If shared liblzma is built, tests/test_* and src/xz/xz are wrapper
scripts created by Libtool. The wrappers set library search path
so that the freshly-built shared library is found.
With a static liblzma, no wrapper scripts are needed, and Libtool
places the real executables to the aforementioned locations. This
speeds up the tests under Valgrind dramatically.
Fixes: 6c095a98fbec ("ci: test Valgrind")
Debian and Ubuntu have a patch that changes the upstream default to
HAVE_DOT = YES. Undo it to have more consistent results across distros.
This was noticed in Ubuntu CI runner where "doxygen" tried to run "dot"
but that failed due to "dot" not being installed. "doxygen" still
finished with exit status 0 until the commit that turned warnings to
errors with WARN_AS_ERROR = FAIL_ON_WARNINGS.
Re-enable CLANG64 environment. Add CLANGARM64. Don't add MINGW64
to slightly reduce the number of runner VMs needed.
Install the required packages using the setup-msys2 action instead
of running the commands separately.
Test Autotools and CMake in the same job to reduce the number of VMs.
This doesn't slow it down too much because the msys2-setup step is
needed by both. However, do only the full builds on ARM64 because
those runners seem to be slower.
Test fewer build configurations. The point of testing on MSYS2 is to
catch Windows-related issues. It should be enough that the more unusual
build configurations are tested in ci.yml.
Run the build commands directly instead of using ci_build.bash. This
makes it easier to see what commands are run even if it is a little
more verbose now.
Run the workflow automatically when commit are pushed to master.
With the fewer build variants it's not too slow.
Don't say that the .lz format allows trailing data. According to the
lzip 1.25 manual, trailing data isn't part of the file format at all.
However, tools are still expected to behave as usefully as possible
when there is trailing data.
Fix the description of lzip >= 1.20 behavior when some of the first
bytes of trailing data match the magic bytes. While the lzip 1.25 manual
recommends that none of the first four bytes in trailing data should
match the magic bytes, the default behavior of lzip 1.25 treats
trailing data as a corrupt member header only if two or three bytes
match the magic bytes; one matching byte isn't enough.
Reported-by: Antonio Diaz Diaz
Link: https://www.mail-archive.com/xz-devel@tukaani.org/msg00702.html
If <getopt.h> had optreset but not getopt_long, xz used optreset while
the replacement getopt_long doesn't support optreset. I'm not aware of
any relevant system where bug is possible. Autotools build didn't have
this bug.
Fixes: af66cd585902 ("CMake: Add support for replacement getopt_long (lib/getopt*).")
Using eight runners seems wasteful. Using only two runners isn't
much slower due to the runner startup overhead.
Also add a comment about the test that fails without b5a5d9e3f702.
On GitHub runners, VS 2019 16.11 (MSVC 19.29.30158) results in
test failures. VS 2022 17.13 (MSVC 19.43.34808) works.
In xz 5.6.x there was a #pragma-based workaround for MSVC builds for
32-bit x86. Another method was thought to work with the new rewritten
CLMUL CRC. Apparently it doesn't. Keep it simple and disable CLMUL CRC
with any non-recent MSVC when building for 32-bit x86.
Fixes: 54eaea5ea49b ("liblzma: x86 CLMUL CRC: Rewrite")
Fixes: https://github.com/tukaani-project/xz/issues/171
Reported-by: Andrew Murray
This makes it easy to crash fuzz_decode_stream_mt when tested
against the code from 5.8.0.
Obviously this might make it harder to reach some other code path now.
The previous code has been in use since 2018 when fuzzing was added
in 106d1a663d4b ("Tests: Add a fuzz test program and a config file
for OSS-Fuzz.").
It doesn't seem possible to trigger the CVE-2025-31115 bug with this
fuzzing target at the moment. It's because the code in fuzz_common.h
passes the whole input buffer to lzma_code() at once.
Single-shot decoding means calling lzma_code() by giving it the whole
input at once and enough output buffer space to store the uncompressed
data, and combining this with LZMA_FINISH and no timeout
(lzma_mt.timeout = 0). This way the file is decoded with a single
lzma_code() call if possible.
The bug prevented the decoder from starting more than one worker thread
in single-shot mode. The issue was noticed when reviewing the code;
there are no bug reports. Thus maybe few have tried this mode.
Fixes: 64b6d496dc81 ("liblzma: Threaded decoder: Always wait for output if LZMA_FINISH is used.")
Don't set thr->in_size = 0 when returning the thread to the stack of
available threads. Not only is it useless, but the main thread may
read the value in SEQ_BLOCK_THR_RUN. With valid inputs, it made
no difference if the main thread saw the original value or 0. With
invalid inputs (when worker thread stops early), thr->in_size was
no longer modified after the previous commit with the security fix
("Don't free the input buffer too early").
So while the bug appears harmless now, it's important to fix it because
the variable was being modified without proper locking. It's trivial
to fix because there is no need to change the value. Only main thread
needs to set the value in (in SEQ_BLOCK_THR_INIT) when starting a new
Block before the worker thread is activated.
Fixes: 4cce3e27f529 ("liblzma: Add threaded .xz decompressor.")
Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Thanks-to: Sam James <sam@gentoo.org>
The input buffer must be valid as long as the main thread is writing
to the worker-specific input buffer. Fix it by making the worker
thread not free the buffer on errors and not return the worker thread to
the pool. The input buffer will be freed when threads_end() is called.
With invalid input, the bug could at least result in a crash. The
effects include heap use after free and writing to an address based
on the null pointer plus an offset.
The bug has been there since the first committed version of the threaded
decoder and thus affects versions from 5.3.3alpha to 5.8.0.
As the commit message in 4cce3e27f529 says, I had made significant
changes on top of Sebastian's patch. This bug was indeed introduced
by my changes; it wasn't in Sebastian's version.
Thanks to Harri K. Koskinen for discovering and reporting this issue.
Fixes: 4cce3e27f529 ("liblzma: Add threaded .xz decompressor.")
Reported-by: Harri K. Koskinen <x64nop@nannu.org>
Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Thanks-to: Sam James <sam@gentoo.org>
The main thread can directly set THR_IDLE in threads_stop() which is
called when errors are detected. threads_stop() won't return the stopped
threads to the pool or free the memory pointed by thr->in anymore, but
it doesn't matter because the existing workers won't be reused after
an error. The resources will be cleaned up when threads_end() is
called (reinitializing the decoder always calls threads_end()).
Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Thanks-to: Sam James <sam@gentoo.org>
Oracle Developer Studio 12.6 on Solaris 10 claims C11 support in
__STDC_VERSION__ and supports _Alignas. However, <stdalign.h> is missing.
We only need alignas, so define it to _Alignas with C11/C17 compilers.
If something included <stdalign.h> later, it shouldn't cause problems.
Thanks to Ihsan Dogan for reporting the issue and testing the fix.
Fixes: c0e7eaae8d6eef1e313c9d0da20ccf126ec61f38
