Commit Graph

2421 Commits

Author SHA1 Message Date
Sam James 73f629e321 ci: rename ci_build.sh -> ci_build.bash
We discussed the name and it's less cognitive load to just call it '.bash'
so you don't have an immediate question about if bashisms are OK.
2024-04-11 00:15:35 +03:00
Sam James 8709407a9e ci: build in parallel by default 2024-04-11 00:15:35 +03:00
Sam James 65bf7e0a1c ci: default to -O2
We need this for when we're passing sanitizer flags or -gdwarf-4 for Clang
with Valgrind. Just always start with -O2 if CFLAGS isn't set in the
environment and append what was passed on the command line.
2024-04-11 00:15:35 +03:00
Sam James bc899f9e07 ci: make automake's test runner verbose on failures
This is a lot easier to work with than the save-logs thing the action
tries to do...
2024-04-11 00:15:35 +03:00
Sam James b5e3470442 ci: make UBSAN abort on errors
Unfortunately, UBSAN doesn't do this by default. See also the change
I made in Meson for this in October [0].

[0] 7b7d2e060b
2024-04-11 00:15:35 +03:00
Sam James 6c095a98fb ci: test Valgrind
Using `--trace-children=yes` has a trade-off here, as it makes
`test_scripts.sh` pretty slow when calling various non-xz utilities.

But I also feel like it's not useless to have Valgrind used there and it's
not easy to exclude Valgrind just for that one test...

I did consider using AX_VALGRIND_CHECK [0][1] but I couldn't get it working
immediately with some conditionally-built tests and I wondered if it was
worth spending time on at least while we're debating xz's future build
system situation.

[0] https://www.gnu.org/software/autoconf-archive/ax_valgrind_check.html
[1] https://tecnocode.co.uk/2014/12/23/automatically-valgrinding-code-with-ax_valgrind_check/
2024-04-11 00:15:35 +03:00
Lasse Collin 6286c1900c liblzma: CRC: Simplify table omission macros
A macro is useful to prevent a single #if directive from
getting too ugly but only one macro is needed for all archs.
2024-04-10 23:33:17 +03:00
Lasse Collin 45da936c87 liblzma: ARM64 CRC: Fix omission of CRC32 table
The macro name had an odd typo so the table wasn't omitted
when it should have.

Fixes: 1940f0ec28
2024-04-10 23:12:23 +03:00
Lasse Collin 308a9af854 Build: If ARM64 feature detection func is found, stop looking for others
This can speed up configure a tiny bit.

Fixes: c5f6d79cc9
2024-04-10 23:12:23 +03:00
Lasse Collin fc43cecd32 liblzma: ARM64 CRC32: Change style of the macOS code to match FreeBSD
I didn't test this but it shouldn't change any functionality.

Fixes: 761f5b69a4
2024-04-10 23:12:23 +03:00
Lasse Collin 1024cd4cd9 liblzma: ARM64 CRC32: Add error checking to FreeBSD-specific code
Also add parenthesis to the return statement.

I didn't test this.

Fixes: 761f5b69a4
2024-04-10 23:12:23 +03:00
Lasse Collin 2337f7021c liblzma: ARM64 CRC32: Use negation instead of subtracting from 8
Subtracting from 0 is negation, this just keeps warnings away.

Fixes: 761f5b69a4
2024-04-10 23:12:11 +03:00
Lasse Collin d8fffd01aa liblzma: ARM64 CRC32: Tweak coding style and comments 2024-04-10 22:53:53 +03:00
Lasse Collin 780d2c236d Update SECURITY.md. 2024-04-09 21:55:01 +03:00
Lasse Collin 986865ea2f CI: Remove ifunc support. 2024-04-09 18:22:27 +03:00
Lasse Collin 689ae24273 liblzma: Remove ifunc support.
This is *NOT* done for security reasons even though the backdoor
relied on the ifunc code. Instead, the reason is that in this
project ifunc provides little benefits but it's quite a bit of
extra code to support it. The only case where ifunc *might* matter
for performance is if the CRC functions are used directly by an
application. In normal compression use it's completely irrelevant.
2024-04-09 18:22:27 +03:00
Lasse Collin 6b4c859059 tests/files/README: Update the main heading. 2024-04-09 18:22:27 +03:00
Lasse Collin 2a851e06b8 tests/files/README: Explain how to recreate the ARM64 test files. 2024-04-09 18:22:27 +03:00
Lasse Collin 3d09b721b9 debug: Add generator for the ARM64 test file data. 2024-04-09 18:22:27 +03:00
Lasse Collin 31ef676567 xz man page: Use .ft CR instead of CW to silence warnings from groff. 2024-04-09 18:22:27 +03:00
Lasse Collin 780cbf29d5 Fix NEWS for 5.6.0 and 5.6.1. 2024-04-09 18:22:27 +03:00
Lasse Collin bfd0c7c478 Remove the XZ logo. 2024-04-09 18:22:27 +03:00
Lasse Collin 77a294d98a Update maintainer and author info.
The other maintainer suddenly disappeared.
2024-04-09 18:22:27 +03:00
Lasse Collin 8dd03d4484 Docs: Update .xz file format specification to 1.2.1.
This only reverts the XZ URL changes.
2024-04-09 18:22:27 +03:00
Lasse Collin 17aa2e1a79 Update website URLs back to tukaani.org.
The XZ projects were moved back to their original URLs.
2024-04-09 18:22:27 +03:00
Lasse Collin 2739db9810 xzdec: Tweak coding style and comments. 2024-04-09 18:22:27 +03:00
Lasse Collin 408b6adb2a tests/ossfuzz: Tiny fix to a comment. 2024-04-09 18:22:27 +03:00
Lasse Collin db4dd74a34 Update THANKS. 2024-04-09 18:22:27 +03:00
Lasse Collin e93e13c8b3 Remove the backdoor found in 5.6.0 and 5.6.1 (CVE-2024-3094).
While the backdoor was inactive (and thus harmless) without inserting
a small trigger code into the build system when the source package was
created, it's good to remove this anyway:

  - The executable payloads were embedded as binary blobs in
    the test files. This was a blatant violation of the
    Debian Free Software Guidelines.

  - On machines that see lots bots poking at the SSH port, the backdoor
    noticeably increased CPU load, resulting in degraded user experience
    and thus overwhelmingly negative user feedback.

  - The maintainer who added the backdoor has disappeared.

  - Backdoors are bad for security.

This reverts the following without making any other changes:

6e636819 Tests: Update two test files.
a3a29bbd Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
0b4ccc91 Tests: Update RISC-V test files.
8c9b8b20 liblzma: Fix typos in crc32_fast.c and crc64_fast.c.
82ecc538 liblzma: Fix false Valgrind error report with GCC.
cf44e4b7 Tests: Add a few test files.
3060e107 Tests: Use smaller dictionary size in RISC-V test files.
e2870db5 Tests: Add two RISC-V Filter test files.

The RISC-V test files also have real content that tests the filter
but the real content would fit into much smaller files. A generator
program would need to be available as well.

Thanks to Andres Freund for finding and reporting it and making
it public quickly so others could act without a delay.
See: https://www.openwall.com/lists/oss-security/2024/03/29/4
2024-04-09 17:57:39 +03:00
Lasse Collin f9cf4c05ed CMake: Fix sabotaged Landlock sandbox check.
It never enabled it.
2024-03-30 14:36:28 +02:00
Jia Tan af071ef770 Docs: Simplify SECURITY.md. 2024-03-26 03:38:12 +08:00
Lasse Collin 0b99783d63 liblzma: memcmplen.h: Add a comment why subtraction is used. 2024-03-22 17:46:30 +02:00
Lasse Collin 8a25ba024d INSTALL: Document arguments of --enable-symbol-versions. 2024-03-15 18:04:10 +02:00
Lasse Collin 49324b711f Build: Use only the generic symbol versioning with NVIDIA HPC Compiler.
This does the previous commit with CMake.

AC_EGREP_CPP uses AC_REQUIRE so the outermost if-commands must
be changed to AS_IF to ensure that things wont break some day.
See 5a5bd7f871.
2024-03-15 18:04:10 +02:00
Lasse Collin c273123ed0 CMake: Use only the generic symbol versioning with NVIDIA HPC Compiler.
It doesn't support the __symver__ attribute or __asm__(".symver ...").
The generic symbol versioning can still be used since it only needs
linker support.
2024-03-15 18:04:10 +02:00
Lasse Collin df7f487648 Update THANKS. 2024-03-15 18:04:10 +02:00
Lasse Collin 3217b82b3e liblzma: Minor comment edits. 2024-03-15 18:03:47 +02:00
Sergey Kosukhin 096bc0e3f8 liblzma: Fix building with NVHPC (NVIDIA HPC SDK).
NVHPC compiler has several issues that make it impossible to
build liblzma:
  - the compiler cannot handle unions that contain pointers that
    are not the first members;
  - the compiler cannot handle the assembler code in range_decoder.h
    (LZMA_RANGE_DECODER_CONFIG has to be set to zero);
  - the compiler fails to produce valid code for delta_decode if the
    vectorization is enabled, which results in failed tests.

This introduces NVHPC-specific workarounds that address the issues.
2024-03-15 17:30:50 +02:00
Lasse Collin 2ad7fad670 CMake: Disable symbol versioning on non-glibc Linux.
This better matches what configure.ac does. For example, musl has
only basic symbol versioning support:

https://wiki.musl-libc.org/functional-differences-from-glibc.html#Symbol_versioning

configure.ac tries to enable symbol versioning only with glibc
so now CMake does the same.
2024-03-15 17:30:50 +02:00
Lasse Collin 82f0c0d39e CMake: Make symbol versioning configurable. 2024-03-15 17:30:50 +02:00
Lasse Collin 45d33bfc45 Build: Style tweaks to configure.ac.
The AC_MSG_ERROR line is overlong anyway as are a few other
AC_MSG_ERROR lines already.
2024-03-13 22:00:23 +02:00
Sergey Kosukhin f56ed6fac6 Build: Let the users override the symbol versioning variant.
There are cases when the users want to decide themselves whether
they want to have the generic (even on GNU/Linux) or the linux
(even if we do not recommend that) symbol versioning variant.
The former might be needed to circumvent compiler issues (i.e.
the compiler does not support all features that are required
for the linux versioning), the latter might help in overriding
the assumptions made in the configure script.
2024-03-13 21:59:55 +02:00
Jia Tan a4f2e20d84 Add NEWS for 5.6.1 2024-03-09 11:27:27 +08:00
Jia Tan f01be8ad75 Translations: Add missing --riscv option to man page translations. 2024-03-09 10:43:20 +08:00
Jia Tan 6e636819e8 Tests: Update two test files.
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
2024-03-09 10:18:29 +08:00
Jia Tan a3a29bbd5d Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
2024-03-09 10:08:32 +08:00
Jia Tan 0b4ccc9145 Tests: Update RISC-V test files.
This increases code coverage and tests for possible shifting bugs.
2024-03-09 10:05:32 +08:00
Jia Tan 8c9b8b2063 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 2024-03-09 09:52:32 +08:00
Jia Tan b93a8d7631 Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.
This reverts commit adaacafde6.
2024-03-09 09:49:55 +08:00
Jia Tan 82ecc53819 liblzma: Fix false Valgrind error report with GCC.
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.

The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-09 09:20:57 +08:00