Remove the backdoor found in 5.6.0 and 5.6.1 (CVE-2024-3094).

While the backdoor was inactive (and thus harmless) without inserting
a small trigger code into the build system when the source package was
created, it's good to remove this anyway:

  - The executable payloads were embedded as binary blobs in
    the test files. This was a blatant violation of the
    Debian Free Software Guidelines.

  - On machines that see lots bots poking at the SSH port, the backdoor
    noticeably increased CPU load, resulting in degraded user experience
    and thus overwhelmingly negative user feedback.

  - The maintainer who added the backdoor has disappeared.

  - Backdoors are bad for security.

This reverts the following without making any other changes:

6e636819 Tests: Update two test files.
a3a29bbd Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
0b4ccc91 Tests: Update RISC-V test files.
8c9b8b20 liblzma: Fix typos in crc32_fast.c and crc64_fast.c.
82ecc538 liblzma: Fix false Valgrind error report with GCC.
cf44e4b7 Tests: Add a few test files.
3060e107 Tests: Use smaller dictionary size in RISC-V test files.
e2870db5 Tests: Add two RISC-V Filter test files.

The RISC-V test files also have real content that tests the filter
but the real content would fit into much smaller files. A generator
program would need to be available as well.

Thanks to Andres Freund for finding and reporting it and making
it public quickly so others could act without a delay.
See: https://www.openwall.com/lists/oss-security/2024/03/29/4

src/liblzma/check/crc32_fast.c

@@ -135,8 +135,11 @@ typedef uint32_t (*crc32_func_type)(
 // This resolver is shared between all three dispatch methods. It serves as
 // the ifunc resolver if ifunc is supported, otherwise it is called as a
 // regular function by the constructor or first call resolution methods.
-// The function attributes are needed for safe IFUNC resolver usage with GCC.
+// The __no_profile_instrument_function__ attribute support is checked when
-lzma_resolver_attributes
+// determining if ifunc can be used, so it is safe to use here.
+#ifdef CRC_USE_IFUNC
+__attribute__((__no_profile_instrument_function__))
+#endif
 static crc32_func_type
 crc32_resolve(void)
 {

src/liblzma/check/crc64_fast.c

@@ -98,7 +98,9 @@ typedef uint64_t (*crc64_func_type)(
 #	pragma GCC diagnostic ignored "-Wunused-function"
 #endif
 
-lzma_resolver_attributes
+#ifdef CRC_USE_IFUNC
+__attribute__((__no_profile_instrument_function__))
+#endif
 static crc64_func_type
 crc64_resolve(void)
 {

src/liblzma/check/crc_common.h

@@ -128,31 +128,6 @@
 #	endif
 #endif
 
-#ifdef CRC_USE_IFUNC
-// Two function attributes are needed to make IFUNC safe with GCC.
-//
-// no-omit-frame-pointer prevents false Valgrind issues when combined with
-// a few other compiler flags. The optimize attribute is supported on
-// GCC >= 4.4 and is not supported with Clang.
-#	if TUKLIB_GNUC_REQ(4,4) && !defined(__clang__)
-#		define no_omit_frame_pointer \
-			__attribute__((optimize("no-omit-frame-pointer")))
-#	else
-#		define no_omit_frame_pointer
-#	endif
-
-// The __no_profile_instrument_function__ attribute support is checked when
-// determining if ifunc can be used, so it is safe to use unconditionally.
-// This attribute is needed because GCC can add profiling to the IFUNC
-// resolver, which calls functions that have not yet been relocated leading
-// to a crash on liblzma start up.
-#	define lzma_resolver_attributes \
-		__attribute__((__no_profile_instrument_function__)) \
-		no_omit_frame_pointer
-#else
-#	define lzma_resolver_attributes
-#endif
-
 // For CRC32 use the generic slice-by-eight implementation if no optimized
 // version is available.
 #if !defined(CRC32_ARCH_OPTIMIZED) && !defined(CRC32_GENERIC)

tests/files/README

@@ -41,8 +41,6 @@
     good-0catpad-empty.xz has two zero-Block Streams concatenated with
     four-byte Stream Padding between the Streams.
 
-    good-2cat.xz has two Streams with one Block each.
-
     good-1-check-none.xz has one Stream with one Block with two
     uncompressed LZMA2 chunks and no integrity check.
 
@@ -83,14 +81,6 @@
     good-1-arm64-lzma2-2.xz is like good-1-arm64-lzma2-1.xz but with
     non-zero start offset. XZ Embedded doesn't support this file.
 
-    good-1-riscv-lzma2-1.xz uses the RISC-V filter and LZMA2. The
-    uncompressed data is constructed so it tests all of the instructions
-    that should be encoded and a few that should not. Additionally, the
-    file contains random bytes to help test unforeseen corner cases.
-
-    good-1-riscv-lzma2-2.xz is like good-1-riscv-lzma2-1.xz but with
-    non-zero start offset. XZ Embedded doesn't support this file.
-
     good-1-lzma2-1.xz has two LZMA2 chunks, of which the second sets
     new properties.
 
@@ -294,11 +284,6 @@
     Uncompressed Size bytes of output will have been produced but
     the LZMA2 decoder doesn't indicate end of stream.
 
-    bad-3-corrupt_lzma2.xz has three Streams in it. The first and third
-    streams are valid xz Streams. The middle Stream has a correct Stream
-    Header, Block Header, Index and Stream Footer. Only the LZMA2 data
-    is corrupt. This file should decompress if --single-stream is used.
-
 
 3. Descriptions of Individual .lzma Files
 
@@ -315,14 +300,6 @@
     will give an error at the end of the file after producing the
     correct uncompressed output.
 
-    good-small_compressed.lzma was created with a small dictionary (2^16).
-    It contains the string "Hello World" repeated 100,000 times. This tests
-    match decoding and wrapping the dictionary.
-
-    good-large_compressed.lzma was created with a mix of repeated
-    characters and random data to test a data stream containing many
-    matches and many literals.
-
 
 3.2. Bad Files
 
@@ -344,10 +321,6 @@
     bad-too_small_size-without_eopm-3.lzma is like -1 above but instead
     of a literal the problem occurs in the middle of a match.
 
-    bad-dict_size.lzma has a valid dictionary size according to the .lzma
-    File Format, but will be rejected by XZ Utils because it is not 2^n or
-    2^n + 2^(n-1).
-
 
 4. Descriptions of Individual .lz (lzip) Files
 

tests/test_files.sh

@@ -149,17 +149,6 @@ else
 	exit 1
 fi
 
-# Test that --single-stream can decompress bad-3-corrupt_lzma2.xz.
-# The first Stream in this file should decompress without errors.
-# This file cannot be decompressed with xzdec.
-I="$srcdir/files/bad-3-corrupt_lzma2.xz"
-if test -z "$XZ" || "$XZ" -dc --single-stream $NO_WARN "$I" > /dev/null; then
-	:
-else
-	echo "Good first Stream failed xz with --single-stream: $I"
-	exit 1
-fi
-
 
 #########
 # .lzma #