In ELF shared libs:
-fvisibility=hidden affects definitions of symbols but not
declarations.[*] This doesn't affect direct calls to functions
inside liblzma as a linker can replace a call to lzma_foo@plt
with a call directly to lzma_foo when -fvisibility=hidden is used.
[*] It has to be like this because otherwise every installed
header file would need to explictly set the symbol visibility
to default.
When accessing extern variables that aren't defined in the
same translation unit, compiler assumes that the variable has
the default visibility and thus indirection is needed. Unlike
function calls, linker cannot optimize this.
Using __attribute__((__visibility__("hidden"))) with the extern
variable declarations tells the compiler that indirection isn't
needed because the definition is in the same shared library.
About 15+ years ago, someone told me that it would be good if
the CRC tables would be defined in the same translation unit
as the C code of the CRC functions. While I understood that it
could help a tiny amount, I didn't want to change the code because
a separate translation unit for the CRC tables was needed for the
x86 assembly code anyway. But when visibility attributes are
supported, simply marking the extern declaration with the
hidden attribute will get identical result. When there are only
a few affected variables, this is trivial to do. I wish I had
understood this back then already.
MinGW (formely a MinGW.org Project, later the MinGW.OSDN Project
at <https://osdn.net/projects/mingw/>) has GCC 9.2.0 as the
most recent GCC package (released 2021-02-02). The project might
still be alive but majority of people have switched to MinGW-w64.
Thus it seems clearer to refer to MinGW-w64 in our API headers too.
Building with MinGW is likely to still work but I haven't tested it
in the recent years.
A CMake option LARGE_FILE_SUPPORT is created if and only if
-D_FILE_OFFSET_BITS=64 affects sizeof(off_t).
This is needed on many 32-bit platforms and even with 64-bit builds
with MinGW-w64 to get support for files larger than 2 GiB.
Autotools based build uses -pthread and thus adds it to Libs.private
in liblzma.pc. CMake doesn't use -pthread at all if pthread functions
are available in libc so Libs.private doesn't get -pthread either.
It properly adds -DLZMA_API_STATIC when compiling code that
will be linked against static liblzma. Having it there on
systems other than Windows does no harm.
See: https://www.msys2.org/docs/pkgconfig/
Using set(ENABLE_THREADS "posix") is confusing because it sets
a new normal variable and leaves the cache entry with the same
name unchanged. The intent wasn't to change the cache entry so
this switches to a different variable name.
This way typos are caught quickly and compounding error messages
are avoided (a single typo could cause more than one error).
This keeps using SEND_ERROR when the system is lacking a feature
(like threading library or sandboxing method). This way the whole
configuration log will be generated in case someone wishes to
report a problem upstream.
The option is enabled by default, but will only be visible to a user
listing cache variables or using a CMake GUI application if the
immintrin.h header file is found.
This mirrors our Autotools build --disable-clmul-crc functionality.
In XZ Utils context this doesn't matter much because
unaligned reads and writes aren't used in hot code
when TUKLIB_FAST_UNALIGNED_ACCESS isn't #defined.
The Ninja Generator for CMake cannot have a custom target and its
BYPRODUCTS have the same name. This has prevented Ninja builds on
Unix-like systems since the xz symlinks were introduced in
80a1a8bb83.
llvm-windres 17.0.0 has more accurate emulation of GNU windres, so
the hack for GNU windres must now be used with llvm-windres too.
LLVM 16.0.6 has the old behavior and there likely won't be more
16.x releases. So we can simply check for >= 17.0.0.
The workaround must not be used with Clang that is acting in
MSVC mode. This checks for the known environments that need
the workaround instead of using "NOT MSVC".
See also:
2bcc0fdc58
When the generic fast crc64 method is used, then we omit
lzma_crc64_table[][].
The C standards don't allow an empty translation unit which can be
avoided by declaring something, without exporting any symbols.
There was a use of AC_COMPILE_IFELSE that didn't use
AC_LANG_SOURCE and Autoconf warned about this. The omission
had been intentional but it turned out that this didn't do
what I thought it would.
Autoconf 2.71 manual gives an impression that AC_LANG_SOURCE
inserts all #defines that have been made with AC_DEFINE so
far (confdefs.h). The idea was that omitting AC_LANG_SOURCE
would mean that only the exact code included in the
AC_COMPILE_IFELSE call would be compiled.
With C programs this is not true: the #defines get added without
AC_LANG_SOURCE too. There seems to be no neat way to avoid this.
Thus, with the C language at least, adding AC_LANG_SOURCE makes
no other difference than silencing a warning from Autoconf. The
generated "configure" remains identical. (Docs of AC_LANG_CONFTEST
say that the #defines have been inserted since Autoconf 2.63b and
that AC_COMPILE_IFELSE uses AC_LANG_CONFTEST. So the behavior is
documented if one also reads the docs of macros that one isn't
calling directly.)
Any extra code, including #defines, can cause problems for
these two tests because these tests must use -Werror.
CC=clang CFLAGS=-Weverything is the most extreme example.
It enables -Wreserved-macro-identifier which warns about
It's possible to write a test file that passes -Weverything but
it becomes impossible when Autoconf inserts confdefs.h.
So this commit adds AC_LANG_SOURCE to silence Autoconf warnings.
A different solution is needed for -Werror tests.
Before this commit, the following writes "foo" to the
console and deletes the input file:
echo foo | xz > con_xz
xz --suffix=_xz --decompress con_xz
It cannot happen without --suffix because names like con.xz
are also special and so attempting to decompress con.xz
(or compress con to con.xz) will already fail when opening
the input file.
Similar thing is possible when compressing. The following
writes to "nul" and the input file "n" is deleted.
echo foo | xz > n
xz --suffix=ul n
Now xz checks if the destination is a special file before
continuing. DOS/DJGPP version had a check for this but
Windows (and OS/2) didn't.
There are several new policies. CMP0149 may affect the Windows SDK
version that CMake will choose by default. The new behavior is more
predictable, always choosing the latest SDK version by default.
The other new policies shouldn't affect this package.
For compatibility with C23's [[noreturn]], tuklib_attr_noreturn
must be at the beginning of declaration (before "extern" or
"static", and even before any GNU C's __attribute__).
This commit also moves all other function attributes to
the beginning of function declarations. "extern" is kept
at the beginning of a line so the attributes are listed on
separate lines before "extern" or "static".
xrealloc() is obviously incorrect, modern GCC docs even
mention realloc() as an example where this attribute
cannot be used.
liblzma's lzma_alloc() and lzma_alloc_zero() would be
correct uses most of the time but custom allocators
may use a memory pool or otherwise hold the pointer
so aliasing issues could happen in theory.
The xstrdup() case likely was correct but I removed it anyway.
Now there are no __malloc__ attributes left in the code.
The allocations aren't in hot paths so this should make
no practical difference.
If CMake was configured more than once, HAVE_CLOCK_GETTIME and
HAVE_CLOCK_MONOTONIC would not be set as compile definitions. The check
for librt being needed to provide HAVE_CLOCK_GETTIME was also
simplified.
Now the two variations of the format strings are created with
a macro, and the whole detection code can be easily disabled
on platforms where thousand separator formatting is known to
not work (MSVC has no support, and on DJGPP 2.05 it can have
problems in some cases).
The argument to vli_ceil4() should always guarantee the return value
is also a valid lzma_vli. Thus the highest three valid lzma_vli values
are invalid arguments. All uses of the function ensure this so the
assert is updated to match this.
This was not a security bug since there was no path to overflow
UINT64_MAX in lzma_index_append() or when it calls index_file_size().
The bug was discovered by a failing assert() in vli_ceil4() when called
from index_file_size() when unpadded_sum (the sum of the compressed size
of current Stream and the unpadded_size parameter) exceeds LZMA_VLI_MAX.
Previously, the unpadded_size parameter was checked to be not greater
than UNPADDED_SIZE_MAX, but no check was done once compressed_base was
added.
This could not have caused an integer overflow in index_file_size() when
called by lzma_index_append(). The calculation for file_size breaks down
into the sum of:
- Compressed base from all previous Streams
- 2 * LZMA_STREAM_HEADER_SIZE (size of the current Streams header and
footer)
- stream_padding (can be set by lzma_index_stream_padding())
- Compressed base from the current Stream
- Unpadded size (parameter to lzma_index_append())
The sum of everything except for Unpadded size must be less than
LZMA_VLI_MAX. This is guarenteed by overflow checks in the functions
that can set these values including lzma_index_stream_padding(),
lzma_index_append(), and lzma_index_cat(). The maximum value for
Unpadded size is enforced by lzma_index_append() to be less than or
equal UNPADDED_SIZE_MAX. Thus, the sum cannot exceed UINT64_MAX since
LZMA_VLI_MAX is half of UINT64_MAX.
Thanks to Joona Kannisto for reporting this.
Jia Tan
Lasse Collin