Call tuklib_mask_nonprint() on filenames and also on a few other
strings from the command line too.
The filename printed by "xz --robot --list" (in list.c) is also masked.
It's good to get rid of tabs and newlines which would desync the output
but masking other chars wouldn't be strictly necessary. It might matter
with sensible filenames if LC_CTYPE is "C" (when iswprint() might reject
non-ASCII chars) and a script wants to read a filename from xz's output.
Hopefully it's an unusual enough corner case to not be a real problem.
With CMake 3.31, there were a few warnings from
CMP0177 "install() DESTINATION paths are normalized".
These occurred because the install(FILES) command in
my_install_man_lang() is called with a DESTINATION path
that contains two consecutive slashes, for example,
"share/man//man1". Such a path is for the English man pages.
With translated man pages, the language code goes between
the slashes. The warning was probably triggered because the
extra slash gets removed by the normalization.
IMPORTANT: This includes a security fix to command line tool
argument handling.
Some toolchains embed an application manifest by default to declare
UAC-compliance. Some also declare compatibility with Vista/8/8.1/10/11
to let the app access features newer than those of Vista.
We want all the above but also two more things:
- Declare that the app is long path aware to support paths longer
than 259 characters (this may also require a registry change).
- Force the code page to UTF-8. This allows the command line tools
to access files whose names contain characters that don't exist
in the current legacy code page (except unpaired surrogates).
The UTF-8 code page also fixes security issues in command line
argument handling which can be exploited with malicious filenames.
See the new file w32_application.manifest.comments.txt.
Thanks to Orange Tsai and splitline from DEVCORE Research Team
for discovering this issue.
Thanks to Vijay Sarvepalli for reporting the issue to me.
Thanks to Kelvin Lee for testing with MSVC and helping with
the required build system fixes.
Autotools-based build has always done this so this is for consistency.
However, the CMake build won't create the DEF file when building
for Cygwin or MSYS2 because in that context it should be useless.
(If Cygwin or MSYS2 is used to host building of normal Windows
binaries then the DEF file is still created.)
This reverts commit dc03f6290f.
OpenBSD 7.6 will support elf_aux_info(3), and the detection code used
on FreeBSD will work on OpenBSD 7.6 too. Keep things simpler and drop
the OpenBSD-specific sysctl() method.
Thanks to Christian Weisgerber.
The liblzma target was recently changed to link against Threads::Threads
with the PRIVATE keyword. I had forgotten that xz itself depends on
pthreads too due to pthread_sigmask(). Thus, the build broke when
building shared liblzma and pthread_sigmask() wasn't in libc.
Thanks to Peter Seiderer for the bug report.
Fixes: ac05f1b0d7
Fixes: https://github.com/tukaani-project/xz/issues/129#issuecomment-2204522994
While po/*.gmo files won't be used from the release tarball,
the generated translated man pages will be used still. Those
are text files and po4a has slightly more dependencies than
gettext tools so installing po4a might be a bit more challenging
in some situations.
When a release tarball is created using Autotools, the tarball includes
po/*.gmo files which are binary files generated from po/*.po. Other
tarball creation methods don't and won't create the .gmo files.
It feels clearer if CMake will never install pre-generated binary files
from the source package. If people are able to install CMake, they
likely are able to install gettext tools as well (assuming they want
translations).
If a user set XZ_NLS=ON but find_package(Intl) failed or CMake version
wasn't at least 3.20, the configuration would fail in a cryptic way.
If XZ_NLS is enabled, require that CMake is new enough and that either
gettext tools or pre-generated .gmo files are available. Otherwise fail
the configuration. Previously missing gettext tools and .gmo files would
only result in a warning.
Missing man page translations are still only a warning.
Thanks to Peter Seiderer for the bug report.
Fixes: https://github.com/tukaani-project/xz/issues/129
Closes: https://github.com/tukaani-project/xz/pull/130
The CMake variables were renamed and accidentally also
the compile definition was renamed. As a result, translation
support wasn't actually enabled in the executables.
Fixes: 29f77c7b70
The crc.w.{b/h/w/d}.w instructions in LoongArch can calculate the CRC32
result for 1/2/4/8 bytes in a single operation. Using these is much
faster compared to the generic method.
Optimized CRC32 is enabled unconditionally on 64-bit LoongArch because
the LoongArch specification says that CRC32 instructions shall be
implemented for 64-bit processors. Optimized CRC32 isn't enabled for
32-bit LoongArch processors because not enough information is available
about them.
Co-authored-by: Lasse Collin <lasse.collin@tukaani.org>
Closes: https://github.com/tukaani-project/xz/pull/86
While the CMake support has gotten a lot less testing than
the Autotools-based build, the supported features should now
be equal. The output may differ slightly, for example,
liblzma.pc may have
Libs.private: -pthread -lpthread
with Autotools on GNU/Linux. CMake doesn't put any options
in Libs.private because on modern glibc the pthread functions
are in libc. The options options aren't required to link static
liblzma into an application.
Autotools-based build doesn't generate or install
lib/cmake/liblzma-*.cmake files. This means that on most
platforms one cannot rely on
find_package(liblzma 5.2.5 REQUIRED CONFIG)
or such finding those files.
It was weird to add CMAKE_THREAD_LIBS_INIT in CMAKE_REQUIRED_LIBRARIES
only if CLOCK_MONOTONIC is available. Alternative would be to remove
the thread libs from CMAKE_REQUIRED_LIBRARIES after the check for
pthread_condattr_setclock() but keeping the libs should be fine too.
Then it's ready in case more pthread functions were wanted some day.
In CMake, check_c_source_compiles() always links too. With
link-time optimization, unused functions may get omitted if
main() doesn't depend on them. Consider the following which
tries to check if somefunction() is available when <someheader.h>
has been included:
#include <someheader.h>
int foo(void) { return somefunction(); }
int main(void) { return 0; }
LTO may omit foo() completely because the program as a whole doesn't
need it and then the program will link even if the symbol somefunction
isn't available in libc or other library being linked in, and then
the test may pass when it shouldn't.
What happens if <someheader.h> doesn't declare somefunction()?
Shouldn't the test fail in the compilation phase already? It should
but many compilers don't follow the C99 and later standards that
prohibit implicit function declarations. Instead such compilers
assume that somefunction() exists, compilation succeeds (with a
warning), and then linker with LTO omits the call to somefunction().
Change the tests so that they are part of main(). If compiler accepts
implicitly declared functions, LTO cannot omit them because it has to
assume that they might have side effects and thus linking will fail.
On the other hand, if the functions/intrinsics being used are supported,
they might get optimized away but in that case it's fine because they
really are supported.
It is fine to use __attribute__((target(...))) for main(). At least
it works with GCC 4.9 to 14.1 on x86-64.
Reported-by: Sam James <sam@gentoo.org>
Now runtime detection of CLMUL support can pick between the CLMUL and
the generic assembly implementations. Whatever overhead this has for
builds that omit CLMUL completely isn't important because builds for
any non-ancient system is likely to include the CLMUL code too.
Handle the CRC tables in crcXX_fast.c files because now these files
are built even when assembly code is used.
If 32-bit x86 assembly is enabled then it will always be built even
if compiler flags were such that CLMUL would be allowed unconditionally.
That is, runtime detection will be used anyway. This keeps the build
rules simpler.
In LZ encoder, build and use lzma_lz_hash_table[256] if CLMUL CRC
is used without runtime detection. Previously this wasn't needed
because crc32_table.c included the lzma_crc32_table[][] in the build
unless encoder support had been disabled. Including an 8 KiB table
was silly when only 1 KiB is actually used. So now liblzma is 7 KiB
smaller if CLMUL is enabled without runtime detection.
It feels clearer this way, and when support for external SHA-256
is added, this will keep the order of the library detection the
same as in configure.ac (check for pthreads before libmd) although
it shouldn't matter in practice.
This simplifies things a little. Building liblzma with VS2013 probably
still worked but building the command line tools was not supported.
Microsoft ended support for VS2013 on 2024-04.
Make the available options and their behavior match
--enable-symbol-versions in configure.ac.
Don't enable symbol versions on Linux if not using glibc. Previously
the generic variant was selected on Microblaze or if using NVHPC
without checking that libc is glibc.
Leave the cache variable to "auto" or "yes" if that was specified
instead of setting it to the autodetected value by default. A downside
is that one cannot easily see which variant the autodetection code
has selected. The same applies to XZ_SANDBOX and XZ_THREADS though.
Also clarify that "yes" will fail if no threading support is found.
If no threading is wanted, it has to be disabled manually.
configure.ac doesn't behave this way at the moment. Instead it
assumes pthreads to be present if not targeting Windows. If pthreads
actually are missing, the build fails later.
Lasse Collin
Xi Ruoyao
Sam James