This only affects builds with UCRT. With legacy MSVCRT, the replacement
functions are always enabled.
Omitting the MinGW-w64 replacements saves over 20 KiB per executable.
The downside is that --enable-small or XZ_SMALL=ON disables thousand
separator support in xz messages. If someone is OK with the slower
speed of slightly smaller builds, lack of thousand separators won't
matter.
Don't override __USE_MINGW_ANSI_STDIO if it is already defined (via
CPPFLAGS or such method).
(cherry picked from commit 4eae859ae8ad7072eaa74aeaee79a2c3c12c55cb)
When the 8-byte method was enabled for ARM64, a check for endianness
wasn't added. This broke the LZMA/LZMA2 encoder. Test suite caught it.
Fixes: cd64dd70d5665b6048829c45772d08606f44672e
Co-authored-by: Marcus Comstedt <marcus@mc.pp.se>
(cherry picked from commit 150356207c8d6a3e0af465b676430d19d62f884c)
lzma_str_to_filters() may call parse_lzma12_preset() in two ways. The
call from str_to_filters() detects the string type from the first
character(s) and as a side-effect it validates the first digit of
the preset string. So this change makes no difference there.
However, the call from parse_options() doesn't pre-validate the string.
parse_lzma12_preset() will return an invalid value which is passed to
lzma_lzma_preset() which safely rejects it. The bug still affects the
the error message:
$ xz --filters=lzma2:preset=X
xz: Error in --filters=FILTERS option:
xz: lzma2:preset=X
xz: ^
xz: Unsupported preset
After the fix:
$ xz --filters=lzma2:preset=X
xz: Error in --filters=FILTERS option:
xz: lzma2:preset=X
xz: ^
xz: Unsupported preset
The ^ now correctly points to the X and not past it because the X itself
is the problematic character.
Fixes: cedeeca2ea6ada5b0411b2ae10d7a859e837f203
(cherry picked from commit 75107217670a97b7b772833669d88c3c2f188e37)
Forgetting the argument (or not using = to separate the option from
the argument) resulted in lzma_str_to_filters() being called with NULL
as input string argument. The function handles it fine but xz passes
the NULL to printf() too:
$ xz --filters
xz: Error in --filters=FILTERS option:
xz: (null)
xz: ^
xz: Unexpected NULL pointer argument(s) to lzma_str_to_filters()
Now it's correct:
$ xz --filters
xz: option '--filters' requires an argument
The --filters-help option doesn't take any arguments.
Fixes: 9ded880a0221f4d1256845fc4ab957ffd377c760
Fixes: d6af7f347077b22403133239592e478931307759
Fixes: a165d7df1964121eb9df715e6f836a31c865beef
(cherry picked from commit 52ff32433734d03befd85a5bf00fba77d6501455)
This fix is similar to 48ff3f06521ca326996ab9a04d1b342098960427.
Fixes: d74fb5f060b76db709b50f5fd37490394e52f975
(cherry picked from commit 2655c81b5e92278b0fd51f6537c1116f8349b02a)
Also remove the recently-added workaround from tuklib_gettext.h.
Requiring a new enough gettext-runtime is cleaner. I guess it's
mostly MSYS2 where xz is built with translation support, so once
MSYS2 has Gettext >= 0.23.1, this requirement shouldn't be a problem
in practice.
(cherry picked from commit 16821252c504071f5c2012e415e59cbf5fb79820)
The DESCRIPTION section always explained it, and the OPTIONS section
only described the differences to the default behavior. However, new
users in a hurry may skip reading DESCRIPTION. The default behavior
is a bit dangerous, thus it's good to repeat in --compress and
--decompress docs that source file is removed after successful operation.
Fixes: https://github.com/tukaani-project/xz/issues/150
(cherry picked from commit 653732bd6f06d8f465bf353bf6e1c16f1405b906)
See the comment. In this package, locale is set at program startup and
not changed later, so the point (2) in the comment isn't a problem.
Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit b40e3321a7fb9dfdf8ffb30e7e0788c2f0abc941)
xzdec isn't translated and doesn't need libintl on Windows even
when NLS is enabled, thus libintl_setlocale() cannot interfere
with the locale settings. Thus, standard setlocale() works perfectly.
In the commit 78868b6e, the explanation in the commit message is wrong.
Fixes: 78868b6ed63fa4c89f73e3dfed27abfb8b0d46db
(cherry picked from commit d6796f9ce5359faaaed82926c1735aee3694430f)
Only leave the FindFileFirstA() notes from 20dfca81, reverting
the incorrect setlocale() notes. On Windows, Gettext's <libintl.h>
overrides setlocale() with libintl_setlocale() wrapper. I hadn't
noticed this, and thus my conclusions were wrong.
Fixes: 20dfca8171dad4c64785ac61d5b68972c444877b
(cherry picked from commit e607329a615759f1519016595dd38df7c89208f2)
If wcwidth() isn't available (Windows), previously it was assumed
that one byte == one column in the terminal. Now it is assumed that
one multibyte character == one column. This works better with UTF-8.
Languages that only use single-width characters without any combining
characters should work correctly with this.
In xz, none of po/*.po contain combining characters and only ko.po,
zh_CN.po, and zh_TW.po contain fullwidth characters. Thus, "only"
those three translations in xz are broken on Windows with the
UTF-8 code page. Broken means that column headings in xz -lvv and
(only in the master branch) strings in --long-help are misaligned,
so it's not a huge problem. I don't know if those three languages
displayed perfectly before the UTF-8 change because I hadn't tested
translations with native Windows builds before.
Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit b797c44c42ea54fe1c52722a2fca0c9618575598)
xzdec isn't translated and didn't have locale-specific behavior
in the past. On Windows with UTF-8 in the application manifest,
setting the locale makes a difference though:
- Without any setlocale() call, non-ASCII filenames don't display
properly in Command Prompt unless one first uses "chcp 65001"
to set the console code page to UTF-8.
- setlocale(LC_ALL, "") is enough to make non-ASCII filenames
print correctly in Command Prompt without using "chcp 65001",
assuming that the non-UTF-8 code page (like 850) supports
those non-ASCII characters.
- setlocale(LC_ALL, ".UTF8") is even better because then mbrtowc() and
such functions use an UTF-8 locale instead of a legacy code page.
The tuklib_gettext_setlocale() macro takes care of this (without
enabling any translations).
Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 78868b6ed63fa4c89f73e3dfed27abfb8b0d46db)
XZ Utils 5.6.3 set the active code page to UTF-8 to fix CVE-2024-47611.
This wasn't paired with UCRT-specific setlocale(LC_ALL, ".UTF8"), thus
non-ASCII characters from translations became mojibake.
Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 0d0b574cc45045d6150d397776340c068df59e2a)
Also warn about unpaired surrogates and (somewhat UTF-8-specific)
MAX_PATH issue in FindFirstFileA().
Fixes: 46ee0061629fb075d61d83839e14dd193337af59
(cherry picked from commit 20dfca8171dad4c64785ac61d5b68972c444877b)
If the early pledge() call on OpenBSD fails, it calls my_errorf()
which requires the "progname" variable.
Fixes: d74fb5f060b76db709b50f5fd37490394e52f975
(cherry picked from commit 4e936f234056e5831013ed922145b666b04bb1e3)
IMPORTANT: This includes a security fix to command line tool
argument handling.
Some toolchains embed an application manifest by default to declare
UAC-compliance. Some also declare compatibility with Vista/8/8.1/10/11
to let the app access features newer than those of Vista.
We want all the above but also two more things:
- Declare that the app is long path aware to support paths longer
than 259 characters (this may also require a registry change).
- Force the code page to UTF-8. This allows the command line tools
to access files whose names contain characters that don't exist
in the current legacy code page (except unpaired surrogates).
The UTF-8 code page also fixes security issues in command line
argument handling which can be exploited with malicious filenames.
See the new file w32_application.manifest.comments.txt.
Thanks to Orange Tsai and splitline from DEVCORE Research Team
for discovering this issue.
Thanks to Vijay Sarvepalli for reporting the issue to me.
Thanks to Kelvin Lee for testing with MSVC and helping with
the required build system fixes.
(cherry picked from commit 46ee0061629fb075d61d83839e14dd193337af59)
Now the information in the "Details" tab in the file properties
dialog matches the naming convention of Cygwin and MSYS2. This
is only a cosmetic change.
(cherry picked from commit dad153091552b52a41b95ec4981c6951f1cae487)
LANGUAGE and VS_VERSION_INFO begin new statements so put an empty line
between them.
(cherry picked from commit 8940ecb96fe9f0f2a9cfb8b66fe9ed31ffbea904)
The MB output can overflow with huge numbers. Most likely these are
invalid .lzma files anyway, but let's avoid garbage output.
lzmadec was adapted from LZMA Utils. The original code with this bug
was written in 2005, over 19 years ago.
Co-authored-by: Lasse Collin <lasse.collin@tukaani.org>
Closes: https://github.com/tukaani-project/xz/pull/144
(cherry picked from commit 76cfd0a9bb33ae8e534b1f73f6359dc825589f2f)
"xzdec -M123" exited with exit status 1 without printing
any messages. The "M:" entry should have been removed when
the memory usage limiter support was removed from xzdec.
Fixes: 792331bdee706aa852a78b171040ebf814c6f3ae
Closes: https://github.com/tukaani-project/xz/pull/143
[ Lasse: Commit message edits ]
(cherry picked from commit 78355aebb7fb654302e5e33692ba109909dacaff)
Support for instruction "movzw" without suffix in "GNU as" was
added in commit [1] and stabilized in binutils 2.27, released
in August 2016. Earlier systems don't accept this instruction
without a suffix, making range_decoder.h's inline assembly
unable to build on old systems such as Ubuntu 16.04, creating
error messages like:
lzma_decoder.c: Assembler messages:
lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi'
lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi'
lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx'
lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi'
Change "movzw" to "movzwl" for compatibility.
[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2
Suggested-by: Lasse Collin <lasse.collin@tukaani.org>
Tested-by: Yifeng Li <tomli@tomli.me>
Signed-off-by: Yifeng Li <tomli@tomli.me>
Fixes: 3182a330c1512cc1f5c87b5c5a272578e60a5158
Fixes: https://github.com/tukaani-project/xz/issues/121
Closes: https://github.com/tukaani-project/xz/pull/136
(cherry picked from commit 6cd7c8607843c337edfe2c472aa316602a393754)
It won't be implemented. find + xargs is more flexible, for example,
it allows compressing small files in parallel. An example for that
has been included in the xz man page since 2010.
(cherry picked from commit baecfa142644eb5f5c6dd6f8e2f531c362fa3747)
On E2K the function compiles only due to compiler emulation but the
function is never used. It's cleaner to omit the function when it's
not needed even though it's a "static inline" function.
Thanks to Ilya Kurdyukov.
(cherry picked from commit 30a2d5d51006301a3ddab5ef1f5ff0a9d74dce6f)
Thanks to Sam James for spotting it.
Fixes: f644473a211394447824ea00518d0a214ff3f7f2
(cherry picked from commit 0a32d2072c598de281058b26dc08920fbf0cd2a1)
GCC 4.2 doesn't have __builtin_bswap16() and friends so tuklib_integer.h
tries to use OS-specific byte swap methods instead. On OpenBSD those
macros are swap16/32/64 instead of bswap16/32/64 like on other *BSDs
and Darwin.
An alternative to "#ifdef __OpenBSD__" could be "#ifdef swap16" as it
is a macro. But since OpenBSD seems to be a special case under this
special case of "*BSDs and Darwin", checking for __OpenBSD__ seems
the more conservative choice now.
Thanks to Christian Weisgerber and Brad Smith who both submitted
the same patch a few hours apart.
Co-authored-by: Christian Weisgerber <naddy@mips.inka.de>
Co-authored-by: Brad Smith <brad@comstyle.com>
Closes: https://github.com/tukaani-project/xz/pull/126
(cherry picked from commit 04b23addf3733873667675df2439725f076c2f36)
Solaris' GCC can't understand that our use is fine, unlike modern compilers:
```
list.c: In function 'print_totals_basic':
list.c:1191:4: error: format not a string literal, argument types not checked [-Werror=format-nonliteral]
uint64_to_str(totals.files, 0));
^~~~~~~~~~~~~
cc1: all warnings being treated as errors
```
It's presumably because of older gettext missing format attributes.
This is with `gcc (GCC) 7.3.0`.
(cherry picked from commit b69768c8bd1a34fde311935c551d061ba52d9a3f)
This is closer to what it was before the --filtersX support was added,
just extended to support for scaling all filter chains. The method
before this commit was an extended version of the original too but
it was done in a more complex way for no clear reason. In case of
an error, the complex version printed fewer informative messages
(a good thing) but it's not a sigificant benefit.
In the limit is too low even for single-threaded mode, the required
amount of memory is now reported like in 5.4.x instead of like in
5.5.1alpha - 5.6.1 which showed the original non-scaled usage. It
had been a FIXME in the old code but it's not clear what message
makes the most sense.
Fixes: 5f0c5a04388f8334962c70bc37a8c2ff8f605e0a
(cherry picked from commit d9e1ae79ec90d6a7eafeaceaf0ece4f0c83d4417)
