Commit Graph

181 Commits

Author SHA1 Message Date
Lasse Collin 73c9e6d6b9 Build: Fix config.h comments. 2023-01-02 17:05:07 +02:00
Jia Tan 74dae7d300 Build: No longer require HAVE_DECL_CLOCK_MONOTONIC to always be set.
Previously, if threading was enabled HAVE_DECL_CLOCK_MONOTONIC would always
be set to 0 or 1. However, this macro was needed in xz so if xz was not
built with threading and HAVE_DECL_CLOCK_MONOTONIC was not defined but
HAVE_CLOCK_GETTIME was, it caused a warning during build. Now,
HAVE_DECL_CLOCK_MONOTONIC has been renamed to HAVE_CLOCK_MONOTONIC and
will only be set if it is 1.
2022-12-30 23:34:31 +08:00
Lasse Collin 764955e2d4 Change the bug report address.
It forwards to me and Jia Tan.

Also update the IRC reference in README as #tukaani was moved
to Libera Chat long ago.
2022-11-30 18:08:34 +02:00
Lasse Collin 06824396b2 Build: Don't put GNU/Linux-specific symbol versions into static liblzma.
It not only makes no sense to put symbol versions into a static library
but it can also cause breakage.

By default Libtool #defines PIC if building a shared library and
doesn't define it for static libraries. This is documented in the
Libtool manual. It can be overriden using --with-pic or --without-pic.
configure.ac detects if --with-pic or --without-pic is used and then
gives an error if neither --disable-shared nor --disable-static was
used at the same time. Thus, in normal situations it works to build
both shared and static library at the same time on GNU/Linux,
only --with-pic or --without-pic requires that only one type of
library is built.

Thanks to John Paul Adrian Glaubitz from Debian for reporting
the problem that occurred on ia64:
https://www.mail-archive.com/xz-devel@tukaani.org/msg00610.html
2022-11-24 14:52:44 +02:00
Lasse Collin f644473a21 liblzma: Add fast CRC64 for 32/64-bit x86 using SSSE3 + SSE4.1 + CLMUL.
It also works on E2K as it supports these intrinsics.

On x86-64 runtime detection is used so the code keeps working on
older processors too. A CLMUL-only build can be done by using
-msse4.1 -mpclmul in CFLAGS and this will reduce the library
size since the generic implementation and its 8 KiB lookup table
will be omitted.

On 32-bit x86 this isn't used by default for now because by default
on 32-bit x86 the separate assembly file crc64_x86.S is used.
If --disable-assembler is used then this new CLMUL code is used
the same way as on 64-bit x86. However, a CLMUL-only build
(-msse4.1 -mpclmul) won't omit the 8 KiB lookup table on
32-bit x86 due to a currently-missing check for disabled
assembler usage.

The configure.ac check should be such that the code won't be
built if something in the toolchain doesn't support it but
--disable-clmul-crc option can be used to unconditionally
disable this feature.

CLMUL speeds up decompression of files that have compressed very
well (assuming CRC64 is used as a check type). It is know that
the CLMUL code is significantly slower than the generic code for
tiny inputs (especially 1-8 bytes but up to 16 bytes). If that
is a real-world problem then there is already a commented-out
variant that uses the generic version for small inputs.

Thanks to Ilya Kurdyukov for the original patch which was
derived from a white paper from Intel [1] (published in 2009)
and public domain code from [2] (released in 2016).

[1] https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf
[2] https://github.com/rawrunprotected/crc
2022-11-14 23:05:46 +02:00
Lasse Collin ac10b1b362 Build: Omit x86_64 from --enable-assembler.
It didn't do anything. There are only 32-bit x86 assembly files
and it feels likely that new files won't be added as intrinsics
in C are more portable across toolchains and OSes.
2022-11-14 17:58:07 +02:00
Lasse Collin eb0f1450ad liblzma: Use __attribute__((__constructor__)) if available.
This uses it for CRC table initializations when using --disable-small.
It avoids mythread_once() overhead. It also means that then
--disable-small --disable-threads is thread-safe if this attribute
is supported.
2022-11-14 16:00:52 +02:00
Lasse Collin 0538db038f liblzma: Add .lz (lzip) decompression support (format versions 0 and 1).
Support for format version 0 was removed from lzip 1.18 for some
reason. .lz format version 0 files are rare (and old) but some
source packages were released in this format, and some people might
have personal files in this format too. It's very little extra code
to support it along side format version 1 so this commits adds
support for both.

The Sync Flush marker extentension to the original .lz format
version 1 isn't supported. It would require changes to the
LZMA decoder itself. Such files are very rare anyway.

See the API doc for lzma_lzip_decoder() for more details about
the .lz format support.

Thanks to Michał Górny for the original patch.
2022-11-09 14:24:20 +02:00
Lasse Collin cf118c0146 Build: Clarify comment in configure.ac about SSE2. 2022-10-31 16:26:05 +02:00
Lasse Collin aad3c609ff Build: Remove obsolete commented-out lines from configure.ac. 2022-10-31 16:16:37 +02:00
Lasse Collin a4b214b93a Build: Use AC_CONFIG_HEADERS instead of the ancient AC_CONFIG_HEADER.
We require Autoconf >= 2.69 and that has AC_CONFIG_HEADERS.

There is a warning about AC_PROG_CC_C99 being obsolete but
it cannot be removed because it is needed with Autoconf 2.69.
2022-10-25 23:45:03 +03:00
Lasse Collin 59c4d6e139 Build: Add configure option --disable-microlzma.
MicroLZMA was made for EROFS and used by erofs-utils.
It might be used by something else in the future but
those wanting a smaller build for specific situations
can now disable this rarely-needed feature.
2022-10-25 23:28:34 +03:00
Lasse Collin 563288ea70 xz: Add support for OpenBSD's pledge() sandbox. 2022-10-25 21:30:48 +03:00
Lasse Collin ecb966de30 liblzma: Add experimental ARM64 BCJ filter with a temporary Filter ID.
That is, the Filter ID will be changed once the design is final.
The current version will be removed. So files created with the
tempoary Filter ID won't be supported in the future.
2022-09-19 20:23:46 +03:00
Lasse Collin 913ddc5572 liblzma: Vaccinate against an ill patch from RHEL/CentOS 7.
RHEL/CentOS 7 shipped with 5.1.2alpha, including the threaded
encoder that is behind #ifdef LZMA_UNSTABLE in the API headers.
In 5.1.2alpha these symbols are under XZ_5.1.2alpha in liblzma.map.
API/ABI compatibility tracking isn't done between development
releases so newer releases didn't have XZ_5.1.2alpha anymore.

Later RHEL/CentOS 7 updated xz to 5.2.2 but they wanted to keep
the exported symbols compatible with 5.1.2alpha. After checking
the ABI changes it turned out that >= 5.2.0 ABI is backward
compatible with the threaded encoder functions from 5.1.2alpha
(but not vice versa as fixes and extensions to these functions
were made between 5.1.2alpha and 5.2.0).

In RHEL/CentOS 7, XZ Utils 5.2.2 was patched with
xz-5.2.2-compat-libs.patch to modify liblzma.map:

  - XZ_5.1.2alpha was added with lzma_stream_encoder_mt and
    lzma_stream_encoder_mt_memusage. This matched XZ Utils 5.1.2alpha.

  - XZ_5.2 was replaced with XZ_5.2.2. It is clear that this was
    an error; the intention was to keep using XZ_5.2 (XZ_5.2.2
    has never been used in XZ Utils). So XZ_5.2.2 lists all
    symbols that were listed under XZ_5.2 before the patch.
    lzma_stream_encoder_mt and _mt_memusage are included too so
    they are listed both here and under XZ_5.1.2alpha.

The patch didn't add any __asm__(".symver ...") lines to the .c
files. Thus the resulting liblzma.so exports the threaded encoder
functions under XZ_5.1.2alpha only. Listing the two functions
also under XZ_5.2.2 in liblzma.map has no effect without
matching .symver lines.

The lack of XZ_5.2 in RHEL/CentOS 7 means that binaries linked
against unpatched XZ Utils 5.2.x won't run on RHEL/CentOS 7.
This is unfortunate but this alone isn't too bad as the problem
is contained within RHEL/CentOS 7 and doesn't affect users
of other distributions. It could also be fixed internally in
RHEL/CentOS 7.

The second problem is more serious: In XZ Utils 5.2.2 the API
headers don't have #ifdef LZMA_UNSTABLE for obvious reasons.
This is true in RHEL/CentOS 7 version too. Thus now programs
using new APIs can be compiled without an extra #define. However,
the programs end up depending on symbol version XZ_5.1.2alpha
(and possibly also XZ_5.2.2) instead of XZ_5.2 as they would
with an unpatched XZ Utils 5.2.2. This means that such binaries
won't run on other distributions shipping XZ Utils >= 5.2.0 as
they don't provide XZ_5.1.2alpha or XZ_5.2.2; they only provide
XZ_5.2 (and XZ_5.0). (This includes RHEL/CentOS 8 as the patch
luckily isn't included there anymore with XZ Utils 5.2.4.)

Binaries built by RHEL/CentOS 7 users get distributed and then
people wonder why they don't run on some other distribution.
Seems that people have found out about the patch and been copying
it to some build scripts, seemingly curing the symptoms but
actually spreading the illness further and outside RHEL/CentOS 7.

The ill patch seems to be from late 2016 (RHEL 7.3) and in 2017 it
had spread at least to EasyBuild. I heard about the events only
recently. :-(

This commit splits liblzma.map into two versions: one for
GNU/Linux and another for other OSes that can use symbol versioning
(FreeBSD, Solaris, maybe others). The Linux-specific file and the
matching additions to .c files add full compatibility with binaries
that have been built against a RHEL/CentOS-patched liblzma. Builds
for OSes other than GNU/Linux won't get the vaccine as they should
be immune to the problem (I really hope that no build script uses
the RHEL/CentOS 7 patch outside GNU/Linux).

The RHEL/CentOS compatibility symbols XZ_5.1.2alpha and XZ_5.2.2
are intentionally put *after* XZ_5.2 in liblzma_linux.map. This way
if one forgets to #define HAVE_SYMBOL_VERSIONS_LINUX when building,
the resulting liblzma.so.5 will have lzma_stream_encoder_mt@@XZ_5.2
since XZ_5.2 {...} is the first one that lists that function.
Without HAVE_SYMBOL_VERSIONS_LINUX @XZ_5.1.2alpha and @XZ_5.2.2
will be missing but that's still a minor problem compared to
only having lzma_stream_encoder_mt@@XZ_5.1.2alpha!

The "local: *;" line was moved to XZ_5.0 so that it doesn't need
to be moved around. It doesn't matter where it is put.

Having two similar liblzma_*.map files is a bit silly as it is,
at least for now, easily possible to generate the generic one
from the Linux-specific file. But that adds extra steps and
increases the risk of mistakes when supporting more than one
build system. So I rather maintain two files in parallel and let
validate_map.sh check that they are in sync when "make mydist"
is run.

This adds .symver lines for lzma_stream_encoder_mt@XZ_5.2.2 and
lzma_stream_encoder_mt_memusage@XZ_5.2.2 even though these
weren't exported by RHEL/CentOS 7 (only @@XZ_5.1.2alpha was
for these two). I added these anyway because someone might
misunderstand the RHEL/CentOS 7 patch and think that @XZ_5.2.2
(@@XZ_5.2.2) versions were exported too.

At glance one could suggest using __typeof__ to copy the function
prototypes when making aliases. However, this doesn't work trivially
because __typeof__ won't copy attributes (lzma_nothrow, lzma_pure)
and it won't change symbol visibility from hidden to default (done
by LZMA_API()). Attributes could be copied with __copy__ attribute
but that needs GCC 9 and a fallback method would be needed anyway.

This uses __symver__ attribute with GCC >= 10 and
__asm__(".symver ...") with everything else. The attribute method
is required for LTO (-flto) support with GCC. Using -flto with
GCC older than 10 is now broken on GNU/Linux and will not be fixed
(can silently result in a broken liblzma build that has dangerously
incorrect symbol versions). LTO builds with Clang seem to work
with the traditional __asm__(".symver ...") method.

Thanks to Boud Roukema for reporting the problem and discussing
the details and testing the fix.
2022-09-08 15:01:29 +03:00
Jia Tan 4d80b463a1 Build: Don't allow empty LIST in --enable-match-finders=LIST.
It's enforced only when a match finder is needed, that is,
when LZMA1 or LZMA2 encoder is enabled.
2022-07-25 18:20:01 +03:00
Lasse Collin 4773608554 Build: Enable Automake's parallel test harness.
It has been the default for quite some time already and
the old serial harness isn't discouraged. The downside is
that with parallel tests one cannot print progress info or
other diagnostics to the terminal; all output from the tests
will be in the log files only. But now that the compression
tests are separated the parallel tests will speed things up.
2022-05-23 21:31:36 +03:00
Lasse Collin feb9c1969b Build: Bump Autoconf and Libtool version requirements.
There is no specific reason for this other than blocking
the most ancient versions. These are still old:

Autoconf 2.69 (2012)
Automake 1.12 (2012)
gettext 0.19.6 (2015)
Libtool 2.4 (2010)
2020-02-15 15:07:11 +02:00
Lasse Collin 3d576cf921 Build: Use AM_GNU_GETTEXT_REQUIRE_VERSION and require 0.19.6.
This bumps the version requirement from 0.19 (from 2014) to
0.19.6 (2015).

Using only the old AM_GNU_GETTEXT_VERSION results in old
gettext infrastructure being placed in the package. By using
both macros we get the latest gettext files while the other
programs in the Autotools family can still see the old macro.
2020-02-15 15:05:35 +02:00
Lasse Collin 986d8c9b52 Build: Workaround a POSIX shell detection problem on Solaris.
I don't know if the problem is in gnulib's gl_POSIX_SHELL macro
or if xzgrep does something that isn't in POSIX. The workaround
adds a special case for Solaris: if /usr/xpg4/bin/sh exists and
gl_cv_posix_shell wasn't overriden on the configure command line,
use that shell for xzgrep and other scripts. That shell is known
to work and exists on most Solaris systems.
2020-02-05 19:33:37 +02:00
Lasse Collin 43ce4ea7c7 Scripts: Put /usr/xpg4/bin to the beginning of PATH on Solaris.
This adds a configure option --enable-path-for-scripts=PREFIX
which defaults to empty except on Solaris it is /usr/xpg4/bin
to make POSIX grep and others available. The Solaris case had
been documented in INSTALL with a manual fix but it's better
to do this automatically since it is needed on most Solaris
systems anyway.

Thanks to Daniel Richard G.
2019-09-24 23:02:40 +03:00
Lasse Collin fc77929e92 configure.ac: Fix a typo in a comment. 2019-06-25 00:16:06 +03:00
Lasse Collin 662b27c417 Update the home page URLs to HTTPS. 2017-04-19 22:17:35 +03:00
Lasse Collin 446e4318fa xz: Fix copying of timestamps on Windows.
xz used to call utime() on Windows, but its result gets lost
on close(). Using _futime() seems to work.

Thanks to Martok for reporting the bug:
http://www.mail-archive.com/xz-devel@tukaani.org/msg00261.html
2016-06-30 20:27:36 +03:00
Lasse Collin ade31a2bfb Build: Bump GNU Gettext version requirement to 0.19.
It silences a few warnings and most people probably have
0.19 even on stable distributions.

Thanks to Christian Kujau.
2016-04-10 20:54:17 +03:00
Lasse Collin ac398c3baf liblzma: Disable external SHA-256 by default.
This is the sane thing to do. The conflict with OpenSSL
on some OSes and especially that the OS-provided versions
can be significantly slower makes it clear that it was
a mistake to have the external SHA-256 support enabled by
default.

Those who want it can now pass --enable-external-sha256 to
configure. INSTALL was updated with notes about OSes where
this can be a bad idea.

The SHA-256 detection code in configure.ac had some bugs that
could lead to a build failure in some situations. These were
fixed, although it doesn't matter that much now that the
external SHA-256 is disabled by default.

MINIX >= 3.2.0 uses NetBSD's libc and thus has SHA256_Init
in libc instead of libutil. Support for the libutil version
was removed.
2016-03-13 20:21:49 +02:00
Lasse Collin 473ef0dc69 Build: Avoid SHA256_Init on FreeBSD and MINIX 3.
On FreeBSD 10 and older, SHA256_Init from libmd conflicts
with libcrypto from OpenSSL. The OpenSSL version has
different sizeof(SHA256_CTX) and it can cause weird
problems if wrong SHA256_Init gets used.

Looking at the source, MINIX 3 seems to have a similar issue but
I'm not sure. To be safe, I disabled SHA256_Init on MINIX 3 too.

NetBSD has SHA256_Init in libc and they had a similar problem,
but they already fixed it in 2009.

Thanks to Jim Wilcoxson for the bug report that helped
in finding the problem.
2016-03-10 20:26:49 +02:00
Lasse Collin c7bc20a6f3 Build: Disable xzdec, lzmadec, and lzmainfo when they cannot be built.
They all need decoder support and if that isn't available,
there's no point trying to build them.
2015-11-03 20:47:07 +02:00
Lasse Collin 5cbca1205d Build: Simplify $enable_{encoders,decoders} usage a bit. 2015-11-03 20:35:19 +02:00
Lasse Collin cb3111e3ed xz: Make xz buildable even when encoders or decoders are disabled.
The patch is quite long but it's mostly about adding new #ifdefs
to omit code when encoders or decoders have been disabled.

This adds two new #defines to config.h: HAVE_ENCODERS and
HAVE_DECODERS.
2015-11-03 20:29:33 +02:00
Lasse Collin 23ed1d4148 Build: Fix configure to handle LZMA1 dependency with LZMA2.
Now it gives an error if LZMA1 encoder/decoder is missing
when LZMA2 encoder/decoder was requested. Even better would
be LZMA2 implicitly enabling LZMA1 but it would need more code.
2015-11-03 17:54:48 +02:00
Lasse Collin cbc9e39bae Build: Minor Cygwin cleanup.
Some tests used "cygwin*" and some used "cygwin". I changed
them all to use "cygwin". Shouldn't affect anything in practice.
2015-08-11 13:23:04 +03:00
Lasse Collin bcacd8ce7a Build: Support building of MSYS2 binaries. 2015-08-11 13:21:52 +03:00
Lasse Collin 6bd0349c58 Revert "xz: Use pipe2() if available."
This reverts commit 7a11c4a8e5.
It is a problem when libc has pipe2() but the kernel is too
old to have pipe2() and thus pipe2() fails. In xz it's pointless
to have a fallback for non-functioning pipe2(); it's better to
avoid pipe2() completely.

Thanks to Michael Fox for the bug report.
2015-04-20 20:17:48 +03:00
Lasse Collin 1238381143 xz: Add support for sandboxing with Capsicum.
The sandboxing is used conditionally as described in main.c.
This isn't optimal but it was much easier to implement than
a full sandboxing solution and it still covers the most common
use cases where xz is writing to standard output. This should
have practically no effect on performance even with small files
as fork() isn't needed.

C and locale libraries can open files as needed. This has been
fine in the past, but it's a problem with things like Capsicum.
io_sandbox_enter() tries to ensure that various locale-related
files have been loaded before cap_enter() is called, but it's
possible that there are other similar problems which haven't
been seen yet.

Currently Capsicum is available on FreeBSD 10 and later
and there is a port to Linux too.

Thanks to Loganaden Velvindron for help.
2015-03-31 22:19:34 +03:00
Lasse Collin 7a11c4a8e5 xz: Use pipe2() if available. 2015-02-22 19:38:48 +02:00
Lasse Collin 7815112153 Build: POSIX shell isn't required if scripts are disabled. 2014-12-26 12:00:05 +02:00
Lasse Collin b13a781833 Build: Replace obsolete AC_HELP_STRING with AS_HELP_STRING. 2014-11-17 18:52:21 +02:00
Lasse Collin 542cac122e Build: Fix Autoconf warnings about escaped backquotes.
Thanks to Daniel Richard G. for pointing out that it's
good to sometimes run autoreconf -fi with -Wall.
2014-11-17 18:43:19 +02:00
Lasse Collin c923b140b2 Build: Prepare to support Automake's subdir-objects.
Due to a bug in Automake, subdir-objects won't be enabled
for now.

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17354

Thanks to Daniel Richard G. for the original patches.
2014-10-29 21:15:35 +02:00
Lasse Collin c4911f2db3 Build: Detect supported compiler warning flags better.
Clang and nowadays also GCC accept any -Wfoobar option
but then may give a warning that an unknown warning option
was specified. To avoid adding unsupported warning options,
the options are now tested with -Werror.

Thanks to Charles Diza.
2014-09-25 18:38:48 +03:00
Lasse Collin e1c8f1d01f liblzma: Add lzma_memcmplen() for fast memory comparison.
This commit just adds the function. Its uses will be in
separate commits.

This hasn't been tested much yet and it's perhaps a bit early
to commit it but if there are bugs they should get found quite
quickly.

Thanks to Jun I Jin from Intel for help and for pointing out
that string comparison needs to be optimized in liblzma.
2014-07-25 20:57:20 +03:00
Lasse Collin 56056571df Build: Add --disable-doc to configure. 2014-04-25 17:44:26 +03:00
Lasse Collin 6b44b4a775 Add native threading support on Windows.
Now liblzma only uses "mythread" functions and types
which are defined in mythread.h matching the desired
threading method.

Before Windows Vista, there is no direct equivalent to
pthread condition variables. Since this package doesn't
use pthread_cond_broadcast(), pre-Vista threading can
still be kept quite simple. The pre-Vista code doesn't
use anything that wasn't already available in Windows 95,
so the binaries should run even on Windows 95 if someone
happens to care.
2013-09-17 11:52:28 +03:00
Lasse Collin ae0ab74a88 Build: Remove a comment about Automake 1.10 from configure.ac.
The previous commit supports silent rules and that requires
Automake 1.11.
2013-09-11 14:40:35 +03:00
Lasse Collin 72975df6c8 Build: Create liblzma.pc in a src/liblzma/Makefile.am.
Previously it was done in configure, but doing that goes
against the Autoconf manual. Autoconf requires that it is
possible to override e.g. prefix after running configure
and that doesn't work correctly if liblzma.pc is created
by configure.

A potential downside of this change is that now e.g.
libdir in liblzma.pc is a standalone string instead of
being defined via ${prefix}, so if one overrides prefix
when running pkg-config the libdir won't get the new value.
I don't know if this matters in practice.

Thanks to Vincent Torri.
2013-09-09 20:37:03 +03:00
Lasse Collin 3e2b198ba3 Build: Fix the detection of missing CRC32.
Thanks to Vincent Torri.
2013-07-15 14:08:02 +03:00
Lasse Collin 9376f5f8f7 Build: Require Automake 1.12 and use serial-tests option.
It should actually still work with Automake 1.10 if
the serial-tests option is removed. Automake 1.13 started
using parallel tests by default and the option to get
the old behavior isn't supported before 1.12.

At least for now, parallel tests don't improve anything
in XZ Utils but they hide the progress output from
test_compress.sh.
2013-06-26 12:17:00 +03:00
Lasse Collin 97379c5ea7 Build: Use -Wvla with GCC if supported.
Variable-length arrays are mandatory in C99 but optional in C11.
The code doesn't currently use any VLAs and it shouldn't in the
future either to stay compatible with C11 without requiring any
optional C11 features.
2013-04-27 22:07:46 +03:00
Lasse Collin ab22562066 A few typo fixes to comments and the xz man page.
Thanks to Jim Meyering.
2012-08-24 16:27:31 +03:00
Lasse Collin d8eaf9d827 Build: Bump gettext version requirement to 0.18.
Otherwise too old version of m4/lib-link.m4 gets included
when autoreconf -fi is run.
2012-08-02 17:13:30 +03:00
Lasse Collin df11317985 Build: Add a comment to configure.ac about symbol versioning. 2012-07-04 17:11:31 +03:00
Lasse Collin 2dcea03712 Fix compiling with IBM XL C on AIX. 2012-02-22 12:00:16 +02:00
Lasse Collin e9ed88126e Run the scripts with the correct shell in test_scripts.sh.
The scripts are now made executable in the build tree.
This way the scripts can be run like programs in
test_scripts.sh. Previously test_scripts.sh always
used sh but it's not correct if @POSIX_SHELL@ is set
to something else by configure.

Thanks to Jonathan Nieder for the patch.
2011-08-06 20:37:28 +03:00
Lasse Collin 492c863455 Build: Make configure print if symbol versioning is enabled or not. 2011-05-28 19:24:56 +03:00
Lasse Collin bd35d903a0 liblzma: Use symbol versioning.
Symbol versioning is enabled by default on GNU/Linux,
other GNU-based systems, and FreeBSD.

I'm not sure how stable this is, so it may need
backward-incompatible changes before the next release.

The idea is that alpha and beta symbols are considered
unstable and require recompiling the applications that
use those symbols. Once a symbol is stable, it may get
extended with new features in ways that don't break
compatibility with older ABI & API.

The mydist target runs validate_map.sh which should
catch some probable problems in liblzma.map. Otherwise
I would forget to update the map file for new releases.
2011-05-28 15:55:39 +03:00
Lasse Collin fe00f95828 Build: Fix checking for system-provided SHA-256. 2011-05-24 00:23:46 +03:00
Lasse Collin b94aa0c838 liblzma: Try to use SHA-256 from the operating system.
If the operating system libc or other base libraries
provide SHA-256, use that instead of our own copy.
Note that this doesn't use OpenSSL or libgcrypt or
such libraries to avoid creating dependencies to
other packages.

This supports at least FreeBSD, NetBSD, OpenBSD, Solaris,
MINIX, and Darwin. They all provide similar but not
identical SHA-256 APIs; everyone is a little different.

Thanks to Wim Lewis for the original patch, improvements,
and testing.
2011-05-21 15:08:44 +03:00
Lasse Collin ec7106309c Build: Fix initialization of enable_check_* variables in configure.ac.
This doesn't matter much in practice since it is unlikely
that anyone would have such environment variable names.

Thanks to Wim Lewis.
2011-05-17 12:01:37 +03:00
Lasse Collin 0b77c4a751 Build: Warn if no supported method to detect the number of CPU cores. 2011-04-19 10:44:48 +03:00
Lasse Collin 9c1b05828a Fix portability problems in mythread.h.
Use gettimeofday() if clock_gettime() isn't available
(e.g. Darwin).

The test for availability of pthread_condattr_setclock()
and CLOCK_MONOTONIC was incorrect. Instead of fixing the
#ifdefs, use an Autoconf test. That way if there exists a
system that supports them but doesn't specify the matching
POSIX #defines, the features will still get detected.

Don't try to use pthread_sigmask() on OpenVMS. It doesn't
have that function.

Guard mythread.h against being #included multiple times.
2011-04-19 09:20:44 +03:00
Lasse Collin de678e0c92 liblzma: Add lzma_stream_encoder_mt() for threaded compression.
This is the simplest method to do threading, which splits
the uncompressed data into blocks and compresses them
independently from each other. There's room for improvement
especially to reduce the memory usage, but nevertheless,
this is a good start.
2011-04-11 22:03:30 +03:00
Lasse Collin 9f0a806aef Revise mythread.h.
This adds:

  - mythread_sync() macro to create synchronized blocks

  - mythread_cond structure and related functions
    and macros for condition variables with timed
    waiting using a relative timeout

  - mythread_create() to create a thread with all
    signals blocked

Some of these wouldn't need to be inline functions,
but I'll keep them this way for now for simplicity.

For timed waiting on a condition variable, librt is
now required on some systems to use clock_gettime().
configure.ac was updated to handle this.
2011-04-10 21:23:21 +03:00
Lasse Collin db33117cc8 Build: Upgrade m4/acx_pthread.m4 to the latest version.
It was renamed to ax_pthread.m4 in Autoconf Archive.
2011-04-05 17:12:20 +03:00
Lasse Collin 1039bfcfc0 xz: Use posix_fadvise() if it is available. 2011-04-05 15:27:26 +03:00
Lasse Collin 9311774c49 Build: Enable ASM on DJGPP by default. 2010-12-12 21:23:55 +02:00
Lasse Collin 084c60d318 configure.ac: Remove two unused defines. 2010-10-08 15:59:25 +03:00
Lasse Collin b1c7368f95 Build: Add options to disable individual command line tools. 2010-10-08 15:25:45 +03:00
Lasse Collin f9907503f8 Build: Remove the static/dynamic tricks.
Most distros want xz linked against shared liblzma, so
it doesn't help much to require --enable-dynamic for that.
Those who want to avoid PIC on x86-32 to get better
performance, can still do it e.g. by using --disable-shared
to compile xz and then another pass to compile shared liblzma.

Part of these static/dynamic tricks were needed for Windows
in the past. Nowadays we rely on GCC and binutils to do the
right thing with auto-import. If the Autotooled build system
needs to support some other toolchain on Windows in the future,
this may need some rethinking.
2010-10-05 14:13:16 +03:00
Lasse Collin fda4724d81 configure.ac: Silence a warning from Autoconf 2.68. 2010-10-05 12:18:58 +03:00
Lasse Collin 17d3c61edd Move version.sh to build-aux. 2010-09-28 10:59:53 +03:00
Lasse Collin bb0b1004f8 xz: Multiple fixes.
The code assumed that printing numbers with thousand separators
and decimal points would always produce only US-ASCII characters.
This was used for buffer sizes (with snprintf(), no overflows)
and aligning columns of the progress indicator and --list. That
assumption was wrong (e.g. LC_ALL=fi_FI.UTF-8 with glibc), so
multibyte character support was added in this commit. The old
way is used if the operating system doesn't have enough multibyte
support (e.g. lacks wcwidth()).

The sizes of buffers were increased to accomodate multibyte
characters. I don't know how big they should be exactly, but
they aren't used for anything critical, so it's not too bad.
If they still aren't big enough, I hopefully get a bug report.
snprintf() takes care of avoiding buffer overflows.

Some static buffers were replaced with buffers allocated on
stack. double_to_str() was removed. uint64_to_str() and
uint64_to_nicestr() now share the static buffer and test
for thousand separator support.

Integrity check names "None" and "Unknown-N" (2 <= N <= 15)
were marked to be translated. I had forgot these, plus they
wouldn't have worked correctly anyway before this commit,
because printing tables with multibyte strings didn't work.

Thanks to Marek Černocký for reporting the bug about
misaligned table columns in --list output.
2010-09-10 10:30:33 +03:00
Lasse Collin d8a55c48b3 Remove the Subblock filter code for now.
The spec isn't finished and the code didn't compile anymore.
It won't be included in XZ Utils 5.0.0. It's easy to get it
back once the spec is done.
2010-05-26 09:55:47 +03:00
Lasse Collin 9886d436ff Change the default of --enable-assume-ram from 32 to 128 MiB.
This is to allow files created with "xz -9" to be decompressed
if the amount of RAM cannot be determined.
2010-03-07 13:34:34 +02:00
Lasse Collin eb7d51a3fa Collection of language fixes to comments and docs.
Thanks to Jonathan Nieder.
2010-02-12 13:16:15 +02:00
Lasse Collin b063cc34a3 Use PACKAGE_URL instead of custom PACKAGE_HOMEPAGE. 2010-01-27 13:31:03 +02:00
Lasse Collin b9b5c54cd4 Enable assembler code only if it is known to work
on that operating system.

I'm too lazy to think how to make a good Autoconf test
for this and it's not that important anyway.

No longer define HAVE_ASM_X86 or HAVE_ASM_X86_64.
Inline assembler (if any) is used if a macro like
__i386__ or __x86_64__ is defined.
2009-11-22 12:00:30 +02:00
Lasse Collin c74c132f7f Update tuklib_cpucores.m4 and tuklib_physmem.m4 from tuklib,
which now use AC_CACHE_CHECK. Using the cache variable,
configure now warns if there is no method to detect the amount
of RAM and recommends using --enable-assume-ram.
2009-11-20 12:51:19 +02:00
Lasse Collin ebfb2c5e1f Use a tuklib module for integer handling.
This replaces bswap.h and integer.h.

The tuklib module uses <byteswap.h> on GNU,
<sys/endian.h> on *BSDs and <sys/byteorder.h>
on Solaris, which may contain optimized code
like inline assembly.
2009-10-04 22:57:12 +03:00
Lasse Collin 29fd321033 Add support for --enable-assume-ram=SIZE. 2009-10-02 14:35:56 +03:00
Lasse Collin e599bba421 Various changes.
Separate a few reusable components from XZ Utils specific
code. The reusable code is now in "tuklib" modules. A few
more could be separated still, e.g. bswap.h.

Fix some bugs in lzmainfo.

Fix physmem and cpucores code on OS/2. Thanks to Elbert Pol
for help.

Add OpenVMS support into physmem. Add a few #ifdefs to ease
building XZ Utils on OpenVMS. Thanks to Jouk Jansen for the
original patch.
2009-09-19 09:47:30 +03:00
Lasse Collin 221be761f4 Use $(LN_EXEEXT) in symlinks to executables.
This fixes "make install" on operating systems using
a suffix for executables.

Cygwin is treated specially. The symlink names won't have
.exe suffix even though the executables themselves have.
Thanks to Charles Wilson.
2009-09-11 10:24:09 +03:00
Lasse Collin 696d7ee395 Require GNU libtool 2.2. 2009-08-27 15:43:54 +03:00
Lasse Collin 71f18e8a06 Make --enable-dynamic a tristate option.
Some programs will by default be linked against static
liblzma and some against shared liblzma. --enable-dynamic
now allows overriding the default to both directions
(all dynamic or all static) even when building both
shared and static liblzma.

This is quite messy compared to how simple thing it is supposed
to be. The complexity is mostly due to Windows support.
2009-08-13 15:00:21 +03:00
Lasse Collin e51b4e49e8 Add lzmainfo for backward compatibility with LZMA Utils.
lzmainfo now links against static liblzma. In contrast
to other command line tools in XZ Utils, linking lzmainfo
against static liblzma by default is dumb. This will be
fixed once I have fixed some related issues in configure.ac.
2009-08-13 12:55:45 +03:00
Lasse Collin 4c9c989d45 Use AC_CONFIG_AUX_DIR to clean up the toplevel directory
a little.

Fixed a related bug in the toplevel Makefile.am.

Added the build-aux directory to .gitignore.
2009-07-18 18:54:55 +03:00
Lasse Collin 96e4b257e1 Major update to the xzgrep and other scripts based on
the latest versions found from gzip CVS repository.

configure will try to find a POSIX shell to be used by
the scripts. This should ease portability on systems
which have pre-POSIX /bin/sh.

xzgrep and xzdiff support .xz, .lzma, .gz, and .bz2 files.
xzmore and xzless support only .xz and .lzma files.

The name of the xz executable used in these scripts is
now correct even if --program-transform-name has been used.
2009-07-05 22:25:17 +03:00
Lasse Collin 0a289c01ac Define PACKAGE_HOMEPAGE in configure.ac and use it in
xz and xzdec.

Use also PACKAGE_NAME instead of hardcoding "XZ Utils".
2009-07-02 14:30:38 +03:00
Lasse Collin 5cc99db5ba Avoid visibility related compiler warnings on Windows. 2009-07-01 12:21:24 +03:00
Lasse Collin f42ee98166 Build system fixes
Don't use libtool convenience libraries to avoid recently
discovered long-standing subtle but somewhat severe bugs
in libtool (at least 1.5.22 and 2.2.6 are affected). It
was found when porting XZ Utils to Windows
<http://lists.gnu.org/archive/html/libtool/2009-06/msg00070.html>
but the problem is significant also e.g. on GNU/Linux.

Unless --disable-shared is passed to configure, static
library built from a set of convenience libraries will
contain PIC objects. That is, while libtool builds non-PIC
objects too, only PIC objects will be used from the
convenience libraries. On 32-bit x86 (tested on mobile XP2400+),
using PIC instead of non-PIC makes the decompressor 10 % slower
with the default CFLAGS.

So while xz was linked against static liblzma by default,
it got the slower PIC objects unless --disable-shared was
used. I tend develop and benchmark with --disable-shared
due to faster build time, so I hadn't noticed the problem
in benchmarks earlier.

This commit also adds support for building Windows resources
into liblzma and executables.
2009-06-30 17:09:57 +03:00
Lasse Collin 390a640856 Basic support for building with Cygwin and MinGW using
the Autotools based build system. It's not good yet, more
fixes will follow.
2009-06-26 15:37:53 +03:00
Lasse Collin a6f43e6412 Use a GCC-specific #pragma instead of GCC-specific
-Wno-uninitialized to silence a bogus warning.
2009-05-02 16:16:28 +03:00
Lasse Collin f6ce63ebdb Removed --disable-encoder and --disable-decoder. Use the values
given to --enable-encoders and --enable-decoders to determine
if any encoder or decoder support is wanted.
2009-05-02 14:46:50 +03:00
Lasse Collin 02ddf09bc3 Put the interesting parts of XZ Utils into the public domain.
Some minor documentation cleanups were made at the same time.
2009-04-13 11:27:40 +03:00
Lasse Collin cf751edfde Added AC_CONFIG_MACRO_DIR to configure.ac. 2009-03-01 09:00:06 +02:00
Lasse Collin 3084d662d2 Cleanups to the code that detects the amount of RAM and
the number of CPU cores. Added support for using sysinfo()
on Linux systems whose libc lacks appropriate sysconf()
support (at least dietlibc). The Autoconf macros were
split into separate files, and CPU core count detection
was moved from hardware.c to cpucores.h. The core count
isn't used for anything real for now, so a problematic
part in process.c was commented out.
2009-02-14 00:45:29 +02:00
Lasse Collin fdbc0cfa71 Changed how the version number is specified in various places.
Now configure.ac will get the version number directly from
src/liblzma/api/lzma/version.h. The intent is to reduce the
number of places where the version number is duplicated. In
future, support for displaying Git commit ID may be added too.
2009-02-13 18:00:03 +02:00
Lasse Collin a3bbbe05d3 Let the user specify custom CFLAGS on the make command
line. Previously custom CFLAGS worked only when they were
passed to configure.
2009-02-09 14:54:31 +02:00
Lasse Collin 880c330938 Make it easy to choose if command line tools should be
linked statically or dynamically against liblzma. The
default is still to use static liblzma, but it can now
be changed by passing --enable-dynamic to configure.
Thanks to Mike Frysinger for the original patch.

Fixed a few minor bugs in configure.ac.
2009-02-07 21:17:07 +02:00
Lasse Collin 99c1c2abfa Updated the x86 assembler code:
- Use call/ret pair to get instruction pointer for PIC.
  - Use PIC only if PIC or __PIC__ is #defined.
  - The code should work on MinGW and Darwin in addition
    to GNU/Linux and Solaris.
2009-02-02 21:19:01 +02:00