Commit Graph

304 Commits

Author SHA1 Message Date
Lasse Collin b0063023f8 Make the default memory usage limit 40 % of RAM for both
compressing and decompressing. This should be OK now that
xz automatically scales down the compression settings if
they would exceed the memory usage limit (earlier, the limit
for compression was increased to 90 % because low limit broke
scripts that used "xz -9" on systems with low RAM).

Support spcifying the memory usage limit as a percentage
of RAM (e.g. --memory=50%).

Support --threads=0 to reset the thread limit to the default
value (number of available CPU cores). Use UINT32_MAX instead
of SIZE_MAX as the maximum in args.c. hardware.c was already
expecting uint32_t value.

Cleaned up the output of --help and --long-help.
2009-05-22 11:29:50 +03:00
Lasse Collin 071b825b23 Support special value "max" where xz and xzdec accept an integer.
Don't round the memory usage limit in xzdec --help to avoid
an integer overflow and to not give wrong impression that
the limit is high enough when it may not actually be.
2009-05-21 17:22:01 +03:00
ABCD 03ca67fd37 Install lzdiff, lzgrep, and lzmore as symlinks
This adds lzdiff, lzgrep, and lzmore to the list of symlinks to install.
It also installs symlinks for the manual pages and removes the new
symlinks on uninstall.
2009-05-21 16:25:36 +03:00
Lasse Collin a6f43e6412 Use a GCC-specific #pragma instead of GCC-specific
-Wno-uninitialized to silence a bogus warning.
2009-05-02 16:16:28 +03:00
Lasse Collin 21c6b94373 Fixed a crash in liblzma.
liblzma tries to avoid useless free()/malloc() pairs in
initialization when multiple files are handled using the
same lzma_stream. This didn't work with filter chains
due to comparison of wrong pointers in lzma_next_coder_init(),
making liblzma think that no memory reallocation is needed
even when it actually is.

Easy way to trigger this bug is to decompress two files with
a single xz command. The first file should have e.g. x86+LZMA2
as the filter chain, and the second file just LZMA2.
2009-04-28 23:08:32 +03:00
Lasse Collin e518d167aa Fix uint32_t -> size_t in ARM and ARM-Thumb filters.
On 64-bit system it would have gone into infinite
loop if a single input buffer was over 4 GiB (unlikely).
2009-04-15 14:13:38 +03:00
Lasse Collin 2f0bc9cd40 Quick & dirty update to support xz in diff/grep/more scripts. 2009-04-13 14:49:48 +03:00
Lasse Collin 02ddf09bc3 Put the interesting parts of XZ Utils into the public domain.
Some minor documentation cleanups were made at the same time.
2009-04-13 11:27:40 +03:00
Lasse Collin e79c42d854 Fix off-by-one in LZ decoder.
Fortunately, this bug had no security risk other than accepting
some corrupt files as valid.
2009-04-10 11:17:02 +03:00
Lasse Collin 390e69887f Fix wrong macro names in lc_cpucores.m4 and cpucores.h.
Thanks to Bert Wesarg.
2009-03-18 16:51:41 +02:00
Lasse Collin fd6a380f4e Add a rough explanation of --extreme to output of --help. 2009-02-22 19:07:54 +02:00
Lasse Collin 68bf7ac298 Fixes to progress message handling in xz:
- Don't use Windows-specific code on Windows. The old code
    required at least Windows 2000. Now it should work on
    Windows 98 and later, and maybe on Windows 95 too.

  - Use less precision when showing estimated remaining time.

  - Fix some small design issues.
2009-02-22 18:52:49 +02:00
Lasse Collin 47c2e21f82 Added files missing from the previous commit. 2009-02-18 13:00:10 +02:00
Lasse Collin 489a3dbaa0 Added lzma_easy_buffer_encode(). Splitted easy.c into small
pieces to avoid unneeded dependencies making statically
linked applications bigger than needed.
2009-02-17 10:43:00 +02:00
Lasse Collin 7494816ab0 Make physmem.h work on old Windows versions.
Thanks to Hongbo Ni for the original patch.
2009-02-15 15:48:45 +02:00
Lasse Collin 11ae4ae35f Fix microsecond vs. nanosecond confusion in my_time(). 2009-02-14 20:44:52 +02:00
Lasse Collin 3084d662d2 Cleanups to the code that detects the amount of RAM and
the number of CPU cores. Added support for using sysinfo()
on Linux systems whose libc lacks appropriate sysconf()
support (at least dietlibc). The Autoconf macros were
split into separate files, and CPU core count detection
was moved from hardware.c to cpucores.h. The core count
isn't used for anything real for now, so a problematic
part in process.c was commented out.
2009-02-14 00:45:29 +02:00
Lasse Collin fdbc0cfa71 Changed how the version number is specified in various places.
Now configure.ac will get the version number directly from
src/liblzma/api/lzma/version.h. The intent is to reduce the
number of places where the version number is duplicated. In
future, support for displaying Git commit ID may be added too.
2009-02-13 18:00:03 +02:00
Lasse Collin 1d924e584b Fix handling of integrity check type in the xz command line tool. 2009-02-13 17:30:30 +02:00
Lasse Collin 96c46df7de Improve support for DOS-like systems.
Here DOS-like means DOS, Windows, and OS/2.
2009-02-13 17:29:02 +02:00
Lasse Collin a3bbbe05d3 Let the user specify custom CFLAGS on the make command
line. Previously custom CFLAGS worked only when they were
passed to configure.
2009-02-09 14:54:31 +02:00
Lasse Collin 53f7598998 Fix aliasing issue in physmem.h. 2009-02-08 21:35:11 +02:00
Lasse Collin 0e27028d74 Add a separate internal function to initialize the CRC32
table, which is used also by LZ encoder. This was needed
because calling lzma_crc32() and ignoring the result is
a no-op due to lzma_attr_pure.
2009-02-08 18:24:50 +02:00
Lasse Collin ae1ad9af54 Make "xz --force" to write to terminal as the error
message suggests.
2009-02-08 18:17:05 +02:00
Lasse Collin 79e25eded4 Support both slash and backslash as path component
separator on Windows when parsing argv[0].
2009-02-08 10:37:50 +02:00
Lasse Collin 880c330938 Make it easy to choose if command line tools should be
linked statically or dynamically against liblzma. The
default is still to use static liblzma, but it can now
be changed by passing --enable-dynamic to configure.
Thanks to Mike Frysinger for the original patch.

Fixed a few minor bugs in configure.ac.
2009-02-07 21:17:07 +02:00
Lasse Collin bd7ca1dad5 Assume 32 MiB of RAM on unsupported operating systems like
the comment in hardware.c already said.
2009-02-07 17:07:52 +02:00
Lasse Collin bfd91198e4 Support LZMA_API_STATIC in assembler files to
avoid __declspec(dllexport) equivalent.
2009-02-07 15:55:47 +02:00
Lasse Collin 3306cf3883 Introduced LZMA_API_STATIC macro, which the applications
need to #define when linking against static liblzma on
platforms like Windows. Most developers don't need to
care about LZMA_API_STATIC at all.
2009-02-07 11:11:50 +02:00
Lasse Collin 75905a9afc Various code cleanups the the xz command line tool.
It now builds with MinGW.
2009-02-05 09:12:57 +02:00
Lasse Collin d0c0b9e94e Another utime() fix. 2009-02-03 12:15:17 +02:00
Lasse Collin ccf92a29e8 Fix wrong filename argument for utime() and utimes().
This doesn't affect most systems, since most systems
have better functions available.
2009-02-03 10:41:11 +02:00
Lasse Collin 99c1c2abfa Updated the x86 assembler code:
- Use call/ret pair to get instruction pointer for PIC.
  - Use PIC only if PIC or __PIC__ is #defined.
  - The code should work on MinGW and Darwin in addition
    to GNU/Linux and Solaris.
2009-02-02 21:19:01 +02:00
Lasse Collin 22a0c6dd94 Modify LZMA_API macro so that it works on Windows with
other compilers than MinGW. This may hurt readability
of the API headers slightly, but I don't know any
better way to do this.
2009-02-02 20:14:03 +02:00
Lasse Collin 8dd7b6052e Fix a bug in lzma_block_buffer_decode(), although this
function should be rewritten anyway.
2009-02-01 22:40:35 +02:00
Lasse Collin 55fd41431e Added initial version of raw buffer-to-buffer coding
functions, and cleaned up filter.h API header a little.
May be very buggy, not tested yet.
2009-02-01 22:39:07 +02:00
Lasse Collin 3e54ecee5c Fix missing newlines in xzdec.c. 2009-02-01 00:11:20 +02:00
Lasse Collin d64ca34f1b Use __cdecl also for function pointers in liblzma API when
on Windows.
2009-02-01 00:10:07 +02:00
Lasse Collin 6a2eb54092 Add LZMA_API to liblzma API headers. It's useful at least
on Windows. sysdefs.h no longer #includes lzma.h, so lzma.h
has to be #included separately where needed.
2009-01-31 11:01:48 +02:00
Lasse Collin d9993fcb4d Use _WIN32 instead of WIN32 in xzdec.c to test if compiling on Windows. 2009-01-31 10:13:09 +02:00
Lasse Collin 2dbdc5befb Fix two lines in lzma.h on which the # wasn't at the
beginning of the line.
2009-01-31 10:02:52 +02:00
Lasse Collin 4ab7601091 Add support for using liblzma headers in MSVC, which has no
stdint.h or inttypes.h.
2009-01-31 09:55:05 +02:00
Lasse Collin b2172cf823 Fix # -> ## in a macro in lzma.h. 2009-01-31 08:49:54 +02:00
Lasse Collin f54bcf6f80 Remove dangling crc64_init.c. 2009-01-30 00:29:58 +02:00
Lasse Collin 3241317093 Fix uninitialized variables in alone_decoder.c. This bug was
triggered by the previous commit, since these variables were
not used by anything before support for a preset dictionary.
2009-01-28 08:43:26 +02:00
Lasse Collin f76e39cf93 Added initial support for preset dictionary for raw LZMA1
and LZMA2. It is not supported by the .xz format or the xz
command line tool yet.
2009-01-27 18:36:05 +02:00
Lasse Collin 449b8c832b Regenerate the CRC tables without trailing blanks. 2009-01-26 20:09:17 +02:00
Jim Meyering 850f740042 remove trailing blanks from all but .xz files 2009-01-26 20:01:51 +02:00
Lasse Collin 667481f1aa Add lzma_block_buffer_decode(). 2009-01-26 14:34:10 +02:00
Lasse Collin 5fb34d8324 Add more sanity checks to lzma_stream_buffer_decode(). 2009-01-26 14:33:28 +02:00
Lasse Collin c129748675 Avoid hardcoded constant in easy.c. 2009-01-26 14:33:13 +02:00
Lasse Collin 1859d22d75 Tiny bit better sanity check in block_util.c 2009-01-26 13:06:49 +02:00
Lasse Collin 2c5fe958e4 Fix a dumb bug in Block decoder, which made it return
LZMA_DATA_ERROR with valid data. The bug was added in
e114502b2b.
2009-01-25 01:35:56 +02:00
Lasse Collin c81f13ff29 Added lzma_stream_buffer_decode() and made minor cleanups. 2009-01-23 22:27:50 +02:00
Lasse Collin 0b3318661c Fix a comment. 2009-01-22 12:53:33 +02:00
Lasse Collin 9ec80355a7 Add some single-call buffer-to-buffer coding functions. 2009-01-20 16:37:27 +02:00
Lasse Collin d8b58d0993 Block encoder cleanups 2009-01-20 13:45:41 +02:00
Lasse Collin 0c09810cb3 Use LZMA_PROG_ERROR in lzma_code() as documented in base.h. 2009-01-20 10:35:15 +02:00
Lasse Collin 2f1a8e8eb8 Fix handling of non-fatal errors in lzma_code(). 2009-01-19 22:53:18 +02:00
Lasse Collin 4810b6bc25 Move some LZMA2 constants to lzma2_encoder.h so that they
can be used outside lzma2_encoder.c.
2009-01-19 14:00:33 +02:00
Lasse Collin 00be5d2e09 Remove dead code. 2009-01-19 13:52:36 +02:00
Lasse Collin 128586213f Beta was supposed to be API stable but I had forgot to rename
lzma_memlimit_encoder and lzma_memlimit_decoder to
lzma_raw_encoder_memlimit and lzma_raw_decoder_memlimit. :-(
Now it is fixed. Hopefully it doesn't cause too much trouble
to those who already thought API is stable.
2009-01-17 14:24:25 +02:00
Lasse Collin 8286a60b8f Use pthread_sigmask() instead of sigprocmask() when pthreads
are enabled.
2009-01-07 18:41:15 +02:00
Lasse Collin 4fd43cb3a9 Bumped version to 4.999.8beta right after the release
of 4.999.7beta.
2008-12-31 20:01:00 +02:00
Lasse Collin d1d17a40d3 Prepare for 4.999.7beta release. 2008-12-31 17:41:46 +02:00
Lasse Collin 88d3e6b0b1 Cleaned up some comments in the API headers. 2008-12-31 17:15:03 +02:00
Lasse Collin 322ecf93c9 Renamed lzma_options_simple to lzma_options_bcj in the API.
The internal implementation is still using the name "simple".
It may need some cleanups, so I look at it later.
2008-12-31 16:29:39 +02:00
Lasse Collin 28e75f7086 Updated src/liblzma/Makefile.am to use liblzma.pc.in, which
should have been in the previous commit.
2008-12-31 00:48:23 +02:00
Lasse Collin 7ed9d943b3 Remove lzma_init() and other init functions from liblzma API.
Half of developers were already forgetting to use these
functions, which could have caused total breakage in some future
liblzma version or even now if --enable-small was used. Now
liblzma uses pthread_once() to do the initializations unless
it has been built with --disable-threads which make these
initializations thread-unsafe.

When --enable-small isn't used, liblzma currently gets needlessly
linked against libpthread (on systems that have it). While it is
stupid for now, liblzma will need threads in future anyway, so
this stupidity will be temporary only.

When --enable-small is used, different code CRC32 and CRC64 is
now used than without --enable-small. This made the resulting
binary slightly smaller, but the main reason was to clean it up
and to handle the lack of lzma_init_check().

The pkg-config file lzma.pc was renamed to liblzma.pc. I'm not
sure if it works correctly and portably for static linking
(Libs.private includes -pthread or other operating system
specific flags). Hopefully someone complains if it is bad.

lzma_rc_prices[] is now included as a precomputed array even
with --enable-small. It's just 128 bytes now that it uses uint8_t
instead of uint32_t. Smaller array seemed to be at least as fast
as the more bloated uint32_t array on x86; hopefully it's not bad
on other architectures.
2008-12-31 00:30:49 +02:00
Lasse Collin 050eb14d29 Revert a change made in 3b34851de1
that was related to LZMA_MODE_FAST. The original code is slightly
faster although it compresses slightly worse. But since it is fast
mode, it is better to select the faster version.
2008-12-27 19:32:20 +02:00
Lasse Collin 4820f10d0f Some xz command line tool improvements. 2008-12-27 19:30:19 +02:00
Lasse Collin e33194e79d Bunch of liblzma tweaks, including some API changes.
The API and ABI should now be very close to stable,
although the code behind it isn't yet.
2008-12-27 19:27:49 +02:00
Lasse Collin 4d00652e75 Updated Makefile.am that was missing from the previous commit. 2008-12-18 13:42:52 +02:00
Lasse Collin 634636fa56 Remove the alignment functions for now. Maybe they will
be added back in some form later, but the current version
wasn't modular, so it would need fixing anyway.
2008-12-17 21:49:53 +02:00
Lasse Collin 4fed98417d xz message handling improvements 2008-12-17 20:11:23 +02:00
Lasse Collin 653e457e37 Fix a dumb bug in .lzma decoder which was introduced in
the previous commit. (Probably the previous commit has
other bugs too, it wasn't tested.)
2008-12-15 23:26:43 +02:00
Lasse Collin 671a5adf1e Bunch of liblzma API cleanups and fixes. 2008-12-15 19:39:13 +02:00
Lasse Collin 17781c2c20 The LZMA2 decoder fix introduced a bug to LZ decoder,
which made LZ decoder return too early after dictionary
reset. This fixes it.
2008-12-15 14:26:52 +02:00
Lasse Collin ff7fb2c605 Fix data corruption in LZMA2 decoder. 2008-12-15 10:01:59 +02:00
Lasse Collin 8582d392ba Remove obsolete comment. 2008-12-10 01:31:00 +02:00
Lasse Collin b1ae6dd731 Use "decompression" consistently in --long-help. 2008-12-10 01:27:15 +02:00
Lasse Collin 1ea9e7f15a Added preset=NUM to --lzma1 and --lzma2. This makes it easy
to take a preset as a template and modify it a little.
2008-12-10 01:23:58 +02:00
Lasse Collin 6efa2d80d4 Make the memusage functions of LZMA1 and LZMA2 encoders
to validate the filter options. Add missing validation
to LZMA2 encoder when options are changed in the middle
of encoding.
2008-12-09 17:41:49 +02:00
Lasse Collin ef7890d564 In command line tool, take advantage of memusage calculation's
ability to also validate the filter chain and options (not
implemented yet for all filters).
2008-12-01 23:04:12 +02:00
Lasse Collin ccd57afa09 Validate the filter chain before checking filter-specific
memory usage.
2008-12-01 22:59:28 +02:00
Lasse Collin c596fda40b Make the memusage functions of LZMA1 and LZMA2 decoders
to validate the filter options.
2008-12-01 22:58:22 +02:00
Lasse Collin c58f469be5 Added the changes for Delta filter that should have been
part of 656ec87882.
2008-12-01 22:55:18 +02:00
Lasse Collin cd70801520 LZMA2 decoder cleanups. Make it require new LZMA properties
also in the first LZMA chunk after a dictionary reset in
uncompressed chunk.
2008-12-01 22:50:28 +02:00
Lasse Collin 656ec87882 Added lzma_delta_coder_memusage() which also validates
the options.
2008-12-01 16:30:11 +02:00
Lasse Collin c7007ddf06 Tested using COLUMNS environment variable to avoid broken
progress indicator but since COLUMNS isn't usually available,
the code was left commented out.
2008-11-28 12:00:48 +02:00
Lasse Collin ae65dcfde2 Cleanups to message.c. 2008-11-27 19:28:59 +02:00
Lasse Collin a8368b75cd Remove the nowadays unneeded memory limitting malloc() wrapper. 2008-11-25 02:37:47 +02:00
Lasse Collin 69472ee5f0 VLI encoder and decoder cleanups. Made encoder return
LZMA_PROG_ERROR in single-call mode if there's no output
space.
2008-11-23 15:09:03 +02:00
Lasse Collin 4249c8c15a Typo fix 2008-11-22 17:44:33 +02:00
Lasse Collin 6d1d6f4598 Support NetBSD's errno for O_NOFOLLOW. 2008-11-20 22:59:10 +02:00
Lasse Collin f901a290ee Build xzdec and lzmadec from xzdec.c. xzdec supports only .xz
files and lzmadec only .lzma files.
2008-11-20 18:05:52 +02:00
Lasse Collin 86a0ed8f01 Minor cleanups to xzdec. 2008-11-20 11:01:29 +02:00
Lasse Collin 1880a3927b Renamed lzma to xz and lzmadec to xzdec. We create symlinks
lzma, unlzma, and lzcat in "make install" for backwards
compatibility with LZMA Utils 4.32.x; I'm not sure if this
should be the default though.
2008-11-19 23:52:24 +02:00
Lasse Collin e114502b2b Oh well, big messy commit again. Some highlights:
- Updated to the latest, probably final file format version.
  - Command line tool reworked to not use threads anymore.
    Threading will probably go into liblzma anyway.
  - Memory usage limit is now about 30 % for uncompression
    and about 90 % for compression.
  - Progress indicator with --verbose
  - Simplified --help and full --long-help
  - Upgraded to the last LGPLv2.1+ getopt_long from gnulib.
  - Some bug fixes
2008-11-19 20:46:52 +02:00
Lasse Collin 0f295bf7a3 Fixed some help messages. 2008-10-07 16:42:18 +03:00