Commit Graph

39 Commits

Author SHA1 Message Date
Lasse Collin e330fb7e6b Fix wrong indentation caused by incorrect settings
in the text editor.
2009-11-15 12:54:45 +02:00
Lasse Collin 418d64a32e Fix a design error in liblzma API.
Originally the idea was that using LZMA_FULL_FLUSH
with Stream encoder would read the filter chain
from the same array that was used to intialize the
Stream encoder. Since most apps wouldn't use
LZMA_FULL_FLUSH, most apps wouldn't need to keep
the filter chain available after initializing the
Stream encoder. However, due to my mistake, it
actually required keeping the array always available.

Since setting the new filter chain via the array
used at initialization time is not a nice way to do
it for a couple of reasons, this commit ditches it
and introduces lzma_filters_update(). This new function
replaces also the "persistent" flag used by LZMA2
(and to-be-designed Subblock filter), which was also
an ugly thing to do.

Thanks to Alexey Tourbin for reminding me about the problem
that Stream encoder used to require keeping the filter
chain allocated.
2009-11-14 18:59:19 +02:00
Lasse Collin ebfb2c5e1f Use a tuklib module for integer handling.
This replaces bswap.h and integer.h.

The tuklib module uses <byteswap.h> on GNU,
<sys/endian.h> on *BSDs and <sys/byteorder.h>
on Solaris, which may contain optimized code
like inline assembly.
2009-10-04 22:57:12 +03:00
Lasse Collin 3782b3fee4 Use unaligned access (if possible) on both endiannesses
in lz_encoder_hash.h.
2009-10-02 11:28:17 +03:00
Lasse Collin c5f68b5cc7 Make liblzma produce the same output on both endiannesses.
Seems that it is a problem in some cases if the same
version of XZ Utils produces different output on different
endiannesses, so this commit fixes that problem. The output
will still vary between different XZ Utils versions, but I
cannot avoid that for now.

This commit bloatens the code on big endian systems by 1 KiB,
which should be OK since liblzma is bloated already. ;-)
2009-10-02 11:03:26 +03:00
Lasse Collin 4ab7b16b95 A few grammar fixes.
Thanks to Christian Weisgerber for pointing out some of these.
2009-09-12 14:07:36 +03:00
Lasse Collin 18a4233a53 Fix a couple of warnings. 2009-09-11 09:25:09 +03:00
Lasse Collin 3ce1916c83 Fix data corruption in LZ/LZMA2 encoder.
Thanks to Jonathan Stott for the bug report.
2009-08-16 22:15:13 +03:00
Lasse Collin f42ee98166 Build system fixes
Don't use libtool convenience libraries to avoid recently
discovered long-standing subtle but somewhat severe bugs
in libtool (at least 1.5.22 and 2.2.6 are affected). It
was found when porting XZ Utils to Windows
<http://lists.gnu.org/archive/html/libtool/2009-06/msg00070.html>
but the problem is significant also e.g. on GNU/Linux.

Unless --disable-shared is passed to configure, static
library built from a set of convenience libraries will
contain PIC objects. That is, while libtool builds non-PIC
objects too, only PIC objects will be used from the
convenience libraries. On 32-bit x86 (tested on mobile XP2400+),
using PIC instead of non-PIC makes the decompressor 10 % slower
with the default CFLAGS.

So while xz was linked against static liblzma by default,
it got the slower PIC objects unless --disable-shared was
used. I tend develop and benchmark with --disable-shared
due to faster build time, so I hadn't noticed the problem
in benchmarks earlier.

This commit also adds support for building Windows resources
into liblzma and executables.
2009-06-30 17:09:57 +03:00
Lasse Collin 1c9360b7d1 Fix @variables@ to $(variables) in Makefile.am files.
Fix the ordering of libgnu.a and LTLIBINTL on the linker
command line and added missing LTLIBINTL to tests/Makefile.am.
2009-06-26 14:47:31 +03:00
Lasse Collin 02ddf09bc3 Put the interesting parts of XZ Utils into the public domain.
Some minor documentation cleanups were made at the same time.
2009-04-13 11:27:40 +03:00
Lasse Collin e79c42d854 Fix off-by-one in LZ decoder.
Fortunately, this bug had no security risk other than accepting
some corrupt files as valid.
2009-04-10 11:17:02 +03:00
Lasse Collin 0e27028d74 Add a separate internal function to initialize the CRC32
table, which is used also by LZ encoder. This was needed
because calling lzma_crc32() and ignoring the result is
a no-op due to lzma_attr_pure.
2009-02-08 18:24:50 +02:00
Lasse Collin 22a0c6dd94 Modify LZMA_API macro so that it works on Windows with
other compilers than MinGW. This may hurt readability
of the API headers slightly, but I don't know any
better way to do this.
2009-02-02 20:14:03 +02:00
Lasse Collin f76e39cf93 Added initial support for preset dictionary for raw LZMA1
and LZMA2. It is not supported by the .xz format or the xz
command line tool yet.
2009-01-27 18:36:05 +02:00
Lasse Collin 7ed9d943b3 Remove lzma_init() and other init functions from liblzma API.
Half of developers were already forgetting to use these
functions, which could have caused total breakage in some future
liblzma version or even now if --enable-small was used. Now
liblzma uses pthread_once() to do the initializations unless
it has been built with --disable-threads which make these
initializations thread-unsafe.

When --enable-small isn't used, liblzma currently gets needlessly
linked against libpthread (on systems that have it). While it is
stupid for now, liblzma will need threads in future anyway, so
this stupidity will be temporary only.

When --enable-small is used, different code CRC32 and CRC64 is
now used than without --enable-small. This made the resulting
binary slightly smaller, but the main reason was to clean it up
and to handle the lack of lzma_init_check().

The pkg-config file lzma.pc was renamed to liblzma.pc. I'm not
sure if it works correctly and portably for static linking
(Libs.private includes -pthread or other operating system
specific flags). Hopefully someone complains if it is bad.

lzma_rc_prices[] is now included as a precomputed array even
with --enable-small. It's just 128 bytes now that it uses uint8_t
instead of uint32_t. Smaller array seemed to be at least as fast
as the more bloated uint32_t array on x86; hopefully it's not bad
on other architectures.
2008-12-31 00:30:49 +02:00
Lasse Collin 17781c2c20 The LZMA2 decoder fix introduced a bug to LZ decoder,
which made LZ decoder return too early after dictionary
reset. This fixes it.
2008-12-15 14:26:52 +02:00
Lasse Collin ff7fb2c605 Fix data corruption in LZMA2 decoder. 2008-12-15 10:01:59 +02:00
Lasse Collin e114502b2b Oh well, big messy commit again. Some highlights:
- Updated to the latest, probably final file format version.
  - Command line tool reworked to not use threads anymore.
    Threading will probably go into liblzma anyway.
  - Memory usage limit is now about 30 % for uncompression
    and about 90 % for compression.
  - Progress indicator with --verbose
  - Simplified --help and full --long-help
  - Upgraded to the last LGPLv2.1+ getopt_long from gnulib.
  - Some bug fixes
2008-11-19 20:46:52 +02:00
Lasse Collin 1dcecfb09b Some API changes, bug fixes, cleanups etc. 2008-09-27 19:09:21 +03:00
Lasse Collin f147666a5c Miscellaneous LZ and LZMA encoder cleanups 2008-09-17 22:11:39 +03:00
Lasse Collin 13d68b0698 LZ decoder cleanup 2008-09-13 13:54:00 +03:00
Lasse Collin 13a74b78e3 Renamed constants:
- LZMA_VLI_VALUE_MAX -> LZMA_VLI_MAX
  - LZMA_VLI_VALUE_UNKNOWN -> LZMA_VLI_UNKNOWN
  - LZMA_HEADER_ERRRO -> LZMA_OPTIONS_ERROR
2008-09-13 12:10:43 +03:00
Lasse Collin 32fe5fa541 Comments 2008-09-06 23:42:50 +03:00
Lasse Collin fc68165745 Some fixes to LZ encoder. 2008-09-02 11:45:39 +03:00
Lasse Collin 3b34851de1 Sort of garbage collection commit. :-| Many things are still
broken. API has changed a lot and it will still change a
little more here and there. The command line tool doesn't
have all the required changes to reflect the API changes, so
it's easy to get "internal error" or trigger assertions.
2008-08-28 22:53:15 +03:00
Lasse Collin 7d17818cec Update the code to mostly match the new simpler file format
specification. Simplify things by removing most of the
support for known uncompressed size in most places.
There are some miscellaneous changes here and there too.

The API of liblzma has got many changes and still some
more will be done soon. While most of the code has been
updated, some things are not fixed (the command line tool
will choke with invalid filter chain, if nothing else).

Subblock filter is somewhat broken for now. It will be
updated once the encoded format of the Subblock filter
has been decided.
2008-06-18 18:02:10 +03:00
Lasse Collin 369f72fd65 Fix a buffer overflow in the LZMA encoder. It was due to my
misunderstanding of the code. There's no tiny fix for this
problem, so I also cleaned up the code in general.

This reduces the speed of the encoder 2-5 % in the fastest
compression mode ("lzma -1"). High compression modes should
have no noticeable performance difference.

This commit breaks things (especially LZMA_SYNC_FLUSH) but I
will fix them once the new format and LZMA2 has been roughly
implemented. Plain LZMA won't support LZMA_SYNC_FLUSH at all
and won't be supported in the new .lzma format. This may
change still but this is what it looks like now.

Support for known uncompressed size (that is, LZMA or LZMA2
without EOPM) is likely to go away. This means there will
be API changes.
2008-06-01 12:48:17 +03:00
Lasse Collin beeb810608 Prevent LZ encoder from hanging with known uncompressed
size. The "fix" breaks LZMA_SYNC_FLUSH at end of stream
with known uncompressed size, but since it currently seems
likely that support for encoding with known uncompressed
size will go away anyway, I'm not fixing this problem now.
2008-04-25 15:39:50 +03:00
Lasse Collin 6c5306e312 Fix wrong return type (uint32_t -> bool). 2008-04-24 18:39:57 +03:00
Lasse Collin 712cfe3ebf Fix data corruption in LZ encoder with LZMA_SYNC_FLUSH. 2008-04-24 18:38:00 +03:00
Lasse Collin f310c50286 Initialize the last byte of the dictionary to zero so that
lz_get_byte(lz, 0) returns zero. This was broken by
1a3b218598.
2008-03-11 15:17:16 +02:00
Lasse Collin 596fa1fac7 Always initialize lz->temp_size in lz_decoder.c. temp_size did
get initialized as a side-effect after allocating a new decoder,
but not when the decoder was reused.
2008-03-10 13:44:29 +02:00
Lasse Collin 1a3b218598 Don't memzero() the history buffer when initializing LZ
decoder. There's no danger of information leak here, so
it isn't required. Doing memzero() takes a lot of time
with large dictionaries, which could make it easier to
construct DoS attack to consume too much CPU time.
2008-02-02 14:51:06 +02:00
Lasse Collin ab5feaf1fc Fix LZMA_SYNC_FLUSH handling in LZ and LZMA encoders.
That code is now almost completely in LZ coder, where
it can be shared with other LZ77-based algorithms in
future.
2008-01-18 20:02:52 +02:00
Lasse Collin e22b37968d Major changes to LZ encoder, LZMA encoder, and range encoder.
These changes implement support for LZMA_SYNC_FLUSH in LZMA
encoder, and move the temporary buffer needed by range encoder
from lzma_range_encoder structure to lzma_lz_encoder.
2008-01-14 13:39:54 +02:00
Lasse Collin 9547e734a0 Don't use coder->lz.stream_end_was_reached in assertions
in match_c.h.
2008-01-14 12:09:52 +02:00
Lasse Collin f73c2ab607 Eliminate lzma_lz_encoder.must_move_pos. It's needed
only in one place which isn't performance criticial.
2008-01-10 17:13:42 +02:00
Lasse Collin 5d018dc035 Imported to git. 2007-12-09 00:42:33 +02:00