Commit Graph

199 Commits

Author SHA1 Message Date
Lasse Collin 050eb14d29 Revert a change made in 3b34851de1
that was related to LZMA_MODE_FAST. The original code is slightly
faster although it compresses slightly worse. But since it is fast
mode, it is better to select the faster version.
2008-12-27 19:32:20 +02:00
Lasse Collin e33194e79d Bunch of liblzma tweaks, including some API changes.
The API and ABI should now be very close to stable,
although the code behind it isn't yet.
2008-12-27 19:27:49 +02:00
Lasse Collin 4d00652e75 Updated Makefile.am that was missing from the previous commit. 2008-12-18 13:42:52 +02:00
Lasse Collin 634636fa56 Remove the alignment functions for now. Maybe they will
be added back in some form later, but the current version
wasn't modular, so it would need fixing anyway.
2008-12-17 21:49:53 +02:00
Lasse Collin 653e457e37 Fix a dumb bug in .lzma decoder which was introduced in
the previous commit. (Probably the previous commit has
other bugs too, it wasn't tested.)
2008-12-15 23:26:43 +02:00
Lasse Collin 671a5adf1e Bunch of liblzma API cleanups and fixes. 2008-12-15 19:39:13 +02:00
Lasse Collin 17781c2c20 The LZMA2 decoder fix introduced a bug to LZ decoder,
which made LZ decoder return too early after dictionary
reset. This fixes it.
2008-12-15 14:26:52 +02:00
Lasse Collin ff7fb2c605 Fix data corruption in LZMA2 decoder. 2008-12-15 10:01:59 +02:00
Lasse Collin 6efa2d80d4 Make the memusage functions of LZMA1 and LZMA2 encoders
to validate the filter options. Add missing validation
to LZMA2 encoder when options are changed in the middle
of encoding.
2008-12-09 17:41:49 +02:00
Lasse Collin ccd57afa09 Validate the filter chain before checking filter-specific
memory usage.
2008-12-01 22:59:28 +02:00
Lasse Collin c596fda40b Make the memusage functions of LZMA1 and LZMA2 decoders
to validate the filter options.
2008-12-01 22:58:22 +02:00
Lasse Collin c58f469be5 Added the changes for Delta filter that should have been
part of 656ec87882.
2008-12-01 22:55:18 +02:00
Lasse Collin cd70801520 LZMA2 decoder cleanups. Make it require new LZMA properties
also in the first LZMA chunk after a dictionary reset in
uncompressed chunk.
2008-12-01 22:50:28 +02:00
Lasse Collin 656ec87882 Added lzma_delta_coder_memusage() which also validates
the options.
2008-12-01 16:30:11 +02:00
Lasse Collin a8368b75cd Remove the nowadays unneeded memory limitting malloc() wrapper. 2008-11-25 02:37:47 +02:00
Lasse Collin 69472ee5f0 VLI encoder and decoder cleanups. Made encoder return
LZMA_PROG_ERROR in single-call mode if there's no output
space.
2008-11-23 15:09:03 +02:00
Lasse Collin e114502b2b Oh well, big messy commit again. Some highlights:
- Updated to the latest, probably final file format version.
  - Command line tool reworked to not use threads anymore.
    Threading will probably go into liblzma anyway.
  - Memory usage limit is now about 30 % for uncompression
    and about 90 % for compression.
  - Progress indicator with --verbose
  - Simplified --help and full --long-help
  - Upgraded to the last LGPLv2.1+ getopt_long from gnulib.
  - Some bug fixes
2008-11-19 20:46:52 +02:00
Lasse Collin 1e8e4fd1f3 Made the preset numbering more logical in liblzma API. 2008-10-07 09:40:31 +03:00
Lasse Collin 8e60c889a2 Fixed Stream decoder to actually use the first_stream variable. 2008-09-30 13:57:44 +03:00
Lasse Collin a6639022fd Fixed uninitialized variable in Stream decoder. 2008-09-30 13:34:07 +03:00
Lasse Collin ea560b0ea8 Fix conflicting Subblock helper filter's ID. 2008-09-27 23:49:24 +03:00
Lasse Collin ad97483b6e Changed magic bytes to match the updated spec. Filename
suffix wasn't changed yet.
2008-09-27 23:37:13 +03:00
Lasse Collin 1dcecfb09b Some API changes, bug fixes, cleanups etc. 2008-09-27 19:09:21 +03:00
Lasse Collin f147666a5c Miscellaneous LZ and LZMA encoder cleanups 2008-09-17 22:11:39 +03:00
Lasse Collin 13d68b0698 LZ decoder cleanup 2008-09-13 13:54:00 +03:00
Lasse Collin 13a74b78e3 Renamed constants:
- LZMA_VLI_VALUE_MAX -> LZMA_VLI_MAX
  - LZMA_VLI_VALUE_UNKNOWN -> LZMA_VLI_UNKNOWN
  - LZMA_HEADER_ERRRO -> LZMA_OPTIONS_ERROR
2008-09-13 12:10:43 +03:00
Lasse Collin 320601b2c7 Improved the Stream Flags handling API. 2008-09-12 22:41:40 +03:00
Lasse Collin 16e8b98f26 Remove a check from Block encoder that should have already
been removed in 2ba01bfa75.
2008-09-11 23:09:24 +03:00
Lasse Collin 5a710c3805 Remove bogus #includes. 2008-09-11 20:02:38 +03:00
Lasse Collin 9373e81e18 Bumped version to 4.999.6alpha. 2008-09-10 19:16:32 +03:00
Lasse Collin cb072b7c84 Check for LZMA_FILTER_RESERVED_START in filter_flags_encoder.c.
Use LZMA_PROG_ERROR instead of LZMA_HEADER_ERROR if the Filter ID
is in the reserved range. This allows Block Header encoder to
detect unallowed Filter IDs, which is good for Stream encoder.
2008-09-10 17:02:00 +03:00
Lasse Collin 123ab0acec Filter handling cleanups 2008-09-10 16:44:32 +03:00
Lasse Collin 9cfcd0c4f2 Comments 2008-09-10 00:33:00 +03:00
Lasse Collin 2ba01bfa75 Cleaned up Block encoder and moved the no longer shared
code from block_private.h to block_decoder.c. Now the Block
encoder doesn't need compressed_size and uncompressed_size
from lzma_block structure to be initialized.
2008-09-10 00:27:02 +03:00
Lasse Collin 07efcb5a6b Changed Filter ID of LZMA to 0x20. 2008-09-07 10:23:13 +03:00
Lasse Collin 32fe5fa541 Comments 2008-09-06 23:42:50 +03:00
Lasse Collin 0a31ed9d5e Some API cleanups 2008-09-06 15:14:30 +03:00
Lasse Collin 2496aee8a7 Don't allow LZMA_SYNC_FLUSH with decoders anymore. There's
simply nothing that would use it. Allow LZMA_FINISH to the
decoders, which will usually ignore it (auto decoder and
Stream decoder being exceptions).
2008-09-04 10:39:15 +03:00
Lasse Collin bab0590504 Auto decoder cleanup 2008-09-02 19:31:42 +03:00
Lasse Collin 689602336d Updated auto decoder to handle LZMA_CONCATENATED when decoding
LZMA_Alone files. Decoding of concatenated LZMA_Alone files is
intentionally not supported, so it is better to put this in
auto decoder than LZMA_Alone decoder.
2008-09-02 19:12:12 +03:00
Lasse Collin 80c4158f19 Stream decoder cleanups 2008-09-02 14:56:52 +03:00
Lasse Collin fc68165745 Some fixes to LZ encoder. 2008-09-02 11:45:39 +03:00
Lasse Collin ede675f9ac Fix wrong pointer calculation in LZMA encoder. 2008-08-31 11:47:01 +03:00
Lasse Collin 3b34851de1 Sort of garbage collection commit. :-| Many things are still
broken. API has changed a lot and it will still change a
little more here and there. The command line tool doesn't
have all the required changes to reflect the API changes, so
it's easy to get "internal error" or trigger assertions.
2008-08-28 22:53:15 +03:00
Lasse Collin eaafc4367c Remove some redundant code from LZMA encoder. 2008-06-20 16:19:54 +03:00
Lasse Collin 0809c46534 Add limit of lc + lp <= 4. Now we can allocate the
literal coder as part of the main LZMA encoder or
decoder structure.

Make the LZMA decoder to rely on the current internal API
to free the allocated memory in case an error occurs.
2008-06-19 16:35:08 +03:00
Lasse Collin d25ab1b961 Comments 2008-06-18 21:45:19 +03:00
Lasse Collin 7d17818cec Update the code to mostly match the new simpler file format
specification. Simplify things by removing most of the
support for known uncompressed size in most places.
There are some miscellaneous changes here and there too.

The API of liblzma has got many changes and still some
more will be done soon. While most of the code has been
updated, some things are not fixed (the command line tool
will choke with invalid filter chain, if nothing else).

Subblock filter is somewhat broken for now. It will be
updated once the encoded format of the Subblock filter
has been decided.
2008-06-18 18:02:10 +03:00
Lasse Collin 803194ddd2 Fix uninitialized variable in LZMA encoder. This was
introduced in 369f72fd65.
2008-06-11 21:42:47 +03:00
Lasse Collin 369f72fd65 Fix a buffer overflow in the LZMA encoder. It was due to my
misunderstanding of the code. There's no tiny fix for this
problem, so I also cleaned up the code in general.

This reduces the speed of the encoder 2-5 % in the fastest
compression mode ("lzma -1"). High compression modes should
have no noticeable performance difference.

This commit breaks things (especially LZMA_SYNC_FLUSH) but I
will fix them once the new format and LZMA2 has been roughly
implemented. Plain LZMA won't support LZMA_SYNC_FLUSH at all
and won't be supported in the new .lzma format. This may
change still but this is what it looks like now.

Support for known uncompressed size (that is, LZMA or LZMA2
without EOPM) is likely to go away. This means there will
be API changes.
2008-06-01 12:48:17 +03:00
Lasse Collin ed6664146f Remove support for pre-C89 libc versions that lack memcpy,
memmove, and memset.
2008-05-11 14:24:42 +03:00
Lasse Collin 11de5d5267 Bunch of grammar fixes from meyering. 2008-05-06 15:15:07 +03:00
Lasse Collin dc192b6343 Typo fix 2008-05-06 13:41:05 +03:00
Lasse Collin 3be21fb12f Fixed wrong spelling "limitter" to "limiter". This affects
liblzma's API.
2008-04-28 17:06:34 +03:00
Lasse Collin beeb810608 Prevent LZ encoder from hanging with known uncompressed
size. The "fix" breaks LZMA_SYNC_FLUSH at end of stream
with known uncompressed size, but since it currently seems
likely that support for encoding with known uncompressed
size will go away anyway, I'm not fixing this problem now.
2008-04-25 15:39:50 +03:00
Lasse Collin c324325f9f Removed src/liblzma/common/sysdefs.h symlink, which was
annoying, because "make dist" put two copies of sysdefs.h
into the tarball instead of the symlink.
2008-04-25 13:58:56 +03:00
Lasse Collin 8f804c29aa Bumped version number to 4.999.3alpha. It will become 5.0.0
once we have a stable release (won't be very soon). The
version number is no longer related to version of LZMA SDK.

Made some small Automake-related changes to toplevel
Makefile.am and configure.ac.
2008-04-25 13:32:35 +03:00
Lasse Collin c99037ea10 Fix a memory leak by calling free(extra->data) in
lzma_extra_free().
2008-04-24 20:25:39 +03:00
Lasse Collin 283f939974 Added two assert()s. 2008-04-24 20:19:20 +03:00
Lasse Collin eb348a60b6 Switch to uint16_t as the type of range coder probabilities. 2008-04-24 19:22:53 +03:00
Lasse Collin 6c5306e312 Fix wrong return type (uint32_t -> bool). 2008-04-24 18:39:57 +03:00
Lasse Collin 712cfe3ebf Fix data corruption in LZ encoder with LZMA_SYNC_FLUSH. 2008-04-24 18:38:00 +03:00
Lasse Collin bc04486e36 Fix fastpos problem in Makefile.am when built with --enable-small. 2008-04-24 17:33:01 +03:00
Lasse Collin 7ab493924e Use 64-bit integer as range encoder's cache size. This fixes a
theoretical data corruption, which should be very hard to trigger
even intentionally.
2008-04-24 17:30:51 +03:00
Lasse Collin 641998c3e1 Replaced the range decoder optimization that used arithmetic
right shift with as fast version that doesn't need
arithmetic right shift. Removed the related check from
configure.ac.
2008-03-24 16:38:40 +02:00
Lasse Collin ad999efd27 Take advantage of arithmetic right shift in range decoder. 2008-03-22 14:39:34 +02:00
Lasse Collin 7521bbdc83 Update a comment to use the variable name rep_len_decoder.
(And BTW, the previous commit actually did change the
program logic slightly.)
2008-03-22 01:26:36 +02:00
Lasse Collin 63b74d000e Demystified the "state" variable in LZMA code. Use the
word literal instead of char for better consistency.
There are still some names with _char instead of _literal
in lzma_optimum, these may be changed later.

Renamed length coder variables.

This commit doesn't change the program logic.
2008-03-22 00:57:33 +02:00
Lasse Collin e6eb0a2675 Fix data corruption in LZMA encoder. Note that this bug was
specific to liblzma and was *not* present in LZMA SDK.
2008-03-14 23:16:11 +02:00
Lasse Collin 7d516f5129 Fix a comment API header. 2008-03-14 21:32:37 +02:00
Lasse Collin 748d6e4274 Make lzma_stream.next_in const. Let's see if anyone complains. 2008-03-12 23:14:50 +02:00
Lasse Collin bfde3b24a5 Apply a minor speed optimization to LZMA decoder. 2008-03-11 15:35:34 +02:00
Lasse Collin f310c50286 Initialize the last byte of the dictionary to zero so that
lz_get_byte(lz, 0) returns zero. This was broken by
1a3b218598.
2008-03-11 15:17:16 +02:00
Lasse Collin 5ead36cf7f Really fix the price count initialization. 2008-03-10 15:57:55 +02:00
Lasse Collin 0541c5ea63 Initialize align_price_count and match_price_count in
lzma_encoder_init.c. While we don't call
fill_distances_prices() and fill_align_prices() in
lzma_lzma_encoder_init(), we still need to initialize
these two variables so that the fill functions get
called in lzma_encoder_getoptimum.c in the beginning
of a stream.
2008-03-10 13:46:48 +02:00
Lasse Collin 596fa1fac7 Always initialize lz->temp_size in lz_decoder.c. temp_size did
get initialized as a side-effect after allocating a new decoder,
but not when the decoder was reused.
2008-03-10 13:44:29 +02:00
Lasse Collin 45e43e1695 Don't fill allocated memory with 0xFD when debugging is
enabled. It hides errors from Valgrind.
2008-03-10 13:41:25 +02:00
Lasse Collin c0e19e0662 Remove two redundant validity checks from the LZMA decoder.
These are already checked elsewhere, so omitting these
gives (very) tiny speed up.
2008-02-28 10:24:31 +02:00
Lasse Collin 1a3b218598 Don't memzero() the history buffer when initializing LZ
decoder. There's no danger of information leak here, so
it isn't required. Doing memzero() takes a lot of time
with large dictionaries, which could make it easier to
construct DoS attack to consume too much CPU time.
2008-02-02 14:51:06 +02:00
Lasse Collin 7e796e312b Do uncompressed size validation in raw encoder. This way
it gets done for not only raw encoder, but also Block
and LZMA_Alone encoders.
2008-02-01 08:39:26 +02:00
Lasse Collin 7dd48578a3 Avoid unneeded function call in raw_common.c. 2008-02-01 08:32:05 +02:00
Lasse Collin e9f6e9c075 Added note.GNU-stack to x86 assembler files. It is needed
when using non-executable stack.
2008-01-26 21:40:23 +02:00
Lasse Collin 4c7ad179c7 Added api/lzma/easy.h. I had forgot to add this to the
git repo. Thanks to Stephan Kulow.
2008-01-26 19:12:50 +02:00
Lasse Collin f9842f7127 Return LZMA_HEADER_ERROR if LZMA_SYNC_FLUSH is used with any
of the so called simple filters. If there is demand, limited
support for LZMA_SYNC_FLUSH may be added in future.

After this commit, using LZMA_SYNC_FLUSH shouldn't cause
undefined behavior in any situation.
2008-01-26 00:25:34 +02:00
Lasse Collin 4441e00418 Combine lzma_options_block validation needed by both Block
encoder and decoder, and put the shared things to
block_private.h. Improved the checks a little so that
they may detect too big Compressed Size at initialization
time if lzma_options_block.total_size or .total_limit is
known.

Allow encoding and decoding Blocks with combinations of
fields that are not allowed by the file format specification.
Doing this requires that the application passes such a
combination in lzma_options_lzma; liblzma doesn't do that,
but it's not impossible that someone could find them useful
in some custom file format.
2008-01-25 23:12:36 +02:00
Lasse Collin 7b8fc7e6b5 Improved the memory limitter:
- Added lzma_memlimit_max() and lzma_memlimit_reached()
    API functions.
  - Added simple estimation of malloc()'s memory usage
    overhead.
  - Fixed integer overflow detection in lzma_memlimit_alloc().
  - Made some white space cleanups and added more comments.

The description of lzma_memlimit_max() in memlimit.h is bad
and should be improved.
2008-01-25 19:20:28 +02:00
Lasse Collin db9df0a960 Fix decoding of empty Metadata Blocks, that don't have
even the Metadata Flags field. Earlier the code allowed
such files; now they are prohibited as the file format
specification requires.
2008-01-23 23:43:00 +02:00
Lasse Collin 765f0b05f6 Fix a bug related to 99e12af4e2.
lzma_metadata.header_metadata_size was not properly set to
zero if the Metadata had only the Metadata Flags field.
2008-01-23 23:38:18 +02:00
Lasse Collin 3a7cc5c3de Fix decoding of Extra Records that have empty Data. 2008-01-23 23:35:49 +02:00
Lasse Collin e5fdec93e2 Add the trailing '\0' to lzma_extra.data as the API header
already documents.
2008-01-23 22:02:38 +02:00
Lasse Collin ae0cd09a66 Return LZMA_STREAM_END instead of LZMA_OK if
LZMA_SYNC_FLUSH or LZMA_FULL_FLUSH is used when
there's no unfinished Block open.
2008-01-23 21:05:33 +02:00
Lasse Collin 8c8eb14055 Fixed a typo. 2008-01-23 13:42:35 +02:00
Lasse Collin 980f65a9a1 Fix a memory leak in the Subblock encoder. 2008-01-23 13:40:45 +02:00
Lasse Collin 99e12af4e2 Fix Size of Header Metadata Block handling. Now
lzma_metadata.header_metadata_size == LZMA_VLI_VALUE_UNKNOWN
is not allowed at all. To indicate missing Header Metadata
Block, header_metadata_size must be set to zero. This is
what Metadata decoder does after this patch too.

Note that other missing fields in lzma_metadata are still
indicated with LZMA_VLI_VALUE_UNKNOWN. This isn't as
illogical as it sounds at first, because missing Size of
Header Metadata Block means that Header Metadata Block is
not present in the Stream. With other Metadata fields,
a missing field means only that the value is unknown.
2008-01-23 13:36:07 +02:00
Lasse Collin 58b78ab20c Fix a memory leak in metadata_decoder.c. 2008-01-23 13:15:55 +02:00
Lasse Collin 4d8cdbdab4 Fix the fix 863028cb7a which
just moved to problem. Now it's really fixed.
2008-01-23 13:13:58 +02:00
Lasse Collin 67321de963 Take advantage of return_if_error() macro in
lzma_info_metadata_set() in info.c.
2008-01-23 00:21:04 +02:00
Lasse Collin 863028cb7a Fixed a dangling pointer that caused invalid free(). 2008-01-23 00:18:32 +02:00
Lasse Collin cf49f42a6b Added lzma_easy_* functions. These should make using
liblzma as easy as using zlib, because the easy API
don't require developers to know any fancy LZMA options.

Note that Multi-Block Stream encoding is currently broken.
The easy API should be OK, the bug(s) are elsewhere.
2008-01-22 22:49:24 +02:00
Lasse Collin 1747b85a43 Fix Multi-Block Stream encoder's EOPM usage. 2008-01-22 21:16:22 +02:00