|
|
|
@ -4,11 +4,11 @@ History of LZMA Utils and XZ Utils |
|
|
|
|
|
|
|
Tukaani distribution |
|
|
|
|
|
|
|
In 2005, there was a small group working on Tukaani distribution, which |
|
|
|
was a Slackware fork. One of the project goals was to fit the distro on |
|
|
|
In 2005, there was a small group working on the Tukaani distribution, which |
|
|
|
was a Slackware fork. One of the project's goals was to fit the distro on |
|
|
|
a single 700 MiB ISO-9660 image. Using LZMA instead of gzip helped a |
|
|
|
lot. Roughly speaking, one could fit data that took 1000 MiB in gzipped |
|
|
|
form into 700 MiB with LZMA. Naturally compression ratio varied across |
|
|
|
form into 700 MiB with LZMA. Naturally, the compression ratio varied across |
|
|
|
packages, but this was what we got on average. |
|
|
|
|
|
|
|
Slackware packages have traditionally had .tgz as the filename suffix, |
|
|
|
@ -30,13 +30,13 @@ Tukaani distribution |
|
|
|
First steps of LZMA Utils |
|
|
|
|
|
|
|
The first version of LZMA Utils (4.22.0) included a shell script called |
|
|
|
lzmash. It was wrapper that had gzip-like command line interface. It |
|
|
|
lzmash. It was a wrapper that had a gzip-like command-line interface. It |
|
|
|
used the LZMA_Alone tool from LZMA SDK to do all the real work. zgrep, |
|
|
|
zdiff, and related scripts from gzip were adapted work with LZMA and |
|
|
|
zdiff, and related scripts from gzip were adapted to work with LZMA and |
|
|
|
were part of the first LZMA Utils release too. |
|
|
|
|
|
|
|
LZMA Utils 4.22.0 included also lzmadec, which was a small (less than |
|
|
|
10 KiB) decoder-only command line tool. It was written on top of the |
|
|
|
10 KiB) decoder-only command-line tool. It was written on top of the |
|
|
|
decoder-only C code found from the LZMA SDK. lzmadec was convenient in |
|
|
|
situations where LZMA_Alone (a few hundred KiB) would be too big. |
|
|
|
|
|
|
|
@ -48,31 +48,31 @@ Second generation |
|
|
|
The lzmash script was an ugly and not very secure hack. The last |
|
|
|
version of LZMA Utils to use lzmash was 4.27.1. |
|
|
|
|
|
|
|
LZMA Utils 4.32.0beta1 introduced a new lzma command line tool written |
|
|
|
LZMA Utils 4.32.0beta1 introduced a new lzma command-line tool written |
|
|
|
by Ville Koskinen. It was written in C++, and used the encoder and |
|
|
|
decoder from C++ LZMA SDK with little modifications. This tool replaced |
|
|
|
both the lzmash script and the LZMA_Alone command line tool in LZMA |
|
|
|
decoder from C++ LZMA SDK with some little modifications. This tool replaced |
|
|
|
both the lzmash script and the LZMA_Alone command-line tool in LZMA |
|
|
|
Utils. |
|
|
|
|
|
|
|
Introducing this new tool caused some temporary incompatibilities, |
|
|
|
because LZMA_Alone executable was simply named lzma like the new |
|
|
|
command line tool, but they had completely different command line |
|
|
|
because the LZMA_Alone executable was simply named lzma like the new |
|
|
|
command-line tool, but they had a completely different command-line |
|
|
|
interface. The file format was still the same. |
|
|
|
|
|
|
|
Lasse wrote liblzmadec, which was a small decoder-only library based |
|
|
|
on the C code found from LZMA SDK. liblzmadec had API similar to zlib, |
|
|
|
on the C code found from LZMA SDK. liblzmadec had an API similar to zlib, |
|
|
|
although there were some significant differences, which made it |
|
|
|
non-trivial to use it in some applications designed for zlib and |
|
|
|
libbzip2. |
|
|
|
|
|
|
|
The lzmadec command line tool was converted to use liblzmadec. |
|
|
|
The lzmadec command-line tool was converted to use liblzmadec. |
|
|
|
|
|
|
|
Alexandre Sauvé helped converting build system to use GNU Autotools. |
|
|
|
This made is easier to test for certain less portable features needed |
|
|
|
by the new command line tool. |
|
|
|
Alexandre Sauvé helped converting the build system to use GNU Autotools. |
|
|
|
This made it easier to test for certain less portable features needed |
|
|
|
by the new command-line tool. |
|
|
|
|
|
|
|
Since the new command line tool never got completely finished (for |
|
|
|
example, it didn't support LZMA_OPT environment variable), the intent |
|
|
|
Since the new command-line tool never got completely finished (for |
|
|
|
example, it didn't support the LZMA_OPT environment variable), the intent |
|
|
|
was to not call 4.32.x stable. Similarly, liblzmadec wasn't polished, |
|
|
|
but appeared to work well enough, so some people started using it too. |
|
|
|
|
|
|
|
@ -85,16 +85,16 @@ Second generation |
|
|
|
|
|
|
|
File format problems |
|
|
|
|
|
|
|
The file format used by LZMA_Alone was primitive. It was designed for |
|
|
|
embedded systems in mind, and thus provided only minimal set of |
|
|
|
features. The two biggest problems for non-embedded use were lack of |
|
|
|
magic bytes and integrity check. |
|
|
|
The file format used by LZMA_Alone was primitive. It was designed with |
|
|
|
embedded systems in mind, and thus provided only a minimal set of |
|
|
|
features. The two biggest problems for non-embedded use were the lack of |
|
|
|
magic bytes and an integrity check. |
|
|
|
|
|
|
|
Igor and Lasse started developing a new file format with some help |
|
|
|
from Ville Koskinen. Also Mark Adler, Mikko Pouru, H. Peter Anvin, |
|
|
|
and Lars Wirzenius helped with some minor things at some point of the |
|
|
|
development. Designing the new format took quite a long time (actually, |
|
|
|
too long time would be more appropriate expression). It was mostly |
|
|
|
too long a time would be a more appropriate expression). It was mostly |
|
|
|
because Lasse was quite slow at getting things done due to personal |
|
|
|
reasons. |
|
|
|
|
|
|
|
@ -102,7 +102,7 @@ File format problems |
|
|
|
that was already used by the old file format. Switching to the new |
|
|
|
format wouldn't have caused much trouble when the old format wasn't |
|
|
|
used by many people. But since the development of the new format took |
|
|
|
so long time, the old format got quite popular, and it was decided |
|
|
|
such a long time, the old format got quite popular, and it was decided |
|
|
|
that the new file format must use a different suffix. |
|
|
|
|
|
|
|
It was decided to use .xz as the suffix of the new file format. The |
|
|
|
@ -125,13 +125,13 @@ Transition to XZ Utils |
|
|
|
The early versions of XZ Utils were called LZMA Utils. The first |
|
|
|
releases were 4.42.0alphas. They dropped the rest of the C++ LZMA SDK. |
|
|
|
The code was still directly based on LZMA SDK but ported to C and |
|
|
|
converted from callback API to stateful API. Later, Igor Pavlov made |
|
|
|
C version of the LZMA encoder too; these ports from C++ to C were |
|
|
|
converted from a callback API to a stateful API. Later, Igor Pavlov made |
|
|
|
a C version of the LZMA encoder too; these ports from C++ to C were |
|
|
|
independent in LZMA SDK and LZMA Utils. |
|
|
|
|
|
|
|
The core of the new LZMA Utils was liblzma, a compression library with |
|
|
|
zlib-like API. liblzma supported both the old and new file format. The |
|
|
|
gzip-like lzma command line tool was rewritten to use liblzma. |
|
|
|
a zlib-like API. liblzma supported both the old and new file format. The |
|
|
|
gzip-like lzma command-line tool was rewritten to use liblzma. |
|
|
|
|
|
|
|
The new LZMA Utils code base was renamed to XZ Utils when the name |
|
|
|
of the new file format had been decided. The liblzma compression |
|
|
|
@ -139,7 +139,7 @@ Transition to XZ Utils |
|
|
|
caused unnecessary breakage in applications already using the early |
|
|
|
liblzma snapshots. |
|
|
|
|
|
|
|
The xz command line tool can emulate the gzip-like lzma tool by |
|
|
|
The xz command-line tool can emulate the gzip-like lzma tool by |
|
|
|
creating appropriate symlinks (e.g. lzma -> xz). Thus, practically |
|
|
|
all scripts using the lzma tool from LZMA Utils will work as is with |
|
|
|
XZ Utils (and will keep using the old .lzma format). Still, the .lzma |
|
|
|
|
Benno Schulenberg
12 years ago
Lasse Collin