From 8d38941baed05de4ff7cc775de14833108f62184 Mon Sep 17 00:00:00 2001 From: Lasse Collin Date: Sun, 18 Feb 2024 17:17:43 +0200 Subject: [PATCH] Windows: Major update to Windows build instructions. --- INSTALL | 54 +++-- windows/INSTALL-MSVC.txt | 23 ++- windows/INSTALL-MinGW-w64_with_Autotools.txt | 49 +++++ windows/INSTALL-MinGW-w64_with_CMake.txt | 203 +++++++++++++++++++ windows/INSTALL-MinGW.txt | 138 ------------- windows/README-Windows.txt | 2 + windows/build-with-cmake.bat | 35 ++++ windows/liblzma-crt-mixing.txt | 70 +++++++ 8 files changed, 397 insertions(+), 177 deletions(-) create mode 100644 windows/INSTALL-MinGW-w64_with_Autotools.txt create mode 100644 windows/INSTALL-MinGW-w64_with_CMake.txt delete mode 100644 windows/INSTALL-MinGW.txt create mode 100644 windows/build-with-cmake.bat create mode 100644 windows/liblzma-crt-mixing.txt diff --git a/INSTALL b/INSTALL index 670e3989..004fc87d 100644 --- a/INSTALL +++ b/INSTALL @@ -143,43 +143,37 @@ XZ Utils Installation 1.2.7. Windows - If it is enough to build liblzma (no command line tools): + The "windows" directory contains instructions for a few types + of builds: - - There is CMake support. It should be good enough to build - static liblzma or liblzma.dll with Visual Studio. The CMake - support may work with MinGW or MinGW-w64. Read the comment - in the beginning of CMakeLists.txt before running CMake! + - INSTALL-MinGW-w64_with_CMake.txt + Simple instructions how to build XZ Utils natively on + Windows using only CMake and a prebuilt toolchain + (GCC + MinGW-w64 or Clang/LLVM + MinGW-w64). - - There are Visual Studio project files under the "windows" - directory. See windows/INSTALL-MSVC.txt. In the future the - project files will be removed when CMake support is good - enough. Thus, please test the CMake version and help fix - possible issues. + - INSTALL-MinGW-w64_with_Autotools.txt + Native build under MSYS2 or cross-compilation from + GNU/Linux using a bash script that creates a .zip + and .7z archives of the binaries and documentation. + The related file README-Windows.txt is for the + resulting binary package. - To build also the command line tools: + - INSTALL-MSVC.txt + Building with MSVC / Visual Studio and CMake. - - MinGW-w64 + MSYS (32-bit and 64-bit x86): This is used - for building the official binary packages for Windows. - There is windows/build.bash to ease packaging XZ Utils with - MinGW(-w64) + MSYS into a redistributable .zip or .7z file. - See windows/INSTALL-MinGW.txt for more information. + - liblzma-crt-mixing.txt + Documentation what to take into account as a programmer + if liblzma.dll and the application don't use the same + CRT (MSVCRT or UCRT). - - MinGW + MSYS (32-bit x86): I haven't recently tested this. + Other choices: - - Cygwin 1.7.35 and later: NOTE that using XZ Utils >= 5.2.0 - under Cygwin older than 1.7.35 can lead to DATA LOSS! If - you must use an old Cygwin version, stick to XZ Utils 5.0.x - which is safe under older Cygwin versions. You can check - the Cygwin version with the command "cygcheck -V". + - Cygwin: https://cygwin.com/ + Building on Cygwin can be done like on many POSIX operating + systems. XZ Utils >= 5.2.0 isn't compatible with Cygwin older + than 1.7.35 (data loss!). 1.7.35 was released on 2015-03-04. - It may be possible to build liblzma with other toolchains too, but - that will probably require writing a separate makefile. Building - the command line tools with non-GNU toolchains will be harder than - building only liblzma. - - Even if liblzma is built with MinGW(-w64), the resulting DLL can - be used by other compilers and linkers, including MSVC. See - windows/README-Windows.txt for details. + - MSYS2: https://www.msys2.org/ 1.2.8. DOS diff --git a/windows/INSTALL-MSVC.txt b/windows/INSTALL-MSVC.txt index d9574388..ba483fb6 100644 --- a/windows/INSTALL-MSVC.txt +++ b/windows/INSTALL-MSVC.txt @@ -5,11 +5,15 @@ Building XZ Utils with Microsoft Visual Studio Introduction ------------ - liblzma got MSVC support in XZ Utils 5.2.0, and the xz and xzdec - command line tools in XZ Utils 5.6.0. + liblzma got MSVC support in XZ Utils 5.2.0, and the xz, xzdec, + lzmadec, and lzmainfo command line tools in XZ Utils 5.6.0. - Note: xz and xzdec depend on the the included GNU getopt - which is licensed under the GNU LGPLv2.1. + NOTE: The *.exe files are linked against GNU getopt_long from + the "lib" directory. That code is under the GNU LGPLv2.1 + and thus the matching source code must be provided when + distributing the *.exe files. The simplest way to comply with + the license requirements is to distribute the matching XZ Utils + source package alongside the *.exe files. MSVC 2013 update 2 and later have enough C99 support to build liblzma from XZ Utils 5.2.0 and later without modifications. @@ -21,12 +25,13 @@ Introduction Building -------- - It is recommended to use CMake to generate build files for MSVC. - Visual Studio project files are no longer provided (XZ Utils 5.4.x - were the last versions to include the project files). + Use CMake to generate build files for MSVC. Visual Studio project + files are no longer provided (XZ Utils 5.4.x were the last versions + to include the project files). - CMake-based build includes tests for liblzma but not for the - command line tools. + NOTE: GCC and Clang compatible inline assembly isn't supported by + the MSVC compiler. Using clang-cl under MSVC should make inline + assembly work (untested). Notes diff --git a/windows/INSTALL-MinGW-w64_with_Autotools.txt b/windows/INSTALL-MinGW-w64_with_Autotools.txt new file mode 100644 index 00000000..89c74409 --- /dev/null +++ b/windows/INSTALL-MinGW-w64_with_Autotools.txt @@ -0,0 +1,49 @@ + +Creating XZ Utils Windows package with build.bash +================================================= + +Introduction +------------ + + The script build.bash can be used for building XZ Utils with + GCC + MinGW-w64 under MSYS2, under the ancient MSYS, or + cross-compiling from GNU/Linux. The script will create a package + with binaries and documentation in a hopefully-convenient bundle. + + NOTE: build.bash requires files that are only included + in release tarballs. If building from xz.git, a distribution + tarball should be created first. + + For native builds on Windows, the CMake-based build described + in the file INSTALL-MinGW-w64_with_CMake.txt is simpler to do as + it has no need for MSYS2 and it works from xz.git without extra + steps. For cross-compilation and package creation the script can + be convenient though. + + These instructions are for making a package with build.bash and thus + don't apply to normal Autotool-based builds under Cygwin or MSYS2. + + +Usage +----- + + First copy the file COPYING.MinGW-w64-runtime.txt from MinGW-w64 + to this directory. It contains copyright and license notices that + apply to the MinGW-w64 runtime that gets statically linked into + the XZ Utils binaries being built. build.bash will include the file + in the final package. + + Put i686 and/or x86_64 GCC-based toolchain in PATH depending on + which builds are wanted. + + Optional: Put the 7z tool from 7-Zip or p7zip in PATH. Without + this, .zip and .7z files won't be created from the finished "pkg" + directory contents. + + Run build.bash: + + bash windows/build.bash + + Note that it does an in-tree build so the build files will be mixed + with the source files in the same directory tree. + diff --git a/windows/INSTALL-MinGW-w64_with_CMake.txt b/windows/INSTALL-MinGW-w64_with_CMake.txt new file mode 100644 index 00000000..5d2be5ba --- /dev/null +++ b/windows/INSTALL-MinGW-w64_with_CMake.txt @@ -0,0 +1,203 @@ + +Building XZ Utils on Windows using MinGW-w64 and CMake +====================================================== + + 1. Introduction + 1.1. Licensing considerations + 2. MSVCRT or UCRT + 3. CMake + 4. MinGW-w64 toolchains + 4.1. MinGW-w64 with GCC + 4.2. MinGW-w64 with Clang/LLVM + 5. Building XZ Utils + 5.1. Advanced build options + 6. Creating an import library for MSVC / Visual Studio + + +1. Introduction +--------------- + + This document explains how to build XZ Utils using MinGW-w64, + GCC or Clang/LLVM, CMake, and GNU make (mingw32-make) natively + on Windows. The resulting XZ Utils library and executable files + will only depend on DLLs that are included in Windows. + + The build tools can be extracted into separate directories and used + directly from there and deleted when no longer needed. There are no + installers to run for these and no configuration needed. + + These instructions don't apply to Cygwin. XZ Utils can be built + under Cygwin in the same way as many other packages. + + +1.1. Licensing considerations + + Parts of MinGW-w64 runtime are statically linked into the binaries + being built. The file COPYING.MinGW-w64-runtime.txt in MinGW-w64 + contains the license notices that apply to some parts of the + runtime. The notices must be distributed alongside the binaries + that have been built with MinGW-w64. + + MinGW-w64 includes getopt_long(). The GNU getopt_long() (LGPLv2.1) + included in XZ Utils isn't used when building with MinGW-w64. + + The code from XZ Utils that ends up liblzma.dll and the *.exe files + is under the BSD Zero Clause License (0BSD) which doesn't require + any copyright or license notices to be included when distributing + the binaries. See the file COPYING in the parent directory. + + +2. MSVCRT or UCRT +----------------- + + Both GCC and Clang/LLVM based MinGW-w64 toolchains come in MSVCRT + and Universal C runtime (UCRT) variants. MSVCRT is the old one. + 32-bit builds of XZ Utils with MSVCRT should run on Windows 2000 + and later (even Windows 95 should still be possible with trivial + edits to the source code). + + UCRT is included in Windows 10, and it's possible to install UCRT + on Windows XP and later. UCRT might be the preferred choice if + out-of-the-box compatibility with Windows versions older than 10 + is not required. Visual Studio 2015 and later produce binaries + that use UCRT. + + If you want to build liblzma.dll for use with your application, + it's recommended to use the same CRT for all components. If this + isn't possible, see the file liblzma-crt-mixing.txt. + + If you only need the command line tools, the choice of CRT isn't + important, at least for now. + + +3. CMake +-------- + + CMake is used for selecting build options and generating makefiles. + It can also be used to extract archives, including .tar.xz and .7z. + + Download a CMake binary package (.zip) from its homepage: + + https://cmake.org/download/ + + Extract it to, for example, C:\devel\cmake so that the executables + end up in C:\devel\cmake\bin. Avoid spaces and other special + characters in the path. + + +4. MinGW-w64 toolchains +----------------------- + + There are a few choices of prebuilt toolchains listed on + the MinGW-w64 homepage: + + https://www.mingw-w64.org/downloads/ + + These instructions list one GCC-based version and one + Clang/LLVM-based version. Both include mingw32-make too. + + +4.1. MinGW-w64 with GCC + + For GCC, download appropriate packages from Mingw-builds depending + on if you want to build 32-bit or 64-bit x86 version of XZ Utils + and if the XZ Utils binaries should link against MSVCRT or UCRT: + + https://github.com/niXman/mingw-builds-binaries/releases + + i686-*-release-win32-*-msvcrt-*.7z 32-bit, uses MSVCRT (old) + i686-*-release-win32-*-ucrt-*.7z 32-bit, uses UCRT (new) + x86_64-*-release-win32-*-msvcrt-*.7z 64-bit, uses MSVCRT (old) + x86_64-*-release-win32-*-ucrt-*.7z 64-bit, uses UCRT (new) + + Extract it, for example, to C:\devel so that the executables are + in C:\devel\mingw32\bin or C:\devel\mingw64\bin. To extract, + you can install 7-Zip from or use CMake + on the command line: + + set PATH=C:\devel\cmake\bin;%PATH% + c: + cd \devel + cmake -E tar xf x86_64-13.1.0-release-win32-seh-ucrt-rt_v11-rev1.7z + + Then skip to the section "Building XZ Utils". + + +4.2. MinGW-w64 with Clang/LLVM + + For Clang/LLVM, download an appropriate package from LLVM-MinGW: + + https://github.com/mstorsjo/llvm-mingw/releases + + llvm-mingw-*-msvcrt-i686.zip 32-bit, uses MSVCRT (old) + llvm-mingw-*-ucrt-i686.zip 32-bit, uses UCRT (new) + llvm-mingw-*-msvcrt-x86_64.zip 64-bit, uses MSVCRT (old) + llvm-mingw-*-ucrt-x86_64.zip 64-bit, uses UCRT (new) + + Extract it, for example, to C:\devel so that the executables end up + in a directory like C:\devel\llvm-mingw-20230919-ucrt-x86_64\bin. + + +5. Building XZ Utils +-------------------- + + For a simple builds, you can use the included build-with-cmake.bat + which takes these arguments: + + %1 = Path to CMake's bin directory. Example: + c:\devel\cmake\bin + + %2 = Path to MinGW-w64's bin directory. Example: + c:\devel\mingw64\bin + + %3 = ON or OFF: Set to ON to build liblzma.dll or OFF for + static liblzma.a. With OFF, the *.exe files won't + depend on liblzma.dll. + + Example: + + build-with-cmake C:\devel\cmake\bin C:\devel\mingw64\bin ON + + If successful, the "build" directory should then contain: + + liblzma.dll liblzma compression library + liblzma.def DEF file for creating an import library + xz.exe xz command line tool + xzdec.exe Decompression-only tool (smaller than xz.exe) + lzmadec.exe Decompression-only tool for legacy .lzma files + lzmainfo.exe Shows header info of legacy .lzma files + + Ignore the other files. :-) + + +5.1. Advanced build options + + For 32-bit x86 builds, adding -msse2 to CFLAGS improves + compression speed a little (but not decompression speed). + There is no runtime detection for SSE2 support. It is + recommended to use 64-bit version when possible. + + It's possible to omit features from the build to reduce code size. + There are several CMake configuration options available. One may + change from CMAKE_BUILD_TYPE=Release to =MinSizeRel as well but + it makes the code slower. + + If building for multiple targets, keep only one toolchain in PATH + at a time. + + +6. Creating an import library for MSVC / Visual Studio +------------------------------------------------------ + + To link against liblzma.dll, you need to create an import library + first. You need the "lib" command from MSVC and liblzma.def. Here + is the command that works on 32-bit x86: + + lib /def:liblzma.def /out:liblzma.lib /machine:ix86 + + On x86-64, the /machine argument has to be changed: + + lib /def:liblzma.def /out:liblzma.lib /machine:x64 + + IMPORTANT: See also the file liblzma-crt-mixing.txt. + diff --git a/windows/INSTALL-MinGW.txt b/windows/INSTALL-MinGW.txt deleted file mode 100644 index 72e2718f..00000000 --- a/windows/INSTALL-MinGW.txt +++ /dev/null @@ -1,138 +0,0 @@ - -Building XZ Utils on Windows -============================ - -Introduction ------------- - - This document explains shortly where to get and how to install the - build tools that are needed to build XZ Utils on Windows. The final - binary package will be standalone in sense that it will depend only - on DLLs that are included in all Windows installations. - - These instructions don't apply to Cygwin. XZ Utils can be built under - Cygwin in the same way as many other packages. - - These instructions don't apply to MinGW and MSYS developers either, - who may want to package XZ Utils for MinGW or MSYS distributions. - You know who you are, and will probably use quite different configure - options etc. than what is described here. - - -Installing the toolchain(s) ---------------------------- - - Some of the following is needed: - - MSYS is always needed to use the GNU Autotools based build system. - - MinGW builds 32-bit x86 binaries. - - 32-bit MinGW-w64 (I call it MingW-w32 here) builds 32-bit x86 - executables too. - - MinGW-w64 builds 64-bit x86-64 binaries. - - So you need to pick between MinGW and MinGW-w32 when building - 32-bit version. You don't need both. - - You might find 7-Zip handy when extracting - some files. The ready-made build script build.bash will also use - 7-Zip to create the distributable .zip and .7z files. - - I used the following directory structure but you can use whatever - you want. Just note that I will use these in my examples. Each of - these should have a subdirectory "bin": - - C:\devel\tools\msys - C:\devel\tools\mingw - C:\devel\tools\mingw-w32 - C:\devel\tools\mingw-w64 - - -Installing MSYS - - You can download MSYS from MinGW's Sourceforge page: - - https://sourceforge.net/projects/mingw/files/MSYS/Base/msys-core/ - - I recommend using MSYS 1.0.11 (MSYS-1.0.11.exe or - msysCORE-1.0.11-bin.tar.gz) because that package includes all the - required tools. At least some of the later versions include only - a subset and thus you would need to download the rest separately. - The old version will work fine for building XZ Utils. - - You can use either the .exe or .tar.gz package. I prefer .tar.gz, - because it can be extracted into any directory and later removed - without worrying about uninstallers. - - -Installing MinGW - - NOTE: This section may be outdated. I haven't tried MinGW recently. - - You can download the required packages from MinGW's Sourceforge page: - - https://sourceforge.net/projects/mingw/files/ - - These version numbers were the latest when I wrote this document, but - you probably should pick the latest versions: - - MinGW Runtime -> mingwrt-3.17-mingw32-dev.tar.gz - MinGW API for MS-Windows -> w32api-3.14-mingw32-dev.tar.gz - GNU Binutils -> binutils-2.20-1-bin.tar.gz - GCC Version 4 -> gcc-full-4.4.0-mingw32-bin-2.tar.lzma - - The full GCC package is quite big, but if you want a smaller - download, you will need to download more than one file, so I'm - using the full package in this document for simplicity. - - Extract the packages in the above order, possibly overwriting files - from packages that were extracted earlier. - - -Installing MinGW-w32 or MinGW-w64 - - I used the packages from Mingw-builds project. With that it is - enough to pick one .7z file for 32-bit and another for 64-bit - toolchain. For XZ Utils 5.2.0 I used the packages from these - directories: - - https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/4.9.2/threads-win32/sjlj/ - - https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.9.2/threads-win32/sjlj/ - - If you install both MinGW-w32 and MinGW-w64, remember to extract - them into different directories. build.bash looks at - C:\devel\tools\mingw-w32 and C:\devel\tools\mingw-w64 by default. - - -Building XZ Utils ------------------ - - Start MSYS by going to the directory C:\devel\tools\msys and running - msys.bat there (double-click or use command prompt). It will start - at "home" directory, which is C:\devel\tools\msys\home\YourUserName. - - If you have xz-5.x.x.tar.gz in C:\devel, you should be able to build - it now with the following commands: - - cd /c/devel - tar xzf xz-5.x.x.tar.gz - cd xz-5.x.x - bash windows/build.bash - - If you used some other directory than C:\devel\tools for the build - tools, edit the variables near the beginning of build.bash first. - - If you want to build manually, read the buildit() function in - build.bash. Look especially at the latter configure invocation. - - Be patient. Running configure and other scripts used by the build - system is (very) slow under Windows. - - -Using a snapshot from the Git repository - - To use a snapshot, the build system files need to be generated with - autogen.sh or "autoreconf -fi" before trying to build using the - above build instructions. You can install the relevant extra packages - from MinGW or use Cygwin or use e.g. a GNU/Linux system to create a - source package with the required build system files. - diff --git a/windows/README-Windows.txt b/windows/README-Windows.txt index 65dc9db1..c7820f8c 100644 --- a/windows/README-Windows.txt +++ b/windows/README-Windows.txt @@ -92,6 +92,8 @@ Creating an import library for MSVC / Visual Studio lib /def:liblzma.def /out:liblzma.lib /machine:x64 + IMPORTANT: See also the file liblzma-crt-mixing.txt. + Reporting bugs -------------- diff --git a/windows/build-with-cmake.bat b/windows/build-with-cmake.bat new file mode 100644 index 00000000..24978ba8 --- /dev/null +++ b/windows/build-with-cmake.bat @@ -0,0 +1,35 @@ +@rem # SPDX-License-Identifier: 0BSD +@rem # Author: Lasse Collin +@rem # +@rem ######################################################################## +@rem # +@rem # This builds XZ Utils with CMake + MinGW-w64 (GCC or Clang/LLVM). +@rem # See INSTALL-MinGW-w64_with_CMake.txt for detailed instructions. +@rem # +@rem # Summary of command line arguments: +@rem # +@rem # %1 = Path to CMake's bin directory. Example: +@rem # C:\devel\cmake\bin +@rem # +@rem # %2 = Path to MinGW-w64's bin directory. Example: +@rem # C:\devel\mingw64\bin +@rem # +@rem # %3 = ON or OFF: Set to ON to build liblzma.dll or OFF for +@rem # static liblzma.a. With OFF, the *.exe files won't +@rem # depend on liblzma.dll. +@rem # +@rem ######################################################################## + +setlocal +set PATH=%1;%2;%PATH% + +md build || exit /b +cd build || exit /b + +cmake -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release -DENABLE_NLS=OFF -DBUILD_SHARED_LIBS=%3 ..\.. || exit /b +mingw32-make || exit /b +mingw32-make test || exit /b + +@rem liblzma.dll might not exist so ignore errors. +strip xz.exe xzdec.exe lzmadec.exe lzmainfo.exe liblzma.dll +exit /b 0 diff --git a/windows/liblzma-crt-mixing.txt b/windows/liblzma-crt-mixing.txt new file mode 100644 index 00000000..4ef2f8ab --- /dev/null +++ b/windows/liblzma-crt-mixing.txt @@ -0,0 +1,70 @@ + +liblzma.dll and mixing C runtimes (CRTs) +---------------------------------------- + + If possible, liblzma.dll should be linked against the same CRT + (MSVCRT or UCRT) as the application calling the liblzma functions. + When this isn't possible, liblzma.dll will still work but there + are a few API functions that need extra care. + + Each CRT has its own memory allocator, stdio FILE implementation, + mapping of file descriptors from _open() to Windows' HANDLEs, and + so on. Mixing CRTs is a problem if, for example, one library calls + fopen() and then passes the resulting FILE* to a second library and + these two libraries use different CRTs. liblzma doesn't expose FILE + pointers or file descriptors in the API but the problem can occur + with memory allocation with a few specific functions. + + The most commonly-used API functions in liblzma are such that both + memory allocation and deallocation is done internally by liblzma, + thus most applications won't have any problems with mixing CRTs + with liblzma.dll. The following API functions are the exception: + + lzma/block.h: + lzma_block_header_decode + + lzma/filter.h: + lzma_filters_copy + lzma_filters_free + lzma_properties_decode + lzma_filter_flags_decode + lzma_str_to_filters + lzma_str_from_filters + lzma_str_list_filters + + Excluding lzma_filters_free(), the above functions allocate memory + and leave it to the caller to free it. lzma_filters_free() frees + memory given to it, and that memory may have been allocated outside + of liblzma. + + For example, if application calls lzma_str_list_filters(&ptr, ...) + and then uses free(ptr), something bad (memory corruption, crash) + will happen if the application and liblzma.dll aren't using the + same CRT. This can be worked around with a few lines of extra code. + + All these functions (and many others too) take a pointer to + lzma_allocator structure as an argument. Typically it is set to + NULL to let liblzma use malloc() and free() (and also calloc() + as it is faster than malloc() + memset()). A custom lzma_allocator + can be used to wrap malloc() and free() from application's CRT: + + static void * LZMA_API_CALL + my_alloc(void *opaque, size_t nmemb, size_t size) + { + // liblzma guarantees that this won't overflow. + return malloc(nmemb * size); + } + + static void LZMA_API_CALL + my_free(void *opaque, void *ptr) + { + free(ptr); + } + + static const lzma_allocator allocator + = { &my_alloc, &my_free, NULL }; + + By passing &allocator to the problematic functions, CRT mixing + should not cause any problems. There is no need to use &allocator + with functions other than those listed above. +