wok-4.x annotate linux/stuff/001-squashfs-decompressors-add-xz-decompressor-module.patch @ rev 11148

xterm: add xterm.desktop & xterm.png
author Pascal Bellard <pascal.bellard@slitaz.org>
date Tue Nov 01 09:40:24 2011 +0100 (2011-11-01)
parents
children
rev   line source
slaxemulator@7650 1 From: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 2 Date: Thu, 2 Dec 2010 19:14:19 +0000 (+0200)
slaxemulator@7650 3 Subject: Decompressors: Add XZ decompressor module
slaxemulator@7650 4 X-Git-Url: http://git.kernel.org/?p=linux%2Fkernel%2Fgit%2Fpkl%2Fsquashfs-xz.git;a=commitdiff_plain;h=3dbc3fe7878e53b43064a12d4ab31ca4c18ce85f
slaxemulator@7650 5
slaxemulator@7650 6 Decompressors: Add XZ decompressor module
slaxemulator@7650 7
slaxemulator@7650 8 In userspace, the .lzma format has become mostly a legacy
slaxemulator@7650 9 file format that got superseded by the .xz format. Similarly,
slaxemulator@7650 10 LZMA Utils was superseded by XZ Utils.
slaxemulator@7650 11
slaxemulator@7650 12 These patches add support for XZ decompression into
slaxemulator@7650 13 the kernel. Most of the code is as is from XZ Embedded
slaxemulator@7650 14 <http://tukaani.org/xz/embedded.html>. It was written for
slaxemulator@7650 15 the Linux kernel but is usable in other projects too.
slaxemulator@7650 16
slaxemulator@7650 17 Advantages of XZ over the current LZMA code in the kernel:
slaxemulator@7650 18 - Nice API that can be used by other kernel modules; it's
slaxemulator@7650 19 not limited to kernel, initramfs, and initrd decompression.
slaxemulator@7650 20 - Integrity check support (CRC32)
slaxemulator@7650 21 - BCJ filters improve compression of executable code on
slaxemulator@7650 22 certain architectures. These together with LZMA2 can
slaxemulator@7650 23 produce a few percent smaller kernel or Squashfs images
slaxemulator@7650 24 than plain LZMA without making the decompression slower.
slaxemulator@7650 25
slaxemulator@7650 26 This patch: Add the main decompression code (xz_dec), testing
slaxemulator@7650 27 module (xz_dec_test), wrapper script (xz_wrap.sh) for the xz
slaxemulator@7650 28 command line tool, and documentation. The xz_dec module is
slaxemulator@7650 29 enough to have a usable XZ decompressor e.g. for Squashfs.
slaxemulator@7650 30
slaxemulator@7650 31 Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 32 ---
slaxemulator@7650 33
slaxemulator@7650 34 diff --git a/Documentation/xz.txt b/Documentation/xz.txt
slaxemulator@7650 35 new file mode 100644
slaxemulator@7650 36 index 0000000..68329ac
slaxemulator@7650 37 --- /dev/null
slaxemulator@7650 38 +++ b/Documentation/xz.txt
slaxemulator@7650 39 @@ -0,0 +1,122 @@
slaxemulator@7650 40 +
slaxemulator@7650 41 +XZ data compression in Linux
slaxemulator@7650 42 +============================
slaxemulator@7650 43 +
slaxemulator@7650 44 +Introduction
slaxemulator@7650 45 +
slaxemulator@7650 46 + XZ is a general purpose data compression format with high compression
slaxemulator@7650 47 + ratio and relatively fast decompression. The primary compression
slaxemulator@7650 48 + algorithm (filter) is LZMA2. Additional filters can be used to improve
slaxemulator@7650 49 + compression ratio even further. E.g. Branch/Call/Jump (BCJ) filters
slaxemulator@7650 50 + improve compression ratio of executable data.
slaxemulator@7650 51 +
slaxemulator@7650 52 + The XZ decompressor in Linux is called XZ Embedded. It supports
slaxemulator@7650 53 + the LZMA2 filter and optionally also BCJ filters. CRC32 is supported
slaxemulator@7650 54 + for integrity checking. The home page of XZ Embedded is at
slaxemulator@7650 55 + <http://tukaani.org/xz/embedded.html>, where you can find the
slaxemulator@7650 56 + latest version and also information about using the code outside
slaxemulator@7650 57 + the Linux kernel.
slaxemulator@7650 58 +
slaxemulator@7650 59 + For userspace, XZ Utils provide a zlib-like compression library
slaxemulator@7650 60 + and a gzip-like command line tool. XZ Utils can be downloaded from
slaxemulator@7650 61 + <http://tukaani.org/xz/>.
slaxemulator@7650 62 +
slaxemulator@7650 63 +XZ related components in the kernel
slaxemulator@7650 64 +
slaxemulator@7650 65 + The xz_dec module provides XZ decompressor with single-call (buffer
slaxemulator@7650 66 + to buffer) and multi-call (stateful) APIs. The usage of the xz_dec
slaxemulator@7650 67 + module is documented in include/linux/xz.h.
slaxemulator@7650 68 +
slaxemulator@7650 69 + The xz_dec_test module is for testing xz_dec. xz_dec_test is not
slaxemulator@7650 70 + useful unless you are hacking the XZ decompressor. xz_dec_test
slaxemulator@7650 71 + allocates a char device major dynamically to which one can write
slaxemulator@7650 72 + .xz files from userspace. The decompressed output is thrown away.
slaxemulator@7650 73 + Keep an eye on dmesg to see diagnostics printed by xz_dec_test.
slaxemulator@7650 74 + See the xz_dec_test source code for the details.
slaxemulator@7650 75 +
slaxemulator@7650 76 + For decompressing the kernel image, initramfs, and initrd, there
slaxemulator@7650 77 + is a wrapper function in lib/decompress_unxz.c. Its API is the
slaxemulator@7650 78 + same as in other decompress_*.c files, which is defined in
slaxemulator@7650 79 + include/linux/decompress/generic.h.
slaxemulator@7650 80 +
slaxemulator@7650 81 + scripts/xz_wrap.sh is a wrapper for the xz command line tool found
slaxemulator@7650 82 + from XZ Utils. The wrapper sets compression options to values suitable
slaxemulator@7650 83 + for compressing the kernel image.
slaxemulator@7650 84 +
slaxemulator@7650 85 + For kernel makefiles, two commands are provided for use with
slaxemulator@7650 86 + $(call if_needed). The kernel image should be compressed with
slaxemulator@7650 87 + $(call if_needed,xzkern) which will use a BCJ filter and a big LZMA2
slaxemulator@7650 88 + dictionary. It will also append a four-byte trailer containing the
slaxemulator@7650 89 + uncompressed size of the file, which is needed by the boot code.
slaxemulator@7650 90 + Other things should be compressed with $(call if_needed,xzmisc)
slaxemulator@7650 91 + which will use no BCJ filter and 1 MiB LZMA2 dictionary.
slaxemulator@7650 92 +
slaxemulator@7650 93 +Notes on compression options
slaxemulator@7650 94 +
slaxemulator@7650 95 + Since the XZ Embedded supports only streams with no integrity check or
slaxemulator@7650 96 + CRC32, make sure that you don't use some other integrity check type
slaxemulator@7650 97 + when encoding files that are supposed to be decoded by the kernel. With
slaxemulator@7650 98 + liblzma, you need to use either LZMA_CHECK_NONE or LZMA_CHECK_CRC32
slaxemulator@7650 99 + when encoding. With the xz command line tool, use --check=none or
slaxemulator@7650 100 + --check=crc32.
slaxemulator@7650 101 +
slaxemulator@7650 102 + Using CRC32 is strongly recommended unless there is some other layer
slaxemulator@7650 103 + which will verify the integrity of the uncompressed data anyway.
slaxemulator@7650 104 + Double checking the integrity would probably be waste of CPU cycles.
slaxemulator@7650 105 + Note that the headers will always have a CRC32 which will be validated
slaxemulator@7650 106 + by the decoder; you can only change the integrity check type (or
slaxemulator@7650 107 + disable it) for the actual uncompressed data.
slaxemulator@7650 108 +
slaxemulator@7650 109 + In userspace, LZMA2 is typically used with dictionary sizes of several
slaxemulator@7650 110 + megabytes. The decoder needs to have the dictionary in RAM, thus big
slaxemulator@7650 111 + dictionaries cannot be used for files that are intended to be decoded
slaxemulator@7650 112 + by the kernel. 1 MiB is probably the maximum reasonable dictionary
slaxemulator@7650 113 + size for in-kernel use (maybe more is OK for initramfs). The presets
slaxemulator@7650 114 + in XZ Utils may not be optimal when creating files for the kernel,
slaxemulator@7650 115 + so don't hesitate to use custom settings. Example:
slaxemulator@7650 116 +
slaxemulator@7650 117 + xz --check=crc32 --lzma2=dict=512KiB inputfile
slaxemulator@7650 118 +
slaxemulator@7650 119 + An exception to above dictionary size limitation is when the decoder
slaxemulator@7650 120 + is used in single-call mode. Decompressing the kernel itself is an
slaxemulator@7650 121 + example of this situation. In single-call mode, the memory usage
slaxemulator@7650 122 + doesn't depend on the dictionary size, and it is perfectly fine to
slaxemulator@7650 123 + use a big dictionary: for maximum compression, the dictionary should
slaxemulator@7650 124 + be at least as big as the uncompressed data itself.
slaxemulator@7650 125 +
slaxemulator@7650 126 +Future plans
slaxemulator@7650 127 +
slaxemulator@7650 128 + Creating a limited XZ encoder may be considered if people think it is
slaxemulator@7650 129 + useful. LZMA2 is slower to compress than e.g. Deflate or LZO even at
slaxemulator@7650 130 + the fastest settings, so it isn't clear if LZMA2 encoder is wanted
slaxemulator@7650 131 + into the kernel.
slaxemulator@7650 132 +
slaxemulator@7650 133 + Support for limited random-access reading is planned for the
slaxemulator@7650 134 + decompression code. I don't know if it could have any use in the
slaxemulator@7650 135 + kernel, but I know that it would be useful in some embedded projects
slaxemulator@7650 136 + outside the Linux kernel.
slaxemulator@7650 137 +
slaxemulator@7650 138 +Conformance to the .xz file format specification
slaxemulator@7650 139 +
slaxemulator@7650 140 + There are a couple of corner cases where things have been simplified
slaxemulator@7650 141 + at expense of detecting errors as early as possible. These should not
slaxemulator@7650 142 + matter in practice all, since they don't cause security issues. But
slaxemulator@7650 143 + it is good to know this if testing the code e.g. with the test files
slaxemulator@7650 144 + from XZ Utils.
slaxemulator@7650 145 +
slaxemulator@7650 146 +Reporting bugs
slaxemulator@7650 147 +
slaxemulator@7650 148 + Before reporting a bug, please check that it's not fixed already
slaxemulator@7650 149 + at upstream. See <http://tukaani.org/xz/embedded.html> to get the
slaxemulator@7650 150 + latest code.
slaxemulator@7650 151 +
slaxemulator@7650 152 + Report bugs to <lasse.collin@tukaani.org> or visit #tukaani on
slaxemulator@7650 153 + Freenode and talk to Larhzu. I don't actively read LKML or other
slaxemulator@7650 154 + kernel-related mailing lists, so if there's something I should know,
slaxemulator@7650 155 + you should email to me personally or use IRC.
slaxemulator@7650 156 +
slaxemulator@7650 157 + Don't bother Igor Pavlov with questions about the XZ implementation
slaxemulator@7650 158 + in the kernel or about XZ Utils. While these two implementations
slaxemulator@7650 159 + include essential code that is directly based on Igor Pavlov's code,
slaxemulator@7650 160 + these implementations aren't maintained nor supported by him.
slaxemulator@7650 161 +
slaxemulator@7650 162 diff --git a/include/linux/xz.h b/include/linux/xz.h
slaxemulator@7650 163 new file mode 100644
slaxemulator@7650 164 index 0000000..64cffa6
slaxemulator@7650 165 --- /dev/null
slaxemulator@7650 166 +++ b/include/linux/xz.h
slaxemulator@7650 167 @@ -0,0 +1,264 @@
slaxemulator@7650 168 +/*
slaxemulator@7650 169 + * XZ decompressor
slaxemulator@7650 170 + *
slaxemulator@7650 171 + * Authors: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 172 + * Igor Pavlov <http://7-zip.org/>
slaxemulator@7650 173 + *
slaxemulator@7650 174 + * This file has been put into the public domain.
slaxemulator@7650 175 + * You can do whatever you want with this file.
slaxemulator@7650 176 + */
slaxemulator@7650 177 +
slaxemulator@7650 178 +#ifndef XZ_H
slaxemulator@7650 179 +#define XZ_H
slaxemulator@7650 180 +
slaxemulator@7650 181 +#ifdef __KERNEL__
slaxemulator@7650 182 +# include <linux/stddef.h>
slaxemulator@7650 183 +# include <linux/types.h>
slaxemulator@7650 184 +#else
slaxemulator@7650 185 +# include <stddef.h>
slaxemulator@7650 186 +# include <stdint.h>
slaxemulator@7650 187 +#endif
slaxemulator@7650 188 +
slaxemulator@7650 189 +/* In Linux, this is used to make extern functions static when needed. */
slaxemulator@7650 190 +#ifndef XZ_EXTERN
slaxemulator@7650 191 +# define XZ_EXTERN extern
slaxemulator@7650 192 +#endif
slaxemulator@7650 193 +
slaxemulator@7650 194 +/**
slaxemulator@7650 195 + * enum xz_mode - Operation mode
slaxemulator@7650 196 + *
slaxemulator@7650 197 + * @XZ_SINGLE: Single-call mode. This uses less RAM than
slaxemulator@7650 198 + * than multi-call modes, because the LZMA2
slaxemulator@7650 199 + * dictionary doesn't need to be allocated as
slaxemulator@7650 200 + * part of the decoder state. All required data
slaxemulator@7650 201 + * structures are allocated at initialization,
slaxemulator@7650 202 + * so xz_dec_run() cannot return XZ_MEM_ERROR.
slaxemulator@7650 203 + * @XZ_PREALLOC: Multi-call mode with preallocated LZMA2
slaxemulator@7650 204 + * dictionary buffer. All data structures are
slaxemulator@7650 205 + * allocated at initialization, so xz_dec_run()
slaxemulator@7650 206 + * cannot return XZ_MEM_ERROR.
slaxemulator@7650 207 + * @XZ_DYNALLOC: Multi-call mode. The LZMA2 dictionary is
slaxemulator@7650 208 + * allocated once the required size has been
slaxemulator@7650 209 + * parsed from the stream headers. If the
slaxemulator@7650 210 + * allocation fails, xz_dec_run() will return
slaxemulator@7650 211 + * XZ_MEM_ERROR.
slaxemulator@7650 212 + *
slaxemulator@7650 213 + * It is possible to enable support only for a subset of the above
slaxemulator@7650 214 + * modes at compile time by defining XZ_DEC_SINGLE, XZ_DEC_PREALLOC,
slaxemulator@7650 215 + * or XZ_DEC_DYNALLOC. The xz_dec kernel module is always compiled
slaxemulator@7650 216 + * with support for all operation modes, but the preboot code may
slaxemulator@7650 217 + * be built with fewer features to minimize code size.
slaxemulator@7650 218 + */
slaxemulator@7650 219 +enum xz_mode {
slaxemulator@7650 220 + XZ_SINGLE,
slaxemulator@7650 221 + XZ_PREALLOC,
slaxemulator@7650 222 + XZ_DYNALLOC
slaxemulator@7650 223 +};
slaxemulator@7650 224 +
slaxemulator@7650 225 +/**
slaxemulator@7650 226 + * enum xz_ret - Return codes
slaxemulator@7650 227 + * @XZ_OK: Everything is OK so far. More input or more
slaxemulator@7650 228 + * output space is required to continue. This
slaxemulator@7650 229 + * return code is possible only in multi-call mode
slaxemulator@7650 230 + * (XZ_PREALLOC or XZ_DYNALLOC).
slaxemulator@7650 231 + * @XZ_STREAM_END: Operation finished successfully.
slaxemulator@7650 232 + * @XZ_UNSUPPORTED_CHECK: Integrity check type is not supported. Decoding
slaxemulator@7650 233 + * is still possible in multi-call mode by simply
slaxemulator@7650 234 + * calling xz_dec_run() again.
slaxemulator@7650 235 + * Note that this return value is used only if
slaxemulator@7650 236 + * XZ_DEC_ANY_CHECK was defined at build time,
slaxemulator@7650 237 + * which is not used in the kernel. Unsupported
slaxemulator@7650 238 + * check types return XZ_OPTIONS_ERROR if
slaxemulator@7650 239 + * XZ_DEC_ANY_CHECK was not defined at build time.
slaxemulator@7650 240 + * @XZ_MEM_ERROR: Allocating memory failed. This return code is
slaxemulator@7650 241 + * possible only if the decoder was initialized
slaxemulator@7650 242 + * with XZ_DYNALLOC. The amount of memory that was
slaxemulator@7650 243 + * tried to be allocated was no more than the
slaxemulator@7650 244 + * dict_max argument given to xz_dec_init().
slaxemulator@7650 245 + * @XZ_MEMLIMIT_ERROR: A bigger LZMA2 dictionary would be needed than
slaxemulator@7650 246 + * allowed by the dict_max argument given to
slaxemulator@7650 247 + * xz_dec_init(). This return value is possible
slaxemulator@7650 248 + * only in multi-call mode (XZ_PREALLOC or
slaxemulator@7650 249 + * XZ_DYNALLOC); the single-call mode (XZ_SINGLE)
slaxemulator@7650 250 + * ignores the dict_max argument.
slaxemulator@7650 251 + * @XZ_FORMAT_ERROR: File format was not recognized (wrong magic
slaxemulator@7650 252 + * bytes).
slaxemulator@7650 253 + * @XZ_OPTIONS_ERROR: This implementation doesn't support the requested
slaxemulator@7650 254 + * compression options. In the decoder this means
slaxemulator@7650 255 + * that the header CRC32 matches, but the header
slaxemulator@7650 256 + * itself specifies something that we don't support.
slaxemulator@7650 257 + * @XZ_DATA_ERROR: Compressed data is corrupt.
slaxemulator@7650 258 + * @XZ_BUF_ERROR: Cannot make any progress. Details are slightly
slaxemulator@7650 259 + * different between multi-call and single-call
slaxemulator@7650 260 + * mode; more information below.
slaxemulator@7650 261 + *
slaxemulator@7650 262 + * In multi-call mode, XZ_BUF_ERROR is returned when two consecutive calls
slaxemulator@7650 263 + * to XZ code cannot consume any input and cannot produce any new output.
slaxemulator@7650 264 + * This happens when there is no new input available, or the output buffer
slaxemulator@7650 265 + * is full while at least one output byte is still pending. Assuming your
slaxemulator@7650 266 + * code is not buggy, you can get this error only when decoding a compressed
slaxemulator@7650 267 + * stream that is truncated or otherwise corrupt.
slaxemulator@7650 268 + *
slaxemulator@7650 269 + * In single-call mode, XZ_BUF_ERROR is returned only when the output buffer
slaxemulator@7650 270 + * is too small or the compressed input is corrupt in a way that makes the
slaxemulator@7650 271 + * decoder produce more output than the caller expected. When it is
slaxemulator@7650 272 + * (relatively) clear that the compressed input is truncated, XZ_DATA_ERROR
slaxemulator@7650 273 + * is used instead of XZ_BUF_ERROR.
slaxemulator@7650 274 + */
slaxemulator@7650 275 +enum xz_ret {
slaxemulator@7650 276 + XZ_OK,
slaxemulator@7650 277 + XZ_STREAM_END,
slaxemulator@7650 278 + XZ_UNSUPPORTED_CHECK,
slaxemulator@7650 279 + XZ_MEM_ERROR,
slaxemulator@7650 280 + XZ_MEMLIMIT_ERROR,
slaxemulator@7650 281 + XZ_FORMAT_ERROR,
slaxemulator@7650 282 + XZ_OPTIONS_ERROR,
slaxemulator@7650 283 + XZ_DATA_ERROR,
slaxemulator@7650 284 + XZ_BUF_ERROR
slaxemulator@7650 285 +};
slaxemulator@7650 286 +
slaxemulator@7650 287 +/**
slaxemulator@7650 288 + * struct xz_buf - Passing input and output buffers to XZ code
slaxemulator@7650 289 + * @in: Beginning of the input buffer. This may be NULL if and only
slaxemulator@7650 290 + * if in_pos is equal to in_size.
slaxemulator@7650 291 + * @in_pos: Current position in the input buffer. This must not exceed
slaxemulator@7650 292 + * in_size.
slaxemulator@7650 293 + * @in_size: Size of the input buffer
slaxemulator@7650 294 + * @out: Beginning of the output buffer. This may be NULL if and only
slaxemulator@7650 295 + * if out_pos is equal to out_size.
slaxemulator@7650 296 + * @out_pos: Current position in the output buffer. This must not exceed
slaxemulator@7650 297 + * out_size.
slaxemulator@7650 298 + * @out_size: Size of the output buffer
slaxemulator@7650 299 + *
slaxemulator@7650 300 + * Only the contents of the output buffer from out[out_pos] onward, and
slaxemulator@7650 301 + * the variables in_pos and out_pos are modified by the XZ code.
slaxemulator@7650 302 + */
slaxemulator@7650 303 +struct xz_buf {
slaxemulator@7650 304 + const uint8_t *in;
slaxemulator@7650 305 + size_t in_pos;
slaxemulator@7650 306 + size_t in_size;
slaxemulator@7650 307 +
slaxemulator@7650 308 + uint8_t *out;
slaxemulator@7650 309 + size_t out_pos;
slaxemulator@7650 310 + size_t out_size;
slaxemulator@7650 311 +};
slaxemulator@7650 312 +
slaxemulator@7650 313 +/**
slaxemulator@7650 314 + * struct xz_dec - Opaque type to hold the XZ decoder state
slaxemulator@7650 315 + */
slaxemulator@7650 316 +struct xz_dec;
slaxemulator@7650 317 +
slaxemulator@7650 318 +/**
slaxemulator@7650 319 + * xz_dec_init() - Allocate and initialize a XZ decoder state
slaxemulator@7650 320 + * @mode: Operation mode
slaxemulator@7650 321 + * @dict_max: Maximum size of the LZMA2 dictionary (history buffer) for
slaxemulator@7650 322 + * multi-call decoding. This is ignored in single-call mode
slaxemulator@7650 323 + * (mode == XZ_SINGLE). LZMA2 dictionary is always 2^n bytes
slaxemulator@7650 324 + * or 2^n + 2^(n-1) bytes (the latter sizes are less common
slaxemulator@7650 325 + * in practice), so other values for dict_max don't make sense.
slaxemulator@7650 326 + * In the kernel, dictionary sizes of 64 KiB, 128 KiB, 256 KiB,
slaxemulator@7650 327 + * 512 KiB, and 1 MiB are probably the only reasonable values,
slaxemulator@7650 328 + * except for kernel and initramfs images where a bigger
slaxemulator@7650 329 + * dictionary can be fine and useful.
slaxemulator@7650 330 + *
slaxemulator@7650 331 + * Single-call mode (XZ_SINGLE): xz_dec_run() decodes the whole stream at
slaxemulator@7650 332 + * once. The caller must provide enough output space or the decoding will
slaxemulator@7650 333 + * fail. The output space is used as the dictionary buffer, which is why
slaxemulator@7650 334 + * there is no need to allocate the dictionary as part of the decoder's
slaxemulator@7650 335 + * internal state.
slaxemulator@7650 336 + *
slaxemulator@7650 337 + * Because the output buffer is used as the workspace, streams encoded using
slaxemulator@7650 338 + * a big dictionary are not a problem in single-call mode. It is enough that
slaxemulator@7650 339 + * the output buffer is big enough to hold the actual uncompressed data; it
slaxemulator@7650 340 + * can be smaller than the dictionary size stored in the stream headers.
slaxemulator@7650 341 + *
slaxemulator@7650 342 + * Multi-call mode with preallocated dictionary (XZ_PREALLOC): dict_max bytes
slaxemulator@7650 343 + * of memory is preallocated for the LZMA2 dictionary. This way there is no
slaxemulator@7650 344 + * risk that xz_dec_run() could run out of memory, since xz_dec_run() will
slaxemulator@7650 345 + * never allocate any memory. Instead, if the preallocated dictionary is too
slaxemulator@7650 346 + * small for decoding the given input stream, xz_dec_run() will return
slaxemulator@7650 347 + * XZ_MEMLIMIT_ERROR. Thus, it is important to know what kind of data will be
slaxemulator@7650 348 + * decoded to avoid allocating excessive amount of memory for the dictionary.
slaxemulator@7650 349 + *
slaxemulator@7650 350 + * Multi-call mode with dynamically allocated dictionary (XZ_DYNALLOC):
slaxemulator@7650 351 + * dict_max specifies the maximum allowed dictionary size that xz_dec_run()
slaxemulator@7650 352 + * may allocate once it has parsed the dictionary size from the stream
slaxemulator@7650 353 + * headers. This way excessive allocations can be avoided while still
slaxemulator@7650 354 + * limiting the maximum memory usage to a sane value to prevent running the
slaxemulator@7650 355 + * system out of memory when decompressing streams from untrusted sources.
slaxemulator@7650 356 + *
slaxemulator@7650 357 + * On success, xz_dec_init() returns a pointer to struct xz_dec, which is
slaxemulator@7650 358 + * ready to be used with xz_dec_run(). If memory allocation fails,
slaxemulator@7650 359 + * xz_dec_init() returns NULL.
slaxemulator@7650 360 + */
slaxemulator@7650 361 +XZ_EXTERN struct xz_dec *xz_dec_init(enum xz_mode mode, uint32_t dict_max);
slaxemulator@7650 362 +
slaxemulator@7650 363 +/**
slaxemulator@7650 364 + * xz_dec_run() - Run the XZ decoder
slaxemulator@7650 365 + * @s: Decoder state allocated using xz_dec_init()
slaxemulator@7650 366 + * @b: Input and output buffers
slaxemulator@7650 367 + *
slaxemulator@7650 368 + * The possible return values depend on build options and operation mode.
slaxemulator@7650 369 + * See enum xz_ret for details.
slaxemulator@7650 370 + *
slaxemulator@7650 371 + * Note that if an error occurs in single-call mode (return value is not
slaxemulator@7650 372 + * XZ_STREAM_END), b->in_pos and b->out_pos are not modified and the
slaxemulator@7650 373 + * contents of the output buffer from b->out[b->out_pos] onward are
slaxemulator@7650 374 + * undefined. This is true even after XZ_BUF_ERROR, because with some filter
slaxemulator@7650 375 + * chains, there may be a second pass over the output buffer, and this pass
slaxemulator@7650 376 + * cannot be properly done if the output buffer is truncated. Thus, you
slaxemulator@7650 377 + * cannot give the single-call decoder a too small buffer and then expect to
slaxemulator@7650 378 + * get that amount valid data from the beginning of the stream. You must use
slaxemulator@7650 379 + * the multi-call decoder if you don't want to uncompress the whole stream.
slaxemulator@7650 380 + */
slaxemulator@7650 381 +XZ_EXTERN enum xz_ret xz_dec_run(struct xz_dec *s, struct xz_buf *b);
slaxemulator@7650 382 +
slaxemulator@7650 383 +/**
slaxemulator@7650 384 + * xz_dec_reset() - Reset an already allocated decoder state
slaxemulator@7650 385 + * @s: Decoder state allocated using xz_dec_init()
slaxemulator@7650 386 + *
slaxemulator@7650 387 + * This function can be used to reset the multi-call decoder state without
slaxemulator@7650 388 + * freeing and reallocating memory with xz_dec_end() and xz_dec_init().
slaxemulator@7650 389 + *
slaxemulator@7650 390 + * In single-call mode, xz_dec_reset() is always called in the beginning of
slaxemulator@7650 391 + * xz_dec_run(). Thus, explicit call to xz_dec_reset() is useful only in
slaxemulator@7650 392 + * multi-call mode.
slaxemulator@7650 393 + */
slaxemulator@7650 394 +XZ_EXTERN void xz_dec_reset(struct xz_dec *s);
slaxemulator@7650 395 +
slaxemulator@7650 396 +/**
slaxemulator@7650 397 + * xz_dec_end() - Free the memory allocated for the decoder state
slaxemulator@7650 398 + * @s: Decoder state allocated using xz_dec_init(). If s is NULL,
slaxemulator@7650 399 + * this function does nothing.
slaxemulator@7650 400 + */
slaxemulator@7650 401 +XZ_EXTERN void xz_dec_end(struct xz_dec *s);
slaxemulator@7650 402 +
slaxemulator@7650 403 +/*
slaxemulator@7650 404 + * Standalone build (userspace build or in-kernel build for boot time use)
slaxemulator@7650 405 + * needs a CRC32 implementation. For normal in-kernel use, kernel's own
slaxemulator@7650 406 + * CRC32 module is used instead, and users of this module don't need to
slaxemulator@7650 407 + * care about the functions below.
slaxemulator@7650 408 + */
slaxemulator@7650 409 +#ifndef XZ_INTERNAL_CRC32
slaxemulator@7650 410 +# ifdef __KERNEL__
slaxemulator@7650 411 +# define XZ_INTERNAL_CRC32 0
slaxemulator@7650 412 +# else
slaxemulator@7650 413 +# define XZ_INTERNAL_CRC32 1
slaxemulator@7650 414 +# endif
slaxemulator@7650 415 +#endif
slaxemulator@7650 416 +
slaxemulator@7650 417 +#if XZ_INTERNAL_CRC32
slaxemulator@7650 418 +/*
slaxemulator@7650 419 + * This must be called before any other xz_* function to initialize
slaxemulator@7650 420 + * the CRC32 lookup table.
slaxemulator@7650 421 + */
slaxemulator@7650 422 +XZ_EXTERN void xz_crc32_init(void);
slaxemulator@7650 423 +
slaxemulator@7650 424 +/*
slaxemulator@7650 425 + * Update CRC32 value using the polynomial from IEEE-802.3. To start a new
slaxemulator@7650 426 + * calculation, the third argument must be zero. To continue the calculation,
slaxemulator@7650 427 + * the previously returned value is passed as the third argument.
slaxemulator@7650 428 + */
slaxemulator@7650 429 +XZ_EXTERN uint32_t xz_crc32(const uint8_t *buf, size_t size, uint32_t crc);
slaxemulator@7650 430 +#endif
slaxemulator@7650 431 +#endif
slaxemulator@7650 432 diff --git a/lib/Kconfig b/lib/Kconfig
slaxemulator@7650 433 index fa9bf2c..6090314 100644
slaxemulator@7650 434 --- a/lib/Kconfig
slaxemulator@7650 435 +++ b/lib/Kconfig
slaxemulator@7650 436 @@ -106,6 +106,8 @@ config LZO_COMPRESS
slaxemulator@7650 437 config LZO_DECOMPRESS
slaxemulator@7650 438 tristate
slaxemulator@7650 439
slaxemulator@7650 440 +source "lib/xz/Kconfig"
slaxemulator@7650 441 +
slaxemulator@7650 442 #
slaxemulator@7650 443 # These all provide a common interface (hence the apparent duplication with
slaxemulator@7650 444 # ZLIB_INFLATE; DECOMPRESS_GZIP is just a wrapper.)
slaxemulator@7650 445 diff --git a/lib/Makefile b/lib/Makefile
slaxemulator@7650 446 index e6a3763..f2f98dd 100644
slaxemulator@7650 447 --- a/lib/Makefile
slaxemulator@7650 448 +++ b/lib/Makefile
slaxemulator@7650 449 @@ -69,6 +69,7 @@ obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
slaxemulator@7650 450 obj-$(CONFIG_REED_SOLOMON) += reed_solomon/
slaxemulator@7650 451 obj-$(CONFIG_LZO_COMPRESS) += lzo/
slaxemulator@7650 452 obj-$(CONFIG_LZO_DECOMPRESS) += lzo/
slaxemulator@7650 453 +obj-$(CONFIG_XZ_DEC) += xz/
slaxemulator@7650 454 obj-$(CONFIG_RAID6_PQ) += raid6/
slaxemulator@7650 455
slaxemulator@7650 456 lib-$(CONFIG_DECOMPRESS_GZIP) += decompress_inflate.o
slaxemulator@7650 457 diff --git a/lib/xz/Kconfig b/lib/xz/Kconfig
slaxemulator@7650 458 new file mode 100644
slaxemulator@7650 459 index 0000000..e3b6e18
slaxemulator@7650 460 --- /dev/null
slaxemulator@7650 461 +++ b/lib/xz/Kconfig
slaxemulator@7650 462 @@ -0,0 +1,59 @@
slaxemulator@7650 463 +config XZ_DEC
slaxemulator@7650 464 + tristate "XZ decompression support"
slaxemulator@7650 465 + select CRC32
slaxemulator@7650 466 + help
slaxemulator@7650 467 + LZMA2 compression algorithm and BCJ filters are supported using
slaxemulator@7650 468 + the .xz file format as the container. For integrity checking,
slaxemulator@7650 469 + CRC32 is supported. See Documentation/xz.txt for more information.
slaxemulator@7650 470 +
slaxemulator@7650 471 +config XZ_DEC_X86
slaxemulator@7650 472 + bool "x86 BCJ filter decoder" if EMBEDDED
slaxemulator@7650 473 + default y
slaxemulator@7650 474 + depends on XZ_DEC
slaxemulator@7650 475 + select XZ_DEC_BCJ
slaxemulator@7650 476 +
slaxemulator@7650 477 +config XZ_DEC_POWERPC
slaxemulator@7650 478 + bool "PowerPC BCJ filter decoder" if EMBEDDED
slaxemulator@7650 479 + default y
slaxemulator@7650 480 + depends on XZ_DEC
slaxemulator@7650 481 + select XZ_DEC_BCJ
slaxemulator@7650 482 +
slaxemulator@7650 483 +config XZ_DEC_IA64
slaxemulator@7650 484 + bool "IA-64 BCJ filter decoder" if EMBEDDED
slaxemulator@7650 485 + default y
slaxemulator@7650 486 + depends on XZ_DEC
slaxemulator@7650 487 + select XZ_DEC_BCJ
slaxemulator@7650 488 +
slaxemulator@7650 489 +config XZ_DEC_ARM
slaxemulator@7650 490 + bool "ARM BCJ filter decoder" if EMBEDDED
slaxemulator@7650 491 + default y
slaxemulator@7650 492 + depends on XZ_DEC
slaxemulator@7650 493 + select XZ_DEC_BCJ
slaxemulator@7650 494 +
slaxemulator@7650 495 +config XZ_DEC_ARMTHUMB
slaxemulator@7650 496 + bool "ARM-Thumb BCJ filter decoder" if EMBEDDED
slaxemulator@7650 497 + default y
slaxemulator@7650 498 + depends on XZ_DEC
slaxemulator@7650 499 + select XZ_DEC_BCJ
slaxemulator@7650 500 +
slaxemulator@7650 501 +config XZ_DEC_SPARC
slaxemulator@7650 502 + bool "SPARC BCJ filter decoder" if EMBEDDED
slaxemulator@7650 503 + default y
slaxemulator@7650 504 + depends on XZ_DEC
slaxemulator@7650 505 + select XZ_DEC_BCJ
slaxemulator@7650 506 +
slaxemulator@7650 507 +config XZ_DEC_BCJ
slaxemulator@7650 508 + bool
slaxemulator@7650 509 + default n
slaxemulator@7650 510 +
slaxemulator@7650 511 +config XZ_DEC_TEST
slaxemulator@7650 512 + tristate "XZ decompressor tester"
slaxemulator@7650 513 + default n
slaxemulator@7650 514 + depends on XZ_DEC
slaxemulator@7650 515 + help
slaxemulator@7650 516 + This allows passing .xz files to the in-kernel XZ decoder via
slaxemulator@7650 517 + a character special file. It calculates CRC32 of the decompressed
slaxemulator@7650 518 + data and writes diagnostics to the system log.
slaxemulator@7650 519 +
slaxemulator@7650 520 + Unless you are developing the XZ decoder, you don't need this
slaxemulator@7650 521 + and should say N.
slaxemulator@7650 522 diff --git a/lib/xz/Makefile b/lib/xz/Makefile
slaxemulator@7650 523 new file mode 100644
slaxemulator@7650 524 index 0000000..a7fa769
slaxemulator@7650 525 --- /dev/null
slaxemulator@7650 526 +++ b/lib/xz/Makefile
slaxemulator@7650 527 @@ -0,0 +1,5 @@
slaxemulator@7650 528 +obj-$(CONFIG_XZ_DEC) += xz_dec.o
slaxemulator@7650 529 +xz_dec-y := xz_dec_syms.o xz_dec_stream.o xz_dec_lzma2.o
slaxemulator@7650 530 +xz_dec-$(CONFIG_XZ_DEC_BCJ) += xz_dec_bcj.o
slaxemulator@7650 531 +
slaxemulator@7650 532 +obj-$(CONFIG_XZ_DEC_TEST) += xz_dec_test.o
slaxemulator@7650 533 diff --git a/lib/xz/xz_crc32.c b/lib/xz/xz_crc32.c
slaxemulator@7650 534 new file mode 100644
slaxemulator@7650 535 index 0000000..34532d1
slaxemulator@7650 536 --- /dev/null
slaxemulator@7650 537 +++ b/lib/xz/xz_crc32.c
slaxemulator@7650 538 @@ -0,0 +1,59 @@
slaxemulator@7650 539 +/*
slaxemulator@7650 540 + * CRC32 using the polynomial from IEEE-802.3
slaxemulator@7650 541 + *
slaxemulator@7650 542 + * Authors: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 543 + * Igor Pavlov <http://7-zip.org/>
slaxemulator@7650 544 + *
slaxemulator@7650 545 + * This file has been put into the public domain.
slaxemulator@7650 546 + * You can do whatever you want with this file.
slaxemulator@7650 547 + */
slaxemulator@7650 548 +
slaxemulator@7650 549 +/*
slaxemulator@7650 550 + * This is not the fastest implementation, but it is pretty compact.
slaxemulator@7650 551 + * The fastest versions of xz_crc32() on modern CPUs without hardware
slaxemulator@7650 552 + * accelerated CRC instruction are 3-5 times as fast as this version,
slaxemulator@7650 553 + * but they are bigger and use more memory for the lookup table.
slaxemulator@7650 554 + */
slaxemulator@7650 555 +
slaxemulator@7650 556 +#include "xz_private.h"
slaxemulator@7650 557 +
slaxemulator@7650 558 +/*
slaxemulator@7650 559 + * STATIC_RW_DATA is used in the pre-boot environment on some architectures.
slaxemulator@7650 560 + * See <linux/decompress/mm.h> for details.
slaxemulator@7650 561 + */
slaxemulator@7650 562 +#ifndef STATIC_RW_DATA
slaxemulator@7650 563 +# define STATIC_RW_DATA static
slaxemulator@7650 564 +#endif
slaxemulator@7650 565 +
slaxemulator@7650 566 +STATIC_RW_DATA uint32_t xz_crc32_table[256];
slaxemulator@7650 567 +
slaxemulator@7650 568 +XZ_EXTERN void xz_crc32_init(void)
slaxemulator@7650 569 +{
slaxemulator@7650 570 + const uint32_t poly = 0xEDB88320;
slaxemulator@7650 571 +
slaxemulator@7650 572 + uint32_t i;
slaxemulator@7650 573 + uint32_t j;
slaxemulator@7650 574 + uint32_t r;
slaxemulator@7650 575 +
slaxemulator@7650 576 + for (i = 0; i < 256; ++i) {
slaxemulator@7650 577 + r = i;
slaxemulator@7650 578 + for (j = 0; j < 8; ++j)
slaxemulator@7650 579 + r = (r >> 1) ^ (poly & ~((r & 1) - 1));
slaxemulator@7650 580 +
slaxemulator@7650 581 + xz_crc32_table[i] = r;
slaxemulator@7650 582 + }
slaxemulator@7650 583 +
slaxemulator@7650 584 + return;
slaxemulator@7650 585 +}
slaxemulator@7650 586 +
slaxemulator@7650 587 +XZ_EXTERN uint32_t xz_crc32(const uint8_t *buf, size_t size, uint32_t crc)
slaxemulator@7650 588 +{
slaxemulator@7650 589 + crc = ~crc;
slaxemulator@7650 590 +
slaxemulator@7650 591 + while (size != 0) {
slaxemulator@7650 592 + crc = xz_crc32_table[*buf++ ^ (crc & 0xFF)] ^ (crc >> 8);
slaxemulator@7650 593 + --size;
slaxemulator@7650 594 + }
slaxemulator@7650 595 +
slaxemulator@7650 596 + return ~crc;
slaxemulator@7650 597 +}
slaxemulator@7650 598 diff --git a/lib/xz/xz_dec_bcj.c b/lib/xz/xz_dec_bcj.c
slaxemulator@7650 599 new file mode 100644
slaxemulator@7650 600 index 0000000..e51e255
slaxemulator@7650 601 --- /dev/null
slaxemulator@7650 602 +++ b/lib/xz/xz_dec_bcj.c
slaxemulator@7650 603 @@ -0,0 +1,561 @@
slaxemulator@7650 604 +/*
slaxemulator@7650 605 + * Branch/Call/Jump (BCJ) filter decoders
slaxemulator@7650 606 + *
slaxemulator@7650 607 + * Authors: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 608 + * Igor Pavlov <http://7-zip.org/>
slaxemulator@7650 609 + *
slaxemulator@7650 610 + * This file has been put into the public domain.
slaxemulator@7650 611 + * You can do whatever you want with this file.
slaxemulator@7650 612 + */
slaxemulator@7650 613 +
slaxemulator@7650 614 +#include "xz_private.h"
slaxemulator@7650 615 +
slaxemulator@7650 616 +/*
slaxemulator@7650 617 + * The rest of the file is inside this ifdef. It makes things a little more
slaxemulator@7650 618 + * convenient when building without support for any BCJ filters.
slaxemulator@7650 619 + */
slaxemulator@7650 620 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 621 +
slaxemulator@7650 622 +struct xz_dec_bcj {
slaxemulator@7650 623 + /* Type of the BCJ filter being used */
slaxemulator@7650 624 + enum {
slaxemulator@7650 625 + BCJ_X86 = 4, /* x86 or x86-64 */
slaxemulator@7650 626 + BCJ_POWERPC = 5, /* Big endian only */
slaxemulator@7650 627 + BCJ_IA64 = 6, /* Big or little endian */
slaxemulator@7650 628 + BCJ_ARM = 7, /* Little endian only */
slaxemulator@7650 629 + BCJ_ARMTHUMB = 8, /* Little endian only */
slaxemulator@7650 630 + BCJ_SPARC = 9 /* Big or little endian */
slaxemulator@7650 631 + } type;
slaxemulator@7650 632 +
slaxemulator@7650 633 + /*
slaxemulator@7650 634 + * Return value of the next filter in the chain. We need to preserve
slaxemulator@7650 635 + * this information across calls, because we must not call the next
slaxemulator@7650 636 + * filter anymore once it has returned XZ_STREAM_END.
slaxemulator@7650 637 + */
slaxemulator@7650 638 + enum xz_ret ret;
slaxemulator@7650 639 +
slaxemulator@7650 640 + /* True if we are operating in single-call mode. */
slaxemulator@7650 641 + bool single_call;
slaxemulator@7650 642 +
slaxemulator@7650 643 + /*
slaxemulator@7650 644 + * Absolute position relative to the beginning of the uncompressed
slaxemulator@7650 645 + * data (in a single .xz Block). We care only about the lowest 32
slaxemulator@7650 646 + * bits so this doesn't need to be uint64_t even with big files.
slaxemulator@7650 647 + */
slaxemulator@7650 648 + uint32_t pos;
slaxemulator@7650 649 +
slaxemulator@7650 650 + /* x86 filter state */
slaxemulator@7650 651 + uint32_t x86_prev_mask;
slaxemulator@7650 652 +
slaxemulator@7650 653 + /* Temporary space to hold the variables from struct xz_buf */
slaxemulator@7650 654 + uint8_t *out;
slaxemulator@7650 655 + size_t out_pos;
slaxemulator@7650 656 + size_t out_size;
slaxemulator@7650 657 +
slaxemulator@7650 658 + struct {
slaxemulator@7650 659 + /* Amount of already filtered data in the beginning of buf */
slaxemulator@7650 660 + size_t filtered;
slaxemulator@7650 661 +
slaxemulator@7650 662 + /* Total amount of data currently stored in buf */
slaxemulator@7650 663 + size_t size;
slaxemulator@7650 664 +
slaxemulator@7650 665 + /*
slaxemulator@7650 666 + * Buffer to hold a mix of filtered and unfiltered data. This
slaxemulator@7650 667 + * needs to be big enough to hold Alignment + 2 * Look-ahead:
slaxemulator@7650 668 + *
slaxemulator@7650 669 + * Type Alignment Look-ahead
slaxemulator@7650 670 + * x86 1 4
slaxemulator@7650 671 + * PowerPC 4 0
slaxemulator@7650 672 + * IA-64 16 0
slaxemulator@7650 673 + * ARM 4 0
slaxemulator@7650 674 + * ARM-Thumb 2 2
slaxemulator@7650 675 + * SPARC 4 0
slaxemulator@7650 676 + */
slaxemulator@7650 677 + uint8_t buf[16];
slaxemulator@7650 678 + } temp;
slaxemulator@7650 679 +};
slaxemulator@7650 680 +
slaxemulator@7650 681 +#ifdef XZ_DEC_X86
slaxemulator@7650 682 +/*
slaxemulator@7650 683 + * This is used to test the most significant byte of a memory address
slaxemulator@7650 684 + * in an x86 instruction.
slaxemulator@7650 685 + */
slaxemulator@7650 686 +static inline int bcj_x86_test_msbyte(uint8_t b)
slaxemulator@7650 687 +{
slaxemulator@7650 688 + return b == 0x00 || b == 0xFF;
slaxemulator@7650 689 +}
slaxemulator@7650 690 +
slaxemulator@7650 691 +static size_t bcj_x86(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 692 +{
slaxemulator@7650 693 + static const bool mask_to_allowed_status[8]
slaxemulator@7650 694 + = { true, true, true, false, true, false, false, false };
slaxemulator@7650 695 +
slaxemulator@7650 696 + static const uint8_t mask_to_bit_num[8] = { 0, 1, 2, 2, 3, 3, 3, 3 };
slaxemulator@7650 697 +
slaxemulator@7650 698 + size_t i;
slaxemulator@7650 699 + size_t prev_pos = (size_t)-1;
slaxemulator@7650 700 + uint32_t prev_mask = s->x86_prev_mask;
slaxemulator@7650 701 + uint32_t src;
slaxemulator@7650 702 + uint32_t dest;
slaxemulator@7650 703 + uint32_t j;
slaxemulator@7650 704 + uint8_t b;
slaxemulator@7650 705 +
slaxemulator@7650 706 + if (size <= 4)
slaxemulator@7650 707 + return 0;
slaxemulator@7650 708 +
slaxemulator@7650 709 + size -= 4;
slaxemulator@7650 710 + for (i = 0; i < size; ++i) {
slaxemulator@7650 711 + if ((buf[i] & 0xFE) != 0xE8)
slaxemulator@7650 712 + continue;
slaxemulator@7650 713 +
slaxemulator@7650 714 + prev_pos = i - prev_pos;
slaxemulator@7650 715 + if (prev_pos > 3) {
slaxemulator@7650 716 + prev_mask = 0;
slaxemulator@7650 717 + } else {
slaxemulator@7650 718 + prev_mask = (prev_mask << (prev_pos - 1)) & 7;
slaxemulator@7650 719 + if (prev_mask != 0) {
slaxemulator@7650 720 + b = buf[i + 4 - mask_to_bit_num[prev_mask]];
slaxemulator@7650 721 + if (!mask_to_allowed_status[prev_mask]
slaxemulator@7650 722 + || bcj_x86_test_msbyte(b)) {
slaxemulator@7650 723 + prev_pos = i;
slaxemulator@7650 724 + prev_mask = (prev_mask << 1) | 1;
slaxemulator@7650 725 + continue;
slaxemulator@7650 726 + }
slaxemulator@7650 727 + }
slaxemulator@7650 728 + }
slaxemulator@7650 729 +
slaxemulator@7650 730 + prev_pos = i;
slaxemulator@7650 731 +
slaxemulator@7650 732 + if (bcj_x86_test_msbyte(buf[i + 4])) {
slaxemulator@7650 733 + src = get_unaligned_le32(buf + i + 1);
slaxemulator@7650 734 + while (true) {
slaxemulator@7650 735 + dest = src - (s->pos + (uint32_t)i + 5);
slaxemulator@7650 736 + if (prev_mask == 0)
slaxemulator@7650 737 + break;
slaxemulator@7650 738 +
slaxemulator@7650 739 + j = mask_to_bit_num[prev_mask] * 8;
slaxemulator@7650 740 + b = (uint8_t)(dest >> (24 - j));
slaxemulator@7650 741 + if (!bcj_x86_test_msbyte(b))
slaxemulator@7650 742 + break;
slaxemulator@7650 743 +
slaxemulator@7650 744 + src = dest ^ (((uint32_t)1 << (32 - j)) - 1);
slaxemulator@7650 745 + }
slaxemulator@7650 746 +
slaxemulator@7650 747 + dest &= 0x01FFFFFF;
slaxemulator@7650 748 + dest |= (uint32_t)0 - (dest & 0x01000000);
slaxemulator@7650 749 + put_unaligned_le32(dest, buf + i + 1);
slaxemulator@7650 750 + i += 4;
slaxemulator@7650 751 + } else {
slaxemulator@7650 752 + prev_mask = (prev_mask << 1) | 1;
slaxemulator@7650 753 + }
slaxemulator@7650 754 + }
slaxemulator@7650 755 +
slaxemulator@7650 756 + prev_pos = i - prev_pos;
slaxemulator@7650 757 + s->x86_prev_mask = prev_pos > 3 ? 0 : prev_mask << (prev_pos - 1);
slaxemulator@7650 758 + return i;
slaxemulator@7650 759 +}
slaxemulator@7650 760 +#endif
slaxemulator@7650 761 +
slaxemulator@7650 762 +#ifdef XZ_DEC_POWERPC
slaxemulator@7650 763 +static size_t bcj_powerpc(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 764 +{
slaxemulator@7650 765 + size_t i;
slaxemulator@7650 766 + uint32_t instr;
slaxemulator@7650 767 +
slaxemulator@7650 768 + for (i = 0; i + 4 <= size; i += 4) {
slaxemulator@7650 769 + instr = get_unaligned_be32(buf + i);
slaxemulator@7650 770 + if ((instr & 0xFC000003) == 0x48000001) {
slaxemulator@7650 771 + instr &= 0x03FFFFFC;
slaxemulator@7650 772 + instr -= s->pos + (uint32_t)i;
slaxemulator@7650 773 + instr &= 0x03FFFFFC;
slaxemulator@7650 774 + instr |= 0x48000001;
slaxemulator@7650 775 + put_unaligned_be32(instr, buf + i);
slaxemulator@7650 776 + }
slaxemulator@7650 777 + }
slaxemulator@7650 778 +
slaxemulator@7650 779 + return i;
slaxemulator@7650 780 +}
slaxemulator@7650 781 +#endif
slaxemulator@7650 782 +
slaxemulator@7650 783 +#ifdef XZ_DEC_IA64
slaxemulator@7650 784 +static size_t bcj_ia64(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 785 +{
slaxemulator@7650 786 + static const uint8_t branch_table[32] = {
slaxemulator@7650 787 + 0, 0, 0, 0, 0, 0, 0, 0,
slaxemulator@7650 788 + 0, 0, 0, 0, 0, 0, 0, 0,
slaxemulator@7650 789 + 4, 4, 6, 6, 0, 0, 7, 7,
slaxemulator@7650 790 + 4, 4, 0, 0, 4, 4, 0, 0
slaxemulator@7650 791 + };
slaxemulator@7650 792 +
slaxemulator@7650 793 + /*
slaxemulator@7650 794 + * The local variables take a little bit stack space, but it's less
slaxemulator@7650 795 + * than what LZMA2 decoder takes, so it doesn't make sense to reduce
slaxemulator@7650 796 + * stack usage here without doing that for the LZMA2 decoder too.
slaxemulator@7650 797 + */
slaxemulator@7650 798 +
slaxemulator@7650 799 + /* Loop counters */
slaxemulator@7650 800 + size_t i;
slaxemulator@7650 801 + size_t j;
slaxemulator@7650 802 +
slaxemulator@7650 803 + /* Instruction slot (0, 1, or 2) in the 128-bit instruction word */
slaxemulator@7650 804 + uint32_t slot;
slaxemulator@7650 805 +
slaxemulator@7650 806 + /* Bitwise offset of the instruction indicated by slot */
slaxemulator@7650 807 + uint32_t bit_pos;
slaxemulator@7650 808 +
slaxemulator@7650 809 + /* bit_pos split into byte and bit parts */
slaxemulator@7650 810 + uint32_t byte_pos;
slaxemulator@7650 811 + uint32_t bit_res;
slaxemulator@7650 812 +
slaxemulator@7650 813 + /* Address part of an instruction */
slaxemulator@7650 814 + uint32_t addr;
slaxemulator@7650 815 +
slaxemulator@7650 816 + /* Mask used to detect which instructions to convert */
slaxemulator@7650 817 + uint32_t mask;
slaxemulator@7650 818 +
slaxemulator@7650 819 + /* 41-bit instruction stored somewhere in the lowest 48 bits */
slaxemulator@7650 820 + uint64_t instr;
slaxemulator@7650 821 +
slaxemulator@7650 822 + /* Instruction normalized with bit_res for easier manipulation */
slaxemulator@7650 823 + uint64_t norm;
slaxemulator@7650 824 +
slaxemulator@7650 825 + for (i = 0; i + 16 <= size; i += 16) {
slaxemulator@7650 826 + mask = branch_table[buf[i] & 0x1F];
slaxemulator@7650 827 + for (slot = 0, bit_pos = 5; slot < 3; ++slot, bit_pos += 41) {
slaxemulator@7650 828 + if (((mask >> slot) & 1) == 0)
slaxemulator@7650 829 + continue;
slaxemulator@7650 830 +
slaxemulator@7650 831 + byte_pos = bit_pos >> 3;
slaxemulator@7650 832 + bit_res = bit_pos & 7;
slaxemulator@7650 833 + instr = 0;
slaxemulator@7650 834 + for (j = 0; j < 6; ++j)
slaxemulator@7650 835 + instr |= (uint64_t)(buf[i + j + byte_pos])
slaxemulator@7650 836 + << (8 * j);
slaxemulator@7650 837 +
slaxemulator@7650 838 + norm = instr >> bit_res;
slaxemulator@7650 839 +
slaxemulator@7650 840 + if (((norm >> 37) & 0x0F) == 0x05
slaxemulator@7650 841 + && ((norm >> 9) & 0x07) == 0) {
slaxemulator@7650 842 + addr = (norm >> 13) & 0x0FFFFF;
slaxemulator@7650 843 + addr |= ((uint32_t)(norm >> 36) & 1) << 20;
slaxemulator@7650 844 + addr <<= 4;
slaxemulator@7650 845 + addr -= s->pos + (uint32_t)i;
slaxemulator@7650 846 + addr >>= 4;
slaxemulator@7650 847 +
slaxemulator@7650 848 + norm &= ~((uint64_t)0x8FFFFF << 13);
slaxemulator@7650 849 + norm |= (uint64_t)(addr & 0x0FFFFF) << 13;
slaxemulator@7650 850 + norm |= (uint64_t)(addr & 0x100000)
slaxemulator@7650 851 + << (36 - 20);
slaxemulator@7650 852 +
slaxemulator@7650 853 + instr &= (1 << bit_res) - 1;
slaxemulator@7650 854 + instr |= norm << bit_res;
slaxemulator@7650 855 +
slaxemulator@7650 856 + for (j = 0; j < 6; j++)
slaxemulator@7650 857 + buf[i + j + byte_pos]
slaxemulator@7650 858 + = (uint8_t)(instr >> (8 * j));
slaxemulator@7650 859 + }
slaxemulator@7650 860 + }
slaxemulator@7650 861 + }
slaxemulator@7650 862 +
slaxemulator@7650 863 + return i;
slaxemulator@7650 864 +}
slaxemulator@7650 865 +#endif
slaxemulator@7650 866 +
slaxemulator@7650 867 +#ifdef XZ_DEC_ARM
slaxemulator@7650 868 +static size_t bcj_arm(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 869 +{
slaxemulator@7650 870 + size_t i;
slaxemulator@7650 871 + uint32_t addr;
slaxemulator@7650 872 +
slaxemulator@7650 873 + for (i = 0; i + 4 <= size; i += 4) {
slaxemulator@7650 874 + if (buf[i + 3] == 0xEB) {
slaxemulator@7650 875 + addr = (uint32_t)buf[i] | ((uint32_t)buf[i + 1] << 8)
slaxemulator@7650 876 + | ((uint32_t)buf[i + 2] << 16);
slaxemulator@7650 877 + addr <<= 2;
slaxemulator@7650 878 + addr -= s->pos + (uint32_t)i + 8;
slaxemulator@7650 879 + addr >>= 2;
slaxemulator@7650 880 + buf[i] = (uint8_t)addr;
slaxemulator@7650 881 + buf[i + 1] = (uint8_t)(addr >> 8);
slaxemulator@7650 882 + buf[i + 2] = (uint8_t)(addr >> 16);
slaxemulator@7650 883 + }
slaxemulator@7650 884 + }
slaxemulator@7650 885 +
slaxemulator@7650 886 + return i;
slaxemulator@7650 887 +}
slaxemulator@7650 888 +#endif
slaxemulator@7650 889 +
slaxemulator@7650 890 +#ifdef XZ_DEC_ARMTHUMB
slaxemulator@7650 891 +static size_t bcj_armthumb(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 892 +{
slaxemulator@7650 893 + size_t i;
slaxemulator@7650 894 + uint32_t addr;
slaxemulator@7650 895 +
slaxemulator@7650 896 + for (i = 0; i + 4 <= size; i += 2) {
slaxemulator@7650 897 + if ((buf[i + 1] & 0xF8) == 0xF0
slaxemulator@7650 898 + && (buf[i + 3] & 0xF8) == 0xF8) {
slaxemulator@7650 899 + addr = (((uint32_t)buf[i + 1] & 0x07) << 19)
slaxemulator@7650 900 + | ((uint32_t)buf[i] << 11)
slaxemulator@7650 901 + | (((uint32_t)buf[i + 3] & 0x07) << 8)
slaxemulator@7650 902 + | (uint32_t)buf[i + 2];
slaxemulator@7650 903 + addr <<= 1;
slaxemulator@7650 904 + addr -= s->pos + (uint32_t)i + 4;
slaxemulator@7650 905 + addr >>= 1;
slaxemulator@7650 906 + buf[i + 1] = (uint8_t)(0xF0 | ((addr >> 19) & 0x07));
slaxemulator@7650 907 + buf[i] = (uint8_t)(addr >> 11);
slaxemulator@7650 908 + buf[i + 3] = (uint8_t)(0xF8 | ((addr >> 8) & 0x07));
slaxemulator@7650 909 + buf[i + 2] = (uint8_t)addr;
slaxemulator@7650 910 + i += 2;
slaxemulator@7650 911 + }
slaxemulator@7650 912 + }
slaxemulator@7650 913 +
slaxemulator@7650 914 + return i;
slaxemulator@7650 915 +}
slaxemulator@7650 916 +#endif
slaxemulator@7650 917 +
slaxemulator@7650 918 +#ifdef XZ_DEC_SPARC
slaxemulator@7650 919 +static size_t bcj_sparc(struct xz_dec_bcj *s, uint8_t *buf, size_t size)
slaxemulator@7650 920 +{
slaxemulator@7650 921 + size_t i;
slaxemulator@7650 922 + uint32_t instr;
slaxemulator@7650 923 +
slaxemulator@7650 924 + for (i = 0; i + 4 <= size; i += 4) {
slaxemulator@7650 925 + instr = get_unaligned_be32(buf + i);
slaxemulator@7650 926 + if ((instr >> 22) == 0x100 || (instr >> 22) == 0x1FF) {
slaxemulator@7650 927 + instr <<= 2;
slaxemulator@7650 928 + instr -= s->pos + (uint32_t)i;
slaxemulator@7650 929 + instr >>= 2;
slaxemulator@7650 930 + instr = ((uint32_t)0x40000000 - (instr & 0x400000))
slaxemulator@7650 931 + | 0x40000000 | (instr & 0x3FFFFF);
slaxemulator@7650 932 + put_unaligned_be32(instr, buf + i);
slaxemulator@7650 933 + }
slaxemulator@7650 934 + }
slaxemulator@7650 935 +
slaxemulator@7650 936 + return i;
slaxemulator@7650 937 +}
slaxemulator@7650 938 +#endif
slaxemulator@7650 939 +
slaxemulator@7650 940 +/*
slaxemulator@7650 941 + * Apply the selected BCJ filter. Update *pos and s->pos to match the amount
slaxemulator@7650 942 + * of data that got filtered.
slaxemulator@7650 943 + *
slaxemulator@7650 944 + * NOTE: This is implemented as a switch statement to avoid using function
slaxemulator@7650 945 + * pointers, which could be problematic in the kernel boot code, which must
slaxemulator@7650 946 + * avoid pointers to static data (at least on x86).
slaxemulator@7650 947 + */
slaxemulator@7650 948 +static void bcj_apply(struct xz_dec_bcj *s,
slaxemulator@7650 949 + uint8_t *buf, size_t *pos, size_t size)
slaxemulator@7650 950 +{
slaxemulator@7650 951 + size_t filtered;
slaxemulator@7650 952 +
slaxemulator@7650 953 + buf += *pos;
slaxemulator@7650 954 + size -= *pos;
slaxemulator@7650 955 +
slaxemulator@7650 956 + switch (s->type) {
slaxemulator@7650 957 +#ifdef XZ_DEC_X86
slaxemulator@7650 958 + case BCJ_X86:
slaxemulator@7650 959 + filtered = bcj_x86(s, buf, size);
slaxemulator@7650 960 + break;
slaxemulator@7650 961 +#endif
slaxemulator@7650 962 +#ifdef XZ_DEC_POWERPC
slaxemulator@7650 963 + case BCJ_POWERPC:
slaxemulator@7650 964 + filtered = bcj_powerpc(s, buf, size);
slaxemulator@7650 965 + break;
slaxemulator@7650 966 +#endif
slaxemulator@7650 967 +#ifdef XZ_DEC_IA64
slaxemulator@7650 968 + case BCJ_IA64:
slaxemulator@7650 969 + filtered = bcj_ia64(s, buf, size);
slaxemulator@7650 970 + break;
slaxemulator@7650 971 +#endif
slaxemulator@7650 972 +#ifdef XZ_DEC_ARM
slaxemulator@7650 973 + case BCJ_ARM:
slaxemulator@7650 974 + filtered = bcj_arm(s, buf, size);
slaxemulator@7650 975 + break;
slaxemulator@7650 976 +#endif
slaxemulator@7650 977 +#ifdef XZ_DEC_ARMTHUMB
slaxemulator@7650 978 + case BCJ_ARMTHUMB:
slaxemulator@7650 979 + filtered = bcj_armthumb(s, buf, size);
slaxemulator@7650 980 + break;
slaxemulator@7650 981 +#endif
slaxemulator@7650 982 +#ifdef XZ_DEC_SPARC
slaxemulator@7650 983 + case BCJ_SPARC:
slaxemulator@7650 984 + filtered = bcj_sparc(s, buf, size);
slaxemulator@7650 985 + break;
slaxemulator@7650 986 +#endif
slaxemulator@7650 987 + default:
slaxemulator@7650 988 + /* Never reached but silence compiler warnings. */
slaxemulator@7650 989 + filtered = 0;
slaxemulator@7650 990 + break;
slaxemulator@7650 991 + }
slaxemulator@7650 992 +
slaxemulator@7650 993 + *pos += filtered;
slaxemulator@7650 994 + s->pos += filtered;
slaxemulator@7650 995 +}
slaxemulator@7650 996 +
slaxemulator@7650 997 +/*
slaxemulator@7650 998 + * Flush pending filtered data from temp to the output buffer.
slaxemulator@7650 999 + * Move the remaining mixture of possibly filtered and unfiltered
slaxemulator@7650 1000 + * data to the beginning of temp.
slaxemulator@7650 1001 + */
slaxemulator@7650 1002 +static void bcj_flush(struct xz_dec_bcj *s, struct xz_buf *b)
slaxemulator@7650 1003 +{
slaxemulator@7650 1004 + size_t copy_size;
slaxemulator@7650 1005 +
slaxemulator@7650 1006 + copy_size = min_t(size_t, s->temp.filtered, b->out_size - b->out_pos);
slaxemulator@7650 1007 + memcpy(b->out + b->out_pos, s->temp.buf, copy_size);
slaxemulator@7650 1008 + b->out_pos += copy_size;
slaxemulator@7650 1009 +
slaxemulator@7650 1010 + s->temp.filtered -= copy_size;
slaxemulator@7650 1011 + s->temp.size -= copy_size;
slaxemulator@7650 1012 + memmove(s->temp.buf, s->temp.buf + copy_size, s->temp.size);
slaxemulator@7650 1013 +}
slaxemulator@7650 1014 +
slaxemulator@7650 1015 +/*
slaxemulator@7650 1016 + * The BCJ filter functions are primitive in sense that they process the
slaxemulator@7650 1017 + * data in chunks of 1-16 bytes. To hide this issue, this function does
slaxemulator@7650 1018 + * some buffering.
slaxemulator@7650 1019 + */
slaxemulator@7650 1020 +XZ_EXTERN enum xz_ret xz_dec_bcj_run(struct xz_dec_bcj *s,
slaxemulator@7650 1021 + struct xz_dec_lzma2 *lzma2,
slaxemulator@7650 1022 + struct xz_buf *b)
slaxemulator@7650 1023 +{
slaxemulator@7650 1024 + size_t out_start;
slaxemulator@7650 1025 +
slaxemulator@7650 1026 + /*
slaxemulator@7650 1027 + * Flush pending already filtered data to the output buffer. Return
slaxemulator@7650 1028 + * immediatelly if we couldn't flush everything, or if the next
slaxemulator@7650 1029 + * filter in the chain had already returned XZ_STREAM_END.
slaxemulator@7650 1030 + */
slaxemulator@7650 1031 + if (s->temp.filtered > 0) {
slaxemulator@7650 1032 + bcj_flush(s, b);
slaxemulator@7650 1033 + if (s->temp.filtered > 0)
slaxemulator@7650 1034 + return XZ_OK;
slaxemulator@7650 1035 +
slaxemulator@7650 1036 + if (s->ret == XZ_STREAM_END)
slaxemulator@7650 1037 + return XZ_STREAM_END;
slaxemulator@7650 1038 + }
slaxemulator@7650 1039 +
slaxemulator@7650 1040 + /*
slaxemulator@7650 1041 + * If we have more output space than what is currently pending in
slaxemulator@7650 1042 + * temp, copy the unfiltered data from temp to the output buffer
slaxemulator@7650 1043 + * and try to fill the output buffer by decoding more data from the
slaxemulator@7650 1044 + * next filter in the chain. Apply the BCJ filter on the new data
slaxemulator@7650 1045 + * in the output buffer. If everything cannot be filtered, copy it
slaxemulator@7650 1046 + * to temp and rewind the output buffer position accordingly.
slaxemulator@7650 1047 + */
slaxemulator@7650 1048 + if (s->temp.size < b->out_size - b->out_pos) {
slaxemulator@7650 1049 + out_start = b->out_pos;
slaxemulator@7650 1050 + memcpy(b->out + b->out_pos, s->temp.buf, s->temp.size);
slaxemulator@7650 1051 + b->out_pos += s->temp.size;
slaxemulator@7650 1052 +
slaxemulator@7650 1053 + s->ret = xz_dec_lzma2_run(lzma2, b);
slaxemulator@7650 1054 + if (s->ret != XZ_STREAM_END
slaxemulator@7650 1055 + && (s->ret != XZ_OK || s->single_call))
slaxemulator@7650 1056 + return s->ret;
slaxemulator@7650 1057 +
slaxemulator@7650 1058 + bcj_apply(s, b->out, &out_start, b->out_pos);
slaxemulator@7650 1059 +
slaxemulator@7650 1060 + /*
slaxemulator@7650 1061 + * As an exception, if the next filter returned XZ_STREAM_END,
slaxemulator@7650 1062 + * we can do that too, since the last few bytes that remain
slaxemulator@7650 1063 + * unfiltered are meant to remain unfiltered.
slaxemulator@7650 1064 + */
slaxemulator@7650 1065 + if (s->ret == XZ_STREAM_END)
slaxemulator@7650 1066 + return XZ_STREAM_END;
slaxemulator@7650 1067 +
slaxemulator@7650 1068 + s->temp.size = b->out_pos - out_start;
slaxemulator@7650 1069 + b->out_pos -= s->temp.size;
slaxemulator@7650 1070 + memcpy(s->temp.buf, b->out + b->out_pos, s->temp.size);
slaxemulator@7650 1071 + }
slaxemulator@7650 1072 +
slaxemulator@7650 1073 + /*
slaxemulator@7650 1074 + * If we have unfiltered data in temp, try to fill by decoding more
slaxemulator@7650 1075 + * data from the next filter. Apply the BCJ filter on temp. Then we
slaxemulator@7650 1076 + * hopefully can fill the actual output buffer by copying filtered
slaxemulator@7650 1077 + * data from temp. A mix of filtered and unfiltered data may be left
slaxemulator@7650 1078 + * in temp; it will be taken care on the next call to this function.
slaxemulator@7650 1079 + */
slaxemulator@7650 1080 + if (s->temp.size > 0) {
slaxemulator@7650 1081 + /* Make b->out{,_pos,_size} temporarily point to s->temp. */
slaxemulator@7650 1082 + s->out = b->out;
slaxemulator@7650 1083 + s->out_pos = b->out_pos;
slaxemulator@7650 1084 + s->out_size = b->out_size;
slaxemulator@7650 1085 + b->out = s->temp.buf;
slaxemulator@7650 1086 + b->out_pos = s->temp.size;
slaxemulator@7650 1087 + b->out_size = sizeof(s->temp.buf);
slaxemulator@7650 1088 +
slaxemulator@7650 1089 + s->ret = xz_dec_lzma2_run(lzma2, b);
slaxemulator@7650 1090 +
slaxemulator@7650 1091 + s->temp.size = b->out_pos;
slaxemulator@7650 1092 + b->out = s->out;
slaxemulator@7650 1093 + b->out_pos = s->out_pos;
slaxemulator@7650 1094 + b->out_size = s->out_size;
slaxemulator@7650 1095 +
slaxemulator@7650 1096 + if (s->ret != XZ_OK && s->ret != XZ_STREAM_END)
slaxemulator@7650 1097 + return s->ret;
slaxemulator@7650 1098 +
slaxemulator@7650 1099 + bcj_apply(s, s->temp.buf, &s->temp.filtered, s->temp.size);
slaxemulator@7650 1100 +
slaxemulator@7650 1101 + /*
slaxemulator@7650 1102 + * If the next filter returned XZ_STREAM_END, we mark that
slaxemulator@7650 1103 + * everything is filtered, since the last unfiltered bytes
slaxemulator@7650 1104 + * of the stream are meant to be left as is.
slaxemulator@7650 1105 + */
slaxemulator@7650 1106 + if (s->ret == XZ_STREAM_END)
slaxemulator@7650 1107 + s->temp.filtered = s->temp.size;
slaxemulator@7650 1108 +
slaxemulator@7650 1109 + bcj_flush(s, b);
slaxemulator@7650 1110 + if (s->temp.filtered > 0)
slaxemulator@7650 1111 + return XZ_OK;
slaxemulator@7650 1112 + }
slaxemulator@7650 1113 +
slaxemulator@7650 1114 + return s->ret;
slaxemulator@7650 1115 +}
slaxemulator@7650 1116 +
slaxemulator@7650 1117 +XZ_EXTERN struct xz_dec_bcj *xz_dec_bcj_create(bool single_call)
slaxemulator@7650 1118 +{
slaxemulator@7650 1119 + struct xz_dec_bcj *s = kmalloc(sizeof(*s), GFP_KERNEL);
slaxemulator@7650 1120 + if (s != NULL)
slaxemulator@7650 1121 + s->single_call = single_call;
slaxemulator@7650 1122 +
slaxemulator@7650 1123 + return s;
slaxemulator@7650 1124 +}
slaxemulator@7650 1125 +
slaxemulator@7650 1126 +XZ_EXTERN enum xz_ret xz_dec_bcj_reset(struct xz_dec_bcj *s, uint8_t id)
slaxemulator@7650 1127 +{
slaxemulator@7650 1128 + switch (id) {
slaxemulator@7650 1129 +#ifdef XZ_DEC_X86
slaxemulator@7650 1130 + case BCJ_X86:
slaxemulator@7650 1131 +#endif
slaxemulator@7650 1132 +#ifdef XZ_DEC_POWERPC
slaxemulator@7650 1133 + case BCJ_POWERPC:
slaxemulator@7650 1134 +#endif
slaxemulator@7650 1135 +#ifdef XZ_DEC_IA64
slaxemulator@7650 1136 + case BCJ_IA64:
slaxemulator@7650 1137 +#endif
slaxemulator@7650 1138 +#ifdef XZ_DEC_ARM
slaxemulator@7650 1139 + case BCJ_ARM:
slaxemulator@7650 1140 +#endif
slaxemulator@7650 1141 +#ifdef XZ_DEC_ARMTHUMB
slaxemulator@7650 1142 + case BCJ_ARMTHUMB:
slaxemulator@7650 1143 +#endif
slaxemulator@7650 1144 +#ifdef XZ_DEC_SPARC
slaxemulator@7650 1145 + case BCJ_SPARC:
slaxemulator@7650 1146 +#endif
slaxemulator@7650 1147 + break;
slaxemulator@7650 1148 +
slaxemulator@7650 1149 + default:
slaxemulator@7650 1150 + /* Unsupported Filter ID */
slaxemulator@7650 1151 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 1152 + }
slaxemulator@7650 1153 +
slaxemulator@7650 1154 + s->type = id;
slaxemulator@7650 1155 + s->ret = XZ_OK;
slaxemulator@7650 1156 + s->pos = 0;
slaxemulator@7650 1157 + s->x86_prev_mask = 0;
slaxemulator@7650 1158 + s->temp.filtered = 0;
slaxemulator@7650 1159 + s->temp.size = 0;
slaxemulator@7650 1160 +
slaxemulator@7650 1161 + return XZ_OK;
slaxemulator@7650 1162 +}
slaxemulator@7650 1163 +
slaxemulator@7650 1164 +#endif
slaxemulator@7650 1165 diff --git a/lib/xz/xz_dec_lzma2.c b/lib/xz/xz_dec_lzma2.c
slaxemulator@7650 1166 new file mode 100644
slaxemulator@7650 1167 index 0000000..ea5fa4f
slaxemulator@7650 1168 --- /dev/null
slaxemulator@7650 1169 +++ b/lib/xz/xz_dec_lzma2.c
slaxemulator@7650 1170 @@ -0,0 +1,1171 @@
slaxemulator@7650 1171 +/*
slaxemulator@7650 1172 + * LZMA2 decoder
slaxemulator@7650 1173 + *
slaxemulator@7650 1174 + * Authors: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 1175 + * Igor Pavlov <http://7-zip.org/>
slaxemulator@7650 1176 + *
slaxemulator@7650 1177 + * This file has been put into the public domain.
slaxemulator@7650 1178 + * You can do whatever you want with this file.
slaxemulator@7650 1179 + */
slaxemulator@7650 1180 +
slaxemulator@7650 1181 +#include "xz_private.h"
slaxemulator@7650 1182 +#include "xz_lzma2.h"
slaxemulator@7650 1183 +
slaxemulator@7650 1184 +/*
slaxemulator@7650 1185 + * Range decoder initialization eats the first five bytes of each LZMA chunk.
slaxemulator@7650 1186 + */
slaxemulator@7650 1187 +#define RC_INIT_BYTES 5
slaxemulator@7650 1188 +
slaxemulator@7650 1189 +/*
slaxemulator@7650 1190 + * Minimum number of usable input buffer to safely decode one LZMA symbol.
slaxemulator@7650 1191 + * The worst case is that we decode 22 bits using probabilities and 26
slaxemulator@7650 1192 + * direct bits. This may decode at maximum of 20 bytes of input. However,
slaxemulator@7650 1193 + * lzma_main() does an extra normalization before returning, thus we
slaxemulator@7650 1194 + * need to put 21 here.
slaxemulator@7650 1195 + */
slaxemulator@7650 1196 +#define LZMA_IN_REQUIRED 21
slaxemulator@7650 1197 +
slaxemulator@7650 1198 +/*
slaxemulator@7650 1199 + * Dictionary (history buffer)
slaxemulator@7650 1200 + *
slaxemulator@7650 1201 + * These are always true:
slaxemulator@7650 1202 + * start <= pos <= full <= end
slaxemulator@7650 1203 + * pos <= limit <= end
slaxemulator@7650 1204 + *
slaxemulator@7650 1205 + * In multi-call mode, also these are true:
slaxemulator@7650 1206 + * end == size
slaxemulator@7650 1207 + * size <= size_max
slaxemulator@7650 1208 + * allocated <= size
slaxemulator@7650 1209 + *
slaxemulator@7650 1210 + * Most of these variables are size_t to support single-call mode,
slaxemulator@7650 1211 + * in which the dictionary variables address the actual output
slaxemulator@7650 1212 + * buffer directly.
slaxemulator@7650 1213 + */
slaxemulator@7650 1214 +struct dictionary {
slaxemulator@7650 1215 + /* Beginning of the history buffer */
slaxemulator@7650 1216 + uint8_t *buf;
slaxemulator@7650 1217 +
slaxemulator@7650 1218 + /* Old position in buf (before decoding more data) */
slaxemulator@7650 1219 + size_t start;
slaxemulator@7650 1220 +
slaxemulator@7650 1221 + /* Position in buf */
slaxemulator@7650 1222 + size_t pos;
slaxemulator@7650 1223 +
slaxemulator@7650 1224 + /*
slaxemulator@7650 1225 + * How full dictionary is. This is used to detect corrupt input that
slaxemulator@7650 1226 + * would read beyond the beginning of the uncompressed stream.
slaxemulator@7650 1227 + */
slaxemulator@7650 1228 + size_t full;
slaxemulator@7650 1229 +
slaxemulator@7650 1230 + /* Write limit; we don't write to buf[limit] or later bytes. */
slaxemulator@7650 1231 + size_t limit;
slaxemulator@7650 1232 +
slaxemulator@7650 1233 + /*
slaxemulator@7650 1234 + * End of the dictionary buffer. In multi-call mode, this is
slaxemulator@7650 1235 + * the same as the dictionary size. In single-call mode, this
slaxemulator@7650 1236 + * indicates the size of the output buffer.
slaxemulator@7650 1237 + */
slaxemulator@7650 1238 + size_t end;
slaxemulator@7650 1239 +
slaxemulator@7650 1240 + /*
slaxemulator@7650 1241 + * Size of the dictionary as specified in Block Header. This is used
slaxemulator@7650 1242 + * together with "full" to detect corrupt input that would make us
slaxemulator@7650 1243 + * read beyond the beginning of the uncompressed stream.
slaxemulator@7650 1244 + */
slaxemulator@7650 1245 + uint32_t size;
slaxemulator@7650 1246 +
slaxemulator@7650 1247 + /*
slaxemulator@7650 1248 + * Maximum allowed dictionary size in multi-call mode.
slaxemulator@7650 1249 + * This is ignored in single-call mode.
slaxemulator@7650 1250 + */
slaxemulator@7650 1251 + uint32_t size_max;
slaxemulator@7650 1252 +
slaxemulator@7650 1253 + /*
slaxemulator@7650 1254 + * Amount of memory currently allocated for the dictionary.
slaxemulator@7650 1255 + * This is used only with XZ_DYNALLOC. (With XZ_PREALLOC,
slaxemulator@7650 1256 + * size_max is always the same as the allocated size.)
slaxemulator@7650 1257 + */
slaxemulator@7650 1258 + uint32_t allocated;
slaxemulator@7650 1259 +
slaxemulator@7650 1260 + /* Operation mode */
slaxemulator@7650 1261 + enum xz_mode mode;
slaxemulator@7650 1262 +};
slaxemulator@7650 1263 +
slaxemulator@7650 1264 +/* Range decoder */
slaxemulator@7650 1265 +struct rc_dec {
slaxemulator@7650 1266 + uint32_t range;
slaxemulator@7650 1267 + uint32_t code;
slaxemulator@7650 1268 +
slaxemulator@7650 1269 + /*
slaxemulator@7650 1270 + * Number of initializing bytes remaining to be read
slaxemulator@7650 1271 + * by rc_read_init().
slaxemulator@7650 1272 + */
slaxemulator@7650 1273 + uint32_t init_bytes_left;
slaxemulator@7650 1274 +
slaxemulator@7650 1275 + /*
slaxemulator@7650 1276 + * Buffer from which we read our input. It can be either
slaxemulator@7650 1277 + * temp.buf or the caller-provided input buffer.
slaxemulator@7650 1278 + */
slaxemulator@7650 1279 + const uint8_t *in;
slaxemulator@7650 1280 + size_t in_pos;
slaxemulator@7650 1281 + size_t in_limit;
slaxemulator@7650 1282 +};
slaxemulator@7650 1283 +
slaxemulator@7650 1284 +/* Probabilities for a length decoder. */
slaxemulator@7650 1285 +struct lzma_len_dec {
slaxemulator@7650 1286 + /* Probability of match length being at least 10 */
slaxemulator@7650 1287 + uint16_t choice;
slaxemulator@7650 1288 +
slaxemulator@7650 1289 + /* Probability of match length being at least 18 */
slaxemulator@7650 1290 + uint16_t choice2;
slaxemulator@7650 1291 +
slaxemulator@7650 1292 + /* Probabilities for match lengths 2-9 */
slaxemulator@7650 1293 + uint16_t low[POS_STATES_MAX][LEN_LOW_SYMBOLS];
slaxemulator@7650 1294 +
slaxemulator@7650 1295 + /* Probabilities for match lengths 10-17 */
slaxemulator@7650 1296 + uint16_t mid[POS_STATES_MAX][LEN_MID_SYMBOLS];
slaxemulator@7650 1297 +
slaxemulator@7650 1298 + /* Probabilities for match lengths 18-273 */
slaxemulator@7650 1299 + uint16_t high[LEN_HIGH_SYMBOLS];
slaxemulator@7650 1300 +};
slaxemulator@7650 1301 +
slaxemulator@7650 1302 +struct lzma_dec {
slaxemulator@7650 1303 + /* Distances of latest four matches */
slaxemulator@7650 1304 + uint32_t rep0;
slaxemulator@7650 1305 + uint32_t rep1;
slaxemulator@7650 1306 + uint32_t rep2;
slaxemulator@7650 1307 + uint32_t rep3;
slaxemulator@7650 1308 +
slaxemulator@7650 1309 + /* Types of the most recently seen LZMA symbols */
slaxemulator@7650 1310 + enum lzma_state state;
slaxemulator@7650 1311 +
slaxemulator@7650 1312 + /*
slaxemulator@7650 1313 + * Length of a match. This is updated so that dict_repeat can
slaxemulator@7650 1314 + * be called again to finish repeating the whole match.
slaxemulator@7650 1315 + */
slaxemulator@7650 1316 + uint32_t len;
slaxemulator@7650 1317 +
slaxemulator@7650 1318 + /*
slaxemulator@7650 1319 + * LZMA properties or related bit masks (number of literal
slaxemulator@7650 1320 + * context bits, a mask dervied from the number of literal
slaxemulator@7650 1321 + * position bits, and a mask dervied from the number
slaxemulator@7650 1322 + * position bits)
slaxemulator@7650 1323 + */
slaxemulator@7650 1324 + uint32_t lc;
slaxemulator@7650 1325 + uint32_t literal_pos_mask; /* (1 << lp) - 1 */
slaxemulator@7650 1326 + uint32_t pos_mask; /* (1 << pb) - 1 */
slaxemulator@7650 1327 +
slaxemulator@7650 1328 + /* If 1, it's a match. Otherwise it's a single 8-bit literal. */
slaxemulator@7650 1329 + uint16_t is_match[STATES][POS_STATES_MAX];
slaxemulator@7650 1330 +
slaxemulator@7650 1331 + /* If 1, it's a repeated match. The distance is one of rep0 .. rep3. */
slaxemulator@7650 1332 + uint16_t is_rep[STATES];
slaxemulator@7650 1333 +
slaxemulator@7650 1334 + /*
slaxemulator@7650 1335 + * If 0, distance of a repeated match is rep0.
slaxemulator@7650 1336 + * Otherwise check is_rep1.
slaxemulator@7650 1337 + */
slaxemulator@7650 1338 + uint16_t is_rep0[STATES];
slaxemulator@7650 1339 +
slaxemulator@7650 1340 + /*
slaxemulator@7650 1341 + * If 0, distance of a repeated match is rep1.
slaxemulator@7650 1342 + * Otherwise check is_rep2.
slaxemulator@7650 1343 + */
slaxemulator@7650 1344 + uint16_t is_rep1[STATES];
slaxemulator@7650 1345 +
slaxemulator@7650 1346 + /* If 0, distance of a repeated match is rep2. Otherwise it is rep3. */
slaxemulator@7650 1347 + uint16_t is_rep2[STATES];
slaxemulator@7650 1348 +
slaxemulator@7650 1349 + /*
slaxemulator@7650 1350 + * If 1, the repeated match has length of one byte. Otherwise
slaxemulator@7650 1351 + * the length is decoded from rep_len_decoder.
slaxemulator@7650 1352 + */
slaxemulator@7650 1353 + uint16_t is_rep0_long[STATES][POS_STATES_MAX];
slaxemulator@7650 1354 +
slaxemulator@7650 1355 + /*
slaxemulator@7650 1356 + * Probability tree for the highest two bits of the match
slaxemulator@7650 1357 + * distance. There is a separate probability tree for match
slaxemulator@7650 1358 + * lengths of 2 (i.e. MATCH_LEN_MIN), 3, 4, and [5, 273].
slaxemulator@7650 1359 + */
slaxemulator@7650 1360 + uint16_t dist_slot[DIST_STATES][DIST_SLOTS];
slaxemulator@7650 1361 +
slaxemulator@7650 1362 + /*
slaxemulator@7650 1363 + * Probility trees for additional bits for match distance
slaxemulator@7650 1364 + * when the distance is in the range [4, 127].
slaxemulator@7650 1365 + */
slaxemulator@7650 1366 + uint16_t dist_special[FULL_DISTANCES - DIST_MODEL_END];
slaxemulator@7650 1367 +
slaxemulator@7650 1368 + /*
slaxemulator@7650 1369 + * Probability tree for the lowest four bits of a match
slaxemulator@7650 1370 + * distance that is equal to or greater than 128.
slaxemulator@7650 1371 + */
slaxemulator@7650 1372 + uint16_t dist_align[ALIGN_SIZE];
slaxemulator@7650 1373 +
slaxemulator@7650 1374 + /* Length of a normal match */
slaxemulator@7650 1375 + struct lzma_len_dec match_len_dec;
slaxemulator@7650 1376 +
slaxemulator@7650 1377 + /* Length of a repeated match */
slaxemulator@7650 1378 + struct lzma_len_dec rep_len_dec;
slaxemulator@7650 1379 +
slaxemulator@7650 1380 + /* Probabilities of literals */
slaxemulator@7650 1381 + uint16_t literal[LITERAL_CODERS_MAX][LITERAL_CODER_SIZE];
slaxemulator@7650 1382 +};
slaxemulator@7650 1383 +
slaxemulator@7650 1384 +struct lzma2_dec {
slaxemulator@7650 1385 + /* Position in xz_dec_lzma2_run(). */
slaxemulator@7650 1386 + enum lzma2_seq {
slaxemulator@7650 1387 + SEQ_CONTROL,
slaxemulator@7650 1388 + SEQ_UNCOMPRESSED_1,
slaxemulator@7650 1389 + SEQ_UNCOMPRESSED_2,
slaxemulator@7650 1390 + SEQ_COMPRESSED_0,
slaxemulator@7650 1391 + SEQ_COMPRESSED_1,
slaxemulator@7650 1392 + SEQ_PROPERTIES,
slaxemulator@7650 1393 + SEQ_LZMA_PREPARE,
slaxemulator@7650 1394 + SEQ_LZMA_RUN,
slaxemulator@7650 1395 + SEQ_COPY
slaxemulator@7650 1396 + } sequence;
slaxemulator@7650 1397 +
slaxemulator@7650 1398 + /* Next position after decoding the compressed size of the chunk. */
slaxemulator@7650 1399 + enum lzma2_seq next_sequence;
slaxemulator@7650 1400 +
slaxemulator@7650 1401 + /* Uncompressed size of LZMA chunk (2 MiB at maximum) */
slaxemulator@7650 1402 + uint32_t uncompressed;
slaxemulator@7650 1403 +
slaxemulator@7650 1404 + /*
slaxemulator@7650 1405 + * Compressed size of LZMA chunk or compressed/uncompressed
slaxemulator@7650 1406 + * size of uncompressed chunk (64 KiB at maximum)
slaxemulator@7650 1407 + */
slaxemulator@7650 1408 + uint32_t compressed;
slaxemulator@7650 1409 +
slaxemulator@7650 1410 + /*
slaxemulator@7650 1411 + * True if dictionary reset is needed. This is false before
slaxemulator@7650 1412 + * the first chunk (LZMA or uncompressed).
slaxemulator@7650 1413 + */
slaxemulator@7650 1414 + bool need_dict_reset;
slaxemulator@7650 1415 +
slaxemulator@7650 1416 + /*
slaxemulator@7650 1417 + * True if new LZMA properties are needed. This is false
slaxemulator@7650 1418 + * before the first LZMA chunk.
slaxemulator@7650 1419 + */
slaxemulator@7650 1420 + bool need_props;
slaxemulator@7650 1421 +};
slaxemulator@7650 1422 +
slaxemulator@7650 1423 +struct xz_dec_lzma2 {
slaxemulator@7650 1424 + /*
slaxemulator@7650 1425 + * The order below is important on x86 to reduce code size and
slaxemulator@7650 1426 + * it shouldn't hurt on other platforms. Everything up to and
slaxemulator@7650 1427 + * including lzma.pos_mask are in the first 128 bytes on x86-32,
slaxemulator@7650 1428 + * which allows using smaller instructions to access those
slaxemulator@7650 1429 + * variables. On x86-64, fewer variables fit into the first 128
slaxemulator@7650 1430 + * bytes, but this is still the best order without sacrificing
slaxemulator@7650 1431 + * the readability by splitting the structures.
slaxemulator@7650 1432 + */
slaxemulator@7650 1433 + struct rc_dec rc;
slaxemulator@7650 1434 + struct dictionary dict;
slaxemulator@7650 1435 + struct lzma2_dec lzma2;
slaxemulator@7650 1436 + struct lzma_dec lzma;
slaxemulator@7650 1437 +
slaxemulator@7650 1438 + /*
slaxemulator@7650 1439 + * Temporary buffer which holds small number of input bytes between
slaxemulator@7650 1440 + * decoder calls. See lzma2_lzma() for details.
slaxemulator@7650 1441 + */
slaxemulator@7650 1442 + struct {
slaxemulator@7650 1443 + uint32_t size;
slaxemulator@7650 1444 + uint8_t buf[3 * LZMA_IN_REQUIRED];
slaxemulator@7650 1445 + } temp;
slaxemulator@7650 1446 +};
slaxemulator@7650 1447 +
slaxemulator@7650 1448 +/**************
slaxemulator@7650 1449 + * Dictionary *
slaxemulator@7650 1450 + **************/
slaxemulator@7650 1451 +
slaxemulator@7650 1452 +/*
slaxemulator@7650 1453 + * Reset the dictionary state. When in single-call mode, set up the beginning
slaxemulator@7650 1454 + * of the dictionary to point to the actual output buffer.
slaxemulator@7650 1455 + */
slaxemulator@7650 1456 +static void dict_reset(struct dictionary *dict, struct xz_buf *b)
slaxemulator@7650 1457 +{
slaxemulator@7650 1458 + if (DEC_IS_SINGLE(dict->mode)) {
slaxemulator@7650 1459 + dict->buf = b->out + b->out_pos;
slaxemulator@7650 1460 + dict->end = b->out_size - b->out_pos;
slaxemulator@7650 1461 + }
slaxemulator@7650 1462 +
slaxemulator@7650 1463 + dict->start = 0;
slaxemulator@7650 1464 + dict->pos = 0;
slaxemulator@7650 1465 + dict->limit = 0;
slaxemulator@7650 1466 + dict->full = 0;
slaxemulator@7650 1467 +}
slaxemulator@7650 1468 +
slaxemulator@7650 1469 +/* Set dictionary write limit */
slaxemulator@7650 1470 +static void dict_limit(struct dictionary *dict, size_t out_max)
slaxemulator@7650 1471 +{
slaxemulator@7650 1472 + if (dict->end - dict->pos <= out_max)
slaxemulator@7650 1473 + dict->limit = dict->end;
slaxemulator@7650 1474 + else
slaxemulator@7650 1475 + dict->limit = dict->pos + out_max;
slaxemulator@7650 1476 +}
slaxemulator@7650 1477 +
slaxemulator@7650 1478 +/* Return true if at least one byte can be written into the dictionary. */
slaxemulator@7650 1479 +static inline bool dict_has_space(const struct dictionary *dict)
slaxemulator@7650 1480 +{
slaxemulator@7650 1481 + return dict->pos < dict->limit;
slaxemulator@7650 1482 +}
slaxemulator@7650 1483 +
slaxemulator@7650 1484 +/*
slaxemulator@7650 1485 + * Get a byte from the dictionary at the given distance. The distance is
slaxemulator@7650 1486 + * assumed to valid, or as a special case, zero when the dictionary is
slaxemulator@7650 1487 + * still empty. This special case is needed for single-call decoding to
slaxemulator@7650 1488 + * avoid writing a '\0' to the end of the destination buffer.
slaxemulator@7650 1489 + */
slaxemulator@7650 1490 +static inline uint32_t dict_get(const struct dictionary *dict, uint32_t dist)
slaxemulator@7650 1491 +{
slaxemulator@7650 1492 + size_t offset = dict->pos - dist - 1;
slaxemulator@7650 1493 +
slaxemulator@7650 1494 + if (dist >= dict->pos)
slaxemulator@7650 1495 + offset += dict->end;
slaxemulator@7650 1496 +
slaxemulator@7650 1497 + return dict->full > 0 ? dict->buf[offset] : 0;
slaxemulator@7650 1498 +}
slaxemulator@7650 1499 +
slaxemulator@7650 1500 +/*
slaxemulator@7650 1501 + * Put one byte into the dictionary. It is assumed that there is space for it.
slaxemulator@7650 1502 + */
slaxemulator@7650 1503 +static inline void dict_put(struct dictionary *dict, uint8_t byte)
slaxemulator@7650 1504 +{
slaxemulator@7650 1505 + dict->buf[dict->pos++] = byte;
slaxemulator@7650 1506 +
slaxemulator@7650 1507 + if (dict->full < dict->pos)
slaxemulator@7650 1508 + dict->full = dict->pos;
slaxemulator@7650 1509 +}
slaxemulator@7650 1510 +
slaxemulator@7650 1511 +/*
slaxemulator@7650 1512 + * Repeat given number of bytes from the given distance. If the distance is
slaxemulator@7650 1513 + * invalid, false is returned. On success, true is returned and *len is
slaxemulator@7650 1514 + * updated to indicate how many bytes were left to be repeated.
slaxemulator@7650 1515 + */
slaxemulator@7650 1516 +static bool dict_repeat(struct dictionary *dict, uint32_t *len, uint32_t dist)
slaxemulator@7650 1517 +{
slaxemulator@7650 1518 + size_t back;
slaxemulator@7650 1519 + uint32_t left;
slaxemulator@7650 1520 +
slaxemulator@7650 1521 + if (dist >= dict->full || dist >= dict->size)
slaxemulator@7650 1522 + return false;
slaxemulator@7650 1523 +
slaxemulator@7650 1524 + left = min_t(size_t, dict->limit - dict->pos, *len);
slaxemulator@7650 1525 + *len -= left;
slaxemulator@7650 1526 +
slaxemulator@7650 1527 + back = dict->pos - dist - 1;
slaxemulator@7650 1528 + if (dist >= dict->pos)
slaxemulator@7650 1529 + back += dict->end;
slaxemulator@7650 1530 +
slaxemulator@7650 1531 + do {
slaxemulator@7650 1532 + dict->buf[dict->pos++] = dict->buf[back++];
slaxemulator@7650 1533 + if (back == dict->end)
slaxemulator@7650 1534 + back = 0;
slaxemulator@7650 1535 + } while (--left > 0);
slaxemulator@7650 1536 +
slaxemulator@7650 1537 + if (dict->full < dict->pos)
slaxemulator@7650 1538 + dict->full = dict->pos;
slaxemulator@7650 1539 +
slaxemulator@7650 1540 + return true;
slaxemulator@7650 1541 +}
slaxemulator@7650 1542 +
slaxemulator@7650 1543 +/* Copy uncompressed data as is from input to dictionary and output buffers. */
slaxemulator@7650 1544 +static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b,
slaxemulator@7650 1545 + uint32_t *left)
slaxemulator@7650 1546 +{
slaxemulator@7650 1547 + size_t copy_size;
slaxemulator@7650 1548 +
slaxemulator@7650 1549 + while (*left > 0 && b->in_pos < b->in_size
slaxemulator@7650 1550 + && b->out_pos < b->out_size) {
slaxemulator@7650 1551 + copy_size = min(b->in_size - b->in_pos,
slaxemulator@7650 1552 + b->out_size - b->out_pos);
slaxemulator@7650 1553 + if (copy_size > dict->end - dict->pos)
slaxemulator@7650 1554 + copy_size = dict->end - dict->pos;
slaxemulator@7650 1555 + if (copy_size > *left)
slaxemulator@7650 1556 + copy_size = *left;
slaxemulator@7650 1557 +
slaxemulator@7650 1558 + *left -= copy_size;
slaxemulator@7650 1559 +
slaxemulator@7650 1560 + memcpy(dict->buf + dict->pos, b->in + b->in_pos, copy_size);
slaxemulator@7650 1561 + dict->pos += copy_size;
slaxemulator@7650 1562 +
slaxemulator@7650 1563 + if (dict->full < dict->pos)
slaxemulator@7650 1564 + dict->full = dict->pos;
slaxemulator@7650 1565 +
slaxemulator@7650 1566 + if (DEC_IS_MULTI(dict->mode)) {
slaxemulator@7650 1567 + if (dict->pos == dict->end)
slaxemulator@7650 1568 + dict->pos = 0;
slaxemulator@7650 1569 +
slaxemulator@7650 1570 + memcpy(b->out + b->out_pos, b->in + b->in_pos,
slaxemulator@7650 1571 + copy_size);
slaxemulator@7650 1572 + }
slaxemulator@7650 1573 +
slaxemulator@7650 1574 + dict->start = dict->pos;
slaxemulator@7650 1575 +
slaxemulator@7650 1576 + b->out_pos += copy_size;
slaxemulator@7650 1577 + b->in_pos += copy_size;
slaxemulator@7650 1578 + }
slaxemulator@7650 1579 +}
slaxemulator@7650 1580 +
slaxemulator@7650 1581 +/*
slaxemulator@7650 1582 + * Flush pending data from dictionary to b->out. It is assumed that there is
slaxemulator@7650 1583 + * enough space in b->out. This is guaranteed because caller uses dict_limit()
slaxemulator@7650 1584 + * before decoding data into the dictionary.
slaxemulator@7650 1585 + */
slaxemulator@7650 1586 +static uint32_t dict_flush(struct dictionary *dict, struct xz_buf *b)
slaxemulator@7650 1587 +{
slaxemulator@7650 1588 + size_t copy_size = dict->pos - dict->start;
slaxemulator@7650 1589 +
slaxemulator@7650 1590 + if (DEC_IS_MULTI(dict->mode)) {
slaxemulator@7650 1591 + if (dict->pos == dict->end)
slaxemulator@7650 1592 + dict->pos = 0;
slaxemulator@7650 1593 +
slaxemulator@7650 1594 + memcpy(b->out + b->out_pos, dict->buf + dict->start,
slaxemulator@7650 1595 + copy_size);
slaxemulator@7650 1596 + }
slaxemulator@7650 1597 +
slaxemulator@7650 1598 + dict->start = dict->pos;
slaxemulator@7650 1599 + b->out_pos += copy_size;
slaxemulator@7650 1600 + return copy_size;
slaxemulator@7650 1601 +}
slaxemulator@7650 1602 +
slaxemulator@7650 1603 +/*****************
slaxemulator@7650 1604 + * Range decoder *
slaxemulator@7650 1605 + *****************/
slaxemulator@7650 1606 +
slaxemulator@7650 1607 +/* Reset the range decoder. */
slaxemulator@7650 1608 +static void rc_reset(struct rc_dec *rc)
slaxemulator@7650 1609 +{
slaxemulator@7650 1610 + rc->range = (uint32_t)-1;
slaxemulator@7650 1611 + rc->code = 0;
slaxemulator@7650 1612 + rc->init_bytes_left = RC_INIT_BYTES;
slaxemulator@7650 1613 +}
slaxemulator@7650 1614 +
slaxemulator@7650 1615 +/*
slaxemulator@7650 1616 + * Read the first five initial bytes into rc->code if they haven't been
slaxemulator@7650 1617 + * read already. (Yes, the first byte gets completely ignored.)
slaxemulator@7650 1618 + */
slaxemulator@7650 1619 +static bool rc_read_init(struct rc_dec *rc, struct xz_buf *b)
slaxemulator@7650 1620 +{
slaxemulator@7650 1621 + while (rc->init_bytes_left > 0) {
slaxemulator@7650 1622 + if (b->in_pos == b->in_size)
slaxemulator@7650 1623 + return false;
slaxemulator@7650 1624 +
slaxemulator@7650 1625 + rc->code = (rc->code << 8) + b->in[b->in_pos++];
slaxemulator@7650 1626 + --rc->init_bytes_left;
slaxemulator@7650 1627 + }
slaxemulator@7650 1628 +
slaxemulator@7650 1629 + return true;
slaxemulator@7650 1630 +}
slaxemulator@7650 1631 +
slaxemulator@7650 1632 +/* Return true if there may not be enough input for the next decoding loop. */
slaxemulator@7650 1633 +static inline bool rc_limit_exceeded(const struct rc_dec *rc)
slaxemulator@7650 1634 +{
slaxemulator@7650 1635 + return rc->in_pos > rc->in_limit;
slaxemulator@7650 1636 +}
slaxemulator@7650 1637 +
slaxemulator@7650 1638 +/*
slaxemulator@7650 1639 + * Return true if it is possible (from point of view of range decoder) that
slaxemulator@7650 1640 + * we have reached the end of the LZMA chunk.
slaxemulator@7650 1641 + */
slaxemulator@7650 1642 +static inline bool rc_is_finished(const struct rc_dec *rc)
slaxemulator@7650 1643 +{
slaxemulator@7650 1644 + return rc->code == 0;
slaxemulator@7650 1645 +}
slaxemulator@7650 1646 +
slaxemulator@7650 1647 +/* Read the next input byte if needed. */
slaxemulator@7650 1648 +static __always_inline void rc_normalize(struct rc_dec *rc)
slaxemulator@7650 1649 +{
slaxemulator@7650 1650 + if (rc->range < RC_TOP_VALUE) {
slaxemulator@7650 1651 + rc->range <<= RC_SHIFT_BITS;
slaxemulator@7650 1652 + rc->code = (rc->code << RC_SHIFT_BITS) + rc->in[rc->in_pos++];
slaxemulator@7650 1653 + }
slaxemulator@7650 1654 +}
slaxemulator@7650 1655 +
slaxemulator@7650 1656 +/*
slaxemulator@7650 1657 + * Decode one bit. In some versions, this function has been splitted in three
slaxemulator@7650 1658 + * functions so that the compiler is supposed to be able to more easily avoid
slaxemulator@7650 1659 + * an extra branch. In this particular version of the LZMA decoder, this
slaxemulator@7650 1660 + * doesn't seem to be a good idea (tested with GCC 3.3.6, 3.4.6, and 4.3.3
slaxemulator@7650 1661 + * on x86). Using a non-splitted version results in nicer looking code too.
slaxemulator@7650 1662 + *
slaxemulator@7650 1663 + * NOTE: This must return an int. Do not make it return a bool or the speed
slaxemulator@7650 1664 + * of the code generated by GCC 3.x decreases 10-15 %. (GCC 4.3 doesn't care,
slaxemulator@7650 1665 + * and it generates 10-20 % faster code than GCC 3.x from this file anyway.)
slaxemulator@7650 1666 + */
slaxemulator@7650 1667 +static __always_inline int rc_bit(struct rc_dec *rc, uint16_t *prob)
slaxemulator@7650 1668 +{
slaxemulator@7650 1669 + uint32_t bound;
slaxemulator@7650 1670 + int bit;
slaxemulator@7650 1671 +
slaxemulator@7650 1672 + rc_normalize(rc);
slaxemulator@7650 1673 + bound = (rc->range >> RC_BIT_MODEL_TOTAL_BITS) * *prob;
slaxemulator@7650 1674 + if (rc->code < bound) {
slaxemulator@7650 1675 + rc->range = bound;
slaxemulator@7650 1676 + *prob += (RC_BIT_MODEL_TOTAL - *prob) >> RC_MOVE_BITS;
slaxemulator@7650 1677 + bit = 0;
slaxemulator@7650 1678 + } else {
slaxemulator@7650 1679 + rc->range -= bound;
slaxemulator@7650 1680 + rc->code -= bound;
slaxemulator@7650 1681 + *prob -= *prob >> RC_MOVE_BITS;
slaxemulator@7650 1682 + bit = 1;
slaxemulator@7650 1683 + }
slaxemulator@7650 1684 +
slaxemulator@7650 1685 + return bit;
slaxemulator@7650 1686 +}
slaxemulator@7650 1687 +
slaxemulator@7650 1688 +/* Decode a bittree starting from the most significant bit. */
slaxemulator@7650 1689 +static __always_inline uint32_t rc_bittree(struct rc_dec *rc,
slaxemulator@7650 1690 + uint16_t *probs, uint32_t limit)
slaxemulator@7650 1691 +{
slaxemulator@7650 1692 + uint32_t symbol = 1;
slaxemulator@7650 1693 +
slaxemulator@7650 1694 + do {
slaxemulator@7650 1695 + if (rc_bit(rc, &probs[symbol]))
slaxemulator@7650 1696 + symbol = (symbol << 1) + 1;
slaxemulator@7650 1697 + else
slaxemulator@7650 1698 + symbol <<= 1;
slaxemulator@7650 1699 + } while (symbol < limit);
slaxemulator@7650 1700 +
slaxemulator@7650 1701 + return symbol;
slaxemulator@7650 1702 +}
slaxemulator@7650 1703 +
slaxemulator@7650 1704 +/* Decode a bittree starting from the least significant bit. */
slaxemulator@7650 1705 +static __always_inline void rc_bittree_reverse(struct rc_dec *rc,
slaxemulator@7650 1706 + uint16_t *probs,
slaxemulator@7650 1707 + uint32_t *dest, uint32_t limit)
slaxemulator@7650 1708 +{
slaxemulator@7650 1709 + uint32_t symbol = 1;
slaxemulator@7650 1710 + uint32_t i = 0;
slaxemulator@7650 1711 +
slaxemulator@7650 1712 + do {
slaxemulator@7650 1713 + if (rc_bit(rc, &probs[symbol])) {
slaxemulator@7650 1714 + symbol = (symbol << 1) + 1;
slaxemulator@7650 1715 + *dest += 1 << i;
slaxemulator@7650 1716 + } else {
slaxemulator@7650 1717 + symbol <<= 1;
slaxemulator@7650 1718 + }
slaxemulator@7650 1719 + } while (++i < limit);
slaxemulator@7650 1720 +}
slaxemulator@7650 1721 +
slaxemulator@7650 1722 +/* Decode direct bits (fixed fifty-fifty probability) */
slaxemulator@7650 1723 +static inline void rc_direct(struct rc_dec *rc, uint32_t *dest, uint32_t limit)
slaxemulator@7650 1724 +{
slaxemulator@7650 1725 + uint32_t mask;
slaxemulator@7650 1726 +
slaxemulator@7650 1727 + do {
slaxemulator@7650 1728 + rc_normalize(rc);
slaxemulator@7650 1729 + rc->range >>= 1;
slaxemulator@7650 1730 + rc->code -= rc->range;
slaxemulator@7650 1731 + mask = (uint32_t)0 - (rc->code >> 31);
slaxemulator@7650 1732 + rc->code += rc->range & mask;
slaxemulator@7650 1733 + *dest = (*dest << 1) + (mask + 1);
slaxemulator@7650 1734 + } while (--limit > 0);
slaxemulator@7650 1735 +}
slaxemulator@7650 1736 +
slaxemulator@7650 1737 +/********
slaxemulator@7650 1738 + * LZMA *
slaxemulator@7650 1739 + ********/
slaxemulator@7650 1740 +
slaxemulator@7650 1741 +/* Get pointer to literal coder probability array. */
slaxemulator@7650 1742 +static uint16_t *lzma_literal_probs(struct xz_dec_lzma2 *s)
slaxemulator@7650 1743 +{
slaxemulator@7650 1744 + uint32_t prev_byte = dict_get(&s->dict, 0);
slaxemulator@7650 1745 + uint32_t low = prev_byte >> (8 - s->lzma.lc);
slaxemulator@7650 1746 + uint32_t high = (s->dict.pos & s->lzma.literal_pos_mask) << s->lzma.lc;
slaxemulator@7650 1747 + return s->lzma.literal[low + high];
slaxemulator@7650 1748 +}
slaxemulator@7650 1749 +
slaxemulator@7650 1750 +/* Decode a literal (one 8-bit byte) */
slaxemulator@7650 1751 +static void lzma_literal(struct xz_dec_lzma2 *s)
slaxemulator@7650 1752 +{
slaxemulator@7650 1753 + uint16_t *probs;
slaxemulator@7650 1754 + uint32_t symbol;
slaxemulator@7650 1755 + uint32_t match_byte;
slaxemulator@7650 1756 + uint32_t match_bit;
slaxemulator@7650 1757 + uint32_t offset;
slaxemulator@7650 1758 + uint32_t i;
slaxemulator@7650 1759 +
slaxemulator@7650 1760 + probs = lzma_literal_probs(s);
slaxemulator@7650 1761 +
slaxemulator@7650 1762 + if (lzma_state_is_literal(s->lzma.state)) {
slaxemulator@7650 1763 + symbol = rc_bittree(&s->rc, probs, 0x100);
slaxemulator@7650 1764 + } else {
slaxemulator@7650 1765 + symbol = 1;
slaxemulator@7650 1766 + match_byte = dict_get(&s->dict, s->lzma.rep0) << 1;
slaxemulator@7650 1767 + offset = 0x100;
slaxemulator@7650 1768 +
slaxemulator@7650 1769 + do {
slaxemulator@7650 1770 + match_bit = match_byte & offset;
slaxemulator@7650 1771 + match_byte <<= 1;
slaxemulator@7650 1772 + i = offset + match_bit + symbol;
slaxemulator@7650 1773 +
slaxemulator@7650 1774 + if (rc_bit(&s->rc, &probs[i])) {
slaxemulator@7650 1775 + symbol = (symbol << 1) + 1;
slaxemulator@7650 1776 + offset &= match_bit;
slaxemulator@7650 1777 + } else {
slaxemulator@7650 1778 + symbol <<= 1;
slaxemulator@7650 1779 + offset &= ~match_bit;
slaxemulator@7650 1780 + }
slaxemulator@7650 1781 + } while (symbol < 0x100);
slaxemulator@7650 1782 + }
slaxemulator@7650 1783 +
slaxemulator@7650 1784 + dict_put(&s->dict, (uint8_t)symbol);
slaxemulator@7650 1785 + lzma_state_literal(&s->lzma.state);
slaxemulator@7650 1786 +}
slaxemulator@7650 1787 +
slaxemulator@7650 1788 +/* Decode the length of the match into s->lzma.len. */
slaxemulator@7650 1789 +static void lzma_len(struct xz_dec_lzma2 *s, struct lzma_len_dec *l,
slaxemulator@7650 1790 + uint32_t pos_state)
slaxemulator@7650 1791 +{
slaxemulator@7650 1792 + uint16_t *probs;
slaxemulator@7650 1793 + uint32_t limit;
slaxemulator@7650 1794 +
slaxemulator@7650 1795 + if (!rc_bit(&s->rc, &l->choice)) {
slaxemulator@7650 1796 + probs = l->low[pos_state];
slaxemulator@7650 1797 + limit = LEN_LOW_SYMBOLS;
slaxemulator@7650 1798 + s->lzma.len = MATCH_LEN_MIN;
slaxemulator@7650 1799 + } else {
slaxemulator@7650 1800 + if (!rc_bit(&s->rc, &l->choice2)) {
slaxemulator@7650 1801 + probs = l->mid[pos_state];
slaxemulator@7650 1802 + limit = LEN_MID_SYMBOLS;
slaxemulator@7650 1803 + s->lzma.len = MATCH_LEN_MIN + LEN_LOW_SYMBOLS;
slaxemulator@7650 1804 + } else {
slaxemulator@7650 1805 + probs = l->high;
slaxemulator@7650 1806 + limit = LEN_HIGH_SYMBOLS;
slaxemulator@7650 1807 + s->lzma.len = MATCH_LEN_MIN + LEN_LOW_SYMBOLS
slaxemulator@7650 1808 + + LEN_MID_SYMBOLS;
slaxemulator@7650 1809 + }
slaxemulator@7650 1810 + }
slaxemulator@7650 1811 +
slaxemulator@7650 1812 + s->lzma.len += rc_bittree(&s->rc, probs, limit) - limit;
slaxemulator@7650 1813 +}
slaxemulator@7650 1814 +
slaxemulator@7650 1815 +/* Decode a match. The distance will be stored in s->lzma.rep0. */
slaxemulator@7650 1816 +static void lzma_match(struct xz_dec_lzma2 *s, uint32_t pos_state)
slaxemulator@7650 1817 +{
slaxemulator@7650 1818 + uint16_t *probs;
slaxemulator@7650 1819 + uint32_t dist_slot;
slaxemulator@7650 1820 + uint32_t limit;
slaxemulator@7650 1821 +
slaxemulator@7650 1822 + lzma_state_match(&s->lzma.state);
slaxemulator@7650 1823 +
slaxemulator@7650 1824 + s->lzma.rep3 = s->lzma.rep2;
slaxemulator@7650 1825 + s->lzma.rep2 = s->lzma.rep1;
slaxemulator@7650 1826 + s->lzma.rep1 = s->lzma.rep0;
slaxemulator@7650 1827 +
slaxemulator@7650 1828 + lzma_len(s, &s->lzma.match_len_dec, pos_state);
slaxemulator@7650 1829 +
slaxemulator@7650 1830 + probs = s->lzma.dist_slot[lzma_get_dist_state(s->lzma.len)];
slaxemulator@7650 1831 + dist_slot = rc_bittree(&s->rc, probs, DIST_SLOTS) - DIST_SLOTS;
slaxemulator@7650 1832 +
slaxemulator@7650 1833 + if (dist_slot < DIST_MODEL_START) {
slaxemulator@7650 1834 + s->lzma.rep0 = dist_slot;
slaxemulator@7650 1835 + } else {
slaxemulator@7650 1836 + limit = (dist_slot >> 1) - 1;
slaxemulator@7650 1837 + s->lzma.rep0 = 2 + (dist_slot & 1);
slaxemulator@7650 1838 +
slaxemulator@7650 1839 + if (dist_slot < DIST_MODEL_END) {
slaxemulator@7650 1840 + s->lzma.rep0 <<= limit;
slaxemulator@7650 1841 + probs = s->lzma.dist_special + s->lzma.rep0
slaxemulator@7650 1842 + - dist_slot - 1;
slaxemulator@7650 1843 + rc_bittree_reverse(&s->rc, probs,
slaxemulator@7650 1844 + &s->lzma.rep0, limit);
slaxemulator@7650 1845 + } else {
slaxemulator@7650 1846 + rc_direct(&s->rc, &s->lzma.rep0, limit - ALIGN_BITS);
slaxemulator@7650 1847 + s->lzma.rep0 <<= ALIGN_BITS;
slaxemulator@7650 1848 + rc_bittree_reverse(&s->rc, s->lzma.dist_align,
slaxemulator@7650 1849 + &s->lzma.rep0, ALIGN_BITS);
slaxemulator@7650 1850 + }
slaxemulator@7650 1851 + }
slaxemulator@7650 1852 +}
slaxemulator@7650 1853 +
slaxemulator@7650 1854 +/*
slaxemulator@7650 1855 + * Decode a repeated match. The distance is one of the four most recently
slaxemulator@7650 1856 + * seen matches. The distance will be stored in s->lzma.rep0.
slaxemulator@7650 1857 + */
slaxemulator@7650 1858 +static void lzma_rep_match(struct xz_dec_lzma2 *s, uint32_t pos_state)
slaxemulator@7650 1859 +{
slaxemulator@7650 1860 + uint32_t tmp;
slaxemulator@7650 1861 +
slaxemulator@7650 1862 + if (!rc_bit(&s->rc, &s->lzma.is_rep0[s->lzma.state])) {
slaxemulator@7650 1863 + if (!rc_bit(&s->rc, &s->lzma.is_rep0_long[
slaxemulator@7650 1864 + s->lzma.state][pos_state])) {
slaxemulator@7650 1865 + lzma_state_short_rep(&s->lzma.state);
slaxemulator@7650 1866 + s->lzma.len = 1;
slaxemulator@7650 1867 + return;
slaxemulator@7650 1868 + }
slaxemulator@7650 1869 + } else {
slaxemulator@7650 1870 + if (!rc_bit(&s->rc, &s->lzma.is_rep1[s->lzma.state])) {
slaxemulator@7650 1871 + tmp = s->lzma.rep1;
slaxemulator@7650 1872 + } else {
slaxemulator@7650 1873 + if (!rc_bit(&s->rc, &s->lzma.is_rep2[s->lzma.state])) {
slaxemulator@7650 1874 + tmp = s->lzma.rep2;
slaxemulator@7650 1875 + } else {
slaxemulator@7650 1876 + tmp = s->lzma.rep3;
slaxemulator@7650 1877 + s->lzma.rep3 = s->lzma.rep2;
slaxemulator@7650 1878 + }
slaxemulator@7650 1879 +
slaxemulator@7650 1880 + s->lzma.rep2 = s->lzma.rep1;
slaxemulator@7650 1881 + }
slaxemulator@7650 1882 +
slaxemulator@7650 1883 + s->lzma.rep1 = s->lzma.rep0;
slaxemulator@7650 1884 + s->lzma.rep0 = tmp;
slaxemulator@7650 1885 + }
slaxemulator@7650 1886 +
slaxemulator@7650 1887 + lzma_state_long_rep(&s->lzma.state);
slaxemulator@7650 1888 + lzma_len(s, &s->lzma.rep_len_dec, pos_state);
slaxemulator@7650 1889 +}
slaxemulator@7650 1890 +
slaxemulator@7650 1891 +/* LZMA decoder core */
slaxemulator@7650 1892 +static bool lzma_main(struct xz_dec_lzma2 *s)
slaxemulator@7650 1893 +{
slaxemulator@7650 1894 + uint32_t pos_state;
slaxemulator@7650 1895 +
slaxemulator@7650 1896 + /*
slaxemulator@7650 1897 + * If the dictionary was reached during the previous call, try to
slaxemulator@7650 1898 + * finish the possibly pending repeat in the dictionary.
slaxemulator@7650 1899 + */
slaxemulator@7650 1900 + if (dict_has_space(&s->dict) && s->lzma.len > 0)
slaxemulator@7650 1901 + dict_repeat(&s->dict, &s->lzma.len, s->lzma.rep0);
slaxemulator@7650 1902 +
slaxemulator@7650 1903 + /*
slaxemulator@7650 1904 + * Decode more LZMA symbols. One iteration may consume up to
slaxemulator@7650 1905 + * LZMA_IN_REQUIRED - 1 bytes.
slaxemulator@7650 1906 + */
slaxemulator@7650 1907 + while (dict_has_space(&s->dict) && !rc_limit_exceeded(&s->rc)) {
slaxemulator@7650 1908 + pos_state = s->dict.pos & s->lzma.pos_mask;
slaxemulator@7650 1909 +
slaxemulator@7650 1910 + if (!rc_bit(&s->rc, &s->lzma.is_match[
slaxemulator@7650 1911 + s->lzma.state][pos_state])) {
slaxemulator@7650 1912 + lzma_literal(s);
slaxemulator@7650 1913 + } else {
slaxemulator@7650 1914 + if (rc_bit(&s->rc, &s->lzma.is_rep[s->lzma.state]))
slaxemulator@7650 1915 + lzma_rep_match(s, pos_state);
slaxemulator@7650 1916 + else
slaxemulator@7650 1917 + lzma_match(s, pos_state);
slaxemulator@7650 1918 +
slaxemulator@7650 1919 + if (!dict_repeat(&s->dict, &s->lzma.len, s->lzma.rep0))
slaxemulator@7650 1920 + return false;
slaxemulator@7650 1921 + }
slaxemulator@7650 1922 + }
slaxemulator@7650 1923 +
slaxemulator@7650 1924 + /*
slaxemulator@7650 1925 + * Having the range decoder always normalized when we are outside
slaxemulator@7650 1926 + * this function makes it easier to correctly handle end of the chunk.
slaxemulator@7650 1927 + */
slaxemulator@7650 1928 + rc_normalize(&s->rc);
slaxemulator@7650 1929 +
slaxemulator@7650 1930 + return true;
slaxemulator@7650 1931 +}
slaxemulator@7650 1932 +
slaxemulator@7650 1933 +/*
slaxemulator@7650 1934 + * Reset the LZMA decoder and range decoder state. Dictionary is nore reset
slaxemulator@7650 1935 + * here, because LZMA state may be reset without resetting the dictionary.
slaxemulator@7650 1936 + */
slaxemulator@7650 1937 +static void lzma_reset(struct xz_dec_lzma2 *s)
slaxemulator@7650 1938 +{
slaxemulator@7650 1939 + uint16_t *probs;
slaxemulator@7650 1940 + size_t i;
slaxemulator@7650 1941 +
slaxemulator@7650 1942 + s->lzma.state = STATE_LIT_LIT;
slaxemulator@7650 1943 + s->lzma.rep0 = 0;
slaxemulator@7650 1944 + s->lzma.rep1 = 0;
slaxemulator@7650 1945 + s->lzma.rep2 = 0;
slaxemulator@7650 1946 + s->lzma.rep3 = 0;
slaxemulator@7650 1947 +
slaxemulator@7650 1948 + /*
slaxemulator@7650 1949 + * All probabilities are initialized to the same value. This hack
slaxemulator@7650 1950 + * makes the code smaller by avoiding a separate loop for each
slaxemulator@7650 1951 + * probability array.
slaxemulator@7650 1952 + *
slaxemulator@7650 1953 + * This could be optimized so that only that part of literal
slaxemulator@7650 1954 + * probabilities that are actually required. In the common case
slaxemulator@7650 1955 + * we would write 12 KiB less.
slaxemulator@7650 1956 + */
slaxemulator@7650 1957 + probs = s->lzma.is_match[0];
slaxemulator@7650 1958 + for (i = 0; i < PROBS_TOTAL; ++i)
slaxemulator@7650 1959 + probs[i] = RC_BIT_MODEL_TOTAL / 2;
slaxemulator@7650 1960 +
slaxemulator@7650 1961 + rc_reset(&s->rc);
slaxemulator@7650 1962 +}
slaxemulator@7650 1963 +
slaxemulator@7650 1964 +/*
slaxemulator@7650 1965 + * Decode and validate LZMA properties (lc/lp/pb) and calculate the bit masks
slaxemulator@7650 1966 + * from the decoded lp and pb values. On success, the LZMA decoder state is
slaxemulator@7650 1967 + * reset and true is returned.
slaxemulator@7650 1968 + */
slaxemulator@7650 1969 +static bool lzma_props(struct xz_dec_lzma2 *s, uint8_t props)
slaxemulator@7650 1970 +{
slaxemulator@7650 1971 + if (props > (4 * 5 + 4) * 9 + 8)
slaxemulator@7650 1972 + return false;
slaxemulator@7650 1973 +
slaxemulator@7650 1974 + s->lzma.pos_mask = 0;
slaxemulator@7650 1975 + while (props >= 9 * 5) {
slaxemulator@7650 1976 + props -= 9 * 5;
slaxemulator@7650 1977 + ++s->lzma.pos_mask;
slaxemulator@7650 1978 + }
slaxemulator@7650 1979 +
slaxemulator@7650 1980 + s->lzma.pos_mask = (1 << s->lzma.pos_mask) - 1;
slaxemulator@7650 1981 +
slaxemulator@7650 1982 + s->lzma.literal_pos_mask = 0;
slaxemulator@7650 1983 + while (props >= 9) {
slaxemulator@7650 1984 + props -= 9;
slaxemulator@7650 1985 + ++s->lzma.literal_pos_mask;
slaxemulator@7650 1986 + }
slaxemulator@7650 1987 +
slaxemulator@7650 1988 + s->lzma.lc = props;
slaxemulator@7650 1989 +
slaxemulator@7650 1990 + if (s->lzma.lc + s->lzma.literal_pos_mask > 4)
slaxemulator@7650 1991 + return false;
slaxemulator@7650 1992 +
slaxemulator@7650 1993 + s->lzma.literal_pos_mask = (1 << s->lzma.literal_pos_mask) - 1;
slaxemulator@7650 1994 +
slaxemulator@7650 1995 + lzma_reset(s);
slaxemulator@7650 1996 +
slaxemulator@7650 1997 + return true;
slaxemulator@7650 1998 +}
slaxemulator@7650 1999 +
slaxemulator@7650 2000 +/*********
slaxemulator@7650 2001 + * LZMA2 *
slaxemulator@7650 2002 + *********/
slaxemulator@7650 2003 +
slaxemulator@7650 2004 +/*
slaxemulator@7650 2005 + * The LZMA decoder assumes that if the input limit (s->rc.in_limit) hasn't
slaxemulator@7650 2006 + * been exceeded, it is safe to read up to LZMA_IN_REQUIRED bytes. This
slaxemulator@7650 2007 + * wrapper function takes care of making the LZMA decoder's assumption safe.
slaxemulator@7650 2008 + *
slaxemulator@7650 2009 + * As long as there is plenty of input left to be decoded in the current LZMA
slaxemulator@7650 2010 + * chunk, we decode directly from the caller-supplied input buffer until
slaxemulator@7650 2011 + * there's LZMA_IN_REQUIRED bytes left. Those remaining bytes are copied into
slaxemulator@7650 2012 + * s->temp.buf, which (hopefully) gets filled on the next call to this
slaxemulator@7650 2013 + * function. We decode a few bytes from the temporary buffer so that we can
slaxemulator@7650 2014 + * continue decoding from the caller-supplied input buffer again.
slaxemulator@7650 2015 + */
slaxemulator@7650 2016 +static bool lzma2_lzma(struct xz_dec_lzma2 *s, struct xz_buf *b)
slaxemulator@7650 2017 +{
slaxemulator@7650 2018 + size_t in_avail;
slaxemulator@7650 2019 + uint32_t tmp;
slaxemulator@7650 2020 +
slaxemulator@7650 2021 + in_avail = b->in_size - b->in_pos;
slaxemulator@7650 2022 + if (s->temp.size > 0 || s->lzma2.compressed == 0) {
slaxemulator@7650 2023 + tmp = 2 * LZMA_IN_REQUIRED - s->temp.size;
slaxemulator@7650 2024 + if (tmp > s->lzma2.compressed - s->temp.size)
slaxemulator@7650 2025 + tmp = s->lzma2.compressed - s->temp.size;
slaxemulator@7650 2026 + if (tmp > in_avail)
slaxemulator@7650 2027 + tmp = in_avail;
slaxemulator@7650 2028 +
slaxemulator@7650 2029 + memcpy(s->temp.buf + s->temp.size, b->in + b->in_pos, tmp);
slaxemulator@7650 2030 +
slaxemulator@7650 2031 + if (s->temp.size + tmp == s->lzma2.compressed) {
slaxemulator@7650 2032 + memzero(s->temp.buf + s->temp.size + tmp,
slaxemulator@7650 2033 + sizeof(s->temp.buf)
slaxemulator@7650 2034 + - s->temp.size - tmp);
slaxemulator@7650 2035 + s->rc.in_limit = s->temp.size + tmp;
slaxemulator@7650 2036 + } else if (s->temp.size + tmp < LZMA_IN_REQUIRED) {
slaxemulator@7650 2037 + s->temp.size += tmp;
slaxemulator@7650 2038 + b->in_pos += tmp;
slaxemulator@7650 2039 + return true;
slaxemulator@7650 2040 + } else {
slaxemulator@7650 2041 + s->rc.in_limit = s->temp.size + tmp - LZMA_IN_REQUIRED;
slaxemulator@7650 2042 + }
slaxemulator@7650 2043 +
slaxemulator@7650 2044 + s->rc.in = s->temp.buf;
slaxemulator@7650 2045 + s->rc.in_pos = 0;
slaxemulator@7650 2046 +
slaxemulator@7650 2047 + if (!lzma_main(s) || s->rc.in_pos > s->temp.size + tmp)
slaxemulator@7650 2048 + return false;
slaxemulator@7650 2049 +
slaxemulator@7650 2050 + s->lzma2.compressed -= s->rc.in_pos;
slaxemulator@7650 2051 +
slaxemulator@7650 2052 + if (s->rc.in_pos < s->temp.size) {
slaxemulator@7650 2053 + s->temp.size -= s->rc.in_pos;
slaxemulator@7650 2054 + memmove(s->temp.buf, s->temp.buf + s->rc.in_pos,
slaxemulator@7650 2055 + s->temp.size);
slaxemulator@7650 2056 + return true;
slaxemulator@7650 2057 + }
slaxemulator@7650 2058 +
slaxemulator@7650 2059 + b->in_pos += s->rc.in_pos - s->temp.size;
slaxemulator@7650 2060 + s->temp.size = 0;
slaxemulator@7650 2061 + }
slaxemulator@7650 2062 +
slaxemulator@7650 2063 + in_avail = b->in_size - b->in_pos;
slaxemulator@7650 2064 + if (in_avail >= LZMA_IN_REQUIRED) {
slaxemulator@7650 2065 + s->rc.in = b->in;
slaxemulator@7650 2066 + s->rc.in_pos = b->in_pos;
slaxemulator@7650 2067 +
slaxemulator@7650 2068 + if (in_avail >= s->lzma2.compressed + LZMA_IN_REQUIRED)
slaxemulator@7650 2069 + s->rc.in_limit = b->in_pos + s->lzma2.compressed;
slaxemulator@7650 2070 + else
slaxemulator@7650 2071 + s->rc.in_limit = b->in_size - LZMA_IN_REQUIRED;
slaxemulator@7650 2072 +
slaxemulator@7650 2073 + if (!lzma_main(s))
slaxemulator@7650 2074 + return false;
slaxemulator@7650 2075 +
slaxemulator@7650 2076 + in_avail = s->rc.in_pos - b->in_pos;
slaxemulator@7650 2077 + if (in_avail > s->lzma2.compressed)
slaxemulator@7650 2078 + return false;
slaxemulator@7650 2079 +
slaxemulator@7650 2080 + s->lzma2.compressed -= in_avail;
slaxemulator@7650 2081 + b->in_pos = s->rc.in_pos;
slaxemulator@7650 2082 + }
slaxemulator@7650 2083 +
slaxemulator@7650 2084 + in_avail = b->in_size - b->in_pos;
slaxemulator@7650 2085 + if (in_avail < LZMA_IN_REQUIRED) {
slaxemulator@7650 2086 + if (in_avail > s->lzma2.compressed)
slaxemulator@7650 2087 + in_avail = s->lzma2.compressed;
slaxemulator@7650 2088 +
slaxemulator@7650 2089 + memcpy(s->temp.buf, b->in + b->in_pos, in_avail);
slaxemulator@7650 2090 + s->temp.size = in_avail;
slaxemulator@7650 2091 + b->in_pos += in_avail;
slaxemulator@7650 2092 + }
slaxemulator@7650 2093 +
slaxemulator@7650 2094 + return true;
slaxemulator@7650 2095 +}
slaxemulator@7650 2096 +
slaxemulator@7650 2097 +/*
slaxemulator@7650 2098 + * Take care of the LZMA2 control layer, and forward the job of actual LZMA
slaxemulator@7650 2099 + * decoding or copying of uncompressed chunks to other functions.
slaxemulator@7650 2100 + */
slaxemulator@7650 2101 +XZ_EXTERN enum xz_ret xz_dec_lzma2_run(struct xz_dec_lzma2 *s,
slaxemulator@7650 2102 + struct xz_buf *b)
slaxemulator@7650 2103 +{
slaxemulator@7650 2104 + uint32_t tmp;
slaxemulator@7650 2105 +
slaxemulator@7650 2106 + while (b->in_pos < b->in_size || s->lzma2.sequence == SEQ_LZMA_RUN) {
slaxemulator@7650 2107 + switch (s->lzma2.sequence) {
slaxemulator@7650 2108 + case SEQ_CONTROL:
slaxemulator@7650 2109 + /*
slaxemulator@7650 2110 + * LZMA2 control byte
slaxemulator@7650 2111 + *
slaxemulator@7650 2112 + * Exact values:
slaxemulator@7650 2113 + * 0x00 End marker
slaxemulator@7650 2114 + * 0x01 Dictionary reset followed by
slaxemulator@7650 2115 + * an uncompressed chunk
slaxemulator@7650 2116 + * 0x02 Uncompressed chunk (no dictionary reset)
slaxemulator@7650 2117 + *
slaxemulator@7650 2118 + * Highest three bits (s->control & 0xE0):
slaxemulator@7650 2119 + * 0xE0 Dictionary reset, new properties and state
slaxemulator@7650 2120 + * reset, followed by LZMA compressed chunk
slaxemulator@7650 2121 + * 0xC0 New properties and state reset, followed
slaxemulator@7650 2122 + * by LZMA compressed chunk (no dictionary
slaxemulator@7650 2123 + * reset)
slaxemulator@7650 2124 + * 0xA0 State reset using old properties,
slaxemulator@7650 2125 + * followed by LZMA compressed chunk (no
slaxemulator@7650 2126 + * dictionary reset)
slaxemulator@7650 2127 + * 0x80 LZMA chunk (no dictionary or state reset)
slaxemulator@7650 2128 + *
slaxemulator@7650 2129 + * For LZMA compressed chunks, the lowest five bits
slaxemulator@7650 2130 + * (s->control & 1F) are the highest bits of the
slaxemulator@7650 2131 + * uncompressed size (bits 16-20).
slaxemulator@7650 2132 + *
slaxemulator@7650 2133 + * A new LZMA2 stream must begin with a dictionary
slaxemulator@7650 2134 + * reset. The first LZMA chunk must set new
slaxemulator@7650 2135 + * properties and reset the LZMA state.
slaxemulator@7650 2136 + *
slaxemulator@7650 2137 + * Values that don't match anything described above
slaxemulator@7650 2138 + * are invalid and we return XZ_DATA_ERROR.
slaxemulator@7650 2139 + */
slaxemulator@7650 2140 + tmp = b->in[b->in_pos++];
slaxemulator@7650 2141 +
slaxemulator@7650 2142 + if (tmp >= 0xE0 || tmp == 0x01) {
slaxemulator@7650 2143 + s->lzma2.need_props = true;
slaxemulator@7650 2144 + s->lzma2.need_dict_reset = false;
slaxemulator@7650 2145 + dict_reset(&s->dict, b);
slaxemulator@7650 2146 + } else if (s->lzma2.need_dict_reset) {
slaxemulator@7650 2147 + return XZ_DATA_ERROR;
slaxemulator@7650 2148 + }
slaxemulator@7650 2149 +
slaxemulator@7650 2150 + if (tmp >= 0x80) {
slaxemulator@7650 2151 + s->lzma2.uncompressed = (tmp & 0x1F) << 16;
slaxemulator@7650 2152 + s->lzma2.sequence = SEQ_UNCOMPRESSED_1;
slaxemulator@7650 2153 +
slaxemulator@7650 2154 + if (tmp >= 0xC0) {
slaxemulator@7650 2155 + /*
slaxemulator@7650 2156 + * When there are new properties,
slaxemulator@7650 2157 + * state reset is done at
slaxemulator@7650 2158 + * SEQ_PROPERTIES.
slaxemulator@7650 2159 + */
slaxemulator@7650 2160 + s->lzma2.need_props = false;
slaxemulator@7650 2161 + s->lzma2.next_sequence
slaxemulator@7650 2162 + = SEQ_PROPERTIES;
slaxemulator@7650 2163 +
slaxemulator@7650 2164 + } else if (s->lzma2.need_props) {
slaxemulator@7650 2165 + return XZ_DATA_ERROR;
slaxemulator@7650 2166 +
slaxemulator@7650 2167 + } else {
slaxemulator@7650 2168 + s->lzma2.next_sequence
slaxemulator@7650 2169 + = SEQ_LZMA_PREPARE;
slaxemulator@7650 2170 + if (tmp >= 0xA0)
slaxemulator@7650 2171 + lzma_reset(s);
slaxemulator@7650 2172 + }
slaxemulator@7650 2173 + } else {
slaxemulator@7650 2174 + if (tmp == 0x00)
slaxemulator@7650 2175 + return XZ_STREAM_END;
slaxemulator@7650 2176 +
slaxemulator@7650 2177 + if (tmp > 0x02)
slaxemulator@7650 2178 + return XZ_DATA_ERROR;
slaxemulator@7650 2179 +
slaxemulator@7650 2180 + s->lzma2.sequence = SEQ_COMPRESSED_0;
slaxemulator@7650 2181 + s->lzma2.next_sequence = SEQ_COPY;
slaxemulator@7650 2182 + }
slaxemulator@7650 2183 +
slaxemulator@7650 2184 + break;
slaxemulator@7650 2185 +
slaxemulator@7650 2186 + case SEQ_UNCOMPRESSED_1:
slaxemulator@7650 2187 + s->lzma2.uncompressed
slaxemulator@7650 2188 + += (uint32_t)b->in[b->in_pos++] << 8;
slaxemulator@7650 2189 + s->lzma2.sequence = SEQ_UNCOMPRESSED_2;
slaxemulator@7650 2190 + break;
slaxemulator@7650 2191 +
slaxemulator@7650 2192 + case SEQ_UNCOMPRESSED_2:
slaxemulator@7650 2193 + s->lzma2.uncompressed
slaxemulator@7650 2194 + += (uint32_t)b->in[b->in_pos++] + 1;
slaxemulator@7650 2195 + s->lzma2.sequence = SEQ_COMPRESSED_0;
slaxemulator@7650 2196 + break;
slaxemulator@7650 2197 +
slaxemulator@7650 2198 + case SEQ_COMPRESSED_0:
slaxemulator@7650 2199 + s->lzma2.compressed
slaxemulator@7650 2200 + = (uint32_t)b->in[b->in_pos++] << 8;
slaxemulator@7650 2201 + s->lzma2.sequence = SEQ_COMPRESSED_1;
slaxemulator@7650 2202 + break;
slaxemulator@7650 2203 +
slaxemulator@7650 2204 + case SEQ_COMPRESSED_1:
slaxemulator@7650 2205 + s->lzma2.compressed
slaxemulator@7650 2206 + += (uint32_t)b->in[b->in_pos++] + 1;
slaxemulator@7650 2207 + s->lzma2.sequence = s->lzma2.next_sequence;
slaxemulator@7650 2208 + break;
slaxemulator@7650 2209 +
slaxemulator@7650 2210 + case SEQ_PROPERTIES:
slaxemulator@7650 2211 + if (!lzma_props(s, b->in[b->in_pos++]))
slaxemulator@7650 2212 + return XZ_DATA_ERROR;
slaxemulator@7650 2213 +
slaxemulator@7650 2214 + s->lzma2.sequence = SEQ_LZMA_PREPARE;
slaxemulator@7650 2215 +
slaxemulator@7650 2216 + case SEQ_LZMA_PREPARE:
slaxemulator@7650 2217 + if (s->lzma2.compressed < RC_INIT_BYTES)
slaxemulator@7650 2218 + return XZ_DATA_ERROR;
slaxemulator@7650 2219 +
slaxemulator@7650 2220 + if (!rc_read_init(&s->rc, b))
slaxemulator@7650 2221 + return XZ_OK;
slaxemulator@7650 2222 +
slaxemulator@7650 2223 + s->lzma2.compressed -= RC_INIT_BYTES;
slaxemulator@7650 2224 + s->lzma2.sequence = SEQ_LZMA_RUN;
slaxemulator@7650 2225 +
slaxemulator@7650 2226 + case SEQ_LZMA_RUN:
slaxemulator@7650 2227 + /*
slaxemulator@7650 2228 + * Set dictionary limit to indicate how much we want
slaxemulator@7650 2229 + * to be encoded at maximum. Decode new data into the
slaxemulator@7650 2230 + * dictionary. Flush the new data from dictionary to
slaxemulator@7650 2231 + * b->out. Check if we finished decoding this chunk.
slaxemulator@7650 2232 + * In case the dictionary got full but we didn't fill
slaxemulator@7650 2233 + * the output buffer yet, we may run this loop
slaxemulator@7650 2234 + * multiple times without changing s->lzma2.sequence.
slaxemulator@7650 2235 + */
slaxemulator@7650 2236 + dict_limit(&s->dict, min_t(size_t,
slaxemulator@7650 2237 + b->out_size - b->out_pos,
slaxemulator@7650 2238 + s->lzma2.uncompressed));
slaxemulator@7650 2239 + if (!lzma2_lzma(s, b))
slaxemulator@7650 2240 + return XZ_DATA_ERROR;
slaxemulator@7650 2241 +
slaxemulator@7650 2242 + s->lzma2.uncompressed -= dict_flush(&s->dict, b);
slaxemulator@7650 2243 +
slaxemulator@7650 2244 + if (s->lzma2.uncompressed == 0) {
slaxemulator@7650 2245 + if (s->lzma2.compressed > 0 || s->lzma.len > 0
slaxemulator@7650 2246 + || !rc_is_finished(&s->rc))
slaxemulator@7650 2247 + return XZ_DATA_ERROR;
slaxemulator@7650 2248 +
slaxemulator@7650 2249 + rc_reset(&s->rc);
slaxemulator@7650 2250 + s->lzma2.sequence = SEQ_CONTROL;
slaxemulator@7650 2251 +
slaxemulator@7650 2252 + } else if (b->out_pos == b->out_size
slaxemulator@7650 2253 + || (b->in_pos == b->in_size
slaxemulator@7650 2254 + && s->temp.size
slaxemulator@7650 2255 + < s->lzma2.compressed)) {
slaxemulator@7650 2256 + return XZ_OK;
slaxemulator@7650 2257 + }
slaxemulator@7650 2258 +
slaxemulator@7650 2259 + break;
slaxemulator@7650 2260 +
slaxemulator@7650 2261 + case SEQ_COPY:
slaxemulator@7650 2262 + dict_uncompressed(&s->dict, b, &s->lzma2.compressed);
slaxemulator@7650 2263 + if (s->lzma2.compressed > 0)
slaxemulator@7650 2264 + return XZ_OK;
slaxemulator@7650 2265 +
slaxemulator@7650 2266 + s->lzma2.sequence = SEQ_CONTROL;
slaxemulator@7650 2267 + break;
slaxemulator@7650 2268 + }
slaxemulator@7650 2269 + }
slaxemulator@7650 2270 +
slaxemulator@7650 2271 + return XZ_OK;
slaxemulator@7650 2272 +}
slaxemulator@7650 2273 +
slaxemulator@7650 2274 +XZ_EXTERN struct xz_dec_lzma2 *xz_dec_lzma2_create(enum xz_mode mode,
slaxemulator@7650 2275 + uint32_t dict_max)
slaxemulator@7650 2276 +{
slaxemulator@7650 2277 + struct xz_dec_lzma2 *s = kmalloc(sizeof(*s), GFP_KERNEL);
slaxemulator@7650 2278 + if (s == NULL)
slaxemulator@7650 2279 + return NULL;
slaxemulator@7650 2280 +
slaxemulator@7650 2281 + s->dict.mode = mode;
slaxemulator@7650 2282 + s->dict.size_max = dict_max;
slaxemulator@7650 2283 +
slaxemulator@7650 2284 + if (DEC_IS_PREALLOC(mode)) {
slaxemulator@7650 2285 + s->dict.buf = vmalloc(dict_max);
slaxemulator@7650 2286 + if (s->dict.buf == NULL) {
slaxemulator@7650 2287 + kfree(s);
slaxemulator@7650 2288 + return NULL;
slaxemulator@7650 2289 + }
slaxemulator@7650 2290 + } else if (DEC_IS_DYNALLOC(mode)) {
slaxemulator@7650 2291 + s->dict.buf = NULL;
slaxemulator@7650 2292 + s->dict.allocated = 0;
slaxemulator@7650 2293 + }
slaxemulator@7650 2294 +
slaxemulator@7650 2295 + return s;
slaxemulator@7650 2296 +}
slaxemulator@7650 2297 +
slaxemulator@7650 2298 +XZ_EXTERN enum xz_ret xz_dec_lzma2_reset(struct xz_dec_lzma2 *s, uint8_t props)
slaxemulator@7650 2299 +{
slaxemulator@7650 2300 + /* This limits dictionary size to 3 GiB to keep parsing simpler. */
slaxemulator@7650 2301 + if (props > 39)
slaxemulator@7650 2302 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2303 +
slaxemulator@7650 2304 + s->dict.size = 2 + (props & 1);
slaxemulator@7650 2305 + s->dict.size <<= (props >> 1) + 11;
slaxemulator@7650 2306 +
slaxemulator@7650 2307 + if (DEC_IS_MULTI(s->dict.mode)) {
slaxemulator@7650 2308 + if (s->dict.size > s->dict.size_max)
slaxemulator@7650 2309 + return XZ_MEMLIMIT_ERROR;
slaxemulator@7650 2310 +
slaxemulator@7650 2311 + s->dict.end = s->dict.size;
slaxemulator@7650 2312 +
slaxemulator@7650 2313 + if (DEC_IS_DYNALLOC(s->dict.mode)) {
slaxemulator@7650 2314 + if (s->dict.allocated < s->dict.size) {
slaxemulator@7650 2315 + vfree(s->dict.buf);
slaxemulator@7650 2316 + s->dict.buf = vmalloc(s->dict.size);
slaxemulator@7650 2317 + if (s->dict.buf == NULL) {
slaxemulator@7650 2318 + s->dict.allocated = 0;
slaxemulator@7650 2319 + return XZ_MEM_ERROR;
slaxemulator@7650 2320 + }
slaxemulator@7650 2321 + }
slaxemulator@7650 2322 + }
slaxemulator@7650 2323 + }
slaxemulator@7650 2324 +
slaxemulator@7650 2325 + s->lzma.len = 0;
slaxemulator@7650 2326 +
slaxemulator@7650 2327 + s->lzma2.sequence = SEQ_CONTROL;
slaxemulator@7650 2328 + s->lzma2.need_dict_reset = true;
slaxemulator@7650 2329 +
slaxemulator@7650 2330 + s->temp.size = 0;
slaxemulator@7650 2331 +
slaxemulator@7650 2332 + return XZ_OK;
slaxemulator@7650 2333 +}
slaxemulator@7650 2334 +
slaxemulator@7650 2335 +XZ_EXTERN void xz_dec_lzma2_end(struct xz_dec_lzma2 *s)
slaxemulator@7650 2336 +{
slaxemulator@7650 2337 + if (DEC_IS_MULTI(s->dict.mode))
slaxemulator@7650 2338 + vfree(s->dict.buf);
slaxemulator@7650 2339 +
slaxemulator@7650 2340 + kfree(s);
slaxemulator@7650 2341 +}
slaxemulator@7650 2342 diff --git a/lib/xz/xz_dec_stream.c b/lib/xz/xz_dec_stream.c
slaxemulator@7650 2343 new file mode 100644
slaxemulator@7650 2344 index 0000000..ac809b1
slaxemulator@7650 2345 --- /dev/null
slaxemulator@7650 2346 +++ b/lib/xz/xz_dec_stream.c
slaxemulator@7650 2347 @@ -0,0 +1,821 @@
slaxemulator@7650 2348 +/*
slaxemulator@7650 2349 + * .xz Stream decoder
slaxemulator@7650 2350 + *
slaxemulator@7650 2351 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 2352 + *
slaxemulator@7650 2353 + * This file has been put into the public domain.
slaxemulator@7650 2354 + * You can do whatever you want with this file.
slaxemulator@7650 2355 + */
slaxemulator@7650 2356 +
slaxemulator@7650 2357 +#include "xz_private.h"
slaxemulator@7650 2358 +#include "xz_stream.h"
slaxemulator@7650 2359 +
slaxemulator@7650 2360 +/* Hash used to validate the Index field */
slaxemulator@7650 2361 +struct xz_dec_hash {
slaxemulator@7650 2362 + vli_type unpadded;
slaxemulator@7650 2363 + vli_type uncompressed;
slaxemulator@7650 2364 + uint32_t crc32;
slaxemulator@7650 2365 +};
slaxemulator@7650 2366 +
slaxemulator@7650 2367 +struct xz_dec {
slaxemulator@7650 2368 + /* Position in dec_main() */
slaxemulator@7650 2369 + enum {
slaxemulator@7650 2370 + SEQ_STREAM_HEADER,
slaxemulator@7650 2371 + SEQ_BLOCK_START,
slaxemulator@7650 2372 + SEQ_BLOCK_HEADER,
slaxemulator@7650 2373 + SEQ_BLOCK_UNCOMPRESS,
slaxemulator@7650 2374 + SEQ_BLOCK_PADDING,
slaxemulator@7650 2375 + SEQ_BLOCK_CHECK,
slaxemulator@7650 2376 + SEQ_INDEX,
slaxemulator@7650 2377 + SEQ_INDEX_PADDING,
slaxemulator@7650 2378 + SEQ_INDEX_CRC32,
slaxemulator@7650 2379 + SEQ_STREAM_FOOTER
slaxemulator@7650 2380 + } sequence;
slaxemulator@7650 2381 +
slaxemulator@7650 2382 + /* Position in variable-length integers and Check fields */
slaxemulator@7650 2383 + uint32_t pos;
slaxemulator@7650 2384 +
slaxemulator@7650 2385 + /* Variable-length integer decoded by dec_vli() */
slaxemulator@7650 2386 + vli_type vli;
slaxemulator@7650 2387 +
slaxemulator@7650 2388 + /* Saved in_pos and out_pos */
slaxemulator@7650 2389 + size_t in_start;
slaxemulator@7650 2390 + size_t out_start;
slaxemulator@7650 2391 +
slaxemulator@7650 2392 + /* CRC32 value in Block or Index */
slaxemulator@7650 2393 + uint32_t crc32;
slaxemulator@7650 2394 +
slaxemulator@7650 2395 + /* Type of the integrity check calculated from uncompressed data */
slaxemulator@7650 2396 + enum xz_check check_type;
slaxemulator@7650 2397 +
slaxemulator@7650 2398 + /* Operation mode */
slaxemulator@7650 2399 + enum xz_mode mode;
slaxemulator@7650 2400 +
slaxemulator@7650 2401 + /*
slaxemulator@7650 2402 + * True if the next call to xz_dec_run() is allowed to return
slaxemulator@7650 2403 + * XZ_BUF_ERROR.
slaxemulator@7650 2404 + */
slaxemulator@7650 2405 + bool allow_buf_error;
slaxemulator@7650 2406 +
slaxemulator@7650 2407 + /* Information stored in Block Header */
slaxemulator@7650 2408 + struct {
slaxemulator@7650 2409 + /*
slaxemulator@7650 2410 + * Value stored in the Compressed Size field, or
slaxemulator@7650 2411 + * VLI_UNKNOWN if Compressed Size is not present.
slaxemulator@7650 2412 + */
slaxemulator@7650 2413 + vli_type compressed;
slaxemulator@7650 2414 +
slaxemulator@7650 2415 + /*
slaxemulator@7650 2416 + * Value stored in the Uncompressed Size field, or
slaxemulator@7650 2417 + * VLI_UNKNOWN if Uncompressed Size is not present.
slaxemulator@7650 2418 + */
slaxemulator@7650 2419 + vli_type uncompressed;
slaxemulator@7650 2420 +
slaxemulator@7650 2421 + /* Size of the Block Header field */
slaxemulator@7650 2422 + uint32_t size;
slaxemulator@7650 2423 + } block_header;
slaxemulator@7650 2424 +
slaxemulator@7650 2425 + /* Information collected when decoding Blocks */
slaxemulator@7650 2426 + struct {
slaxemulator@7650 2427 + /* Observed compressed size of the current Block */
slaxemulator@7650 2428 + vli_type compressed;
slaxemulator@7650 2429 +
slaxemulator@7650 2430 + /* Observed uncompressed size of the current Block */
slaxemulator@7650 2431 + vli_type uncompressed;
slaxemulator@7650 2432 +
slaxemulator@7650 2433 + /* Number of Blocks decoded so far */
slaxemulator@7650 2434 + vli_type count;
slaxemulator@7650 2435 +
slaxemulator@7650 2436 + /*
slaxemulator@7650 2437 + * Hash calculated from the Block sizes. This is used to
slaxemulator@7650 2438 + * validate the Index field.
slaxemulator@7650 2439 + */
slaxemulator@7650 2440 + struct xz_dec_hash hash;
slaxemulator@7650 2441 + } block;
slaxemulator@7650 2442 +
slaxemulator@7650 2443 + /* Variables needed when verifying the Index field */
slaxemulator@7650 2444 + struct {
slaxemulator@7650 2445 + /* Position in dec_index() */
slaxemulator@7650 2446 + enum {
slaxemulator@7650 2447 + SEQ_INDEX_COUNT,
slaxemulator@7650 2448 + SEQ_INDEX_UNPADDED,
slaxemulator@7650 2449 + SEQ_INDEX_UNCOMPRESSED
slaxemulator@7650 2450 + } sequence;
slaxemulator@7650 2451 +
slaxemulator@7650 2452 + /* Size of the Index in bytes */
slaxemulator@7650 2453 + vli_type size;
slaxemulator@7650 2454 +
slaxemulator@7650 2455 + /* Number of Records (matches block.count in valid files) */
slaxemulator@7650 2456 + vli_type count;
slaxemulator@7650 2457 +
slaxemulator@7650 2458 + /*
slaxemulator@7650 2459 + * Hash calculated from the Records (matches block.hash in
slaxemulator@7650 2460 + * valid files).
slaxemulator@7650 2461 + */
slaxemulator@7650 2462 + struct xz_dec_hash hash;
slaxemulator@7650 2463 + } index;
slaxemulator@7650 2464 +
slaxemulator@7650 2465 + /*
slaxemulator@7650 2466 + * Temporary buffer needed to hold Stream Header, Block Header,
slaxemulator@7650 2467 + * and Stream Footer. The Block Header is the biggest (1 KiB)
slaxemulator@7650 2468 + * so we reserve space according to that. buf[] has to be aligned
slaxemulator@7650 2469 + * to a multiple of four bytes; the size_t variables before it
slaxemulator@7650 2470 + * should guarantee this.
slaxemulator@7650 2471 + */
slaxemulator@7650 2472 + struct {
slaxemulator@7650 2473 + size_t pos;
slaxemulator@7650 2474 + size_t size;
slaxemulator@7650 2475 + uint8_t buf[1024];
slaxemulator@7650 2476 + } temp;
slaxemulator@7650 2477 +
slaxemulator@7650 2478 + struct xz_dec_lzma2 *lzma2;
slaxemulator@7650 2479 +
slaxemulator@7650 2480 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 2481 + struct xz_dec_bcj *bcj;
slaxemulator@7650 2482 + bool bcj_active;
slaxemulator@7650 2483 +#endif
slaxemulator@7650 2484 +};
slaxemulator@7650 2485 +
slaxemulator@7650 2486 +#ifdef XZ_DEC_ANY_CHECK
slaxemulator@7650 2487 +/* Sizes of the Check field with different Check IDs */
slaxemulator@7650 2488 +static const uint8_t check_sizes[16] = {
slaxemulator@7650 2489 + 0,
slaxemulator@7650 2490 + 4, 4, 4,
slaxemulator@7650 2491 + 8, 8, 8,
slaxemulator@7650 2492 + 16, 16, 16,
slaxemulator@7650 2493 + 32, 32, 32,
slaxemulator@7650 2494 + 64, 64, 64
slaxemulator@7650 2495 +};
slaxemulator@7650 2496 +#endif
slaxemulator@7650 2497 +
slaxemulator@7650 2498 +/*
slaxemulator@7650 2499 + * Fill s->temp by copying data starting from b->in[b->in_pos]. Caller
slaxemulator@7650 2500 + * must have set s->temp.pos to indicate how much data we are supposed
slaxemulator@7650 2501 + * to copy into s->temp.buf. Return true once s->temp.pos has reached
slaxemulator@7650 2502 + * s->temp.size.
slaxemulator@7650 2503 + */
slaxemulator@7650 2504 +static bool fill_temp(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2505 +{
slaxemulator@7650 2506 + size_t copy_size = min_t(size_t,
slaxemulator@7650 2507 + b->in_size - b->in_pos, s->temp.size - s->temp.pos);
slaxemulator@7650 2508 +
slaxemulator@7650 2509 + memcpy(s->temp.buf + s->temp.pos, b->in + b->in_pos, copy_size);
slaxemulator@7650 2510 + b->in_pos += copy_size;
slaxemulator@7650 2511 + s->temp.pos += copy_size;
slaxemulator@7650 2512 +
slaxemulator@7650 2513 + if (s->temp.pos == s->temp.size) {
slaxemulator@7650 2514 + s->temp.pos = 0;
slaxemulator@7650 2515 + return true;
slaxemulator@7650 2516 + }
slaxemulator@7650 2517 +
slaxemulator@7650 2518 + return false;
slaxemulator@7650 2519 +}
slaxemulator@7650 2520 +
slaxemulator@7650 2521 +/* Decode a variable-length integer (little-endian base-128 encoding) */
slaxemulator@7650 2522 +static enum xz_ret dec_vli(struct xz_dec *s, const uint8_t *in,
slaxemulator@7650 2523 + size_t *in_pos, size_t in_size)
slaxemulator@7650 2524 +{
slaxemulator@7650 2525 + uint8_t byte;
slaxemulator@7650 2526 +
slaxemulator@7650 2527 + if (s->pos == 0)
slaxemulator@7650 2528 + s->vli = 0;
slaxemulator@7650 2529 +
slaxemulator@7650 2530 + while (*in_pos < in_size) {
slaxemulator@7650 2531 + byte = in[*in_pos];
slaxemulator@7650 2532 + ++*in_pos;
slaxemulator@7650 2533 +
slaxemulator@7650 2534 + s->vli |= (vli_type)(byte & 0x7F) << s->pos;
slaxemulator@7650 2535 +
slaxemulator@7650 2536 + if ((byte & 0x80) == 0) {
slaxemulator@7650 2537 + /* Don't allow non-minimal encodings. */
slaxemulator@7650 2538 + if (byte == 0 && s->pos != 0)
slaxemulator@7650 2539 + return XZ_DATA_ERROR;
slaxemulator@7650 2540 +
slaxemulator@7650 2541 + s->pos = 0;
slaxemulator@7650 2542 + return XZ_STREAM_END;
slaxemulator@7650 2543 + }
slaxemulator@7650 2544 +
slaxemulator@7650 2545 + s->pos += 7;
slaxemulator@7650 2546 + if (s->pos == 7 * VLI_BYTES_MAX)
slaxemulator@7650 2547 + return XZ_DATA_ERROR;
slaxemulator@7650 2548 + }
slaxemulator@7650 2549 +
slaxemulator@7650 2550 + return XZ_OK;
slaxemulator@7650 2551 +}
slaxemulator@7650 2552 +
slaxemulator@7650 2553 +/*
slaxemulator@7650 2554 + * Decode the Compressed Data field from a Block. Update and validate
slaxemulator@7650 2555 + * the observed compressed and uncompressed sizes of the Block so that
slaxemulator@7650 2556 + * they don't exceed the values possibly stored in the Block Header
slaxemulator@7650 2557 + * (validation assumes that no integer overflow occurs, since vli_type
slaxemulator@7650 2558 + * is normally uint64_t). Update the CRC32 if presence of the CRC32
slaxemulator@7650 2559 + * field was indicated in Stream Header.
slaxemulator@7650 2560 + *
slaxemulator@7650 2561 + * Once the decoding is finished, validate that the observed sizes match
slaxemulator@7650 2562 + * the sizes possibly stored in the Block Header. Update the hash and
slaxemulator@7650 2563 + * Block count, which are later used to validate the Index field.
slaxemulator@7650 2564 + */
slaxemulator@7650 2565 +static enum xz_ret dec_block(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2566 +{
slaxemulator@7650 2567 + enum xz_ret ret;
slaxemulator@7650 2568 +
slaxemulator@7650 2569 + s->in_start = b->in_pos;
slaxemulator@7650 2570 + s->out_start = b->out_pos;
slaxemulator@7650 2571 +
slaxemulator@7650 2572 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 2573 + if (s->bcj_active)
slaxemulator@7650 2574 + ret = xz_dec_bcj_run(s->bcj, s->lzma2, b);
slaxemulator@7650 2575 + else
slaxemulator@7650 2576 +#endif
slaxemulator@7650 2577 + ret = xz_dec_lzma2_run(s->lzma2, b);
slaxemulator@7650 2578 +
slaxemulator@7650 2579 + s->block.compressed += b->in_pos - s->in_start;
slaxemulator@7650 2580 + s->block.uncompressed += b->out_pos - s->out_start;
slaxemulator@7650 2581 +
slaxemulator@7650 2582 + /*
slaxemulator@7650 2583 + * There is no need to separately check for VLI_UNKNOWN, since
slaxemulator@7650 2584 + * the observed sizes are always smaller than VLI_UNKNOWN.
slaxemulator@7650 2585 + */
slaxemulator@7650 2586 + if (s->block.compressed > s->block_header.compressed
slaxemulator@7650 2587 + || s->block.uncompressed
slaxemulator@7650 2588 + > s->block_header.uncompressed)
slaxemulator@7650 2589 + return XZ_DATA_ERROR;
slaxemulator@7650 2590 +
slaxemulator@7650 2591 + if (s->check_type == XZ_CHECK_CRC32)
slaxemulator@7650 2592 + s->crc32 = xz_crc32(b->out + s->out_start,
slaxemulator@7650 2593 + b->out_pos - s->out_start, s->crc32);
slaxemulator@7650 2594 +
slaxemulator@7650 2595 + if (ret == XZ_STREAM_END) {
slaxemulator@7650 2596 + if (s->block_header.compressed != VLI_UNKNOWN
slaxemulator@7650 2597 + && s->block_header.compressed
slaxemulator@7650 2598 + != s->block.compressed)
slaxemulator@7650 2599 + return XZ_DATA_ERROR;
slaxemulator@7650 2600 +
slaxemulator@7650 2601 + if (s->block_header.uncompressed != VLI_UNKNOWN
slaxemulator@7650 2602 + && s->block_header.uncompressed
slaxemulator@7650 2603 + != s->block.uncompressed)
slaxemulator@7650 2604 + return XZ_DATA_ERROR;
slaxemulator@7650 2605 +
slaxemulator@7650 2606 + s->block.hash.unpadded += s->block_header.size
slaxemulator@7650 2607 + + s->block.compressed;
slaxemulator@7650 2608 +
slaxemulator@7650 2609 +#ifdef XZ_DEC_ANY_CHECK
slaxemulator@7650 2610 + s->block.hash.unpadded += check_sizes[s->check_type];
slaxemulator@7650 2611 +#else
slaxemulator@7650 2612 + if (s->check_type == XZ_CHECK_CRC32)
slaxemulator@7650 2613 + s->block.hash.unpadded += 4;
slaxemulator@7650 2614 +#endif
slaxemulator@7650 2615 +
slaxemulator@7650 2616 + s->block.hash.uncompressed += s->block.uncompressed;
slaxemulator@7650 2617 + s->block.hash.crc32 = xz_crc32(
slaxemulator@7650 2618 + (const uint8_t *)&s->block.hash,
slaxemulator@7650 2619 + sizeof(s->block.hash), s->block.hash.crc32);
slaxemulator@7650 2620 +
slaxemulator@7650 2621 + ++s->block.count;
slaxemulator@7650 2622 + }
slaxemulator@7650 2623 +
slaxemulator@7650 2624 + return ret;
slaxemulator@7650 2625 +}
slaxemulator@7650 2626 +
slaxemulator@7650 2627 +/* Update the Index size and the CRC32 value. */
slaxemulator@7650 2628 +static void index_update(struct xz_dec *s, const struct xz_buf *b)
slaxemulator@7650 2629 +{
slaxemulator@7650 2630 + size_t in_used = b->in_pos - s->in_start;
slaxemulator@7650 2631 + s->index.size += in_used;
slaxemulator@7650 2632 + s->crc32 = xz_crc32(b->in + s->in_start, in_used, s->crc32);
slaxemulator@7650 2633 +}
slaxemulator@7650 2634 +
slaxemulator@7650 2635 +/*
slaxemulator@7650 2636 + * Decode the Number of Records, Unpadded Size, and Uncompressed Size
slaxemulator@7650 2637 + * fields from the Index field. That is, Index Padding and CRC32 are not
slaxemulator@7650 2638 + * decoded by this function.
slaxemulator@7650 2639 + *
slaxemulator@7650 2640 + * This can return XZ_OK (more input needed), XZ_STREAM_END (everything
slaxemulator@7650 2641 + * successfully decoded), or XZ_DATA_ERROR (input is corrupt).
slaxemulator@7650 2642 + */
slaxemulator@7650 2643 +static enum xz_ret dec_index(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2644 +{
slaxemulator@7650 2645 + enum xz_ret ret;
slaxemulator@7650 2646 +
slaxemulator@7650 2647 + do {
slaxemulator@7650 2648 + ret = dec_vli(s, b->in, &b->in_pos, b->in_size);
slaxemulator@7650 2649 + if (ret != XZ_STREAM_END) {
slaxemulator@7650 2650 + index_update(s, b);
slaxemulator@7650 2651 + return ret;
slaxemulator@7650 2652 + }
slaxemulator@7650 2653 +
slaxemulator@7650 2654 + switch (s->index.sequence) {
slaxemulator@7650 2655 + case SEQ_INDEX_COUNT:
slaxemulator@7650 2656 + s->index.count = s->vli;
slaxemulator@7650 2657 +
slaxemulator@7650 2658 + /*
slaxemulator@7650 2659 + * Validate that the Number of Records field
slaxemulator@7650 2660 + * indicates the same number of Records as
slaxemulator@7650 2661 + * there were Blocks in the Stream.
slaxemulator@7650 2662 + */
slaxemulator@7650 2663 + if (s->index.count != s->block.count)
slaxemulator@7650 2664 + return XZ_DATA_ERROR;
slaxemulator@7650 2665 +
slaxemulator@7650 2666 + s->index.sequence = SEQ_INDEX_UNPADDED;
slaxemulator@7650 2667 + break;
slaxemulator@7650 2668 +
slaxemulator@7650 2669 + case SEQ_INDEX_UNPADDED:
slaxemulator@7650 2670 + s->index.hash.unpadded += s->vli;
slaxemulator@7650 2671 + s->index.sequence = SEQ_INDEX_UNCOMPRESSED;
slaxemulator@7650 2672 + break;
slaxemulator@7650 2673 +
slaxemulator@7650 2674 + case SEQ_INDEX_UNCOMPRESSED:
slaxemulator@7650 2675 + s->index.hash.uncompressed += s->vli;
slaxemulator@7650 2676 + s->index.hash.crc32 = xz_crc32(
slaxemulator@7650 2677 + (const uint8_t *)&s->index.hash,
slaxemulator@7650 2678 + sizeof(s->index.hash),
slaxemulator@7650 2679 + s->index.hash.crc32);
slaxemulator@7650 2680 + --s->index.count;
slaxemulator@7650 2681 + s->index.sequence = SEQ_INDEX_UNPADDED;
slaxemulator@7650 2682 + break;
slaxemulator@7650 2683 + }
slaxemulator@7650 2684 + } while (s->index.count > 0);
slaxemulator@7650 2685 +
slaxemulator@7650 2686 + return XZ_STREAM_END;
slaxemulator@7650 2687 +}
slaxemulator@7650 2688 +
slaxemulator@7650 2689 +/*
slaxemulator@7650 2690 + * Validate that the next four input bytes match the value of s->crc32.
slaxemulator@7650 2691 + * s->pos must be zero when starting to validate the first byte.
slaxemulator@7650 2692 + */
slaxemulator@7650 2693 +static enum xz_ret crc32_validate(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2694 +{
slaxemulator@7650 2695 + do {
slaxemulator@7650 2696 + if (b->in_pos == b->in_size)
slaxemulator@7650 2697 + return XZ_OK;
slaxemulator@7650 2698 +
slaxemulator@7650 2699 + if (((s->crc32 >> s->pos) & 0xFF) != b->in[b->in_pos++])
slaxemulator@7650 2700 + return XZ_DATA_ERROR;
slaxemulator@7650 2701 +
slaxemulator@7650 2702 + s->pos += 8;
slaxemulator@7650 2703 +
slaxemulator@7650 2704 + } while (s->pos < 32);
slaxemulator@7650 2705 +
slaxemulator@7650 2706 + s->crc32 = 0;
slaxemulator@7650 2707 + s->pos = 0;
slaxemulator@7650 2708 +
slaxemulator@7650 2709 + return XZ_STREAM_END;
slaxemulator@7650 2710 +}
slaxemulator@7650 2711 +
slaxemulator@7650 2712 +#ifdef XZ_DEC_ANY_CHECK
slaxemulator@7650 2713 +/*
slaxemulator@7650 2714 + * Skip over the Check field when the Check ID is not supported.
slaxemulator@7650 2715 + * Returns true once the whole Check field has been skipped over.
slaxemulator@7650 2716 + */
slaxemulator@7650 2717 +static bool check_skip(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2718 +{
slaxemulator@7650 2719 + while (s->pos < check_sizes[s->check_type]) {
slaxemulator@7650 2720 + if (b->in_pos == b->in_size)
slaxemulator@7650 2721 + return false;
slaxemulator@7650 2722 +
slaxemulator@7650 2723 + ++b->in_pos;
slaxemulator@7650 2724 + ++s->pos;
slaxemulator@7650 2725 + }
slaxemulator@7650 2726 +
slaxemulator@7650 2727 + s->pos = 0;
slaxemulator@7650 2728 +
slaxemulator@7650 2729 + return true;
slaxemulator@7650 2730 +}
slaxemulator@7650 2731 +#endif
slaxemulator@7650 2732 +
slaxemulator@7650 2733 +/* Decode the Stream Header field (the first 12 bytes of the .xz Stream). */
slaxemulator@7650 2734 +static enum xz_ret dec_stream_header(struct xz_dec *s)
slaxemulator@7650 2735 +{
slaxemulator@7650 2736 + if (!memeq(s->temp.buf, HEADER_MAGIC, HEADER_MAGIC_SIZE))
slaxemulator@7650 2737 + return XZ_FORMAT_ERROR;
slaxemulator@7650 2738 +
slaxemulator@7650 2739 + if (xz_crc32(s->temp.buf + HEADER_MAGIC_SIZE, 2, 0)
slaxemulator@7650 2740 + != get_le32(s->temp.buf + HEADER_MAGIC_SIZE + 2))
slaxemulator@7650 2741 + return XZ_DATA_ERROR;
slaxemulator@7650 2742 +
slaxemulator@7650 2743 + if (s->temp.buf[HEADER_MAGIC_SIZE] != 0)
slaxemulator@7650 2744 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2745 +
slaxemulator@7650 2746 + /*
slaxemulator@7650 2747 + * Of integrity checks, we support only none (Check ID = 0) and
slaxemulator@7650 2748 + * CRC32 (Check ID = 1). However, if XZ_DEC_ANY_CHECK is defined,
slaxemulator@7650 2749 + * we will accept other check types too, but then the check won't
slaxemulator@7650 2750 + * be verified and a warning (XZ_UNSUPPORTED_CHECK) will be given.
slaxemulator@7650 2751 + */
slaxemulator@7650 2752 + s->check_type = s->temp.buf[HEADER_MAGIC_SIZE + 1];
slaxemulator@7650 2753 +
slaxemulator@7650 2754 +#ifdef XZ_DEC_ANY_CHECK
slaxemulator@7650 2755 + if (s->check_type > XZ_CHECK_MAX)
slaxemulator@7650 2756 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2757 +
slaxemulator@7650 2758 + if (s->check_type > XZ_CHECK_CRC32)
slaxemulator@7650 2759 + return XZ_UNSUPPORTED_CHECK;
slaxemulator@7650 2760 +#else
slaxemulator@7650 2761 + if (s->check_type > XZ_CHECK_CRC32)
slaxemulator@7650 2762 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2763 +#endif
slaxemulator@7650 2764 +
slaxemulator@7650 2765 + return XZ_OK;
slaxemulator@7650 2766 +}
slaxemulator@7650 2767 +
slaxemulator@7650 2768 +/* Decode the Stream Footer field (the last 12 bytes of the .xz Stream) */
slaxemulator@7650 2769 +static enum xz_ret dec_stream_footer(struct xz_dec *s)
slaxemulator@7650 2770 +{
slaxemulator@7650 2771 + if (!memeq(s->temp.buf + 10, FOOTER_MAGIC, FOOTER_MAGIC_SIZE))
slaxemulator@7650 2772 + return XZ_DATA_ERROR;
slaxemulator@7650 2773 +
slaxemulator@7650 2774 + if (xz_crc32(s->temp.buf + 4, 6, 0) != get_le32(s->temp.buf))
slaxemulator@7650 2775 + return XZ_DATA_ERROR;
slaxemulator@7650 2776 +
slaxemulator@7650 2777 + /*
slaxemulator@7650 2778 + * Validate Backward Size. Note that we never added the size of the
slaxemulator@7650 2779 + * Index CRC32 field to s->index.size, thus we use s->index.size / 4
slaxemulator@7650 2780 + * instead of s->index.size / 4 - 1.
slaxemulator@7650 2781 + */
slaxemulator@7650 2782 + if ((s->index.size >> 2) != get_le32(s->temp.buf + 4))
slaxemulator@7650 2783 + return XZ_DATA_ERROR;
slaxemulator@7650 2784 +
slaxemulator@7650 2785 + if (s->temp.buf[8] != 0 || s->temp.buf[9] != s->check_type)
slaxemulator@7650 2786 + return XZ_DATA_ERROR;
slaxemulator@7650 2787 +
slaxemulator@7650 2788 + /*
slaxemulator@7650 2789 + * Use XZ_STREAM_END instead of XZ_OK to be more convenient
slaxemulator@7650 2790 + * for the caller.
slaxemulator@7650 2791 + */
slaxemulator@7650 2792 + return XZ_STREAM_END;
slaxemulator@7650 2793 +}
slaxemulator@7650 2794 +
slaxemulator@7650 2795 +/* Decode the Block Header and initialize the filter chain. */
slaxemulator@7650 2796 +static enum xz_ret dec_block_header(struct xz_dec *s)
slaxemulator@7650 2797 +{
slaxemulator@7650 2798 + enum xz_ret ret;
slaxemulator@7650 2799 +
slaxemulator@7650 2800 + /*
slaxemulator@7650 2801 + * Validate the CRC32. We know that the temp buffer is at least
slaxemulator@7650 2802 + * eight bytes so this is safe.
slaxemulator@7650 2803 + */
slaxemulator@7650 2804 + s->temp.size -= 4;
slaxemulator@7650 2805 + if (xz_crc32(s->temp.buf, s->temp.size, 0)
slaxemulator@7650 2806 + != get_le32(s->temp.buf + s->temp.size))
slaxemulator@7650 2807 + return XZ_DATA_ERROR;
slaxemulator@7650 2808 +
slaxemulator@7650 2809 + s->temp.pos = 2;
slaxemulator@7650 2810 +
slaxemulator@7650 2811 + /*
slaxemulator@7650 2812 + * Catch unsupported Block Flags. We support only one or two filters
slaxemulator@7650 2813 + * in the chain, so we catch that with the same test.
slaxemulator@7650 2814 + */
slaxemulator@7650 2815 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 2816 + if (s->temp.buf[1] & 0x3E)
slaxemulator@7650 2817 +#else
slaxemulator@7650 2818 + if (s->temp.buf[1] & 0x3F)
slaxemulator@7650 2819 +#endif
slaxemulator@7650 2820 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2821 +
slaxemulator@7650 2822 + /* Compressed Size */
slaxemulator@7650 2823 + if (s->temp.buf[1] & 0x40) {
slaxemulator@7650 2824 + if (dec_vli(s, s->temp.buf, &s->temp.pos, s->temp.size)
slaxemulator@7650 2825 + != XZ_STREAM_END)
slaxemulator@7650 2826 + return XZ_DATA_ERROR;
slaxemulator@7650 2827 +
slaxemulator@7650 2828 + s->block_header.compressed = s->vli;
slaxemulator@7650 2829 + } else {
slaxemulator@7650 2830 + s->block_header.compressed = VLI_UNKNOWN;
slaxemulator@7650 2831 + }
slaxemulator@7650 2832 +
slaxemulator@7650 2833 + /* Uncompressed Size */
slaxemulator@7650 2834 + if (s->temp.buf[1] & 0x80) {
slaxemulator@7650 2835 + if (dec_vli(s, s->temp.buf, &s->temp.pos, s->temp.size)
slaxemulator@7650 2836 + != XZ_STREAM_END)
slaxemulator@7650 2837 + return XZ_DATA_ERROR;
slaxemulator@7650 2838 +
slaxemulator@7650 2839 + s->block_header.uncompressed = s->vli;
slaxemulator@7650 2840 + } else {
slaxemulator@7650 2841 + s->block_header.uncompressed = VLI_UNKNOWN;
slaxemulator@7650 2842 + }
slaxemulator@7650 2843 +
slaxemulator@7650 2844 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 2845 + /* If there are two filters, the first one must be a BCJ filter. */
slaxemulator@7650 2846 + s->bcj_active = s->temp.buf[1] & 0x01;
slaxemulator@7650 2847 + if (s->bcj_active) {
slaxemulator@7650 2848 + if (s->temp.size - s->temp.pos < 2)
slaxemulator@7650 2849 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2850 +
slaxemulator@7650 2851 + ret = xz_dec_bcj_reset(s->bcj, s->temp.buf[s->temp.pos++]);
slaxemulator@7650 2852 + if (ret != XZ_OK)
slaxemulator@7650 2853 + return ret;
slaxemulator@7650 2854 +
slaxemulator@7650 2855 + /*
slaxemulator@7650 2856 + * We don't support custom start offset,
slaxemulator@7650 2857 + * so Size of Properties must be zero.
slaxemulator@7650 2858 + */
slaxemulator@7650 2859 + if (s->temp.buf[s->temp.pos++] != 0x00)
slaxemulator@7650 2860 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2861 + }
slaxemulator@7650 2862 +#endif
slaxemulator@7650 2863 +
slaxemulator@7650 2864 + /* Valid Filter Flags always take at least two bytes. */
slaxemulator@7650 2865 + if (s->temp.size - s->temp.pos < 2)
slaxemulator@7650 2866 + return XZ_DATA_ERROR;
slaxemulator@7650 2867 +
slaxemulator@7650 2868 + /* Filter ID = LZMA2 */
slaxemulator@7650 2869 + if (s->temp.buf[s->temp.pos++] != 0x21)
slaxemulator@7650 2870 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2871 +
slaxemulator@7650 2872 + /* Size of Properties = 1-byte Filter Properties */
slaxemulator@7650 2873 + if (s->temp.buf[s->temp.pos++] != 0x01)
slaxemulator@7650 2874 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2875 +
slaxemulator@7650 2876 + /* Filter Properties contains LZMA2 dictionary size. */
slaxemulator@7650 2877 + if (s->temp.size - s->temp.pos < 1)
slaxemulator@7650 2878 + return XZ_DATA_ERROR;
slaxemulator@7650 2879 +
slaxemulator@7650 2880 + ret = xz_dec_lzma2_reset(s->lzma2, s->temp.buf[s->temp.pos++]);
slaxemulator@7650 2881 + if (ret != XZ_OK)
slaxemulator@7650 2882 + return ret;
slaxemulator@7650 2883 +
slaxemulator@7650 2884 + /* The rest must be Header Padding. */
slaxemulator@7650 2885 + while (s->temp.pos < s->temp.size)
slaxemulator@7650 2886 + if (s->temp.buf[s->temp.pos++] != 0x00)
slaxemulator@7650 2887 + return XZ_OPTIONS_ERROR;
slaxemulator@7650 2888 +
slaxemulator@7650 2889 + s->temp.pos = 0;
slaxemulator@7650 2890 + s->block.compressed = 0;
slaxemulator@7650 2891 + s->block.uncompressed = 0;
slaxemulator@7650 2892 +
slaxemulator@7650 2893 + return XZ_OK;
slaxemulator@7650 2894 +}
slaxemulator@7650 2895 +
slaxemulator@7650 2896 +static enum xz_ret dec_main(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 2897 +{
slaxemulator@7650 2898 + enum xz_ret ret;
slaxemulator@7650 2899 +
slaxemulator@7650 2900 + /*
slaxemulator@7650 2901 + * Store the start position for the case when we are in the middle
slaxemulator@7650 2902 + * of the Index field.
slaxemulator@7650 2903 + */
slaxemulator@7650 2904 + s->in_start = b->in_pos;
slaxemulator@7650 2905 +
slaxemulator@7650 2906 + while (true) {
slaxemulator@7650 2907 + switch (s->sequence) {
slaxemulator@7650 2908 + case SEQ_STREAM_HEADER:
slaxemulator@7650 2909 + /*
slaxemulator@7650 2910 + * Stream Header is copied to s->temp, and then
slaxemulator@7650 2911 + * decoded from there. This way if the caller
slaxemulator@7650 2912 + * gives us only little input at a time, we can
slaxemulator@7650 2913 + * still keep the Stream Header decoding code
slaxemulator@7650 2914 + * simple. Similar approach is used in many places
slaxemulator@7650 2915 + * in this file.
slaxemulator@7650 2916 + */
slaxemulator@7650 2917 + if (!fill_temp(s, b))
slaxemulator@7650 2918 + return XZ_OK;
slaxemulator@7650 2919 +
slaxemulator@7650 2920 + /*
slaxemulator@7650 2921 + * If dec_stream_header() returns
slaxemulator@7650 2922 + * XZ_UNSUPPORTED_CHECK, it is still possible
slaxemulator@7650 2923 + * to continue decoding if working in multi-call
slaxemulator@7650 2924 + * mode. Thus, update s->sequence before calling
slaxemulator@7650 2925 + * dec_stream_header().
slaxemulator@7650 2926 + */
slaxemulator@7650 2927 + s->sequence = SEQ_BLOCK_START;
slaxemulator@7650 2928 +
slaxemulator@7650 2929 + ret = dec_stream_header(s);
slaxemulator@7650 2930 + if (ret != XZ_OK)
slaxemulator@7650 2931 + return ret;
slaxemulator@7650 2932 +
slaxemulator@7650 2933 + case SEQ_BLOCK_START:
slaxemulator@7650 2934 + /* We need one byte of input to continue. */
slaxemulator@7650 2935 + if (b->in_pos == b->in_size)
slaxemulator@7650 2936 + return XZ_OK;
slaxemulator@7650 2937 +
slaxemulator@7650 2938 + /* See if this is the beginning of the Index field. */
slaxemulator@7650 2939 + if (b->in[b->in_pos] == 0) {
slaxemulator@7650 2940 + s->in_start = b->in_pos++;
slaxemulator@7650 2941 + s->sequence = SEQ_INDEX;
slaxemulator@7650 2942 + break;
slaxemulator@7650 2943 + }
slaxemulator@7650 2944 +
slaxemulator@7650 2945 + /*
slaxemulator@7650 2946 + * Calculate the size of the Block Header and
slaxemulator@7650 2947 + * prepare to decode it.
slaxemulator@7650 2948 + */
slaxemulator@7650 2949 + s->block_header.size
slaxemulator@7650 2950 + = ((uint32_t)b->in[b->in_pos] + 1) * 4;
slaxemulator@7650 2951 +
slaxemulator@7650 2952 + s->temp.size = s->block_header.size;
slaxemulator@7650 2953 + s->temp.pos = 0;
slaxemulator@7650 2954 + s->sequence = SEQ_BLOCK_HEADER;
slaxemulator@7650 2955 +
slaxemulator@7650 2956 + case SEQ_BLOCK_HEADER:
slaxemulator@7650 2957 + if (!fill_temp(s, b))
slaxemulator@7650 2958 + return XZ_OK;
slaxemulator@7650 2959 +
slaxemulator@7650 2960 + ret = dec_block_header(s);
slaxemulator@7650 2961 + if (ret != XZ_OK)
slaxemulator@7650 2962 + return ret;
slaxemulator@7650 2963 +
slaxemulator@7650 2964 + s->sequence = SEQ_BLOCK_UNCOMPRESS;
slaxemulator@7650 2965 +
slaxemulator@7650 2966 + case SEQ_BLOCK_UNCOMPRESS:
slaxemulator@7650 2967 + ret = dec_block(s, b);
slaxemulator@7650 2968 + if (ret != XZ_STREAM_END)
slaxemulator@7650 2969 + return ret;
slaxemulator@7650 2970 +
slaxemulator@7650 2971 + s->sequence = SEQ_BLOCK_PADDING;
slaxemulator@7650 2972 +
slaxemulator@7650 2973 + case SEQ_BLOCK_PADDING:
slaxemulator@7650 2974 + /*
slaxemulator@7650 2975 + * Size of Compressed Data + Block Padding
slaxemulator@7650 2976 + * must be a multiple of four. We don't need
slaxemulator@7650 2977 + * s->block.compressed for anything else
slaxemulator@7650 2978 + * anymore, so we use it here to test the size
slaxemulator@7650 2979 + * of the Block Padding field.
slaxemulator@7650 2980 + */
slaxemulator@7650 2981 + while (s->block.compressed & 3) {
slaxemulator@7650 2982 + if (b->in_pos == b->in_size)
slaxemulator@7650 2983 + return XZ_OK;
slaxemulator@7650 2984 +
slaxemulator@7650 2985 + if (b->in[b->in_pos++] != 0)
slaxemulator@7650 2986 + return XZ_DATA_ERROR;
slaxemulator@7650 2987 +
slaxemulator@7650 2988 + ++s->block.compressed;
slaxemulator@7650 2989 + }
slaxemulator@7650 2990 +
slaxemulator@7650 2991 + s->sequence = SEQ_BLOCK_CHECK;
slaxemulator@7650 2992 +
slaxemulator@7650 2993 + case SEQ_BLOCK_CHECK:
slaxemulator@7650 2994 + if (s->check_type == XZ_CHECK_CRC32) {
slaxemulator@7650 2995 + ret = crc32_validate(s, b);
slaxemulator@7650 2996 + if (ret != XZ_STREAM_END)
slaxemulator@7650 2997 + return ret;
slaxemulator@7650 2998 + }
slaxemulator@7650 2999 +#ifdef XZ_DEC_ANY_CHECK
slaxemulator@7650 3000 + else if (!check_skip(s, b)) {
slaxemulator@7650 3001 + return XZ_OK;
slaxemulator@7650 3002 + }
slaxemulator@7650 3003 +#endif
slaxemulator@7650 3004 +
slaxemulator@7650 3005 + s->sequence = SEQ_BLOCK_START;
slaxemulator@7650 3006 + break;
slaxemulator@7650 3007 +
slaxemulator@7650 3008 + case SEQ_INDEX:
slaxemulator@7650 3009 + ret = dec_index(s, b);
slaxemulator@7650 3010 + if (ret != XZ_STREAM_END)
slaxemulator@7650 3011 + return ret;
slaxemulator@7650 3012 +
slaxemulator@7650 3013 + s->sequence = SEQ_INDEX_PADDING;
slaxemulator@7650 3014 +
slaxemulator@7650 3015 + case SEQ_INDEX_PADDING:
slaxemulator@7650 3016 + while ((s->index.size + (b->in_pos - s->in_start))
slaxemulator@7650 3017 + & 3) {
slaxemulator@7650 3018 + if (b->in_pos == b->in_size) {
slaxemulator@7650 3019 + index_update(s, b);
slaxemulator@7650 3020 + return XZ_OK;
slaxemulator@7650 3021 + }
slaxemulator@7650 3022 +
slaxemulator@7650 3023 + if (b->in[b->in_pos++] != 0)
slaxemulator@7650 3024 + return XZ_DATA_ERROR;
slaxemulator@7650 3025 + }
slaxemulator@7650 3026 +
slaxemulator@7650 3027 + /* Finish the CRC32 value and Index size. */
slaxemulator@7650 3028 + index_update(s, b);
slaxemulator@7650 3029 +
slaxemulator@7650 3030 + /* Compare the hashes to validate the Index field. */
slaxemulator@7650 3031 + if (!memeq(&s->block.hash, &s->index.hash,
slaxemulator@7650 3032 + sizeof(s->block.hash)))
slaxemulator@7650 3033 + return XZ_DATA_ERROR;
slaxemulator@7650 3034 +
slaxemulator@7650 3035 + s->sequence = SEQ_INDEX_CRC32;
slaxemulator@7650 3036 +
slaxemulator@7650 3037 + case SEQ_INDEX_CRC32:
slaxemulator@7650 3038 + ret = crc32_validate(s, b);
slaxemulator@7650 3039 + if (ret != XZ_STREAM_END)
slaxemulator@7650 3040 + return ret;
slaxemulator@7650 3041 +
slaxemulator@7650 3042 + s->temp.size = STREAM_HEADER_SIZE;
slaxemulator@7650 3043 + s->sequence = SEQ_STREAM_FOOTER;
slaxemulator@7650 3044 +
slaxemulator@7650 3045 + case SEQ_STREAM_FOOTER:
slaxemulator@7650 3046 + if (!fill_temp(s, b))
slaxemulator@7650 3047 + return XZ_OK;
slaxemulator@7650 3048 +
slaxemulator@7650 3049 + return dec_stream_footer(s);
slaxemulator@7650 3050 + }
slaxemulator@7650 3051 + }
slaxemulator@7650 3052 +
slaxemulator@7650 3053 + /* Never reached */
slaxemulator@7650 3054 +}
slaxemulator@7650 3055 +
slaxemulator@7650 3056 +/*
slaxemulator@7650 3057 + * xz_dec_run() is a wrapper for dec_main() to handle some special cases in
slaxemulator@7650 3058 + * multi-call and single-call decoding.
slaxemulator@7650 3059 + *
slaxemulator@7650 3060 + * In multi-call mode, we must return XZ_BUF_ERROR when it seems clear that we
slaxemulator@7650 3061 + * are not going to make any progress anymore. This is to prevent the caller
slaxemulator@7650 3062 + * from calling us infinitely when the input file is truncated or otherwise
slaxemulator@7650 3063 + * corrupt. Since zlib-style API allows that the caller fills the input buffer
slaxemulator@7650 3064 + * only when the decoder doesn't produce any new output, we have to be careful
slaxemulator@7650 3065 + * to avoid returning XZ_BUF_ERROR too easily: XZ_BUF_ERROR is returned only
slaxemulator@7650 3066 + * after the second consecutive call to xz_dec_run() that makes no progress.
slaxemulator@7650 3067 + *
slaxemulator@7650 3068 + * In single-call mode, if we couldn't decode everything and no error
slaxemulator@7650 3069 + * occurred, either the input is truncated or the output buffer is too small.
slaxemulator@7650 3070 + * Since we know that the last input byte never produces any output, we know
slaxemulator@7650 3071 + * that if all the input was consumed and decoding wasn't finished, the file
slaxemulator@7650 3072 + * must be corrupt. Otherwise the output buffer has to be too small or the
slaxemulator@7650 3073 + * file is corrupt in a way that decoding it produces too big output.
slaxemulator@7650 3074 + *
slaxemulator@7650 3075 + * If single-call decoding fails, we reset b->in_pos and b->out_pos back to
slaxemulator@7650 3076 + * their original values. This is because with some filter chains there won't
slaxemulator@7650 3077 + * be any valid uncompressed data in the output buffer unless the decoding
slaxemulator@7650 3078 + * actually succeeds (that's the price to pay of using the output buffer as
slaxemulator@7650 3079 + * the workspace).
slaxemulator@7650 3080 + */
slaxemulator@7650 3081 +XZ_EXTERN enum xz_ret xz_dec_run(struct xz_dec *s, struct xz_buf *b)
slaxemulator@7650 3082 +{
slaxemulator@7650 3083 + size_t in_start;
slaxemulator@7650 3084 + size_t out_start;
slaxemulator@7650 3085 + enum xz_ret ret;
slaxemulator@7650 3086 +
slaxemulator@7650 3087 + if (DEC_IS_SINGLE(s->mode))
slaxemulator@7650 3088 + xz_dec_reset(s);
slaxemulator@7650 3089 +
slaxemulator@7650 3090 + in_start = b->in_pos;
slaxemulator@7650 3091 + out_start = b->out_pos;
slaxemulator@7650 3092 + ret = dec_main(s, b);
slaxemulator@7650 3093 +
slaxemulator@7650 3094 + if (DEC_IS_SINGLE(s->mode)) {
slaxemulator@7650 3095 + if (ret == XZ_OK)
slaxemulator@7650 3096 + ret = b->in_pos == b->in_size
slaxemulator@7650 3097 + ? XZ_DATA_ERROR : XZ_BUF_ERROR;
slaxemulator@7650 3098 +
slaxemulator@7650 3099 + if (ret != XZ_STREAM_END) {
slaxemulator@7650 3100 + b->in_pos = in_start;
slaxemulator@7650 3101 + b->out_pos = out_start;
slaxemulator@7650 3102 + }
slaxemulator@7650 3103 +
slaxemulator@7650 3104 + } else if (ret == XZ_OK && in_start == b->in_pos
slaxemulator@7650 3105 + && out_start == b->out_pos) {
slaxemulator@7650 3106 + if (s->allow_buf_error)
slaxemulator@7650 3107 + ret = XZ_BUF_ERROR;
slaxemulator@7650 3108 +
slaxemulator@7650 3109 + s->allow_buf_error = true;
slaxemulator@7650 3110 + } else {
slaxemulator@7650 3111 + s->allow_buf_error = false;
slaxemulator@7650 3112 + }
slaxemulator@7650 3113 +
slaxemulator@7650 3114 + return ret;
slaxemulator@7650 3115 +}
slaxemulator@7650 3116 +
slaxemulator@7650 3117 +XZ_EXTERN struct xz_dec *xz_dec_init(enum xz_mode mode, uint32_t dict_max)
slaxemulator@7650 3118 +{
slaxemulator@7650 3119 + struct xz_dec *s = kmalloc(sizeof(*s), GFP_KERNEL);
slaxemulator@7650 3120 + if (s == NULL)
slaxemulator@7650 3121 + return NULL;
slaxemulator@7650 3122 +
slaxemulator@7650 3123 + s->mode = mode;
slaxemulator@7650 3124 +
slaxemulator@7650 3125 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 3126 + s->bcj = xz_dec_bcj_create(DEC_IS_SINGLE(mode));
slaxemulator@7650 3127 + if (s->bcj == NULL)
slaxemulator@7650 3128 + goto error_bcj;
slaxemulator@7650 3129 +#endif
slaxemulator@7650 3130 +
slaxemulator@7650 3131 + s->lzma2 = xz_dec_lzma2_create(mode, dict_max);
slaxemulator@7650 3132 + if (s->lzma2 == NULL)
slaxemulator@7650 3133 + goto error_lzma2;
slaxemulator@7650 3134 +
slaxemulator@7650 3135 + xz_dec_reset(s);
slaxemulator@7650 3136 + return s;
slaxemulator@7650 3137 +
slaxemulator@7650 3138 +error_lzma2:
slaxemulator@7650 3139 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 3140 + xz_dec_bcj_end(s->bcj);
slaxemulator@7650 3141 +error_bcj:
slaxemulator@7650 3142 +#endif
slaxemulator@7650 3143 + kfree(s);
slaxemulator@7650 3144 + return NULL;
slaxemulator@7650 3145 +}
slaxemulator@7650 3146 +
slaxemulator@7650 3147 +XZ_EXTERN void xz_dec_reset(struct xz_dec *s)
slaxemulator@7650 3148 +{
slaxemulator@7650 3149 + s->sequence = SEQ_STREAM_HEADER;
slaxemulator@7650 3150 + s->allow_buf_error = false;
slaxemulator@7650 3151 + s->pos = 0;
slaxemulator@7650 3152 + s->crc32 = 0;
slaxemulator@7650 3153 + memzero(&s->block, sizeof(s->block));
slaxemulator@7650 3154 + memzero(&s->index, sizeof(s->index));
slaxemulator@7650 3155 + s->temp.pos = 0;
slaxemulator@7650 3156 + s->temp.size = STREAM_HEADER_SIZE;
slaxemulator@7650 3157 +}
slaxemulator@7650 3158 +
slaxemulator@7650 3159 +XZ_EXTERN void xz_dec_end(struct xz_dec *s)
slaxemulator@7650 3160 +{
slaxemulator@7650 3161 + if (s != NULL) {
slaxemulator@7650 3162 + xz_dec_lzma2_end(s->lzma2);
slaxemulator@7650 3163 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 3164 + xz_dec_bcj_end(s->bcj);
slaxemulator@7650 3165 +#endif
slaxemulator@7650 3166 + kfree(s);
slaxemulator@7650 3167 + }
slaxemulator@7650 3168 +}
slaxemulator@7650 3169 diff --git a/lib/xz/xz_dec_syms.c b/lib/xz/xz_dec_syms.c
slaxemulator@7650 3170 new file mode 100644
slaxemulator@7650 3171 index 0000000..32eb3c0
slaxemulator@7650 3172 --- /dev/null
slaxemulator@7650 3173 +++ b/lib/xz/xz_dec_syms.c
slaxemulator@7650 3174 @@ -0,0 +1,26 @@
slaxemulator@7650 3175 +/*
slaxemulator@7650 3176 + * XZ decoder module information
slaxemulator@7650 3177 + *
slaxemulator@7650 3178 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3179 + *
slaxemulator@7650 3180 + * This file has been put into the public domain.
slaxemulator@7650 3181 + * You can do whatever you want with this file.
slaxemulator@7650 3182 + */
slaxemulator@7650 3183 +
slaxemulator@7650 3184 +#include <linux/module.h>
slaxemulator@7650 3185 +#include <linux/xz.h>
slaxemulator@7650 3186 +
slaxemulator@7650 3187 +EXPORT_SYMBOL(xz_dec_init);
slaxemulator@7650 3188 +EXPORT_SYMBOL(xz_dec_reset);
slaxemulator@7650 3189 +EXPORT_SYMBOL(xz_dec_run);
slaxemulator@7650 3190 +EXPORT_SYMBOL(xz_dec_end);
slaxemulator@7650 3191 +
slaxemulator@7650 3192 +MODULE_DESCRIPTION("XZ decompressor");
slaxemulator@7650 3193 +MODULE_VERSION("1.0");
slaxemulator@7650 3194 +MODULE_AUTHOR("Lasse Collin <lasse.collin@tukaani.org> and Igor Pavlov");
slaxemulator@7650 3195 +
slaxemulator@7650 3196 +/*
slaxemulator@7650 3197 + * This code is in the public domain, but in Linux it's simplest to just
slaxemulator@7650 3198 + * say it's GPL and consider the authors as the copyright holders.
slaxemulator@7650 3199 + */
slaxemulator@7650 3200 +MODULE_LICENSE("GPL");
slaxemulator@7650 3201 diff --git a/lib/xz/xz_dec_test.c b/lib/xz/xz_dec_test.c
slaxemulator@7650 3202 new file mode 100644
slaxemulator@7650 3203 index 0000000..da28a19
slaxemulator@7650 3204 --- /dev/null
slaxemulator@7650 3205 +++ b/lib/xz/xz_dec_test.c
slaxemulator@7650 3206 @@ -0,0 +1,220 @@
slaxemulator@7650 3207 +/*
slaxemulator@7650 3208 + * XZ decoder tester
slaxemulator@7650 3209 + *
slaxemulator@7650 3210 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3211 + *
slaxemulator@7650 3212 + * This file has been put into the public domain.
slaxemulator@7650 3213 + * You can do whatever you want with this file.
slaxemulator@7650 3214 + */
slaxemulator@7650 3215 +
slaxemulator@7650 3216 +#include <linux/kernel.h>
slaxemulator@7650 3217 +#include <linux/module.h>
slaxemulator@7650 3218 +#include <linux/fs.h>
slaxemulator@7650 3219 +#include <linux/uaccess.h>
slaxemulator@7650 3220 +#include <linux/crc32.h>
slaxemulator@7650 3221 +#include <linux/xz.h>
slaxemulator@7650 3222 +
slaxemulator@7650 3223 +/* Maximum supported dictionary size */
slaxemulator@7650 3224 +#define DICT_MAX (1 << 20)
slaxemulator@7650 3225 +
slaxemulator@7650 3226 +/* Device name to pass to register_chrdev(). */
slaxemulator@7650 3227 +#define DEVICE_NAME "xz_dec_test"
slaxemulator@7650 3228 +
slaxemulator@7650 3229 +/* Dynamically allocated device major number */
slaxemulator@7650 3230 +static int device_major;
slaxemulator@7650 3231 +
slaxemulator@7650 3232 +/*
slaxemulator@7650 3233 + * We reuse the same decoder state, and thus can decode only one
slaxemulator@7650 3234 + * file at a time.
slaxemulator@7650 3235 + */
slaxemulator@7650 3236 +static bool device_is_open;
slaxemulator@7650 3237 +
slaxemulator@7650 3238 +/* XZ decoder state */
slaxemulator@7650 3239 +static struct xz_dec *state;
slaxemulator@7650 3240 +
slaxemulator@7650 3241 +/*
slaxemulator@7650 3242 + * Return value of xz_dec_run(). We need to avoid calling xz_dec_run() after
slaxemulator@7650 3243 + * it has returned XZ_STREAM_END, so we make this static.
slaxemulator@7650 3244 + */
slaxemulator@7650 3245 +static enum xz_ret ret;
slaxemulator@7650 3246 +
slaxemulator@7650 3247 +/*
slaxemulator@7650 3248 + * Input and output buffers. The input buffer is used as a temporary safe
slaxemulator@7650 3249 + * place for the data coming from the userspace.
slaxemulator@7650 3250 + */
slaxemulator@7650 3251 +static uint8_t buffer_in[1024];
slaxemulator@7650 3252 +static uint8_t buffer_out[1024];
slaxemulator@7650 3253 +
slaxemulator@7650 3254 +/*
slaxemulator@7650 3255 + * Structure to pass the input and output buffers to the XZ decoder.
slaxemulator@7650 3256 + * A few of the fields are never modified so we initialize them here.
slaxemulator@7650 3257 + */
slaxemulator@7650 3258 +static struct xz_buf buffers = {
slaxemulator@7650 3259 + .in = buffer_in,
slaxemulator@7650 3260 + .out = buffer_out,
slaxemulator@7650 3261 + .out_size = sizeof(buffer_out)
slaxemulator@7650 3262 +};
slaxemulator@7650 3263 +
slaxemulator@7650 3264 +/*
slaxemulator@7650 3265 + * CRC32 of uncompressed data. This is used to give the user a simple way
slaxemulator@7650 3266 + * to check that the decoder produces correct output.
slaxemulator@7650 3267 + */
slaxemulator@7650 3268 +static uint32_t crc;
slaxemulator@7650 3269 +
slaxemulator@7650 3270 +static int xz_dec_test_open(struct inode *i, struct file *f)
slaxemulator@7650 3271 +{
slaxemulator@7650 3272 + if (device_is_open)
slaxemulator@7650 3273 + return -EBUSY;
slaxemulator@7650 3274 +
slaxemulator@7650 3275 + device_is_open = true;
slaxemulator@7650 3276 +
slaxemulator@7650 3277 + xz_dec_reset(state);
slaxemulator@7650 3278 + ret = XZ_OK;
slaxemulator@7650 3279 + crc = 0xFFFFFFFF;
slaxemulator@7650 3280 +
slaxemulator@7650 3281 + buffers.in_pos = 0;
slaxemulator@7650 3282 + buffers.in_size = 0;
slaxemulator@7650 3283 + buffers.out_pos = 0;
slaxemulator@7650 3284 +
slaxemulator@7650 3285 + printk(KERN_INFO DEVICE_NAME ": opened\n");
slaxemulator@7650 3286 + return 0;
slaxemulator@7650 3287 +}
slaxemulator@7650 3288 +
slaxemulator@7650 3289 +static int xz_dec_test_release(struct inode *i, struct file *f)
slaxemulator@7650 3290 +{
slaxemulator@7650 3291 + device_is_open = false;
slaxemulator@7650 3292 +
slaxemulator@7650 3293 + if (ret == XZ_OK)
slaxemulator@7650 3294 + printk(KERN_INFO DEVICE_NAME ": input was truncated\n");
slaxemulator@7650 3295 +
slaxemulator@7650 3296 + printk(KERN_INFO DEVICE_NAME ": closed\n");
slaxemulator@7650 3297 + return 0;
slaxemulator@7650 3298 +}
slaxemulator@7650 3299 +
slaxemulator@7650 3300 +/*
slaxemulator@7650 3301 + * Decode the data given to us from the userspace. CRC32 of the uncompressed
slaxemulator@7650 3302 + * data is calculated and is printed at the end of successful decoding. The
slaxemulator@7650 3303 + * uncompressed data isn't stored anywhere for further use.
slaxemulator@7650 3304 + *
slaxemulator@7650 3305 + * The .xz file must have exactly one Stream and no Stream Padding. The data
slaxemulator@7650 3306 + * after the first Stream is considered to be garbage.
slaxemulator@7650 3307 + */
slaxemulator@7650 3308 +static ssize_t xz_dec_test_write(struct file *file, const char __user *buf,
slaxemulator@7650 3309 + size_t size, loff_t *pos)
slaxemulator@7650 3310 +{
slaxemulator@7650 3311 + size_t remaining;
slaxemulator@7650 3312 +
slaxemulator@7650 3313 + if (ret != XZ_OK) {
slaxemulator@7650 3314 + if (size > 0)
slaxemulator@7650 3315 + printk(KERN_INFO DEVICE_NAME ": %zu bytes of "
slaxemulator@7650 3316 + "garbage at the end of the file\n",
slaxemulator@7650 3317 + size);
slaxemulator@7650 3318 +
slaxemulator@7650 3319 + return -ENOSPC;
slaxemulator@7650 3320 + }
slaxemulator@7650 3321 +
slaxemulator@7650 3322 + printk(KERN_INFO DEVICE_NAME ": decoding %zu bytes of input\n",
slaxemulator@7650 3323 + size);
slaxemulator@7650 3324 +
slaxemulator@7650 3325 + remaining = size;
slaxemulator@7650 3326 + while ((remaining > 0 || buffers.out_pos == buffers.out_size)
slaxemulator@7650 3327 + && ret == XZ_OK) {
slaxemulator@7650 3328 + if (buffers.in_pos == buffers.in_size) {
slaxemulator@7650 3329 + buffers.in_pos = 0;
slaxemulator@7650 3330 + buffers.in_size = min(remaining, sizeof(buffer_in));
slaxemulator@7650 3331 + if (copy_from_user(buffer_in, buf, buffers.in_size))
slaxemulator@7650 3332 + return -EFAULT;
slaxemulator@7650 3333 +
slaxemulator@7650 3334 + buf += buffers.in_size;
slaxemulator@7650 3335 + remaining -= buffers.in_size;
slaxemulator@7650 3336 + }
slaxemulator@7650 3337 +
slaxemulator@7650 3338 + buffers.out_pos = 0;
slaxemulator@7650 3339 + ret = xz_dec_run(state, &buffers);
slaxemulator@7650 3340 + crc = crc32(crc, buffer_out, buffers.out_pos);
slaxemulator@7650 3341 + }
slaxemulator@7650 3342 +
slaxemulator@7650 3343 + switch (ret) {
slaxemulator@7650 3344 + case XZ_OK:
slaxemulator@7650 3345 + printk(KERN_INFO DEVICE_NAME ": XZ_OK\n");
slaxemulator@7650 3346 + return size;
slaxemulator@7650 3347 +
slaxemulator@7650 3348 + case XZ_STREAM_END:
slaxemulator@7650 3349 + printk(KERN_INFO DEVICE_NAME ": XZ_STREAM_END, "
slaxemulator@7650 3350 + "CRC32 = 0x%08X\n", ~crc);
slaxemulator@7650 3351 + return size - remaining - (buffers.in_size - buffers.in_pos);
slaxemulator@7650 3352 +
slaxemulator@7650 3353 + case XZ_MEMLIMIT_ERROR:
slaxemulator@7650 3354 + printk(KERN_INFO DEVICE_NAME ": XZ_MEMLIMIT_ERROR\n");
slaxemulator@7650 3355 + break;
slaxemulator@7650 3356 +
slaxemulator@7650 3357 + case XZ_FORMAT_ERROR:
slaxemulator@7650 3358 + printk(KERN_INFO DEVICE_NAME ": XZ_FORMAT_ERROR\n");
slaxemulator@7650 3359 + break;
slaxemulator@7650 3360 +
slaxemulator@7650 3361 + case XZ_OPTIONS_ERROR:
slaxemulator@7650 3362 + printk(KERN_INFO DEVICE_NAME ": XZ_OPTIONS_ERROR\n");
slaxemulator@7650 3363 + break;
slaxemulator@7650 3364 +
slaxemulator@7650 3365 + case XZ_DATA_ERROR:
slaxemulator@7650 3366 + printk(KERN_INFO DEVICE_NAME ": XZ_DATA_ERROR\n");
slaxemulator@7650 3367 + break;
slaxemulator@7650 3368 +
slaxemulator@7650 3369 + case XZ_BUF_ERROR:
slaxemulator@7650 3370 + printk(KERN_INFO DEVICE_NAME ": XZ_BUF_ERROR\n");
slaxemulator@7650 3371 + break;
slaxemulator@7650 3372 +
slaxemulator@7650 3373 + default:
slaxemulator@7650 3374 + printk(KERN_INFO DEVICE_NAME ": Bug detected!\n");
slaxemulator@7650 3375 + break;
slaxemulator@7650 3376 + }
slaxemulator@7650 3377 +
slaxemulator@7650 3378 + return -EIO;
slaxemulator@7650 3379 +}
slaxemulator@7650 3380 +
slaxemulator@7650 3381 +/* Allocate the XZ decoder state and register the character device. */
slaxemulator@7650 3382 +static int __init xz_dec_test_init(void)
slaxemulator@7650 3383 +{
slaxemulator@7650 3384 + static const struct file_operations fileops = {
slaxemulator@7650 3385 + .owner = THIS_MODULE,
slaxemulator@7650 3386 + .open = &xz_dec_test_open,
slaxemulator@7650 3387 + .release = &xz_dec_test_release,
slaxemulator@7650 3388 + .write = &xz_dec_test_write
slaxemulator@7650 3389 + };
slaxemulator@7650 3390 +
slaxemulator@7650 3391 + state = xz_dec_init(XZ_PREALLOC, DICT_MAX);
slaxemulator@7650 3392 + if (state == NULL)
slaxemulator@7650 3393 + return -ENOMEM;
slaxemulator@7650 3394 +
slaxemulator@7650 3395 + device_major = register_chrdev(0, DEVICE_NAME, &fileops);
slaxemulator@7650 3396 + if (device_major < 0) {
slaxemulator@7650 3397 + xz_dec_end(state);
slaxemulator@7650 3398 + return device_major;
slaxemulator@7650 3399 + }
slaxemulator@7650 3400 +
slaxemulator@7650 3401 + printk(KERN_INFO DEVICE_NAME ": module loaded\n");
slaxemulator@7650 3402 + printk(KERN_INFO DEVICE_NAME ": Create a device node with "
slaxemulator@7650 3403 + "'mknod " DEVICE_NAME " c %d 0' and write .xz files "
slaxemulator@7650 3404 + "to it.\n", device_major);
slaxemulator@7650 3405 + return 0;
slaxemulator@7650 3406 +}
slaxemulator@7650 3407 +
slaxemulator@7650 3408 +static void __exit xz_dec_test_exit(void)
slaxemulator@7650 3409 +{
slaxemulator@7650 3410 + unregister_chrdev(device_major, DEVICE_NAME);
slaxemulator@7650 3411 + xz_dec_end(state);
slaxemulator@7650 3412 + printk(KERN_INFO DEVICE_NAME ": module unloaded\n");
slaxemulator@7650 3413 +}
slaxemulator@7650 3414 +
slaxemulator@7650 3415 +module_init(xz_dec_test_init);
slaxemulator@7650 3416 +module_exit(xz_dec_test_exit);
slaxemulator@7650 3417 +
slaxemulator@7650 3418 +MODULE_DESCRIPTION("XZ decompressor tester");
slaxemulator@7650 3419 +MODULE_VERSION("1.0");
slaxemulator@7650 3420 +MODULE_AUTHOR("Lasse Collin <lasse.collin@tukaani.org>");
slaxemulator@7650 3421 +
slaxemulator@7650 3422 +/*
slaxemulator@7650 3423 + * This code is in the public domain, but in Linux it's simplest to just
slaxemulator@7650 3424 + * say it's GPL and consider the authors as the copyright holders.
slaxemulator@7650 3425 + */
slaxemulator@7650 3426 +MODULE_LICENSE("GPL");
slaxemulator@7650 3427 diff --git a/lib/xz/xz_lzma2.h b/lib/xz/xz_lzma2.h
slaxemulator@7650 3428 new file mode 100644
slaxemulator@7650 3429 index 0000000..071d67b
slaxemulator@7650 3430 --- /dev/null
slaxemulator@7650 3431 +++ b/lib/xz/xz_lzma2.h
slaxemulator@7650 3432 @@ -0,0 +1,204 @@
slaxemulator@7650 3433 +/*
slaxemulator@7650 3434 + * LZMA2 definitions
slaxemulator@7650 3435 + *
slaxemulator@7650 3436 + * Authors: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3437 + * Igor Pavlov <http://7-zip.org/>
slaxemulator@7650 3438 + *
slaxemulator@7650 3439 + * This file has been put into the public domain.
slaxemulator@7650 3440 + * You can do whatever you want with this file.
slaxemulator@7650 3441 + */
slaxemulator@7650 3442 +
slaxemulator@7650 3443 +#ifndef XZ_LZMA2_H
slaxemulator@7650 3444 +#define XZ_LZMA2_H
slaxemulator@7650 3445 +
slaxemulator@7650 3446 +/* Range coder constants */
slaxemulator@7650 3447 +#define RC_SHIFT_BITS 8
slaxemulator@7650 3448 +#define RC_TOP_BITS 24
slaxemulator@7650 3449 +#define RC_TOP_VALUE (1 << RC_TOP_BITS)
slaxemulator@7650 3450 +#define RC_BIT_MODEL_TOTAL_BITS 11
slaxemulator@7650 3451 +#define RC_BIT_MODEL_TOTAL (1 << RC_BIT_MODEL_TOTAL_BITS)
slaxemulator@7650 3452 +#define RC_MOVE_BITS 5
slaxemulator@7650 3453 +
slaxemulator@7650 3454 +/*
slaxemulator@7650 3455 + * Maximum number of position states. A position state is the lowest pb
slaxemulator@7650 3456 + * number of bits of the current uncompressed offset. In some places there
slaxemulator@7650 3457 + * are different sets of probabilities for different position states.
slaxemulator@7650 3458 + */
slaxemulator@7650 3459 +#define POS_STATES_MAX (1 << 4)
slaxemulator@7650 3460 +
slaxemulator@7650 3461 +/*
slaxemulator@7650 3462 + * This enum is used to track which LZMA symbols have occurred most recently
slaxemulator@7650 3463 + * and in which order. This information is used to predict the next symbol.
slaxemulator@7650 3464 + *
slaxemulator@7650 3465 + * Symbols:
slaxemulator@7650 3466 + * - Literal: One 8-bit byte
slaxemulator@7650 3467 + * - Match: Repeat a chunk of data at some distance
slaxemulator@7650 3468 + * - Long repeat: Multi-byte match at a recently seen distance
slaxemulator@7650 3469 + * - Short repeat: One-byte repeat at a recently seen distance
slaxemulator@7650 3470 + *
slaxemulator@7650 3471 + * The symbol names are in from STATE_oldest_older_previous. REP means
slaxemulator@7650 3472 + * either short or long repeated match, and NONLIT means any non-literal.
slaxemulator@7650 3473 + */
slaxemulator@7650 3474 +enum lzma_state {
slaxemulator@7650 3475 + STATE_LIT_LIT,
slaxemulator@7650 3476 + STATE_MATCH_LIT_LIT,
slaxemulator@7650 3477 + STATE_REP_LIT_LIT,
slaxemulator@7650 3478 + STATE_SHORTREP_LIT_LIT,
slaxemulator@7650 3479 + STATE_MATCH_LIT,
slaxemulator@7650 3480 + STATE_REP_LIT,
slaxemulator@7650 3481 + STATE_SHORTREP_LIT,
slaxemulator@7650 3482 + STATE_LIT_MATCH,
slaxemulator@7650 3483 + STATE_LIT_LONGREP,
slaxemulator@7650 3484 + STATE_LIT_SHORTREP,
slaxemulator@7650 3485 + STATE_NONLIT_MATCH,
slaxemulator@7650 3486 + STATE_NONLIT_REP
slaxemulator@7650 3487 +};
slaxemulator@7650 3488 +
slaxemulator@7650 3489 +/* Total number of states */
slaxemulator@7650 3490 +#define STATES 12
slaxemulator@7650 3491 +
slaxemulator@7650 3492 +/* The lowest 7 states indicate that the previous state was a literal. */
slaxemulator@7650 3493 +#define LIT_STATES 7
slaxemulator@7650 3494 +
slaxemulator@7650 3495 +/* Indicate that the latest symbol was a literal. */
slaxemulator@7650 3496 +static inline void lzma_state_literal(enum lzma_state *state)
slaxemulator@7650 3497 +{
slaxemulator@7650 3498 + if (*state <= STATE_SHORTREP_LIT_LIT)
slaxemulator@7650 3499 + *state = STATE_LIT_LIT;
slaxemulator@7650 3500 + else if (*state <= STATE_LIT_SHORTREP)
slaxemulator@7650 3501 + *state -= 3;
slaxemulator@7650 3502 + else
slaxemulator@7650 3503 + *state -= 6;
slaxemulator@7650 3504 +}
slaxemulator@7650 3505 +
slaxemulator@7650 3506 +/* Indicate that the latest symbol was a match. */
slaxemulator@7650 3507 +static inline void lzma_state_match(enum lzma_state *state)
slaxemulator@7650 3508 +{
slaxemulator@7650 3509 + *state = *state < LIT_STATES ? STATE_LIT_MATCH : STATE_NONLIT_MATCH;
slaxemulator@7650 3510 +}
slaxemulator@7650 3511 +
slaxemulator@7650 3512 +/* Indicate that the latest state was a long repeated match. */
slaxemulator@7650 3513 +static inline void lzma_state_long_rep(enum lzma_state *state)
slaxemulator@7650 3514 +{
slaxemulator@7650 3515 + *state = *state < LIT_STATES ? STATE_LIT_LONGREP : STATE_NONLIT_REP;
slaxemulator@7650 3516 +}
slaxemulator@7650 3517 +
slaxemulator@7650 3518 +/* Indicate that the latest symbol was a short match. */
slaxemulator@7650 3519 +static inline void lzma_state_short_rep(enum lzma_state *state)
slaxemulator@7650 3520 +{
slaxemulator@7650 3521 + *state = *state < LIT_STATES ? STATE_LIT_SHORTREP : STATE_NONLIT_REP;
slaxemulator@7650 3522 +}
slaxemulator@7650 3523 +
slaxemulator@7650 3524 +/* Test if the previous symbol was a literal. */
slaxemulator@7650 3525 +static inline bool lzma_state_is_literal(enum lzma_state state)
slaxemulator@7650 3526 +{
slaxemulator@7650 3527 + return state < LIT_STATES;
slaxemulator@7650 3528 +}
slaxemulator@7650 3529 +
slaxemulator@7650 3530 +/* Each literal coder is divided in three sections:
slaxemulator@7650 3531 + * - 0x001-0x0FF: Without match byte
slaxemulator@7650 3532 + * - 0x101-0x1FF: With match byte; match bit is 0
slaxemulator@7650 3533 + * - 0x201-0x2FF: With match byte; match bit is 1
slaxemulator@7650 3534 + *
slaxemulator@7650 3535 + * Match byte is used when the previous LZMA symbol was something else than
slaxemulator@7650 3536 + * a literal (that is, it was some kind of match).
slaxemulator@7650 3537 + */
slaxemulator@7650 3538 +#define LITERAL_CODER_SIZE 0x300
slaxemulator@7650 3539 +
slaxemulator@7650 3540 +/* Maximum number of literal coders */
slaxemulator@7650 3541 +#define LITERAL_CODERS_MAX (1 << 4)
slaxemulator@7650 3542 +
slaxemulator@7650 3543 +/* Minimum length of a match is two bytes. */
slaxemulator@7650 3544 +#define MATCH_LEN_MIN 2
slaxemulator@7650 3545 +
slaxemulator@7650 3546 +/* Match length is encoded with 4, 5, or 10 bits.
slaxemulator@7650 3547 + *
slaxemulator@7650 3548 + * Length Bits
slaxemulator@7650 3549 + * 2-9 4 = Choice=0 + 3 bits
slaxemulator@7650 3550 + * 10-17 5 = Choice=1 + Choice2=0 + 3 bits
slaxemulator@7650 3551 + * 18-273 10 = Choice=1 + Choice2=1 + 8 bits
slaxemulator@7650 3552 + */
slaxemulator@7650 3553 +#define LEN_LOW_BITS 3
slaxemulator@7650 3554 +#define LEN_LOW_SYMBOLS (1 << LEN_LOW_BITS)
slaxemulator@7650 3555 +#define LEN_MID_BITS 3
slaxemulator@7650 3556 +#define LEN_MID_SYMBOLS (1 << LEN_MID_BITS)
slaxemulator@7650 3557 +#define LEN_HIGH_BITS 8
slaxemulator@7650 3558 +#define LEN_HIGH_SYMBOLS (1 << LEN_HIGH_BITS)
slaxemulator@7650 3559 +#define LEN_SYMBOLS (LEN_LOW_SYMBOLS + LEN_MID_SYMBOLS + LEN_HIGH_SYMBOLS)
slaxemulator@7650 3560 +
slaxemulator@7650 3561 +/*
slaxemulator@7650 3562 + * Maximum length of a match is 273 which is a result of the encoding
slaxemulator@7650 3563 + * described above.
slaxemulator@7650 3564 + */
slaxemulator@7650 3565 +#define MATCH_LEN_MAX (MATCH_LEN_MIN + LEN_SYMBOLS - 1)
slaxemulator@7650 3566 +
slaxemulator@7650 3567 +/*
slaxemulator@7650 3568 + * Different sets of probabilities are used for match distances that have
slaxemulator@7650 3569 + * very short match length: Lengths of 2, 3, and 4 bytes have a separate
slaxemulator@7650 3570 + * set of probabilities for each length. The matches with longer length
slaxemulator@7650 3571 + * use a shared set of probabilities.
slaxemulator@7650 3572 + */
slaxemulator@7650 3573 +#define DIST_STATES 4
slaxemulator@7650 3574 +
slaxemulator@7650 3575 +/*
slaxemulator@7650 3576 + * Get the index of the appropriate probability array for decoding
slaxemulator@7650 3577 + * the distance slot.
slaxemulator@7650 3578 + */
slaxemulator@7650 3579 +static inline uint32_t lzma_get_dist_state(uint32_t len)
slaxemulator@7650 3580 +{
slaxemulator@7650 3581 + return len < DIST_STATES + MATCH_LEN_MIN
slaxemulator@7650 3582 + ? len - MATCH_LEN_MIN : DIST_STATES - 1;
slaxemulator@7650 3583 +}
slaxemulator@7650 3584 +
slaxemulator@7650 3585 +/*
slaxemulator@7650 3586 + * The highest two bits of a 32-bit match distance are encoded using six bits.
slaxemulator@7650 3587 + * This six-bit value is called a distance slot. This way encoding a 32-bit
slaxemulator@7650 3588 + * value takes 6-36 bits, larger values taking more bits.
slaxemulator@7650 3589 + */
slaxemulator@7650 3590 +#define DIST_SLOT_BITS 6
slaxemulator@7650 3591 +#define DIST_SLOTS (1 << DIST_SLOT_BITS)
slaxemulator@7650 3592 +
slaxemulator@7650 3593 +/* Match distances up to 127 are fully encoded using probabilities. Since
slaxemulator@7650 3594 + * the highest two bits (distance slot) are always encoded using six bits,
slaxemulator@7650 3595 + * the distances 0-3 don't need any additional bits to encode, since the
slaxemulator@7650 3596 + * distance slot itself is the same as the actual distance. DIST_MODEL_START
slaxemulator@7650 3597 + * indicates the first distance slot where at least one additional bit is
slaxemulator@7650 3598 + * needed.
slaxemulator@7650 3599 + */
slaxemulator@7650 3600 +#define DIST_MODEL_START 4
slaxemulator@7650 3601 +
slaxemulator@7650 3602 +/*
slaxemulator@7650 3603 + * Match distances greater than 127 are encoded in three pieces:
slaxemulator@7650 3604 + * - distance slot: the highest two bits
slaxemulator@7650 3605 + * - direct bits: 2-26 bits below the highest two bits
slaxemulator@7650 3606 + * - alignment bits: four lowest bits
slaxemulator@7650 3607 + *
slaxemulator@7650 3608 + * Direct bits don't use any probabilities.
slaxemulator@7650 3609 + *
slaxemulator@7650 3610 + * The distance slot value of 14 is for distances 128-191.
slaxemulator@7650 3611 + */
slaxemulator@7650 3612 +#define DIST_MODEL_END 14
slaxemulator@7650 3613 +
slaxemulator@7650 3614 +/* Distance slots that indicate a distance <= 127. */
slaxemulator@7650 3615 +#define FULL_DISTANCES_BITS (DIST_MODEL_END / 2)
slaxemulator@7650 3616 +#define FULL_DISTANCES (1 << FULL_DISTANCES_BITS)
slaxemulator@7650 3617 +
slaxemulator@7650 3618 +/*
slaxemulator@7650 3619 + * For match distances greater than 127, only the highest two bits and the
slaxemulator@7650 3620 + * lowest four bits (alignment) is encoded using probabilities.
slaxemulator@7650 3621 + */
slaxemulator@7650 3622 +#define ALIGN_BITS 4
slaxemulator@7650 3623 +#define ALIGN_SIZE (1 << ALIGN_BITS)
slaxemulator@7650 3624 +#define ALIGN_MASK (ALIGN_SIZE - 1)
slaxemulator@7650 3625 +
slaxemulator@7650 3626 +/* Total number of all probability variables */
slaxemulator@7650 3627 +#define PROBS_TOTAL (1846 + LITERAL_CODERS_MAX * LITERAL_CODER_SIZE)
slaxemulator@7650 3628 +
slaxemulator@7650 3629 +/*
slaxemulator@7650 3630 + * LZMA remembers the four most recent match distances. Reusing these
slaxemulator@7650 3631 + * distances tends to take less space than re-encoding the actual
slaxemulator@7650 3632 + * distance value.
slaxemulator@7650 3633 + */
slaxemulator@7650 3634 +#define REPS 4
slaxemulator@7650 3635 +
slaxemulator@7650 3636 +#endif
slaxemulator@7650 3637 diff --git a/lib/xz/xz_private.h b/lib/xz/xz_private.h
slaxemulator@7650 3638 new file mode 100644
slaxemulator@7650 3639 index 0000000..a65633e
slaxemulator@7650 3640 --- /dev/null
slaxemulator@7650 3641 +++ b/lib/xz/xz_private.h
slaxemulator@7650 3642 @@ -0,0 +1,156 @@
slaxemulator@7650 3643 +/*
slaxemulator@7650 3644 + * Private includes and definitions
slaxemulator@7650 3645 + *
slaxemulator@7650 3646 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3647 + *
slaxemulator@7650 3648 + * This file has been put into the public domain.
slaxemulator@7650 3649 + * You can do whatever you want with this file.
slaxemulator@7650 3650 + */
slaxemulator@7650 3651 +
slaxemulator@7650 3652 +#ifndef XZ_PRIVATE_H
slaxemulator@7650 3653 +#define XZ_PRIVATE_H
slaxemulator@7650 3654 +
slaxemulator@7650 3655 +#ifdef __KERNEL__
slaxemulator@7650 3656 +# include <linux/xz.h>
slaxemulator@7650 3657 +# include <asm/byteorder.h>
slaxemulator@7650 3658 +# include <asm/unaligned.h>
slaxemulator@7650 3659 + /* XZ_PREBOOT may be defined only via decompress_unxz.c. */
slaxemulator@7650 3660 +# ifndef XZ_PREBOOT
slaxemulator@7650 3661 +# include <linux/slab.h>
slaxemulator@7650 3662 +# include <linux/vmalloc.h>
slaxemulator@7650 3663 +# include <linux/string.h>
slaxemulator@7650 3664 +# ifdef CONFIG_XZ_DEC_X86
slaxemulator@7650 3665 +# define XZ_DEC_X86
slaxemulator@7650 3666 +# endif
slaxemulator@7650 3667 +# ifdef CONFIG_XZ_DEC_POWERPC
slaxemulator@7650 3668 +# define XZ_DEC_POWERPC
slaxemulator@7650 3669 +# endif
slaxemulator@7650 3670 +# ifdef CONFIG_XZ_DEC_IA64
slaxemulator@7650 3671 +# define XZ_DEC_IA64
slaxemulator@7650 3672 +# endif
slaxemulator@7650 3673 +# ifdef CONFIG_XZ_DEC_ARM
slaxemulator@7650 3674 +# define XZ_DEC_ARM
slaxemulator@7650 3675 +# endif
slaxemulator@7650 3676 +# ifdef CONFIG_XZ_DEC_ARMTHUMB
slaxemulator@7650 3677 +# define XZ_DEC_ARMTHUMB
slaxemulator@7650 3678 +# endif
slaxemulator@7650 3679 +# ifdef CONFIG_XZ_DEC_SPARC
slaxemulator@7650 3680 +# define XZ_DEC_SPARC
slaxemulator@7650 3681 +# endif
slaxemulator@7650 3682 +# define memeq(a, b, size) (memcmp(a, b, size) == 0)
slaxemulator@7650 3683 +# define memzero(buf, size) memset(buf, 0, size)
slaxemulator@7650 3684 +# endif
slaxemulator@7650 3685 +# define get_le32(p) le32_to_cpup((const uint32_t *)(p))
slaxemulator@7650 3686 +#else
slaxemulator@7650 3687 + /*
slaxemulator@7650 3688 + * For userspace builds, use a separate header to define the required
slaxemulator@7650 3689 + * macros and functions. This makes it easier to adapt the code into
slaxemulator@7650 3690 + * different environments and avoids clutter in the Linux kernel tree.
slaxemulator@7650 3691 + */
slaxemulator@7650 3692 +# include "xz_config.h"
slaxemulator@7650 3693 +#endif
slaxemulator@7650 3694 +
slaxemulator@7650 3695 +/* If no specific decoding mode is requested, enable support for all modes. */
slaxemulator@7650 3696 +#if !defined(XZ_DEC_SINGLE) && !defined(XZ_DEC_PREALLOC) \
slaxemulator@7650 3697 + && !defined(XZ_DEC_DYNALLOC)
slaxemulator@7650 3698 +# define XZ_DEC_SINGLE
slaxemulator@7650 3699 +# define XZ_DEC_PREALLOC
slaxemulator@7650 3700 +# define XZ_DEC_DYNALLOC
slaxemulator@7650 3701 +#endif
slaxemulator@7650 3702 +
slaxemulator@7650 3703 +/*
slaxemulator@7650 3704 + * The DEC_IS_foo(mode) macros are used in "if" statements. If only some
slaxemulator@7650 3705 + * of the supported modes are enabled, these macros will evaluate to true or
slaxemulator@7650 3706 + * false at compile time and thus allow the compiler to omit unneeded code.
slaxemulator@7650 3707 + */
slaxemulator@7650 3708 +#ifdef XZ_DEC_SINGLE
slaxemulator@7650 3709 +# define DEC_IS_SINGLE(mode) ((mode) == XZ_SINGLE)
slaxemulator@7650 3710 +#else
slaxemulator@7650 3711 +# define DEC_IS_SINGLE(mode) (false)
slaxemulator@7650 3712 +#endif
slaxemulator@7650 3713 +
slaxemulator@7650 3714 +#ifdef XZ_DEC_PREALLOC
slaxemulator@7650 3715 +# define DEC_IS_PREALLOC(mode) ((mode) == XZ_PREALLOC)
slaxemulator@7650 3716 +#else
slaxemulator@7650 3717 +# define DEC_IS_PREALLOC(mode) (false)
slaxemulator@7650 3718 +#endif
slaxemulator@7650 3719 +
slaxemulator@7650 3720 +#ifdef XZ_DEC_DYNALLOC
slaxemulator@7650 3721 +# define DEC_IS_DYNALLOC(mode) ((mode) == XZ_DYNALLOC)
slaxemulator@7650 3722 +#else
slaxemulator@7650 3723 +# define DEC_IS_DYNALLOC(mode) (false)
slaxemulator@7650 3724 +#endif
slaxemulator@7650 3725 +
slaxemulator@7650 3726 +#if !defined(XZ_DEC_SINGLE)
slaxemulator@7650 3727 +# define DEC_IS_MULTI(mode) (true)
slaxemulator@7650 3728 +#elif defined(XZ_DEC_PREALLOC) || defined(XZ_DEC_DYNALLOC)
slaxemulator@7650 3729 +# define DEC_IS_MULTI(mode) ((mode) != XZ_SINGLE)
slaxemulator@7650 3730 +#else
slaxemulator@7650 3731 +# define DEC_IS_MULTI(mode) (false)
slaxemulator@7650 3732 +#endif
slaxemulator@7650 3733 +
slaxemulator@7650 3734 +/*
slaxemulator@7650 3735 + * If any of the BCJ filter decoders are wanted, define XZ_DEC_BCJ.
slaxemulator@7650 3736 + * XZ_DEC_BCJ is used to enable generic support for BCJ decoders.
slaxemulator@7650 3737 + */
slaxemulator@7650 3738 +#ifndef XZ_DEC_BCJ
slaxemulator@7650 3739 +# if defined(XZ_DEC_X86) || defined(XZ_DEC_POWERPC) \
slaxemulator@7650 3740 + || defined(XZ_DEC_IA64) || defined(XZ_DEC_ARM) \
slaxemulator@7650 3741 + || defined(XZ_DEC_ARM) || defined(XZ_DEC_ARMTHUMB) \
slaxemulator@7650 3742 + || defined(XZ_DEC_SPARC)
slaxemulator@7650 3743 +# define XZ_DEC_BCJ
slaxemulator@7650 3744 +# endif
slaxemulator@7650 3745 +#endif
slaxemulator@7650 3746 +
slaxemulator@7650 3747 +/*
slaxemulator@7650 3748 + * Allocate memory for LZMA2 decoder. xz_dec_lzma2_reset() must be used
slaxemulator@7650 3749 + * before calling xz_dec_lzma2_run().
slaxemulator@7650 3750 + */
slaxemulator@7650 3751 +XZ_EXTERN struct xz_dec_lzma2 *xz_dec_lzma2_create(enum xz_mode mode,
slaxemulator@7650 3752 + uint32_t dict_max);
slaxemulator@7650 3753 +
slaxemulator@7650 3754 +/*
slaxemulator@7650 3755 + * Decode the LZMA2 properties (one byte) and reset the decoder. Return
slaxemulator@7650 3756 + * XZ_OK on success, XZ_MEMLIMIT_ERROR if the preallocated dictionary is not
slaxemulator@7650 3757 + * big enough, and XZ_OPTIONS_ERROR if props indicates something that this
slaxemulator@7650 3758 + * decoder doesn't support.
slaxemulator@7650 3759 + */
slaxemulator@7650 3760 +XZ_EXTERN enum xz_ret xz_dec_lzma2_reset(struct xz_dec_lzma2 *s,
slaxemulator@7650 3761 + uint8_t props);
slaxemulator@7650 3762 +
slaxemulator@7650 3763 +/* Decode raw LZMA2 stream from b->in to b->out. */
slaxemulator@7650 3764 +XZ_EXTERN enum xz_ret xz_dec_lzma2_run(struct xz_dec_lzma2 *s,
slaxemulator@7650 3765 + struct xz_buf *b);
slaxemulator@7650 3766 +
slaxemulator@7650 3767 +/* Free the memory allocated for the LZMA2 decoder. */
slaxemulator@7650 3768 +XZ_EXTERN void xz_dec_lzma2_end(struct xz_dec_lzma2 *s);
slaxemulator@7650 3769 +
slaxemulator@7650 3770 +#ifdef XZ_DEC_BCJ
slaxemulator@7650 3771 +/*
slaxemulator@7650 3772 + * Allocate memory for BCJ decoders. xz_dec_bcj_reset() must be used before
slaxemulator@7650 3773 + * calling xz_dec_bcj_run().
slaxemulator@7650 3774 + */
slaxemulator@7650 3775 +XZ_EXTERN struct xz_dec_bcj *xz_dec_bcj_create(bool single_call);
slaxemulator@7650 3776 +
slaxemulator@7650 3777 +/*
slaxemulator@7650 3778 + * Decode the Filter ID of a BCJ filter. This implementation doesn't
slaxemulator@7650 3779 + * support custom start offsets, so no decoding of Filter Properties
slaxemulator@7650 3780 + * is needed. Returns XZ_OK if the given Filter ID is supported.
slaxemulator@7650 3781 + * Otherwise XZ_OPTIONS_ERROR is returned.
slaxemulator@7650 3782 + */
slaxemulator@7650 3783 +XZ_EXTERN enum xz_ret xz_dec_bcj_reset(struct xz_dec_bcj *s, uint8_t id);
slaxemulator@7650 3784 +
slaxemulator@7650 3785 +/*
slaxemulator@7650 3786 + * Decode raw BCJ + LZMA2 stream. This must be used only if there actually is
slaxemulator@7650 3787 + * a BCJ filter in the chain. If the chain has only LZMA2, xz_dec_lzma2_run()
slaxemulator@7650 3788 + * must be called directly.
slaxemulator@7650 3789 + */
slaxemulator@7650 3790 +XZ_EXTERN enum xz_ret xz_dec_bcj_run(struct xz_dec_bcj *s,
slaxemulator@7650 3791 + struct xz_dec_lzma2 *lzma2,
slaxemulator@7650 3792 + struct xz_buf *b);
slaxemulator@7650 3793 +
slaxemulator@7650 3794 +/* Free the memory allocated for the BCJ filters. */
slaxemulator@7650 3795 +#define xz_dec_bcj_end(s) kfree(s)
slaxemulator@7650 3796 +#endif
slaxemulator@7650 3797 +
slaxemulator@7650 3798 +#endif
slaxemulator@7650 3799 diff --git a/lib/xz/xz_stream.h b/lib/xz/xz_stream.h
slaxemulator@7650 3800 new file mode 100644
slaxemulator@7650 3801 index 0000000..66cb5a7
slaxemulator@7650 3802 --- /dev/null
slaxemulator@7650 3803 +++ b/lib/xz/xz_stream.h
slaxemulator@7650 3804 @@ -0,0 +1,62 @@
slaxemulator@7650 3805 +/*
slaxemulator@7650 3806 + * Definitions for handling the .xz file format
slaxemulator@7650 3807 + *
slaxemulator@7650 3808 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3809 + *
slaxemulator@7650 3810 + * This file has been put into the public domain.
slaxemulator@7650 3811 + * You can do whatever you want with this file.
slaxemulator@7650 3812 + */
slaxemulator@7650 3813 +
slaxemulator@7650 3814 +#ifndef XZ_STREAM_H
slaxemulator@7650 3815 +#define XZ_STREAM_H
slaxemulator@7650 3816 +
slaxemulator@7650 3817 +#if defined(__KERNEL__) && !XZ_INTERNAL_CRC32
slaxemulator@7650 3818 +# include <linux/crc32.h>
slaxemulator@7650 3819 +# undef crc32
slaxemulator@7650 3820 +# define xz_crc32(buf, size, crc) \
slaxemulator@7650 3821 + (~crc32_le(~(uint32_t)(crc), buf, size))
slaxemulator@7650 3822 +#endif
slaxemulator@7650 3823 +
slaxemulator@7650 3824 +/*
slaxemulator@7650 3825 + * See the .xz file format specification at
slaxemulator@7650 3826 + * http://tukaani.org/xz/xz-file-format.txt
slaxemulator@7650 3827 + * to understand the container format.
slaxemulator@7650 3828 + */
slaxemulator@7650 3829 +
slaxemulator@7650 3830 +#define STREAM_HEADER_SIZE 12
slaxemulator@7650 3831 +
slaxemulator@7650 3832 +#define HEADER_MAGIC "\3757zXZ"
slaxemulator@7650 3833 +#define HEADER_MAGIC_SIZE 6
slaxemulator@7650 3834 +
slaxemulator@7650 3835 +#define FOOTER_MAGIC "YZ"
slaxemulator@7650 3836 +#define FOOTER_MAGIC_SIZE 2
slaxemulator@7650 3837 +
slaxemulator@7650 3838 +/*
slaxemulator@7650 3839 + * Variable-length integer can hold a 63-bit unsigned integer or a special
slaxemulator@7650 3840 + * value indicating that the value is unknown.
slaxemulator@7650 3841 + *
slaxemulator@7650 3842 + * Experimental: vli_type can be defined to uint32_t to save a few bytes
slaxemulator@7650 3843 + * in code size (no effect on speed). Doing so limits the uncompressed and
slaxemulator@7650 3844 + * compressed size of the file to less than 256 MiB and may also weaken
slaxemulator@7650 3845 + * error detection slightly.
slaxemulator@7650 3846 + */
slaxemulator@7650 3847 +typedef uint64_t vli_type;
slaxemulator@7650 3848 +
slaxemulator@7650 3849 +#define VLI_MAX ((vli_type)-1 / 2)
slaxemulator@7650 3850 +#define VLI_UNKNOWN ((vli_type)-1)
slaxemulator@7650 3851 +
slaxemulator@7650 3852 +/* Maximum encoded size of a VLI */
slaxemulator@7650 3853 +#define VLI_BYTES_MAX (sizeof(vli_type) * 8 / 7)
slaxemulator@7650 3854 +
slaxemulator@7650 3855 +/* Integrity Check types */
slaxemulator@7650 3856 +enum xz_check {
slaxemulator@7650 3857 + XZ_CHECK_NONE = 0,
slaxemulator@7650 3858 + XZ_CHECK_CRC32 = 1,
slaxemulator@7650 3859 + XZ_CHECK_CRC64 = 4,
slaxemulator@7650 3860 + XZ_CHECK_SHA256 = 10
slaxemulator@7650 3861 +};
slaxemulator@7650 3862 +
slaxemulator@7650 3863 +/* Maximum possible Check ID */
slaxemulator@7650 3864 +#define XZ_CHECK_MAX 15
slaxemulator@7650 3865 +
slaxemulator@7650 3866 +#endif
slaxemulator@7650 3867 diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
slaxemulator@7650 3868 index 54fd1b7..b862007 100644
slaxemulator@7650 3869 --- a/scripts/Makefile.lib
slaxemulator@7650 3870 +++ b/scripts/Makefile.lib
slaxemulator@7650 3871 @@ -246,6 +246,34 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
slaxemulator@7650 3872 lzop -9 && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
slaxemulator@7650 3873 (rm -f $@ ; false)
slaxemulator@7650 3874
slaxemulator@7650 3875 +# XZ
slaxemulator@7650 3876 +# ---------------------------------------------------------------------------
slaxemulator@7650 3877 +# Use xzkern to compress the kernel image and xzmisc to compress other things.
slaxemulator@7650 3878 +#
slaxemulator@7650 3879 +# xzkern uses a big LZMA2 dictionary since it doesn't increase memory usage
slaxemulator@7650 3880 +# of the kernel decompressor. A BCJ filter is used if it is available for
slaxemulator@7650 3881 +# the target architecture. xzkern also appends uncompressed size of the data
slaxemulator@7650 3882 +# using size_append. The .xz format has the size information available at
slaxemulator@7650 3883 +# the end of the file too, but it's in more complex format and it's good to
slaxemulator@7650 3884 +# avoid changing the part of the boot code that reads the uncompressed size.
slaxemulator@7650 3885 +# Note that the bytes added by size_append will make the xz tool think that
slaxemulator@7650 3886 +# the file is corrupt. This is expected.
slaxemulator@7650 3887 +#
slaxemulator@7650 3888 +# xzmisc doesn't use size_append, so it can be used to create normal .xz
slaxemulator@7650 3889 +# files. xzmisc uses smaller LZMA2 dictionary than xzkern, because a very
slaxemulator@7650 3890 +# big dictionary would increase the memory usage too much in the multi-call
slaxemulator@7650 3891 +# decompression mode. A BCJ filter isn't used either.
slaxemulator@7650 3892 +quiet_cmd_xzkern = XZKERN $@
slaxemulator@7650 3893 +cmd_xzkern = (cat $(filter-out FORCE,$^) | \
slaxemulator@7650 3894 + sh $(srctree)/scripts/xz_wrap.sh && \
slaxemulator@7650 3895 + $(call size_append, $(filter-out FORCE,$^))) > $@ || \
slaxemulator@7650 3896 + (rm -f $@ ; false)
slaxemulator@7650 3897 +
slaxemulator@7650 3898 +quiet_cmd_xzmisc = XZMISC $@
slaxemulator@7650 3899 +cmd_xzmisc = (cat $(filter-out FORCE,$^) | \
slaxemulator@7650 3900 + xz --check=crc32 --lzma2=dict=1MiB) > $@ || \
slaxemulator@7650 3901 + (rm -f $@ ; false)
slaxemulator@7650 3902 +
slaxemulator@7650 3903 # misc stuff
slaxemulator@7650 3904 # ---------------------------------------------------------------------------
slaxemulator@7650 3905 quote:="
slaxemulator@7650 3906 diff --git a/scripts/xz_wrap.sh b/scripts/xz_wrap.sh
slaxemulator@7650 3907 new file mode 100644
slaxemulator@7650 3908 index 0000000..17a5798
slaxemulator@7650 3909 --- /dev/null
slaxemulator@7650 3910 +++ b/scripts/xz_wrap.sh
slaxemulator@7650 3911 @@ -0,0 +1,23 @@
slaxemulator@7650 3912 +#!/bin/sh
slaxemulator@7650 3913 +#
slaxemulator@7650 3914 +# This is a wrapper for xz to compress the kernel image using appropriate
slaxemulator@7650 3915 +# compression options depending on the architecture.
slaxemulator@7650 3916 +#
slaxemulator@7650 3917 +# Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 3918 +#
slaxemulator@7650 3919 +# This file has been put into the public domain.
slaxemulator@7650 3920 +# You can do whatever you want with this file.
slaxemulator@7650 3921 +#
slaxemulator@7650 3922 +
slaxemulator@7650 3923 +BCJ=
slaxemulator@7650 3924 +LZMA2OPTS=
slaxemulator@7650 3925 +
slaxemulator@7650 3926 +case $ARCH in
slaxemulator@7650 3927 + x86|x86_64) BCJ=--x86 ;;
slaxemulator@7650 3928 + powerpc) BCJ=--powerpc ;;
slaxemulator@7650 3929 + ia64) BCJ=--ia64; LZMA2OPTS=pb=4 ;;
slaxemulator@7650 3930 + arm) BCJ=--arm ;;
slaxemulator@7650 3931 + sparc) BCJ=--sparc ;;
slaxemulator@7650 3932 +esac
slaxemulator@7650 3933 +
slaxemulator@7650 3934 +exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB