wok-current annotate linux/stuff/002-squashfs-decompressors-add-boot-time-xz-support.patch @ rev 9117

scim-tables-additional: remove spurrious part of genpkg_rules
author Pascal Bellard <pascal.bellard@slitaz.org>
date Sat Mar 05 13:01:38 2011 +0100 (2011-03-05)
parents
children
rev   line source
slaxemulator@7650 1 From: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 2 Date: Thu, 2 Dec 2010 19:14:37 +0000 (+0200)
slaxemulator@7650 3 Subject: Decompressors: Add boot-time XZ support
slaxemulator@7650 4 X-Git-Url: http://git.kernel.org/?p=linux%2Fkernel%2Fgit%2Fpkl%2Fsquashfs-xz.git;a=commitdiff_plain;h=c64bc9a229b46db75d7761601dd8ca25385a7780
slaxemulator@7650 5
slaxemulator@7650 6 Decompressors: Add boot-time XZ support
slaxemulator@7650 7
slaxemulator@7650 8 This implements the API defined in <linux/decompress/generic.h>
slaxemulator@7650 9 which is used for kernel, initramfs, and initrd decompression.
slaxemulator@7650 10 This patch together with the first patch is enough for
slaxemulator@7650 11 XZ-compressed initramfs and initrd; XZ-compressed kernel will
slaxemulator@7650 12 need arch-specific changes.
slaxemulator@7650 13
slaxemulator@7650 14 In contrast to other initramfs compression methods, support for
slaxemulator@7650 15 XZ-compressed initramfs is not enabled by default in usr/Kconfig.
slaxemulator@7650 16 This is primarily due to the Kconfig options of the xz_dec
slaxemulator@7650 17 module. It can be good to require that xz_dec is enabled
slaxemulator@7650 18 separately so the user can select only the BCJ filters he needs
slaxemulator@7650 19 when EMBEDDED=y.
slaxemulator@7650 20
slaxemulator@7650 21 The buffering requirements described in decompress_unxz.c are
slaxemulator@7650 22 stricter than with gzip, so the relevant changes should be done
slaxemulator@7650 23 to the arch-specific code when adding support for XZ-compressed
slaxemulator@7650 24 kernel. Similarly, the heap size in arch-specific pre-boot code
slaxemulator@7650 25 may need to be increased (30 KiB is enough).
slaxemulator@7650 26
slaxemulator@7650 27 The XZ decompressor needs memmove(), memeq() (memcmp() == 0),
slaxemulator@7650 28 and memzero() (memset(ptr, 0, size)), which aren't available in
slaxemulator@7650 29 all arch-specific pre-boot environments. I'm including simple
slaxemulator@7650 30 versions in decompress_unxz.c, but a cleaner solution would
slaxemulator@7650 31 naturally be nicer.
slaxemulator@7650 32
slaxemulator@7650 33 Signed-off-by: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 34 ---
slaxemulator@7650 35
slaxemulator@7650 36 diff --git a/include/linux/decompress/unxz.h b/include/linux/decompress/unxz.h
slaxemulator@7650 37 new file mode 100644
slaxemulator@7650 38 index 0000000..41728fc
slaxemulator@7650 39 --- /dev/null
slaxemulator@7650 40 +++ b/include/linux/decompress/unxz.h
slaxemulator@7650 41 @@ -0,0 +1,19 @@
slaxemulator@7650 42 +/*
slaxemulator@7650 43 + * Wrapper for decompressing XZ-compressed kernel, initramfs, and initrd
slaxemulator@7650 44 + *
slaxemulator@7650 45 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 46 + *
slaxemulator@7650 47 + * This file has been put into the public domain.
slaxemulator@7650 48 + * You can do whatever you want with this file.
slaxemulator@7650 49 + */
slaxemulator@7650 50 +
slaxemulator@7650 51 +#ifndef DECOMPRESS_UNXZ_H
slaxemulator@7650 52 +#define DECOMPRESS_UNXZ_H
slaxemulator@7650 53 +
slaxemulator@7650 54 +int unxz(unsigned char *in, int in_size,
slaxemulator@7650 55 + int (*fill)(void *dest, unsigned int size),
slaxemulator@7650 56 + int (*flush)(void *src, unsigned int size),
slaxemulator@7650 57 + unsigned char *out, int *in_used,
slaxemulator@7650 58 + void (*error)(char *x));
slaxemulator@7650 59 +
slaxemulator@7650 60 +#endif
slaxemulator@7650 61 diff --git a/init/Kconfig b/init/Kconfig
slaxemulator@7650 62 index 2de5b1c..d9fbb0f 100644
slaxemulator@7650 63 --- a/init/Kconfig
slaxemulator@7650 64 +++ b/init/Kconfig
slaxemulator@7650 65 @@ -123,13 +123,16 @@ config HAVE_KERNEL_BZIP2
slaxemulator@7650 66 config HAVE_KERNEL_LZMA
slaxemulator@7650 67 bool
slaxemulator@7650 68
slaxemulator@7650 69 +config HAVE_KERNEL_XZ
slaxemulator@7650 70 + bool
slaxemulator@7650 71 +
slaxemulator@7650 72 config HAVE_KERNEL_LZO
slaxemulator@7650 73 bool
slaxemulator@7650 74
slaxemulator@7650 75 choice
slaxemulator@7650 76 prompt "Kernel compression mode"
slaxemulator@7650 77 default KERNEL_GZIP
slaxemulator@7650 78 - depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_LZO
slaxemulator@7650 79 + depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO
slaxemulator@7650 80 help
slaxemulator@7650 81 The linux kernel is a kind of self-extracting executable.
slaxemulator@7650 82 Several compression algorithms are available, which differ
slaxemulator@7650 83 @@ -174,6 +177,21 @@ config KERNEL_LZMA
slaxemulator@7650 84 two. Compression is slowest. The kernel size is about 33%
slaxemulator@7650 85 smaller with LZMA in comparison to gzip.
slaxemulator@7650 86
slaxemulator@7650 87 +config KERNEL_XZ
slaxemulator@7650 88 + bool "XZ"
slaxemulator@7650 89 + depends on HAVE_KERNEL_XZ
slaxemulator@7650 90 + help
slaxemulator@7650 91 + XZ uses the LZMA2 algorithm and instruction set specific
slaxemulator@7650 92 + BCJ filters which can improve compression ratio of executable
slaxemulator@7650 93 + code. The size of the kernel is about 30% smaller with XZ in
slaxemulator@7650 94 + comparison to gzip. On architectures for which there is a BCJ
slaxemulator@7650 95 + filter (i386, x86_64, ARM, IA-64, PowerPC, and SPARC), XZ
slaxemulator@7650 96 + will create a few percent smaller kernel than plain LZMA.
slaxemulator@7650 97 +
slaxemulator@7650 98 + The speed is about the same as with LZMA: The decompression
slaxemulator@7650 99 + speed of XZ is better than that of bzip2 but worse than gzip
slaxemulator@7650 100 + and LZO. Compression is slow.
slaxemulator@7650 101 +
slaxemulator@7650 102 config KERNEL_LZO
slaxemulator@7650 103 bool "LZO"
slaxemulator@7650 104 depends on HAVE_KERNEL_LZO
slaxemulator@7650 105 diff --git a/lib/Kconfig b/lib/Kconfig
slaxemulator@7650 106 index 6090314..45960e1 100644
slaxemulator@7650 107 --- a/lib/Kconfig
slaxemulator@7650 108 +++ b/lib/Kconfig
slaxemulator@7650 109 @@ -122,6 +122,9 @@ config DECOMPRESS_BZIP2
slaxemulator@7650 110 config DECOMPRESS_LZMA
slaxemulator@7650 111 tristate
slaxemulator@7650 112
slaxemulator@7650 113 +config DECOMPRESS_XZ
slaxemulator@7650 114 + tristate
slaxemulator@7650 115 +
slaxemulator@7650 116 config DECOMPRESS_LZO
slaxemulator@7650 117 select LZO_DECOMPRESS
slaxemulator@7650 118 tristate
slaxemulator@7650 119 diff --git a/lib/Makefile b/lib/Makefile
slaxemulator@7650 120 index f2f98dd..06e3d8a 100644
slaxemulator@7650 121 --- a/lib/Makefile
slaxemulator@7650 122 +++ b/lib/Makefile
slaxemulator@7650 123 @@ -75,6 +75,7 @@ obj-$(CONFIG_RAID6_PQ) += raid6/
slaxemulator@7650 124 lib-$(CONFIG_DECOMPRESS_GZIP) += decompress_inflate.o
slaxemulator@7650 125 lib-$(CONFIG_DECOMPRESS_BZIP2) += decompress_bunzip2.o
slaxemulator@7650 126 lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o
slaxemulator@7650 127 +lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o
slaxemulator@7650 128 lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o
slaxemulator@7650 129
slaxemulator@7650 130 obj-$(CONFIG_TEXTSEARCH) += textsearch.o
slaxemulator@7650 131 diff --git a/lib/decompress.c b/lib/decompress.c
slaxemulator@7650 132 index a760681..3d766b7 100644
slaxemulator@7650 133 --- a/lib/decompress.c
slaxemulator@7650 134 +++ b/lib/decompress.c
slaxemulator@7650 135 @@ -8,6 +8,7 @@
slaxemulator@7650 136
slaxemulator@7650 137 #include <linux/decompress/bunzip2.h>
slaxemulator@7650 138 #include <linux/decompress/unlzma.h>
slaxemulator@7650 139 +#include <linux/decompress/unxz.h>
slaxemulator@7650 140 #include <linux/decompress/inflate.h>
slaxemulator@7650 141 #include <linux/decompress/unlzo.h>
slaxemulator@7650 142
slaxemulator@7650 143 @@ -23,6 +24,9 @@
slaxemulator@7650 144 #ifndef CONFIG_DECOMPRESS_LZMA
slaxemulator@7650 145 # define unlzma NULL
slaxemulator@7650 146 #endif
slaxemulator@7650 147 +#ifndef CONFIG_DECOMPRESS_XZ
slaxemulator@7650 148 +# define unxz NULL
slaxemulator@7650 149 +#endif
slaxemulator@7650 150 #ifndef CONFIG_DECOMPRESS_LZO
slaxemulator@7650 151 # define unlzo NULL
slaxemulator@7650 152 #endif
slaxemulator@7650 153 @@ -36,6 +40,7 @@ static const struct compress_format {
slaxemulator@7650 154 { {037, 0236}, "gzip", gunzip },
slaxemulator@7650 155 { {0x42, 0x5a}, "bzip2", bunzip2 },
slaxemulator@7650 156 { {0x5d, 0x00}, "lzma", unlzma },
slaxemulator@7650 157 + { {0xfd, 0x37}, "xz", unxz },
slaxemulator@7650 158 { {0x89, 0x4c}, "lzo", unlzo },
slaxemulator@7650 159 { {0, 0}, NULL, NULL }
slaxemulator@7650 160 };
slaxemulator@7650 161 diff --git a/lib/decompress_unxz.c b/lib/decompress_unxz.c
slaxemulator@7650 162 new file mode 100644
slaxemulator@7650 163 index 0000000..cecd23d
slaxemulator@7650 164 --- /dev/null
slaxemulator@7650 165 +++ b/lib/decompress_unxz.c
slaxemulator@7650 166 @@ -0,0 +1,397 @@
slaxemulator@7650 167 +/*
slaxemulator@7650 168 + * Wrapper for decompressing XZ-compressed kernel, initramfs, and initrd
slaxemulator@7650 169 + *
slaxemulator@7650 170 + * Author: Lasse Collin <lasse.collin@tukaani.org>
slaxemulator@7650 171 + *
slaxemulator@7650 172 + * This file has been put into the public domain.
slaxemulator@7650 173 + * You can do whatever you want with this file.
slaxemulator@7650 174 + */
slaxemulator@7650 175 +
slaxemulator@7650 176 +/*
slaxemulator@7650 177 + * Important notes about in-place decompression
slaxemulator@7650 178 + *
slaxemulator@7650 179 + * At least on x86, the kernel is decompressed in place: the compressed data
slaxemulator@7650 180 + * is placed to the end of the output buffer, and the decompressor overwrites
slaxemulator@7650 181 + * most of the compressed data. There must be enough safety margin to
slaxemulator@7650 182 + * guarantee that the write position is always behind the read position.
slaxemulator@7650 183 + *
slaxemulator@7650 184 + * The safety margin for XZ with LZMA2 or BCJ+LZMA2 is calculated below.
slaxemulator@7650 185 + * Note that the margin with XZ is bigger than with Deflate (gzip)!
slaxemulator@7650 186 + *
slaxemulator@7650 187 + * The worst case for in-place decompression is that the beginning of
slaxemulator@7650 188 + * the file is compressed extremely well, and the rest of the file is
slaxemulator@7650 189 + * uncompressible. Thus, we must look for worst-case expansion when the
slaxemulator@7650 190 + * compressor is encoding uncompressible data.
slaxemulator@7650 191 + *
slaxemulator@7650 192 + * The structure of the .xz file in case of a compresed kernel is as follows.
slaxemulator@7650 193 + * Sizes (as bytes) of the fields are in parenthesis.
slaxemulator@7650 194 + *
slaxemulator@7650 195 + * Stream Header (12)
slaxemulator@7650 196 + * Block Header:
slaxemulator@7650 197 + * Block Header (8-12)
slaxemulator@7650 198 + * Compressed Data (N)
slaxemulator@7650 199 + * Block Padding (0-3)
slaxemulator@7650 200 + * CRC32 (4)
slaxemulator@7650 201 + * Index (8-20)
slaxemulator@7650 202 + * Stream Footer (12)
slaxemulator@7650 203 + *
slaxemulator@7650 204 + * Normally there is exactly one Block, but let's assume that there are
slaxemulator@7650 205 + * 2-4 Blocks just in case. Because Stream Header and also Block Header
slaxemulator@7650 206 + * of the first Block don't make the decompressor produce any uncompressed
slaxemulator@7650 207 + * data, we can ignore them from our calculations. Block Headers of possible
slaxemulator@7650 208 + * additional Blocks have to be taken into account still. With these
slaxemulator@7650 209 + * assumptions, it is safe to assume that the total header overhead is
slaxemulator@7650 210 + * less than 128 bytes.
slaxemulator@7650 211 + *
slaxemulator@7650 212 + * Compressed Data contains LZMA2 or BCJ+LZMA2 encoded data. Since BCJ
slaxemulator@7650 213 + * doesn't change the size of the data, it is enough to calculate the
slaxemulator@7650 214 + * safety margin for LZMA2.
slaxemulator@7650 215 + *
slaxemulator@7650 216 + * LZMA2 stores the data in chunks. Each chunk has a header whose size is
slaxemulator@7650 217 + * a maximum of 6 bytes, but to get round 2^n numbers, let's assume that
slaxemulator@7650 218 + * the maximum chunk header size is 8 bytes. After the chunk header, there
slaxemulator@7650 219 + * may be up to 64 KiB of actual payload in the chunk. Often the payload is
slaxemulator@7650 220 + * quite a bit smaller though; to be safe, let's assume that an average
slaxemulator@7650 221 + * chunk has only 32 KiB of payload.
slaxemulator@7650 222 + *
slaxemulator@7650 223 + * The maximum uncompressed size of the payload is 2 MiB. The minimum
slaxemulator@7650 224 + * uncompressed size of the payload is in practice never less than the
slaxemulator@7650 225 + * payload size itself. The LZMA2 format would allow uncompressed size
slaxemulator@7650 226 + * to be less than the payload size, but no sane compressor creates such
slaxemulator@7650 227 + * files. LZMA2 supports storing uncompressible data in uncompressed form,
slaxemulator@7650 228 + * so there's never a need to create payloads whose uncompressed size is
slaxemulator@7650 229 + * smaller than the compressed size.
slaxemulator@7650 230 + *
slaxemulator@7650 231 + * The assumption, that the uncompressed size of the payload is never
slaxemulator@7650 232 + * smaller than the payload itself, is valid only when talking about
slaxemulator@7650 233 + * the payload as a whole. It is possible that the payload has parts where
slaxemulator@7650 234 + * the decompressor consumes more input than it produces output. Calculating
slaxemulator@7650 235 + * the worst case for this would be tricky. Instead of trying to do that,
slaxemulator@7650 236 + * let's simply make sure that the decompressor never overwrites any bytes
slaxemulator@7650 237 + * of the payload which it is currently reading.
slaxemulator@7650 238 + *
slaxemulator@7650 239 + * Now we have enough information to calculate the safety margin. We need
slaxemulator@7650 240 + * - 128 bytes for the .xz file format headers;
slaxemulator@7650 241 + * - 8 bytes per every 32 KiB of uncompressed size (one LZMA2 chunk header
slaxemulator@7650 242 + * per chunk, each chunk having average payload size of 32 KiB); and
slaxemulator@7650 243 + * - 64 KiB (biggest possible LZMA2 chunk payload size) to make sure that
slaxemulator@7650 244 + * the decompressor never overwrites anything from the LZMA2 chunk
slaxemulator@7650 245 + * payload it is currently reading.
slaxemulator@7650 246 + *
slaxemulator@7650 247 + * We get the following formula:
slaxemulator@7650 248 + *
slaxemulator@7650 249 + * safety_margin = 128 + uncompressed_size * 8 / 32768 + 65536
slaxemulator@7650 250 + * = 128 + (uncompressed_size >> 12) + 65536
slaxemulator@7650 251 + *
slaxemulator@7650 252 + * For comparision, according to arch/x86/boot/compressed/misc.c, the
slaxemulator@7650 253 + * equivalent formula for Deflate is this:
slaxemulator@7650 254 + *
slaxemulator@7650 255 + * safety_margin = 18 + (uncompressed_size >> 12) + 32768
slaxemulator@7650 256 + *
slaxemulator@7650 257 + * Thus, when updating Deflate-only in-place kernel decompressor to
slaxemulator@7650 258 + * support XZ, the fixed overhead has to be increased from 18+32768 bytes
slaxemulator@7650 259 + * to 128+65536 bytes.
slaxemulator@7650 260 + */
slaxemulator@7650 261 +
slaxemulator@7650 262 +/*
slaxemulator@7650 263 + * STATIC is defined to "static" if we are being built for kernel
slaxemulator@7650 264 + * decompression (pre-boot code). <linux/decompress/mm.h> will define
slaxemulator@7650 265 + * STATIC to empty if it wasn't already defined. Since we will need to
slaxemulator@7650 266 + * know later if we are being used for kernel decompression, we define
slaxemulator@7650 267 + * XZ_PREBOOT here.
slaxemulator@7650 268 + */
slaxemulator@7650 269 +#ifdef STATIC
slaxemulator@7650 270 +# define XZ_PREBOOT
slaxemulator@7650 271 +#endif
slaxemulator@7650 272 +#ifdef __KERNEL__
slaxemulator@7650 273 +# include <linux/decompress/mm.h>
slaxemulator@7650 274 +#endif
slaxemulator@7650 275 +#define XZ_EXTERN STATIC
slaxemulator@7650 276 +
slaxemulator@7650 277 +#ifndef XZ_PREBOOT
slaxemulator@7650 278 +# include <linux/slab.h>
slaxemulator@7650 279 +# include <linux/xz.h>
slaxemulator@7650 280 +#else
slaxemulator@7650 281 +/*
slaxemulator@7650 282 + * Use the internal CRC32 code instead of kernel's CRC32 module, which
slaxemulator@7650 283 + * is not available in early phase of booting.
slaxemulator@7650 284 + */
slaxemulator@7650 285 +#define XZ_INTERNAL_CRC32 1
slaxemulator@7650 286 +
slaxemulator@7650 287 +/*
slaxemulator@7650 288 + * For boot time use, we enable only the BCJ filter of the current
slaxemulator@7650 289 + * architecture or none if no BCJ filter is available for the architecture.
slaxemulator@7650 290 + */
slaxemulator@7650 291 +#ifdef CONFIG_X86
slaxemulator@7650 292 +# define XZ_DEC_X86
slaxemulator@7650 293 +#endif
slaxemulator@7650 294 +#ifdef CONFIG_PPC
slaxemulator@7650 295 +# define XZ_DEC_POWERPC
slaxemulator@7650 296 +#endif
slaxemulator@7650 297 +#ifdef CONFIG_ARM
slaxemulator@7650 298 +# define XZ_DEC_ARM
slaxemulator@7650 299 +#endif
slaxemulator@7650 300 +#ifdef CONFIG_IA64
slaxemulator@7650 301 +# define XZ_DEC_IA64
slaxemulator@7650 302 +#endif
slaxemulator@7650 303 +#ifdef CONFIG_SPARC
slaxemulator@7650 304 +# define XZ_DEC_SPARC
slaxemulator@7650 305 +#endif
slaxemulator@7650 306 +
slaxemulator@7650 307 +/*
slaxemulator@7650 308 + * This will get the basic headers so that memeq() and others
slaxemulator@7650 309 + * can be defined.
slaxemulator@7650 310 + */
slaxemulator@7650 311 +#include "xz/xz_private.h"
slaxemulator@7650 312 +
slaxemulator@7650 313 +/*
slaxemulator@7650 314 + * Replace the normal allocation functions with the versions from
slaxemulator@7650 315 + * <linux/decompress/mm.h>. vfree() needs to support vfree(NULL)
slaxemulator@7650 316 + * when XZ_DYNALLOC is used, but the pre-boot free() doesn't support it.
slaxemulator@7650 317 + * Workaround it here because the other decompressors don't need it.
slaxemulator@7650 318 + */
slaxemulator@7650 319 +#undef kmalloc
slaxemulator@7650 320 +#undef kfree
slaxemulator@7650 321 +#undef vmalloc
slaxemulator@7650 322 +#undef vfree
slaxemulator@7650 323 +#define kmalloc(size, flags) malloc(size)
slaxemulator@7650 324 +#define kfree(ptr) free(ptr)
slaxemulator@7650 325 +#define vmalloc(size) malloc(size)
slaxemulator@7650 326 +#define vfree(ptr) do { if (ptr != NULL) free(ptr); } while (0)
slaxemulator@7650 327 +
slaxemulator@7650 328 +/*
slaxemulator@7650 329 + * FIXME: Not all basic memory functions are provided in architecture-specific
slaxemulator@7650 330 + * files (yet). We define our own versions here for now, but this should be
slaxemulator@7650 331 + * only a temporary solution.
slaxemulator@7650 332 + *
slaxemulator@7650 333 + * memeq and memzero are not used much and any remotely sane implementation
slaxemulator@7650 334 + * is fast enough. memcpy/memmove speed matters in multi-call mode, but
slaxemulator@7650 335 + * the kernel image is decompressed in single-call mode, in which only
slaxemulator@7650 336 + * memcpy speed can matter and only if there is a lot of uncompressible data
slaxemulator@7650 337 + * (LZMA2 stores uncompressible chunks in uncompressed form). Thus, the
slaxemulator@7650 338 + * functions below should just be kept small; it's probably not worth
slaxemulator@7650 339 + * optimizing for speed.
slaxemulator@7650 340 + */
slaxemulator@7650 341 +
slaxemulator@7650 342 +#ifndef memeq
slaxemulator@7650 343 +static bool memeq(const void *a, const void *b, size_t size)
slaxemulator@7650 344 +{
slaxemulator@7650 345 + const uint8_t *x = a;
slaxemulator@7650 346 + const uint8_t *y = b;
slaxemulator@7650 347 + size_t i;
slaxemulator@7650 348 +
slaxemulator@7650 349 + for (i = 0; i < size; ++i)
slaxemulator@7650 350 + if (x[i] != y[i])
slaxemulator@7650 351 + return false;
slaxemulator@7650 352 +
slaxemulator@7650 353 + return true;
slaxemulator@7650 354 +}
slaxemulator@7650 355 +#endif
slaxemulator@7650 356 +
slaxemulator@7650 357 +#ifndef memzero
slaxemulator@7650 358 +static void memzero(void *buf, size_t size)
slaxemulator@7650 359 +{
slaxemulator@7650 360 + uint8_t *b = buf;
slaxemulator@7650 361 + uint8_t *e = b + size;
slaxemulator@7650 362 +
slaxemulator@7650 363 + while (b != e)
slaxemulator@7650 364 + *b++ = '\0';
slaxemulator@7650 365 +}
slaxemulator@7650 366 +#endif
slaxemulator@7650 367 +
slaxemulator@7650 368 +#ifndef memmove
slaxemulator@7650 369 +/* Not static to avoid a conflict with the prototype in the Linux headers. */
slaxemulator@7650 370 +void *memmove(void *dest, const void *src, size_t size)
slaxemulator@7650 371 +{
slaxemulator@7650 372 + uint8_t *d = dest;
slaxemulator@7650 373 + const uint8_t *s = src;
slaxemulator@7650 374 + size_t i;
slaxemulator@7650 375 +
slaxemulator@7650 376 + if (d < s) {
slaxemulator@7650 377 + for (i = 0; i < size; ++i)
slaxemulator@7650 378 + d[i] = s[i];
slaxemulator@7650 379 + } else if (d > s) {
slaxemulator@7650 380 + i = size;
slaxemulator@7650 381 + while (i-- > 0)
slaxemulator@7650 382 + d[i] = s[i];
slaxemulator@7650 383 + }
slaxemulator@7650 384 +
slaxemulator@7650 385 + return dest;
slaxemulator@7650 386 +}
slaxemulator@7650 387 +#endif
slaxemulator@7650 388 +
slaxemulator@7650 389 +/*
slaxemulator@7650 390 + * Since we need memmove anyway, would use it as memcpy too.
slaxemulator@7650 391 + * Commented out for now to avoid breaking things.
slaxemulator@7650 392 + */
slaxemulator@7650 393 +/*
slaxemulator@7650 394 +#ifndef memcpy
slaxemulator@7650 395 +# define memcpy memmove
slaxemulator@7650 396 +#endif
slaxemulator@7650 397 +*/
slaxemulator@7650 398 +
slaxemulator@7650 399 +#include "xz/xz_crc32.c"
slaxemulator@7650 400 +#include "xz/xz_dec_stream.c"
slaxemulator@7650 401 +#include "xz/xz_dec_lzma2.c"
slaxemulator@7650 402 +#include "xz/xz_dec_bcj.c"
slaxemulator@7650 403 +
slaxemulator@7650 404 +#endif /* XZ_PREBOOT */
slaxemulator@7650 405 +
slaxemulator@7650 406 +/* Size of the input and output buffers in multi-call mode */
slaxemulator@7650 407 +#define XZ_IOBUF_SIZE 4096
slaxemulator@7650 408 +
slaxemulator@7650 409 +/*
slaxemulator@7650 410 + * This function implements the API defined in <linux/decompress/generic.h>.
slaxemulator@7650 411 + *
slaxemulator@7650 412 + * This wrapper will automatically choose single-call or multi-call mode
slaxemulator@7650 413 + * of the native XZ decoder API. The single-call mode can be used only when
slaxemulator@7650 414 + * both input and output buffers are available as a single chunk, i.e. when
slaxemulator@7650 415 + * fill() and flush() won't be used.
slaxemulator@7650 416 + */
slaxemulator@7650 417 +STATIC int INIT unxz(unsigned char *in, int in_size,
slaxemulator@7650 418 + int (*fill)(void *dest, unsigned int size),
slaxemulator@7650 419 + int (*flush)(void *src, unsigned int size),
slaxemulator@7650 420 + unsigned char *out, int *in_used,
slaxemulator@7650 421 + void (*error)(char *x))
slaxemulator@7650 422 +{
slaxemulator@7650 423 + struct xz_buf b;
slaxemulator@7650 424 + struct xz_dec *s;
slaxemulator@7650 425 + enum xz_ret ret;
slaxemulator@7650 426 + bool must_free_in = false;
slaxemulator@7650 427 +
slaxemulator@7650 428 +#if XZ_INTERNAL_CRC32
slaxemulator@7650 429 + xz_crc32_init();
slaxemulator@7650 430 +#endif
slaxemulator@7650 431 +
slaxemulator@7650 432 + if (in_used != NULL)
slaxemulator@7650 433 + *in_used = 0;
slaxemulator@7650 434 +
slaxemulator@7650 435 + if (fill == NULL && flush == NULL)
slaxemulator@7650 436 + s = xz_dec_init(XZ_SINGLE, 0);
slaxemulator@7650 437 + else
slaxemulator@7650 438 + s = xz_dec_init(XZ_DYNALLOC, (uint32_t)-1);
slaxemulator@7650 439 +
slaxemulator@7650 440 + if (s == NULL)
slaxemulator@7650 441 + goto error_alloc_state;
slaxemulator@7650 442 +
slaxemulator@7650 443 + if (flush == NULL) {
slaxemulator@7650 444 + b.out = out;
slaxemulator@7650 445 + b.out_size = (size_t)-1;
slaxemulator@7650 446 + } else {
slaxemulator@7650 447 + b.out_size = XZ_IOBUF_SIZE;
slaxemulator@7650 448 + b.out = malloc(XZ_IOBUF_SIZE);
slaxemulator@7650 449 + if (b.out == NULL)
slaxemulator@7650 450 + goto error_alloc_out;
slaxemulator@7650 451 + }
slaxemulator@7650 452 +
slaxemulator@7650 453 + if (in == NULL) {
slaxemulator@7650 454 + must_free_in = true;
slaxemulator@7650 455 + in = malloc(XZ_IOBUF_SIZE);
slaxemulator@7650 456 + if (in == NULL)
slaxemulator@7650 457 + goto error_alloc_in;
slaxemulator@7650 458 + }
slaxemulator@7650 459 +
slaxemulator@7650 460 + b.in = in;
slaxemulator@7650 461 + b.in_pos = 0;
slaxemulator@7650 462 + b.in_size = in_size;
slaxemulator@7650 463 + b.out_pos = 0;
slaxemulator@7650 464 +
slaxemulator@7650 465 + if (fill == NULL && flush == NULL) {
slaxemulator@7650 466 + ret = xz_dec_run(s, &b);
slaxemulator@7650 467 + } else {
slaxemulator@7650 468 + do {
slaxemulator@7650 469 + if (b.in_pos == b.in_size && fill != NULL) {
slaxemulator@7650 470 + if (in_used != NULL)
slaxemulator@7650 471 + *in_used += b.in_pos;
slaxemulator@7650 472 +
slaxemulator@7650 473 + b.in_pos = 0;
slaxemulator@7650 474 +
slaxemulator@7650 475 + in_size = fill(in, XZ_IOBUF_SIZE);
slaxemulator@7650 476 + if (in_size < 0) {
slaxemulator@7650 477 + /*
slaxemulator@7650 478 + * This isn't an optimal error code
slaxemulator@7650 479 + * but it probably isn't worth making
slaxemulator@7650 480 + * a new one either.
slaxemulator@7650 481 + */
slaxemulator@7650 482 + ret = XZ_BUF_ERROR;
slaxemulator@7650 483 + break;
slaxemulator@7650 484 + }
slaxemulator@7650 485 +
slaxemulator@7650 486 + b.in_size = in_size;
slaxemulator@7650 487 + }
slaxemulator@7650 488 +
slaxemulator@7650 489 + ret = xz_dec_run(s, &b);
slaxemulator@7650 490 +
slaxemulator@7650 491 + if (flush != NULL && (b.out_pos == b.out_size
slaxemulator@7650 492 + || (ret != XZ_OK && b.out_pos > 0))) {
slaxemulator@7650 493 + /*
slaxemulator@7650 494 + * Setting ret here may hide an error
slaxemulator@7650 495 + * returned by xz_dec_run(), but probably
slaxemulator@7650 496 + * it's not too bad.
slaxemulator@7650 497 + */
slaxemulator@7650 498 + if (flush(b.out, b.out_pos) != (int)b.out_pos)
slaxemulator@7650 499 + ret = XZ_BUF_ERROR;
slaxemulator@7650 500 +
slaxemulator@7650 501 + b.out_pos = 0;
slaxemulator@7650 502 + }
slaxemulator@7650 503 + } while (ret == XZ_OK);
slaxemulator@7650 504 +
slaxemulator@7650 505 + if (must_free_in)
slaxemulator@7650 506 + free(in);
slaxemulator@7650 507 +
slaxemulator@7650 508 + if (flush != NULL)
slaxemulator@7650 509 + free(b.out);
slaxemulator@7650 510 + }
slaxemulator@7650 511 +
slaxemulator@7650 512 + if (in_used != NULL)
slaxemulator@7650 513 + *in_used += b.in_pos;
slaxemulator@7650 514 +
slaxemulator@7650 515 + xz_dec_end(s);
slaxemulator@7650 516 +
slaxemulator@7650 517 + switch (ret) {
slaxemulator@7650 518 + case XZ_STREAM_END:
slaxemulator@7650 519 + return 0;
slaxemulator@7650 520 +
slaxemulator@7650 521 + case XZ_MEM_ERROR:
slaxemulator@7650 522 + /* This can occur only in multi-call mode. */
slaxemulator@7650 523 + error("XZ decompressor ran out of memory");
slaxemulator@7650 524 + break;
slaxemulator@7650 525 +
slaxemulator@7650 526 + case XZ_FORMAT_ERROR:
slaxemulator@7650 527 + error("Input is not in the XZ format (wrong magic bytes)");
slaxemulator@7650 528 + break;
slaxemulator@7650 529 +
slaxemulator@7650 530 + case XZ_OPTIONS_ERROR:
slaxemulator@7650 531 + error("Input was encoded with settings that are not "
slaxemulator@7650 532 + "supported by this XZ decoder");
slaxemulator@7650 533 + break;
slaxemulator@7650 534 +
slaxemulator@7650 535 + case XZ_DATA_ERROR:
slaxemulator@7650 536 + case XZ_BUF_ERROR:
slaxemulator@7650 537 + error("XZ-compressed data is corrupt");
slaxemulator@7650 538 + break;
slaxemulator@7650 539 +
slaxemulator@7650 540 + default:
slaxemulator@7650 541 + error("Bug in the XZ decompressor");
slaxemulator@7650 542 + break;
slaxemulator@7650 543 + }
slaxemulator@7650 544 +
slaxemulator@7650 545 + return -1;
slaxemulator@7650 546 +
slaxemulator@7650 547 +error_alloc_in:
slaxemulator@7650 548 + if (flush != NULL)
slaxemulator@7650 549 + free(b.out);
slaxemulator@7650 550 +
slaxemulator@7650 551 +error_alloc_out:
slaxemulator@7650 552 + xz_dec_end(s);
slaxemulator@7650 553 +
slaxemulator@7650 554 +error_alloc_state:
slaxemulator@7650 555 + error("XZ decompressor ran out of memory");
slaxemulator@7650 556 + return -1;
slaxemulator@7650 557 +}
slaxemulator@7650 558 +
slaxemulator@7650 559 +/*
slaxemulator@7650 560 + * This macro is used by architecture-specific files to decompress
slaxemulator@7650 561 + * the kernel image.
slaxemulator@7650 562 + */
slaxemulator@7650 563 +#define decompress unxz
slaxemulator@7650 564 diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
slaxemulator@7650 565 index 5958fff..55caecd 100644
slaxemulator@7650 566 --- a/scripts/gen_initramfs_list.sh
slaxemulator@7650 567 +++ b/scripts/gen_initramfs_list.sh
slaxemulator@7650 568 @@ -243,6 +243,8 @@ case "$arg" in
slaxemulator@7650 569 echo "$output_file" | grep -q "\.gz$" && compr="gzip -9 -f"
slaxemulator@7650 570 echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f"
slaxemulator@7650 571 echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f"
slaxemulator@7650 572 + echo "$output_file" | grep -q "\.xz$" && \
slaxemulator@7650 573 + compr="xz --check=crc32 --lzma2=dict=1MiB"
slaxemulator@7650 574 echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
slaxemulator@7650 575 echo "$output_file" | grep -q "\.cpio$" && compr="cat"
slaxemulator@7650 576 shift
slaxemulator@7650 577 diff --git a/usr/Kconfig b/usr/Kconfig
slaxemulator@7650 578 index e2721f5..9f51a29 100644
slaxemulator@7650 579 --- a/usr/Kconfig
slaxemulator@7650 580 +++ b/usr/Kconfig
slaxemulator@7650 581 @@ -72,6 +72,18 @@ config RD_LZMA
slaxemulator@7650 582 Support loading of a LZMA encoded initial ramdisk or cpio buffer
slaxemulator@7650 583 If unsure, say N.
slaxemulator@7650 584
slaxemulator@7650 585 +config RD_XZ
slaxemulator@7650 586 + bool "Support initial ramdisks compressed using XZ"
slaxemulator@7650 587 + depends on BLK_DEV_INITRD && XZ_DEC=y
slaxemulator@7650 588 + select DECOMPRESS_XZ
slaxemulator@7650 589 + help
slaxemulator@7650 590 + Support loading of a XZ encoded initial ramdisk or cpio buffer.
slaxemulator@7650 591 +
slaxemulator@7650 592 + If this option is inactive, say Y to "XZ decompression support"
slaxemulator@7650 593 + under "Library routines" first.
slaxemulator@7650 594 +
slaxemulator@7650 595 + If unsure, say N.
slaxemulator@7650 596 +
slaxemulator@7650 597 config RD_LZO
slaxemulator@7650 598 bool "Support initial ramdisks compressed using LZO" if EMBEDDED
slaxemulator@7650 599 default !EMBEDDED
slaxemulator@7650 600 @@ -139,6 +151,15 @@ config INITRAMFS_COMPRESSION_LZMA
slaxemulator@7650 601 three. Compression is slowest. The initramfs size is about 33%
slaxemulator@7650 602 smaller with LZMA in comparison to gzip.
slaxemulator@7650 603
slaxemulator@7650 604 +config INITRAMFS_COMPRESSION_XZ
slaxemulator@7650 605 + bool "XZ"
slaxemulator@7650 606 + depends on RD_XZ
slaxemulator@7650 607 + help
slaxemulator@7650 608 + XZ uses the LZMA2 algorithm. The initramfs size is about 30%
slaxemulator@7650 609 + smaller with XZ in comparison to gzip. Decompression speed
slaxemulator@7650 610 + is better than that of bzip2 but worse than gzip and LZO.
slaxemulator@7650 611 + Compression is slow.
slaxemulator@7650 612 +
slaxemulator@7650 613 config INITRAMFS_COMPRESSION_LZO
slaxemulator@7650 614 bool "LZO"
slaxemulator@7650 615 depends on RD_LZO
slaxemulator@7650 616 diff --git a/usr/Makefile b/usr/Makefile
slaxemulator@7650 617 index 6b4b6da..5845a13 100644
slaxemulator@7650 618 --- a/usr/Makefile
slaxemulator@7650 619 +++ b/usr/Makefile
slaxemulator@7650 620 @@ -15,6 +15,9 @@ suffix_$(CONFIG_INITRAMFS_COMPRESSION_BZIP2) = .bz2
slaxemulator@7650 621 # Lzma
slaxemulator@7650 622 suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZMA) = .lzma
slaxemulator@7650 623
slaxemulator@7650 624 +# XZ
slaxemulator@7650 625 +suffix_$(CONFIG_INITRAMFS_COMPRESSION_XZ) = .xz
slaxemulator@7650 626 +
slaxemulator@7650 627 # Lzo
slaxemulator@7650 628 suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZO) = .lzo
slaxemulator@7650 629
slaxemulator@7650 630 @@ -48,7 +51,7 @@ endif
slaxemulator@7650 631 quiet_cmd_initfs = GEN $@
slaxemulator@7650 632 cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
slaxemulator@7650 633
slaxemulator@7650 634 -targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.lzo initramfs_data.cpio
slaxemulator@7650 635 +targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.xz initramfs_data.cpio.lzo initramfs_data.cpio
slaxemulator@7650 636 # do not try to update files included in initramfs
slaxemulator@7650 637 $(deps_initramfs): ;
slaxemulator@7650 638