summaryrefslogtreecommitdiff
path: root/src/static_libs
diff options
context:
space:
mode:
authorStefan Schmidt <s.schmidt@samsung.com>2015-05-07 10:52:54 +0200
committerStefan Schmidt <s.schmidt@samsung.com>2015-05-07 11:15:13 +0200
commit4314257d8cf9c367cab2333cfdc7b10b0109de90 (patch)
treeaf2526ecae97b7cbfb5513b6a4cfebbc3da98935 /src/static_libs
parent6d87ac299158702a6a98f015d2096f8b69fcc06a (diff)
lz4: Update our internal copy to release r128
Looking through the git log it is unclear which release we used before as nobody stated it there. :/ We updated after the security issues last year so my best guess is that we have something like r119. To see what changed I now included the NEWS file and also the LICENSE file from upstream. Upstream in now hosted here: https://github.com/Cyan4973/lz4 and http://www.lz4.info I recommend STRONGLY that you check if your distro ships liblz4 as an up to date library package and use the --enable-liblz4 configure option to use the system version. I consider making the system version default for upcoming releases and only carry the internal one as fallback for systems that do not provide it. Fix T2374
Diffstat (limited to 'src/static_libs')
-rw-r--r--src/static_libs/lz4/LICENSE24
-rw-r--r--src/static_libs/lz4/NEWS153
-rw-r--r--src/static_libs/lz4/README7
-rw-r--r--src/static_libs/lz4/README.md19
-rw-r--r--src/static_libs/lz4/lz4.c918
-rw-r--r--src/static_libs/lz4/lz4.h208
-rw-r--r--src/static_libs/lz4/lz4hc.c886
-rw-r--r--src/static_libs/lz4/lz4hc.h125
8 files changed, 1265 insertions, 1075 deletions
diff --git a/src/static_libs/lz4/LICENSE b/src/static_libs/lz4/LICENSE
new file mode 100644
index 0000000..b566df3
--- /dev/null
+++ b/src/static_libs/lz4/LICENSE
@@ -0,0 +1,24 @@
1LZ4 Library
2Copyright (c) 2011-2014, Yann Collet
3All rights reserved.
4
5Redistribution and use in source and binary forms, with or without modification,
6are permitted provided that the following conditions are met:
7
8* Redistributions of source code must retain the above copyright notice, this
9 list of conditions and the following disclaimer.
10
11* Redistributions in binary form must reproduce the above copyright notice, this
12 list of conditions and the following disclaimer in the documentation and/or
13 other materials provided with the distribution.
14
15THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
16ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
18DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
19ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
20(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
21LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
22ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
23(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
24SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. \ No newline at end of file
diff --git a/src/static_libs/lz4/NEWS b/src/static_libs/lz4/NEWS
new file mode 100644
index 0000000..2eeb948
--- /dev/null
+++ b/src/static_libs/lz4/NEWS
@@ -0,0 +1,153 @@
1r128:
2New : lz4cli sparse file support
3New : command -m, to compress multiple files in a single command
4Fixed : Restored lz4hc compression ratio (was slightly lower since r124)
5New : lz4 cli supports long commands
6New : lz4frame & lz4cli frame content size support
7New : lz4frame supports skippable frames
8Changed:Default "make install" directory is /usr/local
9New : lz4 cli supports "pass-through" mode
10New : datagen can generate sparse files
11New : scan-build tests
12New : g++ compatibility tests
13New : arm cross-compilation test
14Fixed : Fuzzer + frametest compatibility with NetBSD (issue #48)
15Added : Visual project directory
16Updated:Man page & Specification
17
18r127:
19N/A : added a file on SVN
20
21r126:
22New : lz4frame API is now integrated into liblz4
23Fixed : GCC 4.9 bug on highest performance settings, reported by Greg Slazinski
24Fixed : bug within LZ4 HC streaming mode, reported by James Boyle
25Fixed : older compiler don't like nameless unions, reported by Cheyi Lin
26Changed : lz4 is C90 compatible
27Changed : added -pedantic option, fixed a few mminor warnings
28
29r125:
30Changed : endian and alignment code
31Changed : directory structure : new "lib" directory
32Updated : lz4io, now uses lz4frame
33Improved: slightly improved decoding speed
34Fixed : LZ4_compress_limitedOutput(); Special thanks to Christopher Speller !
35Fixed : some alignment warnings under clang
36Fixed : deprecated function LZ4_slideInputBufferHC()
37
38r124:
39New : LZ4 HC streaming mode
40Fixed : LZ4F_compressBound() using null preferencesPtr
41Updated : xxHash to r38
42Updated library number, to 1.4.0
43
44r123:
45Added : experimental lz4frame API, thanks to Takayuki Matsuoka and Christopher Jackson for testings
46Fix : s390x support, thanks to Nobuhiro Iwamatsu
47Fix : test mode (-t) no longer requires confirmation, thanks to Thary Nguyen
48
49r122:
50Fix : AIX & AIX64 support (SamG)
51Fix : mips 64-bits support (lew van)
52Added : Examples directory, using code examples from Takayuki Matsuoka
53Updated : Framing specification, to v1.4.1
54Updated : xxHash, to r36
55
56r121:
57Added : Makefile : install for kFreeBSD and Hurd (Nobuhiro Iwamatsu)
58Fix : Makefile : install for OS-X and BSD, thanks to Takayuki Matsuoka
59
60r120:
61Modified : Streaming API, using strong types
62Added : LZ4_versionNumber(), thanks to Takayuki Matsuoka
63Fix : OS-X : library install name, thanks to Clemens Lang
64Updated : Makefile : synchronize library version number with lz4.h, thanks to Takayuki Matsuoka
65Updated : Makefile : stricter compilation flags
66Added : pkg-config, thanks to Zbigniew Jędrzejewski-Szmek (issue 135)
67Makefile : lz4-test only test native binaries, as suggested by Michał Górny (issue 136)
68Updated : xxHash to r35
69
70r119:
71Fix : Issue 134 : extended malicious address space overflow in 32-bits mode for some specific configurations
72
73r118:
74New : LZ4 Streaming API (Fast version), special thanks to Takayuki Matsuoka
75New : datagen : parametrable synthetic data generator for tests
76Improved : fuzzer, support more test cases, more parameters, ability to jump to specific test
77fix : support ppc64le platform (issue 131)
78fix : Issue 52 (malicious address space overflow in 32-bits mode when using large custom format)
79fix : Makefile : minor issue 130 : header files permissions
80
81r117:
82Added : man pages for lz4c and lz4cat
83Added : automated tests on Travis, thanks to Takayuki Matsuoka !
84fix : block-dependency command line (issue 127)
85fix : lz4fullbench (issue 128)
86
87r116:
88hotfix (issue 124 & 125)
89
90r115:
91Added : lz4cat utility, installed on POSX systems (issue 118)
92OS-X compatible compilation of dynamic library (issue 115)
93
94r114:
95Makefile : library correctly compiled with -O3 switch (issue 114)
96Makefile : library compilation compatible with clang
97Makefile : library is versioned and linked (issue 119)
98lz4.h : no more static inline prototypes (issue 116)
99man : improved header/footer (issue 111)
100Makefile : Use system default $(CC) & $(MAKE) variables (issue 112)
101xxhash : updated to r34
102
103r113:
104Large decompression speed improvement for GCC 32-bits. Thanks to Valery Croizier !
105LZ4HC : Compression Level is now a programmable parameter (CLI from 4 to 9)
106Separated IO routines from command line (lz4io.c)
107Version number into lz4.h (suggested by Francesc Alted)
108
109r112:
110quickfix
111
112r111 :
113Makefile : added capability to install libraries
114Modified Directory tree, to better separate libraries from programs.
115
116r110 :
117lz4 & lz4hc : added capability to allocate state & stream state with custom allocator (issue 99)
118fuzzer & fullbench : updated to test new functions
119man : documented -l command (Legacy format, for Linux kernel compression) (issue 102)
120cmake : improved version by Mika Attila, building programs and libraries (issue 100)
121xxHash : updated to r33
122Makefile : clean also delete local package .tar.gz
123
124r109 :
125lz4.c : corrected issue 98 (LZ4_compress_limitedOutput())
126Makefile : can specify version number from makefile
127
128r108 :
129lz4.c : corrected compression efficiency issue 97 in 64-bits chained mode (-BD) for streams > 4 GB (thanks Roman Strashkin for reporting)
130
131r107 :
132Makefile : support DESTDIR for staged installs. Thanks Jorge Aparicio.
133Makefile : make install installs both lz4 and lz4c (Jorge Aparicio)
134Makefile : removed -Wno-implicit-declaration compilation switch
135lz4cli.c : include <stduni.h> for isatty() (Luca Barbato)
136lz4.h : introduced LZ4_MAX_INPUT_SIZE constant (Shay Green)
137lz4.h : LZ4_compressBound() : unified macro and inline definitions (Shay Green)
138lz4.h : LZ4_decompressSafe_partial() : clarify comments (Shay Green)
139lz4.c : LZ4_compress() verify input size condition (Shay Green)
140bench.c : corrected a bug in free memory size evaluation
141cmake : install into bin/ directory (Richard Yao)
142cmake : check for just C compiler (Elan Ruusamae)
143
144r106 :
145Makefile : make dist modify text files in the package to respect Unix EoL convention
146lz4cli.c : corrected small display bug in HC mode
147
148r105 :
149Makefile : New install script and man page, contributed by Prasad Pandit
150lz4cli.c : Minor modifications, for easier extensibility
151COPYING : added license file
152LZ4_Streaming_Format.odt : modified file name to remove white space characters
153Makefile : .exe suffix now properly added only for Windows target
diff --git a/src/static_libs/lz4/README b/src/static_libs/lz4/README
deleted file mode 100644
index 718f773..0000000
--- a/src/static_libs/lz4/README
+++ /dev/null
@@ -1,7 +0,0 @@
1This iz the lz4 tree copied in:
2 http://lz4.googlecode.com/svn/trunk
3by:
4 yann.collet.73@gmail.com
5Copyright/licensing info in source files here.
6
7this was from revision 84.
diff --git a/src/static_libs/lz4/README.md b/src/static_libs/lz4/README.md
new file mode 100644
index 0000000..636c8be
--- /dev/null
+++ b/src/static_libs/lz4/README.md
@@ -0,0 +1,19 @@
1LZ4 - Library Files
2================================
3
4This directory contains many files, but you don't necessarily need them all.
5
6If you want to integrate LZ4 compression/decompression into your program, you basically need to include "**lz4.c**" and "**lz4.h**" only.
7
8If you want more compression, at the cost of compression speed (but preserving decompression speed), you will also have to include "**lz4hc.c**" and "**lz4hc.h**". Note that lz4hc needs lz4 to work properly.
9
10Next level, if you want to produce files or data streams compatible with lz4 utility, you will have to use and include "**lz4frame.c**" and **lz4frame.h**". This library encapsulate lz4-compressed blocks into the official interoperable frame format. In order to work properly, lz4frame needs lz4 and lz4hc, and also "**xxhash.c**" and "**xxhash.h**", which provide the error detection algorithm.
11
12A more complex "lz4frame_static.h" is also provided, although its usage is not recommended. It contains definitions which are not guaranteed to remain stable within future versions. Use only if you don't plan to update your lz4 version.
13
14The other files are not source code. There are :
15
16 - LICENSE : contains the BSD license text
17 - Makefile : script to compile or install lz4 library (static or dynamic)
18 - liblz4.pc.in : for pkg-config (make install)
19
diff --git a/src/static_libs/lz4/lz4.c b/src/static_libs/lz4/lz4.c
index 482a8ed..881d1af 100644
--- a/src/static_libs/lz4/lz4.c
+++ b/src/static_libs/lz4/lz4.c
@@ -1,6 +1,7 @@
1/* 1/*
2 LZ4 - Fast LZ compression algorithm 2 LZ4 - Fast LZ compression algorithm
3 Copyright (C) 2011-2014, Yann Collet. 3 Copyright (C) 2011-2015, Yann Collet.
4
4 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) 5 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
5 6
6 Redistribution and use in source and binary forms, with or without 7 Redistribution and use in source and binary forms, with or without
@@ -27,82 +28,77 @@
27 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
28 29
29 You can contact the author at : 30 You can contact the author at :
30 - LZ4 source repository : http://code.google.com/p/lz4/ 31 - LZ4 source repository : https://github.com/Cyan4973/lz4
31 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c 32 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
32*/ 33*/
33 34
35
34/************************************** 36/**************************************
35 Tuning parameters 37 Tuning parameters
36**************************************/ 38**************************************/
37/* 39/*
38 * HEAPMODE : 40 * HEAPMODE :
39 * Select how default compression functions will allocate memory for their hash table, 41 * Select how default compression functions will allocate memory for their hash table,
40 * in memory stack (0:default, fastest), or in memory heap (1:requires memory allocation (malloc)). 42 * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
41 */ 43 */
42#define HEAPMODE 0 44#define HEAPMODE 0
43 45
46/*
47 * CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS :
48 * By default, the source code expects the compiler to correctly optimize
49 * 4-bytes and 8-bytes read on architectures able to handle it efficiently.
50 * This is not always the case. In some circumstances (ARM notably),
51 * the compiler will issue cautious code even when target is able to correctly handle unaligned memory accesses.
52 *
53 * You can force the compiler to use unaligned memory access by uncommenting the line below.
54 * One of the below scenarios will happen :
55 * 1 - Your target CPU correctly handle unaligned access, and was not well optimized by compiler (good case).
56 * You will witness large performance improvements (+50% and up).
57 * Keep the line uncommented and send a word to upstream (https://groups.google.com/forum/#!forum/lz4c)
58 * The goal is to automatically detect such situations by adding your target CPU within an exception list.
59 * 2 - Your target CPU correctly handle unaligned access, and was already already optimized by compiler
60 * No change will be experienced.
61 * 3 - Your target CPU inefficiently handle unaligned access.
62 * You will experience a performance loss. Comment back the line.
63 * 4 - Your target CPU does not handle unaligned access.
64 * Program will crash.
65 * If uncommenting results in better performance (case 1)
66 * please report your configuration to upstream (https://groups.google.com/forum/#!forum/lz4c)
67 * This way, an automatic detection macro can be added to match your case within later versions of the library.
68 */
69/* #define CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS 1 */
70
44 71
45/************************************** 72/**************************************
46 CPU Feature Detection 73 CPU Feature Detection
47**************************************/ 74**************************************/
48/* 32 or 64 bits ? */
49#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
50 || defined(__powerpc64__) || defined(__powerpc64le__) \
51 || defined(__ppc64__) || defined(__ppc64le__) \
52 || defined(__PPC64__) || defined(__PPC64LE__) \
53 || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */
54# define LZ4_ARCH64 1
55#else
56# define LZ4_ARCH64 0
57#endif
58
59/* 75/*
60 * Little Endian or Big Endian ? 76 * Automated efficient unaligned memory access detection
61 * Overwrite the #define below if you know your architecture endianess 77 * Based on known hardware architectures
78 * This list will be updated thanks to feedbacks
62 */ 79 */
63#include <stdlib.h> /* Apparently required to detect endianess */ 80#if defined(CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS) \
64#if defined (__GLIBC__) 81 || defined(__ARM_FEATURE_UNALIGNED) \
65# include <endian.h> 82 || defined(__i386__) || defined(__x86_64__) \
66# if (__BYTE_ORDER == __BIG_ENDIAN) 83 || defined(_M_IX86) || defined(_M_X64) \
67# define LZ4_BIG_ENDIAN 1 84 || defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_8__) \
68# endif 85 || (defined(_M_ARM) && (_M_ARM >= 7))
69#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN)) 86# define LZ4_UNALIGNED_ACCESS 1
70# define LZ4_BIG_ENDIAN 1
71#elif defined(__sparc) || defined(__sparc__) \
72 || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
73 || defined(__hpux) || defined(__hppa) \
74 || defined(_MIPSEB) || defined(__s390__)
75# define LZ4_BIG_ENDIAN 1
76#else 87#else
77/* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */ 88# define LZ4_UNALIGNED_ACCESS 0
78#endif 89#endif
79 90
80/* 91/*
81 * Unaligned memory access is automatically enabled for "common" CPU, such as x86. 92 * LZ4_FORCE_SW_BITCOUNT
82 * For others CPU, such as ARM, the compiler may be more cautious, inserting unnecessary extra code to ensure aligned access property 93 * Define this parameter if your target system or compiler does not support hardware bit count
83 * If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance
84 */ 94 */
85#if defined(__ARM_FEATURE_UNALIGNED)
86# define LZ4_FORCE_UNALIGNED_ACCESS 1
87#endif
88
89/* Define this parameter if your target system or compiler does not support hardware bit count */
90#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */ 95#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */
91# define LZ4_FORCE_SW_BITCOUNT 96# define LZ4_FORCE_SW_BITCOUNT
92#endif 97#endif
93 98
94/*
95 * BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE :
96 * This option may provide a small boost to performance for some big endian cpu, although probably modest.
97 * You may set this option to 1 if data will remain within closed environment.
98 * This option is useless on Little_Endian CPU (such as x86)
99 */
100
101/* #define BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE 1 */
102
103 99
104/************************************** 100/**************************************
105 Compiler Options 101* Compiler Options
106**************************************/ 102**************************************/
107#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ 103#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
108/* "restrict" is a known keyword */ 104/* "restrict" is a known keyword */
@@ -112,28 +108,20 @@
112 108
113#ifdef _MSC_VER /* Visual Studio */ 109#ifdef _MSC_VER /* Visual Studio */
114# define FORCE_INLINE static __forceinline 110# define FORCE_INLINE static __forceinline
115# include <intrin.h> /* For Visual 2005 */ 111# include <intrin.h>
116# if LZ4_ARCH64 /* 64-bits */
117# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
118# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
119# else /* 32-bits */
120# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
121# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
122# endif
123# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ 112# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
113# pragma warning(disable : 4293) /* disable: C4293: too large shift (32-bits) */
124#else 114#else
125# ifdef __GNUC__ 115# if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
126# define FORCE_INLINE static inline __attribute__((always_inline)) 116# ifdef __GNUC__
117# define FORCE_INLINE static inline __attribute__((always_inline))
118# else
119# define FORCE_INLINE static inline
120# endif
127# else 121# else
128# define FORCE_INLINE static inline 122# define FORCE_INLINE static
129# endif 123# endif /* __STDC_VERSION__ */
130#endif 124#endif /* _MSC_VER */
131
132#ifdef _MSC_VER /* Visual Studio */
133# define lz4_bswap16(x) _byteswap_ushort(x)
134#else
135# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8)))
136#endif
137 125
138#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) 126#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
139 127
@@ -181,46 +169,132 @@
181 typedef unsigned long long U64; 169 typedef unsigned long long U64;
182#endif 170#endif
183 171
184#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS)
185# define _PACKED __attribute__ ((packed))
186#else
187# define _PACKED
188#endif
189 172
190#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) 173/**************************************
191# if defined(__IBMC__) || defined(__SUNPRO_C) || defined(__SUNPRO_CC) 174 Reading and writing into memory
192# pragma pack(1) 175**************************************/
193# else 176#define STEPSIZE sizeof(size_t)
194# pragma pack(push, 1)
195# endif
196#endif
197 177
198typedef struct { U16 v; } _PACKED U16_S; 178static unsigned LZ4_64bits(void) { return sizeof(void*)==8; }
199typedef struct { U32 v; } _PACKED U32_S;
200typedef struct { U64 v; } _PACKED U64_S;
201typedef struct {size_t v;} _PACKED size_t_S;
202 179
203#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) 180static unsigned LZ4_isLittleEndian(void)
204# if defined(__SUNPRO_C) || defined(__SUNPRO_CC) 181{
205# pragma pack(0) 182 const union { U32 i; BYTE c[4]; } one = { 1 }; /* don't use static : performance detrimental */
206# else 183 return one.c[0];
207# pragma pack(pop) 184}
208# endif 185
186
187static U16 LZ4_readLE16(const void* memPtr)
188{
189 if ((LZ4_UNALIGNED_ACCESS) && (LZ4_isLittleEndian()))
190 return *(U16*)memPtr;
191 else
192 {
193 const BYTE* p = (const BYTE*)memPtr;
194 return (U16)((U16)p[0] + (p[1]<<8));
195 }
196}
197
198static void LZ4_writeLE16(void* memPtr, U16 value)
199{
200 if ((LZ4_UNALIGNED_ACCESS) && (LZ4_isLittleEndian()))
201 {
202 *(U16*)memPtr = value;
203 return;
204 }
205 else
206 {
207 BYTE* p = (BYTE*)memPtr;
208 p[0] = (BYTE) value;
209 p[1] = (BYTE)(value>>8);
210 }
211}
212
213
214static U16 LZ4_read16(const void* memPtr)
215{
216 if (LZ4_UNALIGNED_ACCESS)
217 return *(U16*)memPtr;
218 else
219 {
220 U16 val16;
221 memcpy(&val16, memPtr, 2);
222 return val16;
223 }
224}
225
226static U32 LZ4_read32(const void* memPtr)
227{
228 if (LZ4_UNALIGNED_ACCESS)
229 return *(U32*)memPtr;
230 else
231 {
232 U32 val32;
233 memcpy(&val32, memPtr, 4);
234 return val32;
235 }
236}
237
238static U64 LZ4_read64(const void* memPtr)
239{
240 if (LZ4_UNALIGNED_ACCESS)
241 return *(U64*)memPtr;
242 else
243 {
244 U64 val64;
245 memcpy(&val64, memPtr, 8);
246 return val64;
247 }
248}
249
250static size_t LZ4_read_ARCH(const void* p)
251{
252 if (LZ4_64bits())
253 return (size_t)LZ4_read64(p);
254 else
255 return (size_t)LZ4_read32(p);
256}
257
258
259static void LZ4_copy4(void* dstPtr, const void* srcPtr)
260{
261 if (LZ4_UNALIGNED_ACCESS)
262 {
263 *(U32*)dstPtr = *(U32*)srcPtr;
264 return;
265 }
266 memcpy(dstPtr, srcPtr, 4);
267}
268
269static void LZ4_copy8(void* dstPtr, const void* srcPtr)
270{
271#if GCC_VERSION!=409 /* disabled on GCC 4.9, as it generates invalid opcode (crash) */
272 if (LZ4_UNALIGNED_ACCESS)
273 {
274 if (LZ4_64bits())
275 *(U64*)dstPtr = *(U64*)srcPtr;
276 else
277 ((U32*)dstPtr)[0] = ((U32*)srcPtr)[0],
278 ((U32*)dstPtr)[1] = ((U32*)srcPtr)[1];
279 return;
280 }
209#endif 281#endif
282 memcpy(dstPtr, srcPtr, 8);
283}
210 284
211#define A16(x) (((U16_S *)(x))->v) 285/* customized version of memcpy, which may overwrite up to 7 bytes beyond dstEnd */
212#define A32(x) (((U32_S *)(x))->v) 286static void LZ4_wildCopy(void* dstPtr, const void* srcPtr, void* dstEnd)
213#define A64(x) (((U64_S *)(x))->v) 287{
214#define AARCH(x) (((size_t_S *)(x))->v) 288 BYTE* d = (BYTE*)dstPtr;
289 const BYTE* s = (const BYTE*)srcPtr;
290 BYTE* e = (BYTE*)dstEnd;
291 do { LZ4_copy8(d,s); d+=8; s+=8; } while (d<e);
292}
215 293
216 294
217/************************************** 295/**************************************
218 Constants 296 Common Constants
219**************************************/ 297**************************************/
220#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
221#define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
222#define HASH_SIZE_U32 (1 << LZ4_HASHLOG)
223
224#define MINMATCH 4 298#define MINMATCH 4
225 299
226#define COPYLENGTH 8 300#define COPYLENGTH 8
@@ -228,13 +302,10 @@ typedef struct {size_t v;} _PACKED size_t_S;
228#define MFLIMIT (COPYLENGTH+MINMATCH) 302#define MFLIMIT (COPYLENGTH+MINMATCH)
229static const int LZ4_minLength = (MFLIMIT+1); 303static const int LZ4_minLength = (MFLIMIT+1);
230 304
231#define KB *(1U<<10) 305#define KB *(1 <<10)
232#define MB *(1U<<20) 306#define MB *(1 <<20)
233#define GB *(1U<<30) 307#define GB *(1U<<30)
234 308
235#define LZ4_64KLIMIT ((64 KB) + (MFLIMIT-1))
236#define SKIPSTRENGTH 6 /* Increasing this value will make the compression run slower on incompressible data */
237
238#define MAXD_LOG 16 309#define MAXD_LOG 16
239#define MAX_DISTANCE ((1 << MAXD_LOG) - 1) 310#define MAX_DISTANCE ((1 << MAXD_LOG) - 1)
240 311
@@ -245,129 +316,147 @@ static const int LZ4_minLength = (MFLIMIT+1);
245 316
246 317
247/************************************** 318/**************************************
248 Structures and local types 319* Common Utils
249**************************************/ 320**************************************/
250typedef struct { 321#define LZ4_STATIC_ASSERT(c) { enum { LZ4_static_assert = 1/(int)(!!(c)) }; } /* use only *after* variable declarations */
251 U32 hashTable[HASH_SIZE_U32];
252 U32 currentOffset;
253 U32 initCheck;
254 const BYTE* dictionary;
255 const BYTE* bufferStart;
256 U32 dictSize;
257} LZ4_stream_t_internal;
258 322
259typedef enum { notLimited = 0, limitedOutput = 1 } limitedOutput_directive;
260typedef enum { byPtr, byU32, byU16 } tableType_t;
261 323
262typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive; 324/**************************************
263typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive; 325* Common functions
326**************************************/
327static unsigned LZ4_NbCommonBytes (register size_t val)
328{
329 if (LZ4_isLittleEndian())
330 {
331 if (LZ4_64bits())
332 {
333# if defined(_MSC_VER) && defined(_WIN64) && !defined(LZ4_FORCE_SW_BITCOUNT)
334 unsigned long r = 0;
335 _BitScanForward64( &r, (U64)val );
336 return (int)(r>>3);
337# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
338 return (__builtin_ctzll((U64)val) >> 3);
339# else
340 static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 };
341 return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58];
342# endif
343 }
344 else /* 32 bits */
345 {
346# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
347 unsigned long r;
348 _BitScanForward( &r, (U32)val );
349 return (int)(r>>3);
350# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
351 return (__builtin_ctz((U32)val) >> 3);
352# else
353 static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
354 return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
355# endif
356 }
357 }
358 else /* Big Endian CPU */
359 {
360 if (LZ4_64bits())
361 {
362# if defined(_MSC_VER) && defined(_WIN64) && !defined(LZ4_FORCE_SW_BITCOUNT)
363 unsigned long r = 0;
364 _BitScanReverse64( &r, val );
365 return (unsigned)(r>>3);
366# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
367 return (__builtin_clzll(val) >> 3);
368# else
369 unsigned r;
370 if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
371 if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
372 r += (!val);
373 return r;
374# endif
375 }
376 else /* 32 bits */
377 {
378# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
379 unsigned long r = 0;
380 _BitScanReverse( &r, (unsigned long)val );
381 return (unsigned)(r>>3);
382# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
383 return (__builtin_clz(val) >> 3);
384# else
385 unsigned r;
386 if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
387 r += (!val);
388 return r;
389# endif
390 }
391 }
392}
264 393
265typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive; 394static unsigned LZ4_count(const BYTE* pIn, const BYTE* pMatch, const BYTE* pInLimit)
266typedef enum { full = 0, partial = 1 } earlyEnd_directive; 395{
396 const BYTE* const pStart = pIn;
397
398 while (likely(pIn<pInLimit-(STEPSIZE-1)))
399 {
400 size_t diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn);
401 if (!diff) { pIn+=STEPSIZE; pMatch+=STEPSIZE; continue; }
402 pIn += LZ4_NbCommonBytes(diff);
403 return (unsigned)(pIn - pStart);
404 }
405
406 if (LZ4_64bits()) if ((pIn<(pInLimit-3)) && (LZ4_read32(pMatch) == LZ4_read32(pIn))) { pIn+=4; pMatch+=4; }
407 if ((pIn<(pInLimit-1)) && (LZ4_read16(pMatch) == LZ4_read16(pIn))) { pIn+=2; pMatch+=2; }
408 if ((pIn<pInLimit) && (*pMatch == *pIn)) pIn++;
409 return (unsigned)(pIn - pStart);
410}
267 411
268 412
413#ifndef LZ4_COMMONDEFS_ONLY
269/************************************** 414/**************************************
270 Architecture-specific macros 415* Local Constants
271**************************************/ 416**************************************/
272#define STEPSIZE sizeof(size_t) 417#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
273#define LZ4_COPYSTEP(d,s) { AARCH(d) = AARCH(s); d+=STEPSIZE; s+=STEPSIZE; } 418#define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
274#define LZ4_COPY8(d,s) { LZ4_COPYSTEP(d,s); if (STEPSIZE<8) LZ4_COPYSTEP(d,s); } 419#define HASH_SIZE_U32 (1 << LZ4_HASHLOG) /* required as macro for static allocation */
275 420
276#if (defined(LZ4_BIG_ENDIAN) && !defined(BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE)) 421static const int LZ4_64Klimit = ((64 KB) + (MFLIMIT-1));
277# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { U16 v = A16(p); v = lz4_bswap16(v); d = (s) - v; } 422static const U32 LZ4_skipTrigger = 6; /* Increase this value ==> compression run slower on incompressible data */
278# define LZ4_WRITE_LITTLEENDIAN_16(p,i) { U16 v = (U16)(i); v = lz4_bswap16(v); A16(p) = v; p+=2; }
279#else /* Little Endian */
280# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { d = (s) - A16(p); }
281# define LZ4_WRITE_LITTLEENDIAN_16(p,v) { A16(p) = v; p+=2; }
282#endif
283 423
284 424
285/************************************** 425/**************************************
286 Macros 426* Local Utils
287**************************************/ 427**************************************/
288#define LZ4_STATIC_ASSERT(c) { enum { LZ4_static_assert = 1/(!!(c)) }; } /* use only *after* variable declarations */ 428int LZ4_versionNumber (void) { return LZ4_VERSION_NUMBER; }
289#if LZ4_ARCH64 || !defined(__GNUC__) 429int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); }
290# define LZ4_WILDCOPY(d,s,e) { do { LZ4_COPY8(d,s) } while (d<e); } /* at the end, d>=e; */
291#else
292# define LZ4_WILDCOPY(d,s,e) { if (likely(e-d <= 8)) LZ4_COPY8(d,s) else do { LZ4_COPY8(d,s) } while (d<e); }
293#endif
294 430
295 431
296/**************************** 432/**************************************
297 Private local functions 433* Local Structures and types
298****************************/ 434**************************************/
299#if LZ4_ARCH64 435typedef struct {
436 U32 hashTable[HASH_SIZE_U32];
437 U32 currentOffset;
438 U32 initCheck;
439 const BYTE* dictionary;
440 const BYTE* bufferStart;
441 U32 dictSize;
442} LZ4_stream_t_internal;
300 443
301int LZ4_NbCommonBytes (register U64 val) 444typedef enum { notLimited = 0, limitedOutput = 1 } limitedOutput_directive;
302{ 445typedef enum { byPtr, byU32, byU16 } tableType_t;
303# if defined(LZ4_BIG_ENDIAN)
304# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
305 unsigned long r = 0;
306 _BitScanReverse64( &r, val );
307 return (int)(r>>3);
308# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
309 return (__builtin_clzll(val) >> 3);
310# else
311 int r;
312 if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
313 if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
314 r += (!val);
315 return r;
316# endif
317# else
318# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
319 unsigned long r = 0;
320 _BitScanForward64( &r, val );
321 return (int)(r>>3);
322# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
323 return (__builtin_ctzll(val) >> 3);
324# else
325 static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 };
326 return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58];
327# endif
328# endif
329}
330 446
331#else 447typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive;
448typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive;
332 449
333int LZ4_NbCommonBytes (register U32 val) 450typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive;
334{ 451typedef enum { full = 0, partial = 1 } earlyEnd_directive;
335# if defined(LZ4_BIG_ENDIAN)
336# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
337 unsigned long r = 0;
338 _BitScanReverse( &r, val );
339 return (int)(r>>3);
340# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
341 return (__builtin_clz(val) >> 3);
342# else
343 int r;
344 if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
345 r += (!val);
346 return r;
347# endif
348# else
349# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
350 unsigned long r;
351 _BitScanForward( &r, val );
352 return (int)(r>>3);
353# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
354 return (__builtin_ctz(val) >> 3);
355# else
356 static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
357 return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
358# endif
359# endif
360}
361 452
362#endif
363 453
364 454
365/******************************** 455/********************************
366 Compression functions 456* Compression functions
367********************************/ 457********************************/
368int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); }
369 458
370static int LZ4_hashSequence(U32 sequence, tableType_t tableType) 459static U32 LZ4_hashSequence(U32 sequence, tableType_t const tableType)
371{ 460{
372 if (tableType == byU16) 461 if (tableType == byU16)
373 return (((sequence) * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1))); 462 return (((sequence) * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1)));
@@ -375,15 +464,15 @@ static int LZ4_hashSequence(U32 sequence, tableType_t tableType)
375 return (((sequence) * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG)); 464 return (((sequence) * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG));
376} 465}
377 466
378static int LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return LZ4_hashSequence(A32(p), tableType); } 467static U32 LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return LZ4_hashSequence(LZ4_read32(p), tableType); }
379 468
380static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, tableType_t tableType, const BYTE* srcBase) 469static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, tableType_t const tableType, const BYTE* srcBase)
381{ 470{
382 switch (tableType) 471 switch (tableType)
383 { 472 {
384 case byPtr: { const BYTE** hashTable = (const BYTE**) tableBase; hashTable[h] = p; break; } 473 case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; hashTable[h] = p; return; }
385 case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); break; } 474 case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); return; }
386 case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); break; } 475 case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); return; }
387 } 476 }
388} 477}
389 478
@@ -406,34 +495,14 @@ static const BYTE* LZ4_getPosition(const BYTE* p, void* tableBase, tableType_t t
406 return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase); 495 return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
407} 496}
408 497
409static unsigned LZ4_count(const BYTE* pIn, const BYTE* pRef, const BYTE* pInLimit)
410{
411 const BYTE* const pStart = pIn;
412
413 while (likely(pIn<pInLimit-(STEPSIZE-1)))
414 {
415 size_t diff = AARCH(pRef) ^ AARCH(pIn);
416 if (!diff) { pIn+=STEPSIZE; pRef+=STEPSIZE; continue; }
417 pIn += LZ4_NbCommonBytes(diff);
418 return (unsigned)(pIn - pStart);
419 }
420 if (sizeof(void*)==8) if ((pIn<(pInLimit-3)) && (A32(pRef) == A32(pIn))) { pIn+=4; pRef+=4; }
421 if ((pIn<(pInLimit-1)) && (A16(pRef) == A16(pIn))) { pIn+=2; pRef+=2; }
422 if ((pIn<pInLimit) && (*pRef == *pIn)) pIn++;
423
424 return (unsigned)(pIn - pStart);
425}
426
427
428static int LZ4_compress_generic( 498static int LZ4_compress_generic(
429 void* ctx, 499 void* ctx,
430 const char* source, 500 const char* source,
431 char* dest, 501 char* dest,
432 int inputSize, 502 int inputSize,
433 int maxOutputSize, 503 int maxOutputSize,
434
435 limitedOutput_directive outputLimited, 504 limitedOutput_directive outputLimited,
436 tableType_t tableType, 505 tableType_t const tableType,
437 dict_directive dict, 506 dict_directive dict,
438 dictIssue_directive dictIssue) 507 dictIssue_directive dictIssue)
439{ 508{
@@ -454,7 +523,6 @@ static int LZ4_compress_generic(
454 BYTE* op = (BYTE*) dest; 523 BYTE* op = (BYTE*) dest;
455 BYTE* const olimit = op + maxOutputSize; 524 BYTE* const olimit = op + maxOutputSize;
456 525
457 const int skipStrength = SKIPSTRENGTH;
458 U32 forwardH; 526 U32 forwardH;
459 size_t refDelta=0; 527 size_t refDelta=0;
460 528
@@ -476,8 +544,8 @@ static int LZ4_compress_generic(
476 lowLimit = (const BYTE*)source; 544 lowLimit = (const BYTE*)source;
477 break; 545 break;
478 } 546 }
479 if ((tableType == byU16) && (inputSize>=(int)LZ4_64KLIMIT)) return 0; /* Size too large (not within 64K limit) */ 547 if ((tableType == byU16) && (inputSize>=LZ4_64Klimit)) return 0; /* Size too large (not within 64K limit) */
480 if (inputSize<LZ4_minLength) goto _last_literals; /* Input too small, no compression (all literals) */ 548 if (inputSize<LZ4_minLength) goto _last_literals; /* Input too small, no compression (all literals) */
481 549
482 /* First Byte */ 550 /* First Byte */
483 LZ4_putPosition(ip, ctx, tableType, base); 551 LZ4_putPosition(ip, ctx, tableType, base);
@@ -486,27 +554,26 @@ static int LZ4_compress_generic(
486 /* Main Loop */ 554 /* Main Loop */
487 for ( ; ; ) 555 for ( ; ; )
488 { 556 {
489 const BYTE* ref; 557 const BYTE* match;
490 BYTE* token; 558 BYTE* token;
491 { 559 {
492 const BYTE* forwardIp = ip; 560 const BYTE* forwardIp = ip;
493 unsigned step=1; 561 unsigned step=1;
494 unsigned searchMatchNb = (1U << skipStrength); 562 unsigned searchMatchNb = (1U << LZ4_skipTrigger);
495 563
496 /* Find a match */ 564 /* Find a match */
497 do { 565 do {
498 U32 h = forwardH; 566 U32 h = forwardH;
499 ip = forwardIp; 567 ip = forwardIp;
500 forwardIp += step; 568 forwardIp += step;
501 step = searchMatchNb++ >> skipStrength; 569 step = searchMatchNb++ >> LZ4_skipTrigger;
502 //if (step>8) step=8; // required for valid forwardIp ; slows down uncompressible data a bit
503 570
504 if (unlikely(forwardIp > mflimit)) goto _last_literals; 571 if (unlikely(forwardIp > mflimit)) goto _last_literals;
505 572
506 ref = LZ4_getPositionOnHash(h, ctx, tableType, base); 573 match = LZ4_getPositionOnHash(h, ctx, tableType, base);
507 if (dict==usingExtDict) 574 if (dict==usingExtDict)
508 { 575 {
509 if (ref<(const BYTE*)source) 576 if (match<(const BYTE*)source)
510 { 577 {
511 refDelta = dictDelta; 578 refDelta = dictDelta;
512 lowLimit = dictionary; 579 lowLimit = dictionary;
@@ -520,13 +587,13 @@ static int LZ4_compress_generic(
520 forwardH = LZ4_hashPosition(forwardIp, tableType); 587 forwardH = LZ4_hashPosition(forwardIp, tableType);
521 LZ4_putPositionOnHash(ip, h, ctx, tableType, base); 588 LZ4_putPositionOnHash(ip, h, ctx, tableType, base);
522 589
523 } while ( ((dictIssue==dictSmall) ? (ref < lowRefLimit) : 0) 590 } while ( ((dictIssue==dictSmall) ? (match < lowRefLimit) : 0)
524 || ((tableType==byU16) ? 0 : (ref + MAX_DISTANCE < ip)) 591 || ((tableType==byU16) ? 0 : (match + MAX_DISTANCE < ip))
525 || (A32(ref+refDelta) != A32(ip)) ); 592 || (LZ4_read32(match+refDelta) != LZ4_read32(ip)) );
526 } 593 }
527 594
528 /* Catch up */ 595 /* Catch up */
529 while ((ip>anchor) && (ref+refDelta > lowLimit) && (unlikely(ip[-1]==ref[refDelta-1]))) { ip--; ref--; } 596 while ((ip>anchor) && (match+refDelta > lowLimit) && (unlikely(ip[-1]==match[refDelta-1]))) { ip--; match--; }
530 597
531 { 598 {
532 /* Encode Literal length */ 599 /* Encode Literal length */
@@ -544,12 +611,13 @@ static int LZ4_compress_generic(
544 else *token = (BYTE)(litLength<<ML_BITS); 611 else *token = (BYTE)(litLength<<ML_BITS);
545 612
546 /* Copy Literals */ 613 /* Copy Literals */
547 { BYTE* end = op+litLength; LZ4_WILDCOPY(op,anchor,end); op=end; } 614 LZ4_wildCopy(op, anchor, op+litLength);
615 op+=litLength;
548 } 616 }
549 617
550_next_match: 618_next_match:
551 /* Encode Offset */ 619 /* Encode Offset */
552 LZ4_WRITE_LITTLEENDIAN_16(op, (U16)(ip-ref)); 620 LZ4_writeLE16(op, (U16)(ip-match)); op+=2;
553 621
554 /* Encode MatchLength */ 622 /* Encode MatchLength */
555 { 623 {
@@ -558,10 +626,10 @@ _next_match:
558 if ((dict==usingExtDict) && (lowLimit==dictionary)) 626 if ((dict==usingExtDict) && (lowLimit==dictionary))
559 { 627 {
560 const BYTE* limit; 628 const BYTE* limit;
561 ref += refDelta; 629 match += refDelta;
562 limit = ip + (dictEnd-ref); 630 limit = ip + (dictEnd-match);
563 if (limit > matchlimit) limit = matchlimit; 631 if (limit > matchlimit) limit = matchlimit;
564 matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, limit); 632 matchLength = LZ4_count(ip+MINMATCH, match+MINMATCH, limit);
565 ip += MINMATCH + matchLength; 633 ip += MINMATCH + matchLength;
566 if (ip==limit) 634 if (ip==limit)
567 { 635 {
@@ -572,14 +640,14 @@ _next_match:
572 } 640 }
573 else 641 else
574 { 642 {
575 matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, matchlimit); 643 matchLength = LZ4_count(ip+MINMATCH, match+MINMATCH, matchlimit);
576 ip += MINMATCH + matchLength; 644 ip += MINMATCH + matchLength;
577 } 645 }
578 646
647 if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) + (matchLength>>8) > olimit)))
648 return 0; /* Check output limit */
579 if (matchLength>=ML_MASK) 649 if (matchLength>=ML_MASK)
580 { 650 {
581 if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) + (matchLength>>8) > olimit)))
582 return 0; /* Check output limit */
583 *token += ML_MASK; 651 *token += ML_MASK;
584 matchLength -= ML_MASK; 652 matchLength -= ML_MASK;
585 for (; matchLength >= 510 ; matchLength-=510) { *op++ = 255; *op++ = 255; } 653 for (; matchLength >= 510 ; matchLength-=510) { *op++ = 255; *op++ = 255; }
@@ -598,10 +666,10 @@ _next_match:
598 LZ4_putPosition(ip-2, ctx, tableType, base); 666 LZ4_putPosition(ip-2, ctx, tableType, base);
599 667
600 /* Test next position */ 668 /* Test next position */
601 ref = LZ4_getPosition(ip, ctx, tableType, base); 669 match = LZ4_getPosition(ip, ctx, tableType, base);
602 if (dict==usingExtDict) 670 if (dict==usingExtDict)
603 { 671 {
604 if (ref<(const BYTE*)source) 672 if (match<(const BYTE*)source)
605 { 673 {
606 refDelta = dictDelta; 674 refDelta = dictDelta;
607 lowLimit = dictionary; 675 lowLimit = dictionary;
@@ -613,9 +681,9 @@ _next_match:
613 } 681 }
614 } 682 }
615 LZ4_putPosition(ip, ctx, tableType, base); 683 LZ4_putPosition(ip, ctx, tableType, base);
616 if ( ((dictIssue==dictSmall) ? (ref>=lowRefLimit) : 1) 684 if ( ((dictIssue==dictSmall) ? (match>=lowRefLimit) : 1)
617 && (ref+MAX_DISTANCE>=ip) 685 && (match+MAX_DISTANCE>=ip)
618 && (A32(ref+refDelta)==A32(ip)) ) 686 && (LZ4_read32(match+refDelta)==LZ4_read32(ip)) )
619 { token=op++; *token=0; goto _next_match; } 687 { token=op++; *token=0; goto _next_match; }
620 688
621 /* Prepare next loop */ 689 /* Prepare next loop */
@@ -642,16 +710,16 @@ _last_literals:
642int LZ4_compress(const char* source, char* dest, int inputSize) 710int LZ4_compress(const char* source, char* dest, int inputSize)
643{ 711{
644#if (HEAPMODE) 712#if (HEAPMODE)
645 void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U32, 4); /* Aligned on 4-bytes boundaries */ 713 void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U64, 8); /* Aligned on 8-bytes boundaries */
646#else 714#else
647 U32 ctx[LZ4_STREAMSIZE_U32] = {0}; /* Ensure data is aligned on 4-bytes boundaries */ 715 U64 ctx[LZ4_STREAMSIZE_U64] = {0}; /* Ensure data is aligned on 8-bytes boundaries */
648#endif 716#endif
649 int result; 717 int result;
650 718
651 if (inputSize < (int)LZ4_64KLIMIT) 719 if (inputSize < LZ4_64Klimit)
652 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue); 720 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue);
653 else 721 else
654 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0, notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); 722 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0, notLimited, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
655 723
656#if (HEAPMODE) 724#if (HEAPMODE)
657 FREEMEM(ctx); 725 FREEMEM(ctx);
@@ -662,16 +730,16 @@ int LZ4_compress(const char* source, char* dest, int inputSize)
662int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize) 730int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize)
663{ 731{
664#if (HEAPMODE) 732#if (HEAPMODE)
665 void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U32, 4); /* Aligned on 4-bytes boundaries */ 733 void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U64, 8); /* Aligned on 8-bytes boundaries */
666#else 734#else
667 U32 ctx[LZ4_STREAMSIZE_U32] = {0}; /* Ensure data is aligned on 4-bytes boundaries */ 735 U64 ctx[LZ4_STREAMSIZE_U64] = {0}; /* Ensure data is aligned on 8-bytes boundaries */
668#endif 736#endif
669 int result; 737 int result;
670 738
671 if (inputSize < (int)LZ4_64KLIMIT) 739 if (inputSize < LZ4_64Klimit)
672 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue); 740 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue);
673 else 741 else
674 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); 742 result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, maxOutputSize, limitedOutput, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
675 743
676#if (HEAPMODE) 744#if (HEAPMODE)
677 FREEMEM(ctx); 745 FREEMEM(ctx);
@@ -681,38 +749,48 @@ int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, in
681 749
682 750
683/***************************************** 751/*****************************************
684 Experimental : Streaming functions 752* Experimental : Streaming functions
685*****************************************/ 753*****************************************/
686 754
687void* LZ4_createStream() 755/*
756 * LZ4_initStream
757 * Use this function once, to init a newly allocated LZ4_stream_t structure
758 * Return : 1 if OK, 0 if error
759 */
760void LZ4_resetStream (LZ4_stream_t* LZ4_stream)
761{
762 MEM_INIT(LZ4_stream, 0, sizeof(LZ4_stream_t));
763}
764
765LZ4_stream_t* LZ4_createStream(void)
688{ 766{
689 void* lz4s = ALLOCATOR(4, LZ4_STREAMSIZE_U32); 767 LZ4_stream_t* lz4s = (LZ4_stream_t*)ALLOCATOR(8, LZ4_STREAMSIZE_U64);
690 MEM_INIT(lz4s, 0, LZ4_STREAMSIZE); 768 LZ4_STATIC_ASSERT(LZ4_STREAMSIZE >= sizeof(LZ4_stream_t_internal)); /* A compilation error here means LZ4_STREAMSIZE is not large enough */
769 LZ4_resetStream(lz4s);
691 return lz4s; 770 return lz4s;
692} 771}
693 772
694int LZ4_free (void* LZ4_stream) 773int LZ4_freeStream (LZ4_stream_t* LZ4_stream)
695{ 774{
696 FREEMEM(LZ4_stream); 775 FREEMEM(LZ4_stream);
697 return (0); 776 return (0);
698} 777}
699 778
700 779
701int LZ4_loadDict (void* LZ4_dict, const char* dictionary, int dictSize) 780int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
702{ 781{
703 LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict; 782 LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict;
704 const BYTE* p = (const BYTE*)dictionary; 783 const BYTE* p = (const BYTE*)dictionary;
705 const BYTE* const dictEnd = p + dictSize; 784 const BYTE* const dictEnd = p + dictSize;
706 const BYTE* base; 785 const BYTE* base;
707 786
708 LZ4_STATIC_ASSERT(LZ4_STREAMSIZE >= sizeof(LZ4_stream_t_internal)); /* A compilation error here means LZ4_STREAMSIZE is not large enough */ 787 if (dict->initCheck) LZ4_resetStream(LZ4_dict); /* Uninitialized structure detected */
709 if (dict->initCheck) MEM_INIT(dict, 0, sizeof(LZ4_stream_t_internal)); /* Uninitialized structure detected */
710 788
711 if (dictSize < MINMATCH) 789 if (dictSize < MINMATCH)
712 { 790 {
713 dict->dictionary = NULL; 791 dict->dictionary = NULL;
714 dict->dictSize = 0; 792 dict->dictSize = 0;
715 return 1; 793 return 0;
716 } 794 }
717 795
718 if (p <= dictEnd - 64 KB) p = dictEnd - 64 KB; 796 if (p <= dictEnd - 64 KB) p = dictEnd - 64 KB;
@@ -727,11 +805,11 @@ int LZ4_loadDict (void* LZ4_dict, const char* dictionary, int dictSize)
727 p+=3; 805 p+=3;
728 } 806 }
729 807
730 return 1; 808 return dict->dictSize;
731} 809}
732 810
733 811
734void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src) 812static void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src)
735{ 813{
736 if ((LZ4_dict->currentOffset > 0x80000000) || 814 if ((LZ4_dict->currentOffset > 0x80000000) ||
737 ((size_t)LZ4_dict->currentOffset > (size_t)src)) /* address space overflow */ 815 ((size_t)LZ4_dict->currentOffset > (size_t)src)) /* address space overflow */
@@ -802,19 +880,18 @@ FORCE_INLINE int LZ4_compress_continue_generic (void* LZ4_stream, const char* so
802 } 880 }
803} 881}
804 882
805 883int LZ4_compress_continue (LZ4_stream_t* LZ4_stream, const char* source, char* dest, int inputSize)
806int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize)
807{ 884{
808 return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, 0, notLimited); 885 return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, 0, notLimited);
809} 886}
810 887
811int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize) 888int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize)
812{ 889{
813 return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limitedOutput); 890 return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limitedOutput);
814} 891}
815 892
816 893
817// Hidden debug function, to force separate dictionary mode 894/* Hidden debug function, to force separate dictionary mode */
818int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int inputSize) 895int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int inputSize)
819{ 896{
820 LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_dict; 897 LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_dict;
@@ -835,7 +912,7 @@ int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char*
835} 912}
836 913
837 914
838int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int dictSize) 915int LZ4_saveDict (LZ4_stream_t* LZ4_dict, char* safeBuffer, int dictSize)
839{ 916{
840 LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict; 917 LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict;
841 const BYTE* previousDictEnd = dict->dictionary + dict->dictSize; 918 const BYTE* previousDictEnd = dict->dictionary + dict->dictSize;
@@ -843,28 +920,28 @@ int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int dictSize)
843 if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a dictionary > 64 KB */ 920 if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a dictionary > 64 KB */
844 if ((U32)dictSize > dict->dictSize) dictSize = dict->dictSize; 921 if ((U32)dictSize > dict->dictSize) dictSize = dict->dictSize;
845 922
846 memcpy(safeBuffer, previousDictEnd - dictSize, dictSize); 923 memmove(safeBuffer, previousDictEnd - dictSize, dictSize);
847 924
848 dict->dictionary = (const BYTE*)safeBuffer; 925 dict->dictionary = (const BYTE*)safeBuffer;
849 dict->dictSize = (U32)dictSize; 926 dict->dictSize = (U32)dictSize;
850 927
851 return 1; 928 return dictSize;
852} 929}
853 930
854 931
855 932
856/**************************** 933/*******************************
857 Decompression functions 934* Decompression functions
858****************************/ 935*******************************/
859/* 936/*
860 * This generic decompression function cover all use cases. 937 * This generic decompression function cover all use cases.
861 * It shall be instanciated several times, using different sets of directives 938 * It shall be instantiated several times, using different sets of directives
862 * Note that it is essential this generic function is really inlined, 939 * Note that it is essential this generic function is really inlined,
863 * in order to remove useless branches during compilation optimisation. 940 * in order to remove useless branches during compilation optimization.
864 */ 941 */
865FORCE_INLINE int LZ4_decompress_generic( 942FORCE_INLINE int LZ4_decompress_generic(
866 const char* source, 943 const char* const source,
867 char* dest, 944 char* const dest,
868 int inputSize, 945 int inputSize,
869 int outputSize, /* If endOnInput==endOnInputSize, this value is the max size of Output Buffer. */ 946 int outputSize, /* If endOnInput==endOnInputSize, this value is the max size of Output Buffer. */
870 947
@@ -872,36 +949,32 @@ FORCE_INLINE int LZ4_decompress_generic(
872 int partialDecoding, /* full, partial */ 949 int partialDecoding, /* full, partial */
873 int targetOutputSize, /* only used if partialDecoding==partial */ 950 int targetOutputSize, /* only used if partialDecoding==partial */
874 int dict, /* noDict, withPrefix64k, usingExtDict */ 951 int dict, /* noDict, withPrefix64k, usingExtDict */
875 const char* dictStart, /* only if dict==usingExtDict */ 952 const BYTE* const lowPrefix, /* == dest if dict == noDict */
876 int dictSize /* note : = 0 if noDict */ 953 const BYTE* const dictStart, /* only if dict==usingExtDict */
954 const size_t dictSize /* note : = 0 if noDict */
877 ) 955 )
878{ 956{
879 /* Local Variables */ 957 /* Local Variables */
880 const BYTE* restrict ip = (const BYTE*) source; 958 const BYTE* restrict ip = (const BYTE*) source;
881 const BYTE* ref;
882 const BYTE* const iend = ip + inputSize; 959 const BYTE* const iend = ip + inputSize;
883 960
884 BYTE* op = (BYTE*) dest; 961 BYTE* op = (BYTE*) dest;
885 BYTE* const oend = op + outputSize; 962 BYTE* const oend = op + outputSize;
886 BYTE* cpy; 963 BYTE* cpy;
887 BYTE* oexit = op + targetOutputSize; 964 BYTE* oexit = op + targetOutputSize;
888 const BYTE* const lowLimit = (const BYTE*)dest - dictSize; 965 const BYTE* const lowLimit = lowPrefix - dictSize;
889 966
890 const BYTE* const dictEnd = (const BYTE*)dictStart + dictSize; 967 const BYTE* const dictEnd = (const BYTE*)dictStart + dictSize;
891//#define OLD 968 const size_t dec32table[] = {4, 1, 2, 1, 4, 4, 4, 4};
892#ifdef OLD 969 const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3};
893 const size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */
894#else
895 const size_t dec32table[] = {4-0, 4-3, 4-2, 4-3, 4-0, 4-0, 4-0, 4-0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */
896#endif
897 static const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3};
898 970
899 const int checkOffset = (endOnInput) && (dictSize < (int)(64 KB)); 971 const int safeDecode = (endOnInput==endOnInputSize);
972 const int checkOffset = ((safeDecode) && (dictSize < (int)(64 KB)));
900 973
901 974
902 /* Special cases */ 975 /* Special cases */
903 if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT; /* targetOutputSize too high => decode everything */ 976 if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT; /* targetOutputSize too high => decode everything */
904 if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) && (*ip==0)) ? 0 : -1; /* Empty output buffer */ 977 if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) && (*ip==0)) ? 0 : -1; /* Empty output buffer */
905 if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0?1:-1); 978 if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0?1:-1);
906 979
907 980
@@ -910,8 +983,9 @@ FORCE_INLINE int LZ4_decompress_generic(
910 { 983 {
911 unsigned token; 984 unsigned token;
912 size_t length; 985 size_t length;
986 const BYTE* match;
913 987
914 /* get runlength */ 988 /* get literal length */
915 token = *ip++; 989 token = *ip++;
916 if ((length=(token>>ML_BITS)) == RUN_MASK) 990 if ((length=(token>>ML_BITS)) == RUN_MASK)
917 { 991 {
@@ -922,9 +996,8 @@ FORCE_INLINE int LZ4_decompress_generic(
922 length += s; 996 length += s;
923 } 997 }
924 while (likely((endOnInput)?ip<iend-RUN_MASK:1) && (s==255)); 998 while (likely((endOnInput)?ip<iend-RUN_MASK:1) && (s==255));
925 //if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */ 999 if ((safeDecode) && unlikely((size_t)(op+length)<(size_t)(op))) goto _output_error; /* overflow detection */
926 if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)(op))) goto _output_error; /* quickfix issue 134 */ 1000 if ((safeDecode) && unlikely((size_t)(ip+length)<(size_t)(ip))) goto _output_error; /* overflow detection */
927 if ((endOnInput) && (sizeof(void*)==4) && unlikely((size_t)(ip+length)<(size_t)(ip))) goto _output_error; /* quickfix issue 134 */
928 } 1001 }
929 1002
930 /* copy literals */ 1003 /* copy literals */
@@ -945,16 +1018,18 @@ FORCE_INLINE int LZ4_decompress_generic(
945 memcpy(op, ip, length); 1018 memcpy(op, ip, length);
946 ip += length; 1019 ip += length;
947 op += length; 1020 op += length;
948 break; /* Necessarily EOF, due to parsing restrictions */ 1021 break; /* Necessarily EOF, due to parsing restrictions */
949 } 1022 }
950 LZ4_WILDCOPY(op, ip, cpy); ip -= (op-cpy); op = cpy; 1023 LZ4_wildCopy(op, ip, cpy);
1024 ip += length; op = cpy;
951 1025
952 /* get offset */ 1026 /* get offset */
953 LZ4_READ_LITTLEENDIAN_16(ref,cpy,ip); ip+=2; 1027 match = cpy - LZ4_readLE16(ip); ip+=2;
954 if ((checkOffset) && (unlikely(ref < lowLimit))) goto _output_error; /* Error : offset outside destination buffer */ 1028 if ((checkOffset) && (unlikely(match < lowLimit))) goto _output_error; /* Error : offset outside destination buffer */
955 1029
956 /* get matchlength */ 1030 /* get matchlength */
957 if ((length=(token&ML_MASK)) == ML_MASK) 1031 length = token & ML_MASK;
1032 if (length == ML_MASK)
958 { 1033 {
959 unsigned s; 1034 unsigned s;
960 do 1035 do
@@ -963,36 +1038,38 @@ FORCE_INLINE int LZ4_decompress_generic(
963 s = *ip++; 1038 s = *ip++;
964 length += s; 1039 length += s;
965 } while (s==255); 1040 } while (s==255);
966 //if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */ 1041 if ((safeDecode) && unlikely((size_t)(op+length)<(size_t)op)) goto _output_error; /* overflow detection */
967 if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)op)) goto _output_error; /* quickfix issue 134 */
968 } 1042 }
1043 length += MINMATCH;
969 1044
970 /* check external dictionary */ 1045 /* check external dictionary */
971 if ((dict==usingExtDict) && (ref < (BYTE* const)dest)) 1046 if ((dict==usingExtDict) && (match < lowPrefix))
972 { 1047 {
973 if (unlikely(op+length+MINMATCH > oend-LASTLITERALS)) goto _output_error; 1048 if (unlikely(op+length > oend-LASTLITERALS)) goto _output_error; /* doesn't respect parsing restriction */
974 1049
975 if (length+MINMATCH <= (size_t)(dest-(char*)ref)) 1050 if (length <= (size_t)(lowPrefix-match))
976 { 1051 {
977 ref = dictEnd - (dest-(char*)ref); 1052 /* match can be copied as a single segment from external dictionary */
978 memcpy(op, ref, length+MINMATCH); 1053 match = dictEnd - (lowPrefix-match);
979 op += length+MINMATCH; 1054 memcpy(op, match, length);
1055 op += length;
980 } 1056 }
981 else 1057 else
982 { 1058 {
983 size_t copySize = (size_t)(dest-(char*)ref); 1059 /* match encompass external dictionary and current segment */
1060 size_t copySize = (size_t)(lowPrefix-match);
984 memcpy(op, dictEnd - copySize, copySize); 1061 memcpy(op, dictEnd - copySize, copySize);
985 op += copySize; 1062 op += copySize;
986 copySize = length+MINMATCH - copySize; 1063 copySize = length - copySize;
987 if (copySize > (size_t)((char*)op-dest)) /* overlap */ 1064 if (copySize > (size_t)(op-lowPrefix)) /* overlap within current segment */
988 { 1065 {
989 BYTE* const cpy = op + copySize; 1066 BYTE* const endOfMatch = op + copySize;
990 const BYTE* ref = (BYTE*)dest; 1067 const BYTE* copyFrom = lowPrefix;
991 while (op < cpy) *op++ = *ref++; 1068 while (op < endOfMatch) *op++ = *copyFrom++;
992 } 1069 }
993 else 1070 else
994 { 1071 {
995 memcpy(op, dest, copySize); 1072 memcpy(op, lowPrefix, copySize);
996 op += copySize; 1073 op += copySize;
997 } 1074 }
998 } 1075 }
@@ -1000,34 +1077,32 @@ FORCE_INLINE int LZ4_decompress_generic(
1000 } 1077 }
1001 1078
1002 /* copy repeated sequence */ 1079 /* copy repeated sequence */
1003 if (unlikely((op-ref)<(int)STEPSIZE)) 1080 cpy = op + length;
1081 if (unlikely((op-match)<8))
1004 { 1082 {
1005 const size_t dec64 = dec64table[(sizeof(void*)==4) ? 0 : op-ref]; 1083 const size_t dec64 = dec64table[op-match];
1006 op[0] = ref[0]; 1084 op[0] = match[0];
1007 op[1] = ref[1]; 1085 op[1] = match[1];
1008 op[2] = ref[2]; 1086 op[2] = match[2];
1009 op[3] = ref[3]; 1087 op[3] = match[3];
1010#ifdef OLD 1088 match += dec32table[op-match];
1011 op += 4, ref += 4; ref -= dec32table[op-ref]; 1089 LZ4_copy4(op+4, match);
1012 A32(op) = A32(ref); 1090 op += 8; match -= dec64;
1013 op += STEPSIZE-4; ref -= dec64; 1091 } else { LZ4_copy8(op, match); op+=8; match+=8; }
1014#else 1092
1015 ref += dec32table[op-ref]; 1093 if (unlikely(cpy>oend-12))
1016 A32(op+4) = A32(ref);
1017 op += STEPSIZE; ref -= dec64;
1018#endif
1019 } else { LZ4_COPYSTEP(op,ref); }
1020 cpy = op + length - (STEPSIZE-4);
1021
1022 if (unlikely(cpy>oend-COPYLENGTH-(STEPSIZE-4)))
1023 { 1094 {
1024 if (cpy > oend-LASTLITERALS) goto _output_error; /* Error : last 5 bytes must be literals */ 1095 if (cpy > oend-LASTLITERALS) goto _output_error; /* Error : last LASTLITERALS bytes must be literals */
1025 if (op<oend-COPYLENGTH) LZ4_WILDCOPY(op, ref, (oend-COPYLENGTH)); 1096 if (op < oend-8)
1026 while(op<cpy) *op++=*ref++; 1097 {
1027 op=cpy; 1098 LZ4_wildCopy(op, match, oend-8);
1028 continue; 1099 match += (oend-8) - op;
1100 op = oend-8;
1101 }
1102 while (op<cpy) *op++ = *match++;
1029 } 1103 }
1030 LZ4_WILDCOPY(op, ref, cpy); 1104 else
1105 LZ4_wildCopy(op, match, cpy);
1031 op=cpy; /* correction */ 1106 op=cpy; /* correction */
1032 } 1107 }
1033 1108
@@ -1043,30 +1118,30 @@ _output_error:
1043} 1118}
1044 1119
1045 1120
1046int LZ4_decompress_safe(const char* source, char* dest, int compressedSize, int maxOutputSize) 1121int LZ4_decompress_safe(const char* source, char* dest, int compressedSize, int maxDecompressedSize)
1047{ 1122{
1048 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, noDict, NULL, 0); 1123 return LZ4_decompress_generic(source, dest, compressedSize, maxDecompressedSize, endOnInputSize, full, 0, noDict, (BYTE*)dest, NULL, 0);
1049} 1124}
1050 1125
1051int LZ4_decompress_safe_partial(const char* source, char* dest, int compressedSize, int targetOutputSize, int maxOutputSize) 1126int LZ4_decompress_safe_partial(const char* source, char* dest, int compressedSize, int targetOutputSize, int maxDecompressedSize)
1052{ 1127{
1053 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, partial, targetOutputSize, noDict, NULL, 0); 1128 return LZ4_decompress_generic(source, dest, compressedSize, maxDecompressedSize, endOnInputSize, partial, targetOutputSize, noDict, (BYTE*)dest, NULL, 0);
1054} 1129}
1055 1130
1056int LZ4_decompress_fast(const char* source, char* dest, int originalSize) 1131int LZ4_decompress_fast(const char* source, char* dest, int originalSize)
1057{ 1132{
1058 return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, NULL, 0); 1133 return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, (BYTE*)(dest - 64 KB), NULL, 64 KB);
1059} 1134}
1060 1135
1136
1061/* streaming decompression functions */ 1137/* streaming decompression functions */
1062 1138
1063//#define LZ4_STREAMDECODESIZE_U32 4
1064//#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned int))
1065//typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; } LZ4_streamDecode_t;
1066typedef struct 1139typedef struct
1067{ 1140{
1068 const char* dictionary; 1141 BYTE* externalDict;
1069 int dictSize; 1142 size_t extDictSize;
1143 BYTE* prefixEnd;
1144 size_t prefixSize;
1070} LZ4_streamDecode_t_internal; 1145} LZ4_streamDecode_t_internal;
1071 1146
1072/* 1147/*
@@ -1074,25 +1149,32 @@ typedef struct
1074 * LZ4_createStreamDecode() 1149 * LZ4_createStreamDecode()
1075 * provides a pointer (void*) towards an initialized LZ4_streamDecode_t structure. 1150 * provides a pointer (void*) towards an initialized LZ4_streamDecode_t structure.
1076 */ 1151 */
1077void* LZ4_createStreamDecode() 1152LZ4_streamDecode_t* LZ4_createStreamDecode(void)
1078{ 1153{
1079 void* lz4s = ALLOCATOR(sizeof(U32), LZ4_STREAMDECODESIZE_U32); 1154 LZ4_streamDecode_t* lz4s = (LZ4_streamDecode_t*) ALLOCATOR(1, sizeof(LZ4_streamDecode_t));
1080 MEM_INIT(lz4s, 0, LZ4_STREAMDECODESIZE);
1081 return lz4s; 1155 return lz4s;
1082} 1156}
1083 1157
1158int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream)
1159{
1160 FREEMEM(LZ4_stream);
1161 return 0;
1162}
1163
1084/* 1164/*
1085 * LZ4_setDictDecode 1165 * LZ4_setStreamDecode
1086 * Use this function to instruct where to find the dictionary 1166 * Use this function to instruct where to find the dictionary
1087 * This function is not necessary if previous data is still available where it was decoded. 1167 * This function is not necessary if previous data is still available where it was decoded.
1088 * Loading a size of 0 is allowed (same effect as no dictionary). 1168 * Loading a size of 0 is allowed (same effect as no dictionary).
1089 * Return : 1 if OK, 0 if error 1169 * Return : 1 if OK, 0 if error
1090 */ 1170 */
1091int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int dictSize) 1171int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize)
1092{ 1172{
1093 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode; 1173 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode;
1094 lz4sd->dictionary = dictionary; 1174 lz4sd->prefixSize = (size_t) dictSize;
1095 lz4sd->dictSize = dictSize; 1175 lz4sd->prefixEnd = (BYTE*) dictionary + dictSize;
1176 lz4sd->externalDict = NULL;
1177 lz4sd->extDictSize = 0;
1096 return 1; 1178 return 1;
1097} 1179}
1098 1180
@@ -1101,43 +1183,61 @@ int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int dictS
1101 These decoding functions allow decompression of multiple blocks in "streaming" mode. 1183 These decoding functions allow decompression of multiple blocks in "streaming" mode.
1102 Previously decoded blocks must still be available at the memory position where they were decoded. 1184 Previously decoded blocks must still be available at the memory position where they were decoded.
1103 If it's not possible, save the relevant part of decoded data into a safe buffer, 1185 If it's not possible, save the relevant part of decoded data into a safe buffer,
1104 and indicate where it stands using LZ4_setDictDecode() 1186 and indicate where it stands using LZ4_setStreamDecode()
1105*/ 1187*/
1106int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize) 1188int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize)
1107{ 1189{
1108 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode; 1190 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode;
1109 int result; 1191 int result;
1110 1192
1111 result = LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize); 1193 if (lz4sd->prefixEnd == (BYTE*)dest)
1112 if (result <= 0) return result;
1113 if (lz4sd->dictionary + lz4sd->dictSize == dest)
1114 { 1194 {
1115 lz4sd->dictSize += result; 1195 result = LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
1196 endOnInputSize, full, 0,
1197 usingExtDict, lz4sd->prefixEnd - lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize);
1198 if (result <= 0) return result;
1199 lz4sd->prefixSize += result;
1200 lz4sd->prefixEnd += result;
1116 } 1201 }
1117 else 1202 else
1118 { 1203 {
1119 lz4sd->dictionary = dest; 1204 lz4sd->extDictSize = lz4sd->prefixSize;
1120 lz4sd->dictSize = result; 1205 lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
1206 result = LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
1207 endOnInputSize, full, 0,
1208 usingExtDict, (BYTE*)dest, lz4sd->externalDict, lz4sd->extDictSize);
1209 if (result <= 0) return result;
1210 lz4sd->prefixSize = result;
1211 lz4sd->prefixEnd = (BYTE*)dest + result;
1121 } 1212 }
1122 1213
1123 return result; 1214 return result;
1124} 1215}
1125 1216
1126int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize) 1217int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int originalSize)
1127{ 1218{
1128 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode; 1219 LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode;
1129 int result; 1220 int result;
1130 1221
1131 result = LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize); 1222 if (lz4sd->prefixEnd == (BYTE*)dest)
1132 if (result <= 0) return result;
1133 if (lz4sd->dictionary + lz4sd->dictSize == dest)
1134 { 1223 {
1135 lz4sd->dictSize += result; 1224 result = LZ4_decompress_generic(source, dest, 0, originalSize,
1225 endOnOutputSize, full, 0,
1226 usingExtDict, lz4sd->prefixEnd - lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize);
1227 if (result <= 0) return result;
1228 lz4sd->prefixSize += originalSize;
1229 lz4sd->prefixEnd += originalSize;
1136 } 1230 }
1137 else 1231 else
1138 { 1232 {
1139 lz4sd->dictionary = dest; 1233 lz4sd->extDictSize = lz4sd->prefixSize;
1140 lz4sd->dictSize = result; 1234 lz4sd->externalDict = (BYTE*)dest - lz4sd->extDictSize;
1235 result = LZ4_decompress_generic(source, dest, 0, originalSize,
1236 endOnOutputSize, full, 0,
1237 usingExtDict, (BYTE*)dest, lz4sd->externalDict, lz4sd->extDictSize);
1238 if (result <= 0) return result;
1239 lz4sd->prefixSize = originalSize;
1240 lz4sd->prefixEnd = (BYTE*)dest + originalSize;
1141 } 1241 }
1142 1242
1143 return result; 1243 return result;
@@ -1151,19 +1251,38 @@ Advanced decoding functions :
1151 the dictionary must be explicitly provided within parameters 1251 the dictionary must be explicitly provided within parameters
1152*/ 1252*/
1153 1253
1254FORCE_INLINE int LZ4_decompress_usingDict_generic(const char* source, char* dest, int compressedSize, int maxOutputSize, int safe, const char* dictStart, int dictSize)
1255{
1256 if (dictSize==0)
1257 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, safe, full, 0, noDict, (BYTE*)dest, NULL, 0);
1258 if (dictStart+dictSize == dest)
1259 {
1260 if (dictSize >= (int)(64 KB - 1))
1261 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, safe, full, 0, withPrefix64k, (BYTE*)dest-64 KB, NULL, 0);
1262 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, safe, full, 0, noDict, (BYTE*)dest-dictSize, NULL, 0);
1263 }
1264 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, safe, full, 0, usingExtDict, (BYTE*)dest, (BYTE*)dictStart, dictSize);
1265}
1266
1154int LZ4_decompress_safe_usingDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize) 1267int LZ4_decompress_safe_usingDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize)
1155{ 1268{
1156 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, dictStart, dictSize); 1269 return LZ4_decompress_usingDict_generic(source, dest, compressedSize, maxOutputSize, 1, dictStart, dictSize);
1157} 1270}
1158 1271
1159int LZ4_decompress_fast_usingDict(const char* source, char* dest, int originalSize, const char* dictStart, int dictSize) 1272int LZ4_decompress_fast_usingDict(const char* source, char* dest, int originalSize, const char* dictStart, int dictSize)
1160{ 1273{
1161 return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, dictStart, dictSize); 1274 return LZ4_decompress_usingDict_generic(source, dest, 0, originalSize, 0, dictStart, dictSize);
1275}
1276
1277/* debug function */
1278int LZ4_decompress_safe_forceExtDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize)
1279{
1280 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, (BYTE*)dest, (BYTE*)dictStart, dictSize);
1162} 1281}
1163 1282
1164 1283
1165/*************************************************** 1284/***************************************************
1166 Obsolete Functions 1285* Obsolete Functions
1167***************************************************/ 1286***************************************************/
1168/* 1287/*
1169These function names are deprecated and should no longer be used. 1288These function names are deprecated and should no longer be used.
@@ -1179,7 +1298,7 @@ int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize,
1179 1298
1180int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; } 1299int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; }
1181 1300
1182void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base) 1301static void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base)
1183{ 1302{
1184 MEM_INIT(lz4ds, 0, LZ4_STREAMSIZE); 1303 MEM_INIT(lz4ds, 0, LZ4_STREAMSIZE);
1185 lz4ds->bufferStart = base; 1304 lz4ds->bufferStart = base;
@@ -1194,18 +1313,16 @@ int LZ4_resetStreamState(void* state, const char* inputBuffer)
1194 1313
1195void* LZ4_create (const char* inputBuffer) 1314void* LZ4_create (const char* inputBuffer)
1196{ 1315{
1197 void* lz4ds = ALLOCATOR(4, LZ4_STREAMSIZE_U32); 1316 void* lz4ds = ALLOCATOR(8, LZ4_STREAMSIZE_U64);
1198 LZ4_init ((LZ4_stream_t_internal*)lz4ds, (const BYTE*)inputBuffer); 1317 LZ4_init ((LZ4_stream_t_internal*)lz4ds, (const BYTE*)inputBuffer);
1199 return lz4ds; 1318 return lz4ds;
1200} 1319}
1201 1320
1202char* LZ4_slideInputBuffer (void* LZ4_Data) 1321char* LZ4_slideInputBuffer (void* LZ4_Data)
1203{ 1322{
1204 LZ4_stream_t_internal* lz4ds = (LZ4_stream_t_internal*)LZ4_Data; 1323 LZ4_stream_t_internal* ctx = (LZ4_stream_t_internal*)LZ4_Data;
1205 1324 int dictSize = LZ4_saveDict((LZ4_stream_t*)LZ4_Data, (char*)ctx->bufferStart, 64 KB);
1206 LZ4_saveDict((LZ4_stream_t*)LZ4_Data, (char*)lz4ds->bufferStart, 64 KB); 1325 return (char*)(ctx->bufferStart + dictSize);
1207
1208 return (char*)(lz4ds->bufferStart + 64 KB);
1209} 1326}
1210 1327
1211/* Obsolete compresson functions using User-allocated state */ 1328/* Obsolete compresson functions using User-allocated state */
@@ -1217,10 +1334,10 @@ int LZ4_compress_withState (void* state, const char* source, char* dest, int inp
1217 if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */ 1334 if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */
1218 MEM_INIT(state, 0, LZ4_STREAMSIZE); 1335 MEM_INIT(state, 0, LZ4_STREAMSIZE);
1219 1336
1220 if (inputSize < (int)LZ4_64KLIMIT) 1337 if (inputSize < LZ4_64Klimit)
1221 return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue); 1338 return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue);
1222 else 1339 else
1223 return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); 1340 return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
1224} 1341}
1225 1342
1226int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize) 1343int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize)
@@ -1228,20 +1345,23 @@ int LZ4_compress_limitedOutput_withState (void* state, const char* source, char*
1228 if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */ 1345 if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */
1229 MEM_INIT(state, 0, LZ4_STREAMSIZE); 1346 MEM_INIT(state, 0, LZ4_STREAMSIZE);
1230 1347
1231 if (inputSize < (int)LZ4_64KLIMIT) 1348 if (inputSize < LZ4_64Klimit)
1232 return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue); 1349 return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue);
1233 else 1350 else
1234 return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); 1351 return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
1235} 1352}
1236 1353
1237/* Obsolete streaming decompression functions */ 1354/* Obsolete streaming decompression functions */
1238 1355
1239int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int compressedSize, int maxOutputSize) 1356int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int compressedSize, int maxOutputSize)
1240{ 1357{
1241 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, withPrefix64k, NULL, 64 KB); 1358 return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB);
1242} 1359}
1243 1360
1244int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int originalSize) 1361int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int originalSize)
1245{ 1362{
1246 return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, NULL, 64 KB); 1363 return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB);
1247} 1364}
1365
1366#endif /* LZ4_COMMONDEFS_ONLY */
1367
diff --git a/src/static_libs/lz4/lz4.h b/src/static_libs/lz4/lz4.h
index 1064fa1..de43fc0 100644
--- a/src/static_libs/lz4/lz4.h
+++ b/src/static_libs/lz4/lz4.h
@@ -1,7 +1,8 @@
1/* 1/*
2 LZ4 - Fast LZ compression algorithm 2 LZ4 - Fast LZ compression algorithm
3 Header File 3 Header File
4 Copyright (C) 2011-2014, Yann Collet. 4 Copyright (C) 2011-2015, Yann Collet.
5
5 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) 6 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
6 7
7 Redistribution and use in source and binary forms, with or without 8 Redistribution and use in source and binary forms, with or without
@@ -28,7 +29,7 @@
28 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29 30
30 You can contact the author at : 31 You can contact the author at :
31 - LZ4 source repository : http://code.google.com/p/lz4/ 32 - LZ4 source repository : https://github.com/Cyan4973/lz4
32 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c 33 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
33*/ 34*/
34#pragma once 35#pragma once
@@ -37,17 +38,23 @@
37extern "C" { 38extern "C" {
38#endif 39#endif
39 40
41/*
42 * lz4.h provides block compression functions, for optimal performance.
43 * If you need to generate inter-operable compressed data (respecting LZ4 frame specification),
44 * please use lz4frame.h instead.
45*/
40 46
41/************************************** 47/**************************************
42 Version 48* Version
43**************************************/ 49**************************************/
44#define LZ4_VERSION_MAJOR 1 /* for major interface/format changes */ 50#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
45#define LZ4_VERSION_MINOR 2 /* for minor interface/format changes */ 51#define LZ4_VERSION_MINOR 6 /* for new (non-breaking) interface capabilities */
46#define LZ4_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */ 52#define LZ4_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */
47 53#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)
54int LZ4_versionNumber (void);
48 55
49/************************************** 56/**************************************
50 Tuning parameter 57* Tuning parameter
51**************************************/ 58**************************************/
52/* 59/*
53 * LZ4_MEMORY_USAGE : 60 * LZ4_MEMORY_USAGE :
@@ -60,52 +67,45 @@ extern "C" {
60 67
61 68
62/************************************** 69/**************************************
63 Simple Functions 70* Simple Functions
64**************************************/ 71**************************************/
65 72
66int LZ4_compress (const char* source, char* dest, int inputSize); 73int LZ4_compress (const char* source, char* dest, int sourceSize);
67int LZ4_decompress_safe (const char* source, char* dest, int compressedSize, int maxOutputSize); 74int LZ4_decompress_safe (const char* source, char* dest, int compressedSize, int maxDecompressedSize);
68 75
69/* 76/*
70LZ4_compress() : 77LZ4_compress() :
71 Compresses 'inputSize' bytes from 'source' into 'dest'. 78 Compresses 'sourceSize' bytes from 'source' into 'dest'.
72 Destination buffer must be already allocated, 79 Destination buffer must be already allocated,
73 and must be sized to handle worst cases situations (input data not compressible) 80 and must be sized to handle worst cases situations (input data not compressible)
74 Worst case size evaluation is provided by function LZ4_compressBound() 81 Worst case size evaluation is provided by function LZ4_compressBound()
75 inputSize : Max supported value is LZ4_MAX_INPUT_VALUE 82 inputSize : Max supported value is LZ4_MAX_INPUT_SIZE
76 return : the number of bytes written in buffer dest 83 return : the number of bytes written in buffer dest
77 or 0 if the compression fails 84 or 0 if the compression fails
78 85
79LZ4_decompress_safe() : 86LZ4_decompress_safe() :
80 compressedSize : is obviously the source size 87 compressedSize : is obviously the source size
81 maxOutputSize : is the size of the destination buffer, which must be already allocated. 88 maxDecompressedSize : is the size of the destination buffer, which must be already allocated.
82 return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize) 89 return : the number of bytes decompressed into the destination buffer (necessarily <= maxDecompressedSize)
83 If the destination buffer is not large enough, decoding will stop and output an error code (<0). 90 If the destination buffer is not large enough, decoding will stop and output an error code (<0).
84 If the source stream is detected malformed, the function will stop decoding and return a negative result. 91 If the source stream is detected malformed, the function will stop decoding and return a negative result.
85 This function is protected against buffer overflow exploits : 92 This function is protected against buffer overflow exploits,
86 it never writes outside of output buffer, and never reads outside of input buffer. 93 and never writes outside of output buffer, nor reads outside of input buffer.
87 Therefore, it is protected against malicious data packets. 94 It is also protected against malicious data packets.
88*/
89
90
91/*
92Note :
93 Should you prefer to explicitly allocate compression-table memory using your own allocation method,
94 use the streaming functions provided below, simply reset the memory area between each call to LZ4_compress_continue()
95*/ 95*/
96 96
97 97
98/************************************** 98/**************************************
99 Advanced Functions 99* Advanced Functions
100**************************************/ 100**************************************/
101#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */ 101#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
102#define LZ4_COMPRESSBOUND(isize) ((unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16) 102#define LZ4_COMPRESSBOUND(isize) ((unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16)
103 103
104/* 104/*
105LZ4_compressBound() : 105LZ4_compressBound() :
106 Provides the maximum size that LZ4 may output in a "worst case" scenario (input data not compressible) 106 Provides the maximum size that LZ4 compression may output in a "worst case" scenario (input data not compressible)
107 primarily useful for memory allocation of output buffer. 107 This function is primarily useful for memory allocation purposes (output buffer size).
108 macro is also provided when result needs to be evaluated at compilation (such as stack memory allocation). 108 Macro LZ4_COMPRESSBOUND() is also provided for compilation-time evaluation (stack memory allocation for example).
109 109
110 isize : is the input size. Max supported value is LZ4_MAX_INPUT_SIZE 110 isize : is the input size. Max supported value is LZ4_MAX_INPUT_SIZE
111 return : maximum output size in a "worst case" scenario 111 return : maximum output size in a "worst case" scenario
@@ -116,28 +116,40 @@ int LZ4_compressBound(int isize);
116 116
117/* 117/*
118LZ4_compress_limitedOutput() : 118LZ4_compress_limitedOutput() :
119 Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. 119 Compress 'sourceSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'.
120 If it cannot achieve it, compression will stop, and result of the function will be zero. 120 If it cannot achieve it, compression will stop, and result of the function will be zero.
121 This saves time and memory on detecting non-compressible (or barely compressible) data.
121 This function never writes outside of provided output buffer. 122 This function never writes outside of provided output buffer.
122 123
123 inputSize : Max supported value is LZ4_MAX_INPUT_VALUE 124 sourceSize : Max supported value is LZ4_MAX_INPUT_VALUE
124 maxOutputSize : is the size of the destination buffer (which must be already allocated) 125 maxOutputSize : is the size of the destination buffer (which must be already allocated)
125 return : the number of bytes written in buffer 'dest' 126 return : the number of bytes written in buffer 'dest'
126 or 0 if the compression fails 127 or 0 if compression fails
127*/ 128*/
128int LZ4_compress_limitedOutput (const char* source, char* dest, int inputSize, int maxOutputSize); 129int LZ4_compress_limitedOutput (const char* source, char* dest, int sourceSize, int maxOutputSize);
130
131
132/*
133LZ4_compress_withState() :
134 Same compression functions, but using an externally allocated memory space to store compression state.
135 Use LZ4_sizeofState() to know how much memory must be allocated,
136 and then, provide it as 'void* state' to compression functions.
137*/
138int LZ4_sizeofState(void);
139int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize);
140int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize);
129 141
130 142
131/* 143/*
132LZ4_decompress_fast() : 144LZ4_decompress_fast() :
133 originalSize : is the original and therefore uncompressed size 145 originalSize : is the original and therefore uncompressed size
134 return : the number of bytes read from the source buffer (in other words, the compressed size) 146 return : the number of bytes read from the source buffer (in other words, the compressed size)
135 If the source stream is malformed, the function will stop decoding and return a negative result. 147 If the source stream is detected malformed, the function will stop decoding and return a negative result.
136 Destination buffer must be already allocated. Its size must be a minimum of 'originalSize' bytes. 148 Destination buffer must be already allocated. Its size must be a minimum of 'originalSize' bytes.
137 note : This function is a bit faster than LZ4_decompress_safe() 149 note : This function fully respect memory boundaries for properly formed compressed data.
138 It provides fast decompression and fully respect memory boundaries for properly formed compressed data. 150 It is a bit faster than LZ4_decompress_safe().
139 It does not provide full protection against intentionnally modified data stream. 151 However, it does not provide any protection against intentionally modified data stream (malicious input).
140 Use this function in a trusted environment (data to decode comes from a trusted source). 152 Use this function in trusted environment only (data to decode comes from a trusted source).
141*/ 153*/
142int LZ4_decompress_fast (const char* source, char* dest, int originalSize); 154int LZ4_decompress_fast (const char* source, char* dest, int originalSize);
143 155
@@ -145,117 +157,120 @@ int LZ4_decompress_fast (const char* source, char* dest, int originalSize);
145/* 157/*
146LZ4_decompress_safe_partial() : 158LZ4_decompress_safe_partial() :
147 This function decompress a compressed block of size 'compressedSize' at position 'source' 159 This function decompress a compressed block of size 'compressedSize' at position 'source'
148 into output buffer 'dest' of size 'maxOutputSize'. 160 into destination buffer 'dest' of size 'maxDecompressedSize'.
149 The function tries to stop decompressing operation as soon as 'targetOutputSize' has been reached, 161 The function tries to stop decompressing operation as soon as 'targetOutputSize' has been reached,
150 reducing decompression time. 162 reducing decompression time.
151 return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize) 163 return : the number of bytes decoded in the destination buffer (necessarily <= maxDecompressedSize)
152 Note : this number can be < 'targetOutputSize' should the compressed block to decode be smaller. 164 Note : this number can be < 'targetOutputSize' should the compressed block to decode be smaller.
153 Always control how many bytes were decoded. 165 Always control how many bytes were decoded.
154 If the source stream is detected malformed, the function will stop decoding and return a negative result. 166 If the source stream is detected malformed, the function will stop decoding and return a negative result.
155 This function never writes outside of output buffer, and never reads outside of input buffer. It is therefore protected against malicious data packets 167 This function never writes outside of output buffer, and never reads outside of input buffer. It is therefore protected against malicious data packets
156*/ 168*/
157int LZ4_decompress_safe_partial (const char* source, char* dest, int compressedSize, int targetOutputSize, int maxOutputSize); 169int LZ4_decompress_safe_partial (const char* source, char* dest, int compressedSize, int targetOutputSize, int maxDecompressedSize);
158 170
159 171
160/*********************************************** 172/***********************************************
161 Experimental Streaming Compression Functions 173* Streaming Compression Functions
162***********************************************/ 174***********************************************/
163 175
164#define LZ4_STREAMSIZE_U32 ((1 << (LZ4_MEMORY_USAGE-2)) + 8) 176#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE-3)) + 4)
165#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U32 * sizeof(unsigned int)) 177#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(long long))
166/* 178/*
167 * LZ4_stream_t 179 * LZ4_stream_t
168 * information structure to track an LZ4 stream. 180 * information structure to track an LZ4 stream.
169 * important : set this structure content to zero before first use ! 181 * important : init this structure content before first use !
182 * note : only allocated directly the structure if you are statically linking LZ4
183 * If you are using liblz4 as a DLL, please use below construction methods instead.
170 */ 184 */
171typedef struct { unsigned int table[LZ4_STREAMSIZE_U32]; } LZ4_stream_t; 185typedef struct { long long table[LZ4_STREAMSIZE_U64]; } LZ4_stream_t;
172 186
173/* 187/*
174 * If you prefer dynamic allocation methods, 188 * LZ4_resetStream
175 * LZ4_createStream 189 * Use this function to init an allocated LZ4_stream_t structure
176 * provides a pointer (void*) towards an initialized LZ4_stream_t structure.
177 * LZ4_free just frees it.
178 */ 190 */
179void* LZ4_createStream(); 191void LZ4_resetStream (LZ4_stream_t* LZ4_streamPtr);
180int LZ4_free (void* LZ4_stream);
181 192
193/*
194 * LZ4_createStream will allocate and initialize an LZ4_stream_t structure
195 * LZ4_freeStream releases its memory.
196 * In the context of a DLL (liblz4), please use these methods rather than the static struct.
197 * They are more future proof, in case of a change of LZ4_stream_t size.
198 */
199LZ4_stream_t* LZ4_createStream(void);
200int LZ4_freeStream (LZ4_stream_t* LZ4_streamPtr);
182 201
183/* 202/*
184 * LZ4_loadDict 203 * LZ4_loadDict
185 * Use this function to load a static dictionary into LZ4_stream. 204 * Use this function to load a static dictionary into LZ4_stream.
186 * Any previous data will be forgotten, only 'dictionary' will remain in memory. 205 * Any previous data will be forgotten, only 'dictionary' will remain in memory.
187 * Loading a size of 0 is allowed (same effect as init). 206 * Loading a size of 0 is allowed.
188 * Return : 1 if OK, 0 if error 207 * Return : dictionary size, in bytes (necessarily <= 64 KB)
189 */ 208 */
190int LZ4_loadDict (void* LZ4_stream, const char* dictionary, int dictSize); 209int LZ4_loadDict (LZ4_stream_t* LZ4_streamPtr, const char* dictionary, int dictSize);
191 210
192/* 211/*
193 * LZ4_compress_continue 212 * LZ4_compress_continue
194 * Compress data block 'source', using blocks compressed before as dictionary to improve compression ratio 213 * Compress data block 'source', using blocks compressed before as dictionary to improve compression ratio
195 * Previous data blocks are assumed to still be present at their previous location. 214 * Previous data blocks are assumed to still be present at their previous location.
215 * dest buffer must be already allocated, and sized to at least LZ4_compressBound(inputSize)
196 */ 216 */
197int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize); 217int LZ4_compress_continue (LZ4_stream_t* LZ4_streamPtr, const char* source, char* dest, int inputSize);
198 218
199/* 219/*
200 * LZ4_compress_limitedOutput_continue 220 * LZ4_compress_limitedOutput_continue
201 * Same as before, but also specify a maximum target compressed size (maxOutputSize) 221 * Same as before, but also specify a maximum target compressed size (maxOutputSize)
202 * If objective cannot be met, compression exits, and returns a zero. 222 * If objective cannot be met, compression exits, and returns a zero.
203 */ 223 */
204int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize); 224int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_streamPtr, const char* source, char* dest, int inputSize, int maxOutputSize);
205 225
206/* 226/*
207 * LZ4_saveDict 227 * LZ4_saveDict
208 * If previously compressed data block is not guaranteed to remain at its previous memory location 228 * If previously compressed data block is not guaranteed to remain available at its memory location
209 * save it into a safe place (char* safeBuffer) 229 * save it into a safer place (char* safeBuffer)
210 * Note : you don't need to call LZ4_loadDict() afterwards, 230 * Note : you don't need to call LZ4_loadDict() afterwards,
211 * dictionary is immediately usable, you can therefore call again LZ4_compress_continue() 231 * dictionary is immediately usable, you can therefore call again LZ4_compress_continue()
212 * Return : 1 if OK, 0 if error 232 * Return : saved dictionary size in bytes (necessarily <= dictSize), or 0 if error
213 * Note : any dictSize > 64 KB will be interpreted as 64KB.
214 */ 233 */
215int LZ4_saveDict (void* LZ4_stream, char* safeBuffer, int dictSize); 234int LZ4_saveDict (LZ4_stream_t* LZ4_streamPtr, char* safeBuffer, int dictSize);
216 235
217 236
218/************************************************ 237/************************************************
219 Experimental Streaming Decompression Functions 238* Streaming Decompression Functions
220************************************************/ 239************************************************/
221 240
222#define LZ4_STREAMDECODESIZE_U32 4 241#define LZ4_STREAMDECODESIZE_U64 4
223#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned int)) 242#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * sizeof(unsigned long long))
243typedef struct { unsigned long long table[LZ4_STREAMDECODESIZE_U64]; } LZ4_streamDecode_t;
224/* 244/*
225 * LZ4_streamDecode_t 245 * LZ4_streamDecode_t
226 * information structure to track an LZ4 stream. 246 * information structure to track an LZ4 stream.
227 * important : set this structure content to zero before first use ! 247 * init this structure content using LZ4_setStreamDecode or memset() before first use !
248 *
249 * In the context of a DLL (liblz4) please prefer usage of construction methods below.
250 * They are more future proof, in case of a change of LZ4_streamDecode_t size in the future.
251 * LZ4_createStreamDecode will allocate and initialize an LZ4_streamDecode_t structure
252 * LZ4_freeStreamDecode releases its memory.
228 */ 253 */
229typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; } LZ4_streamDecode_t; 254LZ4_streamDecode_t* LZ4_createStreamDecode(void);
255int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream);
230 256
231/* 257/*
232 * If you prefer dynamic allocation methods, 258 * LZ4_setStreamDecode
233 * LZ4_createStreamDecode() 259 * Use this function to instruct where to find the dictionary.
234 * provides a pointer (void*) towards an initialized LZ4_streamDecode_t structure. 260 * Setting a size of 0 is allowed (same effect as reset).
235 * LZ4_free just frees it. 261 * Return : 1 if OK, 0 if error
236 */ 262 */
237void* LZ4_createStreamDecode(); 263int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize);
238int LZ4_free (void* LZ4_stream); /* yes, it's the same one as for compression */
239 264
240/* 265/*
241*_continue() : 266*_continue() :
242 These decoding functions allow decompression of multiple blocks in "streaming" mode. 267 These decoding functions allow decompression of multiple blocks in "streaming" mode.
243 Previously decoded blocks must still be available at the memory position where they were decoded. 268 Previously decoded blocks *must* remain available at the memory position where they were decoded (up to 64 KB)
244 If it's not possible, save the relevant part of decoded data into a safe buffer, 269 If this condition is not possible, save the relevant part of decoded data into a safe buffer,
245 and indicate where it stands using LZ4_setDictDecode() 270 and indicate where is its new address using LZ4_setStreamDecode()
246*/ 271*/
247int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize); 272int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxDecompressedSize);
248int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize); 273int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int originalSize);
249
250/*
251 * LZ4_setDictDecode
252 * Use this function to instruct where to find the dictionary.
253 * This function can be used to specify a static dictionary,
254 * or to instruct where to find some previously decoded data saved into a different memory space.
255 * Setting a size of 0 is allowed (same effect as no dictionary).
256 * Return : 1 if OK, 0 if error
257 */
258int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int dictSize);
259 274
260 275
261/* 276/*
@@ -263,17 +278,15 @@ Advanced decoding functions :
263*_usingDict() : 278*_usingDict() :
264 These decoding functions work the same as 279 These decoding functions work the same as
265 a combination of LZ4_setDictDecode() followed by LZ4_decompress_x_continue() 280 a combination of LZ4_setDictDecode() followed by LZ4_decompress_x_continue()
266 all together into a single function call. 281 They are stand-alone and don't use nor update an LZ4_streamDecode_t structure.
267 It doesn't use nor update an LZ4_streamDecode_t structure.
268*/ 282*/
269int LZ4_decompress_safe_usingDict (const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize); 283int LZ4_decompress_safe_usingDict (const char* source, char* dest, int compressedSize, int maxDecompressedSize, const char* dictStart, int dictSize);
270int LZ4_decompress_fast_usingDict (const char* source, char* dest, int originalSize, const char* dictStart, int dictSize); 284int LZ4_decompress_fast_usingDict (const char* source, char* dest, int originalSize, const char* dictStart, int dictSize);
271 285
272 286
273 287
274
275/************************************** 288/**************************************
276 Obsolete Functions 289* Obsolete Functions
277**************************************/ 290**************************************/
278/* 291/*
279Obsolete decompression functions 292Obsolete decompression functions
@@ -281,14 +294,11 @@ These function names are deprecated and should no longer be used.
281They are only provided here for compatibility with older user programs. 294They are only provided here for compatibility with older user programs.
282- LZ4_uncompress is the same as LZ4_decompress_fast 295- LZ4_uncompress is the same as LZ4_decompress_fast
283- LZ4_uncompress_unknownOutputSize is the same as LZ4_decompress_safe 296- LZ4_uncompress_unknownOutputSize is the same as LZ4_decompress_safe
284*/ 297These function prototypes are now disabled; uncomment them if you really need them.
285int LZ4_uncompress (const char* source, char* dest, int outputSize); 298It is highly recommended to stop using these functions and migrate to newer ones */
286int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize); 299/* int LZ4_uncompress (const char* source, char* dest, int outputSize); */
300/* int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize); */
287 301
288/* Obsolete functions for externally allocated state; use streaming interface instead */
289int LZ4_sizeofState(void);
290int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize);
291int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize);
292 302
293/* Obsolete streaming functions; use new streaming interface whenever possible */ 303/* Obsolete streaming functions; use new streaming interface whenever possible */
294void* LZ4_create (const char* inputBuffer); 304void* LZ4_create (const char* inputBuffer);
diff --git a/src/static_libs/lz4/lz4hc.c b/src/static_libs/lz4/lz4hc.c
index 6086749..c7a94a0 100644
--- a/src/static_libs/lz4/lz4hc.c
+++ b/src/static_libs/lz4/lz4hc.c
@@ -1,34 +1,35 @@
1/* 1/*
2 LZ4 HC - High Compression Mode of LZ4 2LZ4 HC - High Compression Mode of LZ4
3 Copyright (C) 2011-2014, Yann Collet. 3Copyright (C) 2011-2015, Yann Collet.
4 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) 4
5 5BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
6 Redistribution and use in source and binary forms, with or without 6
7 modification, are permitted provided that the following conditions are 7Redistribution and use in source and binary forms, with or without
8 met: 8modification, are permitted provided that the following conditions are
9 9met:
10 * Redistributions of source code must retain the above copyright 10
11 notice, this list of conditions and the following disclaimer. 11* Redistributions of source code must retain the above copyright
12 * Redistributions in binary form must reproduce the above 12notice, this list of conditions and the following disclaimer.
13 copyright notice, this list of conditions and the following disclaimer 13* Redistributions in binary form must reproduce the above
14 in the documentation and/or other materials provided with the 14copyright notice, this list of conditions and the following disclaimer
15 distribution. 15in the documentation and/or other materials provided with the
16 16distribution.
17 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 17
18 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 18THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 19"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 20LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 21A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 22OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 23SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 24LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 25DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28 28OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29 You can contact the author at : 29
30 - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html 30You can contact the author at :
31 - LZ4 source repository : http://code.google.com/p/lz4/ 31 - LZ4 source repository : https://github.com/Cyan4973/lz4
32 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
32*/ 33*/
33 34
34 35
@@ -36,213 +37,48 @@
36/************************************** 37/**************************************
37 Tuning Parameter 38 Tuning Parameter
38**************************************/ 39**************************************/
39#define LZ4HC_DEFAULT_COMPRESSIONLEVEL 8 40static const int LZ4HC_compressionLevel_default = 9;
40
41
42/**************************************
43 Memory routines
44**************************************/
45#include <stdlib.h> /* calloc, free */
46#define ALLOCATOR(s) calloc(1,s)
47#define FREEMEM free
48#include <string.h> /* memset, memcpy */
49#define MEM_INIT memset
50
51
52/**************************************
53 CPU Feature Detection
54**************************************/
55/* 32 or 64 bits ? */
56#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
57 || defined(__powerpc64__) || defined(__powerpc64le__) \
58 || defined(__ppc64__) || defined(__ppc64le__) \
59 || defined(__PPC64__) || defined(__PPC64LE__) \
60 || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */
61# define LZ4_ARCH64 1
62#else
63# define LZ4_ARCH64 0
64#endif
65
66/*
67 * Little Endian or Big Endian ?
68 * Overwrite the #define below if you know your architecture endianess
69 */
70#include <stdlib.h> /* Apparently required to detect endianess */
71#if defined (__GLIBC__)
72# include <endian.h>
73# if (__BYTE_ORDER == __BIG_ENDIAN)
74# define LZ4_BIG_ENDIAN 1
75# endif
76#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN))
77# define LZ4_BIG_ENDIAN 1
78#elif defined(__sparc) || defined(__sparc__) \
79 || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
80 || defined(__hpux) || defined(__hppa) \
81 || defined(_MIPSEB) || defined(__s390__)
82# define LZ4_BIG_ENDIAN 1
83#else
84/* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */
85#endif
86
87/*
88 * Unaligned memory access is automatically enabled for "common" CPU, such as x86.
89 * For others CPU, the compiler will be more cautious, and insert extra code to ensure aligned access is respected
90 * If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance
91 */
92#if defined(__ARM_FEATURE_UNALIGNED)
93# define LZ4_FORCE_UNALIGNED_ACCESS 1
94#endif
95
96/* Define this parameter if your target system or compiler does not support hardware bit count */
97#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */
98# define LZ4_FORCE_SW_BITCOUNT
99#endif
100
101
102/**************************************
103 Compiler Options
104**************************************/
105#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
106/* "restrict" is a known keyword */
107#else
108# define restrict /* Disable restrict */
109#endif
110
111#ifdef _MSC_VER /* Visual Studio */
112# define FORCE_INLINE static __forceinline
113# include <intrin.h> /* For Visual 2005 */
114# if LZ4_ARCH64 /* 64-bits */
115# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
116# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
117# else /* 32-bits */
118# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
119# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
120# endif
121# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
122# pragma warning(disable : 4701) /* disable: C4701: potentially uninitialized local variable used */
123#else
124# ifdef __GNUC__
125# define FORCE_INLINE static inline __attribute__((always_inline))
126# else
127# define FORCE_INLINE static inline
128# endif
129#endif
130
131#ifdef _MSC_VER /* Visual Studio */
132# define lz4_bswap16(x) _byteswap_ushort(x)
133#else
134# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8)))
135#endif
136 41
137 42
138/************************************** 43/**************************************
139 Includes 44 Includes
140**************************************/ 45**************************************/
141#include "lz4hc.h" 46#include "lz4hc.h"
142#include "lz4.h"
143 47
144 48
145/************************************** 49/**************************************
146 Basic Types 50 Local Compiler Options
147**************************************/ 51**************************************/
148#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ 52#if defined(__GNUC__)
149# include <stdint.h> 53# pragma GCC diagnostic ignored "-Wunused-function"
150 typedef uint8_t BYTE;
151 typedef uint16_t U16;
152 typedef uint32_t U32;
153 typedef int32_t S32;
154 typedef uint64_t U64;
155#else
156 typedef unsigned char BYTE;
157 typedef unsigned short U16;
158 typedef unsigned int U32;
159 typedef signed int S32;
160 typedef unsigned long long U64;
161#endif 54#endif
162 55
163#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS) 56#if defined (__clang__)
164# define _PACKED __attribute__ ((packed)) 57# pragma clang diagnostic ignored "-Wunused-function"
165#else
166# define _PACKED
167#endif 58#endif
168 59
169#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
170# ifdef __IBMC__
171# pragma pack(1)
172# else
173# pragma pack(push, 1)
174# endif
175#endif
176
177typedef struct _U16_S { U16 v; } _PACKED U16_S;
178typedef struct _U32_S { U32 v; } _PACKED U32_S;
179typedef struct _U64_S { U64 v; } _PACKED U64_S;
180
181#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
182# pragma pack(pop)
183#endif
184 60
185#define A64(x) (((U64_S *)(x))->v) 61/**************************************
186#define A32(x) (((U32_S *)(x))->v) 62 Common LZ4 definition
187#define A16(x) (((U16_S *)(x))->v) 63**************************************/
64#define LZ4_COMMONDEFS_ONLY
65#include "lz4.c"
188 66
189 67
190/************************************** 68/**************************************
191 Constants 69 Local Constants
192**************************************/ 70**************************************/
193#define MINMATCH 4
194
195#define DICTIONARY_LOGSIZE 16 71#define DICTIONARY_LOGSIZE 16
196#define MAXD (1<<DICTIONARY_LOGSIZE) 72#define MAXD (1<<DICTIONARY_LOGSIZE)
197#define MAXD_MASK ((U32)(MAXD - 1)) 73#define MAXD_MASK ((U32)(MAXD - 1))
198#define MAX_DISTANCE (MAXD - 1)
199 74
200#define HASH_LOG (DICTIONARY_LOGSIZE-1) 75#define HASH_LOG (DICTIONARY_LOGSIZE-1)
201#define HASHTABLESIZE (1 << HASH_LOG) 76#define HASHTABLESIZE (1 << HASH_LOG)
202#define HASH_MASK (HASHTABLESIZE - 1) 77#define HASH_MASK (HASHTABLESIZE - 1)
203 78
204#define ML_BITS 4
205#define ML_MASK (size_t)((1U<<ML_BITS)-1)
206#define RUN_BITS (8-ML_BITS)
207#define RUN_MASK ((1U<<RUN_BITS)-1)
208
209#define COPYLENGTH 8
210#define LASTLITERALS 5
211#define MFLIMIT (COPYLENGTH+MINMATCH)
212#define MINLENGTH (MFLIMIT+1)
213#define OPTIMAL_ML (int)((ML_MASK-1)+MINMATCH) 79#define OPTIMAL_ML (int)((ML_MASK-1)+MINMATCH)
214 80
215#define KB *(1U<<10) 81static const int g_maxCompressionLevel = 16;
216#define MB *(1U<<20)
217#define GB *(1U<<30)
218
219
220/**************************************
221 Architecture-specific macros
222**************************************/
223#if LZ4_ARCH64 /* 64-bit */
224# define STEPSIZE 8
225# define LZ4_COPYSTEP(s,d) A64(d) = A64(s); d+=8; s+=8;
226# define LZ4_COPYPACKET(s,d) LZ4_COPYSTEP(s,d)
227# define AARCH A64
228# define HTYPE U32
229# define INITBASE(b,s) const BYTE* const b = s
230#else /* 32-bit */
231# define STEPSIZE 4
232# define LZ4_COPYSTEP(s,d) A32(d) = A32(s); d+=4; s+=4;
233# define LZ4_COPYPACKET(s,d) LZ4_COPYSTEP(s,d); LZ4_COPYSTEP(s,d);
234# define AARCH A32
235# define HTYPE U32
236# define INITBASE(b,s) const BYTE* const b = s
237#endif
238
239#if defined(LZ4_BIG_ENDIAN)
240# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { U16 v = A16(p); v = lz4_bswap16(v); d = (s) - v; }
241# define LZ4_WRITE_LITTLEENDIAN_16(p,i) { U16 v = (U16)(i); v = lz4_bswap16(v); A16(p) = v; p+=2; }
242#else /* Little Endian */
243# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { d = (s) - A16(p); }
244# define LZ4_WRITE_LITTLEENDIAN_16(p,v) { A16(p) = v; p+=2; }
245#endif
246 82
247 83
248/************************************** 84/**************************************
@@ -250,312 +86,195 @@ typedef struct _U64_S { U64 v; } _PACKED U64_S;
250**************************************/ 86**************************************/
251typedef struct 87typedef struct
252{ 88{
253 const BYTE* inputBuffer; 89 U32 hashTable[HASHTABLESIZE];
254 const BYTE* base; 90 U16 chainTable[MAXD];
255 const BYTE* end; 91 const BYTE* end; /* next block here to continue on current prefix */
256 HTYPE hashTable[HASHTABLESIZE]; 92 const BYTE* base; /* All index relative to this position */
257 U16 chainTable[MAXD]; 93 const BYTE* dictBase; /* alternate base for extDict */
258 const BYTE* nextToUpdate; 94 const BYTE* inputBuffer;/* deprecated */
95 U32 dictLimit; /* below that point, need extDict */
96 U32 lowLimit; /* below that point, no more dict */
97 U32 nextToUpdate;
98 U32 compressionLevel;
259} LZ4HC_Data_Structure; 99} LZ4HC_Data_Structure;
260 100
261 101
262/************************************** 102/**************************************
263 Macros 103 Local Macros
264**************************************/ 104**************************************/
265#define LZ4_WILDCOPY(s,d,e) do { LZ4_COPYPACKET(s,d) } while (d<e);
266#define LZ4_BLINDCOPY(s,d,l) { BYTE* e=d+l; LZ4_WILDCOPY(s,d,e); d=e; }
267#define HASH_FUNCTION(i) (((i) * 2654435761U) >> ((MINMATCH*8)-HASH_LOG)) 105#define HASH_FUNCTION(i) (((i) * 2654435761U) >> ((MINMATCH*8)-HASH_LOG))
268#define HASH_VALUE(p) HASH_FUNCTION(A32(p))
269#define HASH_POINTER(p) (HashTable[HASH_VALUE(p)] + base)
270#define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK] 106#define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK]
271#define GETNEXT(p) ((p) - (size_t)DELTANEXT(p)) 107#define GETNEXT(p) ((p) - (size_t)DELTANEXT(p))
272 108
109static U32 LZ4HC_hashPtr(const void* ptr) { return HASH_FUNCTION(LZ4_read32(ptr)); }
273 110
274/**************************************
275 Private functions
276**************************************/
277#if LZ4_ARCH64
278
279FORCE_INLINE int LZ4_NbCommonBytes (register U64 val)
280{
281#if defined(LZ4_BIG_ENDIAN)
282# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
283 unsigned long r = 0;
284 _BitScanReverse64( &r, val );
285 return (int)(r>>3);
286# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
287 return (__builtin_clzll(val) >> 3);
288# else
289 int r;
290 if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
291 if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
292 r += (!val);
293 return r;
294# endif
295#else
296# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
297 unsigned long r = 0;
298 _BitScanForward64( &r, val );
299 return (int)(r>>3);
300# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
301 return (__builtin_ctzll(val) >> 3);
302# else
303 static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 };
304 return DeBruijnBytePos[((U64)((val & -val) * 0x0218A392CDABBD3F)) >> 58];
305# endif
306#endif
307}
308
309#else
310 111
311FORCE_INLINE int LZ4_NbCommonBytes (register U32 val)
312{
313#if defined(LZ4_BIG_ENDIAN)
314# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
315 unsigned long r;
316 _BitScanReverse( &r, val );
317 return (int)(r>>3);
318# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
319 return (__builtin_clz(val) >> 3);
320# else
321 int r;
322 if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
323 r += (!val);
324 return r;
325# endif
326#else
327# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
328 unsigned long r;
329 _BitScanForward( &r, val );
330 return (int)(r>>3);
331# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
332 return (__builtin_ctz(val) >> 3);
333# else
334 static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
335 return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
336# endif
337#endif
338}
339 112
340#endif 113/**************************************
341 114 HC Compression
342 115**************************************/
343int LZ4_sizeofStreamStateHC() 116static void LZ4HC_init (LZ4HC_Data_Structure* hc4, const BYTE* start)
344{
345 return sizeof(LZ4HC_Data_Structure);
346}
347
348FORCE_INLINE void LZ4_initHC (LZ4HC_Data_Structure* hc4, const BYTE* base)
349{ 117{
350 MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable)); 118 MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable));
351 MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable)); 119 MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable));
352 hc4->nextToUpdate = base + 1; 120 hc4->nextToUpdate = 64 KB;
353 hc4->base = base; 121 hc4->base = start - 64 KB;
354 hc4->inputBuffer = base; 122 hc4->inputBuffer = start;
355 hc4->end = base; 123 hc4->end = start;
356} 124 hc4->dictBase = start - 64 KB;
357 125 hc4->dictLimit = 64 KB;
358int LZ4_resetStreamStateHC(void* state, const char* inputBuffer) 126 hc4->lowLimit = 64 KB;
359{
360 if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error : pointer is not aligned for pointer (32 or 64 bits) */
361 LZ4_initHC((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer);
362 return 0;
363}
364
365
366void* LZ4_createHC (const char* inputBuffer)
367{
368 void* hc4 = ALLOCATOR(sizeof(LZ4HC_Data_Structure));
369 LZ4_initHC ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer);
370 return hc4;
371}
372
373
374int LZ4_freeHC (void* LZ4HC_Data)
375{
376 FREEMEM(LZ4HC_Data);
377 return (0);
378} 127}
379 128
380 129
381/* Update chains up to ip (excluded) */ 130/* Update chains up to ip (excluded) */
382FORCE_INLINE void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip) 131FORCE_INLINE void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip)
383{ 132{
384 U16* chainTable = hc4->chainTable; 133 U16* chainTable = hc4->chainTable;
385 HTYPE* HashTable = hc4->hashTable; 134 U32* HashTable = hc4->hashTable;
386 INITBASE(base,hc4->base); 135 const BYTE* const base = hc4->base;
136 const U32 target = (U32)(ip - base);
137 U32 idx = hc4->nextToUpdate;
387 138
388 while(hc4->nextToUpdate < ip) 139 while(idx < target)
389 { 140 {
390 const BYTE* const p = hc4->nextToUpdate; 141 U32 h = LZ4HC_hashPtr(base+idx);
391 size_t delta = (p) - HASH_POINTER(p); 142 size_t delta = idx - HashTable[h];
392 if (delta>MAX_DISTANCE) delta = MAX_DISTANCE; 143 if (delta>MAX_DISTANCE) delta = MAX_DISTANCE;
393 DELTANEXT(p) = (U16)delta; 144 chainTable[idx & 0xFFFF] = (U16)delta;
394 HashTable[HASH_VALUE(p)] = (HTYPE)((p) - base); 145 HashTable[h] = idx;
395 hc4->nextToUpdate++; 146 idx++;
396 } 147 }
397}
398
399 148
400char* LZ4_slideInputBufferHC(void* LZ4HC_Data) 149 hc4->nextToUpdate = target;
401{
402 LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data;
403 U32 distance = (U32)(hc4->end - hc4->inputBuffer) - 64 KB;
404 distance = (distance >> 16) << 16; /* Must be a multiple of 64 KB */
405 LZ4HC_Insert(hc4, hc4->end - MINMATCH);
406 memcpy((void*)(hc4->end - 64 KB - distance), (const void*)(hc4->end - 64 KB), 64 KB);
407 hc4->nextToUpdate -= distance;
408 hc4->base -= distance;
409 if ((U32)(hc4->inputBuffer - hc4->base) > 1 GB + 64 KB) /* Avoid overflow */
410 {
411 int i;
412 hc4->base += 1 GB;
413 for (i=0; i<HASHTABLESIZE; i++) hc4->hashTable[i] -= 1 GB;
414 }
415 hc4->end -= distance;
416 return (char*)(hc4->end);
417} 150}
418 151
419 152
420FORCE_INLINE size_t LZ4HC_CommonLength (const BYTE* p1, const BYTE* p2, const BYTE* const matchlimit) 153FORCE_INLINE int LZ4HC_InsertAndFindBestMatch (LZ4HC_Data_Structure* hc4, /* Index table will be updated */
421{ 154 const BYTE* ip, const BYTE* const iLimit,
422 const BYTE* p1t = p1; 155 const BYTE** matchpos,
423 156 const int maxNbAttempts)
424 while (p1t<matchlimit-(STEPSIZE-1))
425 {
426 size_t diff = AARCH(p2) ^ AARCH(p1t);
427 if (!diff) { p1t+=STEPSIZE; p2+=STEPSIZE; continue; }
428 p1t += LZ4_NbCommonBytes(diff);
429 return (p1t - p1);
430 }
431 if (LZ4_ARCH64) if ((p1t<(matchlimit-3)) && (A32(p2) == A32(p1t))) { p1t+=4; p2+=4; }
432 if ((p1t<(matchlimit-1)) && (A16(p2) == A16(p1t))) { p1t+=2; p2+=2; }
433 if ((p1t<matchlimit) && (*p2 == *p1t)) p1t++;
434 return (p1t - p1);
435}
436
437
438FORCE_INLINE int LZ4HC_InsertAndFindBestMatch (LZ4HC_Data_Structure* hc4, const BYTE* ip, const BYTE* const matchlimit, const BYTE** matchpos, const int maxNbAttempts)
439{ 157{
440 U16* const chainTable = hc4->chainTable; 158 U16* const chainTable = hc4->chainTable;
441 HTYPE* const HashTable = hc4->hashTable; 159 U32* const HashTable = hc4->hashTable;
442 const BYTE* ref; 160 const BYTE* const base = hc4->base;
443 INITBASE(base,hc4->base); 161 const BYTE* const dictBase = hc4->dictBase;
162 const U32 dictLimit = hc4->dictLimit;
163 const U32 lowLimit = (hc4->lowLimit + 64 KB > (U32)(ip-base)) ? hc4->lowLimit : (U32)(ip - base) - (64 KB - 1);
164 U32 matchIndex;
165 const BYTE* match;
444 int nbAttempts=maxNbAttempts; 166 int nbAttempts=maxNbAttempts;
445 size_t repl=0, ml=0; 167 size_t ml=0;
446 U16 delta=0; /* useless assignment, to remove an uninitialization warning */
447 168
448 /* HC4 match finder */ 169 /* HC4 match finder */
449 LZ4HC_Insert(hc4, ip); 170 LZ4HC_Insert(hc4, ip);
450 ref = HASH_POINTER(ip); 171 matchIndex = HashTable[LZ4HC_hashPtr(ip)];
451
452#define REPEAT_OPTIMIZATION
453#ifdef REPEAT_OPTIMIZATION
454 /* Detect repetitive sequences of length <= 4 */
455 if ((U32)(ip-ref) <= 4) /* potential repetition */
456 {
457 if (A32(ref) == A32(ip)) /* confirmed */
458 {
459 delta = (U16)(ip-ref);
460 repl = ml = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH;
461 *matchpos = ref;
462 }
463 ref = GETNEXT(ref);
464 }
465#endif
466 172
467 while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) 173 while ((matchIndex>=lowLimit) && (nbAttempts))
468 { 174 {
469 nbAttempts--; 175 nbAttempts--;
470 if (*(ref+ml) == *(ip+ml)) 176 if (matchIndex >= dictLimit)
471 if (A32(ref) == A32(ip))
472 { 177 {
473 size_t mlt = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH; 178 match = base + matchIndex;
474 if (mlt > ml) { ml = mlt; *matchpos = ref; } 179 if (*(match+ml) == *(ip+ml)
180 && (LZ4_read32(match) == LZ4_read32(ip)))
181 {
182 size_t mlt = LZ4_count(ip+MINMATCH, match+MINMATCH, iLimit) + MINMATCH;
183 if (mlt > ml) { ml = mlt; *matchpos = match; }
184 }
475 } 185 }
476 ref = GETNEXT(ref); 186 else
477 }
478
479#ifdef REPEAT_OPTIMIZATION
480 /* Complete table */
481 if (repl)
482 {
483 const BYTE* ptr = ip;
484 const BYTE* end;
485
486 end = ip + repl - (MINMATCH-1);
487 while(ptr < end-delta)
488 { 187 {
489 DELTANEXT(ptr) = delta; /* Pre-Load */ 188 match = dictBase + matchIndex;
490 ptr++; 189 if (LZ4_read32(match) == LZ4_read32(ip))
190 {
191 size_t mlt;
192 const BYTE* vLimit = ip + (dictLimit - matchIndex);
193 if (vLimit > iLimit) vLimit = iLimit;
194 mlt = LZ4_count(ip+MINMATCH, match+MINMATCH, vLimit) + MINMATCH;
195 if ((ip+mlt == vLimit) && (vLimit < iLimit))
196 mlt += LZ4_count(ip+mlt, base+dictLimit, iLimit);
197 if (mlt > ml) { ml = mlt; *matchpos = base + matchIndex; } /* virtual matchpos */
198 }
491 } 199 }
492 do 200 matchIndex -= chainTable[matchIndex & 0xFFFF];
493 {
494 DELTANEXT(ptr) = delta;
495 HashTable[HASH_VALUE(ptr)] = (HTYPE)((ptr) - base); /* Head of chain */
496 ptr++;
497 } while(ptr < end);
498 hc4->nextToUpdate = end;
499 } 201 }
500#endif
501 202
502 return (int)ml; 203 return (int)ml;
503} 204}
504 205
505 206
506FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (LZ4HC_Data_Structure* hc4, const BYTE* ip, const BYTE* startLimit, const BYTE* matchlimit, int longest, const BYTE** matchpos, const BYTE** startpos, const int maxNbAttempts) 207FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (
208 LZ4HC_Data_Structure* hc4,
209 const BYTE* const ip,
210 const BYTE* const iLowLimit,
211 const BYTE* const iHighLimit,
212 int longest,
213 const BYTE** matchpos,
214 const BYTE** startpos,
215 const int maxNbAttempts)
507{ 216{
508 U16* const chainTable = hc4->chainTable; 217 U16* const chainTable = hc4->chainTable;
509 HTYPE* const HashTable = hc4->hashTable; 218 U32* const HashTable = hc4->hashTable;
510 INITBASE(base,hc4->base); 219 const BYTE* const base = hc4->base;
511 const BYTE* ref; 220 const U32 dictLimit = hc4->dictLimit;
221 const BYTE* const lowPrefixPtr = base + dictLimit;
222 const U32 lowLimit = (hc4->lowLimit + 64 KB > (U32)(ip-base)) ? hc4->lowLimit : (U32)(ip - base) - (64 KB - 1);
223 const BYTE* const dictBase = hc4->dictBase;
224 U32 matchIndex;
512 int nbAttempts = maxNbAttempts; 225 int nbAttempts = maxNbAttempts;
513 int delta = (int)(ip-startLimit); 226 int delta = (int)(ip-iLowLimit);
227
514 228
515 /* First Match */ 229 /* First Match */
516 LZ4HC_Insert(hc4, ip); 230 LZ4HC_Insert(hc4, ip);
517 ref = HASH_POINTER(ip); 231 matchIndex = HashTable[LZ4HC_hashPtr(ip)];
518 232
519 while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) 233 while ((matchIndex>=lowLimit) && (nbAttempts))
520 { 234 {
521 nbAttempts--; 235 nbAttempts--;
522 if (*(startLimit + longest) == *(ref - delta + longest)) 236 if (matchIndex >= dictLimit)
523 if (A32(ref) == A32(ip))
524 { 237 {
525#if 1 238 const BYTE* matchPtr = base + matchIndex;
526 const BYTE* reft = ref+MINMATCH; 239 if (*(iLowLimit + longest) == *(matchPtr - delta + longest))
527 const BYTE* ipt = ip+MINMATCH; 240 if (LZ4_read32(matchPtr) == LZ4_read32(ip))
528 const BYTE* startt = ip; 241 {
242 int mlt = MINMATCH + LZ4_count(ip+MINMATCH, matchPtr+MINMATCH, iHighLimit);
243 int back = 0;
529 244
530 while (ipt<matchlimit-(STEPSIZE-1)) 245 while ((ip+back>iLowLimit)
531 { 246 && (matchPtr+back > lowPrefixPtr)
532 size_t diff = AARCH(reft) ^ AARCH(ipt); 247 && (ip[back-1] == matchPtr[back-1]))
533 if (!diff) { ipt+=STEPSIZE; reft+=STEPSIZE; continue; } 248 back--;
534 ipt += LZ4_NbCommonBytes(diff);
535 goto _endCount;
536 }
537 if (LZ4_ARCH64) if ((ipt<(matchlimit-3)) && (A32(reft) == A32(ipt))) { ipt+=4; reft+=4; }
538 if ((ipt<(matchlimit-1)) && (A16(reft) == A16(ipt))) { ipt+=2; reft+=2; }
539 if ((ipt<matchlimit) && (*reft == *ipt)) ipt++;
540_endCount:
541 reft = ref;
542#else
543 /* Easier for code maintenance, but unfortunately slower too */
544 const BYTE* startt = ip;
545 const BYTE* reft = ref;
546 const BYTE* ipt = ip + MINMATCH + LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit);
547#endif
548 249
549 while ((startt>startLimit) && (reft > hc4->inputBuffer) && (startt[-1] == reft[-1])) {startt--; reft--;} 250 mlt -= back;
550 251
551 if ((ipt-startt) > longest) 252 if (mlt > longest)
253 {
254 longest = (int)mlt;
255 *matchpos = matchPtr+back;
256 *startpos = ip+back;
257 }
258 }
259 }
260 else
261 {
262 const BYTE* matchPtr = dictBase + matchIndex;
263 if (LZ4_read32(matchPtr) == LZ4_read32(ip))
552 { 264 {
553 longest = (int)(ipt-startt); 265 size_t mlt;
554 *matchpos = reft; 266 int back=0;
555 *startpos = startt; 267 const BYTE* vLimit = ip + (dictLimit - matchIndex);
268 if (vLimit > iHighLimit) vLimit = iHighLimit;
269 mlt = LZ4_count(ip+MINMATCH, matchPtr+MINMATCH, vLimit) + MINMATCH;
270 if ((ip+mlt == vLimit) && (vLimit < iHighLimit))
271 mlt += LZ4_count(ip+mlt, base+dictLimit, iHighLimit);
272 while ((ip+back > iLowLimit) && (matchIndex+back > lowLimit) && (ip[back-1] == matchPtr[back-1])) back--;
273 mlt -= back;
274 if ((int)mlt > longest) { longest = (int)mlt; *matchpos = base + matchIndex + back; *startpos = ip+back; }
556 } 275 }
557 } 276 }
558 ref = GETNEXT(ref); 277 matchIndex -= chainTable[matchIndex & 0xFFFF];
559 } 278 }
560 279
561 return longest; 280 return longest;
@@ -564,34 +283,44 @@ _endCount:
564 283
565typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive; 284typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive;
566 285
286#define LZ4HC_DEBUG 0
287#if LZ4HC_DEBUG
288static unsigned debug = 0;
289#endif
290
567FORCE_INLINE int LZ4HC_encodeSequence ( 291FORCE_INLINE int LZ4HC_encodeSequence (
568 const BYTE** ip, 292 const BYTE** ip,
569 BYTE** op, 293 BYTE** op,
570 const BYTE** anchor, 294 const BYTE** anchor,
571 int matchLength, 295 int matchLength,
572 const BYTE* ref, 296 const BYTE* const match,
573 limitedOutput_directive limitedOutputBuffer, 297 limitedOutput_directive limitedOutputBuffer,
574 BYTE* oend) 298 BYTE* oend)
575{ 299{
576 int length; 300 int length;
577 BYTE* token; 301 BYTE* token;
578 302
303#if LZ4HC_DEBUG
304 if (debug) printf("literal : %u -- match : %u -- offset : %u\n", (U32)(*ip - *anchor), (U32)matchLength, (U32)(*ip-match));
305#endif
306
579 /* Encode Literal length */ 307 /* Encode Literal length */
580 length = (int)(*ip - *anchor); 308 length = (int)(*ip - *anchor);
581 token = (*op)++; 309 token = (*op)++;
582 if ((limitedOutputBuffer) && ((*op + length + (2 + 1 + LASTLITERALS) + (length>>8)) > oend)) return 1; /* Check output limit */ 310 if ((limitedOutputBuffer) && ((*op + (length>>8) + length + (2 + 1 + LASTLITERALS)) > oend)) return 1; /* Check output limit */
583 if (length>=(int)RUN_MASK) { int len; *token=(RUN_MASK<<ML_BITS); len = length-RUN_MASK; for(; len > 254 ; len-=255) *(*op)++ = 255; *(*op)++ = (BYTE)len; } 311 if (length>=(int)RUN_MASK) { int len; *token=(RUN_MASK<<ML_BITS); len = length-RUN_MASK; for(; len > 254 ; len-=255) *(*op)++ = 255; *(*op)++ = (BYTE)len; }
584 else *token = (BYTE)(length<<ML_BITS); 312 else *token = (BYTE)(length<<ML_BITS);
585 313
586 /* Copy Literals */ 314 /* Copy Literals */
587 LZ4_BLINDCOPY(*anchor, *op, length); 315 LZ4_wildCopy(*op, *anchor, (*op) + length);
316 *op += length;
588 317
589 /* Encode Offset */ 318 /* Encode Offset */
590 LZ4_WRITE_LITTLEENDIAN_16(*op,(U16)(*ip-ref)); 319 LZ4_writeLE16(*op, (U16)(*ip-match)); *op += 2;
591 320
592 /* Encode MatchLength */ 321 /* Encode MatchLength */
593 length = (int)(matchLength-MINMATCH); 322 length = (int)(matchLength-MINMATCH);
594 if ((limitedOutputBuffer) && (*op + (1 + LASTLITERALS) + (length>>8) > oend)) return 1; /* Check output limit */ 323 if ((limitedOutputBuffer) && (*op + (length>>8) + (1 + LASTLITERALS) > oend)) return 1; /* Check output limit */
595 if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length > 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) { length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; } 324 if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length > 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) { length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; }
596 else *token += (BYTE)(length); 325 else *token += (BYTE)(length);
597 326
@@ -603,16 +332,15 @@ FORCE_INLINE int LZ4HC_encodeSequence (
603} 332}
604 333
605 334
606#define MAX_COMPRESSION_LEVEL 16
607static int LZ4HC_compress_generic ( 335static int LZ4HC_compress_generic (
608 void* ctxvoid, 336 void* ctxvoid,
609 const char* source, 337 const char* source,
610 char* dest, 338 char* dest,
611 int inputSize, 339 int inputSize,
612 int maxOutputSize, 340 int maxOutputSize,
613 int compressionLevel, 341 int compressionLevel,
614 limitedOutput_directive limit 342 limitedOutput_directive limit
615 ) 343 )
616{ 344{
617 LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid; 345 LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid;
618 const BYTE* ip = (const BYTE*) source; 346 const BYTE* ip = (const BYTE*) source;
@@ -624,7 +352,7 @@ static int LZ4HC_compress_generic (
624 BYTE* op = (BYTE*) dest; 352 BYTE* op = (BYTE*) dest;
625 BYTE* const oend = op + maxOutputSize; 353 BYTE* const oend = op + maxOutputSize;
626 354
627 const int maxNbAttempts = compressionLevel > MAX_COMPRESSION_LEVEL ? 1 << MAX_COMPRESSION_LEVEL : compressionLevel ? 1<<(compressionLevel-1) : 1<<LZ4HC_DEFAULT_COMPRESSIONLEVEL; 355 unsigned maxNbAttempts;
628 int ml, ml2, ml3, ml0; 356 int ml, ml2, ml3, ml0;
629 const BYTE* ref=NULL; 357 const BYTE* ref=NULL;
630 const BYTE* start2=NULL; 358 const BYTE* start2=NULL;
@@ -635,8 +363,10 @@ static int LZ4HC_compress_generic (
635 const BYTE* ref0; 363 const BYTE* ref0;
636 364
637 365
638 /* Ensure blocks follow each other */ 366 /* init */
639 if (ip != ctx->end) return 0; 367 if (compressionLevel > g_maxCompressionLevel) compressionLevel = g_maxCompressionLevel;
368 if (compressionLevel < 1) compressionLevel = LZ4HC_compressionLevel_default;
369 maxNbAttempts = 1 << (compressionLevel-1);
640 ctx->end += inputSize; 370 ctx->end += inputSize;
641 371
642 ip++; 372 ip++;
@@ -684,10 +414,10 @@ _Search2:
684 414
685_Search3: 415_Search3:
686 /* 416 /*
687 * Currently we have : 417 * Currently we have :
688 * ml2 > ml1, and 418 * ml2 > ml1, and
689 * ip1+3 <= ip2 (usually < ip1+ml1) 419 * ip1+3 <= ip2 (usually < ip1+ml1)
690 */ 420 */
691 if ((start2 - ip) < OPTIMAL_ML) 421 if ((start2 - ip) < OPTIMAL_ML)
692 { 422 {
693 int correction; 423 int correction;
@@ -755,9 +485,9 @@ _Search3:
755 } 485 }
756 486
757 /* 487 /*
758 * OK, now we have 3 ascending matches; let's write at least the first one 488 * OK, now we have 3 ascending matches; let's write at least the first one
759 * ip & ref are known; Now for ml 489 * ip & ref are known; Now for ml
760 */ 490 */
761 if (start2 < ip+ml) 491 if (start2 < ip+ml)
762 { 492 {
763 if ((start2 - ip) < (int)ML_MASK) 493 if ((start2 - ip) < (int)ML_MASK)
@@ -789,7 +519,6 @@ _Search3:
789 ml2 = ml3; 519 ml2 = ml3;
790 520
791 goto _Search3; 521 goto _Search3;
792
793 } 522 }
794 523
795 /* Encode Last Literals */ 524 /* Encode Last Literals */
@@ -809,28 +538,18 @@ _Search3:
809 538
810int LZ4_compressHC2(const char* source, char* dest, int inputSize, int compressionLevel) 539int LZ4_compressHC2(const char* source, char* dest, int inputSize, int compressionLevel)
811{ 540{
812 void* ctx = LZ4_createHC(source); 541 LZ4HC_Data_Structure ctx;
813 int result; 542 LZ4HC_init(&ctx, (const BYTE*)source);
814 if (ctx==NULL) return 0; 543 return LZ4HC_compress_generic (&ctx, source, dest, inputSize, 0, compressionLevel, noLimit);
815
816 result = LZ4HC_compress_generic (ctx, source, dest, inputSize, 0, compressionLevel, noLimit);
817
818 LZ4_freeHC(ctx);
819 return result;
820} 544}
821 545
822int LZ4_compressHC(const char* source, char* dest, int inputSize) { return LZ4_compressHC2(source, dest, inputSize, 0); } 546int LZ4_compressHC(const char* source, char* dest, int inputSize) { return LZ4_compressHC2(source, dest, inputSize, 0); }
823 547
824int LZ4_compressHC2_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel) 548int LZ4_compressHC2_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel)
825{ 549{
826 void* ctx = LZ4_createHC(source); 550 LZ4HC_Data_Structure ctx;
827 int result; 551 LZ4HC_init(&ctx, (const BYTE*)source);
828 if (ctx==NULL) return 0; 552 return LZ4HC_compress_generic (&ctx, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput);
829
830 result = LZ4HC_compress_generic (ctx, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput);
831
832 LZ4_freeHC(ctx);
833 return result;
834} 553}
835 554
836int LZ4_compressHC_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize) 555int LZ4_compressHC_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize)
@@ -840,15 +559,15 @@ int LZ4_compressHC_limitedOutput(const char* source, char* dest, int inputSize,
840 559
841 560
842/***************************** 561/*****************************
843 Using external allocation 562 * Using external allocation
844*****************************/ 563 * ***************************/
845int LZ4_sizeofStateHC() { return sizeof(LZ4HC_Data_Structure); } 564int LZ4_sizeofStateHC(void) { return sizeof(LZ4HC_Data_Structure); }
846 565
847 566
848int LZ4_compressHC2_withStateHC (void* state, const char* source, char* dest, int inputSize, int compressionLevel) 567int LZ4_compressHC2_withStateHC (void* state, const char* source, char* dest, int inputSize, int compressionLevel)
849{ 568{
850 if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state is not aligned for pointers (32 or 64 bits) */ 569 if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state is not aligned for pointers (32 or 64 bits) */
851 LZ4_initHC ((LZ4HC_Data_Structure*)state, (const BYTE*)source); 570 LZ4HC_init ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
852 return LZ4HC_compress_generic (state, source, dest, inputSize, 0, compressionLevel, noLimit); 571 return LZ4HC_compress_generic (state, source, dest, inputSize, 0, compressionLevel, noLimit);
853} 572}
854 573
@@ -859,7 +578,7 @@ int LZ4_compressHC_withStateHC (void* state, const char* source, char* dest, int
859int LZ4_compressHC2_limitedOutput_withStateHC (void* state, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel) 578int LZ4_compressHC2_limitedOutput_withStateHC (void* state, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel)
860{ 579{
861 if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state is not aligned for pointers (32 or 64 bits) */ 580 if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state is not aligned for pointers (32 or 64 bits) */
862 LZ4_initHC ((LZ4HC_Data_Structure*)state, (const BYTE*)source); 581 LZ4HC_init ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
863 return LZ4HC_compress_generic (state, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput); 582 return LZ4HC_compress_generic (state, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput);
864} 583}
865 584
@@ -867,26 +586,171 @@ int LZ4_compressHC_limitedOutput_withStateHC (void* state, const char* source, c
867{ return LZ4_compressHC2_limitedOutput_withStateHC (state, source, dest, inputSize, maxOutputSize, 0); } 586{ return LZ4_compressHC2_limitedOutput_withStateHC (state, source, dest, inputSize, maxOutputSize, 0); }
868 587
869 588
870/****************************
871 Stream functions
872****************************/
873 589
874int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize) 590/**************************************
591 * Streaming Functions
592 * ************************************/
593/* allocation */
594LZ4_streamHC_t* LZ4_createStreamHC(void) { return (LZ4_streamHC_t*)malloc(sizeof(LZ4_streamHC_t)); }
595int LZ4_freeStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr) { free(LZ4_streamHCPtr); return 0; }
596
597
598/* initialization */
599void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)
875{ 600{
876 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, 0, noLimit); 601 LZ4_STATIC_ASSERT(sizeof(LZ4HC_Data_Structure) <= sizeof(LZ4_streamHC_t)); /* if compilation fails here, LZ4_STREAMHCSIZE must be increased */
602 ((LZ4HC_Data_Structure*)LZ4_streamHCPtr)->base = NULL;
603 ((LZ4HC_Data_Structure*)LZ4_streamHCPtr)->compressionLevel = (unsigned)compressionLevel;
877} 604}
878 605
879int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int compressionLevel) 606int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, const char* dictionary, int dictSize)
880{ 607{
881 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, compressionLevel, noLimit); 608 LZ4HC_Data_Structure* ctxPtr = (LZ4HC_Data_Structure*) LZ4_streamHCPtr;
609 if (dictSize > 64 KB)
610 {
611 dictionary += dictSize - 64 KB;
612 dictSize = 64 KB;
613 }
614 LZ4HC_init (ctxPtr, (const BYTE*)dictionary);
615 if (dictSize >= 4) LZ4HC_Insert (ctxPtr, (const BYTE*)dictionary +(dictSize-3));
616 ctxPtr->end = (const BYTE*)dictionary + dictSize;
617 return dictSize;
882} 618}
883 619
620
621/* compression */
622
623static void LZ4HC_setExternalDict(LZ4HC_Data_Structure* ctxPtr, const BYTE* newBlock)
624{
625 if (ctxPtr->end >= ctxPtr->base + 4)
626 LZ4HC_Insert (ctxPtr, ctxPtr->end-3); /* Referencing remaining dictionary content */
627 /* Only one memory segment for extDict, so any previous extDict is lost at this stage */
628 ctxPtr->lowLimit = ctxPtr->dictLimit;
629 ctxPtr->dictLimit = (U32)(ctxPtr->end - ctxPtr->base);
630 ctxPtr->dictBase = ctxPtr->base;
631 ctxPtr->base = newBlock - ctxPtr->dictLimit;
632 ctxPtr->end = newBlock;
633 ctxPtr->nextToUpdate = ctxPtr->dictLimit; /* match referencing will resume from there */
634}
635
636static int LZ4_compressHC_continue_generic (LZ4HC_Data_Structure* ctxPtr,
637 const char* source, char* dest,
638 int inputSize, int maxOutputSize, limitedOutput_directive limit)
639{
640 /* auto-init if forgotten */
641 if (ctxPtr->base == NULL)
642 LZ4HC_init (ctxPtr, (const BYTE*) source);
643
644 /* Check overflow */
645 if ((size_t)(ctxPtr->end - ctxPtr->base) > 2 GB)
646 {
647 size_t dictSize = (size_t)(ctxPtr->end - ctxPtr->base) - ctxPtr->dictLimit;
648 if (dictSize > 64 KB) dictSize = 64 KB;
649
650 LZ4_loadDictHC((LZ4_streamHC_t*)ctxPtr, (const char*)(ctxPtr->end) - dictSize, (int)dictSize);
651 }
652
653 /* Check if blocks follow each other */
654 if ((const BYTE*)source != ctxPtr->end) LZ4HC_setExternalDict(ctxPtr, (const BYTE*)source);
655
656 /* Check overlapping input/dictionary space */
657 {
658 const BYTE* sourceEnd = (const BYTE*) source + inputSize;
659 const BYTE* dictBegin = ctxPtr->dictBase + ctxPtr->lowLimit;
660 const BYTE* dictEnd = ctxPtr->dictBase + ctxPtr->dictLimit;
661 if ((sourceEnd > dictBegin) && ((BYTE*)source < dictEnd))
662 {
663 if (sourceEnd > dictEnd) sourceEnd = dictEnd;
664 ctxPtr->lowLimit = (U32)(sourceEnd - ctxPtr->dictBase);
665 if (ctxPtr->dictLimit - ctxPtr->lowLimit < 4) ctxPtr->lowLimit = ctxPtr->dictLimit;
666 }
667 }
668
669 return LZ4HC_compress_generic (ctxPtr, source, dest, inputSize, maxOutputSize, ctxPtr->compressionLevel, limit);
670}
671
672int LZ4_compressHC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* source, char* dest, int inputSize)
673{
674 return LZ4_compressHC_continue_generic ((LZ4HC_Data_Structure*)LZ4_streamHCPtr, source, dest, inputSize, 0, noLimit);
675}
676
677int LZ4_compressHC_limitedOutput_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* source, char* dest, int inputSize, int maxOutputSize)
678{
679 return LZ4_compressHC_continue_generic ((LZ4HC_Data_Structure*)LZ4_streamHCPtr, source, dest, inputSize, maxOutputSize, limitedOutput);
680}
681
682
683/* dictionary saving */
684
685int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int dictSize)
686{
687 LZ4HC_Data_Structure* streamPtr = (LZ4HC_Data_Structure*)LZ4_streamHCPtr;
688 int prefixSize = (int)(streamPtr->end - (streamPtr->base + streamPtr->dictLimit));
689 if (dictSize > 64 KB) dictSize = 64 KB;
690 if (dictSize < 4) dictSize = 0;
691 if (dictSize > prefixSize) dictSize = prefixSize;
692 memcpy(safeBuffer, streamPtr->end - dictSize, dictSize);
693 {
694 U32 endIndex = (U32)(streamPtr->end - streamPtr->base);
695 streamPtr->end = (const BYTE*)safeBuffer + dictSize;
696 streamPtr->base = streamPtr->end - endIndex;
697 streamPtr->dictLimit = endIndex - dictSize;
698 streamPtr->lowLimit = endIndex - dictSize;
699 if (streamPtr->nextToUpdate < streamPtr->dictLimit) streamPtr->nextToUpdate = streamPtr->dictLimit;
700 }
701 return dictSize;
702}
703
704
705/***********************************
706 * Deprecated Functions
707 ***********************************/
708int LZ4_sizeofStreamStateHC(void) { return LZ4_STREAMHCSIZE; }
709
710int LZ4_resetStreamStateHC(void* state, const char* inputBuffer)
711{
712 if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error : pointer is not aligned for pointer (32 or 64 bits) */
713 LZ4HC_init((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer);
714 return 0;
715}
716
717void* LZ4_createHC (const char* inputBuffer)
718{
719 void* hc4 = ALLOCATOR(1, sizeof(LZ4HC_Data_Structure));
720 LZ4HC_init ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer);
721 return hc4;
722}
723
724int LZ4_freeHC (void* LZ4HC_Data)
725{
726 FREEMEM(LZ4HC_Data);
727 return (0);
728}
729
730/*
731int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize)
732{
733return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, 0, noLimit);
734}
884int LZ4_compressHC_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize) 735int LZ4_compressHC_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize)
885{ 736{
886 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, maxOutputSize, 0, limitedOutput); 737return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, maxOutputSize, 0, limitedOutput);
738}
739*/
740
741int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int compressionLevel)
742{
743 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, compressionLevel, noLimit);
887} 744}
888 745
889int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel) 746int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel)
890{ 747{
891 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput); 748 return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, maxOutputSize, compressionLevel, limitedOutput);
892} 749}
750
751char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
752{
753 LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data;
754 int dictSize = LZ4_saveDictHC((LZ4_streamHC_t*)LZ4HC_Data, (char*)(hc4->inputBuffer), 64 KB);
755 return (char*)(hc4->inputBuffer + dictSize);
756}
diff --git a/src/static_libs/lz4/lz4hc.h b/src/static_libs/lz4/lz4hc.h
index deb2394..4a05845 100644
--- a/src/static_libs/lz4/lz4hc.h
+++ b/src/static_libs/lz4/lz4hc.h
@@ -1,7 +1,7 @@
1/* 1/*
2 LZ4 HC - High Compression Mode of LZ4 2 LZ4 HC - High Compression Mode of LZ4
3 Header File 3 Header File
4 Copyright (C) 2011-2014, Yann Collet. 4 Copyright (C) 2011-2015, Yann Collet.
5 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) 5 BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
6 6
7 Redistribution and use in source and binary forms, with or without 7 Redistribution and use in source and binary forms, with or without
@@ -28,8 +28,8 @@
28 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29 29
30 You can contact the author at : 30 You can contact the author at :
31 - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html 31 - LZ4 source repository : https://github.com/Cyan4973/lz4
32 - LZ4 source repository : http://code.google.com/p/lz4/ 32 - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
33*/ 33*/
34#pragma once 34#pragma once
35 35
@@ -74,12 +74,12 @@ int LZ4_compressHC2_limitedOutput (const char* source, char* dest, int inputSize
74*/ 74*/
75 75
76/* Note : 76/* Note :
77Decompression functions are provided within LZ4 source code (see "lz4.h") (BSD license) 77 Decompression functions are provided within LZ4 source code (see "lz4.h") (BSD license)
78*/ 78*/
79 79
80 80
81/************************************** 81/**************************************
82 Using an external allocation 82* Using an external allocation
83**************************************/ 83**************************************/
84int LZ4_sizeofStateHC(void); 84int LZ4_sizeofStateHC(void);
85int LZ4_compressHC_withStateHC (void* state, const char* source, char* dest, int inputSize); 85int LZ4_compressHC_withStateHC (void* state, const char* source, char* dest, int inputSize);
@@ -95,79 +95,86 @@ int LZ4_sizeofStateHC();
95 95
96Note that tables must be aligned for pointer (32 or 64 bits), otherwise compression will fail (return code 0). 96Note that tables must be aligned for pointer (32 or 64 bits), otherwise compression will fail (return code 0).
97 97
98The allocated memory can be provided to the compressions functions using 'void* state' parameter. 98The allocated memory can be provided to the compression functions using 'void* state' parameter.
99LZ4_compress_withStateHC() and LZ4_compress_limitedOutput_withStateHC() are equivalent to previously described functions. 99LZ4_compress_withStateHC() and LZ4_compress_limitedOutput_withStateHC() are equivalent to previously described functions.
100They just use the externally allocated memory area instead of allocating their own (on stack, or on heap). 100They just use the externally allocated memory for state instead of allocating their own (on stack, or on heap).
101*/ 101*/
102 102
103 103
104/**************************************
105 Streaming Functions
106**************************************/
107/* Note : these streaming functions still follows the older model */
108void* LZ4_createHC (const char* inputBuffer);
109int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize);
110int LZ4_compressHC_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize);
111char* LZ4_slideInputBufferHC (void* LZ4HC_Data);
112int LZ4_freeHC (void* LZ4HC_Data);
113 104
114int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int compressionLevel); 105/*****************************
115int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel); 106* Includes
107*****************************/
108#include <stddef.h> /* size_t */
116 109
110
111/**************************************
112* Experimental Streaming Functions
113**************************************/
114#define LZ4_STREAMHCSIZE 262192
115#define LZ4_STREAMHCSIZE_SIZET (LZ4_STREAMHCSIZE / sizeof(size_t))
116typedef struct { size_t table[LZ4_STREAMHCSIZE_SIZET]; } LZ4_streamHC_t;
117/* 117/*
118These functions allow the compression of dependent blocks, where each block benefits from prior 64 KB within preceding blocks. 118LZ4_streamHC_t
119In order to achieve this, it is necessary to start creating the LZ4HC Data Structure, thanks to the function : 119This structure allows static allocation of LZ4 HC streaming state.
120State must then be initialized using LZ4_resetStreamHC() before first use.
120 121
121void* LZ4_createHC (const char* inputBuffer); 122Static allocation should only be used with statically linked library.
122The result of the function is the (void*) pointer on the LZ4HC Data Structure. 123If you want to use LZ4 as a DLL, please use construction functions below, which are more future-proof.
123This pointer will be needed in all other functions.
124If the pointer returned is NULL, then the allocation has failed, and compression must be aborted.
125The only parameter 'const char* inputBuffer' must, obviously, point at the beginning of input buffer.
126The input buffer must be already allocated, and size at least 192KB.
127'inputBuffer' will also be the 'const char* source' of the first block.
128
129All blocks are expected to lay next to each other within the input buffer, starting from 'inputBuffer'.
130To compress each block, use either LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue().
131Their behavior are identical to LZ4_compressHC() or LZ4_compressHC_limitedOutput(),
132but require the LZ4HC Data Structure as their first argument, and check that each block starts right after the previous one.
133If next block does not begin immediately after the previous one, the compression will fail (return 0).
134
135When it's no longer possible to lay the next block after the previous one (not enough space left into input buffer), a call to :
136char* LZ4_slideInputBufferHC(void* LZ4HC_Data);
137must be performed. It will typically copy the latest 64KB of input at the beginning of input buffer.
138Note that, for this function to work properly, minimum size of an input buffer must be 192KB.
139==> The memory position where the next input data block must start is provided as the result of the function.
140
141Compression can then resume, using LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue(), as usual.
142
143When compression is completed, a call to LZ4_freeHC() will release the memory used by the LZ4HC Data Structure.
144*/ 124*/
145 125
146int LZ4_sizeofStreamStateHC(void);
147int LZ4_resetStreamStateHC(void* state, const char* inputBuffer);
148 126
127LZ4_streamHC_t* LZ4_createStreamHC(void);
128int LZ4_freeStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr);
149/* 129/*
150These functions achieve the same result as : 130These functions create and release memory for LZ4 HC streaming state.
151void* LZ4_createHC (const char* inputBuffer); 131Newly created states are already initialized.
132Existing state space can be re-used anytime using LZ4_resetStreamHC().
133If you use LZ4 as a DLL, please use these functions instead of direct struct allocation,
134to avoid size mismatch between different versions.
135*/
136
137void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel);
138int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, const char* dictionary, int dictSize);
152 139
153They are provided here to allow the user program to allocate memory using its own routines. 140int LZ4_compressHC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* source, char* dest, int inputSize);
141int LZ4_compressHC_limitedOutput_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* source, char* dest, int inputSize, int maxOutputSize);
154 142
155To know how much space must be allocated, use LZ4_sizeofStreamStateHC(); 143int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int maxDictSize);
156Note also that space must be aligned for pointers (32 or 64 bits).
157 144
158Once space is allocated, you must initialize it using : LZ4_resetStreamStateHC(void* state, const char* inputBuffer); 145/*
159void* state is a pointer to the space allocated. 146These functions compress data in successive blocks of any size, using previous blocks as dictionary.
160It must be aligned for pointers (32 or 64 bits), and be large enough. 147One key assumption is that each previous block will remain read-accessible while compressing next block.
161The parameter 'const char* inputBuffer' must, obviously, point at the beginning of input buffer. 148
162The input buffer must be already allocated, and size at least 192KB. 149Before starting compression, state must be properly initialized, using LZ4_resetStreamHC().
163'inputBuffer' will also be the 'const char* source' of the first block. 150A first "fictional block" can then be designated as initial dictionary, using LZ4_loadDictHC() (Optional).
164 151
165The same space can be re-used multiple times, just by initializing it each time with LZ4_resetStreamState(). 152Then, use LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue() to compress each successive block.
166return value of LZ4_resetStreamStateHC() must be 0 is OK. 153They work like usual LZ4_compressHC() or LZ4_compressHC_limitedOutput(), but use previous memory blocks to improve compression.
167Any other value means there was an error (typically, state is not aligned for pointers (32 or 64 bits)). 154Previous memory blocks (including initial dictionary when present) must remain accessible and unmodified during compression.
155
156If, for any reason, previous data block can't be preserved in memory during next compression block,
157you must save it to a safer memory space,
158using LZ4_saveDictHC().
168*/ 159*/
169 160
170 161
162
163/**************************************
164 * Deprecated Streaming Functions
165 * ************************************/
166/* Note : these streaming functions follows the older model, and should no longer be used */
167void* LZ4_createHC (const char* inputBuffer);
168char* LZ4_slideInputBufferHC (void* LZ4HC_Data);
169int LZ4_freeHC (void* LZ4HC_Data);
170
171int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int compressionLevel);
172int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel);
173
174int LZ4_sizeofStreamStateHC(void);
175int LZ4_resetStreamStateHC(void* state, const char* inputBuffer);
176
177
171#if defined (__cplusplus) 178#if defined (__cplusplus)
172} 179}
173#endif 180#endif