RELEASE_NOTES.rst 33 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965
  1. ===========================
  2. Release notes for C-Blosc
  3. ===========================
  4. :Author: Francesc Alted
  5. :Contact: francesc@blosc.org
  6. :URL: http://www.blosc.org
  7. Changes from 1.14.2 to 1.14.3
  8. =============================
  9. - Use win32/pthread.c on all Windows builds, even those with GNU compilers.
  10. Rational: although MinGW provides a more full-featured pthreads replacement,
  11. it doesn't seem to accomplish anything here since the functionality in
  12. win32/pthread.c is sufficient for Blosc. Furthermore, using the MinGW
  13. pthreads adds an additional library dependency to libblosc that is
  14. annoying for binary distribution. For example, it got in the way of
  15. distributing cross-compiled Windows binaries for use with Julia, since they
  16. want the resulting libblosc.dll to be usable on any Windows machine even
  17. where MinGW is not installed. See PR #224. Thanks to Steven G. Johnson.
  18. - Zstd internal sources have been updated to 1.3.4.
  19. Changes from 1.14.1 to 1.14.2
  20. =============================
  21. - Reverted the $Configuration var in CMake configuration for Windows so
  22. as to restore the compatibility with MS VisualStudio compilers.
  23. Changes from 1.14.0 to 1.14.1
  24. =============================
  25. - Fixed a bug that caused C-Blosc to crash on platforms requiring strict
  26. alignment (as in some kinds of ARM CPUs). Fixes #223. Thanks to Elvis
  27. Stansvik and Michael Hudson-Doyle for their help.
  28. - Fixed a piece of code that was not C89 compliant. C89 compliance is
  29. needed mainly by MS VS2008 which is still used for creating Python 2
  30. extensions.
  31. - Remove the (spurious) $Configuration var in cmake config for Windows.
  32. Thanks to Francis Brissette for pointing this out.
  33. Changes from 1.13.7 to 1.14.0
  34. =============================
  35. - New split mode that favors forward compatibility. That means that,
  36. from now on, all the buffers created starting with blosc 1.14.0 will
  37. be forward compatible with any previous versions of the library --at
  38. least until 1.3.0, when support for multi-codec was introduced.
  39. So as to select the split mode, a new API function has been introduced:
  40. https://github.com/Blosc/c-blosc/blob/master/blosc/blosc.h#L500
  41. Also, the BLOSC_SPLITMODE environment variable is honored when using
  42. the `blosc_compress()` function. See
  43. https://github.com/Blosc/c-blosc/blob/master/blosc/blosc.h#L209
  44. There is a dedicated blog entry about this at:
  45. http://blosc.org/posts/new-forward-compat-policy/
  46. More info in PR #216.
  47. Caveat Emptor: Note that Blosc versions from 1.11.0 to 1.14.0 *might*
  48. generate buffers that cannot be read with versions < 1.11.0, so if
  49. forward compatibility is important to you, an upgrade to 1.14.0 is
  50. recommended.
  51. - All warnings during cmake build stage are enabled by default now.
  52. PR #218. Thanks to kalvdans.
  53. - Better checks on versions of formats inside Blosc. PR #219. Thanks
  54. to kalvdans.
  55. - The BLOSC_PRINT_SHUFFLE_ACCEL environment variable is honored now.
  56. This is useful for determining *at runtime* whether the different SIMD
  57. capabilities (only for x86 kind processors) are available to Blosc to get
  58. better performance during shuffle/bitshuffle operation. As an example,
  59. here it is the normal output for the simple.c example::
  60. $ ./simple
  61. Blosc version info: 1.14.0.dev ($Date:: 2018-02-15 #$)
  62. Compression: 4000000 -> 41384 (96.7x)
  63. Decompression succesful!
  64. Succesful roundtrip!
  65. and here with the BLOSC_PRINT_SHUFFLE_ACCEL environment variable set::
  66. $ BLOSC_PRINT_SHUFFLE_ACCEL= ./simple
  67. Blosc version info: 1.14.0.dev ($Date:: 2018-02-15 #$)
  68. Shuffle CPU Information:
  69. SSE2 available: True
  70. SSE3 available: True
  71. SSSE3 available: True
  72. SSE4.1 available: True
  73. SSE4.2 available: True
  74. AVX2 available: True
  75. AVX512BW available: False
  76. XSAVE available: True
  77. XSAVE enabled: True
  78. XMM state enabled: True
  79. YMM state enabled: True
  80. ZMM state enabled: False
  81. Compression: 4000000 -> 41384 (96.7x)
  82. Decompression succesful!
  83. Succesful roundtrip!
  84. Blosc only currently leverages the SSE2 and AVX2 instruction sets, but
  85. it can recognize all of the above. This is useful mainly for debugging.
  86. Changes from 1.13.6 to 1.13.7
  87. =============================
  88. - More tests for binaries in https://bintray.com/blosc/Conan.
  89. Changes from 1.13.5 to 1.13.6
  90. =============================
  91. - More tests for binaries in https://bintray.com/blosc/Conan.
  92. Changes from 1.13.4 to 1.13.5
  93. =============================
  94. - New conan binaries publicly accessible in https://bintray.com/blosc/Conan.
  95. Still experimental, but feedback is appreciated.
  96. Changes from 1.13.3 to 1.13.4
  97. =============================
  98. - Fixed a buffer overrun that happens when compressing small buffers and
  99. len(destination_buffer) < (len(source_buffer) + BLOSC_MAX_OVERHEAD).
  100. Reported by Ivan Smirnov.
  101. Changes from 1.13.2 to 1.13.3
  102. =============================
  103. - Tests work now when external compressors are located in non-system locations.
  104. Fixes #210. Thanks to Leif Walsh.
  105. Changes from 1.13.1 to 1.13.2
  106. =============================
  107. - C-Blosc can be compiled on CentOS 6 now.
  108. - LZ4 internal codec upgraded to 1.8.1.
  109. Changes from 1.13.0 to 1.13.1
  110. =============================
  111. - Fixed a bug uncovered by the python-blosc test suite: when a buffer is
  112. to be copied, then we should reserve space for the header, not block pointers.
  113. Changes from 1.12.1 to 1.13.0
  114. =============================
  115. - Serious optimization of memory copy functions (see new `blosc/fastcopy.c`).
  116. This benefits the speed of all the codecs, but specially the BloscLZ one.
  117. - As a result of the above, the BloscLZ codec received a new adjustment of
  118. knobs so that you should expect better compression ratios with it too.
  119. - LZ4 internal sources have been updated to 1.8.0.
  120. - Zstd internal sources have been updated to 1.3.3.
  121. Changes from 1.12.0 to 1.12.1
  122. =============================
  123. - Backported BloscLZ parameters that were fine-tuned for C-Blosc2.
  124. You should expect better compression ratios and faster operation,
  125. specially on modern CPUs. See:
  126. http://blosc.org/posts/blosclz-tuning/
  127. Changes from 1.11.3 to 1.12.0
  128. =============================
  129. - Snappy, Zlib and Zstd codecs are compiled internally now, even if they are
  130. installed in the machine. This has been done in order to avoid
  131. problems in machines having the shared libraries for the codecs
  132. accessible but not the includes (typical in Windows boxes). Also,
  133. the Zstd codec runs much faster when compiled internally. The
  134. previous behaviour can be restored by activating the cmake options
  135. PREFER_EXTERNAL_SNAPPY, PREFER_EXTERNAL_ZLIB and PREFER_EXTERNAL_ZSTD.
  136. - Zstd internal sources have been updated to 1.3.0.
  137. Changes from 1.11.3 to 1.11.4
  138. =============================
  139. - Internal Zstd codec updated to 1.1.4.
  140. Changes from 1.11.2 to 1.11.3
  141. =============================
  142. - Fixed #181: bitshuffle filter for big endian machines.
  143. - Internal Zstd codec updated to 1.1.3.
  144. - New blocksize for complevel 8 in automatic mode. This should help specially
  145. the Zstd codec to achieve better compression ratios.
  146. Changes from 1.11.1 to 1.11.2
  147. =============================
  148. - Enabled use as a CMake subproject, exporting shared & static library targets
  149. for super-projects to use. See PRs #178, #179 and #180. Thanks to Kevin
  150. Murray.
  151. - Internal LZ4 codec updated to 1.7.5.
  152. - Internal Zstd codec updated to 1.1.2.
  153. Changes from 1.11.0 to 1.11.1
  154. =============================
  155. - Fixed a bug introduced in 1.11.0 and discovered by pandas test suite. This
  156. basically prevented to decompress buffers compressed with previous versions of
  157. C-Blosc. See: https://github.com/Blosc/python-blosc/issues/115
  158. Changes from 1.10.2 to 1.11.0
  159. =============================
  160. - Internal Zstd codec upgraded to 1.0.0.
  161. - New block size computation inherited from C-Blosc2. Benchmarks are saying that
  162. this benefits mainly to LZ4, LZ4HC, Zlib and Zstd codecs, both in speed and in
  163. compression ratios (although YMMV for your case).
  164. - Added the @rpath flag in Mac OSX for shared libraries. Fixes #175.
  165. - Added a fix for VS2008 discovered in: https://github.com/PyTables/PyTables/pull/569/files#diff-953cf824ebfea7208d2a2e312d9ccda2L126
  166. - License changed from MIT to 3-clause BSD style.
  167. Changes from 1.10.1 to 1.10.2
  168. =============================
  169. - Force the use of --std=gnu99 when using gcc. Fixes #174.
  170. Changes from 1.10.0 to 1.10.1
  171. =============================
  172. - Removed an inconsistent check for C11 (__STDC_VERSION__ >= 201112L and
  173. _ISOC11_SOURCE) as this seem to pose problems on compilers doing different
  174. things in this check (e.g. clang). See
  175. https://github.com/Blosc/bloscpack/issues/50.
  176. Changes from 1.9.3 to 1.10.0
  177. ============================
  178. - Initial support for Zstandard (0.7.4). Zstandard (or Zstd for short) is a new
  179. compression library that allows better compression than Zlib, but that works
  180. typically faster (and some times much faster), making of it a good match for
  181. Blosc.
  182. Although the Zstd format is considered stable
  183. (http://fastcompression.blogspot.com.es/2016_07_03_archive.html), its API is
  184. maturing very fast, and despite passing the extreme test suite for C-Blosc,
  185. this codec should be considered in beta for C-Blosc usage purposes. Please
  186. test it and report back any possible issues you may get.
  187. Changes from 1.9.2 to 1.9.3
  188. ===========================
  189. - Reverted a mistake introduced in 1.7.1. At that time, bit-shuffling
  190. was enabled for typesize == 1 (i.e. strings), but the change also
  191. included byte-shuffling accidentally. This only affected performance,
  192. but in a quite bad way (a copy was needed). This has been fixed and
  193. byte-shuffling is not active when typesize == 1 anymore.
  194. Changes from 1.9.1 to 1.9.2
  195. ===========================
  196. - Check whether Blosc is actually initialized before blosc_init(),
  197. blosc_destroy() and blosc_free_resources(). This makes the library
  198. more resistant to different initialization cycles
  199. (e.g. https://github.com/stevengj/Blosc.jl/issues/19).
  200. Changes from 1.9.0 to 1.9.1
  201. ===========================
  202. - The internal copies when clevel=0 are made now via memcpy(). At the
  203. beginning of C-Blosc development, benchmarks where saying that the
  204. internal, multi-threaded copies inside C-Blosc were faster than
  205. memcpy(), but 6 years later, memcpy() made greats strides in terms
  206. of efficiency. With this, you should expect an slight speed
  207. advantage (10% ~ 20%) when C-Blosc is used as a replacement of
  208. memcpy() (which should not be the most common scenario out there).
  209. - Added a new DEACTIVATE_AVX2 cmake option to explicitly disable AVX2
  210. at build-time. Thanks to James Bird.
  211. - The ``make -jN`` for parallel compilation should work now. Thanks
  212. to James Bird.
  213. Changes from 1.8.1 to 1.9.0
  214. ===========================
  215. * New blosc_get_nthreads() function to get the number of threads that
  216. will be used internally during compression/decompression (set by
  217. already existing blosc_set_nthreads()).
  218. * New blosc_get_compressor() function to get the compressor that will
  219. be used internally during compression (set by already existing
  220. blosc_set_compressor()).
  221. * New blosc_get_blocksize() function to get the internal blocksize to
  222. be used during compression (set by already existing
  223. blosc_set_blocksize()).
  224. * Now, when the BLOSC_NOLOCK environment variable is set (to any
  225. value), the calls to blosc_compress() and blosc_decompress() will
  226. call blosc_compress_ctx() and blosc_decompress_ctx() under the hood
  227. so as to avoid the internal locks. See blosc.h for details. This
  228. allows multi-threaded apps calling the non _ctx() functions to avoid
  229. the internal locks in C-Blosc. For the not multi-threaded app
  230. though, it is in general slower to call the _ctx() functions so the
  231. use of BLOSC_NOLOCK is discouraged.
  232. * In the same vein, from now on, when the BLOSC_NTHREADS environment
  233. variable is set to an integer, every call to blosc_compress() and
  234. blosc_decompress() will call blosc_set_nthreads(BLOSC_NTHREADS)
  235. before the actuall compression/decompression process. See blosc.h
  236. for details.
  237. * Finally, if BLOSC_CLEVEL, BLOSC_SHUFFLE, BLOSC_TYPESIZE and/or
  238. BLOSC_COMPRESSOR variables are set in the environment, these will be
  239. also honored before calling blosc_compress().
  240. * Calling blosc_init() before any other Blosc call, although
  241. recommended, is not necessary anymore. The idea is that you can use
  242. just the basic blosc_compress() and blosc_decompress() and control
  243. other parameters (nthreads, compressor, blocksize) by using
  244. environment variables (see above).
  245. Changes from 1.8.0 to 1.8.1
  246. ===========================
  247. * Disable the use of __builtin_cpu_supports() for GCC 5.3.1
  248. compatibility. Details in:
  249. https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/ZM2L65WIZEEQHHLFERZYD5FAG7QY2OGB/
  250. Changes from 1.7.1 to 1.8.0
  251. ===========================
  252. * The code is (again) compatible with VS2008 and VS2010. This is
  253. important for compatibility with Python 2.6/2.7/3.3/3.4.
  254. * Introduced a new global lock during blosc_decompress() operation.
  255. As the blosc_compress() was already guarded by a global lock, this
  256. means that the compression/decompression is again thread safe.
  257. However, when using C-Blosc from multi-threaded environments, it is
  258. important to keep using the *_ctx() functions for performance
  259. reasons. NOTE: _ctx() functions will be replaced by more powerful
  260. ones in C-Blosc 2.0.
  261. Changes from 1.7.0 to 1.7.1
  262. ===========================
  263. * Fixed a bug preventing bitshuffle to work correctly on getitem().
  264. Now, everything with bitshuffle seems to work correctly.
  265. * Fixed the thread initialization for blosc_decompress_ctx(). Issue
  266. #158. Thanks to Chris Webers.
  267. * Fixed a bug in the blocksize computation introduced in 1.7.0. This
  268. could have been creating segfaults.
  269. * Allow bitshuffle to run on 1-byte typesizes.
  270. * New parametrization of the blocksize to be independent of the
  271. typesize. This allows a smoother speed throughout all typesizes.
  272. * lz4 and lz4hc codecs upgraded to 1.7.2 (from 1.7.0).
  273. * When calling set_nthreads() but not actually changing the number of
  274. threads in the internal pool does not teardown and setup it anymore.
  275. PR #153. Thanks to Santi Villalba.
  276. Changes from 1.6.1 to 1.7.0
  277. ===========================
  278. * Added a new 'bitshuffle' filter so that the shuffle takes place at a
  279. bit level and not just at a byte one, which is what it does the
  280. previous 'shuffle' filter.
  281. For activating this new bit-level filter you only have to pass the
  282. symbol BLOSC_BITSHUFFLE to `blosc_compress()`. For the previous
  283. byte-level one, pass BLOSC_SHUFFLE. For disabling the shuffle, pass
  284. BLOSC_NOSHUFFLE.
  285. This is a port of the existing filter in
  286. https://github.com/kiyo-masui/bitshuffle. Thanks to Kiyo Masui for
  287. changing the license and allowing its inclusion here.
  288. * New acceleration mode for LZ4 and BloscLZ codecs that enters in
  289. operation with complevel < 9. This allows for an important boost in
  290. speed with minimal compression ratio loss. Francesc Alted.
  291. * LZ4 codec updated to 1.7.0 (r130).
  292. * PREFER_EXTERNAL_COMPLIBS cmake option has been removed and replaced
  293. by the more fine grained PREFER_EXTERNAL_LZ4, PREFER_EXTERNAL_SNAPPY
  294. and PREFER_EXTERNAL_ZLIB. In order to allow the use of the new API
  295. introduced in LZ4 1.7.0, PREFER_EXTERNAL_LZ4 has been set to OFF by
  296. default, whereas PREFER_EXTERNAL_SNAPPY and PREFER_EXTERNAL_ZLIB
  297. continues to be ON.
  298. * Implemented SSE2 shuffle support for buffers containing a number of
  299. elements which is not a multiple of (typesize * vectorsize). Jack
  300. Pappas.
  301. * Added SSE2 shuffle/unshuffle routines for types larger than 16
  302. bytes. Jack Pappas.
  303. * 'test_basic' suite has been split in components for a much better
  304. granularity on what's a possibly failing test. Also, lots of new
  305. tests have been added. Jack Pappas.
  306. * Fixed compilation on non-Intel archs (tested on ARM). Zbyszek
  307. Szmek.
  308. * Modifyied cmake files in order to inform that AVX2 on Visual Studio
  309. is supported only in 2013 update 2 and higher.
  310. * Added a replacement for stdbool.h for Visual Studio < 2013.
  311. * blosclz codec adds Win64/Intel as a platform supporting unaligned
  312. addressing. That leads to a speed-up of 2.2x in decompression.
  313. * New blosc_get_version_string() function for retrieving the version
  314. of the c-blosc library. Useful when linking with dynamic libraries
  315. and one want to know its version.
  316. * New example (win-dynamic-linking.c) that shows how to link a Blosc
  317. DLL dynamically in run-time (Windows only).
  318. * The `context.threads_started` is initialized now when decompressing.
  319. This could cause crashes in case you decompressed before compressing
  320. (e.g. directly deserializing blosc buffers). @atchouprakov.
  321. * The HDF5 filter has been removed from c-blosc and moved into its own
  322. repo at: https://github.com/Blosc/hdf5
  323. * The MS Visual Studio 2008 has been tested with c-blosc for ensuring
  324. compatibility with extensions for Python 2.6 and up.
  325. Changes from 1.6.0 to 1.6.1
  326. ===========================
  327. * Support for *runtime* detection of AVX2 and SSE2 SIMD instructions.
  328. These changes make it possible to compile one single binary that
  329. runs on a system that supports SSE2 or AVX2 (or neither), so the
  330. redistribution problem is fixed (see #101). Thanks to Julian Taylor
  331. and Jack Pappas.
  332. * Added support for MinGW and TDM-GCC compilers for Windows. Thanks
  333. to yasushima-gd.
  334. * Fixed a bug in blosclz that could potentially overwrite an area
  335. beyond the output buffer. See #113.
  336. * New computation for blocksize so that larger typesizes (> 8 bytes)
  337. would benefit of much better compression ratios. Speed is not
  338. penalized too much.
  339. * New parametrization of the hash table for blosclz codec. This
  340. allows better compression in many scenarios, while slightly
  341. increasing the speed.
  342. Changes from 1.5.4 to 1.6.0
  343. ===========================
  344. * Support for AVX2 is here! The benchmarks with a 4-core Intel
  345. Haswell machine tell that both compression and decompression are
  346. accelerated around a 10%, reaching peaks of 9.6 GB/s during
  347. compression and 26 GB/s during decompression (memcpy() speed for
  348. this machine is 7.5 GB/s for writes and 11.7 GB/s for reads). Many
  349. thanks to @littlezhou for this nice work.
  350. * Support for HPET (high precision timers) for the `bench` program.
  351. This is particularly important for microbenchmarks like bench is
  352. doing; since they take so little time to run, the granularity of a
  353. less-accurate timer may account for a significant portion of the
  354. runtime of the benchmark itself, skewing the results. Thanks to
  355. Jack Pappas.
  356. Changes from 1.5.3 to 1.5.4
  357. ===========================
  358. * Updated to LZ4 1.6.0 (r128).
  359. * Fix resource leak in t_blosc. Jack Pappas.
  360. * Better checks during testing. Jack Pappas.
  361. * Dynamically loadable HDF5 filter plugin. Kiyo Masui.
  362. Changes from 1.5.2 to 1.5.3
  363. ===========================
  364. * Use llabs function (where available) instead of abs to avoid
  365. truncating the result. Jack Pappas.
  366. * Use C11 aligned_alloc when it's available. Jack Pappas.
  367. * Use the built-in stdint.h with MSVC when available. Jack Pappas.
  368. * Only define the __SSE2__ symbol when compiling with MS Visual C++
  369. and targeting x64 or x86 with the correct /arch flag set. This
  370. avoids re-defining the symbol which makes other compilers issue
  371. warnings. Jack Pappas.
  372. * Reinitializing Blosc during a call to set_nthreads() so as to fix
  373. problems with contexts. Francesc Alted.
  374. Changes from 1.5.1 to 1.5.2
  375. ===========================
  376. * Using blosc_compress_ctx() / blosc_decompress_ctx() inside the HDF5
  377. compressor for allowing operation in multiprocess scenarios. See:
  378. https://github.com/PyTables/PyTables/issues/412
  379. The drawback of this quick fix is that the Blosc filter will be only
  380. able to use a single thread until another solution can be devised.
  381. Changes from 1.5.0 to 1.5.1
  382. ===========================
  383. * Updated to LZ4 1.5.0. Closes #74.
  384. * Added the 'const' qualifier to non SSE2 shuffle functions. Closes #75.
  385. * Explicitly call blosc_init() in HDF5 blosc_filter.c, fixing a
  386. segfault.
  387. * Quite a few improvements in cmake files for HDF5 support. Thanks to
  388. Dana Robinson (The HDF Group).
  389. * Variable 'class' caused problems compiling the HDF5 filter with g++.
  390. Thanks to Laurent Chapon.
  391. * Small improvements on docstrings of c-blosc main functions.
  392. Changes from 1.4.1 to 1.5.0
  393. ===========================
  394. * Added new calls for allowing Blosc to be used *simultaneously*
  395. (i.e. lock free) from multi-threaded environments. The new
  396. functions are:
  397. - blosc_compress_ctx(...)
  398. - blosc_decompress_ctx(...)
  399. See the new docstrings in blosc.h for how to use them. The previous
  400. API should be completely unaffected. Thanks to Christopher Speller.
  401. * Optimized copies during BloscLZ decompression. This can make BloscLZ
  402. to decompress up to 1.5x faster in some situations.
  403. * LZ4 and LZ4HC compressors updated to version 1.3.1.
  404. * Added an examples directory on how to link apps with Blosc.
  405. * stdlib.h moved from blosc.c to blosc.h as suggested by Rob Lathm.
  406. * Fix a warning for {snappy,lz4}-free compilation. Thanks to Andrew Schaaf.
  407. * Several improvements for CMakeLists.txt (cmake).
  408. * Fixing C99 compatibility warnings. Thanks to Christopher Speller.
  409. Changes from 1.4.0 to 1.4.1
  410. ===========================
  411. * Fixed a bug in blosc_getitem() introduced in 1.4.0. Added a test for
  412. blosc_getitem() as well.
  413. Changes from 1.3.6 to 1.4.0
  414. ===========================
  415. * Support for non-Intel and non-SSE2 architectures has been added. In
  416. particular, the Raspberry Pi platform (ARM) has been tested and all
  417. tests pass here.
  418. * Architectures requiring strict access alignment are supported as well.
  419. Due to this, arquitectures with a high penalty in accessing unaligned
  420. data (e.g. Raspberry Pi, ARMv6) can compress up to 2.5x faster.
  421. * LZ4 has been updated to r119 (1.2.0) so as to fix a possible security
  422. breach.
  423. Changes from 1.3.5 to 1.3.6
  424. ===========================
  425. * Updated to LZ4 r118 due to a (highly unlikely) security hole. For
  426. details see:
  427. http://fastcompression.blogspot.fr/2014/06/debunking-lz4-20-years-old-bug-myth.html
  428. Changes from 1.3.4 to 1.3.5
  429. ===========================
  430. * Removed a pointer from 'pointer from integer without a cast' compiler
  431. warning due to a bad macro definition.
  432. Changes from 1.3.3 to 1.3.4
  433. ===========================
  434. * Fixed a false buffer overrun condition. This bug made c-blosc to
  435. fail, even if the failure was not real.
  436. * Fixed the type of a buffer string.
  437. Changes from 1.3.2 to 1.3.3
  438. ===========================
  439. * Updated to LZ4 1.1.3 (improved speed for 32-bit platforms).
  440. * Added a new `blosc_cbuffer_complib()` for getting the compression
  441. library for a compressed buffer.
  442. Changes from 1.3.1 to 1.3.2
  443. ===========================
  444. * Fix for compiling Snappy sources against MSVC 2008. Thanks to Mark
  445. Wiebe!
  446. * Version for internal LZ4 and Snappy are now supported. When compiled
  447. against the external libraries, this info is not available because
  448. they do not support the symbols (yet).
  449. Changes from 1.3.0 to 1.3.1
  450. ===========================
  451. * Fixes for a series of issues with the filter for HDF5 and, in
  452. particular, a problem in the decompression buffer size that made it
  453. impossible to use the blosc_filter in combination with other ones
  454. (e.g. fletcher32). See
  455. https://github.com/PyTables/PyTables/issues/21.
  456. Thanks to Antonio Valentino for the fix!
  457. Changes from 1.2.4 to 1.3.0
  458. ===========================
  459. A nice handful of compressors have been added to Blosc:
  460. * LZ4 (http://code.google.com/p/lz4/): A very fast
  461. compressor/decompressor. Could be thought as a replacement of the
  462. original BloscLZ, but it can behave better is some scenarios.
  463. * LZ4HC (http://code.google.com/p/lz4/): This is a variation of LZ4
  464. that achieves much better compression ratio at the cost of being
  465. much slower for compressing. Decompression speed is unaffected (and
  466. sometimes better than when using LZ4 itself!), so this is very good
  467. for read-only datasets.
  468. * Snappy (http://code.google.com/p/snappy/): A very fast
  469. compressor/decompressor. Could be thought as a replacement of the
  470. original BloscLZ, but it can behave better is some scenarios.
  471. * Zlib (http://www.zlib.net/): This is a classic. It achieves very
  472. good compression ratios, at the cost of speed. However,
  473. decompression speed is still pretty good, so it is a good candidate
  474. for read-only datasets.
  475. With this, you can select the compression library with the new
  476. function::
  477. int blosc_set_complib(char* complib);
  478. where you pass the library that you want to use (currently "blosclz",
  479. "lz4", "lz4hc", "snappy" and "zlib", but the list can grow in the
  480. future).
  481. You can get more info about compressors support in you Blosc build by
  482. using these functions::
  483. char* blosc_list_compressors(void);
  484. int blosc_get_complib_info(char *compressor, char **complib, char **version);
  485. Changes from 1.2.2 to 1.2.3
  486. ===========================
  487. - Added a `blosc_init()` and `blosc_destroy()` so that the global lock
  488. can be initialized safely. These new functions will also allow other
  489. kind of initializations/destructions in the future.
  490. Existing applications using Blosc do not need to start using the new
  491. functions right away, as long as they calling `blosc_set_nthreads()`
  492. previous to anything else. However, using them is highly recommended.
  493. Thanks to Oscar Villellas for the init/destroy suggestion, it is a
  494. nice idea!
  495. Changes from 1.2.1 to 1.2.2
  496. ===========================
  497. - All important warnings removed for all tested platforms. This will
  498. allow less intrusiveness compilation experiences with applications
  499. including Blosc source code.
  500. - The `bench/bench.c` has been updated so that it can be compiled on
  501. Windows again.
  502. - The new web site has been set to: http://www.blosc.org
  503. Changes from 1.2 to 1.2.1
  504. =========================
  505. - Fixed a problem with global lock not being initialized. This
  506. affected mostly to Windows platforms. Thanks to Christoph
  507. Gohlke for finding the cure!
  508. Changes from 1.1.5 to 1.2
  509. =========================
  510. - Now it is possible to call Blosc simultaneously from a parent threaded
  511. application without problems. This has been solved by setting a
  512. global lock so that the different calling threads do not execute Blosc
  513. routines at the same time. Of course, real threading work is still
  514. available *inside* Blosc itself. Thanks to Thibault North.
  515. - Support for cmake is now included. Linux, Mac OSX and Windows
  516. platforms are supported. Thanks to Thibault North, Antonio Valentino
  517. and Mark Wiebe.
  518. - Fixed many compilers warnings (specially about unused variables).
  519. - As a consequence of the above, as minimal change in the API has been
  520. introduced. That is, the previous API::
  521. void blosc_free_resources(void)
  522. has changed to::
  523. int blosc_free_resources(void)
  524. Now, a return value of 0 means that the resources have been released
  525. successfully. If the return value is negative, then it is not
  526. guaranteed that all the resources have been freed.
  527. - Many typos were fixed and docs have been improved. The script for
  528. generating nice plots for the included benchmarks has been improved
  529. too. Thanks to Valetin Haenel.
  530. Changes from 1.1.4 to 1.1.5
  531. ===========================
  532. - Fix compile error with msvc compilers (Christoph Gohlke)
  533. Changes from 1.1.3 to 1.1.4
  534. ===========================
  535. - Redefinition of the BLOSC_MAX_BUFFERSIZE constant as (INT_MAX -
  536. BLOSC_MAX_OVERHEAD) instead of just INT_MAX. This prevents to produce
  537. outputs larger than INT_MAX, which is not supported.
  538. - `exit()` call has been replaced by a ``return -1`` in blosc_compress()
  539. when checking for buffer sizes. Now programs will not just exit when
  540. the buffer is too large, but return a negative code.
  541. - Improvements in explicit casts. Blosc compiles without warnings
  542. (with GCC) now.
  543. - Lots of improvements in docs, in particular a nice ascii-art diagram
  544. of the Blosc format (Valentin Haenel).
  545. - Improvements to the plot-speeds.py (Valentin Haenel).
  546. - [HDF5 filter] Adapted HDF5 filter to use HDF5 1.8 by default
  547. (Antonio Valentino).
  548. - [HDF5 filter] New version of H5Z_class_t definition (Antonio Valentino).
  549. Changes from 1.1.2 to 1.1.3
  550. ===========================
  551. - Much improved compression ratio when using large blocks (> 64 KB) and
  552. high compression levels (> 6) under some circumstances (special data
  553. distribution). Closes #7.
  554. Changes from 1.1.1 to 1.1.2
  555. ===========================
  556. - Fixes for small typesizes (#6 and #1 of python-blosc).
  557. Changes from 1.1 to 1.1.1
  558. =========================
  559. - Added code to avoid calling blosc_set_nthreads more than necessary.
  560. That will improve performance up to 3x or more, specially for small
  561. chunksizes (< 1 MB).
  562. Changes from 1.0 to 1.1
  563. =======================
  564. - Added code for emulating pthreads API on Windows. No need to link
  565. explicitly with pthreads lib on Windows anymore. However, performance
  566. is a somewhat worse because the new emulation layer does not support
  567. the `pthread_barrier_wait()` call natively. But the big improvement
  568. in installation easiness is worth this penalty (most specially on
  569. 64-bit Windows, where pthreads-win32 support is flaky).
  570. - New BLOSC_MAX_BUFFERSIZE, BLOSC_MAX_TYPESIZE and BLOSC_MAX_THREADS
  571. symbols are available in blosc.h. These can be useful for validating
  572. parameters in clients. Thanks to Robert Smallshire for suggesting
  573. that.
  574. - A new BLOSC_MIN_HEADER_LENGTH symbol in blosc.h tells how many bytes
  575. long is the minimum length of a Blosc header. `blosc_cbuffer_sizes()`
  576. only needs these bytes to be passed to work correctly.
  577. - Removed many warnings (related with potentially dangerous type-casting
  578. code) issued by MSVC 2008 in 64-bit mode.
  579. - Fixed a problem with the computation of the blocksize in the Blosc
  580. filter for HDF5.
  581. - Fixed a problem with large datatypes. See
  582. http://www.pytables.org/trac/ticket/288 for more info.
  583. - Now Blosc is able to work well even if you fork an existing process
  584. with a pool of threads. Bug discovered when PyTables runs in
  585. multiprocess environments. See http://pytables.org/trac/ticket/295
  586. for details.
  587. - Added a new `blosc_getitem()` call to allow the retrieval of items in
  588. sizes smaller than the complete buffer. That is useful for the carray
  589. project, but certainly for others too.
  590. Changes from 0.9.5 to 1.0
  591. =========================
  592. - Added a filter for HDF5 so that people can use Blosc outside PyTables,
  593. if they want to.
  594. - Many small improvements, specially in README files.
  595. - Do not assume that size_t is uint_32 for every platform.
  596. - Added more protection for large buffers or in allocation memory
  597. routines.
  598. - The src/ directory has been renamed to blosc/.
  599. - The `maxbytes` parameter in `blosc_compress()` has been renamed to
  600. `destsize`. This is for consistency with the `blosc_decompress()`
  601. parameters.
  602. Changes from 0.9.4 to 0.9.5
  603. ===========================
  604. - Now, compression level 0 is allowed, meaning not compression at all.
  605. The overhead of this mode will be always BLOSC_MAX_OVERHEAD (16)
  606. bytes. This mode actually represents using Blosc as a basic memory
  607. container.
  608. - Supported a new parameter `maxbytes` for ``blosc_compress()``. It
  609. represents a maximum of bytes for output. Tests unit added too.
  610. - Added 3 new functions for querying different metadata on compressed
  611. buffers. A test suite for testing the new API has been added too.
  612. Changes from 0.9.3 to 0.9.4
  613. ===========================
  614. - Support for cross-platform big/little endian compatibility in Blosc
  615. headers has been added.
  616. - Fixed several failures exposed by the extremesuite. The problem was a
  617. bad check for limits in the buffer size while compressing.
  618. - Added a new suite in bench.c called ``debugsuite`` that is
  619. appropriate for debugging purposes. Now, the ``extremesuite`` can be
  620. used for running the complete (and extremely long) suite.
  621. Changes from 0.9.0 to 0.9.3
  622. ===========================
  623. - Fixed several nasty bugs uncovered by the new suites in bench.c.
  624. Thanks to Tony Theodore and Gabriel Beckers for their (very)
  625. responsive beta testing and feedback.
  626. - Added several modes (suites), namely ``suite``, ``hardsuite`` and
  627. ``extremehardsuite`` in bench.c so as to allow different levels of
  628. testing.
  629. Changes from 0.8.0 to 0.9
  630. =========================
  631. - Internal format version bumped to 2 in order to allow an easy way to
  632. indicate that a buffer is being saved uncompressed. This is not
  633. supported yet, but it might be in the future.
  634. - Blosc can use threads now for leveraging the increasing number of
  635. multi-core processors out there. See README-threaded.txt for more
  636. info.
  637. - Added a protection for MacOSX so that it has to not link against
  638. posix_memalign() funtion, which seems not available in old versions of
  639. MacOSX (for example, Tiger). At nay rate, posix_memalign() is not
  640. necessary on Mac because 16 bytes alignment is ensured by default.
  641. Thanks to Ivan Vilata. Fixes #3.