RELEASE_NOTES.rst 26 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763
  1. =======================
  2. Release notes for bcolz
  3. =======================
  4. Changes from 1.1.0 to 1.1.1
  5. ===========================
  6. - Allow to delete all the columns in a ctable. Fixes #306.
  7. - Double-check the value of a column that is being overwritten. Fixes
  8. #307.
  9. - Now all the columns in a ctable are enforced to be a carray instance
  10. in order to simplify the internal logic for handling columns.
  11. - Now, the cparams are preserved during column replacement, e.g.:
  12. `ct['f0'] = x + 1`
  13. - C-Blosc updated to 1.9.3.
  14. - Added a new `defaults_ctx` context so that users can select defaults
  15. easily without changing global behaviour. For example::
  16. with bcolz.defaults_ctx(vm="python", cparams=bcolz.cparams(clevel=0)):
  17. cout = bcolz.eval("(x + 1) < 0")
  18. Changes from 1.0.0 to 1.1.0
  19. ===========================
  20. - Defaults when creating carray/ctable objects are always scalars now.
  21. The new approach follows what was documented and besides it prevents
  22. storing too much JSON data in meta/ directory.
  23. - Fixed an issue with bcolz.iterblocks() not working on multidimensional
  24. carrays.
  25. - It is possible now to create ctables with more than 255 columns. Thanks
  26. to Skipper Seabold. Fixes #131 (via PR #303).
  27. - Added a new `quantize` filter for allowing lossy compression on
  28. floating point data. Data is quantized using
  29. np.around(scale*data)/scale, where scale is 2**bits, and bits is
  30. determined from the quantize value. For example, if quantize=1, bits
  31. will be 4. 0 means that the quantization is disabled.
  32. Here it is an example of what you can get from the new quantize::
  33. In [9]: a = np.cumsum(np.random.random_sample(1000*1000)-0.5)
  34. In [10]: bcolz.carray(a, cparams=bcolz.cparams(quantize=0)) # no quantize
  35. Out[10]:
  36. carray((1000000,), float64)
  37. nbytes: 7.63 MB; cbytes: 6.05 MB; ratio: 1.26
  38. cparams := cparams(clevel=5, shuffle=1, cname='blosclz', quantize=0)
  39. [ -2.80946077e-01 -7.63925274e-01 -5.65575047e-01 ..., 3.59036158e+02
  40. 3.58546624e+02 3.58258860e+02]
  41. In [11]: bcolz.carray(a, cparams=bcolz.cparams(quantize=1))
  42. Out[11]:
  43. carray((1000000,), float64)
  44. nbytes: 7.63 MB; cbytes: 1.41 MB; ratio: 5.40
  45. cparams := cparams(clevel=5, shuffle=1, cname='blosclz', quantize=1)
  46. [ -2.50000000e-01 -7.50000000e-01 -5.62500000e-01 ..., 3.59036158e+02
  47. 3.58546624e+02 3.58258860e+02]
  48. In [12]: bcolz.carray(a, cparams=bcolz.cparams(quantize=2))
  49. Out[12]:
  50. carray((1000000,), float64)
  51. nbytes: 7.63 MB; cbytes: 2.20 MB; ratio: 3.47
  52. cparams := cparams(clevel=5, shuffle=1, cname='blosclz', quantize=2)
  53. [ -2.81250000e-01 -7.65625000e-01 -5.62500000e-01 ..., 3.59036158e+02
  54. 3.58546624e+02 3.58258860e+02]
  55. In [13]: bcolz.carray(a, cparams=bcolz.cparams(quantize=3))
  56. Out[13]:
  57. carray((1000000,), float64)
  58. nbytes: 7.63 MB; cbytes: 2.30 MB; ratio: 3.31
  59. cparams := cparams(clevel=5, shuffle=1, cname='blosclz', quantize=3)
  60. [ -2.81250000e-01 -7.63671875e-01 -5.65429688e-01 ..., 3.59036158e+02
  61. 3.58546624e+02 3.58258860e+02]
  62. As you can see, the compression ratio can improve pretty significantly
  63. when using the quantize filter. It is important to note that by using
  64. quantize you are loosing precision on your floating point data.
  65. Also note how the first elements in the quantized arrays have less
  66. significant digits, but not the last ones. This is a side effect due
  67. to how bcolz stores the trainling data that do not fit in a whole
  68. chunk. But in general you should expect a loss in precision.
  69. - Fixed a bug in carray.__getitem__() when the chunksize was not an
  70. exact multiple of the blocksize. Added test:
  71. test_carray.py::getitemMemoryTest::test06.
  72. - bcolz now follows the convention introduced in NumPy 1.11 for
  73. representing datetime types with TZ="naive" (i.e. with no TZ info in
  74. the representation). See https://github.com/numpy/numpy/blob/master/doc/release/1.11.0-notes.rst#datetime64-changes.
  75. - bcolz now releases the GIL during Blosc compression/decompression. In
  76. multi-threaded environments, a single-threaded, contextual version of
  77. Blosc is used instead (this is useful for frameworks like Dask).
  78. - Removed from the ``cbytes`` count the storage overhead due to the
  79. internal container. This overhead was media-dependent, and it was
  80. just a guess anyway.
  81. - The -O1 compilation flag has been removed and bcolz is compiled now at
  82. full optimization. I have tested that for several weeks, without any
  83. segfault, so this should be pretty safe.
  84. - Added information about the chunklen, chunksize and blocksize (the
  85. size of the internal blocks in a Blosc chunk) in the repr() of a
  86. carray.
  87. - New accelerated codepath for `carray[:] = array` assignation. This
  88. operation should be close in performance to `carray.copy()` now.
  89. - carray object does implement the __array__() special method
  90. (http://docs.scipy.org/doc/numpy-1.10.1/reference/arrays.classes.html#numpy.class.__array__)
  91. now. With this, interoperability with numpy arrays is easier and
  92. faster:
  93. Before __array__()::
  94. >>> a = np.arange(1e7)
  95. >>> b = np.arange(1e7)
  96. >>> ca = bcolz.carray(a)
  97. >>> cb = bcolz.carray(b)
  98. >>> %timeit ca + a
  99. 1 loop, best of 3: 1.06 s per loop
  100. >>> %timeit np.array(bcolz.eval("ca*(cb+1)"))
  101. 1 loop, best of 3: 1.18 s per loop
  102. After __array__()::
  103. >>> %timeit ca + a
  104. 10 loops, best of 3: 45.2 ms per loop
  105. >>> %timeit np.array(bcolz.eval("ca*(cb+1)"))
  106. 1 loop, best of 3: 133 ms per loop
  107. And it also allows to use bcolz carrays more efficiently in some scenarios::
  108. >>> import numexpr
  109. >>> %timeit numexpr.evaluate("ca*(cb+1)")
  110. 10 loops, best of 3: 76.2 ms per loop
  111. >>> %timeit numexpr.evaluate("a*(b+1)")
  112. 10 loops, best of 3: 25.5 ms per loop # ndarrays are still faster
  113. - Internal C-Blosc sources bumped to 1.9.2.
  114. - Dask (dask.pydata.org) is supported as another virtual machine backed
  115. for bcolz.eval(). Now, either Numexpr (the default) or Dask or even
  116. the Python interpreter can be used to evaluate complex expressions.
  117. - The default compressor has been changed from 'blosclz' to 'lz4'.
  118. BloscLZ tends to be a bit faster when decompressing, but LZ4 is
  119. quickly catching up as the compilers are making progress with memory
  120. access optimizations. Also, LZ4 is considerably faster during
  121. compression and in general compresses better too.
  122. - The supported SIMD extensions (SSE2 and AVX2) of the current platform
  123. are auto-detected so that the affected code will selectively be
  124. included from vendored C-Blosc sources.
  125. - Added a new `blen` parameter to bcolz.eval() so that the user can
  126. select the length of the operand blocks to be operated with.
  127. - New fine-tuning of the automatically computed blen in bcolz.eval() for
  128. better times and reduced memory consumption.
  129. - Added a new `out_flavor` parameter to the ctable.iter() and
  130. ctable.where() for specifying the type of result rows. Now one can
  131. select namedtuple (default), tuple or ndarray.
  132. - The performance of carray.whereblocks() has been accelerated 2x due to
  133. the internal use of tuples instead of named tuples.
  134. - New ctable.fetchwhere() method for getting the rows fulfilling some
  135. condition in one go.
  136. - Parameter `outfields` in ctable.whereblocks has been renamed to
  137. `outcols` for consistency with the other methods. The previous
  138. 'outfields' name is considered a bug and hence is not supported
  139. anymore.
  140. - bcolz.fromiter() has been streamlined and optimized. The result is
  141. that it uses less memory and can go faster too (20% ~ 50%, depending
  142. on the use).
  143. - The values for defaults.eval_out_flavor has been changed to ['bcolz',
  144. 'numpy'] instead of previous ['carray', 'numpy']. For backward
  145. compatibility the 'carray' value is still allowed.
  146. - The `bcolz.defaults.eval_out_flavor` and `bcolz.defaults.eval_vm` have
  147. been renamed to `bcolz.defaults.out_flavor` and `bcolz.defaults.vm`
  148. because they can be used in other places than just bcolz.eval(). The
  149. old `eval_out_flavor` and `eval_vm` properties of the `defaults`
  150. object are still kept for backward compatibility, but they are not
  151. documented anymore and its use is discouraged.
  152. - Added a new `user_dict` parameter in all ctable methods that evaluate
  153. expressions. For convenience, this dictionary is updated internally
  154. with ctable columns, locals and globals from the caller.
  155. - Small optimization for using the recently added re_evaluate() function
  156. in numexpr for faster operation of numexpr inside loops using the same
  157. expression (quite common scenario).
  158. - Unicode strings are recognized now when imported from a pandas
  159. dataframe, making the storage much more efficient. Before unicode was
  160. converted into 'O'bject type, but the change to 'U'nicode should be
  161. backward compatible.
  162. - Added a new `vm` parameter to specify the virtual machine for doing
  163. internal operations in ctable.where(), ctable.fetchwhere() and
  164. ctable.whereblocks().
  165. Changes from 0.12.1 to 1.0.0
  166. ============================
  167. - New version of embedded C-Blosc (bumped to 1.8.1). This allows for
  168. using recent C-Blosc features like the BITSHUFFLE filter that
  169. generally allows for better compression ratios at the expense of some
  170. slowdown. Look into the carray tutorial on how to use the new
  171. BITSHUFFLE filter.
  172. - Use the -O1 flag for compiling the included C-Blosc sources on Linux.
  173. This represents slower performance, but fixes nasty segfaults as can
  174. be seen in issue #110 of python-blosc. Also, it prints a warning for
  175. using an external C-Blosc library.
  176. - Improved support for operations with carrays of shape (N, 1). PR #296.
  177. Fixes #165 and #295. Thanks to Kevin Murray.
  178. - Check that column exists before inserting a new one in a ctable via
  179. `__setitem__`. If it exists, the existing column is overwritten.
  180. Fixes #291.
  181. - Some optimisations have been made within ``carray.__getitem__`` to
  182. improve performance when extracting a slice of data from a
  183. carray. This is particularly relevant when running some computation
  184. chunk-by-chunk over a large carray. (#283 @alimanfoo).
  185. Changes from 0.12.0 to 0.12.1
  186. =============================
  187. - ``setup.py`` now defers operations requiring ``numpy`` and ``Cython``
  188. until after those modules have been installed by ``setuptools``. This
  189. means that users no longer need to pre-install ``numpy`` and
  190. ``Cython`` to install ``bcolz``.
  191. Changes from 0.11.4 to 0.12.0
  192. =============================
  193. - Fixes an installation glitch for Windows. (#268 @cgohlke).
  194. - The tutorial is now a Jupyter notebook. (#261 @FrancescElies).
  195. - Replaces numpy float string specifier in test with numpy.longdouble
  196. (#271 @msarahan).
  197. - Fix for allowing the use of variables of type string in `eval()` and
  198. other queries. (#273, @FrancescAlted).
  199. - The size of the tables during import/export to HDF5 are honored now
  200. via the `expectedlen` (bcolz) and `expectedrows` (PyTables)
  201. parameters (@FrancescAlted).
  202. - Update only the valid part of the last chunk during boolean
  203. assignments. Fixes a VisibleDeprecationWarning with NumPy 1.10
  204. (@FrancescAlted).
  205. - More consistent string-type checking to allow use of unicode strings
  206. in Python 2 for queries, column selection, etc. (#274 @BrenBarn).
  207. - Installation no longer fails when listed as dependency of project
  208. installed via setup.py develop or setup.py install. (#280 @mindw,
  209. fixes #277).
  210. - Paver setup has been deprecated (see #275).
  211. Changes from 0.11.3 to 0.11.4
  212. =============================
  213. - The .pyx extension is not packed using the absolute path anymore.
  214. (#266 @FrancescAlted)
  215. Changes from 0.11.2 to 0.11.3
  216. =============================
  217. - Implement feature #255 bcolz.zeros can create new ctables too, either
  218. empty or filled with zeros. (#256 @FrancescElies @FrancescAlted)
  219. Changes from 0.11.1 to 0.11.2
  220. =============================
  221. - Changed the `setuptools>18.3` dependency to `setuptools>18.0` because
  222. Anaconda does not have `setuptools > 18.1` yet.
  223. Changes from 0.11.0 to 0.11.1
  224. =============================
  225. - Do not try to flush when a ctable is opened in 'r'ead-only mode.
  226. See issue #252.
  227. - Added the mock dependency for Python2.
  228. - Added a `setuptools>18.3` dependency.
  229. - Several fixes in the tutorial (Francesc Elies).
  230. Changes from 0.10.0 to 0.11.0
  231. =============================
  232. - Added support for appending a np.void to ctable objects
  233. (closes ticket #229 @eumiro)
  234. - Do not try to flush when an carray is opened in 'r'ead-only mode.
  235. (closes #241 @FrancescAlted).
  236. - Fix appending of object arrays to already existing carrays
  237. (closes #243 @cpcloud)
  238. - Great modernization of setup.py by using new versioning and many
  239. other improvements (PR #239 @mindw).
  240. Changes from 0.9.0 to 0.10.0
  241. ============================
  242. - Fix pickle for in-memory carrays. (#193 #194 @dataisle @esc)
  243. - Implement chunks iterator, which allows the following syntax
  244. ``for chunk_ in ca._chunks``, added "internal use" indicator to carray
  245. chunks attribute. (#153 @FrancescElies and @esc)
  246. - Fix a memory leak and avoid copy in ``chunk.getudata``. (#201 #202 @esc)
  247. - Fix the error message when trying to open a fresh ctable in an existing
  248. rootdir. (#191 @twiecki @esc)
  249. - Solve #22 and be more specific about ``carray`` private methods.
  250. (#209 @FrancescElies @FrancescAlted)
  251. - Implement context manager for ``carray`` and ``ctable``.
  252. (#135 #210 @FrancescElies and @esc)
  253. - Fix handling and API for leftovers. (#72 #132 #211 #213 @FrancescElies @esc)
  254. - Fix bug for incorrect leftover value. (#208 @waylonflinn)
  255. - Documentation: document how to write extensions, update docstrings and
  256. mention the with statement / context manager. (#214 @FrancescElies)
  257. - Various refactorings and cleanups. (#190 #198 #197 #199 #200)
  258. - Fix bug creating carrays from transposed arrays without explicit dtype.
  259. (#217 #218 @sdvillal)
  260. Changes from 0.8.1 to 0.9.0
  261. ===========================
  262. - Implement ``purge``, which removes data for on-disk carrays. (#130 @esc)
  263. - Implement ``addcol/delcol`` to properly handle on-disk ctable (#112/#151 @cpcloud @esc)
  264. - Adding io-mode to the ``repr`` for carrays. (#124 @esc)
  265. - Implement ``auto_flush`` which allows ctables to flush themselves during
  266. operations that modify (write) data.
  267. (#140 #152 @FrancescElies @CarstVaartjes @esc)
  268. - Implement ``move`` for ctable, which allows disk-based carray to be moved
  269. (``mv``) into the root directory of the ctable.
  270. (#140 #152 #170 @FrancescElies @CarstVaartjes @esc)
  271. - Distribute ``carray_ext.pxd`` as part of the package. (#159 @ARF)
  272. - Add ``safe=`` keyword argument to control dtype/stride checking on append
  273. (#163 @mrocklin)
  274. - Hold GIL during c-blosc compression/decompression, avoiding some segfaults
  275. (#166 @mrocklin)
  276. - Fix ``dtype`` for multidimensional columns in a ctable (#136 #172 @alimanfoo)
  277. - Fix to allow adding strings > len 1 to ctable (#178 @brentp)
  278. - Sphinx based API documentation is now built from the docstrings in the Python
  279. sourcecode (#171 @esc)
  280. Changes from 0.8.0 to 0.8.1
  281. ===========================
  282. - Downgrade to Blosc v1.4.1 (#144 @esc)
  283. - Fix license include (#143 @esc)
  284. - Upgrade to Cython 0.22 (#145 @esc)
  285. Changes from 0.7.3 to 0.8.0
  286. ===========================
  287. - Public API for ``carray`` (#98 @FrancescElies and #esc)
  288. A Cython definition file ``carrat_ext.pxd`` was added that contains the
  289. definitions for the ``carray``, ``chunks`` and ``chunk`` classes. This was
  290. done to allow more complex programs to be built on the compressed container
  291. primitives provided by bcolz.
  292. - Overhaul the release procedure
  293. - Other miscellaneous fixes and improvements
  294. Changes from 0.7.2 to 0.7.3
  295. ===========================
  296. - Update to Blosc ``v1.5.2``
  297. - Added support for pickling persistent carray/ctable objects. Basically,
  298. what is serialized is the ``rootdir`` so the data is still sitting on disk
  299. and the original contents in ``rootdir`` are still needed for unpickling.
  300. (#79 @mrocklin)
  301. - Fixed repr-ing of ``datetime64`` ``carray`` objects (#99 @cpcloud)
  302. - Fixed Unicode handling for column addressing (#91 @CarstVaartjes)
  303. - Conda recipe and ``.binstar.yml`` (#88 @mrocklin and @cpcloud)
  304. - Removed ``unittest2`` as a run-time dependency (#90 @mrocklin)
  305. - Various typo fixes. (#75 @talumbau, #86 @catawbasam and #83 @bgrant)
  306. - Other miscellaneous fixes and improvements
  307. Changes from 0.7.1 to 0.7.2
  308. ===========================
  309. - Fix various test that were failing on 32 bit, especially a segfault
  310. - Fix compatibility with Numpy 1.9
  311. - Fix compatibility with Cython 0.21.
  312. - Allow tests to be executed with ``nosetests``.
  313. - Include git hash in version info when applicable.
  314. - Initial support for testing on Travis CI.
  315. - Close file handle when ``nodepath`` arg to ``ctable.fromhdf5`` is incorrect.
  316. - Introduced a new ``carray.view()`` method returning a light-weight
  317. carray object describing the same data than the original carray. This
  318. is mostly useful for iterators, but other uses could be devised as
  319. well.
  320. - Each iterator now return a view (see above) of the original object, so
  321. things like::
  322. >>> bc = bcolz.ctable([[1, 2, 3], [10, 20, 30]], names=['a', 'b'])
  323. >>> bc.where('a >= 2') # call .where but don't do anything with it
  324. <itertools.imap at 0x7fd7a84f5750>
  325. >>> list(bc['b']) # later iterate over table, get where result
  326. [10, 20, 30]
  327. works as expected now.
  328. - Added a workaround for dealing with Unicode types.
  329. - Fix writing absolute paths into the persistent metadata.
  330. - ``next(carray)`` calls now work as they should.
  331. - Fix the ``__repr__`` method of the ``chunk`` class.
  332. - Prevent sometimes incorrect assignment of dtype to name with fromhdf5.
  333. - Various miscellaneous bug-fixes, pep8 improvements and typo-fixes.
  334. Changes from 0.7.0 to 0.7.1
  335. ===========================
  336. - Return the outcome of the test for checking that in standalone
  337. programs. Thanks to Ilan Schnell for suggesting that.
  338. - Avoiding importing lists of ints as this has roundtrip problems in
  339. 32-bit platforms.
  340. - Got rid of the nose dependency for Python 2.6. Thanks to Ilan Schnell
  341. for the suggestion.
  342. Changes from 0.5.1 to 0.7.0
  343. ===========================
  344. - Renamed the ``carray`` package to ``bcolz``.
  345. - Added support for Python 3.
  346. - Added a new function `iterblocks` for quickly returning blocks of
  347. data, not just single elements. ctable receives a new `whereblocks`
  348. method, which is the equivalent of `where` but returning data blocks.
  349. - New pandas import/export functionality via `ctable.fromdataframe()`
  350. and `ctable.todataframe()`.
  351. - New HDF5/PyTables import/export functionality via `ctable.fromhdf5()`
  352. and `ctable.tohdf5()`.
  353. - Support for c-blosc 1.4.1. This allows the use of different
  354. compressors via the new `cname` parameter in the `cparams` class, and
  355. also to be used in platforms not supporting unaligned access.
  356. - Objects are supported in carray containers (not yet for ctable).
  357. - Added a new `free_cachemem()` method for freeing internal caches after
  358. reading/querying carray/ctable objects.
  359. - New `cparams.setdefaults()` method for globally setting defaults in
  360. compression parameters during carray/ctable creation.
  361. - Disabled multi-threading in both Blosc and numexpr because it is not
  362. delivering the promised speedups yet. This can always be re-activated
  363. by using `blosc_set_nthreads(nthreads)` and
  364. `numexpr.set_num_threads(nthreads)`.
  365. Changes from 0.5 to 0.5.1
  366. =========================
  367. - Added the missing bcolz.tests module in setup.py.
  368. Changes from 0.4 to 0.5
  369. =======================
  370. - Introduced support for persistent objects. Now, every carray and
  371. ctable constructor support a new `rootdir` parameter where you can
  372. specify the path where you want to make the data stored.
  373. The format chosen is explained in the 'persistence.rst' file, except
  374. that the blockpack format is still version 1 (that will probably
  375. change in future versions). Also, JSON is used for storing metadata
  376. instead of YAML. This is mainly for avoiding a new library
  377. dependency.
  378. - New `open(rootdir, mode='a')` top level function so as to open on-disk
  379. bcolz objects.
  380. - New `flush()` method for `carray` and `ctable` objects. This is
  381. useful for flushing data to disk in persistent objects.
  382. - New `walk(dir, classname=None, mode='a')` top level function for
  383. listing carray/ctable objects handing from `dir`.
  384. - New `attrs` accessor is provided, so that users can store
  385. its own metadata (in a persistent way, if desired).
  386. - Representation of carray/ctable objects is based now on the same code
  387. than NumPy.
  388. - Reductions (`sum` and `prod`) work now, even with the `axis` parameter
  389. (when using the Numexpr virtual machine).
  390. Changes from 0.3.2 to 0.4
  391. =========================
  392. - Implemented a `skip` parameter for iterators in `carray` and `ctable`
  393. objects. This complements `limit` for selecting the number of
  394. elements to be returned by the iterator.
  395. - Implemented multidimensional indexing for carrays. Than means that
  396. you can do::
  397. >>> a = ca.zeros((2,3))
  398. Now, you can access any element in any dimension::
  399. >>> a[1]
  400. array([ 0., 0., 0.])
  401. >>> a[1,::2]
  402. array([ 0., 0.])
  403. >>> a[1,1]
  404. 0.0
  405. - `dtype` and `shape` attributes follow now ndarray (NumPy) convention.
  406. The `dtype` is always a scalar and the dimensionality is added to the
  407. `shape` attribute. Before, all the additional dimensionality was in
  408. the `dtype`. The new convention should be more familiar for
  409. everybody.
  410. Changes from 0.3.1 to 0.3.2
  411. ===========================
  412. - New `vm` parameter for `eval()` that allows to choose a 'python' or
  413. 'numexpr' virtual machine during operations. If numexpr is not
  414. detected, the default will be 'python'.
  415. That means that you can use any function available in Python for
  416. evaluating bcolz expressions and that numexpr is not necessary
  417. anymore for using `eval()`.
  418. - New `out_flavor` parameter for `eval()` that allows to choose the
  419. output type. It can be 'bcolz' or 'numpy'.
  420. - New `defaults.py` module that enables the user to modify the defaults
  421. for internal bcolz operation. Defaults that are currently
  422. implemented: `eval_out_flavor` and `eval_vm`.
  423. - Fixed a bug with `carray.sum()` for multidimensional types.
  424. Changes from 0.3 to 0.3.1
  425. =========================
  426. - Added a `limit` parameter to `iter`, `where` and `wheretrue` iterators
  427. of carray object and to `iter` and `where` of ctable object.
  428. - Full support for multidimensional carrays. All types are supported,
  429. except the 'object' type (that applies to unidimensional carrays too).
  430. - Added a new `reshape()` for reshaping to new (multidimensional)
  431. carrays. This supports the same functionality than `reshape()` in
  432. NumPy.
  433. - The behaviour of a carray was altered after using an iterator. This
  434. has been fixed. Thanks to Han Genuit for reporting.
  435. Changes from 0.2 to 0.3
  436. =======================
  437. - Added a new `ctable` class that implements a compressed, column-wise
  438. table.
  439. - New `arange()` constructor for quickly building carray objects (this
  440. method is much faster than using `fromiter()`).
  441. - New `zeros()` constructor for quickly building zeroed carray objects.
  442. This is way faster than its NumPy counterpart.
  443. - New `ones()` constructor for quickly building 1's carray objects.
  444. Very fast.
  445. - New `fill()` constructor for quickly building carray objects with a
  446. filling value. This is very fast too.
  447. - New `trim()` method for `carray` and `ctable` objects for trimming
  448. items.
  449. - New `resize()` method for `carray` and `ctable` objects for resizing
  450. lengths.
  451. - New `test()` function that runs the complete test suite.
  452. - Added a new `eval()` function to evaluate expressions including any
  453. combination of carrays, ndarrays, sequences or scalars. Requires
  454. Numexpr being installed.
  455. - Added new `__len__()` and `__sizeof__()` special methods for both
  456. `carray` and `ctable` objects.
  457. - New `sum()` method for `carray` that computes the sum of the array
  458. elements.
  459. - Added new `nbytes` and `cbytes` properties for `carray` and `ctable`
  460. objects. The former accounts for the size of the original
  461. (non-compressed) object, and the later for the actual compressed
  462. object.
  463. - New algorithm for computing an optimal chunk size for carrays based on
  464. the new `expectedlen` argument.
  465. - Added `chunklen` property for `carray` that allows querying the chunk
  466. length (in rows) for the internal I/O buffer.
  467. - Added a new `append(rows)` method to `ctable` class.
  468. - Added a new `wheretrue()` iterator for `carray` that returns the
  469. indices for true values (only valid for boolean arrays).
  470. - Added a new `where(boolarr)` iterator for `carray` that returns the
  471. values where `boolarr` is true.
  472. - New idiom ``carray[boolarr]`` that returns the values where `boolarr`
  473. is true.
  474. - New idiom ``ctable[boolarr]`` that returns the rows where `boolarr` is
  475. true.
  476. - Added a new `eval()` method for `ctable` that is able to evaluate
  477. expressions with columns. It needs numexpr to be installed.
  478. - New idiom ``ctable[boolexpr]`` that returns the rows fulfilling the
  479. boolean expression. Needs numexpr.
  480. - Added fancy indexing (as a list of integers) support to `carray` and
  481. `ctable`.
  482. - Added `copy(clevel, shuffle)` method to both `carray` and `ctable`
  483. objects.
  484. - Removed the `toarray()` method in `carray` as this was equivalent to
  485. ``carray[:]`` idiom.
  486. - Renamed `setBloscMaxThreads()` to `blosc_set_num_threads()` and
  487. `whichLibVersion()` to `blosc_version()` to follow bcolz name
  488. conventions more closely.
  489. - Added a new `set_num_threads()` to set the number of threads in both
  490. Blosc and Numexpr (if available).
  491. - New `fromiter()` constructor for creating `carray` objects from
  492. iterators. It follows the NumPy API convention.
  493. - New `cparams(clevel=5, shuffle=True)` class to host all params related
  494. with compression.
  495. - Added more indexing support for `carray.__getitem__()`. All indexing
  496. modes present in NumPy are supported now, including fancy indexing.
  497. The only exception are negative steps in ``carray[start:stop:-step]``.
  498. - Added support for `bcolz.__setitem__()`. All indexing modes present
  499. in NumPy are supported, including fancy indexing. The only exception
  500. are negative steps in ``carray[start:stop:-step] = values``.
  501. - Added support for `ctable.__setitem__()`. All indexing modes present
  502. in NumPy are supported, including fancy indexing. The only exception
  503. are negative steps in ``ctable[start:stop:-step] = values``.
  504. - Added new `ctable.__iter__()`, `ctable.iter()` and `ctable.where()`
  505. iterators mimicking the functionality in carray object.
  506. Changes from 0.1 to 0.2
  507. =======================
  508. - Added a couple of iterators for carray: `__iter__()` and `iter(start,
  509. stop, step)`. The difference is that the later does accept slices.
  510. - Added a `__len__()` method.
  511. .. Local Variables:
  512. .. mode: rst
  513. .. coding: utf-8
  514. .. fill-column: 72
  515. .. End: