Up to [cvs.NetBSD.org] / pkgsrc / databases / py-lmdb
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
py-lmdb: updated to 2.3.0
2.3.0
Bug fixes
- ``lmdb.aio``: an async write transaction is now pinned to a private
single-thread executor, so its ``begin``, operations, and
``commit``/``abort`` all run on one OS thread. Previously they were
dispatched across the shared, multi-threaded default executor, which could
release LMDB's thread-bound robust write mutex on the wrong thread (the
failure mode of 465). ``AsyncCursor``/``AsyncEnvironment`` close on
context exit now also acquire the lock and run on the executor, so closing
can no longer race an in-flight operation on the same transaction.
- Fix committing values larger than ~2 GiB. A single LMDB value is written
in one call, exceeding per-call platform write limits: on POSIX,
``write()`` returns a short count that was treated as fatal, so the
commit failed with ``mdb_txn_commit: Input/output error``; the bundled
liblmdb now resumes after short writes. On Windows, the compacting-copy
writer thread passed the full length to ``WriteFile``, whose 32-bit
length truncates; it now caps each write at 1 GiB.
bundled liblmdb — not applied when building with ``LMDB_PURE`` or a
system liblmdb)
- Fix silent data loss on Windows committing values near the 4 GiB maximum.
``mdb_page_flush`` wrote each overflow extent with a single ``WriteFile``
whose 32-bit length truncates — an extent of exactly 4 GiB wrote nothing
at all, committing a value of zeroes. Writes are now chunked with the
bytes-written count checked.
applied when building with ``LMDB_PURE`` or a system liblmdb)
- Fix 32-bit overflow in the bundled liblmdb's overflow-page byte-length
math. For values within one page of the 4 GiB maximum, the products
``num * me_psize`` (dirty-page spill) and ``me_psize * (mp_pages - 1)``
(compacting copy) wrapped on 64-bit builds, truncating the copied value
or crashing. Both multiplications are widened to ``size_t``.
patch to the bundled liblmdb — not applied when building with
``LMDB_PURE`` or a system liblmdb)
- Fix ``Environment.set_mapsize()`` invalidating open database handles.
Since 2.2.0
invalidated all named- and main-database handles, so a handle opened
before the resize raised ``lmdb.Error: Database handle belongs to
another environment`` (CPython) or an invalid-dbi ``TypeError`` (CFFI)
on next use. Database handles hold only an integer ``MDB_dbi`` that
stays valid across the remap, so they are now preserved; only open
transactions and cursors are invalidated.
- Fix races between ``Environment.set_mapsize()`` and concurrent operations
on other threads. An operation starting during the resize's remap window
could touch the old memory map after ``munmap`` (crash or corruption); a
write-transaction abort racing the resize could deadlock it or leave a
transaction alive at remap time (failing the resize and invalidating the
environment); and in the CFFI backend a write transaction beginning
during the resize could orphan its thread-id record, deadlocking a later
``close()``. LMDB operations that would race a resize now fail fast with
``lmdb.Error`` instead; retry them after the resize completes. The CFFI
backend now also serializes ``Transaction.get``/``put``/``delete``/
``stat``/``drop`` and the environment-level ``stat``/``info``/``sync``/
``copy``/``copyfd``/``readers``/``reader_check`` against
``close()``/``set_mapsize()``, closing long-standing crash races with
``close()`` as well. Operations racing a ``set_mapsize()`` on another
thread now raise a transient ``lmdb.Error`` (CPython) or the CFFI
invalid-handle ``TypeError`` instead of crashing or deadlocking.
Improvements
- The unit test suite is now type-checked in CI with both mypy and pyright,
and the shipped stubs are validated against the runtime module with
stubtest, so the public typed API stays usable regardless of which type
checker downstream users run.
- Add missing ``lmdb.aio.AsyncEnvironment`` method stubs.
- ``lmdb.version(subpatch=True)`` is now correctly typed via overloads,
returning a 4-tuple instead of a 3-tuple.
- The CFFI backend's internal reference-holding attributes ``Transaction.env``,
``Cursor.db``, and ``Cursor.txn`` are now private (``_pyenv``, ``_pydb``,
``_pytxn``). They were never exposed by the CPython extension, so this
aligns the two backends. These were undocumented implementation details.
py-lmdb: updated to 2.2.1
2.2.1
Improvements
- ``lmdb.aio`` async wrapper methods (on ``AsyncEnvironment``,
``AsyncTransaction``, and ``AsyncCursor``) are now real class
attributes rather than synthesized by ``__getattr__``, making them
visible to ``dir()``, ``inspect.signature()``, ``help()``, IPython,
and stubtest. Non-callable attributes still proxy through.
- Added type stubs for the ``lmdb.aio`` module.
Fixes
- Fix a segmentation fault on Linux/aarch64 when an ``Environment`` is
closed from a different thread than the one holding an active write
transaction. ``Environment.close()`` now waits for the owning thread
to release the write transaction before unmapping the lock file.
- Reject adversarial on-disk values that could overflow size arithmetic
or cause implementation-defined narrowing of ``mv_size`` when reading
crafted databases.
- Fix crashes on out-of-memory conditions in the CPython extension's
error paths.
- Fix and modernize the type stubs, including ``__all__`` exports and the
``Cursor.iter*`` parameter defaults in ``__init__.pyi``.
Other
- Remove remaining Python 2 compatibility code.
- Run the test suite natively on Linux/aarch64 in CI; previously aarch64
wheels were built but never tested.
py-lmdb: updated to 2.2.0
2.2.0
New features
- New ``lmdb.aio`` module providing asyncio support with
``AsyncEnvironment`` and ``AsyncTransaction``.
- ``Environment.dbs()`` lists all named databases in an environment.
- ``Cursor.getmulti()`` accepts ``values=False`` for batch key-existence
checks without fetching values.
- ``Transaction.stat()`` now defaults to the main database when no ``db``
argument is given.
- ``Environment.set_mapsize()`` now safely invalidates open transactions
and cursors before remapping, preventing use-after-free crashes.
- Enable sparse files on Windows so NTFS does not preallocate the full
``map_size`` on disk.
Bug fixes
- Fix type stubs: return types now use ``Union[bytes, memoryview]`` to
reflect the ``buffers=`` parameter, and ``dbs()`` is included.
- Declare ``python_requires='>=3.9'`` so pip skips this package on
Python 2.
Documentation
- Modernize documentation for Python 3 and current project state.
- Add API reference for ``lmdb.aio`` module.
- Document fork safety and multiprocessing guidance.
- Document duplicate-sort (dupsort) databases.
- Document cursor iteration and navigation patterns.
- Add "When to use py-lmdb" comparison section.
- Render ChangeLog as native HTML instead of a code block.
Other
- Add LMDB data layout validation tests and patched/pure equivalence
checks.
py-lmdb: updated to 1.7.5 1.7.5 * CI-only: Fix generation of 3.14 binaries. 1.7.4 -- yanked * CI-only: Generate Python 3.14 binaries. 1.7.3 * Fix CFFI build on some platforms by ensuring paths are absolute. * Correct CI badge URL in README. 1.7.2 * CI-only fix 1.7.1 * CI-only fix 1.7.0 * Rewrite CI to use cibuildwheel. * Update bundled LMDB to 0.9.33, plus a patch to fix ITS-10346. * Prevent some accidental use of LMDB objects by child processes.
Fix PLIST after py-setuptools update; bump depends and revision
py-lmdb: updated to 1.6.2 2025-01-05 1.6.2 * CI-only fix. 2025-01-05 1.6.1 * CI-only fix. 2025-01-05 1.6.0 * Support for Python 3.13.
py-*: remove unused tool dependency py-setuptools includes the py-wheel functionality nowadays
py-lmdb: updated to 1.5.1 1.5.1 * CI-only fix. 1.5.0 * Add Python 3.12 binaries. * Update bundled LMDB to 0.9.31. * Remove Python 2.7 support.
py-lmdb: seems to use patch during build From SunOS build: sh: /usr/bin/patch: not found [No such file or directory]
py-lmdb: updated to 1.4.1 v1.4.1 * Update CI to build manylinux binaries.
py-lmdb: updated to 1.4.0 2022-12-06 v1.4.0 * Add Python 3.11 support. 2021-12-30 v1.3.0 * Add aarch64 architecture builds. Contributed by odidev. * Add Python 3.10 support. * Fix crash relating to caching of transactions. The 'max_spare_txns' parameter to Environment/open is currently ignored.
*: bump PKGREVISION for egg.mk users They now have a tool dependency on py-setuptools instead of a DEPENDS
py-lmdb: updated to 1.2.1 2021-04-19 v1.2.1 * Resolve CI bug where non-Linux wheels were not being published to PyPI. 2021-04-15 v1.2.0 * Update bundled LMDB to 0.9.29. * Add non-bundled testing to CI. * Remove wheel generation for 2.7 because the manylinux images no longer support it. * Allow passing None as a value to transaction.del in CFFI implementation for parity with cpython implementation. * Fix Cursor.put behavior on a dupsort DB with append=True. * Add warning to docs about use of Environment.set_mapsize. This is currently an unresolved issue with upstream LMDB. * CFFI implementation: fix a seg fault when open_db returns map full. * CFFI implementation: fix a bug in open_db in a read-only environment. 2021-02-05 v1.1.1 * Dowgrade underlying LMDB to 0.9.26. 0.9.27 has a minor defect that will need to get resolved. 2021-02-04 v1.1.0 * Migrate CI pipeline from Travis and AppVeyor to Github Actions. Now includes comprehensive testing across 4 dimensions (OS, Python version, cpython/CFFI, pure/with mods). Also includes publishing to PyPI. * Prevent invalid flag combinations when creating a database. * Add a Cursor.getmulti method with optional buffer support. Contributed by Will Thompson <willsthompson@gmail.com>. * Upgrade underlying LMDB to 0.9.27. 2020-08-28 v1.0.0 * Start of new semantic versioning scheme. This would be a minor version bump from the 0.99 release if it were semantically versioned. * Allow environment copy to take a passed-in transaction. This is the first released feature that requires a (very small) patch to the underlying C library. By default, the patch will be applied unless this module is built with LMDB_PURE environment variable set. 2020-08-13 v0.99 * Fix lmdb.tool encoding issues. * Fix -l lmdb invocation issue. * Minor documentation improvements. * Update LMDB to version 0.9.24. * Update for Python 3.9 (current release candidate) support. * Resolve a bug when using cursor.putmulti and append=True on dupsort DBs. * Allow _Database.flags method to take no arguments since the one argument wasn't being used. 2019-11-06 v0.98 * Fix that a duplicate argument to a lmdb method would cause an assert. * Solaris needs ```#include "python.h"``` as soon as possible. Fix contributed by Jesús Cea. * Fix crash under debug cpython when mdb_cursor_open failed 2019-08-11 v0.97 * Fix a missed GIL unlock sequence. Reported by ajschorr. * Fix argv check in JEP (cpython under Java) environment. Contributed by de-code. 2019-07-14 v0.96 * First release under new maintainer, Nic Watson. * Doc updates. * More removal of code for now-unsupported Python versions. * Only preload the value with the GIL unlocked when the value is actually requested. This significantly improves read performance to retrieve keys with large values when the value isn't retrieved. Reported by Dan Patton. 2019-06-08 v0.95 * The minimum supported version of Python is now 2.7. * The library is no longer tested on Python 3.2. * The address-book.py example was updated for Python 3. Contributed by Jamie Bliss. * Development-related files were removed from the distribution tarball. * Handling of the Environment(create=True) flag was improved. Fix contributed by Nir Soffer. * Database names may be reused after they are dropped on CFFI, without reopening the environment. Fix contributed by Gareth Bult. 2018-04-09 v0.94 * CPython argument parsing now matches the behaviour of CFFI, and most sane Python APIs: a bool parameter is considered to be true if it is any truthy value, not just if it is exactly True. Reported by Nic Watson. * Removed Python 2.6 support due to urllib3 warnings and pytest dropping it. * Updared LMDB to version 0.9.22. * Fixed several 2.7/3 bugs in command line tool.
databases/py-lmdb: Import version 0.94 Provides Python binding for the LMDB "Lightning" Database.