Up to [cvs.NetBSD.org] / pkgsrc / sysutils / py-fsspec
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
py-fsspec: updated to 2026.9.0 2026.9.0 Enhancements ia:// filesystem for files in Internet Archive items: an HTTPFileSystem reading from archive.org/download with credentials from ia configure's ia.ini Allow instance-local memory stores with global_store=False while preserving normal instance caching and the default shared store Add an adaptive readahead cache Implement created and modified for SFTPFileSystem Allow filesystem implementations to assign protocol per instance Delegate sizes and cat_ranges from DirFileSystem to the wrapped filesystem Delegate invalidate_cache, ukey and checksum from DirFileSystem to the wrapped filesystem Support negative offsets in TarFileSystem.cat_file Mirror readinto from the PyArrow stream onto ArrowFile Allow simplecache files to be removed by age using their modification times HTTP: retry transient failures (408/425/429/5xx, dropped connections, timeouts, truncated range bodies, a 416 inside the file) of cat_file, HTTPFile block reads and the size lookup behind open, controlled by the new retries, retry_wait, retry_statuses and retry_exceptions options Fixes Compute HadoopFileSystem.fsid from its configured host and port instead of accessing attributes no longer exposed by PyArrow Normalize dot path components in TarFileSystem listings and lookups, while preserving original archive names for reading Make merge_offset_ranges O(n log n) and keep merged blocks within max_block, replacing the quadratic nested-range filter added Fix transactions with compressed files Make MemoryFileSystem.rm raise FileNotFoundError for a path that does not exist, as rm_file and the other filesystems do, so deleting a missing key from a memory-backed mapper raises KeyError Create missing ZIP archives in append mode without truncating existing archives Close cached readers when exiting an open_files context Avoid mutating live BlockCache and BackgroundBlockCache instances when pickling End the transaction even when a commit or discard raises, so the filesystem is not left in transaction mode and deferred temporary files are cleaned up Honor a caller-owned output file in get_file Preserve existing files when checking FSMap write access Normalize ZIP paths in pipe_file and find Reject get destinations that resolve above the target Pass maxdepth through in AsyncFileSystem._rm Do not let isfile swallow BaseException Keep reference filesystem get destinations inside the target Return relative names from DirFileSystem.glob and find Raise FileNotFoundError for missing ZIP and TAR members Stop collapsing three-character drive-relative paths to the drive root Use / as the only glob separator on every platform Distinguish omitted and None defaults in FSMap.pop Forward an Arrow file's flush and closed state to its stream Keep reference filesystem cat_file offsets inside the referenced part Raise FileNotFoundError from archive ls for a missing path Download uncached files in simplecache.cat_ranges Fix tail on simplecache and filecache Get whole-file reference sizes asynchronously in ReferenceFileSystem._info Report the target size for links in TarFileSystem Keep the async generator returned by AsyncFileSystemWrapper._walk Join the default output filename with / in _expand_paths Make names relative in DirFileSystem.walk(detail=True) Ignore a leading slash in TarFileSystem paths Fix cat_ranges on whole-file caches Normalize paths in AsyncFileSystemWrapper like the wrapped filesystem The SFTP host key policy is configurable with the new host_key_policy argument of SFTPFileSystem, which accepts "auto_add" (default, unchanged behaviour), "warning" or "reject", or any paramiko.MissingHostKeyPolicy
py-fsspec: updated to 2025.10.0 2025.10.0 Enhancements - specify storage_options not use use for tokenising; test instance cache - base class "chained" for FSs that pass through operations Fixes - strip protocol for sftp - use one-step mv in jupyterfs - raise errors in jupyterfs, not return - add protocol attribute to http - allow ls() on a single file for arrowFS - support kwargs in webhdfs ls() - better discrimination for parquet in referenceFS - accept all mode= types in memoryFS - pass path list to put() in localtempfile for efficiency - fix docs errors Other - add CoC - restrict github/gist tests because of rate limiting
py-fsspec: updated to 2025.5.1 2025.5.1 Enhancements - file system for GitHub gists Other - doc fixes
py-fsspec: updated to 2025.3.0 2025.3.0 Enhancements - add pipe_file to HTTP - add sync http for pyodide - ls performance for local and detail=False Fixes - dir/info consistency in dirfs - referenceFS async consistency - CI
py-fsspec: updated to 2024.12.0 2024.12.0 Enhancements - "exclusive" mode for writing - "tree" text display of filesystem contents - async wrapper for sync FSs - new known implementation: tosfs - consilidate block fetch requests Fixes - better webHDFS proxies - syn FSs in referenceFS - don't serialize file caches - race condition in local ls() - missing/nan references in parquet - _un_chain kwargs (@1736) - async _cat_file in referenceFS Other - fallback implementation for _fetch_range
py-fsspec: updated to 2024.6.0 2024.6.0 Enhancements - allow dicts (not just bytes) for referenceFS - make filesystems JSON serializeable - implement multifile cat() for github Fixes - implement auto_mkdir for SMB Other - add doc deps to pyproject - re-remove test from package - formatting - change monthly download badge
py-fsspec: updated to 2024.5.0 2024.5.0 Enhancements - log hits/misses in bytes cachers Fixes - SMB flaky tests - rsync: only delete files if there are some to delete - don't let files equal bytes objects - url_to_fs to stringify paths - assert types in MemoryFS - restore _strip_protocol signature for local - convert list to set when loading cache metadata Other - remove mv_file - mv() should not swallow errors - change versioning template, allows easier co-install of dev s3fs - in ls_from_cache, avoid dounble lookup - expand=True in open() - build system to hatch
py-fsspec: updated to 2023.12.2 2023.12.2 Fixes top-level glob in ZIP append mode on local ZIP files/truncate restrict ":" as protocol marker to data: sftp relative paths http encoding in HTTP FS put_file 2023.12.1 Fixes Remove trailing "/" from directory names in zipFS/archive 2023.12.0 Enhancements allow HTTP size guess in more circumstances allow kwargs passed to GUI to be dict transaction support for writing via a cache FS make cached FSs work better with async backends allow FSs to set their transaction implementation add dataFS allow basic auth in webHDFS Fixes in referenceFS, maintain order when some keys are omitted in cat nested subdirectory listing in referenceFS allow "=" in webHDF paths fix file mode to consistent "r+b" format pass on kwargs in HTTP glob allow Path in can_be_local and open_local fix parent for cachedFS "ends" list in _cat_ranges was incorrect Other smarter handling of exceptions when doing auto_mkdir
python/wheel.mk: simplify a lot, and switch to 'installer' for installation This follows the recommended bootstrap method (flit_core, build, installer). However, installer installs different files than pip, so update PLISTs for all packages using wheel.mk and bump their PKGREVISIONs.
py-fsspec: updated to 2023.9.2 2023.9.2 Fixes revert: auto_mkdir in open() Other code style updates update CI setup 2023.9.1 Enhancements * save file cache metadata in JSON * remove some unnecessary list iterations Fixes * re-allow None for default port for SMB * initialising GUI widget FS with kwargs * pass auto_mkdir vi url_to_fs again Other * auto delete temp cache directory 2023.9.0 Enhancements * add ocilake protocol * implement async-sync and async-async generic cp and rsync * add lakefs protocol * add goatcounter to docs * add xethub protocol * in HTTP, check content-encoding when getting length * add on_error in walk * add dirfs attribute to mappers * configure port for smb Fixes * don't reorder paths in bulk ops if source and dest are both lists * allow mode="x" in get_fs_token_paths * allow generic to work with complex URLs * exclude bytes-cache kwargs in url_to_fs * remote utcnow/utcfromtimestamp * dirFS's protocol * use get_file rather than get in file caching * allow bz2 to be optional Other * more bulk ops testing * separate out classes in file caching for future enhancements
py-fsspec: updated to 2023.6.0 2023.6.0 Enhancements * add maxdepth fo cp/get/put * allow dir modification during walk() * add boxfs to registry * optimise referenceFS lazy lookups, especially for writing parquet * "encoding" for FTP Fixes * (re)allow reading .zstd reference sets * resource.error for win32 * range reads in dbfs * create parent directories in get_file * off-by-one in reference block writing * strip protocol in local rm_file Other * async bulk tests * types and mypy * drop outdated forms io.open, IOError 2023.5.0 Enhancements * allow writing ReferenceFS references directly to parquet Fixes * copy of glob to single output directory * non-recursive copy of directory (no-op) * cleanup fix on close of ZIP FS * ignore dirs when copying list of files * don't error on register without clobber is registering same thing again * special case for other_files and relative path Other * add test harness into released wheel package * docs and tests around common bulk file operations
py-fsspec: added version 2023.4.0 Filesystem Spec (fsspec) is a project to provide a unified pythonic interface to local, remote and embedded file systems and bytes storage.