The NetBSD Project

CVS log for pkgsrc/sysutils/py-fsspec/PLIST

[BACK] Up to [cvs.NetBSD.org] / pkgsrc / sysutils / py-fsspec

Request diff between arbitrary revisions


Keyword substitution: kv
Default branch: MAIN


Revision 1.12: download - view: text, markup, annotated - select for diffs
Wed Oct 28 08:57:52 2026 UTC (2 weeks, 4 days ago) by adam
Branches: MAIN
CVS tags: HEAD
Diff to: previous 1.11: preferred, colored
Changes since revision 1.11: +4 -1 lines
py-fsspec: updated to 2026.9.0

2026.9.0

Enhancements

ia:// filesystem for files in Internet Archive items: an HTTPFileSystem reading from archive.org/download with credentials from ia configure's ia.ini
Allow instance-local memory stores with global_store=False while preserving normal instance caching and the default shared store
Add an adaptive readahead cache
Implement created and modified for SFTPFileSystem
Allow filesystem implementations to assign protocol per instance
Delegate sizes and cat_ranges from DirFileSystem to the wrapped filesystem
Delegate invalidate_cache, ukey and checksum from DirFileSystem to the wrapped filesystem
Support negative offsets in TarFileSystem.cat_file
Mirror readinto from the PyArrow stream onto ArrowFile
Allow simplecache files to be removed by age using their modification times
HTTP: retry transient failures (408/425/429/5xx, dropped connections, timeouts, truncated range bodies, a 416 inside the file) of cat_file, HTTPFile block reads and the size lookup behind open, controlled by the new retries, retry_wait, retry_statuses and retry_exceptions options

Fixes

Compute HadoopFileSystem.fsid from its configured host and port instead of accessing attributes no longer exposed by PyArrow
Normalize dot path components in TarFileSystem listings and lookups, while preserving original archive names for reading
Make merge_offset_ranges O(n log n) and keep merged blocks within max_block, replacing the quadratic nested-range filter added
Fix transactions with compressed files
Make MemoryFileSystem.rm raise FileNotFoundError for a path that does not exist, as rm_file and the other filesystems do, so deleting a missing key from a memory-backed mapper raises KeyError
Create missing ZIP archives in append mode without truncating existing archives
Close cached readers when exiting an open_files context
Avoid mutating live BlockCache and BackgroundBlockCache instances when pickling
End the transaction even when a commit or discard raises, so the filesystem is not left in transaction mode and deferred temporary files are cleaned up
Honor a caller-owned output file in get_file
Preserve existing files when checking FSMap write access
Normalize ZIP paths in pipe_file and find
Reject get destinations that resolve above the target
Pass maxdepth through in AsyncFileSystem._rm
Do not let isfile swallow BaseException
Keep reference filesystem get destinations inside the target
Return relative names from DirFileSystem.glob and find
Raise FileNotFoundError for missing ZIP and TAR members
Stop collapsing three-character drive-relative paths to the drive root
Use / as the only glob separator on every platform
Distinguish omitted and None defaults in FSMap.pop
Forward an Arrow file's flush and closed state to its stream
Keep reference filesystem cat_file offsets inside the referenced part
Raise FileNotFoundError from archive ls for a missing path
Download uncached files in simplecache.cat_ranges
Fix tail on simplecache and filecache
Get whole-file reference sizes asynchronously in ReferenceFileSystem._info
Report the target size for links in TarFileSystem
Keep the async generator returned by AsyncFileSystemWrapper._walk
Join the default output filename with / in _expand_paths
Make names relative in DirFileSystem.walk(detail=True)
Ignore a leading slash in TarFileSystem paths
Fix cat_ranges on whole-file caches
Normalize paths in AsyncFileSystemWrapper like the wrapped filesystem
The SFTP host key policy is configurable with the new host_key_policy argument of SFTPFileSystem, which accepts "auto_add" (default, unchanged behaviour), "warning" or "reject", or any paramiko.MissingHostKeyPolicy

Revision 1.11: download - view: text, markup, annotated - select for diffs
Tue Nov 11 10:22:05 2025 UTC (12 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2026Q3-base, pkgsrc-2026Q3, pkgsrc-2026Q2-base, pkgsrc-2026Q2, pkgsrc-2026Q1-base, pkgsrc-2026Q1, pkgsrc-2025Q4-base, pkgsrc-2025Q4
Diff to: previous 1.10: preferred, colored
Changes since revision 1.10: +4 -1 lines
py-fsspec: updated to 2025.10.0

2025.10.0

Enhancements

- specify storage_options not use use for tokenising; test instance cache
- base class "chained" for FSs that pass through operations

Fixes

- strip protocol for sftp
- use one-step mv in jupyterfs
- raise errors in jupyterfs, not return
- add protocol attribute to http
- allow ls() on a single file for arrowFS
- support kwargs in webhdfs ls()
- better discrimination for parquet in referenceFS
- accept all mode= types in memoryFS
- pass path list to put() in localtempfile for efficiency
- fix docs errors

Other

- add CoC
- restrict github/gist tests because of rate limiting

Revision 1.10: download - view: text, markup, annotated - select for diffs
Sun May 25 07:28:51 2025 UTC (17 months, 3 weeks ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2025Q3-base, pkgsrc-2025Q3, pkgsrc-2025Q2-base, pkgsrc-2025Q2
Diff to: previous 1.9: preferred, colored
Changes since revision 1.9: +4 -1 lines
py-fsspec: updated to 2025.5.1

2025.5.1

Enhancements
- file system for GitHub gists

Other
- doc fixes

Revision 1.9: download - view: text, markup, annotated - select for diffs
Tue Mar 11 13:47:41 2025 UTC (20 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2025Q1-base, pkgsrc-2025Q1
Diff to: previous 1.8: preferred, colored
Changes since revision 1.8: +4 -1 lines
py-fsspec: updated to 2025.3.0

2025.3.0

Enhancements

- add pipe_file to HTTP
- add sync http for pyodide
- ls performance for local and detail=False

Fixes

- dir/info consistency in dirfs
- referenceFS async consistency
- CI

Revision 1.8: download - view: text, markup, annotated - select for diffs
Tue Dec 24 17:53:54 2024 UTC (22 months, 3 weeks ago) by adam
Branches: MAIN
Diff to: previous 1.7: preferred, colored
Changes since revision 1.7: +10 -1 lines
py-fsspec: updated to 2024.12.0

2024.12.0

Enhancements

- "exclusive" mode for writing
- "tree" text display of filesystem contents
- async wrapper for sync FSs
- new known implementation: tosfs
- consilidate block fetch requests

Fixes

- better webHDFS proxies
- syn FSs in referenceFS
- don't serialize file caches
- race condition in local ls()
- missing/nan references in parquet
- _un_chain kwargs (@1736)
- async _cat_file in referenceFS

Other

- fallback implementation for _fetch_range

Revision 1.7: download - view: text, markup, annotated - select for diffs
Thu Jun 6 05:03:14 2024 UTC (2 years, 5 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2024Q4-base, pkgsrc-2024Q4, pkgsrc-2024Q3-base, pkgsrc-2024Q3, pkgsrc-2024Q2-base, pkgsrc-2024Q2
Diff to: previous 1.6: preferred, colored
Changes since revision 1.6: +4 -154 lines
py-fsspec: updated to 2024.6.0

2024.6.0

Enhancements

- allow dicts (not just bytes) for referenceFS
- make filesystems JSON serializeable
- implement multifile cat() for github

Fixes

- implement auto_mkdir for SMB

Other

- add doc deps to pyproject
- re-remove test from package
- formatting
- change monthly download badge

Revision 1.6: download - view: text, markup, annotated - select for diffs
Wed May 15 15:19:03 2024 UTC (2 years, 6 months ago) by adam
Branches: MAIN
Diff to: previous 1.5: preferred, colored
Changes since revision 1.5: +158 -3 lines
py-fsspec: updated to 2024.5.0

2024.5.0

Enhancements

- log hits/misses in bytes cachers

Fixes

- SMB flaky tests
- rsync: only delete files if there are some to delete
- don't let files equal bytes objects
- url_to_fs to stringify paths
- assert types in MemoryFS
- restore _strip_protocol signature for local
- convert list to set when loading cache metadata

Other

- remove mv_file
- mv() should not swallow errors
- change versioning template, allows easier co-install of dev s3fs
- in ls_from_cache, avoid dounble lookup
- expand=True in open()
- build system to hatch

Revision 1.5: download - view: text, markup, annotated - select for diffs
Sun Dec 17 20:25:29 2023 UTC (2 years, 10 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2024Q1-base, pkgsrc-2024Q1, pkgsrc-2023Q4-base, pkgsrc-2023Q4
Diff to: previous 1.4: preferred, colored
Changes since revision 1.4: +4 -1 lines
py-fsspec: updated to 2023.12.2

2023.12.2

Fixes

top-level glob in ZIP
append mode on local ZIP files/truncate
restrict ":" as protocol marker to data:
sftp relative paths
http encoding in HTTP FS put_file

2023.12.1

Fixes

Remove trailing "/" from directory names in zipFS/archive

2023.12.0

Enhancements

allow HTTP size guess in more circumstances
allow kwargs passed to GUI to be dict
transaction support for writing via a cache FS
make cached FSs work better with async backends
allow FSs to set their transaction implementation
add dataFS
allow basic auth in webHDFS

Fixes

in referenceFS, maintain order when some keys are omitted in cat
nested subdirectory listing in referenceFS
allow "=" in webHDF paths
fix file mode to consistent "r+b" format
pass on kwargs in HTTP glob
allow Path in can_be_local and open_local
fix parent for cachedFS
"ends" list in _cat_ranges was incorrect

Other

smarter handling of exceptions when doing auto_mkdir

Revision 1.4: download - view: text, markup, annotated - select for diffs
Sat Oct 28 19:57:16 2023 UTC (3 years ago) by wiz
Branches: MAIN
Diff to: previous 1.3: preferred, colored
Changes since revision 1.3: +49 -4 lines
python/wheel.mk: simplify a lot, and switch to 'installer' for installation

This follows the recommended bootstrap method (flit_core, build, installer).

However, installer installs different files than pip, so update PLISTs
for all packages using wheel.mk and bump their PKGREVISIONs.

Revision 1.3: download - view: text, markup, annotated - select for diffs
Mon Oct 16 13:01:44 2023 UTC (3 years, 1 month ago) by adam
Branches: MAIN
Diff to: previous 1.2: preferred, colored
Changes since revision 1.2: +7 -1 lines
py-fsspec: updated to 2023.9.2

2023.9.2

Fixes

revert: auto_mkdir in open()

Other

code style updates
update CI setup

2023.9.1

Enhancements

* save file cache metadata in JSON
* remove some unnecessary list iterations

Fixes

* re-allow None for default port for SMB
* initialising GUI widget FS with kwargs
* pass auto_mkdir vi url_to_fs again

Other

* auto delete temp cache directory

2023.9.0

Enhancements

* add ocilake protocol
* implement async-sync and async-async generic cp and rsync
* add lakefs protocol
* add goatcounter to docs
* add xethub protocol
* in HTTP, check content-encoding when getting length
* add on_error in walk
* add dirfs attribute to mappers
* configure port for smb

Fixes

* don't reorder paths in bulk ops if source and dest are both lists
* allow mode="x" in get_fs_token_paths
* allow generic to work with complex URLs
* exclude bytes-cache kwargs in url_to_fs
* remote utcnow/utcfromtimestamp
* dirFS's protocol
* use get_file rather than get in file caching
* allow bz2 to be optional

Other

* more bulk ops testing
* separate out classes in file caching for future enhancements

Revision 1.2: download - view: text, markup, annotated - select for diffs
Fri Aug 25 22:14:01 2023 UTC (3 years, 2 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2023Q3-base, pkgsrc-2023Q3
Diff to: previous 1.1: preferred, colored
Changes since revision 1.1: +9 -1 lines
py-fsspec: updated to 2023.6.0

2023.6.0

Enhancements

* add maxdepth fo cp/get/put
* allow dir modification during walk()
* add boxfs to registry
* optimise referenceFS lazy lookups, especially for writing parquet
* "encoding" for FTP

Fixes

* (re)allow reading .zstd reference sets
* resource.error for win32
* range reads in dbfs
* create parent directories in get_file
* off-by-one in reference block writing
* strip protocol in local rm_file

Other

* async bulk tests
* types and mypy
* drop outdated forms io.open, IOError


2023.5.0

Enhancements

* allow writing ReferenceFS references directly to parquet

Fixes

* copy of glob to single output directory
* non-recursive copy of directory (no-op)
* cleanup fix on close of ZIP FS
* ignore dirs when copying list of files
* don't error on register without clobber is registering same thing again
* special case for other_files and relative path

Other

* add test harness into released wheel package
* docs and tests around common bulk file operations

Revision 1.1: download - view: text, markup, annotated - select for diffs
Mon Apr 24 07:01:43 2023 UTC (3 years, 6 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2023Q2-base, pkgsrc-2023Q2
py-fsspec: added version 2023.4.0

Filesystem Spec (fsspec) is a project to provide a unified pythonic interface
to local, remote and embedded file systems and bytes storage.

Diff request

This form allows you to request diffs between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.

Log view options

CVSweb <webmaster@jp.NetBSD.org>