From aac9586ef241aa47973ac83aca264fd07dcf86a6 Mon Sep 17 00:00:00 2001 From: Leon Anavi Date: Tue, 20 Oct 2026 15:32:25 +0300 Subject: [PATCH] python3-fsspec: Upgrade 2026.7.0 -> 2026.9.0 Switch SRC_URI to enable tests and upgrade to release 2026.9.0: - ia:// filesystem for files in Internet Archive items: an HTTPFileSystem reading from archive.org/download with credentials from ia configure's ia.ini - Allow instance-local memory stores with global_store=False while preserving normal instance caching and the default shared store - Add an adaptive readahead cache - Implement created and modified for SFTPFileSystem - Allow filesystem implementations to assign protocol per instance - Delegate sizes and cat_ranges from DirFileSystem to the wrapped filesystem - Delegate invalidate_cache, ukey and checksum from DirFileSystem to the wrapped filesystem - Support negative offsets in TarFileSystem.cat_file - Mirror readinto from the PyArrow stream onto ArrowFile - Allow simplecache files to be removed by age using their modification times - HTTP: retry transient failures (408/425/429/5xx, dropped connections, timeouts, truncated range bodies, a 416 inside the file) of cat_file, HTTPFile block reads and the size lookup behind open, controlled by the new retries, retry_wait, retry_statuses and retry_exceptions options Fixes: - Compute HadoopFileSystem.fsid from its configured host and port instead of accessing attributes no longer exposed by PyArrow - Normalize dot path components in TarFileSystem listings and lookups, while preserving original archive names for reading - Make merge_offset_ranges O(n log n) and keep merged blocks within max_block, replacing the quadratic nested-range filter - Fix transactions with compressed files - Make MemoryFileSystem.rm raise FileNotFoundError for a path that does not exist, as rm_file and the other filesystems do, so deleting a missing key from a memory-backed mapper raises KeyError - Create missing ZIP archives in append mode without truncating existing archives - Close cached readers when exiting an open_files context - Avoid mutating live BlockCache and BackgroundBlockCache instances when pickling - End the transaction even when a commit or discard raises, so the filesystem is not left in transaction mode and deferred temporary files are cleaned up - Honor a caller-owned output file in get_file - Preserve existing files when checking FSMap write access - Normalize ZIP paths in pipe_file and find - Reject get destinations that resolve above the target - Pass maxdepth through in AsyncFileSystem._rm - Do not let isfile swallow BaseException - Keep reference filesystem get destinations inside the target - Return relative names from DirFileSystem.glob and find - Raise FileNotFoundError for missing ZIP and TAR members - Stop collapsing three-character drive-relative paths to the drive root - Use / as the only glob separator on every platform - Distinguish omitted and None defaults in FSMap.pop - Forward an Arrow file's flush and closed state to its stream - Keep reference filesystem cat_file offsets inside the referenced part - Raise FileNotFoundError from archive ls for a missing path - Download uncached files in simplecache.cat_ranges - Fix tail on simplecache and filecache - Get whole-file reference sizes asynchronously in ReferenceFileSystem._info - Report the target size for links in TarFileSystem - Keep the async generator returned by AsyncFileSystemWrapper._walk - Join the default output filename with / in _expand_paths - Make names relative in DirFileSystem.walk(detail=True) - Ignore a leading slash in TarFileSystem paths - Fix cat_ranges on whole-file caches - Normalize paths in AsyncFileSystemWrapper like the wrapped filesystem - The SFTP host key policy is configurable with the new host_key_policy argument of SFTPFileSystem, which accepts "auto_add" (default, unchanged behaviour), "warning" or "reject", or any paramiko.MissingHostKeyPolicy Other: - Use trusted publishing for releases and trigger the release workflow on tags - Remove the stale async wrapper documentation warning - Add Zarr to the test_downstream extra so the Xarray/Zarr test runs again - Clarify that rm handles directories - Fix grammar in features.rst - Patch HAS_CPYTHON_API where _fast_slice reads it in tests - Percent-decode the username and password parsed from URLs in infer_storage_options so that backends (ftp, sftp, smb, ...) receive the real credentials rather than their URL-encoded form; a password whose bare % produces bytes that are not valid UTF-8 (for example pass%ab) is left untouched so that pre-existing URLs with unescaped % continue to work Signed-off-by: Leon Anavi Signed-off-by: Khem Raj --- .../python/python3-fsspec_2026.7.0.bb | 24 ------------ .../python/python3-fsspec_2026.9.0.bb | 37 +++++++++++++++++++ 2 files changed, 37 insertions(+), 24 deletions(-) delete mode 100644 meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb create mode 100644 meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb diff --git a/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb b/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb deleted file mode 100644 index d34b67c428..0000000000 --- a/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb +++ /dev/null @@ -1,24 +0,0 @@ -SUMMARY = "A specification that python filesystems should adhere to." -HOMEPAGE = "https://github.com/fsspec/filesystem_spec" -LICENSE = "BSD-3-Clause" -LIC_FILES_CHKSUM = "file://LICENSE;md5=b38a11bf4dcdfc66307f8515ce1fbaa6" - -SRC_URI[sha256sum] = "c803c40f4cf860b49dea58ee3e1c33cb9c790520e233537e1340049f89b82a88" - -inherit pypi python_hatchling ptest - - -DEPENDS = "python3-hatch-vcs-native python3-hatchling-native" - -RDEPENDS:${PN}-ptest += "\ - python3-pytest \ - python3-pytest-mock \ - python3-pytest-asyncio \ - python3-pytest-cov \ - python3-pytest-benchmark \ - python3-aiohttp \ - python3-numpy \ - python3-requests \ -" - -BBCLASSEXTEND = "native nativesdk" diff --git a/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb b/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb new file mode 100644 index 0000000000..43d405f017 --- /dev/null +++ b/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb @@ -0,0 +1,37 @@ +SUMMARY = "A specification that python filesystems should adhere to." +HOMEPAGE = "https://github.com/fsspec/filesystem_spec" +LICENSE = "BSD-3-Clause" +LIC_FILES_CHKSUM = "file://LICENSE;md5=b38a11bf4dcdfc66307f8515ce1fbaa6" + +SRC_URI = "git://github.com/fsspec/filesystem_spec;protocol=https;branch=master;tag=${PV}" +SRCREV = "0f76baaae4a227b085b18e8469e6cc1792a95ade" + +inherit python_hatchling ptest-python-pytest + +DEPENDS = "python3-hatch-vcs-native python3-hatchling-native" + +RDEPENDS:${PN}-ptest += "\ + python3-pytest \ + python3-pytest-mock \ + python3-pytest-asyncio \ + python3-pytest-cov \ + python3-pytest-benchmark \ + python3-aiohttp \ + python3-numpy \ + python3-requests \ + python3-pandas \ + python3-paramiko \ +" + +do_install_ptest:append() { + install -d ${D}${PTEST_PATH}/tests ${D}${PTEST_PATH}/implementations + cp -rf ${S}/fsspec/tests/* ${D}${PTEST_PATH}/tests/ + cp -rf ${S}/fsspec/implementations/tests ${D}${PTEST_PATH}/implementations/ + touch ${D}${PTEST_PATH}/implementations/__init__.py + # Tests import helpers via fsspec.tests / fsspec.implementations.tests, + # which aren't in the installed package + grep -rlE 'fsspec\.(implementations\.)?tests' ${D}${PTEST_PATH} \ + | xargs -r sed -i -E 's/fsspec\.(implementations\.)?tests\b/\1tests/g' +} + +BBCLASSEXTEND = "native nativesdk"