fsspeckit¶
fsspeckit enhances fsspec with utilities for multi-format I/O, storage
configuration, and dataset processing. It gives you one interface for reading
and writing datasets across local and cloud storage, with SQL-based filtering
and DuckDB-backed analytics built in.
Start here¶
New to fsspeckit? Work through the Local Dataset Lifecycle tutorial. It is a single, copyable, offline script: configure a local filesystem, write a Parquet dataset, read it back, verify the result, and clean up a named sandbox. This is the recommended first path for anyone using fsspeckit as a library.
Where to go next¶
Once you have the basics, pick a lane:
- How-to Guides - practical recipes for specific tasks
- Reference - the complete API tour; see also Installation for setup and the dependency matrix
- Explanation - the concepts and architecture behind fsspeckit
- Migration - moving from older module layouts
- Integrations - wire up cloud backends in Configure Cloud Storage and query layers in Use SQL Filters; check Installation for the full extras matrix
Key features¶
- Multi-cloud support: unified interface for AWS S3, Azure Blob Storage, and Google Cloud Storage
- Dataset operations: Parquet processing with DuckDB and PyArrow backends
- SQL filter translation: write filters once and run them across PyArrow and Polars
- Enhanced filesystem API: extended I/O methods with automatic batching and threading
- Path safety by default: built-in protection against directory traversal attacks
- Domain package architecture: organized APIs for discoverability and type safety
Contributing¶
fsspeckit is open source. Read the Contributing Guide if you would like to help out, and file bugs or feature requests on GitHub.
Badges¶
