Skip to content

fsspeckit

fsspeckit enhances fsspec with advanced utilities for multi-format I/O, cloud storage configuration, and high-performance data processing.

Quick Start

New to fsspeckit? Start with our Getting Started tutorial for a complete walkthrough.

Looking for specific tasks? Browse our How-to Guides for practical recipes:

Key Features

  • Multi-Cloud Support: Unified interface for AWS S3, Azure Blob Storage, Google Cloud Storage
  • Advanced Dataset Operations: High-performance Parquet processing with DuckDB integration
  • SQL Filter Translation: Write filters once, use across PyArrow and Polars
  • Enhanced Filesystem API: Extended I/O methods with automatic batching and threading
  • Path Safety by Default: Built-in protection against directory traversal attacks
  • Domain Package Architecture: Organized APIs for better discoverability and type safety

Documentation

Learning Paths

Beginners: Start with Getting Started for hands-on learning

Practical Users: Jump to How-to Guides for specific task solutions

Developers: Reference API Guide for capability overview

Architects: Understand design decisions in Architecture & Concepts

Reference Materials

Architecture Overview

fsspeckit is organized into domain-specific packages:

  • fsspeckit.core - Filesystem creation and extended I/O
  • fsspeckit.storage_options - Cloud provider configuration
  • fsspeckit.datasets - Large-scale dataset operations
  • fsspeckit.sql - Cross-framework SQL translation
  • fsspeckit.common - Shared utilities and helpers
  • fsspeckit.utils - Backwards compatibility façade

Getting Help

Contributing: See our Contributing Guide to help improve fsspeckit

Issues: Report bugs and request features on GitHub

Community: Join discussions and connect with other users

Badges

GitHub PyPI