Installs just this skill. Get the whole plugin for auto-invocation.
⚡ How it fires
How this skill gets triggered: by you, by Claude, or both.
Fires itselfClaude auto-loads it when your prompt matches the work.
You can call itInvoke it directly when you want it.
Slash command/zarr-python
👁️ Context preview
The summary Claude sees to decide when to auto-load this skill.
Chunked N-D arrays for cloud storage (Zarr-Python 3). Compressed arrays, parallel I/O, S3/GCS via fsspec, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
📊 Stats
Stars31,542
Forks3,146
LanguagePython
LicenseMIT
📦 Ships with scientific-agent-skills
</> SKILL.md
zarr-python.SKILL.md
---name: zarr-python
description: Chunked N-D arrays for cloud storage (Zarr-Python 3). Compressed arrays, parallel I/O, S3/GCS via fsspec, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
allowed-tools: Read Write Edit Bash
license: MIT license
compatibility: Requires Python 3.12+ and zarr 3.x. Cloud I/O needs zarr[remote] plus pinned s3fs or gcsfs. Legacy Zarr v2 workflows need exact 2.x pins on older Python.
metadata: {"version": "1.1", "skill-author": "K-Dense Inc."}
---# Zarr Python
## Overview
Zarr is a Python library for storing large N-dimensional arrays with chunking and compression. Apply this skill for efficient parallel I/O, cloud-native workflows, and seamless integration with NumPy, Dask, and Xarray.
**Current upstream:** zarr **3.2.1** (released 2026-05-05). Docs: [zarr.readthedocs.io](https://zarr.readthedocs.io/en/stable/). New arrays default to **Zarr format 3**; set `zarr_format=2` for legacy interop. Zarr 3.2 adds rectilinear chunks and continues to refine the v3 codec pipeline. This skill is a **community guide** maintained by K-Dense Inc., not an official zarr-developers package.
## Quick Start
### Installation
```bash
uv pip install "zarr==3.2.1"
```
Requires **Python 3.12+** and NumPy 2.0+ for current stable Zarr-Python. For remote stores (S3, GCS, HTTP), pin the optional extras/backends in your project lockfile:
Use a version range such as `zarr>=3,<4` only when your project has a committed lockfile and compatibility tests. For Zarr-Python 2 / Python 3.10–3.11 workflows, choose an exact `zarr==2.x.y` patch version from the support-v2 release notes and commit the resulting lockfile.
### Basic Array Creation
```python
import zarr
import numpy as np
# Create a 2D array with chunking and compression
z = zarr.create_array(
store="data/my_array.zarr",
shape=(10000, 10000),
chunks=(1000, 1000),
dtype="f4"
)
# Write data using NumPy-style indexing
z[:, :] = np.random.random((10000, 10000))
# Read data
data = z[0:100, 0:100] # Returns NumPy array
```
## Core Operations
### Creating Arrays
Zarr provides multiple convenience functions for array creation:
```python
# Create empty array
z = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000), dtype='f4',
store='data.zarr')
# Create filled arrays
z = zarr.ones((5000, 5000), chunks=(500, 500))
z = zarr.full((1000, 1000), fill_value=42, chunks=(100, 100))
# Create from existing data
data = np.arange(10000).reshape(100, 100)
z = zarr.array(data, chunks=(10, 10), store='data.zarr')
# Create like another array
z2 = zarr.zeros_like(z) # Matches shape, chunks, dtype of z
```
### Opening Existing Arrays
```python
# Open array (read/write mode by default)
z = zarr.open_array('data.zarr', mode='r+')
# Read-only mode
z = zarr.open_array('data.zarr', mode='r')
# The open() function auto-detects arrays vs groups
z = zarr.open('data.zarr') # Returns Array or Group
# Never print, log, or copy credential values into prompts or notebooks.
z = zarr.create_array(
store="s3://my-bucket/path/to/array.zarr",
shape=(1000, 1000),
chunks=(100, 100),
dtype="f4",
storage_options={"anon": False},
)
z[:] = data
# GCS — prefer workload identity or gcloud application-default credentials.
z = zarr.open_array(
"gs://my-bucket/path/to/array.zarr",
mode="r",
storage_options={"project": "my-project"},
)
# Explicit store (any fsspec filesystem)
from zarr.storage import FsspecStore
store = FsspecStore.from_url("s3://my-bucket/data.zarr", storage_options={"anon": False})
root = zarr.open_group(store=store, mode="r+")
```
Cloud backends read credentials through the provider SDK/fsspec backend. Do not inspect broad `.env` files; if a user explicitly needs help debugging auth, ask for redacted configuration and read only the named provider variables they approve. Treat all `import zarr`, `import dask`, `import h5py`, and `import xarray` examples as third-party package imports, not bundled script files.
**Cloud Storage Best Practices**:
- Use consolidated metadata to reduce latency: `zarr.consolidate_metadata(store)`
For Dask-heavy workloads, estimate total concurrent I/O as roughly `dask_threads × async.concurrency` and lower Zarr's concurrency settings if the store or memory becomes saturated.
## Consolidated Metadata
For hierarchical stores with many arrays, consolidate metadata into a single file to reduce I/O operations:
```python
import zarr
# After creating arrays/groups
root = zarr.group('data.zarr')
# ... create multiple arrays/groups ...
# Consolidate metadata
zarr.consolidate_metadata('data.zarr')
# Open with consolidated metadata (faster, especially on cloud storage)
root = zarr.open_consolidated('data.zarr')
```
**Benefits**:
- Reduces metadata read operations from N (one per array) to 1
- Critical for cloud storage (reduces latency)
- Speeds up `tree()` operations and group traversal
**Cautions**:
- Metadata can become stale if arrays update without re-consolidation
- Not suitable for frequently-updated datasets
- Multi-writer scenarios may have inconsistent reads
z = zarr.array(data, chunks='auto', store='data.zarr')
# Zarr to NetCDF (via Xarray)
import xarray as xr
ds = xr.open_zarr('data.zarr')
ds.to_netcdf('data.nc')
```
## Common Issues and Solutions
### Issue: Slow Performance
**Diagnosis**: Check chunk size and alignment
```python
print(z.chunks) # Are chunks appropriate size?
print(z.info) # Check compression ratio
```
**Solutions**:
- Increase chunk size to 1-10 MB
- Align chunks with access pattern
- Try different compression codecs
- Use Dask for parallel operations
### Issue: High Memory Usage
**Cause**: Loading entire array or large chunks into memory
**Solutions**:
```python
# Don't load entire array
# Bad: data = z[:]
# Good: Process in chunks
for i in range(0, z.shape[0], 1000):
chunk = z[i:i+1000, :]
process(chunk)
# Or use Dask for automatic chunking
import dask.array as da
dask_z = da.from_zarr('data.zarr')
result = dask_z.mean().compute() # Processes in chunks
```
### Issue: Cloud Storage Latency
**Solutions**:
```python
# 1. Consolidate metadata
zarr.consolidate_metadata(store)
z = zarr.open_consolidated(store)
# 2. Use appropriate chunk sizes (5-100 MB for cloud)
chunks = (2000, 2000) # Larger chunks for cloud
# 3. Enable sharding
shards = (10000, 10000) # Groups many chunks
```
### Issue: Concurrent Write Conflicts
**Solution**: Design workflows so each process/thread writes to separate chunks. Zarr-Python 3 does not yet support `ThreadSynchronizer` / `ProcessSynchronizer`; see `references/v3_migration.md`.
## Additional Resources
### Bundled references
| File | Contents |
|------|----------|
| `references/api_reference.md` | Function signatures, stores, codecs, indexing |
| `references/v3_migration.md` | Zarr-Python 2→3 breaking changes and WIP features |
