/polars

name: polars
description: >-
  Polars DataFrame library for high-performance data manipulation. Lazy/eager execution, expressions, I/O (CSV, Parquet, JSON), aggregations, joins, string/datetime ops, pandas interop. Use for Polars DataFrames or reading/writing Parquet files.
metadata:
  audience: research-coders
  domain: python-library
  library-version: "1.x"
  skill-last-updated: "2026-03-26"

Polars Skill

Polars DataFrame library for high-performance data manipulation in Python. Covers lazy/eager execution, expressions, I/O (CSV, Parquet, JSON, database), aggregations, joins, string/datetime operations, pandas/NumPy interop, and performance optimization. Use when working with Polars DataFrames, migrating from pandas, reading Parquet files, or optimizing data pipeline performance.

Comprehensive skill for high-performance data manipulation with Polars. Use decision trees below to find the right guidance, then load detailed references.

What is Polars?

Polars is a **fast** DataFrame library for Python (and Rust):

• **Fast**: Written in Rust, optimized for modern CPUs with SIMD and parallelism
• **Lazy Evaluation**: Build query plans that get optimized before execution
• **Expressive**: Powerful expression API for complex transformations
• **Memory Efficient**: Columnar format, streaming for larger-than-memory data
• **No Dependencies**: Pure Rust core, no NumPy/Pandas required

Version Notes

This skill targets **Polars 1.x** (tested with 1.37.1). Key changes from 0.x:

• `apply` renamed to `map_elements` (0.19+)
• `groupby` renamed to `group_by` (0.19+)
• `melt` renamed to `unpivot` (1.0+)
• Streaming engine improvements in 1.x
• `pl.Utf8` is now `pl.String` (1.0+, Utf8 still works as alias)

How to Use This Skill

Reference File Structure

Each topic in `./references/` contains focused documentation:

| File | Purpose | When to Read | |------|---------|--------------| | `quickstart.md` | Installation, concepts, first DataFrame | Starting with Polars | | `dataframes-series.md` | Creation, selection, filtering, modification | Basic data manipulation | | `io-data.md` | CSV, Parquet, JSON, database I/O | Loading/saving data | | `expressions.md` | Expression system, contexts, chaining | Understanding Polars idioms | | `aggregations-grouping.md` | GroupBy, window functions, statistics | Summarizing data | | `joins-concat.md` | Joins, concatenation, pivot/unpivot | Combining DataFrames | | `strings-datetime-categorical.md` | String ops, datetime, categoricals | Type-specific operations | | `performance.md` | Lazy execution, optimization, anti-patterns | Making code faster | | `interop.md` | Pandas, NumPy, PyArrow, DuckDB | Working with other tools | | `gotchas.md` | Common errors, anti-patterns, migration | Debugging issues |

Reading Order

1. **New to Polars?** Start with `quickstart.md` then `expressions.md` 2. **Coming from Pandas?** Read `quickstart.md`, `expressions.md`, then `interop.md` 3. **Performance issues?** Check `performance.md` first

Quick Decision Trees

"I need to get started"

Getting started?
├─ Install Polars → ./references/quickstart.md
├─ Create first DataFrame → ./references/quickstart.md
├─ Understand lazy vs eager → ./references/quickstart.md
├─ Learn expression syntax → ./references/expressions.md
└─ Coming from Pandas → ./references/interop.md

"I need to load or save data"

Loading/saving data?
├─ Read CSV file → ./references/io-data.md
├─ Read Parquet (recommended) → ./references/io-data.md
├─ Read JSON/NDJSON → ./references/io-data.md
├─ Read from database → ./references/io-data.md
├─ Read multiple files (glob) → ./references/io-data.md
├─ Write to file → ./references/io-data.md
└─ Larger-than-memory data → ./references/performance.md

"I need to filter or select data"

Filtering/selecting?
├─ Select columns by name → ./references/dataframes-series.md
├─ Select by pattern/regex → ./references/dataframes-series.md
├─ Select by data type → ./references/dataframes-series.md
├─ Filter rows by condition → ./references/dataframes-series.md
├─ Filter with multiple conditions → ./references/dataframes-series.md
├─ Handle null values → ./references/dataframes-series.md
└─ Add/modify columns → ./references/dataframes-series.md

"I need to aggregate or group data"

Aggregating data?
├─ Basic statistics (sum, mean, etc.) → ./references/aggregations-grouping.md
├─ Group by columns → ./references/aggregations-grouping.md
├─ Multiple aggregations → ./references/aggregations-grouping.md
├─ Window functions (over) → ./references/aggregations-grouping.md
├─ Rolling/moving averages → ./references/aggregations-grouping.md
├─ Cumulative operations → ./references/aggregations-grouping.md
└─ Ranking within groups → ./references/aggregations-grouping.md

"I need to combine DataFrames"

Combining data?
├─ Join two DataFrames → ./references/joins-concat.md
├─ Left/right/outer join → ./references/joins-concat.md
├─ Anti-join (not in) → ./references/joins-concat.md
├─ Concatenate vertically → ./references/joins-concat.md
├─ Pivot (long to wide) → ./references/joins-concat.md
└─ Unpivot/melt (wide to long) → ./references/joins-concat.md

"I need better performance"

Performance issues?
├─ Use lazy evaluation → ./references/performance.md
├─ Avoid row iteration → ./references/performance.md
├─ Reduce memory usage → ./references/performance.md
├─ Process large files → ./references/performance.md
├─ Optimize query plan → ./references/performance.md
└─ Common anti-patterns → ./references/performance.md

"Something isn't working"

Having issues?
├─ Type errors → ./references/gotchas.md
├─ Null handling → ./references/gotchas.md
├─ Expression context errors → ./references/gotchas.md
├─ String operations → ./references/strings-datetime-categorical.md
├─ Date parsing issues → ./references/strings-datetime-categorical.md
├─ Performance problems → ./references/gotchas.md
├─ Pandas migratio