Polars

The Polars skill enables AI agents to perform high-performance data manipulation using the Rust-based Polars library. It allows for efficient processing of large datasets through lazy evaluation and parallel execution, significantly speeding up complex analytical tasks compared to standard Python libraries.

When to use it

• Processing massive CSV or Parquet files that cause memory errors in Pandas.
• Performing complex aggregations and joins on multi-terabyte datasets.
• Executing time-sensitive data transformations where latency is critical.
• Running parallel queries across multiple CPU cores for faster computation.

Key capabilities

• Lazy evaluation engine for optimized query planning before execution.
• Native support for streaming data and out-of-core processing.
• High-performance multi-threaded execution for parallel computing.
• Efficient handling of various file formats including CSV, Parquet, JSON, and Avro.

Example prompts

• "Load this 5GB CSV file into a Polars DataFrame and filter rows where the sales amount exceeds $10,000."
• "Group the dataset by region and calculate the average transaction value using Polars."
• "Join two large datasets on customer ID and sort the result by date in descending order using Polars."

Tips & gotchas

Ensure your AI agent has access to the polars Python package installed via pip before attempting to execute code. While Polars is incredibly fast, it requires understanding its specific API syntax, which differs slightly from Pandas, so verify that the generated code uses correct method names for operations like filtering or grouping.