Pandas Data Processing

What it does

This skill enables AI agents to process data within Pandas DataFrames using Python code. It's specifically designed for time series analysis, handling OrcaFlex simulation results, and creating configuration-driven data workflows common in marine and offshore engineering contexts. The skill facilitates cleaning, transforming, analyzing, and visualizing datasets from various formats like CSV, Excel, HDF5, and Parquet.

When to use it

• Time series analysis: Analyzing wave elevation, vessel motions, or mooring tensions over time.
• OrcaFlex results processing: Analyzing load simulation results, Response Amplitude Operators (RAOs), and dynamics data from OrcaFlex simulations.
• Combining multiple data formats: Working with datasets in CSV, Excel, HDF5, or Parquet formats.
• Statistical analysis: Calculating summary statistics, applying rolling windows, or resampling time series data.
• Automated report generation: Extracting and summarizing data for engineering reports.

Key capabilities

• Time Series Analysis: Loading and processing time series data from CSV files, setting a specified column as the index (and parsing it as dates if desired).
• Data Loading: Supports loading data from CSV, Excel, HDF5, and Parquet file formats.
• Pandas DataFrame Manipulation: Provides capabilities for pivot, melt, merge, and group operations on DataFrames.
• Statistical Analysis: Enables the calculation of summary statistics and resampling time series.

Example prompts

• "Load the vessel motions data from data/processed/vessel_motions.csv and show me the time range."
• "Analyze the OrcaFlex results to calculate the average RAO over a rolling window of 10 seconds."
• “Create a summary table of the mooring tensions, including min, max, and mean values.”

Tips & gotchas

• Dataset Size: This skill is best suited for datasets up to approximately 100GB. For larger datasets, consider using alternatives like Dask, Vaex, or PySpark.
• Real-time Data: This skill isn't designed for real-time streaming data; use libraries optimized for that purpose instead (e.g., Polars).
• Numerical Computation: If your task involves primarily numerical computation without the need for DataFrame manipulation, consider using NumPy directly.