Exploratory Data Analysis

What it does

This skill, Exploratory Data Analysis (EDA), helps AI agents systematically examine datasets to understand their characteristics and assess data quality. It performs data profiling, distribution analysis, relationship discovery, anomaly detection, and evaluates completeness and consistency of data. The tool uses Python libraries like pandas, numpy, matplotlib, and seaborn for these analyses.

When to use it

• Starting a new dataset analysis project.
• Before building models to understand the underlying data.
• When you need to identify potential data quality issues.
• To generate hypotheses that can be tested with further investigation.
• For communicating insights and findings about the data to stakeholders.

Key capabilities

• Data Profiling: Calculates basic statistics and identifies data types within a dataset.
• Distribution Analysis: Visualizes how variables are distributed using histograms and other plots.
• Relationship Discovery: Identifies patterns and correlations between different variables.
• Anomaly Detection: Uses boxplots to help find outliers in the data.
• Data Quality Assessment: Evaluates missing values, duplicate entries, and overall consistency of the dataset.
• Skewness and Kurtosis Analysis: Calculates these statistical measures for numerical columns.
• Percentile Analysis: Determines specific percentile values within a column (e.g., 25th, 50th, 75th).

Example prompts

• "Perform an exploratory data analysis on this customer dataset."
• "Show me the distribution of ages in the 'customer_data.csv' file."
• "Identify any outliers in the income column and show a boxplot."

Tips & gotchas

• This skill requires access to Python libraries like pandas, numpy, matplotlib, and seaborn.
• The skill operates on data loaded into a Pandas DataFrame; ensure your data is properly formatted for analysis.
• The provided code examples use 'customer_data.csv' as an example file name - you will need to provide the correct filename when using this skill.