Spark Sql Optimizer
Optimizes Spark SQL queries for performance by analyzing and suggesting improvements to execution plans – boosting query speed and efficiency.
Install on your platform
We auto-selected Claude Code based on this skill’s supported platforms.
Run in terminal (recommended)
claude mcp add spark-sql-optimizer npx -- -y @trustedskills/spark-sql-optimizer
Or manually add to ~/.claude/settings.json
{
"mcpServers": {
"spark-sql-optimizer": {
"command": "npx",
"args": [
"-y",
"@trustedskills/spark-sql-optimizer"
]
}
}
}Requires Claude Code (claude CLI). Run claude --version to verify your install.
About This Skill
What it does
This skill optimizes Spark SQL queries to improve performance and efficiency. It analyzes query plans, identifies bottlenecks, and suggests or automatically applies transformations like predicate pushdown, join reordering, and broadcast joins. The goal is to reduce execution time and resource consumption for data processing tasks within a Spark environment.
When to use it
- Slow Query Performance: When existing Spark SQL queries are taking an unacceptably long time to complete.
- Resource Constraints: When running Spark jobs on limited hardware resources (e.g., smaller clusters).
- Complex Joins: When dealing with complex join operations that significantly impact query execution time.
- Large Datasets: When processing large datasets where even small optimizations can yield substantial performance gains.
Key capabilities
- Query plan analysis
- Predicate pushdown optimization
- Join reordering
- Broadcast join implementation
- Automatic query transformation suggestions
Example prompts
- "Optimize this Spark SQL query:
SELECT * FROM table1 JOIN table2 ON table1.id = table2.id" - "Analyze the execution plan for my Spark SQL job and suggest improvements."
- "Can you rewrite this query to use a broadcast join?
SELECT ... FROM large_table JOIN small_table ON ..."
Tips & gotchas
- Requires access to a running Spark environment. The agent needs permissions to analyze and potentially modify queries within the cluster.
Tags
TrustedSkills Verification
Unlike other registries that point to live repositories, TrustedSkills pins every skill to a verified commit hash. This protects you from malicious updates — what you install today is exactly what was reviewed and verified.
Security Audits
| Gen Agent Trust Hub | Pass |
| Socket | Pass |
| Snyk | Pass |
🌐 Community
Passed automated security scans.