Databricks 2025

The databricks-2025 skill provides AI agents with direct access to the Databricks ecosystem, enabling seamless interaction with data engineering and machine learning workflows. It allows agents to execute SQL queries, manage notebooks, and orchestrate jobs within a Databricks workspace without manual API intervention.

When to use it

• Automating routine data transformation tasks in Delta Lake tables.
• Triggering scheduled ML training pipelines based on data availability.
• Generating and executing SQL queries for ad-hoc data exploration.
• Managing cluster lifecycles to optimize resource usage and costs.

Key capabilities

• Execute SQL statements against Databricks SQL warehouses.
• Create, edit, and run Jupyter notebooks programmatically.
• Trigger and monitor Databricks Jobs and Workflows.
• Interact with Unity Catalog for metadata and governance tasks.
• Manage cluster configurations and scaling policies.

Example prompts

• "Create a new notebook in the analytics workspace that calculates daily sales totals from the orders table."
• "Run the scheduled job named 'model-retrain-daily' and report its current status."
• "Execute a SQL query to find the top 10 customers by revenue for Q4 2025."

Tips & gotchas

Ensure your AI agent has valid authentication credentials (e.g., service principals or personal access tokens) with appropriate permissions before attempting to execute commands. Be mindful of workspace quotas and cluster costs when instructing the agent to spin up new resources for testing.