Dbt Data Transformation

What it does

This skill enables AI agents to automate and manage dbt (data build tool) projects for analytics engineering. It covers a wide range of tasks including model development, testing, documentation generation, incremental builds, macro creation, package management, and deployment workflows. Essentially, the agent can assist in building and maintaining data transformation pipelines using dbt's software engineering best practices.

When to use it

• Building data transformation pipelines for analytics and business intelligence.
• Creating a data warehouse with modular, testable SQL transformations.
• Implementing ELT (Extract, Load, Transform) workflows.
• Developing dimensional models (facts, dimensions).
• Managing complex SQL dependencies and data lineage.

Key capabilities

• Model Development: Writing SQL SELECT statements as dbt models using CTEs and referencing sources with {{ source() }}.
• Testing: Defining and executing data quality tests within the dbt framework.
• Documentation: Generating searchable data dictionaries automatically.
• Macro Creation: Developing reusable Jinja-SQL functions (macros).
• Package Management: Handling dependencies and managing dbt packages.
• Deployment: Supporting CI/CD pipelines for deploying data transformations.
• Incremental Builds: Creating incremental models for large datasets.

Example prompts

• "Create a dbt model to transform the 'orders' table from the 'jaffle_shop' source."
• "Define a data quality test for the 'stg_orders' model."
• "Generate documentation for all dbt models in my project."

Tips & gotchas

• This skill requires familiarity with dbt concepts and terminology.
• Models are defined using only SELECT statements; no Data Definition Language (DDL) is supported within the models themselves.
• The agent will leverage Jinja templating for SQL transformation logic.