October 2021 dbt Update: Metrics and Hat Tricks 🎩

Hello there,

While I have a lot of fun things to share this month, I can't start with anything other than this:

Yep, it's official:

💥dbt will support metric definitions💥

With this feature, you'll be able to centrally define rules for aggregating metrics (think, "active users" or "MRR") in version controlled, tested, documented dbt project code.

We still have a ways to go, but in future, you'll be able to explore these metrics in the BI and analytics tools that you know and love.

Jeremy (dbt PM) will share more about the metrics layer in his v1.0 reveal at Coalesce.

While this topic plays a significant supporting role in Drew's keynote, it's not the whole story🍿. Drew's talk will tie together several threads across the industry right now, and introduce some very exciting futures for dbt and the community as a whole.

You really don't want to miss this one - register for free here.

What's New

---

I've got three really exciting things to share this month!

dbt v0.21: 

• Check out the #dbt-releases channel in the dbt Community Slack for full detail!

• dbt build is here! 🙌 This command executes everything you'd want to do in the DAG, in order, and does it with attitude opinions: Run models, test tests, snapshot snapshots and seed seeds while prioritizing quality and resiliency. Reduce several steps to a single command and bring best practices along for the ride 🚗

v1.0 beta: 

• Check out the #dbt-prereleases channel in the dbt Community Slack**, and Jeremy's Discourse post!*

• v1.0 is a huge milestone with all the trimmings, including 100x faster project parsing compared to v0.19.0 ⚡. We're excited to celebrate with you during Jeremy's session at Coalesce, but until then, we hope you give the beta a spin! And don't forget to join the #dbt-v1-readiness channel in Slack.

dbt Cloud v1.1.36 - v1.1.37

• Changelog + docs located here.

• Model bottlenecks beta: Identify long-running models ripe for refactoring (or re-scheduling). The new model timing dashboard in the run detail page helps you quickly assess job composition, order, and duration to optimize your workflows and cut costs💰

The Model Timing tab in dbt Cloud highlights models taking particularly long to run.

New Resources 

---

Things to Try 🛠️

• Nearly 500 dbt Cloud accounts are using CI. Want to know why? (or maybe... how?) Julia breaks it down in her latest blog and shares how to choose and configure continuous delivery or continuous deployment at your organization.
• Hex just launched an integration with dbt! It uses the dbt Cloud Metadata API to surface metadata from dbt right in Hex, letting you quickly get the context you need on things like data freshness without juggling multiple apps and browser tabs. Get started here.
• The dbt-Rockset adapter (now in beta) just received a major update. It now supports View, Table, Incremental, and Ephemeral materializations to help you perform real-time data transformations on Rockset. Read more here..

Things to Read 📚

• Everyone is talking about the next layer of the modern data stack. It's not a new conversation, but it is starting to heat up. Anna (dbt Labs Director of Community), does a phenomenal job connecting this week's events in the latest issue of the Analytics Engineering Roundup.

Things to Watch 📺

• Infrastructure-as-Code and the Modern Data Experience

  At the Future Data Conference last week Tristan noted that data workflows borrow much from software engineering, but haven't really crossed the DevOps chasm. What's missing? Spreadsheets? Actually... maybe. 😅 Okay you had to be there. Luckily you still can! Check out the recording.

• Modeling behavioral data with Snowplow and dbt (coming up on 10/27). Our own Sanjana Sen joins the Snowplow team to talk modeling Snowplow event data in dbt -- including how to structure your data models, best practices to follow, and key pitfalls to avoid.

• How Blend Eliminated Data Silos with dbt and Hightouch. Fin-tech behemoth, Blend, processes trillions of dollars in loans (and recently IPO'd). Join this talk with William Tsu (Customer Success Operations at Blend) to learn how adopting dbt and Hightouch has helped them overcome data silos to keep kicking a$$.

That's all for now! Thanks for reading, and as always, let me know if there's anything else you want to see in these updates!

Lauren Craigie
Director of Product Marketing, dbt Labs