Agile Data N’ Info

AgileData

Join the Agile Data N' Info Podcast where we discuss how data and analytics teams can apply agile, product and data patterns to change the way they work.

  1. 4D AGO

    Data Match, AgileData Engineering Pattern #7 - Episode #75

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the Data Match.   The Data Match pattern provides an automated, granular comparison capability to efficiently identify and report discrepancies between two datasets, moving from row counts to specific data values. This 'data diff' solution transforms hours of manual data reconciliation into minutes by optimising comparisons for cloud analytics databases like BigQuery, serving as a support feature for on-demand exception handling rather than a continuous trust rule.   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.   If you want a copy of the pattern template head over to: https://agiledata.substack.com/i/172820886/pattern-name Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    10 min

  2. 4D AGO

    Patterns to define the ROI of a data product with Nick Zervoudis - Episode #74

    Join Shane Gibson as he chats with Nick Zervoudis about patterns that you can use to quickly and easily define the ROI of your data products, before you build them. You can get in touch with Nick via LinkedIn or over at https://blog.valuefromdata.ai If you want want to read a summary generated with GenAI, head over to: https://agiledata.substack.com/i/172748370/google-notebooklm-briefing   If you want to download the transcript for the podcast, head over to: https://agiledata.substack.com/i/172748370/transcript   Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/ Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    1h 7m

  3. AUG 18

    Dimensional Data Modeling Patterns with Johnny Winter - Episode #73

    Join Shane Gibson as he chats with Johnny Winter about the core patterns that make up Dimensional (Star Schema) Modeling. You can get in touch with Johnny via LinkedIn or over at https://greyskullanalytics.com/ If you want want to read a summary generated with GenAI, head over to: https://agiledata.substack.com/i/171239242/google-notebooklm-briefing   If you want to download the transcript for the podcast, head over to: https://agiledata.substack.com/i/171239242/transcript     Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/ Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    1h 9m

  4. AUG 7

    Activity Event Tile, AgileData Engineering Pattern #6 - Episode #72

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Activity Event Tile.   The Activity Event Tile pattern tracks a sequence of events (e.g., subscription signups, subscription payments) for a core business concept by storing minimal event data and using a hydration layer to automatically generate pre-calculated metrics as views. This allows business users and analysts to easily understand business performance and event sequencing without complex SQL, providing immediate, consistent insights into activity movements and financial impacts   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.   If you want a copy of the pattern template head over to: https://agiledata.substack.com/i/170387919/pattern-name Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    12 min

  5. JUL 30

    Data Engineering Patterns for the AgileData Event Tile, AgileData Engineering Pattern #5 - Episode #71

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Event Tile.   The Event Tile Data Modeling pattern captures business processes and transactions within AgileData's three-layered data architecture, modeling "who does what" at a specific point in time. It achieves this by storing only a small number of core business keys in insert-only event tables. These event records are then automatically hydrated into a "consume tile" by joining with all relevant detail attributes that were current at the exact moment the event occurred, ensuring an accurate, time-staged narrative. This provides data consumers with a rich, wide, and historically accurate view of events, simplifying the querying of complex historical changes and state transitions.   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.   If you want a copy of the pattern template head over to: https://agiledata.substack.com/i/169697974/pattern-name Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    16 min

  6. JUL 22

    Automated Load Patterns based on Source Data Profiles, AgileData Engineering Pattern #4 - Episode #70

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Automated Load Patterns based on Source Data Profiles.   The Automated Load Patterns based on Source Data Profiles pattern automatically profiles incoming data to determine its optimal loading method. It classifies data as either event data (immutable, append-only) or change data (evolving records) by analysing characteristics like unique key volume and column names over time. This automated classification then dictates the load strategy: an efficient partition replace for event data or a cost-effective upsert (end-dating historical records) for change data. This capability removes the manual burden and accelerates data onboarding, providing a trustworthy and efficient process even when the data type is initially unknown.   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.   If you want a copy of the pattern template head over to: https://agiledata.substack.com/i/168912285/pattern-name Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns     If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    12 min

  7. JUL 13

    Trust Rules to validate your data, AgileData Engineering Pattern #3 - Episode #69

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Trust Rules to validate data.   The Trust Rules pattern provides automated data validation to ensure all incoming data is fit for purpose and trustworthy. It bakes in essential checks such as unique business keys and business effective dates, which run automatically upon data load or table refresh. Users can also define custom validation rules for specific columns. Results are collected, persisted, and surfaced via applications or alerts, with the system optimising validation for cost and speed through smart partitioning and clustered columns.   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.   If you want a copy of the pattern template head over to: https://agiledata.substack.com/i/167964917/pattern-name Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns     If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    12 min

  8. JUL 3

    Union two or more tables together automatically, AgileData Engineering Pattern #2 - Episode #68

    Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Unioning two or more tables together automatically.   The Automated Table Unioning pattern automatically combines two or more tables by intelligently looking up column names and data types to generate safe SQL under the covers. It supports disparate data sources, such as those from multiple publishers in a data clean room, and creates a view or incrementally loads the unified data into a physical table while tracking load watermarks to prevent duplicates.   An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.     If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |  Buy the Green Book now!   Simply Magical Data Ways of Working

    11 min
See All (75)

About

Join the Agile Data N' Info Podcast where we discuss how data and analytics teams can apply agile, product and data patterns to change the way they work.

Information

  • Creator
    AgileData
  • Years Active
    2019 - 2025
  • Episodes
    75
  • Rating
    Clean
  • Copyright
    © Copyright 2024 All rights reserved.
  • Show Website
    Agile Data N’ Info

You Might Also Like