Data Science Decoded

Mike E
  • 3,8 (5)
  • TOÁN HỌC
  • HẰNG TUẦN

We discuss seminal mathematical papers (sometimes really old 😎 ) that have shaped and established the fields of machine learning and data science as we know them today. The goal of the podcast is to introduce you to the evolution of these fields from a mathematical and slightly philosophical perspective. We will discuss the contribution of these papers, not just from pure a math aspect but also how they influenced the discourse in the field, which areas were opened up as a result, and so on. Our podcast episodes are also available on our youtube: https://youtu.be/wThcXx_vXjQ?si=vnMfs

  1. 6 GIỜ TRƯỚC

    Data Science #34 - The deep learning original paper review, Hinton, Rumelhard & Williams (1985)

    On the 34th episode, we review the 1986 paper, "Learning representations by back-propagating errors" , which was pivotal because it provided a clear, generalized framework for training neural networks with internal 'hidden' units. The core of the procedure, back-propagation, repeatedly adjusts the weights of connections in the network to minimize the error between the actual and desired output vectors. Crucially, this process forces the hidden units, whose desired states aren't specified, to develop distributed internal representations of the task domain's important features.This capability to construct useful new features distinguishes back-propagation from earlier, simpler methods like the perceptron-convergence procedure. The authors demonstrate its power on non-trivial problems, such as detecting mirror symmetry in an input vector and storing information about isomorphic family trees. By showing how the network generalizes correctly from one family tree to its Italian equivalent, the paper illustrated the algorithm's ability to capture the underlying structure of the task domain.Despite recognizing that the procedure was not guaranteed to find a global minimum due to local minima in the error-surface , the paper's clear formulation (using equations 1-9 ) and its successful demonstration of learning complex, non-linear representations served as a powerful catalyst. It fundamentally advanced the field of connectionism and became the standard, foundational algorithm used today to train multi-layered networks, or deep learning models, despite the earlier, lesser-known work by Werbos

    47 phút

  2. 3 THG 11

    Data Science #33 - The Backpropagation method, Paul Werbos (1980)

    On the 33rd episdoe we review Paul Werbos’s “Applications of Advances in Nonlinear Sensitivity Analysis” which presents efficient methods for computing derivatives in nonlinear systems, drastically reducing computational costs for large-scale models. Werbos, Paul J. "Applications of advances in nonlinear sensitivity analysis." System Modeling and Optimization: Proceedings of the 10th IFIP Conference New York City, USA, August 31–September 4, 1981 These methods, especially the backward differentiation technique, enable better sensitivity analysis, optimization, and stochastic modeling across economics, engineering, and artificial intelligence. The paper also introduces Generalized Dynamic Heuristic Programming (GDHP) for adaptive decision-making in uncertain environments.Its importance to modern data science lies in laying the foundation for backpropagation, the core algorithm behind training neural networks. Werbos’s work bridged traditional optimization and today’s AI, influencing machine learning, reinforcement learning, and data-driven modeling.

    58 phút

  3. 19 THG 9

    Data Science #32 - A Markovian Decision Process, Richard Bellman (1957)

    We reviewed Richard Bellman’s “A Markovian Decision Process” (1957), which introduced a mathematical framework for sequential decision-making under uncertainty. By connecting recurrence relations to Markov processes, Bellman showed how current choices shape future outcomes and formalized the principle of optimality, laying the groundwork for dynamic programming and the Bellman equationThis paper is directly relevant to reinforcement learning and modern AI: it defines the structure of Markov Decision Processes (MDPs), which underpin algorithms like value iteration, policy iteration, and Q-learning. From robotics to large-scale systems like AlphaGo, nearly all of RL traces back to the foundations Bellman set in 1957

    46 phút

  4. 26 THG 7

    Data Science #31 - Correlation and causation (1921), Wright Sewall

    On the 31st episode of the podcast, we add Liron to the team, we review a gem from 1921, where Sewall Wright introduced path analysis, mapping hypothesized causal arrows into simple diagrams and proving that any sample correlation can be written as the sum of products of “path coefficients.” By treating each arrow as a standardised regression weight, he showed how to split the variance of an outcome into direct, indirect, and joint pieces, then solve for unknown paths from an ordinary correlation matrix—turning the slogan “correlation ≠ causation” into a workable calculus for observational data.Wright’s algebra and diagrams became the blueprint for modern graphical causal models, structural‑equation modelling, and DAG‑based inference that power libraries such as DoWhy, Pyro and CausalNex. The same logic underlies feature‑importance decompositions, counterfactual A/B testing, fairness audits, and explainable‑AI tooling, making a century‑old livestock‑breeding study a foundation stone of present‑day data‑science and AI practice.

    48 phút

  5. 30 THG 5

    Data Science #30 - The Bootstrap Method (1977)

    In the 30th episode we review the the bootstrap, method which was introduced by Bradley Efron in 1979, is a non-parametric resampling technique that approximates a statistic’s sampling distribution by repeatedly drawing with replacement from the observed data, allowing estimation of standard errors, confidence intervals, and bias without relying on strong distributional assumptions. Its ability to quantify uncertainty cheaply and flexibly underlies many staples of modern data science and AI, powering model evaluation and feature stability analysis, inspiring ensemble methods like bagging and random forests, and informing uncertainty calibration for deep-learning predictions—thereby making contemporary models more reliable and robust.Efron, B. "Bootstrap methods: Another look at the bootstrap." The Annals of Statistics 7 (1977): 1-26.

    41 phút

  6. 23 THG 5

    Data Science #29 - The Chi-square automatic interaction detection(CHAID) algorithm (1979)

    In the 29th episode, we go over the 1979 paper by Gordon Vivian Kass that introduced the CHAID algorithm.CHAID (Chi-squared Automatic Interaction Detection) is a tree-based partitioning method introduced by G. V. Kass for exploring large categorical data sets by iteratively splitting records into mutually exclusive, exhaustive subsets based on the most statistically significant predictors rather than maximal explanatory power. Unlike its predecessor, AID, CHAID embeds each split in a chi-squared significance test (with Bonferroni‐corrected thresholds), allows multi-way divisions, and handles missing or “floating” categories gracefully.In practice, CHAID proceeds by merging predictor categories that are least distinguishable (stepwise grouping) and then testing whether any compound categories merit a further split, ensuring parsimonious, stable groupings without overfitting. Through its significance‐driven, multi-way splitting and built-in bias correction against predictors with many levels, CHAID yields intuitive decision trees that highlight the strongest associations in high-dimensional categorical data In modern data science, CHAID’s core ideas underpin contemporary decision‐tree algorithms (e.g., CART, C4.5) and ensemble methods like random forests, where statistical rigor in splitting criteria and robust handling of missing data remain critical. Its emphasis on automated, hypothesis‐driven partitioning has influenced automated feature selection, interpretable machine learning, and scalable analytics workflows that transform raw categorical variables into actionable insights.

    41 phút

  7. 23 THG 5

    Data Science #28 - The Bloom filter algorithm

    In the 28th episode, we go over Burton Bloom's Bloom filter from 1970, a groundbreaking data structure that enables fast, space-efficient set membership checks by allowing a small, controllable rate of false positives.Unlike traditional methods that store full data, Bloom filters use a compact bit array and multiple hash functions, trading exactness for speed and memory savings. This idea transformed modern data science and big data systems, powering tools like Apache Spark, Cassandra, and Kafka, where fast filtering and memory efficiency are critical for performance at scale.

    39 phút

  8. 2 THG 4

    Data Science #27 - The History of Least Squares (1877)

    Mansfield Merriman's 1877 paper traces the historical development of the Method of Least Squares, crediting Legendre (1805) for introducing the method, Adrain (1808) for the first formal probabilistic proof, and Gauss (1809) for linking it to the normal distribution. He evaluates multiple proofs, including Laplace’s (1810) general probability-based derivation, and highlights later refinements by various mathematicians. The paper underscores the method’s fundamental role in statistical estimation, probability theory, and error minimization, solidifying its place in scientific and engineering applications.

    32 phút
Xem tất cả (34)

Xếp Hạng & Nhận Xét

3,8
/5
5 Xếp hạng

Giới Thiệu

We discuss seminal mathematical papers (sometimes really old 😎 ) that have shaped and established the fields of machine learning and data science as we know them today. The goal of the podcast is to introduce you to the evolution of these fields from a mathematical and slightly philosophical perspective. We will discuss the contribution of these papers, not just from pure a math aspect but also how they influenced the discourse in the field, which areas were opened up as a result, and so on. Our podcast episodes are also available on our youtube: https://youtu.be/wThcXx_vXjQ?si=vnMfs

Thông Tin

  • Nhà sáng tạo
    Mike E
  • Năm hoạt động
    2024 - 2025
  • Tập
    34
  • Xếp hạng
    Sạch
  • Bản quyền
    © Mike E
  • Trang web chương trình
    Data Science Decoded

Có Thể Bạn Cũng Thích