LearnByDoing With Steven

•

Chaîne • 4 émissions

Éducation

Éducation

Chaque semaine
Éducation

Éducation

Chaque semaine
Éducation

Éducation

Chaque semaine
Éducation

Éducation

Chaque semaine

28 juin

🚀 Stanford CS336 Lecture 13: The Evolution of Language Model Data. -Notebooklm Summary

Stanford CS336 Lecture 13 focuses on the critical evolution of language model training data. While model architectures are widely disclosed, dataset details remain highly proprietary due to commercial competition and copyright considerations. The lifecycle of language model training spans pre-training, mid-training, and post-training. The mid-training phase curates high-quality datasets to enhance specific capabilities like coding, mathematics, and long-context reasoning. Ultimately, data curation shifts from high-volume, low-quality web crawls to low-volume, high-quality specialized datasets. Data filtering methodologies have evolved from basic rule-based heuristics and language identification to advanced model-based approaches. Modern curation pipelines leverage large language models to assess the educational value of documents, rewrite low-quality texts, and synthesize high-quality QA pairs to scale data efficiently. Legal and compliance challenges, including copyright and fair use, remain central to data acquisition. As models risk memorizing training text, developers navigate the balance between direct commercial licensing and fair use arguments. Key Takeaways: Mid-training acts as a crucial bridge, refining models for targeted reasoning tasks.Advanced LLM-driven filtering and synthesis scale high-quality data while avoiding rule-based bias.Copyright compliance and memorization concerns limit public dataset disclosures.All my links: https://linktr.ee/learnbydoingwithsteven #learnbydoingwithsteven #AI #DeepLearning #Research #TechSummary #MachineLearning #LLM #DataScience #DataCuration #Copyright #ArtificialIntelligence

28 juin

•

7 min
18 juin

Stanford CS336 Language Modeling from Scratch Lecture 12 highlights - Evaluation Overview

Stanford CS336 Language Modeling from Scratch Lecture 12 Evaluation Overview Evaluating language models may seem as simple as measuring a specific model's performance, but it is actually fraught with challenges. The industry currently evaluates models through various metrics, such as benchmark scores like MMLU, cost-effectiveness indicators combining model accuracy and per-token cost, OpenRouter platform data based on user traffic routing, and Chatbot Arena which relies on human pairwise preference comparisons. However, an evaluation crisis currently exists, as some benchmarks may have reached saturation or been gamed, making it difficult to determine the most accurate evaluation method amidst a plethora of models and benchmark data. Key Takeaways: The fundamental purpose of evaluation depends on specific needs, and there is no single true evaluat...All my links: https://linktr.ee/learnbydoingwithsteven #learnbydoingwithsteven #AI #DeepLearning #Research #TechSummary #MachineLearning #LLM #ScalingLaws #NeuralNetworks #Innovation

18 juin

•

5 min
18 juin

Stanford University CS336 Lecture 11 highlights Application of Scaling Laws in Large Language Models and Maximal Update Parameterization

Stanford University CS336 Lecture 11 Application of Scaling Laws in Large Language Models and Maximal Update Parameterization This lecture explores how modern large language model builders use scaling laws as part of their model design process, and details case studies from relevant papers alongside the mathematical specifics of maximal update parameterization. Following the release of the Chinchilla model, due to intensified industry competition, many frontier labs stopped publicly sharing specific details regarding data and model scaling. However, some highly capable research teams have still openly shared their rigorous studies on scaling laws when executing large-scale model training. Key Takeaways: In the case of scaling strategies, the Cerebras GPT series applied the Chinchilla recipe across para...All my links: https://linktr.ee/learnbydoingwithsteven #learnbydoingwithsteven #AI #DeepLearning #Research #TechSummary #MachineLearning #LLM #ScalingLaws #NeuralNetworks #Innovation

18 juin

•

7 min
25 avr.

The future of engineering is no longer about writing code, but building the "harness" for AI.

The future of engineering is no longer about writing code, but building the "harness" for AI. 🚀 1️⃣ Harness Engineering: Redesigning the entire organization around the assumption that AI is the primary builder. 2️⃣ Monorepo Revival: Giving agents 100% context to eliminate architectural blind spots. 3️⃣ Self-healing: An automated closed loop from error triage to fix validation. 4️⃣ Efficiency Jump: Enabling 10-person teams to deliver at the scale of 100+. The edge in the AI era belongs to those who design the most efficient "control plane." full video in all my channels, youtube, substack, etc All my links: https://linktr.ee/learnbydoingwithsteven #learnbydoingwithsteven #AIEngineering #SystemDesign #HarnessEngineering #FutureOfWork #DevOps #AI #TechInnovation #SoftwareArchitecture #Productivity

25 avr.

•

6 min
31 janv.

Hydra in the Machine: The Evolution of Ransomware and AI Threats

This IBM Security Intelligence podcast explores the evolving intersection of artificial intelligence and modern cybersecurity threats. Experts discuss how ransomware groups have become more resilient and decentralized, increasingly utilizing AI agents to automate complex attack chains. The dialogue highlights persistent vulnerabilities in identity management, where a failure to implement multi-factor authentication or passkeys allows single actors to breach dozens of corporate clouds. Furthermore, the participants warn that improperly governed AI agents may become the next major insider threat due to their autonomous nature and susceptibility to prompt injection. The discussion concludes with a look at physical security risks, demonstrating how humanoid robots can be hijacked through simple voice commands. Ultimately, the sources emphasize that defenders must adopt AI-driven safeguards and Zero Trust principles to counter these sophisticated, high-speed digital and physical dangers.

31 janv.

•

8 min
11 avr.

Demystifying the Foundations of Neural Networks with MIT

Demystifying the Foundations of Neural Networks with MIT The revolution in artificial intelligence over the past decade has been nothing short of extraordinary. From solving insurmountable scientific challenges to hyper-realistic content generation, deep learning has moved from a computation-expensive luxury to a highly commoditized technology accessible to everyone. MIT 6.S191 (Lecture One) provides a foundational systematic review of the core intelligence hierarchy. By defining intelligence as information processing for decision-making, it clarifies the distinct roles of AI, Machine Learning, and Deep Learning. The growth we see today is driven by three pillars: massive datasets, efficient GPUs, and mature open-source software. Key takeaways include: The Layered Hierarchy: AI (Science) -> ML (Learning from Data) -> DL (Neural Networks).The Perceptron: The fundamental building block using dot products and non-linear activation.Training Mechanics: Using Loss Functions and Backpropagation for weight optimization.Generalization: Combatting overfitting with techniques like Dropout and early stopping.All my links: ⁠https://linktr.ee/learnbydoingwithsteven⁠ #DeepLearning #MIT #ArtificialIntelligence #MachineLearning #NeuralNetworks #LearnByDoingWithSteven #StevenDataTalk #AIResearch #ComputerScience #TechInnovation

11 avr.

•

7 min
1 févr.

Building Agentic Workflows with Langflow Visual Studio

Langflow serves as a versatile visual framework designed to help developers architect sophisticated AI workflows through a low-code, drag-and-drop interface. This open-source platform utilizes a node-based system that allows users to seamlessly connect various generative AI models, vector databases, and custom Python components. By remaining provider-agnostic, the tool supports a wide array of local and cloud-based services while offering native integration for the Model Context Protocol (MCP). Developers can instantly test agent logic within a built-in playground and subsequently deploy their creations into external applications via dedicated APIs. Ultimately, the software simplifies the creation of complex agentic systems by providing a clear graphical representation of intricate logic and data streams.

1 févr.

•

6 min
18 févr.

Navigating the Great Divide: China vs. US Venture Strategy

The provided text examines the structural divergence between the American and Chinese venture capital ecosystems as they enter 2026. While the United States maintains its lead in foundational AI and software through private capital, China has pivoted toward state-guided "hard tech" and industrial efficiency. This shift has introduced stark differences in legal protections, with Chinese investors often requiring personal guarantees from founders that are absent in Western contracts. Consequently, entrepreneurs are adopting decentralized corporate structures and seeking "middle-ground" capital in Singapore and the Middle East to navigate geopolitical tensions. Regulatory hurdles like CFIUS and data sovereignty laws are further fragmenting the market, forcing a dual-track approach to global fundraising and operations. Ultimately, the sources suggest that surviving this era requires a sophisticated understanding of these competing investment philosophies and their respective exit pathways.

18 févr.

•

18 min
17 févr.

Intelligent AI Delegation: A Framework for Agentic Coordination

The provided text explores intelligent delegation, defined as a structured framework for assigning complex tasks between humans and AI agents while maintaining accountability and trust. To ensure safety, the process involves task decomposition, where objectives are broken down into verifiable sub-tasks based on their difficulty and importance. The authors propose a dynamic coordination cycle that allows for real-time adjustments, such as switching assistants if performance drops or environmental conditions change. Success is supported by reputation systems and smart contracts that formalize agreements and automate penalties to ensure reliability in decentralized markets. Additionally, the research highlights the need for transitive monitoring and cryptographic verification to prevent security threats like data poisoning or unauthorized access. Ultimately, the framework aims to scale autonomous systems effectively by balancing operational efficiency with human oversight and ethical labor standards.

17 févr.

•

17 min
5 mai

Stanford CS336 Lecture 6: Mastering GPU Programming Models, Performance, and Triton Kernels

🚀 Deep dive into GPU architecture! Just summarized Stanford CS336 Lecture 6: Mastering GPU Programming Models, Performance, and Triton Kernels. ⚡️ Key takeaways: Memory hierarchy: Registers > Shared Memory > HBM Kernel Fusion to beat the Memory Wall Tiling strategies for MatMul Why Triton is a game-changer for custom kernels Full video in my channels. linktr.ee/learnbydoingwithsteven #GPUProgramming #TritonKernels #StanfordCS336 #DeepLearning #CUDA #PerformanceOptimization #LanguageModeling #LearnByDoingWithSteven #StevenDataTalk #AIInfrastructure #LLM

5 mai

•

10 min

Learn By Doing With Steven 数能生智