Semi Doped

Vikram Sekar and Austin Lyons

The business and technology of semiconductors. Alpha for engineers and investors alike.

  1. 2 dgn geleden

    Datacenter Interconnects: Copper vs. Optics, Nvidia's 78-Layer PCB, Co-Packaged Optics (CPO)

    Vik Sekar and Austin Lyons tackle the biggest bottleneck in inference: moving data. They break down the three tiers of datacenter networking — scale up, scale out, scale across — and the core engineering trade-off at each layer: copper vs. optics. Topics include Nvidia's extreme measures to keep scale-up fabric electrical (a 78-layer mid-plane PCB), why Co-Packaged Optics is the "holy grail" everyone wants and no one can ship, and the serviceability problem standing in its way. Key Takeaways: A single 72-GPU scale-up rack needs 5,000+ cables spanning ~2 km — at that density, the power and cost of every connection becomes a system-level constraint, not an implementation detail.The scale-up rule is "copper when you can, optics when you must": each pluggable optical module adds ~30W, and with thousands of links in the fabric, that penalty compounds fast.Nvidia is pushing copper past its usual limits with a 78-layer mid-plane PCB — 3x+ the layer count of a typical complex board — specifically to avoid paying the optics power tax in its scale-up fabric.Scale-up isn't just a training problem. Frontier MoE models need 72-GPU domains to hit memory bandwidth targets, which pulls high-performance interconnect into the inference conversation.Scale-up has the highest connection density of the three tiers, making it the largest TAM and the sharpest three-way fight between NVLink, UA-Link, and Ethernet.Co-Packaged Optics could cut interconnect power by two-thirds — but a single failed laser could brick an entire multi-thousand-dollar GPU package, and that serviceability risk is what's keeping it on the roadmap instead of in racks.Chapters: 0:00 The Biggest Problem in Computing 7:14 The Three Tiers of Networking 13:49 Scale Up: Copper vs. Optics 17:59 Front-End vs. Back-End Networks 22:12 The Physical Scale of Cabling 28:46 Nvidia's 78-Layer Mid-Plane 33:16 How Optical Transceivers Work 39:16 The Power Penalty of Pluggables 42:09 The Business of Speed Transitions 44:40 The Promise and Peril of CPO 51:45 The Holy Grail of Networking Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/austinsemis Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/

    Datacenter Interconnects: Copper vs. Optics, Nvidia's 78-Layer PCB, Co-Packaged Optics (CPO)

  2. 16 jul

    PicoJool's Al Yuen: The Case for GaAs VCSELs in Scale-Up Interconnects

    Al Yuen, CEO of PicoJool, talks with Austin about using VCSELs for scale-up optical interconnects in AI data centers. Al explains why gallium arsenide (GaAs) supply is unconstrained while indium phosphide (InP) is limited, and how PicoJool can leverage existing supply chains to ship in the millions per month. They cover the roadmap from 1.6T to 12.8T, the trade-offs among 8×200G, 16×100G, and 32×50G, and the push to train the next generation of photonics engineers. The spec that changed is error rate. AI needs error-free links (10⁻¹⁰–10⁻¹² vs Ethernet's old 10⁻⁶) because GPUs act as one system — one error stalls the whole job.Single-mode optics can't hit the volume. Built for ~100K city-to-city links; datacenters need millions/month — a 10–50× gapGaAs vs InP is the whole supply story. VCSELs run on unconstrained GaAs; single-mode on constrained InP. 1M units: 8–10 weeks vs a sold-out 8–18 months.One platform, three flavors to 1.6T: 8×200G "fast and narrow," 16×100G LPO low-power, 32×50G NRZ "slow and wide" — pick on power, cost, and BER.12.8T needs no new tech — just more of the same: 64 channels (4×16 array in a finger-sized connector) × 200G, scaling channels, bi-di wavelengths, and lane speed.Capacity isn't the time consuming part — qualification is. WIN runs ~1,000 wafers/week at ~240K VCSELs each, so 1M units = ~10 wafers. The wait is tier-1 (>6 mo) and tier-2 (~3 mo) quals. Chapters: 0:00 Meet Al Yuen and PicoJool 2:29 Inventing the active optical cable 5:03 Engineering mindset, copper limits 8:43 Why VCSELs 13:45 Scale-up and bit error rate 20:09 Unconstrained vs constrained supply 21:53 Indium phosphide bottleneck 25:49 VCSEL design and foundry handoff 31:59 Product road map, 200G launch 34:16 Path to 3.2T and 12.8T 40:10 Ordering a million VCSELs 45:10 Ramp timing and training new engineers Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free! Sign up: https://daily.semidoped.com/

  3. 10 jul

    WEKA's Val Bercovici: KV Cache, DeepSeek V4, HBF, SLC vs QLC NAND, CXL, NVLink, Tokenomics

    Vik welcomes Val Bercovici from Weka to discuss the rapidly evolving landscape of AI memory and storage. Val explains how Weka's architecture leverages high-bandwidth networks to make storage faster than motherboard DRAM. They dive into KV cache optimizations, the future of NAND flash tiers, and the role of CXL in AI inference. The episode concludes with a look at predictive memory offloading and the AI flywheel. Chapters: 0:00 Welcome Val Bercovici, Weka 1:59 Memory situation and model routing 3:50 KV cache offloading to CMX 6:10 Network faster than motherboard 13:10 Weka as AI memory infrastructure 14:45 Inference market is different 16:06 Memory hierarchy and KV cache 19:40 KV cache optimizations and demand 25:20 DeepSeek's cache read pricing 34:49 NAND flash tiers: SLC vs QLC 43:01 High Bandwidth Flash (HBF) 49:59 CXL versus other interconnects Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free! Sign up: https://daily.semidoped.com/

  4. 3 jul

    Micron's Record Profits, Apple's CXMT Plea: AI is Eating All the Memory

    Austin and Vik break down the current memory crisis, where AI demand is driving unprecedented profits for memory makers like Micron, while simultaneously causing inflation and price hikes for consumer electronics. They discuss how even premium brands like Apple are struggling to secure supply, leading them to controversial suppliers. The hosts also explore the long-term implications of AI's insatiable memory appetite and its impact on both data center and consumer markets. Chapters: 0:00 Memory crisis hits 1:20 AI impacting consumers 3:00 AI causing inflation 6:48 Consumer demand drop? 8:49 AI demand inelastic 10:56 Long-term memory outlook 11:02 GoPro's memory woes 12:24 Apple's pricing power 18:20 Apple seeks CXMT DRAM 21:38 Shrinkflation for phones 23:16 Korea's memory investment 26:10 Micron's killing profits 33:00 Why AI needs so much DRAM 40:27 Future of AI training 44:34 Cost-optimizing inference Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free! Sign up: https://www.semidoped.com/

  5. 29 jun

    Qualcomm's HBC Memory, Alphawave, Modular, and more

    Qualcomm flew Austin to New York for its investor day, where the communications company laid out a plan to make data center, automotive, and IoT two-thirds of its business by FY29. Austin was in the room and asked Cristiano Amon a question on the record. Vik watched the whole thing on YouTube. Together they break down what actually changed. The technical centerpiece is High Bandwidth Compute (HBC): stacking LPDDR on top of logic to expose the whole face of the chip for interconnects, claiming up to 100x more lanes and a path around the HBM bandwidth bottleneck. Austin and Vik dig into what's really under that memory, why "no advanced packaging needed" just moves the hard problem somewhere worse, and how it compares to d-Matrix. And as Austin put it: don't let anyone tell you HBM is dead. MOAR memory. Then the roadmap: the AI200/250/300 accelerators, the C1000 server CPU (5 GHz, 250+ cores, Meta as a customer), the Alphawave and Modular acquisitions, and Chris Lattner's Mojo. The most interesting takeaway might not be the data center at all. Qualcomm's edge play — AI-defined vehicles, cars as token generators, and a $1T robotics opportunity by 2040 — could be where High Bandwidth Compute matters most.  Chapters:  0:00 Communications? That's just the start  4:08 Inside Qualcomm's investor day  9:16 Can Qualcomm build a data center business?  13:09 Disaggregated inference opens the door  17:57 High Bandwidth Compute: memory on the XPU  30:29 "No advanced packaging" just moves the problem  36:20 The roadmap, Alphawave, and Modular  46:00 The C1000 CPU and the agentic shortage  50:40 Cars as token generators, the $1T robotics bet  57:32 The memory market: MOAR Follow Semi Doped:  Get more of Austin and Vik daily, free!  Sign up: https://www.semidoped.com/ Connect with Vik and Austin:  Vik's Paid Substack: https://www.viksnewsletter.com Austin's Paid Substack: https://www.chipstrat.com

  6. 19 jun

    Advanced Packaging, TSMC CoWoS, Intel EMIB

    New episode: Advanced packaging for AI chips, from wire bonds to TSMC CoWOS and Intel EMIB. Packaging is no longer an afterthought. It is the chip, and Intel's EMIB challenges TSMC's CoWOS. Three CoWOS flavors: silicon, organic RDL, local bridgesEMIB embeds tiny bridges into the substrate, no interposerEMIB-T and EMIB-M add through-silicon vias and power capacitorsGoogle is booking 3M TPUs on EMIB via MediaTek by 2028Package sizes keep climbing: 5.5x reticle today, 40x aheadThis episode is brought to you by SambaNova. Try SambaNova's fast inference today at the SambaNova Dashboard! Connect with Vik and Austin via a daily free newsletter: https://www.semidoped.com Vik's Paid Substack: https://www.viksnewsletter.com Austin's Paid Substack: https://www.chipstrat.com Chapters: (0:00) "There Is No Chip Without the Packaging" (0:28) Intro and SpaceX IPO Day (5:15) What We're Covering: CoWOS, EMIB, Google (7:40) Simple Packaging: Wire Bonds to Flip Chip (17:07) What Makes Packaging "Advanced" (33:44) CoWOS: Three Flavors Explained (45:30) EMIB: Intel's Embedded Bridge Approach (52:47) EMIB-T and EMIB-M (57:31) CoWOS vs. EMIB Trade-offs (1:02:18) Google's 3M TPU EMIB Order

  7. 12 jun

    Computex Mania 2026: Optics and Power

    Austin and Vik discuss their recent experience at Computex, where they met for the first time in person after six months of podcasting together. They share insights about the massive show, the people they connected with, and the exciting developments in AI hardware and interconnect technology. Connect with Vik and Austin via a daily free newsletter: https://www.semidoped.com Vik's Paid Substack: https://www.viksnewsletter.com Austin's Paid Substack: https://www.chipstrat.com Chapters 00:00 Meeting in Person for the First Time 03:05 Experiencing Computex: A Massive Show 05:17 Connecting with the Audience: Real-Life Encounters 06:46 Networking with Industry Leaders 10:42 Keynote Highlights: Marvell's Vision 15:11 The Future of Interconnects: CPO and Beyond 22:54 Exploring Optical Interconnects and Future Technologies 25:56 Micro LED Developments and Future Conferences 27:34 Power Innovations in Data Centers 30:54 Intel's Keynote and New CPU Technologies 36:31 Intel Foundry's Advancements and Industry Implications

    Computex Mania 2026: Optics and Power

  8. 29 mei

    Huawei's Tau Scaling Law: Is the "EUV Killer" Real?

    Huawei dropped a paper claiming 1.4nm-class performance without EUV, and the internet immediately declared ASML dead and US export controls useless. Austin and Vik recorded one day after Memorial Day to unpack what Huawei actually announced at ISCAS 2026 — and why the "EUV killer" headline gets the story backwards. They walk through the tau scaling law (tau is delay, and the idea is to attack it at the system level instead of the transistor), logic folding via hybrid bonding, the Kirin 2026 that doubles transistor count without shrinking, and who can actually manufacture stacked logic. Then the other tau knobs: a unified memory bus and near-packaged optics. Along the way: BESI vs EV Group, die-to-wafer vs wafer-to-wafer bonding, and why hybrid bonding isn't export-controlled the way EUV is. The takeaway is the opposite of the headline. Tau scaling is rational engineering under constraint, it's bullish for ASML (two DUV wafers per product, not fewer), and the moment EUV-enabled fabs stack their own advanced-node wafers, the gap widens instead of narrowing. Bullish advanced packaging, bullish EDA and multiphysics. Chapters: 0:00 The "EUV killer" paper that broke the internet 2:28 What Huawei actually announced at ISCAS 4:00 Tau scaling: optimize delay, not transistors 8:58 The equation and the 10x AI claim 11:05 Logic folding: stacking logic on logic 17:24 Who builds it, and can hybrid bonding be banned? 24:16 Why this is bullish for ASML 29:49 The other tau knobs: memory and optics 35:18 Takeaways: packaging, EDA, multiphysics Follow Semi Doped: Get more of Austin and Vik daily, free! Sign up: https://www.semidoped.com/ Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com X: https://x.com/vikramskr

    Huawei's Tau Scaling Law: Is the "EUV Killer" Real?
Bekijk alles (38)

Info

The business and technology of semiconductors. Alpha for engineers and investors alike.

Informatie

  • Maker
    Vikram Sekar and Austin Lyons
  • Jaren actief
    2026
  • Afleveringen
    38
  • Beoordeling
    Veilig
  • Copyright
    © 2026 Semi Doped
  • Website van aflevering
    Semi Doped
  • Provider

Suggesties voor jou