Program-as-Weights for Compiling Fuzzy Functions

This episode explores Program-as-Weights, a paper that asks whether a natural-language description of a fuzzy software task can be compiled once into a small reusable neural artifact instead of sending every request to a larger remote model. It explains the paper’s architecture in concrete terms: a 4B pseudo-compiler rewrites the task and generates example I/O pairs, a trained 4B compiler plus LoRA mapper turns that specification into adapter weights, and a frozen Qwen3-0.6B interpreter runs the task locally on new inputs. The discussion focuses on why this matters for real problems like log triage, malformed JSON repair, and intent-based reranking, highlighting the promised gains in cost, latency, privacy, offline use, and reproducibility. It also digs into the paper’s broader claim, debating whether this is truly a new programming paradigm or a sharp repackaging of PEFT and LoRA-based adaptation, which makes the episode interesting for listeners thinking about practical deployment rather than just model benchmarks. Sources: 1. Program-as-Weights: A Programming Paradigm for Fuzzy Functions — Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng, 2026 http://arxiv.org/abs/2607.02512 2. HyperNetworks — David Ha, Andrew Dai, Quoc V. Le, 2016 https://scholar.google.com/scholar?q=HyperNetworks 3. LoRA: Low-Rank Adaptation of Large Language Models — Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, et al., 2021 https://scholar.google.com/scholar?q=LoRA:+Low-Rank+Adaptation+of+Large+Language+Models 4. Learning to Compile Programs to Neural Networks — Logan Weber, Jesse Michel, Alex Renda, Michael Carbin, 2024 https://scholar.google.com/scholar?q=Learning+to+Compile+Programs+to+Neural+Networks 5. Text-to-LoRA: Instant Transformer Adaption — Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange, 2025 https://scholar.google.com/scholar?q=Text-to-LoRA:+Instant+Transformer+Adaption 6. SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass — Y. Liu, X. Wang, Y. Mao, Y. Gelberg, H. Maron, and M. Zhang, 2026 https://scholar.google.com/scholar?q=SHINE:+A+Scalable+In-Context+Hypernetwork+for+Mapping+Context+to+LoRA+in+a+Single+Pass 7. Doc-to-LoRA: Learning to Instantly Internalize Contexts — R. Charakorn, E. Cetin, S. Uesaka, and R. T. Lange, 2026 https://scholar.google.com/scholar?q=Doc-to-LoRA:+Learning+to+Instantly+Internalize+Contexts 8. Latent Context Compilation: Distilling Long Context into Compact Portable Memory — Z. Li, Y. Zhou, and Q. Xu, 2026 https://scholar.google.com/scholar?q=Latent+Context+Compilation:+Distilling+Long+Context+into+Compact+Portable+Memory 9. The Alchemist: Automated Labeling 500x Cheaper than LLM Data Annotators — T. Huang, C. Cao, V. Bhargava, and F. Sala, 2024 https://scholar.google.com/scholar?q=The+Alchemist:+Automated+Labeling+500x+Cheaper+than+LLM+Data+Annotators 10. Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models — Vladimir Araujo, Marie-Francine Moens, Tinne Tuytelaars, 2024 https://arxiv.org/abs/2408.09053 11. X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design — Eric L. Buehler, Markus J. Buehler, 2024 https://arxiv.org/abs/2402.07148 12. CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning — Yu Feng, Zhen Tian, Yifan Zhu, Zongfu Han, Haoran Luo, Guangwei Zhang, Meina Song, 2024 https://arxiv.org/abs/2407.21043 13. Gradient Projection For Continual Parameter-Efficient Tuning — Jingyang Qiao, Zhizhong Zhang, Xin Tan, Yanyun Qu, Wensheng Zhang, Zhi Han, Yuan Xie, 2024 https://arxiv.org/abs/2405.13383 14. Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading — Minrui Xu, Dusit Niyato, Christopher G. Brinton, 2025 https://arxiv.org/abs/2501.14205 15. AI Post Transformers: SGLang for Faster Structured LLM Programs — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-06-sglang-for-faster-structured-llm-program-c59f1c.mp3 16. AI Post Transformers: OpenSkill for Open-World Self-Evolution in LLM Agents — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-08-openskill-for-open-world-self-evolution-19762a.mp3 17. AI Post Transformers: Learning Facts at Scale with Active Reading — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-25-learning-facts-at-scale-with-active-read-161bea.mp3 18. AI Post Transformers: Fine-Tuning LLMs for Human Behavior Prediction — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-21-fine-tuning-llms-for-human-behavior-pred-c79163.mp3