Building Things with Machine Learning Yaoshiang ho

A true “aha” conversation! Learn how deep learning techniques from natural language processing (NLP) are applied to drug discovery, specifically, protein to protein interactions. Includes a quick and dirty primer on just enough biology to understand the training data A-Alpha Bio uses for their ML models.
 
Show Notes:
 
0:37 - The basics of synthetic biology for machine learning practitioners
0:50 - What are proteins and why do they matter?
1:50 - A protein is a string of 20 amino acids… which means it starts looking like a Natural Language Processing problem.
2:35 - DeepMind’s AlphaFold and Meta FAIR’s ESMFold: taking as input a string of amino acids, and then predicting the 3D structure of proteins.
6:23: Where Alphafold got their training data: The Protein Data Bank.
8:07: A Alpha Bio’s product: AlphaSeq. 10:45: The source of the name “A Alpha Bio”: yeast genders. 11:36: Applications of synthetic biology: pharmaceuticals, agriculture.
15:00: Applying ML to predict protein to protein interactions.
20:30: !!! The actual ML techniques applied: treating proteins as strings and applying NLP architectures: RNNs, LSTMs, Attention, and Transformers.
22:50: Discrete Optimization problem to then generate proteins.
28:30: The insights behind why applying ML would work.
31:20: The rise of deep learning in the field of computational biology.
32:50: Ryan’s journey into machine learning and data science
35:20: Advice for deep learning people interested in applying ML to biology
 
Additional papers covering the topic of ML in biology:
https://www.nature.com/articles/s41586-021-03819-2 - The AlphaFold paper.
https://pubmed.ncbi.nlm.nih.gov/35830864/ - A broad overview of deep learning in biology.
https://pubmed.ncbi.nlm.nih.gov/35862514/ - A paper out of the Baker lab in which the authors use deep learning to design proteins from scratch.
https://pubmed.ncbi.nlm.nih.gov/35099535/ - From Charlotte Deane’s lab with collaborators from Roche, this paper presents a deep learning approach to rapidly and accurately model the structure of antibody CDR3 loops. One of the papers mentioned in the review above.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9129155/ - This is recent work from A-Alpha; this paper doesn’t include any ML but does include some great examples of AlphaSeq data and how it can be applied.

Ciro Greco has built ML systems used at many named-brand retailers. In this episode, he gives us tips on getting value out of ML at “reasonable scale” companies with NLP and information retrieval. The concept of “reasonable scale” was one he returned to, and he clearly has a very nuanced understanding of that segment and how they are different from the hyper scale tech giants. He also brings advanced ideas like embeddings from NLP towards e-commerce personalization. 
 
1:36: Key differences in applying ML at “reasonable scale” companies like major retailers where you can’t just “big-data” your way out of problems, compared to the hyper scale tech giants. 
3:22: The basics of personalization: suggestions, search, recommendations, and categories. 
4:38: A non-obvious challenge: how to personalize for non-logged-in users without a profile who visit infrequently. 
9:00: Different incentives for reasonable scale vs hyper scale companies.
9:44: Getting your data right: data ingestion, data practices, organizing teams around data, transforming data, infrastructure for flexible data access, so that you can make developers productive when you have finite resources.
11:23: Learning from experience that data - replayability and replicitability - is more important than modeling.
12:58: Learnings from experiences at presenting at top tier conferences: so many published papers come from the hyper scale companies.
14:19: Taking session data and catalog data to create a “product to vector” embedding to personalize an experience.
19:20: Requirements on how to sell: the sales people must communicate to the “people who write the check” that data integration is a first class citizen, not a downstream task, to achieve ROI.
21:09: Dynamics of regulatory and privacy issues, and how to tackle them as an organization.
24:10: Ciro talks about his personal journey into ML, starting with a PhD in neuroscience and linguistics. 
25:46: Early challenges in applying deep learning to NLP.
26:22: The “a ha” moment that led to Ciro’s first startup delivering search products.
27:55: Changes in the role of a data scientist over the past decade. From the role of PhDs who had to tackle problems with very little tooling, to today where there are so many tools available. And a  shift towards understanding products and customers. 
 
 
 

Yihua Liao is Head of Data Science at Netskope, a next-generation cybersecurity firm. Yihua talks about using both CV and NLP to create novel cybersecurity features. Yihua Liao’s PhD research was on security and machine learning, and he previously worked at Microsoft, Facebook, Uber, and his own startup.
00:24 - How Netskope addresses cybersecurity.
00:57 - Netskope’s unique approach to cybersecurity through network traffic routing.
02:51 - The prior approach to cybersecurity: a focus on the physical perimeter and firewalls.
03:44 - A unique application of Image Classification in cybersecurity: identifying sensitive documents like driver’s licenses so CISOs (chief information security officer) can set security rules for them.
07:45 - Challenges of building Image Classifiers #1: High quality data.
08:45 - Challenges of building Image Classifiers #2: Managing false positive and false negatives (recall and precision).
09:15 - Challenges of building Image Classifiers #3: Managing latency (15 ms) for a real-time use case.
10:38 - An application of NLP (natural language processing) in cybersecurity: classifying phishing websites.
13:46 - Optimizing LLMs (Large Language Models) through quantization and distillation.
14:45 - How Yihua got into ML. 16:10 - How ML has evolved over the past 15 years.
Show Notes: https://www.netskope.com/ https://www.netskope.com/blog/enhancing-security-with-ai-ml

Ted tells us about applying machine learning to the field of baseball cards! 33% of Americans have trading cards, making this a very large addressable market. Learn some tips on scrappy ways to launch an app, and how similarity search powers one of the killer features of the CollX app. 
Key Moments: 
Building an application that works around the potential errors of an ML model (15:10). The data and ML behind his trading card valuation model, especially when recent transactions don’t exist. (18:30). Dealing with the latency inherent in ML and networking through the concept of “building lists” (18:25). Early work on product search (24:00). Working with bad training data and adding a “wizard behind the curtains” to deliver value while labeling data (26:18). More UX techniques to reduce perceived latency (28:00). Helping users understand that ML models are not 100% accurate (30:45). Advice for entrepreneurs trying to launch an app (35:20).  

In this episode, I interview my colleague Tom Rikert at Masterful AI. Tom is building the "AutoML 2.0" platform for computer vision. We talk about the product for the first 10 minutes, and then spend some learning about his work at MIT CSAIL, which got him into robotics and computer vision, as well as his experiences selling a startup to Google and his time as a venture capitalist at Andreessen Horowitz and Nextworld Capital. 

Welcome to the Building Things with Machine Learning Podcast.  Every episode, I’ll be interviewing someone who building really interesting products using machine learning. 
Our focus is really on applications:
Medical diagnostics
Autonomous vehicles  & advanced driver assistance systems (ADAS)
Geospatial analytics
Media and Content analysis
Manufacturing
Logistics
And AEC, Architecture / Engineering / Construction
What you won’t get are coding tips or research papers. Although ML developers are definitely part of our audience, so are product managers and marketers and entrepreneurs - anyone who wants to see how machine learning is being used in action. 
 
 
I wanted to start this podcast because Building things with machine learning is a different discipline than traditional software development. 
Some big differences: 
A developer has a lot less control than traditional software.  ML is also used to solve many problems in the real world where it must be paired with physical sensors and actuators and robotics. Finally, ML is never perfect - it’s always a game of probabilities - so this is a new way of thinking about things than the traditional concept of software, where it’s realistic to think about stamping out every single bug and defect. In ML, you have to build applications that you accept will not be 100% accurate.  I hope you get something out of the “Building Things with Machine Learning” podcast! 
If you enjoy this podcast, please give us a rating on your podcast store. It helps others find our podcast.