Episode 65: Scaling Data Pipelines with Nick Schrock, Founder/CTO of Dagster Labs

Intro

Mike Schwartz: Hello and welcome to Open Source Underdogs! I’m your host Mike Schwartz, and this is episode 65 with Nick Schrock, Founder and CTO of Dagster, a platform that helps companies create data pipelines, which is critical to transform and update data in order to make it useful, for example, to generate reports, content, or other actionable information.
Dagster might not be a blueprint you can emulate. Like all start-ups, there are some hard to replicate serendipity that enables Nick and his team to build this amazing company. But as Machiavelli says, “Great leaders need both – fortune and virtue.” In other words, you need to be good at what you do, i.e. virtue, but they also need some good old-fashioned luck.

But what separates a really successful founders, like Nick, is the ability to harness fortune and virtue and combine it with some deep insights about the market, and turn it into a profitable and fast-growing venture not easy to do.

So, with that said, let’s cut to the interview, and let Nick tell you, in his own words, how Dagster evolves.

Early Career

Nick Schrock: Great to be with you.

Mike: Nick, thanks for joining us today.

Mike: Can I just go back a little bit and ask you to share some of your story about how you ended from going from the University of Michigan Computer Science to working at Facebook? So, that early period – how that happened?

Nick: Oh, I wasn’t expecting to talk about the preface book days. I’ll do the quick version of that. I graduated from Michigan in 2003, and I actually went to work at Microsoft, right out of school. And Microsoft’s a great company, and they treated me well, but… 

And actually, the division I was in was the developer division. And I thought that they were just extraordinarily talented, but at that time of my life, that wasn’t for me, in terms of working at a big company.
I wasn’t actually sure if I wanted to do software anymore, so I went to the London School of Economics for a year, because I thought I might want to go more into finance, or even government service – you know, I was a young man kind of searching around.

But I ended up getting back into software. I worked for a healthcare start-up out of Ann Arbor, which is where Michigan is, for what – 2 and a half years.

And then, I went to Chicago to try to do a start-up. That was very quickly spun down because me and a friend, who had worked in the finance industry, we wanted to do it, but then, it was about 6 months before the financial crisis.

So, that was incredibly poor timing. I spun that down, and actually, turns out a friend of mine, who I knew from Microsoft, kind of heard that was on the open market, and he just reached out and was like, “Hey, I’m working at Facebook, it’s really a special place. You should consider looking at it.”

And I was looking at staying in finance in the Chicago area. And I flew out to Facebook, and it’s just the vibe difference between a place like Facebook and a hedge fund in Chicago cannot be overstated.

You know, everyone at Facebook was young, super excited, idealistic, the office was incredible – there was just all this energy versus all these miserable people working in the hedge fund. So, the choice was obvious from there. And then, off to the races after that.

Why was Facebook so innovative in 2009-2015?

Mike: So, what was it about Facebook in 2009 that made it such a hotbed of innovation? Like, what new problems were they trying to solve?

Nick: The engineering-driven culture there, combined with the actual product that was being built. So, the produc