Decoded — How Video Really Works

Fora Soft

Decoded is a podcast about how video really works — the engineering behind every stream, video call, and clip you watch. Each episode takes one piece of the video pipeline — pixels and color, codecs and compression, quality metrics, and the latest in AI — and explains it clearly, from first principles to what actually ships in production. The series is based on The Fora Soft Video Engineering Handbook and is made by Fora Soft, a software company that has done one thing since 2005: video, real-time communication, and AI systems. 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. Whether you're a developer moving into video, a product person who needs to reason about codecs and bitrate, or simply curious how a camera becomes a stream, Decoded turns a dense field into something you can actually follow — one short episode at a time. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  1. Episode 1

    Episode 1. What Video Delivery Is

    A photo that loads a second late is merely annoying; a video frame that arrives late is garbage. That single difference — video as data with a deadline — is why streaming is its own engineering discipline rather than file serving at scale. Decoded's second season opens with the problem statement: what separates a stream from a download, why one giant server can't carry a big audience, and what the industry built instead. In this episode: — file vs stream: why a video stream is data with a schedule, bound by three requirements — time, order, and continuity — the 33 ms deadline per frame at 30 frames per second, why a late frame is as good as lost, and what rebuffering is — how impatient viewers are: the 2012 Akamai study behind the 2-second startup threshold and 5.8 more percentage points of abandonment for every extra second — why one giant server fails: 100,000 viewers need about 450 Gbps of output, while an ordinary machine pushes roughly 10 — distance as a hard limit: why a server in Amsterdam gives a viewer in Sydney a slow start no matter how much bandwidth it has — the industry's answer: cut the stream into few-second segments, serve them over plain HTTP, and cache copies close to viewers Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  2. Episode 2

    Episode 2. Codec, Container, Protocol, Profile

    When half an audience's devices refuse to play a stream, the root cause is often vocabulary: a contract promised 4K without naming a codec or profile, and codec, container, protocol, and profile all got lumped together as "the format." Episode two pulls the four layers apart — what each one decides, why the decisions are independent, and which mix-ups break playback. In this episode: — four words teams constantly mix up — codec, container, protocol, profile — and the single question each layer answers — what the codec does: raw 1080p at about 1.5 Gbps squeezed roughly 330-fold, and why only the codec compresses — why an .mp4 extension names the container rather than the codec, and why a plain MP4 with its table of contents at the tail is awkward to stream — fragmented MP4 as the shape of modern streaming segments, and MPEG-TS as the digital-TV veteran still common in live broadcast — protocols as rules of carriage: why "we stream via HLS" says nothing about H.264 or fMP4, and where WebRTC fits — profile and level as a promise to the decoder: what H.264 High@L4.1 guarantees an aging smart TV or a budget phone Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  3. Episode 3

    Episode 3. Live, VOD, and Near-Live: Three Delivery Problems

    Behind an identical play button, a finished film and a championship game are different engineering problems, and the difference comes down to one thing: the deadline. Here streaming gets sorted into live, video on demand, and near-live — where the three share everything, right down to the bytes on the wire, and where their costs, failure modes, and deliberate delays split apart. In this episode: — live, VOD, and near-live as three delivery problems that differ in one thing — the deadline between producing a frame and showing it — why the bytes on the wire are identical and the first real difference is the playlist: a finished VOD list with an end tag vs a sliding live window — two cost structures: a 12-minute film's whole ladder fits in under a gigabyte, while live pays for real-time transcoding and about 450 Gbps of peak for 100,000 viewers — why live gets no second take: a minute lost at ingest is a minute of the event gone, and no downstream autoscaling brings it back — near-live as delay used on purpose — typically 5–10 seconds — to keep betting fair, give live-shopping moderators time, and sync fast internet paths with TV — how appending a single end tag turns a finished live broadcast into VOD without re-encoding a byte Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  4. Episode 4

    Episode 4. Latency: Glass-to-Glass and the Latency Budget

    Ask a vendor about latency and the answer is a second and a half; ask a viewer and the live match trails the stadium by twenty-odd seconds — and both can be telling the truth. This episode takes glass-to-glass latency apart term by term: where the seconds actually go, why the network is almost never the culprit, and what it really costs to push a stream below one second. In this episode: — glass-to-glass latency — from the camera lens to the viewer's screen — and why it's the only number that matches what the viewer feels — why a vendor's "1.5 seconds" means nothing until you ask between which two points it was measured — the seven-term latency budget, from capture and encoder to CDN and decoder, and why everything but packaging and the player buffer sums to under a second — the two real culprits: a segment can't ship until it ends, and classic HLS makes the player hold three segments — 18 seconds with 6-second segments — why the buffer can't simply be torn down: a thinner reserve turns every Wi-Fi blip or lost packet into a stall — the price of sub-second: WebRTC trades cheap HTTP caches for a real-time connection per viewer, worth it only when viewers bid or talk back Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  5. Episode 5

    Episode 5. The Network Reality: Bandwidth, Throughput, Jitter, Loss

    Most streaming complaints — the stream froze, the picture fell apart, the quality dropped and never came back — trace back to one of four network numbers, and the one an internet provider advertises is usually the least useful of them. The episode separates bandwidth, throughput, jitter, and packet loss, walks through the classic ways real networks fail, and runs the arithmetic on a stream that doesn't fit the link it was sold. In this episode: — bandwidth vs throughput: the pipe's diameter versus the water actually flowing, and the three forces — distance, loss, competition — that pull them apart — why the adaptive player ignores your speed test and judges the link by how fast the last segment downloaded — jitter as unpredictability, not delay: a steady 200 ms is fine, a delay bouncing between 50 and 250 ms is hostile — plus the 1/16 smoothing in RFC 3550 — why 1% packet loss shows in live video: one lost packet can spoil a keyframe and up to two seconds of picture, and the retransmit-vs-redundancy trade-off — four recognizable failures — bufferbloat, repeated slow start, last-mile peak hour, mobile handover — and why buying more bandwidth can't fix three of them — the arithmetic of a 5 Mbps stream failing on a "6 Mbps" link: Apple's 1.25× peak headroom plus audio pushes the real need to about 6.4 Mbps Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  6. Episode 6

    Episode 6. The End-to-End Streaming Pipeline

    In November 2024 Netflix streamed the Jake Paul–Mike Tyson fight to a peak of around 65 million concurrent streams, and for many viewers it stuttered — at a company that knows its pipeline as well as anyone. When live video breaks, the viewer's complaint almost never points at the stage that failed. Episode six of Decoded's second season draws the full delivery pipeline as a nine-station map, with a rule for finding the guilty stage and the seam that splits it into push and pull. In this episode: — the nine stages from camera to screen — capture, contribution, transcode and package, origin, CDN, last mile, buffer, decoder, screen — the debugging rule: a fault at stage n shows up at n+1 and later, never earlier — so look for the first bad metric, not the loudest symptom — why a frozen player usually means an empty buffer, starved by a congested last mile or a CDN cache miss further upstream — push vs pull — the delivery truck versus the warehouse counter — and what follows from it: cacheability and the latency floor — contribution vs distribution: one fragile uplink sized at 1.5× the bitrate to absorb keyframe bursts, versus fan-out to millions through a cache — how Media over QUIC aims to remove the seam between push and pull Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  7. Episode 7

    Episode 7. TCP and UDP: The Choice Every Protocol Makes

    Why does a recorded film play flawlessly while a video call on the same connection freezes mid-sentence? The answer sits one layer beneath every streaming protocol. At stake is the first architectural choice any streaming product makes — TCP or UDP — explained through the one idea that decides it, head-of-line blocking, followed by a look at how QUIC starts to dissolve the old boundary. In this episode: — the six jobs of a transport protocol: TCP takes on all six, UDP handles just one and leaves the rest to the application — head-of-line blocking frame by frame: how losing a packet of frame 100 makes TCP hold frames 101–103 hostage and freeze a call — three ways to survive loss and what each costs: retransmission pays in delay, FEC in bandwidth, loss concealment in a brief local glitch — why HLS and DASH are happy on TCP: a player buffer of around 12 seconds hides any resend while segments stay cacheable at scale — why real-time protocols like WebRTC and SRT ride UDP to reach sub-second latency — how QUIC runs over UDP yet restores TCP's guarantees, moving head-of-line blocking from the whole connection to the individual stream Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

  8. Episode 8

    Episode 8. Congestion Control: CUBIC, BBR, Copa

    Radio noise on Wi-Fi drops a packet, and the internet's default algorithm reacts as if it had hit a traffic jam — slowing down for no reason. That choice of signal is the heart of congestion control, the rule that quietly decides how fast segments reach viewers and how long queues grow along the way. Episode eight traces it from a 1980s network meltdown to CUBIC, BBR, and the delay-based control behind WebRTC. In this episode: — the 1986 congestion collapse, when a Berkeley link fell from 32 kbit/s to 40 bit/s, and why every sender has had to police its own speed ever since — CUBIC, the default on Linux, Windows, and Apple: why cutting speed by 30% on every loss hurts on Wi-Fi and cellular and packs router queues into bufferbloat — BBR's model of bottleneck bandwidth and round-trip time, and how moving YouTube to it cut median RTT by 53% — the fairness catch: a single BBRv1 flow holding about 40% of a link against sixteen CUBIC flows — why real time needs delay-based control, and how WebRTC's GCC trims the encoder's bitrate instead of letting packets queue — how swapping CUBIC for BBR at the edge can lift a mobile viewer on a lossy link from 360p to 480p without touching the video Decoded is a podcast by Fora Soft, based on The Fora Soft Video Engineering Handbook book series. Season two is about streaming and follows the book Video Streaming. Since 2005 Fora Soft has built video products, real-time communication, and AI systems: 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

See All Season 2 (26)

About

Decoded is a podcast about how video really works — the engineering behind every stream, video call, and clip you watch. Each episode takes one piece of the video pipeline — pixels and color, codecs and compression, quality metrics, and the latest in AI — and explains it clearly, from first principles to what actually ships in production. The series is based on The Fora Soft Video Engineering Handbook and is made by Fora Soft, a software company that has done one thing since 2005: video, real-time communication, and AI systems. 50 in-house engineers, 250+ delivered projects, live streaming with 0.4–0.5-second glass-to-glass latency for 10,000 simultaneous viewers, and solutions used by NBC, CBS, and Red Bull. Whether you're a developer moving into video, a product person who needs to reason about codecs and bitrate, or simply curious how a camera becomes a stream, Decoded turns a dense field into something you can actually follow — one short episode at a time. If you're building a product with video or online calls, come talk to us: https://www.forasoft.com/

Information