Before two browsers can establish a peer-to-peer media stream across the Internet, they face a classic bootstrapping paradox: how can two distributed entities establish direct cryptographic communication when neither knows the other's network IP address, supported audio/video codecs, encryption keys, or streaming capabilities?
To solve this initialization challenge, WebRTC relies on an out-of-band communication phase termed Signaling. Unlike traditional telecommunications standards (such as SIP or H.323), the WebRTC specification deliberately leaves the signaling protocol undefined, granting developers complete architectural freedom to use WebSockets, HTTP REST endpoints, gRPC, Server-Sent Events (SSE), or MQTT. During signaling, peers exchange their capabilities using the Session Description Protocol (SDP) under the formal rules of the JavaScript Session Establishment Protocol (JSEP) Offer/Answer state machine.
This guide delivers an exhaustive, university-grade examination of WebRTC signaling. We dissect the syntax and anatomy of SDP documents, analyze codec payload negotiation, trace the internal state machine transitions of the RTCPeerConnection API, resolve signaling collision race conditions (glare) via the Perfect Negotiation pattern, and derive media bandwidth constraints.
The Signaling Abstraction & Transport Agnosticism#
The WebRTC working group at the W3C made a foundational architectural decision: the browser handles the media transport, but the application developer controls the signaling.
+---------------------------------------------------------------------------------------------------+
| The WebRTC Triangular Signaling Model |
+---------------------------------------------------------------------------------------------------+
| |
| +-----------------------------------------------+ |
| | Application Signaling Server (WebSocket) | |
| | (Relays JSON Offer / Answer / ICE) | |
| +-----------------------------------------------+ |
| ^ ^ |
| Signaling Message 1 | | Signaling Message 2 |
| (SDP Offer) | | (SDP Answer) |
| v v |
| +------------+ +------------+ |
| | Peer A | | Peer B | |
| +------------+ +------------+ |
| \ / |
| \ / |
| v v |
| +===============================+ |
| | Direct P2P Media Stream (SRTP)| |
| +===============================+ |
| |
+---------------------------------------------------------------------------------------------------+Why Signaling Is Not Standardized in WebRTC#
- Protocol Flexibility: Modern web applications already possess existing communication channels (e.g., an established WebSocket connection for chat, Firebase real-time database, or a push notification service). Standardizing a rigid signaling protocol would force developers to re-architect their backend infrastructure.
- Identity & Authentication: User authentication, permissions, call routing, and contact lists belong to the application domain, not the browser network transport layer.
- Decoupling State: The signaling server acts merely as a transparent message relay; once the peer connection is established, the signaling server can experience transient restarts without terminating active media calls.
Session Description Protocol (SDP) Anatomy#
The Session Description Protocol (SDP, standardized in RFC 4566 and updated for WebRTC in RFC 8866) is a plain-text, line-oriented format used to negotiate media capabilities, transport addresses, cryptographic fingerprints, and streaming parameters.
The Syntax Rule#
Every line in an SDP document follows a strict, whitespace-free syntax:
<type>=<value>where <type> is exactly one single case-sensitive ASCII character, followed immediately by an equals sign =, followed by the structured value string.
Structural Decomposition of an SDP Document#
An SDP document is partitioned into two distinct hierarchical scopes:
- Session-Level Description: Applies to the entire conference session (lines preceding the first
m=line). - Media-Level Description: Applies to a specific audio, video, or data channel stream (begins with an
m=line and continues until the nextm=line).
+---------------------------------------------------------------------------------------+
| Anatomy of a WebRTC SDP |
+---------------------------------------------------------------------------------------+
| [ SESSION-LEVEL ATTRIBUTES ] |
| v=0 <-- Protocol Version |
| o=- 4820194820 2 IN IP4 127.0.0.1 <-- Session Owner ID & Version |
| s=- <-- Session Name (Unused in WebRTC) |
| t=0 0 <-- Time Active (Permanent: 0 0) |
| a=group:BUNDLE 0 1 2 <-- BUNDLE: Multiplex audio, video, data |
| a=msid-semantic: WMS <-- WebRTC Media Stream ID Mapping |
+---------------------------------------------------------------------------------------+
| [ MEDIA-LEVEL ATTRIBUTE BLOCK 0: AUDIO ] |
| m=audio 9 UDP/TLS/RTP/SAVPF 111 9 <-- Media type, port, transport, codecs |
| c=IN IP4 0.0.0.0 <-- Connection data |
| a=rtcp:9 IN IP4 0.0.0.0 <-- RTCP port |
| a=mid:0 <-- Media Identification tag |
| a=sendrecv <-- Directionality: Bidirectional |
| a=rtcp-mux <-- Multiplex RTCP on RTP port |
| a=rtpmap:111 opus/48000/2 <-- Codec mapping: Opus, 48kHz, 2 channels |
| a=fmtp:111 minptime=10;useinbandfec=1 <-- Format parameters: FEC enabled |
| a=fingerprint:sha-256 8A:4F:... <-- DTLS Certificate Hash |
| a=setup:actpass <-- DTLS Role Negotiation |
+---------------------------------------------------------------------------------------+
| [ MEDIA-LEVEL ATTRIBUTE BLOCK 1: VIDEO ] |
| m=video 9 UDP/TLS/RTP/SAVPF 96 97 <-- Video block, payload types 96, 97 |
| a=mid:1 <-- Media Identification tag 1 |
| a=sendrecv <-- Directionality |
| a=rtpmap:96 VP8/90000 <-- Codec mapping: VP8, 90kHz clock |
| a=rtpmap:97 H264/90000 <-- Codec mapping: H.264, 90kHz clock |
| a=fmtp:97 profile-level-id=42e01f <-- H.264 Constrained Baseline Profile |
| a=rtcp-fb:96 nack <-- Feedback: Negative Acknowledgement |
| a=rtcp-fb:96 nack pli <-- Feedback: Picture Loss Indication |
| a=rtcp-fb:96 ccm fir <-- Feedback: Full Intra Request |
+---------------------------------------------------------------------------------------+Critical SDP Fields Explained#
- `m=audio 9 UDP/TLS/RTP/SAVPF 111 9`:
* audio: Media stream category.
* 9: Port placeholder (under BUNDLE, the actual port is dictated by the ICE candidate).
* UDP/TLS/RTP/SAVPF: The transport profile: Secure Real-time Transport Protocol with Audio-Visual Profile with Feedback, secured over DTLS/UDP.
* 111 9: List of supported codec payload types in order of preference.
- `a=rtpmap:<payload_type> <encoding_name>/<clock_rate>[/<channels>]`:
* Maps numeric payload type 111 to the Opus codec operating at a 48,000 Hz sampling clock across 2 stereo channels.
- `a=fmtp:<payload_type> <parameters>`:
* Supplies codec-specific configuration: useinbandfec=1 instructs the Opus decoder that the encoder will embed Forward Error Correction bits directly into audio packets, enabling loss recovery.
- `a=fingerprint:<hash_func> <fingerprint_hash>`:
* The cryptographic hash of the endpoint's self-signed x.509 certificate used during the subsequent DTLS handshake. When the DTLS connection opens, the browser verifies that the certificate presented matches this fingerprint, preventing Man-in-the-Middle (MitM) attacks.
- `a=setup:<role>`:
* Controls the DTLS client/server roles: actpass (willing to be either client or server), active (initiates DTLS handshake), passive (listens for handshake).
Codec Negotiation & Dynamic Payload Types#
During signaling, the Offer contains all codecs supported by the initiating peer. The answering peer inspects the list, compares it against its own locally supported hardware and software decoders, and selects the mutual intersection.
Standard WebRTC Codec Profiles#
- Audio Codecs:
Opus*: The mandatory default audio codec for WebRTC (RFC 7587). Highly dynamic: scales seamlessly from 6 kbps narrowband speech to 510 kbps multi-channel lossless audio, featuring built-in Forward Error Correction (FEC) and Packet Loss Concealment (PLC).
G.711 (PCMU / PCMA)*: Legacy 64 kbps PSTN telephony fallback operating at 8 kHz sampling rate.
- Video Codecs:
VP8*: Open-source codec developed by Google; highly optimized software decoding across all modern browsers.
VP9*: Next-generation open codec supporting profile modes for 4K video and High Dynamic Range (HDR) with ~35% bandwidth savings over VP8.
H.264 (AVC)*: Ubiquitous hardware-accelerated codec on mobile devices (iOS / Android). The SDP must negotiate matching profile-level-id (e.g., 42e01f represents Constrained Baseline Profile Level 3.1).
AV1*: Modern royalty-free open video codec providing massive compression efficiency (~50% over VP8), ideal for bandwidth-constrained networks.
The JavaScript Session Establishment Protocol (JSEP) State Machine#
JSEP (standardized in RFC 8829) formalizes the exact sequence of JavaScript API calls and internal state transitions required to initiate and accept peer connections.
+---------------------------------------------------------------------------------------+
| JSEP Offer / Answer Flow |
+---------------------------------------------------------------------------------------+
| Peer A (Offerer) Peer B (Answerer) |
| State: STABLE State: STABLE |
| | | |
| 1. pc.createOffer() | |
| v | |
| 2. pc.setLocalDescription(offer) | |
| State: HAVE-LOCAL-OFFER | |
| | | |
| +---- 3. Transmit Offer via Signaling WebSocket ----------->| |
| v |
| 4. pc.setRemoteDescription(offer) |
| State: HAVE-REMOTE-OFFER |
| | |
| 5. pc.createAnswer() |
| v |
| 6. pc.setLocalDescription(answer) |
| State: STABLE |
| | |
| |<--- 7. Transmit Answer via Signaling WebSocket -----------+ |
| v |
| 8. pc.setRemoteDescription(answer) |
| State: STABLE |
| |
+---------------------------------------------------------------------------------------+Detailed State Transition Breakdown#
The pc.signalingState property of an RTCPeerConnection instance traverses an explicit finite state machine:
- `stable`: The baseline resting state. No offer or answer is currently in-flight.
- `have-local-offer`: The local peer generated an SDP offer (
createOffer()) and successfully applied it locally viasetLocalDescription(offer). The peer is waiting for a remote answer. - `have-remote-offer`: The remote peer generated an offer, transmitted it over signaling, and the local peer applied it via
setRemoteDescription(offer). The local peer must now generate and return an answer. - `have-local-pranswer` / `have-remote-pranswer`: Provisional answer states, used when a peer returns early media descriptors (e.g., ringback tones) before final connection establishment.
- `closed`: The connection has terminated and underlying OS socket resources have been destroyed.
Renegotiation & The Perfect Negotiation Pattern#
Once a WebRTC connection is active and streaming in the stable state, application state can change dynamically:
- A user clicks "Enable Camera" (adding a new video track via
pc.addTrack()). - A user clicks "Screen Share" (adding a desktop capture video track).
- A network bandwidth constraint forces downscaling or disabling a track.
When tracks are added or removed, the browser automatically fires the onnegotiationneeded event, signaling that a fresh Offer/Answer cycle must occur.
The "Glare" Problem (Signaling Collision)#
If Peer A and Peer B simultaneously add a track at the exact same millisecond:
- Peer A creates Offer A and enters
have-local-offer. - Peer B creates Offer B and enters
have-local-offer. - Both peers receive each other's offers over the signaling channel while in the
have-local-offerstate. - Under standard JSEP, calling
setRemoteDescription(offer)while inhave-local-offerthrows a fatalInvalidStateErrorexception! This collision condition is known as Glare.
The W3C Perfect Negotiation Pattern#
The W3C WebRTC working group resolved Glare by introducing the Perfect Negotiation Pattern. One peer is designated as Polite, and the other as Impolite:
+---------------------------------------------------------------------------------------+
| The Perfect Negotiation Pattern |
+---------------------------------------------------------------------------------------+
| Role Determination: |
| Peer A: Polite = false (Impolite) |
| Peer B: Polite = true (Polite) |
| |
| When Glare Occurs (Both peers send simultaneous offers): |
| |
| - Impolite Peer (Peer A): |
| Ignores the incoming remote offer from Peer B! |
| Maintains its own local offer and waits for Peer B to surrender. |
| |
| - Polite Peer (Peer B): |
| Receives Peer A's offer while waiting for its own answer. |
| Recognizes Glare collision: Polite peer surrenders! |
| Executes rollback: pc.setLocalDescription({ type: "rollback" }) |
| Accepts Peer A's offer: pc.setRemoteDescription(offerA) |
| Generates and returns Answer B to Peer A! |
| |
| ===> Result: Glare is resolved deterministically without race conditions! |
+---------------------------------------------------------------------------------------+Production JavaScript Implementation#
// Production Perfect Negotiation Pattern
const polite = isCaller ? false : true;
let makingOffer = false;
let ignoreOffer = false;
pc.onnegotiationneeded = async () => {
try {
makingOffer = true;
await pc.setLocalDescription();
signaling.send({ description: pc.localDescription });
} catch (err) {
console.error("Negotiation failed:", err);
} finally {
makingOffer = false;
}
};
signaling.onmessage = async ({ description, candidate }) => {
try {
if (description) {
// Glare Detection: Remote offer received while making local offer or not in stable
const offerCollision = description.type === "offer" &&
(makingOffer || pc.signalingState !== "stable");
ignoreOffer = !polite && offerCollision;
if (ignoreOffer) {
console.warn("Impolite peer: Colliding remote offer ignored.");
return;
}
await pc.setRemoteDescription(description);
if (description.type === "offer") {
await pc.setLocalDescription();
signaling.send({ description: pc.localDescription });
}
} else if (candidate) {
try {
await pc.addIceCandidate(candidate);
} catch (err) {
if (!ignoreOffer) throw err;
}
}
} catch (err) {
console.error("Signaling error:", err);
}
};Advanced Multi-Stream Signaling: Simulcast & Scalable Video Coding (SVC)#
In multi-party video conferences, participants have wildly varying downstream network bandwidth: a desktop user on gigabit fiber can consume 1080p 60fps video, while a mobile user on a congested 3G network can barely sustain 240p.
If the publisher encodes and transmits a single 1080p stream, the mobile receiver experiences total buffer collapse and dropped frames. If the publisher downscales to 240p, the desktop user receives degraded video quality.
1. WebRTC Simulcast Signaling#
Under Simulcast, the publishing client's hardware encoder concurrently generates multiple distinct spatial and temporal resolutions of the same camera feed (e.g., High: 1080p @ 2.5 Mbps, Medium: 720p @ 800 kbps, Low: 360p @ 200 kbps).
In the SDP, Simulcast is signaled via the Restriction Identifier (a=rid) and a=simulcast attributes (RFC 8853):
m=video 9 UDP/TLS/RTP/SAVPF 96
a=rtpmap:96 VP8/90000
a=rid:h send
a=rid:m send
a=rid:l send
a=simulcast:send h;m;lrid:h: High resolution layer.rid:m: Medium resolution layer.rid:l: Low resolution layer.
An intermediary Selective Forwarding Unit (SFU) inspects each receiver's downstream bandwidth and selectively forwards only the layer each receiver can accommodate, without decoding or re-encoding video frames.
2. Scalable Video Coding (SVC)#
While Simulcast transmits three separate video streams (consuming ~1.5x the upload bandwidth of 1080p), Scalable Video Coding (supported in VP9 and AV1) encodes video into a single layered bitstream:
- Base Layer: Lowest spatial resolution and framerate (e.g., 240p @ 15fps).
- Enhancement Layers: Contain differential residual information that increases spatial resolution (720p, 1080p) or temporal framerate (30fps, 60fps).
- If a packet for an enhancement layer drops, the decoder drops the enhancement but smoothly renders the base layer without visual artifacts.
Cryptographic Key Exchange: DTLS-SRTP Key Derivation#
A common misconception is that the encryption keys protecting real-time video are exchanged in the SDP signaling document.
In early VoIP protocols (SDES, RFC 4568), base64-encoded AES encryption keys were placed directly in plaintext a=crypto: SDP lines. If an attacker compromised the signaling server or intercepted the SIP/HTTP traffic, they could decrypt all audio and video calls instantly.
The Modern WebRTC Security Invariant: DTLS Handshake#
WebRTC strictly mandates DTLS-SRTP (RFC 5763 / RFC 5764):
- Fingerprint Verification: The SDP contains only the cryptographic hash of the endpoint's public certificate (
a=fingerprint:sha-256 ...). No private keys or encryption keys are ever transmitted over signaling! - Datagram Handshake: Once the ICE layer establishes UDP connectivity, the peers initiate a full TLS 1.2 / 1.3 handshake over the UDP socket (
ClientHello,ServerHello,Certificate,KeyExchange). - Identity Binding: Each peer computes the SHA-256 hash of the certificate presented by the remote party and verifies that it exactly matches the fingerprint received earlier in the signed SDP answer.
- SRTP Key Derivation: The peers use the DTLS Pseudo-Random Function (PRF) with key material exporters to derive four symmetrical session keys:
* Client SRTP Encryption Key (AES-128 / AES-256)
* Client SRTP Authentication Salt
* Server SRTP Encryption Key
* Server SRTP Authentication Salt
- Decoupled Security: Even if a hostile actor intercepts, logs, or tampers with the entire signaling server, they possess zero cryptographic material to decrypt the SRTP media streams.
Scalable Signaling Architecture: Room State & Message Brokers#
In enterprise production environments, signaling servers manage millions of concurrent WebSocket connections across global regions.
+---------------------------------------------------------------------------------------+
| Enterprise Distributed Signaling Cluster |
+---------------------------------------------------------------------------------------+
| |
| Client 1 (Tokyo) ------------> [ Edge WebSocket Gateway A ] |
| | |
| v Publish Event |
| +-------------------------+ |
| | Redis Cluster Pub/Sub | |
| | (Room State Key-Value) | |
| +-------------------------+ |
| ^ |
| | Subscribe Event |
| Client 2 (London) -----------> [ Edge WebSocket Gateway B ] |
| |
+---------------------------------------------------------------------------------------+The Distributed Room State Problem#
- WebSocket Gateway Tier: Stateless Node.js, Go, or Rust edge servers that maintain persistent WebSocket connections with browsers.
- Message Bus / Pub-Sub (Redis / NATS): When Client 1 in "Room-482" emits an SDP offer, Gateway A publishes the payload to Redis topic
room:482. - Cross-Gateway Delivery: Gateway B, which holds the active WebSocket connection for Client 2, receives the message from the Redis subscription and pushes the offer to Client 2 within milliseconds, regardless of which physical server each client connected to.
Checkpoint Practice Problems#
Problem 1 (SDP Codec Payload Intersection Negotiation): Peer A initiates an offer with the following video media attributes:
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99
a=rtpmap:96 AV1/90000
a=rtpmap:97 VP9/90000
a=rtpmap:98 VP8/90000
a=rtpmap:99 H264/90000
a=fmtp:99 profile-level-id=42e01fPeer B is an older mobile smartphone that supports only H.264 (Constrained Baseline Profile 42e01f) and VP8. Peer B's internal codec priority prefers H.264 over VP8.
- Determine the intersecting set of codecs mutually supported by both peers.
- In the resulting SDP Answer constructed by Peer B, what will the
m=videoline look like? - Which codec will be selected as the primary active video stream for the call?
Solution:
- Compute Intersection:
* Peer A supports: { AV1, VP9, VP8, H.264 (42e01f) }
* Peer B supports: { H.264 (42e01f), VP8 }
Mutual Intersection: { H.264, VP8 }*
- Constructed
m=videoline in Answer:
Because Peer B's local configuration prioritizes H.264 over VP8, Peer B lists H.264 first:
```
m=video 9 UDP/TLS/RTP/SAVPF 99 98
a=rtpmap:99 H264/90000
a=fmtp:99 profile-level-id=42e01f
a=rtpmap:98 VP8/90000
```
- Primary codec selection:
The first payload type listed in the Answer (99) dictates the active codec. The call will transmit video encoded via H.264.
Problem 2 (JSEP Signaling State Machine Sequence Validation): An engineer inspects WebRTC telemetry logs and observes the following sequence of method calls on a local RTCPeerConnection:
- Step 0:
pc.signalingStateis initialized. - Step 1:
pc.setRemoteDescription(offer)is executed. - Step 2:
pc.createAnswer()is executed. - Step 3:
pc.setLocalDescription(answer)is executed. - Step 4: The peer attempts to execute
pc.setLocalDescription(offer)while still in the resulting state.
- State the exact value of
pc.signalingStateafter Step 0, Step 1, and Step 3. - Explain what happens at Step 4, and identify the specific JavaScript DOMException thrown by the browser.
Solution:
- State evolution:
After Step 0 (Initialization): `stable`*.
After Step 1 (`setRemoteDescription` with an offer): `have-remote-offer`*.
After Step 3 (`setLocalDescription` with an answer): Transitions cleanly back to `stable`*.
- Evaluation of Step 4:
* At Step 4, the peer is in state stable.
* Calling setLocalDescription(offer) directly without prior createOffer() is valid in modern WebRTC (it implicitly calls createOffer()), transitioning the state from stable to have-local-offer.
However, if the parameter passed to `setLocalDescription()` is an incompatible SDP object or an old cached offer whose session version has not incremented, the browser throws an `InvalidModificationError` or `InvalidStateError`*.
Problem 3 (Signaling Race Condition & Glare Probability): In an asynchronous peer connection, both peers trigger track renegotiation independently.
- The signaling transit time between Peer A and Peer B over the WebSocket server is
T_transit = 150 milliseconds. - Peer A triggers an offer at timestamp
t = 1,000 ms. - Peer B triggers an offer at timestamp
t = 1,000 + Δt ms.
- What is the critical time window for
Δtduring which a Glare collision is guaranteed to occur? - If Peer B is designated as the Polite Peer, describe the exact step-by-step actions executed by Peer B when Peer A's offer arrives at
t = 1,150 ms.
Solution:
- Glare Collision Window:
Peer A sends its offer at t = 1,000 ms. This offer arrives at Peer B at t = 1,000 + 150 = 1,150 ms.
If Peer B initiates its own local offer before t = 1,150 ms, Peer B will be in the have-local-offer state when Peer A's offer arrives.
Similarly, Peer B's offer arrives at Peer A at t + Δt + 150 ms.
Glare occurs whenever Peer B initiates its offer before receiving Peer A's offer:
```
0 ≤ Δt < 150 milliseconds
```
If Peer B creates an offer within this 150 ms transit window, Glare is guaranteed.
- Polite Peer B's Execution Sequence:
When Peer A's offer arrives at t = 1,150 ms:
* Peer B inspects its state: pc.signalingState == "have-local-offer".
* Peer B recognizes Glare (offerCollision == true).
Because Peer B is polite*, it yields precedence:
1. Peer B rolls back its uncommitted local offer:
await pc.setLocalDescription({ type: "rollback" })
State returns from have-local-offer to stable.
2. Peer B accepts Peer A's offer:
await pc.setRemoteDescription(offerA)
State transitions to have-remote-offer.
3. Peer B generates an answer incorporating both peers' changes:
const answer = await pc.createAnswer()
await pc.setLocalDescription(answer)
State returns to stable.
4. Peer B transmits the answer back to Peer A over the signaling socket.
Problem 4 (SDP Application-Specific Bandwidth Sizing): In an SDP media block, the bandwidth attribute b=AS:<kbps> specifies the Application Specific maximum bandwidth in kilobits per second (including RTP, UDP, and IP headers):
- A video stream declares
b=AS:1500(1,500 kbps total IP bandwidth). - The video is transmitted at
30 frames per second. - Each video frame is packetized into
3 discrete RTP packets. - The protocol headers per packet are:
* IPv4 Header: 20 bytes
* UDP Header: 8 bytes
* SRTP Header: 12 bytes
* SRTP Authentication Tag: 10 bytes
- Compute the total protocol header overhead (in bytes and bits) per individual packet.
- Compute the total protocol header bandwidth consumed per second (in kbps).
- Compute the maximum net payload bandwidth remaining strictly for compressed video data.
Solution:
- Header overhead per packet:
```
Header Bytes = 20 (IPv4) + 8 (UDP) + 12 (SRTP) + 10 (Auth Tag) = 50 bytes per packet
Header Bits = 50 bytes · 8 bits/byte = 400 bits per packet
```
- Header bandwidth per second:
* Packets per second:
30 frames/sec · 3 packets/frame = 90 packets per second.
* Header bit rate:
90 packets/sec · 400 bits/packet = 36,000 bits per second = 36.0 kbps.
- Net video payload bandwidth:
* Total allocated bandwidth = 1,500 kbps.
* Net video bandwidth:
BW_video = Total_BW - Header_BW = 1,500 kbps - 36.0 kbps = 1,464.0 kbps.
The video encoder must target a maximum compressed bitrate of 1,464.0 kbps to honor the b=AS:1500 ceiling constraint.