Media & WebRTC

Video conferencing, live streaming, and on-demand playback powered by LiveKit's SFU architecture. AI agents participate as active media participants — seeing, hearing, and speaking in real-time.

LiveKit SFU

GoRust SDK

LiveKit provides a Selective Forwarding Unit (SFU) that handles WebRTC media routing at scale. Its Agents Framework enables AI agents to join rooms as participants with full audio/video capabilities.

Capabilities

Video conferencing (multi-party SFU)
Live streaming (WHIP/WHEP ingress/egress)
Recording & compositing
Data channels for arbitrary messaging
Simulcast & adaptive bitrate

Agent Integration

Agents join as room participants
STT/TTS pipeline for voice agents
Vision model receives video frames
Rust SDK (livekit-rust-ffi)
Dispatch via agent DID resolution

Video Lexicons

Custom lexicons define video content as AT Protocol records, making streams discoverable, portable, and indexable by the AppView.

com.network.video.stream

Live stream record with HLS/WebRTC endpoint, title, status, and streamer DID

endpoint: string (URL)title: stringstatus: 'live' | 'ended'streamerDid: did

com.network.video.conference

Conference room with participant list, SFU endpoint, and access permissions

roomId: stringparticipants: did[]sfuEndpoint: stringaccess: 'public' | 'invite'

com.network.video.recording

On-demand VOD record with manifest URL, duration, and transcode status

manifestUrl: string (HLS)duration: integer (seconds)thumbnail: blobstatus: 'processing' | 'ready'

Alternative: str0m (Pure Rust)

For scenarios requiring a fully Rust-native WebRTC stack without C dependencies, str0m provides a sans-IO implementation. It gives you complete control over the network layer and integrates cleanly with tokio-based services.

// str0m: sans-IO WebRTC — you control the event loop
use str0m::Rtc;

let mut rtc = Rtc::builder()
  .set_ice_lite(true)
  .build();

// Add media tracks for agent audio/video
let audio = rtc.add_media(
  MediaKind::Audio, Direction::SendRecv, None, None
);

// Process in your own event loop
loop {
  let timeout = rtc.poll_output()?;
  // handle Output::Transmit, Output::Event, etc.
}

Fig 5 — WebRTC media flow with AI agent participation via LiveKit SFU

Recommendation: Use LiveKit for production video infrastructure (conferences, streaming, recording). Use str0m only if you need a custom SFU or have strict no-C-dependency requirements for embedded scenarios.