System Architecture

High-Level Overview

This pipeline ingests 4K video from DSLR/mirrorless cameras, synchronizes audio via Dante network, enables low-latency NDI caller integration, and outputs to streaming platforms (YouTube, Twitch) via OBS.

Design principle: Separation of concerns — capture, sync, mixing, output are independent subsystems connected via standardized protocols (HDMI, Dante, NDI, RTMP).

Signal Flow Diagram

┌─────────────────────────────────────────────────────────┐
│ INPUT LAYER                                             │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  Cameras (HDMI)              Cameras (Audio)            │
│  ├─ Panasonic G7             ├─ USB or XLR             │
│  ├─ GoPro Hero 9             └─ Dante converters       │
│  ├─ GoPro Max                                           │
│  └─ DJI Osmo                 Callers (Remote)          │
│       ↓                        └─ NDI OBS stream       │
│  DeckLink Quad HDMI                ↓                   │
│  (Echo Express SE I)          M1 Mac Ultra             │
│       ↓                        (OBS NDI input)         │
│                                                         │
├─────────────────────────────────────────────────────────┤
│ CAPTURE LAYER (M1 Mac Ultra)                            │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  OBS Studio                                             │
│  ├─ Scene 1: Camera quad (4K mosaic)                   │
│  ├─ Scene 2: Single camera + caller                    │
│  ├─ Scene 3: Screen share (Blender)                    │
│  └─ Audio Input: MOTU 8PRE-ES (Dante interface)        │
│                                                         │
│  Dante Network (audio sync)                            │
│  ├─ MOTU 8PRE-ES (Dante I/O)                           │
│  ├─ Dante AVIO nodes (camera audio converters)         │
│  └─ Sample clock: 48 kHz (broadcast standard)          │
│                                                         │
├─────────────────────────────────────────────────────────┤
│ PROCESSING LAYER                                        │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  Ableton Live (audio mixing, clock master)             │
│  ├─ Input: Camera feeds (via Dante)                    │
│  ├─ Master clock: Drives all Dante devices             │
│  └─ Output: Master mix → MOTU → OBS                    │
│                                                         │
│  Blender (graphics, optional)                          │
│  ├─ Input: Generative parameters (MIDI from Ableton)  │
│  └─ Output: Screen share → OBS                         │
│                                                         │
├─────────────────────────────────────────────────────────┤
│ OUTPUT / ENCODING LAYER                                 │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  OBS Encoder (H.264 or H.265)                          │
│  ├─ Video bitrate: 8–25 Mbps (codec-dependent)         │
│  ├─ Resolution: 4K or 1080p mosaic                     │
│  └─ Framerate: 30fps or 60fps                          │
│       ↓                                                  │
│  RTMP Output (streaming protocol)                      │
│       ↓                                                  │
├─────────────────────────────────────────────────────────┤
│ DISTRIBUTION LAYER                                      │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  Streaming Platforms:                                   │
│  ├─ YouTube Live                                        │
│  ├─ Twitch                                              │
│  └─ Custom RTMP server (Nginx, Wowza)                  │
│                                                         │
│  Archive & Analytics:                                   │
│  ├─ Local recording (OBS → H.264 file)                 │
│  └─ Optional: S3 backup of raw recordings              │
│                                                         │
└─────────────────────────────────────────────────────────┘

Key Components

1. Capture Subsystem (HDMI + DeckLink)

  • DeckLink Quad HDMI: 4× HDMI input, outputs to PCIe
  • Echo Express SE I: TB3 enclosure, PCIe slot for DeckLink
  • Protocol: Uncompressed HDMI or HDCP-encrypted video
  • Latency: <1ms (hardware-based capture)
  • Bandwidth: ~1.5 Gbps per 4K30 stream

2. Audio Synchronization (Dante)

Why Dante?

  • All audio sources share a global sample clock
  • Avoids drift between video and audio
  • Supports flexible routing without hardware reconfiguration
  • Industry standard for broadcast

Components:

  • Dante Network: Dedicated managed Ethernet switch
  • MOTU 8PRE-ES: Primary Dante I/O, master clock source
  • Dante AVIO USB: Bridges USB audio devices to Dante network
  • Sample Rate: 48 kHz (locked to Ableton’s clock)

Clock Hierarchy:

Ableton Live (Master Clock, 48 kHz)
    ↓ sync to
Dante Network (all devices phase-locked)
    ↓ sync to
Camera audio converters (lock to Dante clock, eliminate drift)

3. Caller Integration (NDI)

  • Protocol: NDI (Network Device Interface)
  • Latency: <200ms local, <500ms internet
  • Caller machine: OBS instance streaming via NDI
  • Your OBS: NDI input plugin (receives caller video)
  • Audio path: Separate Dante return (or Zoom audio as fallback)

4. Video Mixing (OBS)

Responsibilities:

  • Ingest 4 HDMI feeds (DeckLink)
  • Ingest 1–4 NDI feeds (callers)
  • Compose into scenes (mosaic, PiP, full-screen)
  • Apply transitions, overlays, text
  • Encode to H.264/H.265
  • Output to RTMP server

Resource allocation:

  • GPU: ~50–70% (encoding)
  • CPU: ~20–30% (scene mixing, transitions)
  • RAM: <10 GB (typical)

5. Audio Mixing (Ableton Live)

Responsibilities:

  • Input: 4 camera feeds (via Dante interface)
  • Mixing: EQ, gain, effects per camera
  • Master: Acts as Dante clock master
  • Output: Stereo mix → MOTU → OBS

Performance Targets

Metric Target Measured (Jan 2025)
Video latency (capture→stream) <5 sec 2–3 sec (RTMP platform-dependent)
Audio sync drift <100ms over 1 hour <50ms observed
Caller latency (RTT) <500ms 150–200ms (Dante local)
Dropped frames 0 0 (2-hour test)
CPU sustained load <70% 50–60% (M1 Ultra)
GPU load <80% 65–70% (H.265 encoding)
Network jitter <10ms <2ms (dedicated Dante switch)

Scaling: Single Room → Entire House

Phase 1: Single Setup (Current, 4 cameras)

M1 Ultra (all processing)
    ├─ Video: DeckLink Quad (4 cameras)
    ├─ Audio: MOTU 8PRE-ES + Dante switch (cameras + mics)
    └─ Output: RTMP to YouTube/Twitch

Limits:

  • 4 simultaneous camera inputs (DeckLink Quad limit)
  • CPU: 60–70% sustained load
  • Network: Single gigabit upstream (25–50 Mbps)

Phase 2: Multi-Room Expansion (8–10 cameras)

Room A: Local OBS instance
    └─ Camera 1–4 + local audio
    └─ Output: NDI to master M1 Ultra

Room B: Local OBS instance
    └─ Camera 5–8 + local audio
    └─ Output: NDI to master M1 Ultra

Master (M1 Ultra):
    ├─ Input: NDI streams from rooms
    ├─ Audio: Dante network (per-room mixing)
    └─ Output: Composite RTMP to YouTube

New Requirements:

  • Multiple machines (M1 mac mini per room)
  • Dante redundant network (dual switches)
  • Higher bandwidth (100 Mbps+ upstream)

Phase 3: Building-Scale Reality Show

[Building with 10–15 rooms]
    ├─ Each room: Local capture + NDI out
    ├─ Central control: Master M1 Ultra
    ├─ Dante backbone: Redundant audio network
    ├─ Archive server: NAS or S3 backup
    └─ Analytics dashboard: Viewer engagement tracking

Reliability & Redundancy

Current (Single M1 Ultra)

Single point of failure: M1 Ultra crash → entire stream down

Mitigation:

  • Regular thermal monitoring (avoid throttle)
  • Staged restart: Dante → DeckLink → OBS → Ableton
  • Local recording (continuous backup)

Production (Multi-Room)

Redundancy layers:

  1. Network: Dual Dante switches (RSTP auto-failover)
  2. Encoding: Two M1 Ultras (active/passive or load-balanced)
  3. Archive: Dual storage paths (local + S3)
  4. Streaming: Multiple RTMP keys (YouTube + Twitch simultaneously)

Monitoring & Observability

Key metrics to track (during broadcast):

  • OBS frame drops (target: 0)
  • Dante clock phase (should show <1µs deviation)
  • MOTU CPU/DSP load (target: <80%)
  • Upstream bandwidth utilization
  • Caller RTT latency

Tools:

  • macOS Activity Monitor (CPU, memory, network)
  • OBS built-in stats (FPS, dropped frames, render latency)
  • Dante Controller (network status, device sync)
  • MOTU Console (audio levels, DSP load)

Last Updated: 2025-01-22