Audio Package

Audio processing framework for speech and multimedia applications.

Design Goals

1. Real-time Processing: Low-latency audio mixing, encoding, and streaming
2. Format Flexibility: Support common audio formats (PCM, Opus, MP3, OGG)
3. Cross-platform: FFI bindings to native libraries (libopus, libsoxr, lame)
4. Streaming-first: Designed for continuous audio streams, not just files

Architecture

graph TB
    subgraph audio["audio/"]
        subgraph row1[" "]
            pcm["pcm/<br/>- Format<br/>- Chunk<br/>- Mixer"]
            codec["codec/<br/>- opus/<br/>- mp3/<br/>- ogg/"]
            resampler["resampler/<br/>- soxr<br/>- Format<br/>- Convert"]
        end
        subgraph row2[" "]
            opusrt["opusrt/<br/>- Buffer<br/>- Realtime<br/>- OGG R/W"]
            songs["songs/<br/>- Catalog<br/>- Notes<br/>- PCM gen"]
            portaudio["portaudio/<br/>(Go only)<br/>- Stream<br/>- Device"]
        end
    end

Submodules

Audio Formats

PCM Formats (Predefined)

Codec Support

Common Workflows

Voice Chat (Low Latency)

flowchart LR
    A[Microphone] --> B[PCM 16kHz]
    B --> C[Opus Encode]
    C --> D[Network]
    D --> E[Opus Decode]
    E --> F[Mixer]
    F --> G[Speaker]

Speech Synthesis Playback

flowchart LR
    A[API Response<br/>Base64 MP3] --> B[MP3 Decode]
    B --> C[Resample<br/>24K→16K]
    C --> D[Mixer]
    D --> E[Speaker]

Audio Recording

flowchart LR
    A[PCM Stream] --> B[Opus Encode]
    B --> C[OGG Writer]
    C --> D[File]

Native Dependencies

Examples Directory

• examples/go/audio/ - Go audio examples
• examples/rust/audio/ - Rust audio examples

Related Packages

• buffer - Used for audio data buffering
• speech - High-level speech synthesis/recognition
• minimax, doubaospeech - TTS/ASR APIs returning audio