Buffer Package - Rust Implementation

Crate: giztoy-buffer

📚 Rust Documentation

Types

Buffer

Growable buffer using VecDeque<T> for O(1) front operations.

#![allow(unused)]
fn main() {
pub struct Buffer<T> {
    inner: Arc<BufferInner<T>>,
}

struct BufferInner<T> {
    state: Mutex<BufferState<T>>,
    write_notify: Condvar,
}

struct BufferState<T> {
    buf: VecDeque<T>,
    close_write: bool,
    close_err: Option<Arc<dyn Error + Send + Sync>>,
}
}

Key Methods:

BlockBuffer

Fixed-size circular buffer with blocking semantics.

#![allow(unused)]
fn main() {
pub struct BlockBuffer<T> {
    inner: Arc<BlockBufferInner<T>>,
}

struct BlockBufferInner<T> {
    state: Mutex<BlockBufferState<T>>,
    not_full: Condvar,
    not_empty: Condvar,
}

struct BlockBufferState<T> {
    buf: Vec<Option<T>>,
    head: usize,
    tail: usize,
    count: usize,
    close_write: bool,
    close_err: Option<Arc<dyn Error + Send + Sync>>,
}
}

Key Methods:

RingBuffer

Fixed-size circular buffer with overwrite semantics.

#![allow(unused)]
fn main() {
pub struct RingBuffer<T> {
    inner: Arc<RingBufferInner<T>>,
}

struct RingBufferInner<T> {
    state: Mutex<RingBufferState<T>>,
    write_notify: Condvar,
}

struct RingBufferState<T> {
    buf: Vec<Option<T>>,
    head: usize,  // virtual counter (wraps)
    tail: usize,  // virtual counter (wraps)
    close_write: bool,
    close_err: Option<Arc<dyn Error + Send + Sync>>,
}
}

Key Methods:

Error Types

#![allow(unused)]
fn main() {
#[derive(Debug, Clone)]
pub enum BufferError {
    Closed,
    ClosedWithError(Arc<dyn Error + Send + Sync>),
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Done;
}

Convenience Functions

#![allow(unused)]
fn main() {
// Growable buffers
pub fn bytes() -> Buffer<u8>         // 1KB
pub fn bytes_1kb() -> Buffer<u8>     // 1KB
pub fn bytes_4kb() -> Buffer<u8>     // 4KB
pub fn bytes_16kb() -> Buffer<u8>    // 16KB
pub fn bytes_64kb() -> Buffer<u8>    // 64KB
pub fn bytes_256b() -> Buffer<u8>    // 256B

// Blocking buffers
pub fn block_bytes() -> BlockBuffer<u8>      // 1KB
pub fn block_bytes_1kb() -> BlockBuffer<u8>  // 1KB
pub fn block_bytes_4kb() -> BlockBuffer<u8>  // 4KB
pub fn block_bytes_16kb() -> BlockBuffer<u8> // 16KB
pub fn block_bytes_64kb() -> BlockBuffer<u8> // 64KB

// Ring buffers
pub fn ring_bytes(size: usize) -> RingBuffer<u8>
pub fn ring_bytes_1kb() -> RingBuffer<u8>    // 1KB
pub fn ring_bytes_4kb() -> RingBuffer<u8>    // 4KB
pub fn ring_bytes_16kb() -> RingBuffer<u8>   // 16KB
pub fn ring_bytes_64kb() -> RingBuffer<u8>   // 64KB
}

Thread Safety

All types implement Send + Sync and support Clone:

#![allow(unused)]
fn main() {
// Clone shares the underlying buffer via Arc
let buf = Buffer::<i32>::new();
let buf_clone = buf.clone();  // Same underlying buffer

// Safe to send to other threads
std::thread::spawn(move || {
    buf_clone.add(42).unwrap();
});
}

Usage Patterns

Producer-Consumer

#![allow(unused)]
fn main() {
use giztoy_buffer::{BlockBuffer, Done};
use std::thread;

let buf = BlockBuffer::<i32>::new(4);
let producer_buf = buf.clone();

let producer = thread::spawn(move || {
    for i in 0..100 {
        producer_buf.add(i).unwrap();
    }
    producer_buf.close_write().unwrap();
});

let mut collected = Vec::new();
loop {
    match buf.next() {
        Ok(item) => collected.push(item),
        Err(Done) => break,
    }
}

producer.join().unwrap();
}

Sliding Window

#![allow(unused)]
fn main() {
use giztoy_buffer::RingBuffer;

let buf = RingBuffer::<f32>::new(100);

// Write more than capacity - old data overwritten
for i in 0..200 {
    buf.add(i as f32).unwrap();
}

// Buffer contains only last 100 values
assert_eq!(buf.len(), 100);
let window = buf.to_vec();  // [100.0, 101.0, ..., 199.0]
}

Implementation Details

VecDeque vs Vec

• Buffer: Uses VecDeque<T> for O(1) pop_front()
• BlockBuffer/RingBuffer: Use Vec<Option<T>> for circular buffer

Wrapping Arithmetic

RingBuffer uses wrapping_add for counters to handle overflow:

#![allow(unused)]
fn main() {
state.tail = state.tail.wrapping_add(1);
if state.tail.wrapping_sub(state.head) > capacity {
    state.head = state.head.wrapping_add(1);
}
}

Dual Condvar Pattern (BlockBuffer)

BlockBuffer uses two Condvars for precise signaling:

#![allow(unused)]
fn main() {
not_full: Condvar,   // Signals writers when space available
not_empty: Condvar,  // Signals readers when data available
}

Differences from Go Implementation