Noundry.Streams

High-performance, self-contained event stream ingestion and relay service. Capture webhooks, buffer events, and relay to downstream systems with 50K+ messages/second throughput and sub-millisecond latency.

Enterprise-Grade Performance

Built for high-throughput event streaming with durable persistence and guaranteed delivery.

50K+
Messages per second
<1ms
Ingestion latency (p99)
Zero
Message loss
Crash
Safe persistence

Powerful Features

Everything you need for reliable event stream processing and relay.

High Throughput

Lock-free channels, batched writes, and optimized SQLite persistence deliver exceptional performance.

  • 50,000+ messages/second
  • Sub-millisecond latency
  • Configurable batching

Durable Persistence

SQLite with WAL mode provides crash-safe persistence with configurable retention policies.

  • Zero message loss
  • Crash recovery
  • Configurable retention

Smart Relay

Built-in relay workers with exponential backoff retry logic and parallel processing.

  • Automatic retries
  • Multiple endpoints
  • Ordered or parallel

Use Cases

Perfect for event-driven architectures and webhook processing.

Webhook Ingestion

Capture webhooks from Stripe, GitHub, Twilio, and other services with guaranteed delivery.

Event Streaming

Build event pipelines between microservices with durable buffering.

Audit Logging

Durable audit trails with replay capability for compliance requirements.

API Rate Limiting

Buffer requests and relay at controlled rates to downstream APIs.

Service Decoupling

Decouple producers from consumers with reliable message buffering.

Data Replication

Replicate events to multiple downstream systems reliably.

Architecture

Simple, powerful design for reliable event streaming.

HTTP Requests ───> Ingest Channel ───> BatchWriter ───> SQLite DB
                                                                      
                                                                      
                                                              RelayChannel
                                                                      
                                                                      
                                                              RelayWorkers
                                                                      
                                                                      
                                                         External Endpoints

Ingestion Flow

  1. 1 HTTP POST received at endpoint
  2. 2 Message queued to ingest channel
  3. 3 202 Accepted returned (<1ms)
  4. 4 BatchWriter persists to SQLite

Relay Flow

  1. 1 Persisted messages forwarded to relay channel
  2. 2 RelayWorkers POST to configured endpoints
  3. 3 Failed deliveries retried with backoff
  4. 4 Attempts logged for observability

Installation

Deploy as a standalone service with zero external dependencies.

Docker

$ docker run -p 5000:5000 -v streams-data:/data noundry/streams

Direct Download

# Linux/macOS
$ curl -sSL https://noundry.com/install/streams.sh | bash
# Windows
$ iwr https://noundry.com/install/streams.ps1 | iex

.NET Tool

$ dotnet tool install -g Noundry.Streams

Requirements

  • No external dependencies - SQLite is embedded
  • Cross-platform: Linux, macOS, Windows
  • .NET 9.0 runtime (included in self-contained builds)

Usage Examples

Simple REST API for stream management and event ingestion.

1. Create a Stream

$ curl -X POST http://localhost:5000/admin/streams \
  -H "Content-Type: application/json" \
  -d '{
    "streamId": "webhook-events",
    "displayName": "Webhook Events",
    "retentionHours": 72,
    "orderingRequired": false,
    "relayWorkerCount": 4
  }'

# Response: 201 Created

2. Configure Relay Endpoint

$ curl -X POST http://localhost:5000/admin/streams/webhook-events/relays \
  -H "Content-Type: application/json" \
  -d '{
    "endpointUrl": "https://my-backend.com/process",
    "httpMethod": "POST",
    "headers": {"Authorization": "Bearer token123"},
    "timeoutMs": 5000,
    "retryAttempts": 3,
    "retryBackoffMs": 1000
  }'

# Response: 201 Created

3. Ingest Messages

Path-based routing

$ curl -X POST http://localhost:5000/streams/webhook-events/ingest \
  -H "Content-Type: application/json" \
  -d '{"event": "user.signup", "userId": 12345}'

# Response: 202 Accepted (<1ms)

Header-based routing

$ curl -X POST http://localhost:5000/ingest \
  -H "X-Stream-Id: webhook-events" \
  -H "Content-Type: application/json" \
  -d '{"event": "order.created"}'

# Response: 202 Accepted

4. Monitor Metrics

$ curl http://localhost:5000/admin/streams/webhook-events/metrics

# Response:
{
  "streamId": "webhook-events",
  "totalMessagesReceived": 125430,
  "totalMessagesRelayed": 125428,
  "pendingMessages": 2,
  "failedRelays": 0,
  "avgLatencyMs": 0.42
}

Configuration

Flexible configuration via JSON or environment variables.

appsettings.json

{
  "NoundryStreams": {
    "DataPath": "./data/noundry-streams.db",
    "ApiKey": "your-secret-key",
    "Defaults": {
      "ChannelCapacity": 10000,
      "BatchIntervalMs": 50,
      "BatchMaxSize": 500,
      "RetentionHours": 168,
      "RelayWorkerCount": 4
    },
    "Sqlite": {
      "CacheSizeMB": 256,
      "MmapSizeMB": 1024
    }
  },
  "Kestrel": {
    "Endpoints": {
      "Http": { "Url": "http://0.0.0.0:5000" }
    }
  }
}

Performance Settings

  • ChannelCapacity: Ingest buffer size (default: 10000)
  • BatchIntervalMs: Max batch wait (default: 50ms)
  • BatchMaxSize: Messages per batch (default: 500)
  • RelayWorkerCount: Parallel relays (default: 4)

Storage Settings

  • DataPath: SQLite database location
  • RetentionHours: Message retention (default: 168)
  • CacheSizeMB: SQLite page cache (default: 256)
  • MmapSizeMB: Memory-mapped I/O (default: 1024)

Observability

Built-in OpenTelemetry support for distributed tracing.

OpenTelemetry Tracing

Per-stream distributed tracing with OTLP/gRPC export to your observability platform.

  • Trace ingestion to relay
  • Jaeger, Zipkin, Tempo compatible
  • Per-stream configuration

Structured Logging

Serilog-based structured logging with configurable sinks and log levels.

  • JSON log output
  • Request correlation IDs
  • ELK, Seq, Loki compatible

API Reference

Complete REST API for stream management and ingestion.

Ingestion Endpoints

POST /streams/{streamId}/ingest Primary ingestion endpoint
POST /ingest Header/query-based routing

Admin Endpoints

POST /admin/streams Create stream
GET /admin/streams List all streams
GET /admin/streams/{streamId} Get stream details
POST /admin/streams/{streamId}/relays Add relay config
GET /admin/streams/{streamId}/metrics Get stream metrics

Health Endpoints

GET /health Liveness check
GET /health/ready Readiness check

Ready for Reliable Event Streaming?

Deploy Noundry.Streams today and handle 50K+ messages/second with sub-millisecond latency and zero message loss.