SyndrDB Documentation

🔌 Connection & Configuration Reference

Complete guide to connecting to SyndrDB, configuring the server, and tuning settings for optimal performance.

📑 Table of Contents

🔗 Connection String Format

Basic Connection String

SyndrDB uses a colon-delimited connection string format:

syndrdb://host:port:database:user:password

Components:

Component Description Example
Protocol Always syndrdb:// syndrdb://
Host Server hostname or IP address localhost, 192.168.1.100
Port TCP port the server listens on 1776
Database Target database name MyDB
User Authentication username admin
Password Authentication password secret123

Example:

syndrdb://localhost:1776:MyDB:admin:secret123

💡 Tip: The default port for SyndrDB is 1776. If you haven't changed it, use this port in your connection string.

Connection with Options

Append a sixth colon-delimited segment for connection options. Multiple options are separated by commas:

syndrdb://host:port:database:user:password:option1=value1,option2=value2

Examples:

-- Enable zstd compression
syndrdb://localhost:1776:MyDB:admin:secret123:compress=zstd

-- Enable pipeline mode
syndrdb://localhost:1776:MyDB:admin:secret123:pipeline=true

-- Enable chunked streaming
syndrdb://localhost:1776:MyDB:admin:secret123:streaming=chunked

-- All options combined
syndrdb://localhost:1776:MyDB:admin:secret123:compress=zstd,pipeline=true,streaming=chunked

⚙️ Connection Options

Connection options are appended to the connection string after the password, separated by a colon. Multiple options use comma separation.

📦 compress=zstd

Enables zstd compression on the wire protocol. This reduces bandwidth usage between client and server, which is especially beneficial for large result sets or high-latency networks.

  • Values: zstd
  • Default: Disabled (no compression)
  • When to use: Remote connections, large result sets, bandwidth-constrained environments
  • Trade-off: Adds small CPU overhead for compression/decompression
syndrdb://localhost:1776:MyDB:admin:secret123:compress=zstd

🚀 pipeline=true

Enables pipeline mode for batch command execution. In pipeline mode, the server sends a READY\n sentinel after each response, allowing the client to send multiple commands without waiting for each response.

  • Values: true
  • Default: Disabled
  • When to use: Batch operations, bulk inserts, scripted workflows
  • Benefit: Eliminates round-trip latency between sequential commands
syndrdb://localhost:1776:MyDB:admin:secret123:pipeline=true

🌊 streaming=chunked

Enables chunked streaming for large result sets. Instead of buffering the entire result in memory, the server streams results in chunks as they are produced.

  • Values: chunked
  • Default: Disabled (full materialization)
  • When to use: Large SELECT queries, cursor-based iteration, memory-constrained clients
  • Benefit: Reduces memory usage on both client and server; first results arrive faster
syndrdb://localhost:1776:MyDB:admin:secret123:streaming=chunked

📡 Wire Protocol Overview

SyndrDB communicates over TCP using a custom binary-framed protocol. Understanding the wire protocol is essential for building custom clients or debugging connection issues.

Command Framing

Element Byte Value Description
Command terminator \x04 (EOT) Marks the end of a command or response
Escaped EOT \x04\x04 Represents a literal EOT character within data
Parameter delimiter \x05 (ENQ) Separates parameters within a command

Basic command flow:

Client sends:  SELECT * FROM "Users"\x04
Server sends:  [JSON result data]\x04

⚠️ Important: If your data contains the literal byte \x04, it must be escaped as \x04\x04 to avoid premature command termination.

Pipeline Protocol

When pipeline mode is enabled (pipeline=true), the server appends a READY\n sentinel after each response. This allows the client to distinguish between individual responses in a batch.

Client sends:  SELECT * FROM "Users"\x04
Client sends:  SELECT * FROM "Orders"\x04

Server sends:  [Users result]\x04
Server sends:  READY\n
Server sends:  [Orders result]\x04
Server sends:  READY\n

Streaming Protocol

When streaming mode is enabled (streaming=chunked), the server uses a chunked transfer format for large result sets.

Streaming frame format:

Frame Format Description
Header STREAM:v1\n Indicates the start of a streamed response
Data chunk CHUNK:<len>\n<data> Uncompressed data chunk with byte length
Compressed chunk ZCHUNK:<compressed>:<uncompressed>\n<data> zstd-compressed chunk with both sizes
End marker END:<count>,<timeMS>\n Total document count and execution time in ms

Example streaming flow:

Client sends:  SELECT * FROM "LargeBundle"\x04

Server sends:  STREAM:v1\n
Server sends:  CHUNK:4096\n[4096 bytes of JSON data]
Server sends:  CHUNK:4096\n[4096 bytes of JSON data]
Server sends:  CHUNK:2048\n[2048 bytes of JSON data]
Server sends:  END:50000,1250\n

With compression enabled:

Server sends:  STREAM:v1\n
Server sends:  ZCHUNK:2048:4096\n[2048 bytes of compressed data]
Server sends:  ZCHUNK:1890:4096\n[1890 bytes of compressed data]
Server sends:  END:50000,980\n

🏳️ Server CLI Flags

Start the SyndrDB server with the following command-line flags to override default settings.

./server --port 1776 --host 0.0.0.0 --datadir /var/syndrdb/data --auth --config /etc/syndrdb/config.yaml
Flag Default Description
--datadir ./database Data storage directory for bundles, indexes, and WAL files
--port 1776 TCP port the server listens on
--host localhost Bind address (use 0.0.0.0 for all interfaces)
--config none Path to YAML configuration file
--graphql false Enable the GraphQL query endpoint
--auth false Enable authentication (requires user credentials)
--mode standalone Server mode: standalone or cluster
--wal-mode sync WAL write mode: sync (fsync each op) or async (buffered)
--max-connections 100 Maximum number of concurrent client connections

Examples:

# Start with default settings
./server

# Production setup with auth, custom data directory, and config file
./server --datadir /var/syndrdb/data --auth --config /etc/syndrdb/config.yaml

# Development setup listening on all interfaces
./server --host 0.0.0.0 --port 1776 --graphql

# High-connection server with async WAL
./server --max-connections 500 --wal-mode async

💡 Tip: CLI flags take precedence over YAML configuration file values. Use flags for quick overrides and YAML for persistent configuration.

📄 YAML Configuration

For production deployments, use a YAML configuration file to manage all server settings. Pass the file path with --config:

./server --config /etc/syndrdb/config.yaml

Configuration Categories

🖥️ Server Settings

Setting Default Description
host localhost Bind address
port 1776 TCP listen port
mode standalone Server mode (standalone / cluster)
maxConnections 100 Maximum concurrent connections
tlsEnabled false Enable TLS encryption
tlsCertFile none Path to TLS certificate file
tlsKeyFile none Path to TLS private key file

💾 Storage Settings

Setting Default Description
bundleFileMaxSizeMB 32 Maximum segment file size before rotation (MB)
bundleStorageFormat binary Storage format (binary BSON only)

📝 WAL Settings

Setting Default Description
walEnabled true Enable Write-Ahead Log
walMode sync WAL write mode (sync / async)
durabilityMode balanced strict (fsync each op), balanced (group commit), performance (async flush)
groupCommit true Enable group commit for reduced fsync contention

🧱 Cache Settings

Setting Default Description
maxLoadedDocumentPages 500 Maximum document pages kept in memory
bundleAdapterMaxCachedPages 500 Maximum cached pages per bundle adapter

🔍 Query Settings

Setting Default Description
queryTimeoutSeconds 300 Maximum query execution time (seconds)
queryMaxMemoryMB 25 Maximum memory per query (MB)
planCacheCapacity 1000 Plan cache entries per shard (8 shards total)

📊 Index Settings

Setting Default Description
bTreeSyncMode batched B-Tree flush strategy (batched / immediate)
indexMaintenanceEnabled true Enable automatic index maintenance on writes

🔄 MVCC Settings

Setting Default Description
enableRCUWrites true Enable Read-Copy-Update for lock-free reads
rcuGracePeriodMs 100 RCU grace period before old versions are invisible (ms)
maxOCCRetries 3 Maximum Optimistic Concurrency Control retries

🧹 Vacuum Settings

Setting Default Description
vacuumEnabled true Enable automatic dead version reclamation
vacuumDeadRatioThreshold 0.3 Dead version ratio that triggers vacuum (0.0 - 1.0)
vacuumMaxPagesPerCycle 100 Maximum pages processed per vacuum cycle

🌊 Streaming & Cursor Settings

Setting Default Description
streamingChunkSize 256 Documents per streaming chunk
maxOpenCursorsPerSession 64 Maximum open cursors per client session
cursorIdleTimeoutSeconds 300 Idle cursor timeout before automatic cleanup

🏎️ Sort & Performance Settings

Setting Default Description
sortSIMDEnabled true Enable SIMD-accelerated sorting
sortParallelEnabled true Enable parallel sorting for large result sets
sortParallelMinSize 10000 Minimum documents before parallel sort activates

📝 Example YAML Configuration

A comprehensive example configuration file for a production deployment:

# SyndrDB Server Configuration
# =============================

# Server Settings
host: "0.0.0.0"
port: 1776
mode: "standalone"
maxConnections: 200
authEnabled: true

# TLS Settings
tlsEnabled: true
tlsCertFile: "/etc/syndrdb/certs/server.crt"
tlsKeyFile: "/etc/syndrdb/certs/server.key"

# Storage Settings
dataDir: "/var/syndrdb/data"
bundleFileMaxSizeMB: 64
bundleStorageFormat: "binary"

# WAL Settings
walEnabled: true
walMode: "sync"
durabilityMode: "balanced"
groupCommit: true

# Cache Settings
maxLoadedDocumentPages: 1000
bundleAdapterMaxCachedPages: 1000

# Query Settings
queryTimeoutSeconds: 300
queryMaxMemoryMB: 50
planCacheCapacity: 2000

# Index Settings
bTreeSyncMode: "batched"
indexMaintenanceEnabled: true

# MVCC Settings
enableRCUWrites: true
rcuGracePeriodMs: 100
maxOCCRetries: 3

# Vacuum Settings
vacuumEnabled: true
vacuumDeadRatioThreshold: 0.3
vacuumMaxPagesPerCycle: 200

# Streaming Settings
streamingChunkSize: 512
maxOpenCursorsPerSession: 64
cursorIdleTimeoutSeconds: 300

# Sort & Performance
sortSIMDEnabled: true
sortParallelEnabled: true
sortParallelMinSize: 10000

# GraphQL
graphqlEnabled: false

📋 Key Settings Reference

The most important settings at a glance, grouped by impact area.

⚡ Performance-Critical Settings

Setting Type Default Description
maxLoadedDocumentPages int 500 More pages in cache = fewer disk reads, but more memory
queryMaxMemoryMB int 25 Per-query memory limit; raise for complex aggregations
planCacheCapacity int 1000 Per-shard plan cache size; raise for diverse query workloads
sortParallelMinSize int 10000 Lower = more parallel sorts; higher = less CPU overhead
sortSIMDEnabled bool true Hardware-accelerated sorting; disable on unsupported platforms
streamingChunkSize int 256 Docs per stream chunk; larger = fewer frames, more latency

🛡️ Durability Settings

Setting Type Default Description
walEnabled bool true Disabling WAL risks data loss on crash
durabilityMode string balanced strict: safest, slowest; performance: fastest, least safe
groupCommit bool true Batches fsync calls; ~10x improvement in write throughput
maxOCCRetries int 3 Retries on transaction conflicts before aborting

📦 Storage Settings

Setting Type Default Description
bundleFileMaxSizeMB int 32 Segment file rotation size; larger = fewer files, longer compaction
vacuumEnabled bool true Automatic cleanup of dead MVCC versions
vacuumDeadRatioThreshold float 0.3 Trigger vacuum when 30%+ of page versions are dead
vacuumMaxPagesPerCycle int 100 Limits vacuum work per cycle to avoid I/O spikes

🔐 Connection & Security Settings

Setting Type Default Description
maxConnections int 100 Maximum simultaneous client connections
authEnabled bool false Require username/password authentication
tlsEnabled bool false Encrypt connections with TLS
queryTimeoutSeconds int 300 Kill queries running longer than this (5 min default)
maxOpenCursorsPerSession int 64 Prevents cursor leaks from consuming server resources

🎯 Best Practices

✅ DO:

  1. Use a YAML config file for production:

    ./server --config /etc/syndrdb/config.yaml

    Keep all settings version-controlled and documented.

  2. Enable TLS for remote connections:

    tlsEnabled: true
tlsCertFile: "/etc/syndrdb/certs/server.crt"
tlsKeyFile: "/etc/syndrdb/certs/server.key"

  3. Enable authentication in production:

    ./server --auth

    Then create users with appropriate permissions:

    CREATE USER "app_user" PASSWORD "strong_password_here"
GRANT "read" TO USER "app_user"
GRANT "write" TO USER "app_user"

  4. Use balanced durability mode for most workloads:

    It provides group commit (batched fsync) for ~10x write throughput improvement over strict mode while still maintaining crash safety.

  5. Enable compression for remote clients:

    syndrdb://db.example.com:1776:MyDB:admin:secret:compress=zstd

  6. Use streaming for large result sets:

    syndrdb://localhost:1776:MyDB:admin:secret:streaming=chunked

    Combine with cursors for maximum control:

    DECLARE my_cursor CURSOR FOR SELECT * FROM "LargeBundle" WHERE status == "active"
FETCH 100 FROM my_cursor
FETCH 100 FROM my_cursor
CLOSE my_cursor

  7. Tune cache sizes based on available memory:

    Each document page holds ~4096 documents. With maxLoadedDocumentPages: 1000, you can cache ~4 million documents in memory.

❌ DON'T:

  1. Don't use performance durability mode with critical data:

    Async WAL flush can lose the most recent transactions on crash.

  2. Don't disable WAL in production:

    # DON'T do this in production
walEnabled: false  # Risk of data loss!

  3. Don't bind to 0.0.0.0 without authentication:

    # Dangerous: open to the network without auth
./server --host 0.0.0.0

# Safe: require authentication
./server --host 0.0.0.0 --auth

  4. Don't set queryTimeoutSeconds too low:

    Complex aggregations and large joins may legitimately take time. The default of 300 seconds (5 minutes) is suitable for most workloads.

  5. Don't set maxConnections higher than your system can handle:

    Each connection consumes memory for session state, buffers, and goroutine stack. Monitor system resources when tuning this value.

🔗 Related Documentation

Version: SyndrDB 1.0
Last Updated: March 2026
Status: ✅ Implemented and tested