GitHub - shannonxu-2018/PistaDB: The embedded vector database for LLM-native applications.

The embedded vector database for LLM-native applications.
RAG-ready · Zero dependencies · Single-file storage · MIT Licensed

Documentation & Guides → shannonxu-2018.github.io/PistaDB

Every LLM application eventually needs a vector database. For retrieval-augmented generation (RAG), semantic search, agent memory, or embedding caches — the standard answer is a cloud service or a containerized cluster. PistaDB disagrees.

Small, dense, and full of value — like the nut it's named after — PistaDB gives you a production-grade vector store in a single .pst file and a C library with zero dependencies. Ship it inside a desktop app. Bundle it in an edge device. Drop it next to your Python script. No Docker. No API keys. No data leaving the machine.

Why PistaDB?

	PistaDB	Cloud / Server Vector DB
Deployment	Copy a `.dll` / `.so`	Docker, Kubernetes, cloud subscriptions
Storage	One `.pst` file	Separate data + WAL + config + sidecar files
Privacy	All data stays local	Embeddings sent over the network
Memory	Configurable, minimal	GBs of JVM / runtime overhead
Dependencies	None (pure C99)	Dozens of packages
Latency	Sub-millisecond on a laptop	Network round-trips
Cost	Free forever (MIT)	Per-query or per-vector pricing

PistaDB is purpose-built for local RAG pipelines, offline AI agents, privacy-sensitive applications, edge inference, and anywhere shipping a full vector database cluster is impractical — which, honestly, is most places.

Key Features

8 Production-Ready Index Algorithms

Index	Algorithm	Best For
`LINEAR`	Brute-force exact scan	Ground truth, small embedding sets
`HNSW`	Hierarchical Navigable Small World	Recommended for RAG — best speed/recall tradeoff
`IVF`	Inverted File Index (k-means)	Large knowledge bases with a training budget
`IVF_PQ`	IVF + Product Quantization	Memory-constrained deployments
`DISKANN`	Vamana graph (DiskANN)	Billion-scale embedding collections
`LSH`	Locality-Sensitive Hashing	Ultra-low memory footprint
`SCANN`	Anisotropic Vector Quantization (Google ScaNN)	Maximum recall on MIPS / cosine workloads
`SQ`	Scalar Quantization (uint8)	4× memory & storage savings, no training needed

5 Distance Metrics — All LLM Embedding Models Covered

Metric	LLM / Embedding Use Case
`COSINE`	Text embeddings — OpenAI `text-embedding-3`, Cohere, `sentence-transformers`, BGE, GTE
`IP`	Inner product — embeddings already L2-normalised (same result as cosine, faster)
`L2`	Image / multimodal embeddings (CLIP, ImageBind)
`L1`	Sparse feature vectors, BM25-style hybrid retrieval
`HAMMING`	Binary embeddings, hash-based deduplication

Production Feature Set

SIMD-accelerated distance kernels — AVX2+FMA on x86-64, NEON on ARM, runtime-dispatched (4–8× scalar)
VecStore chunked storage — no scale ceiling; verified at 10 M vectors (HNSW) and 9 M full CRUD (IVF)
Multi-modal retrieval — multiple named vector fields per record + payload blob + hybrid search with Reciprocal-Rank-Fusion ranker; crash-safe via dedicated MM-WAL (see below)
Transactions — ACID-style atomic multi-op groups with full undo-on-failure rollback
Multi-threaded batch insert — thread-pool + ring-buffer API for high-throughput embedding pipelines
Embedding cache — persistent LRU cache (.pcc) that eliminates redundant model calls
Single-file storage — CRC32-verified .pst format (lookup-table accelerated); atomic save, no partial writes
O(1) vector count — cached active-vector count maintained on insert/delete, no linear scans
Hardened internals — bitset bounds checks, heap empty-access guards, HNSW neighbor bounds validation on file load
12 language bindings — C, C++, Python, Go, Java, Kotlin, Swift, Objective-C, C#, Rust, Julia, WASM
169 / 169 tests passing across all features and platforms

Capabilities & Limitations

A precise picture of what PistaDB does and does not do, so callers can size the upstream layer correctly.

What PistaDB does provide

O(1) id point CRUD. pistadb_insert / update / delete / get all dispatch through a per-handle open-addressing hash map (IdMap, ≤ 50% load factor) in src/utils.c. Single-row update is not O(n); it is amortised O(1) on every index variant (LINEAR, HNSW, IVF, IVF_PQ, DiskANN, LSH, ScaNN, SQ).
- Caveat: ScaNN's get does an O(L) inverted-list walk only when reconstructing the raw vector — id→slot lookup itself is still O(1).
- Caveat: DiskANN update may pay graph-maintenance cost on top of the O(1) lookup.
Partial update in the MultiModal layer (pdb_mm_update, src/pistadb_mm.h). The patch record carries only the fields you supply — other vector fields are untouched, and the payload supports a KEEP_PAYLOAD sentinel (preserve), explicit bytes (replace), or None (clear).
K-NN vector search via 8 index algorithms and 5 distance metrics (see Key Features above).
Hybrid multi-field search with Reciprocal-Rank-Fusion ranking across named vector fields (pdb_mm_hybrid_search).
Transactions (pistadb_txn_*) for atomic multi-op groups under a single exclusive lock.
Crash safety via per-handle WAL (.wal) and MM-WAL, with replay on open and idempotent checkpoint markers.

What PistaDB does not provide — upstream must supply

No full-table enumeration / iterator / scan API. There is no pistadb_list_ids, pistadb_iter, or pistadb_scan in either the core (pistadb.h) or the multi-modal (pistadb_mm.h) headers. The application must maintain its own id set if it needs to walk all records; PistaDB only answers point lookups by id and k-NN by vector.
No secondary / attribute index. Every query is k-NN by vector. There is no WHERE category='X' style filter. Two workarounds, both pushed up to the caller:
- Encode the attribute in label (≤ 255 B) or in the MM payload, then over-fetch top-K′ and filter in application code.
- Maintain an external id→attribute map upstream.
No partial update of label in the core API. pistadb_update(db, id, vec) only replaces vector data; changing only the label requires delete + insert. (The MultiModal layer does support per-field / per-payload patching — see above.)
No graph-traversal primitives. PistaDB is a vector database, not a graph database. The HNSW / DiskANN proximity graphs are private index structures and are not exposed as adjacency APIs. Application-level graphs (knowledge graphs, citation graphs, document linkage) must be stored upstream — this is a property of vector databases in general, not a PistaDB-specific O(n) limitation.

Common upstream patterns

Need	Pattern
"List all records I have stored"	Track the id set in the caller (a `set[int]`, a SQLite sidecar, etc.) at insert/delete time
"Search within category X"	Store category in `label` / MM payload; over-fetch (e.g. k′ = 4k) then filter
"Update only the label"	Read vector via `get`, then `delete` + `insert` with the new label
"Walk a knowledge graph"	Store adjacency upstream (relational DB, JSON sidecar); use PistaDB only for the vector-similarity hops

Language & Platform Support

Language	Binding mechanism	Where to find it
C / C++	Direct `#include`	`src/pistadb.h` / `wrap/cpp/pistadb.hpp`
Python	`ctypes` (no Cython)	`wrap/python/`
Go	CGO	`wrap/go/`
Java	JNI	`wrap/android/src/main/java/`
Kotlin	JNI + extension functions	`wrap/android/src/main/kotlin/`
Objective-C	Direct C interop	`wrap/ios/Sources/PistaDBObjC/`
Swift	ObjC bridge	`wrap/ios/Sources/PistaDB/`
C#	P/Invoke	`wrap/csharp/`
Rust	FFI (`extern "C"`)	`wrap/rust/`
Julia	`ccall` / `Libdl`	`wrap/julia/PistaDB/`
WASM	Emscripten / Embind	`wrap/wasm/`

Platform	Library output	ABI targets
Windows	`pistadb.dll`	x86_64
Linux	`libpistadb.so`	x86_64, aarch64
macOS	`libpistadb.dylib`	x86_64, arm64
Android	`libpistadb_jni.so`	arm64-v8a, armeabi-v7a, x86_64, x86
iOS / macOS	Static library (SPM)	arm64, arm64-Simulator, x86_64-Simulator
WASM	`.wasm`	— (planned)
ESP32 / MCU	`libpistadb.a` (ESP-IDF component)	xtensa-esp32-s3, esp32-c series (experimental)

Installation

1. Build the C Library

Windows (MSVC):

scripts\windows\build.bat Release

Linux (GCC / Clang):

bash scripts/linux/build.sh Release

macOS (Apple Clang):

bash scripts/macos/build.sh Release

Each script auto-detects the host architecture and copies the artifact into libs/<os>/<arch>/ (e.g. libs/linux/x86_64/libpistadb.so). The legacy build.bat / build.sh at the repo root still work — they forward to the per-OS scripts. Produced library has zero external dependencies.

Alternative: single-file amalgamation (SQLite-style). The entire engine can be generated as one pistadb.c + one pistadb.h — drop the two files into any project and compile with no special flags (SIMD kernels are enabled per-function and selected at runtime):

python3 tools/mkamalgamation.py      # writes amalgamation/pistadb.c + pistadb.h
cc -O2 -c amalgamation/pistadb.c     # zero-flag compile; link -lpthread -lm

The same output is available via the amalgamation CMake target, and -DPISTADB_BUILD_FROM_AMALGAMATION=ON builds the regular library targets from the single file to validate it against the split sources.

2. Install the Python Binding

pip install -e wrap/python/

The wrapper auto-discovers libs/<os>/<arch>/ at import time, so no environment variable is required when working inside this checkout.

Using PistaDB from a separate Python project? See INTEGRATION.md for the end-to-end Linux deployment flow (vendoring, PISTADB_LIB_DIR / PISTADB_LIB_PATH, Docker recipe).

No Rust compiler. No CMake for the Python step. No surprises.

3. Android Integration

Open wrap/android/ as a library module in Android Studio, or declare it in settings.gradle:

include ':android'
project(':android').projectDir = new File('<path-to-PistaDB>/wrap/android')

The NDK build is handled automatically by wrap/android/CMakeLists.txt. Ensure NDK 26.x is installed and ndkVersion in wrap/android/build.gradle matches.

4. iOS / macOS Integration (Swift Package Manager)

In Xcode: File → Add Package Dependencies → point to this repository (or local checkout). Or add to your own Package.swift:

.package(path: "../PistaDB")

The Package.swift at the project root declares three targets — CPistaDB (C core), PistaDBObjC, and PistaDB (Swift) — wired together automatically by SPM.

5. WASM Integration

source /path/to/emsdk/emsdk_env.sh
cd wrap/wasm && bash build.sh
# → wrap/wasm/build/pistadb.js + pistadb.wasm

Serve both files from the same HTTP origin, or use directly in Node.js.

6. C++ Integration

add_subdirectory(PistaDB)
add_subdirectory(PistaDB/wrap/cpp)
target_link_libraries(my_app PRIVATE pistadb_cpp)

7. Go Integration

// go.mod
replace pistadb.io/go => ../PistaDB/wrap/go

export CGO_LDFLAGS="-L../PistaDB/build -lpistadb"
go get pistadb.io/go/pistadb
go build ./...

8. Rust Integration

cd wrap/rust
PISTADB_LIB_DIR=../../build cargo build --release

9. C# / .NET Integration

<!-- In your .csproj -->
<ItemGroup>
  <ProjectReference Include="../PistaDB/wrap/csharp/PistaDB.csproj" />
</ItemGroup>

# Windows: copy pistadb.dll next to your executable
copy build\Release\pistadb.dll MyApp\bin\Debug\net8.0\

# Linux: set LD_LIBRARY_PATH or copy libpistadb.so
export LD_LIBRARY_PATH=$PWD/build:$LD_LIBRARY_PATH

10. ESP-IDF / Embedded Integration (experimental)

# In your ESP-IDF project's top-level CMakeLists.txt, before project(...)
set(EXTRA_COMPONENT_DIRS /path/to/PistaDB/components/pistadb)

Add pistadb to your component's REQUIRES, then mount a filesystem (LittleFS / SPIFFS / FAT) before calling pistadb_open — the library uses stdio paths under the mount point. The component sets PISTADB_EMBEDDED automatically, which selects the FreeRTOS mutex backend, disables batch insert / transactions, and shrinks the vector chunk size for MCU RAM budgets. See src/pistadb_config.h for the individual knobs.

Quick Start

import numpy as np
from pistadb import PistaDB, Metric, Index, Params

params = Params(hnsw_M=16, hnsw_ef_construction=200, hnsw_ef_search=50)
db = PistaDB("mydb.pst", dim=1536, metric=Metric.COSINE, index=Index.HNSW, params=params)

vec = np.random.rand(1536).astype("float32")
db.insert(1, vec, label="chunk_0001")

query = np.random.rand(1536).astype("float32")
results = db.search(query, k=10)
for r in results:
    print(f"id={r.id}  dist={r.distance:.4f}  label={r.label!r}")

db.save()
db.close()

# Context manager — auto-closed on exit
with PistaDB("docs.pst", dim=768, metric=Metric.COSINE) as db:
    db.insert(1, vec, label="document excerpt")
    results = db.search(query, k=5)
    db.save()

For more examples — RAG pipelines, agent memory, advanced indexes, transactions, batch insert, embedding cache, and per-language integration guides — see the docs below.

Schema-Based Collections (Milvus-style)

The base PistaDB API stores (id, label, vector) triples — perfect when your metadata fits in a 256-byte label. When you need multiple typed fields per row (section, key, language, line number, token count, …) on top of the embedding, the Collection layer adds a Milvus-compatible schema API:

FieldSchema / CollectionSchema / DataType — declare INT64, VARCHAR, FLOAT, DOUBLE, BOOL, JSON, FLOAT_VECTOR fields with is_primary / auto_id / max_length / dim semantics that mirror pymilvus line for line.
Collection.insert(rows) — accepts a list of dicts keyed by field name, validates types and lengths, auto-generates ids when auto_id=True.
Collection.search(query, k, output_fields=…) — returns hits enriched with the projected scalar columns.
JSON sidecar (<path>.meta.json) — vectors stay in the .pst file; scalar fields go to a sibling JSON file with a stable wire format, so a collection created from one language opens cleanly from any other.

import numpy as np
from pistadb import (
    FieldSchema, DataType, create_collection,
    Metric, Index,
)

fields = [
    FieldSchema("lc_id",      DataType.INT64,        is_primary=True, auto_id=True),
    FieldSchema("lc_section", DataType.VARCHAR,      max_length=100),
    FieldSchema("lc_key",     DataType.VARCHAR,      max_length=200),
    FieldSchema("lc_lang",    DataType.VARCHAR,      max_length=10),
    FieldSchema("lc_lineno",  DataType.INT64),
    FieldSchema("lc_tokens",  DataType.INT64),
    FieldSchema("lc_vector",  DataType.FLOAT_VECTOR, dim=1536),
]
coll = create_collection(
    "common_text", fields, "Common text search",
    metric=Metric.COSINE, index=Index.HNSW, base_dir="./db",
)

ids = coll.insert([
    {"lc_section": "common", "lc_key": "btn_ok",
     "lc_lang": "en", "lc_lineno": 12, "lc_tokens": 3,
     "lc_vector": np.random.rand(1536).astype("float32")},
])

hits = coll.search(query, limit=10, output_fields=["lc_key", "lc_lang"])[0]
for h in hits:
    print(h.id, h.distance, h["lc_key"], h["lc_lang"])

coll.flush()                       # persist .pst + sidecar
coll.close()

A complete runnable port of the typical Milvus create_database() snippet lives at examples/example_schema.py.

The same API is available in every language wrapper:

// Go — wrap/go/pistadb/schema.go
fields := []pistadb.FieldSchema{
    {Name: "lc_id", DType: pistadb.DTypeInt64, IsPrimary: true, AutoID: true},
    {Name: "lc_vector", DType: pistadb.DTypeFloatVector, Dim: 1536},
}
coll, _ := pistadb.CreateCollection("common_text", fields, "...",
    pistadb.CollectionOptions{Metric: pistadb.MetricCosine, Index: pistadb.IndexHNSW})
ids, _ := coll.Insert([]map[string]any{{"lc_vector": vec}})
hits, _ := coll.Search(query, 10, nil)

// Rust — cargo build --features schema
use pistadb::schema::{create_collection, CollectionOptions, DataType, FieldSchema};
use pistadb::{IndexType, Metric};

let fields = vec![
    FieldSchema { name: "lc_id".into(),     dtype: DataType::Int64,
                  is_primary: true, auto_id: true,    ..Default::default() },
    FieldSchema { name: "lc_section".into(),dtype: DataType::VarChar,
                  max_length: Some(100),             ..Default::default() },
    FieldSchema { name: "lc_vector".into(), dtype: DataType::FloatVector,
                  dim: Some(1536),                    ..Default::default() },
];
let coll = create_collection("common_text", fields, "Common text search",
    CollectionOptions { metric: Metric::Cosine, index: IndexType::HNSW,
                        base_dir: Some("./db".into()), ..Default::default() })?;

// C# — wrap/csharp/Collection.cs
var fields = new[] {
    new FieldSchema("lc_id",     DataType.Int64,       isPrimary: true, autoId: true),
    new FieldSchema("lc_section",DataType.VarChar,     maxLength: 100),
    new FieldSchema("lc_vector", DataType.FloatVector, dim: 1536),
};
var coll = Collection.Create("common_text", fields, "Common text search",
    metric: Metric.Cosine, indexType: IndexType.HNSW, baseDir: "./db");
var ids = coll.Insert(new[] {
    new Dictionary<string, object?> {
        ["lc_section"] = "common",
        ["lc_vector"]  = vec,
    },
});

// C++ — #include "pistadb_schema.hpp"
using namespace pistadb;
std::vector<FieldSchema> fields = {
    { "lc_id",     DataType::Int64,       /*primary=*/true, /*auto_id=*/true   },
    { "lc_section",DataType::VarChar,     false, false, /*max_length=*/100      },
    { "lc_vector", DataType::FloatVector, false, false, std::nullopt, /*dim=*/1536 },
};
auto coll = create_collection("common_text", std::move(fields), "Common text search",
    { Metric::Cosine, IndexType::HNSW, std::nullopt, std::string("./db") });
coll.insert({{ {"lc_section", Value::str("common")},
               {"lc_vector",  Value::floats(vec)} }});

// Java — wrap/android/.../Collection.java
List<FieldSchema> fields = Arrays.asList(
    new FieldSchema.Builder("lc_id",     DataType.INT64).primary(true).autoId(true).build(),
    new FieldSchema.Builder("lc_vector", DataType.FLOAT_VECTOR).dim(1536).build());
Collection coll = Collection.create("common_text", fields, "...",
    new Collection.Options().metric(Metric.COSINE).index(IndexType.HNSW));

// Kotlin DSL — wrap/android/.../CollectionExtensions.kt
val coll = collection("common_text", fields = listOf(
    field("lc_id",     DataType.INT64) { primary(true).autoId(true) },
    field("lc_vector", DataType.FLOAT_VECTOR) { dim(1536) },
)) { metric = Metric.COSINE; index = IndexType.HNSW }

// Swift — wrap/ios/Sources/PistaDB/PistaDBSchema.swift
let fields: [FieldSchema] = [
    try FieldSchema(name: "lc_id",     dtype: .int64,       isPrimary: true, autoId: true),
    try FieldSchema(name: "lc_section",dtype: .varchar,     maxLength: 100),
    try FieldSchema(name: "lc_vector", dtype: .floatVector, dim: 1536),
]
let coll = try createCollection(
    name: "common_text", fields: fields, description: "Common text search",
    options: .init(metric: .cosine, indexType: .hnsw, baseDir: "./db"))

// WASM — import from pistadb_schema.js, then attachSchema(M)
const fields = [
    new M.FieldSchema("lc_id",     M.DataType.INT64,        { isPrimary: true, autoId: true }),
    new M.FieldSchema("lc_section",M.DataType.VARCHAR,      { maxLength: 100 }),
    new M.FieldSchema("lc_vector", M.DataType.FLOAT_VECTOR, { dim: 1536 }),
];
const coll = M.createCollection("common_text", fields, "Common text search", {
    metric: M.Metric.Cosine, indexType: M.IndexType.HNSW,
});
coll.insert([{ lc_section: "common", lc_vector: new Float32Array(1536) }]);

Schema rules: exactly one is_primary field (must be INT64), exactly one FLOAT_VECTOR field (with positive dim), unique field names. Validation happens at construction — the same constraints in every wrapper.

Multi-Modal Retrieval (Hybrid Search)

Real applications often need to search multiple modalities at once — a product with text + image embeddings, a clip with audio + video, a document with dense + sparse representations. The MultiModal layer adds first-class support for multiple named vector fields per record plus Reciprocal-Rank-Fusion (RRF) hybrid search, while keeping the zero-dependency and single-machine character of the core library.

What you get:

Multi-vector schema — up to 16 named fields per record, each with its own dim, metric, and index algorithm (HNSW for text + IVF for image, for example). Each field is internally a standard .pst file, so you can mix any of the 8 index types freely.
Atomic multi-field writes — mm.insert(id, vecs={...}) either commits every field, payload, and catalog entry, or rolls every one of them back. Powered by a dedicated MM-layer WAL that survives os._exit / power loss.
Hybrid search with RRF — parallel per-field k-NN, fused by score = Σ 1 / (k + rank_i) with configurable rrf_k (default 60).
Binary payload per record — bring your own caption / URL / tags as a blob; readers borrow the bytes from an mmap-style read buffer.
Backward-compatible — does not change the .pst file format or any existing API. Old single-modal .pst files keep working unchanged.

On-disk layout — a bundle is a directory:

mybundle.pmm/
├── pmm.manifest          (128-byte header + schema)
├── pmm.catalog           (64-byte rows, id → flags/mask/payload_off/label)
├── pmm.payload           (append-only blob log; compacted at checkpoint)
├── pmm.wal               (canonical multi-field WAL; CRC32-tail-safe replay)
└── fields/<name>.pst     (one standard PistaDB file per field)

Python example:

import numpy as np
from pistadb import MultiModal, FieldSpec, Metric, Index, Params

mm = MultiModal.create("products.pmm", [
    FieldSpec("text",  dim=384, metric=Metric.COSINE, index_type=Index.HNSW),
    FieldSpec("image", dim=512, metric=Metric.COSINE, index_type=Index.IVF),
])
mm.train_field("image")     # IVF needs training before insert

mm.insert(
    id=1,
    label="red leather wallet",
    payload=b'{"sku":"W-1042","price":49.99}',
    vecs={
        "text":  text_emb,     # numpy float32, shape (384,)
        "image": image_emb,    # numpy float32, shape (512,)
    },
)

# Search both modalities in parallel and fuse with RRF:
hits = mm.hybrid_search(
    {"text":  (query_text_emb,  20),     # per-field top-k before fusion
     "image": (query_image_emb, 20)},
    top_k=10, rrf_k=60, parallel=True,
)
for h in hits:
    print(h.id, h.score, h.label, mm.get_payload(h.id))

mm.checkpoint()              # snapshot + compact + truncate WAL
mm.close()

C API — declared in src/pistadb_mm.h; 14 entry points (pdb_mm_create / _open / _close / _save / _checkpoint / _insert / _update / _delete / _get / _hybrid_search / _train_field / _count / _schema / _last_error). Same opaque-handle pattern as the existing pistadb_batch.h / pistadb_txn.h / pistadb_cache.h modules.

What's not in MVP: scalar-metadata filtering (category == 'shoes' AND price < 100), sparse vector fields (BM25 / SPLADE), late-interaction models (ColBERT / BGE-M3). These are on the roadmap; vote with an issue.

Running Tests

# Windows
set PISTADB_LIB_DIR=build\Release
pytest tests\ -v

# Linux / macOS
PISTADB_LIB_DIR=build pytest tests/ -v

169 / 169 tests passing — recall benchmarks, roundtrip persistence, corrupt-file rejection, metric correctness, ScaNN two-phase search, transaction atomicity / rollback, plus the new multi-modal suite (tests/test_pistadb_mm.py) covering schema round-trip, hybrid-search RRF fusion, crash recovery from os._exit, and torn-WAL-tail tolerance.

Shuckr — Visual Database Browser

Shuckr is a standalone GUI tool for visually browsing and managing PistaDB .pst files — inspired by DB Browser for SQLite. Built with Python + PyQt6, it talks to the compiled native library (pistadb.dll / libpistadb.so) via ctypes.

Features: Create / open .pst files · Browse vectors with pagination · Insert / edit / delete vectors · k-NN search with random query generation · Database metadata & raw header inspection · Unsaved-changes tracking

Quick Start

cd Shuckr
pip install -r requirements.txt
python main.py

Or on Windows, simply double-click run.bat.

Screenshots

Database Info	Browse Data	Search

Documentation

Document	Contents
docs/examples.md	RAG pipelines, agent memory, all index types, transactions, batch insert, embedding cache
docs/language-bindings.md	Android, iOS/macOS, .NET, WASM, C++, Rust, Go — full integration guides
docs/benchmarks.md	Large-scale CRUD benchmarks, SIMD details, file format, project structure

Roadmap

Multi-modal retrieval — multiple named vector fields per record + RRF hybrid search (shipped)
Scalar-metadata filtering on multi-modal bundles (filter by tag / category / numeric range)
Sparse vector field (BM25 / SPLADE) for hybrid dense+sparse retrieval
Late-interaction multi-vector field (ColBERT / BGE-M3 MaxSim)
LangChain and LlamaIndex integration (drop-in vectorstore)
Full in-browser RAG pipeline via WASM (IDBFS persistence, SharedArrayBuffer workers)
HTTP microserver mode (optional, single binary, for multi-process access)

Contributing

Pull requests are warmly welcomed. Whether it's a new index algorithm, a language binding, a performance improvement, an LLM integration, or documentation — every contribution makes PistaDB better for the whole community.

Fork the repository
Create your feature branch (git checkout -b feat/langchain-integration)
Commit your changes
Open a Pull Request

Please ensure all 169 tests continue to pass before submitting.

Built in C99 · C++ · WASM · Python · Go · Java · Kotlin · Swift · Objective-C · C# · Rust · Julia · Runs anywhere · Keeps your data private
The best infrastructure for an LLM app is the kind you never have to think about.

Navigation Menu

Repository files navigation

Why PistaDB?

Key Features

8 Production-Ready Index Algorithms

5 Distance Metrics — All LLM Embedding Models Covered

Production Feature Set

Capabilities & Limitations

What PistaDB does provide

What PistaDB does not provide — upstream must supply

Common upstream patterns

Language & Platform Support

Installation

1. Build the C Library

2. Install the Python Binding

3. Android Integration

4. iOS / macOS Integration (Swift Package Manager)

5. WASM Integration

6. C++ Integration

7. Go Integration

8. Rust Integration

9. C# / .NET Integration

10. ESP-IDF / Embedded Integration (experimental)

Quick Start

Schema-Based Collections (Milvus-style)

Multi-Modal Retrieval (Hybrid Search)

Running Tests

Shuckr — Visual Database Browser

Quick Start

Screenshots

Documentation

Roadmap

Contributing

About

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github		.github
Shuckr		Shuckr
benchmarks		benchmarks
components/pistadb		components/pistadb
docs		docs
example_dbs		example_dbs
examples		examples
img		img
libs		libs
scripts		scripts
skills/pistadb		skills/pistadb
src		src
tests		tests
tools		tools
website		website
wrap		wrap
.gitignore		.gitignore
CMakeLists.txt		CMakeLists.txt
DEPLOY.md		DEPLOY.md
INTEGRATION.md		INTEGRATION.md
LICENSE		LICENSE
Package.swift		Package.swift
README.md		README.md
README_CN.md		README_CN.md
build.bat		build.bat
build.sh		build.sh
build_error.txt		build_error.txt
build_output.txt		build_output.txt
build_vs.bat		build_vs.bat

Latest commit

History

Folders and files

Repository files navigation

Why PistaDB?

Key Features

8 Production-Ready Index Algorithms

5 Distance Metrics — All LLM Embedding Models Covered

Production Feature Set

Capabilities & Limitations

What PistaDB does provide

What PistaDB does not provide — upstream must supply

Common upstream patterns

Language & Platform Support

Installation

1. Build the C Library

2. Install the Python Binding

3. Android Integration

4. iOS / macOS Integration (Swift Package Manager)

5. WASM Integration

6. C++ Integration

7. Go Integration

8. Rust Integration

9. C# / .NET Integration

10. ESP-IDF / Embedded Integration (experimental)

Quick Start

Schema-Based Collections (Milvus-style)

Multi-Modal Retrieval (Hybrid Search)

Running Tests

Shuckr — Visual Database Browser

Quick Start

Screenshots

Documentation

Roadmap

Contributing

About

Topics

Resources

Stars

Watchers

Forks

Contributors

Languages