corvid-dart

corvid-dart is the Dart binding: it links the engine’s published FFI artifacts (the platform cdylib and corvid.h) through dart:ffi, with the raw layer generated by ffigen from the release’s own header (committed and drift-gated in CI), and carries an idiomatic Dart API on top. Deliberately the corvid-c/corvid-go pattern (a fetched, checksummed shared library), not the node/python one (Rust-source builds): ./fetch.sh (macOS/Linux) or ./fetch.ps1 (Windows) downloads and sha256-verifies the pinned release archive, dart test does the rest — no Rust toolchain, no vendored binaries. Dart VM on desktop (macOS, Linux, Windows x64; macOS + Linux arm64), CI-verified on all three operating systems; Flutter plugin packaging is a documented follow-up with a trigger (see the repo’s docs/PLAN.md platform story).

When to choose this binding: your project is Dart — a VM app, a CLI, a server — and you want corvid embedded as a plain pub dependency: documents are Map<String, Object?>, vectors are Float32List, keys are Uint8List, and every failure is a CorvidException.

The idiom mapping

C ABI	corvid-dart
opaque handles (corvid_db*, …)	Db / Collection / Query classes with explicit idempotent close() and a NativeFinalizer backstop — close deliberately, the finalizer only keeps a leak from pinning engine memory
CORVID_ERR + thread-local last error	CorvidException carrying the ABI code + message, read off the thread-local slot immediately after the failing call
frozen enums	Metric, Quant, FieldType, … re-declared idiomatic Dart enums (exact ABI values)
consumed-by-call args (pred trees, builders)	Predicate/Query consumed by their terminal (filter/and/run/aggregates); the finalizer is detached on consumption, close() frees an abandoned builder
borrowed views (_ref buffers, row docs, callback keys)	copies at the boundary — every key/doc/vector is copied into Dart-owned memory inside the wrapper call before the next next() or free; nothing borrowed is retained
corvid_update_fn / corvid_scan_fn	ordinary Dart closures over NativeCallable.isolateLocal trampolines; a thrown exception is captured (never unwinding through native frames), the scan/update is aborted at the ABI level, and the original error + stack trace is rethrown at the call site
strings / bytes / vectors	String (UTF-8 at the boundary), Uint8List, Float32List — NaN/±inf/-0.0 cross bit-exact
documents	Map<String, Object?> / List<Object?> / null / bool / int (int64) / double

The raw ffigen layer stays importable as package:corvid_dart/src/... (internal); the public package:corvid_dart/corvid.dart leaks no dart:ffi type in any signature (the ABI’s ruling 3).

Install

From the pinned release artifacts:

./fetch.sh                     # fetch + sha256-verify corvid v0.4.1 into deps/current
dart pub get
dart test                      # the golden suite (267 lines) + supplemental contracts
dart run examples/quickstart.dart

Requirements: the Dart SDK ^3.10 floor; CI runs the stable channel and 3.10.0 across linux/macos/windows (the toolchain policy — modern minimums, latest + previous in CI). Until the pub.dev publish is announced, consume from git (publish_to: none today):

dependencies:
  corvid_dart:
    git: https://github.com/corvid-db/corvid-dart.git

Documents, maps, and phrases

Engine v0.3.0’s ABI additions are first-class here:

• Complete map decode — decoding a document enumerates every map’s keys through corvid_value_map_keys (the §4.4 string cursor), so whatever wrote the data, the Dart side sees the whole document. The mapkeys_test.dart contract pins the across-a-reopen shape.
• Collection.phraseSearch(field, phrase, k) — the DIRECT positional search: consecutive, in-order analyzed tokens, stop words collapsing out of adjacency ("leaps over a sleeping" matching "jumps over a sleeping dog"-style text), rows carrying the BM25 phrase score (the phrase scale, not the builder’s fused RRF scale); k == 0 answers empty — inert, never an error. The text_search example demonstrates all of it, CJK bigram phrases included.

The examples

Six runnable programs under the repo’s examples/ directory (dart run examples/<name>.dart), executed on every CI leg with deterministic output: quickstart, hybrid (the flagship below), vector_index (in-memory / on-disk / binary-quantized HNSW vs the exact scan, plus a close/reopen), text_search (BM25 incl. CJK bigram segmentation, plus the v0.3.0 direct phraseSearch), graph (neighbors/traverse + delete cascade), and geo (radius / bbox / nearest with haversine kilometres). The quickstart and hybrid sources are embedded below — imported from the repo so they cannot drift from what CI executes (scripts/sync-binding-examples.sh; the drift gate reddens docs CI if they diverge).

Quickstart

void main() {
  final db = Db.openMemory();
  final docs = db.collection('docs');

  docs.insert(kb('p1'), {
    'title': 'rust embedded database',
    'kind': 'doc',
    'v': Float32List.fromList([1.0, 0.0]),
  });
  docs.insert(kb('p2'), {
    'title': 'python web frameworks',
    'kind': 'doc',
    'v': Float32List.fromList([0.0, 1.0]),
  });
  docs.insert(kb('p3'), {
    'title': 'rust again database',
    'kind': 'doc',
    'v': Float32List.fromList([0.9, 0.1]),
  });

  // kNN: the 3 nearest documents to (1, 0) under cosine. Row.doc is
  // materialized only under select — retrieval rows carry keys and
  // scores, so select the field the printout needs.
  final rows = docs
      .query()
      .vector('v', Float32List.fromList([1.0, 0.0]), 3, Metric.cosine)
      .select(['title'])
      .run();
  var rank = 0;
  for (final r in rows) {
    print('${++rank}. ${String.fromCharCodes(r.key)} '
        'score=${r.score.toStringAsFixed(6)} ${r.doc}');
  }

  docs.close();
  db.close();
}

Hybrid retrieval

void main() {
  final db = Db.openMemory();
  final docs = db.collection('docs');

  docs.insert(kb('s1'), {
    'kind': 'doc',
    'body': 'rust embedded database',
    'v': Float32List.fromList([1.0, 0.0]),
  });
  docs.insert(kb('s2'), {
    'kind': 'doc',
    'body': 'python web frameworks',
    'v': Float32List.fromList([0.0, 1.0]),
  });
  docs.insert(kb('s3'), {
    'kind': 'doc',
    'body': 'rust again database',
    'v': Float32List.fromList([0.9, 0.1]),
  });
  docs.insert(kb('m1'), {'kind': 'meta'}); // filtered out below

  // The flagship query: filter + vector + text, RRF + MMR + limit.
  final rows = docs
      .query()
      .filter(field('kind').eq('doc'))
      .vector('v', Float32List.fromList([1.0, 0.0]), 2, Metric.cosine)
      .text('body', 'rust database', 2)
      .fuseRRF(60)
      .rerankMMR(1.0)
      .limit(2)
      .select(['body'])
      .run();
  var rank = 0;
  for (final r in rows) {
    print('${++rank}. ${String.fromCharCodes(r.key)} '
        'score=${r.score.toStringAsFixed(6)} ${r.doc}');
  }

  docs.close();
  db.close();
}

Vector indexes (ANN vs exact)

void main() {
  final path = '${Directory.systemTemp.path}/corvid-dart-vector-index.redb';
  final f = File(path);
  if (f.existsSync()) f.deleteSync(); // reruns start clean (single-file db)

  final db = Db.open(path);
  final items = db.collection('items');
  for (final (key, v) in corpus) {
    items.insert(kb(key), {
      'v_mem': Float32List.fromList(v),
      'v_disk': Float32List.fromList(v),
      'v_q': Float32List.fromList(v),
    });
  }
  items.createVectorIndex('v_mem', Metric.cosine);
  items.createVectorIndexOnDisk('v_disk', Metric.cosine);
  items.createVectorIndexQuantized('v_q', Metric.cosine, Quant.binary);

  print('top-4 nearest to (1,0,0,0) under cosine:');
  runQuery(items, 'v_mem', false, 'exact (scan):');
  runQuery(items, 'v_mem', true, 'ann in-memory HNSW:');
  runQuery(items, 'v_disk', true, 'ann on-disk HNSW:');
  runQuery(items, 'v_q', true, 'ann binary-quantized:');
  print('(the quantized lane trades recall for a ~32x smaller index)');

  items.close();
  db.close();

  // Reopen: the on-disk graph reloads (no rebuild) and answers again.
  final db2 = Db.open(path);
  final items2 = db2.collection('items');
  runQuery(items2, 'v_disk', true, 'ann on-disk after reopen:');
  items2.close();
  db2.close();

  f.deleteSync();
}

Text search (BM25, CJK, phrases)

void main() {
  final db = Db.openMemory();
  final notes = db.collection('notes');

  for (final (key, body) in corpus) {
    notes.insert(kb(key), {'body': body});
  }
  notes.createTextIndex('body');

  search(notes, 'quick fox', 'bm25 "quick fox":');
  search(notes, 'quick dog', 'bm25 "quick dog":');
  search(notes, '城市', 'bm25 CJK 城市 (city):');
  search(notes, '数据库', 'bm25 CJK 数据库 (database):');

  phrase(notes, 'fox jumps over', 'phrase "fox jumps over":');
  phrase(notes, 'over jumps fox', 'phrase "over jumps fox" (reversed — no match):');
  phrase(notes, 'leaps over a sleeping', 'phrase with stop words collapsed:');

  notes.close();
  db.close();
}

Graph (neighbors, traverse, delete cascade)

void main() {
  final db = Db.openMemory();
  final nodes = db.collection('nodes');

  for (final key in ['ga', 'gb', 'gc']) {
    nodes.insert(kb(key), {'n': key});
  }

  nodes.link(kb('ga'), 'parent_of', kb('gb'));
  nodes.link(kb('ga'), 'parent_of', kb('gc'));
  nodes.link(kb('gb'), 'parent_of', kb('gd')); // gd never exists as a document
  nodes.linkWeighted(kb('ga'), 'route', kb('gb'), 2.5);
  nodes.linkWeighted(kb('ga'), 'route', kb('gd'), 0.75);

  final ga = kb('ga'), gb = kb('gb');

  show('neighbors(ga)', nodes.neighbors(ga, 'parent_of'));
  show('in_neighbors(gb)', nodes.inNeighbors(gb, 'parent_of'));

  final routes = nodes.neighborsWeighted(ga, 'route');
  final parts = [
    for (final r in routes)
      '${String.fromCharCodes(r.key)}=${r.weight.toStringAsFixed(2)}',
  ];
  print('${'routes from ga (weighted):'.padRight(36)} [${parts.join(' ')}]');

  show('traverse(ga, 1 hop)', nodes.traverse(ga, 'parent_of', 1));
  show('traverse(ga, 2 hops)', nodes.traverse(ga, 'parent_of', 2));

  // Delete cascade: remove gc (a document) and gd (never a document).
  print('delete gc: existed = ${nodes.delete(kb('gc'))}');
  print('delete gd: existed = ${nodes.delete(kb('gd'))} '
      '(never a document; its edges still cascade)');

  show('neighbors(ga) after deletes', nodes.neighbors(ga, 'parent_of'));
  show('neighbors(gb) after deletes', nodes.neighbors(gb, 'parent_of'));
  show('traverse(ga, 2 hops) after', nodes.traverse(ga, 'parent_of', 2));

  nodes.close();
  db.close();
}

Geo (radius, bbox, nearest)

void main() {
  final db = Db.openMemory();
  final places = db.collection('places');

  for (final (name, lat, lon) in cities) {
    places.insert(kb(name), {
      'name': name,
      'loc': [lat, lon], // the [lat, lon] array encoding
    });
  }
  places.createGeoIndex('loc');

  show('within 600km of Berlin:', places.geoWithinRadius('loc', 52.52, 13.40, 600.0));
  show('bbox 47..55N, 5..15E:', places.geoWithinBBox('loc', 47, 5, 55, 15));
  show('nearest 2 to Berlin:', places.geoNearest('loc', 52.52, 13.40, 2));

  places.close();
  db.close();
}

API at a glance

Generated from the binding’s docs/SURFACE.tsv (every engine construct at the pinned tag mapped or N/A with a reason) — regenerated by the docs sync, so it cannot drift.

API group	engine constructs	proven by
the Dart value mapping (null/bool/int/double/String/Uint8List/Float32List/List<Object?>/Map<String,Object?>)	10	golden:values.txt:VTYPE
maxNesting — the encode-side depth cap (the engine's decode bound, 128): converter-accepted == decodable, deeper graphs throw CorvidException(argument)	1	depthcap_test.dart
FieldExpr eq/ne/lt/le/gt/ge	7	golden:queries.txt:QF_*
Predicate via field()/not()	27	golden:queries.txt:QF_* + golden:mutations.txt:DELETE_IN
Metric enum (cosine/dot/l2)	4	golden:queries.txt:QVEC
Quant enum (none/binary/scalar)	4	golden:schema.txt:IDX_VEC_Q
throws CorvidException (code + message)	1	golden:mutations.txt:INSERT_ERR
CorvidException.code (CorvidErrorCode — the frozen table)	1	errcodes_test.dart
ErrDatabase (code 1)	1	TestErrorCodeTable
ErrTransaction (code 2)	1	TestErrorCodeTable
ErrTable (code 3)	1	TestErrorCodeTable
ErrStorage (code 4)	1	TestErrorCodeTable
ErrCommit (code 5)	1	TestErrorCodeTable
ErrSetDurability (code 6)	1	TestErrorCodeTable
ErrCompaction (code 7)	1	TestErrorCodeTable
ErrDecode (code 8)	1	TestErrorCodeTable
ErrCorruptIndex (code 9)	1	TestErrorCodeTable
ErrReservedCollection (code 10)	1	TestErrorCodeTable; golden:mutations.txt:INSERT_ERR(err:10)
ErrInvalidName (code 11)	1	TestErrorCodeTable; golden:mutations.txt:INSERT_ERR(err:11)
ErrArgument (code 12)	1	TestErrorCodeTable; golden:mutations.txt:UPDATE_ABORT(err:12)
ErrIncompatibleFormat (code 13)	1	TestErrorCodeTable
ErrEmptyIndexTraining (code 14)	1	TestErrorCodeTable; golden:schema.txt:IDX_PQ_ERR(err:14)
ErrSchemaViolation (code 15)	1	TestErrorCodeTable; golden:schema.txt:SCHEMA_ERR(err:15)
ErrInvalidDump (code 16)	1	TestErrorCodeTable
ErrBackupTargetExists (code 17)	1	TestErrorCodeTable; golden:admin.txt:BACKUP_DUP(err:17)
ErrIO (code 18)	1	TestErrorCodeTable
Row { key, doc, score }	1	golden:queries.txt
Query (Collection.query())	2	golden:queries.txt
Query.filter	1	golden:queries.txt:QF_COUNT
Query.vector	1	golden:queries.txt:QVEC
Query.text	1	golden:queries.txt:QTEXT
Query.fuseRRF	1	golden:queries.txt:HYBRID_F
Query.rerankMMR	1	golden:queries.txt:HYBRID
Query.limit	1	golden:queries.txt:ORDER_BY
Query.offset	1	golden:queries.txt:ORDER_BY
Query.orderBy	1	golden:queries.txt:ORDER_BY
Query.approx	1	golden:queries.txt:APPROX
Query.select	1	golden:queries.txt:SELECT
Query.count	1	golden:queries.txt:AGG_COUNT
Query.groupCount	1	golden:queries.txt:AGG_GCOUNT
Query.sum	1	golden:queries.txt:AGG_SUM
Query.avg	1	golden:queries.txt:AGG_AVG
Query.min	1	golden:queries.txt:AGG_MIN
Query.max	1	golden:queries.txt:AGG_MAX
Query.countDistinct	1	golden:queries.txt:AGG_DISTINCT
Query.groupSum	1	golden:queries.txt:AGG_GSUM
Query.groupAvg	1	golden:queries.txt:AGG_GAVG
Query.run	1	golden:queries.txt:QVEC
Db	1	golden:admin.txt:FILEDB
Db.open/openMemory/collection/collections/backup/compact	6	golden:admin.txt (COLLECTIONS/BACKUP/COMPACT)
Collection	1	golden:mutations.txt:COLL
Collection insert/update/patch/compareAndSet	4	golden:mutations.txt (INSERT/UPDATE/PATCH/CAS)
Collection.scan(callback) — return false for the early stop	1	golden:mutations.txt:SCAN/SCAN_STOP
Collection.length (length()==0 for empty)	2	golden:mutations.txt:LEN
Collection.putMany	1	golden:mutations.txt:PUTMANY + golden:schema.txt:PUTMANY_ROLLBACK
Collection.insertAuto	1	golden:mutations.txt:INSERT_AUTO
Collection.get	1	golden:mutations.txt:GET
Collection delete/deleteWhere/deleteBatch	3	golden:mutations.txt (DELETE/DELETE_WHERE/DELETE_BATCH)
Collection.scan	1	golden:mutations.txt:SCAN
Collection.page -> Page { rows, next }	2	golden:mutations.txt:PAGE
Row.score (Query.vector().run())	1	golden:queries.txt:QVEC
Row.score (Query.text().run())	1	golden:queries.txt:QTEXT
Collection.phraseSearch(field, phrase, k) — the direct positional search (corvid_phrase_search, v0.3.0) over the rows cursor	1	golden:queries.txt:PHRASE
Query.fuseRRF (RRF constant, e.g. 60)	1	golden:queries.txt:HYBRID
GeoHit { Key, Doc, DistanceKm }	1	golden:geo.txt:RADIUS/NEAREST/BBOX
Collection geoWithinRadius/geoNearest/geoWithinBBox/createGeoIndex	4	golden:geo.txt (RADIUS/NEAREST/BBOX/IDX_GEO)
Collection link/linkWeighted/unlink/neighbors/inNeighbors/neighborsWeighted/traverse	7	golden:graph.txt
Collection.createScalarIndex/createCompoundIndex/createTextIndex[/OnDisk]/createGeoIndex/createVectorIndex* (6 variants)	10	golden:schema.txt:IDX_*
FieldType enum (any/boolean/integer/float/text/bytes/vector/array/map)	10	golden:schema.txt:SET_SCHEMA/SCHEMA
Collection.setSchema/schema + FieldDef { name, type, required, unique }	10	golden:schema.txt:SET_SCHEMA/SCHEMA/SCHEMA_ERR
Collection insertTTL/setTTL/getTTL/purgeExpired	4	golden:mutations.txt (INSERT_TTL/SET_TTL/GET_TTL/PURGE)
Db dump/load/loadWithRenames	3	golden:admin.txt (DUMP/LOAD/LOAD_RENAMES)

158 engine constructs are deliberately not exposed (each with its reason in the repo’s docs/SURFACE.tsv).

API reference

pub.dev renders the API from the doc comments: pub.dev/documentation/corvid.

The correctness floor

dart test replays the engine’s entire golden fixture suite — 267 executable lines across 8 files, including the v0.3.0 VMAP_KEYS (map-key iteration) and PHRASE/PHRASE_K0 (direct positional search) lines — against the downloaded cdylib, through the binding (test/golden_test.dart): every counted line must dispatch, the first failure names file:line + OP + expected-vs-got, and one SMOKE <file> lines=<n> executed=<n> line per fixture proves the dispatch count. Supplemental contracts pin the §1.6 callback exception-surfacing (callback_test.dart — a thrown error surfaces verbatim at the call site and the engine stays usable), the complete across-a-reopen map decode (mapkeys_test.dart), and the frozen 0..19 error table (errcodes_test.dart). The fixtures are vendored in the repo and byte-compared against the release’s copies at fetch time, so a bad artifact is a loud fetch failure, never a silent skip.

On top sits docs/SURFACE.tsv — every construct of the engine’s public surface (331 rows at this pin) resolved to the Dart API exposing it plus the golden line that proves it, or N/A with the ABI’s §9 reason, gated in CI (scripts/surface-gate.sh). A fourth CI job regenerates the ffigen layer (lib/src/bindings.dart) from the freshly fetched header and byte-diffs the commit, so an engine pin bump can never change the ABI surface silently.

Next: corvid-js.