btrdb - B-tree DataBase
btrdb is a NoSQL database engine with Copy-on-Write inspired by btrfs.
- Deno runtime
- Node.js runtime
- Compatibility layer
- ES module bundle
- CommonJS bundle
- Publish to NPM registry
- B-Tree
- Single file
- Fully Copy-on-Write and log-structured
- Good performance even written in pure TypeScript
- Snapshots
- Named snapshots
- Key-Value sets
- Document sets
- Auto-id
- Indexes
- BSON instead of JSON on disk (?)
- ACID
- Readers/writer lock
- Isolation with concurrent reader on snapshots
- Client / Server (?)
- Replication (?)
- GC (?)
- Auto-commit (?)
⚠️ Warning ⚠️
This project is just started. It's under heavy development!
The on-disk format structure and the API are NOT stable yet.
Please do NOT use it in any serious production.
Usage
Import the module
Deno:
import { Database } from "https://github.com/lideming/btrdb/raw/main/mod.ts";
Node.js:
Install from NPM registry:
npm i @yuuza/btrdb
Import from ES module:
import { Database } from "@yuuza/btrdb";
Import from CommonJS module:
import("@yuuza/btrdb").then(({ Database }) => {
// Use Database here...
});
Create/open database file
const db = new Database();
await db.openFile("data.db");
// Will create new database if the file doesn't exist.
Use key-value set
const configSet = await db.createSet("config");
// Get the set or create if not exist.
await configSet.set("username", "yuuza");
console.info(await configSet.get("username")); // "yuuza"
await db.commit();
// Commit to persist the changes.
Use document set
Create set
interface User {
id: number; // A property named "id" is required.
username: string;
status: "online" | "offline";
}
const userSet = await db.createSet<User>("users", "doc");
// Get the set or create if not exist.
Insert
await userSet.insert({ username: "yuuza", status: "offline" });
// Insert a new document, auto id when it's not specified.
console.info(await userSet.get(1));
// { id: 1, username: "yuuza", status: "offline" }
await db.commit();
// Commit to persist the changes.
Upsert
upsert
will update the document with the same id, or insert a new document if
the id does not exist.
const user = await userSet.get(1);
user.status = "online";
// Get user and set its status
await userSet.upsert(user);
// Use upsert to apply the change.
console.info(await userSet.get(1));
// { id: 1, username: "yuuza", status: "online" }
await db.commit();
// Commit to persist the changes.
Indexes
interface User {
id: number;
username: string;
status: "online" | "offline";
role: "admin" | "user";
}
const userSet = await db.createSet<User>("users", "doc");
// Define indexes on the set and update indexes if needed.
userSet.useIndexes({
status: (u) => u.status,
// define "status" index, which indexing the value of user.status for each user in the set
username: { unique: true, key: (u) => u.username },
// define "username" unique index, which does not allow duplicated username.
onlineAdmin: (u) => u.status == "online" && u.role == "admin",
// define "onlineAdmin" index, the value is a computed boolean.
});
await userSet.insert({ username: "yuuza", status: "online", role: "admin" });
await userSet.insert({ username: "foo", status: "offline", role: "user" });
await userSet.insert({ username: "bar", status: "online", role: "admin" });
await db.commit();
// Get all online users
console.info(await userSet.findIndex("status", "online"));
// [
// { username: "yuuza", status: "online", role: "user", id: 1 },
// { username: "bar", status: "online", role: "admin", id: 3 }
// ]
// Get all users named 'yuuza'
console.info(await userSet.findIndex("username", "yuuza"));
// [ { username: "yuuza", status: "online", role: "user", id: 1 } ]
// Get all online admins
console.info(await userSet.findIndex("onlineAdmin", true));
// [ { username: "bar", status: "online", role: "admin", id: 3 } ]
Use snapshots
Since btrdb uses CoW and never overwrites data on-disk, creating "snapshot" have almost no cost.
const dataSet = await db.createSet("data");
await dataSet.set("foo", "bar");
// Commit then create a "named snapshot"
await db.createSnapshot("backup");
await dataSet.set("someone", "messed up your data!");
await dataSet.set("foo", "no bar!");
await db.commit();
// Get a "named snapshot".
const snap = await db.getSnapshot("backup");
// Read data from the snapshot
console.info(await snap.getSet("data").get("foo"));
Also, db.getPrevCommit()
can be used to get previous commit as a snapshot.
More example in the test code
See test.ts.
Design
(Outdated. To be added: documents tree, indexes tree, data pages, named snapshots)
License
MIT License