JSPM

  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 128
  • Score
    100M100P100Q102137F

EJDB - Embedded JSON Database engine

Package Exports

  • ejdb

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (ejdb) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

EJDB - Embedded JSON Database engine

It aims to be a fast MongoDB-like library which can be embedded into C/C++ applications under terms of LGPL license.

EJDB is the C library based on modified version of Tokyo Cabinet.

JSON representation of queries and data implemented with API based on C BSON

Features

  • LGPL license allows you to embed this library into proprietary software.
  • MongoDB-like queries and overall philosophy.
  • Collection level write locking.
  • Collection level transactions.
  • String token matching queries: $stror $strand
  • Node.js binding

Documentation

Community

We use EJDB Google group as our mailing list.

NodeJS binding

One snippet intro

var EJDB = require("ejdb");
//Open zoo DB
var jb = EJDB.open("zoo", EJDB.DEFAULT_OPEN_MODE | EJDB.JBOTRUNC);

var parrot1 = {
    "name" : "Grenny",
    "type" : "African Grey",
    "male" : true,
    "age" : 1,
    "birthdate" : new Date(),
    "likes" : ["green color", "night", "toys"],
    "extra1" : null
};
var parrot2 = {
    "name" : "Bounty",
    "type" : "Cockatoo",
    "male" : false,
    "age" : 15,
    "birthdate" : new Date(),
    "likes" : ["sugar cane"]
};

jb.save("parrots", [parrot1, parrot2], function(err, oids) {
    if (err) {
        console.error(err);
        return;
    }
    console.log("Grenny OID: " + parrot1["_id"]);
    console.log("Bounty OID: " + parrot2["_id"]);

    jb.find("parrots",
            {"likes" : "toys"},
            {"$orderby" : {"name" : 1}},
            function(err, cursor, count) {
                if (err) {
                    console.error(err);
                    return;
                }
                console.log("Found " + count + " parrots");
                while (cursor.next()) {
                    console.log(cursor.field("name") + " likes toys!");
                }
                cursor.close(); //It's not mandatory to close cursor explicitly
                jb.close(); //Close the database
            });
});

Installation

System libraries:

  • g++
  • cunit
  • zlib

On Debian/Ubuntu linux you can install it as follows:

sudo apt-get install g++ libcunit1 libcunit1-dev zlib1g zlib1g-dev

Installation from node package manager:

npm install ejdb

EJDB NodeJS samples

EJDB NodeJS API

### EJDB.open(dbFile, openMode)

Open database. Return database instance handle object.
Default open mode: JBOWRITER | JBOCREAT.
This is blocking function.

Arguments

  • {String} dbFile Database main file name
  • {Number} [openMode=JBOWRITER | JBOCREAT] Bitmast of open modes: - JBOREADER Open as a reader. - JBOWRITER Open as a writer. - JBOCREAT Create if db file not exists - JBOTRUNC Truncate db.
 ### close()

Close database.
If database was not opened it does nothing.
This is blocking function.

 ### isOpen() Check if database in opened state.
### ensureCollection(cname, copts)

Automatically creates new collection if it does't exists. Collection options copts applied only for newly created collection. For existing collections copts takes no effect.
This is blocking function.

Arguments

  • {String} cname Name of collection.
  • {Object} [copts] Collection options.
 ### removeCollection(cname, prune, cb)

Remove collection.

Call variations:

removeCollection(cname)
removeCollection(cname, cb)
removeCollection(cname, prune, cb)

Arguments

  • {String} cname Name of collection.
  • {Boolean} [prune=false] If true the collection data will erased from disk.
  • {Function} [cb] Callback args: (error)
 ### save(cname, jsarr, cb)

Save/update specified JSON objects in the collection. If collection with cname does not exists it will be created.

Each persistent object has unique identifier (OID) placed in the _id property. If a saved object does not have _id it will be autogenerated. To identify and update object it should contains _id property.

Call variations:

save(cname, json object, [cb])
save(cname, <Array of json objects>, [cb])

Arguments

  • {String} cname Name of collection.
  • {Array|Object} jsarr Signle JSON object or array of JSON objects to save
  • {Function} [cb] Callback args: (error, {Array} of OIDs for saved objects)
 ### load(cname, oid, cb)

Loads JSON object identified by OID from the collection.

Arguments

  • {String} cname Name of collection
  • {String} oid Object identifier (OID)
  • {Function} cb Callback args: (error, obj) obj: Retrieved JSON object or NULL if it is not found.
 ### remove(cname, oid, cb)

Removes JSON object from the collection.

Arguments

  • {String} cname Name of collection
  • {String} oid Object identifier (OID)
  • {Function} cb Callback args: (error)
 ### find(cname, qobj, orarr, hints, cb) Execute query on collection. EJDB queries inspired by MongoDB (mongodb.org) and follows same philosophy. 
Supported queries:
  - Simple matching of String OR Number OR Array value:
      -   {'json.field.path' : 'val', ...}
  - $not Negate operation.
      -   {'json.field.path' : {'$not' : val}} //Field not equal to val
      -   {'json.field.path' : {'$not' : {'$begin' : prefix}}} //Field not begins with val
  - $begin String starts with prefix
      -   {'json.field.path' : {'$begin' : prefix}}
  - $gt, $gte (>, >=) and $lt, $lte for number types:
      -   {'json.field.path' : {'$gt' : number}, ...}
  - $bt Between for number types:
      -   {'json.field.path' : {'$bt' : [num1, num2]}}
  - $in String OR Number OR Array val matches to value in specified array:
      -   {'json.field.path' : {'$in' : [val1, val2, val3]}}
  - $nin - Not IN
  - $strand String tokens OR String array val matches all tokens in specified array:
      -   {'json.field.path' : {'$strand' : [val1, val2, val3]}}
  - $stror String tokens OR String array val matches any token in specified array:
      -   {'json.field.path' : {'$stror' : [val1, val2, val3]}}
  - $exists Field existence matching:
      -   {'json.field.path' : {'$exists' : true|false}}
  - $icase Case insensitive string matching:
      -  {'json.field.path' : {'$icase' : 'val1'}} //icase matching
      Ignore case matching with '$in' operation:
      -  {'name' : {'$icase' : {'$in' : ['tHéâtre - театр', 'heLLo WorlD']}}}
       For case insensitive matching you can create special type of string index.

Queries can be used to update records:
  `$set` and `$inc` operations are supported.
  `$set` Field set operation.
       - {some fields for selection, '$set' : {'field1' : {obj}, ...,  'field1' : {obj}}}
   `$inc` Increment operation. Only number types are supported.
       - {some fields for selection, '$inc' : {'field1' : number, ...,  'field1' : {number}}

NOTE: It is better to execute update queries with `$onlycount=true` hint flag
     or use the special `update()` method to avoid unnecessarily rows fetching.

NOTE: Negate operations: $not and $nin not using indexes
      so they can be slow in comparison to other matching operations.

NOTE: Only one index can be used in search query operation.

QUERY HINTS (specified by `hints` argument):
  - $max Maximum number in the result set
  - $skip Number of skipped results in the result set
  - $orderby Sorting order of query fields.
  - $onlycount true|false If `true` only count of matching records will be returned
                          without placing records in result set.
  - $fields Set subset of fetched fields
       Example:
       hints:    {
                   "$orderby" : { //ORDER BY field1 ASC, field2 DESC
                       "field1" : 1,
                       "field2" : -1
                   },
                   "$fields" : { //SELECT ONLY {_id, field1, field2}
                       "field1" : 1,
                       "field2" : 1
                   }
                 }

Many C API query examples can be found in `tcejdb/testejdb/t2.c` test case.

To traverse selected records cursor object is used:
  - Cursor#next() Move cursor to the next record and returns true if next record exists.
  - Cursor#hasNext() Returns true if cursor can be placed to the next record.
  - Cursor#field(name) Retrieve value of the specified field of the current JSON object record.
  - Cursor#object() Retrieve whole JSON object with all fields.
  - Cursor#reset() Reset cursor to its initial state.
  - Cursor#length Read-only property: Number of records placed into cursor.
  - Cursor#pos Read/Write property: You can set cursor position: 0 <= pos < length
  - Cursor#close() Closes cursor and free cursor resources. Cursor cant be used in closed state.

Call variations of find():
   - find(cname, qobj, cb)
   - find(cname, qobj, hints, cb)
   - find(cname, qobj, qobjarr, cb)
   - find(cname, qobj, qobjarr, hints, cb)

Arguments

  • {String} cname Name of collection
  • {Object} qobj Main JSON query object
  • {Array} [orarr] Array of additional OR query objects (joined with OR predicate).
  • {Object} [hints] JSON object with query hints.
  • {Function} cb Callback args: (error, cursor, count) cursor: Cursor object to traverse records count: Total number of selected records
 ### findOne(cname, qobj, orarr, hints, cb) Same as #find() but retrieves only one matching JSON object.

Call variations of findOne():

findOne(cname, qobj, cb)
findOne(cname, qobj, hints, cb)
findOne(cname, qobj, qobjarr, cb)
findOne(cname, qobj, qobjarr, hints, cb)

Arguments

  • {String} cname Name of collection
  • {Object} qobj Main JSON query object
  • {Array} [orarr] Array of additional OR query objects (joined with OR predicate).
  • {Object} [hints] JSON object with query hints.
  • {Function} cb Callback args: (error, obj) obj Retrieved JSON object or NULL if it is not found.
 ### update(cname, qobj, orarr, hints, cb) Convenient method to execute update queries. The `$set` and `$inc` operations are supported.
  • $set Field set operation:
    • {some fields for selection, '$set' : {'field1' : {obj}, ..., 'field1' : {obj}}}
  • $inc Increment operation. Only number types are supported.
    • {some fields for selection, '$inc' : {'field1' : number, ..., 'field1' : {number}}

Call variations of update():

update(cname, qobj, cb)
update(cname, qobj, hints, cb)
update(cname, qobj, qobjarr, cb)
update(cname, qobj, qobjarr, hints, cb)

Arguments

  • {String} cname Name of collection
  • {Object} qobj Update JSON query object
  • {Array} [orarr] Array of additional OR query objects (joined with OR predicate).
  • {Object} [hints] JSON object with query hints.
  • {Function} cb Callback args: (error, count) count The number of updated records.
 ### count(cname, qobj, orarr, hints, cb) Convenient count(*) operation.

Call variations of count():

count(cname, qobj, cb)
count(cname, qobj, hints, cb)
count(cname, qobj, qobjarr, cb)
count(cname, qobj, qobjarr, hints, cb)

Arguments

  • {String} cname Name of collection
  • {Object} qobj Main JSON query object
  • {Array} [orarr] Array of additional OR query objects (joined with OR predicate).
  • {Object} [hints] JSON object with query hints.
  • {Function} cb Callback args: (error, count) count: Number of matching records.
 ### sync(cb) Synchronize entire EJDB database with disk.

Arguments

  • {Function} cb Callback args: (error)
 ### dropIndexes(cname, path, cb) Drop indexes of all types for JSON field path.

Arguments

  • {String} cname Name of collection
  • {String} path JSON field path
  • {Function} [cb] Optional callback function. Callback args: (error)
 ### optimizeIndexes(cname, path, cb) Optimize indexes of all types for JSON field path. Performs B+ tree index file optimization.

Arguments

  • {String} cname Name of collection
  • {String} path JSON field path
  • {Function} [cb] Optional callback function. Callback args: (error)
 ### ensureStringIndex(cname, path, cb) ### ensureIStringIndex(cname, path, cb) ### ensureNumberIndex(cname, path, cb) ### ensureArrayIndex(cname, path, cb)

Ensure index presence of String|Number|Array type for JSON field path. IString is the special type of String index for case insensitive matching.

Arguments

  • {String} cname Name of collection
  • {String} path JSON field path
  • {Function} [cb] Optional callback function. Callback args: (error)
 ### rebuildStringIndex(cname, path, cb) ### rebuildIStringIndex(cname, path, cb) ### rebuildNumberIndex(cname, path, cb) ### rebuildArrayIndex(cname, path, cb)

Rebuild index of String|Number|Array type for JSON field path. IString is the special type of String index for case insensitive matching.

Arguments

  • {String} cname Name of collection
  • {String} path JSON field path
  • {Function} [cb] Optional callback function. Callback args: (error)
 ### dropStringIndex(cname, path, cb) ### dropIStringIndex(cname, path, cb) ### dropNumberIndex(cname, path, cb) ### dropArrayIndex(cname, path, cb)

Drop index of String|Number|Array type for JSON field path. IString is the special type of String index for case insensitive matching.

Arguments

  • {String} cname Name of collection
  • {String} path JSON field path
  • {Function} [cb] Optional callback function. Callback args: (error)

EJDB C Library

One snippet intro

#include <tcejdb/ejdb.h>

static EJDB *jb;

int main() {
    jb = ejdbnew();
    if (!ejdbopen(jb, "addressbook", JBOWRITER | JBOCREAT | JBOTRUNC)) {
        return 1;
    }
    //Get or create collection 'contacts'
    EJCOLL *coll = ejdbcreatecoll(jb, "contacts", NULL);

    bson bsrec;
    bson_oid_t oid;

    //Insert one record:
    //JSON: {'name' : 'Bruce', 'phone' : '333-222-333', 'age' : 58}
    bson_init(&bsrec);
    bson_append_string(&bsrec, "name", "Bruce");
    bson_append_string(&bsrec, "phone", "333-222-333");
    bson_append_int(&bsrec, "age", 58);
    bson_finish(&bsrec);
    //Save BSON
    ejdbsavebson(coll, &bsrec, &oid);
    fprintf(stderr, "\nSaved Bruce");
    bson_destroy(&bsrec);

    //Now execute query
    //QUERY: {'name' : {'$begin' : 'Bru'}} //Name starts with 'Bru' string
    bson bq1;
    bson_init_as_query(&bq1);
    bson_append_start_object(&bq1, "name");
    bson_append_string(&bq1, "$begin", "Bru");
    bson_append_finish_object(&bq1);
    bson_finish(&bq1);

    EJQ *q1 = ejdbcreatequery(jb, &bq1, NULL, 0, NULL);

    uint32_t count;
    TCLIST *res = ejdbqrysearch(coll, q1, &count, 0, NULL);
    fprintf(stderr, "\n\nRecords found: %d\n", count);

    //Now print the result set records
    for (int i = 0; i < TCLISTNUM(res); ++i) {
        void *bsdata = TCLISTVALPTR(res, i);
        bson_print_raw(stderr, bsdata, 0);
    }
    fprintf(stderr, "\n");

    //Dispose result set
    tclistdel(res);

    //Dispose query
    ejdbquerydel(q1);
    bson_destroy(&bq1);

    //Close database
    ejdbclose(jb);
    ejdbdel(jb);
    return 0;
}

You can save this code in csnippet.c And build:

gcc -std=c99 -Wall -pedantic  -c -o csnippet.o csnippet.c
gcc -std=c99 -Wall -pedantic  -o csnippet csnippet.o  -ltcejdb

Building & Installation

Prerequisites

System libraries:

  • gcc
  • cunit
  • zlib

On Debian/Ubuntu linux you can install it as follows:

   sudo apt-get install gcc libcunit1 libcunit1-dev zlib1g zlib1g-dev

Building

   cd ./tcejdb
   ./configure --disable-bzip --prefix=<installation prefix> && make && make check
   make install
  • library name: tcejdb (with pkgconfig)
  • main include header: <tcejdb/ejdb.h>

C API

EJDB API presented in ejdb.h C header file.

JSON processing API: bson.h

Queries

/**
 * Create query object.
 * Sucessfully created queries must be destroyed with ejdbquerydel().
 *
 * EJDB queries inspired by MongoDB (mongodb.org) and follows same philosophy.
 *
 *  - Supported queries:
 *      - Simple matching of String OR Number OR Array value:
 *          -   {'json.field.path' : 'val', ...}
 *      - $not Negate operation.
 *          -   {'json.field.path' : {'$not' : val}} //Field not equal to val
 *          -   {'json.field.path' : {'$not' : {'$begin' : prefix}}} //Field not begins with val
 *      - $begin String starts with prefix
 *          -   {'json.field.path' : {'$begin' : prefix}}
 *      - $gt, $gte (>, >=) and $lt, $lte for number types:
 *          -   {'json.field.path' : {'$gt' : number}, ...}
 *      - $bt Between for number types:
 *          -   {'json.field.path' : {'$bt' : [num1, num2]}}
 *      - $in String OR Number OR Array val matches to value in specified array:
 *          -   {'json.field.path' : {'$in' : [val1, val2, val3]}}
 *      - $nin - Not IN
 *      - $strand String tokens OR String array val matches all tokens in specified array:
 *          -   {'json.field.path' : {'$strand' : [val1, val2, val3]}}
 *      - $stror String tokens OR String array val matches any token in specified array:
 *          -   {'json.field.path' : {'$stror' : [val1, val2, val3]}}
 *      - $exists Field existence matching:
 *          -   {'json.field.path' : {'$exists' : true|false}}
 *      - $icase Case insensitive string matching:
 *          -    {'json.field.path' : {'$icase' : 'val1'}} //icase matching
 *          Ignore case matching with '$in' operation:
 *          -    {'name' : {'$icase' : {'$in' : ['tHéâtre - театр', 'heLLo WorlD']}}}
 *          For case insensitive matching you can create special index of type: `JBIDXISTR`
 *
 *  NOTE: Negate operations: $not and $nin not using indexes
 *  so they can be slow in comparison to other matching operations.
 *
 *  NOTE: Only one index can be used in search query operation.
 *
 *  QUERY HINTS (specified by `hints` argument):
 *      - $max Maximum number in the result set
 *      - $skip Number of skipped results in the result set
 *      - $orderby Sorting order of query fields.
 *      - $fields Set subset of fetched fields
 *          Example:
 *          hints:    {
 *                      "$orderby" : { //ORDER BY field1 ASC, field2 DESC
 *                          "field1" : 1,
 *                          "field2" : -1
 *                      },
 *                      "$fields" : { //SELECT ONLY {_id, field1, field2}
 *                          "field1" : 1,
 *                          "field2" : 1
 *                      }
 *                    }
 *
 * Many query examples can be found in `testejdb/t2.c` test case.
 *
 * @param EJDB database handle.
 * @param qobj Main BSON query object.
 * @param orqobjs Array of additional OR query objects (joined with OR predicate).
 * @param orqobjsnum Number of OR query objects.
 * @param hints BSON object with query hints.
 * @return On success return query handle. On error returns NULL.
 */
EJDB_EXPORT EJQ* ejdbcreatequery(EJDB *jb, bson *qobj, bson *orqobjs, int orqobjsnum, bson *hints);

EJDB C Samples

You can find some code samples in:

Basic EJDB architecture

EJDB database files structure

.
├── <dbname>
├── <dbname>_<collection1>
├── ...
├── <dbname>_<collectionN>
└── <dbname>_<collectionN>_<fieldpath>.<index ext>

Where

  • <dbname> - name of database. It is metadata DB.
  • <collectionN> - name of collection. Collection database.
  • <fieldpath> - JSON field path used in index
  • <index ext> - Collection index extension:
    • .lex String index
    • .dec Number index
    • .tok Array index

Limitations/TODOs

  • Collect collection index statistic
  • Windows port