Package Exports

bulksearch

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (bulksearch) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

BulkSearch

Superfast, lightweight and read-write optimized full text search library.

When it comes to the overall speed, BulkSearch outperforms every searching library out there and also provides flexible search capabilities like multi-word, phonetics or partial matching. Adding, updating or removing items are also fast as searching them. When your index don't needs to be updated continuously then FlexSearch may be a better choice. BulkSearch also provides you a asynchronous processing model to perform any updates on the index in background.

Benchmark: https://jsperf.com/bulksearch

All Features:

Partial Words
Multiple Words
Flexible Word Order
Phonetic Search
Limit Results
Caching
Asynchronous Mode
Custom Matchers
Custom Encoders

Installation

Node.js

npm install bulksearch

In your code include as follows:

var BulkSearch = require("BulkSearch");

HTML / Javascript

<html>
<head>
    <script src="https://cdn.rawgit.com/nextapps-de/bulksearch/master/bulksearch.min.js"></script>
</head>
...

AMD

var BulkSearch = require("BulkSearch");

Usage (API)

Create a new index

var index = new BulkSearch();

alternatively you can also use:

var index = BulkSearch.create();

Create a new index with custom options

BulkSearch.create(options)

var index = new BulkSearch({

    // default values:

    type: "integer",
    encode: "icase",
    boolean: "and",
    size: 4000,
    depth: 3,
    multi: false,
    strict: false,
    ordered: false,
    async: false,
    cache: false
});

Add items to an index

Index.add_(id, string)

index.add(10025, "John Doe");

Search items

Index.search(string, limit, callback)

index.search("John");

Limit the result

index.search("John", 10);

Perform queries asynchronously

index.search("John", function(result){
    
    // array of results
});

Update item to the index

Index.update(id, string)

index.update(10025, "Road Runner");

Remove item to the index

Index.remove(id)

index.remove(10025);

Destroy the index

index.destroy();

Initialize the index

Index.init(options)

index.init();

Add custom matcher

Index.addMatcher(KEY_VALUE_PAIRS)

index.addMatcher({

    'ä': 'a', // replaces all 'ä' to 'a'
    'ö': 'o',
    'Ü': 'u'
});

Add custom encoder

var index = new BulkSearch({

    encode: function(str){
    
        // do something with str ...
        
        return str;
    }
});

Get info

index.info();

Returns information about the index, e.g.:

{
    "bytes": 3600356288,
    "chunks": 9,
    "fragmentation": 0,
    "fragments": 0,
    "id": 0,
    "length": 7798,
    "matchers": 0,
    "size": 10000,
    "status": false
}

Note: When the fragmentation value is about 50% or higher, your should consider using cleanup() to free all fragmented available memory.

Optimize / Cleanup index

index.cleanup();

Options

Option	Values	Description
type	"byte" "short" "integer" "float" "string"	The data type of passed IDs has to be specified on creation. It is recommended to uses to most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.
encode	false "icase" "simple" "advanced" "extra" {function}	The encoding type. Choose one of the built-ins or pass a custom encoding function.
boolean	"and" "or"	The applied boolean model when comparing multiple words. Note: When using "or" the first word is also compared with "and". Example: a query with 3 words, results has either: matched word 1 & 2 and matched word 1 & 3.
size	2500 - 10000	The size of chunks. It depends on content length which value fits best. Short content length (e.g. User names) are faster with a chunk size of 2,500. Bigger text runs faster with a chunk size of 10,000. Note: It is recommended to use a minimum chunk size of the maximum content length which has to be indexed to prevent fragmentation.
depth	0 - 6	Set the depth of register. It is recommended to use a value in relation to the number of stored index and content length for an optimum performance-memory value. Note: Increase this options carefully!
multi	true false	Enable multi word processing.
ordered	true false	Multiple words has to be the same order as the matched entry.
strict	true false	Matches exactly needs to be started with the query.
cache	true false	Enable caching.

Phonetic Encoding

Option	Description	Example	False Positives	Compression Level
false	Turn off encoding	Reference: "Björn-Phillipp Mayer" Matches: "Phil"	no	no
icase	Case in-sensitive encoding	Reference: "Björn-Phillipp Mayer" Matches: "phil"	no	no
simple	Phonetic normalizations	Reference: "Björn-Phillipp Mayer" Matches: "bjoern fillip"	no	~ 3%
advanced	Phonetic normalizations + Literal transformations	Reference: "Björn-Phillipp Mayer" Matches: "filip meier"	no	~ 25%
extra	Phonetic normalizations + Soundex transformations	Reference: "Björn-Phillipp Mayer" Matches: "byorn mair"	yes	~ 50%

Compare Phonetic Search Results

Reference String: "Björn-Phillipp Mayer"

Query	ElasticSearch	BulkSearch (iCase)	BulkSearch (Simple)	BulkSearch (Adv.)	BulkSearch (Extra)
björn	yes	yes	yes	yes	yes
björ	no	yes	yes	yes	yes
bjorn	no	no	yes	yes	yes
bjoern	no	no	no	yes	yes
philipp	no	no	no	yes	yes
filip	no	no	no	yes	yes
björnphillip	no	no	yes	yes	yes
meier	no	no	no	yes	yes
björn meier	no	no	no	yes	yes
meier fhilip	no	no	no	yes	yes
byorn mair	no	no	no	no	yes
(false positives)	yes	no	no	no	yes

Compare Memory Usage

Note: The data type of passed IDs has to be specified on creation. It is recommended to uses to most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.

ID Type	Range of Values	Memory usage of every ~ 100,000 indexed words (Index + Content)
Byte	0 - 255	683 kb + 2.4 Mb
Short	0 - 65,535	1.3 Mb + 2.4 Mb
Integer	0 - 4,294,967,295	2.7 Mb + 2.4 Mb
Float	0 - * (16 digits)	5.3 Mb + 2.4 Mb
String	* (unlimited)	1.3 Mb * char length of IDs + 2.4 Mb

Author BulkSearch: Thomas Wilkerling
License: Apache 2.0 License