Advanced Firestore Functions

These are the back-end firestore functions that will allow you to create easy-to-use indexes.

Installation

Install the package into your firebase functions directory.

npm i adv-firestore-functions

Import the necessary functions at the top of your firebase function file:

import { eventExists, fullTextIndex } from 'adv-firestore-functions';

All of these functions are called on an async onWrite firebase firestore function like so:

functions.firestore
    .document('posts/{docId}')
    .onWrite(async (change: any, context: any) => {
//... code
}

Full-text search

WARNING! - This function can create A LOT of documents if you have a big text field. However, it is worth it if you only write sporatically.

This will index your fields so that you can search them. No more Algolia or Elastic Search! It will create documents based on the number of words in the field. So a blog post with 100 words, will create 100 documents indexing 6 words at a time. You can change this number. Since you generally write / update fields in firebase rarely, 100 documents is not a big deal to index, and will save you money on searching. The size of the document is just 6 words, plus the other foreign key fields you want to index. This function will automatically create, delete, and update the indexes when necessary. All of these functions use transactions, batching, and chunking (100 documents at a time) to provide the best performance.

Events -- VERY IMPORTANT!

Anytime you use a counter function, or a complicated function like fullTextIndex that you only want run once, make sure to add the event function at the top of your code. Firebase functions can run functions more than once, messing up your indexes and counters.

// don't run if repeated function
if (await eventExists(context.eventId)) {
    return null;
}

So, in order to index the title and the content of your posts function, you could have something like this:

// index the posts
const searchable = ['content', 'title'];
searchable.forEach(async (field: string) => {
    await fullTextIndex(change, context, field);
});

--options--

await fullTextIndex(change, context, 'field-to-index', ['foreign', 'keys', 'to', 'index']);

The foreign keys to index will be all of the fields you will get back in a search. It defaults to ONLY the document id, however, you can add or change this to whatever fields you like.

await fullTextIndex(change, context, field, foreign-keys, type);

The type input defaults to 'id', and is indexed on all options.
--id - just makes the document searchable from the id field using the ~ trick on the 6 word chunk you are searching.
--map - makes the document searchable using a map of _terms (same as document id)
--array - makes the document searchable using an array of _terms (same as document id)

// map
{
    _terms: {
        a: true,
        al: true,
        also: true,
        also : true,
        also t: true
        ...
    }
}
// array
{
    _terms: [
        a,
        al,
        als,
        also,
        also ,
        also t,
        ...
    ]
}

Maps and Arrays are useful when you want to do complex searching, depending on what your constraints are. They do require more space on your documents and database size, but do not create any additional documents. Obviously searching is still limited to firestore's limits, but you can now use something similar to startsWith(), if it were a real function.

Front-end: This will depend on your implementation, but generally, you will use something like the following code:

let id = firebase.firestore.FieldPath.documentId();
const col = `_search/COLLECTION_NAME/COLLECTION_FIELD`;
db.collection(col).orderBy(id).startAt(term).endAt(term + '~').limit(5);

I will eventually make a front-end package for vanilla js or angular. You can search mulitple fields at the same time by combining the promises or combineLatest, for example, and sorting them on the front end with map. It will automatically index the correct fields and collection names. Use term to search. I would also recommend using a debounceTime function with rxjs to only search when you quit typing.

If you are using map or array, you may have something like this:

const col = `_search/COLLECTION_NAME/COLLECTION_FIELD`;
db.collection(col).where('_terms.' + term, '==', true); // map
db.collection(col).where('_terms', 'array-contains', term); // array

Index unique fields

Unique fields is pretty simple, add the unique field function and it will update automatically everytime there is a change. Here you can index the unique 'title' field. Check code for options like friendlyURL.

await uniqueField(change, context, 'title');

Front-end: Again, this will depend, but generally speaking you search like so:

db.doc(`_uniques/COLLECTION_NAME/${title}`);

on the document name to see if the 'title', in this case, is a unique value. Just use snapshot.exists depending on your front-end code.

Note: The following indexes care created automatically if they do not exist.

Collection counters

This will create a counter every time the collection is changed.

await colCounter(change, context);

Query counters

Query counters are very interesting, and will save you a lot of time. For example, you can count the number of documents a user has, or the number of categories a post has, and save it on the original document.

(See below for the getValue function)

// postsCount on usersDoc
import { eventExists, queryCounter, getValue } from 'adv-firestore-functions';

const userRef = getValue(change, 'userDoc');
const postsQuery = db.collection('posts').where('userDoc', "==", userRef);
await queryCounter(change, context, postsQuery, userRef);

This assumes you saved the userDoc as a reference, but you could easily create one with the document id:

const userId = getValue(change, 'userId');
const userRef = db.doc(`users/${userId}`);

Trigger Functions

You can change the trigger functions to update the same document with a filtered or new value. For example, if you have a value that you want to create on a function, and then go back and update it (a friendly title in lowercase).

// define data
const data: any = {};
data.someValue = doSomething();

// run trigger
await triggerFunction(change, data);

This will also automatically update createdAt and updatedAt dates in your document. This is good if you don't want the user to be able to hack these dates on the front end. You can turn this off by passing in false as the last paramenter.

Note: If you only want to update the dates createdAt and updatedAt, simply leave out the data parameter.

However, you need to add isTriggerFunction to the top of your code to prevent infinite loops:

// don't run if repeated function
if (await eventExists(context.eventId) || isTriggerFunction(change, context.eventId)) {
    return null;
}

There are many options for these as well, see actual code for changing default parameters.

The default counter variable can be changed on all documents. See the code for each function. You can also change the name of the index collections. The defaults are _tags, _search, _uniques, _counters, _categories, _events.

There are also some helper functions like valueIsChanged to see if a field has changed:

if (valueIsChanged(change, 'category')) {
// do something
}

and getValue to get the latest value of a field:

const category = getValue(change, 'category');

Last, but not least I have these specific functions for categories. I will explain these in a front-end module eventually, but until then don't worry about them. I am adding the usage case just for completeness.

Category counters

This is specific if you want to index categories. I will eventually post code on this to explain it, but usage is like so:

On the categories collection:

// update all sub category counters
await subCatCounter(change, context);

On the posts collection, or whatever your categories contain:

// [collection]Count on categories and its subcategories
await catDocCounter(change, context);

I will try and update the documention as these functions progress. There is plenty of logging, so check your logs for problems!

There is more to come as I simplify my firebase functions! See Fireblog.io for more examples (whenever I finally update it)!