Firelord (BETA, Nodejs)

🐤 Write truly scalable Firestore code with complete deep typing Firestore wrapper.

💪🏻 Type object, array, any combination of array and object, regardless of the nesting level.

🚀 The wrapper type all read and write operations, query field path, field value, collection path, document path.

🥙 All Snapshot(response) are recursively typed, no more type casting.

🔥 Convert all value types to corresponding read types, write types and compare types (good at handling timestamp and field values).

💥 Safe typing with masked Firestore Field Value(serverTimestamp, arrayRemove, arrayUnion and increment) types.

🌈 Strictly one-time setup per document. Once configured, you are ready. No more confusing setup in the future, simplicity at its finest.

🍡 Prevent empty array from hitting in, not-in, array-contains-any, arrayUnion and arrayRemove, peace in mind.

🍧 Use in, not-in and array-contains-any with more than 10 elements array. (not-in has a caveat)

🍁 write operations reject unknown member and enforce partial but no undefined.

🍹 Avoid order and query limitations for you, stopping potential run-time errors before they happen.

✨ API closely resembles Firestore API, low learning curve.

🦊 Zero dependencies.

⛲️ Out of box typescript support.

Variants:

1. react native
2. js

The package is only 22 KB before zipping and uglify, it looks big due to the images in the documentation.

require typescript 4.1 and above

Table of Contents

• What Is Firelord
• Getting Started
  • Collection
  • Sub-Collection
• Conversion Table
• Document Operations: Write, Read and Listen
• Document Operations: Batch
• Document Operations: Transaction
• Collection Operations: Query
• Collection Operations: Order And Limit
• Collection Operations: Paginate And Cursor
• Collection Group
• Complex Data Typing
• Set, Create and Add
  • Update
  • Handling Firestore Field Value: Masking
• Circumvented Firestore Limitations
• Advices
  • You can ≠ You should
  • Nested Object
  • Do Not Bother Cost Focus Data Modelling
  • One Collection One Document Type
  • Speed
• Caveats
  • Error Hint
  • not-in
• Opinionated Elements
• Limitation
• Utilities
• Road Map
• Change Log (any version that is not mentioned in the changelog is document update)

🐉 What Is Firelord

You may not notice this but you need to prepare 3 sets of data types to use Firestore properly, best example is sever timestamp, when read, it is Firestore.Timestamp; when write, it is Firestore.FieldValue; and finally when compare, it is Date|Firestore.Timestamp.

Unfortunately withConverter is not enough to solve the type problems, there is still no feasible solutions to deal with type like date, Firestore.Timestamp, number and array where different types are needed in read, write and compare(query).

Firelord introduces deep typing solutions to handle every case, other than that it also provides type safety for collection path, document path, Firestore limitations(whenever is possible).

The typing solution is powerful enough for any combination of array and object regardless of nesting level.

Other than type issues, Firestore also suffers from runtime errors:

• when you hit in, not-in, array-contains-any, arrayUnion and arrayRemove with empty array.
• when you hit in, not-in and array-contains-any with 10+ elements array.
• In a compound query, range (<, <=, >, >=) and not equals (!=, not-in) comparisons do not filter on the same field.
• Use over one array-contains clause per query.
• If you include a filter with a range comparison (<, <=, >, >=), your first ordering is not on the same field.
• Order your query by a field included in an equality == or in clause.

Firelord aims to eradicate preventable errors when developing with Firestore by implementing strategies that make the most sense.

It is designed to be as friendly as possible while offering a complete Firestore typing solution.

Long thing short, Firelord make sure that:
-It is virtually impossible to make any typing mistake.
-It is virtually impossible to run into any runtime errors as stated in Firestore query and order limitations.

Overview:

• generate read(get operation), write type(set/update operation) and compare type(for query) for field value, example:

  • server timestamp: {write: Firestore.FieldValue, read: Firestore.Timestamp, compare: Date | Firestore.Timestamp}
  • number: {write: FieldValue | number, read: number, compare:number}
  • xArray: {write: x[] | FieldValue, read: x[], compare: x[]}
  • see conversion table for more.

• One time setting per document type: define a data type, a collection path and a document path, and you are ready to go.

  • type collection path, collection group path and document path.
  • auto generate sub collection path type.

• auto generate updatedAt andcreatedAt timestamp.

  • auto update updatedAt server timestamp to update operation.
  • auto add createdAt and updatedAt server timestamp to create and add operation.
  • set operation does not auto-add or auto-update createdAt and updatedAt because it is impossible to know whether you are using this to create or to update. However set accepts createdAt(Firelord.ServerTimestamp) and updatedAt(Firelord.ServerTimestamp | null) as data optionally, so you can decide on yourself how to use it.

• type complex data type like nested object, nested array, object array, array object and all their operations regardless of their nesting level. Read Complex Data Typing for more info. NOTE: There is no path for d.e.g.h.a because it is inside an array, read Complex Data Typing for more info.

  

• Check your type regardless of how deep it is.

  

• Partial but no undefined: Prevent you from explicitly assigning undefined to a partial member in operation like set(with merge options) or update while still allowing you to skip that member regardless of how deep it is(You can override this behaviour by explicitly union undefined in the base type). Do note that you cannot skip any member of an object in an array due to how Firestore array works, read Complex Data Typing for more info.

  

• Reject unknown member: prevent you from writing unknown member (not exist in type) into set,create and update operations, stop unnecessary data from entering firestore, regardless of how deep the strange member is located.

  

• Prevent you from chaining <offset> or <limit and limit to last> for the 2nd time no matter how you chain them.

  

• much better where and orderBy clause

  • field values are typed accordingly to field path
  • comparators depend on field value type, eg you cannot apply array-contains operator onto non-array field value
  • whether you can chain orderBy clause or not is depends on the comparator's value, this is according to Order Limitation, see image below. Go to Order And Limit for documentation.

  

• avoid Query Limitation for you, no more runtime surprises, prevention >>> cures.

  

• The 4 musketeers: serverTimestamp(FieldValue), arrayRemove(FieldValue), arrayUnion(FieldValue) and increment(FieldValue) are now typed, see Handling Firestore Field Value: Masking for more info.

  

• Any document reference or query document reference that read or query operation returns are recursively typed, which means it has exactly the same read, write and compare data type for all read and write operations.

  • document reference return by get and onSnapshot are typed.
  • document reference return by querySnapshot's forEach, docChanges, and doc are typed.

• Firestore trigger runtime error if your array is empty when dealing with in, not-in, array-contains-any, arrayUnion and arrayRemove. The wrapper automatic handle this problem for you, you are free to use an empty array.

• Use in, not-in and array-contains-any with over 10 elements array, the wrapper chunk the array and create extra queries. not-in is not without a caveat, read not-in caveat.

🦜 Getting Started

npm i firelord
npm i -D ts-essentials

The wrapper requires ts-essentials to work, install it as dev-dependency.

Collection

import { firelord, Firelord } from 'firelord'
import { Firestore } from 'firebase-admin'

// create wrapper
const {
    fieldValue: { increment, arrayUnion, serverTimestamp, arrayRemove },
    wrapper,
} = firelord(firestore)

// use base type to generate read and write type
type User = FirelordUtils.ReadWriteCreator<
    {
        name: string
        age: number
        birthday: Date
        joinDate: Firelord.ServerTimestamp
        beenTo: ('USA' | 'CANADA' | 'RUSSIA' | 'CHINA')[]
    }, // base type
    'Users', // collection path type
    string // document ID type
>

// implement wrapper
const userCreator = wrapper<User>()
// collection reference
const users = userCreator.col('Users') // collection path type is "Users"
// collection group reference
const userGroup = userCreator.colGroup('Users') // collection group path type is "Users"
// document reference
const user = users.doc('1234567890') // document ID is string

if you need the types, here is how you get it.

// import User

// read type
type UserRead = User['read'] // {name: string, age:number, birthday: Firestore.Timestamp, joinDate: Firestore.Timestamp, beenTo:('USA' | 'CANADA' | 'RUSSIA' | 'CHINA')[], createdAt: Firestore.Timestamp, updatedAt: Firestore.Timestamp}

// write type
type UserWrite = User['write'] // {name: string, age:number|FirebaseFirestore.FieldValue, birthday:Firestore.Timestamp | Date, joinDate:FirebaseFirestore.FieldValue, beenTo:('USA' | 'CANADA' | 'RUSSIA' | 'CHINA')[] | FirebaseFirestore.FieldValue, createdAt: FirebaseFirestore.FieldValue, updatedAt: FirebaseFirestore.FieldValue}

// compare type
type UserCompare = User['compare'] // {name: string, age:number, birthday:Date | Firestore.Timestamp, joinDate: Date | Firestore.Timestamp, beenTo:('USA' | 'CANADA' | 'RUSSIA' | 'CHINA')[], createdAt: Date | Firestore.Timestamp, updatedAt: Date | Firestore.Timestamp}

// collection name
type UserColName = User['colName'] //"Users"

// collection path
type UserColPath = User['colPath'] // "Users"

// document ID
type UserDocId = User['docID'] // string

// documentPath
type UserDocPath = User['docPath']

Sub-Collection

This is how you define a sub-collection, just plug in parent type into the type generator's 4th parameter and the wrapper know this is a sub-collection.

The wrapper will constructs all the collection, document and collection group path types for you, easy-peasy.

// import User
// import FirelordUtils
// import wrapper

// subCollection of User
type Transaction = FirelordUtils.ReadWriteCreator<
    {
        amount: number
        date: Firelord.ServerTimestamp
        status: 'Fail' | 'Success'
    }, // base type
    'Transactions', // collection path type
    string, // document path type
    User // insert parent collection, it will auto construct the collection path for you
>

// implement the wrapper
const transactionCreator = wrapper<Transaction>()
const transactionsCol = transactionCreator.col('Users/283277782/Transactions') // the type for col is `User/${string}/Transactions`
const transactionGroup = transactionCreator.colGroup('Transactions') // the type for collection group is `Transactions`
const transaction = transactionsCol.doc('1234567890') // document path is string

I strongly against defining over one document type per collection, read one collection one document type.

🦔 Conversion Table

The wrapper generate 3 types based on base type:

1. read type: the type you get when you call get or onSnapshot.

2. write type: the type for add, set, create and update operations, the type is further split into:

   -write type: flattened type for update operation.

   -writeNested type: normal object type for add, set and create operations.

3. compare type: the type you use in where clause.

All read operations return read type data, all write operations require write type data and all queries require compare type data, you only need to define base type and the wrapper will generate the other 3 types for you.

You don't need to do any kind of manipulation onto read, write and compare types, nor do you need to use them.

The documentation explains how the types work, the wrapper itself is intuitive in practice. thoroughly refer to the documentation only if you hit the dead end.

You SHOULD NOT try to memorize how the typing work, keep in mind the purpose is not for you to fit into the type, but is to let the type GUIDE you.

you can union any types, it will generate the types distributively, for example type string | number | number[] | (string | number)[] | (string | number)[][] | (string | number)[][][] generates:

read type: string | number | number[] | (string | number)[] | (string | number)[][] | (string | number)[][][]

write type: string | number | FirebaseFirestore.FieldValue | number[] | (string | number)[] | (string | number)[][] | (string | number)[][][]

compare type: string | number | number[] | (string | number)[] | (string | number)[][] | (string | number)[][][]

In practice, any union is not recommended, data should has only one type, except undefined or null union that bear certain meaning(value missing or never initialized).

NOTE: Date | Firestore.Timestamp, (Date | Firestore.Timestamp)[], and Date[] | Firestore.Timestamp[] unions are redundant, because Date and Firestore.Timestamp generate same read, write and compare types.

* Any FirebaseFirestore.FieldValue type will be replaced by masked type, see Handling Firestore Field Value: Masking for more info.

** object type refer to object literal type(typescript) or map type(firestore). The wrapper flatten nested object, however, there are not many things to do with object[] type due to how Firestore work, read Complex Data Typing for more info.

*** Firelord.ServerTimestamp is a reserved type. You cannot use it as a string literal type, use this type if you want your type to be Firestore.ServerTimestamp. Also do note that you cannot use serverTimestamp or any Firestore field value in an array, see Complex Data Typing for more info.

🐘 Document Operations: Write, Read and Listen

All the document operations API is like Firestore write, read and listen.

// import user

import { Firestore } from 'firebase-admin'

// get data(type is `read type`)
user.get().then(snapshot => {
    const data = snapshot.data()
})

// listen to data(type is `read type`)
user.onSnapshot(snapshot => {
    const data = snapshot.data()
})

const ServerTimestamp = Firestore.FieldValue.ServerTimestamp()

// create if only exist, else fail
// require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
// auto add `createdAt` and `updatedAt`
user.create({
    name: 'John',
    age: 24,
    birthday: new Date(1995, 11, 17),
    joinDate: ServerTimestamp,
    beenTo: ['RUSSIA'],
})

// create if not exist, else overwrite
// require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
user.set({
    name: 'John',
    age: 24,
    birthday: new Date(1995, 11, 17),
    joinDate: ServerTimestamp,
    beenTo: ['RUSSIA'],
})

// create if not exist, else update
// all member are partial members, you can leave any of the member out, however typescript will stop you from explicitly assign `undefined` value to any of the member unless you union the type with `undefined` in the `base type`
// the only value for `merge` is `true`
// NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ merge: true }` in the option, as shown below.
user.set({ name: 'Michael' }, { merge: true })

// create if not exist, else update
// all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
// the only value for `merge` is `true`
// NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ mergeField: fieldPath[] }` in the option, as shown below.
user.set(
    { name: 'Michael', age: 32, birthday: new Date(1987, 8, 9) },
    { mergeField: ['name', 'age'] } // update only `name` and `age` fields
)

// update if exist, else fail
// all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
// auto update `updatedAt`
user.update({ name: 'Michael' })

// delete document
user.delete()

🦩 Document Operations: Batch

all API are similar to firestore batch, the only difference is, the batch is member of doc, hence you don't need to define document reference.

// import user
import { Firestore } from 'firebase-admin'

// implement the wrapper
const user = wrapper<User>().col('Users').doc('1234567890')

// create batch
const batch = firestore().batch()
const userBatch = user.batch(batch)

// delete document
userBatch.delete()

// create if exist, else fail
// require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
// auto add `updatedAt` and `createdAt`
userBatch.create({ name: 'Michael', age: 32, birthday: new Date(1987, 8, 9) })

// create if not exist, else overwrite
// require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
userBatch.set({
    name: 'John',
    age: 24,
    birthday: new Date(1995, 11, 17),
    joinDate: ServerTimestamp,
    beenTo: ['RUSSIA'],
})

// create if not exist, else update
// all member are partial members, you can leave any of the member out, however typescript will stop you from explicitly assign `undefined` value to any of the member unless you union the type with `undefined` in the `base type`
// the only value for `merge` is `true`
// NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ merge: true }` in the option, as shown below.
userBatch.set({ name: 'Michael' }, { merge: true })

// create if not exist, else update
// all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
// the only value for `merge` is `true`
// NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ mergeField: fieldPath[] }` in the option, as shown below.
userBatch.set(
    { name: 'Michael', age: 32, birthday: new Date(1987, 8, 9) },
    { mergeField: ['name', 'age'] } // update only `name` and `age` fields
)

// update if exist, else fail
// all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
// auto update `updatedAt`
userBatch.update({ name: 'Ozai' })

//commit
batch.commit()

🐠 Document Operations: Transaction

all API is like firestore transaction, the only difference is, the batch is a member of doc, hence you don't need to define document reference.

// import user
// import firestore

firestore().runTransaction(async transaction => {
    // get `read type` data
    await user
        .transaction(transaction)
        .get()
        .then(snapshot => {
            const data = snapshot.data()
        })

    // create if only exist, else fail
    // require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
    // auto add `createdAt` and `updatedAt`
    user.transaction(transaction).create({
        name: 'John',
        age: 24,
        birthday: new Date(1995, 11, 17),
        joinDate: ServerTimestamp,
        beenTo: ['RUSSIA'],
    })

    // create if not exist, else overwrite
    // require all `write type` members(including partial member in the `base type`) except `updatedAt` and `createdAt`
    user.transaction(transaction).set({
        name: 'John',
        age: 24,
        birthday: new Date(1995, 11, 17),
        joinDate: ServerTimestamp,
        beenTo: ['RUSSIA'],
    })

    // create if not exist, else update
    // all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
    // the only value for `merge` is `true`
    // NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ merge: true }` in the option, as shown below.
    user.transaction(transaction).set({ name: 'Michael' }, { merge: true })

    // create if not exist, else update
    // all members are partial members, you can leave any of the members out, however, typescript will stop you from explicitly assigning `undefined` value to any of the members unless you union the type with `undefined` in the `base type`
    // the only value for `merge` is `true`
    // NOTE: there will be a missing property error from typescript if all the members are not present. To fix this, just fill in `{ mergeKey: fieldPath[] }` in the option, as shown below.
    user.transaction(transaction).set(
        { name: 'Michael', age: 32, birthday: new Date(1987, 8, 9) },
        { mergeField: ['name', 'age'] } // update only `name` and `age` fields
    )

    // update if exist, else fail
    // all member are partial members, you can leave any of the member out, however typescript will stop you from explicitly assign `undefined` value to any of the member unless you union the type with `undefined` in the `base type`
    // auto update `updatedAt`
    user.transaction(transaction).update({ name: 'Michael' })
    // delete document
    user.transaction(transaction).delete()

    return
})

🌞 Collection Operations: Query

All the API are like firestore query, clauses are chain-able.

// import users

// non array data type
// the field path is the keys of the `base type`
// type of opStr is '<' | '<=' | '==' | '!=' | '>=' | '>' | 'not-in' | 'in'
// if type of opStr is '<' | '<=' | '==' | '!=' | '>=' | '>', the value type is same as the member's type in `compare type`
users.where('name', '==', 'John').get()
// if type of opStr is 'not-in' | 'in', the value type is array of member's type in `compare type`
users.where('name', 'in', ['John', 'Michael']).get()

// array data type
// the field path is the keys of the `base type`
// type of `opStr` is  'in' | 'array-contains-any'
// if type of opStr is 'array-contains', the value type is the non-array version of member's type in `compare type`
users.where('beenTo', 'array-contains', 'USA').get()
// if type of opStr is 'array-contains-any', the value type is same as the member's type in `compare type`
users.where('beenTo', 'array-contains-any', ['USA']).get()
// if type of opStr is 'in', the value type is the array of member's type in `compare type`
users.where('beenTo', 'in', [['CANADA', 'RUSSIA']]).get()

🐳 Collection Operations: Order And Limit

all the API are like firestore order and limit with slight differences, but work the same, clauses are chain-able.

The type rule obey Firestore Order Limitation.

Read this before proceeding: Firestore OrderBy and Where conflict and firestore index on how to overcome certain order limitation, this is also considered into typing.

Any orderBy that does not follow where clause does not abide by the rule and limitation mentioned above.

Note: The wrapper will not stop you from using multiple orderBy clause because multiple orderBy clause is possible, read Multiple orderBy in firestore and Ordering a Firestore query on multiple fields.

NOTE: in and array-contains-any return array of query which in the end return array of Promise, please use Promise.all or Promise.allSettled to resolve the promises, this is to overcome the 10 elements limitation

Tips: to make things easier, whenever you want to use where + orderBy, use the shorthand form (see example code below).

// import users

// the field path is the keys of the `compare type`(basically keyof `base type` plus `createdAt` and `updatedAt`)

// if the member value type is array, type of `opStr` is  'in' | 'array-contains'| 'array-contains-any'
// if type of opStr is 'array-contains', the value type is the non-array version of member's type in `compare type`
users.where('beenTo', 'array-contains', 'USA').get()
// if type of opStr is 'array-contains-any', the value type is same as the member's type in `compare type`
Promise.all(
    users.where('beenTo', 'array-contains-any', ['USA']).map(query => {
        return query.get()
    })
)
// if type of opStr is 'in', the value type is the array of member's type in `compare type`
Promise.all(
    users.where('beenTo', 'in', [['CANADA', 'RUSSIA']]).map(query => {
        return query.get()
    })
)
// orderBy field path only include members that is NOT array type in `compare type`
users.orderBy('name', 'desc').limit(3).get()

// for `array-contains` and `array-contains-any` comparators, you can chain `orderBy` clause with DIFFERENT field path
users.where('beenTo', 'array-contains', 'USA').orderBy('age', 'desc').get()
Promise.all(
    users.where('beenTo', 'array-contains-any', ['USA', 'CHINA']).map(query => {
        return query.orderBy('age', 'desc').get()
    })
)

// for '==' | 'in' comparators:
// no order for '==' | 'in' comparator for SAME field name, read https://stackoverflow.com/a/56620325/5338829 before proceed
users.where('age', '==', 20).orderBy('age', 'desc').get() // ERROR
// '==' | 'in' is order-able with DIFFERENT field name but need to use SHORTHAND form to ensure type safety
users.where('age', '==', 20).orderBy('name', 'desc').get() // ERROR
// shorthand ensure type safety, equivalent to where('age', '>', 20).orderBy('name','desc')
users
    .where('age', '==', 20, { fieldPath:: 'name', directionStr: 'desc' })
    .get() // OK
// again, no order for '==' | 'in' comparator for SAME field name
users
    .where('age', '==', 20, { fieldPath:: 'age', directionStr: 'desc' })
    .get() // ERROR

// for '<' | '<=]| '>'| '>=' comparator
// no order for '<' | '<=]| '>'| '>=' comparator for DIFFERENT field name
users.where('age', '>', 20).orderBy('name', 'desc').get() // ERROR
// '<' | '<=]| '>'| '>=' is oder-able with SAME field name but need to use SHORTHAND form to ensure type safety
users.where('age', '>', 20).orderBy('age', 'desc').get() // ERROR
// equivalent to where('age', '>', 20).orderBy('age','desc')
users
    .where('age', '>', 20, { fieldPath:: 'age', directionStr: 'desc' })
    .get() // OK
// again, no order for '<' | '<=]| '>'| '>=' comparator for DIFFERENT field name
users
    .where('age', '>', 20, { fieldPath:: 'name', directionStr: 'desc' })
    .get() // ERROR

// for `not-in` and `!=` comparator, you can use normal and  shorthand form for both same and different name path
// same field path
users.where('name', 'not-in', ['John', 'Ozai']).orderBy('name', 'desc').get()
// different field path
users.where('name', 'not-in', ['John', 'Ozai']).orderBy('age', 'desc').get()
// shorthand different field path:
users
    .where('name', 'not-in', ['John', 'Ozai'], {
        fieldPath:: 'age',
        directionStr: 'desc',
    })
    .get() // equivalent to where('name', 'not-in', ['John', 'Ozai']).orderBy('age','desc')
// shorthand same field path:
users
    .where('name', 'not-in', ['John', 'Ozai'], {
        fieldPath:: 'name',
        directionStr: 'desc',
    })
    .get() // equivalent to where('name', 'not-in', ['John', 'Ozai']).orderBy('name','desc')

// same field path
users.where('name', '!=', 'John').orderBy('name', 'desc').get()
// different field path
users.where('name', '!=', 'John').orderBy('age', 'desc').get()
// shorthand different field path:
users
    .where('name', '!=', 'John', {
        fieldPath:: 'age',
        directionStr: 'desc',
    })
    .get() // equivalent to where('name', '!=', 'John').orderBy('age','desc')
// shorthand same field path:
users
    .where('name', '!=', 'John', {
        fieldPath:: 'name',
        directionStr: 'desc',
    })
    .get() // equivalent to where('name', '!=', 'John').orderBy('name','desc')

//pagination
// field path only include members that is NOT array type in `base type`
// field value type is the corresponding field path value type in `compare type`
// value of cursor clause is 'startAt' | 'startAfter' | 'endAt' | 'endBefore'
users.orderBy('age', 'asc', { clause: 'startAt', fieldValue: 20 }).offset(5) // equivalent to orderBy("age").startAt(20).offset(5)
// usage with where
users
    .where('name', '!=', 'John')
    .orderBy('age', 'desc', { clause: 'endAt', fieldValue: 50 })
// equivalent to shorthand
users
    .where('name', '!=', 'John', {
        fieldPath:: 'age',
        directionStr: 'desc',
        cursor: { clause: 'endAt', fieldValue: 50 },
    })
    .get() // equivalent to where('name', '!=', 'John').orderBy('age','desc').endAt(50)

🌺 Collection Operations: Paginate And Cursor

API differ slightly from firestore paginate and cursor, the cursors became orderBy parameter, it still works the same as Firestore original API, clauses are chain-able.

// import users

// field path only include members that is NOT array type in the `base type`
// field value type is the corresponding field path value type in `compare type`
// value of cursor clause is 'startAt' | 'startAfter' | 'endAt' | 'endBefore'
users.orderBy('age', 'asc', { clause: 'startAt', fieldValue: 20 }).offset(5) // equivalent to orderBy("age").startAt(20).offset(5)
// usage with where
users
    .where('name', '!=', 'John')
    .orderBy('age', 'desc', { clause: 'endAt', fieldValue: 50 })
// equivalent to shorthand
users
    .where('name', '!=', 'John', {
        fieldPath:: 'age',
        directionStr: 'desc',
        cursor: { clause: 'endAt', fieldValue: 50 },
    })
    .get() // equivalent to where('name', '!=', 'John').orderBy('age','desc').endAt(50)

🌵 Collection Group

Api is exactly the same as Collection Operations: Query, Order And Limit, Paginate And Cursor

simply use collection group reference instead of collection reference, refer back Getting Started on how to create collection group reference.

🌻 Complex Data Typing

As for (nested or not)object[] type, its document/collection operations work the same as other arrays: it will not flatten down due to how Firestore work, read Firestore how to query nested object in array. You cannot query(or set, update, etc) specific object member or array member in the array, nested or not, similar rule applies to a nested array.

Long thing short, any data type that is in an array, be it another array or another object with array member:

1. will not get flattened and will not have its field path built nor you can use field value (arrayRemove, arrayUnion and increment, serverTimestamp). Read Firestore append to array (field type) an object with server timestamp and How to increment a map value in a Firestore array, both are negative.
2. unable to skip any member, object must have exact shape. If you need undefined, assign undefined to the member in the base type instead.

However, it is possible to query or write a specific object member (nested or not), as long as it is not in an array, the typing logic works just like other primitive data types' document/collection operation because the wrapper will flatten all the members in object type, nested or not.

NOTE: read type does not flatten, because there is no need to.

Last, all the object, object[], array, array object, nested or not, no matter how deep, all the field values (not specifically referring to Firestore.FieldValue) of all data types will undergo data type conversion according to the conversion table.

consider this example

// import FirelordUtils
// import wrapper

type Nested = FirelordUtils.ReadWriteCreator<
    {
        a: number
        b: { c: string }
        d: { e: { f: Date[]; g: { h: { i: { j: Date }[] }[] } } }
    },
    'Nested',
    string
>
const nestedCreator = wrapper<Nested>()

const nestedCol = nestedCreator.col('Nested')

// read type, does not flatten because no need to
type NestedRead = Nested['read'] // {a: number, b: { c: string }, d: { e: { f: FirebaseFirestore.Timestamp[], g: { h: { i: {j: Firestore.Timestamp}[] }[] } } }	}
// write type
type NestedWrite = Nested['write'] // {a: number | FirebaseFirestore.FieldValue, "b.c": string, "d.e.f": FirebaseFirestore.FieldValue | (FirebaseFirestore.Timestamp | Date)[], "d.e.g.h": FirebaseFirestore.FieldValue | { i: {j: Firestore.Timestamp | Date}[] }[], createdAt: FirebaseFirestore.FieldValue, updatedAt: FirebaseFirestore.FieldValue}
// compare type
type NestedCompare = Nested['compare'] // {a: number, "b.c": string, "d.e.f": (FirebaseFirestore.Timestamp | Date)[], "d.e.g.h": FirebaseFirestore.FieldValue | { i: {j: Firestore.Timestamp | Date}[] }[], createdAt: Date | Firestore.Timestamp, updatedAt: Date | Firestore.Timestamp}

As you can see, the object flattens down and the wrapper converted all the value types

so the next question is, how are you going to shape your object so you can use it in set, create and update operation?

🍣 Set, Create and Add

Please read set and dot syntax before you proceed.

In short, you cannot use dot syntax with set (create should have the same behaviour, need more clarification).

consider this example:

// import nestedCol
const completeData = {
    a: 1,
    b: { c: 'abc' },
    d: { e: { f: [new Date(0)], g: { h: [{ i: [{ j: new Date(0) }] }] } } },
}
const data = {
    a: 1,
    d: { e: { f: [new Date(0)], g: { h: [{ i: [{ j: new Date(0) }] }] } } },
}

nestedCol.doc('123456').set(completeData) // set needs complete data if no merge option
nestedCol.doc('123456').create(completeData) // create also requires complete data
nestedCol.add(completeData) // add also requires complete data
nestedCol.doc('123456').set(data, { merge: true })

Update

Please read Cloud Firestore: Update fields in nested objects with dynamic key before you proceed,

Yes, update can use dot syntax to update specific fields.

consider this example:

// import nested

const data = {
    a: 1,
    d: { e: { f: [new Date(0)], g: { h: [{ i: [{ j: new Date(0) }] }] } } },
}

nestedCol.doc('123456').update(data) // ERROR, type mismatch, if your data is nested object, please flatten your data first

If you want to update fields in nested objects, there are 2 ways:

1. create the flattened object by yourself.
2. use helper function: import flatten from firelord. (recommended, because it is easier)

Reminder:

1. you don't need to flatten non-nested object, but nothing will happen if you accidentally did it.
2. update does not require all members to exist, it will simply update that particular field.

solution:

// import nestedCol

import { flatten } from 'firelord'

// flatten by yourself(?)
const flattenedData = {
    a: 1,
    'd.e.f': [new Date(0)],
    'd.e.g.h': [{ i: [{ j: new Date(0) }] }],
}

nestedCol.doc('123456').update(flattenedData) // ok

// use helper function
const data = {
    a: 1,
    d: { e: { f: [new Date(0)], g: { h: [{ i: [{ j: new Date(0) }] }] } } },
}
nestedCol.doc('123456').update(flatten(data)) // ok, recommended, because it is easier

As for query, since the type is flattened, just query like you would normally query in firelord.

That is all. Call flatten to flatten the complex data and the rest work just like simple data, clean and simple.

🍱 Handling Firestore Field Value: Masking

Firestore field value, aka serverTimestamp, arrayRemove, arrayUnion and increment, they all return FieldValue, this is problematic, as you may use increment on an array or serverTimeStamp on a number. Kudo to whoever design this for making our life harder.

The wrapper forbids you to use any Firestore field value(serverTimestamp, arrayRemove, arrayUnion and increment) instance. We prepare another field value generator for you with the return type masked.

It still returns the same Firestore field value but with a masked return type, conversion table below shows what mask the types.

the mask types purposely looks weird, so nobody accidentally uses it for something else(as it could be dangerous, because the underneath value is Firestore field value, not what typescript really think it is).

this is how you use it

// import firestore

const {
    fieldValue: { increment, arrayUnion, serverTimestamp, arrayRemove },
    wrapper,
} = firelord(firestore)

type Example = FirelordUtils.ReadWriteCreator<
    {
        aaa: number | undefined
        bbb: Firelord.ServerTimestamp
        ddd: string[]
    },
    'Example',
    string
>

const exampleCreator = wrapper<Example>()

const exampleCol = exampleCreator.col('Example')

exampleCol.doc('1234567').set({
    aaa: arrayUnion('123', '456'), // ERROR
    bbb: increment(11), // ERROR
    ddd: arrayUnion('ddd', 123, 456), // ERROR <- at first typescript will not highlight this error unless you solve other error first, which mean in the end it still able to catch all error, so don't panic
})

exampleCol.doc('1234567').set({
    aaa: increment(1), // ok
    bbb: serverTimestamp(), // ok
    ...arrayUnion('ddd', '123', '456'), // ok, you can also write this as `...arrayUnion('ddd', ...['123', '456'])`
})

The API is like Firestore API, except arrayUnion and arrayRemove, the API is designed in such a way to deal with empty array errors while keeping your type safe.

same working logic apply to complex data type.

if you try to use the original Firestore field value, the wrapper will stop you.

🍝 Circumvented Firestore Limitations

🐬 Advices

You can ≠ You should

Although the wrapper can handle virtually any complex data type, it doesn't mean you should model the data in such a way, especially when dealing with the array.

When dealing with an array, avoid array of objects

not only data types are hard to query and hard to massage, but they also pose great difficulty to security rule, especially if the permission is relying on the data.

Use array on primitive data types, or timestamp and geo point(objects that have consistent structure).

Theoretically speaking, it is possible to create a flat structure for all kinds of data types, but this is harder to be done in Firestore because your data model affects the pricing, Firestore incentives you to put more data in the same document, hence you see all kind of array of objects types.

Anyway, do not resort to array of objects types easily, create a new collection instead. Always keep your data type straightforward if possible.

Nested Object

In firestore, nested object working logic is like a flat object, as long as you get the path right, you can query and write them with no issue. A nested object is fine, as long as you structure it in how a human mind can easily comprehend it, eg do not nest too deep.

Do Not Bother Cost Focus Data Modelling

Firestore may look simple, but it is incredibly difficult to model especially if you aim to save as much as cost as possible, that is aggregating your data. My advice is, do not bother, because it increases your project complexity, and it doesn't worth the time and money to aggregate the data.

However, if aggregation can save you a significant amount of money(assuming you already model your data correctly), chances are, you are probably not using the right database, use other databases (PSQL or MongoDB), it is much better and easier in terms of cost handling.

One Collection One Document Type

It is possible to have multiple document types under the same collection. For example, under a User collection, you may be tempted to create profile and setting(User/{userId}/Account/[profile and setting]) documents in it. Or you may create two collections under User that contains only a single document:User/{userId}/Profile/profile and User/{userId}/Setting/setting.

I would recommend instead of creating profile and setting, you create two top collections: profile and setting that contain all users' profiles and settings instead.

Logically, there should be one type of document in on collection(hence the name collections).

If this is not enough to convince you, then this will: type safety.

If you have over one type of document, how do you ensure the document you query matches the type you want?

Answer: it is possible, by union the two types but maintenance difficulty escalate quickly, imagine union bunch of types.

Avoid multi document types per collection, I strongly recommend enforcing 1 collection 1 document type coding policy.

Speed

Don't use Firestore if speed matter. The query time of Firestore is depend on result set, not total data set, which means the more data set you have the better Firestore performs against other database.

Though it is unsure at what point Firestore performance start to exceed other databases, my guess is most people are not likely to hit that point.

Thus don't expect much from the speed, use Firestore for task that is not time critical.

Do Not Use Firestore Rules For Write Operation

Firestore rule sucks big time:

1. You need to learn a language that is somehow similar to Javascript but is useless anywhere else
2. No type safety, full YOLO mode.
3. Code is not scalable and maintainable, it is nightmare to debug.
4. No open source and tooling support like Javascript, you going to miss a lot of powerful validation libraries like joi, yup and zod.

Run write operations in cloud function. Yes, cloud function cost you money per invoke, but write operation is much less frequent compare to read operation, the cost is definitely lower than the cost to maintain Firestore rules.

If the cost is your concern, you can always set up a custom backend.

Read operation requires only simple authentication, but some applications may require complicated authentication, in that case, it is also better to drop all the Firestore rules and validate it via a custom backend or cloud function.

One thing you will miss is the optimistic update, well until Firestore allows us to write rules in mainstream languages, we need to create our own optimistic update solutions.

🦎 Caveats

Error Hint

Because of the heavy use of generic types and utility types, typescript hints may look chaotic. The wrapper priority is to make sure you cannot go wrong. There will be no false positive, only misplaced negative like:

1. no error shown on member that has type error if other members also having type errors
2. error shown on member that is ok instead
3. everything has type error

However, there is no need to be panic. As long as there is an error, then something must be wrong. Simply check your data type carefully.

not-in

Although the wrapper can help you with array-contains-any and in comparator 10 elements limit, it is virtually impossible to overcome the similar limitation on not-in comparator.

Here are the solutions that will NOT work:

• chunk the array and create multiple queries just like what we did to array-contains-any and in.

This will not work, in fact, this is a very terrible idea. Here is why:

you have array like this [1,2,3,4,5,6,7,8,9,10,11], after chunking, you have 2 arrays: [1,2,3,4,5,6,7,8,9,10] and [11] and you run 2 queries with it.

The first query will return all documents that do not contain 1,2,3,4,5,6,7,8,9 or 10.

The 2nd query, however, will return all documents that do not contain 11, this includes documents that were filtered by query 1.

As you can see, chunking makes thing worse than what it is now.

• using typescript to circumvent it.

This is impossible as array size can only be determined on runtime, so neither can typescript help you.

• Checking the array size in runtime, if over 10, prevent the not-in query from running.

So you skip the query, then what? How are you going to fill the data void after that?

You need the query. It is inevitable.

• not using not-in at all.

Seriously doubt this is doable, not-in query is like one of the most useful queries in all kinds of databases. I don't think you can survive without it.

====

There is only one solution to this problem: first, filter with 10 elements, then filter the rest of the query result with the rest of the elements.

We implemented the solution in the wrapper; you don't need to do anything. The querySnapshot's forEach, docChanges, docs, empty and size values are all recomputed base on the extra elements to filter.

Not the ideal solution, but this is the only option we have, the wrapper simply help you filter the rest so you don't have to.

There is nothing can be done on our side. It is up to Google to improve this.

🐕 Opinionated Elements

Code wise, there is one opinionated element in the wrapper, that is createdAt and updatedAt timestamp that add or update automatically.

when a document is created via add, create without option, two things will happen:

1. createdAt field path is created, and the value is Firestore server timestamp(current server timestamp).
2. updatedAt field path is created, and the value is null.

when a document is updated via update with option:

1. updatedAt field path is updated and the value is Firestore server timestamp.

This behaviour may be undesirable for some people. I will improve this in future by giving the developer choice.

Finally there is no auto update/add updatedAt and created for set operation as it is not possible to tell whether you want to create or update.

However set now accepts createdAt(Firelord.ServerTimestamp) and updatedAt(Firelord.ServerTimestamp | null) as data optionally, so you can decide on yourself how to use it.

Typing wise, there are few opinionated elements:

1. set(without option) and create operations require all member to present.
2. all write operations reject unknown members.
3. although updatedAt and createdAt is included in type, all write operation exclude them, which mean you cant write the value of updatedAt and createdAt.

I believe this decision is practical for cases and not planning to change it in the forseeable future.

🐇 Limitation

While the wrapper try to safeguard as much type as possible, some problem cannot be solved due to typing difficulty, or require huge effort to implement, or straight up not can be solved.

1. Firelord.ServerTimestamp is a reserved type, underneath it is a string and you cannot use it as a string literal type. Use it when only you need the serverTimestamp type.
2. All mask types are passive reserved types, you cannot use them as object type nor use them for any purpose(the wrapper will turn mask types into never if you use them).

💍 Utilities

Since write operations reject unknown members (member that are not defined in base type), you can use object-exact to remove the unknown members, the library returns the exact type, so it should work well with the wrapper.

Do not use flatten for other purposes. If you need it, see object-flat, it is a general purpose library. Do not use object-flat in firelord as it is not specifically tailored for firelord, use firelord native flatten instead.

If you are looking to chunk an array: array-chop

Disclaimer: I am the author of all the packages.

🐎 Road Map

• allow update to accept both flatten and nested object thus able to automate flatten internally (difficult)
• jsDoc
• validation with zod
• jest