Multer's GridFS storage engine

GridFS storage engine for Multer to store uploaded files directly to MongoDb

Installation

Using npm

$ npm install multer-gridfs-storage --save

Basic usage example:

var express = require('express');
var multer  = require('multer');
// Create a storage object with a given configuration
var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database'
});
// Set multer storage engine to the newly created object
var upload = multer({ storage: storage });

var app = express()

// Upload your files as usual
var sUpload = upload.single('avatar');
app.post('/profile', sUpload, function (req, res, next) { 
    /*....*/ 
})

var arrUpload = upload.array('photos', 12);
app.post('/photos/upload', arrUpload, function (req, res, next) {
    /*....*/ 
})

var fUpload = upload.fields([{ name: 'avatar', maxCount: 1 }, { name: 'gallery', maxCount: 8 }])
app.post('/cool-profile', fUpload, function (req, res, next) {
    /*....*/ 
})

API

module(options) : function

The module returns a function that can be invoked with options to create a Multer storage engine.

The options parameter is an object with the following properties.

gfs

Type: Object or Promise

Required if url option is not present

The gridfs-stream instance to use or a promise that resolves with the instance.

If this option is provided, files are stored using this stream. This is useful when you have an existing GridFS object and want to reuse it to upload your files.

Example:

var Grid = require('gridfs-stream');
var mongo = require('mongodb');
var GridFsStorage = require('multer-gridfs-storage');

var db = new mongo.Db('database', new mongo.Server("127.0.0.1", 27017));

db.open(function (err) {
  if (err) return handleError(err);
  var gfs = Grid(db, mongo);

  var storage = GridFsStorage({
     gfs: gfs
  });
  var upload = multer({ storage: storage });
})

url

Type: String

Required if gfs option is not present

The mongodb connection uri.

A string pointing to the database used to store the incoming files. This must be a standard mongodb connection string.

With this option the module will create a GridFS stream instance for you instead.

Note: If the gfs option is specified this setting is ignored.

Example:

var storage = require('multer-gridfs-storage')({
    url: 'mongodb://localhost:27017/database'
});
var upload = multer({ storage: storage });

Creating the connection string from a host, port and database object

var url = require('url');

var settings = {
    host: '127.0.0.1',
    port: 27017,
    database: 'database'
};

var connectionString = url.format({
    protocol: 'mongodb',
    slashes: true,
    hostname: settings.host,
    port: settings.port,
    pathname: settings.database
});

var storage = require('multer-gridfs-storage')({
    url: connectionString
});
var upload = multer({ storage: storage });

filename

Type: Function

Not required

A function to control the file naming in the database. Is invoked with the parameters req, file and callback, in that order, like all the Multer configuration functions.

By default this module behaves exactly like the default Multer disk storage does. It generates a 16 bytes long name in hexadecimal format with no extension for the file to guarantee that there are very low probabilities of naming collisions. You can override this by passing your own function.

Example:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   filename: function(req, file, cb) {
      cb(null, file.originalname);
   }
});
var upload = multer({ storage: storage });

In this example the original filename and extension in the user's computer are used to name each of the uploaded files. Please note that this will not guarantee that file names are unique and you might have files with duplicate names in your database.

var crypto = require('crypto');
var path = require('path');

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   filename: function(req, file, cb) {
       crypto.randomBytes(16, function (err, raw) {
           cb(err, err ? undefined : raw.toString('hex') + path.extname(file.originalname));
       });
   }
});
var upload = multer({ storage: storage });

To ensure that names are unique a random name is used and the file extension is preserved as well. You could also use the user's file name plus a timestamp to generate unique names.

identifier

Type: Function

Not required

A function to control the unique identifier of the file.

This function is invoked as all the others with the req, file and callback parameters and can be used to change the default identifier ( the _id property) created by MongoDb. You must guarantee that this value is unique otherwise you will get an error.

Please note that the identifiers must conform to the MongoDb spec for ObjectID, that is, a 24 bytes hex string, 12 byte binary string or a Number.

To use the default generated identifier invoke the callback with a falsey value like null or undefined.

Example:

var path = require('path');
var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   identifier: function(req, file, cb) {
      cb(null, Math.floor(Math.random() * 1000000));
   }
});
var upload = multer({ storage: storage });

In this example a random number is used for the file identifier.

Important note

Normally you shouldn't use this function unless you want granular control of your file ids because auto-generated identifiers are guaranteed to be unique.

metadata

Type: Function

Not required

A function to control the metadata object associated to the file.

This function is called with the req, file and callback parameters and is used to store metadata with the file.

By default the stored metadata value for uploaded files is null.

Example:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   metadata: function(req, file, cb) {
      cb(null, req.body);
   }
});
var upload = multer({ storage: storage });

In this example the contents of the request body are stored with the file. This is only for illustrative purposes. If your users send passwords or other sensitive data in the request those will be stored unencrypted in the database as well, inside the metadata of the file.

chunkSize

Type: Number or Function

Not required

The preferred size of file chunks. Default value is 261120. You can use a fixed number as the value or a function to use different values per file.

Example using fixed value:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   chunkSize: 2048
});
var upload = multer({ storage: storage });

Example using dynamic value:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   chunkSize: function(req, file, cb) {
       if (file.originalname === 'myphoto.jpg') {
           cb(null, 12345);
       } else {
           cb(null, 261120);
       }
   }
});
var upload = multer({ storage: storage });

root

Type: String or Function

Not required

The root collection to store the files. By default this value is null. When the value of this property is null MongoDb will use the default collection name 'fs' to store files. This value can be changed with this option and you can use a different fixed value or a dynamic one per file.

Example using a fixed value:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   root: 'myfiles'
});
var upload = multer({ storage: storage });

Later on you can query the GridFS collection using

db.collection('myfiles.files')//...
db.collection('myfiles.chunks')//...

Example using a dynamic value:

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   root: function(req, file, cb) {
       if (file.fieldname = 'plants') {
           cb(null, 'plants');
       } else {
           cb(null, 'animals')
       }
   }
});
var upload = multer({ storage: storage });

This will create two collections of files for animals and plants based on the fieldname used to upload the file.

File information

Each file in req.file and req.files contain the following properties in addition to the ones that Multer create by default.

To see all the other properties of the file object check the Multer's documentation.

Events

Each storage object is also a standard Node.js Event Emitter. This is done to ensure that some objects are made available after they are instantiated since some options generate asynchronous database connections that needs to be established before those objects are referenced.

Event: 'connection'

Only available when the storage is created with the url option.

This event is emitted when the MongoDb connection is opened. This is useful if you want access to the internal GridFS instance for later use.

Event arguments

• gfs: The newly created GridFS instance
• db: The native MongoDb database object.

Example:

var url = 'mongodb://localhost:27017/database';
var storage = require('multer-gridfs-storage')({
   url: url
});
var upload = multer({ storage: storage });

storage.once('connection', function(gfs, db) {
   // Do something with the GridFS or the MongoDb instance
   console.log('MongoDb connected in url ' + url);
});

This event is only triggered once. Note that if you only want to log events there is an api option for that

Event: 'file'

This event is ememitted every time a new file is stored in the db. This is useful when you have a custom logging mechanism and want to record every uploaded file.

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database'
});
var upload = multer({ storage: storage });

storage.on('file', function(file) {
   // Do something with the file
   console.log('New file uploaded ' + file.originalname);
});

Debugging

To make debugging easy you can use any of the logging options in the storage constructor.

log

Type: Boolean or Function

Default: false

Not required

Enable or disable logging.

By default the module will not output anything. Set this option to true to log when the connection is opened, files are stored or an error occurs. This is useful when you want to see logging about incoming files.

If a function is provided it will be called in every log event with two arguments err y log with the error or the message respectively. The log object contains two properties message and extra corresponding to the event that triggered the log and any additional info, eg. the uploaded file

var storage = require('multer-gridfs-storage')({
   url: 'mongodb://localhost:27017/database',
   log: function(err, log) {
      if (error) {
        console.error(err);
      } else {
        console.log(log.message, log.extra);
      }
   }
});
var upload = multer({ storage: storage });

See logLevel for more information on how logging behaves on different options.

The console is used to log information to stdout or stderr

logLevel

Not required

The events to be logged out. Only applies if logging is enabled.

Type: string

Default: 'file'

Possible values: 'all' or 'file'

If set to 'all' and the connection is established using the url option some events are attached to the MongoDb connection to output to stdout and stderr when the connection is established and files are uploaded.

If set to 'file' only successful file uploads will be registered. This setting applies to both the gfs and the url configuration.

This option is useful when you also want to log when the connection is opened or an error has occurs. Setting it to 'all' and using the gfs option has no effect and behaves like if it were set to 'file'.

Test

To run the test suite, first install the dependencies, then run npm test:

$ npm install
$ npm test

Tests are written with mocha and chai. You can also run the tests with:

In case you don't have mocha installed

$ npm install mocha -g

and then run

$ mocha

Code coverage thanks to istanbul

$ npm coverage

License

MIT