rotating-file-stream

Usage

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M', // rotate every 10 MegaBytes written
    interval: '1d',  // rotate daily
    compress: 'gzip' // compress rotated files
});

under development

This package is currently under development.

Please check the TODO list to be aware of what is missing.

Installation

With npm:

npm install rotating-file-stream

API

rfs(filename, options)

Returns a new stream.Writable to filename as fs.createWriteStream does. The file is rotated following options rules.

filename {String|Function}

The most complex problem about file name is: "how to call the rotated file name?"

The answer to this question may vary in many forms depending on application requirements and/or specifications. If there are no requirements, a String can be used and default rotated file name generator will be used; otherwise a Function which returns the rotated file name can be used.

function filename(time, index)

• time: {Date} The start time of rotation period. If null, the not rotated file name must be returned.
• index {Number} The progressive index of rotation by size in the same rotation period. Starts from 1.

An example of a complex rotated file name generator function could be:

function pad(num) {
    return (num + "").length == 1 ? "0" + num : num;
}

function generator(time, index) {
    if(! time)
        return "file.log";

    var month  = time.getFullYear() + "" + pad(time.getMonth() + 1);
    var day    = pad(time.getDate());
    var hour   = pad(time.getHours());
    var minute = pad(time.getMinutes());

    return "/storage/" + month + "/" + month + day + "-" + hour + minute + "-" + index + "-" + filename;
}

var rfs    = require('rotating-file-stream');
var stream = rfs(generator, {
    size:     '10M',
    interval: '1d'
});

Note:

If part of returned destination path does not exists, the rotation job will try to create it.

options {Object}

• compress: {String} (default: null) Specifies compression method of rotated files.
• interval: {String} (default: null) Specifies the time interval to rotate the file.
• size: {String} (default: null) Specifies the file size to rotate the file.
• highWaterMark: {Number} (default: 16K) Proxied to new stream.Writable
• mode: {Integer} (default: 0o666) Proxied to fs.open

size

Accepts a positive integer followed by one of these possible letters:

• K: KiloBites
• M: MegaBytes
• G: GigaBytes

  size: '300K', // rotates the file when its size exceeds 300 KiloBytes

  size: '100M', // rotates the file when its size exceeds 100 MegaBytes

  size: '1G', // rotates the file when its size exceeds a GigaBytes

interval

Accepts a positive integer followed by one of these possible letters:

• m: minutes. Accepts integer divider of 60.
• h: hours. Accepts integer divider of 24.
• d: days

  interval: '5m', // rotates the file at minutes 0, 5, 10, 15 and so on

  interval: '2h', // rotates the file at midnight, 02:00, 04:00 and so on

  interval: '1d', // rotates the file at every midnight

compress

Due the nature of Node.js compression may be done with an external command (to use other CPUs than the one used by Node.js to not subtract CPU power to our application) or with internal code (to use the CPU used by Node.js to not subtract more CPU power than expected to the system). This decision is left to you.

Following fixed strings are allowed to compress the files with internal libraries:

• bzip
• gzip
• zip

To enable external compression, a function can be used or simple the boolean true value to use default external compression. The two following code snippets have exactly the same effect:

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M',
    compress: true
});

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M',
    compress: function(src, dst) {
        return "cat " + src + " | gzip -t9 > " + dst;
    }
});

Events

Custom Events are emitted by the stream.

var rfs    = require('rotating-file-stream');
var stream = rfs(...);

stream.on('error', function(err) {
    // here are reported errors occurred while rotating as well write errors
});

stream.on('rotation', function() {
    // rotation job started
});

stream.on('rotated', function(filename) {
    // rotation job completed with success and produced given filename
});

Under the hood

Logs should be handled so carefully, so this package tries to never overwrite files.

At stream creation, if the not rotated log file already exists and its size exceeds the rotation size, an initial rotation attempt is done.

At each rotation attempt a check is done to verify that destination rotated file does not exists yet; if this is not the case a new destination rotated file name is generated and the same check is performed before going on. This is repeated until a not existing destination file name is found or the package is exhausted. For this reason the rotated file name generator function may be called several times for each rotation job.

To not waste CPU power checking size for rotation at each write, a timer is set up to check size at every second. This means that rotated file size will be a bit greater than how much specified with options.size parameter.

Licence

MIT Licence

Bugs

Do not hesitate to report any bug or inconsistency @github.

TODO

• Write tests
• Write code
• Emit events
• External compression
• Internal compression gzip
• Internal compression bzip
• Internal compression zip

Changelog

• 2015-09-14 - v0.0.1
  • README.md
• 2015-09-10 - v0.0.0
  • Embryonal stage