dl-tar

A Node.js module to download and extract a tar archive with the Observable API

const {readdirSync} = require('fs');
const dlTar = require('dl-tar');

const url = 'https://****.org/my-archive.tar';
/* my-archive
   ├── LICENSE
   ├── README.md
   ├── INSTALL
   └── bin
       └── app.exe
*/

dlTar(url, 'my/dir').subscribe({
  next({entry}) {
    if (entry.bytes !== entry.header.size) {
      return;
    }

    console.log(`✓ ${entry.header.name}`);
  },
  complete() {
    readdirSync('my/dir'); //=> ['INSTALL', LICENSE', 'README.md', 'bin']

    console.log('\nCompleted.')
  }
});

✓ bin/
✓ bin/app.exe
✓ README.md
✓ LICENSE
✓ install

Completed.

For gzipped tar (tar.gz), use dl-tgz instead.

Installation

Use npm.

npm install dl-tar

API

const dlTar = require('dl-tar');

dlTar(tarArchiveUrl, extractDir [, options])

tarArchiveUrl: String
extractDir: String (a path where the archive will be extracted)
options: Object
Return: Observable (zenparsing's implementation)

When the observable is subscribed, it starts to download the tar archive, extract it and successively send extraction progress to its observer.

When the subscription is unsubscribed, it stops downloading and extracting.

Progress

Every progress object have two properties entry and response.

entry

Type: Object {bytes: <Number>, header: <Object>}

entry.header is a header of the entry, and entry.bytes is the total size of currently extracted entry. bytes is always 0 if the entry is not a file but directory, link or symlink.

For example you can get the progress of each entry as a percentage by (progress.entry.bytes / progress.entry.header.size || 0) * 100.

dlTar('https://****.org/my-archive.tar', 'my/dir')
.filter(progress => progress.entry.header.type === 'file')
.subscribe(progress => {
  console.log(`${(progress.entry.bytes / progress.entry.header.size * 100).toFixed(1)} %`);

  if (progress.entry.bytes === progress.entry.header.size) {
    console.log(`>> OK ${progress.entry.header.name}`);
  }
});

0.0 %
0.1 %
0.3 %
0.4 %
︙
99.6 %
99.8 %
99.9 %
100.0 %
>> OK bin/app.exe
0.0 %
0.1 %
0.2 %
0.3 %
︙

response

Type: Object {bytes: <Number>, headers: <Object>}

response.headers is a response header object derived from http.IncomingMessage, and response.bytes is a total content length of the downloaded archive. content-length header will be converted to Number if it is String.

Options

You can pass options to Request and tar-fs's extract method. Note that:

• ignore option is applied before map option modifies filenames.
• strip option defaults to 1, not 0. That means the top level directory is stripped off by default.
• fs option defaults to graceful-fs for more stability.

Additionally, you can use the following:

tarTransform

Type: Stream

A transform stream to modify the archive before extraction.

For example, pass gunzip-maybe to this option and you can download both gzipped and non-gzipped tar.

const dlTar = require('dl-tar');
const gunzipMaybe = require('gunzip-maybe');

const observable = dlTar('https://github.com/nodejs/node/archive/master.tar.gz', './', {
  tarTransform: gunzipMaybe()
});

License

Copyright (c) 2017 Shinnosuke Watanabe

Licensed under the MIT License.