Package Exports

s3-zip
s3-zip/s3-zip.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (s3-zip) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

s3-zip

Download selected files from an Amazon S3 bucket as a zip file.

Install

npm install s3-zip

AWS Configuration

Refer to the AWS SDK for authenticating to AWS prior to using this plugin.

Usage

Zip specific files

const fs = require('fs')
const join = require('path').join
const s3Zip = require('s3-zip')

const region = 'bucket-region'
const bucket = 'name-of-s3-bucket'
const folder = 'name-of-bucket-folder/'
const file1 = 'Image A.png'
const file2 = 'Image B.png'
const file3 = 'Image C.png'
const file4 = 'Image D.png'

const output = fs.createWriteStream(join(__dirname, 'use-s3-zip.zip'))

s3Zip
  .archive({ region: region, bucket: bucket}, folder, [file1, file2, file3, file4])
  .pipe(output)

You can also pass a custom S3 client. For example if you want to zip files from a S3 compatible storage:

const { S3Client } = require('@aws-sdk/client-s3')

const s3Client = new S3Client({
  region: 'us-east-1',
  endpoint: 'http://localhost:9000',
  forcePathStyle: true
})

s3Zip
  .archive({ s3: s3Client, bucket: bucket }, folder, [file1, file2])
  .pipe(output)

Note: When passing a custom S3 client, it must be an AWS SDK v3 client (from @aws-sdk/client-s3). AWS SDK v2 clients are not supported and will result in an error.

Zip files with AWS Lambda

Example of s3-zip in combination with AWS Lambda.

Zip a whole bucket folder

const fs = require('fs')
const join = require('path').join
const {
  S3Client
} = require("@aws-sdk/client-s3")
const s3Zip = require('s3-zip')
const XmlStream = require('xml-stream')

const region = 'bucket-region'
const bucket = 'name-of-s3-bucket'
const folder = 'name-of-bucket-folder/'
const s3 = new S3Client({ region: region })
const params = {
  Bucket: bucket,
  Prefix: folder
}

const filesArray = []
const files = s3.listObjects(params).createReadStream()
const xml = new XmlStream(files)
xml.collect('Key')
xml.on('endElement: Key', function(item) {
  filesArray.push(item['$text'].substr(folder.length))
})

xml
  .on('end', function () {
    zip(filesArray)
  })

function zip(files) {
  console.log(files)
  const output = fs.createWriteStream(join(__dirname, 'use-s3-zip.zip'))
  s3Zip
   .archive({ region: region, bucket: bucket, preserveFolderStructure: true }, folder, files)
   .pipe(output)
}

Tar format support

s3Zip
  .setFormat('tar')
  .archive({ region: region, bucket: bucket }, folder, [file1, file2])
  .pipe(output)

Zip a file with protected password

s3Zip
  .setRegisterFormatOptions('zip-encrypted', require("archiver-zip-encrypted"))
  .setFormat('zip-encryptable')
  .setArchiverOptions({zlib: {level: 8}, encryptionMethod: 'aes256', password: '123'})
  .archive({ region: region, bucket: bucket }, folder, [file1, file2])
  .pipe(output)

Archiver options

We use archiver to create archives. To pass your options to it, use setArchiverOptions method:

s3Zip
  .setFormat('tar')
  .setArchiverOptions({ gzip: true })
  .archive({ region: region, bucket: bucket }, folder, [file1, file2])

Organize your archive with custom paths and permissions

You can pass an array of objects with type EntryData to organize your archive.

const files = ['flower.jpg', 'road.jpg']
const archiveFiles = [
  { name: 'newFolder/flower.jpg' },

  /* _rw_______ */
  { name: 'road.jpg', mode: parseInt('0600', 8)  }
];
s3Zip.archive({ region: region, bucket: bucket }, folder, files, archiveFiles)

Using with ExpressJS

s3-zip works with any framework which leverages on NodeJS Streams including ExpressJS.

const s3Zip = require('s3-zip')

app.get('/download', (req, res) => {
  s3Zip
    .archive({ region: region, bucket: bucket }, '', 'abc.jpg')
    .pipe(res)
})

Above should stream out the file in the response of the request.

Debug mode

Enable debug mode to see the logs:

s3Zip.archive({ region: region, bucket: bucket, debug: true }, folder, files)

Testing

Tests are written in Node Tap, run them like this:

npm t

If you would like a more fancy coverage report:

npm run coverage

Publishing

This package is automatically published to NPM when a new release is created on GitHub. The publishing workflow:

Triggers on GitHub releases
Runs tests to ensure quality
Publishes to NPM using the NPM_TOKEN secret

Setup for Maintainers

To enable automatic publishing, the repository requires an NPM_TOKEN secret to be configured in GitHub:

Generate an NPM access token with publish permissions
Add it as a repository secret named NPM_TOKEN in GitHub Settings > Secrets and variables > Actions

The workflow can also be triggered manually from the Actions tab for testing purposes.