puppeteer-extra-plugin

Installation

yarn add puppeteer-extra-plugin

API

Table of Contents

• PuppeteerExtraPlugin
  • name
  • defaults
  • requirements
  • dependencies
  • data
  • opts
  • debug
  • beforeLaunch
  • afterLaunch
  • onTargetCreated
  • onPageCreated
  • onTargetChanged
  • onTargetDestroyed
  • onDisconnected
  • onClose
  • onPluginRegistered
  • getDataFromPlugins

PuppeteerExtraPlugin

Base class for puppeteer-extra plugins.

Provides convenience methods to avoid boilerplate.

All common puppeteer browser events will be bound to the plugin instance, if a respectively named class member is found.

Please refer to the puppeteer API documentation as well.

Type: function (opts)

• opts (optional, default {})

Example:

// hello-world-plugin.js
const PuppeteerExtraPlugin = require('puppeteer-extra-plugin')

class Plugin extends PuppeteerExtraPlugin {
  constructor (opts = { }) { super(opts) }

  get name () { return 'hello-world' }

  async onPageCreated (page) {
    this.debug('page created', page.url())
    const ua = await page.browser().userAgent()
    this.debug('user agent', ua)
  }
}

module.exports = function (pluginConfig) { return new Plugin(pluginConfig) }


// foo.js
const puppeteer = require('puppeteer-extra')
puppeteer.use(require('./hello-world-plugin')())

(async () => {
  const browser = await puppeteer.launch({headless: false})
  const page = await browser.newPage()
  await page.goto('http://example.com', {waitUntil: 'domcontentloaded'})
  await browser.close()
})()

name

Plugin name (required).

Convention:

• Package: puppeteer-extra-plugin-anonymize-ua
• Name: anonymize-ua

Type: string

Example:

get name () { return 'anonymize-ua' }

defaults

• See: opts

Plugin defaults (optional).

If defined will be (deep-)merged with the (optional) user supplied options (supplied during plugin instantiation).

The result of merging defaults with user supplied options can be accessed through this.opts.

Type: Object

Example:

get defaults () {
  return {
    stripHeadless: true,
    makeWindows: true,
    customFn: null
  }
}

// Users can overwrite plugin defaults during instantiation:
puppeteer.use(require('puppeteer-extra-plugin-foobar')({ makeWindows: false }))

requirements

Plugin requirements (optional).

Signal certain plugin requirements to the base class and the user.

Currently supported:

• headful
  • If the plugin doesn't work in headless: true mode, will output a warning to the user.
• dataFromPlugins
  • In case the plugin requires data from other plugins. will enable usage of this.getDataFromPlugins().
• runLast
  • In case the plugin prefers to run after the others. Useful when the plugin needs data from others.

Type: Set<string>

Example:

get requirements () {
  return new Set(['runLast', 'dataFromPlugins'])
}

dependencies

Plugin dependencies (optional).

Missing plugins will be required() by puppeteer-extra.

Type: Set<string>

Example:

get dependencies () {
  return new Set(['user-preferences'])
}
// Will ensure the 'puppeteer-extra-plugin-user-preferences' plugin is loaded.

data

• See: getDataFromPlugins

Plugin data (optional).

Plugins can expose data (an array of objects), which in turn can be consumed by other plugins, that list the dataFromPlugins requirement (by using this.getDataFromPlugins()).

Convention: [ {name: 'Any name', value: 'Any value'} ]

Type: function ()

Example:

// plugin1.js
get data () {
  return [
    {
      name: 'userPreferences',
      value: { foo: 'bar' }
    },
    {
      name: 'userPreferences',
      value: { hello: 'world' }
    }
  ]

// plugin2.js
get requirements () { return new Set(['dataFromPlugins']) }

async beforeLaunch () {
  const prefs = this.getDataFromPlugins('userPreferences').map(d => d.value)
  this.debug(prefs) // => [ { foo: 'bar' }, { hello: 'world' } ]
}

opts

• See: defaults

Access the plugin options (usually the defaults merged with user defined options)

To skip the auto-merging of defaults with user supplied opts don't define a defaults property and set the this._opts Object in your plugin constructor directly.

Type: Object

Example:

get defaults () { return { foo: "bar" } }

async onPageCreated (page) {
  this.debug(this.opts.foo) // => bar
}

debug

Convenience debug logger based on the debug module. Will automatically namespace the logging output to the plugin package name.

# toggle output using environment variables
DEBUG=puppeteer-extra-plugin:<plugin_name> node foo.js
# to debug all the things:
DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node foo.js

Type: Function

Example:

this.debug('hello world')
// will output e.g. 'puppeteer-extra-plugin:anonymize-ua hello world'

beforeLaunch

Can be used to modify the puppeteer launch options by modifying or returning them.

Plugins using this method will be called in sequence to each be able to update the launch options.

Type: function (options)

• options Object Puppeteer launch options

Example:

async beforeLaunch (options) {
  if (this.opts.flashPluginPath) {
    options.args.push(`--ppapi-flash-path=${this.opts.flashPluginPath}`)
  }
}

afterLaunch

After the browser has launched.

Note: Don't assume that there will only be a single browser instance during the lifecycle of a plugin. It's possible that pupeeteer.launch will be called multiple times and more than one browser created. In order to make the plugins as stateless as possible don't store a reference to the browser instance in the plugin but rather consider alternatives.

E.g. when using onPageCreated you can get a browser reference by using page.browser().

Alternatively you could expose a class method that takes a browser instance as a parameter to work with:

const fancyPlugin = require('puppeteer-extra-plugin-fancy')()
puppeteer.use(fancyPlugin)
const browser = await puppeteer.launch()
await fancyPlugin.killBrowser(browser)

Type: function (browser, options)

• browser Puppeteer.Browser The puppeteer browser instance.
• options Object? The launch options used. (optional, default {})

Example:

async afterLaunch (browser, options) {
  this.debug('browser has been launched', options)
}

onTargetCreated

Called when a target is created, for example when a new page is opened by window.open or browser.newPage.

Note: This includes target creations in incognito browser contexts.

Type: function (target)

• target Puppeteer.Target

onPageCreated

Same as onTargetCreated but prefiltered to only contain Pages, for convenience.

Note: This includes page creations in incognito browser contexts.

Type: function (target)

• target Puppeteer.Target

Example:

async onPageCreated (page) {
  let ua = await page.browser().userAgent()
  if (this.opts.stripHeadless) {
    ua = ua.replace('HeadlessChrome/', 'Chrome/')
  }
  this.debug('new ua', ua)
  await page.setUserAgent(ua)
}

onTargetChanged

Called when the url of a target changes.

Note: This includes target changes in incognito browser contexts.

Type: function (target)

• target Puppeteer.Target

onTargetDestroyed

Called when a target is destroyed, for example when a page is closed.

Note: This includes target destructions in incognito browser contexts.

Type: function (target)

• target Puppeteer.Target

onDisconnected

Called when Puppeteer gets disconnected from the Chromium instance. This might happen because of one of the following:

• Chromium is closed or crashed
• The browser.disconnect method was called

Type: function ()

onClose

Sometimes onDisconnected is not catching all exit scenarios. In order for plugins to clean up properly (e.g. deleting temporary files) the onClose method can be used.

Note: Might be called multiple times on exit.

Type: function ()

onPluginRegistered

After the plugin has been registered in puppeteer-extra.

Normally right after puppeteer.use(plugin) is called

Type: function ()

getDataFromPlugins

• See: data
• See: requirements

Helper method to retrieve data objects from other plugins.

A plugin needs to state the dataFromPlugins requirement in order to use this method. Will be mapped to puppeteer.getPluginData.

Type: function (name)

• name string? Filter data by name property (optional, default null)