Package Exports

html-get

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (html-get) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

html-get

Last version

Get the HTML from any website, using prerendering when is necessary.

Features

Get HTML markup from any website (client side apps as well)
Prerendering detection based on domains whitelist.
Speed up process blocking ads trackers.
Encoding body response properly.

Install

$ npm install html-get --save

Usage

'use strict'

const getHTML = require('html-get')
;(async () => {
  const url = 'https://kikobeats.com'
  const { html, stats } = await getHTML(url)
  console.log(html)
  console.log(stats)
})()

API

getHTML(url, [options])

url

Required
Type: string

The target URL for getting the HTML markup.

options

prerender

Type: boolean|string
Default: 'auto'

Enable or disable prerendering as mechanism for getting the HTML markup explicitly.

The value auto means that that internally use a list of whitelist website that don't need to use prerendering by default. This list is used for speedup the process, using fetch mode for these websites.

See fetchMode parameter for know more.

browserless

Type: object

A browserless instance to be used for interact with puppeteer. If you don't provide one, a browser instance will be created in each library call.

encoding

Type: string
Default: 'utf-8'

Encoding the HTML markup properly from the body response.

It determines the encode to use A Node.js library for converting HTML documents of arbitrary encoding into a target encoding (utf8, utf16, etc).

fetchMode

Type: function

A function evaluation that will be invoked to determinate the resolutive mode for getting the HTML markup from the target URL.

The default fetchMode is:

const getFetchMode = (url, { prerender }) => {
  if (prerender === false) return 'fetch'
  if (prerender !== 'auto') return 'prerender'
  return autoDomains.includes(parseDomain(url).domain) ? 'fetch' : 'prerender'
}

gotOptions

Type: object

Under mode=fetch, pass configuration object to got.

puppeteerOpts

Type: object

Under non mode=fetch, pass configuration object to puppeteer.

License

html-get © Kiko Beats, released under the MIT License.
Authored and maintained by Kiko Beats with help from contributors.

kikobeats.com · GitHub Kiko Beats · Twitter @Kikobeats