JSPM

  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 7
  • Score
    100M100P100Q40073F
  • License GPL-3.0-only

The purpose of the neume-network/core Extraction Worker (short: "EW") is to parallelize retrieving distributed information from various data sources by abstracting away the complexity of scaling processes accross a distributed system such as e.g. multiple node.js Worker threads.

Package Exports

  • @neume-network/extraction-worker
  • @neume-network/extraction-worker/src/worker.mjs

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (@neume-network/extraction-worker) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

Extractor Worker API

The purpose of the neume-network/extraction-worker (short: "EW") is to parallelize retrieving distributed information from various data sources by abstracting away the complexity of scaling processes accross a distributed system such as e.g. multiple node.js Worker threads.

Installation

npm i @neume-network/extraction-worker
npm i eth-fun --no-save # to install eth-fun as a peer dependency

Interface

A single interface allows sending new jobs to the neume-network-core Worker. Using postMessage and by sending a specific Extractor Worker JSON message object, within a reasonable timeframe, the worker will send back the retrieved record.

Versioning

The API uses semantic versioning to identify temporally different manifestations of the API.

Status Of This Document

The goal is to keep this document up-to-date with the actual implementation of the Extractor Worker API. If you find this document out-of-date, please consider contributing.

Extractor Worker Flow

The Extractor Worker JSON Message Object is sent via postMessage to a Worker instance. An answer of this message can then be expected within a reasonable timeframe. The EW implements a minimal queue, which is why a weak prioritization of submitted tasks can be expected.

Below is a sequence diagram of the entire flow:

JSON Message Object

The JSON object that is sent to a Worker via postMessage is versioned using semver and schematized via JSON schema. To keep things easily updatable, for now see api.mjs const schema to get an idea of what structure is mandated.