pymport

Use Python libraries from Node.js

Installation

Fully self-contained package

This is supported only on Windows x64, Linux x64 and macOS x64.

npm i pymport

This will install the pre-built pymport binaries and a self-contained Python 3.10 environment.

You should use pympip3 (or npx pympip3 if node_modules/.bin is not in your PATH) to install packages into this environment. Running pympip3 show pip will show you where these packages live.

pympip3 is simply a redirection to node_modules/pymport/lib/binding/<platform>/python -m pip.

Using an existing Python environment

npm i pymport --build-from-source

This will download and rebuild pymport against your own already existing Python environment.

You will need a working C++ development environment. Additionally, on Linux you will need the libpython3-dev package. On macOS the Homebrew install has everything required. On Windows you should have a working python command in your shell.

Usage

Basic Principle

All Python objects co-exist with the JavaScript objects. The Python GC manages the Python objects, the V8 GC manages the JS objects. The Python GC cannot free Python objects while they are referenced by a JavaScript object.

Python objects have a PyObject type in JavaScript. When calling a Python function, input JavaScript arguments are automatically converted. Automatic conversion to JavaScript is possible if the context permits it through valueOf and Symbol.toPrimitive. In all other cases, an explicit conversion, using fromJS()/toJS() is needed.

An additional (and optional) convenience layer, pymport.proxify, allows wrapping a PyObject in a JavaScript Proxy object that creates the illusion of directly accessing the PyObject from JavaScript.

pymport itself supports worker_thread but does not provide any locking. Unlike Node.js, Python threads share the same single environment and PyObjects will be shared among all threads.

Examples

Directly use the raw PyObject object:

import { pymport } from "pymport";

// Python: import numpy as np
// np is a PyObject
const np = pymport("numpy");

// Python: a = np.arange(15).reshape(3, 5)
// a is a PyObject
const a = np.get("arange").call(15).get("reshape").call(3, 5);

// Python: a = np.ones((2, 3), dtype=int16)
// np.get('int16') is a PyObject
const b = np.get("ones").call([2, 3], { dtype: np.get("int16") });

// Python: print(a.tolist())
// PyObject.toJS() converts to JS
console.log(a.get("tolist").call().toJS());

With proxify:

import { pymport, proxify } from "pymport";

// Python: import numpy as np
// np is a JS proxy object
const np = proxify(pymport("numpy"));

// Python: a = np.arange(15).reshape(3, 5)
// a is a JS proxy object
const a = np.arange(15).reshape(3, 5);

// Python: a = np.ones((2, 3), dtype=int16)
// np.int16 is a callable PyFunction
const b = np.ones([2, 3], { dtype: np.int16 });

console.log(a.tolist().toJS());

Even the most perverted pandas syntax can be expressed:

// df = pd.DataFrame(np.arange(15).reshape(5, 3), columns=list(['ABC']) })
const df = pd.DataFrame(np.arange(15).reshape(5, 3), {
  columns: PyObject.list(["A", "B", "C"]),
});
assert.deepEqual(df.columns.tolist().toJS(), ["A", "B", "C"]);

// df[2:3]
// In Python this is equivalent to df.__getitem__(2:3)
// In pymport item is a shortcut for __getitem__
const df2 = df.item(PyObject.slice([2, 3, null]));
assert.deepEqual(df2.values.tolist().toJS(), [[6, 7, 8]]);

// df[df["C"] <= 3]
// In Python this is equivalent to df.__getitem__(df.__getitem__("C").__le__(3))
const df3 = df.item(df.item("C").__le__(3));
assert.deepEqual(df3.values.tolist().toJS(), [[0, 1, 2]]);

Inline Python is supported through pyval (Python eval):

// fn is a PyObject
const fn = pyval("lambda x: (x + 42)");

assert.instanceOf(py_fn, PyObject);
assert.isTrue(py_fn.callable);
assert.strictEqual(py_fn.call(-42).toJS(), 0);

// with eval arguments
const array = pyval("list([1, x, 3])", { x: 4 });
assert.instanceOf(py_array, PyObject);
assert.deepEqual(py_array.toJS(), [1, 4, 3]);

// PyObjects can be passed too
const np = pymport("numpy");
const py_array = pyval("np.array([2, 1, 0]).tolist()", { np });
assert.instanceOf(py_array, PyObject);
assert.deepEqual(py_array.toJS(), [2, 1, 0]);

API

Table of Contents

• PyObject
  • callable
  • type
  • length
  • get
    • Parameters
  • has
    • Parameters
  • item
    • Parameters
  • call
    • Parameters
  • toJS
  • valueOf
  • toString
  • int
    • Parameters
  • float
    • Parameters
  • string
    • Parameters
  • dict
    • Parameters
  • list
    • Parameters
  • tuple
    • Parameters
  • slice
  • fromJS
    • Parameters
• pymport
  • Parameters
• proxify
  • Parameters
• pyval
  • Parameters

PyObject

JavaScript representation of a Python object

callable

Is the property callable

Type: boolean

type

The underlying Python type

Type: string

length

Length of the underlying object if it is defined

Type: (number | undefined)

get

Get a property from the object

Type: function (name: string): PyObject

Parameters

• name string property name

Returns PyObject

has

Check if a property exists

Type: function (name: string): boolean

Parameters

• name string property name

Returns boolean

item

Retrieve an element by index, equivalent to Python subscript operator[]

Type: function (index: any): PyObject

Parameters

• index any index

Returns boolean

call

Call a callable property from the object

Type: function (...args: Array<any>): PyObject

Parameters

• args ...Array<any> function arguments

Returns PyObject

toJS

Transform the PyObject to a plain JS object. Equivalent to valueOf().

Type: function (): any

Returns any

valueOf

Transform the PyObject to a plain JS object. Equivalent to toJS().

Type: function (): any

Returns any

toString

Use the Python str() built-in on the object

Type: function (): string

Returns string

int

Construct a PyObject integer from a JS number

Type: function (v: number): PyObject

Parameters

• number number

Returns PyObject

float

Construct a PyObject float from a JS number

Type: function (v: number): PyObject

Parameters

• number number

Returns PyObject

string

Construct a PyObject string from a JS string

Type: function (v: string): PyObject

Parameters

• string string

Returns PyObject

dict

Construct a PyObject dictionary from a JS object

Type: function (v: Record<string, any>): PyObject

Parameters

• object Record<string, any>

Returns PyObject

list

Construct a PyObject list from a JS array

Type: function (v: Array<any>): PyObject

Parameters

• array Array<any>

Returns PyObject

tuple

Construct a PyObject tuple from a JS array

Type: function (v: Array<any>): PyObject

Parameters

• array Array<any>

Returns PyObject

slice

Construct a PyObject slice from three elements (start, stop, step)

Type: function (v: any): PyObject

Returns PyObject

fromJS

Construct an automatically typed PyObject from a plain JS value

Type: function (v: any): PyObject

Parameters

• value any

Returns PyObject

pymport

Import a Python module

Parameters

• name string Python module name

Returns PyObject

proxify

Create a profixied version of a PyObject that works like a native Python object

Parameters

• v PyObject
• name string? optional name to be assigned to a proxified function
• object PyObject object to proxify

Returns any

pyval

Eval a Python fragment

Parameters

• code string
• globals (PyObject | Record<string, any>)? Optional global context
• locals (PyObject | Record<string, any>)? Optional local context
• name string Python module name

Returns PyObject

Alternatives

There is an alternative package that is more mature but with slightly different target use called node-calls-python.

node-calls-python is geared towards calling large monolithic Python subroutines. It supports asynchronous calling as it is expected that those will take significant amount of CPU time. node-calls-python does type conversions on each call.

pymport is geared towards intensive use of Python libraries in Node.js. It may support asynchronous calling in the future. The main difference is that pymport keeps the PyObject objects visible in JavaScript. For example, it allows creating a numpy array, then using the various numpy methods without converting the array back to JavaScript.