React-Tensorflow

A library of React hooks and HOCs written in Typescript to use Tensorflow models in your application! 🤖🧠

Installation

yarn add react-tensorflow

npm i react-tensorflow -S

Peer dependencies

• react >=16.8.0
• @tensorflow/tfjs >=2.0.0

Basic usage

import { useModel } from 'react-tensorflow'

const MyModelComponent = () => {
  const model = useModel(`${PATH_TO_MODEL}`)

  // ...do something with the model

  return null
}

API

useModel

useModel(url?: string, opts?: { layers: boolean }): GraphModel | LayersModel | null

If url is omitted useModel will look to find the ModelProvider as it's context for returning the model. When loading a model with this hook, the opts.layers boolean is passed if your TF model should be loaded with the function tf.loadLayersModel otherwise it is assumed the model should be loaded with tf.loadGraphModel.

ModelProvider

<ModelProvider url={string} layers={boolean}>
  <App />
</ModelProvider>

Wraps the children in a React Provider to be consumed by Context's in either the useModel hook or withModel HOC. The props passed to this provider are the same as the documented props for useModel.

withModel

withModel(Component: React.ComponentType): JSX.Element

Wraps the provided component in a React Context, passing the model give to the provider as a prop.

useWebcam

useWebcam (options?: {
    width?: number
    height?: number
    facingMode?: string
  }): [React.MutableRefObject, tf.Tensor | null]

Provides a ref to be used on a video element, the hook then returns a tensor with shape [1, width, height, 3] where the width and height are either dictated by the element's width & height or the provided argument documented above. The options argument while documented above can infact take all the properties of the MediaStreamConstraints.

usePrediction

usePrediction (options?: {
  modelUrl?: string
}): [React.MutableRefObject, tf.Tensor | tf.Tensor[] | tf.NamedTensorMap | null]

Provides a ref to the data you want to use to create a prediction. The data must be in the form of a tensor. It then returns a new Tensor as the prediction using either the model set with the ModelProvider component or by passing a url in the options argument as it uses useModel under the hood.

Roadmap

So far there is only one hook to include the model in your react-app. As this library goes the plan would be to add more specific hooks to use with specific models e.g usePredicition as well as other helper hooks such as useWebcam.

• useModel as hook and HOX
• useWebcam
• usePrediction
• useClassifier
• useObjects
• usePoses

Suggestions should be made through the issues tab.