JSPM

JS SDK for running quantized small language models in the browser

use `npm i --save llama.native.js` to run lama.cpp models on your local machine. features a socket.io server and client that can do inference with the host of the model.

serve websocket GGML 4/5bit Quantized LLM's based on Meta's LLaMa model with llama.ccp