shiki-stream

Streaming highlighting with Shiki. Useful for highlighting text streams like LLM outputs.

[!IMPORTANT] API Unstable. Pin your version on use.

Usage

import { createHighlighter, createJavaScriptRegexEngine } from 'shiki'
import { CodeToTokenTransformStream } from 'shiki-stream'

// Initialize the Shiki highlighter somewhere in your app
const highlighter = await createHighlighter({
  langs: [/* ... */],
  themes: [/* ... */],
  engine: createJavaScriptRegexEngine()
})

// The ReadableStream<string> you want to highlight
const textStream = getTextStreamFromSomewhere()

// Pipe the text stream through the token stream
const tokensStream = textStream
  .pipeThrough(new CodeToTokenTransformStream({
    highlighter,
    lang: 'javascript',
    theme: 'nord'
  }))

Consume the token stream

Manually

for await (const token of tokensStream) {
  console.log(token)
}

tokensStream.pipeTo(new WritableStream({
  async write(token) {
    console.log(token)
  }
}))

Vue

<script setup lang="ts">
import { ShikiStreamRenderer } from 'shiki-stream/vue'

// get the token stream
</script>

<template>
  <ShikiStreamRenderer :stream="tokensStream" />
</template>

React

import { ShikiStreamRenderer } from 'shiki-stream/react'

export function MyComponent() {
  // get the token stream
  return <ShikiStreamRenderer stream={tokensStream} />
}

JSPM

shiki-stream

Package Exports

Readme

shiki-stream

Usage

Consume the token stream

Manually

Vue

React

Sponsors

License