Oniguruma-To-ES

A lightweight Oniguruma to JavaScript RegExp transpiler that runs in the browser and on your server. Use it to:

• Take advantage of Oniguruma's extended regex capabilities in JavaScript.
• Run regexes intended for Oniguruma in JavaScript, such as those used in TextMate grammars (used by VS Code, Shiki syntax highlighter, etc.).
• Share regexes across your Ruby and JavaScript code.

Compared to running the actual Oniguruma C library in JavaScript via WASM bindings (e.g. via vscode-oniguruma), this library is much lighter weight and its regexes run much faster since they run as native JavaScript.

[!WARNING] This library is currently in alpha and has known bugs.

Try the demo REPL

Oniguruma-To-ES deeply understands all of the hundreds of large and small differences in Oniguruma and JavaScript regex syntax and behavior across multiple JavaScript version targets. It's obsessive about precisely following Oniguruma syntax rules and ensuring that the emulated features it supports have exactly the same behavior, even in extreme edge cases. A few uncommon features can't be perfectly emulated and allow rare differences, but if you don't want to allow this, you can disable the allowBestEffort option to throw for such patterns (see details below).

📜 Contents

• Install and use
• API
• Options
• Supported features
• Unicode / mixed case-sensitivity

🕹️ Install and use

npm install oniguruma-to-es

import {compile} from 'oniguruma-to-es';

In browsers:

<script type="module">
  import {compile} from 'https://esm.run/oniguruma-to-es';
  compile(String.raw`…`);
</script>

<script src="https://cdn.jsdelivr.net/npm/oniguruma-to-es/dist/index.min.js"></script>
<script>
  const {compile} = OnigurumaToES;
</script>

🔑 API

compile

Transpiles an Oniguruma regex pattern and flags to native JavaScript.

function compile(
  pattern: string,
  flags?: OnigurumaFlags,
  options?: CompileOptions
): {
  pattern: string;
  flags: string;
};

The returned pattern and flags can be provided directly to the JavaScript RegExp constructor. Various JavaScript flags might have been added or removed compared to the Oniguruma flags provided, as part of the emulation process.

Type OnigurumaFlags

A string with i, m, and x in any order (all optional).

[!IMPORTANT] Oniguruma and JavaScript both have an m flag but with different meanings. Oniguruma's m is equivalent to JavaScript's s (dotAll).

Type CompileOptions

type CompileOptions = {
    allowBestEffort?: boolean;
    global?: boolean;
    hasIndices?: boolean;
    maxRecursionDepth?: number | null;
    optimize?: boolean;
    target?: 'ES2018' | 'ES2024' | 'ESNext';
};

See Options for more details.

toRegExp

Transpiles an Oniguruma regex pattern and flags and returns a native JavaScript RegExp.

function toRegExp(
  pattern: string,
  flags?: OnigurumaFlags,
  options?: CompileOptions
): RegExp;

[!TIP] Try it in the demo REPL.

toOnigurumaAst

Generates an Oniguruma AST from an Oniguruma pattern and flags.

function toOnigurumaAst(
  pattern: string,
  flags?: OnigurumaFlags
): OnigurumaAst;

toRegexAst

Generates a regex AST from an Oniguruma pattern and flags.

function toRegexAst(
  pattern: string,
  flags?: OnigurumaFlags
): RegexAst;

regex's syntax and behavior is a strict superset of native JavaScript, so the AST is very close to representing native ESNext RegExp but with some added features (atomic groups, possessive quantifiers, recursion). The regex AST doesn't use some of regex's extended features like flag x or subroutines because they follow PCRE behavior and work somewhat differently than in Oniguruma. The AST represents what's needed to precisely reproduce the Oniguruma behavior using regex.

🔩 Options

These options are shared by functions compile and toRegExp.

allowBestEffort

Allows results that differ from Oniguruma in rare cases. If false, throws if the pattern can't be emulated with identical behavior for the given target.

Default: true.

global

Include JavaScript flag g (global) in results.

Default: false.

hasIndices

Include JavaScript flag d (hasIndices) in results.

Default: false.

maxRecursionDepth

If null, any use of recursion throws. If an integer between 2 and 100 (and allowBestEffort is true), common recursion forms are supported and recurse up to the specified max depth.

Default: 6.

optimize

Simplify the generated pattern when it doesn't change the meaning.

Default: true.

target

Sets the JavaScript language version for generated patterns and flags. Later targets allow faster processing, simpler generated source, and support for additional features.

Default: 'ES2024'.

✅ Supported features

Following are the supported features by target.

Targets ES2024 and ESNext have the same emulation capabilities. Resulting regexes might have different source and flags, but they match the same strings.

Notice that nearly every feature below has at least subtle differences from JavaScript. Some features and subfeatures listed as unsupported are not emulatable using native JavaScript regexes, but support for others might be added in future versions of Oniguruma-To-ES. Unsupported features throw an error.

The table above doesn't include all aspects that Oniguruma-To-ES emulates (including error handling, most aspects that work the same as in JavaScript, and many aspects of non-JavaScript features that work the same in the other regex flavors that support them).

Footnotes

1. Target ES2018 doesn't allow Unicode property names added in JavaScript specifications after ES2018 to be used.
2. Unicode blocks are easily emulatable but their character data would significantly increase library weight. They're also a deeply flawed and arguably-unuseful feature, given the ability to use Unicode scripts and other properties instead.
3. With target ES2018, the specific POSIX classes [:graph:] and [:print:] are an error if option allowBestEffort is false, and they use ASCII-based versions rather than the Unicode versions available for target ES2024 and later.
4. Target ES2018 doesn't support nested negated character classes.
5. It's not an error for numbered backreferences to come before their referenced group in Oniguruma, but an error is the best path for Oniguruma-To-ES because (1) most placements are mistakes and can never match (based on the Oniguruma behavior for backreferences to nonparticipating groups), (2) erroring matches the behavior of named backreferences, and (3) the edge cases where they're matchable rely on rules for backreference resetting within quantified groups that are different in JS and aren't emulatable. Note that it's not a backreference in the first place if using \10 or higher and not as many capturing groups are defined to the left (it's an octal or identity escape).
6. Recursion depth is limited, and specified by option maxRecursionDepth. Any use of recursion results in an error if maxRecursionDepth is null or allowBestEffort is false. Additionally, some forms of recursion are not yet supported, including mixing recursion with backreferences, using multiple recursions in the same pattern, and recursion by group number. Because recursion is bounded, patterns that fail due to infinite recursion in Oniguruma might find a match in Oniguruma-To-ES. Future versions will detect this and throw an error.

㊗️ Unicode / mixed case-sensitivity

Oniguruma-To-ES fully supports mixed case-sensitivity (and handles the Unicode edge cases) regardless of JavaScript target. It also restricts Unicode properties to those supported by Oniguruma and the target JavaScript version.

Oniguruma-To-ES focuses on being lightweight to make it better for use in browsers. This is partly achieved by not including heavyweight Unicode character data, which imposes a couple of minor/rare restrictions:

• Character class intersection and nested negated character classes are unsupported with target ES2018. Use target ES2024 or later if you need support for these Oniguruma features.
• With targets before ESNext, a handful of Unicode properties that target a specific character case (ex: \p{Lower}) can't be used case-insensitively in patterns that contain other characters with a specific case that are used case-sensitively.
  • In other words, almost every usage is fine, including A\p{Lower}, (?i:A\p{Lower}), (?i:A)\p{Lower}, (?i:A(?-i:\p{Lower})), and \w(?i:\p{Lower}), but not A(?i:\p{Lower}).
  • Using these properties case-insensitively is basically never done intentionally, so you're unlikely to encounter this error unless it's catching a mistake.

👀 Similar projects

JsRegex transpiles Onigmo regexes to JavaScript (Onigmo is a fork of Oniguruma with mostly shared syntax and behavior). It's written in Ruby and relies on the Regexp::Parser Ruby gem, which means regexes must be pre-transpiled on the server to use them in JavaScript. Note that JsRegex doesn't always translate edge case behavior differences.

🏷️ About

Oniguruma-To-ES was created by Steven Levithan.

If you want to support this project, I'd love your help by contributing improvements, sharing it with others, or sponsoring ongoing development.

© 2024–present. MIT License.