Package Exports

felig-toolkit
felig-toolkit/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (felig-toolkit) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

Felig Toolkit

A toolset for Amharic Language pre-processing 🔧

What is felig-toolkit?

It is a toolset for Amharic Language pre-processing. It includes an Amharic Stemmer, Amharic Transliterator, Amharic Stopword remover and Amharic Lexical analyzer.

Amharic Lexical Analyzer

Breaks down Amharic language corpus and returns tokens by removing any whitespace, expanding abbreviations(አ.አ -> አዲስ አበበ), removing numbers, breaking up hyphenated words, and removing punctuation (፡ ። ! ? ...).

Amharic Stopword remover

Removes commonly occuring words that have no contribution to the semantics of the corpus. Eg: እና ፡ ስለዚህ ፡ በመሆኑም...

Amharic Transliterator

Changes Unicode Amharic characters to ASCII. Exmaple: ልጆች -> ልጅኦች -> ljoc. This tool implements two types of Amharic transliteration lookup tables.

SERA (System for Ethiopic Representation in ASCII) - This system maps alphabets with similar sounds separately. Eg: (ሀ፣ሐ፡ኀ)፣(ሰ፡ሠ)፡(ጸ፡ፀ)፡(ዐ፡አ). However, in practice, these alphabets are used interchangeably and use of SERA would greatly decrease recall. NOT RECOMMENDED!
Felig - Normalizes the redundant symbols into a common symbol. RECOMMENDED!

Amharic Stemmer

Reduces the different morphological (e.g. inflectional or derivational) variations of Amharic word forms by taking an Amharic word and returning the stem through affix-removal with longest match.

Exmaple: ልጆች -> ልጅኦች -> ljoc -> lj -> ልጅ

Installation

Felig Toolkit is available as a package on NPM for use in a Node application:

# NPM
npm install felig-toolkit

Example

note: this package uses es-modules

import felig_toolkit from 'felig-toolkit'

What's Included

transliterate.felig_transliterate(word,lang): takes a single word and its' language (am/en) and returns felig-transliterated string
transliterate.sera_transliterate(word,lang): takes a single word and its' language (am/en) and returns SERA-transliterated string
removeStopwords(corpus): takes an Amharic corpus text (sentence/paragraph/multiple-paragraphs) and removes stop wprds
lexAnalyze(): takes an Amharic corpus text returns a string of tokens
stem(word): takes an Amharic word string and returns the stem as a string (async)

Contributions

felig-toolkit is open to contributions, but it is recommend to create an issue or reply in a comment to let others know what you are working on first.

How to run locally

Prerequisites

nodejs

Clone the repository
Run node index.js on the root directory

Attribution

To prepare the following tools, these academic papers were used