Package Exports

stopword

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (stopword) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

stopword

stopword is a node module that allows you to strip stopwords from an input text. In natural language processing, "Stopwords" are words that are so frequent that they can safely be removed from a text without altering its meaning.

Usage

Default (English)

By default, stopword will strip an array of "meaningless" English words

sw = require('stopword')
const oldString = 'a really Interesting string with some words'.split(' ')
const newString = sw.removeStopwords(oldString)
// newString is now [ 'really', 'Interesting', 'string', 'words' ]

Other languages

You can also specify a language other than English:

sw = require('stopword')
const oldString = 'Trädgårdsägare är beredda att pröva vad som helst för att bli av med de hatade mördarsniglarna åäö'.split(' ')
// sw.sv contains swedish stopwords
const newString = sw.removeStopwords(oldString, sw.sv)
// newString is now [ 'Trädgårdsägare', 'beredda', 'pröva', 'helst', 'hatade', 'mördarsniglarna', 'åäö' ]

Custom list of stopwords

And last, but not least, it is possible to use your own, custom list of stopwords:

sw = require('stopword')
const oldString = 'you can even roll your own custom stopword list'.split(' ')
// Just add your own list/array of stopwords
const newString = sw.removeStopwords(oldString, [ 'even', 'a', 'custom', 'stopword', 'list', 'is', 'possible']
// newString is now [ 'you', 'can', 'roll', 'your', 'own']

API

<language code>

Arrays of stopwords for the following 32 languages are supplied:

af - Afrikaans
ar - Modern Standard Arabic
bn - Bengali
br - Brazilian Portuguese
da - Danish
de - German
en - English
es - Spanish
fa - Farsi
fi - Finnish
fr - French
ha - Hausa
he - Hebrew
hi - Hindi
id - Indonesian
it - Italian
ja - Japanese
lgg - Lugbara (without diacritics)
lggo - Lugbara official (with diacritics)
nl - Dutch
no - Norwegian
pl - Polish
pt - Portuguese
pa - Punjabi Gurmukhi
ru - Russian
so - Somali
st - Sotho
sv - Swedish
sw - Swahili
vi - Vietnamese
yo - Yoruba
zh - Chinese Simplified
zu - Zulu

sw = require('stopword')
norwegianStopwords = sw.no
// norwegianStopwords now contains an Array of norwgian stopwords

Languages with no space between words

ja Japanese and zh Chinese Simplified have no space between words. For these languages you need to split the text into words before feeding it to the stopword module. You can check out TinySegmenter for Japanese and chinese-tokenizer for Chinese.

Your language missing?

If you can't find a stopword file for your language, you can try creating one with stopword-trainer. We're happy to help you in the process.

removeStopwords

Returns an Array that represents the text with the specified stopwords removed.

text An array of words
stopwords An array of stopwords

sw = require('stopword')
var text = sw.removeStopwords(text[, stopwords])
// text is now an array of given words minus specified stopwords