JSPM

What

Wikipedia scrapper that returns a JSON-formatted object of all the links and images in a wiki article.

Why

For one of my projects I needed a list of related links from a wiki article including text exerpt and an image. This is a small experiment based on @bcoe's npm module which I rewrote to ES6, Promises and it's now usable outside of terminal.

How

Based on cheerio for parsing the HTML, request-promise for getting data and Bluebird for promises. Sample data returned by this module :

    {
        "title": "Foobar Article",
        "links": {
            "Link_to_another_article: {
                "text": "Another article.", // the text that was linked.
                "title": "Another_article.", // title attribute <a/> tag.
                "occurrences": 1 // number of times this article was linked.
            }
        },
        "sections": {
            "Section Heading": {
                text: "text contents of section.",
                images: ["http://foobar.jpg"] // images occurring within this section.
            }
        }
    }

Usage

Install npm module:

npm install wikifetch-modern

Use wikifetch-modern in your code. Calling wikifetch returns a Promise :

import wikifetch from 'wikifetch-modern';
// or var wikifetch = require('wikifetch-modern').default;

wikifetch('javascript')
.then(article => {
  console.log('JSON ARTICLE: ', article);
})
.catch(err => {
  // handle error
});

Building

To build the project run:

npm install && gulp

Testing

To test the project run:

gulp test

Development

Easiest way to develop is to run the watcher :

gulp watch

Credits

Based on: WikiFetch