price-finder

The price-finder module helps find the price of retail items online. It does this by taking in a URI for a webpage that displays the product information, and scrapes the page to find the price (optionally able to find the name and category of products as well).

Quick Examples

Find an item's current price online

var PriceFinder = require("price-finder");

var priceFinder = new PriceFinder();

// Atoms for Peace : Amok  (from Amazon)
var uri = "http://www.amazon.com/Amok/dp/B00BIQ1EL4";
priceFinder.findItemPrice(uri, function(err, price) {
    console.log(price); // 8.91
});

// Ferris Bueller's Day Off  (from BestBuy)
uri = "http://www.bestbuy.com/site/ferris-buellers-day-off-dvd/7444513.p?id=47476&skuId=7444513";
priceFinder.findItemPrice(uri, function(err, price) {
    console.log(price); // 3.99
});

Find additional details on an item, including price

var PriceFinder = require("price-finder");

var priceFinder = new PriceFinder();

// Plants vs Zombies  (from Google Play)
var uri = "https://play.google.com/store/apps/details?id=com.popcap.pvz_na";
priceFinder.findItemDetails(uri, function(err, itemDetails) {
    console.log(itemDetails.price);    // 0.99
    console.log(itemDetails.name);     // Plants vs. Zombies™
    console.log(itemDetails.category); // Mobile Apps
});

// Blues Brothers  (from Amazon)
uri = "http://www.amazon.com/Blues-Brothers-Blu-ray-John-Belushi/dp/B001AQO446";
priceFinder.findItemDetails(uri, function(err, itemDetails) {
    console.log(itemDetails.price);    // 13.01
    console.log(itemDetails.name);     // The Blues Brothers [Blu-ray] (1980)
    console.log(itemDetails.category); // Movies & TV
});

Price Finder Documentation

Configuration Options

When creating a new PriceFinder object, a configuration object can be specified. The following options are configurable:

• headers : Any HTTP headers that should be sent within the page scrape request. Defaults to "User-Agent": "Mozilla/5.0".
• retryStatusCodes : An array of status codes (Numbers) which when returned from the page scrape request, will trigger a retry request (meaning it will attempt to scrape the page again). Defaults to [503].
• retrySleepTime : If a retry status code is returned from a page scrape request, this is the amount of time (in milliseconds) that the code will sleep prior to re-issuing the request. Defaults to 1000 (ms).

For example:

var PriceFinder = require("price-finder");

var priceFinder = new PriceFinder({
    retrySleepTime: 2000
});

API

findItemPrice(uri, callback)

Given a uri (that is for a supported site), this function will scrape the page and attempt to find the current price listed on the page, sending it to the callback. The callback's arguments are:

• error : If an error occurred during processing, this will contain the error information. If no errors occurred, this will be null.
• price : The current price of the item listed on the page (a Number).

findItemDetails(uri, callback)

Given a uri (that is for a supported site), this function will scrape the page and attempt to find the item details listed on the page, sending it to the callback. The callback's arguments are:

• error : If an error occurred during processing, this will contain the error information. If no errors occurred, this will be null.
• itemDetails : This object contains three things:
  • price : The current price of the item listed on the page (a Number).
  • name : The name of the product (if supported by the site implementation).
  • category : The category of the product (if supported by the site implementation).

Debugging Price Finder

The debug package is used within price-finder to output debugging information useful in tracking down any potential issues. To enable, export the DEBUG environment variable set to price-finder* to pick up all files (or include a specific library to only enable a certain module). For example:

$ DEBUG=price-finder* node app.js

Supported Sites

The current supported sites, along with categories supported within each site, are listed below.

• Amazon
• Digital Music
• Video Games
• Mobile Apps
• Movies & TV
• Camera & Video
• Toys & Games
• Kindle Books
• Books
• Household
• Health & Personal Care
• Google Play
• Music
• Movies & TV
• Android Apps
• Best Buy
• Movies & TV
• Music
• Video Games

Don't see your site listed? Please consider contributing to the project!

Contributing

The price-finder project is a Node.js module, so before cloning the repository make sure node is installed. Once cloned, install dependencies by issuing:

npm install

Tests

The project uses Jasmine for tests (please add tests for any new features). To run the unit tests execute:

npm test

These tests can additionally be run once, while watching the files for any changes, and if that occurs, re-run the tests. To do so execute:

npm run test-watch

End-to-end tests exist which will test the price-finder module using real URIs, scraping the pages to verify the code works correctly. Note that these tests should be run on a limited basis while coding since some sites have been known to throw up CAPTCHA's after repeated, automated page requests. To execute these tests run:

npm run test-e2e

Adding Sites

This project was built to easily drop in support for new sites. The site-manager iterates over all files contained within the sites directory, and adds it to the list of available sites. When a request is issued to price-finder to look up a price, it asks each site if the uri is supported by the site, and if so, uses that site to find the price (or name, category).

The site interface is:

function Site(uri) {
    
    /**
     * Returns the URI used to find the page data
     * (most likely the same URI used in constructing this Site)
     * 
     * @return {String} The URI used to find the page data
     */
    function getURIForPageData();

    /**
     * Returns true if the page data is JSON
     *  
     * @return {Boolean} true if the page data is JSON, false otherwise
     */
    function isJSON();

    /**
     * Returns the price found on the page
     * 
     * @param  {Object} $ jQuery object used to search the page
     * @return {String}   The price found on the page
     */
    function findPriceOnPage($);

    /**
     * Returns the category of the item found on the page
     * 
     * @param  {Object} $ jQuery object used to search the page
     * @return {String}   The category found on the page
     */
    function findCategoryOnPage($);

    /**
     * Returns the name of the item found on the page
     * 
     * @param  {Object} $        jQuery object used to search the page
     * @param  {String} category The product's category
     * @return {String}          The category found on the page
     */
    function findNameOnPage($, category);
}

/**
 * Returns true if this site supports the incoming URI
 * 
 * @param  {String}  uri The URI to test
 * @return {Boolean}     true if this Site supports the URI, false otherwise
 */
Site.isSite = function(uri);

License

MIT