scroungejs

(c)Bumblehead, 2012,2013 MIT-license

OVERVIEW:

Scroungejs is a javascript program that processes .js and .css files. With it you may compress, concatenate and/or timestamp your files as part of a build process for your web site. It is made to be as flexible as possible and its features are selectively enabled as needed. Scroungejs concatenates files in an ordered way -each file appears after the files that it depends on.

Scroungejs provides advantages over other deployment tools:

• usable as a command line tool -no config file or framework.
• callable from a javascript file using node.js.
• it does not interfere with the node.js requirejs module system.
• compression and concatenation are enabled/disabled for each build.
• html files do not need unusual tags or attributes added to them.
• it works with almost any collection of javascript files.
• by default, it does not add javascript to files it generates.
• it is packaged with an emacs config file.

Scroungejs relies on other packages, notably: css-clean and UglifyJS2.

Read about how it works to learn more, or get started with scroungejs.

HOW IT WORKS:

Scroungejs reads files from a 'source' directory and it writes files to an 'output' directory. The output directory will have the minified and concatenated result of your web application files (.js and .css files).

Scroungejs handles burdensome tasks related to web application development and deployment. It will optionally perform these tasks:

• Update an html-formatted file to include references to web application files.

• Concatenate files. When an html file is updated, it will be updated with references to the resulting concatenated or unconcatenated files.

• Timestamp files. Scroungejs will write files that have a timestamp in the filename. When an html file is updated, it will be updated with references to the resulting timestamped filenames. Timestamped filenames ease development and deployment -no need to clear the browser cache to use the most recent file. An updated timestamp yields a unique filename and so a cached version of the file will not be used by a web browser.

• Produce output for one or more specific web applications when a directory contains files that are used by multiple web applications.

• Find console.log statements among the source files of your web application. It may also be used to remove console.log from the files that it generates.

• Remove requires statements from files that it generates. This is useful for application files that are shared by a node.js environment and a browser environment. A node.js environment should access files found in the 'source' directory and a web browser should access files found in the 'output' directory.

INSTALL:

Scroungejs may be downloaded directly or installed through npm.

• npm

$ npm install scroungejs

• Direct Download

$ git clone https://github.com/iambumblehead/scroungejs.git
$ cd scroungejs && npm install

GET STARTED:

1. Before Starting...

 > *javascript file*

 > ```javascript
var scroungejs = require('scroungejs');
scroungejs.build({  
  inputPath : './getStarted',
  isTimestamped : true,  
  isComressed : true
}, function (err, res) {
  if (err) return console.log(err);
  console.log('finished!')
});

$ cd /path/to/scroungejs

$ node ./scrounge.js
[...] read: files (2/2)
[...] write: cmpr/scrounge.js
[...] write: cmpr/index.js
[...] finish: 00:00:23 (mm:ss:ms)

The directory named 'getStarted' and its contents are provided with the Scrounge package.

$ node ./scrounge.js -i ./getStarted
[...] read: files (2/2) 
[...] write: getStarted/cmpr/fileB.js
[...] write: getStarted/cmpr/fileA.js
[...] finish: 00:00:25 (mm:ss:ms)

5. Use compression and timestamping modifiers.

$ node ./scrounge.js -i ./getStarted \
   --isTimestamped=true --isCompressed=true
[...] read: files (2/2) 
[...] ugly: (fileA.js, .js, 1/2) getStarted/fileB.js
[...] write: getStarted/cmpr/fileB_2012.07.07-15:25:57.js
[...] ugly: (fileB.js, .js, 2/2) getStarted/fileA.js
[...] write: getStarted/cmpr/fileA_2012.07.07-15:25:46.js
[...] finish: 00:00:25 (mm:ss:ms)

6. Define a dependency in fileB.js.

scroungejs will concatenate dependency-related files. Dependencies are defined in .js and .css files using the 'Requires' property.

Open fileB.js and add the following line to the top: // Requires: fileA.js.

File properties are explained in section File Properties.

./getStarted/fileB.js:

// Requires: fileA.js

7. Concatenate files

Dependency-related files are recognized as a tree. A tree is composed of one file that depends on other files 'and so on. Here fileB.js is a source file that begins a dependency tree.

When tree information is collected a visual representation of each tree is printed.

isConcatenated may be defined as true or false or it may be defined with comma separated names of trees or extension types, such as '.js'. Only matching files will be concatenated.

$ node ./scrounge.js -i ./getStarted --isConcatenated=true
[...] read: files (2/2)  
 
fileB.js  
└── fileA.js  
 
[...] join: (fileA.js, .js, 1/2) getStarted/fileA.js
[...] join: (fileB.js, .js, 2/2) getStarted/fileB.js
[...] write: getStarted/cmpr/fileB.js
[...] finish: 00:00:27 (mm:ss:ms)

8. Specify an Output Directory

If a specified output directory does not exist it is created.

$ node ./scrounge.js -i ./getStarted --isConcatenated=true \
 --outputPath=./app/public/cmpr
[...] read: files (2/2)  
 
fileB.js  
└── fileA.js  
 
[...] join: (fileA.js, .js, 1/2) getStarted/fileA.js
[...] join: (fileB.js, .js, 2/2) getStarted/fileB.js
[...] write: app/public/cmpr/fileB.js
[...] finish: 00:00:27 (mm:ss:ms)

9. Build a larger tree found in ./getStarted/app/.

Multiple trees may be discovered and concatenated and .css files that associate with a tree will be discovered as well.

$ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
  --outputPath=./app/public/cmpr --isRecursive=true --trees=app.js
[...] read: files (12/12)  
 
app.js
└─┬ ViewsAll.js
. ├─┬ ViewB.js
. │ └─┬ CtrlsAll.js
. │   ├─┬ CtrlB.js
. │   │ └── ModelB.js
. │   └─┬ CtrlA.js
. │     └── ModelA.js
. └── ViewA.js  
 
[...] join: (app.js, .css, 1/1) getStarted/app/views/ViewA.css
[...] write: app/public/cmpr/app.css
[...] join: (app.js, .js, 1/9) getStarted/app/models/ModelB.js
[...] join: (app.js, .js, 2/9) getStarted/app/controls/CtrlB.js
[...] join: (app.js, .js, 3/9) getStarted/app/models/ModelA.js
[...] join: (app.js, .js, 4/9) getStarted/app/controls/CtrlA.js
[...] join: (app.js, .js, 5/9) getStarted/app/controls/CtrlsAll.js
[...] join: (app.js, .js, 6/9) getStarted/app/views/ViewB.js
[...] join: (app.js, .js, 7/9) getStarted/app/views/ViewA.js
[...] join: (app.js, .js, 8/9) getStarted/app/views/ViewsAll.js
[...] join: (app.js, .js, 9/9) getStarted/app/app.js
[...] write: app/public/cmpr/app.js
[...] finish: 00:00:17 (mm:ss:ms)

10. Define filters for the build process.

When --extensionType=js is given, files without a js extension are ignored by the build process.

Other modifiers are explained in section Modifiers.

$ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
  --outputPath=./app/public/cmpr --isRecursive=true --tress=app.js \
  --extensionType=.js
[...] read: files (12/12)  
 
app.js
└─┬ ViewsAll.js
. ├─┬ ViewB.js
. │ └─┬ CtrlsAll.js
. │   ├─┬ CtrlB.js
. │   │ └── ModelB.js
. │   └─┬ CtrlA.js
. │     └── ModelA.js
. └── ViewA.js  
 
[...] join: (app.js, .css, 1/1) getStarted/app/views/ViewA.css
[...] write: app/public/cmpr/app.css
[...] join: (app.js, .js, 1/9) getStarted/app/models/ModelB.js
[...] join: (app.js, .js, 2/9) getStarted/app/controls/CtrlB.js
[...] join: (app.js, .js, 3/9) getStarted/app/models/ModelA.js
[...] join: (app.js, .js, 4/9) getStarted/app/controls/CtrlA.js
[...] join: (app.js, .js, 5/9) getStarted/app/controls/CtrlsAll.js
[...] join: (app.js, .js, 6/9) getStarted/app/views/ViewB.js
[...] join: (app.js, .js, 7/9) getStarted/app/views/ViewA.js
[...] join: (app.js, .js, 8/9) getStarted/app/views/ViewsAll.js
[...] join: (app.js, .js, 9/9) getStarted/app/app.js
[...] write: app/public/cmpr/app.js
[...] finish: 00:00:12 (mm:ss:ms)

11. Controlling the tree display.

To make scroungejs display the tree and stop, use the modifier stop=tree

By default, each dependency is only displayed once. This makes the tree smaller when, for example, a library is required by many files. To view the entire tree, use the modifier treeView=full

$ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
  --isRecursive=true --stop=tree --treeView=full
[...] read: files (12/12)  
 
app.js
└─┬ ViewsAll.js
. ├─┬ ViewB.js
. │ └─┬ CtrlsAll.js
. │   ├─┬ CtrlB.js
. │   │ └── ModelB.js
. │   └─┬ CtrlA.js
. │     └── ModelA.js
. └─┬ ViewA.js
.   └── CtrlsAll.js  
 

12. Update index.mustache using scroungejs.

The index file contains a scrounge element.

./getStarted/index.mustache

<!doctype html>
<html>
  <head>
    <script src="/scr/libFile.js" type="text/javascript"></script>
  </head>
  <body>  
    <!-- <scrounge.js> -->
    <!-- </scrounge> -->
  </body>
</html>

13. modify the basepage index.mustache with scroungejs

Call scroungejs without concatenation on the basepage.

using -s makes the output 'silent'

$ node ./scrounge.js -s -i ./getStarted/app \
--basepage=./getStarted/index.mustache --isRecursive=true \
--outputPath=./app/public/cmpr

./getStarted/index.mustache

<!doctype html>
<html>
  <head>
    <script src="/app/lib/library.js" type="text/javascript"></script>
  </head>
  <body>
    <!-- <scrounge.js> -->
    <script src="/app/public/cmpr/library.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/app2.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ModelB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ModelA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlsAll.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewsAll.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/app.js" type="text/javascript"></script>     
    <!-- </scrounge> -->
  </body>
</html>

13. modify the basepage index.mustache and use concatenation.

Call scroungejs with concatenation on this basepage. scroungejs correctly modifies the basepage for concatenated files.

$ node ./scrounge.js -s -i ./getStarted/app --isConcatenated=true \
--basepage=./getStarted/index.mustache --isRecursive=true \
--outputPath=./app/public/cmpr

./getStarted/index.mustache

<!doctype html>  
<html>  
  <head>  
    <script src="/app/lib/library.js" type="text/javascript"></script>  
  </head>  
  <body>  
    <!-- <scrounge.js> -->  
    <script src="/app/public/cmpr/library.js" type="text/javascript"></script>  
    <script src="/app/public/cmpr/app2.js" type="text/javascript"></script>  
    <script src="/app/public/cmpr/app.js" type="text/javascript"></script>  
    <!-- </scrounge> -->  
  </body>  
</html>  

14. define a public path and update index.mustache.

This index page may reference a root web directory served from app/public. In this case, files copied to /app/public/cmpr/ should be accessed through the path /cmpr.

Scroungejs will modify a basepage to reference scripts from a publicPath.

$ node ./scrounge.js -s -i ./getStarted/app --isConcatenated=true \
  --basepage=./getStarted/index.mustache --isRecursive=true \
  --outputPath=./app/public/cmpr --publicPath=/cmpr

./getStarted/index.mustache

<!doctype html>  
<html>  
  <head>  
    <script src="/app/lib/library.js" type="text/javascript"></script>
  </head>  
  <body>  
    <!-- <scrounge.js> -->  
    <script src="/cmpr/library.js" type="text/javascript"></script>
    <script src="/cmpr/app.js" type="text/javascript"></script>  
    <script src="/cmpr/app2.js" type="text/javascript"></script>  
    <!-- </scrounge> -->  
  </body>  
</html>

15. define tree attributes.

If a tree attribute is defined in a scrounge element, only the file or files that result from the named tree are added to the scrounge element.

Trees may be defined in a scrounge element attribute or by using the --trees modifier. When trees are defined, scrounge will output the defined trees only.

./getStarted/index.mustache

<!doctype html>  
<html>  
  <head>  
    <script src="/app/lib/library.js" type="text/javascript"></script>  
    <!-- <scrounge.css trees="app.js"> -->     
    <!-- </scrounge> -->  
  </head>  
  <body>  
    <!-- <scrounge.js trees="app.js"> -->  
    <!-- </scrounge> -->  
  </body>  
</html>  

$ node ./scrounge.js -s -i ./getStarted/app
  --basepage=./getStarted/index.mustache --isRecursive=true --publicPath=/cmpr

./getStarted/index.mustache

<!doctype html>  
<html>  
  <head>  
    <script src="/app/lib/library.js" type="text/javascript"></script>  
    <!-- <scrounge.css trees="app.js"> -->
    <script src="/cmpr/app.css" type="text/javascript"></script>  
    <!-- </scrounge> -->     
  </head>  
  <body>  
    <!-- <scrounge.js trees="app.js"> -->  
    <script src="/cmpr/app.js" type="text/javascript"></script>  
    <!-- </scrounge> -->  
  </body>  
</html>  

16. You are now ready to use scroungejs.

FILE PROPERTIES:

Each file processed by scroungejs may affect the build process. This is done through information added to each file.

 > *./getStarted/app/views/ViewA.css*

 > ```css
/* Filename: Main.css
 * Timestamp: 2011.06.03-22:10:20
 * Author(s): Bumblehead (www.bumblehead.com)
 */

BASEPAGE:

scroungejs may add include elements for the js and css files it processes. Only a basepage that contains one or more scrounge elements will be modified.

<!doctype html>  
<html>  
  <head>  
    <!-- <scrounge.js> -->  
    <!-- </scrounge> -->  
  </head>  
  <body></body>   
</html>  

<!-- <scrounge.js> -->
<script src="cmpr/app1.js" type="text/javascript"></script>
<script src="cmpr/app2.js" type="text/javascript"></script>
<script src="cmpr/app3.js" type="text/javascript"></script>
<!-- </scrounge> -->

<!-- <scrounge.css tree="app1.js,app2"> -->
<script src="cmpr/app1.js" type="text/javascript"></script>
<script src="cmpr/app2.js" type="text/javascript"></script>
<!-- </scrounge> -->

<!-- <scrounge.css tree="Map.css,Main.js"> -->
<!-- </scrounge> -->
<!-- <scrounge.js tree="Main.js,Crypto.js"> -->
<!-- </scrounge> -->
<!-- <scrounge.css tree="Main.js"> -->
<!-- </scrounge> -->

MODIFIERS:

• --inputPath= path, -i path, default: ./

  a systempath to a directory or file.

• --outputPath= path, -o path, default: ./cmpr

  a systempath to a directory or file.

• --publicPath= path, -p path, default: null
  a path to the files created by scrounge.

  a valid public path is discoverable on the full system path to the file. for example, /cmpr is found on the path ./getStarted/app/cmpr/app.js.

  this file's full public path would be /cmpr/app.js.

  a public path is not useful without a -basepage argument as it affects only paths generated for the specified basepage.

• --isBasepageSourcePaths= bool, default: false
  basepage include tags will reference scripts in source directory. disables compression, concatenation and copying of files. uses publicPath

  ex.

  $ node scrounge.js -l --isRecursive=true \        
    --isBasepageSourcePaths=true \  
    --basepage=~/Software/kuaweb/sources/index.html \  
    --inputPath=~/Software/kuaweb/sources/appSrc --publicPath=/appSrc

• --isCompressed= bool, extension, tree, default: false
  compress all files and/or trees before writing them to disk. If an extension or a treename are given only files that associate with the treename or extension will be compressed.

• --isConcatenated= bool, extension, tree, default: false
  concatenate all js/css files that form a tree before writing them to disk. If an extension or a treename are given only files that associate with the treename or extension will be concatenated.

• --isRecursive= bool, default: false
  discover files in nested directories on the given path.

• --isWarning= bool, default: false
  raise warnings around trace statements.

• --isLines= bool, default: false
  each compressed script on its own line.

• --isClosure= bool, default: false
  compressed js code will added in the body of anonymous self-calling function.

• --isSilent= bool, -s, default: false
  supress console messages.

• --isMintFilter= bool, default: false
  only process files that include _mint in their filename. this should only be used for the special case when a build directory has many files that should not be included in the build process. _mint distinguishes a file that should be included in the build process from a file that should not be included in the build process.

• --isTimestamped= bool, extension, tree, default: false
  add a timestamp to name of the output files. If an extension or treename are given only files that associate with the treename or extension will be timestamped

• --isRemoveRequires= bool, default: true
  remove 'requires' statements from javascript files. true by default.

• --isRemoveConsole= bool, default: false
  remove 'console.log' statements from javascript files. uses UglifyJS2 ast to comprehensively remove console.log

• --isUpdateOnly= bool, default: false
  modify include tags in a basepage only. do not build scrounge elements.

• --extensionType= type, -t type, default: null
  process files of one type only, js or css.

• --forceTimestamp= timestamp, default: null
  all timestamped files will use the given timestamp.

• --basepage= basepage, -b basepage, default: null
  update scrounge tags in the defined basepage.

  scroungejs will not modify or process a file that does not contain scrounge tags.

License:

Copyright (c) 2012 Bumblehead chris@bumblehead.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.