solpp

A preprocessor and flattener for Ethereum's Solidity source files.

solpp is designed specifically for Solidity, which means it actually understands some of Solidity's grammar and offers high-precision math and builtin functions suitable for use with Solidity's primitives.

Features

• Flattens your source files for easy contract verification on etherscan.io by merging all naked imports.
• Will even include URL imports, along with their dependencies.
• Simple, practical language inspired by C preprocessor directives, python, and javascript.
• Easily declare symbols and macro functions in your source file with #def and #macro if directives.
• #if/#elif/#else blocks for conditional code rendering.
• #for blocks for repeating code.
• Expand (substitute) with ${...} or evaluate with $${...} symbols, macros, and expressions anywhere in your code.
• All math is done in extremely high precision (120 digits) and can represent both positive and negative integers or decimals.
• Robust expression syntax with many useful builtin functions.

Topics

• Example
• Installation
• Command Line Usage
  • CLI Options
  • Instructions
    • External Symbols
• Library Usage
• Language Reference
  • Directive Syntax
  • Symbols
  • Macros
  • Expansion
  • Evaluation
    • Inline Evaluation
  • If/Elif/Else Blocks
  • For Loops
  • Expressions
    • Operations
    • Literals
  • Builtins
    • Language Functions
    • Numerical Functions
    • String Functions
    • List Functions
    • Ethereum Functions
    • Constants

Example

pragma solidity^0.4.24;

// solpp will inline this file and any of its dependencies.
import './MyLibrary.sol';
// solpp can also do the same with URLs!
import 'https://raw.githubusercontent.com/OpenZeppelin/openzeppelin-solidity/master/contracts/token/ERC20/IERC20.sol';

contract MyContract {
   // Define and use a symbol.
   // #def EIGHT_QUARTERS 8 / 4
   uint256 _var1 = ${EIGHT_QUARTERS}; // -> uint256 _var1 = 8 / 4;
   // Evaluate a symbol as an expression. Note the double $.
   uint256 _var2 = $${EIGHT_QUARTERS}; // -> uint256 _var2 = 2;
   // We can even remove the symbol if we don't need it anymore.
   // #undef EIGHT_QUARTERS

   // Define and use a macro.
   // #macro POW(a, b) a ** b
   // Here we expand then evaluate the symbol.
   uint256 _var3 = ${POW(2, 3)} + $${POW(16, 0.5)}; // -> uint256 _var3 = 2 ** 3 + 4;
   // Macros can call other macros and symbols during evaluation.
   // #macro SQUARE(x) POW(x, 2)
   uint256 _var4 = $${SQUARE(10)}; // -> uint256 _var4 = 100;
   // You can also evaluate expressions inline.
   // Here we compute LOG(10) expressed as parts per million
   // using some builtin functions.
   uint256 _log10 = $${int(log(10) * 1e6)}; // -> uint256 _log10 = 2302585;
   // All math supports decimals. here we output sqrt(2) as a
   // decimal string with 16 digits of precision.
   string _sqrt2 = $${quote(sd(2**0.5, 16))}; // -> string _sqrt2 = "1.414213562373095";
   // We can even do bitwise operations on integers and output hex.
   bytes32 _bytes = $${hex(int((2**0.5) * 1e18) << 8)}; // -> bytes32 _bytes = 0x13a04bbdfdc9be8800;
   // Maybe we want to combine some strings.
   string _fullName = $${quote(join(['Bob', 'Smith'], ' '))}; // -> string _fullName = "Bob Smith";
   // Convert a private key to an address? Sure!
   // #def PRIV_KEY 0x563b99585e0709e3a7ac78b8957aa0f53bc874a86f288884d7ccafe3b9e9b934
   address _addr = $${key2addr(PRIV_KEY)}; // -> _addr = 0x86c0bfFbA7b505c82f1533aFe6C69A604c3e2870;

   function foo(uint256 x) pure returns (uint256) {
      // #if EXT_SYMBOL
      // If the symbol EXT_SYMBOL is defined externally or in a source file
      // as "truthy" (non-false, non-zero), this code will be rendered.
      return x * 100;
      // #else
      // Otherwise, this code will be rendered.
      return x / 100;
      // #endif
   }

   function bar(uint256 x) pure returns (uint256) {
     // Repeat code with a a for loop.
     return x /* #for V in range(1,4) */+ $${V+1}/* #done */; // -> return x + 1 + 2 + 3;
   }
}

Installation

If your project scaffolding is node-based, you can install it as a development dependency inside your project directory. As a development dependency, you should be able to call solpp from within your project as if it were installed globally.

# Install as a project development dependency with npm.
npm install -D solpp
# or if using yarn
yarn add -D solpp

If that doesn't work, or if you want solpp to be accessible from anywhere in your filesystem, you'll need to install it globally.

# Install globally with npm.
npm install -g solpp
# or if using yarn
yarn global add solpp

Command Line Usage

CLI Options

Usage: solpp [options] <source-file>

Options:
  -v --version                 output the version number
  -o, --output <file>          write output to a file (instead of stdout)
  --no-flatten                 do not flatten (include) naked imports
  -D, --define <name>[=value]  define a preprocessor symbol (can be repeated)
  --defs <file>                preprocessor definitions JS or JSON file
  --tolerant                   ignore missing imports when flattening
  -h, --help                   output usage information

Instructions

You should invoke solpp for the contract file that you want to preprocess. It will automatically include and preprocess all naked imports found within the file and subsequent files. Naked imports are of the form import "path/to/file";. Any other import scheme will be ignored.

solpp is not a compiler, so you will still need to compile the generated code yourself, via solc or a build pipeline like truffle.

If you are using a build pipeline such as truffle, you should keep your raw source files in a separate directory and run solpp to output code into the pipeline's source directory before compiling your project.

External Symbols

You can define preprocessor symbols inside source files using the #def directive or you can define them externally on the command line (with the -D flag) or in a JS/JSON file. The order of priority in which redundantly declared symbols override each other, from highest priority to lowest priority, is Source -> Command Line -> Definitions File.

A JSON definitions file is just a plain object such as:

{
   "MY_SYMBOL_1": 100,
   "MY_SYMBOL_2": true,
   "MY_SYMBOL_3": "48192.418291248",
   "MY_SYMBOL_4": "blah blah"
}

If using a JS file, it will be imported as a nodejs module, so you must assign your object to module.exports. The advantage of using a JS file is you can define symbols as functions, which can be called from evaluations in your code.

Library Usage

If you require('solpp') into your nodejs project, you can use it programmatically. This allows solpp to be integrated into any pipeline.

The library exposes 3 functions:

const solpp = require('solpp');

// Preprocess code. Returns processed code.
PROCESSED_CODE: String = solpp.processCode(
   // The raw code to process.
   code: String,
   // Options object. All fields are optional.
   opts={
      // Dictionary of preprocessor symbols.
      defs: {...},
      // Unique name for the source unit (like a full path).
      name: String,
      // The current working directory.
      cwd: String,
      // Function to resolve imported files.
      resolver: Function(path, cwd, from),
      // Whether or not to collapse 2 or more empty lines. Defaults to true.
      collapseEmptyLines: Boolean,
      // Whether to ignore unresolved imports. Defaults to false.
      tolerant: Boolean
});

// Preprocess code in a file. Returns processed code.
PROCESSED_CODE: String = solpp.processFile(
   // The path of the raw code file.
   path: String,
   // Same as in processCode().
   opts={...});

// The default resolver used when none is set.
// Will either resolve filesystem files or remote files (URLs).
// Returns an object.
{
   // The raw code in the file.
   code: String
   // A unique name for the file (usually just the full path or URL)
   name: String,
   // The parent directory (or URL path) of the file.
   cwd: String,
} = solpp.resolver(
      // The import path. May be a relative or absolute path, or even a URL.
      path: String,
      // The current working directory, or parent URL path if coming from a remote file.
      cwd: String,
      // The name of the importing file.
      from: String
   );

Language Reference

Directive Syntax

All solpp directives are contained inside either line comments or block comments. This design is to maintain compatibility with existing Solidity editor highlighters. You can use line and block comment directives interchangeably. Line comment directives terminate at the end of the line, unless the line ends in a \ character, which allows you to continue the directive on the next immediate line comment.

// Single line comment directive.
// #def LDEF All this text is the value of LDEF.

// A line comment directive spread across two lines.
// #def MULTILINE_LDEF If you start running out of room, you can continue \
// on the next line like so.

// Block comment directives are great in one-line #for loops or #if blocks.
// This will render: uint256 fac5 = 1 * 2 * 3 * 4 * 5;
uint256 fac5 = 1/* #for i in range(2, 6) */ * $${i}/* #done */;

Symbols

Symbols can be declared externally on the command line or in a definitions file, or from right within a source file with the #def directive. Symbols can take on arbitrary values: code snippets, expressions, strings, numbers (with or without decimals), and booleans.

Once defined, symbols (and macros) can be expanded (${...}) or evaluated ($${...}) in your code. Symbols do not have to hold valid expressions, but only those that do can be evaluated.

If you no longer need a symbol, you can undefine it with the #undef directive.

Example

// Define a symbol
// #def MY_EXPR 1 + 1
// Expand it.
${MY_EXPR} // -> 1 + 1
// Now evaluate it.
$${MY_EXPR} // -> 2

// Remove this symbol.
// #undef MY_EXPR

Macros

Macros are similar to symbols except that they take some arguments, like a function. They can be defined externally in a (javascript) definitions file or from within your code with the #macro directive.

Expanding a macro will perform a substitution of the arguments across its contents. Evaluating a macro will evaluate its contents as an expression, just like calling a function. You can even call other macros or reference other symbols inside macros during evaluation. Like symbols, macros do not have to hold valid expressions, but only those that do can be evaluated.

If you no longer need a macro, you can undefine it with the #undef directive.

Example

// Define a macro
// #macro MY_MACRO(x) (x / 2) + 1
// Expand it.
${MY_MACRO(10)} // -> (10/2) + 1
// Evaluate it.
$${MY_MACRO(10)} // -> 6

// Define a new macro that calls the other macro.
// #macro OTHER_MACRO(x) MY_MACRO(x * 10)
// Evaluate it.
$${OTHER_MACRO(8)} // -> 41

// Remove this macro.
// #undef OTHER_MACRO

Expansion

Expansion occurs whenever a ${...} block is encountered in regular code (i.e., not inside a comment or string literal). When expanding a symbol, the contents of that symbol are simply put in the place of the expansion block.

A similar result occurs when expanding a macro, except arguments will be substituted throughout the macro's contents.

Example

// Expanding a symbol.
// #def SYM_1 foo / 2
uint256 v = 1 + ${SYM_1}; // -> uint256 v = 1 + foo / 2;

// Expanding a macro.
// #def MACRO_1(x) x / 2
uint256 v2 = 1 + ${MACRO_1(100)}; // -> uint256 v = 1 + 100 / 2;

Evaluation

Evaluation occurs whenever a $${...} block is encountered in regular code (i.e., not inside a comment or string literal). When evaluating a symbol, the contents of that symbol are parsed and evaluated as an expression, with the ultimate result put in the place of the evaluation block. Macros work the same way, except they can accept arguments which will also be evaluated.

Macros and symbols call and refer to other macros, symbols, and builtins during evaluation.

If a macro or symbol does not contain a valid expression during an evaluation operation, an error will be raised. If a symbol is encountered that is undefined, it will take on the value of 0.

Example

// Evaluating a symbol.
// #def SYM_1 5 * 2
uint256 v = 1 + $${SYM_1}; // -> uint256 v = 1 + 10;

// Evaluating a macro.
// #def MACRO_1(x) x * 2
uint256 v2 = 1 + $${MACRO_1(2)}; // -> uint256 v2 = 1 + 4;

// Macro calling another macro and referencing a symbol.
// #def SYM_2 2 ** 3
// #def MACRO_2(x) MACRO_1(x) + SYM_2
uint256 v3 = $${MACRO_2(4)}; // -> uint256 v3 = 16;

Inline Evaluation

You don't have to use symbols or macros in your evaluation blocks. You can put any valid expression inside of them as well.

Example

// Compute the sqrt of 2 as parts per million.
// Here we use the builtin function 'int' to make the result an integer.
uint256 sqrt2 = $${int(2**0.5 * 1e6)};

If/Elif/Else Blocks

One of the most common uses of a preprocessor is conditional code generation. This is easily accomplished through the #if, #elif, #else, and #endif directives. Code between an #if/else/elif and #endif directive will only be rendered if the directive's condition evaluates to "truthy" (either true, non-zero, or a non-empty string). Conditions are normal preprocessor expressions, which can be as simple as a single value or a complex sequence of operations.

Blocks may also be nested, with inner blocks depending on outer blocks.

Example

// #if true
// This block will always render.
uint256 x = 100;
// #endif

// #if 0
// This block will never render
uint256 x = 200;
// #endif

// #if EXT_SYM_1
// This block will only render if EXT_SYM_1 is defined as truthy.
uint256 foo = 1;
// #elif EXT_SYM_2 == 'foobar'
// This Block will only render if EXT_SYM_1 is falsey (or undefined)
// AND EXT_SYM_1 is set to the string 'foobar'
uint256 foo = 2;
// #else
// This block will only render if the preceding two conditions fail.
uint256 foo = 3;
// #endif

Expressions

Preprocessor expressions are similar to javascript expressions, with many of the usual operations and precedence. All mathematical operations are done using high-precision (up to 120 significant digits) math, which can hold many more digits than the largest Solidity type (uint256).

It's worth noting that bitwise operations, however, will only operate on 256-bit integers, and will always result in a 256-bit unsigned integer. Attempting to perform a bitwise operation on a number with a decimal component will result in an error.

Operations

Literals

There are 3 types of literals: booleans, strings, and numbers. Internally, numbers are stored as decimal strings, so they do not lose precision.

Builtins

Many general-purpose and domain-specific builtin functions come already defined and can be called during evaluation.

Language Functions

Numerical Functions

String Functions

List Functions

Ethereum Functions

Constants