Preprocessor: Components

The preprocessor uses a number of classes for tracking the state within the file whilst evaluating it. It monitors every line in the input file, separating them into blocks created by macros (such as #for / #endfor and #if / #endif), and including lines from other files where required by a #include macro. The API of each of these tracking classes is documented below.

PreprocessorScope

class blade.preprocessor.scope.PreprocessorScope(name, deps=None, defines=None)

Holds a scope of the preprocessor, containing its own files, defined values, and dependencies on other scopes.

add_dependency(dependency)

Add a new dependency to this scope

Parameters

dependency – The name of the dependency to add

add_file(file_path, pre_file)

Add a new file to this scope, checks if it clashes with an existing file.

Parameters

• file_path – Path to this file on the filesystem

• pre_file – PreprocessorFile instance representing this file

property defines

Returns the map of defined values within this scope

property dependencies

Returns the names of scopes that this scope depends upon

property files

Returns the list of files held within this scope

get_file(file_path)

Return a file if it exists within the scope (else returns None)

Parameters

file_path – Full or partial path to the file to find

property name

Returns the name of the scope

set_definition(key, value)

Add a new key-value pair to the map of defined values

Parameters

• key – The key for the definition

• value – The value for the definition

PreprocessorFile

class blade.preprocessor.file.PreprocessorFile(path, scope, preprocessor, evaluated=False)

Represents a file that has been loaded for preprocessing, allows it to carry state such as what other files have been included, what values have been defined, etc. The preprocessor runs on a iterative basis, attempting to resolve files one at a time - this gets around the issue of ‘#include’ and ‘#define’ statements being made conditional on other variable defined in other files.

add_parsed_document(document)

Link a parsed document to this source file

Link a parsed version of the document back to the source code - object type is arbitrary and not checked. Multiple documents can come from one source file.

Parameters

document – The object parsed from this source file

all_included_files(top=True, recursive=False)

Return a list of all included PreprocessorFile’s

Parameters

• top – List the files that this one directly includes (default: True)

• recursive – List all files referenced by included files (default: False)

append_context(block)

Extend the current context by adding the next level of hierarchy

Parameters

block – PreprocessorBlock to append to the context stack

evaluate()

Evaluate the contents of this PreprocessorFile

Run the evaluation to construct the ‘final’ line array, expanding all blocks based on the evaluated result of statements. Note that this call returns the PreprocessorFile object and not the line array, this is so later stages can relate parsed documents back to the source code.

Returns

This instance, allowing for chaining of commands.

Return type

PreprocessorFile

property evaluated

Has this file been successfully evaluated?

get_current_context()

Returns the current context of the parse (block most recently assembled).

Returns

The block if within a context, otherwise None

Return type

PreprocessorBlock

get_input_line_file(line_no)

Maps from line in evaluated output to the input PreprocessorFile.

Parameters

line_no – Line number within the evaluation output (indexed 0 upwards)

Returns

Returns the file responsible for a line in the output

Return type

PreprocessorFile

get_input_line_number(line_no)

Maps from line in evaluated output to line number in the input file

Parameters

line_no – Line number within the evaluation output (indexed 0 upwards)

Returns

The line number within the input file

Return type

int

get_parsed_documents(en_includes=False, included=[])

Return all documents that have been parsed from this file.

Return all of the parsed documents generated from this source file. Note that the object types within the array are arbitrary and may differ. Optionally the call can also return all documents in included files as well.

Parameters

• en_includes – Includes parsed documents from #include’d files, this performs a recursive call to get_parsed_documents.

• included – Tracks already included files to avoid infinite loop

Returns

All of the documents attached to this file

Return type

list

get_result()

Return the evaluated file contents

Returns

List of output lines from evaluation as PreprocessorLine

instances

Return type

list

include_file(file, bypass=False)

Add a new file to be included from the scope when evaluating this file.

Parameters

• file – The file to include

• bypass – Bypass evaluation check, used to inject fake files (default: False)

list_all_defines()

Return all of the values defined in the scope.

Returns

Returns the map of defined values.

Return type

map

load_file()

Load file from disk and parse preprocessor syntax

Load the contents of the file from disk, evaluating each line and generating the full hierarchy of PreprocessorBlocks with PreprocessorStatements.

property loaded

Has the file been loaded from disk?

property path

Path to the file on disk

pop_context()

Move a layer up the block hierarchy.

push_to_current_context(item)

Add a new item to the current context

This can be a PreprocessorBlock, PreprocessorStatement, or a basic PreprocessorLine. If a block is pushed it will not be pushed onto the context stack, instead you should use ‘append_context’.

Parameters

item – The instance to add to the current context

resolve_value(value, line=None)

Convert any string into its final value using defined constants

Parameters

• value – The value to resolve

• line – The PreprocessorLine instance this value came from

Returns

The result of the evaluation

Return type

value

property scope

PreprocessorScope instance that contains this file

set_definition(key, value)

Add a new key-value pair to the scope’s definitions map.

Parameters

• key – The key for the definition

• value – The value of the definition

PreprocessorBlock

class blade.preprocessor.block.PreprocessorBlock(statement, file, lines=None)

Represents a block of code within the file that is encased by some form of condition. This block can be optionally included, or replicated multiple times based on the outcome of evaluate the PreprocessorStatement.

add_line(line)

Append a new line to this block

Parameters

line – The line to add which can be a PreprocessorLine, or a nested PreprocessorBlock, or PreprocessorStatement.

evaluate()

Evaluate all lines within this block and child blocks.

Works through the stored lines, flattening out nested PreprocessorBlocks, evaluating PreprocessorStatements, and returning final array of lines. Note that this will throw exceptions if the evaluation fails to complete.

Returns

A list of PreprocessorLines of the evaluated content

Return type

list

property file

Returns the file that contains this block

property statement

Returns the statement opening this block

PreprocessorForBlock

class blade.preprocessor.for_block.PreprocessorForBlock(statement, file)

Represents a block that should be repeatedly evaluated based on the iterable defined in the statement. This block doesn’t directly contain lines, but instead holds the loop’s contents in an instance of PreprocessorBlock.

add_line(line)

Add line to the block.

Overriding the inherited method, instead this appends the line to the encased PreprocessorBlock that represents the loop’s contents.

Parameters

line – The PreprocessorLine, PreprocessorStatement, or PreprocessorBlock instance to add to the block.

evaluate()

Evaluate the PreprocessorForBlock replacing uses of the iteration variable.

Evaluate the opening statement to determine how many times the loop should be repeated. The generate a result, replacing any usage of the iteration variable on each pass.

Returns

A list of lines from the evaluated block

Return type

list

PreprocessorIfBlock

class blade.preprocessor.if_block.PreprocessorIfBlock(file)

Represents a series of blocks of code encased in a ‘if-elif-else’ relationship evaluating to only return the valid section. The block itself does not contain any lines, but adding a line will append it to the last active block (e.g. whichever if/elif/else section was last added).

add_line(line)

Add a line to the last section.

Overriding the inherited method, instead this appends the line to the last added section. As with inherited method this can be a string or other preprocessor object.

Parameters

line – The line to add, this can be a PreprocessorLine, PreprocessorBlock, or PreprocessorStatement

add_section(statement)

Append a new if/elif/else section to the IF block.

Note that the statement is contained within the PreprocessorBlock in the section.

Parameters

statement – The PreprocessorStatement opening the section

evaluate()

Evaluate the PreprocessorIfBlock choosing the right section.

Work through the sections, evaluating the guard statements until a ‘True’ value is returned. For the one section with a ‘True’ value, the contents of the block will be evaluated and returned.

Returns

A list of lines from the evaluated block

Return type

list

PreprocessorLine

class blade.preprocessor.line.PreprocessorLine

Extends from ‘str’ to provide a custom string class that can track the input and output line numbers to allow us to relate parse errors back to original lines of source code.

property input_line

Get the line number in the input file this line came from

property output_line

Get the line number within the output file that this line appears at

property source_file

Return the PreprocessorFile that contains this line

strip()

Override the normal strip method to return another PreprocessorLine object

PreprocessorStatement

class blade.preprocessor.statement.PreprocessorStatement(line, regex, file)

Defines a statement that needs to be evaluated, for example an ‘IF’ condition where the evaluated result is returned and used to decide which section of code is included into the final file.

property condition

Return the condition of this statement, i.e. what needs to be evaluated

evaluate()

Evaluate the statement and return the result.

Evaluates the statement based on the extracted type and condition and returns a value. This could be true/false in the case of an IF statement, or it could be more complex like a key-value pair for a DEFINE statement.

Returns

Depending on the type of statement this could be a primitive

value (string, integer, boolean) or a complex value which is returned as a map.

Return type

value

property line

Return the raw string this statement was derived from

property type

Return the type of this statement (which regex was matched)