fparser2

Fparser2 provides support for parsing Fortran up to and including Fortran 2003. This is implemented in the Fortran2003.py file and contains an entirely separate parser to fparser1 that includes rules for Fortran 2003 syntax. Support for Fortran 2008 is being added in the Fortran2008.py file which extends the Fortran2003 rules appropriately. At this time fparser2 supports the following Fortran2008 features: submodules, co-arrays, the ‘mold’ argument to allocate, the ‘contiguous’ keyword, the ‘BLOCK’ construct, the ‘CRITICAL’ construct, the optional ‘::’ for a procedure statement and the ‘do concurrent’ construct.

Getting Going : Script

Once installed, fparser2 can be run from the command line by using the fparser2 script. One or more input files can be provided. These files are parsed in turn and the parsed Fortran is output to the screen (most likely with a different formatting to the input as fparser2 does not preserve format), or an appropriate error is output.

> cat simple.f90
program simple
end
> fparser2 simple.f90
PROGRAM simple
END PROGRAM simple
> cat error.f90
prog error
end
> fparser2 error.f90
File: 'error.f90'
Syntax error: at line 1
>>>prog error

fparser2 provides a number of command line options

> fparser2 -h
Usage: fparser2 [options] <Fortran files>

Description:
  fparser2 parses Fortran code.

Options:
  -h, --help     show this help message and exit
  --task=TASK    Specify parsing result task. Default: show.
  --std=STD      Specify the Fortran standard to use. Default: f2003.

The --task option supports show (the default) which outputs the parsed code to stdout, repr which outputs the fparser2 representation of its internal parse tree and none which outputs nothing.

The --std option chooses the flavour of Fortran to parse. Valid options are currently limited to f2003 (the default) and f2008.

Getting Going : Python

As with the other parser (fparser), the source code to parse must be provided via a Fortran reader which is an instance of either FortranFileReader or FortranStringReader (see readfortran.py).

The required parser is then created using an instance of the ParserFactory class. ParserFactory either returns a Fortran2003-compliant parser or a Fortran2008-compliant parser depending on the std argument provided to its create method.

Finally the parser is provided with the Fortran reader and returns an abstract representation (a parse-tree) of the code, if the code is valid Fortran. This parse-tree can be output as Fortran by printing it. The parse-tree hierarchy can also be output in textual form by executing it. For example:

>>> from fparser.two.parser import ParserFactory
>>> from fparser.common.readfortran import FortranFileReader
>>> reader = FortranFileReader("compute_unew_mod.f90",
                               ignore_comments=False)
>>> f2008_parser = ParserFactory().create(std="f2008")
>>> parse_tree = f2008_parser(reader)
>>> print parse_tree
MODULE compute_unew_mod
  USE :: kind_params_mod
  USE :: kernel_mod
  USE :: argument_mod
  USE :: grid_mod
  USE :: field_mod
  IMPLICIT NONE
  PRIVATE
  PUBLIC :: invoke_compute_unew
  PUBLIC :: compute_unew, compute_unew_code
  TYPE, EXTENDS(kernel_type) :: compute_unew
  ...
>>> parse_tree
Program(Module(Module_Stmt('MODULE', Name('compute_unew_mod')),Spec
ification_Part(Use_Stmt(None, Name('kind_params_mod'), '', None),Us
e_Stmt(None, Name('kernel_mod'), '', None),Use_Stmt(None, Name('arg
ument_mod'), '', None),Use_Stmt(None, Name('grid_mod'), '', None),U
se_Stmt(None, Name('field_mod'), '', None),Implicit_Part(Implicit_S
tmt('NONE')), Access_Stmt('PRIVATE', None),Access_Stmt('PUBLIC', Na
me('invoke_compute_unew')),Access_Stmt('PUBLIC', Access_Id_List(','
, (Name('compute_unew'),Name('compute_unew_code')))),Derived_Type_D
ef(Derived_Type_Stmt(Type_Attr_Spec('EXTENDS',Name('kernel_type')),
Type_Name('compute_unew'), None), ...

Note that the two readers will ignore (and dispose of) comments by default. If you wish comments to be retained then you must set ignore_comments=False when creating the reader. The parse tree created by fparser2 will then have Comment nodes representing any comments found in the code. Nodes representing in-line comments will be added immediately following the node representing the code in which they were encountered.

Preprocessing directives are retained as CppDirective objects by the readers and are represented by matching nodes in the parse tree created by fparser2. See section Preprocessing Directives for more details.

Note that empty input, or input that consists of purely white space and/or newlines, is not treated as invalid Fortran and an empty parse tree is returned. Whilst this is not strictly valid, most compilers have this behaviour so we follow their lead.

If the code is invalid Fortran then a FortranSyntaxError exception will be raised which indicates the offending line of code and its line number. For example:

>>> from fparser.common.readfortran import FortranStringReader
>>> code = "program test\nen"
>>> reader = FortranStringReader(code)
>>> from fparser.two.parser import ParserFactory
>>> f2008_parser = ParserFactory().create(std="f2008")
>>> parse_tree = f2008_parser(reader)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fparser/two/Fortran2003.py", line 1300, in __new__
    raise FortranSyntaxError(error)
fparser.two.Fortran2003.FortranSyntaxError: at line 2
>>>en

Matching Multiple Rules

There are a number of situations where more than one Fortran rule could match some input text. For example, the following text could be an array section or a substring:

a(1:2)

The Fortran specifications deal with such ambiguities by specifying constraints. In this case, constraint C608 in the Fortran2003 specification states that this text can only be a substring if the variable a is of type character.

At the moment, fparser2 attempts to match rules but does very little constraint checking and is implemented such that when there is a choice of rules, the first matching rule in the associated internal rule-list is returned. This approach is fine for parsing and de-parsing code, but it does means that the resultant parse tree might not be what would be expected. This problem is therefore passed on to any tool that makes use of the parse tree.

For example, the text a(1:2) would always match an array section (Fortran2003 rule R617) and never a substring (Fortran2003 rule R609), even when variable a is declared as type character. The reason for this is that Fortran2003 rule R603 specifies the matching of array section before the matching of substring which is how it is implemented in fparser2.

In many cases any ambiguity in rule matching can and will be removed by adding constraint checking to fparser2 and the first step in supporting this is the subject of issue #201.

However, in some situations it is not possible to disambiguate rules via constraints by simply parsing a valid Fortran file. As an illustration, if we again take the example of the text a(1:2) and constraint C608 where we need to determine whether the variable a is of type character, it may be that this variable is brought into scope via a use statement and so its datatype is unknown without finding its declaration in another module.

Fortran compilers deal with this problem by requiring code associated with use statements to have already been compiled creating mod files which contain the required information. It has not yet been decided how to deal with this problem in fparser2, but it is likely that some on-demand parsing of other files will be used to determine appropriate types.

Situations where fparser2 is known to choose the wrong parsing rule are given in the following sections.

Array element, array section or substring

A substring will always be matched as an array element. An array section will also be matched as an array element unless the array section contains an additional substring range (see Fortran 2003 rule R617).

Array or function

A function will always be matched as an array element unless that function is an intrinsic. An array access with the same name as an intrinsic function will be matched as an intrinsic function but will raise an exception if the number of dimensions of the array does not match the number of arguments expected by the intrinsic.

Statement function or array assignment

Fparser2 is currently not able to distinguish between statement functions and array assignments when one or more array assignment statements are the first statements after a declaration section. This limitation would lead to these particular array assignments being incorrectly parsed as statement functions.

To avoid this behaviour, support for statement functions has been temporarily removed from fparser2. However, with this change, statement functions will now be incorrectly parsed as array assignments when one or more statement function statements are the last statements in a declaration section.

Whilst any incorrect behaviour should be avoided, the behaviour of this temporary change is considered preferable to the former case, as array assignments are more common than statement functions.

Extensions

Many compilers support extensions to standard Fortran and codes often make use of them. This section documents the extensions to the standard that are supported by fparser2.

Cray Pointers

Cray pointers are part of a non-standard extension that provides a C-like pointer in Fortran. This is accomplished through a pair of variables: an integer “pointer” that holds a memory address, and a “pointee” that is used to dereference the pointer. For example:

pointer (my_pointer, my_pointee)

For a specification and a more detailed explanation see http://pubs.cray.com/content/S-3901/8.6/cray-fortran-reference-manual-s-3901-86/types or https://gcc.gnu.org/onlinedocs/gfortran/Cray-pointers.html.

X Format

An X edit descriptor in a format statement specifies the position (forward from the current position) at which the next character will be transmitted to or from a record. In standard Fortran2003 the X edit descriptor must be preceeded by an integer which specifies how far forward from the current position. The ‘x-format’ extension allows the X edit descriptor to be specified without a preceeding integer. When omitted, the value is implicitly assumed to be one. For example:

100 format (x)

For more information see https://gcc.gnu.org/onlinedocs/gfortran/X-format-descriptor-without-count-field.html

Hollerith Constant

A Hollerith constant is a way of specifying a string as a sequence of characters preceded by the string length and separated by an ‘H’. For example:

5Hhello
11Hhello there

Hollerith constants were used in Fortran before character strings were introduced in Fortran 77. In Fortran 77 the use of Hollerith constants was deprecated and in Fortran 95 they were removed from the language. However, many compilers still support Hollerith constants for legacy Fortran code.

The ‘hollerith’ extension adds support in fparser for Hollerith constants. This support is currently limited to those specified in format statements. There is currently no support for 1) constants in DATA statements and 2) constant actual arguments in subroutine CALL statements (which are its other uses as specified in the Fortran 66 standard).

For more information see https://gcc.gnu.org/onlinedocs/gfortran/Hollerith-constants-support.html or https://en.wikipedia.org/wiki/Hollerith_constant

Dollar Descriptor

A dollar descriptor is used to specify that the carriage return in a write statement should be suppressed. For example:

100 FORMAT ('Enter your name ',$)

This extension is not specified in the Fortran standard but is implemented by a number of compilers. The ‘dollar-descriptor’ extension adds support for the dollar descriptor in fparser.

For more information see https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-dollar-sign-and-backslash-editing

CONVERT argument to OPEN

The CONVERT argument may be used to specify how unformatted data being read from file is to be converted before being stored. For example:

OPEN(unit=23, file="some_data", form='unformatted',access='sequential', &
     convert="LITTLE_ENDIAN")

This extension is supported by (at least) the Gnu, Intel and Cray compilers but is not a part of any Fortran standard. More details can be found at https://gcc.gnu.org/onlinedocs/gfortran/CONVERT-specifier.html

Classes

class fparser.common.readfortran.FortranFileReader(file_candidate, include_dirs=None, source_only=None, ignore_comments=True, ignore_encoding=True)[source]

Constructs a FortranFileReader object from a file.

Parameters:

file_candidate – A filename or file-like object.
include_dirs (list) – Directories in which to look for inclusions.
source_only (list) – Fortran source files to search for modules required by “use” statements.
ignore_comments (bool) – Whether or not to ignore comments
ignore_encoding (Optional[bool]) – whether or not to ignore Python-style encoding information (e.g. “-- fortran --”) when attempting to determine the format of the file. Default is True.

For example:

>>> from fparser.common.readfortran import FortranFileReader
>>> import os
>>> reader = FortranFileReader('myfile.f90')

close_source()[source]: Called when self.source.next() raises StopIteration.

class fparser.common.readfortran.FortranStringReader(string, include_dirs=None, source_only=None, ignore_comments=True, ignore_encoding=True)[source]

Reads Fortran source code as a string.

Parameters:

string (str) – string to read
include_dirs (list) – List of dirs to search for include files
source_only (list) – Fortran source files to search for modules required by “use” statements.
ignore_comments (bool) – Whether or not to ignore comments
ignore_encoding (Optional[bool]) – whether or not to ignore Python-style encoding information (e.g. “-- fortran --”) when attempting to determine the format of the source. Default is True.

For example:

>>> from fparser.common.readfortran import FortranStringReader
>>> code = '''
         subroutine foo(a)
            integer a
            print*,"a=",a
          end
    '''
>>> reader = FortranStringReader(code)

class fparser.two.parser.ParserFactory[source]

Creates a parser suitable for the specified Fortran standard.

create(std=None)[source]

Creates a class hierarchy suitable for the specified Fortran standard. Also sets-up the list of classes that define scoping regions in the global SymbolTables object and clears any existing symbol table information.

Parameters:: std (str) – the Fortran standard. Choices are ‘f2003’ or ‘f2008’. ‘f2003’ is the default.
Returns:: a Program class (not object) for use with the Fortran reader
Return type:: fparser.two.Fortran2003.Program
Raises:: ValueError – if the supplied value for the std parameter is invalid

For example:

>>> from fparser.two.parser import ParserFactory
>>> f2003_parser = ParserFactory().create()
>>> f2003_parser = ParserFactory().create(std='f2003')
>>> f2008_parser = ParserFactory().create(std='f2008')
>>> # Assuming that a reader has already been created ...
>>> ast = f2008_parser(reader)
>>> print(ast)

Includes

Fortran supports the include statement in order to inline code from other locations into the current file.

The interpretation of where the include content comes from is defined as being processor dependent in the Fortran standard. Fparser assumes that the content comes from another another file. For example

program x
include 'content.inc'
end program

implies that content.inc is a file containing Fortran code. For example

print *, "Hello"

The above example would then be equivalent to the following Fortran code

program x
print *, "Hello"
end program

The Fortran standard specifies that the include statement is not a Fortran statement, rather it indicates that when the code is processed the contents of the include file should replace the include statement as the code is parsed.

fparser supports include statements in its reader classes (FortranStringReader and FortranFileReader), so the parser itself need not be aware of the original include statements.

The directory locations in which to search for include files can be specified to a reader class instance via the optional include_dirs argument, which should be iterable. For example:

reader = FortranFileReader(my_file, include_dirs=["dir1", "dir2"])

If a relative directory is specified then the location is with respect to the input file. The reader class instance will search in the specified directory order and include the contents of the first matching file found.

If no include_dirs argument is supplied then the default search location is ['.']. Therefore include files would need to be in the same directory as the input file for them to be found.

Note

At the moment it is not possible to specify include directories in the fparser2 script.

In a compiler, all include files must be found otherwise there is an error. However, with a code parser this is not necessarily the case. A user might not want to include all files when parsing, perhaps for simplicity, or perhaps because the files are not readily available. To support this, fparser has been extended to recognise include statements as part of the parse tree. If a particular include file is not found then the associated include statement will be parsed like a standard Fortran statement. If we again consider the simple example from earlier:

program x
include 'content.inc'
end program

with content.inc containing the following Fortran code:

print *, "Hello"

then if the content.inc file is found the output of fparser would be:

PROGRAM x
PRINT *, "Hello"
END PROGRAM x

but if the content.inc file is not found then the output of fparser would be:

PROGRAM x
INCLUDE 'content.inc'
END PROGRAM x

Clearly in the latter case the parser does require the code to be valid with the include statement in it. For example, assuming the content of endprogram.inc is end program then the following example would be parsed successfully if the endprogram.inc include file was found but would fail if the include file was not found:

program x
include 'endprogram.inc'

Preprocessing Directives

Preprocessing directives are language constructs that specify how a source file should be processed upon compilation or translation. They are most commonly used to conditionally include or exclude parts of a source file, include other source files in place, or define macros for expansion. As part of the compilation process these directives are interpreted to produce a preprocessed variant of the source file before translating it to the target language (e.g., assembly or machine code). Most compilers have a separate program (“preprocessor”) that is invoked by the compiler to carry out preprocessing.

While directives are not specified in any Fortran standard itself, Fortran compilers often support preprocessing of Fortran source files nonetheless. Consequently, the extent and limitations of this support depends on the compiler toolchain used.

fparser2 does not support preprocessing of source files but it allows to represent preprocessor directives as dedicated nodes in the parse tree.

Note

With all preprocessor directives removed the source code must reduce to valid Fortran. This is due to the fact that fparser2 only keeps directives but does not interpret them (thus essentially treats them like comments).

The support for preprocessing directives in fparser2 comprises that defined by the C99 standard. Left out are pragma directives as those are typically specified in the form of comments in Fortran. Preprocessing directives consist of the character # as the first character in a line (optionally after white space containing no new-line characters) followed by the actual directive and is ended by a new-line character. Line continuation is specified by a single backslash character \ at the end of the line.

The implementation of directives is in the C99Preprocessor.py file with support for the following:

#if ...
#ifdef ...
#ifndef ...
#elif ...
#else
#endif
#include ...
#define ...
#undef ...
#line ...
#error
#warning
#

Walking the Parse Tree

Properties and Methods of Nodes

The majority of the nodes in the parse tree produced by fparser2 are instances of (sub-classes of) the fparser.two.utils.Base class. This class provides the parent and children properties which return the immediate parent/children of a given node.

In addition to these properties, the Base class also provides the method:

Base.get_root()[source]

Gets the node at the root of the parse tree to which this node belongs.

Returns:: the node at the root of the parse tree.
Return type:: fparser.two.utils.Base

Note

The parse tree produced by fparser2 can contain some nodes that are not instances of fparser.two.utils.Base (e.g. None or str). Obviously such nodes do not have the parent and children properties.

Utilities

The utils module of fparser2 provides two utility functions to support the traversal of the parse tree that it constructs:

fparser.two.utils.walk(node_list, types=None, indent=0, debug=False)[source]

Walk down the parse tree produced by fparser2. Returns a list of all nodes with the specified type(s).

Parameters:

node_list ((list of) :py:class:fparser.two.utils.Base) – node or list of nodes from which to walk.
types (type or tuple of types) – type or tuple of types of Node to return. (Default is to return all nodes.)
indent (int) – extent to which to indent debug output.
debug (bool) – whether or not to write textual representation of AST to stdout.

Returns:

a list of nodes

Return type:

list of fparser.two.utils.Base

fparser.two.utils.get_child(node, node_type)[source]

Searches for the first, immediate child of the supplied node that is of the specified type.

Parameters:

node (fparser.two.utils.Base) – the node whose children will be searched.
node_type (type) – the class of child node to search for.

Returns:

the first child node of type node_type that is encountered or None.

Return type:

fparser.two.utils.Base