Difference between revisions of "Contrib/PyFoam"

From OpenFOAMWiki
(pyFoamAPoMaFoX.py: A Poor Man's FoamX)
Line 630: Line 630:
==== <tt>pyFoamAPoMaFoX.py</tt>: ''A Poor Man's FoamX'' ====
==== <tt>pyFoamAPoMaFoX.py</tt>: ''A Poor Man's FoamX'' ====
TUI (terminal/curses based) interface to the [[constrib_pyfoamcasebuilder|CaseBuilder]]-functionality
TUI (terminal/curses based) interface to the [[Contrib_pyFoamCaseBuilder|CaseBuilder]]-functionality
==== <tt>pyFoamAPoMaFoXiiQt.py</tt> ====
==== <tt>pyFoamAPoMaFoXiiQt.py</tt> ====

Revision as of 12:34, 22 May 2011

A python library to control OpenFOAM-runs and manipulate OpenFOAM-data. Comes with a number of utilities that should make your life easier if you're not scared by commandlines

Valid versions: OF version 141.png OF version 15.png OF version 16.png OF version 17.png works with older versions (but I don't support that)


1 Short description

This Python-library can be used to

  • analyze the logs produced by OpenFoam-solvers
  • execute OpenFoam-solvers and utilities and analyze their output simultaneously
  • manipulate the parameter files and the initial-conditions of a run in a non-destructive manner
  • plots the residuals of OpenFOAM solvers

1.1 Motivation

This library was developed to control OpenFOAM-simulations with a decent (sorry Perl) scripting language to do parameter-variations and results analysis.

It is an ongoing effort. I add features on an As-Needed basis but am open to suggestions.

1.2 Compatibility

This Library tries to be as conservative as possible in the libraries it uses. This means that it tries to use only libraries that are in the standard Python-distribution. Currently it needs at least version 2.3 of Python because of the threading-support but it has been tested on all versions up to 2.6 (Python 3 is currently not supported and will not be for the next time)

For plotting support (which is pretty rudimentary anyway) it needs the Gnuplot-library, but installs without.

It should run on any system with OpenFOAM and python2.3. Currently it has been tested on

  • Linux (various flavours)
  • MacOS X

1.3 Other material

This presentation "Happy Foaming with Python" from the 4th Workshop gives a short overview of the things that can be done with PyFoam.

There is another presentation on the "Automatization with pyFoam" from the 5th Workshop in Gothenburg avalaible.

2 Examples

These examples should demonstrate the possible applications of the library. They are enclosed in the source distribution.

2.1 Compact output

This example is a wrapper for solvers. For each time-step only one line is output: the time and the initial residuals of the linear solvers (with the name of the solved variable). The complete output of the solver is written to a log-file. The residuals are written to separate files in a special directory in the simulation-directory (for plotting).

import re,sys
from PyFoam.LogAnalysis.LogLineAnalyzer import LogLineAnalyzer
from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer
from PyFoam.Execution.AnalyzedRunner import AnalyzedRunner
class CompactLineAnalyzer(LogLineAnalyzer):
    def __init__(self):
        self.exp=re.compile("^(.+):  Solving for (.+), Initial residual = (.+), Final residual = (.+), No Iterations (.+)$")
    def doAnalysis(self,line):
        if m!=None:
            if time!=self.told:
                print "\n t = %6g : " % ( float(time) ),
            print " %5s: %6e " % (name,float(resid)),
class CompactAnalyzer(BoundingLogAnalyzer):
    def __init__(self):

2.2 Parameter Variation

This example does 10 runs of the same case. For each case a different velocity at the inlet is set. After the simulation has ended the mass flow (with this utility) and the pressure difference (with this utility) are calculated and written to the file InflowVariationResults. In addition to this the data of the last time-step is saved to a separate directory (for further analysis):

from PyFoam.Execution.ConvergenceRunner import ConvergenceRunner
from PyFoam.Execution.UtilityRunner import UtilityRunner
from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer
from PyFoam.RunDictionary.SolutionFile import SolutionFile
from PyFoam.RunDictionary.SolutionDirectory import SolutionDirectory
for i in range(nr+1):
    # Set the boundary condition at the inlet
    print "Inlet velocity:",val
    sol.replaceBoundary("inlet","(%f 0 0)" %(val))
    # Run the solver
    print "Last Time = ",dire.getLast()
    # Get the pressure difference (Using an external utility)
    pUtil.add("deltaP","Pressure at .* Difference .*\] (.+)")
    # Get the mass flow
    mUtil.add("mass","Flux at (.+?) .*\] (.+)",idNr=1)
    # Archive the results
    dire.lastToArchive("vel=%g" % (val))
    # Clear results
    # Output current stuff
    print "Vel: ",val,"DeltaP: ",deltaP,"Mass Flow:",massFlow
    f.writeLine( (val,deltaP,massFlow) )

2.3 Manipulating dictionaries

Between Version 1.3 and 1.4 the format for the specification of the linear solvers was changed. The following script takes a fvSolution-file in the old format and rewrites it in the equivalent new format (only works for some solvers, but should work for all the tutorials):

#! /usr/bin/python
"""Given a pre-OpenFOAM-1.4 fvSolution file, this script transforms the solvers
section into the equivalen 1.4-format
Incomplete, because it doesn't support (B)DCG and GaussSeidl but should work for
instance for all the tutorials"""
import sys
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
if len(sys.argv)<2:
    print "I need at least one fvSolution-file to work with"
for fName in sys.argv[1:]:
    print "Working on",fName
        for name,val in sol.iteritems():
            if type(name)!=str or type(val)!=tuple or len(val)<3:
                # this is not an old-school entry
            if solver=="ICCG":
                new=( "PCG" , { "preconditioner":"DIC" } )
            elif solver=="BICCG":
                new=( "PBiCG" , { "preconditioner":"DILU" } )
            elif solver=="AMG":
                new=( "GAMG" , { "agglomerator":"faceAreaPair",
                                 "smoother":"GaussSeidel"} )
                print "Unsupported solver",solver
    except IOError:
        print "File",fName,"does not exist"
    except KeyError:
        print "File",fName,"is not a fvSolution-file"
    except IOError:
        print "Can't write file. Content would have been:"
        print f

Of course 1.4 can read the old format, so this script is not needed. It's only an example.

2.4 Setting boundary conditions for walls

This example sets the boundary condition for all patches whose names contain the string "Wall" to zero-velocity:

from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
for b in f["boundaryField"]:
    if "Wall" in b:
        f["boundaryField"][b]["value"]="uniform (0 0 0)"

This can serve as a starting-point for automatically setting up cases where the patches are named after a specific scheme

3 Installation

The easies way to install PyFoam is as root, but it is also possible to do it as a regular user.

3.1 Installation as root

  1. Download the latest tar
  2. Untar it
  3. Go to the directory PyFoam-0.5.5
  4. Install it with the command
python setup.py install

Should step 4 fail one of the most common causes is that some Linux-distributions (usually Debian-based, don't know about Mandrake) don't install the packages that are necessary for setup.py. Usually they are in the Developer-package which is installed with

apt-get install python-dev

(on Debian) or similar.

3.2 Installation as a regular user

If you don't want to (or can't) install it as root things are a bit more complicated. Let's assume that you have directory /home/nonroot/my_python to which you have write access:

  • Download PyFoam-0.x.x (0.x.x being the current version)
  • Untar it
  • Go to the directory PyFoam-0.x.x
  • Install it with the command
python setup.py install --prefix=/home/nonroot/my_python
In /home/nonroot/my_python there should be two directories: bin and lib
  • Add the path to the library to the PYTHONPATH environment-variable
export PYTHONPATH=/home/nonroot/my_python/lib/python-2.3/site-packages:$PYTHONPATH

(the lib/python-2.3/site-packages portion of the path may depend on your Python-version; in that directory there must be a directory named PyFoam)

  • If you want to use the scripts distributed with PyFoam extend your path with
export PATH=/home/nonroot/my_python/bin:$PATH

3.3 Installation via RPMs

Starting with version 0.4.0 a source rpm and a regular rpm are provided for those who prefer to install it that way (The rpms have not been extensivly tested. If there is a problem: please contact me)

For recent versions no RPMs were made available. You can produced them from the tarballs with something like python setup.py bdist

3.4 Installation under gentoo Linux

PyFoam is now in the main portage tree, so you can install it with emerge.

emerge PyFoam

3.5 Installation under Debian/Ubuntu

Files to package PyFoam for the Debian packaging system are in the tar.

3.6 Testing the installation

To test whether the installation of PyFoam is working start the interactive Python shell with


(The shell can be left with Control-D). In that shell type

import PyFoam
import PyFoam.FoamInformation
print PyFoam.FoamInformation.foamTutorials()

If the last command produces an output similar to


everything is well.

If things don't work check in the Python-shell with the commands

import sys
print sys.path

whether the directory in which PyFoam resides is on the path searched by Python.

4 Installed Utilities

All the utilities described in this section are installed to /usr/bin (if installed as root) and should be in the standard path for execution. They can all be called with the --help-option which gives all the available options and the latest, up-to-date information about the utility.

One recommended usage of these utilities would be to start a simulation with foamJob and display the residuals with pyFoamPlotWatcher.py without interfering with the simulation.

The following descriptions are not necessarily up to date. The text generated with the --help-option is

4.1 Runner-Utilities

These are utilities that start and control OpenFOAM-Runs. Apart from the features described below they start a little XMLRPC-server through which these runs can be controlled (see Network-Utilities).

Most of the utilities can be used to start parallel runs (the number of processors and a host-file have to be provided, depending on the MPI-implementation. The parallel environment is set up automatically).

4.1.1 pyFoamRunner.py

Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information:

  • Residuals and other information of the linear solvers
  • Execution time
  • continuity information
  • bounding of variables

4.1.2 pyFoamUtilityRunner.py

Runs a OpenFoam Utility an analyzes the output. Needs a regular expression to look for. The next 3 arguments are the usual OpenFoam arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamUtility.logfile). The Directory PyFoamUtility.analyzed contains a file with the information of the regexp (the pattern groups) at every time-step.

Doesn't start an XMLRPC-Server

4.1.3 pyFoamSteadyRunner.py

Runs an OpenFoam steady solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments) Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information

  • Residuals and other information of the linear solvers
  • Execution time
  • continuity information
  • bounding of variables

If the solver has converged (all linear solvers below threshold) it is stopped and the last simulation state is written to disk

4.1.4 pyFoamPlotRunner.py

Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). Information that is output as graphs are

  • residuals of the linear solver
  • continuity
  • bounding of variables (if it occurs)
  • Courant-number (optional)
  • Execution time of a time-step (optional)
  • number of iterations of the linear solver (optional)
  • arbitrary regular expressions can used to look for solver-specific data
    • The groups in the regular expressions (patterns that are in brackets) are plotted
    • in the regular expressions %f% can be used as a shortcut for floating-point numbers
    • Any number of expressions can be specified and will be plotted in a separate window
  • For steady runs it behave like pyFoamSteadyRunner.py

4.1.5 pyFoamMeshUtilityRunner.py

Some utilities that manipulate meshes store the new mesh into the first timestep (so that they don't destroy the original mesh). This can be inconvenient because you have to copy the mesh back to constant for all further steps. This utility

  1. Runs a mesh-utility
  2. Moves the resulting mesh from the first time-step to the constant-directory

Be careful: This utility deletes the old mesh. If you don't know what you're doing, it will be gone forever

4.1.6 pyFoamPotentialRunner.py

Runs potentialFoam on a case and writes the results to p and U. Before running the solver it manipulates the dictionaries in system in such a way that potentialFoam runs and afterwards resets them to their old form.

4.1.7 pyFoamRunAtMultipleTimes.py

Runs a utility multiple times using different values for the -time option. Workaround for utilities that only allow a single time-step

4.2 Utilities for Logfiles

These utilities can be used to analyze the output of an OpenFOAM-solver that has been written to a file

4.2.1 pyFoamPlotWatcher.py

Gets the name of a logfile which is assumed to be the output of a OpenFOAM- solver. Parses the logfile for information about the convergence of the solver and generates gnuplot-graphs. Watches the file until interrupted.

4.2.2 pyFoamStandardLogAnalyzer.py

Analyzes a Log written by foamJob. Needs the name of the Logfile to be analyzed the data is being written to a directory that has the same name with _analyzed appended

4.3 Networking Utilities

Most Runner-Utilities start a Server-thread that makes it possible to contact control an OpenFOAM-run and get information about it. These utilities help to access these servers.


  • These utilities are beta-qualitiy. They work for my network but have not been tested in other environments
    • The address of the pyFoamMetaServer.py is hardcoded into the sources, but can be overwritten using system-wide configuration-files
  • pyFoamNetShell.py doesn't need any additional infrastructure. Only the address and the port have to be known

4.3.1 pyFoamNetList.py

This command lists all the OpenFOAM-simulations in your network environment and gives you information about them. Amongst the information is:

  • Hostname and port (useful for pyFoamNetShell.py)
  • Details about the run (solver, working directory ...)
  • Estimates about the time at which the run will end (using the time since the start, current timestep and the endTime)

This utilitiy only works if a pyFoamMetaServer.py can be contacted at the configured address pyFoamMetaServer.py

This program spawns a daemon process that runs forever. It stores information about the running processes. There are two mechanisms for it to get that information

  • scanning the network (at the start and on demand)
  • Runner-utilities try to register with it, when they start an OpenFOAM-run
    • The utilities also dregister their process when they exit gracefully

4.3.2 pyFoamNetShell.py

Connects to an OpenFOAM-run started by a runner-Utility using

  • the hostname of the machine it runs on
  • the port number (usually 18000, if more than one run is running one of the following ports)

and starts an interactive shell there. On the shells the XMLRPC-commands provided by the server can be used to

  • get information about the run
    • environment
    • start-times
  • manipulate the run
    • for instance stop() tells the run to write at the next time-step and then end
    • kill() brutally terminates the run

Help about the available commands is provided.

Can also be used to control the pyFoamMetaServer.py (which is usually contacted on port 17999)

4.4 Utilities for Manipulating case data

These utilities are used for manipulating case date. They are especially useful in scripts that automatically set up simulation runs.

4.4.1 pyFoamAddEmptyBoundary.py

Adds an empty patch to the boundary file. Such a patch is needed for some mesh-maipulation utilities.

4.4.2 pyFoamChangeBoundaryType.py

Changes the type of a boundary patch.

4.4.3 pyFoamChangeBoundaryName.py

Changes the name of a boundary patch.

4.4.4 pyFoamCreateBoundaryPatches.py

Creates boundary patches in a field-file by looking at the boundary-file

4.4.5 pyFoamClearCase.py

Removes all the timesteps except for the first from a case directory.

4.4.6 pyFoamPackCase.py

Packs the essential files (the ones that are needed to run it) of a case into a tar-file for archiving/mailing-purposes

4.4.7 pyFoamCloneCase.py

Creates a copy of a case with only the most essential files

4.4.8 pyFoamCopyLastToFirst.py

Copy last time-step from one case to the first one of another (making it the initial condition)

4.4.9 pyFoamClearInternalField.py

Clears the solution from the internal field

4.4.10 pyFoamClearBoundaryValue.py

Clear non-uniform values from the boundary-fields

4.4.11 pyFoamInitVCS.py

Initialize the case for the use with the Version Control System (VCS) of your choice

4.5 Manipulating dictionaries (from scripts)

For more complex cases these utilities require an understanding of the syntax/semantics of Python-lists and dictionaries

4.5.1 pyFoamReadDictionary.py

Reads data from the root of a OpenFOAM-dictionary and prints it. To access subexpressions of a dictionary entry Python-expressions can be used (note: the expression is evaluated by the Python-interpreter, so quoting has to be used for strings, otherwise Python confuses them with variables)

4.5.2 pyFoamWriteDictionary.py

Writes data to the root of a OpenFOAM-dictionary and writes it to disk. Subexpressions can be selected (see note: pyFoamReadDictionary.py)

4.5.3 pyFoamFromTemplate.py

Generate a dictionary from a template file

4.5.4 pyFoamCompareDictionary.py

Compares two dictionaries structurally (not textually). Useful when the order of the entries or the formating have changed and diff won't give useful results

4.5.5 pyFoamUpdateDictionary.py

Updates a dictionary from another by adding/removing entries that are not in both.

4.6 Paraview related utilities

Thes utilities require a paraview that is compiled with python-support. They have been only tested with ParaView 3.4 and may not work with other versions.

4.6.1 pyFoamPVSnapshot.py

Generates snapshots from OpenFOAM-data using a ParaView-state-file that was generated with another, similar case

4.6.2 pyFoamPVLoadState.py

Loads OpenFOAM-data and a Paraview-state-file that was generated with another similar case and lets you continue work like regular paraFoam

4.7 Other

Utilities that don't fit any of the other categories.

4.7.1 pyFoamListCases.py

Gives a ls-like overview of the OpenFOAM-cases in a directory

4.7.2 pyFoamDecompose.py

  • Generates a decomposeParDict according to the user specifications and
  • runs dcomposePar

This utility is quite useful when writing scripts where the number of available processors is not known beforehand (for instance if you are using a queueing-system)

4.7.3 pyFoamComparator.py

A utilitiy that automatically does a series of variations on a run (for benchmarking, testing and verification purposes). The behaviour is specified by an XML-file.

Work in progress. Therefor no example is present and the DTD that assists editing the XML-file is not included. Contact me, if you want to give it a spin (and want the DTD)

4.7.4 pyFoamVersion.py

Prints the Version of OpenFOAM and pyFoam. Used for information purposes (when reporting bugs). Also lists the Foam-versions installed in the OpenFOAM-directory

4.7.5 pyFoamExecute.py

Execute a command specifying a OpenFOAM-Version. Useful for compiling/executing a debug-version without changing the shell

4.7.6 pyFoamDumpConfiguration.py

Dumps the values that can be changed via configuration files. The location of these configuration values are (lower overrides higer):

  1. Hardcoded in the Library
  2. Systemwide configuration file: /etc/pyfoamrc
  3. User configuration file: ~/pyFoam/pyfoamrc

Parts of the output of the utilitiy can be pasted into the configuration files and modified to change the behaviour of PyFoam

4.7.7 pyFoamSamplePlot.py

Creates Gnuplot commands that plot the results of the sample-Utility (sorted by timestep, field etc) or the sample-function object

4.7.8 pyFoamSurfacePlot.py

Uses VTK to plot surfaces generated by the sampleSurface-Utility (sorted by timestep, field etc) or the sample-function object

4.7.9 pyFoamTimelinePlot.py

Creates Gnuplot commands to plot timelines produced by various functionObjects

4.7.10 pyFoamRedoPlot.py

Generates images by either connecting to a running server-process (one that is started for instance by pyFoamRunner.py or by reading a pickle-file (found in the .analyzed-directory)

4.7.11 pyFoamCaseReport.py

Prints small reports about a case. Currently implemnted:

  • Table of the boundary conditions
  • Boundary conditions on a per-patch basis (as opposed to the files where they are listed per-field)
  • Dimensions of the fields
  • Internal fields

4.7.12 pyFoamEchoDictionary.py

Just outputs a dictionary. Used to find bugs in the dictionary-parser

4.7.13 pyFoamCaseBuilder.py

Command line interface to the CaseBuilder-functionality

4.7.14 pyFoamJoinCSV.py

Joins different CSV-files and resamples them if needed

4.7.15 pyFoamConvertToCSV.py

Takes a file with column oriented data and converts it into a CSV

4.8 GUI-Tools

Stuff that is not purely command-line oriented

4.8.1 pyFoamDisplayBlockMesh.py

Graphically displays a blockMesh and helps highlights selected blocks and patches. It is not a blockMesh-editor, but it helps to find errors in the order of vertices etc (editing of the blockMesh is still done in a text editor).

This needs an installation of the VTK with Python-bindings. The ways to get this are very different on various systems. But usually you have to compile it from the sources. Sorry.

Since 0.5.4 this is implemented in PyQT4. If that library is not found it falls back to the old (less feature-rich) version implemented in Tkinter

4.8.2 pyFoamAPoMaFoX.py: A Poor Man's FoamX

TUI (terminal/curses based) interface to the CaseBuilder-functionality

4.8.3 pyFoamAPoMaFoXiiQt.py

Implementation of pyFoamAPoMaFoX.py using PyQT

4.9 Special utilities

Utilities for special applications

4.9.1 pyFoamModifyGGIBoundary.py

Modify GGI boundary condition parameters

5 Library Documentation

The tar-file contains the documentation of the Library as HTML (in the folder doc)

6 Rocks-Cluster support

There is library support to run OpenFOAM on a link Rocks cluster. It has only been tested on that type of cluster and probably has to be extended/adapted for other Clusters/Queuing-systems. There is a good chance that it might work on vanilla SunGridEngine installations, but this has not been tested.

An example cluster script would be

#$ -cwd
#$ -j y
#$ -S /opt/rocks/bin/python
#$ -m be
import sys,os
from os import path
from PyFoam.Infrastructure.ClusterJob import SolverJob
from PyFoam.Error import error
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
class Pitz(SolverJob):
    def __init__(self):
    def setup(self,parameters):
        vel["boundaryField"]["inlet"]["value"] = "uniform (%s 0 0)" % velIn

If this script (assume it has the name runPitzDaily.py) is submitted to a Rocks-Cluster with the SunGridEngine installed via

qsub runPitzDaily.py 5

(serial variant) or

qsub -pe mpi 4 runPitzDaily.py 5

the following things happen once the Job gets scheduled:

  1. it is ensured that version 1.5 of OpenFOAM gets used
  2. the template-case pitzDaily gets copied to a case pitzDaily.run.5 (automatically)
  3. blockMesh is run on that case (by the script)
  4. the inlet velocity is set to 5 (by the script)
  5. if the case was submitted in parallel then it is decomposed (using the Metis-algorithm) - automatically
  6. the case is run
  7. if it was run in parallel it is reconstructed (automatically)

Apart from the setup-method the SolverJob-class has a number of other hooks that allow to manipulate the case before/after decomposition/reconstruction

A script can be tested on the local machine using

pyFoamClusterTester.py runPitzDaily.py 5

or in parallel

pyFoamClusterTester.py --proc=4 runPitzDaily.py 5

The tester-utility does not need the SGE (SunGridEngine)

6.1 Preparation of a user account on the SGE

For the Cluster to be able to find the PyFoam-stuff a user has to create a file .sge_request with the contents

-v PYTHONPATH=/home/common/python

(assuming that the PyFoam-library resides in a directory /home/common/python/PyFoam that is reachable from all cluster nodes)

7 Technical notes/known problems

This is a list of the currently known problems. If you find any other bugs (also known as "surprising features") don't hesitate to contact the author

  • if a run-fails the error message might be "eaten" by the utility controlling it. I'm working on this
  • the only MPI-environment I currently have available is OpenMPI. The pyFoam-Libraries used to work with LAM (still should, but I can not test it). If you have problems, please contact me (also "success-stories": "it works with MPIch" are most welcome)

7.1 Bug reporting

Bugs can be reported on the Message board or on the Mantis Bug-Tracker at openfoam-extend.sourceforge.net. The later is preferred

8 Additional stuff

Since Version 0.2 the distribution contains the benchFoam.py-script (Since 0.2.4 the benchmark suite is stable enough to be used by the public). From version 0.3.0 on it can be used as pyFoamBench.py

9 Other topics

9.1 Plotting with customRegexp-files

The utilities that do the plotting of time-graphs (pyFoamPlotRunner.py and pyFoamPlotWatcher.py) automatically plot user defined quantities if they find a file customRegexp in the case directory. Since Version 0.4.3 this file may have a more complicated syntax. One line of the file may for instance look like this (connaisseurs will recognize a Python-dictionary):

{"expr":"Reaction Speed: (%f%) min (%f%) max","name":"Reaction speed","titles":["min","max"]}

If the output of the solver has lines that look like this

Reaction Speed: -0.43 min 0.23 max

a plot with the name "Reaction speed" and two data sets ("min" and "max") will be created.

The fields in the dictionary are:

This is a regular expression that is looked for in the output of the solver. The string %t% is shorthand for a more complicated expression that fits any real number. Any Regexp-Groups (the stuff between "()") is used as data for plotting. Please note that regular expression need some getting used to. One extra character can completely change the meaning of an expression and cause the expression to fail
The title of the plot
The "legend" of the data lines found during fitting. These names are used in the order in which they are found in the Regexp

9.1.1 New format

Starting with version 0.5.3 a new format for the customRegexp is introduced. It basically looks like a OF-dictionary where each plot is specified by a separate sub-dictionary. The above custom plot can be written as

  expr "Reaction Speed: (%f%) min (%f%) max";
  name Custom01_Reaction_speed;
  theTitle "Custom 1 - Reaction speed";
  type regular;

customRegexp-files in the old format are still recognized and can be dumped in the new format using the --dump-custom-regegexp-option

It is possible to let the titles be determined by the regular expression (idNr identifies the sub-expression that is the name):

  expr "Value of species (.+) is (%f%)";
  theTitle "Don't know how many species we have";
  type dynamic;
  idNr 1;
  with steps;

Instead of plotting data can be appended to a different plot (and will be plotted there):

  expr "Average reaction speed (%f%)";
  titles ( avg );
  type slave;
  master ReactionSpeed;

9.2 Settings

Several aspects of pyFoam can be modified using settings-files

The current settings can be viewed with the pyFoamDumpConfiguration.py-command. They are output in the format that is needed in pyfoamrc (so you can paste them in there and modify them) (BTW: the format is basically similar to the INI-files in Windows ca. 3.XX - in the days before the registry. It's primitive, but it's sufficient for our purposes)

The precedence of settings is:

  1. Hardcoded in the PyFoam-sources
  2. Systemwide in /etc/pyfoamrc
  3. User-specific in ~/.pyFoam/pyfoamrc

Highest number wins

Settings can be overwritten for specific OF versions by adding new sections that are of the format MPI-1.7 for instance (entries in that section would overwrite the settings in MPI if the used version of OpenFOAM starts with 1.7 (1.7.x, 1.7 etc)

9.2.1 Modifying the call to mpirun

Different MPI-implementations and sites may need different options to mpirun.

The section of variables that is relevant to this is

options_openmpi_post: ["-x","LD_LIBRARY_PATH","-x","WM_PROJECT_DIR","-x","FOAM_MPI_LIBBIN","-x","MPI_B UFFER_SIZE"]
options_openmpi_pre: ["--mca","pls_rsh_agent","rsh"]

The _pre-option is the command-line arguments that are added to the mpirun command-line BEFRORE the -np/-machinefie argument the _post-option is for stuff that comes afterward (but before the actual OpenFOAM-command)

The middle-part of the variable name depends on the value of the $WM_MPILIB-variable. So for LAM this would be options_lam_pre/post.

9.3 Version control support

PyFoam adds some facilities to work with cases under version control. The advantage of this is that changes to a case are recorded when experimenting with a case (when using a Distributed VCS this also has some interesting applications for cloning/branching/merging cases). The disadvantage is that huge amounts of disc space are wasted if this is used unwisely.

Currently only support for mercurial is implemented (it was the natural choice as it is mainly implemented in Python). The implementation is generic enough to support other VCS but some low-level support has to be added

The case is initialized for VCS with the pyFoamInitVCS.py-command. This initializes the VCS-repository and adds the most important stuff in the case (the directories 0, constant and system. After that further files can be added using the usual commands of the used VCS. Options of the runner-utilities allow "commit before running"

The contrib-folder of the distribution contains a Mercurial-extension foamdiff. When this extension is installed a command like

hg foamdiff -r 23 system/controlDict

compares different revisions of a file not for textual differences but for semantic differences (using the same mechanisms that pyFoamCompareDictionaries.py uses)

10 paraFoam/paraview 'support'

If you have an installation of paraview that is 'Python-enabled' and PyFoam is found on the PYTHON_PATH then PyFoam can be used in the 'Programmable Filter'/'Programmable Source' of paraview by importing it as usual

import PyFoam

This is especially useful for data that was written by OpenFOAM but is not recognized by the regular importer. It can be loaded as usual with the ParsedParameterFile-class

Please note:

  • To visualize the data you need to know how to program VTK
  • Be aware, that due to the very general parser of PyFoam large datasets might consume huge amounts of memory and time during loading

10.1 Displaying the gravitation direction

10.1.1 As a Programmable Source

This little example displays the gravitation vector as a line. Just use it as a Programmable Source. It assumes that paraFoam was called in the case directory (otherwise casepath has to be modified)

from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
from PyFoam.Paraview import caseDirectory
from os import path
import paraview
pdo.InsertNextCell(line.GetCellType(), line.GetPointIds())

Using the timeDirectory()-function you can program similar Programmable Filters for data that is stored in the time directories

10.1.2 As a script

An alternative is running this script via "Tools -> Python Shell":

from PyFoam.Paraview import readerObject,caseDirectory
from PyFoam.Paraview.SimpleSources import Glyph,Arrow
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
from os import path
# gly=Glyph("Gravity",ro.getCenter(),ro.getCenter()+0.5*g*abs(ro.getExtent())/abs(g))

10.2 Other example scripts

Can be found in the examples/paraview3-directory of the release. These are:

Displays the most important feature of a snappyHexMeshDict including the bounding-box and the STL-files
displays the probes specified in system/controlDict

These Utilities are quite useful to find mistakes when setting up these files

10.3 General Remarks on the Programming

PyFoam takes care of some of the things described in this document when creating a new object. The two aspects of the object are accessible as

the actual VTK-object
its paraview representation

You can access their properties as described in the document

There are still some issues with the deletion of these objects

  • sometimes they are still visible but not deleted
  • sometimes paraview crashes after deleting them using the "Delete"-button on the GUI

but your milage may vary

All this has been tested with ParaView 3.4 and may not work with other versions as the Python-API of paraview seems to be in flux

10.4 Necessary changes in the OF-installation to use the paraview-stuff

The etc/apps/paraview3/bashrc must be extended to extend the Python-PATH:

if [ "$PYTHONPATH" ]; then
    export PYTHONPATH=$PYTHONPATH:$ParaView_DIR/Utilities/VTKPythonWrapping
    export PYTHONPATH=$ParaView_DIR/Utilities/VTKPythonWrapping
export PYTHONPATH=$ParaView_DIR/lib/paraview-3.3/:$PYTHONPATH
# export PYTHONPATH=$ParaView_DIR/lib/paraview-3.4/:$PYTHONPATH

For the Version 3.3-cvs that is distributed with 1.5 a little bug has to be fixed: look for the file ~/OpenFOAM/ThirdParty/ParaView3.3-cvs/platforms/linuxGcc/Utilities/VTKPythonWrapping/paraview/servermanager.py and there change the line 1110 from




11 Other OpenFOAM-related Python libraries

PyFoam can't do everything (and it doesn't even try) but there are other efforts that help to use OpenFOAM programmed in Python

11.1 pythonFlu

pythonFlu is a wrapper to OpenFOAM that allows the programming of top-level solvers in Python (and I'm sure there will be other great applications that can't be easily done otherwise). More information can be found on the official pythonFlu website.


12.1 Subversion

The current version can also be fetched using the command

svn co https://openfoam-extend.svn.sourceforge.net/svnroot/openfoam-extend/trunk/Breeder/other/scripting/PyFoam/

This is the current stable version plus

* Bugfixes
* User contributions

not the development version. User contributions will be considered for inclusion in the next stable version

12.2 Current version (0.5.5)

12.3 Previous version (0.5.4)

12.4 Previous version (0.5.3)

12.5 Previous version (0.5.2)

12.6 Previous version (0.5.1)

12.7 Previous version (0.5.0)

12.8 Previous version (0.4.3)

12.8.1 Buggy version (0.4.2)

From this version the Gnuplot-package was missing. Forget it

12.9 Ancient history

12.10 Previous version (

12.11 Previous version (0.4.1)

12.11.1 Previous version (0.4.0)

There were no complaints about this version

12.11.2 Very old versions

13 History

  • 2005-08-15: Version 0.1
  • 2006-01-13: Version 0.2
  • 2006-01-16: Version 0.2.1
  • 2006-01-16: Version 0.2.2
  • 2006-01-23: Version 0.2.3
  • 2006-01-23: Version 0.2.4
  • 2006-02-03: Version 0.2.5
    • Improves the stability of the benchmark-script for parallel runs
    • adds benchmakr suite for ancient versions of OpenFOAM
  • 2006-02-03: Version 0.2.6
    • Minor changes to the benchmark-scripts (other cases than the tutorial cases can now be used for benching)
  • 2006-02-23: Version 0.3.0
    • live plotting of residuals of solvers
    • benchmark script moved to bin-directory
  • 2006-03-22: Version 0.3.1
    • BugFix: Plotting residuals in interFoam did not work
    • Extending the Resiudals-Plotting (Courant-Numbers etc)
    • Parsing of Dictionary-files into Python-Dictionaries (not yet fully tested)
  • 2006-03-22: Version 0.3.2
    • Plotting resiudals now works with OpenFOAM 1.3
  • 2006-10-18: Version 0.3.3
    • pyFoam distinguishes between the used MPI-implementations when starting parallel runs (currently LAM and OpenMPI)
    • BugFixes
    • added Utility pyFoamAddEmptyBoundary
  • 2006-11-29: Version 0.3.4
    • An error pointed out by Andrea Rimondi: the last version was incomplete. Due to a code reorganisation and missing files in the distribution half of the scripts in bin stopped working, but apart from Andrea everyone was too polite to mention it
    • added Utility pyFoamClearCase.py
  • 2007-03-29: Version 0.4.0
    • Reimplementation of the parsing of OpenFOAM-Dictionaries
      • Can be manipulated like regular Python-Data-Structures
    • Extension of the plotting-Utilities: arbitrary expressions are possible
      • The data can also be written to the disk
    • Runs that are started by pyFoam can be accessed and controlled over the network (using XMLRPC)
    • The runner-Utilities can start parallel-runs
      • A utility for generating a dictionary and decomposing a case is added
    • Bug removed that didn't let pyFoamSteadyRunner.py work properly with OpenFOAM 1.3 (due to changes in the solver output)
  • 2007-08-29: Version 0.4.1
    • Debugging and stabilizing of the parsed OpenFOAM-Dictionaries
      • Changes in semantics: numbers are converted to numbers (not strings as before)
      • Can read and write gnuzipped files
      • More elegant access of the data
      • Order of entries in dictionaries is preserved, if possible
    • Gnuplot-library enclosed in distribution (to ease installation)
    • Enhancement of utilities
      • Runner-utilities can choose the Foam-version that should be used
      • Plot-Runner automaticaly loads regular expressions for a case
      • pyFoamBench now writes CSV-files which can be loaded into a spreadsheet-program
      • other stuff
    • New utilities
      • Creation of boundary conditions
      • Cloning and packing cases
      • VTK-based utility to display blockMeshes
      • Specialized runner for mesh utilities
    • Implementation of Unit-Tests for base classes to ensure stability
    • Other changes I can't remember
  • 2007-11-24: Version 0.4.2
    • A framework to run jobs on a Rocks-cluster was implemented
    • Support for cases that have more than one mesh-region (in the library and the utilities)
    • Changes in the Library
      • the Application-classes have been changed, so that they can be called from other scripts
      • the OpenFOAM-version can be changed for the called applications
      • Extension of the Parser (Tensors etc)
      • Updated Parser Ply to newest version 2.3
      • A symbolic link controlDict.foam is automaticaly created (for the Paraview3.x-reader)
      • Gnuplot.py-library is adapted to use either numpy or Numeric (whatever is available)
      • mpirun can now have configurable parameters
    • Changes in Utilities
      • the Runner-Utilities now have a --restart-option
      • All Utilities that start an OpenFOAM-application now can select the OF-Version with a --foam-version-option
      • Possibility to temporarily lower the relaxation factors for steady runs
      • Runner-Utilities can report memory usage
      • Clone-Case is now securer
    • New Utilities
      • pyFoamDumpConfiguration.py: Dump the current configuration file for the PyFoam-installation
      • pyFoamExecute.py: Execute a command specifying a OpenFOAM-Version
      • pyFoamFromTemplate.py: Generate a dictionary from a template file
      • pyFoamUpdateDictionary.py: Updates a dictionary from another by adding/removing entries that are not in both
      • pyFoamCompareDictionary.py: Compares two dictionieries structurally (not textually)
      • pyFoamClusterTester.py: Test pyFoam-Scripts before submitting them to the cluster
    • Semantic changes
      • Counting of linear iterations now accumulates over various solutions during the same timestep (different numbers in Plots!!)
    • Removed Bugs: many
  • 2007-12-04: Uploaded a bugfix version (Missing Gnuplot-library)
  • 2008-04-21: Version 0.4.3
    • Changes in the Library
      • Faster parsing of long lists
      • Parsing works with unicode
      • Better handling of solution-directories
    • Changes in Utilities
      • Logfiles are now created using the name of the application
      • Various extensions of the Comparator-Utility (CSV-files etc)
      • Advanced syntax for the customRegexp-files
    • New Utilities
      • pyFoamCopyLastToFirst.py: Copy last timestep to another case (as initial condition)
      • pyFoamPotentialRunner.py: Runs potentialFoam on a case (leaving everything else unchanged)
      • pyFoamSamplePlot.py: Generates graphs from data generated by the sampe-Utility
      • pyFoamClearInternalField.py: Resets the whole internal field to one value
      • pyFoamCaseReport.py: Information about a case (table of boundary conditions)
    • includes new Version 1.8 of the Gnuplot-Library
    • Removed Bugs: all that were reported (and some more)
  • 2008-09-09: Version 0.5.0
    • This version detects whether a OF-1.5 or previous is used and acts according to the convention for utilities in that version
      • the --foamVersion-option of the utilities is also aware of switches that change the convention
    • Plot utilities do Postscript-Hardcopies if requested
    • CaseReporter now also reports data about the decomposition
    • Other extension of the CaseReporter
    • Added an option to the runner-utils that temporarily disabel libs and functions in the cointrolDict (for incompatible utilities)
    • Parser extended to handle special dictionary files (files without headers. Chemical reactions)
    • added pyFoamEchoDictionary.py
    • Various bugfixes (especially in the parser)
  • 2008-10-30: Version 0.5.1
    • Moved common functionality of the application to common classes
    • Options get grouped in the online-help
    • Parser is aware of include and parameter substitution
    • Profiling of applications enabled
    • Example for a Paraview-Source added
    • Error with 1.5 and MultiRegion-cases fixed
    • Parser fixed so that it can work with patch-names that are generated by snappyHexMesh
    • CaseBuiilder-functionality added
      • Small TUI added as a demo
    • Parser is now Version 2.5 of PLY (20% speed improvement)
    • Bug in the potentialRunner-application fixed
  • 2009-03-12: Version 0.5.2
    • Uses color on the terminal to format the output of certain utilities (pyFoamCompareDictionary.py etc)
      • Can be configured
    • User can be selected for pyFoamNetList
    • Log-File can be surpressed
    • The XMLRPC-server is more tolerant if the port is already in use
    • Removed incompatibility with python 2.6 (clashes with the with-keyword)
      • 2.6 now complains about deprecated Libraries
    • CaseBuilder allows the grouping of arguments
    • Extended support for Paraview 3.4
      • Easy creation of graphics primitives (Spheres, Glyphs etc)
      • Automatically finds the case-directory of an OpenFOAM-case loaded in paraFoam
    • Allows to chose whether symlinks will be followed when cloning a case
    • Added pyFoamPVSnapshot.py and pyFoamPVLoadState.py
    • Vector-class for vector-objects read during case-parsing (allows vector-arithmetics)
    • Repaired pyFoamPackCase.py
    • added pyFoamClearBoundaryValue.py
    • added pyFoamListCases.py
    • switch of the usage of the XMLRPC-server for certain utilities
    • extension of pyFoamClearCase.py
    • Enhancements of the dictionary parser
      • tries to preserve comments when writing the results to disk
      • is now based on PLY 3.1
    • the plot-utilities can accumulate in different ways if a match is found multiple times per timestep
      • Display of data for cases with >>2000 time-steps is improved (better downsampling)
    • pyFoamWatcher.py now can make hardcopies of the graphs
      • format of the hard-copies can be choosen
  • 2009-11-10: Version 0.5.3
    • General new stuff
      • Support for the regular-expressions that come with 1.6
      • Matplotlib can be used as an additional backend for plotting (this is beta-quality)
      • improved format for the customRegexp that looks more 'FOAMlike'. Old files can be dumped in the new format using the --dump-option of the PlotWatcher
      • two new plot types in the customRegexp ('dynamic' for getting the data names from the regular expression and 'slave' to add data to another plot instead of plotting itself)
    • Additional utilities
      • GGI-script by Martin Beaudoin
      • pyFoamRunAtMultipleTimes.py after an idea by waterboy (MessageBoard-name)
    • Bugfixes
      • Amount of files opened by FileCollection is limited (won't hit the OS-limit on the number of open files
      • Fixed a hang at the end of a run that occured if no meta-server was present
      • FoamServer reports actual IP (not
    • Changes
      • Additional parameter to the Decomposer-script (Martin Beaudoin)
      • Decomposer-script now recognizes scotch as a valid decompositon method and uses it as a default when used with OF-1.6
      • Plotter-Utilities write data to file
      • Added general options --force-debug --force-opt --force-32 --force-64 to switch to a specific subversion of OF
      • Changes that concern the network-infrastructure
      • Mesh can be excluded for the Pack-script
      • more stable method to handle temporary changes to the controlDict
      • for-loops over a PArsedParameter-file are possible
      • unit-tests work with the new tutorial-structure in 1.6
      • upgraded parser to PLY 3.2
      • Created files can be forced to gz
      • Various upgrades to the ParsedParameterFile-family
      • Utilities write a history to the case-directory
      • further adaptions for Python2.6
      • Watcher can read compressed logfiles
      • Written logs can be compressed on-the-fly
      • Clone works with cases that don't have a time-step
      • zsh is recognized (for switching OF-versions)
    • Known limitations
      • Currently does not work with Paraview 3.6 (which comes with OF 1.6)
  • 2010-06-11: Version 0.5.4
    • Utilities
      • DisplayBlockmesh now converted to PyQT4.
        • Major enhancements (built-in editor)
        • If no PyQT4 found the old Tkinter-version will be used
      • PlotWatcher can be stopped on a file that is no longer updated (Mantis Bug #8)
      • Decomposer can treat regions separatly
      • Decomposer can be called directly in the case-directory
      • Runners can now switch on purgeWrite
      • RedoPlot adds titles
      • PotentialRunner works with 1.6
      • Enhancments to the CaseReporter
        • Writes to files
        • Report on linear solvers and relaxation
      • New utility: pyFoamSurfacePlot.py to generate images from suface-samples
      • New utility: pyFoamTimelinePlot.py that plots timelines that were generated by functionObjects. Also can plot Bar-plots
      • New utility: pyFoamRedoPlot.py to plot timelines from running processes (requires a server-process - for instance one started by pyFoamRunner) and pickled data from finished runs
      • ClearCase tries to remove data from functionObjects, too
      • SamplePlot can now write the Gnuplot-commands to a file
    • Library
      • General application-class for PyQT4-classes
      • TimeLineAnalyzer is more tolerant
      • TemplateFile now shows the expression that gives a problem
      • numpy now preferred to Numeric
      • Pickle-file with the plot-data is written
      • Race condition with the pickle-file avoided
      • Configuration can now overwrite parameters depending on the OF-version
      • Configuration now allows directories with configuration snipplets (profile.d-style)
      • local configuration in a case-directory possible
      • Regular expression for "Time =" can now be configured
      • Paraview-stuff now accepts the native reader
      • SolutionDirectory automatically creates a .foam-stub-file for the native reader
      • FoamInformation more tolerant if no OF-installation is found
    • Packaging
      • Basic framework for debian-packaging is there (only in SVN)

... and a number of other enhancements

  • 2011-04-06 Version 0.5.5
    • Utilities
      • CaseReporter now generates restructured text (this allows easy generation of PDFs and HTML using the right tools)
      • CaseReporter can now work with regions
      • Paraview-Utilities now know th native reader in Paraview 3.8
      • PyFoamChangeBoundaryName by Martin Beaudoin added
      • CloneCase now knows how to handle parallel runs
      • SamplePlot and TimelinePlot now can plot reference data
      • In the Runners the writing of data files now MUST be switched on
      • ListCases reports the state-files written by BasicRunner (if present)
      • ListCases has additional reporting options
      • CompareDictionary can now compare more than one dictionary at once
      • Option --autosense for the Runner-utilities now automatically determines the number of processors to use
      • SamplePlot, TimelinePlot and RedoPlot now can write CSV-Files
      • PyQT-variant of 'A Poor Mans FoamX' added to the distro
      • Decomposer now can handle parMetis
      • Utility pyFoamInitVCS.py added
      • --current uses the current OF-version (when switching to Debug) in all utilties that support version-switching
      • CloneCase now uses the VCS for cloning if the case is under version control
      • TimelinePlot now allows the usage of the time of the reference data for scaling
      • TimelinePlot and SamplePlot now can compare the data to the reference data quantitatively
      • Utility JoinCSV to join different CSV-files (resamples data if needed)
      • Runner-Utilites now can commit if the case is under VCS
      • Utility ConvertToCSV added
      • RunAtMultipleTimes now also works for parallel cases
    • customRegexp
      • New entry progress now allows addition of custom values to the progress output of runners (for instance "h: $0" for addition of the first value found)
      • Plots now can be selected from the command line
      • All Utilities have an option to switch on/off the stack-trace in the event of an error
    • Library
      • Gnuplot-Library bumped to 1.8
      • PLY updated to 3.3
      • Improved handling of Macros in the Dictionary Parser
      • Paraview-Sublibrary adapted to Paraview 3.8
      • Unittests now can be done using the nose-library
      • BasicRunner now writes some state-files
      • New SpreadsheetData-class generalizes the handling of data that will go into CSV-files
      • Enhancements to the CaseBuilder-backend
      • error-function now throws an Exception (this allows graceful error-handling)
      • Parser now knows that he can't handle binary files
    • Bugfixes
      • WriteDictionary now can write negative values
      • BlockMeshes without edges not correctly parsed
      • If more than one triggered restore was accessing the file it was not correctly restored
    • Other enhancements
      • In the configuration files features can be switched depening on the version. For instance the section MPI-1.7 overrides everything specified in MPI if the OpenFOAM-version starts with 1.7
      • File LocalConfigPyFoam in a case can be used to override the global settings for this case
      • Automatic reconstruction can be switched off for ClusterJobs
      • ClusterJobs can now also live with alread reconstructed Jobs
      • Addition of a Mercurial-extension foamdiff that can show changes between revisions like CompareDictionary does
      • Installed OpenFOAM-versions now can be looked for in more than one directory

--Bgschaid 21:50, 15 Aug 2005 (CEST)