Contrib/PyFoam

A python library to control OpenFOAM-runs and manipulate OpenFOAM-data. Comes with a number of utilities that should make your life easier if you're not scared by commandlines

Valid versions: works with older versions (but I don't support that)

1 Short description
2 Examples
3 Installation
4 Installed Utilities
5 Library Documentation
6 Rocks-Cluster support
- 6.1 Preparation of a user account on the SGE
7 Technical notes/known problems
- 7.1 Bug reporting
8 Additional stuff
9 Other topics
10 paraFoam/paraview 'support'
11 Other OpenFOAM-related Python libraries
- 11.1 pythonFlu
12 Downloads
13 History

1 Short description

This Python-library can be used to

analyze the logs produced by OpenFoam-solvers
execute OpenFoam-solvers and utilities and analyze their output simultaneously
manipulate the parameter files and the initial-conditions of a run in a non-destructive manner
plots the residuals of OpenFOAM solvers

1.1 Motivation

This library was developed to control OpenFOAM-simulations with a decent (sorry Perl) scripting language to do parameter-variations and results analysis.

It is an ongoing effort. I add features on an As-Needed basis but am open to suggestions.

1.2 Compatibility

This Library tries to be as conservative as possible in the libraries it uses. This means that it tries to use only libraries that are in the standard Python-distribution. Currently it needs at least version 2.5 of Python but it has been tested on all versions up to 2.7. Python 3 is currently supported (but currently only 3.4 is tested)

For plotting support it needs the Numpy-library. It is recommended to use the version that is supplied by the package manager of the distribution (PyFoam does not require a specific version).

It should run on any system with OpenFOAM and python2.5 (but some features require at least 2.6). Currently it has been tested on

Linux (various flavours)
MacOS X

1.3 Other material

This presentation "Happy Foaming with Python" from the 4th Workshop gives a short overview of the things that can be done with PyFoam.

There is another presentation on the "Automatization with pyFoam" from the 5th Workshop in Gothenburg avalaible (the material is available at this link).

The presentation PyFoam - the dark, unknown corners about the tools for quantitative analysis, handling cases with version control and the server mechanism held at the PFAU (Austrian User Meeting) at the TU Wien in March 2012

Complementary there is a complete tutorial on "Particles with pyFoam - The air classification test case" available here.

A presentation about automatic solver testing using PyFoam, a feature that was added in PyFoam 0.5.7, and a file with the data described there (rename file appropriately before using)

The presentation held at the 8th OpenFOAM Workshop gives a nice introduction to pyFoam from a user perspective (the first part. The second is a bit more advanced). The material mentioned is available here

The presentation Building Block for Automatization (a HTML5-browser is required) from the OpenSource CFD International Conference 2013 gives an overview of newer features of 0.6.2.

The Presentation Before and after the case held at the Austrian User Group Metting PFAU 8.0 demonstrates the use of a new utility to set up cases and interaction with IPython-notebooks. The files to reproduce the presentation are available here.

There was a a basic training about swak4foam and PyFoam at the 9th OpenFOAM Workshop which gives an overview over the work-flow. Updated version of this training from the 10th OpenFOAM Workshop which was updated once more at the 11th OpenFOAM Workshop.

At the 10th OpenFOAM Workshop a comprehensive about the pyFoamPrepareCase.py utility was given. An updated version (without cat pictures) was given at the 13th OpenFOAM Workshop

At the 11th OpenFOAM Workshop the presentation PyFoam 4 the lazy gives an overview of some utilities that allow processing simulation results (case file for this presentation is found here)

At the PFAU 14 (Austrian User Meeting) the talk Talking to PyFoam (and it talks back) demonstrated two features that have been in PyFoam for some time: the server-process that allows controlling a PyFoam-controlled OpenFOAM-run over the net and hooks: little Python-programs that are executed at the start and the end of an execution

A training about using the PyFoam library to program automatic workflows was File:BernhardGschaider-OFW15 ProgrammingPyFoamTraining.pdf

Introductory training for swak4Foam and PyFoam at the 12th International OpenFOAM Workshop. This is a completely new training. An updated version of the Training held at the 2019 Workshop in Duisburg. Most recent version File:BernhardGschaider-OFW15 swakPyFoamBasicTraining.pdf. The hopefully last version File:BernhardGschaider-OFW17 swakPyFoamBasicTraining.pdf with pointers to other material

2 Examples

These examples should demonstrate the possible applications of the library. They are enclosed in the source distribution.

2.1 Compact output

This example is a wrapper for solvers. For each time-step only one line is output: the time and the initial residuals of the linear solvers (with the name of the solved variable). The complete output of the solver is written to a log-file. The residuals are written to separate files in a special directory in the simulation-directory (for plotting).

 
import re,sys
 
from PyFoam.LogAnalysis.LogLineAnalyzer import LogLineAnalyzer
from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer
from PyFoam.Execution.AnalyzedRunner import AnalyzedRunner
 
class CompactLineAnalyzer(LogLineAnalyzer):
    def __init__(self):
        LogLineAnalyzer.__init__(self)
 
        self.told=""
        self.exp=re.compile("^(.+):  Solving for (.+), Initial residual = (.+), Final residual = (.+), No Iterations (.+)$")
 
    def doAnalysis(self,line):
        m=self.exp.match(line)
        if m!=None:
            name=m.group(2)
            resid=m.group(3)
            time=self.getTime()
            if time!=self.told:
                self.told=time
                print "\n t = %6g : " % ( float(time) ),
            print " %5s: %6e " % (name,float(resid)),
            sys.stdout.flush()
 
class CompactAnalyzer(BoundingLogAnalyzer):
    def __init__(self):
        BoundingLogAnalyzer.__init__(self)
        self.addAnalyzer("Compact",CompactLineAnalyzer())
 
run=AnalyzedRunner(CompactAnalyzer(),silent=True)
run.start()

2.2 Parameter Variation

This example does 10 runs of the same case. For each case a different velocity at the inlet is set. After the simulation has ended the mass flow (with this utility) and the pressure difference (with this utility) are calculated and written to the file InflowVariationResults. In addition to this the data of the last time-step is saved to a separate directory (for further analysis):

 
from PyFoam.Execution.ConvergenceRunner import ConvergenceRunner
from PyFoam.Execution.UtilityRunner import UtilityRunner
from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer
from PyFoam.RunDictionary.SolutionFile import SolutionFile
from PyFoam.RunDictionary.SolutionDirectory import SolutionDirectory
 
solver="simpleFoam"
case="pitzDaily"
pCmd="calcPressureDifference"
mCmd="calcMassFlow"
 
dire=SolutionDirectory(case,archive="InletVariation")
dire.clearResults()
dire.addBackup("PyFoamSolve.logfile")
dire.addBackup("PyFoamSolve.analyzed")
dire.addBackup("Pressure.analyzed")
dire.addBackup("MassFlow.analyzed")
 
sol=SolutionFile(dire.initialDir(),"U")
 
maximum=1.
nr=10
 
f=dire.makeFile("InflowVariationResults")
 
for i in range(nr+1):
    # Set the boundary condition at the inlet
    val=(maximum*i)/nr
    print "Inlet velocity:",val
    sol.replaceBoundary("inlet","(%f 0 0)" %(val))
 
    # Run the solver
    run=ConvergenceRunner(BoundingLogAnalyzer(),argv=[solver,".",case],silent=True)
    run.start()
    
    print "Last Time = ",dire.getLast()
 
    # Get the pressure difference (Using an external utility)
    pUtil=UtilityRunner(argv=[pCmd,".",case],silent=True,logname="Pressure")
    pUtil.add("deltaP","Pressure at .* Difference .*\] (.+)")
    pUtil.start()
 
    deltaP=pUtil.get("deltaP")[0]
 
    # Get the mass flow
    mUtil=UtilityRunner(argv=[mCmd,".",case,"-latestTime"],silent=True,logname="MassFlow")
    mUtil.add("mass","Flux at (.+?) .*\] (.+)",idNr=1)
    mUtil.start()
 
    massFlow=mUtil.get("mass",ID="outlet")[0]
 
    # Archive the results
    dire.lastToArchive("vel=%g" % (val))
 
    # Clear results
    dire.clearResults()
 
    # Output current stuff
    print "Vel: ",val,"DeltaP: ",deltaP,"Mass Flow:",massFlow
    f.writeLine( (val,deltaP,massFlow) )
    
sol.purgeFile()

2.3 Manipulating dictionaries

Between Version 1.3 and 1.4 the format for the specification of the linear solvers was changed. The following script takes a fvSolution-file in the old format and rewrites it in the equivalent new format (only works for some solvers, but should work for all the tutorials):

 
#! /usr/bin/python
 
"""Given a pre-OpenFOAM-1.4 fvSolution file, this script transforms the solvers
section into the equivalen 1.4-format
Incomplete, because it doesn't support (B)DCG and GaussSeidl but should work for
instance for all the tutorials"""
 
import sys
 
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
 
if len(sys.argv)<2:
    print "I need at least one fvSolution-file to work with"
    sys.exit(1)
    
for fName in sys.argv[1:]:
    print "Working on",fName
    try:
        f=ParsedParameterFile(fName)
        sol=f["solvers"]
        for name,val in sol.iteritems():
            if type(name)!=str or type(val)!=tuple or len(val)<3:
                # this is not an old-school entry
                continue
            solver=val[0]
            tol=val[1]
            rTol=val[2]
            if solver=="ICCG":
                new=( "PCG" , { "preconditioner":"DIC" } )
            elif solver=="BICCG":
                new=( "PBiCG" , { "preconditioner":"DILU" } )
            elif solver=="AMG":
                new=( "GAMG" , { "agglomerator":"faceAreaPair",
                                 "nCellsInCoarsestLevel":val[3],
                                 "mergeLevels":1,
                                 "smoother":"GaussSeidel"} )
            else:
                print "Unsupported solver",solver
                continue
            new[1]["tolerance"]=tol
            new[1]["relTol"]=rTol
            sol[name]=new
    except IOError:
        print "File",fName,"does not exist"
    except KeyError:
        print "File",fName,"is not a fvSolution-file"
 
    try:
        f.writeFile()
    except IOError:
        print "Can't write file. Content would have been:"
        print f

Of course 1.4 can read the old format, so this script is not needed. It's only an example.

2.4 Setting boundary conditions for walls

This example sets the boundary condition for all patches whose names contain the string "Wall" to zero-velocity:

 
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
 
f=ParsedParameterFile("pitzDaily/0/U")
 
for b in f["boundaryField"]:
    if "Wall" in b:
        f["boundaryField"][b]["value"]="uniform (0 0 0)"
        f["boundaryField"][b]["type"]="fixedValue"
 
f.writeFile()

This can serve as a starting-point for automatically setting up cases where the patches are named after a specific scheme

3 Installation

The easies way to install PyFoam is as root, but it is also possible to do it as a regular user.

3.1 Prerequisites

To install PyFoam you need (obviously) Python.

The smallest Python-version known to work is 2.5 but higher versions are better (not all features might be available with smaller versions). Basically the python that is preinstalled with most Linux-versions currently in use should be fine.

3.1.1 Libraries

PyFoam brings some essential libraries. For plotting and other data handling operations you will need numpy. Use the version provided by the package manager of your distribution.

Some utilities need additional libraries. Once installed you can use the pyFoamVersion.py-utility to find out who they are

If one gets a warning message:

 Neither numpy nor Numeric python-package installed. Plotting won't work

that can be fixed by installing the necessary package, by running the respective installation command, depending on your Linux Distribution:

Debian/Ubuntu:
```
sudo apt-get install python-numpy
```
Fedora/CentOS/RHEL/SL:
```
sudo yum install numpy
```
openSUSE:
```
sudo zypper install python-numpy
```

3.2 Installing with `pip`

This is now the preferred way of installing PyFoam.

PyFoam is now available at the Python Package Index. This means that if pip is installed a single command as root:

pip install PyFoam

or

sudo pip install PyFoam

installs PyFoam on the machine. The library numpy is also installed if it is not yet installed on the machine as it is required for plotting

If you can not write to the global Python installation because you don't have administrative rights try

pip install --user PyFoam

but you'l have to make sure that the location the utilities are installed to are in your PATH

An existing installation can be easily upgraded via pip:

pip install --upgrade PyFoam

3.3 Installation as root

This used to be the old way to install PyFoam but now should only be used in emergency cases. If possible use pip.

Download the latest tar
Untar it, e.g.:
```
tar -xf PyFoam-0.6.6.tar.gz
```
Go to the directory PyFoam-0.6.6 (or for whichever version you downloaded):
```
cd PyFoam-0.6.6
```
Install it with the command:
```
sudo python setup.py install
```

Should step 4 fail one of the most common causes is that some Linux-distributions (usually Debian-based, don't know about Mandrake) don't install the packages that are necessary for setup.py. Usually they are in the Developer-package which is installed by running the respective installation command, depending on your Linux Distribution:

Debian/Ubuntu:

sudo apt-get install python-dev python-setuptools

Fedora/CentOS/RHEL/SL:
```
sudo yum install python-devel
```
openSUSE:
```
sudo zypper install python-devel
```

3.4 Installation as a regular user

If you don't want to (or can't) install it as root things are a bit more complicated. Let's assume that you have directory /home/nonroot/my_python to which you have write access:

Download PyFoam-0.x.x (0.x.x being the current version)
Untar it, e.g.:
```
tar -xf PyFoam-0.6.5.tar.gz
```
Go to the directory PyFoam-0.6.5 (or for whichever version you downloaded):
```
cd PyFoam-0.6.5
```
Add the path to the library to the PYTHONPATH environment-variable, e.g.:
```
export PYTHONPATH=$HOME/my_python/lib/python-2.3/site-packages:$PYTHONPATH
```
- The lib/python-2.3/site-packages portion of the path may depend on your Python-version; in that directory there must be a directory named PyFoam.
Install it with the command
```
python setup.py install --prefix=$HOME/my_python
```
- After installing, in $HOME/my_python there should be two directories that contain the shell scripts and the Python libraries for PyFoam: bin and lib
If you want to use the scripts distributed with PyFoam extend your path with the following command:
```
export PATH=$HOME/my_python/bin:$PATH
```

3.5 Installation via RPMs

Starting with version 0.4.0 a source rpm and a regular rpm are provided for those who prefer to install it that way (The rpms have not been extensively tested. If there is a problem: please contact me)

For recent versions no RPMs were made available. You can produced them from the tarballs with something like:

python setup.py bdist

3.6 Obsolete: Installation under gentoo Linux

PyFoam is now in the main portage tree, so you can install it with emerge.

 
emerge PyFoam

Note: According to the information found at Gentoo this is quite an old version. It seems like the port is unmaintained

3.7 Installation under Debian/Ubuntu

Files to package PyFoam for the Debian packaging system are in the downloaded tarball or repository.

3.8 Testing the installation

To test whether the installation of PyFoam is working start the interactive Python shell with

python

(The shell can be left with Control-D). In that shell type

import PyFoam
import PyFoam.FoamInformation
print PyFoam.FoamInformation.foamTutorials()

If the last command produces an output similar to

/home/nonroot/OpenFOAM/OpenFOAM-1.2-devel/tutorials

everything is well.

If things don't work check in the Python-shell with the commands

import sys
print sys.path

whether the directory in which PyFoam resides is on the path searched by Python.

It can be useful to use the utility pyFoamVersion.py to view the PYTHONPATH, PyFoam version, OpenFOAM version and the installed libraries.

3.9 Docker containers

At this page there are instructions to install Docker containers with preinstalled versions of swak4Foam and PyFoam. These containers are based on publicly available containers of others (so your preferred OpenFOAM-version might not be there) and might not be the newest versions. But they should be fine for testing

4 Installed Utilities

All the utilities described in this section are installed to /usr/bin (if installed as root) and should be in the standard path for execution. They can all be called with the --help-option which gives all the available options and the latest, up-to-date information about the utility.

One recommended usage of these utilities would be to start a simulation with foamJob and display the residuals with pyFoamPlotWatcher.py without interfering with the simulation.

The following descriptions are not necessarily up to date. The text generated with the --help-option is

4.1 Runner-Utilities

These are utilities that start and control OpenFOAM-Runs. Apart from the features described below they start a little XMLRPC-server through which these runs can be controlled (see Network-Utilities).

Most of the utilities can be used to start parallel runs (the number of processors and a host-file have to be provided, depending on the MPI-implementation. The parallel environment is set up automatically).

4.1.1 `pyFoamRunner.py`

Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information:

Residuals and other information of the linear solvers
Execution time
continuity information
bounding of variables

4.1.2 `pyFoamUtilityRunner.py`

Runs a OpenFoam Utility an analyzes the output. Needs a regular expression to look for. The next 3 arguments are the usual OpenFoam arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamUtility.logfile). The Directory PyFoamUtility.analyzed contains a file with the information of the regexp (the pattern groups) at every time-step.

Doesn't start an XMLRPC-Server

4.1.3 `pyFoamSteadyRunner.py`

Runs an OpenFoam steady solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments) Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information

Residuals and other information of the linear solvers
Execution time
continuity information
bounding of variables

If the solver has converged (all linear solvers below threshold) it is stopped and the last simulation state is written to disk

4.1.4 `pyFoamPlotRunner.py`

Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). Information that is output as graphs are

residuals of the linear solver
continuity
bounding of variables (if it occurs)
Courant-number (optional)
Execution time of a time-step (optional)
number of iterations of the linear solver (optional)
arbitrary regular expressions can used to look for solver-specific data
- The groups in the regular expressions (patterns that are in brackets) are plotted
- in the regular expressions %f% can be used as a shortcut for floating-point numbers
- Any number of expressions can be specified and will be plotted in a separate window
For steady runs it behave like pyFoamSteadyRunner.py

4.1.5 `pyFoamMeshUtilityRunner.py`

Some utilities that manipulate meshes store the new mesh into the first timestep (so that they don't destroy the original mesh). This can be inconvenient because you have to copy the mesh back to constant for all further steps. This utility

Runs a mesh-utility
Moves the resulting mesh from the first time-step to the constant-directory

Be careful: This utility deletes the old mesh. If you don't know what you're doing, it will be gone forever

4.1.6 `pyFoamPotentialRunner.py`

Runs potentialFoam on a case and writes the results to p and U. Before running the solver it manipulates the dictionaries in system in such a way that potentialFoam runs and afterwards resets them to their old form.

4.1.7 `pyFoamRunAtMultipleTimes.py`

Runs a utility multiple times using different values for the -time option. Workaround for utilities that only allow a single time-step

4.1.8 `pyFoamRunParameterVariation.py`

This utility takes a template case and a file specifying the parameter variation and creates cases with the pyFoamPrepareCase.py-engine, runs a solver on these cases and collects the data into a database. The database can then be extracted with pyFoamDumpRunDatabaseToCSV.py

4.2 Utilities for Logfiles

These utilities can be used to analyze the output of an OpenFOAM-solver that has been written to a file

4.2.1 `pyFoamPlotWatcher.py`

Gets the name of a logfile which is assumed to be the output of a OpenFOAM- solver. Parses the logfile for information about the convergence of the solver and generates gnuplot-graphs. Watches the file until interrupted.

4.2.2 `pyFoamStandardLogAnalyzer.py`

Analyzes a Log written by foamJob. Needs the name of the Logfile to be analyzed the data is being written to a directory that has the same name with _analyzed appended

4.3 Networking Utilities

Most Runner-Utilities start a Server-thread that makes it possible to contact control an OpenFOAM-run and get information about it. These utilities help to access these servers.

Notes:

These utilities are beta-qualitiy. They work for my network but have not been tested in other environments
- The address of the pyFoamMetaServer.py is hardcoded into the sources, but can be overwritten using system-wide configuration-files
pyFoamNetShell.py doesn't need any additional infrastructure. Only the address and the port have to be known

4.3.1 `pyFoamNetList.py`

This command lists all the OpenFOAM-simulations in your network environment and gives you information about them. Amongst the information is:

Hostname and port (useful for pyFoamNetShell.py)
Details about the run (solver, working directory ...)
Estimates about the time at which the run will end (using the time since the start, current timestep and the endTime)

This utilitiy only works if a pyFoamMetaServer.py can be contacted at the configured address

4.3.1.1 `pyFoamMetaServer.py`

This program spawns a daemon process that runs forever. It stores information about the running processes. There are two mechanisms for it to get that information

scanning the network (at the start and on demand)
Runner-utilities try to register with it, when they start an OpenFOAM-run
- The utilities also dregister their process when they exit gracefully

4.3.2 `pyFoamNetShell.py`

Connects to an OpenFOAM-run started by a runner-Utility using

the hostname of the machine it runs on
the port number (usually 18000, if more than one run is running one of the following ports)

and starts an interactive shell there. On the shells the XMLRPC-commands provided by the server can be used to

get information about the run
- environment
- start-times
manipulate the run
- for instance stop() tells the run to write at the next time-step and then end
- kill() brutally terminates the run

Help about the available commands is provided.

Can also be used to control the pyFoamMetaServer.py (which is usually contacted on port 17999)

4.4 Utilities for Manipulating case data

These utilities are used for manipulating case date. They are especially useful in scripts that automatically set up simulation runs.

4.4.1 `pyFoamAddEmptyBoundary.py`

Adds an empty patch to the boundary file. Such a patch is needed for some mesh-maipulation utilities.

4.4.2 `pyFoamChangeBoundaryType.py`

Changes the type of a boundary patch.

4.4.3 `pyFoamChangeBoundaryName.py`

Changes the name of a boundary patch.

4.4.4 `pyFoamCreateBoundaryPatches.py`

Creates boundary patches in a field-file by looking at the boundary-file

4.4.5 `pyFoamClearCase.py`

Removes all the timesteps except for the first from a case directory.

4.4.6 `pyFoamPackCase.py`

Packs the essential files (the ones that are needed to run it) of a case into a tar-file for archiving/mailing-purposes

4.4.7 `pyFoamCloneCase.py`

Creates a copy of a case with only the most essential files

4.4.8 `pyFoamCopyLastToFirst.py`

Copy last time-step from one case to the first one of another (making it the initial condition)

4.4.9 `pyFoamClearInternalField.py`

Clears the solution from the internal field

4.4.10 `pyFoamClearBoundaryValue.py`

Clear non-uniform values from the boundary-fields

4.4.11 `pyFoamInitVCS.py`

Initialize the case for the use with the Version Control System (VCS) of your choice

4.4.12 `pyFoamSymlinkToFile.py`

This utility replaces a symlink with a copy of the file/directories it points to. To be used after a pyFoamCloneCase.py in --symlink-mode

4.4.13 `pyFoamCompressCases.py`

Goes through a case and compresses single files that are bigger than a certain threshold. Purpose is to shrink cases that were run with the setting uncompressed in the controlDict

4.5 Manipulating dictionaries (from scripts)

For more complex cases these utilities require an understanding of the syntax/semantics of Python-lists and dictionaries

4.5.1 `pyFoamReadDictionary.py`

Reads data from the root of a OpenFOAM-dictionary and prints it. To access subexpressions of a dictionary entry Python-expressions can be used (note: the expression is evaluated by the Python-interpreter, so quoting has to be used for strings, otherwise Python confuses them with variables)

4.5.2 `pyFoamWriteDictionary.py`

Writes data to the root of a OpenFOAM-dictionary and writes it to disk. Subexpressions can be selected (see note: pyFoamReadDictionary.py)

4.5.3 `pyFoamFromTemplate.py`

Generate a dictionary from a template file

4.5.4 `pyFoamCompareDictionary.py`

Compares two dictionaries structurally (not textually). Useful when the order of the entries or the formating have changed and diff won't give useful results

4.5.5 `pyFoamUpdateDictionary.py`

Updates a dictionary from another by adding/removing entries that are not in both.

4.5.6 `pyFoamUpgradeDictionariesTo17.py`

Checks the case and if fvSolution is in the old format rewrites it to the new format

4.5.7 `pyFoamUpgradeDictionariesTo20.py`

Checks the case and if blockMeshDict and thermophysicalProperties is in the old format tries to rewrite them to the new format. Also applies the changes pyFoamUpgradeDictionariesTo17.py would do

4.6 Paraview related utilities

Thes utilities require a paraview that is compiled with python-support. They have been only tested with ParaView 3.4 and may not work with other versions.

4.6.1 pyFoamPVSnapshot.py

Generates snapshots from OpenFOAM-data using a ParaView-state-file that was generated with another, similar case

4.6.2 pyFoamPVLoadState.py

Loads OpenFOAM-data and a Paraview-state-file that was generated with another similar case and lets you continue work like regular paraFoam

4.7 Other

Utilities that don't fit any of the other categories.

4.7.1 `pyFoamListCases.py`

Gives a ls-like overview of the OpenFOAM-cases in a directory

4.7.2 `pyFoamDecompose.py`

Generates a decomposeParDict according to the user specifications and
runs dcomposePar

This utility is quite useful when writing scripts where the number of available processors is not known beforehand (for instance if you are using a queueing-system)

4.7.3 `pyFoamComparator.py`

A utilitiy that automatically does a series of variations on a run (for benchmarking, testing and verification purposes). The behaviour is specified by an XML-file.

Work in progress. Therefor no example is present and the DTD that assists editing the XML-file is not included. Contact me, if you want to give it a spin (and want the DTD)

4.7.4 `pyFoamVersion.py`

Prints the Version of OpenFOAM and pyFoam. Used for information purposes (when reporting bugs). Also lists the Foam-versions installed in the OpenFOAM-directory

4.7.5 `pyFoamExecute.py`

Execute a command specifying a OpenFOAM-Version. Useful for compiling/executing a debug-version without changing the shell

4.7.6 `pyFoamBuildHelper.py`

Helper utility to build a project and its prerequisites (OF, libraries etc)

4.7.7 `pyFoamDumpConfiguration.py`

Dumps the values that can be changed via configuration files. The location of these configuration values are (lower overrides higer):

Hardcoded in the Library
Systemwide configuration file: /etc/pyfoamrc
User configuration file: ~/pyFoam/pyfoamrc

Parts of the output of the utilitiy can be pasted into the configuration files and modified to change the behaviour of PyFoam

4.7.8 `pyFoamSamplePlot.py`

Creates Gnuplot commands that plot the results of the sample-Utility (sorted by timestep, field etc) or the sample-function object

4.7.9 `pyFoamSurfacePlot.py`

Uses VTK to plot surfaces generated by the sampleSurface-Utility (sorted by timestep, field etc) or the sample-function object

4.7.10 `pyFoamTimelinePlot.py`

Creates Gnuplot commands to plot timelines produced by various functionObjects

4.7.11 `pyFoamRedoPlot.py`

Generates images by either connecting to a running server-process (one that is started for instance by pyFoamRunner.py or by reading a pickle-file (found in the .analyzed-directory)

4.7.12 `pyFoamCaseReport.py`

Prints small reports about a case. Currently implemnted:

Table of the boundary conditions
Boundary conditions on a per-patch basis (as opposed to the files where they are listed per-field)
Dimensions of the fields
Internal fields

4.7.13 `pyFoamEchoDictionary.py`

Just outputs a dictionary. Used to find bugs in the dictionary-parser

4.7.14 `pyFoamEchoPickledApplicationData.py`

Outputs the data from a pickle file. Either to the terminal or to a pipe

4.7.15 `pyFoamCaseBuilder.py`

Command line interface to the CaseBuilder-functionality

4.7.16 `pyFoamJoinCSV.py`

Joins different CSV-files and resamples them if needed

4.7.17 `pyFoamConvertToCSV.py`

Takes a file with column oriented data and converts it into a CSV

4.7.18 `pyFoamSTLUtility.py`

Takes a number of STL-files and joins them into one (with unique patch names)

4.7.19 `pyFoamPrintData2DStatistics.py`

Prints data generated from the Data2DStatistics data structure

4.7.20 `pyFoamCreateModuleFile.py`

Utility to create files for [1] that help switching between OF-versions

4.7.21 `pyFoamBinarySize.py`

Calculates the size of the binaries in an OpenFOAM-installation separated by compile-option

4.7.22 `pyFoamBlockMeshRewrite.py`

Assists the user in rewriting the blockMeshDict by doing simple, but error-prone transformations. Assumes "sensible" formatting: one block/vertex etc per line.

Sub-commands are:

refine: refines mesh by multiplying cell numbers in the blocks
number: Adds comments with the vertex numbers. Should help the user when editing/modifying the mesh
stripNumber: Remove the comments added by number
mergeVertices: Adds vertices from other blockMeshes that are not present in the current blockMesh
renumberVertices: Take another blockMeshDict, copy over the vertices-section of that mesh and rewrite blocks and patches so that they conform to these vertices. The original vertices have to be a sub-set of the vertices in the other mesh
normalizePatches: Rotates patches so that the lowest number is in front

4.8 GUI-Tools

Stuff that is not purely command-line oriented

4.8.1 `pyFoamDisplayBlockMesh.py`

Graphically displays a blockMesh and helps highlights selected blocks and patches. It is not a blockMesh-editor, but it helps to find errors in the order of vertices etc (editing of the blockMesh is still done in a text editor).

This needs an installation of the VTK with Python-bindings. The ways to get this are very different on various systems. But usually you have to compile it from the sources. Sorry.

Since 0.5.4 this is implemented in PyQT4. If that library is not found it falls back to the old (less feature-rich) version implemented in Tkinter

4.8.2 `pyFoamAPoMaFoX.py`: A Poor Man's FoamX

TUI (terminal/curses based) interface to the CaseBuilder-functionality (discontinued in PyFoam 0.5.6 as the TUI library was not available for most platforms)

4.8.3 `pyFoamAPoMaFoXiiQt.py`

Implementation of pyFoamAPoMaFoX.py using PyQt

4.9 Special utilities

Utilities for special applications

4.9.1 `pyFoamModifyGGIBoundary.py`

Modify GGI boundary condition parameters

5 Library Documentation

The tar-file contains the documentation of the Library as HTML (in the folder doc)

6 Rocks-Cluster support

There is library support to run OpenFOAM on a link Rocks cluster. It has only been tested on that type of cluster and probably has to be extended/adapted for other Clusters/Queuing-systems. There is a good chance that it might work on vanilla SunGridEngine installations, but this has not been tested.

An example cluster script would be

 
#!/usr/bin/python
#
#$ -cwd
#$ -j y
#$ -S /opt/rocks/bin/python
#$ -m be
#
 
import sys,os
from os import path
 
os.environ["WM_64"]="1"
 
from PyFoam.Infrastructure.ClusterJob import SolverJob
from PyFoam.Error import error
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
 
velIn=sys.argv[1]
 
name="pitzDaily"
 
class Pitz(SolverJob):
    def __init__(self):
        SolverJob.__init__(self,
                           name+".run."+velIn,
                           "simpleFoam",
                           template=name,
                           steady=True,
                           foamVersion="1.5")
 
    def setup(self,parameters):
        self.foamRun("blockMesh")
        vel=ParsedParameterFile(path.join(self.casedir(),"0","U"))
        vel["boundaryField"]["inlet"]["value"] = "uniform (%s 0 0)" % velIn
        vel.writeFile()
        
Pitz().doIt()

If this script (assume it has the name runPitzDaily.py) is submitted to a Rocks-Cluster with the SunGridEngine installed via

qsub runPitzDaily.py 5

(serial variant) or

qsub -pe mpi 4 runPitzDaily.py 5

the following things happen once the Job gets scheduled:

it is ensured that version 1.5 of OpenFOAM gets used
the template-case pitzDaily gets copied to a case pitzDaily.run.5 (automatically)
blockMesh is run on that case (by the script)
the inlet velocity is set to 5 (by the script)
if the case was submitted in parallel then it is decomposed (using the Metis-algorithm) - automatically
the case is run
if it was run in parallel it is reconstructed (automatically)

Apart from the setup-method the SolverJob-class has a number of other hooks that allow to manipulate the case before/after decomposition/reconstruction

A script can be tested on the local machine using

pyFoamClusterTester.py runPitzDaily.py 5

or in parallel

pyFoamClusterTester.py --proc=4 runPitzDaily.py 5

The tester-utility does not need the SGE (SunGridEngine)

6.1 Preparation of a user account on the SGE

For the Cluster to be able to find the PyFoam-stuff a user has to create a file .sge_request with the contents

-v PYTHONPATH=/home/common/python

(assuming that the PyFoam-library resides in a directory /home/common/python/PyFoam that is reachable from all cluster nodes)

7 Technical notes/known problems

This is a list of the currently known problems. If you find any other bugs (also known as "surprising features") don't hesitate to contact the author

if a run-fails the error message might be "eaten" by the utility controlling it. I'm working on this
the only MPI-environment I currently have available is OpenMPI. The pyFoam-Libraries used to work with LAM (still should, but I can not test it). If you have problems, please contact me (also "success-stories": "it works with MPIch" are most welcome)

7.1 Bug reporting

Bugs can be reported on the Bug-Tracker at openfoam-extend.sourceforge.net.

In reporting a bug, it can be useful to add the information given by the utility pyFoamVersion.py.

8 Additional stuff

Since Version 0.2 the distribution contains the benchFoam.py-script (Since 0.2.4 the benchmark suite is stable enough to be used by the public). From version 0.3.0 on it can be used as pyFoamBench.py

9 Other topics

9.1 Plotting with `customRegexp`-files

The utilities that do the plotting of time-graphs (pyFoamPlotRunner.py and pyFoamPlotWatcher.py) automatically plot user defined quantities if they find a file customRegexp in the case directory. Since Version 0.4.3 this file may have a more complicated syntax. One line of the file may for instance look like this (connaisseurs will recognize a Python-dictionary):

{"expr":"Reaction Speed: (%f%) min (%f%) max","name":"Reaction speed","titles":["min","max"]}

If the output of the solver has lines that look like this

Reaction Speed: -0.43 min 0.23 max

a plot with the name "Reaction speed" and two data sets ("min" and "max") will be created.

The fields in the dictionary are:

expr: This is a regular expression that is looked for in the output of the solver. The string %t% is shorthand for a more complicated expression that fits any real number. Any Regexp-Groups (the stuff between "()") is used as data for plotting. Please note that regular expression need some getting used to. One extra character can completely change the meaning of an expression and cause the expression to fail
name: The title of the plot
titles: The "legend" of the data lines found during fitting. These names are used in the order in which they are found in the Regexp

9.1.1 New format

Starting with version 0.5.3 a new format for the customRegexp is introduced. It basically looks like a OF-dictionary where each plot is specified by a separate sub-dictionary. The above custom plot can be written as

ReactionSpeed
{
  expr "Reaction Speed: (%f%) min (%f%) max";
  name Custom01_Reaction_speed;
  theTitle "Custom 1 - Reaction speed";
  titles
   (
     min
     max
   );
  type regular;
}

customRegexp-files in the old format are still recognized and can be dumped in the new format using the --dump-custom-regegexp-option

It is possible to let the titles be determined by the regular expression (idNr identifies the sub-expression that is the name):

ValuesOfSpecies
{
  expr "Value of species (.+) is (%f%)";
  theTitle "Don't know how many species we have";
  type dynamic;
  idNr 1;
  with steps;
}

Instead of plotting data can be appended to a different plot (and will be plotted there):

AddToSpeed
{
  expr "Average reaction speed (%f%)";
  titles ( avg );
  type slave;
  master ReactionSpeed;
}

9.1.2 Information on regular expressions

If a line contains a special character, one must indicate that for the parser just like in case of a regular expression. Solver output with multiple parentheses and square brackets:

Integral of U = (-0.05 0.02 0.05) [0 4 -1 0 0 0 0]

The matching expression:

expr "Integral of U = \((%f%) (%f%) (%f%)\) \[0 4 -1 0 0 0 0\]";

If in trouble, some nice summaries about regular expressions can be found with this search.

9.2 Settings

Several aspects of pyFoam can be modified using settings-files

The current settings can be viewed with the pyFoamDumpConfiguration.py-command. They are output in the format that is needed in pyfoamrc (so you can paste them in there and modify them) (BTW: the format is basically similar to the INI-files in Windows ca. 3.XX - in the days before the registry. It's primitive, but it's sufficient for our purposes)

The precedence of settings is:

Hardcoded in the PyFoam-sources
Systemwide in /etc/pyfoamrc
User-specific in ~/.pyFoam/pyfoamrc

Highest number wins

Settings can be overwritten for specific OF versions by adding new sections that are of the format MPI-1.7 for instance (entries in that section would overwrite the settings in MPI if the used version of OpenFOAM starts with 1.7 (1.7.x, 1.7 etc)

9.2.1 Modifying the call to `mpirun`

Different MPI-implementations and sites may need different options to mpirun.

The section of variables that is relevant to this is

[MPI]
options_openmpi_post: ["-x","LD_LIBRARY_PATH","-x","WM_PROJECT_DIR","-x","FOAM_MPI_LIBBIN","-x","MPI_B UFFER_SIZE"]
options_openmpi_pre: ["--mca","pls_rsh_agent","rsh"]

The _pre-option is the command-line arguments that are added to the mpirun command-line BEFRORE the -np/-machinefie argument the _post-option is for stuff that comes afterward (but before the actual OpenFOAM-command)

The middle-part of the variable name depends on the value of the $WM_MPILIB-variable. So for LAM this would be options_lam_pre/post.

9.3 Version control support

PyFoam adds some facilities to work with cases under version control. The advantage of this is that changes to a case are recorded when experimenting with a case (when using a Distributed VCS this also has some interesting applications for cloning/branching/merging cases). The disadvantage is that huge amounts of disc space are wasted if this is used unwisely.

Currently only support for mercurial is implemented (it was the natural choice as it is mainly implemented in Python). The implementation is generic enough to support other VCS but some low-level support has to be added

The case is initialized for VCS with the pyFoamInitVCS.py-command. This initializes the VCS-repository and adds the most important stuff in the case (the directories 0, constant and system. After that further files can be added using the usual commands of the used VCS. Options of the runner-utilities allow "commit before running"

The contrib-folder of the distribution contains a Mercurial-extension foamdiff. When this extension is installed a command like

hg foamdiff -r 23 system/controlDict

compares different revisions of a file not for textual differences but for semantic differences (using the same mechanisms that pyFoamCompareDictionaries.py uses)

10 paraFoam/paraview 'support'

If you have an installation of paraview that is 'Python-enabled' and PyFoam is found on the PYTHON_PATH then PyFoam can be used in the 'Programmable Filter'/'Programmable Source' of paraview by importing it as usual

import PyFoam

This is especially useful for data that was written by OpenFOAM but is not recognized by the regular importer. It can be loaded as usual with the ParsedParameterFile-class

Please note:

To visualize the data you need to know how to program VTK
Be aware, that due to the very general parser of PyFoam large datasets might consume huge amounts of memory and time during loading

10.1 Displaying the gravitation direction

10.1.1 As a Programmable Source

This little example displays the gravitation vector as a line. Just use it as a Programmable Source. It assumes that paraFoam was called in the case directory (otherwise casepath has to be modified)

 
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
from PyFoam.Paraview import caseDirectory
from os import path
import paraview
 
env=ParsedParameterFile(path.join(caseDirectory().constantDir(),"environmentalProperties"))
g=env["g"][2]
pdo=self.GetPolyDataOutput()
pdo.Allocate(1,1)
pts=paraview.vtk.vtkPoints()
pts.InsertPoint(0,(0.0,0.0,0.0))
pts.InsertPoint(1,g)
line=paraview.vtk.vtkPolyLine()
line.GetPointIds().SetNumberOfIds(2)
line.GetPointIds().SetId(0,0)
line.GetPointIds().SetId(1,1)
pdo.InsertNextCell(line.GetCellType(), line.GetPointIds())
pdo.SetPoints(pts)

Using the timeDirectory()-function you can program similar Programmable Filters for data that is stored in the time directories

10.1.2 As a script

An alternative is running this script via "Tools -> Python Shell":

 
from PyFoam.Paraview import readerObject,caseDirectory
from PyFoam.Paraview.SimpleSources import Glyph,Arrow
 
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile
from os import path
 
ro=readerObject()
 
env=ParsedParameterFile(path.join(caseDirectory().constantDir(),"environmentalProperties"))
g=env["g"][2]
 
# gly=Glyph("Gravity",ro.getCenter(),ro.getCenter()+0.5*g*abs(ro.getExtent())/abs(g))
gly=Arrow("Gravity",ro.getCenter(),ro.getCenter()+0.5*g*abs(ro.getExtent())/abs(g))
 
gly.repr.Color=(0,0,0)

10.2 Other example scripts

Can be found in the examples/paraview3-directory of the release. These are:

snappyDisplay: Displays the most important feature of a snappyHexMeshDict including the bounding-box and the STL-files
probeDisplay: displays the probes specified in system/controlDict

These Utilities are quite useful to find mistakes when setting up these files

10.3 General Remarks on the Programming

PyFoam takes care of some of the things described in this document when creating a new object. The two aspects of the object are accessible as

src: the actual VTK-object
repr: its paraview representation

You can access their properties as described in the document

There are still some issues with the deletion of these objects

sometimes they are still visible but not deleted
sometimes paraview crashes after deleting them using the "Delete"-button on the GUI

but your milage may vary

All this has been tested with ParaView 3.4 and may not work with other versions as the Python-API of paraview seems to be in flux

10.4 Necessary changes in the OF-installation to use the paraview-stuff

The etc/apps/paraview3/bashrc must be extended to extend the Python-PATH:

 
if [ "$PYTHONPATH" ]; then
    export PYTHONPATH=$PYTHONPATH:$ParaView_DIR/Utilities/VTKPythonWrapping
else
    export PYTHONPATH=$ParaView_DIR/Utilities/VTKPythonWrapping
fi
 
export PYTHONPATH=$ParaView_DIR/lib/paraview-3.3/:$PYTHONPATH
# export PYTHONPATH=$ParaView_DIR/lib/paraview-3.4/:$PYTHONPATH

For the Version 3.3-cvs that is distributed with 1.5 a little bug has to be fixed: look for the file ~/OpenFOAM/ThirdParty/ParaView3.3-cvs/platforms/linuxGcc/Utilities/VTKPythonWrapping/paraview/servermanager.py and there change the line 1110 from

 
            loader.SetRenderViewXMLName(rvname)

to

 
            loader.SetViewXMLName(rvname)

11 Other OpenFOAM-related Python libraries

PyFoam can't do everything (and it doesn't even try) but there are other efforts that help to use OpenFOAM programmed in Python

11.1 pythonFlu

pythonFlu is a wrapper to OpenFOAM that allows the programming of top-level solvers in Python (and I'm sure there will be other great applications that can't be easily done otherwise). More information can be found on the official pythonFlu website.

12 Downloads

12.1 Development version

The current development version can also be fetched using the command

 hg clone http://hg.code.sf.net/p/openfoam-extend/PyFoam PyFoam
 cd PyFoam

This is the current stable version. To get the current development version you also have to

 hg update develop

12.1.1 Updating existing working copies

If you downloaded this repository before January 2013 and want to keep this working copy see this article on information how to switch it to the new link.

12.2 Python Package Index

The latest released version is available via the Python Package Index

12.18.1 Buggy version (0.4.2)

From this version the Gnuplot-package was missing. Forget it

12.19 Ancient history

12.20 Previous version (0.4.2.1)

12.21 Previous version (0.4.1)

12.21.1 Previous version (0.4.0)

There were no complaints about this version

12.21.2 Very old versions

Media:PyFoam-0.3.4.tar.gz

13 History

2005-08-15: Version 0.1

2006-01-13: Version 0.2

2006-01-16: Version 0.2.1

2006-01-16: Version 0.2.2

2006-01-23: Version 0.2.3

2006-01-23: Version 0.2.4

2006-02-03: Version 0.2.5
- Improves the stability of the benchmark-script for parallel runs
- adds benchmakr suite for ancient versions of OpenFOAM

2006-02-03: Version 0.2.6
- Minor changes to the benchmark-scripts (other cases than the tutorial cases can now be used for benching)

2006-02-23: Version 0.3.0
- live plotting of residuals of solvers
- benchmark script moved to bin-directory

2006-03-22: Version 0.3.1
- BugFix: Plotting residuals in interFoam did not work
- Extending the Resiudals-Plotting (Courant-Numbers etc)
- Parsing of Dictionary-files into Python-Dictionaries (not yet fully tested)

2006-03-22: Version 0.3.2
- Plotting resiudals now works with OpenFOAM 1.3

2006-10-18: Version 0.3.3
- pyFoam distinguishes between the used MPI-implementations when starting parallel runs (currently LAM and OpenMPI)
- BugFixes
- added Utility pyFoamAddEmptyBoundary

2006-11-29: Version 0.3.4
- An error pointed out by Andrea Rimondi: the last version was incomplete. Due to a code reorganisation and missing files in the distribution half of the scripts in bin stopped working, but apart from Andrea everyone was too polite to mention it
- added Utility pyFoamClearCase.py

2007-03-29: Version 0.4.0
- Reimplementation of the parsing of OpenFOAM-Dictionaries
  - Can be manipulated like regular Python-Data-Structures
- Extension of the plotting-Utilities: arbitrary expressions are possible
  - The data can also be written to the disk
- Runs that are started by pyFoam can be accessed and controlled over the network (using XMLRPC)
- The runner-Utilities can start parallel-runs
  - A utility for generating a dictionary and decomposing a case is added
- Bug removed that didn't let pyFoamSteadyRunner.py work properly with OpenFOAM 1.3 (due to changes in the solver output)

2007-08-29: Version 0.4.1
- Debugging and stabilizing of the parsed OpenFOAM-Dictionaries
  - Changes in semantics: numbers are converted to numbers (not strings as before)
  - Can read and write gnuzipped files
  - More elegant access of the data
  - Order of entries in dictionaries is preserved, if possible
- Gnuplot-library enclosed in distribution (to ease installation)
- Enhancement of utilities
  - Runner-utilities can choose the Foam-version that should be used
  - Plot-Runner automaticaly loads regular expressions for a case
  - pyFoamBench now writes CSV-files which can be loaded into a spreadsheet-program
  - other stuff
- New utilities
  - Creation of boundary conditions
  - Cloning and packing cases
  - VTK-based utility to display blockMeshes
  - Specialized runner for mesh utilities
- Implementation of Unit-Tests for base classes to ensure stability
- Other changes I can't remember

2007-11-24: Version 0.4.2
- A framework to run jobs on a Rocks-cluster was implemented
- Support for cases that have more than one mesh-region (in the library and the utilities)
- Changes in the Library
  - the Application-classes have been changed, so that they can be called from other scripts
  - the OpenFOAM-version can be changed for the called applications
  - Extension of the Parser (Tensors etc)
  - Updated Parser Ply to newest version 2.3
  - A symbolic link controlDict.foam is automaticaly created (for the Paraview3.x-reader)
  - Gnuplot.py-library is adapted to use either numpy or Numeric (whatever is available)
  - mpirun can now have configurable parameters
- Changes in Utilities
  - the Runner-Utilities now have a --restart-option
  - All Utilities that start an OpenFOAM-application now can select the OF-Version with a --foam-version-option
  - Possibility to temporarily lower the relaxation factors for steady runs
  - Runner-Utilities can report memory usage
  - Clone-Case is now securer
- New Utilities
  - pyFoamDumpConfiguration.py: Dump the current configuration file for the PyFoam-installation
  - pyFoamExecute.py: Execute a command specifying a OpenFOAM-Version
  - pyFoamFromTemplate.py: Generate a dictionary from a template file
  - pyFoamUpdateDictionary.py: Updates a dictionary from another by adding/removing entries that are not in both
  - pyFoamCompareDictionary.py: Compares two dictionieries structurally (not textually)
  - pyFoamClusterTester.py: Test pyFoam-Scripts before submitting them to the cluster
- Semantic changes
  - Counting of linear iterations now accumulates over various solutions during the same timestep (different numbers in Plots!!)
- Removed Bugs: many

2007-12-04: Uploaded a bugfix version (Missing Gnuplot-library)

2008-04-21: Version 0.4.3
- Changes in the Library
  - Faster parsing of long lists
  - Parsing works with unicode
  - Better handling of solution-directories
- Changes in Utilities
  - Logfiles are now created using the name of the application
  - Various extensions of the Comparator-Utility (CSV-files etc)
  - Advanced syntax for the customRegexp-files
- New Utilities
  - pyFoamCopyLastToFirst.py: Copy last timestep to another case (as initial condition)
  - pyFoamPotentialRunner.py: Runs potentialFoam on a case (leaving everything else unchanged)
  - pyFoamSamplePlot.py: Generates graphs from data generated by the sampe-Utility
  - pyFoamClearInternalField.py: Resets the whole internal field to one value
  - pyFoamCaseReport.py: Information about a case (table of boundary conditions)
- includes new Version 1.8 of the Gnuplot-Library
- Removed Bugs: all that were reported (and some more)

2008-09-09: Version 0.5.0
- This version detects whether a OF-1.5 or previous is used and acts according to the convention for utilities in that version
  - the --foamVersion-option of the utilities is also aware of switches that change the convention
- Plot utilities do Postscript-Hardcopies if requested
- CaseReporter now also reports data about the decomposition
- Other extension of the CaseReporter
- Added an option to the runner-utils that temporarily disabel libs and functions in the cointrolDict (for incompatible utilities)
- Parser extended to handle special dictionary files (files without headers. Chemical reactions)
- added pyFoamEchoDictionary.py
- Various bugfixes (especially in the parser)

2008-10-30: Version 0.5.1
- Moved common functionality of the application to common classes
- Options get grouped in the online-help
- Parser is aware of include and parameter substitution
- Profiling of applications enabled
- Example for a Paraview-Source added
- Error with 1.5 and MultiRegion-cases fixed
- Parser fixed so that it can work with patch-names that are generated by snappyHexMesh
- CaseBuiilder-functionality added
  - Small TUI added as a demo
- Parser is now Version 2.5 of PLY (20% speed improvement)
- Bug in the potentialRunner-application fixed

2009-03-12: Version 0.5.2
- Uses color on the terminal to format the output of certain utilities (pyFoamCompareDictionary.py etc)
  - Can be configured
- User can be selected for pyFoamNetList
- Log-File can be surpressed
- The XMLRPC-server is more tolerant if the port is already in use
- Removed incompatibility with python 2.6 (clashes with the with-keyword)
  - 2.6 now complains about deprecated Libraries
- CaseBuilder allows the grouping of arguments
- Extended support for Paraview 3.4
  - Easy creation of graphics primitives (Spheres, Glyphs etc)
  - Automatically finds the case-directory of an OpenFOAM-case loaded in paraFoam
- Allows to chose whether symlinks will be followed when cloning a case
- Added pyFoamPVSnapshot.py and pyFoamPVLoadState.py
- Vector-class for vector-objects read during case-parsing (allows vector-arithmetics)
- Repaired pyFoamPackCase.py
- added pyFoamClearBoundaryValue.py
- added pyFoamListCases.py
- switch of the usage of the XMLRPC-server for certain utilities
- extension of pyFoamClearCase.py
- Enhancements of the dictionary parser
  - tries to preserve comments when writing the results to disk
  - is now based on PLY 3.1
- the plot-utilities can accumulate in different ways if a match is found multiple times per timestep
  - Display of data for cases with >>2000 time-steps is improved (better downsampling)
- pyFoamWatcher.py now can make hardcopies of the graphs
  - format of the hard-copies can be choosen

2009-11-10: Version 0.5.3
- General new stuff
  - Support for the regular-expressions that come with 1.6
  - Matplotlib can be used as an additional backend for plotting (this is beta-quality)
  - improved format for the customRegexp that looks more 'FOAMlike'. Old files can be dumped in the new format using the --dump-option of the PlotWatcher
  - two new plot types in the customRegexp ('dynamic' for getting the data names from the regular expression and 'slave' to add data to another plot instead of plotting itself)
- Additional utilities
  - GGI-script by Martin Beaudoin
  - pyFoamRunAtMultipleTimes.py after an idea by waterboy (MessageBoard-name)
- Bugfixes
  - Amount of files opened by FileCollection is limited (won't hit the OS-limit on the number of open files
  - Fixed a hang at the end of a run that occured if no meta-server was present
  - FoamServer reports actual IP (not 127.0.0.1)
- Changes
  - Additional parameter to the Decomposer-script (Martin Beaudoin)
  - Decomposer-script now recognizes scotch as a valid decompositon method and uses it as a default when used with OF-1.6
  - Plotter-Utilities write data to file
  - Added general options --force-debug --force-opt --force-32 --force-64 to switch to a specific subversion of OF
  - Changes that concern the network-infrastructure
  - Mesh can be excluded for the Pack-script
  - more stable method to handle temporary changes to the controlDict
  - for-loops over a PArsedParameter-file are possible
  - unit-tests work with the new tutorial-structure in 1.6
  - upgraded parser to PLY 3.2
  - Created files can be forced to gz
  - Various upgrades to the ParsedParameterFile-family
  - Utilities write a history to the case-directory
  - further adaptions for Python2.6
  - Watcher can read compressed logfiles
  - Written logs can be compressed on-the-fly
  - Clone works with cases that don't have a time-step
  - zsh is recognized (for switching OF-versions)
- Known limitations
  - Currently does not work with Paraview 3.6 (which comes with OF 1.6)

2010-06-11: Version 0.5.4
- Utilities
  - DisplayBlockmesh now converted to PyQT4.
    - Major enhancements (built-in editor)
    - If no PyQT4 found the old Tkinter-version will be used
  - PlotWatcher can be stopped on a file that is no longer updated (Mantis Bug #8)
  - Decomposer can treat regions separatly
  - Decomposer can be called directly in the case-directory
  - Runners can now switch on purgeWrite
  - RedoPlot adds titles
  - PotentialRunner works with 1.6
  - Enhancments to the CaseReporter
    - Writes to files
    - Report on linear solvers and relaxation
  - New utility: pyFoamSurfacePlot.py to generate images from suface-samples
  - New utility: pyFoamTimelinePlot.py that plots timelines that were generated by functionObjects. Also can plot Bar-plots
  - New utility: pyFoamRedoPlot.py to plot timelines from running processes (requires a server-process - for instance one started by pyFoamRunner) and pickled data from finished runs
  - ClearCase tries to remove data from functionObjects, too
  - SamplePlot can now write the Gnuplot-commands to a file
- Library
  - General application-class for PyQT4-classes
  - TimeLineAnalyzer is more tolerant
  - TemplateFile now shows the expression that gives a problem
  - numpy now preferred to Numeric
  - Pickle-file with the plot-data is written
  - Race condition with the pickle-file avoided
  - Configuration can now overwrite parameters depending on the OF-version
  - Configuration now allows directories with configuration snipplets (profile.d-style)
  - local configuration in a case-directory possible
  - Regular expression for "Time =" can now be configured
  - Paraview-stuff now accepts the native reader
  - SolutionDirectory automatically creates a .foam-stub-file for the native reader
  - FoamInformation more tolerant if no OF-installation is found
- Packaging
  - Basic framework for debian-packaging is there (only in SVN)

... and a number of other enhancements

2011-04-06 Version 0.5.5
- Utilities
  - CaseReporter now generates restructured text (this allows easy generation of PDFs and HTML using the right tools)
  - CaseReporter can now work with regions
  - Paraview-Utilities now know th native reader in Paraview 3.8
  - PyFoamChangeBoundaryName by Martin Beaudoin added
  - CloneCase now knows how to handle parallel runs
  - SamplePlot and TimelinePlot now can plot reference data
  - In the Runners the writing of data files now MUST be switched on
  - ListCases reports the state-files written by BasicRunner (if present)
  - ListCases has additional reporting options
  - CompareDictionary can now compare more than one dictionary at once
  - Option --autosense for the Runner-utilities now automatically determines the number of processors to use
  - SamplePlot, TimelinePlot and RedoPlot now can write CSV-Files
  - PyQT-variant of 'A Poor Mans FoamX' added to the distro
  - Decomposer now can handle parMetis
  - Utility pyFoamInitVCS.py added
  - --current uses the current OF-version (when switching to Debug) in all utilties that support version-switching
  - CloneCase now uses the VCS for cloning if the case is under version control
  - TimelinePlot now allows the usage of the time of the reference data for scaling
  - TimelinePlot and SamplePlot now can compare the data to the reference data quantitatively
  - Utility JoinCSV to join different CSV-files (resamples data if needed)
  - Runner-Utilites now can commit if the case is under VCS
  - Utility ConvertToCSV added
  - RunAtMultipleTimes now also works for parallel cases
- customRegexp
  - New entry progress now allows addition of custom values to the progress output of runners (for instance "h: $0" for addition of the first value found)
  - Plots now can be selected from the command line
  - All Utilities have an option to switch on/off the stack-trace in the event of an error
- Library
  - Gnuplot-Library bumped to 1.8
  - PLY updated to 3.3
  - Improved handling of Macros in the Dictionary Parser
  - Paraview-Sublibrary adapted to Paraview 3.8
  - Unittests now can be done using the nose-library
  - BasicRunner now writes some state-files
  - New SpreadsheetData-class generalizes the handling of data that will go into CSV-files
  - Enhancements to the CaseBuilder-backend
  - error-function now throws an Exception (this allows graceful error-handling)
  - Parser now knows that he can't handle binary files
- Bugfixes
  - WriteDictionary now can write negative values
  - BlockMeshes without edges not correctly parsed
  - If more than one triggered restore was accessing the file it was not correctly restored
- Other enhancements
  - In the configuration files features can be switched depening on the version. For instance the section MPI-1.7 overrides everything specified in MPI if the OpenFOAM-version starts with 1.7
  - File LocalConfigPyFoam in a case can be used to override the global settings for this case
  - Automatic reconstruction can be switched off for ClusterJobs
  - ClusterJobs can now also live with alread reconstructed Jobs
  - Addition of a Mercurial-extension foamdiff that can show changes between revisions like CompareDictionary does
  - Installed OpenFOAM-versions now can be looked for in more than one directory

2011-07-26 Version 0.5.6
- Changes to the parser
  - Now can parse the codestreams introduced in OpenFOAM 2.0
  - ParsedBlockMeshDict understands the new format
- Utilities
  - pyFoamUpgradeDictionaries17.py and pyFoamUpgradeDictionaries20.py to automatically upgrade selected dictionaries (fvSolution, blockMeshDict, thermoPhysicalProperties) to new formats
- Internal changes
  - Direct calls to shell commands for copying and removing files now replaced by Python-library calls
- Bugfixes
  - pyFoamCaseReporter can now handle all known formats of fvSolution/solvers
  - Fixed a problem where the Library was still using the old OF-calling convention when starting parallel runs
- Mercurial extension foamdiff
  - Now works with Mercurial 1.9 (new version untested with older Mercurial-version)

2012-04-13 Version 0.5.7
- Parser improvements
  - Problem with nested comments
  - Parse code streams
  - Preserving of comments can be switched off
  - Ignore extra semicolons
  - Allows parsing lists of length 3 and 9 as lists and not as vectors and tensors
  - "lookup redirection" in OF-dictionaries now works
- Utility improvements
  - pyFoamSamplePlot.py stops if the same directory is compared
  - pyFoamSamplePlot.py shows the location of the biggest difference
  - pyFoamSamplePlot.py allows only same ranges for comparisons
  - Generation of pickled-files can be switched of for runner-utilities
  - Frequency with which the pickled file is written is adapted (so that it doesn't use ALL the CPU-time)
  - pyFoamVersion.py improved (Version of Python is checked etc)
  - pyFoamRedoPlot.py: fixed parameter checking
  - pyFoamPotentialRunner.py: temporarily disable libs and functions
  - Only write last N loglines from Runner-utility
  - pyFoamClearCase.py: allow local specification of additional files that should be cleared
  - Runner utilities can report data about the run
  - pyFoamConvertToCSV.py now allows selection of columns
  - Utilities for quantative analysis now can return data
  - Utilities for quantative now correctly return data for multiple places
  - pyFoamSamplePlot.py now allows specification of valid variable pre and postfixes to allow correct parsing of variable names with a _
  - endTime can be specified by the runner-utilities
  - Utilities now allow piping (using pickled data)
  - pyFoamSamplePlot.py now allows the specification of a reference time
  - Nomenclature of pyFoamSamplePlot.py and pyFoamTimelinePlots.py now similar (both call it fields)
  - pyFoamDecompose.py now can use the -region-option if the OF-version is right
  - Forcing debug-mode fixed for older OF-versions
  - pyFoamDecomposer.py now accepts globalFaceZones in Python or OpenFOAM-syntax
  - Plot-utilities now don't interpret _ in names not as LaTeX
- New Utilities
  - pyFoamEchoPickledApplicationData to output pickled data
  - pyFoamPrintData2DStatistics.py to output data from comparisons
  - pyFoamBuildHelper.py to build project and its prerequisites (work in progress)
  - pyFoamCreateModuleFile.py to create files for http://modules.sourceforge.net/ (by Martin Beaudoin)
  - pyFoamSTLUtility.py to join STL-files
- Library improvements
  - stabler comparisons
  - Paraview-Utilities work with 1.x and 2.x
  - Runner classes return a dictionary with data
  - TimelineDirectory ignores dot-files
  - Application-objects can now be used like dictionaries to access data
  - New class TableData for simple data tables
  - New class Data2DStatistics for statistics about tables
  - new class CTestRun as basis for automated testing
  - FoamOptionParser now resets environment variables so that application-classes can call other application classes
  - Improvements to HgInterface
  - Additional VCS-subclasses for git, svn and svk (implementation only as needed)
  - SolutionDir now uses 0.org as initial directory if no valid initial directory is present (this affects clearing and cloning)
  - FoamFileGenerator now more flexible for long lists
  - ParsedBlockMeshDict now doesn't introduce prefixes for 'long' lists
- Removed utilities
  - pyFoamAPoMaFoX.py
  - pyFoamPlotResiduals.py
- Thirdparty
  - Got rid of Numeric-support in Gnuplot-library
- Other
  - script to generate man-pages for the utilities
  - Paraview3-example probeDisplay.py now renamed to probeAndSetDisplay.py and reads sampledSets from controlDict and sampleDict

13.1 Version 0.6.0 - Release 2013-03-14

13.1.1 Major changes

13.1.1.1 Adaption to work with Python3

Sources are adapted so that PyFoam works with Python3 too. This breaks support for Python 2.4 and earlier (possibly also Python 2.5)

Some of the Libraries in PyFoam.ThirdParty had to be adapted to work with Python3:

Gnuplot.py: The original version 1.8 is quite old. It was adapted with the help of the six-library (see below) to work with Python2 and Python3 (inspired by [2] which is a pure port to Python3 without backwards compatibility)

13.1.1.2 New ThirdParty-Libraries

six: Library that helps supporting Python 2 and Python 3 in the same source code. Currently version 1.2 from [3] is used
pyratemp: Templating library to support the new templating format. Version 0.2.0 from [4] is used

13.1.1.3 Porting to `Windows`

Port to allow running PyFoam on Windows was done by Bruno Santos of blueCAPE (bruno.santos@bluecape.com.pt)

Patch was originally posted at http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166

Note: many of PyFoam's features are not yet fully functional on Windows.

13.1.1.4 Experimental port to `pypy`

Sources are executed in pypy but it seems there are problems with numpy and also with code like for l in open(f).readlines()

13.1.2 Third-Party

13.1.2.1 Upgraded `ply` to 3.4

This brings virtually no changes. README with copyright information has been added

13.1.3 Infrastructure

13.1.3.1 Parameters can't be modified in `CTestRun` after initialization

This should help to avoid side-effects

13.1.3.2 Treat timeouts in the `MetaServer` right

Due to a previous workaround timeouts when collecting information about new machines was not treated correctly

13.1.3.3 Add `execute`-method to `ClusterJob`

This allows the execution of a shell-script in the directory of the case

13.1.3.4 Add possibility to run specific modules before or after the solver

These modules are found in PyFoam.Infrastructure.RunHooks. Two concrete implementations:

PrintMessageHook: to print a text to the terminal
SendToWebservice: encode an URL and send it to a webservice (example for pushover.net added)

Hooks are automatically instantiated from the configuration data (examples are hardcoded))

13.1.3.5 Parameters added to the info about the run

The Runner-classes now have a parameter parameters. This data (usually it would be a dictionary) is added verbatim to the run info.

Most runner applications now have the possibility to add this info.

Purpose of this facility is to identify different runs in the database better.

13.1.3.6 Parameter handling in `ClusterJob` extended

Parameter values are now handed to the actual job. Also a dictionary with parameters can be handed to the constructor and will be used in the relevant callbacks

13.1.3.7 Run data written alongside `PickledPlots`

During the run whenever the PickledPlots-file is written a file pickledUnfinishedData gets written. This has the current solver data and is similar to pickledData.

Also a file pickledStartData gets written that has the data that is available at the start of the run.

13.1.3.8 `BasicRunner` collects error and warning texts

The runner collects

at every warning the next 20 lines of the output until a total of 500 lines is reached (this avoids filling disk and memory if the solver produces too many warnings)
All output from an error message until the end

And stores them in the application data

13.1.4 Library

13.1.4.1 `TemplateFile` now uses `pyratemp`

The class TempalteFile now uses an enhanced templating engine. The old implementation is in the class TemplateFileOldFormat

13.1.4.2 Clearer error message in Application-classes

If used as classes (not as utilities) these classes print the class name instead of the calling utilities name

13.1.4.3 Output is only colored if it goes to the terminal

Error and warning messages don't decorate the output if it goes to files or other non-terminal streams

13.1.4.4 `error`-method of application classes now raises an exception

An exception is now raised by self.error(). This makes it easier to handle such errors if the application class is used. The exception is passed up until there is a "real" application

13.1.4.5 `ParsedParameterFile` now knows how to handle binary files

When the format of a file is binary lists with a length prefix are being read as binary blobs.

For reading the blobs a simple heuristics is used: a multiple of the length in bytes is read. If the next character is a ) and the characters after that are a certain combination of characters (newlines and ;) then it is assumed that the blob has ended. This may fail on certain occasions:

if the combination of characters appears at these places
if the objects inside the binary data are of different sizes

It would be hard to work around these restrictions without reprogramming the full functionality of OpenFOAM

13.1.4.6 `LabledReSTTable` for more flexible table generation

New class in the RestructuredTextHelper allows more flexible generation of tables. Items are added with column and row and if these don't exist in the first row/column the table is extended appropriately

13.1.4.7 Plotting classes now allow setting of `xlabel`

This is implemented for Gnuplot and Matplotlib. Default for the label on the x-Axis is now "Time [s]"

13.1.5 Utilities

13.1.5.1 `pyFoamFromTemplate.py` with new templating engine

The utility can now use the pyratemp-templating engine which allows templates with loops, conditions and other fancy stuff

13.1.5.2 `pyFoamSamplePlot.py` allows using the reference data as basis for comparison

Instead of using the x-values from the original data the y-values of the reference data can be used for comparing (with the --use-reference-option)

Same for pyFoamTimelimePlot.py

13.1.5.3 Scaling and offsets are now used in plots of `pyFoamSamplePlot.py`

If scales not equal to $1$ and offsets not equal to $0$ are specified they are used in the gnuplot-output

13.1.5.4 `pyFoamPrintData2DStatistics.py` prints relative average error

With the --relative-average-error-option

13.1.5.5 Enhancements to `pyFoamVersion.py`

More tolerant if no library was found
Reports the location of the PyFoam-Library
Checks whether utility version is consistent the library found

13.1.5.6 `pyFoamRunner.py` allows hooks

Hooks can be added at the start and the end of a run

13.1.5.7 `pyFoamRedoPlots.py` supports range for plots

Added -end and -start-option to select a range that should be plotted.

Currently not working with the Matplotlib-implementation (only gnuplot)

13.1.5.8 `pyFoamDisplayBlockMesh.py` no supports templates

If a file with values is specified then the utility assumes you're editing a template file and will evaluate it before displaying it

13.1.5.9 `pyFoamCaseReport.py` is tolerant towards binary files

New switch that makes the parser treat files that are declared binary in the header as if they were ascii

13.1.5.10 `pyFoamSamplePlot.py` and `pyFoamTimelinePlot.py` raise error if no plots are generated

This makes it easier to catch faulty specifications (or empty timeline-files)

13.1.5.11 `pyFoamSurfacePlot.py` can wait for a key

An option --wait has been added that makes the utility wait before displaying the next picture

13.1.5.12 `pyFoamEchoDictionary.py` is more flexible with binary files

Switch allows forcing it to read a binary File as an ASCII

13.1.5.13 All utilities now have a switch that starts the debugger even with syntax-errors

Previously the option --interactive-debug only started the debugger if the error was no syntax error. This is still the default behavior, but can be overruled

13.1.5.14 Utilities now can be killed with `USR1` and will give a traceback

The option --catch-USR1-signal now installs a signal-handler that prints a traceback and finishes the run. If the interactive debugger is enabled then it goes to the debugger-shell.

Option --keyboard-interrupt-trace triggers the same behaviour for keyboard interrupts with <Ctrl>-C

13.1.5.15 Switch to switch on all debug options

For the purpose of developing a switch --i-am-a-developer has been added.

13.1.5.16 Plotting utilities now allow specification of x-Axis label

With the option xlabel in the customRegexp-file the label on the x-axis of the plot can be changed. Setting ylabel and y2label (for the secondary axis) was already possible

13.1.5.17 Metrics and compare for `pyFoamTimelinePlot.py` and `pyFoamSamplePlot.py` support time ranges

Now the options --min-time and --max-time are supported by --metrics and --compare

13.1.5.18 `pyFoamDisplayBlockMesh.py` allows graphical selection of blocks and patches

New addition by Marc Immer allows the graphical selection of blocks and patches and adds them to the blockMeshDict

13.1.5.19 `pyFoamCloneCase.py` and `pyFoamPackCase.py` accept additional parameters

The file LocalConfigPyFoam is read by these utilities and if there is a parameter addItem in the section Cloning defined then these files are cloned/packed automatically (no user specification required)

13.1.5.20 `pyFoamListCases.py` now calculates estimated end-times

Additional option to print the estimated end times. These can be wrong if the case did not start from the startTime in the controlDict.

Also now allows printing the end and the start-time according to the controlDict

13.1.6 New features

13.1.6.1 Different "phases" for multi-region solvers

Plots of type phase in customRegexp don't actually plot anything. The set a phase-name that is used for subsequent values (for instance to distinguish the different residuals)

13.1.6.2 `pyFoamChangeBoundaryType.py` allows selection of region and time

Options --region and --time-directory added that allow selecting different boundary-files

13.1.6.3 New class for storing case data in a sqlite-database and associated utilities

The class RunDatabase stores the data from runs. Utility pyFoamAddCaseDataToDatabase.py is one way to populate the database. pyFoamDumpRunDatabaseToCSV.py allows dumping that data to a file for further processing (in a spreadsheet for instance)

Database can also be populated using a special post-run hook

13.1.7 Bugfixes

13.1.7.1 Only binary packages of 1.x were found

Pattern had to start with 1 (now every digit is possible))

13.1.7.2 Option group Regular expressions was listed twice

No harm done. But fixed

13.1.7.3 `--clear`-option for `pyFoamDecompose.py` not working

Reason was that rmtree does not allow wildcards. Fixed

13.1.7.4 `pyFoamDisplayBlockmesh.py` not working with variable substitution

The DictRedirect would not convert to float. Fixed. Although it might happen again for other data types

13.1.7.5 Option `--function-object-data` of `pyFoamClearCase.py` not working with directories

The option was only implemented for the list-form of the functions entry in controlDict

Now fixed to also work with the dictionary-form

13.1.7.6 `nonuniform` of length 0 not correctly printed

Seems like the length was interpreted as the name of the list. Fixed

13.1.7.7 Building of pseudocases with `pyFoamRunner.py` broken

Only worked if no region was specified (= not at all). Fixed

13.1.7.8 `pyFoamRedoPlot.py` did not correctly honor `--end` and `--start`

Plots were over the whole data range. This is now fix (also the issue that --end alone did not work)

13.1.7.9 `WriteParameterFile` does not preserve the order of additions

Contents was "only" set as a dictionary which does not preserve the order in which entries are added. Replaced with a DictProxy

13.1.7.10 Wrong number of arguments when using `TimelinePlot` in `positions`-mode

Problem that was introduced by changes in the fields-mode

13.1.7.11 `ClusterJob` uses only `metis` for decomposition

For OpenFOAM-versions 1.6 and higher the automatic decomposition used is now scotch

13.1.7.12 `pyFoamSamplePlot.py` and `pyFoamTimelinePlot.py` produced no pictures for regions

As regions have their own subdirectories the / from the directory name was inserted into the filename and if the subdirectory did not exist gnuplot did not create the picture

13.1.7.13 Barplots in `pyFoamTimelinePlot.py` not working if value is a vector

The base class didn't correctly handle the ( and ). Fixed

13.1.7.14 Mysterious deadlocks while plotting long logfiles

The problem was that during splitting the timeline data an exception was raised. This exception was caught by another part of PyFoam. This left a lock on the data structure locked and the next access to the structure was held indefinitely. Fixed

13.1.7.15 Scanning linear expressions form the block coupled solver failed

As there is a tuple of residuals the scanner did not analyze the output of the output of the block-coupled solver from 1.6-ext correctly. This is now treated as a special case and each residual is plotted separately (distinguished by a [x] with x being the number of the component)

13.1.7.16 `#include` not correctly working with macros in the included file

Macros $var were not correctly expanded. Fixed

13.1.7.17 Macros not correctly expanded to strings

When being expanded to string form macros were not correctly expanded

13.1.7.18 `pyFoamPackCase.py` in the working directory produces 'invisible' tar

If the utility was used in the form

pyFoamPackCase.py .

then an 'invisible' tar ..tgz was produced. Fixed

13.1.7.19 String at the end of a linear solver output makes parsing fail

Reported in [5] the string is assumed to be part of the iteration number. Fixed

13.1.7.20 Paraview utilities not working with higher Paraview versions

At least for PV 3.14 and 3.98 the way the version number is determined has changed and the PV-utilities failed. This has been fixed but is untested with old versions

13.1.7.21 Camera settings not honored with `pyFoamPVSnapshot.py`

For the first rendered view Paraview automatically resets the camera. This has now been switched off (so the snapshot is rendered correctly)

13.2 Version 0.6.1- Release 2013-05-24

13.2.1 Bugfixes

13.2.1.1 Restoring of `controlDict` after `write`

When activating an on-demand write the constrolDict was not restored because the output-line about the file being read was not recognized (due to a change in the output in recent OF-versions). Now a number of different formats is recognized

13.2.1.2 Custom-plot type `slave` not working if no `master` defined

That plot-type needs a master. Fixed to fail if none is defined

13.2.1.3 `-list-only` did not correctly parse lists with a numeric prefix

This did affect all utilities that use that option and also calls with listOnly to the library class

13.2.2 Utilities

13.2.2.1 `pyFoamBuildHelper.py` now allow more than one action

If multiple actions like --update and --build are specified they are executed in a sensible order (update before build etc)

13.2.2.2 Utilities warn if OpenFOAM-version is unset

If the environment variable that determines the OpenFOAM-version is unset a warning is issued by the utilities

13.2.2.3 `pyFoamUpgradeDictionariesTo20.py` allows single files

If single file is specified then the action to transform it has can be specified

13.2.2.4 `pyFoamUpgradeDictionariesTo20.py` transforms reaction-schemes

Now knows how to transform "old" reaction files (where the reactions-entry was a list) to the new format (where it is a dictionary). Only a limited number of reaction types is supported.

13.2.2.5 `pyFoamUpgradeDictionariesTo20.py` transforms thermophysical data

Now the old form of thermophysical data (lists) is transformed into the new dictionary-form

13.2.2.6 `pyFoamCloneCase` now allows creating directory that symlinks to the original

Now with the option --symlink-mode instead of copying the directories from the original new directories art created and populated with symlinks to the files in the original. The depth until which no symlinks to directories are created can be specified. This allows the clone to share the configuration files with the original

13.2.2.7 `pyFoamClearCase.py` now removes `postProcessing` and allows removal of additional files

The directory postProcessing is now automatically removed (can be switched off with --keep-postprocessing). Also with the --additional-option patterns with additional files to remove can be specified.

13.2.2.8 Improvements to `pyFoamVersion.py`

Now reports the location of the python-executable
Reports locations of used libraries

13.2.2.9 Additional files automatically cloned

The files Allrun, Allclean and 0.org are automatically added during cloning as these are often used by the standard-utilities

13.2.2.10 `pyFoamDisplayBlockMesh.py` uses the same options for template format as `pyFoamFromTemplate.py`

This makes sure that templates are handled consistently and also allows different delimiters in the blockMeshDict.template

13.2.3 Library

13.2.3.1 Improvements in syntax of `ParsedParameterFile`

Now the new relative scoping that was introduced in OF 2.2 is supported

13.2.3.2 `Utilities`-class now function to find files matching a pattern

Added a function find that approxiamtes the find-command

13.2.3.3 VCS ignores more files

Some more patterns have been added that will be ignored in a VSC-controlled case. All of them concerning files that PyFoam creates during operation

13.2.4 New features/Utilities

13.2.4.1 New Utility `pyFoamSymlinkToFile.py`

This utility replaces a symlink with a copy of the file/directories it points to. To be used after a pyFoamCloneCase.py in --symlink-mode

13.3 Version 0.6.2 - 2013-11-03

13.3.1 Major changes

13.3.1.1 Use of `pandas`-library

Starting with this version the pandas-library is used fordata-analysis. When possible and appropriate classes returnpandas-objects. Currently these are:* CSVCollection. With a call-operator this class returns the collected data as a DataFrame of the collected data* SpreadsheetData now has methods to return Series and DataFrame objectsIt is not necessary to install pandas if these classes are notused (and even then most of their functionality works)

13.3.2 Incompatibilities

13.3.2.1 Different separator for databases in CSV-files

The class RunDatabase (and therefor also the utilitypyFoamDumpRunDatabaseToCSV.py) now write as a separator for datafrom sub-tables a // instead of the space. This especially meansthat scripts that rely on a data-item foo in analyzed mightbreak because this is now called analyzed//foo instead of=analyzed foo=. On the other hand this makes the names moreconsistent and easier to parse as // is the saperator for otherlevels of dictionaries

13.3.2.2 Change of independent variable name in sample data

Instead of col0 this is now coord. This could cause problemswith scripts that use that column name in the resultingSpreadsheetData-object

13.3.3 Bugfixes

13.3.3.1 `pyFoamPackCase.py` does not handle symbolic links correctly

Symbolic links were copied as is and did not work correctlyafterwards. This is fixed. If the symbolic link is an absolutepath or points outside the case directory it is replaced with thefile it points to. Otherwise it is preserved as a symbolic link

13.3.3.2 `pyFoamPotentialRunner.py` not working with OpenFOAM 2.0 or newer

These versions require an entry potentialFlow in thefvSolution-file instead of the old SIMPLE

13.3.3.3 `pyFoamListCase.py` fails with `controlDict` that use preprocessing

Fixed by first trying to read that with preprocessing. Without ifthat fails

13.3.3.4 Cloning fails in symlink-mode if files are specified twice

Now using a set instead of a list makes sure that no file iscloned twice

13.3.4 Utilities

13.3.4.1 `pyFoamPotentialRunner.py` now allows removing of `functions` and `libs`

The utility now allows removing these entries in case that theydon't work with potentialFoam

13.3.4.2 The Runner-utilities now have more options for clearing

Some of the options of pyFoamClearCase.py for clearing cases(for instance specifying additional files) have been ported to theRunner-utilities. Also is the postProcessing-directoryremoved by default

13.3.5 Library

13.3.5.1 `SolutionDirectory` and `TimeDirectory` are more tolerant

If there are field files and their zipped counterpart thaninstead of an error a warning *can* be given

13.3.5.2 `ClusterJob` now handles template files

A new method templateFile gets the name of a file which isconstructed from a template of the same name plus the extension.template

13.3.5.3 Additional parameters in `ClusterJob`

The method additionalParameters can return a dictionary withadditional parameters

13.3.5.4 Custom data in directory easier accessible

In the written data in the sub-dictionary analyzed there is nowa subdictionary Custom with the values of the custom expressionswith the prefix CustomXX_ removed. This means that values thatwere available as data['Custom02_velCheck']['min']are now available as data['Custom']['velCheck']['min']The advantage is that the number part which was dependent on theorder the expressions were specified is now no longer necessary(this should make scripts more stable)The old notation is still available but deprecated

13.3.5.5 `SolverJob` now allows compression of output

The parameter solverLogCompress compresses the log-file whilewriting it to disc. *Attention:* This may lead to corruptedlog-files if the run crashes

13.3.5.6 `PyFoamApplication`-class now allows quick access to data

The dictionary returned by getData() now allows access to allthe elements as attributes.

13.3.6 New features/Utilities

13.3.6.1 Post-run hook that sends mail at the end of run

The hook-module MailToAddress sends a mail at the end of arun. Prerequisite is an SMTP-Server that doesn't needauthentication

13.3.6.2 New utility `pyFoamCompressCases.py`

This utility goes through cases and compresses single files. Thecases can be searched recursively to.Purpose of this utility is to shrink cases wherewriteCompression was not turned on during the run

13.3.6.3 Paraview-module to read additional data

A new module PyFoam.Paraview.Data reads additional data usuallywritten by OpenFOAM. These are converted to vtkArray using thefollowing functions and can be used in =Programmable filters=:- setSampleData :: reads the data from sampled sets- setTimelineData :: reads data from a timeline directory- setPlotData :: reads pickled plot data using RedoPlot

13.3.7 Enhancements

13.3.7.1 `pyFoamRedoPlot.py` can plot in XKCD-mode

When used with the option --implementationxkcd= and version ofmatplotlib that supports it is installed then plots are done inthe style of the webcomics [[6]]

13.3.7.2 `pyFoamListCases.py` now displays disk usage in human readable form

If the disk usage of the cases is calculated then it is displayedin human readable form (as KB, MB, GB or TB) for sizes larger thanone Kilobyte

13.3.7.3 `pyFoamClearCase.py` more flexible in selection of data to be removed

Options to be more flexible in removing data are added:; keep-interval : keep timesteps at a specified interval. For instance --keep-interval0.1= will keep times like $1$, $1.1$ etc but remove $1.05$; keep-parallel : this will not remove any times in the processor-directories. Also are things like keep-last now honored for processor directories; remove-analyzed : Remove the directories with the analyzed datatoo. Old behavior was to remove them. Now they are kept by default

13.3.7.4 `pyFoamFromTemplate.py` automatically chooses template and default values

If an output file foo is specified and no template then theutility looks for a file foo.template as a template.If a file foo.defaults is present then this file is read andused as default parameter values. Other specifications overridethese defaults

13.3.7.5 `pyFoamDumpRunDatabaseToCSV.py` can disable standard-fields

Additional option --disable-run-data

13.3.7.6 `pyFoamDumpRunDatabaseToCSV.py` prints `pandas`-object

With the -pandas-print-option a DataFrame is generated andprinted

13.3.7.7 Better debugging with `ipdb`

If the ipdb-package (basically pdb with IPython-additions)is installed then it is used. This gives additions liketab-completion

13.3.7.8 Interactive shell after execution for utilities

The option --interactive-after-execution drops the user to aninteractive shell where the namespace can be inspected. If presentIPython will be used, otherwise the regular shell is used

13.3.7.9 Utilities that read quantitative data convert to `pandas`-data and/or `numpy`

This is mainly to be used on the interactive shell to do furtheranalysis or write this data out. The utilities are:; pyDumpRunDatabaseToCSV.py : add an item dump with the wholedata as a DataFrame; pyFoamTimelinePlot.py : add element series with all the dataas Series and dataFrame with the same data as a DataFrame; pyFoamSamplePlot.py : Like pyFoamTimelinePlot.py; pyFoamRedoPlot.py : Now can get series and the whole plot dataas pandas-objects

13.3.7.10 Utilities that read quantitative data write Excel files

The utilities pyDumpRunDatabaseToCSV.py,pyFoamTimelinePlot.py, pyFoamSamplePlot.py andpyFoamRedoPlot.py now have options to write Excel-files

13.3.7.11 Specify additional settings for `GnuPlot` in `customRegexp`

If an item in customRegexp has an item gnuplotCommands thenit is assumed that this is a list of strings which are executedbefore the first plotting. For instance gnuplotCommands ( "set format y '%.2e'" );changes the number format on the y-axis

13.3.7.12 More flexible data specification for `pyFoamSamplePlot.py`

Instead of determining the names of the fields and lines form thefilenames it is now also possible to specify them through options.The option --is-distribution is a shorthand that sets theseoptions for distribution files

13.3.7.13 `pyFoamSamplePlot.py` now allows specification of x-range

The range of the x-axis of the plots can either be set byautomatically scaling to the domains of all the data sets with--scale-domain or by specifying them with --domain-minimum or--domain-maximum.These domains are set for all plots