A python library to control OpenFOAM-runs and manipulate OpenFOAM-data. Comes with a number of utilities that should make your life easier if you're not scared by commandlines
Valid versions:
works with older versions (but I don't support that)
Contents
- 1 Short description
- 2 Examples
- 3 Installation
- 4 Installed Utilities
- 4.1 Runner-Utilities
- 4.2 Utilities for Logfiles
- 4.3 Networking Utilities
- 4.4 Utilities for Manipulating case data
- 4.4.1 pyFoamAddEmptyBoundary.py
- 4.4.2 pyFoamChangeBoundaryType.py
- 4.4.3 pyFoamChangeBoundaryName.py
- 4.4.4 pyFoamCreateBoundaryPatches.py
- 4.4.5 pyFoamClearCase.py
- 4.4.6 pyFoamPackCase.py
- 4.4.7 pyFoamCloneCase.py
- 4.4.8 pyFoamCopyLastToFirst.py
- 4.4.9 pyFoamClearInternalField.py
- 4.4.10 pyFoamClearBoundaryValue.py
- 4.4.11 pyFoamInitVCS.py
- 4.4.12 pyFoamSymlinkToFile.py
- 4.4.13 pyFoamCompressCases.py
- 4.5 Manipulating dictionaries (from scripts)
- 4.6 Paraview related utilities
- 4.7 Other
- 4.7.1 pyFoamListCases.py
- 4.7.2 pyFoamDecompose.py
- 4.7.3 pyFoamComparator.py
- 4.7.4 pyFoamVersion.py
- 4.7.5 pyFoamExecute.py
- 4.7.6 pyFoamBuildHelper.py
- 4.7.7 pyFoamDumpConfiguration.py
- 4.7.8 pyFoamSamplePlot.py
- 4.7.9 pyFoamSurfacePlot.py
- 4.7.10 pyFoamTimelinePlot.py
- 4.7.11 pyFoamRedoPlot.py
- 4.7.12 pyFoamCaseReport.py
- 4.7.13 pyFoamEchoDictionary.py
- 4.7.14 pyFoamEchoPickledApplicationData.py
- 4.7.15 pyFoamCaseBuilder.py
- 4.7.16 pyFoamJoinCSV.py
- 4.7.17 pyFoamConvertToCSV.py
- 4.7.18 pyFoamSTLUtility.py
- 4.7.19 pyFoamPrintData2DStatistics.py
- 4.7.20 pyFoamCreateModuleFile.py
- 4.7.21 pyFoamBinarySize.py
- 4.7.22 pyFoamBlockMeshRewrite.py
- 4.8 GUI-Tools
- 4.9 Special utilities
- 5 Library Documentation
- 6 Rocks-Cluster support
- 7 Technical notes/known problems
- 8 Additional stuff
- 9 Other topics
- 10 paraFoam/paraview 'support'
- 11 Other OpenFOAM-related Python libraries
- 12 Downloads
- 12.1 Development version
- 12.2 Python Package Index
- 12.3 Current version (0.6.6)
- 12.4 Current version (0.6.5)
- 12.5 Previous version (0.6.4)
- 12.6 Previous version (0.6.3)
- 12.7 Previous version (0.6.2)
- 12.8 Previous version (0.6.1)
- 12.9 Previous version (0.6.0)
- 12.10 Previous version (0.5.7)
- 12.11 Previous version (0.5.6)
- 12.12 Previous version (0.5.5)
- 12.13 Previous version (0.5.4)
- 12.14 Previous version (0.5.3)
- 12.15 Previous version (0.5.2)
- 12.16 Previous version (0.5.1)
- 12.17 Previous version (0.5.0)
- 12.18 Previous version (0.4.3)
- 12.19 Ancient history
- 12.20 Previous version (0.4.2.1)
- 12.21 Previous version (0.4.1)
- 13 History
- 13.1 Version 0.6.0 - Release 2013-03-14
- 13.1.1 Major changes
- 13.1.2 Third-Party
- 13.1.3 Infrastructure
- 13.1.3.1 Parameters can't be modified in CTestRun after initialization
- 13.1.3.2 Treat timeouts in the MetaServer right
- 13.1.3.3 Add execute-method to ClusterJob
- 13.1.3.4 Add possibility to run specific modules before or after the solver
- 13.1.3.5 Parameters added to the info about the run
- 13.1.3.6 Parameter handling in ClusterJob extended
- 13.1.3.7 Run data written alongside PickledPlots
- 13.1.3.8 BasicRunner collects error and warning texts
- 13.1.4 Library
- 13.1.4.1 TemplateFile now uses pyratemp
- 13.1.4.2 Clearer error message in Application-classes
- 13.1.4.3 Output is only colored if it goes to the terminal
- 13.1.4.4 error-method of application classes now raises an exception
- 13.1.4.5 ParsedParameterFile now knows how to handle binary files
- 13.1.4.6 LabledReSTTable for more flexible table generation
- 13.1.4.7 Plotting classes now allow setting of xlabel
- 13.1.5 Utilities
- 13.1.5.1 pyFoamFromTemplate.py with new templating engine
- 13.1.5.2 pyFoamSamplePlot.py allows using the reference data as basis for comparison
- 13.1.5.3 Scaling and offsets are now used in plots of pyFoamSamplePlot.py
- 13.1.5.4 pyFoamPrintData2DStatistics.py prints relative average error
- 13.1.5.5 Enhancements to pyFoamVersion.py
- 13.1.5.6 pyFoamRunner.py allows hooks
- 13.1.5.7 pyFoamRedoPlots.py supports range for plots
- 13.1.5.8 pyFoamDisplayBlockMesh.py no supports templates
- 13.1.5.9 pyFoamCaseReport.py is tolerant towards binary files
- 13.1.5.10 pyFoamSamplePlot.py and pyFoamTimelinePlot.py raise error if no plots are generated
- 13.1.5.11 pyFoamSurfacePlot.py can wait for a key
- 13.1.5.12 pyFoamEchoDictionary.py is more flexible with binary files
- 13.1.5.13 All utilities now have a switch that starts the debugger even with syntax-errors
- 13.1.5.14 Utilities now can be killed with USR1 and will give a traceback
- 13.1.5.15 Switch to switch on all debug options
- 13.1.5.16 Plotting utilities now allow specification of x-Axis label
- 13.1.5.17 Metrics and compare for pyFoamTimelinePlot.py and pyFoamSamplePlot.py support time ranges
- 13.1.5.18 pyFoamDisplayBlockMesh.py allows graphical selection of blocks and patches
- 13.1.5.19 pyFoamCloneCase.py and pyFoamPackCase.py accept additional parameters
- 13.1.5.20 pyFoamListCases.py now calculates estimated end-times
- 13.1.6 New features
- 13.1.7 Bugfixes
- 13.1.7.1 Only binary packages of 1.x were found
- 13.1.7.2 Option group Regular expressions was listed twice
- 13.1.7.3 --clear-option for pyFoamDecompose.py not working
- 13.1.7.4 pyFoamDisplayBlockmesh.py not working with variable substitution
- 13.1.7.5 Option --function-object-data of pyFoamClearCase.py not working with directories
- 13.1.7.6 nonuniform of length 0 not correctly printed
- 13.1.7.7 Building of pseudocases with pyFoamRunner.py broken
- 13.1.7.8 pyFoamRedoPlot.py did not correctly honor --end and --start
- 13.1.7.9 WriteParameterFile does not preserve the order of additions
- 13.1.7.10 Wrong number of arguments when using TimelinePlot in positions-mode
- 13.1.7.11 ClusterJob uses only metis for decomposition
- 13.1.7.12 pyFoamSamplePlot.py and pyFoamTimelinePlot.py produced no pictures for regions
- 13.1.7.13 Barplots in pyFoamTimelinePlot.py not working if value is a vector
- 13.1.7.14 Mysterious deadlocks while plotting long logfiles
- 13.1.7.15 Scanning linear expressions form the block coupled solver failed
- 13.1.7.16 #include not correctly working with macros in the included file
- 13.1.7.17 Macros not correctly expanded to strings
- 13.1.7.18 pyFoamPackCase.py in the working directory produces 'invisible' tar
- 13.1.7.19 String at the end of a linear solver output makes parsing fail
- 13.1.7.20 Paraview utilities not working with higher Paraview versions
- 13.1.7.21 Camera settings not honored with pyFoamPVSnapshot.py
- 13.2 Version 0.6.1- Release 2013-05-24
- 13.2.1 Bugfixes
- 13.2.2 Utilities
- 13.2.2.1 pyFoamBuildHelper.py now allow more than one action
- 13.2.2.2 Utilities warn if OpenFOAM-version is unset
- 13.2.2.3 pyFoamUpgradeDictionariesTo20.py allows single files
- 13.2.2.4 pyFoamUpgradeDictionariesTo20.py transforms reaction-schemes
- 13.2.2.5 pyFoamUpgradeDictionariesTo20.py transforms thermophysical data
- 13.2.2.6 pyFoamCloneCase now allows creating directory that symlinks to the original
- 13.2.2.7 pyFoamClearCase.py now removes postProcessing and allows removal of additional files
- 13.2.2.8 Improvements to pyFoamVersion.py
- 13.2.2.9 Additional files automatically cloned
- 13.2.2.10 pyFoamDisplayBlockMesh.py uses the same options for template format as pyFoamFromTemplate.py
- 13.2.3 Library
- 13.2.4 New features/Utilities
- 13.3 Version 0.6.2 - 2013-11-03
- 13.3.1 Major changes
- 13.3.2 Incompatibilities
- 13.3.3 Bugfixes
- 13.3.4 Utilities
- 13.3.5 Library
- 13.3.5.1 SolutionDirectory and TimeDirectory are more tolerant
- 13.3.5.2 ClusterJob now handles template files
- 13.3.5.3 Additional parameters in ClusterJob
- 13.3.5.4 Custom data in directory easier accessible
- 13.3.5.5 SolverJob now allows compression of output
- 13.3.5.6 PyFoamApplication-class now allows quick access to data
- 13.3.6 New features/Utilities
- 13.3.7 Enhancements
- 13.3.7.1 pyFoamRedoPlot.py can plot in XKCD-mode
- 13.3.7.2 pyFoamListCases.py now displays disk usage in human readable form
- 13.3.7.3 pyFoamClearCase.py more flexible in selection of data to be removed
- 13.3.7.4 pyFoamFromTemplate.py automatically chooses template and default values
- 13.3.7.5 pyFoamDumpRunDatabaseToCSV.py can disable standard-fields
- 13.3.7.6 pyFoamDumpRunDatabaseToCSV.py prints pandas-object
- 13.3.7.7 Better debugging with ipdb
- 13.3.7.8 Interactive shell after execution for utilities
- 13.3.7.9 Utilities that read quantitative data convert to pandas-data and/or numpy
- 13.3.7.10 Utilities that read quantitative data write Excel files
- 13.3.7.11 Specify additional settings for GnuPlot in customRegexp
- 13.3.7.12 More flexible data specification for pyFoamSamplePlot.py
- 13.3.7.13 pyFoamSamplePlot.py now allows specification of x-range
- 13.4 Release 0.6.3 - 2014-06-23
- 13.5 Release 0.6.4 - 2014-11-24
- 13.6 Release 0.6.5 - 2015-06-23
- 13.7 Release 0.6.6 - 2016-07-15
- 13.8 Release 0.6.7 - 2017-06-04
- 13.9 Release 0.6.8 - 2017-07-06
- 13.10 Release 0.6.9 - 2018-02-25
- 13.1 Version 0.6.0 - Release 2013-03-14
1 Short description
This Python-library can be used to
- analyze the logs produced by OpenFoam-solvers
- execute OpenFoam-solvers and utilities and analyze their output simultaneously
- manipulate the parameter files and the initial-conditions of a run in a non-destructive manner
- plots the residuals of OpenFOAM solvers
1.1 Motivation
This library was developed to control OpenFOAM-simulations with a decent (sorry Perl) scripting language to do parameter-variations and results analysis.
It is an ongoing effort. I add features on an As-Needed basis but am open to suggestions.
1.2 Compatibility
This Library tries to be as conservative as possible in the libraries it uses. This means that it tries to use only libraries that are in the standard Python-distribution. Currently it needs at least version 2.5 of Python but it has been tested on all versions up to 2.7. Python 3 is currently supported (but currently only 3.4 is tested)
For plotting support it needs the Numpy-library. It is recommended to use the version that is supplied by the package manager of the distribution (PyFoam does not require a specific version).
It should run on any system with OpenFOAM and python2.5 (but some features require at least 2.6). Currently it has been tested on
- Linux (various flavours)
- MacOS X
1.3 Other material
This presentation "Happy Foaming with Python" from the 4th Workshop gives a short overview of the things that can be done with PyFoam.
There is another presentation on the "Automatization with pyFoam" from the 5th Workshop in Gothenburg avalaible (the material is available at this link).
The presentation PyFoam - the dark, unknown corners about the tools for quantitative analysis, handling cases with version control and the server mechanism held at the PFAU (Austrian User Meeting) at the TU Wien in March 2012
Complementary there is a complete tutorial on "Particles with pyFoam - The air classification test case" available here.
A presentation about automatic solver testing using PyFoam, a feature that was added in PyFoam 0.5.7, and a file with the data described there (rename file appropriately before using)
The presentation held at the 8th OpenFOAM Workshop gives a nice introduction to pyFoam from a user perspective (the first part. The second is a bit more advanced). The material mentioned is available here
The presentation Building Block for Automatization (a HTML5-browser is required) from the OpenSource CFD International Conference 2013 gives an overview of newer features of 0.6.2.
The Presentation Before and after the case held at the Austrian User Group Metting PFAU 8.0 demonstrates the use of a new utility to set up cases and interaction with IPython-notebooks. The files to reproduce the presentation are available here.
There was a a basic training about swak4foam and PyFoam at the 9th OpenFOAM Workshop which gives an overview over the work-flow. Updated version of this training from the 10th OpenFOAM Workshop which was updated once more at the 11th OpenFOAM Workshop.
At the 10th OpenFOAM Workshop a comprehensive about the pyFoamPrepareCase.py utility was given. An updated version (without cat pictures) was given at the 13th OpenFOAM Workshop
At the 11th OpenFOAM Workshop the presentation PyFoam 4 the lazy gives an overview of some utilities that allow processing simulation results (case file for this presentation is found here)
At the PFAU 14 (Austrian User Meeting) the talk Talking to PyFoam (and it talks back) demonstrated two features that have been in PyFoam for some time: the server-process that allows controlling a PyFoam-controlled OpenFOAM-run over the net and hooks: little Python-programs that are executed at the start and the end of an execution
A training about using the PyFoam library to program automatic workflows was File:BernhardGschaider-OFW15 ProgrammingPyFoamTraining.pdf
Introductory training for swak4Foam and PyFoam at the 12th International OpenFOAM Workshop. This is a completely new training. An updated version of the Training held at the 2019 Workshop in Duisburg. Most recent version File:BernhardGschaider-OFW15 swakPyFoamBasicTraining.pdf. The hopefully last version File:BernhardGschaider-OFW17 swakPyFoamBasicTraining.pdf with pointers to other material
2 Examples
These examples should demonstrate the possible applications of the library. They are enclosed in the source distribution.
2.1 Compact output
This example is a wrapper for solvers. For each time-step only one line is output: the time and the initial residuals of the linear solvers (with the name of the solved variable). The complete output of the solver is written to a log-file. The residuals are written to separate files in a special directory in the simulation-directory (for plotting).
import re,sys from PyFoam.LogAnalysis.LogLineAnalyzer import LogLineAnalyzer from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer from PyFoam.Execution.AnalyzedRunner import AnalyzedRunner class CompactLineAnalyzer(LogLineAnalyzer): def __init__(self): LogLineAnalyzer.__init__(self) self.told="" self.exp=re.compile("^(.+): Solving for (.+), Initial residual = (.+), Final residual = (.+), No Iterations (.+)$") def doAnalysis(self,line): m=self.exp.match(line) if m!=None: name=m.group(2) resid=m.group(3) time=self.getTime() if time!=self.told: self.told=time print "\n t = %6g : " % ( float(time) ), print " %5s: %6e " % (name,float(resid)), sys.stdout.flush() class CompactAnalyzer(BoundingLogAnalyzer): def __init__(self): BoundingLogAnalyzer.__init__(self) self.addAnalyzer("Compact",CompactLineAnalyzer()) run=AnalyzedRunner(CompactAnalyzer(),silent=True) run.start()
2.2 Parameter Variation
This example does 10 runs of the same case. For each case a different velocity at the inlet is set. After the simulation has ended the mass flow (with this utility) and the pressure difference (with this utility) are calculated and written to the file InflowVariationResults. In addition to this the data of the last time-step is saved to a separate directory (for further analysis):
from PyFoam.Execution.ConvergenceRunner import ConvergenceRunner from PyFoam.Execution.UtilityRunner import UtilityRunner from PyFoam.LogAnalysis.BoundingLogAnalyzer import BoundingLogAnalyzer from PyFoam.RunDictionary.SolutionFile import SolutionFile from PyFoam.RunDictionary.SolutionDirectory import SolutionDirectory solver="simpleFoam" case="pitzDaily" pCmd="calcPressureDifference" mCmd="calcMassFlow" dire=SolutionDirectory(case,archive="InletVariation") dire.clearResults() dire.addBackup("PyFoamSolve.logfile") dire.addBackup("PyFoamSolve.analyzed") dire.addBackup("Pressure.analyzed") dire.addBackup("MassFlow.analyzed") sol=SolutionFile(dire.initialDir(),"U") maximum=1. nr=10 f=dire.makeFile("InflowVariationResults") for i in range(nr+1): # Set the boundary condition at the inlet val=(maximum*i)/nr print "Inlet velocity:",val sol.replaceBoundary("inlet","(%f 0 0)" %(val)) # Run the solver run=ConvergenceRunner(BoundingLogAnalyzer(),argv=[solver,".",case],silent=True) run.start() print "Last Time = ",dire.getLast() # Get the pressure difference (Using an external utility) pUtil=UtilityRunner(argv=[pCmd,".",case],silent=True,logname="Pressure") pUtil.add("deltaP","Pressure at .* Difference .*\] (.+)") pUtil.start() deltaP=pUtil.get("deltaP")[0] # Get the mass flow mUtil=UtilityRunner(argv=[mCmd,".",case,"-latestTime"],silent=True,logname="MassFlow") mUtil.add("mass","Flux at (.+?) .*\] (.+)",idNr=1) mUtil.start() massFlow=mUtil.get("mass",ID="outlet")[0] # Archive the results dire.lastToArchive("vel=%g" % (val)) # Clear results dire.clearResults() # Output current stuff print "Vel: ",val,"DeltaP: ",deltaP,"Mass Flow:",massFlow f.writeLine( (val,deltaP,massFlow) ) sol.purgeFile()
2.3 Manipulating dictionaries
Between Version 1.3 and 1.4 the format for the specification of the linear solvers was changed. The following script takes a fvSolution-file in the old format and rewrites it in the equivalent new format (only works for some solvers, but should work for all the tutorials):
#! /usr/bin/python """Given a pre-OpenFOAM-1.4 fvSolution file, this script transforms the solvers section into the equivalen 1.4-format Incomplete, because it doesn't support (B)DCG and GaussSeidl but should work for instance for all the tutorials""" import sys from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile if len(sys.argv)<2: print "I need at least one fvSolution-file to work with" sys.exit(1) for fName in sys.argv[1:]: print "Working on",fName try: f=ParsedParameterFile(fName) sol=f["solvers"] for name,val in sol.iteritems(): if type(name)!=str or type(val)!=tuple or len(val)<3: # this is not an old-school entry continue solver=val[0] tol=val[1] rTol=val[2] if solver=="ICCG": new=( "PCG" , { "preconditioner":"DIC" } ) elif solver=="BICCG": new=( "PBiCG" , { "preconditioner":"DILU" } ) elif solver=="AMG": new=( "GAMG" , { "agglomerator":"faceAreaPair", "nCellsInCoarsestLevel":val[3], "mergeLevels":1, "smoother":"GaussSeidel"} ) else: print "Unsupported solver",solver continue new[1]["tolerance"]=tol new[1]["relTol"]=rTol sol[name]=new except IOError: print "File",fName,"does not exist" except KeyError: print "File",fName,"is not a fvSolution-file" try: f.writeFile() except IOError: print "Can't write file. Content would have been:" print f
Of course 1.4 can read the old format, so this script is not needed. It's only an example.
2.4 Setting boundary conditions for walls
This example sets the boundary condition for all patches whose names contain the string "Wall" to zero-velocity:
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile f=ParsedParameterFile("pitzDaily/0/U") for b in f["boundaryField"]: if "Wall" in b: f["boundaryField"][b]["value"]="uniform (0 0 0)" f["boundaryField"][b]["type"]="fixedValue" f.writeFile()
This can serve as a starting-point for automatically setting up cases where the patches are named after a specific scheme
3 Installation
The easies way to install PyFoam is as root, but it is also possible to do it as a regular user.
3.1 Prerequisites
To install PyFoam you need (obviously) Python.
The smallest Python-version known to work is 2.5 but higher versions are better (not all features might be available with smaller versions). Basically the python that is preinstalled with most Linux-versions currently in use should be fine.
3.1.1 Libraries
PyFoam brings some essential libraries. For plotting and other data handling operations you will need numpy. Use the version provided by the package manager of your distribution.
Some utilities need additional libraries. Once installed you can use the pyFoamVersion.py-utility to find out who they are
If one gets a warning message:
Neither numpy nor Numeric python-package installed. Plotting won't work
that can be fixed by installing the necessary package, by running the respective installation command, depending on your Linux Distribution:
- Debian/Ubuntu:
sudo apt-get install python-numpy
- Fedora/CentOS/RHEL/SL:
sudo yum install numpy
- openSUSE:
sudo zypper install python-numpy
3.2 Installing with pip
This is now the preferred way of installing PyFoam.
PyFoam is now available at the Python Package Index. This means that if pip is installed a single command as root:
pip install PyFoam
or
sudo pip install PyFoam
installs PyFoam on the machine. The library numpy is also installed if it is not yet installed on the machine as it is required for plotting
If you can not write to the global Python installation because you don't have administrative rights try
pip install --user PyFoam
but you'l have to make sure that the location the utilities are installed to are in your PATH
An existing installation can be easily upgraded via pip:
pip install --upgrade PyFoam
3.3 Installation as root
This used to be the old way to install PyFoam but now should only be used in emergency cases. If possible use pip.
- Download the latest tar
- Untar it, e.g.:
tar -xf PyFoam-0.6.6.tar.gz
- Go to the directory PyFoam-0.6.6 (or for whichever version you downloaded):
cd PyFoam-0.6.6
- Install it with the command:
sudo python setup.py install
Should step 4 fail one of the most common causes is that some Linux-distributions (usually Debian-based, don't know about Mandrake) don't install the packages that are necessary for setup.py. Usually they are in the Developer-package which is installed by running the respective installation command, depending on your Linux Distribution:
- Debian/Ubuntu:
sudo apt-get install python-dev python-setuptools
- Fedora/CentOS/RHEL/SL:
sudo yum install python-devel
- openSUSE:
sudo zypper install python-devel
3.4 Installation as a regular user
If you don't want to (or can't) install it as root things are a bit more complicated. Let's assume that you have directory /home/nonroot/my_python to which you have write access:
- Download PyFoam-0.x.x (0.x.x being the current version)
- Untar it, e.g.:
tar -xf PyFoam-0.6.5.tar.gz
- Go to the directory PyFoam-0.6.5 (or for whichever version you downloaded):
cd PyFoam-0.6.5
- Add the path to the library to the PYTHONPATH environment-variable, e.g.:
export PYTHONPATH=$HOME/my_python/lib/python-2.3/site-packages:$PYTHONPATH
- The lib/python-2.3/site-packages portion of the path may depend on your Python-version; in that directory there must be a directory named PyFoam.
- Install it with the command
python setup.py install --prefix=$HOME/my_python
- After installing, in $HOME/my_python there should be two directories that contain the shell scripts and the Python libraries for PyFoam: bin and lib
- If you want to use the scripts distributed with PyFoam extend your path with the following command:
export PATH=$HOME/my_python/bin:$PATH
3.5 Installation via RPMs
Starting with version 0.4.0 a source rpm and a regular rpm are provided for those who prefer to install it that way (The rpms have not been extensively tested. If there is a problem: please contact me)
For recent versions no RPMs were made available. You can produced them from the tarballs with something like:python setup.py bdist
3.6 Obsolete: Installation under gentoo Linux
PyFoam is now in the main portage tree, so you can install it with emerge.
emerge PyFoam
Note: According to the information found at Gentoo this is quite an old version. It seems like the port is unmaintained
3.7 Installation under Debian/Ubuntu
Files to package PyFoam for the Debian packaging system are in the downloaded tarball or repository.
3.8 Testing the installation
To test whether the installation of PyFoam is working start the interactive Python shell with
python
(The shell can be left with Control-D). In that shell type
import PyFoam import PyFoam.FoamInformation print PyFoam.FoamInformation.foamTutorials()
If the last command produces an output similar to
/home/nonroot/OpenFOAM/OpenFOAM-1.2-devel/tutorials
everything is well.
If things don't work check in the Python-shell with the commands
import sys print sys.path
whether the directory in which PyFoam resides is on the path searched by Python.
It can be useful to use the utility pyFoamVersion.py to view the PYTHONPATH, PyFoam version, OpenFOAM version and the installed libraries.
3.9 Docker containers
At this page there are instructions to install Docker containers with preinstalled versions of swak4Foam and PyFoam. These containers are based on publicly available containers of others (so your preferred OpenFOAM-version might not be there) and might not be the newest versions. But they should be fine for testing
4 Installed Utilities
All the utilities described in this section are installed to /usr/bin (if installed as root) and should be in the standard path for execution. They can all be called with the --help-option which gives all the available options and the latest, up-to-date information about the utility.
One recommended usage of these utilities would be to start a simulation with foamJob and display the residuals with pyFoamPlotWatcher.py without interfering with the simulation.
The following descriptions are not necessarily up to date. The text generated with the --help-option is
4.1 Runner-Utilities
These are utilities that start and control OpenFOAM-Runs. Apart from the features described below they start a little XMLRPC-server through which these runs can be controlled (see Network-Utilities).
Most of the utilities can be used to start parallel runs (the number of processors and a host-file have to be provided, depending on the MPI-implementation. The parallel environment is set up automatically).
4.1.1 pyFoamRunner.py
Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information:
- Residuals and other information of the linear solvers
- Execution time
- continuity information
- bounding of variables
4.1.2 pyFoamUtilityRunner.py
Runs a OpenFoam Utility an analyzes the output. Needs a regular expression to look for. The next 3 arguments are the usual OpenFoam arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamUtility.logfile). The Directory PyFoamUtility.analyzed contains a file with the information of the regexp (the pattern groups) at every time-step.
Doesn't start an XMLRPC-Server
4.1.3 pyFoamSteadyRunner.py
Runs an OpenFoam steady solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments) Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). The Directory PyFoamSolver.analyzed contains this information
- Residuals and other information of the linear solvers
- Execution time
- continuity information
- bounding of variables
If the solver has converged (all linear solvers below threshold) it is stopped and the last simulation state is written to disk
4.1.4 pyFoamPlotRunner.py
Runs an OpenFoam solver. Needs the usual 3 arguments (<solver> <directory> <case>) and passes them on (plus additional arguments). Output is sent to stdout and a logfile inside the case directory (PyFoamSolver.logfile). Information that is output as graphs are
- residuals of the linear solver
- continuity
- bounding of variables (if it occurs)
- Courant-number (optional)
- Execution time of a time-step (optional)
- number of iterations of the linear solver (optional)
- arbitrary regular expressions can used to look for solver-specific data
- The groups in the regular expressions (patterns that are in brackets) are plotted
- in the regular expressions %f% can be used as a shortcut for floating-point numbers
- Any number of expressions can be specified and will be plotted in a separate window
- For steady runs it behave like pyFoamSteadyRunner.py
4.1.5 pyFoamMeshUtilityRunner.py
Some utilities that manipulate meshes store the new mesh into the first timestep (so that they don't destroy the original mesh). This can be inconvenient because you have to copy the mesh back to constant for all further steps. This utility
- Runs a mesh-utility
- Moves the resulting mesh from the first time-step to the constant-directory
Be careful: This utility deletes the old mesh. If you don't know what you're doing, it will be gone forever
4.1.6 pyFoamPotentialRunner.py
Runs potentialFoam on a case and writes the results to p and U. Before running the solver it manipulates the dictionaries in system in such a way that potentialFoam runs and afterwards resets them to their old form.
4.1.7 pyFoamRunAtMultipleTimes.py
Runs a utility multiple times using different values for the -time option. Workaround for utilities that only allow a single time-step
4.1.8 pyFoamRunParameterVariation.py
This utility takes a template case and a file specifying the parameter variation and creates cases with the pyFoamPrepareCase.py-engine, runs a solver on these cases and collects the data into a database. The database can then be extracted with pyFoamDumpRunDatabaseToCSV.py
4.2 Utilities for Logfiles
These utilities can be used to analyze the output of an OpenFOAM-solver that has been written to a file
4.2.1 pyFoamPlotWatcher.py
Gets the name of a logfile which is assumed to be the output of a OpenFOAM- solver. Parses the logfile for information about the convergence of the solver and generates gnuplot-graphs. Watches the file until interrupted.
4.2.2 pyFoamStandardLogAnalyzer.py
Analyzes a Log written by foamJob. Needs the name of the Logfile to be analyzed the data is being written to a directory that has the same name with _analyzed appended
4.3 Networking Utilities
Most Runner-Utilities start a Server-thread that makes it possible to contact control an OpenFOAM-run and get information about it. These utilities help to access these servers.
Notes:
- These utilities are beta-qualitiy. They work for my network but have not been tested in other environments
- The address of the pyFoamMetaServer.py is hardcoded into the sources, but can be overwritten using system-wide configuration-files
- pyFoamNetShell.py doesn't need any additional infrastructure. Only the address and the port have to be known
4.3.1 pyFoamNetList.py
This command lists all the OpenFOAM-simulations in your network environment and gives you information about them. Amongst the information is:
- Hostname and port (useful for pyFoamNetShell.py)
- Details about the run (solver, working directory ...)
- Estimates about the time at which the run will end (using the time since the start, current timestep and the endTime)
This utilitiy only works if a pyFoamMetaServer.py can be contacted at the configured address
4.3.1.1 pyFoamMetaServer.py
This program spawns a daemon process that runs forever. It stores information about the running processes. There are two mechanisms for it to get that information
- scanning the network (at the start and on demand)
- Runner-utilities try to register with it, when they start an OpenFOAM-run
- The utilities also dregister their process when they exit gracefully
4.3.2 pyFoamNetShell.py
Connects to an OpenFOAM-run started by a runner-Utility using
- the hostname of the machine it runs on
- the port number (usually 18000, if more than one run is running one of the following ports)
and starts an interactive shell there. On the shells the XMLRPC-commands provided by the server can be used to
- get information about the run
- environment
- start-times
- manipulate the run
- for instance stop() tells the run to write at the next time-step and then end
- kill() brutally terminates the run
Help about the available commands is provided.
Can also be used to control the pyFoamMetaServer.py (which is usually contacted on port 17999)
4.4 Utilities for Manipulating case data
These utilities are used for manipulating case date. They are especially useful in scripts that automatically set up simulation runs.
4.4.1 pyFoamAddEmptyBoundary.py
Adds an empty patch to the boundary file. Such a patch is needed for some mesh-maipulation utilities.
4.4.2 pyFoamChangeBoundaryType.py
Changes the type of a boundary patch.
4.4.3 pyFoamChangeBoundaryName.py
Changes the name of a boundary patch.
4.4.4 pyFoamCreateBoundaryPatches.py
Creates boundary patches in a field-file by looking at the boundary-file
4.4.5 pyFoamClearCase.py
Removes all the timesteps except for the first from a case directory.
4.4.6 pyFoamPackCase.py
Packs the essential files (the ones that are needed to run it) of a case into a tar-file for archiving/mailing-purposes
4.4.7 pyFoamCloneCase.py
Creates a copy of a case with only the most essential files
4.4.8 pyFoamCopyLastToFirst.py
Copy last time-step from one case to the first one of another (making it the initial condition)
4.4.9 pyFoamClearInternalField.py
Clears the solution from the internal field
4.4.10 pyFoamClearBoundaryValue.py
Clear non-uniform values from the boundary-fields
4.4.11 pyFoamInitVCS.py
Initialize the case for the use with the Version Control System (VCS) of your choice
4.4.12 pyFoamSymlinkToFile.py
This utility replaces a symlink with a copy of the file/directories it points to. To be used after a pyFoamCloneCase.py in --symlink-mode
4.4.13 pyFoamCompressCases.py
Goes through a case and compresses single files that are bigger than a certain threshold. Purpose is to shrink cases that were run with the setting uncompressed in the controlDict
4.5 Manipulating dictionaries (from scripts)
For more complex cases these utilities require an understanding of the syntax/semantics of Python-lists and dictionaries
4.5.1 pyFoamReadDictionary.py
Reads data from the root of a OpenFOAM-dictionary and prints it. To access subexpressions of a dictionary entry Python-expressions can be used (note: the expression is evaluated by the Python-interpreter, so quoting has to be used for strings, otherwise Python confuses them with variables)
4.5.2 pyFoamWriteDictionary.py
Writes data to the root of a OpenFOAM-dictionary and writes it to disk. Subexpressions can be selected (see note: pyFoamReadDictionary.py)
4.5.3 pyFoamFromTemplate.py
Generate a dictionary from a template file
4.5.4 pyFoamCompareDictionary.py
Compares two dictionaries structurally (not textually). Useful when the order of the entries or the formating have changed and diff won't give useful results
4.5.5 pyFoamUpdateDictionary.py
Updates a dictionary from another by adding/removing entries that are not in both.
4.5.6 pyFoamUpgradeDictionariesTo17.py
Checks the case and if fvSolution is in the old format rewrites it to the new format
4.5.7 pyFoamUpgradeDictionariesTo20.py
Checks the case and if blockMeshDict and thermophysicalProperties is in the old format tries to rewrite them to the new format. Also applies the changes pyFoamUpgradeDictionariesTo17.py would do
Thes utilities require a paraview that is compiled with python-support. They have been only tested with ParaView 3.4 and may not work with other versions.
4.6.1 pyFoamPVSnapshot.py
Generates snapshots from OpenFOAM-data using a ParaView-state-file that was generated with another, similar case
4.6.2 pyFoamPVLoadState.py
Loads OpenFOAM-data and a Paraview-state-file that was generated with another similar case and lets you continue work like regular paraFoam
4.7 Other
Utilities that don't fit any of the other categories.
4.7.1 pyFoamListCases.py
Gives a ls-like overview of the OpenFOAM-cases in a directory
4.7.2 pyFoamDecompose.py
- Generates a decomposeParDict according to the user specifications and
- runs dcomposePar
This utility is quite useful when writing scripts where the number of available processors is not known beforehand (for instance if you are using a queueing-system)
4.7.3 pyFoamComparator.py
A utilitiy that automatically does a series of variations on a run (for benchmarking, testing and verification purposes). The behaviour is specified by an XML-file.
Work in progress. Therefor no example is present and the DTD that assists editing the XML-file is not included. Contact me, if you want to give it a spin (and want the DTD)
4.7.4 pyFoamVersion.py
Prints the Version of OpenFOAM and pyFoam. Used for information purposes (when reporting bugs). Also lists the Foam-versions installed in the OpenFOAM-directory
4.7.5 pyFoamExecute.py
Execute a command specifying a OpenFOAM-Version. Useful for compiling/executing a debug-version without changing the shell
4.7.6 pyFoamBuildHelper.py
Helper utility to build a project and its prerequisites (OF, libraries etc)
4.7.7 pyFoamDumpConfiguration.py
Dumps the values that can be changed via configuration files. The location of these configuration values are (lower overrides higer):
- Hardcoded in the Library
- Systemwide configuration file: /etc/pyfoamrc
- User configuration file: ~/pyFoam/pyfoamrc
Parts of the output of the utilitiy can be pasted into the configuration files and modified to change the behaviour of PyFoam
4.7.8 pyFoamSamplePlot.py
Creates Gnuplot commands that plot the results of the sample-Utility (sorted by timestep, field etc) or the sample-function object
4.7.9 pyFoamSurfacePlot.py
Uses VTK to plot surfaces generated by the sampleSurface-Utility (sorted by timestep, field etc) or the sample-function object
4.7.10 pyFoamTimelinePlot.py
Creates Gnuplot commands to plot timelines produced by various functionObjects
4.7.11 pyFoamRedoPlot.py
Generates images by either connecting to a running server-process (one that is started for instance by pyFoamRunner.py or by reading a pickle-file (found in the .analyzed-directory)
4.7.12 pyFoamCaseReport.py
Prints small reports about a case. Currently implemnted:
- Table of the boundary conditions
- Boundary conditions on a per-patch basis (as opposed to the files where they are listed per-field)
- Dimensions of the fields
- Internal fields
4.7.13 pyFoamEchoDictionary.py
Just outputs a dictionary. Used to find bugs in the dictionary-parser
4.7.14 pyFoamEchoPickledApplicationData.py
Outputs the data from a pickle file. Either to the terminal or to a pipe
4.7.15 pyFoamCaseBuilder.py
Command line interface to the CaseBuilder-functionality
4.7.16 pyFoamJoinCSV.py
Joins different CSV-files and resamples them if needed
4.7.17 pyFoamConvertToCSV.py
Takes a file with column oriented data and converts it into a CSV
4.7.18 pyFoamSTLUtility.py
Takes a number of STL-files and joins them into one (with unique patch names)
4.7.19 pyFoamPrintData2DStatistics.py
Prints data generated from the Data2DStatistics data structure
4.7.20 pyFoamCreateModuleFile.py
Utility to create files for [1] that help switching between OF-versions
4.7.21 pyFoamBinarySize.py
Calculates the size of the binaries in an OpenFOAM-installation separated by compile-option
4.7.22 pyFoamBlockMeshRewrite.py
Assists the user in rewriting the blockMeshDict by doing simple, but error-prone transformations. Assumes "sensible" formatting: one block/vertex etc per line.
Sub-commands are:
- refine
- refines mesh by multiplying cell numbers in the blocks
- number
- Adds comments with the vertex numbers. Should help the user when editing/modifying the mesh
- stripNumber
- Remove the comments added by number
- mergeVertices
- Adds vertices from other blockMeshes that are not present in the current blockMesh
- renumberVertices
- Take another blockMeshDict, copy over the vertices-section of that mesh and rewrite blocks and patches so that they conform to these vertices. The original vertices have to be a sub-set of the vertices in the other mesh
- normalizePatches
- Rotates patches so that the lowest number is in front
4.8 GUI-Tools
Stuff that is not purely command-line oriented
4.8.1 pyFoamDisplayBlockMesh.py
Graphically displays a blockMesh and helps highlights selected blocks and patches. It is not a blockMesh-editor, but it helps to find errors in the order of vertices etc (editing of the blockMesh is still done in a text editor).
This needs an installation of the VTK with Python-bindings. The ways to get this are very different on various systems. But usually you have to compile it from the sources. Sorry.
Since 0.5.4 this is implemented in PyQT4. If that library is not found it falls back to the old (less feature-rich) version implemented in Tkinter
4.8.2 pyFoamAPoMaFoX.py: A Poor Man's FoamX
TUI (terminal/curses based) interface to the CaseBuilder-functionality (discontinued in PyFoam 0.5.6 as the TUI library was not available for most platforms)
4.8.3 pyFoamAPoMaFoXiiQt.py
Implementation of pyFoamAPoMaFoX.py using PyQt
4.9 Special utilities
Utilities for special applications
4.9.1 pyFoamModifyGGIBoundary.py
Modify GGI boundary condition parameters
5 Library Documentation
The tar-file contains the documentation of the Library as HTML (in the folder doc)
6 Rocks-Cluster support
There is library support to run OpenFOAM on a link Rocks cluster. It has only been tested on that type of cluster and probably has to be extended/adapted for other Clusters/Queuing-systems. There is a good chance that it might work on vanilla SunGridEngine installations, but this has not been tested.
An example cluster script would be
#!/usr/bin/python # #$ -cwd #$ -j y #$ -S /opt/rocks/bin/python #$ -m be # import sys,os from os import path os.environ["WM_64"]="1" from PyFoam.Infrastructure.ClusterJob import SolverJob from PyFoam.Error import error from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile velIn=sys.argv[1] name="pitzDaily" class Pitz(SolverJob): def __init__(self): SolverJob.__init__(self, name+".run."+velIn, "simpleFoam", template=name, steady=True, foamVersion="1.5") def setup(self,parameters): self.foamRun("blockMesh") vel=ParsedParameterFile(path.join(self.casedir(),"0","U")) vel["boundaryField"]["inlet"]["value"] = "uniform (%s 0 0)" % velIn vel.writeFile() Pitz().doIt()
If this script (assume it has the name runPitzDaily.py) is submitted to a Rocks-Cluster with the SunGridEngine installed via
qsub runPitzDaily.py 5
(serial variant) or
qsub -pe mpi 4 runPitzDaily.py 5
the following things happen once the Job gets scheduled:
- it is ensured that version 1.5 of OpenFOAM gets used
- the template-case pitzDaily gets copied to a case pitzDaily.run.5 (automatically)
- blockMesh is run on that case (by the script)
- the inlet velocity is set to 5 (by the script)
- if the case was submitted in parallel then it is decomposed (using the Metis-algorithm) - automatically
- the case is run
- if it was run in parallel it is reconstructed (automatically)
Apart from the setup-method the SolverJob-class has a number of other hooks that allow to manipulate the case before/after decomposition/reconstruction
A script can be tested on the local machine using
pyFoamClusterTester.py runPitzDaily.py 5
or in parallel
pyFoamClusterTester.py --proc=4 runPitzDaily.py 5
The tester-utility does not need the SGE (SunGridEngine)
6.1 Preparation of a user account on the SGE
For the Cluster to be able to find the PyFoam-stuff a user has to create a file .sge_request with the contents
-v PYTHONPATH=/home/common/python
(assuming that the PyFoam-library resides in a directory /home/common/python/PyFoam that is reachable from all cluster nodes)
7 Technical notes/known problems
This is a list of the currently known problems. If you find any other bugs (also known as "surprising features") don't hesitate to contact the author
- if a run-fails the error message might be "eaten" by the utility controlling it. I'm working on this
- the only MPI-environment I currently have available is OpenMPI. The pyFoam-Libraries used to work with LAM (still should, but I can not test it). If you have problems, please contact me (also "success-stories": "it works with MPIch" are most welcome)
7.1 Bug reporting
Bugs can be reported on the Bug-Tracker at openfoam-extend.sourceforge.net.
In reporting a bug, it can be useful to add the information given by the utility pyFoamVersion.py.
8 Additional stuff
Since Version 0.2 the distribution contains the benchFoam.py-script (Since 0.2.4 the benchmark suite is stable enough to be used by the public). From version 0.3.0 on it can be used as pyFoamBench.py
9 Other topics
9.1 Plotting with customRegexp-files
The utilities that do the plotting of time-graphs (pyFoamPlotRunner.py and pyFoamPlotWatcher.py) automatically plot user defined quantities if they find a file customRegexp in the case directory. Since Version 0.4.3 this file may have a more complicated syntax. One line of the file may for instance look like this (connaisseurs will recognize a Python-dictionary):
{"expr":"Reaction Speed: (%f%) min (%f%) max","name":"Reaction speed","titles":["min","max"]}
If the output of the solver has lines that look like this
Reaction Speed: -0.43 min 0.23 max
a plot with the name "Reaction speed" and two data sets ("min" and "max") will be created.
The fields in the dictionary are:
- expr
- This is a regular expression that is looked for in the output of the solver. The string %t% is shorthand for a more complicated expression that fits any real number. Any Regexp-Groups (the stuff between "()") is used as data for plotting. Please note that regular expression need some getting used to. One extra character can completely change the meaning of an expression and cause the expression to fail
- name
- The title of the plot
- titles
- The "legend" of the data lines found during fitting. These names are used in the order in which they are found in the Regexp
9.1.1 New format
Starting with version 0.5.3 a new format for the customRegexp is introduced. It basically looks like a OF-dictionary where each plot is specified by a separate sub-dictionary. The above custom plot can be written as
ReactionSpeed { expr "Reaction Speed: (%f%) min (%f%) max"; name Custom01_Reaction_speed; theTitle "Custom 1 - Reaction speed"; titles ( min max ); type regular; }
customRegexp-files in the old format are still recognized and can be dumped in the new format using the --dump-custom-regegexp-option
It is possible to let the titles be determined by the regular expression (idNr identifies the sub-expression that is the name):
ValuesOfSpecies { expr "Value of species (.+) is (%f%)"; theTitle "Don't know how many species we have"; type dynamic; idNr 1; with steps; }
Instead of plotting data can be appended to a different plot (and will be plotted there):
AddToSpeed { expr "Average reaction speed (%f%)"; titles ( avg ); type slave; master ReactionSpeed; }
9.1.2 Information on regular expressions
If a line contains a special character, one must indicate that for the parser just like in case of a regular expression. Solver output with multiple parentheses and square brackets:
Integral of U = (-0.05 0.02 0.05) [0 4 -1 0 0 0 0]
The matching expression:
expr "Integral of U = \((%f%) (%f%) (%f%)\) \[0 4 -1 0 0 0 0\]";
If in trouble, some nice summaries about regular expressions can be found with this search.
9.2 Settings
Several aspects of pyFoam can be modified using settings-files
The current settings can be viewed with the pyFoamDumpConfiguration.py-command. They are output in the format that is needed in pyfoamrc (so you can paste them in there and modify them) (BTW: the format is basically similar to the INI-files in Windows ca. 3.XX - in the days before the registry. It's primitive, but it's sufficient for our purposes)
The precedence of settings is:
- Hardcoded in the PyFoam-sources
- Systemwide in /etc/pyfoamrc
- User-specific in ~/.pyFoam/pyfoamrc
Highest number wins
Settings can be overwritten for specific OF versions by adding new sections that are of the format MPI-1.7 for instance (entries in that section would overwrite the settings in MPI if the used version of OpenFOAM starts with 1.7 (1.7.x, 1.7 etc)
9.2.1 Modifying the call to mpirun
Different MPI-implementations and sites may need different options to mpirun.
The section of variables that is relevant to this is
[MPI] options_openmpi_post: ["-x","LD_LIBRARY_PATH","-x","WM_PROJECT_DIR","-x","FOAM_MPI_LIBBIN","-x","MPI_B UFFER_SIZE"] options_openmpi_pre: ["--mca","pls_rsh_agent","rsh"]
The _pre-option is the command-line arguments that are added to the mpirun command-line BEFRORE the -np/-machinefie argument the _post-option is for stuff that comes afterward (but before the actual OpenFOAM-command)
The middle-part of the variable name depends on the value of the $WM_MPILIB-variable. So for LAM this would be options_lam_pre/post.
9.3 Version control support
PyFoam adds some facilities to work with cases under version control. The advantage of this is that changes to a case are recorded when experimenting with a case (when using a Distributed VCS this also has some interesting applications for cloning/branching/merging cases). The disadvantage is that huge amounts of disc space are wasted if this is used unwisely.
Currently only support for mercurial is implemented (it was the natural choice as it is mainly implemented in Python). The implementation is generic enough to support other VCS but some low-level support has to be added
The case is initialized for VCS with the pyFoamInitVCS.py-command. This initializes the VCS-repository and adds the most important stuff in the case (the directories 0, constant and system. After that further files can be added using the usual commands of the used VCS. Options of the runner-utilities allow "commit before running"
The contrib-folder of the distribution contains a Mercurial-extension foamdiff. When this extension is installed a command like
hg foamdiff -r 23 system/controlDict
compares different revisions of a file not for textual differences but for semantic differences (using the same mechanisms that pyFoamCompareDictionaries.py uses)
10 paraFoam/paraview 'support'
If you have an installation of paraview that is 'Python-enabled' and PyFoam is found on the PYTHON_PATH then PyFoam can be used in the 'Programmable Filter'/'Programmable Source' of paraview by importing it as usual
import PyFoam
This is especially useful for data that was written by OpenFOAM but is not recognized by the regular importer. It can be loaded as usual with the ParsedParameterFile-class
Please note:
- To visualize the data you need to know how to program VTK
- Be aware, that due to the very general parser of PyFoam large datasets might consume huge amounts of memory and time during loading
10.1 Displaying the gravitation direction
10.1.1 As a Programmable Source
This little example displays the gravitation vector as a line. Just use it as a Programmable Source. It assumes that paraFoam was called in the case directory (otherwise casepath has to be modified)
from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile from PyFoam.Paraview import caseDirectory from os import path import paraview env=ParsedParameterFile(path.join(caseDirectory().constantDir(),"environmentalProperties")) g=env["g"][2] pdo=self.GetPolyDataOutput() pdo.Allocate(1,1) pts=paraview.vtk.vtkPoints() pts.InsertPoint(0,(0.0,0.0,0.0)) pts.InsertPoint(1,g) line=paraview.vtk.vtkPolyLine() line.GetPointIds().SetNumberOfIds(2) line.GetPointIds().SetId(0,0) line.GetPointIds().SetId(1,1) pdo.InsertNextCell(line.GetCellType(), line.GetPointIds()) pdo.SetPoints(pts)
Using the timeDirectory()-function you can program similar Programmable Filters for data that is stored in the time directories
10.1.2 As a script
An alternative is running this script via "Tools -> Python Shell":
from PyFoam.Paraview import readerObject,caseDirectory from PyFoam.Paraview.SimpleSources import Glyph,Arrow from PyFoam.RunDictionary.ParsedParameterFile import ParsedParameterFile from os import path ro=readerObject() env=ParsedParameterFile(path.join(caseDirectory().constantDir(),"environmentalProperties")) g=env["g"][2] # gly=Glyph("Gravity",ro.getCenter(),ro.getCenter()+0.5*g*abs(ro.getExtent())/abs(g)) gly=Arrow("Gravity",ro.getCenter(),ro.getCenter()+0.5*g*abs(ro.getExtent())/abs(g)) gly.repr.Color=(0,0,0)
10.2 Other example scripts
Can be found in the examples/paraview3-directory of the release. These are:
- snappyDisplay
- Displays the most important feature of a snappyHexMeshDict including the bounding-box and the STL-files
- probeDisplay
- displays the probes specified in system/controlDict
These Utilities are quite useful to find mistakes when setting up these files
10.3 General Remarks on the Programming
PyFoam takes care of some of the things described in this document when creating a new object. The two aspects of the object are accessible as
- src
- the actual VTK-object
- repr
- its paraview representation
You can access their properties as described in the document
There are still some issues with the deletion of these objects
- sometimes they are still visible but not deleted
- sometimes paraview crashes after deleting them using the "Delete"-button on the GUI
but your milage may vary
All this has been tested with ParaView 3.4 and may not work with other versions as the Python-API of paraview seems to be in flux
10.4 Necessary changes in the OF-installation to use the paraview-stuff
The etc/apps/paraview3/bashrc must be extended to extend the Python-PATH:
if [ "$PYTHONPATH" ]; then export PYTHONPATH=$PYTHONPATH:$ParaView_DIR/Utilities/VTKPythonWrapping else export PYTHONPATH=$ParaView_DIR/Utilities/VTKPythonWrapping fi export PYTHONPATH=$ParaView_DIR/lib/paraview-3.3/:$PYTHONPATH # export PYTHONPATH=$ParaView_DIR/lib/paraview-3.4/:$PYTHONPATH
For the Version 3.3-cvs that is distributed with 1.5 a little bug has to be fixed: look for the file ~/OpenFOAM/ThirdParty/ParaView3.3-cvs/platforms/linuxGcc/Utilities/VTKPythonWrapping/paraview/servermanager.py and there change the line 1110 from
loader.SetRenderViewXMLName(rvname)
to
loader.SetViewXMLName(rvname)
PyFoam can't do everything (and it doesn't even try) but there are other efforts that help to use OpenFOAM programmed in Python
11.1 pythonFlu
pythonFlu is a wrapper to OpenFOAM that allows the programming of top-level solvers in Python (and I'm sure there will be other great applications that can't be easily done otherwise). More information can be found on the official pythonFlu website.
12 Downloads
12.1 Development version
The current development version can also be fetched using the command
hg clone http://hg.code.sf.net/p/openfoam-extend/PyFoam PyFoam cd PyFoam
This is the current stable version. To get the current development version you also have to
hg update develop
12.1.1 Updating existing working copies
If you downloaded this repository before January 2013 and want to keep this working copy see this article on information how to switch it to the new link.
12.2 Python Package Index
The latest released version is available via the Python Package Index
12.3 Current version (0.6.6)
- the tar file with the sources and documentation (recommended for users)
12.4 Current version (0.6.5)
- the tar file with the sources and documentation (recommended for users)
12.5 Previous version (0.6.4)
- the tar file with the sources and documentation (recommended for users)
12.6 Previous version (0.6.3)
- the tar file with the sources and documentation (recommended for users)
12.7 Previous version (0.6.2)
- the tar file with the sources and documentation (recommended for users)
12.8 Previous version (0.6.1)
- the tar file with the sources and documentation (recommended for users)
12.9 Previous version (0.6.0)
- the tar file with the sources and documentation (recommended for users)
- the tar file with the sources and unit-tests (for developers)
12.10 Previous version (0.5.7)
12.11 Previous version (0.5.6)
12.12 Previous version (0.5.5)
12.13 Previous version (0.5.4)
12.14 Previous version (0.5.3)
12.15 Previous version (0.5.2)
12.16 Previous version (0.5.1)
12.17 Previous version (0.5.0)
12.18 Previous version (0.4.3)
12.18.1 Buggy version (0.4.2)
From this version the Gnuplot-package was missing. Forget it
12.19 Ancient history
12.20 Previous version (0.4.2.1)
12.21 Previous version (0.4.1)
12.21.1 Previous version (0.4.0)
There were no complaints about this version
12.21.2 Very old versions
13 History
- 2005-08-15: Version 0.1
- 2006-01-13: Version 0.2
- 2006-01-16: Version 0.2.1
- 2006-01-16: Version 0.2.2
- 2006-01-23: Version 0.2.3
- 2006-01-23: Version 0.2.4
- 2006-02-03: Version 0.2.5
- Improves the stability of the benchmark-script for parallel runs
- adds benchmakr suite for ancient versions of OpenFOAM
- 2006-02-03: Version 0.2.6
- Minor changes to the benchmark-scripts (other cases than the tutorial cases can now be used for benching)
- 2006-02-23: Version 0.3.0
- live plotting of residuals of solvers
- benchmark script moved to bin-directory
- 2006-03-22: Version 0.3.1
- BugFix: Plotting residuals in interFoam did not work
- Extending the Resiudals-Plotting (Courant-Numbers etc)
- Parsing of Dictionary-files into Python-Dictionaries (not yet fully tested)
- 2006-03-22: Version 0.3.2
- Plotting resiudals now works with OpenFOAM 1.3
- 2006-10-18: Version 0.3.3
- pyFoam distinguishes between the used MPI-implementations when starting parallel runs (currently LAM and OpenMPI)
- BugFixes
- added Utility pyFoamAddEmptyBoundary
- 2006-11-29: Version 0.3.4
- An error pointed out by Andrea Rimondi: the last version was incomplete. Due to a code reorganisation and missing files in the distribution half of the scripts in bin stopped working, but apart from Andrea everyone was too polite to mention it
- added Utility pyFoamClearCase.py
- 2007-03-29: Version 0.4.0
- Reimplementation of the parsing of OpenFOAM-Dictionaries
- Can be manipulated like regular Python-Data-Structures
- Extension of the plotting-Utilities: arbitrary expressions are possible
- The data can also be written to the disk
- Runs that are started by pyFoam can be accessed and controlled over the network (using XMLRPC)
- The runner-Utilities can start parallel-runs
- A utility for generating a dictionary and decomposing a case is added
- Bug removed that didn't let pyFoamSteadyRunner.py work properly with OpenFOAM 1.3 (due to changes in the solver output)
- Reimplementation of the parsing of OpenFOAM-Dictionaries
- 2007-08-29: Version 0.4.1
- Debugging and stabilizing of the parsed OpenFOAM-Dictionaries
- Changes in semantics: numbers are converted to numbers (not strings as before)
- Can read and write gnuzipped files
- More elegant access of the data
- Order of entries in dictionaries is preserved, if possible
- Gnuplot-library enclosed in distribution (to ease installation)
- Enhancement of utilities
- Runner-utilities can choose the Foam-version that should be used
- Plot-Runner automaticaly loads regular expressions for a case
- pyFoamBench now writes CSV-files which can be loaded into a spreadsheet-program
- other stuff
- New utilities
- Creation of boundary conditions
- Cloning and packing cases
- VTK-based utility to display blockMeshes
- Specialized runner for mesh utilities
- Implementation of Unit-Tests for base classes to ensure stability
- Other changes I can't remember
- Debugging and stabilizing of the parsed OpenFOAM-Dictionaries
- 2007-11-24: Version 0.4.2
- A framework to run jobs on a Rocks-cluster was implemented
- Support for cases that have more than one mesh-region (in the library and the utilities)
- Changes in the Library
- the Application-classes have been changed, so that they can be called from other scripts
- the OpenFOAM-version can be changed for the called applications
- Extension of the Parser (Tensors etc)
- Updated Parser Ply to newest version 2.3
- A symbolic link controlDict.foam is automaticaly created (for the Paraview3.x-reader)
- Gnuplot.py-library is adapted to use either numpy or Numeric (whatever is available)
- mpirun can now have configurable parameters
- Changes in Utilities
- the Runner-Utilities now have a --restart-option
- All Utilities that start an OpenFOAM-application now can select the OF-Version with a --foam-version-option
- Possibility to temporarily lower the relaxation factors for steady runs
- Runner-Utilities can report memory usage
- Clone-Case is now securer
- New Utilities
- pyFoamDumpConfiguration.py: Dump the current configuration file for the PyFoam-installation
- pyFoamExecute.py: Execute a command specifying a OpenFOAM-Version
- pyFoamFromTemplate.py: Generate a dictionary from a template file
- pyFoamUpdateDictionary.py: Updates a dictionary from another by adding/removing entries that are not in both
- pyFoamCompareDictionary.py: Compares two dictionieries structurally (not textually)
- pyFoamClusterTester.py: Test pyFoam-Scripts before submitting them to the cluster
- Semantic changes
- Counting of linear iterations now accumulates over various solutions during the same timestep (different numbers in Plots!!)
- Removed Bugs: many
- 2007-12-04: Uploaded a bugfix version (Missing Gnuplot-library)
- 2008-04-21: Version 0.4.3
- Changes in the Library
- Faster parsing of long lists
- Parsing works with unicode
- Better handling of solution-directories
- Changes in Utilities
- Logfiles are now created using the name of the application
- Various extensions of the Comparator-Utility (CSV-files etc)
- Advanced syntax for the customRegexp-files
- New Utilities
- pyFoamCopyLastToFirst.py: Copy last timestep to another case (as initial condition)
- pyFoamPotentialRunner.py: Runs potentialFoam on a case (leaving everything else unchanged)
- pyFoamSamplePlot.py: Generates graphs from data generated by the sampe-Utility
- pyFoamClearInternalField.py: Resets the whole internal field to one value
- pyFoamCaseReport.py: Information about a case (table of boundary conditions)
- includes new Version 1.8 of the Gnuplot-Library
- Removed Bugs: all that were reported (and some more)
- Changes in the Library
- 2008-09-09: Version 0.5.0
- This version detects whether a OF-1.5 or previous is used and acts according to the convention for utilities in that version
- the --foamVersion-option of the utilities is also aware of switches that change the convention
- Plot utilities do Postscript-Hardcopies if requested
- CaseReporter now also reports data about the decomposition
- Other extension of the CaseReporter
- Added an option to the runner-utils that temporarily disabel libs and functions in the cointrolDict (for incompatible utilities)
- Parser extended to handle special dictionary files (files without headers. Chemical reactions)
- added pyFoamEchoDictionary.py
- Various bugfixes (especially in the parser)
- This version detects whether a OF-1.5 or previous is used and acts according to the convention for utilities in that version
- 2008-10-30: Version 0.5.1
- Moved common functionality of the application to common classes
- Options get grouped in the online-help
- Parser is aware of include and parameter substitution
- Profiling of applications enabled
- Example for a Paraview-Source added
- Error with 1.5 and MultiRegion-cases fixed
- Parser fixed so that it can work with patch-names that are generated by snappyHexMesh
- CaseBuiilder-functionality added
- Small TUI added as a demo
- Parser is now Version 2.5 of PLY (20% speed improvement)
- Bug in the potentialRunner-application fixed
- 2009-03-12: Version 0.5.2
- Uses color on the terminal to format the output of certain utilities (pyFoamCompareDictionary.py etc)
- Can be configured
- User can be selected for pyFoamNetList
- Log-File can be surpressed
- The XMLRPC-server is more tolerant if the port is already in use
- Removed incompatibility with python 2.6 (clashes with the with-keyword)
- 2.6 now complains about deprecated Libraries
- CaseBuilder allows the grouping of arguments
- Extended support for Paraview 3.4
- Easy creation of graphics primitives (Spheres, Glyphs etc)
- Automatically finds the case-directory of an OpenFOAM-case loaded in paraFoam
- Allows to chose whether symlinks will be followed when cloning a case
- Added pyFoamPVSnapshot.py and pyFoamPVLoadState.py
- Vector-class for vector-objects read during case-parsing (allows vector-arithmetics)
- Repaired pyFoamPackCase.py
- added pyFoamClearBoundaryValue.py
- added pyFoamListCases.py
- switch of the usage of the XMLRPC-server for certain utilities
- extension of pyFoamClearCase.py
- Enhancements of the dictionary parser
- tries to preserve comments when writing the results to disk
- is now based on PLY 3.1
- the plot-utilities can accumulate in different ways if a match is found multiple times per timestep
- Display of data for cases with >>2000 time-steps is improved (better downsampling)
- pyFoamWatcher.py now can make hardcopies of the graphs
- format of the hard-copies can be choosen
- Uses color on the terminal to format the output of certain utilities (pyFoamCompareDictionary.py etc)
- 2009-11-10: Version 0.5.3
- General new stuff
- Support for the regular-expressions that come with 1.6
- Matplotlib can be used as an additional backend for plotting (this is beta-quality)
- improved format for the customRegexp that looks more 'FOAMlike'. Old files can be dumped in the new format using the --dump-option of the PlotWatcher
- two new plot types in the customRegexp ('dynamic' for getting the data names from the regular expression and 'slave' to add data to another plot instead of plotting itself)
- Additional utilities
- GGI-script by Martin Beaudoin
- pyFoamRunAtMultipleTimes.py after an idea by waterboy (MessageBoard-name)
- Bugfixes
- Amount of files opened by FileCollection is limited (won't hit the OS-limit on the number of open files
- Fixed a hang at the end of a run that occured if no meta-server was present
- FoamServer reports actual IP (not 127.0.0.1)
- Changes
- Additional parameter to the Decomposer-script (Martin Beaudoin)
- Decomposer-script now recognizes scotch as a valid decompositon method and uses it as a default when used with OF-1.6
- Plotter-Utilities write data to file
- Added general options --force-debug --force-opt --force-32 --force-64 to switch to a specific subversion of OF
- Changes that concern the network-infrastructure
- Mesh can be excluded for the Pack-script
- more stable method to handle temporary changes to the controlDict
- for-loops over a PArsedParameter-file are possible
- unit-tests work with the new tutorial-structure in 1.6
- upgraded parser to PLY 3.2
- Created files can be forced to gz
- Various upgrades to the ParsedParameterFile-family
- Utilities write a history to the case-directory
- further adaptions for Python2.6
- Watcher can read compressed logfiles
- Written logs can be compressed on-the-fly
- Clone works with cases that don't have a time-step
- zsh is recognized (for switching OF-versions)
- Known limitations
- Currently does not work with Paraview 3.6 (which comes with OF 1.6)
- General new stuff
- 2010-06-11: Version 0.5.4
- Utilities
- DisplayBlockmesh now converted to PyQT4.
- Major enhancements (built-in editor)
- If no PyQT4 found the old Tkinter-version will be used
- PlotWatcher can be stopped on a file that is no longer updated (Mantis Bug #8)
- Decomposer can treat regions separatly
- Decomposer can be called directly in the case-directory
- Runners can now switch on purgeWrite
- RedoPlot adds titles
- PotentialRunner works with 1.6
- Enhancments to the CaseReporter
- Writes to files
- Report on linear solvers and relaxation
- New utility: pyFoamSurfacePlot.py to generate images from suface-samples
- New utility: pyFoamTimelinePlot.py that plots timelines that were generated by functionObjects. Also can plot Bar-plots
- New utility: pyFoamRedoPlot.py to plot timelines from running processes (requires a server-process - for instance one started by pyFoamRunner) and pickled data from finished runs
- ClearCase tries to remove data from functionObjects, too
- SamplePlot can now write the Gnuplot-commands to a file
- DisplayBlockmesh now converted to PyQT4.
- Library
- General application-class for PyQT4-classes
- TimeLineAnalyzer is more tolerant
- TemplateFile now shows the expression that gives a problem
- numpy now preferred to Numeric
- Pickle-file with the plot-data is written
- Race condition with the pickle-file avoided
- Configuration can now overwrite parameters depending on the OF-version
- Configuration now allows directories with configuration snipplets (profile.d-style)
- local configuration in a case-directory possible
- Regular expression for "Time =" can now be configured
- Paraview-stuff now accepts the native reader
- SolutionDirectory automatically creates a .foam-stub-file for the native reader
- FoamInformation more tolerant if no OF-installation is found
- Packaging
- Basic framework for debian-packaging is there (only in SVN)
- Utilities
... and a number of other enhancements
- 2011-04-06 Version 0.5.5
- Utilities
- CaseReporter now generates restructured text (this allows easy generation of PDFs and HTML using the right tools)
- CaseReporter can now work with regions
- Paraview-Utilities now know th native reader in Paraview 3.8
- PyFoamChangeBoundaryName by Martin Beaudoin added
- CloneCase now knows how to handle parallel runs
- SamplePlot and TimelinePlot now can plot reference data
- In the Runners the writing of data files now MUST be switched on
- ListCases reports the state-files written by BasicRunner (if present)
- ListCases has additional reporting options
- CompareDictionary can now compare more than one dictionary at once
- Option --autosense for the Runner-utilities now automatically determines the number of processors to use
- SamplePlot, TimelinePlot and RedoPlot now can write CSV-Files
- PyQT-variant of 'A Poor Mans FoamX' added to the distro
- Decomposer now can handle parMetis
- Utility pyFoamInitVCS.py added
- --current uses the current OF-version (when switching to Debug) in all utilties that support version-switching
- CloneCase now uses the VCS for cloning if the case is under version control
- TimelinePlot now allows the usage of the time of the reference data for scaling
- TimelinePlot and SamplePlot now can compare the data to the reference data quantitatively
- Utility JoinCSV to join different CSV-files (resamples data if needed)
- Runner-Utilites now can commit if the case is under VCS
- Utility ConvertToCSV added
- RunAtMultipleTimes now also works for parallel cases
- customRegexp
- New entry progress now allows addition of custom values to the progress output of runners (for instance "h: $0" for addition of the first value found)
- Plots now can be selected from the command line
- All Utilities have an option to switch on/off the stack-trace in the event of an error
- Library
- Gnuplot-Library bumped to 1.8
- PLY updated to 3.3
- Improved handling of Macros in the Dictionary Parser
- Paraview-Sublibrary adapted to Paraview 3.8
- Unittests now can be done using the nose-library
- BasicRunner now writes some state-files
- New SpreadsheetData-class generalizes the handling of data that will go into CSV-files
- Enhancements to the CaseBuilder-backend
- error-function now throws an Exception (this allows graceful error-handling)
- Parser now knows that he can't handle binary files
- Bugfixes
- WriteDictionary now can write negative values
- BlockMeshes without edges not correctly parsed
- If more than one triggered restore was accessing the file it was not correctly restored
- Other enhancements
- In the configuration files features can be switched depening on the version. For instance the section MPI-1.7 overrides everything specified in MPI if the OpenFOAM-version starts with 1.7
- File LocalConfigPyFoam in a case can be used to override the global settings for this case
- Automatic reconstruction can be switched off for ClusterJobs
- ClusterJobs can now also live with alread reconstructed Jobs
- Addition of a Mercurial-extension foamdiff that can show changes between revisions like CompareDictionary does
- Installed OpenFOAM-versions now can be looked for in more than one directory
- Utilities
- 2011-07-26 Version 0.5.6
- Changes to the parser
- Now can parse the codestreams introduced in OpenFOAM 2.0
- ParsedBlockMeshDict understands the new format
- Utilities
- pyFoamUpgradeDictionaries17.py and pyFoamUpgradeDictionaries20.py to automatically upgrade selected dictionaries (fvSolution, blockMeshDict, thermoPhysicalProperties) to new formats
- Internal changes
- Direct calls to shell commands for copying and removing files now replaced by Python-library calls
- Bugfixes
- pyFoamCaseReporter can now handle all known formats of fvSolution/solvers
- Fixed a problem where the Library was still using the old OF-calling convention when starting parallel runs
- Mercurial extension foamdiff
- Now works with Mercurial 1.9 (new version untested with older Mercurial-version)
- Changes to the parser
- 2012-04-13 Version 0.5.7
- Parser improvements
- Problem with nested comments
- Parse code streams
- Preserving of comments can be switched off
- Ignore extra semicolons
- Allows parsing lists of length 3 and 9 as lists and not as vectors and tensors
- "lookup redirection" in OF-dictionaries now works
- Utility improvements
- pyFoamSamplePlot.py stops if the same directory is compared
- pyFoamSamplePlot.py shows the location of the biggest difference
- pyFoamSamplePlot.py allows only same ranges for comparisons
- Generation of pickled-files can be switched of for runner-utilities
- Frequency with which the pickled file is written is adapted (so that it doesn't use ALL the CPU-time)
- pyFoamVersion.py improved (Version of Python is checked etc)
- pyFoamRedoPlot.py: fixed parameter checking
- pyFoamPotentialRunner.py: temporarily disable libs and functions
- Only write last N loglines from Runner-utility
- pyFoamClearCase.py: allow local specification of additional files that should be cleared
- Runner utilities can report data about the run
- pyFoamConvertToCSV.py now allows selection of columns
- Utilities for quantative analysis now can return data
- Utilities for quantative now correctly return data for multiple places
- pyFoamSamplePlot.py now allows specification of valid variable pre and postfixes to allow correct parsing of variable names with a _
- endTime can be specified by the runner-utilities
- Utilities now allow piping (using pickled data)
- pyFoamSamplePlot.py now allows the specification of a reference time
- Nomenclature of pyFoamSamplePlot.py and pyFoamTimelinePlots.py now similar (both call it fields)
- pyFoamDecompose.py now can use the -region-option if the OF-version is right
- Forcing debug-mode fixed for older OF-versions
- pyFoamDecomposer.py now accepts globalFaceZones in Python or OpenFOAM-syntax
- Plot-utilities now don't interpret _ in names not as LaTeX
- New Utilities
- pyFoamEchoPickledApplicationData to output pickled data
- pyFoamPrintData2DStatistics.py to output data from comparisons
- pyFoamBuildHelper.py to build project and its prerequisites (work in progress)
- pyFoamCreateModuleFile.py to create files for http://modules.sourceforge.net/ (by Martin Beaudoin)
- pyFoamSTLUtility.py to join STL-files
- Library improvements
- stabler comparisons
- Paraview-Utilities work with 1.x and 2.x
- Runner classes return a dictionary with data
- TimelineDirectory ignores dot-files
- Application-objects can now be used like dictionaries to access data
- New class TableData for simple data tables
- New class Data2DStatistics for statistics about tables
- new class CTestRun as basis for automated testing
- FoamOptionParser now resets environment variables so that application-classes can call other application classes
- Improvements to HgInterface
- Additional VCS-subclasses for git, svn and svk (implementation only as needed)
- SolutionDir now uses 0.org as initial directory if no valid initial directory is present (this affects clearing and cloning)
- FoamFileGenerator now more flexible for long lists
- ParsedBlockMeshDict now doesn't introduce prefixes for 'long' lists
- Removed utilities
- pyFoamAPoMaFoX.py
- pyFoamPlotResiduals.py
- Thirdparty
- Got rid of Numeric-support in Gnuplot-library
- Other
- script to generate man-pages for the utilities
- Paraview3-example probeDisplay.py now renamed to probeAndSetDisplay.py and reads sampledSets from controlDict and sampleDict
- Parser improvements
13.1 Version 0.6.0 - Release 2013-03-14
13.1.1 Major changes
13.1.1.1 Adaption to work with Python3
Sources are adapted so that PyFoam works with Python3 too. This breaks support for Python 2.4 and earlier (possibly also Python 2.5)
Some of the Libraries in PyFoam.ThirdParty had to be adapted to work with Python3:
- Gnuplot.py
- The original version 1.8 is quite old. It was adapted with the help of the six-library (see below) to work with Python2 and Python3 (inspired by [2] which is a pure port to Python3 without backwards compatibility)
13.1.1.2 New ThirdParty-Libraries
- six
- Library that helps supporting Python 2 and Python 3 in the same source code. Currently version 1.2 from [3] is used
- pyratemp
- Templating library to support the new templating format. Version 0.2.0 from [4] is used
13.1.1.3 Porting to Windows
Port to allow running PyFoam on Windows was done by Bruno Santos of blueCAPE (bruno.santos@bluecape.com.pt)
Patch was originally posted at http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166
Note: many of PyFoam's features are not yet fully functional on Windows.
13.1.1.4 Experimental port to pypy
Sources are executed in pypy but it seems there are problems with numpy and also with code like for l in open(f).readlines()
13.1.2 Third-Party
13.1.2.1 Upgraded ply to 3.4
This brings virtually no changes. README with copyright information has been added
13.1.3 Infrastructure
13.1.3.1 Parameters can't be modified in CTestRun after initialization
This should help to avoid side-effects
13.1.3.2 Treat timeouts in the MetaServer right
Due to a previous workaround timeouts when collecting information about new machines was not treated correctly
13.1.3.3 Add execute-method to ClusterJob
This allows the execution of a shell-script in the directory of the case
13.1.3.4 Add possibility to run specific modules before or after the solver
These modules are found in PyFoam.Infrastructure.RunHooks. Two concrete implementations:
- PrintMessageHook
- to print a text to the terminal
- SendToWebservice
- encode an URL and send it to a webservice (example for pushover.net added)
Hooks are automatically instantiated from the configuration data (examples are hardcoded))
13.1.3.5 Parameters added to the info about the run
The Runner-classes now have a parameter parameters. This data (usually it would be a dictionary) is added verbatim to the run info.
Most runner applications now have the possibility to add this info.
Purpose of this facility is to identify different runs in the database better.
13.1.3.6 Parameter handling in ClusterJob extended
Parameter values are now handed to the actual job. Also a dictionary with parameters can be handed to the constructor and will be used in the relevant callbacks
13.1.3.7 Run data written alongside PickledPlots
During the run whenever the PickledPlots-file is written a file pickledUnfinishedData gets written. This has the current solver data and is similar to pickledData.
Also a file pickledStartData gets written that has the data that is available at the start of the run.
13.1.3.8 BasicRunner collects error and warning texts
The runner collects
- at every warning the next 20 lines of the output until a total of 500 lines is reached (this avoids filling disk and memory if the solver produces too many warnings)
- All output from an error message until the end
And stores them in the application data
13.1.4 Library
13.1.4.1 TemplateFile now uses pyratemp
The class TempalteFile now uses an enhanced templating engine. The old implementation is in the class TemplateFileOldFormat
13.1.4.2 Clearer error message in Application-classes
If used as classes (not as utilities) these classes print the class name instead of the calling utilities name
13.1.4.3 Output is only colored if it goes to the terminal
Error and warning messages don't decorate the output if it goes to files or other non-terminal streams
13.1.4.4 error-method of application classes now raises an exception
An exception is now raised by self.error(). This makes it easier to handle such errors if the application class is used. The exception is passed up until there is a "real" application
13.1.4.5 ParsedParameterFile now knows how to handle binary files
When the format of a file is binary lists with a length prefix are being read as binary blobs.
For reading the blobs a simple heuristics is used: a multiple of the length in bytes is read. If the next character is a ) and the characters after that are a certain combination of characters (newlines and ;) then it is assumed that the blob has ended. This may fail on certain occasions:
- if the combination of characters appears at these places
- if the objects inside the binary data are of different sizes
It would be hard to work around these restrictions without reprogramming the full functionality of OpenFOAM
13.1.4.6 LabledReSTTable for more flexible table generation
New class in the RestructuredTextHelper allows more flexible generation of tables. Items are added with column and row and if these don't exist in the first row/column the table is extended appropriately
13.1.4.7 Plotting classes now allow setting of xlabel
This is implemented for Gnuplot and Matplotlib. Default for the label on the x-Axis is now "Time [s]"
13.1.5 Utilities
13.1.5.1 pyFoamFromTemplate.py with new templating engine
The utility can now use the pyratemp-templating engine which allows templates with loops, conditions and other fancy stuff
13.1.5.2 pyFoamSamplePlot.py allows using the reference data as basis for comparison
Instead of using the x-values from the original data the y-values of the reference data can be used for comparing (with the --use-reference-option)
Same for pyFoamTimelimePlot.py
13.1.5.3 Scaling and offsets are now used in plots of pyFoamSamplePlot.py
If scales not equal to \(1\) and offsets not equal to \(0\) are specified they are used in the gnuplot-output
13.1.5.4 pyFoamPrintData2DStatistics.py prints relative average error
With the --relative-average-error-option
13.1.5.5 Enhancements to pyFoamVersion.py
- More tolerant if no library was found
- Reports the location of the PyFoam-Library
- Checks whether utility version is consistent the library found
13.1.5.6 pyFoamRunner.py allows hooks
Hooks can be added at the start and the end of a run
13.1.5.7 pyFoamRedoPlots.py supports range for plots
Added -end and -start-option to select a range that should be plotted.
Currently not working with the Matplotlib-implementation (only gnuplot)
13.1.5.8 pyFoamDisplayBlockMesh.py no supports templates
If a file with values is specified then the utility assumes you're editing a template file and will evaluate it before displaying it
13.1.5.9 pyFoamCaseReport.py is tolerant towards binary files
New switch that makes the parser treat files that are declared binary in the header as if they were ascii
13.1.5.10 pyFoamSamplePlot.py and pyFoamTimelinePlot.py raise error if no plots are generated
This makes it easier to catch faulty specifications (or empty timeline-files)
13.1.5.11 pyFoamSurfacePlot.py can wait for a key
An option --wait has been added that makes the utility wait before displaying the next picture
13.1.5.12 pyFoamEchoDictionary.py is more flexible with binary files
Switch allows forcing it to read a binary File as an ASCII
13.1.5.13 All utilities now have a switch that starts the debugger even with syntax-errors
Previously the option --interactive-debug only started the debugger if the error was no syntax error. This is still the default behavior, but can be overruled
13.1.5.14 Utilities now can be killed with USR1 and will give a traceback
The option --catch-USR1-signal now installs a signal-handler that prints a traceback and finishes the run. If the interactive debugger is enabled then it goes to the debugger-shell.
Option --keyboard-interrupt-trace triggers the same behaviour for keyboard interrupts with <Ctrl>-C
13.1.5.15 Switch to switch on all debug options
For the purpose of developing a switch --i-am-a-developer has been added.
13.1.5.16 Plotting utilities now allow specification of x-Axis label
With the option xlabel in the customRegexp-file the label on the x-axis of the plot can be changed. Setting ylabel and y2label (for the secondary axis) was already possible
13.1.5.17 Metrics and compare for pyFoamTimelinePlot.py and pyFoamSamplePlot.py support time ranges
Now the options --min-time and --max-time are supported by --metrics and --compare
13.1.5.18 pyFoamDisplayBlockMesh.py allows graphical selection of blocks and patches
New addition by Marc Immer allows the graphical selection of blocks and patches and adds them to the blockMeshDict
13.1.5.19 pyFoamCloneCase.py and pyFoamPackCase.py accept additional parameters
The file LocalConfigPyFoam is read by these utilities and if there is a parameter addItem in the section Cloning defined then these files are cloned/packed automatically (no user specification required)
13.1.5.20 pyFoamListCases.py now calculates estimated end-times
Additional option to print the estimated end times. These can be wrong if the case did not start from the startTime in the controlDict.
Also now allows printing the end and the start-time according to the controlDict
13.1.6 New features
13.1.6.1 Different "phases" for multi-region solvers
Plots of type phase in customRegexp don't actually plot anything. The set a phase-name that is used for subsequent values (for instance to distinguish the different residuals)
13.1.6.2 pyFoamChangeBoundaryType.py allows selection of region and time
Options --region and --time-directory added that allow selecting different boundary-files
13.1.6.3 New class for storing case data in a sqlite-database and associated utilities
The class RunDatabase stores the data from runs. Utility pyFoamAddCaseDataToDatabase.py is one way to populate the database. pyFoamDumpRunDatabaseToCSV.py allows dumping that data to a file for further processing (in a spreadsheet for instance)
Database can also be populated using a special post-run hook
13.1.7 Bugfixes
13.1.7.1 Only binary packages of 1.x were found
Pattern had to start with 1 (now every digit is possible))
13.1.7.2 Option group Regular expressions was listed twice
No harm done. But fixed
13.1.7.3 --clear-option for pyFoamDecompose.py not working
Reason was that rmtree does not allow wildcards. Fixed
13.1.7.4 pyFoamDisplayBlockmesh.py not working with variable substitution
The DictRedirect would not convert to float. Fixed. Although it might happen again for other data types
13.1.7.5 Option --function-object-data of pyFoamClearCase.py not working with directories
The option was only implemented for the list-form of the functions entry in controlDict
Now fixed to also work with the dictionary-form
13.1.7.6 nonuniform of length 0 not correctly printed
Seems like the length was interpreted as the name of the list. Fixed
13.1.7.7 Building of pseudocases with pyFoamRunner.py broken
Only worked if no region was specified (= not at all). Fixed
13.1.7.8 pyFoamRedoPlot.py did not correctly honor --end and --start
Plots were over the whole data range. This is now fix (also the issue that --end alone did not work)
13.1.7.9 WriteParameterFile does not preserve the order of additions
Contents was "only" set as a dictionary which does not preserve the order in which entries are added. Replaced with a DictProxy
13.1.7.10 Wrong number of arguments when using TimelinePlot in positions-mode
Problem that was introduced by changes in the fields-mode
13.1.7.11 ClusterJob uses only metis for decomposition
For OpenFOAM-versions 1.6 and higher the automatic decomposition used is now scotch
13.1.7.12 pyFoamSamplePlot.py and pyFoamTimelinePlot.py produced no pictures for regions
As regions have their own subdirectories the / from the directory name was inserted into the filename and if the subdirectory did not exist gnuplot did not create the picture
13.1.7.13 Barplots in pyFoamTimelinePlot.py not working if value is a vector
The base class didn't correctly handle the ( and ). Fixed
13.1.7.14 Mysterious deadlocks while plotting long logfiles
The problem was that during splitting the timeline data an exception was raised. This exception was caught by another part of PyFoam. This left a lock on the data structure locked and the next access to the structure was held indefinitely. Fixed
13.1.7.15 Scanning linear expressions form the block coupled solver failed
As there is a tuple of residuals the scanner did not analyze the output of the output of the block-coupled solver from 1.6-ext correctly. This is now treated as a special case and each residual is plotted separately (distinguished by a [x] with x being the number of the component)
13.1.7.16 #include not correctly working with macros in the included file
Macros $var were not correctly expanded. Fixed
13.1.7.17 Macros not correctly expanded to strings
When being expanded to string form macros were not correctly expanded
13.1.7.18 pyFoamPackCase.py in the working directory produces 'invisible' tar
If the utility was used in the form
pyFoamPackCase.py .
then an 'invisible' tar ..tgz was produced. Fixed
13.1.7.19 String at the end of a linear solver output makes parsing fail
Reported in [5] the string is assumed to be part of the iteration number. Fixed
13.1.7.20 Paraview utilities not working with higher Paraview versions
At least for PV 3.14 and 3.98 the way the version number is determined has changed and the PV-utilities failed. This has been fixed but is untested with old versions
13.1.7.21 Camera settings not honored with pyFoamPVSnapshot.py
For the first rendered view Paraview automatically resets the camera. This has now been switched off (so the snapshot is rendered correctly)
13.2 Version 0.6.1- Release 2013-05-24
13.2.1 Bugfixes
13.2.1.1 Restoring of controlDict after write
When activating an on-demand write the constrolDict was not restored because the output-line about the file being read was not recognized (due to a change in the output in recent OF-versions). Now a number of different formats is recognized
13.2.1.2 Custom-plot type slave not working if no master defined
That plot-type needs a master. Fixed to fail if none is defined
13.2.1.3 -list-only did not correctly parse lists with a numeric prefix
This did affect all utilities that use that option and also calls with listOnly to the library class
13.2.2 Utilities
13.2.2.1 pyFoamBuildHelper.py now allow more than one action
If multiple actions like --update and --build are specified they are executed in a sensible order (update before build etc)
13.2.2.2 Utilities warn if OpenFOAM-version is unset
If the environment variable that determines the OpenFOAM-version is unset a warning is issued by the utilities
13.2.2.3 pyFoamUpgradeDictionariesTo20.py allows single files
If single file is specified then the action to transform it has can be specified
13.2.2.4 pyFoamUpgradeDictionariesTo20.py transforms reaction-schemes
Now knows how to transform "old" reaction files (where the reactions-entry was a list) to the new format (where it is a dictionary). Only a limited number of reaction types is supported.
13.2.2.5 pyFoamUpgradeDictionariesTo20.py transforms thermophysical data
Now the old form of thermophysical data (lists) is transformed into the new dictionary-form
13.2.2.6 pyFoamCloneCase now allows creating directory that symlinks to the original
Now with the option --symlink-mode instead of copying the directories from the original new directories art created and populated with symlinks to the files in the original. The depth until which no symlinks to directories are created can be specified. This allows the clone to share the configuration files with the original
13.2.2.7 pyFoamClearCase.py now removes postProcessing and allows removal of additional files
The directory postProcessing is now automatically removed (can be switched off with --keep-postprocessing). Also with the --additional-option patterns with additional files to remove can be specified.
13.2.2.8 Improvements to pyFoamVersion.py
- Now reports the location of the python-executable
- Reports locations of used libraries
13.2.2.9 Additional files automatically cloned
The files Allrun, Allclean and 0.org are automatically added during cloning as these are often used by the standard-utilities
13.2.2.10 pyFoamDisplayBlockMesh.py uses the same options for template format as pyFoamFromTemplate.py
This makes sure that templates are handled consistently and also allows different delimiters in the blockMeshDict.template
13.2.3 Library
13.2.3.1 Improvements in syntax of ParsedParameterFile
Now the new relative scoping that was introduced in OF 2.2 is supported
13.2.3.2 Utilities-class now function to find files matching a pattern
Added a function find that approxiamtes the find-command
13.2.3.3 VCS ignores more files
Some more patterns have been added that will be ignored in a VSC-controlled case. All of them concerning files that PyFoam creates during operation
13.2.4 New features/Utilities
13.2.4.1 New Utility pyFoamSymlinkToFile.py
This utility replaces a symlink with a copy of the file/directories it points to. To be used after a pyFoamCloneCase.py in --symlink-mode
13.3 Version 0.6.2 - 2013-11-03
13.3.1 Major changes
13.3.1.1 Use of pandas-library
Starting with this version the pandas-library is used fordata-analysis. When possible and appropriate classes returnpandas-objects. Currently these are:* CSVCollection. With a call-operator this class returns the collected data as a DataFrame of the collected data* SpreadsheetData now has methods to return Series and DataFrame objectsIt is not necessary to install pandas if these classes are notused (and even then most of their functionality works)
13.3.2 Incompatibilities
13.3.2.1 Different separator for databases in CSV-files
The class RunDatabase (and therefor also the utilitypyFoamDumpRunDatabaseToCSV.py) now write as a separator for datafrom sub-tables a // instead of the space. This especially meansthat scripts that rely on a data-item foo in analyzed mightbreak because this is now called analyzed//foo instead of=analyzed foo=. On the other hand this makes the names moreconsistent and easier to parse as // is the saperator for otherlevels of dictionaries
13.3.2.2 Change of independent variable name in sample data
Instead of col0 this is now coord. This could cause problemswith scripts that use that column name in the resultingSpreadsheetData-object
13.3.3 Bugfixes
13.3.3.1 pyFoamPackCase.py does not handle symbolic links correctly
Symbolic links were copied as is and did not work correctlyafterwards. This is fixed. If the symbolic link is an absolutepath or points outside the case directory it is replaced with thefile it points to. Otherwise it is preserved as a symbolic link
13.3.3.2 pyFoamPotentialRunner.py not working with OpenFOAM 2.0 or newer
These versions require an entry potentialFlow in thefvSolution-file instead of the old SIMPLE
13.3.3.3 pyFoamListCase.py fails with controlDict that use preprocessing
Fixed by first trying to read that with preprocessing. Without ifthat fails
13.3.3.4 Cloning fails in symlink-mode if files are specified twice
Now using a set instead of a list makes sure that no file iscloned twice
13.3.4 Utilities
13.3.4.1 pyFoamPotentialRunner.py now allows removing of functions and libs
The utility now allows removing these entries in case that theydon't work with potentialFoam
13.3.4.2 The Runner-utilities now have more options for clearing
Some of the options of pyFoamClearCase.py for clearing cases(for instance specifying additional files) have been ported to theRunner-utilities. Also is the postProcessing-directoryremoved by default
13.3.5 Library
13.3.5.1 SolutionDirectory and TimeDirectory are more tolerant
If there are field files and their zipped counterpart thaninstead of an error a warning *can* be given
13.3.5.2 ClusterJob now handles template files
A new method templateFile gets the name of a file which isconstructed from a template of the same name plus the extension.template
13.3.5.3 Additional parameters in ClusterJob
The method additionalParameters can return a dictionary withadditional parameters
13.3.5.4 Custom data in directory easier accessible
In the written data in the sub-dictionary analyzed there is nowa subdictionary Custom with the values of the custom expressionswith the prefix CustomXX_ removed. This means that values thatwere available as data['Custom02_velCheck']['min']are now available as data['Custom']['velCheck']['min']The advantage is that the number part which was dependent on theorder the expressions were specified is now no longer necessary(this should make scripts more stable)The old notation is still available but deprecated
13.3.5.5 SolverJob now allows compression of output
The parameter solverLogCompress compresses the log-file whilewriting it to disc. *Attention:* This may lead to corruptedlog-files if the run crashes
13.3.5.6 PyFoamApplication-class now allows quick access to data
The dictionary returned by getData() now allows access to allthe elements as attributes.
13.3.6 New features/Utilities
13.3.6.1 Post-run hook that sends mail at the end of run
The hook-module MailToAddress sends a mail at the end of arun. Prerequisite is an SMTP-Server that doesn't needauthentication
13.3.6.2 New utility pyFoamCompressCases.py
This utility goes through cases and compresses single files. Thecases can be searched recursively to.Purpose of this utility is to shrink cases wherewriteCompression was not turned on during the run
13.3.6.3 Paraview-module to read additional data
A new module PyFoam.Paraview.Data reads additional data usuallywritten by OpenFOAM. These are converted to vtkArray using thefollowing functions and can be used in =Programmable filters=:- setSampleData :: reads the data from sampled sets- setTimelineData :: reads data from a timeline directory- setPlotData :: reads pickled plot data using RedoPlot
13.3.7 Enhancements
13.3.7.1 pyFoamRedoPlot.py can plot in XKCD-mode
When used with the option --implementationxkcd= and version ofmatplotlib that supports it is installed then plots are done inthe style of the webcomics [[6]]
13.3.7.2 pyFoamListCases.py now displays disk usage in human readable form
If the disk usage of the cases is calculated then it is displayedin human readable form (as KB, MB, GB or TB) for sizes larger thanone Kilobyte
13.3.7.3 pyFoamClearCase.py more flexible in selection of data to be removed
Options to be more flexible in removing data are added:; keep-interval : keep timesteps at a specified interval. For instance --keep-interval0.1= will keep times like $1$, $1.1$ etc but remove $1.05$; keep-parallel : this will not remove any times in the processor-directories. Also are things like keep-last now honored for processor directories; remove-analyzed : Remove the directories with the analyzed datatoo. Old behavior was to remove them. Now they are kept by default
13.3.7.4 pyFoamFromTemplate.py automatically chooses template and default values
If an output file foo is specified and no template then theutility looks for a file foo.template as a template.If a file foo.defaults is present then this file is read andused as default parameter values. Other specifications overridethese defaults
13.3.7.5 pyFoamDumpRunDatabaseToCSV.py can disable standard-fields
Additional option --disable-run-data
13.3.7.6 pyFoamDumpRunDatabaseToCSV.py prints pandas-object
With the -pandas-print-option a DataFrame is generated andprinted
13.3.7.7 Better debugging with ipdb
If the ipdb-package (basically pdb with IPython-additions)is installed then it is used. This gives additions liketab-completion
13.3.7.8 Interactive shell after execution for utilities
The option --interactive-after-execution drops the user to aninteractive shell where the namespace can be inspected. If presentIPython will be used, otherwise the regular shell is used
13.3.7.9 Utilities that read quantitative data convert to pandas-data and/or numpy
This is mainly to be used on the interactive shell to do furtheranalysis or write this data out. The utilities are:; pyDumpRunDatabaseToCSV.py : add an item dump with the wholedata as a DataFrame; pyFoamTimelinePlot.py : add element series with all the dataas Series and dataFrame with the same data as a DataFrame; pyFoamSamplePlot.py : Like pyFoamTimelinePlot.py; pyFoamRedoPlot.py : Now can get series and the whole plot dataas pandas-objects
13.3.7.10 Utilities that read quantitative data write Excel files
The utilities pyDumpRunDatabaseToCSV.py,pyFoamTimelinePlot.py, pyFoamSamplePlot.py andpyFoamRedoPlot.py now have options to write Excel-files
13.3.7.11 Specify additional settings for GnuPlot in customRegexp
If an item in customRegexp has an item gnuplotCommands thenit is assumed that this is a list of strings which are executedbefore the first plotting. For instance gnuplotCommands ( "set format y '%.2e'" );changes the number format on the y-axis
13.3.7.12 More flexible data specification for pyFoamSamplePlot.py
Instead of determining the names of the fields and lines form thefilenames it is now also possible to specify them through options.The option --is-distribution is a shorthand that sets theseoptions for distribution files
13.3.7.13 pyFoamSamplePlot.py now allows specification of x-range
The range of the x-axis of the plots can either be set byautomatically scaling to the domains of all the data sets with--scale-domain or by specifying them with --domain-minimum or--domain-maximum.These domains are set for all plots
13.4 Release 0.6.3 - 2014-06-23
For release notes see here
13.5 Release 0.6.4 - 2014-11-24
The release notes for this release are found here
13.6 Release 0.6.5 - 2015-06-23
The release notes for this release are found here
13.7 Release 0.6.6 - 2016-07-15
The release notes for this release are found here
13.8 Release 0.6.7 - 2017-06-04
The release notes for this release are found here
13.9 Release 0.6.8 - 2017-07-06
The release notes for this release are found here
13.10 Release 0.6.9 - 2018-02-25
The release notes for this release are found here
--Bgschaid 21:50, 15 Aug 2005 (CEST)