Difference between revisions of "Contrib Parallelized Native OpenFOAM Reader for ParaView"

From OpenFOAMWiki
m (Installation: serial reader)
(posted version 20090920)
Line 1: Line 1:
 
=Short description=
 
=Short description=
  
This is an OpenFOAM native reader for ParaView 3.3/3.4/3.5 and VTK (vtkOpenFOAMReader) and its parallelization extension for decomposePar-ed cases (vtkPOpenFOAMReader).
+
This is an OpenFOAM native reader for ParaView 3.6/3.7 and VTK (vtkOpenFOAMReader) and its parallelization extension for decomposePar-ed cases (vtkPOpenFOAMReader).
  
 
=Features=
 
=Features=
==Features: main features==
+
* Fast. For a rough performance indication, the reader loads an ascii-gzipped mesh with 3.5 million hexahedral cells in 9 seconds on the developer's 3.0 GHz Mac Pro.
* Fast. For a rough performance indication, the serial reader loads an ascii-gzipped mesh with 3.5 million hexahedral cells in 9 seconds on my 3.0 GHz Mac Pro.
+
 
+
 
* Memory efficient. The readers typically requires less memory than PV3FoamReader. The above mentioned case crashes a 32-bit version of ParaView due to out of memory if loaded by PV3FoamReader, while the reader does not.
 
* Memory efficient. The readers typically requires less memory than PV3FoamReader. The above mentioned case crashes a 32-bit version of ParaView due to out of memory if loaded by PV3FoamReader, while the reader does not.
 
 
* Portable. The reader works under Linux, Mac OS X and Windows without any porting works.
 
* Portable. The reader works under Linux, Mac OS X and Windows without any porting works.
 
 
* Supports parallel decomposed cases and multi-region cases.
 
* Supports parallel decomposed cases and multi-region cases.
  
==Features: new features/changes from the last release (20080728)==
+
=Manifest and Licenses=
 
+
After unpacking the package, the following contents will be found under the top directory.
* Removed the Accumulate Patches option. Zero-sized patches are now always preserved in the selection list.
+
BSD/ contains the reader provided under the ParaView License 1.2
 
+
GPL/ contains additional UI controls provided under the GPL
* Fixed cell normal directions (they all have been inverted!).
+
CMakeLists.txt (CMake configuration file)
 
+
[http://sourceforge.net/projects/of-interfaces/files/vtkPOpenFOAMReader/vtkPOpenFOAMReader-r173-20090920/README.txt/download README]
* Experimental parallelized reader plugin for decomposed cases (vtk"P"OpenFOAMReader).
+
[http://sourceforge.net/projects/of-interfaces/files/vtkPOpenFOAMReader/vtkPOpenFOAMReader-r173-20090920/readerPanel.pdf/download readerPanel.pdf] (quick reference guide)
 
+
* Support for standard multi-region cases where mesh for each region is separeted into separate subdirectory. This means chtMultiRegionFoam is not supported.
+
 
+
* Dropped ParaView 3.2.x support.
+
 
+
* Support for old mesh format (points/faces/cells).
+
 
+
* Addition of several extra mesh consistency checks.
+
 
+
* The license of the reader is now compatible with ParaView License v1.2, except for the GPLed materials under the GPL subdirectory.
+
 
+
* Fixed compressed lagrangian positions not being detected.
+
 
+
* Now warns about the existence of different time directories with the same time value and removes the duplicates (e.g. "0" and "0.000").
+
 
+
* Added a Refresh button to refresh case information (timesteps etc.).
+
 
+
* Added a feature to attach the case path as field data to the output. This can be accesible from a downstream programmable filter as follows.
+
<python>input = self.GetInputDataObject(0, 0)
+
casePath = input.GetFieldData().GetArray("CasePath").GetPointer(0)</python>NB: In client/server mode, the case path is not added to the output of pvserver processes whose IDs are equal or larger than the number of available processor subdirectories, since it is the behavior specified in The ParaView Guide.
+
 
+
* Added labelField support for lagrangians.
+
 
+
* Added feature to change case without starting over.
+
 
+
=Licenses=
+
  
The serial part of the package is licensed under the ParaView License (a BSD license, "the code may be used in commercial products without condition" in Kitware's words). On the other hand the parallel part and the additional UI are licensed under GPL in order to avoid complicated licensing issues coming from the fact that I am using an open source edition of Qt. I am willing to hear suggestions (except "buy a commercial edition of Qt").
+
The <tt>BSD/</tt> subdirectory contains the main reader sources which are licensed under the ParaView License (a BSD license, "the code may be used in commercial products without condition" in Kitware's words). The <tt>GPL/</tt> subdirectory contains the additional UI which is licensed under the GPL in order to avoid complicated licensing issues coming from the fact that the developer is using an open source edition of Qt 4.3. The GPLed materials are required only for "refresh (rescan timesteps and reload mesh and fields with a single click)" and "watch a running case (poll the case directory with a specified interval and render a new timestep if one is found)" capabilities. Other reader functionalities work without the GPLed materials. (Furthermore: this wiki page is licensed [[Main Policy#Information submitted to this Wiki|under the GFDL]])
  
 
=Installation=
 
=Installation=
 +
The reader can be installed as either plugins or a builtin reader. Unlike the previous versions of the reader, if one chooses to install the reader as plugins the main ParaView sources no longer have to be patched. Thus in general installing as plugins is recommended.
  
For now the installation configuration of the reader is a bit irregular: the installation of the serial part will be done by overwriting the existing reader in the ParaView source tree, while the parallel part and the additional UI will be built as plugins.
+
On the other hand, installing as a builtin reader is meant for Linux package maintainers who intend to distribute their own versions of ParaView sources and binaries.
  
First install the serial reader as noted by BSD/ReleaseNotes, then proceed to install the parallel reader following GPL/ReleaseNotes.
+
==Prerequisites==
 +
* CMake 2.6 or later is required.
 +
* The reader has absolutely no dependence on any of the OpenFOAM libraries (which is why the reader is called native). However the reader can be installed somewhat easier as plugins if installed with ParaView 3.6.1 that comes with the ThirdParty package of OpenFOAM 1.6.
 +
* When configuring the reader with CMake, a system POSIX regex (regular expression) library is searched for and the reader uses the library if found in order to support the new regex'ed dictionary keyword syntax in OpenFOAM 1.6. If one is not found but the reader is built with Visual Studio 2008 SP1 or later, the regex library in STL TR1 extension that comes with Visual Studio 2008 SP1 or later is used instead. If none were found, the regex library included in VTK is used as the final fallback option. This is because at least a pattern in the extended regular expression (ERE) is known not supported by the VTK regex library. Hence on Windows, building the reader with Visual Studio 2008 SP1 or later is strongly recommended.
  
<b>[IMPORTANT]</b> You must install both of the serial and parallel reader even if you only need the serial reader.
+
==Install as plugins (Linux and OS X)==
 +
===Building and installing ParaView with parallelization support===
 +
First one has to do a full build, installation and configuration of ParaView with parallelization support. See
 +
http://paraview.org/Wiki/ParaView:Build_And_Install
 +
http://paraview.org/Wiki/Starting_the_server
 +
for full instructions. When configuring with CMake, at least the following variables should be set:
 +
BUILD_SHARED_LIBS = ON
 +
CMAKE_BUILD_TYPE = Release
 +
PARAVIEW_USE_MPI = ON
 +
. After installation, do not remove the ParaView build directory but keep it since it is required in building the plugins afterwards.
  
==Installation: serial reader==
+
Alternatively, a parallelized ParaView-3.6.1 can be built using the OpenFOAM ThirdParty-1.6 source package (<tt>ThirdParty-1.6.General.gtgz</tt>) by issuing the following commands under OpenFOAM environment:
 +
<bash>cd $WM_THIRD_PARTY_DIR
 +
rm -rf paraview-3.6.1/platforms
 +
./makeParaView -mpi
 +
</bash>
  
1. Only required for ThirdParty paraview-3.6.1: Setup your OF environment.<bash>. ~/OpenFOAM/OpenFOAM-1.6/etc/bashrc</bash>
+
===Building and installing the plugins===
 +
The BSDed and GPLed plugins can be installed at once. Create a build directory of the plugins and point CMake to the top directory of the reader source tree (the directory where the <tt>README</tt> file is in).
 +
<bash>mkdir <the plugin build directory>
 +
cd <the plugin build directory>
 +
ccmake <the top directory of the reader source tree>
 +
</bash>
  
2. Untar the package at the parent directory of ParaView3 source tree.<bash>tar zxf <packageName>-<releaseDate>.tar.gz</bash>
+
Then set
 +
CMAKE_BUILD_TYPE = Release
 +
CMAKE_INSTALL_PREFIX = <one of the local plugin directories of the ParaView installation>
 +
CMAKE_OSX_ARCHITECTURES = <the value used for the main ParaView build>
 +
ParaView_DIR = <the directory where ParaView was built>
 +
where <tt>CMAKE_OSX_ARCHITECTURES</tt> is only required under OS X. If one is unsure about where to install the plugins, the plugin directories can be checked by launching ParaView and choosing the Tools -> Manage Plugins/Extensions... menu. There one will see "Local plugins are automatically searched for in..." followed by a list of directories. Alternatively, if the plugins are being built within OpenFOAM environment in conjunction with the ThirdParty-1.6 ParaView, all of these variables are set automatically as default values.
  
3. Run the preInstall script.<bash><packageDirectory>/BSD/preInstall.sh</bash>
+
Then build and install the plugins.
 +
<bash>make
 +
make install
 +
</bash>
  
4. Build ParaView3.
+
By these steps the reader and the UI plugins (<tt>libPOpenFOAMReaderPlugin</tt> and <tt>libPOpenFOAMPanelPlugin</tt>) are installed to a local plugin directory.
  
For general ParaView 3.3/3.4: Build ParaView3 as usual. You would want to turn on compiler optimization (set CMAKE_BUILD_TYPE to Release). Without it the reader is dead slow. For a rough performance indication, the reader loads an ascii-gzipped mesh with 3.5 million hexahedral cells in 9 seconds on my 3.0 GHz Mac Pro.
+
'''[!!IMPORTANT!!]''' ParaView may have different remote plugin directories than the local plugin directories when in client/server mode. In such cases the installed BSDed plugin (<tt>libPOpenFOAMReaderPlugin</tt>) under the local plugin directory must be manually copied (or symbolic linked) by hand to one of the remote plugin directories. To check remote plugin directories, start ParaView in client/server configuration and choose Tools -> Manage Plugins/Extensions... . Attempting to read OpenFOAM cases in client/server mode without the plugin loaded as a remote plugin '''will''' crash ParaView.
  
For ThirdParty paraview-3.X:
+
===Building and installing only the BSDed plugin===
* If you have built your own binary of ParaView,
+
If one would not like the GPLed plugin to be installed, point CMake to the BSD subdirectory of the reader source tree when configuring the plugin with CMake. The remaining steps are identical to the previous section.
<bash>$WM_THIRD_PARTY/makeParaView -rebuild      # should finish in a minute or two!</bash>
+
* If you have not built your own binary of ParaView,
+
<bash>$WM_THIRD_PARTY/makeParaView              # wait a while until it finishes</bash>
+
  
==Installation: parallelized reader==
+
==Install as plugins (Windows)==
The package can be installed with the usual installation procedure of ParaView plugins. For details you would want to refer to Section 19.2 of The ParaView Guide. In brief, it goes as follows.
+
The overall installation steps are identical to those for Linux and OS X except that one would probably use <tt>cmake-gui</tt> instead of <tt>ccmake</tt> and Visual Studio IDE (open <tt>Project.sln</tt> under the plugin build directory, set Configuration to <tt>Release</tt> and build <tt>ALL_BUILD</tt> followed by <tt>INSTALL</tt>) instead of <tt>make</tt>. The reader has been only tested with Visual Studio, and as written in the Prerequisites section, Visual Studio 2008 SP1 or later is strongly recommended. Building under MinGW has not been tested.
  
0. <b>[IMPORTANT]</b>
+
==Install as a builtin reader==
* Installation of vtkOpenFOAMReader (the serial version of the reader) is required before installation of the parallel reader.
+
Follow the "Install by patching the ParaView sources" section of each <tt>INSTALL</tt> file under the <tt>BSD/</tt> and <tt>GPL/</tt> subdirectories. After that, build ParaView as usual.
* In order for parallelization to work you need to build ParaView with parallelization support (PARAVIEW_USE_MPI=ON). However if you just like to read decomposed cases in serial the parallelization support is not mandatory.
+
 
+
1. Create a build directory.
+
 
+
2. Run ccmake from within the build directory. Set ParaView_DIR to your ParaView build directory (not the source directory) at the command line.<bash>cd <buildDirectory>
+
ccmake <packageDirectory>/GPL -DParaView_DIR=<path to your paraview build></bash>
+
 
+
3. Configure the build. Note that CMAKE_BUILD_TYPE must be same as the one used for the ParaView build (otherwise the plugin will not load).
+
 
+
4. Run make.<bash>make</bash>
+
 
+
5. Copy the following produced libraries to your plugin directory.<bash>[lib]POpenFOAMPanel.*
+
[lib]POpenFOAMReaderSMPlugin.*</bash>where * is the extension given to shared libraries on your platform(so, dylib or dll).
+
  
 
=Usage=
 
=Usage=
==Usage: general==
+
In order to open OpenFOAM case, either
 +
* create an empty stub file <tt><caseName>.foam</tt> right under the case directory, where <tt><caseName></tt> is an arbitrary string other than those that starts with "<tt>controlDict</tt>," and open <tt><caseName>.foam</tt> or
 +
* create a symbolic link from <tt>controlDict</tt> to <tt>controlDict.foam</tt> and open <tt>controlDict.foam</tt>.
  
*To load an OpenFOAM case, you either have to
+
There is also a script for user convenience that works like <tt>paraFoam</tt>, called <tt>pvFoam</tt>, under the <tt>BSD/</tt> subdirectory. The script creates the stub file and launches ParaView in background. For available options, run
** create an empty stub file <caseName>.foam right under the case directory, where <caseName> is an arbitrary string other than strings that starts with "controlDict," and open <caseName>.foam or
+
<bash>pvFoam -help
** create a symbolic link from controlDict to controlDict.foam and open controlDict.foam.
+
</bash>
 +
in a terminal.
  
* There is a user convenience script "pvFoam" which works just like paraFoam.
+
For general operations of the reader, see [http://sourceforge.net/projects/of-interfaces/files/vtkPOpenFOAMReader/vtkPOpenFOAMReader-r173-20090920/readerPanel.pdf/download <tt>readerPanel.pdf</tt>].
  
*There is a "Refresh" button on the top right of the reader panel which triggers the reader to refresh the case information, including timesteps, when clicked. This is supposed to be useful when watching a running simulation.
+
=Notes (general)=
 +
# Dictionary syntax <tt>~OpenFOAM</tt> expands only to <tt>$HOME/.OpenFOAM</tt> where <tt>$HOME</tt> is the home directory of the user.
 +
# The <tt>#remove</tt> dictionary directive is not supported.
 +
# The builtin cell-to-point filter works like a Cell Data to Point Data filter in ParaView in that it just takes the average of cell values connected to a point. The difference is that the builtin filter takes boundary patch values into account. The filter is faster but less accurate than the volPoint interpolator in <tt>paraFoam</tt>, which further does inverse distance weighting of cell values. The cell-to-point filter is still computationally demanding thus can be turned off by unchecking "Create cell-to-point filtered data" on the reader panel.
 +
# In order to create slices with volField data one may have to extract internal mesh or turn off reading patches in the reader panel before running the Slice filter, otherwise the sliced data may be garbled.
 +
# When the List timesteps according to controlDict checkbox is turned on, the reader lists time instances according to descriptions in <tt>controlDict</tt> when (<tt>adjustTimeStep</tt>, <tt>writeControl</tt>) == (<tt>yes</tt>, <tt>adjustableRunTime</tt>) or (<tt>no</tt>, <tt>timeStep</tt>) (i. e. writing interval is supposed to be constant in simulated time), or otherwise, lists all valid time directories.
 +
# The reader lists field names according to the "<tt>object</tt>" entry in <tt>FoamFile</tt> header, whereas <tt>paraFoam</tt> lists according to the file name of the object. Hence a valid <tt>FoamFile</tt> header is required for every object.
 +
# Reading cellZones for polyhedral mesh with the Read Zones feature may cause ParaView to thrash.
 +
# The reader only takes the "<tt>value</tt>" and "<tt>uniformValue</tt>" entries into account for boundary fields. If none of the entries exists, the corresponding patch-internal field values are used (i. e. equivalent to <tt>zeroGradient</tt> b. c.).
 +
# When watching running case, one may occasionally see reader errors due to race conditions where the reader reads up until the end of file while the file is being written. Although the developer has put much effort in avoiding crashes in such cases, there may still be ones. Doing important tasks while watching case is thus not recommended.
 +
# Multi-region case is detected by existence of mesh files under subdirectories found in first two time directories. If a case is determined to be a multi-region case, the selection of internalMesh of the defaultRegion in the reader panel is turned off by default.
  
==Usage: parallel-specific==
+
=Notes (Parallel-specific)=
 
+
# The Case Type selection combo box determines whether the case to be read is a reconstructed case (a serial case) or a decomposed case (a case decomposed into processorX subdirectories). The reader can read decomposed cases even when ParaView is run in builtin server (serial) mode. If ParaView is running in client/server mode and the servers (pvservers) are running in parallel, the decomposed mesh regions are read in parallel.
* <b>[IMPORTANT]</b> The POpenFOAMReaderSM plugin must be loaded as a remote plugin before loading a parallel case when ParaView is run in client/server mode. Otherwise ParaView will crash.
+
# The number of decomposed mesh regions and the number of <tt>pvserver</tt> processes do not have to be identical, with a known exception of the note 6 below. Processor subdirectories are detected by server process 0, and each <tt>processor<i>X</i></tt> subdirectory is assigned to a server process in an interleaved way. For example, if one has 5 processor subdirectories and running 2 <tt>pvserver</tt>s, the <tt>pvserver</tt> process 0 reads <tt>processor0</tt>, <tt>processor2</tt> and <tt>processor4</tt> subdirectories while <tt>pvserver</tt> process 1 reads <tt>procssor1</tt> and <tt>processor3</tt> subdirectories. Which process owns which decomposed regions can be checked by Filters -> Process Id Scalars.
 
+
# Timesteps are taken from a first processor subdirectory (typically <tt>processor0</tt>) by server process 0 and broadcasted to other processes.
* The Case Type selection combo box determines whether a reconstructed case or a decomposed case will be loaded. The parallelization works only for decomposed cases in client/server mode. If you try to load a reconstructed case in server/client mode, it will be loaded into pvserver process 0 (then you can apply Merge Blocks -> D3 to distribute the data).
+
# In client/server mode, reconstructed case is loaded by pvserver process 0. Merge Blocks -> D3 can be used in order to distribute the data.
 
+
# The reader does nothing about adding ghost cells. Hence one has to see processor boundaries when the geometry is represented in Wireframe or Surface. This could be especially problematic if one wants to extract surfaces. In order to remove internal processor boundaries from the output of the Extract Surface filter, run Merge Blocks -> D3 -> Extract Surface. Similarly, when the reader is run in serial or in builtin mode, the Clean to Grid filter can be used in order to remove internal processor boundaries.
* You can use the reader either in client/server mode or in builtin server mode. In the latter case the reader works as a serial reader that can read all decomposePar-ed processor subdirectories at once.
+
# In order to run Stream Tracer when in parallel, a <tt>pvserver</tt> process should be assigned to each decomposed mesh region, since the tracer can handle inter-server process boundaries but cannot for processor mesh region boundaries. For example if a case has four decomposed regions, four <tt>pvserver</tt> processes should be used. On the other hand, when in serial Clean to Grid can be used in order to remove the region boundaries as noted in 5. If none of the two applies, the streamlines will stop at appended region boundaries.
 
+
# The reader does not output processor boundary patches of decomposed regions. In order to see the processor patches, each processor subdirectory has to be opened as a separate case.
* For general instructions on configuring ParaView in server/client mode, have a look at the Server Setup section of [http://www.paraview.org/Wiki/ParaView ParaView Wiki].
+
 
+
=Notes=
+
 
+
==Notes: general==
+
1. $-expansion only works for complete syntactic units. e. g.<bash>fieldValue      uniform 0;
+
internalField    $fieldValue;</bash>is supposed to work, while<bash>fieldValue      0;
+
internalField    uniform $fieldValue;</bash>and<bash>uniformity      uniform;
+
internalField    $uniformity 0;</bash>are not.
+
 
+
2. ~OpenFOAM expands only to $HOME/.OpenFOAM where $HOME is the user's home directory.
+
 
+
3. The #remove directive is not supported.
+
 
+
4. The builtin cell-to-point filter works like a Cell Data to Point Data filter in ParaView in that it just takes the average of cell values connected to a point. The difference is that the builtin filter takes boundary patch values into account. The filter is still not as accurate as the volPoint interpolator in paraFoam, which further does inverse distance weighting of cell values.
+
 
+
5. The cell-to-point filter is computationally demanding so you can turn it off by deselecting "Create cell-to-point filtered data" on the reader panel.
+
 
+
6. With the default the reader omits CRC checks for gzipped files (except on Windows). If you feel it is too adventurous set VTK_FOAMFILE_OMIT_CRCCHECK to 0 at the beginning of vtkOpenFOAMReader.cxx.
+
 
+
7. To create a slice with volField data you have to extract internal mesh or turn off reading patches in the reader panel.
+
 
+
8. When the List timesteps according to controlDict checkbox is turned on the reader lists time instances according to descriptions in controlDict when (adjustTimeStep, writeControl) == (yes, adjustableRunTime) or (no, timeStep) (i. e. writing interval is supposed to be constant in simulated time), or otherwise, lists all valid time directories. This is a different behavior from paraFoam which always lists all valid time directories in a case directory.
+
 
+
9. The reader lists field names according to the "object" entry in FoamFile header, whereas paraFoam lists according to the file name of the object. So you need a valid FoamFile header for every object.
+
 
+
10. Reading cellZones for polyhedral mesh with the Read Zones feature may cause ParaView to thrash.
+
 
+
11. Handling of boundary conditions is quite incomplete as of now; if a boundary patch doesn't have a value nor a uniformValue entry the corresponding patch-internal field value is used (i. e. equivalent to zeroGradient b. c.).
+
 
+
12. Single precision binary format is not supported.
+
 
+
13. The reader fails to load mesh that has negative labels in owner. This will be fixed in the next release.
+
 
+
14. The reader fails to restore mesh/array selection status from state file. This will be fixed in the next release.
+
 
+
==Notes: parallel-specific==
+
 
+
1. The number of decomposed mesh regions and the number of pvserver processes do not have to be identical, with a known exception of the following note 4.
+
 
+
2. The reader does nothing about adding ghost cells. So you have to see processor boundaries when your geometry is represented in Wireframe or Surface. If you want to remove internal processor boundaries from the output of Extract Surface, you can run Merge Blocks -> D3 -> Extract Surface.
+
 
+
3. When the reader is run in serial, you can use the Clean to Grid filter to remove internal processor boundaries.
+
 
+
4. To run Stream Tracer when in parallel, a pvserver process should be assigned for each decomposed mesh region (since the tracer handles inter-server process boundaries -- but not processor boundaries -- properly). For example if your case was decomposed into four processor regions, you should run four pvserver processes. On the other hand when in serial you can use Clean to Grid as noted in 3. Otherwise the streamlines will stop at appended processor boundaries.
+
 
+
5. Processor subdirectories are detected by the server process 0 and distributed to other processes in an interleaved way. For example, if you have processor0 - processor4 subdirectories while having 3 pvservers running, process 0 owns processor0 and processor3 subdirectories, process 1 owns processor1 and processor4, and process 2 owns processor2. The reader does not support distributed cases yet.
+
 
+
6. Timesteps are taken from the first processor subdirectory (typically processor0) by server process 0 and broadcasted to other processes.
+
 
+
7. The reader does not output processor boundary patches when reading a decomposed case. To do so you have to read each processor subdirectory as a separate case.
+
  
 
=Miscellaneous=
 
=Miscellaneous=
Line 169: Line 115:
 
* The parallel extension part is quite a straightforward implementation of what are written in the Section 21.3 of The ParaView Guide.
 
* The parallel extension part is quite a straightforward implementation of what are written in the Section 21.3 of The ParaView Guide.
  
* In my testing I get typically 2.3x - 2.5x speedups in mesh loading time by 4 process parallel execution. However everything is not so rosy especially in that I often get slower rendering times by parallel executions. As no one but me being quite new to parallel visualization, I'm interested in your findings. You are welcomed to post them to the forum.
+
* In my testing I get typically 2.3x - 2.5x speedups in mesh loading time by 4 process parallel execution. However everything is not so rosy especially in that I often get slower rendering times by parallel executions. As no one but the developer being quite new to parallel visualization, the developer is interested in your findings. You are welcomed to post them to the forum.
  
 
* The reader was once checked in as a part of ParaView but reverted due to too many errors by older compilers. The author is trying hard to resolve the isssue.
 
* The reader was once checked in as a part of ParaView but reverted due to too many errors by older compilers. The author is trying hard to resolve the isssue.
Line 175: Line 121:
 
=Downloads=
 
=Downloads=
 
Releases:
 
Releases:
*[http://downloads.sourceforge.net/of-interfaces/vtkPOpenFOAMReader-r64-20081005.tar.gz?use_mirror= vtkPOpenFOAMReader-r64-20081005.tar.gz]: The source package
+
*[http://sourceforge.net/projects/of-interfaces/files/vtkPOpenFOAMReader/vtkPOpenFOAMReader-r173-20090920/vtkPOpenFOAMReader-r173-20090920.tar.gz/download vtkPOpenFOAMReader-r173-20090920.tar.gz]: The source package
 
*[http://downloads.sourceforge.net/of-interfaces/ParaView-3.4.0-win32-x86-mpich2-OpenFOAM-r109.zip?use_mirror= ParaView-3.4.0-win32-x86-mpich2-OpenFOAM-r109.zip]: Windows x86 32bit binary package of ParaView 3.4.0 (with parallelization support by MPICH2) patched with the reader. Copy the unpacked package to wherever you like, and see <tt>README-ParaView-mpich2-OpenFOAM.txt</tt> in the package.
 
*[http://downloads.sourceforge.net/of-interfaces/ParaView-3.4.0-win32-x86-mpich2-OpenFOAM-r109.zip?use_mirror= ParaView-3.4.0-win32-x86-mpich2-OpenFOAM-r109.zip]: Windows x86 32bit binary package of ParaView 3.4.0 (with parallelization support by MPICH2) patched with the reader. Copy the unpacked package to wherever you like, and see <tt>README-ParaView-mpich2-OpenFOAM.txt</tt> in the package.
*Linux ParaView packages with the reader included: [http://bugs.gentoo.org/243362 Gentoo], [http://www.cfd-online.com/Forums/openfoam-installation/57200-debian-package-ubuntu-810-a.html Ubuntu 8.10]
+
*Linux ParaView packages with the reader included (the versions of the reader may not be up-to-date): [http://bugs.gentoo.org/243362 Gentoo], [http://www.cfd-online.com/Forums/openfoam-installation/57200-debian-package-ubuntu-810-a.html Ubuntu 8.10], [http://www.cfd-online.com/Forums/openfoam-installation/67676-openfoam-1-6-x-centos-5-3-x86_64-a.html CentOS 5.3 x86_64]
 
+
Subversion repository:
+
*To Linux package maintainers: there's a branch that is meant to be included in ParaView source trees as a builtin reader. See README under the top directory for how to install.
+
svn co https://of-interfaces.svn.sourceforge.net/svnroot/of-interfaces/branches/vtkPOpenFOAMReader-r23-VTKcontrib
+
  
 
Technical documentation:
 
Technical documentation:
Line 187: Line 129:
  
 
Past versions:
 
Past versions:
 +
*[http://downloads.sourceforge.net/of-interfaces/vtkPOpenFOAMReader-r64-20081005.tar.gz?use_mirror= vtkPOpenFOAMReader-r64-20081005.tar.gz]
 
*Parallel reader development snapshot: [[Media:vtkPOpenFOAMReader-20080831.tar.gz|vtkPOpenFOAMReader-20080831.tar.gz]]
 
*Parallel reader development snapshot: [[Media:vtkPOpenFOAMReader-20080831.tar.gz|vtkPOpenFOAMReader-20080831.tar.gz]]
*Serial versions: Download from [http://openfoam.cfd-online.com/cgi-bin/forum/show.cgi?tpc=1&post=17466#POST17466 the native reader thread] on the Message Board.
+
*Ancient serial versions: Download from [http://openfoam.cfd-online.com/cgi-bin/forum/show.cgi?tpc=1&post=17466#POST17466 the native reader thread] on the Message Board.
  
=Feedback=
+
=Feedbacks=
 
* A use case on a supercomputing cluster: [http://www.csc.fi/csc/kurssit/arkisto/ofud09 "Parallelized OpenFOAM reader for Paraview" - An example] by CSC Finland
 
* A use case on a supercomputing cluster: [http://www.csc.fi/csc/kurssit/arkisto/ofud09 "Parallelized OpenFOAM reader for Paraview" - An example] by CSC Finland
* [http://serverlab.unab.edu.co:8080/wikimedia/memorias/shortspapers.html "Construcción de un túnel de viento virtual sobre AccessGrid y ParaView"] (Construction of a virtual wind tunnel on AccessGrid and ParaView) by Camilo Cortés et al (in Spanish). One comment is that the bad parallel scaling of ParaView as shown in Tabla 1 is expected, because [http://www.vtk.org/doc/release/5.2/html/a00337.html the Stream Tracer is essentially a serial algorithm thus only one process is active at one time].
+
* [http://serverlab.unab.edu.co:8080/wikimedia/memorias/shortspapers.html "Construcción de un túnel de viento virtual sobre AccessGrid y ParaView"] (Construction of a virtual wind tunnel on AccessGrid and ParaView) by Camilo Cortés et al (in Spanish). One comment is that the bad parallel scaling of ParaView as shown in Tabla 1 is expected, because [http://www.vtk.org/doc/release/5.2/html/a00337.html the Stream Tracer is essentially a serial algorithm thus only one process is active at one time]. Nonetheless the developer appreciates the fair citation.
  
 
If you found bugs or have comments, post to [http://openfoam.cfd-online.com/cgi-bin/forum/show.cgi?tpc=1&post=17466 the native reader thread].
 
If you found bugs or have comments, post to [http://openfoam.cfd-online.com/cgi-bin/forum/show.cgi?tpc=1&post=17466 the native reader thread].
  
 
=History=
 
=History=
 +
[[User:7islands|7islands]] 09:55, 20 September 2009 (UTC): Released 20090920. Changes from the 20081005 version are
 +
* Added support for extended dictionary syntaxes of OpenFOAM 1.6, notably the regex'ed dictionary keywords.
 +
* The reader now compiles with a drastically wider range of compilers. The tested compilers include many GCC versions from 3.3.2 to 4.4.1 (including Apple's compilers under 64bit mode), Intel Compiler 11/11.1, Sun Studio 12/12.1, Open64 4.2.1/4.2.2, Visual Studio 2003.net/2008SP1.
 +
* Added "watching a running case" functionality.
 +
* Better detection and handling of multi region cases. The reader now can read <tt>chtMultiRegionFoam/multiRegionHeater</tt> case. Note that one will still get an error about missing "<tt>uniform</tt>" keywords in <tt>buoyantPressure</tt> BC sections of the field files when reading the case as is. In order to eliminate the error, either add the uniform keyword to or remove entirely the value entries of the <tt>buoyantPressure</tt> sections of the field files and <tt>cahgeDictionaryDict</tt>s under the <tt>system</tt> subdirectory.
 +
* Added support for single precision binary format.
 +
* Improved performance in reading files in double precision binary format.
 +
* Improved installation steps.
 +
* Rewritten pvFoam script.
 +
* Changed the license of some more files to the BSDL.
 +
* Workaround for a locale problem in string-to-floating conversion under Qt 4.5/Linux.
 +
* Fixed many bugs, including crashes (thanks to all the reporters).
 +
* Other features that I don't remember.
 +
 
[[User:7islands|7islands]] 06:10, 23 March 2009 (CET) (Takuya OSHIMA): Added a win32 binary package release.
 
[[User:7islands|7islands]] 06:10, 23 March 2009 (CET) (Takuya OSHIMA): Added a win32 binary package release.
  

Revision as of 09:55, 20 September 2009

1 Short description

This is an OpenFOAM native reader for ParaView 3.6/3.7 and VTK (vtkOpenFOAMReader) and its parallelization extension for decomposePar-ed cases (vtkPOpenFOAMReader).

2 Features

  • Fast. For a rough performance indication, the reader loads an ascii-gzipped mesh with 3.5 million hexahedral cells in 9 seconds on the developer's 3.0 GHz Mac Pro.
  • Memory efficient. The readers typically requires less memory than PV3FoamReader. The above mentioned case crashes a 32-bit version of ParaView due to out of memory if loaded by PV3FoamReader, while the reader does not.
  • Portable. The reader works under Linux, Mac OS X and Windows without any porting works.
  • Supports parallel decomposed cases and multi-region cases.

3 Manifest and Licenses

After unpacking the package, the following contents will be found under the top directory.

BSD/ contains the reader provided under the ParaView License 1.2
GPL/ contains additional UI controls provided under the GPL
CMakeLists.txt (CMake configuration file)
README
readerPanel.pdf (quick reference guide)

The BSD/ subdirectory contains the main reader sources which are licensed under the ParaView License (a BSD license, "the code may be used in commercial products without condition" in Kitware's words). The GPL/ subdirectory contains the additional UI which is licensed under the GPL in order to avoid complicated licensing issues coming from the fact that the developer is using an open source edition of Qt 4.3. The GPLed materials are required only for "refresh (rescan timesteps and reload mesh and fields with a single click)" and "watch a running case (poll the case directory with a specified interval and render a new timestep if one is found)" capabilities. Other reader functionalities work without the GPLed materials. (Furthermore: this wiki page is licensed under the GFDL)

4 Installation

The reader can be installed as either plugins or a builtin reader. Unlike the previous versions of the reader, if one chooses to install the reader as plugins the main ParaView sources no longer have to be patched. Thus in general installing as plugins is recommended.

On the other hand, installing as a builtin reader is meant for Linux package maintainers who intend to distribute their own versions of ParaView sources and binaries.

4.1 Prerequisites

  • CMake 2.6 or later is required.
  • The reader has absolutely no dependence on any of the OpenFOAM libraries (which is why the reader is called native). However the reader can be installed somewhat easier as plugins if installed with ParaView 3.6.1 that comes with the ThirdParty package of OpenFOAM 1.6.
  • When configuring the reader with CMake, a system POSIX regex (regular expression) library is searched for and the reader uses the library if found in order to support the new regex'ed dictionary keyword syntax in OpenFOAM 1.6. If one is not found but the reader is built with Visual Studio 2008 SP1 or later, the regex library in STL TR1 extension that comes with Visual Studio 2008 SP1 or later is used instead. If none were found, the regex library included in VTK is used as the final fallback option. This is because at least a pattern in the extended regular expression (ERE) is known not supported by the VTK regex library. Hence on Windows, building the reader with Visual Studio 2008 SP1 or later is strongly recommended.

4.2 Install as plugins (Linux and OS X)

4.2.1 Building and installing ParaView with parallelization support

First one has to do a full build, installation and configuration of ParaView with parallelization support. See

http://paraview.org/Wiki/ParaView:Build_And_Install
http://paraview.org/Wiki/Starting_the_server

for full instructions. When configuring with CMake, at least the following variables should be set:

BUILD_SHARED_LIBS = ON
CMAKE_BUILD_TYPE = Release
PARAVIEW_USE_MPI = ON

. After installation, do not remove the ParaView build directory but keep it since it is required in building the plugins afterwards.

Alternatively, a parallelized ParaView-3.6.1 can be built using the OpenFOAM ThirdParty-1.6 source package (ThirdParty-1.6.General.gtgz) by issuing the following commands under OpenFOAM environment:

cd $WM_THIRD_PARTY_DIR
rm -rf paraview-3.6.1/platforms
./makeParaView -mpi

4.2.2 Building and installing the plugins

The BSDed and GPLed plugins can be installed at once. Create a build directory of the plugins and point CMake to the top directory of the reader source tree (the directory where the README file is in).

mkdir <the plugin build directory>
cd <the plugin build directory>
ccmake <the top directory of the reader source tree>

Then set

CMAKE_BUILD_TYPE = Release
CMAKE_INSTALL_PREFIX = <one of the local plugin directories of the ParaView installation>
CMAKE_OSX_ARCHITECTURES = <the value used for the main ParaView build>
ParaView_DIR = <the directory where ParaView was built>

where CMAKE_OSX_ARCHITECTURES is only required under OS X. If one is unsure about where to install the plugins, the plugin directories can be checked by launching ParaView and choosing the Tools -> Manage Plugins/Extensions... menu. There one will see "Local plugins are automatically searched for in..." followed by a list of directories. Alternatively, if the plugins are being built within OpenFOAM environment in conjunction with the ThirdParty-1.6 ParaView, all of these variables are set automatically as default values.

Then build and install the plugins.

make
make install

By these steps the reader and the UI plugins (libPOpenFOAMReaderPlugin and libPOpenFOAMPanelPlugin) are installed to a local plugin directory.

[!!IMPORTANT!!] ParaView may have different remote plugin directories than the local plugin directories when in client/server mode. In such cases the installed BSDed plugin (libPOpenFOAMReaderPlugin) under the local plugin directory must be manually copied (or symbolic linked) by hand to one of the remote plugin directories. To check remote plugin directories, start ParaView in client/server configuration and choose Tools -> Manage Plugins/Extensions... . Attempting to read OpenFOAM cases in client/server mode without the plugin loaded as a remote plugin will crash ParaView.

4.2.3 Building and installing only the BSDed plugin

If one would not like the GPLed plugin to be installed, point CMake to the BSD subdirectory of the reader source tree when configuring the plugin with CMake. The remaining steps are identical to the previous section.

4.3 Install as plugins (Windows)

The overall installation steps are identical to those for Linux and OS X except that one would probably use cmake-gui instead of ccmake and Visual Studio IDE (open Project.sln under the plugin build directory, set Configuration to Release and build ALL_BUILD followed by INSTALL) instead of make. The reader has been only tested with Visual Studio, and as written in the Prerequisites section, Visual Studio 2008 SP1 or later is strongly recommended. Building under MinGW has not been tested.

4.4 Install as a builtin reader

Follow the "Install by patching the ParaView sources" section of each INSTALL file under the BSD/ and GPL/ subdirectories. After that, build ParaView as usual.

5 Usage

In order to open OpenFOAM case, either

  • create an empty stub file <caseName>.foam right under the case directory, where <caseName> is an arbitrary string other than those that starts with "controlDict," and open <caseName>.foam or
  • create a symbolic link from controlDict to controlDict.foam and open controlDict.foam.

There is also a script for user convenience that works like paraFoam, called pvFoam, under the BSD/ subdirectory. The script creates the stub file and launches ParaView in background. For available options, run

pvFoam -help

in a terminal.

For general operations of the reader, see readerPanel.pdf.

6 Notes (general)

  1. Dictionary syntax ~OpenFOAM expands only to $HOME/.OpenFOAM where $HOME is the home directory of the user.
  2. The #remove dictionary directive is not supported.
  3. The builtin cell-to-point filter works like a Cell Data to Point Data filter in ParaView in that it just takes the average of cell values connected to a point. The difference is that the builtin filter takes boundary patch values into account. The filter is faster but less accurate than the volPoint interpolator in paraFoam, which further does inverse distance weighting of cell values. The cell-to-point filter is still computationally demanding thus can be turned off by unchecking "Create cell-to-point filtered data" on the reader panel.
  4. In order to create slices with volField data one may have to extract internal mesh or turn off reading patches in the reader panel before running the Slice filter, otherwise the sliced data may be garbled.
  5. When the List timesteps according to controlDict checkbox is turned on, the reader lists time instances according to descriptions in controlDict when (adjustTimeStep, writeControl) == (yes, adjustableRunTime) or (no, timeStep) (i. e. writing interval is supposed to be constant in simulated time), or otherwise, lists all valid time directories.
  6. The reader lists field names according to the "object" entry in FoamFile header, whereas paraFoam lists according to the file name of the object. Hence a valid FoamFile header is required for every object.
  7. Reading cellZones for polyhedral mesh with the Read Zones feature may cause ParaView to thrash.
  8. The reader only takes the "value" and "uniformValue" entries into account for boundary fields. If none of the entries exists, the corresponding patch-internal field values are used (i. e. equivalent to zeroGradient b. c.).
  9. When watching running case, one may occasionally see reader errors due to race conditions where the reader reads up until the end of file while the file is being written. Although the developer has put much effort in avoiding crashes in such cases, there may still be ones. Doing important tasks while watching case is thus not recommended.
  10. Multi-region case is detected by existence of mesh files under subdirectories found in first two time directories. If a case is determined to be a multi-region case, the selection of internalMesh of the defaultRegion in the reader panel is turned off by default.

7 Notes (Parallel-specific)

  1. The Case Type selection combo box determines whether the case to be read is a reconstructed case (a serial case) or a decomposed case (a case decomposed into processorX subdirectories). The reader can read decomposed cases even when ParaView is run in builtin server (serial) mode. If ParaView is running in client/server mode and the servers (pvservers) are running in parallel, the decomposed mesh regions are read in parallel.
  2. The number of decomposed mesh regions and the number of pvserver processes do not have to be identical, with a known exception of the note 6 below. Processor subdirectories are detected by server process 0, and each processorX subdirectory is assigned to a server process in an interleaved way. For example, if one has 5 processor subdirectories and running 2 pvservers, the pvserver process 0 reads processor0, processor2 and processor4 subdirectories while pvserver process 1 reads procssor1 and processor3 subdirectories. Which process owns which decomposed regions can be checked by Filters -> Process Id Scalars.
  3. Timesteps are taken from a first processor subdirectory (typically processor0) by server process 0 and broadcasted to other processes.
  4. In client/server mode, reconstructed case is loaded by pvserver process 0. Merge Blocks -> D3 can be used in order to distribute the data.
  5. The reader does nothing about adding ghost cells. Hence one has to see processor boundaries when the geometry is represented in Wireframe or Surface. This could be especially problematic if one wants to extract surfaces. In order to remove internal processor boundaries from the output of the Extract Surface filter, run Merge Blocks -> D3 -> Extract Surface. Similarly, when the reader is run in serial or in builtin mode, the Clean to Grid filter can be used in order to remove internal processor boundaries.
  6. In order to run Stream Tracer when in parallel, a pvserver process should be assigned to each decomposed mesh region, since the tracer can handle inter-server process boundaries but cannot for processor mesh region boundaries. For example if a case has four decomposed regions, four pvserver processes should be used. On the other hand, when in serial Clean to Grid can be used in order to remove the region boundaries as noted in 5. If none of the two applies, the streamlines will stop at appended region boundaries.
  7. The reader does not output processor boundary patches of decomposed regions. In order to see the processor patches, each processor subdirectory has to be opened as a separate case.

8 Miscellaneous

  • The parallel extension part is quite a straightforward implementation of what are written in the Section 21.3 of The ParaView Guide.
  • In my testing I get typically 2.3x - 2.5x speedups in mesh loading time by 4 process parallel execution. However everything is not so rosy especially in that I often get slower rendering times by parallel executions. As no one but the developer being quite new to parallel visualization, the developer is interested in your findings. You are welcomed to post them to the forum.
  • The reader was once checked in as a part of ParaView but reverted due to too many errors by older compilers. The author is trying hard to resolve the isssue.

9 Downloads

Releases:

Technical documentation:

Past versions:

10 Feedbacks

If you found bugs or have comments, post to the native reader thread.

11 History

7islands 09:55, 20 September 2009 (UTC): Released 20090920. Changes from the 20081005 version are

  • Added support for extended dictionary syntaxes of OpenFOAM 1.6, notably the regex'ed dictionary keywords.
  • The reader now compiles with a drastically wider range of compilers. The tested compilers include many GCC versions from 3.3.2 to 4.4.1 (including Apple's compilers under 64bit mode), Intel Compiler 11/11.1, Sun Studio 12/12.1, Open64 4.2.1/4.2.2, Visual Studio 2003.net/2008SP1.
  • Added "watching a running case" functionality.
  • Better detection and handling of multi region cases. The reader now can read chtMultiRegionFoam/multiRegionHeater case. Note that one will still get an error about missing "uniform" keywords in buoyantPressure BC sections of the field files when reading the case as is. In order to eliminate the error, either add the uniform keyword to or remove entirely the value entries of the buoyantPressure sections of the field files and cahgeDictionaryDicts under the system subdirectory.
  • Added support for single precision binary format.
  • Improved performance in reading files in double precision binary format.
  • Improved installation steps.
  • Rewritten pvFoam script.
  • Changed the license of some more files to the BSDL.
  • Workaround for a locale problem in string-to-floating conversion under Qt 4.5/Linux.
  • Fixed many bugs, including crashes (thanks to all the reporters).
  • Other features that I don't remember.

7islands 06:10, 23 March 2009 (CET) (Takuya OSHIMA): Added a win32 binary package release.

7islands 06:13, 5 October 2008 (CEST) (Takuya OSHIMA): Released 20081005. Notable changes from the 20080831 snapshot are

  • Unified application interfaces of the serial/parallel readers. Both can be accessed by opening a .foam file. Subsequently, readers.xml no longer have to be patched -- which makes tracking the CVS head of ParaView easier.
  • Fixed several bugs, notably deadlocks.
  • Added lagrangian labelField support, casePath addition to output.
  • Added feature to change case without reinstanciating another reader.

7islands 06:57, 31 August 2008 (CEST) (Takuya OSHIMA): moved to Contrib section. Initial upload of the parallelized version.

7islands 02:58, 28 April 2008 (CEST) (Takuya OSHIMA): update for the 20080324 version.

7islands 09:18, 29 Dec 2007 (CET): some addenda.

7islands 04:36, 16 Dec 2007 (CET) (Takuya OSHIMA): Created this page.