Difference between revisions of "HowTo debugging"
(→Other locations to turn on the built-in debug flags: Added detail of inline options/flags for foam-extend 3.2 and changed the formatting a bit and added more version numbers) |
m (some formatting) |
||
(3 intermediate revisions by 2 users not shown) | |||
Line 2: | Line 2: | ||
If your application crashes it will usually output a stack trace, e.g. | If your application crashes it will usually output a stack trace, e.g. | ||
− | < | + | <blockquote> |
− | #0 Foam::error::printStack(Foam::-Ostream&) in "/home/ | + | #0 Foam::error::printStack(Foam::-Ostream&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"<br/> |
− | #1 Foam::sigFpe::sigFpeHandler(int) in "/home/ | + | #1 Foam::sigFpe::sigFpeHandler(int) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"<br/> |
#2 Uninterpreted: [0xb7f8b420]<br/> | #2 Uninterpreted: [0xb7f8b420]<br/> | ||
− | #3 Foam::divide(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&) in "/home/ | + | #3 Foam::divide(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"<br/> |
− | #4 void Foam::divide<foam::fvpatchfield,>(Foam::GeometricField<double,>&, Foam::GeometricField<double,> const&, Foam::GeometricField<double,> const&) in "/home/ | + | #4 void Foam::divide<foam::fvpatchfield,>(Foam::GeometricField<double,>&, Foam::GeometricField<double,> const&, Foam::GeometricField<double,> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so" <br/> |
− | #5 Foam::tmp<foam::geometricfield<double,> > Foam::operator/<foam::fvpatchfield,>(Foam::tmp<foam::geometricfield<double,> > const&, Foam::GeometricField<double,> const&) in "/home/ | + | #5 Foam::tmp<foam::geometricfield<double,> > Foam::operator/<foam::fvpatchfield,>(Foam::tmp<foam::geometricfield<double,> > const&, Foam::GeometricField<double,> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so" <br/> |
− | #6 Foam::turbulenceModels::kEpsilon::correct() in "/home/ | + | #6 Foam::turbulenceModels::kEpsilon::correct() in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so" <br/> |
− | </ | + | </blockquote> |
There is lots of interesting information in there. It shows the type of error (sigFpe which means a division by zero or any other operation causing an invalid floating point number) and who caused it (operator/ of an fvPatchField). Further down is the origin, kEpsilon::correct(), which obviously does some divisions. A good guess is that one of the patch fields of k or epsilon is 0. | There is lots of interesting information in there. It shows the type of error (sigFpe which means a division by zero or any other operation causing an invalid floating point number) and who caused it (operator/ of an fvPatchField). Further down is the origin, kEpsilon::correct(), which obviously does some divisions. A good guess is that one of the patch fields of k or epsilon is 0. | ||
Line 57: | Line 57: | ||
==== ddd ==== | ==== ddd ==== | ||
− | Is another more complex frontend for | + | Is another more complex frontend for gdb. You can launch your solver with the following command |
ddd --args xxxFoam <FoamOptions> | ddd --args xxxFoam <FoamOptions> | ||
Line 180: | Line 180: | ||
− | <li>{{version2.2}} {{version2.3}} {{version2.4}} {{version3.0}} In addition to the previous feature, as of OpenFOAM 2.2.0, one can set these flags directly inside a case's {{tt|system/controlDict}} file, as documented at the official website: [http:// | + | <li>{{version2.2}} {{version2.3}} {{version2.4}} {{version3.0}} {{version4.0}} In addition to the previous feature, as of OpenFOAM 2.2.0, one can set these flags directly inside a case's {{tt|system/controlDict}} file, as documented at the official website: [http://openfoam.org/release/2-2-0/runtime-control/ OpenFOAM v2.2.0: Run-time Control] |
</li> | </li> | ||
Line 204: | Line 204: | ||
* Figuring out how the {{tt|SaffmanMeiLiftForce}} Lagrangian model can be used: [http://www.cfd-online.com/Forums/openfoam-pre-processing/124795-saffmanmeiliftforce.html#post512967 SaffmanMeiLiftForce - post #9] | * Figuring out how the {{tt|SaffmanMeiLiftForce}} Lagrangian model can be used: [http://www.cfd-online.com/Forums/openfoam-pre-processing/124795-saffmanmeiliftforce.html#post512967 SaffmanMeiLiftForce - post #9] | ||
** It also includes how to figure out why the solver was then crashing when using this particular model. | ** It also includes how to figure out why the solver was then crashing when using this particular model. | ||
+ | |||
+ | |||
+ | == Other instructions found online == | ||
+ | |||
+ | The following are links to pages and documents on other websites that also discuss this topic: | ||
+ | * Presentation [http://www.tfd.chalmers.se/~hani/kurser/OS_CFD_2008/debugging.pdf Debugging OpenFOAM implementations with GDB], by based on material from Dr.Fabian Peng-Kärrholm, 2007, for the PhD course in CFD with OpenSource software, Quarter 2, 2008 at Chalmers. | ||
[[Category:Develop]] | [[Category:Develop]] |
Latest revision as of 08:15, 20 September 2018
Contents
1 Motivation
If your application crashes it will usually output a stack trace, e.g.
#0 Foam::error::printStack(Foam::-Ostream&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"
#1 Foam::sigFpe::sigFpeHandler(int) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"
#2 Uninterpreted: [0xb7f8b420]
#3 Foam::divide(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libOpenFOAM.so"
#4 void Foam::divide<foam::fvpatchfield,>(Foam::GeometricField<double,>&, Foam::GeometricField<double,> const&, Foam::GeometricField<double,> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so"
#5 Foam::tmp<foam::geometricfield<double,> > Foam::operator/<foam::fvpatchfield,>(Foam::tmp<foam::geometricfield<double,> > const&, Foam::GeometricField<double,> const&) in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so"
#6 Foam::turbulenceModels::kEpsilon::correct() in "/home/<user>/OpenFOAM/OpenFOAM-1.4.1/lib/linuxGccDPOpt/libincompressibleTurbulenceModels.so"
There is lots of interesting information in there. It shows the type of error (sigFpe which means a division by zero or any other operation causing an invalid floating point number) and who caused it (operator/ of an fvPatchField). Further down is the origin, kEpsilon::correct(), which obviously does some divisions. A good guess is that one of the patch fields of k or epsilon is 0.
From experience sigfpe originate from three sources:
- as above - division by 0 from having an initial field set to 0.
- when using floatTransfer = 1. This will truncate doubles into floats before doing parallel transfer so if the double does not fit it will produce a sigfpe. Check the traceback for a call to 'compressedSend'.
- when using FOAM_SETNAN (initialises allocated memory to NaN) and accessing uninitialised memory.
The other common error is a segmentation violation (sigSegv) which is caused by an application accessing memory outside the allocated space. These are nearly always caused by a programming error.
2 FULLDEBUG - libraries
To enable the most thorough level of debugging, you'll have to recompile OpenFOAM with the debug switch enabled. To do so, set WM_COMPILE_OPTION=Debug. To go back to a normal build, set WM_COMPILE_OPTION to 'Opt'. You can set WM_COMPILE_OPTION in the bashrc file of OpenFOAM, or temporarily using 'export WM_COMPILE_OPTION=Debug', which will only last for the life of the current terminal session. Keep in mind that though very useful, a full debug build will double up the hard disk space that OpenFOAM needs and run much slower.
3 Top-Level debugging
If you want to find out more but not create a complete debugging build.
- Find out from the printed stack trace which files contain the functions that crash. Copy these into your local directory.
- Add the files to your Make/files
- in Make/options, add
-DFULLDEBUG -g -O0
to EXE_INC and recompile. The FULLDEBUG-flag causes amongst others full range checking on Lists.
In order to go step by step through the sources of the full debug objects, you'll need a debugger.
4 Tools
4.1 Serial debuggers
4.1.1 gdb
Can be invoked on the command line like
gdb xxxFoam
Note: You can further extend functionality in gdb by using this contribution: Contrib_gdbOF
4.1.2 nemiver
Is a nice GTK+ based GUI frontend for gdb. Your solver can be launched like
nemiver xxxFoam <FoamOptions>
4.1.3 ddd
Is another more complex frontend for gdb. You can launch your solver with the following command
ddd --args xxxFoam <FoamOptions>
4.1.4 Limitations
gdb seems to have problems to step into expressions like
return autoPtr<basicThermo>(cstrIter()(mesh));
4.2 Parallel debuggers
4.2.1 mpirunDebug
Is a bash script which can start each process of the parallel run in an extra gdb session. This script can easily extended to start a gdb frontend for each process (download patched mpirunDebug file). Once this is done you'll get a separate GUI instance for each process, where you can set breakpoints etc. separately. This behaviour is similar to Totalview. Maybe one can utilise the session features from the GUI's in order to remember e.g. breakpoints.
mpirunDebug -np 2 xxxFoam -parallel
4.2.2 Eclipse PTP
Eclipse PTP - Parallel Tools Platform [1] is an open-source platform that provides a highly integrated environment specifically designed for parallel application development. In parallel it provides and manages a graphical user interface to a number of serial gdb processes.
4.2.3 Totalview
Totalview is a commercial debugger with many features. It can debug your application in parallel out of the box.
5 Additional Info
5.1 Manual Assistance for Breakpoints
Sometimes it might be helpful to set an endless loop somewhere into solver, and change the variable inside the debugger after launching. This is similar, but less nice, to setting a breakpoint.
int myi = 0; while (0 == myi) Foam::sleep(5);
5.2 Getting built-in feedback from OpenFOAM
OpenFOAM provides a built-in feature for providing debug output on-demand. This section will detail how and where these options can be found and subsequently used.
5.2.1 Looking at the source code
For example, check the source code for polyMesh.C (see online OpenFOAM-2.1.x/src/OpenFOAM/meshes/polyMesh/polyMesh.C at Github), where the following code snippets can be found:
- Near the top, you'll find this piece of code:
defineTypeNameAndDebug(Foam::polyMesh, 0);
It defines that this class Foam::polyMesh should have the type name "Foam::polyMesh" associated to the class and that it should start by default with the debug flag at 0.
- At around the line #866 you'll find this piece of code:
if (debug) { WarningIn("const labelList& polyMesh::tetBasePtIs() const") << "Tet base point indices not available. " << "Forcing storage of base points." << endl; }
Notice the variable debug? This variable is the one that was implicitly initialized in the previous code block. Whenever debug is set to a value different than 0, it will trigger this if block and therefore issue this warning. Said warning will appear while an OpenFOAM solver or utility executed and only if there is a reason for this warning, as indicated in the line #864:
if (tetBasePtIsPtr_.empty())
- Wondering where the macro defineTypeNameAndDebug is defined in the first place? It is defined in the file src/OpenFOAM/db/typeInfo/className.H (see online OpenFOAM-2.1.x/src/OpenFOAM/db/typeInfo/className.H at Github)
5.2.2 Global location to turn on the built-in debug flags
First of all, there is a global file where these flags can be defined: it is etc/controlDict, located in the main OpenFOAM folder. For example, one such file can be seen online for OpenFOAM 2.1.x: OpenFOAM-2.1.x/etc/controlDict at Github.
In this example file, search for the following block of code:
DebugSwitches { Analytical 0; APIdiffCoefFunc 0; Ar 0; //... it's a long list, so I'll snip most of it ... zoneToCell 0; zoneToFace 0; zoneToPoint 0; }
Each one of those lines refer to a class that has a pre-defined debug variable.
For the example given in the previous section Looking at the source code, follow these steps:
- Run a solver or utility and send the output to a log file. For example (using the tutorial incompressible/icoFoam/cavity, after blockMesh was executed), run:
icoFoam > log.icoFoam.before
- Find this line in the etc/controlDict file:
polyMesh 0;
- Then change 0 to 1 and run the solver or utility once again. For example:
icoFoam > log.icoFoam.after
Note: If you do not have permissions to edit the global file, check the next section Other locations to turn on the built-in debug flags. - Compare the two files and there should be some small differences in the outputs. For example, the following command can be used for creating a comparison of the files:
diff -Nur log.icoFoam.before log.icoFoam.after > log.icoFoam.diff
Note: this assumes that the diff application is installed in your machine. - From the previous diff command, the file log.icoFoam.diff should have been created and should have for example the following block of text:
@@ -22,6 +22,7 @@ Create mesh for time = 0 +void polyMesh::initMesh() : initialising primitiveMesh Reading transportProperties Reading field p
The line that starts with the plus sign, indicates the new line, which is clearly an output provided by the polyMesh class, more specifically by the initMesh() method, which can be found inside the file mentioned in the previous section Looking at the source code.
5.2.3 Other locations to turn on the built-in debug flags
Since having to edit the global location can be a serious problem, specially in machines where root (super-user) permissions are required, then there are a few other locations where these flags can be turned on:
-
At least as of OpenFOAM 2.0, there is a folder dedicated to personal environment configurations, located at $HOME/.OpenFOAM/$WM_PROJECT_VERSION/, which can be created by running the following command:
mkdir -p $HOME/.OpenFOAM/$WM_PROJECT_VERSION
Copy the global controlDict to this folder by running:
cp $WM_PROJECT_DIR/etc/controlDict $HOME/.OpenFOAM/$WM_PROJECT_VERSION/
Then feel free to edit this new copy, which can be located by running:
echo $HOME/.OpenFOAM/$WM_PROJECT_VERSION/controlDict
Note: As of OpenFOAM 2.0.x, partial personal controlDict files can be used, so instead of using a complete copy of the controlDict file, one can merely define the following inside the personal file:
DebugSwitches { polyMesh 1; }
-
In addition to the previous feature, as of OpenFOAM 2.2.0, one can set these flags directly inside a case's system/controlDict file, as documented at the official website: OpenFOAM v2.2.0: Run-time Control
-
The personal controlDict file was deprecated in favour of setting the environment variable named FOAM_GLOBAL_CONTROLDICT. For example, to reproduce the previous behavious, set this variable by running:
export FOAM_GLOBAL_CONTROLDICT=$HOME/.foam/3.0/controlDict
But keep in mind that this allows to place this controlDict file with any name and anywhere, for example, directly in the case's folder:
export FOAM_GLOBAL_CONTROLDICT=./controlDictDebugFlags
In addition, if the file is absent and the flag is set, it will look for the standard file in the standard locations; this is useful if you wish to set this as the personal default file name and location for case-by-case debug options. For further information, see this thread: controlDict in foam-extend-3.0
-
The default controlDict file was also deprecated in favour of setting the flags directly when calling the application. For example:
foamJob -s -p simpleFoam -OptimisationSwitches commsType=nonBlocking
For further information on this feature, see this commit: FEATURE: Global control switches can now be used as command line parameters.
6 Step-by-step examples
There are a few examples online demonstrating how this can be done. A few examples are listed here:
- Figuring out how the SaffmanMeiLiftForce Lagrangian model can be used: SaffmanMeiLiftForce - post #9
- It also includes how to figure out why the solver was then crashing when using this particular model.
7 Other instructions found online
The following are links to pages and documents on other websites that also discuss this topic:
- Presentation Debugging OpenFOAM implementations with GDB, by based on material from Dr.Fabian Peng-Kärrholm, 2007, for the PhD course in CFD with OpenSource software, Quarter 2, 2008 at Chalmers.