WM_NCOMPPROCS and full optimization

Hi Bruno!

Don't want to add noise to your tip about WM_NCOMPPROCS=4 for compilation. But that may be problematic for some people: certain compilers (for instance CLang) use HUGE amounts of memory when doing full optimiziation. The files generated by the grammars are very large and the problem is that three grammars from the base library get compiled at approximately the same time. I've seen cases where this forces machines with medium amount of memory (8Gig to be precise) into swapping and making them almost non-responsive. So I don't endorse this tip. I'd rather have people go "took me long to compile" than "compilation frooze my machine" (and I think I remember at least one such instance on the MessageBoard)

Bernhard

• More

• History
• View source
• Link to

Bgschaid (talk)‎12:21, 5 October 2013

Hi Bernhard,

Mmm... and I thought that .NET was annoying enough... I've never used CLang myself, so I had no idea it could be such a memory hog :(

Then how about an if and only if indication that one can only use this option if using GCC or ICC? Something like:

Note

If you are using a GCC or ICC compiler, then before running one of the commands above, you can run this command:

... and so on.

Best regards, Bruno

• Parent
• More

• History
• View source
• Link to

Wyldckat (talk)‎23:01, 5 October 2013

You do not have permission to edit this page, for the following reasons:

• The action you have requested is limited to users in the group: Users.
• You must confirm your email address before editing pages. Please set and validate your email address through your user preferences.

---

You can view and copy the source of this page:

The problem is in my opinion not that CLang is that bad. On the contrary: I think the problem is that it tries to optimize the output of bison when other compilers don't. At least in the past a sure-fire way to get your code NOT (or only superficially) optimized was to add goto to it. Most optimization algorithms assume that there is only one way to enter a code block and if you use goto you break that promise and most compilers say "OK. I will not even try to optimize this garbage". The bison/flex code contains quite a number of AND the source files generated are quite large. CLang seems to try to optimize them with O3. The memory usage seems to vary for different versions of CLang. Ah. And by the way: the error messages of CLange are MUCH more readable than the stuff g++ prints

Return to Thread:Talk:Contrib/swak4Foam/WM NCOMPPROCS and full optimization/reply (2).

Nice! I already knew that CLang provides a better warning/error messages, as well as helping find some issues without the need of some other lint software, but I had no idea that it tried to optimize the output from bison et al!

OK, I'll update the note with this in mind.

• Parent
• More

• History
• View source
• Link to

Wyldckat (talk)‎11:41, 6 October 2013

The problem is that optimizing the parser for swak is futile: the time spent doing the actual parsing is small compard to the actual "OF-work": calculations on the fields. That's why I added the special optimiziation switch for the parsers.

PS: and this "tries to optimize goto-stained code" is only a theory (I haven't looked at the source. But its behaviour - certain CLang versions freeze a 8 Gig machine when compiling FieldParser with O3 - make me pretty sure that this is the case)

• Parent
• More

• History
• View source
• Link to

Bgschaid (talk)‎12:11, 6 October 2013