How to find out why the compiler hangs ? lazarus / 62-General ---------------------------------------------------------------------- Nitorami · 2018-01-09 20:03 Today I encountered an issue I never had before. Normally my program compiles within two seconds, then I made a small modification to the Class hierarchy in a bottom unit, and when I now try to build, the compiler keeps hanging in an endless loop, after having compiled all except the topmost unit. It says "Compiling - press ESC to cancel", but does not react to ESC nor anything else. Processor load is at 60% permanently. I reverted the code change and it compiles, I can live with that, that is not the issue. But this should be reported, only, I cannot possibly submit the entire program for diagnosis. Have tried many other compiler settings, target processors etc. but to no avail. Tried changing from objfpc to delphi mode; well, the compiler then does not hang but throws a lot of error messages because my code is not delphi compatible. Have deleted the old .o and .ppu files, and the fp.ini, fp.dsk, fp.cfg, but nothing helps. I shuffled the order of units in the "uses" section but no sucess either. How can I narrow down such a problem ? Each time I try a new compilation with modified settings or whatever, I'll have to kill the IDE and start a new command processor. Getting tired of that. I am using the textmode IDE with FPC3.0.2 and win32. Don't think anything is wrong with my installation, as only this specific code change causes the problem. Will try FPC 3.0.4 next but am not too optimistic. ---------------------------------------------------------------------- Leledumbo · 2018-01-09 20:16 That's one kind of hard to reproduce problem. The compiler might be trapped in an infinite loop somehow (well, it's not in anyway magical). Still, a small source that can reproduce the behavior is expected. If it's hard to write one, then I guess you can help by debugging the compiler yourself, using your project source code. There's a .lpi in the compiler source directory, you can use that for debugging the compiler in lazarus. Maybe it will help if you at least mention what changes to the source code trigger this behavior so a compiler dev can point where to put breakpoint closer to the problem source. ---------------------------------------------------------------------- molly · 2018-01-09 20:26 As TS is not using lazarus and the problem seems related to text mode ide (internal compiler) TS could perhaps try to compile sources using only the commandline compiler. IF you do not have it it would be a separate installation. That would be able to better pinpoint if it is compiler or (only) textmode related. @Nitorami, be aware that compiling generics can take a (very) long while. for instance compiling the delphi generics collection took ages, and it gets worse when there is  something actually wrong in the code. But seeing that when you revert it compiles fine, would probably indicate something else is amiss. Something that is able to tell you where things go wrong is using verbose all option, but i take it that this only points to compiling the unit that contains the problematic code. if not, then please dump that log for us to see. Perhaps the compiler gets stuck somewhere else. Other than that: make a copy of your current code and then start chopping away to see if you are able to reproduce with a smaller example code. ---------------------------------------------------------------------- Nitorami · 2018-01-09 21:01 Yes, certainly the compiler is trapped in an endless loop. I don't use generics so that cannot be the reason for long compilation times. The verbose option seemed like a good idea but as the compiler hangs, the messages are lost But when I compile the units individually, the messages do not seem to indicate anything unusual. Another funny thing is, if I insert a typo at the end of the main program, the compiler complains, but when I correct it and recompile, it hangs. The issue may be close to the linker stage. The textmode IDE displays a compiler progress window, informing about the currently compiled unit and line number. I hoped that this might be useful, but the text is different on each try. I guess the window content is only loosely synchronised to the compilation process. Recompiling and debugging the sources is somewhat beyond my skills I fear. I'll try the "chopping" approach and see whether I can reproduce the issue with much less code. ---------------------------------------------------------------------- Leledumbo · 2018-01-09 21:19 Quote from: Nitorami on January 09, 2018, 10:01:23 pmAnother funny thing is, if I insert a typo at the end of the main program, the compiler complains, but when I correct it and recompile, it hangs. The issue may be close to the linker stage. I'm not too sure about it, if you make a typo that would be on parsing stage. But if the source is syntactically valid, the infinite loop can be anywhere inside the compiler (sub)processes. Also, did you use any options related to code generation (e.g. -O, -Oo, -Op, -Cp, -Cf)? ---------------------------------------------------------------------- Nitorami · 2018-01-09 21:32 QuoteAlso, did you use any options related to code generation (e.g. -O, -Oo, -Op, -Cp, -Cf)? Maybe not all of them but definitely tried the most conservative of all settings, no optimisation, 80386 processor etc. I just found that compilation from the commandline with fpc.exe works. Surprise. I am not familiar with the commandline switches and it took me a while to put something together, but got it compiled after all. Maybe some switches are still different than in the IDE.. not sure. It also compiles with Lazarus, but my installation is 64bit and does not compare. So - the issue might really be related to the textmode IDE only. I am used to it and still love it, but I think it gets little care from the developers. Maybe I should switch to Lazarus, finally ---------------------------------------------------------------------- engkin · 2018-01-09 21:55 Quote from: Nitorami on January 09, 2018, 10:32:21 pmI just found that compilation from the commandline with fpc.exe works. Surprise. I am not familiar with the commandline switches and it took me a while to put something together, but got it compiled after all. Maybe some switches are still different than in the IDE.. not sure. Pass -s in the Options >> Compiler >> Additional compiler args (I guess). It should generate a batch file PPAS.BAT (and link.res) ---------------------------------------------------------------------- Nitorami · 2018-01-09 22:12 I understand -s means "do not call assembler and linker". Still hangs, and does not generate the said files. Tried again with a hello world program, and it does generate a link.res, but also an exe file...1000kB size - what ? Don't understand this - if no assembler and linker is called, how can an exe file be generated ?  EDIT - sorry, had "Call linker after" ticked. Without that, the hello world program has reasonable size, and the said files are generated. But compiling my program still hangs. ---------------------------------------------------------------------- engkin · 2018-01-09 22:15 What about passing -Cn instead? ---------------------------------------------------------------------- Nitorami · 2018-01-09 22:22 Same. Hangs. ---------------------------------------------------------------------- engkin · 2018-01-09 22:23 Ah, I see, you edited your answer. Can you share the change that sparked the problem? ---------------------------------------------------------------------- Nitorami · 2018-01-09 22:30 I could share that, but don't think it helps, as the unit with the change compiles ok on its own. The issue only appears when compiling the entire program, or at least a high level unit. ---------------------------------------------------------------------- Nitorami · 2018-01-10 10:17 Update: Same issue on an office PC at win7 (rather than my home win 10). I found the problem is related to the generation of debug information. It compiles ok without generating debug information, but the -g option reliably reproduces the freeze. When compiling from the command line, I can at least interrupt the process by hitting Ctrl-C, but the IDE hangs. ---------------------------------------------------------------------- Thaddy · 2018-01-10 10:29 Try specifying -B which forces a total rebuild of your project. If you change a base class (in a separate unit from derived classes) that might be the issue. You can not simply change a base class and expect derived classes to work. In any compiled language... If I am right, then kindly hit yourself firmly with a baseball bat...slap on the wrist is also ok... ---------------------------------------------------------------------- Nitorami · 2018-01-10 10:43 No need to hit myself over the head unfortunately, I already deleted all .o and .ppu and did fpc -g -B. The compiler hangs on generating the debug information for a full build. When it finds precompiled units, the issue may not occur. I.e. fpc -B myprogram.pas fpc -g myprogram.pas is ok. But fpc -g -B myprogram.pas hangs. EDIT: Found a workaround. It works when using the DWARF debug format fpc -B -gw. Only I have no idea what dwarf and stabs mean, and whether I can use the debug features in the IDE in the same way when switching to dwarf. ---------------------------------------------------------------------- Thaddy · 2018-01-10 11:19 In that case it depends on platform: dwarf does not work properly on all platforms, some platforms require stabs.... That is documented. If you leave out such information in your original report I can not test for that. A clean build seems to resolve it in your case. (since -B helps) As it should in the general case where you change a base class as I described. ---------------------------------------------------------------------- Nitorami · 2018-01-10 12:55 Alright, I'll give dwarf a try. Have no other option anyway, except disabling debug info entirely, or changing to Lazarus. @Thaddy: We may have a slight misunderstanding. I did not leave out any information, I just stumbled into this problem, tried to find the root cause, asked for some help with it,  and finally described what I found. Summarising: - The compiler freezes when compiling my code. - This was after I had changed a base class but that is essentially without importance. It has nothing to do with the act of changing the code, or the compiler using a wrong object file or such, or bad code. It happens on a clean fresh build. - The issue is triggered by the specific code, but the code is correct. The units alone compile fine. There are no error messages when compiling the whole program, but just before generating the exe, the compiler feezes. Except when disabling debug info generation or using dwarf format for debug information. -> The compiler has a bug which makes it freeze when compiling this specific piece of code with debug information in default format (presumably stabs). I would file this as bug report but I cannot throw the entire program at the developers, and I did not succeed in chopping it down to a small piece of code which still reproduces the issue. ---------------------------------------------------------------------- jamie · 2018-01-10 23:14 Have you tried "-Xg"  Create External Debug file ? ---------------------------------------------------------------------- Noodly · 2018-01-10 23:43 Quote from: Nitorami on January 09, 2018, 09:03:06 pmI cannot possibly submit the entire program for diagnosis. Well that's obviously not true. You might be worried about IP theft or you happen to be writing the ICBM guidance code for Kim Jong Un and he has a tank on your lawn keeping an eye on you, but you should trust the people who develop and support FPC/Lazarus and find a way of submitting the code to one of the recognised experts on the forum who I'm sure will step forward to try and help resolve this. ---------------------------------------------------------------------- molly · 2018-01-11 01:01 Quote from: Nitorami on January 09, 2018, 10:01:23 pmThe verbose option seemed like a good idea but as the compiler hangs, the messages are lost Oops, yes of course my bad  Nothing of interest to see what does show up ? using -g would indicate to use the debug format as is common for the target. No reason to doubt that ? It seems the strangest of things you've ran into, and haven't experienced that myself (but i'm using fp-ide less and less these days). Are you able to reproduce with 3.0.4 (or perhaps even trunk) ? ---------------------------------------------------------------------- Nitorami · 2018-01-11 17:29 QuoteQuote from: Nitorami on January 09, 2018, 10:01:23 pm     The verbose option seemed like a good idea but as the compiler hangs, the messages are lost Oops, yes of course my bad  Maybe not quite; as this turns out to be related to the complier rather than the IDE, I can compile from the commandline and pipe the compiler messages into a text file, such as "fpc -B -g myprogram.pas > bugger.txt"; once the compiler hangs, I can interrupt it with Ctrl-C, and the OS will write the messages to the text file. However there is nothing unusual to find, and I would  not expect anything - the compile is ok, it's only building the exe file with debug information which fails. Quoteusing -g would indicate to use the debug format as is common for the target. No reason to doubt that ?Not quite sure what you say. I understand that -g generates debug info in standard format for win32 (I think stabs, whatever this is - it hangs), while -gw forces dwarf format (whatever...it works ok). QuoteAre you able to reproduce with 3.0.4 (or perhaps even trunk) ?Yes, same with 3.0.4. Not tried trunk. QuoteHave you tried "-Xg"  Create External Debug file ?Yes, same. Hangs before writing the .dbg file. But works with dwarf. QuoteYou might be worried about IP theft or you happen to be writing the ICBM guidance code for Kim Jong Un and he has a tank on your lawn keeping an eye on you, Well my life does not depend on writing software, and it is less a matter of trust but rather the fpc devs usually asking for a small and comprehensive piece of code to reproduce the issue. But the case may be different, I'll ask on the mailing list. ---------------------------------------------------------------------- soerensen3 · 2018-01-11 20:57 I was having a similar problem myself until yesterday and now the problem magically disappeared. Maybe it had to do with switching the debugging info to automatic but I'm not sure maybe it also was some heavy use of generics which in the end I did not really need. Those compiler/debugger errors can be so frustrating, for me it is mostly guessing. For me first the compiler crashed on both Linux and Windows and later I somehow resolved that but then the debugger was failing. ---------------------------------------------------------------------- Nitorami · 2018-01-14 10:53 I spend the entire morning stripping down the code. Attached a bare skeleton, which still hangs on compilation. If anyone wants to try, simply use Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i fpc -Mobjfpc -B -g fta_analyzer.pas with FPC3.0.2 or 3.0.4, win32. Mode Delphi will hang as well. The problem is not in the compilation, but the generation of debug information in default format (-g). It seems this has to do with how my Classes are nested. It works in dwarf format (-gw), though. ---------------------------------------------------------------------- Leledumbo · 2018-01-14 16:00 The issue seems to be platform specific as well, using fpc 3.0.4 on x86_64-linux with the same exact compiler options doesn't hang the compiler. And yes, even on trunk if I target i386-win32 it hangs. ---------------------------------------------------------------------- PascalDragon · 2018-01-14 16:49 Quote from: Nitorami on January 14, 2018, 11:53:28 amI spend the entire morning stripping down the code. Attached a bare skeleton, which still hangs on compilation. Thank you for redrucing the test case. With this I was able to reproduce your issue and fix it in revision 37969 (trunk only) [and even if it wouldn't be fixed, I added an internal error in r37968 so that the compiler won't loop ad infinitum anymore for this problem ) As a work around for 3.0.x you'll have to either use DWARF (don't know whether the 32-bit Windows GDB likes DWARF) or switch to the 64-bit compiler as that only uses DWARF. ---------------------------------------------------------------------- molly · 2018-01-15 01:49 @nitorami: Thank you for having taken the time to produce a reduced example. Worked like a charm (or perhaps better worded: made the compiler hang). @PascalDragon That seems a nasty one. Thank you for addressing it.