[Partially Solved] My main form crashes Lazarus IDE
lazarus / 9-Editor

----------------------------------------------------------------------
lainz · 2018-12-12 22:09

Hi, In the project I'm working I have a huge form (2 mb) that when
it's opened it crashes the Lazarus IDE with 'access violation'.

If the form is closed Lazarus works normally.

How I can debug that? The ide in some time was at 900 MB of RAM usage,
so it's leaking somewhere.

----------------------------------------------------------------------
wp · 2018-12-12 22:18

If you are working with Laz trunk: This version has been leaking for a
few days recently. Try a more recent trunk version.

----------------------------------------------------------------------
lainz · 2018-12-12 22:28

Thanks I will try, I'm using trunk. It happens about a month ago or
more, so I wish this can be solved or if I can debug it in some way.

----------------------------------------------------------------------
Martin_fr · 2018-12-12 23:00

There are several ways to debug the IDE.

First of all for any of them make sure you have debug info. In Tools >
Configure build Lazarus
-gw -godwarfset -O- -gh
-Sa -gt -Criot

the 2nd line is optional.
Make sure this applies to any package you use. (installed in the IDE)


1) run the IDE with a logfile (on windows edit properties of a
shortcut)
lazarus.exe --debug-log=/path/file.log

Crashes will likely write logs to the file

2) If you need very basic info only, or no trace was written in (1)
start gdb on commandline
  gdb lazarus.exe
and then use "r" to run it.
"bt" to get a trace when it stops

3) In the ide open the project ide/lazarus.lpi
Hit F9
A 2nd IDE will be started and is being debugged by the first.

------
Since you know aprox when it started, consider bisecting the guilty
revision

----------------------------------------------------------------------
lainz · 2018-12-16 14:21

Ok what I get is this:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
TApplication.HandleException: EOutOfMemoryOut of memory  Stack trace: 
$00413B9A  $00413EDA  $0061B2B2  ALLOCDATA,  line 329 of
include/imglist.inc  $0061BF46  INTERNALINSERT,  line 626 of
include/imglist.inc  $0061E112  ADDSLICE,  line 1390 of
include/imglist.inc  $0062227A  ADDNEWBTNIMAGE,  line 2745 of
include/imglist.inc  $00621F76  GETIMAGEINDEX,  line 2781 of
include/imglist.inc  $007F96CD  GETIMAGEINDEX,  line 290 of
ideimagesintf.pas  $008494C8  $008663DE  $00875CE8  $0043A61E
 GETCHILDREN,  line 1072 of include/customform.inc  $008755B0 
$00875E7A  $008760E7  $00875F59  $008785B8ERROR in code: Creating gdb
catchable error:   $004E4BC4  DEBUGLNSTACK,  line 868 of
lazloggerbase.pas  $004E20CC  DEBUGLNSTACK,  line 12 of
LazLoggerImpl.inc  $004E61E7  RAISEGDBEXCEPTION,  line 48 of
laztracer.pas  $00CDBECA  $00CDBE78  $008712E7  $0087407E  $005A339D
 WNDPROC,  line 2186 of include/control.inc  $005A1D2A  PERFORM,  line
1580 of include/control.inc  $00595CF4  PAINTCONTROLS,  line 4959 of
include/wincontrol.inc  $00595AE0  PAINTHANDLER,  line 4871 of
include/wincontrol.inc  $00599D2B  WMPAINT,  line 6841 of
include/wincontrol.inc  $005ACD4D  WMPAINT,  line 98 of
include/customcontrol.inc  $00410449  $0087407E  $005A339D  WNDPROC,
 line 2186 of include/control.inc  $00596E2B  WNDPROC,  line 5425 of
include/wincontrol.inc  $0068472A  DELIVERMESSAGE,  line 112 of
lclmessageglue.pas  $00572FDE  SENDPAINTMESSAGE,  line 710 of
win32/win32callback.inc  $0057749D  DOWINDOWPROC,  line 2297 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AABECA  $76AABC57 
$723A5F23  $73489FBF  $734C0F7E  $734C0E64  $7346FA96  $76ACBF1B 
$76AC83EA  $76AA7AFD  $00571DD9  CALLDEFAULTWINDOWPROC,  line 97 of
win32/win32callback.inc  $005779E6  DOWINDOWPROC,  line 2445 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AC7F8A  $76ACA6D9 
$7748CD3D  $6B677DE6  $76ABA989  $7748CD3D  $76AC90A4  $76AA8FB3 
$76AA4B0A  $76AA4A22  $76AA49DB  $7357D5A3  $734A6A35  $734A7526 
$00583A3B  PROMPTUSER,  line 773 of win32/win32lclintf.inc  $0052F1AD
 PROMPTUSER,  line 376 of include/lclintf.inc  $0052F7EF  PROMPTUSER,
 line 535 of include/lclintf.inc  $00446DD0  SHOWEXCEPTION,  line 1591
of include/application.inc  $0078A169  HANDLEAPPLICATIONEXCEPTION,
 line 28 of lclexceptionstacktrace.pas  $00445EC0  HANDLEEXCEPTION,
 line 1258 of include/application.inc  $004465EB  RUNLOOP,  line 1421
of include/application.inc  $0048F2EF  APPRUN,  line 54 of
include/interfacebase.inc  $00446556  RUN,  line 1407 of
include/application.inc  $00403A9B  main,  line 152 of lazarus.pp 
$00418C76ERROR in code: Creating gdb catchable error:   $004E4BC4
 DEBUGLNSTACK,  line 868 of lazloggerbase.pas  $004E20CC
 DEBUGLNSTACK,  line 12 of LazLoggerImpl.inc  $004E61E7
 RAISEGDBEXCEPTION,  line 48 of laztracer.pas  $00CDBECA  $00CDBE78 
$008712E7  $0087407E  $005A339D  WNDPROC,  line 2186 of
include/control.inc  $00596E2B  WNDPROC,  line 5425 of
include/wincontrol.inc  $0068472A  DELIVERMESSAGE,  line 112 of
lclmessageglue.pas  $00572FDE  SENDPAINTMESSAGE,  line 710 of
win32/win32callback.inc  $0057749D  DOWINDOWPROC,  line 2297 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AABECA  $76AABC57 
$723A5F23  $73489FBF  $734C0F7E  $734C0E64  $7346FA96  $76ACBF1B 
$76AC83EA  $76AA7AFD  $00571DD9  CALLDEFAULTWINDOWPROC,  line 97 of
win32/win32callback.inc  $005779E6  DOWINDOWPROC,  line 2445 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AC7F8A  $76ACA6D9 
$7748CD3D  $0044542E  UPDATEVISIBLE,  line 992 of
include/application.inc  $004391F6  SETVISIBLE,  line 493 of
include/customform.inc  $0043DFA0  HIDE,  line 2318 of
include/customform.inc  $00438502  BEFOREDESTRUCTION,  line 183 of
include/customform.inc  $00438560  DESTROY,  line 198 of
include/customform.inc  $0040FF42  $00433E3D  BEFOREFINALIZATION,
 line 1935 of forms.pp  $00412C01  $006847A3  DELIVERMESSAGE,  line
116 of lclmessageglue.pas  $00572FDE  SENDPAINTMESSAGE,  line 710 of
win32/win32callback.inc  $0057749D  DOWINDOWPROC,  line 2297 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AABECA  $76AABC57 
$723A5F23  $73489FBF  $734C0F7E  $734C0E64  $7346FA96  $76ACBF1B 
$76AC83EA  $76AA7AFD  $00571DD9  CALLDEFAULTWINDOWPROC,  line 97 of
win32/win32callback.inc  $005779E6  DOWINDOWPROC,  line 2445 of
win32/win32callback.inc  $00578589  WINDOWPROC,  line 2699 of
win32/win32callback.inc  $76ACBF1B  $76AC83EA  $76AC7F8A  $76ACA6D9 
$7748CD3D  $6B677DE6  $76ABA989  $7748CD3D  $76AC90A4  $76AA8FB3 
$76AA4B0A  $76AA4A22  $76AA49DB  $7357D5A3  $734A6A35  $734A7526 
$00583A3B  PROMPTUSER,  line 773 of win32/win32lclintf.inc  $0052F1AD
 PROMPTUSER,  line 376 of include/lclintf.inc  $0052F7EF  PROMPTUSER,
 line 535 of include/lclintf.inc  $00446DD0  SHOWEXCEPTION,  line 1591
of include/application.inc  $0078A169  HANDLEAPPLICATIONEXCEPTION,
 line 28 of lclexceptionstacktrace.pas  $00445EC0  HANDLEEXCEPTION,
 line 1258 of include/application.inc  $0044
As well as the attached image.

----------------------------------------------------------------------
Thaddy · 2018-12-16 14:27

EOutOfMemory is - as in Delphi - pre-allocated so it is able to report
the error and should show a nice exception dialog?
If this is not the case then Lazarus should take the same approach and
pre-allocate the EOutOfMemory dialog beforehand.
Note you still need to allow for EOutOfMemory explicitly.

----------------------------------------------------------------------
lainz · 2018-12-16 14:31

Hi Thaddy, I readed the docs and it says is pre allocated in FPC as
well.

How I can turn it on to display the nice exception dialog? What I get
is a unresponsive IDE that closes itself.

In the log shows that is image list the problem or i'm pointing it
wrong?

Edit: I reported a bug
https://bugs.freepascal.org/view.php?id=34708

I think is image list, it allocates memory in allocdata line 329 and
seems that is not freed.

----------------------------------------------------------------------
lainz · 2018-12-16 23:06

Well, I can't provide a form that replicates the issue so there's no
way to get help in the bugtracker and they're right, they can't
reproduce so can't test.

I will try to split the form in data modules to see if that solves the
problem.

----------------------------------------------------------------------
engkin · 2018-12-17 03:53

You obviously caught a bug. Line 329 is:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TCustomImageListResolution.AllocData(ACount: Integer);... 
SetLength(FData, ACount * FWidth * FHeight * SizeOf(FData[0])); Your
image shows that Lazarus did not reserve more than 240MB.
EOutOfMemory shows that ACount * FWidth * FHeight * SizeOf(FData[0])
is more than what's possible for 32-bit Lazarus.

I would suggest you catch the EOutOfMemory exception around SetLength
and check the values involved in ACount * FWidth * FHeight *
SizeOf(FData[0]). The odd value would give a direction toward the
source of the bug, I guess.

The comments you received in your bug report are disappointing.

----------------------------------------------------------------------
Cyrax · 2018-12-17 10:10

Maybe removing SizeOf(FData[0]) from that line helps. With dynamic
arrays SetLength knows that arrays element type size automatically, so
it doesn't need to be told to it explicitly.

----------------------------------------------------------------------
engkin · 2018-12-17 18:19

Quote from: Cyrax on December 17, 2018, 11:10:59 amMaybe removing
SizeOf(FData[0]) from that line helps. With dynamic arrays SetLength
knows that arrays element type size automatically, so it doesn't need
to be told to it explicitly.

I believe you are right about removing SizeOf(FData[0]). Still
SizeOf(FData[0]) is 4 which makes me think there might be another
problem.

----------------------------------------------------------------------
wp · 2018-12-17 18:59

Quote from: engkin on December 17, 2018, 04:53:22 amYou obviously
caught a bug. Line 329 is:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TCustomImageListResolution.AllocData(ACount: Integer);... 
SetLength(FData, ACount * FWidth * FHeight * SizeOf(FData[0])); Your
image shows that Lazarus did not reserve more than 240MB.
EOutOfMemory shows that ACount * FWidth * FHeight * SizeOf(FData[0])
is more than what's possible for 32-bit Lazarus.You lost me. How do
you know what ACount, FWidth and FHeight are? Nothing has been
mentioned so far. Where do you see the 240MB in the screenshot?

Quote from: engkin on December 17, 2018, 04:53:22 amThe comments you
received in your bug report are disappointing.
Sorry. But what do you expect when we cannot reproduce the issue and
the author cannot provide precise steps how to do that.

----------------------------------------------------------------------
engkin · 2018-12-18 02:14

Quote from: wp on December 17, 2018, 07:59:57 pmYou lost me. How do
you know what ACount, FWidth and FHeight are? Nothing has been
mentioned so far.
I am not sure why I lost you. The stack trace shows:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
TApplication.HandleException: EOutOfMemoryOut of memory  Stack trace: 
$00413B9A  $00413EDA  $0061B2B2  ALLOCDATA,  line 329 of
include/imglist.incEOutOfMemory at line 329.

Line 329 is:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
SetLength(FData, ACount * FWidth * FHeight * SizeOf(FData[0]));
I know ACount * FWidth * FHeight * SizeOf(FData[0]) is too much. I
don't know why, nor what each value is. That is why I suggested to
catch the exception and find these values.

Cyrax spotted the wrong SizeOf(FData[0]) in the call to SetLength. Do
you agree with him? is this line buggy or not?

Quote from: wp on December 17, 2018, 07:59:57 pmWhere do you see the
240MB in the screenshot?
Check the attached image.

----------------------------------------------------------------------
lainz · 2018-12-19 20:43

When using 64 bit IDE it goes worst, it uses more than 1 gb of ram and
then crashes.

But, I found the problem, I have a progress bar with style marquee on
that form, and seems that when that animation is running even if I'm
designing in the IDE, so it refreshes the GUI a lot, and for sure
there's something causing the out of memory error, is a big form as I
already say.

To solve I set the style to normal and set marquee by code. But memory
still growing over time, not too much and the IDE has not crashed now.

Tried to create another form to replicate and did not work, there must
be a component that uses image list and causes this issue. But for now
I'm good.

----------------------------------------------------------------------
wp · 2018-12-20 09:04

Quote from: engkin on December 18, 2018, 03:14:32 amI am not sure why
I lost you. Thanks for clarification, I now understand your arguments.

I tried to remove the FData[0] in the AllocData method; the effect is
that the IDE hangs when restarting after building. The
TCustomImageListResolution makes use of both byte-wise
(System.Move(...) ) and element-wise memory (FData[0]) access to the
FData array. Something else must be wrong, too.

----------------------------------------------------------------------
Ondrej Pokorny · 2018-12-20 13:55

Quote from: wp on December 20, 2018, 10:04:02 amQuote from: engkin on
December 18, 2018, 03:14:32 amI am not sure why I lost you. Thanks for
clarification, I now understand your arguments.

I tried to remove the FData[0] in the AllocData method; the effect is
that the IDE hangs when restarting after building. The
TCustomImageListResolution makes use of both byte-wise
(System.Move(...) ) and element-wise memory (FData[0]) access to the
FData array. Something else must be wrong, too.

The AllocData method has 2 bugs:
1.) SetLength(FData, ACount * FWidth * FHeight * SizeOf(FData[0]));
2.) Inc(FAllocCount, ACount);

It should read:
1.) SetLength(FData, ACount * FWidth * FHeight);
2.) FAllocCount := ACount;

If you fix both, your IDE should work fine. The bugs have been in the
LCL for years (I checked 1.4.4 and they are there) - strange that
nobody noticed them, including myself when I rewrote the imagelist in
1.9.

Btw. I wanted to send a patch to Mantis but you closed
https://bugs.freepascal.org/view.php?id=34708 as "no change required"
:/

Code: [Select]procedure TCustomImageListResolution.AllocData(ACount:
Integer);
var
  n: Integer;
begin
  if FAllocCount >= ACount
  then Exit;

  // calculate number of blocks, add an extra block for the remainder.
  n := ACount mod FImageList.FAllocBy;
  if n <> 0
  then Inc(ACount, FImageList.FAllocBy - n);

  SetLength(FData, ACount * FWidth * FHeight);

  FAllocCount := ACount;
end;

----------------------------------------------------------------------
lainz · 2018-12-20 14:58

Ok I reopened it

----------------------------------------------------------------------
wp · 2018-12-20 16:55

I applied Ondrej's patch - thank you. I understand that the bug was
responsible for allocation of 4 times more memory than needed. But
does this explain the gradual allocation of memory consumption
reported by lainz?

----------------------------------------------------------------------
Ondrej Pokorny · 2018-12-20 17:09

Quote from: wp on December 20, 2018, 05:55:35 pmBut does this explain
the gradual allocation of memory consumption reported by lainz?

No, not directly.

----------------------------------------------------------------------
zeljko · 2018-12-21 16:16

r59880 introduced memory corruption. See why in r59888. In any case
r59880 and r59888 should be reverted back until it's properly fixed
because now some imagelists (haven't tested all scenarios yet), does
not work as expected.
Edit: without r59888 win32 works fine, but gtk2, qt and qt5 crashes
because of memory corruption. With r59888, there's no crashes, but
some bitmaps are invisible when loaded from imagelist - reproducable
on win32 (via wine), gtk2 and qt/qt5.