Correct way to plot rather large amount of data lazarus / 55-TAChart ---------------------------------------------------------------------- Kwadrum · 2020-08-11 00:33 Hi, I have a question on how to plot rather large amount of data correctly. The program I’ve made basically reads raw data from a file, does calculations and the output is two arrays of numbers, from which one is small (~20 numbers) and the other is quite big (several hundred to a thousand numbers). I want to plot the both datasets on separate graphs. My first guess would be to use UserDefined ChartSource, as described here: https://wiki.freepascal.org/TAChart_Tutorial:_Userdefined_ChartSource. But I have a problem that all the calculations are performed inside the procedure which is a handler for a “Start” button and therefore the output data arrays are currently local. The handler for the OnGetChartDataItem needs access to the data, so I made those output data arrays global variables (which works for now, but maybe is not a very good idea). My question is how should it be done correctly? Is it possible to make the OnGetChartDataItem handler nested in the main calculation procedure (probably not, but still)? I do not have much experience, so kind of lost so far… Thank you. ---------------------------------------------------------------------- wp · 2020-08-11 07:51 The UserDefinedChartSource provides an interface to make data of any structure accessible by a chart series. For example, when you perform a calculation and have the calculated data in an array then you can plot these data directly (by means of the UserDefinedchartSource); the conventional "series.AddXY()" would dupliate the data in an internal listsource of the series and waste memory. But of course, the array must exist as long as the series exists. If you have your data in a local array you must either make the array "global" (you can also add them to the corresponding form of the application), or you should use the internal chartsource of the series or an external ListChartSource. I think there is no clear right or wrong; it depends on what you are doing and what you want to achieve. The ListChartSource reserves space for additional information (multiple x, y values, individual point color, data point label), i.e. storing data in simple x/y arrays and using a UserDefinedChartSource is more memory efficient. And if you want a string grid to display the plotted values you'll have the data in memory a strings another time. Morover, to me it is also clearer to separate the data from the series. Data in ListChartSources, however, can be added simply by calling the series' AddXY() method. Data can be sorted directly (well, using a UserDefinedChartSource you can do this too, but you need Laz trunk for it). My personal strategy: Use the (internal/external) ListChartSource for "simple" plots and applications. More sophisticated programs where the same data are used at several places require data in their own data structures and UserDefinedChartSources. ---------------------------------------------------------------------- Kwadrum · 2020-08-11 23:41 WP, thank you for detailed explanations, So if I understood it correctly, it is advisable to use UserDefinedChartSource in order to avoid moving (raw) data into other types of sources (i.e. no duplication of big volumes of data). I am not planning to do anything specific with the plot (no sorting, etc), so I won't need any additional options provided by, say, ListChartSource. Then if I have an array with data, generated in a procedure which in turn is a handler for one of the buttons (and I want to keep it like this), is making this data array "global" the only reasonable option for plotting via UserDefinedChartSource? I guess I can't nest UserDefinedChartSource0GetChartDataItem handler into a button handler (to make it "local" like the data array), right? And I do not want to copy the data from this array into other data source either. Thank you. ---------------------------------------------------------------------- wp · 2020-08-12 07:57 So, if you do not want to use TListChartSource and want to keep your data arrays local to the button-click handler then you only have the option to create the plot there and destroy it when you exit the click handler. This is because the chart must be able to redraw itself at any time, amd for this it needs the data. As a consequence your program will not be able to do anything else while the click handler runs. OK - there is another option to make the chart more persistent: Draw it to a bitmap and display the bitmap after exiting the click handler. But you give up many advantages of the living chart: redraw when it is resized, zooming, panning, etc. ---------------------------------------------------------------------- Kwadrum · 2020-08-12 20:46 WP, thanks for the explanations! As I mentioned, right now everything seems to work with data array made "global" and I'm kind of OK with that. Here is a heavily stripped excerpt of the code, but I hope it is clear enough. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Main; {$mode objfpc}{$H+} interface uses  Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls, ExtCtrls,  Menus, ComCtrls, TAGraph, TASeries, TASources, StrUtils, Math, DateUtils,  MMSystem, TACustomSource; type   { TfMain }   TfMain = class(TForm)    Chart1: TChart;    Chart1BarSeries0: TBarSeries;        .......    UserDefinedChartSource0: TUserDefinedChartSource;        .......    procedure bRunClick(Sender: TObject);       //main calculations, filling AR array            procedure UserDefinedChartSource0GetChartDataItem(      ASource: TUserDefinedChartSource; AIndex: Integer;      var AItem: TChartDataItem);        .......          private   public   end; var  fMain: TfMain;  AR: array [0..4,0..4] of cardinal;  ........  implementation {$R *.lfm} { TfMain } ........ procedure TfMain.UserDefinedChartSource0GetChartDataItem(  ASource: TUserDefinedChartSource; AIndex: Integer; var AItem: TChartDataItem);begin  AItem.X:=AIndex;  AItem.Y:=AR[0][AIndex];end;.........procedure TfMain.bRunClick(Sender: TObject);   //main calculations + filling data array AR.........UserDefinedChartSource0.PointsNumber := 5;UserDefinedChartSource0.Reset;                      //bar graph plotting.......... end. However I'm still curious about how it could be done with "local" data array. Is it possible "simply" to move the whole procedure UserDefinedChartSource0GetChartDataItem into the procedure TfMain.bRunClick or do you mean anything else? Thank you. ---------------------------------------------------------------------- wp · 2020-08-12 21:18 Your code seems to be correct. I probably misunderstood you because you were saying "output data arrays are currently local". But instead of a global array AR I would prefer to have the array within the form because when there are multiple instances of TfMain each one of them would access the same data - probably not what you want.: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type   { TfMain }   TfMain = class(TForm)    Chart1: TChart;    Chart1BarSeries0: TBarSeries;        .......    UserDefinedChartSource0: TUserDefinedChartSource;        .......    procedure bRunClick(Sender: TObject);       //main calculations, filling AR array           procedure UserDefinedChartSource0GetChartDataItem(      ASource: TUserDefinedChartSource; AIndex: Integer;      var AItem: TChartDataItem);        .......         private    AR: array[0..4, 0..4] of Cardinal;  // or move it to "public" when you need the array from somewhere else   public   end; Quote from: Kwadrum on August 12, 2020, 10:46:52 pmIs it possible "simply" to move the whole procedure UserDefinedChartSource0GetChartDataItem into the procedure TfMain.bRunClick No. The OnGetChartDataItem event is declared as "procedure(...) of object" -- this means it must be used as a procedure (with the given parameters) of an object, in other words: as a method of the form (or some other class). In principle, it could also have been declared within TAChart as "nested" (i.e. a "local" procedure within another procedure) but this will not work because the event handler is needed as long as the UserDefinedChartSource exists since the chart must be able to redraw itself and its series at any time; in case of a nested procedure the event handler would no longer be available once the outer procedure has been left. ---------------------------------------------------------------------- Kwadrum · 2020-08-13 00:01 Quote from: wp on August 12, 2020, 11:18:41 pmI probably misunderstood you because you were saying "output data arrays are currently local". Sorry, that were me who was not clear enough. I wrote the "calculation" part of the code separately and then, trying to fit it into the form, realized the problem with local data array and UserDefinedChartSource handler. The first instinctive move was to make the array global, which I did, and it works fine. However, I thought that there might be another way of doing the same thing, while keeping the data array local. Quote from: wp on August 12, 2020, 11:18:41 pmIn principle, it could also have been declared within TAChart as "nested" (i.e. a "local" procedure within another procedure) but this will not work because the event handler is needed as long as the UserDefinedChartSource exists since the chart must be able to redraw itself and its series at any time; in case of a nested procedure the event handler would no longer be available once the outer procedure has been left.That might work for me, actually. So, if I declare procedure UserDefinedChartSource0GetChartDataItem within procedure bRunClick, I think I still will be able to plot the graph from within procedure bRunClick. This is the end of the whole program, and any other manipulations would mean starting the whole program from the very beginning and the current graph won't be needed any more. Not sure if I managed to explain it, though... I can show the whole code or, if I manage to nest UserDefinedChartSource0GetChartDataItem in bRunClick I'll see if the result suits my needs  . Or do I have to declare it somewhere else? Do I need to specifically alter the syntax of declaration of UserDefinedChartSource0GetChartDataItem when it's inside bRunClick or is it considered as a "regular" procedure within another procedure? Thank you. P.S. Sorry for bugging, just really want to know how to make it one and another way. ---------------------------------------------------------------------- wp · 2020-08-13 09:18 Quote from: Kwadrum on August 13, 2020, 02:01:28 amSo, if I declare procedure UserDefinedChartSource0GetChartDataItem within procedure bRunClick, I think I still will be able to plot the graph from within procedure bRunClick. This is the end of the whole program, and any other manipulations would mean starting the whole program from the very beginning and the current graph won't be needed any more. Forget it. The compiler will not allow it because a "nested" procedure is not a "procedure of class" - that's what I tried to explain in the previous post. If you don't believe me try to compile the attached project - you will get compilation error message Code: [Select]unit1.pas(55,72) Error: Incompatible type for arg no. 1: Got "
", expected ""On the other hand, if you select the non-nested event handler the program compiles and runs fine. Quote from: Kwadrum on August 13, 2020, 02:01:28 amThis is the end of the whole program, and any other manipulations would mean starting the whole program from the very beginning and the current graph won't be needed any more. Since you have a GUI program the program flow is determined by user actions, not by your intentions. Even if nothing new may happen in your program after showing the chart the chart will still be visible and the user, for example could drag another window over the chart, and in this case the chart would have to be redrawn. And with every redraw the chart needs the data and, in the case of the UserdefinedChartSource, the OnGetChartDataItem event handler. ---------------------------------------------------------------------- Kwadrum · 2020-08-17 16:14 Thank you, I see now how it works (or, better to say, why it does not). If you do not mind, I have a couple of additional questions. 1. Do I understand it correctly, that in your code Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i UserDefinedChartSource1.OnGetChartDataItem := @UserDefinedChartSource1GetChartDataItem;you point OnGetChartDataItem event to UserDefinedChartSource1GetChartDataItem procedure at run time instead of setting it ahead in the Object Inspector? 2. What is that parameter "ASource", which is not used and pops up in the compiler output as a hint? I understand it is a preset thing that should not be altered and the hints are not errors, but I did not find anything on how this parameter should be addressed (maybe some additional features?). 3. If I, for example, plot several (5-7) series on the same graph, is it any way to address them at ones (or to put them into a cycle) when resetting, etc? I tried a couple of things and walked though the properties/functions in the pop-up assistant in Lazarus, but do not see anything I could use (maybe missing?). What I mean are fragments like these: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Chart1BarSeries0.Active := False;Chart1BarSeries1.Active := False;// + several more UserDefinedChartSource0.PointsNumber := 5;UserDefinedChartSource1.PointsNumber := 5;// + several more with the same value of 5 UserDefinedChartSource0.Reset;UserDefinedChartSource1.Reset;// + several more  I think I can replace ChartBarSeries Active/Inactive with simple hiding the graph (Graph1.Hide), but all the rest look way too long I need this part for a case when the program (without restarting) opens another data file and creates another set of bar series (old series should disappear). Thank you. ---------------------------------------------------------------------- wp · 2020-08-17 17:18 Quote from: Kwadrum on August 17, 2020, 06:14:27 pm1. Do I understand it correctly, that in your code Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i UserDefinedChartSource1.OnGetChartDataItem := @UserDefinedChartSource1GetChartDataItem;you point OnGetChartDataItem event to UserDefinedChartSource1GetChartDataItem procedure at run time instead of setting it ahead in the Object Inspector?Yes, this is the way how to assign an event handler to an event at runtime. Don't miss the '@' when your code is in {$mode objfpc}, but do not use it in {$mode Delphi}. The procedure itself can have any name, it is only required that it is a method (i.e. is declared within the context of a class) and it must have the same parameter list as required by the event. Quote from: Kwadrum on August 17, 2020, 06:14:27 pm2. What is that parameter "ASource", which is not used and pops up in the compiler output as a hint? I understand it is a preset thing that should not be altered and the hints are not errors, but I did not find anything on how this parameter should be addressed (maybe some additional features?).The parameter "ASource" identifies the UserdefinedChartSource which is requesting the data values. Sometimes it is useful to reuse the same event handler for different sources. This way you can determine the source which is currently working. When the warning annoys you you can right-click on the warning message and select an option from the context menu to hide the "unused parameter" warning either for the current line or for the entire unit. Quote from: Kwadrum on August 17, 2020, 06:14:27 pm3. If I, for example, plot several (5-7) series on the same graph, is it any way to address them at ones (or to put them into a cycle) when resetting, etc? I tried a couple of things and walked though the properties/functions in the pop-up assistant in Lazarus, but do not see anything I could use (maybe missing?). What I mean are fragments like these: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Chart1BarSeries0.Active := False;Chart1BarSeries1.Active := False;// + several more UserDefinedChartSource0.PointsNumber := 5;UserDefinedChartSource1.PointsNumber := 5;// + several more with the same value of 5 UserDefinedChartSource0.Reset;UserDefinedChartSource1.Reset;// + several more The chart contains a list of all series, named Series[index]; their count is SeriesCount. However, since TAChart supports many series types, this list returns only the must basic series type, TBasicChartSeries. If you want to address some particular properties of the series you must check the series type and make a type-cast. Please look at the TAChart documentation to learn about the inheritance of the series types (https://wiki.lazarus.freepascal.org/TAChart_documentation#Series). The most elemental series type which supports a chart source is TChartSeries. In the same way you can address the source of the chart series. You can use the following code instead of yours, and it will be valid for all series inheriting from TChartSeries, i.e. TLineSeries, TBarSeries, TAreaSeries etc. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  i: Integer;  ser: TChartSeries;  src: TUserDefinedChartSource;begin  for i := 0 to Chart1.SeriesCount-1 do    if (Chart1.Series[i] is TChartSeries) then    begin      ser := TChartSeries(Chart1.Series[i]);      ser.Active := false;      if ser.Source is TUserDefinedChartSource then      begin        src := TUserDefinedChartSource(ser.Source);        src.PointsNumber := 5;      ser.Reset;      end;    end;end; Quote from: Kwadrum on August 17, 2020, 06:14:27 pmI think I can replace ChartBarSeries Active/Inactive with simple hiding the graph (Graph1.Hide), but all the rest look way too long I need this part for a case when the program (without restarting) opens another data file and creates another set of bar series (old series should disappear).I think hiding and showing the Chart causes some disturbing flicker. In order to remove the old series you can set its Active to false, or you can delete it altogether (FreeAndNil(Chart1BarSeries1)), or you can keep the old series and you just make the data array used by the UserDefinedChartSource point to the new data.