[solved] adding text notes to charts lazarus / 55-TAChart ---------------------------------------------------------------------- Muso · 2022-07-11 17:02 I wrote some programs that display live sensor data. Since it is live, incidents can occur. When there is for example a loose contact of a cable, I see it in the chart and fix the cable. However, 2 weeks later I don't know this anymore or a colleague looks at the chart and wonders about e.g. a sudden high peak in the chart. I thought about it for a while and think that a kind of a text note would help me. So for example then I get a sudden peak, I can add a note like "loose cable, already fixed".The note must be attached to a certain x value so that it is at he right position when zooming in/scrolling etc. I found the marks feature and its example file: https://wiki.freepascal.org/TAChart_documentation#Marksand that is what I need I think. however, I cannot find a way, than I can click into the chart on a data point so that a dialog pops up that I can add a mark. I am pretty sure this is somehow doable so any hint would be nice. (In my charts I use a lot of tools, see attached and maybe I block myself somehow.) ---------------------------------------------------------------------- wp · 2022-07-11 18:14 Use a TDatapointClickTool and assign it to the double-click, for example (*). In its OnPointClick handler you can determine which data point and which series was double-clicked, you can prompt for a message text, and assign it to the Text element of the data point. And when the series' Marks.Style is at the smsLabel setting, this text will be overlaid on the chart. See the attached demo. (*) You may have to add other Shift modifiers to distinguish this tool from the effect of other tools. ---------------------------------------------------------------------- Muso · 2022-07-11 18:44 Quote from: wp on July 11, 2022, 08:14:43 pmSee the attached demo. Many, many thanks. Works perfectly and is indeed exactly what I was looking for!  Quote from: wp on July 11, 2022, 08:14:43 pm(*) You may have to add other Shift modifiers to distinguish this tool from the effect of other tools. Yes, this is a surprisingly hard task since I use so many tools. ---------------------------------------------------------------------- wp · 2022-07-11 20:42 Quote from: Muso on July 11, 2022, 08:44:12 pmQuote from: wp on July 11, 2022, 08:14:43 pm(*) You may have to add other Shift modifiers to distinguish this tool from the effect of other tools. Yes, this is a surprisingly hard task since I use so many tools. A different approach is to add a toolbar with buttons for zooming, panning, etc., and to restrict the shift modifiers to a  small number of combinations. You can find an example in https://sourceforge.net/p/wp-laz/code/HEAD/tree/MathPlotter/trunk/ ---------------------------------------------------------------------- Muso · 2022-07-11 21:51 Quote from: Muso on July 11, 2022, 08:44:12 pmMany, many thanks. Works perfectly and is indeed exactly what I was looking for!  I was a bit too rash. I bound the tool to Ctrl+leftclick and get the input dialog. But the mark is never displayed. it cost me about an hour to realize that one must not set the Marks style in the .lfm file. Only when I keep the .lfm file untouched and set the marks style in the OnCreate form event, it works.Maybe the DataPointHint tool sneaks in here? Now I need a way to save the mark with my data file (a CSV file that  continuously fill whenever new data arrives). Is there a proven method I should/can use for this? ---------------------------------------------------------------------- wp · 2022-07-12 08:08 Quote from: Muso on July 11, 2022, 11:51:31 pmI was a bit too rash. I bound the tool to Ctrl+leftclick and get the input dialog. But the mark is never displayed. it cost me about an hour to realize that one must not set the Marks style in the .lfm file. Only when I keep the .lfm file untouched and set the marks style in the OnCreate form event, it works.Maybe the DataPointHint tool sneaks in here? Cannot imagine that this is an issue. Please post a project showing this issue. Did you "invalidate" the chart after adding a data point label? Quote from: Muso on July 11, 2022, 11:51:31 pmNow I need a way to save the mark with my data file (a CSV file that  continuously fill whenever new data arrives). Is there a proven method I should/can use for this? Simply add a column for the data point label after the x/y values: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  F: TextFile;  i: Integer;begin  AssignFile(F, 'data.txt');  Rewrite(F);  for i := 0 to Chart1LineSeries1.Count-1 do    WriteLn(F, Chart1LineSeries1.XValue[i]:0:6, #9, Chart1LineSeries1.YValue[i]:0:6, #9, Chart1LineSeries1.Source[i]^.Text);  CloseFile(F);end; Or, alternatively, you could take advantage of the ListSource.DataPoints property of the TListChartSource in which the series data values are accessible as a TStrings instance so that you can save/load data in a CSV-like file by calling its SaveToFile and LoadFromFile methods (note: the field delimiter is '|', i.e. the following code cannot load the file written by the previous code): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button2Click(Sender: TObject);begin  Chart1LineSeries1.ListSour ce.DataPoints.SaveToFile('data.txt');end; procedure TForm1.Button3Click(Sender: TObject);begin  Chart1LineSeries1.ListSource.DataPoints.LoadFromFile('data.txt');end; ---------------------------------------------------------------------- Muso · 2022-07-12 14:22 Quote from: wp on July 12, 2022, 10:08:24 amCannot imagine that this is an issue. Please post a project showing this issue. Did you "invalidate" the chart after adding a data point label? I spent now an hour to get an example file. But neither extending your example nor simplifying my program leads to the issue. So I have no clue why this occurs. I invalidated the Chart. ------------------ However, while trying to make an example, I got a reproducible crash when editing charts:- take the attached example- right-click in Lazarus into the chart and edit the series- delete the 4th series- now use the Menu Edit -> Undo result: access violation Should I open a separate thread for this? ---------------------------------- I need linebreaks in the marks text. Is this possible and if so how? I see there is an HTML font style available for marks. Can i use this and input there e.g.
? -------------------------------- Quote from: wp on July 12, 2022, 10:08:24 amSimply add a column for the data point label after the x/y values: Every few seconds I get new data. As user I later add the notes. I cannot seek in the result file around to add there marks. Later marks are deleted or changed. So I think I need a kind of separate .marks file that I can write in a separate thread. I thought maybe that there exists something. ---------------------------------------------------------------------- Muso · 2022-07-12 14:32 Quote from: Muso on July 12, 2022, 04:22:01 pmI need linebreaks in the marks text. Is this possible and if so how? I see there is an HTML font style available for marks. Can i use this and input there e.g.
? OK, I found now out that I can use Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i LineEnding in the string I pass to the marks text. ---------------------------------------------------------------------- wp · 2022-07-12 15:05 Quote from: Muso on July 12, 2022, 04:22:01 pmHowever, while trying to make an example, I got a reproducible crash when editing charts:- take the attached example- right-click in Lazarus into the chart and edit the series- delete the 4th series- now use the Menu Edit -> Undo result: access violation Maybe I don't understand: "right-click in Lazarus into the chart and edit the series- delete the 4th series" refers to the designmode in Lazarus? Unfortunately I cannot reproduce the bug exactly here, because the "Undo" command is disabled in the Edit menu after this... There is a crash, however, when I press Ctrl+Backspace. I am not sure if this is a TAChart issue at all, IIRC there are issues with the Undo function in the form designer for other controls as well. Quote from: Muso on July 12, 2022, 04:22:01 pmI need linebreaks in the marks text. Is this possible and if so how? I see there is an HTML font style available for marks. Can i use this and input there e.g.
? In this case, I'd design my own InputQuery form containing a memo rather than an edit. This way you can insert line breaks easily. You could also use the HTML mode, by activating the tfHTML TextFormat of the Marks; in this mode you can insert simple HTML tags such as
, or you can also switch font styles. Quote from: Muso on July 12, 2022, 04:22:01 pmEvery few seconds I get new data. As user I later add the notes. I cannot seek in the result file around to add there marks. Later marks are deleted or changed. So I think I need a kind of separate .marks file that I can write in a separate thread. I thought maybe that there exists something. Again I don't understand. You do not have to search in the data file to add a mark - you simply click on the data point and add or edit the mark text (or erase the mark text to delete the mark). If you have many data points it may be difficult to hit an already labeled data point again for editing or deletion of the mark - maybe this is what you mean. In this case I'd populate a string grid with the indices and mark labels of all data points having marks. A separate marks file will make things much more complicated because the mark texts are part of the data point record: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  TChartDataItem = packed record  public    X, Y: Double;    Color: TChartColor;    Text: String;    XList: TDoubleDynArray;    YList: TDoubleDynArray;    ...  end;  PChartDataItem = ^TChartDataItem; ---------------------------------------------------------------------- Muso · 2022-07-12 16:46 Quote from: wp on July 12, 2022, 05:05:14 pmMaybe I don't understand: "right-click in Lazarus into the chart and edit the series- delete the 4th series" refers to the designmode in Lazarus? Unfortunately I cannot reproduce the bug exactly here, because the "Undo" command is disabled in the Edit menu after this... There is a crash, however, when I press Ctrl+Backspace. Hmm, here Undo is enabled and crashes, see attached. Should I open a bugreport or is this known? QuoteIn this case, I'd design my own InputQuery form containing a memo rather than an edit.Yes, this is what I have done now. QuoteAgain I don't understand. You do not have to search in the data file to add a mark - you simply click on the data point and add or edit the mark text (or erase the mark text to delete the mark).OK, I was not clear: - every 2 seconds I get new data (chart is a LiveChart), it is validated, if OK, added to the graph and also saved immediately to a CSV file- Assume I have now 2000 data points and point 1329 gets now a mark. I need to store it.- later I have 5000 points and the mark is changed or deleted, then I need to update the saved markThe key feature of my program is that when the program was killed, one can reload the data and continue the measurement. That the program is suddenly closed happens more often than I thought (Windows has issues, users thought they could pull the power plug of the PC, transfer the measurement to another room etc.) In these cases I can load the existing measurements to the chart. I cannot save the marks to the same data file because when e.g. data point number 5000 was just written, and the user add a mark for point 1200, I must seek in the file to this position to insert there the mark. I hope it is now more clear what I need. I think I will now write a separate file only with all marks. QuoteIf you have many data points it may be difficult to hit an already labeled data point again for editing or deletion of the mark - maybe this is what you mean.No, this works OK. QuoteA separate marks file will make things much more complicated because the mark texts are part of the data point record:I have maybe one mark a day, so in total maybe 10 marks per measurement. I thought I can parse the .marks file and set the marks for the data points. ---------------------------------------------------------------------- wp · 2022-07-12 17:20 What is your Laz/FPC version? Operating system? Bitness? ---------------------------------------------------------------------- Muso · 2022-07-12 18:03 Quote from: wp on July 12, 2022, 07:20:39 pmWhat is your Laz/FPC version? Operating system? Bitness? Lazarus 2.2.2 (rev lazarus_2_2_2) FPC 3.2.2 x86_64-win64-win32/win64 ---------------------------------------------------------------------- wp · 2022-07-12 20:55 OK, the Undo item is active now - 64bit seems to behave differently from 32bit which I am working with normally. Found a missing nil check in a property editor, and after fixing this the crash seems to be gone. But note that the Undo does not restore the deleted series - this seems to be a different issue. At least, it does not crash any more. Please confirm. If you don't have access to Laz/main, replace the following methods in TASeriesPropEditors and rebuild the IDE: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TAxisIndexPropertyEditor.GetValues(AProc: TGetStrProc);var  s: TCustomChartSeries;  ch: TChart;  i: Integer;begin  s := GetComponent(0) as TCustomChartSeries;  ch := s.ParentChart;  AProc('-1 None');  if ch <> nil then           // <--- ADDED    for i := 0 to ch.AxisList.Count - 1 do      AProc(IntToStr(i) + ' ' + ch.AxisList[i].DisplayName);end; function TAxisIndexPropertyEditor.OrdValueToVisualValue(  AOrdValue: Longint): String;var  s: TCustomChartSeries;  ch: TChart;begin  s := GetComponent(0) as TCustomChartSeries;  ch := s.ParentChart;  Result := IntToStr(AOrdValue) + ' ';  if Assigned(ch) and InRange(AOrdValue, 0, ch.AxisList.Count - 1) then   // <--- ADDED: "Assigned(ch) and"    Result += ch.AxisList[AOrdValue].DisplayName  else    Result += 'None';end; ---------------------------------------------------------------------- Muso · 2022-07-13 12:41 Quote from: wp on July 12, 2022, 10:55:00 pmPlease confirm. If you don't have access to Laz/main, replace the following methods in TASeriesPropEditors and rebuild the IDE: Many thanks! I did so and can confirm the bug is fixed. Please also backport this if possible. --------------------------------- Meanwhile the marks work fine for me. However, I have another thing I would like to improve and maybe there is already a method to do this: - Since the measurements take weeks, I added a feature to change the live chart x-axis unit between minutes, hours and days. But now there happens something, one zooms in to see what is happening. To see a sensible x-value, one switches the unit to minutes but sees then then of course huge numbers, see attached.So maybe I could retranslate the x-axis unit somehow to the format day:hour:minute. I searched the forum and found this post from you: https://forum.lazarus.freepascal.org/index.php/topic,33570.msg217785.h tml#msg217785 My application requires that the measurement stats at zero, so the data points x-values must be minutes, only the display should be different, if the user likes to. Maybe I can just use https://www.freepascal.org/docs- html/rtl/sysutils/formatdatetime.html to reformat the x-axis labels without any internal chart recalculation? If so this would be perfect because my charts have quickly 100k data points. ---------------------------------------------------------------------- Muso · 2022-07-13 13:08 Quote from: Muso on July 13, 2022, 02:41:00 pmMaybe I can just use https://www.freepascal.org/docs-html/rtl/sysutils/formatdatetime.html to reformat the x-axis labels without any internal chart recalculation? If so this would be perfect because my charts have quickly 100k data points. I found now this post from you: https://forum.lazarus.freepascal.org/index.php/topic,27750.msg172374.h tml#msg172374 and setup a https://wiki.lazarus.freepascal.org/TAChart_Tutorial:_Userdefined_Char tSource but I am still stuck on how to display my x-date (which is always in minutes) to the format 00:00:00. ---------------------------------------------------------------------- Muso · 2022-07-14 14:09 Since I succeeded to store and loading the marks, here is my solution in case someone needs the same: setting marks and saving:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TMainForm.DataPointClickToolPointClick(ATool: TChartTool;  APoint: TPoint);var NoteText, NotesFile, OutputLine : string; tool : TDataPointTool; series : TChartSeries; NotesFileStream : TFileStream; MousePointer : TPoint;begin MousePointer:= Mouse.CursorPos; tool:= ATool as TDataPointTool; if tool.PointIndex < 0 then  exit;  series:= tool.Series as TChartSeries; NoteText:= series.Source[tool.PointIndex]^.Text; with NoteEditingF do begin  NoteLabelL.Caption:= 'Note for ' + series.Title + ':';  NoteTextM.Lines.Text:= NoteText;  ShowModal;  series.Source[tool.PointIndex]^.Text:= NoteTextM.Lines.Text; end; // force a redraw of the chart (chart name is "SIXCH") SIXCH.Invalidate;  // write mark to the .notes file // we write into the same folder than the program .exe if InNameSensor.IsEmpty then  NotesFile:= 'unknown.notes' else  NotesFile:= ExtractFileName(InNameSensor) + '.notes'; NotesFile:= ExtractFilePath(Application.ExeName) + NotesFile;  try  try   if FileExists(NotesFile) then    NotesFileStream:= TFileStream.Create(NotesFile, fmOpenReadWrite or fmShareDenyNone)   else    NotesFileStream:= TFileStream.Create(NotesFile, fmCreate or fmShareDenyNone);  except   on EFOpenError do   begin    MessageDlgPos('Notes file is used by another program and cannot be opened.'                  + LineEnding + 'New notes cannot be saved.',                  mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);    exit;   end;  end;  // seek to end and append the mark  NotesFileStream.Seek(0, soFromEnd);  OutputLine:= series.Name + #9 + IntToStr(tool.PointIndex) + LineEnding               + NoteEditingF.NoteTextM.Lines.Text + LineEnding + LineEnding;  try   NotesFileStream.Write(OutputLine[1], Length(OutputLine));  except   NotesFileStream.Free;   MessageDlgPos('Notes could not be written to the .notes file.',                 mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);  end;  finally  NotesFileStream.Free; end; end; Reading existing marks and applying them: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TMainForm.ReadNotes;// reads marks from .notes file ad adds them to the chartvar NotesFileStream : TFileStream; LineReader : TStreamReader; StringArray : TStringArray; series : TChartSeries; ReadLine, NotesFile, OutputLine : string; MousePointer : TPoint; rowCounter, PointIndex : integer; skipNote : Boolean;begin // initialize rowCounter:= 0; series:= nil; MousePointer:= Mouse.CursorPos;  NotesFile:= ExtractFileName(InNameSensor) + '.notes'; NotesFile:= ExtractFilePath(Application.ExeName) + NotesFile; if not FileExists(NotesFile) then  // nothing to do  exit; try try  NotesFileStream:= TFileStream.Create(NotesFile, fmOpenRead or fmShareDenyNone) except  on EFOpenError do  begin   MessageDlgPos('Notes file is used by another program and cannot be opened.'                 + LineEnding + 'New notes cannot be saved.',                 mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);   exit;  end; end; LineReader:= TStreamReader.Create(NotesFileStream);  // parse the file to the end while not LineReader.Eof do begin  skipNote:= false;  inc(rowCounter);  LineReader.ReadLine(ReadLine);  StringArray:= ReadLine.Split(#9);  // the first part is the series name the second one the point index  series:= (FindComponent(StringArray[0]) as TChartSeries);  if series = nil then  begin   MessageDlgPos('In line ' + IntToStr(rowCounter) + ' "' + StringArray[0]                 + '" is no valid chart series name.'                 + LineEnding + 'This note could not be read.',                 mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);   skipNote:= true;  end;  if not skipNote then  begin   if not TryStrToInt(StringArray[1], PointIndex) then   begin    MessageDlgPos('Point index in line ' + IntToStr(rowCounter)                  + ' is no valid integer.' + LineEnding                  + 'This note could not be read.',                mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);    skipNote:= true;   end;  if PointIndex < 0 then   begin   MessageDlgPos('Point index in line ' + IntToStr(rowCounter)                + ' is negative.' + LineEnding                  + 'This note could not be read.',                 mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);    skipNote:= true;   end   else if PointIndex > series.Count then   begin    MessageDlgPos('Point index in line ' + IntToStr(rowCounter)                  + ' is "' + IntToStr(PointIndex)                  + '" and thus larger than "' + IntToStr(series.Count)                  + '", the number of points in the series.'                  + LineEnding + 'This note could not be read.',                  mtError, [mbOK], 0, MousePointer.X, MousePointer.Y);    skipNote:= true;   end;  end; // end if not skipNote   // now read as many lines until there is an empty line (2 subsequent LineEndings)  OutputLine:= '';  repeat   inc(rowCounter);   LineReader.ReadLine(ReadLine);   OutputLine:= OutputLine + ReadLine + LineEnding;  // we can have an empty line for a deleted note, therefore until we  // have at least 2 LineEndings   until ReadLine.IsEmpty and (Length(OutputLine) >= 2 * Length(LineEnding));  // remove the two LineEndings at the end  OutputLine:= Copy(OutputLine, 0, Length(OutputLine) - 2 * Length(LineEnding));  // set the mark  if not skipNote then   series.Source[PointIndex]^.Text:= OutputLine; end; // end of while not LineReader.Eof do finally LineReader.Free; NotesFileStream.Free; // force a redraw of the chart (chart name is "SIXCH") SIXCH.Invalidate;end; end; I will open a new thread for the axis number formatting. ---------------------------------------------------------------------- wp · 2022-07-15 11:14 Quote from: Muso on July 13, 2022, 02:41:00 pmHowever, I have another thing I would like to improve and maybe there is already a method to do this: - Since the measurements take weeks, I added a feature to change the live chart x-axis unit between minutes, hours and days. But now there happens something, one zooms in to see what is happening. To see a sensible x-value, one switches the unit to minutes but sees then then of course huge numbers, see attached.So maybe I could retranslate the x-axis unit somehow to the format day:hour:minute. I searched the forum and found this post from you: https://forum.lazarus.freepascal.org/index.php/topic,33570.msg217785.h tml#msg217785 My application requires that the measurement stats at zero, so the data points x-values must be minutes, only the display should be different, if the user likes to. Maybe I can just use https://www.freepascal.org/docs- html/rtl/sysutils/formatdatetime.html to reformat the x-axis labels without any internal chart recalculation? If so this would be perfect because my charts have quickly 100k data points. In the attachment there is a demo project in which the time axis labels format changes depending on the time range displayed. Not 100% perfect, yet, but maybe you get the idea how this could be achieved. Basically it uses a TDateTimeIntervalChartSource (which requires the x values to be in units of days (TDateTime)), and it simply changes the DateTimeFormat property of the source according to the time range. Unfortunately, FPC's FormatDateTime() function does not allow days in interval mode, and this makes the display of day count more complicated: I remove the DateTimeIntervalchartSource in this case and implement an OnGetMarkText handler to format the day interval correctly (unfortunately I did not manage to get "nice" interval labels - but I am quite sure that it must be possible...).