[SOLVED] Save/Load Series data to/from file (fastest way)
lazarus / 55-TAChart

----------------------------------------------------------------------
ahmadian · 2014-05-19 13:54

Hi all
Is there a fast way to save/load LineSeries (x,y) to a file instead of
writing them in a for loop?
My line Series have more than 10000 points and my Hardware is
Raspberry Pi.
My file can't be Typed file, because I want to add a Record before
LineSeries data.
This is my record:
Code: [Select]type
    TNewSample = Packed Record
      ID               : PtrUint;
      Name             : string[100];
      StandardName     : string[100];
      Material         : string[100];
      TestType         : PtrUint;
      TestSpeed        : PtrUint;
      Shape            : PtrUint;
      Length           : real;
      Width            : real;
      Height           : real;
      Weight           : real;
      SampleUserParam1 : string[100];
      SampleUserParam2 : string[100];
      SampleUserParam3 : string[100];
    end;

    TTest = Packed Record
      TestSample       : TNewSample;
      TestDate         : string[20];
      TestUserParam1   : string[100];
      TestUserParam2   : string[100];
      TestUserParam3   : string[100];
    end;

I want to save TTest record plus a LineSeries (array of x,y) to a file
and load them again. My hardware is slow and I want to use fastest
method. (If available on Linux)
Thanks for any help.

----------------------------------------------------------------------
Leledumbo · 2014-05-19 15:04

Using untyped file and big memory buffer, you can reduce I/O call.

----------------------------------------------------------------------
ahmadian · 2014-05-19 15:24

Thanks Leledumbo
Is this true:
Code: [Select]procedure TForm1.Svae_test;
var
  MyFile: file;
  Data: TTest;
begin
  AssignFile(MyFile, 'test1.bin');
  Rewrite(MyFile);
  try
    BlockWrite(MyFile, Data, SizeOf(Data));
    BlockWrite(MyFile, Chart1LineSeries1.Source,
SizeOf(Chart1LineSeries1.Source));
  finally
    CloseFile(MyFile);
  end;
end;

----------------------------------------------------------------------
wp · 2014-05-19 15:30

I don't understand in your posting which one of the records is the
header and which contains the data.

You can mix different record types by writing them into a binary file
or a TFileStream. Let me suppose that the header is a TNewSample
record that you give and the data points are in the ListSource of a
TLineSeries. Then you can write them in this way:

Code: [Select]type
  TDataRecord = record x,y: Double; end;

var
  header: TNewSample;

procedure SaveToFile(ASeries: TChartSeries; AFileName: String);
var
  MS: TMemoryStream;
  i: Integer;
  data: TDataRecord;
begin
  MS := TMemoryStream.Create;
  try
    MS.WriteBuffer(header, SizeOf(header));
    for i:=0 to ASeries.ListSource.Count-1 do begin
      data,X] := ASeries.ListSource.Item[i]^.X;
      data[Y] := ASeries.ListSource.Item[i]^.Y;
      MS.WriteBuffer(data, SizeOf(data));
    end;
    MS.SaveToFile(AFileName);
  finally
    MS.Free;
  end;
end;

procedure ReadDataFromFile(ASeries: TChartSeries; AFileName: String);
var
  FS: TFileStream;
  MS: TMemoryStream;
  data: TDataRecord;
begin
  ASeries.Clear;
  MS := TMemoryStream.Create;
  try
    FS := TFileStream.Create(AFileName, fmOpenRead);
    try
      MS.CopyFrom(FS);
    finally
      FS.Free;
    end;
    MS.Read(header, SizeOf(header));
    while MS.Position < MS.Size do begin
      MS.ReadBuffer(data, Sizeof(data));
      ASeries.AddXY(data.x, data.y);
    end;
  finally
    MS.Free;
  end;
end;

----------------------------------------------------------------------
ahmadian · 2014-05-19 15:52

Thanks wp
I test your code.The CopyFrom procedure need size (maybe filesize).

----------------------------------------------------------------------
wp · 2014-05-19 16:14

The size of the filestream, FS.Size

----------------------------------------------------------------------
ahmadian · 2014-05-19 16:18

Hi wp
I attached test program(on windows).
It saves data to file but don't load it from file.

----------------------------------------------------------------------
ahmadian · 2014-05-19 17:09

Thanks a lot 'wp'
I changed the code and remove the FileStream.
Now it works OK 
I attached code for others.
I don't know how to add (Solved) to topic name!
Regards

(Attached program Save and Load a LineSeries with Record header
to/from a Untyped file)

----------------------------------------------------------------------
wp · 2014-05-19 20:44

QuoteI don't know how to add (Solved) to topic name!Just edit the
title of the first posting

----------------------------------------------------------------------
wp · 2014-05-19 22:32

A remark on your code of adding data to the series:
Quote  for i := 0 to 100000 do
  begin
    LineSeries1.BeginUpdate;
    LineSeries1.AddXY ( i/100 , SIN(pi*i / 10000) );
    LineSeries1.EndUpdate;
  end;

Calling Begin/EndUpdate inside the loop has no effect. Your chart will
be much faster if you call them before/after the loop. BeginUpdate
just blocks the 100000 unnecessary redrawing operations after adding
new data points until the EndUpdate occurs. Your code blocks the
drawing, adds one data point and redraws, and repeats this 100000
times. My code below blocks redrawing, adds all data without redrawing
and redraws only afterwards.

And don't forget the try/finally block - if the AddXY crashes for some
reason your series would never be redrawn without it in this case:

Code: [Select]  LineSeries1.BeginUpdate;
  try
    for i := 0 to 100000 do
      LineSeries1.AddXY ( i/100 , SIN(pi*i / 10000) );
  finally
    LineSeries1.EndUpdate;
  end;

----------------------------------------------------------------------
ahmadian · 2014-05-20 07:09

Thanks wp
That was fast test program and my unaware mistake   (I will change
last attached code for others).
But a bit question:
What is difference between "TMemoryStream.Write" and 
"TMemoryStream.WriteBuffer" ?
So for Read and ReadBuffer?

----------------------------------------------------------------------
wp · 2014-05-20 08:26

From looking at the source code (you can do this too by clicking at
the "WriteBuffer" of "MS.WriteBuffer" with the CTRL-Key down) I'd
guess that there is not difference on this level. In detail, however,
there is a big difference because Read and Write are virtual, i.e.
they are responsible for the correct read/write behaviors of the
various stream types.  Reading and writing has different requirements
when you use a Memorystream or a FileStream. When you write an
Integer, for example, four bytes are written to the stream my means of
the method implemented by "Write". The same for "WriteBuffer" which
writes a given number of bytes. From this aspect, I'd say that
"WriteBuffer" is more "correct" than "Write", but in practice there is
no difference.

If you want to learn more about streams read the short article
http://delphi.about.com/od/vclusing/l/aa110803a.htm

----------------------------------------------------------------------
ahmadian · 2014-05-20 09:10

Thanks wp
I read the article and it compare Arrays and Stream. But the Stream is
look like Pointer. Do you see any difference?

----------------------------------------------------------------------
wp · 2014-05-20 10:34

QuoteBut the Stream is look like PointerNo, a pointer is a variable
that holds the address of something. A stream is a very generalized
file, a collection of data stored in a way defined by the stream type.

----------------------------------------------------------------------
howardpc · 2014-05-20 13:12

A stream is a class, so whatever kind of stream it is (file, string,
memory etc.) an instantiated stream has not only its own data, but its
own methods for interacting with that data and with other classes etc.
A stream variable is a pointer in that it is a reference variable. It
is a reference that identifies the particular memory location where
that instance of the stream has been set up (or will be set up if it
is presently unassigned).

----------------------------------------------------------------------
ahmadian · 2014-05-20 13:46

Thanks wp and howardpc
My questions are for speed-up purposes on Linux embedded PCs, like
Raspberry Pi.

----------------------------------------------------------------------
PeterX · 2023-12-23 20:39

Thanks for this example, useful for me.