fpsStreams unit problems
lazarus / 49-FPSpreadsheet

----------------------------------------------------------------------
AlexTP · 2023-10-03 06:36

I got this unit
https://sourceforge.net/p/lazarus-ccr/svn/HEAD/tree/components/fpsprea
dsheet/source/common/fpsstreams.pas
and replaced TFileStream with it in ATSynEdit.
You can get ATSynEdit and do it too.
Change is in atsynedit/atstrings_save.inc,
procedure TATStrings.SaveToFile.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  procedure
TATStrings.SaveToFile(const AFilename: string);var  fs: TBufStream; 
NMode: word;begin  NMode:= fmOpenReadWrite or fmShareDenyWrite; 
//don't set fmCreate for existing file, to keep NTFS file streams  if
not FileExists(AFilename) then    NMode:= NMode or fmCreate;   fs:=
TBufStream.Create(AFilename, NMode, 64*1024);  try    fs.Size:= 0;
//important to avoid tail from old contents    fs.Seek(0,
soFromBeginning);    SaveToStream(fs, FEncoding,
IsSavingWithSignature);  finally    FreeAndNil(fs);  end;  
DoFinalizeSaving;end; 
Problems when ATSynEdit saves 30M big log file

- before, saving took 10-20 seconds with TFileStream, or 1-2 seconds
with TMemoryStream+TFileStream. Now it is forever. ATSynedit saving
progressbar stops. Disk file grows to 20G and app freezes.

- you always add fmCreate to FileMode so NTFS file streams are not
preserved

----------------------------------------------------------------------
wp · 2023-10-03 10:45

I cannot reproduce the issue with the following minimal project:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project1;uses  SysUtils, Classes, fpsStreams;const  N = 8*1024*1024;
 // there are N*4 Bytes in the file --> 32 MBvar  stream: TStream; 
data: array[0..N-1] of DWord;  i: Integer;  t: TDateTime;  fn:
String;begin  for i := 0 to N-1 do    data[i] := i;  fn :=
ExtractFilepath(ParamStr(0)) + 'test.dat';   t := Now();  stream :=
TBufStream.Create(fn, fmCreate, 64*1024);  // --> 0.09 seconds 
//stream := TFileStream.Create(fn, fmCreate);        // --> 19
seconds  //stream := TMemoryStream.Create;                      // -->
0.22 seconds  try    for i := 0 to N-1 do      stream.Write(data[i],
SizeOf(DWord));    if stream is TMemoryStream then
TMemoryStream(stream).SaveToFile(fn);  finally    stream.Free;  end; 
t := Now() - t;   WriteLn('Done ', FormatDateTime('s.zzz', t), '
seconds.');  ReadLn;end.

----------------------------------------------------------------------
AlexTP · 2023-10-03 14:00

Maybe you can repro on the maximal project, ie do as I wrote in the
1st post and save the 30M file using ATSynEdit demo.

----------------------------------------------------------------------
wp · 2023-10-03 14:15

Sorry, I promised myself that I will not try to fix bugs occuring only
in third-party software.

----------------------------------------------------------------------
AlexTP · 2023-10-03 16:54

If i run this example, i get the file test.log with content
Code: [Select]line one;
line two;
line end;which is not expected.
see the code, test.log must have
Code: [Select]first.
Code
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$h+} uses sysutils, classes, fpsStreams; procedure Save(const
filename: string; L: TStringList);var  fs: TBufStream;  NMode: word; 
buf: string;  i: integer;begin  NMode:= fmOpenReadWrite or
fmShareDenyWrite;  //don't set fmCreate for existing file, to keep
NTFS file streams  if not FileExists(filename) then    NMode:= NMode
or fmCreate;   fs:= TBufStream.Create(filename, NMode);  try   
fs.Size:= 0;    for i:= 0 to L.Count-1 do    begin      buf:=
L[i]+#10;      fs.WriteBuffer(buf[1], length(buf));    end;  finally 
 FreeAndNil(fs);  end;end; var  L: TStringList;begin  L:=
TStringList.Create;  L.Add('line one;');  L.Add('line two;'); 
L.Add('line end;');  Save('test.log', L);   L.Clear; 
L.Add('first.');  Save('test.log', L);   L.Free; end.

----------------------------------------------------------------------
AlexTP · 2023-10-03 16:57

(Linux x64, FPC 3.2.3)

----------------------------------------------------------------------
wp · 2023-10-03 20:44

OK, now I understand. Never had thought of such a requirement to set
the Size of the TBufStream...

Please test the new version at ccr.

However, having to consider both buffer state and existing file size,
I think it will be very complicated to set the Size to an arbitrary
value. Therefore, I decided to allow only a new size of zero; any
other value raises an exception. For simplification, there is also a
TBufStream.Clear method. (Why does the abstract class TStream declare
Size to be public?)