How to load CSVDataset to DBGrid? CSV overwritten after first run.
lazarus / 16-Databases

----------------------------------------------------------------------
Vodnik · 2020-02-11 14:06

Following SQLdb Tutorial, I successfully access DBs with SQL and show
data in DBGrid.
But how the CSVDataset should be opened in DBGrid?

I have added TCSVDataset, TDataSource and TDBGrid to the form and set
properties accordingly.
I can see the data at design time, but not at run time.
CSVDataset1.Open seems not enough.

Do we need SQLQuery and SQLTransaction in that case?

----------------------------------------------------------------------
mangakissa · 2020-02-11 14:34

CSVDataset1.Open;

----------------------------------------------------------------------
Vodnik · 2020-02-11 15:11

Yes, but this works for the first time only.
After exit from the application CSV file is modified to binary format:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
inBufDataSet    Column1   Column1я      Column2   Column2я   яяяяю   
301   Resp. Buryatiyaю     302   Buryatskij AOю     302   CHitinskaya
obl.ю     341   Resp. Udmurtiyaю     342   Permskaya obl.ю     343  
Sverdlovskaya obl.ю     345   Tyumenskaya obl.ю     346   Tyumenskaya
obl.ю     347   Resp. Bashkortostanю     349   YAmalo-Neneckij Okrugю
   351   CHelyabinskaya obl.ю     352   Kurganskaya obl.ю     353  
Orenburgskaya obl.ю     Setting CSVDataset1.Readonly=True doesn't
help.

----------------------------------------------------------------------
Vodnik · 2020-02-11 21:34

Dear gurus, please help to understand why CSV file is overwritten
after application first run.
Project attached.

----------------------------------------------------------------------
ezlage · 2020-02-11 21:40

If you have "FirstLineAsFieldNames=False" you probably need to create
the FieldDefs manually. Maybe the absence of this information causes
the file to be recreated.

----------------------------------------------------------------------
Vodnik · 2020-02-11 22:26

With FirstLineAsFieldNames=True CSV-file is overwritten, too.
If FieldDefs are created manually, CSV-file is overwritten, too.
Why at all application should overwrite it?

----------------------------------------------------------------------
winni · 2020-02-11 22:31

Hi!

Set the CSV to read only and then check on which line it crashes.

Winni

----------------------------------------------------------------------
ezlage · 2020-02-11 23:23

I don't know what is happening - maybe a bug -, but I was able to
open, write and save this way.

Before, I cleared the CSVDataset1.FileName property!

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var  Form1:
TForm1;  CSVdata: TMemoryStream; implementation {$R *.lfm} { TForm1
} procedure TForm1.FormCreate(Sender: TObject);begin 
CSVdata:=TMemoryStream.Create; 
CSVdata.LoadFromFile('citycodes.csv'); 
CSVDataset1.LoadFromCSVStream(CSVdata); 
CSVDataset1.Open;end; procedure TForm1.FormClose(Sender: TObject; var
CloseAction: TCloseAction);begin  CSVdata.Clear; 
CSVDataset1.SaveToCSVStream(CSVdata); 
CSVdata.SaveToFile('citycodes.csv');  CSVDataset1.Close; 
CSVdata.Free;end;

----------------------------------------------------------------------
mangakissa · 2020-02-12 08:36

It seems that TCSVdataset is doing strange things in the textfile.
You can use TBufdataset combined with this piece of code:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i type {
TMyCSV }  TMyCSV = class    fBufdataset : TBufdataset;    fTextfile  
: TStrings;    fUseFirstRow: boolean;     procedure CreateDataset;   
procedure fillDataset;   public      constructor create(aBufdataset :
TBufdataset; aTextfile : string; const aUseFirstRow : boolean =
true);      destructor  destroy; override;  end; implementation {
TMyCSV } constructor TMyCSV.create(aBufdataset: TBufdataset;
aTextfile: string; const aUseFirstRow : boolean);begin  fBufdataset :=
aBufdataset;  fTextfile   := TStringlist.create; 
fTextfile.LoadFromFile(aTextfile);  fUseFirstRow:= aUseFirstRow; 
CreateDataset;  fillDataset;end; destructor TMyCSV.destroy;begin 
fTextfile.free;  inherited destroy;end; procedure
TMyCSV.CreateDataset;var FirstTextline : string;    slRegel       :
TStrings;    columnindex   : integer;begin  slRegel :=
TStringlist.create;  try    slRegel.Delimiter := ',';   
slRegel.StrictDelimiter := true;    slRegel.CommaText :=
fTextfile[0];    for columnindex := 0 to slRegel.count - 1 do   
begin      if fUseFirstRow then     
 fBufdataset.FieldDefs.Add(slRegel[columnindex],ftstring,30)     else 
     fBufdataset.FieldDefs.Add('line_' + inttostr(columnindex +
1),ftstring,30);    end;    fBufdataset.CreateDataset;   
fBufdataset.Active := true;  finally    slRegel.free; 
end;end; procedure TMyCSV.fillDataset;var slRegel       : TStrings;  
columnindex   : integer;    firstrowindex : integer;    subindex    
 : integer;begin  slRegel := TStringlist.create;  try   
slRegel.Delimiter := ',';    slRegel.StrictDelimiter := true;    if
fUseFirstRow then      firstrowindex := 1    else      firstrowindex
:= 0;    for columnindex := firstrowindex to fTextfile.Count - 1 do  
begin      slRegel.CommaText := fTextfile[columnindex];     
fBufdataset.Append;      for subindex := 0 to slRegel.Count - 1 do   
   fBufdataset.Fields[subindex].Value := slRegel[subindex];     
fBufdataset.Post;    end;  finally    slRegel.free;  end;end; 
Use can also look at JanSQL

----------------------------------------------------------------------
Vodnik · 2020-02-12 14:52

Quote from: winni on February 11, 2020, 11:31:12 pmSet the CSV to read
only and then check on which line it crashes.
Destructor TLCLComponent.Destroy on calling inherited Destroy.

----------------------------------------------------------------------
wp · 2020-02-12 15:11

Your description somehow reminds me of a bugreport of mine which was
fixed recently in fpc trunk and 3.2-to-come:
https://bugs.freepascal.org/view.php?id=34435.

----------------------------------------------------------------------
Vodnik · 2020-02-12 20:23

Thanks, @wp, this is the same bug.
I have updated Lazarus trunk version (SVN 62624M), but issue still
exists.

----------------------------------------------------------------------
wp · 2020-02-12 20:37

No, the basic CSVDataset code belongs to FPC, not to Lazarus.
Therefore, updating Lazarus is usless. Unfortunately, FPC 3.2 has not
yet been released. Unless you want to build FPC from code you can
download a precompiled snapshot from
http://ftp.icm.edu.pl/packages/fpc/snapshot/.

----------------------------------------------------------------------
Vodnik · 2020-02-12 20:40

@ezlage, I have tested your code, it works.
I'm trying to understand what is the keypoint.
Similar code that I expect to run, too:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.FormCreate(Sender: TObject);begin 
CSVDataset1.LoadFromCSVFile(GetCurrentDir+'\citycodes.csv'); 
CSVDataset1.Open;end; procedure TForm1.FormClose(Sender: TObject; var
CloseAction: TCloseAction);begin 
CSVDataset1.SaveToCSVFile(GetCurrentDir+'\citycodes.csv'); 
CSVDataset1.Close;end; causes "Stream write error" on exit.

----------------------------------------------------------------------
ezlage · 2020-02-13 09:52

Quote from: Vodnik on February 12, 2020, 09:40:23 pm@ezlage, I have
tested your code, it works.
I'm trying to understand what is the keypoint.
Similar code that I expect to run, too:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.FormCreate(Sender: TObject);begin 
CSVDataset1.LoadFromCSVFile(GetCurrentDir+'\citycodes.csv'); 
CSVDataset1.Open;end; procedure TForm1.FormClose(Sender: TObject; var
CloseAction: TCloseAction);begin 
CSVDataset1.SaveToCSVFile(GetCurrentDir+'\citycodes.csv'); 
CSVDataset1.Close;end; causes "Stream write error" on exit.

Yeah, I tried this way too and it failed the same way. Strange.

----------------------------------------------------------------------
Vodnik · 2020-02-13 20:35

Quote from: wp on February 12, 2020, 09:37:12 pmNo, the basic
CSVDataset code belongs to FPC, not to Lazarus. Therefore, updating
Lazarus is usless. Unfortunately, FPC 3.2 has not yet been released.
Unless you want to build FPC from code you can download a precompiled
snapshot from http://ftp.icm.edu.pl/packages/fpc/snapshot/.
Well, I downloaded the snapshot and tried to replace FPC 3.0.4 with
FPC 3.2.0. This was the pain. I didn't find clear description how this
should be done, so tried step by step correcting every error reported
by Lazarus, until completely destroyed it. One who decide to walk this
way should know in details how IDE works, that's not for me

----------------------------------------------------------------------
wp · 2020-02-13 22:14

This is how I got it running on Windows (I don't know your OS):
Download the 32-bit binaries from http://ftp.icm.edu.pl/packages/fpc/s
napshot/v32/i386-win32/fpc-3.2.0-beta.i386-win32.zip (or for 64 bit
use the x86_64_win64 folder - in the following description, I'll
assume 32 bit however)Unzip the file to something like
d:\fpc320Download the fpc sources from
http://ftp.icm.edu.pl/packages/fpc/snapshot/v32/source/fpc.zipUnzip
the folders "packages" and "rtl" to d:\fpc320 (the other folders are
not needed). Note the folder structure must be retained, i.e.
"packages" and "rtl" must be folders within d:\fpc320. Copy the
following code to a batch file and execute it in order to create the
fpc.cfg file (adjust the DEST_DIR if you extracted to a different
folder)Code: Bash  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i set
DEST_DIR=d:\fpc320path %DEST_DIR%\bin\i386-win32\fpcmkcfg -d
basepath=%DEST_DIR% -o %DEST_DIR%\bin\i386-win32\fpc.cfg Open the
Lazarus version for which you want to switch fpc to v3.2. But note
that the current release 2.0.6 does not compile with fpc 3.2, you must
use Lazarus fixes or trunk.Go to "Tools" > "Options" > "Files.Make the
"compiler executable" point to the new location of fpc. In my case it
is d:\fpc320\bin\i386-win32\fpc.exeMake the "FPC Source directory"
point to the folder which contains the "Packages" and "rtl"
directories, in my case: d:\fpc320Rebuild the IDE by going to "Tools"
> "Configure Build Lazarus" ; in the "Clean-up" box check "Clean all"
and "Switch after building to automatically". Click "Build"After a
while Lazarus restarts and will use fpc 3.2 now.
Of course, you can also install a new version with fpcupdeluxe which
is less work.

----------------------------------------------------------------------
Vodnik · 2020-02-14 14:51

Thanks a lot, wp!
I nearly came to the same.
Lazarus also asks path to make.exe file, which doesn't come with
snapshot.
I tried to use a one from FPC3.0.4, but this doesn't work:
Panic: make: Entering directory `C:/lazarus'
Panic: make: Leaving directory `C:/lazarus'
Panic: Makefile:29: *** You need the GNU utils package to use this
Makefile.  Stop.