Lazarus and XML
lazarus / 21-General
----------------------------------------------------------------------
skom · 2023-09-30 15:37
This is my first contact with XML support and I have a problem. My XML
input file looks like this:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
2023
340-RK1-1PNN
340-PS1-2LNN
30.0
2024-01-18
2023-12-21
2023-12-07
2023-11-23
2023-11-09
2023-10-26
2023-10-12
30.0
2024-01-25
2024-01-11
340-ON2-2EG
30.0
340-PS1-3VPR
15.0
and the program
in lazarus:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project2test; {$mode objfpc}{$H+} uses SysUtils, laz2_DOM,
laz2_XMLRead, laz2_XMLWrite; procedure ProcessXMLtoCSV(inputFileName,
outputFileName: string);var XMLDocument: TXMLDocument; RootNode,
CykleNode, PrzedmiotNode, GrupaNode, SpotkanieNode: TDOMNode;
CSVFile: TextFile;begin // Initialization of the XML document
ReadXMLFile(XMLDocument, inputFileName); RootNode :=
XMLDocument.DocumentElement; // Opening the CSV file for writing
AssignFile(CSVFile, outputFileName); Rewrite(CSVFile); try //
Processing cykleDydaktycznych CykleNode :=
RootNode.FindNode('.//cykleDydaktyczne/cyklDydaktyczny'); while
Assigned(CykleNode) do begin // Processing przedmiot
PrzedmiotNode := CykleNode.FindNode('.//przedmiot'); while
Assigned(PrzedmiotNode) do begin // Processing
grupaZajęciowa GrupaNode :=
PrzedmiotNode.FindNode('.//grupaZajeciowa'); while
Assigned(GrupaNode) do begin // Processing spotkanie
SpotkanieNode := GrupaNode.FindNode('.//spotkanie');
while Assigned(SpotkanieNode) do begin // Geting
data Write(CSVFile,
GrupaNode.FindNode('liczbaGodzin').TextContent, #9);
Write(CSVFile, PrzedmiotNode.Attributes.GetNamedItem('kod').NodeValue,
#9); Write(CSVFile,
SpotkanieNode.FindNode('data').TextContent, #9);
Writeln(CSVFile); SpotkanieNode :=
SpotkanieNode.NextSibling; end; GrupaNode :=
GrupaNode.NextSibling; end; PrzedmiotNode :=
PrzedmiotNode.NextSibling; end; CykleNode :=
CykleNode.NextSibling; end; finally CloseFile(CSVFile);
end;end; begin try ProcessXMLtoCSV('plan3.xml', 'plan.csv');
Writeln('The CSV file has been generated.'); except on E:
Exception do Writeln('An error occured: ', E.Message);
end;end. Unfortunately, the program finds nothing and the csv file is
empty.
Where is my mistake?
----------------------------------------------------------------------
jamie · 2023-09-30 15:47
I assume your test files are within the same folder as your project
files? Otherwise, without a full path to it, you get nothing.
----------------------------------------------------------------------
skom · 2023-09-30 15:52
yes, both are in the same folder
----------------------------------------------------------------------
Bart · 2023-09-30 15:56
Is the './/' part correct (wouldn't '/' at the star suffice?).
B.t.w. I get massive memory leaks reports when running the program...
Bart
----------------------------------------------------------------------
skom · 2023-09-30 16:19
I don't really understand, it should be */ ? And how can I prevent
excessive memory consumption in this case?
----------------------------------------------------------------------
jamie · 2023-09-30 16:30
The Nodes (TDomNode) all have owners.
So look at your code at the very start, who is the owner?
Maybe the XMLDocument.Free ? I don't know, I am just guessing, like
everyone else!
----------------------------------------------------------------------
Bart · 2023-09-30 16:33
I know next to nothing about XML.
With your supplied xml:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
writeln('RootNode.NodeName=',RootNode.NodeName);
//TDOMElement.findnode writeln('RootNode.FirstChild.NodeName=',Root
Node.FirstChild.NodeName);
writeln('RootNode.FindNode(jednostkiOrganizacyjne) ->
',Assigned(RootNode.FindNode('jednostkiOrganizacyjne')));
writeln('* NextSibling ->
',Assigned(RootNode.FindNode('jednostkiOrganizacyjne').NextSibling));
writeln('* NextSibling.NodeName=',RootNode.FindNode('jednostkiOrgani
zacyjne').NextSibling.NodeName); writeln('* NextSibling.NextSibling
-> ',Assigned(RootNode.FindNode('jednostkiOrganizacyjne').NextSibling.
NextSibling));
Gives:
Code: [Select]RootNode.NodeName=plan
RootNode.FirstChild.NodeName=jednostkiOrganizacyjne
RootNode.FindNode(jednostkiOrganizacyjne) -> TRUE
* NextSibling -> TRUE
* NextSibling.NodeName=cykleDydaktyczne
* NextSibling.NextSibling -> FALSE
It does not find any sibling node for 'cykleDydaktyczne' node, and
therefore (?) FindNode will return False (FindNode calls NextSibling).
RootNode.NodeName=plan
RootNode.FirstChild.NodeName=jednostkiOrganizacyjne
----------------------------------------------------------------------
Bart · 2023-09-30 16:57
Changed the program and at least now it has output.
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
test; {$mode objfpc}{$H+} uses SysUtils, laz2_DOM, laz2_XMLRead,
laz2_XMLWrite; procedure ProcessXMLtoCSV(inputFileName,
outputFileName: string);var XMLDocument: TXMLDocument; RootNode,
CykleNode, PrzedmiotNode, GrupaNode, SpotkanieNode, ANode: TDOMNode;
CSVFile: TextFile; S: DOMString;begin // Initialization of the XML
document ReadXMLFile(XMLDocument, inputFileName); RootNode :=
XMLDocument.DocumentElement; // Opening the CSV file for writing
AssignFile(CSVFile, outputFileName); Rewrite(CSVFile); try //
Processing cykleDydaktycznych CykleNode :=
RootNode.FindNode('cykleDydaktyczne'); writeln('cykleDydaktyczne:
',Assigned(CykleNode)); if Assigned(CykleNode) then begin
writeln('Found: cykleDydaktyczne'); CykleNode :=
CykleNode.FirstChild; if Assigned(CykleNode) then
writeln('CykleNode.NodeName=',CykleNode.NodeName) else
writeln('CykleNode=nil'); end; while Assigned(CykleNode) do
begin // Processing przedmiot PrzedmiotNode :=
CykleNode.FindNode('przedmiot'); while Assigned(PrzedmiotNode)
do begin writeln('Found przedmiot'); // Processing
grupaZajęciowa GrupaNode :=
PrzedmiotNode.FindNode('grupaZajeciowa'); while
Assigned(GrupaNode) do begin writeln('Found
grupaZajeciowa'); // Processing spotkanie
SpotkanieNode := GrupaNode.FindNode('spotkanie'); while
Assigned(SpotkanieNode) do begin writeln('Found
spotkanie'); // Geting data ANode :=
GrupaNode.FindNode('liczbaGodzin'); if Assigned(ANode)
then S := ANode.TextContent else
S := '[empty]'; Write(CSVFile,
S{GrupaNode.FindNode('liczbaGodzin').TextContent}, #9);
ANode := PrzedmiotNode.Attributes.GetNamedItem('kod'); if
Assigned(ANode) then S := ANode.NodeValue
else S := '[empty]'; Write(CSVFile,
S{PrzedmiotNode.Attributes.GetNamedItem('kod').NodeValue}, #9);
ANode := SpotkanieNode.FindNode('data'); if
Assigned(ANode) then S := ANode.TextContent
else S := '[empty]'; Write(CSVFile,
S{SpotkanieNode.FindNode('data').TextContent}, #9);
Writeln(CSVFile); SpotkanieNode :=
SpotkanieNode.NextSibling; end; GrupaNode :=
GrupaNode.NextSibling; end; PrzedmiotNode :=
PrzedmiotNode.NextSibling; end; CykleNode :=
CykleNode.NextSibling; end; finally CloseFile(CSVFile);
end;end; begin //try ProcessXMLtoCSV('plan3.xml', 'plan.csv');
Writeln('The CSV file has been generated.'); //except // on E:
Exception do // Writeln('An error occured: ', E.Message);
//end;end.
Notice that I removed all starting slashes from the nodenames in
FindNode().
Notice that I also changed the writing to CSV file, since FindNode and
Attributes.GetNamedItem can return nil, so you need to check that
(without the check the program crashes with "EObjectCheck: Object
reference is Nil").
It looks cumbersome this way, and some more experienced person migt
have a better way of fixing this.
I hope this can get you started.
Bart
----------------------------------------------------------------------
wp · 2023-09-30 17:18
I normally iterate through the xml tree myself, without the FindNode
methods because in my way of thinking I too often miss some nodes this
way or get lost in endless loops. I do use FindNode to arrive at the
non-repetitive part of the tree, and then I use FirstChild and
NextSibling to walk along the tree. I always add a nodeName variable
which makes the NodeName of the currently worked node available to the
debugger so that I know where I am.
The xml gurus certainly know better ways...
With the xml presented I end up with the following code and the
following csv output (Not sure whether the WriteLn instruction is at
the correct location because I do not understand the contents of the
xml file and what should be exported):
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
ProcessXMLtoCSV(inputFileName, outputFileName: string);var
XMLDocument: TXMLDocument; RootNode, CykleNode, cyklDydaktycznyNode,
PrzedmiotNode, przedmiot_childNode, grupaZajeciowa_childNode:
TDOMNode; CSVFile: TextFile; nodeName: String; kod_value,
liczbaGodzin_Value, data_Value: String;begin // Initialization of the
XML document ReadXMLFile(XMLDocument, inputFileName); RootNode :=
XMLDocument.DocumentElement; // Opening the CSV file for writing
AssignFile(CSVFile, outputFileName); Rewrite(CSVFile); try //
Processing cykleDydaktycznych CykleNode :=
RootNode.FindNode('cykleDydaktyczne'); if Assigned(CykleNode)
then begin cyklDydaktycznyNode :=
CykleNode.FindNode('cyklDydaktyczny'); if
Assigned(cyklDydaktycznyNode) then begin // Processing
przedmiot PrzedmiotNode :=
cyklDydaktycznyNode.FindNode('przedmiot'); while
Assigned(PrzedmiotNode) do begin nodeName :=
PrzedmiotNode.NodeName; przedmiot_ChildNode :=
PrzedmiotNode.FirstChild; while Assigned(przedmiot_childnode)
do begin // Processing children of przedmiot
nodeName := przedmiot_childNode.NodeName; if nodeName
= 'kod' then kod_value :=
przedmiot_childNode.TextContent else if nodeName
= 'grupaZajeciowa' then begin // Processing
children of grupaZajeciowa grupaZajeciowa_childNode :=
przedmiot_childNode.FirstChild; while
Assigned(grupaZajeciowa_childNode) do begin
nodeName := grupaZajeciowa_childNode.NodeName; case
nodeName of 'liczbaGodzin':
liczbaGodzin_Value := grupaZajeciowa_childNode.TextContent;
'spotkanie': begin
data_Value := grupaZajeciowa_childNode.TextContent;
WriteLn(CSVFile, liczbaGodzin_value, #9, kod_value, #9,
data_value); end; end;
grupaZajeciowa_ChildNode := grupaZajeciowa_childNode.NextSibling;
end; end; przedmiot_childnode :=
przedmiot_childnode.NextSibling; end; przedmiotNode
:= przedmiotNode.NextSibling; end; end; end; finally
CloseFile(CSVFile); end;end;
Code: [Select]30.0 340-PS1-2LNN 2024-01-18
30.0 340-PS1-2LNN 2023-12-21
30.0 340-PS1-2LNN 2023-12-07
30.0 340-PS1-2LNN 2023-11-23
30.0 340-PS1-2LNN 2023-11-09
30.0 340-PS1-2LNN 2023-10-26
30.0 340-PS1-2LNN 2023-10-12
30.0 340-PS1-2LNN 2024-01-25
30.0 340-PS1-2LNN 2024-01-11
----------------------------------------------------------------------
dsiders · 2023-09-30 20:22
Quote from: skom on September 30, 2023, 06:19:32 pmI don't really
understand, it should be */ ?
FindNode accepts a child node name and not an XPath expression. You'd
need to use TXPathExpression classes for that functionality.
https://lazarus-ccr.sourceforge.io/docs/lazutils/laz2_xpath/evaluatexp
athexpression.html
----------------------------------------------------------------------
skom · 2023-10-01 04:19
thank you all for your help, I'm checking all the tips you provided
#Bart
In your code I always have "kod" [empty], even though it is in xml?
skom
----------------------------------------------------------------------
Bart · 2023-10-01 07:38
Use wp's code, not mine.
Bart
----------------------------------------------------------------------
jamie · 2023-10-01 13:42
Is it me or does the XLMDocument instant need to be FREED at the end?
I am a sticker for resources usage.
----------------------------------------------------------------------
wp · 2023-10-01 13:49
You are right. I saw it yesterday, but then forgot to write a note.
----------------------------------------------------------------------
skom · 2023-10-01 15:59
Thank you Bart and wp
#wp
I'm working on your code now and I have a problem, I've been working
on it for an hour and unfortunately without a positive result -
yesterday I gave a simplified sentence (I thought I could still handle
it myself, unfortunately not )
If the xml file has the form:
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
26000000
Szkoła Doktorska Nauk Humanistycznych
WFIL Budynek Wydziału
Filologicznego
47
75
359
992
True
Internet dla prowadzącego
Nagłośnienie
48
50
360
992
True
Rzutnik multimedialny
54
19
361
992
True
Sala komputerowa
Internet dla prowadzącego
Tablica (ścieralna)
A
K 36
Pan
A
D
P 31
Pan
A
K
K 43
Pani
wykładowca
E F
L 34
Pan
A
2023-10-01
00:00:00 2024-06-30
00:00:00 2023
Rok 2023/24 Praktyczna nauka
drugiego języka obcego - j. niemiecki
340-RK1-1PNN
Literatura najnowsza
340-PS1-2LNN
1
30.0
228703
KON
2024-01-18
11:30:00
13:00:00
2023-12-21
11:30:00
13:00:00
2023-12-07
11:30:00
13:00:00
2023-11-23
11:30:00
13:00:00
2023-11-09
11:30:00
13:00:00
2023-10-26
11:30:00
13:00:00
2023-10-12
11:30:00
13:00:00
2024-01-25
12:15:00
13:00:00
co dwa tygodnie (nieparzyste)
3
11:30:00
13:00:00
85899
2
30.0
228703
KON
2024-01-25
11:30:00
12:15:00
2024-01-11
11:30:00
13:00:00
2023-12-14
11:30:00
13:00:00
2023-11-30
11:30:00
13:00:00
2023-11-16
11:30:00
13:00:00
2023-11-02
11:30:00
13:00:00
2023-10-19
11:30:00
13:00:00
2023-10-05
11:30:00
13:00:00
co dwa tygodnie (parzyste)
3
11:30:00
13:00:00
85900
Emisja głosu
340-ON2-2EG
1
30.0
228523
CW
Vademecum pracy
redaktora
340-PS1-3VPR
1
15.0
228774
CW
co dwa tygodnie (parzyste)
3
09:45:00
11:15:00
85931
and I would like to write and from the
in the csv,
and ultimately for I would substitute the appropriate
employee ( and ) with and
the id number is equal to the ref number, and after
substitute the appropriate from
(id=ref).
----------------------------------------------------------------------
paweld · 2023-10-01 21:25
I prefer to process xml files using SAX. Any extension is much
simpler. The only condition is that you must know the file format.
Sample project in attachment.
----------------------------------------------------------------------
wp · 2023-10-01 21:50
My solution uses XPath to simplify finding the starting nodes for each
query. Once such a node has been found the required information is
extracted by climbing down the subtrees similar to my previous post.
At first I am traversing the pracownicy tree, some information is
stored in a list for easier lookup later. In the same way budynek tree
could be traversed (which I didn't do). Finally I am traversing the
cyklDydaktyczny tree collecting the information of some subnodes. The
prowadzacy ref is replaced by the associated name in the list prepared
earlier. Finally the results are written to screen.
Code: Pascal [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project1; {$mode objfpc}{$h+} uses SysUtils, fgl, laz2_dom,
laz2_xmlread, laz2_xpath; const XML_FILE_NAME = 'plan.xml'; type
TNameList = specialize TFPGMap; var XPathResult:
TXPathVariable; XMLDocument: TXMLDocument; ns: TNodeSet; i, j:
Integer; rootNode: TDOMNode; pracownicyNode, grupaNode,
spotkanieNode: TDOMNode; nodeName: String; idStr, imieStr,
nazwiskoStr: String; numerStr, typStr: String; dataStr,
godzinaOdStr, godzinaDoStr, prowadzacyRefStr, pracownicyStr: String;
pracownicyList: TNameList; begin pracownicyList := TNameList.Create;
pracownicyList.Sorted := true; ReadXMLFile(XMLDocument,
XML_FILE_NAME); rootNode := XMLDocument.DocumentElement;
XPathResult := EvaluateXPathExpression('/plan/pracownicy', rootNode);
ns := XPathResult.AsNodeSet; for i := 0 to ns.Count-1 do begin
pracownicyNode := TDOMNode(ns[i]); if pracownicyNode.HasChildNodes
then begin pracownicyNode := pracownicyNode.FirstChild;
while Assigned(pracownicyNode) do begin nodeName :=
pracownicyNode.NodeName; if nodeName = 'pracownik' then
begin idStr :=
pracownicyNode.Attributes.GetNamedItem('id').NodeValue;
imieStr := pracownicyNode.FindNode('imie').TextContent;
nazwiskoStr := pracownicyNode.FindNode('nazwisko').TextContent;
end; WriteLn(idStr, ' --> ', imieStr, ' ', nazwiskoStr);
pracownicyList.Add(idStr, imieStr + ' ' + nazwiskoStr);
pracownicyNode := pracownicyNode.NextSibling; end; end; end;
XPathResult.Free; WriteLn; WriteLn('numer', #9, 'typ', #9, 'data',
#9, 'godzinaOd', #9, 'godzinaDo', #9, 'imie nazwisko'); XPathResult
:= EvaluateXPathExpression('/plan/cykleDydaktyczne/cyklDydaktyczny/prz
edmiot/grupaZajeciowa', rootNode); ns := XPathResult.AsNodeSet; for
i:=0 to ns.Count-1 do begin grupaNode := TDOMNode(ns[i]);
nodeName := grupaNode.NodeName; numerStr :=
grupaNode.FindNode('numer').TextContent; typStr :=
grupaNode.FindNode('typ').TextContent; spotkanieNode :=
grupaNode.FindNode('spotkanie'); while spotkanieNode <> nil do
begin nodeName := spotkanieNode.NodeName; if nodeName =
'spotkanie' then begin dataStr :=
spotkanieNode.FindNode('data').TextContent; godzinaOdStr :=
spotkanieNode.FindNode('godzinaOd').TextContent; godzinaDoStr
:= spotkanieNode.FindNode('godzinaDo').TextContent;
prowadzacyRefStr := spotkanieNode.FindNode('prowadzacy').Attributes.Ge
tNamedItem('ref').NodeValue; if
pracownicyList.Find(prowadzacyRefStr, j) then pracownicyStr
:= pracownicyList.Data[j] + ' (' + prowadzacyRefStr + ')' else
pracownicyStr := '(' + prowadzacyRefStr + ')';
WriteLn(numerStr, #9, typStr, #9, dataStr, #9, godzinaOdStr, #9,
godzinaDoStr, #9, pracownicyStr); end; spotkanieNode :=
spotkanieNode.NextSibling; end; end; XPathResult.Free;
XMLDocument.Free; pracownicyList.Free;end.