Query.RecordCount is wrong lazarus / 16-Databases ---------------------------------------------------------------------- nugax · 2022-03-14 00:59 I have the following code. I have 40 records in my sqlite db. I can manually look and see the mailid, which is an autoincrement integer in the database. However, checking the code by a simple write returns a record count of 10??? Any idea why? Its a very simple routine, and the table is correct. I ensured it was looking at the right table. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetAllMailFromConference(sConferenceGiven: string): ConferenceArray;var  Connect: TSQLite3Connection;  Trans: TSQLTransaction;  Query: TSQLQuery;  db_name: string[100];  sqlhandle: string[255];  MailRecordArray: ConferenceArray;  iCnt: integer;  sTableName: string; begin  //Setup SQL  db_name := programinfo.cbbsInfo.sMessagesDB;  sTableName := cConferences.GetTableName(sConferenceGiven);  write('Tble: ' + sTableName);   sqlhandle := 'SELECT mailid FROM ' + sTableName;   write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;  // opens Connect, EInOutError if SQLite not installed  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  //Connection for SQL  Query.Open;   write('sql here2');  iCnt := 0;  write('Cnt: ' + Query.RecordCount.Tostring);  if (Query.RecordCount > 0) then  begin  SetLength(MailRecordArray, Query.RecordCount);    repeat      MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;      query.Next;      iCnt := iCnt + 1;    until (iCnt = Query.RecordCount);   end;   //Free Resources  Query.Close;  Query.Free;  Trans.Free;  Connect.Free;   if (length(MailRecordArray) > 0) then  begin    exit(MailRecordArray); //Return Mail Array  end  else    exit(); end; ---------------------------------------------------------------------- paweld · 2022-03-14 05:34 you need to fetch all records to get the correct count, and sqlquery by default fetches in batches of 10 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   ...  write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;    Query.PacketRecords := -1;  // <-- FetchAll  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  Query.Open;  write('sql here2');  ... ---------------------------------------------------------------------- Handoko · 2022-03-14 05:53 I use this function to get the query for record count: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function SQGetRecordCount(const TableName: string; const Param: string): string;begin  Result := '';  If TableName.IsEmpty then Exit;  Result := 'SELECT COUNT(*) FROM ' + TableName + ' ' + Param + ' ;';end; Note: The function only provides you query string not the actually result. I learn SQL and CSS from w3schools, they have good documentations: https://www.w3schools.com/sql/sql_count_avg_sum.asp ---------------------------------------------------------------------- rvk · 2022-03-14 06:57 Quote from: nugax on March 14, 2022, 01:59:28 amCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   write('sql here2');  iCnt := 0;  write('Cnt: ' + Query.RecordCount.Tostring);  if (Query.RecordCount > 0) then  begin  SetLength(MailRecordArray, Query.RecordCount);    repeat      MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;      query.Next;      iCnt := iCnt + 1;    until (iCnt = Query.RecordCount);  end; As said above by others... The TDataset.RecordCount is not wrong. You are just using it wrong. It doesn't mean the count off all records in your table. It means the number of records in the dataset (retrieved) and that can differ. BTW. This is well documented (read here https://www.freepascal.org/docs- html/fcl/db/tdataset.recordcount.html) But you are also using RecordCount wrong. You shouldn't use RecordCount to iterate through a dataset. You should use "while not TDataset.EOF". Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   write('sql here2');  iCnt := 0;  SetLength(MailRecordArray, iCnt);  while not Query.EOF do  begin    iCnt := iCnt + 1;    SetLength(MailRecordArray, iCnt);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;  end;  write('Cnt: ' + iCnt.ToString); // although you could also use Query.RecordCount.Tostring here                                  // because NOW the dataset is itterated all the way though. So, although you could use a COUNT(*) or set PacketRecords to -1, like mentoined above, it's better to iterate though a dataset using TDataset.EOF and not use TDataset.RecordCount for that !! ---------------------------------------------------------------------- Zvoni · 2022-03-14 07:47 IMO, the only real world use for RecordCount is a check for RecordCount=0 (though there might be a valid argument for using EOF and BOF to be True) Everything else as said above ---------------------------------------------------------------------- nugax · 2022-03-14 14:19 thanks for the info as always! I didnt know that about query.recordcount  - i thought it returned, well the count of records of a query! Also, didnt know that query needed any extra arguement to return all records. Thank you! ill make the changes! Quote from: Zvoni on March 14, 2022, 08:47:51 amIMO, the only real world use for RecordCount is a check for RecordCount=0 (though there might be a valid argument for using EOF and BOF to be True) Everything else as said above ---------------------------------------------------------------------- nugax · 2022-03-14 14:27 I put the suggested changes in, it still fails after the test: sql here2.. Am I missing something? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetAllMailFromConference(sConferenceGiven: string): ConferenceArray;var  Connect: TSQLite3Connection;  Trans: TSQLTransaction;  Query: TSQLQuery;  db_name: string[100];  sqlhandle: string[255];  MailRecordArray: ConferenceArray;  iCnt: integer;  sTableName: string; begin  //Setup SQL  db_name := programinfo.cbbsInfo.sMessagesDB;  sTableName := cConferences.GetTableName(sConferenceGiven);  write('Tble: ' + sTableName);   sqlhandle := 'SELECT mailid FROM ' + sTableName;   write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;  // opens Connect, EInOutError if SQLite not installed  Query.PacketRecords:= 1; //Fetch All Records  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  //Connection for SQL  Query.Open;   write('sql here2');  iCnt := 0;  SetLength(MailRecordArray, iCnt);  while not Query.EOF do  begin    iCnt := iCnt + 1;    SetLength(MailRecordArray, iCnt);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;  end;   write('Record Cnt: ' + iCnt.ToString);   //Free Resources  Query.Close;  Query.Free;  Trans.Free;  Connect.Free;   if (length(MailRecordArray) > 0) then  begin    exit(MailRecordArray); //Return Mail Array  end  else    exit(); end;[code=pascal][/code] Quote from: paweld on March 14, 2022, 06:34:46 amyou need to fetch all records to get the correct count, and sqlquery by default fetches in batches of 10 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   ...  write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;    Query.PacketRecords := -1;  // <-- FetchAll  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  Query.Open;  write('sql here2');  ... ---------------------------------------------------------------------- rvk · 2022-03-14 15:10 You can remove the Query.PacketRecords:= 1; For one thing, it should have been -1 and not 1. Second, you don't need it because you are woring with Query.EOF now. You say it fails. With what error???? ---------------------------------------------------------------------- nugax · 2022-03-14 16:04 I cant see the error because the application is a telnet server so it just drops the connection but the program stays running. I think I can put in a way to write out the error, I just dont know how at this point. I removed the packet records and it still just drops. How can I put in a  error check so it will print to console? Quote from: rvk on March 14, 2022, 04:10:09 pmYou can remove the Query.PacketRecords:= 1; For one thing, it should have been -1 and not 1. Second, you don't need it because you are woring with Query.EOF now. You say it fails. With what error???? ---------------------------------------------------------------------- rvk · 2022-03-14 16:07 Quote from: nugax on March 14, 2022, 05:04:27 pmHow can I put in a  error check so it will print to console? If it's just console writeln and readln are your friend. This will write something to the screen: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i writeln('debugging: ' + iCnt.ToString); // or something And this will pause the program until you press enter. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i readln;So, if you put the readln below the while loop, you can see if it ran correctly. ---------------------------------------------------------------------- nugax · 2022-03-14 16:09 Oh yea, been doing that. Its not populating the array to return. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString; So, it doesnt have the primary key mailid when the next loop tries to print each mail item in a seperate routine. The SQ: shows that mailid isnt being returned. ---------------------------------------------------------------------- rvk · 2022-03-14 16:15 Quote from: nugax on March 14, 2022, 05:09:53 pmThe SQ: shows that mailid isnt being returned. Shouldn't the line Query.FieldByName('mailid').AsString in that case crash your program. FieldByName returns nil if mailid isn't in your fields so the .AsString would crap out your program. Are you sure all the records are iterated? Or does your query even return any records ??? (you could check that with a writeln inside the loop) ---------------------------------------------------------------------- Thaddy · 2022-03-14 16:22 Do not forget about assertions. ---------------------------------------------------------------------- nugax · 2022-03-14 16:28 You were right, its failing to assign the return (which DOES contain mailid) to the array that I have it returning to, in the calling routine. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Write('here2');       arrayConfMessages := sqlite_mail.GetAllMailFrom Conference(userhandler.recCurrentUser.LastConference);      sMsgListDisplay := 'All';      Write('here3');      write('ID: ' + arrayConfMessages[0].mailid); Quote from: Thaddy on March 14, 2022, 05:22:58 pmDo not forget about assertions. ---------------------------------------------------------------------- nugax · 2022-03-14 16:30 Here is the full routine that gets the mailid from the sqlite db: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetAllMailFromConference(sConferenceGiven: string): ConferenceArray;var  Connect: TSQLite3Connection;  Trans: TSQLTransaction;  Query: TSQLQuery;  db_name: string[100];  sqlhandle: string[255];  MailRecordArray: ConferenceArray;  iCnt: integer;  sTableName: string; begin  //Setup SQL  db_name := programinfo.cbbsInfo.sMessagesDB;  sTableName := cConferences.GetTableName(sConferenceGiven);  write('Tble: ' + sTableName);   sqlhandle := 'SELECT mailid FROM ' + sTableName;   write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;  // opens Connect, EInOutError if SQLite not installed  //Query.PacketRecords:= -1; //Fetch All Records  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  //Connection for SQL  Query.Open;   write('sql here2');  iCnt := 0;  SetLength(MailRecordArray, iCnt);  while not Query.EOF do  begin    iCnt := iCnt + 1;    SetLength(MailRecordArray, iCnt);    writeln('MailID: ' + Query.FieldByName('mailid').AsString);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;  end;   write('Record Cnt: ' + iCnt.ToString);   //Free Resources  Query.Close;  Query.Free;  Trans.Free;  Connect.Free;   if (length(MailRecordArray) > 0) then  begin    exit(MailRecordArray); //Return Mail Array  end  else    exit(); end; ---------------------------------------------------------------------- nugax · 2022-03-14 16:32 And this is the calling routine, that takes the mailid, looks up the details, and then displays a list to select from: Error has to be around here somewhere: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i //List Messages From A Local or EchoMail Conferenceprocedure ListConferenceMessages(iUnreadOnly: integer);var  // array_messages: messagetools.LocalMailArray;  iCnt: integer;  iNumbMessages: integer;  recCurrentMail: messagetools.ConfMessageRecord;  sTypeConference: string;  sFromName: string;  sMsgListDisplay: string;  iCheckUnread: integer; begin   Write('here');   //Clear Array  //setlength(arrayConfMessages, 0);    //Set Counter  icnt := 0;  iNumbMessages := 0;   //Load Conference   case (iUnreadOnly) of    1: begin      //  arrayLocalMessages := sqlite_mail.GetUnreadMailFromLocalMailConf();      sMsgListDisplay := 'Unread';     end;    0: begin      Write('here2');       arrayConfMessages := sqlite_mail.GetAllMailFromConference(userhandler. recCurrentUser.LastConference);      sMsgListDisplay := 'All';      Write('here3');      write('ID: ' + arrayConfMessages[0].mailid);    end;    else    begin      //  arrayLocalMessages := sqlite_mail.GetAllMailFromLocalMailConf();      sMsgListDisplay := 'All';    end;  end;   Write('here');   //Get/Set Number Of Messages In Array  iNumbMessages := length(arrayConfMessages);  Write('nmb: ' + iNumbMessages.ToString);   //Set Blinking For Unread  if ((sMsgListDisplay = 'Unread') and (iNumbMessages > 0)) then  begin    sMsgListDisplay := '|BLO|12Unread|BLF|10';  end;   cyberio.PipeWriteCRLF;  cyberio.PipeWriteCRLF;  cyberio.PipeWriteLNString('|10' + sMsgListDisplay + ' Messages In Conference: |11' + UserHandler.recCurrentUser.LastConference);  cyberio.PipeWriteLNString('|14ID:     Date:       From:         Subject:                        ');  cyberio.PipeWriteCRLF;  repeat  if (iNumbMessages > 0) then    begin       recCurrentMail := sqlite_mail.GetConfMailByID(arrayConfMessages[iCnt].mailid);      cyberio.PipeWriteString('|15[' + recCurrentMail.mailid + ']     ' + '|11' + recCurrentMail.date.Substring(0, 9) + '   |11' + recCurrentMail.from_name + '  -> |10' + recCurrentMail.subject.Substring(0, 30));         {      iCheckUnread := messagetools.MessageRead(UserHandler.recCurrentUser.LastConference, arrayLocalMessages[iCnt].mailid);      //Check To See If Message Is Unread      if (iCheckUnread = 0) then      begin        cyberio.PipeWriteString('   |11[|14UNREAD|11]');      end;      }     //Finish Writing Line      cyberio.PipeWriteCRLF;      iCnt := iCnt + 1;    end;     if (iNumbMessages = 0) then    begin      cyberio.PipeWriteLnString('|11 No |14' + sMsgListDisplay + '|11 messages in conference');    end;   until (iCnt = iNumbMessages);   write('at prompt');   //Read Message Prompt  readmessages.MessageListPrompt(); end; ---------------------------------------------------------------------- nugax · 2022-03-14 16:33 The write with ID: in case 0:, is empty. So the returned array is empty. ---------------------------------------------------------------------- Zvoni · 2022-03-14 17:46 Quote from: nugax on March 14, 2022, 05:30:39 pmHere is the full routine that gets the mailid from the sqlite db: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetAllMailFromConference(sConferenceGiven: string): ConferenceArray;var  Connect: TSQLite3Connection;  Trans: TSQLTransaction;  Query: TSQLQuery;  db_name: string[100];  sqlhandle: string[255];  MailRecordArray: ConferenceArray;  iCnt: integer;  sTableName: string; begin  //Setup SQL  db_name := programinfo.cbbsInfo.sMessagesDB;  sTableName := cConferences.GetTableName(sConferenceGiven);  write('Tble: ' + sTableName);   sqlhandle := 'SELECT mailid FROM ' + sTableName;   write('sql here');  //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;  // opens Connect, EInOutError if SQLite not installed  //Query.PacketRecords:= -1; //Fetch All Records  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  //Connection for SQL  Query.Open;   write('sql here2');  iCnt := 0;  SetLength(MailRecordArray, iCnt);  while not Query.EOF do  begin    iCnt := iCnt + 1;    SetLength(MailRecordArray, iCnt);    writeln('MailID: ' + Query.FieldByName('mailid').AsString);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;  end;   write('Record Cnt: ' + iCnt.ToString);   //Free Resources  Query.Close;  Query.Free;  Trans.Free;  Connect.Free;   if (length(MailRecordArray) > 0) then  begin    exit(MailRecordArray); //Return Mail Array  end  else    exit(); end;        Beginning in line 34 iCnt is zero In line 38 you increase iCnt by one (iCnt=1) In line 39 you set the lengrh of the Array to 1 In line 41 you try to access array-member „1“ which doesn’t exist ---------------------------------------------------------------------- Zvoni · 2022-03-14 17:49 In line 34 initialize with 1, remove line 35, move line 38 after line 42, And in line 41 use iCnt-1 as access-variable ---------------------------------------------------------------------- rvk · 2022-03-14 18:42 Quote from: Zvoni on March 14, 2022, 06:49:23 pmIn line 34 initialize with 1, remove line 35, move line 38 after line 42, And in line 41 use iCnt-1 as access-variable Just changing MailRecordArray[iCnt].mailid to MailRecordArray[iCnt - 1].mailid in the given code is enough. ---------------------------------------------------------------------- nugax · 2022-03-14 19:02 Geez, thanks. How did I miss that? ---------------------------------------------------------------------- nugax · 2022-03-14 21:43 Here is how i fixed it: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetAllMailFromConference(sConferenceGiven: string): ConferenceArray;var  Connect: TSQLite3Connection;  Trans: TSQLTransaction;  Query: TSQLQuery;  db_name: string[100];  sqlhandle: string[255];  MailRecordArray: ConferenceArray;  iCnt: integer;  sTableName: string; begin  //Setup SQL  db_name := programinfo.cbbsInfo.sMessagesDB;  sTableName := cConferences.GetTableName(sConferenceGiven);   sqlhandle := 'SELECT mailid FROM ' + sTableName;   //Code to update password from handle  Connect := TSQLite3Connection.Create(nil);  Query := TSQLQuery.Create(nil);  Trans := TSQLTransaction.Create(Connect);  Connect.Transaction := Trans;  Connect.DatabaseName := db_name;  Trans.StartTransaction;  // opens Connect, EInOutError if SQLite not installed  //Query.PacketRecords:= -1; //Fetch All Records  Query.SQL.Text := sqlhandle;  Query.Database := Connect;  //Connection for SQL  Query.Open;   iCnt := 0;  SetLength(MailRecordArray, 1);  while not Query.EOF do  begin    SetLength(MailRecordArray, iCnt + 1);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;    iCnt := iCnt + 1;  end;   //Free Resources  Query.Close;  Query.Free;  Trans.Free;  Connect.Free;   if (length(MailRecordArray) > 0) then  begin    exit(MailRecordArray); //Return Mail Array  end  else    exit(); end; ---------------------------------------------------------------------- Zvoni · 2022-03-14 21:46 Remove line 32. It‘s superfluous ---------------------------------------------------------------------- rvk · 2022-03-14 22:50 Quote from: nugax on March 14, 2022, 10:43:30 pmCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   iCnt := 0;  while not Query.EOF do  begin    SetLength(MailRecordArray, iCnt + 1);    MailRecordArray[iCnt].mailid := Query.FieldByName('mailid').AsString;    Query.Next;    iCnt := iCnt + 1;  end;   Which is the same as Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i iCnt := 0;while not Query.EOF dobegin  inc(iCnt);  SetLength(MailRecordArray, iCnt);  MailRecordArray[iCnt - 1].mailid := Query.FieldByName('mailid').AsString; // 0-based  Query.Next;end; I find it more clear to inc(iCnt) before setting SetLength to a new length. But that could be just personal preference. ---------------------------------------------------------------------- Zvoni · 2022-03-15 07:24 Quote from: rvk on March 14, 2022, 11:50:17 pmI find it more clear to inc(iCnt) before setting SetLength to a new length. But that could be just personal preference. Agreed. Initializing iCnt with 0 makes sense. If the Query doesn't return anything, EOF is true, and the While-Loop is never entered, ergo: the "Counter" iCnt stays 0