[SOLVED]How to convert an Integer Array to a Byte Array use the same memory? lazarus / 16-Databases ---------------------------------------------------------------------- tomitomy · 2017-10-09 04:36 这里可以说中文吗?我的英语很差。 我想把一个整数数组存入数据库,但是数据库只接受 TBytes,请问如何把整数数组转换为字节数组? 为了提高读写速度,我不想申请新的内存,而是直接使用整数数组的内存,这可以实现吗? 我目前的做法是直接在 TBytes 中读写整数,不过这样的代码不清晰,很难读,是否有更好的方法? 下面是我用的方法: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetIntFromBytes(index: integer; bytes: TBytes): integer;begin  Result := PInteger(@bytes[index * Sizeof(integer)])^;end; procedure SetIntToBytes(index: integer; bytes: TBytes; data: integer);begin  PInteger(@bytes[index * Sizeof(integer)])^ := data;end; function CountIntsInBytes(bytes: TBytes): integer;begin  Result := Length(bytes) div Sizeof(integer);end; function InsertIntIntoBytes(data: integer; refData: integer; bytes: TBytes;  before: boolean = True): TBytes;var  i, j: integer;begin  // Get insert point  for i := 0 to CountIntsInBytes(bytes) - 1 do    if GetIntFromBytes(i, bytes) = refData then      Break;  // Resize TBytes  SetLength(bytes, (CountIntsInBytes(bytes) + 1) * Sizeof(integer));  // Move data  if not before then    i := i + 1;  for j := CountIntsInBytes(bytes) - 1 downto i + 1 do    SetIntToBytes(j, bytes, GetIntFromBytes(j - 1, bytes));  // Insert data  SetIntToBytes(i, bytes, data);  Result := bytes;end; procedure Test;var  i: integer;  bytes: TBytes;begin  bytes := InsertIntIntoBytes(1, 0, bytes);  bytes := InsertIntIntoBytes(2, 1, bytes);  bytes := InsertIntIntoBytes(3, 1, bytes);  bytes := InsertIntIntoBytes(4, 1, bytes, False);  bytes := InsertIntIntoBytes(5, 1, bytes, False);  for i := 0 to CountIntsInBytes(bytes) - 1 do    ShowMessage(IntToStr(GetIntFromBytes(i, bytes)));end;          ------------------------------------------------------------ Translated by Google: Can I speak Chinese here? My English is poor. I want to put an integer array into a database, but the database only accept TBytes, how to convert an integer array into a byte array? In order to improve read and write speed, I do not want to allocate new memory, but direct use the momory of integer array, which can be achieved? My current practice is to read and write integers directly in TBytes, but this code is not clear, it is difficult to read, is there a better way? Here's my practice: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function GetIntFromBytes(index: integer; bytes: TBytes): integer;begin  Result := PInteger(@bytes[index * Sizeof(integer)])^;end; procedure SetIntToBytes(index: integer; bytes: TBytes; data: integer);begin  PInteger(@bytes[index * Sizeof(integer)])^ := data;end; function CountIntsInBytes(bytes: TBytes): integer;begin  Result := Length(bytes) div Sizeof(integer);end; function InsertIntIntoBytes(data: integer; refData: integer; bytes: TBytes;  before: boolean = True): TBytes;var  i, j: integer;begin  // Get insert point  for i := 0 to CountIntsInBytes(bytes) - 1 do    if GetIntFromBytes(i, bytes) = refData then      Break;  // Resize TBytes  SetLength(bytes, (CountIntsInBytes(bytes) + 1) * Sizeof(integer));  // Move data  if not before then    i := i + 1;  for j := CountIntsInBytes(bytes) - 1 downto i + 1 do    SetIntToBytes(j, bytes, GetIntFromBytes(j - 1, bytes));  // Insert data  SetIntToBytes(i, bytes, data);  Result := bytes;end; procedure Test;var  i: integer;  bytes: TBytes;begin  bytes := InsertIntIntoBytes(1, 0, bytes);  bytes := InsertIntIntoBytes(2, 1, bytes);  bytes := InsertIntIntoBytes(3, 1, bytes);  bytes := InsertIntIntoBytes(4, 1, bytes, False);  bytes := InsertIntIntoBytes(5, 1, bytes, False);  for i := 0 to CountIntsInBytes(bytes) - 1 do    ShowMessage(IntToStr(GetIntFromBytes(i, bytes)));end; ---------------------------------------------------------------------- taazz · 2017-10-09 06:11 tbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. ---------------------------------------------------------------------- Thaddy · 2017-10-09 07:25 As long as the size of the integer is known (2,4,8? bytes) Simply cast it or use absolute. example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program program1;var  a:array[0..9] of integer = (0,1,2,3,4,5,6,7,8,9);  b:array[0..9*SizeOf(Integer)] of byte absolute a;  c:byte;begin  for c in b do writeln(c);// should write some extra zeros depending on int size and compiler mode...end.  - A standard compile will write one extra zero (integer =16 bit = 2 bytes in TP or default mode) - Mode Delphi or ObjFPC will write 3 extra zero's (integer = 32 bit = 4 ) ---------------------------------------------------------------------- tomitomy · 2017-10-09 09:26 Quote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? ---------------------------------------------------------------------- Thaddy · 2017-10-09 09:33 Quote from: tomitomy on October 09, 2017, 11:26:25 amQuote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? Yes, but for your purpose you should not use TBytes. See my example for sharing an integer and a byte array in the exact same memory. That can not be guaranteed with a dynamic array (they can be relocated...). Usually I use absolute if array size is known. @Taazz plz read subject! A better way is maybe a variant record with an array of integer and an array of byte.... He needs the same memory, w/o copy. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program program1;{$mode fpc}var  overlay: packed record  case boolean of    false:(a:array[0..9] of integer);    true :(b:array[0..9*SizeOf(Integer)] of byte);  end;beginoverlay.a[0]:=1;writeln(overlay.b[0],#9,overlay.b[1]);  overlay.a[0]:=256;writeln(overlay.b[0],#9,overlay.b[1]);  end. Same can be done with pointers, of course, but that is an excercise. ---------------------------------------------------------------------- tomitomy · 2017-10-09 10:00 Quote from: Thaddy on October 09, 2017, 11:33:20 amQuote from: tomitomy on October 09, 2017, 11:26:25 amQuote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? Yes, but for your purpose you should not use TBytes. See my example for sharing an integer and a byte array in the exact same memory. That can not be guaranteed with a dynamic array (they can be relocated...). @Taazz plz read subject! A better way is maybe a variant record with an array of integer and an array of byte.... He needs the same memory, w/o copy. But my integer array is a dynamic array, I need to dynamically add and delete elements, and I want write it to a SQLite database, if I don't use TBytes, I don't know how to write it to the database. ---------------------------------------------------------------------- taazz · 2017-10-09 10:02 Quote from: tomitomy on October 09, 2017, 11:26:25 amQuote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? I need to see the code that uses the TBytes in order to provide a sample. ---------------------------------------------------------------------- tomitomy · 2017-10-09 10:51 Quote from: taazz on October 09, 2017, 12:02:04 pmQuote from: tomitomy on October 09, 2017, 11:26:25 amQuote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? I need to see the code that uses the TBytes in order to provide a sample. this is my code (Simplified): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Unit1; {$mode objfpc}{$H+} interface uses  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, sqlite3conn, sqldb; type  TTestDB = class(TObject)    SQLConn: TSQLite3Connection;    SQLQuery: TSQLQuery;    SQLTran: TSQLTransaction;  private    procedure CreateTable;  public    constructor Create;    destructor Destroy; override;     function OpenDB(FileName: string): boolean;    procedure CloseDB(Save: boolean = True);     function GetData(ID: integer): TBytes;    procedure SetData(ID: integer; Data: TBytes);   function NewRecord(Data: TBytes): integer;  end;   { TForm1 }   TForm1 = class(TForm)    procedure FormCreate(Sender: TObject);  end; var  Form1: TForm1; implementation {$R *.lfm} procedure Test;var  FileName: string;  bytes: TBytes;  TestDB: TTestDB;    i: integer;begin  FileName := ExtractFilePath(Application.ExeName);  FileName := ConcatPaths([FileName, 'test.db']);  DeleteFile(FileName);  TestDB := TTestDB.Create;  TestDB.OpenDB(FileName);  SetLength(bytes, 5 * sizeof(integer));  for i := 0 to 4 do    PInteger(@bytes[i * Sizeof(integer)])^ := i + 1;  // Write  TestDB.NewRecord(bytes);  // Read  bytes := TestDB.GetData(1);  for i := 0 to 4 do  begin    ShowMessage(IntToStr(PInteger(@bytes[i * Sizeof(integer)])^));  end;end; procedure TForm1.FormCreate(Sender: TObject);begin  Test;end; constructor TTestDB.Create;var  LibFile: string;  AppPath: string;begin  inherited Create;  SQLConn := TSQLite3Connection.Create(nil);  SQLTran := TSQLTransaction.Create(nil);  SQLQuery := TSQLQuery.Create(nil);   SQLConn.Transaction := SQLTran;  SQLTran.DataBase := SQLConn;  SQLQuery.DataBase := SQLConn;   AppPath := ExtractFilePath(Application.ExeName);{$ifdef MSWINDOWS}  LibFile := ConcatPaths([AppPath, 'lib', 'sqlite3.dll']);{$else}  LibFile := ConcatPaths([AppPath, 'lib', 'libsqlite3.so']);{$endif}  if FileExists(LibFile) then    SQLiteLibraryName := LibFile;end; destructor TTestDB.Destroy;begin  FreeAndNil(SQLQuery);  FreeAndNil(SQLTran);  FreeAndNil(SQLConn);  inherited Destroy;end; procedure TTestDB.CreateTable;begin  SQLConn.ExecuteDirect('Create Table "main"("data" Blob);');end; function TTestDB.OpenDB(FileName: string): boolean;begin  SQLConn.DatabaseName := FileName;  SQLConn.Open;  SQLTran.Active := True;  CreateTable;end; procedure TTestDB.CloseDB(Save: boolean = True);begin  if SQLTran.Active then  if Save then      SQLTran.Commit    else      SQLTran.Rollback;  SQLConn.Close(True);end; function TTestDB.NewRecord(Data: TBytes): integer;begin  SQLQuery.SQL.Text :=    'Insert Into main (data) Values (:data)';  SQLQuery.Params.ParamByName('data').AsBytes := Data;  SQLQuery.ExecSQL;  SQLQuery.SQL.Text := 'Select last_insert_rowid() From main';  SQLQuery.Open;  Result := SQLQuery.Fields[0].AsInteger;  SQLQuery.Close;end; function TTestDB.GetData(ID: integer): TBytes;begin  SQLQuery.SQL.Text := 'Select data From main where rowid=:rowid';  SQLQuery.Params.ParamByName('rowid').AsInteger := ID;  SQLQuery.Open;  Result := SQLQuery.Fields[0].AsBytes;  SQLQuery.Close;end; procedure TTestDB.SetData(ID: integer; Data: TBytes);begin  SQLQuery.SQL.Text := 'Update main Set data=:data Where rowid=:rowid';  SQLQuery.Params.ParamByName('data').AsBytes := Data;  SQLQuery.Params.ParamByName('rowid').AsInteger := ID;  SQLQuery.ExecSQL;end; end. ---------------------------------------------------------------------- taazz · 2017-10-09 20:12 Quote from: tomitomy on October 09, 2017, 12:51:20 pmQuote from: taazz on October 09, 2017, 12:02:04 pmQuote from: tomitomy on October 09, 2017, 11:26:25 amQuote from: taazz on October 09, 2017, 08:11:47 amtbytes is a dynamic array that means that the length and size of the array are managed internally with hidden fields it is impossible to cast one dynamic array to an other and have the rtl function properly, it might be easier to extend the database component to support a pbyte and a length parameter instead of a tbytes. I do not know how to implement, is there a sample code? I need to see the code that uses the TBytes in order to provide a sample. this is my code (Simplified): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Unit1; {$mode objfpc}{$H+} interface uses  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, sqlite3conn, sqldb; type  TTestDB = class(TObject)    SQLConn: TSQLite3Connection;    SQLQuery: TSQLQuery;    SQLTran: TSQLTransaction;  private    procedure CreateTable;  public    constructor Create;    destructor Destroy; override;     function OpenDB(FileName: string): boolean;    procedure CloseDB(Save: boolean = True);     function GetData(ID: integer): TBytes;    procedure SetData(ID: integer; Data: TBytes);   function NewRecord(Data: TBytes): integer;  end;   { TForm1 }   TForm1 = class(TForm)    procedure FormCreate(Sender: TObject);  end; var  Form1: TForm1; implementation {$R *.lfm} procedure Test;var  FileName: string;  bytes: TBytes;  TestDB: TTestDB;    i: integer;begin  FileName := ExtractFilePath(Application.ExeName);  FileName := ConcatPaths([FileName, 'test.db']);  DeleteFile(FileName);  TestDB := TTestDB.Create;  TestDB.OpenDB(FileName);  SetLength(bytes, 5 * sizeof(integer));  for i := 0 to 4 do    PInteger(@bytes[i * Sizeof(integer)])^ := i + 1;  // Write  TestDB.NewRecord(bytes);  // Read  bytes := TestDB.GetData(1);  for i := 0 to 4 do  begin    ShowMessage(IntToStr(PInteger(@bytes[i * Sizeof(integer)])^));  end;end; procedure TForm1.FormCreate(Sender: TObject);begin  Test;end; constructor TTestDB.Create;var  LibFile: string;  AppPath: string;begin  inherited Create;  SQLConn := TSQLite3Connection.Create(nil);  SQLTran := TSQLTransaction.Create(nil);  SQLQuery := TSQLQuery.Create(nil);   SQLConn.Transaction := SQLTran;  SQLTran.DataBase := SQLConn;  SQLQuery.DataBase := SQLConn;   AppPath := ExtractFilePath(Application.ExeName);{$ifdef MSWINDOWS}  LibFile := ConcatPaths([AppPath, 'lib', 'sqlite3.dll']);{$else}  LibFile := ConcatPaths([AppPath, 'lib', 'libsqlite3.so']);{$endif}  if FileExists(LibFile) then    SQLiteLibraryName := LibFile;end; destructor TTestDB.Destroy;begin  FreeAndNil(SQLQuery);  FreeAndNil(SQLTran);  FreeAndNil(SQLConn);  inherited Destroy;end; procedure TTestDB.CreateTable;begin  SQLConn.ExecuteDirect('Create Table "main"("data" Blob);');end; function TTestDB.OpenDB(FileName: string): boolean;begin  SQLConn.DatabaseName := FileName;  SQLConn.Open;  SQLTran.Active := True;  CreateTable;end; procedure TTestDB.CloseDB(Save: boolean = True);begin  if SQLTran.Active then  if Save then      SQLTran.Commit    else      SQLTran.Rollback;  SQLConn.Close(True);end; function TTestDB.NewRecord(Data: TBytes): integer;begin  SQLQuery.SQL.Text :=    'Insert Into main (data) Values (:data)';  SQLQuery.Params.ParamByName('data').AsBytes := Data;  SQLQuery.ExecSQL;  SQLQuery.SQL.Text := 'Select last_insert_rowid() From main';  SQLQuery.Open;  Result := SQLQuery.Fields[0].AsInteger;  SQLQuery.Close;end; function TTestDB.GetData(ID: integer): TBytes;begin  SQLQuery.SQL.Text := 'Select data From main where rowid=:rowid';  SQLQuery.Params.ParamByName('rowid').AsInteger := ID;  SQLQuery.Open;  Result := SQLQuery.Fields[0].AsBytes;  SQLQuery.Close;end; procedure TTestDB.SetData(ID: integer; Data: TBytes);begin  SQLQuery.SQL.Text := 'Update main Set data=:data Where rowid=:rowid';  SQLQuery.Params.ParamByName('data').AsBytes := Data;  SQLQuery.Params.ParamByName('rowid').AsInteger := ID;  SQLQuery.ExecSQL;end; end.The only thing you can do is to convert your data to TBytes or to a string and pass that to the parameter. I would recomend converting it to a string, for the correct procedure of doing so see the method TParam.GetAsString in the db unit, because this is done for your data internally when you execute it, this way you avoid a 3 way convertion. ---------------------------------------------------------------------- tomitomy · 2017-10-10 05:34 thanks @taazz and @Thaddy, all your replies are helpful, thank you very much. I don't want allocate new memory, so i don't want to convert my data to another type, I just want to use the same memory of my data. now, with your help, I have two ways to meet my needs, the one is only use 'absolute', another is change the hidden field which stone the length of the array. method 1 (only use 'absolute'): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  TInts = array of integer; procedure Test;var  FileName: string;  TestDB: TTestDB;  i: integer;  bytes: TBytes;  ints: TInts absolute bytes;  HighInt: integer;begin  FileName := ExtractFilePath(Application.ExeName);  FileName := ConcatPaths([FileName, 'test.db']);  DeleteFile(FileName);  TestDB := TTestDB.Create;  TestDB.OpenDB(FileName);   // Initialize Ints  HighInt := 4;  // don't use SetLength(ints, HighInt +1)  // Bytes are larger in length than ints, they are use the same length field.  SetLength(bytes, (HighInt + 1) * sizeof(integer));  for i := Low(Ints) to HighInt do    Ints[i] := i + 1;  // Write Ints  TestDB.NewRecord(bytes);  // Clear Ints  SetLength(Ints,0);    // Read Ints  bytes := TestDB.GetData(1);  for i := Low(Ints) to HighInt do  begin    ShowMessage(IntToStr(Ints[i]));  end;end; method 2 (change the hidden field): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  TInts = array of integer; function IntsToBytes(ints: TInts):TBytes;var  bytes: TBytes absolute ints;  p: PQword;begin  // Get hidden field address  p := @ints[0];  Dec(p);  // Change hidden field  p^ := Length(Ints) * Sizeof(integer) - 1;  Result := bytes;end; function BytesToInts(bytes: TBytes):TInts;var  ints: TInts absolute bytes;  p: PQword;begin  // Get hidden field address  p := @ints[0];  Dec(p);  // Change hidden field  p^ := Length(bytes) div Sizeof(integer) - 1;  Result := ints;end; procedure Test;var  FileName: string;  TestDB: TTestDB;  i: integer;  ints: TInts;begin  FileName := ExtractFilePath(Application.ExeName);  FileName := ConcatPaths([FileName, 'test.db']);  DeleteFile(FileName);  TestDB := TTestDB.Create;  TestDB.OpenDB(FileName);   // Initialize Ints  SetLength(ints, 5);  for i := Low(Ints) to High(ints) do    Ints[i] := i + 1;  // Write Ints  TestDB.NewRecord(IntsToBytes(Ints));  // Clear Ints  SetLength(Ints,0);    // Read Ints  Ints := BytesToInts(TestDB.GetData(1));  for i := Low(Ints) to High(ints) do  begin    ShowMessage(IntToStr(Ints[i]));  end;end; I don't know if the second method is correct, but it can work properly. ---------------------------------------------------------------------- ASerge · 2017-10-10 22:05 Quote from: tomitomy on October 10, 2017, 07:34:52 amI don't know if the second method is correct, but it can work properly. A little safer with the test: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$APPTYPE CONSOLE}{$MODE OBJFPC}program project1; uses SysUtils; function GetArrHigh(P: Pointer): PDynArrayIndex; inline;begin  Result := P - SizeOf(IntPtr);  // See TDynArray in dynarr.inc end; function IntsToBytes(const Ints: TBoundArray): TBytes;begin  if Assigned(Ints) then    GetArrHigh(@Ints[0])^ := Length(Ints) * SizeOf(Ints[0]) - 1;  Result := TBytes(Ints);end; function BytesToInts(const Bytes: TBytes): TBoundArray;begin  if Assigned(Bytes) then    GetArrHigh(@Bytes[0])^ := Length(Bytes) div SizeOf(TBoundArray[0]) - 1;  Result := TBoundArray(Bytes);end; procedure PrintInts(const Ints: TBoundArray);var  i: Integer;begin  Write(Format('Ints  %p: ', [@Ints[0]]));  for i := Low(Ints) to High(Ints) do    Write(Ints[i], ' ');  Writeln;end; procedure PrintBytes(const Bytes: TBytes);var  i: Integer;begin  Write(Format('Bytes %p: ', [@Bytes[0]]));  for i := Low(Bytes) to High(Bytes) do    Write(Bytes[i], ' ');  Writeln;end; procedure Test;var  Ints: TBoundArray;  Bytes: TBytes;  i: Integer;begin  SetLength(Ints, 5);  for i := Low(Ints) to High(ints) do    Ints[i] := i + 1;  PrintInts(Ints);  Bytes := IntsToBytes(Ints);  //Ints := nil;  PrintBytes(Bytes);  Ints := BytesToInts(Bytes);  //Bytes := nil;  PrintInts(Ints);end; begin  Test;  Readln;end. ---------------------------------------------------------------------- tomitomy · 2017-10-11 01:21 Quote from: ASerge on October 11, 2017, 12:05:18 amA little safer with the test: Your code is very clear and safer, I can use it with confidence, thank you very much.