[SOLVED] Tiff 16 bit grayscale pixel access? lazarus / 39-Graphics ---------------------------------------------------------------------- Frate · 2021-05-17 22:21 Hello, I'm working on a project with DEM tiff data and i need to access pixel color to retrieve elevation value of mutiple points. I was successfull in loading the tiff image onto bitmap using this example https://bugs.freepascal.org/file_download.php?file_id=28524&type=bug The problem is that i'm not able to get the correct color value from bitmap pixel access. The tiff image i'm trying to read is 16 bit grayscale, so i'm using Lsa color space. The issue present itself as following: using photoshop on a pixel i get a reading of L = 2, while using TColorToFPColor(newImage.Canvas.Pixels[x,y]).ToLabA.L I get L = 0.008 . What is going on? Do I need to set the colorspace on the bitimap in advance? Actually I dont need to display the image at all, i just need to do calculations, so skipping the bitmap step would be good. Is it possible to use FPReadTiff directly to get a buffer of the pixels? (Documentation on this?) I hope I could exlain mysef clearly enough. Any help/suggestion appreciated Thanks Frate ---------------------------------------------------------------------- engkin · 2021-05-18 17:54 Notice that 2/256=0.0078125 ~0.008 According to the comments in FPReadTiff: Quote ToDo:    Compression: FAX, Jpeg...    Color format: YCbCr, Lab...    PlanarConfiguration: 2 (one chunk for each channel)    bigtiff 64bit offsets    XMP tag 700    ICC profile tag 34675 and Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     case IFD.PhotoMetricInterpretation of    0,1: // 0:bilevel grayscale 0 is white; 1:0 is black...    2: // RGB(A)...    3: //3 Palette/color map indexed...    //4 Mask/holdout mask (obsolete by TIFF 6.0 specification)     5: // CMYK plus optional alpha...      //6: YCBCR: CCIR 601     //8: CIELAB: 1976 CIE L*a*b*     //9: ICCLAB: ICC L*a*b*. Introduced post TIFF rev 6.0 by Adobe TIFF Technote 4     //10: ITULAB: ITU L*a*b*     //32844: LOGL: CIE Log2(L)     //32845: LOGLUV: CIE Log2(L) (u',v')    else      TiffError('PhotometricInterpretation= '+IntToStr(IFD.PhotoMetricInterpretation)+' not supported');    end; You might want to give BGRABitmap library a try. Read this post for instance. ---------------------------------------------------------------------- winni · 2021-05-18 20:39 Hi! As far as I know there is no simple solution to read 16 bit grayscale GeoTiffs. The Gimp moans something about unknown Tags, BGRAbitmap is not prepared for that, the LCL not. This is a special enhancement of arcGis and so you should search at arcGIS to find some software to get the height level depending on the 16-bit-Grayscale. I dont know any library or software which is ready for this exotic enhancement. Winni ---------------------------------------------------------------------- winni · 2021-05-18 21:26 Hi! I am wrong. Besides all that Python, arcGIS and other rubbish there is still our beloved tool ImageMagic. I did not know that ImageMagic can do that. Converting 16-Bit- Grayscale-Tiff to png. * Solution 1: Install ImageMagic on your machine and then: Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i convert infile.tif -depth 16 -type Grayscale outfile.png * Solution 2: Install PascalMagic on your Lazarus and do the same: https://wiki.freepascal.org/PascalMagick Winni ---------------------------------------------------------------------- Frate · 2021-05-19 13:07 Quote from: engkin on May 18, 2021, 07:54:16 pmNotice that 2/256=0.0078125 ~0.008 Thanks engkin, I'll take a look at cosintency on that calculation. Quote from: engkin on May 18, 2021, 07:54:16 pmYou might want to give BGRABitmap library a try. Read this post for instance. Good news, seems like they have some sort of Tiff Lab support! Quote from: winni on May 18, 2021, 11:26:47 pmI did not know that ImageMagic can do that. Converting 16-Bit-Grayscale-Tiff to png. Thanks Winni, I didnt want to convert to another format because i may loose detail (maybe not?). If i have no other solution I will batch convert al data to some format i can read easily. Its a pretty big dataset around 300 GB so if i can minimize the work on it I'm happy I'll be doing my homeworks and let you know. Frate ---------------------------------------------------------------------- wp · 2021-05-19 13:24 Did you try the large imaging libraries? - FreeImage: https://freeimage.sourceforge.io/ - Vampyre Imaging: https://github.com/galfar/imaginglib or Online- Package-Manager They support a huge selection of image formats. Vampyre directly supports Lazarus, FreeImage has a Delphi wrapper which works in Lazarus as well (if I am wrong here ask again because I once used FreeImage in a project, and I certainly can post the Lazarus version of the wrapper unit). ---------------------------------------------------------------------- winni · 2021-05-19 16:05 Hi! GeoTiff is an enhancement by the Geo fraggles to the Tiff specification. BGRAbitmap cannot do it, even ImageMagic fails although they can read 16 bit grayscal. The only source to read the GeoTiffs is from the inventors of that specialized format themselves. This is GDAL and here are the sources: https://github.com/OSGeo/GDAL Everything in C. Winni ---------------------------------------------------------------------- engkin · 2021-05-19 17:05 It seems to me he is after Lab color space, not the meta data. Specifically the L value of each pixel ---------------------------------------------------------------------- winni · 2021-05-19 18:00 Hi! Unless it is not quiet clear what Frate wants perhaps this canadian page about DEM helps him https://library.carleton.ca/help/dem-formats Winni ---------------------------------------------------------------------- Frate · 2021-05-20 10:13 Quote from: engkin on May 19, 2021, 07:05:06 pmIt seems to me he is after Lab color space, not the meta data. Specifically the L value of each pixel Yep, i need to get the correct pixel color to be used in elevation calculations. Lab color space seemed the best option. I can even go to other color spaces (needs to be 16-bit) as long as no conversion math is happening and I get in some way the actual pixel value. This is what the DEM data docs says about the format : "DEM: GeoTiff, signed 16 bits, and 1m/DN for DEM;" Unfortunately this trick is not working QuoteNotice that 2/256=0.0078125 ~0.008 imaginglib is not recognizing this specific Tif format... and reads as RGBA... Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     if DataFormat = ifUnknown then    begin      // Use RGBA interface to read A8R8G8B8 TIFFs and mainly TIFFs in various      // formats with no Imaging equivalent, exotic color spaces etc. and yes, i don't care about the geotif tags. I have yet to try other solutions you mentioned. Frate ---------------------------------------------------------------------- Frate · 2021-05-20 10:45 with BGRABitmap... DecompressLZW goes wrong ---------------------------------------------------------------------- winni · 2021-05-20 10:53 Quote from: Frate on May 20, 2021, 12:45:03 pmwith BGRABitmap... DecompressLZW goes wrong I told you that before ---------------------------------------------------------------------- wp · 2021-05-20 10:56 Quote from: Frate on May 18, 2021, 12:21:44 amHello, I'm working on a project with DEM tiff data and i need to access pixel color to retrieve elevation value of mutiple points. I was successfull in loading the tiff image onto bitmap using this example https://bugs.freepascal.org/file_download.php?file_id=28524&type=bug The problem is that i'm not able to get the correct color value from bitmap pixel access. The tiff image i'm trying to read is 16 bit grayscale, so i'm using Lsa color space. I am confused by this specification: you say the image is 16-bit grayscale - I assume that the pixels are 16-bit grayscale values, but you are talking of "color space". How does this fit together? Can you upload one of these images, or at least provide us a link to get one of them? ---------------------------------------------------------------------- Frate · 2021-05-20 12:24 Quote from: wp on May 20, 2021, 12:56:41 pmQuote from: Frate on May 18, 2021, 12:21:44 amHello, I'm working on a project with DEM tiff data and i need to access pixel color to retrieve elevation value of mutiple points. I was successfull in loading the tiff image onto bitmap using this example https://bugs.freepascal.org/file_download.php?file_id=28524&type=bug The problem is that i'm not able to get the correct color value from bitmap pixel access. The tiff image i'm trying to read is 16 bit grayscale, so i'm using Lsa color space. I am confused by this specification: you say the image is 16-bit grayscale - I assume that the pixels are 16-bit grayscale values, but you are talking of "color space". How does this fit together? Can you upload one of these images, or at least provide us a link to get one of them? The thing with colorspace is that when i try to get pixel value from an image raster it is returned not as a raw pixel value (which I'm trying to get) but as color in whatever colorspace RGBA, FPcolor etc... This make some conversion in between which i dont want. This is a sample tile (maybe you need to register, so i'll upload one to OneDrive) https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod- protected/ASTGTM.003/ASTGTMV003_N45E012_dem.tif https://1drv.ms/u/s!Ao9fJmV_S5FXiZYTnqNXlSW1qL9-pw?e=sby5Iq This is documentation with tiles specs https://ssl.jspacesystems.or.jp/ersdac/GDEM/ver3Validation/GDEM3_Quick Guide.pdf I think at this time I will convert the database to another raw file format and work with that. Frate ---------------------------------------------------------------------- wp · 2021-05-20 15:22 IrfanView was the only program on my PC to display this file. However, a Lazarus program based on the freeimage.dll is able to display something too, although it is in some kind of false colors - but at least it is something. I don't want to spend too much time in this, but when you play with freeimage you can probably find settings which show the image correctly. My demo program is in the attachment. If you are on Windows you can download the pre-compiled freeimage.dll from https://freeimage.sourceforge.io/download.html; copy it into the folder with the exe of the demo program. ---------------------------------------------------------------------- DomingoGP · 2021-05-20 20:34 Hi You can read the elevation data using the libTiff library. You can use TIFFReadScanline for reading the raw data. Here is a quick and dirty sample loading the raw data without color conversions. If the format of the tiff is tiled then the sample will not work. The code reads a 'test.tif' image in the same folder. I don't include the image because the size is > 500kb and the forum dont allow me to attach it. I hope this code works for you. ---------------------------------------------------------------------- engkin · 2021-05-20 22:22 The image is not black. To be precise, the max value of the sample image is 1566. This value is close to black: 0. On a linear scale it is about 2.4% The sample file is tiled 15x15 tiles. It has 9 "pages" or images. FPReadTiff is able to read it. It does not recognize 6 tags from the first image: QuoteTFPReaderTiff.ReadTiffHeader Endian Big=FALSE ReverseEndian=FALSE ReadIFD Start=8 TFPReaderTiff.ReadDirectoryEntry Tag 256: ImageWidth=3601 TFPReaderTiff.ReadDirectoryEntry Tag 257: ImageHeight=3601 TFPReaderTiff.ReadDirectoryEntry Tag 258: BitsPerSample: 16 TFPReaderTiff.ReadDirectoryEntry Tag 259: Compression=5=LZW TFPReaderTiff.ReadDirectoryEntry Tag 262: PhotometricInterpretation=1=bilevel grayscale 0 is black TFPReaderTiff.ReadDirectoryEntry Tag 277: SamplesPerPixel=1 TFPReaderTiff.ReadDirectoryEntry Tag 284: PlanarConfiguration=chunky format TFPReaderTiff.ReadDirectoryEntry Tag 317: Predictor="1" TFPReaderTiff.ReadDirectoryEntry Tag 322: TileWidth=256 TFPReaderTiff.ReadDirectoryEntry Tag 323: TileLength=256 TFPReaderTiff.ReadDirectoryEntry Tag 324: TileOffsets=130 TFPReaderTiff.ReadDirectoryEntry Tag 325: TileByteCounts=142 TFPReaderTiff.ReadDirectoryEntry Tag=339 Type=3 Count=1 ValuesStart=2 TFPReaderTiff.ReadDirectoryEntry Tag=33550 Type=12 Count=3 ValuesStart=2182 TFPReaderTiff.ReadDirectoryEntry Tag=33922 Type=12 Count=6 ValuesStart=2206 TFPReaderTiff.ReadDirectoryEntry Tag=34735 Type=3 Count=32 ValuesStart=2254 TFPReaderTiff.ReadDirectoryEntry Tag=34736 Type=12 Count=2 ValuesStart=2318 TFPReaderTiff.ReadDirectoryEntry Tag=34737 Type=2 Count=8 ValuesStart=2334 TFPReaderTiff.ReadDirectoryEntry Tag=42112 Type=2 Count=139 ValuesStart=2042 Sample code to read the image, find the max value, and "brighten" the image before saving it to "out1.jpg" Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses...  FPImage, FPReadTiff; function FindMax(Img:TFPCustomImage):word;var  y, x: Integer;  v: Word;begin  Result := 0;  for y:=0 to Img.Height-1 do  begin    for x := 0 to Img.Width-1 do    begin      v := Img.Colors[x,y].Red;      if v>Result then        Result := v;    end;  end;end; procedure BrightenImg(Img: TFPCustomImage; maxV: word);var  y, x: Integer;  v: Word;begin  for y:=0 to Img.Height-1 do  begin    for x := 0 to Img.Width-1 do    begin      v := Img.Colors[x,y].Red;      v := (($FFFF*v) div maxV) and $FFFF;      Img.Colors[x,y] := FPColor(v,v,v);    end;  end;end; procedure TiffToJpg(fn:String='ASTGTMV003_N45E012_dem.tif');var  r:TFPReaderTiff;  Img:TFPMemoryImage;  maxV: Word;begin   Img:=TFPMemoryImage.Create(0,0);  try    r := TFPReaderTiff.Create;  r.Debug:=True;    try      Img.loadfromfile (fn, r);      WriteLn('Img.Width: ', Img.Width);      WriteLn('Img.Height: ', Img.Height);       maxV := FindMax(Img);      WriteLn('Max value: ', maxV);       BrightenImg(Img, maxV);      Img.SaveToFile('out1.jpg');    finally      r.Free;    end;  finally    Img.Free;  end;end; ---------------------------------------------------------------------- Frate · 2021-05-20 23:28 Quote from: wp on May 20, 2021, 05:22:24 pmIrfanView was the only program on my PC to display this file. However, a Lazarus program based on the freeimage.dll is able to display something too, although it is in some kind of false colors - but at least it is something. I don't want to spend too much time in this, but when you play with freeimage you can probably find settings which show the image correctly. My demo program is in the attachment. If you are on Windows you can download the pre-compiled freeimage.dll from https://freeimage.sourceforge.io/download.html; copy it into the folder with the exe of the demo program. Working good! However i couldnt figure out the correct reading settings, tried numerous combinations but no luck. I  looked also to the raw binary value to see if i could identify some patterns in the value but its taking too much time as you said. Thanks for your help. Quote from: engkin on May 21, 2021, 12:22:11 amThe image is not black. To be precise, the max value of the sample image is 1566. This value is close to black: 0. On a linear scale it is about 2.4% The sample file is tiled 15x15 tiles. It has 9 "pages" or images. FPReadTiff is able to read it. It does not recognize 6 tags from the first image: QuoteTFPReaderTiff.ReadTiffHeader Endian Big=FALSE ReverseEndian=FALSE ReadIFD Start=8 TFPReaderTiff.ReadDirectoryEntry Tag 256: ImageWidth=3601 TFPReaderTiff.ReadDirectoryEntry Tag 257: ImageHeight=3601 TFPReaderTiff.ReadDirectoryEntry Tag 258: BitsPerSample: 16 TFPReaderTiff.ReadDirectoryEntry Tag 259: Compression=5=LZW TFPReaderTiff.ReadDirectoryEntry Tag 262: PhotometricInterpretation=1=bilevel grayscale 0 is black TFPReaderTiff.ReadDirectoryEntry Tag 277: SamplesPerPixel=1 TFPReaderTiff.ReadDirectoryEntry Tag 284: PlanarConfiguration=chunky format TFPReaderTiff.ReadDirectoryEntry Tag 317: Predictor="1" TFPReaderTiff.ReadDirectoryEntry Tag 322: TileWidth=256 TFPReaderTiff.ReadDirectoryEntry Tag 323: TileLength=256 TFPReaderTiff.ReadDirectoryEntry Tag 324: TileOffsets=130 TFPReaderTiff.ReadDirectoryEntry Tag 325: TileByteCounts=142 TFPReaderTiff.ReadDirectoryEntry Tag=339 Type=3 Count=1 ValuesStart=2 TFPReaderTiff.ReadDirectoryEntry Tag=33550 Type=12 Count=3 ValuesStart=2182 TFPReaderTiff.ReadDirectoryEntry Tag=33922 Type=12 Count=6 ValuesStart=2206 TFPReaderTiff.ReadDirectoryEntry Tag=34735 Type=3 Count=32 ValuesStart=2254 TFPReaderTiff.ReadDirectoryEntry Tag=34736 Type=12 Count=2 ValuesStart=2318 TFPReaderTiff.ReadDirectoryEntry Tag=34737 Type=2 Count=8 ValuesStart=2334 TFPReaderTiff.ReadDirectoryEntry Tag=42112 Type=2 Count=139 ValuesStart=2042 Sample code to read the image, find the max value, and "brighten" the image before saving it to "out1.jpg" Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses...  FPImage, FPReadTiff; function FindMax(Img:TFPCustomImage):word;var  y, x: Integer;  v: Word;begin  Result := 0;  for y:=0 to Img.Height-1 do  begin    for x := 0 to Img.Width-1 do    begin      v := Img.Colors[x,y].Red;      if v>Result then        Result := v;    end;  end;end; procedure BrightenImg(Img: TFPCustomImage; maxV: word);var  y, x: Integer;  v: Word;begin  for y:=0 to Img.Height-1 do  begin    for x := 0 to Img.Width-1 do    begin      v := Img.Colors[x,y].Red;      v := (($FFFF*v) div maxV) and $FFFF;      Img.Colors[x,y] := FPColor(v,v,v);    end;  end;end; procedure TiffToJpg(fn:String='ASTGTMV003_N45E012_dem.tif');var  r:TFPReaderTiff;  Img:TFPMemoryImage;  maxV: Word;begin   Img:=TFPMemoryImage.Create(0,0);  try    r := TFPReaderTiff.Create;  r.Debug:=True;    try      Img.loadfromfile (fn, r);      WriteLn('Img.Width: ', Img.Width);      WriteLn('Img.Height: ', Img.Height);       maxV := FindMax(Img);      WriteLn('Max value: ', maxV);       BrightenImg(Img, maxV);      Img.SaveToFile('out1.jpg');    finally      r.Free;    end;  finally    Img.Free;  end;end; Engkin this is perfect! All fp also, best solution. It's also quite fast, I'm able to make 1000000 pixel calls a second which is more than good enough for now. About the unrecognized tags, i'm actually not interested in them. they basically give geo position info of the tile (i guess), which i dont need. Quote from: DomingoGP on May 20, 2021, 10:34:10 pmHi You can read the elevation data using the libTiff library. You can use TIFFReadScanline for reading the raw data. Here is a quick and dirty sample loading the raw data without color conversions. If the format of the tiff is tiled then the sample will not work. The code reads a 'test.tif' image in the same folder. I don't include the image because the size is > 500kb and the forum dont allow me to attach it. I hope this code works for you. Thanks DomingoGP, I'll compare your solution to the one from Engkin. Lets see which one is faster. About tiled format I think I need only the first image, but I'm checking. Thank you all for helping out! I'll keep you posted on the final choice Frate ---------------------------------------------------------------------- Frate · 2021-05-21 09:53 Allright, in the end Engkin's code works the best. It allows me to get a pretty big improvement in data quality. See attached image (its a terrain section), green is old, red new data. Again thanks all Great community! Best wishes for everyone Frate