FFT scaling (old chestnut) lazarus / 21-General ---------------------------------------------------------------------- MarkMLl · 2021-01-27 10:59 I wonder whether I could bore the community with a question which is more about using FPC/Lazarus effectively than the language or tools. I have a cheap oscilloscope as a pocket tool: as described at https://hackaday.com/2017/08/08/touchscreen-oscilloscope/ it's a handy bit of kit for its price, although in practice it will need new leads with the appropriate (MMCX?) connectors. I've written a program that captures data over USB, where- as is fairly common- the device looks like a serial port (CP2102). This is entirely adequate to save screenshots to go into project documentation etc., at least in the time domain. Running on Linux x86_64 with FPC 3.2.0 and Lazarus 2.1.0, but also tested on other versions/variants. My problems start when I try to convert the collected time domain data to frequency domain using FFT. I was initially using the code discussed at e.g. https://lists.freepascal.org/pipermail/fpc- pascal/2017-April/050717.html but have also got the same questionable results using FFTW, so it's obviously my interpretation of the results that's at fault. The scope's calibration signal is 1kHz 3.3V. If I capture that with the timebase set to 1 mSec per division I get 512 values, and can derive appropriate voltages etc. for on-screen annotation. There's 20 divisions, so it's 20 mSec across the entire 512 values or 39.06 uSec per value hence a sampling rate of 25.6 kHz. /If/ my understanding is correct, that implies that each bin in the FFT output is 25.6 kHz/512, or 50Hz. However because this is mirrored with the zero in the middle and +-Nyquist at each end what I've actually got is a usable 256 bins from 0 to 12.8 kHz (still 50Hz per bin). Displaying that with an arbitrary 32 divisions should give me 400 Hz per division. Please could somebody who knows what they're doing confirm my logic there before I wade into my code again. I attach a couple of screenshots, showing the sort of problem I'm having. The bigger picture is that I'm trying to get the graphical side of this wrapped up as a way of revisiting TAChart. Then I'm going to add graphical output to a program that reads data from a multimeter (Mastech MS2115B), and the code that does that will be adapted in short order to read a pulse oximeter which is obviously of topical interest. MarkMLl ---------------------------------------------------------------------- Mr.Madguy · 2021-01-27 11:52 I'm not sure about math, because I'm not strong in it, but I use following code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit FFT; {$mode objfpc}{$H+} interface uses Complex; procedure FFTMain(ADepth:Integer;ASource, ADest:PComplexArray); implementation procedure DoFFT(ADepth, AMaxDepth:Integer;ASource, ADest:PComplexArray;ATable:PComplexArray);  var Shift, SourceSpacing, DestSpacing, I:Integer;Temp:TComplex;begin  if ADepth = 0 then begin    ADest^[0] := ASource^[0];    Exit;  end;  Shift := AMaxDepth - ADepth;  SourceSpacing := 1 shl Shift;  DestSpacing := 1 shl (ADepth - 1);  DoFFT(ADepth - 1, AMaxDepth, ASource, ADest, ATable);  DoFFT(ADepth - 1, AMaxDepth, @ASource^[SourceSpacing], @ADest^[DestSpacing], ATable);  for I := 0 to DestSpacing - 1 do begin    Temp := ADest^[DestSpacing + I] * ATable^[I shl Shift];    ADest^[DestSpacing + I] := ADest^[I] - Temp;    ADest^[I] := ADest^[I] + Temp;  end;end; procedure FFTMain(ADepth:Integer;ASource, ADest:PComplexArray);  var Table:PComplexArray;Count, I:Integer;begin  Count := 1 shl ADepth;  Table := GetMem(Count * SizeOf(TComplex));  for I := 0 to Count - 1 do begin    Table^[I].Re := Cos(-(2 * Pi) * I / Count);    Table^[I].Im := Sin(-(2 * Pi) * I / Count);  end;  DoFFT(ADepth, ADepth, ASource, ADest, Table);  for I := 0 to Count - 1 do begin    ADest^[I] := ADest^[I] / Count;  end;  FreeMem(Table);end; end. And only problem I have - is mirroring. Picture 1 shows "left" mirroring and picture 2 shows "right" one. I don't know, how to beat it and I don't care, because it's just example of uncertainty principle in quantum mechanic. ---------------------------------------------------------------------- avra · 2021-01-27 12:15 Quote from: MarkMLl on January 27, 2021, 11:59:31 am...hence a sampling rate of 25.6 kHz.This does not sound right. Can you test with 0.1 ms/div and report difference in findings? How do you read data? Do you get CSV file as explained at http://www.j yetech.com/Products/LcdScope/CaptureUploadingAndWaveformFileFormat.pdf or you use USB Scope mode and DataBlock read function as in http://www.jyetech.com/Products/LcdScope/TheDataInterface_068.pdf ? Do you use Auto state or Manual state? You can compare FFT results of your sample data online https://duckduckgo.com/?q=fft+online&ia=web. Btw, there is an unofficial sigrok driver for your device: https://sourceforge.net/p/sigrok/mailman/message/35238342/ https://github.com/mwm/libsigrok/tree/dso112a/src/hardware/jyetech- dso112a EDIT: From Data Interface manual: QuoteNote that when the timebase is set between 20ms/div – 0.5us/div (inclusive) the whole buffer of data is sent (as DataBlock frame) after each capture. When the timebase is set between 10min/div and 50ms/div (inclusive) single sample is sent (as DataSample frame) after each sampling.Maybe buffer length is 512 and your perception is wrong? If you get the same 512 bytes as block size for both 1 ms/div and 0.1 ms/div then it might be the case. Or device always sends fixed 512 bytes for 20 divisions? ---------------------------------------------------------------------- MarkMLl · 2021-01-27 12:28 Quote from: avra on January 27, 2021, 01:15:02 pmQuote from: MarkMLl on January 27, 2021, 11:59:31 am...hence a sampling rate of 25.6 kHz.This does not sound right. Can you test with 0.1 ms/div and report difference in findings? That was a worked-out figure rather than an observed one, so I'm very interested to know /why/ it doesn't sound right since if I've got it wrong there it might explain what's wrong. QuoteHow do you read data? Do you get CSV file as explained at http:// www.jyetech.com/Products/LcdScope/CaptureUploadingAndWaveformFileForma t.pdf or you use USB Scope mode and DataBlock read function as in http://www.jyetech.com/Products/LcdScope/TheDataInterface_068.pdf ? CSV via Xmodem, since other devices I hope to support at some point only have a "dump" rather than a "live-control" facility and much of this is already in a separate dynamically-linked protocol unit. QuoteYou can compare FFT results of your sample data online https://duckduckgo.com/?q=fft+online&ia=web. Thanks. I'll check. QuoteBtw, there is an unofficial sigrok driver for your device: https://sourceforge.net/p/sigrok/mailman/message/35238342/ https://github.com/mwm/libsigrok/tree/dso112a/src/hardware/jyetech- dso112a Yes, but some of the other devices I hope to support aren't so I thought it worth using this as (in part) a testbed for dynamic linkage stuff I've had running in the past in a different context. Problems raised by that aren't exactly on the critical path at the moment, so I'll raise them some other time :-) MarkMLl ---------------------------------------------------------------------- avra · 2021-01-27 13:05 Quote from: MarkMLl on January 27, 2021, 01:28:49 pmThat was a worked- out figure rather than an observed one, so I'm very interested to know /why/ it doesn't sound right since if I've got it wrong there it might explain what's wrong. I have seen somewhere that they used AVR MEGA64 in some of their DSOs, so I will assume that your model uses AVR too. 2.5MHz sample rate is not achievable with AVR internal ADC, so I will also assume that they use some fast external ADC. That ADC must be tied to some clock, and 25.6 kHz could be achieved from something like 16MHz clock by using some prescaler. That could be partially determined by opening device and examination of board components and ADC datasheet, but I do not have a device to examine so I am just guessing. I have also updated my first post with a question if datablock has fixed size of 512 bytes but you have replied before my edit was finished so you probably missed it. You could also check if you can try to receive CSV with TeraTerm as described in CaptureUploadingAndWaveformFileFormat.pdf because that CSV shows RecLen and SampleRate for the transferred data which could be used for clarification of your doubt. Btw. SampleRate in that pdf is shown to be 50000, so if that is right then your 25600 is questionable. Also RecLen is 256, so fixed buffer of 512 bytes is also questionable. Again, only if that DSO068 screenshot is real and applicable to your DSO model (I have seen some forum message which says that DSO068 protocol  is compatible with your DSO). ---------------------------------------------------------------------- wp · 2021-01-27 14:20 Quote from: MarkMLl on January 27, 2021, 11:59:31 amThe bigger picture is that I'm trying to get the graphical side of this wrapped up as a way of revisiting TAChart. Then I'm going to add graphical output to a program that reads data from a multimeter (Mastech MS2115B), and the code that does that will be adapted in short order to read a pulse oximeter which is obviously of topical interest. Your projects seem to be similar to two of mine: oscilloscope reading data from the audio input connector (using the BASS libarary; https://sourceforge.net/p/wp-laz/code/HEAD/tree/Oscilloscope/), and logging analog values from a digital multimeter based on the devices FS9721 and FS9922 (https://sourceforge.net/p/wp- laz/code/HEAD/tree/datalogger/). I am using TAChart a lot, and maybe you can find some ideas there. ---------------------------------------------------------------------- MarkMLl · 2021-01-27 14:42 Quote from: avra on January 27, 2021, 02:05:16 pmQuote from: MarkMLl on January 27, 2021, 01:28:49 pmThat was a worked-out figure rather than an observed one, so I'm very interested to know /why/ it doesn't sound right since if I've got it wrong there it might explain what's wrong. I have seen somewhere that they used AVR MEGA64 in some of their DSOs, so I will assume that your model uses AVR too. 2.5MHz sample rate is not achievable with AVR internal ADC, so I will also assume that they use some fast external ADC. That ADC must be tied to some clock, and 25.6 kHz could be achieved from something like 16MHz clock by using some prescaler. That could be partially determined by opening device and examination of board components and ADC datasheet, but I do not have a device to examine so I am just guessing. Photo at http://www.hezkyden.cz/i/osciloskop-dso112a-konstrukce.jpg plus I've got mine open. 2x Atmels (I've previously remarked on this since the loader is written using Delphi), and various other stuff under a non-removable screening PCB. There's a 20MHz xtal in there but I suspect that that's on an external circuit supplying both chips... HP bench 'scope is happy enough with that being a 20MHz signal. Under the circumstances I'm not bothered with it being out by a few percent, just trying to get it better than a factor of 2. [/quote] QuoteI have also updated my first post with a question if datablock has fixed size of 512 bytes but you have replied before my edit was finished so you probably missed it. You could also check if you can try to receive CSV with TeraTerm as described in CaptureUploadingAndWaveformFileFormat.pdf because that CSV shows RecLen and SampleRate for the transferred data which could be used for clarification of your doubt. Who is this Teraterm of which you speak? :-) OK, I know perfectly well what it is but I'm running Linux as stated in the first message. Download debug messages confirm 512 readings (not bytes) being transferred, but I've found a slightly dodgy assumption relating to the time-per-reading that I need to check. QuoteBtw. SampleRate in that pdf is shown to be 50000, so if that is right then your 25600 is questionable. Also RecLen is 256, so fixed buffer of 512 bytes is also questionable. Again, only if that DSO068 screenshot is real and applicable to your DSO model (I have seen some forum message which says that DSO068 protocol  is compatible with your DSO). But it's XModem and transferring multiple records... again I can see the entire session in debug messages: ... > 06 < 01 1A E5 < 30 38 38 0A 30 30 30 38 38 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 30 38 < D7 > 06 < 01 1B E4 < 37 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 31 37 30 0A 30 30 30 38 37 0A 30 30 30 38 38 0A 30 30 30 38 37 0A 30 30 30 38 38 0A < AF > 06 < 04 00 00 > 06 Note also that I think that that edit you put into your original message applies to live data, not for a dump (which needs action both on the PC and on the 'scope, and is a one-shot operation). So leaving aside for the moment that my understanding of the sample rate might be a few percent adrift, do you see any howlers in my original logic? If it looks more or less OK I'll fix my frequency domain bin size to conform to it, and will then review my assumptions about time domain scaling... I suspect that I'm doing something like neglecting the size of the display window when working out the samples-per-division figure. MarkMLl ---------------------------------------------------------------------- avra · 2021-01-27 14:55 16MHz can be divided with 25.6 kHz, but 20MHz can not, so if you can not find any other crystals except already mentioned 20MHz, I highly doubt that sampling rate is 25600. As mentioned, CaptureUploadingAndWaveformFileFormat.pdf on page 3 figure 7 (which I have attached here) shows CSV received with TeraTerrm and procedure to get is described. I know it's windows only, but I do not see any other way to determine exact SampleRate and RecLen filelds for received data. Once you get them, you can compare to your assumptions and make corrections. ---------------------------------------------------------------------- MarkMLl · 2021-01-27 15:06 Quote from: wp on January 27, 2021, 03:20:54 pmQuote from: MarkMLl on January 27, 2021, 11:59:31 amThe bigger picture is that I'm trying to get the graphical side of this wrapped up as a way of revisiting TAChart. Then I'm going to add graphical output to a program that reads data from a multimeter (Mastech MS2115B), and the code that does that will be adapted in short order to read a pulse oximeter which is obviously of topical interest. Your projects seem to be similar to two of mine: oscilloscope reading data from the audio input connector (using the BASS libarary; https://sourceforge.net/p/wp-laz/code/HEAD/tree/Oscilloscope/), and logging analog values from a digital multimeter based on the devices FS9721 and FS9922 (https://sourceforge.net/p/wp- laz/code/HEAD/tree/datalogger/). I am using TAChart a lot, and maybe you can find some ideas there. Thanks, I noticed your comment about TAChart a few days ago and intended to follow up your link. In my case I've not so much got end- user applications in mind but am trying to wring every bit of functionality out of what-I've got test equipment... hence my occasional comment about GPIB etc. :-) I can't remember what chip's in the 2115B although I'm obviously aware that there's a limited number used in practice. I chose that meter because it had the useful combination of a current clamp and data download capability (albeit with a fairly slow sample rate); data format is a bit more advanced than what's described in your PDFs with lots of info on scaling etc. I uploaded a text-only program to read it to Github over the weekend (and then had to correct it due to the FileExists() issue which was messing up the code that worked out what port it was on) https://github.com/MarkMLl/Mastech_ms2115b In practical terms very few changes will be needed to handle a cheap pulse oximeter. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2021-01-27 15:09 Quote from: avra on January 27, 2021, 03:55:18 pm16MHz can be divided with 25.6 kHz, but 20MHz can not, so if you can not find any other crystals except already mentioned 20MHz, I highly doubt that sampling rate is 25600. As mentioned, CaptureUploadingAndWaveformFileFormat.pdf on page 3 figure 7 (which I have attached here) shows CSV received with TeraTerrm and procedure to get is described. I know it's windows only, but I do not see any other way to determine exact SampleRate and RecLen filelds for received data. Once you get them, you can compare to your assumptions and make corrections. Remember that that's not the same kind of meter so the fact that the number of samples differs is hardly surprising (it's also got a 1024-sample mode but I've not tried that at all with FFT etc. yet). I've got no problem at all capturing the data, the problem's in my interpretation when FFTing it. MarkMLl ---------------------------------------------------------------------- Fred vS · 2021-01-27 16:52 Quote from: wp on January 27, 2021, 03:20:54 pmYour projects seem to be similar to two of mine: oscilloscope reading data from the audio input connector (using the BASS libarary; https://sourceforge.net/p/wp-laz/code/HEAD/tree/Oscilloscope/) Hello Wp. Nice project! I try to compile it but there is a missing component: uEControls. Where can I find the component? Your project seems to be Windows only. What would it needed to make it work under Unix OS (like Raspbian for Rpi) ? Your project uses Bass closed source library. What methods of Bass are you using, do you think that open source libraries like PortAudio + LibSndFile could do the job? Thanks. Fre;D ---------------------------------------------------------------------- wp · 2021-01-27 17:07 "Oscilloscope" was some kind of fun-project just to show the feasiblity. Don't expect too much of it. UEControls is available via OPM. Windows-only? Really? Where is it not cross-platform? It should be, although I never tested it. Other audio libs will certainly work as well. I selected BASS because I had used it in an old Delphi project and was trying to avoid learning how to install and apply other libs. The "Oscilloscope" application uses some kind of abstract TDataCollector class to access the sound card. For BASS a special TBASSDataCollector is implemented. Likewise any another audio lib could be used provided  that a TDatacollector descendant can be written for it. Again: not severely tested, probably needing some tweaks here and there. ---------------------------------------------------------------------- Fred vS · 2021-01-27 17:34 Quote from: wp on January 27, 2021, 06:07:55 pmUEControls is available via OPM. Ha, ok, thanks. Quote from: wp on January 27, 2021, 06:07:55 pmWindows-only? Really? Where is it not cross-platform? It should be, although I never tested it. So, no "Windows only" things, ok, perfect. Quote from: wp on January 27, 2021, 06:07:55 pmThe "Oscilloscope" application uses some kind of abstract TDataCollector class OK, I will re-jump into Bass code to remember what it is about. Many thanks. PS: No plan to migrate to GitHub (easier for pull-request) ? ---------------------------------------------------------------------- MarkMLl · 2021-01-27 18:11 I think I've found out what's going on. In the time domain, I was displaying a fixed number of horizontal divisions (20) and saying that the the timing (secs per division) was the same as that reported by the 'scope. However, I was also displaying the entire buffer: 512 or 1024 values rather than the 25x12=300 (I think) values in the 'scope's display window: if I put a label on the waveform the time and voltage were both plausible, but the horizontal spacing of the grid was slightly out when working with 512 values and grossly out with 1024. Because of the way I was working with divisions, when I started converting from time divisions to frequency bins things went badly wrong. All of that is obviously fixable, but it means I have to go back and get my time domain grid right before I can really tackle the frequency domain. In order to keep things looking like a traditional scope, I'm seriously considering dropping the rightmost few readings, since otherwise I'll either have to move the timing divisions away from the conventional 1.000 mSec settings or won't have a grid line in the middle of the display as is traditional. MarkMLl ---------------------------------------------------------------------- wp · 2021-01-27 18:33 Quote from: Fred vS on January 27, 2021, 06:34:34 pmNo plan to migrate to GitHub (easier for pull-request) ? Here you go: https://github.com/wp-xyz/Oscilloscope.git ---------------------------------------------------------------------- Fred vS · 2021-01-27 19:01 Quote from: wp on January 27, 2021, 07:33:58 pmQuote from: Fred vS on January 27, 2021, 06:34:34 pmNo plan to migrate to GitHub (easier for pull-request) ? Here you go: https://github.com/wp-xyz/Oscilloscope.git OK, perfect. ---------------------------------------------------------------------- avra · 2021-01-28 07:32 Quote from: MarkMLl on January 27, 2021, 07:11:58 pmI think I've found out what's going on. In the time domain, I was displaying a fixed number of horizontal divisions (20) and saying that the the timing (secs per division) was the same as that reported by the 'scope. However, I was also displaying the entire buffer: 512 or 1024 values rather than the 25x12=300 (I think) values in the 'scope's display windowThat's why I insisted on trying TeraTerm because according to screenshot it allows downloading data along with metadata fields SampleRate and RecLen, which would be enough to understand that downloaded buffer data and oscilloscope display divisions are a miss match. ---------------------------------------------------------------------- MarkMLl · 2021-01-28 08:37 Quote from: avra on January 28, 2021, 08:32:29 amThat's why I insisted on trying TeraTerm because according to screenshot it allows downloading data along with metadata fields SampleRate and RecLen, which would be enough to understand that downloaded buffer data and oscilloscope display divisions are a miss match. All of which is already in debugging output. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2021-01-29 10:04 Wrapping this up: worst of all, I had a totally and unforgivably silly mistake in the RMS function that merged the FFT's real and imaginary output, which is why I was getting the same strange output from both a local FFT implementation and the (better, but not universally-bundled) FFTW. Fixing it so that the time domain display has an integer-number of (25-sample) divisions but the FFT always sees an integer power of 2 samples, fixing the RMS problem, and with few other changes of significance, I can get things like the attached which is the output of a pretty-but-noisy USB PSU. For a cheap and pocketable scope I feel that's a bit better than adequate. Frequency (measuring its own calibration signal) is a bit suspect, but I can fairly easily check that against a pulse-per-second GPS signal. Anybody who wants a copy is welcome, but I'm not sticking it onto Github yet. The issue is that the project can use dynamic linkage between the frontend (generic) and backend (device-specific) parts, and while the frontend could call the backend I was having problems with calls from the backend to frontend. The worrying thing is that it worked properly with 32-bit code (on a 64-bit Linux system with multiarch, FPC 3.0.4) but not with 64-bit code, and when I last looked at this I got bemired a long way into the runtimes and standard libraries. MarkMLl