Real time charting lazarus / 55-TAChart ---------------------------------------------------------------------- ahmadian · 2014-04-26 21:35 I have a worker thread that read an integer value  from spi port every 10ms. The hardware is Raspberry Pi and the worker thread works nice. It contains a sleep(10) for timing. When thread read 100 numbers from poet, main thread must plot this values on a LineSeries. But during LineSeries1.EndUpdate, the worker thread stops and my data is lost. How can I write a thread that not interrupted by GUI? ---------------------------------------------------------------------- engkin · 2014-04-26 22:58 Quote from: ahmadian on April 26, 2014, 11:35:13 pmBut during LineSeries1.EndUpdate, the worker thread stops and my data is lost. How can I write a thread that not interrupted by GUI? Maybe your data should not be part of the worker thread, or may be you should *not* use FreeOnTerminate := True. ---------------------------------------------------------------------- ahmadian · 2014-04-27 17:35 Thanks engkin The FreeOnTerminate didn't solve problem. You can see two simple procedure of code: Code: [Select]procedure TForm1.Button1Click(Sender: TObject); var i: integer; begin   LineSeries1.BeginUpdate;   LineSeries1.Clear;   for i:= 1 to 10000 do LineSeries1.AddXY(i,i);   test_flag:= false;   LineSeries1.EndUpdate;   if test_flag then Form1.Caption:= 'Worker Thread was alive'; end; procedure TMyThread.Execute; var   dt: real;   Present, Last: TdateTime; begin   Last:= Now;   while true do   begin     Repeat       Present:= Now;       dt:= (Present - Last) * 86400; //sec     until dt >= 0.01;     Last:= Present;     test_flag:= true;     sleep(10);   end; end;The 'test_flag' is set to false before EndUpdate. If TMyThread works between EndUpdate execution, 'test_flag' must be true after EndUpdate. But it was not true; ---------------------------------------------------------------------- engkin · 2014-04-27 20:12 Quote from: ahmadian on April 27, 2014, 07:35:17 pmThe 'test_flag' is set to false before EndUpdate. If TMyThread works between EndUpdate execution, 'test_flag' must be true after EndUpdate. But it was not true; Maybe because EndUpdate takes a very small amount of time before execution gets transferred to TMyThread. Which means, if true, that it does *not* block your thread. ---------------------------------------------------------------------- wp · 2014-04-27 22:29 Sorry, maybe my stupidity, but I don't understand your code: - How does the while-loop in the thread terminate? Normally there is a "while not Terminated...". - Why do you need that "sleep"? Can't you just loop until a value is available, use "synchronize" to pass the value to the series? The timing to catch values that come 10 ms apart by using a sleep value of 10 ms seems to be very tight to me. But maybe I don't understand your code at all. - Can you simulate the issue on a little demo which runs on a PC? I don't have a Raspberry Pi, but would like to help debugging. ---------------------------------------------------------------------- ahmadian · 2014-04-28 11:55 Thanks engkin and wp My story is: I make circuit boards for Tensile Tester Machines. Up to now i used Delphi and Cport component. The board send 8byte to PC every 10ms. Serial port of PC has large buffer (over 4096 bytes) and the software has many free  time for charting and etc. TeeChart component of Delphi is very fast too (1ms for each point of FastLine). 6 month ago, I decided to replace Raspberry Pi with PC. Best choose was Lazarus. I change my PCB and Remove MCU of the circuit, because Raspberry Pi has GPIO. I was successful and all works fine. But there is no any buffer on GPIO and my software must read(non stop) every 10ms. I measured charting speed of TAchart on Raspberry Pi. It take 25ms fo revery  point  of LineSeries that is slow and i need below 10ms. But when using BeginUpdate/EndUpdate, it draw 100 point in 50ms, that is fast. I decided to write a thread to buffer every 100 points that take 1 second and draw these 100 points simultaneously with BeginUpdate/EndUpdate. But The tread is stopped on running EndUpdate and some data is lost. I need a thread that works non stop within another procedure. Same as kernel drivers that stop any work for reading hardwares (mouse , keyboard , ...). 'Ex me for my poor English' ---------------------------------------------------------------------- taazz · 2014-04-28 12:23 thats an interesting background but still with out any code to look at it is improbable that someone will come up with the solution to a problem he has not seen or know what it is. Calling endupdate should not have that effect on the thread or any thread for that matter what happens if instead of reading from the com port as you do now you use a file of previously recorded values and send them to the main thread add some more threads one to read from the com one or more to read from a file and report back one on 100 values the other on 1000 etc or any number of values you see that it will create a more random effect. now when endupdate is called are all the threads stopped? only the one that it communicated with the main thread last or only one specific thread? IT would be easier though if you could create a demo application using the file approach I described above to show us the problem and we could have something to look at and grin our teeth at. ---------------------------------------------------------------------- ahmadian · 2014-04-28 21:27 Thanks taazz I write a simple code that you can see on the top of topic. If this code work on Windows PC, I can move it raspberry PI and complete my software. In that code I define a simple flag to check thread status when EndUpdate is working. I am Hardware engineer and my main experiences is Assembly programming. I cant understand threading without interrupts! I doubt thread Preempt method in Lazarus! Any help or example that show a thread can run when EndUpdate is running, will be appropriated :-) Simply, the thread must work like hardware interrupt. Regards ---------------------------------------------------------------------- taazz · 2014-04-28 23:48 OK lets go with the code provided, the main problem I see is that you make assumptions on the synchronization of the thread and the main thread execution time. You say that it should be true after the call to end update but I do not see anywhere code that makes sure that the thread had enough time to reach and execute the code at all, depending on the number of tests you might have different results each time why not introduce a sleep(X) before checking the test_flag where X must be equal to the total time required to execute a single iteration inside the while loop. Still the code you provide is not enough for example how is the test_flag declared is it var or threadvar? inside the form or global to the unit/application, as you can see there are a number of reasons that your test fails and the thread is still executing behind your back so a complete demo that shows the problem with everything visible would be better for us to test. ---------------------------------------------------------------------- CaptBill · 2014-04-28 23:55 Since you are already dealing with GPIO pins you could simply pass the data to a micro controller for buffering and timing. There is no need for 'Preempting', 'blocking', 'locks','priorities' etc. etc. Those are all 'contentious' types of coding where you are fighting the system for powers that aren't normally allowed. Those are all 'work arounds' to a problem that even the little Rasberry Pi OS has: poor 'real-time' skills. Micro controllers don't have those problems. ---------------------------------------------------------------------- ahmadian · 2014-04-29 09:23 Quote from: taazz on April 29, 2014, 01:48:06 amOK lets go with the code provided, the main problem I see is that you make assumptions on the synchronization of the thread and the main thread execution time. You say that it should be true after the call to end update but I do not see anywhere code that makes sure that the thread had enough time to reach and execute the code at all, depending on the number of tests you might have different results each time why not introduce a sleep(X) before checking the test_flag where X must be equal to the total time required to execute a single iteration inside the while loop. Still the code you provide is not enough for example how is the test_flag declared is it var or threadvar? inside the form or global to the unit/application, as you can see there are a number of reasons that your test fails and the thread is still executing behind your back so a complete demo that shows the problem with everything visible would be better for us to test. Thanks taazz The test_flag is a global variable, not a thread var. I hered somewhere that boolean vars don't need to be defines in 'Critical Section'. I can't add a Sleep(x) to main thread, because this don't change EndUpdate problem! I attached the my test code. I am waiting any suggestion. Thanks CaptBill (Great man of TinyCore) QuoteSince you are already dealing with GPIO pins you could simply pass the data to a micro controller for buffering and timing. There is no need for 'Preempting', 'blocking', 'locks','priorities' etc. etc. Those are all 'contentious' types of coding where you are fighting the system for powers that aren't normally allowed. Those are all 'work arounds' to a problem that even the little Rasberry Pi OS has: poor 'real-time' skills. Micro controllers don't have those problems. Yes, Doing this job with MCUs is 'ULTRA SIMPLE'. But I come from there to use a 1GHz CPU!!! Recent years, I made many boards with AVR + WIN PC + Wireless dongle (that i made by NRF24L01+). Everything works OK. I removed AVR, Wirless Link and  connect all ICs directly to GPIO of  Raspberry Pi. This Reduces the board size and eliminate an expensive Windows PC. I expect the 1GHz CPU works better than a 20MHz AVR! Linux has many advantages over Windows but I disappointed, If i can't have a simple interrupt (or fixed time thread) My last solution is to: 1- Add an AVR to the board and buffer 100 samples 2- Add a 1 Sec timer in Lazarus Program 3- In the on timer event handler, simultaneously read all 100 samples from SPI and add them to LineSeries of chart. This is very simple, but was not my first goal Regards ---------------------------------------------------------------------- taazz · 2014-04-29 09:52 well the attached project will show you that I was right, the problem is simple one of synchronization between main thread and your thread run it and see for your self the callback counter I added on the status bar. ---------------------------------------------------------------------- ahmadian · 2014-04-29 12:13 You are great taazz ...  It Works OK and the AsyncCall is strange. ---------------------------------------------------------------------- taazz · 2014-04-29 12:26 QueueAsyncCall is a method of TApplication it has been written with multi threading in mind so it is safe to call it from multiple threads at the same time. What I did was to instract the application that when all the messages (this is a windows behavior where event generates a message that is send to active windows) have finished processing and the application is about to enter its idle state then run the method I provided that is why I used a TForm1 method instead of a TMyThread Method since the calling thread will be the main thread this way I keep the thread free of synchronization code. It is a quick way to check various things and probably to implement a limited information wake up calls eg percentage completed of a file decoding in the background but it is not to be used for more complicated data exchange with out proper data protection. ---------------------------------------------------------------------- ahmadian · 2014-04-29 13:04 Thanks a lot taazz. This is new highway for me and my hardwares. But I have another problem. My data is an array and must be plotted on chart. I changed the code to make an example array. I use the dt time for data for test and evaluation of thread timing. But I don't know how to send this array asynchronously to chart. I attached the new code. Regards. ---------------------------------------------------------------------- taazz · 2014-04-29 13:15 there are 2 main methods of data exchange between threads. 1) copy data to an address not accessed by the thread and send address to the main thread. 2) use some sort of synchronization mechanism critical section, multi read single write or some other mechanism. In your case I would go with 1 ee create an array when it fills create a new array to work with and send the old to the application. I'll attach an application to demonstrate the 1st technique later today or tomorrow don't know for sure yet. ---------------------------------------------------------------------- CaptBill · 2014-04-29 16:20 QuoteYes, Doing this job with MCUs is 'ULTRA SIMPLE'. But I come from there to use a 1GHz CPU!!! Recent years, I made many boards with AVR + WIN PC + Wireless dongle (that i made by NRF24L01+). Everything works OK. I removed AVR, Wirless Link and  connect all ICs directly to GPIO of  Raspberry Pi. This Reduces the board size and eliminate an expensive Windows PC. I expect the 1GHz CPU works better than a 20MHz AVR! Linux has many advantages over Windows but I disappointed, If i can't have a simple interrupt (or fixed time thread) Ah, see? You defined the problem right there. You are disappointed that the Pi is only a little better at doing real-time. Also you are equating available clock cycles with 'good timing skills'. Even a 20mhz clock that runs on time is better than a clock that can't keep time. Hahaha  4ghz or not. Even ARM Linux has this problem. 'PC OS's' have this serious weakness where the cannot maintain a steady beat. How could they be expected too in a shared multi-tasking environment? You got the solution too. You need a buffer so you can pile your array data into to be processed and fed out perfectly timed. This is what the AVR is good at. Very good at. It 'single tasks' well. Just program it as your buffer and process the data, within this more suitable real-time platform. Then you only have one last concern: latency. Perhaps you need a speedier AVR to match the latency requirements you have. "Great man of Tinycore"? Hahaha .Maybe "Tenacious man of Tinycore". Have you attempted an install on the Pi? You know there is a Tinycore ARM port for the Pi, right? ---------------------------------------------------------------------- ahmadian · 2014-04-29 17:35 Quote from: taazz on April 29, 2014, 03:15:25 pmthere are 2 main methods of data exchange between threads. 1) copy data to an address not accessed by the thread and send address to the main thread. 2) use some sort of synchronization mechanism critical section, multi read single write or some other mechanism. In your case I would go with 1 ee create an array when it fills create a new array to work with and send the old to the application. I'll attach an application to demonstrate the 1st technique later today or tomorrow don't know for sure yet. Thanks a lot taaz. Your guide works OK. I made a global array and fill it every 1 sec. I attached result. It work same as buffered Oscilloscope that show thread timing detail. I will mix this code with SPI reading on Raspberry Pi and replay you soon. ---------------------------------------------------------------------- ahmadian · 2014-04-29 18:07 Quote from: CaptBill on April 29, 2014, 06:20:51 pmAh, see? You defined the problem right there. You are disappointed that the Pi is only a little better at doing real-time. Also you are equating available clock cycles with 'good timing skills'. Even a 20mhz clock that runs on time is better than a clock that can't keep time. Hahaha  4ghz or not. Even ARM Linux has this problem. 'PC OS's' have this serious weakness where they cannot maintain a steady beat. How could they be expected too in a shared multi-tasking environment? You got the solution too. You need a buffer so you can pile your array data into to be processed and fed out perfectly timed. This is what the AVR is good at. Very good at. It 'single tasks' well. Just program it as your buffer and process the data, within this more suitable real-time platform. Then you only have one last concern: latency. Perhaps you need a speedier AVR to match the latency requirements you have. Even a 12MHz AVR is fast enough for my purpose, when using Interrupt of SPI in Slave mode. But when I send a package for a costumer I want to support one software, not 2s. I can change one software to newer version and email it to the costumer. Thanks to "taazz" that he solved my problem of threading and i will test it on the Raspberry-Pi. QuoteHave you attempted an install on the Pi? You know there is a Tinycore ARM port for the Pi, right? yes, but Results was not different from Whezzy. You may see below that "bmarkus" disappointed me in PiCore. http://forum.tinycorelinux.net/index.php/topic,16727 I have a dream to have real time Linux on Raspberry Pi ---------------------------------------------------------------------- CaptBill · 2014-04-29 19:29 QuoteHave you attempted an install on the Pi? You know there is a Tinycore ARM port for the Pi, right? yes, but Results was not different from Whezzy. You may see below that "bmarkus" disappointed me in PiCore. http://forum.tinycorelinux.net/index.php/topic,16727 I have a deam to have real time Linux on Raspberry Pi  Hey, I missed your post. Very interesting. BMarkus is correct. Just because it's small doesn't mean it's real-time. As a matter of fact, there is no 'ARM Rt-kernel build' at all, that I am aware of yet.  The real problem you have deployment wise is that you must have complete control of the complete system, if you are attempting real- time. So to prevent a deployment nightmare, you have to provide a custom OS pre-configured for RT. You can't just write an app that is 'Rt-ready', unfortunately. You can't even allow your customer to install any more apps, hahaha. You have to configure the kernel itself to do bona-fide Rt. That's the sad reality. The good news is there seems to be a very nice, elegant solution with the AVR route. Seen it with my own eyes. Total game changer for CNC, which is heavily reliant on Rt. With AVR controlling the actual pulsing previously done with a Linux RT-kernel, Linux is free to just be a front end while the AVR is running circles around the Rt-kernel's performance. Even running 5 axis simultaneously. No need for Rt- kernels anymore. Whoo hoo. ---------------------------------------------------------------------- ahmadian · 2014-05-01 13:09 Hi taazz You are great. It works like a  Jet on Raspberry Pi.  I'm writing this post on Raspberry Pi too. I attached source but you must have Rpi and my hardware for testing. The code have a data logger thread that reads a Loadcell and ShaftEncoder every 10ms from SPI port and copy it to an array[0..99]. When the array was full, It copied to another array and this new array is plotted on the chart. Simultaneous plotting of TAchart component is very fast and 1sec time is so enough for 100 points. The data logger thread works in background even when plotting is done. I use Szollosi's Pigpio code from http://wiki.lazarus.freepascal.org/L azarus_on_Raspberry_Pi#5._PiGpio_Low-level_native_pascal_unit_.28GPIO_ control_instead_of_wiringPi_c_library.29 But I added low level SPI driver to it. This driver works directly with CPU registers (faster than any other driver of linux). I can't  donate Lazarus/FreePascal developers (here is IRAN and no VISA card is found  ) but I can share all of my practical codes on this forum (or wiki). Lazarus/FreePascal is the only RAD available for Raspberry Pi and same boards. It has many potential for consumer/industrial electronics. But without a systematic donate, it propagates slow. ---------------------------------------------------------------------- taazz · 2014-05-01 14:27 I'm glad you made it work. Let me pock a small hole in your code not big enough to make any difference I guess but it is important for me. you say Quote from: ahmadian on May 01, 2014, 03:09:32 pmWhen the array was full, It copied to another array and this new array is plotted on the chart. So let me get this straight for a second, in pseudo code you have something along the lines of Code: [Select]    getmemory(MyVar, 100*4{bytes});     While Not terminated do begin       FillMem(MyVar,0);//init;       Cnt := 0; //init       repeat         MyVar[Cnt] := readValue;         Inc(Cnt)       until Cnt >= 100;       GetMemory(MyOtherVar,100*4);       copy(MyVar, MyOtherVar, 100*4);       SendToChart(MyOtherVar);     end; Given the fact that this runs once a second and it only copy 100 values I don't think this would be of any measurable help but it would be faster if you avoided the copy altogether. eg Code: [Select]    getmemory(MyVar, 100*4{bytes});     While Not terminated do begin       FillMem(MyVar,0);//init;       Cnt := 0; //init       repeat         MyVar[Cnt] := readValue;         Inc(Cnt)       until Cnt >= 100;       SendToChart(MyVar);       getmemory(MyVar, 100*4);     end; Just in case you missed it, no I haven't looked at your code yet, so fill free to ignore me. ---------------------------------------------------------------------- ahmadian · 2014-05-01 15:07 You are ashamed me taazz  I can search Google for small problems and refer to you for large problems. Before using your suggestions (or other improvements), Now I am trying to write a high precision timer to measure small improvements of speed. I am googling to find equivalent to QueryPerformanceCounter for Linux ARM. This will have me an insight to small speedup of code. ---------------------------------------------------------------------- taazz · 2014-05-01 15:38 there is a control called epictimer that if I understand correctly it has good precision on both windows and linux. I have never used it although it is in my list to check. Try googling it. ---------------------------------------------------------------------- ahmadian · 2014-05-07 09:53 Thanks taazz I used rtl function of Linux and It was precise. Istalling Lazarus components on Rraspberry is not simple. But I successfully installed LazReport. Every thing is OK. But I can't decide on selecting a database that  is suitable for saving Series and link to LazReport. In Delphi, I used ClientDataset in nested mode. I don't need query or other advanced options. Only save a LineSerie and its metadata on file. But want to load multiple Series and report those. Lazarus offer many database that I can't select one for my program. Regards. ---------------------------------------------------------------------- taazz · 2014-05-07 18:25 personally I would go with a type file in your case everything is small and self contained as far as I can see, a database no matter how small will add overhead for features that you do not use and all solutions on lazarus I have seen so far are not cached. They load all from file in to memory, nice easy small solutions that do not lend them selfs to big datasets, reporting or anything else of that group of processes. In the case that you do not really care for memory consumption for what ever reason, then I would go with TDBF or TBufDataset as local databases. ---------------------------------------------------------------------- avra · 2014-05-08 09:30 ahmadian, you should know that with linux approach you have an error in your tension break force calculation, so you should check if such error is acceptable (different force will be calculated if wire break is caught after dt+9ms then after dt+12ms). When you read a measurement over SPI, you can turn on and off a single GPIO pin, and then monitor timings on oscilloscope. If error is acceptable you are done. If not you examine if you can improve it (like doing nothing except filling an array for a few seconds and update screen after measurement is done, or drawing directly on canvas, or...). If nothing helps, then you are left with two options: to program RPi without any OS and use ARM interrupts directly (this is really not for a newbie, both for bare bones ARM hw access and for visualization part), or to keep AVR and use it's interrupt for precise 10ms measurements which you can send to RPi even via simple serial (even speeds like 115200, 57600 or 38400 bps will be enough since you have 8 bytes * 100 times / per second which is just 800 bytes). ---------------------------------------------------------------------- ahmadian · 2014-05-10 19:28 Thanks avra I read many posts from you about RPI (thanks) I used AD7190 from Analog Device for reading LoadCell. This IC convert voltage to data at a programmable update rate. In my app 100 sample per second. When a data is ready the REY pin goes Low and we have 10ms time to read data. In my tests on RPI, this 10ms is enough. I think refresh time of scheduler of Raspbian is less than 10ms. Next, in charting of data on Tensile test, we can modify chart ofter break. Tensile testers is used for many materials and 10ms is enough for more of them. I have a Repaint_Chart procedure on my software that filter and modify raw data of initial chart. I workd many years on bare-metal programming with 8051, AVR and ARM. But Now I want to use OS for faster development and many other benefits. I'm not student. I must make machines for living. But here is Iran and team working is very rare. I must work alone: basic design an ordered machine, draw CAD detail of every parts, send to CNC worker, assemble parts, draw schematic, draw PCB, soldering parts, initial test of circuit, initial test of machine, write software, write helps, send to costumer ... You can see bare-metal programming is not suitable for me. I attached my code in last post. It works very well by offering AsynCall from taazz (You may use or offer to others my SPI lib that directly command to BCM2835). I converted my Delphi program of Tensile Testers to Lazarus and it works cool on RPI( overclocked to turbo mode). My new problem is selecting a method for data storing and manipulation: Database or Record. Taazz say, Record is faster than Database for small softwares and my test approve it. Now I'm working on LazReport to complete my program and release version 1 to costumer ( get some money to be alive for writing version 2)  Regards ---------------------------------------------------------------------- Paul Breneman · 2014-05-21 12:22 This is a very interesting thread!  I know a fellow who tried very hard to get the Raspberry Pi working with linux-cnc (which is sort of real time) but he couldn't.  Mainly because the 3D graphics part of the CPU came up in control (I don't remember the specifics).  He switched to the Beaglebone Black and that does work.  So if you are interested in real time Linux you might check this blog out more: http://blog.machinekit.io/ There are similar links on this page: http://www.turbocontrol.com/monitor.htm ---------------------------------------------------------------------- ahmadian · 2014-05-21 15:36 Thanks Paul It is very useful Link.