Read OpenGLContent mouse position and convert back to coords. lazarus / 70-OpenGL ---------------------------------------------------------------------- ranny · 2024-05-16 14:41 Hello, I have a couple of programs I use for graphing 3D polynomials where the data and curve shape is displayed in a OpenGLContent object, I got a lot of help with this from the forum in the past.  I am now working on a 3D beam analysis program for which I have all the maths and can draw the model in 3D in my OpenGLContent and it is beginning to look good.  I am having some scaling problems but I think I will get there. My difficulty now is finding a way of clicking the mouse near a drawn node point on the OpenGLContent object and getting back world coordinates I can then use in other calculations, for example, being able to add another node offset to the one I have selected.  I am having no luck with this at all yet I am sure there is a neat method out there.  I am struggling to understand how, I think, that anything I draw on the OpenGLContent object always has it's centre 0,0,0 in the middle of the object but the mouse position is relative to the top and left of the object.  That's my first problem! Does anybody have an idea where I can look for a solution? Thanks in advance. ---------------------------------------------------------------------- Dzandaa · 2024-05-16 16:51 Hi, I have the same problem I didn't find any clue in the OpenGl manual. Most of the solutions use ray casting: https://antongerdelan.net/opengl/raycasting.html Also look a the feedback mode glRenderMode(GL_FEEDBACK), not sure it is related. B-> ---------------------------------------------------------------------- ranny · 2024-05-16 20:48 Quote from: Dzandaa on May 16, 2024, 06:51:24 pmHi, I have the same problem I didn't find any clue in the OpenGl manual. Most of the solutions use ray casting: https://antongerdelan.net/opengl/raycasting.html Also look a the feedback mode glRenderMode(GL_FEEDBACK), not sure it is related. B-> Hi Dzandaa, So I am plotting lines to represent a beam structure which displays the beams and the numbers at the beam ends and middle of each beam.  The 3D coords from the file of data get scaled and drawn in the OpenGLContent object.  I was hoping I could "reverse" the scaling routine from a cursor position and get at least an X Y pair of coords in world terms that I can check against all of the nodes and find the nearest node to the cursor.  However, as it appears to me, how OpenGL plots lines and polygons etc. is a bit hard to work backwards from.  I also thought that I could store the OpenGLContent position in an array for interrogating later but not sure about how I will get it to work. I would have thought there was some sort of routine that can do this, there must be OpenGL gaming situations where you place the cursor at a location and have code that responds to it..... ---------------------------------------------------------------------- H₂SO₄ · 2024-05-17 05:17 Are you using "modern" OpenGL (shaders, vertex buffer objects/arrays, framebuffers,  glDrawElements)? Or are you using the old fixed- function pipeline (glBegin, glEnd, glVertex3f, glCallList  etc.)? I'm asking because these are very different approaches, with the former being insanely more flexible compared to the latter. In my (modern) OpenGL project for example I'm rendering the scene again with a shader that outputs (among other things) the position of each fragment (interpreted as RGB it may look something like the attached image). Then I do  glReadPixels at the mouse position to get the position. The same thing should be possible by sampling the Z-buffer instead (which is pretty much always present), although that isn't ideal (nonlinear, loss of precision for objects far away from the camera, needs more camera information). Unfortunately this technique isn't implementable in the fixed-function pipeline (glReadPixels  may still be useful however, for example by enumerating objects with specific color values to at least check what is under the mouse pointer). In that case I'm afraid you'll have to use CPU-based ray casting. In any case, the fixed-function pipeline makes anything slightly advanced immediately much, much harder if not outright impossible to accomplish (while also being deprecated since 2008 + shaders have been available for 20 years). ---------------------------------------------------------------------- KodeZwerg · 2024-05-17 06:58 Isnt that task not just a normalization? Like ClientToScreen/ScreenToClient methods. ---------------------------------------------------------------------- Dzandaa · 2024-05-17 09:24 Hi, This could be interesting: https://stackoverflow.com/questions/29912559/opengl- get-3d-coordinates-of-nearest-world-3d-point-to-the-current-mouse- locatio B-> ---------------------------------------------------------------------- Seenkao · 2024-05-17 09:43 Привет! Решения были давно найдены, это: - glReadPixels - glRenderMode - прямые вычисления (не используя OpenGL, для 3D мало подходит, но при желании можно использовать). Все три способа подойдут для любого созданного контекста. Видео с примерами. Извиняюсь, но на русском, думаю если это проблема, то нужную информацию вы сможете найти на других языках программирования. ---------------- Google translate: Hello! Solutions were found a long time ago, these are: - glReadPixels - glRenderMode - direct calculations (not using OpenGL, not very suitable for 3D, but can be used if desired). All three methods are suitable for any created context. Video with examples. I apologize, but the video is in Russian, I think if this is a problem, then you can find the necessary information in other programming languages. ---------------------------------------------------------------------- ranny · 2024-05-21 06:51 Hello again, Thanks everybody for the comments, sorry for the late reply - it was a nice sunny weekend and the garden needed attention! I am surprised that there appears to be little about this issue of mine that has an obvious solution.  I am a novice coder and not very good at graphics (I prefer the maths part) but I would have thought this was a common requirement.  Apologies if that seems rude but this is a make or break for my program.  This from Dzandaa seems interesting right enough, will explore (thanks Dzandaa):- Quote from: Dzandaa on May 17, 2024, 11:24:39 amHi, This could be interesting: https://stackoverflow.com/questions/29912559/opengl- get-3d-coordinates-of-nearest-world-3d-point-to-the-current-mouse- locatio B-> A little bit more about the program I am working on.  Many years ago (more than 20 I am horrified to report) I got some Fortran code for a 2D beam analysis program and converted it to 3D in Delphi.  I wrote my own "3D graphics" routines that showed the image (which is just lines and text mostly) which could be rotated and scaled to some degree but it was very crude. To build a 3D model for analysis one specifies nodes and the beam elements join two nodes.  The beam elements have mechanical properties.  Loads can be applied to nodes and when the analysis is run there are a load of matrix multiplications and solving which then provides detailed mechanical info for each node and element. To build a model it is much easier to start with some nodes placed at their coordinates then one can select a node by using the mouse and then add new nodes by an offset dimension.  Beam elements can be added by selecting two nodes and entering (or selecting already established) properties. So it is only a worthwhile coding exercise if I can get to read capturing a screen position, getting it back to world coordinates to find the nearest node or beam, then carrying on from there. Here is a pic of my old program with my crude graphics. Thanks all. ---------------------------------------------------------------------- H₂SO₄ · 2024-05-21 08:50 If you're only drawing thin lines between a few points, then OpenGL is definitely not the right tool for calculating the closest point to some reference. OpenGL isn't some all-encompassing fully-fledged 3D library; it's for coloring pixels using the GPU - massively parallel tasks involving lots of data, we're talking hundreds of thousands to millions of pixels. In this context, a few hundred nodes is nothing in comparison. Quote from: ranny on May 21, 2024, 08:51:05 am[...] I am a novice coder and not very good at graphics (I prefer the maths part) [...] Since you're not afraid of maths and matrices and need to locate only a few points, I suggest doing the calculations on the CPU like this: Transform the clicked point into normalized device coordinates (NDC; axis range is  [-1, 1], i.e. in NDC the viewport's left edge has  x = -1  and top edge has  y = 1)Find the camera's combined projection matrix; OpenGL uses it (*) to transform the points you feed it into NDC space.Find each node's NDC location by multiplying its position (with W = 1) by the combined projection matrix and then dividing the result's components by its  W  component).Find the closest node by comparing X-Y plane distances in NDC space. * Typically 3 matrices lie between a source and a destination point in OpenGL: The model matrix (per-object offset, scale, rotation; transforms from model space to world space)The view matrix (camera position and orientation; transforms from world space to camera space)The projection matrix (transforms from world space to clip space)Going from clip space to NDC space involves doing a perspective divide, which is done by dividing all components by the  W  component (OpenGL uses homogenous coordinates, so for input points it is assumed that  W = 1). To find the combined matrix you'd need to multiply these matrices together (in the right order of course). I'm not familiar with fixed- function OpenGL, but I think  glGet with  GL_PROJECTION_MATRIX and  GL_MODELVIEW_MATRIX  should give you the matrices you need. ---------------------------------------------------------------------- ranny · 2024-05-21 09:36 Thanks Khrys. I found this:- Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  type T3D_Point = array [1..3] of Double; function GetOGLPos(X, Y: Integer): T3D_Point;var    viewport:   array [1..4]  of Integer;    modelview:  array [1..16] of Double;    projection: array [1..16] of Double;    winZ: Single;begin    glGetDoublev( GL_MODELVIEW_MATRIX, @modelview );    glGetDoublev( GL_PROJECTION_MATRIX, @projection );    glGetIntegerv( GL_VIEWPORT, @viewport );     // In Delphi A Y Value Of 0 Returns An Unknown Value    // I Discovered This While I Was Testing A Crosshair    if( Y = 0 )then Y := 1;     glReadPixels(   X, -Y, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, @winZ );    gluUnProject(   X, viewport[4]-Y, winZ,            @modelview, @projection, @viewport,          Result[1], Result[2], Result[3]);end; However, I cannot get it to work with the following errors:- Incompatible types: got "Double" expected "PGLdouble" which I cannot resolve. And sometimes Foward declaration not solved "GetOGLPos(Longint,Longint):Array[1..3]of PGLDouble I feel this is what I am looking for, but cannot get it to work, don't know what PGLDouble is either. Actually at one point I did get it to work but it just stated Access violation and stopped. I understand I may be using OpenGL for something so small but I have the rest of the code working well to display what I want with the ability to rotate the model and zoom in etc. if I can fix this I can go forward... ---------------------------------------------------------------------- wp · 2024-05-21 10:13 Why is "viewport" an array of Integer when you put it in a call to glGetDoublev? I'd rather declare it as an array of double or GLDouble. PGLDouble is a pointer to GLDouble: CTRL-click on it, and the IDE opens the unit in which it is declared: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PGLdouble  = ^GLdouble; At which line is that error reported? I assume in the glUnproject line because the result is returned as a PGLdouble and you assign it to the fields of a T3D_Point which probably is a record of three doubles (hopefully, rather than single!) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i // unit glu;...  gluUnProject : function(winX:GLdouble; winY:GLdouble; winZ:GLdouble; model:PGLdouble; proj:PGLdouble;     view:PGLint; objX:PGLdouble; objY:PGLdouble; objZ:PGLdouble):GLint;extdecl; This might work (but not tested): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  objx, objY, objZ: GLDouble;...    gluUnProject(   X, viewport[4]-Y, winZ,          @modelview, @projection, @viewport,            @objx, @objY, @objZ);    Result[1] := objX;    Result[2] := objY;    Result[3] := objZ;   // or maybe simpler, if the elements of Result are doubles    gluUnProject(   X, viewport[4]-Y, winZ,            @modelview, @projection, @viewport,            @Result[1], @Result[2], @Result[3]); ---------------------------------------------------------------------- ranny · 2024-05-21 10:49 Thanks WP, always useful, but alas I still get this when compiling... Error:Forward declaration not solved "GetOGLPos(Longint;Longint):Array[1..3]Of Double I have declared T3D_Point after uses in the main form, don't know if that makes a difference. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  TPaletteItem = record    Value: GLFloat;    Color: TColor;  end;   TPalette = array of TPaletteItem;   T3D_Point = array[1..3]of Double;    { TForm1 }   TForm1 = class(TForm)    btnReset: TButton; ---------------------------------------------------------------------- H₂SO₄ · 2024-05-21 11:33 I think I forgot to clarify why only drawing thin lines makes OpenGL unsuitable for the task of finding the closest point... It's perfectly fine and reasonable to use OpenGL for drawing lines, that's (part of) what it was designed for. What doesn't really work in that case is relying on sampled buffer values (glReadPixels); that's because the buffers are only updated where OpenGL actually draws something. Lines are thin and only span a few pixels, and so trying to hit one with the mouse pointer is difficult and frustrating. Look at the attached example of a tilted triangle: on the left it's drawn as a solid face and has a well-defined depth value at each pixel inside it, while on the right it's drawn in wireframe mode and is mostly undefined (technically  glClear's argument). So to reiterate, I'm afraid you'll have to do the calculation yourself, without using  glReadPixels. ---------------------------------------------------------------------- ranny · 2024-05-21 12:14 Thanks Khrys  for your comments. If you look further back in this thread I provided a screenshot from my old program of what I am working with.  What I am trying to read from the screen is nearby nodes and elements, not really in any great detail, just approximately where they are.  So if I can get a screen position that I can convert back to the 3D world of the model, all I have to do is step through the node (or element) coordinates and find the closest one to that 3D world position.  Once selected I can carry out the rest of my code to add new nodes, change position etc.  So I don't need to try and hit any item too accurately.  This is what I did in my early program. I enjoy doing this sort of stuff, not very good at it but it is rewarding when it works out..... Cheers ---------------------------------------------------------------------- H₂SO₄ · 2024-05-21 12:41 OpenGL won't be of any help here unless you click directly on the drawn geometry (which, as I said, isn't something to rely on when doing wireframe). I'd still recommend manually projecting everything to 2D and only then looking for the closest element (in 2D). The user only sees the 2D projection, the depth doesn't matter and in is ill- defined in many cases anyway. What if multiple nodes are collinear from your POV, so that they are exactly on top of each other in the 2D projection? You're only trying to find an approximation of the nearest relevant thing's position, you don't need a rigorous 3D inverse for every pixel on the screen. ---------------------------------------------------------------------- wp · 2024-05-21 14:58 I have the impression that you want to click on the vertices of the OGL model, "nodes" in your words, right? In this case, since there are only a few nodes you could iterate over the nodes, calculate the screen point and compare with the click position (and allow for some tolerance). Calculation of the screen point can be done by the inverse of your gluUnproject: gluProject Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  TOpenGLPoint = record    X, Y, Z: GLDouble;  end; function CalcOpenGLScreenPoint(P: TOpenGLPoint): TPoint;var  viewport:   array[0..3]  of Integer;  modelview:  array[0..15] of GLDouble;  projection: array[0..15] of GLDouble;  X,Y,Z: Double;begin  glGetDoublev(GL_MODELVIEW_MATRIX, @modelview );  glGetDoublev(GL_PROJECTION_MATRIX, @projection );  glGetIntegerv(GL_VIEWPORT, @viewport);   gluProject(P.X, P.Y, P.Z, @modelView, @projection, @viewport, @X, @Y, @Z);  Result.X := round(X);  Result.Y := viewport[3] - round(Y);end;          function TForm1.FindClickedNodeIndex(X, Y: Integer): Integer;// Assuming that your model consists of an array of vertices (TOpenGLPoint)const  Tolerance = 5;var  i: Integer;  P: TPoint;begin  for i := 0 to Length(Vertices)-1 do  begin    P := CalcOpenGLScreenPoint(Vertices[i]);    if (abs(P.X - X) < Tolerance)) and (abs(P.Y - Y) < Tolerance) then    begin      Result := i;      exit;    end;  end;  Result := -1;end; Attaching a small OpenGL project which demonstrates this. The nodes in the demo are at the corners/endpoints of the table-like object. Ctrl- click on them and the coordinates of the clicked vertex (in model coordinates) will be displayed in the blue label at the bottom. P.S. Sorry for mentioning the integer issue in my previous post, I was just confused by the different order of declarations and procedure calls. ---------------------------------------------------------------------- ranny · 2024-06-26 14:31 Hi, Sorry for so long since I started this post.  I have had a lot of difficulties with other parts of my code, but I did find a fix for this - not glamorous but it works perfectly well. I have nodes, elements, loads, restrictions and fixings in my model that I want to be able to select.  if I want to select a node then the nearest node to the cursor is what I want.  For loads etc. it has to be the nearest load that is selected but loads can only exist at nodes and there may be 100 nodes and only 2 or 3 loads anywhere in the model. Considering selection of nodes, currently when the node positions are plotted the node number is plotted next to it.  The routine to write the node number is beyond my total understanding but it places it based on the actual viewport location based on the top left of the form (opengl object in the form actually).  So every time a node number is plotted, I store that X,Y pair in an array.  When I want to select a node, on MouseDown the screen position X,Y is compared to the stored X,Y's from placement of the node numbers and works out which is closest.  Works perfectly. For loads etc. the X,Y is stored where the load number would be in a separate array and that is used when I want to select only loads.  And so on for other features like restraints. I know it is not very clever but it does work very well and it has allowed me to progress with other aspects of the program. Thanks for the support. ---------------------------------------------------------------------- ranny · 2024-06-27 14:37 WP - I owe you an apology.  When I went to add my latest post on this subject, I did not see your post as it had moved to a new page and I just assumed that the last comments were those on page 1. Now that i have seen your suggestion, it is a similar approach to what I took, although your version is much tidier and surely the best way to do it, although I worked out my version not knowing your post existed - that could have saved me some time! You are working with vertices as a record but the original code for the maths (20 years old actually) were a bunch of individual arrays that I am working with, but it works fine.  The only comment I have on your code is that in your loop to find the closest "node" you jump if the check for less than tolerance is met whereas there could be a closer node.  Also you could be selecting from a greater distance than tolerance could ever be fulfilled, as your demo shows.  So I opted for checking the distance between mouse position and all nodes and selection is the shortest distance from amongst all. Thanks for your support, your comments and code are very helpful and it helps me develop better solutions. Apologies again - Ranny