Is code completion playing dice? lazarus / 9-Editor ---------------------------------------------------------------------- WooBean · 2023-03-03 14:20 When I use in code editor a dot to see available record fields/functions I see unstable behavior of code completion mechanism. Is it by design? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i //Lazarus 2.2.4 (rev lazarus_2_2_4) FPC 3.2.2 x86_64-win64-win32/win64program project1;{$mode objfpc}{$H+}{$modeswitch advancedrecords on}//{$mode delphi} uses sysutils;  type  myRec = record    a:byte;    x,y:double;    b:word;    function foo:string;    function afoo:string;    case byte of    0:(cc:byte);    1:(dd:int64);  end;   function myRec.foo:string;  begin    result:=self.dd.ToString;    //after writing a dot after "self" I can see an order in code completion "offer":    //dd    //cc    //y    //x    //a    //afoo    //b    //foo     //it was before writing main program code, after second attempt to enter a dot here set an order is as below    //foo  //dd    //cc    //y    //x    //a    //afoo    //b     //after reloading Lazarus  attempt to enter a dot here makes an order as below     //a    //afoo    //b    //cc    //dd    //foo    //x    //y     end;    function myRec.afoo:string;  begin    result:=self.cc.ToString;  end;  var rec:myRec;begin   rec.a:=1;  //after writing a dot after "rec" I can see the same order in code completion "offer"  //foo  //dd  //cc  //y  //x  //a  //afoo  //b   //after reloading Lazarus order is  //a  //afoo  //b  //cc  //dd  //foo  //x  //y   rec.b:=2;  rec.x:=pi;  rec.y:=2*pi;  rec.cc:=5;  writeln(rec.foo);   readln;end. I would like to control a way how fields of a record are presented in code completion (at least to choose between order by name or by declaration sequence as it was in Delphi IDE - by popup menu). Is it possible? ---------------------------------------------------------------------- Martin_fr · 2023-03-03 14:41 Tools > Options: Codetools > Identifier Completion There are 2 options about "sorting" - recently used at top - sort for scope Try adjusting them. You may not like the "recently used" to be sorted to the top. Scope likely means: locals first, then class, then unit, .... As for alphabetic/declaration => no idea. ---------------------------------------------------------------------- WooBean · 2023-03-03 15:39 Quote from: Martin_fr on March 03, 2023, 03:41:41 pmTools > Options: Codetools > Identifier Completion There are 2 options about "sorting" - recently used at top - sort for scope Try adjusting them. You may not like the "recently used" to be sorted to the top. Scope likely means: locals first, then class, then unit, .... As for alphabetic/declaration => no idea. Tried all the combinations - now all is sorted by name in each setting - ignoring my intentions, what a pity. Archaic version of Delphi managed to do it - see a picture. ---------------------------------------------------------------------- Blaazen · 2023-03-03 18:14 If you set - [X] Show recent identifier at top - it is forgotten when you restart Lazarus. Lazarus probably cannot order items by declaration, only alphabetically + by scope. ---------------------------------------------------------------------- WooBean · 2023-03-04 08:43 Regarding "Scope likely means: locals first, then class, then unit, ...." I think that code completion during writing code in IDE with variable identifier of a record (which always starts by pressing "." ie. "recName.") should work only with two options - ordered by name or by declaration/definition. Order by scope in that case should mean definition order. Such approach I can see in Delphi and it is good. What is the aim to propose idenitifiers for fields (and functions/procedures) of record in that situation in an order based on previous actions of a programmer during writing code? Pure playing dice? ---------------------------------------------------------------------- Martin_fr · 2023-03-04 08:53 "scope ..." was an assumption on my side. I did not check if it is so. But that aside. Currently there is simply no option how to sort identifiers that are in the same group by "sort for scope". Or at least I haven't seen it. IMHO both alphabetically and by-declaration are valid ideas. I have no idea how much work it would be though. You can add a feature request on the issue tracker, but that might only be for the long term.... You could look at code completion (IIRC ide/sourceeditor.pp is a good start point for the search). The list is retrieved from codetools (package). I don't know if the sorting can be skipped, and if in that case maybe they come in the desired order.... If not, it will be a lot of work. ---------------------------------------------------------------------- WooBean · 2023-06-12 09:22 Next steps ... I have managed (?) to understand how codetools work in aspect of identifier completion. The list of identifiers is generated from source code buffer in context of cursor position. Initially this list is ordered in reversed order of apperance (ie. a record defined as record a,b,c:integer; end is diveded to fields ordered c,b,a). But this list is then ordered/sorted according by two (or three - including parameters) settings - sort by history and sort by scope  (if they are set to TRUE). There is yet one sorting always active - by alphabetic order - no change setting possible. I would like to get list of idendifiers prepared for displaying for a hint / popup window ordered as they are defined in source code. Just to get it possible I prepared a first step (changes in code tools). Can someone judge whether it is OK? Note: Lazarus 2.2.4 ---------------------------------------------------------------------- Martin_fr · 2023-06-12 10:34 I have taken a look at your file. It describes how to add a new checkbox to the options, and how to store that checkbox. I assume Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i SortForAlphaOrder = trueis for the current order? And switching it off, would be the new order? That is ok, but personally I would suggest to make "SortByDeclaration" as now option. That way to change away from the current behaviour people will turn an option on (rather than off). In any case, one important bit. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i XMLConfig.S etDeleteValue('CodeToolsOptions/IdentifierCompletion/SortForAlphaOrder ',      FIdentComplSortForAlphaOrder, TRUE);The "TRUE" means that is the default. If the value is true, then nothing will be written into the xml file. And the "XMLConfig.GetValue(...., TRUE)" must then have the same default, so if the XML has no value, "true" will be read. The "FIdentComplSortForAlphaOrder" must also  be initialized (that should happen for either true or false anyway). There should be a method where all the options are initialized. ===> The important bit is: If the setting is at its default => the xml should not contain anything. For all else, your suggestions look ok. Of course you still need the code that do the actual work. If you have further suggestions, please provide them as patch. Even if you just put comments (or dummy code) in there. If I apply a patch, I can open the files, and have tools to navigate to the lines that changed. ---------------------------------------------------------------------- WooBean · 2023-06-12 11:31 @Martin Quote.. That is ok, but personally I would suggest to make "SortByDeclaration" as now option. That way to change away from the current behaviour people will turn an option on (rather than off). It is a little complicated as just to have effect of displaying identifiers "by declaration" we shoud unmark both "Sort by history" and "Sort by scope" and unmark (now not existing option "Sort alphabetically"). When  "Sort alphabetically" option will be set to FALSE any setting "Sort by history" or "Sort by scope" to TRUE will/may produce something not expected in list of identifiers. Too much freedom hurts (joke). Maybe some hint in proper area (Identifier Cempletion/Sorting) would be a solution. Thanks for your time. ---------------------------------------------------------------------- Martin_fr · 2023-06-12 12:19 Quote from: WooBean on June 12, 2023, 01:31:55 pm@Martin Quote.. That is ok, but personally I would suggest to make "SortByDeclaration" as now option. That way to change away from the current behaviour people will turn an option on (rather than off). It is a little complicated as just to have effect of displaying identifiers "by declaration" we shoud unmark both "Sort by history" and "Sort by scope" and unmark (now not existing option "Sort alphabetically"). When  "Sort alphabetically" option will be set to FALSE any setting "Sort by history" or "Sort by scope" to TRUE will/may produce something not expected in list of identifiers. "Sort by Scope" can always be mixed with any other sort order (assuming that in (at least some) scope(s) there is more than one identifier. "Sort by history", depends. There always can be identifiers that are not yet in history. Though, of course eventually the remaining sort order will disappear.... Unless an option makes the history limited to N entries. (Usually the last ~3 entries are common to be used again soon). "Sort by declaration" of course includes some scope anyway. Well, it has too. Without scope, the first possible identifier (even in a different unit / e.g. in the base class) would be the first declared identifier... not really desirable. So when enabling ""Sort by declaration" then the "Sort by scope" checkbox has to be disabled. (it can't be changed by the user.) Or alternatively (maybe better), the current "Sort by Scope" becomes a drop-down (or radio-group): "Alphabetically", "Scope/Alphabetically", "Scope/Declaration". ---------------------------------------------------------------------- Martin_fr · 2023-06-12 12:21 Note that making it a drop down (or radio), might change how the current settings are stored. That is the boolean would be removed, and become part of an enum. => That can be done. However, before going to deep into that, I would suggest getting the actual functionality. ---------------------------------------------------------------------- WooBean · 2023-06-13 16:52 I am pleased to announce ... (sorry for too high tone) I have reached the goal to display identifier completion (part of CodeTools) using items declaration order. How to do it is presented in attached files. Technically I tried to follow coding style I met in needed units and make as less addings as possible. I have no experience how to share/publish a solution for changing Lazarus. If this will be found useful I can post modified units files of CodeTools package (Lazarus 2.2.4). WooBean ---------------------------------------------------------------------- Martin_fr · 2023-06-13 19:30 First of all, I will be away for a few days. I may be on the forum or not, but I likely wont deal with any code ... Ideally a patch, or better yet a git merge request. Which version of Lazarus do you have? Any new code should really be based on the git main branch (aka "trunk") => you can get that with FpcUpDeluxe. If you do FpcUpDeluxe then you also have a git dir, in wich you can edit the files, and "git diff" will give you a patch, that you can post. (Other tools like WinMerge, can also produce patches: you do need the unmodified file too, so the files can be compared) ---------------------------------------------------------------------- WooBean · 2023-06-19 17:27 Issue created at GitLab and should be considered SOLVED (thanks to Martin). Lazarus users may see final effect when trunk becomes stable, I hope. See https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40332 Note: removed ---------------------------------------------------------------------- Martin_fr · 2023-06-20 16:37 From the issue report. Quote'To reverse the order. This should be done in TIdentifierList.CompareIdentListItems' - I think there is too late - items order is partially predefined before CompareIdentListItems is used for sorting. (BTW - as it is proceeded during Application.OnIdle event I have difficulties to follow control flow in debugged Lazarus IDE). I expect some clues from Code Explorer Tab - it can display structures much better than code/identifier completion is able now. Yet remark, if source code contains errors even Code Explorer becomes stupid. Afaik Code explorer also uses codetools. "OnIdle" => That would be the pascal parsing? Or what? The actual collecting of the idents happens when you press ctrl-space. Well ok, if you type a dot, and wait => then it is in a TTimer. (not Idle but similar) If you want it being called without having to interact with the IDE => write a testcase. I am not 100% sure what final order you want. But you do have the position in the source. So that might help. "Item.Node.StartPos" A little help with debugging, if you are in CompareIdentListItems and want to inspect an item. Or anywhere were you have a NODE and a TOOL. In Lazarus 2.3: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i @Item1.Tool.Src[Item1.FNode.StartPos] In older Lazarus you need to typecast this to pchar Only in 2.3 Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Item1.Tool.Src[Item1.FNode.StartPos..Item1.FNode.EndPos] ---------------------------------------------------------------------- Martin_fr · 2023-06-20 16:43 Back to the forum. General: - How to debug - How do codetool work -... Are better discussed here. Items that pertain to the actual feature (like the naming of options, the desired order, ...) can be on the issue. That way, if someone looks at the code in a few years, and wants to know why it was done, they can read up on the issue (which should be linked in the commit). And they don't need to read up on too generic discussions, which they already may know. ---------------------------------------------------------------------- WooBean · 2023-06-20 17:38 As I see my first attempt was not mature enough. I think that ways how identifiers are gathered and initially ordered by identifier completion and Code Explorer Tab/Window differ - completion travels source code buffer from current cursor position back to tree root but Code Explorer does it from the root through all the source code tree. Now only Code Explorer has / can have full control how items are declared by its order of definition. If I am right, such 'travelling' for items in records or classes should be modified for identifier completion. What I observed for identifier completion - fields of records / classes are initially gathered in TIdentifierList as reverted order from declaration in source code. It is not true for class methods or functions and procedures in advanced records. ---------------------------------------------------------------------- Martin_fr · 2023-06-20 17:58 Quote from: WooBean on June 20, 2023, 07:38:37 pmI think that ways how identifiers are gathered and initially ordered by identifier completion and Code Explorer Tab/Window differ - completion travels source code buffer from current cursor position back to tree root but Code Explorer does it from the root through all the source code tree. Now only Code Explorer has / can have full control how items are declared by its order of definition. If I am right, such 'travelling' for items in records or classes should be modified for identifier completion. They may "travel" in different ways => They have different scopes to cope with. "Code Explorer" probably just goes for one unit at a time. "Ident completion" needs content from many sources, but according to pascal scoping rules. No, you can't change the order of "travel" => it can't be guaranteed. It may change in future for whatever new reasons there may be. (I.e. it may change anyway, as bugs with generics get fixed, or maybe it wont change for that..., but then something else). Also I don't know, but it may be that the current collection order is needed to filter duplicates (maybe not, I have not checked). See example at the bottom If that gets proper tested, then maybe it can be changed. But it must also still be able to produce the current scoping order. And allow fixes to it, should that be needed. QuoteWhat I observed for identifier completion - fields of records / classes are initially gathered in TIdentifierList as reverted order from declaration in source code. It is not true for class methods, functions, procedures. But you have the position in the source. So you can order them according to the order they have in the source. (3rd time, I am pointing this out). That leaves the finer questions on how you want them to be ordered Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure foo;var  a: byte;  b: word;beginShould offer "a" on top, or "b" on top? Then the scoping outwards. I assume you want to see items in the dropdown as follows 1)  local vars 1a) self/result (before/after locals?) 2)  function params (result may be in this part?) 3) structure fields and methods 4) inherited fields and methods     inherited inherited fields and methods     ..... 5) global vars from current unit 6) globals from other units ? or do you want globals to be shown at an earlier point? Most of this order is in the "Level" field. So "Level" and "StartPos" should give you something closer to what you want. Then you may have to adjust how level is calculated. Or you have to post process it, before sorting.... Duplicates Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program indentcomplete_order;var  g1: Integer;  g2: Boolean;type  TFoo = class    f1: byte;    f2: single;    function Bar1(p1: string; p2: qword): string;    function Bar2(p3: string; p4: word): string;  end;var  g3: cardinal;  f1:string;function Func: word;begin {} end; function TFoo.Bar1(p1: string; p2: qword): string;begin {} end; function TFoo.Bar2(p3: string; p4: word): string;var  l1: word;  l2: byte;begin    // invoke completion at the start of this line    // Only f1:byte should be in the list. f1: string (global) should notend; beginend. ---------------------------------------------------------------------- Martin_fr · 2023-06-21 12:35 QuoteNow I am back in the lesson "How to use git?". Do you have tortoise git? Right click the lazarus folder in the explorer, there should be entries by TortoiseGit: one of them "show log". - Select the "Working tree changes ..." on the top of the log. - At the bottom, the "view" dropdown, select "show patch" - copy the patch. ------------- Or command line, install Git4Windows. the open "git bash", go to the lazarus folder, enter "git diff" ---------------------------------------------------------------------- WooBean · 2023-06-22 06:27 After some hesitation I am close to propose such settings for identifier completion: 1- by scope (it means items ordered by its sequence of declaration) + former logic about scoping 2- by name (= alphabetic) 3- by name + history (sort alphabetically then used/touched items move first) Old style option configuration  is ready - see a picture. It would be good to invoke popup menu directly from completion hint window and choose/change an option from there. It needs a bit digging (not much, I hope). Note, all refers to Lazarus 2.3.0 ---------------------------------------------------------------------- Martin_fr · 2023-06-22 07:26 I would leave "history" as an "add on" option, such that it can be combined with any other setting. I could see, that in future history could be limited to a number, e.g. 3 most recent, or 5 most recent on top. And after that, any sorting can apply. ---------------------------------------------------------------------- WooBean · 2023-06-24 08:17 It is time to give an answer to the initial question (title) -  is code completion playing dice? Yes, at some level and for now. Things can change in the future - identifier completion mechanism can be clearly tailored and some steps towards implementing it have been done. Respect for Martin! Goodbye picure: