How to export Project data to UML? lazarus / 64-FPC development ---------------------------------------------------------------------- garlar27 · 2019-01-21 18:09 Hi!! I want to make a plugin to import (reverse engineering) FPC/Lazarus project to StarUML 3 (http://staruml.io/). I've been reading their tutorials (in case someone wish to look at them https://docs.staruml.io/developing-extensions/getting-started). I don't want to write a tokenizer/parser in Java Script when everything is already made in pascal. My plan is to generate a JSON file with the info needed to at least create a Class Diagram (in a first stage) or at least import the classes and types without a diagram. So the question are:    o- Where do I start reading?    o- Someone has tried to do something like that? By the way. I've read the thread Wiki stub: Make your own compiler, interpreter, parser, or expression analyzer and Make your own compiler, interpreter, parser, or expression analyzer and it is quite interesting and I can use that info.- ETA:    I want to build a Package to export project types, classes, etc. to a JSON file (JSON definition will be decided afterwards) ---------------------------------------------------------------------- marcov · 2019-01-21 19:43 Post on the staruml forum? They are probably more knowledgable about how to instrument their parsers than we are?!? ---------------------------------------------------------------------- garlar27 · 2019-01-21 20:06 Quote from: marcov on January 21, 2019, 08:43:57 pmPost on the staruml forum? They are probably more knowledgable about how to instrument their parsers than we are?!? It seems pretty easy to access diagrams and elements on them. But Quote from: garlar27 on January 21, 2019, 07:09:50 pmI don't want to write a tokenizer/parser in Java Script when everything is already made in pascal. I want to build a package for lazarus to export a JSON with info of all classes a project has ---------------------------------------------------------------------- valdir.marcos · 2019-01-21 21:15 Quote from: garlar27 on January 21, 2019, 07:09:50 pmHi!! I want to make a plugin to import (reverse engineering) FPC/Lazarus project to StarUML 3 (http://staruml.io/). I've been reading their tutorials (in case someone wish to look at them https://docs.staruml.io/developing-extensions/getting-started). I don't want to write a tokenizer/parser in Java Script when everything is already made in pascal. My plan is to generate a JSON file with the info needed to at least create a Class Diagram (in a first stage) or at least import the classes and types without a diagram. So the question are:    o- Where do I start reading?    o- Someone has tried to do something like that? By the way. I've read the thread Wiki stub: Make your own compiler, interpreter, parser, or expression analyzer and Make your own compiler, interpreter, parser, or expression analyzer and it is quite interesting and I can use that info.- ETA:    I want to build a Package to export project types, classes, etc. to a JSON file (JSON definition will be decided afterwards)What about studying? C:\lazarus\components\ideintf\objectinspector.pp C:\lazarus\components\ideintf\objectinspector.lfm C:\lazarus\examples\objectinspector\oiexample.lpi ---------------------------------------------------------------------- avra · 2019-01-21 23:38 Quote from: garlar27 on January 21, 2019, 07:09:50 pmI want to make a plugin to import (reverse engineering) FPC/Lazarus project to StarUML 3Are you aware that StarUML used to be an open source Delphi project? There is a fork of last Delphi version here: https://sourceforge.net/projects/whitestaruml/ Quote from: garlar27 on January 21, 2019, 07:09:50 pmI don't want to write a tokenizer/parser in Java Script when everything is already made in pascal.Well, maybe you might want to write in pascal but just output javascript? http://wiki.freepascal.org/pas2js Quote from: garlar27 on January 21, 2019, 07:09:50 pmMy plan is to generate a JSON file with the info needed to at least create a Class Diagram (in a first stage) or at least import the classes and types without a diagramYou might want to take a look at EssModel (https://sourceforge.net/projects/essmodelforlaza/), or use something like Gold (http://wiki.lazarus.freepascal.org/Gold). Gold and other parsers are fine when zooming into 99% of your code (grammars can not handle every possible case). When you need that 1% left then you need to go manual way. ---------------------------------------------------------------------- garlar27 · 2019-01-22 00:01 AFAIK Object inspector can access published methods and properties and I need private protected and public too. also I need the parameter list with name and type, if the method is abstract, virtual, overridden, overloaded. And collect as much data as possible. Lets say I have this code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type// C:\Lazarus\Lazarus.1.8.2-FPC.3.0.4-(32)\fpc\3.0.4\source\packages\fcl- json\src\fpjsonrtti.pp   TJSONFiler = Class(TComponent)  Protected    Procedure Error(Const Msg : String);    Procedure Error(Const FMT : String;  Args : Array of const);  end;   TJSONStreamer = Class(TJSONFiler)  private    FAfterStreamObject: TJSONStreamEvent;  FBeforeStreamObject: TJSONStreamEvent;    FChildProperty: String;    FDateTimeFormat: String;    FOnStreamProperty: TJSONPropertyEvent;    FOptions: TJSONStreamOptions;    function GetChildProperty: String;  function IsChildStored: boolean;    function StreamChildren(AComp: TComponent): TJSONArray;  protected    function StreamClassProperty(Const AObject: TObject): TJSONData; virtual;    Function StreamProperty(Const AObject : TObject; Const PropertyName : String) : TJSONData;    Function StreamProperty(Const AObject : TObject; PropertyInfo : PPropInfo) : TJSONData;    Function FormatDateProp(const DateTime : TDateTime) : TJSONString;  Public    Constructor Create(AOwner : TComponent); override;    Destructor Destroy;override;    //    // Basic functions    //    // Use RTTI to stream object.    // If AObject is of type TStrings or TCollection, special treatment occurs:    // TStrings results in { Strings: [S,S,S] } or { Strings: { "S1" : O1, "S2" : O2 }} depending on Options.    // Collection results in { Items: [I,I,I] }    Function ObjectToJSON(Const AObject : TObject) : TJSONObject;    // Stream a collection - always returns an array    function StreamCollection(Const ACollection: TCollection): TJSONArray;    // Stream an objectlist - always returns an array    function StreamObjectList(Const AnObjectList: TObjectList): TJSONArray;    // Stream a List - always returns an array    function StreamTList(Const AList: TList): TJSONArray;    // Stream a TStrings instance as an array    function StreamTStringsArray(Const AStrings: TStrings): TJSONArray;    // Stream a TStrings instance as an object    function StreamTStringsObject(Const AStrings: TStrings): TJSONObject;    // Stream a TStrings instance. Takes into account Options.    function StreamTStrings(Const AStrings: TStrings): TJSONData;    // Stream a variant as JSON.    function StreamVariant(const Data: Variant): TJSONData; virtual;    //    // Some utility functions.    //    // Call ObjectToJSON and convert result to JSON String.    Function ObjectToJSONString(AObject : TObject) : TJSONStringType;    // Convert TSTrings to JSON string with array or Object.    Function StringsToJSON(Const Strings : TStrings; AsObject : Boolean = False) : TJSONStringType;    // Convert collection to JSON string    Function CollectionToJSON(Const ACollection : TCollection) : TJSONStringType;  // Convert variant to JSON String    Function VariantToJSON(Const Data : Variant) : TJSONStringType;  Published    // Format used when formatting DateTime values. Only used in conjunction with jsoDateTimeToString    Property DateTimeFormat : String Read FDateTimeFormat Write FDateTimeFormat;    // Options to use when streaming    Property Options : TJSONStreamOptions Read FOptions Write FOptions;    // Called before streaming an object with ObjectToJSON  Property BeforeStreamObject : TJSONStreamEvent Read FBeforeStreamObject Write FBeforeStreamObject;    // Called After streaming an object with ObjectToJSON    Property AfterStreamObject : TJSONStreamEvent Read FAfterStreamObject Write FAfterStreamObject;    // Called whenever a property was streamed. If Res is nil on return, no property is added.    Property OnStreamProperty : TJSONPropertyEvent Read FOnStreamProperty Write FOnStreamProperty;    // Property name to use when streaming child components. Default is "Children"    Property ChildProperty : String Read GetChildProperty Write FChildProperty Stored IsChildStored;  end;   TJSONDeStreamer = Class(TJSONFiler)  private    FAfterReadObject: TJSONStreamEvent;    FBeforeReadObject: TJSONStreamEvent;    FDateTimeFormat: String;    FOnGetObject: TJSONGetObjectEvent;    FOnPropError: TJSONpropertyErrorEvent;    FOnRestoreProp: TJSONRestorePropertyEvent;    FCaseInsensitive : Boolean;    FOptions: TJSONDestreamOptions;    procedure DeStreamClassProperty(AObject: TObject; PropInfo: PPropInfo; PropData: TJSONData);    function GetCaseInsensitive: Boolean;    procedure SetCaseInsensitive(AValue: Boolean);  protected    // Try to parse a date.    Function ExtractDateTime(S : String): TDateTime;    function GetObject(AInstance : TObject; const APropName: TJSONStringType; D: TJSONObject; PropInfo: PPropInfo): TObject;    procedure DoRestoreProperty(AObject: TObject; PropInfo: PPropInfo;  PropData: TJSONData); virtual;    Function ObjectFromString(Const JSON : TJSONStringType) : TJSONData; virtual;    procedure RestoreProperty(AObject: TObject; PropInfo: PPropInfo; PropData: TJSONData);  Public    Constructor Create(AOwner : TComponent); override;    Destructor Destroy; override;    // Convert JSON object to properties of AObject    Procedure JSONToObject(Const JSON : TJSONStringType; AObject : TObject);    Procedure JSONToObject(Const JSON : TJSONObject; AObject : TObject);    // Convert JSON object/array to collection.    Procedure JSONToCollection(Const JSON : TJSONStringType; ACollection : TCollection);    Procedure JSONToCollection(Const JSON : TJSONData; ACollection : TCollection);  // Convert JSON array/object/string to TStrings    Procedure JSONToStrings(Const JSON : TJSONStringType; AStrings : TSTrings);    Procedure JSONToStrings(Const JSON : TJSONData; AStrings : TSTrings);    // Convert JSON data to a variant. Supports simple data types and arrays.    Function JSONToVariant(Data: TJSONData): Variant;    Function JSONToVariant(Data: TJSONStringType): Variant;    // Triggered at the start of each call to JSONToObject    Property BeforeReadObject : TJSONStreamEvent Read FBeforeReadObject Write FBeforeReadObject;    // Triggered at the end of each call to JSONToObject (not if exception happens)    Property AfterReadObject : TJSONStreamEvent Read FAfterReadObject Write FAfterReadObject;    // Called when a property will be restored. If 'Handled' is True on return, property is considered restored.    Property OnRestoreProperty : TJSONRestorePropertyEvent Read FOnRestoreProp Write FOnRestoreProp;    // Called when an error occurs when restoring a property. If Continue is False on return, exception is re-raised.    Property OnPropertyError : TJSONpropertyErrorEvent Read FOnPropError Write FOnPropError;    // Called when a object-typed property must be restored, and the property is Nil. Must return an instance for the property.    // Published Properties of the instance will be further restored with available data.    Property OngetObject : TJSONGetObjectEvent Read FOnGetObject Write FOnGetObject;    // JSON is by definition case sensitive. Should properties be looked up case- insentive ?    Property CaseInsensitive : Boolean Read GetCaseInsensitive Write SetCaseInsensitive ; deprecated;    // DateTime format. If not set, RFC3339DateTimeFormat is assumed.    // If set, it will be used as an argument to ScanDateTime. If that fails, StrToDateTime is used.    Property DateTimeFormat : String Read FDateTimeFormat Write FDateTimeFormat;    // Options overning the behaviour    Property Options : TJSONDestreamOptions Read FOptions Write FOptions;  end;  I would like to buid a JSON like this (more or less): Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {   "element" :       [         {            "name" : "TJSONFiler", "type" : "Class", "inheritsFrom" : "TComponent", "visibility" : "public", "isAbstract" : "false", "documentation" : "",            "operations" :               [                  {"name" : "Error", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "false", "returns" : "", "documentation" : "", "parameters" :                    [                        {"name" : "Msg", "type" : "String"}                     ]                  },                  {"name" : "Error", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "false", "returns" : "", "documentation" : "", "parameters" :                      [                        {"name" : "FMT", "type" : "String"},                         {"name" : "Args", "type" : "Array of const"}                     ]                  }             ]         },         {            "name" : "TJSONStreamer", "type" : "Class", "inheritsFrom" : "TJSONFiler", "visibility" : "public", "isAbstract" : "false", "documentation" : "",            "attributes" :               [                  {"name" : "FAfterStreamObject", "type" : "TJSONStreamEvent", "visibility" : "private", "documentation" : ""},                  {"name" : "FBeforeStreamObject", "type" : "TJSONStreamEvent", "visibility" : "private", "documentation" : ""},                  {"name" : "FChildProperty", "type" : "String", "visibility" : "private", "documentation" : ""},                  {"name" : "FDateTimeFormat", "type" : "String", "visibility" : "private", "documentation" : ""},                {"name" : "FOnStreamProperty", "type" : "TJSONPropertyEvent", "visibility" : "private", "documentation" : ""},                  {"name" : "FOptions", "type" : "TJSONStreamOptions", "visibility" : "private", "documentation" : ""}               ],            "operations" :               [                {"name" : "GetChildProperty", "visibility" : "private", "returns" : "String", "documentation" : "", "parameters" : []},                {"name" : "IsChildStored", "visibility" : "private", "returns" : "boolean", "documentation" : "", "parameters" : []},                  {"name" : "StreamChildren", "visibility" : "private", "returns" : "TJSONArray", "documentation" : "", "parameters" :                    [                        {"name" : "AComp", "type" : "TComponent"}                   ]                  },                  {"name" : "StreamClassProperty", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "true", "returns" : "TJSONData", "documentation" : "", "parameters" :                      [                      {"name" : "AObject", "type" : "TObject"}                   ]                  },                  {"name" : "StreamProperty", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "false", "returns" : "TJSONData", "documentation" : "", "parameters" :                      [                        {"name" : "AObject", "type" : "TObject"},                        {"name" : "PropertyName", "type" : "String"}                     ]                  },                {"name" : "StreamProperty", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "false", "returns" : "TJSONData", "documentation" : "", "parameters" :                      [                        {"name" : "AObject", "type" : "TObject"},                      {"name" : "PropertyInfo", "type" : "PPropInfo"}                   ]                  },                  {"name" : "FormatDateProp", "visibility" : "protected", "isAbstract" : "false", "isVirtual" : "false", "returns" : "TJSONString", "documentation" : "", "parameters" :                      [                        {"name" : "DateTime", "type" : "TDateTime"}                     ]                },                  {"name" : "ObjectToJSON", "visibility" : "public", "isAbstract" : "false", "isVirtual" : "false", "returns" : "TJSONObject", "documentation" : "Basic functions\n\nUse RTTI to stream object.\nIf AObject is of type TStrings or TCollection, special treatment occurs:\nTStrings results in { Strings: [S,S,S] } or { Strings: { "S1" : O1, "S2" : O2 }} depending on Options.\nCollection results in { Items: [I,I,I] }\n", "parameters" :                      [                        {"name" : "AObject", "type" : "TObject"}                   ]                  },                  {"name" : "", "visibility" : "public", "isAbstract" : "false", "isVirtual" : "false", "returns" : "", "documentation" : "", "parameters" :                    [                        {"name" : "", "type" : ""},                      {"name" : "", "type" : ""},                        {"name" : "", "type" : ""},                        {"name" : "", "type" : ""}                     ]                  },                {"name" : "", "visibility" : "public", "isAbstract" : "false", "isVirtual" : "false", "returns" : "", "documentation" : "", "parameters" :                      [                        {"name" : "", "type" : ""},                        {"name" : "", "type" : ""},                      {"name" : "", "type" : ""},                      {"name" : "", "type" : ""}                     ]                  },                                 ]         }      ]}  ETA: I plan to feed the StarUML extension with a file like that. I have a long way to go..... ---------------------------------------------------------------------- garlar27 · 2019-01-22 00:05 Quote from: avra on January 22, 2019, 12:38:21 amQuote from: garlar27 on January 21, 2019, 07:09:50 pmI want to make a plugin to import (reverse engineering) FPC/Lazarus project to StarUML 3Are you aware that StarUML used to be an open source Delphi project? There is a fork of last Delphi version here: https://sourceforge.net/projects/whitestaruml/ Yeah, I knew it was made with Delphi but didn't know of the fork... Quote from: avra on January 22, 2019, 12:38:21 amQuote from: garlar27 on January 21, 2019, 07:09:50 pmI don't want to write a tokenizer/parser in Java Script when everything is already made in pascal.Well, maybe you might want to write in pascal but just output javascript? http://wiki.freepascal.org/pas2js Not ATM Quote from: avra on January 22, 2019, 12:38:21 amQuote from: garlar27 on January 21, 2019, 07:09:50 pmMy plan is to generate a JSON file with the info needed to at least create a Class Diagram (in a first stage) or at least import the classes and types without a diagramYou might want to take a look at EssModel (https://sourceforge.net/projects/essmodelforlaza/), or use something like Gold (http://wiki.lazarus.freepascal.org/Gold). Gold and other parsers are fine when zooming into 99% of your code (grammars can not handle every possible case). When you need that 1% left then you need to go manual way. Tomorrow will take a look at them!!! ---------------------------------------------------------------------- af0815 · 2019-01-22 05:39 Some time ago, i have looked into fpdoc, and it is also IMHO building a parsertree for making the docs, maybe one idea/input more.