Need library for numerical (and symbolic) math lazarus / 21-General ---------------------------------------------------------------------- suleyman · 2024-01-20 20:51 Hello, I'm in the engineering field. Therefore, numerical calculations are very important to me. As I read there is only Numlib in FreePascal (Lazarus). Are there any free alternatives? What I exactly need is: 1) Solving nonlinear systems of equations 2) Levenberg-Marquardt method. Is there already a single library for the Levenberg-Marquardt method? beste regards ---------------------------------------------------------------------- wp · 2024-01-20 22:26 https://sourceforge.net/p/lmath-library/code/HEAD/tree/LMath/: contains Levenberg-Marquardt - see attached test project as an example. https://github.com/chadilukito/www.wolfgang-ehrhardt.de/tree/master (folder src/misc)https://sourceforge.net/projects/jedimath/maybe (well: probably) more... ---------------------------------------------------------------------- suleyman · 2024-01-20 23:56 Thank you very much, now I can let off steam in Pascal again Is there also an example (since I can learn better with examples) for solving non-linear systems of equations? There are files in the folder (\lmNonLinEq), but how can I make them executable? ---------------------------------------------------------------------- TRon · 2024-01-21 03:58 Quote from: suleyman on January 21, 2024, 12:56:47 amIs there also an example (since I can learn better with examples) for solving non- linear systems of equations? see console example ? Quotehere are files in the folder (\lmNonLinEq), but how can I make them executable? see wiki adding new packages ---------------------------------------------------------------------- mtrsoft · 2024-01-21 06:10 Quote2) Levenberg-Marquardt method. Is there already a single library for the Levenberg-Marquardt method? Not exactly what you are asking for but there is a commercial application that uses the Marquardt alogrithm to fit data to nonlinear equations. Check the Downloads > FitAll page at www.fitall.com for a very limited (free) evaluation edition of FitAll. Regards, John ---------------------------------------------------------------------- Thaddy · 2024-01-21 08:07 Well, sort of, not tested:[bing]Certainly! The Levenberg-Marquardt algorithm is a popular method for solving non-linear least squares problems, especially in least squares curve fitting ¹. FreePascal provides a `Math` unit that contains a number of mathematical functions, including the `Sqrt` function for calculating square roots and the `Power` function for raising a number to a power ². Here's an example implementation of the Levenberg-Marquardt algorithm in FreePascal using the `Math` unit: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses Math;// TODO: change array of double everywhere to the type TDoubleArray, declared as an array of double.// TODO2: change the function parameters f and df to a function prototype and implement them.// TODO3: test if something else won't compile// it does not work yet// have fun, Thaddy function LevenbergMarquardt(x: array of Double; y: array of Double; f: function(x: Double; a: array of Double): Double; df: function(x: Double; a: array of Double): Double; a: array of Double; lambda: Double; maxIter: Integer): array of Double;var    n, m, i, j, k: Integer;    J, JTJ, r, delta, aNew: array of Double;  alpha: Double;begin    n := Length(x);    m := Length(a);    SetLength(J, n * m);    SetLength(JTJ, m * m);    SetLength(r, n);    SetLength(delta, m);    SetLength(aNew, m);     for i := 0 to maxIter - 1 do    begin        // Calculate Jacobian matrix        for j := 0 to n - 1 do        begin            for k := 0 to m - 1 do            begin                J[j * m + k] := df(x[j], a)[k];            end;          r[j] := y[j] - f(x[j], a);        end;         // Calculate JTJ and JTr        for j := 0 to m - 1 do        begin            for k := 0 to m - 1 do            begin                JTJ[j * m + k] := 0;                for n := 0 to Length(x) - 1 do                begin                    JTJ[j * m + k] := JTJ[j * m + k] + J[n * m + j] * J[n * m + k];                end;            end;            delta[j] := 0;            for n := 0 to Length(x) - 1 do            begin              delta[j] := delta[j] + J[n * m + j] * r[n];            end;      end;         // Add damping parameter to JTJ        for j := 0 to m - 1 do        begin            JTJ[j * m + j] := JTJ[j * m + j] * (1 + lambda);        end;         // Solve linear system        for j := 0 to m - 1 do        begin            alpha := 0;            for k := 0 to m - 1 do            begin                alpha := alpha + JTJ[j * m + k] * delta[k];            end;            aNew[j] := a[j] + alpha;        end;         // Check for convergence        if (MaxValue(delta) < 1e-6) then        begin            Result := aNew;            Exit;        end;         // Update damping parameter      if (MaxValue(delta) > MaxValue(a) / 10) then        begin            lambda := lambda * 10;        end        else if (MaxValue(delta) < MaxValue(a) / 100) then        begin            lambda := lambda / 10;        end;         a := aNew;    end;     Result := a;end;  This implementation takes in the following parameters: - `x`: An array of independent variable values. - `y`: An array of dependent variable values. - `f`: A function that takes in an independent variable value and an array of parameters and returns a predicted dependent variable value. - `df`: A function that takes in an independent variable value and an array of parameters and returns the derivative of `f` with respect to each parameter. - `a`: An array of initial parameter values. - `lambda`: A damping parameter that controls the step size in each iteration. - `maxIter`: The maximum number of iterations to perform. The function returns an array of parameter values that minimize the sum of squared residuals between the predicted and actual dependent variable values. I hope this helps! Bron: Gesprek met Bing, 21-1-2024 (1) Levenberg–Marquardt algorithm - Wikipedia. https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm. (2) arXiv:1807.02176v3 [math.OC] 10 Jul 2021. https://arxiv.org/pdf/1807.02176v3.pdf. (3) Levenberg-Marquardt Method -- from Wolfram MathWorld. https://mathworld.wolfram.com/Levenberg-MarquardtMethod.html. ---------------------------------------------------------------------- Thaddy · 2024-01-21 09:34 For completeness: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i // the types you needtype    TDoubleArray = array of Double;    TFunction = function(x: Double; a: TDoubleArray): Double;    TDFunction = function(x: Double; a: TDoubleArray): TDoubleArray; An here the suggested implementation for the function prototypes f and df:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function MyFunction(x: Double; a: TDoubleArray): Double;begin    Result := a[0] * Sin(a[1] * x) + a[2] * Exp(-a[3] * x);end; function MyDFunction(x: Double; a: TDoubleArray): TDoubleArray;begin    SetLength(Result, 4);    Result[0] := Sin(a[1] * x);    Result[1] := a[0] * x * Cos(a[1] * x);    Result[2] := Exp(-a[3] * x);    Result[3] := -a[2] * x * Exp(-a[3] * x);end; After that it should work, since the math is otherwise correct. ---------------------------------------------------------------------- Thaddy · 2024-01-21 10:04 Here's the programmer touched version (me), compiles in 3.2.2 Debian Bookworm:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$mode delphi}{$H+}uses Math; // have fun, Thaddy// the types you needtype  TDoubleArray = array of Double;    TFunction = function(x: Double; a: TDoubleArray): Double;    TDFunction = function(x: Double; a: TDoubleArray): TDoubleArray; var F:TFunction; DF:TDFunction; function MyFunction(x: Double; a: TDoubleArray): Double;begin    Result := a[0] * Sin(a[1] * x) + a[2] * Exp(-a[3] * x);end; function MyDFunction(x: Double; a: TDoubleArray): TDoubleArray;begin    SetLength(Result, 4);    Result[0] := Sin(a[1] * x);    Result[1] := a[0] * x * Cos(a[1] * x);    Result[2] := Exp(-a[3] * x);    Result[3] := -a[2] * x * Exp(-a[3] * x);end; function LevenbergMarquardt(x: TDoubleArray; y: TDoubleArray; f: TFunction; df:TDFunction; a: TDoubleArray; lambda: Double; maxIter: Integer): TDoubleArray;var    n, m, i, j, k: Integer;    JJ, JTJ, r, delta, aNew: TDoubleArray;    alpha: Double;begin    n := Length(x);    m := Length(a);    SetLength(JJ, n * m);    SetLength(JTJ, m * m);    SetLength(r, n);    SetLength(delta, m);    SetLength(aNew, m);     for i := 0 to maxIter - 1 do    begin        // Calculate Jacobian matrix        for j := 0 to n - 1 do        begin            for k := 0 to m - 1 do            begin                JJ[j * m + k] := df(x[j], a)[k];            end;            r[j] := y[j] - f(x[j], a);        end;         // Calculate JTJ and JTr        for j := 0 to m - 1 do        begin            for k := 0 to m - 1 do            begin                JTJ[j * m + k] := 0;                for n := 0 to Length(x) - 1 do                begin                    JTJ[j * m + k] := JTJ[j * m + k] + JJ[n * m + j] * JJ[n * m + k];                end;            end;            delta[j] := 0;            for n := 0 to Length(x) - 1 do            begin              delta[j] := delta[j] + JJ[n * m + j] * r[n];            end;        end;         // Add damping parameter to JTJ        for j := 0 to m - 1 do        begin            JTJ[j * m + j] := JTJ[j * m + j] * (1 + lambda);        end;         // Solve linear system        for j := 0 to m - 1 do        begin            alpha := 0;            for k := 0 to m - 1 do            begin                alpha := alpha + JTJ[j * m + k] * delta[k];            end;            aNew[j] := a[j] + alpha;        end;         // Check for convergence        if (MaxValue(delta) < 1e-6) then        begin            Result := aNew;            Exit;        end;         // Update damping parameter      if (MaxValue(delta) > MaxValue(a) / 10) then        begin            lambda := lambda * 10;        end        else if (MaxValue(delta) < MaxValue(a) / 100) then        begin            lambda := lambda / 10;        end;         a := aNew;    end;     Result := a;end;beginend.95% GPT4 and 5% programmer.... ---------------------------------------------------------------------- wp · 2024-01-21 10:21 Quote from: Thaddy on January 21, 2024, 11:04:12 am95% GPT4 and 5% programmer.... I already wanted to write a comment saying: "Reads like code written by ChatGPT: Looking excellent, but when it comes to running the compiler it does not work". [EDIT] Looks like the programmer's contribution is too small since the code does not fit even a straight line - see attachment. BTW, how to determine the initial Lambda value? ---------------------------------------------------------------------- marcov · 2024-01-21 10:54 I've tried to use the Jedi Math LM a long time ago, but it seemed to easily give a weird line if there were outliers. I then switched to TPMath's  Gaussian regression, but in the end ended up writing an harness around that (that throws away outliers and recalcs). Maybe I didn't understand the checking for errors(and the per point residu) of the LM solution at the time though.