Translation from Python to Lazarus lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- mpknap · 2018-07-18 05:24 I have a request. The following code sends the file to the server. The main project is in Python, I do not know this programming environment. I do not know JSON. Anyone can write this in Lazarus Pascal for windows? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i import_ requestsimport_ jsonimport_ timeimport_ base64import_ urllib3 urllib3.disable_warnings()picture = open('test4.png', 'rb') # read picturepicture_read = picture.read()base64_picture = base64.b64encode(picture_read) # save picture as  base 64print(len((str(base64_picture)[2:]))) # cutting off the first two characters from byte encoding, you may not need tolinux_time = time.time()linux_time_millisecund = [item for item in str(linux_time) if item != '.'] # formatting time according to the linux era # Below you have two variables in the form of a dictionary, which are protocols for logging in and sending data, this is what I sent youearlier and comes from the creed page (with minor changes) # data in the login in addition to the password and e-mail are free, if you registered a cell phone it will match. login = {'app_version': '1', 'device_model': 'zero', 'device_type': 'raspberry','system_version': 'raspbian','device_id': 'raspberry','password': 'hasło', "email": "email"}  # in detection the data in addition to frame content and timestamp are also arbitrary, remember to limit the linux time to 13 characters as on the server,otherwise you will have problems with the date. detection = {"detections": [{"frame_content": '%s'% (str(base64_picture)[2:]), "timestamp": int(''.join(linux_time_millisecund)[:-4]), "latitude": 51.708491,"longitude": 19.476362, "altitude": 12, "accuracy": 1, "provider": 'google map', "width": 45, "height": 45, "id": 1}],"device_id": "raspberry", "androidVersion": 'none', "device_model": "zero", "app_version": '1',"system_version": 'raspbian', "device_type": 'raspberry'} # Once I've created variables, I save them in the json format. with open("login.json", "w") as login_file:    json.dump(login, login_file)with open("login.json", "r") as read_login:    data_login = json.load(read_login) with open("detection.json", "w") as detection_file:    json.dump(detection, detection_file)with open("detection.json", "r") as read_detection:    data_detetion = json.load(read_detection) #request login is to get the authentication token needed to send the image to the server request_login = requests.post('https://api.credo.science/api/v2/user/login', verify=False, json=data_login)print(request_login.status_code, request_login.reason)token = request_login.json()['token'] #in request detection you are sending a json file and a token code, and that's all, the file is on the server request_detection = requests.post('https://api.credo.science/api/v2/detection', verify=False,                                  json=data_detetion, headers={'Authorization': 'Token %s' % token})print(request_detection.status_code, request_detection.reason)print(request_detection.content) ---------------------------------------------------------------------- sash · 2018-07-18 12:05 Quote from: mpknap on July 18, 2018, 07:24:26 amI do not know this programming environment. I do not know JSON. Anyone can write this ... Shouldn't it be posted in "jobs" then? ---------------------------------------------------------------------- mpknap · 2018-07-19 18:59 it's not a job offer, it's a free project I do not expect a ready solution (but it would be nice), I'm asking for tips, advice, whatever. I do not know how to start and whether it is possible to translate at all ... ---------------------------------------------------------------------- sash · 2018-07-19 20:14 QuoteI do not know how to start Well, then learn basics about JSON, HTTP APIs, Base64 encoding, then basic Python syntax, and you'll be able to translate it for yourself. I don't know Python either, but given code has comments, and as I can see, it uses some HTTP API (and API provider's address - so you can check their documentation). Maybe you won't ever need a Python knowledge, if you know exact task, with their protocol description, you will be able to recreate Pascal code from scratch, with some tweaking and debugging. ---------------------------------------------------------------------- engkin · 2018-07-20 04:22 To read the picture: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses Classes... var  picture: TFileStream;  picture_read: array of byte;...  picture := TFileStream.Create('test4.png',fmOpenRead or fmShareDenyNone);  try    SetLength(picture_read, picture.Size);    picture.Read(picture_read[1], Length(picture_read));  finally    picture.Free;  end; To convert the read picture to base64: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses base64... var  b64: TBase64EncodingStream;  base64_picture: TStringStream;  picture_read: array of byte;...  base64_picture := TStringStream.Create('');  try    b64 := TBase64EncodingStream.Create(base64_picture);    try      b64.write(picture_read[1], Length(picture_read));    finally      b64.Free;    end; //Do whatever with the result    WriteLn(base64_picture.DataString);//*************   finally    base64_picture.Free;  end; To generate JSON in accordance with their specs: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses fpjson, ....var   O : TJSONObject;   O := TJSONObject.Create([       'body',       TJSONObject.Create([       'detection',       TJSONObject.Create([                           'accuracy',1.0,                         'altitude',1.0,                           'frame_content',base64_picture.DataString                            //add the rest here//}                            ]),//detection      'device_info',       TJSONObject.Create([                           'adnroidVersion','7.0',                           'deviceId','device Id',                           'deviceModel','device model'                         ]),//device_info        'user_info',       TJSONObject.Create([                           'email','user@email.com',                           'key','access_key'                           ])//user_info                        ]),//body       'header',       TJSONObject.Create([                         'application','1.3',                           'frame_type','detection',                           'protocol','1.0',                           'time_stamp',1518707872                         ])//header                         ]);//root   //See the result  Writeln(O.FormatJSon);  //********** There is a better way to generate the JSON using classes. Take a look at Laz1.8.2\fpc\3.0.4\source\packages\fcl-json\examples specifically demortti.pp ---------------------------------------------------------------------- mpknap · 2018-07-20 06:15 Thanks ENGKIN, it looks like it works. There is the last thing I have a problem with from the beginning. How to send it. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i request_login = requests.post('https://api.credo.science/api/v2/user/login', verify=False, json=data_login)print(request_login.status_code, request_login.reason)token = request_login.json()['token']   request_detection = requests.post('https://api.credo.science/api/v2/detection', verify=False,                                  json=data_detetion, headers={'Authorization': 'Token %s' % token}) ---------------------------------------------------------------------- Trenatos · 2018-07-20 15:12 Take a look at fphttp simple post. ---------------------------------------------------------------------- engkin · 2018-07-21 02:13 Quote from: mpknap on July 20, 2018, 08:15:54 amThere is the last thing I have a problem with from the beginning. How to send it. To login/send your data use fphttpclient. Here is logging in: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses fphttpcleint,......var  requests: TFPHTTPClient;...  requests := TFPHTTPClient.Create(nil);  requests.AllowRedirect:=True;  try    data := TStringStream.Create('');    try      O.DumpJSON(data);  //<--- O/data holds whatever you want to send      requests.AddHeader('User- Agent','Your prefered agent');      requests.AddHeader('Content- type','application/json');      requests.RequestBody := data;      WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/user/login')); //<-- extract the token for here      WriteLn('Code: ', requests.ResponseStatusCode, 'Text: ',requests.ResponseStatusText);    WriteLn();    finally      data.Free;    end;  finally    requests.Free;  end; To use secured connection "HTTPS" you need to get OpenSSL library. Have the files available to your application (same folder/add them to the correct system folder/add their path) ---------------------------------------------------------------------- mpknap · 2018-07-26 06:28 It does not quite work for me. I did this procedure Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button3Click(Sender: TObject);var  requests: TFPHTTPClient;  data: TStringStream; begin  requests := TFPHTTPClient.Create(nil);  requests.AllowRedirect:=True;  try    data := TStringStream.Create('');   finally  end;  try    o := TJSONObject.Create([       'login',       TJSONObject.Create([                      'app_version',1,                        'device_model', 1,                       'device_type',1,                       'system_version',1,                         'device_id',1,                      'password', 'S73071YT6',                        'email', 'mpknap@wp.pl'                             ])                   ]);          try      o.DumpJSON(data);  //<--- O/data holds whatever you want to send      requests.AddHeader('User-Agent','Your prefered agent');      requests.AddHeader('Content-type','application/json');    requests.RequestBody := data;     LABEL1.caption:=('Response: '+requests.Post('https://api.credo.science/api/v2/user/login')); //<-- extract the token for here      memo3.caption:=('Code: '+ inttostr(requests.ResponseStatusCode)+ 'Text: '+ (requests.ResponseStatusText));     finally      data.Free;    end;  finally    requests.Free;  end;  After Break (Attached Image), the line from LABEL1 is underlined ... ---------------------------------------------------------------------- engkin · 2018-07-26 15:36 Your JSON is wrong. According to their data it should be: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {"app_versi on":1,"device_model":1,"device_type":1,"system_version":1,"device_id": 1,"password":"S73071YT6","email":"mpknap@wp.pl"} You can do that either directly: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     data := TStringStream.Create('{"app_version":1,"device_model":1,"device_type": 1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpk nap@wp.pl"}'); or: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i       o := TJSONObject.Create([                          'app_version',1,                        'device_model', 1,                         'device_type',1,                         'system_version',1,                         'device_id',1,                          'password', 'S73071YT6',                          'email', 'mpknap@wp.pl'                   ]);      O.DumpJSON(data); I just tried it and the response was: QuoteResponse: {"username":"mpknap","display_name":"Marek .K","language":"pl","token":"S73071YT6","team":"credo Forum","email":"mpknap@wp.pl"} Edit: The code I used in my test: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   requests := TFPHTTPClient.Create(nil);  requests.AllowRedirect:=True;  try    //https://github.com/credo-science/credo- webapp/blob/master/credoapiv2    data := TStringStream.Create('{"app_v ersion":1,"device_model":1,"device_type":1,"system_version":1,"device_ id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');    try      requests.AddHeader('User-Agent','YourPreferedAgent');      requests.AddHeader('Content-type','application/json');      requests.RequestBody := data;      WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/user/login'));      WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);      WriteLn();    finally      data.Free;    end;  finally    requests.Free;  end; ---------------------------------------------------------------------- mpknap · 2018-07-28 07:08 Dear Engkin I will not hesitate well. Json is not a subject for me, however. Since you have already helped a lot, I am asking you for the last step. We have received the Token. How to send the content of Detection. I am doing something wrong because I get the answer" Invalid authorization header". Please help and close the topic I will be grateful Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  data := TS tringStream.Create('{"app_version":1,"device_model":1,"device_type":1, "system_version":1,"device_id":1,"password":"S73071YT6","email":"mpkna p@wp.pl"}');    detection := TStringStream.Create( '{"detections": [{"frame_content": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAAAAAA7VNdtAAACv0 lEQVRIDVXBMVbrMBRAwfue5O8YQxYEHUVKejg07L+HJBLyv40KZuIZVbSiE9rRhhaUqKGG 4hlVtKIT2tGGFpSooYbiGVW0ohPa0YYWlKihhuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFp SooYbiBe3ohDb0gCoqqKAf1FG8oB2d0IYeUEUFFfSDOooXtKMT2tADqqiggn5QR/GCdnRC G3pAFRVU0A/qKF7Qjk5oQw+oooIK+kEdxSt6QhtaUUWJkqmjhuIVPaENraiiRMnUUUPxip 7QhlZUUaJk6qiheEVPaEMrqihRMnXUULyiJ7ShFVWUKJk6aigu6Iw2tPDHggYaqKG4oDPa 0MIfCxpooIbigs5oQwt/LGiggRqKCzqjDS38saCBBmooLuiMNrTwx4IGGqihuKAz2lFBgQ JV1NEvGigu6Ix2VFCgQBV19IsGigs6ox0VFChQRR39ooHigs5oRwUFClRRR79ooLigM9pR QYECVdTRLxoo3tAj+ocKKiiZOmroG8UbekT/UEEFJVNHDX2jeEOP6B8qqKBk6qihbxRv6B H9QwUVlEwdNfSN4g09on+ooIKSqaOGvlF8oIoqCpQoUaA7uqIbig9UUUWBEiUKdEdXdEPx gSqqKFCiRIHu6IpuKD5QRRUFSpQo0B1d0Q3FB6qookCJEgW6oyu6oXhHK0oUKFCggRq6o4 7iHa0oUaBAgQZq6I46ine0okSBAgUaqKE76ije0YoSBQoUaKCG7qijeEcrShQoUKCBGrqj juITJUqUKNGBBhrohjqKT5QoUaJEBxpooBvqKD5RokSJEh1ooIFuqKP4RIkSJUp0oIEGuq GO4hMlSpQo0YEGGuiGOoovpgMVpsHU0BUdKL6YDlSYBlNDV3Sg+GI6UGEaTA1d0YHii+lA hWkwNXRFB4ovpgMVpsHU0BUd6D8pRLGV1zBGtwAAAABJRU5ErkJggg==", "timestamp": "1528481412762", "latitude": "51.708491", "longitude": "19.476362", "altitude": "12", "accuracy": "1", "provider": "google map", "width": "680", "height": "460", "id": "1"}], "device_id": "raspberry", "androidVersion": "none", "device_model": "zero", "app_version": "1", "system_version": "raspbian", "device_type": "raspberry"}'); ---------------------------------------------------------------------- engkin · 2018-07-28 15:36 No problem at all. I'll be glad to help finish it as soon as I get a chance. Most likely tomorrow. ---------------------------------------------------------------------- engkin · 2018-07-29 05:53 Get the token: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  token: string;...  try    //https://github.com/credo-science/credo- webapp/blob/master/credoapiv2    data := TStringStream.Create('{"app_v ersion":1,"device_model":1,"device_type":1,"system_version":1,"device_ id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');    try      requests.AddHeader('User-Agent','YourPreferedAgent');      requests.AddHeader('Content-type','application/json');      requests.RequestBody := data;      response := requests.Post('https://api.credo.science/api/v2/user/login');      WriteLn('Response: ',response);      WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);       O := TJSONObject(GetJSON(response));      try        token := O.Get('token');      finally        O.Free;      end;       WriteLn('Token: ',token);    finally      data.Free;    end;  finally    requests.Free;  end; According to the docs, pass the token in the header: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   requests := TFPHTTPClient.Create(nil);  requests.AllowRedirect:=True;  try    detection:=TStringStream.Create('{"detections": [{"frame_content": "iV BORw0KGgoAAAANSUhEUgAAADIAAAAyCAAAAAA7VNdtAAACv0lEQVRIDVXBMVbrMBRAwfue 5O8YQxYEHUVKejg07L+HJBLyv40KZuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiGV W0ohPa0YYWlKihhuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiBe3ohDb0gCoqqKAf 1FG8oB2d0IYeUEUFFfSDOooXtKMT2tADqqiggn5QR/GCdnRCG3pAFRVU0A/qKF7Qjk5oQw +oooIK+kEdxSt6QhtaUUWJkqmjhuIVPaENraiiRMnUUUPxip7QhlZUUaJk6qiheEVPaEMr qihRMnXUULyiJ7ShFVWUKJk6aigu6Iw2tPDHggYaqKG4oDPa0MIfCxpooIbigs5oQwt/LG iggRqKCzqjDS38saCBBmooLuiMNrTwx4IGGqihuKAz2lFBgQJV1NEvGigu6Ix2VFCgQBV1 9IsGigs6ox0VFChQRR39ooHigs5oRwUFClRRR79ooLigM9pRQYECVdTRLxoo3tAj+ocKKi iZOmroG8UbekT/UEEFJVNHDX2jeEOP6B8qqKBk6qihbxRv6BH9QwUVlEwdNfSN4g09on+o oIKSqaOGvlF8oIoqCpQoUaA7uqIbig9UUUWBEiUKdEdXdEPxgSqqKFCiRIHu6IpuKD5QRR UFSpQo0B1d0Q3FB6qookCJEgW6oyu6oXhHK0oUKFCggRq6o47iHa0oUaBAgQZq6I46ine0 okSBAgUaqKE76ije0YoSBQoUaKCG7qijeEcrShQoUKCBGrqjjuITJUqUKNGBBhrohjqKT5 QoUaJEBxpooBvqKD5RokSJEh1ooIFuqKP4RIkSJUp0oIEGuqGO4hMlSpQo0YEGGuiGOoov pgMVpsHU0BUdKL6YDlSYBlNDV3Sg+GI6UGEaTA1d0YHii+lAhWkwNXRFB4ovpgMVpsHU0B Ud6D8pRLGV1zBGtwAAAABJRU5ErkJggg==", "timestamp": "1528481412762", "latitude": "51.708491", "longitude": "19.476362", "altitude": "12", "accuracy": "1", "provider": "google map", "width": "680", "height": "460", "id": "1"}], "device_id": "raspberry", "androidVersion": "none", "device_model": "zero", "app_version": "1", "system_version": "raspbian", "device_type": "raspberry"}');    try      requests.AddHeader('User-Agent','YourPreferedAgent');      requests.AddHeader('Content-type','application/json');      requests.AddHeader('Authorization','Token '+token);      requests.RequestBody := detection;      WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/detection'));      WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);      WriteLn();    finally      detection.Free;    end;   finally    requests.Free;  end; The previous code resulted in: QuoteResponse: {"username":"mpknap","display_name":"Marek .K","language":"pl","token":"S73071YT6","team":"credo Forum","email":"mpknap@wp.pl"} Code: 200, Text: OK Token: S73071YT6 Response: {"detections":[{"id":1194285}]} Code: 200, Text: OK ---------------------------------------------------------------------- mpknap · 2018-07-29 14:18 Looking at your code in the last answer, I see that I was close .  Everything works, no errors, but why is not the upload done? It should show a new detection on https://api.credo.science/web/user/mpknap/ In any case, thank you for everything, I would never solve this problem myself. ---------------------------------------------------------------------- engkin · 2018-07-29 16:05 Yes, you were almost there, and I am glad to help. Quote from: mpknap on July 29, 2018, 04:18:43 pmwhy is not the upload done? It should show a new detection on https://api.credo.science/web/user/mpknap/ I don't know precisely, but I can try to guess: The list is sorted based on [json reported] date. The one I did is 1528481412762ms ~ 2018.06.08Maybe they filter detections that seem wrong. Like mismatching width/height compared to image details. (Actual: 50x50. Reported: 680x460)Possibly you had uploaded that picture before? You can download your detections using /api/v2/data_export to see if the data is there. Maybe it is time to get in touch with them and see if they have a better answer. ---------------------------------------------------------------------- mpknap · 2018-07-29 18:52 I changed the timstamp to the current one, I also changed the size of the image and then nothing. As a friend says only FRAME_CONTENT and TIMESTAMP are required, the rest is not important. Strange, because should work: / ---------------------------------------------------------------------- engkin · 2018-07-29 21:38 In this case we only need to double check the two needed values. Looking at the Python code you posted above, I see the code skips one or two chars: str(base64_picture)[2:]. That might be it. 1st Edit: I doubt that it matters, but just in case, the TIMESTAMP numerical value does not need quotation marks, otherwise it would be considered text. I can't test, yet. 2nd Edit: It seems that api.credo.science is down. 3rd Edit: According to their source code, for a detection to be visible it has to fulfill two conditions 1-frame_content is a "valid image": Code: Python  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i         frame_content = base64.b64decode(d['frame_content'])        visible = True        if (not frame_content) or (not validate_image(frame_content)):            visible = False         if visible:            if not r:                r = get_redis_connection(write=False) 2-timestamp is later than the user's "start_time": Code: Python  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i           start_time = r.zscore(cache.make_key('start_time'), request.user.id)          if start_time:                visible = d['timestamp'] > start_time            else:                visible = False I think the tests do not fulfill the second condition. They do get stored, but they are not visible. 4th Edit: I tried again with a recent timestamp value: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  dt: TDateTime;  ut: int64;...begin  dt := Now();  ut := (DateTimeToUnix(dt)-1)*1000+248; passing it in the detection stream: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     detection:=TStringStream.Create('{"detections": [{"frame_content": ......, "timestamp": '+intToStr(ut)+',.... It is visible, dated 2018-07-30. ---------------------------------------------------------------------- mpknap · 2018-07-31 04:26 Cool!!! I do not know how to thank you. You are fantastic!!! I always try to solve the problems myself but  I struggled with this thread for 3 months, and you did it yourself. Thanks thanks !!!. I also invite you to the CREDO project, there everyone is welcome. Engkin, you are great!