Help with an SFTP script reddit / freebsd ---------------------------------------------------------------------- iCashMon3y · 2023-05-16 22:46 I am trying create a recurring job that connects to an sftp server, checks to see if there have been any new files put there in the past 10 minutes and then download them. Below is the code that I have so far, and I have been using ChatGPT to help because I have never done this before in FreeBSD #!/usr/local/bin/expect # Define the SFTP server, username, and password set SFTP_SERVER "redacted" set SFTP_USERNAME "redacted" set SFTP_PASSWORD "redacted" # Define the local destination folder to download the files set DEST_FOLDER "/usr/home/sftp/Test/files/test" # Define the remote SFTP folder path set SFTP_FOLDER "/" # Define the path to the private key file set KEY_FILE "/usr/home/sftp/test/ssh_host_rsa_key_test.priv" # Define the maximum file age in minutes set MAX_FILE_AGE 10 # Enable verbose output for debugging log_user 1 # Spawn the SFTP command spawn sftp -i $KEY_FILE $SFTP_USERNAME@$SFTP_SERVER # Expect password prompt expect { "password:" { send -- "$SFTP_PASSWORD\r" exp_continue } "sftp>" { # Change to the SFTP folder send -- "cd $SFTP_FOLDER\r" expect "sftp>" # List the files in the SFTP folder send -- "ls -l\r" expect "sftp>" # Get the current timestamp set current_time [clock seconds] # Download the new files set files [split $expect_out(buffer) "\n"] foreach file $files { if {[string length $file] > 0 && ![string match "*total*" $file]} { # Extract the file timestamp set file_info [split $file] set file_timestamp [concat [lindex $file_info 5] [lindex $file_info 6] [lindex $file_info 7]] set file_time [clock scan $file_timestamp -format "%b %e %H:%M"] # Calculate the file age in minutes set age [expr {($current_time - $file_time) / 60}] if {$age " } } } # Quit the SFTP session send -- "quit\r" expect eof exit 0 } timeout { exit 1 } } I am getting the following error when I try and run this input string does not match supplied format while executing "return -options $opts $result" (procedure "::tcl::clock::scan" line 83) invoked from within "::tcl::clock::scan {} -format {%b %e %H:%M}" ("uplevel" body line 1) invoked from within "uplevel 1 [info level 0]" (procedure "::tcl::clock::scan" line 4) invoked from within "clock scan $file_timestamp -format "%b %e %H:%M"" ("foreach" body line 7) invoked from within "foreach file $files { if {[string length $file] > 0 && ![string match "*total*" $file]} { # Extract the file timestamp ..." invoked from within "expect { "password:" { send -- "$SFTP_PASSWORD\r" exp_continue } "sftp>" { # Change to the SFTP folder sen..." (file "con_and_download.expect" line 27) I know there is something wrong with the formating here but I can't seem to figure it out. The output from the -l from the server is like this -rw-rw-rw- 1 user group xxxx May 2 11:54 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (filename) Any help would be greatly appreciated. Thanks! ---------------------------------------------------------------------- dkh · 2023-05-16 23:45 I know it"s not what you asked but two things... From the admittedly small sample set of programming things I've tried with chatgpt, it is horrible unless you ask it for something really simple. Could certainly be down to the way I have done my prompts. For an sftp task like this I would use perl and Net::SFTP::Foriegn. I imagine there is something similar for python but I don't enjoy using python. ---------------------------------------------------------------------- Queueded · 2023-05-17 00:02 I'm going to back up a bit, and first ask why you're not using keys. You don't want to type a password for something that's going to run periodically, that's stupid. Second, why on Earth use SFTP for this? You can check for the presence of the file using ssh, if you really need to check for the presence of a file. Or, you could just use rsync and save yourself a lot of time and trouble if you just want to update everything. ---------------------------------------------------------------------- iCashMon3y · 2023-05-17 00:11 Because we were told this is how we have to connect by the company we are working with. ---------------------------------------------------------------------- GunsKnivesRadios · 2023-05-17 00:33 You should still use rsync, but use sshpass too for the password. Just curious, but is this the real estate industry? I did about a decade of hard time myself and saw sysadmin practices that were so bad it was kind of amazing. This looks familiar. ---------------------------------------------------------------------- iCashMon3y · 2023-05-17 01:41 No it is not real estate. I am a straight FreeBSD noob so I am completely unfamiliar with rsync. ---------------------------------------------------------------------- DimestoreProstitute · 2023-05-17 01:53 If you're doing comparisons, could you use fusefs-sshfs instead of sftp? ---------------------------------------------------------------------- laffer1 · 2023-05-17 03:20 Sometimes you can tell Chatgpt the error with the code it provided and it will fix it. ---------------------------------------------------------------------- [deleted] · 2023-05-17 05:47 [deleted] ---------------------------------------------------------------------- iCashMon3y · 2023-05-17 05:52 yeah that's not possible, we have 0 control on the other side. ---------------------------------------------------------------------- [deleted] · 2023-05-17 08:46 Oh no, anything but EXPECT :D Here you go, this is just an example you can tweak it to your needs - [https://pastebin.com/UTAZscxX](https://pastebin.com/UTAZscxX) A few notes, if you are running the script in Linux and the SFTP system is FreeBSD change /usr/local/bin/bash to /bin/bash. If you are running the script in FreeBSD, run pkg install bash sshpass before that so you can get the required packages. If you are running it on Linux, well check your packages manager for sshpass (Linux has bash by default). Let me know if you need further help. Don't use Expect unless you absolutely have to and in this case you don't have to. ---------------------------------------------------------------------- rodrigo975 · 2023-05-17 12:20 >A few notes, if you are running the script in Linux and the SFTP system is FreeBSD change /usr/local/bin/bash to /bin/bash. If you are running the script in FreeBSD, run pkg install bash sshpass before that so you can get the required packages. If you are running it on Linux, well check your packages manager for sshpass (Linux has bash by default). Or just use #! /usr/bin/env bash .... ---------------------------------------------------------------------- iCashMon3y · 2023-05-17 14:37 Thank you very much, quick question on sshpass, will this allow me to pass a key as well? We pass a key on the initial connection and then it prompts us for a password. ---------------------------------------------------------------------- yadad · 2023-05-17 14:37 It's most likely installed on the other side. Please check as you'll save yourself time ---------------------------------------------------------------------- [deleted] · 2023-05-17 14:44 Hm interesting system. Yes you can pass a key in the ssh arguments - something like sshpass ssh -i /path/to/key. Same with scp. Test it if it’s working, if not ping me either here on in a DM to see what isn’t working. ---------------------------------------------------------------------- iCashMon3y · 2023-05-17 14:52 Yeah that is why chatGPT introduced expect into the equation lol. I will give this a shot later on this afternoon and let you know. Thanks again!