*Please* finish reading this readme file. It will tell you everything I can think of regarding the scripts in this package. I have tried to break things up into some sort of logical order, but I tend to ramble on, sort of _Stream of Conciousness_ documentation if you will. Please forgive the poor spelling, grammer, etc. There are additional comments in many of the scripts. The Minimum You Have To Do - At the very least you have to set a few variables in the runme file, and a few defines in ftpmenu.c Please take the time to do that. If you dont nothing will work right. If you're like me you typed 'make' before 'more readme', well I suppose you'll live. It probably didnt work due to the fact I dont understand 'make' yet, so I used a shell script to do everything, you can use 'runme', just pretend its a make file, it would be if I knew what I was doing. Anyway, almost all of these scripts rely on csh. If youre csh _isn't_ at /bin/csh then they wont run. They also rely on the berkeley curses C library, and a pager (more, less) Ive tried to note restrictions and requirements. Then just type ftpnightly, this should get the ball rolling. Things should take care of updating themselves after that, you can just use the routines. If you just want to see how things work then skip typing ftpnightly. Youll be working with old data, but at least youll be able to see what the interfaces to ftpnew, and ftpmenu are like. The Makefile - See the comments at the front of the makefile for a list of everything it will do. It adds lines to your .cshrc, and adds a directory at your home level among others. The Scripts - The basic idea - At a high level heres how things work. You set up an 'at' job that runs every night. It will do a bunch of stuff for you automatically. In general it gets the new listings and abstract files. Then when you log in for the day, you can use some other scripts to view the information. Of course things get filtered along the way, and all this information eats up disk space, but it sure speeds up routine archive checking. If you let things go it doesnt matter, since todays files replace yesterdays, and the onlything wasted is the automatic ftp to your archive sites in the middle of the night. More recently Ive added the ability to automatically mirror the last few days of new stuff to a local directory. If you always want to look at the new things, this is a godsend. (A friend of mine hacked this one actually.) _Beware_ a hell of a lot of stuff can get posted in two days. It averages around 1M, but its been as high as 15M, and 3-4M is not uncommmon. If you want to enable this, you'll have to take a look at $FTPEXTRAS/bin/ftpnightly. $FTPEXTRAS is an environment variable that tells where all this stuff goes. The make file will create FtpExtra/ in your home directory unless you tell it otherwise. Oh yea, I almost forgot, script files, in $FTPEXTRAS/scripts/ tell this nightly job exactly what it is looking for. My examples should be explanation enough. Reading the comments in the script files, really will aid your understanding. Yipes.. Ive used script in two differnet ways. Shell script, and ftpcheck script. The files in $FTPEXTRAS/scripts/ are internally used. Things in $FTPEXTRAS/bin/ are shell scripts you can run from the prompt. These are the ones with comments. The scripts you will use - 'call ' - uses the info in your .netrc file to save you typing e.g. I type 'call sum' instead of 'ftp sumex-aim.stanford.edu' Basically it does a substring search and ftp's to any match. 'ftpnew [month]' - uses data files that get automatically generated to show you a listing of whats new this (or the specified) month. I originally used this as an easy way to see if anything new was posted to sumex, long before I knew about info-mac.digest. 'ftpmenu' - Menu driven program based on the curses package allows you to retrieve recent posts releatively easily. A good tool for browsing if you've been away for a few weeks. 'undo' - unbinhexes everything in the current dierectory, and trashes the .hqx if successful The scripts that run behind the scenes - 'ftpcheck' - Runs script files you set up to get information from your favorite ftp sites. 'ftpdoc' - If you are running the mirroring option, then this script takes care of exttracting doc files. 'ftpmenubase' - the real meat of ftpmenu 'ftpnightly' - runs automatically every night. 'mirror.etc.etc.etc.and.more.stuff' - if it starts off mirror. then it has something to do with mirroring. (Obviously I suppose) Some aliases - sumex - Goes directly to the directory holding the sumex mirror. Dependancies - less,more - If you dont have a one of these pagers the abstract function of 'ftpmenu' wont work. mcread - Available at sumex-aim, this utility can translate MacWrite macbinary files into text. I use it to extract docs with ftpdoc during the mirroring operation. unsit - Also used by ftpdoc. mcvert - Debinhexes files. Used by undo, and during mirroring. macbin - used by ftpdoc. All of these should be availible at sumex-aim.stanford.edu Questions - Q: How can I add sites to the ones already being watched? How can I add remove directories from those being watched? A: This is controlled by two things. Your .netrc, and the scripts in $FTPEXTRAS/scripts/ When the automatic stuff runs, it looks in your .netrc. It steps to each line and reads the entry, if a script (internal) is present then it gets run. So to add a site: add it to your .netrc, and then create a script file. The script file should be named script.host.domain and be in $FTPEXTRAS/scripts/ Please note that the spacing of the .netrc file is important, since I check specific columns for this or that info. To change directories, simply modify an existing script. It should be obvious if you look at one of them, they are simply sequential ftp commands. Q: How can I change the number of days I am mirroring? How can just mirror permanently? A: These are controled in mirror.sumex-aim.stanford.edu and mirror.sumex.filter. To change the number of days, look for the foreach, and include more or less numbers depending on your preference. To miror permanently, comment out the 'rm *' line that gets rid of yesterdays files. Q: How do I use ftpmenu? A: The idea behind this program is to let you easily retrieve files from ftp sites. When you run the program, it should tell you that it is loading some information. You will then see a menu of sites. Pick a site, then pick a directory from the new menu. You will then see a list of files, just go to the file, you want and hit space. It will get marked (visibly). You can mark other files, etc.. When you are done, exit the program, and the files will be retrieved into the current directory. A short help screen is available by hitting 'h' or '?' Q: When I use ftpmenu, and try and look at an abstract, I dont see one. Not only that the controls stop working right, and when I hit ^C to get out of the program, my keypresses dont echo properly. A: This occurs because somefile doesnt exist. Heres what you should check. The path to 'less' or 'more' at the very beginning of ftpmenu.c. Make sure this points to a valid copy. If this path is invalid you will get screwy behavior as above. Another thing that could be wrong is the absense of an abstract file. Make sure the parameter that ftpmenu (the script) passes to ftpmenubase (the realprogram) is valid. If you havent been fiddling, then It should work the way I have it set up. The last thing you should do is mail me, and tell me to include real error checking. (Like checking whether files exist) I have really poor checking in this program, since I orginally developed it for _me_ and of course I know what should go where. Q: How can I change the order of things when using 'ftpnew'? A: If you chnage the order things are listed in your .netrc then the ordering in both ftpnew and ftpmenu, will change. To change the order of directories, change the order of your $FTPEXTRAS/scripts/whatever file. Q: When I run ftpmenu, all the choices appear twice, how come? A: Look in your .netrc. Probably you have the site listed twice. This could happen easily if you ran 'runme' more than once. Youve probably got duplicate lines in youre .cshrc also. Q: How do I get ftpmenu to show me abstracts from other sites besides Sumex? A: You need to see if the site has an abstract file you can get. For instance mac.archive.umich.edu could be added by following these steps. 1) Modify the $FTPEXTRAS/script/script.mac.archive.umich.edu file so that it gets the /mac/00help/index.txt file. Take a look at the script for sumex to see how to do this. 2) Add a line to ftpnightly, that will 'cat' the two abstract files together. such as 'cat abstract.sumex-aim.stanford.edu abstract.umich > totalabstract' 3) Change 'ftpmenu' so that the abstract file is totalabstract instead of the sumex abstract. Make sense? Q: Some other miscellaneous problem? A: Mail me, and ask. Maybe Ill know. About Me - I'm a Junior in computer science at University California, Davis. You can reach me at jedavis@ucdavis.edu. Id be happy to explain, or make small changes, if I have time. As soon as school starts back up (its summer now) Ill be fairly busy though. But please feel free to ask. Now could you do me a favor: write me a brief note saying what you, like dont like about this whole thing. Or if you got it running at all. If you work somewhere that has official letterhead, and you use these, then take the time to boost my ego by expressing it on that official letterhead, and sending it to: James Davis 6708 Austin Way Sacramento, CA 95823 Incidentally thats my parents address so it should be good for quite a while, as opposed to my local address which changes quite often. At this point you should go modify: ftpmenu.c, runme, bin/ftpnightly .