The http Package

The standard Tcl library includes an http package that is based on the code I wrote for this chapter. This section documents the package, which has a slightly different interface. The library version uses namespaces and combines the Http_Get, Http_Head, and Http_Post procedures into a single http::geturl procedure. The examples in this chapter are still interesting, but you should use the standard http package for your production code.

http::config

The http::config command is used to set the proxy information, time-outs, and the User-Agent and Accept headers that are generated in the HTTP request. You can specify the proxy host and port, or you can specify a Tcl command that is run to determine the proxy. With no arguments, http::config returns the current settings:

http::config
=> -accept */* -proxyfilter http::ProxyRequired
-proxyhost {} -proxyport {}
-useragent {Tcl http client package 2.4}

If you specify just one option, its value is returned:

http::config -proxyfilter
=> http::ProxyRequired

You can set one or more options:

http::config -proxyhost webcache.eng -proxyport 8080

The default proxy filter just returns the -proxyhost and -proxyport values if they are set. You can supply a smarter filter that picks a proxy based on the host in the URL. The proxy filter is called with the hostname and should return a list of two elements, the proxy host and port. If no proxy is required, return an empty list.

http::geturl

The http::geturl procedure does a GET, POST, or HEAD transaction depending on its arguments. By default, http::geturl blocks until the request completes and it returns a token that represents the transaction. As described below, you use the token to get the results of the transaction. If you supply a -command callback option, then http::geturl returns immediately and invokes callback when the transaction completes. The callback is passed the token that represents the transaction.

For simple applications you can simply block on the transaction:

set token [http::geturl www.beedub.com/index.html]
=> http::1

The leading http:// in the URL is optional. The return value is a token that represents the transaction. There are other http:: commands that return information when passed the token. The token is also the name of an array that contains state about the transaction. Make sure to clean up this array to free memory when you are done:

http::cleanup $token

If you need to access the array directly, use upvar to create an alias:

upvar #0 $token data

Table 17-1 lists the options to http::geturl.

Table 17-1. Options to the http::geturl command

-binary boolean	Specifies whether we should do a binary transfer of the data. (Tcl 8.3)
-blocksize num	Block size when copying to a channel.
-channel fileID	The fileID is an open file or socket. The URL data is copied to this channel instead of saving it in memory.
-command callback	Calls callback when the transaction completes. The token from http::geturl is passed to callback.
-handler command	Called from the event handler to read data from the URL.
-headers list	The list specifies a set of headers that are included in the HTTP request. The list alternates between header keys and values.
-progress command	Calls command after each block is copied to a channel. It gets called with three parameters: command token totalsize currentsize
-query codedstring	Issues a POST request with the codedstring form data.
-queryblocksize num	Block size when copying to the query channel.
-querychannel fileID	The fileID is an open file or socket. The query data is copied from this channel instead of passed in a string.
-queryprogress command	Calls command after each block is copied from the query channel. It gets called with three parameters: command token totalsize currentsize
-timeout msec	Aborts the request after msec milliseconds have elapsed.
-type mime-type	Use mime-type as the Content-Type value during a POST operation.
-validate bool	If bool is true, a HEAD request is made.

Table 17-2 lists the access functions to the state array.

Table 17-2. The http support procedures

http::cleanup $token	Unsets the state array named by $token.
http::code $token	Returns state(http).
http::data $token	Returns state(body).
http::error $token	Returns state(error).
http::ncode $token	Returns the numeric return code contained in state(http).
http::size $token	Return the number of bytes read from the URL so far.
http::status $token	Returns state(status).
http::wait $token	Blocks until the transaction completes.

The array elements are listed in Table 17-3:

Table 17-3. Elements of the http::geturl state array

body	The contents of the URL.
charset	The value of the charset attribute from the Content-Type meta-data value. If none was specified, this defaults to the RFC standard iso8859-1.
coding	A copy of the Content-Encoding meta-data value.
currentsize	The current number of bytes transferred.
error	An explanation of why the transaction was aborted.
http	The HTTP reply status.
meta	A list of the keys and values in the reply header.
posterror	An explanation of why the transaction was aborted when writing post query data, if any.
status	The current status: pending, ok, eof, or reset.
totalsize	The expected size of the returned data.
type	The content type of the returned data.
url	The URL of the request.

You can take advantage of the asynchronous interface by specifying a command that is called when the transaction completes. The callback is passed the token returned from http::geturl so that it can access the transaction state:

http::geturl $url -command [list Url_Display $text $url]
proc Url_Display {text url token} {
   upvar #0 $token state
   # Display the url in text
}

You can have http::geturl copy the URL to a file or socket with the -channel option. This is useful for downloading large files or images. In this case, you can get a progress callback so that you can provide user feedback during the transaction. Example 17-12 shows a simple downloading script:

Example 17-12 Downloading files with http::geturl

#!/usr/local/bin/tclsh8.4
if {$argc < 2} {
   puts stderr "Usage: $argv0 url file"
   exit 1
}
package require http
set url [lindex $argv 0]
set file [lindex $argv 1]
set out [open $file w]
proc progress {token total current} {
   puts -nonewline "."
}
http::config -proxyhost webcache.eng -proxyport 8080
set token [http::geturl $url -progress progress \
   -headers {Pragma no-cache} -channel $out]
close $out
# Print out the return header information
puts ""
upvar #0 $token state
puts $state(http)
foreach {key value} $state(meta) {
   puts "$key: $value"
}
exit 0

http::formatQuery

If you specify form data with the -query option, then http::geturl does a POST transaction. You need to encode the form data for safe transmission. The http::formatQuery procedure takes a list of keys and values and encodes them in x-www-url-encoded format. Pass this result as the query data:

http::formatQuery name "Brent Welch" title "Tcl Programmer"
=> name=Brent+Welch&title=Tcl+Programmer

http::register and http::unregister

The http::register procedure registers a protocol handler for URL protocols other than HTTP. The http::unregister procedure removes the handler registration. The primary application is to provide secure web access via HTTPS and the TLS extension.

package require tls
http::register https 443 ::tls::socket
set token [http::geturl https://my.secure.site/]

http::reset

You can cancel an outstanding transaction with http::reset:

http::reset $token

This is done automatically when you setup a -timeout with http::config.

http::cleanup

When you are done with the data returned from http::geturl, use the http::cleanup procedure to unset the state variable used to store the data.