hurl.md - www.codemadness.org - www.codemadness.org saait content files
 (HTM) git clone git://git.codemadness.org/www.codemadness.org
 (DIR) Log
 (DIR) Files
 (DIR) Refs
 (DIR) README
 (DIR) LICENSE
       ---
       hurl.md (2257B)
       ---
            1 hurl is a relatively simple HTTP, HTTPS and Gopher client/file grabber.
            2 
            3 
            4 ## Why?
            5 
            6 Sometimes (or most of the time?) you just want to fetch a file via the HTTP,
            7 HTTPS or Gopher protocol.
            8 
            9 The focus of this tool is only this.
           10 
           11 
           12 ## Features
           13 
           14 * Uses OpenBSD pledge(2) and unveil(2). Allow no filesystem access (writes to
           15   stdout).
           16 * Impose time-out and maximum size limits.
           17 * Use well-defined exitcodes for reliable scripting (curl sucks at this).
           18 * Send as little information as possible (no User-Agent etc by default).
           19 
           20 
           21 ## Anti-features
           22 
           23 * No HTTP byte range support.
           24 * No HTTP User-Agent.
           25 * No HTTP If-Modified-Since/If-* support.
           26 * No HTTP auth support.
           27 * No HTTP/2+ support.
           28 * No HTTP keep-alive.
           29 * No HTTP chunked-encoding support.
           30 * No HTTP redirect support.
           31 * No (GZIP) compression support.
           32 * No cookie-jar or cookie parsing support.
           33 * No Gopher text handling (".\r\n").
           34 * ... etc...
           35 
           36 
           37 ## Dependencies
           38 
           39 * C compiler (C99).
           40 * libc + some BSD functions like err() and strlcat().
           41 * LibreSSL(-portable)
           42 * libtls (part of LibreSSL).
           43 
           44 
           45 ## Optional dependencies
           46 
           47 * POSIX make(1) (for Makefile).
           48 * mandoc for documentation: <https://mdocml.bsd.lv/>
           49 
           50 
           51 ## Clone
           52 
           53         git clone git://git.codemadness.org/hurl
           54 
           55 
           56 ## Browse
           57 
           58 You can browse the source-code at:
           59 
           60 * <https://git.codemadness.org/hurl/>
           61 * <gopher://codemadness.org/1/git/hurl>
           62 
           63 
           64 ## Download releases
           65 
           66 Releases are available at:
           67 
           68 * <https://codemadness.org/releases/hurl/>
           69 * <gopher://codemadness.org/1/releases/hurl>
           70 
           71 
           72 ## Build and install
           73 
           74         $ make
           75         # make install
           76 
           77 
           78 ## Examples
           79 
           80 Fetch the Atom feed from this site using a maximum filesize limit of 1MB and
           81 a time-out limit of 15 seconds:
           82 
           83         hurl -m 1048576 -t 15 "https://codemadness.org/atom.xml"
           84 
           85 There is an -H option to add custom headers. This way some of the anti-features
           86 listed above are supported. For example some CDNs like Cloudflare are known to
           87 block empty or certain User-Agents.
           88 
           89 User-Agent:
           90 
           91         hurl -H 'User-Agent: some browser' 'https://codemadness.org/atom.xml'
           92 
           93 HTTP Basic Auth (base64-encoded username:password):
           94 
           95         hurl -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' \
           96                 'https://codemadness.org/atom.xml'
           97 
           98 GZIP (this assumes the served response Content-Type is gzip):
           99 
          100         hurl -H 'Accept-Encoding: gzip' 'https://somesite/' | gzip -d