From nobody@FreeBSD.org  Thu Dec 16 12:57:31 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 753C41065672
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 16 Dec 2010 12:57:31 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (unknown [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 4D82D8FC12
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 16 Dec 2010 12:57:31 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id oBGCvULx076926
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 16 Dec 2010 12:57:30 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id oBGCvUE9076924;
	Thu, 16 Dec 2010 12:57:30 GMT
	(envelope-from nobody)
Message-Id: <201012161257.oBGCvUE9076924@red.freebsd.org>
Date: Thu, 16 Dec 2010 12:57:30 GMT
From: Mario Kleinboelting <mario@kleinboelting.de>
To: freebsd-gnats-submit@FreeBSD.org
Subject: freebsd-update can not fetch updates over a proxy with mandatory authorization
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         153211
>Category:       bin
>Synopsis:       freebsd-update(8) can not fetch updates over a proxy with mandatory authorization
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    cperciva
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 16 13:00:18 UTC 2010
>Closed-Date:    
>Last-Modified:  Fri Dec 17 20:40:07 UTC 2010
>Originator:     Mario Kleinboelting
>Release:        8.1-RELEASE (Version from CD without any updates)
>Organization:
>Environment:
FreeBSD mr0111.de.deutz.de 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 2010     root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
>Description:
I installed a FreeBSD 8.1 i386 Server from CD-ROM.
After installation i tried to update the server with freebsd-update and portsnap. 
The freebsd-update did not work properly in our environment.
freebsd-update can not fetch updates because the internet connection is established with an auth proxy. Our proxy requires username an password.

Two environment variables are set for the proxy connection. http_proxy and ftp_proxy.
Both contains the proxy URL and the proxy credentials and are exported.

http_proxy="http://username:password@http-proxy:3128"
ftp_proxy="http://username:password@ftp-proxy:3428"

The start of freebsd-update -v debug fetch says:
----------------------------------------------------------------------
# freebsd-update -v debug fetch
Looking up update.FreeBSD.org mirrors... none found.
Fetching metadata signature for 8.1-RELEASE from update.FreeBSD.org... 
latest.ssl                                    100% of  512  B 1667 kBps
done.
Fetching metadata index... 
14b0dfddc908e278d231e3290507fe1ab01e6ceb553a9d100% of  225  B  552 kBps
done.
Fetching 2 metadata files... 
/usr/libexec/phttpget update.FreeBSD.org 8.1-RELEASE/i386/m/61dfab427765793984bc911a1eab992cffc22c26bba438fe11a665820fcf97f2.gz 8.1-RELEASE/i386/m/e376c4c604af7362453546c3ebb4ad439f900173000d8412aabcf0101c92311d.gz
phttpget: host = username, port = password@http-proxy:3128: servname not supported for ai_socktype
failed.
----------------------------------------------------------------------
freebsd-update seems to evaluate the variable and cuts server and port with ":".
Now we have ":" two times. Delimiter for username and passwort and delimiter for proxyserver and port.


I have tested the variable http_proxy and ftp_proxy with only the proxy server and the variable http_proxy_auth with the proxy credentials. 

Then freebsd-update says:
----------------------------------------------------------------------
# export http_proxy="http://http-proxy:3128"
# export ftp_proxy="http://ftp-proxy:3428"
# export http_proxy_auth="basic:*:username:password"
# freebsd-update -v debug fetch
Looking up update.FreeBSD.org mirrors... none found.
Fetching metadata signature for 8.1-RELEASE from update.FreeBSD.org... 
fetch: http://update.FreeBSD.org/8.1-RELEASE/i386/latest.ssl: Proxy Authentication Required
failed.
No mirrors remaining, giving up.
----------------------------------------------------------------------
It seems that freebsd-update did not recognize the http_proxy_auth variable

portsnap and /usr/bin/fetch works fine with our proxy an handles the authentication info in http_proxy and ftp_proxy correct.

Our Proxyserver is squid on solaris. Every connection to the internet must use username and a password by corporate policy.

FYI: Username and password are not my real proxy credentials...
>How-To-Repeat:
Use freebsd-update behind a (squid) proxy with mandatory username and password authorization.
>Fix:
Rewrite of /usr/libexec/phttpget and freebsd-update to work with auth information in proxy URL's.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-www->cperciva 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Dec 16 16:31:46 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153211 

From: Mario Kleinboelting <mario@kleinboelting.de>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/153211: freebsd-update(8) can not fetch updates over a proxy
 with mandatory authorization
Date: Fri, 17 Dec 2010 21:21:48 +0100

 I did a little bit of testing to reproduce the problem in my private 
 environment.
 I can reproduce the problem on two patched FreeBSD 8.1 systems with 
 8.1-RELEASE-p2 in my LAN also.
 
 # uname -a
 FreeBSD hera 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 
 2010     root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
 
 # uname -a
 FreeBSD horus 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 
 UTC 2010     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
 
 
 To build a testproxy I used this squid config to reconfigure my 
 testproxy (squid-3.0.25_2 of FreeBSD 8.1):
 auth_param basic program /usr/local/libexec/squid/ncsa_auth 
 /usr/local/etc/squid/squidpasswd
 acl password proxy_auth REQUIRED
 http_access allow password
 
 /usr/local/etc/squid/squidpasswd contains the credentials in htpasswd 
 format.
 
 
 When /var/db/freebsd-update contains old cached data the freebsd-update 
 call is not a problem:
 ------------------------------------------------
 # export http_proxy="http://username:password@192.168.0.5:3128/"
 # find /var/db/freebsd-update/
 /var/db/freebsd-update/
 /var/db/freebsd-update/files
 /var/db/freebsd-update/files/61dfab427765793984bc911a1eab992cffc22c26bba438fe11a665820fcf97f2.gz
 /var/db/freebsd-update/files/e376c4c604af7362453546c3ebb4ad439f900173000d8412aabcf0101c92311d.gz
 /var/db/freebsd-update/tINDEX.present
 /var/db/freebsd-update/serverlist_tried
 /var/db/freebsd-update/serverlist_full
 /var/db/freebsd-update/serverlist
 /var/db/freebsd-update/pub.ssl
 /var/db/freebsd-update/tag
 # freebsd-update -v debug fetch
 Looking up update.FreeBSD.org mirrors... 4 mirrors found.
 Fetching metadata signature for 8.1-RELEASE from update5.FreeBSD.org...
 latest.ssl                                    100% of  512  B 1004 kBps
 done.
 Fetching metadata index...
 14b0dfddc908e278d231e3290507fe1ab01e6ceb553a9d100% of  225  B  426 kBps
 done.
 Inspecting system... done.
 Preparing to download files... done.
 
 No updates needed to update system to 8.1-RELEASE-p2.
 ------------------------------------------------
 
 
 When the cached content in /var/db/freebsd-update/files is deleted or 
 the directory /var/db/freebsd-update is empty (on a fresh installed 
 system) the problem is there:
 ------------------------------------------------
 # export http_proxy="http://username:password@192.168.0.5:3128/"
 # rm 
 /var/db/freebsd-update/files/61dfab427765793984bc911a1eab992cffc22c26bba438fe11a665820fcf97f2.gz 
 /var/db/freebsd-update/files/e376c4c604af7362453546c3ebb4ad439f900173000d8412aabcf0101c92311d.gz
 # freebsd-update -v debug fetch
 Looking up update.FreeBSD.org mirrors... 4 mirrors found.
 Fetching metadata signature for 8.1-RELEASE from update5.FreeBSD.org...
 latest.ssl                                    100% of  512  B  948 kBps
 done.
 Fetching metadata index...
 14b0dfddc908e278d231e3290507fe1ab01e6ceb553a9d100% of  225  B   39 kBps
 done.
 Fetching 2 metadata files...
 /usr/libexec/phttpget update5.FreeBSD.org 
 8.1-RELEASE/i386/m/61dfab427765793984bc911a1eab992cffc22c26bba438fe11a665820fcf97f2.gz 
 8.1-RELEASE/i386/m/e376c4c604af7362453546c3ebb4ad439f900173000d8412aabcf0101c92311d.gz
 phttpget: host = username, port = password@192.168.0.5:3128: servname 
 not supported for ai_socktype
 failed.
 ------------------------------------------------
 
 
 Regards
 Mario
 
>Unformatted:
