From odip@pierino.bionet.nsc.ru  Tue Jun  1 03:07:56 2004
Return-Path: <odip@pierino.bionet.nsc.ru>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 170CD16A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  1 Jun 2004 03:07:56 -0700 (PDT)
Received: from pierino.bionet.nsc.ru (pierino.bionet.nsc.ru [193.125.179.50])
	by mx1.FreeBSD.org (Postfix) with ESMTP id BEA8943D5D
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  1 Jun 2004 03:07:53 -0700 (PDT)
	(envelope-from odip@pierino.bionet.nsc.ru)
Received: from pierino.bionet.nsc.ru (localhost [127.0.0.1])
	by pierino.bionet.nsc.ru (8.12.8p1/8.12.8) with ESMTP id i51A7iRl029227
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 1 Jun 2004 17:07:44 +0700 (NOVST)
	(envelope-from odip@pierino.bionet.nsc.ru)
Received: (from odip@localhost)
	by pierino.bionet.nsc.ru (8.12.8p1/8.12.8/Submit) id i51A7hBh029226;
	Tue, 1 Jun 2004 17:07:43 +0700 (NOVST)
	(envelope-from odip)
Message-Id: <200406011007.i51A7hBh029226@pierino.bionet.nsc.ru>
Date: Tue, 1 Jun 2004 17:07:43 +0700 (NOVST)
From: Dmitry A Grigorovich <odip@bionet.nsc.ru>
Reply-To: Dmitry A Grigorovich <odip@bionet.nsc.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: [PATCH] mirror don't fetch files with $ in names when update_local=yes
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         67445
>Category:       ports
>Synopsis:       [PATCH] mirror don't fetch files with $ in names when update_local=yes
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pav
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jun 01 03:10:14 PDT 2004
>Closed-Date:    Tue Jun 01 07:29:03 PDT 2004
>Last-Modified:  Tue Jun 01 07:29:03 PDT 2004
>Originator:     Dmitry A Grigorovich
>Release:        FreeBSD 4.8-RELEASE i386
>Organization:
ICiG SB RAS
>Environment:
System: FreeBSD pierino.bionet.nsc.ru 4.8-RELEASE FreeBSD 4.8-RELEASE #5: Sat Oct 4 02:28:14 NOVST 2003 root@pierino.bionet.nsc.ru:/usr/obj/usr/src/sys/ODIP i386
>Description:

Use port ftp/mirror and include update_local=yes in config
This mean, that mirror must update only files, which present in local directory

To do this mirror scan local directory and build get_patt pattern.
Pattern get_patt have perl RE syntax, like '^file1$|^file2$'.
This pattern must match only local files and no more.

Next step, mirror list remote files and update only files,
which match get_patt.

But mirror incorrectly build get_patt pattern
if file name have special characters like '^', '$' and other.
Pattern don't match this files and accordingly mirror
don't update this files from remote site.

>How-To-Repeat:

1) Create file config with content:

package=test
        comment=test
        site=ftp.bionet.nsc.ru
        passive_ftp=true
        # where to start pulling files back from
        remote_dir=/test
        # where to put the files on your machine
        local_dir=/home/odip/mirror/local
	update_local=yes

2) Put non empty files to ftp://ftp.bionet.nsc.ru/test/
    with names: normal_file, $bad_file, bad_file$

root# ls -l /home/ftp/test
total 6
-rw-r--r--  1 root  wheel   9 Jun  1 16:44 $bad_file
-rw-r--r--  1 root  wheel   9 Jun  1 16:44 bad_file$
-rw-r--r--  1 root  wheel  12 Jun  1 16:16 normal_file

3) Create empty dir local
   Create empty files with names: normal_file, $bad_file, bad_file$

odip$ ls -l local
total 0
-rw-r--r--  1 odip  odip  0  1  16:58 $bad_file
-rw-r--r--  1 odip  odip  0  1  16:58 bad_file$
-rw-r--r--  1 odip  odip  0  1  16:58 normal_file

4) Run mirror in debug mode

odip@hydra$ mirror -d -p test ./config
package=test ftp.bionet.nsc.ru:/test -> /home/odip/mirror/local
Scanning local directory /home/odip/mirror/local
get_patt = ^$bad_file|^bad_file$|^normal_file
Scanning remote directory /test
compare directories (src 3, dest 3)
No files to transfer
unlink /home/odip/mirror/local/bad_file$
unlink /home/odip/mirror/local/$bad_file

disconnecting from ftp.bionet.nsc.ru
All done, Exiting


ERROR: get_patt in invalid and mirror remove two files !!!

5) Show local directory

odip@hydra$ ls -l local
total 2
-r--r--r--  1 odip  odip  12  1  09:16 normal_file

>Fix:

Put follow file into /usr/ports/ftp/mirror/files
Remove port ftp/mirror
Build and install port ftp/mirror

--- patch-ba begins here ---
--- mirror.pl.orig	Thu Aug  7 11:47:06 2003
+++ mirror.pl	Fri Apr 30 14:21:48 2004
@@ -1323,8 +1323,11 @@
 
 		# Create a get_patt from the contents of the local directory
 		if( $update_local && $#get_top >= 0 ){
-			$get_patt = '^' . join( '|^', @get_top );
-			$get_patt =~ s/$squished//g;
+			my ($path,$re,@re_patt);
+			#$get_patt = '^' . join( '|^', @get_top );
+			#$get_patt =~ s/$squished//g;
+			foreach $path ( @get_top ) { push @re_patt, "\^\Q$path\E\$"; }
+			$get_patt= join( '|', @re_patt );
 			&msg( "get_patt = $get_patt\n" ) if $debug;
 		}
 	
--- patch-ba ends here ---


Try again test above

3) Create empty dir local
   Create empty files with names: normal_file, $bad_file, bad_file$

odip$ ls -l local
total 0
-rw-r--r--  1 odip  odip  0  1  17:03 $bad_file
-rw-r--r--  1 odip  odip  0  1  17:03 bad_file$
-rw-r--r--  1 odip  odip  0  1  17:04 normal_file

4) Run mirror in debug mode

odip@hydra$ mirror -d -p test ./config
package=test ftp.bionet.nsc.ru:/test -> /home/odip/mirror/local
Scanning local directory /home/odip/mirror/local
get_patt = ^\$bad_file$|^bad_file\$$|^normal_file$
Scanning remote directory /test
compare directories (src 3, dest 3)
Got bad_file$ 9 0
Got $bad_file 9 0
Got normal_file 12 0

disconnecting from ftp.bionet.nsc.ru
All done, Exiting

5) Show local directory

odip@hydra$ ls -l local
total 6
-r--r--r--  1 odip  odip   9  1  09:44 $bad_file
-r--r--r--  1 odip  odip   9  1  09:44 bad_file$
-r--r--r--  1 odip  odip  12  1  09:16 normal_file


All is ok - files mirrored
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: pav 
State-Changed-When: Tue Jun 1 04:32:46 PDT 2004 
State-Changed-Why:  
Asked maintainer for approval. 


Responsible-Changed-From-To: freebsd-ports-bugs->pav 
Responsible-Changed-By: pav 
Responsible-Changed-When: Tue Jun 1 04:32:46 PDT 2004 
Responsible-Changed-Why:  
Handle. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=67445 

From: Pav Lucistnik <pav@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org, odip@bionet.nsc.ru,
	ryan@sasknow.com
Cc:  
Subject: Re: ports/67445: [PATCH] mirror don't fetch files with $ in names
	when update_local=yes
Date: Tue, 01 Jun 2004 13:32:42 +0200

 Dear maintainer of FreeBSD port ftp/mirror, please take a look at
 
 http://www.freebsd.org/cgi/query-pr.cgi?q=67445
 
 Do you approve this patch?
 
 -- 
 Pav Lucistnik <pav@oook.cz>
               <pav@FreeBSD.org>
 
 You take the red pill, you stay in Wonderland, and I show you how deep
 the rabbit hole goes....

From: Ryan Thompson <ryan@sasknow.com>
To: Pav Lucistnik <pav@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org, odip@bionet.nsc.ru
Subject: Re: ports/67445: [PATCH] mirror don't fetch files with $ in namesn
 update_local=yes
Date: Tue, 1 Jun 2004 08:10:18 -0600 (CST)

 Pav Lucistnik wrote to freebsd-gnats-submit@FreeBSD.org, odip@bionet.nsc.ru...:
 
 > Dear maintainer of FreeBSD port ftp/mirror, please take a look at
 >
 > http://www.freebsd.org/cgi/query-pr.cgi?q=67445
 >
 > Do you approve this patch?
 
 It looks fine to me. Thanks!
 
 - Ryan
 
 -- 
   Ryan Thompson <ryan@sasknow.com>
 
   SaskNow Technologies - http://www.sasknow.com
   901-1st Avenue North - Saskatoon, SK - S7K 1Y4
 
         Tel: 306-664-3600   Fax: 306-244-7037   Saskatoon
   Toll-Free: 877-727-5669     (877-SASKNOW)     North America
State-Changed-From-To: feedback->closed 
State-Changed-By: pav 
State-Changed-When: Tue Jun 1 07:28:45 PDT 2004 
State-Changed-Why:  
Committed, thanks! Please also try to contact author 
about this patch 

http://www.freebsd.org/cgi/query-pr.cgi?pr=67445 
>Unformatted:
