From edwin@mavetju.org  Sun Sep 22 18:46:54 2002
Return-Path: <edwin@mavetju.org>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id B021A37B401
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 22 Sep 2002 18:46:54 -0700 (PDT)
Received: from topaz.mdcc.cx (topaz.mdcc.cx [212.204.230.141])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 20AE343E4A
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 22 Sep 2002 18:46:54 -0700 (PDT)
	(envelope-from edwin@mavetju.org)
Received: from k7.mavetju (topaz.mdcc.cx [212.204.230.141])
	by topaz.mdcc.cx (Postfix) with ESMTP id E4A2B2B8BD
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 23 Sep 2002 03:46:50 +0200 (CEST)
Received: by k7.mavetju (Postfix, from userid 1001)
	id 0C1706A7124; Mon, 23 Sep 2002 11:46:32 +1000 (EST)
Message-Id: <20020923014632.0C1706A7124@k7.mavetju>
Date: Mon, 23 Sep 2002 11:46:32 +1000 (EST)
From: Edwin Groothuis <edwin@mavetju.org>
Reply-To: Edwin Groothuis <edwin@mavetju.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: [patch] make bsd.port.mk checksum check aware of multiple people fetching/compiling 
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         43273
>Category:       ports
>Synopsis:       [patch] make bsd.port.mk checksum check aware of multiple people fetching/compiling
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    portmgr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sun Sep 22 18:50:01 PDT 2002
>Closed-Date:    Fri Mar 18 23:04:30 GMT 2005
>Last-Modified:  Fri Mar 18 23:04:30 GMT 2005
>Originator:     Edwin Groothuis
>Release:        FreeBSD 4.5-RELEASE i386
>Organization:
-
>Environment:
System: FreeBSD k7.mavetju 4.5-RELEASE FreeBSD 4.5-RELEASE #4: Sat Sep 14 10:41:50 EST 2002 edwin@k7.mavetju:/usr/src/sys/compile/k7 i386

$FreeBSD: ports/Mk/bsd.port.mk,v 1.424 2002/09/19 00:16:39 kris Exp $

>Description:

Sometimes, two people are running make at the same time for a
related port, for example Alice is running make for port A which
depends on port C and Bob is running make for port B which depend
on port C too.

So Alice is downloading the tarball for port C and Bob runs make for
port B. Bobs make will see that port C isn't installed, but it finds
a file in /usr/ports/distfiles and will do the checksum match. It
fails, because the tarball is still incomplete. Bob removed the
tarball from /usr/ports/distfiles and tries again. Now it downloads.
In the mean time, Alice has completed her download and her make
finds that the tarball (which is now being downloaded bij Bob) has
the wrong checksum (because it's still being downloaded). Ad
infinitum until one gives up and gets a cup of coffee, in the mean
time the other will complete the download, install port C and install
his/her port B or A.

>How-To-Repeat:

See description

>Fix:

This patch will check if the file for which the checksum fails is
being opened in write-mode for somebody else, giving it a different
message:

--- bsd.port.mk.old	Mon Sep 23 11:26:02 2002
+++ bsd.port.mk	Mon Sep 23 11:33:51 2002
@@ -2882,8 +2882,11 @@
 				OK="false"; \
 			elif ${EXPR} "$$CKSUM2" : ".*$$CKSUM" > /dev/null; then \
 				${ECHO_MSG} ">> Checksum OK for $$file."; \
-			else \
+			elif [ ! -z "`/usr/bin/fstat $$file | ${AWK} '{ print $9 }' | ${GREP} w`" ]; then \
+				${ECHO_MSG} ">> Checksum mismatch for $$file, somebody is writing to it right now."; \
+				OK="false"; \
+			else \
 				${ECHO_MSG} ">> Checksum mismatch for $$file."; \
 				OK="false"; \
 			fi; \
 		  done; \

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-ports->portmgr 
Responsible-Changed-By: okazaki 
Responsible-Changed-When: Sun Sep 22 19:09:39 PDT 2002 
Responsible-Changed-Why:  
Over to the MAINTAINER. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=43273 
State-Changed-From-To: open->feedback 
State-Changed-By: will 
State-Changed-When: Wed Oct 29 15:40:44 PST 2003 
State-Changed-Why:  
Sorry for the (really really) late response. 

I think it would be better to develop something similar to this: 
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/25815 

i.e. a patch to lock targets that are in process.  This would solve 
the problem in your PR, I believe. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=43273 

From: Edwin Groothuis <edwin@mavetju.org>
To: Will Andrews <will@FreeBSD.org>
Cc: FreeBSD Gnats Submit <freebsd-gnats-submit@freebsd.org>
Subject: Re: ports/43273: [patch] make bsd.port.mk checksum check aware of multiple people fetching/compiling
Date: Thu, 30 Oct 2003 11:10:15 +1100

 > I think it would be better to develop something similar to this:
 > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/25815
 > 
 > i.e. a patch to lock targets that are in process.  This would solve
 > the problem in your PR, I believe.
 
 The PR mentioned does prevent the same dependency port to be build
 at the same time, but it doesn't prevent two people downloading the
 file at the same time. Even worse, with todays autoresume I have
 no idea how to the story will end.
 
 Edwin
 -- 
 Edwin Groothuis      |            Personal website: http://www.mavetju.org
 edwin@mavetju.org    |    Weblog: http://www.mavetju.org/weblog/weblog.php 

From: Will Andrews <will@csociety.org>
To: FreeBSD GNATS DB <FreeBSD-gnats-submit@FreeBSD.org>
Cc: Edwin Groothuis <edwin@mavetju.org>
Subject: Re: ports/43273: [patch] make bsd.port.mk checksum check aware of multiple people fetching/compiling
Date: Wed, 29 Oct 2003 18:49:16 -0800

 On Wed, Oct 29, 2003 at 06:23:55PM -0800, Kris Kennaway wrote:
 > I really don't know if this is something we want to support in the
 > ports collection.  It's not a common situation, and it adds complexity
 > to the ports system.
 
 What Edwin's talking about is indeed uncommon.  But one would
 think that if one starts installing two ports, they will both
 complete even if they depend on similar things and happen to
 cross paths during the build.  That was Alfred's problem.
 Lockfiles are a good solution for that.
 
 Oh.  Lockfiles don't work between users, do they?
 
 I guess the only solution in that case is to make people use a
 particular account for ports installs.  Say, root.  8)
 
 Shall I close this PR?
 
 Regards,
 -- 
 wca
State-Changed-From-To: feedback->suspended 
State-Changed-By: kris 
State-Changed-When: Fri Oct 15 23:25:33 GMT 2004 
State-Changed-Why:  


http://www.freebsd.org/cgi/query-pr.cgi?pr=43273 
State-Changed-From-To: suspended->closed 
State-Changed-By: krion 
State-Changed-When: Fri Mar 18 22:57:08 GMT 2005 
State-Changed-Why:  
Provided patch does not resolve conflicts and can 
only warn users.  The real solution would be more  
difficult to implement.  Submitter agrees with closing 
this PR.  Thanks. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=43273 
>Unformatted:
