From rea-fbsd@codelabs.ru  Sun Apr  8 19:15:32 2007
Return-Path: <rea-fbsd@codelabs.ru>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 77E1916A400
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  8 Apr 2007 19:15:32 +0000 (UTC)
	(envelope-from rea-fbsd@codelabs.ru)
Received: from pobox.codelabs.ru (pobox.codelabs.ru [144.206.177.45])
	by mx1.freebsd.org (Postfix) with ESMTP id EE5DB13C45D
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  8 Apr 2007 19:15:31 +0000 (UTC)
	(envelope-from rea-fbsd@codelabs.ru)
Received: from localdomain
	by pobox.codelabs.ru with local
	id 1HacrI-0003q5-Fi; Sun, 08 Apr 2007 23:15:28 +0400
Message-Id: <E1HacrI-0003q5-Fi@pobox.codelabs.ru>
Date: Sun, 08 Apr 2007 23:15:28 +0400
From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
Sender: rea-fbsd@codelabs.ru
Reply-To: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc: fjoe@freebsd.org, harti@freebsd.org
Subject: prevent infinite loops for make's "Remaking Makefiles" feature
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         111387
>Category:       bin
>Synopsis:       [patch] prevent infinite loops for make(1) "Remaking Makefiles" feature
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    fjoe
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Apr 08 19:20:01 GMT 2007
>Closed-Date:    Fri Apr 20 06:38:23 GMT 2007
>Last-Modified:  Fri Apr 20 06:38:23 GMT 2007
>Originator:     Eygene Ryabinkin
>Release:        FreeBSD 7.0-CURRENT i386
>Organization:
Code Labs
>Environment:
System: FreeBSD XXX 7.0-CURRENT FreeBSD 7.0-CURRENT #10: Sat Mar 31 16:23:39 MSD 2007 root@XXX:/usr/obj/usr/src/sys/XXX i386
>Description:

After the new make's "Remaking Makefiles" feature was introduced I
started to see the ports that were unable to build properly and
make was terminating with the "Max recursion level (500) exceeded".
The investigation showed that the problem was in the Makefile targets
that were considered out-of-date and had some rules to build them,
but the actual rules were not touching the target in any way.

>How-To-Repeat:

The following shell script will show the essence of the problem.
--- create-test.sh begins here ---
#!/bin/sh

echo -n "Creating test files: Makefile and Makefile.in..."
cat > Makefile << "EOF"
all:	Makefile

Makefile: Makefile.in
	echo "Wow, wow! I am the dummy target."
EOF

touch Makefile
sleep 2
touch Makefile.in
echo "OK"
echo
echo "OK, spawning 'make'"
make
--- create-test.sh ends here ---

Additionally, you can try to build the mail/mutt-ng port on the
7-CURRENT using the tree with src/usr.bin/make/main.c >= 1.161
(other parts of the "Remaking Makefiles" feature should be present
too, but if you have the consistent tree with the aforementioned
main.c -- you got the buggy code).

>Fix:

I've been straight: just make two 'stat' calls before and after the
Makefile build and compare the whole 'struct stat' contents.  If
something has changed -- we're considering the target to be really
rebuilt and will try to restart make. This helped me to overcome
the trouble with my simple example as well as to build the mail/mutt-ng
and the recent development version of cURL that had this issue too.

The patch is below.
--- main.c.patch begins here ---
--- usr.bin/make/main.c.orig	Sun Apr  8 22:13:46 2007
+++ usr.bin/make/main.c	Sun Apr  8 22:42:52 2007
@@ -62,6 +62,7 @@
  */
 
 #include <sys/param.h>
+#include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/sysctl.h>
 #include <sys/time.h>
@@ -694,6 +695,7 @@
 	LstNode *ln;
 	int error_cnt = 0;
 	int remade_cnt = 0;
+	struct stat pre_sb, post_sb;
 
 	Compat_InstallSignalHandlers();
 
@@ -767,7 +769,11 @@
 		/*
 		 * Check and remake the makefile
 		 */
+		if (stat(gn->name, &pre_sb) != 0)
+			memset(&pre_sb, 0, sizeof(pre_sb));
 		Compat_Make(gn, gn);
+		if (stat(gn->name, &post_sb) != 0)
+			memset(&post_sb, 0, sizeof(post_sb));
 
 		/*
 		 * Restore -t, -q and -n behaviour
@@ -784,8 +790,20 @@
 		 *	ERROR	  An error occurred while gn was being created
 		 *	ABORTED	  gn was not remade because one of its inferiors
 		 *		  could not be made due to errors.
+		 *
+		 * We are checking if file's 'struct stat' was changed,
+		 * since we do not want to restart the process if there
+		 * were some makefile(s) that was not actually remade --
+		 * this can cause loops. The simplest example is:
+		 * -----
+		 * Makefile: Makefile.in
+		 * 	echo "Yo, all is done, no need to worry."
+		 * -----
+		 * and the Makefile is out-of-date.
 		 */
-		if (gn->made == MADE)
+		if (gn->made == MADE &&
+		    memcmp((const void*)&pre_sb, (const void*)&post_sb,
+		    sizeof(pre_sb)) != 0)
 			remade_cnt++;
 		else if (gn->made == ERROR)
 			error_cnt++;
--- main.c.patch ends here ---
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->fjoe 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun Apr 8 21:58:23 UTC 2007 
Responsible-Changed-Why:  
Max, since you did the commit, can you please take a look at this? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111387 

From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: bin/111387: prevent infinite loops for make's "Remaking Makefiles" feature
Date: Sat, 14 Apr 2007 14:28:28 +0400

 Just for the record: I've successfully recompiled all my ports
 (total number is 389, though I am not sure how much of them are
 using pmake and rebuilding their Makefile components on the fly)
 with the patched version of make. No problems were seen.
 -- 
 Eygene
State-Changed-From-To: open->closed 
State-Changed-By: fjoe 
State-Changed-When: Fri Apr 20 06:34:54 UTC 2007 
State-Changed-Why:  
I had slightly different patch (do not consider targets out-of-date 
if it has no commands and no chidlren were actually remade, 
GNU make behaves this way), but it was rather intrusive, so I committed a 
variation of your patch. By the way, the example you mentioned 
in the comments is not correct (see my commit message). 

Anyway, thanks for the idea! PR can now be closed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111387 
>Unformatted:
