CVS log for src/usr.bin/make/make.h

Up to [cvs.NetBSD.org] / src / usr.bin / make

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

make: reduce line length in error messages

The error messages for deeply nested parse errors were hard to decipher,
due to the large amount of text.  Split these messages into individual
lines, just as in the backtrace for .include files and .for loops.  This
unified backtrace makes the output more uniform.

make: treat recursive variables non-fatally

A recursive variable is no worse than an unknown modifier, so treat them
in the same way by continuing parsing until the end of the makefile.

make: add more context to error message about recursive variables

Compat_RunCommand use tempfile if cmd too big

Extract the logic recently added to Cmd_Exec to handle
long commands via temp file to Cmd_Argv,
so it can also be leveraged by Compat_RunCommand

Reviewed by: christos

make: error out on parse/evaluation errors in shell commands

The expression ${VAR:X} has an unknown modifier ':X'.  Previously, this
expression errored out when the expression was evaluated at parse time,
but not when the expression was evaluated when generating the commands
to bring a target up to date.  The errors were previously reported, they
didn't affect the exit status, though.

Now, errors in expressions are handled in the same way, regardless of
the time at which they are evaluated.

make: move initialization of variable scopes to targ.c

The variable scopes are freed by Targ_End, so initialize them there as
well.  Separate printing statistics and freeing memory, which makes
Var_End unnecessary.

make: only generate code for cleanup functions in CLEANUP mode

make: reduce lint-specific comments about ARGSUSED

make: clean up collection of context information for error messages

make: sync VarEvalMode constant names with their debug log names

make: add .export-all

An explicit syntax for exporting all global variables is much safer
than allowing .export with no argument to do the same.

Add .export-all and have .export with no argument throw a warning saying
to use .export-all

Reviewed by: rillig

make: remove unused VARE_EVAL_KEEP_DOLLAR

make: fix memory leak for command strings

make: fix a few more memory leaks

make: all command line overrides go in .MAKEOVERRIDES

Not all variables that start with '.' are internals,
and unless they are explicitly flagged as internal should go
into .MAKEOVERRIDES

Update varname-dot-makeoverrides to check this.

Also avoid using SCOPE_CMDLINE when ReadOnly will do.

make: clean up, test .NOPATH

Trim down the comments in the archive module, as they mainly repeated
the code.  Trim down the binary code size in the archive module, as it
is rarely used.

In Var_Parse, delay two variable assignments until they are actually
needed.

make: clean up comments, code and tests

make: provide more context information for parse/evaluate errors

make: record exit status in GNode

SetErrorVars can now set .ERROR_EXIT which allows
a .ERROR target to ignore the case of .ERROR_EXIT == 6
which means failure happened elsewhere.

Reviewed by:

make: export target scope values

Pass target scope to Var_ReexportVars so that a target process
will see the correct values in its env.
We must then mark any Global scope variable as unexported
so targets without local value get the Global one.

make: clean up unused return value of str2Lst_Append

No functional change.

make: inline a single-line cross-file function

No functional change.

make: remove redundant 'extern' from function declaration

make: remove redundant 'extern' in function declaration

No binary change.

make: allow targets to be used as multiple-inclusion guards

This style is used by FreeBSD, among others.

make: if a makefile is protected by a guard, only include it once

"looks reasonable" sjg@

make: remove parameter names from function declarations

No binary change.

make: shorten function names, clean up comments

No functional change.

make: declare all common symbols in headers, unexport others

No functional change.

make: document how read-only variables can be made read-write

make: fix parsing of unevaluated subexpressions with unbalanced '{}'

Since var.c 1.323 from 2020-07-26, modifiers containing unbalanced
braces or parentheses were parsed differently, depending on whether they
were relevant or not.

For example, the expression '${VAR:...}' is enclosed with braces. When
this expression has a modifier ':S,},}},g' that would double each '}' in
that expression, the parser got confused:

If the expression was relevant, the modifier was parsed as usual, taking
into account that the 3 '}' in the modifier are ordinary characters.

If the expression was irrelevant, the parser only counted the '{' and
the '}', without taking into account that a '}' might be escaped by a
'\' or be an ordinary character.  Parsing therefore stopped at the first
'}', assuming it would finish the expression '${VAR:S,}'.

This parsing mode of only counting balanced '{' and '}' makes sense for
the modifier ':@var@...@', which expands each word of the expression
using the template from the '...'.  These templates tend to be simple
enough that counting the '{' and '}' suffices.

make: inline macros for variable names

The variable name '.MAKEOVERRIDES' was already used in the non-macro
form.

No binary change.

make: inline macro for variable name ".MAKE.EXPORTED"

The variable name is distinctive enough to be searched directly in the
code instead of having a named constant for it.

No binary change.

make: remove redundant type VarParseResult

No functional change.

make: clean up calls to Var_Subst

None of the calls to Var_Subst used the return value, and the return
value was always VPR_OK.

No functional change.

make: reduce complexity of evaluating expressions

No functional change.

make: some variables should be read-only

Make variables like .newline and .MAKE.{GID,PID,PPID,UID} read-only.

Reviewed by: rillig

make: .[NO]READONLY for control of read-only variables

Reviewed by: rillig

make: inline macro for variable name

This fixes the inconsistency of using the macro name in one place and
its value in another place (since 2010).

No binary change.

make: change return type of unlink_file back to int

As unlink_file is a wrapper around unlink, use the same encoding for the
possible return values as in the wrapped function.  This consistency is
more important than expressing all possible return values in the return
type 'bool'.

https://mail-index.netbsd.org/tech-toolchain/2022/10/06/msg004155.html

No functional change.

make: clean up tracking of depth of nested .if directives

The variable cond_min_depth was redundant.  It was only accessed while
parsing the makefiles.  Merging it into struct IncludedFile removes the
possible confusion between cond_min_depth and including_cond_min_depth.

No functional change.

make: fix variable and function names relating to .if nesting

The previous names were confusing since they suggested that cond_depth
instead of cond_min_depth would be saved and restored.

No functional change.

Cond_reset_depth just use cond_min_depth

To avoid errors from unclosed conditionals on .break
it is sufficient to just set cond_depth = cond_min_depth.

Patch from rillig

make: add .break to terminate .for loop early

When .break is encountered within a .for loop
it causes immediate termination.

Outside of a .for loop .break causes a parse error.

Reviewed by: christos

make: reorganize Parse_Error

Determining the location where the error occurred is now done by
ParseVErrorInternal.  This frees the remaining code from keeping the
filename and the line number together.  It also makes Parse_Error short
enough that it might be worth providing a separate function for each of
the 3 log levels.

No functional change.

make: allow to randomize build order of targets

In complex dependency structures, when a build fails, a probable cause
is a missing dependency declaration between some files.  In compat mode,
the build order is deterministic, in jobs mode, it is somewhat
deterministic.  To explore more edge cases, add the line ".MAKE.MODE +=
randomize-targets" somewhere in the makefile.

Fixes PR bin/45226 by riastradh.  Reviewed by christos.

make: rename Compat_Run to Compat_MakeAll

No functional change.

make: only switch to POSIX mode if '.POSIX:' is the first line

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
says that in order to make a makefile POSIX-conforming, its first
non-comment line must be the special dependency line '.POSIX:' without
any source dependencies.

Previously, make switched to POSIX mode even if such a line occurred
anywhere else, which was allowed by POSIX but was deep in the
"unspecified behavior" area.  For NetBSD make, there is no big
difference since it doesn't ship any <posix.mk> file, this change mainly
affects the bmake distribution.

Previously, makefiles that contain '.POSIX:' somewhere in the middle
could fail due to <posix.mk> resetting .SUFFIXES, among other things.

Suggested by Simon J. Gerraty, who also reviewed an earlier version of
this change.

make: prefer 'long long' over 'long' on 32-bit C99 platforms

When sorting the words of an expression numerically using the modifier
':On' (added on 2021-07-30), use 64-bit numbers even on 32-bit
platforms.  A typical use case is comparing file sizes.

When tracing the execution of jobs, fix an integer overflow after 2038.
32-bit platforms that use a pre-C99 compiler still have this problem.

No change to the test suite since most tests simply skip any potential
differences between 32-bit platforms and 64-bit platforms (see
varmod-order-numeric.mk) or already account for both variants (see
varmod-localtime.mk).

make: improve C90 support

Do not use inline functions, remove trailing comma in enum declaration,
do not use 'long long' for printing a timestamp.  This re-introduces the
Year 2038 Problem for pre-C99 compilers when printing the trace log, but
that is a seldom used feature.

make: use unsigned int for line numbers everywhere

Previously, some line numbers were stored as signed int while others
were stored as size_t.  Since line numbers are never negative, use an
unsigned type.  Since the maximum file size for makefiles is 1 GB (see
loadfile), unsigned int is large enough even on 64-bit platforms.

Using a single data types reduces the number of type conversions.  Using
unsigned int improves compatibility with C90 (printf %u instead of %zu),
which is needed by bmake, which is derived from usr.bin/make.

No functional change.

make: make DEBUG0 simpler

All arguments to DEBUG0 are string literals, and none of them contains a
'%', which makes them safe to pass to printf directly.  Any accidental
'%' would be caught by the compiler.

No functional change.

Make the GNode lineno unsigned to fix lint warning in var.c calling
PrintLocation()

make: for recursive variables in commands, print location

Print the approximate location based on the last command that has been
defined for the target.  It would be possible to get more detailed
location information by counting the number of commands of the target,
but that would get messy due to .USEBEFORE, .USE and .DEFAULT, and
still, this is an edge case, so don't waste too much code for it now.
Having this hint about the location is more helpful than just a plain
"Variable X is recursive" without any further details.

make: print stack trace on fatal errors

The only fatal error that occurs while the makefiles are read in is the
one about recursive variables, which didn't give any hint about the
location before.

If a recursive variable is detected while evaluating the commands of a
target to be made, there is no location information, as before.

make: inline functions that have a single call site

No functional change.

Allow local variable assignments in dependency lines

The variable is set in the context of the target.
This syntax has been supported by gmake for ~ever.
If necessary a makefile can set .MAKE.TARGET_LOCAL_VARIABLES=false
to disable this.

Expose GetBooleanExpr so parse.c can use it.

make: merge nonints.h into make.h

No functional change.

make: use islower for parsing directives

None of the directives has an uppercase letter, so there is no need to
test for it.

No functional change.

make: extract low-level character operations into utility function

Suggested by nia.
https://mail-index.netbsd.org/source-changes-d/2022/01/09/msg013564.html

No functional change.

make: merge duplicate variables for the main target

No functional change.

make: rename and inline Targ_Precious

No functional change.

make: rename IFile and its fields to match their actual content

For lines that use backslash continuation, the human-readable line
number does not equal the number of raw lines that have been read from
the file.

The big comment in PrintStackTrace has become outdated, it still
referred to first_lineno.  Due to the bugs documented in
opt-debug-parse.mk, that function needs to be redone completely.

No functional change.

make: merge duplicate types CondEvalResult and CondResult

No binary change.

make: merge duplicate enum constants for CondEvalResult and CondResult

No binary change.

make: merge types CondResult and CondEvalResult

No functional change.

make: do not treat .USEBEFORE as candidate for the main target

A .USE target is not a candidate, so .USEBEFORE shouldn't either.

Since make.h 1.36 from 2001-07-03.  In that commit, OP_USEBEFORE should
have been added to OP_NOTARGET.

make: extract OP_NOTARGET into separate function

No binary change, except for line numbers in assertions.

make: remove unnecessary words from command line options

Several years ago, the command line options were individual global
variables.  The global variable could therefore not be named 'silent'
since that would have conflicted with local variables of the same name.
After moving the global variable to the namespace 'struct CmdOpts',
there is no conflict anymore.

There doesn't seem to be any risk of naming collisions for the names
'touch' and 'query'.

No functional change.

make: amend leftover cleanups from the previous commits

No functional change.

make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.

make: remove redundant comments for multiple-inclusion guards

make: remove space after ':' in bit-field declarations

As seen in /usr/share/misc/style.

make: mark several functions whose result must be used

Suggested by sjg, to catch more bugs like the memory leak in cond.c
1.303 from 2021-12-13.

No binary change.

make: prevent memory leaks from buffers

The warning about unused function results would have prevented the
memory leak that was fixed in cond.c 1.303 from 2021-12-13.

make: convert debugging flags from enum to bit-field

This gets rid of the magic numbers, making it possible to add another
debug flag without renumbering the others.

No functional change.

make: fix memory leak for filenames in .for loops (since 2013-06-18)

Previously, each time a .for directive pushed its buffer on the input
file stack, the current filename was duplicated.  This was a waste of
memory.

The name of a file is typically only used while it is read in.  There is
one situation when the filename is needed for longer, which is when a
target is defined.

Since .for loops are implemented as a special form of included files,
each .for loop duplicated the current filename as well.

$ cat << EOF > for.mk
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor

all:
	@ps -o rsz -p ${.MAKE.PID}
EOF

$ make-2021.12.13.03.55.16 -r -f for.mk
  RSZ
10720

$ ./make -r -f for.mk
 RSZ
1716

The difference is 8 MB, which amounts to 1 million .for loops.

make: fix a few lint warnings about type mismatch in enum comparisons

These warnings were triggered with the lint flag '-e', which enables
additional checks on enums.  This check would have detected the type
mismatch from the previous commit.

The check has a few strange warnings though, complaining about
initialization of 'unsigned long' with 'unsigned long', so don't enable
it for the official builds.

No functional change.

make: fix leftover typo from previous refactoring

make: convert GNodeFlags from enum into bit-fields

Now that Enum_ToString is implemented for each type separately, it's
easy to convert them to bit-fields.  This gets rid of the magic numbers
12 for CYCLE and 13 for DONECYCLE that left a suspicious gap in the
numbers.  This gap was not needed since the code didn't make use of the
relative ordering of the enum constants.

The effects of this conversion are fewer capital letters in the code,
smaller scope for the GNode flags, and clearer code especially when
setting a flag back to false.

One strange thing is that GCC 10.3.0 doesn't optimize GNodeFlags_IsNone
to an single bitmasking instruction, at least on x86_64.  Instead it
generates a testb instruction for each of the flags, even loading bit 8
separately from the others.  Clang 12.0.1 knows this optimization
though and generates the obvious sequence of movzwl, testl, jz.

No functional change.

make: replace bloated bit-set-to-string code with simple code

It was a nice idea to implement a bit-set using an enum type and have a
generic ToString function for them.  In the end, the implementation
involved really heavy preprocessor magic and was probably difficult to
understand.  Replace all the code with a few bits of straight-forward
preprocessor magic that can be readily understood by just looking 5
lines around, instead of digging through 130 lines of lengthy macro
definitions.

Curiously, this reduces the binary size even though the 3 ToString
functions now have a few lines of duplicate code and there are more
explicit function calls.

The ToString functions are only seldom used, so the additional memory
allocation is acceptable.

No functional change.

make: remove CONSTCOND comments, lint no longer needs them

make: error out if a pre-C99 platform defines bool in some cases

On NetBSD/amd64 9.99.88, when compiling make in C90 mode, make.h defined
its own boolean type as an alias for unsigned int.  Not plain int since
that would make the value of bit-fields -1 instead of 1.

This worked fine for all files except main.c, which includes
<sys/sysctl.h>, which in turn includes <stdbool.h> unconditionally, even
in C90 mode.  This meant that in main.c, sizeof(bool) was 1, while in
all other files it was 4.

This in turn led to a segmentation fault when ParseDependencySourceMain
tried to access opts.create.  Since parse.c assumed sizeof(bool) == 4,
it computed an offset outside of struct CmdOpts, which was defined in
main.c with sizeof(bool) == 1.

Rather than risking these segmentation faults, prevent building make on
platforms like these and suggest a proper workaround.

make: fix lint warnings

The string functions from str.h are declared as 'static __unused' when
compiled with GCC, but lint explicitly undefines __GCC__ during
preprocessing.  Therefore, make those functions inline, to prevent
warnings that they are unused.

The macro UNCONST is used in a few places, and (again) since lint
undefines __GCC__, that macro expanded to a simple type cast, which lint
warned about.  To prevent this warning, implement UNCONST as a function
that works everywhere and hides the type cast.

In filemon_open, the code for closing F->in was obviously unreachable.

No functional change.

make: document where to find tests for the dependency lines

Pull up following revision(s) (requested by mrg in ticket #1678):

	bin/pax/options.c: revision 1.119
	usr.bin/config/scan.l: revision 1.32
	usr.bin/make/main.c: revision 1.274
	bin/pax/extern.h: revision 1.60
	usr.bin/config/defs.h: revision 1.106
	usr.bin/make/make.h: revision 1.105
	sbin/newfs_udf/udf_create.c: revision 1.27 (patch)
	sbin/newfs_udf/newfs_udf.h: revision 1.7 (patch)
	usr.bin/config/main.c: revision 1.100
	usr.sbin/installboot/Makefile:	revisions 1.53,1.54 (patch)
	usr.sbin/installboot/installboot.h: revision 1.41 (patch)
	usr.sbin/installboot/machines.c: revisisons 1.41,1.42 (patch)

Avoid depending on common symbols.

Pull up following revision(s) (requested by mrg in ticket #1263):

	bin/pax/options.c: revision 1.119
	usr.bin/config/scan.l: revision 1.32
	usr.bin/make/main.c: revision 1.274
	bin/pax/extern.h: revision 1.60
	usr.bin/config/defs.h: revision 1.106
	usr.bin/make/make.h: revision 1.105
	sbin/newfs_udf/udf_create.c: revision 1.27 (patch)
	sbin/newfs_udf/newfs_udf.h: revision 1.7 (patch)
	usr.bin/config/main.c: revision 1.100

Avoid depending on common symbols.

make: let the compiler decide whether to inline string functions

On x86_64, this reduces the binary size by 2 kB.

make: add types Substring and LazyBuf

These will be used for making the string handling more efficient,
avoiding allocations, especially when evaluating variable expressions.

Since the string handling has grown quite a bit in the last months,
extract it into its own header file.

No functional change.

make: rename a few functions to be more descriptive

No functional change.

make: revert accidental change from the previous commit

The definition of MAKE_GNUC_PREREQ was not supposed to be changed.  This
change only slipped accidentally, the test for __STDC_VERSION__ was only
supposed to be for UNCONST.

make: backport to C90

In the past few months I had accidentally used C99 features in the make
code.  According to tools/README, tools that are used in the build
system should restrict themselves to C90.

This allows make to build with GCC's options "-pedantic
-Wno-system-headers -Dinline= -Wno-error=cast-qual".

I didn't notice anyone actively complaining though, I just wanted to see
how much work this backporting would be.  The identifier __func__ is
still used, as in other tools.

No functional change.

make: use C99 bool type instead of defining its own

No functional change.

Avoid strdup in mkTempFile

Require caller to pass a buffer and size if they
want the tempfile not unlinked.

Add Job_TempFile to handle blocking signals around
call to mkTempFile, so that meta_open_filemon can use it
in jobs mode.

make: improve documentation about variable scopes

In an experiment, I tried to separate the concepts of a GNode and a
variable scope.  The global variables SCOPE_GLOBAL, SCOPE_INTERNAL and
SCOPE_CMDLINE are implemented as GNode even though they only need the
members 'name' and 'vars'.  All their other members are unused.
Therefore it seemed natural to extract this part of the GNode into a
separate type called Scope.

The resulting code was harder to read though since it had split the
namespace of the functions into several parts that were not obviously
related: The Var_ functions, the Scope_ functions, and the short-cut
Global_ functions.  Because of this, I threw away the experiment.

All that is left are a few updated comments.

make: rename Var_ValueDirect to GNode_ValueDirect

make: rename context and ctxt to scope

This continues the previous commit, in which VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE were renamed.

Renaming the variable 'ctxt' was trivial since that word is used nowhere
else.  In the comments though, each occurrence of the word 'context' had
to be checked individually since the word 'context' was not only used
for referring to a variable scope.  It is also used to distinguish
different situations where characters are escaped in a certain way
('parsing context') and in a few other expressions.

make: rename some VAR constants to SCOPE

The word "context" does not fit perfectly to the variables that are
associate with a GNode, as the context is usually something from the
outside and the variables are more like properties inherent to the
GNode.

The term "global context" fits even less.  Since the thing where
variables are looked up is commonly named a scope, use that term
instead.

This commit only renames the global variables VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE, plus a few very closely related comments.  These are:

	GNode.vars (because of line breaks)
	GNode_Free (dito)
	varname-make_print_var_on_error.mk
	varname-make_print_var_on_error-jobs.mk

The debug message in Var_Stats is left as-is since there is no unit test
for it yet.

The other renamings (variable names "context", "ctxt", as well as
further comments) will be done in a follow-up commit.

make: remove unused INTERNAL flag

It had been used for cached_realpaths, until this variable had its type
changed from GNode to HashTable in main.c 1.469 from 2020-11-14.

make: clean up comments

make: indent preprocessor directives consistently

As seen in share/misc/style.

make: always use vfork, never fork

Before compat.c 1.217, job.c 1.390 and main.c 1.504 from 2020-12-27, the
exported make variables were exported from each freshly forked child
process.  There was no practical difference though between exporting the
variables from the parent process or the child process since these two
processes share the same address space, except that the forked process
is very limited in what it may actually do.  This limitation was
violated on a regular basis.

When an exported variable referred to a variable that used the :sh
variable modifier, this led to a fork from within vfork, which is not
allowed.  Since 2020-12-27, exporting the variables is done from the
main process, which prevents this situation from ever occurring.

Since that day, there is no need anymore to distinguish between vfork
and fork, which removes any need for the macro.

make: use bit shifts in enum constants for GNodeFlags

Same as in the other enum bit sets.  This makes it easier to spot the
gap between bit 6 and bit 12.

make(1): convert SearchPath to struct

This prepares for making dotLast a simple struct member instead of a
fake CachedDir, which is easier to understand.

make(1): merge duplicate code in Parse_MainName

make(1): remove do-not-format markers from comments

These markers had been used inconsistently.  Furthermore the source code
had not been formatted automatically before 2020 at all, otherwise there
wouldn't have been any trailing whitespace left.

make(1): fix a few inconsistencies for lint's strict bool mode

make(1): consistently use boolean expressions in conditions

Most of the make code already followed the style of explicitly writing
(ptr != NULL) instead of the shorter (ptr) in conditions.

The remaining 50 instances have been found by an experimental,
unpublished check in lint(1) that treats bool expressions as
incompatible to any other scalar type, just as in Java, C#, Pascal and
several other languages.

The only unsafe operation on Boolean that is left over is (flags &
FLAG), for an enum implementing a bit set.  If Boolean is an ordinary
integer type (the default), some high bits may get lost.  But if Boolean
is the same as _Bool (by compiling with -DUSE_C99_BOOLEAN), C99 6.3.1.2
defines that a conversion from any scalar to the type _Bool acts as a
comparison to 0, which cannot lose any bits.

make(1): format multi-line comments

make(1): replace global preserveUndefined with VARE_KEEP_UNDEF

Controlling the expansion of variable expressions using a global
variable and a VARE flag was inconsistent.

Converting the global variable into a flag had to prerequisites:

1.  The unintended duplicate variable assignment had to be fixed, as
done in parse.c 1.520 from 2020-12-27.  Without this fix, it would have
been necessary to add more flags to Var_Exists and Var_SetWithFlags, and
this would have become too complex.

2.  There had to be a unit test demonstrating that VARE_KEEP_DOLLAR only
applies to the top-level expression and is not passed to the
subexpressions, while VARE_KEEP_UNDEF applies to all subexpressions as
well.  This test is in var-op-expand.mk 1.10 from 2020-12-28, at least
for the ':@word@' modifier.  In ParseModifierPartSubst, VARE_KEEP_UNDEF
is not passed down either, in the same way.

make(1): fix lint warnings for constant condition in DEBUG calls

make(1): fix MAKE_RCSID for lint mode

Previously, running lint mode didn't define MAKE_RCSID at all, which
resulted in a syntax error.

While here, reduced the indentation and nesting of the preprocessor
directives.

make(1): rename CmdOpts.lint to strict

When running lint(1) on the code, it defines the preprocessor macro
"lint" to 1, which generated a syntax error in the declaration "Boolean
lint", as that became "Boolean 1".

make(1): fix return type of macro DEBUG

This macro was supposed to return a boolean expression all the time, it
just hadn't been implemented this way.  This resulted in wrong output
for the test sh-flags, in compilation modes -DUSE_UCHAR_BOOLEAN and
-DUSE_CHAR_BOOLEAN, since in ParseCommandFlags, the expression
DEBUG(LOUD) didn't fit into a boolean.

make(1): support using C99 bool for Boolean

make(1): constify progname

make(1): clean up comments for command line options

make(1): clean up macros for debug logging

Using a do-while loop prevents compiler warnings about possible dangling
else.  It also removes the unnecessary negation.

make(1): define constants for enum zero-values

make(1): use space instead of tab for preprocessor directives

make(1): reduce memory allocation for dirSearchPath

make(1): reduce memory allocation for CmdOpts.create

make(1): reduce memory allocation in CmdOpts.variables

make(1): reduce memory allocation in CmdOpts.makefiles

make(1): reduce pointer indirection for GNode.implicitParents

make(1): reduce pointer indirection for GNode.cohorts

make(1): reduce pointer indirection for GNode.order_pred and order_succ

make(1): reduce memory allocation for GNode.parents and GNode.children

make(1): remove pointer indirection from GNode.commands

Just to save a few memory allocations.  No noticeable effect on the
performance though.

make(1): allow multiple RCS IDs in all-in-one mode

make(1): indent make.h with tabs instead of spaces

make(1): add high-level API for GNode.made

Having an enum whose constants must be ordered in a certain way may be
unexpected to casual readers.  Hide this implementation detail in
separate functions.

make(1): fix error handling for dependency of .END in -k mode

Fix one bug, find 4 new ones.  All these bugs have been around since
2005-05-08, when dependencies on the .BEGIN, .END and .INTERRUPT nodes
were implemented.  Before that, checking gn->made == ERROR was
appropriate, but adding the dependencies made ABORTED a new possible
error value from Compat_Make.

make(1): document the enum GNodeMade

Given only the state names and their individual documentation, it is
hard to see the full picture.  To make this easier, provide typical
examples of the ways that a GNode takes through these states.

make(1): align end-of-line comments with tabs

make(1): add more debugging for searching transformation rules

make(1): renumber the flags in GNodeType

In plain C, bit 31 of an enum is not guaranteed to play well with binary
operators.  Bit 23 was unused until now, so shift them all down.

The order of the bits is only used in Targ_PrintType.

make(1): rename type Suff to Suffix

This avoids confusion with the module suff.c of the same name.

The existing naming convention is that the module's public functions are
named Suff_Func and its private functions are named SuffFunc.  This
collides with the naming convention that a the functions of the type Suff
are called Suff_Func as well.

To resolve this collision, rename Suff to Suffix.  The related types like
SuffList and the affected functions like SuffRef will be renamed in a
follow-up commit.

make(1): rename GNode.context to vars

Especially in the variables module, the expression 'ctxt->context' looked
redundant.  Having a GNode as a context sounds ok, but a context of a
context just doesn't make sense.

make(1): clean up and extend comments in targ.c

make(1): clean up make.h, meta.c, parse.c, str.c

The main changes are in the comments, which have been shortened and
corrected.

Some local variables changed their names.

In ParseErrorInternal, the scope of va_start is now narrower.

In ParseDoDependency, the type of tOp has been fixed.

ParseGetLine doesn't take flags anymore but instead a parsing mode.
Previously, the flags had not been combined anyway.

At the beginning of Parse_File, fatals is already guaranteed to be 0, and
even if not, it would be wrong to just discard the fatal errors.

make(1): remove redundant parameter from str2Lst_Append

make(1): rename DEFAULT to defaultNode

In C, uppercase names are typically used for constants, and this is not
a constant.

Pass a writable flag to Main_SetObjdir to control writable check

For curdir and an explicit .OBJDIR target, we allow for
the directory to be read-only.
During InitObjdir we otherwise default to requiring objdir to be
writable - this can be controlled by env variable
MAKE_OBJDIR_CHECK_WRITABLE

Add unit-tests/objdir-writable

Reviewed by: christos rillig

make(1): use consistent definition for MAKE_INLINE

make(1): move lint option away from the debug flags

The lint option affects much more than a typical debug flag.  Not only
does it produce additional diagnostics, it also changes the control flow
and fixes the traditional error handling in a few places.

No functional change.

make(1): sort debug flags alphabetically

DEBUG_FOR had been accidentally left at the end of the list.

make(1): fix documentation of preserveUndefined

Since it is a global variable and not a VarEvalFlags, it affects nested
subexpressions as well, and there is no way to escape it.

make(1): fix debug output in out-of-date for cohorts (since 2003-11-14)

Before, a wrong cause for being out-of-date was printed in the debug log,
for optional cohorts.  This was caused by having the same conditions
duplicated in the code, instead of putting them in a separate function.

Now the optional cohort is correctly identified as using the '::'
dependency operator.

make(1): rename Make_OODate to GNode_IsOODate

It doesn't matter which of the make modules is in charge of determining
whether a node is out-of-date.  Therefore, remove the module name from
the function name.

make(1): rename Make_TimeStamp to GNode_UpdateYoungestChild

make(1): fix type mismatch between int and PrintVarsMode

make(1): add pp_skip_hspace to skip horizontal whitespace during parsing

make(1): merge duplicate code for skipping horizontal whitespace

make(1): move name of inline functions to the beginning of the line

make(1): rename dieQuietly to shouldDieQuietly

It was too confusing to have a function named die that doesn't actually
die.  Plus, the return type int didn't give any clue about what the
function actually returns.

make(1): document DEBUG_LINT

make(1): sort and renumber debug flags

make(1): document progname

make(1): clean up comments in make.h

make(1): extend documentation for GNodeType constants

make(1): group GNode fields by topic, document ParseErrorLevel

make(1): align comments on a few commonly used variables

make(1): move declaration of parseIncPath from main.c to make.h

make(1): rename MAKEFILE_PREFERENCE for consistency

The names of the other special variables are all prefixed with MAKE_.

make(1): remove unused names DTARGET, FPREFIX, etc.

make(1): negate discardUndefined to preserveUndefined

make(1): rename oldVars to discardUndefined

While here, moved all the documentation about this variable into a
single place.

make(1): add missing enum tag for GNodeMade

make(1): document that "old-style" variables are older than 1993

make(1): negate NoExecute to GNode_ShouldExecute

make(1): do not look up local variables like .TARGET anywhere else

Nobody defines a global variable named .TARGET since that would have
many unpredictable effects, applying to all targets at once.

Nobody defines an environment variable named .TARGET since that's
against the naming conventions for environment variables and would have
the same effect.

Because of this, there is no point looking up the variables that are
local to a GNode anywhere else.  This means they cannot come from the
environment and thus their value doesn't need to be freed after use,
which makes the code simpler.

The newly added accessor functions in make.h refer to external
functions, but since that header is not used anywhere outside of
usr.bin/make, it doesn't matter.  Between 2020-08-25 and 2020-10-30,
that header had been referenced by usr.bin/xinstall.

make(1): clean up comments and local variables in var.c

make(1): rename VAR_CMD to VAR_CMDLINE

Since make has to do with both the command line and child commands, the
former name was confusing.

make(1): rename defIncPath to defSysIncPath

There are two variables, parseIncPath and sysIncPath, which made the
name defIncPath ambiguous.

make(1): move debugVflag to CmdOpts

make(1): group the command line options and arguments

By having a single struct that holds all command line options and
arguments, it is easy to see in the code when such a command line
argument is modified.  It also cleans up the namespace since the command
line options don't follow a common naming style.  Having them in a
struct also means that there is a single place for putting the
documentation, not two as before.

The struct also suggests to extract the initialization code out of main,
which is still too large, having more than 400 lines of code and
covering far too many topics.

make(1): add GNode_Path to access the path of a GNode

make(1): remove macros MIN and MAX

These macros typically evaluate one of their arguments twice.  Until
2020-08-31, they had not parenthesized their arguments properly.  They
are only used in a few places, therefore it doesn't hurt much to have
them expanded.

make(1): convert macros for debug flags into enum

make(1): remove lengthy comments from archive handling

The relevant information is already documented on OP_ARCHV, but only
since a few weeks.

make(1): fix typos in comments

make(1): convert error level for Parse_Error to an enum

make(1): rename GNode.cmgn to youngestChild

The name is longer than before but also clearer.

make(1): add test for the '::' dependency operator

make(1): negate OP_NOP and rename it to GNode_IsTarget

make(1): merge duplicate documentation for OP_NOP

make(1): only define TRUE and FALSE if necessary

For -DUSE_ENUM_BOOLEAN, the macros are not necessary.

make(1): allow compilation with Boolean implemented as char

make(1): clean up headers, no functional change

make(1): add tags to enum types

This allows IDEs to offer better type information than "anonymous enum".

make(1): remove underscore from Hash_Table and Hash_Entry

For consistency with the other type names, such as GNodeListNode.

make(1): convert NoExecute to non-inline

The header make.h is not only used by usr.bin/make but also by
usr.bin/xinstall, to get the needshell function.

That program does not declare the variables noRecursiveExecute and
noExecute, leading to an "unresolved reference" at link time.  To fix
this, move the inline functions away from make.h.

It's a quirk that make.h gets included by other programs since it
contains a whole lot of definitions that are only useful in make itself.
If any, there should be a separate header that defines the public
library API of make, and only that.

make(1): enable format string truncation warnings for parse.c

No node is ever going to have a million or more cohorts.  And even if
there were, this name is only used for printing informational messages.

make(1): revert previous commit

It had accidentally reverted all the work from the past few days.

make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25.  In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath.  Dir_ClearPath takes full ownership of the given list and
empties it.  This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table.  This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes.  This by itself
was not a problem since the hash table would be freed afterwards.  But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again.  Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.

make(1): replace GNode macros with inline functions

Curiously, this affects the size of the generated code: it gets smaller
than before.

make(1): use consistent pattern for parsing whitespace

The pp and cpp in the function names stand for "parsing position" and
"const parsing position".

make(1): remove the inline attribute from debug_printf

The code size overhead of a varargs function is much more than I had
expected.  Since GCC 5 didn't inline the function anyway and it is only
used in debug mode, there is no point keeping multiple copies of that
code around.

make(1): make debug logging simpler

This avoids referring to the debug_file variable in many places where
this implementation detail is not necessary.

make(1): make debugging code shorter

make(1): normalize whitespace in source code

There is no more space tab.  Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.

make(1): clean up API for finding and creating GNodes

The previous API had complicated rules for the cases in which the single
function returned NULL or what it did.  The flags for that function were
confusing since passing TARG_NOHASH would create a new node even though
TARG_CREATE was not included in that bit mask.

Splitting the function into 3 separate functions avoids this confusion.
It also reveals several places where the complicated API led to
unreachable code.  Such code has been removed.

make(1): migrate MakeTimeStamp and MakeUnmark to Lst_ForEach

make(1): fix assertion failure in -j mode with .END node

There had been two separate global variables for the .END node, and in
parallel mode, only the one in jobs.c was initialized.

The code in JobRun heads over to Compat_Make without calling Compat_Run
first, which left the variable ENDNode uninitialized.

make(1): use fine-grained type names for lists and their nodes

This is only intended to help the human reader.  There is no additional
type safety yet.

make(1): add specific typedefs for lists

These typedefs are only intended to help human readers, they do not
provide any type-safety.  They also make the pointers explicit, which
had been hidden before by the typedef for Lst and LstNode.  Typing a few
'*' is less work than finding out which of the many types are pointers
and which aren't.

In meta.c, the variable "ln" served two completely different purposes,
which have been split again.  Register allocation is the job of the
compiler, not of the human source code reader.

make(1): clean up RCSID blocks

These blocks mostly consisted of redundant structure, following the same
#ifndef pattern over and over, with only minimal variation.

It's easier to maintain if the common structure is only written once and
encapsulated in a macro.

To avoid "defined but unused" warnings from GCC in the case where
MAKE_NATIVE is not defined, I had to add volatile.  Adding
MAKE_ATTR_UNUSED alone would not preserve the rcsid variable in the
resulting binary.

make(1): clean up API for evaluating conditions

There was no need to make struct If publicly visible.

There was no need to have parameters in the public API that were passed
the same constants all the time.

The former function names had not been distinctive.

make(1): remove #endif comments for very short blocks

make(1): add wrappers around ctype.h functions

This avoids casting the argument to unsigned char, and to cast the
result of toupper/tolower back to char.

make(1): add missing space in source code

make(1): document the value restrictions for Boolean variables

The previous lenient rule came from the sprite.h header that was not
specific to make.  To avoid confusion, only the expected values should
be stored in a Boolean variable.  To help find obvious violations and
inconsistencies, there are different possibilities for the Boolean type,
during development.

In C there is no way to actually enforce this restriction at runtime.
It would be possible in C++, but the code is not ready to be compiled
with a C++ compiler.

make(1): reduce number of stat fields returned by cached_stat

Only st_mtime and st_mode are actually filled, the remaining fields had
been set to zero.  To prevent these from ever being accessed, a custom
struct make_stat replaces the previously used struct stat.

The fields in struct make_stat are intentionally named different from
the fields in struct stat because NetBSD and some other operating
systems define st_mtime as a macro, and that would not work in a field
declaration.

make(1): use proper types in API of cached_stat and cached_lstat

make(1): parenthesize macro arguments

Just in case anyone wants to use them for copy-and-paste.

The invocations of these macros are left cautious since the
system-provided definition of these macros may have forgotten the
parentheses as well.

make(1): rename GNode.iParents to implicitParents

The i alone was too ambiguous.  It could have meant ignore, implicit,
interactive, and probably many more.

make(1): remove ReturnStatus, SUCCESS and FAILURE

These are used in so few places now that it is easier to use a simple
Boolean for them.

make(1): add another Boolean variant to check during development

make(1): allow for strict type checking for Boolean

Having Boolean aliased to int creates ambiguities since int is widely
used.  Allow to occasionally compile make with -DUSE_DOUBLE_BOOLEAN to
check that the type definitions still agree.

make(1): clean up targ.c

The main part is removing redundant or outdated information from
comments.  In a few places, the expression cond ? TRUE : FALSE has been
simplified.

make(1): print suffix flags in the standard way

This changes the output (it is now SUFF_NULL instead of just NULL), and
the order of the flags in the output is reversed.

make(1): remove header sprite.h

Make is independent of the Sprite operating system.

make(1): in debug mode, print GNode details in symbols

A string like OP_DEPENDS|OP_OPTIONAL|OP_PRECIOUS is much easier to read
and understand than the bit pattern 00000089.

The implementation in enum.h looks really bloated and ugly, but using
this API is as simple and natural as possible.  That's the trade-off.

In enum.h, I thought about choosing the numbers in the macros such that
it is always possible to combine two of them in order to reach an
arbitrary number, because of the "part1, part2" in the ENUM__SPEC macro.
The powers of 2 are not these numbers, as 7 cannot be expressed as the
sum of two of them.  Neither are the fibonacci numbers since 12 cannot
be expressed as the sum of 2 fibonacci numbers.  I tried to find a
general pattern to generate these minimal 2-sum numbers, but failed.

make(1): remove unused declarations from header files

make(1): use proper data type for GNode.type

Theoretically this should show the enum constant names when displaying
the field in the debugger.  This only happens for GNode.flags though,
but not for GNode.type.  Not sure what the exact cause is.

make(1): clean up headers

Remove redundant headers that are already included by "make.h".
Make <assert.h> available to all compilation units that use "make.h".

make(1): remove wrong comment about possible Y2038 problem

The "time" that is mentioned is not clock time but a sequence number.

make(1): remove unused variable jobServer

make(1): convert GNode type constants to enum

This allows debuggers to print symbolic names for the type flags.

The names of these constants need to be revised.  These flags probably
started as the 3 dependency operators (:, !, ::), and at that time, the
prefix OP_ made sense.  The other flags are not related to the type of
the dependency operator and could have been placed in GNode.flags as
well.

make(1): restructure GNode types and documentation

Having a numbered list above the type was not helpful since the numbers
didn't serve any purpose, they just consumed screen space.

Several of these list items didn't have an obvious relationship to the
struct fields.  It's better to have just a rough introduction as the
type level documentation, leaving the details to the individual fields.

Converting the types and flags and other constants into separate types
and defining them outside the struct leaves more space to see the
relationship of the struct fields.

Limiting the documentation of each field to a single line, as suggested
by the end-of-line comments, reduces clarity since several of the fields
need way more documentation to be properly understood.

make(1): remove unused function declarations

make(1): fix wrong or outdated comments

make(1): follow naming conventions for multiple-inclusion guards

This avoids undefined behavior.

make(1): convert Suff.flags from #define to enum

This increases debugging support since the debugger can now display
symbolic names instead of an integer bit mask.

make(1): fix parameter name of str_concat

The previous documentation mentioned Str_Concat, but str_concat has been
written in lowercase for years.  The "flags" are not flags since they
cannot be combined, not even when they are written in hex.

make(1): use enum for return values of Cond_Eval and friends

Add -dL for LINT

When parsing variable assignments other than := and if
value contains '$' attempt Var_Subst the same as for :=,
if the value does not parse correctly, we get a fatal error
including file an line number.

This can greatly help with finding the cause of problems.

Reviewed by: christos

make(1): clean up various things in var.c

Add -dh for DEBUG_HASH

Allow tracking of max chain length, to see how well the hash
tables are working.
Pull the actual hash operation into a marco so it can be
easily changed - for experimenting.

The current hash, is pretty good.

Reviewed by: christos

make(1): add more type safety for enums

There are several types of flags for variables, and these cannot be
mixed.  To prevent accidental typos, these are defined in separate enum
types.  Clang warns about direct assignments between distinct types, but
not about mixing distinct types in binary expressions like A | B.  GCC
does not warn at all.

Avoid unnecessary noise when sub-make or sibling dies

When analyzing a build log, the first 'stopped' output
from make, is the end of interesting output.

Normally when a build fails deep down in a parallel build
the log ends with many blockes of error output from make,
with all but the fist being unhelpful.

We add a function dieQuietly() which will return true
if we should supress the error output from make.
If the failing node was a sub-make, we want to die quietly.

Also when we read an abort token we call dieQuietly telling we
want to die quietly.

This behavior is suppressed by -dj or
setting .MAKE.DIE_QUIETLY=no

Reviewed by: christos

Merge changes from current as of 20200406

Revert arraycount

CHECK_VALID_META is too agressive for CMD check.

An empty CMD is perfectly valid.
So using CHECK_VALID_META results in too much rebuilding.

make: fix -fno-common build

debug was declared extern, but debug_file was not; correct this and define
debug_file in main.c (as debug is) to fix the -fno-common build.

-fno-common will become the default with GCC10/LLVM11.

Patch from kevans at freebsd

Do not treat .info as warning for -W

Reported by: lwhsu at FreeBSD.org

Make compat.c handle SIGINT etc more like job.c

If there is a running child, pass the signal on, and
wait for it to exit before we self-terminate.

Reviewed by: christos

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Refactor and simplify objdir setting code.

Add a .DELETE_ON_ERROR: magic target that causes *failed* targets as
well as *interrupted* targets to be deleted. The name and behavior of
the variable matches gmake.

Also fix a glitch in newline output on error in compat mode that I
discovered while doing it.

Closes PR 51376.

Extend the mtimes cache used by dir.c so it can be used by others.

We store both st_mtime and st_mode, since some callers care about the
later.

Reviewed by: christos

Add cached_realpath()

realpath(3) is expensive, and meta mode at least uses it extensively.
We use cached_realpath() to save the result of successful calls to
realpath(3) in a private variable context.

This improves the worst case performance (eg examining libc with
nothing to do) by a factor of 4.

Reviewed by: christos

Collapse the 3 boolean parameter to 1 flags parameter. No functional change.

provide missing FD_CLOEXEC for the havenots.

Add more compat defs for code compiled for !NetBSD targets.

At least GNU Hurd lacks MAXHOSTNAMELEN/MAXPATHLEN/PATH_MAX, so make sure
those have some arbitrary semi-sane values.

from Robert Millan in PR misc/50166, with some modifications from myself

Revert all make changes except the unit tests to the state of three
weeks ago. Individual changes can be reapplied after review.

PR/46096: Jarmo Jaakkola: fix many problems with dependencies (PR 49086)

Quite extensive rewrite of the Suff module.  Some ripple effects into
Parse and Targ modules too.

Dependency searches in general were made to honor explicit rules so
implicit and explicit sources are no longer applied on targets that
do not invoke a transformation rule.

Archive member dependency search was rewritten.  Explicit rules now
work properly and $(.TARGET) is set correctly.  POSIX semantics for
lib(member.o) and .s1.a rules are supported.

.SUFFIXES list maintenance was rewritten so that scanning of existing
rules works when suffixes are added and that clearing the suffix list
removes single suffix rules too.  Transformation rule nodes are now
mixed with regular nodes so they are available as regular targets too
if needed (especially after the known suffixes are cleared).

The .NULL target was documented in the manual page, especially to
warn against using it when a single suffix rule would work.
A deprecation warning was also added to the manual and make also
warns the user if it encounters .NULL.

Search for suffix rules no longer allows the explicit dependencies
to override the selected transformation rule.  A check is made in
the search that the transformation that would be tried does not
already exist in the chain.  This prevents getting stuck in an infinite
loop under specific circumstances.  Local variables are now set
before node's children are expanded so dynamic sources work in
multi-stage transformations.  Make_HandleUse() no longer expands
the added children for transformation nodes, preventing triple
expansion and allowing the Suff module to properly postpone their
expansion until proper values are set for the local variables.

Directory prefix is no longer removed from $(.PREFIX) if the target
is found via directory search.

The last rule defined is now used instead of the first one (POSIX
requirement) in case a rule is defined multiple times.  Everything
defined in the first instance is undone, but things added "globally"
are honored.  To implement this, each node tracks attribute bits
which have been set by special targets (global) instead of special
sources (local).  They also track dependencies that were added by
a rule with commands (local) instead of rule with no commands (global).

New attribute, OP_FROM_SYS_MK is introduced.  It is set on all targets
found in system makefiles so that they are not eligible to become
the main target.  We cannot just set OP_NOTMAIN because it is one of
the attributes inherited from transformation and .USE rules and would
make any eligible target that uses a built-in inference rule ineligible.

The $(.IMPSRC) local variable now works like in gmake: it is set to
the first prerequisite for explicit rules.  For implicit rules it
is still the implied source.

The manual page is improved regarding the fixed features.  Test cases
for the fixed problems are added.

Other improvements in the Suff module include:
  - better debug messages for transformation rule search (length of
    the chain is now visualized by indentation)
  - Suff structures are created, destroyed and moved around by a set
    of maintenance functions so their reference counts are easier
    to track (this also gets rid of a lot of code duplication)
  - some unreasonably long functions were split into smaller ones
  - many local variables had their names changed to describe their
    purpose instead of their type

Rebase to HEAD as of a few days ago.

Rebase.

PR/48367: David A. Holland: Mark possible submake nodes so that we can
avoid closing the job pipe on exec for them in order to make recursive
makes work in parallel.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Add VAR_INTERNAL as a context for variables set by make itself,
which should not override those set by makefiles.
Currently MAKEFILE is the only variable affected.

Reviewed by: christos

resync from head

Use a #define for the variable name we put in environment to pass .MAKE.LEVEL
in case we don't want to use gmake's MAKELEVEL in a different way.

Fix obvious typo.

sync with head

Replace __dead, __unused and the various printf format attributes
with versions prefixed by MAKE_ATTR_* to avoid modifying the
implementation namespace. Make sure they are available in all places
using nonints.h to fix bootstrap on Linux.

Var_UnExport() and setenv() can both realloc environ.
Use a common variable (savedEnv) to track that to avoid wasting memory.
Also, if providing setenv and unsetenv, do getenv too to ensure a consistent
set.

Use __dead consistently. If it doesn't exist, define it away.

Add .MAKE.META.BAILIWICK - to identify the scope of make's control.
meta_oodate: if a file that was written or linked within our bailiwick,
but outside of .OBJDIR is missing, add it to missingFiles.
If we get to the end of the .meta file without seeing it [re]moved,
then consider the target out-of-date.

Sync with HEAD

Use the current directory at start time for .PARSEDIR, if the filename
contains no /.

Instead of keeping around the mtime of the youngest child, keep a pointer
to it, so that we can print it when we do the out of date determination.

Add meta.c which implements "meta" mode for make.
In this mode, a .meta file is created for each target, capturing
the expanded commands used, any command output, and if filemon(9)
is available, a record of system calls which are of interest.
Not enabled unless USE_META=yes is set when building make.
Also, if FILEMON_H exists, meta.c will be compiled to use filemon(9).

On darwin at least, vfork() fails in child of vfork().
It probably shouldn't work anyway, so avoid this.
We use the macro vFork() - a function seems to cause problems
and is unnecessary overhead.

Just because $TMPDIR is set does not mean it is valid.
Add a central function for creating temp files so we have one place to
audit.

Reviewed by: dh

Add:

.error "message"
.warning "message"
	based on FreeBSD implementation.
	add .info while were at it.

.ERROR:	a target to run on error.
	We pass the failing GNode to PrintOnError so it can set
	.ERROR_TARGET.

.MAKE.MAKEFILE_PREFERENCE
	As a means to control make's list of prefered makefile names.
	(Default: "makefile Makefile")

.MAKE.DEPENDFILE
	Names the file to read dependencies from
	(Default ".depend")

.MAKE.MODE
	Processed after all makefiles are read.
	Can put make into "compat" mode (more to come).

Fix:

compat.c: Error code should not be sent to debug_file.
Make_DoAllVar: use DONE_ALLSRC to avoid processing a node multiple times.
ReadMakefile: we can simply use doing_depend to control setting MAKEFILE.

Reviewed by: apb

Use .MAKE.LEVEL to track recursion.
The first instance of make will have .MAKE.LEVEL 0, which
can be handy for excluding rules which should not apply
in a sub-make.
gmake and freebsd's make have a similar mechanism, but each
uses a different variable to track it.  Since we cannot be
compatible with both, we allow the makefiles to cope if they want
by handling the export of .MAKE.LEVEL+1 in Var_Set().

Fixes to includes of make_malloc.h to that it actually builds when
USE_EMALLOC is undefined.
Fixes earlier fixes :-)

Move the bmake_malloc() functions into their own .c and .h files.
Include instead of make.h in a few places.

Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)

add debug flag for cwd.

sync with HEAD

back all changes out until I fix it properly.

- use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4

Implement -dl (aka LOUD) to override '@' at the start of script lines.
Based on supplied patch.

PR: 37202

sync with HEAD

If make can't find a rule to create a file it needs to satisfy a rule
that came from .depend then just output a message and contine.

sync with HEAD

Use .MAKE.MAKEFILES to track all the makefiles that have been read
so they can be used in dependency rules.

Add the ability to .export variables to the environment.

Add the ability to tweak the token output before targets in job mode.
Eg.
.MAKE.JOB.PREFIX=${.newline}---${.MAKE:T}[${.MAKE.PID}]
would produce
---make[1234] target ---

Sync w/ NetBSD-4-RC_1

Apply patch (requested by tron in ticket #696):
	usr.bin/make/compat.c		patch
	usr.bin/make/cond.c		patch
	usr.bin/make/dir.c		patch
	usr.bin/make/for.c		patch
	usr.bin/make/main.c		patch
	usr.bin/make/make.1		patch
	usr.bin/make/make.c		patch
	usr.bin/make/make.h		patch
	usr.bin/make/nonints.h		patch
	usr.bin/make/parse.c		patch
	usr.bin/make/str.c		patch
	usr.bin/make/targ.c		patch
	usr.bin/make/util.c		patch
	usr.bin/make/var.c		patch

Synchronize make(1) with HEAD branch to increase perfomance and
improve stability.

Remove the (now redundant) MAKE_BSIZE, BUF_DEF_SIZE from buf.c is used instead.

Pull up following revision(s) (requested by dsl in ticket #285):
	usr.bin/make/cond.c: revision 1.36
	usr.bin/make/parse.c: revision 1.121-1.124
	usr.bin/make/make.h: revision 1.65
	usr.bin/make/targ.c: revision 1.46
	usr.bin/make/main.c: revision 1.138-1.139
Simplify the code that processes .if lines.
It doesn't need a two-dimensional array to remember the states of .if lines.
It would be even simpler if we didn't try to detect .else and .elif lines
that follow .else lines.
Unfortunately this isn't the code that is stupendously slow...
Unknot this code slightly by avoiding 'break/return; else' and by putting
the short parts of some conditionals first.
First step towards speeding up the parsing of makefiles (esp. the unwanted
parts of .if clauses).
There should be no changes to the logic.
There really is no point calling estrdup() to get a copy of the makefile name
into every 'gnode' (aka target), instead just copy a pointer to the string
and avoid freeing the original name when we close the file.
I can't imagine any makefile set where this gives a larger footprint!
Add anewline to the end of the error message output when the debug log
file cannot be opened.
Put the big block of code that was common to ParseDoInclude() and
ParseTraditionalInclude() into a separate routine.
Fix the 'use after free' and 'free on item not malloced' that got added
to ParseTraditionalInclude() in Feb 2006 (rev 1.111).
Kill the 'PTR' struct and put both its members into IFile.
Remove the parameter from ParseEOF(), 1 of the 3 calls passed the wrong value!
Fortunately another test stopped anything nasty happening, we'll use that
test instead.
Revert part of a recent commit.
ParseEOF() shouldn't close the original file.
Some minor changes:
- Only test 'printVars' once.
  Has side effect of not trying to find the 'main' target for '-V varname'.
- Only reap the single child process in Cmd_Exec(),
  I think this is for 'xx != cmd' so probobably no other children should exist.
- Don't read and parse .depend if '-r' and '-V varname' are set.
  I suspect that .depend shouldn't affect the output of any -V command...

Fix detection and reporting of dependency loops in parallel makes.
make's unit_tests should now run again.

There really is no point calling estrdup() to get a copy of the makefile name
into every 'gnode' (aka target), instead just copy a pointer to the string
and avoid freeing the original name when we close the file.
I can't imagine any makefile set where this gives a larger footprint!

A rather large rototil in the way the parallel make code schedules jobs.
This gives a considerable speedup in the processing of .WAIT and .ORDER.
Both .WAIT and .ORDER stop both the commands of the node, and its dependant
nodes being built until the LH nodes are complete.
.WAIT only applies to the dependency line on which it appears, whereas
.ORDER applies globally between the two nodes.
In both cases dependant nodes can be built because other targets need them.
make now processes the target list left to right, scheduling child nodes
as they are needed to make other nodes (instead of attempting to generate
a bottom-up dependency graph at the start).  This means that 'make -j1'
will tend to build in the same order as a non-parallel make.
Note that:
    all: x y
    x: a .WAIT b
    y: b .WAIT a
does not generate a dependency loop.
But
    x: y
    .ORDER y x
does (unless something elswhere causes 'y' to be built).

Output all debug trace output through 'debug_file' defaulting to 'stdout'.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.

Rip out the code for the undocumented -P (don't use pipes for command
output) option.  I'm sure it is baggage from the past.

Don't include <util.h> if we are not NetBSD. Other OS's don't have it.

More programs using efun.

PR/33049: OKANO Takayoshi: Fix the spelling of NetBSD RCS tags.

Make ".WAIT" apply recursively to all children of nodes on the right
hand side of the .WAIT, except when the recursive interpretation would
cause a cycle in the dependency graph.

Discussed in tech-toolchain.  Reviewed by christos, sjg.

Make .NOTPARALLEL stop the job engine issuing more than one token (instead
of trying to de-jobify the make.
You can now put .NOTPARALLEL in a submake of a recursive make (where it is
using a job-token pipe from the outer make and have it only run a single job.
You can also specify .NOTPARRALEL in the root makefile of a large recursive
make and have the submakes run multiple commands.
Add some diagnostics printfs (enabled with -dp) to the parser.

Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

Add new debugging option '-dn' which will leave the scripts fed into make on
-j jobs in /tmp for debugging purposes. Add a note to the man page that
this could cause problems if run a lot (due to the number of files created)

Now that dependencies in .BEGIN, .END, and .INTERRUPT work, allow them.

Don't accept dependencies for .BEGIN, .END, and .INTERRUPT since they
don't make sense or work anyway.

define __GNUC_PREREQ__ if it isn't available on host.

Change to use __unused instead and provide a compat definition in make.h if
not already defined from cdefs.h

Add a -Dg3 which outputs the 'input graph' only on error exit.
Lets you see the wood for the trees...

Implement :[] modifier to allow picking a range of words out of a variable.
Also :tW and a W flag to :C and :S to allow treating value as a single word.
Add unit tests for the above, and fix some corner cases.
Based on patches supplied by Alan Barrett <apb@cequrux.com>

Add the 'e' debug flag (i.e., '-d e'):  when enabled, show the "target
failed" and "command failed" messages added recently.  These introduce
too much noise when debugging some kind of problems, specially in pkgsrc.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

Pass WARNS=3

Add a -X option, which prevents make(1) from putting variables
set on the command line into the environment individually.  This
can be useful on systems which have a tight limit on the size
of the argument space.

Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.

Xref: parse.c, rev 1.46; make.c, rev 1.23

In these revisions `::' dependency handling was simplified by not linking
the cohort nodes into the dependency graph. This broke dependency checking
on all but the first instance of a `::' target since all of the cohort nodes
now just form a collection of disconnected dependency graphs.

Fix this by keeping a back-reference in each cohort to its leader (the
first instance of a :: node with the same name) and a count of the number
of cohorts that need to be made before dependent nodes are scheduled.

Classically, we'd need six centurions for cohort, but in this case one
suffices...

Make_HandleUse() no longer has a return value,

Remove OP_NOSUFF, fix OP_MADE in compat mode [from pk], and make the OP_MADE
comment reflect reality.

Bring down the number of stat(2) system calls from 682 to 294. This change
adds a .NOSUFF directive that has been applied to targets that have been
already made, and are not supposed to have suffix rules applied to them.

Overhaul the initialization and handling of .OBJDIR:

* Replace chdir_verify_path() with Main_SetObjdir(), which can be called
  externally, and can take a "const char *".  (There's a lot of non-const
  "char *" passing around in var.c of what should be const strings....)

* Rewrite the initial "find my .OBJDIR" code to make use of the new
  function.  This still functions as it had in the past, but the comment
  above this block was changed to reflect reality:  if MAKEOBJDIRPREFIX
  or MAKEOBJDIR are set in the environment, then *only that value* is
  tried; make does not fall back to obj.MACHINE, obj, and /usr/obj/`pwd`
  as it would without these env vars set.

* Add a new special target, .OBJDIR:, which when parsed will cause make to
  change to a new object directory and reset .OBJDIR, and PWD in the
  environment.  This will allow some makefiles (mainly, src/tools)
  to override the default objdir semantics in order to add custom logic.

Allow "-m" to be used in a .MAKEFLAGS: special target and get it to work.
(This splits out the "default system include paths" into its own Lst
variable, and uses it only if sysIncPath is empty.  This allows sysIncPath
to be filled in by the Makefile itself.)

Spell 'occurred' with two 'r's.

add .USEBEFORE Attribute

Modify handling of command line variable assignments and their exporting
via MAKEFLAGS.  Instead of appending them directly to .MAKEFLAGS, put
them in .MAKEOVERRIDES (and ensure they are quoted).  This is now done
in Var_Set when it exports VAR_CMD's.
Use ExportMAKEFLAGS() to export MAKEFLAGS, using the combined content
of .MAKEFLAGS and .MAKEOVERRIDES (with duplicate supression).
If .MAKEFLAGS is assigned to in a Makefile, ExportMAKEFLAGS is called again.
This allows a line like:
.MAKEOVERRIDES=
to effectively stop the exporting of the command line vars in MAKEFLAGS.

A number of semi-related changes.
1. make -dx turns on DEBUG_SHELL which causes sh -x to be used where
   possible.
2. PrintOnError() is now called when make is stopping due to an error.
   This routine reports the curdir and the value of any variables listed
   in MAKE_PRINT_VAR_ON_ERROR.
3. Variables set via command line, are propagated to child-makes via
   MAKEFLAGS.  This behaviour appears to be necessary for POSIX (according
   to the GNU folk anyway).
4. Do not reset MAKEFILE when reading ".depend" as this rather eliminates the
   usefulness of ${MAKEFILE}.
5. Added ${.newline} as a simple means of being able to include \n in the
   result of a :@ loop expansion.
6. Set ${MAKE_VERSION} if defined.  Need to come up with a useful value.

Reviewed: christos

- make the second list of commands override the first like gmake does
- remember where the first command definition occured so that we can debug
  things.

PR/11800: Chris Demetriou: Add -W flag to exit on parse warnings.
While I am here, s/make:.*"/%s:\1", progname/

Missing bit of last change

Add a token-passing scheme to allow a recursive make to successfully
use -j; all make's in a recursive build cooperate to limit the total
number of jobs, using a token-passing scheme.

The current token passing algorithm is similar to the one implemented
by gmake; there is a single pipe which is inherited through the entire
process hierarchy; tokens are obtained by reading a byte from the
"read end" of the pipe, and are returned by writing them to the "write
end".  This exact algorithm is likely to change in the future.

Implementation details:
 - Use the new trace facility to allow measurement of the
effectiveness of different token-passing schemes
 - Get a token in MakeStartJobs(), return it in Make_Update()
 - Eliminate Job_Full() and the jobFull global since they are
redundant with token system.
 - Add an "internal" -J option (to pass the token pipe fd's down to
submakes) and a -T option for tracing.
 - Change how compatMake is forced so that -j means something when
inherited by submakes.
 - When waiting for a token, poll the token-passing pipe as well as
the output pipes of existing jobs.

Sync w/ netbsd-1-5-base.

Readd optimization last night.  Problems earlier were partially due to the
arguments names on one function being swapped (by a previous author).

Do not do any duplicate suppression when a source list is created.  Instead:
* OP_MADE protects against trying to make the source multiple times.
* A new OP_MARK flag is introduced to suppress duplicates while expanding
  the .ALLSRC variable and .USE targets.
This turns the O(n^2) insertion into O(n) in most cases.

This is tested with a `make build' and some special test cases.

Back out last night's optimization for now.

Introduce an OP_MARK bit, and use it to suppress duplicates during .ALLSRC
and .USE expansion.  Also, remove some more Lst_Member() checks that are now
redundant.

Create a `for' context, and substitute iteration variable from it using the
normal Var_Parse() path.  This allows :R, etc. to work on iteration variables.

make cleaner C for more picky compilers

Changes to make MAKEOBJDIRPREFIX useful.

Firstly, we ignore getenv("PWD") if MAKEOBJDIRPREFIX is set so that we always
get the same value for .CURDIR regardless of how make was invoked.

Second, when executing a command we check if it is ${.MAKE} or ${.MAKE:T}
without a preceeding chdir, if so we insert a chdir(${.CURDIR}) so that
the Makefile will be found by the child make.  Note that this behaviour is
dissabled if MAKEOBJDIRPREFIX is not set or if NOCHECKMAKECHDIR is set.
See the comments in main.c for more detail.

With these two changes, one can successfully build usr/src using MAKEOBJDIRPREFIX
allowing the src to be mounted from a CD-ROM.

Make this compile on Solaris...

Replace the linear variable list in the GNode with a hash table.  Gives a
small (~10% on my machine) speedup building libc.
Changes by: perry (+ bug fix and minor change by me)
Reviewed by: mycroft sommerfeld

This patch fixes the problem introduced in the previous commit where
parents would be get remade, even if children were not really updated
by the commands executed for them. It also makes all the children have
the real modification time set if possible, so it should fix some other
timing weirdnesses...

- collapse childMade and make fields into flags and convert them to bits
  CHILDMADE and REMAKE
- introduce FORCE flag that gets set in all the parents of a child that
  has no sources and does not exist.
- set oodate if the FORCE flag is set, and not if CHILDMADE
- centralize the RECHECK into Make_Recheck() and use this in make.c and compat.c
- use Make_TimeStamp for all child -> parent timestamp propagations

Only define const and volatile when __STDC__ is not defined (for BOOTSTRAP)

Make a MAKE_BOOTSTRAP case work on NetBSD, too, for cross-compile cases

fix some ints that should be "time_t" per PR#4139

Add the new .NOPATH feature which can be used to disable .PATH search
for particular targets, i.e. .depend, objects, etc.  (from Christos).

- Target searching addition:
	Make used to only use the search path for nodes that were pure
	sources (not targets of other sources). This has been corrected
	and now gnu-autoconf generated Makefiles work in directories other
	than the source one.

- Suffix transformation rescanning:
	Suffix transformations (.c.o:; cc ...) were only recognized in
	the past when both suffixes were members of the suffix list.
	Thus a sequence like:
		.z.b:
			echo ${.TARGET}
		.SUFFIXES: .z
	would cause .z.b: to be inserted as a regular target (and the main
	target in this case). Other make programs always add rules that
	start with a period in the transformation list and never consider
	them as targets. We cannot do that (consider .depend files) so we
	resort to scanning the list of the current targets every time a
	suffix gets added, and we mutate existing targets that are now
	valid transformation rules into transformation rules. If the
	transformed target was also the main target, we set the main target
	to be the next target in the targets list.

Add a .MADE directive to indicated that the children of a target are
up-to-date, even when they are not. This is to simulate our current
make install behavior with proper dependencies.

Reported by cgd: fix .USE directive problems:
    1. ${.*} variables did not get expanded in dependencies.
    2. expanded ${.*} variables in .USE dependencies can cause tree
       restructuring; handle it.
    3. in compat mode, expand .USE before evaluating the list of targets,
       instead of doing .USE expansions on demand, because they can cause
       tree restructuring.

Update make(1) from trunk, by request from Christos Zoulas.  Fixes many bugs.

BSD did not traditionally have <sys/cdefs.h>; use BSD4_4 instead and include
<sys/param.h> to grab it.

define const and volatile for non __STDC__

Import 4.4BSD-Lite2 sources onto CSRG branch (already merged at head)

- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
  be considered to be out of date, since it does not have a TOC.

Protect __P from being multiply defined (for systems that already define it)

Fix PR/2118; apply sysincludes patch.

fix pr/1421 and pr/1997

- Added .WAIT to synchronize between sources like other pmake variants.
- Updated documentation to include .ORDER .PARALLEL .NO_PARALLEL .NONPARALLEL

- $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
  Posix mandates
- Added .PHONY target keyword

bootstrap improvements

bootstrap changes

fixes/improvements from Christos Zoulas <christos@deshaw.com>.

Add RCS identifiers.

initial import of 386bsd-0.1 sources

Initial revision