CVS log for src/bin/sh/exec.c

Up to [cvs.NetBSD.org] / src / bin / sh

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Improve safety in var imports from the environment.

Add a new var flag VUNSAFE - set on all vars imported from the environment.

Add setvareqsafe() (which is to setvareq() as setvarsafe() is to setvar())
and use that instead of setvareq() when processing the environment, so
errors don't cause the shell to abort.  Use VUNSAFE in that call.

Add flags arguments to all var callback functions which are used when setting
variables, and pass the flags given to the setvar*() functions to those
functions, so they can act differently in different situations (if desired).
Most of them just ignore the flags.

When unsetting a variable, call setvar() to clear things (and call the
callback function) both when the variable had a value which needs to be freed,
and when unsetting a variable which wasn't unset previously, so the VUNSET
flag can be seen by that callback func.

When setting HISTSIZE, use the flags passed to determine whether to ignore
bad values (if VUNSAFE) or treat them as an error.  This replaces the
earlier temporary hack to always ignore bad data there (histedit.c 1.68).

Miscellaneous associated minor changes.

These changes should largely be invisible in normal use.

Do a better job handling EACCES errors from exec() calls.   If the
EACCES is from the namei(), treat it just like ENOENT or ENOTDIR
(and if that is the final error, the exit status from a failed exec
will be 127).   If the EACCES is from the exec() itself, that indicates
the file to be run exists, but has no 'x' permission.   That's a
meaningful error (as distinct from just "yet another PATH element
search failure").

While here, return the first meaingful error we encountered while
searching PATH, rather than the last (and ENOENT if there are none
of those).

This change results in some failed command executions returning status
127 now, where they returned 126 before - which better reflects the
intent of those values (127 is simply "not found" whereas 126 is "found
but couldn't be executed").

We still do nothing to distinguish errors encountered looking up the
command name give, with errors encountered (by the kernel) attempting to
run an interpreter needed for the exec to succeed (#! line path, or
/libexec/ld.elf_so and similar - or anything else of a similar nature).

PR bin/56491

Make "hash" exit(!=0) (ie: exit(1)) if it writes an error message to
stderr as required by POSIX (it was writing "not found" errors, yet
still doing exit(0)).

Whether, when doing "hash foobar", and "foobar" is not found as a command
(not a built-in, not a function, and not found via a PATH search), that
should be considered an error differs between shells.  All of the ksh
descendant shells say "no", write no error message in this case, and
exit(0) if no other errors occur.   Other shells (essentially all) do
consider it an error, write a message to stderr, and exit(1) when this happens.

POSIX isn't clear, the bug report:
     https://austingroupbugs.net/view.php?id=1460
which is not yet resolved, suggests that the outcome will be that
this is to be unspecified.   Given the diversity, there might be no
other choice.

Have a foot in both camps - default to the "other shell" behaviour,
but add a -e option (no errors ... applies only to these "not found"
errors) to generate the ksh behaviour.   Without other errors (like an
unknown option, etc) "hash -e anyname" will always exit(0).

See the PR for details on how it all works now, or read the updated man page.

While here, when hash is in its other mode (reporting what is in the
table) check for I/O errors on stdout, and exit(1) (with an error
message!) if any occurred.   This does not apply to output generated
by the -v option when command names are given (that output is incidental).

In sh.1 document all of this.   Also add documentation for a bunch of
other options the hash command has had for years, but which were never
documented.   And while there, clean up some other sections I noticed
needed improving (either formatting or content or both).

sh: make find_command simpler

Lint complained about the do-while-0 loop that contained a continue. It
didn't state the reason for it, but indeed the code looked complicated.
Rewrite the code to be less verbose and to use common coding patterns.

No functional change.

Pull up following revision(s) (requested by kre in ticket #1242):

	bin/sh/input.c: revision 1.72
	bin/sh/exec.c: revision 1.55

PR bin/55979

This fixes the MSAN detected reference to an unitialised variable
(an unitialised field in a struct) which happens when a command is
not found after a PATH search.
Aside from skipping some known to be going to fail exec*() calls
in some cases, the setting of the relevant field is irrelevant,
so this problem makes no practical difference to the shell, or any
shell script.

XXX (maybe) pullup -9


PR bin/55979

Correctly handle (ie: ignore completely) \0 chars (nuls) in the
shell command input stream (script, dot file, or stdin).
Previously nul chars were ignored correctly in the line in which
they occurred, but would cause trailing chars of that line to reappear
as the start of the following line.   If there was just one \0 skipped,
this would generally result in an extra \n in the sh input, which in
most cases has no effect.   With multiple \0's in a single line, more
of the end of that line was duplicated into the following one.  This
usually manifested as a weird "command not found" error.

Note that any \0 chars in the sh input make the script non-conforming,
so fixing this is not crucial (no \0's should really ever be seen) but
it was an obvious bug in the code, which was attempting to ignore nul
chars (as do many other shells), so let it be fixed.

XXX pullup -9

PR bin/55979

This fixes the MSAN detected reference to an unitialised variable
(an unitialised field in a struct) which happens when a command is
not found after a PATH search.

Aside from skipping some known to be going to fail exec*() calls
in some cases, the setting of the relevant field is irrelevant,
so this problem makes no practical difference to the shell, or any
shell script.

XXX (maybe) pullup -9

Pull up following revision(s) (requested by kre in ticket #1629):

	bin/sh/exec.c: revision 1.54

PR bin/55526

Fix a bug that has existed since the "command" command was added in
2003.   "command foo" would cause the definition of a function "foo"
to be lost (not freed, simply discarded) if "foo" is (in addition to
being a function) a filesystem command.   The case where "foo" is
a builtin was handled.

For now, when a function exists with the same name as a filesystem
command, the latter can never appear in the command hash table, and
when used (which can only be via "command foo", just "foo" finds
the function) will always result in a full PATH search.

XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)

Pull up following revision(s) (requested by kre in ticket #1064):

	bin/sh/exec.c: revision 1.54

PR bin/55526

Fix a bug that has existed since the "command" command was added in
2003.   "command foo" would cause the definition of a function "foo"
to be lost (not freed, simply discarded) if "foo" is (in addition to
being a function) a filesystem command.   The case where "foo" is
a builtin was handled.

For now, when a function exists with the same name as a filesystem
command, the latter can never appear in the command hash table, and
when used (which can only be via "command foo", just "foo" finds
the function) will always result in a full PATH search.

XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)

PR bin/55526

Fix a bug that has existed since the "command" command was added in
2003.   "command foo" would cause the definition of a function "foo"
to be lost (not freed, simply discarded) if "foo" is (in addition to
being a function) a filesystem command.   The case where "foo" is
a builtin was handled.

For now, when a function exists with the same name as a filesystem
command, the latter can never appear in the command hash table, and
when used (which can only be via "command foo", just "foo" finds
the function) will always result in a full PATH search.

XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)

Ooops, restore accidently removed files from merge mishap

Sync with HEAD

Sync with HEAD

Pull up following revision(s) (requested by kre in ticket #989):

	bin/sh/eval.c: revision 1.156
	bin/sh/eval.h: revision 1.20
	bin/sh/exec.c: revision 1.53

Fix several bugs in the command / type builtin ( including PR bin/48499 )

1. Make command -pv (and -pV) work (which is not as easy as the PR
   suggests it might be (the "check and cause error" was there because
   it did not work, not in order to prevent it from working).

2. Stop -v and -V being both used (that makes no sense).

3. Stop the "type" builtin inheriting the args (-pvV) that "command" has
   (which it did, as when -v -or -V is used with command, it and type are
    implemented using the same code).

4. make "command -v word" DTRT for sh keywords (was treating them as an error).

5. Require at least one arg for "command -[vV]" or "type" else usage & error.
   Strictly this should also apply to "command" and "command -p" (no -v)
   but that's handled elsewhere, so perhaps some other time.   Perhaps
   "command -v" (and -V) should be limited to 1 command name (where "type"
   can have many) as in the POSIX definitions, but I don't think that matters.

6. With "command -V alias", (or "type alias" which is the same thing),
   (but not "command -v alias") alter the output format, so we get
	ll is an alias for: ls -al
   instead of the old
	ll is an alias for
	ls -al
   (and note there was a space, for some reason, after "for")
   That is, unless the alias value contains any \n characters, in which
   case (something approximating) the old multi-line format is retained.
   Also note: that if code wants to parse/use the value of an alias, it
   should be using the output of "alias name", not command or type.

Note that none of the above affects "command [-p] cmd" (no -v or -V options)
only "command -[vV]" and "type".

Note also that the changes to eval.[ch] are merely to make syspath()
visible in exec.c rather than static in eval.c

Sync with HEAD

Fix several bugs in the command / type builtin ( including PR bin/48499 )

1. Make command -pv (and -pV) work (which is not as easy as the PR
   suggests it might be (the "check and cause error" was there because
   it did not work, not in order to prevent it from working).

2. Stop -v and -V being both used (that makes no sense).

3. Stop the "type" builtin inheriting the args (-pvV) that "command" has
   (which it did, as when -v -or -V is used with command, it and type are
    implemented using the same code).

4. make "command -v word" DTRT for sh keywords (was treating them as an error).

5. Require at least one arg for "command -[vV]" or "type" else usage & error.
   Strictly this should also apply to "command" and "command -p" (no -v)
   but that's handled elsewhere, so perhaps some other time.   Perhaps
   "command -v" (and -V) should be limited to 1 command name (where "type"
   can have many) as in the POSIX definitions, but I don't think that matters.

6. With "command -V alias", (or "type alias" which is the same thing),
   (but not "command -v alias") alter the output format, so we get
	ll is an alias for: ls -al
   instead of the old
	ll is an alias for
	ls -al
   (and note there was a space, for some reason, after "for")

   That is, unless the alias value contains any \n characters, in which
   case (something approximating) the old multi-line format is retained.
   Also note: that if code wants to parse/use the value of an alias, it
   should be using the output of "alias name", not command or type.

Note that none of the above affects "command [-p] cmd" (no -v or -V options)
only "command -[vV]" and "type".

Note also that the changes to eval.[ch] are merely to make syspath()
visible in exec.c rather than static in eval.c

Pull up following revision(s) (requested by kre in ticket #906):

	bin/sh/eval.c: revision 1.155
	bin/sh/mknodes.sh: revision 1.3
	bin/sh/nodes.c.pat: revision 1.14
	bin/sh/exec.h: revision 1.27
	bin/sh/exec.c: revision 1.52

Deal with ref after free found by ASAN when a function redefines
itself, or some other function which is still active.

This was a long known bug (fixed ages ago in the FreeBSD sh) which
hadn't been fixed as in practice, the situation that causes the
problem simply doesn't arise .. ASAN found it in the sh dotcmd
tests which do have this odd "feature" in the way they are written
(but where it never caused a problem, as the tests are so simple
that no mem is ever allocated between when the old version of the
function was deleted, and when it finished executing, so its code
all remained intact, despite having been freed.)

The fix is taken from the FreeBSD sh.

XXX -- pullup-8 (after a while to ensure no other problems arise).

Sync with HEAD

Deal with ref after free found by ASAN when a function redefines
itself, or some other function which is still active.
This was a long known bug (fixed ages ago in the FreeBSD sh) which
hadn't been fixed as in practice, the situation that causes the
problem simply doesn't arise .. ASAN found it in the sh dotcmd
tests which do have this odd "feature" in the way they are written
(but where it never caused a problem, as the tests are so simple
that no mem is ever allocated between when the old version of the
function was deleted, and when it finished executing, so its code
all remained intact, despite having been freed.)

The fix is taken from the FreeBSD sh.

XXX -- pullup-8 (after a while to ensure no other problems arise).

Pull up following revision(s) (requested by kre in ticket #103):
	bin/kill/kill.c: 1.28
	bin/sh/Makefile: 1.111-1.113
	bin/sh/arith_token.c: 1.5
	bin/sh/arith_tokens.h: 1.2
	bin/sh/arithmetic.c: 1.3
	bin/sh/arithmetic.h: 1.2
	bin/sh/bltin/bltin.h: 1.15
	bin/sh/cd.c: 1.49-1.50
	bin/sh/error.c: 1.40
	bin/sh/eval.c: 1.142-1.151
	bin/sh/exec.c: 1.49-1.51
	bin/sh/exec.h: 1.26
	bin/sh/expand.c: 1.113-1.119
	bin/sh/expand.h: 1.23
	bin/sh/histedit.c: 1.49-1.52
	bin/sh/input.c: 1.57-1.60
	bin/sh/input.h: 1.19-1.20
	bin/sh/jobs.c: 1.86-1.87
	bin/sh/main.c: 1.71-1.72
	bin/sh/memalloc.c: 1.30
	bin/sh/memalloc.h: 1.17
	bin/sh/mknodenames.sh: 1.4
	bin/sh/mkoptions.sh: 1.3-1.4
	bin/sh/myhistedit.h: 1.12-1.13
	bin/sh/nodetypes: 1.16-1.18
	bin/sh/option.list: 1.3-1.5
	bin/sh/parser.c: 1.133-1.141
	bin/sh/parser.h: 1.22-1.23
	bin/sh/redir.c: 1.58
	bin/sh/redir.h: 1.24
	bin/sh/sh.1: 1.149-1.159
	bin/sh/shell.h: 1.24
	bin/sh/show.c: 1.43-1.47
	bin/sh/show.h: 1.11
	bin/sh/syntax.c: 1.4
	bin/sh/syntax.h: 1.8
	bin/sh/trap.c: 1.41
	bin/sh/var.c: 1.56-1.65
	bin/sh/var.h: 1.29-1.35
An initial attempt at implementing LINENO to meet the specs.
Aside from one problem (not too hard to fix if it was ever needed) this version
does about as well as most other shell implementations when expanding
$((LINENO)) and better for ${LINENO} as it retains the "LINENO hack" for the
latter, and that is very accurate.
Unfortunately that means that ${LINENO} and $((LINENO)) do not always produce
the same value when used on the same line (a defect that other shells do not
share - aside from the FreeBSD sh as it is today, where only the LINENO hack
exists and so (like for us before this commit) $((LINENO)) is always either
0, or at least whatever value was last set, perhaps by
	LINENO=${LINENO}
which does actually work ... for that one line...)
This could be corrected by simply removing the LINENO hack (look for the string
LINENO in parser.c) in which case ${LINENO} and $((LINENO)) would give the
same (not perfectly accurate) values, as do most other shells.
POSIX requires that LINENO be set before each command, and this implementation
does that fairly literally - except that we only bother before the commands
which actually expand words (for, case and simple commands).   Unfortunately
this forgot that expansions also occur in redirects, and the other compound
commands can also have redirects, so if a redirect on one of the other compound
commands wants to use the value of $((LINENO)) as a part of a generated file
name, then it will get an incorrect value.  This is the "one problem" above.
(Because the LINENO hack is still enabled, using ${LINENO} works.)
This could be fixed, but as this version of the LINENO implementation is just
for reference purposes (it will be superseded within minutes by a better one)
I won't bother.  However should anyone else decide that this is a better choice
(it is probably a smaller implementation, in terms of code & data space then
the replacement, but also I would expect, slower, and definitely less accurate)
this defect is something to bear in mind, and fix.
This version retains the *BSD historical practice that line numbers in functions
(all functions) count from 1 from the start of the function, and elsewhere,
start from 1 from where the shell started reading the input file/stream in
question.  In an "eval" expression the line number starts at the line of the
"eval" (and then increases if the input is a multi-line string).
Note: this version is not documented (beyond as much as LINENO was before)
hence this slightly longer than usual commit message.
A better LINENO implementation.   This version deletes (well, #if 0's out)
the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)).
(Code to invert the LINENO hack when required, like when de-compiling the
execution tree to provide the "jobs" command strings, is still included,
that can be deleted when the LINENO hack is completely removed - look for
refs to VSLINENO throughout the code.  The var funclinno in parser.c can
also be removed, it is used only for the LINENO hack.)
This version produces accurate results: $((LINENO)) was made as accurate
as the LINENO hack made ${LINENO} which is very good.  That's why the
LINENO hack is not yet completely removed, so it can be easily re-enabled.
If you can tell the difference when it is in use, or not in use, then
something has broken (or I managed to miss a case somewhere.)
The way that LINENO works is documented in its own (new) section in the
man page, so nothing more about that, or the new options, etc, here.
This version introduces the possibility of having a "reference" function
associated with a variable, which gets called whenever the value of the
variable is required (that's what implements LINENO).  There is just
one function pointer however, so any particular variable gets at most
one of the set function (as used for PATH, etc) or the reference function.
The VFUNCREF bit in the var flags indicates which func the variable in
question uses (if any - the func ptr, as before, can be NULL).
I would not call the results of this perfect yet, but it is close.
Unbreak (at least) i386 build .... I have no idea why this built for me on
amd64 (problem was missing prototype for snprintf witout <stdio.h>)
While here, add some (DEBUG mode only) tracing that proved useful in
solving another problem.
Set the line number before expanding args, not after.   As the line_number
would have usually been set earlier, this change is mostly an effective
no-op, but it is better this way (just in case) - not observed to have
caused any problems.
Undo some over agressive fixes for a (pre-commit) bug that did not
need these changes to be fixed - and these cause problems in another
absurd use case.   Either of these issues is unlikely to be seen by
anyone who isn't an idiot masochist...
PR bin/52280
removescapes_nl in expari() even when not quoted,
CRTNONL's appear regardless of quoting (unlike CTLESC).
New sentence, new line. Whitespace.
Improve the (new) LINENO section, markup changes (with thanks to wiz@ for
assistace) and some better wording in a few placed.
I am an idiot...  revert the previous unintended commit.
Remove some left over baggage from the LINENO v1 implementation that
didn't get removed with v2, and should have.   This would have had
(I think, without having tested it) one very minor effect on the way
LINENO worked in the v2 implementation, but my guess is it would have
taken a long time before anyone noticed...
Correct spelling in comments of DEBUG only code...
(Perhaps) temporary fix to pkgtools (cwrappers) build (configure).
Expanding  `` containing \ \n sequences looks to have been giving
problems.   I don't think this is the correct fix, but it will do
no worse harm than (perhaps) incorrectly calculating LINENO in this
kind of (rare) circumstance.   I'll look and see if there should be
a better fix later.
s/volatile/const/ -- wonderful how opposites attract like this.
NFC (normal use) - DEBUG only change, when showing empty arg list don't
omit terminating \n.
Free stack memory in a couple of obscure cases where it wasn't
being done (one in probably dead code that is never compiled, the other
in a very rare error case.)   Since it is stack memory it wasn't lost
in any case, just held longer than needed.
Many internal memory management type fixes.
PR bin/52302   (core dump with interactive shell, here doc and error
on same line) is fixed.   (An old bug.)
echo "$( echo x; for a in $( seq 1000 ); do printf '%s\n'; done; echo y )"
consistently prints 1002 lines (x, 1000 empty ones, then y) as it should
(And you don't want to know what it did before, or why.) (Another old one.)
(Recently added) Problems with ~ expansion fixed (mem management related).
Proper fix for the cwrappers configure problem (which includes the quick
fix that was done earlier, but extends upon that to be correct). (This was
another newly added problem.)
And the really devious (and rare) old bug - if STACKSTRNUL() needs to
allocate a new buffer in which to store the \0, calculate the size of
the string space remaining correctly, unlike when SPUTC() grows the
buffer, there is no actual data being stored in the STACKSTRNUL()
case - the string space remaining was calculated as one byte too few.
That would be harmless, unless the next buffer also filled, in which
case it was assumed that it was really full, not one byte less, meaning
one junk char (a nul, or anything) was being copied into the next (even
bigger buffer) corrupting the data.
Consistent use of stalloc() to allocate a new block of (stack) memory,
and grabstackstr() to claim a block of (stack) memory that had already
been occupied but not claimed as in use.  Since grabstackstr is implemented
as just a call to stalloc() this is a no-op change in practice, but makes
it much easier to comprehend what is really happening.  Previous code
sometimes used stalloc() when the use case was really for grabstackstr().
Change grabstackstr() to actually use the arg passed to it, instead of
(not much better than) guessing how much space to claim,
More care when using unstalloc()/ungrabstackstr() to return space, and in
particular when the stack must be returned to its previous state, rather than
just returning no-longer needed space, neither of those work.  They also don't
work properly if there have been (really, even might have been) any stack mem
allocations since the last stalloc()/grabstackstr().   (If we know there
cannot have been then the alloc/release sequence is kind of pointless.)
To work correctly in general we must use setstackmark()/popstackmark() so
do that when needed.  Have those also save/restore the top of stack string
space remaining.
	[Aside: for those reading this, the "stack" mentioned is not
	in any way related to the thing used for maintaining the C
	function call state, ie: the "stack segment" of the program,
	but the shell's internal memory management strategy.]
More comments to better explain what is happening in some cases.
Also cleaned up some hopelessly broken DEBUG mode data that were
recently added (no effect on anyone but the poor semi-human attempting
to make sense of it...).
User visible changes:
Proper counting of line numbers when a here document is delimited
by a multi-line end-delimiter, as in
	cat << 'REALLY
	END'
	here doc line 1
	here doc line 2
	REALLY
	END
(which is an obscure case, but nothing says should not work.)  The \n
in the end-delimiter of the here doc (the last one) was not incrementing
the line number, which from that point on in the script would be 1 too
low (or more, for end-delimiters with more than one \n in them.)
With tilde expansion:
	unset HOME; echo ~
changed to return getpwuid(getuid())->pw_home instead of failing (returning ~)
POSIX says this is unspecified, which makes it difficult for a script to
compensate for being run without HOME set (as in env -i sh script), so
while not able to be used portably, this seems like a useful extension
(and is implemented the same way by some other shells).
Further, with
	HOME=; printf %s ~
we now write nothing (which is required by POSIX - which requires ~ to
expand to the value of $HOME if it is set) previously if $HOME (in this
case) or a user's directory in the passwd file (for ~user) were a null
STRING, We failed the ~ expansion and left behind '~' or '~user'.
Changed the long name for the -L option from lineno_fn_relative
to local_lineno as the latter seemed to be marginally more popular,
and perhaps more importantly, is the same length as the peviously
existing quietprofile option, which means the man page indentation
for the list of options can return to (about) what it was before...
(That is, less indented, which means more data/line, which means less
lines of man page - a good thing!)
Cosmetic changes to variable flags - make their values more suited
to my delicate sensibilities...  (NFC).
Arrange not to barf (ever) if some turkey makes _ readonly.  Do this
by adding a VNOERROR flag that causes errors in var setting to be
ignored (intended use is only for internal shell var setting, like of "_").
(nb: invalid var name errors ignore this flag, but those should never
occur on a var set by the shell itself.)
From FreeBSD: don't simply discard memory if a variable is not set for
any reason (including because it is readonly) if the var's value had
been malloc'd.  Free it instead...
NFC - DEBUG changes, update this to new TRACE method.
KNF - white space and comment formatting.
NFC - DEBUG mode only change - convert this to the new TRACE() format.
NFC - DEBUG mode only change - complete a change made earlier (marking
the line number when included in the trace line tag to show whether it
comes from the parser, or the elsewhere as they tend to be quite different).
Initially only one case was changed, while I pondered whether I liked it
or not.  Now it is all done...   Also when there is a line tag at all,
always include the root/sub-shell indicator character, not only when the
pid is included.
NFC: DEBUG related comment change - catch up with reality.
NFC: DEBUG mode only change.  Fix botched cleanup of one TRACE().
"b" more forgiving when sorting options to allow reasonable (and intended)
flexibility in option.list format.   Changes nothing for current option.list.
Now that excessive use of STACKSTRNUL has served its purpose (well, accidental
purpose) in exposing the bug in its implementation, go back to not using
it when not needed for DEBUG TRACE purposes.   This change should have no
practical effect on either a DEBUG shell (where the STACKSTRNUL() calls
remain) or a non DEBUG shell where they are not needed.
Correct the initial line number used for processing -c arg strings.
(It was inheriting the value from end of profile file processing) - I didn't
notice before as I usually test with empty or no profile files to avoid
complications.   Trivial change which should have very limited impact.
Fix from FreeBSD (applied there in July 2008...)
Don't dump core with input like sh -c 'x=; echo >&$x' - that is where
the word after a >& or <& redirect expands to nothing at all.
Another fix from FreeBSD (this one from April 2009).
When processing a string (as in eval, trap, or sh -c) don't allow
trailing \n's to destroy the exit status of the last command executed.
That is:
	sh -c 'false
	'
	echo $?
should produce 1, not 0.
It is amazing what nonsense appears to work sometimes... (all my nonsense too!)
Two bugs here, one benign because of the way the script is used.
The other hidden by NetBSD's sort being stable, and the data not really
requiring sorting at all...
So as it happens these fixes change nothing, but they are needed anyway.
(The contents of the generated file are only used in DEBUG shells, so
this is really even less important than it seems.)
Another ancient (highly improbable) bug bites the dust.   This one
caused by incorrect macro usage (ie: using the wrong one) which has
been in the sources since version 1.1 (ie: forever).
Like the previous (STACKSTRNUL) bug, the probability of this one
actually occurring has been infinitesimal but the LINENO code increases
that to infinitesimal and a smidgen... (or a few, depending upon usage).
Still, apparently that was enough, Kamil Rytarowski discovered that the
zsh configure script (damn competition!) managed to trigger this problem.
source .editrc after we initialize so that commands persist!
Make arg parsing in kill POSIX compatible with POSIX (XBD 2.12) by
parsing the way getopt(3) would, if only it could handle the (required)
-signumber and -signame options.  This adds two "features" to kill,
-ssigname and -lstatus now work (ie: one word with all of the '-', the
option letter, and its value) and "--" also now works (kill -- -pid1 pid2
will not attempt to send the pid1 signal to pid2, but rather SIGTERM
to the pid1 process group and pid2).  It is still the case that (apart
from --) at most 1 option is permitted (-l, -s, -signame, or -signumber.)
Note that we now have an ambiguity, -sname might mean "-s name" or
send the signal "sname" - if one of those turns out to be valid, that
will be accepted, otherwise the error message will indicate that "sname"
is not a valid signal name, not that "name" is not.   Keeping the "-s"
and signal name as separate words avoids this issue.
Also caution: should someone be weird enough to define a new signal
name (as in the part after SIG) which is almost the same name as an
existing name that starts with 'S' by adding an extra 'S' prepended
(eg: adding a SIGSSYS) then the ambiguity problem becomes much worse.
In that case "kill -ssys" will be resolved in favour of the "-s"
flag being used (the more modern syntax) and would send a SIGSYS, rather
that a SIGSSYS.    So don't do that.
While here, switch to using signalname(3) (bye bye NSIG, et. al.), add
some constipation, and show a little pride in formatting the signal names
for "kill -l" (and in the usage when appropriate -- same routine.)   Respect
COLUMNS (POSIX XBD 8.3) as primary specification of the width (terminal width,
not number of columns to print) for kill -l, a very small value for COLUMNS
will cause kill -l output to list signals one per line, a very large
value will cause them all to be listed on one line.) (eg: "COLUMNS=1 kill -l")
TODO: the signal printing for "trap -l" and that for "kill -l"
should be switched to use a common routine (for the sh builtin versions.)
All changes of relevance here are to bin/kill - the (minor) changes to bin/sh
are only to properly expose the builtin version of getenv(3) so the builtin
version of kill can use it (ie: make its prototype available.)
Properly support EDITRC - use it as (naming) the file when setting
up libedit, and re-do the config whenever EDITRC is set.
Get rid of workarounds for ancient groff html backend.
Simplify macro usage.
Make one example more like a real world possibility (it still isn't, but
is closer) - though the actual content is irrelevant to the point being made.
Add literal prompt support this allows one to do:
CA="$(printf '\1')"
PS1="${CA}$(tput bold)${CA}\$${CA}$(tput sgr0)${CA} "
Now libedit supports embedded mode switch sequence, improve sh
support for them (adds PSlit variable to set the magic character).
NFC: DEBUG only change - provide an externally visible (to the DEBUG sh
internals) interface to one of the internal (private to trace code) functions
Include redirections in trace output from "set -x"
Implement PS1, PS2 and PS4 expansions (variable expansions, arithmetic
expansions, and if enabled by the promptcmds option, command substitutions.)
Implement a bunch of new shell environment variables. many mostly useful
in prompts when expanded at prompt time, but all available for general use.
Many of the new ones are not available in SMALL shells (they work as normal
if assigned, but the shell does not set or use them - and there is no magic
in a SMALL shell (usually for install media.))
Omnibus manual update for prompt expansions and new variables.  Throw in
some random cleanups as a bonus.
Correct a markup typo (why did I not see this before the prev commit??)
Sort options (our default is 0..9AaBbZz).
Fix markup problems and a typo.
Make $- list flags in the same order they appear in sh(1)
Do a better job of detecting the error in pkgsrc/devel/libbson-1.6.3's
configure script, ie: $(( which is intended to be a sub-shell in a
command substitution, but is an arith subst instead, it needs to be
written $( ( to do as intended.   Instead of just blindly carrying on to
find the missing )) somewhere, anywhere, give up as soon as we have seen
an unbalanced ')' that isn't immediately followed by another ')' which
in a valid arith subst it always would be.
While here, there has been a comment in the code for quite a while noting a
difference in the standard between the text descr & grammar when it comes to
the syntax of case statements.   Add more comments to explain why parsing it
as we do is in fact definitely the correct way (ie: the grammar wins arguments
like this...).
DEBUG and white space changes only.   Convert TRACE() calls for DEBUg mode
to the new style.   NFC (when not debugging sh).
Mostly DEBUG and white space changes.   Convert DEEBUG TRACE() calls to
the new format.   Also #if 0 a function definition that is used nowhere.
While here, change the function of pushfile() slightly - it now sets
the buf pointer in the top (new) input descriptor to NULL, instead of
simply leaving it - code that needs a buffer always (before and after)
must malloc() one and assign it after the call.  But code which does not
(which will be reading from a string or similar) now does not have to
explicitly set it to NULL (cleaner interface.)   NFC intended (or observed.)
DEBUG changes: convert DEBUG TRACE() calls to new format.
ALso, cause exec failures to always cause the shell to exit with
status 126 or 127, whatever the cause.  127 is intended for lookup
failures (and is used that way), 126 is used for anything else that
goes wrong (as in several other shells.)  We no longer use 2 (more easily
confused with an exit status of the command exec'd) for shell exec failures.
DEBUG only changes.  Convert the TRACE() calls in the remaining files
that still used it to the new format.   NFC.
Fix a reference after free (and consequent nonsense diagnostic for
attempts to set readonly variables) I added in 1.60 by incompletely
copying the FreeBSD fix for the lost memory issue.

DEBUG changes: convert DEBUG TRACE() calls to new format.
ALso, cause exec failures to always cause the shell to exit with
status 126 or 127, whatever the cause.  127 is intended for lookup
failures (and is used that way), 126 is used for anything else that
goes wrong (as in several other shells.)  We no longer use 2 (more easily
confused with an exit status of the command exec'd) for shell exec failures.

Many internal memory management type fixes.

PR bin/52302   (core dump with interactive shell, here doc and error
on same line) is fixed.   (An old bug.)

echo "$( echo x; for a in $( seq 1000 ); do printf '%s\n'; done; echo y )"
consistently prints 1002 lines (x, 1000 empty ones, then y) as it should
(And you don't want to know what it did before, or why.) (Another old one.)

(Recently added) Problems with ~ expansion fixed (mem management related).

Proper fix for the cwrappers configure problem (which includes the quick
fix that was done earlier, but extends upon that to be correct). (This was
another newly added problem.)

And the really devious (and rare) old bug - if STACKSTRNUL() needs to
allocate a new buffer in which to store the \0, calculate the size of
the string space remaining correctly, unlike when SPUTC() grows the
buffer, there is no actual data being stored in the STACKSTRNUL()
case - the string space remaining was calculated as one byte too few.
That would be harmless, unless the next buffer also filled, in which
case it was assumed that it was really full, not one byte less, meaning
one junk char (a nul, or anything) was being copied into the next (even
bigger buffer) corrupting the data.

Consistent use of stalloc() to allocate a new block of (stack) memory,
and grabstackstr() to claim a block of (stack) memory that had already
been occupied but not claimed as in use.  Since grabstackstr is implemented
as just a call to stalloc() this is a no-op change in practice, but makes
it much easier to comprehend what is really happening.  Previous code
sometimes used stalloc() when the use case was really for grabstackstr().
Change grabstackstr() to actually use the arg passed to it, instead of
(not much better than) guessing how much space to claim,

More care when using unstalloc()/ungrabstackstr() to return space, and in
particular when the stack must be returned to its previous state, rather than
just returning no-longer needed space, neither of those work.  They also don't
work properly if there have been (really, even might have been) any stack mem
allocations since the last stalloc()/grabstackstr().   (If we know there
cannot have been then the alloc/release sequence is kind of pointless.)
To work correctly in general we must use setstackmark()/popstackmark() so
do that when needed.  Have those also save/restore the top of stack string
space remaining.

	[Aside: for those reading this, the "stack" mentioned is not
	in any way related to the thing used for maintaining the C
	function call state, ie: the "stack segment" of the program,
	but the shell's internal memory management strategy.]

More comments to better explain what is happening in some cases.
Also cleaned up some hopelessly broken DEBUG mode data that were
recently added (no effect on anyone but the poor semi-human attempting
to make sense of it...).

User visible changes:

Proper counting of line numbers when a here document is delimited
by a multi-line end-delimiter, as in

	cat << 'REALLY
	END'
	here doc line 1
	here doc line 2
	REALLY
	END

(which is an obscure case, but nothing says should not work.)  The \n
in the end-delimiter of the here doc (the last one) was not incrementing
the line number, which from that point on in the script would be 1 too
low (or more, for end-delimiters with more than one \n in them.)

With tilde expansion:
	unset HOME; echo ~
changed to return getpwuid(getuid())->pw_home instead of failing (returning ~)

POSIX says this is unspecified, which makes it difficult for a script to
compensate for being run without HOME set (as in env -i sh script), so
while not able to be used portably, this seems like a useful extension
(and is implemented the same way by some other shells).

Further, with
	HOME=; printf %s ~
we now write nothing (which is required by POSIX - which requires ~ to
expand to the value of $HOME if it is set) previously if $HOME (in this
case) or a user's directory in the passwd file (for ~user) were a null
STRING, We failed the ~ expansion and left behind '~' or '~user'.

A better LINENO implementation.   This version deletes (well, #if 0's out)
the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)).
(Code to invert the LINENO hack when required, like when de-compiling the
execution tree to provide the "jobs" command strings, is still included,
that can be deleted when the LINENO hack is completely removed - look for
refs to VSLINENO throughout the code.  The var funclinno in parser.c can
also be removed, it is used only for the LINENO hack.)

This version produces accurate results: $((LINENO)) was made as accurate
as the LINENO hack made ${LINENO} which is very good.  That's why the
LINENO hack is not yet completely removed, so it can be easily re-enabled.
If you can tell the difference when it is in use, or not in use, then
something has broken (or I managed to miss a case somewhere.)

The way that LINENO works is documented in its own (new) section in the
man page, so nothing more about that, or the new options, etc, here.

This version introduces the possibility of having a "reference" function
associated with a variable, which gets called whenever the value of the
variable is required (that's what implements LINENO).  There is just
one function pointer however, so any particular variable gets at most
one of the set function (as used for PATH, etc) or the reference function.
The VFUNCREF bit in the var flags indicates which func the variable in
question uses (if any - the func ptr, as before, can be NULL).

I would not call the results of this perfect yet, but it is close.

Pull up following revision(s) (requested by kre in ticket #5):
	bin/sh/cd.c: revision 1.48
	bin/sh/eval.c: revision 1.141
	bin/sh/exec.c: revision 1.48
	bin/sh/exec.h: revision 1.25
	bin/sh/mail.c: revisions 1.17, 1.18
	bin/sh/sh.1: revision 1.147
Make cd (really) do cd -P, and not just claim that is what it is doing
while doing a half-hearted, broken, partial, version of cd -L instead.
The latter (as the manual says) is not supported, what's more, it is an
abomination, and should never be supported (anywhere.)
Fix the doc so that the pretense that we notice when a path given crosses
a symlink (and turns on printing of the destination directory) is claimed
no more (that used to be true until late Dec 2016, but was changed).  Now
the print happens if -o cdprint is set, or if an entry from CDPATH that is
not "" or "." is used (or if the "cd dest repl" cd cmd variant is used.)
Fix CDPATH processing: avoid the magic '%' processing that is used for
PATH and MAILPATH from corrupting CDPATH.  The % magic (both variants)
remains undocumented.
Also, don't double the '/' if an entry in PATH or CDPATH ends in '/'
(as in CDPATH=":/usr/src/").  A "cd usr.bin" used to do
chdir("/usr/src//usr.bin").  No more.  This is almost invisible,
and relatively harmless, either way....
Also fix a bug where if a plausible destination directory in CDPATH
was located, but the chdir() failed (eg: permission denied) and then
a later "." or "" CDPATH entry succeeded, "print" mode was turned on.
That is:
	cd /tmp; mkdir bin
	mkdir -p P/bin; chmod 0 P/bin
	CDPATH=/tmp/P:
	cd bin
would cd to /tmp/bin (correctly) but print it (incorrectly).
Also when in "cd dest replace" mode, if the result of the replacement
generates '-' as the path named, as in:
	cd $PWD -
then simply change to '-' (or attempt to, with CDPATH search), rather
than having this being equivalent to "cd -")
Because of these changes, the pwd command (and $PWD) essentially
always acts as pwd -P, even when called as pwd -L (which is still
the default.)   That is, even more than it did before.
Also fixed a (kind of minor) mem management error (CDPATH related)
"whosoever shall padvance must stunalloc before repeating" (and the
same for MAILPATH).
--
If we are going to keep the MAILPATH % hack, then at least do something
rational.  Since it isn't documented, what "rational" is is up for
discussion, but what it did before was not it (it was nonsense...).

Make cd (really) do cd -P, and not just claim that is what it is doing
while doing a half-hearted, broken, partial, version of cd -L instead.
The latter (as the manual says) is not supported, what's more, it is an
abomination, and should never be supported (anywhere.)

Fix the doc so that the pretense that we notice when a path given crosses
a symlink (and turns on printing of the destination directory) is claimed
no more (that used to be true until late Dec 2016, but was changed).  Now
the print happens if -o cdprint is set, or if an entry from CDPATH that is
not "" or "." is used (or if the "cd dest repl" cd cmd variant is used.)

Fix CDPATH processing: avoid the magic '%' processing that is used for
PATH and MAILPATH from corrupting CDPATH.  The % magic (both variants)
remains undocumented.

Also, don't double the '/' if an entry in PATH or CDPATH ends in '/'
(as in CDPATH=":/usr/src/").  A "cd usr.bin" used to do
chdir("/usr/src//usr.bin").  No more.  This is almost invisible,
and relatively harmless, either way....

Also fix a bug where if a plausible destination directory in CDPATH
was located, but the chdir() failed (eg: permission denied) and then
a later "." or "" CDPATH entry succeeded, "print" mode was turned on.
That is:
	cd /tmp; mkdir bin
	mkdir -p P/bin; chmod 0 P/bin
	CDPATH=/tmp/P:
	cd bin
would cd to /tmp/bin (correctly) but print it (incorrectly).

Also when in "cd dest replace" mode, if the result of the replacement
generates '-' as the path named, as in:
	cd $PWD -
then simply change to '-' (or attempt to, with CDPATH search), rather
than having this being equivalent to "cd -")

Because of these changes, the pwd command (and $PWD) essentially
always acts as pwd -P, even when called as pwd -L (which is still
the default.)   That is, even more than it did before.

Also fixed a (kind of minor) mem management error (CDPATH related)
"whosoever shall padvance must stunalloc before repeating" (and the
same for MAILPATH).

Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)

(Perhaps temporarary) updated "hash" command.  New options, and
more flexible behaviour.

Pull up following revision(s) (requested by dholland in ticket #1412):
	bin/sh/exec.c: revision 1.45
	bin/sh/var.c: revision 1.43
PR/48312: Dieter Roelants: According to TOG, unset should not return an error
for functions are variables that were not previously set:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

add missing forward declaration for the STATIC= case.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

PR/48312: Dieter Roelands: According to TOG, unset should not return an error
for functions are variables that were not previously set:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

resync with head

sync with head

Add support for '%n' being a shorthand for 'fg %n'.

sync with head

Use C89 function definitions

Wrap declaration of a STATIC function that's only conditionally defined
in a suitable ifdef, so things still compile if STATIC is defined as
"static", which is for some reason not the default.

(In the long run STATIC should go away - it might have once been a
portability hack but now definitely serves no purpose.)

sync with HEAD

Fix inconsistent definitions

PR/36531: Greg A. Woods: another very helpful DEBUG TRACE() call for execve()
failures in /bin/sh

Coverity CID 890: Possible NULL pointer deref.

Coverity CID 1329: Possible NULL deref.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22249, verified by myself.

Fix bin/20185 - builtin called from function of same name mustn't be hashed.
Make 'hash' only report utilities that are not builtins (posix), the
non-posix 'hash -v' will report everything.
(agreed by christos)

Support command -p, -v and -V as posix
Stop temporary PATH assigments messing up hash table
Fix sh -c -e "echo $0 $*" -a x (as posix)
(agreed by christos)

Fixes from David Laight:
- ansification
- format of output of jobs command (etc)
- job identiers %+, %- etc
- $? and $(...)
- correct quoting of output of set, export -p and readonly -p
- differentiation between nornal and 'posix special' builtins
- correct behaviour (posix) for errors on builtins and special builtins
- builtin printf and kill
- set -o debug (if compiled with DEBUG)
- cd src obj (as ksh - too useful to do without)
- unset -e name, remove non-readonly variable from export list.
  (so I could unset -e PS1 before running the test shell...)

VFork()ing shell: From elric@netbsd.org:
Plus my changes:
	- walking process group fix in foregrounding a job.
	- reset of process group in parent shell if interrupted before the wait.
	- move INTON lower in the dowait so that the job structure is
	  consistent.
	- error check all setpgid(), tcsetpgrp() calls.
	- eliminate unneeded strpgid() call.
	- check that we don't belong in the process group before we try to
	  set it.

Doing the vfork work on ash on a branch to try to shake out the
problems before I expose everyone to them.  This checkin represents
a merge of the prior work, which I backed out a while ago, to the
HEAD only and does not incorporate any additional bugfixes.  The
additional bugfixes and code-cleanup will occur in later checkins.

For reference the patches that were used are:
cvs diff -kk -r1.51 -r1.55 eval.c  | patch
cvs diff -kk -r1.27 -r1.28 exec.c  | patch
cvs diff -kk -r1.15 -r1.16 exec.h  | patch
cvs diff -kk -r1.32 -r1.33 input.c | patch
cvs diff -kk -r1.10 -r1.11 input.h | patch
cvs diff -kk -r1.32 -r1.35 jobs.c  | patch
cvs diff -kk -r1.9  -r1.11 jobs.h  | patch
cvs diff -kk -r1.36 -r1.37 main.c  | patch
cvs diff -kk -r1.20 -r1.21 redir.c | patch
cvs diff -kk -r1.10 -r1.11 redir.h | patch
cvs diff -kk -r1.10 -r1.12 shell.h | patch
cvs diff -kk -r1.22 -r1.23 trap.c  | patch
cvs diff -kk -r1.12 -r1.13 trap.h  | patch
cvs diff -kk -r1.23 -r1.24 var.c   | patch
cvs diff -kk -r1.16 -r1.17 var.h   | patch

All other changes were simply the resolution of the resulting
conflicts, which occured only in the merge of jobs.c.

Begins to address PR: bin/5475

remove redundant declarations and nexted externs.

Pullup 1.31 [hubertf]:
Fixes bug in 'type' command WRT slashes.

handle type command on names that contain slashes, and print a : in the not
found case. From FreeBSD.

include <stdlib.h>, <string.h>, or whatever as appropriate to shut up
gcc 2.96

Back out previous vfork changes.

Now we use vfork(2) instead of fork(2) when we can.

compile with WARNS = 2

Be more retentive about use of NOTREACHED and noreturn.

Delint.

PR/3888: Chris Demetriou: type command-with-slash prints
	$PATH[0]/command-with-slash...

Fix compiler warnings.

Pull up latest sh(1).  Fixes yet more bugs.

add type builtin.

Update /bin/sh from trunk per request of Christos Zoulas.  Fixes
many bugs.

kill 'register'

Remove leftover printf("here") from last change.

PR/287: Shell does not exit with 126/127 when permission denied/
	command not found. Add extra exception type and generalize
	error handling routines to take that exception type. Use
	a global variable exerrno to keep the last exec error.

const poisoning.

Changed so that 'PATH=newpath command' works, instead of looking at the
old path. Synced input.c with vangogh.

Merge in my changes from vangogh, and fix the x=`false`; echo $? == 0
bug.

convert to new RCS id conventions.

Use S_IS*().

Remove the gratuitous `security' warning.

clean up further.  more patches from Jim Jegers

Eliminate uses of some obsolete functions.

Add RCS ids.

Include appropriate header files to bring function prototypes into scope.

sync with 4.4lite

44lite code

Getgroup's second argument is now a gid_t ptr.

Root can execute anything with at least one execute bit set.

Add RCS identifiers.

from "Andrew A. Chernov, Black Mage" <ache@astral.msk.su>
When sh tried to find_command() for execution (without full path),
it checked ONLY ONE current effective group execution permissions,
but didn't check another valid groups from /etc/group.

changed "Id" to "Header" for rcsids

added rcs ids to all files

initial import of 386bsd-0.1 sources

Initial revision