Up to [cvs.NetBSD.org] / src / bin / sh
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
Apologies for that commit message ... it should have been the same as for revision 1.124 of Makefile (and as below). (This change changes nothing). Reject nul characters in shell input. At the request of Thomas Klausner (wiz@) copy an idea from OpenBSD, and have the shell simply reject any (sh) input containing a \0 (nul) character. Previously nul characters were simply ignored (removed from the input before it was examined in any other way). Note this affects data read by the shell itself only, and has no impact on other utilities, including those that are built into the shell (so 'read' can still use -d '' to process output using \0 as the record separator). While I have tested that this works, there are so many places where a nul might appear, that I cannot possibly test them all (or even imagine all the possible places), so if this change causes any problems, let me know (if from a script, send me the script). To undo this change, simply comment out (or delete) the line CPPFLAGS+= -DREJECT_NULS from the Makefile, and build again. Eventually if this causes no problems, that option (conditional compilation) will probably just go away, and this change would be permanent. While the conditional compilation (on or off) persists, the NETBSD_SHELL variable value will contain the word REJECT_NULLS so you can easily verify if you are running a shell with this included or not. No pullups planned.
# $NetBSD: Makefile,v 1.123 2023/10/19 04:27:24 mrg Exp $ # @(#)Makefile 8.4 (Berkeley) 5/5/95 .include <bsd.own.mk> PROG= sh SHSRCS= alias.c arith_token.c arithmetic.c cd.c echo.c error.c eval.c exec.c \ expand.c histedit.c input.c jobs.c mail.c main.c memalloc.c \ miscbltin.c mystring.c options.c parser.c redir.c show.c trap.c \ output.c var.c test.c kill.c syntax.c GENSRCS=builtins.c init.c nodes.c GENHDRS=builtins.h nodes.h token.h nodenames.h optinit.h SRCS= ${SHSRCS} ${GENSRCS} DPSRCS+=${GENHDRS} LDADD+= -ledit -lterminfo DPADD+= ${LIBEDIT} ${LIBTERMINFO} # Environment for scripts executed during build. SCRIPT_ENV= \ AWK=${TOOL_AWK:Q} \ MKTEMP=${TOOL_MKTEMP:Q} \ SED=${TOOL_SED:Q} CPPFLAGS+=-DSHELL -I. -I${.CURDIR} -I${NETBSDSRCDIR}/lib/libedit CPPFLAGS+= -DUSE_LRAND48 CPPFLAGS+= -DREJECT_NULS #XXX: For testing only. #CPPFLAGS+=-DDEBUG=1 #COPTS+=-g #CFLAGS+=-funsigned-char #TARGET_CHARFLAG?= -DTARGET_CHAR="unsigned char" -funsigned-char # Reproducible build parameters ... export into sh for NETBSD_SHELL setting .if ${MKREPRO_TIMESTAMP:Uno} != "no" BUILD_DATE!= ${TOOL_DATE} -u -r "${MKREPRO_TIMESTAMP}" "+%Y%m%d%H%M%S" # These are (should be) equivalent, but the 2nd is easier to understand #CPPFLAGS+= -DBUILD_DATE='"${BUILD_DATE:C/([^0]0?)(00)*$/\1/}Z"' CPPFLAGS+= -DBUILD_DATE='"${BUILD_DATE:S/00$//:S/00$//:S/00$//}Z"' .endif .ifdef SMALLPROG CPPFLAGS+=-DSMALL .endif .ifdef TINYPROG CPPFLAGS+=-DTINY .else SRCS+=printf.c .endif .PATH: ${.CURDIR}/bltin ${NETBSDSRCDIR}/bin/test \ ${NETBSDSRCDIR}/usr.bin/printf \ ${NETBSDSRCDIR}/bin/kill CLEANFILES+= ${GENSRCS} ${GENHDRS} sh.html1 CLEANFILES+= trace.* token.h: mktokens ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} .ORDER: builtins.h builtins.c builtins.h builtins.c: mkbuiltins shell.h builtins.def ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} ${.OBJDIR} [ -f builtins.h ] init.c: mkinit.sh ${SHSRCS} ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} .ORDER: nodes.h nodes.c nodes.c nodes.h: mknodes.sh nodetypes nodes.c.pat ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} ${.OBJDIR} [ -f nodes.h ] nodenames.h: mknodenames.sh nodes.h ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} > ${.TARGET} optinit.h: mkoptions.sh option.list ${_MKTARGET_CREATE} ${SCRIPT_ENV} ${HOST_SH} ${.ALLSRC} ${.TARGET} ${.OBJDIR} .if ${USETOOLS} == "yes" NBCOMPATLIB= -L${TOOLDIR}/lib -lnbcompat .endif SUBDIR.roff+=USD.doc COPTS.printf.c = -Wno-format-nonliteral COPTS.jobs.c = -Wno-format-nonliteral COPTS.var.c = -Wno-format-nonliteral # XXXGCC12 - only on some targets COPTS.parser.c+= ${${ACTIVE_CC} == "gcc" && ${HAVE_GCC:U0} >= 12:? -Wno-error=clobbered :} .include <bsd.prog.mk> .include <bsd.subdir.mk> ${OBJS}: Makefile
Change the "string" argument to evalstring() and setinputstring() from being "char *" to being "const char *". This is needed for a forthcoming change which needs to pass a const char * to evalstring (and through it to setinputstring) and be assured that nothing will alter the characters in the string supplied. This is (aside from the additional compile time protection provided) a no-op change, all evalstring() does with its string is pass it to setinputstring() and all that does with it is determine its length (strlen() which expects a const char *) and assign the string pointer to parsenextc which is already a const char * - there never has been any reason for these two functions to not include the "const" in the arg declaration -- except that when originally written (early 1990's) I suspect "const" either didn't exist at all, or wasn't supported by relevant compilers. NFCI. Most probably (though I didn't check) no binary change at all.
Implement the HISTFILE and HISTAPPEND variables. See the (newly updated) sh(1) for details. Also add the -z option to fc (clear history). None of this exists in SMALL shells.
Pull up following revision(s) (requested by kre in ticket #1242): bin/sh/input.c: revision 1.72 bin/sh/exec.c: revision 1.55 PR bin/55979 This fixes the MSAN detected reference to an unitialised variable (an unitialised field in a struct) which happens when a command is not found after a PATH search. Aside from skipping some known to be going to fail exec*() calls in some cases, the setting of the relevant field is irrelevant, so this problem makes no practical difference to the shell, or any shell script. XXX (maybe) pullup -9 PR bin/55979 Correctly handle (ie: ignore completely) \0 chars (nuls) in the shell command input stream (script, dot file, or stdin). Previously nul chars were ignored correctly in the line in which they occurred, but would cause trailing chars of that line to reappear as the start of the following line. If there was just one \0 skipped, this would generally result in an extra \n in the sh input, which in most cases has no effect. With multiple \0's in a single line, more of the end of that line was duplicated into the following one. This usually manifested as a weird "command not found" error. Note that any \0 chars in the sh input make the script non-conforming, so fixing this is not crucial (no \0's should really ever be seen) but it was an obvious bug in the code, which was attempting to ignore nul chars (as do many other shells), so let it be fixed. XXX pullup -9
PR bin/55979 Correctly handle (ie: ignore completely) \0 chars (nuls) in the shell command input stream (script, dot file, or stdin). Previously nul chars were ignored correctly in the line in which they occurred, but would cause trailing chars of that line to reappear as the start of the following line. If there was just one \0 skipped, this would generally result in an extra \n in the sh input, which in most cases has no effect. With multiple \0's in a single line, more of the end of that line was duplicated into the following one. This usually manifested as a weird "command not found" error. Note that any \0 chars in the sh input make the script non-conforming, so fixing this is not crucial (no \0's should really ever be seen) but it was an obvious bug in the code, which was attempting to ignore nul chars (as do many other shells), so let it be fixed. XXX pullup -9
Ooops, restore accidently removed files from merge mishap
Sync with HEAD
Sync with HEAD
KNF - white space changes, indent using tabs not spaces. NFC.
INTON / INTOFF audit and cleanup. No visible differences expected - there is a remote chance that some internal lossage may no longer occur in interactive shells that receive SIGINT (untrapped) at inopportune times, but you would have had to have been very unlucky to have ever suffered from that.
Synch with HEAD
Redo 1.65 in a simpler way. This is the bit rot avoidance code that is #if 0'd and (still) has never been compiled (most likely never will be.) While here, in the same uncompiled code, deal with line number counting. Whether this is correct depends upon how this code is used, and as it never is (and never has been since line numbers first started being counted), this is somewhat speculative, but it seems likely to be the correct way to handle things. NFC (this code is still all #if 0).
Don't use quoteflag when deciding if the word after an alias should be looked up as a potential following alias - if the first expands to a string that ends with a space (any space, quoted or not) then the next word is to be treated as an alias candidate. (POSIX was to specify only unquoted spaces, but is now going to leave that unspecified, and the "any space" version turns out to be more useful. And besides, the quoteflag test didn't work properly, and would have been very messy to fix ... if in a word (as if we have a quoted space) it means that the word has been quoted, which meant that quoted spaces were correctly detected, but it outside a word, it just means that the previous word was quoted, so it would sometimes reject alias lookup on the next word in cases where it is unquestioned it should be done.
Correct an (old) typo in a comment. NFC - it is just a comment.
Fix the code taken from FreeBSD 2 revisions back, which fixed aliases, to actually do what it was supposed to do, and not just come close by accident. (How broken this was, while still seeming to work perfectly most of the time was truly amazing!) This corrects the behaviour of an alias defined with a blank char as the last of its value, to correctly do an alias lookup on the word that follows the alias.
Update some dead (#if 0'd) code that is never called to cope with the changes made in the previous revision, in an attempt to avoid bit rot. Untested (uncompiled) - though it should work. NFC: this change doesn't get compiled, let alone used.
Sync with HEAD, resolve a few conflicts
Revamp aliases - as dumb an idea as they are, if we're going to have them, they should work as documented, not cause core dumps, reference after free, incorrect replacements, failing to implement alias after alias, ... The big comment that ended: This is a good idea ------- ***NOT*** and the hack it was describing are gone. Note that most of this was from original CVS version 1.1 code (ie: came from the original import, even before 4.4-Lite was merged. That is, May 1994. And no-one in 24.5 years noticed (or at least complained about) all the bugs (or at least, most of them)). With these changes, aliases ought to work (if you can call it that) as they are expected to by POSIX. Now if only we could get POSIX to delete them (or make them optional)... Changes partly inspired by similar changes made by FreeBSD, (as was the previous change to alias.c, forgot ack in commit log for that one, apologies) but done a little differently, and perhaps with a slightly better outcome.
Sync with HEAD Resolve a couple of conflicts (result of the uimin/uimax changes)
PR bin/48875 (is related, and ameliorated, but not exactly "fixed") Import a whole set of tree evaluation enhancements from FreeBSD. With these, before forking, the shell predicts (often) when all it will have to do after forking (in the parent) is wait for the child and then exit with the status from the child, and in such a case simply does not fork, but rather allows the child to take over the parent's role. This turns out to handle the particular test case from PR bin/48875 in such a way that it works as hoped, rather than as it did (the delay there was caused by an extra copy of the shell hanging around waiting for the background child to complete ... and keeping the command substitution stdout open, so the "real" parent had to wait in case more output appeared). As part of doing this, redirection processing for compound commands gets moved out of evalsubshell() and into a new evalredir(), which allows us to properly handle errors occurring while performing those redirects, and not mishandle (as in simply forget) fd's which had been moved out of the way temporarily. evaltree() has its degree of recursion reduced by making it loop to handle the subsequent operation: that is instead of (for any binop like ';' '&&' (etc)) where it used to evaltree(node->left); evaltree(node->right); return; it now does (kind of) next = node; while ((node = next) != NULL) { next = NULL; if (node is a binary op) { evaltree(node->left); if appropriate /* if && test for success, etc */ next = node->right; continue; } /* similar for loops, etc */ } which can be a good saving, as while the left side (now) tends to be (usually) a simple (or simpleish) command, the right side can be many commands (in a command sequence like a; b; c; d; ... the node at the top of the tree will now have "a" as its left node, and the tree for b; c; d; ... as its right node - until now everything was evaluated recursively so it made no difference, and the tree was constructed the other way). if/while/... statements are done similarly, recurse to evaluate the condition, then if the (or one of the) body parts is to be evaluated, set next to that, and loop (previously it recursed). There is more to do in this area (particularly in the way that case statements are processed - we can avoid recursion there as well) but that can wait for another day. While doing all of this we keep much better track of when the shell is just going to exit once the current tree is evaluated (with a new predicate at_eof() to tell us that we have, for sure, reached the end of the input stream, that is, this shell will, for certain, not be reading more command input) and use that info to avoid unneeded forks. For that we also need another new predicate (have_traps()) to determine of there are any caught traps which might occur - if there are, we need to remain to (potentially) handle them, so these optimisations will not occur (to make the issue in PR 48875 appear again, run the same code, but with a trap set to execute some code when a signal (or EXIT) occurs - note that the trap must be set in the appropriate level of sub-shell to have this effect, any caught traps are cleared in a subshell whenever one is created). There is still work to be done to handle traps properly, whatever weirdness they do (some of which is related to some of this.) These changes do not need man page updates, but 48875 does - an update to sh.1 will be forthcoming once it is decided what it should say... Once again, all the heavy lifting for this set of changes comes directly (with thanks) from the FreeBSD shell. XXX pullup-8 (but not very soon)
NFC - DEBUG mode change only - add some sanity to a debug printf format string
Pull up following revision(s) (requested by kre in ticket #199): bin/sh/input.c: revision 1.61 bin/sh/parser.c: revision 1.143 PR bin/52458 Avoid mangling history when editing is enabled, and the prompt contains a \n Also, allow empty input lines into history when they are being appended to a previous (partial) command (but not when they would just make an empty entry) . For all the gory details, see the PR. Note nothing here actually makes prompts containing \n work correctly when editing is enabled, that's a libedit issue, which will be addressed some other time.
PR bin/52458 Avoid mangling history when editing is enabled, and the prompt contains a \n Also, allow empty input lines into history when they are being appended to a previous (partial) command (but not when they would just make an empty entry). For all the gory details, see the PR. Note nothing here actually makes prompts containing \n work correctly when editing is enabled, that's a libedit issue, which will be addressed some other time.
Pull up following revision(s) (requested by kre in ticket #103): bin/kill/kill.c: 1.28 bin/sh/Makefile: 1.111-1.113 bin/sh/arith_token.c: 1.5 bin/sh/arith_tokens.h: 1.2 bin/sh/arithmetic.c: 1.3 bin/sh/arithmetic.h: 1.2 bin/sh/bltin/bltin.h: 1.15 bin/sh/cd.c: 1.49-1.50 bin/sh/error.c: 1.40 bin/sh/eval.c: 1.142-1.151 bin/sh/exec.c: 1.49-1.51 bin/sh/exec.h: 1.26 bin/sh/expand.c: 1.113-1.119 bin/sh/expand.h: 1.23 bin/sh/histedit.c: 1.49-1.52 bin/sh/input.c: 1.57-1.60 bin/sh/input.h: 1.19-1.20 bin/sh/jobs.c: 1.86-1.87 bin/sh/main.c: 1.71-1.72 bin/sh/memalloc.c: 1.30 bin/sh/memalloc.h: 1.17 bin/sh/mknodenames.sh: 1.4 bin/sh/mkoptions.sh: 1.3-1.4 bin/sh/myhistedit.h: 1.12-1.13 bin/sh/nodetypes: 1.16-1.18 bin/sh/option.list: 1.3-1.5 bin/sh/parser.c: 1.133-1.141 bin/sh/parser.h: 1.22-1.23 bin/sh/redir.c: 1.58 bin/sh/redir.h: 1.24 bin/sh/sh.1: 1.149-1.159 bin/sh/shell.h: 1.24 bin/sh/show.c: 1.43-1.47 bin/sh/show.h: 1.11 bin/sh/syntax.c: 1.4 bin/sh/syntax.h: 1.8 bin/sh/trap.c: 1.41 bin/sh/var.c: 1.56-1.65 bin/sh/var.h: 1.29-1.35 An initial attempt at implementing LINENO to meet the specs. Aside from one problem (not too hard to fix if it was ever needed) this version does about as well as most other shell implementations when expanding $((LINENO)) and better for ${LINENO} as it retains the "LINENO hack" for the latter, and that is very accurate. Unfortunately that means that ${LINENO} and $((LINENO)) do not always produce the same value when used on the same line (a defect that other shells do not share - aside from the FreeBSD sh as it is today, where only the LINENO hack exists and so (like for us before this commit) $((LINENO)) is always either 0, or at least whatever value was last set, perhaps by LINENO=${LINENO} which does actually work ... for that one line...) This could be corrected by simply removing the LINENO hack (look for the string LINENO in parser.c) in which case ${LINENO} and $((LINENO)) would give the same (not perfectly accurate) values, as do most other shells. POSIX requires that LINENO be set before each command, and this implementation does that fairly literally - except that we only bother before the commands which actually expand words (for, case and simple commands). Unfortunately this forgot that expansions also occur in redirects, and the other compound commands can also have redirects, so if a redirect on one of the other compound commands wants to use the value of $((LINENO)) as a part of a generated file name, then it will get an incorrect value. This is the "one problem" above. (Because the LINENO hack is still enabled, using ${LINENO} works.) This could be fixed, but as this version of the LINENO implementation is just for reference purposes (it will be superseded within minutes by a better one) I won't bother. However should anyone else decide that this is a better choice (it is probably a smaller implementation, in terms of code & data space then the replacement, but also I would expect, slower, and definitely less accurate) this defect is something to bear in mind, and fix. This version retains the *BSD historical practice that line numbers in functions (all functions) count from 1 from the start of the function, and elsewhere, start from 1 from where the shell started reading the input file/stream in question. In an "eval" expression the line number starts at the line of the "eval" (and then increases if the input is a multi-line string). Note: this version is not documented (beyond as much as LINENO was before) hence this slightly longer than usual commit message. A better LINENO implementation. This version deletes (well, #if 0's out) the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)). (Code to invert the LINENO hack when required, like when de-compiling the execution tree to provide the "jobs" command strings, is still included, that can be deleted when the LINENO hack is completely removed - look for refs to VSLINENO throughout the code. The var funclinno in parser.c can also be removed, it is used only for the LINENO hack.) This version produces accurate results: $((LINENO)) was made as accurate as the LINENO hack made ${LINENO} which is very good. That's why the LINENO hack is not yet completely removed, so it can be easily re-enabled. If you can tell the difference when it is in use, or not in use, then something has broken (or I managed to miss a case somewhere.) The way that LINENO works is documented in its own (new) section in the man page, so nothing more about that, or the new options, etc, here. This version introduces the possibility of having a "reference" function associated with a variable, which gets called whenever the value of the variable is required (that's what implements LINENO). There is just one function pointer however, so any particular variable gets at most one of the set function (as used for PATH, etc) or the reference function. The VFUNCREF bit in the var flags indicates which func the variable in question uses (if any - the func ptr, as before, can be NULL). I would not call the results of this perfect yet, but it is close. Unbreak (at least) i386 build .... I have no idea why this built for me on amd64 (problem was missing prototype for snprintf witout <stdio.h>) While here, add some (DEBUG mode only) tracing that proved useful in solving another problem. Set the line number before expanding args, not after. As the line_number would have usually been set earlier, this change is mostly an effective no-op, but it is better this way (just in case) - not observed to have caused any problems. Undo some over agressive fixes for a (pre-commit) bug that did not need these changes to be fixed - and these cause problems in another absurd use case. Either of these issues is unlikely to be seen by anyone who isn't an idiot masochist... PR bin/52280 removescapes_nl in expari() even when not quoted, CRTNONL's appear regardless of quoting (unlike CTLESC). New sentence, new line. Whitespace. Improve the (new) LINENO section, markup changes (with thanks to wiz@ for assistace) and some better wording in a few placed. I am an idiot... revert the previous unintended commit. Remove some left over baggage from the LINENO v1 implementation that didn't get removed with v2, and should have. This would have had (I think, without having tested it) one very minor effect on the way LINENO worked in the v2 implementation, but my guess is it would have taken a long time before anyone noticed... Correct spelling in comments of DEBUG only code... (Perhaps) temporary fix to pkgtools (cwrappers) build (configure). Expanding `` containing \ \n sequences looks to have been giving problems. I don't think this is the correct fix, but it will do no worse harm than (perhaps) incorrectly calculating LINENO in this kind of (rare) circumstance. I'll look and see if there should be a better fix later. s/volatile/const/ -- wonderful how opposites attract like this. NFC (normal use) - DEBUG only change, when showing empty arg list don't omit terminating \n. Free stack memory in a couple of obscure cases where it wasn't being done (one in probably dead code that is never compiled, the other in a very rare error case.) Since it is stack memory it wasn't lost in any case, just held longer than needed. Many internal memory management type fixes. PR bin/52302 (core dump with interactive shell, here doc and error on same line) is fixed. (An old bug.) echo "$( echo x; for a in $( seq 1000 ); do printf '%s\n'; done; echo y )" consistently prints 1002 lines (x, 1000 empty ones, then y) as it should (And you don't want to know what it did before, or why.) (Another old one.) (Recently added) Problems with ~ expansion fixed (mem management related). Proper fix for the cwrappers configure problem (which includes the quick fix that was done earlier, but extends upon that to be correct). (This was another newly added problem.) And the really devious (and rare) old bug - if STACKSTRNUL() needs to allocate a new buffer in which to store the \0, calculate the size of the string space remaining correctly, unlike when SPUTC() grows the buffer, there is no actual data being stored in the STACKSTRNUL() case - the string space remaining was calculated as one byte too few. That would be harmless, unless the next buffer also filled, in which case it was assumed that it was really full, not one byte less, meaning one junk char (a nul, or anything) was being copied into the next (even bigger buffer) corrupting the data. Consistent use of stalloc() to allocate a new block of (stack) memory, and grabstackstr() to claim a block of (stack) memory that had already been occupied but not claimed as in use. Since grabstackstr is implemented as just a call to stalloc() this is a no-op change in practice, but makes it much easier to comprehend what is really happening. Previous code sometimes used stalloc() when the use case was really for grabstackstr(). Change grabstackstr() to actually use the arg passed to it, instead of (not much better than) guessing how much space to claim, More care when using unstalloc()/ungrabstackstr() to return space, and in particular when the stack must be returned to its previous state, rather than just returning no-longer needed space, neither of those work. They also don't work properly if there have been (really, even might have been) any stack mem allocations since the last stalloc()/grabstackstr(). (If we know there cannot have been then the alloc/release sequence is kind of pointless.) To work correctly in general we must use setstackmark()/popstackmark() so do that when needed. Have those also save/restore the top of stack string space remaining. [Aside: for those reading this, the "stack" mentioned is not in any way related to the thing used for maintaining the C function call state, ie: the "stack segment" of the program, but the shell's internal memory management strategy.] More comments to better explain what is happening in some cases. Also cleaned up some hopelessly broken DEBUG mode data that were recently added (no effect on anyone but the poor semi-human attempting to make sense of it...). User visible changes: Proper counting of line numbers when a here document is delimited by a multi-line end-delimiter, as in cat << 'REALLY END' here doc line 1 here doc line 2 REALLY END (which is an obscure case, but nothing says should not work.) The \n in the end-delimiter of the here doc (the last one) was not incrementing the line number, which from that point on in the script would be 1 too low (or more, for end-delimiters with more than one \n in them.) With tilde expansion: unset HOME; echo ~ changed to return getpwuid(getuid())->pw_home instead of failing (returning ~) POSIX says this is unspecified, which makes it difficult for a script to compensate for being run without HOME set (as in env -i sh script), so while not able to be used portably, this seems like a useful extension (and is implemented the same way by some other shells). Further, with HOME=; printf %s ~ we now write nothing (which is required by POSIX - which requires ~ to expand to the value of $HOME if it is set) previously if $HOME (in this case) or a user's directory in the passwd file (for ~user) were a null STRING, We failed the ~ expansion and left behind '~' or '~user'. Changed the long name for the -L option from lineno_fn_relative to local_lineno as the latter seemed to be marginally more popular, and perhaps more importantly, is the same length as the peviously existing quietprofile option, which means the man page indentation for the list of options can return to (about) what it was before... (That is, less indented, which means more data/line, which means less lines of man page - a good thing!) Cosmetic changes to variable flags - make their values more suited to my delicate sensibilities... (NFC). Arrange not to barf (ever) if some turkey makes _ readonly. Do this by adding a VNOERROR flag that causes errors in var setting to be ignored (intended use is only for internal shell var setting, like of "_"). (nb: invalid var name errors ignore this flag, but those should never occur on a var set by the shell itself.) From FreeBSD: don't simply discard memory if a variable is not set for any reason (including because it is readonly) if the var's value had been malloc'd. Free it instead... NFC - DEBUG changes, update this to new TRACE method. KNF - white space and comment formatting. NFC - DEBUG mode only change - convert this to the new TRACE() format. NFC - DEBUG mode only change - complete a change made earlier (marking the line number when included in the trace line tag to show whether it comes from the parser, or the elsewhere as they tend to be quite different). Initially only one case was changed, while I pondered whether I liked it or not. Now it is all done... Also when there is a line tag at all, always include the root/sub-shell indicator character, not only when the pid is included. NFC: DEBUG related comment change - catch up with reality. NFC: DEBUG mode only change. Fix botched cleanup of one TRACE(). "b" more forgiving when sorting options to allow reasonable (and intended) flexibility in option.list format. Changes nothing for current option.list. Now that excessive use of STACKSTRNUL has served its purpose (well, accidental purpose) in exposing the bug in its implementation, go back to not using it when not needed for DEBUG TRACE purposes. This change should have no practical effect on either a DEBUG shell (where the STACKSTRNUL() calls remain) or a non DEBUG shell where they are not needed. Correct the initial line number used for processing -c arg strings. (It was inheriting the value from end of profile file processing) - I didn't notice before as I usually test with empty or no profile files to avoid complications. Trivial change which should have very limited impact. Fix from FreeBSD (applied there in July 2008...) Don't dump core with input like sh -c 'x=; echo >&$x' - that is where the word after a >& or <& redirect expands to nothing at all. Another fix from FreeBSD (this one from April 2009). When processing a string (as in eval, trap, or sh -c) don't allow trailing \n's to destroy the exit status of the last command executed. That is: sh -c 'false ' echo $? should produce 1, not 0. It is amazing what nonsense appears to work sometimes... (all my nonsense too!) Two bugs here, one benign because of the way the script is used. The other hidden by NetBSD's sort being stable, and the data not really requiring sorting at all... So as it happens these fixes change nothing, but they are needed anyway. (The contents of the generated file are only used in DEBUG shells, so this is really even less important than it seems.) Another ancient (highly improbable) bug bites the dust. This one caused by incorrect macro usage (ie: using the wrong one) which has been in the sources since version 1.1 (ie: forever). Like the previous (STACKSTRNUL) bug, the probability of this one actually occurring has been infinitesimal but the LINENO code increases that to infinitesimal and a smidgen... (or a few, depending upon usage). Still, apparently that was enough, Kamil Rytarowski discovered that the zsh configure script (damn competition!) managed to trigger this problem. source .editrc after we initialize so that commands persist! Make arg parsing in kill POSIX compatible with POSIX (XBD 2.12) by parsing the way getopt(3) would, if only it could handle the (required) -signumber and -signame options. This adds two "features" to kill, -ssigname and -lstatus now work (ie: one word with all of the '-', the option letter, and its value) and "--" also now works (kill -- -pid1 pid2 will not attempt to send the pid1 signal to pid2, but rather SIGTERM to the pid1 process group and pid2). It is still the case that (apart from --) at most 1 option is permitted (-l, -s, -signame, or -signumber.) Note that we now have an ambiguity, -sname might mean "-s name" or send the signal "sname" - if one of those turns out to be valid, that will be accepted, otherwise the error message will indicate that "sname" is not a valid signal name, not that "name" is not. Keeping the "-s" and signal name as separate words avoids this issue. Also caution: should someone be weird enough to define a new signal name (as in the part after SIG) which is almost the same name as an existing name that starts with 'S' by adding an extra 'S' prepended (eg: adding a SIGSSYS) then the ambiguity problem becomes much worse. In that case "kill -ssys" will be resolved in favour of the "-s" flag being used (the more modern syntax) and would send a SIGSYS, rather that a SIGSSYS. So don't do that. While here, switch to using signalname(3) (bye bye NSIG, et. al.), add some constipation, and show a little pride in formatting the signal names for "kill -l" (and in the usage when appropriate -- same routine.) Respect COLUMNS (POSIX XBD 8.3) as primary specification of the width (terminal width, not number of columns to print) for kill -l, a very small value for COLUMNS will cause kill -l output to list signals one per line, a very large value will cause them all to be listed on one line.) (eg: "COLUMNS=1 kill -l") TODO: the signal printing for "trap -l" and that for "kill -l" should be switched to use a common routine (for the sh builtin versions.) All changes of relevance here are to bin/kill - the (minor) changes to bin/sh are only to properly expose the builtin version of getenv(3) so the builtin version of kill can use it (ie: make its prototype available.) Properly support EDITRC - use it as (naming) the file when setting up libedit, and re-do the config whenever EDITRC is set. Get rid of workarounds for ancient groff html backend. Simplify macro usage. Make one example more like a real world possibility (it still isn't, but is closer) - though the actual content is irrelevant to the point being made. Add literal prompt support this allows one to do: CA="$(printf '\1')" PS1="${CA}$(tput bold)${CA}\$${CA}$(tput sgr0)${CA} " Now libedit supports embedded mode switch sequence, improve sh support for them (adds PSlit variable to set the magic character). NFC: DEBUG only change - provide an externally visible (to the DEBUG sh internals) interface to one of the internal (private to trace code) functions Include redirections in trace output from "set -x" Implement PS1, PS2 and PS4 expansions (variable expansions, arithmetic expansions, and if enabled by the promptcmds option, command substitutions.) Implement a bunch of new shell environment variables. many mostly useful in prompts when expanded at prompt time, but all available for general use. Many of the new ones are not available in SMALL shells (they work as normal if assigned, but the shell does not set or use them - and there is no magic in a SMALL shell (usually for install media.)) Omnibus manual update for prompt expansions and new variables. Throw in some random cleanups as a bonus. Correct a markup typo (why did I not see this before the prev commit??) Sort options (our default is 0..9AaBbZz). Fix markup problems and a typo. Make $- list flags in the same order they appear in sh(1) Do a better job of detecting the error in pkgsrc/devel/libbson-1.6.3's configure script, ie: $(( which is intended to be a sub-shell in a command substitution, but is an arith subst instead, it needs to be written $( ( to do as intended. Instead of just blindly carrying on to find the missing )) somewhere, anywhere, give up as soon as we have seen an unbalanced ')' that isn't immediately followed by another ')' which in a valid arith subst it always would be. While here, there has been a comment in the code for quite a while noting a difference in the standard between the text descr & grammar when it comes to the syntax of case statements. Add more comments to explain why parsing it as we do is in fact definitely the correct way (ie: the grammar wins arguments like this...). DEBUG and white space changes only. Convert TRACE() calls for DEBUg mode to the new style. NFC (when not debugging sh). Mostly DEBUG and white space changes. Convert DEEBUG TRACE() calls to the new format. Also #if 0 a function definition that is used nowhere. While here, change the function of pushfile() slightly - it now sets the buf pointer in the top (new) input descriptor to NULL, instead of simply leaving it - code that needs a buffer always (before and after) must malloc() one and assign it after the call. But code which does not (which will be reading from a string or similar) now does not have to explicitly set it to NULL (cleaner interface.) NFC intended (or observed.) DEBUG changes: convert DEBUG TRACE() calls to new format. ALso, cause exec failures to always cause the shell to exit with status 126 or 127, whatever the cause. 127 is intended for lookup failures (and is used that way), 126 is used for anything else that goes wrong (as in several other shells.) We no longer use 2 (more easily confused with an exit status of the command exec'd) for shell exec failures. DEBUG only changes. Convert the TRACE() calls in the remaining files that still used it to the new format. NFC. Fix a reference after free (and consequent nonsense diagnostic for attempts to set readonly variables) I added in 1.60 by incompletely copying the FreeBSD fix for the lost memory issue.
Mostly DEBUG and white space changes. Convert DEEBUG TRACE() calls to the new format. Also #if 0 a function definition that is used nowhere. While here, change the function of pushfile() slightly - it now sets the buf pointer in the top (new) input descriptor to NULL, instead of simply leaving it - code that needs a buffer always (before and after) must malloc() one and assign it after the call. But code which does not (which will be reading from a string or similar) now does not have to explicitly set it to NULL (cleaner interface.) NFC intended (or observed.)
Implement PS1, PS2 and PS4 expansions (variable expansions, arithmetic expansions, and if enabled by the promptcmds option, command substitutions.)
A better LINENO implementation. This version deletes (well, #if 0's out) the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)). (Code to invert the LINENO hack when required, like when de-compiling the execution tree to provide the "jobs" command strings, is still included, that can be deleted when the LINENO hack is completely removed - look for refs to VSLINENO throughout the code. The var funclinno in parser.c can also be removed, it is used only for the LINENO hack.) This version produces accurate results: $((LINENO)) was made as accurate as the LINENO hack made ${LINENO} which is very good. That's why the LINENO hack is not yet completely removed, so it can be easily re-enabled. If you can tell the difference when it is in use, or not in use, then something has broken (or I managed to miss a case somewhere.) The way that LINENO works is documented in its own (new) section in the man page, so nothing more about that, or the new options, etc, here. This version introduces the possibility of having a "reference" function associated with a variable, which gets called whenever the value of the variable is required (that's what implements LINENO). There is just one function pointer however, so any particular variable gets at most one of the set function (as used for PATH, etc) or the reference function. The VFUNCREF bit in the var flags indicates which func the variable in question uses (if any - the func ptr, as before, can be NULL). I would not call the results of this perfect yet, but it is close.
An initial attempt at implementing LINENO to meet the specs. Aside from one problem (not too hard to fix if it was ever needed) this version does about as well as most other shell implementations when expanding $((LINENO)) and better for ${LINENO} as it retains the "LINENO hack" for the latter, and that is very accurate. Unfortunately that means that ${LINENO} and $((LINENO)) do not always produce the same value when used on the same line (a defect that other shells do not share - aside from the FreeBSD sh as it is today, where only the LINENO hack exists and so (like for us before this commit) $((LINENO)) is always either 0, or at least whatever value was last set, perhaps by LINENO=${LINENO} which does actually work ... for that one line...) This could be corrected by simply removing the LINENO hack (look for the string LINENO in parser.c) in which case ${LINENO} and $((LINENO)) would give the same (not perfectly accurate) values, as do most other shells. POSIX requires that LINENO be set before each command, and this implementation does that fairly literally - except that we only bother before the commands which actually expand words (for, case and simple commands). Unfortunately this forgot that expansions also occur in redirects, and the other compound commands can also have redirects, so if a redirect on one of the other compound commands wants to use the value of $((LINENO)) as a part of a generated file name, then it will get an incorrect value. This is the "one problem" above. (Because the LINENO hack is still enabled, using ${LINENO} works.) This could be fixed, but as this version of the LINENO implementation is just for reference purposes (it will be superseded within minutes by a better one) I won't bother. However should anyone else decide that this is a better choice (it is probably a smaller implementation, in terms of code & data space then the replacement, but also I would expect, slower, and definitely less accurate) this defect is something to bear in mind, and fix. This version retains the *BSD historical practice that line numbers in functions (all functions) count from 1 from the start of the function, and elsewhere, start from 1 from where the shell started reading the input file/stream in question. In an "eval" expression the line number starts at the line of the "eval" (and then increases if the input is a multi-line string). Note: this version is not documented (beyond as much as LINENO was before) hence this slightly longer than usual commit message.
Sync with HEAD
Another fix from FreeBSD. I'm not sure how to trigger the problem fixed (there might be no way) - but it "feels right"! When popping an (exhausted) input string off the input stack, allow for the possibility that the previous string might also just happened to have run out of steam as well, so keep poppin' along until we run out of pop, or find something to consume.
Deal with \newline line continuations more correctly. They can occur anywhere (*anywhere*) not only where it happens to be convenient to the parser... This fix from FreeBSD (thanks again folks). To make this work, pushstring()'s signature needed to change to allow a const char * as its string arg, which meant sprinkling some const other places for a brighter appearance (and handling fallout). All this because I wanted to see what number would come from echo $\ {\ L\ I\ N\ E\ N\ O\ } and was surprised at the result! That works now... The bug would also affect stuff like true &\ & false and all kinds of other uses where the \newline occurred in the "wrong" place. An ATF test for sh syntax is coming... (sometime.)
Fix idiot typos in previous (this is not the advertised :next commit") Same typo - two different places. Ugh!
NFC: Change prototype of pushstring() to give a real type for the 3rd arg (struct alias *) rather than using void * and then casting it when used. For callers, the arg either is a struct alias *, or is NULL, so nothing to adjust there. NB: This change untested by itself, it was going to be a part of the next change (coming in a few minutes) but is logically unrelated, so ...
Sync with HEAD - tag prg-localcount2-base1
Keep track of which file descriptors the shell is using for its own purposes, and move them elsewhere whenever a user redirection happens to pick the same number. With this we can move the shell file descriptors back to lower values (be slightly kinder to the kernel) since we can no longer clash. (Also get rid of a little old unneeded code.) This also completes the fdflags command, which no longer permits access to (by way or either obtaining, or changing) the shell's internal fds.
PR bin/51207 Only check for ELF bnaries in regular files.
PR bin/51119 - don't leak FDs in unusual error cases. OK christos@
Fix handing of user file descriptors outside the 0..9 range. Also, move (most of) the shell's internal use fd's to much higher values (depending upon what ulimit -n allows) so they are less likely to clash with user supplied fd numbers. A future patch will (hopefully) avoid this problem completely by dynamically moving the shell's internal fds around as needed. (From kre@)
General KNF and source code cleanups, avoid scattering the magic string " \t\n" all over the place, slightly improved syntax error messages, restructured some of the code for clarity, don't allow IFS to be imported through the environment, and remove the (never) conditionally compiled ATTY option. Apart from one or two syntax error messages, and ignoring IFS if present in the environment, this is intended to have no user visible changes. (from kre@)
Don't leak redirected rescriptors to exec'ed processes. This is what ksh does, but bash does not. For example: $ cat test1 #!/bin/sh exec 6> out echo "test" >&6 sh ./test2 exec 6>&- $ cat test2 echo "test2" >&6 $ ./test1 ./test2: 6: Bad file descriptor This fixes by side effect the problem of the rc system leaking file descriptors 7 and 8 to all starting daemons: $ fstat -p 1359 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root powerd 1359 wd / 2 drwxr-xr-x 512 r root powerd 1359 0 / 63029 crw-rw-rw- null rw root powerd 1359 1 / 63029 crw-rw-rw- null rw root powerd 1359 2 / 63029 crw-rw-rw- null rw root powerd 1359 3* kqueue pending 0 root powerd 1359 4 / 64463 crw-r----- power r root powerd 1359 7 flags 0x80034<ISTTY,MPSAFE,LOCKSWORK,CLEAN> root powerd 1359 8 flags 0x80034<ISTTY,MPSAFE,LOCKSWORK,CLEAN> root powerd 1359 9* pipe 0xfffffe815d7bfdc0 -> 0x0 w Note fd=7,8 pointing to the revoked pty from the parent rc process.
Rebase to HEAD as of a few days ago.
sync with head. for a reference, the tree before this commit was tagged as yamt-pagecache-tag8. this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
#ifdef a variable decl/setting with it's use.
sync with head
include <limits.h> for CHAR_MIN/CHAR_MAX
Sync with HEAD
Tell copyfd if the caller wants the exact tofd to just fd >= tofd. Fixes "echo foo > /rump/bar" in a rump hijacked shell. reviewed by christos
dprintf is claimed by posix.
Sync with HEAD. Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
el_gets now sets el_len to -1 on error so we can distinguish between a NULL string and an error. This fixes sh from exiting with libedit now allowing EINTR to return. We may need to expand this to an errno check in the future.
Use "extern" properly for referencing globals defined in other modules. Now builds cleanly with -warn-common.
Since interpreting ELF binaries as shell scripts is not very useful, and since the current error message is less than helpful, improve it.
Move UCB-licensed code from 4-clause to 3-clause licence. Patches provided by Joel Baker in PR 22249, verified by myself.
Don't close any script files if vfork is set. If a fork() is done later in order to feed a 'here' document into a pipe then it is possible that one end of the pipe might get closed.
Fixes from David Laight: - ansification - format of output of jobs command (etc) - job identiers %+, %- etc - $? and $(...) - correct quoting of output of set, export -p and readonly -p - differentiation between nornal and 'posix special' builtins - correct behaviour (posix) for errors on builtins and special builtins - builtin printf and kill - set -o debug (if compiled with DEBUG) - cd src obj (as ksh - too useful to do without) - unset -e name, remove non-readonly variable from export list. (so I could unset -e PS1 before running the test shell...)
VFork()ing shell: From elric@netbsd.org: Plus my changes: - walking process group fix in foregrounding a job. - reset of process group in parent shell if interrupted before the wait. - move INTON lower in the dowait so that the job structure is consistent. - error check all setpgid(), tcsetpgrp() calls. - eliminate unneeded strpgid() call. - check that we don't belong in the process group before we try to set it.
Doing the vfork work on ash on a branch to try to shake out the problems before I expose everyone to them. This checkin represents a merge of the prior work, which I backed out a while ago, to the HEAD only and does not incorporate any additional bugfixes. The additional bugfixes and code-cleanup will occur in later checkins. For reference the patches that were used are: cvs diff -kk -r1.51 -r1.55 eval.c | patch cvs diff -kk -r1.27 -r1.28 exec.c | patch cvs diff -kk -r1.15 -r1.16 exec.h | patch cvs diff -kk -r1.32 -r1.33 input.c | patch cvs diff -kk -r1.10 -r1.11 input.h | patch cvs diff -kk -r1.32 -r1.35 jobs.c | patch cvs diff -kk -r1.9 -r1.11 jobs.h | patch cvs diff -kk -r1.36 -r1.37 main.c | patch cvs diff -kk -r1.20 -r1.21 redir.c | patch cvs diff -kk -r1.10 -r1.11 redir.h | patch cvs diff -kk -r1.10 -r1.12 shell.h | patch cvs diff -kk -r1.22 -r1.23 trap.c | patch cvs diff -kk -r1.12 -r1.13 trap.h | patch cvs diff -kk -r1.23 -r1.24 var.c | patch cvs diff -kk -r1.16 -r1.17 var.h | patch All other changes were simply the resolution of the resulting conflicts, which occured only in the merge of jobs.c. Begins to address PR: bin/5475
remove redundant declarations and nexted externs.
Back out previous vfork changes.
Now we use vfork(2) instead of fork(2) when we can.
compile with WARNS = 2
Simplify interface for general use.
fix bug where if moving in history during a multi-line event, the append to history event would end up in the last event where the history was moved to instead of the multi-line event; reported by Mycroft
Sync with trunk, per request of christos.
- change "extern" variables into int's - remove extern'd variables not actually referenced - don't use char as an array index
PR/4257: Jaromir Dolecek: Update for libedit interface change.
Fix compiler warnings.
Rename pread to preadfd
NO_HISTORY->SMALL
Fix problems with -DNO_HISTORY
Set input files to close-on-exec mode.
Update /bin/sh from trunk per request of Christos Zoulas. Fixes many bugs.
kill 'register'
Fix problems that gcc -Wall found (from Todd Miller, OpenBSD)
PR/2808: Remove trailing whitespace (from FreeBSD)
- fix PR1620, -DNO_HISTORY did not work. - restore parsing state after parsing old style command substitution. The ';' in '`echo z;`' broke the following: for i in 1; do cat > /dev/tty << __EOF__ `echo z;` __EOF__ done cVS: Enter Log. Lines beginning with `CVS: ' are removed automatically
fixed previous booboo that broke command line editing input.
- Fix -v flag, so that it works properly when the shell reads from scripts. - Bad style to fix my own PR, but I'd like to commit the parallel make changes soon, and this is a necessary prerequisite.
Changed so that 'PATH=newpath command' works, instead of looking at the old path. Synced input.c with vangogh.
needs string.h
Merge in my changes from vangogh, and fix the x=`false`; echo $? == 0 bug.
convert to new RCS id conventions.
pull prototypes into scope for string functions.
from James Jegers <jimj@miller.cs.uwm.edu>: quiet -Wall, and squelch some of the worst style errors.
Add RCS ids.
Fix from Christos for when NO_HISTORY is defined
a few more things to omit when NO_HISTORY defined. from noel@cs.oberlin.edu
add back in support for building w/o obj dir. also, add NO_HISTORY define, which (if you invoke mkbuiltins properly) gets you a sh w/o history of command line editing (for floppy sh).
Include appropriate header files to bring function prototypes into scope.
sync with 4.4lite
44lite code
Add RCS identifiers.
changed "Id" to "Header" for rcsids
added rcs ids to all files
initial import of 386bsd-0.1 sources
Initial revision