CVS log for src/usr.bin/xinstall/xinstall.c

Up to [cvs.NetBSD.org] / src / usr.bin / xinstall

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

nstall(1): add support for -v

Reviewed by rillig@ and thorpej@, thanks!

tools: binstall can use posix_spawn() instead of vfork()

install(1): copy metachar table from usr.bin/make

It is not worth having a dependency on usr.bin/make just for including
this small table.  The header file usr.bin/make/make.h, which is
included for consistency with the rest of the usr.bin/make code, is
using inline functions a lot and refers to several parts of make that
are completely irrelevant to usr.bin/install.

Move __MKTEMP_OK up earlier so it has an actual impact.  Reduces the
number of warnings during build.

XXX There are still some other warnings remaining to be resolved.

Fixes PR bin/48195 although we really should go back someday and fix
this correctly (by replacing all uses of mktemp(3)!)

remove unused label

remove useless assignment and explain why.

restore use of whitespace needing shell.

restore unlink file if strip fails (from Takeshi Nakayama)

Use the bourne shell if the after command has shell metachars.

- improve error printing
- deduplicate run functions and don't use the shell so that we handle
  filenames with spaces and metacharacters consistently.

Add optional support for posix_spawn (instead of vfork) and enable it for
the in-tree build.

Rebase to HEAD as of a few days ago.

Rebase.

mtree spec files want nanoseconds, so multiply the microseconds by 1000.

When printing a timestamp to a metalog, use 9 digits
for the nanoseconds part, except use only 1 digit if
the value was zero.

Use static and __dead.

sync to netbsd-5

Don't assume that "id_t", "gid_t" and "uid_t" use the same underlying type
and use the appropriate types instead. Tested under NetBSD 5.99.22
and Darwin 10.2.0.

This should fix one of the cross-builds under FreeBSD as reported by
Rhialto on the "current-users" mailing list.

Only trust STRIP from environment if it is non-empty. Some packages set
STRIP in Makefiles and that makes install -s fail badly. OK apb@

When installing via a temporary file, base the name of the temporary
file on the name of the target file, not just the target directory, to
ensure uniqueness when multiple concurrent invocations of install(1)
simultaneously install files in the same directory.  Fixes bin/41512.

tabify and wrap long lines.

Fix race condition causing "install -d" to randomly fail when multiple
concurrent install processes try to create the same directory.
Modelled after the code handling the "mkdir -p" case in mkdir(1).

Pull up following revision(s) (requested by snj in ticket #790):
	share/mk/bsd.lib.mk: revision 1.298
	share/mk/bsd.man.mk: revision 1.100
	share/mk/bsd.hostprog.mk: revision 1.55
	distrib/sets/maketars: revision 1.66
	share/zoneinfo/Makefile: revision 1.43
	share/mk/bsd.kmodule.mk: revision 1.19
	usr.bin/xinstall/xinstall.c: revisions 1.106 - 1.108 via patch
	share/mk/bsd.prog.mk: revision 1.241
	share/dict/Makefile: revision 1.17
	share/mk/bsd.README: revision 1.249
	distrib/common/parselist.awk: revision 1.16
	share/mk/bsd.links.mk: revision 1.34
Explicitly sort entries on preparing set files from METALOG.
METALOG could have different order due to install(1) race
on parallel builds, and mtree(8) doesn't sort files.
Should fix inconsistent shared sets among builds as seen in
/pub/NetBSD-daily/netbsd-5/200904010000Z/shared/ and
/pub/NetBSD-daily/netbsd-5/200904010002Z/shared/ dirs.
Okay'ed by snj@.
Changes for installing with a metalog:
* When installing hard links and using a metalog, if -o, -g, -m, or -f
  args were explicitly specified on the command line, then believe them,
  but do not implicitly believe uname/gname/mode/flags from the file
  system.
* Output fields in the same order used by mtree.
Fix -Wshadow -Wcast-qual issues
Re-calculate size and digestresult after stripping, to ensure that
correct values appear in the metalog.
Add LINKSOWN, LINKSGRP, and LINKSMODE variables for use by bsd.links.mk
when installing hard links.  They have no effect except when using a
metalog, in which case the information is added to the metalog.  In
the future, these variables may be replaced by a method for explicitly
recording hard links in a metadata log.
Also change a few things that called ${INSTALL_LINK} without going
through bsd.links.mk.
Reviewed by perry and joerg.  This should fix PR 24457 and PR 41155.

Pull up following revision(s) (requested by snj in ticket #790):
	share/mk/bsd.lib.mk: revision 1.298
	share/mk/bsd.man.mk: revision 1.100
	share/mk/bsd.hostprog.mk: revision 1.55
	distrib/sets/maketars: revision 1.66
	share/zoneinfo/Makefile: revision 1.43
	share/mk/bsd.kmodule.mk: revision 1.19
	usr.bin/xinstall/xinstall.c: revisions 1.106 - 1.108 via patch
	share/mk/bsd.prog.mk: revision 1.241
	share/dict/Makefile: revision 1.17
	share/mk/bsd.README: revision 1.249
	distrib/common/parselist.awk: revision 1.16
	share/mk/bsd.links.mk: revision 1.34
Explicitly sort entries on preparing set files from METALOG.
METALOG could have different order due to install(1) race
on parallel builds, and mtree(8) doesn't sort files.
Should fix inconsistent shared sets among builds as seen in
/pub/NetBSD-daily/netbsd-5/200904010000Z/shared/ and
/pub/NetBSD-daily/netbsd-5/200904010002Z/shared/ dirs.
Okay'ed by snj@.
Changes for installing with a metalog:
* When installing hard links and using a metalog, if -o, -g, -m, or -f
  args were explicitly specified on the command line, then believe them,
  but do not implicitly believe uname/gname/mode/flags from the file
  system.
* Output fields in the same order used by mtree.
Fix -Wshadow -Wcast-qual issues
Re-calculate size and digestresult after stripping, to ensure that
correct values appear in the metalog.
Add LINKSOWN, LINKSGRP, and LINKSMODE variables for use by bsd.links.mk
when installing hard links.  They have no effect except when using a
metalog, in which case the information is added to the metalog.  In
the future, these variables may be replaced by a method for explicitly
recording hard links in a metadata log.
Also change a few things that called ${INSTALL_LINK} without going
through bsd.links.mk.
Reviewed by perry and joerg.  This should fix PR 24457 and PR 41155.

Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html

Add support for writing sha256, sha384, and sha512 digests to
a metalog.  mtree(8) has supported these for a long time.

Re-calculate size and digestresult after stripping, to ensure that
correct values appear in the metalog.

Fix -Wshadow -Wcast-qual issues

Changes for installing with a metalog:
* When installing hard links and using a metalog, if -o, -g, -m, or -f
  args were explicitly specified on the command line, then believe them,
  but do not implicitly believe uname/gname/mode/flags from the file
  system.
* Add a size= field (only for plain files).
* Output fields in the same order used by mtree.

back out previous, and do it the better way.

In the "tool" case, pretend we don't have futimes because there is no visible
prototype for futimes without _NETBSD_SOURCE being defined. When we will do
a __RENAME() in futimes this will not work. XXX: This is a hack, better ideas
welcome.

Sync with wrstuden-revivesa-base-2.

Remove the \n and tabs from the __COPYRIGHT() strings.
Tweak to use a consistent format.

Sync w/ NetBSD-4-RC_1

Pull up following revision(s) (requested by joerg in ticket #823):
	usr.bin/xinstall/xinstall.c: revision 1.102
	tools/binstall/Makefile: revision 1.6
	tools/binstall/Makefile: revision 1.7
	share/mk/bsd.own.mk: revision 1.496
- When compiling nbinstall for the tool chain, use the target strip
  and not /usr/bin/strip. This makes nbinstall usable for
  cross-compiling in pkgsrc.
- Fix a typo in a comment.
- Fix strip() in the context of (v)fork:
  - Build the argument string first using asprintf (simpler) and
    avoid leaking memory from the child in the parent.
  - Don't use warn in the child, as stdio should be avoided.
OK christos@, tested on Solaris by dmcmahill@.
Rename nbinstall to ${MACHINE_GNU_PLATFORM}-install as it calls the
target strip. Don't pass down strip as it is not needed any longer.

- When compiling nbinstall for the tool chain, use the target strip
  and not /usr/bin/strip. This makes nbinstall usable for
  cross-compiling in pkgsrc.
- Fix a typo in a comment.
- Fix strip() in the context of (v)fork:
  - Build the argument string first using asprintf (simpler) and
    avoid leaking memory from the child in the parent.
  - Don't use warn in the child, as stdio should be avoided.

OK christos@, tested on Solaris by dmcmahill@.

Complete the move of string_to_flags() and flags_to_string() from the
bin/ls sources to libutil:
 o Bump libutil minor version number
 o Fix uses to include <util.h> to pick up the function definitions
 o Fix most uses of flags_to_string() to release the now-malloc()ed result

kill crypto/rmd160.h and crypto/sha2.h, and instead make symlinks to
/usr/include from /usr/include/sys. This makes all the one way hash
header handling identical.

When trying to copy many files into another file, provide a useful and
more specific error message instead of the usage().

PR/30698: install(1) stripping flags in METALOG [patch] (Jukka Salmi)

Apply patch, thanks!

PR/32589: install -d ignores existing non-directories (joerg@netbsd.org)

Raise an error if an argument to 'install -d' exists as a non-directory,
as suggested by joerg@ (and acked by some developers on #netbsd-code).
With this change behavior is equal in this respect to FreeBSD and
DragonFly.

fixes for installing into a case insensitive $DESTDIR
when files may have hard links to a a name that only differs by case
  - change install to unconditionally remove its temporary file
    when installing hard links with -r.  This avoids problems when
    built with posix rename(2) semantics and reinstalling an existing
    hard link.
  - rework hard link targets in bsd.man.mk and bsd.links.mk
    to use makefile constructs instead of shell constructs
  - always reinstall hard links that may have case conflicts, even
    when MKUPDATE=yes, this ensures that they get added to METALOG
  - remove man pages which were hard linked to themselves in libform
  - remove improper hard link command for existing man page in libkrb5
  - fix libl's makefile to include bsd.lib.mk at end
  - remove shell quoting in link target for test's [.1 man page

char -> u_char for a buffer passed to MD5*().

Coverity CID 95: Fix resource leak.

fix setmode error handling.

Use crypto/rmd160.h.

mktemp() uses are ok here, so no more warnings.

Pull up revision 1.90 (requested by lukem in ticket #287):
When creating a relative symlink (-lr), use the originally provided `to_name'
to create and metalog, rather than using the realpath(3) modified version.
Fixes a problem with -lr when -ddestdir contains a symlink.
Add some XXX comments reminding us that the `from_name' in a symlink may
point outside of destdir in the metalog, even though our build process
doesn't trigger this.

Pull up revision 1.89 (requested by lukem in ticket #271):
strvis(3) encode the link=... argument in the metalog.
(mtree(8) already requires this.)

Pull up revision 1.88 (requested by lukem in ticket #270):
metadata_log(): remove leading  -D destdir  before strsvis(3) encoding
the path, not after.

When creating a relative symlink (-lr), use the originally provided `to_name'
to create and metalog, rather than using the realpath(3) modified version.
Fixes a problem with -lr when -ddestdir contains a symlink.

Add some XXX comments reminding us that the `from_name' in a symlink may
point outside of destdir in the metalog, even though our build process
doesn't trigger this.

strvis(3) encode the link=... argument in the metalog.
(mtree(8) already requires this.)

metadata_log(): remove leading  -D destdir  before strsvis(3) encoding
the path, not after.

Apply patch (request by dbj in ticket 1723):
Fix building the netbsd-1-6 branch under Darwin/MacOS.

Pull up revision 1.87 (requested by jmc in ticket #527):
Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')
Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).
Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.
Fixes PR's: PR#17762 PR#25944

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

Fix bug when calculating digest when copying from dev/null.
Problem noted by Matthias Scheler and Tomasz Luchowski.

Auso support -h hash for hard-linked files.

Implement  -h {none,md5,rmd160,sha1}  to determine the appropriate hash
(digest) when copying a file to store in the -M METALOG.

Make "copy the file" (the -c option) the default behaviour.  The -c option
is kept for backwards compatibility but now has no effect.

Patch from Bruce Nourish in PR bin/23874, with some minor changes by me.

Initialize uid and gid so that the owner/group is not inadvertantly changed
when only one of -g or -o are given by the super-user and -g works for others.

munmap() so that we don't accumulate memory on multiple files.

Overhaul how `build.sh tools' are used:

    *	Rename "config.h" to "nbtool_config.h" and
	HAVE_CONFIG_H to HAVE_NBTOOL_CONFIG_H.
	This makes in more obvious in the source when we're using
	tools/compat/config.h versus "standard autoconf" config.h

    *	Consistently move the inclusion of nbtool_config.h to before
    	<sys/cdefs.h> so that the former can provide __RCSID() (et al),
    	and there's no need to protect those macros any more.

These changes should make it easier to "tool-ify" a program by adding:
	#if HAVE_NBTOOL_CONFIG_H
	#include "nbtool_config.h"
	#endif
to the top of the source files (for the general case).

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

strlcat

Don't do the madvise(2) call if __APPLE__.  Even though madvise(2)
exists on Mac OS X, and we pass valid arguments, the call always
results in EINVAL, and the warning annoys me.

DTRT with -D destdir when the target == destdir

Implement '-D destdir'; if '-M metalog' is being used, remove a
leading `destdir' from the file name logged to the metalog.
The -D option has no effect on where the actual files are installed.

Add '-N dbdir', to specify that user & group lookups are to be done from
dbdir/master.passwd and dbdir/group, using simple text file parsing
routines from ../../usr.sbin/mtree/getid.c.

Convert from flock() to the more portable fcntl().
As discussed with Mason Loring Bliss <mason@netbsd.org>

make an auto const variable static.

doh.  don't try and update from_sb and tv  if dolink is set

always stat into &from_sb and setup tv[]; stops weird problem noticed
by Simon Burge when installing from /dev/null resulted in weird dates.

- Add "-a afterinstallcommand", which runs "sh -c afterinstallcmd target"
  after target has been installed and possibly stripped with -s, but
  before ownership, permissions or timestamps are set, and before
  renaming (with -r) occurs.  Per discussion with simonb.
- When metalogging hardlinks, log the mode of the existing target
  (rather than the default 0755), and ignore owner/group/fflags.
- Improve man page description of various options to include argument name.
- Clean up usage() to use getprogname(), etc.

When metalogging a hard link, don't log it as "type=hlink".
Instead, stat the type, and only log an entry if it's a "type=file".
We could log hard links to symlinks, but they're not portable and
their use has been deprecated in our source tree.
Hard links to other types aren't as useful at this stage; if the need
arises we can change this to log the appropriate information then.

Roll in fixes to permit cross-compiling from non-NetBSD hosts.  This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
  sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

Identify where errors come from, rather than just doing err[x]("%s", foo).

Add non-NetBSD host hooks.

ADD MISSING SLASH.
Broken insall had been made a temporary file at the parent directory
(may cause Cross-device link(EXDEV)).

fix an accidental use of dirname instead of xdirname

add comments noting the mktemp usage is safe.

long run I think we have to abandon using the linker for warnings and
beef up lint instead.

Overhaul link creation - splitting out hard and symbolic link handling
to separate functions and handling -r (renaming) correctly for both
cases.
Also fix most lint(1) warnings.

Make metadata logging of set-id bits work for unprived installs.
These bits were cleared from mode before a call to fchmod and later
that changed value was logged.

simplify metadata logging, and add metadata logging for hard links.

Actually do a chmod if -U, but only try chmod'ing the lower 0777 bits.
This is needed to make things executable, where appropriate.

change metadata_log() to support logging symlink info as well
(including target name).  pointed out by perry.

Do not attempt to stat the source if we are dealing with a link. We
aren't going to do anything with the information anyway and there is
error checking later anyway.
We can now succeed in creating symlinks to locations that don't exist,
just as ln -s will let us do, and we can use install instead of ln -s
in several Makefiles. The code was written with the obvious intent to
let you do this but apparently it was never tested.

1) Remove a comment that bore no resemblance to reality. It was worse
   than having no comment at all by far.
2) fix an instance in which we checked if "from" wasn't a regular file
   and then bitched about "to".

Add '-T tags', to specify mtree(8) tags to be written out to the '-M metalog'
for the current entry.  Concept suggested by Perry Metzger.

minor knf

If unprivileged, we may be running on a system that doesn't have the proper
/etc/passwd and /etc/group (as well as concept of file flags) as the target.

Rather than look up users, groups, or flags with -U, do what the manpage
indicates:  don't even try changing them.  If -M is specified, use the
strings that are passed in via options with "gname", "uname", and "flags"
in the resultant mtree file, rather than retranslating them back from numbers.

oops, got chflags() error checking around the wrong way

- Implement "-M log", which appends the metadata associated with the
  installed file or directory to `log'. This is primarily designed
  to be used in conjunction with -U, but that's not mandatory.
  Provide metadata_log() to do the hard work. The log file is opened
  for append at the start, and flock(LOCK_EX) is used to prevent
  interleaved output (from multiple concurrent install(8) operations).
- Improve parsing of owner and groups (e.g., strtoul() instead of atoi()),
  and don't rely upon uid==-1 to indicate "no owner"; use a flag instead.
- Make fileflags a global variable; uid, gid, and mode are, so lets be
  consistent...
- Improve -U (unpriv); work on file flags as well, and only try to change
  the file flags after the temporary file is renamed (since certain flags
  will prevent the rename from working).

Throw out home-grown strdup().
Error check malloc/strdup return values.

ANSIfy.

heirarchy -> hierarchy

Add the -r flag to usage output.

fall back to read() if mmap() returns an error.

slight portability improvement: use madvise(... MADV_SEQUENTIAL ...)
iff MADV_SEQUENTIAL is actually defined.

call setprogname(argv[0]) at start of main()

- The type of return value of setmode is a void * and getmode takes it,
  rather than mode_t *.
- Free the storage allocated by setmode unless it is obvious that program
  exits immediately.

If unprivileged, don't even look up user or group name.  (always set uid and
gid to -1.)  Don't bother checking 'unpriv' when it's redundant with the
uid/gid == -1 check.  (Doing the uid/gid handling consistently also fixes
the directory-creation code, which didn't check unpriv and did chmod/chown
anyway.)  One minor spaces/tabs cleanup at one of the uid/gid checks.

invoke stripprog using execlp.  (this mirrors the behaviour that would
be had from using the shell to invoke strip, and removes an unnecessary
limitation that the STRIP environment variable be an absolute path.)

don't bitch if madvise returns EOPNOTSUPP

Add `-U' flag (for "unprivileged").  Currently just skips the
chown/chgrp stages which would otherwise blow out if not running as
root.  A future version may log the owner/group to allow correction by
a later privileged cleanup pass.

Just warn about madvise failure; it's not something to blow up about.

Document new option, and clean up option grouping for -d (install
directory) option.

pacify gcc-2.8 uninitialized variable warnings, and only use timespecs in
struct stat on BSD4_4 systems.

If we choose the mmap() method of copying the file, advise the VM
system that we'll be doing sequential access of the file.

Check if given suffix really generates different suffixes - catch e.g. ".%"
Submitted in PR 7278 by NAKAJIMA Yoshihiro <nakayosh@kcn.ne.jp>

Don't unlink the file first if we are renaming. We want to be as atomic as
possible, and this not only introduces time where the file is not valid,
but also breaks in the ld.so install where strip does not find ld.so, since
it has not been renamed yet.

PR/2936: Brian C. Grayson: Avoid truncating precious files when installing
and the filesystem is full. Added -r flag to install, which installs on
a temporary file and then renames to the final destination.

enable numbered backups via -B.old%02d

Add a -b(ackup) flag to move any existing file(s) aside by renaming
them to file.old instead of simply deleting them. A different backup
suffix may be chosen with the -B option's argument.

char -> unsigned char

create a "stat_flags.h" to go with stat_flags.c

Add new option -S, which is like -s, but takes a string of options to pass to strip,
  rather than "-s". For example:
	install -S "-i -s symbolfile" foo /usr/bin/foo
  Invokes:
	/bin/sh -c "strip -i -s /usr/bin/foo"
  This is needed, because we need different strip options for dynamic libraries and
  Mach-O bundles than for executables; our strip supports more directed stripping.
  /bin/sh is used in this case, to parse the arguments.

Don't bother mmap()ing files of zero length. Was a workaround for a bug in our mmap(),
  which didn't get along with such files, but makes sense anyway.

Add -p flag to usage().

add -p flag, similar to cp(1)

Fill in missing (default) mmap(2) flags.

use MAP_PRIVATE for mmap flags, not the non-standard MAP_FILE (aka 0)

accept -c silently with -d; fixes GNU autoconf (among others) that assume
you can use -c with -d
...but this is NOT a reason for those of you making install procedures
to think this behaviour is `correct'... <grin>

Pull up from trunk:

In strip():
- Use the saved errno to print the error message if the vfork() fails.
- In the child, if the execl() fails, use warn()/_exit(), not err(),
  as the latter is not safe in the presence of a Real vfork().

In strip():
- Use the saved errno to print the error message if the vfork() fails.
- In the child, if the execl() fails, use warn()/_exit(), not err(),
  as the latter is not safe in the presence of a Real vfork().

getopt returns -1 not EOF, fix .Nm

WARNSify;  deprecate register;  clean up .Nm

use proper pathname when unable to exec strip

Allow the environment variable $STRIP to override _PATH_STRIP for
`-s', helping out cross-compiling.

Fix pasto in error message.

Add support for installing hard and symbolic links via the [-l harms] flag.

Remove kluge for GCC bug on m68k.

Use a slightly more sane way to cheat around XXX_BROKEN_GCC on the m68k.
The old way was more complex than necessary, and buggy, as pointed out
by Bruce Albrecht <bruce@zuhause.mn.org> in PR #3309.

warnx --> warn, warnx wasn't needed anymore after previous change
(pointed out by enami tsugutomo)

Correct error in previous.

Don't bail out when chflags() fails with EOPNOTSUPP. Fixes PR #3288

XXX Kludge to make this compile with large MAXBSIZE until the
XXX GCC bug it tickles is fixed.

Change xinstall to work with strip from GNU binutils, for mips and alpha ports.

When using a strip that uses a temp file and moves that to the target,
unlinking the original, install would set the modes of the
original (now unlinked) file, not the new stripped one.
Fixed by re-opening the fd used for fchown/chmod after stripping.

Accept numeric uid or gid for -o and -g flags, for consistency with
chown and chgrp.

added prototype for install_dir()

added exit(1) to usage()

Merged with 4.4lite.
Changed to conform to NetBSD's new RCS Id convention.
Changed to use err(),errx(),warn(),warnx().

imported from 4.4lite

be a bit more careful abotu what we can stat()/mkdir()

Add RCS identifiers.

Added -d option to install

initial import of 386bsd-0.1 sources

Initial revision