CVS log for src/usr.sbin/mtree/compare.c

Up to [cvs.NetBSD.org] / src / usr.sbin / mtree

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

Deal with the case when a device gets recreated (eg when mtree is called by
MAKEDEV) and the stat buffer contains information about the original device
before checking/setting the owner/group/mode of the new device file.

mtree: use POSIX type uint32_t instead of u_int32_t

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

From Brooks Davis:
When comparing a directory to an mtree file NetBSD mtree has output like
this:

xxx:	size (4, 5)
	md5digest (0x6de9439834c9147569741d3c9c9fc010, 0xa5d119ab8edcda0ef7f381da8=
f52f92f)
=2E/yyy missing

FreeBSD's historical behavior is this:

xxx changed
	size expected 4 found 5
	md5digest expected 6de9439834c9147569741d3c9c9fc010 found a5d119ab8edcda0e=
f7f381da8f52f92f
=2E/yyy missing

FreeBSD's mergemaster -U command depends on the latter behavior so
the following patch alters the freebsd9 mode of mtree to output
that format.  I don't think programs should rely on this output one way
or another since it's clearly meant for humans, but this seems like the
easiest short term fix.

Our <sys/param.h> ends up calling header files that define intmax_t. This
should not be the case (but >sys/param.h> is not a standard header so all bets
are off). FreeBSD's does not, so explicitly include <stdint.h> to get it.

use intmax_t instead of long long. fixes wrong cast for time_t.

Resync to 2012-11-19 00:00:00 UTC

sync with head

account for differences in digest names

explicitly include <sys/stat.h>

more const
check if we support SHA384 before using
(brooks)

fix dev_t formats.

The hashing routines (MD5File() and co.) can do dynamic allocation, so
take advantage of that instead of an ugly hard-coded MAXHASHLEN limit
that needs updating.

I think this was suggested by cube@ years ago.

Fix mtree to release the returned strings from flags_to_string()
following the move of it from bin/ls to libutil and the change of
it to return malloc()ed memory.

kill crypto/rmd160.h and crypto/sha2.h, and instead make symlinks to
/usr/include from /usr/include/sys. This makes all the one way hash
header handling identical.

Use crypto/rmd160.h.

Apply patch (requested by elad in ticket #755):
Add support for SHA2 in mtree(8).

Apply patch (requested by elad in ticket #755):
Add support for SHA2 in mtree(8).

Use MAXHASHLEN here too.

Add SHA2 support in mtree.

Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Pull up revision 1.44 (requested by jmc in ticket #527):
Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')
Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).
Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.
Fixes PR's: PR#17762 PR#25944

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Remove convoluted code to work around missing lch{flags,mod,own}();
instead, rely upon libnbcompat to provide a workaround (which returns
an error if the target is a symlink).
Should fix another MacOS X build issue due to missing lchown(2).

- Check HAVE_LCHFLAGS for lchflags(2) support, rather than assuming
  that HAVE_STRUCT_STAT_ST_FLAGS implies this.
- Set HAVE_LCHFLAGS for native builds
- Clean up {CLEAR,SET,CHANGE}FLAGS macros, and only provide if
  HAVE_STRUCT_STAT_ST_FLAGS is set.

(Fixes compilation as a tool on MacOS X, noted by Allen Briggs.)

Funtion appropriately in the absence of a lchmod call on the host.

- use lch*() instead of ch*() to change permissions, so that links in
  the specfile don't confuse things.  doesn't affect the default
  (physical) operation, and shouldn't affect logical either.
- slightly rework how miss() is implemented so that duplicated code is
  reduced and links may optionally have a mode, uname and/or gname.

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

Pull up revision 1.26 (requested by lukem):
  Add an ``-l'' flag which does ``loose'' permission checks.

provide the ability to remove support for generating or checking various
digests by #define-ing NO_MD5, NO_RMD160, or NO_SHA1.  the keywords are
still parsed.

Generally don't need the extra byte in MAXPATHLEN sized buffers, because
pathnames are guaranteed by POSIX to be MAXPATHLEN-1 bytes long.
Kindly pointed out by Warner Losh in private email.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
  the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
  and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
  directories from its traversal.  excludes-file contains fnmatch(3)
  patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
  it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
  previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
	- Device type (retaining existing ownership).
	- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
  The previous code got majorly confused if fts(3) couldn't descend
  into a subdir, resulting in leaf nodes getting attached to the wrong
  directory.
  XXX: This new method is much much more robust, even though it's not 100%
  perfect; it might result in a couple of following entries in the spec
  to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
  doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
  there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
  the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

Add -W: don't attempt to change the perms/time/flags on existing entries, nor
set them when creating new directories. (AKA the ``don't whack'' option).
Concept suggested by Todd Vierling.

- enforce the top-most entry must be "." and must be a directory; avoids
  various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
  other programs (via .PATHing spec.c and misc.c):
	- move excludetags, includetags and keys from mtree.c to misc.c
	- implement mtree_err() using vwarnx() instead of assuming name is
	  "mtree"
	- move inotype() and nodetype() from compare.c to misc.c
	- add nodetoinode(), to convert from an mtree F_* type to a
	  mode_t S_IF* type
	- clean up #include use; don't assume "mtree.h" pulls in some
	  standard includes
	- change spec() to take a FILE * arg (where the specfile is read from)

need semicolon between goto label and closing brace

make this mostly WARNS=2 safe

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

features:
- add a new keyword - "tags" - which is a comma separated list of tags
  associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
  (default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D  - dump the specfile out in a manner that's parseable
  by various tools
- implement -R key  - remove the given key(s) from the list of keys to print
  (now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

Add a -l flag which does "loose" permissions checks, i.e. a mode 444
matches a check for mode 644 (though obviously not the reverse). This
can be used by the nightly security run, making the output1 more useful
by having it contain fewer spurious permissions violations.

Note that I did not make -l work if you have a sgid/suid/sticky bit
set. I don't know how you could cause security trouble with more
stringent settings and a suid file, but I don't want to find out the
hard way.

- add parsetype() (a la parsekey()) to parse the "type=xxx" arg. this improves
  the error detection for invalid types, as well as shrinking binary size
- implement inotype() using ftype(), rather than duplicating the switch
- change "char *" arguments to "const char *"

ANSIfy, KNF, in preperation for more work.

use %ll_ instead of the less standard %q_

fix thinko in mtime stuff.

portability fixes:
- don't use timespecs in struct stat on non 4.4BSD systems
- don't assume that tv_sec is a time_t

- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

Appease gcc-2.8.1, make printf formats more portable.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT.  There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags.  we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

Add {} pairs to shut up egcs.

Pull rev 1.14 up from trunk

Make this compile again on NetBSD/alpha; cast off_t variable to long long
so that it matches printf format `%qd'.

WARNSify, getopt returns -1 not EOF

fix printf() formats, make explicit comparisons to NULL (gcc -Wall)

From trunk:
Don`t record sizes for things other than regular files.
Add a `-t' option to update time stamps.

Add a `-t' option to update time stamps.

Rename struct timespec fields to conform to POSIX.1b

Insert parentheses to get correct precedence when comparing time-stamps
(from Arne Juul; PR#1660).

add NetBSD id's

clean up import, fix a few bugs, etc.

from 4.4-Lite

off_t exp.

make things a bit more consistent

latest mtree from uunet.  doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

update for fts library change

Add RCS identifiers.

initial import of 386bsd-0.1 sources

Initial revision