CVS log for src/usr.sbin/makefs/ffs/mkfs.c

Up to [cvs.NetBSD.org] / src / usr.sbin / makefs / ffs

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Pull up following revision(s) (requested by chs in ticket #160):

	usr.sbin/makefs/ffs/ffs_alloc.c: revision 1.31
	sbin/tunefs/tunefs.c: revision 1.58
	sbin/fsck_ffs/setup.c: revision 1.105
	sbin/fsck_ffs/pass5.c: revision 1.56
	usr.sbin/makefs/ffs.c: revision 1.74
	usr.sbin/makefs/ffs/mkfs.c: revision 1.42
	usr.sbin/makefs/Makefile: revision 1.40
	sys/ufs/ffs/fs.h: revision 1.71
	sbin/fsdb/fsdb.c: revision 1.54
	sbin/resize_ffs/resize_ffs.c: revision 1.58
	sbin/fsck_ffs/pass4.c: revision 1.29
	usr.sbin/makefs/ffs/ffs_extern.h: revision 1.9
	sbin/newfs/mkfs.c: revision 1.133
	sys/ufs/ffs/ffs_alloc.c: revision 1.172
	sbin/fsck_ffs/pass1b.c: revision 1.24
	usr.sbin/dumpfs/dumpfs.c: revision 1.68
	sys/ufs/ffs/ffs_extern.h: revision 1.88
	usr.sbin/quotacheck/quotacheck.c: revision 1.51
	sys/ufs/ffs/ffs_subr.c: revision 1.54
	sbin/fsck_ffs/main.c: revision 1.91
	sbin/fsck_ffs/pass1.c: revision 1.63

ufs: fixed signed/unsigned bugs affecting large file systems

Apply these commits from FreeBSD:
  commit e870d1e6f97cc73308c11c40684b775bcfa906a2
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Wed Feb 10 20:10:35 2010 +0000
    This fix corrects a problem in the file system that treats large
    inode numbers as negative rather than unsigned. For a default
    (16K block) file system, this bug began to show up at a file system
    size above about 16Tb.
    To fully handle this problem, newfs must be updated to ensure that
    it will never create a filesystem with more than 2^32 inodes. That
    patch will be forthcoming soon.
    Reported by: Scott Burns, John Kilburg, Bruce Evans
    Followup by: Jeff Roberson
    PR:          133980
    MFC after:   2 weeks

  commit 81479e688b0f643ffacd3f335b4b4bba460b769d
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Thu Feb 11 18:14:53 2010 +0000
    One last pass to get all the unsigned comparisons correct.

In additional to the changes from FreeBSD, this commit includes quite a few
related changes to appease -Wsign-compare.

ufs: fixed signed/unsigned bugs affecting large file systems

Apply these commits from FreeBSD:

  commit e870d1e6f97cc73308c11c40684b775bcfa906a2
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Wed Feb 10 20:10:35 2010 +0000

    This fix corrects a problem in the file system that treats large
    inode numbers as negative rather than unsigned. For a default
    (16K block) file system, this bug began to show up at a file system
    size above about 16Tb.

    To fully handle this problem, newfs must be updated to ensure that
    it will never create a filesystem with more than 2^32 inodes. That
    patch will be forthcoming soon.

    Reported by: Scott Burns, John Kilburg, Bruce Evans
    Followup by: Jeff Roberson
    PR:          133980
    MFC after:   2 weeks

  commit 81479e688b0f643ffacd3f335b4b4bba460b769d
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Thu Feb 11 18:14:53 2010 +0000

    One last pass to get all the unsigned comparisons correct.


In additional to the changes from FreeBSD, this commit includes quite a few
related changes to appease -Wsign-compare.

Restore backward compatibility of UFS2 with previous NetBSD releases by
disabling support in UFS2 for extended attributes (including ACLs).
Add a new variant of UFS2 called "UFS2ea" that does support extended attributes.
Add new	fsck_ffs operations "-c	ea" and	"-c no-ea" to convert file systems
from UFS2 to UFS2ea and	vice-versa (both of which delete all existing extended
attributes in the process).

Avoid integer overflow on systems with 32bit disk addresses.

Merge changes from current as of 20200406

Fix the build, use %jd and (intmax_t) cast for big numbers, off_t
and ptrdiff_t aren't always the same size.

improve error messages.

Sync with HEAD

Sync with HEAD

Revert the part of mkfs.c 1.36 that "gutted the const" (while
retaining the part that added a different one).

That is, re-constipate makefs (well, just a bit, no real pain here.)

gut const for now.

Don't store random (from ASLR) pointers into the superblock. Should be the
last (famous last words) problem with reproducible builds!

tidy up error messages

more deterministic ffs for reproducible builds.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

fix unused variable warnings.

Stick ffs_ in front of the following macros:
   fragstoblks()
   blkstofrags()
   fragnum()
   blknum()

to finish the job of distinguishing them from the lfs versions, which
Christos renamed the other day.

I believe this is the last of the overtly ambiguous exported symbols
from ffs... or at least, the last of the ones that conflicted with lfs.
ffs still pollutes the C namespace very broadly (as does ufs) and this
needs quite a bit more cleanup.

XXX: boo on macros with lowercase names. But I'm not tackling that just yet.

Stick ffs_, ext2_, chfs_, filecore_, cd9660_, or mfs_ in front of
the following symbols so as to disambiguate fully. (Christos already
did the lfs ones.)

   lblkno
   lblktosize
   lfragtosize
   numfrags
   blkroundup
   fragroundup

resync from head

fsbtodb() -> FFS_FSBTODB(), EXT2_FSBTODB(), or MFS_FSBTODB()
dbtofsb() -> FFS_DBTOFSB() or EXT2_DBTOFSB()

(Christos already did the lfs ones a few days back)

Rename ambiguous macros:
   MAXDIRSIZE -> UFS_MAXDIRSIZE or LFS_MAXDIRSIZE
   NINDIR -> FFS_NINDIR, EXT2_NINDIR, LFS_NINDIR, or MFS_NINDIR
   INOPB -> FFS_INOPB, LFS_INOPB
   INOPF -> FFS_INOPF, LFS_INOPF
   blksize -> ffs_blksize, ext2_blksize, or lfs_blksize
   sblksize -> ffs_blksize

These are not the only ambiguously defined filesystem macros, of
course, there's a pile more. I may not have found all the ambiguous
definitions of blksize(), too, as there are a lot of other things
called 'blksize' in the system.

resync with head

- more changes to make -O work
- fix err* calls.

- use emalloc and friends
- kill a bunch of global variables, more work to be done here
- homogenize option parsing. more work for cd9660
- use the new options parsing code to print an fs-specific usage

sync with head

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
   ROOTINO -> UFS_ROOTINO
   WINO -> UFS_WINO
   NXADDR -> UFS_NXADDR
   NDADDR -> UFS_NDADDR
   NIADDR -> UFS_NIADDR
   MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
   MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

sync with head.

Fix build failure reported by bracket.

(shouldn't this size_t be ino_t?)

WARNS=5

add support for setting the ufs label. (Nathan Whitehorn)

Refactor the filesystem specific portions out of makefs.c/makefs.h completely.
Instead of extending fsinfo_t it now holds a void * to file system specific
data. This is then setup/cleaned up by the additional of 2 additional
callbacks. Makes adding new filesystems simpler as almost no code has
to be updated in the generic makefs code now.

Pullup rev 1.20 (requested by lukem in ticket #549)

Define FFS_MAXBSIZE to 65536 and use instead of MAXBSIZE.
Deprecate unused MAXINOPB. PR#25603 fixed.

Define FFS_MAXBSIZE to 65536 and use instead of MAXBSIZE.
Deprecate unused MAXINOPB.
Should fix PR [toolchain/25603].

Pull up revision 1.19 (requested by jmc in ticket #527):
Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')
Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).
Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.
Fixes PR's: PR#17762 PR#25944

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

Revert arc4random usage. makefs is only used for install images,
filehandle randomization is not an issue there, and using arc4random
broke building makefs as a host tool.

use arc4random

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Protect use of __RCSID() with a check to ensure that it's defined.
Suggested by Ian Lance Taylor on tech-userlevel@ to help cross-compilation
on Red Hat 7.3.

Add support for UFS2. UFS2 is an enhanced FFS, adding support for
64 bit block pointers, extended attribute storage, and a few
other things.

This commit does not yet include the code to manipulate the extended
storage (for e.g. ACLs), this will be done later.

Originally written by Kirk McKusick and Network Associates Laboratories for
FreeBSD.

make this build on alpha after daddr_t->64bit

Bump daddr_t to 64 bits. Replace it with int32_t in all places where
it was used on-disk, so that on-disk formats remain the same.
Remove ufs_daddr_t and ufs_lbn_t for the time being.

Remove extraneous \n's in {err,warn}{,x} that used to be printfs.

don't need <ufs/ufs/dir.h> here

Roll in fixes to permit cross-compiling from non-NetBSD hosts.  This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
  sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

- crank default cpg from 16 to 65536, and let the auto-sizing code in
  mkfs() do its job
- pass cpgflg=1 to mkfs() if the user provided cpg, =0 otherwise
- improve error reporting in ffs_write_file()
- only add superblock & block map slop for one cylinder group.
  trying to calculate the number of required cylinder groups caused
  too many problems when tweaking parameters for small file systems.
  rely upon slop from minfree, and upon ffs_write_file() to complain
  if the user is too aggressive about tweaking the parameters

apply changes from revs 1.56-1.59 of newfs/mkfs.c:
* revision 1.56
	the change of calculation of inodes per group in rev 1.52 was far too
	aggressive; rework to be a bit less susceptable to round-off error.
	now it's likely that the density might not be obtained with a small
	filesystem with a large number of inodes (e.g -s 4M -i 1k), but that's
	an extremely unlikely corner case that can easily be rectified with
	command-line arguments.
	fixed provided in private email by Takao Shinohara <shin@sm.sony.co.jp>
	should resolve PRs [bin/14049] and [bin/14046]
    [ this fixes a problem in makefs(8) that matt green reported ]

* revision 1.58
	clamp bsize to MAXBSIZE

* revision 1.59
	don't bother printing a warning about the cylinder group size being
	restricted if -c isn't given; it just confuses a user of newfs (and
	the cpg info is printed as part of the display anyway), and prints an
	unnecessary warning for mount_mfs.

XXX: I must rework newfs/mkfs.c to make it easier to just .PATH into makefs(8)

need ufs/ufs/dinode.h for ufs/ffs/fs.h

add "ffs/ufs_inode.h" with a minimal struct inode, to replace
<ufs/ufs/inode.h>, since the latter has a lot of cruft we don't need
and it #include's way more stuff in <sys/*> (etc) than is needed here.
yet another nail in the "let's make makefs a proper host tool" coffin.

- pull in <ufs/ufs/dinode.h> before <ufs/ffs/fs.h>
- #include <ufs/[uf]fs/*.h> instead of "ufs/[uf]fs/*.h", and don't bother
  with -I../../sys; we'll solve the hostprog problem another way and
  unconditionally pulling in NetBSD-current's usr/src/sys on a host system
  could be a Bad Thing.

fix bugs reported by matt green:
- when calculating file size, round up to frag size not block size
- #define FFS_EI in makefs.h, and include that before [uf]fs/*.h

other stuff:
- round up final size to next block (instead of next sector)
- protect makefs.h from multiple inclusion

reference the ufs/[uf]fs headers in ../../sys rather than using those
in /usr/include. this should make it easier to make this a host tool

makefs - create a file system image from a directory tree.
It doesn't need any special privileges or kernel devices.

Only ffs image creation is supported at this time, although makefs has been
designed to allow the addition of other file system formats by writing new
back-ends.

This program was designed & implemented by Luke Mewburn of Wasabi Systems.

Initial revision