CVS log for src/sys/ufs/lfs/lfs_vfsops.c

Up to [cvs.NetBSD.org] / src / sys / ufs / lfs

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Remove now unused VV_LOCKSWORK, all file systems support locking.

Remove unused predicates vn_locked() and vn_anylocked().

Welcome to 9.99.95

Sync with HEAD.

s/threshhold/threshold

Round of uvm.h cleanup.

The poorly named uvm.h is generally supposed to be for uvm-internal
users only.

- Narrow it to files that actually need it -- mostly files that need
  to query whether curlwp is the pagedaemon, which should maybe be
  exposed by an external header.

- Use uvm_extern.h where feasible and uvm_*.h for things not exposed
  by it.  We should split up uvm_extern.h but this will serve for now
  to reduce the uvm.h dependencies.

- Use uvm_stat.h and #ifdef UVMHIST uvm.h for files that use
  UVMHIST(ubchist), since ubchist is declared in uvm.h but the
  reference evaporates if UVMHIST is not defined, so we reduce header
  file dependencies.

- Make uvm_device.h and uvm_swap.h independently includable while
  here.

ok chs@

Pull up following revision(s) (requested by riastradh in ticket #1050):

	sys/ufs/lfs/lfs_subr.c: revision 1.101
	sys/ufs/lfs/lfs_subr.c: revision 1.102
	sys/ufs/lfs/lfs_inode.c: revision 1.158
	sys/ufs/lfs/lfs_inode.h: revision 1.25
	sys/ufs/lfs/lfs_balloc.c: revision 1.95
	sys/ufs/lfs/lfs_pages.c: revision 1.21
	sys/ufs/lfs/lfs_vnops.c: revision 1.330
	sys/ufs/lfs/lfs_alloc.c: revision 1.140 (patch)
	sys/ufs/lfs/lfs_alloc.c: revision 1.141 (patch)
	lib/libp2k/p2k.c: revision 1.72
	sys/ufs/lfs/lfs.h: revision 1.205
	sys/ufs/lfs/lfs.h: revision 1.206
	sys/ufs/lfs/lfs_segment.c: revision 1.284
	sys/ufs/lfs/lfs.h: revision 1.207
	sys/ufs/lfs/lfs_segment.c: revision 1.285
	sys/ufs/lfs/lfs_debug.c: revision 1.55
	sys/ufs/lfs/lfs_rename.c: revision 1.23
	usr.sbin/dumplfs/dumplfs.c: revision 1.65
	sys/ufs/lfs/lfs_vfsops.c: revision 1.371
	sys/arch/i386/stand/efiboot/bootx64/Makefile: revision 1.3
	sys/ufs/lfs/lfs_vfsops.c: revision 1.372
	sys/ufs/lfs/lfs_vfsops.c: revision 1.373
	sbin/fsck_lfs/pass1.c: revision 1.46
	sys/ufs/lfs/lfs_vnops.c: revision 1.326
	sys/ufs/lfs/lfs_vnops.c: revision 1.327
	sys/ufs/lfs/lfs_vfsops.c: revision 1.375 (patch)
	sys/ufs/lfs/lfs_vnops.c: revision 1.328
	sys/ufs/lfs/lfs_subr.c: revision 1.98
	sys/ufs/lfs/lfs_extern.h: revision 1.116
	sys/ufs/lfs/lfs_vnops.c: revision 1.329
	sys/ufs/lfs/lfs_subr.c: revision 1.99
	sys/ufs/lfs/lfs_extern.h: revision 1.117
	sys/ufs/lfs/lfs_accessors.h: revision 1.49
	sys/ufs/lfs/lfs_extern.h: revision 1.118
	sys/rump/fs/lib/liblfs/Makefile: revision 1.15
	sys/ufs/lfs/lfs_bio.c: revision 1.146 (patch)
	sys/ufs/lfs/lfs_bio.c: revision 1.147
	sys/ufs/lfs/lfs_subr.c: revision 1.100

Fix kassert in lfs by initializing vp first.

Use a marker node to iterate lfs_dchainhd / i_lfs_dchain.

I believe elements can be removed while the lock is dropped,
including the next node we're hanging on to.

Just use VOP_BWRITE for lfs_bwrite_log.
Hope this doesn't cause trouble with vfs_suspend.

Teach lfs to transition ro<->rw.

Prevent new dirops while we issue lfs_flush_dirops.

lfs_flush_dirops assumes (by KASSERT((ip->i_state & IN_ADIROP) == 0))
that vnodes on the dchain will not become involved in active dirops
even while holding no other locks (lfs_lock, v_interlock), so we must
set lfs_writer here.  All other callers already set lfs_writer.

We set fs->lfs_writer++ without explicitly doing lfs_writer_enter
because
(a) we already waited for the dirops to drain, and
(b) we hold lfs_lock and cannot drop it before setting lfs_writer.

Assert lfs_writer where I think we can now prove it.

Serialize access to the splay tree with lfs_lock.

Change some cheap KDASSERT into KASSERT.

Take a reference and fix assertions in lfs_flush_dirops.
Fixes panic:
KASSERT((ip->i_state & IN_ADIROP) == 0) at lfs_vnops.c:1670
lfs_flush_dirops
lfs_check
lfs_setattr
VOP_SETATTR
change_mode
sys_fchmod
syscall

This assertion -- and the assertion that vp->v_uflag has VU_DIROP set
-- is valid only until we release lfs_lock, because we may race with
lfs_unmark_dirop which will remove the nodes and change the flags.

Further, vp itself is valid only as long as it is referenced, which it
is as long as it's on the dchain, but lfs_unmark_dirop drops the
dchain's reference.

Don't lfs_writer_enter while holding v_interlock.

There's no need to lfs_writer_enter at all here, as far as I can see.
lfs_flush_fs will do it for us.

Break deadlock in PR kern/52301.

The lock order is lfs_writer -> lfs_seglock.  The problem in 52301 is
that lfs_segwrite violates this lock order by sometimes doing
lfs_seglock -> lfs_writer, either (a) when doing a checkpoint or (b),
opportunistically, when there are no dirops pending.  Both cases can
deadlock, because dirops sometimes take the seglock (lfs_truncate,
lfs_valloc, lfs_vfree):
(a) There may be dirops pending, and they may be waiting for the
seglock, so we can't wait for them to complete while holding the
seglock.
(b) The test for fs->lfs_dirops == 0 happens unlocked, and the state
may change by the time lfs_writer_enter acquires lfs_lock.

To resolve this in each case:
(a) Do lfs_writer_enter before lfs_seglock, since we will need it
unconditionally anyway.  The worst performance impact of this should
be that some dirops get delayed a little bit.
(b) Create a new lfs_writer_tryenter to use at this point so that the
test for fs->lfs_dirops == 0 and the acquisition of lfs_writer happen
atomically under lfs_lock.

Initialize/destroy lfs_allclean_wakeup in modcmd, not lfs_mountfs.

Fixes reloading lfs.kmod.

In lfs_update, hold lfs_writer around lfs_vflush.

Otherwise, we might do
lfs_vflush
-> lfs_seglock
-> lfs_segwait(SEGM_CKP)
   -> lfs_writer_enter
which is the reverse of the lfs_writer -> lfs_seglock ordering.

Call lfs_orphan in lfs_rename while we're still in the dirop.
lfs_writer_enter can't fail; keep it simple and don't pretend it can.

Assert that mtsleep can't fail either -- it doesn't catch signals and
there's no timeout.

Teach LFS_ORPHAN_NEXTFREE about lfs64.

Dust off the orphan detection code and try to make it work.

Fix !DIAGNOSTIC compile

Fix userland references to LFS_ORPHAN_NEXTFREE.

Forgot to grep for these or do a full distribution build, oops!

Fix missing <sys/evcnt.h> by removing the evcnts instead.

Just wanted to confirm that a race might happen, and indeed it did.
These serve little diagnostic value otherwise.

OR into bp->b_cflags; don't overwrite.

CTASSERT lfs on-disk structure sizes.

Avoid misaligned access to lfs64 on-disk records in memory.
lfs64 directory entries are only 32-bit aligned in order to conserve
space in directory blocks, and we had a hack to stuff a 64-bit inode
in them.  This replaces the hack by __aligned(4) __packed, and goes
further:

1. It's not clear that all the other lfs64 data structures are 64-bit
   aligned on disk to begin with.  We can go through these later and
   upgrade them from
        struct foo64 {
                ...
        } __aligned(4) __packed;
        union foo {
                struct foo64 f64;
                ...
        };
   to
        struct foo64 {
                ...
        };
        union foo {
                struct foo64 f64 __aligned(8);
                ...
        } __aligned(4) __packed;
   if we really want to take advantage of 64-bit memory accesses.
   However, the __aligned(4) __packed must remain on the union
   because:
2. We access even the lfs32 data structures via a union that has
   lfs64 members, and it turns out that compilers will assume access
   through a union with 64-bit aligned members implies the whole
   union has 64-bit alignment, even if we're only accessing a 32-bit
   aligned member.

Fix clang build after packed lfs64 accessor change.

Suppress spurious address-of-packed error in rump lfs too.

Mark lfs vnodes with VV_LOCKSWORK, same as ffs.

Merge changes from current as of 20200406

Merge the remaining changes from the ad-namecache branch, affecting namei()
and getcwd():

- push vnode locking back as far as possible.
- do most lookups directly in the namecache, avoiding vnode locks & refs.
- don't block new refs to vnodes across VOP_INACTIVE().
- get shared locks for VOP_LOOKUP() if the file system supports it.
- correct lock types for VOP_ACCESS() / VOP_GETATTR() in a few places.

Possible future enhancements:

- make the lookups lockless.
- support dotdot lookups by being lockless and inferring absence of chroot.
- maybe make it work for layered file systems.
- avoid vnode references at the root & cwd.

Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself.  These are not changed.

Make page waits (WANTED vs BUSY) interlocked by pg->interlock.  Gets RW
locks out of the equation for sleep/wakeup, and allows observing+waiting
for busy pages when holding only a read lock.  Proposed on tech-kern.

OR into bp->b_cflags; don't overwrite.

Sync with head.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart.  v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap.  Others to follow later.

Dust off the orphan detection code and try to make it work.

Initialize/destroy lfs_allclean_wakeup in modcmd, not lfs_mountfs.

Fixes reloading lfs.kmod.

Teach lfs to transition ro<->rw.

remove the aiodoned thread.  I originally added this to provide a thread context
for doing page cache iodone work, but since then biodone() has changed to
hand off all iodone work to a softint thread, so we no longer need the
special-purpose aiodoned thread.

Set IMNT_SHRLOOKUP and use it for the in-cache case.  Need to check what
more can be done with tmpfs though, it can probably do the whole lookup.

Sync with head.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode.  Matches
FreeBSD.

Merge from yamt-pagecache (after much testing):

- Reduce unnecessary page scan in putpages esp. when an object has a ton of
  pages cached but only a few of them are dirty.

- Reduce the number of pmap operations by tracking page dirtiness more
  precisely in uvm layer.

- Add and use wrapper functions that take and acquire page interlocks, and pairs
  of page interlocks.  Require that the page interlock be held over calls to
  uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state.  Rather than
  updating the global state synchronously, set an intended state on
  individual pages (active, inactive, enqueued, dequeued) while holding the
  page interlock.  After the interlock is released put the pages on a 128
  entry per-CPU queue for their state changes to be made real in batch.
  This results in in a ~400 fold decrease in contention on my test system.
  Proposed on tech-kern but modified to use the page interlock rather than
  atomics to synchronise as it's much easier to maintain that way, and
  cheaper.

Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code.  Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour

Sync with HEAD

s/recieve/receive/

Synch with HEAD

Add "void *extra" argument to vcache_new() so a file system may
pass more information about the file to create.

Welcome to 8.99.30

Sync with HEAD, resolve a few conflicts

Remove unused mbuf.h includes.

Sync with HEAD

add a genfs method to allow a file system to limit the range of pages
that are given to a single GOP_WRITE() call.  needed by ZFS.

update from HEAD

Pull up following revision(s) (requested by pgoyette in ticket #335):
	share/man/man9/kernhist.9: 1.5-1.8
	sys/arch/acorn26/acorn26/pmap.c: 1.39
	sys/arch/arm/arm32/fault.c: 1.105 via patch
	sys/arch/arm/arm32/pmap.c: 1.350, 1.359
	sys/arch/arm/broadcom/bcm2835_bsc.c: 1.7
	sys/arch/arm/omap/if_cpsw.c: 1.20
	sys/arch/arm/omap/tiotg.c: 1.7
	sys/arch/evbarm/conf/RPI2_INSTALL: 1.3
	sys/dev/ic/sl811hs.c: 1.98
	sys/dev/usb/ehci.c: 1.256
	sys/dev/usb/if_axe.c: 1.83
	sys/dev/usb/motg.c: 1.18
	sys/dev/usb/ohci.c: 1.274
	sys/dev/usb/ucom.c: 1.119
	sys/dev/usb/uhci.c: 1.277
	sys/dev/usb/uhub.c: 1.137
	sys/dev/usb/umass.c: 1.160-1.162
	sys/dev/usb/umass_quirks.c: 1.100
	sys/dev/usb/umass_scsipi.c: 1.55
	sys/dev/usb/usb.c: 1.168
	sys/dev/usb/usb_mem.c: 1.70
	sys/dev/usb/usb_subr.c: 1.221
	sys/dev/usb/usbdi.c: 1.175
	sys/dev/usb/usbdi_util.c: 1.67-1.70
	sys/dev/usb/usbroothub.c: 1.3
	sys/dev/usb/xhci.c: 1.75
	sys/external/bsd/drm2/dist/drm/i915/i915_gem.c: 1.34
	sys/kern/kern_history.c: 1.15
	sys/kern/kern_xxx.c: 1.74
	sys/kern/vfs_bio.c: 1.275-1.276
	sys/miscfs/genfs/genfs_io.c: 1.71
	sys/sys/kernhist.h: 1.21
	sys/ufs/ffs/ffs_balloc.c: 1.63
	sys/ufs/lfs/lfs_vfsops.c: 1.361
	sys/ufs/lfs/ulfs_inode.c: 1.21
	sys/ufs/lfs/ulfs_vnops.c: 1.52
	sys/ufs/ufs/ufs_inode.c: 1.102
	sys/ufs/ufs/ufs_vnops.c: 1.239
	sys/uvm/pmap/pmap.c: 1.37-1.39
	sys/uvm/pmap/pmap_tlb.c: 1.22
	sys/uvm/uvm_amap.c: 1.108
	sys/uvm/uvm_anon.c: 1.64
	sys/uvm/uvm_aobj.c: 1.126
	sys/uvm/uvm_bio.c: 1.91
	sys/uvm/uvm_device.c: 1.66
	sys/uvm/uvm_fault.c: 1.201
	sys/uvm/uvm_km.c: 1.144
	sys/uvm/uvm_loan.c: 1.85
	sys/uvm/uvm_map.c: 1.353
	sys/uvm/uvm_page.c: 1.194
	sys/uvm/uvm_pager.c: 1.111
	sys/uvm/uvm_pdaemon.c: 1.109
	sys/uvm/uvm_swap.c: 1.175
	sys/uvm/uvm_vnode.c: 1.103
	usr.bin/vmstat/vmstat.c: 1.219
Reorder to test for null before null deref in debug code
--
Reorder to test for null before null deref in debug code
--
KNF
--
No need for '\n' in UVMHIST_LOG
--
normalise a BIOHIST log message
--
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
   the kernel and in the structures used for exporting the history data
   to userland via sysctl(9).  This avoids problems on some architectures
   where passing a 64-bit (or larger) value to printf(3) can cause it to
   process the value as multiple arguments.  (This can be particularly
   problematic when printf()'s format string is not a literal, since in
   that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
   include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
   updated.  Each format specifier now includes an explicit length
   modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
   updated to replace uses of "%p" with "%#jx", and the pointer
   arguments are now cast to (uintptr_t) before being subsequently cast
   to (uintmax_t).  This is needed to avoid compiler warnings about
   casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
   "%c" format strings replaced with numeric formats; several instances
   of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
   history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
   the -u option does not exist (previously, this condition was silently
   ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
   data exported via sysctl(9) and exits if they do not match the values
   with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
   requirements imposed on the format strings, along with several other
   minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
     uint64_t) for the history arguments.  But that would require another
     "rototill" of all the users in the future when we add support for an
     architecture that supports a larger size.  Also, the printf(3)
format
     specifiers for explicitly-sized values, such as "%"PRIu64, are much
     more verbose (and less aesthetically appealing, IMHO) than simply
     using "%ju".
[2] I've tried very hard to find "all [the] existing users of
kernhist(9)"
     but it is possible that I've missed some of them.  I would be glad
to
     update any stragglers that anyone identifies.
--
For some reason this single kernel seems to have outgrown its declared
size as a result of the kernhist(9) changes.  Bump the size.
XXX The amount of increase may be excessive - anyone with more detailed
XXX knowledge please feel free to further adjust the value
appropriately.
--
Misssed one cast of pointer --> uintptr_t in previous kernhist(9) commit
--
And yet another one.  :(
--
Use correct mark-up for NetBSD version.
--
More improvements in grammar and readability.
--
Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.
--
And replace an instance of "%p" conversion with "%#jx"
--
Whitespace fix. Give Bl tag table a width. Fix Xr.

Pull up following revision(s) (requested by maya in ticket #330):
	sbin/fsck_lfs/inode.c: 1.69
	sbin/fsck_lfs/lfs.c: 1.73
	sbin/fsck_lfs/pass6.c: 1.50
	sbin/fsck_lfs/segwrite.c: 1.46
	sys/ufs/lfs/lfs.h: 1.202-1.203
	sys/ufs/lfs/lfs_accessors.h: 1.48
	sys/ufs/lfs/lfs_alloc.c: 1.136-1.137
	sys/ufs/lfs/lfs_balloc.c: 1.94
	sys/ufs/lfs/lfs_bio.c: 1.141
	sys/ufs/lfs/lfs_extern.h: 1.113
	sys/ufs/lfs/lfs_inode.c: 1.156-1.157
	sys/ufs/lfs/lfs_inode.h: 1.20, 1.21, 1.23
	sys/ufs/lfs/lfs_itimes.c: 1.20
	sys/ufs/lfs/lfs_pages.c: 1.13-1.15
	sys/ufs/lfs/lfs_rename.c: 1.22
	sys/ufs/lfs/lfs_segment.c: 1.270-1.275
	sys/ufs/lfs/lfs_subr.c: 1.94-1.97
	sys/ufs/lfs/lfs_syscalls.c: 1.175
	sys/ufs/lfs/lfs_vfsops.c: 1.360
	sys/ufs/lfs/lfs_vnops.c: 1.316-1.321
	sys/ufs/lfs/ulfs_inode.c: 1.20
	sys/ufs/lfs/ulfs_inode.h: 1.24
	sys/ufs/lfs/ulfs_lookup.c: 1.41
	sys/ufs/lfs/ulfs_quota2.c: 1.31
	sys/ufs/lfs/ulfs_readwrite.c: 1.24
	sys/ufs/lfs/ulfs_vnops.c: 1.49-1.50
Update inode member i_flag --> i_state to keep up with kernel changes
Move definition of IN_ALLMOD near the flag it's a mask for.
Now we can see that it doesn't match all the flags, but changing that will
require more careful thought.
Correct confusion between i_flag and i_flags
These will have to be renamed.
Spotted by Riastradh, thanks!
Add an XXX about the missing flags so it's not buried in a commit
message.
now the XXX count for LFS is 260
Rename i_flag to i_state.
The similarity to i_flags has previously caused errors.
Use continue to denote the no-op loop to match netbsd style
newline for extra clarity.
It isn't safe to drain dirops with seglock held, it'll deadlock if there
are any dirops. drain before grabbing seglock.
lfs_dirops == 0 is always true (as we already drained dirops), so omit
that part of the comparison.
Fixes a lot of LFS deadlocks. PR kern/52301
Many thanks to dholland for help analyzing coredumps
Ifdef out KDASSERT which fires on my machine.
Deduplicate sanity check that seglock is held on segunlock
Revert r1.272 fix to PR kern/52301, the performance hit is making things
unusable.
change lfs_nextsegsleep and lfs_allclean_wakeup to use condvar
XXX had to use lfs_lock in lfs_segwait, removed kernel_lock, is this
appropriate?
fix buffer overflow/KASSERT when cookies are supplied
lfs no longer uses the ffs-style struct direct, use the correct minimum
size
from dholland
XXX more wrong
Consistently use {,UN}MARK_VNODE macros rather than function calls.
Not much point doing anything after a panic call
Ask some question about the code in a XXX comment
XXX question our double-flushing of dirops
Fix typo in comment

Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
  the kernel and in the structures used for exporting the history data
  to userland via sysctl(9).  This avoids problems on some architectures
  where passing a 64-bit (or larger) value to printf(3) can cause it to
  process the value as multiple arguments.  (This can be particularly
  problematic when printf()'s format string is not a literal, since in
  that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
  include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
  updated.  Each format specifier now includes an explicit length
  modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
  updated to replace uses of "%p" with "%#jx", and the pointer
  arguments are now cast to (uintptr_t) before being subsequently cast
  to (uintmax_t).  This is needed to avoid compiler warnings about
  casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
  "%c" format strings replaced with numeric formats; several instances
  of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
  history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
  the -u option does not exist (previously, this condition was silently
  ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
  data exported via sysctl(9) and exits if they do not match the values
  with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
  requirements imposed on the format strings, along with several other
  minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
    uint64_t) for the history arguments.  But that would require another
    "rototill" of all the users in the future when we add support for an
    architecture that supports a larger size.  Also, the printf(3) format
    specifiers for explicitly-sized values, such as "%"PRIu64, are much
    more verbose (and less aesthetically appealing, IMHO) than simply
    using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
    but it is possible that I've missed some of them.  I would be glad to
    update any stragglers that anyone identifies.

Sync with HEAD

change lfs_nextsegsleep and lfs_allclean_wakeup to use condvar

XXX had to use lfs_lock in lfs_segwait, removed kernel_lock, is this
appropriate?

Restore all work from the former pgoyette-localcount branch (which is
now abandoned doe to cvs merge botch).

The branch now builds, and installs via anita.  There are still some
problems (cgd is non-functional and all atf tests time-out) but they
will get resolved soon.

Sync with HEAD

Sync with HEAD

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Add vfs_ref(mp) and vfs_rele(mp) to add or remove a reference to
struct mount.  Rename vfs_destroy(mp) to vfs_rele(mp) and replace
incrementing mp->mnt_refcnt with vfs_ref(mp).

Switch lfs_flush() and lfs_writerd() to mountlist iterator.

Switch lfs_writer_daemon to use condvar instead of mtsleep.
track thread existence with struct lwp instead of pid + lid,
it's more useful from ddb.

switch lfs_dirops to condvar (from mtsleep)

switch lfs_sleepers to condvar (from mtsleep)

Sync with HEAD

#if DIAGNOSTIC panic ---> KASSERT

Replace some #if DEBUG by this too.  DEBUG is only for expensive
assertions; these are not.

Add generic genfs_suspendctl() and use it for all file systems.
Layered file systems need work.

Adapt machine-independant code to the new {b,c}devsw reference-counting
(using localcount(9)).  All callers of {b,c}devsw_lookup() now call
{b,c}devsw_lookup_acquire() which retains a reference on the 'struct
{b,c}devsw'.  This reference must be released by the caller once it is
finished with the structure's content (or other data that would disappear
if the 'struct {b,c}devsw' were to disappear).

Sync with HEAD

KNF. Remove extra spaces. No functional change.

Merge -r1.44 of ufs_extattr.c and related change -r1.302 of ffs_vfops.c:
fix use-after-free on failed unmount with extended attributes enabled.

Sync with HEAD (as of 26th Dec)

Set the legacy ulfs fstype field to ULFS2 when mounting lfs64. Oops.

Enable mounting lfs64 volumes.

Move stuff from struct ulfsmount to struct lfs.

Remove no longer needed explicit 32->64 sign extension.

This is the last 32-bit-on-disk item among those that were either
already tagged or readily discoverable.

Sync with HEAD

Add new accessors for the d_type and d_namlen fields of struct lfs_direct.
Napalm the old byteswap access logic for these.

Make the inode fields in the 64-bit superblock 64 bits wide.
Reasoning as before.

Note that I am not going through and checking for 64->32 truncations
in inode numbers; I'm sure there are quite a few, but that's a project
for later.

Add byteswapping to the dinode accessors.

This prevents regressions in the ulfs code when switching to the new
accessors. Note that while adding byteswapping to the other accessors
is straightforward, I haven't done it yet; and that also is not enough
to make LFS_EI work, because there are places lying around that bypass
the accessors for one reason and another and all of them need to be
updated. That is going to have to wait for a later day as LFS_EI is
not on the critical path right now.

Use the lfs dinode accessors in place of the ufs-derived ones.
(Mostly.)

The ufs-derived ones are fake structure member macros, which are gross
and not very safe. Also, it seems that a lot of places in the lfs code
were using the ffsv1 branch of them unconditionally, and this way it's
guaranteed all those places have been updated.

Found while doing this: for non-devices, have getattr produce NODEV
in the rdev field instead of leaking the address of the first direct
block.

Part two of dinodes; use the same union everywhere.
(previously the ufs-derived code had things set up slightly different)

Remove a bunch of associated mess.

Hack up dinode usage to be 64 vs. 32 as needed. Part 1.

(This part changes the native lfs code; the ufs-derived code already
has 64 vs. 32 logic, but as aspects of it are unsafe, and don't
entirely interoperate cleanly with the lfs 64/32 stuff, pass 2 will be
rehashing that.)

Provide 32-bit and 64-bit versions of FINFO.

This also entailed sorting out part of struct segment, as that
contains a pointer into the current FINFO data.

Make 32-bit and 64-bit versions of SEGSUM.
Also fix some of the FINFO handling as it's closely entangled.

Add IFILE32 and IFILE64 structures for the on-disk ifile entries.
Add and use accessors. There are also a bunch of places that cast and
I hope I've found them all...

Make 32-bit and 64-bit versions of CLEANERINFO.

XXX: while this is written to disk, it seems like much of it would
XXX: be better set up as a commpage shared with the cleaner.

Fix botched syscall_package. HI CHRISTOS

Pass the fs object to LFS_MAX_DADDR so it can check lfs_is64.

Remove some hackish intentional 64->32 truncations next to the checks
using LFS_MAX_DADDR, and tackle the problem they handled in bmap
instead.

The problem: the magic block pointer value UNWRITTEN has magic value
-2, and if it's not handled specifically, uint32 -> uint64 promotion
turns it into 4294967294, which then causes consternation and
monkeyhouse downstream.

What's here is still kind of a hack, but it's a step forward.

Add a (draft) 64-bit superblock. Make things build again.

Add pieces of support for using both superblock types where
convenient, and specifically to the superblock accessors, but don't
actually enable it anywhere.

First substantive step on PR 50000.

Use accessor functions for the version field of the lfs superblock.
I thought at first maybe the cases that test the version should be
rolled into the accessors, but on the whole I think the conclusion on
that is no.

Second batch of 64 -> 32 truncations in lfs, along with more minor
tidyups and corrections in passing.

Fix assorted 64 -> 32 truncations in lfs. Also, some minor tidyups and
corrections in passing.

Add a new lfs header file: lfs_accessors.h.

This contains all the accessor functions and macros out of lfs.h.
Add an include of lfs_accessors.h after all uses of lfs.h... except
for code that wants to define its own struct lfs-alike that the
accessors are supposed to play along with. For these, set STRUCT_LFS
and include lfs_accessors.h after the necessary structure has been
defined, so that lfs_accessors.h can emit functions in terms of it.

More lfs superblock accessors.
(This changes the rest of the code over; all the accessors were
already added.)

The difference between this commit and the previous one is arbitrary,
but the previous one passed the regression tests on its own so I'm
keeping it separate to help with any bisections that might be needed
in the future.

Switch to accessor functions for elements of the LFS on-disk
superblock. This will allow switching between 32/64 bit forms on the
fly; it will also allow handling LFS_EI reasonably tidily. (That
currently doesn't work on the superblock.)

It also gets rid of cpp abuse in the form of fake structure member
macros.

Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg
inside the on-disk superblock, add extra elements to the in-memory
struct lfs for this. (XXX: these should be changed to condvars, but
not right now)

XXX: this migrates a structure needed by the lfs code in libsa (struct
salfs) into lfs.h, where it doesn't belong, but for the time being
this is necessary in order to allow the accessors (and the various
lfs macros and other goop that relies on them) to compile.

Don't cast the return value of malloc.

Fix copy and paste errors from last commits.
- Kernel i386/ALL and amd64/ALL compile again.
- Resolves CID 1304138 (DEADCODE) and 1304139 (IDENTICAL_BRANCHES).

Sync with HEAD

Change lfs from hash table to vcache.

- Change lfs_valloc() to return an inode number and version instead of
  a vnode and move lfs_ialloc() and lfs_vcreate() to new lfs_init_vnode().

- Add lfs_valloc_fixed() to allocate a known inode, used by kernel
  roll forward.

- Remove lfs_*ref(), these functions cannot coexist with vcache and
  their commented behaviour is far away from their implementation.

- Add the cleaner lwp and blockinfo to struct ulfsmount so lfs_loadvnode()
  may use hints from the cleaner.

- Remove vnode locks from ulfs_lookup() like we did with ufs_lookup().

Use VFS_PROTOS() for lfs.
Rename conflicting struct lfs field "lfs_start" to "lfs_s0addr".

No functional change.

Sync with HEAD

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

Rebase to HEAD as of a few days ago.

Rebase.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Pull up the following revisions(s) (requested by maxv in ticket #1901):
	sys/kern/vfs_syscalls.c:	revision 1.478, 1.480 via patch
	sys/coda/coda_vfsops.c:		revision 1.81
	sys/fs/adosfs/advfsops.c:	revision 1.70
	sys/fs/cd9660/cd9660_vfsops.c:	revision 1.84
	sys/fs/efs/efs_vfsops.c:	revision 1.25
	sys/fs/filecorefs/filecore_vfsops.c:	revision 1.76
	sys/fs/hfs/hfs_vfsops.c:	revision 1.31
	sys/fs/msdosfs/msdosfs_vfsops.c:	revision 1.107
	sys/fs/ntfs/ntfs_vfsops.c:	revision 1.94
	sys/fs/ptyfs/ptyfs_vfsops.c:	revision 1.50 via patch
	sys/fs/puffs/puffs_vfsops.c:	revision 1.110 via patch
	sys/fs/smbfs/smbfs_vfsops.c:	revision 1.100
	sys/fs/sysvbfs/sysvbfs_vfsops.c:	revision 1.43
	sys/fs/tmpfs/tmpfs_vfsops.c:	revision 1.59 via patch
	sys/fs/udf/udf_vfsops.c:	revision 1.67
	sys/fs/union/union_vfsops.c:	revision 1.72
	sys/fs/unionfs/unionfs_vfsops.c:	revision 1.13
	sys/kern/vfs_syscalls.c:	revision 1.479
	sys/miscfs/nullfs/null_vfsops.c:	revision 1.88 via patch
	sys/miscfs/overlay/overlay_vfsops.c:	revision 1.61
	sys/miscfs/procfs/procfs_vfsops.c:	revision 1.91
	sys/miscfs/umapfs/umap_vfsops.c:	revision 1.92
	sys/nfs/nfs_vfsops.c:		revision 1.227
	sys/ufs/ext2fs/ext2fs_vfsops.c:	revision 1.180
	sys/ufs/ffs/ffs_vfsops.c:	revision 1.297
	sys/ufs/lfs/lfs_vfsops.c:	revision 1.321
	sys/ufs/mfs/mfs_vfsops.c:	revision 1.107

Due to missing checks in the mount syscall, and a wrong assumption on the
file systems side, the kernel could allocate an unbounded or zero-sized
memory buffer, and could dereference a NULL pointer when particular
arguments are given by a user.

Pull up the following revisions(s) (requested by maxv in ticket #1901):
	sys/kern/vfs_syscalls.c:	revision 1.478, 1.480 via patch
	sys/coda/coda_vfsops.c:		revision 1.81
	sys/fs/adosfs/advfsops.c:	revision 1.70
	sys/fs/cd9660/cd9660_vfsops.c:	revision 1.84
	sys/fs/efs/efs_vfsops.c:	revision 1.25
	sys/fs/filecorefs/filecore_vfsops.c:	revision 1.76
	sys/fs/hfs/hfs_vfsops.c:	revision 1.31
	sys/fs/msdosfs/msdosfs_vfsops.c:	revision 1.107
	sys/fs/ntfs/ntfs_vfsops.c:	revision 1.94
	sys/fs/ptyfs/ptyfs_vfsops.c:	revision 1.50 via patch
	sys/fs/puffs/puffs_vfsops.c:	revision 1.110 via patch
	sys/fs/smbfs/smbfs_vfsops.c:	revision 1.100
	sys/fs/sysvbfs/sysvbfs_vfsops.c:	revision 1.43
	sys/fs/tmpfs/tmpfs_vfsops.c:	revision 1.59 via patch
	sys/fs/udf/udf_vfsops.c:	revision 1.67
	sys/fs/union/union_vfsops.c:	revision 1.72
	sys/fs/unionfs/unionfs_vfsops.c:	revision 1.13
	sys/kern/vfs_syscalls.c:	revision 1.479
	sys/miscfs/nullfs/null_vfsops.c:	revision 1.88 via patch
	sys/miscfs/overlay/overlay_vfsops.c:	revision 1.61
	sys/miscfs/procfs/procfs_vfsops.c:	revision 1.91
	sys/miscfs/umapfs/umap_vfsops.c:	revision 1.92
	sys/nfs/nfs_vfsops.c:		revision 1.227
	sys/ufs/ext2fs/ext2fs_vfsops.c:	revision 1.180
	sys/ufs/ffs/ffs_vfsops.c:	revision 1.297
	sys/ufs/lfs/lfs_vfsops.c:	revision 1.321
	sys/ufs/mfs/mfs_vfsops.c:	revision 1.107

Due to missing checks in the mount syscall, and a wrong assumption on the
file systems side, the kernel could allocate an unbounded or zero-sized
memory buffer, and could dereference a NULL pointer when particular
arguments are given by a user.

Pull up the following revisions(s) (requested by maxv in ticket #1901):
	sys/kern/vfs_syscalls.c:	revision 1.478, 1.480 via patch
	sys/coda/coda_vfsops.c:		revision 1.81
	sys/fs/adosfs/advfsops.c:	revision 1.70
	sys/fs/cd9660/cd9660_vfsops.c:	revision 1.84
	sys/fs/efs/efs_vfsops.c:	revision 1.25
	sys/fs/filecorefs/filecore_vfsops.c:	revision 1.76
	sys/fs/hfs/hfs_vfsops.c:	revision 1.31
	sys/fs/msdosfs/msdosfs_vfsops.c:	revision 1.107
	sys/fs/ntfs/ntfs_vfsops.c:	revision 1.94
	sys/fs/ptyfs/ptyfs_vfsops.c:	revision 1.50 via patch
	sys/fs/puffs/puffs_vfsops.c:	revision 1.110 via patch
	sys/fs/smbfs/smbfs_vfsops.c:	revision 1.100
	sys/fs/sysvbfs/sysvbfs_vfsops.c:	revision 1.43
	sys/fs/tmpfs/tmpfs_vfsops.c:	revision 1.59 via patch
	sys/fs/udf/udf_vfsops.c:	revision 1.67
	sys/fs/union/union_vfsops.c:	revision 1.72
	sys/fs/unionfs/unionfs_vfsops.c:	revision 1.13
	sys/kern/vfs_syscalls.c:	revision 1.479
	sys/miscfs/nullfs/null_vfsops.c:	revision 1.88 via patch
	sys/miscfs/overlay/overlay_vfsops.c:	revision 1.61
	sys/miscfs/procfs/procfs_vfsops.c:	revision 1.91
	sys/miscfs/umapfs/umap_vfsops.c:	revision 1.92
	sys/nfs/nfs_vfsops.c:		revision 1.227
	sys/ufs/ext2fs/ext2fs_vfsops.c:	revision 1.180
	sys/ufs/ffs/ffs_vfsops.c:	revision 1.297
	sys/ufs/lfs/lfs_vfsops.c:	revision 1.321
	sys/ufs/mfs/mfs_vfsops.c:	revision 1.107

Due to missing checks in the mount syscall, and a wrong assumption on the
file systems side, the kernel could allocate an unbounded or zero-sized
memory buffer, and could dereference a NULL pointer when particular
arguments are given by a user.

Pull up following revision(s) (requested by maxv in ticket #1050):
	sys/ufs/chfs/chfs_vfsops.c: revision 1.11
	sys/fs/unionfs/unionfs_vfsops.c: revision 1.13
	sys/fs/nilfs/nilfs_vfsops.c: revision 1.16
	sys/ufs/mfs/mfs_vfsops.c: revision 1.107
	sys/fs/sysvbfs/sysvbfs_vfsops.c: revision 1.43
	sys/ufs/ffs/ffs_vfsops.c: revision 1.297
	sys/kern/vfs_syscalls.c: revision 1.478
	sys/kern/vfs_syscalls.c: revision 1.479
	sys/fs/puffs/puffs_vfsops.c: revision 1.110
	sys/fs/cd9660/cd9660_vfsops.c: revision 1.84
	sys/nfs/nfs_vfsops.c: revision 1.227
	sys/fs/v7fs/v7fs_vfsops.c: revision 1.10
	sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.180
	sys/miscfs/umapfs/umap_vfsops.c: revision 1.92
	sys/fs/filecorefs/filecore_vfsops.c: revision 1.76
	sys/miscfs/nullfs/null_vfsops.c: revision 1.88
	sys/fs/ptyfs/ptyfs_vfsops.c: revision 1.50
	sys/coda/coda_vfsops.c: revision 1.81
	sys/ufs/lfs/lfs_vfsops.c: revision 1.321
	sys/fs/tmpfs/tmpfs_vfsops.c: revision 1.59
	sys/fs/hfs/hfs_vfsops.c: revision 1.31
	sys/miscfs/overlay/overlay_vfsops.c: revision 1.61
	sys/fs/union/union_vfsops.c: revision 1.72
	sys/fs/ntfs/ntfs_vfsops.c: revision 1.94
	sys/kern/vfs_syscalls.c: revision 1.480
	sys/fs/efs/efs_vfsops.c: revision 1.25
	sys/kern/vfs_syscalls.c: revision 1.482
	sys/fs/msdosfs/msdosfs_vfsops.c: revision 1.107
	external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c: revision 1.12
	sys/miscfs/procfs/procfs_vfsops.c: revision 1.91
	sys/fs/smbfs/smbfs_vfsops.c: revision 1.100
	sys/fs/adosfs/advfsops.c: revision 1.70
	sys/fs/udf/udf_vfsops.c: revision 1.67
Limit check for 'data_len'. Otherwise a (un)privileged user can easily
panic the system by passing a huge size.
ok christos@
An (un)privileged user can easily make the kernel dereference a NULL
pointer.
The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).
ok christos@
Some fs's - like kernfs - set their vfs_min_mount_data to zero. Add a check
to prevent an (un)privileged user from requesting a zero-sized allocation
(and thus a panic).
This thing is totally buggy: 'data_len' is modified by the fs, so calling
kmem_free with it while its value has changed since the kmem_alloc is far
from being a good idea.
If the kernel figures out that something mismatches, it will panic
(typically with kernfs).

Pull up following revision(s) (requested by maxv in ticket #1050):
	sys/ufs/chfs/chfs_vfsops.c: revision 1.11
	sys/fs/unionfs/unionfs_vfsops.c: revision 1.13
	sys/fs/nilfs/nilfs_vfsops.c: revision 1.16
	sys/ufs/mfs/mfs_vfsops.c: revision 1.107
	sys/fs/sysvbfs/sysvbfs_vfsops.c: revision 1.43
	sys/ufs/ffs/ffs_vfsops.c: revision 1.297
	sys/kern/vfs_syscalls.c: revision 1.478
	sys/kern/vfs_syscalls.c: revision 1.479
	sys/fs/puffs/puffs_vfsops.c: revision 1.110
	sys/fs/cd9660/cd9660_vfsops.c: revision 1.84
	sys/nfs/nfs_vfsops.c: revision 1.227
	sys/fs/v7fs/v7fs_vfsops.c: revision 1.10
	sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.180
	sys/miscfs/umapfs/umap_vfsops.c: revision 1.92
	sys/fs/filecorefs/filecore_vfsops.c: revision 1.76
	sys/miscfs/nullfs/null_vfsops.c: revision 1.88
	sys/fs/ptyfs/ptyfs_vfsops.c: revision 1.50
	sys/coda/coda_vfsops.c: revision 1.81
	sys/ufs/lfs/lfs_vfsops.c: revision 1.321
	sys/fs/tmpfs/tmpfs_vfsops.c: revision 1.59
	sys/fs/hfs/hfs_vfsops.c: revision 1.31
	sys/miscfs/overlay/overlay_vfsops.c: revision 1.61
	sys/fs/union/union_vfsops.c: revision 1.72
	sys/fs/ntfs/ntfs_vfsops.c: revision 1.94
	sys/kern/vfs_syscalls.c: revision 1.480
	sys/fs/efs/efs_vfsops.c: revision 1.25
	sys/kern/vfs_syscalls.c: revision 1.482
	sys/fs/msdosfs/msdosfs_vfsops.c: revision 1.107
	external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c: revision 1.12
	sys/miscfs/procfs/procfs_vfsops.c: revision 1.91
	sys/fs/smbfs/smbfs_vfsops.c: revision 1.100
	sys/fs/adosfs/advfsops.c: revision 1.70
	sys/fs/udf/udf_vfsops.c: revision 1.67
Limit check for 'data_len'. Otherwise a (un)privileged user can easily
panic the system by passing a huge size.
ok christos@
An (un)privileged user can easily make the kernel dereference a NULL
pointer.
The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).
ok christos@
Some fs's - like kernfs - set their vfs_min_mount_data to zero. Add a check
to prevent an (un)privileged user from requesting a zero-sized allocation
(and thus a panic).
This thing is totally buggy: 'data_len' is modified by the fs, so calling
kmem_free with it while its value has changed since the kmem_alloc is far
from being a good idea.
If the kernel figures out that something mismatches, it will panic
(typically with kernfs).

Pull up following revision(s) (requested by maxv in ticket #1050):
	sys/ufs/chfs/chfs_vfsops.c: revision 1.11
	sys/fs/unionfs/unionfs_vfsops.c: revision 1.13
	sys/fs/nilfs/nilfs_vfsops.c: revision 1.16
	sys/ufs/mfs/mfs_vfsops.c: revision 1.107
	sys/fs/sysvbfs/sysvbfs_vfsops.c: revision 1.43
	sys/ufs/ffs/ffs_vfsops.c: revision 1.297
	sys/kern/vfs_syscalls.c: revision 1.478
	sys/kern/vfs_syscalls.c: revision 1.479
	sys/fs/puffs/puffs_vfsops.c: revision 1.110
	sys/fs/cd9660/cd9660_vfsops.c: revision 1.84
	sys/nfs/nfs_vfsops.c: revision 1.227
	sys/fs/v7fs/v7fs_vfsops.c: revision 1.10
	sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.180
	sys/miscfs/umapfs/umap_vfsops.c: revision 1.92
	sys/fs/filecorefs/filecore_vfsops.c: revision 1.76
	sys/miscfs/nullfs/null_vfsops.c: revision 1.88
	sys/fs/ptyfs/ptyfs_vfsops.c: revision 1.50
	sys/coda/coda_vfsops.c: revision 1.81
	sys/ufs/lfs/lfs_vfsops.c: revision 1.321
	sys/fs/tmpfs/tmpfs_vfsops.c: revision 1.59
	sys/fs/hfs/hfs_vfsops.c: revision 1.31
	sys/miscfs/overlay/overlay_vfsops.c: revision 1.61
	sys/fs/union/union_vfsops.c: revision 1.72
	sys/fs/ntfs/ntfs_vfsops.c: revision 1.94
	sys/kern/vfs_syscalls.c: revision 1.480
	sys/fs/efs/efs_vfsops.c: revision 1.25
	sys/kern/vfs_syscalls.c: revision 1.482
	sys/fs/msdosfs/msdosfs_vfsops.c: revision 1.107
	external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c: revision 1.12
	sys/miscfs/procfs/procfs_vfsops.c: revision 1.91
	sys/fs/smbfs/smbfs_vfsops.c: revision 1.100
	sys/fs/adosfs/advfsops.c: revision 1.70
	sys/fs/udf/udf_vfsops.c: revision 1.67
Limit check for 'data_len'. Otherwise a (un)privileged user can easily
panic the system by passing a huge size.
ok christos@
An (un)privileged user can easily make the kernel dereference a NULL
pointer.
The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).
ok christos@
Some fs's - like kernfs - set their vfs_min_mount_data to zero. Add a check
to prevent an (un)privileged user from requesting a zero-sized allocation
(and thus a panic).
This thing is totally buggy: 'data_len' is modified by the fs, so calling
kmem_free with it while its value has changed since the kmem_alloc is far
from being a good idea.
If the kernel figures out that something mismatches, it will panic
(typically with kernfs).

An (un)privileged user can easily make the kernel dereference a NULL
pointer.

The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).

ok christos@

- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add  vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Change all vfsops to use C99 designated initializers.

No functional changes intended.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

Change the queue.3 *_END(&head) macros to NULL. Since we don't have CIRCLEQ
anymore, all the macros expand to NULL anyway, so this improves readability.
Requested by rmind@

change the mountlist CIRCLEQ into a TAILQ

- remove unused variables
- add debug ifdefs for debugging variables
- __USE() where appropriate.

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

sync with head

Merge in some of the stuff for supporting the extended attributes code.

Add more of the bits for supporting quotas.

Bring in a copy of ffs_quota2_mount() for reference.
Add stuff to struct lfs that it needs to initialize.
Clear these fields in mount as there's no on-disk support for quota2;
but this increases the chances of being able to add it (or something
like it) in the future.

Migrate the miscellaneous ulfs-level info from struct ulfsmount to
struct lfs.

Put them inside #ifdef _KERNEL there. They are not the only such
members, gross as that is. Unfortunately, moving struct lfs to
lfs_kernel.h does not work.

Add lfs_kernel.h for declarations that don't need to be exposed to userland.

lfs currently has the following headers:
   lfs.h - on-disk structures and stuff needed for userlevel tools
   lfs_inode.h - additional restricted materials for userlevel tools
                 that operate the fs (newfs_lfs, fsck_lfs, lfs_cleanerd)
   lfs_kernel.h - stuff needed only in the kernel

and the following legacy headers that are expected to be mopped up and
folded into one of the above:
   lfs_extern.h - function prototypes
   ulfs_bswap.h - endian-independent support
   ulfs_dinode.h - now contains very little
   ulfs_dirhash.h - dirhash support
   ulfs_extattr.h - extattr support
   ulfs_extern.h - more function prototypes
   ulfs_inode.h - assorted kernel-only declarations
   ulfs_quota.h - quota support
   ulfs_quota1.h - more quota support
   ulfs_quota2.h - more quota support
   ulfs_quotacommon.h - more quota support
   ulfsmount.h - legacy copy of ufsmount material

Get rid of the ulfs_ops table as we only have one fs in here now.

resync from head

Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs.
This was done so that boot blocks that want to compile both FFS and LFS in
the same file work.

LFS module does not depend on FFS anymore. (NAKAJIMA Yoshihiro)

Make DEBUG kernel compile: di_u.inumber -> di_inumber

DIRBLKSIZ -> LFS_DIRBLKSIZ
DIRECTSIZ -> LFS_DIRECTSIZ
DIRSIZ -> LFS_DIRSIZ
OLDDIRFMT -> LFS_OLDDIRFMT
NEWDIRFMT -> LFS_NEWDIRFMT
IFTODT -> LFS_IFTODT
DTTOIF -> LFS_DTTOIF

Stick LFS_ in front of IFMT, IFIFO, IFREG, etc. so as not to conflict
with the UFS copies of these symbols. (Which themselves ought to have
UFS_ stuck on.)

Add lfs_ or ulfs_ in front of extern symbols lacking them, mostly
quota-related (and particularly quota2-related) stuff.

Split lfs from ufs step 4:

Massedit all ufs symbols to be "ulfs" instead, to make sure there are
no conflicts with ufs. Confirmed with grep.

(This required changing a few comments that maybe should have been
left alone to say "ulfs", but we'll survive that.)

Split lfs from ufs step 3: rearrange config stuff.
Add new options:
   LFS_EI
   LFS_DIRHASH
   LFS_EXTATTR
   LFS_EXTATTR_AUTOSTART
   LFS_QUOTA
   LFS_QUOTA2

and update code referring to the corresponding FFS and UFS config
symbols to use the LFS versions. Disable the one extant reference
to APPLE_UFS in the ulfs files. Use opt_lfs.h only, not opt_ffs.h.

Split lfs from ufs, part 2:

Change all <ufs/ufs/foo.h> includes to <ufs/lfs/ulfs_foo.h>.

resync with head

sync with head

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
   ROOTINO -> UFS_ROOTINO
   WINO -> UFS_WINO
   NXADDR -> UFS_NXADDR
   NDADDR -> UFS_NDADDR
   NIADDR -> UFS_NIADDR
   MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
   MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

sync to latest -current.

sync with head.

- Replace some malloc(9) uses with kmem(9).
- G/C M_IPMOPTS, M_IPMADDR and M_BWMETER.

sync with head

sync to latest -current.

Pull up following revision(s) (requested by perseant in ticket #116):
	sys/ufs/lfs/lfs_alloc.c: revision 1.112
	tests/fs/vfs/t_rmdirrace.c: revision 1.9
	tests/fs/vfs/t_renamerace.c: revision 1.25
	sys/ufs/lfs/lfs_vnops.c: revision 1.240
	sys/ufs/lfs/lfs_segment.c: revision 1.224
	sys/ufs/lfs/lfs_bio.c: revision 1.122
	sys/ufs/lfs/lfs_vfsops.c: revision 1.294
	sbin/newfs_lfs/make_lfs.c: revision 1.19
	sys/ufs/lfs/lfs.h: revision 1.136
Pass t_renamerace and t_rmdirrace tests.
Adapt dholland@'s fix to ufs_rename to fix PR kern/43582.  Address several
other MP locking issues discovered during the course of investigating the
same problem.
Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
are controlled by the segment lock.
Fix PR kern/45982 by deemphasizing the estimate of how much metadata
will fill the empty space on disk when the disk is nearly empty
(t_renamerace crates a lot of inode blocks on a tiny empty disk).

Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

    http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
    http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
    http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

merge to -current.

Pass t_renamerace and t_rmdirrace tests.

Adapt dholland@'s fix to ufs_rename to fix PR kern/43582.  Address several
other MP locking issues discovered during the course of investigating the
same problem.

Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
are controlled by the segment lock.

Fix PR kern/45982 by deemphasizing the estimate of how much metadata
will fill the empty space on disk when the disk is nearly empty
(t_renamerace crates a lot of inode blocks on a tiny empty disk).

Change to use the updated uvm_pageout_* signature.

lfs_writerd thread exits when no more LFSs are mounted.

* Remove PGO_RECLAIM during lfs_putpages()' call to genfs_putpages(),
  to avoid a live lock in the latter when reclaiming a vnode with
  dirty pages.

* Add a new segment flag, SEGM_RECLAIM, to note when a segment is
  being written for vnode reclamation, and record which inode is being
  reclaimed, to aid in forensic debugging.

* Add a new segment flag, SEGM_SINGLE, so that opportunistic writes
  can write a single segment's worth of blocks and then stop, rather
  than writing all the way up to the cleaner's reserved number of
  segments.

* Add assert statements to check mutex ownership is the way it ought
  to be, mostly in lfs_putpages; fix problems uncovered by this.

* Don't clear VU_DIROP until the inode actually makes its way to disk,
  avoiding a problem where dirop inodes could become separated
  (uncovered by a modified version of the "ckckp" forensic regression
  test).

* Move the vfs_getopsbyname() call into lfs_writerd.  Prepare code to
  make lfs_writerd notice when there are no more LFSs, and exit losing
  the reference, so that, in theory, the module can be unloaded.  This
  code is not enabled, since it causes a crash on exit.

* Set IN_MODIFIED on inodes flushed by lfs_flush_dirops.  Really we
  only need to set IN_MODIFIED if we are going to write them again
  (e.g., to write pages); need to think about this more.

Finally, several changes to help avoid "no clean segments" panics:

* In lfs_bmapv, note when a vnode is loaded only to discover whether
  its blocks are live, so it can immediately be recycled.  Since the
  cleaner will try to choose ~empty segments over full ones, this
  prevents the cleaner from (1) filling the vnode cache with junk, and
  (2) squeezing any unwritten writes to disk and running the fs out of
  segments.

* Overestimate by half the amount of metadata that will be required
  to fill the clean segments.  This will make the disk appear smaller,
  but should help avoid a "no clean segments" panic.

* Rearrange lfs_writerd.  In particular, lfs_writerd now pays
  attention to the number of clean segments available, and holds off
  writing until there is room.

VOP_OPEN() needs a locked vnode.  All these copy-and-pasted xxxfs_mount()
implementations need more review.

page cache related changes

- maintain object pages in radix tree rather than rb tree.
- reduce unnecessary page scan in putpages.  esp. when an object has a ton of
  pages cached but only a few of them are dirty.
- reduce the number of pmap operations by tracking page dirtiness more
  precisely in uvm layer.
- fix nfs commit range tracking.
- fix nfs write clustering.  XXX hack

Change VOP_BWRITE() to take a vnode as its first argument like all other
VOPs do.  Layered file systems no longer have to modify bp->b_vp and run
into trouble when an async VOP_BWRITE() uses the wrong vnode.

- change all occurences of VOP_BWRITE(bp) to VOP_BWRITE(bp->b_vp, bp).
- remove layer_bwrite().
- welcome to 5.99.55

Adresses PR kern/38762 panic: vwakeup: neg numoutput

No objections from tech-kern@.

Catchup with rmind-uvmplock merge.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

Sync with HEAD.

Implement sharing of vnode_t::v_interlock amongst vnodes:
- Lock is shared amongst UVM objects using uvm_obj_setlock() or getnewvnode().
- Adjust vnode cache to handle unsharing, add VI_LOCKSHARE flag for that.
- Use sharing in tmpfs and layerfs for underlying object.
- Simplify locking in ubc_fault().
- Sprinkle some asserts.

Discussed with ad@.

sync with head

merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.

Various build fixes

Sync with HEAD.

sync with head.

sync with head

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Sync with HEAD.

Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc().  It allows us to share the locks among UVM objects.

sync with head

load lfs syscalls in modload

/*
 * XXX: Get extra reference to LFS vfsops.  This prevents unload,
 * but also prevents kernel panic due to text being unloaded
 * from below lfs_writerd.  When lfs_writerd can exit, remove
 * this!!!
 */

Fix root filesystem support.

Three changes in a single commit.

- drop the notion of frags (LFS fragments) vs fsb (FFS fragments)
  The code uses a complicated unity function that just makes the
  code difficult to understand.

- support larger sector sizes. Fix disk address computations
  to use DEV_BSIZE in the kernel as required by device drivers
  and to use sector sizes in userland.

- Fix several locking bugs in lfs_bio.c and lfs_subr.c.

The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase).  Plenty of mix'n match upper/lowercase has creeped
into the tree since then.  Nuke the macros and convert all callsites
to lowercase.

no functional change

Fix some more hangs and deadlocks.

Create unwind log in global variable instead of automatic variable.

memory leak spotted by njoly's valgrind run

Fix up numoutput accounting.

sync with head

Move declaration of ufs_hashlock into <ufs/ufs_extern.h> from each c source.

sync with head.

Compensate v_numoutput & nestbuf for lfs's rather peculiar I/O habits.

remember to nestiobuf_done() too

Use nestiobuf instead of homerolled equivalent.

Sync with HEAD.

sync with head.

Convert 67 namei call sites to use namei_simple, in these functions:

check_console, veriexecclose, veriexec_delete, veriexec_file_add,
emul_find_root, coff_load_shlib (sh3 version), coff_load_shlib,
compat_20_sys_statfs, compat_20_netbsd32_statfs,
ELFNAME2(netbsd32,probe_noteless), darwin_sys_statfs,
ibcs2_sys_statfs, ibcs2_sys_statvfs, linux_sys_uselib,
osf1_sys_statfs, sunos_sys_statfs, sunos32_sys_statfs,
ultrix_sys_statfs, do_sys_mount, fss_create_files (3 of 4),
adosfs_mount, cd9660_mount, coda_ioctl, coda_mount, ext2fs_mount,
ffs_mount, filecore_mount, hfs_mount, lfs_mount, msdosfs_mount,
ntfs_mount, sysvbfs_mount, udf_mount, union_mount, sys_chflags,
sys_lchflags, sys_chmod, sys_lchmod, sys_chown, sys_lchown,
sys___posix_chown, sys___posix_lchown, sys_link, do_sys_pstatvfs,
sys_quotactl, sys_revoke, sys_truncate, do_sys_utimes, sys_extattrctl,
sys_extattr_set_file, sys_extattr_set_link, sys_extattr_get_file,
sys_extattr_get_link, sys_extattr_delete_file,
sys_extattr_delete_link, sys_extattr_list_file, sys_extattr_list_link,
sys_setxattr, sys_lsetxattr, sys_getxattr, sys_lgetxattr,
sys_listxattr, sys_llistxattr, sys_removexattr, sys_lremovexattr

All have been scrutinized (several times, in fact) and compile-tested,
but not all have been explicitly tested in action.

XXX: While I haven't (intentionally) changed the use or nonuse of
XXX: TRYEMULROOT in any of these places, I'm not convinced all the
XXX: uses are correct; an audit might be desirable.

sync with head

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

Use genfs_can_mount().

sync with head.

Sync with HEAD.

Pull up following revision(s) (requested by ad in ticket #662):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.272
Turn up the volume on the warning message a bit.

Turn up the volume on the warning message a bit.

ansify function definitions

Sync with HEAD.

PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.

Sync with HEAD.

Sync with HEAD.

Update haad-dm branch to haad-dm-base2.

These depend on ffs.

Remove #ifdef LFS from the ufs code.

Sync with wrstuden-revivesa-base-2.

Sync with head.

Sync with HEAD.

Fix lkm fallout from previous sysctl changes. This largely duplicates
sysctl creation code, but lkms are going away soon(ish) anyway.

Spotted by Chris Gilbert.

Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.

Sync w/ -current. 34 merge conflicts to follow.

sync with head

Sync with HEAD.

s/log file system/log-structured file system/

Don't moan about LFS unless the mount succeeds.

Until these get fixed or replaced:

WARNING: the foo file system is experimental and may be unstable

sync with head.

Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write.  Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn().  If set the caller
  intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
  may clear the buffer and runs copy-on-write.  Process possible errors
  from getblk() or fscow_run().  Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

sync with head.

Convert file systems to dynamically attach with the new module interface.
Make VFS hooks dynamic while we're here and say farewell to VFS_ATTACH and
VFS_HOOKS_ATTACH linksets.

As a consequence, most of the file systems can now be loaded as new style
modules.

Quick sanity check by ad@.

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
  sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
  and is only ever write locked in dounmount(). A write hold can't be taken
  on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
  example when going r/o -> r/w, and is only present to serialize updates.
  In order to take this lock, a read hold must first be taken on
  mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

kern/38135 vfs_busy/vfs_trybusy confusion

The symptom was that sometimes file systems would occasionally not appear
in output from 'df' or 'mount' if the system was busy. Resolution:

- Make mount locks work somewhat like vm_map locks.
- vfs_trybusy() now only fails if the mount is gone, or if someone is
  unmounting the file system. Simple contention on mnt_lock doesn't
  cause it to fail.
- vfs_busy() will wait even if the file system is being unmounted.

PR kern/38057 ffs makes assuptions about devvp file system
PR kern/33406 softdeps get stuck in endless loop

Introduce VFS_FSYNC() and call it when syncing a block device, if it
has a mounted file system.

Remove clause 3 and 4 from TNF licenses

sync with HEAD

Sync with HEAD.

sync with head.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
  mount takes a reference, and in turn the mount takes a reference to the
  vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
  locking inherited from 4.4BSD with a recursable rwlock.

Fix some race conditions in rename.
Introduce a per-FS rename lock and new vfsops to manipulate it.
Get this lock while renaming. Also add another relookup() in do_sys_rename,
which is a hack to kludge around some of the worst deficiencies of
ufs_rename.
reviewed-by: pooka (and an earlier rev by ad)
posted on tech-kern with no objections.

specfs changes for PR kern/37717 (raidclose() is no longer called on
shutdown). There are still problems with device access and a PR will be
filed.

- Kill checkalias(). Allow multiple vnodes to reference a single device.

- Don't play dangerous tricks with block vnodes to ensure that only one
  vnode can describe a block device. Instead, prohibit concurrent opens of
  block devices. As a bonus remove the unreliable code that prevents
  multiple file system mounts on the same device. It's no longer needed.

- Track opens by vnode and by device. Issue cdev_close() when the last open
  goes away, instead of abusing vnode::v_usecount to tell if the device is
  open.

sync with head

sync with HEAD

Sync with HEAD

Merge vmlocking2 to head.

Sync with HEAD.

Sync with head.

Use a global lfs_lock.

Fix some more problems w/lfs on this branch.

Get lfs mostly working.

Sync with HEAD

sync with head.

defflag LFS_KERNEL_RFW (in opt_lfs.h).
Note: lfs_rfw.c doesn't compile if you define the option; locking API fallout?

Sync with HEAD.

Remove cn_lwp from struct componentname.  curlwp should be used
from on.  The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

Sync with HEAD.

sync with head

Pull the vmlocking changes into a new branch.

Sync with HEAD. amd64 Xen support needs testing.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start.  In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

Sync with HEAD

lfs_mountroot: use vfs_destroy.

Sync with HEAD.

sync with head.

Sync with HEAD

Sync with HEAD.

Use PRI_BIO for kthreads instead of PINOD.  Fixes a missed case of priority
inversion, which caused LFS to fire some assertions.

Reported by Kurt Schreiner on <current-users>.

sync with HEAD

sync with head.

Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.

sync with head.

Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge ffs locking & brelse changes from the vmlocking branch.

- Checkpoint work in progress on the vnode lifecycle and reference counting
  stuff. This makes it work properly without kernel_lock and fixes a few
  quite old bugs. See vfs_subr.c 1.283.2.17 for details.

- Fix some problems with softdep. Unfortunately our softdep code appears
  to have some longstanding bugs that cause it fail under stress test.

sync with head.

Sync w/ NetBSD-4-RC_1

Sync with buffer cache locking changes. See buf.h/vfs_bio.c for details.
Some minor portions are incomplete and needs to be verified as a whole.

fix some races around pagedaemon and uvm_wait.  ok'ed by Andrew Doran.

Sync with HEAD.

- Back out the biodone() changes.
- Eliminate B_ERROR (from HEAD).

Sync with HEAD.

* nuke the nameidata parameter from VFS_MOUNT().  Nobody on tech-kern
  knew what it was supposed to be used for and wrstuden gave a go-ahead
* while rototilling, convert file systems which went easily to
  use VFS_PROTOS() instead of manually prototyping the methods

file lfs_vfsops.c was added on branch matt-mips64 on 2007-07-31 21:14:21 +0000

* nuke the nameidata parameter from VFS_MOUNT().  Nobody on tech-kern
  knew what it was supposed to be used for and wrstuden gave a go-ahead
* while rototilling, convert file systems which went easily to
  use VFS_PROTOS() instead of manually prototyping the methods

It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.

Use eopnotsupp() instead of vfs_stdsuspendctl() and retire the latter.

Workaround the ufs_haslock/ufs_ihash_lock deadlock. From a patch
posted by Blair Sadewitz.

Eliminate MFSNAMELEN

Make set_statvfs_info() take a parameter for the vfs name instead
of always retrieving it from mp->mnt_op->vfs_name

christos ok

Sync with head.

Change the VFS_MOUNT() interface so that the 'data' buffer passed to the
fs code is a kernel buffer, pass though the length of the buffer as well.
Since the length of the userspace buffer isn'it (yet) passed through the mount
system call, add a field to the vfsops structure containing the default length.
Split sys_mount() for calls from compat code.
Ride one of the recent kernel version changes - old fs LKMs will load, but
sys_mount() will reject any attempt to use them.

Sync with head.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Using POOL_INIT here makes no sense, since file systems always have
an init method.  So get rid of it and #ifdef _LKM and just always
init in the init method.  Give malloc types the same treatment.
Makes file systems nicer to work with in linksetless environments
and fixes a few LKM discrepancies.

- Lock v_cleanblkhd, v_dirtyblkhd, v_numoutput with the vnode's interlock.
  Get rid of global_v_numoutput_lock. Partially incomplete as the buffer
  cache locking doesn't work very well and needs an overhaul.
- Some changes to try and make softdep MP safe. Untested.

Sync with head.

Pull up following revision(s) (requested by perseant in ticket #703):
	sys/miscfs/genfs/genfs.h              1.21
	sys/miscfs/genfs/genfs_vnops.c        1.151
	sys/ufs/lfs/lfs.h                     1.119, 1.120
	sys/ufs/lfs/lfs_bio.c                 1.99-101
	sys/ufs/lfs/lfs_extern.h              1.89
	sys/ufs/lfs/lfs_inode.c               1.108, 1.109
	sys/ufs/lfs/lfs_segment.c             1.197, 1.199, 1.200
	sys/ufs/lfs/lfs_subr.c                1.69, 1.70
	sys/ufs/lfs/lfs_syscalls.c            1.119
	sys/ufs/lfs/lfs_vfsops.c              1.234, 1.235
	sys/ufs/lfs/lfs_vnops.c               1.195, 1.196, 1.200, 1.202-206

Reduce busy waiting in lfs_putpages(), and other LFS improvements.

sync with head.

Change references to SEGM_W_DIROPS to SEGM_CKP, and replace the logic that
formerly used SEGM_W_DIROPS in lfs_segwrite() appropriately.  This prevents
a problem in which processes could get stuck in "buffers" sleep forever.

- Pass the error number and residual count to biodone(), and let it handle
  setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
  of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
  create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.

sync with head.

Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore
the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS.  The
default is "on", i.e., ignore lazy sync.

Reduce the amount of polling/busy-waiting done by lfs_putpages().  To
accomplish this, copied genfs_putpages() and modified it to indicate which
page it was that caused it to return with EDEADLK.  fsync()/fdatasync()
should no longer ever fail with EAGAIN, and should not consume huge
quantities of cpu.

Also, try to make dirops less likely to be written as the result of a
VOP_PUTPAGES(), while ensuring that they are written regularly.

Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.

- Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.

sync with head.

Pull in the initial set of changes for the vmlocking branch.

Sync with head.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

- sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.

sync with head.

TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type.  A
future commit will replace use of TRUE and FALSE with true and false.

Release ufs_hashlock before calling ungetnewvnode().

Destroy the fraglock on unmount.

Replace some uses of lockmgr() / simplelocks.

Sync with head.

New file system suspension API to replace vn_start_write and vn_finished_write.
The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.

Implemented for file systems of type ffs.

The new API is enabled on a kernel option NEWVNGATE.  This option is
not enabled by default in any kernel config.

Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.

Welcome to 4.99.9 (new vfs op vfs_suspendctl).

Sync with head.

Consistent usage of KAUTH_GENERIC_ISSUSER.

sync with head.

sync with head.

Sync with head.

__unused removal on arguments; approved by core.

sync with head

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

fix empty if

Use lockstatus instead of a homebrewed locking system to control
LFCNWRAPSTOP and LFCNWRAPGO.

Be less verbose about the various looping checks: use log() rather than
printf(), and only log anything if we are really looping ("count = 2" is
not an error condition).

Allow dirops sleeping on available space to be interruptible.

sync with head

sync with head.

- add missing initializers
- comment out impossible code

Changes to help the roll-forward agent, to wit:

* Mark being-deleted files in the Ifile so we can finish deleting them
  at fs mount time.
* Flag the Ifile with "cleaner must clean" when writers are waiting for
  the cleaner, rather than relying solely on the cleaner's estimation of
  whether it should clean or not.
* Note partial segments written by a user agent (in particular,
  fsck_lfs) so that repeated rolls forward don't interfere with one
  another.
* Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once,
  for better testing of the validity of checkpoints.
* Keep track of the on-disk nlink count when cleaning, so that we don't
  partially complete directory operations while cleaning.
* Ensure that every single Ifile inode write represents a consistent
  view of the filesystem.  In particular, the accounting for the segment
  we are writing the inode into must be correct, and the accounting for
  the segment that inode used to reside in must be correct.  Rather than
  just rewriting the inode if we wrote it wrong, rewrite the necessary
  ifile blocks before writing the inode so we never write it wrong.
* Don't unmark any VDIROP vnodes if we haven't written them to disk,
  avoiding yet another problem with the "wait for the cleaner" error
  return from lfs_putpages().

Also, move the last callback to an aiodone call, so we no longer do any
memory management from interrupt context.

sync with head

Apply patch (requested by fair in perseant #1457):
Bring LFS up to current, including a patch (1.95 lfs_alloc.c) that
should prevent the inode free list errors seen on the STABLE branch
subsequent to pullup ticket #1327.

Use the LWP cached credentials where sane.

Separate the (non-working) LFS kernel roll-forward code into its own file,
lfs_rfw.c.

Merge from HEAD.

Fix alignement problems for fhandle_t, exposed by gcc4.1.

While touching all vptofh/fhtovp functions, get rid of VFS_MAXFIDSIZ,
version the getfh(2) syscall and explicitly pass the size available in
the filehandle from userland.

Discussed on tech-kern, with lots of help from yamt (thanks!).

Fix a typo that caused a "multiple free" panic on unmounting a resized lfs.

Don't wake up the cleaner if the filesystem is unwrappable, and fix the
compatibility fcntls.

Also includes one-line fixes for an MP locking bug and a zero-length FINFO
problem that manifested during testing.

sync with head.

sync with head.

Sync with head.

Sync with head.

Read the inode version number fro a more reliable source, quelling a
diagnostic assertion panic.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_alloc.c: revision 1.93
	sys/ufs/lfs/lfs.h: revision 1.106
	sys/ufs/lfs/lfs_vfsops.c: revision 1.209
	sys/ufs/lfs/lfs_vnops.c: revision 1.175
	sys/ufs/lfs/lfs_segment.c: revision 1.178
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when
many inodes are cleaned at once.  Make sure that we write all the pages
on vnodes that are being flushed, even if we don't think there's room;
drain v_numoutput before lfs_vflush() completes.
Also, don't allow a vnode that is in the process of being cleaned to be
chosen by getnewvnode(); this avoids a segment accounting panic in the case
that a large number of inodes are fed to lfs_markv() all at once.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_alloc.c: revision 1.92
	sys/ufs/lfs/lfs.h: revision 1.105
	sys/ufs/lfs/lfs_vfsops.c: revision 1.207
	sys/ufs/lfs/lfs_subr.c: revision 1.59
	sys/ufs/lfs/lfs_vnops.c: revision 1.173
	sys/ufs/lfs/lfs_bio.c: revision 1.92
Introduce another per-filesystem parameter, lfs_resvseg, to separate the
notion of "how many segments are reserved for the cleaner" from that of
"how many segments are not counted in lfs_bfree".  The default value
used for existing filesystems is the same as the previous implicit value
of (lfs_minfreeseg / 2 + 1), modulo some sanity checking.
Count pending dirops on a per-filesystem basis, since once we start
writing them we can't stop until we're done.  This seems to help stave off
the "no clean segments" panic in the case of filling the filesystem with
directories and small files (e.g. simultaneously unpacking more copies of
pkgsrc than will fit).

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs.h: revision 1.104
	sys/ufs/lfs/lfs_vfsops.c: revision 1.206
	sys/ufs/lfs/lfs_vnops.c: revision 1.170
	sys/ufs/lfs/lfs_extern.h: revision 1.80
	sys/ufs/lfs/lfs_segment.c: revision 1.176
	sys/ufs/lfs/lfs_inode.c: revision 1.103 via patch
	sys/ufs/lfs/lfs_alloc.c: revision 1.90
Postpone the segment accounting changes coming from truncation until the
inode that makes those changes valid is either written to disk by
lfs_writeinode() or discarded by lfs_vfree().
A couple of locking fixes are also included as well.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.205
Don't roll forward if we aren't given a process context.  Coverity CID 1076.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.204 via patch
Coverity CID 2499: Fix uninitialize variable use.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.203 via patch
Remove mostly useless BUFPAGES warning message from lfs_{un,}mount.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs.h: revision 1.101
	sys/ufs/lfs/lfs_vfsops.c: revision 1.202
	sys/ufs/lfs/lfs_alloc.c: revision 1.88
Optimize the free list search a little more; in particular use words
instead of bytes for the index, and never search below fs->lfs_freehd.
Fix a bug in the previous version of the search (an erroneous assumption
that ino_t was signed).
Free the bitmap when we unmount the filesystem.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.201
Correct a locking bug in the recent pager optimization.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.200
	sys/ufs/lfs/lfs_vnops.c: revision 1.164
	sys/ufs/lfs/lfs_inode.c: revision 1.101
	sys/ufs/lfs/lfs_extern.h: revision 1.78
	sys/ufs/lfs/lfs.h: revision 1.100
Implement a somewhat finer-grained mechanism for paging LFS-backed pages.
The writer daemon, if it does not need to flush the whole filesystem,
now only writes the vnodes for which the pagedaemon has requested pageouts
(although it does not pay attention to the page ranges the pagedaemon
supplies).

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_alloc.c: revision 1.87
	sys/ufs/lfs/lfs.h: revision 1.99
	sys/ufs/lfs/lfs_vfsops.c: revision 1.199
	sys/ufs/lfs/lfs_extern.h: revision 1.77 via patch
Keep the free list ordered.  This solves a problem first pointed out to me
by Michel Oey, in which an aged LFS writes up to an extra Ifile block for
every file created; and paves the way for the truncation of the Ifile when
many files are deleted.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.198
	sys/ufs/lfs/lfs_vnops.c: revision 1.161
Handle the "filesystem is clean" flag correctly when upgrading from
read-only to read-write mount.  This makes "root on lfs" work for me,
although it looks like a different traceback from PR#32667.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.196
Double-checkpoint on unmount.  This ensures that vnodes belonging to removed
files are really freed, preventing occasional spurious EBUSY returns from
vflush().

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vnops.c: revision 1.158
	sys/ufs/lfs/lfs_subr.c: revision 1.57
	sys/ufs/lfs/lfs_segment.c: revision 1.171
	sys/ufs/lfs/lfs.h: revision 1.97
	sys/ufs/lfs/lfs_vfsops.c: revision 1.195
	sys/ufs/lfs/lfs_extern.h: revision 1.76
Improvements to LFS's paging mechanism, to wit:
* Acknowledge that sometimes there are more dirty pages to be written to
   disk than clean segments.  When we reach the danger line,
   lfs_gop_write() now returns EAGAIN.  The caller of VOP_PUTPAGES(), if
   it holds the segment lock, drops it and waits for the cleaner to make
   room before continuing.
* Note and avoid a three-way deadlock in lfs_putpages (a writer holding
   a page busy blocks on the cleaner while the cleaner blocks on the
   segment lock while lfs_putpages blocks on the page).

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_segment.c: revision 1.170
	sys/ufs/lfs/lfs.h: revision 1.96
	sys/ufs/lfs/lfs_vfsops.c: revision 1.194
	sys/ufs/lfs/lfs_syscalls.c: revision 1.109
From Konrad Schroeder, in response to strange df output on anoncvs.netbsd.org:
We were returning the wrong value for free space.  Now we're not.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vnops.c: revision 1.153
	sys/ufs/lfs/lfs_debug.c: revision 1.32
	sys/ufs/lfs/lfs_alloc.c: revision 1.84
	sys/ufs/lfs/lfs_vfsops.c: revision 1.185
	sys/ufs/lfs/lfs_segment.c: revision 1.165
64 bit inode changes.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vnops.c: revision 1.152
	sys/ufs/lfs/lfs_debug.c: revision 1.31
	sys/ufs/lfs/lfs_subr.c: revision 1.53
	sys/ufs/lfs/lfs_extern.h: revision 1.68
	sys/ufs/lfs/lfs_inode.c: revision 1.96
	sys/ufs/lfs/lfs_bio.c: revision 1.86
	sys/ufs/lfs/lfs_alloc.c: revision 1.83
	sys/ufs/lfs/lfs_vfsops.c: revision 1.181
	sys/ufs/lfs/lfs.h: revision 1.88
	sys/ufs/lfs/lfs_segment.c: revision 1.164
- sprinkle const
- avoid shadow variables.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.180
	sys/ufs/lfs/lfs_syscalls.c: revision 1.106
	sys/ufs/lfs/lfs.h: revision 1.87
Keep track of the number of segments reclaimed, since the cleaner doesn't
do this anymore (it hasn't for quite some time).  Add a couple of conditional
debugging messages to indicate why segments are not cleaned, in the event
that lfs_segclean is used.
Make the LFCNSEGWAITALL fcntl work again.

Pull up following revision(s) (requested by perseant in ticket #1327):
	sys/ufs/lfs/lfs_vfsops.c: revision 1.179
Fill in the lfs_fsmnt field in the superblock when we mount the filesystem,
so fsck(8) can tell where it was last mounted.

Break out the finfo array manipulation code into two new functions,
lfs_acquire_finfo() and lfs_release_finfo().  Add a debugging check
for zero-length finfo arrays in the segment summary to avoid future
regressions.

Don't duplicate the LFS_STARVED_FOR_SEGS check (an oversight that came
in with rev 1.210).

integrate kauth.

Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when
many inodes are cleaned at once.  Make sure that we write all the pages
on vnodes that are being flushed, even if we don't think there's room;
drain v_numoutput before lfs_vflush() completes.

Also, don't allow a vnode that is in the process of being cleaned to be
chosen by getnewvnode(); this avoids a segment accounting panic in the case
that a large number of inodes are fed to lfs_markv() all at once.

sync with head

quell GCC 4.1 uninitialised variable warnings.

XXX: we should audit the tree for which old ones are no longer needed
after getting the older compilers out of the tree..

- Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
  that need it.

Approved by core.

Introduce another per-filesystem parameter, lfs_resvseg, to separate the
notion of "how many segments are reserved for the cleaner" from that of
"how many segments are not counted in lfs_bfree".  The default value
used for existing filesystems is the same as the previous implicit value
of (lfs_minfreeseg / 2 + 1), modulo some sanity checking.

Count pending dirops on a per-filesystem basis, since once we start
writing them we can't stop until we're done.  This seems to help stave off
the "no clean segments" panic in the case of filling the filesystem with
directories and small files (e.g. simultaneously unpacking more copies of
pkgsrc than will fit).

Postpone the segment accounting changes coming from truncation until the
inode that makes those changes valid is either written to disk by
lfs_writeinode() or discarded by lfs_vfree().

A couple of locking fixes are also included as well.

Sync with head.

sync with head.