CVS log for src/sys/kern/vfs_vnode.c

Up to [cvs.NetBSD.org] / src / sys / kern

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

vfs(9): Sprinkle SET_ERROR dtrace probes.

PR kern/58378: Kernel error code origination lacks dtrace probes

vfs(9): Fix some more whitespace issues.

No functional change intended.

vfs(9): Sprinkle KNF.

No functional change intended.

Restore kpause() accidentially removed with last commit.

Implement and use an iterator over LRU lists.

Replace the vdrain kernel thread with two threadpool jobs,
one to process deferred vrele and
one to keep the number of allocated vnodes below limit.

vfs(9): Make sure to kpause at least one tick, not zero.

kpause(9) forbids zero.

Local workaround for wider problem in PR kern/57718, to address
immediate symptom of crash on any system with hz=50, e.g. alpha in
qemu:

panic: kernel diagnostic assertion "timo != 0 || intr" failed: file "/usr/src/sys/kern/kern_synch.c", line 249

XXX pullup-10
XXX pullup-9
XXX pullup-8

As the number of allocated vnodes goes beyond 106% of desiredvnodes
start throttling threads allocating new vnodes at a rate of ~100 new
vnodes per second and thread.

kern: Eliminate most __HAVE_ATOMIC_AS_MEMBAR conditionals.

I'm leaving in the conditional around the legacy membar_enters
(store-before-load, store-before-store) in kern_mutex.c and in
kern_lock.c because they may still matter: store-before-load barriers
tend to be the most expensive kind, so eliding them is probably
worthwhile on x86.  (It also may not matter; I just don't care to do
measurements right now, and it's a single valid and potentially
justifiable use case in the whole tree.)

However, membar_release/acquire can be mere instruction barriers on
all TSO platforms including x86, so there's no need to go out of our
way with a bad API to conditionalize them.  If the procedure call
overhead is measurable we just could change them to be macros on x86
that expand into __insn_barrier.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/02/23/msg028729.html

_vstate_assert: Use atomic_load/store_relaxed.  Omit membar_enter.

Can't find anything this is supposed to pair with.  Pretty sure this
is just an optimistic unlocked test, not actually reliant on memory
ordering.  But as it is unlocked, it needs to be coordinated with
atomic_load/store_relaxed, not ordinary loads or stores, if for no
other reason than to pacify sanitizers.

No need in vnalloc_marker or vcache_alloc because these still have
exclusive access to the vnode at that point.

XXX Should deduplicate the logic in vstate_assert_change and
vstate_change.

miscfs/specfs/specdev.h: New home for extern spec_vnodeop_opv_desc.

Also use it for extern spec_vnodeop_p, which is already there.

miscfs/deadfs/deadfs.h: New home for deadfs-related externs.

XXX regen sys/kern/vnode_if.c and the others

In vcache_reclaim(), post NOTE_REVOKE immediately after changing the
vnode state to VS_RECLAIMING, before we actually call VOP_RECLAIM(),
which will release the reference on the lower node of a stacked FS
vnode, which is likely to free the upper node's v_klist backing store.

Acquire the vnode interlock when checking for kevent interest now,
because the vp->v_klist pointer is now volatile.

PR kern/56950

Make kqueue event status for vnodes shareable, and for stacked file systems
like nullfs, make the upper vnode share that status with the lower vnode.

And, lo, NetBSD 9.99.99.

Fixes PR kern/56713.

vfs(9): Add XXX comment about unclear membar_enter.

sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.

specfs: Remove specnode from hash table in spec_node_revoke.

Previously, it was possible for spec_node_lookup_by_dev to handle a
speconde that a concurrent spec_node_destroy is about to remove from
the hash table and then free, as soon as spec_node_lookup_by_dev
releases device_lock.

Now, the ordering is:

1. Remove specnode from hash table in spec_node_revoke.  At this
   point, no _new_ vnode references are possible (other than possibly
   one acquired by vcache_vget under v_interlock), but there may be
   existing ones.

2. Mark vnode reclaimed so vcache_vget will fail.

3. The last vrele (or equivalent logic in vcache_vget) will then free
   the specnode in spec_node_destroy.

This way, _if_ a thread in spec_node_lookup_by_dev finds a specnode
in the hash table under device_lock/v_interlock, _then_ it will not
be freed until the thread completes vcache_vget.

This change requires calling spec_node_revoke unconditionally for
device special nodes, not just for active ones.  Might introduce
slightly more contention on device_lock but not much because we
already have to take it in this path anyway a little later in
spec_node_destroy.

specfs: Let spec_node_lookup_by_dev wait for reclaim to finish.

vdevgone relies on this to ensure that if there is a concurrent
revoke in progress, it will wait for that revoke to finish -- that
way, it can guarantee all I/O operations have completed and the
device is closed.

Remove now unused VV_LOCKSWORK, all file systems support locking.

Remove unused predicates vn_locked() and vn_anylocked().

Welcome to 9.99.95

Switch spec_vnodeop vector to real vnode locking, VV_LOCKSWORK now.

vrelel(): No need to test usecount if VGET marker is clear.
Assert "usecount == 1" instead.

sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

	  Thread A			  Thread B
	obj->foo = 42;			obj->baz = 73;
	mumble(&obj->bar);		grumble(&obj->quux);
	/* membar_exit(); */		/* membar_exit(); */
	atomic_dec -- not last		atomic_dec -- last
					/* membar_enter(); */
					KASSERT(invariant(obj->foo,
					    obj->bar));
					free_stuff(obj);

The memory barriers ensure that

	obj->foo = 42;
	mumble(&obj->bar);

in thread A happens before

	KASSERT(invariant(obj->foo, obj->bar));
	free_stuff(obj);

in thread B.  Without them, this ordering is not guaranteed.

So in general it is necessary to do

	membar_exit();
	if (atomic_dec_uint_nv(&obj->refcnt) != 0)
		return;
	membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting.  (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these.  Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that.  It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.

vrelel(): after all locks are in place check for new reference again.

Should fix assertion "vp->v_iflag & VI_TEXT" under load.

vrelel(): no VOP_UNLOCK() with v_interlock or vmobjlock held.

Do the space accounting before VOP_INACTIVE() so we may
unlock the vnode after VOP_INCATIVE().

This was the last call from vrelel() to VOP_UNLOCK() with v_interlock held.

Add a marker VUSECOUNT_VGET to v_usecount that gets set whenever
vcache_vget() or vache_tryvget() succeeds.

Use it to rerun VOP_INACTIVE() if another thread ran a vget()..vrele()
cycle while we inactivated our last reference.

If the vnode to vrelel() is already reclaimed there is no need
to lock or defer it.  Jump straight to decrement usecount and requeue.

Add inline functions to manipulate the klists that link up knotes
via kn_selnext:

- klist_init()
- klist_fini()
- klist_insert()
- klist_remove()

These provide some API insulation from the implementation details of these
lists (but not completely; see vn_knote_attach() and vn_knote_detach()).
Currently just a wrapper around SLIST(9).

This will make it significantly easier to switch kn_selnext linkage
to a different kind of list.

Operation vfs_suspend() returns ENOENT if the mount is gone (IMNT_GONE).

Adjust the KASSERT() appropriately.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
  forcing each individual file system to deal with it (except VOP_RENAME(),
  because VOP_RENAME() is a mess and we currently have 2 different ways
  of handling it; at least it's reasonably well-centralized in the "new"
  way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
  compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
  to avoid doing work for events no one cares about (avoiding, e.g.
  taking locks and traversing the klist to send a NOTE_WRITE when
  someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
  to be invoked before and after vop_pre() and vop_post(), respectively.
  Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
  vop_*_args structures.  These context fields are used to convey information
  between the file system VOP function and the VOP wrapper, but do not
  occupy an argument slot in the VOP_*() call itself.  These context fields
  are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
  back the resulting link count of the target vnode.  Return this in tmpfs,
  udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Sync with HEAD.

Sync with HEAD.

Add a sysctl hashstat collector for vcache.

Fix bogus fast path in vput.

If we can't discern whether we have an exclusive or shared lock, then
just unlock and don't play fast and loose with pretending that we
have an exclusive lock will work -- it won't.

If a vnode is marked with VI_EXECMAP then in all likelyhood it has pages.

Counter tweaks:

- Don't need to count anonpages+filepages any more; clean+unknown+dirty for
  each kind of page can be summed to get the totals.

- Track the number of free pages with a counter so that it's one less thing
  for the allocator to do, which opens up further options there.

- Remove cpu_count_sync_one().  It has no users and doesn't save a whole lot.
  For the cheap option, give cpu_count_sync() a boolean parameter indicating
  that a cached value is okay, and rate limit the updates for cached values
  to hz.

Make vcache_tryvget() lockless.  Reviewed by hannken@.

vrele_flush(): yield() every 100ms like we do it in vflush().

Sync with HEAD

Sync with HEAD

Take some pressure from vdrain lock:

- Use cv_signal() instead of cv_broadcast(), there is only one waiter.
- No need to signal if number of vnodes doesn't increase.
- Use kpause(1) instead of yield().

Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed().  Signature matches
FreeBSD.

hardclock_ticks -> getticks()

Merge changes from current as of 20200406

vrelel(): clear VV_MAPPED with the vnode still locked.

Merge the remaining changes from the ad-namecache branch, affecting namei()
and getcwd():

- push vnode locking back as far as possible.
- do most lookups directly in the namecache, avoiding vnode locks & refs.
- don't block new refs to vnodes across VOP_INACTIVE().
- get shared locks for VOP_LOOKUP() if the file system supports it.
- correct lock types for VOP_ACCESS() / VOP_GETATTR() in a few places.

Possible future enhancements:

- make the lookups lockless.
- support dotdot lookups by being lockless and inferring absence of chroot.
- maybe make it work for layered file systems.
- avoid vnode references at the root & cwd.

Process concurrent page faults on individual uvm_objects / vm_amaps in
parallel, where the relevant pages are already in-core.  Proposed on
tech-kern.

Temporarily disabled on MP architectures with __HAVE_UNLOCKED_PMAP until
adjustments are made to their pmaps.

Fix build failure.

Merge vfs_cache.c from the ad-namecache branch.  With this the namecache
index becomes per-directory (initially, a red-black tree).  The remaining
changes on the branch to namei()/getcwd() will be merged in the future.

Sync with head.

Tighten up the locking around vp->v_iflag a little more after the recent
split of vmobjlock & v_interlock.

Merge from ad-namecache:

- Have a stab at clustering the members of vnode_t and vnode_impl_t in a
  more cache-conscious way.  With that done, go back to adjusting v_usecount
  with atomics and keep vi_lock directly in vnode_impl_t (saves KVA).

- Allow VOP_LOCK(LK_NONE) for the benefit of VFS_VGET() and VFS_ROOT().
  Make sure LK_UPGRADE always comes with LK_NOWAIT.

- Make cwdinfo use mostly lockless.

Turns out there's no point adjusting v_holdcnt with atomics.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart.  v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap.  Others to follow later.

Sync with head.

Make cwdinfo use mostly lockless, and largely hide the details in vfs_cwd.c.

vnodes:

- Have own v_usecount again, don't share the uvm_object's refcount.
- Cluster the members of vnode_t and vnode_impl_t in a cache-concious way.
- Go back to having vi_lock directly in vnode_impl_t.
- Go back to having v_usecount adjusted with atomics.
- Start adjusting v_holdcnt with atomics, too.
- Put all the namecache stuff back into vnode_impl_t.

Do not clean up segvguard while holding v_interlock.c

vcache_reclaim(): purge namecache immediately after setting vnode to
VS_RECLAIMED.

#ifdef _KERNEL_OPT for previous

PAX_SEGVGUARD doesn't seem to work properly in testing for me, but at least
make it not cause problems:

- Cover it with exec_lock so the updates are not racy.
- Using fileassoc is silly.  Just hang a pointer off the vnode.

vrelel: don't change the vnode state to VS_BLOCKED for VOP_INACTIVE(), it's
not needed (at least not for the usual case).  Will revist before merge.

Sync with head.

vput(): don't drop the vnode lock, carry the hold over into vrelel() which
might need it anyway.

- options NAMECACHE_ENTER_REVERSE is no more.

- Partially sort the list of per-vnode namecache entries by using a TAILQ.
  Put the real name to the head, and put dot and dotdot to the tail so that
  cache_lookup_reverse() doesn't have to consider them.

Redo the namecache to focus on per-directory data structures, removing the
huge hashtable and nasty locking scheme.

Initially this uses rbtrees (because that's what's there).  The intent is
experiment with other data structures.

- Extend the per-CPU counters matt@ did to include all of the hot counters
  in UVM, excluding uvmexp.free, which needs special treatment and will be
  done with a separate commit.  Cuts system time for a build by 20-25% on
  a 48 CPU machine w/DIAGNOSTIC.

- Avoid 64-bit integer divide on every fault (for rnd_add_uint32).

Minor vnode locking changes:

- Stop using atomics to maniupulate v_usecount.  It was a mistake to begin
  with.  It doesn't work as intended unless the XLOCK bit is incorporated in
  v_usecount and we don't have that any more.  When I introduced this 10+
  years ago it was to reduce pressure on v_interlock but it doesn't do that,
  it just makes stuff disappear from lockstat output and introduces problems
  elsewhere.  We could do atomic usecounts on vnodes but there has to be a
  well thought out scheme.

- Resurrect LK_UPGRADE/LK_DOWNGRADE which will be needed to work effectively
  when there is increased use of shared locks on vnodes.

- Allocate the vnode lock using rw_obj_alloc() to reduce false sharing of
  struct vnode.

- Put all of the LRU lists into a single cache line, and do not requeue a
  vnode if it's already on the correct list and was requeued recently (less
  than a second ago).

Kernel build before and after:

119.63s real  1453.16s user  2742.57s system
115.29s real  1401.52s user  2690.94s system

Sync with HEAD

Attach "mnt_transinfo" to "dead_rootmount" so every mount has a
valid "mnt_transinfo" and remove now unneeded flag IMNT_HAS_TRANS.

Run fstrans_start()/fstrans_done() on dead_rootmount if FSTRANS_DEAD_ENABLED.
Should become the default for DIAGNOSTIC in the future.

Assign vnode to dead_rootmount before vcache_dealloc() releases it.

Now v_mount is never NULL.

Synch with HEAD

Add "void *extra" argument to vcache_new() so a file system may
pass more information about the file to create.

Welcome to 8.99.30

update from HEAD

Pull up following revision(s) (requested by hannken in ticket #309):
	sys/sys/vnode_impl.h: revision 1.17
	sys/kern/vfs_vnode.c: revision 1.99, 1.100

Change the VSTATE_ASSERT_UNLOCKED code by pushing the potential lock
handling into the backend and doing an optimistic (unlocked) check
first. Always taking the vnode interlock makes this assertion otherwise
very heavy for multi-processor machines.
-
Fix non-DIAGNOSTICS build by adjusting _vstate_assert here too.

Fix non-DIAGNOSTICS build by adjusting _vstate_assert here too.

Change the VSTATE_ASSERT_UNLOCKED code by pushing the potential lock
handling into the backend and doing an optimistic (unlocked) check
first. Always taking the vnode interlock makes this assertion otherwise
very heavy for multi-processor machines. Ride the kernel version bump.

Sync with HEAD

Pull up following revision(s) (requested by hannken in ticket #227):
	sys/sys/vnode_impl.h: revision 1.16
	sys/kern/vfs_vnode.c: revision 1.97
	sys/kern/vfs_vnode.c: revision 1.98
	sys/kern/vfs_mount.c: revision 1.67
	sys/miscfs/deadfs/dead_vfsops.c: revision 1.8
No need to cache anonymous device vnodes, they will never be looked up.
Set key to (dead_rootmount, 0, NULL) and add assertions.
--
Change forced unmount to revert open device vnodes to anonymous devices.

Change forced unmount to revert open device vnodes to anonymous devices.

No need to cache anonymous device vnodes, they will never be looked up.

Set key to (dead_rootmount, 0, NULL) and add assertions.

pullup the following revisions, requested by hannken in ticket #2:
	src/share/man/man9/fstrans.9			1.25
	src/sys/kern/vfs_mount.c			1.66
	src/sys/kern/vfs_subr.c				1.468
	src/sys/kern/vfs_trans.c			1.46
	src/sys/kern/vfs_vnode.c			1.94, 1.95, 1.96
	src/sys/kern/vnode_if.c				1.105, 1.106
	src/sys/kern/vnode_if.sh			1.65, 1.66
	src/sys/kern/vnode_if.src			1.76
	src/sys/miscfs/genfs/genfs_io.c			1.69
	src/sys/miscfs/genfs/genfs_vnops.c		1.196, 1.197
	src/sys/miscfs/genfs/layer_extern.h		1.40
	src/sys/miscfs/genfs/layer_vfsops.c		1.51
	src/sys/miscfs/genfs/layer_vnops.c		1.67
	src/sys/miscfs/nullfs/null_vnops.c		1.42
	src/sys/miscfs/overlay/overlay_vnops.c		1.24
	src/sys/miscfs/umapfs/umap_vnops.c		1.60
	src/sys/rump/include/rump/rumpvnode_if.h	1.29, 1.30
	src/sys/rump/librump/rumpkern/emul.c		1.182
	src/sys/rump/librump/rumpvfs/rumpvnode_if.c	1.29, 1.30
	src/sys/sys/fstrans.h				1.11
	src/sys/sys/vnode.h				1.278
	src/sys/sys/vnode_if.h				1.100, 1.101
	src/sys/sys/vnode_impl.h			1.14, 1.15
	src/sys/ufs/lfs/lfs_pages.c			1.12

Vnode state, lock and fstrans cleanup:
- Rename vnode state "VS_ACTIVE" to "VS_LOADED" and add synthetic
  state "VS_ACTIVE" to assert a loaded vnode with usecount > 0.

- Redo FSTRANS in vnode_if.c and use it for VOP_LOCK and VOP_UNLOCK.

- Cleanup the genfs lock operations.

- Make "struct vnode_impl" member "vi_lock" a krwlock_t again.

- Remove the lock type argument from fstrans_start and
  fstrans_start_nowait,
  remove now unused FSTRANS state "FSTRANS_SUSPENDING".

Operations fstrans_start() and fstrans_start_nowait() now always
use FSTRANS_SHARED as lock type so remove the lock type argument.

File system state FSTRANS_SUSPENDING is now unused so remove it.

Regen vnode_if files.

Ride 8.99.1 less than a hour ago.

Locking a layer vnode using the regular bypass routine is no longer
racy.  Undo the change from 2017-03-30 11:16:52, commitid eurqbzuGxGRlryLz
and make vi_lock a krwlock_t again.

A vnode is usually called "active", if it has an associated file system
node and a usecount greater zero.  Therefore rename state "VS_ACTIVE"
to "VS_LOADED" and add a new synthetic state "VS_ACTIVE" for VSTATE_ASSERT()
to assert an active vnode.

Add VSTATE_ASSERT_UNLOCKED() to be used with v_interlock unheld and
move the state assertion macros to sys/vnode_impl.h.

Restrict vgone() to suspended file systems only.

Welcome to 7.99.75, old file system modules would cause a diagnostic
assertion with new kernel.

Add a helper to propagate file system suspension for vrevoke().

Take care to retry suspension on interrupt as vrevoke must succeed.

Check VOP_INACTIVE contract with a judicious assert.

Clarify comment.

Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)

Suspend file system while revoking a vnode.  This way no operations run
on the mounted file system during revoke and all operations see
the state before or after the revoke.

Sync with HEAD

Sync with HEAD

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Add vfs_ref(mp) and vfs_rele(mp) to add or remove a reference to
struct mount.  Rename vfs_destroy(mp) to vfs_rele(mp) and replace
incrementing mp->mnt_refcnt with vfs_ref(mp).

Back out previous.

Breaks file systems for which VOP_UNLOCK doesn't work on a reclaimed
vnode.

The only case in tree right now is sys/fs/union -- most file systems
use genfs_unlock, which does work on a reclaimed vnode.

Maybe we can work around this -- and still enable VOP_RECLAIM's
callees to assert lock ownership -- by having VOP_RECLAIM unlock the
vnode instead.

Keep vnode locked during VOP_RECLAIM.

No bump because it wouldn't have been possible to acquire the lock in
VOP_RECLAIM anyway -- instant deadlock because vn_lock waits to
transition out of the RECLAIMING state first.  Benefit is that we can
now assert ownership of the lock in any operations called by
VOP_RECLAIM.

Discussed on tech-kern:

https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Simplify: eliminate a now-needless unlock/lock cycle.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Locking a layer vnode is racy as it may become reclaimed before
calling the operation on the lower vnode.

Replace vi_lock with a rw_obj and change layered file systems
to share the lock with the lower vnode.

Layered file systems now use genfs_lock()/_unlock/_islocked().

Welcome to 7.99.67

Change the operations vector before changing the mount.

Vnode operations enter the mount before using the vector.

Change vrelel() to defer the test for a reclaimed vnode until
we hold both the interlock and the vnode lock.

Add a common operation to deallocate a vnode in state LOADING.

Add flag VRELEL_FORCE_RELE to vrelel() to force release and
use it from vdrain_vrele() and vrele_flush() to prevent a
possible live lock from vrele_flush().

Change last users of FSTRANS_LAZY to FSTRANS_SHARED and change
genfs_suspendctl() to move from FSTRANS_NORMAL to FSTRANS_SUSPENDED
and vice versa.

Sync with HEAD

Change vrecycle() and vgone() to lock with LK_RETRY.  If this node is
a layerfs node the lower node(s) may already be reclaimed.

Bring back vrele_flush() to flush deferred vrele() o an suspended file system.

Make sure vcache_reclaim() will complete before file system suspension.

Sync with HEAD

Vrecycle() cannot wait for the vnode lock.  On a leaf file system this lock
will always succeed as we hold the last reference and prevent further
references.  On layered file systems waiting for the lock would open a can of
deadlocks as the lower vnodes may have other active references.

Move vnode member v_lock as vi_lock to vnode_impl.h.

Move vnode members v_dnclist and v_nclist as vi_dnclist and
vi_nclist to vnode_impl.h.

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Name all "vnode_impl_t" variables "vip".
No functional change.

Expand struct vcache to individual variables (vcache.* -> vcache_*).
No functional change.

Now that v_usecount tracks valid references add some "v_usecount == 1"
assertions.

Change vcache_*vget() to increment v_usecount on success only.
Increment v_holdcnt to prevent the vnode from disappearing while
vcache_vget() waits for a stable state.

Now v_usecount tracks the number of successfull references.

Rename vget() to vcache_vget() and vcache_tryvget() respectively and
move the definitions to sys/vnode_impl.h.

No functional change intended.

Welcome to 7.99.54

It is wrong to block the vnode during vcache_rekey.  The vnode may be looked
up using the old key until vcache_rekey_exit changes the key to the new one.

Add an assertion that the temporary key is different from the current one.

Restructure vdrain_vrele().  While it is not possible for another thread
to lock this vnodes v_interlock -> vdrain_lock another vnode sharing the
v_interlock may lock this order.
While here, restore fstrans_start_nowait arg to FSTRANS_LAZY.

Fixes a deadlock seen recently on some pbulk environments.

Change the freelists to lrulists, all vnodes are always on one
of the lists.  Speeds up namei on cached vnodes by ~3 percent.

Merge "vrele_thread" into "vdrain_thread" so we have one thread
working on the lrulists.  Adapt vfs_drainvnodes() to always wait
for a complete cycle of vdrain_thread().

Move vnode members "v_freelisthd" and "v_freelist" from "struct vnode"
to "struct vnode_impl" and rename to "vi_lrulisthd" and "vi_lrulist".

No functional change intended.

Welcome to 7.99.48

Remove the "target" argment from vfs_drainvnodes() as it is
always equal to "desiredvnodes" and move its definition
from sys/vnode.h to sys/vnode_impl.h.

Extend vfs_drainvnodes() to also wait for deferred vrele to flush
and replace the call to vrele_flush() with a call to vfs_drainvnodes().

Sync with HEAD

- Change vcache_reclaim() to always call VOP_INACTIVE() before VOP_RECLAIM().
  When called from vrecycle() or vgone() there is a window where the refcount
  is greater than zero and another thread could get and release a reference
  that would miss VOP_INACTIVE() as the refcount doesn't drop to zero.

  Adjust test fs/puffs/t_basic:  test VOP_INACTIVE count being greater zero.

- Make vrecycle() more robust by checking v_usecount first and preventing
  further references across vn_lock().  Fixes a deadlock where one thread
  starts unmount, second thread locks a directory and allocates a vnode
  and first thread tries to vrecycle() the directory.
  First thread holds vfs_busy and wants vnode, second thread holds vnode
  and wants vfs_busy.

- With these fixes in place change cleanvnode() to use vget()/vrecycle()
  to reclaim the vnode.

Sync with HEAD

Add a function to print the fields of a vnode including its implementation
and use it from vprint() and vfs_vnode_print().

Move vstate_name() to vfs_subr.c.

Split sys/vnode.h into sys/vnode.h and sys/vnode_impl.h
- Move _VFS_VNODE_PRIVATE protected operations into vnode_impl.h.
- Move struct vnode_impl definition and operations into vnode_impl.h.
- Include vnode_impl.h where we include vnode.h with _VFS_VNODE_PRIVATE defined.
- Get rid of _VFS_VNODE_PRIVATE.

Prepare the split of sys/vnode.h into sys/vnode.h and sys/vnode_impl.h
- Rename struct vcache_node to vnode_impl, start its fields with vi_.
- Rename enum vcache_state to vnode_state, start its elements with VS_.
- Rename macros VN_TO_VP and VP_TO_VN to VIMPL_TO_VNODE and VNODE_TO_VIMPL.
- Add typedef struct vnode_impl vnode_impl_t.

Sync with HEAD

Remove now obsolete operation vcache_remove().

Welcome to 7.99.36

Change vcache_reclaim() to remove vnode from vnode cache once the
vnode was reclaimed from the file system.

Rename vclean() to vcache_reclaim().
No functional change.

KNF. Remove extra spaces. No functional change.

Sync with HEAD

Use vnode state to replace VI_MARKER, VI_CHANGING, VI_XLOCK and VI_CLEAN.

Presented on tech-kern@

Add vnode state and supporting operations and diagnostics.

Presented on tech-kern@

Merge the vnode and its corresponding vcache_node into one
vcache_node structure.

Print the vcache_node part in vprint() and vfs_vnode_print().

Presented on tech-kern@

Keep the old vcache node on rekey.  Change its key and remove the
new vcache node now used as placeholder only.

Add VFS_VNODE_PRIVATE protected operations vnalloc_marker() to create,
vnfree_marker() to destroy and vnis_marker() to test for marker vnodes.

Make operations vnalloc() and vnfree() local to vfs_vnode.c.

Report what error was if nonzero with KASSERTMSG.

Pull up following revision(s) (requested by hannken in ticket #1070):
	sys/kern/vfs_vnode.c: revision 1.46 via patch
Take the vnode lock before the vnode is marked VI_CHANGING and fed
to vclean().  Prevents a deadlock with two null mounts on the same
physical mount where one thread tries to vclean() a layer node and
another thread tries to vget() a layer node pointing to the same
physical node.
Fixes PR kern/50375 layerfs (nullfs) locking problem leading to livelock

Pull up following revision(s) (requested by hannken in ticket #1070):
	sys/kern/vfs_vnode.c: revision 1.46 via patch
Take the vnode lock before the vnode is marked VI_CHANGING and fed
to vclean().  Prevents a deadlock with two null mounts on the same
physical mount where one thread tries to vclean() a layer node and
another thread tries to vget() a layer node pointing to the same
physical node.
Fixes PR kern/50375 layerfs (nullfs) locking problem leading to livelock

Sync with HEAD (as of 26th Dec)

Take the vnode lock before the vnode is marked VI_CHANGING and fed
to vclean().  Prevents a deadlock with two null mounts on the same
physical mount where one thread tries to vclean() a layer node and
another thread tries to vget() a layer node pointing to the same
physical node.

Fixes PR kern/50375 layerfs (nullfs) locking problem leading to livelock

Sync with HEAD

Operations getnewvnode() and ungetnewvnode() have been replaced with vcache.
- Remove now obsolete functions getnewvnode() and ungetnewvnode().
- Document vcache operations.

Welcome to 7.99.20

Use VFS_PROTOS() for deadfs.  Rename dead_mount to dead_rootmount.

Remove the test for mounted-on block devices in vclean() and
always close the vnode here.

A forced unmount of a file system holding a mounted-on
block device will make this mounted-on file system unusable.

Sync with HEAD

Cull unused vnode v_iflags: VI_LAYER, VI_LOCKSHARE.

Make vget always return vnode unlocked.

Convert callers who want locks to use vn_lock afterward.

Add extra argument so the compiler will report stragglers.

Sync with HEAD

Add new operation "vcache_new()" to allocate and initialise a new
vnode/fsnode pair:

int
vcache_new(struct mount *mp, struct vnode *dvp, struct vattr *vap,
    kauth_cred_t cred, struct vnode **vpp)

where dvp is the (referenced) directory where we want to create the
new node, vap passes va_type, va_mode and possibly va_rdev and cred
gives the credentials to setup uid/guid.

The node returned from vcache_new() is referenced, fully initialised
and has link count zero.

Welcome to NetBSD 7.99.7

Pull up following revision(s) (requested by hannken in ticket #150):
	sys/kern/vfs_vnode.c: revision 1.39
When creating a vnode with vcache_get() mark the vnode VI_CHANGING until
it is fully initialised.  It may be on the specnode list before it is
fully initialised and revoking it then would panic.
Should prevent the panic from PR kern/49171 (panic when closing a pty).

When creating a vnode with vcache_get() mark the vnode VI_CHANGING until
it is fully initialised.  It may be on the specnode list before it is
fully initialised and revoking it then would panic.

Should prevent the panic from PR kern/49171 (panic when closing a pty).

Don't next structure and enum definitions.
Don't use C++ keywords new, try, class, private, etc.

Rebase to HEAD as of a few days ago.

Rebase.

Add vcache operations to support key changes:

vcache_rekey_enter locks the old cache node and creates and locks the
  new cache node.  It is an error if the new cache node exists.

vcache_rekey_exit removes the old cache node and finalizes and
  unlocks the new cache node.

No objections on tech-kern@

Welcome to 6.99.46

adapt assertions to this branch

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Add a global vnode cache:

- vcache_get() retrieves a referenced and initialised vnode / fs node pair.
- vcache_remove() removes a vnode / fs node pair from the cache.

On cache miss vcache_get() calls new vfs operation vfs_loadvnode() to
initialise a vnode / fs node pair.  This call is guaranteed exclusive,
no other thread will try to load this vnode / fs node pair.

Convert ufs/ext2fs, ufs/ffs and ufs/mfs to use this interface.

Remove now unused ufs/ufs_ihash

Discussed on tech-kern.

Welcome to 6.99.41

- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add  vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Add fstrans_startnowait()/fstrans_done() to vrele_thread().

Current support for iterating over mnt_vnodelist is rudimentary.  Every
caller has to care about list and vnode mutexes, reference count being zero,
intermediate vnode states like VI_CLEAN, VI_XLOCK, VI_MARKER and so on.

Add an interface to iterate over a vnode list:

void vfs_vnode_iterator_init(struct mount *mp, struct vnode_iterator **marker)
void vfs_vnode_iterator_destroy(struct vnode_iterator *marker)
bool vfs_vnode_iterator_next(struct vnode_iterator *marker, struct vnode **vpp)

vfs_vnode_iterator_next() returns either "false / *vpp == NULL" when done
or "true / *vpp != NULL" to return the next referenced vnode from the list.

To make vrecycle() work in this environment change it to

bool vrecycle(struct vnode *vp)

where "vp" is a referenced vnode to be destroyed if this is the last reference.

Discussed on tech-kern.

Welcome to 6.99.34

The current implementation of vn_lock() is racy.  Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
  the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Currently dead vnodes still reside on the vnodelist of the file system
they have been removed from.

Create a "dead mount" that takes dead vnodes until they get freed.

Discussed on tech-kern.

When deciding to defer in vrelel():
- No need to always defer layer vnodes, if we get the vnode lock it
  is safe to inactivate.
- Always use VOP_LOCK(), it makes no sense to use vn_lock() here.
- No need to drop v_interlock for VOP_LOCK(... LK_NOWAIT).

Put back the vnode changes I backed out yesterday; they were not the problem.
I've tested them with 2 -j 20 builds on an 8 cpu box. It crashed reliably
with the pcu changes present before.

Revert recent vnode changes per PR/48411, I still have deadlocks with
build -j 20 on an 8 cpu machine.

Change vrelel() to mark the vnode as changing after it has aquired
the vnode lock but before it calls VOP_INACTIVE().

Should fix the race between layer_node_find() trying to vget(, LK_NOWAIT)
a locked vnode when vrelel() marked it as changing and wants its lock.

PR kern/48411 (repeatable SMP crashes in amd64-current)

Replace VI_INACTNOW and VI_INACTREDO with a new flag VI_CHANGING that gets
set while a vnode changes state from active to inactive or from active
or inactive to clean and protects "vclean(); vrelel()" and "vrelel()"
against "vget()".

Presented on tech-kern.

Make vclean static (ride 6.99.2).

DOCLOSE is no longer needed -- remove.

cleanvnode():
- VC_XLOCK/VC_MASK are not used anymore, remove.
- If we get a reference while cleaning, there is no need to retry as
  these reference and this vnode will disappear soon.
- Make sure we run inside a fstrans transaction to prevent deadlocks
  against vget().

vrecycle():
- don't even try to recycle a vnode currently cleaning.

Vnode API cleanup pass 1.

- Make these defines and functions private to vfs_vnode.c:

  VC_MASK, VC_LOCK, DOCLOSE, VI_IANCTREDO and VI_INACTNOW
  vclean() and vrelel()

- Remove the long time unused lwp argument from vrecycle().

- Remove vtryget(), it is responsible for ugly hacks and doesn't
  look that effective.

Presented on tech-kern.

Welcome to 6.99.25

Mark diagnostic-only variables

Remove VI_INACTPEND. Last consumer was vcount() which got removed 2010-01-08.

Reviewed by: David Holland <dholland@netbsd.org>

In description of a locking mess, add reference to the PR for the bug
the mess is supposed to help with.

resync with head

Make the spec_node table implementation private to spec_vnops.c.

To retrieve a spec_node, two new lookup functions (by device or by mount)
are implemented.  Both return a referenced vnode, for an opened block device
the opened vnode is returned so further diagnostic checks "vp == ... sd_bdevvp"
will not fire.  Otherwise any vnode matching the criteria gets returned.

No objections on tech-kern.

Welcome to 6.99.17

printflike maintenance.

sync with (a bit old) head

Pull up following revision(s) (requested by hannken in ticket #692):
	sys/kern/vfs_vnode.c: revision 1.17
	sys/kern/vfs_vnops.c: revision 1.186
Bring back Manuel Bouyers patch to resolve races between vget() and vrelel()
resulting in vget() returning dead vnodes.
It is impossible to resolve these races in vn_lock().
Needs pullup to NetBSD-6.

Pull up following revision(s) (requested by hannken in ticket #692):
	sys/kern/vfs_vnode.c: revision 1.17
	sys/kern/vfs_vnops.c: revision 1.186
Bring back Manuel Bouyers patch to resolve races between vget() and vrelel()
resulting in vget() returning dead vnodes.
It is impossible to resolve these races in vn_lock().
Needs pullup to NetBSD-6.

Resync to 2012-11-19 00:00:00 UTC

Bring back Manuel Bouyers patch to resolve races between vget() and vrelel()
resulting in vget() returning dead vnodes.
It is impossible to resolve these races in vn_lock().

Needs pullup to NetBSD-6.

sync with head

Update comment on vnode life-cycle a little.

sync with head

merge to -current.

Move the diagnostic check for a missing VOP_CLOSE() to the top of vrelel().
As long as we hold the vnode interlock there is no chance for this vnode
to gain new references.

Fixes false alarms observed by Thor Lancelot Simon and reported on tech-kern.

Ok: David Holland <dholland@netbsd.org>

remove uobj->memq

As vnalloc() always allocates with PR_WAITOK there is no longer the need
to test its result for NULL.

As getnewvnode() is prepared to wait for an allocation change vnalloc()
to always use PR_WAITOK.

No more 'WARNING: unable to allocate new vnode, retrying...' messages.

The path getnewvnode()->getcleanvnode()->vclean()->VOP_LOCK() will panic
if the vnode we want to clean is a layered vnode and the caller already
locked its lower vnode.

Change getnewvnode() to always allocate a fresh vnode and add a helper
thread (vdrain) to keep the number of allocated vnodes within desiredvnodes.

Rename getcleanvnode() to cleanvnode() and let it take a vnode from the
lists, clean and free it.

Reviewed by: David Holland <dholland@netbsd.org>

Should fix:
PR #19110 (nullfs mounts over NFS cause lock manager problems)
PR #34102 (ffs panic in NetBSD 3.0_STABLE)
PR #45115 (lock error panic when build.sh*3 and daily script is running)
PR #45355 (Reader/writer lock error:  rw_vector_enter: locking against myself)

rename vpanic() to vnpanic() and make it varyadic. While there, fix the
broken formats, always call panic() from vnpanic() and make all the calls
use vnpanic(). We only call vprint() on DIAGNOSTIC now.

Check for v_type before v_rdev because it is cheaper and safer.

Catchup with rmind-uvmplock merge.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

Sync with HEAD.

sync with head

- Amend getnewvnode(9) to take the lock for sharing, not a vnode.
- Update tmpfs to perform vnode and UVM object lock sharing correctly.

Fix vnode interlock sharing.

Implement sharing of vnode_t::v_interlock amongst vnodes:
- Lock is shared amongst UVM objects using uvm_obj_setlock() or getnewvnode().
- Adjust vnode cache to handle unsharing, add VI_LOCKSHARE flag for that.
- Use sharing in tmpfs and layerfs for underlying object.
- Simplify locking in ubc_fault().
- Sprinkle some asserts.

Discussed with ad@.

file vfs_vnode.c was added on branch jruoho-x86intr on 2011-06-06 09:09:41 +0000

Add some general description of vnode life-cycle.

Remove cache_purge(9) calls from reclamation routines in the file systems,
as vclean(9) performs it for us since Lite2 merge.

Sprinkle __cacheline_aligned and __read_mostly.

sync with head

file vfs_vnode.c was added on branch rmind-uvmplock on 2011-04-21 01:42:12 +0000

getcleanvnode: make static, add few comments, convert checks to asserts.

KNF, slightly improve few comments.

vfs_drainvnodes: drop lwp argument, remove variable name in prototype.

- Move vrele_list flush notify code into vrele_flush() routine.
- Make some structures static.

Split off parts of vfs_subr.c into vfs_vnode.c and vfs_mount.c modules.

No functional change.  Discussed on tech-kern@.