CVS log for src/sys/ufs/ufs/ufs_vfsops.c

Up to [cvs.NetBSD.org] / src / sys / ufs / ufs

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

ufs: Nix trailing whitespace and tidy up some other minor KNF.

There is no difference between a zero-sized and not yet
reclaimed directory vnode and a non-existent vnode.

Teach ufs_fhtovp() to treat zero-sized directories as stale.

PR kern/55211 (fs/vfs/t_vnops:nfs_dir_rmdirdotdot test fails)

Mostly merge changes from HEAD upto 20200411

Merge changes from current as of 20200406

Sync with head.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode.  Matches
FreeBSD.

Make mntvnode_lock per-mount, and address false sharing of struct mount.

Split the ufs code out of the ffs module and into its own module.

Adapt chfs and ext2fs modules accordingly.

Sync with HEAD

Sync with HEAD, resolve a few conflicts

Remove unused mbuf.h includes.

update from HEAD

remove rebase merge artifacts

Sync with HEAD

Sync with HEAD

Sync with HEAD

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Sync with HEAD

Change ffs to use vcache_new:
- Change ffs_valloc to return an inode number.
- Remove now obsolete UFS operations UFS_VALLOC and UFS_VFREE.
- Make ufs_makeinode private to ufs_vnops.c and pass vattr instead of mode.

Rebase to HEAD as of a few days ago.

Rebase.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Add a global vnode cache:

- vcache_get() retrieves a referenced and initialised vnode / fs node pair.
- vcache_remove() removes a vnode / fs node pair from the cache.

On cache miss vcache_get() calls new vfs operation vfs_loadvnode() to
initialise a vnode / fs node pair.  This call is guaranteed exclusive,
no other thread will try to load this vnode / fs node pair.

Convert ufs/ext2fs, ufs/ffs and ufs/mfs to use this interface.

Remove now unused ufs/ufs_ihash

Discussed on tech-kern.

Welcome to 6.99.41

resync with head

Add an accessor -- ufs_maxphys() -- to check the maximum transfer size
for a given UFS mountpoint, and move the code from mount that finds
the underlying disk and resets the mountpoint max transfer size into a
utility function, ufs_update_maxphys().

Add a global serial number that counts disk property changes to which
filesystems are meant to accomodate themselves.  Make ufs_maxphys()
check it.  This is a sort of flag-polling interface that avoids callbacks
into the filesystem code, but will require freezing filesystems and
draining in-flight transactions before a decrease in size that is
mandatory (like attaching a disk with a smaller maximum transfer size
as a spare in a RAIDframe set), rather than "advisory", like finding
out set geometry from a RAID controller long after boot and deciding
a smaller transfer size would be optimal, can be signalled.  Still, the
"advisory" case is the common one so this is progress.

Make a bit of an example of RAIDframe by making it bump this new
serial number when disks are added to the subsystem.  I will attack
one of the hardware RAID drivers (probably arcmsr) next.

sync with head

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
   ROOTINO -> UFS_ROOTINO
   WINO -> UFS_WINO
   NXADDR -> UFS_NXADDR
   NDADDR -> UFS_NDADDR
   NIADDR -> UFS_NIADDR
   MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
   MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

sync with head

sync to latest -current.

Assert that we can a valid inode when looking up a file handle.

merge to -current.

Change the syscall API for quotas over to the new non-proplib one.

   - struct vfs_quotactl_args -> struct quotactl_args
   - add sys/stdint.h to sys/quotactl.h for clean userland build
   - install sys/quotactl.h in /usr/include
   - update set lists for same
   - add new marshalling code in libquota
   - add new unmarshalling code in vfs_syscalls.c
   - discard proplib interpreter code in vfs_quotactl.c
   - add dispatching code for the 14 quotactl ops in vfs_quotactl.c
   - mark the proplib quotactl syscall obsolete
   - add a new syscall number for the new quotactl syscall
   - change the name of the syscall to __quotactl()
   - remove the decl of the old quotactl from quota/quotaprop.h
   - add a decl of the new quotactl to sys/quotactl.h
   - update the libc build
   - update ktruss
   - remove proplib marshalling code from libquota
   - update copy of syscall table in gdb ppc sources
   - hack rumphijack to accomodate new quotactl name (as I recall,
     pooka wanted such a name change to simplify something, but I
     don't really see what/how)

This change appears to require a kernel version bump for rumpish
reasons.

Fix errors in !defined(QUOTA) && !defined(QUOTA2) case.

Remove references to <quota/quotaprop.h> in src/sys/ufs.
The remaining references in the kernel are in vfs_quotactl.c, the
compat_50 code for the old quotactl (to be fixed up), and the
code compiled from src/common/lib/libquota.

Remove the extra op argument to VFS_QUOTACTL() - the op is now stored
purely in the args structure.

This change requires a kernel version bump.

Introduce struct vfs_quotactl_args. Use it.

This change uglifies vfs_quotactl some in order to make room for
moving operation-specific but FS-independent logic out of ufs_quota.c.

Note: this change requires a kernel version bump.

Move the proplib-based quota command dispatching (that is, the code
that knows the magic string names for the allowed actions) out of
UFS-specific code and to fs-independent code.

This introduces QUOTACTL_* operation codes and changes the signature
of VFS_QUOTACTL() again for compile safety.

Note: this change requires a kernel version bump.

Move the code for iterating over the multiple RPC calls in a quota
proplib XML packet to vfs_quotactl.c out of sys/ufs/ufs.

Add a dummy extra arg to VFS_QUOTACTL for compile safety.

Note: this change requires a kernel version bump.

converting readdir in ffs ext2fs from malloc(9) to kmem(9)
while there allocate ufs mount structs from kmem(9) too
preceding kmem-vmem-pool-patch

releng@ acknowledged

Sync with HEAD.

sync with head

Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.

Fix build without quotas

Snapshot of work in progress on a modernised disk quota system:
- new quotactl syscall (versionned for backward compat), which takes
  as parameter a path to a mount point, and a prop_dictionary
  (in plistref format) describing commands and arguments.
  For each command, status and data are returned as a prop_dictionary.
  quota commands features will be added to take advantage of this,
  exporting quota data or getting quota commands as plists.

- new on disk-format storage (all 64bit wide), integrated to metadata for
  ffs (and playing nicely with wapbl).
  Quotas are enabled on a ffs filesystem via superblock flags.
  tunefs(8) can enable or disable quotas.
  On a quota-enabled filesystem, fsck_ffs(8) will track per-uid/gid
  block and inode usages, and will check and update quotas in Pass 6.
  quota usage and limits are stored in unliked files (one for users,
  one for groups)l fsck_ffs(8) will create the files if needed, or
  free them if needed. This means that after enabling or disabling
  quotas on a filesystem; a fsck_ffs(8) run is required.
  quotacheck(8) is not needed any more, on a unclean shutdown
  fsck or journal replay will take care of fixing quotas.
  newfs(8) can create a ready-to-mount quota-enabled filesystem
  (superblock flags are set and quota inodes are created).
  Other new features or semantic changes:
  - default quota datas, applied to users or groups which don't already
    have a quota entry
  - per-user/group grace time (instead of a filesystem global one)
  - 0 really means "nothing allowed at all", not "no limit".
    If you want "no limit", set the limit to UQUAD_MAX (tools will
    understand "unlimited" and "-")

  A quota file is structured as follow:
  it starts with a header, containing a few per-filesystem values,
  and the default quota limits.
  Quota entries are linked together as a simple list, each entry has a
  pointer (as an offset withing the file) to the next.
  The header has a pointer to a list of free quota entries, and
  a hash table of in-use entries. The size of the hash table depends
  on the filesystem block size (header+hash table should fit in the
  first block). The file is not sparse and is a multiple of
  filesystem block size (when the free quota entry list is empty a new
  filesystem block is allocated). quota entries to not cross
  filesystem block boundaries.

  In memory, the kernel keeps a cache of recently used quota entries
  as a reference to the block number, and offset withing the block.
  The quota entry itself is keept in the buf cache.

fsck_ffs(8), tunefs(8) and newfs(8) supports are completed (with
related atf tests :)
The kernel can update disk usage and report it via quotactl(2).

Todo: enforce quotas limits (limits are not checked by kernel yet)
      update repquota, edquota and rpc.rquotad to the new world
      implement compat_50_quotactl ioctl.
      update quotactl(2) man page

fsck_ffs required fixes so that allocating new blocks or inodes will
properly update the superblock and cg sumaries. This was not an issue up
to now because superblock and cg sumaries check happened last, but now
allocations or frees can happen in pass 6.

sync with head

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

Introduce several actions/requests for authorizing file-system related
operations, specifically quota and block allocation from reserved space.

Modify ufs_quotactl() to accomodate passing "mp" earlier by vfs_busy()ing
it a little bit higher.

Mailing list reference:

	http://mail-index.netbsd.org/tech-kern/2009/04/26/msg004936.html

Note that the umapfs request mentioned in this thread was NOT added as
there is still on-going discussion regarding the proper implementation.

Sync with HEAD.

sync with head.

sync with head.

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
  sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
  and is only ever write locked in dounmount(). A write hold can't be taken
  on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
  example when going r/o -> r/w, and is only present to serialize updates.
  In order to take this lock, a read hold must first be taken on
  mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

sync with HEAD

Sync with HEAD.

sync with head.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
  mount takes a reference, and in turn the mount takes a reference to the
  vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
  locking inherited from 4.4BSD with a recursable rwlock.

sync with head

sync with HEAD

Sync with HEAD

Use pool_cache.

Sync with HEAD.

sync with head

Sync with HEAD. amd64 Xen support needs testing.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start.  In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

sync with head.

Sync with head.

Sync with head.

Using POOL_INIT here makes no sense, since file systems always have
an init method.  So get rid of it and #ifdef _LKM and just always
init in the init method.  Give malloc types the same treatment.
Makes file systems nicer to work with in linksetless environments
and fixes a few LKM discrepancies.

sync with head.

Sync with head.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

sync with head.

Sync with head.

Consistent usage of KAUTH_GENERIC_ISSUSER.

sync with head.

sync with head.

Sync with head.

__unused removal on arguments; approved by core.

sync with head

ufs_quotactl(): consume the arguments even if QUOTAS is not defined.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

sync with head

sync with head

Use the LWP cached credentials where sane.

sync with head.

Sync with head.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

integrate kauth.

- Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
  that need it.

Approved by core.

generic_authorize() -> kauth_authorize_generic().

Adapt to kernel authorization KPI.

merge ktrace-lwp.

Sync with HEAD. Here we go again...

Apply the NFS exports list rototill patch:

- Remove all NFS related stuff from file system specific code.
- Drop the vfs_checkexp hook and generalize it in the new nfs_check_export
  function, thus removing redundancy from all file systems.
- Move all NFS export-related stuff from kern/vfs_subr.c to the new
  file sys/nfs/nfs_export.c.  The former was becoming large and its code
  is always compiled, regardless of the build options.  Using the latter,
  the code is only compiled in when NFSSERVER is enabled.  While doing this,
  also make some functions in nfs_subs.c conditional to NFSSERVER.
- Add a new command in nfssvc(2), called NFSSVC_SETEXPORTSLIST, that takes a
  path and a set of export entries.  At the moment it can only clear the
  exports list or append entries, one by one, but it is done in a way that
  allows setting the whole set of entries atomically in the future (see the
  comment in mountd_set_exports_list or in doc/TODO).
- Change mountd(8) to use the nfssvc(2) system call instead of mount(2) so
  that it becomes file system agnostic.  In fact, all this whole thing was
  done to remove a 'XXX' block from this utility!
- Change the mount*, newfs and fsck* userland utilities to not deal with NFS
  exports initialization; done internally by the kernel when initializing
  the NFS support for each file system.
- Implement an interface for VFS (called VFS hooks) so that several kernel
  subsystems can run arbitrary code upon receipt of specific VFS events.
  At the moment, this only provides support for unmount and is used to
  destroy NFS exports lists from the file systems being unmounted, though it
  has room for extension.

Thanks go to yamt@, chs@, thorpej@, wrstuden@ and others for their comments
and advice in the development of this patch.

Defflag UFS_DIRHASH.

- Use ANSI function decls.
- Sprinkle some static.

sync with -current

Sync with HEAD.

Bring in Ian Dowse's Dirhash from FreeBSD. Hash tables of
directories are created on the fly and used to increase
performance by circumventing ufs_lookup's linear search.

Dirhash is enabled by the UFS_DIRHASH option, but not
by default.

Sync with HEAD.

use #if defined(_KERNEL_OPT) around opt includes
fix arg to pool_init() when _LKM is defined

Remove the struct lwp * arguments from qsync and ufs_checkpath that are
no longer (read: were never) required.

Fix the sync with head I botched.

Sync with HEAD.

Undo part of the ktrace/lwp changes.  In particular:
* Remove the "lwp *" argument that was added to vget().  Turns out
  that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
  and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
  above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
  just to appease the above.

Sync with HEAD

Use a pool for struct direct instead of kernel stack.
Reduces the kernel stack usage by 264 bytes.

Approved by: Jason R. Thorpe <thorpej@netbsd.org>

First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

	- removed casts from unionfs, deadfs and fdesc
	  (there are more to hunt down still)
	- changed vfs_quotactl args argumet from caddr_t to void *
	- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

Apply the aborted ktrace-lwp changes to a specific branch.  This is just for
others to review, I'm concerned that patch fuziness may have resulted in some
errant code being generated but I'll look at that later by comparing the diff
from the base to the branch with the file I attempt to apply to it.  This will,
at the very least, put the changes in a better context for others to review
them and attempt to tinker with removing passing of 'struct lwp' through
the kernel.

Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes.  In particular:
* Remove the "lwp *" argument that was added to vget().  Turns out
  that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
  and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
  above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
  just to appease the above.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records.  The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Add support for UFS2. UFS2 is an enhanced FFS, adding support for
64 bit block pointers, extended attribute storage, and a few
other things.

This commit does not yet include the code to manipulate the extended
storage (for e.g. ACLs), this will be done later.

Originally written by Kirk McKusick and Network Associates Laboratories for
FreeBSD.

Merge pcred and ucred, and poolify.  TBD: check backward compatibility
and factor-out some higher-level functionality.

Sync kqueue branch with -current.

Catch up to -current.

Sync the thorpej-mips-cache branch with -current.

add RCSID

Catch up with -current.

Catch up to -current.

add a new VFS op, vfs_reinit, which is called when desiredvnodes is
adjusted via sysctl.  file systems that have hash tables which are
sized based on the value of this variable now resize those hash tables
using the new value.  the max number of FFS softdeps is also recalculated.

convert various file systems to use the <sys/queue.h> macros for
their hash tables.

Update thorpej_scsipi to -current as of a month ago

Remove register declarations.

Change ufs_init() to keep global count of how many times it was called.
Resources are initialized still just once (on first call).

Add ufs_done(), which takes care of freeing all resources allocated in
ufs_init(). The resources are freed only when last user of the code exits.

Modify vfsops to seperate vfs_fhtovp() into two routines. vfs_fhtovp() now
only handles the file handle to vnode conversion, and a new call,
vfs_checkexp(), performs the export verification.

minr KNF nit ..

Use the newly-defined opt_quota.h.

Merge with Lite2 + local changes

Import 4.4BSD-Lite2

Import 4.4BSD-Lite for reference

Add support for ext2fs, this needed a few modifications to ufs/ufs/inode.h:
- added an "union inode_ext" to struct inode, for the per-fs extentions.
  For now only ext2fs uses it.
- i_din is now an union:
	union {
		struct  dinode ffs_din; /* 128 bytes of the on-disk dinode. */
		struct ext2fs_dinode e2fs_din; /* 128 bytes of the on-disk dinode. */
	} i_din
  Added a lot of #define i_ffs_* and i_e2fs_* to access the fields.
- Added two macros: FFS_ITIMES and EXT2FS_ITIMES. ITIMES calls the rigth
  macro, depending on the time of the inode. ITIMES is used where necessary,
  FFS_ITIMES and EXT2FS_ITIMES in other places.

ufs prototype changes

from Mike Karels:
allow Q_SYNC regardless of "target" uid, we allow it with -1;
fix bug that caused all ops to refer to user quotas, not group.
[finally had a chance to check this!]

New RCS ID's, take two.  they're more aesthecially pleasant, and use 'NetBSD'

Update to 4.4-Lite fs code, with local changes.