CVS log for src/sys/uvm/uvm_pager.c

Up to [cvs.NetBSD.org] / src / sys / uvm

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Fix !VMSWAP build:
Added __unused for few local variables, which are used in VMSWAP block only.
Adjust !VMSWAP uvm_swap_stats() definition to make it build with compat code.
Copied "int (*uvm_swap_stats50)(...)" definition from uvm_swap to uvm_swapstub
to avoid missing uvm_swap_stats50 reference on linking.

Fixes INSTALL_CPMBR1400, INSTALL_ZYXELKX evbmips kernel configs as a result.

Reviewed by simon and phone in IRC (thanks).

Pull up following revision(s) - all via patch -
(requested by riastradh in ticket #1317):

	sys/uvm/uvm_page.c: revision 1.248
	sys/uvm/uvm_anon.c: revision 1.80
	sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.40
	sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.41
	sys/rump/librump/rumpkern/vm.c: revision 1.191
	sys/uvm/uvm_pager.c: revision 1.130
	external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vnops.c: revision 1.71
	tests/rump/rumpkern/t_vm.c: revision 1.5
	tests/rump/rumpkern/t_vm.c: revision 1.6
	sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.39

Move the handling of PG_PAGEOUT from uvm_aio_aiodone_pages() to
uvm_page_unbusy() so that all callers of uvm_page_unbusy() don't need to
handle this flag separately.  Split out the pages part of uvm_aio_aiodone()
into uvm_aio_aiodone_pages() in rump just like in the real kernel.

In ZFS functions that can fail to copy data between the ARC and VM pages,
use uvm_aio_aiodone_pages() rather than uvm_page_unbusy() so that we can
handle these "I/O" errors.  Fixes PR 55702.

fix an incorrect assertion in the previous commit.

Handle PG_PAGEOUT in uvm_anon_release() too.

Commit the ZFS file that I forgot in this previous commit:

Move the handling of PG_PAGEOUT from uvm_aio_aiodone_pages() to
uvm_page_unbusy() so that all callers of uvm_page_unbusy() don't need to
handle this flag separately.  Split out the pages part of uvm_aio_aiodone()
into uvm_aio_aiodone_pages() in rump just like in the real kernel.

In ZFS functions that can fail to copy data between the ARC and VM pages,
use uvm_aio_aiodone_pages() rather than uvm_page_unbusy() so that we can
handle these "I/O" errors.  Fixes PR 55702.
update the rump copy of uvm_page_unbusy() to match the real version,
in particular handle PG_PAGEOUT.  fixes a few atf tests.
the busypage test is buggy, expect it to fail.

make rump's uvm_aio_aiodone_pages() look more like the kernel version.
fixes some more rumpy assertions.

for the busypage test, replace atf_tc_expect_fail() with atf_tc_skip()
because atf apparently has no way to expect a test program to crash.
fixes PR 55945.

Move the handling of PG_PAGEOUT from uvm_aio_aiodone_pages() to
uvm_page_unbusy() so that all callers of uvm_page_unbusy() don't need to
handle this flag separately.  Split out the pages part of uvm_aio_aiodone()
into uvm_aio_aiodone_pages() in rump just like in the real kernel.
In ZFS functions that can fail to copy data between the ARC and VM pages,
use uvm_aio_aiodone_pages() rather than uvm_page_unbusy() so that we can
handle these "I/O" errors.  Fixes PR 55702.

centralize calls from UVM to radixtree into a few functions.
in those functions, assert that the object lock is held in
the correct mode.

Consistently use UVMHIST(__func__)

Convert UVMHIST_{CALLED,LOG} into UVMHIST_CALLARGS

Trailing whitespace

use maximum-size fixed size array instead of variable-length array
in uvm_aio_aiodone() so that the stack usage can be determined and
checked in compile time; this is not called recursively not
particularly deep in call stack, so there is no need to save every
last drop of stack space here

Sync with HEAD

Sync with HEAD

uvm_aio_aiodone_pages(): only call uvm_pageout_done() if work was done for
the page daemon.

Mostly merge changes from HEAD upto 20200411

Merge changes from current as of 20200406

For single page I/O, use direct mapping if available.

Sync with head.

0x%#x --> %#x for non-external codes.
Also, stop mixing up 0x%x and %#x in single files as far as possible.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart.  v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap.  Others to follow later.

remove the aiodoned thread.  I originally added this to provide a thread context
for doing page cache iodone work, but since then biodone() has changed to
hand off all iodone work to a softint thread, so we no longer need the
special-purpose aiodoned thread.

Sync with head.

Merge from yamt-pagecache (after much testing):

- Reduce unnecessary page scan in putpages esp. when an object has a ton of
  pages cached but only a few of them are dirty.

- Reduce the number of pmap operations by tracking page dirtiness more
  precisely in uvm layer.

- Add and use wrapper functions that take and acquire page interlocks, and pairs
  of page interlocks.  Require that the page interlock be held over calls to
  uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state.  Rather than
  updating the global state synchronously, set an intended state on
  individual pages (active, inactive, enqueued, dequeued) while holding the
  page interlock.  After the interlock is released put the pages on a 128
  entry per-CPU queue for their state changes to be made real in batch.
  This results in in a ~400 fold decrease in contention on my test system.
  Proposed on tech-kern but modified to use the page interlock rather than
  atomics to synchronise as it's much easier to maintain that way, and
  cheaper.

Pull up following revision(s) (requested by ad in ticket #584):

	sys/uvm/uvm_pager.c: revision 1.118

PR kern/48044: panic: kernel diagnostic assertion "uvmexp.swpgonly + npages <= uvmexp.swpginuse" failed
swpgonly is updated asynchronously with regard to swap use.  We can't assert
this condition with confidence in the post-5.0 world, at least not without
broader changes.  swpgonly's ultimate use is of a heuristic nature so this
is no problem at all.

PR kern/48044: panic: kernel diagnostic assertion "uvmexp.swpgonly + npages <= uvmexp.swpginuse" failed

swpgonly is updated asynchronously with regard to swap use.  We can't assert
this condition with confidence in the post-5.0 world, at least not without
broader changes.  swpgonly's ultimate use is of a heuristic nature so this
is no problem at all.

- Rename VM_PGCOLOR_BUCKET() to VM_PGCOLOR().  I want to reuse "bucket" for
  something else soon and TBH it matches what this macro does better.

- Add inlines to set/get locator values in the unused lower bits of
  pg->phys_addr.  Begin by using it to cache the freelist index, because
  computing it is expensive and that shows up during profiling.  Discussed
  on tech-kern.

The uvmexp.pdpending change was incorrect - revert for now.

Adjust pdpending in uvm_pageout_start() and uvm_pageout_done() to avoid
the value going temporarily negative.

Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code.  Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour

Add missing #include <sys/atomic.h>

- Adjust uvmexp.swpgonly with atomics, and make uvm_swap_data_lock static.
- A bit more __cacheline_aligned on mutexes.

update from HEAD

Pull up following revision(s) (requested by pgoyette in ticket #335):
	share/man/man9/kernhist.9: 1.5-1.8
	sys/arch/acorn26/acorn26/pmap.c: 1.39
	sys/arch/arm/arm32/fault.c: 1.105 via patch
	sys/arch/arm/arm32/pmap.c: 1.350, 1.359
	sys/arch/arm/broadcom/bcm2835_bsc.c: 1.7
	sys/arch/arm/omap/if_cpsw.c: 1.20
	sys/arch/arm/omap/tiotg.c: 1.7
	sys/arch/evbarm/conf/RPI2_INSTALL: 1.3
	sys/dev/ic/sl811hs.c: 1.98
	sys/dev/usb/ehci.c: 1.256
	sys/dev/usb/if_axe.c: 1.83
	sys/dev/usb/motg.c: 1.18
	sys/dev/usb/ohci.c: 1.274
	sys/dev/usb/ucom.c: 1.119
	sys/dev/usb/uhci.c: 1.277
	sys/dev/usb/uhub.c: 1.137
	sys/dev/usb/umass.c: 1.160-1.162
	sys/dev/usb/umass_quirks.c: 1.100
	sys/dev/usb/umass_scsipi.c: 1.55
	sys/dev/usb/usb.c: 1.168
	sys/dev/usb/usb_mem.c: 1.70
	sys/dev/usb/usb_subr.c: 1.221
	sys/dev/usb/usbdi.c: 1.175
	sys/dev/usb/usbdi_util.c: 1.67-1.70
	sys/dev/usb/usbroothub.c: 1.3
	sys/dev/usb/xhci.c: 1.75
	sys/external/bsd/drm2/dist/drm/i915/i915_gem.c: 1.34
	sys/kern/kern_history.c: 1.15
	sys/kern/kern_xxx.c: 1.74
	sys/kern/vfs_bio.c: 1.275-1.276
	sys/miscfs/genfs/genfs_io.c: 1.71
	sys/sys/kernhist.h: 1.21
	sys/ufs/ffs/ffs_balloc.c: 1.63
	sys/ufs/lfs/lfs_vfsops.c: 1.361
	sys/ufs/lfs/ulfs_inode.c: 1.21
	sys/ufs/lfs/ulfs_vnops.c: 1.52
	sys/ufs/ufs/ufs_inode.c: 1.102
	sys/ufs/ufs/ufs_vnops.c: 1.239
	sys/uvm/pmap/pmap.c: 1.37-1.39
	sys/uvm/pmap/pmap_tlb.c: 1.22
	sys/uvm/uvm_amap.c: 1.108
	sys/uvm/uvm_anon.c: 1.64
	sys/uvm/uvm_aobj.c: 1.126
	sys/uvm/uvm_bio.c: 1.91
	sys/uvm/uvm_device.c: 1.66
	sys/uvm/uvm_fault.c: 1.201
	sys/uvm/uvm_km.c: 1.144
	sys/uvm/uvm_loan.c: 1.85
	sys/uvm/uvm_map.c: 1.353
	sys/uvm/uvm_page.c: 1.194
	sys/uvm/uvm_pager.c: 1.111
	sys/uvm/uvm_pdaemon.c: 1.109
	sys/uvm/uvm_swap.c: 1.175
	sys/uvm/uvm_vnode.c: 1.103
	usr.bin/vmstat/vmstat.c: 1.219
Reorder to test for null before null deref in debug code
--
Reorder to test for null before null deref in debug code
--
KNF
--
No need for '\n' in UVMHIST_LOG
--
normalise a BIOHIST log message
--
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
   the kernel and in the structures used for exporting the history data
   to userland via sysctl(9).  This avoids problems on some architectures
   where passing a 64-bit (or larger) value to printf(3) can cause it to
   process the value as multiple arguments.  (This can be particularly
   problematic when printf()'s format string is not a literal, since in
   that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
   include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
   updated.  Each format specifier now includes an explicit length
   modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
   updated to replace uses of "%p" with "%#jx", and the pointer
   arguments are now cast to (uintptr_t) before being subsequently cast
   to (uintmax_t).  This is needed to avoid compiler warnings about
   casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
   "%c" format strings replaced with numeric formats; several instances
   of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
   history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
   the -u option does not exist (previously, this condition was silently
   ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
   data exported via sysctl(9) and exits if they do not match the values
   with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
   requirements imposed on the format strings, along with several other
   minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
     uint64_t) for the history arguments.  But that would require another
     "rototill" of all the users in the future when we add support for an
     architecture that supports a larger size.  Also, the printf(3)
format
     specifiers for explicitly-sized values, such as "%"PRIu64, are much
     more verbose (and less aesthetically appealing, IMHO) than simply
     using "%ju".
[2] I've tried very hard to find "all [the] existing users of
kernhist(9)"
     but it is possible that I've missed some of them.  I would be glad
to
     update any stragglers that anyone identifies.
--
For some reason this single kernel seems to have outgrown its declared
size as a result of the kernhist(9) changes.  Bump the size.
XXX The amount of increase may be excessive - anyone with more detailed
XXX knowledge please feel free to further adjust the value
appropriately.
--
Misssed one cast of pointer --> uintptr_t in previous kernhist(9) commit
--
And yet another one.  :(
--
Use correct mark-up for NetBSD version.
--
More improvements in grammar and readability.
--
Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.
--
And replace an instance of "%p" conversion with "%#jx"
--
Whitespace fix. Give Bl tag table a width. Fix Xr.

Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
  the kernel and in the structures used for exporting the history data
  to userland via sysctl(9).  This avoids problems on some architectures
  where passing a 64-bit (or larger) value to printf(3) can cause it to
  process the value as multiple arguments.  (This can be particularly
  problematic when printf()'s format string is not a literal, since in
  that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
  include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
  updated.  Each format specifier now includes an explicit length
  modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
  updated to replace uses of "%p" with "%#jx", and the pointer
  arguments are now cast to (uintptr_t) before being subsequently cast
  to (uintmax_t).  This is needed to avoid compiler warnings about
  casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
  "%c" format strings replaced with numeric formats; several instances
  of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
  history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
  the -u option does not exist (previously, this condition was silently
  ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
  data exported via sysctl(9) and exits if they do not match the values
  with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
  requirements imposed on the format strings, along with several other
  minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
    uint64_t) for the history arguments.  But that would require another
    "rototill" of all the users in the future when we add support for an
    architecture that supports a larger size.  Also, the printf(3) format
    specifiers for explicitly-sized values, such as "%"PRIu64, are much
    more verbose (and less aesthetically appealing, IMHO) than simply
    using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
    but it is possible that I've missed some of them.  I would be glad to
    update any stragglers that anyone identifies.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

only check that npages fit, if we are going to add npages to swpgonly.

Mark diagnostic-only variables

The pagedaemon ca emit write as large as the underlying device's maxphys,
so emergva size needs to be MACHINE_MAXPHYS, MAXPHYS is not enough.

- fix integrity sync.
  putpages for integrity sync (fsync, msync with MS_SYNC, etc) should not
  skip pages being written back by other threads.

- adapt to radix tree tag api changes.

sync with head

merge to -current.

Add more KASSERTs (more! more! more!).
When returning page to the free pool, make sure to dequeue the pages before
hand or free page queue corruption will happen.

Major changes to uvm.
Support multiple collections (groups) of free pages and run the page
reclaimation algorithm on each group independently.

extending vmem(9) to be able to allocated resources for it's own needs.
simplifying uvm_map handling (no special kernel entries anymore no relocking)
make malloc(9) a thin wrapper around kmem(9)
(with private interface for interrupt safety reasons)

releng@ acknowledged

remove a redundant assertion

page cache related changes

- maintain object pages in radix tree rather than rb tree.
- reduce unnecessary page scan in putpages.  esp. when an object has a ton of
  pages cached but only a few of them are dirty.
- reduce the number of pmap operations by tracking page dirtiness more
  precisely in uvm layer.
- fix nfs commit range tracking.
- fix nfs write clustering.  XXX hack

comment

Correct pagermap emergva allocation.  From yamt@.

Tested by building i386 kernel with DTRACE defined which died 100%.

Reallocate emergency pager va when ncolors is increased. (modication of
patch from mrg).

Forward some UVM from matt-nb5-mips64.  Add UVM_KMF_COLORMATCH flag.
When uvm_map gets passed UVM_FLAG_COLORMATCH, the align argument contains
the color of the starting address to be allocated (0..colormask).
When uvm_km_alloc is passed UVM_KMF_COLORMATCH (which can only be used with
UVM_KMF_VAONLY), the align argument contain the color of the starting address
to be allocated.
Change uvm_pagermapin to use this.  When mapping user pages in the kernel,
if colormatch is used with the color of the starting user page then the kernel
mapping will be congruent with the existing user mappings.

make compile without VMSWAP.  no functional change.

uvm_aio_aiodone_pages: check disposed anon correctly.

Catchup with rmind-uvmplock merge.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

Sync with HEAD.

Restore $NetBSD$

Rework page free lists to be sorted by color first rather than free_list.
Kept per color PGFL_* counter in each page free list.
Minor cleanups.

sync with head

Make uvm_map recognize UVM_FLAG_COLORMATCH which tells uvm_map that the
'align' argument specifies the starting color of the KVA range to be returned.

When calling uvm_km_alloc with UVM_KMF_VAONLY, also specify the starting
color of the kva range returned (UMV_KMF_COLORMATCH) and pass those to
uvm_map.

In uvm_pglistalloc, make sure the pages being returned have sequentially
advancing colors (so they can be mapped in a contiguous address range).
Add a few missing UVM_FLAG_COLORMATCH flags to uvm_pagealloc calls.

Make the socket and pipe loan color-safe.

Make the mips pmap enforce strict page color (color(VA) == color(PA)).

Replace "malloc" in comments, remove unnecessary header inclusions.

sync with head

Sync with HEAD

udpate license clauses on my code to match the new-style BSD licenses.
based on diff that rmind@ sent me.

no functional change with this commit.

Pull up following revision(s) (requested by rmind in ticket #1421):
	sys/uvm/uvm_bio.c: revision 1.70
	sys/uvm/uvm_map.c: revision 1.292
	sys/uvm/uvm_pager.c: revision 1.98
	sys/uvm/uvm_fault.c: revision 1.175
	sys/uvm/uvm_bio.c: revision 1.69
ubc_fault: split-off code part handling a single page into ubc_fault_page().
Keep the lock around pmap_update() where required.  While fixing this
in ubc_fault(), rework logic to "remember" the last object of page and
reduce locking overhead, since in common case pages belong to one and
the same UVM object (but not always, therefore add a comment).
Unlocks before pmap_update(), on removal of mappings, might cause TLB
coherency issues, since on architectures like x86 and mips64 invalidation
IPIs are deferred to pmap_update().  Hence, VA space might be globally
visible before IPIs are sent or while they are still in-flight.
OK ad@.

Sync with HEAD.

sync with head.

sync with head

Keep the lock around pmap_update() where required.  While fixing this
in ubc_fault(), rework logic to "remember" the last object of page and
reduce locking overhead, since in common case pages belong to one and
the same UVM object (but not always, therefore add a comment).

Unlocks before pmap_update(), on removal of mappings, might cause TLB
coherency issues, since on architectures like x86 and mips64 invalidation
IPIs are deferred to pmap_update().  Hence, VA space might be globally
visible before IPIs are sent or while they are still in-flight.

OK ad@.

Reorganise UVM locking to protect P->V state and serialise pmap(9)
operations on the same page(s) by always locking their owner.  Hence
lock order: "vmpage"-lock -> pmap-lock.

Patch, proposed on tech-kern@, from Andrew Doran.

Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc().  It allows us to share the locks among UVM objects.

sync with head

Add a flags argument to pmap_kenter_pa(9).
Patch showed on tech-kern@ http://mail-index.netbsd.org/tech-kern/2009/11/04/msg006434.html
No objections.

sync with head.

kill uvm_aio_biodone1().  only user was lfs and that uses nestiobuf now.

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

g/c uvm_aiobuf_pool.

Sync with HEAD.

PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.

Sync with HEAD.

Sync with HEAD.

Update haad-dm branch to haad-dm-base2.

more <sys/buf.h> police

Sync with HEAD.

Set up uvmhist in uvm_aio_aiodone_pages().

Sync with HEAD.

sync with head.

sync with HEAD

sync with head.

uvm_swap_io: if pagedaemon, don't wait for iobuf.

Sync with HEAD.

sync with head

sync with HEAD

Sync with HEAD

Merge vmlocking2 to head.

Sync with HEAD.

sync with head

Pull the vmlocking changes into a new branch.

Sync with HEAD.

constify pagerops.

uvm_pager_init: use __arraycount.

Sync with HEAD

sync with HEAD

Sync with HEAD.

sync with head.

defparam PAGER_MAP_SIZE.

Sync with head.

Sync with HEAD.

sync with head.

Sync with HEAD.

Make bioops a pointer and point it to the softdeps struct in softdep
init.  Decouples "options SOFTDEP" from the main kernel and ffs code.

Sync with buffer cache locking changes. See buf.h/vfs_bio.c for details.
Some minor portions are incomplete and needs to be verified as a whole.

softdep_pageiodone: softdep_pageiodone() acquires bqueue_lock.

fix some races around pagedaemon and uvm_wait.  ok'ed by Andrew Doran.

softdep locking improvements. It hangs looping in flush_inodedep_deps(),
more work required.

- Back out the biodone() changes.
- Eliminate B_ERROR (from HEAD).

Sync with HEAD.

It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.

file uvm_pager.c was added on branch matt-mips64 on 2007-07-29 13:31:18 +0000

It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.

Merge unobtrusive locking changes from the vmlocking branch.

Sync with head.

Implementation of per-CPU work-queues support for workqueue(9) interface.
WQ_PERCPU flag for workqueue and additional argument for workqueue_enqueue()
to assign a CPU might be used. Notes:
 - For now, the list is used for workqueue_queue, which is non-optimal,
   and will be changed with array, where index would be CPU ID.
 - The data structures should be changed to be cache-friendly.

Reviewed by: <yamt>, <tech-kern>

Sync with head.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

- Increase the number of thread priorities from 128 to 256. How the space
  is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
  is provided, with hooks for fast-path MD code that can run the interrupt
  threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
  locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.

- Pass the error number and residual count to biodone(), and let it handle
  setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
  of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
  create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.

- Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.

- Replace more simple_locks, and fix up in a few places.
- Use condition variables.
- LOCK_ASSERT -> KASSERT.

Pull in the initial set of changes for the vmlocking branch.

- sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.

sync with head.

TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type.  A
future commit will replace use of TRUE and FALSE with true and false.

Sync with head.

sync with head.

merge yamt-splraiseipl branch.

	- finish implementing splraiseipl (and makeiplcookie).
	  http://mail-index.NetBSD.org/tech-kern/2006/07/01/0000.html
	- complete workqueue(9) and fix its ipl problem, which is reported
	  to cause audio skipping.
	- fix netbt (at least compilation problems) for some ports.
	- fix PR/33218.

Sync with head.

use workqueue for aiodoned.

merge yamt-pdpolicy branch.
	- separate page replacement policy from the rest of kernel
	- implement an alternative replacement policy

sync with head

sync with head.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

Sync with head.

oops - *really* sync to head this time.

Change previous to KASSERT per yamt's request.

Coverity CID 835: Check before dereferencing pg->uanon.

sync with head

uvm_pagermapin: nowait allocation for pagedaemon.

- change the way to account read-ahead stats.
- fix UVM_PQFLAGBITS.

separate page replacement policy from the rest of kernel.

sync with head.

remove the following options.  no objections on tech-kern@.

	UVM_PAGER_INLINE
	UVM_AMAP_INLINE
	UVM_PAGE_INLINE
	UVM_MAP_INLINE

sync with head.

- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

Sync with head.

sync with head.

read-ahead statistics.

Sync with HEAD. Here we go again...

wrap swap related code by #ifdef VMSWAP.  always #define VMSWAP for now.

revert "defflag VMSWAP" changes for now.
there seems to be far more people who don't want to edit
their kernel config files than i thought.

defflag VMSWAP.

Clean up the cpp macro used to say "we're compiling this specific C file".

Use ANSI function decls.

sync with -current

Sync with HEAD.

merge yamt-km branch.
- don't use managed mappings/backing objects for wired memory allocations.
  save some resources like pv_entry.  also fix (most of) PR/27030.
- simplify kernel memory management API.
- simplify pmap bootstrap of some ports.
- some related cleanups.

- don't use uvm_object or managed mappings for wired allocations.
  (eg. malloc(9))
- simplify uvm_km_* apis.

Sync with HEAD.

for in-kernel maps,
- allocate kva for vm_map_entry from the map itsself and
  remove the static limit, MAX_KMAPENT.
- keep merged entries for later splitting to fix allocate-to-free problem.
  PR/24039.

Sync with HEAD

Count obj pages freed by pagedaemon.

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Pull up revision 1.63 (requested by yamt in ticket #271):
fix a amap_wirerange deadlock problem by re-introducing
PG_RELEASED for anon pages.  PR/23171 from Christian Limpach.
for details, see discussion filed in the PR database.
uvm_anon_release: a new function to free anon-owned PG_RELEASED page.
uvm_anfree: we can't wait for the page here because the caller might hold
	amap lock.  instead, just mark the page as PG_RELEASED.
	who unbusy the page should check the PG_RELEASED.
uvm_aio_aiodone: uvm_anon_release() instead of uvm_page_unbusy()
	if appropriate.
uvmfault_anonget: check PG_RELEASED.

fix a amap_wirerange deadlock problem by re-introducing
PG_RELEASED for anon pages.  PR/23171 from Christian Limpach.
for details, see discussion filed in the PR database.

uvm_anon_release: a new function to free anon-owned PG_RELEASED page.
uvm_anfree: we can't wait for the page here because the caller might hold
	amap lock.  instead, just mark the page as PG_RELEASED.
	who unbusy the page should check the PG_RELEASED.
uvm_aio_aiodone: uvm_anon_release() instead of uvm_page_unbusy()
	if appropriate.
uvmfault_anonget: check PG_RELEASED.

Can't rely on side-effects in KASSERT expressions which was pointed out to
me by YAMAMOTO Takashi.

When retiring a swap device with marked bad blocks on it we should update
the `# swap page in use' and `# swap page only' counters.  However, at the
time of swap device removal we can no longer figure out how many of the
bad swap pages are actually also `swap only' pages.

So, on swap I/O errors arrange things to not include the bad swap pages in
the `swpgonly' counter as follows: uvm_swap_markbad() decrements `swpgonly'
by the number of bad pages, and the various VM object deallocation routines
do not decrement `swpgonly' for swap slots marked as SWSLOT_BAD.

Pull up revision 1.60 (requested by tls in ticket #1434):
Correct use of MAXBSIZE where MAXPHYS was intended.  This is a necessary
first step towards per-device MAXPHYS, and has the beneficial side effect
of allowing clustering to MAXPHYS even on systems that need to run with
a reduced MAXBSIZE to get more metadata buffers.

Correct use of MAXBSIZE where MAXPHYS was intended.  This is a necessary
first step towards per-device MAXPHYS, and has the beneficial side effect
of allowing clustering to MAXPHYS even on systems that need to run with
a reduced MAXBSIZE to get more metadata buffers.

Catch up to -current

Fix signed/unsigned comparison warnings.

Catch up to -current.

sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work

uao_find_swslot()'s second argument is in units of pages, not bytes.
spotted by Doug Donsbach.

pagedaemon_proc really should be a proc, not a LWP.

Curproc->curlwp renaming.

Change uses of "curproc->l_proc" back to "curproc", which is more like the
original use. Bare uses of "curproc" are now "curlwp".

"curproc" is now #defined in proc.h as ((curlwp) ? (curlwp)->l_proc) : NULL)
so that it is always safe to reference curproc (*de*referencing curproc
is another story, but that's always been true).

catch up with -current on kqueue branch

Catch up to -current.

When core dumping a process, don't dump maps backed up by the device pager.
(move the pagerops externs to uvm_object.h and out the C files).

- In genfs_putpages(), no need to restrict the cluster within the given
  region.
- In uvm_aio_aiodone(), remove assertions no longer true.

Convert swap_syscall_lock and uvm.swap_data_lock to adaptive mutexes,
and rename them apporpriately.

Sync kqueue branch with -current.

Catch up to -current.

fix locking for loaning.  in general we should be looking at the page's
uobject and uanon pointers rather than at the PQ_ANON flag to determine
which lock to hold, since PQ_ANON can be clear even when the anon's lock
is the one which we should hold (if the page was loaned from an object
and then freed by the object).

Catch up to -current.

Sync the thorpej-mips-cache branch with -current.

add RCSIDs, and in some cases, slightly cleanup #include order

several changes prompted by loaning problems:
 - fix the loaned case in uvm_pagefree().
 - redo uvmexp.swpgonly accounting to work with page loaning.
   add an assertion before each place we adjust uvmexp.swpgonly.
 - fix uvm_km_pgremove() to always free any swap space associated with
   the range being removed.
 - get rid of UVM_LOAN_WIRED flag.  instead, we just make sure that
   pages loaned to the kernel are never on the page queues.
   this allows us to assert that pages are not loaned and wired
   at the same time.
 - add yet more assertions.

Remove some variables that are set but never used.

Catch up to -current.

fix an uninitialized-variable problem in an error case.
pointed out by Simon Burge.

Catch up with -current.

Catch up to -current.

a whole bunch of changes to improve performance and robustness under load:

 - remove special treatment of pager_map mappings in pmaps.  this is
   required now, since I've removed the globals that expose the address range.
   pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
   no longer any need to special-case it.
 - eliminate struct uvm_vnode by moving its fields into struct vnode.
 - rewrite the pageout path.  the pager is now responsible for handling the
   high-level requests instead of only getting control after a bunch of work
   has already been done on its behalf.  this will allow us to UBCify LFS,
   which needs tighter control over its pages than other filesystems do.
   writing a page to disk no longer requires making it read-only, which
   allows us to write wired pages without causing all kinds of havoc.
 - use a new PG_PAGEOUT flag to indicate that a page should be freed
   on behalf of the pagedaemon when it's unlocked.  this flag is very similar
   to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
   pageout fails due to eg. an indirect-block buffer being locked.
   this allows us to remove the "version" field from struct vm_page,
   and together with shrinking "loan_count" from 32 bits to 16,
   struct vm_page is now 4 bytes smaller.
 - no longer use PG_RELEASED for swap-backed pages.  if the page is busy
   because it's being paged out, we can't release the swap slot to be
   reallocated until that write is complete, but unlike with vnodes we
   don't keep a count of in-progress writes so there's no good way to
   know when the write is done.  instead, when we need to free a busy
   swap-backed page, just sleep until we can get it busy ourselves.
 - implement a fast-path for extending writes which allows us to avoid
   zeroing new pages.  this substantially reduces cpu usage.
 - encapsulate the data used by the genfs code in a struct genfs_node,
   which must be the first element of the filesystem-specific vnode data
   for filesystems which use genfs_{get,put}pages().
 - eliminate many of the UVM pagerops, since they aren't needed anymore
   now that the pager "put" operation is a higher-level operation.
 - enhance the genfs code to allow NFS to use the genfs_{get,put}pages
   instead of a modified copy.
 - clean up struct vnode by removing all the fields that used to be used by
   the vfs_cluster.c code (which we don't use anymore with UBC).
 - remove kmem_object and mb_object since they were useless.
   instead of allocating pages to these objects, we now just allocate
   pages with no object.  such pages are mapped in the kernel until they
   are freed, so we can use the mapping to find the page to free it.
   this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.

Update the kqueue branch to HEAD.

Update pmap_update to now take the updated pmap as an argument.
This will allow improvements to the pmaps so that they can more easily defer expensive operations, eg tlb/cache flush, til the last possible moment.

Currently this is a no-op on most platforms, so they should see no difference.

Reviewed by Jason.

Catch up with -current.

clean up the transient error case in uvm_pager_put().

Catch up to -current.

replace vm_map{,_entry}_t with struct vm_map{,_entry} *.

replace {simple_,}lock{_data,}_t with struct {simple,}lock {,*}.

remove trailing whitespace.

Sprinkle pmap_update() calls after calls to:
- pmap_enter()
- pmap_remove()
- pmap_protect()
- pmap_kenter_pa()
- pmap_kremove()
as described in pmap(9).

These calls are relatively conservative.  It may be possible to
optimize these a little more.

Catch up with -current.

Sync with HEAD.

eliminate the KERN_* error codes in favor of the traditional E* codes.
the mapping is:

KERN_SUCCESS			0
KERN_INVALID_ADDRESS		EFAULT
KERN_PROTECTION_FAILURE		EACCES
KERN_NO_SPACE			ENOMEM
KERN_INVALID_ARGUMENT		EINVAL
KERN_FAILURE			various, mostly turn into KASSERTs
KERN_RESOURCE_SHORTAGE		ENOMEM
KERN_NOT_RECEIVER		<unused>
KERN_NO_ACCESS			<unused>
KERN_PAGES_LOCKED		<unused>

Sync with HEAD.

eliminate the VM_PAGER_* error codes in favor of the traditional E* codes.
the mapping is:

VM_PAGER_OK		        0
VM_PAGER_BAD		        <unused>
VM_PAGER_FAIL		        <unused>
VM_PAGER_PEND		        0 (see below)
VM_PAGER_ERROR		        EIO
VM_PAGER_AGAIN		        EAGAIN
VM_PAGER_UNLOCK		        EBUSY
VM_PAGER_REFAULT	        ERESTART

for async i/o requests, it used to be possible for the request to
be convert to sync, and the pager would return VM_PAGER_OK or VM_PAGER_PEND
to indicate whether the caller should perform post-i/o cleanup.
this is no longer allowed; pagers must now return 0 to indicate that
the async i/o was successfully started, and the caller never needs to
worry about doing the post-i/o cleanup.

in uvm_aio_aiodone(), don't mark the page(s) clean if the pageout
failed because we failed to acquire some resource needed to initiate
the pageout (such as failing to lock an indirect buffer) rather than
a hard i/o error.  in this case we just want to reactivate the page(s)
so that we'll try to write them again later.

while I'm here, clean up some DIAGNOSTIC code.

Sync with HEAD.

add a KASSERT(pp) in the uvm_pagermapin() loop.

Page scanner improvements, behavior is actually a bit more like
Mach VM's now.  Specific changes:
- Pages now need not have all of their mappings removed before being
  put on the inactive list.  They only need to have the "referenced"
  attribute cleared.  This makes putting pages onto the inactive list
  much more efficient.  In order to eliminate redundant clearings of
  "refrenced", callers of uvm_pagedeactivate() must now do this
  themselves.
- When checking the "modified" attribute for a page (for clearing
  PG_CLEAN), make sure to only do it if PG_CLEAN is currently set on
  the page (saves a potentially expensive pmap operation).
- When scanning the inactive list, if a page is referenced, reactivate
  it (this part was actually added in uvm_pdaemon.c,v 1.27).  This
  now works properly now that pages on the inactive list are allowed to
  have mappings.
- When scanning the inactive list and considering a page for freeing,
  remove all mappings, and then check the "modified" attribute if the
  page is marked PG_CLEAN.
- When scanning the active list, if the page was referenced since its
  last sweep by the scanner, don't deactivate it.  (This part was
  actually added in uvm_pdaemon.c,v 1.28.)

These changes greatly improve interactive performance during
moderate to high memory and I/O load.

Sync with HEAD (for UBC fixes).

in uvm_pagermapin(), for now, don't pass the flag to pmap_enter()
which presets the page modified bit if the page is already initialized.
we don't actually want to modify such pages.

Sync with HEAD.

make sure that pages are on an paging queue before unlocking them.

allow building without SOFTDEP by adding the pageiodone hook to bio_ops.

Initial integration of the Unified Buffer Cache project.

increase PAGER_MAP_SIZE to 16MB and move it to uvm_pager.h
since the alpha and mips pmaps use it.

Update thorpej_scsipi to -current as of a month ago

Add an align argument to uvm_map() and some callers of that
routine.  Works similarly fto pmap_prefer(), but allows callers
to specify a minimum power-of-two alignment of the region.
How we ever got along without this for so long is beyond me.

remove include of <vm/vm.h>

remove/move more mach vm header files:

	<vm/pglist.h> -> <uvm/uvm_pglist.h>
	<vm/vm_inherit.h> -> <uvm/uvm_inherit.h>
	<vm/vm_kern.h> -> into <uvm/uvm_extern.h>
	<vm/vm_object.h> -> nothing
	<vm/vm_pager.h> -> into <uvm/uvm_pager.h>

also includes a bunch of <vm/vm_page.h> include removals (due to redudancy
with <vm/vm.h>), and a scattering of other similar headers.

Remove VM_PROT_EXECUTE from the permissions used to map the page
for pager I/O -- it is not needed, and including it leads to
unnecessary I-cache flushes.

Tell uvm_pagermapin() the direction of the I/O so that it can map
with only the protection that it needs.

remove uvm_shareprot().  no longer needed since the demise of share maps.

Delete redundant decl of aobj_pager - it's in <uvm/uvm_aobj.h>.

Merge parts of chs-ubc2 into the trunk:
Add a new type voff_t (defined as a synonym for off_t) to describe offsets
into uvm objects, and update the appropriate interfaces to use it, the
most visible effect being the ability to mmap() file offsets beyond
the range of a vaddr_t.

Originally by Chuck Silvers; blame me for problems caused by merging this
into non-UBC.

add support for ``swapctl -d'' (removing swap space).
improve handling of i/o errors in swap space.

reviewed by:  Chuck Cranor

Pull up to last week's -current.

Sync with -current

Change the pmap_enter() API slightly; pmap_enter() now returns an error
value (KERN_SUCCESS or KERN_RESOURCE_SHORTAGE) indicating if it succeeded
or failed.  Change the `wired' and `access_type' arguments to a single
`flags' argument, which includes the access type, and flags:

	PMAP_WIRED	the old `wired' boolean
	PMAP_CANFAIL	pmap_enter() is allowed to fail

If PMAP_CANFAIL is not specified, the pmap should behave as it always
has in the face of a drastic resource shortage: fall over dead.

Change the fault handler to deal with failure (which indicates resource
shortage) by unlocking everything, waiting for the pagedaemon to free
more memory, then retrying the fault.

eliminate the PMAP_NEW option by making it required for all ports.
ports which previously had no support for PMAP_NEW now implement
the pmap_k* interfaces as wrappers around the non-k versions.

create a new type "voff_t" for uvm_object offsets
and define it to be "off_t".  also, remove pgo_asyncget().

Fix a merge error.

in uvm_pagermapout(), be sure to pmap_remove() the range we're unmapping.
rewrite uvm_aio_aiodone() to be mostly the same as uvm_pager_dropcluster()
but using the pages' PG_FAKE and the buf's B_READ to decide whether the
each page should have the mod/ref bits cleared.

Garbage collect thread_sleep()/thread_wakeup() left over from the old
Mach VM code.  Also nuke iprintf(), which was no longer used anywhere.

Add proclist locking where appropriate.

add uvm_errno2vmerror().
in uvm_aio_biodone1() use b_resid rather than b_bcount to track
how much of the nested i/o is left to be done.

Teeny bit of style policing.

remove uvm_aiobuf_pool.  remove aiop arg to uvm_pagermapin().
re-enable pageout clustering.
revamp the uvm_aio_*iodone() functions to use bufs instead of aiobufs.

Sync w/ -current.

merge everything from chs-ubc branch.

in uvm_mk_pcluster(), remove the vm_page-blkno-based clustering
(and just disable clustering totally for the moment).

in uvm_pager_put(), remove hack from before where all swap pageouts
were made synchronous, it's not needed anymore.

in uvm_pager_dropcluster(), also clear the PG_FAKE vm_page flag.
a page being dropcluster'd must always either contain valid data
or else be marked PG_RELEASE.

in uvm_aio_aiodone(), mark pages PG_RELEASE when we get a read error.

Change the vm_map's "entries_pageable" member to a r/o flags member, which
has PAGEABLE and INTRSAFE flags.  PAGEABLE now really means "pageable",
not "allocate vm_map_entry's from non-static pool", so update all map
creations to reflect that.  INTRSAFE maps are maps that are used in
interrupt context (e.g. kmem_map, mb_map), and thus use the static
map entry pool (XXX as does kernel_map, for now).  This will eventually
change now these maps are locked, as well.

In uvm_pagermapin(), pass VM_PROT_READ|VM_PROT_WRITE as access_type, to
ensure we don't take mod/ref emulation faults in an interrupt context
(e.g. during the i/o operation).  This is safe because:
	- For a pageout operation, the page is already known to be
	  modified, and the pagedaemon will pmap_clear_modify() after
	  the pageout has completed.
	- For a pagein operation, pagers must already pmap_clear_modify()
	  after the pagein operation is complete, because the i/o may have
	  been done with e.g. programmed i/o.
XXX It would be nice to know the i/o direction so that we can call
XXX pmap_enter() with only the protection and access_type necessary.

Remove a comment in uvm_pager_dropcluster() about PMAP_NEW and mod/ref
attributes for the page; it no longer applies, since we don't use
pmap_kenter_pgs() anymore.

Don't use pmap_kenter_pgs() for entering pager_map mappings.  The pages
are still owned by the object which is paging, and so the test for a kernel
object in uvm_unmap_remove() will cause pmap_remove() to be used instead
of pmap_kremove().

This was a MAJOR source of pmap_remove() vs pmap_kremove() inconsistency
(which caused the busted kernel pmap statistics, and a cause of much
locking hair on MP systems).

remove some useless code.

move uvmexp.paging handling back into pagedaemon code.

Add a new `access type' argument to pmap_enter().  This indicates what type of
memory access a mapping was caused by.  This is passed through from uvm_fault()
and udv_fault(), and in most other cases is 0.
The pmap module may use this to preset R/M information.  On MMUs which require
R/M emulation, the implementation may preset the bits and avoid taking another
fault.  On MMUs which keep R/M information in hardware, the implementation may
preset its cached bits to speed up the next call to pmap_is_modified() or
pmap_is_referenced().

remove now >1 year old pre-release message.

use a pool for vnode aio buffers, like the uvm_swap code does.
fix a precendence error in uvm_mk_pcluster() which prevented it
from doing anything useful (already in -current).
have uvm_mk_pcluster() check physical adjacency as well as logical.
bump uvmexp.paging in uvm_pager_put() instead of wherever it used to be.
(actually this last one is bogus, but I'll fix it shortly.)
in uvm_pager_dropcluster(), skip NULL entries in the array of pages
(needed since this function is now used by several more callers who
do not always have a full array).
add iodone handlers for vnode aio.

fix a precedence problem in uvm_mk_pcluster() which prevented
clustering of vnode pageouts.  this probably makes no difference
since most apps don't write via the pagecache anyway... yet.

initial snapshot.  lots left to do.

be consistent with locking of amaps and anons when freeing them.

shift by PAGE_SHIFT instead of multiplying or dividing by PAGE_SIZE.

remove unused share map code from UVM:
 - update calls to uvm_unmap_remove/uvm_unmap (mainonly boolean arg
	has been removed)
 - replace UVM_ET_ISMAP checks with UVM_ET_ISSUBMAP checks

Make sure the aobj_pager gets initialized!

Merge paddr_t changes into the main branch.

Split vm_offset_t and vm_size_t into paddr_t, psize_t, vaddr_t, and vsize_t.

Remove inclusions of syscall (and syscall argument) related header files;
we don't need them here.

fix released pg bugs detected by Chuck S.:
 - release the correct page (ppsp[lcv], not pg)
 - don't access the page's fields after we have released it
 - in the uvm_objct case: move on to the next page if we've released
[should have been merged in on 1998/02/12, but we somehow missed it]

KNF.

- add defopt's for UVM, UVMHIST and PMAP_NEW.
- remove unnecessary UVMHIST_DECL's.

Allow callers of uvm_km_suballoc() to specify where the base of the
submap _must_ begin, by adding a "fixed" boolean argument.

restore rcsids

RCS ID police.

initial import of the new virtual memory system, UVM, into -current.

UVM was written by chuck cranor <chuck@maria.wustl.edu>, with some
minor portions derived from the old Mach code.  i provided some help
getting swap and paging working, and other bug fixes/ideas.  chuck
silvers <chuq@chuq.com> also provided some other fixes.

this is the UVM kernel code portion.


this will be KNF'd shortly.  :-)

Initial revision