CVS log for src/sys/uvm/uvm_map.c

Up to [cvs.NetBSD.org] / src / sys / uvm

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Pull up following revision(s) (requested by riastradh in ticket #804):

	sys/uvm/uvm_map.c: revision 1.423 (patch)
	sys/uvm/uvm_map.c: revision 1.425 (patch)

uvm_map(9): Make sure search in the nearest gap is monotonic.

The algorithm, on a hint clamped to the VM bounds, works as follows
(assuming topdown VM):

1. Make sure the hint is aligned, by subtracting the remainderin
   uvm_map_align_va.
2. If the hint is equal to the VM max, try the first free gap.
3. If the hint is not equal to the VM max, but is already in use, try
   the next gap _below_ the entry covering hint.
4. If the hint is not equal to the VM max and is not already in use,
   try gap between the entry below hint and the next entry after it,
   above hint.

In the last case, `entry' is the one below hint, and `entry->next' is
the one above it.  We would take
        entry->next->start - length
as the next candidate hint.

However, this algorithm is supposed to be a monotonic search through
the address space, and we might wind up with something like:

[0x7defb000,0x7defc000)         entry above hint (entry->next)
0x77895000                      hint
[0x77894000,0x77895000)         entry below hint (entry)

In this case, if length=0x1000, we would take
        0x7defb000 - 0x1000 = 0x7defa000
as the next candidate hint, but this violates monotonicity of the
search.

Instead, take the _smallest_ of orig_hint or entry->next->start -
length, to avoid violating monotonicity, so hint <= orig_hint.

I didn't commit this change before because it didn't seem to fix all
the manifestations of the problem, but we have more diagnostics now
so maybe we will find there is a _different_ violation of the same
invariants once this is committed -- and I'm pretty sure this change
is necessary to guarantee monotonicity in some cases (but I'm still
not sure why we're only hitting the problem on sh3).

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed
uvm_map(9): Apply the same orig_hint clamp again to the same entry.

Previous change dealt with case like length=0x1000 and:

[0x7defb000,0x7defc000)         entry above orig_hint (entry->next)
0x77895000                      orig_hint
[0x77894000,0x77895000)         entry below orig_hint (entry)

by changing
        entry->next->start - length
to
        MIN(orig_hint, entry->next->start - length)

in order to enforce monotonicity of search.

In this case, if the tree search for a gap has failed, we retry with
a list search with exactly the same orig_hint and entry -- nothing
has changed them (only hint and tmp).  So apply the same clamping.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

Pull up following revision(s) (requested by riastradh in ticket #804):

	sys/uvm/uvm_map.c: revision 1.423
	sys/uvm/uvm_map.c: revision 1.425

uvm_map(9): Make sure search in the nearest gap is monotonic.

The algorithm, on a hint clamped to the VM bounds, works as follows
(assuming topdown VM):

1. Make sure the hint is aligned, by subtracting the remainderin
   uvm_map_align_va.
2. If the hint is equal to the VM max, try the first free gap.
3. If the hint is not equal to the VM max, but is already in use, try
   the next gap _below_ the entry covering hint.
4. If the hint is not equal to the VM max and is not already in use,
   try gap between the entry below hint and the next entry after it,
   above hint.

In the last case, `entry' is the one below hint, and `entry->next' is
the one above it.  We would take
        entry->next->start - length
as the next candidate hint.

However, this algorithm is supposed to be a monotonic search through
the address space, and we might wind up with something like:

[0x7defb000,0x7defc000)         entry above hint (entry->next)
0x77895000                      hint
[0x77894000,0x77895000)         entry below hint (entry)

In this case, if length=0x1000, we would take
        0x7defb000 - 0x1000 = 0x7defa000
as the next candidate hint, but this violates monotonicity of the
search.

Instead, take the _smallest_ of orig_hint or entry->next->start -
length, to avoid violating monotonicity, so hint <= orig_hint.

I didn't commit this change before because it didn't seem to fix all
the manifestations of the problem, but we have more diagnostics now
so maybe we will find there is a _different_ violation of the same
invariants once this is committed -- and I'm pretty sure this change
is necessary to guarantee monotonicity in some cases (but I'm still
not sure why we're only hitting the problem on sh3).

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed
uvm_map(9): Apply the same orig_hint clamp again to the same entry.

Previous change dealt with case like length=0x1000 and:

[0x7defb000,0x7defc000)         entry above orig_hint (entry->next)
0x77895000                      orig_hint
[0x77894000,0x77895000)         entry below orig_hint (entry)

by changing
        entry->next->start - length
to
        MIN(orig_hint, entry->next->start - length)

in order to enforce monotonicity of search.

In this case, if the tree search for a gap has failed, we retry with
a list search with exactly the same orig_hint and entry -- nothing
has changed them (only hint and tmp).  So apply the same clamping.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

Pull up following revision(s) (requested by rin in ticket #783):

	sys/uvm/uvm_map.c: revision 1.407
	sys/uvm/uvm_map.c: revision 1.412
	sys/uvm/uvm_map.c: revision 1.413

uvm_findspace(): For sh3, convert a KASSERTMSG(9) into printf(9)
XXX

Work around for PR kern/51254 until it gets fixed.

With this change, landisk survives full ATF with DIAGNOSTIC enabled.
uvm_map.c: Fix kassertmsg/printf newline mismatch in PR 51254 note.
uvm_findspace_invariants: don't repeat the message three times

The topdown and bottomup messages were exactly the same and sh3 printf
hack added the third copy.  Restructure the code so that there's only
one message and make the message more obvious - the topdown condition
in the assertions was confusing b/c it's inverted (!topdown || ...
means it's the topdown map).

PR 51254

uvm_map(9): Make KASSERTMSG unconditional for findspace invariants.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Apply the same orig_hint clamp again to the same entry.

Previous change dealt with case like length=0x1000 and:

[0x7defb000,0x7defc000)		entry above orig_hint (entry->next)
0x77895000			orig_hint
[0x77894000,0x77895000)		entry below orig_hint (entry)

by changing

	entry->next->start - length

to

	MIN(orig_hint, entry->next->start - length)

in order to enforce monotonicity of search.

In this case, if the tree search for a gap has failed, we retry with
a list search with exactly the same orig_hint and entry -- nothing
has changed them (only hint and tmp).  So apply the same clamping.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map: Fix build failure with DIAGNOSTIC for rev 1.422

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Make sure search in the nearest gap is monotonic.

The algorithm, on a hint clamped to the VM bounds, works as follows
(assuming topdown VM):

1. Make sure the hint is aligned, by subtracting the remainderin
   uvm_map_align_va.

2. If the hint is equal to the VM max, try the first free gap.

3. If the hint is not equal to the VM max, but is already in use, try
   the next gap _below_ the entry covering hint.

4. If the hint is not equal to the VM max and is not already in use,
   try gap between the entry below hint and the next entry after it,
   above hint.

In the last case, `entry' is the one below hint, and `entry->next' is
the one above it.  We would take

	entry->next->start - length

as the next candidate hint.

However, this algorithm is supposed to be a monotonic search through
the address space, and we might wind up with something like:

[0x7defb000,0x7defc000)         entry above hint (entry->next)
0x77895000                      hint
[0x77894000,0x77895000)         entry below hint (entry)

In this case, if length=0x1000, we would take

	0x7defb000 - 0x1000 = 0x7defa000

as the next candidate hint, but this violates monotonicity of the
search.

Instead, take the _smallest_ of orig_hint or entry->next->start -
length, to avoid violating monotonicity, so hint <= orig_hint.

I didn't commit this change before because it didn't seem to fix all
the manifestations of the problem, but we have more diagnostics now
so maybe we will find there is a _different_ violation of the same
invariants once this is committed -- and I'm pretty sure this change
is necessary to guarantee monotonicity in some cases (but I'm still
not sure why we're only hitting the problem on sh3).

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Show the next entry when findspace invariants fail too.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Take vm map lock around uvm_unmap_remove.

This was tripping one of the assertions I added.  While it is safe
here not to hold the lock -- caller has exclusive access to the map
at this point -- it is better if we can annotate the functions in
question with executable notes about locking rules, and taking a
single uncontended lock in the vm map destruction path is probably a
tiny cost worth those executable notes.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Show the entry in findspace invariants.

No functional change intended in the non-crash3 case.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Avoid potential arithmetic overflow.

Should be harmless in this case because vaddr_t is unsigned, so
there's no undefined behaviour here, but let's make it unnecessary to
wonder whether overflow is a problem.

No functional change intended.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Assert another loop invariant in entry lookup.

No functional change intended.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

Redo uvm_map.c 1.414 without the null pointer dereference.

uvm_map(9): Sprinkle assertions and interface contract comments.

No functional change intended.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

Revert uvm_map.c 1.414.

This was:

   uvm_map(9): Sprinkle assertions and interface contract comments.

Apparently, you have to actually test changes, not just prove they
are correct.  Who knew??

(And the incantation `No functional change intended.' didn't work
either!)

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Sprinkle invariant assertions into uvm_map_space_avail.

No functional change intended.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_map(9): Sprinkle assertions and interface contract comments.

No functional change intended.

PR kern/51254: uvm assertion "!topdown || hint <= orig_hint" failed

uvm_findspace_invariants: don't repeat the message three times

The topdown and bottomup messages were exactly the same and sh3 printf
hack added the third copy.  Restructure the code so that there's only
one message and make the message more obvious - the topdown condition
in the assertions was confusing b/c it's inverted (!topdown || ...
means it's the topdown map).

PR 51254

uvm_map.c: Fix kassertmsg/printf newline mismatch in PR 51254 note.

fix spelling mistakes, mainly in comments and log messages.

Repply this change with a couple of bugs fixed:

- Do away with separate pool_cache for some kernel objects that have no special
  requirements and use the general purpose allocator instead. On one of my
  test systems this makes for a small (~1%) but repeatable reduction in system
  time during builds presumably because it decreases the kernel's cache /
  memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.

Back out recent change to replace pool_cache with then general allocator.
Will return to this when I have time again.

- Do away with separate pool_cache for some kernel objects that have no special
  requirements and use the general purpose allocator instead.  On one of my
  test systems this makes for a small (~1%) but repeatable reduction in system
  time during builds presumably because it decreases the kernel's cache /
  memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.

Pull up following revision(s) (requested by riastradh in ticket #1880):

	sys/uvm/uvm_map.c: revision 1.403 (patch)

mmap(2): Avoid arithmetic overflow in search for free space.

PR kern/56900

uvm_findspace(): For sh3, convert a KASSERTMSG(9) into printf(9)

XXX
Work around for PR kern/51254 until it gets fixed.

With this change, landisk survives full ATF with DIAGNOSTIC enabled.

Pull up following revision(s) (requested by chs in ticket #167):

	sys/uvm/uvm_map.c: revision 1.406

uvm: avoid a deadlock in uvm_map_clean()

The locking order between map locks and page "busy" locks
is that the page "busy" lock comes first, but uvm_map_clean()
breaks this rule by holding a map locked (as reader) while
waiting for page "busy" locks.

If another thread is in the page-fault path holding a page
"busy" lock while waiting for the map lock (as a reader)
and at the same time a third thread is blocked waiting for
the map lock as a writer (which blocks the page-fault thread),
then these three threads will all deadlock with each other.

Fix this by marking the map "busy" (to block any modifications)
and unlocking the map lock before possibly waiting for any
page "busy" locks.

Martin Pieuchot reported that the same problem existed in OpenBSD
he applied this fix there after several people tested it.

fixes PR 56952

uvm: avoid a deadlock in uvm_map_clean()

The locking order between map locks and page "busy" locks
is that the page "busy" lock comes first, but uvm_map_clean()
breaks this rule by holding a map locked (as reader) while
waiting for page "busy" locks.

If another thread is in the page-fault path holding a page
"busy" lock while waiting for the map lock (as a reader)
and at the same time a third thread is blocked waiting for
the map lock as a writer (which blocks the page-fault thread),
then these three threads will all deadlock with each other.

Fix this by marking the map "busy" (to block any modifications)
and unlocking the map lock before possibly waiting for any
page "busy" locks.

Martin Pieuchot reported that the same problem existed in OpenBSD
he applied this fix there after several people tested it.

fixes PR 56952

Pull up following revision(s) (requested by riastradh in ticket #1625):

	sys/uvm/uvm_map.c: revision 1.403

mmap(2): Avoid arithmetic overflow in search for free space.

PR kern/56900

Pull up following revision(s) (requested by riastradh in ticket #1816):

	sys/uvm/uvm_map.c: revision 1.396

uvm(9): Fix mmap optimization for topdown case.

PR kern/51393

Pull up following revision(s) (requested by riastradh in ticket #1623):

	sys/uvm/uvm_map.c: revision 1.396

uvm(9): Fix mmap optimization for topdown case.

PR kern/51393

Pull up following revision(s) (requested by riastradh in ticket #1815):

	sys/uvm/uvm_map.c: revision 1.395

uvm(9): Fix 19-year-old bug in assertion about mmap hint.

Previously this would _first_ remember the original hint, and _then_
clamp the hint to the VM map's range:

        orig_hint = hint;
        if (hint < vm_map_min(map)) {   /* check ranges ... */
                if (flags & UVM_FLAG_FIXED) {
                        UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
                        return (NULL);
                }
                hint = vm_map_min(map);
        ...
        KASSERTMSG(!topdown || hint <= orig_hint, "hint: %#jx, orig_hint: %#jx",
            (uintmax_t)hint, (uintmax_t)orig_hint);

Even if nothing else happens in the ellipsis, taking the branch
guarantees the assertion will fail in the topdown case.

Pull up following revision(s) (requested by riastradh in ticket #1622):

	sys/uvm/uvm_map.c: revision 1.395

uvm(9): Fix 19-year-old bug in assertion about mmap hint.

Previously this would _first_ remember the original hint, and _then_
clamp the hint to the VM map's range:

        orig_hint = hint;
        if (hint < vm_map_min(map)) {   /* check ranges ... */
                if (flags & UVM_FLAG_FIXED) {
                        UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
                        return (NULL);
                }
                hint = vm_map_min(map);
        ...
        KASSERTMSG(!topdown || hint <= orig_hint, "hint: %#jx, orig_hint: %#jx",
            (uintmax_t)hint, (uintmax_t)orig_hint);

Even if nothing else happens in the ellipsis, taking the branch
guarantees the assertion will fail in the topdown case.

Unwrap. NFCI.

uvm(9): KASSERT(A && B) -> KASSERT(A); KASSERT(B)

While here, print some of the inputs with KASSERTMSG.

mmap(2): Avoid arithmetic overflow in search for free space.

PR kern/56900

Reported-by: syzbot+3833ae1d38037a263d05@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=e542bcf59b2564cca1cb38c12f076fb08dcac37e

initialize a variable to appease clang

PR kern/51254
uvm_map_findspace(): Output current value of "entry" when KASSERT fires.

uvm(9): Sprinkle assertions into uvm_map_findspace.

May help to diagnose PR kern/51254.

uvm(9): Don't duplicate vm_map_min/max in `show map' output.

Didn't notice these were already there, oops!

uvm(9): Sprinkle more info into hint/orig_hint assertions.

May help to diagnose PR kern/51254.

uvm(9): Print min/max address and first_free entry in ddb `show map'.

May help to diagnose PR kern/51254.

uvm(9): Fix mmap optimization for topdown case.

PR kern/51393

uvm(9): Fix 19-year-old bug in assertion about mmap hint.

Previously this would _first_ remember the original hint, and _then_
clamp the hint to the VM map's range:

	orig_hint = hint;
	if (hint < vm_map_min(map)) {	/* check ranges ... */
		if (flags & UVM_FLAG_FIXED) {
			UVMHIST_LOG(maphist,"<- VA below map range",0,0,0,0);
			return (NULL);
		}
		hint = vm_map_min(map);
	...
	KASSERTMSG(!topdown || hint <= orig_hint, "hint: %#jx, orig_hint: %#jx",
	    (uintmax_t)hint, (uintmax_t)orig_hint);

Even if nothing else happens in the ellipsis, taking the branch
guarantees the assertion will fail in the topdown case.

fix various typos in comments and output/log messages.

sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.

sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

	  Thread A			  Thread B
	obj->foo = 42;			obj->baz = 73;
	mumble(&obj->bar);		grumble(&obj->quux);
	/* membar_exit(); */		/* membar_exit(); */
	atomic_dec -- not last		atomic_dec -- last
					/* membar_enter(); */
					KASSERT(invariant(obj->foo,
					    obj->bar));
					free_stuff(obj);

The memory barriers ensure that

	obj->foo = 42;
	mumble(&obj->bar);

in thread A happens before

	KASSERT(invariant(obj->foo, obj->bar));
	free_stuff(obj);

in thread B.  Without them, this ordering is not guaranteed.

So in general it is necessary to do

	membar_exit();
	if (atomic_dec_uint_nv(&obj->refcnt) != 0)
		return;
	membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting.  (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these.  Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that.  It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.

More / improved debug

Sync with HEAD.

in uvm_mapent_forkzero(), if the old entry was an object mapping,
appease a debug check by setting the new entry offset to zero along with
setting the new entry object pointer to NULL.

Reported-by: syzbot+de8e4b223a3838c7307b@syzkaller.appspotmail.com
Reported-by: syzbot+efaea991addfdcc5abd4@syzkaller.appspotmail.com
Reported-by: syzbot+15d1e19dff9209c2e40b@syzkaller.appspotmail.com

remove diag-only printf() that fires when an unlinked file is mmapped
and someone runs ps(1) or similar.

fix error in previous: UVMHIST_PDHIST_SIZE needs to stay next to pdhistbuf[].

Sync with HEAD.

remove KERNHIST_INIT_STATIC().  it stradles the line between usable
early in boot and broken early in boot by requiring a partly static
structure with another structure that must be present by the time
any uses are performed.  theoretically platform code could allocate
a chunk while seting up memory and assign it here, giving a dynamic
sizing for the entry list, but the reality is that all users have
a statically allocated entry list as well.

the existing KERNHIST_LINK_STATIC() is used in conjunction with
KERNHIST_INITIALIZER() instead.

this stops a NULL pointer deref when the _LOG() macro is called
before the storage is linked in, which happens with GCC 10 on OCTEON
with UVMHIST enabled, crashing in very early kernel init.

Sync with HEAD.

Consistently use %#jx instead of 0x%jx or just %jx in UVMHIST_LOG formats

Consistently use UVMHIST(__func__)

Convert UVMHIST_{CALLED,LOG} into UVMHIST_CALLARGS

Avoid passing file paths in panic strings, this results in extra long
output that is annoying and that syzbot classifies as independent reports
due to the instances having different build paths.

Make the uvm_voaddr structure more compact, only occupying 2 pointers
worth of space, by encoding the type in the lower bits of the object
pointer.

- In uvm_voaddr_acquire(), take an extra hold on the anon lock obj.
- In uvm_voaddr_release(), if the anon ref count drops to 0, call
  uvm_anfree() rather than uvm_anon_release().  Unconditionally drop
  the anon lock, and release the extra hold on the anon lock obj.

Fixes a panic that occurs if the backing store for a futex backed by
an anon memory location is unmapped while a thread is waiting in the
futex.

Add a test case that reproduced the panic to verify that it's fixed.

Sync with HEAD

Sync with HEAD

Fix UVMHIST_LOG compile on 32bit platforms

Fix trailing whitespace.

Add an API to get a reference on the identity of an individual byte of
virtual memory, a "virtual object address".  This is not a reference to
a physical byte of memory, per se, but a reference to a byte residing
in a page, owned by a unique UVM object (either a uobj or an anon).  Two
separate address+addresses space tuples that reference the same byte in
an object (such as a location in a shared memory segment) will resolve
to equivalent virtual object addresses.  Even if the residency status
of the page changes, the virtual object address remains unchanged.

struct uvm_voaddr -- a structure that encapsulates this address reference.

uvm_voaddr_acquire() -- a function to acquire this address reference,
given a vm_map and a vaddr_t.

uvm_voaddr_release() -- a function to release this address reference.

uvm_voaddr_compare() -- a function to compare two such address references.

uvm_voaddr_acquire() resolves the COW status of the object address before
acquiring.

In collaboration with riastradh@ and chs@.

Mostly merge changes from HEAD upto 20200411

uvmspace_exec(): set VM_MAP_DYING for the duration, so pmap_update() is not
called until the pmap has been totally cleared out after pmap_remove_all(),
or it can confuse some pmap implementations.

Mark uvm_map_entry_cache with PR_LARGECACHE.

Process concurrent page faults on individual uvm_objects / vm_amaps in
parallel, where the relevant pages are already in-core.  Proposed on
tech-kern.

Temporarily disabled on MP architectures with __HAVE_UNLOCKED_PMAP until
adjustments are made to their pmaps.

Go back to freeing struct vm_anon one by one.  There may have been an
advantage circa ~2008 but there isn't now.

uvm_map_lookup_entry(): save the hint even on failure, since code elsewhere
relies on it pointing to the previous entry.

- uvmspace_exec(), uvmspace_free(): if pmap_remove_all() returns true the
  pmap is emptied.  Pass UVM_FLAG_VAONLY when clearing out the map and avoid
  needless extra work to tear down each mapping individually.

- uvm_map_lookup_entry(): remove the code to do a linear scan of map entries
  for small maps, in preference to using the RB tree.  It's questionable,
  and I think the code is almost never triggered because the average number
  of map entries has probably exceeded the hard-coded threshold for quite
  some time.

- vm_map_entry: get it aligned on a cacheline boundary, and cluster fields
  used during rbtree lookup at the beginning.

Sync with head.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart.  v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap.  Others to follow later.

Sync with head.

- uvm_unmap_remove(): need to call pmap_update() with the object still
  locked, otherwise the page could gain a new identity and still be visible
  via a stale mapping.

- Adjust reference counts with atomics.

remove unused predicate function

likely unused since kmem changes

- Add and use wrapper functions that take and acquire page interlocks, and pairs
  of page interlocks.  Require that the page interlock be held over calls to
  uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state.  Rather than
  updating the global state synchronously, set an intended state on
  individual pages (active, inactive, enqueued, dequeued) while holding the
  page interlock.  After the interlock is released put the pages on a 128
  entry per-CPU queue for their state changes to be made real in batch.
  This results in in a ~400 fold decrease in contention on my test system.
  Proposed on tech-kern but modified to use the page interlock rather than
  atomics to synchronise as it's much easier to maintain that way, and
  cheaper.

s/referece/reference/ in comment.

Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code.  Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour

Addionally pull up the following revision for ticket #388:

	sys/uvm/uvm_map.c			1.366

Fix previous; semantics of align argument of uvm_map() is different
when UVM_FLAG_COLORMATCH is specified.

Should fix PR kern/54669.

Fix previous; semantics of align argument of uvm_map() is different
when UVM_FLAG_COLORMATCH is specified.

Should fix PR kern/54669.

Pull up following revision(s) (requested by rin in ticket #388):

	sys/uvm/uvm_map.c: revision 1.365

PR kern/54395

- Align hint for virtual address at the beginning of uvm_map() if
   required. Otherwise, it will be rounded up/down in an unexpected
   way by uvm_map_space_avail(), which results in assertion failure.
   Fix kernel panic when executing earm binary (8KB pages) on aarch64
   (4KB pages), which relies on mmap(2) with MAP_ALIGNED flag.
- Use inline functions/macros consistently.
- Add some more KASSERT's.

For more details, see the PR as well as discussion on port-kern:
http://mail-index.netbsd.org/tech-kern/2019/10/27/msg025629.html

PR kern/54395

- Align hint for virtual address at the beginning of uvm_map() if
  required. Otherwise, it will be rounded up/down in an unexpected
  way by uvm_map_space_avail(), which results in assertion failure.

  Fix kernel panic when executing earm binary (8KB pages) on aarch64
  (4KB pages), which relies on mmap(2) with MAP_ALIGNED flag.

- Use inline functions/macros consistently.

- Add some more KASSERT's.

For more details, see the PR as well as discussion on port-kern:
http://mail-index.netbsd.org/tech-kern/2019/10/27/msg025629.html

KASSERT -> KASSERTMSG so we actually display the overflowed values.

Pull up following revision(s) (requested by maxv in ticket #1320):

	sys/uvm/uvm_map.c: revision 1.361

Fix info leak: 'map_attrib' is not used in UVM, and contains uninitialized
heap garbage. Return zero. Maybe we should remove the field completely.

Remove last trace of never-used map_attrib.

Add missing lock around pmap_protect.
ok, chs@

Reported-by: syzbot+6bfd0be70896fc9e9a3d@syzkaller.appspotmail.com

Fix info leak: 'map_attrib' is not used in UVM, and contains uninitialized
heap garbage. Return zero. Maybe we should remove the field completely.

Sync with HEAD

in uvm_map_protect(), do a pmap_update() before possibly switching from
removing pmap entries to creating them.  this fixes the problem reported in
https://syzkaller.appspot.com/bug?id=cc89e47f05e4eea2fd69bcccb5e837f8d1ab4d60

Avoid a panic from the sequence

	mlock(buf, 0);
	munlock(buf, 0);
	mlock(buf, page);
	munlock(buf, page);

where buf is page aligned, and page is actually anything > 0
(but not too big) which will get rounded up to the next multiple
of the page size.

In that sequence, it is possible that the 1st munlock() is optional.

Add a KASSERT() (or two) to detect the first effects of the problem
(without that, or in !DIAGNOSTIC kernels) the problem eventually
causes some kind of problem or other (most often still a panic.)

After this, mlock(anything, 0) (or munlock) validates "anything"
but is otherwise a no-op (regardless of the alignment of anything).

Also, don't treat mlock(buf, verybig) as equivalent to mlock(buf, 0)
which is (more or less) what we had been doing.

XXX pullup -8 (maybe -7 as well, need to check).

Fix bug, the entry we're iterating on is 'current', not 'entry'. Here only
the first entry gets wired in.

Sync with HEAD, resolve a few conflicts

Raise the fill_vmentries() E2BIG limit from 1MB to 10MB

The previous limit was not enough for libFuzzer as it requires up to 2.5MB
in test-suite.

Alternative approaches to retrieve larger address map during happened to be
worse during the evaluation due to difficulties in locking and atomicity.

Discussed with <christos>

Ssync with HEAD

Remove this check, it has never protected against mmap on page zero, and
has since been replaced by the code in exec_vm_minaddr.

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int.  The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER!  Some subsystems have

	#define min(a, b)	((a) < (b) ? (a) : (b))
	#define max(a, b)	((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX.  Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate.  But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all.  (Who knows, maybe in some cases integer
truncation is actually intended!)

uvm_map_extract() has an indentation issue.

update from HEAD

Pull up following revision(s) (requested by pgoyette in ticket #335):
	share/man/man9/kernhist.9: 1.5-1.8
	sys/arch/acorn26/acorn26/pmap.c: 1.39
	sys/arch/arm/arm32/fault.c: 1.105 via patch
	sys/arch/arm/arm32/pmap.c: 1.350, 1.359
	sys/arch/arm/broadcom/bcm2835_bsc.c: 1.7
	sys/arch/arm/omap/if_cpsw.c: 1.20
	sys/arch/arm/omap/tiotg.c: 1.7
	sys/arch/evbarm/conf/RPI2_INSTALL: 1.3
	sys/dev/ic/sl811hs.c: 1.98
	sys/dev/usb/ehci.c: 1.256
	sys/dev/usb/if_axe.c: 1.83
	sys/dev/usb/motg.c: 1.18
	sys/dev/usb/ohci.c: 1.274
	sys/dev/usb/ucom.c: 1.119
	sys/dev/usb/uhci.c: 1.277
	sys/dev/usb/uhub.c: 1.137
	sys/dev/usb/umass.c: 1.160-1.162
	sys/dev/usb/umass_quirks.c: 1.100
	sys/dev/usb/umass_scsipi.c: 1.55
	sys/dev/usb/usb.c: 1.168
	sys/dev/usb/usb_mem.c: 1.70
	sys/dev/usb/usb_subr.c: 1.221
	sys/dev/usb/usbdi.c: 1.175
	sys/dev/usb/usbdi_util.c: 1.67-1.70
	sys/dev/usb/usbroothub.c: 1.3
	sys/dev/usb/xhci.c: 1.75
	sys/external/bsd/drm2/dist/drm/i915/i915_gem.c: 1.34
	sys/kern/kern_history.c: 1.15
	sys/kern/kern_xxx.c: 1.74
	sys/kern/vfs_bio.c: 1.275-1.276
	sys/miscfs/genfs/genfs_io.c: 1.71
	sys/sys/kernhist.h: 1.21
	sys/ufs/ffs/ffs_balloc.c: 1.63
	sys/ufs/lfs/lfs_vfsops.c: 1.361
	sys/ufs/lfs/ulfs_inode.c: 1.21
	sys/ufs/lfs/ulfs_vnops.c: 1.52
	sys/ufs/ufs/ufs_inode.c: 1.102
	sys/ufs/ufs/ufs_vnops.c: 1.239
	sys/uvm/pmap/pmap.c: 1.37-1.39
	sys/uvm/pmap/pmap_tlb.c: 1.22
	sys/uvm/uvm_amap.c: 1.108
	sys/uvm/uvm_anon.c: 1.64
	sys/uvm/uvm_aobj.c: 1.126
	sys/uvm/uvm_bio.c: 1.91
	sys/uvm/uvm_device.c: 1.66
	sys/uvm/uvm_fault.c: 1.201
	sys/uvm/uvm_km.c: 1.144
	sys/uvm/uvm_loan.c: 1.85
	sys/uvm/uvm_map.c: 1.353
	sys/uvm/uvm_page.c: 1.194
	sys/uvm/uvm_pager.c: 1.111
	sys/uvm/uvm_pdaemon.c: 1.109
	sys/uvm/uvm_swap.c: 1.175
	sys/uvm/uvm_vnode.c: 1.103
	usr.bin/vmstat/vmstat.c: 1.219
Reorder to test for null before null deref in debug code
--
Reorder to test for null before null deref in debug code
--
KNF
--
No need for '\n' in UVMHIST_LOG
--
normalise a BIOHIST log message
--
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
   the kernel and in the structures used for exporting the history data
   to userland via sysctl(9).  This avoids problems on some architectures
   where passing a 64-bit (or larger) value to printf(3) can cause it to
   process the value as multiple arguments.  (This can be particularly
   problematic when printf()'s format string is not a literal, since in
   that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
   include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
   updated.  Each format specifier now includes an explicit length
   modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
   updated to replace uses of "%p" with "%#jx", and the pointer
   arguments are now cast to (uintptr_t) before being subsequently cast
   to (uintmax_t).  This is needed to avoid compiler warnings about
   casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
   "%c" format strings replaced with numeric formats; several instances
   of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
   history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
   the -u option does not exist (previously, this condition was silently
   ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
   data exported via sysctl(9) and exits if they do not match the values
   with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
   requirements imposed on the format strings, along with several other
   minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
     uint64_t) for the history arguments.  But that would require another
     "rototill" of all the users in the future when we add support for an
     architecture that supports a larger size.  Also, the printf(3)
format
     specifiers for explicitly-sized values, such as "%"PRIu64, are much
     more verbose (and less aesthetically appealing, IMHO) than simply
     using "%ju".
[2] I've tried very hard to find "all [the] existing users of
kernhist(9)"
     but it is possible that I've missed some of them.  I would be glad
to
     update any stragglers that anyone identifies.
--
For some reason this single kernel seems to have outgrown its declared
size as a result of the kernhist(9) changes.  Bump the size.
XXX The amount of increase may be excessive - anyone with more detailed
XXX knowledge please feel free to further adjust the value
appropriately.
--
Misssed one cast of pointer --> uintptr_t in previous kernhist(9) commit
--
And yet another one.  :(
--
Use correct mark-up for NetBSD version.
--
More improvements in grammar and readability.
--
Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.
--
And replace an instance of "%p" conversion with "%#jx"
--
Whitespace fix. Give Bl tag table a width. Fix Xr.

Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
  the kernel and in the structures used for exporting the history data
  to userland via sysctl(9).  This avoids problems on some architectures
  where passing a 64-bit (or larger) value to printf(3) can cause it to
  process the value as multiple arguments.  (This can be particularly
  problematic when printf()'s format string is not a literal, since in
  that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
  include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
  updated.  Each format specifier now includes an explicit length
  modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
  updated to replace uses of "%p" with "%#jx", and the pointer
  arguments are now cast to (uintptr_t) before being subsequently cast
  to (uintmax_t).  This is needed to avoid compiler warnings about
  casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
  "%c" format strings replaced with numeric formats; several instances
  of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
  history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
  the -u option does not exist (previously, this condition was silently
  ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
  data exported via sysctl(9) and exits if they do not match the values
  with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
  requirements imposed on the format strings, along with several other
  minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
    uint64_t) for the history arguments.  But that would require another
    "rototill" of all the users in the future when we add support for an
    architecture that supports a larger size.  Also, the printf(3) format
    specifiers for explicitly-sized values, such as "%"PRIu64, are much
    more verbose (and less aesthetically appealing, IMHO) than simply
    using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
    but it is possible that I've missed some of them.  I would be glad to
    update any stragglers that anyone identifies.

Pull up following revision(s) (requested by pgoyette in ticket #294):
	sys/uvm/uvm_map.c: revision 1.352
Fix user-triggerable kernel crash as reported in PR kern/52573 (from
Bruno Haible).

Fix user-triggerable kernel crash as reported in PR kern/52573 (from
Bruno Haible).

XXX Pull-up to netbsd-8

Sync with HEAD

add assertions that would have caught the recent audio mmap bugs.

sprinkle __diagused to fix the powerpc build, which is not DIAGNOSTIC.

MAP_FIXED means something different for mremap() than it does for mmap(),
so we cannot use UVM_FLAG_FIXED to specify both behaviors.
keep UVM_FLAG_FIXED with its earlier meaning (prior to my previous change)
of whether to use uvm_map_findspace() to locate space for the new mapping or
to use the hint address that the caller passed in, and add a new flag
UVM_FLAG_UNMAP to indicate that any existing entries in the range should be
unmapped as part of creating the new mapping.  the new UVM_FLAG_UNMAP flag
may only be used if UVM_FLAG_FIXED is also specified.

Add missing , to fix syntax

Unbreaks build after recent change adding a message for vm.user_va0_disable

make MAP_FIXED mapping operations atomic. fixes PR 52239.
previously, unmapping any entries being replaced was done separately
from entering the new mapping, which allowed another thread doing
a non-MAP_FIXED mapping to allocate the range out from under the
MAP_FIXED thread.

mention the man page instead of the command.

Provide a helpful message to the user trying to run an birary that needs page
0 access.

Sync with HEAD

Extend the mmap(2) interface to allow requesting protections for later
use with mprotect(2), but without enabling them immediately.

Extend the mremap(2) interface to allow duplicating mappings, i.e.
create a second range of virtual addresses references the same physical
pages. Duplicated mappings can have different effective protections.

Adjust PAX mprotect logic to disallow effective protections of W&X, but
allow one mapping W and another X protections. This obsoletes using
temporary files for purposes like JIT.

Adjust PAX logic for mmap(2) and mprotect(2) to fail if W&X is requested
and not silently drop the X protection.

Improve test cases to ensure correct operation of the changed
interfaces.

Sync with HEAD

Sync with HEAD

PR/52078: Don't panic on 0 allocation, check more bounds.

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Sync with HEAD

allow the sizes of the maphist and pdhist to be set in the config
file via UVMHIST_MAPHIST_SIZE and UVMHIST_PDHIST_SIZE.

Sync with HEAD

The way the kernel tries to prevent a userland process from allocating page
zero is hugely flawed. It is easy to demonstrate that one can trick UVM
into chosing a NULL hint after the user_va0_disable check from uvm_map.
Such a bypass allows kernel NULL pointer dereferences to be exploitable on
architectures with a shared userland<->kernel VA, like amd64.

Fix this by increasing the limit of the vm space made available for
userland processes. This way, UVM will never chose a NULL hint, since it
would be outside of the vm space.

The user_va0_disable sysctl still controls this feature.

Sync with HEAD

KNF. Remove extra spaces. No functional change.

Change two KASSERT to KASSERTMSG to provide better diagnostics.

Avoid locking issues when copying out requires taking a fault and we are
finding out our own maps, by allocating a buffer and copying out after
we collected the information.

Sync with HEAD

Introduce security.pax.mprotect.ptrace sysctl which can be used to bypass
mprotect settings so that debuggers can write to the text segment of traced
processes so that they can insert breakpoints. Turned off by default.
Ok: chuq (for now)

Sync with HEAD (as of 26th Dec)

Now that SYSVSHM is modularized, reattach the linkages from uvm so that
we can correctly clean up on process exit or fork.

Without this, firefox attaches to a shared memory segment but doesn't
detach before exit.  Thus once firefox causes an autoload for sysv_ipc
it cannot be unloaded since the segment still retains references.

implement VM_PROC_MAP

Sync with HEAD

Use %p, %#xl etc. for pointers and addresses.

Sync with HEAD

The diagnostic function uvm_km_check_empty() takes a mutex, so don't call it
if we are using UVM_FLAG_NOWAIT.

Pull up following revision(s) (requested by chs in ticket #447):
	sys/uvm/uvm_map.c: revision 1.332
skip busy anon pages in uvm_map_clean().
we shouldn't be messing with pages that someone else has busy,
and uvm_map_clean() is just advisory for amap mappings.

skip busy anon pages in uvm_map_clean().
we shouldn't be messing with pages that someone else has busy,
and uvm_map_clean() is just advisory for amap mappings.

Define UVMDEBUG for expensive debugging operations. Idea from chuq.

Rebase to HEAD as of a few days ago.

Rebase.

Add MAP_INHERIT_ZERO

Split out the minherit code into separate functions for readability (allows
us to indent them properly), and merge the new vm_map_entry creation into
a common function to avoid code duplication. No functional change.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Use UVMHIST_INITIALIZER (KERNHIST_INITIALIZER) to statically initialize
maphist.  This allows maphist to used very very early in boot well before
uvm has been initialized.

As discussed on tech-kern: make TOPDOWN-VM runtime selectable per process
(offer MD code or emulations to override it).

Mark diagnostic-only variables

Some pmaps may not consume all arguments of pmap_copy()

sync with (a bit old) head

Resync to 2012-11-19 00:00:00 UTC

When uvm_io reserves kernel address space, make sure it's starts with the
same color as the user address space being copied.

sync with netbsd-6-0-RELEASE.

sync with head

get rid of not used uvm_map flag (UVM_MAP_KMAPENT)

Initial snapshot of work to eliminate 64K MAXPHYS.  Basically works for
physio (I/O to raw devices); needs more doing to get it going with the
filesystems, but it shouldn't damage data.

All work's been done on amd64 so far.  Not hard to add support to other
ports.  If others want to pitch in, one very helpful thing would be to
sort out when and how IDE disks can do 128K or larger transfers, and
adjust the various PCI IDE (or at least ahcisata) drivers and wd.c
accordingly -- it would make testing much easier.  Another very helpful
thing would be to implement a smart minphys() for RAIDframe along the
lines detailed in the MAXPHYS-NOTES file.

Pull up following revision(s) (requested by para in ticket #547):
	sys/uvm/uvm_map.c: revision 1.320
	sys/uvm/uvm_map.c: revision 1.321
	sys/uvm/uvm_map.c: revision 1.322
	sys/uvm/uvm_km.c: revision 1.130
	sys/uvm/uvm_km.c: revision 1.131
	sys/uvm/uvm_km.c: revision 1.132
	sys/uvm/uvm_km.c: revision 1.133
	sys/uvm/uvm_km.c: revision 1.134
	sys/uvm/uvm_km.c: revision 1.135
	sys/uvm/uvm_km.c: revision 1.129
Fix a bug where the kernel was never grown to accomodate the kmem VA space
since that happens before the kernel_map is set.
Don't try grow the entire kmem space but just do as needed in
uvm_km_kmem_alloc
Shut up gcc printf warning.
Cleanup comment.  Change panic to KASSERTMSG.
Use kernel_map->misc_lock to make sure we don't call pmap_growkernel
concurrently and possibly mess up uvm_maxkaddr.
Switch to a spin lock (uvm_kentry_lock) which, fortunately, was
sitting there
unused.
Remove locking since it isn't needed.  As soon as the 2nd
uvm_map_entry in kernel_map
is created, uvm_map_prepare will call pmap_growkernel and the
pmap_growkernel call in
uvm_km_mem_alloc will never be called again.
call pmap_growkernel once after the kmem_arena is created
to make the pmap cover it's address space
assert on the growth in uvm_km_kmem_alloc
for the 3rd uvm_map_entry uvm_map_prepare will grow the kernel,
but we might call into uvm_km_kmem_alloc through imports to
the kmem_meta_arena earlier
while here guard uvm_km_va_starved_p from kmem_arena not yet created
thanks for tracking this down to everyone involved

Remove locking since it isn't needed.  As soon as the 2nd uvm_map_entry in kernel_map
is created, uvm_map_prepare will call pmap_growkernel and the pmap_growkernel call in
uvm_km_mem_alloc will never be called again.

Switch to a spin lock (uvm_kentry_lock) which, fortunately, was sitting there
unused.

Cleanup comment.  Change panic to KASSERTMSG.
Use kernel_map->misc_lock to make sure we don't call pmap_growkernel
concurrently and possibly mess up uvm_maxkaddr.

Pull up following revision(s) (requested by chs in ticket #1790):
	sys/uvm/uvm_map.c: revision 1.319
avoid leaking a uvm_object reference when merging a new map entry
with the entries on both sides.  fixes PR 46807.

Pull up following revision(s) (requested by chs in ticket #508):
	sys/uvm/uvm_map.c: revision 1.319
avoid leaking a uvm_object reference when merging a new map entry
with the entries on both sides.  fixes PR 46807.

avoid leaking a uvm_object reference when merging a new map entry
with the entries on both sides.  fixes PR 46807.

-fno-common broke kernhist since it used commons.
Add a KERNHIST_DEFINE which is define the kernel history.
Change UVM to deal with the new usage.

Use uvm_km_pagefree to free a kmap entry page.

sync to latest -current.

sync with head

Pull up following revision(s) (requested by martin in ticket #175):
	sys/kern/kern_exit.c: revision 1.238
	tests/lib/libc/gen/posix_spawn/t_fileactions.c: revision 1.4
	tests/lib/libc/gen/posix_spawn/t_fileactions.c: revision 1.5
	sys/uvm/uvm_extern.h: revision 1.183
	lib/libc/gen/posix_spawn_fileactions.c: revision 1.2
	sys/kern/kern_exec.c: revision 1.348
	sys/kern/kern_exec.c: revision 1.349
	sys/compat/netbsd32/syscalls.master: revision 1.95
	sys/uvm/uvm_glue.c: revision 1.159
	sys/uvm/uvm_map.c: revision 1.317
	sys/compat/netbsd32/netbsd32.h: revision 1.95
	sys/kern/exec_elf.c: revision 1.38
	sys/sys/spawn.h: revision 1.2
	sys/sys/exec.h: revision 1.135
	sys/compat/netbsd32/netbsd32_execve.c: revision 1.34
Rework posix_spawn locking and memory management:
 - always provide a vmspace for the new proc, initially borrowing from proc0
   (this part fixes PR 46286)
 - increase parallelism between parent and child if arguments allow this,
   avoiding a potential deadlock on exec_lock
 - add a new flag for userland to request old (lockstepped) behaviour for
   better error reporting
 - adapt test cases to the previous two and add a new variant to test the
   diagnostics flag
 - fix a few memory (and lock) leaks
 - provide netbsd32 compat
Fix asynchronous posix_spawn child exit status (and test for it).

Rework posix_spawn locking and memory management:
 - always provide a vmspace for the new proc, initially borrowing from proc0
   (this part fixes PR 46286)
 - increase parallelism between parent and child if arguments allow this,
   avoiding a potential deadlock on exec_lock
 - add a new flag for userland to request old (lockstepped) behaviour for
   better error reporting
 - adapt test cases to the previous two and add a new variant to test the
   diagnostics flag
 - fix a few memory (and lock) leaks
 - provide netbsd32 compat

sync to latest -current.

Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

    http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
    http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
    http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

Improve UVM_PAGE_TRKOWN.
Add more asserts to uvm_page.

sync to -current.

Pull up following revision(s) (requested by bouyer in ticket #29):
	sys/arch/xen/x86/x86_xpmap.c: revision 1.39
	sys/arch/xen/include/hypervisor.h: revision 1.37
	sys/arch/xen/include/intr.h: revision 1.34
	sys/arch/xen/x86/xen_ipi.c: revision 1.10
	sys/arch/x86/x86/cpu.c: revision 1.97
	sys/arch/x86/include/cpu.h: revision 1.48
	sys/uvm/uvm_map.c: revision 1.315
	sys/arch/x86/x86/pmap.c: revision 1.165
	sys/arch/xen/x86/cpu.c: revision 1.81
	sys/arch/x86/x86/pmap.c: revision 1.167
	sys/arch/xen/x86/cpu.c: revision 1.82
	sys/arch/x86/x86/pmap.c: revision 1.168
	sys/arch/xen/x86/xen_pmap.c: revision 1.17
	sys/uvm/uvm_km.c: revision 1.122
	sys/uvm/uvm_kmguard.c: revision 1.10
	sys/arch/x86/include/pmap.h: revision 1.50
Apply patch proposed in PR port-xen/45975 (this does not solve the exact
problem reported here but is part of the solution):
xen_kpm_sync() is not working as expected,
leading to races between CPUs.
1 the check (xpq_cpu != &x86_curcpu) is always false because we
  have different x86_curcpu symbols with different addresses in the kernel.
  Fortunably, all addresses dissaemble to the same code.
  Because of this we always use the code intended for bootstrap, which doesn't
  use cross-calls or lock.
2 once 1 above is fixed, xen_kpm_sync() will use xcalls to sync other CPUs,
  which cause it to sleep and pmap.c doesn't like that. It triggers this
  KASSERT() in pmap_unmap_ptes():
  KASSERT(pmap->pm_ncsw == curlwp->l_ncsw);
3 pmap->pm_cpus is not safe for the purpose of xen_kpm_sync(), which
  needs to know on which CPU a pmap is loaded *now*:
  pmap->pm_cpus is cleared before cpu_load_pmap() is called to switch
  to a new pmap, leaving a window where a pmap is still in a CPU's
  ci_kpm_pdir but not in pm_cpus. As a virtual CPU may be preempted
  by the hypervisor at any time, it can be large enough to let another
  CPU free the PTP and reuse it as a normal page.
To fix 2), avoid cross-calls and IPIs completely, and instead
use a mutex to update all CPU's ci_kpm_pdir from the local CPU.
It's safe because we just need to update the table page, a tlbflush IPI will
happen later. As a side effect, we don't need a different code for bootstrap,
fixing 1). The mutex added to struct cpu needs a small headers reorganisation.
to fix 3), introduce a pm_xen_ptp_cpus which is updated from
cpu_pmap_load(), whith the ci_kpm_mtx mutex held. Checking it with
ci_kpm_mtx held will avoid overwriting the wrong pmap's ci_kpm_pdir.
While there I removed the unused pmap_is_active() function;
and added some more details to DIAGNOSTIC panics.
When using uvm_km_pgremove_intrsafe() make sure mappings are removed
before returning the pages to the free pool. Otherwise, under Xen,
a page which still has a writable mapping could be allocated for
a PDP by another CPU and the hypervisor would refuse it (this is
PR port-xen/45975).
For this, move the pmap_kremove() calls inside uvm_km_pgremove_intrsafe(),
and do pmap_kremove()/uvm_pagefree() in batch of (at most) 16 entries
(as suggested by Chuck Silvers on tech-kern@, see also
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012727.html and
followups).
Avoid early use of xen_kpm_sync(); locks are not available at this time.
Don't call cpu_init() twice.
Makes LOCKDEBUG kernels boot again
Revert pmap_pte_flush() -> xpq_flush_queue() in previous.

When using uvm_km_pgremove_intrsafe() make sure mappings are removed
before returning the pages to the free pool. Otherwise, under Xen,
a page which still has a writable mapping could be allocated for
a PDP by another CPU and the hypervisor would refuse it (this is
PR port-xen/45975).
For this, move the pmap_kremove() calls inside uvm_km_pgremove_intrsafe(),
and do pmap_kremove()/uvm_pagefree() in batch of (at most) 16 entries
(as suggested by Chuck Silvers on tech-kern@, see also
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012727.html and
followups).

Remove VM_MAP_INTRSAFE and related code.  Not used since the "kmem changes".

merge to -current.

Fix another merge botch - bracket vm space assignement with kpreempt-
disable/enable.

Major changes to uvm.
Support multiple collections (groups) of free pages and run the page
reclaimation algorithm on each group independently.

pool_page_alloc, pool_page_alloc_meta: avoid extra compare, use const.
ffs_mountfs,sys_swapctl: replace memset with kmem_zalloc.
sys_swapctl: move kmem_free outside the lock path.
uvm_init: fix comment, remove pointless numeration of steps.
uvm_map_enter: remove meflagval variable.
Fix some indentation.

extending vmem(9) to be able to allocated resources for it's own needs.
simplifying uvm_map handling (no special kernel entries anymore no relocking)
make malloc(9) a thin wrapper around kmem(9)
(with private interface for interrupt safety reasons)

releng@ acknowledged

Revert MAP_NOSYSCALLS patch.

- use O->A loan to serve read(2).  based on a patch from Chuck Silvers
- associated O->A loan fixes.

Redo uvm_map_setattr() to never fail and remove the possible panic. The
possibility of failure was a C&P error.

Add a MAP_NOSYSCALLS flag to mmap. This flag prohibits executing of system
calls from the mapped region. This can be used for emulation perposed or for
extra security in the case of generated code.

Its implemented by adding mapping-attributes to each uvm_map_entry. These can
then be queried when needed.

Currently the MAP_NOSYSCALLS is only implemented for x86 but other
architectures are easy to adapt; see the sys/arch/x86/x86/syscall.c patch.
Port maintainers are encouraged to add them for their processor ports too.
When this feature is not yet implemented for an architecture the
MAP_NOSYSCALLS is simply ignored with virtually no cpu cost..

comment and assertion

When allocating pages for kernel map entries and PMAP_ALLOC_POOLPAGE is
defined, use it.  (allows a MIPS N32 kernel to boot when there is memory
outside of KSEG0).

Modify *ASSERTMSG() so they are now used as variadic macros. The main goal
is to provide routines that do as KASSERT(9) says: append a message
to the panic format string when the assertion triggers, with optional
arguments.

Fix call sites to reflect the new definition.

Discussed on tech-kern@. See
http://mail-index.netbsd.org/tech-kern/2011/09/07/msg011427.html

Forward some UVM from matt-nb5-mips64.  Add UVM_KMF_COLORMATCH flag.
When uvm_map gets passed UVM_FLAG_COLORMATCH, the align argument contains
the color of the starting address to be allocated (0..colormask).
When uvm_km_alloc is passed UVM_KMF_COLORMATCH (which can only be used with
UVM_KMF_VAONLY), the align argument contain the color of the starting address
to be allocated.
Change uvm_pagermapin to use this.  When mapping user pages in the kernel,
if colormatch is used with the color of the starting user page then the kernel
mapping will be congruent with the existing user mappings.

- Rework uvm_anfree() into uvm_anon_freelst(), which always drops the lock.
- Free anons in uvm_anon_freelst() without lock held.
- Mechanic sync to unused loaning code.

Make uvmspace_exec() deal with procs that have no vmspace (yet) at all.
Greatly simplifies the upcoming posix_spawn implementation.

Get rid of #ifdef __sparc__ in uvm code - as noted by cgd back 1996,
now that we have __HAVE_CPU_VMSPACE_EXEC/cpu_vmspace_exec().

- fix a use-after-free bug in uvm_km_free.
  (after uvm_km_pgremove frees pages, the following pmap_remove touches them.)
- acquire the object lock for operations on pmap_kernel as it can actually be
  raced with P->V operations.  eg. pagedaemon.

Catchup with rmind-uvmplock merge.

uvm_map_lock_entry: fix the order of locking.  Spotted by yamt@.
Also, keep uvm_map_unlock_entry() symmetric.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

Sync with HEAD.

Restore $NetBSD$

Rework page free lists to be sorted by color first rather than free_list.
Kept per color PGFL_* counter in each page free list.
Minor cleanups.

sync with head

Make uvm_map recognize UVM_FLAG_COLORMATCH which tells uvm_map that the
'align' argument specifies the starting color of the KVA range to be returned.

When calling uvm_km_alloc with UVM_KMF_VAONLY, also specify the starting
color of the kva range returned (UMV_KMF_COLORMATCH) and pass those to
uvm_map.

In uvm_pglistalloc, make sure the pages being returned have sequentially
advancing colors (so they can be mapped in a contiguous address range).
Add a few missing UVM_FLAG_COLORMATCH flags to uvm_pagealloc calls.

Make the socket and pipe loan color-safe.

Make the mips pmap enforce strict page color (color(VA) == color(PA)).

move and rename the uvm history code out of uvm_stat to "kernhist".

rename "UVMHIST" option to enable the uvm histories.

TODO:
- make UVMHIST properly depend upon KERNHIST
- enable dynamic registration of histories.  this is mostly just
  allocating something in a bitmap, and is only for viewing multiple
  histories in a merged form.


tested on amd64 and sparc64.

sync with head

comment

sync with head

Sync with HEAD

udpate license clauses on my code to match the new-style BSD licenses.
based on second diff that rmind@ sent me.

no functional change with this commit.

Add a MD hook to indicate a change of vmspace due to exec.  (This is useful
to update any cpu flag due to a change to/from a 64bit and a 32bit address
space).  This can set the state needed for copyout/copyin before setregs
is invoked.

Pull up following revision(s) (requested by rmind in ticket #1421):
	sys/uvm/uvm_bio.c: revision 1.70
	sys/uvm/uvm_map.c: revision 1.292
	sys/uvm/uvm_pager.c: revision 1.98
	sys/uvm/uvm_fault.c: revision 1.175
	sys/uvm/uvm_bio.c: revision 1.69
ubc_fault: split-off code part handling a single page into ubc_fault_page().
Keep the lock around pmap_update() where required.  While fixing this
in ubc_fault(), rework logic to "remember" the last object of page and
reduce locking overhead, since in common case pages belong to one and
the same UVM object (but not always, therefore add a comment).
Unlocks before pmap_update(), on removal of mappings, might cause TLB
coherency issues, since on architectures like x86 and mips64 invalidation
IPIs are deferred to pmap_update().  Hence, VA space might be globally
visible before IPIs are sent or while they are still in-flight.
OK ad@.

Sync with HEAD (-D20101022).

sync with head

Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
   would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
   already existing one.
4. Amend the interface to manipulate the actual object, instead of the
   rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@

Use __HAVE_CPU_VMSPACE_EXEC instead of a mips-specific #ifdef.

Add a hook so that MD code has handle the change in address space limits
when an exec happens.
Add a routine to turn on/off UX when an address space changes due to an exec
(N32 execing a N64 for instance).

Sync with HEAD.

sync with head.

sync with head

Keep the lock around pmap_update() where required.  While fixing this
in ubc_fault(), rework logic to "remember" the last object of page and
reduce locking overhead, since in common case pages belong to one and
the same UVM object (but not always, therefore add a comment).

Unlocks before pmap_update(), on removal of mappings, might cause TLB
coherency issues, since on architectures like x86 and mips64 invalidation
IPIs are deferred to pmap_update().  Hence, VA space might be globally
visible before IPIs are sent or while they are still in-flight.

OK ad@.

sync with head

Move PMAP_KMPAGE to be used in pmap_kenter_pa flags argument.
'Looks good to me' gimpy@

Forgot to commit this piece in previous commit

Sync with HEAD.

Reorganise UVM locking to protect P->V state and serialise pmap(9)
operations on the same page(s) by always locking their owner.  Hence
lock order: "vmpage"-lock -> pmap-lock.

Patch, proposed on tech-kern@, from Andrew Doran.

Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc().  It allows us to share the locks among UVM objects.

sync with head

rename the va0_disabled option and cpp conditional to "disable" as well,
for consistency, and document option and sysctl flag

rename the new sysctl to "vm.user_va0_disable", for consistency
with the majority of existing sysctl flags, suggested by yamt

Disable mapping of virtual address 0 by user programs per default.
This blocks an easy exploit of kernel bugs leading to dereference
of a NULL pointer on some architectures (eg i386).
The check can be disabled in various ways:
-by CPP definitions in machine/types.h (portmaster's choice)
-by a kernel config option USER_VA0_DISABLED_DEFAULT=0
-at runtime by sysctl vm.user_va0_disabled (cannot be cleared
 at securelevel>0)

Remove separate mb_map. The nmbclusters is computed at boot time based
on the amount of physical memory and limited by NMBCLUSTERS if present.
Architectures without direct mapping also limit it based on the kmem_map
size, which is used as backing store. On i386 and ARM, the maximum KVA
used for mbuf clusters is limited to 64MB by default.

The old default limits and limits based on GATEWAY have been removed.
key_registered_sb_max is hard-wired to a value derived from 2048
clusters.

Use PRIxVADDR... (change a printf/panic -> panic)

Use PRIxVADDR ...

Add a flags argument to pmap_kenter_pa(9).
Patch showed on tech-kern@ http://mail-index.netbsd.org/tech-kern/2009/11/04/msg006434.html
No objections.

Consistently call amap / uobj layers as upper / lower, because UVM has only
those two layers by design.  Approved by Chuck Cranor some time ago.

sync with head

uvmspace_unshare: #if 0-out this function.  Q: perhaps remove?
AFAIK it was not used for 11 years.

PRIxVADDR, PRIdVSIZE, PRIxVSIZE, or PRIxPADDR as appropriate.
Use __intXX_t or __uintXX_t as appropriate in <mips/types.h>

sync with head.

In uvm_kmapent_alloc, Make sure entry is initialized.
Spotted by msaitoh.

Move uvm_object-related DDB hooks into uvm_object.c.  Put all of the
uvm_map-related DDB stuff in one spot in the file.

Move uvm_page-related DDB hooks into uvm_page.c.

Fix KASSERT() failure reported by Geoff Wing.

Compare vaddr_t against 0, not NULL.

If PMAP_MAP_POOLPAGE is defined, use it to map kernel map entries.  This
avoids TLB pollution on those platforms that define it.

- uvm_map_extract: update map->size correctly for !UVM_EXTRACT_CONTIG.
- uvm_map_extract: panic on zero-sized entries.
- make uvm_map_replace static.

don't call uvm_map_check with map unlocked.

_uvm_tree_sanity: fix an assertion.

_uvm_map_sanity: fix a race which causes "stale hint".

Sync with HEAD.

sync with head

on MADV_WILLNEED, start prefetching backing object's pages.

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Include some debug print routines if DEBUGPRINT is defined.  This
way they can be included without having to include DDB.
(arguably all print routines should be behind #ifdef DEBUGPRINT
and options DDB should define that macro, but I'll tackle that later)

Pull up following revision(s) (requested by mrg in ticket #708):
	sys/uvm/uvm_km.c: revision 1.102
	sys/uvm/uvm_km.h: revision 1.18
	sys/uvm/uvm_map.c: revision 1.264
PR port-amd64/32816 amd64 can not load lkms
Change some assertions to partially allow for VM_MAP_IS_KERNEL(map) where
map is outside the range of kernel_map.

Pull up following revision(s) (requested by ad in ticket #354):
	sys/uvm/uvm_fault.c: revision 1.126
	sys/uvm/uvm_map.c: revision 1.268
Move a couple of calls to pmap_update().

Sync with HEAD.

Sync with HEAD.

vm_map_locked_p: add comments

Pull up following revision(s) (requested by bouyer in ticket #211):
	sys/uvm/uvm_km.c: revision 1.103
	sys/uvm/uvm_map.c: revision 1.265
	sys/uvm/uvm_page.c: revision 1.141
It's easier for kernel reserve pages to be consumed because the pagedaemon
serves as less of a barrier these days. Restrict provision of kernel reserve
pages to kmem and one of these cases:
- doing a NOWAIT allocation
- caller is a realtime thread
- caller is a kernel thread
- explicitly requested, for example by the pmap

Move a couple of calls to pmap_update().

kill MALLOC and FREE macros.

replace bitmask_snprintf(9) with snprintb(3)

It's easier for kernel reserve pages to be consumed because the pagedaemon
serves as less of a barrier these days. Restrict provision of kernel reserve
pages to kmem and one of these cases:

- doing a NOWAIT allocation
- caller is a realtime thread
- caller is a kernel thread
- explicitly requested, for example by the pmap

Update haad-dm branch to haad-dm-base2.

PR port-amd64/32816 amd64 can not load lkms

Change some assertions to partially allow for VM_MAP_IS_KERNEL(map) where
map is outside the range of kernel_map.

Sync with HEAD.

Sync with HEAD.

Sync with wrstuden-revivesa-base-2.

Sync with head.

Make uvm_map.? use <sys/rb.h> instead of <sys/tree.h>.  Change the
ambiguous members ownspace/space to gap/maxgap.  Add some evcnt for
evaluation of lookups using tree/list.  Drop threshold of using
tree for lookups from > 30 to > 15.

Bump kernel version to 4.99.71

Sync with head.

Add PMAP_KMPAGE flag for pmap_kenter_pa.  This allows pmaps to know that
the page being entered is being for the kernel memory allocator.  Such pages
should have no references and don't need bookkeeping.

English improvement in comments.

"seems good to me :)" from yamt.

Sync w/ -current. 34 merge conflicts to follow.

sync with head.

Back out previous.

Wrap an expensive check in DIAGNOSTIC.

Sync with HEAD.

Also fix build.

- Switch off the map evcnts by default.
- SAVE_HINT() doesn't need to be atomic.

vm_page: put TAILQ_ENTRY into a union with LIST_ENTRY, so we can use both.

sync with head

Don't needlessly acquire v_interlock.

Sync with HEAD.

Missing cv_destroy().

sync with head.

Disable preemption while swapping pmap.

fix a locking botch.  PR/38415 from Wolfgang Solfrank.

Sync with HEAD.

sync with head.

sync with HEAD

sync with head.

fix "stale map" assertions.  PR/38153 from Sarton O'Brien.

sync with head.

Add some more missing pmap_update()s following pmap_kremove()s.

Sync with HEAD.

sync with head

Sync with HEAD

push pmap_clear_reference calls into pdpolicy code, where reference bits
actually matter.

sync with HEAD

Sync with HEAD

simplify locking and remove vm_map_upgrade/downgrade.
this fixes a deadlock due to read-lock recursion of map->lock.

Sync with HEAD

Merge vmlocking2 to head.

Fix merge error.

- Move remaining map locking functions into uvm_map.c. They depend on proc.h.
- Lock vm_map_kernel::vmk_merged_entries with the map's own lock. There was
  a race where a thread legitimately expects to find cached entries, but can
  find none because they have not been freed yet.

Be more paranoid with vm_map_lock/vm_map_unbusy.

Sync with HEAD.

Sync with head.

Kill vm_map::hint_lock.

Sync with HEAD

sync with head.

add ddb "whatis" command.  inspired from solaris ::whatis dcmd.

uvm_map_checkprot: a kludge for mem(4).

- separate kernel va allocation (kernel_va_arena) from
  in-kernel fault handling (kernel_map).
- add vmem bootstrap code.  vmem doesn't rely on malloc anymore.
- make kmem_alloc interrupt-safe.
- kill kmem_map.  make malloc a wrapper of kmem_alloc.

Sync with HEAD.

sync with head

Pull the vmlocking changes into a new branch.

Sync with HEAD. amd64 Xen support needs testing.

Make this build without LOCKDEBUG (the if statement that uses
LOCKDEBUG_MEM_CHECK).

uvm_map_extract: for UVM_EXTRACT_QREF, mark entries UVM_MAP_NOMERGE.

uvm_unmap1: LOCKDEBUG_MEM_CHECK for kernel_map.

uvm_map_prepare: ignore UVM_FLAG_TRYLOCK for intrsafe maps.
otherwise pool_cache_get(NOWAIT) often fails even if there are plenty of
free memory and causes eg. "no pv entries available" panic on x86.

sync with HEAD

Catch up with 4.0 RC3

sync with head.

Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.

Sync with head.

sync with head.

uvm_map_reserve: don't ignore alignment.  fixes mremap.

sync with head.

Pull up following revision(s) (requested by skrll in ticket #928):
	sys/uvm/uvm_map.c: revision 1.242
Don't restrict the offset when allocating a map entry for in-kernel map -
use UVM_UNKNOWN_OFFSET in the call to uvm_map_prepare.
This fixes a '"panic: malloc: out of space in kmem_map" when it's not
really' testcase of mine, and one reported to me by chuq. This is likely
to fix PR/35587 as well.
Looks/seems fine to me from chuq and yamt. Thanks.

Don't restrict the offset when allocating a map entry for in-kernel map -
use UVM_UNKNOWN_OFFSET in the call to uvm_map_prepare.

This fixes a '"panic: malloc: out of space in kmem_map" when it's not
really' testcase of mine, and one reported to me by chuq. This is likely
to fix PR/35587 as well.

Looks/seems fine to me from chuq and yamt. Thanks.

Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Sync with HEAD.

sync with head.

Sync with HEAD.

Use pool_cache for allocating a few more types of objects.

Sync with HEAD.

Also initialize map->lock for INTRSAFE maps.

uvmspace_free: destroy locks.

Sync with HEAD.

Destroy the map's locks before it's freed.

Merge unobtrusive locking changes from the vmlocking branch.

file uvm_map.c was added on branch matt-mips64 on 2007-07-21 19:21:55 +0000

Merge unobtrusive locking changes from the vmlocking branch.

Sync with head.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

- Fix a (new) bug where vget tries to acquire freed vnodes' interlocks.
- Minor locking fixes.

- Put a per-LWP lock around swapin / swapout.
- Replace use of lockmgr().
- Minor locking fixes and assertions.
- uvm_map.h no longer pulls in proc.h, etc.
- Use kpause where appropriate.

sync with head.