CVS log for src/sys/arch/sparc64/sparc64/pmap.c

Up to [cvs.NetBSD.org] / src / sys / arch / sparc64 / sparc64

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

fix spelling mistakes, mainly in comments and log messages.

Remove unnecessary <sys/malloc.h>.

follow the steps of Andrew Doran (ad) commit and fix more s/loose/lose/ typos.
also s/beyound/beyond/ and few others along the way, mainly in comments.

sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.

sparc64/pmap: Nix trailing whitespace.

sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

	  Thread A			  Thread B
	obj->foo = 42;			obj->baz = 73;
	mumble(&obj->bar);		grumble(&obj->quux);
	/* membar_exit(); */		/* membar_exit(); */
	atomic_dec -- not last		atomic_dec -- last
					/* membar_enter(); */
					KASSERT(invariant(obj->foo,
					    obj->bar));
					free_stuff(obj);

The memory barriers ensure that

	obj->foo = 42;
	mumble(&obj->bar);

in thread A happens before

	KASSERT(invariant(obj->foo, obj->bar));
	free_stuff(obj);

in thread B.  Without them, this ordering is not guaranteed.

So in general it is necessary to do

	membar_exit();
	if (atomic_dec_uint_nv(&obj->refcnt) != 0)
		return;
	membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting.  (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these.  Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that.  It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.

Remove an "#ifdef DIAGNOSTIC", KASSERT() always conumes its expression.

Merge changes from current as of 20200406

pmap_remove_all(): Return a boolean value to indicate the behaviour.  If
true, all mappings have been removed, the pmap is totally cleared out, and
UVM can then avoid doing the work to call pmap_remove() for each map entry.
If false, either nothing has been done, or some helpful arch-specific voodoo
has taken place.

Merge from yamt-pagecache:

- do gang lookup of pages using radixtree.
- remove now unused uvm_object::uo_memq and vm_page::listq.queue.

Sync with HEAD

Synch with HEAD

- move pmap_ctx() macro into pmap.h instead of duplicating it.
- fix description of some mach ddb commands.

Sync with HEAD, resolve a few conflicts

sun4v: fix incorrect calculation of phys addr for the mdesc - now the virtual-devices can (again) be detected (worked previously...)

Pull up following revision(s) (requested by mrg in ticket #593):
	sys/dev/marvell/mvxpsec.c: revision 1.2
	sys/arch/m68k/m68k/pmap_motorola.c: revision 1.70
	sys/opencrypto/crypto.c: revision 1.102
	sys/arch/sparc64/sparc64/pmap.c: revision 1.308
	sys/ufs/chfs/chfs_malloc.c: revision 1.5
	sys/arch/powerpc/oea/pmap.c: revision 1.95
	sys/sys/pool.h: revision 1.80,1.82
	sys/kern/subr_pool.c: revision 1.209-1.216,1.219-1.220
	sys/arch/alpha/alpha/pmap.c: revision 1.262
	sys/kern/uipc_mbuf.c: revision 1.173
	sys/uvm/uvm_fault.c: revision 1.202
	sys/sys/mbuf.h: revision 1.172
	sys/kern/subr_extent.c: revision 1.86
	sys/arch/x86/x86/pmap.c: revision 1.266 (via patch)
	sys/dev/dtv/dtv_scatter.c: revision 1.4

Allow only one pending call to a pool's backing allocator at a time.
Candidate fix for problems with hanging after kva fragmentation related
to PR kern/45718.

Proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2017/10/23/msg022472.html
Tested by bouyer@ on i386.

This makes one small change to the semantics of pool_prime and
pool_setlowat: they may fail with EWOULDBLOCK instead of ENOMEM, if
there is a pending call to the backing allocator in another thread but
we are not actually out of memory.  That is unlikely because nearly
always these are used during initialization, when the pool is not in
use.

Define the new flag too for previous commit.

pool_grow can now fail even when sleeping is ok. Catch this case in pool_get
and retry.

Assert that pool_get failure happens only with PR_NOWAIT.
This would have caught the mistake I made last week leading to null
pointer dereferences all over the place, a mistake which I evidently
poorly scheduled alongside maxv's change to the panic message on x86
for null pointer dereferences.

Since pr_lock is now used to wait for two things now (PR_GROWING and
PR_WANTED) we need to loop for the condition we wanted.
make the KASSERTMSG/panic strings consistent as '%s: [%s], __func__, wchan'
Handle the ERESTART case from pool_grow()

don't pass 0 to the pool flags
Guess pool_cache_get(pc, 0) means PR_WAITOK here.
Earlier on in the same context we use kmem_alloc(sz, KM_SLEEP).

use PR_WAITOK everywhere.
use PR_NOWAIT.

Don't use 0 for PR_NOWAIT

use PR_NOWAIT instead of 0

panic ex nihilo -- PR_NOWAITing for zerot

Add assertions that either PR_WAITOK or PR_NOWAIT are set.
- fix an assert; we can reach there if we are nowait or limitfail.
- when priming the pool and failing with ERESTART, don't decrement the number
  of pages; this avoids the issue of returning an ERESTART when we get to 0,
  and is more correct.
- simplify the pool_grow code, and don't wakeup things if we ENOMEM.

In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them.  This implements the requirement that
pmap_enter(PMAP_CANFAIL) must not fail when replacing an existing
mapping with the first mapping of a new page, which is an unintended
consequence of the changes from the rmind-uvmplock branch in 2011.

The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries.  This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before.  It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.

This somewhat indirectly fixes PR 52706, as well as the failing assertion
about "uvm_page_locked_p(old_pg)".  (but only on x86, various other platforms
will need their own changes to handle this issue.)
In uvm_fault_upper_enter(), if pmap_enter(PMAP_CANFAIL) fails, assert that
the pmap did not leave around a now-stale pmap mapping for an old page.
If such a pmap mapping still existed after we unlocked the vm_map,
the UVM code would not know later that it would need to lock the
lower layer object while calling the pmap to remove or replace that
stale pmap mapping.  See PR 52706 for further details.
hopefully workaround the irregularly "fork fails in init" problem.
if a pool is growing, and the grower is PR_NOWAIT, mark this.
if another caller wants to grow the pool and is also PR_NOWAIT,
busy-wait for the original caller, which should either succeed
or hard-fail fairly quickly.

implement the busy-wait by unlocking and relocking this pools
mutex and returning ERESTART.  other methods (such as having
the caller do this) were significantly more code and this hack
is fairly localised.
ok chs@ riastradh@

Don't release the lock in the PR_NOWAIT allocation. Move flags setting
after the acquiring the mutex. (from Tobias Nygren)
apply the change from arch/x86/x86/pmap.c rev. 1.266 commitid vZRjvmxG7YTHLOfA:

In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them.  If we are replacing an existing mapping,
reuse the pv structure where possible.

This implements the requirement that pmap_enter(PMAP_CANFAIL) must not fail
when replacing an existing mapping with the first mapping of a new page,
which is an unintended consequence of the changes from the rmind-uvmplock
branch in 2011.

The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries.  This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before.  It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.

This somewhat indirectly fixes PR 52706 on the remaining platforms where
this problem existed.

apply the change from arch/x86/x86/pmap.c rev. 1.266 commitid vZRjvmxG7YTHLOfA:

In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them.  If we are replacing an existing mapping,
reuse the pv structure where possible.

This implements the requirement that pmap_enter(PMAP_CANFAIL) must not fail
when replacing an existing mapping with the first mapping of a new page,
which is an unintended consequence of the changes from the rmind-uvmplock
branch in 2011.

The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries.  This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before.  It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.

This somewhat indirectly fixes PR 52706 on the remaining platforms where
this problem existed.

update from HEAD

Sync with HEAD

Sync with HEAD

Sync with HEAD

sun4v: Added handling of trap type 0x034 (address alignment error) + fixed typo mmfsa -> mmufsa. Verified for both sun4u and sun4v using qemu.

Sync with HEAD

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Rename the "flush" macro to "sparc_flush_icache", the former is way too
generic for a macro name.

switch all ports to use uvm_init.c:uvm_md_init()

uvm_setpagesize() is now subsumed within this funciton.

Sync with HEAD

Sync with HEAD

add plumbing to support bus_space_mmap() with:
- write combining allowed via BUS_SPACE_MAP_PREFETCHABLE
- byte order translation via BUS_SPACE_MAP_LITTLE

KNF. Remove extra spaces. No functional change.

Sync with HEAD

Move the SMP-disabled hack for SUN4V a bit earlier

Sync with HEAD

sun4v: extend dmmu_set_secondary_context() to handle both sun4u and sun4v. ok martin@

Sync with HEAD (as of 26th Dec)

Left-shift of negative values is UB.

Sync with HEAD

Fix build w/o options SUN4V.

mdesc_get_len() and mdesc_init() are used outside of #ifdef SUN4V,
and eliminated by optimization if SUN4V isn't defined.

Use <machine/*.h> instead of <sparc64/*.h> for consistency.

On SUN4V make the mdesc data available early in pmap_bootstrap, so we
can set up the uvm page colors correctly.
Based on code from palle@.

Pull up following revision(s) (requested by martin in ticket #842):
	sys/arch/sparc/stand/ofwboot/loadfile_machdep.c: revision 1.14
	sys/arch/sparc/stand/ofwboot/version: revision 1.21
	sys/arch/sparc64/include/cpu.h: revision 1.115
	sys/arch/sparc64/sparc64/autoconf.c: revision 1.203
	sys/arch/sparc64/sparc64/mp_subr.S: revision 1.9 via patch
	sys/arch/sparc64/sparc64/pmap.c: revisions 1.295, 1.295 via patch
Fix available length calculation in kvamap_extract when reusing existing
mappings.
--
Bump version now that we can load kernels with sizeof(.data)+sizeof(.bss)> 4 MB
--
Do not hardcode the assumption that .data and .bss fit together in a single
4 MB page. This allows booting kernels with options USB_DEBUG again.
--
Slightly optimize the ITLB usage on secondary processors

Slightly optimize the ITLB usage on secondary processors

Do not hardcode the assumption that .data and .bss fit together in a single
4 MB page. This allows booting kernels with options USB_DEBUG again.

Sync with HEAD

make some DEBUG code in pmap_clear_reference() UP-only

Sync with HEAD

sun4v: Ensure that the W (Write) bit is not present in the TTEs unless for the data mapping. This way the permanent mappings for the primary cpu done by ofwboot are the same as the permanent mappings for the secondary cpus done by the kernel. This should fix an issue when booting the kernel on a T2 based system (T5120).

sun4v: Removed a bunch of ifdef SUN4V since the CPU_ISSUN4V already has the necessary logic. Noticed by mrg@

sun4v: ensure that the X (eXecute) bit is set in the TTE

sun4v: All cpus must be setup with a TSB descriptor, so pmap_setup_tsb_sun4v() must take a pointer to the TSB descriptor

Rebase to HEAD as of a few days ago.

Rebase.

Increase maximum MSGBUFSIZE to 248k.
While here, remove 2 sparc(32)-only entries (from mrg@).

sun4v: note missing handling in pmap_calculate_colors()

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Removed duplicate call to prom_map_phys()

sync with head

Coding style fixes (whitespace) - no functional changes

sun4v: trap table setup - currently populated with dummy entries which will be properly implemented later - parts from OpenBSD - OK martin@

Make 32-bit kernel buildable again.

a) make the hunt through the cpus list more generic b) sun4v: ensure that the interrupt stack is mapped permanently and the TSB is setup properly - parts from OpenBSD. OK martin@

Updates to pmap_bootstrap() for sun4v - parts from OpenBSD - OK martin@

Disallow MAP_FIXED mmap's in the sparc64 VA hole (XXX - need to deal with
different variants of the hole for different CPU types)

resync with head

Update the DEBUG section at the tail of pmap_clear_modify().

Nothing prevents page modification after the modify bit was
cleared but before the DEBUG section checks pmap_is_modified(),
so remove this check.

For the same reason "modified" and "changed" may differ, so only
check "modified" implies "changed" here.

Ok: Matthew Green <mrg@netbsd.org>

sync with head

Rearange VA layout for 64 bit kernels, as suggested by Lars Heidieker:
move the dynamic kernel memory past 4GB (basically unlimiting it) and
use the now freed address range past kernel data upto 2GB for modules.

sync with head

sync to latest -current.

reduce the #ifdef DEBUG usage by using the optimiser.

take the pmap_lock in pmap_protect().

hopefully this will avoid a problem in pmap_clear_reference()
detecting a reference remaining at the end.

when we remove most/all pmap_lock uses we probably need to remove
this check entirely, but for now this seems better.

merge to -current.

Provide a module_map (16 MB for now) to load modules close to kernel text
and data.

Fixes PR port-sparc64/45895. Ok: releng

adapt sparc64.  compile tested.

rename sparc64 BLOCK_SIZE and BLOCK_ALIGN to have SPARC64_ prefixes.
for the assembler files, define the old names to the new names
since using the new names cause ugliness due to longer identifer
names, and reduces churn.

fixes build issues in dtv and vaguely makes <machine/psl.h> slightly
less name-space invasive.

Catchup with rmind-uvmplock merge.

Add fast softint(9) support for sparc64.

Reviewed on port-sparc64.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

sync with head

Sync with HEAD.

re-implement blast_dcache() and blast_icache() via function pointers so that
we can set these to functions that simply return on sun4us/sun4v systems.

also include some work-in-progress #if 0'ed code to do the same for the
dcache_flush*() functions.


(some of these could probably move into the cpuinfo except that sparc64
systems don't really ever work with mismatches AFAICT.)

sync with head

Implement sharing of vnode_t::v_interlock amongst vnodes:
- Lock is shared amongst UVM objects using uvm_obj_setlock() or getnewvnode().
- Adjust vnode cache to handle unsharing, add VI_LOCKSHARE flag for that.
- Use sharing in tmpfs and layerfs for underlying object.
- Simplify locking in ubc_fault().
- Sprinkle some asserts.

Discussed with ad@.

first steps towards fujitsu SPARC64 support:

- use cputyp when writing the kernel core

don't take pmap_lock in a few places.  there's a bunch more work
we can do in here, but i've run with this change for ages.

sync with head

Sync with HEAD

blast_dcache() does the right thing for MP or !MP, so just use it.

Put VM_PAGE_TO_MD() definition in one place.  No functional changes.

Sync with HEAD.

Oops, fix build !DIAGNOSTICS again.

Fix DIAGNOSTIC build.  Pointed out by Juergen Hannken-Illjes.

We already have a flag PMAP_NOCACHE.  s/PMAP_UNMANAGED/PMAN_NOCACHE/.
Pointed out by Chuck Silvers, thanks.

Use VM_PAGE_TO_MD() to locate struct vm_page_md.  No functional
changes.

Implement pmap_physload_device(9) to replace xmd(4) MD backend.
Implement pmap_mmap(9) and use it from mem(4) and xmd(4).

Sync with HEAD.

sync with head.

Add vm page flag PG_MARKER and use it to tag dummy marker pages
in genfs_do_putpages() and uao_put().
Use 'v_uobj.uo_npages' to check for an empty memq.
Put some assertions where these marker pages may not appear.

Ok: YAMAMOTO Takashi <yamt@netbsd.org>

sync with head

move the mutex_init() for ci_ctx_lock inside cpu_attach() so that
we don't try to mutex_init() before lockdebug is available.

fixes LOCKDEBUG crashes at startup (for USII systems.)

Sync with HEAD.

Support PMAP_UNMANAGED in some pmaps.

(Others should be converted eventually, but no problem while managed
device page is not used.)

sync to netbsd-5

fix a KASSERT() in the previous.  add a couple of comments.

- add a kmutex_t ci_ctx_lock to struct cpu_info, and initialise it in
  cpu_pmap_init() and replace pmap_ctx_lock usage with this new ci_ctx_lock

- replace smp_dcache_flush_page_all() with smp_dcache_flush_page_cpuset(),
  that flushes only on the set of CPUs not, everyone.  add new
  dcache_flush_page_cpuset() to flush this page from the D$ only
  on the specified set of CPUs.

- add a cpuset to pmap_free_page() and use it when freeing PTE pages
  when a pmap is destroyed

- introduce pmap_free_page_noflush(), and use it when we allocated a
  page for PTEs but didn't use it and don't need to flush it

- don't bother with pmap_lock in pmap_extract(), the only potential
  issue is pseg_get() which is already safe


tested on sb2000, sb2500 and ultra80 with a bunch of various heavy
workloads, and seems to give a clear 1-2% speed up for high-forking /
short lived processes, such as ./configure.

since pmap_activate/pmap_deactivate need to not block, but the rest of
pmap is OK, add a new IPL_VM pmap_ctx_lock that is held while allocating
or freeing a context, and revert pmap_lock back to IPL_NONE.

i considered a per-cpu context lock, but that would require locking
each active CPUs lock in turn in pmap_remove_all() and pmap_destroy().


thanks to ad for explaining how these parts interact.

convert pmap_lock into an IPL_VM lock, and avoid calling
KASSERT(!mutex_owned(&pmap_lock)) since that's no longer valid.

this fixes hangs on 4-core machines like the U80.

tested on U80, SB2500 and SB2000.


XXX: my initial attempts to replace pseg_lock with this new equivalent
XXX: lock hang in pmap_bootstrap it seems..  but this should be able
XXX: to go away now.

Apply patch (requested by mrg in ticket #1343):
- flush the dcache around pmap_{zero,copy}_page()
- convert all blast_dcache() / dcache_flush_page() calls to
  properly handle flushes in all cpus as necessary

Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc().  It allows us to share the locks among UVM objects.

sync with head

XXX: workaround we'd like to remove when pmap / uvm locking is cleaned up:

- rename pseg_get() and pseg_set() to pseg_get_real() and pseg_set_real().
- if USE_LOCKSAFE_PSEG_GETSET is defined, which it current is by default,
  define pseg_[gs]et() in terms of functions that take a new pseg_lock
  mutex at IPL_VM while calling into the real functions.

this seems to avoid the pseg_set() crashes we've seen:
  1 - spare needed, when pseg_get() just worked for this pmap
  2 - the 2rd ldxa via ASI_PHYS_CACHED in pseg_set() loads garbage
      into %o4, and causes the 3rd ldxa to fault

blast_dcache() SMP friendly:

- blast_dcache() becomes sp_blast_dcache(dcache_size, dcache_line_size)
- new smp_blast_dcache(sparc64_cpuset_t) that blasts the D$ on this cpuset
- sparc64_ipi_blast_dcache() to support the above
- in pmap_remove_all(), when freeing mmu contexts for this pmap, mark
  the set of cpus to blast the d$ on as well and convert the
  blast_dcache() call into smp_blast_dcache() on the cpus who ran this
  pmap, or, sp_blast_dcache(dcache_size, dcache_line_size)
- convert the remaining blast_dcache() in machdep.c to sp_blast_dcache()
- in pmap_destroy()/pmap_remove_all() take the pmap_lock() always since
  we assert it is held always.

with these changes, NFS builds on the U60 seem to be stable now, and
the USIII machines also can often complete a single build.sh run now,
diskful or diskless.


reviewed by mlelstv and partially by martin, tested by martin and myself,
with some ideas from chuq as well.

clean up a bunch of MULTIPROCESSOR:

- always include ci_ipi_evcnt[] in cpuinfo
- #define sparc_ncpus 1 for !MULTIPROCESSOR
- make struct pmap::pm_list an pm_ctx always be an array, and simplify
  several functions and lookups to always be the same

tested on U60 and SB2500 before and after with one and two cpus in an
MP kernel, and UP kernels, and i can't find anything besides noise for
benchmark issues.  (infact, i can't really tell the difference between
GENERIC and GENERIC.MP on these systems...)

avoid a build error in the previous for !DEBUG kernels.

- in _bus_dmamap_unload(), pmap_page_protect() and pmap_clear_reference()
  switch the dcache_flush_page() into a dcache_flush_page_all()
- in both pmap_kremove()/pmap_remove(), remove the blast_dcache() call
  and replace it with dcache_flush_page_all()
- in pmap_get_page() [used to allocate PTP's], always call pmap_zero_page(pa)
- flush the dcache of the dst page in pmap_{copy,zero}_page() by redirecting
  throught a C function that calls the (renamed) asm.  the old asm code had a
  comment about needing to do this...
- add a couple of membar #Sync's that the USIII manual recommends


based on discussions with chuq@, skrll@ and martin@.


these help my SB2000 / SB2500 with both disk / nfs builds and other tasks,
sometimes lasting for several hours before failing or asserting.

Use VM_PAGE_TO_MD().  Only compile tested.

check for TLB_V bit for valid tte entries, not against zero.
KASSERT() the 3rd call to pseg_get() in pmap_page_cache() like the other 2.

when handing back page table pages to UVM, flush them from the dcache
on all CPUs.

idea from skrll@, and makes my disk-based SB2000 stable.

tested on U60, SB2000 and SB2500.

- re-introduce sp_tlb_flush_pte() and sp_tlb_flush_all() that call
  the right USI/USIII version.
- define tlb_flush_pte() in terms of sp_tlb_flush_pte() for the
  !MULTIPROCESSOR case

this centralises some CPU_IS_USIII_UP() tests.

in pmap_growkernel(), make sure we only mutex_exit() if we mutex_enter()ed.

Avoid taking a (not yet) initialized mutex early during bootstrap.

- split sp_tlb_flush_pte() and switchtoctx() into sp_tlb_flush_pte_us()/
  sp_tlb_flush_pte_usiii() and switchtoctx_us()/switchtoctx_usiii() and
  implement the latter while i'm here.  it works ... sometimes i think,
  but also sometimes panics/hangs.
- fix a comment in sparc64_ipi_flush_pte_usiii()

- sp_tlb_flush_all() becomes sp_tlb_flush_all_us() and sp_tlb_flush_all_usiii()
- sparc64_ipi_flush_pte() becomes sparc64_ipi_flush_pte_us() and
  sparc64_ipi_flush_pte_usiii()
- add some commented out code to disable interrupts and raise the traplevel
  in sparc64_ipi_flush_pte_usiii()
- cache_flush_phys() was missing a little of code in the cheetah case

- add a PDB_CTX_FLUSHALL debug type to pmap.c, and also log the cpu_number()
  in a bunch more cases

- make sparc64_ipi_halt_thiscpu() and sparc64_ipi_pause_thiscpu() return void,
  their callers never checked anyway.

- remove prototypes for sparc64_ipi_flush_ctx() and sparc64_ipi_flush_all(),
  there are no such functions

s/cpu_switch/cpu_switchto/ in comment.

from openbsd:
>Don't use u_int to store a 64-bit address. Found with help from miod.
>Makes a Sun Fire V490 boot without spending ages in bus_dmamem_alloc(9).

- Use uvm_lwp_setuarea() instead of directly setting address to lwp_t::l_addr.
- Replace most remaining uses of l_addr with uvm_lwp_getuarea() or lwp_getpcb().
- Amend assembly in ports where it accesses PCB via struct user.
- Rename L_ADDR to L_PCB in few places.  Reduce sys/user.h inclusions.

Kill proc0paddr.  Use lwp0.l_addr instead.

Add a flags argument to pmap_kenter_pa(9).
Patch showed on tech-kern@ http://mail-index.netbsd.org/tech-kern/2009/11/04/msg006434.html
No objections.

kill extra whitespaces
reviewed by tsutsui@

Use trunc_page or round_page where appropriate.
No functional changes intended, and generate same binaries.

Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code.  Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

sync with head.

Include <sys/exec_aout.h> explicitly instead of relying on <sys/exec.h> to
do it for us.

Fix misplaced #endif, from Henning Petersen in PR port-sparc64/41856

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

change pmap flags argument from int to u_int.
discussed with christos@ on source-changes-d@

bcopy -> memcpy

Ansify function definitions w/o arguments. Generated with sed.

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

Sync with HEAD.

Sync with HEAD.

Update haad-dm branch to haad-dm-base2.

Make kernel_pmap_ptr a const.  Requested by steve_martin.

Backout previous - Matt, you probably did a partial update and didn't
pick up the sys/uvm changes.

fix sparc64 build: provide pmap_kernel(), re-provide pmap_t.

fixes builds for my U60.

Get rid of get_maxctx() - it doesn't work on newer cpus and we will have
to find a better way for (some) sun4v machines - all sun4u use 13 bit
contexts anyway.

Make pmap_kernel() a MI macro for struct pmap *kernel_pmap_ptr,
which is now the "API" provided by the pmap module.  pmap_kernel()
remains as the syntactic sugar.

Bonus cosmetics round: move all the pmap_t pointer typedefs into
uvm_pmap.h.

Thanks to Greg Oster for providing cpu muscle for doing test builds.

Pull up following revision(s) (requested by spz in ticket #107):
	sys/arch/sparc64/sparc64/pmap.c: revision 1.226
pmap_is_referenced_locked() is relevant for non-MULTIPROCESSOR systems too,
says martin@

pmap_is_referenced_locked() is relevant for non-MULTIPROCESSOR systems too,
says martin@

Sync with HEAD.

- fix corner case bugs.
- avoid unnecessary write backs.
- remove meaningless local variable usages.

Avoid local include.

Sync with HEAD.

Conform previous commit to KNF, pointed out by ad@.

Sync with HEAD.

pmap_cache: use BLOCK_SIZE as alignment.
pmap_pv_cache: use large size groups (PR_LARGECACHE) like x86.

Sync with HEAD.

pmap_is_modified() and pmap_is_referenced(): when we don't find a cached
positive answer in the list head, make sure to aquire pmap_lock before
traversing the pv list (and update the cache in head).
Fixes a diagnostic debugger call reported by Havard Eidnes.

Sync with wrstuden-revivesa-base-2.

Sync with the following revisions (requested by skrll in ticket #1196):
	gnu/dist/gdb						removed
	gnu/usr.bin/gdb53					removed
	distrib/cats/instkernel/Makefile			1.14.6.1
	gnu/dist/gdb6/bfd/config.bfd				1.3.6.1
	gnu/dist/gdb6/bfd/elfxx-sparc.c				1.1.1.2.6.1
	gnu/dist/gdb6/bfd/elfxx-sparc.h				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/Makefile.in				1.2.2.1.2.2
	gnu/dist/gdb6/gdb/alpha-tdep.c				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/alpha-tdep.h				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/alphabsd-nat.c			1.1.1.2.6.2
	gnu/dist/gdb6/gdb/alphabsd-nat.h			1.1.2.1
	gnu/dist/gdb6/gdb/alphabsd-tdep.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/alphabsd-tdep.h			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/alphanbsd-nat.c			1.1.2.1
	gnu/dist/gdb6/gdb/alphanbsd-tdep.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/amd64-nat.c				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/amd64bsd-nat.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/amd64nbsd-nat.c			1.1.1.2.6.3
	gnu/dist/gdb6/gdb/amd64nbsd-tdep.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/arm-tdep.h				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/armbsd-tdep.c				1.1.2.1
	gnu/dist/gdb6/gdb/armnbsd-nat.c				1.1.1.2.6.2
	gnu/dist/gdb6/gdb/armnbsd-tdep.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/configure				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/configure.ac				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/i386bsd-nat.c				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/i386nbsd-tdep.c			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/m68kbsd-nat.c				1.1.1.2.6.2
	gnu/dist/gdb6/gdb/mipsnbsd-nat.c			1.1.1.2.6.2
	gnu/dist/gdb6/gdb/nbsd-thread.c				1.1.2.3
	gnu/dist/gdb6/gdb/ppcnbsd-nat.c				1.1.1.2.6.2
	gnu/dist/gdb6/gdb/ppcnbsd-tdep.c			1.3.6.1
	gnu/dist/gdb6/gdb/sh-tdep.c				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/shnbsd-nat.c				1.1.1.2.6.3
	gnu/dist/gdb6/gdb/shnbsd-tdep.c				1.1.1.2.6.4
	gnu/dist/gdb6/gdb/shnbsd-tdep.h				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/sparc-nat.c				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/sparc64nbsd-nat.c			1.1.1.2.6.2
	gnu/dist/gdb6/gdb/sparcnbsd-nat.c			1.1.1.2.6.2
	gnu/dist/gdb6/gdb/tramp-frame.h				1.1.1.2.6.1
	gnu/dist/gdb6/gdb/vaxbsd-nat.c				1.1.1.2.6.2
	gnu/dist/gdb6/gdb/config/alpha/nbsd.mh			1.1.1.2.6.1
	gnu/dist/gdb6/gdb/config/arm/nbsd.mt			1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/arm/nbsdelf.mh			1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/i386/nbsd64.mh			1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/m68k/nbsdelf.mh		1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/mips/nbsd.mh			1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/powerpc/nbsd.mh		1.1.1.2.6.1
	gnu/dist/gdb6/gdb/config/sh/nbsd.mh			1.1.1.1.6.2
	gnu/dist/gdb6/gdb/config/sh/tm-nbsd.h			1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/sparc/nbsd64.mh		1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/sparc/nbsdelf.mh		1.1.1.1.6.1
	gnu/dist/gdb6/gdb/config/vax/nbsdelf.mh			1.1.1.1.6.1
	gnu/dist/gdb6/opcodes/configure				1.1.1.2.6.1
	gnu/dist/gdb6/opcodes/configure.in			1.1.1.2.6.1
	gnu/usr.bin/Makefile					1.126.4.1
	gnu/usr.bin/gdb6/arch/alpha/config.h			1.3.4.1
	gnu/usr.bin/gdb6/arch/alpha/defs.mk			1.2.6.1
	gnu/usr.bin/gdb6/arch/alpha/init.c			1.2.6.1
	gnu/usr.bin/gdb6/arch/alpha/nm.h			1.2.6.1
	gnu/usr.bin/gdb6/arch/arm/defs.mk			1.2.6.2
	gnu/usr.bin/gdb6/arch/arm/init.c			1.1.6.1
	gnu/usr.bin/gdb6/arch/armeb/config.h			1.1.6.2
	gnu/usr.bin/gdb6/arch/armeb/defs.mk			1.1.6.3
	gnu/usr.bin/gdb6/arch/armeb/init.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/armeb/tm.h			1.1.6.2
	gnu/usr.bin/gdb6/arch/armeb/version.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/i386/defs.mk			1.4.4.1
	gnu/usr.bin/gdb6/arch/i386/init.c			1.3.6.1
	gnu/usr.bin/gdb6/arch/m68000/config.h			1.1.6.2
	gnu/usr.bin/gdb6/arch/m68000/defs.mk			1.1.6.2
	gnu/usr.bin/gdb6/arch/m68000/init.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/m68000/tm.h			1.1.6.2
	gnu/usr.bin/gdb6/arch/m68000/version.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/m68k/defs.mk			1.1.4.1
	gnu/usr.bin/gdb6/arch/m68k/init.c			1.1.4.1
	gnu/usr.bin/gdb6/arch/mipseb/config.h			1.3.4.1
	gnu/usr.bin/gdb6/arch/mipseb/defs.mk			1.2.6.2
	gnu/usr.bin/gdb6/arch/mipseb/init.c			1.2.6.2
	gnu/usr.bin/gdb6/arch/mipsel/config.h			1.2.6.3
	gnu/usr.bin/gdb6/arch/mipsel/defs.mk			1.2.6.3
	gnu/usr.bin/gdb6/arch/mipsel/init.c			1.2.6.3
	gnu/usr.bin/gdb6/arch/mipsel/tm.h			1.2.6.2
	gnu/usr.bin/gdb6/arch/mipsel/version.c			1.2.6.2
	gnu/usr.bin/gdb6/arch/powerpc/defs.mk			1.3.6.1
	gnu/usr.bin/gdb6/arch/powerpc/init.c			1.3.6.1
	gnu/usr.bin/gdb6/arch/sh3eb/config.h			1.2.2.2
	gnu/usr.bin/gdb6/arch/sh3eb/defs.mk			1.2.8.3
	gnu/usr.bin/gdb6/arch/sh3eb/init.c			1.1.8.3
	gnu/usr.bin/gdb6/arch/sh3eb/nm.h			1.1.8.2
	gnu/usr.bin/gdb6/arch/sh3eb/tm.h			1.1.8.2
	gnu/usr.bin/gdb6/arch/sh3eb/version.c			1.1.8.2
	gnu/usr.bin/gdb6/arch/sh3el/config.h			1.2.2.2
	gnu/usr.bin/gdb6/arch/sh3el/defs.mk			1.2.8.3
	gnu/usr.bin/gdb6/arch/sh3el/init.c			1.1.8.3
	gnu/usr.bin/gdb6/arch/sh3el/nm.h			1.1.8.2
	gnu/usr.bin/gdb6/arch/sh3el/tm.h			1.1.8.2
	gnu/usr.bin/gdb6/arch/sh3el/version.c			1.1.8.2
	gnu/usr.bin/gdb6/arch/sparc/defs.mk			1.2.6.1
	gnu/usr.bin/gdb6/arch/sparc/init.c			1.1.6.1
	gnu/usr.bin/gdb6/arch/sparc64/defs.mk			1.2.6.1
	gnu/usr.bin/gdb6/arch/sparc64/init.c			1.1.6.1
	gnu/usr.bin/gdb6/arch/vax/config.h			1.1.6.2
	gnu/usr.bin/gdb6/arch/vax/defs.mk			1.1.6.2
	gnu/usr.bin/gdb6/arch/vax/init.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/vax/tm.h				1.1.6.2
	gnu/usr.bin/gdb6/arch/vax/version.c			1.1.6.2
	gnu/usr.bin/gdb6/arch/x86_64/defs.mk			1.2.6.1
	gnu/usr.bin/gdb6/arch/x86_64/init.c			1.1.6.1
	gnu/usr.bin/gdb6/bfd/arch/armeb/bfd.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/armeb/bfdver.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/armeb/config.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/armeb/defs.mk			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/m68000/bfd.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/m68000/bfdver.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/m68000/config.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/m68000/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/mipsel/bfd.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/mipsel/bfdver.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/mipsel/config.h		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/mipsel/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/sh3eb/bfd.h			1.1.8.3
	gnu/usr.bin/gdb6/bfd/arch/sh3eb/bfdver.h		1.1.8.2
	gnu/usr.bin/gdb6/bfd/arch/sh3eb/config.h		1.1.8.2
	gnu/usr.bin/gdb6/bfd/arch/sh3eb/defs.mk			1.1.8.3
	gnu/usr.bin/gdb6/bfd/arch/sh3el/bfd.h			1.1.8.3
	gnu/usr.bin/gdb6/bfd/arch/sh3el/bfdver.h		1.1.8.2
	gnu/usr.bin/gdb6/bfd/arch/sh3el/config.h		1.1.8.2
	gnu/usr.bin/gdb6/bfd/arch/sh3el/defs.mk			1.1.8.3
	gnu/usr.bin/gdb6/bfd/arch/vax/bfd.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/vax/bfdver.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/vax/config.h			1.1.6.2
	gnu/usr.bin/gdb6/bfd/arch/vax/defs.mk			1.1.6.2
	gnu/usr.bin/gdb6/gdb/Makefile				1.5.2.1.2.2
	gnu/usr.bin/gdb6/gdbtui/Makefile			1.2.6.1
	gnu/usr.bin/gdb6/libiberty/arch/armeb/config.h		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/armeb/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/m68000/config.h		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/m68000/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/mipsel/config.h		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/mipsel/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/sh3eb/config.h		1.1.8.2
	gnu/usr.bin/gdb6/libiberty/arch/sh3eb/defs.mk		1.1.8.2
	gnu/usr.bin/gdb6/libiberty/arch/sh3el/config.h		1.1.8.2
	gnu/usr.bin/gdb6/libiberty/arch/sh3el/defs.mk		1.1.8.2
	gnu/usr.bin/gdb6/libiberty/arch/vax/config.h		1.1.6.2
	gnu/usr.bin/gdb6/libiberty/arch/vax/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/armeb/config.h		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/armeb/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/m68000/config.h		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/m68000/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/mipsel/config.h		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/mipsel/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/sh3eb/config.h		1.1.8.2
	gnu/usr.bin/gdb6/opcodes/arch/sh3eb/defs.mk		1.1.8.3
	gnu/usr.bin/gdb6/opcodes/arch/sh3el/config.h		1.1.8.2
	gnu/usr.bin/gdb6/opcodes/arch/sh3el/defs.mk		1.1.8.3
	gnu/usr.bin/gdb6/opcodes/arch/vax/config.h		1.1.6.2
	gnu/usr.bin/gdb6/opcodes/arch/vax/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/armeb/config.h		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/armeb/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/m68000/config.h		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/m68000/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/mipsel/config.h		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/mipsel/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/sh3eb/config.h		1.1.8.2
	gnu/usr.bin/gdb6/readline/arch/sh3eb/defs.mk		1.1.8.2
	gnu/usr.bin/gdb6/readline/arch/sh3el/config.h		1.1.8.2
	gnu/usr.bin/gdb6/readline/arch/sh3el/defs.mk		1.1.8.2
	gnu/usr.bin/gdb6/readline/arch/vax/config.h		1.1.6.2
	gnu/usr.bin/gdb6/readline/arch/vax/defs.mk		1.1.6.2
	gnu/usr.bin/gdb6/sim/arch/mipseb/cconfig.h		1.1.2.1
	gnu/usr.bin/gdb6/sim/arch/mipseb/config.h		1.1.2.1
	gnu/usr.bin/gdb6/sim/arch/mipseb/defs.mk		1.1.2.1
	gnu/usr.bin/gdb6/sim/arch/mipsel/cconfig.h		1.1.2.1
	gnu/usr.bin/gdb6/sim/arch/mipsel/config.h		1.1.2.1
	gnu/usr.bin/gdb6/sim/arch/mipsel/defs.mk		1.1.2.1
	lib/libkvm/kvm_sparc64.c				1.10.18.2
	lib/libpthread/pthread.c				1.48.6.4
	lib/libpthread/pthread_barrier.c			1.6.18.1
	lib/libpthread/pthread_cond.c				1.18.12.2
	lib/libpthread/pthread_debug.h				1.8.18.1
	lib/libpthread/pthread_int.h				1.34.4.5
	lib/libpthread/pthread_lock.c				1.14.6.1
	lib/libpthread/pthread_mutex.c				1.22.4.2
	lib/libpthread/pthread_run.c				1.18.12.4
	lib/libpthread/pthread_rwlock.c				1.13.6.2
	lib/libpthread/pthread_sa.c				1.37.6.5
	lib/libpthread/pthread_sig.c				1.47.4.8
	lib/libpthread/pthread_sleep.c				1.7.6.2
	lib/libpthread/sem.c					1.9.6.2
	lib/libpthread/arch/sh3/pthread_md.h			1.3.6.1
	regress/lib/libpthread/resolv/Makefile			1.1.12.1
	regress/lib/libpthread/sigrunning/Makefile		1.1.2.1
	regress/lib/libpthread/sigrunning/sigrunning.c		1.1.2.1
	share/mk/bsd.own.mk					1.489.4.3
	sys/arch/amd64/amd64/locore.S				1.18.14.1
	sys/arch/amd64/amd64/machdep.c				1.44.2.3.2.1
	sys/arch/amd64/conf/kern.ldscript			1.1.70.1
	sys/arch/cats/conf/Makefile.cats.inc			1.17.30.1
	sys/arch/shark/conf/Makefile.shark.inc			1.6.30.1
	sys/arch/sparc64/conf/kern.ldscript			1.7.26.2
	sys/arch/sparc64/conf/kern32.ldscript			1.6.26.2
	sys/arch/sparc64/include/kcore.h			1.4.92.2
	sys/arch/sparc64/sparc64/locore.s			1.232.4.4
	sys/arch/sparc64/sparc64/machdep.c			1.193.4.3
	sys/arch/sparc64/sparc64/pmap.c				1.184.2.1.2.4
	sys/conf/newvers.sh					1.42.26.2
	sys/kern/kern_sa.c					1.87.4.11
	sys/kern/kern_synch.c					1.173.4.2
	sys/sys/savar.h						1.20.10.2
	tools/gdb/Makefile					1.9.4.1
	tools/gdb/mknative-gdb					1.1.6.1

pullup the wrstuden-fixsa CVS branch to netbsd-4:
toolchain/35540 - GDB 6 support for pthreads.
port-sparc64/37534 - ktrace firefox gives
    kernel trap 30: data access expection
GDB changes:
- delete gdb53
- enable gdb6 on all architectures
- add support for amd64 crash dumps
- add support for sparc64 crash dumps
- add support for /proc pid to executable filename for all archs
- enable thread support for all architectures
- add a note section to kernels to all platforms
- support detection/unwinding of signals for most architectures.
- Fix PTHREAD_UCONTEXT_TO_REG / PTHREAD_REG_TO_UCONTEXT on sh3.
- Apply fix from binutils-current so that sparc gdb can be cross built
  on a 64bit host.
SA/pthread changes:
Pre-allocate memory needed for event delivery. Eliminates dropped
interrupts under load.
Deliver intra-process signals to running threads
Eliminate some deadlock scenarios
Fix intra-process signal delivery when delivering to a thread waiting
for signals. Makes afs work again!

Turn a printf into a DEB

Sync with HEAD.

Sync w/ -current. 34 merge conflicts to follow.

Sync with head.

Use UVM_OBJ_INIT/DESTROY.

sync with head.

Sync with HEAD.

Also fix build.

vm_page: put TAILQ_ENTRY into a union with LIST_ENTRY, so we can use both.

Sync with HEAD.

Don't allocate or free pv_entry_t with pmap_lock held. Requested by martin@.
Untested.

Sync with HEAD.

Now that we have per-cpu-contexts, checking the context on the local cpu
to find out if a pmap is on some mmu does not realy work any more.

Make the debug message that triggered in PR 38297 slightly more verbose

sync with head.

sync with head.

sync with HEAD

Add more assertion.

Lock also on freeing own context, and add assertion.

sync with head.

Complete per-CPU TSBs and MMU contexts changes.

- make interrupt pending list per-CPU.
- make tickintr() MP-safe.
- remove unused port-sparc derived interrupt code.

Ok by martin@.

Make TSBs and MMU contexts per-cpu.

Sync with HEAD.

Pull up following revision(s) (requested by martin in ticket #983):
	sys/arch/sparc64/include/kcore.h: revision 1.5
	lib/libkvm/kvm_sparc64.c: revision 1.11
	sys/arch/sparc64/sparc64/pmap.c: revision 1.201
Extend the sparc64 cpu kcore segment so that it records all kernel mappings
done via locked 4 MB pages. The old format could only record physically
continous kernel text mappings.
For compatibility reasons, just extend the structure and fill in all the
old fields as well.
Use the new sparc64 kcore cpu segment extensions to properly map VA/PAs
on kernels with more then 4 MB text/rodata segment.

sync with head

Sync with HEAD

Backported version of martin's changes to:

Extend the MD kernel core header, so that crashdumps record enough
information to recover all per-cpu mappings. This provides access to
the interrupt stack for example.

Extend the MD kernel core header, so that crashdumps record enough
information to recover all per-cpu mappings. This provides access to
the interrupt stack for example.

sync with HEAD

Sync with HEAD

pmap_destroy: cast return of atomic_dec_uint_nv to (int) to avoid leaking
pmaps. The refcount can be zero due to pmap_remove_all. Problem found by
tnn@.

Remove superflouse extern decls.

Move the mutex_init of pmap_lock even later into pmap_init().
This makes LOCKDEBUG kernels work again.

We can not mutex_init() before setting up the per-CPU mappings needed
by curcpu() - a LOCKDEBUG kernel will die otherwise.

Don't need sys/lock.h

Sync with HEAD

Merge vmlocking2 to head.

Apply changes from ticket #983.

Modified Files:
	src/sys/arch/sparc64/include: kcore.h
	src/sys/arch/sparc64/sparc64: pmap.c

Log Message:
Extend the sparc64 cpu kcore segment so that it records all kernel mappings
done via locked 4 MB pages. The old format could only record physically
continous kernel text mappings.
For compatibility reasons, just extend the structure and fill in all the
old fields as well.

Modified Files:
	src/lib/libkvm: kvm_sparc64.c

Log Message:
Use the new sparc64 kcore cpu segment extensions to properly map VA/PAs
on kernels with more then 4 MB text/rodata segment.

Sync with head.

sync with head.

Checkpoint work in progress.

Provide cpu_intr_p(), at least for non-MULTIPROCESSOR kernels.
Based on suggestions by Andrew Doran.

Sync with HEAD.

sync with head.

Sync with HEAD

sync with HEAD

Sync with HEAD.

Extend the sparc64 cpu kcore segment so that it records all kernel mappings
done via locked 4 MB pages. The old format could only record physically
continous kernel text mappings.
For compatibility reasons, just extend the structure and fill in all the
old fields as well.

Sync with HEAD.

cosmetics: use the "extern char ..[]" idiom for linker supplied addresses.

sync with head.

Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree.  Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches.  The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

Sync with head.

sync with head.

Sync with HEAD

Sync with HEAD.

On second thought: a valid mapping is a valid mapping, period.
No need to special case the kernel pmap here.

Add support to pmap_extract() for kernal mappings with PA 0.

Cleanup cpu_info: get rid of ci_number and ci_upaid, use ci_index
and ci_cpuid instead.

Sync with HEAD.

Remove the (now unused) second 64k page mapped per CPU.
From matthew green, with small changes by me. All bugs are mine.

Do not call smp_tlb_flush_all() on every pmap_update() - it is too big
a hammer and papers over other problems. We need to revisit this and
fix it correctly soon.

Sync with HEAD.

sync with head.

Sync with HEAD.

Sync w/ NetBSD-4-RC_1

Pull up following (requested by martin in ticket #854):

Now that we have fixed the per-cpu mappings, adapt pmap_extract() accordingly.
No equivalent change on head, second per-cpu page is not used there any more.

Remove INITSTACK completely - at the time we used to switch to it, we
already have access to all of lwp0 and it's uarea - so we can switch
to the correct lwp0 stack easily before calling main.

Sync with HEAD.

Explicitly use trapframe64 instead of trapframe.

Sync with HEAD.

Sync with HEAD.

Complete initialization of lwp0 in pmap_bootstrap()

Sync with head.

Sync with HEAD.

Update to today's netbsd-4.

Sync with head.

Pull up following revision(s) (requested by martin in ticket #669):
	sys/arch/sparc64/sparc64/pmap.c: revision 1.190
In pmap_kenter_pa honour VM_PROT_EXECUTE - otherwise LKMs are not mapped
executable.
Fixes PR port-sparc64/36376.

In pmap_kenter_pa honour VM_PROT_EXECUTE - otherwise LKMs are not mapped
executable.
Fixes PR port-sparc64/36376.

Update to HEAD.

merge yamt-idlelwp branch.  asked by core@.  some ports still needs work.

from doc/BRANCHES:

	idle lwp, and some changes depending on it.

	1. separate context switching and thread scheduling.
	   (cf. gmcgarry_ctxsw)
	2. implement idle lwp.
	3. clean up related MD/MI interfaces.
	4. make scheduler(s) modular.

port yamt-idlelwp to sparc64.  this does not work yet, but others
have asked that i commit this work-in-progress.

currently it gets a double fault during a window save operation
and crashes to the prom.  (it seems very much like the problem
that i see on SMP sparc64, but there is no real evidence it is
the same root cause.)

Sync with head.

Pull up to the branch:
	cvs rdiff -r1.42 -r1.43 src/sys/arch/sparc64/include/ctlreg.h
	cvs rdiff -r1.187 -r1.188 src/sys/arch/sparc64/sparc64/pmap.c
Requested by mrg@

Pull up following revision(s) (requested by martin in ticket #547):
	sys/arch/sparc64/sparc64/pmap.c: revision 1.188
	sys/arch/sparc64/include/ctlreg.h: revision 1.43
When invalidating a context, make sure to flush user windows upfront.
Do not set the DMMU secondary context to 0 (that would be kernel), and
add a few required membars after switching the secondary dmmu context.
This avoids SIRs caused by double kernel_data_faults, caused by spills
of obsolete user windows after the context for that user pmap is gone.

When invalidating a context, make sure to flush user windows upfront.
Do not set the DMMU secondary context to 0 (that would be kernel), and
add a few required membars after switching the secondary dmmu context.

This avoids SIRs caused by double kernel_data_faults, caused by spills
of obsolete user windows after the context for that user pmap is gone.

sync with head.

Sync with head.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

Sync with HEAD.

Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

- sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.

sync with head.

Replace the Mach-derived boolean_t type with the C99 bool type.  A
future commit will replace use of TRUE and FALSE with true and false.

Sync with head.

sync with head.

sync with head.

s/existance/existence/, from Zafer.

Sync with head.

In the INTSTACK and KSTACK_VA special case treatment in pmap_exctract,
do not add the offset to the result - the calculated PA is the right
value - not the start of the page as in the other cases.
This fixes DMA to kernel stack, and avoids "fabricating a geometry"
warnings.

sync with head

correct a debug statement.

avoid yet another GCC uninitialised warning at -O3.

Ooops, array initialization for the page size map differs in DEBUG and
other kernels - hide details in a macro.
Pointed out by Matthias Scheler.

Initializers

allocate 128KB (as opposed to 64KB) per-cpu, and place the interrupt
stack, cpu_info structure, panicstack, idle uarea and normal kernel
stack in here.  the idle uarea and kernel stack are in the 2nd 64KB
page.  the interrupt stack and cpu_info structure are as the same
place in the 1st 64KB page, and the panicstack grows down from the
top of this page.  update comments in param.h to suit (XXX: a lot
of these constants should move to a header file that will not pollute
the name space so much.)  map these pages identically between
cpu_initialise() and cpu_mp_startup(), and deal with the two 64KB
pages in pmap_extract().  keep an unmapped 64KB page between them.

don't initialise the unused "ci_eintstack" member of the cpu_info
structure

use constants to initialise many addresses in the cpu_info structure,
consolidating these assignments from cpu0 and cpu>0

delete u0/estack0/panicstack from locore.s

clean up some wrong comments in locore.s

fix a 32bit-only bug in cpu_mp_startup().

delete 'u0' entirely.


this fixes recent random lossage seen on port-sparc64: processes
stuck in RUN state, data faults, and hangs.

Make -Wextra save

redo previous: print these int64_t with PRIx64.

in pmap_mp_init(), tag & data are 64 bit int's, print them as such.
also, fix up some comments and function names.

pmap_bootstrap() gets a bit of SMP help:
	- initialise bootcpu->ci_{cpcb,initstack}/proc0paddr from the
	  idle u area.
pmap_extract():
	- make this work on the kernel interrupt stack as well

both of these are from petrov@ via martin@.

pmap_update(): XXX mark the MULTIPROCESSOR call to smp_tlb_flush_all().

sync with head.

SMP cleanup.  provide support for multiple CPUs in DDB.  (SMP itself
is still not working.)

cpu.h:
- add a pointer for DDB regs in SMP environment to struct cpu_info
- remove the #defines for mp_pause_cpus() and mp_resume_cpus()
cpuset.h:
- remove CPUSET_ALL() and rename CPUSET_ALL_BUT() to CPUSET_EXCEPT()
  from petrov.
db_machdep.h:
- rename the members of db_regs_t to be the same as sparc
- change "db_regs_t ddb_regs" to "db_regs_t *ddb_regp" and change
  all references to suit
- redo DDB_REGS to no longer be a pointer to a fixed data structure
  but to one allocated per-cpu when ddb is entered
- move a bunch of prototypes in here
intr.h:
- remove SPARC64_IPI_* macros, no longer used
db_interface.c:
- change "db_regs_t ddb_regs" to "db_regs_t *ddb_regp" and change
  all references to suit
- make "nil" a 64 bit entity
- change the ddb register access methods to work in multiprocessor
  environment, it is now very much like sparc does it
- in kdb_trap() avoid accessing ddb_regp when it is NULL
- update several messages to include the cpu number
- unpause other cpus much later when resuming from ddb
- rename db_lock() to db_lock_cmd(), as the sparc-like code has
  db_lock as a simple lock
- remove "mach cpus" command, and replace it with "mach cpu" (which
  does the same) and also implement "mach cpu N" to switch to
  another cpus saved trapframe
db_trace.c:
- update for the ddb_regs -> ddb_regp change
genassym.cf:
- add TF_KSTACK as offsetof(struct trapframe64, tf_kstack)
ipifuncs.c:
- overhaul extensively
- remove all normal interrupt handlers as IPI's, we now handle
  them all specially in locore.s:interrupt_vector
- add a simplelock around all ipi functions - it's not safe for
  multiple cpus to be sending IPI's to each other right now
- rename sparc64_ipi_pause() to sparc64_ipi_pause_thiscpu() and,
  if DDB is configured, enable it to save the passed-in trapframe
  to a db_regs_t for this cpu's saved DDB registers.
- remove the "ipimask" system (SPARC64_IPI_* macros) and instead
  pass functions directly
- in sparc64_send_ipi() always set the interrupt arguments to 0,
  the address and argument of the to be called function.  (the
  argument right now is the address of ipi_tlb_args variable, and
  part of the reason why only one CPU can send IPI's at a time.)
  don't wait forever for an IPI to complete.  some of this is
  from petrov.
- rename sparc64_ipi_{halt,pause,resume}_cpus() to
  mp_{halt,pause,resume}_cpus()
- new function mp_cpu_is_paused() used to avoid access missing
  saved DDB registers
- actually broadcast the flush in smp_tlb_flush_pte(),
  smp_tlb_flush_ctx() and smp_tlb_flush_all().  the other end may
  not do anything yet in the pte/ctx cases yet...
kgdb_machdep.c:
- rework for changed member names in db_regs_t.
locore.s:
- shave an instruction from syscall_setup() (set + ld -> sethi + ld)
- remove some old dead debug code
- add new sparc64_ipi_halt IPI entry point, it just calls the C
  vector to shutdown.
- add new sparc64_ipi_pause IPI entry point, which just traps into
  the debugger using the normal breakpoint trap.  these cpus usually
  lose the race in db_interface.c:db_suspend_others() and end up
  calling the C vector sparc64_ipi_pause_thiscpu().
- add #if 0'ed code to sparc64_ipi_flush_{pte,ctx}() IPI entry
  points to call the sp_ version of these functions.
- in rft_kernel (return from trap, kernel), check to see if the
  %tpc is at the sparc64_ipi_pause_trap_point and if so, call
  "done" not "retry"
- rework cpu_switch slightly:  save the passed-in lwp instead of
  using the one in curlwp
- in cpu_loadproc(), save the new lwp not the old lwp, to curlwp
- in cpu_initialize(), set %tl to zero as well.  from petrov.
- in cpu_exit(), fix a load register confusion.  from petrov.
- change some "set" in delay branch to "mov".
machdep.c:
- deal with function renames
pmap.c:
- remove a spurious space
trap.c:
- remove unused "trapstats" variable
- add cpu number to a couple of messages

sync with head

sync with head.

sync with head.

Sync with head.

Add opt_multiprocessor.h.

Sync with head.

sync with head.

Use ANSI-style function definitions and declarations.

sync with head.

ANSIfication: u_intN_t -> uintN_t, use ANSI function declarations/definitions
instead of K&R ones.

sync with head.

Fix the new ofwboot boot problem with kernels where .text segment is smaller
than 4MB.

Unless this fix, boot failed with the message as below.

| panic: pmap_kextract: Address 0x1400000 is not from kernel space.
| Data segment is too small?

Alter sparc64 bootstrap, catch up to ofwboot v1.9:

 - Accept bootinfo structure passed down from ofwboot v1.9
 - Drop kernel re-mapping code
 - Use permanent 4MB mappings provided by the loader instead
 - Change kernel entry address to point directly at the code instead of pointing
   at the trap table's first slot. This allows the bootloader to detect
   those kernels which are aware of the new boot scheme
 - Due to the changes in kernel mapping code, alter secondary CPU bootstrap
   code to use trampoline just like FreeBSD does (some FreeBSD code is used
   here as well)

sync with head.

Fix 'memsize' value. pmap_bootstrap() calculates it as a PROM property
length but it is expected to contain a number of available memory regions
(e.g. in pmap_dumpsize() and pmap_dumpmmu()).

Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

Sync with HEAD. Here we go again...

Rename 'ncpus' to 'sparc_ncpus' to avoid shadow warnings in m.i. code.
Also sprinkle an __UNVOLATILE() for sparc.
n.b. sparc64 'cpus' should probably be renamed to 'cpu_info_list' to
     match i386 et al.

Add const (catching up with additions to MI ddb code).

- sprinkle const
- avoid shadowed variables

sync with -current

Sync with HEAD.

merge yamt-km branch.
- don't use managed mappings/backing objects for wired memory allocations.
  save some resources like pv_entry.  also fix (most of) PR/27030.
- simplify kernel memory management API.
- simplify pmap bootstrap of some ports.
- some related cleanups.

sync with head.  xen and whitespace.  xen part is not finished.

Sync with HEAD.

Hi Perry!

Cleanup: the PG_SZ macro was buggy (pointed out by Tim Newsham).
It was only used by (the also buggy/not ready for prime time) PMAP_PAGE_SIZE,
which in turn only was used in one place.

Get rid of all this and hardcode the result at the single place of use.

convert arch/sparc64 to new apis.

Pull up revision 1.158 (requested by martin in ticket #1014).

Better return value checking for pseg_get/pseg_set. Turn Debugger() calls
into proper panics and sprinkle some KASSERTs. Closes PR 27288.

Pull up revision 1.158 (requested by martin in ticket #1014).

Better return value checking for pseg_get/pseg_set. Turn Debugger() calls
into proper panics and sprinkle some KASSERTs. Closes PR 27288.

Pull up revision 1.157 (requested by martin in ticket #1013).

do the invalidation of the (virtually-addressed) L1 cache in pmap_remove_all()
(ie. just before we skip the per-page cache invalidations while tearing down
each context) instead of just before we recycle all the contexts in ctx_alloc().
the latter is not good enough since a (physically tagged) L1 cache line is
valid for all contexts, not just the one that was used to instantiate it.
fixes PRs 24126, 25669 and 27730.

Pull up revision 1.157 (requested by martin in ticket #1013).

do the invalidation of the (virtually-addressed) L1 cache in pmap_remove_all()
(ie. just before we skip the per-page cache invalidations while tearing down
each context) instead of just before we recycle all the contexts in ctx_alloc().
the latter is not good enough since a (physically tagged) L1 cache line is
valid for all contexts, not just the one that was used to instantiate it.
fixes PRs 24126, 25669 and 27730.

Sync with HEAD.

remove dead code and other minor cleaning.

Better return value checking for pseg_get/pseg_set. Turn Debugger() calls
into proper panics and sprinkle some KASSERTs. Closes PR 27288.

do the invalidation of the (virtually-addressed) L1 cache in pmap_remove_all()
(ie. just before we skip the per-page cache invalidations while tearing down
each context) instead of just before we recycle all the contexts in ctx_alloc().
the latter is not good enough since a (physically tagged) L1 cache line is
valid for all contexts, not just the one that was used to instantiate it.
fixes PRs 24126, 25669 and 27730.

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Mostly switch to promlib interface. Postpone the use of prom_makememarr()
since the sparc64 memory range properties aren't compatible yet.

Fix pmap_calculate_colors() in the process.

checkpoint of MP work from dennis and myself.  includes cross-processor
interrupt framework, a sledgehammer TLB invalidation and misc MP fixes.
doesn't work at all yet.

pmap_activate() on pmap_kernel() should be a no-op.
needed for now while exiting processes "borrow" vmspace0,
but it's reasonable to check anyway.

make sure we free the context for the pmap in pmap_destroy().
this usually isn't necessary since we freed it earlier in pmap_remove_all(),
but since uvmspace_free() is now called in the context of the exiting
process, a new context might be allocated if uvm_unmap_detach() decides
to sleep (since cpu_switch() will allocate a new context when it switches
back to the exiting process).

Spinup secondary cpus. Based on codes sent to me by Dennis Chernoivanov
and Chuck Silvers.

Use pmap_get_page return code instead, remove unused pmap_stats,
delete incorrect KASSERT in pmap_kenter_pa.

Use per-cpu pcb, curlwp and fplwp rather than global ones. This brings
GENERIC.MP configuration to a usable state.

Approved by petrov@.

bzero/bcopy -> memset/memcpy

Fix uninitialized variable warning.

Don't use NULL in integer-type assignments/comparisons.

don't assign/compare integer and NULL.

Pull up revision 1.145 via patch (requested by martin in ticket #1522):
cache_flush_virt() is buggy, just use blast_dcache() instead.
this should fix PR 23093.

cache_flush_virt() is buggy, just use blast_dcache() instead.
this should fix PR 23093.

in pmap_enter(), invalidate the TSBs before pre-loading the new values,
since we only pre-load one and the other one could be stale.
also, remove the bogus dcache_flush_page() from pmap_clear_modify() again,
since the problem it was intended to mask seems to have disappeared.

add support for non-executable mappings (where the hardware allows this)
and make the stack and heap non-executable by default.  the changes
fall into two basic catagories:

 - pmap and trap-handler changes.  these are all MD:
   = alpha: we already track per-page execute permission with the (software)
	PG_EXEC bit, so just have the trap handler pay attention to it.
   = i386: use a new GDT segment for %cs for processes that have no
	executable mappings above a certain threshold (currently the
	bottom of the stack).  track per-page execute permission with
	the last unused PTE bit.
   = powerpc/ibm4xx: just use the hardware exec bit.
   = powerpc/oea: we already track per-page exec bits, but the hardware only
	implements non-exec mappings at the segment level.  so track the
	number of executable mappings in each segment and turn on the no-exec
	segment bit iff the count is 0.  adjust the trap handler to deal.
   = sparc (sun4m): fix our use of the hardware protection bits.
	fix the trap handler to recognize text faults.
   = sparc64: split the existing unified TSB into data and instruction TSBs,
	and only load TTEs into the appropriate TSB(s) for the permissions.
	fix the trap handler to check for execute permission.
   = not yet implemented: amd64, hppa, sh5

 - changes in all the emulations that put a signal trampoline on the stack.
   instead, we now put the trampoline into a uvm_aobj and map that into
   the process separately.

originally from openbsd, adapted for netbsd by me.

__KERNEL_RCSID()

Avoid strict-alias warnings in gcc 3.3.

Backout previous, it was apparently ovelooked when backing out the
change in other places.

Simplify the way the bounds of the managed kernel virtual address
space is advertised to UVM by making virtual_avail and virtual_end
first-class exported variables by UVM.  Machine-dependent code is
responsible for initializing them before main() is called.  Anything
that steals KVA must adjust these variables accordingly.

This reduces the number of instances of this info from 3 to 1, and
simplifies the pmap(9) interface by removing the pmap_virtual_space()
function call, and removing two arguments from pmap_steal_memory().

This also eliminates some kludges such as having to burn kernel_map
entries on space used by the kernel and stolen KVA.

This also eliminates use of VM_{MIN,MAX}_KERNEL_ADDRESS from MI code,
this giving MD code greater flexibility over the bounds of the managed
kernel virtual address space if a given port's specific platforms can
vary in this regard (this is especially true of the evb* ports).

Use PAGE_SIZE rather than NBPG.

Add some braces to make a block of code a bit easier to read.

Band aid - the infamous dcache_flush_page change independently suggested
by Andrey Petrov and Jason Thorpje (AFAIK).

This is a hack that covers some symptoms while we have no idea where
the real problem is. Anyway, since this avoids random data corruption
we better be safe and have this in-tree until the problem is solved the
right way.

Make the pmap count resident/wired mappings on the fly instead of
walking the page tables whenever this information is needed.

Add an option PMAP_COUNT_DEBUG to assert the new counts and the
page table walk agree.

The old solution had very bad performance impact, for example
by the high CPU load when running top(1).

Thanks to Simon Burge for pointing at the cause of the problem and
to Valeriy E. Ushakov for optimizing my simple minded assembler code.

Remove #if 0'd pmap_zero_page and pmap_copy_page, which are now
implemented in assembler in locore.s.

Merge the nathanw_sa branch.

Sync with HEAD.

fix two bugs in my last pmap rework:
 - in tlb_flush_all(), don't skip TLB entries with the high bit on,
   I was confused about which MMU register it was using.  it's also fine
   to use the last hardware context.
 - in pmap_create(), don't allocate a hardware context for the new pmap.
   it's unnecessary, and when this would cause us to recycle all the contexts,
   it would result in the current process's context being set to 0
   (ie. the kernel's context).  the current process could then return to
   userspace without going through the context-switch code (and thus without
   having a hardware context reallocated).  this would lead to user mappings
   being entered in the kernel's context later, causing all sorts of trouble.
   add some assertions to catch this kind of thing.