CVS log for src/sys/arch/x86/x86/vm_machdep.c

Up to [cvs.NetBSD.org] / src / sys / arch / x86 / x86

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Convert the l2->l_md.md_astpending assignments into KASSERTs.

l_md is zeroised by lwp_create with

         memset(&l2->l_startzero, 0, sizeof(*l2) -
                    offsetof(lwp_t, l_startzero));

Sync with HEAD.

Sync with HEAD.

fix a comment that has been c&p'ed around and not updated

Sync w/ HEAD.

s/ we we / we /

In cpu_uarea_{alloc,free}:

 - My previous change in this file was not correct, kremove does not free
   the underlying PA, which caused a very slow leak under memory pressure.
   Rework to correctly free the PA.
 - Add a second redzone, this time after the stack, to catch several stack
   overflows. The main concern is read overflows which leak the heap that
   follows the stack.
 - UVM_KMF_WAITVA doesn't fail, so remove error check.
 - Add KASSERTs.

Mostly merge changes from HEAD upto 20200411

Add a redzone between the pcb and the stack. Sent to port-amd64@.

Sync with head.

cpu_lwp_free() can be called with (l != curlwp) in error paths, so don't
detonate.

Sync with head.

x86 pmap:

- It turns out that every page the pmap frees is necessarily zeroed.  Tell
  the VM system about this and use the pmap as a source of pre-zeroed pages.

- Redo deferred freeing of PTPs more elegantly, including the integration with
  pmap_remove_all().  This fixes problems with nvmm, and possibly also a crash
  discovered during fuzzing.

Reported-by: syzbot+a97186518c84f1d85c0c@syzkaller.appspotmail.com

Remove unused call to savectx().

Rewrite the FPU code on x86. This greatly simplifies the logic and removes
the dependency on IPL_HIGH. NVMM is updated accordingly. Posted on
port-amd64 a week ago.

Bump the kernel version to 9.99.16.

Sync with HEAD

We reorganise definitions for XEN source support as follows:

XEN - common sources required for baseline XEN support.
XENPV - sources required for support of XEN in PV mode.
XENPVHVM - sources required for support for XEN in HVM mode.
XENPVH - sources required for support for XEN in PVH mode.

Sync with HEAD

Rework dbregs, to switch the registers during context switches, and not on
each user->kernel transition via userret. Reloads of DR6/DR7 are expensive
on both native and xen.

Use a variable-sized memcpy, instead of copying the PCB and then adding
the extra bytes. The PCB embeds the biggest static FPU state, but our
real FPU state may be smaller (FNSAVE), so we don't need to memcpy the
extra unused bytes.

Sync with HEAD

Pull up the following, via patch, requested by maxv in ticket #897:

	sys/arch/amd64/amd64/locore.S           1.166 (patch)
	sys/arch/i386/i386/locore.S             1.157 (patch)
	sys/arch/x86/include/cpu.h              1.92 (patch)
	sys/arch/x86/include/fpu.h              1.9 (patch)
	sys/arch/x86/x86/fpu.c                  1.33-1.39 (patch)
	sys/arch/x86/x86/identcpu.c             1.72 (patch)
	sys/arch/x86/x86/vm_machdep.c           1.34 (patch)
	sys/arch/x86/x86/x86_machdep.c          1.116,1.117 (patch)

Support eager fpu switch, to work around INTEL-SA-00145.
Provide a sysctl machdep.fpu_eager, which gets automatically
initialized to 1 on affected CPUs.

Explicitly clear l2's pcb_fpcpu when forking.

A context switch (preemption) could occur between

	fpusave_lwp(l1, true);
and
	memcpy(pcb2, pcb1, sizeof(struct pcb));

In this case, l1's FPU state is re-installed on the current CPU, and
pcb1->pcb_fpcpu becomes non NULL. While it's fine to have l1's state
installed, we don't want to indicate l2's state is installed too.

With lazy fpu this was not a problem, because the context-switch would
not re-install the state, so pcb1->pcb_fpcpu was NULL.

Should fix PR/53383.

Pull up the following revisions, requested by maxv in ticket #652:

	sys/arch/amd64/amd64/amd64_trap.S	upto 1.39 (partial, patch)
	sys/arch/amd64/amd64/db_machdep.c	1.6 (patch)
	sys/arch/amd64/amd64/genassym.cf	1.65,1.66,1.67 (patch)
	sys/arch/amd64/amd64/locore.S		upto 1.159 (partial, patch)
	sys/arch/amd64/amd64/machdep.c		1.299-1.302 (patch)
	sys/arch/amd64/amd64/trap.c		upto 1.113 (partial, patch)
	sys/arch/amd64/amd64/amd64/vector.S	upto 1.61 (partial, patch)
	sys/arch/amd64/conf/GENERIC		1.477,1.478 (patch)
	sys/arch/amd64/conf/kern.ldscript	1.26 (patch)
	sys/arch/amd64/include/frameasm.h	upto 1.37 (partial, patch)
	sys/arch/amd64/include/param.h		1.25 (patch)
	sys/arch/amd64/include/pmap.h		1.41,1.43,1.44 (patch)
	sys/arch/x86/conf/files.x86		1.91,1.93 (patch)
	sys/arch/x86/include/cpu.h		1.88,1.89 (patch)
	sys/arch/x86/include/pmap.h		1.75 (patch)
	sys/arch/x86/x86/cpu.c			1.144,1.146,1.148,1.149 (patch)
	sys/arch/x86/x86/pmap.c			upto 1.289 (partial, patch)
	sys/arch/x86/x86/vm_machdep.c		1.31,1.32 (patch)
	sys/arch/x86/x86/x86_machdep.c		1.104,1.106,1.108 (patch)
	sys/arch/x86/x86/svs.c			1.1-1.14
	sys/arch/xen/conf/files.compat		1.30 (patch)

Backport SVS. Not enabled yet.

Synch with HEAD, resolve conflicts

Pull up the following revisions, requested by maxv in ticket #637:

	sys/arch/amd64/amd64/process_machdep.c	1.33,1.34,1.35 (patch)
	sys/arch/amd64/include/types.h		1.55 (patch)
	sys/arch/x86/x86/vm_machdep.c		1.33 (patch)

- Reduce the number of places where segment register faults can
  occur.
- Remove __HAVE_CPU_UAREA_ROUTINES.

Remove the __HAVE_CPU_UAREA_ROUTINES code from x86.

It was available only in amd64, and I disabled it a few months ago in
order to support SVS. Regardless of SVS this option was questionable,
since it made stack overflows more difficult to detect.

Pull up following revision(s) (requested by maxv in ticket #1517):
	sys/arch/amd64/amd64/machdep.c: 1.280 via patch
	sys/arch/amd64/include/segments.h: 1.34 via patch
	sys/arch/i386/i386/machdep.c: 1.800
	sys/arch/i386/include/segments.h: 1.64
	sys/arch/x86/x86/vm_machdep.c: 1.30
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Pull up following revision(s) (requested by maxv in ticket #1517):
	sys/arch/amd64/amd64/machdep.c: 1.280 via patch
	sys/arch/amd64/include/segments.h: 1.34 via patch
	sys/arch/i386/i386/machdep.c: 1.800
	sys/arch/i386/include/segments.h: 1.64
	sys/arch/x86/x86/vm_machdep.c: 1.30
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Pull up following revision(s) (requested by maxv in ticket #1517):
	sys/arch/amd64/amd64/machdep.c: 1.280 via patch
	sys/arch/amd64/include/segments.h: 1.34 via patch
	sys/arch/i386/i386/machdep.c: 1.800
	sys/arch/i386/include/segments.h: 1.64
	sys/arch/x86/x86/vm_machdep.c: 1.30
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Pull up following revision(s) (requested by maxv in ticket #1550):
	sys/arch/amd64/amd64/machdep.c: revision 1.280 via patch
	sys/arch/amd64/include/segments.h: revision 1.34 via patch
	sys/arch/i386/i386/machdep.c: revision 1.800 via patch
	sys/arch/i386/include/segments.h: revision 1.64 via patch
	sys/arch/x86/x86/vm_machdep.c: revision 1.30 via patch
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Pull up following revision(s) (requested by maxv in ticket #1550):
	sys/arch/amd64/amd64/machdep.c: revision 1.280 via patch
	sys/arch/amd64/include/segments.h: revision 1.34 via patch
	sys/arch/i386/i386/machdep.c: revision 1.800 via patch
	sys/arch/i386/include/segments.h: revision 1.64 via patch
	sys/arch/x86/x86/vm_machdep.c: revision 1.30 via patch
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Pull up following revision(s) (requested by maxv in ticket #1550):
	sys/arch/amd64/amd64/machdep.c: revision 1.280 via patch
	sys/arch/amd64/include/segments.h: revision 1.34 via patch
	sys/arch/i386/i386/machdep.c: revision 1.800 via patch
	sys/arch/i386/include/segments.h: revision 1.64 via patch
	sys/arch/x86/x86/vm_machdep.c: revision 1.30 via patch
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Unmap the kernel heap from the user page tables (SVS).

This implementation is optimized and organized in such a way that we
don't need to copy the kernel stack to a safe place during user<->kernel
transitions. We create two VAs that point to the same physical page; one
will be mapped in userland and is offset in order to contain only the
trapframe, the other is mapped in the kernel and maps the entire stack.

Sent on tech-kern@ a week ago.

The uarea must always be page-aligned.

Pull up following revision(s) (requested by maxv in ticket #477):
	sys/arch/amd64/amd64/machdep.c: revision 1.280
	sys/arch/amd64/include/segments.h: revision 1.34
	sys/arch/i386/i386/machdep.c: revision 1.800
	sys/arch/i386/include/segments.h: revision 1.64 via patch
	sys/arch/x86/x86/vm_machdep.c: revision 1.30
Fix a huge privilege separation vulnerability in Xen-amd64.
On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.
It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.
Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

Fix a huge privilege separation vulnerability in Xen-amd64.

On amd64 the kernel runs in ring3, like userland, and therefore SEL_KPL
equals SEL_UPL. While Xen can make a distinction between usermode and
kernelmode in %cs, it can't when it comes to iopl. Since we set SEL_KPL
in iopl, Xen sees SEL_UPL, and allows (unprivileged) userland processes
to read and write to the CPU ports.

It is easy, then, to completely escalate privileges; by reprogramming the
PIC, by reading the ATA disks, by intercepting the keyboard interrupts
(keylogger), etc.

Declare IOPL_KPL, set to 1 on Xen-amd64, which allows the kernel to use
the ports but not userland. I didn't test this change on i386, but it
seems fine enough.

update from HEAD

Sync with HEAD

Check (inside), not (!outside). It explains the two install failures
reported between pmap.h::r1.65 and vmparam.h::r1.40.

Sync with HEAD

Sync with HEAD

Introduce PT_GETDBREGS and PT_SETDBREGS in ptrace(2) on i386 and amd64

This interface is modeled after FreeBSD API with the usage.

This replaced previous watchpoint API. The previous one was introduced
recently in NetBSD-current and remove its spurs without any
backward-compatibility.

Design choices for Debug Register accessors:
 - exec() (TRAP_EXEC event) must remove debug registers from LWP
 - debug registers are only per-LWP, not per-process globally
 - debug registers must not be inherited after (v)forking a process
 - debug registers must not be inherited after forking a thread
 - a debugger is responsible to set global watchpoints/breakpoints with the
   debug registers, to achieve this PTRACE_LWP_CREATE/PTRACE_LWP_EXIT event
   monitoring function is designed to be used
 - debug register traps must generate SIGTRAP with si_code TRAP_DBREG
 - debugger is responsible to retrieve debug register state to distinguish
   the exact debug register trap (DR6 is Status Register on x86)
 - kernel must not remove debug register traps after triggering a trap event
   a debugger is responsible to detach this trap with appropriate PT_SETDBREGS
   call (DR7 is Control Register on x86)
 - debug registers must not be exposed in mcontext
 - userland must not be allowed to set a trap on the kernel

Implementation notes on i386 and amd64:
 - the initial state of debug register is retrieved on boot and this value is
   stored in a local copy (initdbregs), this value is used to initialize dbreg
   context after PT_GETDBREGS
 - struct dbregs is stored in pcb as a pointer and by default not initialized
 - reserved registers (DR4-DR5, DR9-DR15) are ignored

Further ideas:
 - restrict this interface with securelevel

Tested on real hardware i386 (Intel Pentium IV) and amd64 (Intel i7).

This commit enables 390 debug register ATF tests in kernel/arch/x86.
All tests are passing.

This commit does not cover netbsd32 compat code. Currently other interface
PT_GET_SIGINFO/PT_SET_SIGINFO is required in netbsd32 compat code in order to
validate reliably PT_GETDBREGS/PT_SETDBREGS.

This implementation does not cover FreeBSD specific defines in their
<x86/reg.h>: DBREG_DR7_LOCAL_ENABLE, DBREG_DR7_GLOBAL_ENABLE, DBREG_DR7_LEN_1
etc. These values tend to be reinvented by each tracer on its own. GNU
Debugger (GDB) works with NetBSD debug registers after adding this patch:

--- gdb/amd64bsd-nat.c.orig	2016-02-10 03:19:39.000000000 +0000
+++ gdb/amd64bsd-nat.c
@@ -167,6 +167,10 @@ amd64bsd_target (void)

 #ifdef HAVE_PT_GETDBREGS

+#ifndef DBREG_DRX
+#define	DBREG_DRX(d,x)	((d)->dr[(x)])
+#endif
+
 static unsigned long
 amd64bsd_dr_get (ptid_t ptid, int regnum)
 {


Another reason to stop introducing unpopular defines covering machine
specific register macros is that these value varies across generations of
the same CPU family.

GDB demo:
  (gdb) c
  Continuing.

  Watchpoint 2: traceme

  Old value = 0
  New value = 16
  main (argc=1, argv=0x7f7fff79fe30) at test.c:8
  8               printf("traceme=%d\n", traceme);

(Currently the GDB interface is not reliable due to NetBSD support bugs)

Sponsored by <The NetBSD Foundation>

Sync with HEAD

Sync with netbsd-5

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Pull up following revision(s) (requested by riastradh in ticket #1316):
	sys/arch/x86/x86/pmap.c: revision 1.223
	sys/arch/x86/x86/vm_machdep.c: revision 1.26
	sys/arch/x86/include/pmap.h: revision 1.61
PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7

Pull up following revision(s) (requested by riastradh in ticket #1316):
	sys/arch/x86/x86/pmap.c: revision 1.223
	sys/arch/x86/x86/vm_machdep.c: revision 1.26
	sys/arch/x86/include/pmap.h: revision 1.61
PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7

Add support for hardware assisted watchpoints/breakpoints API in ptrace(2)

Add new ptrace(2) calls:
 - PT_COUNT_WATCHPOINTS - count the number of available hardware watchpoints
 - PT_READ_WATCHPOINT   - read struct ptrace_watchpoint from the kernel state
 - PT_WRITE_WATCHPOINT  - write new struct ptrace_watchpoint state, this
                          includes enabling and disabling watchpoints

The ptrace_watchpoint structure contains MI and MD parts:

typedef struct ptrace_watchpoint {
	int		pw_index;	/* HW Watchpoint ID (count from 0) */
	lwpid_t		pw_lwpid;	/* LWP described */
	struct mdpw	pw_md;		/* MD fields */
} ptrace_watchpoint_t;

For example amd64 defines MD as follows:
struct mdpw {
	void	*md_address;
	int	 md_condition;
	int	 md_length;
};

These calls are protected with the __HAVE_PTRACE_WATCHPOINTS guard.

Tested on amd64, initial support added for i386 and XEN.

Sponsored by <The NetBSD Foundation>

Sync with HEAD

PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

mark a diagnostic only variable

Add support for saving the AVX-256 ymm registers during FPU context switches.
Add support for the forthcoming AVX-512 registers.
Code compiled with -mavx seems to work, but I've not tested context
  switches with live ymm registers.
There is a small cost on fork/exec (a larger area is copied/zerod),
  but I don't think the ymm registers are read/written unless they
  have been used.
The code use XSAVE on all cpus, I'm not brave enough to enable XSAVEOPT.

Move the amd64 and i386 pcb to the bottom of the uarea, and move the
  kernel stack to the top.
Change the pcb layouts so that fpu save area is at the end and is
  64byte aligned ready for xsave (saving the ymm registers).
Welcome to 6.99.32

Remove all references to MDL_USEDFPU and deferred fpu initialisation.
The cost of zeroing the save area on exec is minimal.
This stops the FP registers of a random process being used the first
  time an lwp uses the fpu.
sendsig_siginfo() and get_mcontext() now unconditionally copy the FP
registers.
I'll remove the double-copy for signal handlers soon.
get_mcontext() might have been leaking kernel memory to userspace - and
  may still do so if i386_use_fxsave is false (short copies).

Move sys/arch/amd64/amd64/fpu.c and sys/arch/amd64/include/fpu.h
into sys/arch/x86 in preparation for using the same code for i386.

Remove support for 'external' floating point units and the MS-DOS
  compatible method of handling floating point exceptions.
Make kernel support for teh fpu non-optional (486SX should still work).
Only 386 cpus support external fpu, and i386 support was removed years ago.
This means that the npx code no longer uses port 0xf0 or interupt 13.
All the "npx at isa" lines go from the configs, arch/i386/isa/npx.c
  is now mandatory for all i386 kernels.
I've renamed npxinit() to fpuinit() and npxinit_cpu() to fpuinit_cpu()
  to match the very similar amd64 functions.
The fpu of the boot cpu is now initialised by a direct call from
  cpu_configure(), this enables FP emulation for a 486SX.
  (for amd64 the cr0 values are set in locore.S and similar).
This fixes a long-standing bug in linux_setregs() - which did not
  save the fpu regsiters if they were active.
I've test booted a single cpu i386 kernel (using anita).
amd64 builds - none of teh changes should affect it.
The i386 XEN kernels build, but I'm not sure where they set cr0, and
  it might have got lost!

Comment about missing stackframe member initialization (Richard Hansen)

I haven't studied the code, but I'm concerned that not initializing
sf->sf_edi could potentially leak a few bytes of information to a new
userspace process.

revert fpu/pcu changes until we figure out what's wrong; they cause random
freezes

Use the MI "pcu" framework for bookkeeping of npx/fpu states on x86.
This reduces the amount of MD code enormously, and makes it easier
to implement support for newer CPU features which require more fpu
state, or for fpu usage by the kernel.
For access to FPU state across CPUs, an xcall kthread is used now
rather than a dedicated IPI.
No user visible changes intended.

sync with head

Rename MDP_IRET to MDL_IRET since it is an lwp flag, not a proc one.
Add an MDL_COMPAT32 flag to the lwp's md_flags, set it for 32bit lwps
  and use it to force 'return to user' with iret (as is done when
  MDL_IRET is set).
Split the iret/sysret code paths much later.
Remove all the replicated code for 32bit system calls - which was only
  needed so that iret was always used.
frameasm.h for XEN contains '#define swapgs', while XEN probable never
  needs swapgs, this is likely to be confusing.
Add a SWAPGS which is a nop on XEN and swapgs otherwise.
(I've not yet checked all the swapgs in files that include frameasm.h)
Simple x86 programs still work.
Hijack 6.99.9 kernel bump (needed for compat32 modules)

sync with head

sync to -current.

Remove COMPAT_SA / KERN_SA.  Welcome to 6.99.3!
Approved by core@.

merge to -current.

allocate uareas contiguously and access them via the direct map.

Sync with HEAD.

Sync with HEAD. TODO before merge:
- shortcut for suspend code in sysmon, when powerd(8) is not running.
Borrow ``xs_watch'' thread context?
- bug hunting in xbd + xennet resume. Rings are currently thrashed upon
resume, so current implementation force flush them on suspend. It's not
really needed.

sync with head

Sync with HEAD

Make vmapbuf() return success/error and make physio deal with a
failure.

Sync with HEAD

cpu_lwp_free2: add assertions

Copy PK_32 to p2->p_flag instead of doing it in the cpu_proc_fork hook.

Sync with HEAD

Sync with HEAD.

sync with head.

implement cpu_lwp_setprivate() on several platforms.

sync with head

Sync with HEAD.

Use struct segment_descriptor for pcb_fsd and pcb_gsd instead of int[2].

sync with head

Replace l_addr with uvm_lwp_getuarea() in various MD code, mostly cpu_lwp_fork().

Disable kstack red-zone for now, while we decide on a nice way to fix it.

Use lwp_getpcb() on x86 MD code, clean from struct user usage.

Add a flags argument to pmap_kenter_pa(9).
Patch showed on tech-kern@ http://mail-index.netbsd.org/tech-kern/2009/11/04/msg006434.html
No objections.

Sync with HEAD.

fix a typo in a comment.

cpu_proc_fork: use pcb1 and pcb2, and thus make routine more readable.
Remove or update outdated comments, add new ones.  Clean-up.

Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code.  Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

file vm_machdep.c was added on branch jym-xensuspend on 2009-05-13 17:18:45 +0000

file vm_machdep.c was added on branch yamt-nfs-mp on 2009-05-04 08:12:11 +0000

file vm_machdep.c was added on branch nick-hppapmap on 2009-04-28 07:34:57 +0000

Merge i386 and amd64 vm_machdep.c into x86.  No functional changes intended.
Note: some #ifdefs will be removed with macros.