CVS log for src/sys/arch/amd64/amd64/genassym.cf

Up to [cvs.NetBSD.org] / src / sys / arch / amd64 / amd64

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

amd64: Allocate FPU save state outside pcb if it's too large.

We have seen x86_fpu_save_size values (CPUID[EAX=0x0d, ECX=0].ECX) as
large as 11008 bytes, notably with Intel AMX TILEDATA's 8192-byte
state.

We only do this for user threads, and only on machines where it's
necessary, to avoid incurring much overhead.  There is still a tiny
bit of overhead when saving and restoring the FPU state by using a
pointer indirection instead of arithmetic indirection for access to
struct pcb::pcb_savefpu, but this is probably a drop in the bucket
compared to the memory traffic incurred by the FPU state save/restore
anyway.

For now, these paths are mostly disabled on i386.  We could enable
them but it will require either rewriting cpu_uarea_alloc/free for
i386, or adopting a guard page like amd64 does, which might be costly
and so should be undertaken only with some thought and care.  And
since Intel AMX instructions only work in 64-bit mode, it's not
likely to be useful on i386.

PR port-amd64/57661: Crash when booting on Xeon Silver 4416+ in
KVM/Qemu

These changes, as a side effect, may fix:

PR kern/57258: kthread_fpu_enter/exit problem

by making sure to allocate an FPU save space that is large enough to
guarantee fpu_kern_enter/leave work safely, instead of just using a
union savefpu object on the stack (which, at 576 bytes, may be too
small on some machines, particularly with AVX512 requiring ~2.5K).
(But we'll have to do some extra work with kthread_fpu_enter/exit_md
-- if we try doing them again on x86 -- to actually allocate the
separate pcb on these machines!)

Pull up following revision(s) (requested by imil in ticket #1074):

	sys/arch/x86/x86/x86_machdep.c: revision 1.155
	sys/arch/x86/include/cpu.h: revision 1.137
	sys/arch/x86/x86/x86_machdep.c: revision 1.156
	sys/arch/x86/include/cpu.h: revision 1.138
	sys/arch/x86/x86/consinit.c: revision 1.40
	sys/arch/x86/acpi/acpi_machdep.c: revision 1.37
	sys/arch/x86/acpi/acpi_machdep.c: revision 1.38
	sys/arch/amd64/amd64/machdep.c: revision 1.370
	sys/arch/xen/xen/hypervisor.c: revision 1.97
	sys/arch/xen/xen/hypervisor.c: revision 1.98
	sys/arch/amd64/amd64/genassym.cf: revision 1.98
	sys/arch/x86/x86/x86_autoconf.c: revision 1.88
	sys/arch/x86/x86/x86_autoconf.c: revision 1.89
	sys/arch/amd64/amd64/locore.S: revision 1.226
	sys/arch/amd64/amd64/locore.S: revision 1.227
	sys/arch/x86/x86/identcpu.c: revision 1.131

Add support for non-Xen PVH guests to amd64. Patch from
Emile 'iMil' Heitor in PR kern/57813, with some cosmetic tweaks by me.
Tested on bare metal, Xen PV and Xen PVH by me.

Get one more change from PR kern/57813, needed for non-Xen PVH.

Introduce vm_guest_is_pvh() and use it in place of
        (vm_guest == VM_GUEST_XENPVH || vm_guest == VM_GUEST_GENPVH)

Add support for non-Xen PVH guests to amd64. Patch from
Emile 'iMil' Heitor in PR kern/57813, with some cosmetic tweaks by me.
Tested on bare metal, Xen PV and Xen PVH by me.

Eliminate l->l_ncsw and l->l_nivcsw.  From memory think they were added
before we had per-LWP struct rusage; the same is now tracked there.

Remove unused L_KPRIORITY.

Revert "x86: Add kthread_fpu_enter/exit support, take two."

kthread_fpu_enter/exit changes broke some hardware, unclear why, to
investigate before fixing and reapplying these changes.

x86: Add kthread_fpu_enter/exit support, take two.

This time, make sure to restore the FPU state when switching to a
kthread in the middle of kthread_fpu_enter/exit.

This adds a single predicted-taken branch for the case of kthreads
that are not in kthread_fpu_enter/exit, so it incurs a penalty only
for threads that actually use it.  Since it avoids FPU state
switching in kthreads that do use the FPU, namely cgd worker threads,
this should be a net performance win on systems using it and have
negligible impact otherwise.

XXX pullup-10

Move some definitions. Add comment. No functional change.

Reduce diff between i386/i386/genassym.cf and amd64/amd64/genassym.cf. NFCI.

Sort lines. Add comment. No functional change.

Reduce diff between i386/i386/genassym.cf and amd64/amd64/genassym.cf. NFCI.

 - Whitespace fix.
 - Sort lines.

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
    https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implemented by nonaka@n.o, updated by me.

x86: Split most of pmap.h into pmap_private.h or vmparam.h.

This way pmap.h only contains the MD definition of the MI pmap(9)
API, which loads of things in the kernel rely on, so changing x86
pmap internals no longer requires recompiling the entire kernel every
time.

Callers needing these internals must now use machine/pmap_private.h.
Note: This is not x86/pmap_private.h because it contains three parts:

1. CPU-specific (different for i386/amd64) definitions used by...

2. common definitions, including Xenisms like xpmap_ptetomach,
   further used by...

3. more CPU-specific inlines for pmap_pte_* operations

So {amd64,i386}/pmap_private.h defines 1, includes x86/pmap_private.h
for 2, and then defines 3.  Maybe we should split that out into a new
pmap_pte.h to reduce this trouble.

No functional change intended, other than that some .c files must
include machine/pmap_private.h when previously uvm/uvm_pmap.h
polluted the namespace with pmap internals.

Note: This migrates part of i386/pmap.h into i386/vmparam.h --
specifically the parts that are needed for several constants defined
in vmparam.h:

VM_MAXUSER_ADDRESS
VM_MAX_ADDRESS
VM_MAX_KERNEL_ADDRESS
VM_MIN_KERNEL_ADDRESS

Since i386 needs PDP_SIZE in vmparam.h, I added it there on amd64
too, just to keep things parallel.

XenPV: in cpu_switchto(), reset ci_xen_clockf_usermode/ci_xen_clockf_pc,
in case a clock interrupt is deffered while we're switching lwp.
Fix a (rare) panic:
panic: kernel diagnostic assertion "p != NULL" failed: file "/usr/src/sys/kern/kern_clock.c", line 438
seen when xen_timer_handler() is called from the splx() at the end of
mi_switch()

Some devices (e.g. ixg in MSI-X mode) don't to have their handlers called
when no interrupt are pending. So add an extra ih_pending field
to struct intrhand, which is incremeted when the handler is not called because
of IPL level and reset to 0 when called. Check this in Xen's resume
assembly to call only handlers that are really pending.

Add ACL support for FFS. From FreeBSD.

Introduce Xen PVH support in GENERIC.
This is compiled in with
options XENPVHVM
x86 changes:
- add Xen section and xen pvh entry points to locore.S. Set vm_guest
  to VM_GUEST_XENPVH in this entry point.
  Most of the boot procedure (especially page table setup and switch to
  paged mode) is shared with native.
- change some x86_delay() to delay_func(), which points to x86_delay() for
  native/HVM, and xen_delay() for PVH

Xen changes:
- remove Xen bits from init_x86_64_ksyms() and init386_ksyms()
  and move to xen_init_ksyms(), used for both PV and PVH
- set ISA no-legacy-devices property for PVH
- factor out code from Xen's cpu_bootconf() to xen_bootconf()
  in xen_machdep.c
- set up a specific pvh_consinit() which starts with printk()
  (which uses a simple hypercall that is available early) and switch to
  xencons when we can use pmap_kenter_pa().

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

Mostly merge changes from HEAD upto 20200411

Get rid of xen-specific ci_x* interrupt handling:
- use the general SIR mechanism, reserving 3 more slots for IPL_VM, IPL_SCHED
  and IPL_HIGH
- remove specific handling from C sources, or change to ipending
- convert IPL number to SIR number in various places
- Remove XUNMASK/XPENDING in assembly or change to IUNMASK/IPENDING
- remove Xen-specific ci_xsources, ci_xmask, ci_xunmask, ci_xpending from
  struct cpu_info
- for now remove a KASSERT that there are no pending interrupts in
  idle_block(). We can get there with some software interrupts pending
  in autoconf XXX needs to be looked at.

Include ci_isources[] for XenPV too.
Adjust spllower() to XenPV needs, and switch XenPV to the native spllower().
Remove xen_spllower().

Remove VM_GUEST_XEN and define only Xen subtypes:
VM_GUEST_XENPV
VM_GUEST_XENPVH
VM_GUEST_XENHVM
VM_GUEST_XENPVHVM

Set vm_guest in the start routine, if it is hypervisor-specific (e.g Xen PV).
If vm_guest was not set early and we detect Xen in identify_hypervisor(),
assume it is VM_GUEST_XENHVM. Refine to VM_GUEST_PVXENHVM in
hypervisor_match().

Merge changes from current as of 20200406

Sync with head.

G/C LS{SLEEP,RUN,ONPROC}

LWP status manipulation was moved out of assembly long ago.

Back out previous.

Adaptive rwlocks proposed on tech-kern and working well on this branch
with vnode locks.

Sync with head.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
  calling cpu_switchto().  It's not safe to let other actors mess with the
  LWP (in particular l->l_cpu) while it's still context switching.  This
  removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
  it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead.  Everything
  is in cache anyway so it wasn't buying much by trying to avoid saving old
  state.  This means cpu_switchto() will never be called with prevlwp ==
  NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

Fix a problem with intr_unmask() that can cause a forever-loop:
- When handling the source-is-masked case in the interrupt vector, set the
  interrupt bit in a new ci_imasked field and ensure the bit is cleared
  from ci_ipending.
- In intr_unmask(), transfer the bit from ci_imasked to ci_ipending for
  non-level-sensitive interrupts (the PIC does the work for us in the
  level-sensitive case), and only force pending interrupts to be processed
  in this case.  (In all cases, make sure the now-unmasked bit is cleared
  from ci_imasked.)

Before, the bit was left in ci_ipending so as not to use edge-triggered
interrupts while the source is masked, but Xspllower() relies on the
pending bits getting cleared.

Tested by forcing all wm(4) interrupts on my test system though an
intr_mask() / softint / intr_unmask() cycle and exercising the network
heavily.

Add intr_mask() and corresponding intr_unmask() calls that allow specific
interrupt lines / sources to be masked as needed (rather than making a
set of sources by IPL as with spl*()).

x86: abort pageidlezero unconditionally if there is something to run.

Rewrite the FPU code on x86. This greatly simplifies the logic and removes
the dependency on IPL_HIGH. NVMM is updated accordingly. Posted on
port-amd64 a week ago.

Bump the kernel version to 9.99.16.

Sync with HEAD

Add PCID support in SVS. This avoids TLB flushes during kernel<->user
transitions, which greatly reduces the performance penalty introduced by
SVS.

We use two ASIDs, 0 (kern) and 1 (user), and use invpcid to flush pages
in both ASIDs.

The read-only machdep.svs.pcid={0,1} sysctl is added, and indicates whether
SVS+PCID is in use.

Remove 'ci_svs_kpdirpa', unused. While here fix a few comments here and
there, reduces a future diff.

Pull up following revision(s) via patch (requested by nonaka in ticket #1210):

	sys/dev/hyperv/vmbusvar.h: revision 1.1
	sys/dev/hyperv/hvs.c: revision 1.1
	sys/dev/hyperv/if_hvn.c: revision 1.1
	sys/dev/hyperv/vmbusic.c: revision 1.1
	sys/arch/x86/x86/lapic.c: revision 1.69
	sys/arch/x86/isa/clock.c: revision 1.34
	sys/arch/x86/include/intrdefs.h: revision 1.22
	sys/arch/i386/conf/GENERIC: revision 1.1201
	sys/arch/x86/x86/hyperv.c: revision 1.1
	sys/arch/x86/include/cpu.h: revision 1.105
	sys/arch/x86/x86/x86_machdep.c: revision 1.124
	sys/arch/i386/conf/GENERIC: revision 1.1203
	sys/arch/amd64/amd64/genassym.cf: revision 1.74
	sys/arch/i386/conf/GENERIC: revision 1.1204
	sys/arch/amd64/conf/GENERIC: revision 1.520
	sys/arch/x86/x86/hypervreg.h: revision 1.1
	sys/arch/amd64/amd64/vector.S: revision 1.69
	sys/dev/hyperv/hvshutdown.c: revision 1.1
	sys/dev/hyperv/hvshutdown.c: revision 1.2
	sys/dev/usb/if_urndisreg.h: file removal
	sys/arch/x86/x86/cpu.c: revision 1.167
	sys/arch/x86/conf/files.x86: revision 1.107
	sys/dev/usb/if_urndis.c: revision 1.20
	sys/dev/hyperv/vmbusicreg.h: revision 1.1
	sys/dev/hyperv/hvheartbeat.c: revision 1.1
	sys/dev/hyperv/vmbusicreg.h: revision 1.2
	sys/dev/hyperv/hvheartbeat.c: revision 1.2
	sys/dev/hyperv/files.hyperv: revision 1.1
	sys/dev/ic/rndisreg.h: revision 1.1
	sys/arch/i386/i386/genassym.cf: revision 1.111
	sys/dev/ic/rndisreg.h: revision 1.2
	sys/dev/hyperv/hyperv_common.c: revision 1.1
	sys/dev/hyperv/hvtimesync.c: revision 1.1
	sys/dev/hyperv/hypervreg.h: revision 1.1
	sys/dev/hyperv/hvtimesync.c: revision 1.2
	sys/dev/hyperv/vmbusicvar.h: revision 1.1
	sys/dev/hyperv/if_hvnreg.h: revision 1.1
	sys/arch/x86/x86/lapic.c: revision 1.70
	sys/arch/amd64/amd64/vector.S: revision 1.70
	sys/dev/ic/ndisreg.h: revision 1.1
	sys/arch/amd64/conf/GENERIC: revision 1.516
	sys/dev/hyperv/hypervvar.h: revision 1.1
	sys/arch/amd64/conf/GENERIC: revision 1.518
	sys/arch/amd64/conf/GENERIC: revision 1.519
	sys/arch/i386/conf/files.i386: revision 1.400
	sys/dev/acpi/vmbus_acpi.c: revision 1.1
	sys/dev/hyperv/vmbus.c: revision 1.1
	sys/dev/hyperv/vmbus.c: revision 1.2
	sys/arch/x86/x86/intr.c: revision 1.144
	sys/arch/i386/i386/vector.S: revision 1.83
	sys/arch/amd64/conf/files.amd64: revision 1.112

separate RNDIS definitions from urndis(4) for use with Hyper-V NetVSC.

 -

Added Microsoft Hyper-V support.  It ported from OpenBSD and FreeBSD.
graphical console is not work on Gen.2 VM yet. To use the serial console,
enter "consdev com,0x3f8,115200" on efiboot.

 -

Add __diagused.

 -

PR/53984: Partial revert of modify lapic_calibrate_timer() in lapic.c r1.69.

 -

Update Hyper-V related drivers description.

 -

Remove unused definition.

 -

Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly
discussed on irc.
NFCI intended.

 -

commented out hvkvp entry.

 -

fix typo. pointed out by pgoyette@n.o.

 -

Use IDTVEC instead of NENTRY for handle_hyperv_hypercall.

 -

Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly
discussed on irc.

Added Microsoft Hyper-V support.  It ported from OpenBSD and FreeBSD.

graphical console is not work on Gen.2 VM yet. To use the serial console,
enter "consdev com,0x3f8,115200" on efiboot.

We reorganise definitions for XEN source support as follows:

XEN - common sources required for baseline XEN support.
XENPV - sources required for support of XEN in PV mode.
XENPVHVM - sources required for support for XEN in HVM mode.
XENPVH - sources required for support for XEN in PVH mode.

Switch NetBSD/xen to use XEN api tag RELEASE-4.11.1

The headers for this api are in sys/external/mit/xen-include-public/dist/

Sync with HEAD, resolve a few conflicts

Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

More ASLR: randomize the location of the PTE area. The PTE slot is not
created in locore anymore, but a little later; by using the already
entered L4 page, rather than the recursive slot itself (which doesn't
exist yet).

In the prekern we still map the slot - the prekern behaves as an external
locore -, because we need it as part of the randomization/relocation
work. The kernel then removes this slot, and regenerates a randomized
one.

Tested on GENERIC and GENERIC_KASLR, Xen doesn't have it and dom0 still
boots fine.

Eliminate the only ASM reference to VM_MIN_KERNEL_ADDRESS. Rename the
value to VM_SPACE_SEP_HIGH32, it is now the highest 32bits of the first
va of the higher half of the address space (right after the canonical
hole).

Sync with HEAD.  77 conflicts resolved - all of them $NetBSD$

Fix warning when compiling Xen; FLAT_RING3_CS64 is defined in a child of
xen.h, which is already included in genassym.cf. So don't redefine it.

Pull up the following revisions, requested by maxv in ticket #652:

	sys/arch/amd64/amd64/amd64_trap.S	upto 1.39 (partial, patch)
	sys/arch/amd64/amd64/db_machdep.c	1.6 (patch)
	sys/arch/amd64/amd64/genassym.cf	1.65,1.66,1.67 (patch)
	sys/arch/amd64/amd64/locore.S		upto 1.159 (partial, patch)
	sys/arch/amd64/amd64/machdep.c		1.299-1.302 (patch)
	sys/arch/amd64/amd64/trap.c		upto 1.113 (partial, patch)
	sys/arch/amd64/amd64/amd64/vector.S	upto 1.61 (partial, patch)
	sys/arch/amd64/conf/GENERIC		1.477,1.478 (patch)
	sys/arch/amd64/conf/kern.ldscript	1.26 (patch)
	sys/arch/amd64/include/frameasm.h	upto 1.37 (partial, patch)
	sys/arch/amd64/include/param.h		1.25 (patch)
	sys/arch/amd64/include/pmap.h		1.41,1.43,1.44 (patch)
	sys/arch/x86/conf/files.x86		1.91,1.93 (patch)
	sys/arch/x86/include/cpu.h		1.88,1.89 (patch)
	sys/arch/x86/include/pmap.h		1.75 (patch)
	sys/arch/x86/x86/cpu.c			1.144,1.146,1.148,1.149 (patch)
	sys/arch/x86/x86/pmap.c			upto 1.289 (partial, patch)
	sys/arch/x86/x86/vm_machdep.c		1.31,1.32 (patch)
	sys/arch/x86/x86/x86_machdep.c		1.104,1.106,1.108 (patch)
	sys/arch/x86/x86/svs.c			1.1-1.14
	sys/arch/xen/conf/files.compat		1.30 (patch)

Backport SVS. Not enabled yet.

Pullup the following revisions via patch, requested by maxv in ticket #629:

	sys/arch/amd64/amd64/genassym.cf		1.63,1.64
	sys/arch/amd64/amd64/locore.S			1.144
	sys/arch/amd64/amd64/machdep.c			1.281-1.283
	sys/arch/i386/i386/genassym.cf			1.105-1.106
	sys/arch/i386/i386/locore.S			1.155
	sys/arch/i386/i386/machdep.c			1.802 (adapted),1.803
	sys/arch/x86/include/cpu.h			1.85
	sys/arch/x86/x86/intr.c				1.115-1.116
	sys/arch/x86/x86/pmap.c				1.275
	sys/arch/x86/x86/sys_machdep.c			1.45
	sys/arch/xen/x86/cpu.c				1.117

Stop sharing the double-fault stack.
Merge the TSS structures into one single cpu_tss structure, and
allocate it dynamically.

Unmap the kernel heap from the user page tables (SVS).

This implementation is optimized and organized in such a way that we
don't need to copy the kernel stack to a safe place during user<->kernel
transitions. We create two VAs that point to the same physical page; one
will be mapped in userland and is offset in order to contain only the
trapframe, the other is mapped in the kernel and maps the entire stack.

Sent on tech-kern@ a week ago.

make this compile again

Add a new option, SVS (for Separate Virtual Space), that unmaps kernel
pages when running in userland. For now, only the PTE area is unmapped.

Sent on tech-kern@.

Allocate the TSS area dynamically. This way cpu_info and cpu_tss can be
put in separate pages.

Group the different TSSes into a cpu_tss structure. And pack this
structure to make sure there is no padding between 'tss' and 'iomap'.

update from HEAD

Use FLAT_RING3_CS64 (defined in Xen public headers) instead of numeric
value.

Sync with HEAD

2302677

file genassym.cf was added on branch perseant-stdc-iso10646 on 2017-07-16 14:02:49 +0000

Unify the xen and native x86/ interrupt setup functions and
spl traversal data structures.

This is towards PVHVM.

Sync with HEAD (as of 26th Dec)

A few changes:
 - remove cpu_id and cpu_brand_id (unused)
 - copy a comment from i386 about fillkpt
 - define PDE_SIZE (i386)

Sync with HEAD

Have MI genassym.cf.

Replace COHERENCY_UNIT in ldscript.

Improve spelling: regsister -> register

This change is non-functional.

Approved by <riastradh>, <pgoyette>

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Move the amd64 and i386 pcb to the bottom of the uarea, and move the
  kernel stack to the top.
Change the pcb layouts so that fpu save area is at the end and is
  64byte aligned ready for xsave (saving the ymm registers).
Welcome to 6.99.32

Remove all references to MDL_USEDFPU and deferred fpu initialisation.
The cost of zeroing the save area on exec is minimal.
This stops the FP registers of a random process being used the first
  time an lwp uses the fpu.
sendsig_siginfo() and get_mcontext() now unconditionally copy the FP
registers.
I'll remove the double-copy for signal handlers soon.
get_mcontext() might have been leaking kernel memory to userspace - and
  may still do so if i386_use_fxsave is false (short copies).

revert fpu/pcu changes until we figure out what's wrong; they cause random
freezes

Use the MI "pcu" framework for bookkeeping of npx/fpu states on x86.
This reduces the amount of MD code enormously, and makes it easier
to implement support for newer CPU features which require more fpu
state, or for fpu usage by the kernel.
For access to FPU state across CPUs, an xcall kthread is used now
rather than a dedicated IPI.
No user visible changes intended.

Pull up following revision(s) (requested by chs in ticket #690):
	external/cddl/osnet/dev/dtrace/amd64/dtrace_isa.c: revision 1.4
	sys/arch/amd64/include/Makefile.inc: revision 1.4
	sys/arch/amd64/include/pmap.h: revision 1.33
	external/cddl/osnet/dev/dtrace/amd64/dtrace_subr.c: revision 1.6
	sys/arch/amd64/include/asm.h: revision 1.15
	sys/arch/amd64/amd64/genassym.cf: revision 1.51
	external/cddl/osnet/dev/dtrace/amd64/dtrace_asm.S: revision 1.4
make dtrace work on amd64.
allow more space for modules.

sync with head

Rename MDP_IRET to MDL_IRET since it is an lwp flag, not a proc one.
Add an MDL_COMPAT32 flag to the lwp's md_flags, set it for 32bit lwps
  and use it to force 'return to user' with iret (as is done when
  MDL_IRET is set).
Split the iret/sysret code paths much later.
Remove all the replicated code for 32bit system calls - which was only
  needed so that iret was always used.
frameasm.h for XEN contains '#define swapgs', while XEN probable never
  needs swapgs, this is likely to be confusing.
Add a SWAPGS which is a nop on XEN and swapgs otherwise.
(I've not yet checked all the swapgs in files that include frameasm.h)
Simple x86 programs still work.
Hijack 6.99.9 kernel bump (needed for compat32 modules)

make dtrace work on amd64.

sync with head.

Pull up following revision(s) (requested by rmind in ticket #202):
	sys/arch/x86/include/cpuvar.h: revision 1.46
	sys/arch/xen/include/xenpmap.h: revision 1.34
	sys/arch/i386/include/param.h: revision 1.77
	sys/arch/x86/x86/pmap_tlb.c: revision 1.5
	sys/arch/x86/x86/pmap_tlb.c: revision 1.6
	sys/arch/i386/i386/genassym.cf: revision 1.92
	sys/arch/xen/x86/cpu.c: revision 1.91
	sys/arch/x86/x86/pmap.c: revision 1.177
	sys/arch/xen/x86/xen_pmap.c: revision 1.21
	sys/arch/x86/acpi/acpi_wakeup.c: revision 1.31
	sys/kern/subr_kcpuset.c: revision 1.5
	sys/arch/amd64/include/param.h: revision 1.18
	sys/sys/kcpuset.h: revision 1.5
	sys/arch/x86/x86/mtrr_i686.c: revision 1.26
	sys/arch/x86/x86/mtrr_i686.c: revision 1.27
	sys/arch/xen/x86/x86_xpmap.c: revision 1.43
	sys/arch/x86/x86/cpu.c: revision 1.98
	sys/arch/amd64/amd64/mptramp.S: revision 1.14
	sys/kern/sys_sched.c: revision 1.42
	sys/arch/amd64/amd64/genassym.cf: revision 1.50
	sys/arch/i386/i386/mptramp.S: revision 1.24
	sys/arch/x86/include/pmap.h: revision 1.52
	sys/arch/x86/include/cpu.h: revision 1.50
- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
  kcpuset(9) and thus replace hardcoded CPU bitmasks.  This removes the
  limitation of maximum CPUs.
- Support up to 256 CPUs on amd64 architecture by default.
Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.
- pmap_tlb_shootdown: do not overwrite tp_cpumask with pm_cpus, but merge
  like pm_kernel_cpus.  Remove unecessary intersection with kcpuset_running.
  Do not reset tp_userpmap if pmap_kernel().
- Remove pmap_tlb_mailbox_t wrapping, which is pointless after recent changes.
- pmap_tlb_invalidate, pmap_tlb_intr: constify for packet structure.
i686_mtrr_init_first: handle the case when there are no variable-size MTRR
registers available (i686_mtrr_vcnt == 0).

sync to latest -current.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
  kcpuset(9) and thus replace hardcoded CPU bitmasks.  This removes the
  limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

sync with head

merge to -current.

switch from xen3-public to xen-public.

Sync with HEAD. Most notably: uvm/pmap work done by rmind@, and MP Xen
work of cherry@.

No regression observed on suspend/restore.

#include <sys/bus.h> instead of <machine/bus.h>.

Catchup with rmind-uvmplock merge.

Welcome to 5.99.53!  Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

- Fix tlbflushg() to behave like tlbflush(), if page global extension (PGE)
  is not (yet) enabled.  This fixes the issue of stale TLB entry, experienced
  early on boot, when PGE is not yet set on primary CPU.
- Rewrite i386/amd64 TLB interrupt handlers in C (only stubs are in assembly),
  which simplifies and unifies (under x86) code, plus fixes few bugs.
- cpu_attach: remove assignment to cpus_running, as primary CPU might not be
  attached first, which causes reset (and thus missed secondary CPUs).

sync with head

Sync with HEAD

Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits.  Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.

Sync with HEAD

Sync with HEAD.

sync with head.

add the guts of TLS support on amd64.  based on joerg's patch,
reworked by me to support 32-bit processes as well.
we now keep %fs and %gs loaded with the user values
while in the kernel, which means we don't need to
reload them when returning to user mode.

sync with head

Sync with HEAD.

On x86, change the bus_space_tag_t to a pointer to a struct
bus_space_tag.  For now, bus_space_tag's only member is
bst_type, the type of space, which is either X86_BUS_SPACE_IO
or X86_BUS_SPACE_MEM.  In the future, new bus_space_tag members
will refer to override-functions installed by a new function,
bus_space_tag_create(9).

Add pointers to constant struct bus_space_tag, x86_bus_space_io and
x86_bus_space_mem.  Use them to replace most uses of X86_BUS_SPACE_IO
and X86_BUS_SPACE_MEM.

Add an x86-specific bus_space_is_equal(9) implementation that compares
the two tags' bst_type.

Partly rewrite amd64 TLB shutdown handler for the changes in x86 pmap.
At this point, branch seems to pass preliminar stress tests on amd64.

sync with head

- Use uvm_lwp_setuarea() instead of directly setting address to lwp_t::l_addr.
- Replace most remaining uses of l_addr with uvm_lwp_getuarea() or lwp_getpcb().
- Amend assembly in ports where it accesses PCB via struct user.
- Rename L_ADDR to L_PCB in few places.  Reduce sys/user.h inclusions.

Remove IPL_LPT and IPL_IPI aliases, use the actual IPLs.
Fix some broken comments.

Use lwp_getpcb() on x86 MD code, clean from struct user usage.

sync with head.

Pull up following revision(s) (requested by ad in ticket #345):
	sys/arch/amd64/amd64/genassym.cf: revision 1.39
	sys/arch/i386/i386/genassym.cf: revision 1.79
	sys/arch/i386/i386/locore.S: revision 1.82
	sys/arch/x86/x86/x86_machdep.c: revision 1.26
Don't abort pageidlezero unless a realtime thread wants to run.

Sync with HEAD.

Sync with HEAD.

!defined(_LKM) -> defined(_KERNEL_OPT)

Update haad-dm branch to haad-dm-base2.

Don't abort pageidlezero unless a realtime thread wants to run.

Pull up following revision(s) (requested by ad in ticket #74):
	sys/arch/i386/isa/npx.c: revision 1.131
	sys/arch/amd64/amd64/fpu.c: revision 1.28
	sys/arch/i386/i386/genassym.cf: revision 1.77
	sys/arch/i386/i386/autoconf.c: revision 1.93
	sys/arch/amd64/amd64/locore.S: revision 1.48
	sys/arch/amd64/amd64/machdep.c: revision 1.104
	sys/arch/i386/i386/machdep.c: revision 1.646
	sys/arch/amd64/amd64/genassym.cf: revision 1.38
	sys/arch/i386/i386/locore.S: revision 1.79
PR port-amd64/38293 panic: fp_save ipi didn't
Fix race conditions in FPU IPI handling.

PR port-amd64/38293 panic: fp_save ipi didn't

Fix race conditions in FPU IPI handling.

Sync with HEAD.

Sync with HEAD.

Sync with HEAD.

Remove advertising clause for UCB in various genassym.cf files,
which were derived from genassym.c in 4.4BSD-Lite2 (or 386BSD).
Closes PR misc/39573.  Approved by martin@.

Sync with wrstuden-revivesa-base-2.

Sync with head.

Remove "struct device" from "struct pic", where it was only real
for ioapics and faked up for others. Add it to "struct ioapic_softc"
for now, until device/softc get split.
This required all typecasts between "struct pic" and "struct ioapic_softc"
to be replaced, I hope I got them all.
functionally tested on i386, compile-tested on xen, untested on amd64

Sync w/ -current. 34 merge conflicts to follow.

sync with head.

Sync with HEAD.

Also fix build.

Remove ref to uvm.page_idle_zero

sync with head

Sync with HEAD.

Oops, add CPU_INFO_NTRAP.

sync with head.

sync with head.

Don't reload LDTR unless a new value, which only happens for USER_LDT.

Share cpu.h between the x86 ports.

Improve x86 tsc handling:

- Ditch the cross-CPU calibration stuff. It didn't work properly, and it's
  near impossible to synchronize the CPUs in a running system, because bus
  traffic will interfere with any calibration attempt, messing up the
  timings.

- Only enable the TSC on CPUs where we are sure it does not drift. If we are
  On a known good CPU, give the TSC high timecounter quality, making it the
  default.

- When booting CPUs, detect TSC skew and account for it. Most Intel MP
  systems have synchronized counters, but that need not be true if the
  system has a complicated bus structure. As far as I know, AMD systems
  do not have synchronized TSCs and so we need to handle skew.

- While an AP is waiting to be set running, try and make the TSC drift by
  entering a reduced power state. If we detect drift, ensure that the TSC
  does not get a high timecounter quality. This should not happen and is
  only for safety.

- Make cpu_counter() stuff LKM safe.

Add support for kernel preeemption to the i386 and amd64 ports. Notes:

- I have seen one isolated panic in the x86 pmap, but otherwise i386
  seems stable with preemption enabled.

- amd64 is missing the FPU handling changes and it's not yet safe to
  enable it there.

- The usual level for kern.sched.kpreempt_pri will be 128 once enabled
  by default. For testing, setting it to 0 helps to shake out bugs.

Remove clause 3 and 4 from TNF licenses

Make kernel pagefault handling preeemption safe.

Access Xen's vcpu info structure per-CPU.
Tested on i386 and amd64 (both dom0 and domU) by me.
Xen2 tested (both dom0 and domU) by bouyer.
OK bouyer

Sync with HEAD.

Correct pasto in the cc_microtime defs. From tsutsui@.

sync with head.

sync with HEAD

sync with head.

Make context switch + syscall counters optionally per-CPU and accumulate
in schedclock() at "about 16 hz".

Implement an optimized, preemption-safe asm version of tsc_get_timecount().
The C version needs work to be preemption safe. Cuts the clock cycles
for microtime() from 950 down to 300 on a Pentium D.

Sync with HEAD.

sync with head

sync with HEAD

Sync with HEAD

- make amd64 use per-cpu tss.
- fix iopl syscall for amd64+xen.

Sync with HEAD

make the nmi handler safer.

XXX should use IST.

Sync with HEAD.

Sync with head.

Create the trap/syscall frame space for all the registers in one go.
Use the tramp-frame offsets (TF_foo) for all references to the registers.
Sort the saving of the GP registers into the same order as the trap frame
because consequetive memory accesses are liekly to be faster.

Add new IPI for saving CPU state explicitly, share high-level part of
ACPI wakeup code and teach it how to start the APs again. As a side
effect the CPU_START interface allows choosing between different
bootstrap codes more easily now.

Sync with HEAD.

Sync with HEAD.

sync with head

Sync with HEAD.

Sync with HEAD.

Interrupt handling changes, in discussion since February:

- Reduce available SPL levels for hardware devices to none, vm, sched, high.
- Acquire kernel_lock only for interrupts at IPL_VM.
- Implement threaded soft interrupts.

Sync with HEAD. amd64 Xen support needs testing.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

Sync with HEAD.

Sync with HEAD

put back IPL_IPI as it's still used.

IPL_IPI -> IPL_HIGH

Initial domain0 support for xenamd64. The kernel boots multiuser, but
xen tools have not been tried yet.
In this process, cleanup some more the page table bootstrap, and properly
handle event counters for soft interrupts.

sync with head.

Sync with HEAD

sync with HEAD

Sync with HEAD.

Fallout from fixing the linux32 system calls to use the correct kernel
functions and parameter layouts.

sync with head.

Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.

Finish sync with HEAD. Especially use the new x86 pmap for xenamd64.
For this:
- rename pmap_pte_set() to pmap_pte_testset()
- make pmap_pte_set() a function or macro for non-atomic PTE write
- define and use pmap_pa2pte()/pmap_pte2pa() to read/write PTE entries
- define pmap_pte_flush() which is a nop in x86 case, and flush the
  MMUops queue in the Xen case

Sync with head.

merge yamt-x86pmap branch.

- reduce differences between amd64 and i386.  notably, share pmap.c
  between them.  it makes several i386 pmap improvements available to
  amd64, including tlb shootdown reduction and bug fixes from Stephan Uphoff.
- implement deferred pmap switching for amd64.
- remove LARGEPAGES option.  always use large pages if available.
  also, make it work on amd64.

amd64 (aka x86-64) support for Xen. Based on the OpenBSD port done by
Mathieu Ropert in 2006.
DomU-only for now. An INSTALL_XEN3_DOMU kernel with a ramdisk will boot to
sysinst if you're lucky. Often it panics because a runable LWP has
a NULL stack (really, it's all of l->l_addr which is has been zeroed out
while the process was on the queue !)
TODO:
- bug fixes :)
- Most of the xpq_* functions should be shared with xen/i386
- The xen/i386 assembly bootstrap code should be remplaced with the C
  version in xenamd64/amd64/xpmap.c
- see if a config(5) trick could allow to merge xenamd64 back to xen or amd64.

Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree.  Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches.  The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

Sync with head.

fix DIAGNOSTIC

add KERNTEXTOFF.

whitespace changes to reduce diffs.

reduce diffs from i386.

remove some #ifdef _LOCORE and use genassym instead.

sync with head.

Sync with HEAD

Sync with HEAD.

implement deferred pmap switching for amd64, and make amd64 use
x86 shared pmap code.  it makes several i386 pmap improvements available
to amd64, including tlb shootdown reduction and bug fixes from Stephan Uphoff.

x86 changes for pcc and LKMs.

- Replace most inline assembly with proper functions. As a side effect
  this reduces the size of amd64 GENERIC by about 120kB, and i386 by a
  smaller amount. Nearly all of the inlines did something slow, or something
  that does not need to be fast.
- Make curcpu() and curlwp functions proper, unless __GNUC__ && _KERNEL.
  In that case make them inlines. Makes curlwp LKM and preemption safe.
- Make bus_space and bus_dma more LKM friendly.
- Share a few more files between the ports.
- Other minor changes.

ci_astpending is no more.

Sync with HEAD.

sync with head.

Sync with HEAD.

Merge most x86 changes from the vmlocking branch, except the threaded soft
interrupt stuff. This is mostly comprised of changes to the pmap modules to
work on multiprocessor systems without kernel_lock, and changes to speed up
tlb shootdowns.

amd64 changes, as yet untested:

- Adapt to vmlocking branch.
- Apply TLB shootdown and pv allocation changes to the pmap.
- Make it build.

Sync with head.

Sync with head.

Update to HEAD.

merge yamt-idlelwp branch.  asked by core@.  some ports still needs work.

from doc/BRANCHES:

	idle lwp, and some changes depending on it.

	1. separate context switching and thread scheduling.
	   (cf. gmcgarry_ctxsw)
	2. implement idle lwp.
	3. clean up related MD/MI interfaces.
	4. make scheduler(s) modular.

PK_SYSTEM -> LW_SYSTEM

remove L_FORW and L_BACK.

adapt amd64.

XXX changes in identcpu.c is minmum for MONITOR.
XXX identcpu.c should be shared with i386.

- sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.

sync with head.

Fix a couple missed occurrences of P_32 -> PK_32.

Change the process/lwp flags seen by userland via sysctl back to the
P_*/L_* naming convention, and rename the in-kernel flags to avoid
conflict. (P_ -> PK_, L_ -> LW_ ). Add back the (now unused) LSDEAD
constant.

Restores source compatibility with pre-newlock2 tools like ps or top.

Reviewed by Andrew Doran.

Merge newlock2 to head.

If running on a PPro or later, at boot patch in versions of spllower() and
similar that use cmpxchg8b instead of cli/sti. Cuts the clock cycles for
splx() by a factor of ~6 on the P4, and ~3 on the PIII when bracketed by
serializing instructions (and hopefully more when not).

Make amd64 build again.

Checkpoint work in progress.

Checkpoint work in progress.

Checkpoint work in progress.

Export/remove new symbols to asm code.

Add updated locking primatives.

sync with head

sync with head.

Sync with head.

sync with head.

Add initial (but unfinished) COMPAT_LINUX32 for amd64. This is good enough so
that the i386 license manager part of amd64 version of Fluent works.

While I'm here, add SysV IPC to COMPAT_LINUX/amd64

merge ktrace-lwp.

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Remove p_nras from struct proc - use LIST_EMPTY(&p->p_raslist) instead.
Remove p_raslock and rename p_lwplock p_lock (one lock is enough).
Simplify window test when adding a ras and correct test on VM_MAXUSER_ADDRESS.
Avoid unpredictable branch in i386 locore.S
(pad fields left in struct proc to avoid kernel bump)

Follow i386, and mask deferred level-triggered interrupts at the ioapic.

Rename the x86_64 port to amd64, as this is the actual name used for
the processor family now. x86_64 is kept as the MACHINE_ARCH value,
since it's already widely used (by e.g. the toolchain, etc), and
by other operating systems.