CVS log for src/sys/arch/xen/xen/xen_machdep.c

Up to [cvs.NetBSD.org] / src / sys / arch / xen / xen

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Pull up following revision(s) (requested by bouyer in ticket #428):

	sys/arch/xen/xen/xen_machdep.c: revision 1.28
	sys/arch/x86/pci/pci_machdep.c: revision 1.97
	sys/arch/xen/xen/genfb_xen.c: revision 1.1
	sys/arch/xen/xen/genfb_xen.c: revision 1.2
	sys/arch/xen/include/hypervisor.h: revision 1.59
	sys/arch/i386/conf/XEN3PAE_DOM0: revision 1.41 (patch)
	sys/arch/x86/x86/genfb_machdep.c: revision 1.22
	sys/arch/xen/x86/consinit.c: revision 1.18
	sys/arch/xen/x86/autoconf.c: revision 1.26
	sys/external/mit/xen-include-public/dist/xen/include/public/platform.h: revision 1.2
	sys/arch/xen/conf/files.xen: revision 1.188
	sys/arch/x86/x86/consinit.c: revision 1.37
	sys/arch/xen/conf/files.xen: revision 1.189
	sys/arch/x86/x86/consinit.c: revision 1.38
	sys/external/mit/xen-include-public/dist/xen/include/public/xen.h: revision 1.2
	sys/arch/x86/include/genfb_machdep.h: revision 1.7
	sys/arch/xen/x86/pvh_consinit.c: revision 1.5
	sys/arch/xen/x86/pvh_consinit.c: revision 1.6
	sys/arch/amd64/conf/XEN3_DOM0: revision 1.201

Move the pvh_xencons so xen_machdep.c as early_xencons, so it can be
used in the future as early ouput for plain PV guests too.

Support non-VGA framebuffers for Xen dom0. This is mandatory for graphic
console on EFI-only hardware.

Add a xen_genfb_getbtinfo() function which will return a btinfo_framebuffer
    structure, filled in with parameters provided by Xen

when runing as a Xen dom0, call xen_genfb_getbtinfo() instead of
    lookup_bootinfo(BTINFO_FRAMEBUFFER) when adding properties to the
    PCI graphic device (when genfb is attached) and in x86_genfb_init()
    when genfb is used as console.

x86/x86/consinit.c: If running as a Xen dom0, use xen_genfb_getbtinfo()
    to check if we have a genfb console

xen/x86/consinit.c: support genfb as possible console

xen/x86/consinit.c: use the hypervior IO as console until a better one
    is found. If the hypervisor is using a serial port for boot messages,
    we'll get NetBSD's boot message on the serial port too until
    the real console takes over.

xen/x86/autoconf.c: rework device_register() to be closer to the x86 version.
    Especially make sure that device_pci_register() is called.

Make sure to always fall back to xen_early_console, even for dom0

Enable genfb in DOM0 kernels

Add ext_lfb_base to dom0_vga_console_info, from recent Xen. We know if it's
present or not by checking dom0.info_size

Add XENPF_get_dom0_console, which gets a dom0_vga_console_info stucture
from the hypervisor. To be used by PVH dom0 kernels.

XENPVH option is not used. Fix consinit.c to use XENPVHVM as intended
and XENPVH from defflag
for a dom0 PVH, the dom0_vga_console_info structure has to be retrieved
using a platform hypercall; do so in the XENPVHVM case.

Now genfb works in a PVH dom0 running on Xen 4.18 (Xen 4.15 doesn't support
this platoform op, so no way to make it work here).

xen_machdep.c: Nix trailing whitespace.

No functional change intended.

Move the pvh_xencons so xen_machdep.c as early_xencons, so it can be
used in the future as early ouput for plain PV guests too.

x86: Split most of pmap.h into pmap_private.h or vmparam.h.

This way pmap.h only contains the MD definition of the MI pmap(9)
API, which loads of things in the kernel rely on, so changing x86
pmap internals no longer requires recompiling the entire kernel every
time.

Callers needing these internals must now use machine/pmap_private.h.
Note: This is not x86/pmap_private.h because it contains three parts:

1. CPU-specific (different for i386/amd64) definitions used by...

2. common definitions, including Xenisms like xpmap_ptetomach,
   further used by...

3. more CPU-specific inlines for pmap_pte_* operations

So {amd64,i386}/pmap_private.h defines 1, includes x86/pmap_private.h
for 2, and then defines 3.  Maybe we should split that out into a new
pmap_pte.h to reduce this trouble.

No functional change intended, other than that some .c files must
include machine/pmap_private.h when previously uvm/uvm_pmap.h
polluted the namespace with pmap internals.

Note: This migrates part of i386/pmap.h into i386/vmparam.h --
specifically the parts that are needed for several constants defined
in vmparam.h:

VM_MAXUSER_ADDRESS
VM_MAX_ADDRESS
VM_MAX_KERNEL_ADDRESS
VM_MIN_KERNEL_ADDRESS

Since i386 needs PDP_SIZE in vmparam.h, I added it there on amd64
too, just to keep things parallel.

 G.C. No functional change. OK'd by bouyer@.

Remove duplicate def of tsc_get_timecount().

Introduce Xen PVH support in GENERIC.
This is compiled in with
options XENPVHVM
x86 changes:
- add Xen section and xen pvh entry points to locore.S. Set vm_guest
  to VM_GUEST_XENPVH in this entry point.
  Most of the boot procedure (especially page table setup and switch to
  paged mode) is shared with native.
- change some x86_delay() to delay_func(), which points to x86_delay() for
  native/HVM, and xen_delay() for PVH

Xen changes:
- remove Xen bits from init_x86_64_ksyms() and init386_ksyms()
  and move to xen_init_ksyms(), used for both PV and PVH
- set ISA no-legacy-devices property for PVH
- factor out code from Xen's cpu_bootconf() to xen_bootconf()
  in xen_machdep.c
- set up a specific pvh_consinit() which starts with printk()
  (which uses a simple hypercall that is available early) and switch to
  xencons when we can use pmap_kenter_pa().

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

Reorganise sources to make it possible to include Xen PVHVM support in
native kernels. Among others:
- move xen/include/amd64/hypercall.h to amd64/include/xen and
  xen/include/i386/hypercall.h to i386/include/xen
- exclude some native files from the build for xenpv
- add xen to "machine" config statement for amd64 and i386
- split arch/xen/conf/files.xen to arch/xen/conf/files.xen (for pv drivers)
  and arch/xen/conf/files.xen.pv (for full pv support)
- add GENERIC_XENHVM kernel config which includes GENERIC and add Xen PV
  drivers.

Sync with HEAD

Start replacing the x86 PTE bits.

Further restrict the scope of XENPV to relevant parts.

Switch NetBSD/xen to use XEN api tag RELEASE-4.11.1

The headers for this api are in sys/external/mit/xen-include-public/dist/

Sync with HEAD

Merge the content of xen_debug.c into xen_machdep.c, there is only one
function.

Build fix bandaid.

This allows the builds including XEN to complete, but XEN kernels
built from these sources (at least, DomU) do not boot successfully.

Use xcall to suspend/resume clocks on _all_ CPUs, not just current.

Compile-tested only -- kernel still runs but I haven't exercised this
code path.

By code inspection, this change or something like it is obviously
_necessary_ for Xen MP suspend/resume -- otherwise there's no logic
to suspend/resume the clocks on any CPU except the one that actually
triggers sysctl -w machdep.xen.suspend=1.

Whether this change is _sufficient_ for Xen MP suspend/resume to
work, I don't know.

ok cherry

Sync with HEAD

mark with XXXSMP all remaining spl*() and tsleep() calls

update from HEAD

Sync with HEAD

Feeding uninitialised garbage to the hypervisor is likely to be a bad idea.

Switch to elf notes for amd64 instead of the old key=value list to describe the
  guest requirements and support.
Add infrastructure to query the hypervisor about features support.
For verbose boot, print the features suppoted by the hypervisor for this
  guest.

sync with head

Fix some -fno-common fallout.

Pull up revision 1.12 (requested by jym in ticket #437).

Don't leak stack content when reading the Xen suspend value.

XXX pull-up to -6.

Don't leak stack content when reading the Xen suspend value.

XXX pull-up to -6.

Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map()      map a pseudo-phys address to a machine address
xpmap_ptom_unmap()    unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid()  check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

sync with head

merge to -current.

Xen clock management routines keep track of CPU (following MP merge).
Reflect this change in the suspend/resume routines so they can cope with
domU CPU suspend, instead of setting their cpu_info pointer to NULL.

Avoid copy/pasting by using the resume routines during attachement.

ok releng@.

No regression observed, and allows domU to suspend successfully again.
Restore is a different beast as PD/PT flags are marked "invalid" by Xen-4
hypervisor, and blocks resuming. Looking into it.

Expose pmap_pdp_cache publicly to x86/xen pmap. Provide suspend/resume
callbacks for Xen pmap.

Turn static internal callbacks of pmap_pdp_cache.

XXX the implementation of pool_cache_invalidate(9) is still wrong, and
IMHO this needs fixing before -6. See
http://mail-index.netbsd.org/tech-kern/2011/11/18/msg011924.html

Merge jym-xensuspend branch in -current. ok bouyer@.

Goal: save/restore support in NetBSD domUs, for i386, i386 PAE and amd64.

Executive summary:
- split all Xen drivers (xenbus(4), grant tables, xbd(4), xennet(4))
in two parts: suspend and resume, and hook them to pmf(9).
- modify pmap so that Xen hypervisor does not cry out loud in case
it finds "unexpected" recursive memory mappings
- provide a sysctl(7), machdep.xen.suspend, to command suspend from
userland via powerd(8). Note: a suspend can only be handled correctly
when dom0 requested it, so provide a mechanism that will prevent
kernel to blindly validate user's commands

The code is still in experimental state, use at your own risk: restore
can corrupt backend communications rings; this can completely thrash
dom0 as it will loop at a high interrupt level trying to honor
all domU requests.

XXX PAE suspend does not work in amd64 currently, due to (yet again!)
page validation issues with hypervisor. Will fix.

XXX secondary CPUs are not suspended, I will write the handlers
in sync with cherry's Xen MP work.

Tested under i386 and amd64, bear in mind ring corruption though.

No build break expected, GENERICs and XEN* kernels should be fine.
./build.sh distribution still running. In any case: sorry if it does
break for you, contact me directly for reports.

Rename the functions for suspend to reflect that Xen does not hijack
the ACPI "sleepstate" sysctl(7) node anymore.

Add a boolean value to mark that the save/suspend operation has been
notified by dom0, so as to avoid possible errors where admin would like
to schedule the domain for sleep without dom0 being prepared for that. Fail
with EAGAIN in this case.

Sprinkle some KNF.

Now that Xen sysctl(7) moved under machdep, create a machdep.xen.suspend
node to command suspension.

balloon is now supported, update comment.

Sync with HEAD

sync to netbsd-5

sync with head

- Upgrade suspend/resume code to comply with Xen2 removal.
- Add support for PAE domUs suspend/resume.
- Fix an issue regarding initialization of the xbd ring I/O that could end
badly during resume, with invalid block operations submitted to dom0 backend.

NetBSD supports PAE under x86_32 by considering the L2 page as being
4 pages long instead of 1.

Xen validates the page types during resume. Sadly, the hypervisor handles
alternative recursive mappings (== PG/PD entries pointing to pages other
than self) inadequately, which could lead to incorrect page pinning.

As a result, the important change with this patch is to clear these alternative
mappings during suspend, and reset them back to their former self upon
resume. For PAE, approx. all 4 PDIR_SLOT_PTEs could be considered as
alternative recursive mappings.

See comments in pmap.c for further details.

Now, let the testing and bug hunting begin.

Sync with HEAD.

Remove 3rd and 4th clauses.  OK cl@ (copyright holder).

Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !

Pull up following revision(s) (requested by bouyer in ticket #1054):
	sys/arch/xen/conf/files.xen: revision 1.93
	sys/arch/xen/include/pci_machdep.h: revision 1.11
	sys/arch/xen/include/xen.h: revision 1.31
	sys/arch/xen/xen/hypervisor.c: revision 1.44
	sys/arch/xen/xen/pci_intr_machdep.c: revision 1.8
	sys/arch/xen/xen/pciback.c: revision 1.1
	sys/arch/xen/xen/xen_machdep.c: revision 1.5
	sys/arch/xen/xen/xpci_xenbus.c: revision 1.1
Work in progress on PCI front-end/back-end support
front-end:
- add a xpci* at xenbus? which provides pci busses from the dom0
  xpci provides support routines for PCI config space operations and
  enumeration in xpci_xenbus.c
- hypervisor.c: do dom0-style PCI attach only ifdef DOM0OPS
- pci_intr_machdep.c:  check line value only if DOM0OPS
back-end:
- add a pciback* at pci? device which takes precedences over all
  other PCI devices (match return 500) and matches all devices passed
  to pciback.hide option on boot command line.
  It exports the PCI device informations to files in /kern/xen/pci/
- hypervisor.c: create /kern/xen earlier so pciback can create its
  entries while PCI devices are probed
- xen_machdep.c: add handling for pciback.hide=
frontend is know working on Xen 3.1.x dom0 with ahc(4) and pciide(4)
devices. uhci(4) fail when trying to allocate a large contigous DMA
buffer.
backend is work in progress; support in xentools is not there yet.

- rework the page pinning API, so that now a function is provided for
each level of indirection encountered during virtual memory translations. Update
pmap accordingly. Pinning looks cleaner that way, and it offers the possibility
to pin lower level pages if necessary (NetBSD does not do it currently).

- some fixes and comments to explain how page validation/invalidation take
place during save/restore/migrate under Xen. L2 shadow entries from PAE are now
handled, so basically, suspend/resume works with PAE.

- fixes an issue reported by Christoph (cegger@) for xencons suspend/resume
in dom0.

TODO:

- PAE save/restore is currently limited to single-user only, multi-user
support requires modifications in PAE pmap that should be discussed first. See
the comments about the L2 shadow pages cached in pmap_pdp_cache in this commit.

- grant table bug is still there; do not use the kernels of this branch
to test suspend/resume, unless you want to experience bad crashes in dom0,
and push the big red button.

Now there is light at the end of the tunnel :)

Note: XEN2 kernels will neither build nor work with this branch.

Modifications for the Xen suspend/migrate/resume branch:

- introduce xenbus_device_{suspend,resume}() functions. These are routines
used to suspend/resume MI parts of the Xenbus device interfaces, like updating
frontend/backend devices' paths found in XenStore.

- introduce HYPERVISOR_sysctl(), an hypercall used only by Xentools to obtain
information from hypervisor (listing VMs, printing console, etc.). I use it
to query xenconsole from ddb(), as a last resort in case of a panic() in
dom0 (xm being not available). Currently unused in the branch; could be, if
requested.

- disable the rwlock(9) used to protect code that could use transient MFNs.
It could trigger nasty context switches in place it should not to.

- fix some bugs in the xennet/xbd suspend/resume pmf(9) handlers.

- following XenSource's design, talk_to_otherend() is now called
watch_otherend(), and free_otherend_details() is used by Xenbus device
suspend/resume routines.

- some slight modifications in pmap regarding APDP. Introduce an inline
function (pmap_unmap_apdp_pde()) that clears APDP entry for the current pmap.

- similarly, implement pmap_unmap_all_apdp_pdes() that iterates through all
pmaps and tears down APDP, as Xen does not handle them properly.

TODO/XXX:

- pmap_unmap_apdp_pde() does not handle APDP shadow entry of PAE. It will,
once I figure out how PAE uses it.

- revisit the pmap locking issue regarding transient MFNs. As NetBSD does not
use kernel preemption and MP for Xen, this could be skipped momentarily. See
http://mail-index.netbsd.org/port-xen/2009/04/27/msg004903.html for details.

- fix a bug regarding grant tables which could technically DoS a dom0 if
ridiculously high consumer/producer indexes are passed down in the ring during
a resume.

All in all, once the grant table index issue and APDP PAE are fixed, next step
is to torture test this branch.

Tested under i386 PAE and non-PAE, Xen3 dom0 and domU. amd64 is only compile
tested.

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

Work in progress on PCI front-end/back-end support
front-end:
- add a xpci* at xenbus? which provides pci busses from the dom0
  xpci provides support routines for PCI config space operations and
  enumeration in xpci_xenbus.c
- hypervisor.c: do dom0-style PCI attach only ifdef DOM0OPS
- pci_intr_machdep.c:  check line value only if DOM0OPS

back-end:
- add a pciback* at pci? device which takes precedences over all
  other PCI devices (match return 500) and matches all devices passed
  to pciback.hide option on boot command line.
  It exports the PCI device informations to files in /kern/xen/pci/
- hypervisor.c: create /kern/xen earlier so pciback can create its
  entries while PCI devices are probed
- xen_machdep.c: add handling for pciback.hide=

frontend is know working on Xen 3.1.x dom0 with ahc(4) and pciide(4)
devices. uhci(4) fail when trying to allocate a large contigous DMA
buffer.

backend is work in progress; support in xentools is not there yet.

Initial code for xen save/restore/migrate facilities.

- split the attach code of frontends in two half: one that is only needed
during autoconf(9) attach/detach phases, and one used at each save/restore
of device state (between suspend and resume).

Applies to hypervisor, xencons, xenbus, xbd, and xennet.

- add a rwlock(9) ("ptom_lock") to protect the different parts in the kernel
 that manipulate MFNs (which could change between a suspend and a resume,
without the kernel noticing it). Parts that require MFNs acquire a reader lock,
while suspend code will acquire a writer lock to ensure that no-other parts
in kernel still use MFNs.

- integrate the suspend code with sysmon.

- various things in pmap(9), and clock.

TODO:
- factorize code a bit more inside frontends drivers.
- remove all alternative recursive (APDP_PDE) mappings found in PD/PT during
suspend, as Xen does not support them.
- abstract the ptom_lock locking, it is only required when kernel preemption
is enabled, or on MP systems.

Current code works mostly. You may experience difficulties in some corner
cases (dom0 warnings about xennet interface errors, and Xen tools failing to
 validate NetBSD's alternative pmaps).

Sync w/ -current. 34 merge conflicts to follow.

Sync with HEAD.

sync with head.

sync with head.

Make xen build after tsc changes.

Export tsc_freq via sysctl.

sync with HEAD

Sync with HEAD.

sync with head

Sync with HEAD.

Sync with HEAD. amd64 Xen support needs testing.

file xen_machdep.c was added on branch matt-armv6 on 2008-01-09 01:50:22 +0000

file xen_machdep.c was added on branch yamt-lazymbuf on 2007-12-07 17:27:24 +0000

file xen_machdep.c was added on branch vmlocking on 2007-12-03 19:04:46 +0000

file xen_machdep.c was added on branch jmcneill-pm on 2007-11-27 19:36:32 +0000

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

Prepare for xenamd64:
- kill xen/i386/identcpu.c, use i386/i386/identcpu.c instead (with a few
  #ifndef XEN)
- move some files that can be shared between i386 and amd64 from
  xen/i386 to xen/x86 (or to xen/xen for non-cpu-specific code)
- split assembly out of xen/include/hypervisor.h to xen/include/hypercalls.h
- use <xen/...> instead of <machine/...> for cpu-independant include files.

more work needed here, i386-specific files should got out of arch/xen to
arch/xeni386, and more code shared with arch/i386.

file xen_machdep.c was initially added on branch bouyer-xenamd64.