The NetBSD Project

CVS log for src/sys/arch/x86/x86/svs.c

[BACK] Up to [cvs.NetBSD.org] / src / sys / arch / x86 / x86

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.42 / (download) - annotate - [select for diffs], Sat Sep 24 11:05:18 2022 UTC (18 months, 2 weeks ago) by riastradh
Branch: MAIN
CVS Tags: thorpej-ifq-base, thorpej-ifq, thorpej-altq-separation-base, thorpej-altq-separation, netbsd-10-base, netbsd-10-0-RELEASE, netbsd-10-0-RC6, netbsd-10-0-RC5, netbsd-10-0-RC4, netbsd-10-0-RC3, netbsd-10-0-RC2, netbsd-10-0-RC1, netbsd-10, bouyer-sunxi-drm-base, bouyer-sunxi-drm, HEAD
Changes since 1.41: +4 -2 lines
Diff to previous 1.41 (colored)

x86: Support EFI runtime services.

This creates a special pmap, efi_runtime_pmap, which avoids setting
PTE_U but allows mappings to lie in what would normally be user VM --
this way we don't fall afoul of SMAP/SMEP when executing EFI runtime
services from CPL 0.  SVS does not apply to the EFI runtime pmap.

The mechanism is intended to work with either physical addressing or
virtual addressing; currently the bootloader does physical addressing
but in principle it could be modified to do virtual addressing
instead, if it allocated virtual pages, assigned them in the memory
map, and issued RT->SetVirtualAddressMap.

Not sure pmap_activate_sync and pmap_deactivate_sync are correct,
need more review from an x86 wizard.

If this causes fallout, it can be disabled temporarily without
reverting anything by just making efi_runtime_init return immediately
without doing anything, or by removing options EFI_RUNTIME.

amd64-only for now pending type fixes and testing on i386.

Revision 1.41 / (download) - annotate - [select for diffs], Sat Aug 20 23:48:51 2022 UTC (19 months, 3 weeks ago) by riastradh
Branch: MAIN
Changes since 1.40: +4 -2 lines
Diff to previous 1.40 (colored)

x86: Split most of pmap.h into pmap_private.h or vmparam.h.

This way pmap.h only contains the MD definition of the MI pmap(9)
API, which loads of things in the kernel rely on, so changing x86
pmap internals no longer requires recompiling the entire kernel every
time.

Callers needing these internals must now use machine/pmap_private.h.
Note: This is not x86/pmap_private.h because it contains three parts:

1. CPU-specific (different for i386/amd64) definitions used by...

2. common definitions, including Xenisms like xpmap_ptetomach,
   further used by...

3. more CPU-specific inlines for pmap_pte_* operations

So {amd64,i386}/pmap_private.h defines 1, includes x86/pmap_private.h
for 2, and then defines 3.  Maybe we should split that out into a new
pmap_pte.h to reduce this trouble.

No functional change intended, other than that some .c files must
include machine/pmap_private.h when previously uvm/uvm_pmap.h
polluted the namespace with pmap internals.

Note: This migrates part of i386/pmap.h into i386/vmparam.h --
specifically the parts that are needed for several constants defined
in vmparam.h:

VM_MAXUSER_ADDRESS
VM_MAX_ADDRESS
VM_MAX_KERNEL_ADDRESS
VM_MIN_KERNEL_ADDRESS

Since i386 needs PDP_SIZE in vmparam.h, I added it there on amd64
too, just to keep things parallel.

Revision 1.40 / (download) - annotate - [select for diffs], Thu Oct 7 12:52:27 2021 UTC (2 years, 6 months ago) by msaitoh
Branch: MAIN
Changes since 1.39: +4 -4 lines
Diff to previous 1.39 (colored)

KNF. No functional change.

Revision 1.39 / (download) - annotate - [select for diffs], Sun Jul 19 07:35:08 2020 UTC (3 years, 8 months ago) by maxv
Branch: MAIN
CVS Tags: thorpej-i2c-spi-conf2-base, thorpej-i2c-spi-conf2, thorpej-i2c-spi-conf-base, thorpej-i2c-spi-conf, thorpej-futex2-base, thorpej-futex2, thorpej-futex-base, thorpej-futex, thorpej-cfargs2-base, thorpej-cfargs2, thorpej-cfargs-base, thorpej-cfargs, cjep_sun2x-base1, cjep_sun2x-base, cjep_sun2x, cjep_staticlib_x-base1, cjep_staticlib_x-base, cjep_staticlib_x
Changes since 1.38: +3 -3 lines
Diff to previous 1.38 (colored)

Revert most of ad's movs/stos change. Instead do a lot simpler: declare
svs_quad_copy() used by SVS only, with no need for instrumentation, because
SVS is disabled when sanitizers are on.

Revision 1.38 / (download) - annotate - [select for diffs], Tue Jul 14 00:45:53 2020 UTC (3 years, 9 months ago) by yamaguchi
Branch: MAIN
Changes since 1.37: +5 -3 lines
Diff to previous 1.37 (colored)

Introduce per-cpu IDTs

This is realized by following modifications:
- Add IDT pages and its allocation maps for each cpu in "struct cpu_info"
- Load per-cpu IDTs at cpu_init_idt(struct cpu_info*)
- Copy the IDT entries for cpu0 to other CPUs at attach
   - These are, for example, exceptions, db, system calls, etc.

And, added a kernel option named PCPU_IDT to enable the feature.

Revision 1.37 / (download) - annotate - [select for diffs], Wed May 27 19:40:29 2020 UTC (3 years, 10 months ago) by ad
Branch: MAIN
Changes since 1.36: +7 -24 lines
Diff to previous 1.36 (colored)

svs_pdir_switch(): Use MOVS to copy the PTES.

Revision 1.36 / (download) - annotate - [select for diffs], Wed May 27 19:15:08 2020 UTC (3 years, 10 months ago) by ad
Branch: MAIN
Changes since 1.35: +15 -3 lines
Diff to previous 1.35 (colored)

svs_pmap_sync(): Fast-path the curcpu case.  Could be improved further
with a kcpuset iterator thing.

Revision 1.35 / (download) - annotate - [select for diffs], Sat May 2 11:37:17 2020 UTC (3 years, 11 months ago) by maxv
Branch: MAIN
Changes since 1.34: +91 -43 lines
Diff to previous 1.34 (colored)

Modify the hotpatch mechanism, in order to make it much less ROP-friendly.

Currently x86_patch_window_open is a big problem, because it is a perfect
function to inject/modify executable code with ROP.

 - Remove x86_patch_window_open(), along with its x86_patch_window_close()
   counterpart.
 - Introduce a read-only link-set of hotpatch descriptor structures,
   which reference a maximum of two read-only hotpatch sources.
 - Modify x86_hotpatch() to open a window and call the new
   x86_hotpatch_apply() function in a hard-coded manner.
 - Modify x86_hotpatch() to take a name and a selector, and have
   x86_hotpatch_apply() resolve the descriptor from the name and the
   source from the selector, before hotpatching.
 - Move the error handling in a separate x86_hotpatch_cleanup() function,
   that gets called after we closed the window.

The resulting implementation is a bit complex and non-obvious. But it
gains the following properties: the code executed in the hotpatch window
is strictly hard-coded (no callback and no possibility to execute your own
code in the window) and the pointers this code accesses are strictly
read-only (no possibility to forge pointers to hotpatch an area that was
not designated as hotpatchable at compile-time, and no possibility to
choose what bytes to write other than the maximum of two read-only
templates that were designated as valid for the given destination at
compile-time).

With current CPUs this slightly improves a situation that is already
pretty bad by definition on x86. Assuming CET however, this change closes
a big hole and is kinda great.

The only ~problem there is, is that dtrace-fbt tries to hotpatch random
places with random bytes, and there is just no way to make it safe.
However dtrace is only in a module, that is rarely used and never compiled
into the kernel, so it's not a big problem; add a shitty & vulnerable
independent hotpatch window in it, and leave big XXXs. It looks like fbt
is going to collapse soon anyway.

Revision 1.34 / (download) - annotate - [select for diffs], Sat Apr 25 15:26:18 2020 UTC (3 years, 11 months ago) by bouyer
Branch: MAIN
Changes since 1.33: +2 -2 lines
Diff to previous 1.33 (colored)

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

Revision 1.32.4.1 / (download) - annotate - [select for diffs], Sat Apr 25 11:23:57 2020 UTC (3 years, 11 months ago) by bouyer
Branch: bouyer-xenpvh
Changes since 1.32: +21 -6 lines
Diff to previous 1.32 (colored) next main 1.33 (colored)

Sync with bouyer-xenpvh-base2 (HEAD)

Revision 1.33 / (download) - annotate - [select for diffs], Fri Apr 24 16:27:28 2020 UTC (3 years, 11 months ago) by maxv
Branch: MAIN
CVS Tags: bouyer-xenpvh-base2
Changes since 1.32: +21 -6 lines
Diff to previous 1.32 (colored)

Give the ldt a fixed size of one page (512 slots), and drop the variable-
sized mechanism that was too complex.

This fixes a race between USER_LDT and SVS: during context switches, the
way SVS installs the new ldt relies on the ldt pointer AND the ldt size,
but both cannot be accessed atomically at the same time.

Revision 1.18.2.3 / (download) - annotate - [select for diffs], Mon Apr 13 08:04:12 2020 UTC (4 years ago) by martin
Branch: phil-wifi
Changes since 1.18.2.2: +25 -4 lines
Diff to previous 1.18.2.2 (colored) to branchpoint 1.18 (colored) next main 1.19 (colored)

Mostly merge changes from HEAD upto 20200411

Revision 1.18.2.2 / (download) - annotate - [select for diffs], Wed Apr 8 14:07:59 2020 UTC (4 years ago) by martin
Branch: phil-wifi
Changes since 1.18.2.1: +4 -4 lines
Diff to previous 1.18.2.1 (colored) to branchpoint 1.18 (colored)

Merge changes from current as of 20200406

Revision 1.31.2.1 / (download) - annotate - [select for diffs], Sat Feb 29 20:18:33 2020 UTC (4 years, 1 month ago) by ad
Branch: ad-namecache
Changes since 1.31: +3 -3 lines
Diff to previous 1.31 (colored) next main 1.32 (colored)

Sync with head.

Revision 1.32 / (download) - annotate - [select for diffs], Fri Jan 31 08:55:38 2020 UTC (4 years, 2 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20200421, phil-wifi-20200411, phil-wifi-20200406, is-mlppp-base, is-mlppp, bouyer-xenpvh-base1, bouyer-xenpvh-base, ad-namecache-base3
Branch point for: bouyer-xenpvh
Changes since 1.31: +3 -3 lines
Diff to previous 1.31 (colored)

'oldlwp' is never NULL now, so remove the NULL checks.

Revision 1.31 / (download) - annotate - [select for diffs], Sun Dec 8 20:42:48 2019 UTC (4 years, 4 months ago) by ad
Branch: MAIN
CVS Tags: ad-namecache-base2, ad-namecache-base1, ad-namecache-base
Branch point for: ad-namecache
Changes since 1.30: +3 -3 lines
Diff to previous 1.30 (colored)

Merge x86 pmap changes from yamt-pagecache:

- Deal better with the multi-level pmap object locking kludge.
- Handle uvm_pagealloc() being able to block.

Revision 1.30 / (download) - annotate - [select for diffs], Wed Aug 7 06:23:48 2019 UTC (4 years, 8 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20191119
Changes since 1.29: +27 -6 lines
Diff to previous 1.29 (colored)

Add support for USER_LDT in SVS. This allows us to have both enabled at
the same time.

We allocate an LDT for each CPU in the GDT and map an area for it, in
addition to the default LDT already present. In context switches between
different processes, we choose between the default or the per-cpu LDT
selector: if the user set specific LDT entries, we memcpy them to the
per-cpu LDT and load the per-cpu selector.

Tested by Naveen Narayanan (with Wine on amd64).

Revision 1.18.2.1 / (download) - annotate - [select for diffs], Mon Jun 10 22:06:54 2019 UTC (4 years, 10 months ago) by christos
Branch: phil-wifi
Changes since 1.18: +84 -180 lines
Diff to previous 1.18 (colored)

Sync with HEAD

Revision 1.29 / (download) - annotate - [select for diffs], Wed May 29 16:54:41 2019 UTC (4 years, 10 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20190609, netbsd-9-base, netbsd-9-3-RELEASE, netbsd-9-2-RELEASE, netbsd-9-1-RELEASE, netbsd-9-0-RELEASE, netbsd-9-0-RC2, netbsd-9-0-RC1, netbsd-9
Changes since 1.28: +37 -5 lines
Diff to previous 1.28 (colored)

Add PCID support in SVS. This avoids TLB flushes during kernel<->user
transitions, which greatly reduces the performance penalty introduced by
SVS.

We use two ASIDs, 0 (kern) and 1 (user), and use invpcid to flush pages
in both ASIDs.

The read-only machdep.svs.pcid={0,1} sysctl is added, and indicates whether
SVS+PCID is in use.

Revision 1.28 / (download) - annotate - [select for diffs], Mon May 27 18:36:37 2019 UTC (4 years, 10 months ago) by maxv
Branch: MAIN
Changes since 1.27: +22 -11 lines
Diff to previous 1.27 (colored)

Change the effect of SVS on the TLB. Keep CR4_PGE set when SVS is enabled,
but don't use PTE_G on the kernel PTEs in general.

Add PTE_G on only a few pages, that are already leaked to userland and do
not contain secrets.

This slightly improves syscall performance.

Revision 1.27 / (download) - annotate - [select for diffs], Mon May 27 17:32:36 2019 UTC (4 years, 10 months ago) by maxv
Branch: MAIN
Changes since 1.26: +3 -7 lines
Diff to previous 1.26 (colored)

Remove 'ci_svs_kpdirpa', unused. While here fix a few comments here and
there, reduces a future diff.

Revision 1.26 / (download) - annotate - [select for diffs], Wed May 15 17:31:41 2019 UTC (4 years, 11 months ago) by maxv
Branch: MAIN
Changes since 1.25: +7 -144 lines
Diff to previous 1.25 (colored)

Change the way SVS is disabled. Now you have to pass "boot -3" from the
bootloader. The machdep.svs.enabled sysctl becomes read-only, and just
indicates whether SVS is enabled.

Sent on port-amd64@.

Revision 1.25 / (download) - annotate - [select for diffs], Sun Apr 21 06:37:21 2019 UTC (4 years, 11 months ago) by maxv
Branch: MAIN
Changes since 1.24: +7 -7 lines
Diff to previous 1.24 (colored)

Rename the PTE bits.

Revision 1.24 / (download) - annotate - [select for diffs], Sat Mar 23 10:02:05 2019 UTC (5 years ago) by maxv
Branch: MAIN
CVS Tags: isaki-audio2-base, isaki-audio2
Changes since 1.23: +2 -21 lines
Diff to previous 1.23 (colored)

In fact, xc_broadcast also applies to offline CPUs, so we don't need to
make sure each CPU is online. Remove the checks, I suspect they weren't
totally correct by the way.

Revision 1.23 / (download) - annotate - [select for diffs], Sat Mar 9 08:42:26 2019 UTC (5 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.22: +4 -4 lines
Diff to previous 1.22 (colored)

Start replacing the x86 PTE bits.

Revision 1.13.2.8 / (download) - annotate - [select for diffs], Wed Dec 26 14:01:45 2018 UTC (5 years, 3 months ago) by pgoyette
Branch: pgoyette-compat
CVS Tags: pgoyette-compat-merge-20190127
Changes since 1.13.2.7: +7 -10 lines
Diff to previous 1.13.2.7 (colored) to branchpoint 1.13 (colored) next main 1.14 (colored)

Sync with HEAD, resolve a few conflicts

Revision 1.22 / (download) - annotate - [select for diffs], Thu Dec 6 17:44:28 2018 UTC (5 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-20190127, pgoyette-compat-20190118, pgoyette-compat-1226
Changes since 1.21: +7 -10 lines
Diff to previous 1.21 (colored)

Simplify, use _pi instead of modulos, no real functional change.

Revision 1.13.2.7 / (download) - annotate - [select for diffs], Mon Nov 26 01:52:28 2018 UTC (5 years, 4 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.6: +3 -3 lines
Diff to previous 1.13.2.6 (colored) to branchpoint 1.13 (colored)

Sync with HEAD, resolve a couple of conflicts

Revision 1.21 / (download) - annotate - [select for diffs], Mon Nov 19 20:28:01 2018 UTC (5 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-1126
Changes since 1.20: +3 -3 lines
Diff to previous 1.20 (colored)

Rename 'mask' -> 'frame', we will use the real 'mask' soon.

Revision 1.13.2.6 / (download) - annotate - [select for diffs], Thu Sep 6 06:55:44 2018 UTC (5 years, 7 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.5: +4 -4 lines
Diff to previous 1.13.2.5 (colored) to branchpoint 1.13 (colored)

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Revision 1.20 / (download) - annotate - [select for diffs], Sun Aug 12 12:23:33 2018 UTC (5 years, 8 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-1020, pgoyette-compat-0930, pgoyette-compat-0906
Changes since 1.19: +4 -4 lines
Diff to previous 1.19 (colored)

Introduce PDIR_SLOT_USERLIM, which indicates the limit of the user slots.
Use it instead of PDIR_SLOT_PTE when we just want to iterate over the
user slots. Also use it in SVS, I had hardcoded 255 because there was no
proper define (which there now is).

Revision 1.13.2.5 / (download) - annotate - [select for diffs], Sat Jul 28 04:37:42 2018 UTC (5 years, 8 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.4: +26 -2 lines
Diff to previous 1.13.2.4 (colored) to branchpoint 1.13 (colored)

Sync with HEAD

Revision 1.19 / (download) - annotate - [select for diffs], Thu Jul 12 19:48:16 2018 UTC (5 years, 9 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0728
Changes since 1.18: +26 -2 lines
Diff to previous 1.18 (colored)

Handle NMIs correctly when SVS is enabled. We store the kernel's CR3 at the
top of the NMI stack, and we unconditionally switch to it, because we don't
know with which page tables we received the NMI. Hotpatch the whole thing as
usual.

This restores the ability to use PMCs on Intel CPUs.

Revision 1.14.2.4 / (download) - annotate - [select for diffs], Sat May 5 15:11:53 2018 UTC (5 years, 11 months ago) by martin
Branch: netbsd-8
CVS Tags: netbsd-8-2-RELEASE, netbsd-8-1-RELEASE, netbsd-8-1-RC1, netbsd-8-0-RELEASE, netbsd-8-0-RC2
Changes since 1.14.2.3: +9 -6 lines
Diff to previous 1.14.2.3 (colored) to branchpoint 1.14 (colored) next main 1.15 (colored)

Pull up following revision(s) (requested by alnsn in ticket #790):

	share/man/man9/secmodel_securelevel.9: revision 1.16
	sys/secmodel/suser/secmodel_suser.c: revision 1.44
	sys/secmodel/securelevel/secmodel_securelevel.c: revision 1.31
	sys/sys/kauth.h: revision 1.76
	sys/arch/x86/x86/svs.c: revision 1.18

Add KAUTH_MACHDEP_SVS_DISABLE and add support to secmodel_securelevel(9).
Disabling SVS is denied at securelevel 1 and above.

Add SVS. It may not be disabled at securelevel 1 and above.

Revision 1.13.2.4 / (download) - annotate - [select for diffs], Wed May 2 07:20:06 2018 UTC (5 years, 11 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.3: +9 -6 lines
Diff to previous 1.13.2.3 (colored) to branchpoint 1.13 (colored)

Synch with HEAD

Revision 1.18 / (download) - annotate - [select for diffs], Thu Apr 26 18:54:09 2018 UTC (5 years, 11 months ago) by alnsn
Branch: MAIN
CVS Tags: phil-wifi-base, pgoyette-compat-0625, pgoyette-compat-0521, pgoyette-compat-0502
Branch point for: phil-wifi
Changes since 1.17: +9 -6 lines
Diff to previous 1.17 (colored)

Add KAUTH_MACHDEP_SVS_DISABLE and add support to secmodel_securelevel(9).

Disabling SVS is denied at securelevel 1 and above.

Revision 1.13.2.3 / (download) - annotate - [select for diffs], Sat Apr 7 04:12:14 2018 UTC (6 years ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.2: +15 -2 lines
Diff to previous 1.13.2.2 (colored) to branchpoint 1.13 (colored)

Sync with HEAD.  77 conflicts resolved - all of them $NetBSD$

Revision 1.14.2.3 / (download) - annotate - [select for diffs], Mon Apr 2 08:43:58 2018 UTC (6 years ago) by martin
Branch: netbsd-8
CVS Tags: netbsd-8-0-RC1
Changes since 1.14.2.2: +22 -5 lines
Diff to previous 1.14.2.2 (colored) to branchpoint 1.14 (colored)

Pull up the following revisions, requested by maxv in ticket #683:

	sys/arch/x86/x86/svs.c          1.15-1.17

Fix sysctl type, should be bool.

Use EOPNOTSUPP instead of EINVAL.

Improve the detection. Future generations of Intel CPUs will have a bit to
say they are not affected by Meltdown.

Revision 1.17 / (download) - annotate - [select for diffs], Fri Mar 30 19:58:05 2018 UTC (6 years ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0422, pgoyette-compat-0415, pgoyette-compat-0407
Changes since 1.16: +15 -2 lines
Diff to previous 1.16 (colored)

Improve the detection. Future generations of Intel CPUs will have a bit to
say they are not affected by Meltdown.

Revision 1.13.2.2 / (download) - annotate - [select for diffs], Fri Mar 30 06:20:13 2018 UTC (6 years ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13.2.1: +9 -5 lines
Diff to previous 1.13.2.1 (colored) to branchpoint 1.13 (colored)

Resolve conflicts between branch and HEAD

Revision 1.16 / (download) - annotate - [select for diffs], Thu Mar 29 07:24:26 2018 UTC (6 years ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0330
Changes since 1.15: +6 -3 lines
Diff to previous 1.15 (colored)

Use EOPNOTSUPP instead of EINVAL.

Revision 1.15 / (download) - annotate - [select for diffs], Thu Mar 29 07:15:12 2018 UTC (6 years ago) by maxv
Branch: MAIN
Changes since 1.14: +5 -4 lines
Diff to previous 1.14 (colored)

Fix sysctl type, should be bool.

Revision 1.14.2.2 / (download) - annotate - [select for diffs], Thu Mar 22 16:59:04 2018 UTC (6 years ago) by martin
Branch: netbsd-8
Changes since 1.14.2.1: +753 -0 lines
Diff to previous 1.14.2.1 (colored) to branchpoint 1.14 (colored)

Pull up the following revisions, requested by maxv in ticket #652:

	sys/arch/amd64/amd64/amd64_trap.S	upto 1.39 (partial, patch)
	sys/arch/amd64/amd64/db_machdep.c	1.6 (patch)
	sys/arch/amd64/amd64/genassym.cf	1.65,1.66,1.67 (patch)
	sys/arch/amd64/amd64/locore.S		upto 1.159 (partial, patch)
	sys/arch/amd64/amd64/machdep.c		1.299-1.302 (patch)
	sys/arch/amd64/amd64/trap.c		upto 1.113 (partial, patch)
	sys/arch/amd64/amd64/amd64/vector.S	upto 1.61 (partial, patch)
	sys/arch/amd64/conf/GENERIC		1.477,1.478 (patch)
	sys/arch/amd64/conf/kern.ldscript	1.26 (patch)
	sys/arch/amd64/include/frameasm.h	upto 1.37 (partial, patch)
	sys/arch/amd64/include/param.h		1.25 (patch)
	sys/arch/amd64/include/pmap.h		1.41,1.43,1.44 (patch)
	sys/arch/x86/conf/files.x86		1.91,1.93 (patch)
	sys/arch/x86/include/cpu.h		1.88,1.89 (patch)
	sys/arch/x86/include/pmap.h		1.75 (patch)
	sys/arch/x86/x86/cpu.c			1.144,1.146,1.148,1.149 (patch)
	sys/arch/x86/x86/pmap.c			upto 1.289 (partial, patch)
	sys/arch/x86/x86/vm_machdep.c		1.31,1.32 (patch)
	sys/arch/x86/x86/x86_machdep.c		1.104,1.106,1.108 (patch)
	sys/arch/x86/x86/svs.c			1.1-1.14
	sys/arch/xen/conf/files.compat		1.30 (patch)

Backport SVS. Not enabled yet.

Revision 1.13.2.1 / (download) - annotate - [select for diffs], Thu Mar 15 09:12:04 2018 UTC (6 years, 1 month ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.13: +3 -2 lines
Diff to previous 1.13 (colored)

Synch with HEAD

Revision 1.14.2.1, Tue Mar 13 16:45:52 2018 UTC (6 years, 1 month ago) by martin
Branch: netbsd-8
Changes since 1.14: +0 -753 lines
FILE REMOVED

file svs.c was added on branch netbsd-8 on 2018-03-22 16:59:04 +0000

Revision 1.14 / (download) - annotate - [select for diffs], Tue Mar 13 16:45:52 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0322, pgoyette-compat-0315
Branch point for: netbsd-8
Changes since 1.13: +3 -2 lines
Diff to previous 1.13 (colored)

Mmh, add a missing x86_disable_intr(). My intention there was to ensure
interrupts were disabled before the barriers.

Revision 1.13 / (download) - annotate - [select for diffs], Thu Mar 1 16:49:06 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-base
Branch point for: pgoyette-compat
Changes since 1.12: +2 -5 lines
Diff to previous 1.12 (colored)

Remove these two KASSERTs. Thinking about it, they may fire when the user
enters "sysctl -w machdep.svs.enabled=0", if the xcall is received between
the 'svs_enabled' check in the caller and the same check in these KASSERTs.

In such a case we perform an SVS operation with svs_enabled set to false,
but that's intentional: after it is done svs_pmap_sync and svs_lwp_switch
won't be called anymore, the pdir synchronization is dropped.

Having said that, I didn't see these KASSERTs getting triggered.

Revision 1.12 / (download) - annotate - [select for diffs], Sun Feb 25 13:15:35 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.11: +2 -4 lines
Diff to previous 1.11 (colored)

Remove the first entry from the todo list, it's handled properly now.

Revision 1.11 / (download) - annotate - [select for diffs], Sat Feb 24 19:52:46 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.10: +7 -7 lines
Diff to previous 1.10 (colored)

Fix one thing in the documentation, I meant to say only SVS_UTLS.

Revision 1.10 / (download) - annotate - [select for diffs], Sat Feb 24 10:31:30 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.9: +175 -43 lines
Diff to previous 1.9 (colored)

Document SVS. Also, remove an entry from the todo list.

Revision 1.9 / (download) - annotate - [select for diffs], Fri Feb 23 19:39:27 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.8: +4 -2 lines
Diff to previous 1.8 (colored)

Add a new entry in the TODO list.

Revision 1.8 / (download) - annotate - [select for diffs], Thu Feb 22 13:27:18 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.7: +12 -94 lines
Diff to previous 1.7 (colored)

Remove svs_pgg_update(). Instead of manually changing PG_G on each page,
we can disable the global-paging mechanism in %cr4 with CR4_PGE. Do that.

In addition, install CR4_PGE when SVS is disabled manually (via the
sysctl).

Now, doing "sysctl -w machdep.svs_enabled=0" restores the performance
completely, exactly as if SVS hadn't been enabled in the first place.

Revision 1.7 / (download) - annotate - [select for diffs], Thu Feb 22 11:57:39 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.6: +26 -8 lines
Diff to previous 1.6 (colored)

Ensure the CPUs are all online. We take cpu_lock, so nobody can go offline
in the meantime.

Revision 1.6 / (download) - annotate - [select for diffs], Thu Feb 22 10:42:11 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.5: +129 -2 lines
Diff to previous 1.5 (colored)

Make the machdep.svs_enabled sysctl writable, and add the kernel code
needed to disable SVS at runtime.

We set 'svs_enabled' to false, and hotpatch the kernel entry/exit points
to eliminate the context switch code.

We need to make sure there is no remote CPU that is executing the code we
are hotpatching. So we use two barriers:

 * After the first one each CPU is guaranteed to be executing in
   svs_disable_cpu with interrupts disabled (this way it can't leave this
   place).

 * After the second one it is guaranteed that SVS is disabled, so we flush
   the cache, enable interrupts and continue execution normally.

Between the two barriers, cpu0 will disable SVS (svs_enabled=false and
hotpatch), and each CPU will restore the generic syscall entry point.

Three notes:

 * We should call svs_pgg_update(true) afterwards, to put back PG_G on
   the kernel pages (for better performance). This will be done in another
   commit.

 * The fact that we disable interrupts does not prevent us from receiving
   an NMI, and it would be problematic. So we need to add some code to
   verify that PMCs are disabled before hotpatching. This will be done
   in another commit.

 * In svs_disable() we expect each CPU to be online. We need to add a
   check to make sure they indeed are.

The sysctl allows only a 1->0 transition. There is no point in doing 0->1
transitions anyway, and it would be complicated to implement because we
need to re-synchronize the CPU user page tables with the current ones (we
lost track of them in the last 1->0 transition).

Revision 1.5 / (download) - annotate - [select for diffs], Thu Feb 22 09:41:06 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.4: +51 -5 lines
Diff to previous 1.4 (colored)

Improve the SVS initialization.

Declare x86_patch_window_open() and x86_patch_window_close(), and globalify
x86_hotpatch().

Introduce svs_enable() in x86/svs.c, that does the SVS hotpatching.

Change svs_init() to take a bool. This function gets called twice; early
when the system just booted (and nothing is initialized), lately when at
least pmap_kernel has been initialized.

Revision 1.4 / (download) - annotate - [select for diffs], Thu Feb 22 08:56:52 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.3: +9 -6 lines
Diff to previous 1.3 (colored)

Add a dynamic detection for SVS.

The SVS_* macros are now compiled as skip-noopt. When the system boots, if
the cpu is from Intel, they are hotpatched to their real content.
Typically:

		jmp	1f
		int3
		int3
		int3
		... int3 ...
	1:

gets hotpatched to:

		movq	SVS_UTLS+UTLS_KPDIRPA,%rax
		movq	%rax,%cr3
		movq	CPUVAR(KRSP0),%rsp

These two chunks of code being of the exact same size. We put int3 (0xCC)
to make sure we never execute there.

In the non-SVS (ie non-Intel) case, all it costs is one jump. Given that
the SVS_* macros are small, this jump will likely leave us in the same
icache line, so it's pretty fast.

The syscall entry point is special, because there we use a scratch uint64_t
not in curcpu but in the UTLS page, and it's difficult to hotpatch this
properly. So instead of hotpatching we declare the entry point as an ASM
macro, and define two functions: syscall and syscall_svs, the latter being
the one used in the SVS case.

While here 'syscall' is optimized not to contain an SVS_ENTER - this way
we don't even need to do a jump on the non-SVS case.

When adding pages in the user page tables, make sure we don't have PG_G,
now that it's dynamic.

A read-only sysctl is added, machdep.svs_enabled, that tells whether the
kernel uses SVS or not.

More changes to come, svs_init() is not very clean.

Revision 1.3 / (download) - annotate - [select for diffs], Sun Feb 18 14:07:29 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.2: +5 -2 lines
Diff to previous 1.2 (colored)

Add svs_enabled, which defaults to 'true' when SVS is compiled (no dynamic
detection yet).

Revision 1.2 / (download) - annotate - [select for diffs], Sat Feb 17 17:44:09 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.1: +84 -2 lines
Diff to previous 1.1 (colored)

Add svs_init. This is where we will detect the CPU and decide whether
to turn SVS on or not.

Add svs_pgg_update to dynamically add/remove PG_G from all the kernel
pages. Use it now.

Revision 1.1 / (download) - annotate - [select for diffs], Sun Feb 11 09:39:37 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN

Move SVS into x86/svs.c

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb <webmaster@jp.NetBSD.org>