The NetBSD Project

CVS log for src/sys/arch/amd64/amd64/amd64_trap.S

[BACK] Up to [cvs.NetBSD.org] / src / sys / arch / amd64 / amd64

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.55 / (download) - annotate - [select for diffs], Mon Feb 27 16:24:28 2023 UTC (13 months, 2 weeks ago) by riastradh
Branch: MAIN
CVS Tags: thorpej-ifq-base, thorpej-ifq, thorpej-altq-separation-base, thorpej-altq-separation, HEAD
Changes since 1.54: +26 -26 lines
Diff to previous 1.54 (colored)

amd64_trap.S: Annotate trap vectors with their mnemonics.

Derived from Intel and AMD manuals.

Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3
(3A, 3B, 3C, & 3D: System Programming Guide, Order Number:
325384-077US, April 2022.
https://cdrdv2.intel.com/v1/dl/getContent/671447

AMD64 Technology: AMD64 Architecture Programmers' Manual, Volume 2:
System Programming, Publication No. 24953, Revision 3.40, January 2023.
https://www.amd.com/system/files/TechDocs/24593.pdf

No functional change intended.

XXX Should apply the same treatment to i386_trap.S.

Revision 1.54 / (download) - annotate - [select for diffs], Wed Sep 7 00:40:18 2022 UTC (19 months ago) by knakahara
Branch: MAIN
CVS Tags: netbsd-10-base, netbsd-10-0-RELEASE, netbsd-10-0-RC6, netbsd-10-0-RC5, netbsd-10-0-RC4, netbsd-10-0-RC3, netbsd-10-0-RC2, netbsd-10-0-RC1, netbsd-10, bouyer-sunxi-drm-base, bouyer-sunxi-drm
Changes since 1.53: +8 -8 lines
Diff to previous 1.53 (colored)

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
    https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implemented by nonaka@n.o, updated by me.

Revision 1.53 / (download) - annotate - [select for diffs], Mon Jun 29 23:04:56 2020 UTC (3 years, 9 months ago) by riastradh
Branch: MAIN
CVS Tags: thorpej-i2c-spi-conf2-base, thorpej-i2c-spi-conf2, thorpej-i2c-spi-conf-base, thorpej-i2c-spi-conf, thorpej-futex2-base, thorpej-futex2, thorpej-futex-base, thorpej-futex, thorpej-cfargs2-base, thorpej-cfargs2, thorpej-cfargs-base, thorpej-cfargs, cjep_sun2x-base1, cjep_sun2x-base, cjep_sun2x, cjep_staticlib_x-base1, cjep_staticlib_x-base, cjep_staticlib_x
Changes since 1.52: +2 -2 lines
Diff to previous 1.52 (colored)

Nix trailing whitespace.

Revision 1.52 / (download) - annotate - [select for diffs], Sun Jun 21 16:57:18 2020 UTC (3 years, 9 months ago) by bouyer
Branch: MAIN
Changes since 1.51: +17 -9 lines
Diff to previous 1.51 (colored)

On amd64, Xen PV calls syscalls and traps with events enabled.
Disable events on entry to be safe.
It should have been mostly safe for most cases, but for FPU traps
we need to reload the FPU state if we got interrupted at trap entry.

Hopefully fixes:
panic: kernel diagnostic assertion "curlwp->l_md.md_flags & MDL_FPU_IN_CPU" failed: file "/home/source/ab/HEAD/src/sys/arch/x86/x86/fpu.c", line 524

when running tests.

Revision 1.42.2.3 / (download) - annotate - [select for diffs], Mon Apr 13 08:03:30 2020 UTC (4 years ago) by martin
Branch: phil-wifi
Changes since 1.42.2.2: +5 -0 lines
Diff to previous 1.42.2.2 (colored) to branchpoint 1.42 (colored) next main 1.43 (colored)

Mostly merge changes from HEAD upto 20200411

Revision 1.42.2.2 / (download) - annotate - [select for diffs], Wed Apr 8 14:07:25 2020 UTC (4 years ago) by martin
Branch: phil-wifi
Changes since 1.42.2.1: +3 -8 lines
Diff to previous 1.42.2.1 (colored) to branchpoint 1.42 (colored)

Merge changes from current as of 20200406

Revision 1.51 / (download) - annotate - [select for diffs], Sat Dec 7 10:19:35 2019 UTC (4 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20200421, phil-wifi-20200411, phil-wifi-20200406, is-mlppp-base, is-mlppp, bouyer-xenpvh-base2, bouyer-xenpvh-base1, bouyer-xenpvh-base, bouyer-xenpvh, ad-namecache-base3, ad-namecache-base2, ad-namecache-base1, ad-namecache-base, ad-namecache
Changes since 1.50: +3 -8 lines
Diff to previous 1.50 (colored)

Panic instead of printf, same as syscall.

Revision 1.50 / (download) - annotate - [select for diffs], Thu Nov 14 16:23:52 2019 UTC (4 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20191119
Changes since 1.49: +5 -1 lines
Diff to previous 1.49 (colored)

Add support for Kernel Memory Sanitizer (kMSan). It detects uninitialized
memory used by the kernel at run time, and just like kASan and kCSan, it
is an excellent feature. It has already detected 38 uninitialized variables
in the kernel during my testing, which I have since discreetly fixed.

We use two shadows:
 - "shad", to track uninitialized memory with a bit granularity (1:1).
   Each bit set to 1 in the shad corresponds to one uninitialized bit of
   real kernel memory.
 - "orig", to track the origin of the memory with a 4-byte granularity
   (1:1). Each uint32_t cell in the orig indicates the origin of the
   associated uint32_t of real kernel memory.

The memory consumption of these shadows is consequent, so at least 4GB of
RAM is recommended to run kMSan.

The compiler inserts calls to specific __msan_* functions on each memory
access, to manage both the shad and the orig and detect uninitialized
memory accesses that change the execution flow (like an "if" on an
uninitialized variable).

We mark as uninit several types of memory buffers (stack, pools, kmem,
malloc, uvm_km), and check each buffer passed to copyout, copyoutstr,
bwrite, if_transmit_lock and DMA operations, to detect uninitialized memory
that leaves the system. This allows us to detect kernel info leaks in a way
that is more efficient and also more user-friendly than KLEAK.

Contrary to kASan, kMSan requires comprehensive coverage, ie we cannot
tolerate having one non-instrumented function, because this could cause
false positives. kMSan cannot instrument ASM functions, so I converted
most of them to __asm__ inlines, which kMSan is able to instrument. Those
that remain receive special treatment.

Contrary to kASan again, kMSan uses a TLS, so we must context-switch this
TLS during interrupts. We use different contexts depending on the interrupt
level.

The orig tracks precisely the origin of a buffer. We use a special encoding
for the orig values, and pack together in each uint32_t cell of the orig:
 - a code designating the type of memory (Stack, Pool, etc), and
 - a compressed pointer, which points either (1) to a string containing
   the name of the variable associated with the cell, or (2) to an area
   in the kernel .text section which we resolve to a symbol name + offset.

This encoding allows us not to consume extra memory for associating
information with each cell, and produces a precise output, that can tell
for example the name of an uninitialized variable on the stack, the
function in which it was pushed on the stack, and the function where we
accessed this uninitialized variable.

kMSan is available with LLVM, but not with GCC.

The code is organized in a way that is similar to kASan and kCSan, so it
means that other architectures than amd64 can be supported.

Revision 1.49 / (download) - annotate - [select for diffs], Sat Oct 12 06:31:03 2019 UTC (4 years, 6 months ago) by maxv
Branch: MAIN
Changes since 1.48: +2 -1 lines
Diff to previous 1.48 (colored)

Rewrite the FPU code on x86. This greatly simplifies the logic and removes
the dependency on IPL_HIGH. NVMM is updated accordingly. Posted on
port-amd64 a week ago.

Bump the kernel version to 9.99.16.

Revision 1.42.2.1 / (download) - annotate - [select for diffs], Mon Jun 10 22:05:46 2019 UTC (4 years, 10 months ago) by christos
Branch: phil-wifi
Changes since 1.42: +32 -18 lines
Diff to previous 1.42 (colored)

Sync with HEAD

Revision 1.48 / (download) - annotate - [select for diffs], Sat May 18 13:32:12 2019 UTC (4 years, 10 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-20190609, netbsd-9-base, netbsd-9-3-RELEASE, netbsd-9-2-RELEASE, netbsd-9-1-RELEASE, netbsd-9-0-RELEASE, netbsd-9-0-RC2, netbsd-9-0-RC1, netbsd-9
Changes since 1.47: +20 -8 lines
Diff to previous 1.47 (colored)

Two changes in the CPU mitigations:

 * Micro-optimize: put every mitigation in the same branch. This removes
   two branches in each exc/int return path, and removes all branches in
   the syscall return path.

 * Modify the SpectreV2 mitigation to be compatible with SpectreV4. I
   recently realized that both couldn't be enabled at the same time on
   Intel. This is because initially, when there was just SpectreV2, we
   could reset the whole IA32_SPEC_CTRL MSR. But then Intel added another
   bit in it for SpectreV4, so it isn't right to reset it entirely
   anymore. SSBD needs to stay.

Revision 1.5.6.4 / (download) - annotate - [select for diffs], Tue May 14 17:12:19 2019 UTC (4 years, 11 months ago) by martin
Branch: netbsd-8
CVS Tags: netbsd-8-2-RELEASE, netbsd-8-1-RELEASE, netbsd-8-1-RC1
Changes since 1.5.6.3: +4 -1 lines
Diff to previous 1.5.6.3 (colored) to branchpoint 1.5 (colored) next main 1.6 (colored)

Pull up following revision(s) (requested by maxv in ticket #1269):

	sys/arch/amd64/amd64/locore.S: revision 1.181 (adapted)
	sys/arch/amd64/amd64/amd64_trap.S: revision 1.47 (adapted)
	sys/arch/x86/include/specialreg.h: revision 1.144 (adapted)
	sys/arch/amd64/include/frameasm.h: revision 1.43 (adapted)
	sys/arch/x86/x86/spectre.c: revision 1.27 (adapted)

Mitigation for INTEL-SA-00233: Microarchitectural Data Sampling (MDS).
It requires a microcode update, now available on the Intel website. The
microcode modifies the behavior of the VERW instruction, and makes it flush
internal CPU buffers. We hotpatch the return-to-userland path to add VERW.

Two sysctls are added:

	machdep.mds.mitigated = {0/1} user-settable
	machdep.mds.method = {string} constructed by the kernel

The kernel will automatically enable the mitigation if the updated
microcode is present. If the new microcode is not present, the user can
load it via cpuctl, and set machdep.mds.mitigated=1.

Revision 1.47 / (download) - annotate - [select for diffs], Tue May 14 16:59:25 2019 UTC (4 years, 11 months ago) by maxv
Branch: MAIN
Changes since 1.46: +3 -1 lines
Diff to previous 1.46 (colored)

Mitigation for INTEL-SA-00233: Microarchitectural Data Sampling (MDS).

It requires a microcode update, now available on the Intel website. The
microcode modifies the behavior of the VERW instruction, and makes it flush
internal CPU buffers. We hotpatch the return-to-userland path to add VERW.

Two sysctls are added:

	machdep.mds.mitigated = {0/1} user-settable
	machdep.mds.method = {string} constructed by the kernel

The kernel will automatically enable the mitigation if the updated
microcode is present. If the new microcode is not present, the user can
load it via cpuctl, and set machdep.mds.mitigated=1.

Revision 1.46 / (download) - annotate - [select for diffs], Mon Feb 11 14:59:32 2019 UTC (5 years, 2 months ago) by cherry
Branch: MAIN
CVS Tags: isaki-audio2-base, isaki-audio2
Changes since 1.45: +6 -6 lines
Diff to previous 1.45 (colored)

We reorganise definitions for XEN source support as follows:

XEN - common sources required for baseline XEN support.
XENPV - sources required for support of XEN in PV mode.
XENPVHVM - sources required for support for XEN in HVM mode.
XENPVH - sources required for support for XEN in PVH mode.

Revision 1.37.2.6 / (download) - annotate - [select for diffs], Thu Sep 6 06:55:24 2018 UTC (5 years, 7 months ago) by pgoyette
Branch: pgoyette-compat
CVS Tags: pgoyette-compat-merge-20190127
Changes since 1.37.2.5: +3 -3 lines
Diff to previous 1.37.2.5 (colored) to branchpoint 1.37 (colored) next main 1.38 (colored)

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Revision 1.45 / (download) - annotate - [select for diffs], Sun Aug 12 06:11:47 2018 UTC (5 years, 8 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-20190127, pgoyette-compat-20190118, pgoyette-compat-1226, pgoyette-compat-1126, pgoyette-compat-1020, pgoyette-compat-0930, pgoyette-compat-0906
Changes since 1.44: +3 -3 lines
Diff to previous 1.44 (colored)

Eliminate the only ASM reference to VM_MIN_KERNEL_ADDRESS. Rename the
value to VM_SPACE_SEP_HIGH32, it is now the highest 32bits of the first
va of the higher half of the address space (right after the canonical
hole).

Revision 1.37.2.5 / (download) - annotate - [select for diffs], Sat Jul 28 04:37:26 2018 UTC (5 years, 8 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.37.2.4: +5 -5 lines
Diff to previous 1.37.2.4 (colored) to branchpoint 1.37 (colored)

Sync with HEAD

Revision 1.44 / (download) - annotate - [select for diffs], Sat Jul 14 14:29:40 2018 UTC (5 years, 9 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0728
Changes since 1.43: +2 -2 lines
Diff to previous 1.43 (colored)

Drop NENTRY() from the x86 kernels, use ENTRY(). With PMCs (and other hardware
tracing facilities) we have a much better ways of monitoring the CPU activity
than GPROF, without software modification.

Also I think GPROF has never worked, because the 'start' functions of both
i386 and amd64 use ENTRY(), and it would have caused a function call while the
kernel was not yet relocated.

Revision 1.43 / (download) - annotate - [select for diffs], Thu Jul 12 19:48:16 2018 UTC (5 years, 9 months ago) by maxv
Branch: MAIN
Changes since 1.42: +4 -4 lines
Diff to previous 1.42 (colored)

Handle NMIs correctly when SVS is enabled. We store the kernel's CR3 at the
top of the NMI stack, and we unconditionally switch to it, because we don't
know with which page tables we received the NMI. Hotpatch the whole thing as
usual.

This restores the ability to use PMCs on Intel CPUs.

Revision 1.37.2.4 / (download) - annotate - [select for diffs], Mon Jun 25 07:25:38 2018 UTC (5 years, 9 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.37.2.3: +2 -1 lines
Diff to previous 1.37.2.3 (colored) to branchpoint 1.37 (colored)

Sync with HEAD

Revision 1.42 / (download) - annotate - [select for diffs], Fri May 25 15:33:56 2018 UTC (5 years, 10 months ago) by maxv
Branch: MAIN
CVS Tags: phil-wifi-base, pgoyette-compat-0625
Branch point for: phil-wifi
Changes since 1.41: +2 -1 lines
Diff to previous 1.41 (colored)

When the previous contrext is in kernel mode we are not guaranteed to have
a 16-byte-aligned stack pointer, so align it. That's what the CPU would do
on exception entry.

Revision 1.37.2.3 / (download) - annotate - [select for diffs], Mon May 21 04:35:57 2018 UTC (5 years, 10 months ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.37.2.2: +119 -1 lines
Diff to previous 1.37.2.2 (colored) to branchpoint 1.37 (colored)

Sync with HEAD

Revision 1.41 / (download) - annotate - [select for diffs], Tue May 8 17:20:44 2018 UTC (5 years, 11 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0521
Changes since 1.40: +119 -1 lines
Diff to previous 1.40 (colored)

Mitigation for the SS bug, CVE-2018-8897. We disabled dbregs a month ago
in -current and -8 so we are not particularly affected anymore.

The #DB handler runs on ist3, if we decide to process the exception we
copy the iret frame on the correct non-ist stack and continue as usual.

Revision 1.5.6.3 / (download) - annotate - [select for diffs], Sat Apr 14 10:11:49 2018 UTC (6 years ago) by martin
Branch: netbsd-8
CVS Tags: netbsd-8-0-RELEASE, netbsd-8-0-RC2, netbsd-8-0-RC1
Changes since 1.5.6.2: +3 -1 lines
Diff to previous 1.5.6.2 (colored) to branchpoint 1.5 (colored)

Pullup the following revisions via patch, requested by maxv in ticket #748:

sys/arch/amd64/amd64/copy.S		1.29 (adapted, via patch)
sys/arch/amd64/amd64/amd64_trap.S	1.16,1.19 (partial) (via patch)
sys/arch/amd64/amd64/trap.c		1.102,1.106 (partial),1.110 (via patch)
sys/arch/amd64/include/frameasm.h	1.22,1.24 (via patch)
sys/arch/x86/x86/cpu.c			1.137 (via patch)
sys/arch/x86/x86/patch.c		1.23,1.26 (partial) (via patch)

Backport of SMAP support.

Revision 1.37.2.2 / (download) - annotate - [select for diffs], Fri Mar 30 06:20:10 2018 UTC (6 years ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.37.2.1: +5 -1 lines
Diff to previous 1.37.2.1 (colored) to branchpoint 1.37 (colored)

Resolve conflicts between branch and HEAD

Revision 1.40 / (download) - annotate - [select for diffs], Wed Mar 28 16:02:49 2018 UTC (6 years ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0502, pgoyette-compat-0422, pgoyette-compat-0415, pgoyette-compat-0407, pgoyette-compat-0330
Changes since 1.39: +5 -1 lines
Diff to previous 1.39 (colored)

Add the IBRS mitigation for SpectreV2 on amd64.

Different operations are performed during context transitions:

	user->kernel: IBRS <- 1
	kernel->user: IBRS <- 0

And during context switches:

	user->user:   IBPB <- 0
	kernel->user: IBPB <- 0
	[user->kernel:IBPB <- 0 this one may not be needed]

We use two macros, IBRS_ENTER and IBRS_LEAVE, to set the IBRS bit. The
thing is hotpatched for better performance, like SVS.

The idea is that IBRS is a "privileged" bit, which is set to 1 in kernel
mode and 0 in user mode. To protect the branch predictor between user
processes (which are of the same privilege), we use the IBPB barrier.

The Intel manual also talks about (MWAIT/HLT)+HyperThreading, and says
that when using either of the two instructions IBRS must be disabled for
better performance on the core. I'm not totally sure about this part, so
I'm not adding it now.

IBRS is available only when the Intel microcode update is applied. The
mitigation must be enabled manually with machdep.spectreV2.mitigated.

Tested by msaitoh a week ago (but I adapted a few things since). Probably
more changes to come.

Revision 1.5.6.2 / (download) - annotate - [select for diffs], Thu Mar 22 16:59:03 2018 UTC (6 years ago) by martin
Branch: netbsd-8
Changes since 1.5.6.1: +128 -50 lines
Diff to previous 1.5.6.1 (colored) to branchpoint 1.5 (colored)

Pull up the following revisions, requested by maxv in ticket #652:

	sys/arch/amd64/amd64/amd64_trap.S	upto 1.39 (partial, patch)
	sys/arch/amd64/amd64/db_machdep.c	1.6 (patch)
	sys/arch/amd64/amd64/genassym.cf	1.65,1.66,1.67 (patch)
	sys/arch/amd64/amd64/locore.S		upto 1.159 (partial, patch)
	sys/arch/amd64/amd64/machdep.c		1.299-1.302 (patch)
	sys/arch/amd64/amd64/trap.c		upto 1.113 (partial, patch)
	sys/arch/amd64/amd64/amd64/vector.S	upto 1.61 (partial, patch)
	sys/arch/amd64/conf/GENERIC		1.477,1.478 (patch)
	sys/arch/amd64/conf/kern.ldscript	1.26 (patch)
	sys/arch/amd64/include/frameasm.h	upto 1.37 (partial, patch)
	sys/arch/amd64/include/param.h		1.25 (patch)
	sys/arch/amd64/include/pmap.h		1.41,1.43,1.44 (patch)
	sys/arch/x86/conf/files.x86		1.91,1.93 (patch)
	sys/arch/x86/include/cpu.h		1.88,1.89 (patch)
	sys/arch/x86/include/pmap.h		1.75 (patch)
	sys/arch/x86/x86/cpu.c			1.144,1.146,1.148,1.149 (patch)
	sys/arch/x86/x86/pmap.c			upto 1.289 (partial, patch)
	sys/arch/x86/x86/vm_machdep.c		1.31,1.32 (patch)
	sys/arch/x86/x86/x86_machdep.c		1.104,1.106,1.108 (patch)
	sys/arch/x86/x86/svs.c			1.1-1.14
	sys/arch/xen/conf/files.compat		1.30 (patch)

Backport SVS. Not enabled yet.

Revision 1.37.2.1 / (download) - annotate - [select for diffs], Thu Mar 22 01:44:41 2018 UTC (6 years ago) by pgoyette
Branch: pgoyette-compat
Changes since 1.37: +27 -28 lines
Diff to previous 1.37 (colored)

Synch with HEAD, resolve conflicts

Revision 1.39 / (download) - annotate - [select for diffs], Tue Mar 20 18:27:58 2018 UTC (6 years ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-0322
Changes since 1.38: +21 -16 lines
Diff to previous 1.38 (colored)

(Re)Fix handling of segment register faults. My previous attempt did fix
faults occuring when reloading %es/%ds/%fs/%gs, but it did not fix faults
occuring when executing 'iretq', because before iretq we needed to do +16
in %rsp, and the resulting stack layout was not the one kernuser_reenter()
expected (tf_trapno and tf_err were not there).

So now: pop tf_trapno and tf_err right away in intrfastexit(), and update
the layout in kernuser_reenter() accordingly. The resulting code is
actually simpler.

Tested by "hardcoding" an iretq fault; the process correctly receives a
SIGSEGV.

(Note that segment register faults do not happen in the wild, you really
need to try hard to trigger one.)

Revision 1.38 / (download) - annotate - [select for diffs], Tue Mar 20 14:26:49 2018 UTC (6 years ago) by maxv
Branch: MAIN
Changes since 1.37: +7 -13 lines
Diff to previous 1.37 (colored)

Remove the sysretq fault handler. It is broken with SVS, and not really
needed anyway. Initially I had added it so that if such a fault was
received the kernel would panic "cleanly" instead of crashing in a
potentially undefined way.

I'll re-add this handler later.

Revision 1.5.6.1 / (download) - annotate - [select for diffs], Wed Mar 7 14:50:56 2018 UTC (6 years, 1 month ago) by martin
Branch: netbsd-8
Changes since 1.5: +211 -154 lines
Diff to previous 1.5 (colored)

Pull up the following revisions (via patch), requested by maxv in ticket #610:

sys/arch/amd64/amd64/amd64_trap.S	1.8,1.10,1.12 (partial),1.13-1.15,
					1.19 (partial),1.20,1.21,1.22,1.24
					(via patch)
sys/arch/amd64/amd64/locore.S		1.129 (partial),1.132 (via patch)
sys/arch/amd64/amd64/trap.c		1.97 (partial),1.111 (via patch)
sys/arch/amd64/amd64/vector.S		1.54,1.55 (via patch)
sys/arch/amd64/include/frameasm.h	1.21,1.23 (via patch)
sys/arch/x86/x86/cpu.c			1.138 (via patch)
sys/arch/xen/conf/Makefile.xen		1.45 (via patch)

Rename and reorder several things in amd64_trap.S.
Compile amd64_trap.S as a file.
Introduce nmitrap and doubletrap.
Have the CPU clear PSL_D automatically in the syscall entry point.

Revision 1.37 / (download) - annotate - [select for diffs], Sun Feb 25 12:37:16 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-compat-base, pgoyette-compat-0315
Branch point for: pgoyette-compat
Changes since 1.36: +110 -40 lines
Diff to previous 1.36 (colored)

Fix handling of segment register faults when running with SVS. The behavior
is changed also in the non-SVS case.

I've put a documentation in amd64_trap.S. Basically, the problem with SVS
is that if iret faults, we already have a full trapframe pushed on the
stack and the CPU will push another frame on this stack (nested), but it
hits the redzone below the stack since it is still running with the user
page table loaded.

To fix that, we pop a good part of the trapframe earlier in intrfastexit.
If iret faults, the current %rsp has enough room for an iret frame, and
the CPU can push that without problem. We then switch back to the outer
iret frame (the frame the CPU was trying to pop by executing iret, but that
it didn't pop for real because iret faulted), call INTRENTRY, and handle
the trap as if it had been received from userland directly.

Revision 1.36 / (download) - annotate - [select for diffs], Sun Feb 25 11:57:44 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.35: +9 -8 lines
Diff to previous 1.35 (colored)

Ah. Don't use NENTRY() to declare check_swapgs, use LABEL() instead. NENTRY
puts the code in the .text section, so the effect of TEXT_USER_BEGIN was
overwritten, and check_swapgs was not put in the .text.user section.

As a result kernels running SVS would crash when jumping here - because we
execute this place with the user page table loaded, and in this page table
only .text.user is mapped.

While here, rename check_swapgs -> kernuser_reenter, because we do more
things than just SWAPGS.

Revision 1.35 / (download) - annotate - [select for diffs], Sun Feb 25 08:28:55 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.34: +3 -3 lines
Diff to previous 1.34 (colored)

Replace %rax -> %rdi, so that check_swapgs clobbers only one register.

Revision 1.34 / (download) - annotate - [select for diffs], Sun Feb 25 08:09:07 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.33: +7 -7 lines
Diff to previous 1.33 (colored)

There are two places where we reload %gs:

 * In setusergs. Here we can't fault. So we don't need to handle this
   case.

 * In intrfastexit for 32bit processes. This case needs to be handled,
   and we already have a label.

So use the label instead of disassembling %rip.

Revision 1.33 / (download) - annotate - [select for diffs], Thu Feb 22 08:36:31 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.32: +144 -127 lines
Diff to previous 1.32 (colored)

Revert all my latest changes, and restore this file back to how it was
in rev1.24. I wanted to replace the functions dynamically for SVS, but
that was a dumb idea, we'll just hotpatch instead.

Revision 1.32 / (download) - annotate - [select for diffs], Sun Feb 18 14:32:31 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.31: +5 -5 lines
Diff to previous 1.31 (colored)

Pass the name of the function as argument in SWAPGS_HANDLER.

Revision 1.31 / (download) - annotate - [select for diffs], Sat Feb 17 21:05:58 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.30: +46 -40 lines
Diff to previous 1.30 (colored)

Declare check_swapgs in an ASM macro. No real functional change.

Revision 1.30 / (download) - annotate - [select for diffs], Sat Feb 17 20:59:14 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.29: +79 -76 lines
Diff to previous 1.29 (colored)

Use ASM macros for the rest of the entry points. No real functional
change. Now the format of the entry points is:

	.macro	TRAP_ENTRY_POINT_xx	arg1,arg2,arg3
		...the asm code...
	.endm

		TEXT_USER_BEGIN
	TRAP_ENTRY_POINT_xx	arg1,arg2,arg3
		TEXT_USER_END

Revision 1.29 / (download) - annotate - [select for diffs], Sat Feb 17 20:47:04 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.28: +16 -13 lines
Diff to previous 1.28 (colored)

Declare and use TRAP_ENTRY_POINT_DNA. This time we don't give an is_ztrap
argument, because the macro is tied to the entry point, and it would be
wrong to suggest the paramater is controllable. No real functional change.

Revision 1.28 / (download) - annotate - [select for diffs], Sat Feb 17 20:41:57 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.27: +19 -15 lines
Diff to previous 1.27 (colored)

Now that [Z]TRAP and [Z]TRAP_NJ are identical, put back the

	INTRENTRY
	jmp	.Lalltraps_noentry

instructions for Xen, and remove [Z]TRAP_NJ.

Revision 1.27 / (download) - annotate - [select for diffs], Sat Feb 17 20:33:28 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.26: +18 -19 lines
Diff to previous 1.26 (colored)

Declare and use TRAP_ENTRY_POINT_SPUR. No real functional change.

Revision 1.26 / (download) - annotate - [select for diffs], Sat Feb 17 20:28:18 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.25: +20 -19 lines
Diff to previous 1.25 (colored)

Declare and use TRAP_ENTRY_POINT_FPU. No real functional change.

Revision 1.25 / (download) - annotate - [select for diffs], Sat Feb 17 20:22:05 2018 UTC (6 years, 1 month ago) by maxv
Branch: MAIN
Changes since 1.24: +59 -92 lines
Diff to previous 1.24 (colored)

Start using ASM macros to define the trap entry points. No real functional
change.

Revision 1.24 / (download) - annotate - [select for diffs], Fri Feb 9 08:54:11 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.23: +1 -3 lines
Diff to previous 1.23 (colored)

Don't restore segment registers when leaving NMIs. In nmitrap (and the
functions it later calls), we are not allowing the trap frame to change;
so the segregs don't change since we are running with interrupts disabled
and there is no rescheduling in this case.

Revision 1.23 / (download) - annotate - [select for diffs], Sun Jan 21 11:21:40 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.22: +30 -21 lines
Diff to previous 1.22 (colored)

Unmap the kernel from userland in SVS, and leave only the needed
trampolines. As explained below, SVS should now completely mitigate
Meltdown on GENERIC kernels, even though it needs some more tweaking
for GENERIC_KASLR.

Until now the kernel entry points looked like:

	FUNC(intr)
		pushq	$ERR
		pushq	$TRAPNO
		INTRENTRY
		... handle interrupt ...
		INTRFASTEXIT
	END(intr)

With this change they are split and become:

	FUNC(handle)
		... handle interrupt ...
		INTRFASTEXIT
	END(handle)

		TEXT_USER_BEGIN
	FUNC(intr)
		pushq	$ERR
		pushq	$TRAPNO
		INTRENTRY
		jmp	handle
	END(intr)
		TEXT_USER_END

A new section is introduced, .text.user, that contains minimal kernel
entry/exit points. In order to choose what to put in this section, two
macros are introduced, TEXT_USER_BEGIN and TEXT_USER_END.

The section is mapped in userland with normal 4K pages.

In GENERIC, the section is 4K-page-aligned and embedded in .text, which
is mapped with large pages. That is to say, when an interrupt comes in,
the CPU has the user page tables loaded and executes the 'intr' functions
on 4K pages; after calling SVS_ENTER (in INTRENTRY) these 4K pages become
2MB large pages, and remain so when executing in kernel mode.

In GENERIC_KASLR, the section is 4K-page-aligned and independent from the
other kernel texts. The prekern just picks it up and maps it at a random
address.

In GENERIC, SVS should now completely mitigate Meltdown: what we put in
.text.user is not secret.

In GENERIC_KASLR, SVS would have to be improved a bit more: the
'jmp handle' instruction is actually secret, since it leaks the address
of the section we are jumping into. By exploiting Meltdown on Intel, this
theoretically allows a local user to reconstruct the address of the first
text section. But given that our KASLR produces several texts, and that
each section is not correlated with the others, the level of protection
KASLR provides is still good.

Revision 1.22 / (download) - annotate - [select for diffs], Sat Jan 20 14:27:15 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.21: +13 -6 lines
Diff to previous 1.21 (colored)

Compile amd64_trap.S as a file instead of including it.

Revision 1.21 / (download) - annotate - [select for diffs], Sat Jan 20 13:45:15 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.20: +18 -17 lines
Diff to previous 1.20 (colored)

Eliminate a '.text'.

Revision 1.20 / (download) - annotate - [select for diffs], Sat Jan 20 13:42:07 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.19: +23 -21 lines
Diff to previous 1.19 (colored)

Don't declare exceptions[] with IDTVEC, it's an array, not a function.
Rename it to x86_exceptions[], and move it to .rodata.

Revision 1.19 / (download) - annotate - [select for diffs], Sat Jan 20 08:30:53 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.18: +38 -3 lines
Diff to previous 1.18 (colored)

Fix the double-fault handler. We're executing on ist1 and must not jump
out of it, so don't enable interrupts. And use the SVS_*_ALTSTACK macros.

While here, fix the NMI handler too: it should use SVS_LEAVE_ALTSTACK.

Revision 1.18 / (download) - annotate - [select for diffs], Thu Jan 18 07:25:34 2018 UTC (6 years, 2 months ago) by maxv
Branch: MAIN
Changes since 1.17: +4 -4 lines
Diff to previous 1.17 (colored)

Unmap the kernel heap from the user page tables (SVS).

This implementation is optimized and organized in such a way that we
don't need to copy the kernel stack to a safe place during user<->kernel
transitions. We create two VAs that point to the same physical page; one
will be mapped in userland and is offset in order to contain only the
trapframe, the other is mapped in the kernel and maps the entire stack.

Sent on tech-kern@ a week ago.

Revision 1.17 / (download) - annotate - [select for diffs], Sun Jan 7 16:10:16 2018 UTC (6 years, 3 months ago) by maxv
Branch: MAIN
Changes since 1.16: +5 -2 lines
Diff to previous 1.16 (colored)

Add a new option, SVS (for Separate Virtual Space), that unmaps kernel
pages when running in userland. For now, only the PTE area is unmapped.

Sent on tech-kern@.

Revision 1.16 / (download) - annotate - [select for diffs], Sun Jan 7 12:42:46 2018 UTC (6 years, 3 months ago) by maxv
Branch: MAIN
Changes since 1.15: +3 -3 lines
Diff to previous 1.15 (colored)

Implement a real hotpatch feature.

Define a HOTPATCH() macro, that puts a label and additional information
in the new .rodata.hotpatch kernel section. In patch.c, scan the section
and patch what needs to be. Now it is possible to hotpatch the content of
a macro.

SMAP is switched to use this new system; this saves a call+ret in each
kernel entry/exit point.

Many other operating systems do the same.

Revision 1.15 / (download) - annotate - [select for diffs], Sat Jan 6 08:44:01 2018 UTC (6 years, 3 months ago) by maxv
Branch: MAIN
Changes since 1.14: +3 -3 lines
Diff to previous 1.14 (colored)

Mmh, I made a mistake in r1.10 - I forgot to update this function call.

Revision 1.14 / (download) - annotate - [select for diffs], Tue Jan 2 18:41:14 2018 UTC (6 years, 3 months ago) by maxv
Branch: MAIN
Changes since 1.13: +62 -56 lines
Diff to previous 1.13 (colored)

Use decimal numbering - hex is just misleading -, use ZTRAP_NJ for NMIs,
and declare intrspurious independently.

Revision 1.2.8.3 / (download) - annotate - [select for diffs], Sun Dec 3 11:35:47 2017 UTC (6 years, 4 months ago) by jdolecek
Branch: tls-maxphys
Changes since 1.2.8.2: +113 -71 lines
Diff to previous 1.2.8.2 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

update from HEAD

Revision 1.13 / (download) - annotate - [select for diffs], Sun Nov 26 14:54:43 2017 UTC (6 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: tls-maxphys-base-20171202
Changes since 1.12: +6 -6 lines
Diff to previous 1.12 (colored)

Hide a bunch of raw symbols.

Revision 1.12 / (download) - annotate - [select for diffs], Tue Oct 17 07:33:44 2017 UTC (6 years, 5 months ago) by maxv
Branch: MAIN
Changes since 1.11: +4 -2 lines
Diff to previous 1.11 (colored)

Have the cpu clear PSL_D automatically when entering the kernel via a
syscall. Then, don't clear PSL_D and PSL_AC in the syscall entry point,
they are now both cleared by the cpu (faster). However they still need to
be manually cleared in the interrupt/trap entry points.

Revision 1.11 / (download) - annotate - [select for diffs], Fri Sep 15 17:32:12 2017 UTC (6 years, 6 months ago) by maxv
Branch: MAIN
Changes since 1.10: +6 -5 lines
Diff to previous 1.10 (colored)

Declare INTRFASTEXIT as a function, so that there is only one iretq in the
kernel. Then, check %rip against the address of this iretq instead of
disassembling (%rip) - which could fault again, or point at some random
address which happens to contain the iretq opcode. The same is true for gs
below, but I'll fix that in another commit.

Revision 1.10 / (download) - annotate - [select for diffs], Sun Sep 3 08:52:18 2017 UTC (6 years, 7 months ago) by maxv
Branch: MAIN
Changes since 1.9: +3 -3 lines
Diff to previous 1.9 (colored)

Remove useless debug code, and split trap() into smaller functions, easier
to understand. NMIs take another, faster path now. No functional change
beyond that.

Revision 1.9 / (download) - annotate - [select for diffs], Thu Aug 31 10:30:58 2017 UTC (6 years, 7 months ago) by maxv
Branch: MAIN
Changes since 1.8: +8 -2 lines
Diff to previous 1.8 (colored)

Add a layer of mitigation against the intel sysret vuln: restore %gs when
sysretq faults. Right now we try to make sure that %rip is canonical by
performing sanity checks in several places, but I've already found missing
checks two times already, and there may be others.

By performing an additional swapgs here, we are turning ring0 exploits to
simple DoSes - which are still security bugs, but of a lower impact.

Revision 1.8 / (download) - annotate - [select for diffs], Thu Aug 31 09:33:19 2017 UTC (6 years, 7 months ago) by maxv
Branch: MAIN
Changes since 1.7: +70 -61 lines
Diff to previous 1.7 (colored)

Reorder for clarity, and style.

Revision 1.2.10.3 / (download) - annotate - [select for diffs], Mon Aug 28 17:51:27 2017 UTC (6 years, 7 months ago) by skrll
Branch: nick-nhusb
Changes since 1.2.10.2: +31 -16 lines
Diff to previous 1.2.10.2 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

Sync with HEAD

Revision 1.7 / (download) - annotate - [select for diffs], Fri Aug 18 14:52:19 2017 UTC (6 years, 7 months ago) by maxv
Branch: MAIN
CVS Tags: nick-nhusb-base-20170825
Changes since 1.6: +40 -0 lines
Diff to previous 1.6 (colored)

Revert my previous change. I hadn't checked carefully enough: the
symbols are used in src/external. There is a number of things that seem
wrong to me here, but I'm not changing them for now.

Revision 1.6 / (download) - annotate - [select for diffs], Fri Aug 18 10:02:37 2017 UTC (6 years, 7 months ago) by maxv
Branch: MAIN
Changes since 1.5: +2 -42 lines
Diff to previous 1.5 (colored)

Remove unused and broken code. On amd64 we won't want int3 from kernel
mode to be valid.

Revision 1.3.2.1 / (download) - annotate - [select for diffs], Wed Apr 26 02:52:59 2017 UTC (6 years, 11 months ago) by pgoyette
Branch: pgoyette-localcount
Changes since 1.3: +31 -16 lines
Diff to previous 1.3 (colored) next main 1.4 (colored)

Sync with HEAD

Revision 1.4.2.1 / (download) - annotate - [select for diffs], Fri Apr 21 16:53:21 2017 UTC (6 years, 11 months ago) by bouyer
Branch: bouyer-socketcan
Changes since 1.4: +31 -16 lines
Diff to previous 1.4 (colored) next main 1.5 (colored)

Sync with HEAD

Revision 1.5 / (download) - annotate - [select for diffs], Fri Mar 24 18:03:32 2017 UTC (7 years ago) by maxv
Branch: MAIN
CVS Tags: prg-localcount2-base3, prg-localcount2-base2, prg-localcount2-base1, prg-localcount2-base, prg-localcount2, pgoyette-localcount-20170426, perseant-stdc-iso10646-base, perseant-stdc-iso10646, netbsd-8-base, matt-nb8-mediatek-base, matt-nb8-mediatek, jdolecek-ncq-base, jdolecek-ncq, bouyer-socketcan-base1
Branch point for: netbsd-8
Changes since 1.4: +31 -16 lines
Diff to previous 1.4 (colored)

Unconditionnally save the segment registers - because we could have a
kernel %gs and a userland %es/%ds -, and explain why T_NMI is a special
case.

Note that checking %gs directly is not a good idea: recent CPUs have the
FSGSBASE instruction set, which allows userland to directly modify %gs
without going through the kernel. If we ever enable this set, we will have
to change this function, since we won't be able to test %gs against
VM_MIN_KERNEL_ADDRESS anymore.

Revision 1.2.10.2 / (download) - annotate - [select for diffs], Wed Oct 5 20:55:23 2016 UTC (7 years, 6 months ago) by skrll
Branch: nick-nhusb
Changes since 1.2.10.1: +22 -13 lines
Diff to previous 1.2.10.1 (colored) to branchpoint 1.2 (colored)

Sync with HEAD

Revision 1.4 / (download) - annotate - [select for diffs], Sun Aug 7 09:04:55 2016 UTC (7 years, 8 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-localcount-20170320, pgoyette-localcount-20170107, pgoyette-localcount-20161104, nick-nhusb-base-20170204, nick-nhusb-base-20161204, nick-nhusb-base-20161004, localcount-20160914, bouyer-socketcan-base
Branch point for: bouyer-socketcan
Changes since 1.3: +20 -11 lines
Diff to previous 1.3 (colored)

Explain a little.

Revision 1.2.10.1 / (download) - annotate - [select for diffs], Sun Dec 27 12:09:28 2015 UTC (8 years, 3 months ago) by skrll
Branch: nick-nhusb
Changes since 1.2: +7 -7 lines
Diff to previous 1.2 (colored)

Sync with HEAD (as of 26th Dec)

Revision 1.3 / (download) - annotate - [select for diffs], Sun Nov 22 13:41:24 2015 UTC (8 years, 4 months ago) by maxv
Branch: MAIN
CVS Tags: pgoyette-localcount-base, pgoyette-localcount-20160806, pgoyette-localcount-20160726, nick-nhusb-base-20160907, nick-nhusb-base-20160529, nick-nhusb-base-20160422, nick-nhusb-base-20160319, nick-nhusb-base-20151226
Branch point for: pgoyette-localcount
Changes since 1.2: +7 -7 lines
Diff to previous 1.2 (colored)

KNF a bit, so I don't get scared each time I open a file

Revision 1.2.8.2 / (download) - annotate - [select for diffs], Wed Aug 20 00:02:42 2014 UTC (9 years, 7 months ago) by tls
Branch: tls-maxphys
Changes since 1.2.8.1: +428 -0 lines
Diff to previous 1.2.8.1 (colored) to branchpoint 1.2 (colored)

Rebase to HEAD as of a few days ago.

Revision 1.2.4.2 / (download) - annotate - [select for diffs], Thu May 22 11:39:28 2014 UTC (9 years, 10 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2.4.1: +428 -0 lines
Diff to previous 1.2.4.1 (colored) to branchpoint 1.2 (colored) next main 1.3 (colored)

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Revision 1.1.2.1 / (download) - annotate - [select for diffs], Sun May 18 17:44:54 2014 UTC (9 years, 10 months ago) by rmind
Branch: rmind-smpnet
Changes since 1.1: +4 -9 lines
Diff to previous 1.1 (colored) next main 1.2 (colored)

sync with head

Revision 1.2.8.1, Wed Feb 12 19:53:49 2014 UTC (10 years, 2 months ago) by tls
Branch: tls-maxphys
Changes since 1.2: +0 -428 lines
FILE REMOVED

file amd64_trap.S was added on branch tls-maxphys on 2014-08-20 00:02:42 +0000

Revision 1.2.4.1, Wed Feb 12 19:53:49 2014 UTC (10 years, 2 months ago) by yamt
Branch: yamt-pagecache
Changes since 1.2: +0 -428 lines
FILE REMOVED

file amd64_trap.S was added on branch yamt-pagecache on 2014-05-22 11:39:28 +0000

Revision 1.2 / (download) - annotate - [select for diffs], Wed Feb 12 19:53:49 2014 UTC (10 years, 2 months ago) by dsl
Branch: MAIN
CVS Tags: yamt-pagecache-base9, tls-maxphys-base, tls-earlyentropy-base, tls-earlyentropy, rmind-smpnet-nbase, rmind-smpnet-base, riastradh-xf86-video-intel-2-7-1-pre-2-21-15, riastradh-drm2-base3, nick-nhusb-base-20150921, nick-nhusb-base-20150606, nick-nhusb-base-20150406, nick-nhusb-base, netbsd-7-nhusb-base-20170116, netbsd-7-nhusb-base, netbsd-7-nhusb, netbsd-7-base, netbsd-7-2-RELEASE, netbsd-7-1-RELEASE, netbsd-7-1-RC2, netbsd-7-1-RC1, netbsd-7-1-2-RELEASE, netbsd-7-1-1-RELEASE, netbsd-7-1, netbsd-7-0-RELEASE, netbsd-7-0-RC3, netbsd-7-0-RC2, netbsd-7-0-RC1, netbsd-7-0-2-RELEASE, netbsd-7-0-1-RELEASE, netbsd-7-0, netbsd-7
Branch point for: yamt-pagecache, tls-maxphys, nick-nhusb
Changes since 1.1: +4 -9 lines
Diff to previous 1.1 (colored)

Change the argument to fpudna() to be the trapframe.
Move the checks for fpu traps in kernel into x86/fpu.c.
Remove the code from amd64/trap.c related to fpu traps (they've not gone
  there for ages - expect to panic in kernel mode).
In fpudna():
- Don't actually enable hardware interrupts unless we need to
  allow in IPIs.
- There is no point in enabling them when they are blocked in software
  (by splhigh()).
- Keep the splhigh() to avoid a load of the KASSERTS() firing.

Revision 1.1 / (download) - annotate - [select for diffs], Tue Jun 25 00:27:22 2013 UTC (10 years, 9 months ago) by uebayasi
Branch: MAIN
CVS Tags: riastradh-drm2-base2, riastradh-drm2-base1, riastradh-drm2-base, riastradh-drm2
Branch point for: rmind-smpnet

Split these to improve diffability.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb <webmaster@jp.NetBSD.org>