CVS log for src/sys/arch/mips/mips/cache.c

Up to [cvs.NetBSD.org] / src / sys / arch / mips / mips

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

s/hander/handler/ and s/hader/header/ in comments and documentation.

Add all the Cavium Networks cpu ids

Pull up following revision(s) (requested by tsutsui in ticket #964):

	sys/arch/mips/mips/cache.c: revision 1.67

Use 32 byte cacheline ops (not 16 byte ones) for R5000 picache.  PR/55138

Commented "I think this is bad copy&paste" from skrll@.

No visible regression on Cobalt Qube 2700 (Rm5230) through
whole installation using netbsd-9 based Cobalt RestoreCD/USB.

Use 32 byte cacheline ops (not 16 byte ones) for R5000 picache.  PR/55138

Commented "I think this is bad copy&paste" from skrll@.
No visible regression on Cobalt Qube 2700 (Rm5230) through
whole installation using netbsd-9 based Cobalt RestoreCD/USB.

Support Octeon Cavium cnMIPS I, II and III cores that have various
non-standard cache configurations (in terms of following MIPS spec
for defining cache configurations).

Move (most) Octeon support into a single place.

Move some "case 0" statements to be first in their switch statements.

Adjust previous - move consolidated debug printfs _after_ core specific
overrides, not before them.

Make core specific overrides a bit more readable / scalable (switch
instead of if / else / ...).
Move debug printfs to after core specific overrides in case any config
is updated.

KNF police - long lines and comments.

Merge changes from current as of 20200406

s/defintion/definition/ in comment.

Sync with HEAD

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int.  The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER!  Some subsystems have

	#define min(a, b)	((a) < (b) ? (a) : (b))
	#define max(a, b)	((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX.  Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate.  But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all.  (Who knows, maybe in some cases integer
truncation is actually intended!)

Pull up following revision(s) (requested by flxd in ticket #1578):
	sys/arch/mips/mips/cache.c: revision 1.59
Add missing call to mips_dcache_compute_align() affecting "modern" MIPS
(MIPS32{,R2}/MIPS64{,R2}). Thanks jmcneill@; OK skrll@.

Pull up following revision(s) (requested by flxd in ticket #601):
	sys/arch/mips/mips/cache.c: revision 1.59
Add missing call to mips_dcache_compute_align() affecting "modern" MIPS
(MIPS32{,R2}/MIPS64{,R2}). Thanks jmcneill@; OK skrll@.

Add missing call to mips_dcache_compute_align() affecting "modern" MIPS
(MIPS32{,R2}/MIPS64{,R2}). Thanks jmcneill@; OK skrll@.

update from HEAD

Sync with HEAD

Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)

Set mci_{,i}cache_alias_mask for all variants that can have virtual cache
aliases

Set ncolors appropriately

These align to dcache and expect icache aliases to be dealt with by the
pmap directly.

Really fix typo that got dcache alias mask set from icache way_mask

Fix typo that got dcache alias mask set from icache way_mask

Sync with HEAD

Comment indentation

Sync with HEAD

Remove old and incorrect comments

Trailing whitespace

Use sdcache routines.
Remove old cache support.
Switch to new cache routines.

Change MIPS to use the common pmap code.
Switch to 8KB pages on CPUs with a R4K MMU.
Simplify cache code.
Merge in most of changes from matt-mips64 branch

Sync with HEAD

mci_cache_virtual_alias is bool so use true and false, not 1 and 0.

Sync with HEAD

Initial import of Cavium Octeon and Octeon Plus SoC and
specifically Ubiquiti Networks EdgeRouter LITE support.
Currently the ethernet and uart are worked.
This support was contributed by Internet Initiative Japan Inc.

sync with head

When running an N32 kernel, run it with 64-bit addresses even though the
kernel itself will only use 32-bit addresses.  There are exceptions.
bus_space_handles are now register_t instead of intptr_t.  This allows them
to contain XKPHYS addresses.  Now bus_space can use XKPHYS addresses and
not required non-KSEG1 devices to be mapped in KSEG2 thereby leaving more
KSEG2 space for the kernel to use.

The cache range routines (but not index routines) now take a register_t
instead of vaddr_t so they can too take a XKPHYS address.  This allows the
pmap to use a page's XKPHYS address to clean sync the icache thereby avoiding
massive icache invalidations.

Since "cache" instruction effects are global to all CPUs and their caches, we
can use the above to greatly simplify MP page isyncs.  If using an O32 kernel
with pages outside KSEG0, index ops still need to be performed since there
isn't an a quick way of mapping the page.

Add an optimization for UP system with non-virtually tagged caches (which are
most of them these days).

If a page needs to be have an icache_sync performed and the page has a direct
map alias (XKPHYS or KSEG0), then don't do an index op; instead do a range op
on the XKPHYS or KSEG0 address.  This results in unneeded fewer cache line
invalidations.

Use MIPS_*CACHE_VIRTUAL_ALIAS instead of mci->mci_*

Rework (rewrite) the cache code for MIPS32/MIPS64.  Use an assembly stub
for generate the cache instructions.  Add some more intern routines so that
cache_mipsNN.c only had the "generic" all/index routines.

Add code to deal SDcache settings in CFG2.
Add support for CFG7 handling for MTI cores.
Cleanup cache alias handling.

Add workaround for RMI cpus (not really needed but good for completeness)

For RMI, use wbinv for wb ops since there is no wb.

Catchup with rmind-uvmplock merge.

sync with head

Make GDIUM build again after matt-nb5-mips64 merge. untested as I don't have
this hardware, but I'll use this as a base for Lemote Fulong support.

Sync with HEAD.

Add MIPS64_RMIXL (XLR/XLS) and MIPS64R2_RMIXL (XLP).  This allows the kernel
to treat this special which is needed for MP support. When accessing the TLB,
always lock the TLB before hand.  If in the miss handlers, the TLB is already
locked let trap deal with the exeception.

Make uvm_map recognize UVM_FLAG_COLORMATCH which tells uvm_map that the
'align' argument specifies the starting color of the KVA range to be returned.

When calling uvm_km_alloc with UVM_KMF_VAONLY, also specify the starting
color of the kva range returned (UMV_KMF_COLORMATCH) and pass those to
uvm_map.

In uvm_pglistalloc, make sure the pages being returned have sequentially
advancing colors (so they can be mapped in a contiguous address range).
Add a few missing UVM_FLAG_COLORMATCH flags to uvm_pagealloc calls.

Make the socket and pipe loan color-safe.

Make the mips pmap enforce strict page color (color(VA) == color(PA)).

Major merge to/from current.
Adds MIPS32/MIPS64 R2 support (24k, 74k, etc.) including COP0_USERLOCAL
Adds support for emulation of rdhwr $3,$29 instruction.
Major cleanup of SMP code.  (stable on multi-core / single thread per core)
llsc locking code only used in MP capable kernels.

sync with head

Add separate support for MIPS32R2 and MIPS64R2.
Use EHB/SSNOP and jr.hb ra as appropriate (COP0_SYNC now uses them).
Add support for COP_0_HWRENA and COP_0_USERLOCAL (use by rdhwr $3,$29
instruction for TLS support).
Add mips3+ reserved instruction handler to emulate rdhwr is many fewer
instructions.

sync with head

Sync with HEAD

Major merge forward from matt-nb5-mips64.
New fixup code.
New common SPL code.
New common interrupt code.
Move related variables into structures.
Cleanup locore (move MD variable into it).
Kill StudlyCaps
Use PCU for FPU

Sync with HEAD

Fix build of malta/sbmips.

Add support for the Extensible MIPS ("eMIPS") platform.  The
NetBSD/emips port runs on Xilinx and Beecube FPGA systems and the
Giano system simulator.

eMIPS is a platform developed at Microsoft Research for researching
reconfigurable computing.  eMIPS allows dynamic loading and scheduling
of application-specific circuits for the purpose of accelerating
computations based on the current workload.

NetBSD eMIPS support for NetBSD 4.x was written at Microsoft Research
by Alessandro Forin and Neil Pittman.  Microsoft Corporation has
donated full copyright to The NetBSD Foundation.

Platform support for eMIPS is the first part of Microsoft's
contribution.  The second part includes the hardware accelerator
framework and will be proposed on tech-kern soon.

Revamp pmap.  Add exec page caching logic from powerpc oea pmap.  Shrink struct
vm_page by placing the first pv_entry in it.  Remove pv_flags since nothing
really needed it.  Add pmap counters.  Rework virtual cache alias logic.
Allow pmap_copy_page and pmap_zero_page to deal with non-KSEG0 mappable pages.

Adjust things to the new world order.

Revamp things a bit.  Move of the globals mips uses into either cpu_info,
mips_options, or mips_cache_info.  Make MALT64 be able to boot MULTIPROCESSOR.
(some pmap MP work).

sync with head.

Fix loongson2 sdcache init.

Flush by increasing way, then increasing addr.  flush L1 before L2 (even
though according to the specification it should be needed).  Reset
mips_sdcache_size to 0 so we will configure it.

If Loongson2 way size is <= page size, don't enable mips_virtual_alias

Add loongson2 specific cache ops

Use mips3_get_cache_config for Loongson2

Update Loongston.  Set colors and mark it as having virtual aliases.

Change MIPS64_LOONGSON2 to MIP3_LOONGSON2 since it's a MIPS3 and not MIPS64.

The Loongson2 secondary cache is unified.

Add Loongson2 support.

sync with head.

__asm__ -> __asm
__const__ -> const
__inline__ -> inline
__volatile__ -> volatile

Sync with head.

sync with head.

Use PAGE_MASK macro instead of (PAGE_SIZE - 1).

Pull up following revision(s) (requested by tsutsui in ticket #961):
	sys/arch/mips/mips/cache.c: revision 1.31
Check possibility of virtual alias correctly in primaly_cache_is_2way case
(though I guess all those CPUs have L1 cache larger than 8KB).

Pull up following revision(s) (requested by tsutsui in ticket #961):
	sys/arch/mips/mips/cache.c: revision 1.30
Partially revert change in sys/arch/mips/mips/cache.c rev 1.27,
i.e. override mips_cache_alias_mask and mips_cache_prefer_mask
with values which match MIPS3_MAX_PCACHE_SIZE (32KB), rather than
leave them actual primary virtual indexed cache size (8KB or 16KB).
Also add comments about what the value means there.
I thought the VCE on R4000/R4400 occurred only if actual virtual alias
was detected because there was an article which mentioned that VCE
detection logic was different according to primary cache size and
it looked reasonable. But all other articles I can find later
(http://www.linux-mips.org/archives/linux-mips/1998-05/msg00084.html etc.)
claimes that VCE detection logic always verifies all 3 bits
between vaddr[14:12] and PIdx[2:0] regardless of primary cache size
(i.e. VCE could occur even if there is no actual virtual alias), and
in fact VCED still happens with the mask values adjusted for 16KB L1
but it doesn't with ones for MIPS3_MAX_PCACHE_SIZE on my R4400 news5000.

Pull up following revision(s) (requested by tsutsui in ticket #961):
	sys/arch/mips/mips/cache.c: revision 1.27
	sys/arch/mips/include/cache.h: revision 1.8
	sys/arch/mips/mips/pmap.c: revision 1.158
	sys/arch/mips/mips/vm_machdep.c: revision 1.106
	sys/arch/mips/mips/mem.c: revision 1.30
	sys/arch/mips/include/pmap.h: revision 1.47
Add a workaround to handle virtual alias which may cause data corruption
on R5000/Rm52xx machines:
- Add a new global variable mips_cache_virtual_alias in mips/cache.c,
  which indicates that VIPT cache on the CPU could cause virtual alias
  and software support is required to handle it. (i.e. no VCED/VCEI)
- Add several cache flush/invalidate ops around KSEG0 access which
  might cause virtual alias if mips_cache_virtual_alias is true.
  (note checking mips_sdcache_line_size isn't valid for R5000/Rm52xx
   because only R4000/R4400 with L2 cache have VCED/VCEI)
- Remove a global variable mips_sdcache_forceinv, which is now superseded
  by new mips_cache_virtual_alias.
While here, also change some R4000/R4400 cache ops:
- Don't override mips_cache_alias_mask and mips_cache_prefer_mask with
  values based on MIPS3_MAX_PCACHE_SIZE for R4000/R4400 with L2 cache
  because it's still worth to reduce VCED/VCEI.
- Flush dcache in pmap_zero_page(9) unconditionally on all MIPS_HAS_R4K_MMU
  CPUs and remove cache flush code from cpu_lwp_fork() in vm_machdep.c.
Thanks to Markus W Kilbinger for testing patches on port-cobalt/port-mips.
XXX This fix is just a workaround because it doesn't handle all possible
XXX virtual aliases. As discussed on port-mips, maybe the real fix
XXX for virtual alias is to change MI UVM to adapt it to VIPT cache.
XXX (all VA mappings against the same PA must have the same VAC index etc.)

Check possibility of virtual alias correctly in primaly_cache_is_2way case
(though I guess all those CPUs have L1 cache larger than 8KB).

Sync with HEAD. Here we go again...

Partially revert change in sys/arch/mips/mips/cache.c rev 1.27,
i.e. override mips_cache_alias_mask and mips_cache_prefer_mask
with values which match MIPS3_MAX_PCACHE_SIZE (32KB), rather than
leave them actual primary virtual indexed cache size (8KB or 16KB).
Also add comments about what the value means there.

I thought the VCE on R4000/R4400 occurred only if actual virtual alias
was detected because there was an article which mentioned that VCE
detection logic was different according to primary cache size and
it looked reasonable. But all other articles I can find later
(http://www.linux-mips.org/archives/linux-mips/1998-05/msg00084.html etc.)
claimes that VCE detection logic always verifies all 3 bits
between vaddr[14:12] and PIdx[2:0] regardless of primary cache size
(i.e. VCE could occur even if there is no actual virtual alias), and
in fact VCED still happens with the mask values adjusted for 16KB L1
but it doesn't with ones for MIPS3_MAX_PCACHE_SIZE on my R4400 news5000.

Check MIPS3_CONFIG_CS and adjust csizebase at runtime on MIPS_R4100 CPUs,
and remove "XXXCDC: THIS MIPS3_4100 SPECIAL CASE SHOULD GO AWAY" part
from cpuregs.h. Tested on gxemul.

BTW, cache.c doesn't have MIPS_RC32364 config which was added
in mips_machdep.c rev 1.101?

Cast the various uses of cache_noop to the proper function pointer
type of the target instead of (void*).  Appeases -Wcast-qual.

sync with -current

Sync with HEAD.

sync with head.

Add a workaround to handle virtual alias which may cause data corruption
on R5000/Rm52xx machines:
- Add a new global variable mips_cache_virtual_alias in mips/cache.c,
  which indicates that VIPT cache on the CPU could cause virtual alias
  and software support is required to handle it. (i.e. no VCED/VCEI)
- Add several cache flush/invalidate ops around KSEG0 access which
  might cause virtual alias if mips_cache_virtual_alias is true.
  (note checking mips_sdcache_line_size isn't valid for R5000/Rm52xx
   because only R4000/R4400 with L2 cache have VCED/VCEI)
- Remove a global variable mips_sdcache_forceinv, which is now superseded
  by new mips_cache_virtual_alias.

While here, also change some R4000/R4400 cache ops:
- Don't override mips_cache_alias_mask and mips_cache_prefer_mask with
  values based on MIPS3_MAX_PCACHE_SIZE for R4000/R4400 with L2 cache
  because it's still worth to reduce VCED/VCEI.
- Flush dcache in pmap_zero_page(9) unconditionally on all MIPS_HAS_R4K_MMU
  CPUs and remove cache flush code from cpu_lwp_fork() in vm_machdep.c.

Thanks to Markus W Kilbinger for testing patches on port-cobalt/port-mips.


XXX This fix is just a workaround because it doesn't handle all possible
XXX virtual aliases. As discussed on port-mips, maybe the real fix
XXX for virtual alias is to change MI UVM to adapt it to VIPT cache.
XXX (all VA mappings against the same PA must have the same VAC index etc.)

sync with head.  xen and whitespace.  xen part is not finished.

Sync with HEAD.

Hi Perry!

Add a hint variable (mips_sdcache_forceinv, explicitly initialized to zero)
that tells pmap_zero_page() and pmap_copy_page() to unconditionally invalidate
pages for r5k-class CPUs with secondary cache.

This behavior must be explicitly enabled by setting mips_sdcache_forceinv to 1.

This is the last bit of a patch that has been kicked around since 2000 between
rafal@, tsutsui@, and myself.

Sync with HEAD.

Implement r5k indexed writeback-invalidate, and fix usage of Page_Invalidate_S.

Originally written by rafal@ back in April 2003.  Field-tested by many
people since.

(I am not committing the pmap hack at this time; although pmap changes are
necessary to fully address the r5k panic/coma problems, the implementation
needs further thought)

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Vr4100 and Vr4300 are not capable of having external caches.

cache_r10k.c rev. 1.1 is broken. Because,

	1) R10k uses VA0 to select cache ways, but in rev. 1.1, VA14
	   is used instead.
	2) R10k does not support HitWriteBack and should map HitWriteBack
	   to HitWriteBackInvalidate, but in rev. 1.1, HitWriteBack is not
	   handled properly.

So, cache_r10k.c rev. 1.1 was replaced by new implementation.

It seems r4k_sdcache_wb_range_NN() function can't handle
R10000 L2 cache (which is 2-way set-associative write-back),
so use r4k_sdcache_wbinv_range_NN() for workaround until someone
implement proper r10k_sdcache_*() ops.

Problem reported by Christopher SEKIYA.

Add R10000 cache ops, written by KIYOHARA Takashi and posted on port-sgimips.
Enabled by options ENABLE_MIPS4_CACHE_R10K for now.

__KERNEL_RCSID()

Protect uses of MIPS_R5000 with #ifndef ENABLE_MIPS_R3NKK in new code just
as the old code does.

Add support for R5k secondary caches, from code Chris Sekiya sent me a long
time ago, with small tweaks by me.  Since the R5k doesn't do VCE, the pmap
still needs to be whacked for R5kSC CPUs to work correctly, but this is a
start.

Update to consistently use Broadcom GPL-compatible license on all SiByte code.

Sync with HEAD.

Add the MIPS3_CONFIG_SE (name taken from Rm52xx manual) bit, which is the
external cache enable bit -- this allows software to enable or disable the
(external) L2 cache on the R5k and Rm527x and the (external) L3 cache on
the Rm7k.  If the (external) cache is disabled, treat it as if there were
no cache for the purposes of the cache setup code.

Also, update sgimips code to use the new name.

Sync with HEAD.

Add support for caches where the data cache is fully coherent, and
either requires flushing either only when the I cache ops are used
or not at all.  Currently only used by MIPS32/MIPS64 cache code.

Sync with HEAD.

New generic way-aware MIPS32/64 range-index cache functions with proper
handling for phyiscally-indexed caches where the way size is greater than
the page size.
These work fine with pass 1 SB1 cores, so g/c those workarounds.

Much thanks to Chris Demetriou for many suggestions and helping me get
my head around all this.

Catch up to -current

Make cache size/mask variables unsigned.

Catch up to -current.

sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work

remove trailing \n in panic().  approved perry.

catch up with -current on kqueue branch

Catch up to -current.

Use the new 2way mips{32,64} cache ops.

Catch up to -current.
(CVS: It's not just a program. It's an adventure!)

Catch up with -current.

Add MIPS32/64 cache setup code (from Broadcom Corp).

Catch up to -current.

Sync w/ -current.

move TX39 specific cache configuration code to cache.c

add VR4131 cache-op bug workaround code.
we can't use Hit_WriteBack_Invalidate.

Sync kqueue branch with -current.

file cache.c was added on branch kqueue on 2002-01-10 19:46:03 +0000

R4000/R4400 always detects virtual alias as if
primary cache size is 32KB. Actual primary cache size
is ignored wrt VCED/VCEI.

Added Vr4131 support.

Add 32B/l L1 D/I-cache ops for newer ARC machines.

Add mips_dcache_align and mips_dcache_align_mask variables that
contain information suitable for allowing other parts of the kernel
to determine if a memory region is aligned to the largest data cache
line size present in the system.

Add a mips_dcache_compute_align() function which must be called whenever
one of the data cache line size variables is changed, in order to
compute mips_dcache_align and mips_dcache_align_mask.

Add 128b/l L2 cache ops.

file cache.c was added on branch nathanw_sa on 2002-02-28 04:10:43 +0000

Merge the thorpej-mips-cache branch onto the trunk.  This is an
overhaul of how caches are handled for NetBSD's MIPS ports.

improve  R4000/4400 secondary cache support.
add support for secondary cache line sizes 16, 64, 128.

new cache code for R5900 and playstation2

make this compile with TX39

New MIPS cache primitives and code to configure which ones are used.

file cache.c was initially added on branch thorpej-mips-cache.