CVS log for src/sys/arch/mips/mips/lock_stubs_llsc.S

Up to [cvs.NetBSD.org] / src / sys / arch / mips / mips

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Pull up following revision(s) (requested by andvar in ticket #1773):

	sys/arch/mips/mips/lock_stubs_llsc.S: revision 1.18

Add missing PTR_WORD command in front of 0.
Likely accidentally missed in the commit, since rev 1.9.

Pull up following revision(s) (requested by andvar in ticket #484):

	sys/arch/mips/mips/lock_stubs_llsc.S: revision 1.18

Add missing PTR_WORD command in front of 0.
Likely accidentally missed in the commit, since rev 1.9.

Add missing PTR_WORD command in front of 0.

Likely accidentally missed in the commit, since rev 1.9.

In theory needs pull-ups for netbsd-9, -10, but nobody noticed for 4 years...

mips: Issue a sync plunger at the end of mutex_spin_exit.

Same as mutex_exit.  Relevant only on cnMIPS where the store buffers
get clogged.  Recommended by the Cavium documentation.  No semantic
change, only performance -- this only adds a barrier in some cases
where there was none before, so it can't hurt correctness.

mips: Omit needless SYNC in mutex_exit.

This change deletes a memory barrier.  However, it should be safe:
The semantic requirement for this is already provided by the SYNC_REL
above, before the ll.  And as currently defined, SYNC_REL is at least
as strong as SYNC, so this change can't hurt correctness on its own
(barring CPU errata, which would apply to other users of SYNC_REL and
can be addressed in the definition of SYNC_REL).

Later, perhaps we can relax SYNC_REL to syncw on Octeon if we prove
that it is correct (e.g., if Octeon follows the SPARCv9 partial store
order semantics).

Nix now-unused SYNC macro in asm.h.

mips: Membar audit.

This change should be safe because it doesn't remove or weaken any
memory barriers, but does add, clarify, or strengthen barriers.

Goals:

- Make sure mutex_enter/exit and mutex_spin_enter/exit have
  acquire/release semantics.

- New macros make maintenance easier and purpose clearer:

  . SYNC_ACQ is for load-before-load/store barrier, and BDSYNC_ACQ
    for a branch delay slot -- currently defined as plain sync for MP
    and nothing, or nop, for UP; thus it is no weaker than SYNC and
    BDSYNC as currently defined, which is syncw on Octeon, plain sync
    on non-Octeon MP, and nothing/nop on UP.

    It is not clear to me whether load-then-syncw or ll/sc-then-syncw
    or even bare load provides load-acquire semantics on Octeon -- if
    no, this will fix bugs; if yes (like it is on SPARC PSO), we can
    relax SYNC_ACQ to be syncw or nothing later.

  . SYNC_REL is for load/store-before-store barrier -- currently
    defined as plain sync for MP and nothing for UP.

    It is not clear to me whether syncw-then-store is enough for
    store-release on Octeon -- if no, we can leave this as is; if
    yes, we can relax SYNC_REL to be syncw on Octeon.

  . SYNC_PLUNGER is there to flush clogged Cavium store buffers, and
    BDSYNC_PLUNGER for a branch delay slot -- syncw on Octeon,
    nothing or nop on non-Octeon.

    => This is not necessary (or, as far as I'm aware, sufficient)
       for acquire semantics -- it serves only to flush store buffers
       where stores might otherwise linger for hundreds of thousands
       of cycles, which would, e.g., cause spin locks to be held for
       unreasonably long durations.

  Newerish revisions of the MIPS ISA also have finer-grained sync
  variants that could be plopped in here.

Mechanism:

Insert these barriers in the right places, replacing only those where
the definition is currently equivalent, so this change is safe.

- Replace #ifdef _MIPS_ARCH_OCTEONP / syncw / #endif at the end of
  atomic_cas_* by SYNC_PLUNGER, which is `sync 4' (a.k.a. syncw) if
  __OCTEON__ and empty otherwise.

  => From what I can tell, __OCTEON__ is defined in at least as many
     contexts as _MIPS_ARCH_OCTEONP -- i.e., there are some Octeons
     with no _MIPS_ARCH_OCTEONP, but I don't know if any of them are
     relevant to us or ever saw the light of day outside Cavium; we
     seem to buid with `-march=octeonp' so this is unlikely to make a
     difference.  If it turns out that we do care, well, now there's
     a central place to make the distinction for sync instructions.

- Replace post-ll/sc SYNC by SYNC_ACQ in _atomic_cas_*, which are
  internal kernel versions used in sys/arch/mips/include/lock.h where
  it assumes they have load-acquire semantics.  Should move this to
  lock.h later, since we _don't_ define __HAVE_ATOMIC_AS_MEMBAR on
  MIPS and so the extra barrier might be costly.

- Insert SYNC_REL before ll/sc, and replace post-ll/sc SYNC by
  SYNC_ACQ, in _ucas_*, which is used without any barriers in futex
  code and doesn't mention barriers in the man page so I have to
  assume it is required to be a release/acquire barrier.

- Change BDSYNC to BDSYNC_ACQ in mutex_enter and mutex_spin_enter.
  This is necessary to provide load-acquire semantics -- unclear if
  it was provided already by syncw on Octeon, but it seems more
  likely that either (a) no sync or syncw is needed at all, or (b)
  syncw is not enough and sync is needed, since syncw is only a
  store-before-store ordering barrier.

- Insert SYNC_REL before ll/sc in mutex_exit and mutex_spin_exit.
  This is currently redundant with the SYNC already there, but
  SYNC_REL more clearly identifies the necessary semantics in case we
  want to define it differently on different systems, and having a
  sync in the middle of an ll/sc is a bit weird and possibly not a
  good idea, so I intend to (carefully) remove the redundant SYNC in
  a later change.

- Change BDSYNC to BDSYNC_PLUNGER at the end of mutex_exit.  This has
  no semantic change right now -- it's syncw on Octeon, sync on
  non-Octeon MP, nop on UP -- but we can relax it later to nop on
  non-Cavium MP.

- Leave LLSCSYNC in for now -- it is apparently there for a Cavium
  erratum, but I'm not sure what the erratum is, exactly, and I have
  no reference for it.  I suspect these can be safely removed, but we
  might have to double up some other syncw instructions -- Linux uses
  it only in store-release sequences, not at the head of every ll/sc.

mips: Make sure that mutex_spin_exit works even if !DIAGNOSTIC.

The critical store has been under #ifdef DIAGNOSTIC since, uh, 2011.

Expose the atomicvec vectors via EXPORT_OBJECT so ksyms(4) address
lookups can find them.

This file is only ever used when MULTIPROCESSOR so unifdef MULTIPROCESSOR

Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.

Add a comment to say that an Octeon "sync 4" is "syncw" - sync all writes.

Sync with HEAD

Overhaul the API used to fetch and store individual memory cells in
userspace.  The old fetch(9) and store(9) APIs (fubyte(), fuword(),
subyte(), suword(), etc.) are retired and replaced with new ufetch(9)
and ustore(9) APIs that can return proper error codes, etc. and are
implemented consistently across all platforms.  The interrupt-safe
variants are no longer supported (and several of the existing attempts
at fuswintr(), etc. were buggy and not actually interrupt-safe).

Also augmement the ucas(9) API, making it consistently available on
all plaforms, supporting uniprocessor and multiprocessor systems, even
those that do not have CAS or LL/SC primitives.

Welcome to NetBSD 8.99.37.

update from HEAD

Sync with HEAD

Eliminate redundant load delays.

Machines that need load delays do not have ll/sc instructions.

Sync with HEAD

Trailing whitespace

Sync with HEAD

Sprinle RCSID

Change MIPS to use the common pmap code.
Switch to 8KB pages on CPUs with a R4K MMU.
Simplify cache code.
Merge in most of changes from matt-mips64 branch

Sync with HEAD

assembly no longer include <machine/cpu.h>.  Instead MIPS_CURLWP is gotten
from regdef.h and everything else from assym.h.  <mips/mips_param.h> no
longer include <machine/cpu.h>

Sync with HEAD

Use syncw on OCTEON

More <machine/ include cleanup

Sync with HEAD.

Add MIPS64_RMIXL (XLR/XLS) and MIPS64R2_RMIXL (XLP).  This allows the kernel
to treat this special which is needed for MP support. When accessing the TLB,
always lock the TLB before hand.  If in the miss handlers, the TLB is already
locked let trap deal with the exeception.

Major merge to/from current.
Adds MIPS32/MIPS64 R2 support (24k, 74k, etc.) including COP0_USERLOCAL
Adds support for emulation of rdhwr $3,$29 instruction.
Major cleanup of SMP code.  (stable on multi-core / single thread per core)
llsc locking code only used in MP capable kernels.

sync with head

Sync with HEAD

file lock_stubs_llsc.S was added on branch jruoho-x86intr on 2011-06-06 09:06:06 +0000

file lock_stubs_llsc.S was added on branch matt-nb5-mips64 on 2011-04-29 08:26:26 +0000

file lock_stubs_llsc.S was added on branch rmind-uvmplock on 2011-03-05 20:51:06 +0000

file lock_stubs_llsc.S was added on branch bouyer-quota2 on 2011-03-05 15:09:49 +0000

Major merge forward from matt-nb5-mips64.
New fixup code.
New common SPL code.
New common interrupt code.
Move related variables into structures.
Cleanup locore (move MD variable into it).
Kill StudlyCaps
Use PCU for FPU