CVS log for src/sys/dev/raidframe/rf_dagutils.c

Up to [cvs.NetBSD.org] / src / sys / dev / raidframe

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

Sync with HEAD.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools.  Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

Mostly merge changes from HEAD upto 20200411

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
  and the callback structure; separate them into "function" and "value"
  callbacks.
- make the wait function signature consistent.

Sync with HEAD

Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

update from HEAD

Sync with HEAD

Don't use for (...); by using an explicit continue as body.

Sync with HEAD.

sync with head

convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

ansify function definitions

sync with head.

Sync with HEAD.

Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

sync with head.

sync with head.

Sync with head.

__unused removal on arguments; approved by core.

sync with head

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

sync with head.

sync with head.

rf_DiskUnlockFunc and rf_DiskUnlockFuncForThreads are never used.  Punt them.
rf_DiskUnlockUndoFunc is in the same boat.  Punt it too.

merge ktrace-lwp.

Sync with HEAD. Here we go again...

Pull up revision 1.46 (requested by oster in ticket #472):
- avoid variable shadowing
- add a lot of const
- remove parameters from function declarations

- avoid variable shadowing
- add a lot of const
- remove parameters from functin declarations

sync with -current

sync with head.  xen and whitespace.  xen part is not finished.

Sync with HEAD.

Hi Perry!

nuke trailing whitespace

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Pull up revision 1.44 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe.  They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space.  rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer().  rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure().  In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move	RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s.  This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer().  Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191

These changes complete the effective removal of malloc() from all
write paths within RAIDframe.  They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.

With this new code the 5-component RAID 5 set panics are now gone.

It is also now also possible to swap to RAID 5.

The changes made are:

1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space.  rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer().  rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.

2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure().  In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().

3) Move	RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s.  This is more consistent with the
original code, and will not result in items being freed "too early".

4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.

5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().

6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).

7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer().  Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).

8) Remove RF_IOBufHeader and all references to it.

9) Remove desc->cleanupList and all references to it.

Fixes PR#20191

Partially back out some changes that were causing grief with
RAID5 sets with more than 3 drives.  Still need to figure out why
the original changes were losing, but need the version in tree reliable
first!

Huge THANKS to Juergen Hannken-Illjes for helping track down
the changes that were causing the lossage.

Change signature of rf_AllocBuffer() to take a dag_h and buffer size
instead of an PDA and an alloclist.  This lets us do the vple dance
inside of rf_AllocBuffer().

Cleanup usage of rf_AllocIOBuffer() and use rf_AllocBuffer() instead.

Fix all uses of rf_AllocBuffer() to conform to the new way of doing
things.

For each RAID set, pre-allocate a number of "emergency buffers" to be
used in the event that we can't malloc a buffer of the appropriate
size in the traditional way.  rf_AllocIOBuffer() and rf_FreeIOBuffer()
deal with allocating/freeing these structures.  These buffers are
stored in a list on the 'iobuf' list.  iobuf_count keeps track of how
many buffers are available, and numEmergencyBuffers is the effective
"high-water" mark for the freelist.  The buffers allocated by
rf_AllocIOBuffer() are stripe-unit sized, which is the maximum
size requested by any of the callers.

Add an iobufs entry to RF_DagHeader_s.  Use it for keeping track of
buffers that get allocated from the free-list.

Add a "generic list" pool (VoidPointerListElement Pool) for elements
used to maintain a list of allocated memory.  [It is somewhat less
than ideal to add another little pool to handle this...]

Teach rf_AllocBuffer() to use the new rf_AllocIOBuffer().  Modify
other Mallocs to use rf_AllocIOBuffer(), and to update dag_h->iobufs as
appropriate.

Update rf_FreeDAG() to handle cleanup of dag_h->iobufs.

While here, add some missing pool_destroy() calls for a number of pools.

With these changes, it should (in theory) be possible to swap on
RAID 5 sets again.  That said, I've not had any success there yet --
but the last issue I saw at least wasn't in RAIDframe. :-}

[There is room for this code to become a bit more consise, but I
wanted to do a checkpoint here with something known to work :) ]

Introduce a dual-purpose pool for providing pointer and param "caches"
for RF_DagNode_t's.  Scale the structure size based on RF_MAXCOL.
Use the new allocation method in InitNode().  Note that we can't get
rid of the mallocs in there until we can prove that this new
allocation method is a strict upper bound.  Unless someone tries
running a RAID set with 40 components, the mallocs here shouldn't
shouldn't be an issue.  (and if someone does make a set with 40 components
they will run into other issues with other constants long before
then)

Take care of six more mallocs:

- Pull rf_FreePhysDiskAddr() out from under a #ifdef, since we're now
going to use it.

- Add a pda_cleanup_list into the DAG header.  Use it in rf_FreeDAG() to
cleanup any PDA's that get allocated but have no "easy" way of being
located and freed when the DAG completes.

- numStripeUnitsAccessed is a per-stripe value, and has a maximum
value equal to the number of colums (thus limited by RF_MAXCOL).
Use this knowledge to set a high-bound on overlappingPDAs, and stuff
it on the stack instead of malloc'ing it all the time!  This costs us
a whopping 40 bytes on the stack, but saves a malloc() and a free().

- Introduce a 'dagnode' pool.  Initialize it and allow for cleanup.
Provide rf_AllocDAGNode() and rf_FreeDAGNode() to handle
allocation/freeing.

- Introduce a "nodes" linked list of RF_DagNode_t's into the DAG header.
Initialize nodes in InitHdrNode().  Arrange for nodes cleanup in rf_FreeDAG().

- Add a "list_next" to RF_DagNode_t to keep track of nodes on the
above "nodes" list.  (This is distinct from the "next" field of
RF_DagNode_t, which keeps track of the firing order of nodes.)
"list_next" gets used in the cleanup routines, and in traversing
through a set of nodes that belong to a particular set of nodes
(e.g. those belonging to xorNodes for a given DAG).

- use rf_AllocDAGNode() instead of mallocs of variable-sized arrays of
RF_DagNode_t's.  Mostly mechanical changes to convert the DAG construction
from "access nodes via an array index" to "access nodes via a 'nextnode'
pointer".

- rework a couple of tricky spots where assumptions about the node order
was being abused.

- performance remains consistent with performance before these changes.

[Thanks to Simon Burge (simonb at you.know.where) for looking over
the mechanical changes to make sure I didn't biff anything.]

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe.  Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe.  Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

- Introduce rf_pools which contains all of the various global pools used
by RAIDframe.  Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe.  Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)

Re-work rf_GenerateFailedAccessASMs() to simplify things a bit.
rf_AllocBuffer() is available, so use it to get buffer space instead
of the previous RF_Malloc() bits.  Saves a few bytes, but more
importantly makes the code much more readable.

Pretty up a bit of unused code.

rf_AllocBuffer() doesn't do anything with its dag_h parameter.  Nuke
it, and adjust callers.

Minor tabbing cleanup.  No functional change.

Introduce RF_DEBUG_DAG and use it to #if-out rf_dagDebug sections.
(i386 GENERIC kernel shrinks by 1.6K)

- remove the RF_*_INC's, as necessary.  They are not needed any more.
- introduce RF_MIN_*'s, as necessary.  These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.

Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory.  Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing.  Shaves another 600 bytes off of an i386 GENERIC kernel.

We'd better have gotten a dag header from the pool.  In any event, callers
arn't checking what we return anyway. (Cleanup memory allocations.)

Stripe functions are now handled by a linked-list instead of a
runtime-variable array.

Fix a bug where stripeFuncs was being freed, and then being used after
(in the case of numStripesBailed > 0).

Add forgotten pool_destroy().

Use a dynamically allocated linked list of dagLists instead of using a
dynamically allocated variable-sized array (dagArray).  Convert code
to use the new linked list stuff instead of the array stuff (the ratio
of one dagList per stripe still applies).  The big advantage is in
being able to more efficiently allocate the dagLists on-the-fly, and
not have to know the size(s) of the array beforehand.

Since the LOCK and UNLOCK flags are never used, no need in extracting them.
Collapse the related variables down to zero.  That means 'flags' is 0
as well.  Nuke the extraction macros, a bunch of the variables, and replace
'flags' as well.

Some days you wonder if some of the function declaration consistency
was just an accident in the first place.  Cleanup function decls and
a few comments.  [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

- first kick at a major reworking of RAIDframe's memory allocation code:
  - all freelists converted to pools
  - initialization of structure members in certain cases where
	code was relying on specific allocation and usage properties
	to keep structures in a "known state" (that doesn't work with
	pools!).
  - make most pool_get() be "PR_WAITOK" until they can be analyzed
	further, and/or have proper error handling added.
  - all RF_Mallocs zero the space returned, so there is no difference
	between RF_Calloc and RF_Malloc.  In fact, all the RF_Calloc()'s
	do is tend to do is get things horribly confused.
	Make RF_Malloc() the "general memory allocator", with
	RF_MallocAndAdd() the "general memory allocator with
	allocation list".
  - some of these RF_Malloc's et al. are destined to disappear.
  - remove rf_rdp_freelist entirely (it's not used anywhere!)
  - remove: #include "rf_freelist.h"
  - to the files that were relying on the above, add: #include "rf_general.h"
  - add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
	about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K.  RAIDframe now
weighs in at about 162K on i386.

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc.  All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K.  For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

constify some

Sync with HEAD.

rf_SelectMirrorDiskPartition() is only needed in a few cases.  #if it
out in the rest.  Thanks to Krister!

Catch up to -current.

sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work

Remove unneeded variables and lame assignments.  Thanks Simon B.!

rf_MakePropListEntry isn't used anywhere, so nuke it.  Thanks Krister!

Another couple of functions that arn't used unless one is debugging RAIDframe.

Catch up to -current.

Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space.  Also improves readability of code.

sync kqueue branch with HEAD

catch up with -current.

Catch up to -current.

Nuke stuff dealing with the experimental memChunk code.  It's unused, and
currently only contributing to bloat.

Catch up to -current.

catch up with -current.

Most folks won't need the DAG printing and verification routines.
Introduce a #define to toggle them on/off.  Disable calls to
rf_PrintDAGList().  Saves ~6K on GENERIC+DEBUG kernel on i386.

Minor cleanup.

rf_compute_workload_shift() is only used by the CHAINDECLUSTER stuff,
so only include it if needed.

Catch up to -current.
(CVS: It's not just a program. It's an adventure!)

Catch up with -current.

Correct misspellings of "failed".

Sync kqueue branch with -current.

Catch up to -current.

add RCSIDs

Catch up to -current.

Catch up with -current. Fix some bogons in the sparc64 kbd/ms
attach code. cd18xx conversion provided by mrg.

Step 2 of the disentanglement.  We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

Catch up with -current.

update to -current

bzero -> memset

Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.

Pull up to last week's -current.

Pull up revision 1.6 (requested by oster):
  Trust only the data disk if the mirror is not known to
  be up-to-date.

Trust only the data disk if the mirror is not known to be up-to-date.
(this should have been committed with a previous fix for the same
problem in another function in this file :( )

Sync with -current

Pull up revision 1.5 (requested by oster):
  Do not trust the parity disk for read balancing in a RAID 1 set
  if we know that the parity might not be up-to-date (and if we
  have a choice in the matter).

If we have a choice: do not trust the parity disk for read
balancing in a RAID 1 set if we know that the parity might not
be up-to-date.  Thanks to Thor for bringing this to my attention.

rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.

Phase 2 of the RAIDframe cleanup.  The source is now closer to KNF
and is much easier to read.  No functionality changes.

RAIDframe cleanup, phase 1.  Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff.  This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University.  Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.