CVS log for src/sys/dev/raidframe/Attic/rf_copyback.c

Up to [cvs.NetBSD.org] / src / sys / dev / raidframe

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

We no longer need the deprecated copyback functionality now that
incorporating a used spare is automatic.

Copyback has always been an issue, as to do a copyback all IO to
the array had to be suspended, and so was very, very unlikely to
have been used in anything resembling a production system.

Sync with HEAD.

rf_CreateDiskQueueData() no longer uses waitflag, and will always succeed.
Cleanup the error path for the (no longer needed) PR_NOWAIT cases.

Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools.  Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.

Mostly merge changes from HEAD upto 20200411

Merge changes from current as of 20200406

Switch to vn_bdev_open* functions.

fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
  and the callback structure; separate them into "function" and "value"
  callbacks.
- make the wait function signature consistent.

Sync with HEAD

- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.

Rebase to HEAD as of a few days ago.

Rebase.

Change dk_lookup() to return an anonymous vnode not associated with
any file system.  Change all consumers of dk_lookup() to get the
device from "v_rdev" instead of VOP_GETATTR() as specfs does not
support VOP_GETATTR().  Devices obtained with dk_lookup() will no
longer disappear on forced unmounts.

Fix for PR kern/48849 (root mirror raid fails on shutdown)

Welcome to 6.99.44

Pull up the following revisions(s) (requested by mrg in ticket #1774):
	sbin/raidctl/raidctl.c:			revision 1.52
	sys/dev/raidframe/raidframevar.h:	revision 1.15
	sys/dev/raidframe/rf_copyback.c:	revision 1.45
	sys/dev/raidframe/rf_disks.c:		revision 1.78
	sys/dev/raidframe/rf_netbsdkintf.c:	revision 1.282,1.284
	sys/dev/raidframe/rf_reconstruct.c:	revision 1.111

Fix garbage values in partitionSizeHi with RAID array > 2TB. Stops the check against
rf_component_label_partitionsize() failing and stopping auto-configure.

Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock.  Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.

Remove unused 'struct lwp *' from rf_getdisksize.  No functional changes.

Sync with HEAD.

sync with head

convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.

convert mcpair to kmutex.

sync with head

Sync with HEAD

Define accessors for number of blocks and partition size in the
component label and use them where appropriate.  Disscussed on tech-kern.

Pull up following revision(s) (requested by mrg in ticket #1468):
	sys/dev/raidframe/rf_disks.c: revision 1.74
	sys/dev/raidframe/raidframevar.h: revision 1.14
	sys/dev/raidframe/rf_netbsdkintf.c: revision 1.275
	sys/dev/raidframe/rf_copyback.c: revision 1.43
	sys/dev/raidframe/rf_reconstruct.c: revision 1.109
add support for >2TB raid devices.
- - add two new members to the component label:
     u_int numBlocksHi
     u_int partitionSizeHi
  and store the top 32 bits of the real number of blocks and
  partition size.  modify rf_print_component_label(),
  rf_does_it_fit(), rf_AutoConfigureDisks() and
  rf_ReconstructFailedDiskBasic().
- - call disk_blocksize() after disk_attach() [ from mlelstv ]
- - shift the block number relative to DEV_BSHIFT in raidstart()
  and InitBP() so that accesses work for non 512-byte devices.
  [ from mlelstv ]
- - update rf_getdisksize() to use the new getdisksize() [ from
  mlelstv.  this part needs a separate change for netbsd-5. ]
reviewed by: oster, christos and darrenr

Introduce struct pathbuf. This is an abstraction to hold a pathname
and the metadata required to interpret it. Callers of namei must now
create a pathbuf and pass it to NDINIT (instead of a string and a
uio_seg), then destroy the pathbuf after the namei session is
complete.

Update all namei call sites accordingly. Add a pathbuf(9) man page and
update namei(9).

The pathbuf interface also now appears in a couple of related
additional places that were passing string/uio_seg pairs that were
later fed into NDINIT. Update other call sites accordingly.

Sync with HEAD.

add support for >2TB raid devices.

- add two new members to the component label:
     u_int numBlocksHi
     u_int partitionSizeHi
  and store the top 32 bits of the real number of blocks and
  partition size.  modify rf_print_component_label(),
  rf_does_it_fit(), rf_AutoConfigureDisks() and
  rf_ReconstructFailedDiskBasic().

- call disk_blocksize() after disk_attach() [ from mlelstv ]

- shift the block number relative to DEV_BSHIFT in raidstart()
  and InitBP() so that accesses work for non 512-byte devices.
  [ from mlelstv ]

- update rf_getdisksize() to use the new getdisksize() [ from
  mlelstv.  this part needs a separate change for netbsd-5. ]


reviewed by: oster, christos and darrenr

sync to netbsd-5

sync with head

Pull up following revision(s) (requested by tron in ticket #1187):
	sbin/raidctl/raidctl.8: revisions 1.57-1.59 via patch
	sbin/raidctl/raidctl.c: revision 1.42 via patch
	sys/dev/raidframe/files.raidframe: revision 1.8 via patch
	sys/dev/raidframe/rf_copyback.c: revision 1.42 via patch
	sys/dev/raidframe/rf_disks.c: revision 1.72 via patch
	sys/dev/raidframe/rf_driver.c: revision 1.122 via patch
	sys/dev/raidframe/rf_engine.c: revision 1.40 via patch
	sys/dev/raidframe/rf_kintf.h: revision 1.21 via patch
	sys/dev/raidframe/rf_netbsdkintf.c: revision 1.269 via patch
	sys/dev/raidframe/rf_paritymap.c: revisions 1.1-1.3 via patch
	sys/dev/raidframe/rf_paritymap.h: revision 1.1 via patch
	sys/dev/raidframe/rf_parityscan.c: revision 1.33 via patch
	sys/dev/raidframe/rf_parityscan.h: revision 1.8 via patch
	sys/dev/raidframe/rf_raid.h: revision 1.38 via patch
	sys/dev/raidframe/rf_reconstruct.c: revision 1.108 via patch
	sys/dev/raidframe/rf_states.c: revision 1.44 via patch
	sys/dev/raidframe/raidframeio.h: revision 1.6 via patch
	sys/dev/raidframe/raidframevar.h: revision 1.13 via patch
Pull up the RAIDframe parity map Summer Of Code project.
Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes.  Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Finally commit the RAIDframe parity map Summer Of Code project.

Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes.  Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Discussed on tech-kern@ to a general air of approval; exhortations to
commit from mrg@, christos@, and others.

Thanks to Google for their sponsorship, oster@ for mentoring the
project, assorted developers for trying very hard to break it, and
probably more I'm forgetting.

sync with HEAD

Sync with HEAD.

sync with head.

In a land before time, when kernel processes roamed the system, we
needed to keep track of the kernel process that opened a device in
order to close it with the right credentials.  Flash forward to today
where curlwp is now quite sufficient.

sync with HEAD

Sync with HEAD.

sync with head

Sync with HEAD. amd64 Xen support needs testing.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start.  In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

sync with head.

Sync w/ NetBSD-4-RC_1

Sync with HEAD.

Sync with HEAD.

Fix fallout from recent kthread changes.

file rf_copyback.c was added on branch matt-mips64 on 2007-07-18 19:04:59 +0000

Fix fallout from recent kthread changes.

Sync with head.

Sync with head.

Pull up following revision(s) (requested by cube in ticket #748):
	sys/dev/dksubr.c: revision 1.29
	sys/dev/ccd.c: revision 1.120
	sys/dev/raidframe/rf_disks.c: revision 1.66
	sys/dev/raidframe/rf_reconstruct.c: revision 1.96
	sys/dev/cgd.c: revision 1.45
	sys/dev/dkvar.h: revision 1.11
	sys/dev/raidframe/rf_copyback.c: revision 1.38
Change dk_lookup() to accept an additional argument of the type enum uio_seg
that tells whether the given path is in user space or kernel space, so it
can tell NDINIT().
While the raidframe calls were ok, both ccd(4) and cgd(4) were passing
pointers to user space data, which leads to strange error on i386, as
reported by Jukka Salmi on current-users.

Change dk_lookup() to accept an additional argument of the type enum uio_seg
that tells whether the given path is in user space or kernel space, so it
can tell NDINIT().

While the raidframe calls were ok, both ccd(4) and cgd(4) were passing
pointers to user space data, which leads to strange error on i386, as
reported by Jukka Salmi on current-users.

The issue has been there since last august, I'm actually a bit surprised
that no one in the meantime has used ccd(4) or cgd(4) on an arch where it
would have simply faulted.

sync with head.

sync with head.

Sync with head.

__unused removal on arguments; approved by core.

sync with head

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

sync with head

sync with head.

- use dk_lookup instead of our home-spun version.
- allow raid to be configured in a wedge
- allow wedges to be configured in a raid
- add autoconfiguration of wedges in a raid

sync with head

- Use the LWP cached credentials where sane.
- Minor cosmetic changes.

sync with head.

Sync with head.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

integrate kauth.

Adapt to kernel authorization KPI.

merge ktrace-lwp.

sync with -current

sync with head.  xen and whitespace.  xen part is not finished.

Sync with HEAD.

Hi Perry!

nuke trailing whitespace

Sync with HEAD.

sync with head.

The 'next' argument to rf_CreateDiskQueueData is always NULL.  Since
there is no particular reason to pass an extra NULL argument, turf it,
and initialize p->next to NULL within the function.

Add a 'waitflag' argument to rf_CreateDiskQueueData() and use it to
determine if we are willing to wait for memory to come from the
diskqueuedata (dqd) and bufpool pools.  Cleanup the mess related to
code calling rf_CreateDiskQueueData() with different expectations
(and/or blatent disregard) of what might happen if there were
insufficient pool resources.

Fix the sync with head I botched.

Sync with HEAD.

Sync with HEAD

Fix mcpair locking issues -- we really don't need the lock held
the entire time we spend in calling rf_DiskIOEnqueue().

Some days you wonder if some of the function declaration consistency
was just an accident in the first place.  Cleanup function decls and
a few comments.  [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]

[Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc.  All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K.  For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.

Apply the aborted ktrace-lwp changes to a specific branch.  This is just for
others to review, I'm concerned that patch fuziness may have resulted in some
errant code being generated but I'll look at that later by comparing the diff
from the base to the branch with the file I attempt to apply to it.  This will,
at the very least, put the changes in a better context for others to review
them and attempt to tinker with removing passing of 'struct lwp' through
the kernel.

Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records.  The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

Use 'void *' instead of 'caddr_t' in prototypes of VOP_IOCTL, VOP_FCNTL
and VOP_ADVLOCK, delete casts from callers (and some to copyin/out).

Sync with HEAD.

Clean up error reporting, and nuke some printf verbosity.

After a rebuild-in-place, a reconstruct, or a copyback, we should
really be updating the component labels.

Catch up to -current.

sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work

Cast the RF_DEBUG_RECON net a little wider.

Sync kqueue branch with -current.

Catch up to -current.

don't need <sys/types.h> when including <sys/param.h>

Catch up to -current.

add RCSIDs

Catch up to -current.

Catch up with -current. Fix some bogons in the sparc64 kbd/ms
attach code. cd18xx conversion provided by mrg.

More #if 0's bite the dust.

Step 2 of the disentanglement.  We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.

Commit my "devvp" changes to the thorpej-devvp branch.  This
replaces the use of dev_t in most places with a struct vnode *.

This will form the basic infrastructure for real cloning device
support (besides being architecurally cleaner -- it'll be good
to get away from using numbers to represent objects).

Sync with HEAD.

Pull up revision 1.15 (requested by oster):
  Make sure we update the ``partitionSize'' field of the component
  labels when doing a reconstruct or copyback, instead of leaving
  behind possibly uninitialized junk, which could cause autoconfig
  failure on reboot.

Ensure we update the 'partitionSize' field of the component labels
when doing a reconstruct or a copyback.  If we don't, junk might be
there, and that could cause the component to be not correctly
autoconfigured on reboot.  Thanks to Simon Burge for helping track this down.

Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.

Create a new rf_close_component() to handle vnode operations for closing
components.  Teach rf_UnconfigureVnodes() how to use it, and tell
the copyback and reconstruction code about it too.

Do a better job of (re)initializing the component labels after
a reconstruct or a copyback.

- move a bunch of function prototypes to rf_kintf.h
- general cleanup of a number of prototypes that were scattered around.

Nuke: #include "rf_threadid.h".

- update RF_CREATE_THREAD to handle a 'process name' argument.
- fire up a new thread for parity re-writes, copybacks, and reconstructs.
  The ioctl's which trigger these actions now return immediately.
- add progress accounting for the above actions.
- minor rototillage of rf_netbsdkintf.c to deal with all of the above.

Remove a 'struct proc *'-passing abomination that's been bugging me
for quite some time.

rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.

Update for recent changes including component label support, clean
bits, rebuilding components in-place, adding hot spares, shutdownhooks, etc.

After a vn_close(), set the vnode * to NULL so we don't try closing it
again later.

Do the VOP_UNLOCK() on an initialized vp, rather than random garbage.

Do a VOP_UNLOCK() before a vn_close().

Phase 2 of the RAIDframe cleanup.  The source is now closer to KNF
and is much easier to read.  No functionality changes.

RAIDframe cleanup, phase 1.  Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff.  This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.

RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University.  Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.