CVS log for src/sys/netinet/tcp_subr.c

Up to [cvs.NetBSD.org] / src / sys / netinet

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

sys: Drop redundant NULL check before m_freem(9)

m_freem(9) safely has accepted NULL argument at least since 4.2BSD:
https://www.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/sys/uipc_mbuf.c

Compile-tested on amd64/ALL.

Suggested by knakahara@

inpcb: rename functions to in6pcb_*

inpcb: rename functions to inpcb_*

Inspired by rmind-smpnet patches.

tcp: fix wrong logic in tcp_drop

Pointed out by mlelstv@

inpcb: separate inpcb again to reduce the size of PCB for IPv4

The data size of PCB for IPv4 increased because of the merge of
struct in6pcb.  The change decreases the size to the original size by
separating struct inpcb (again).  struct in4pcb and in6pcb that embed
struct inpcb are introduced.

Even after the separation, users don't need to realize the separation
and only have to use some macros to access dedicated data.  For example,
inp->inp_laddr is now accessed through in4p_laddr(inp).

inpcb: integrate data structures of PCB into one

Data structures of network protocol control blocks (PCBs), i.e.,
struct inpcb, in6pcb and inpcb_hdr, are not organized well.  Users of
the data structures have to handle them separately and thus the code
is cluttered and duplicated.

The commit integrates the data structures into one, struct inpcb.  As a
result, users of PCBs only have to handle just one data structure, so
the code becomes simple.

One drawback is that the data size of PCB for IPv4 increases by 40 bytes
(from 248 bytes to 288 bytes).

tcp: separate syn cache stuffs into tcp_syncache.[ch] files

No functional change.

Remove confusable comment.

The comment was added in tcp_subr.c:r1.124 (2002/03/15).
tcp_drain() is called from softint context only, now.

Sync with HEAD.

s/threshhold/threshold

Sync with HEAD.

Pull up following revision(s) (requested by christos in ticket #1662):

	sys/netinet/tcp_subr.c: revision 1.286
	sys/netinet/tcp_timer.c: revision 1.96
	sys/netinet/in_var.h: revision 1.102
	sys/netinet/in_var.h: revision 1.99

Don't increment the iss sequence on each connection because it exposes
information (Amit Klein)

Add some randomness to the iss offset

Use a random IPv4 ID because the shuffling algorithm used before could expose
information (Amit Klein)

mv <sys/cprng.h> include to the kernel portion

Pull up following revision(s) (requested by christos in ticket #1229):

	sys/netinet/tcp_subr.c: revision 1.286
	sys/netinet/tcp_timer.c: revision 1.96
	sys/netinet/in_var.h: revision 1.102
	sys/netinet/in_var.h: revision 1.99

Don't increment the iss sequence on each connection because it exposes
information (Amit Klein)

Add some randomness to the iss offset

Use a random IPv4 ID because the shuffling algorithm used before could expose
information (Amit Klein)

mv <sys/cprng.h> include to the kernel portion

Move the offset addition in one place and mask the random generated value
to make sure that the isn is monotonic.

Remove the unused "addin" argument (it was always 0) and go back using
a random iss by default (instead of rfc1948)

Don't increment the iss sequence on each connection because it exposes
information (Amit Klein)

Pull up following revision(s) (requested by christos in ticket #1661):

	sys/netinet6/ip6_id.c: revision 1.19-1.21
	sys/netinet6/ip6_var.h: revision 1.88
	sys/netinet/ip_input.c: revision 1.400
	sys/netinet/tcp_subr.c: revision 1.285
	sys/netinet/ip6.h: revision 1.30

netinet: Enable random IP fragment ids by default (from riastradh)

netinet: Enable RFC 1948 pseudorandom TCP ISS selection by default.
(from riastradh)

netinet6: Mark randomid unused.

Will make merging and bisection easier if anything goes wrong with
flow label or fragment id randomization changes.
(from riastradh)

netinet/netinet6: Add necessary includes to make these standalone.
(from riastradh)

Replace randomid() by cprng_fast32()

Pull up following revision(s) (requested by christos in ticket #1226):

	sys/netinet6/ip6_id.c: revision 1.19-1.21
	sys/netinet6/ip6_var.h: revision 1.88
	sys/netinet/ip_input.c: revision 1.400
	sys/netinet/tcp_subr.c: revision 1.285
	sys/netinet/ip6.h: revision 1.30

netinet: Enable random IP fragment ids by default (from riastradh)

netinet: Enable RFC 1948 pseudorandom TCP ISS selection by default.
(from riastradh)

netinet6: Mark randomid unused.

Will make merging and bisection easier if anything goes wrong with
flow label or fragment id randomization changes.
(from riastradh)

netinet/netinet6: Add necessary includes to make these standalone.
(from riastradh)

Replace randomid() by cprng_fast32()

netinet: Enable RFC 1948 pseudorandom TCP ISS selection by default.
(from riastradh)

Remove in-kernel handling of Router Advertisements

This is much better handled by a user-land tool.
Proposed on tech-net here:
https://mail-index.netbsd.org/tech-net/2020/04/22/msg007766.html

Note that the ioctl SIOCGIFINFO_IN6 no longer sets flags. That now
needs to be done using the pre-existing SIOCSIFINFO_FLAGS ioctl.

Compat is fully provided where it makes sense, but trying to turn on
RA handling will obviously throw an error as it no longer exists.

Note that if you use IPv6 temporary addresses, this now needs to be
turned on in dhcpcd.conf(5) rather than in sysctl.conf(5).

Mostly merge changes from HEAD upto 20200411

Pull up following revision(s) (requested by maxv in ticket #193):

	sys/netinet/tcp_timer.h: revision 1.30
	sys/netinet/tcp_input.c: revision 1.415
	sys/netinet/tcp_usrreq.c: revision 1.225
	sys/netinet/tcp_subr.c: revision 1.283

Clamp tcp timer quantities to reasonable ranges.

Clamp tcp timer quantities to reasonable ranges.

Reported-by: syzbot+259675123340bf46a6de@syzkaller.appspotmail.com

Sync with HEAD

Synch with HEAD

Remove unused arguments.

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int.  The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER!  Some subsystems have

	#define min(a, b)	((a) < (b) ? (a) : (b))
	#define max(a, b)	((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX.  Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate.  But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all.  (Who knows, maybe in some cases integer
truncation is actually intended!)

Sync with HEAD

Add XXX.

Sync with HEAD

Remove now unused tcpip.h includes. Some were already unused before.

Sync with HEAD

Remove unused netipsec/xform.h includes.

Remove misleading comments.

Resolve conflicts between branch and HEAD

Remove TCPREASS_DEBUG. It was introduced 20 years ago when the reassembler
was being developed, but it's irrelevant today. Makes the code clearer.

Misc changes; no real functional change.

Remove #ifdef INET. Same as tcp_input.c. Makes the code easier to
understand.

Also make tcp6_mtudisc() static in tcp_subr.c.

Dedup: merge ipsec4_hdrsiz and ipsec6_hdrsiz into ipsec_hdrsiz.

ok ozaki-r@

Pull up following revision(s) (requested by ozaki-r in ticket #514):
	sys/net/route.c: 1.205
	sys/net/rtsock.c: 1.237-1.238
	sys/netinet/in.c: 1.215
	sys/netinet/tcp_subr.c: 1.272
	sys/netinet/tcp_timer.c: 1.93
	sys/netinet/tcp_timer.h: 1.29
	sys/netinet/tcp_var.h: 1.182
	sys/netinet6/in6.c: 1.258
Remove extra pserialize_perform from in_purgeaddr
It's already performed in ifa_remove. Note so there (in in6_unlink_ifa too).
Release rt_so_mtx on updating a rtentry to avoid a deadlock with route_intr
The deadlock happened only if NET_MPSAFE on.
Run tcp_slowtimo in workqueue if NET_MPSAFE
If NET_MPSAFE is enabled, we have to avoid taking softnet_lock in softint as
much as possible to prevent any softint handlers including callout handlers
such as tcp_slowtimo from sticking on softnet_lock because it results in
undesired delays of executing subsequent softint handlers.
NFCI for !NET_MPSAFE
Fix a return value of rt_update_prepare
Callers expect it to be an errno.
Fix another deadlock
When waiting for a route update to finish, a waiter has to release its reference
to the route to avoid a deadlock. Because a updater tries to wait for references
to a target route (except for a reference by the updater itself) to be released.

Run tcp_slowtimo in workqueue if NET_MPSAFE

If NET_MPSAFE is enabled, we have to avoid taking softnet_lock in softint as
much as possible to prevent any softint handlers including callout handlers
such as tcp_slowtimo from sticking on softnet_lock because it results in
undesired delays of executing subsequent softint handlers.

NFCI for !NET_MPSAFE

update from HEAD

Sync with HEAD

Forgot to commit this file yesterday.

Sync with HEAD

Sync with HEAD

Pass inpcb/in6pcb instead of socket to ip_output/ip6_output

- Passing a socket to Layer 3 is layer violation and even unnecessary
- The change makes codes of callers and IPsec a bit simple

Sync with HEAD

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

Fix TCP signature code:
1. pack options more tightly instead of being generous with no/op
2. put TCP_SIGNATURE option before SACK
3. fix computation of options length, by deferring it
XXX: Really we should move the options setting code in one place instead
of having two copies one for input and one for output.
XXX: tcp_optlen/tcp_hdrsiz need to be fixed; they were wrong before too.

Add rtcache_unref to release points of rtentry stemming from rtcache

In the MP-safe world, a rtentry stemming from a rtcache can be freed at any
points. So we need to protect rtentries somehow say by reference couting or
passive references. Regardless of the method, we need to call some release
function of a rtentry after using it.

The change adds a new function rtcache_unref to release a rtentry. At this
point, this function does nothing because for now we don't add a reference
to a rtentry when we get one from a rtcache. We will add something useful
in a further commit.

This change is a part of changes for MP-safe routing table. It is separated
to avoid one big change that makes difficult to debug by bisecting.

Sync with HEAD

Cleanup/KNF tcp6_mtudisc

No functional change.

Sync with HEAD

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

Sync with HEAD

Reduce code duplication.

Split creation of IPv4-Mapped IPv6 addresses into its own function
and use it.

No functional change intended.  As posted to tech-net@

Sync with HEAD

Refactor tcp_mtudisc

No functional change.

sprinkle _KERNEL_OPT

Sync with HEAD

Use RUN_ONCE to initialize iss secret. Suggested by riastradh@

Don't overexpose tcp_iss_secret and don't bother compute it unless
RFC1948 compliance is activated

Introduce in6_selecthlim_rt to consolidate an idiom for rt->rt_ifp

It consolidates a scattered routine:
(rt = rtcache_validate(&in6p->in6p_route)) != NULL ? rt->rt_ifp : NULL

cprng_strong(kern_cprng, ...) never blocks, pass 0 for flags.

FASYNC was wrong anyway!  It's FNONBLOCK.

Sync with HEAD

Pull up following revision(s) (requested by he in ticket #530):
	sys/netinet/tcp_output.c: revision 1.180
	sys/netinet/tcp_input.c: revision 1.336
	sys/netinet/tcp_usrreq.c: revision 1.203
	share/man/man4/tcp.4: revision 1.30
	sys/netinet/tcp.h: revision 1.31
	sys/netinet/tcp_subr.c: revision 1.258
	sys/netinet/tcp_var.h: revision 1.176
	sys/netinet/tcp_var.h: revision 1.177
	sys/sys/param.h: bump revision

Port over the TCP_INFO socket option from FreeBSD, originally from
the Linux 2.6 TCP API.  This permits the caller to query certain information
about a TCP connection, and is used by pkgsrc's net/iperf3 test program
if available.

This extends struct tcbcb with three fields to count retransmits,
out-of-sequence receives and zero window announcements, and will
therefore warrant a kernel revision bump (done separately).

Change the new counter variables in struct tcpcb to uint32_t, as
per christos' comments.

Port over the TCP_INFO socket option from FreeBSD, originally from
the Linux 2.6 TCP API.  This permits the caller to query certain information
about a TCP connection, and is used by pkgsrc's net/iperf3 test program
if available.

This extends struct tcbcb with three fields to count retransmits,
out-of-sequence receives and zero window announcements, and will
therefore warrant a kernel revision bump (done separately).

Pull up following revision(s) (requested by maxv in ticket #427):
	sys/compat/svr4/svr4_schedctl.c: revision 1.8
	sys/netinet/tcp_timer.c: revision 1.88
	sys/miscfs/genfs/layer_vfsops.c: revision 1.45
	sys/compat/svr4/svr4_ioctl.c: revision 1.37
	sys/ufs/chfs/chfs_vfsops.c: revision 1.14
	sys/miscfs/fdesc/fdesc_vfsops.c: revision 1.91
	sys/compat/linux/arch/i386/linux_ptrace.c: revision 1.30
	sys/compat/common/kern_time_50.c: revision 1.28
	sys/netinet6/ip6_forward.c: revision 1.74
	sys/miscfs/umapfs/umap_vnops.c: revision 1.57
	sys/compat/svr4/svr4_fcntl.c: revision 1.74
	distrib/sets/lists/comp/mi: revision 1.1931
	sys/netinet6/udp6_output.c: revision 1.46
	sys/fs/puffs/puffs_compat.c: revision 1.3
	sys/fs/udf/udf_rename.c: revision 1.11
	sys/compat/svr4/svr4_filio.c: revision 1.24
	sys/fs/udf/udf_rename.c: revision 1.12
	sys/netinet/tcp_usrreq.c: revision 1.202
	sys/miscfs/umapfs/umap_subr.c: revision 1.29
	sys/compat/linux/common/linux_fadvise64.c: revision 1.3
	sys/netinet/if_atm.c: revision 1.34
	sys/miscfs/procfs/procfs_subr.c: revision 1.106
	sys/miscfs/genfs/layer_subr.c: revision 1.37
	sys/netinet/tcp_sack.c: revision 1.30
	sys/compat/freebsd/freebsd_misc.c: revision 1.33
	sys/compat/freebsd/freebsd_file.c: revision 1.33
	sys/ufs/chfs/chfs_vnode.c: revision 1.12
	sys/compat/svr4/svr4_ttold.c: revision 1.34
	sys/compat/linux/common/linux_file.c: revision 1.114
	sys/compat/linux/arch/mips/linux_machdep.c: revision 1.43
	sys/compat/linux/common/linux_signal.c: revision 1.76
	sys/compat/common/compat_util.c: revision 1.46
	sys/compat/linux/arch/arm/linux_ptrace.c: revision 1.18
	sys/compat/svr4/svr4_sockio.c: revision 1.36
	sys/compat/linux/arch/arm/linux_machdep.c: revision 1.32
	sys/compat/svr4/svr4_signal.c: revision 1.66
	sys/kern/kern_exec.c: revision 1.410
	sys/fs/puffs/puffs_vfsops.c: revision 1.115
	sys/compat/svr4/svr4_exec_elf64.c: revision 1.15
	sys/compat/linux/arch/i386/linux_machdep.c: revision 1.159
	sys/compat/linux/arch/alpha/linux_machdep.c: revision 1.50
	sys/compat/linux32/common/linux32_misc.c: revision 1.24
	sys/netinet/in_pcb.c: revision 1.153
	sys/sys/malloc.h: revision 1.116
	sys/compat/common/if_43.c: revision 1.9
	share/man/man9/Makefile: revision 1.380
	sys/netinet/tcp_vtw.c: revision 1.12
	sys/miscfs/umapfs/umap_vfsops.c: revision 1.95
	sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.186
	sys/compat/common/uipc_syscalls_43.c: revision 1.46
	sys/ufs/ext2fs/ext2fs_vnops.c: revision 1.115
	sys/fs/puffs/puffs_msgif.c: revision 1.97
	sys/compat/svr4/svr4_ipc.c: revision 1.27
	sys/compat/linux/common/linux_exec.c: revision 1.117
	sys/ufs/ext2fs/ext2fs_readwrite.c: revision 1.66
	sys/netinet/tcp_output.c: revision 1.179
	sys/compat/svr4/svr4_termios.c: revision 1.28
	sys/fs/udf/udf_strat_bootstrap.c: revision 1.4
	sys/fs/puffs/puffs_subr.c: revision 1.67
	sys/fs/puffs/puffs_node.c: revision 1.36
	sys/miscfs/overlay/overlay_vnops.c: revision 1.21
	sys/fs/cd9660/cd9660_node.c: revision 1.34
	sys/netinet/raw_ip.c: revision 1.146
	sys/sys/mallocvar.h: revision 1.13
	sys/miscfs/overlay/overlay_vfsops.c: revision 1.63
	share/man/man9/malloc.9: revision 1.50
	sys/netinet6/dest6.c: revision 1.18
	sys/compat/linux/common/linux_uselib.c: revision 1.33
	sys/compat/linux/common/linux_socket.c: revision 1.120
	share/man/man9/malloc.9: revision 1.51
	sys/netinet/tcp_subr.c: revision 1.257
	sys/compat/linux/common/linux_socketcall.c: revision 1.45
	sys/compat/linux/common/linux_fadvise64_64.c: revision 1.3
	sys/compat/freebsd/freebsd_ipc.c: revision 1.17
	sys/compat/linux/common/linux_misc_notalpha.c: revision 1.109
	sys/compat/linux/arch/alpha/linux_pipe.c: revision 1.17
	sys/netinet6/in6_pcb.c: revision 1.132
	sys/netinet6/in6_ifattach.c: revision 1.94
	sys/compat/svr4/svr4_exec_elf32.c: revision 1.15
	sys/miscfs/nullfs/null_vfsops.c: revision 1.90
	sys/fs/cd9660/cd9660_util.c: revision 1.12
	sys/compat/linux/arch/powerpc/linux_machdep.c: revision 1.48
	sys/compat/freebsd/freebsd_exec_elf32.c: revision 1.20
	sys/miscfs/procfs/procfs_vfsops.c: revision 1.94
	sys/compat/linux/arch/powerpc/linux_ptrace.c: revision 1.28
	sys/compat/linux/common/linux_sched.c: revision 1.67
	sys/compat/linux/common/linux_exec_aout.c: revision 1.67
	sys/compat/linux/common/linux_pipe.c: revision 1.67
	sys/compat/linux/common/linux_llseek.c: revision 1.34
	sys/compat/linux/arch/mips/linux_ptrace.c: revision 1.10
Do not uselessly include <sys/malloc.h>.
Cleanup:
 - remove struct kmembuckets (dead)
 - correctly deadify MALLOC_XX
 - remove MALLOC_DEFINE_LIMIT and MALLOC_JUSTDEFINE_LIMIT (dead)
 - remove malloc_roundup(), malloc_type_setlimit(), MALLOC_DEFINE_LIMIT()
   and MALLOC_JUSTDEFINE_LIMIT() from man 9 malloc
New sentence, new line. Bump date for previous.
Obsolete malloc_roundup(9), malloc_type_setlimit(9) and MALLOC_DEFINE_LIMIT(9)
man pages.

Do not uselessly include <sys/malloc.h>.

Don't use C++ keyword (template) as variable.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

need atomic.h, from uwe

Allow kernels compiled with INET+INET6 to be booted as IPv4-only or IPv6-only.

convert from CIRCLEQ to TAILQ.

* implement TCP CUBIC congestion control algorithm
* move tcp_sack_newack bits inside reno and newreno_fast_retransmit_newack
* notify ECN peer about cwnd shrink in [new]reno_slow_retransmit

Based on the patch proposed on tech-net@ on Nov 7 with minor improvments:
 * adapt wmax for no-fast convergence case
 * correct cbrt calculation for big window sizes (>750KB)

- Add some initial locking to the IPv4 PCB.
- Rename inpcb_lookup_*() routines to be more accurate and add comments.
- Add some comments about connection life-cycle WRT socket layer.

Checkpoint work in progress:
- Move PCB structures under __INPCB_PRIVATE, adjust most of the callers
  and thus make IPv4 PCB structures mostly opaque.  Any volunteers for
  merging in6pcb with inpcb (see rpaulo-netinet-merge-pcb branch)?
- Move various global vars to the modules where they belong, make them static.
- Some preliminary work for IPv4 PCB locking scheme.
- Make raw IP code mostly MP-safe.  Simplify some of it.
- Rework "fast" IP forwarding (ipflow) code to be mostly MP-safe.  It should
  run from a software interrupt, rather than hard.
- Rework tun(4) pseudo interface to be MP-safe.
- Work towards making some other interfaces more strict.

resync from head

IPSEC has not come in two speeds for a long time now (IPSEC == kame,
FAST_IPSEC). Make everything refer to IPSEC to avoid confusion.

Limit the tcp initial window setting to 10, leaving it by default to 4
and simplifying the code in process. Per draft-ietf-initcwnd-08.txt.

Pull up following revision(s) (requested by msaitoh in ticket #644):
	sys/netinet/tcp_subr.c: revision 1.248
	sys/kern/subr_cprng.c: revision 1.12
Fix a bug that kmem_alloc() is called from the interrupt context.

sync with head

Fix a bug that kmem_alloc() is called from the interrupt context.

sync with head

sync to latest -current.

remove KAME IPSEC, replaced by FAST_IPSEC

merge to -current.

- fix offsetof usage, and redundant defines
- kill pointer casts to 0

rename the IPSEC in-kernel CPP variable and config(8) option to
KAME_IPSEC, and make IPSEC define it so that existing kernel
config files work as before
Now the default can be easily be changed to FAST_IPSEC just by
setting the IPSEC alias to FAST_IPSEC.

Separate /dev/random pseudodevice implemenation from kernel entropy pool
implementation.  Rewrite pseudodevice code to use cprng_strong(9).

The new pseudodevice is cloning, so each caller gets bits from a stream
generated with its own key.  Users of /dev/urandom get their generators
keyed on a "best effort" basis -- the kernel will rekey generators
whenever the entropy pool hits the high water mark -- while users of
/dev/random get their generators rekeyed every time key-length bits
are output.

The underlying cprng_strong API can use AES-256 or AES-128, but we use
AES-128 because of concerns about related-key attacks on AES-256.  This
improves performance (and reduces entropy pool depletion) significantly
for users of /dev/urandom but does cause users of /dev/random to rekey
twice as often.

Also fixes various bugs (including some missing locking and a reseed-counter
overflow in the CTR_DRBG code) found while testing this.

For long reads, this generator is approximately 20 times as fast as the
old generator (dd with bs=64K yields 53MB/sec on 2Ghz Core2 instead of
2.5MB/sec) and also uses a separate mutex per instance so concurrency
is greatly improved.  For reads of typical key sizes for modern
cryptosystems (16-32 bytes) performance is about the same as the old
code: a little better for 32 bytes, a little worse for 16 bytes.

First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>.  This change includes
the following:

	An initial cleanup and minor reorganization of the entropy pool
	code in sys/dev/rnd.c and sys/dev/rndpool.c.  Several bugs are
	fixed.  Some effort is made to accumulate entropy more quickly at
	boot time.

	A generic interface, "rndsink", is added, for stream generators to
	request that they be re-keyed with good quality entropy from the pool
	as soon as it is available.

	The arc4random()/arc4randbytes() implementation in libkern is
	adjusted to use the rndsink interface for rekeying, which helps
	address the problem of low-quality keys at boot time.

	An implementation of the FIPS 140-2 statistical tests for random
	number generator quality is provided (libkern/rngtest.c).  This
	is based on Greg Rose's implementation from Qualcomm.

	A new random stream generator, nist_ctr_drbg, is provided.  It is
	based on an implementation of the NIST SP800-90 CTR_DRBG by
	Henric Jungheim.  This generator users AES in a modified counter
	mode to generate a backtracking-resistant random stream.

	An abstraction layer, "cprng", is provided for in-kernel consumers
	of randomness.  The arc4random/arc4randbytes API is deprecated for
	in-kernel use.  It is replaced by "cprng_strong".  The current
	cprng_fast implementation wraps the existing arc4random
	implementation.  The current cprng_strong implementation wraps the
	new CTR_DRBG implementation.  Both interfaces are rekeyed from
	the entropy pool automatically at intervals justifiable from best
	current cryptographic practice.

	In some quick tests, cprng_fast() is about the same speed as
	the old arc4randbytes(), and cprng_strong() is about 20% faster
	than rnd_extract_data().  Performance is expected to improve.

	The AES code in src/crypto/rijndael is no longer an optional
	kernel component, as it is required by cprng_strong, which is
	not an optional kernel component.

	The entropy pool output is subjected to the rngtest tests at
	startup time; if it fails, the system will reboot.  There is
	approximately a 3/10000 chance of a false positive from these
	tests.  Entropy pool _input_ from hardware random numbers is
	subjected to the rngtest tests at attach time, as well as the
	FIPS continuous-output test, to detect bad or stuck hardware
	RNGs; if any are detected, they are detached, but the system
	continues to run.

	A problem with rndctl(8) is fixed -- datastructures with
	pointers in arrays are no longer passed to userspace (this
	was not a security problem, but rather a major issue for
	compat32).  A new kernel will require a new rndctl.

	The sysctl kern.arandom() and kern.urandom() nodes are hooked
	up to the new generators, but the /dev/*random pseudodevices
	are not, yet.

	Manual pages for the new kernel interfaces are forthcoming.

tcp_drain: grab softnet_lock where appropriate

Sync with HEAD.

sync with head

Reduces the resources demanded by TCP sessions in TIME_WAIT-state using
methods called Vestigial Time-Wait (VTW) and Maximum Segment Lifetime
Truncation (MSLT).

MSLT and VTW were contributed by Coyote Point Systems, Inc.

Even after a TCP session enters the TIME_WAIT state, its corresponding
socket and protocol control blocks (PCBs) stick around until the TCP
Maximum Segment Lifetime (MSL) expires.  On a host whose workload
necessarily creates and closes down many TCP sockets, the sockets & PCBs
for TCP sessions in TIME_WAIT state amount to many megabytes of dead
weight in RAM.

Maximum Segment Lifetimes Truncation (MSLT) assigns each TCP session to
a class based on the nearness of the peer.  Corresponding to each class
is an MSL, and a session uses the MSL of its class.  The classes are
loopback (local host equals remote host), local (local host and remote
host are on the same link/subnet), and remote (local host and remote
host communicate via one or more gateways).  Classes corresponding to
nearer peers have lower MSLs by default: 2 seconds for loopback, 10
seconds for local, 60 seconds for remote.  Loopback and local sessions
expire more quickly when MSLT is used.

Vestigial Time-Wait (VTW) replaces a TIME_WAIT session's PCB/socket
dead weight with a compact representation of the session, called a
"vestigial PCB".  VTW data structures are designed to be very fast and
memory-efficient: for fast insertion and lookup of vestigial PCBs,
the PCBs are stored in a hash table that is designed to minimize the
number of cacheline visits per lookup/insertion.  The memory both
for vestigial PCBs and for elements of the PCB hashtable come from
fixed-size pools, and linked data structures exploit this to conserve
memory by representing references with a narrow index/offset from the
start of a pool instead of a pointer.  When space for new vestigial PCBs
runs out, VTW makes room by discarding old vestigial PCBs, oldest first.
VTW cooperates with MSLT.

It may help to think of VTW as a "FIN cache" by analogy to the SYN
cache.

A 2.8-GHz Pentium 4 running a test workload that creates TIME_WAIT
sessions as fast as it can is approximately 17% idle when VTW is active
versus 0% idle when VTW is inactive.  It has 103 megabytes more free RAM
when VTW is active (approximately 64k vestigial PCBs are created) than
when it is inactive.

*_drain() routines may be called with locks held, so instead of doing
any work in *_drain(), set a drain-needed flag.  Do the work in the
fasttimo handler.

Contributed by Coyote Point Systems, Inc.

sync with head

Rewrite comments about TCP RTO calculations.

Long ago, the storage representations of srtt and rttvar were changed
from the 4.4BSD scheme, and the comments are out of sync with the
code.  This commit rewrites most of the comments that explain the RTO
calculations, and points out some issues in the code.

Joint work with Bev Schwartz of BBN (original analysis and comments),
but I have rewritten and extended them, so errors are mine.

This material is based upon work supported by the Defense Advanced
Research Projects Agency and Space and Naval Warfare Systems Center,
Pacific, under Contract No. N66001-09-C-2073.  Approved for Public
Release, Distribution Unlimited

sync with head

Replace a large number of link set based sysctl node creations with
calls from subsystem constructors.  Benefits both future kernel
modules and rump.

no change to sysctl nodes on i386/MONOLITHIC & build tested i386/ALL

Sync with HEAD.

sync with head

POOL_INIT -> pool_init

Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.

sync with head.

Sync with HEAD.

bzero -> memset

bcmp -> memcmp

Sync with HEAD.

stinkset purge: POOL_INIT -> pool_init
also, make the syncache pool static in scope

Sync with HEAD.

Sync with HEAD.

POOL_INIT -> pool_init

Sync with HEAD.

tcp_close: rearrange sequence of events slightly to make this atomic.
It was possible for a half-destroyed tcpcb to be visble, as softnet_lock
was being dropped.

Sync with netbsd-4.

Sync with HEAD.

sync with head.

sync with head.

PR kern/38497 Out of memory allocating ksiginfo

Work around: don't acquire softnet_lock in protocol drain routines.

Remove clause 3 and 4 from TNF licenses

tcp_init: don't forget to allocate tcpstat_percpu.

Merge the socket locking patch:

- Socket layer becomes MP safe.
- Unix protocols become MP safe.
- Allows protocol processing interrupts to safely block on locks.
- Fixes a number of race conditions.

With much feedback from matt@ and plunky@.

Make IP, TCP, UDP, and ICMP statistics per-CPU.  The stats are collated
when the user requests them via sysctl.

Change TCP stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old tcpstat structure; old netstat
binaries will continue to work properly.

Sync with HEAD.

Pull up revisions:
  src/sys/netinet/ip_input.c    1.263
  src/sys/netinet/tcp_subr.c    1.225
(requested by cube in ticket #1109).

- Make sure we send a reasonable fragment size when IPSEC is configured.
  Otherwise we end up sending a dubious "0" whenever we cannot find a
  proper association for the packet.
- Reset sack_newdata along with snd_nxt to avoid improper integer
  arithmetics that lead to sending data from an incorrect place in the
  stream, making it appear as corrupted.

Patch by Michael Van Elst, based on an analysis by Michael for the IPSEC
stuff and I for the SACK issue.

Pull up revisions:
  src/sys/netinet/ip_input.c    1.263
  src/sys/netinet/tcp_subr.c    1.225
(requested by cube in ticket #1109).

- Make sure we send a reasonable fragment size when IPSEC is configured.
  Otherwise we end up sending a dubious "0" whenever we cannot find a
  proper association for the packet.
- Reset sack_newdata along with snd_nxt to avoid improper integer
  arithmetics that lead to sending data from an incorrect place in the
  stream, making it appear as corrupted.

Patch by Michael Van Elst, based on an analysis by Michael for the IPSEC
stuff and I for the SACK issue.

- Make sure we send a reasonable fragment size when IPSEC is configured.
  Otherwise we end up sending a dubious "0" whenever we cannot find a
  proper association for the packet.
- Reset sack_newdata along with snd_nxt to avoid improper integer
  arithmetics that lead to sending data from an incorrect place in the
  stream, making it appear as corrupted.

Patch by Michael Van Elst, based on an analysis by Michael for the IPSEC
stuff and I for the SACK issue.

sync with head.

sync with HEAD

sync with head.

Rework tcp congctl selection code so that the congctl entries can be const.
Don't access tcp_congctl stuff outside of tcp_congctl.c, use routines to
update t_congctl.  This code is slightly now more complicated.

Convert stragglers to ansi definitions from old-style definitons.
Remember that func() is not ansi, func(void) is.

Sync with HEAD.

sync with head.

- start tcp timestamp from 1 instead of 0.
- add a comment to explain why:
+        * We start with 1, because 0 doesn't work with linux, which
+        * considers timestamp 0 in a SYN packet as a bug and disables
+        * timestamps.

sync with head

Sync with HEAD

Use rtcache_validate() instead of rtcache_getrt().  Shorten staircase
in in_losing().

sync with HEAD

Sync with HEAD

Sync with head.

A few missing ifdefs to make non-INET6 kernels build again.

Poison struct route->ro_rt uses in the kernel by changing the name
to _ro_rt.  Use rtcache_getrt() to access a route cache's struct
rtentry *.

Introduce struct ifnet->if_dl that always points at the interface
identifier/link-layer address.  Make code that treated the first
ifaddr on struct ifnet->if_addrlist as the interface address use
if_dl, instead.

Remove stale debugging code from net/route.c.  Move the rtflush()
code into rtcache_clear() and delete rtflush().  Delete rtalloc(),
because nothing uses it any more.

Make ND6_HINT an inline, lowercase subroutine, nd6_hint.

I've done my best to convert IP Filter, the ISO stack, and the
AppleTalk stack to rtcache_getrt().  They compile, but I have not
tested them.  I have given the changes to PF, GRE, IPv4 and IPv6
stacks a lot of exercise.

sync with head.

Sync with HEAD.

Sync with HEAD.

TCP socket buffers automatic sizing - ported from FreeBSD.
http://mail-index.netbsd.org/tech-net/2007/02/04/0006.html

! Disabled by default, marked as experimental. Testers are very needed.
! Someone should thoroughly test this, and improve if possible.

Discussed on <tech-net>:
http://mail-index.netbsd.org/tech-net/2007/07/12/0002.html
Thanks Greg Troxel for comments.

OK by the long silence on <tech-net>.

file tcp_subr.c was added on branch matt-mips64 on 2007-08-02 02:42:42 +0000

TCP socket buffers automatic sizing - ported from FreeBSD.
http://mail-index.netbsd.org/tech-net/2007/02/04/0006.html

! Disabled by default, marked as experimental. Testers are very needed.
! Someone should thoroughly test this, and improve if possible.

Discussed on <tech-net>:
http://mail-index.netbsd.org/tech-net/2007/07/12/0002.html
Thanks Greg Troxel for comments.

OK by the long silence on <tech-net>.

Take steps to hide the radix_node implementation of the forwarding table
from the forwarding table's users:

        Introduce rt_walktree() for walking the routing table and
        applying a function to each rtentry.  Replace most
        rn_walktree() calls with it.

        Use rt_getkey()/rt_setkey() to get/set a route's destination.
        Keep a pointer to the sockaddr key in the rtentry, so that
        rtentry users do not have to grovel in the radix_node for
        the key.

        Add a RTM_GET method to rtrequest.  Use that instead of
        radix_node lookups in, e.g., carp(4).

Add sys/net/link_proto.c, which supplies sockaddr routines for
link-layer socket addresses (sockaddr_dl).

Cosmetic:

        Constify.  KNF.  Stop open-coding LIST_FOREACH, TAILQ_FOREACH,
        et cetera.  Use NULL instead of 0 for null pointers.  Use
        __arraycount().  Reduce gratuitous parenthesization.

        Stop using variadic arguments for rip6_output(), it is
        unnecessary.

        Remove the unnecessary rtentry member rt_genmask and the
        code to maintain it, since nothing actually used it.

        Make rt_maskedcopy() easier to read by using meaningful variable
        names.

        Extract a subroutine intern_netmask() for looking up a netmask in
        the masks table.

        Start converting backslash-ridden IPv6 macros in
        sys/netinet6/in6_var.h into inline subroutines that one
        can read without special eyeglasses.

One functional change: when the kernel serves an RTM_GET, RTM_LOCK,
or RTM_CHANGE request, it applies the netmask (if supplied) to a
destination before searching for it in the forwarding table.

I have changed sys/netinet/ip_carp.c, carp_setroute(), to remove
the unlawful radix_node knowledge.

Apart from the changes to carp(4), netiso, ATM, and strip(4), I
have run the changes on three nodes in my wireless routing testbed,
which involves IPv4 + IPv6 dynamic routing acrobatics, and it's
working beautifully so far.

Sync with head.

Sync with head.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Adapt to callout API change.

- per socket keepalive settings
- settable connection establishment timeout

Sync with head.

Update to today's netbsd-4.

Pull up following revision(s) (requested by degroote in ticket #667):
	sys/netinet/tcp_input.c: revision 1.260
	sys/netinet/tcp_output.c: revision 1.154
	sys/netinet/tcp_subr.c: revision 1.210
	sys/netinet6/icmp6.c: revision 1.129
	sys/netinet6/in6_proto.c: revision 1.70
	sys/netinet6/ip6_forward.c: revision 1.54
	sys/netinet6/ip6_input.c: revision 1.94
	sys/netinet6/ip6_output.c: revision 1.114
	sys/netinet6/raw_ip6.c: revision 1.81
	sys/netipsec/ipcomp_var.h: revision 1.4
	sys/netipsec/ipsec.c: revision 1.26 via patch,1.31-1.32
	sys/netipsec/ipsec6.h: revision 1.5
	sys/netipsec/ipsec_input.c: revision 1.14
	sys/netipsec/ipsec_netbsd.c: revision 1.18,1.26
	sys/netipsec/ipsec_output.c: revision 1.21 via patch
	sys/netipsec/key.c: revision 1.33,1.44
	sys/netipsec/xform_ipcomp.c: revision 1.9
	sys/netipsec/xform_ipip.c: revision 1.15
	sys/opencrypto/deflate.c: revision 1.8
Commit my SoC work
Add ipv6 support for fast_ipsec
Note that currently, packet with extensions headers are not correctly
supported
Change the ipcomp logic

Add sysctl tree to modify the fast_ipsec options related to ipv6. Similar
to the sysctl kame interface.

Choose the good default policy, depending of the adress family of the
desired policy

Increase the refcount for the default ipv6 policy so nobody can reclaim it

Always compute the sp index even if we don't have any sp in spd. It will
let us to choose the right default policy (based on the adress family
requested).
While here, fix an error message

Use dynamic array instead of an static array to decompress. It lets us to
decompress any data, whatever is the radio decompressed data / compressed
data.
It fixes the last issues with fast_ipsec and ipcomp.
While here, bzero -> memset, bcopy -> memcpy, FREE -> free
Reviewed a long time ago by sam@

sync with head.

Eliminate address family-specific route caches (struct route, struct
route_in6, struct route_iso), replacing all caches with a struct
route.

The principle benefit of this change is that all of the protocol
families can benefit from route cache-invalidation, which is
necessary for correct routing.  Route-cache invalidation fixes an
ancient PR, kern/3508, at long last; it fixes various other PRs,
also.

Discussions with and ideas from Joerg Sonnenberger influenced this
work tremendously.  Of course, all design oversights and bugs are
mine.

DETAILS

1 I added to each address family a pool of sockaddrs.  I have
  introduced routines for allocating, copying, and duplicating,
  and freeing sockaddrs:

        struct sockaddr *sockaddr_alloc(sa_family_t af, int flags);
        struct sockaddr *sockaddr_copy(struct sockaddr *dst,
                                       const struct sockaddr *src);
        struct sockaddr *sockaddr_dup(const struct sockaddr *src, int flags);
        void sockaddr_free(struct sockaddr *sa);

  sockaddr_alloc() returns either a sockaddr from the pool belonging
  to the specified family, or NULL if the pool is exhausted.  The
  returned sockaddr has the right size for that family; sa_family
  and sa_len fields are initialized to the family and sockaddr
  length---e.g., sa_family = AF_INET and sa_len = sizeof(struct
  sockaddr_in).  sockaddr_free() puts the given sockaddr back into
  its family's pool.

  sockaddr_dup() and sockaddr_copy() work analogously to strdup()
  and strcpy(), respectively.  sockaddr_copy() KASSERTs that the
  family of the destination and source sockaddrs are alike.

  The 'flags' argumet for sockaddr_alloc() and sockaddr_dup() is
  passed directly to pool_get(9).

2 I added routines for initializing sockaddrs in each address
  family, sockaddr_in_init(), sockaddr_in6_init(), sockaddr_iso_init(),
  etc.  They are fairly self-explanatory.

3 structs route_in6 and route_iso are no more.  All protocol families
  use struct route.  I have changed the route cache, 'struct route',
  so that it does not contain storage space for a sockaddr.  Instead,
  struct route points to a sockaddr coming from the pool the sockaddr
  belongs to.  I added a new method to struct route, rtcache_setdst(),
  for setting the cache destination:

        int rtcache_setdst(struct route *, const struct sockaddr *);

  rtcache_setdst() returns 0 on success, or ENOMEM if no memory is
  available to create the sockaddr storage.

  It is now possible for rtcache_getdst() to return NULL if, say,
  rtcache_setdst() failed.  I check the return value for NULL
  everywhere in the kernel.

4 Each routing domain (struct domain) has a list of live route
  caches, dom_rtcache.  rtflushall(sa_family_t af) looks up the
  domain indicated by 'af', walks the domain's list of route caches
  and invalidates each one.

sync with head.

Sync with head.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

Sync with HEAD.

Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

- sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.

sync with head.

KNF: de-__P, bzero -> memset, bcmp -> memcmp.  Remove extraneous
   parentheses in return statements.

Cosmetic: don't open-code TAILQ_FOREACH().

Cosmetic: change types of variables to avoid oodles of casts: in
   in6_src.c, avoid casts by changing several route_in6 pointers
   to struct route pointers.  Remove unnecessary casts to caddr_t
   elsewhere.

Pave the way for eliminating address family-specific route caches:
   soon, struct route will not embed a sockaddr, but it will hold
   a reference to an external sockaddr, instead.  We will set the
   destination sockaddr using rtcache_setdst().  (I created a stub
   for it, but it isn't used anywhere, yet.)  rtcache_free() will
   free the sockaddr.  I have extracted from rtcache_free() a helper
   subroutine, rtcache_clear().  rtcache_clear() will "forget" a
   cached route, but it will not forget the destination by releasing
   the sockaddr.  I use rtcache_clear() instead of rtcache_free()
   in rtcache_update(), because rtcache_update() is not supposed
   to forget the destination.

Constify:

   1 Introduce const accessor for route->ro_dst, rtcache_getdst().

   2 Constify the 'dst' argument to ifnet->if_output().  This
     led me to constify a lot of code called by output routines.

   3 Constify the sockaddr argument to protosw->pr_ctlinput.  This
     led me to constify a lot of code called by ctlinput routines.

   4 Introduce const macros for converting from a generic sockaddr
     to family-specific sockaddrs, e.g., sockaddr_in: satocsin6,
     satocsin, et cetera.

Commit my SoC work
Add ipv6 support for fast_ipsec
Note that currently, packet with extensions headers are not correctly
supported
Change the ipcomp logic

Sync with head.

sync with head.

sync with head.

add some more tcp mowners.

Sync with head.

__unused removal on arguments; approved by core.

sync with head

implement RFC3465 appropriate byte counting.
from Kentaro A. Kurahone, with minor adjustments by me.
the ack prediction part of the original patch was omitted because
it's a separate change.  reviewed by Rui Paulo.

now that we have -Wno-unused-parameter, back out all the tremendously ugly
code to gratuitously access said parameters.

more unused variable fallout.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

change the MOWNER_INIT define to take two args; fix extant struct mowner
decls to use it. Makes options MBUFTRACE compile again and not whinge about
missing structure declarations. (Also makes initialization consistent.)

Modular (I tried ;-) TCP congestion control API. Whenever certain conditions
happen in the TCP stack, this interface calls the specified callback to
handle the situation according to the currently selected congestion
control algorithm.
A new sysctl node was created: net.inet.tcp.congctl.{available,selected}
with obvious meanings.
The old net.inet.tcp.newreno MIB was removed.
The API is discussed in tcp_congctl(9).

In the near future, it will be possible to selected a congestion control
algorithm on a per-socket basis.

Discussed on tech-net and reviewed by <yamt>.

revert tcp_close part of tcp_subr.c rev.1.200 because it's unnecessary.
all callers of tcp_close are at splsoftnet already:
	tcp_close
		tcp_input ok
		tcp_disconnect
			tcp_usrreq ok
		tcp_usrclosed
			tcp_usrreq ok
			tcp_disconnect
		tcp_timer_2msl ok
		tcp_drop
			tcp_usrreq
			tcp_disconnect
			tcp_timer_rexmt ok
			tcp_timer_persist ok
			tcp_timer_keep ok
			tcp_input
			syn_cache_get
				tcp_input

Protect calls to pool_put/pool_get that may occur in interrupt context
with spl used to protect other allocations and frees, or datastructure
element insertion and removal, in adjacent code.

It is almost unquestionably the case that some of the spl()/splx() calls
added here are superfluous, but it really seems wrong to see:

	s=splfoo();
	/* frob data structure */
	splx(s);
	pool_put(x);

and if we think we need to protect the first operation, then it is hard
to see why we should not think we need to protect the next.  "Better
safe than sorry".

It is also almost unquestionably the case that I missed some pool
gets/puts from interrupt context with my strategy for finding these
calls; use of PR_NOWAIT is a strong hint that a pool may be used from
interrupt context but many callers in the kernel pass a "can wait/can't
wait" flag down such that my searches might not have found them.  One
notable area that needs to be looked at is pf.

See also:

http://mail-index.netbsd.org/tech-kern/2006/07/19/0003.html
http://mail-index.netbsd.org/tech-kern/2006/07/19/0009.html

sync with head.

sync with head

Import of TCP ECN algorithm for congestion control.
Both available for IPv4 and IPv6.
Basic implementation test results are available at
http://netbsd-soc.sourceforge.net/projects/ecn/testresults.html.

Work sponsored by the Google Summer of Code project 2006.
Special thanks to Kentaro Kurahone, Allen Briggs and Matt Thomas for their
help, comments and support during the project.

sync with head.

Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.

sync with head.

Sync with head.

sync with head.

Coverity CID 1149: Add KASSERT before deref.

Coverity CID 1148: Add KASSERT before deref.

Adapt to in6pcb removal.

Bye netinet6/in6_pcb.h.

merge ktrace-lwp.

Sync with HEAD. Here we go again...

move {tcp,udp}_do_loopback_cksum back to tcp/udp
so that they can be referenced by ipv6.

device independent part of ipv6 rx checksum offloading.

Make this build without INET6.

Implement PMTU checks from:

        http://www.gont.com.ar/drafts/icmp-attacks-against-tcp.html

1. Don't act on ICMP-need-frag immediately if adhoc checks on the
advertised MTU fail. The MTU update is delayed until a TCP retransmit
happens.
2. Ignore ICMP Source Quench messages meant for TCP connections.

From OpenBSD.

- add const
- remove bogus casts
- avoid nested variables

Pull up revision 1.190 (requested by yamt in ticket #251):
fix problems related to loopback interface checksum omission.  PR/29971.
- for ipv4, defer decision to ip layer as h/w checksum offloading does
  so that it can check the actual interface the packet is going to.
- for ipv6, disable it.
  (maybe will be revisited when it implements h/w checksum offloading.)
ok'ed by Jason Thorpe.

Pull up revision 1.189 (requested by kurahone in ticket #199):
Added sysctl tunable limits for the number of maximum SACK holes
per connection and per system.
Idea taken from FreeBSD.

sync with -current

fix problems related to loopback interface checksum omission.  PR/29971.

- for ipv4, defer decision to ip layer as h/w checksum offloading does
  so that it can check the actual interface the packet is going to.
- for ipv6, disable it.
  (maybe will be revisited when it implements h/w checksum offloading.)

ok'ed by Jason Thorpe.

Added sysctl tunable limits for the number of maximum SACK holes
per connection and per system.

Idea taken from FreeBSD.

Pull up revision 1.188 (requested by yamt in ticket #90):
protect tcpipqent with splvm.

Sync with HEAD.

protect tcpipqent with splvm.

sync with head.  xen and whitespace.  xen part is not finished.

simplify data receiver side sack processing.
- introduce t_segqlen, the number of segments in segq/timeq.
  the name is from freebsd.
- rather than maintaining a copy of sack blocks (rcv_sack_block[]),
  build it directly from the segment list when needed.

- use full sized segments unless we actually have SACKs to send.
- avoid TSO duplicate D-SACK.
- send SACKs regardless of TF_ACKNOW.
- don't clear rcv_sack_num when transmitting.

discussed on tech-net@.

s/quence/quench/.

Add an extra `i' to notifes/notifed.

Sync with HEAD.

Hi Perry!

Commit TCP SACK patches from Kentaro A. Karahone's patch at:
   http://www.sigusr1.org/~kurahone/tcp-sack-netbsd-02152005.diff.gz

Fixes in that patch for pre-existing TCP pcb initializations were already
committed to NetBSD-current, so are not included in this commit.

The SACK patch has been observed to correctly negotiate and respond,
to SACKs in wide-area traffic.

There are two indepenently-observed, as-yet-unresolved anomalies:
First, seeing unexplained delays between in fast retransmission
(potentially explainable by an 0.2sec RTT between adjacent
ethernet/wifi NICs); and second, peculiar and unepxlained TCP
retransmits observed over an ath0 card.

After discussion with several interested developers, I'm committing
this now, as-is, for more eyes to use and look over.  Current hypothesis
is that the anomalies above may in fact be due to link/level (hardware,
driver, HAL, firmware) abberations in the test setup, affecting  both
Kentaro's  wired-Ethernet NIC and in my two (different) WiFi NICs.

nuke trailing whitespace

Sync with HEAD.

Initialize t_partialacks in the tcpcb template.
From Kentaro A. Kurahone.

Sync with HEAD.

sync with head.

ntohs->htons for ip6 plen (payload length).
It is not technically necessary to set plen here, since ip6_output() starts
off by calculating it, but leaving it keeps it consistent with other code.

Sync with HEAD.

ANSIfy function declarations

de-__P -- will ANSIfy .c files later.

Sync with HEAD.

In tcp_respond(), clear the m_pkthdr.csum_flags that was inherited from the
received packet so that the checksum is not performed twice.  Also,
tcp_respond() does not fill-in the m_pkthdr.csum_data, so a h/w checksum may
have the wrong offset.

OK from Jason Thorpe.

yamt's changes seem to fix all the checksumming issues. Turn the loopback
checksums back off so we can make sure that everything works.

Sync with HEAD.

Turn checksumming on loopback back on until we fix the bugs in it.
Connect over tcp on the loopback is broken:

  4729 amq      0.000007 CALL  connect(4,0x804f2a0,0x1c)
  4729 amq      75.007420 RET   connect -1 errno 60 Connection timed out

Don't perform checksums on loopback interfaces.  They can be reenabled with
the net.inet.*.do_loopback_cksum sysctl.

Approved by: groo

Fix the sync with head I botched.

Apply patch (requested by yamt in ticket #861):
  Fix this so it compiles again; we cannot use the link
  set macros for pool initialization on this release branch.

Pull up revision 1.173 (requested by yamt in ticket #861):
  Fix ipqent pool corruption problems.  Make the TCP reassembly
  code use its own pool of ipqent rather than sharing it with
  the IP reassembly code.  Fixes PR#24782.

Sync with HEAD.

fix ipqent pool corruption problems.  make tcp reass code use
its own pool of ipqent rather than sharing it with ip reass code.
PR/24782.

Sync with HEAD

fix MD5 signature support to actually validate inbound signature, and
drop packet if fails.

Use EVCNT_ATTACH_STATIC{,2}

zero-clear ip6?pseudo before use

declare ip6_hdr_pseudo (for kernel only) and use it for TCP MD5 signature

sync comment with reality

make TCP MD5 signature work with KAME IPSEC (#define IPSEC).

support IPv6 if KAME IPSEC (RFC is not explicit about how we make data stream
for checksum with IPv6, but i'm pretty sure using normal pseudo-header is the
right thing).

XXX
current TCP MD5 signature code has giant flaw:
it does not validate signature on input (can't believe it! what is the point?)

Initial commit of a port of the FreeBSD implementation of RFC 2385
(MD5 signatures for TCP, as used with BGP).  Credit for original
FreeBSD code goes to Bruce M. Simpson, with FreeBSD sponsorship
credited to sentex.net.  Shortening of the setsockopt() name
attributed to Vincent Jardin.

This commit is a minimal, working version of the FreeBSD code, as
MFC'ed to FreeBSD-4. It has received minimal testing with a ttcp
modified to set the TCP-MD5 option; BMS's additions to tcpdump-current
(tcpdump -M) confirm that the MD5 signatures are correct.  Committed
as-is for further testing between a NetBSD BGP speaker (e.g., quagga)
and industry-standard BGP speakers (e.g., Cisco, Juniper).


NOTE: This version has two potential flaws. First, I do see any code
that verifies recieved TCP-MD5 signatures.  Second, the TCP-MD5
options are internally padded and assumed to be 32-bit aligned. A more
space-efficient scheme is to pack all TCP options densely (and
possibly unaligned) into the TCP header ; then do one final padding to
a 4-byte boundary.  Pre-existing comments note that accounting for
TCP-option space when we add SACK is yet to be done. For now, I'm
punting on that; we can solve it properly, in a way that will handle
SACK blocks, as a separate exercise.

In case a pullup to NetBSD-2 is requested, this adds sys/netipsec/xform_tcp.c
,and modifies:

sys/net/pfkeyv2.h,v 1.15
sys/netinet/files.netinet,v 1.5
sys/netinet/ip.h,v 1.25
sys/netinet/tcp.h,v 1.15
sys/netinet/tcp_input.c,v 1.200
sys/netinet/tcp_output.c,v 1.109
sys/netinet/tcp_subr.c,v 1.165
sys/netinet/tcp_usrreq.c,v 1.89
sys/netinet/tcp_var.h,v 1.109
sys/netipsec/files.netipsec,v 1.3
sys/netipsec/ipsec.c,v 1.11
sys/netipsec/ipsec.h,v 1.7
sys/netipsec/key.c,v 1.11
share/man/man4/tcp.4,v 1.16
lib/libipsec/pfkey.c,v 1.20
lib/libipsec/pfkey_dump.c,v 1.17
lib/libipsec/policy_token.l,v 1.8
sbin/setkey/parse.y,v 1.14
sbin/setkey/setkey.8,v 1.27
sbin/setkey/token.l,v 1.15

Note that the preceding two revisions to tcp.4 will be
required to cleanly apply this diff.

Initialise (most) pools from a link set instead of explicit calls
to pool_init.  Untouched pools are ones that either in arch-specific
code, or aren't initialiased during initial system startup.

 Convert struct session, ucred and lockf to pools.

Pull up revision 1.164 (requested by tls in ticket #172):
Change the default state of two tunables; bring our TCP a little bit
closer to normal behaviour for the current century.
New Reno is now on by default (which is really the only reasonable
choice, since we don't do SACK); instead of an initial window of 1
for non-local nets, we now use Sally Floyd's magic 4K rule.

Change the default state of two tunables; bring our TCP a little bit
closer to normal behaviour for the current century.

New Reno is now on by default (which is really the only reasonable
choice, since we don't do SACK); instead of an initial window of 1
for non-local nets, we now use Sally Floyd's magic 4K rule.

Pullup patch (requested by itojun in ticket #169)

If a segment is received with RST set and the segment is completely to the
left of the receive window, ignore it.  Add some additional comments to
the code that deals with received segemnts that are completely to the right
of the receive window.  If an invalid SYN is received, force an ACK and
drop it; if the other side really sent the SYN; it'll respond with a reset.
Respond to RST by ACK, as suggested in NISCC recommendation.
Rate-limit ACKs against RSTs and SYNs.
If SYN is coming and RCV.NXT == SEG.SEQ, then ACK with value - 1.

Pullup patch (requested by itojun in ticket #1680)

If a segment is received with RST set and the segment is completely to the
left of the receive window, ignore it.  Add some additional comments to
the code that deals with received segemnts that are completely to the right
of the receive window.  If an invalid SYN is received, force an ACK and
drop it; if the other side really sent the SYN; it'll respond with a reset.
Respond to RST by ACK, as suggested in NISCC recommendation.
Rate-limit ACKs against RSTs and SYNs.
If SYN is coming and RCV.NXT == SEG.SEQ, then ACK with value - 1.

- respond to RST by ACK, as suggested in NISCC recommendation
- rate-limit ACKs against RSTs and SYNs

adjust to the sbreserve prototype change.

Pullup rev 1.161 (requested by christos in ticket #69)

Invoking tcpcb's get erroneously free'd resulting in to_ticks <= 0 assertion.
PR#22551

PR/22551: Invoking tcpcb's get erroneously free'd resulting in to_ticks <= 0
assertion. Approved by he.

When accepting a peer's MSS, never let it drop below 256 (SLIP + TCP will
be the lowest MSS we should ever enounter).

- Change callout_setfunc() to require that the callout handle is already
  initialized.  Update the txp(4) to compensate.
- Statically initialize the TCP timer callout handles in the tcpcb
  template.  We still use callout_setfunc(), but that call is now much
  less expensive.  Add a comment that the compiler is likely to unroll
  the loop (so don't sweat that it's there).

initialize off

Pullup rev 1.144 (requested by he in ticket #1530)


        Introduce a new INVOKING status for callouts, and use it to close
        a race condition in the TCP code.  Fixes PR#20390.

Oops, a little to aggressive in the previous patch; TCP_TIMER_INIT()
still needs to be in tcp_newtcpcb(), for now.  Pointed out by enami.

Rather than zeroing a tcpcb structure and filling in all the fields
individually, create a tcpcb template pre-initialized (and pre-zero'd)
with the static and mostly-static tcpcb parameters.  The template is
now copied into the new tcpcb, which zeros and initializes most of the
tcpcb in one pass.  The template is kept up-to-date as TCP sysctl
variables are changed.

Combined with the previous sb_max change, TCP socket creation is now
25% faster.

Add event counters that measure FAST_MBSEARCH.

Fix glaring errors in recent changes.

Pull up revision 1.153 (requested by itojun in ticket #1452):
initialize ip_hl for ipsec policy lookup.  PR kern/22715

Pull up rev. 1.153 (requested by itojun in ticket #78):
 Initialize ip_hl for ipsec policy lookup. Fixes PR 22715.

initialize ip_hl for ipsec policy lookup.  PR kern/22715

randomize IPv4/v6 fragment ID and IPv6 flowlabel.  avoids predictability
of these fields.  ip_id.c is from openbsd.  ip6_id.c is adapted by kame.

Pull up revision 1.128 (requested by tls in ticket #1445):
path MTU discovery blackhole detection.
PR 12790 (sorry for not committing it for a long time)

revamp inpcb/in6pcb so that they are more aligned with each other.
in6pcb lookup now uses hash(9).

tp could be null in tcp_respond()

remove ipsec_set/getsocket.  now we explicitly pass socket * to ip{,6}_output.

change the additional arg to be passed to ip{,6}_output to struct socket *.

this fixes KAME policy lookup which was broken by the previous commit.

Replace the set_socket() method of passing an extra struct socket*
argument to ip6_output() with a new explicit struct in6pcb* argument.
(The underlying socket can be obtained via in6pcb->inp6_socket.)

In preparation for fast-ipsec.  Reviewed by itojun.

(fast-ipsec): Add hooks to pass IPv4 IPsec traffic into fast-ipsec, if
configured with ``options FAST_IPSEC''.  Kernels with KAME IPsec or
with no IPsec should work as before.

All calls to ip_output() now always pass an additional compulsory
argument: the inpcb associated with the packet being sent,
or 0 if no inpcb is available.

Fast-ipsec tested with ICMP or UDP over ESP. TCP doesn't work, yet.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

As a temporary workaround, apply the fix from PR#20390, thereby
cooperating with the callout code in working around the race
condition caused by the TCP code's use of the callout facility.

Instead of unconditionally releasing memory in tcp_close() and
SYN_CACHE_PUT(), check whether any of the related callout handlers
are about to be invoked (but have not yet done callout_ack()), and
if so, just mark the associated data structure (tcpcb or syn cache
entry) as "dead", and test for this (and release storage) in the
callout handler functions.

Make it possible to set TCP_INIT_WIN and TCP_INIT_WIN_LOCAL in the config
file as options.

Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Add code to remember where in the send queue of mbufs the last packet was
sent from. This change avoid a linear search through all mbufs when using
large TCP windows, and therefore permit high-speed connections on long
distances.

Tested on a 1 Gigabit connection between Luleå and San Francisco, a distance
of about 15000km.  With TCP windows of just over 20 Mbytes it could keep up
with 950Mbit/s.

After discussions with Matt Thomas and Jason Thorpe.

Make sure to include opt_foo.h if a defflag option FOO is used.

Allow TCP connections to hosts on a local network to use a larger
slow start initial window.  Default this larger initial window to
4 packets, allowing it to be adjusted with net.inet.tcp.init_win_local.

Add MBUFTRACE kernel option.
Do a little mbuf rework while here.  Change all uses of MGET*(*, M_WAIT, *)
to m_get*(M_WAIT, *).  These are not performance critical and making them
call m_get saves considerable space.  Add m_clget analogue of MCLGET and
make corresponding change for M_WAIT uses.
Modify netinet, gem, fxp, tulip, nfs to support MBUFTRACE.
Begin to change netstat to use sysctl.

Sync with HEAD.

Quell an uninitialised variable warning.

Catch up to -current

fix typo in previous: s/tip/top/

Micro-optimisation: don't check if the high bit is set and then mask it
off - just mask it off anyways.  Saves a branch 50% of the time.

Catch up to -current.

sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work

minor KNF

sync kqueue branch with HEAD

catch up with -current.

Catch up to -current.

avoid swapping endian of ip_len and ip_off on mbuf, to meet with M_LEADINGSPACE
optimization made last year.  should solve PR 17867 and 10195.

IP_HDRINCL behavior of raw ip socket is kept unchanged.  we may want to
provide IP_HDRINCL variant that does not swap endian.

Catch up to -current.

catch up with -current.

Pull up revision 1.132 (requested by itojun in ticket #422):
check AF_INET6 socketes when IPv4 "too big" messages arrive.
PR 17448

check AF_INET6 socketes when IPv4 "too big" messages arrive.
PR 17448

catch up with -current on kqueue branch

catch up with -current.

Catch up to -current.

whitespace

Catch up with -current.

attach nd_ifinfo structure into if_afdata.
split IPv6 link MTU (advertised by RA) from real link MTU.
sync with kame

use arc4random() on tcp iss generation

path MTU discovery blackhole detection.
PR 12790 (sorry for not committing it for a long time)

Eliminate commons.

Change struct ipqe to use TAILQ's instead of LIST's (primarily for TCP's
benefit currently).  Rework tcp_reass code to optimize the 4 most likely causes
of out-of-order packets: first OoO pkt, next OoO pkt in seq, OoO pkt is part
of new chuck of OoO packets, and the OoO pkt fills the first hole.  Add evcnts
to instrument tcp_reass (enabled by the options TCP_REASS_COUNTERS).  This is
part 1/2 of tcp_reass changes.

Update from trunk.

* Instrument tcp_build_datapkt().
* Remove the code that allocates a cluster if the packet would
  fit in one; it totally defeats doing references to M_EXT mbufs
  in the socket buffer.  This drastically reduces the number of
  data copies in the tcp_output() path for applications which use
  large writes.  Kudos to Matt Thomas for pointing me in the right
  direction.

Catch up to -current.
(CVS: It's not just a program. It's an adventure!)

Catch up with -current.

have tcp6_drain

Pool deals fairly well with physical memory shortage, but it doesn't
deal with shortages of the VM maps where the backing pages are mapped
(usually kmem_map).  Try to deal with this:

* Group all information about the backend allocator for a pool in a
  separate structure.  The pool references this structure, rather than
  the individual fields.
* Change the pool_init() API accordingly, and adjust all callers.
* Link all pools using the same backend allocator on a list.
* The backend allocator is responsible for waiting for physical memory
  to become available, but will still fail if it cannot callocate KVA
  space for the pages.  If this happens, carefully drain all pools using
  the same backend allocator, so that some KVA space can be freed.
* Change pool_reclaim() to indicate if it actually succeeded in freeing
  some pages, and use that information to make draining easier and more
  efficient.
* Get rid of PR_URGENT.  There was only one use of it, and it could be
  dealt with by the caller.

From art@openbsd.org.

Sync kqueue branch with -current.

Catch up to -current.

add RCSIDs

Sync the thorpej-mips-cache branch with -current.

Convert netinet to not use the internal <sys/queue.h> field names
but instead the access macros.  Use the FOREACH macros where appropriate.

Change a few variable/tables to const since they are read-only.

Catch up with -current.

Catch up to -current.

Update the kqueue branch to HEAD.

Use callouts for SYN cache timers, rather than traversing time queues
in tcp_slowtimo().

Use callouts for TCP timers, rather than traversing the list of
all open TCP connections in tcp_slowtimo() (which is called 2x
per second).  It's fairly rare for TCP timers to actually fire,
so saving this list traversal is good, especially if you want
to scale to thousands of open connections.

Initialize TCP timer variables in a new function, tcp_timer_init().

Add explicit initialization of TCP timer state.  A noop right now.

Use a callout for the delayed ACK timer, and delete tcp_fasttimo().
Expose the delayed ACK timer as net.inet.tcp.delack_ticks.

Catch up with -current.

update to -current

wrap IPv6 code by #ifdef INET6

use in6_maxmtu, not in_maxmtu, for IPv6 mss computation

Catch up to -current.