CVS log for src/sys/net/npf/npf_impl.h

Up to [cvs.NetBSD.org] / src / sys / net / npf

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

npf: Make sure to initialize portmap_lock only once.

PR kern/55586

Pull up following revision(s) (requested by rmind in ticket #956):

	usr.sbin/npf/npf-params.7: revision 1.4
	sys/net/npf/npf_worker.c: revision 1.9
	usr.sbin/npf/npftest/npftest.h: revision 1.17
	usr.sbin/npf/npfctl/npf_bpf_comp.c: revision 1.16
	usr.sbin/npf/npf-params.7: revision 1.5
	sys/net/npf/npf_state_tcp.c: revision 1.21
	usr.sbin/npf/npfctl/npf_build.c: revision 1.55
	usr.sbin/npf/npf-params.7: revision 1.6
	sys/net/npf/npfkern.h: revision 1.5
	lib/libnpf/npf.c: revision 1.49
	usr.sbin/npf/npf-params.7: revision 1.7
	sys/net/npf/npf_impl.h: revision 1.81
	sys/net/npf/npf_ext_log.c: revision 1.17
	usr.sbin/npf/npfctl/npfctl.h: revision 1.53
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.11
	sys/net/npf/npf_nat.c: revision 1.50
	sys/net/npf/npf_mbuf.c: revision 1.24
	sys/net/npf/npf_alg.c: revision 1.22
	usr.sbin/npf/npftest/libnpftest/npf_nat_test.c: revision 1.14
	usr.sbin/npf/npftest/libnpftest/npf_conn_test.c: file removal
	usr.sbin/npf/npftest/libnpftest/npf_state_test.c: revision 1.10
	sys/net/npf/npf.h: revision 1.63
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.21
	usr.sbin/npf/npfctl/npf_var.c: revision 1.13
	sys/net/npf/files.npf: revision 1.23
	usr.sbin/npf/npfctl/npf_show.c: revision 1.32
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.91
	sys/net/npf/npf_os.c: revision 1.18
	sys/net/npf/npf_connkey.c: revision 1.2
	sys/net/npf/npf_conf.c: revision 1.17
	lib/libnpf/libnpf.3: revision 1.12
	usr.sbin/npf/npftest/npftest.c: revision 1.25
	usr.sbin/npf/npftest/libnpftest/npf_gc_test.c: revision 1.1
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.51
	sys/net/npf/npf_tableset.c: revision 1.35
	usr.sbin/npf/npftest/npftest.conf: revision 1.9
	sys/net/npf/npf_sendpkt.c: revision 1.22
	usr.sbin/npf/npfctl/npf_var.h: revision 1.10
	sys/net/npf/npf_state.c: revision 1.23
	sys/net/npf/npf_conn.h: revision 1.20
	usr.sbin/npf/npfctl/npfctl.c: revision 1.64
	usr.sbin/npf/npfctl/npf_cmd.c: revision 1.1
	sys/net/npf/npf_portmap.c: revision 1.5
	sys/net/npf/npf_params.c: revision 1.3
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.32
	tests/net/npf/t_npf.sh: revision 1.4
	sys/net/npf/npf_ext_rndblock.c: revision 1.9
	lib/libnpf/npf.h: revision 1.39
	sys/net/npf/npf_ruleset.c: revision 1.51
	sys/net/npf/npf_alg_icmp.c: revision 1.33
	sys/net/npf/npf.c: revision 1.43
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.17
	usr.sbin/npf/npfctl/npfctl.8: revision 1.25
	sys/net/npf/npf_ctl.c: revision 1.60
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.18
	usr.sbin/npf/npftest/libnpftest/Makefile: revision 1.11
	sys/net/npf/npf_handler.c: revision 1.49
	sys/net/npf/npf_inet.c: revision 1.57
	sys/net/npf/npf_ifaddr.c: revision 1.7
	sys/net/npf/npf_conndb.c: revision 1.9
	sys/net/npf/npf_if.c: revision 1.13
	usr.sbin/npf/npfctl/Makefile: revision 1.15
	sys/net/npf/npf_conn.c: revision 1.32
	sys/net/npf/npf_ext_normalize.c: revision 1.10
	sys/net/npf/npf_rproc.c: revision 1.20
	sys/net/npf/npf_worker.c: revision 1.8

Major NPF improvements (merge from upstream):
- Switch to the C11-style atomic primitives using atomic_loadstore(9).
- npfkern: introduce the 'state.key.interface' and 'state.key.direction'
  settings.  Users can now choose whether the connection state should be
  strictly per-interface or global at the configuration level.  Keep NAT
  logic to be always per-interface, though.
- npfkern: rewrite the G/C worker logic and make it self-tuning.
- npfkern and libnpf: multiple bug fixes; add param exporting; introduce
  more parameters.  Remove npf_nvlist_{copyin,copyout}() functions and
  refactor npfctl_load_nvlist() with others; add npfctl_run_op() to have
  a single entry point for operations.  Introduce npf_flow_t and clean up
  some code.
- npfctl: lots of fixes for the 'npfctl show' logic; make 'npfctl list'
  more informative; misc usability improvements and more user-friendly
  error messages.
- Amend and improve the manual pages.

npf_worker_sys{init,fini}: initialize/destroy the exit_cv condvar.

npftest -- npf_test_init(): add a workaround for NetBSD.

npf-params(7): fix the state.key defaults.

npf-params.7: s/filer/filter/

Adjust to "npfctl debug" command line changes, from rmind@.

Use more markup.

Major NPF improvements (merge from upstream):

- Switch to the C11-style atomic primitives using atomic_loadstore(9).

- npfkern: introduce the 'state.key.interface' and 'state.key.direction'
  settings.  Users can now choose whether the connection state should be
  strictly per-interface or global at the configuration level.  Keep NAT
  logic to be always per-interface, though.

- npfkern: rewrite the G/C worker logic and make it self-tuning.

- npfkern and libnpf: multiple bug fixes; add param exporting; introduce
  more parameters.  Remove npf_nvlist_{copyin,copyout}() functions and
  refactor npfctl_load_nvlist() with others; add npfctl_run_op() to have
  a single entry point for operations.  Introduce npf_flow_t and clean up
  some code.

- npfctl: lots of fixes for the 'npfctl show' logic; make 'npfctl list'
  more informative; misc usability improvements and more user-friendly
  error messages.

- Amend and improve the manual pages.

Mostly merge changes from HEAD upto 20200411

Pull up following revision(s) (requested by rmind in ticket #282):

	usr.sbin/npf/npfctl/npf_build.c: revision 1.53
	lib/libnpf/npf.c: revision 1.48
	usr.sbin/npf/npfctl/npfctl.h: revision 1.50
	sys/net/npf/npf_impl.h: revision 1.80
	usr.sbin/npf/npfctl/npfctl.h: revision 1.51
	sys/net/npf/npf_ruleset.c: revision 1.49
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.90
	sys/net/npf/npf_ctl.c: revision 1.59
	lib/libnpf/libnpf.3: revision 1.11
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.50
	usr.sbin/npf/npftest/npftest.conf: revision 1.8
	usr.sbin/npf/npfctl/npfctl.c: revision 1.62
	usr.sbin/npf/npfctl/npfctl.c: revision 1.63
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.30
	usr.sbin/npf/npfctl/npfctl.8: revision 1.22
	lib/libnpf/npf.h: revision 1.38
	usr.sbin/npf/npfctl/npfctl.8: revision 1.23
	usr.sbin/npf/npfctl/npfctl.8: revision 1.24
	sys/net/npf/npf_if.c: revision 1.11
	sys/net/npf/npf_if.c: revision 1.12
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.89
	sys/net/npf/npf_conn.c: revision 1.30
	usr.sbin/npf/npfctl/npf_build.c: revision 1.52

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.

NPF ifmap: rework and fix a few small bugs.

npfctl: implement table replace subcommand.
Contributed by Timshel Knoll-Miller.
(missed a file in previous commit; cvs is so helpful..)

libnpf/npfctl: support dynamic NAT rulesets using a name prefix.

Use -width Pa for FILES.

Fix pasto in table replace -t type

Use -width Pa for FILES.

npf_ifmap_copylogname: be more defensive.

NPF ifmap: rework and fix a few small bugs.

Pull up following revision(s) (requested by rmind in ticket #141):

	usr.sbin/npf/npfctl/npf_bpf_comp.c: revision 1.15
	sys/net/npf/npf_alg.c: revision 1.21
	sys/net/npf/npf.h: revision 1.62
	sys/net/npf/npf_ctl.c: revision 1.57
	sys/net/npf/npf_ctl.c: revision 1.58
	sys/net/npf/npf_os.c: revision 1.16
	sys/net/npf/npf_os.c: revision 1.17
	sys/net/npf/npf_conf.c: revision 1.15
	sys/net/npf/npf_impl.h: revision 1.78
	sys/sys/mbuf.h: revision 1.220
	sys/net/npf/npf_impl.h: revision 1.79
	sys/net/npf/npf.c: revision 1.41
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.19
	sys/net/npf/npf_nat.c: revision 1.48
	sys/net/npf/npf_handler.c: revision 1.48
	sys/net/npf/npf_ifaddr.c: revision 1.6

- npfctl_load_nvlist: simplify the config loading logic.
- Fix a small race condition in npf_nat_getaddr().
- Rework pserialize/EBR wrappers, make it easier to maintain.
Move PACKET_TAG_NPF where it belongs to.
Make npfctl_switch() and pfil private to OS-specific module.

Pull up following revision(s) (requested by rmind in ticket #139):

	lib/libnpf/npf.c: revision 1.47
	usr.sbin/npf/npftest/libnpftest/npf_nbuf_test.c: revision 1.10
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.10
	sys/net/npf/npf.h: revision 1.61
	sys/net/npf/npf_ctl.c: revision 1.56
	sys/net/npf/npf_os.c: revision 1.15
	lib/libnpf/libnpf.3: revision 1.10
	sys/net/npf/npf_tableset.c: revision 1.34
	usr.sbin/npf/npfctl/npfctl.c: revision 1.61
	sys/net/npf/npf_impl.h: revision 1.77
	lib/libnpf/npf.h: revision 1.37

- npftest: fix a memleak in a unit test (standalone path only).
- Minor style fixes.  No functional change.
npfkern/libnpf: Add support for the table replace/swap operation.
Contributed by Timshel Knoll-Miller.

 ake npfctl_switch() and pfil private to OS-specific module.

- npfctl_load_nvlist: simplify the config loading logic.
- Fix a small race condition in npf_nat_getaddr().
- Rework pserialize/EBR wrappers, make it easier to maintain.

npfkern/libnpf: Add support for the table replace/swap operation.
Contributed by Timshel Knoll-Miller.

Pull up following revision(s) (requested by rmind in ticket #49):

	usr.sbin/npf/npf.7: revision 1.7
	sys/net/npf/npfkern.h: revision 1.4
	sys/net/npf/npf_conn.h: revision 1.18
	usr.sbin/npf/npftest/libnpftest/npf_nat_test.c: revision 1.13
	sys/net/npf/npf_ctl.c: revision 1.55
	sys/net/npf/npf_os.c: revision 1.14
	sys/net/npf/npf_conf.c: revision 1.14
	usr.sbin/npf/npftest/libnpftest/npf_conn_test.c: revision 1.3
	usr.sbin/npf/npftest/libnpftest/npf_perf_test.c: revision 1.9
	sys/net/npf/npf_impl.h: revision 1.76
	sys/net/npf/npf_portmap.c: revision 1.4
	sys/net/npf/npf_params.c: revision 1.2
	sys/net/npf/npf.c: revision 1.40
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.16
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.18
	sys/net/npf/npf_nat.c: revision 1.47
	sys/net/npf/npf_handler.c: revision 1.47
	sys/net/npf/npf_inet.c: revision 1.55
	sys/net/npf/npf_if.c: revision 1.10
	sys/net/npf/npf_worker.c: revision 1.7
	usr.sbin/npf/npf-params.7: revision 1.3

npf-params(7): add more bpf.jit details.
From David H. Gutteridge.

Adjust some internal NPF APIs:
* npfkern: use the npfk_ prefix.
* NPF portmap: amend the API so it could be used elsewhere.
* Make npf_connkey_t public.

npf.7: add xref to npf-params.7
(Adding directly here since this particular file isn't included in
rmind@'s upstream GitHub repo at present.)

Adjust some internal NPF APIs:
* npfkern: use the npfk_ prefix.
* NPF portmap: amend the API so it could be used elsewhere.
* Make npf_connkey_t public.

NPF improvements:
- Add support for dynamic NETMAP algorithm (stateful net-to-net).
- Add most of the support for the dynamic NAT rules; a little bit more
  userland work is needed to finish this up and enable.
- Replace 'stateful-ends' with more permissive 'stateful-all'.
- Add various tunable parameters and document them, see npf-params(7).
- Reduce the memory usage of the connection state table (conndb).
- Portmap rewrite: use memory more efficiently, handle addresses dynamically.
- Bug fix: add splsoftnet()/splx() around the thmap writers and comment.
- npftest: clean up and simplify; fix some memleaks to make ASAN happy.

Sync with HEAD

Sync with HEAD

Major NPF improvements:
- Convert NPF connection table to thmap.  State lookup is now lock-free.
- Improve connection state G/C: it is now incremental and tunable.
- Add support for dynamic NAT address.  Translation addresses can now be
  selected from a pool of addresses.  There are two selection algorithms,
  "ip-hash" and "round-robin" (see the man page).
- Translation address can be specified as e.g. ifaddrs(wm0) in npf.conf
  to dynamically choose an IP from the interface address(es).
- Add support for the NETMAP algorithm with static NAT for net-to-net
  translation (it is equivalent to iptables NETMAP logic).
- Convert 'ipset' tables to use thmap; the table lookup is now lock-free.
- Misc improvements, bug fixes and more unit tests.
- Bump NPF_VERSION (will also bump libnpf).

Ssync with HEAD

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
  a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
  previous versions.  Also, different serialisation format means NPF
  connection/config saving and loading is not compatible with the
  previous versions either.

Thanks to christos@ for extra testing.

Fix lockdebug diagnostic error of trying to acquire an rw_lock from a
pserialized active context. From riastradh@

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Introduce npf_set_mss(). When the MSS is not 16bit-aligned, it sets:

	0      8           16          24    32
	+------+-----------+-----------+------+
	| data | MSS (low) | MSS (hig) | data |
	+------+-----------+-----------+------+
	^                  ^
	old[0]             old[1]

And sets new[0,1] accordingly with the new value. The MSS-clamping code
then adjusts twice the checksum on a 16bit boundary:

	from old[0] to new[0]
	from old[1] to new[1]

Fixes PR/53479, opened by myself. Tested with wireshark and kASan.

- npf_mk_rules: enforce unique names for the dynamic rulesets.
- npf_worker_unregister: merge fix for the standalone NPF.

update from HEAD

Sync with HEAD

Sync with HEAD

Sync with HEAD

forgot to commit this (new prototype)

Sync with HEAD

- Increase copyin buffer size to 4M
- Change log output format to be like the OpenBSD's pf including in
  the header the matching rule etc, and fill in the matching info.

Sync with netbsd-5

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

NPF: fix the interface table initialisation on load.

NPF: implement dynamic handling of interface addresses (the kernel part).

export rprocs too so we don't lose them.

Sync NPF with the version on github: backport standalone NPF changes,
which allow us to create and run separate NPF instances. Minor fixes.
(from rmind@)

Pull up following revision(s) (requested by rmind in ticket #1319):
	sys/modules/npf/Makefile: revision 1.19
	sys/net/npf/files.npf: revision 1.18
	sys/net/npf/lpm.c: revision 1.1
	sys/net/npf/lpm.h: revision 1.1
	sys/net/npf/npf_impl.h: revision 1.62
	sys/net/npf/npf_tableset.c: revision 1.24
	sys/net/npf/npf_tableset_ptree.c: file removal
	sys/rump/net/lib/libnpf/Makefile: revision 1.18
This patches ditches the ptree(3) library, because it is broken (you
can get missing entries!).  Instead, as a temporary solution, we switch
to a simple linear scan of the hash tables for the longest-prefix-match
(lpm.c lpm.h) algorithm. In fact, with few unique prefixes in the set,
on modern hardware this simple algorithm is pretty fast anyway!
--
ditch ptree and use lpm
--
remove ptree add lpm

add functionality to lookup a nat entry from the connection list.

This patches ditches the ptree(3) library, because it is broken (you
can get missing entries!).  Instead, as a temporary solution, we switch
to a simple linear scan of the hash tables for the longest-prefix-match
(lpm.c lpm.h) algorithm. In fact, with few unique prefixes in the set,
on modern hardware this simple algorithm is pretty fast anyway!

Sync with HEAD

Pull up following revision(s) (requested by rmind in ticket #479):
	lib/libnpf/npf.c: revision 1.35
	lib/libnpf/npf.h: revision 1.28
	sys/net/npf/npf_conn.c: revision 1.15
	sys/net/npf/npf_impl.h: revision 1.61
	sys/net/npf/npf_ruleset.c: revision 1.41
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.44
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.37
	usr.sbin/npf/npfctl/npf_show.c: revisions 1.16, 1.17
	usr.sbin/npf/npfctl/npfctl.c: revision 1.46
load the config file before bpfjit so that we can disable the warning.
--
Don't depend on yacc to include stdlib.h or string.h.
--
- npf_conn_establish: remove a rare race condition when we might destroy a
  connection when it is still referenced by another thread.
- npf_conn_destroy: remove the backwards entry using the saved key, PR/49488.
- Sprinkle some asserts.
--
npf.conf(5): mention alg, include in the example, minor fix.
--
npfctl(8): report dynamic rule ID in a comment, print the case when libpcap
is used correctly.  Also, add npf_ruleset_dump() helper in the kernel.
--
libnpf: add npf_rule_getid() and npf_rule_getcode().
Missed in the previous commit.
--
npfctl_print_rule: print the ID in hex, not decimal.

npfctl(8): report dynamic rule ID in a comment, print the case when libpcap
is used correctly.  Also, add npf_ruleset_dump() helper in the kernel.

Pull up following revision(s) (requested by rmind in ticket #280):
	sys/net/npf/npf_ruleset.c: revision 1.40
	sys/net/npf/npf_nat.c: revision 1.36
	sys/net/npf/npf_nat.c: revision 1.37
	sys/net/npf/npf_conn.h: revision 1.7
	sys/net/npf/npf_conf.c: revision 1.9
	sys/net/npf/npf_ruleset.c: revision 1.39
	sys/net/npf/npf_conn.c: revision 1.13
	sys/net/npf/npf_impl.h: revision 1.60
NPF:
- npf_nat_import: take the port only if using the portmap.
- Sprinkle some comments and asserts.
- npf_config_load: if loading the connections, do not perform any actice
  NAT policy take over or or portmap sharing - just replace them all.
- npf_config_fini: flush with the empty connection database.
- npf_nat_import: fix the stat counter.

- npf_config_load: if loading the connections, do not perform any actice
  NAT policy take over or or portmap sharing - just replace them all.
- npf_config_fini: flush with the empty connection database.
- npf_nat_import: fix the stat counter.

Pull up following revision(s) (requested by rmind in ticket #56):
	sys/net/npf/npf_ctl.c: revision 1.39
	usr.sbin/npf/npfctl/npfctl.c: revision 1.43
	lib/libnpf/npf.c: revision 1.33
	lib/libnpf/npf.c: revision 1.34
	sys/net/npf/npf_impl.h: revision 1.59
	sys/net/npf/npf_ctl.c: revision 1.40
	sys/net/npf/npf_conn.c: revision 1.11
	sys/net/npf/npf_alg.c: revision 1.15
	sys/net/npf/npf_conn.c: revision 1.12
	sys/net/npf/npf_nat.c: revision 1.33
	sys/net/npf/npf_nat.c: revision 1.34
Add and use npf_alg_export().
npf_conn_import: handle NAT metadata correctly.
npf_nat_newpolicy: restore the policy ID.
npfctl_load: fix error code handling for the limit cases.
npf_config_import: fix the inverted logic.
npfctl_load: improve error handling.
npf_conn_import: add a missing stat counter increment.
npf_nat_import: add a missing reference and make a comment.
npf_config_submit: finally, include the saved connections.

Rebase to HEAD as of a few days ago.

- Add and use npf_alg_export().
- npf_conn_import: handle NAT metadata correctly.
- npf_nat_newpolicy: restore the policy ID.
- npfctl_load: fix error code handling for the limit cases.
- npf_config_import: fix the inverted logic.
- npfctl_load: improve error handling.

NPF: finish up the rework of npfctl_save() mechanism.

- Add npf_ruleset_export(), npf_rule_export() and npf_nat_policyexport().
- Split off npf_conn_export().  Add npf_ifmap_getname() and use it to save
  the interface name; pick it up on npf_conn_import().
- Misc fixes.  Bump NPF_VERSION.

Rebase.

NPF: rework of the connection saving and restoring:
- Add support for saving a snapshot of the current connections together
  with a full configuration.  Support a reverse load operation.  Eliminate
  the old 'sess-save' and 'sess-load' in favour of the new mechanism.
- Share code between load and reload operations: the latter performs
  load from npf.conf without affecting the connections.
- Simplify and fix races with connection loading.
- Bump NPF_VERSION.

NPF: add nbuf_t * into npf_cache_t and remove unnecessary carrying by argument.

NPF: partially rewrite the connection tracking mechanism:
- Separate the tracking interface from the storage (state table)
  and thus prepare to use a new data structure for the storage.
- Fix some race conditions in NAT association logic.

Adjust NPF to the recent BPF / BPF JIT changes and make it work again.
All regression tests are happy now (hi alnsn!).

- npf_nat_freepolicy: handle a race condition when a new connection might
  be associated with a NAT policy which is going away and npfctl reload
  would wait for its natural expiration (potentially long time).
- Remove npf_ruleset_natreload() by merging into npf_ruleset_reload().
- npf_ruleset_reload: eliminate a small time period when a valid NAT
  policy might be inactive during the reload operation.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

Add ability to have mbufs disappear (to another interface) during
npf_rproc_run().  For upcoming npf_ext_route extension.

Guidance and ok by rmind@.

sync with head

NPF: add support for "stateful-ends".

NPF: fix the recent breakage of the traceroute ALG.  Also, simplify and
refactor a little bit.

NPF: pass ALG functions via npfa_funcs_t structure.

NPF: add support for IPv6-to-IPv6 Network Prefix Translation (NPTv6),
as per RFC 6296.  Add a unit test.  Also, bump NPF_VERSION.

Thanks to S.P.Zeidler for the help with NPTv6 work!

Add support for CDB based NPF tables.

NPF:
- Adjust NAT to not assume flow direction in some cases and thus support
  less usual setups which are possible when using 'map' with a custom
  filter criteria.
- Introduce NPF_SRC/NPF_DST and replace npc_src/npc_dst with npc_ips[2]
  for more convenient handling.
- ICMP ALG: restrict matching only to the outgoing traffic, but be more
  direction-agnostic elsewhere.

- npf_do_nat: fix a race condition and simplify the logic.
- npf_session_setnat: clear the NAT association on failure.

Move initialisation of bpf_args_t into the npf_ruleset_inspect().
This allows us to reuse the BPF memory store as a cache.

npf_addr_mix: use xor rather than sum.

Add npf_tableset_syncdict() to sync the table IDs in the proplib dictionary,
as they can change on reload now.  Also, fix table name checking in npfctl.

Pull up following revision(s) (requested by rmind in ticket #985):
	sys/net/npf/npf_impl.h: revision 1.35
	sys/net/npf/npf_nat.c: revision 1.21
	sys/net/npf/npf_session.c: revision 1.26
npf_session_setnat: fix the race condition when the old connection is still
being expired while a new/duplicate is being created.

Pull up following revision(s) (requested by rmind in ticket #985):
	sys/net/npf/npf_impl.h: revision 1.35
	sys/net/npf/npf_nat.c: revision 1.21
	sys/net/npf/npf_session.c: revision 1.26
npf_session_setnat: fix the race condition when the old connection is still
being expired while a new/duplicate is being created.

NPF: convert to bpf_jit_generate()/bpf_jit_freecode().

- Add bpf_args_t and convert bpf_filter_ext() to use it.  This allows the
  caller to initialise (and re-use) the memory store.
- Add bpf_jit_generate() and bpf_jit_freecode() wrappers.

NPF: add support for table naming and remove NPF_TABLE_SLOTS (there is
just an arbitrary sanity limit of NPF_MAX_TABLES currently set to 128).

Few misc fixes.  Bump NPF_VERSION.

NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections).  Once the interface is reattached they will
become active.

Bump NPF_VERSION.

npf_generic_fsm and npf_tcp_fsm: use uint8_t and make the arrays more dense.

npf_session_setnat: fix the race condition when the old connection is still
being expired while a new/duplicate is being created.

Add NPF_MAX_RULES, an artificial limit (set it to 1M).

NPF: G/C n-code in favour of BPF byte-code.  Delete lots of code, mmm!

- Convert NPF to use BPF byte-code by default.  Compile BPF byte-code in
  npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

resync from head

- NPF connection tracking: rework synchronisation on tracking disable/enable
  points and document it.  Split the worker thread into a separate module
  with an interface, so it could be re-used for other tasks.
- Replace ALG list with arrays and thus hit fewer cache lines.
- Misc bug fixes.

- Add NPF table flushing functionality.
- Fix line numbering for npfctl debug command.

Make ALG's autoloadable by providing in the config file:
	alg "algname"

resync with head

Pull up following revision(s) (requested by rmind in ticket #829):
	usr.sbin/npf/npfctl/npfctl.8: revision 1.13
	usr.sbin/npf/npfctl/npf_build.c: revision 1.21
	lib/libnpf/npf.c: revision 1.18
	sys/net/npf/npf_ctl.c: revision 1.23
	usr.sbin/npf/npfctl/npfctl.h: revision 1.27
	lib/libnpf/npf.h: revision 1.15
	sys/net/npf/npf_ruleset.c: revision 1.19
	sys/net/npf/npf_impl.h: revision 1.28
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.17
	usr.sbin/npf/npfctl/npfctl.c: revision 1.31
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.6
- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs.  Also, update the man page.

- Convert NPF dynamic rule ID to just incremented 64-bit counter.
- Fix multiple bugs.  Also, update the man page.

Pull up following revision(s) (requested by rmind in ticket #817):
	usr.sbin/npf/npfctl/npfctl.8: revision 1.12
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.27
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.18
	usr.sbin/npf/npfctl/npf_build.c: revision 1.20
	usr.sbin/npf/npfctl/npfctl.c: revision 1.28
	lib/libnpf/npf.c: revision 1.16
	usr.sbin/npf/npfctl/npfctl.c: revision 1.29
	lib/libnpf/npf.c: revision 1.17
	sys/modules/npf/Makefile: revision 1.12
	sys/net/npf/npf_rproc.c: revision 1.6
	usr.sbin/npf/npftest/README: revision 1.4
	sys/net/npf/npf_tableset.c: revision 1.17
	sys/net/npf/npf_ctl.c: revision 1.21
	sys/net/npf/npf_ctl.c: revision 1.22
	usr.sbin/npf/npfctl/npfctl.h: revision 1.25
	lib/libnpf/npf.h: revision 1.13
	usr.sbin/npf/npftest/npftest.conf: revision 1.2
	usr.sbin/npf/npfctl/npfctl.h: revision 1.26
	sys/net/npf/npf_ruleset.c: revision 1.17
	lib/libnpf/npf.h: revision 1.14
	sys/net/npf/npf_ruleset.c: revision 1.18
	sys/net/npf/npf_conf.c: revision 1.1
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.10
	sys/net/npf/npf_conf.c: revision 1.2
	sys/net/npf/npf_instr.c: revision 1.16
	sys/net/npf/npf_handler.c: revision 1.26
	sys/net/npf/npf_impl.h: revision 1.26
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.14
	sys/net/npf/npf_processor.c: revision 1.15
	sys/net/npf/npf_impl.h: revision 1.27
	sys/net/npf/npf_alg_icmp.c: revision 1.15
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.15
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.16
	sys/net/npf/npf_ncode.h: revision 1.11
	sys/net/npf/files.npf: revision 1.10
	usr.sbin/npf/npftest/Makefile: revision 1.4
	usr.sbin/npf/npfctl/npfctl.c: revision 1.30
	lib/libnpf/npf.3: revision 1.8
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.4
	sys/net/npf/npf_session.c: revision 1.21
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.5
	usr.sbin/npf/npfctl/npf_build.c: revision 1.18
	usr.sbin/npf/npfctl/npf_build.c: revision 1.19
	sys/net/npf/npf_alg.c: revision 1.7
	usr.sbin/npf/npfctl/Makefile: revision 1.10
	sys/net/npf/npf_inet.c: revision 1.21
	sys/net/npf/npf.h: revision 1.26
	sys/net/npf/npf.h: revision 1.27
	usr.sbin/pf/ftp-proxy/Makefile: revision 1.8
	sys/net/npf/npf_nat.c: revision 1.19
	sys/net/npf/npf.c: revision 1.15
	sys/net/npf/npf_state.c: revision 1.14
	sys/net/npf/npf_sendpkt.c: revision 1.14
	sys/rump/net/lib/libnpf/Makefile: revision 1.4
IPv6 linklocal address printing cosmetics
NPF:
- Implement dynamic NPF rules.  Controlled through npf(3) library of via
  npfctl rule command.  A rule can be removed using a unique identifier,
  returned on addition, or using a key which is SHA1 hash of the rule.
  Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.
Disable -DWITH_NPF for now; will be converted to BPF mechanism.
- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.
Allow filtering on IP addresses even if the L4 protocol is unknown.
Patch from spz@.
npftest: adjust for recent change.

- Fix NPF config reload with dynamic rules present.
- Implement list and flush commands on a dynamic ruleset.

NPF:
- Implement dynamic NPF rules.  Controlled through npf(3) library of via
  npfctl rule command.  A rule can be removed using a unique identifier,
  returned on addition, or using a key which is SHA1 hash of the rule.
  Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

Pull up following revision(s) (requested by rmind in ticket #777):
	usr.sbin/npf/npfctl/npfctl.c: revision 1.27
	sys/net/npf/npf_session.c: revision 1.19
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.4
	sys/net/npf/npf_rproc.c: revision 1.5
	usr.sbin/npf/npftest/README: revision 1.3
	sys/sys/mbuf.h: revision 1.151
	sys/net/npf/npf_ruleset.c: revision 1.15
	usr.sbin/npf/npftest/libnpftest/npf_nbuf_test.c: revision 1.3
	sys/net/npf/npf_ruleset.c: revision 1.16
	usr.sbin/npf/npftest/libnpftest/npf_state_test.c: revision 1.4
	usr.sbin/npf/npftest/libnpftest/npf_nbuf_test.c: revision 1.4
	sys/net/npf/npf_inet.c: revision 1.19
	sys/net/npf/npf_instr.c: revision 1.15
	sys/net/npf/npf_handler.c: revision 1.24
	sys/net/npf/npf_handler.c: revision 1.25
	sys/net/npf/npf_state_tcp.c: revision 1.12
	sys/net/npf/npf_processor.c: revision 1.13
	sys/net/npf/npf_impl.h: revision 1.25
	sys/net/npf/npf_processor.c: revision 1.14
	sys/net/npf/npf_mbuf.c: revision 1.10
	sys/net/npf/npf_alg_icmp.c: revision 1.14
	sys/net/npf/npf_mbuf.c: revision 1.9
	usr.sbin/npf/npftest/libnpftest/npf_nat_test.c: revision 1.2
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.3
	sys/net/npf/npf_session.c: revision 1.20
	sys/net/npf/npf_alg.c: revision 1.6
	sys/kern/uipc_mbuf.c: revision 1.148
	sys/net/npf/npf_inet.c: revision 1.20
	sys/net/npf/npf.h: revision 1.25
	sys/net/npf/npf_nat.c: revision 1.18
	sys/net/npf/npf_state.c: revision 1.13
	sys/net/npf/npf_sendpkt.c: revision 1.13
	sys/net/npf/npf_ext_log.c: revision 1.2
	usr.sbin/npf/npftest/libnpftest/npf_processor_test.c: revision 1.4
	sys/net/npf/npf_ext_normalise.c: revision 1.2
- Rework NPF's nbuf interface: use advancing and ensuring as a main method.
  Eliminate unnecessary copy and simplify.  Adapt regression tests.
- Simplify ICMP ALG a little.  While here, handle ICMP ECHO for traceroute.
- Minor fixes, misc cleanup.
Silence gcc in npf_recache().
Add m_ensure_contig() routine, which is equivalent to m_pullup, but does not
destroy the mbuf chain on failure (it is kept valid).
- nbuf_ensure_contig: rework to use m_ensure_contig(9), which will not free
  the mbuf chain on failure.  Fixes some corner cases.  Improve regression
  test and sprinkle some asserts.
- npf_reassembly: clear nbuf on IPv6 reassembly failure path (partial fix).
  The problem was found and fix provided by Anthony Mallet.

sync with head

- Rework NPF's nbuf interface: use advancing and ensuring as a main method.
  Eliminate unnecessary copy and simplify.  Adapt regression tests.
- Simplify ICMP ALG a little.  While here, handle ICMP ECHO for traceroute.
- Minor fixes, misc cleanup.

Pull up following revision(s) (requested by rmind in ticket #718):
	usr.sbin/npf/npfctl/npfctl.c: revision 1.22
	usr.sbin/npf/npfctl/npfctl.c: revision 1.23
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.15
	usr.sbin/npf/npfctl/npfctl.c: revision 1.24
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.16
	usr.sbin/npf/npfctl/npfctl.h: revision 1.22
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.14
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.15
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.11
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.12
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.7
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.8
	usr.sbin/npf/npfctl/npf_extmod.c: revision 1.2
	usr.sbin/npf/npfctl/npf_extmod.c: revision 1.3
	usr.sbin/npf/npfctl/npf_var.c: revision 1.6
	usr.sbin/npf/npfctl/npf_var.c: revision 1.7
gcc 4.1 is not smart enough to notice "arg" is only used when initialized
correctly and produces a "might be used unintialized" warning.
npfctl: switch to efun(3) routines.
npfctl: switch to ecalloc(3).

Pull up following revision(s) (requested by rmind in ticket #702):
	sys/net/npf/npf_tableset.c: revision 1.15
	usr.sbin/npf/npfctl/npfctl.h: revision 1.21
	usr.sbin/npf/npftest/libnpftest/npf_table_test.c: revision 1.6
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.10
	sys/net/npf/npf_state_tcp.c: revision 1.11
	sys/net/npf/npf_impl.h: revision 1.24
	sys/net/npf/npf.h: revision 1.22
	sys/net/npf/npf_ctl.c: revision 1.19
	sys/net/npf/npf.c: revision 1.14
	usr.sbin/npf/npfctl/npfctl.8: revision 1.10
	usr.sbin/npf/npfctl/npfctl.c: revision 1.21
npf_tcp_inwindow: inspect the sequence numbers even if the packet contains no
data, fixing up only the RST to the initial SYN.  This makes off-path attacks
more difficult.  For the reference, see "Reflection Scan: an Off-Path Attack
on TCP" by Jan Wrobel.
Implement NPF table listing and preservation of entries on reload.
Bump the version.
npfctl(8): mention table listing.

Resync to 2012-11-19 00:00:00 UTC

Pull up following revision(s) (requested by rmind in ticket #693):
	lib/npf/ext_normalise/shlib_version: revision 1.1
	lib/libnpf/npf.c: revision 1.13
	distrib/sets/lists/modules/mi: revision 1.48
	sys/net/npf/npf_rproc.c: revision 1.3
	sys/net/npf/npf_rproc.c: revision 1.4
	sys/modules/npf/Makefile: revision 1.11
	usr.sbin/npf/npfctl/npfctl.h: revision 1.20
	lib/npf/ext_log/npfext_log.c: revision 1.1
	lib/libnpf/npf.h: revision 1.11
	sys/net/npf/npf_inet.c: revision 1.17
	sys/net/npf/npf_log.c: file removal
	sys/net/npf/npf_handler.c: revision 1.22
	distrib/sets/lists/base/shl.mi: revision 1.636
	sys/net/npf/npf_impl.h: revision 1.23
	usr.sbin/npf/npfctl/Makefile: revision 1.8
	lib/npf/Makefile: revision 1.1
	lib/npf/ext_log/shlib_version: revision 1.1
	lib/Makefile: revision 1.189
	distrib/sets/lists/comp/shl.mi: revision 1.236
	usr.sbin/npf/npfctl/npf_build.c: revision 1.14
	distrib/sets/lists/base/mi: revision 1.1007
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.6
	distrib/sets/lists/base/mi: revision 1.1009
	sys/net/npf/npf.h: revision 1.21
	lib/npf/ext_normalise/npfext_normalise.c: revision 1.1
	etc/mtree/NetBSD.dist.base: revision 1.105
	lib/libnpf/Makefile: revision 1.3
	etc/mtree/NetBSD.dist.base: revision 1.106
	usr.sbin/npf/npfctl/npf_extmod.c: revision 1.1
	sys/net/npf/npf_ctl.c: revision 1.18
	lib/npf/ext_log/Makefile: revision 1.1
	distrib/sets/lists/comp/mi: revision 1.1781
	usr.sbin/npf/npfctl/npf_var.h: revision 1.4
	sys/net/npf/npf.c: revision 1.13
	sys/modules/Makefile: revision 1.111
	sys/net/npf/npf_ext_log.c: revision 1.1
	lib/npf/Makefile.inc: revision 1.1
	sys/net/npf/npf_ext_normalise.c: revision 1.1
	sys/net/npf/files.npf: revision 1.8
	sys/rump/net/lib/libnpf/Makefile: revision 1.2
	sys/modules/npf_ext_log/Makefile: revision 1.1
	lib/npf/ext_normalise/Makefile: revision 1.1
	usr.sbin/npf/npfctl/npfctl.c: revision 1.20
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.13
	sys/modules/npf_ext_normalise/Makefile: revision 1.1
Implement dynamic NPF extensions interface.  An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module.  Move normalisation and logging functionality into their own
extensions.  More improvements to come.
Add /usr/lib/npf.
Add ./usr/libdata/debug/usr/lib/npf for rmind
Fix MKDEBUG set lists
ext_ops does not change during the life cycle and can be fetched without
the mutex held. This avoids confusion in the compiler about an uninitialized
variable ext_ops.
ok rmind@

Pull up following revision(s) (requested by rmind in ticket #678):
	sys/rump/librump/rumpkern/rump.c: revision 1.243
	sys/rump/librump/rumpkern/rump.c: revision 1.244
	sys/rump/librump/rumpkern/rump.c: revision 1.245
	sys/rump/librump/rumpkern/rump.c: revision 1.246
	usr.sbin/npf/npftest/npftest.c: revision 1.5
	usr.sbin/npf/npftest/README: revision 1.2
	usr.sbin/npf/npftest/npftest.h: revision 1.5
	sys/rump/net/Makefile.rumpnetcomp: revision 1.5
	sys/rump/net/lib/libnpf/shlib_version: revision 1.1
	sys/net/npf/npf_impl.h: revision 1.22
	sys/rump/dev/lib/libnpf/Makefile: file removal
	usr.sbin/npf/npftest/Makefile: revision 1.3
	sys/rump/dev/lib/libnpf/component.c: file removal
	sys/rump/dev/lib/libnpf/shlib_version: file removal
	sys/net/npf/npf_state.c: revision 1.12
	sys/rump/net/lib/libnpf/component.c: revision 1.1
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.4
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.6
	sys/rump/net/lib/libnpf/Makefile: revision 1.1
Move and rename librumpdev_npf to librumpnet_npf.
Enable the build of librumpnet_npf.
Add npf_state_setsampler() for _NPF_TESTING case.  This also fixes the build.
Call pserialize_init() during rump start-up, since librump/net/npf
uses it.
It helps to include the declaration of the routine being called.
We also need kcpuset_init() now.
Use correct routine name - kcpuset_sysinit() vs kcpuset_init()

sync with head

Implement NPF table listing and preservation of entries on reload.
Bump the version.

Implement dynamic NPF extensions interface.  An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module.  Move normalisation and logging functionality into their own
extensions.  More improvements to come.

Add npf_state_setsampler() for _NPF_TESTING case.  This also fixes the build.

Pull up following revision(s) (requested by rmind in ticket #485):
	lib/libnpf/npf.c: revision 1.11
	sys/net/npf/npf_session.c: revision 1.17
	sys/modules/npf/Makefile: revision 1.10
	usr.sbin/npf/npftest/npftest.c: revision 1.4
	usr.sbin/npf/npftest/README: revision 1.1
	sys/net/npf/npf_tableset.c: revision 1.14
	usr.sbin/npf/npftest/npftest.h: revision 1.4
	lib/libnpf/npf.h: revision 1.10
	sys/net/npf/npf_ruleset.c: revision 1.14
	usr.sbin/npf/npfctl/npf_data.c: revision 1.18
	usr.sbin/npf/npftest/npftest.conf: revision 1.1
	sys/net/npf/npf_handler.c: revision 1.21
	sys/net/npf/npf_impl.h: revision 1.21
	usr.sbin/npf/npfctl/npfctl.c: revision 1.18
	usr.sbin/npf/npftest/libnpftest/npf_nat_test.c: revision 1.1
	usr.sbin/npf/npfctl/npf_build.c: revision 1.13
	usr.sbin/npf/npftest/libnpftest/npf_rule_test.c: revision 1.1
	usr.sbin/npf/npftest/npfstream.c: revision 1.3
	usr.sbin/npf/npftest/libnpftest/Makefile: revision 1.4
	usr.sbin/npf/npfctl/npfctl.h: revision 1.19
	sys/net/npf/npf_nat.c: revision 1.16
	sys/net/npf/npf_state.c: revision 1.11
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.3
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.5
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.12
- Extend npftest: add ruleset inspection testing from the config generated
  by npfctl debug functionality.  Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
  share between the entries and thus fix the handling of them.  Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

- Extend npftest: add ruleset inspection testing from the config generated
  by npfctl debug functionality.  Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
  share between the entries and thus fix the handling of them.  Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

Fix -fno-common found by building i386/conf/ALL

Pull up revisions:
  src/usr.sbin/npf/npfctl/npfctl.c revisions 1.16,1.17
  src/sys/net/npf/npf.h revision 1.20
  src/sys/net/npf/npf_alg_icmp.c revision 1.11
  src/sys/net/npf/npf_impl.h revision 1.19
  src/sys/net/npf/npf_inet.c revisions 1.15,1.16
  src/sys/net/npf/npf_instr.c revision 1.14
  src/sys/net/npf/npf_ncode.h revision 1.10
  src/sys/net/npf/npf_processor.c revision 1.12
  src/sys/net/npf/npf_session.c revision 1.16
  src/usr.sbin/npf/npfctl/npf_build.c revision 1.12
  src/usr.sbin/npf/npfctl/npf_data.c revisions 1.16,1.17
  src/usr.sbin/npf/npfctl/npf_disassemble.c revision 1.8
  src/usr.sbin/npf/npfctl/npf_ncgen.c revision 1.13
  src/usr.sbin/npf/npfctl/npf_parse.y revision 1.11
  src/usr.sbin/npf/npfctl/npf_scan.l revision 1.5
  src/usr.sbin/npf/npfctl/npf_var.h revision 1.3
  src/usr.sbin/npf/npfctl/npfctl.h revision 1.18
  src/sys/net/npf/npf_state.c revision 1.10
  src/sys/net/npf/npf_state_tcp.c revision 1.10
  src/usr.sbin/npf/npftest/npfstream.c revision 1.2
  src/usr.sbin/npf/npftest/libnpftest/npf_test_subr.c revision 1.2
(requested by rmind in ticket #435).

Add missing __dead.

teach npf ipv6-icmp
reviewed by rmind@

- npfctl_print_stats: beautification a la French style.
- npfctl_icmpcode: fix the build break.

- npf_fetch_tcpopts: fix off-by-one when validating TCP option length
  against the maximum allowed.
- npf_tcp_inwindow: be more liberal with npf_fetch_tcpopts().
- Few minor improvements to npftest.

teach npf ipv6-icmp
reviewed by rmind@

Pull up following revision(s) (requested by rmind in ticket #421):
	lib/libnpf/npf.c: revision 1.10
	sys/net/npf/npf_session.c: revision 1.15
	sys/net/npf/npf_tableset.c: revision 1.13
	sys/net/npf/npf_state_tcp.c: revision 1.9
	usr.sbin/npf/npfctl/npf_data.c: revision 1.15
	sys/net/npf/npf_inet.c: revision 1.14
	sys/net/npf/npf_ruleset.c: revision 1.13
	sys/net/npf/npf.h: revision 1.19
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.12
	sys/net/npf/npf_instr.c: revision 1.13
	sys/net/npf/npf_handler.c: revision 1.20
	usr.sbin/npf/npftest/libnpftest/npf_table_test.c: revision 1.4
	sys/net/npf/npf_alg_icmp.c: revision 1.10
	usr.sbin/npf/npfctl/npfctl.c: revision 1.15
	usr.sbin/npf/npfctl/npf_build.c: revision 1.11
	lib/libnpf/npf.h: revision 1.9
	sys/net/npf/npf_alg.c: revision 1.5
	sys/rump/dev/lib/libnpf/Makefile: revision 1.4
	usr.sbin/npf/npfctl/npfctl.h: revision 1.17
	sys/net/npf/npf_ctl.c: revision 1.16
	sys/net/npf/npf_nat.c: revision 1.15
	sys/net/npf/npf_tableset_ptree.c: revision 1.1
	sys/net/npf/npf.c: revision 1.12
	sys/net/npf/npf_sendpkt.c: revision 1.12
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.7
	sys/net/npf/npf_impl.h: revision 1.18
	sys/net/npf/files.npf: revision 1.7
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.10
- Rework NPF tables and fix support for IPv6.  Implement tree table type
  using radix / Patricia tree.  Universal IPv4/IPv6 comparator for ptree(3)
  was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

- Rework NPF tables and fix support for IPv6.  Implement tree table type
  using radix / Patricia tree.  Universal IPv4/IPv6 comparator for ptree(3)
  was contributed by Matt Thomas.
- NPF tables: update regression tests, improve npfctl(8) error messages.
- Fix few bugs when using kernel modules and handle module autounloader.
- Few other fixes and misc cleanups.
- Bump the version.

Pull up following revision(s) (requested by rmind in ticket #399):
	sys/net/npf/npf_session.c: revision 1.14
	sys/net/npf/npf_tableset.c: revision 1.12
	sys/net/npf/npf_state_tcp.c: revision 1.8
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.3
	usr.sbin/npf/npfctl/npf_data.c: revision 1.14
	sys/net/npf/npf_inet.c: revision 1.13
	sys/net/npf/npf_ruleset.c: revision 1.12
	sys/net/npf/npf.h: revision 1.18
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.11
	usr.sbin/npf/npfctl/npfctl.8: revision 1.7
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.9
	usr.sbin/npf/npftest/libnpftest/npf_state_test.c: revision 1.2
	usr.sbin/npf/npfctl/npfctl.8: revision 1.8
	sys/net/npf/npf_instr.c: revision 1.12
	usr.sbin/npf/npftest/libnpftest/npf_table_test.c: revision 1.3
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.13
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.14
	sys/net/npf/npf_state.c: revision 1.9
	sys/net/npf/npf_processor.c: revision 1.11
	usr.sbin/npf/npfctl/npfctl.c: revision 1.13
	usr.sbin/npf/npfctl/npfctl.c: revision 1.14
	usr.sbin/npf/npfctl/npf_build.c: revision 1.10
	lib/libnpf/npf.3: revision 1.5
	lib/libnpf/npf.h: revision 1.8
	share/man/man9/npf_ncode.9: revision 1.9
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.4
	lib/libnpf/npf.c: revision 1.9
	usr.sbin/npf/npfctl/npfctl.h: revision 1.16
	sys/net/npf/npf_nat.c: revision 1.14
	usr.sbin/npf/npftest/libnpftest/npf_processor_test.c: revision 1.2
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.6
	sys/net/npf/npf_impl.h: revision 1.17
	sys/net/npf/npf_handler.c: revision 1.18
	sys/net/npf/npf_handler.c: revision 1.19
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.4
	sys/net/npf/npf_ncode.h: revision 1.9
Fix and update npf.conf(5), npfctl(8) and its usage message.
npf_state_tcp: fix for FIN retransmission and out-of-order ACK case.
NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.
npf_packet_handler: fix gcc unused warning.

NPF improvements:
- Add NPF_OPCODE_PROTO to match the address and/or protocol only.
- Update parser to support arbitrary "pass proto <name/number>".
- Fix IPv6 address and protocol handling (add a regression test).
- Fix few theorethical races in session handling module.
- Misc fixes, simplifications and some clean up.

Pull up following revision(s) (requested by rmind in ticket #365):
	sys/rump/librump/rumpkern/rumpcpu_generic.c: revision 1.4
	sys/net/npf/npf_session.c: revision 1.13
	sys/net/npf/npf_tableset.c: revision 1.11
	sys/net/npf/npf_state_tcp.c: revision 1.7
	sys/net/npf/npf_inet.c: revision 1.12
	sys/net/npf/npf.h: revision 1.17
	sys/net/npf/npf_instr.c: revision 1.11
	usr.sbin/npf/npftest/libnpftest/npf_table_test.c: revision 1.2
	sys/net/npf/npf_state.c: revision 1.8
	sys/net/npf/npf_log.c: revision 1.4
	sys/net/npf/npf_alg.c: revision 1.4
	sys/rump/librump/rumpkern/Makefile.rumpkern: revision 1.118
	sys/net/npf/npf_nat.c: revision 1.13
	sys/net/npf/npf.c: revision 1.11
	sys/net/npf/npf_sendpkt.c: revision 1.11
	sys/net/npf/npf_impl.h: revision 1.16
	sys/rump/librump/rumpkern/scheduler.c: revision 1.28
rumpkern:
- Add subr_kcpuset.c and subr_pserialize.c modules.
- Add kcpuset_{running,attached} for RUMP env.
NPF:
- Rename some functions for consistency and de-inline them.
- Fix few invalid asserts (add regressoin test).
- Use pserialize(9) for ALG interface.
- Minor fixes, sprinkle many comments.

Pull up following revision(s) (requested by rmind in ticket #354):
	sys/net/npf/npf_state_tcp.c: revision 1.4
	sys/net/npf/npf_state_tcp.c: revision 1.5
	sys/net/npf/npf_state_tcp.c: revision 1.6
	usr.sbin/npf/npftest/npftest.c: revision 1.1
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.1
	usr.sbin/npf/npftest/npftest.c: revision 1.2
	usr.sbin/npf/npftest/libnpftest/npf_mbuf_subr.c: revision 1.2
	usr.sbin/npf/npfctl/npf_data.c: revision 1.11
	usr.sbin/npf/npftest/npftest.c: revision 1.3
	usr.sbin/npf/npfctl/npf_data.c: revision 1.12
	usr.sbin/npf/npftest/npftest.h: revision 1.1
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.5
	usr.sbin/npf/npfctl/npf_data.c: revision 1.13
	sys/net/npf/npf.h: revision 1.16
	usr.sbin/npf/npftest/npftest.h: revision 1.2
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.6
	usr.sbin/npf/npftest/npftest.h: revision 1.3
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.7
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.10
	usr.sbin/npf/npfctl/npf_build.c: revision 1.6
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.8
	usr.sbin/npf/npfctl/npf_build.c: revision 1.7
	usr.sbin/npf/npftest/libnpftest/npf_state_test.c: revision 1.1
	usr.sbin/npf/npftest/libnpftest/npf_nbuf_test.c: revision 1.1
	usr.sbin/npf/npfctl/npf_build.c: revision 1.8
	usr.sbin/npf/npftest/libnpftest/npf_table_test.c: revision 1.1
	usr.sbin/npf/npfctl/npf_build.c: revision 1.9
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.10
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.11
	usr.sbin/npf/npfctl/npf.conf.5: revision 1.12
	sys/net/npf/npf_state.c: revision 1.7
	usr.sbin/npf/npfctl/npfctl.c: revision 1.11
	usr.sbin/npf/npfctl/npfctl.c: revision 1.12
	usr.sbin/npf/npfctl/Makefile: revision 1.7
	sys/rump/net/lib/libnet/Makefile: revision 1.14
	sys/net/npf/npf_mbuf.c: revision 1.7
	usr.sbin/npf/npftest/Makefile: revision 1.1
	usr.sbin/npf/npftest/Makefile: revision 1.2
	usr.sbin/npf/npftest/libnpftest/Makefile: revision 1.1
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.2
	usr.sbin/npf/npftest/npfstream.c: revision 1.1
	usr.sbin/npf/npftest/libnpftest/Makefile: revision 1.2
	usr.sbin/npf/npfctl/npf_scan.l: revision 1.3
	usr.sbin/npf/npftest/libnpftest/Makefile: revision 1.3
	usr.sbin/npf/npfctl/npfctl.h: revision 1.12
	sys/rump/dev/lib/libnpf/Makefile: revision 1.2
	usr.sbin/npf/npfctl/npfctl.h: revision 1.14
	sys/rump/dev/lib/libnpf/Makefile: revision 1.3
	usr.sbin/npf/npfctl/npfctl.h: revision 1.15
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.9
	sys/net/npf/npf_ctl.c: revision 1.15
	usr.sbin/npf/npfctl/npf_var.c: revision 1.4
	usr.sbin/npf/npfctl/npf_var.h: revision 1.2
	usr.sbin/npf/npfctl/npf_var.c: revision 1.5
	sys/net/npf/npf_impl.h: revision 1.13
	sys/net/npf/npf_sendpkt.c: revision 1.10
	sys/net/npf/npf_impl.h: revision 1.14
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.4
	sys/net/npf/npf_impl.h: revision 1.15
	sys/net/npf/npf_handler.c: revision 1.16
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.1
	usr.sbin/npf/npftest/libnpftest/npf_processor_test.c: revision 1.1
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.5
	sys/net/npf/npf_handler.c: revision 1.17
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.2
	sys/net/npf/npf_ncode.h: revision 1.7
	usr.sbin/npf/npftest/libnpftest/npf_test_subr.c: revision 1.1
	usr.sbin/npf/npftest/libnpftest/npf_test.h: revision 1.3
	sys/net/npf/npf_ncode.h: revision 1.8
npf_tcp_inwindow: in a case of negative skew, bump the maximum seen value of
SEQ+LEN in the receiver's side correctly (using ACK from the sender's side).
PR/46265 from Changli Gao.
rumpnet_net: add pfil.c
Update rumpdev_npf; use WARNS=4.
Add initial NPF regression tests integrated with RUMP framework (running the
kernel part of NPF in userland).  Other tests will be added once converted to
RUMP framework.  All tests are in the public domain.
Some Makefile fixes from christos@.
- Fix double-free case on ICMP return case.
- npf_pfil_register: handle kernels without INET6 option correctly.
- Reduce some #ifdefs.
npfctl(8): add show-config command.  Also, update syntax.
npftest: add a stream processor, which prints out the TCP state information.
A tool for debugging connection tracking from tcpdump -w captured data.
npftest: add a module for TCP state tracking and add few test cases.
npf_state_tcp: add an assert; fix some comments while here.
- Rework NPF NAT syntax to be more structured and support future additions
  of different types and configurations of NAT.
- npfctl: improve disassemble and show-config command functionality.
- Fix custom ICMP code and type filtering.
make this compile again.
remove error(1) output
Remove superfluous Pp
- make each element of a variable hold a type
- change get_type to take an index, so we can get the individual types of
  each element (since primitive elements can be in lists)
- make port_range primitive
- add a routine to convert a variable of primitives to a variable containing
- only port ranges.
remove extra rule that got merged...

NPF:
- Rename some functions for consistency and de-inline them.
- Fix few invalid asserts (add regressoin test).
- Use pserialize(9) for ALG interface.
- Minor fixes, sprinkle many comments.

sync to latest -current.

npftest: add a stream processor, which prints out the TCP state information.
A tool for debugging connection tracking from tcpdump -w captured data.

sync with head.

- Fix double-free case on ICMP return case.
- npf_pfil_register: handle kernels without INET6 option correctly.
- Reduce some #ifdefs.

sync to latest -current.

sync with head

Update rumpdev_npf; use WARNS=4.

sync to latest -current.

Pull up following revision(s) (requested by rmind in ticket #158):
	sys/net/npf/npf_session.c: revision 1.12
	sys/net/npf/npf_tableset.c: revision 1.10
	sys/net/npf/npf_rproc.c: revision 1.2
	usr.sbin/npf/npfctl/npf_parse.y: revision 1.4
	sys/net/npf/npf_inet.c: revision 1.11
	sys/net/npf/npf.h: revision 1.15
	usr.sbin/npf/npfctl/npf_build.c: revision 1.5
	sys/net/npf/npf_ruleset.c: revision 1.11
	sys/net/npf/npf_instr.c: revision 1.10
	usr.sbin/npf/npfctl/Makefile: revision 1.6
	sys/net/npf/npf_processor.c: revision 1.10
	sys/net/npf/npf_log.c: revision 1.3
	lib/libnpf/npf.h: revision 1.7
	sys/net/npf/npf_alg.c: revision 1.3
	sys/net/npf/npf_sendpkt.c: revision 1.9
	lib/libnpf/npf.c: revision 1.8
	usr.sbin/npf/npfctl/npfctl.h: revision 1.13
	sys/net/npf/npf_ctl.c: revision 1.13
	usr.sbin/npf/npfctl/npf_ncgen.c: revision 1.8
	sys/net/npf/npf_ctl.c: revision 1.14
	sys/net/npf/npf_nat.c: revision 1.11
	sys/net/npf/npf_nat.c: revision 1.12
	sys/net/npf/npf_impl.h: revision 1.11
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.1
	sys/net/npf/npf_impl.h: revision 1.12
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.2
	sys/net/npf/npf_handler.c: revision 1.14
	usr.sbin/npf/npfctl/npf_disassemble.c: revision 1.3
	sys/net/npf/npf_handler.c: revision 1.15
	sys/net/npf/npf_ncode.h: revision 1.6
	sys/net/npf/npf.c: revision 1.8
	sys/net/npf/npf.c: revision 1.9
	sys/net/npf/npf_alg_icmp.c: revision 1.9
	sys/net/npf/npf_session.c: revision 1.11
- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS.  Be more defensive in the
  packet handler.  Change the default policy to block when the config is
  loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
  of rule procedure might happen in the interrupt handler (under a very rare
  condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.
Simplify slightly: merge iface into addr_or_iface, use it in filt_addr.
Add a small disassembler.
definitions used by the disassembler.
- better printing of type/code flags/mask
- pass the instruction start pointer, instead of subtracting 1 to account for it
- Save active config in proplib dictionary; add GETCONF ioctl to retrieve.
- Few fixes.  Improve some comments.
don't leak the branch target array.
Add NPF config retrieval routines.

- Save active config in proplib dictionary; add GETCONF ioctl to retrieve.
- Few fixes.  Improve some comments.

sync to -current.

- Add NPF_DECISION_BLOCK and NPF_DECISION_PASS.  Be more defensive in the
  packet handler.  Change the default policy to block when the config is
  loaded and set it to pass when flush operation is performed.
- Use kmem_zalloc(9) instead of kmem_alloc(9) in few places.
- npf_rproc_{create,release}: use kmem_intr_{alloc,free} as the destruction
  of rule procedure might happen in the interrupt handler (under a very rare
  condition, if config reload races with the handler).
- npf_session_establish: check whether layer 3 and 4 are cached.
- npfctl_build_group: do not make groups as passing rules.
- Remove some unecessary header inclusion.

merge to -current.

- Split NPF rule procedure code into a separate module (no functional changes).
- Simplify some code, add more comments, some asserts.
- G/C unused rule hook code.

- Rework and improve TCP state tracking.
- Fix regressions after IPv6 patch merge.

Note: npfctl(8) rework will come soon.

sync with head

Add IPv6 support for NPF.

Sync with HEAD.

sync with head

Sync with HEAD

file npf_impl.h was added on branch rmind-uvmplock on 2011-03-05 20:55:55 +0000

NPF checkpoint:
- Add libnpf(3) - a library to control NPF (configuration, ruleset, etc).
- Add NPF support for ftp-proxy(8).
- Add rc.d script for NPF.
- Convert npfctl(8) to use libnpf(3) and thus make it less depressive.
  Note: next clean-up step should be a parser, once dholland@ will finish it.
- Add more documentation.
- Various fixes.

NPF checkpoint:
- Add the concept of rule procedure: separate normalization, logging and
  potentially other functions from the rule structure.  Rule procedure can be
  shared amongst the rules.  Separation is both at kernel level (npf_rproc_t)
  and configuration ("procedure" + "apply").
- Fix portmap sharing for NAT policy.
- Update TCP state tracking logic.  Use TCP FSM definitions.
- Add if_byindex(), OK by matt@.  Use in logging for the lookup.
- Fix traceroute ALG and many other bugs; misc clean-up.

NPF checkpoint:
- Add support for session saving/restoring.
- Add packet logging support (can tcpdump a pseudo-interface).
- Support reload without flushing of sessions; rework some locking.
- Revisit session mangement, replace linking with npf_sentry_t entries.
- Add some counters for statistics, using percpu(9).
- Add IP_DF flag cleansing.
- Fix various bugs; misc clean-up.

NPF checkpoint:
- Add proper TCP state tracking as described in Guido van Rooij paper,
  plus handle TCP Window Scaling option.
- Completely rework npf_cache_t, reduce granularity, simplify code.
- Add npf_addr_t as an abstraction, amend session handling code, as well
  as NAT code et al, to use it.  Now design is prepared for IPv6 support.
- Handle IPv4 fragments i.e. perform packet reassembly.
- Add support for IPv4 ID randomization and minimum TTL enforcement.
- Add support for TCP MSS "clamping".
- Random bits for IPv6.  Various fixes and clean-up.

Sync with HEAD (-D20101022).

sync with head

file npf_impl.h was added on branch uebayasi-xip on 2010-10-22 09:23:14 +0000

file npf_impl.h was added on branch yamt-nfs-mp on 2010-10-09 03:32:37 +0000

Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h).  Also
helps find code that hasn't been updated to use the new rbtree API.

NPF checkpoint:
- Add support for bi-directional NAT and redirection / port forwarding.
- Finish filtering on ICMP type/code and add filtering on TCP flags.
- Add support for TCP reset (RST) or ICMP destination unreachable on block.
- Fix a bunch of bugs; misc cleanup.

Import NPF - a packet filter.  Some features:

- Designed to be fully MP-safe and highly efficient.

- Tables/IP sets (hash or red-black tree) for high performance lookups.

- Stateful filtering and Network Address Port Translation (NAPT).
  Framework for application level gateways (ALGs).

- Packet inspection engine called n-code processor - inspired by BPF -
  supporting generic RISC-like and specific CISC-like instructions for
  common patterns (e.g. IPv4 address matching).  See npf_ncode(9) manual.

- Convenient userland utility npfctl(8) with npf.conf(8).

NOTE: This is not yet a fully capable alternative to PF or IPFilter.
Further work (support for binat/rdr, return-rst/return-icmp, common ALGs,
state saving/restoring, logging, etc) is in progress.

Thanks a lot to Matt Thomas for various useful comments and code review.
Aye by: board@