CVS log for src/sys/dev/pci/ixgbe/ixv.c

Up to [cvs.NetBSD.org] / src / sys / dev / pci / ixgbe

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

ixgbe: Remove ifdef NET_MPSAFE. Make ixgbe MP-scalable by default.

 Same as wm(4).

ixgbe: ixgbe is always MP-safe, so remove IXGBE_MPSAFE macro.

 Before this change, IXGBE_MPSAFE is not defined if NET_MPSAFE is not set.
In this case, the PCI_INTR_MPSAFE flag is not set by pci_intr_setattr()
and interrupt_distribute() fails on arm. Always enable IXGBE_MPSAFE
funciton by removing the macro.

Pull up the following, requested by msaitoh in ticket #1933:

	sys/dev/pci/ixgbe/ix_txrx.c			1.105-1.116 via patch
	sys/dev/pci/ixgbe/ixgbe.c			1.345-1.346,1.349 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.94-1.98
	sys/dev/pci/ixgbe/ixgbe_type.h			1.62
	sys/dev/pci/ixgbe/ixv.c				1.193,1.195-1.196

- Clear the WTHRESH bit field before writing it.
- Optimize ixgbe_txeof().
- Use kmem_zalloc() instead of malloc(,M_ZERO).
- Add QPRDC(Queue Packet Receive Drop Count) into iqdrops.
- No functional change
  - Move assignment of TXD.
  - ixv(4): Remove unused IFF_OACTIVE.
  - Don't include the Flow Director related members to reduce the
    size of struct tx_ring. On amd64 and aarch64, the real size is
    not changed because of the alignment.
  - The descriptor ring size and the alignment are tested in the
    attach function, so it's not required to use
    roundup2(size, DBA_ALIGN).
  - Use #ifdef LRO more to reduce the size of struct rx_ring.
  - Change "me" from 32bit to 8bit because the max is 128.
    This commit doesn't change the real size of ix_queue, tx_ring
    and rx_ring because of the alignment.
  - Th RSC (hardware receive side coalescing) feature has been
    disabled all along, so enclose the code with #ifdef RSC.
  - Remove unused.
  - Modify for the readability.
  - Modify comment.
  - Fix comment. Whitespace.

Pull up the following, requested by msaitoh in ticket #1792:

	sys/dev/pci/ixgbe/ix_txrx.c			1.105-1.116 via patch
	sys/dev/pci/ixgbe/ixgbe.c			1.345-1.346,1.349 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.94-1.98
	sys/dev/pci/ixgbe/ixgbe_type.h			1.62
	sys/dev/pci/ixgbe/ixv.c				1.193,1.195-1.196

- Clear the WTHRESH bit field before writing it.
- Optimize ixgbe_txeof().
- Use kmem_zalloc() instead of malloc(,M_ZERO).
- Add QPRDC(Queue Packet Receive Drop Count) into iqdrops.
- No functional change
  - Move assignment of TXD.
  - ixv(4): Remove unused IFF_OACTIVE.
  - Don't include the Flow Director related members to reduce the
    size of struct tx_ring. On amd64 and aarch64, the real size is
    not changed because of the alignment.
  - The descriptor ring size and the alignment are tested in the
    attach function, so it's not required to use
    roundup2(size, DBA_ALIGN).
  - Use #ifdef LRO more to reduce the size of struct rx_ring.
  - Change "me" from 32bit to 8bit because the max is 128.
    This commit doesn't change the real size of ix_queue, tx_ring
    and rx_ring because of the alignment.
  - Th RSC (hardware receive side coalescing) feature has been
    disabled all along, so enclose the code with #ifdef RSC.
  - Remove unused.
  - Modify for the readability.
  - Modify comment.
  - Fix comment. Whitespace.

Pull up following revision(s) (requested by msaitoh in ticket #563):

	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.110
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.345
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.111
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.346
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.112
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.62
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.113
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.348
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.114
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.115
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.116
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.105
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.106
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.107
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.108
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.109
	sys/dev/pci/ixgbe/ixv.c: revision 1.193
	sys/dev/pci/ixgbe/ixv.c: revision 1.195
	sys/dev/pci/ixgbe/ixv.c: revision 1.196
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.94
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.95
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.96
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.97
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.98

ixgbe: Fix comment. No functional change.
ixgbe: Whitespace. No functional change.
ixgbe(4): Move assignment of TXD. NFCI.
ixgbe(4): Modify comment. No functional change.
 ixgbe_tx_ctx_setup() may or may not consume one TX descriptor.
ixv(4): Remove unused IFF_OACTIVE. No functional change.
ixgbe: Clear the WTHRESH bit field before writing it.
ixgbe: Modify for the readability. No functional change.
ixgbe: micro-optimize ixgbe_txeof()
 Update txr->tx_avail and txr->txr_no_space outside the loop in ixgbe_txeof().
ixgbe: Update if_opackets outside the loop in ixgbe_txeof().
ixgbe: micro-optimize ixgbe_txeof()
 Update txr->packets outside the loop in ixgbe_txeof().
ixgbe: Use #ifdef IXGBE_FDIR more
 Don't include the Flow Director related members to reduce the size of
struct tx_ring. On amd64 and aarch64, the real size is not changed
because of the alignment.
ixgbe: Simplify. No functional change.
 The descriptor ring size and the alignment are tested in the attach
function, so it's not required to use roundup2(size, DBA_ALIGN).
ixgbe: Use kmem_zalloc() instead of malloc(,M_ZERO).
ixgbe: Remove unused to reduce the size of struct rx_ring.
ixgbe: Use #ifdef LRO more to reduce the size of struct rx_ring.
ixgbe: Change "me" from 32bit to 8bit because the max is 128.
 This commit doesn't change the real size of ix_queue, tx_ring and rx_ring
because of the alignment.
ixgbe: Use #ifdef RSC
 This feature (hardware receive side coalescing) has been disabled all along,
so enclose the code with #ifdef RSC.

ixgbe: Clear the WTHRESH bit field before writing it.

ixv(4): Remove unused IFF_OACTIVE. No functional change.

Pull up following revision(s) (requested by yamaguchi in ticket #455):
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.347
	sys/net/if_l2tp.c: revision 1.49
	tests/net/if_vlan/t_vlan.sh: revision 1.25
	sys/net/if_vlan.c: revision 1.171
	sys/net/if_ethersubr.c: revision 1.326
	sys/dev/pci/ixgbe/ixv.c: revision 1.194
Use ether_bpf_mtap only when the device supports vlan harware tagging
The function is bpf_mtap() for ethernet devices and *currently*
it is just handling VLAN tag stripped by the hardware.
l2tp(4): use ether_ifattach() to initialize ethercom
Support vlan(4) over l2tp(4)
Added the test for vlan over l2tp

Use ether_bpf_mtap only when the device supports vlan harware tagging

The function is bpf_mtap() for ethernet devices and *currently*
it is just handling VLAN tag stripped by the hardware.

ixgbe: Whitespace. No functional change.

Pull up the following, requested by msaitoh in ticket #1914:

	sys/dev/pci/ixgbe/ix_txrx.c			1.103-1.104
	sys/dev/pci/ixgbe/ixgbe.c			1.334-1.338,
							1.341-1.344 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.90-1.93
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.31-1.32
	sys/dev/pci/ixgbe/ixgbe_api.c			1.29
	sys/dev/pci/ixgbe/ixgbe_bypass.h		1.5
	sys/dev/pci/ixgbe/ixgbe_common.c		1.46-1.47
	sys/dev/pci/ixgbe/ixgbe_common.h		1.18
	sys/dev/pci/ixgbe/ixgbe_dcb.c			1.14-1.15
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c		1.13
	sys/dev/pci/ixgbe/ixgbe_type.h			1.59-1.61
	sys/dev/pci/ixgbe/ixgbe_x540.c			1.24
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.28
	sys/dev/pci/ixgbe/ixv.c				1.187-1.192 via patch

- ixg(4): Add 82599 LS support once again.
- ixg(4): Filter out spurious link up indication more.
- ixg(4): Print DEVICE_CAPS register.
- ixg(4): Fix a bug that the number of queues is unintentionally
  limited to a small number or wrong error message may be printed
  when two devices' number of MSI-X vectors are different.
- Modify error message of wrong TX/RX descriptor size.
- Enable interrupt after setting IFF_RUNNING.
- Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change
  all devices all queues default interrupt rate.
- Cleanup the code.

Pull up the following, requested by msaitoh in ticket #1753:

	sys/dev/pci/ixgbe/ix_txrx.c			1.103-1.104
	sys/dev/pci/ixgbe/ixgbe.c			1.334-1.338,
							1.341-1.344 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.90-1.93
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.31-1.32
	sys/dev/pci/ixgbe/ixgbe_api.c			1.29
	sys/dev/pci/ixgbe/ixgbe_bypass.h		1.5
	sys/dev/pci/ixgbe/ixgbe_common.c		1.46-1.47
	sys/dev/pci/ixgbe/ixgbe_common.h		1.18
	sys/dev/pci/ixgbe/ixgbe_dcb.c			1.14-1.15
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c		1.13
	sys/dev/pci/ixgbe/ixgbe_type.h			1.59-1.61
	sys/dev/pci/ixgbe/ixgbe_x540.c			1.24
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.28
	sys/dev/pci/ixgbe/ixv.c				1.187-1.192

- ixg(4): Add 82599 LS support once again.
- ixg(4): Filter out spurious link up indication more.
- ixg(4): Print DEVICE_CAPS register.
- ixg(4): Fix a bug that the number of queues is unintentionally
  limited to a small number or wrong error message may be printed
  when two devices' number of MSI-X vectors are different.
- Modify error message of wrong TX/RX descriptor size.
- Enable interrupt after setting IFF_RUNNING.
- Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change
  all devices all queues default interrupt rate.
- Cleanup the code.

Pull up following revision(s) (requested by msaitoh in ticket #420):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.340
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.341
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.342
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.343
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.344
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.61
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.104
	sys/dev/pci/ixgbe/ixv.c: revision 1.190
	sys/dev/pci/ixgbe/ixv.c: revision 1.191
	sys/dev/pci/ixgbe/ixv.c: revision 1.192
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.93

ixg(4): Print DEVICE_CAPS register.

ixgbe: Whitespace. No functional change.

ixg(4): Don't print wrong error message about ixgbe_num_queues.
 Don't override the ixgbe_num_queues global variable. It's the default
value of the number of queues and should not override it because it
will be referenced by later device attach. For example, the number of
MSI-X vector is 64 on X540 and 18 on 82599. When both cards are inserted
to a machine that the number of CPU is 24 and X540 is probed earlier,
ixgbe_num_queues is overridden to 24 and the following error message is
printed when attaching 82599:
	ixg2: autoconfiguration error: ixgbe_num_queues (24) is too large,
	using reduced amount (17).

Note that the number of queues is in sc->num_queuss and referenced
by hw.ixgN.num_queues sysctl.

ixgbe: Don't override the {ixgbe,ixv}_max_interrupt_rate global variable.

 Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change all
devices all queues default interrupt rate.

ixgbe: Whitespace. No functional change.

ixgbe: Whitespace. No functional change.

Pull up following revision(s) (requested by msaitoh in ticket #407):

	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.47
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.28
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.60
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.31
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.32
	sys/dev/pci/ixgbe/ixv.c: revision 1.187
	sys/dev/pci/ixgbe/ixv.c: revision 1.188
	sys/dev/pci/ixgbe/ixv.c: revision 1.189
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.18
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.24
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.334
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.335
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.336
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.337
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.103
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.338
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.59
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_dcb.c: revision 1.14
	sys/dev/pci/ixgbe/ixgbe_dcb.c: revision 1.15
	sys/dev/pci/ixgbe/ixgbe_api.c: revision 1.29
	sys/dev/pci/ixgbe/ixgbe_bypass.h: revision 1.5
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.90
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.91
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.92
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.46

ixg(4): Remove unused and old function. No functional change.
 - From FreeBSD's ix-3.3.32.

ixg(4): Add 82599 LS once again.
 - From:
        FreeBSD: 9228ac3a69c4c7401a743e6465e118101a2beeb0
        DPDK:    549ccd3dc01539e060597b503f2b65b272de3347
 - This was removed 5 years ago. From the commit message:
   > Remove IXGBE_DEV_ID_82599_LS(0x154f) support again. I don't know why. This
   > was added in ix-3.2.18.tar.gz(NetBSD: ixgbe_82599.c rev. 1.20) and
   > removed in ix-3.3.6.tar.gz.

ixg(4): Filter out spurious link up indication
 - Extend SFP+ cage crosstalk fix by re-checking link state after 5ms delay
   to filter out spurious link up indication by transceiver with no fiber
   cable connected.
 - From FreeBSD:
     In-tree: 04a5e6d7cadd06b10169c3c3a560649e7dc7444c
     Out of tree: ix-3.3.33.

ixgbe: Simplify definitions. No functional change.
 Both DEFAULT_{TX,RX}D and PERFORM_{TX,RX}D are 2048. Use DEFAULT_{TX,RX}D.
Same as FreeBSD.

ixgbe: Modify error message of wrong TX/RX descriptor size.
  - Based from FreeBSD ix-3.3.35. I think ix-3.3.35's RING_INCREMENT(== 32)
    is wrong. It should be 8(DBA_ALIGN / sizeof(union ixgbe_adv_[tr]x_desc)).
    Linux also uses 8.

ixgbe: Remove NO_82599_SUPPORT and NO_X540_SUPPORT support. NFCI.
 From FreeBSD ix-3.3.35. Note that this file is not used in NetBSD.

ixgbe: Enable interrupt after setting IFF_RUNNING. Same as FreeBSD x-3.3.35.

ixgbe: Cleanup. No fucntional change.

 Remove unused code, fix whitespace and modify comment to reduce against
FreeBSD(mainly from ix-3.3.35).

Pull up following revision(s) (requested by msaitoh in ticket #1912):

	sys/dev/pci/ixgbe/if_bypass.c		1.10
	sys/dev/pci/ixgbe/if_fdir.c		1.6 via patch
	sys/dev/pci/ixgbe/if_sriov.c		1.18 via patch
	sys/dev/pci/ixgbe/ix_txrx.c		1.102 via patch
	sys/dev/pci/ixgbe/ixgbe.c		1.333,1.339 via patch
	sys/dev/pci/ixgbe/ixgbe.h		1.89
	sys/dev/pci/ixgbe/ixgbe_api.h		1.17
	sys/dev/pci/ixgbe/ixgbe_common.h	1.17
	sys/dev/pci/ixgbe/ixgbe_bypass.h	1.4
	sys/dev/pci/ixgbe/ixgbe_common.c	1.45
	sys/dev/pci/ixgbe/ixgbe_fdir.h		1.5
	sys/dev/pci/ixgbe/ixgbe_netmap.h	1.3
	sys/dev/pci/ixgbe/ixgbe_netmap.c	1.6
	sys/dev/pci/ixgbe/ixgbe_osdep.c		1.9
	sys/dev/pci/ixgbe/ixgbe_sriov.h		1.6
	sys/dev/pci/ixgbe/ixgbe_type.h		1.58
	sys/dev/pci/ixgbe/ixgbe_x550.c		1.27
	sys/dev/pci/ixgbe/ixv.c			1.186 via patch

ixgbe: Rename some definitions, modify comment. No functional change.
 Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
    - struct adapter *adapter -> struct ixgbe_softc *sc
    - master -> primary
    - black -> block

ixg(4): Whitespace. No functional change.

Pull up following revision(s) (requested by msaitoh in ticket #1749):

	sys/dev/pci/ixgbe/if_bypass.c		1.10
	sys/dev/pci/ixgbe/if_fdir.c		1.6 via patch
	sys/dev/pci/ixgbe/if_sriov.c		1.18 via patch
	sys/dev/pci/ixgbe/ix_txrx.c		1.102 via patch
	sys/dev/pci/ixgbe/ixgbe.c		1.333,1.339 via patch
	sys/dev/pci/ixgbe/ixgbe.h		1.89
	sys/dev/pci/ixgbe/ixgbe_api.h		1.17
	sys/dev/pci/ixgbe/ixgbe_common.h	1.17
	sys/dev/pci/ixgbe/ixgbe_bypass.h	1.4
	sys/dev/pci/ixgbe/ixgbe_common.c	1.45
	sys/dev/pci/ixgbe/ixgbe_fdir.h		1.5
	sys/dev/pci/ixgbe/ixgbe_netmap.h	1.3
	sys/dev/pci/ixgbe/ixgbe_netmap.c	1.6
	sys/dev/pci/ixgbe/ixgbe_osdep.c		1.9
	sys/dev/pci/ixgbe/ixgbe_sriov.h		1.6
	sys/dev/pci/ixgbe/ixgbe_type.h		1.58
	sys/dev/pci/ixgbe/ixgbe_x550.c		1.27
	sys/dev/pci/ixgbe/ixv.c			1.186 via patch

ixgbe: Rename some definitions, modify comment. No functional change.
 Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
    - struct adapter *adapter -> struct ixgbe_softc *sc
    - master -> primary
    - black -> block

ixg(4): Whitespace. No functional change.

Pull up following revision(s) (requested by msaitoh in ticket #405):

	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.27
	sys/dev/pci/ixgbe/ixgbe_fdir.h: revision 1.5
	sys/dev/pci/ixgbe/ixv.c: revision 1.186
	sys/dev/pci/ixgbe/ixgbe_osdep.c: revision 1.9
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.17
	sys/dev/pci/ixgbe/ixgbe_api.h: revision 1.17
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.89
	sys/dev/pci/ixgbe/if_fdir.c: revision 1.6
	sys/dev/pci/ixgbe/if_sriov.c: revision 1.18
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.333
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.102
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.339
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.58
	sys/dev/pci/ixgbe/ixgbe_sriov.h: revision 1.6
	sys/dev/pci/ixgbe/if_bypass.c: revision 1.10
	sys/dev/pci/ixgbe/ixgbe_bypass.h: revision 1.4
	sys/dev/pci/ixgbe/ixgbe_netmap.h: revision 1.3
	sys/dev/pci/ixgbe/ixgbe_netmap.c: revision 1.6
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.45

ixgbe: Rename some definitions, modify comment. No functional change.
 Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
    - struct adapter *adapter -> struct ixgbe_softc *sc
    - master -> primary
    - black -> block

ixg(4): Whitespace. No functional change.

ixgbe: Don't override the {ixgbe,ixv}_max_interrupt_rate global variable.

 Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change all
devices all queues default interrupt rate.

ixgbe: Whitespace. No functional change.

Pull up following revision(s) (requested by msaitoh in ticket #1745):

	sys/dev/pci/ixgbe/ixgbe.c		1.327-1.332 via patch
	sys/dev/pci/ixgbe/ixgbe.h		1.87-1.88
	sys/dev/pci/ixgbe/ixv.c 		1.184-1.185
	sys/dev/pci/ixgbe/ix_txrx.c		1.101
	sys/dev/pci/ixgbe/ixgbe_82599.c 	1.30
	sys/dev/pci/ixgbe/ixgbe_vf.c		1.32-1.33
	sys/dev/pci/ixgbe/ixgbe_vf.h		1.18
	sys/dev/pci/ixgbe/ixgbe_mbx.h		1.20
	sys/dev/pci/ixgbe/ixgbe_type.h		1.57

- Reorder some event counters for readability.
- Rename some descriptions of event counters.
- Count Queue Bytes {Transmit, Receive} counter.
- Improve error check in ixgbe_check_mac_link_vf().
- Add new IXGBE_VF_GET_LINK_STATE message support.
  The VF's link state can be forced to down by PF.
- Update FCTRL after writing multicast filter.
- Update comments.

Pull up following revision(s) (requested by msaitoh in ticket #1745):

	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.30
	sys/dev/pci/ixgbe/ixv.c: revision 1.184
	sys/dev/pci/ixgbe/ixv.c: revision 1.185
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.32
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.33
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.87
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.88
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.330
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.331
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.332
	sys/dev/pci/ixgbe/ixgbe_vf.h: revision 1.18
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.101
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.57
	sys/dev/pci/ixgbe/ixgbe_mbx.h: revision 1.20
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.327
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.328
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.329

ixgbe: Reorder some event counters for readability.

ixg(4): Rename some descriptions of event counters.
 - Rename some descriptions from register name to the meaning.
 - For the same meaning's counters, add "(soft)" or "(reg)".
   The former is for a software level counter and the latter is for a
   statistics counter register based.

ixg(4): Count Queue Bytes {Transmit, Receive} counter.

ixg(4): Reorder some flow control related event counters for readability.

ixg(4): Rename some descriptions of flow control related event conters.
Remove obsolete comment.

ixgbe: Fix typo in comment. No functional change.

ixv(4): Improve error check.
 ixgbe_vf.c rev. 1.31 changed the behavior of the ixgbe_check_mac_link_vf()
function. It was from FreeBSD's ixv-1.5.25 to resolve mailbox collision
problem. The change had a problem that error checks have not done at all if
the API version >= 1.5. Fix it. From FreeBSD ixv-1.5.27.

ixv(4): Add new IXGBE_VF_GET_LINK_STATE message support.
 PF can control vf's link state by this change. Note that Linux's PF driver
can't control the link to force up (i.e. ip link set XXX vf Y state enable).
From FreeBSD ixv-1.5.30.

ixg(4): Update FCTRL after writing multicast filter. Same as other OSes.
 From FreeBSD 395cc55d896654b8f75071e71e856b22aed87da5.

Pull up following revision(s) (requested by msaitoh in ticket #395):

	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.30
	sys/dev/pci/ixgbe/ixv.c: revision 1.184
	sys/dev/pci/ixgbe/ixv.c: revision 1.185
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.32
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.33
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.87
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.88
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.330
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.331
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.332
	sys/dev/pci/ixgbe/ixgbe_vf.h: revision 1.18
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.101
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.57
	sys/dev/pci/ixgbe/ixgbe_mbx.h: revision 1.20
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.327
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.328
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.329

ixgbe: Reorder some event counters for readability.

ixg(4): Rename some descriptions of event counters.
 - Rename some descriptions from register name to the meaning.
 - For the same meaning's counters, add "(soft)" or "(reg)".
   The former is for a software level counter and the latter is for a
   statistics counter register based.

ixg(4): Count Queue Bytes {Transmit, Receive} counter.

ixg(4): Reorder some flow control related event counters for readability.

ixg(4): Rename some descriptions of flow control related event conters.
Remove obsolete comment.

ixgbe: Fix typo in comment. No functional change.

ixv(4): Improve error check.
 ixgbe_vf.c rev. 1.31 changed the behavior of the ixgbe_check_mac_link_vf()
function. It was from FreeBSD's ixv-1.5.25 to resolve mailbox collision
problem. The change had a problem that error checks have not done at all if
the API version >= 1.5. Fix it. From FreeBSD ixv-1.5.27.

ixv(4): Add new IXGBE_VF_GET_LINK_STATE message support.
 PF can control vf's link state by this change. Note that Linux's PF driver
can't control the link to force up (i.e. ip link set XXX vf Y state enable).
From FreeBSD ixv-1.5.30.

ixg(4): Update FCTRL after writing multicast filter. Same as other OSes.
 From FreeBSD 395cc55d896654b8f75071e71e856b22aed87da5.

ixgbe: Enable interrupt after setting IFF_RUNNING. Same as FreeBSD x-3.3.35.

ixgbe: Modify error message of wrong TX/RX descriptor size.

  - Based from FreeBSD ix-3.3.35. I think ix-3.3.35's RING_INCREMENT(== 32)
    is wrong. It should be 8(DBA_ALIGN / sizeof(union ixgbe_adv_[tr]x_desc)).
    Linux also uses 8.

ixgbe: Simplify definitions. No functional change.

 Both DEFAULT_{TX,RX}D and PERFORM_{TX,RX}D are 2048. Use DEFAULT_{TX,RX}D.
Same as FreeBSD.

ixgbe: Rename some definitions, modify comment. No functional change.

 Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
    - struct adapter *adapter -> struct ixgbe_softc *sc
    - master -> primary
    - black -> block

ixv(4): Add new IXGBE_VF_GET_LINK_STATE message support.

 PF can control vf's link state by this change. Note that Linux's PF driver
can't control the link to force up (i.e. ip link set XXX vf Y state enable).
From FreeBSD ixv-1.5.30.

ixgbe: Reorder some event counters for readability.

Pull up the following revisions, requested by msaitoh in ticket #1796:

	sys/dev/pci/files.pci				1.442
	sys/dev/pci/ixgbe/ix_txrx.c			1.99-1.100
	sys/dev/pci/ixgbe/ixgbe.c			1.320-1.324 via patch
	sys/dev/pci/ixgbe/ixgbe_82598.c 		1.19
	sys/dev/pci/ixgbe/ixgbe_api.c			1.28
	sys/dev/pci/ixgbe/ixgbe_common.c		1.43
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.17
	sys/dev/pci/ixgbe/ixv.c 			1.183

- Add an option for Tx to use deferred softint regardless of whether
  can get txq lock or not. It's off by default.
- Call txeof first, then rxeof for the consistency.
- Make three "Unsupported SFP+ module..." messages the same.
- KNF. Modify comment. Fix typo.

Pull up the following revisions, requested by msaitoh in ticket #1579:

	sys/dev/pci/files.pci				1.442
	sys/dev/pci/ixgbe/ix_txrx.c			1.99-1.100
	sys/dev/pci/ixgbe/ixgbe.c			1.320-1.324 via patch
	sys/dev/pci/ixgbe/ixgbe_82598.c 		1.19
	sys/dev/pci/ixgbe/ixgbe_api.c			1.28
	sys/dev/pci/ixgbe/ixgbe_common.c		1.43
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.17
	sys/dev/pci/ixgbe/ixv.c 			1.183

- Add an option for Tx to use deferred softint regardless of whether
  can get txq lock or not. It's off by default.
- Call txeof first, then rxeof for the consistency.
- Make three "Unsupported SFP+ module..." messages the same.
- KNF. Modify comment. Fix typo.

Call txeof first, then rxeof for the consistency.

 There are three functions where the txeof and rxeof are called. The legacy
interrupt function and MSI-X function call rxeof first, then rxeof.
For the workqueue function. rxeof is called first. Modify it to match with
other two.

Pull up the following revisions, requestes by msaitoh in ticket #1746:

	sys/dev/pci/ixgbe/ix_txrx.c			1.95
	sys/dev/pci/ixgbe/ixgbe.c			1.261,1.263,
							1.265-1.268,1.273,
							1.275-1.277,1.305,
							1.312,
							1.316-1.321 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.85 via patch
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.19
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.15-1.16
	sys/dev/pci/ixgbe/ixgbe_type.h			1.46-1.47
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.26
	sys/dev/pci/ixgbe/ixv.c				1.178,1.182 via patch

- Reduce code duplication between ixgbe_msix_admin() and
  ixgbe_legacy_irq().
  - Add missing code which was not in ixgbe_msix_admin() from
    ixgbe_legacy_irq() and vice versa.
  - Reorder code.
- Disable/enable the OTHER interrupts correctly.
- Don't return in the middle of ixgbe_msix_admin() when an flow
  director reinit failed. NetBSD currently doesn't support flow
  director, so this is not a real bug.
- Print ECC, PHY and temp error log using with ratecheck().
- Correctly re-enable queue interrupt in ixgbe_legacy_irq().
- Correctly enter the recovery mode.
- No functional change:
  - Add some debug printf()s.
  - Don't use "more" flag for simplify.
  - Use macro.
  - Fix typos in comment.
  - KNF.

Pull up the following revisions, all via patch, requested by msaitoh
in ticket #1459:

	sys/dev/pci/ixgbe/ixgbe.c	1.261,1.265-1.268,1.273,1.275-1.277,
					1.312,1.316-1.319 via patch
	sys/dev/pci/ixgbe/ixgbe.h	1.85 via patch
	sys/dev/pci/ixgbe/ixgbe_type.h	1.46-1.47
	sys/dev/pci/ixgbe/ixgbe_x550.c	1.26
	sys/dev/pci/ixgbe/ixv.c		1.182

- Reduce code duplication between ixgbe_msix_admin() and
  ixgbe_legacy_irq().
  - Add missing code which was not in ixgbe_msix_admin() from
    ixgbe_legacy_irq() and vice versa.
  - Reorder code.
- Disable/enable the OTHER interrupts correctly.
- Don't return in the middle of ixgbe_msix_admin() when an flow
  director reinit failed. NetBSD currently doesn't support flow
  director, so this is not a real bug.
- Print ECC, PHY and temp error log using with ratecheck().
- Correctly re-enable queue interrupt in ixgbe_legacy_irq().
- Correctly enter the recovery mode.
- No functional change:
  - Add some debug printf()s.
  - Don't use "more" flag for simplify.
  - Fix typos in comment.
  - KNF.

KNF. No functional change.

Pull up following revision(s) (requested by msaitoh in ticket #1745):

	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.98
	sys/dev/pci/ixgbe/ixv.c: revision 1.181
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.315
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.86

bus_dmamem_unmap() before bus_dmamem_free(), otherwise we may give back meomry
which is still (and will stay) mapped.

Fixes one instance of "panic: HYPERVISOR_mmu_update failed" on Xen.
There may be others.

Fix a bug that the legacy interrupt doesn't work when MSI-X allocation failed.
Fixes PR kern/56857.

Remove unused adapter->msix_mem.

Pull up following revision(s) (requested by msaitoh in ticket #1458):

	sys/dev/pci/ixgbe/ixv.c: revision 1.181
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.315
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.86

Fix a bug that the legacy interrupt doesn't work when MSI-X allocation failed.
Fixes PR kern/56857.

Remove unused adapter->msix_mem.

Pull up the following revisions, requested by msaitoh in ticket #1744:

	sys/dev/pci/ixgbe/ixgbe.c			1.270,1.280,1.307-1.311,
							1.313-1.314 via patch
	sys/dev/pci/ixgbe/ix_txrx.c			1.96-1.97
	sys/dev/pci/ixgbe/ixv.c				1.179-1.180 via patch


- ixg(4): Print Printed Board Assembly (PBA) number.
- ixg(4): Add IFF_RUNNING check in ixgbe_legacy_irq() again. this might
  fix small race but it's not so dangerous.
- Add value check for {tx,rx}_process_limit sysctl to avoid setting
  wrong value.
- Add missing num_tx_desc sysctl.
- No functional change:
  - KNF a bit.
  - Simplify setting of EIAC register.
  - Move the definition of eicr_mask variable.
  - Enclose flow director stuff in ixgbe_intr_admin_common() with
    IXGBE_FIR which is not defined in NetBSD.
  - Modify comment for consistency.
  - Use cached rx_copy_len in ixgbe_rxeof().

Pull up the following revisions, requested by msaitoh:

	sys/dev/pci/ixgbe/ixgbe.c	1.270,1.280,1.307-1.311,
					1.313-1.314 via patch
	sys/dev/pci/ixgbe/ix_txrx.c	1.96-1.97
	sys/dev/pci/ixgbe/ixv.c		1.158,1.179-1.180 via patch


- ixg(4): Print Printed Board Assembly (PBA) number.
- ixg(4): Add IFF_RUNNING check in ixgbe_legacy_irq() again. this might
  fix small race but it's not so dangerous.
- Add value check for {tx,rx}_process_limit sysctl to avoid setting
  wrong value.
- Add missing num_tx_desc sysctl.
- No functional change:
  - KNF a bit.
  - Simplify setting of EIAC register.
  - Move the definition of eicr_mask variable.
  - Enclose flow director stuff in ixgbe_intr_admin_common() with
    IXGBE_FIR which is not defined in NetBSD.
  - Modify comment for consistency.
  - Use cached rx_copy_len in ixgbe_rxeof().

Remove unused adapter->msix_mem.

Add missing num_{tx,rx}_desc sysctl to sync with ixgbe.c.

Limit {tx,rx}_process_limit sysctl values from 1 to num_{tx,rx}_desc.

Pull up the following revisions (requested by msaitoh in ticket #1424):

	sys/dev/pci/ixgbe/ix_txrx.c			1.95
	sys/dev/pci/ixgbe/ixgbe.c			1.305 via patch
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.19
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.15-1.16
	sys/dev/pci/ixgbe/ixv.c				1.178 via patch

Use atomic_{load,store}_relaxed() for event counters.

Pull up following revision(s) (requested by msaitoh in ticket #1730):

	sys/dev/pci/ixgbe/ixgbe_mbx.h: revision 1.19
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.31
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.23
	sys/dev/pci/ixgbe/if_sriov.c: revision 1.17
	sys/dev/pci/ixgbe/ixv.c: revision 1.172
	sys/dev/pci/ixgbe/ixv.c: revision 1.173
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.301
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.55
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.29
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.16
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.17
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.18

Add code to support API version 1.5. No functional change.
   - This change adds almost all code to support API 1.5 except real negotiation
     and upgrade mailbox functions.
   - From ix-3.3.18, ix-3.3.22 and ixv-1.5.24.

Enable mailbox API 1.5 support. Tested on ESXi with ixgben 1.10.3.0.

Don't clear mailbox related counters in ixgbe_upgrade_mbx_params_vf().
Don't clear mailbox related counters in ixgbe_upgrade_mbx_params_pf().

Pull up following revision(s) (requested by msaitoh in ticket #1416):

	sys/dev/pci/ixgbe/ixgbe_mbx.h: revision 1.19
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.31
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.23
	sys/dev/pci/ixgbe/if_sriov.c: revision 1.17
	sys/dev/pci/ixgbe/ixv.c: revision 1.172
	sys/dev/pci/ixgbe/ixv.c: revision 1.173
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.301
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.55
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.29
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.16
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.17
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.18

Add code to support API version 1.5. No functional change.
   - This change adds almost all code to support API 1.5 except real negotiation
     and upgrade mailbox functions.
   - From ix-3.3.18, ix-3.3.22 and ixv-1.5.24.

Enable mailbox API 1.5 support. Tested on ESXi with ixgben 1.10.3.0.

Don't clear mailbox related counters in ixgbe_upgrade_mbx_params_vf().
Don't clear mailbox related counters in ixgbe_upgrade_mbx_params_pf().

Pull up the following, requested by msaitoh in ticket #1729:

	sys/dev/pci/ixgbe/if_sriov.c			1.12-1.16
	sys/dev/pci/ixgbe/ixgbe.c			1.295-1.297,1.300,1.304 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.84
	sys/dev/pci/ixgbe/ixgbe_82598.c			1.17-1.18
	sys/dev/pci/ixgbe/ixgbe_82598.h			1.9
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.24-1.28
	sys/dev/pci/ixgbe/ixgbe_82599.h			1.8
	sys/dev/pci/ixgbe/ixgbe_api.c			1.26-1.27
	sys/dev/pci/ixgbe/ixgbe_api.h			1.16
	sys/dev/pci/ixgbe/ixgbe_bypass.h		1.3
	sys/dev/pci/ixgbe/ixgbe_common.c		1.34-1.42
	sys/dev/pci/ixgbe/ixgbe_common.h		1.15-1.16
	sys/dev/pci/ixgbe/ixgbe_dcb.c			1.12-1.13
	sys/dev/pci/ixgbe/ixgbe_dcb.h			1.8-1.9
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c		1.10-1.12
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h		1.8
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c		1.10-1.11
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h		1.8
	sys/dev/pci/ixgbe/ixgbe_fdir.h			1.4
	sys/dev/pci/ixgbe/ixgbe_features.h		1.4
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.13-1.15
	sys/dev/pci/ixgbe/ixgbe_mbx.h			1.15-1.18
	sys/dev/pci/ixgbe/ixgbe_netmap.c		1.5
	sys/dev/pci/ixgbe/ixgbe_osdep.c			1.8
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.31
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.25-1.29
	sys/dev/pci/ixgbe/ixgbe_phy.h			1.13
	sys/dev/pci/ixgbe/ixgbe_rss.h			1.6
	sys/dev/pci/ixgbe/ixgbe_sriov.h			1.5
	sys/dev/pci/ixgbe/ixgbe_type.h			1.51-1.54
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.28-1.29
	sys/dev/pci/ixgbe/ixgbe_vf.h			1.15
	sys/dev/pci/ixgbe/ixgbe_x540.c			1.20-1.22
	sys/dev/pci/ixgbe/ixgbe_x540.h			1.10
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.21-1.25
	sys/dev/pci/ixgbe/ixgbe_x550.h			1.7
	sys/dev/pci/ixgbe/ixv.c				1.170, 1.174-1.175 via patch

- Add typecast for type mismatch.
- Fix retry count calculation of I2C read/write.
- Wait longer for link after fiber MAC setup.
- ixv(4): Use adapter->mta for the multicast array memory instead of
  the on-stack array.
- Match X550_PHY_ID correctly on X550.
- Print NVM image version on 82598.
- Use 64bit for lxon + lxoff.
- Don't expose garbage data of hw.ixvN.debug.
- Some NetBSD unrelated changes:
  - Fix infinite recursion on PCIe link down if VMDQ is used.
  - Move PF mailbox initialization from ixgbe_attach() to
    ixgbe_init_iov().
  - Add IPv6 mask for flow director.
  - Change error level in ixgbe_fc_autoneg().
  - Check host interface return status when writing NVM.
  - Change DCB credit parameters.
- Restore some mailbox related functions. Revert part of ixgbe_mbx.c
  rev. 1.7 and ixgbe_mbx.h rev. 1.11. No functional change.
- Rename IXGBE_VT_MSGTYPE_{ACK,NACK} to
  IXGBE_VT_MSGTYPE_{SUCCESS,FAILURE}. No functional change.
- Remove unused argument. Change argument.
- Remove unnecessary return value check.
- Remove debug error message.
- Remove dead code.
- Add some unused macros.
- Fix typo in comment.
- Rename some functions.
- Sort lines, modify comment.
- Whitespace fix.

Pull up the following revisions, requested by msaitoh in ticket #1414:

	sys/dev/pci/ixgbe/if_sriov.c			1.12-1.16
	sys/dev/pci/ixgbe/ixgbe.c			1.295-1.297,1.300,1.304 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.84
	sys/dev/pci/ixgbe/ixgbe_82598.c			1.17-1.18
	sys/dev/pci/ixgbe/ixgbe_82598.h			1.9
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.24-1.28
	sys/dev/pci/ixgbe/ixgbe_82599.h			1.8
	sys/dev/pci/ixgbe/ixgbe_api.c			1.26-1.27
	sys/dev/pci/ixgbe/ixgbe_api.h			1.16
	sys/dev/pci/ixgbe/ixgbe_bypass.h		1.3
	sys/dev/pci/ixgbe/ixgbe_common.c		1.34-1.42
	sys/dev/pci/ixgbe/ixgbe_common.h		1.15-1.16
	sys/dev/pci/ixgbe/ixgbe_dcb.c			1.12-1.13
	sys/dev/pci/ixgbe/ixgbe_dcb.h			1.8-1.9
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c		1.10-1.12
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h		1.8
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c		1.10-1.11
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h		1.8
	sys/dev/pci/ixgbe/ixgbe_fdir.h			1.4
	sys/dev/pci/ixgbe/ixgbe_features.h		1.4
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.13-1.15
	sys/dev/pci/ixgbe/ixgbe_mbx.h			1.15-1.18
	sys/dev/pci/ixgbe/ixgbe_netmap.c		1.5
	sys/dev/pci/ixgbe/ixgbe_osdep.c			1.8
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.31
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.25-1.29
	sys/dev/pci/ixgbe/ixgbe_phy.h			1.13
	sys/dev/pci/ixgbe/ixgbe_rss.h			1.6
	sys/dev/pci/ixgbe/ixgbe_sriov.h			1.5
	sys/dev/pci/ixgbe/ixgbe_type.h			1.51-1.54
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.28-1.29
	sys/dev/pci/ixgbe/ixgbe_vf.h			1.15
	sys/dev/pci/ixgbe/ixgbe_x540.c			1.20-1.22
	sys/dev/pci/ixgbe/ixgbe_x540.h			1.10
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.21-1.25
	sys/dev/pci/ixgbe/ixgbe_x550.h			1.7
	sys/dev/pci/ixgbe/ixv.c				1.170, 1.174-1.175 via patch

- Add typecast for type mismatch.
- Fix retry count calculation of I2C read/write.
- Wait longer for link after fiber MAC setup.
- ixv(4): Use adapter->mta for the multicast array memory instead of
  the on-stack array.
- Match X550_PHY_ID correctly on X550.
- Print NVM image version on 82598.
- Use 64bit for lxon + lxoff.
- Don't expose garbage data of hw.ixvN.debug.
- Some NetBSD unrelated changes:
  - Fix infinite recursion on PCIe link down if VMDQ is used.
  - Move PF mailbox initialization from ixgbe_attach() to
    ixgbe_init_iov().
  - Add IPv6 mask for flow director.
  - Change error level in ixgbe_fc_autoneg().
  - Check host interface return status when writing NVM.
  - Change DCB credit parameters.
- Restore some mailbox related functions. Revert part of ixgbe_mbx.c
  rev. 1.7 and ixgbe_mbx.h rev. 1.11. No functional change.
- Rename IXGBE_VT_MSGTYPE_{ACK,NACK} to
  IXGBE_VT_MSGTYPE_{SUCCESS,FAILURE}. No functional change.
- Remove unused argument. Change argument.
- Remove unnecessary return value check.
- Remove debug error message.
- Remove dead code.
- Add some unused macros.
- Fix typo in comment.
- Rename some functions.
- Sort lines, modify comment.
- Whitespace fix.

Pull up the following revisions (all via patch), requested by msaitoh
in ticket #1725:

	sys/dev/pci/ixgbe/ixgbe_vf.h			1.16-1.17
	sys/dev/pci/ixgbe/ixv.c				1.176-1.177

Make ifconfig -z ixvN clear event counter.

Pull up the following revisions (all via patch), requested by msaitoh
in ticket #1408:

	sys/dev/pci/ixgbe/ixgbe_vf.h			1.16-1.17
	sys/dev/pci/ixgbe/ixv.c				1.176-1.177

Make ifconfig -z ixvN clear event counter.

Use atomic_{load,store}_relaxed() for event counters.

 Fix ixv.c rev. 1.176. Calculate 36bit counter correctly.

Improve ixv(4)'s some event counters.

 - The virtual function's packet counter registers are not cleared on read.
   To solve this problem, <REGNAME>, base_<REGNAME>, last_<REGNAME> and
   saved_reset_<REGNAME> are in the struct ixgbevf_hw_stats and some
   functions use them. However, saved_reset_<REGNAME> is set but never
   referenced. base_<REGNAME> is set and it's only used for
   saved_reset_<REGNAME>, so it's also unused in reality. THERE ARE A LOT
   OF GARBAGE. Remove them.
 - Call ixv_init_stats() in ixv_clear_evcnt() to make ifconfig -z ixvN
   work correctly.

Update ixv_print_debug_info() to print similar data to ixg(4).

  The old hw.ixvN.debug=1 printed evcnt(9) data. The same values are printed
  by vmstat -e, so it's not worth to print. Remove all of the old output and
  make it similar to ixgbe_print_debug_info().

Don't expose garbage data of hw.ixvN.debug.

Enable mailbox API 1.5 support. Tested on ESXi with ixgben 1.10.3.0.

Add code to support API version 1.5. No functional change.

  - This change adds almost all code to support API 1.5 except real negotiation
    and upgrade mailbox functions.
  - From ix-3.3.18, ix-3.3.22 and ixv-1.5.24.

Whitespace fix. Remove old comment. No functional change.

  - From FreeBSD ixv-1.5.22.

Use adapter->mta for the multicast array memory instead of the on-stack array.

  - Same as ixv-1.5.22.
  - Same as ixg(4).

Pull up the following, requested by msaitoh in ticket #1708:

	sys/dev/pci/ixgbe/ixgbe.h			1.81-1.83
	sys/dev/pci/ixgbe/ixgbe.c			1.291-1.292 via patch
	sys/dev/pci/ixgbe/ixgbe_type.h			1.50
	sys/dev/pci/ixgbe/ixv.c				1.167-1.168 via patch
	sys/dev/pci/ixgbe/ix_txrx.c			1.94

- Fix a bug that a near 64KB TSO segment can't send.
- Reduce bus_dmamap_sync() cost.
- Use macro. Fix typos in comment.

Pull up the following, requested by msaitoh in ticket #1374:

	sys/dev/pci/ixgbe/ixgbe.h			1.81-1.83
	sys/dev/pci/ixgbe/ixgbe.c			1.291-1.292 via patch
	sys/dev/pci/ixgbe/ixgbe_type.h			1.50
	sys/dev/pci/ixgbe/ixv.c				1.167-1.168 via patch
	sys/dev/pci/ixgbe/ix_txrx.c			1.94

- Fix a bug that a near 64KB TSO segment can't send.
- Reduce bus_dmamap_sync() cost.
- Use macro. Fix typos in comment.

Remove vlan_ifdetach() from ixv_detach()

The removed function is called in ether_ifdetach().

 Fix a bug that an mbuf chain which has more than 63488bytes MAY fail on TX.

 - Currently, The TX buffer dmamap's max number of segments is set to 32.
   MCLBYTES(== 2048) * 32 = 65536 and it's enough for IP_MAXPACKET(65535).
   If an mbuf chain has more than 32 mbufs, we call m_defrag() to make
   it lower than equal to 32, but it might not work. The reason is that
   our m_defrag() don't modify the first mbuf entry of the chain. e.g.:
   if an mbuf chain contains 63600bytes data and the first mbuf has 100bytes
   in the m_data, the new chain has 100+2048+2048+...+12 and the total number
   of the chain is not 32 but 33. It result in 43 TCP packets will drop.
 - One of the way to fix this problem is to change m_defrag() which add a
   new mbuf cluster to the first mbuf. It's need discussion.
 - Another solution is to change the max number of the TX DMA segment.
   It should be at least 33 to avoid the m_defrag()'s current limitation.
   The document (82599-X550 DS 7.2.1.1 "Transmit Storage in system Memory")
   says that a packet can be fragmented into 40 - WTHRESH - 2 (for 82598,
   the document say nothing and some people and code says it's unlimited).
   Currently WTHRESH is set to 8. 40 - 8 - 2 = 30. !?!?!? {Net,Free,Open}BSD
   and Linux use 32. Is that safe? Anyway, we change WTHRESH from 8 to 5
   to fit it. The added comment in ixgbe.h is based on DragonFly's
   though they don't use WTHRESH.
 - Yet another solution is to use the Tx Head Pointer Write Back function
   instead of WTHRESH based write back (see 82599-X550 DS 7.2.3.5.2
   "Tx Head Pointer Write Back" or 82598 "Transmit Completions Head Write Back"
   ). DragonFly, illumos and DPDK use it.
 - Yet yet another solution is to add tso_maxsize entry to struct ifnet
   and use it in the TCP stack.

Use IXGBE_TXDCTL_WTHRESH_SHIFT macro. No functional change.

Pull up the following (all via patch), requested by msaitoh in ticket #1696:

	sysdev/pci/ixgbe/ixgbe.c			1.252, 1.280-1.283, 1.286-1.287, 1.289-1.290 via patch
	sysdev/pci/ixgbe/ixgbe.h			1.73, 1.76-1.80 via patch
	sysdev/pci/ixgbe/ix_txrx.c			1.68-1.93
	sysdev/pci/ixgbe/ixv.c				1.153, 1.157-1.161, 1.163-1.166 via patch
	sysdev/pci/ixgbe/if_bypass.c			1.7-1.9
	sysdev/pci/ixgbe/if_fdir.c			1.4-1.5
	sysdev/pci/ixgbe/if_sriov.c			1.10-1.11
	sysdev/pci/ixgbe/ixgbe_82598.c			1.16
	sysdev/pci/ixgbe/ixgbe_82599.c			1.23
	sysdev/pci/ixgbe/ixgbe_api.c			1.25
	sysdev/pci/ixgbe/ixgbe_bypass.h			1.2
	sysdev/pci/ixgbe/ixgbe_common.c			1.30-1.33
	sysdev/pci/ixgbe/ixgbe_dcb.c			1.10-1.11
	sysdev/pci/ixgbe/ixgbe_dcb.h			1.7
	sysdev/pci/ixgbe/ixgbe_dcb_82598.c		1.8-1.9
	sysdev/pci/ixgbe/ixgbe_dcb_82598.h		1.7
	sysdev/pci/ixgbe/ixgbe_dcb_82599.c		1.8-1.9
	sysdev/pci/ixgbe/ixgbe_dcb_82599.h		1.7
	sysdev/pci/ixgbe/ixgbe_fdir.h			1.3
	sysdev/pci/ixgbe/ixgbe_features.h		1.3
	sysdev/pci/ixgbe/ixgbe_mbx.c			1.12
	sysdev/pci/ixgbe/ixgbe_netbsd.c			1.16-1.17
	sysdev/pci/ixgbe/ixgbe_netbsd.h			1.13-1.14
	sysdev/pci/ixgbe/ixgbe_netmap.c			1.3-1.4
	sysdev/pci/ixgbe/ixgbe_netmap.h			1.2
	sysdev/pci/ixgbe/ixgbe_osdep.c			1.7
	sysdev/pci/ixgbe/ixgbe_osdep.h			1.29-1.30
	sysdev/pci/ixgbe/ixgbe_phy.c			1.24
	sysdev/pci/ixgbe/ixgbe_rss.h			1.5
	sysdev/pci/ixgbe/ixgbe_sriov.h			1.4
	sysdev/pci/ixgbe/ixgbe_type.h			1.49
	sysdev/pci/ixgbe/ixgbe_vf.c			1.27
	sysdev/pci/ixgbe/ixgbe_x540.c			1.18-1.19
	sysdev/pci/ixgbe/ixgbe_x540.h			1.9
	sysdev/pci/ixgbe/ixgbe_x550.c			1.19-1.20
	sysdev/pci/ixgbe/ixgbe_x550.h			1.6
	sysdev/pci/files.pci				1.438
	share/man/man4/ixg.4				1.15
	share/man/man4/ixv.4				1.8

- Use MCLGET() instead of homegrown cluster (jcl) allocation mechanism.
  Before this commit, resource shortage was easily occurred because
  the total number of the clusters is small.
- Improve performance:
  - Use m_adj(ETHER_ALIGN) more.
  - Sprinkle __predict_false() in the RX path.
  - Don't pre-allocate a cluster for RXCOPY case to improve short
    packet's performance.
- Call bus_dmamap_unload(9) via ixgbe_dmamap_unload(), before freeing
  DMA buffer. Also, when the buffer is already freed, do not call
  bus_dmamap_unload(9) (no resource leaks with this change). This
  change is required to make ixg(4) work on alpha.
- Keep m_len and m_pkthdr.len consistent to prevent panic on arm.
- Fix panic when bus_dmamap_load_mbuf() failed in
  ixgbe_setup_receive_ring().
- Added BUS_DMA_COHERENT flag to bus_dmamem_map() to improve stability
  on aarch64.
- Use uint64_t instead of bus_addr_t for the TX descriptor's buffer
  address. At least, this change is required for macppc
  (sizeof(bus_addr_t) == 4) to make TX work.
- Fix little-endian dependence.
- Set rxr->next_to_refresh correctly in ixgbe_setup_receive_ring().
- Refresh unrefreshed descriptors' buffers correctly.
- Don't call bus_dmamap_sync with rx_mbuf_sz(== MCLBYTES) to prevent
  panic.
- Save the discard_multidesc state to not to forget the state by
  exiting rxeof().
- Add missing increment of no_mbuf error counter.
- Don't increment no_mbuf evcnt(9) when discarding multi-descriptor
  packet.
- ixv: Modify error message to sync with ixgbe.c
- Print the error value of ixgbe_reset_hw() for debugging.
- Remove extra unlock/lock processing around if_percpuq_enqueue().
- Refactor rxr->next_to_check updating.
- Add new sysctl "rx_copy_len".
- Add a new sysctl to read rxr->next_to_refresh.
- Print error number when error occurred.
- Rename ix{gbe,v}_stop() with ix{gbe,v}_stop_locked(). No functional
  change.
- Don't use fixed value.
- Comment out flow director processing in fast path.
- Add missing NetBSD RCS IDs and __KERNEL_RCSID()s.
- KNF.
- Fix typos.

Pull up the following (via patch), requested by msaitoh in ticket #1346:

	sys/dev/pci/ixgbe/ixgbe.c			1.252, 1.280-1.283, 1.286-1.287, 1.289-1.290 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.73, 1.76-1.80 via patch
	sys/dev/pci/ixgbe/ix_txrx.c			1.68-1.93
	sys/dev/pci/ixgbe/ixv.c				1.153, 1.157-1.161, 1.163-1.166 via patch
	sys/dev/pci/ixgbe/if_bypass.c			1.7-1.9
	sys/dev/pci/ixgbe/if_fdir.c			1.4-1.5
	sys/dev/pci/ixgbe/if_sriov.c			1.10-1.11
	sys/dev/pci/ixgbe/ixgbe_82598.c			1.16
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.23
	sys/dev/pci/ixgbe/ixgbe_api.c			1.25
	sys/dev/pci/ixgbe/ixgbe_bypass.h		1.2
	sys/dev/pci/ixgbe/ixgbe_common.c		1.30-1.33
	sys/dev/pci/ixgbe/ixgbe_dcb.c			1.10-1.11
	sys/dev/pci/ixgbe/ixgbe_dcb.h			1.7
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c		1.8-1.9
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h		1.7
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c		1.8-1.9
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h		1.7
	sys/dev/pci/ixgbe/ixgbe_fdir.h			1.3
	sys/dev/pci/ixgbe/ixgbe_features.h		1.3
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.12
	sys/dev/pci/ixgbe/ixgbe_netbsd.c		1.13, 1.16-1.17
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.13-1.14
	sys/dev/pci/ixgbe/ixgbe_netmap.c		1.3-1.4
	sys/dev/pci/ixgbe/ixgbe_netmap.h		1.2
	sys/dev/pci/ixgbe/ixgbe_osdep.c			1.7
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.29-1.30
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.24
	sys/dev/pci/ixgbe/ixgbe_rss.h			1.5
	sys/dev/pci/ixgbe/ixgbe_sriov.h			1.4
	sys/dev/pci/ixgbe/ixgbe_type.h			1.49
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.27
	sys/dev/pci/ixgbe/ixgbe_x540.c			1.18-1.19
	sys/dev/pci/ixgbe/ixgbe_x540.h			1.9
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.19-1.20
	sys/dev/pci/ixgbe/ixgbe_x550.h			1.6
	sys/dev/pci/files.pci				1.438
	share/man/man4/ixg.4				1.15
	share/man/man4/ixv.4				1.8

- Use MCLGET() instead of homegrown cluster (jcl) allocation mechanism.
  Before this commit, resource shortage was easily occurred because
  the total number of the clusters is small.
- Improve performance:
  - Use m_adj(ETHER_ALIGN) more.
  - Sprinkle __predict_false() in the RX path.
  - Don't pre-allocate a cluster for RXCOPY case to improve short
    packet's performance.
- Call bus_dmamap_unload(9) via ixgbe_dmamap_unload(), before freeing
  DMA buffer. Also, when the buffer is already freed, do not call
  bus_dmamap_unload(9) (no resource leaks with this change). This
  change is required to make ixg(4) work on alpha.
- Keep m_len and m_pkthdr.len consistent to prevent panic on arm.
- Fix panic when bus_dmamap_load_mbuf() failed in
  ixgbe_setup_receive_ring().
- Added BUS_DMA_COHERENT flag to bus_dmamem_map() to improve stability
  on aarch64.
- Use uint64_t instead of bus_addr_t for the TX descriptor's buffer
  address. At least, this change is required for macppc
  (sizeof(bus_addr_t) == 4) to make TX work.
- Fix little-endian dependence.
- Set rxr->next_to_refresh correctly in ixgbe_setup_receive_ring().
- Refresh unrefreshed descriptors' buffers correctly.
- Don't call bus_dmamap_sync with rx_mbuf_sz(== MCLBYTES) to prevent
  panic.
- Save the discard_multidesc state to not to forget the state by
  exiting rxeof().
- Add missing increment of no_mbuf error counter.
- Don't increment no_mbuf evcnt(9) when discarding multi-descriptor
  packet.
- ixv: Modify error message to sync with ixgbe.c
- Print the error value of ixgbe_reset_hw() for debugging.
- Remove extra unlock/lock processing around if_percpuq_enqueue().
- Refactor rxr->next_to_check updating.
- Add new sysctl "rx_copy_len".
- Add a new sysctl to read rxr->next_to_refresh.
- Print error number when error occurred.
- Rename ix{gbe,v}_stop() with ix{gbe,v}_stop_locked(). No functional
  change.
- Don't use fixed value.
- Comment out flow director processing in fast path.
- Add missing NetBSD RCS IDs and __KERNEL_RCSID()s.
- KNF.
- Fix typos.

 Remove "j" or "jumbo" because we always use MCLBYTES sized buffer.

 Use MCLGET() instead of homegrown cluster (jcl) allocation mechanism.

- Before this commit, resource shortage was easily occurred because the total
  number of the clusters is small.

- Reviewed by knakahara and ryo.

Sync with HEAD.

 Add a new sysctl to read rxr->next_to_refresh.

Add new sysctl "rx_copy_len".

 ixgbe_rxeof() has an optimization "RX_COPY" to reduce costs of
bus_dmamap_load_mbuf() and bus_dmamap_unload() by copying a mbuf cluster's
memory to a newly allocated mbuf's MH_databuf[] and recycle the original map.
The optimization is used when a length of a packet is smaller than a specific
value. The value is calculated based on MHLEN. The size of MHLEN is
architecture specific. It's 256 or 512. Make the threshold controllable by
adding a new sysctl.

Sync w/ HEAD.

if_attach and if_initialize cannot fail, don't test return value

These were originally made failable back in 2017 when if_initialize
allocated a softint in every interface for link state changes, so
that it could fail gracefully instead of panicking:

https://mail-index.NetBSD.org/source-changes/2017/10/23/msg089053.html

However, this spawned many seldom- or never-tested error branches,
which are risky to have around.  And that softint in every interface
has since been replaced by a single global workqueue, because link
state changes require thread context but not low latency or high
throughput:

https://mail-index.NetBSD.org/source-changes/2020/02/06/msg113759.html

So there is no longer any reason for if_initialize to fail.  (The
subroutine if_stats_init can't fail because percpu_alloc can't fail
either.)

There is a snag: the softint_establish in if_percpuq_create could
fail, potentially leading to bad consequences later on trying to use
the softint.  This change doesn't introduce any new bugs because of
the snag -- if_percpuq_attach was already broken.  However, the snag
can be better addressed without spawning error branches, either by
using a single softint or making softints less scarce.

(Separate commit will change the signatures of if_attach and
if_initialize to return void, scheduled to ride whatever is the next
convenient kernel bump.)

Patch and testing on amd64 and evbmips64-eb by maya@; commit message
soliloquy, and compile-testing on evbppc/i386/earmv7hf, by me.

sync with head

No functional change:

 - Add NetBSD RCS IDs.
 - KNF.

- Cleanup an rxbuf entry when bus_dmamap_load_mbuf() failed to prevent panic.
- Print error number when error occurred.

Sync with HEAD.

 Add missing __KERNEL_RCSID().

Sync with HEAD.

Sync with HEAD.

KNF a bit. No functional change.

Modify error message to sync with ixgbe.c

Pull up the following (all via patch), requested by msaitoh in ticket #1663:

	sys/dev/pci/ixgbe/ixgbe.c			1.259, 1.278-1.279
	sys/dev/pci/ixgbe/ixgbe.h			1.75
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.12
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.24-1.26
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.17
	sys/dev/pci/ixgbe/ixv.c				1.155-1.156
	sys/dev/pci/ixgbe/ix_txrx.c			1.64-67
	sys/dev/pci/files.pci				1.436
	share/man/man4/ixg.4				1.13-1.14
	share/man/man4/ixv.4				1.3, 1.6-1.7

- Fix a problem that the RX path stalled when the mbuf cluster is
  exhausted.
- Modify some parameters to reduce packet dropping. See also the
  manual's OPTIONS section for the detail.
- ixv(4): The max number of queue(pair) is not 7 but 8. Correctly
  reset the hardware.
- Add "TX " to "Queue No Descriptor Available" evcnt(9) name to make
  it more understandable.
- Fix a bug that some advertise speeds can't be set with
  hw.ixgN.advertise_speed if both 2.5G and 5G are set. Fix the error
  message, too.
- Fix typo in comment or debug message.

Pull up the following (all via patch) requested by msaitoh in ticket #1231:

	sys/dev/pci/ixgbe/ixgbe.c			1.259, 1.278-1.279
	sys/dev/pci/ixgbe/ixgbe.h			1.75
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.12
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.24-1.26
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.17
	sys/dev/pci/ixgbe/ixv.c				1.155-1.156
	sys/dev/pci/ixgbe/ix_txrx.c			1.64-67
	sys/dev/pci/files.pci				1.436
	share/man/man4/ixg.4				1.13-1.14
	share/man/man4/ixv.4				1.6-1.7

- Fix a problem that the RX path stalled when the mbuf cluster is
  exhausted.
- Modify some parameters to reduce packet dropping. See also the
  manual's OPTIONS section for the detail.
- ixv(4): The max number of queue(pair) is not 7 but 8. Correctly
  reset the hardware.
- Add "TX " to "Queue No Descriptor Available" evcnt(9) name to make
  it more understandable.
- Fix a bug that some advertise speeds can't be set with
  hw.ixgN.advertise_speed if both 2.5G and 5G are set. Fix the error
  message, too.
- Fix typo in comment or debug message.

Add "TX " to "Queue No Descriptor Available" to make it more understandable.
Same as ixgbe.c rev. 1.278.

Modify some parameters to reduce packet dropping.

 - Background: ixgbe doesn't use common MCLGET() interface and use the
   driver specific cluster allocation mechanism (jcl). The cluster is
   pre-allocated with a fixed number and the current number per queue
   is num_rx_desc * 2 (2048*2=4096). It's too small. It also has a problem
   that the max length of the pcq which is used in the TX path is big
   (4096). Example:

    100M <----- [ixg0                                      ixg1] <----- 1G
                2048 TX descs <--- 4096 pcqs <---- 2048 RX descs

   If a machine forwards a traffic from 1G interface to 100M interface,
   It would require 2048+4096+2048=8192 descriptors, but the current number
   is 2048*2=4096. It's too small. Even if the both interface's link speed
   is the same and only small number of packet is queued in the pcq, 4096
   jcl is small because 2048(RX)+TX(2048)=4096. If jcl is exhausted, not only
   forwarding from ixg1 to ixg0 is dropped, but also another forwarding path
   from ixg1 to another interface(e.g. wm0) is also dropped. Sockets also
   queue packets, so if a lot of sockets are used and/or a socket buffer
   size is changed to bigger one, it'll also become a problem. If the jcl
   is exhausted, evcnt(9) counter "ixgX qY Rx no jumbo mbuf" is incremented.
   Example:
     vmstat -ev | grep ixg1 | grep "no jumbo"
     ixg1 q0 Rx no jumbo mbuf                           0     0 misc
     ixg1 q1 Rx no jumbo mbuf                           0     0 misc
     ixg1 q2 Rx no jumbo mbuf                      141326     0 misc
     ixg1 q3 Rx no jumbo mbuf                           0     0 misc


 - To solve this problem:
   - Add new config parameter IXGBE_JCLNUM_MULTI and set the default to 3
     (2048 * 3). The minimum number is 2. The total number of jcl per queue
     is available with hw.ixgN.num_jcl_per_queue sysctl.
   - Reduce the max length of the pcq() which is used in the TX path from
     4096 to 2048.

 - Reviewed by knakahara@ and ozaki-r@.

 - TODO: Use MCLGET().

- Remove extra callout_stop() in ixgbe_detach(). Found by knakahara@.
- Rename ix{gbe,v}_free_workqueue() to ix{gbe,v}_free_deferred_handlers().
- Add KASSERT() to functions who are called from ixgbe_handle_admin().

 Rename ix{gbe,v}_stop() with ix{gbe,v}_stop_locked(). No functional change.

 Use atomic_cas_uint() and atomic_store_relaxed(). Advised by thorpej@.
Tested by me. OK'd by knakahara.

Pull up the following revisions, requested by msaitoh in ticket #1587:

	sys/dev/pci/ixgbe/ix_txrx.c			1.62-1.63 via patch
	sys/dev/pci/ixgbe/ixgbe.c			1.225, 1.228-1.229,
							1.232 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.64, 1.66
	sys/dev/pci/ixgbe/ixv.c				1.146, 1.148-1.150
							via patch
	sys/dev/pci/ixgbe/ixgbe_common.c		1.27
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.23
	sys/dev/pci/ixgbe/ixgbe_82598.c			1.15
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.18
	sys/dev/pci/ixgbe/ixgbe_netbsd.c		1.14
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.21
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.26

- Fix IXGBE_LE32_TO_CPUS() macro for big endian machine. This problem
  was only on X550.
- Add debug printf()s.
- Use unsigned to avoid undefined behavior in
  ixgbe_fc_enable_generic().
- Modify a little to reduce diff between ixgbe.c and ixv.c.
  No functional change.
- Modify comment.
- Remove unused macros.
- Whitespace fix.
- Fix typos.

Pull up the following revisions, requested by msaitoh in ticket #997:

	sys/dev/pci/ixgbe/ix_txrx.c			1.62-1.63 via patch
	sys/dev/pci/ixgbe/ixgbe.c			1.225, 1.228-1.229,
							1.232 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.64, 1.66
	sys/dev/pci/ixgbe/ixv.c				1.146, 1.148-1.150
	sys/dev/pci/ixgbe/ixgbe_common.c		1.27
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.23
	sys/dev/pci/ixgbe/ixgbe_82598.c			1.15
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.18
	sys/dev/pci/ixgbe/ixgbe_netbsd.c		1.14
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.21
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.26

- Fix IXGBE_LE32_TO_CPUS() macro for big endian machine. This problem
  was only on X550*.
- Add debug printf()s.
- Use unsigned to avoid undefined behavior in
  ixgbe_fc_enable_generic().
- Modify a little to reduce diff between ixgbe.c and ixv.c.
  No functional change.
- Modify comment.
- Remove unused macros.
- Whitespace fix.
- Fix typos.

Reduce ixgbe's busy loop using with workqueue and kpause.

- Use workqueue instead of softint to make some functions sleepable.
- Use new workqueue and enqueue it in ixgbe_local_timer() and
  ixgbe_recovery_mode_timer() to make them sleepable.
- Make new ixgbe_delay() and use it. This functions sleeps if the time is
  more than equals 1 tick. If it's not, do delay().

Modify a liitle to reduce diff between ixgbe.c and ixv.c. No functional change.

 Fix typo. No functional change.

Sync with HEAD

Sync with HEAD

No functional change:
 - modify comment
 - whitespace fix

Mostly merge changes from HEAD upto 20200411

Define and implement a locking protocol for the ifmedia / mii layers:
- MP-safe drivers provide a mutex to ifmedia that is used to serialize
  access to media-related structures / hardware regsiters.  Converted
  drivers use the new ifmedia_init_with_lock() function for this.  The
  new name is provided to ease the transition.
- Un-converted drivers continue to call ifmedia_init(), which will supply
  a compatibility lock to be used instead.  Several media-related entry
  points must be aware of this compatibility lock, and are able to acquire
  it recursively a limited number of times, if needed.  This is a SPIN
  mutex with priority IPL_NET.
- This same lock is used to serialize access to PHY registers and other
  MII-related data structures.

The PHY drivers are modified to acquire and release the lock, as needed,
and assert the lock is held as a diagnostic aid.

The "usbnet" framework has had an overhaul of its internal locking
protocols to fit in with the media / mii changes, and the drivers adapted.

USB wifi drivers have been changed to provide their own adaptive mutex
to the ifmedia later via a new ieee80211_media_init_with_lock() function.
This is required because the USB drivers need an adaptive mutex.

Besised "usbnet", a few other drivers are converted: vmx, wm, ixgbe / ixv.

mcx also now calls ifmedia_init_with_lock() because it needs to also use
an adaptive mutex.  The mcx driver still needs to be fully converted to
NET_MPSAFE.

Sync with head.

 Modify note in ixv_update_stats(). VF doesn't count errors by hardware.

Use ifmedia_fini().

Pull up the following (via patch), requested by msaitoh in ticket #648

	sys/dev/pci/ixgbe/if_bypass.c		1.5
	sys/dev/pci/ixgbe/ixgbe_osdep.c		1.5
	sys/dev/pci/ixgbe/ix_txrx.c		1.58-1.60
	sys/dev/pci/ixgbe/ixgbe.c		1.220-1.221
	sys/dev/pci/ixgbe/ixgbe.h		1.60-1.2
	sys/dev/pci/ixgbe/ixgbe_api.c		1.24
	sys/dev/pci/ixgbe/ixgbe_common.c	1.26
	sys/dev/pci/ixgbe/ixgbe_netbsd.c	1.11-1.12
	sys/dev/pci/ixgbe/ixgbe_osdep.h		1.24
	sys/dev/pci/ixgbe/ixgbe_phy.c		1.19
	sys/dev/pci/ixgbe/ixgbe_82598.c		1.14
	sys/dev/pci/ixgbe/ixv.c			1.142,1.144

- Free RX structure correctly when detaching.
- Remove unused code.
- Fix some typos in comment.
- Remove extra spaces.
- KNF.

Sync with head.

Pull up the following, requested by msaitoh in ticket #1490:

	sys/dev/pci/ixgbe/if_bypass.c		1.5
	sys/dev/pci/ixgbe/ixgbe_osdep.c		1.5
	sys/dev/pci/ixgbe/ix_txrx.c		1.58-1.60
	sys/dev/pci/ixgbe/ixgbe.c		1.195,1.220-1.221 via patch
	sys/dev/pci/ixgbe/ixgbe.h		1.60-1.2
	sys/dev/pci/ixgbe/ixgbe_api.c		1.24
	sys/dev/pci/ixgbe/ixgbe_common.c	1.26
	sys/dev/pci/ixgbe/ixgbe_netbsd.c	1.11-1.12
	sys/dev/pci/ixgbe/ixgbe_osdep.h		1.24
	sys/dev/pci/ixgbe/ixgbe_phy.c		1.19
	sys/dev/pci/ixgbe/ixgbe_82598.c		1.14
	sys/dev/pci/ixgbe/ixv.c			1.122,1.142,1.144 via patch

- Use unsigned to avoid undefined behavior in
  ix{gbe,v}_[un]register_vlan().
- Free RX structure correctly when detaching.
- Remove unused code.
- Remove extra spaces.
- Fix some typos in comment.
- KNF.

 Fix the freeing code for some error paths. Found and tested by Patrick Welche.

Pull up following revision(s) (requested by msaitoh in ticket #580):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.217
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.218
	sys/dev/pci/ixgbe/ixgbe_osdep.c: revision 1.6
	sys/dev/pci/ixgbe/ixv.c: revision 1.143
	sys/dev/pci/ixgbe/ixgbe_osdep.h: revision 1.25

 Use bus_space_barrier() instead of x86 specific *fence instruction.
Written by riastradh@.

 Add missing core lock in ixgbe_handle_mod().

 Use bus_space_barrier() instead of x86 specific *fence instruction.
Written by riastradh@.

No functional change:

 - Remove unused code.
 - Remove extra spaces.
 - KNF.

Pull up following revision(s) (requested by msaitoh in ticket #453):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.216
	sys/dev/pci/ixgbe/ixv.c: revision 1.141

Print MAC address.

 XXX Should we move such type of printf() to ether_ifattach?

Pull up following revision(s) (requested by msaitoh in ticket #1451):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.216
	sys/dev/pci/ixgbe/ixv.c: revision 1.141

Print MAC address.

XXX Should we move such type of printf() to ether_ifattach?

 Print MAC address.

 XXX Should we move such type of printf() to ether_ifattach?

Pull up following revision(s) (requested by msaitoh in ticket #1435):

	sys/dev/pci/ixgbe/ixv.c: revision 1.140

ixv(4): disable RSS configuration on 82599 and X540 VFs.

Those VFs share their RSS configuration with PF and, thus,
they cannot be configured independently. From FreeBSD r354349.

Pull up following revision(s) (requested by msaitoh in ticket #422):

	sys/dev/pci/ixgbe/ixv.c: revision 1.140

ixv(4): disable RSS configuration on 82599 and X540 VFs.

Those VFs share their RSS configuration with PF and, thus,
they cannot be configured independently. From FreeBSD r354349.

ixv(4): disable RSS configuration on 82599 and X540 VFs.

Those VFs share their RSS configuration with PF and, thus,
they cannot be configured independently. From FreeBSD r354349.

Pull up following revision(s) (requested by msaitoh in ticket #1403):

	sys/dev/pci/ixgbe/ixv.c: revision 1.139

Set MTU correctly if mtu > 1500.

Pull up following revision(s) (requested by msaitoh in ticket #293):

	sys/dev/pci/ixgbe/ixv.c: revision 1.139

Set MTU correctly if mtu > 1500.

 Set MTU correctly if mtu > 1500.

Pull up the following revisions, requested by msaitoh in ticket #246:

	sys/dev/pci/ixgbe/ixgbe_type.h			1.42-1.43
	sys/dev/pci/ixgbe/ixgbe.c			1.209-1.213
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.16
	sys/dev/pci/ixgbe/ixv.c				1.131-1.138
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.19-1.22
	sys/dev/pci/ixgbe/ixgbe_vf.h			1.14
	sys/dev/pci/ixgbe/ixgbe.h			1.57-1.58
	share/man/man4/ixv.4				1.5

- Fix a bug that MBSDC (Bad SFD Count) isn't counted on X550EM_X and
  X550EM_A. The register is for X550 and newer.
- ixv(4): Make SIOCADDMULTI returns ENOSPC and print error message
  when the Ethernet multicast address list exceeds the limit(30) and
  can't be ALLMULTI.
- ixv(4): SIOCZIFDATA clear the event counters as ixgbe.c.
- Reduce ixv(4)'s multicast table array size in ixv_set_multi from
  MAX_NUM_MULTICAST_ADDRESSES(128) to IXGBE_MAX_VF_MC(30).
- ixv(4): Add support ALLMULTI and PROMISC.
- if_flags is neither int nor short. It's unsigned short.
- ixg(4): Fix a bug that the multicast filter isn't correctly
  initialized when the total number of the Ethernet multicast
  addresses is just 128.
- Make ixv_set_multi() work correctly (especially for PROMISC) when the
  function is called from if_init().
- Remove *_set_promisc() and use *_set_multi(). And then, rename
  *_set_multi() to *_set_rxfilter().
- ixv(4): If a multicast entry has range, use ALLMULTI like others.
- Fix typo in comment. Found by Wataru Ashihara.

Pull up the following revisions, requested by msaitoh in ticket #1389:

	sys/dev/pci/ixgbe/ixgbe_type.h			1.42-1.43
	sys/dev/pci/ixgbe/ixgbe.c			1.209-1.213 via patch
	sys/dev/pci/ixgbe/ixv.c				1.131-1.138 via patch
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.19-1.22
	sys/dev/pci/ixgbe/ixgbe_vf.h			1.14
	sys/dev/pci/ixgbe/ixgbe.h			1.57-1.58 via patch
	share/man/man4/ixv.4				1.5 via patch

- Fix a bug that MBSDC (Bad SFD Count) isn't counted on X550EM_X and
  X550EM_A. The register is for X550 and newer.
- ixv(4): Make SIOCADDMULTI returns ENOSPC and print error message
  when the Ethernet multicast address list exceeds the limit(30) and
  can't be ALLMULTI.
- ixv(4): SIOCZIFDATA clear the event counters as ixgbe.c.
- Reduce ixv(4)'s multicast table array size in ixv_set_multi from
  MAX_NUM_MULTICAST_ADDRESSES(128) to IXGBE_MAX_VF_MC(30).
- ixv(4): Add support ALLMULTI and PROMISC.
- if_flags is neither int nor short. It's unsigned short.
- Fix a bug that the multicast filter isn't correctly initialized when
  the total number of the Ethernet multicast addresses is just 128.
- Make ixv_set_multi() work correctly (especially for PROMISC) when the
  function is called from if_init().
- Remove *_set_promisc() and use *_set_multi(). And then, rename
  *_set_multi() to *_set_rxfilter().
- ixv(4): If a multicast entry has range, use ALLMULTI like others.

- Make ixv_set_multi() work correctly (especially for PROMISC) when the
  function is called from if_init().
- If a multicast entry has range, use ALLMULTI like others.
- Remove ixv_set_promisc() and use ixv_set_multi(). And then, rename
  *_set_multi() to *_set_rxfilter(). Same as ixgbe.c.
- The promisc mode can't be enabled if the PF is not in promisc mode.
  Identify that state and report it as "the PF may not in promisc mode"
  (though it might not be perfect).

 Add PROMISC support.

 if_flags is neither int nor short. It's unsigned short.

- Return EPERM when the interface is not trusted.
- Treat IXGBE_ERR_FEATURE_NOT_SUPPORTED.

Add support ALLMULTI on ixv(4):
 - Negotiate API version up to 1.3.
 - On linux's PF driver implementation, the PF replies VF's XCAST_MODE_ALLMULTI
   message not with NACK but with ACK even if the virtual function is NOT
   marked "trust" and act as XCAST_MODE_"MULTI". If ixv(4) simply check the
   return vaule of update_xcast_mode(XCAST_MODE_ALLMULTI), SIOCSADDMULTI
   success and the user may have trouble with some addresses. Fortunately,
   the Linux's PF driver's "ACK" message has not XCAST_MODE_"ALL"MULTI but
   XCAST_MODE_MULTI, so we can check this state by checking if the send
   message's argument and the reply message's argument are different.
 - Noy yet for PROMISC.

 Reduce ixv(4)'s multicast table array size in ixv_set_multi from
MAX_NUM_MULTICAST_ADDRESSES(128) to IXGBE_MAX_VF_MC(30).

- Make SIOCADDMULTI returns ENOSPC when the Ethenet multicast address list
  exceeds the limit.
- State "Ethernet" multicast address.
- Use macro.

 Add support SIOCZIFDATA as ixgbe.c.

BUGS
     The following event counters are not cleared by SIOCZIFDATA because the
     corresponding registers are read only and not cleared on read:

           Good Packets Received
           Good Octets Received
           Multicast Packets Received
           Good Packets Transmitted
           Good Octets Transmitted

Pull up following revision(s) (requested by msaitoh in ticket #180):

	sys/dev/pci/ixgbe/ixv.c: revision 1.130
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.207
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.208
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.55
	sys/dev/pci/ixgbe/ixv.c: revision 1.129
	sys/dev/pci/ixgbe/ixgbe_netbsd.c: revision 1.10

 printf -> device_printf

 Set IFM_1000_BX10 correctly.

 Use aprint_*() in the attach function.

Apply patch, requested by msaitoh in ticket #1367, to pull up the following
revisions:

	sys/dev/pci/ixgbe/ixgbe_x550.c                  1.16
	sys/dev/pci/ixgbe/ixgbe.c                       1.180,1.203-1.204,
							1.207-1.208 via patch
	sys/dev/pci/ixgbe/ix_txrx.c                     1.55
	sys/dev/pci/ixgbe/ixgbe_netbsd.c                1.10
	sys/dev/pci/ixgbe/ixgbe_common.c                1.25
	sys/dev/pci/ixgbe/ixv.c                         1.129-1.130

- X550EM supports QSFP, so check ixgbe_media_type_fiber_qsfp too.
- An interrupt might not arrive when a module is inserted. When an link
  status change interrupt occurred and the driver still regard SFP as
  unplugged, issue the module softint before issuing LSC softint.
- Add 10000BASE-LX media if it's 1000BASE-BX.
- printf -> device_printf
- Avoid undefined behavior of VLAN filter setting.
- Simplify code.
- Fix typo in unused code.
- Fix typo in comment. Found by Wataru Ashihara.

 Use aprint_*() in the attach function.

 printf -> device_printf

Pull up following revision(s) (requested by msaitoh in ticket #134):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.202
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.203
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.204
	sys/dev/pci/ixgbe/ixv.c: revision 1.128

Simplify ix{gbe,v}_[un]register_vlan() API suggested by knakahara.

 The API was the same as FreeBSD's pre-iflib's. They use iflib now and it's
not required for us to keep the old API.
X550EM supports QSFP, so check ixgbe_media_type_fiber_qsfp too.

 An interrupt might not arrive when a module is inserted. When an link status
change interrupt occurred and the driver still regard SFP as unplugged, link
becomes up and the real media type is unknown. e.g:

 % ifconfig -m ixg0
 (snip)
         media: Ethernet autoselect (autoselect rxpause,txpause)
         status: active
         supported Ethernet media:
                 media none
                 media autoselect
 (snip)

To resolve this problem, when an link status change interrupt occurred and the
driver still regard SFP as unplugged, issue the module softint before issuing
LSC interrupt.

Pull up following revision(s) (requested by msaitoh in ticket #133):

	sys/dev/pci/ixgbe/ixgbe.c: revision 1.200
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.201
	sys/dev/pci/ixgbe/ixv.c: revision 1.126
	sys/dev/pci/ixgbe/ixv.c: revision 1.127
	sys/net/if_vlan.c: revision 1.142
	sys/net/if_vlan.c: revision 1.143
	sys/net/if_vlan.c: revision 1.144
	sys/net/if_vlan.c: revision 1.145
	sys/net/if_vlan.c: revision 1.146

 Check ec_capenable instead of ec_capabilities to control TX side of VLAN HW
tagging correctly.
XXX pullup-9

 Add missing IFNET_LOCK() and IFNET_UNLOCK() in vlan_config().
XXX pullup-9

 Fix a bug that VLAN HW "tagging" enable/disable may not reflect correctly.
  - Always call ec_vlan_cb() if it exists.
  - Some (or all?) ethernet drivers don't enable HW tagging if no any vlan is
    attached. ixgbe is one of them. Check the the transition and update
    VLAN HW tagging function.
XXX pullup-9

 Use ETHER_LOCK()/ETHER_UNLOCK() suggested by knakahara.
- kmem_alloc(,KM_SLEEP) never return NULL, so remove NULL check.
- VLAN ID is never duplicated, so break the loop when found. Also move
  kmen_free() outside of ETHER_LOCK(ec)/ETHER_UNLOCK(ec) to reduce the hold
  time. suggested by ozaki-r.
- Whitespace fix.

Simplify ix{gbe,v}_[un]register_vlan() API suggestesd by knakahara.

 The API was the same as FreeBSD's pre-iflib's. They use iflib now and it's
not required for us to keep the old API.

 Use ETHER_LOCK()/ETHER_UNLOCK() suggested by knakahara.

 Fix a bug that VLAN HW "tagging" enable/disable may not refrect correctly.

  - Always call ec_vlan_cb() if it exists.
  - Some (or all?) ethernet drivers don't enable HW tagging if no any vlan is
    attached. ixgbe is one of them. Check the the transition and update
    VLAN HW tagging function.

XXX pullup-9

Pull up the following revisions, requested by msaitoh in ticket #1313:

	sys/dev/pci/ixgbe/ixgbe_type.h			1.41
	sys/dev/pci/ixgbe/ixgbe.c			1.194,1.197-1.199
							via patch
	sys/dev/pci/ixgbe/ixgbe_common.c		1.24
	sys/dev/pci/ixgbe/ixv.c				1.121,1.124-1.125

 - Avoid undefined behavior of interrupt vector setting.
 - Avoid undefined behavior of TX/RX queue statistics calculation.
 - Avoid undefined behavior of X550EM's PHY accesses.
 - Avoid undefined behavior of Flow Control Transmit Timer setting.
 - Don't call {ixgbe,ixv}_stop() twice while detaching.

 It's not required to call ixv_stop() twice in ixv_detach().

 Simplify and avoid undefined behavior. This is not a real bug because
the max number of queue is 2.

 Fix VLAN hardware filter initialization. Use unsigned. Found by KUBSan.

 Fix VLAN filter setting. Found by KUBSan.

Set IVAR (Interrupt Vector Allocation Register) correctly in ixgbe_set_ivar().
Found by KUBSan.

Pull up the following revisions (via patch), requested by msaitoh in
ticket #1301:

	sys/dev/pci/ixgbe/if_sriov.c			1.5-1.6
	sys/dev/pci/ixgbe/ix_txrx.c			1.53-1.54
	sys/dev/pci/ixgbe/ixgbe_x550.h			1.5
	sys/dev/pci/ixgbe/ixgbe.c			1.169-1.170,1.176,1.179,1.181,1.185-1.186,1.188-1.192 via patch
	sys/dev/pci/ixgbe/ixgbe.h			1.53,1.55
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.21
	sys/dev/pci/ixgbe/ixgbe_api.c			1.22-1.23
	sys/dev/pci/ixgbe/ixgbe_api.h			1.14-1.15
	sys/dev/pci/ixgbe/ixgbe_x550.c			1.14-1.15
	sys/dev/pci/ixgbe/ixgbe_common.c		1.23
	sys/dev/pci/ixgbe/ixgbe_common.h		1.14
	sys/dev/pci/ixgbe/ixgbe_mbx.c			1.11
	sys/dev/pci/ixgbe/ixgbe_mbx.h			1.14
	sys/dev/pci/ixgbe/ixgbe_netmap.c		1.2
	sys/dev/pci/ixgbe/ixgbe_features.h		1.2
	sys/dev/pci/ixgbe/ixgbe_netbsd.c		1.9
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.9-1.10
	sys/dev/pci/ixgbe/ixgbe_phy.c			1.18
	sys/dev/pci/ixgbe/ixgbe_type.h			1.37,1.39-1.40
	sys/dev/pci/ixgbe/ixgbe_vf.c			1.18
	sys/dev/pci/ixgbe/ixv.c				1.112-1.114,1.117,1.119 via patch

Sync ixgbe up to 20190717 except ETHERCAP or ixv's VLAN stuff:
- Add firmware recovery mode for X550, X550A(Xeon D) and X550EM(C3000).
- Remove IXGBE_DEV_ID_82599_LS(0x154f) support again.
- On X550EMU, use ixgbe_identify_sfp_module_X550em() instead of
  ixgbe_identify_module_generic(). ixgbe_identify_sfp_module_X550em()
  has extra check (e.g. exclude 1G copper).
- It's not required to calculate unused queues' statistics.
- Remove ETHERCAP_VLAN_HWFILTER's definition.
- Match 82598_BX(0x1508), 82599_KR(0x1517), 82599_SFP_EM(0x1507),
  X550EM_X_XFI(0x15b0), X550EM_A_QSFP(0x15ca) and
  X550EM_A_QSFP_N(0x15cc)
- Add missing XFI support into ixgbe_get_link_capabilities_X550em().
- Other than IXGBE_VF_RESET should wait ACK, so use
  ixgbevf_write_msg_read_ack() instead of write_posted() in
  ixgbe_update_mc_addr_list_vf().
- When ixv_check_link() failed in the watchdog function, reset the
  interface.
- Remove RXCSUM register modification in
  ixv_initialize_receive_units(). It seems it's not required.
- Remove some debug printf in ixv_print_debug_info().
- Calculate vector's bit location correctly when the vector >= 31
  in ixgbe_allocate_msix().
- Fix hung queue check when the queue number >= 31.
- On ENETRESET case, not continue and quit the ifflags_cb() function
  because if_init() will do the same thing.
- Fix bugs in unused code.
- Fix typo in comment.
- Fix typo in unused code.
- Whitespace fixes. KNF.

 Implement VLAN hardware filter function(ETHERCAP_VLAN_HWFILTER).
First proposed by jmcneill in 2017 and modified by me.

How to use:

 - Set callback function:

	ether_set_vlan_cb(struct ethercom *, ether_vlancb_t)

 - Callback. This function is called when a vlan is attached/detached to the
   parent interface:

	int (*ether_vlancb_t)(struct ethercom *ec, uint16_t vlanid, bool set);

 - ifconfig(8)

	ifconfig ixg0 [-]vlan-hwfilter

 Note that ETHERCAP_VLAN_HWFILTER is set by default on ixg(4) because
the PF driver usually enable "all block" filter by default.

 KNF. No functional change.

 On ENETRESET case, not continue and quit the ifflags_cb() function because
if_init() will do the same thing.

Sync with FreeBSD's ixv-1.5.17:
 - When ixv_check_link() failed in the watchdog function, reset the interface.
 - Remove RXCSUM register modification in ixv_initialize_receive_units().
   It seems it's not required.
 - Remove some debug printf in ixv_print_debug_info().
 - Modify to match the newer NETMAP interface. No functional change.

Don't call set_vfta() if any VLAN is attached.
XXX pullup-8.

Sync with HEAD

Even if we don't use MII(4), use the common path of SIOC[GS]IFMEDIA in
sys/net/if_ethersubr.c if we can.
 - Add ec_ifmedia into struct ethercom.
 - ec_mii in struct ethercom is kept and used as it is. It might be used in
   future. Note that some Ethernet drivers which _DOESN'T_ use mii(4) use
   ec_mii for keeping the if_media. Those should be changed in future.

 Whitespace fix (mainly tabify).

-No functional change:
  - KNF
  - u_int*_t -> uint*_t.

Pull up the following, requested by msaitohin ticket #1225:

	sys/dev/pci/ixgbe/ixgbe.c		1.175-1.178
	sys/dev/pci/ixgbe/ixv.c			1.110-1.111 via patch

- NetBSD currently uses traffic class 0 only. Other traffic classes
  aren't used yet. When IXGBE_TC_COUNTER_NUM is set to lower than
  IXGBE_DCB_MAX_TRAFFIC_CLASS (e.g. 1), other traffic classes' counters
  are not used. It means we don't generate evcnt for them and don't
  add the values in ixgbe_update_stats_counters().
- It's not required to calculate unused queues' statistics.
- Fix a bug that the VLAN HW tagging function is not correctly disabled
  when all vlan is detached.
- Fix a bug that VLAN HW tagging function is not correctly controlled
  on 82598.
- Control VLAN HW filter function correctly. Note that currently
  VLAN HW filter function doesn't work because NetBSD doesn't support
  it yet.
- Don't clear IXGBE_VLNCTRL_CFIEN bit When ETHERCAP_VLAN_HWFILTER is
  set. I think it's not required (and Linux doesn't do it). This change
  has no effect to NetBSD because ETHERCAP_VLAN_HWFILTER is not
  supported yet.

 s/pakcet/packet/ in comment.

- Simplily. Suggested by knakahara.
- Modify comment. Per queue VLAN enable flags is on 82599 and later.
- Fix typo in comment.

 Fix a bug that the VLAN HW tagging function is not correctly disabled when
all vlan is detached. Part of ixgbe.c rev. 1.177.

Pull up following revision(s) (requested by msaitoh in ticket #1199):

	sys/dev/pci/ixgbe/ixv.c: revision 1.109
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.54
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.52
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.174

  Fix a bug that if_link_state_change(ifp, LINK_STATE_DOWN) isn't called unless
link goes up at least one time. Without this change, never linkup-ed media
keeps LINK_STATE_UNKNOWN instead of LINK_STATE_DOWN.

 Fix a bug that if_link_state_change(ifp, LINK_STATE_DOWN) isn't called unless
link goes up at least one time. Without this change, never linkup-ed media
keeps LINK_STATE_UNKNOWN instead of LINK_STATE_DOWN.

Pull up following revision(s) (requested by msaitoh in ticket #1172):

	sys/dev/pci/nvme_pci.c: revision 1.26
	sys/dev/pci/xhci_pci.c: revision 1.21
	sys/dev/pci/ixgbe/ixv.c: revision 1.108
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.171
	sys/dev/pci/if_fxp_pci.c: revision 1.84
	sys/dev/pci/if_fxp_pci.c: revision 1.85
	sys/dev/pci/xhci_pci.c: revision 1.16

remove #ifdef DEBUG printf, it seems to have outlived it's usefulness

 -

  KNF. No functional change.

 -

  Nowadays some UEFI BIOSes don't enable some PCI devices' address decoding.
To resolve this problem, pci_map.c rev. 1.34-1.36 changed the
pci_mapreg_(sub)map()'s to set the decode bit if it's not set. It's good for
almost all drivers, but some other drivers don't use pci_mapreg_map().
In drivers which don't use pci_mapreg_map(), some of them explicitly enable
decoding but others don't. Add code to enable decoding to them.

  See also the following discussion:
	http://mail-index.netbsd.org/tech-kern/2017/03/22/msg021678.html

Sync with HEAD

 Nowadays some UEFI BIOSes don't enable some PCI devices' address decoding.
To resolve this problem, pci_map.c rev. 1.34-1.36 changed the
pci_mapreg_(sub)map()'s to set the decode bit if it's not set. It's good for
almost all drivers, but some other drivers don't use pci_mapreg_map().
In drivers which don't use pci_mapreg_map(), some of them expilicitly enable
decoding but others don't. Add code to enable decoding to them.

 See also the following discussion:
	http://mail-index.netbsd.org/tech-kern/2017/03/22/msg021678.html

Pull up following revision(s) (requested by msaitoh in ticket #1084):

	sys/dev/pci/ixgbe/ixv.c: revision 1.107
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.167

  Don't set LRO capability flag because we don't support it yet.

This doesn't change any behavior other than if_capabilities' visibility
(e.g. ifconfig).

Ssync with HEAD

 Don't set LRO capability flags because we don't support it yet.
This doesn't change any behavior other than if_capabilities' visibility
(e.g. ifconfig).

Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int.  The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER!  Some subsystems have

	#define min(a, b)	((a) < (b) ? (a) : (b))
	#define max(a, b)	((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX.  Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate.  But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all.  (Who knows, maybe in some cases integer
truncation is actually intended!)

Sync with HEAD

Pull up following revision(s) (requested by msaitoh in ticket #864):

	sys/dev/pci/ixgbe/ix_txrx.c			1.40-1.47 (patch)
	sys/dev/pci/ixgbe/ixgbe.c			1.148,1.149,1.151,
							1.152,1.154,
							1.155,1.157-1.160 (patch)
	sys/dev/pci/ixgbe/ixgbe.h			1.43,1.44,1.46,1.49 (patch)
	sys/dev/pci/ixgbe/ixgbe_netbsd.c		1.7 (patch)
	sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.8 (patch)
	sys/dev/pci/ixgbe/ixgbe_osdep.h			1.22 (patch)
	sys/dev/pci/ixgbe/ixv.c				1.100-1.104 (patch)
	sys/dev/pci/ixgbe/ixv.c				1.94,1.95,1.99 (patch)

 Remove unused structure entries. No functional change.
 -
 Remove unused IXGBE_FC_HI and IXGBE_FC_LO. The watermark of the flow control
is automatically calculated from the size of the packet buffer.
 -
 Use ixgbe_eitr_write() when writing the EITR for the link interrupt like
queue's EITR to write the register safely. This change is not relatively
so important than queue's EITR because link's EITR is written in if_init().
 -
 Don't free and reallocate bus_dmamem when it's not required. Currently,
the watchdog timer is completely broken and never fire (it's from FreeBSD
(pre iflib)). If the problem is fixed and watchdog fired, ixgbe_init() always
calls ixgbe_jcl_reinit() and it causes panic. The reason is that
ixgbe_local_timer1(it includes watchdog function) is softint and
xgbe_jcl_reinit() calls bus_dmamem*() functions. bus_dmamem*() can't be called
from interrupt context.

 One of the way to prevent panic is use worqueue for the timer, but it's
not a small change. (I'll do it in future).

 Another way is not reallocate dmamem if it's not required. If both the MTU
(rx_mbuf_sz in reality) and the number of RX descriptors are not changed, it's
not required to call bus_dmamem_{unmap,free}(). Even if we use workque, this
change save time of ixgbe_init().

 I have a code to fix broken watchdog timer but it sometime causes watchdog
timeout, so I don't commit it yet.
 -
Count some register correctly:
- QPRDC register is only for 82599 and newer.
- Count IXGBE_QPRDC, PX{ON,OFF}{T,R}XC[NT].
 The TQSMR register is not for receiving but for transmitting, so move the
initialization from ixgbe_initialize_receive_units() to
ixgbe_initialize_transmit_units(). No functional change.
 -
 Whitespace fix. No functional change.
 -
 Add rxd_nxck (Receive Descriptor next to check) read only sysctl.
 Don't check IFF_RUNNING in ixgbe_rxeof(). Doing break and leaving a deacriptor
with DD bit is worse than just processing the entry. And it's also racy to
check IFF_RUNNING in rxeof(). If you'd like to strictly obey IFF_RUNNING,
it would be better to do it in the upper layer.
 Same as DragonFly (a part of 79251f5ebe4cf9dd2f3e6aed590e09d756d39922).
 Add "bool txr_no_space" for TX descriptor shortage. Use it like IFF_OACTIVE.
 Clear que->disabled_count in {ixgbe,ixv}_init_locked(). Without this,
interrupt mask state and EIMS may mismatch and if_init doesn't recover
from TX/RX stall problem.
 This change itself doesn't fix PR#53294.
 -
 Add hw.ixgN.debug sysctl. "sysctl -w hw.ixgN.debug=1" dumps some registers
to console.
 -
Constify several variables in ixgbe/ so that they land in .rodata (1038
bytes).
 -
 Don't call ixgbe_rearm_queues() in ixgbe_local_timer1().
   ixgbe_enable_queue() and ixgbe_disable_queue() try to enable/disable queue
  interrupt safely. It has the internal counter. When a queue's MSI-X is
  received, ixgbe_msix_que() is called (IPL_NET). This function disable the
  queue's interrupt by ixgbe_disable_queue() and issues an softint.
  ixgbe_handle() queue is called by the softint (IPL_SOFTNET), process TX, RX
  and call ixgbe_enable_queue() at the end.

   ixgbe_local_timer1() is a callout and run always on CPU 0 (IPL_SOFTCLOCK).
  When ixgbe_rearm_queues() called, an MSI-X interrupt is issued for a specific
  queue. It may not CPU 0. If this interrupt's ixgbe_msix_que() is called and
  sofint_schedule() is called before the last sofint's softint_execute() is not
  called, the softint_schedule() fails because of SOFTINT_PENDING. It result
  in breaking ixgbe_{enable,disable}_queue()'s internal counter.
   ixgbe_local_timer1() is written not to call ixgbe_rearm_queues() if
  the interrupt is disabled, but it's called because of unknown bug or a race.

  One solution to avoid this problem is to not to use the internal counter,
  but it's little difficult. Another solution is stop using
  ixgbe_rearm_queues() at all. Essentially, ixgbe_rearm_queues() is not
  required (it was added in ixgbe.c rev. 1.43 (2016/12/01)).
  ixgbe_rearm_queues() helps for lost interrupt problem but I've never seen it
  other than ixgbe_rearm_queues() problem.

XXX pullup-8.

Pull up following revision(s) (requested by ozaki-r in ticket #843):

	sys/dev/pci/ixgbe/ixv.c: revision 1.101
	sys/net/if_bridge.c: revision 1.156
	sys/net/if_pppoe.c: revision 1.138
	sys/dev/pci/if_wm.c: revision 1.580
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.156
	sys/net/if_gif.c: revision 1.142

Ensure to call if_register after interface initializations finish

Comment out unused functions in order to fix Clang build

ixv_rearm_queues() and ixgbe_rearm_queues() are unused.

Sponsored by <The NetBSD Foundation>

 Don't call ixgbe_rearm_queues() in ixgbe_local_timer1().

   ixgbe_enable_queue() and ixgbe_disable_queue() try to enable/disable queue
  interrupt safely. It has the internal counter. When a queue's MSI-X is
  received, ixgbe_msix_que() is called (IPL_NET). This function disable the
  queue's interrupt by ixgbe_disable_queue() and issues an softint.
  ixgbe_handle() queue is called by the softint (IPL_SOFTNET), process TX, RX
  and call ixgbe_enable_queue() at the end.

   ixgbe_local_timer1() is a callout and run always on CPU 0 (IPL_SOFTCLOCK).
  When ixgbe_rearm_queues() called, an MSI-X interrupt is issued for a specific
  queue. It may not CPU 0. If this interrupt's ixgbe_msix_que() is called and
  sofint_schedule() is called before the last sofint's softint_execute() is not
  called, the softint_schedule() fails because of SOFTINT_PENDING. It result
  in breaking ixgbe_{enable,disable}_queue()'s internal counter.

   ixgbe_local_timer1() is written not to call ixgbe_rearm_queues() if
  the interrupt is disabled, but it's called because of unknown bug or a race.
  One solution to avoid this problem is to not to use the internal counter,
  but it's little difficult. Another solution is stop using
  ixgbe_rearm_queues() at all. Essentially, ixgbe_rearm_queues() is not
  required (it was added in ixgbe.c rev. 1.43 (2016/12/01)).
  ixgbe_rearm_queues() helps for lost interrupt problem but I've never seen it
  other than ixgbe_rearm_queues() problem.

XXX pullup-8.

Constify several variables in ixgbe/ so that they land in .rodata (1038
bytes).

 Clear que->disabled_count in {ixgbe,ixv}_init_locked(). Without this,
interrupt mask state and EIMS may mismatch and if_init doesn't recover
from TX/RX stall problem.

 This change itself doesn't fix PR#53294.

Ensure to call if_register after interface initializations finish

 Add "bool txr_no_space" for TX descriptor shortage. Use it like IFF_OACTIVE.

 Add rxd_nxck (Receive Descriptor next to check) read only sysctl.

Sync with HEAD

 Revert new watchdog timer commits. The new watchdog timer made stability
worse than before. It seems unknown problems exists.

http://mail-index.netbsd.org/source-changes/2018/05/08/msg095020.html
http://mail-index.netbsd.org/source-changes/2018/05/16/msg095240.html

Pull up following revision(s) (requested by msaitoh in ticket #824):

	sys/dev/pci/ixgbe/ixv.c: revision 1.97

Fix panic or hangup when "sysctl -w hw.ixgN.debug=1".
XXX pullup-8

 Fix panic or hangup when "sysctl -w hw.ixgN.debug=1".

XXX pullup-8

- Fix broken watchdog timer. This change detects TX device timeout correctly.
  NOTE: It's supporsed not to be called {ixgbe,ixv}_rearm_queues() in the
  timer. Those are not required if any chip have no bug. In reality,
  ixgbe_rearm_queues() is required on 82599 and newer chip AND other than
  queue 0 to prevent device timeout. When it occured, packet was sent but the
  descriptor's DD bit wasn't set even though IXGBE_TXD_CMD_EOP and
  IXGBE_TXD_CMD_RS were set. After forcing interrupt by writing EICS register
  in ixgbe_rearm_queues(), DD is set. Why? Is this an undocumented errata? It
  might be possible not call rearm_queues on 82598 or queue 0, we call in any
  cases in case the problem occurs. On ixv(4), I have not seen this problem yet
  (though I tested only on X550_X(Xeon D 12xx)'s virtual function), but we
  do rearm in case TX device timeout happen.
- ixv(4): Call callout_stop() earlier in ixv_stop() like ixgbe_stop().
- KNF.

Sync with HEAD

 Use ixgbe_eitr_write() when writing the EITR for the link interrupt like
queue's EITR to write the register safely. This change is not relatively
so important than queue's EITR because link's EITR is written in if_init().

 Remove unused structure entries. No functional change.

Sync with HEAD, resolve some conflicts

Pull up following revision(s) (requested by msaitoh in ticket #750):

	sys/dev/pci/ixgbe/ixgbe_x540.h: revision 1.8
	sys/dev/pci/ixgbe/ixgbe_82598.c: revision 1.11
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c: revision 1.6
	sys/dev/pci/ixgbe/ixgbe_82598.c: revision 1.12
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c: revision 1.7
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h: revision 1.6
	sys/dev/pci/ixgbe/ixgbe_fdir.h: revision 1.2
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h: revision 1.6
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c: revision 1.6
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.18
	sys/dev/pci/ixgbe/if_sriov.c: revision 1.3
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c: revision 1.7
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.19
	sys/dev/pci/ixgbe/ixgbe_mbx.h: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_dcb.c: revision 1.7
	sys/dev/pci/ixgbe/ixgbe_api.c: revision 1.19
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.16
	sys/dev/pci/ixgbe/ixgbe_dcb.c: revision 1.8
	sys/dev/pci/ixgbe/ixgbe_vf.c: revision 1.17
	sys/dev/pci/ixgbe/ixgbe_dcb.h: revision 1.6
	sys/dev/pci/ixgbe/if_bypass.c: revision 1.4
	sys/dev/pci/ixgbe/ixv.c: revision 1.91
	sys/dev/pci/ixgbe/ixgbe_rss.h: revision 1.4
	sys/dev/pci/ixgbe/ixv.c: revision 1.92
	sys/dev/pci/ixgbe/ixv.c: revision 1.93
	sys/dev/pci/ixgbe/ixgbe_osdep.c: revision 1.4
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.12
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_api.h: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.12
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.33
	sys/dev/pci/ixgbe/if_fdir.c: revision 1.2
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.34
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.39
	sys/dev/pci/ixgbe/ixgbe_vf.h: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_api.c: revision 1.20
	sys/dev/pci/ixgbe/ixgbe_sriov.h: revision 1.3
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.21
	sys/dev/pci/ixgbe/ixgbe_osdep.h: revision 1.21
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.22
	sys/dev/pci/ixgbe/ixgbe_82599.h: revision 1.7
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.144
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.145
	sys/dev/pci/ixgbe/ixgbe_phy.c: revision 1.16
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.146
	sys/dev/pci/ixgbe/ixgbe_phy.c: revision 1.17
	sys/dev/pci/ixgbe/ixgbe_82598.h: revision 1.8
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.41
	sys/dev/pci/ixgbe/ixgbe_mbx.c: revision 1.10
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.14
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.15
	sys/dev/pci/ixgbe/ixgbe_phy.h: revision 1.11

Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
 - Always schedule module intterrupt in ixgbe_config_link() when a device is
   SFP+ based.
 - Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
   It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
   NetBSD.
 - Simplify ixgbe_bp_wd_set(). No functional change.
 - Whitespace.

Sync with FreeBSD r331224 except ixv_if_update_admin_status()'s change:
 - Fix length of reading buffer when the command is Read Flash in
   ixgbe_host_interface_command().
 - Add missing start_hw() call in ixv_init_locked(). This is not a real bug
   because start_hw just set hw->adapter_stopped to false and anyone refer it.
 - Style change.
 - Update comment.

Read sc_if_flags after taking core lock. Same as if_wm.c rev. 1.418.

Read sc_if_flags after taking core lock. Same as if_wm.c rev. 1.418.

Sync with HEAD.  77 conflicts resolved - all of them $NetBSD$

Pull up following revision(s) (requested by msaitoh in ticket #690):
	sys/dev/pci/ixgbe/ixgbe_82599.c: revision 1.17
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.138
	sys/dev/pci/ixgbe/ixv.c: revision 1.90
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.139
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.19
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.11
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.38
	sys/dev/pci/ixgbe/ixgbe_api.h: revision 1.12
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.39
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.10
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.11
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.37
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.38
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.140
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.20
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.141
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.142
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.143
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.40
	sys/dev/pci/ixgbe/ixgbe_x550.h: revision 1.4

Fix the problem between eitr and link_speed.

In ixgbe_msix_que(), que->eitr_setting is limited to IXGBE_MIN_RSC_EITR_10G1G
when link_speed is 1Gbps or 10Gbps. However, que->eitr_setting is set to EITR
register in the *next* Tx/Rx interrupt. If link_speed changes from 100Mbps to
1Gbps ro 10Gbps, que->eitr_setting which is not limited can be set to EITR
register, that is, the problem fixed by ixgbe.c:r1.124 can occur in this case.

To fix this case, que->eitr_setting should be clear when link_speed is changed
or link state is changed.

Furthermore, expand the variants used for AIM (txr->bytes, txr->packets,
rxr->bytes and rxr->packets) from u32 to u64 to avoid wraparound which causes
que->eitr_setting calculation mistake.
XXX pullup-8

Don't write EIMC directly. It is required to manage with struct ix_queue status.
XXX pullup-8

-  Add missing IFM_NONE support. If a interface support linkdown,
  "ifconfig ixgN media none" drpos link.  Not all interface can do link down.
 Tested:
        82598 AT2 (T)
        92599 SF+(SFI) (X520-DA2)
        X540
        X550-T1
        X550EM_x (X10SDV-8C-TLN4F)
        X550EM_a (A2SDi-H-TP4F port 0, 1 (T))
 Doesn't work:
        X550EM_a (A2SDi-H-TP4F port 2, 3 (SFP+ (KR)))
        X550EM_a (MA10-ST0 port 2, 3 (SFP+ (SFI)))
        (Denverton SFP+ can't force link down because SFP+'s TX_DISABLE pin is
        pull down. Is there a way to shutdown SFP+ cage's power?)
 Not tested:
        82598 fiber.
- Change some functions static.

Avoid issues caused by sending old packets at next link-up time.

This modification consists by the following two parts.
    - drain packets in if_snd queue or corresponding txr->txr_interq
      when link_active == false in ifp->if_start(), ifp->if_transmit(),
      and deferred Tx processing
    - drain packets in if_snd queue and all of txr->txr_interq's
      at link-down time
ok by msaitoh@n.o.

 Fix long standing bug that 82598 SFP+ panics in ixgbe_handle_mod() because
hw->mac.ops.setup_sfp is NULL. This change is a part of FreeBSD r327031.

Fix a bug that "ifconfig ixgN down up" forgot IFM_NONE setting.
 - Don't assume autoneg == 0 is the first call of ixgbe_config_link().
   Check ifm_media, too.
 - Don't override autoneg_advertised in ixgbe_get_phy_id_fw() to not to
   be inconsistent with if_media value.

Sync with FreeBSD r331224 except ixv_if_update_admin_status()'s change:
 - Fix length of reading buffer when the command is Read Flash in
   ixgbe_host_interface_command().
 - Add missing start_hw() call in ixv_init_locked(). This is not a real bug
   because start_hw just set hw->adapter_stopped to false and anyone refer it.
 - Style change.
 - Update comment.

Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
 - Always schedule module intterrupt in ixgbe_config_link() when a device is
   SFP+ based.
 - Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
   It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
   NetBSD.
 - Simplify ixgbe_bp_wd_set(). No functional change.
 - Whitespace.

Pull up following revision(s) (requested by msaitoh in ticket #673):

	sys/dev/pci/ixgbe/if_sriov.c: revision 1.2
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.135
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.136
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.137
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.17
	sys/dev/pci/ixgbe/if_bypass.c: revision 1.3
	sys/dev/pci/ixgbe/ixgbe_common.c: revision 1.18
	sys/dev/pci/ixgbe/ixgbe_common.h: revision 1.10
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.36
	sys/dev/pci/ixgbe/ixgbe_osdep.c: revision 1.3
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.37
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.36
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.32
	sys/dev/pci/ixgbe/ixgbe_vf.h: revision 1.12
	sys/dev/pci/ixgbe/ixgbe_sriov.h: revision 1.2
	sys/dev/pci/ixgbe/ixgbe_osdep.h: revision 1.20
	sys/dev/pci/ixgbe/ixv.c: revision 1.88
	sys/dev/pci/ixgbe/ixgbe_phy.c: revision 1.15
	sys/dev/pci/ixgbe/ixv.c: revision 1.89
	sys/dev/pci/ixgbe/ixgbe_x540.c: revision 1.13
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.8
	sys/dev/pci/ixgbe/ixgbe_x550.c: revision 1.9

Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.

Some others (e.g. sfp cage interrupt and bypass adapter  stuff) will be merged
later:

- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
  and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
  NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
  X550 and newer on some environment. X550 changed FWSM bit definition.
  See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
  not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.
- Check offset correctly in ixgbe_get_oem_prod_version(). Note that this
  function is not used.
- Set PHY correctly in ixgbe_setup_mac_link_sfp_x550a() if a device is a
  C3000 KR SFP+.

Fix race about writing adapter->link_active for ixg(4).
adapter->link_active is updated by ixgbe_update_link_status() only.

The function is called from the following four functions.
    - ixgbe_media_status()
    - ixgbe_local_timer1()
    - ixgbe_stop()
    - ixgbe_handle_link()

The functions other than ixgbe_handle_link() call ixgbe_update_link_status()
with holding IXGBE_CORE_LOCK, however ixgbe_handle_link() calls it without
holding IXGBE_CORE_LOCK. That can cause race. So, add IXGBE_CORE_LOCK to
ixgbe_handle_link().

Tested by msaitoh@n.o and me.

Fix race about writing adapter->link_active for ixv(4).
adapter->link_active is updated by ixv_update_link_status() only.

The function is called from the following two functions.
    - ixv_media_status()
    - ixv_handle_link()

ixv_media_status() calls ixv_update_link_status() with holding
IXGBE_CORE_LOCK, however ixv_handle_link() calls it without
holding IXGBE_CORE_LOCK, the same as ixg(4).

ok by msaitoh@n.o.

- Add link related softint's counter.
- Fix indent.

Resolve conflicts between branch and HEAD

Don't write EIMC directly. It is required to manage with struct ix_queue status.

XXX pullup-8

Synch with HEAD, resolve conflicts

Fix race about writing adapter->link_active for ixv(4).

adapter->link_active is updated by ixv_update_link_status() only.
The function is called from the following two functions.
    - ixv_media_status()
    - ixv_handle_link()

ixv_media_status() calls ixv_update_link_status() with holding
IXGBE_CORE_LOCK, however ixv_handle_link() calls it without
holding IXGBE_CORE_LOCK, the same as ixg(4).

ok by msaitoh@n.o.

Synch with HEAD

 Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.
Some others (e.g. sfp cage interrupt and bypass adapter  stuff) will be merged
later:
- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
  and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
  NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
  X550 and newer on some environment. X550 changed FWSM bit definition.
  See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
  not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.

Pull up following revision(s) (requested by msaitoh in ticket #626):
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.134
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.35
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.35
	sys/dev/pci/ixgbe/ixv.c: revision 1.87
  Make some event counters MP safe. Now all of the event counters are
MP safe.

 Make some event counters MP safe. Now all of the event counters are
MP safe.

Pull up following revision(s) (requested by knakahara in ticket #612):
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.129-1.133
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.34
	sys/dev/pci/ixgbe/ixv.c: revision 1.85,1.86

- Make "Handled queue in softint" and "Requeued in softint" evcnt(9) per queue
  and count them correctly.
- Remove #if 0'ed code.

Don't increment que->req.ev_count in MSI-X interrupt because it's not
reschedule.

Fix another poll mode assumption breaking. Implemented by msaitoh@n.o, I just commit by proxy.
ixgbe_rearm_queues() writes EICS register(s). 82599, X540 and X550
specifications say "Following a write of 1b to any bit in the EICS register
(interrupt cause set), its corresponding bit in the EIMS register is auto
set as well enabling its interrupt." in "Extended Interrupt Auto Mask Enable
(EIAM) Register" section. That is, ixgbe_rearm_queues() causes interrupts
regardless of the status managed by ixgbe_enable_queue()/ixgbe_disable_queue().

That can break poll mode assumption.

In fact, the problem occurs in the following situation
    - CPU#A has high load traffic, in contrast, CPU#B has not so high load traffic
    - CPU#A is occurred interrupt by its NIC queue
      - CPU#A calls ixgbe_disable_queue() in interrupt handler(ixgbe_msix_que())
      - CPU#A kick softint handler(ixgbe_handle_que())
        - CPU#A begins softint
        - CPU#A's NIC queue is set que->txr->busy flag
        - With some reason, CPU#A can do ixg interrupt handler
          E.g. when one of CPU#A's softnet handlers sleeps, ipl is lowered
    - CPU#B starts callout
      - CPU#B calls ixgbe_local_timer1()
        - CPU#B writes EICS bit corresponding CPU#A's NIC queue bit
    - CPU#A's NIC queue causes interrupt whie CPU#A is running in poll mode
      - CPU#A calls ixgbe_disable_queue() in interrupt handler *again*
    - CPU#A has done polling, and then CPU#A calls ixgbe_enable_queue() *once*
    - CPU#A's NIC queue interrupt is disabled until ixg is detached as
      ixgbe_disable_queue() is called twice though ixgbe_disable_queue() is
      called once only
NOTE:
82598 does not say so, but it is treated in the same way because of no harm.
By the way, we will refactor ixgbe_local_timer(watchdog processing) later.

Fix INTx/MSI handler did not schedule workqueue. Pointed out by msaitoh@n.o.

Reduce duplicated code which schedule deferred packet processing. No functional change.

 Don't increment que->req.ev_count in MSI-X interrupt because it's not
reschedule.

- Make "Handled queue in softint" and "Requeued in softint" evcnt(9) per queue
  and count them correctly.
- Remove #if 0'ed code.

Pull up following revision(s) (requested by knakahara in ticket #605):
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.32
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.33
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.34
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.128
	sys/dev/pci/ixgbe/ixv.c: revision 1.83
	sys/dev/pci/ixgbe/ixv.c: revision 1.84

Add hw.ixvM.q[01].{interrupt_rate,[tr]xd_head,[tr]xd_tail} sysctls as ixg(4).

ixg(4) supports workqueue poll mode, but not enabled by default yet. (that is, the default behavior is *not* changed)

At the time of high load near the wire rate, the turnaround time
of update/delete processing such as "ifconfig ixg0 inet XXX" or
"ifconfig ixg0 delete" is very long. The main reason is CPU
starvation caused by ixg(4)'s softint poll mode. ixg(4) uses
workqueue poll mode instead of softint poll mode, so that this
problem will be fix.

This change may cause performance issues, so it is not enabled
by default yet. Although there are that issues, if you want to use
workqueue poll mode, do "sysctl -w hw.ixgXX.txrx_workqueue=1" while
there is no traffic on the ixgXX.

ok by msaitoh@n.o.

ixv(4) also supports workqueue poll mode, but not enabled by default yet, either.
ok by msaitoh@n.o.

Move the location of "struct work" as FreeBSD's "struct task" in ixgbe.h.
No functional change.

ixv(4) also supports workqueue poll mode, but not enabled by default yet, either.

ok by msaitoh@n.o.

Pull up following revision(s) (requested by knakahara in ticket #597):
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.31
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.33
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.127
	sys/dev/pci/ixgbe/ixv.c: revision 1.82
Fix poll mode assumption breaking.
ixgbe_{enable,disable}_intr() forcibly enable/disable all interrupts
regardless of current state. That can break poll mode assumption,
that is, queue interrupts must not occur while polling Tx/Rx rings.
E.g. "ifconfig ixg0 delete && ifconfig ixg0 192.168.0.1" on heavy
load traffic can causes this issue.
This fix may have 1% or 2% performance impact at short packets.
XXX
ixgbe_rearm_queues() which is called only via watchdog can also break
this poll mode assumption because writing EICS casues interrupts
immediately when interrupt auto mask enabled.
We will fix it with other issues about watchdog later.
ok by msaitoh@n.o.
Apply ixgbe.c:r1.127 to ixv.c. Pointed out by msaitoh@n.o.

 Add hw.ixvM.q[01].{interrupt_rate,[tr]xd_head,[tr]xd_tail} sysctls as ixg(4).

Pull up following revision(s) (requested by msaitoh in ticket #592):
	sys/dev/pci/ixgbe/ixv.c: revision 1.79
	sys/dev/pci/ixgbe/ixgbe.h: revision 1.30
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.31
	sys/dev/pci/ixgbe/ix_txrx.c: revision 1.32
	sys/dev/pci/ixgbe/ixgbe_type.h: revision 1.31
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.120
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.121
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.123
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.124
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.125
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.126
	sys/dev/pci/ixgbe/ixv.c: revision 1.80
	sys/dev/pci/ixgbe/ixv.c: revision 1.81
 CID-1427719: Integer handling issues  (BAD_SHIFT). Print bus/slot info
correctly on 82599_SFP_SF_QP(DID 0x154a) and 82599_QSFP_SF_QP(DID 0x1558).
 Fix a bug that RX may stall on heavy load on ixg(4). ixgbe_rxeof() has loop
limit and the function returns true if a packet are still in the RX ring.
ixgbe_handle_que() didn't check the return value. Check the return vaule
and issue a softint.
 This bug is derived from FreeBSD and ixv(4) has no this bug.
XXX pullup-8
 Fix a bug that the first call of ifflags_cb() causes linkdown. The first
call of ix(gbe|v)_ifflags_cb() refered uninitialized adapter->if_flags.
adapter->if_flags should be initialized in the end of xxx_init().
XXX pullup-[678] (ixgbe)
XXX pullup-8 (ixv)
- Fix a bug that RX may stall on heavy load on ixg(4) derived from FreeBSD's
 AIM (Auto Interrupt Moderation) bug.
 When I use a machine as a NFS client, sometimes one of queue pairs doesn't
 get any interrupt other than every second tick via ixgbe_local_timer1().
 When the problem occured, the queue pair's hw.ixgM.qN.interrupt_rate is
 always 500000. When this problem occuring, set hw.ixgM.qN.interrupt_rate lower
 than 166667 recover from stall. i.e.:
  sysctl -w hw.ixgM.qN.interrupt_rate=166667 (don't revocer)
  sysctl -w hw.ixgM.qN.interrupt_rate=166666 (recover)
  Relatios between the interrupt_rate and EICR's ITR_INTERVAL field is as
 follows:
 int_rate | EICR[11:0]   | interval in us | recover |
          |(ITR_INTERVAL)| (10G and 1G)   |         |
 ---------+--------------+----------------+---------+
   500000 | 0x008(0)     |              2 |     not |
   166667 | 0x010(1)     |              4 |     not |
   166666 | 0x018(2)     |              6 | recover |
  The reason why int_rate becomes 500000 is that xgbe_tx_eof() doesn't
 increment rxr->packets(*1). Even if we fix rxr->packets' bug, interrupt_rate
 might become greater than 166666 and it might cause stall.
  While reading datasheets, knakahara noticed a section titled with "ITR
 Affect on RSC Functionality". It says "When RSC is enabled on specific RX
 queues, the associated ITR interval with these queus must be enabled and must
 be larger (in time uints) than RSC delay". Currently, RSC_DELAY field in the
 GPIE register is 0 and it means 4us for 10G and 1G. The greater ITR_INTERVAL
 value of 4us is 6us == 166666. Yes, BINGO!
  This description is noted in 82599 and newer datasheets and not in 82598
 datasheet. I don't know if 82598 has this limitation but, I apply this
 limitation all of chips.
 (*1) Note that this bug is going to be fixed in the next commit to distinct
 between two different bugs.
- The bitfield of EITR register is different between 82598 and others.
 Only ixgbe_msix_que() taken care of it. Make new function ixgbe_eitr_write()
 and use it in all of functions which modify ITR_INTERVAL.
XXX pullup-8
 Increment rxr->packets correctly in ixgbe_rxeof() to calculate ITR value
of AIM (Auto Interrupt Moderation) correctly. See also ixgbe.c rev. 1.124.
XXX pullup-8
Improve a comment about reading EICS register defined write-only by spec.
It seems that is workaround for silicon errata.
ok by msaitoh@n.o.
- Apply ixgbe.c rev. 1.124 to ixv.c. Fix a bug that RX may stall on heavy load
 on ixv(4) derived from FreeBSD's AIM (Auto Interrupt Moderation) bug.
 ITR_INTERVAL value must be larger than 4us.
- The bitfield of EITR register is different between 82598 and others.
 ixv.c had a bug that it accessed 82598's way even though only 82599 and
 newer support virtual function. Fix it using with new ixv_eitr_write()
 function.
 Fix a potential bug that TX/RX might stall when TX rate limit reached.
This change is almost the same as the RX rate limit bug fix in ixgbe.c
rev. 1.121. I've never got any stall, but this must be a bug.

Apply ixgbe.c:r1.127 to ixv.c. Pointed out by msaitoh@n.o.

Pull up following revision(s) (requested by knakahara in ticket #567):
	distrib/sets/lists/comp/mi: 1.2182-1.2183
	sys/dev/pci/if_wm.c: 1.564
	sys/dev/pci/ixgbe/ixgbe.c: 1.122
	sys/dev/pci/ixgbe/ixgbe_rss.h: 1.3
	sys/dev/pci/ixgbe/ixv.c: 1.78
	sys/net/Makefile: 1.35-1.36
	sys/net/files.net: 1.15
	sys/net/rss_config.c: 1.1
	sys/net/rss_config.h: 1.1
Introduce very simple Receive Side Scaling (RSS) utility.
ok by msaitoh@n.o.
--
Apply RSS utility to wm(4).
ok by msaitoh@n.o.
--
Apply RSS utility to ixg(4) and ixv(4).
ok by msaitoh@n.o.
--
Fix build failure, sorry.
--
Currently, it is not necessary to install rss_config.h. Pointed out by msaitoh@n.o.

 Fix a potential bug that TX/RX might stall when TX rate limit reached.
This change is almost the same as the RX rate limit bug fix in ixgbe.c
rev. 1.121. I've never got any stall, but this must be a bug.

- Apply ixgbe.c rev. 1.124 to ixv.c. Fix a bug that RX may stall on heavy load
 on ixv(4) derived from FreeBSD's AIM (Auto Interrupt Moderation) bug.
 ITR_INTERVAL value must be larger than 4us.

- The bitfield of EITR register is different between 82598 and others.
 ixv.c had a bug that it accessed 82598's way even though only 82599 and
 newer support virtual function. Fix it using with new ixv_eitr_write()
 function.

 Fix a bug that the first call of ifflags_cb() causes linkdown. The first
call of ix(gbe|v)_ifflags_cb() refered uninitialized adapter->if_flags.
adapter->if_flags should be initialized in the end of xxx_init().

XXX pullup-[678] (ixgbe)
XXX pullup-8 (ixv)

Apply RSS utility to ixg(4) and ixv(4).

ok by msaitoh@n.o.

Pull up following revision(s) (requested by msaitoh in ticket #490):
	sys/dev/pci/ixgbe/ixgbe.c: revision 1.116-1.119
	sys/dev/pci/ixgbe/ixv.c: revision 1.76-1.77
Don't panic when resource shortage occured. Fixes PR#52820 reported by
kardel@:
 - Don't use if_free() because ixgbe(4) don't use if_alloc().
 - Move location of {ixgbe,ixv}_setup_interface() call at a position that
   any error don't occur. One of the reason is that it should be. Another
   reason is that it's hard to call ether_ifdetach() and if_detach() when
   cold == 1 (because of pserialize_perform, xc_wait, timing of domaininit
   and maybe more).
--
Don't panic when resource shortage occured. Like ixgbe.c rev. 1.116.
 - Move location of {ixgbe,ixv}_setup_interface() call at a position that
   any error don't occur. One of the reason is that it should be. Another
   reason is that it's hard to call ether_ifdetach() and if_detach() when
   cold == 1 (because of pserialize_perform, xc_wait, timing of domaininit
   and maybe more).
--
Fix panic when only link interrupt can't be established.
--
- If MSI can't be used on some environment, fallback to INTx correctly.
- Use single vector MSI when number of CPU is 1 to save interrupt slot.
--
Fallback from MSI-X to MSI or INTx if MSI-X setup failed.

Pull up following revision(s) (requested by ozaki-r in ticket #456):
	sys/arch/arm/sunxi/sunxi_emac.c: 1.9
	sys/dev/ic/dwc_gmac.c: 1.43-1.44
	sys/dev/pci/if_iwm.c: 1.75
	sys/dev/pci/if_wm.c: 1.543
	sys/dev/pci/ixgbe/ixgbe.c: 1.112
	sys/dev/pci/ixgbe/ixv.c: 1.74
	sys/kern/sys_socket.c: 1.75
	sys/net/agr/if_agr.c: 1.43
	sys/net/bpf.c: 1.219
	sys/net/if.c: 1.397, 1.399, 1.401-1.403, 1.406-1.410, 1.412-1.416
	sys/net/if.h: 1.242-1.247, 1.250, 1.252-1.257
	sys/net/if_bridge.c: 1.140 via patch, 1.142-1.146
	sys/net/if_etherip.c: 1.40
	sys/net/if_ethersubr.c: 1.243, 1.246
	sys/net/if_faith.c: 1.57
	sys/net/if_gif.c: 1.132
	sys/net/if_l2tp.c: 1.15, 1.17
	sys/net/if_loop.c: 1.98-1.101
	sys/net/if_media.c: 1.35
	sys/net/if_pppoe.c: 1.131-1.132
	sys/net/if_spppsubr.c: 1.176-1.177
	sys/net/if_tun.c: 1.142
	sys/net/if_vlan.c: 1.107, 1.109, 1.114-1.121
	sys/net/npf/npf_ifaddr.c: 1.3
	sys/net/npf/npf_os.c: 1.8-1.9
	sys/net/rtsock.c: 1.230
	sys/netcan/if_canloop.c: 1.3-1.5
	sys/netinet/if_arp.c: 1.255
	sys/netinet/igmp.c: 1.65
	sys/netinet/in.c: 1.210-1.211
	sys/netinet/in_pcb.c: 1.180
	sys/netinet/ip_carp.c: 1.92, 1.94
	sys/netinet/ip_flow.c: 1.81
	sys/netinet/ip_input.c: 1.362
	sys/netinet/ip_mroute.c: 1.147
	sys/netinet/ip_output.c: 1.283, 1.285, 1.287
	sys/netinet6/frag6.c: 1.61
	sys/netinet6/in6.c: 1.251, 1.255
	sys/netinet6/in6_pcb.c: 1.162
	sys/netinet6/ip6_flow.c: 1.35
	sys/netinet6/ip6_input.c: 1.183
	sys/netinet6/ip6_output.c: 1.196
	sys/netinet6/mld6.c: 1.90
	sys/netinet6/nd6.c: 1.239-1.240
	sys/netinet6/nd6_nbr.c: 1.139
	sys/netinet6/nd6_rtr.c: 1.136
	sys/netipsec/ipsec_output.c: 1.65
	sys/rump/net/lib/libnetinet/netinet_component.c: 1.9-1.10
kmem_intr_free kmem_intr_[z]alloced memory
the underlying pools are the same but api-wise those should match
Unify IFEF_*_MPSAFE into IFEF_MPSAFE
There are already two flags for if_output and if_start, however, it seems such
MPSAFE flags are eventually needed for all if_XXX operations. Having discrete
flags for each operation is wasteful of if_extflags bits. So let's unify
the flags into one: IFEF_MPSAFE.
Fortunately IFEF_*_MPSAFE flags have never been included in any releases, so
we can change them without breaking backward compatibility of the releases
(though the kernel version of -current should be bumped).
Note that if an interface have both MP-safe and non-MP-safe operations at a
time, we have to set the IFEF_MPSAFE flag and let callees of non-MP-safe
opeartions take the kernel lock.
Proposed on tech-kern@ and tech-net@
Provide macros for softnet_lock and KERNEL_LOCK hiding NET_MPSAFE switch
It reduces C&P codes such as "#ifndef NET_MPSAFE KERNEL_LOCK(1, NULL); ..."
scattered all over the source code and makes it easy to identify remaining
KERNEL_LOCK and/or softnet_lock that are held even if NET_MPSAFE.
No functional change
Hold KERNEL_LOCK on if_ioctl selectively based on IFEF_MPSAFE
If IFEF_MPSAFE is set, hold the lock and otherwise don't hold.
This change requires additions of KERNEL_LOCK to subsequence functions from
if_ioctl such as ifmedia_ioctl and ifioctl_common to protect non-MP-safe
components.
Proposed on tech-kern@ and tech-net@
Ensure to hold if_ioctl_lock when calling if_flags_set
Fix locking against myself on ifpromisc
vlan_unconfig_locked could be called with holding if_ioctl_lock.
Ensure to not turn on IFF_RUNNING of an interface until its initialization completes
And ensure to turn off it before destruction as per IFF_RUNNING's description
"resource allocated". (The description is a bit doubtful though, I believe the
change is still proper.)
Ensure to hold if_ioctl_lock on if_up and if_down
One exception for if_down is if_detach; in the case the lock isn't needed
because it's guaranteed that no other one can access ifp at that point.
Make if_link_queue MP-safe if IFEF_MPSAFE
if_link_queue is a queue to store events of link state changes, which is
used to pass events from (typically) an interrupt handler to
if_link_state_change softint. The queue was protected by KERNEL_LOCK so far,
but if IFEF_MPSAFE is enabled, it becomes unsafe because (perhaps) an interrupt
handler of an interface with IFEF_MPSAFE doesn't take KERNEL_LOCK. Protect it
by a spin mutex.
Additionally with this change KERNEL_LOCK of if_link_state_change softint is
omitted if NET_MPSAFE is enabled.
Note that the spin mutex is now ifp->if_snd.ifq_lock as well as the case of
if_timer (see the comment).
Use IFADDR_WRITER_FOREACH instead of IFADDR_READER_FOREACH
At that point no other one modifies the list so IFADDR_READER_FOREACH
is unnecessary. Use of IFADDR_READER_FOREACH is harmless in general though,
if we try to detect contract violations of pserialize, using it violates
the contract. So avoid using it makes life easy.
Ensure to call if_addr_init with holding if_ioctl_lock
Get rid of outdated comments
Fix build of kernels without ether
By throwing out if_enable_vlan_mtu and if_disable_vlan_mtu that
created a unnecessary dependency from if.c to if_ethersubr.c.
PR kern/52790
Rename IFNET_LOCK to IFNET_GLOBAL_LOCK
IFNET_LOCK will be used in another lock, if_ioctl_lock (might be renamed then).
Wrap if_ioctl_lock with IFNET_* macros (NFC)
Also if_ioctl_lock perhaps needs to be renamed to something because it's now
not just for ioctl...
Reorder some destruction routines in if_detach
- Destroy if_ioctl_lock at the end of the if_detach because it's used in various
  destruction routines
- Move psref_target_destroy after pr_purgeif because we want to use psref in
  pr_purgeif (otherwise destruction procedures can be tricky)
Ensure to call if_mcast_op with holding IFNET_LOCK
Note that CARP doesn't deal with IFNET_LOCK yet.
Remove IFNET_GLOBAL_LOCK where it's unnecessary because IFNET_LOCK is held
Describe which lock is used to protect each member variable of struct ifnet
Requested by skrll@
Write a guideline for converting an interface to IFEF_MPSAFE
Requested by skrll@
Note that IFNET_LOCK must not be held in softint
Don't set IFEF_MPSAFE unless NET_MPSAFE at this point
Because recent investigations show that interfaces with IFEF_MPSAFE need to
follow additional restrictions to work with the flag safely. We should enable it
on an interface by default only if the interface surely satisfies the
restrictions, which are described in if.h.
Note that enabling IFEF_MPSAFE solely gains a few benefit on performance because
the network stack is still serialized by the big kernel locks by default.

Pull up following revision(s) (requested by msaitoh in ticket #441):
	sys/dev/pci/files.pci: 1.390
	sys/dev/pci/ixgbe/if_bypass.c: new
	sys/dev/pci/ixgbe/if_fdir.c: new
	sys/dev/pci/ixgbe/if_sriov.c: new
	sys/dev/pci/ixgbe/ixgbe_bypass.h: new
	sys/dev/pci/ixgbe/ixgbe_fdir.h: new
	sys/dev/pci/ixgbe/ixgbe_features.h: new
	sys/dev/pci/ixgbe/ixgbe_netmap.c: new
	sys/dev/pci/ixgbe/ixgbe_netmap.h: new
	sys/dev/pci/ixgbe/ixgbe_rss.h: new
	sys/dev/pci/ixgbe/ixgbe_sriov.h: new
	sys/dev/pci/ixgbe/ix_txrx.c: 1.28, 1.30 via patch
	sys/dev/pci/ixgbe/ixgbe.c: 1.97-1.105, 1.107-1.111, 1.113-1.115
	sys/dev/pci/ixgbe/ixgbe.h: 1.26-1.29 via patch
	sys/dev/pci/ixgbe/ixv.c: 1.57-1.58, 1.60-1.61, 1.63-1.64, 1.66-1.72, 1.75 via patch
	sys/dev/pci/ixgbe/ixgbe_api.c: 1.17-1.18 via patch
	sys/dev/pci/ixgbe/ixgbe_type.h: 1.26-1.30 via patch
	sys/dev/pci/ixgbe/ixgbe_82598.c: 1.9-1.10 via patch
	sys/dev/pci/ixgbe/ixgbe_82598.h: 1.6-1.7
	sys/dev/pci/ixgbe/ixgbe_82599.c: 1.15-1.16 via patch
	sys/dev/pci/ixgbe/ixgbe_82599.h: 1.5-1.6
	sys/dev/pci/ixgbe/ixgbe_api.h: 1.10-1.11 via patch
	sys/dev/pci/ixgbe/ixgbe_common.c: 1.14-1.16 via patch
	sys/dev/pci/ixgbe/ixgbe_common.h: 1.8-1.9 via patch
	sys/dev/pci/ixgbe/ixgbe_dcb.c: 1.5-1.6
	sys/dev/pci/ixgbe/ixgbe_dcb.h: 1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c: 1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h: 1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c: 1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h: 1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_mbx.c: 1.7-1.9 via patch
	sys/dev/pci/ixgbe/ixgbe_mbx.h: 1.11-1.12 via patch
	sys/dev/pci/ixgbe/ixgbe_osdep.c: 1.2
	sys/dev/pci/ixgbe/ixgbe_osdep.h: 1.18-1.19 via patch
	sys/dev/pci/ixgbe/ixgbe_phy.c: 1.12-1.14 via patch
	sys/dev/pci/ixgbe/ixgbe_phy.h: 1.9-1.10 via patch
	sys/dev/pci/ixgbe/ixgbe_vf.c: 1.13-1.15 via patch
	sys/dev/pci/ixgbe/ixgbe_vf.h: 1.9-1.11 via patch
	sys/dev/pci/ixgbe/ixgbe_x540.c: 1.10-1.12
	sys/dev/pci/ixgbe/ixgbe_x540.h: 1.6-1.7
	sys/dev/pci/ixgbe/ixgbe_x550.c: 1.6-1.7
	sys/dev/pci/ixgbe/ixgbe_x550.h: 1.3
Sync with FreeBSD's up to r326022, fix bugs and some improvements:
- Don't limit number of queue pair to 8. Take chip's max TX queues and
  max RX queues into account.
- Add C3000 (Denverton) support.
- Add bypass function support for bypass adapters.
- Change EEE sysctl.
- Remove thermal test sysctl.
- ixv(4): set RSS mapping.
- ixv(4): Add TSOv6.
- ixv(4): Fix hardware counter.
- ixv(4): Make TX/RX descriptors size the same as ixg(4).
- ixv(4): Print device name in ixv_attach().
- ixv(4): Make mailbox statistic counters evcnt(9).
- ixv(4): Fix a problem that mailbox interrupt never occurred.
- ixv(4): Don't check neither VTEICR nor VTEICS same as Linux. It
  seems that both registers can't be used to check which MSI-X vector
  is triggered.
- ixv(4): Print Mailbox API version.
- Print driver feature capabilities and enable bits when verbose boot.
- Fix a bug that X550EM_A (Denverton) can't force 10BaseT.
- Remove all half-duplex setting because of the chip specification.
- Add 2.5GBASE-T and 5GBASE-T support.
- Set ifm_baudrate correctly.
- Protect ec_multi* with mutex like other MP safe Ethernet drivers.
- On X550 and newer, print NVM Image Version with %u.%02x.
- On X540, print PHY FW Revision with %u.%x. 0x4030 will be printed
  as "Revision 4.3 ID 0x0"
- Fix value check of OEM_NVM_IMAGE_VER.
- Print PHY ID only for copper PHY.
- Fix a bug that X550 and newer didn't linkup if it forces
  100BaseTX-FDX or 10BaseT-FDX.
- Fix a bug that RX may accesses freed area. It also fixes a problem
  that if_init() takes long time on many core machine.
- Check ETHERCAP_VLAN_HWTAGGING in ixgbe_setup_vlan_hw_support(). This
  change has no influence to netbsd because it's enabled by default
  and NetBSD has no API to disable it.
- Whitespace fix.

 Fix panic when only link interrupt can't be established.

 Don't panic when resource shortage occured. Like ixgbe.c rev. 1.116.
 - Move location of {ixgbe,ixv}_setup_interface() call at a position that
   any error don't occur. One of the reason is that it should be. Another
   reason is that it's hard to call ether_ifdetach() and if_detach() when
   cold == 1 (because of pserialize_perform, xc_wait, timing of domaininit
   and maybe more).

Pull up following revision(s) (requested by msaitoh in ticket #427):
	sys/arch/amiga/dev/if_bah_zbus.c: 1.17
	sys/arch/arm/broadcom/bcm53xx_eth.c: 1.30
	sys/arch/powerpc/booke/dev/pq3etsec.c: 1.32
	sys/arch/usermode/dev/if_veth.c: 1.9
	sys/dev/ic/an.c: 1.66
	sys/dev/ic/athn.c: 1.17
	sys/dev/ic/atw.c: 1.162
	sys/dev/ic/bwi.c: 1.33
	sys/dev/ic/dwc_gmac.c: 1.41-1.42
	sys/dev/ic/malo.c: 1.10
	sys/dev/ic/rt2560.c: 1.31
	sys/dev/ic/rt2661.c: 1.36
	sys/dev/ic/rt2860.c: 1.29
	sys/dev/ic/rtw.c: 1.127
	sys/dev/ic/rtwvar.h: 1.46
	sys/dev/ic/smc90cx6.c: 1.71
	sys/dev/ic/smc90cx6var.h: 1.12
	sys/dev/ic/wi.c: 1.244
	sys/dev/pci/if_ipw.c: 1.66
	sys/dev/pci/if_iwi.c: 1.104
	sys/dev/pci/if_iwm.c: 1.76
	sys/dev/pci/if_iwn.c: 1.86
	sys/dev/pci/if_rtwn.c: 1.13
	sys/dev/pci/if_wm.c: 1.541
	sys/dev/pci/if_wpi.c: 1.79
	sys/dev/pci/ixgbe/ixgbe.c: 1.106
	sys/dev/pci/ixgbe/ixv.c: 1.73 via patch
	sys/dev/pcmcia/if_malo_pcmcia.c: 1.15
	sys/dev/scsipi/if_se.c: 1.95
	sys/dev/usb/if_upl.c: 1.60
	sys/net/if.c: 1.396
	sys/net/if.h: 1.241
	sys/net/if_arc.h: 1.23
	sys/net/if_arcsubr.c: 1.78
	sys/net/if_bridge.c: 1.136-1.137
	sys/net/if_etherip.c: 1.39
	sys/net/if_faith.c: 1.56
	sys/net/if_gif.c: 1.131
	sys/net/if_loop.c: 1.96
	sys/net/if_mpls.c: 1.30
	sys/net/if_pppoe.c: 1.129
	sys/net/if_srt.c: 1.27
	sys/net/if_stf.c: 1.102
	sys/net/if_tap.c: 1.100
	sys/net/if_vlan.c: 1.105
	sys/netinet/ip_carp.c: 1.91
	sys/rump/net/lib/libshmif/if_shmem.c: 1.73-1.74
	sys/rump/net/lib/libvirtif/if_virt.c: 1.55-1.56
 if_initalize() and if_attach() failed when resource allocation failed
(e.g. allocating softint). Without this change, it panics. It's bad because
resource shortage really occured when a lot of pseudo interface is created.
To avoid this problem, don't panic and change return value of if_initialize()
and if_attach() to int. Caller fanction will be recover from error cleanly by
checking the return value.
 Return if bah_attach_subr() failed.
 If if_attach() failed in the attach function, return.
- If if_initialize() failed in the attach function, free resources and return.
- Add some missing frees in bridge_clone_destroy().
- KNF
 If error occured in bcmeth_ccb_attach(), free resources and return.
 If error occured in pq3etsec_attach(), free resources and return.
 If error occured in the attach function, free resources and return.
- If if_initialize() failed in athn_attach(), free resources and return.
- Add missing pmf_event_deregister() in athn_detach().
- Free resources correctly on some errors in atw_attach().
- Use apint*() insread of printf() in the attach function.
 If if_initialize() failed in the attach function, return.
- If if_initialize() failed in the attach function, free resources and return.
- Add missing dwc_gmac_free_dma_rings() and mutex_destroy() when attach
   failed.
- If if_initialize() failed in the attach function, free resources and return.
- ifp is always not NULL in iwi_detach(). Check correctly with ifp->if_softc.
- If if_initialize() failed in the attach function, free resources and return.
- Fix error path in the attach function correctly.
 If if_initialize() failed in the attach function, free resources and return.
 If if_attach() failed in the attach function, free resources and return.
 - If if_initialize() failed in the attach function, free resources and return.
 - KNF
 - If if_attach() failed in the attach function, free resources and return.
 - KNF
 Fix compile error.
 Fix compile error.
We don't need '&mii', but just 'mii' for mii_detach().
Don't free sc_rthash twice

Sync with FreeBSD's r326022. All of the following changes have no influence
to netbsd:
- Check ETHERCAP_VLAN_HWTAGGING in ixgbe_setup_vlan_hw_support(). This change
  has no influence to netbsd because it's enabled by default and NetBSD has
  no API to disable it.
- Fix for netmap module.
- Remove never defined UDP_IPV4_EX
- Add SPDX-License-Identifier

update from HEAD

Unify IFEF_*_MPSAFE into IFEF_MPSAFE

There are already two flags for if_output and if_start, however, it seems such
MPSAFE flags are eventually needed for all if_XXX operations. Having discrete
flags for each operation is wasteful of if_extflags bits. So let's unify
the flags into one: IFEF_MPSAFE.

Fortunately IFEF_*_MPSAFE flags have never been included in any releases, so
we can change them without breaking backward compatibility of the releases
(though the kernel version of -current should be bumped).

Note that if an interface have both MP-safe and non-MP-safe operations at a
time, we have to set the IFEF_MPSAFE flag and let callees of non-MP-safe
opeartions take the kernel lock.

Proposed on tech-kern@ and tech-net@

 If if_initialize() failed in the attach function, free resources and return.

 Protect ec_multi* with mutex like other MP safe Ethernet drivers.
See if_ether.h rev. 1.66 and related stuff:
http://mail-index.netbsd.org/source-changes/2016/12/28/msg080289.html

 Show 2.5GBASE-T and 5GBASE-T correctly on ixv(4). Tested on a virtual function
on X550 T1.

 Set baudrate correctly. A device which use mii(4) interface can set
if_baudrate atomatically by mii_phy_statusmsg(). ixg(4) and ixv(4) don't
use mii(4), so calling ifmedia_baudrate() directly is required.

 XXX need the same same work for wm(4)'s TBI and SERDEV devices.

- ixv_enableintr(): Fix a bug that ixv_enable_queue() is called with incorrect
  argument. This bug was added in rev. 1.68.
- Don't check neither VTEICR nor VTEICS same as Linux. It seems that both
  registers can't be used to check which MSI-X vector is triggered. A bit is
  set on very low probability and the bit location is incorrect. Is this an
  errata?

 Fix a problem that mailbox interrupt never occured. Bit definitions of PF's
interrupt related registers and VF's interrupt registers a completely
different. VF's registers (VF{EICR, EICS, EIMS, EIMC, EIAC, EIAM})'s bits are
just MSI-X vector bitmask, so read/write correctly. Read VTEICR instead of
VTEICS in ixv_msix_mbx (note that "PF" has a errata that EICS is required to
read the cause). Don't write IXGBE_VTEICR in ixv_msix_mbx() because we use
auto-clear for mailbox interrupt.

 We have gotten link status change by timer instead of interrupt before
this fix...

 Make mailbox statistic counters evcnt(9).
Example:
 > ixv0 message TXs                                            23    0 misc
 > ixv0 message RXs                                          2911    0 misc
 > ixv0 ACKs                                                   23    0 misc
 > ixv0 REQs                                                   22    0 misc
 > ixv0 RSTs                                                    0    0 misc

 Print device name in ixv_attach().

Pull up following revision(s) (requested by msaitoh in ticket #288):
	sys/dev/pci/ixgbe/ixv.c: 1.59 and 1.62 via patch
- Use cprng_strong64() instead of cprng_fast64() to prevent panic.
- Set get_link_status to true before calling ixv_config_link() in
  ixv_init_locked() to get the first link status correctly.

xv(4) VLAN fixes:
- Dirty hack for VID mask bits. On ixg(4), VLAN filter is disabled. So any
  vlan ID's packet passes RX filter. On ixv(4), usually, PF driver uses
  VLAN filter enabled. There is no way to disable PF's VLAN filter function
  itself. NetBSD's Ethernet driver has currently no API to know which VLAN ID
  should be accepted. To avoid this proble, enable all VIDs... Yes, I know
  this is dirty hack. We should rethink.
- Call ixv_setup_vlan_support() in ixv_ifflags_cb().
- Don't use global ixv_shadow_vfta[].
- Use local variable (rxr->) to reduce diff against ixgbe.c. No functional
  change.

 Make ixv(4) TX/RX descriptors size the same as ixg(4).

- Print driver feature capabilities and enable bits when verbose boot as
  ixgbe.c rev. 1.99.
- Whitespace fix.

 Set get_link_status to true in ixv_init_locked() to get the first link status
correctly.

 Fix panic caused by rev. 1.58's change. Don't if_start from interrupt context.
The restart code exists in ixv_handle_que().

 Print Mailbox API version.

 Use cprng_strong64() insread of cprng_fast64() to prevent panic.

- Sync with FreeBSD r320688 (and up to r322865):
  - Add C3000(Denvertion(-NS)) support.
  - Add bypass function support for bypass adapters. Sysctls are provided.
    Not tested because I have no any bypass adapter.
  - ixv(4): set RSS mapping.
  - Change EEE sysctl.
  - ixv(4): Add TSOv6.
  - ixv(4): Mailbox API 1.2 and more are implemented and comment says it
    negotiate with 1.2 but it really does 1.1...
  - Remove thermal test sysctl.
  - Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

 FreeBSD r320688 sorted a lot of functions. Merging that change by one commit
makes difficult to know what was really changed. To make change understandable,
commit only function sort and white space changes before real change.

Sync with HEAD

 Add missing {ixgbe,ixv}_stop() in the detach function. Without this change,
shutdown won't finish or panic on heay traffic.

Sync with HEAD

Sync with HEAD

 Set PCI_INTR_MPSAFE, SOFTINT_MPSAFE, CALLOUT_MPSAFE and IFEF_START_MPSAFE
if NET_MPSAFE is defined.

 Do if_snd check even if IXGBE_LEGACY_TX is used. It's used by ALTQ.

Remove more ix*_start_locked() from interrupt context like previous commit.
 > Fix a bug that ix*_start_locked() is called in interrput context
 > (ix*_msix_que). The function is called in softint(ix*_handle_que()). OK'd by
 >k-nakahara.

Use percpuq.

 Fix a bug that ix*_start_locked() is called in interrput context
(ix*_msix_que). The function is called in softint(ix*_handle_que()). OK'd by
k-nakahara.

 Change the Interrupt Moderation flag from global variable to per device.

- Both mbuf_defrag_failed and m_defrag_failed existed. Remove m_defrag_failed.
- rx_irq, dropped_pkts, morerx, moretx and txloops counters are uncounted, so
  remove them.
- Count handleq in ixv_handle_que().
- Detach event counters correctly.
- Set some per-queue event names correctly.

No functional change:
 - Rename ixv_sysctl_attach() to ixv_add_device_sysctls().
 - Move sysct and evcnt related functions to match ixgbe.c.

 Print CPU number correctly.

- Fix aprint_*()s in ixv_allocate_msix().
- Style fix. Reduce diff against ixgbe.c. No functional change.

 Call IFQ_SET_MAXLEN() even if TX multiqueue is used. ALTQ uses it.

 Enable multiqueue.

Use dmat64 if available. Same as ixgbe.c rev. 1.60.

 Fix a bug that 1Gbps is printed even if the phyiscal media is at 100Mbps.

Fix detach path:
 - Call softint_disestablish() and pci_intr_disestablish() for all queues
   correctly.
 - Add missing sysctl_teardown() to delete sysctl entries.
 - Call pci_intr_release(). Now NETBSD_MSI_OR_MSIX is completely deleted.

Add queue number into interrupt name.

 Set ifm_active to IFM_NONE when the physical interface's link is down.
Now ifconfig shows "media: Ethernet autoselect '(none)'".

Set IFM_10G_T on 10G...

Fix link status stuff:
 - Set get_link_status in ixv_init_locked() to get link status correctly.
 - Don't discard checked speed in ixv_config_link()...

Fix device attach:
 - Even if ixgbevf_negotiate_api_version() failed, continue attaching.
   Same as Linux. Tested on ESXi 6.0.
 - Fix argument of pci_msix_alloc_exact().
 - Use different intr_xname for each interrupt.
 - Add missing pci_intr_release() on error.
 - Add missign kcpuset_destroy().
 - Calculate MSI-X vectors correctly.

Sync with HEAD

 TX multiqueue. If you want to disable it, enable IXGBE_LEGACY_TX
in ixgbe_netbsd.h

Fix to use softint-based if_input in ixv(4).

See:
  http://mail-index.netbsd.org/source-changes/2016/02/09/msg072521.html
  http://mail-index.netbsd.org/tech-kern/2016/01/14/msg019997.html

Reduce diff against FreeBSD. No functional change.

 Now we can use multiqueue. It's default on ixg(4). Not yet for ixv(4).

Sync with HEAD.  (Note that most of these changes are simply $NetBSD$
tag issues.)

 Fix INIT_DEBUGOUT() messages.

Remove extra newline in debug printfs. ixgbe's *DEBUGOUT* macros automatically
add newline.

Sync with HEAD

Update ixg(4) and ixv(4) up to FreeBSD r303890:
- Configure ixgbe phy & gbic power.
  Setup phy and gbic power as per Linux 4.3.13 driver.
  This fixes link not detected on X540-AT2 after booting to Linux which
  turns the phy power off on detach. (FreeBSD r295093)
- Fixup DA cable detection routine. (FreeBSD r303032)
- Some minor changes

Update ixg(4) and ixv(4) up to FreeBSD r294578:
- Fixup SFP module insertion on the 82599 when insertion happens after
  the system is booted and running. Add PHY detection logic to
  ixgbe_handle_mod() and add locking to ixgbe_handle_msf() as well.
  FreeBSD r293334.
- Fix ix advertise value after media change. When ifconfig sets media then the
  values displayed by the advertise_speed value are invalidated.
  Fix this by setting the bits correctly including setting advertise to 0 for
  media = auto. FreeBSD r294578.
- Some others (e.g. LRO(not used by NetBSD)).

Fix ix{gbe,v}_set_sysctlvalue().

Update ixg(4) and ixv(4) up to FreeBSD r292674:
- Add X552 SFP+ and X550T single port.
- Bug fixes.

Update ixg(4) and ixv(4) up to FreeBSD r289238:
- Add support for sysctl knobs to live tune the per interrupt rx/tx packet
  processing limits in ixgbe(4)
- Some others (netmap, etc.)

Update ixg(4) and ixv(4) up to FreeBSD r285590:
- Fix igxbe SRIOV VF (if_ixv) initialization bugs.  The MAC address for
  an if_ixv instance can now set at creation time, and the receive ring
  tail pointer is correctly initialized (previously, things still worked
  because the receive ring tail pointer was being fixed up as a side
  effect of other activity).

Update ixg(4) and ixv(4) up to FreeBSD r283881:
- SRIOV support (not enabled because NetBSD doesn't support it).

Update ixg(4) and ixv(4) up to FreeBSD r282299:
- Add support for X55x.
- ADD EEE support (not tested).
- Add WOL support (not tested).
- Add suspend/resume support (not testd).
- Add LPLU support (not tested).
- Add DMA Coalescing (note verified).
- Not tested well for sysctls.
- Fix ixgbe_set_advertise() a bit. At least, FreeBSD r294578 is required
  to work hw.ixg0.advertise_speed sysctl correctly.

Update ixg(4) and ixv(4) up to FreeBSD r280197:
- Add support for 82599_QSFP_SF_QP and X540T1.
- Add partial support for X55x. It's required to sync with FreeBSD r282299
  to support X55x.
- Bugfixes.

Fix vlan check bug in ixv_setup_vlan_support().

fix build of ixv enabled NO_PCI_MSI_MSIX option.

 Remove #ifdef NETBSD_MSI_OR_MSIX to be simple. ixv(4) isn't attached if
__HAVE_PCI_MSI_MSIX isn't set because MSI-X must be used.

Sync with HEAD

Pull up following revision(s) (requested by msaitoh in ticket #1174):
	sys/dev/pci/ixgbe/README: 1.2
	sys/dev/pci/ixgbe/ixgbe.c: 1.30-1.33, 1.35 via patch
	sys/dev/pci/ixgbe/ixgbe.h: 1.8 via patch
	sys/dev/pci/ixgbe/ixgbe_82598.c: 1.5 via patch
	sys/dev/pci/ixgbe/ixgbe_82599.c: 1.10 via patch
	sys/dev/pci/ixgbe/ixgbe_api.c: 1.8-1.11 via patch
	sys/dev/pci/ixgbe/ixgbe_api.h: 1.5-1.6 via patch
	sys/dev/pci/ixgbe/ixgbe_common.c: 1.6-1.7 via patch
	sys/dev/pci/ixgbe/ixgbe_common.h: 1.5 via patch
	sys/dev/pci/ixgbe/ixgbe_osdep.h: 1.9 via patch
	sys/dev/pci/ixgbe/ixgbe_phy.c: 1.6 via patch
	sys/dev/pci/ixgbe/ixgbe_phy.h: 1.5 via patch
	sys/dev/pci/ixgbe/ixgbe_type.h: 1.11, 1.13 via patch
	sys/dev/pci/ixgbe/ixgbe_vf.c: 1.5-1.6 via patch
	sys/dev/pci/ixgbe/ixv.c: 1.9-1.10, 1.12-1.14 via patch
	sys/dev/pci/ixgbe/ixv.h: 1.6 via patch
	sys/dev/pci/ixgbe/ixgbe_dcb.c: 1.1
	sys/dev/pci/ixgbe/ixgbe_dcb.h: 1.1
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.c: 1.1
	sys/dev/pci/ixgbe/ixgbe_dcb_82598.h: 1.1
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.c: 1.1
	sys/dev/pci/ixgbe/ixgbe_dcb_82599.h: 1.1
Sync ixg(4) up to ixgbe.c rev. 1.35 except MSI/MSI-X and NET_MPSAFE:
- Sync ixg(4) up to FreeBSD r279805 (or r280181) which include some
  bugfixes.
- Add missing evcnt_detach(&rxr->rx_copies) in ixgbe_detach().
- Fix error path in ixgbe_attach() and ixgbe_detatch() to prevent
  panic.
- Remove unreachable code.
- Remove duplicated assignment of mh->m_len.
- Reduce the diff against FreeBSD (tabs, spaces and comments)
- ixv.c: Fix unintialized data. Include vlan.h and check NVLAN.
- Avoid shadowing global `min'.

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

Sync with HEAD

Introduce softint-based if_input

This change intends to run the whole network stack in softint context
(or normal LWP), not hardware interrupt context. Note that the work is
still incomplete by this change; to that end, we also have to softint-ify
if_link_state_change (and bpf) which can still run in hardware interrupt.

This change softint-ifies at ifp->if_input that is called from
each device driver (and ieee80211_input) to ensure Layer 2 runs
in softint (e.g., ether_input and bridge_input). To this end,
we provide a framework (called percpuq) that utlizes softint(9)
and percpu ifqueues. With this patch, rxintr of most drivers just
queues received packets and schedules a softint, and the softint
dequeues packets and does rest packet processing.

To minimize changes to each driver, percpuq is allocated in struct
ifnet for now and that is initialized by default (in if_attach).
We probably have to move percpuq to softc of each driver, but it's
future work. At this point, only wm(4) has percpuq in its softc
as a reference implementation.

Additional information including performance numbers can be found
in the thread at tech-kern@ and tech-net@:
http://mail-index.netbsd.org/tech-kern/2016/01/14/msg019997.html

Acknowledgment: riastradh@ greatly helped this work.
Thank you very much!

Sync with HEAD

Add kernel code to support intrctl(8).

Make it compilable without DIAGNOSTIC

 - Fix uninitialized data.
 - include vlan.h and check NVLAN.

 Reduce diff against FreeBSD r280181.

 - Add MSI/MSI-X support. The multiqueue function is not supported yet.
 - Make ixv.c compilable. _NOT_TESTED_YET_

 Sync ixg(4) up to FreeBSD r279805 (or r280181) which include some bugfixes.
TODO:
 - Merge r280182 and newer. It's required to support X55x.
 - MSI/MSI-X support.

Sync with HEAD

remove duplicated assignment of mh->m_len

Pull up following revisions (requested by msaitoh in ticket #734):
sys/dev/pci/ixgbe/LICENSE			1.2
sys/dev/pci/ixgbe/ixgbe.c			1.28-1.29
sys/dev/pci/ixgbe/ixgbe.h			1.7
sys/dev/pci/ixgbe/ixgbe_82598.c			1.4
sys/dev/pci/ixgbe/ixgbe_82598.h			1.3
sys/dev/pci/ixgbe/ixgbe_82599.c			1.9
sys/dev/pci/ixgbe/ixgbe_82599.h			1.2
sys/dev/pci/ixgbe/ixgbe_api.c			1.7
sys/dev/pci/ixgbe/ixgbe_api.h			1.4
sys/dev/pci/ixgbe/ixgbe_common.c		1.5
sys/dev/pci/ixgbe/ixgbe_common.h		1.4
sys/dev/pci/ixgbe/ixgbe_mbx.h			1.4
sys/dev/pci/ixgbe/ixgbe_netbsd.h		1.3
sys/dev/pci/ixgbe/ixgbe_osdep.h			1.7
sys/dev/pci/ixgbe/ixgbe_phy.c			1.5
sys/dev/pci/ixgbe/ixgbe_phy.h			1.4
sys/dev/pci/ixgbe/ixgbe_type.h			1.10
sys/dev/pci/ixgbe/ixgbe_vf.c			1.4
sys/dev/pci/ixgbe/ixgbe_vf.h			1.5
sys/dev/pci/ixgbe/ixgbe_x540.c			1.3
sys/dev/pci/ixgbe/ixgbe_x540.h			1.2
sys/dev/pci/ixgbe/ixv.c				1.8

	- Sync ixg(4) up to FreeBSD r250108:
	  - Cleanup some unused counters and some unused code.
	  - Improve performance.
	  - Fix flow control - don't override user value on re-init
	  - Fix to make 1G optics work correctly
	  - Change to interrupt enabling - some bits were incorrect
	    for certain hardware.
	  - Certain stats fixes, remove a duplicate increment of
	    ierror, thanks to Scott Long for pointing these out.
	  - Fix the setting of RX which related to multicast.
	  - Some netmap related fixes.
	- Fixes link error without vlan. PR#49879 reported by Hauke Fath.
	[msaitoh, ticket #734]

Sync ixg(4) up to FreeBSD r250108:
 - Cleanup some unused counters and some unused code.
 - Improve performance.
 - Fix flow control - don't override user value on re-init
 - Fix to make 1G optics work correctly
 - Change to interrupt enabling - some bits were incorrect
   for certain hardware.
 - Certain stats fixes, remove a duplicate increment of
   ierror, thanks to Scott Long for pointing these out.
 - Fix the setting of RX which related to multicast.
 - Some netmap related fixes.

Apply patch (requested by msaitoh in ticket #697):
sys/dev/pci/files.pci				1.375 via patch
sys/dev/pci/ixgbe/ixgbe.c			1.24-1.27 via patch
sys/dev/pci/ixgbe/ixgbe.h			1.4-1.6 via patch
sys/dev/pci/ixgbe/ixgbe_82598.c			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_82598.h			1.1-1.2 via patch
sys/dev/pci/ixgbe/ixgbe_82599.c			1.6-1.8 via patch
sys/dev/pci/ixgbe/ixgbe_82599.h			1.1 via patch
sys/dev/pci/ixgbe/ixgbe_api.c			1.5-1.6 via patch
sys/dev/pci/ixgbe/ixgbe_api.h			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_common.c		1.3-1.4 via patch
sys/dev/pci/ixgbe/ixgbe_common.h		1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_mbx.c			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_mbx.h			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_osdep.h			1.4-1.6 via patch
sys/dev/pci/ixgbe/ixgbe_phy.c			1.3-1.4 via patch
sys/dev/pci/ixgbe/ixgbe_phy.h			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_type.h			1.8-1.9 via patch
sys/dev/pci/ixgbe/ixgbe_vf.c			1.2-1.3 via patch
sys/dev/pci/ixgbe/ixgbe_vf.h			1.3-1.4 via patch
sys/dev/pci/ixgbe/ixgbe_x540.c			1.1-1.2 via patch
sys/dev/pci/ixgbe/ixgbe_x540.h			1.1 via patch
sys/dev/pci/ixgbe/ixv.c				1.5-1.7 via patch
sys/dev/pci/ixgbe/ixv.h				1.4-1.5 via patch

	Synchronize our ixg(4) driver up to FreeBSD r243716:
	 - Add X540 support.
	 - Add TSO6 support.
	 - Add 100BaseTX support.
	 - The max size in dma tag is changed from 65535 to 262140 (IXGBE_TSO_SIZE).
	   The value is the same as other *BSDs. The change might cause a address
	   space shortage (ixgbe_dmamap_create() might fail) on some machines.
	 - Show 1000Base-SX correctly.
	 - Fix if_baudrate from 1G to 10G.
	 - Fix a bug that ifconfig -z (SOICZIFDATA) doesn't work.
	 - Fix a lot of bugs.
	 - Improve performance.
	[msaitoh, ticket #697]

Sync ixg(4) up to FreeBSD r243716:
 - A lot of bugfixes. Some of them are realted to multi queue and those
   have not affected in NetBSD because we have not used it yet.
 - Show 1000Base-SX correctly.
 - Fix if_baudrate from 1G to 10G.
 - Improve performance.

Sync with HEAD

Update our ixg(4) driver up to FreeBSD r238149:
- Add TSO6 support.
- The max size in dma tag is changed from 65535 to 262140 (IXGBE_TSO_SIZE).
  The value is the same as other *BSDs. The change might cause a address
  space shortage (ixgbe_dmamap_create() might fail) on some machines.
- Fix a lot of bugs.
- Improve performance.

Synchronize our ixg(4) driver up to FreeBSD r230775:
 - Add X540 support.
 - Add 100BaseTX support.
 - Fix a lot of bugs.
 - Improve performance.

Pull up the following revisions (via patch), requested by msaitoh in #641:

	sys/dev/pci/files.pci				1.374
	sys/dev/pci/ixgbe/ixgbe.h			1.3
	sys/dev/pci/ixgbe/ixgbe.c			1.20-1.23
	sys/dev/pci/ixgbe/ixgbe_type.h			1.3-1.7
	sys/dev/pci/ixgbe/ixgbe_82599.c			1.4-1.5
	sys/dev/pci/ixgbe/ixgbe_api.c			1.3-1.4
	sys/dev/pci/ixgbe/ixgbe_vf.h			1.2
	sys/dev/pci/ixgbe/ixv.c				1.3-1.4
	sys/dev/pci/ixgbe/ixv.h				1.3

 - Sync ixg(4) up to FreeBSD r230572.
   - Fix in the interrupt handler to make sure the stack TX queue is processed.
     (FreeBSD r222588)
   - The maximum read size of incoming packets is done in 1024-byte increments.
     The current code was rounding down the maximum frame size instead of
     routing up, resulting in a read size of 1024 bytes, in the non-jumbo
     frame case, and splitting the packets across multiple mbufs.
     (FreeBSD r225045)
   - Consequently the above problem exposed another issue, which is when
     packets were splitted across multiple mbufs, and all of the mbufs in the
     chain have the M_PKTHDR flag set. (FreeBSD r225045)
   - Use the correct constant for conversion between interrupt rate
     and EITR values (the previous values were off by a factor of 2)
     (FreeBSD r230572)
   - Make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing
     individual values affects the queue immediately, and propagates to all
     interfaces at the next reinit. (FreeBSD r230572)
   - Add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt
     counts. (FreeBSD r230572)
   - Some netmap related changes.
 - Fix a bug that vlan setting may not apply to the parent interface correctly.
 - Add 82599EN_SFP and 82599_SFP_SF_QP.
 - Modify to make Intel Intel 10G Ethernet (ixg(4)) virtual function ixv(4)
   compilable. Not completed yet. It's required to use MSI-X.
 - Change comments, tabs and spaces to make difference among *BSDs small.
 - Enclose macro argument correctly. Same as *BSDs. No binary change.
 - Add missing parenthesis in IXGBE_CORE_LOCK_ASSERT() macro. No binary change.

Sync ixg(4) up to FreeBSD r230572.

 - Fix in the interrupt handler to make sure the stack TX queue is processed.
   (FreeBSD r222588)
 - The maximum read size of incoming packets is done in 1024-byte increments.
   The current code was rounding down the maximum frame size instead of
   routing up, resulting in a read size of 1024 bytes, in the non-jumbo
   frame case, and splitting the packets across multiple mbufs.
   (FreeBSD r225045)
 - Consequently the above problem exposed another issue, which is when
   packets were splitted across multiple mbufs, and all of the mbufs in the
   chain have the M_PKTHDR flag set. (FreeBSD r225045)
 - Use the correct constant for conversion between interrupt rate
   and EITR values (the previous values were off by a factor of 2)
   (FreeBSD r230572)
 - Make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing
   individual values affects the queue immediately, and propagates to all
   interfaces at the next reinit. (FreeBSD r230572)
 - Add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt
   counts. (FreeBSD r230572)
 - Some netmap related changes.

 Modify to make Intel Intel 10G Ethernet (ixg(4)) virtual function ixv(4)
compilable. Not completed yet. It's required to use MSI-X.

Rebase to HEAD as of a few days ago.

sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs.  ("Protocol error: too many arguments")

sync with head

kill sprintf

Add sources for ixgbe(4), a Intel 82599 10-gigabit ethernet driver
ported from FreeBSD.