Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Daniel Borkmann says:
====================
pull-request: bpf-next 2023-04-21
We've added 71 non-merge commits during the last 8 day(s) which contain
a total of 116 files changed, 13397 insertions(+), 8896 deletions(-).
The main changes are:
1) Add a new BPF netfilter program type and minimal support to hook
BPF programs to netfilter hooks such as prerouting or forward,
from Florian Westphal.
2) Fix race between btf_put and btf_idr walk which caused a deadlock,
from Alexei Starovoitov.
3) Second big batch to migrate test_verifier unit tests into test_progs
for ease of readability and debugging, from Eduard Zingerman.
4) Add support for refcounted local kptrs to the verifier for allowing
shared ownership, useful for adding a node to both the BPF list and
rbtree, from Dave Marchevsky.
5) Migrate bpf_for(), bpf_for_each() and bpf_repeat() macros from BPF
selftests into libbpf-provided bpf_helpers.h header and improve
kfunc handling, from Andrii Nakryiko.
6) Support 64-bit pointers to kfuncs needed for archs like s390x,
from Ilya Leoshkevich.
7) Support BPF progs under getsockopt with a NULL optval,
from Stanislav Fomichev.
8) Improve verifier u32 scalar equality checking in order to enable
LLVM transformations which earlier had to be disabled specifically
for BPF backend, from Yonghong Song.
9) Extend bpftool's struct_ops object loading to support links,
from Kui-Feng Lee.
10) Add xsk selftest follow-up fixes for hugepage allocated umem,
from Magnus Karlsson.
11) Support BPF redirects from tc BPF to ifb devices,
from Daniel Borkmann.
12) Add BPF support for integer type when accessing variable length
arrays, from Feng Zhou.
* tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (71 commits)
selftests/bpf: verifier/value_ptr_arith converted to inline assembly
selftests/bpf: verifier/value_illegal_alu converted to inline assembly
selftests/bpf: verifier/unpriv converted to inline assembly
selftests/bpf: verifier/subreg converted to inline assembly
selftests/bpf: verifier/spin_lock converted to inline assembly
selftests/bpf: verifier/sock converted to inline assembly
selftests/bpf: verifier/search_pruning converted to inline assembly
selftests/bpf: verifier/runtime_jit converted to inline assembly
selftests/bpf: verifier/regalloc converted to inline assembly
selftests/bpf: verifier/ref_tracking converted to inline assembly
selftests/bpf: verifier/map_ptr_mixing converted to inline assembly
selftests/bpf: verifier/map_in_map converted to inline assembly
selftests/bpf: verifier/lwt converted to inline assembly
selftests/bpf: verifier/loops1 converted to inline assembly
selftests/bpf: verifier/jeq_infer_not_null converted to inline assembly
selftests/bpf: verifier/direct_packet_access converted to inline assembly
selftests/bpf: verifier/d_path converted to inline assembly
selftests/bpf: verifier/ctx converted to inline assembly
selftests/bpf: verifier/btf_ctx_access converted to inline assembly
selftests/bpf: verifier/bpf_get_stack converted to inline assembly
...
====================
Link: https://lore.kernel.org/r/20230421211035.9111-1-daniel@iogearbox.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
diff --git a/.gitignore b/.gitignore
index 70ec603..51117ba 100644
--- a/.gitignore
+++ b/.gitignore
@@ -105,6 +105,7 @@
!.gitignore
!.mailmap
!.rustfmt.toml
+!.kunitconfig
#
# Generated include files
diff --git a/.mailmap b/.mailmap
index e424863..6686879 100644
--- a/.mailmap
+++ b/.mailmap
@@ -232,6 +232,8 @@
John Crispin <john@phrozen.org> <blogic@openwrt.org>
John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
John Stultz <johnstul@us.ibm.com>
+<jon.toppins+linux@gmail.com> <jtoppins@cumulusnetworks.com>
+<jon.toppins+linux@gmail.com> <jtoppins@redhat.com>
Jordan Crouse <jordan@cosmicpenguin.net> <jcrouse@codeaurora.org>
<josh@joshtriplett.org> <josh@freedesktop.org>
<josh@joshtriplett.org> <josh@kernel.org>
@@ -297,6 +299,8 @@
Martin Kepplinger <martink@posteo.de> <martin.kepplinger@theobroma-systems.com>
Martyna Szapar-Mudlaw <martyna.szapar-mudlaw@linux.intel.com> <martyna.szapar-mudlaw@intel.com>
Mathieu Othacehe <m.othacehe@gmail.com>
+Mat Martineau <martineau@kernel.org> <mathew.j.martineau@linux.intel.com>
+Mat Martineau <martineau@kernel.org> <mathewm@codeaurora.org>
Matthew Wilcox <willy@infradead.org> <matthew.r.wilcox@intel.com>
Matthew Wilcox <willy@infradead.org> <matthew@wil.cx>
Matthew Wilcox <willy@infradead.org> <mawilcox@linuxonhyperv.com>
diff --git a/Documentation/admin-guide/kernel-parameters.rst b/Documentation/admin-guide/kernel-parameters.rst
index 19600c5..6ae5f12 100644
--- a/Documentation/admin-guide/kernel-parameters.rst
+++ b/Documentation/admin-guide/kernel-parameters.rst
@@ -128,6 +128,7 @@
KVM Kernel Virtual Machine support is enabled.
LIBATA Libata driver is enabled
LP Printer support is enabled.
+ LOONGARCH LoongArch architecture is enabled.
LOOP Loopback device support is enabled.
M68k M68k architecture is enabled.
These options have more detailed description inside of
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 6221a1d..7016cb1 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6933,6 +6933,12 @@
When enabled, memory and cache locality will be
impacted.
+ writecombine= [LOONGARCH] Control the MAT (Memory Access Type) of
+ ioremap_wc().
+
+ on - Enable writecombine, use WUC for ioremap_wc()
+ off - Disable writecombine, use SUC for ioremap_wc()
+
x2apic_phys [X86-64,APIC] Use x2apic physical mode instead of
default x2apic cluster mode on platforms
supporting x2apic.
diff --git a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
index fe9ebe2..df64eeb 100644
--- a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
+++ b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
@@ -18,6 +18,8 @@
PHY it is connected to. In this config, an internal mdio-bus is registered and
the MDIO master is used for communication. Mixed external and internal
mdio-bus configurations are not supported by the hardware.
+ Each phy has at most 3 LEDs connected and can be declared
+ using the standard LEDs structure.
properties:
compatible:
@@ -117,6 +119,7 @@
examples:
- |
#include <dt-bindings/gpio/gpio.h>
+ #include <dt-bindings/leds/common.h>
mdio {
#address-cells = <1>;
@@ -226,6 +229,25 @@
label = "lan1";
phy-mode = "internal";
phy-handle = <&internal_phy_port1>;
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_WHITE>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+
+ led@1 {
+ reg = <1>;
+ color = <LED_COLOR_ID_AMBER>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@2 {
diff --git a/Documentation/devicetree/bindings/net/ethernet-controller.yaml b/Documentation/devicetree/bindings/net/ethernet-controller.yaml
index 00be387..6b0d359 100644
--- a/Documentation/devicetree/bindings/net/ethernet-controller.yaml
+++ b/Documentation/devicetree/bindings/net/ethernet-controller.yaml
@@ -205,7 +205,7 @@
duplex is assumed.
pause:
- $ref: /schemas/types.yaml#definitions/flag
+ $ref: /schemas/types.yaml#/definitions/flag
description:
Indicates that pause should be enabled.
@@ -222,6 +222,41 @@
required:
- speed
+ leds:
+ description:
+ Describes the LEDs associated by Ethernet Controller.
+ These LEDs are not integrated in the PHY and PHY doesn't have any
+ control on them. Ethernet Controller regs are used to control
+ these defined LEDs.
+
+ type: object
+
+ properties:
+ '#address-cells':
+ const: 1
+
+ '#size-cells':
+ const: 0
+
+ patternProperties:
+ '^led@[a-f0-9]+$':
+ $ref: /schemas/leds/common.yaml#
+
+ properties:
+ reg:
+ maxItems: 1
+ description:
+ This define the LED index in the PHY or the MAC. It's really
+ driver dependent and required for ports that define multiple
+ LED for the same port.
+
+ required:
+ - reg
+
+ unevaluatedProperties: false
+
+ additionalProperties: false
+
dependencies:
pcs-handle-names: [pcs-handle]
diff --git a/Documentation/devicetree/bindings/net/ethernet-phy.yaml b/Documentation/devicetree/bindings/net/ethernet-phy.yaml
index ac04f8e..4f57453 100644
--- a/Documentation/devicetree/bindings/net/ethernet-phy.yaml
+++ b/Documentation/devicetree/bindings/net/ethernet-phy.yaml
@@ -197,6 +197,35 @@
PHY's that have configurable TX internal delays. If this property is
present then the PHY applies the TX delay.
+ leds:
+ type: object
+
+ properties:
+ '#address-cells':
+ const: 1
+
+ '#size-cells':
+ const: 0
+
+ patternProperties:
+ '^led@[a-f0-9]+$':
+ $ref: /schemas/leds/common.yaml#
+
+ properties:
+ reg:
+ maxItems: 1
+ description:
+ This define the LED index in the PHY or the MAC. It's really
+ driver dependent and required for ports that define multiple
+ LED for the same port.
+
+ required:
+ - reg
+
+ unevaluatedProperties: false
+
+ additionalProperties: false
+
required:
- reg
@@ -204,6 +233,8 @@
examples:
- |
+ #include <dt-bindings/leds/common.h>
+
ethernet {
#address-cells = <1>;
#size-cells = <0>;
@@ -219,5 +250,17 @@
reset-gpios = <&gpio1 4 1>;
reset-assert-us = <1000>;
reset-deassert-us = <2000>;
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_WHITE>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
};
diff --git a/Documentation/devicetree/bindings/net/ethernet-switch.yaml b/Documentation/devicetree/bindings/net/ethernet-switch.yaml
index 2ceccce..f1b9075 100644
--- a/Documentation/devicetree/bindings/net/ethernet-switch.yaml
+++ b/Documentation/devicetree/bindings/net/ethernet-switch.yaml
@@ -55,7 +55,7 @@
$defs:
base:
description: An ethernet switch without any extra port properties
- $ref: '#/'
+ $ref: '#'
patternProperties:
"^(ethernet-)?port@[0-9]+$":
diff --git a/Documentation/devicetree/bindings/net/snps,dwmac.yaml b/Documentation/devicetree/bindings/net/snps,dwmac.yaml
index da311c1..363b3e3 100644
--- a/Documentation/devicetree/bindings/net/snps,dwmac.yaml
+++ b/Documentation/devicetree/bindings/net/snps,dwmac.yaml
@@ -30,6 +30,7 @@
- snps,dwmac-4.10a
- snps,dwmac-4.20a
- snps,dwmac-5.10a
+ - snps,dwmac-5.20
- snps,dwxgmac
- snps,dwxgmac-2.10
@@ -90,8 +91,10 @@
- snps,dwmac-4.10a
- snps,dwmac-4.20a
- snps,dwmac-5.10a
+ - snps,dwmac-5.20
- snps,dwxgmac
- snps,dwxgmac-2.10
+ - starfive,jh7110-dwmac
reg:
minItems: 1
@@ -134,12 +137,16 @@
- ptp_ref
resets:
- maxItems: 1
- description:
- MAC Reset signal.
+ minItems: 1
+ items:
+ - description: GMAC stmmaceth reset
+ - description: AHB reset
reset-names:
- const: stmmaceth
+ minItems: 1
+ items:
+ - const: stmmaceth
+ - const: ahb
power-domains:
maxItems: 1
@@ -579,6 +586,7 @@
- snps,dwmac-3.50a
- snps,dwmac-4.10a
- snps,dwmac-4.20a
+ - snps,dwmac-5.20
- snps,dwxgmac
- snps,dwxgmac-2.10
- st,spear600-gmac
@@ -636,6 +644,7 @@
- snps,dwmac-4.10a
- snps,dwmac-4.20a
- snps,dwmac-5.10a
+ - snps,dwmac-5.20
- snps,dwxgmac
- snps,dwxgmac-2.10
- st,spear600-gmac
diff --git a/Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml b/Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml
new file mode 100644
index 0000000..5e7cfbb
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml
@@ -0,0 +1,144 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2022 StarFive Technology Co., Ltd.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/starfive,jh7110-dwmac.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: StarFive JH7110 DWMAC glue layer
+
+maintainers:
+ - Emil Renner Berthing <kernel@esmil.dk>
+ - Samin Guo <samin.guo@starfivetech.com>
+
+select:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - starfive,jh7110-dwmac
+ required:
+ - compatible
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - starfive,jh7110-dwmac
+ - const: snps,dwmac-5.20
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: GMAC main clock
+ - description: GMAC AHB clock
+ - description: PTP clock
+ - description: TX clock
+ - description: GTX clock
+
+ clock-names:
+ items:
+ - const: stmmaceth
+ - const: pclk
+ - const: ptp_ref
+ - const: tx
+ - const: gtx
+
+ interrupts:
+ minItems: 3
+ maxItems: 3
+
+ interrupt-names:
+ minItems: 3
+ maxItems: 3
+
+ resets:
+ items:
+ - description: MAC Reset signal.
+ - description: AHB Reset signal.
+
+ reset-names:
+ items:
+ - const: stmmaceth
+ - const: ahb
+
+ starfive,tx-use-rgmii-clk:
+ description:
+ Tx clock is provided by external rgmii clock.
+ type: boolean
+
+ starfive,syscon:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ items:
+ - items:
+ - description: phandle to syscon that configures phy mode
+ - description: Offset of phy mode selection
+ - description: Shift of phy mode selection
+ description:
+ A phandle to syscon with two arguments that configure phy mode.
+ The argument one is the offset of phy mode selection, the
+ argument two is the shift of phy mode selection.
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - interrupts
+ - interrupt-names
+ - resets
+ - reset-names
+
+allOf:
+ - $ref: snps,dwmac.yaml#
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ ethernet@16030000 {
+ compatible = "starfive,jh7110-dwmac", "snps,dwmac-5.20";
+ reg = <0x16030000 0x10000>;
+ clocks = <&clk 3>, <&clk 2>, <&clk 109>,
+ <&clk 6>, <&clk 111>;
+ clock-names = "stmmaceth", "pclk", "ptp_ref",
+ "tx", "gtx";
+ resets = <&rst 1>, <&rst 2>;
+ reset-names = "stmmaceth", "ahb";
+ interrupts = <7>, <6>, <5>;
+ interrupt-names = "macirq", "eth_wake_irq", "eth_lpi";
+ phy-mode = "rgmii-id";
+ snps,multicast-filter-bins = <64>;
+ snps,perfect-filter-entries = <8>;
+ rx-fifo-depth = <2048>;
+ tx-fifo-depth = <2048>;
+ snps,fixed-burst;
+ snps,no-pbl-x8;
+ snps,tso;
+ snps,force_thresh_dma_mode;
+ snps,axi-config = <&stmmac_axi_setup>;
+ snps,en-tx-lpi-clockgating;
+ snps,txpbl = <16>;
+ snps,rxpbl = <16>;
+ starfive,syscon = <&aon_syscon 0xc 0x12>;
+ phy-handle = <&phy0>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "snps,dwmac-mdio";
+
+ phy0: ethernet-phy@0 {
+ reg = <0>;
+ };
+ };
+
+ stmmac_axi_setup: stmmac-axi-config {
+ snps,lpi_en;
+ snps,wr_osr_lmt = <4>;
+ snps,rd_osr_lmt = <4>;
+ snps,blen = <256 128 64 32 0 0 0>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/net/wireless/mediatek,mt76.yaml b/Documentation/devicetree/bindings/net/wireless/mediatek,mt76.yaml
index 7d526ff..67b63f1 100644
--- a/Documentation/devicetree/bindings/net/wireless/mediatek,mt76.yaml
+++ b/Documentation/devicetree/bindings/net/wireless/mediatek,mt76.yaml
@@ -111,6 +111,11 @@
$ref: /schemas/leds/common.yaml#
additionalProperties: false
properties:
+ led-active-low:
+ description:
+ LED is enabled with ground signal.
+ type: boolean
+
led-sources:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt b/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt
deleted file mode 100644
index b61c2d5..0000000
--- a/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt
+++ /dev/null
@@ -1,215 +0,0 @@
-* Qualcomm Atheros ath10k wireless devices
-
-Required properties:
-- compatible: Should be one of the following:
- * "qcom,ath10k"
- * "qcom,ipq4019-wifi"
- * "qcom,wcn3990-wifi"
-
-PCI based devices uses compatible string "qcom,ath10k" and takes calibration
-data along with board specific data via "qcom,ath10k-calibration-data".
-Rest of the properties are not applicable for PCI based devices.
-
-AHB based devices (i.e. ipq4019) uses compatible string "qcom,ipq4019-wifi"
-and also uses most of the properties defined in this doc (except
-"qcom,ath10k-calibration-data"). It uses "qcom,ath10k-pre-calibration-data"
-to carry pre calibration data.
-
-In general, entry "qcom,ath10k-pre-calibration-data" and
-"qcom,ath10k-calibration-data" conflict with each other and only one
-can be provided per device.
-
-SNOC based devices (i.e. wcn3990) uses compatible string "qcom,wcn3990-wifi".
-
-- reg: Address and length of the register set for the device.
-- reg-names: Must include the list of following reg names,
- "membase"
-- interrupts: reference to the list of 17 interrupt numbers for "qcom,ipq4019-wifi"
- compatible target.
- reference to the list of 12 interrupt numbers for "qcom,wcn3990-wifi"
- compatible target.
- Must contain interrupt-names property per entry for
- "qcom,ath10k", "qcom,ipq4019-wifi" compatible targets.
-
-- interrupt-names: Must include the entries for MSI interrupt
- names ("msi0" to "msi15") and legacy interrupt
- name ("legacy") for "qcom,ath10k", "qcom,ipq4019-wifi"
- compatible targets.
-
-Optional properties:
-- resets: Must contain an entry for each entry in reset-names.
- See ../reset/reseti.txt for details.
-- reset-names: Must include the list of following reset names,
- "wifi_cpu_init"
- "wifi_radio_srif"
- "wifi_radio_warm"
- "wifi_radio_cold"
- "wifi_core_warm"
- "wifi_core_cold"
-- clocks: List of clock specifiers, must contain an entry for each required
- entry in clock-names.
-- clock-names: Should contain the clock names "wifi_wcss_cmd", "wifi_wcss_ref",
- "wifi_wcss_rtc" for "qcom,ipq4019-wifi" compatible target and
- "cxo_ref_clk_pin" and optionally "qdss" for "qcom,wcn3990-wifi"
- compatible target.
-- qcom,msi_addr: MSI interrupt address.
-- qcom,msi_base: Base value to add before writing MSI data into
- MSI address register.
-- qcom,ath10k-calibration-variant: string to search for in the board-2.bin
- variant list with the same bus and device
- specific ids
-- qcom,ath10k-calibration-data : calibration data + board specific data
- as an array, the length can vary between
- hw versions.
-- qcom,ath10k-pre-calibration-data : pre calibration data as an array,
- the length can vary between hw versions.
-- <supply-name>-supply: handle to the regulator device tree node
- optional "supply-name" are "vdd-0.8-cx-mx",
- "vdd-1.8-xo", "vdd-1.3-rfa", "vdd-3.3-ch0",
- and "vdd-3.3-ch1".
-- memory-region:
- Usage: optional
- Value type: <phandle>
- Definition: reference to the reserved-memory for the msa region
- used by the wifi firmware running in Q6.
-- iommus:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: A list of phandle and IOMMU specifier pairs.
-- ext-fem-name:
- Usage: Optional
- Value type: string
- Definition: Name of external front end module used. Some valid FEM names
- for example: "microsemi-lx5586", "sky85703-11"
- and "sky85803" etc.
-- qcom,snoc-host-cap-8bit-quirk:
- Usage: Optional
- Value type: <empty>
- Definition: Quirk specifying that the firmware expects the 8bit version
- of the host capability QMI request
-- qcom,xo-cal-data: xo cal offset to be configured in xo trim register.
-
-- qcom,msa-fixed-perm: Boolean context flag to disable SCM call for statically
- mapped msa region.
-
-- qcom,coexist-support : should contain eithr "0" or "1" to indicate coex
- support by the hardware.
-- qcom,coexist-gpio-pin : gpio pin number information to support coex
- which will be used by wifi firmware.
-
-* Subnodes
-The ath10k wifi node can contain one optional firmware subnode.
-Firmware subnode is needed when the platform does not have TustZone.
-The firmware subnode must have:
-
-- iommus:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: A list of phandle and IOMMU specifier pairs.
-
-
-Example (to supply PCI based wifi block details):
-
-In this example, the node is defined as child node of the PCI controller.
-
-pci {
- pcie@0 {
- reg = <0 0 0 0 0>;
- #interrupt-cells = <1>;
- #size-cells = <2>;
- #address-cells = <3>;
- device_type = "pci";
-
- wifi@0,0 {
- reg = <0 0 0 0 0>;
- qcom,ath10k-calibration-data = [ 01 02 03 ... ];
- ext-fem-name = "microsemi-lx5586";
- };
- };
-};
-
-Example (to supply ipq4019 SoC wifi block details):
-
-wifi0: wifi@a000000 {
- compatible = "qcom,ipq4019-wifi";
- reg = <0xa000000 0x200000>;
- resets = <&gcc WIFI0_CPU_INIT_RESET>,
- <&gcc WIFI0_RADIO_SRIF_RESET>,
- <&gcc WIFI0_RADIO_WARM_RESET>,
- <&gcc WIFI0_RADIO_COLD_RESET>,
- <&gcc WIFI0_CORE_WARM_RESET>,
- <&gcc WIFI0_CORE_COLD_RESET>;
- reset-names = "wifi_cpu_init",
- "wifi_radio_srif",
- "wifi_radio_warm",
- "wifi_radio_cold",
- "wifi_core_warm",
- "wifi_core_cold";
- clocks = <&gcc GCC_WCSS2G_CLK>,
- <&gcc GCC_WCSS2G_REF_CLK>,
- <&gcc GCC_WCSS2G_RTC_CLK>;
- clock-names = "wifi_wcss_cmd",
- "wifi_wcss_ref",
- "wifi_wcss_rtc";
- interrupts = <0 0x20 0x1>,
- <0 0x21 0x1>,
- <0 0x22 0x1>,
- <0 0x23 0x1>,
- <0 0x24 0x1>,
- <0 0x25 0x1>,
- <0 0x26 0x1>,
- <0 0x27 0x1>,
- <0 0x28 0x1>,
- <0 0x29 0x1>,
- <0 0x2a 0x1>,
- <0 0x2b 0x1>,
- <0 0x2c 0x1>,
- <0 0x2d 0x1>,
- <0 0x2e 0x1>,
- <0 0x2f 0x1>,
- <0 0xa8 0x0>;
- interrupt-names = "msi0", "msi1", "msi2", "msi3",
- "msi4", "msi5", "msi6", "msi7",
- "msi8", "msi9", "msi10", "msi11",
- "msi12", "msi13", "msi14", "msi15",
- "legacy";
- qcom,msi_addr = <0x0b006040>;
- qcom,msi_base = <0x40>;
- qcom,ath10k-pre-calibration-data = [ 01 02 03 ... ];
- qcom,coexist-support = <1>;
- qcom,coexist-gpio-pin = <0x33>;
-};
-
-Example (to supply wcn3990 SoC wifi block details):
-
-wifi@18000000 {
- compatible = "qcom,wcn3990-wifi";
- reg = <0x18800000 0x800000>;
- reg-names = "membase";
- clocks = <&clock_gcc clk_rf_clk2_pin>;
- clock-names = "cxo_ref_clk_pin";
- interrupts =
- <GIC_SPI 414 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 415 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 417 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 419 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 420 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 421 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 423 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 424 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 425 IRQ_TYPE_LEVEL_HIGH>;
- vdd-0.8-cx-mx-supply = <&pm8998_l5>;
- vdd-1.8-xo-supply = <&vreg_l7a_1p8>;
- vdd-1.3-rfa-supply = <&vreg_l17a_1p3>;
- vdd-3.3-ch0-supply = <&vreg_l25a_3p3>;
- vdd-3.3-ch1-supply = <&vreg_l26a_3p3>;
- memory-region = <&wifi_msa_mem>;
- iommus = <&apps_smmu 0x0040 0x1>;
- qcom,msa-fixed-perm;
- wifi-firmware {
- iommus = <&apps_iommu 0xc22 0x1>;
- };
-};
diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml b/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml
new file mode 100644
index 0000000..c85ed33
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml
@@ -0,0 +1,358 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/wireless/qcom,ath10k.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Technologies ath10k wireless devices
+
+maintainers:
+ - Kalle Valo <kvalo@kernel.org>
+
+description:
+ Qualcomm Technologies, Inc. IEEE 802.11ac devices.
+
+properties:
+ compatible:
+ enum:
+ - qcom,ath10k # SDIO-based devices
+ - qcom,ipq4019-wifi
+ - qcom,wcn3990-wifi # SNoC-based devices
+
+ reg:
+ maxItems: 1
+
+ reg-names:
+ items:
+ - const: membase
+
+ interrupts:
+ minItems: 12
+ maxItems: 17
+
+ interrupt-names:
+ minItems: 12
+ maxItems: 17
+
+ memory-region:
+ maxItems: 1
+ description:
+ Reference to the MSA memory region used by the Wi-Fi firmware
+ running on the Q6 core.
+
+ iommus:
+ minItems: 1
+ maxItems: 2
+
+ clocks:
+ minItems: 1
+ maxItems: 3
+
+ clock-names:
+ minItems: 1
+ maxItems: 3
+
+ resets:
+ maxItems: 6
+
+ reset-names:
+ items:
+ - const: wifi_cpu_init
+ - const: wifi_radio_srif
+ - const: wifi_radio_warm
+ - const: wifi_radio_cold
+ - const: wifi_core_warm
+ - const: wifi_core_cold
+
+ ext-fem-name:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: Name of external front end module used.
+ enum:
+ - microsemi-lx5586
+ - sky85703-11
+ - sky85803
+
+ wifi-firmware:
+ type: object
+ additionalProperties: false
+ description: |
+ The ath10k Wi-Fi node can contain one optional firmware subnode.
+ Firmware subnode is needed when the platform does not have Trustzone.
+ properties:
+ iommus:
+ maxItems: 1
+ required:
+ - iommus
+
+ qcom,ath10k-calibration-data:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Calibration data + board-specific data as a byte array. The length
+ can vary between hardware versions.
+
+ qcom,ath10k-calibration-variant:
+ $ref: /schemas/types.yaml#/definitions/string
+ description:
+ Unique variant identifier of the calibration data in board-2.bin
+ for designs with colliding bus and device specific ids
+
+ qcom,ath10k-pre-calibration-data:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Pre-calibration data as a byte array. The length can vary between
+ hardware versions.
+
+ qcom,coexist-support:
+ $ref: /schemas/types.yaml#/definitions/uint8
+ enum: [0, 1]
+ description:
+ Indicate coex support by the hardware.
+
+ qcom,coexist-gpio-pin:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ COEX GPIO number provided to the Wi-Fi firmware.
+
+ qcom,msa-fixed-perm:
+ type: boolean
+ description:
+ Whether to skip executing an SCM call that reassigns the memory
+ region ownership.
+
+ qcom,smem-states:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ description: State bits used by the AP to signal the WLAN Q6.
+ items:
+ - description: Signal bits used to enable/disable low power mode
+ on WCN in the case of WoW (Wake on Wireless).
+
+ qcom,smem-state-names:
+ description: The names of the state bits used for SMP2P output.
+ items:
+ - const: wlan-smp2p-out
+
+ qcom,snoc-host-cap-8bit-quirk:
+ type: boolean
+ description:
+ Quirk specifying that the firmware expects the 8bit version
+ of the host capability QMI request
+
+ qcom,xo-cal-data:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ XO cal offset to be configured in XO trim register.
+
+ vdd-0.8-cx-mx-supply:
+ description: Main logic power rail
+
+ vdd-1.8-xo-supply:
+ description: Crystal oscillator supply
+
+ vdd-1.3-rfa-supply:
+ description: RFA supply
+
+ vdd-3.3-ch0-supply:
+ description: Primary Wi-Fi antenna supply
+
+ vdd-3.3-ch1-supply:
+ description: Secondary Wi-Fi antenna supply
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,ipq4019-wifi
+ then:
+ properties:
+ interrupts:
+ minItems: 17
+ maxItems: 17
+
+ interrupt-names:
+ items:
+ - const: msi0
+ - const: msi1
+ - const: msi2
+ - const: msi3
+ - const: msi4
+ - const: msi5
+ - const: msi6
+ - const: msi7
+ - const: msi8
+ - const: msi9
+ - const: msi10
+ - const: msi11
+ - const: msi12
+ - const: msi13
+ - const: msi14
+ - const: msi15
+ - const: legacy
+
+ clocks:
+ items:
+ - description: Wi-Fi command clock
+ - description: Wi-Fi reference clock
+ - description: Wi-Fi RTC clock
+
+ clock-names:
+ items:
+ - const: wifi_wcss_cmd
+ - const: wifi_wcss_ref
+ - const: wifi_wcss_rtc
+
+ required:
+ - clocks
+ - clock-names
+ - interrupts
+ - interrupt-names
+ - resets
+ - reset-names
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,wcn3990-wifi
+
+ then:
+ properties:
+ clocks:
+ minItems: 1
+ items:
+ - description: XO reference clock
+ - description: Qualcomm Debug Subsystem clock
+
+ clock-names:
+ minItems: 1
+ items:
+ - const: cxo_ref_clk_pin
+ - const: qdss
+
+ interrupts:
+ items:
+ - description: CE0
+ - description: CE1
+ - description: CE2
+ - description: CE3
+ - description: CE4
+ - description: CE5
+ - description: CE6
+ - description: CE7
+ - description: CE8
+ - description: CE9
+ - description: CE10
+ - description: CE11
+
+ interrupt-names: false
+
+ required:
+ - interrupts
+
+examples:
+ # SNoC
+ - |
+ #include <dt-bindings/clock/qcom,rpmcc.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ wifi@18800000 {
+ compatible = "qcom,wcn3990-wifi";
+ reg = <0x18800000 0x800000>;
+ reg-names = "membase";
+ memory-region = <&wlan_msa_mem>;
+ clocks = <&rpmcc RPM_SMD_RF_CLK2_PIN>;
+ clock-names = "cxo_ref_clk_pin";
+ interrupts = <GIC_SPI 413 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 414 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 415 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 416 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 417 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 418 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 420 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 421 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 422 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 423 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 424 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 425 IRQ_TYPE_LEVEL_HIGH>;
+ iommus = <&anoc2_smmu 0x1900>,
+ <&anoc2_smmu 0x1901>;
+ qcom,snoc-host-cap-8bit-quirk;
+ vdd-0.8-cx-mx-supply = <&vreg_l5a_0p8>;
+ vdd-1.8-xo-supply = <&vreg_l7a_1p8>;
+ vdd-1.3-rfa-supply = <&vreg_l17a_1p3>;
+ vdd-3.3-ch0-supply = <&vreg_l25a_3p3>;
+ vdd-3.3-ch1-supply = <&vreg_l23a_3p3>;
+
+ wifi-firmware {
+ iommus = <&apps_smmu 0x1c02 0x1>;
+ };
+ };
+
+ # AHB
+ - |
+ #include <dt-bindings/clock/qcom,gcc-ipq4019.h>
+
+ wifi@a000000 {
+ compatible = "qcom,ipq4019-wifi";
+ reg = <0xa000000 0x200000>;
+ resets = <&gcc WIFI0_CPU_INIT_RESET>,
+ <&gcc WIFI0_RADIO_SRIF_RESET>,
+ <&gcc WIFI0_RADIO_WARM_RESET>,
+ <&gcc WIFI0_RADIO_COLD_RESET>,
+ <&gcc WIFI0_CORE_WARM_RESET>,
+ <&gcc WIFI0_CORE_COLD_RESET>;
+ reset-names = "wifi_cpu_init",
+ "wifi_radio_srif",
+ "wifi_radio_warm",
+ "wifi_radio_cold",
+ "wifi_core_warm",
+ "wifi_core_cold";
+ clocks = <&gcc GCC_WCSS2G_CLK>,
+ <&gcc GCC_WCSS2G_REF_CLK>,
+ <&gcc GCC_WCSS2G_RTC_CLK>;
+ clock-names = "wifi_wcss_cmd",
+ "wifi_wcss_ref",
+ "wifi_wcss_rtc";
+ interrupts = <GIC_SPI 32 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 33 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 34 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 35 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 36 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 37 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 38 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 39 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 40 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 41 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 42 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 43 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 44 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 45 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 46 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 47 IRQ_TYPE_EDGE_RISING>,
+ <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "msi0",
+ "msi1",
+ "msi2",
+ "msi3",
+ "msi4",
+ "msi5",
+ "msi6",
+ "msi7",
+ "msi8",
+ "msi9",
+ "msi10",
+ "msi11",
+ "msi12",
+ "msi13",
+ "msi14",
+ "msi15",
+ "legacy";
+ };
diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath11k-pci.yaml b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k-pci.yaml
new file mode 100644
index 0000000..817f02a
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k-pci.yaml
@@ -0,0 +1,58 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (c) 2023 Linaro Limited
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/wireless/qcom,ath11k-pci.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Technologies ath11k wireless devices (PCIe)
+
+maintainers:
+ - Kalle Valo <kvalo@kernel.org>
+
+description: |
+ Qualcomm Technologies IEEE 802.11ax PCIe devices
+
+properties:
+ compatible:
+ enum:
+ - pci17cb,1103 # WCN6855
+
+ reg:
+ maxItems: 1
+
+ qcom,ath11k-calibration-variant:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: |
+ string to uniquely identify variant of the calibration data for designs
+ with colliding bus and device ids
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ pcie {
+ #address-cells = <3>;
+ #size-cells = <2>;
+
+ pcie@0 {
+ device_type = "pci";
+ reg = <0x0 0x0 0x0 0x0 0x0>;
+ #address-cells = <3>;
+ #size-cells = <2>;
+ ranges;
+
+ bus-range = <0x01 0xff>;
+
+ wifi@0 {
+ compatible = "pci17cb,1103";
+ reg = <0x10000 0x0 0x0 0x0 0x0>;
+
+ qcom,ath11k-calibration-variant = "LE_X13S";
+ };
+ };
+ };
diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
index bfb5168..c3851fe 100644
--- a/Documentation/kbuild/llvm.rst
+++ b/Documentation/kbuild/llvm.rst
@@ -171,6 +171,10 @@
Getting LLVM
-------------
+We provide prebuilt stable versions of LLVM on `kernel.org <https://kernel.org/pub/tools/llvm/>`_.
+Below are links that may be useful for building LLVM from source or procuring
+it through a distribution's package manager.
+
- https://releases.llvm.org/download.html
- https://github.com/llvm/llvm-project
- https://llvm.org/docs/GettingStarted.html
diff --git a/Documentation/leds/well-known-leds.txt b/Documentation/leds/well-known-leds.txt
index 2160382..e9c30dc 100644
--- a/Documentation/leds/well-known-leds.txt
+++ b/Documentation/leds/well-known-leds.txt
@@ -70,3 +70,33 @@
* Screen
Good: ":backlight" (Motorola Droid 4)
+
+* Ethernet LEDs
+
+Currently two types of Network LEDs are support, those controlled by
+the PHY and those by the MAC. In theory both can be present at the
+same time for one Linux netdev, hence the names need to differ between
+MAC and PHY.
+
+Do not use the netdev name, such as eth0, enp1s0. These are not stable
+and are not unique. They also don't differentiate between MAC and PHY.
+
+** MAC LEDs
+
+Good: f1070000.ethernet:white:WAN
+Good: mdio_mux-0.1:00:green:left
+Good: 0000:02:00.0:yellow:top
+
+The first part must uniquely name the MAC controller. Then follows the
+colour. WAN/LAN should be used for a single LED. If there are
+multiple LEDs, use left/right, or top/bottom to indicate their
+position on the RJ45 socket.
+
+** PHY LEDs
+
+Good: f1072004.mdio-mii:00: white:WAN
+Good: !mdio-mux!mdio@2!switch@0!mdio:01:green:right
+Good: r8169-0-200:00:yellow:bottom
+
+The first part must uniquely name the PHY. This often means uniquely
+identifying the MDIO bus controller, and the address on the bus.
diff --git a/Documentation/netlink/specs/handshake.yaml b/Documentation/netlink/specs/handshake.yaml
new file mode 100644
index 0000000..614f1a5
--- /dev/null
+++ b/Documentation/netlink/specs/handshake.yaml
@@ -0,0 +1,124 @@
+# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+#
+# Author: Chuck Lever <chuck.lever@oracle.com>
+#
+# Copyright (c) 2023, Oracle and/or its affiliates.
+#
+
+name: handshake
+
+protocol: genetlink
+
+doc: Netlink protocol to request a transport layer security handshake.
+
+definitions:
+ -
+ type: enum
+ name: handler-class
+ value-start: 0
+ entries: [ none, tlshd, max ]
+ -
+ type: enum
+ name: msg-type
+ value-start: 0
+ entries: [ unspec, clienthello, serverhello ]
+ -
+ type: enum
+ name: auth
+ value-start: 0
+ entries: [ unspec, unauth, psk, x509 ]
+
+attribute-sets:
+ -
+ name: x509
+ attributes:
+ -
+ name: cert
+ type: u32
+ -
+ name: privkey
+ type: u32
+ -
+ name: accept
+ attributes:
+ -
+ name: sockfd
+ type: u32
+ -
+ name: handler-class
+ type: u32
+ enum: handler-class
+ -
+ name: message-type
+ type: u32
+ enum: msg-type
+ -
+ name: timeout
+ type: u32
+ -
+ name: auth-mode
+ type: u32
+ enum: auth
+ -
+ name: peer-identity
+ type: u32
+ multi-attr: true
+ -
+ name: certificate
+ type: nest
+ nested-attributes: x509
+ multi-attr: true
+ -
+ name: done
+ attributes:
+ -
+ name: status
+ type: u32
+ -
+ name: sockfd
+ type: u32
+ -
+ name: remote-auth
+ type: u32
+ multi-attr: true
+
+operations:
+ list:
+ -
+ name: ready
+ doc: Notify handlers that a new handshake request is waiting
+ notify: accept
+ -
+ name: accept
+ doc: Handler retrieves next queued handshake request
+ attribute-set: accept
+ flags: [ admin-perm ]
+ do:
+ request:
+ attributes:
+ - handler-class
+ reply:
+ attributes:
+ - sockfd
+ - message-type
+ - timeout
+ - auth-mode
+ - peer-identity
+ - certificate
+ -
+ name: done
+ doc: Handler reports handshake completion
+ attribute-set: done
+ do:
+ request:
+ attributes:
+ - status
+ - sockfd
+ - remote-auth
+
+mcast-groups:
+ list:
+ -
+ name: none
+ -
+ name: tlshd
diff --git a/Documentation/networking/device_drivers/ethernet/amd/pds_core.rst b/Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
new file mode 100644
index 0000000..9e8a16c
--- /dev/null
+++ b/Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
@@ -0,0 +1,139 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+========================================================
+Linux Driver for the AMD/Pensando(R) DSC adapter family
+========================================================
+
+Copyright(c) 2023 Advanced Micro Devices, Inc
+
+Identifying the Adapter
+=======================
+
+To find if one or more AMD/Pensando PCI Core devices are installed on the
+host, check for the PCI devices::
+
+ # lspci -d 1dd8:100c
+ b5:00.0 Processing accelerators: Pensando Systems Device 100c
+ b6:00.0 Processing accelerators: Pensando Systems Device 100c
+
+If such devices are listed as above, then the pds_core.ko driver should find
+and configure them for use. There should be log entries in the kernel
+messages such as these::
+
+ $ dmesg | grep pds_core
+ pds_core 0000:b5:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
+ pds_core 0000:b5:00.0: FW: 1.60.0-73
+ pds_core 0000:b6:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
+ pds_core 0000:b6:00.0: FW: 1.60.0-73
+
+Driver and firmware version information can be gathered with devlink::
+
+ $ devlink dev info pci/0000:b5:00.0
+ pci/0000:b5:00.0:
+ driver pds_core
+ serial_number FLM18420073
+ versions:
+ fixed:
+ asic.id 0x0
+ asic.rev 0x0
+ running:
+ fw 1.51.0-73
+ stored:
+ fw.goldfw 1.15.9-C-22
+ fw.mainfwa 1.60.0-73
+ fw.mainfwb 1.60.0-57
+
+Info versions
+=============
+
+The ``pds_core`` driver reports the following versions
+
+.. list-table:: devlink info versions implemented
+ :widths: 5 5 90
+
+ * - Name
+ - Type
+ - Description
+ * - ``fw``
+ - running
+ - Version of firmware running on the device
+ * - ``fw.goldfw``
+ - stored
+ - Version of firmware stored in the goldfw slot
+ * - ``fw.mainfwa``
+ - stored
+ - Version of firmware stored in the mainfwa slot
+ * - ``fw.mainfwb``
+ - stored
+ - Version of firmware stored in the mainfwb slot
+ * - ``asic.id``
+ - fixed
+ - The ASIC type for this device
+ * - ``asic.rev``
+ - fixed
+ - The revision of the ASIC for this device
+
+Parameters
+==========
+
+The ``pds_core`` driver implements the following generic
+parameters for controlling the functionality to be made available
+as auxiliary_bus devices.
+
+.. list-table:: Generic parameters implemented
+ :widths: 5 5 8 82
+
+ * - Name
+ - Mode
+ - Type
+ - Description
+ * - ``enable_vnet``
+ - runtime
+ - Boolean
+ - Enables vDPA functionality through an auxiliary_bus device
+
+Firmware Management
+===================
+
+The ``flash`` command can update a the DSC firmware. The downloaded firmware
+will be saved into either of firmware bank 1 or bank 2, whichever is not
+currently in use, and that bank will used for the next boot::
+
+ # devlink dev flash pci/0000:b5:00.0 \
+ file pensando/dsc_fw_1.63.0-22.tar
+
+Health Reporters
+================
+
+The driver supports a devlink health reporter for FW status::
+
+ # devlink health show pci/0000:2b:00.0 reporter fw
+ pci/0000:2b:00.0:
+ reporter fw
+ state healthy error 0 recover 0
+ # devlink health diagnose pci/0000:2b:00.0 reporter fw
+ Status: healthy State: 1 Generation: 0 Recoveries: 0
+
+Enabling the driver
+===================
+
+The driver is enabled via the standard kernel configuration system,
+using the make command::
+
+ make oldconfig/menuconfig/etc.
+
+The driver is located in the menu structure at:
+
+ -> Device Drivers
+ -> Network device support (NETDEVICES [=y])
+ -> Ethernet driver support
+ -> AMD devices
+ -> AMD/Pensando Ethernet PDS_CORE Support
+
+Support
+=======
+
+For general Linux networking support, please use the netdev mailing
+list, which is monitored by AMD/Pensando personnel::
+
+ netdev@vger.kernel.org
diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
index 6e9e701..417ca51 100644
--- a/Documentation/networking/device_drivers/ethernet/index.rst
+++ b/Documentation/networking/device_drivers/ethernet/index.rst
@@ -14,6 +14,7 @@
3com/vortex
amazon/ena
altera/altera_tse
+ amd/pds_core
aquantia/atlantic
chelsio/cxgb
cirrus/cs89x0
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 10f282c..2f60e34 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -7,6 +7,21 @@
This document describes the devlink features implemented by the ``ice``
device driver.
+Parameters
+==========
+
+.. list-table:: Generic parameters implemented
+
+ * - Name
+ - Mode
+ - Notes
+ * - ``enable_roce``
+ - runtime
+ - mutually exclusive with ``enable_iwarp``
+ * - ``enable_iwarp``
+ - runtime
+ - mutually exclusive with ``enable_roce``
+
Info versions
=============
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index 24bb256..a164ff0 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -36,6 +36,7 @@
scaling
tls
tls-offload
+ tls-handshake
nfc
6lowpan
6pack
diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst
index 58a78a3..6ec06a3 100644
--- a/Documentation/networking/ip-sysctl.rst
+++ b/Documentation/networking/ip-sysctl.rst
@@ -2721,6 +2721,13 @@
Default: 0
+error_anycast_as_unicast - BOOLEAN
+ If set to 1, then the kernel will respond with ICMP Errors
+ resulting from requests sent to it over the IPv6 protocol destined
+ to anycast address essentially treating anycast as unicast.
+
+ Default: 0
+
xfrm6_gc_thresh - INTEGER
(Obsolete since linux-4.14)
The threshold at which we will start garbage collecting for IPv6
diff --git a/Documentation/networking/page_pool.rst b/Documentation/networking/page_pool.rst
index 30f1344..873efd9 100644
--- a/Documentation/networking/page_pool.rst
+++ b/Documentation/networking/page_pool.rst
@@ -165,6 +165,7 @@
pp_params.pool_size = DESC_NUM;
pp_params.nid = NUMA_NO_NODE;
pp_params.dev = priv->dev;
+ pp_params.napi = napi; /* only if locking is tied to NAPI */
pp_params.dma_dir = xdp_prog ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE;
page_pool = page_pool_create(&pp_params);
diff --git a/Documentation/networking/tls-handshake.rst b/Documentation/networking/tls-handshake.rst
new file mode 100644
index 0000000..a2817a8
--- /dev/null
+++ b/Documentation/networking/tls-handshake.rst
@@ -0,0 +1,217 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================
+In-Kernel TLS Handshake
+=======================
+
+Overview
+========
+
+Transport Layer Security (TLS) is a Upper Layer Protocol (ULP) that runs
+over TCP. TLS provides end-to-end data integrity and confidentiality in
+addition to peer authentication.
+
+The kernel's kTLS implementation handles the TLS record subprotocol, but
+does not handle the TLS handshake subprotocol which is used to establish
+a TLS session. Kernel consumers can use the API described here to
+request TLS session establishment.
+
+There are several possible ways to provide a handshake service in the
+kernel. The API described here is designed to hide the details of those
+implementations so that in-kernel TLS consumers do not need to be
+aware of how the handshake gets done.
+
+
+User handshake agent
+====================
+
+As of this writing, there is no TLS handshake implementation in the
+Linux kernel. To provide a handshake service, a handshake agent
+(typically in user space) is started in each network namespace where a
+kernel consumer might require a TLS handshake. Handshake agents listen
+for events sent from the kernel that indicate a handshake request is
+waiting.
+
+An open socket is passed to a handshake agent via a netlink operation,
+which creates a socket descriptor in the agent's file descriptor table.
+If the handshake completes successfully, the handshake agent promotes
+the socket to use the TLS ULP and sets the session information using the
+SOL_TLS socket options. The handshake agent returns the socket to the
+kernel via a second netlink operation.
+
+
+Kernel Handshake API
+====================
+
+A kernel TLS consumer initiates a client-side TLS handshake on an open
+socket by invoking one of the tls_client_hello() functions. First, it
+fills in a structure that contains the parameters of the request:
+
+.. code-block:: c
+
+ struct tls_handshake_args {
+ struct socket *ta_sock;
+ tls_done_func_t ta_done;
+ void *ta_data;
+ unsigned int ta_timeout_ms;
+ key_serial_t ta_keyring;
+ key_serial_t ta_my_cert;
+ key_serial_t ta_my_privkey;
+ unsigned int ta_num_peerids;
+ key_serial_t ta_my_peerids[5];
+ };
+
+The @ta_sock field references an open and connected socket. The consumer
+must hold a reference on the socket to prevent it from being destroyed
+while the handshake is in progress. The consumer must also have
+instantiated a struct file in sock->file.
+
+
+@ta_done contains a callback function that is invoked when the handshake
+has completed. Further explanation of this function is in the "Handshake
+Completion" sesction below.
+
+The consumer can fill in the @ta_timeout_ms field to force the servicing
+handshake agent to exit after a number of milliseconds. This enables the
+socket to be fully closed once both the kernel and the handshake agent
+have closed their endpoints.
+
+Authentication material such as x.509 certificates, private certificate
+keys, and pre-shared keys are provided to the handshake agent in keys
+that are instantiated by the consumer before making the handshake
+request. The consumer can provide a private keyring that is linked into
+the handshake agent's process keyring in the @ta_keyring field to prevent
+access of those keys by other subsystems.
+
+To request an x.509-authenticated TLS session, the consumer fills in
+the @ta_my_cert and @ta_my_privkey fields with the serial numbers of
+keys containing an x.509 certificate and the private key for that
+certificate. Then, it invokes this function:
+
+.. code-block:: c
+
+ ret = tls_client_hello_x509(args, gfp_flags);
+
+The function returns zero when the handshake request is under way. A
+zero return guarantees the callback function @ta_done will be invoked
+for this socket. The function returns a negative errno if the handshake
+could not be started. A negative errno guarantees the callback function
+@ta_done will not be invoked on this socket.
+
+
+To initiate a client-side TLS handshake with a pre-shared key, use:
+
+.. code-block:: c
+
+ ret = tls_client_hello_psk(args, gfp_flags);
+
+However, in this case, the consumer fills in the @ta_my_peerids array
+with serial numbers of keys containing the peer identities it wishes
+to offer, and the @ta_num_peerids field with the number of array
+entries it has filled in. The other fields are filled in as above.
+
+
+To initiate an anonymous client-side TLS handshake use:
+
+.. code-block:: c
+
+ ret = tls_client_hello_anon(args, gfp_flags);
+
+The handshake agent presents no peer identity information to the remote
+during this type of handshake. Only server authentication (ie the client
+verifies the server's identity) is performed during the handshake. Thus
+the established session uses encryption only.
+
+
+Consumers that are in-kernel servers use:
+
+.. code-block:: c
+
+ ret = tls_server_hello_x509(args, gfp_flags);
+
+or
+
+.. code-block:: c
+
+ ret = tls_server_hello_psk(args, gfp_flags);
+
+The argument structure is filled in as above.
+
+
+If the consumer needs to cancel the handshake request, say, due to a ^C
+or other exigent event, the consumer can invoke:
+
+.. code-block:: c
+
+ bool tls_handshake_cancel(sock);
+
+This function returns true if the handshake request associated with
+@sock has been canceled. The consumer's handshake completion callback
+will not be invoked. If this function returns false, then the consumer's
+completion callback has already been invoked.
+
+
+Handshake Completion
+====================
+
+When the handshake agent has completed processing, it notifies the
+kernel that the socket may be used by the consumer again. At this point,
+the consumer's handshake completion callback, provided in the @ta_done
+field in the tls_handshake_args structure, is invoked.
+
+The synopsis of this function is:
+
+.. code-block:: c
+
+ typedef void (*tls_done_func_t)(void *data, int status,
+ key_serial_t peerid);
+
+The consumer provides a cookie in the @ta_data field of the
+tls_handshake_args structure that is returned in the @data parameter of
+this callback. The consumer uses the cookie to match the callback to the
+thread waiting for the handshake to complete.
+
+The success status of the handshake is returned via the @status
+parameter:
+
++------------+----------------------------------------------+
+| status | meaning |
++============+==============================================+
+| 0 | TLS session established successfully |
++------------+----------------------------------------------+
+| -EACCESS | Remote peer rejected the handshake or |
+| | authentication failed |
++------------+----------------------------------------------+
+| -ENOMEM | Temporary resource allocation failure |
++------------+----------------------------------------------+
+| -EINVAL | Consumer provided an invalid argument |
++------------+----------------------------------------------+
+| -ENOKEY | Missing authentication material |
++------------+----------------------------------------------+
+| -EIO | An unexpected fault occurred |
++------------+----------------------------------------------+
+
+The @peerid parameter contains the serial number of a key containing the
+remote peer's identity or the value TLS_NO_PEERID if the session is not
+authenticated.
+
+A best practice is to close and destroy the socket immediately if the
+handshake failed.
+
+
+Other considerations
+--------------------
+
+While a handshake is under way, the kernel consumer must alter the
+socket's sk_data_ready callback function to ignore all incoming data.
+Once the handshake completion callback function has been invoked, normal
+receive operation can be resumed.
+
+Once a TLS session is established, the consumer must provide a buffer
+for and then examine the control message (CMSG) that is part of every
+subsequent sock_recvmsg(). Each control message indicates whether the
+received message data is TLS record data or session metadata.
+
+See tls.rst for details on how a kTLS consumer recognizes incoming
+(decrypted) application data, alerts, and handshake packets once the
+socket has been promoted to use the TLS ULP.
diff --git a/Documentation/riscv/vm-layout.rst b/Documentation/riscv/vm-layout.rst
index 3be44e7..5462c84 100644
--- a/Documentation/riscv/vm-layout.rst
+++ b/Documentation/riscv/vm-layout.rst
@@ -47,7 +47,7 @@
| Kernel-space virtual memory, shared between all processes:
____________________________________________________________|___________________________________________________________
| | | |
- ffffffc6fee00000 | -228 GB | ffffffc6feffffff | 2 MB | fixmap
+ ffffffc6fea00000 | -228 GB | ffffffc6feffffff | 6 MB | fixmap
ffffffc6ff000000 | -228 GB | ffffffc6ffffffff | 16 MB | PCI io
ffffffc700000000 | -228 GB | ffffffc7ffffffff | 4 GB | vmemmap
ffffffc800000000 | -224 GB | ffffffd7ffffffff | 64 GB | vmalloc/ioremap space
@@ -83,7 +83,7 @@
| Kernel-space virtual memory, shared between all processes:
____________________________________________________________|___________________________________________________________
| | | |
- ffff8d7ffee00000 | -114.5 TB | ffff8d7ffeffffff | 2 MB | fixmap
+ ffff8d7ffea00000 | -114.5 TB | ffff8d7ffeffffff | 6 MB | fixmap
ffff8d7fff000000 | -114.5 TB | ffff8d7fffffffff | 16 MB | PCI io
ffff8d8000000000 | -114.5 TB | ffff8f7fffffffff | 2 TB | vmemmap
ffff8f8000000000 | -112.5 TB | ffffaf7fffffffff | 32 TB | vmalloc/ioremap space
@@ -119,7 +119,7 @@
| Kernel-space virtual memory, shared between all processes:
____________________________________________________________|___________________________________________________________
| | | |
- ff1bfffffee00000 | -57 PB | ff1bfffffeffffff | 2 MB | fixmap
+ ff1bfffffea00000 | -57 PB | ff1bfffffeffffff | 6 MB | fixmap
ff1bffffff000000 | -57 PB | ff1bffffffffffff | 16 MB | PCI io
ff1c000000000000 | -57 PB | ff1fffffffffffff | 1 PB | vmemmap
ff20000000000000 | -56 PB | ff5fffffffffffff | 16 PB | vmalloc/ioremap space
diff --git a/Documentation/rust/arch-support.rst b/Documentation/rust/arch-support.rst
index ed7f4f5b..b91e9ef 100644
--- a/Documentation/rust/arch-support.rst
+++ b/Documentation/rust/arch-support.rst
@@ -15,7 +15,7 @@
============ ================ ==============================================
Architecture Level of support Constraints
============ ================ ==============================================
-``x86`` Maintained ``x86_64`` only.
``um`` Maintained ``x86_64`` only.
+``x86`` Maintained ``x86_64`` only.
============ ================ ==============================================
diff --git a/Documentation/sound/hd-audio/models.rst b/Documentation/sound/hd-audio/models.rst
index 9b52f50..1204304 100644
--- a/Documentation/sound/hd-audio/models.rst
+++ b/Documentation/sound/hd-audio/models.rst
@@ -704,7 +704,7 @@
no-jd
BIOS setup but without jack-detection
intel
- Intel DG45* mobos
+ Intel D*45* mobos
dell-m6-amic
Dell desktops/laptops with analog mics
dell-m6-dmic
diff --git a/MAINTAINERS b/MAINTAINERS
index b8b275e2..6ac562e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1041,6 +1041,15 @@
F: include/uapi/linux/kfd_ioctl.h
F: include/uapi/linux/kfd_sysfs.h
+AMD PDS CORE DRIVER
+M: Shannon Nelson <shannon.nelson@amd.com>
+M: Brett Creeley <brett.creeley@amd.com>
+L: netdev@vger.kernel.org
+S: Supported
+F: Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
+F: drivers/net/ethernet/amd/pds_core/
+F: include/linux/pds/
+
AMD SPI DRIVER
M: Sanjay R Mehta <sanju.mehta@amd.com>
S: Maintained
@@ -8947,6 +8956,17 @@
T: git git://linuxtv.org/anttip/media_tree.git
F: drivers/media/usb/hackrf/
+HANDSHAKE UPCALL FOR TRANSPORT LAYER SECURITY
+M: Chuck Lever <chuck.lever@oracle.com>
+L: kernel-tls-handshake@lists.linux.dev
+L: netdev@vger.kernel.org
+S: Maintained
+F: Documentation/netlink/specs/handshake.yaml
+F: Documentation/networking/tls-handshake.rst
+F: include/net/handshake.h
+F: include/trace/events/handshake.h
+F: net/handshake/
+
HANTRO VPU CODEC DRIVER
M: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
M: Philipp Zabel <p.zabel@pengutronix.de>
@@ -14612,11 +14632,14 @@
NETWORKING [MPTCP]
M: Matthieu Baerts <matthieu.baerts@tessares.net>
+M: Mat Martineau <martineau@kernel.org>
L: netdev@vger.kernel.org
L: mptcp@lists.linux.dev
S: Maintained
W: https://github.com/multipath-tcp/mptcp_net-next/wiki
B: https://github.com/multipath-tcp/mptcp_net-next/issues
+T: git https://github.com/multipath-tcp/mptcp_net-next.git export-net
+T: git https://github.com/multipath-tcp/mptcp_net-next.git export
F: Documentation/networking/mptcp-sysctl.rst
F: include/net/mptcp.h
F: include/trace/events/mptcp.h
@@ -17218,7 +17241,7 @@
W: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k
T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git
F: drivers/net/wireless/ath/ath10k/
-F: Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt
+F: Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml
QUALCOMM ATHEROS ATH11K WIRELESS DRIVER
M: Kalle Valo <kvalo@kernel.org>
@@ -19926,6 +19949,13 @@
S: Maintained
F: arch/riscv/boot/dts/starfive/
+STARFIVE DWMAC GLUE LAYER
+M: Emil Renner Berthing <kernel@esmil.dk>
+M: Samin Guo <samin.guo@starfivetech.com>
+S: Maintained
+F: Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml
+F: drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c
+
STARFIVE JH7100 CLOCK DRIVERS
M: Emil Renner Berthing <kernel@esmil.dk>
S: Maintained
diff --git a/Makefile b/Makefile
index 5aeea3d..b5c48e3 100644
--- a/Makefile
+++ b/Makefile
@@ -2,7 +2,7 @@
VERSION = 6
PATCHLEVEL = 3
SUBLEVEL = 0
-EXTRAVERSION = -rc6
+EXTRAVERSION = -rc7
NAME = Hurr durr I'ma ninja sloth
# *DOCUMENTATION*
diff --git a/arch/arm/boot/dts/armada-370-rd.dts b/arch/arm/boot/dts/armada-370-rd.dts
index be005c9..2586f32 100644
--- a/arch/arm/boot/dts/armada-370-rd.dts
+++ b/arch/arm/boot/dts/armada-370-rd.dts
@@ -20,6 +20,7 @@
/dts-v1/;
#include <dt-bindings/input/input.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/leds/common.h>
#include <dt-bindings/gpio/gpio.h>
#include "armada-370.dtsi"
@@ -135,6 +136,17 @@ &mdio {
pinctrl-names = "default";
phy0: ethernet-phy@0 {
reg = <0>;
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_WHITE>;
+ function = LED_FUNCTION_WAN;
+ default-state = "keep";
+ };
+ };
};
switch: switch@10 {
diff --git a/arch/arm/boot/dts/imx6ull-colibri.dtsi b/arch/arm/boot/dts/imx6ull-colibri.dtsi
index bf64ba8..fde8a19 100644
--- a/arch/arm/boot/dts/imx6ull-colibri.dtsi
+++ b/arch/arm/boot/dts/imx6ull-colibri.dtsi
@@ -33,15 +33,9 @@ connector {
self-powered;
type = "micro";
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- usb_dr_connector: endpoint {
- remote-endpoint = <&usb1_drd_sw>;
- };
+ port {
+ usb_dr_connector: endpoint {
+ remote-endpoint = <&usb1_drd_sw>;
};
};
};
diff --git a/arch/arm/boot/dts/imx7d-remarkable2.dts b/arch/arm/boot/dts/imx7d-remarkable2.dts
index 8b2f11e..427f8d0 100644
--- a/arch/arm/boot/dts/imx7d-remarkable2.dts
+++ b/arch/arm/boot/dts/imx7d-remarkable2.dts
@@ -118,8 +118,6 @@ sy7636a: pmic@62 {
reg = <0x62>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_epdpmic>;
- #address-cells = <1>;
- #size-cells = <0>;
#thermal-sensor-cells = <0>;
epd-pwr-good-gpios = <&gpio6 21 GPIO_ACTIVE_HIGH>;
diff --git a/arch/arm/boot/dts/qcom-ipq8064-rb3011.dts b/arch/arm/boot/dts/qcom-ipq8064-rb3011.dts
index f908889..4d50987 100644
--- a/arch/arm/boot/dts/qcom-ipq8064-rb3011.dts
+++ b/arch/arm/boot/dts/qcom-ipq8064-rb3011.dts
@@ -38,8 +38,6 @@ mdio0: mdio-0 {
switch0: switch@10 {
compatible = "qca,qca8337";
- #address-cells = <1>;
- #size-cells = <0>;
dsa,member = <0 0>;
@@ -67,26 +65,86 @@ fixed-link {
port@1 {
reg = <1>;
label = "sw1";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@2 {
reg = <2>;
label = "sw2";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@3 {
reg = <3>;
label = "sw3";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@4 {
reg = <4>;
label = "sw4";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@5 {
reg = <5>;
label = "sw5";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
};
};
@@ -105,8 +163,6 @@ mdio1: mdio-1 {
switch1: switch@14 {
compatible = "qca,qca8337";
- #address-cells = <1>;
- #size-cells = <0>;
dsa,member = <1 0>;
@@ -134,26 +190,86 @@ fixed-link {
port@1 {
reg = <1>;
label = "sw6";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@2 {
reg = <2>;
label = "sw7";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@3 {
reg = <3>;
label = "sw8";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@4 {
reg = <4>;
label = "sw9";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
port@5 {
reg = <5>;
label = "sw10";
+
+ leds {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@0 {
+ reg = <0>;
+ color = <LED_COLOR_ID_GREEN>;
+ function = LED_FUNCTION_LAN;
+ default-state = "keep";
+ };
+ };
};
};
};
diff --git a/arch/arm/boot/dts/rk3288.dtsi b/arch/arm/boot/dts/rk3288.dtsi
index 2ca76b6..511ca86 100644
--- a/arch/arm/boot/dts/rk3288.dtsi
+++ b/arch/arm/boot/dts/rk3288.dtsi
@@ -942,7 +942,7 @@ wdt: watchdog@ff800000 {
status = "disabled";
};
- spdif: sound@ff88b0000 {
+ spdif: sound@ff8b0000 {
compatible = "rockchip,rk3288-spdif", "rockchip,rk3066-spdif";
reg = <0x0 0xff8b0000 0x0 0x10000>;
#sound-dai-cells = <0>;
diff --git a/arch/arm/configs/imx_v6_v7_defconfig b/arch/arm/configs/imx_v6_v7_defconfig
index 6dc6fed..8d002c6 100644
--- a/arch/arm/configs/imx_v6_v7_defconfig
+++ b/arch/arm/configs/imx_v6_v7_defconfig
@@ -76,7 +76,7 @@
CONFIG_RFKILL_INPUT=y
CONFIG_PCI=y
CONFIG_PCI_MSI=y
-CONFIG_PCI_IMX6=y
+CONFIG_PCI_IMX6_HOST=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
index 123a56f..feb27a0 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
@@ -1571,15 +1571,20 @@ usb2_phy0: phy@36000 {
dmc: bus@38000 {
compatible = "simple-bus";
- reg = <0x0 0x38000 0x0 0x400>;
#address-cells = <2>;
#size-cells = <2>;
- ranges = <0x0 0x0 0x0 0x38000 0x0 0x400>;
+ ranges = <0x0 0x0 0x0 0x38000 0x0 0x2000>;
canvas: video-lut@48 {
compatible = "amlogic,canvas";
reg = <0x0 0x48 0x0 0x14>;
};
+
+ pmu: pmu@80 {
+ reg = <0x0 0x80 0x0 0x40>,
+ <0x0 0xc00 0x0 0x40>;
+ interrupts = <GIC_SPI 52 IRQ_TYPE_EDGE_RISING>;
+ };
};
usb2_phy1: phy@3a000 {
@@ -1705,12 +1710,6 @@ internal_ephy: ethernet-phy@8 {
};
};
- pmu: pmu@ff638000 {
- reg = <0x0 0xff638000 0x0 0x100>,
- <0x0 0xff638c00 0x0 0x100>;
- interrupts = <GIC_SPI 52 IRQ_TYPE_EDGE_RISING>;
- };
-
aobus: bus@ff800000 {
compatible = "simple-bus";
reg = <0x0 0xff800000 0x0 0x100000>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
index d1a6390..3f9dfd4 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
@@ -194,7 +194,7 @@ pmic@4b {
rohm,reset-snvs-powered;
#clock-cells = <0>;
- clocks = <&osc_32k 0>;
+ clocks = <&osc_32k>;
clock-output-names = "clk-32k-out";
regulators {
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
index 88321b5..6f08115 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-verdin.dtsi
@@ -99,7 +99,7 @@ reg_ethphy: regulator-ethphy {
compatible = "regulator-fixed";
enable-active-high;
gpio = <&gpio2 20 GPIO_ACTIVE_HIGH>; /* PMIC_EN_ETH */
- off-on-delay = <500000>;
+ off-on-delay-us = <500000>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_reg_eth>;
regulator-always-on;
@@ -139,7 +139,7 @@ reg_usdhc2_vmmc: regulator-usdhc2 {
enable-active-high;
/* Verdin SD_1_PWR_EN (SODIMM 76) */
gpio = <&gpio3 5 GPIO_ACTIVE_HIGH>;
- off-on-delay = <100000>;
+ off-on-delay-us = <100000>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_usdhc2_pwr_en>;
regulator-max-microvolt = <3300000>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-verdin-dev.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-verdin-dev.dtsi
index 361426c..c296225 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-verdin-dev.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-verdin-dev.dtsi
@@ -10,7 +10,7 @@ reg_eth2phy: regulator-eth2phy {
compatible = "regulator-fixed";
enable-active-high;
gpio = <&gpio_expander_21 4 GPIO_ACTIVE_HIGH>; /* ETH_PWR_EN */
- off-on-delay = <500000>;
+ off-on-delay-us = <500000>;
regulator-max-microvolt = <3300000>;
regulator-min-microvolt = <3300000>;
regulator-name = "+V3.3_ETH";
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-verdin.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-verdin.dtsi
index 0dd6180..1608775 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-verdin.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-verdin.dtsi
@@ -87,7 +87,7 @@ reg_module_eth1phy: regulator-module-eth1phy {
compatible = "regulator-fixed";
enable-active-high;
gpio = <&gpio2 20 GPIO_ACTIVE_HIGH>; /* PMIC_EN_ETH */
- off-on-delay = <500000>;
+ off-on-delay-us = <500000>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_reg_eth>;
regulator-always-on;
@@ -128,7 +128,7 @@ reg_usdhc2_vmmc: regulator-usdhc2 {
enable-active-high;
/* Verdin SD_1_PWR_EN (SODIMM 76) */
gpio = <&gpio4 22 GPIO_ACTIVE_HIGH>;
- off-on-delay = <100000>;
+ off-on-delay-us = <100000>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_usdhc2_pwr_en>;
regulator-max-microvolt = <3300000>;
diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
index 2dd60e3..a237275 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
@@ -1128,7 +1128,7 @@ aips4: bus@32c00000 {
lcdif2: display-controller@32e90000 {
compatible = "fsl,imx8mp-lcdif";
- reg = <0x32e90000 0x238>;
+ reg = <0x32e90000 0x10000>;
interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk IMX8MP_CLK_MEDIA_DISP2_PIX_ROOT>,
<&clk IMX8MP_CLK_MEDIA_APB_ROOT>,
diff --git a/arch/arm64/boot/dts/qcom/ipq8074-hk01.dts b/arch/arm64/boot/dts/qcom/ipq8074-hk01.dts
index ca3f966..5cf07ca 100644
--- a/arch/arm64/boot/dts/qcom/ipq8074-hk01.dts
+++ b/arch/arm64/boot/dts/qcom/ipq8074-hk01.dts
@@ -62,11 +62,11 @@ &pcie1 {
perst-gpios = <&tlmm 58 GPIO_ACTIVE_LOW>;
};
-&pcie_phy0 {
+&pcie_qmp0 {
status = "okay";
};
-&pcie_phy1 {
+&pcie_qmp1 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/ipq8074-hk10.dtsi b/arch/arm64/boot/dts/qcom/ipq8074-hk10.dtsi
index 651a231..1b8379b 100644
--- a/arch/arm64/boot/dts/qcom/ipq8074-hk10.dtsi
+++ b/arch/arm64/boot/dts/qcom/ipq8074-hk10.dtsi
@@ -48,11 +48,11 @@ &pcie1 {
perst-gpios = <&tlmm 61 GPIO_ACTIVE_LOW>;
};
-&pcie_phy0 {
+&pcie_qmp0 {
status = "okay";
};
-&pcie_phy1 {
+&pcie_qmp1 {
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/qrb5165-rb5.dts b/arch/arm64/boot/dts/qcom/qrb5165-rb5.dts
index aa0a7bd..dd92433 100644
--- a/arch/arm64/boot/dts/qcom/qrb5165-rb5.dts
+++ b/arch/arm64/boot/dts/qcom/qrb5165-rb5.dts
@@ -1012,7 +1012,7 @@ &swr0 {
left_spkr: speaker@0,3 {
compatible = "sdw10217211000";
reg = <0 3>;
- powerdown-gpios = <&tlmm 130 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&tlmm 130 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrLeft";
#sound-dai-cells = <0>;
@@ -1021,7 +1021,7 @@ left_spkr: speaker@0,3 {
right_spkr: speaker@0,4 {
compatible = "sdw10217211000";
reg = <0 4>;
- powerdown-gpios = <&tlmm 130 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&tlmm 130 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrRight";
#sound-dai-cells = <0>;
diff --git a/arch/arm64/boot/dts/qcom/sc7280-herobrine.dtsi b/arch/arm64/boot/dts/qcom/sc7280-herobrine.dtsi
index b613781..313083e 100644
--- a/arch/arm64/boot/dts/qcom/sc7280-herobrine.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7280-herobrine.dtsi
@@ -464,7 +464,7 @@ &mdss_dp {
&mdss_dp_out {
data-lanes = <0 1>;
- link-frequencies = /bits/ 64 <1620000000 2700000000 5400000000 8100000000>;
+ link-frequencies = /bits/ 64 <1620000000 2700000000 5400000000>;
};
&mdss_mdp {
diff --git a/arch/arm64/boot/dts/qcom/sc8280xp-pmics.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp-pmics.dtsi
index df7d28f..be446eb 100644
--- a/arch/arm64/boot/dts/qcom/sc8280xp-pmics.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc8280xp-pmics.dtsi
@@ -59,8 +59,9 @@ pmk8280: pmic@0 {
#size-cells = <0>;
pmk8280_pon: pon@1300 {
- compatible = "qcom,pm8998-pon";
- reg = <0x1300>;
+ compatible = "qcom,pmk8350-pon";
+ reg = <0x1300>, <0x800>;
+ reg-names = "hlos", "pbs";
pmk8280_pon_pwrkey: pwrkey {
compatible = "qcom,pmk8350-pwrkey";
diff --git a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
index 67d2a66..5c688cb 100644
--- a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
+++ b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts
@@ -753,7 +753,7 @@ swm: swm@c85 {
left_spkr: speaker@0,3 {
compatible = "sdw10217211000";
reg = <0 3>;
- powerdown-gpios = <&wcdgpio 1 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&wcdgpio 1 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrLeft";
#sound-dai-cells = <0>;
@@ -761,7 +761,7 @@ left_spkr: speaker@0,3 {
right_spkr: speaker@0,4 {
compatible = "sdw10217211000";
- powerdown-gpios = <&wcdgpio 2 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&wcdgpio 2 GPIO_ACTIVE_LOW>;
reg = <0 4>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrRight";
diff --git a/arch/arm64/boot/dts/qcom/sdm850-samsung-w737.dts b/arch/arm64/boot/dts/qcom/sdm850-samsung-w737.dts
index 9850140..41f59e3 100644
--- a/arch/arm64/boot/dts/qcom/sdm850-samsung-w737.dts
+++ b/arch/arm64/boot/dts/qcom/sdm850-samsung-w737.dts
@@ -662,7 +662,7 @@ swm: swm@c85 {
left_spkr: speaker@0,3 {
compatible = "sdw10217211000";
reg = <0 3>;
- powerdown-gpios = <&wcdgpio 1 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&wcdgpio 1 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrLeft";
#sound-dai-cells = <0>;
@@ -670,7 +670,7 @@ left_spkr: speaker@0,3 {
right_spkr: speaker@0,4 {
compatible = "sdw10217211000";
- powerdown-gpios = <&wcdgpio 2 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&wcdgpio 2 GPIO_ACTIVE_LOW>;
reg = <0 4>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrRight";
diff --git a/arch/arm64/boot/dts/qcom/sm8250-mtp.dts b/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
index e54cdc8..4c9de23 100644
--- a/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
+++ b/arch/arm64/boot/dts/qcom/sm8250-mtp.dts
@@ -764,7 +764,7 @@ &swr0 {
left_spkr: speaker@0,3 {
compatible = "sdw10217211000";
reg = <0 3>;
- powerdown-gpios = <&tlmm 26 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&tlmm 26 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrLeft";
#sound-dai-cells = <0>;
@@ -773,7 +773,7 @@ left_spkr: speaker@0,3 {
right_spkr: speaker@0,4 {
compatible = "sdw10217211000";
reg = <0 4>;
- powerdown-gpios = <&tlmm 127 GPIO_ACTIVE_HIGH>;
+ powerdown-gpios = <&tlmm 127 GPIO_ACTIVE_LOW>;
#thermal-sensor-cells = <0>;
sound-name-prefix = "SpkrRight";
#sound-dai-cells = <0>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3326-anbernic-rg351m.dts b/arch/arm64/boot/dts/rockchip/rk3326-anbernic-rg351m.dts
index 61b3168..ce318e0 100644
--- a/arch/arm64/boot/dts/rockchip/rk3326-anbernic-rg351m.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3326-anbernic-rg351m.dts
@@ -24,6 +24,8 @@ vibrator {
&internal_display {
compatible = "elida,kd35t133";
+ iovcc-supply = <&vcc_lcd>;
+ vdd-supply = <&vcc_lcd>;
};
&pwm0 {
diff --git a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go.dtsi b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go.dtsi
index 04eba43..80fc53c 100644
--- a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go.dtsi
@@ -235,10 +235,8 @@ mipi_out_panel: endpoint {
internal_display: panel@0 {
reg = <0>;
backlight = <&backlight>;
- iovcc-supply = <&vcc_lcd>;
reset-gpios = <&gpio3 RK_PC0 GPIO_ACTIVE_LOW>;
rotation = <270>;
- vdd-supply = <&vcc_lcd>;
port {
mipi_in_panel: endpoint {
diff --git a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2-v11.dts b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2-v11.dts
index 139c898..d94ac81 100644
--- a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2-v11.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2-v11.dts
@@ -83,6 +83,8 @@ button-sw21 {
&internal_display {
compatible = "elida,kd35t133";
+ iovcc-supply = <&vcc_lcd>;
+ vdd-supply = <&vcc_lcd>;
};
&rk817 {
diff --git a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2.dts b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2.dts
index 4702183..aa6f5b1 100644
--- a/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3326-odroid-go2.dts
@@ -59,6 +59,8 @@ battery: battery {
&internal_display {
compatible = "elida,kd35t133";
+ iovcc-supply = <&vcc_lcd>;
+ vdd-supply = <&vcc_lcd>;
};
&rk817_charger {
diff --git a/arch/arm64/boot/dts/rockchip/rk3368-evb.dtsi b/arch/arm64/boot/dts/rockchip/rk3368-evb.dtsi
index 083452c6..e47d139 100644
--- a/arch/arm64/boot/dts/rockchip/rk3368-evb.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3368-evb.dtsi
@@ -61,7 +61,6 @@ backlight: backlight {
pinctrl-names = "default";
pinctrl-0 = <&bl_en>;
pwms = <&pwm0 0 1000000 PWM_POLARITY_INVERTED>;
- pwm-delay-us = <10000>;
};
emmc_pwrseq: emmc-pwrseq {
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
index ee6095b..5c1929d 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
@@ -198,7 +198,6 @@ backlight: backlight {
power-supply = <&pp3300_disp>;
pinctrl-names = "default";
pinctrl-0 = <&bl_en>;
- pwm-delay-us = <10000>;
};
gpio_keys: gpio-keys {
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
index a47d9f7..c5e7de6 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
@@ -167,7 +167,6 @@ backlight: backlight {
pinctrl-names = "default";
pinctrl-0 = <&bl_en>;
pwms = <&pwm1 0 1000000 0>;
- pwm-delay-us = <10000>;
};
dmic: dmic {
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-pinebook-pro.dts b/arch/arm64/boot/dts/rockchip/rk3399-pinebook-pro.dts
index 194e48c..ddd45de 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-pinebook-pro.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3399-pinebook-pro.dts
@@ -50,19 +50,9 @@ edp_panel: edp-panel {
pinctrl-0 = <&panel_en_pin>;
power-supply = <&vcc3v3_panel>;
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
-
- port@0 {
- reg = <0>;
- #address-cells = <1>;
- #size-cells = <0>;
-
- panel_in_edp: endpoint@0 {
- reg = <0>;
- remote-endpoint = <&edp_out_panel>;
- };
+ port {
+ panel_in_edp: endpoint {
+ remote-endpoint = <&edp_out_panel>;
};
};
};
@@ -943,7 +933,7 @@ &sdmmc {
disable-wp;
pinctrl-names = "default";
pinctrl-0 = <&sdmmc_clk &sdmmc_cmd &sdmmc_bus4>;
- sd-uhs-sdr104;
+ sd-uhs-sdr50;
vmmc-supply = <&vcc3v0_sd>;
vqmmc-supply = <&vcc_sdio>;
status = "okay";
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dtsi
index 7815752..bca2b50 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dtsi
@@ -647,16 +647,10 @@ mipi_panel: panel@0 {
avdd-supply = <&avdd>;
backlight = <&backlight>;
dvdd-supply = <&vcc3v3_s0>;
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
- port@0 {
- reg = <0>;
-
- mipi_in_panel: endpoint {
- remote-endpoint = <&mipi_out_panel>;
- };
+ port {
+ mipi_in_panel: endpoint {
+ remote-endpoint = <&mipi_out_panel>;
};
};
};
diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
index 1881b4b..40e7c4a 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
@@ -552,7 +552,7 @@ gic: interrupt-controller@fee00000 {
<0x0 0xfff10000 0 0x10000>, /* GICH */
<0x0 0xfff20000 0 0x10000>; /* GICV */
interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH 0>;
- its: interrupt-controller@fee20000 {
+ its: msi-controller@fee20000 {
compatible = "arm,gic-v3-its";
msi-controller;
#msi-cells = <1>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg353x.dtsi b/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg353x.dtsi
index 65a80d1..9a0e217 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg353x.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg353x.dtsi
@@ -16,8 +16,10 @@ backlight: backlight {
};
&cru {
- assigned-clocks = <&cru PLL_GPLL>, <&pmucru PLL_PPLL>, <&cru PLL_VPLL>;
- assigned-clock-rates = <1200000000>, <200000000>, <241500000>;
+ assigned-clocks = <&pmucru CLK_RTC_32K>, <&cru PLL_GPLL>,
+ <&pmucru PLL_PPLL>, <&cru PLL_VPLL>;
+ assigned-clock-rates = <32768>, <1200000000>,
+ <200000000>, <241500000>;
};
&gpio_keys_control {
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg503.dts b/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg503.dts
index b4b2df8..c763c7f3 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg503.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3566-anbernic-rg503.dts
@@ -105,8 +105,10 @@ spk_amp: audio-amplifier {
};
&cru {
- assigned-clocks = <&cru PLL_GPLL>, <&pmucru PLL_PPLL>, <&cru PLL_VPLL>;
- assigned-clock-rates = <1200000000>, <200000000>, <500000000>;
+ assigned-clocks = <&pmucru CLK_RTC_32K>, <&cru PLL_GPLL>,
+ <&pmucru PLL_PPLL>, <&cru PLL_VPLL>;
+ assigned-clock-rates = <32768>, <1200000000>,
+ <200000000>, <500000000>;
};
&dsi_dphy0 {
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-soquartz.dtsi b/arch/arm64/boot/dts/rockchip/rk3566-soquartz.dtsi
index ce7165d..102e448 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-soquartz.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3566-soquartz.dtsi
@@ -598,7 +598,7 @@ &sdmmc1 {
non-removable;
pinctrl-names = "default";
pinctrl-0 = <&sdmmc1_bus4 &sdmmc1_cmd &sdmmc1_clk>;
- sd-uhs-sdr104;
+ sd-uhs-sdr50;
vmmc-supply = <&vcc3v3_sys>;
vqmmc-supply = <&vcc_1v8>;
status = "okay";
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index 005cde6..a506948 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -222,6 +222,7 @@ l2_cache_l0: l2-cache-l0 {
cache-size = <131072>;
cache-line-size = <64>;
cache-sets = <512>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -230,6 +231,7 @@ l2_cache_l1: l2-cache-l1 {
cache-size = <131072>;
cache-line-size = <64>;
cache-sets = <512>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -238,6 +240,7 @@ l2_cache_l2: l2-cache-l2 {
cache-size = <131072>;
cache-line-size = <64>;
cache-sets = <512>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -246,6 +249,7 @@ l2_cache_l3: l2-cache-l3 {
cache-size = <131072>;
cache-line-size = <64>;
cache-sets = <512>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -254,6 +258,7 @@ l2_cache_b0: l2-cache-b0 {
cache-size = <524288>;
cache-line-size = <64>;
cache-sets = <1024>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -262,6 +267,7 @@ l2_cache_b1: l2-cache-b1 {
cache-size = <524288>;
cache-line-size = <64>;
cache-sets = <1024>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -270,6 +276,7 @@ l2_cache_b2: l2-cache-b2 {
cache-size = <524288>;
cache-line-size = <64>;
cache-sets = <1024>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -278,6 +285,7 @@ l2_cache_b3: l2-cache-b3 {
cache-size = <524288>;
cache-line-size = <64>;
cache-sets = <1024>;
+ cache-level = <2>;
next-level-cache = <&l3_cache>;
};
@@ -286,6 +294,7 @@ l3_cache: l3-cache {
cache-size = <3145728>;
cache-line-size = <64>;
cache-sets = <4096>;
+ cache-level = <3>;
};
};
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 7fd5125..3ddde33 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -447,6 +447,22 @@
protection support. However, you can enable LoongArch DMW-based
ioremap() for better performance.
+config ARCH_WRITECOMBINE
+ bool "Enable WriteCombine (WUC) for ioremap()"
+ help
+ LoongArch maintains cache coherency in hardware, but when paired
+ with LS7A chipsets the WUC attribute (Weak-ordered UnCached, which
+ is similar to WriteCombine) is out of the scope of cache coherency
+ machanism for PCIe devices (this is a PCIe protocol violation, which
+ may be fixed in newer chipsets).
+
+ This means WUC can only used for write-only memory regions now, so
+ this option is disabled by default, making WUC silently fallback to
+ SUC for ioremap(). You can enable this option if the kernel is ensured
+ to run on hardware without this bug.
+
+ You can override this setting via writecombine=on/off boot parameter.
+
config ARCH_STRICT_ALIGN
bool "Enable -mstrict-align to prevent unaligned accesses" if EXPERT
default y
diff --git a/arch/loongarch/include/asm/acpi.h b/arch/loongarch/include/asm/acpi.h
index 4198753..976a810 100644
--- a/arch/loongarch/include/asm/acpi.h
+++ b/arch/loongarch/include/asm/acpi.h
@@ -41,8 +41,11 @@ extern void loongarch_suspend_enter(void);
static inline unsigned long acpi_get_wakeup_address(void)
{
+#ifdef CONFIG_SUSPEND
extern void loongarch_wakeup_start(void);
return (unsigned long)loongarch_wakeup_start;
+#endif
+ return 0UL;
}
#endif /* _ASM_LOONGARCH_ACPI_H */
diff --git a/arch/loongarch/include/asm/addrspace.h b/arch/loongarch/include/asm/addrspace.h
index 8fb699b..5c9c03b 100644
--- a/arch/loongarch/include/asm/addrspace.h
+++ b/arch/loongarch/include/asm/addrspace.h
@@ -71,9 +71,9 @@ extern unsigned long vm_map_base;
#define _ATYPE32_ int
#define _ATYPE64_ __s64
#ifdef CONFIG_64BIT
-#define _CONST64_(x) x ## L
+#define _CONST64_(x) x ## UL
#else
-#define _CONST64_(x) x ## LL
+#define _CONST64_(x) x ## ULL
#endif
#endif
diff --git a/arch/loongarch/include/asm/bootinfo.h b/arch/loongarch/include/asm/bootinfo.h
index 0051b52..c607968 100644
--- a/arch/loongarch/include/asm/bootinfo.h
+++ b/arch/loongarch/include/asm/bootinfo.h
@@ -13,7 +13,6 @@ const char *get_system_type(void);
extern void init_environ(void);
extern void memblock_init(void);
extern void platform_init(void);
-extern void plat_swiotlb_setup(void);
extern int __init init_numa_memory(void);
struct loongson_board_info {
diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h
index b079742..f6177f1 100644
--- a/arch/loongarch/include/asm/cpu-features.h
+++ b/arch/loongarch/include/asm/cpu-features.h
@@ -42,6 +42,7 @@
#define cpu_has_fpu cpu_opt(LOONGARCH_CPU_FPU)
#define cpu_has_lsx cpu_opt(LOONGARCH_CPU_LSX)
#define cpu_has_lasx cpu_opt(LOONGARCH_CPU_LASX)
+#define cpu_has_crc32 cpu_opt(LOONGARCH_CPU_CRC32)
#define cpu_has_complex cpu_opt(LOONGARCH_CPU_COMPLEX)
#define cpu_has_crypto cpu_opt(LOONGARCH_CPU_CRYPTO)
#define cpu_has_lvz cpu_opt(LOONGARCH_CPU_LVZ)
diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h
index c3da917..88773d8 100644
--- a/arch/loongarch/include/asm/cpu.h
+++ b/arch/loongarch/include/asm/cpu.h
@@ -78,25 +78,26 @@ enum cpu_type_enum {
#define CPU_FEATURE_FPU 3 /* CPU has FPU */
#define CPU_FEATURE_LSX 4 /* CPU has LSX (128-bit SIMD) */
#define CPU_FEATURE_LASX 5 /* CPU has LASX (256-bit SIMD) */
-#define CPU_FEATURE_COMPLEX 6 /* CPU has Complex instructions */
-#define CPU_FEATURE_CRYPTO 7 /* CPU has Crypto instructions */
-#define CPU_FEATURE_LVZ 8 /* CPU has Virtualization extension */
-#define CPU_FEATURE_LBT_X86 9 /* CPU has X86 Binary Translation */
-#define CPU_FEATURE_LBT_ARM 10 /* CPU has ARM Binary Translation */
-#define CPU_FEATURE_LBT_MIPS 11 /* CPU has MIPS Binary Translation */
-#define CPU_FEATURE_TLB 12 /* CPU has TLB */
-#define CPU_FEATURE_CSR 13 /* CPU has CSR */
-#define CPU_FEATURE_WATCH 14 /* CPU has watchpoint registers */
-#define CPU_FEATURE_VINT 15 /* CPU has vectored interrupts */
-#define CPU_FEATURE_CSRIPI 16 /* CPU has CSR-IPI */
-#define CPU_FEATURE_EXTIOI 17 /* CPU has EXT-IOI */
-#define CPU_FEATURE_PREFETCH 18 /* CPU has prefetch instructions */
-#define CPU_FEATURE_PMP 19 /* CPU has perfermance counter */
-#define CPU_FEATURE_SCALEFREQ 20 /* CPU supports cpufreq scaling */
-#define CPU_FEATURE_FLATMODE 21 /* CPU has flat mode */
-#define CPU_FEATURE_EIODECODE 22 /* CPU has EXTIOI interrupt pin decode mode */
-#define CPU_FEATURE_GUESTID 23 /* CPU has GuestID feature */
-#define CPU_FEATURE_HYPERVISOR 24 /* CPU has hypervisor (running in VM) */
+#define CPU_FEATURE_CRC32 6 /* CPU has CRC32 instructions */
+#define CPU_FEATURE_COMPLEX 7 /* CPU has Complex instructions */
+#define CPU_FEATURE_CRYPTO 8 /* CPU has Crypto instructions */
+#define CPU_FEATURE_LVZ 9 /* CPU has Virtualization extension */
+#define CPU_FEATURE_LBT_X86 10 /* CPU has X86 Binary Translation */
+#define CPU_FEATURE_LBT_ARM 11 /* CPU has ARM Binary Translation */
+#define CPU_FEATURE_LBT_MIPS 12 /* CPU has MIPS Binary Translation */
+#define CPU_FEATURE_TLB 13 /* CPU has TLB */
+#define CPU_FEATURE_CSR 14 /* CPU has CSR */
+#define CPU_FEATURE_WATCH 15 /* CPU has watchpoint registers */
+#define CPU_FEATURE_VINT 16 /* CPU has vectored interrupts */
+#define CPU_FEATURE_CSRIPI 17 /* CPU has CSR-IPI */
+#define CPU_FEATURE_EXTIOI 18 /* CPU has EXT-IOI */
+#define CPU_FEATURE_PREFETCH 19 /* CPU has prefetch instructions */
+#define CPU_FEATURE_PMP 20 /* CPU has perfermance counter */
+#define CPU_FEATURE_SCALEFREQ 21 /* CPU supports cpufreq scaling */
+#define CPU_FEATURE_FLATMODE 22 /* CPU has flat mode */
+#define CPU_FEATURE_EIODECODE 23 /* CPU has EXTIOI interrupt pin decode mode */
+#define CPU_FEATURE_GUESTID 24 /* CPU has GuestID feature */
+#define CPU_FEATURE_HYPERVISOR 25 /* CPU has hypervisor (running in VM) */
#define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG)
#define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM)
@@ -104,6 +105,7 @@ enum cpu_type_enum {
#define LOONGARCH_CPU_FPU BIT_ULL(CPU_FEATURE_FPU)
#define LOONGARCH_CPU_LSX BIT_ULL(CPU_FEATURE_LSX)
#define LOONGARCH_CPU_LASX BIT_ULL(CPU_FEATURE_LASX)
+#define LOONGARCH_CPU_CRC32 BIT_ULL(CPU_FEATURE_CRC32)
#define LOONGARCH_CPU_COMPLEX BIT_ULL(CPU_FEATURE_COMPLEX)
#define LOONGARCH_CPU_CRYPTO BIT_ULL(CPU_FEATURE_CRYPTO)
#define LOONGARCH_CPU_LVZ BIT_ULL(CPU_FEATURE_LVZ)
diff --git a/arch/loongarch/include/asm/io.h b/arch/loongarch/include/asm/io.h
index 402a7d9..545e2708f 100644
--- a/arch/loongarch/include/asm/io.h
+++ b/arch/loongarch/include/asm/io.h
@@ -54,8 +54,10 @@ static inline void __iomem *ioremap_prot(phys_addr_t offset, unsigned long size,
* @offset: bus address of the memory
* @size: size of the resource to map
*/
+extern pgprot_t pgprot_wc;
+
#define ioremap_wc(offset, size) \
- ioremap_prot((offset), (size), pgprot_val(PAGE_KERNEL_WUC))
+ ioremap_prot((offset), (size), pgprot_val(pgprot_wc))
#define ioremap_cache(offset, size) \
ioremap_prot((offset), (size), pgprot_val(PAGE_KERNEL))
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 65b7dcd..83da5d2 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -117,7 +117,7 @@ static inline u32 read_cpucfg(u32 reg)
#define CPUCFG1_EP BIT(22)
#define CPUCFG1_RPLV BIT(23)
#define CPUCFG1_HUGEPG BIT(24)
-#define CPUCFG1_IOCSRBRD BIT(25)
+#define CPUCFG1_CRC32 BIT(25)
#define CPUCFG1_MSGINT BIT(26)
#define LOONGARCH_CPUCFG2 0x2
@@ -423,9 +423,9 @@ static __always_inline void iocsr_write64(u64 val, u32 reg)
#define CSR_ASID_ASID_WIDTH 10
#define CSR_ASID_ASID (_ULCAST_(0x3ff) << CSR_ASID_ASID_SHIFT)
-#define LOONGARCH_CSR_PGDL 0x19 /* Page table base address when VA[47] = 0 */
+#define LOONGARCH_CSR_PGDL 0x19 /* Page table base address when VA[VALEN-1] = 0 */
-#define LOONGARCH_CSR_PGDH 0x1a /* Page table base address when VA[47] = 1 */
+#define LOONGARCH_CSR_PGDH 0x1a /* Page table base address when VA[VALEN-1] = 1 */
#define LOONGARCH_CSR_PGD 0x1b /* Page table base */
diff --git a/arch/loongarch/include/asm/module.lds.h b/arch/loongarch/include/asm/module.lds.h
index 438f09d..88554f9 100644
--- a/arch/loongarch/include/asm/module.lds.h
+++ b/arch/loongarch/include/asm/module.lds.h
@@ -2,8 +2,8 @@
/* Copyright (C) 2020-2022 Loongson Technology Corporation Limited */
SECTIONS {
. = ALIGN(4);
- .got : { BYTE(0) }
- .plt : { BYTE(0) }
- .plt.idx : { BYTE(0) }
- .ftrace_trampoline : { BYTE(0) }
+ .got 0 : { BYTE(0) }
+ .plt 0 : { BYTE(0) }
+ .plt.idx 0 : { BYTE(0) }
+ .ftrace_trampoline 0 : { BYTE(0) }
}
diff --git a/arch/loongarch/include/uapi/asm/ptrace.h b/arch/loongarch/include/uapi/asm/ptrace.h
index cc48ed2..82d811b 100644
--- a/arch/loongarch/include/uapi/asm/ptrace.h
+++ b/arch/loongarch/include/uapi/asm/ptrace.h
@@ -47,11 +47,12 @@ struct user_fp_state {
};
struct user_watch_state {
- uint16_t dbg_info;
+ uint64_t dbg_info;
struct {
uint64_t addr;
uint64_t mask;
uint32_t ctrl;
+ uint32_t pad;
} dbg_regs[8];
};
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
index 3a3fce2..5adf0f7 100644
--- a/arch/loongarch/kernel/cpu-probe.c
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -60,7 +60,7 @@ static inline void set_elf_platform(int cpu, const char *plat)
/* MAP BASE */
unsigned long vm_map_base;
-EXPORT_SYMBOL_GPL(vm_map_base);
+EXPORT_SYMBOL(vm_map_base);
static void cpu_probe_addrbits(struct cpuinfo_loongarch *c)
{
@@ -94,13 +94,18 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c)
c->options = LOONGARCH_CPU_CPUCFG | LOONGARCH_CPU_CSR |
LOONGARCH_CPU_TLB | LOONGARCH_CPU_VINT | LOONGARCH_CPU_WATCH;
- elf_hwcap = HWCAP_LOONGARCH_CPUCFG | HWCAP_LOONGARCH_CRC32;
+ elf_hwcap = HWCAP_LOONGARCH_CPUCFG;
config = read_cpucfg(LOONGARCH_CPUCFG1);
if (config & CPUCFG1_UAL) {
c->options |= LOONGARCH_CPU_UAL;
elf_hwcap |= HWCAP_LOONGARCH_UAL;
}
+ if (config & CPUCFG1_CRC32) {
+ c->options |= LOONGARCH_CPU_CRC32;
+ elf_hwcap |= HWCAP_LOONGARCH_CRC32;
+ }
+
config = read_cpucfg(LOONGARCH_CPUCFG2);
if (config & CPUCFG2_LAM) {
diff --git a/arch/loongarch/kernel/proc.c b/arch/loongarch/kernel/proc.c
index 5c67cc4..0d82907 100644
--- a/arch/loongarch/kernel/proc.c
+++ b/arch/loongarch/kernel/proc.c
@@ -76,6 +76,7 @@ static int show_cpuinfo(struct seq_file *m, void *v)
if (cpu_has_fpu) seq_printf(m, " fpu");
if (cpu_has_lsx) seq_printf(m, " lsx");
if (cpu_has_lasx) seq_printf(m, " lasx");
+ if (cpu_has_crc32) seq_printf(m, " crc32");
if (cpu_has_complex) seq_printf(m, " complex");
if (cpu_has_crypto) seq_printf(m, " crypto");
if (cpu_has_lvz) seq_printf(m, " lvz");
diff --git a/arch/loongarch/kernel/ptrace.c b/arch/loongarch/kernel/ptrace.c
index 06bceae..5fcffb4 100644
--- a/arch/loongarch/kernel/ptrace.c
+++ b/arch/loongarch/kernel/ptrace.c
@@ -391,10 +391,10 @@ static int ptrace_hbp_fill_attr_ctrl(unsigned int note_type,
return 0;
}
-static int ptrace_hbp_get_resource_info(unsigned int note_type, u16 *info)
+static int ptrace_hbp_get_resource_info(unsigned int note_type, u64 *info)
{
u8 num;
- u16 reg = 0;
+ u64 reg = 0;
switch (note_type) {
case NT_LOONGARCH_HW_BREAK:
@@ -524,15 +524,16 @@ static int ptrace_hbp_set_addr(unsigned int note_type,
return modify_user_hw_breakpoint(bp, &attr);
}
-#define PTRACE_HBP_CTRL_SZ sizeof(u32)
#define PTRACE_HBP_ADDR_SZ sizeof(u64)
#define PTRACE_HBP_MASK_SZ sizeof(u64)
+#define PTRACE_HBP_CTRL_SZ sizeof(u32)
+#define PTRACE_HBP_PAD_SZ sizeof(u32)
static int hw_break_get(struct task_struct *target,
const struct user_regset *regset,
struct membuf to)
{
- u16 info;
+ u64 info;
u32 ctrl;
u64 addr, mask;
int ret, idx = 0;
@@ -545,7 +546,7 @@ static int hw_break_get(struct task_struct *target,
membuf_write(&to, &info, sizeof(info));
- /* (address, ctrl) registers */
+ /* (address, mask, ctrl) registers */
while (to.left) {
ret = ptrace_hbp_get_addr(note_type, target, idx, &addr);
if (ret)
@@ -562,6 +563,7 @@ static int hw_break_get(struct task_struct *target,
membuf_store(&to, addr);
membuf_store(&to, mask);
membuf_store(&to, ctrl);
+ membuf_zero(&to, sizeof(u32));
idx++;
}
@@ -582,7 +584,7 @@ static int hw_break_set(struct task_struct *target,
offset = offsetof(struct user_watch_state, dbg_regs);
user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf, 0, offset);
- /* (address, ctrl) registers */
+ /* (address, mask, ctrl) registers */
limit = regset->n * regset->size;
while (count && offset < limit) {
if (count < PTRACE_HBP_ADDR_SZ)
@@ -602,7 +604,7 @@ static int hw_break_set(struct task_struct *target,
break;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &mask,
- offset, offset + PTRACE_HBP_ADDR_SZ);
+ offset, offset + PTRACE_HBP_MASK_SZ);
if (ret)
return ret;
@@ -611,8 +613,8 @@ static int hw_break_set(struct task_struct *target,
return ret;
offset += PTRACE_HBP_MASK_SZ;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &mask,
- offset, offset + PTRACE_HBP_MASK_SZ);
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &ctrl,
+ offset, offset + PTRACE_HBP_CTRL_SZ);
if (ret)
return ret;
@@ -620,6 +622,11 @@ static int hw_break_set(struct task_struct *target,
if (ret)
return ret;
offset += PTRACE_HBP_CTRL_SZ;
+
+ user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ offset, offset + PTRACE_HBP_PAD_SZ);
+ offset += PTRACE_HBP_PAD_SZ;
+
idx++;
}
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
index bae84cc..4444b13 100644
--- a/arch/loongarch/kernel/setup.c
+++ b/arch/loongarch/kernel/setup.c
@@ -160,6 +160,27 @@ static void __init smbios_parse(void)
dmi_walk(find_tokens, NULL);
}
+#ifdef CONFIG_ARCH_WRITECOMBINE
+pgprot_t pgprot_wc = PAGE_KERNEL_WUC;
+#else
+pgprot_t pgprot_wc = PAGE_KERNEL_SUC;
+#endif
+
+EXPORT_SYMBOL(pgprot_wc);
+
+static int __init setup_writecombine(char *p)
+{
+ if (!strcmp(p, "on"))
+ pgprot_wc = PAGE_KERNEL_WUC;
+ else if (!strcmp(p, "off"))
+ pgprot_wc = PAGE_KERNEL_SUC;
+ else
+ pr_warn("Unknown writecombine setting \"%s\".\n", p);
+
+ return 0;
+}
+early_param("writecombine", setup_writecombine);
+
static int usermem __initdata;
static int __init early_parse_mem(char *p)
@@ -368,8 +389,8 @@ static void __init arch_mem_init(char **cmdline_p)
/*
* In order to reduce the possibility of kernel panic when failed to
* get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
- * low memory as small as possible before plat_swiotlb_setup(), so
- * make sparse_init() using top-down allocation.
+ * low memory as small as possible before swiotlb_init(), so make
+ * sparse_init() using top-down allocation.
*/
memblock_set_bottom_up(false);
sparse_init();
diff --git a/arch/loongarch/kernel/stacktrace.c b/arch/loongarch/kernel/stacktrace.c
index 3a690f9..2463d2f 100644
--- a/arch/loongarch/kernel/stacktrace.c
+++ b/arch/loongarch/kernel/stacktrace.c
@@ -30,7 +30,7 @@ void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie,
regs->regs[1] = 0;
for (unwind_start(&state, task, regs);
- !unwind_done(&state); unwind_next_frame(&state)) {
+ !unwind_done(&state) && !unwind_error(&state); unwind_next_frame(&state)) {
addr = unwind_get_return_address(&state);
if (!addr || !consume_entry(cookie, addr))
break;
diff --git a/arch/loongarch/kernel/unwind.c b/arch/loongarch/kernel/unwind.c
index a463d69..ba324ba 100644
--- a/arch/loongarch/kernel/unwind.c
+++ b/arch/loongarch/kernel/unwind.c
@@ -28,5 +28,6 @@ bool default_next_frame(struct unwind_state *state)
} while (!get_stack_info(state->sp, state->task, info));
+ state->error = true;
return false;
}
diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c
index 9095fde..55afc27 100644
--- a/arch/loongarch/kernel/unwind_prologue.c
+++ b/arch/loongarch/kernel/unwind_prologue.c
@@ -211,7 +211,7 @@ static bool next_frame(struct unwind_state *state)
pc = regs->csr_era;
if (user_mode(regs) || !__kernel_text_address(pc))
- return false;
+ goto out;
state->first = true;
state->pc = pc;
@@ -226,6 +226,8 @@ static bool next_frame(struct unwind_state *state)
} while (!get_stack_info(state->sp, state->task, info));
+out:
+ state->error = true;
return false;
}
diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c
index e018aed..3b7d812 100644
--- a/arch/loongarch/mm/init.c
+++ b/arch/loongarch/mm/init.c
@@ -41,7 +41,7 @@
* don't have to care about aliases on other CPUs.
*/
unsigned long empty_zero_page, zero_page_mask;
-EXPORT_SYMBOL_GPL(empty_zero_page);
+EXPORT_SYMBOL(empty_zero_page);
EXPORT_SYMBOL(zero_page_mask);
void setup_zero_pages(void)
@@ -270,7 +270,7 @@ pud_t invalid_pud_table[PTRS_PER_PUD] __page_aligned_bss;
#endif
#ifndef __PAGETABLE_PMD_FOLDED
pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
-EXPORT_SYMBOL_GPL(invalid_pmd_table);
+EXPORT_SYMBOL(invalid_pmd_table);
#endif
pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss;
EXPORT_SYMBOL(invalid_pte_table);
diff --git a/arch/loongarch/power/suspend_asm.S b/arch/loongarch/power/suspend_asm.S
index 90da899..e2fc3b4 100644
--- a/arch/loongarch/power/suspend_asm.S
+++ b/arch/loongarch/power/suspend_asm.S
@@ -80,6 +80,10 @@
JUMP_VIRT_ADDR t0, t1
+ /* Enable PG */
+ li.w t0, 0xb0 # PLV=0, IE=0, PG=1
+ csrwr t0, LOONGARCH_CSR_CRMD
+
la.pcrel t0, acpi_saved_sp
ld.d sp, t0, 0
SETUP_WAKEUP
diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index b44ce71..16cfe56 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -366,6 +366,7 @@ void update_numa_distance(struct device_node *node)
WARN(numa_distance_table[nid][nid] == -1,
"NUMA distance details for node %d not provided\n", nid);
}
+EXPORT_SYMBOL_GPL(update_numa_distance);
/*
* ibm,numa-lookup-index-table= {N, domainid1, domainid2, ..... domainidN}
diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index 2f83855..1a53e04 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -1428,6 +1428,13 @@ static int papr_scm_probe(struct platform_device *pdev)
return -ENODEV;
}
+ /*
+ * open firmware platform device create won't update the NUMA
+ * distance table. For PAPR SCM devices we use numa_map_to_online_node()
+ * to find the nearest online NUMA node and that requires correct
+ * distance table information.
+ */
+ update_numa_distance(dn);
p = kzalloc(sizeof(*p), GFP_KERNEL);
if (!p)
diff --git a/arch/riscv/boot/dts/canaan/k210.dtsi b/arch/riscv/boot/dts/canaan/k210.dtsi
index 07e2e26..f87c516 100644
--- a/arch/riscv/boot/dts/canaan/k210.dtsi
+++ b/arch/riscv/boot/dts/canaan/k210.dtsi
@@ -259,7 +259,6 @@ spi2: spi@50240000 {
<&sysclk K210_CLK_APB0>;
clock-names = "ssi_clk", "pclk";
resets = <&sysrst K210_RST_SPI2>;
- spi-max-frequency = <25000000>;
};
i2s0: i2s@50250000 {
diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h
index 5c3e7b9..0a55099 100644
--- a/arch/riscv/include/asm/fixmap.h
+++ b/arch/riscv/include/asm/fixmap.h
@@ -22,6 +22,14 @@
*/
enum fixed_addresses {
FIX_HOLE,
+ /*
+ * The fdt fixmap mapping must be PMD aligned and will be mapped
+ * using PMD entries in fixmap_pmd in 64-bit and a PGD entry in 32-bit.
+ */
+ FIX_FDT_END,
+ FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
+
+ /* Below fixmaps will be mapped using fixmap_pte */
FIX_PTE,
FIX_PMD,
FIX_PUD,
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index ab05f89..f641837 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -87,9 +87,13 @@
#define FIXADDR_TOP PCI_IO_START
#ifdef CONFIG_64BIT
-#define FIXADDR_SIZE PMD_SIZE
+#define MAX_FDT_SIZE PMD_SIZE
+#define FIX_FDT_SIZE (MAX_FDT_SIZE + SZ_2M)
+#define FIXADDR_SIZE (PMD_SIZE + FIX_FDT_SIZE)
#else
-#define FIXADDR_SIZE PGDIR_SIZE
+#define MAX_FDT_SIZE PGDIR_SIZE
+#define FIX_FDT_SIZE MAX_FDT_SIZE
+#define FIXADDR_SIZE (PGDIR_SIZE + FIX_FDT_SIZE)
#endif
#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 376d282..a059b73 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -278,12 +278,8 @@ void __init setup_arch(char **cmdline_p)
#if IS_ENABLED(CONFIG_BUILTIN_DTB)
unflatten_and_copy_device_tree();
#else
- if (early_init_dt_verify(__va(XIP_FIXUP(dtb_early_pa))))
- unflatten_device_tree();
- else
- pr_err("No DTB found in kernel mappings\n");
+ unflatten_device_tree();
#endif
- early_init_fdt_scan_reserved_mem();
misc_mem_init();
init_resources();
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index bfb2afa..dee66c9 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -19,6 +19,7 @@
#include <asm/signal32.h>
#include <asm/switch_to.h>
#include <asm/csr.h>
+#include <asm/cacheflush.h>
extern u32 __user_rt_sigreturn[2];
@@ -181,6 +182,7 @@ static int setup_rt_frame(struct ksignal *ksig, sigset_t *set,
{
struct rt_sigframe __user *frame;
long err = 0;
+ unsigned long __maybe_unused addr;
frame = get_sigframe(ksig, regs, sizeof(*frame));
if (!access_ok(frame, sizeof(*frame)))
@@ -209,7 +211,12 @@ static int setup_rt_frame(struct ksignal *ksig, sigset_t *set,
if (copy_to_user(&frame->sigreturn_code, __user_rt_sigreturn,
sizeof(frame->sigreturn_code)))
return -EFAULT;
- regs->ra = (unsigned long)&frame->sigreturn_code;
+
+ addr = (unsigned long)&frame->sigreturn_code;
+ /* Make sure the two instructions are pushed to icache. */
+ flush_icache_range(addr, addr + sizeof(frame->sigreturn_code));
+
+ regs->ra = addr;
#endif /* CONFIG_MMU */
/*
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 478d676..0f14f4a 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -57,7 +57,6 @@ unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)]
EXPORT_SYMBOL(empty_zero_page);
extern char _start[];
-#define DTB_EARLY_BASE_VA PGDIR_SIZE
void *_dtb_early_va __initdata;
uintptr_t _dtb_early_pa __initdata;
@@ -236,31 +235,22 @@ static void __init setup_bootmem(void)
set_max_mapnr(max_low_pfn - ARCH_PFN_OFFSET);
reserve_initrd_mem();
+
+ /*
+ * No allocation should be done before reserving the memory as defined
+ * in the device tree, otherwise the allocation could end up in a
+ * reserved region.
+ */
+ early_init_fdt_scan_reserved_mem();
+
/*
* If DTB is built in, no need to reserve its memblock.
* Otherwise, do reserve it but avoid using
* early_init_fdt_reserve_self() since __pa() does
* not work for DTB pointers that are fixmap addresses
*/
- if (!IS_ENABLED(CONFIG_BUILTIN_DTB)) {
- /*
- * In case the DTB is not located in a memory region we won't
- * be able to locate it later on via the linear mapping and
- * get a segfault when accessing it via __va(dtb_early_pa).
- * To avoid this situation copy DTB to a memory region.
- * Note that memblock_phys_alloc will also reserve DTB region.
- */
- if (!memblock_is_memory(dtb_early_pa)) {
- size_t fdt_size = fdt_totalsize(dtb_early_va);
- phys_addr_t new_dtb_early_pa = memblock_phys_alloc(fdt_size, PAGE_SIZE);
- void *new_dtb_early_va = early_memremap(new_dtb_early_pa, fdt_size);
-
- memcpy(new_dtb_early_va, dtb_early_va, fdt_size);
- early_memunmap(new_dtb_early_va, fdt_size);
- _dtb_early_pa = new_dtb_early_pa;
- } else
- memblock_reserve(dtb_early_pa, fdt_totalsize(dtb_early_va));
- }
+ if (!IS_ENABLED(CONFIG_BUILTIN_DTB))
+ memblock_reserve(dtb_early_pa, fdt_totalsize(dtb_early_va));
dma_contiguous_reserve(dma32_phys_limit);
if (IS_ENABLED(CONFIG_64BIT))
@@ -279,9 +269,6 @@ pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss;
pgd_t early_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE);
-static p4d_t __maybe_unused early_dtb_p4d[PTRS_PER_P4D] __initdata __aligned(PAGE_SIZE);
-static pud_t __maybe_unused early_dtb_pud[PTRS_PER_PUD] __initdata __aligned(PAGE_SIZE);
-static pmd_t __maybe_unused early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAGE_SIZE);
#ifdef CONFIG_XIP_KERNEL
#define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops))
@@ -626,9 +613,6 @@ static void __init create_p4d_mapping(p4d_t *p4dp,
#define trampoline_pgd_next (pgtable_l5_enabled ? \
(uintptr_t)trampoline_p4d : (pgtable_l4_enabled ? \
(uintptr_t)trampoline_pud : (uintptr_t)trampoline_pmd))
-#define early_dtb_pgd_next (pgtable_l5_enabled ? \
- (uintptr_t)early_dtb_p4d : (pgtable_l4_enabled ? \
- (uintptr_t)early_dtb_pud : (uintptr_t)early_dtb_pmd))
#else
#define pgd_next_t pte_t
#define alloc_pgd_next(__va) pt_ops.alloc_pte(__va)
@@ -636,7 +620,6 @@ static void __init create_p4d_mapping(p4d_t *p4dp,
#define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \
create_pte_mapping(__nextp, __va, __pa, __sz, __prot)
#define fixmap_pgd_next ((uintptr_t)fixmap_pte)
-#define early_dtb_pgd_next ((uintptr_t)early_dtb_pmd)
#define create_p4d_mapping(__pmdp, __va, __pa, __sz, __prot) do {} while(0)
#define create_pud_mapping(__pmdp, __va, __pa, __sz, __prot) do {} while(0)
#define create_pmd_mapping(__pmdp, __va, __pa, __sz, __prot) do {} while(0)
@@ -860,32 +843,28 @@ static void __init create_kernel_page_table(pgd_t *pgdir, bool early)
* this means 2 PMD entries whereas for 32-bit kernel, this is only 1 PGDIR
* entry.
*/
-static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
+static void __init create_fdt_early_page_table(pgd_t *pgdir,
+ uintptr_t fix_fdt_va,
+ uintptr_t dtb_pa)
{
-#ifndef CONFIG_BUILTIN_DTB
uintptr_t pa = dtb_pa & ~(PMD_SIZE - 1);
- create_pgd_mapping(early_pg_dir, DTB_EARLY_BASE_VA,
- IS_ENABLED(CONFIG_64BIT) ? early_dtb_pgd_next : pa,
- PGDIR_SIZE,
- IS_ENABLED(CONFIG_64BIT) ? PAGE_TABLE : PAGE_KERNEL);
+#ifndef CONFIG_BUILTIN_DTB
+ /* Make sure the fdt fixmap address is always aligned on PMD size */
+ BUILD_BUG_ON(FIX_FDT % (PMD_SIZE / PAGE_SIZE));
- if (pgtable_l5_enabled)
- create_p4d_mapping(early_dtb_p4d, DTB_EARLY_BASE_VA,
- (uintptr_t)early_dtb_pud, P4D_SIZE, PAGE_TABLE);
-
- if (pgtable_l4_enabled)
- create_pud_mapping(early_dtb_pud, DTB_EARLY_BASE_VA,
- (uintptr_t)early_dtb_pmd, PUD_SIZE, PAGE_TABLE);
-
- if (IS_ENABLED(CONFIG_64BIT)) {
- create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA,
+ /* In 32-bit only, the fdt lies in its own PGD */
+ if (!IS_ENABLED(CONFIG_64BIT)) {
+ create_pgd_mapping(early_pg_dir, fix_fdt_va,
+ pa, MAX_FDT_SIZE, PAGE_KERNEL);
+ } else {
+ create_pmd_mapping(fixmap_pmd, fix_fdt_va,
pa, PMD_SIZE, PAGE_KERNEL);
- create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA + PMD_SIZE,
+ create_pmd_mapping(fixmap_pmd, fix_fdt_va + PMD_SIZE,
pa + PMD_SIZE, PMD_SIZE, PAGE_KERNEL);
}
- dtb_early_va = (void *)DTB_EARLY_BASE_VA + (dtb_pa & (PMD_SIZE - 1));
+ dtb_early_va = (void *)fix_fdt_va + (dtb_pa & (PMD_SIZE - 1));
#else
/*
* For 64-bit kernel, __va can't be used since it would return a linear
@@ -1055,7 +1034,8 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
create_kernel_page_table(early_pg_dir, true);
/* Setup early mapping for FDT early scan */
- create_fdt_early_page_table(early_pg_dir, dtb_pa);
+ create_fdt_early_page_table(early_pg_dir,
+ __fix_to_virt(FIX_FDT), dtb_pa);
/*
* Bootime fixmap only can handle PMD_SIZE mapping. Thus, boot-ioremap
@@ -1097,6 +1077,16 @@ static void __init setup_vm_final(void)
u64 i;
/* Setup swapper PGD for fixmap */
+#if !defined(CONFIG_64BIT)
+ /*
+ * In 32-bit, the device tree lies in a pgd entry, so it must be copied
+ * directly in swapper_pg_dir in addition to the pgd entry that points
+ * to fixmap_pte.
+ */
+ unsigned long idx = pgd_index(__fix_to_virt(FIX_FDT));
+
+ set_pgd(&swapper_pg_dir[idx], early_pg_dir[idx]);
+#endif
create_pgd_mapping(swapper_pg_dir, FIXADDR_START,
__pa_symbol(fixmap_pgd_next),
PGDIR_SIZE, PAGE_TABLE);
diff --git a/arch/riscv/purgatory/Makefile b/arch/riscv/purgatory/Makefile
index d16bf71..5730797 100644
--- a/arch/riscv/purgatory/Makefile
+++ b/arch/riscv/purgatory/Makefile
@@ -84,12 +84,7 @@
CFLAGS_REMOVE_ctype.o += $(PURGATORY_CFLAGS_REMOVE)
CFLAGS_ctype.o += $(PURGATORY_CFLAGS)
-AFLAGS_REMOVE_entry.o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_memcpy.o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_memset.o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_strcmp.o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_strlen.o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_strncmp.o += -Wa,-gdwarf-2
+asflags-remove-y += $(foreach x, -g -gdwarf-4 -gdwarf-5, $(x) -Wa,$(x))
$(obj)/purgatory.ro: $(PURGATORY_OBJS) FORCE
$(call if_changed,ld)
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 7102e4b..f95d7e4 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -539,7 +539,7 @@ static void bpf_jit_plt(void *plt, void *ret, void *target)
{
memcpy(plt, bpf_plt, BPF_PLT_SIZE);
*(void **)((char *)plt + (bpf_plt_ret - bpf_plt)) = ret;
- *(void **)((char *)plt + (bpf_plt_target - bpf_plt)) = target;
+ *(void **)((char *)plt + (bpf_plt_target - bpf_plt)) = target ?: ret;
}
/*
@@ -2015,7 +2015,9 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t,
} __packed insn;
char expected_plt[BPF_PLT_SIZE];
char current_plt[BPF_PLT_SIZE];
+ char new_plt[BPF_PLT_SIZE];
char *plt;
+ char *ret;
int err;
/* Verify the branch to be patched. */
@@ -2037,12 +2039,15 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t,
err = copy_from_kernel_nofault(current_plt, plt, BPF_PLT_SIZE);
if (err < 0)
return err;
- bpf_jit_plt(expected_plt, (char *)ip + 6, old_addr);
+ ret = (char *)ip + 6;
+ bpf_jit_plt(expected_plt, ret, old_addr);
if (memcmp(current_plt, expected_plt, BPF_PLT_SIZE))
return -EINVAL;
/* Adjust the call address. */
+ bpf_jit_plt(new_plt, ret, new_addr);
s390_kernel_write(plt + (bpf_plt_target - bpf_plt),
- &new_addr, sizeof(void *));
+ new_plt + (bpf_plt_target - bpf_plt),
+ sizeof(void *));
}
/* Adjust the mask of the branch. */
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index ef80d36..10622cf 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -33,8 +33,8 @@ static int __init iommu_init_noop(void) { return 0; }
static void iommu_shutdown_noop(void) { }
bool __init bool_x86_init_noop(void) { return false; }
void x86_op_int_noop(int cpu) { }
-static __init int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
-static __init void get_rtc_noop(struct timespec64 *now) { }
+static int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
+static void get_rtc_noop(struct timespec64 *now) { }
static __initconst const struct of_device_id of_cmos_match[] = {
{ .compatible = "motorola,mc146818" },
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
index 17f09dc..82fec66 100644
--- a/arch/x86/purgatory/Makefile
+++ b/arch/x86/purgatory/Makefile
@@ -69,8 +69,7 @@
CFLAGS_REMOVE_string.o += $(PURGATORY_CFLAGS_REMOVE)
CFLAGS_string.o += $(PURGATORY_CFLAGS)
-AFLAGS_REMOVE_setup-x86_$(BITS).o += -Wa,-gdwarf-2
-AFLAGS_REMOVE_entry64.o += -Wa,-gdwarf-2
+asflags-remove-y += $(foreach x, -g -gdwarf-4 -gdwarf-5, $(x) -Wa,$(x))
$(obj)/purgatory.ro: $(PURGATORY_OBJS) FORCE
$(call if_changed,ld)
diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
index 7b4801c..e8492b3 100644
--- a/drivers/acpi/resource.c
+++ b/drivers/acpi/resource.c
@@ -440,6 +440,13 @@ static const struct dmi_system_id asus_laptop[] = {
},
},
{
+ .ident = "Asus ExpertBook B1502CBA",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_BOARD_NAME, "B1502CBA"),
+ },
+ },
+ {
.ident = "Asus ExpertBook B2402CBA",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
diff --git a/drivers/acpi/x86/utils.c b/drivers/acpi/x86/utils.c
index da57270..ba420a2 100644
--- a/drivers/acpi/x86/utils.c
+++ b/drivers/acpi/x86/utils.c
@@ -213,6 +213,7 @@ bool acpi_device_override_status(struct acpi_device *adev, unsigned long long *s
disk in the system.
*/
static const struct x86_cpu_id storage_d3_cpu_ids[] = {
+ X86_MATCH_VENDOR_FAM_MODEL(AMD, 23, 24, NULL), /* Picasso */
X86_MATCH_VENDOR_FAM_MODEL(AMD, 23, 96, NULL), /* Renoir */
X86_MATCH_VENDOR_FAM_MODEL(AMD, 23, 104, NULL), /* Lucienne */
X86_MATCH_VENDOR_FAM_MODEL(AMD, 25, 80, NULL), /* Cezanne */
diff --git a/drivers/bcma/main.c b/drivers/bcma/main.c
index 5e438f7..7061d3e 100644
--- a/drivers/bcma/main.c
+++ b/drivers/bcma/main.c
@@ -14,6 +14,7 @@
#include <linux/slab.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
+#include <linux/of_device.h>
#include <linux/of_platform.h>
MODULE_DESCRIPTION("Broadcom's specific AMBA driver");
diff --git a/drivers/clk/clk-renesas-pcie.c b/drivers/clk/clk-renesas-pcie.c
index f91f305..ff3a52d 100644
--- a/drivers/clk/clk-renesas-pcie.c
+++ b/drivers/clk/clk-renesas-pcie.c
@@ -143,8 +143,9 @@ static int rs9_regmap_i2c_read(void *context,
static const struct regmap_config rs9_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
- .cache_type = REGCACHE_NONE,
+ .cache_type = REGCACHE_FLAT,
.max_register = RS9_REG_BCP,
+ .num_reg_defaults_raw = 0x8,
.rd_table = &rs9_readable_table,
.wr_table = &rs9_writeable_table,
.reg_write = rs9_regmap_i2c_write,
diff --git a/drivers/clk/imx/clk-imx6ul.c b/drivers/clk/imx/clk-imx6ul.c
index 2836adb..e3696a8 100644
--- a/drivers/clk/imx/clk-imx6ul.c
+++ b/drivers/clk/imx/clk-imx6ul.c
@@ -95,14 +95,16 @@ static const struct clk_div_table video_div_table[] = {
{ }
};
-static const char * enet1_ref_sels[] = { "enet1_ref_125m", "enet1_ref_pad", };
+static const char * enet1_ref_sels[] = { "enet1_ref_125m", "enet1_ref_pad", "dummy", "dummy"};
static const u32 enet1_ref_sels_table[] = { IMX6UL_GPR1_ENET1_TX_CLK_DIR,
- IMX6UL_GPR1_ENET1_CLK_SEL };
+ IMX6UL_GPR1_ENET1_CLK_SEL, 0,
+ IMX6UL_GPR1_ENET1_TX_CLK_DIR | IMX6UL_GPR1_ENET1_CLK_SEL };
static const u32 enet1_ref_sels_table_mask = IMX6UL_GPR1_ENET1_TX_CLK_DIR |
IMX6UL_GPR1_ENET1_CLK_SEL;
-static const char * enet2_ref_sels[] = { "enet2_ref_125m", "enet2_ref_pad", };
+static const char * enet2_ref_sels[] = { "enet2_ref_125m", "enet2_ref_pad", "dummy", "dummy"};
static const u32 enet2_ref_sels_table[] = { IMX6UL_GPR1_ENET2_TX_CLK_DIR,
- IMX6UL_GPR1_ENET2_CLK_SEL };
+ IMX6UL_GPR1_ENET2_CLK_SEL, 0,
+ IMX6UL_GPR1_ENET2_TX_CLK_DIR | IMX6UL_GPR1_ENET2_CLK_SEL };
static const u32 enet2_ref_sels_table_mask = IMX6UL_GPR1_ENET2_TX_CLK_DIR |
IMX6UL_GPR1_ENET2_CLK_SEL;
diff --git a/drivers/clk/sprd/common.c b/drivers/clk/sprd/common.c
index ce81e40..2bfbab8 100644
--- a/drivers/clk/sprd/common.c
+++ b/drivers/clk/sprd/common.c
@@ -17,7 +17,6 @@ static const struct regmap_config sprdclk_regmap_config = {
.reg_bits = 32,
.reg_stride = 4,
.val_bits = 32,
- .max_register = 0xffff,
.fast_io = true,
};
@@ -43,6 +42,8 @@ int sprd_clk_regmap_init(struct platform_device *pdev,
struct device *dev = &pdev->dev;
struct device_node *node = dev->of_node, *np;
struct regmap *regmap;
+ struct resource *res;
+ struct regmap_config reg_config = sprdclk_regmap_config;
if (of_find_property(node, "sprd,syscon", NULL)) {
regmap = syscon_regmap_lookup_by_phandle(node, "sprd,syscon");
@@ -59,12 +60,14 @@ int sprd_clk_regmap_init(struct platform_device *pdev,
return PTR_ERR(regmap);
}
} else {
- base = devm_platform_ioremap_resource(pdev, 0);
+ base = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (IS_ERR(base))
return PTR_ERR(base);
+ reg_config.max_register = resource_size(res) - reg_config.reg_stride;
+
regmap = devm_regmap_init_mmio(&pdev->dev, base,
- &sprdclk_regmap_config);
+ ®_config);
if (IS_ERR(regmap)) {
pr_err("failed to init regmap\n");
return PTR_ERR(regmap);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 73c7643..8dd46fa 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -840,22 +840,20 @@ static int amd_pstate_update_status(const char *buf, size_t size)
switch(mode_idx) {
case AMD_PSTATE_DISABLE:
- if (!current_pstate_driver)
- return -EINVAL;
- if (cppc_state == AMD_PSTATE_ACTIVE)
- return -EBUSY;
- cpufreq_unregister_driver(current_pstate_driver);
- amd_pstate_driver_cleanup();
+ if (current_pstate_driver) {
+ cpufreq_unregister_driver(current_pstate_driver);
+ amd_pstate_driver_cleanup();
+ }
break;
case AMD_PSTATE_PASSIVE:
if (current_pstate_driver) {
if (current_pstate_driver == &amd_pstate_driver)
return 0;
cpufreq_unregister_driver(current_pstate_driver);
- cppc_state = AMD_PSTATE_PASSIVE;
- current_pstate_driver = &amd_pstate_driver;
}
+ current_pstate_driver = &amd_pstate_driver;
+ cppc_state = AMD_PSTATE_PASSIVE;
ret = cpufreq_register_driver(current_pstate_driver);
break;
case AMD_PSTATE_ACTIVE:
@@ -863,10 +861,10 @@ static int amd_pstate_update_status(const char *buf, size_t size)
if (current_pstate_driver == &amd_pstate_epp_driver)
return 0;
cpufreq_unregister_driver(current_pstate_driver);
- current_pstate_driver = &amd_pstate_epp_driver;
- cppc_state = AMD_PSTATE_ACTIVE;
}
+ current_pstate_driver = &amd_pstate_epp_driver;
+ cppc_state = AMD_PSTATE_ACTIVE;
ret = cpufreq_register_driver(current_pstate_driver);
break;
default:
diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c
index 29619f4..d9629ff 100644
--- a/drivers/firmware/psci/psci.c
+++ b/drivers/firmware/psci/psci.c
@@ -167,7 +167,8 @@ int psci_set_osi_mode(bool enable)
err = invoke_psci_fn(PSCI_1_0_FN_SET_SUSPEND_MODE, suspend_mode, 0, 0);
if (err < 0)
- pr_warn("failed to set %s mode: %d\n", enable ? "OSI" : "PC", err);
+ pr_info(FW_BUG "failed to set %s mode: %d\n",
+ enable ? "OSI" : "PC", err);
return psci_to_linux_errno(err);
}
diff --git a/drivers/i2c/busses/i2c-mchp-pci1xxxx.c b/drivers/i2c/busses/i2c-mchp-pci1xxxx.c
index 09af759..b21ffd6 100644
--- a/drivers/i2c/busses/i2c-mchp-pci1xxxx.c
+++ b/drivers/i2c/busses/i2c-mchp-pci1xxxx.c
@@ -48,9 +48,9 @@
* SR_HOLD_TIME_XK_TICKS field will indicate the number of ticks of the
* baud clock required to program 'Hold Time' at X KHz.
*/
-#define SR_HOLD_TIME_100K_TICKS 133
-#define SR_HOLD_TIME_400K_TICKS 20
-#define SR_HOLD_TIME_1000K_TICKS 11
+#define SR_HOLD_TIME_100K_TICKS 150
+#define SR_HOLD_TIME_400K_TICKS 20
+#define SR_HOLD_TIME_1000K_TICKS 12
#define SMB_CORE_COMPLETION_REG_OFF3 (SMBUS_MAST_CORE_ADDR_BASE + 0x23)
@@ -65,17 +65,17 @@
* the baud clock required to program 'fair idle delay' at X KHz. Fair idle
* delay establishes the MCTP T(IDLE_DELAY) period.
*/
-#define FAIR_BUS_IDLE_MIN_100K_TICKS 969
-#define FAIR_BUS_IDLE_MIN_400K_TICKS 157
-#define FAIR_BUS_IDLE_MIN_1000K_TICKS 157
+#define FAIR_BUS_IDLE_MIN_100K_TICKS 992
+#define FAIR_BUS_IDLE_MIN_400K_TICKS 500
+#define FAIR_BUS_IDLE_MIN_1000K_TICKS 500
/*
* FAIR_IDLE_DELAY_XK_TICKS field will indicate the number of ticks of the
* baud clock required to satisfy the fairness protocol at X KHz.
*/
-#define FAIR_IDLE_DELAY_100K_TICKS 1000
-#define FAIR_IDLE_DELAY_400K_TICKS 500
-#define FAIR_IDLE_DELAY_1000K_TICKS 500
+#define FAIR_IDLE_DELAY_100K_TICKS 963
+#define FAIR_IDLE_DELAY_400K_TICKS 156
+#define FAIR_IDLE_DELAY_1000K_TICKS 156
#define SMB_IDLE_SCALING_100K \
((FAIR_IDLE_DELAY_100K_TICKS << 16) | FAIR_BUS_IDLE_MIN_100K_TICKS)
@@ -105,7 +105,7 @@
*/
#define BUS_CLK_100K_LOW_PERIOD_TICKS 156
#define BUS_CLK_400K_LOW_PERIOD_TICKS 41
-#define BUS_CLK_1000K_LOW_PERIOD_TICKS 15
+#define BUS_CLK_1000K_LOW_PERIOD_TICKS 15
/*
* BUS_CLK_XK_HIGH_PERIOD_TICKS field defines the number of I2C Baud Clock
@@ -131,7 +131,7 @@
*/
#define CLK_SYNC_100K 4
#define CLK_SYNC_400K 4
-#define CLK_SYNC_1000K 4
+#define CLK_SYNC_1000K 4
#define SMB_CORE_DATA_TIMING_REG_OFF (SMBUS_MAST_CORE_ADDR_BASE + 0x40)
@@ -142,25 +142,25 @@
* determines the SCLK hold time following SDAT driven low during the first
* START bit in a transfer.
*/
-#define FIRST_START_HOLD_100K_TICKS 22
-#define FIRST_START_HOLD_400K_TICKS 16
-#define FIRST_START_HOLD_1000K_TICKS 6
+#define FIRST_START_HOLD_100K_TICKS 23
+#define FIRST_START_HOLD_400K_TICKS 8
+#define FIRST_START_HOLD_1000K_TICKS 12
/*
* STOP_SETUP_XK_TICKS will indicate the number of ticks of the baud clock
* required to program 'STOP_SETUP' timer at X KHz. This timer determines the
* SDAT setup time from the rising edge of SCLK for a STOP condition.
*/
-#define STOP_SETUP_100K_TICKS 157
+#define STOP_SETUP_100K_TICKS 150
#define STOP_SETUP_400K_TICKS 20
-#define STOP_SETUP_1000K_TICKS 12
+#define STOP_SETUP_1000K_TICKS 12
/*
* RESTART_SETUP_XK_TICKS will indicate the number of ticks of the baud clock
* required to program 'RESTART_SETUP' timer at X KHz. This timer determines the
* SDAT setup time from the rising edge of SCLK for a repeated START condition.
*/
-#define RESTART_SETUP_100K_TICKS 157
+#define RESTART_SETUP_100K_TICKS 156
#define RESTART_SETUP_400K_TICKS 20
#define RESTART_SETUP_1000K_TICKS 12
@@ -169,7 +169,7 @@
* required to program 'DATA_HOLD' timer at X KHz. This timer determines the
* SDAT hold time following SCLK driven low.
*/
-#define DATA_HOLD_100K_TICKS 2
+#define DATA_HOLD_100K_TICKS 12
#define DATA_HOLD_400K_TICKS 2
#define DATA_HOLD_1000K_TICKS 2
@@ -190,35 +190,35 @@
* Bus Idle Minimum time = BUS_IDLE_MIN[7:0] x Baud_Clock_Period x
* (BUS_IDLE_MIN_XK_TICKS[7] ? 4,1)
*/
-#define BUS_IDLE_MIN_100K_TICKS 167UL
-#define BUS_IDLE_MIN_400K_TICKS 139UL
-#define BUS_IDLE_MIN_1000K_TICKS 133UL
+#define BUS_IDLE_MIN_100K_TICKS 36UL
+#define BUS_IDLE_MIN_400K_TICKS 10UL
+#define BUS_IDLE_MIN_1000K_TICKS 4UL
/*
* CTRL_CUM_TIME_OUT_XK_TICKS defines SMBus Controller Cumulative Time-Out.
* SMBus Controller Cumulative Time-Out duration =
* CTRL_CUM_TIME_OUT_XK_TICKS[7:0] x Baud_Clock_Period x 2048
*/
-#define CTRL_CUM_TIME_OUT_100K_TICKS 159
-#define CTRL_CUM_TIME_OUT_400K_TICKS 159
-#define CTRL_CUM_TIME_OUT_1000K_TICKS 159
+#define CTRL_CUM_TIME_OUT_100K_TICKS 76
+#define CTRL_CUM_TIME_OUT_400K_TICKS 76
+#define CTRL_CUM_TIME_OUT_1000K_TICKS 76
/*
* TARGET_CUM_TIME_OUT_XK_TICKS defines SMBus Target Cumulative Time-Out duration.
* SMBus Target Cumulative Time-Out duration = TARGET_CUM_TIME_OUT_XK_TICKS[7:0] x
* Baud_Clock_Period x 4096
*/
-#define TARGET_CUM_TIME_OUT_100K_TICKS 199
-#define TARGET_CUM_TIME_OUT_400K_TICKS 199
-#define TARGET_CUM_TIME_OUT_1000K_TICKS 199
+#define TARGET_CUM_TIME_OUT_100K_TICKS 95
+#define TARGET_CUM_TIME_OUT_400K_TICKS 95
+#define TARGET_CUM_TIME_OUT_1000K_TICKS 95
/*
* CLOCK_HIGH_TIME_OUT_XK defines Clock High time out period.
* Clock High time out period = CLOCK_HIGH_TIME_OUT_XK[7:0] x Baud_Clock_Period x 8
*/
-#define CLOCK_HIGH_TIME_OUT_100K_TICKS 204
-#define CLOCK_HIGH_TIME_OUT_400K_TICKS 204
-#define CLOCK_HIGH_TIME_OUT_1000K_TICKS 204
+#define CLOCK_HIGH_TIME_OUT_100K_TICKS 97
+#define CLOCK_HIGH_TIME_OUT_400K_TICKS 97
+#define CLOCK_HIGH_TIME_OUT_1000K_TICKS 97
#define TO_SCALING_100K \
((BUS_IDLE_MIN_100K_TICKS << 24) | (CTRL_CUM_TIME_OUT_100K_TICKS << 16) | \
diff --git a/drivers/i2c/busses/i2c-ocores.c b/drivers/i2c/busses/i2c-ocores.c
index a0af027..2e57585 100644
--- a/drivers/i2c/busses/i2c-ocores.c
+++ b/drivers/i2c/busses/i2c-ocores.c
@@ -342,18 +342,18 @@ static int ocores_poll_wait(struct ocores_i2c *i2c)
* ocores_isr(), we just add our polling code around it.
*
* It can run in atomic context
+ *
+ * Return: 0 on success, -ETIMEDOUT on timeout
*/
-static void ocores_process_polling(struct ocores_i2c *i2c)
+static int ocores_process_polling(struct ocores_i2c *i2c)
{
- while (1) {
- irqreturn_t ret;
- int err;
+ irqreturn_t ret;
+ int err = 0;
+ while (1) {
err = ocores_poll_wait(i2c);
- if (err) {
- i2c->state = STATE_ERROR;
+ if (err)
break; /* timeout */
- }
ret = ocores_isr(-1, i2c);
if (ret == IRQ_NONE)
@@ -364,13 +364,15 @@ static void ocores_process_polling(struct ocores_i2c *i2c)
break;
}
}
+
+ return err;
}
static int ocores_xfer_core(struct ocores_i2c *i2c,
struct i2c_msg *msgs, int num,
bool polling)
{
- int ret;
+ int ret = 0;
u8 ctrl;
ctrl = oc_getreg(i2c, OCI2C_CONTROL);
@@ -388,15 +390,16 @@ static int ocores_xfer_core(struct ocores_i2c *i2c,
oc_setreg(i2c, OCI2C_CMD, OCI2C_CMD_START);
if (polling) {
- ocores_process_polling(i2c);
+ ret = ocores_process_polling(i2c);
} else {
- ret = wait_event_timeout(i2c->wait,
- (i2c->state == STATE_ERROR) ||
- (i2c->state == STATE_DONE), HZ);
- if (ret == 0) {
- ocores_process_timeout(i2c);
- return -ETIMEDOUT;
- }
+ if (wait_event_timeout(i2c->wait,
+ (i2c->state == STATE_ERROR) ||
+ (i2c->state == STATE_DONE), HZ) == 0)
+ ret = -ETIMEDOUT;
+ }
+ if (ret) {
+ ocores_process_timeout(i2c);
+ return ret;
}
return (i2c->state == STATE_DONE) ? num : -EIO;
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 3081559..6b9563d 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -624,22 +624,11 @@ static inline unsigned short cma_family(struct rdma_id_private *id_priv)
return id_priv->id.route.addr.src_addr.ss_family;
}
-static int cma_set_qkey(struct rdma_id_private *id_priv, u32 qkey)
+static int cma_set_default_qkey(struct rdma_id_private *id_priv)
{
struct ib_sa_mcmember_rec rec;
int ret = 0;
- if (id_priv->qkey) {
- if (qkey && id_priv->qkey != qkey)
- return -EINVAL;
- return 0;
- }
-
- if (qkey) {
- id_priv->qkey = qkey;
- return 0;
- }
-
switch (id_priv->id.ps) {
case RDMA_PS_UDP:
case RDMA_PS_IB:
@@ -659,6 +648,16 @@ static int cma_set_qkey(struct rdma_id_private *id_priv, u32 qkey)
return ret;
}
+static int cma_set_qkey(struct rdma_id_private *id_priv, u32 qkey)
+{
+ if (!qkey ||
+ (id_priv->qkey && (id_priv->qkey != qkey)))
+ return -EINVAL;
+
+ id_priv->qkey = qkey;
+ return 0;
+}
+
static void cma_translate_ib(struct sockaddr_ib *sib, struct rdma_dev_addr *dev_addr)
{
dev_addr->dev_type = ARPHRD_INFINIBAND;
@@ -1229,7 +1228,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,
*qp_attr_mask = IB_QP_STATE | IB_QP_PKEY_INDEX | IB_QP_PORT;
if (id_priv->id.qp_type == IB_QPT_UD) {
- ret = cma_set_qkey(id_priv, 0);
+ ret = cma_set_default_qkey(id_priv);
if (ret)
return ret;
@@ -4569,7 +4568,10 @@ static int cma_send_sidr_rep(struct rdma_id_private *id_priv,
memset(&rep, 0, sizeof rep);
rep.status = status;
if (status == IB_SIDR_SUCCESS) {
- ret = cma_set_qkey(id_priv, qkey);
+ if (qkey)
+ ret = cma_set_qkey(id_priv, qkey);
+ else
+ ret = cma_set_default_qkey(id_priv);
if (ret)
return ret;
rep.qp_num = id_priv->qp_num;
@@ -4774,9 +4776,7 @@ static void cma_make_mc_event(int status, struct rdma_id_private *id_priv,
enum ib_gid_type gid_type;
struct net_device *ndev;
- if (!status)
- status = cma_set_qkey(id_priv, be32_to_cpu(multicast->rec.qkey));
- else
+ if (status)
pr_debug_ratelimited("RDMA CM: MULTICAST_ERROR: failed to join multicast. status %d\n",
status);
@@ -4804,7 +4804,7 @@ static void cma_make_mc_event(int status, struct rdma_id_private *id_priv,
}
event->param.ud.qp_num = 0xFFFFFF;
- event->param.ud.qkey = be32_to_cpu(multicast->rec.qkey);
+ event->param.ud.qkey = id_priv->qkey;
out:
if (ndev)
@@ -4823,8 +4823,11 @@ static int cma_ib_mc_handler(int status, struct ib_sa_multicast *multicast)
READ_ONCE(id_priv->state) == RDMA_CM_DESTROYING)
goto out;
- cma_make_mc_event(status, id_priv, multicast, &event, mc);
- ret = cma_cm_event_handler(id_priv, &event);
+ ret = cma_set_qkey(id_priv, be32_to_cpu(multicast->rec.qkey));
+ if (!ret) {
+ cma_make_mc_event(status, id_priv, multicast, &event, mc);
+ ret = cma_cm_event_handler(id_priv, &event);
+ }
rdma_destroy_ah_attr(&event.param.ud.ah_attr);
WARN_ON(ret);
@@ -4877,9 +4880,11 @@ static int cma_join_ib_multicast(struct rdma_id_private *id_priv,
if (ret)
return ret;
- ret = cma_set_qkey(id_priv, 0);
- if (ret)
- return ret;
+ if (!id_priv->qkey) {
+ ret = cma_set_default_qkey(id_priv);
+ if (ret)
+ return ret;
+ }
cma_set_mgid(id_priv, (struct sockaddr *) &mc->addr, &rec.mgid);
rec.qkey = cpu_to_be32(id_priv->qkey);
@@ -4956,9 +4961,6 @@ static int cma_iboe_join_multicast(struct rdma_id_private *id_priv,
cma_iboe_set_mgid(addr, &ib.rec.mgid, gid_type);
ib.rec.pkey = cpu_to_be16(0xffff);
- if (id_priv->id.ps == RDMA_PS_UDP)
- ib.rec.qkey = cpu_to_be32(RDMA_UDP_QKEY);
-
if (dev_addr->bound_dev_if)
ndev = dev_get_by_index(dev_addr->net, dev_addr->bound_dev_if);
if (!ndev)
@@ -4984,6 +4986,9 @@ static int cma_iboe_join_multicast(struct rdma_id_private *id_priv,
if (err || !ib.rec.mtu)
return err ?: -EINVAL;
+ if (!id_priv->qkey)
+ cma_set_default_qkey(id_priv);
+
rdma_ip2gid((struct sockaddr *)&id_priv->id.route.addr.src_addr,
&ib.rec.port_gid);
INIT_WORK(&mc->iboe_join.work, cma_iboe_join_work_handler);
@@ -5009,6 +5014,9 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
READ_ONCE(id_priv->state) != RDMA_CM_ADDR_RESOLVED))
return -EINVAL;
+ if (id_priv->id.qp_type != IB_QPT_UD)
+ return -EINVAL;
+
mc = kzalloc(sizeof(*mc), GFP_KERNEL);
if (!mc)
return -ENOMEM;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 11b1c16..b99b3cc 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -532,6 +532,8 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
else
ret = device->ops.create_ah(ah, &init_attr, NULL);
if (ret) {
+ if (ah->sgid_attr)
+ rdma_put_gid_attr(ah->sgid_attr);
kfree(ah);
return ERR_PTR(ret);
}
diff --git a/drivers/infiniband/hw/erdma/erdma_cq.c b/drivers/infiniband/hw/erdma/erdma_cq.c
index cabd867..7bc3542 100644
--- a/drivers/infiniband/hw/erdma/erdma_cq.c
+++ b/drivers/infiniband/hw/erdma/erdma_cq.c
@@ -65,7 +65,7 @@ static const enum ib_wc_opcode wc_mapping_table[ERDMA_NUM_OPCODES] = {
[ERDMA_OP_LOCAL_INV] = IB_WC_LOCAL_INV,
[ERDMA_OP_READ_WITH_INV] = IB_WC_RDMA_READ,
[ERDMA_OP_ATOMIC_CAS] = IB_WC_COMP_SWAP,
- [ERDMA_OP_ATOMIC_FAD] = IB_WC_FETCH_ADD,
+ [ERDMA_OP_ATOMIC_FAA] = IB_WC_FETCH_ADD,
};
static const struct {
diff --git a/drivers/infiniband/hw/erdma/erdma_hw.h b/drivers/infiniband/hw/erdma/erdma_hw.h
index 4c38d99..37ad1bb 100644
--- a/drivers/infiniband/hw/erdma/erdma_hw.h
+++ b/drivers/infiniband/hw/erdma/erdma_hw.h
@@ -441,7 +441,7 @@ struct erdma_reg_mr_sqe {
};
/* EQ related. */
-#define ERDMA_DEFAULT_EQ_DEPTH 256
+#define ERDMA_DEFAULT_EQ_DEPTH 4096
/* ceqe */
#define ERDMA_CEQE_HDR_DB_MASK BIT_ULL(63)
@@ -491,7 +491,7 @@ enum erdma_opcode {
ERDMA_OP_LOCAL_INV = 15,
ERDMA_OP_READ_WITH_INV = 16,
ERDMA_OP_ATOMIC_CAS = 17,
- ERDMA_OP_ATOMIC_FAD = 18,
+ ERDMA_OP_ATOMIC_FAA = 18,
ERDMA_NUM_OPCODES = 19,
ERDMA_OP_INVALID = ERDMA_NUM_OPCODES + 1
};
diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c
index 5dc31e5..4a29a53 100644
--- a/drivers/infiniband/hw/erdma/erdma_main.c
+++ b/drivers/infiniband/hw/erdma/erdma_main.c
@@ -56,7 +56,7 @@ static int erdma_netdev_event(struct notifier_block *nb, unsigned long event,
static int erdma_enum_and_get_netdev(struct erdma_dev *dev)
{
struct net_device *netdev;
- int ret = -ENODEV;
+ int ret = -EPROBE_DEFER;
/* Already binded to a net_device, so we skip. */
if (dev->netdev)
diff --git a/drivers/infiniband/hw/erdma/erdma_qp.c b/drivers/infiniband/hw/erdma/erdma_qp.c
index d088d6be..44923c5 100644
--- a/drivers/infiniband/hw/erdma/erdma_qp.c
+++ b/drivers/infiniband/hw/erdma/erdma_qp.c
@@ -405,7 +405,7 @@ static int erdma_push_one_sqe(struct erdma_qp *qp, u16 *pi,
FIELD_PREP(ERDMA_SQE_MR_MTT_CNT_MASK,
mr->mem.mtt_nents);
- if (mr->mem.mtt_nents < ERDMA_MAX_INLINE_MTT_ENTRIES) {
+ if (mr->mem.mtt_nents <= ERDMA_MAX_INLINE_MTT_ENTRIES) {
attrs |= FIELD_PREP(ERDMA_SQE_MR_MTT_TYPE_MASK, 0);
/* Copy SGLs to SQE content to accelerate */
memcpy(get_queue_entry(qp->kern_qp.sq_buf, idx + 1,
@@ -439,7 +439,7 @@ static int erdma_push_one_sqe(struct erdma_qp *qp, u16 *pi,
cpu_to_le64(atomic_wr(send_wr)->compare_add);
} else {
wqe_hdr |= FIELD_PREP(ERDMA_SQE_HDR_OPCODE_MASK,
- ERDMA_OP_ATOMIC_FAD);
+ ERDMA_OP_ATOMIC_FAA);
atomic_sqe->fetchadd_swap_data =
cpu_to_le64(atomic_wr(send_wr)->compare_add);
}
diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h
index e0a993b..131cf5f 100644
--- a/drivers/infiniband/hw/erdma/erdma_verbs.h
+++ b/drivers/infiniband/hw/erdma/erdma_verbs.h
@@ -11,7 +11,7 @@
/* RDMA Capability. */
#define ERDMA_MAX_PD (128 * 1024)
-#define ERDMA_MAX_SEND_WR 4096
+#define ERDMA_MAX_SEND_WR 8192
#define ERDMA_MAX_ORD 128
#define ERDMA_MAX_IRD 128
#define ERDMA_MAX_SGE_RD 1
diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
index 195aa9e..8817864 100644
--- a/drivers/infiniband/hw/irdma/cm.c
+++ b/drivers/infiniband/hw/irdma/cm.c
@@ -1458,13 +1458,15 @@ static int irdma_send_fin(struct irdma_cm_node *cm_node)
* irdma_find_listener - find a cm node listening on this addr-port pair
* @cm_core: cm's core
* @dst_addr: listener ip addr
+ * @ipv4: flag indicating IPv4 when true
* @dst_port: listener tcp port num
* @vlan_id: virtual LAN ID
* @listener_state: state to match with listen node's
*/
static struct irdma_cm_listener *
-irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, u16 dst_port,
- u16 vlan_id, enum irdma_cm_listener_state listener_state)
+irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, bool ipv4,
+ u16 dst_port, u16 vlan_id,
+ enum irdma_cm_listener_state listener_state)
{
struct irdma_cm_listener *listen_node;
static const u32 ip_zero[4] = { 0, 0, 0, 0 };
@@ -1477,7 +1479,7 @@ irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, u16 dst_port,
list_for_each_entry (listen_node, &cm_core->listen_list, list) {
memcpy(listen_addr, listen_node->loc_addr, sizeof(listen_addr));
listen_port = listen_node->loc_port;
- if (listen_port != dst_port ||
+ if (listen_node->ipv4 != ipv4 || listen_port != dst_port ||
!(listener_state & listen_node->listener_state))
continue;
/* compare node pair, return node handle if a match */
@@ -2902,9 +2904,10 @@ irdma_make_listen_node(struct irdma_cm_core *cm_core,
unsigned long flags;
/* cannot have multiple matching listeners */
- listener = irdma_find_listener(cm_core, cm_info->loc_addr,
- cm_info->loc_port, cm_info->vlan_id,
- IRDMA_CM_LISTENER_EITHER_STATE);
+ listener =
+ irdma_find_listener(cm_core, cm_info->loc_addr, cm_info->ipv4,
+ cm_info->loc_port, cm_info->vlan_id,
+ IRDMA_CM_LISTENER_EITHER_STATE);
if (listener &&
listener->listener_state == IRDMA_CM_LISTENER_ACTIVE_STATE) {
refcount_dec(&listener->refcnt);
@@ -3153,6 +3156,7 @@ void irdma_receive_ilq(struct irdma_sc_vsi *vsi, struct irdma_puda_buf *rbuf)
listener = irdma_find_listener(cm_core,
cm_info.loc_addr,
+ cm_info.ipv4,
cm_info.loc_port,
cm_info.vlan_id,
IRDMA_CM_LISTENER_ACTIVE_STATE);
diff --git a/drivers/infiniband/hw/irdma/cm.h b/drivers/infiniband/hw/irdma/cm.h
index 19c2849..7feadb3 100644
--- a/drivers/infiniband/hw/irdma/cm.h
+++ b/drivers/infiniband/hw/irdma/cm.h
@@ -41,7 +41,7 @@
#define TCP_OPTIONS_PADDING 3
#define IRDMA_DEFAULT_RETRYS 64
-#define IRDMA_DEFAULT_RETRANS 8
+#define IRDMA_DEFAULT_RETRANS 32
#define IRDMA_DEFAULT_TTL 0x40
#define IRDMA_DEFAULT_RTT_VAR 6
#define IRDMA_DEFAULT_SS_THRESH 0x3fffffff
diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c
index 2e1e2ba..43dfa47 100644
--- a/drivers/infiniband/hw/irdma/hw.c
+++ b/drivers/infiniband/hw/irdma/hw.c
@@ -41,6 +41,7 @@ static enum irdma_hmc_rsrc_type iw_hmc_obj_types[] = {
IRDMA_HMC_IW_XFFL,
IRDMA_HMC_IW_Q1,
IRDMA_HMC_IW_Q1FL,
+ IRDMA_HMC_IW_PBLE,
IRDMA_HMC_IW_TIMER,
IRDMA_HMC_IW_FSIMC,
IRDMA_HMC_IW_FSIAV,
@@ -827,6 +828,8 @@ static int irdma_create_hmc_objs(struct irdma_pci_f *rf, bool privileged,
info.entry_type = rf->sd_type;
for (i = 0; i < IW_HMC_OBJ_TYPE_NUM; i++) {
+ if (iw_hmc_obj_types[i] == IRDMA_HMC_IW_PBLE)
+ continue;
if (dev->hmc_info->hmc_obj[iw_hmc_obj_types[i]].cnt) {
info.rsrc_type = iw_hmc_obj_types[i];
info.count = dev->hmc_info->hmc_obj[info.rsrc_type].cnt;
diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c
index 445e69e8..7887230 100644
--- a/drivers/infiniband/hw/irdma/utils.c
+++ b/drivers/infiniband/hw/irdma/utils.c
@@ -2595,7 +2595,10 @@ void irdma_generate_flush_completions(struct irdma_qp *iwqp)
/* remove the SQ WR by moving SQ tail*/
IRDMA_RING_SET_TAIL(*sq_ring,
sq_ring->tail + qp->sq_wrtrk_array[sq_ring->tail].quanta);
-
+ if (cmpl->cpi.op_type == IRDMAQP_OP_NOP) {
+ kfree(cmpl);
+ continue;
+ }
ibdev_dbg(iwqp->iwscq->ibcq.device,
"DEV: %s: adding wr_id = 0x%llx SQ Completion to list qp_id=%d\n",
__func__, cmpl->cpi.wr_id, qp->qp_id);
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 5b988db..5d45de2 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -442,6 +442,10 @@ static int translate_eth_ext_proto_oper(u32 eth_proto_oper, u16 *active_speed,
*active_width = IB_WIDTH_2X;
*active_speed = IB_SPEED_NDR;
break;
+ case MLX5E_PROT_MASK(MLX5E_400GAUI_8):
+ *active_width = IB_WIDTH_8X;
+ *active_speed = IB_SPEED_HDR;
+ break;
case MLX5E_PROT_MASK(MLX5E_400GAUI_4_400GBASE_CR4_KR4):
*active_width = IB_WIDTH_4X;
*active_speed = IB_SPEED_NDR;
diff --git a/drivers/memstick/core/memstick.c b/drivers/memstick/core/memstick.c
index bf76678..bbfaf65 100644
--- a/drivers/memstick/core/memstick.c
+++ b/drivers/memstick/core/memstick.c
@@ -410,6 +410,7 @@ static struct memstick_dev *memstick_alloc_card(struct memstick_host *host)
return card;
err_out:
host->card = old_card;
+ kfree_const(card->dev.kobj.name);
kfree(card);
return NULL;
}
@@ -468,8 +469,10 @@ static void memstick_check(struct work_struct *work)
put_device(&card->dev);
host->card = NULL;
}
- } else
+ } else {
+ kfree_const(card->dev.kobj.name);
kfree(card);
+ }
}
out_power_off:
diff --git a/drivers/mmc/host/sdhci_am654.c b/drivers/mmc/host/sdhci_am654.c
index 8995309..672d37e 100644
--- a/drivers/mmc/host/sdhci_am654.c
+++ b/drivers/mmc/host/sdhci_am654.c
@@ -351,8 +351,6 @@ static void sdhci_am654_write_b(struct sdhci_host *host, u8 val, int reg)
*/
case MMC_TIMING_SD_HS:
case MMC_TIMING_MMC_HS:
- case MMC_TIMING_UHS_SDR12:
- case MMC_TIMING_UHS_SDR25:
val &= ~SDHCI_CTRL_HISPD;
}
}
diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 0904eb4..ad025b2e 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -666,12 +666,6 @@ static int io_init(struct ubi_device *ubi, int max_beb_per1024)
ubi->ec_hdr_alsize = ALIGN(UBI_EC_HDR_SIZE, ubi->hdrs_min_io_size);
ubi->vid_hdr_alsize = ALIGN(UBI_VID_HDR_SIZE, ubi->hdrs_min_io_size);
- if (ubi->vid_hdr_offset && ((ubi->vid_hdr_offset + UBI_VID_HDR_SIZE) >
- ubi->vid_hdr_alsize)) {
- ubi_err(ubi, "VID header offset %d too large.", ubi->vid_hdr_offset);
- return -EINVAL;
- }
-
dbg_gen("min_io_size %d", ubi->min_io_size);
dbg_gen("max_write_size %d", ubi->max_write_size);
dbg_gen("hdrs_min_io_size %d", ubi->hdrs_min_io_size);
@@ -689,6 +683,21 @@ static int io_init(struct ubi_device *ubi, int max_beb_per1024)
ubi->vid_hdr_aloffset;
}
+ /*
+ * Memory allocation for VID header is ubi->vid_hdr_alsize
+ * which is described in comments in io.c.
+ * Make sure VID header shift + UBI_VID_HDR_SIZE not exceeds
+ * ubi->vid_hdr_alsize, so that all vid header operations
+ * won't access memory out of bounds.
+ */
+ if ((ubi->vid_hdr_shift + UBI_VID_HDR_SIZE) > ubi->vid_hdr_alsize) {
+ ubi_err(ubi, "Invalid VID header offset %d, VID header shift(%d)"
+ " + VID header size(%zu) > VID header aligned size(%d).",
+ ubi->vid_hdr_offset, ubi->vid_hdr_shift,
+ UBI_VID_HDR_SIZE, ubi->vid_hdr_alsize);
+ return -EINVAL;
+ }
+
/* Similar for the data offset */
ubi->leb_start = ubi->vid_hdr_offset + UBI_VID_HDR_SIZE;
ubi->leb_start = ALIGN(ubi->leb_start, ubi->min_io_size);
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 40f39e5..26a214f 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -575,7 +575,7 @@ static int erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk,
* @vol_id: the volume ID that last used this PEB
* @lnum: the last used logical eraseblock number for the PEB
* @torture: if the physical eraseblock has to be tortured
- * @nested: denotes whether the work_sem is already held in read mode
+ * @nested: denotes whether the work_sem is already held
*
* This function returns zero in case of success and a %-ENOMEM in case of
* failure.
@@ -1131,7 +1131,7 @@ static int __erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk)
int err1;
/* Re-schedule the LEB for erasure */
- err1 = schedule_erase(ubi, e, vol_id, lnum, 0, false);
+ err1 = schedule_erase(ubi, e, vol_id, lnum, 0, true);
if (err1) {
spin_lock(&ubi->wl_lock);
wl_entry_destroy(ubi, e);
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 8cc9a74..710548d 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -1777,14 +1777,15 @@ void bond_lower_state_changed(struct slave *slave)
/* The bonding driver uses ether_setup() to convert a master bond device
* to ARPHRD_ETHER, that resets the target netdevice's flags so we always
- * have to restore the IFF_MASTER flag, and only restore IFF_SLAVE if it was set
+ * have to restore the IFF_MASTER flag, and only restore IFF_SLAVE and IFF_UP
+ * if they were set
*/
static void bond_ether_setup(struct net_device *bond_dev)
{
- unsigned int slave_flag = bond_dev->flags & IFF_SLAVE;
+ unsigned int flags = bond_dev->flags & (IFF_SLAVE | IFF_UP);
ether_setup(bond_dev);
- bond_dev->flags |= IFF_MASTER | slave_flag;
+ bond_dev->flags |= IFF_MASTER | flags;
bond_dev->priv_flags &= ~IFF_TX_SKB_SHARING;
}
@@ -5696,9 +5697,13 @@ static int bond_ethtool_get_ts_info(struct net_device *bond_dev,
struct ethtool_ts_info *info)
{
struct bonding *bond = netdev_priv(bond_dev);
+ struct ethtool_ts_info ts_info;
const struct ethtool_ops *ops;
struct net_device *real_dev;
+ bool sw_tx_support = false;
struct phy_device *phydev;
+ struct list_head *iter;
+ struct slave *slave;
int ret = 0;
rcu_read_lock();
@@ -5717,10 +5722,36 @@ static int bond_ethtool_get_ts_info(struct net_device *bond_dev,
ret = ops->get_ts_info(real_dev, info);
goto out;
}
+ } else {
+ /* Check if all slaves support software tx timestamping */
+ rcu_read_lock();
+ bond_for_each_slave_rcu(bond, slave, iter) {
+ ret = -1;
+ ops = slave->dev->ethtool_ops;
+ phydev = slave->dev->phydev;
+
+ if (phy_has_tsinfo(phydev))
+ ret = phy_ts_info(phydev, &ts_info);
+ else if (ops->get_ts_info)
+ ret = ops->get_ts_info(slave->dev, &ts_info);
+
+ if (!ret && (ts_info.so_timestamping & SOF_TIMESTAMPING_TX_SOFTWARE)) {
+ sw_tx_support = true;
+ continue;
+ }
+
+ sw_tx_support = false;
+ break;
+ }
+ rcu_read_unlock();
}
+ ret = 0;
info->so_timestamping = SOF_TIMESTAMPING_RX_SOFTWARE |
SOF_TIMESTAMPING_SOFTWARE;
+ if (sw_tx_support)
+ info->so_timestamping |= SOF_TIMESTAMPING_TX_SOFTWARE;
+
info->phc_index = -1;
out:
diff --git a/drivers/net/dsa/microchip/ksz8795.c b/drivers/net/dsa/microchip/ksz8795.c
index 23614a9..f56fca1 100644
--- a/drivers/net/dsa/microchip/ksz8795.c
+++ b/drivers/net/dsa/microchip/ksz8795.c
@@ -96,7 +96,7 @@ static int ksz8795_change_mtu(struct ksz_device *dev, int frame_size)
if (frame_size > KSZ8_LEGAL_PACKET_SIZE)
ctrl2 |= SW_LEGAL_PACKET_DISABLE;
- else if (frame_size > KSZ8863_NORMAL_PACKET_SIZE)
+ if (frame_size > KSZ8863_NORMAL_PACKET_SIZE)
ctrl1 |= SW_HUGE_PACKET;
ret = ksz_rmw8(dev, REG_SW_CTRL_1, SW_HUGE_PACKET, ctrl1);
diff --git a/drivers/net/dsa/mt7530-mdio.c b/drivers/net/dsa/mt7530-mdio.c
index 34a547b..0885336 100644
--- a/drivers/net/dsa/mt7530-mdio.c
+++ b/drivers/net/dsa/mt7530-mdio.c
@@ -81,14 +81,17 @@ static const struct regmap_bus mt7530_regmap_bus = {
};
static int
-mt7531_create_sgmii(struct mt7530_priv *priv)
+mt7531_create_sgmii(struct mt7530_priv *priv, bool dual_sgmii)
{
- struct regmap_config *mt7531_pcs_config[2];
+ struct regmap_config *mt7531_pcs_config[2] = {};
struct phylink_pcs *pcs;
struct regmap *regmap;
int i, ret = 0;
- for (i = 0; i < 2; i++) {
+ /* MT7531AE has two SGMII units for port 5 and port 6
+ * MT7531BE has only one SGMII unit for port 6
+ */
+ for (i = dual_sgmii ? 0 : 1; i < 2; i++) {
mt7531_pcs_config[i] = devm_kzalloc(priv->dev,
sizeof(struct regmap_config),
GFP_KERNEL);
@@ -208,11 +211,8 @@ mt7530_probe(struct mdio_device *mdiodev)
if (IS_ERR(priv->regmap))
return PTR_ERR(priv->regmap);
- if (priv->id == ID_MT7531) {
- ret = mt7531_create_sgmii(priv);
- if (ret)
- return ret;
- }
+ if (priv->id == ID_MT7531)
+ priv->create_sgmii = mt7531_create_sgmii;
return dsa_register_switch(priv->ds);
}
diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index e4bb5037..c680873 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -3018,6 +3018,12 @@ mt753x_setup(struct dsa_switch *ds)
if (ret && priv->irq)
mt7530_free_irq_common(priv);
+ if (priv->create_sgmii) {
+ ret = priv->create_sgmii(priv, mt7531_dual_sgmii_supported(priv));
+ if (ret && priv->irq)
+ mt7530_free_irq(priv);
+ }
+
return ret;
}
diff --git a/drivers/net/dsa/mt7530.h b/drivers/net/dsa/mt7530.h
index 01db5c9..5084f48 100644
--- a/drivers/net/dsa/mt7530.h
+++ b/drivers/net/dsa/mt7530.h
@@ -748,10 +748,10 @@ struct mt753x_info {
* registers
* @p6_interface Holding the current port 6 interface
* @p5_intf_sel: Holding the current port 5 interface select
- *
* @irq: IRQ number of the switch
* @irq_domain: IRQ domain of the switch irq_chip
* @irq_enable: IRQ enable bits, synced to SYS_INT_EN
+ * @create_sgmii: Pointer to function creating SGMII PCS instance(s)
*/
struct mt7530_priv {
struct device *dev;
@@ -770,7 +770,6 @@ struct mt7530_priv {
unsigned int p5_intf_sel;
u8 mirror_rx;
u8 mirror_tx;
-
struct mt7530_port ports[MT7530_NUM_PORTS];
struct mt753x_pcs pcs[MT7530_NUM_PORTS];
/* protect among processes for registers access*/
@@ -778,6 +777,7 @@ struct mt7530_priv {
int irq;
struct irq_domain *irq_domain;
u32 irq_enable;
+ int (*create_sgmii)(struct mt7530_priv *priv, bool dual_sgmii);
};
struct mt7530_hw_vlan_entry {
diff --git a/drivers/net/dsa/ocelot/felix.c b/drivers/net/dsa/ocelot/felix.c
index 6dcebcf..80861ac 100644
--- a/drivers/net/dsa/ocelot/felix.c
+++ b/drivers/net/dsa/ocelot/felix.c
@@ -1550,11 +1550,6 @@ static int felix_connect_tag_protocol(struct dsa_switch *ds,
}
}
-/* Hardware initialization done here so that we can allocate structures with
- * devm without fear of dsa_register_switch returning -EPROBE_DEFER and causing
- * us to allocate structures twice (leak memory) and map PCI memory twice
- * (which will not work).
- */
static int felix_setup(struct dsa_switch *ds)
{
struct ocelot *ocelot = ds->priv;
diff --git a/drivers/net/dsa/ocelot/felix_vsc9959.c b/drivers/net/dsa/ocelot/felix_vsc9959.c
index dddb289..cfb3fae 100644
--- a/drivers/net/dsa/ocelot/felix_vsc9959.c
+++ b/drivers/net/dsa/ocelot/felix_vsc9959.c
@@ -1424,6 +1424,7 @@ static int vsc9959_qos_port_tas_set(struct ocelot *ocelot, int port,
mutex_lock(&ocelot->tas_lock);
if (!taprio->enable) {
+ ocelot_port_mqprio(ocelot, port, &taprio->mqprio);
ocelot_rmw_rix(ocelot, 0, QSYS_TAG_CONFIG_ENABLE,
QSYS_TAG_CONFIG, port);
@@ -1436,15 +1437,19 @@ static int vsc9959_qos_port_tas_set(struct ocelot *ocelot, int port,
return 0;
}
+ ret = ocelot_port_mqprio(ocelot, port, &taprio->mqprio);
+ if (ret)
+ goto err_unlock;
+
if (taprio->cycle_time > NSEC_PER_SEC ||
taprio->cycle_time_extension >= NSEC_PER_SEC) {
ret = -EINVAL;
- goto err;
+ goto err_reset_tc;
}
if (taprio->num_entries > VSC9959_TAS_GCL_ENTRY_MAX) {
ret = -ERANGE;
- goto err;
+ goto err_reset_tc;
}
/* Enable guard band. The switch will schedule frames without taking
@@ -1468,7 +1473,7 @@ static int vsc9959_qos_port_tas_set(struct ocelot *ocelot, int port,
val = ocelot_read(ocelot, QSYS_PARAM_STATUS_REG_8);
if (val & QSYS_PARAM_STATUS_REG_8_CONFIG_PENDING) {
ret = -EBUSY;
- goto err;
+ goto err_reset_tc;
}
ocelot_rmw_rix(ocelot,
@@ -1503,12 +1508,19 @@ static int vsc9959_qos_port_tas_set(struct ocelot *ocelot, int port,
!(val & QSYS_TAS_PARAM_CFG_CTRL_CONFIG_CHANGE),
10, 100000);
if (ret)
- goto err;
+ goto err_reset_tc;
ocelot_port->taprio = taprio_offload_get(taprio);
vsc9959_tas_guard_bands_update(ocelot, port);
-err:
+ mutex_unlock(&ocelot->tas_lock);
+
+ return 0;
+
+err_reset_tc:
+ taprio->mqprio.qopt.num_tc = 0;
+ ocelot_port_mqprio(ocelot, port, &taprio->mqprio);
+err_unlock:
mutex_unlock(&ocelot->tas_lock);
return ret;
@@ -1612,6 +1624,13 @@ static int vsc9959_qos_port_cbs_set(struct dsa_switch *ds, int port,
static int vsc9959_qos_query_caps(struct tc_query_caps_base *base)
{
switch (base->type) {
+ case TC_SETUP_QDISC_MQPRIO: {
+ struct tc_mqprio_caps *caps = base->caps;
+
+ caps->validate_queue_counts = true;
+
+ return 0;
+ }
case TC_SETUP_QDISC_TAPRIO: {
struct tc_taprio_caps *caps = base->caps;
@@ -1635,6 +1654,8 @@ static int vsc9959_port_setup_tc(struct dsa_switch *ds, int port,
return vsc9959_qos_query_caps(type_data);
case TC_SETUP_QDISC_TAPRIO:
return vsc9959_qos_port_tas_set(ocelot, port, type_data);
+ case TC_SETUP_QDISC_MQPRIO:
+ return ocelot_port_mqprio(ocelot, port, type_data);
case TC_SETUP_QDISC_CBS:
return vsc9959_qos_port_cbs_set(ds, port, type_data);
default:
@@ -2498,6 +2519,7 @@ static void vsc9959_cut_through_fwd(struct ocelot *ocelot)
for (port = 0; port < ocelot->num_phys_ports; port++) {
struct ocelot_port *ocelot_port = ocelot->ports[port];
+ struct ocelot_mm_state *mm = &ocelot->mm[port];
int min_speed = ocelot_port->speed;
unsigned long mask = 0;
u32 tmp, val = 0;
@@ -2538,10 +2560,12 @@ static void vsc9959_cut_through_fwd(struct ocelot *ocelot)
/* Enable cut-through forwarding for all traffic classes that
* don't have oversized dropping enabled, since this check is
- * bypassed in cut-through mode.
+ * bypassed in cut-through mode. Also exclude preemptible
+ * traffic classes, since these would hang the port for some
+ * reason, if sent as cut-through.
*/
if (ocelot_port->speed == min_speed) {
- val = GENMASK(7, 0);
+ val = GENMASK(7, 0) & ~mm->active_preemptible_tcs;
for (tc = 0; tc < OCELOT_NUM_TC; tc++)
if (vsc9959_port_qmaxsdu_get(ocelot, port, tc))
@@ -2610,12 +2634,9 @@ static const struct felix_info felix_info_vsc9959 = {
static irqreturn_t felix_irq_handler(int irq, void *data)
{
struct ocelot *ocelot = (struct ocelot *)data;
- int port;
ocelot_get_txtstamp(ocelot);
-
- for (port = 0; port < ocelot->num_phys_ports; port++)
- ocelot_port_mm_irq(ocelot, port);
+ ocelot_mm_irq(ocelot);
return IRQ_HANDLED;
}
diff --git a/drivers/net/dsa/qca/Kconfig b/drivers/net/dsa/qca/Kconfig
index ba33974..4347b42 100644
--- a/drivers/net/dsa/qca/Kconfig
+++ b/drivers/net/dsa/qca/Kconfig
@@ -15,3 +15,11 @@
help
This enables support for the Qualcomm Atheros QCA8K Ethernet
switch chips.
+
+config NET_DSA_QCA8K_LEDS_SUPPORT
+ bool "Qualcomm Atheros QCA8K Ethernet switch family LEDs support"
+ depends on NET_DSA_QCA8K
+ depends on LEDS_CLASS=y || LEDS_CLASS=NET_DSA_QCA8K
+ help
+ This enabled support for LEDs present on the Qualcomm Atheros
+ QCA8K Ethernet switch chips.
diff --git a/drivers/net/dsa/qca/Makefile b/drivers/net/dsa/qca/Makefile
index 701f1d1..ce66b19 100644
--- a/drivers/net/dsa/qca/Makefile
+++ b/drivers/net/dsa/qca/Makefile
@@ -2,3 +2,6 @@
obj-$(CONFIG_NET_DSA_AR9331) += ar9331.o
obj-$(CONFIG_NET_DSA_QCA8K) += qca8k.o
qca8k-y += qca8k-common.o qca8k-8xxx.o
+ifdef CONFIG_NET_DSA_QCA8K_LEDS_SUPPORT
+qca8k-y += qca8k-leds.o
+endif
diff --git a/drivers/net/dsa/qca/qca8k-8xxx.c b/drivers/net/dsa/qca/qca8k-8xxx.c
index 6281090..6d5ac75 100644
--- a/drivers/net/dsa/qca/qca8k-8xxx.c
+++ b/drivers/net/dsa/qca/qca8k-8xxx.c
@@ -22,6 +22,7 @@
#include <linux/dsa/tag_qca.h>
#include "qca8k.h"
+#include "qca8k_leds.h"
static void
qca8k_split_addr(u32 regaddr, u16 *r1, u16 *r2, u16 *page)
@@ -772,21 +773,6 @@ qca8k_phy_eth_command(struct qca8k_priv *priv, bool read, int phy,
return ret;
}
-static u32
-qca8k_port_to_phy(int port)
-{
- /* From Andrew Lunn:
- * Port 0 has no internal phy.
- * Port 1 has an internal PHY at MDIO address 0.
- * Port 2 has an internal PHY at MDIO address 1.
- * ...
- * Port 5 has an internal PHY at MDIO address 4.
- * Port 6 has no internal PHY.
- */
-
- return port - 1;
-}
-
static int
qca8k_mdio_busy_wait(struct mii_bus *bus, u32 reg, u32 mask)
{
@@ -1797,6 +1783,10 @@ qca8k_setup(struct dsa_switch *ds)
if (ret)
return ret;
+ ret = qca8k_setup_led_ctrl(priv);
+ if (ret)
+ return ret;
+
qca8k_setup_pcs(priv, &priv->pcs_port_0, 0);
qca8k_setup_pcs(priv, &priv->pcs_port_6, 6);
diff --git a/drivers/net/dsa/qca/qca8k-leds.c b/drivers/net/dsa/qca/qca8k-leds.c
new file mode 100644
index 0000000..b883692
--- /dev/null
+++ b/drivers/net/dsa/qca/qca8k-leds.c
@@ -0,0 +1,277 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/regmap.h>
+#include <net/dsa.h>
+
+#include "qca8k.h"
+#include "qca8k_leds.h"
+
+static int
+qca8k_get_enable_led_reg(int port_num, int led_num, struct qca8k_led_pattern_en *reg_info)
+{
+ switch (port_num) {
+ case 0:
+ reg_info->reg = QCA8K_LED_CTRL_REG(led_num);
+ reg_info->shift = QCA8K_LED_PHY0123_CONTROL_RULE_SHIFT;
+ break;
+ case 1:
+ case 2:
+ case 3:
+ /* Port 123 are controlled on a different reg */
+ reg_info->reg = QCA8K_LED_CTRL3_REG;
+ reg_info->shift = QCA8K_LED_PHY123_PATTERN_EN_SHIFT(port_num, led_num);
+ break;
+ case 4:
+ reg_info->reg = QCA8K_LED_CTRL_REG(led_num);
+ reg_info->shift = QCA8K_LED_PHY4_CONTROL_RULE_SHIFT;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+qca8k_led_brightness_set(struct qca8k_led *led,
+ enum led_brightness brightness)
+{
+ struct qca8k_led_pattern_en reg_info;
+ struct qca8k_priv *priv = led->priv;
+ u32 mask, val;
+
+ qca8k_get_enable_led_reg(led->port_num, led->led_num, ®_info);
+
+ val = QCA8K_LED_ALWAYS_OFF;
+ if (brightness)
+ val = QCA8K_LED_ALWAYS_ON;
+
+ /* HW regs to control brightness is special and port 1-2-3
+ * are placed in a different reg.
+ *
+ * To control port 0 brightness:
+ * - the 2 bit (15, 14) of:
+ * - QCA8K_LED_CTRL0_REG for led1
+ * - QCA8K_LED_CTRL1_REG for led2
+ * - QCA8K_LED_CTRL2_REG for led3
+ *
+ * To control port 4:
+ * - the 2 bit (31, 30) of:
+ * - QCA8K_LED_CTRL0_REG for led1
+ * - QCA8K_LED_CTRL1_REG for led2
+ * - QCA8K_LED_CTRL2_REG for led3
+ *
+ * To control port 1:
+ * - the 2 bit at (9, 8) of QCA8K_LED_CTRL3_REG are used for led1
+ * - the 2 bit at (11, 10) of QCA8K_LED_CTRL3_REG are used for led2
+ * - the 2 bit at (13, 12) of QCA8K_LED_CTRL3_REG are used for led3
+ *
+ * To control port 2:
+ * - the 2 bit at (15, 14) of QCA8K_LED_CTRL3_REG are used for led1
+ * - the 2 bit at (17, 16) of QCA8K_LED_CTRL3_REG are used for led2
+ * - the 2 bit at (19, 18) of QCA8K_LED_CTRL3_REG are used for led3
+ *
+ * To control port 3:
+ * - the 2 bit at (21, 20) of QCA8K_LED_CTRL3_REG are used for led1
+ * - the 2 bit at (23, 22) of QCA8K_LED_CTRL3_REG are used for led2
+ * - the 2 bit at (25, 24) of QCA8K_LED_CTRL3_REG are used for led3
+ *
+ * To abstract this and have less code, we use the port and led numm
+ * to calculate the shift and the correct reg due to this problem of
+ * not having a 1:1 map of LED with the regs.
+ */
+ if (led->port_num == 0 || led->port_num == 4) {
+ mask = QCA8K_LED_PATTERN_EN_MASK;
+ val <<= QCA8K_LED_PATTERN_EN_SHIFT;
+ } else {
+ mask = QCA8K_LED_PHY123_PATTERN_EN_MASK;
+ }
+
+ return regmap_update_bits(priv->regmap, reg_info.reg,
+ mask << reg_info.shift,
+ val << reg_info.shift);
+}
+
+static int
+qca8k_cled_brightness_set_blocking(struct led_classdev *ldev,
+ enum led_brightness brightness)
+{
+ struct qca8k_led *led = container_of(ldev, struct qca8k_led, cdev);
+
+ return qca8k_led_brightness_set(led, brightness);
+}
+
+static enum led_brightness
+qca8k_led_brightness_get(struct qca8k_led *led)
+{
+ struct qca8k_led_pattern_en reg_info;
+ struct qca8k_priv *priv = led->priv;
+ u32 val;
+ int ret;
+
+ qca8k_get_enable_led_reg(led->port_num, led->led_num, ®_info);
+
+ ret = regmap_read(priv->regmap, reg_info.reg, &val);
+ if (ret)
+ return 0;
+
+ val >>= reg_info.shift;
+
+ if (led->port_num == 0 || led->port_num == 4) {
+ val &= QCA8K_LED_PATTERN_EN_MASK;
+ val >>= QCA8K_LED_PATTERN_EN_SHIFT;
+ } else {
+ val &= QCA8K_LED_PHY123_PATTERN_EN_MASK;
+ }
+
+ /* Assume brightness ON only when the LED is set to always ON */
+ return val == QCA8K_LED_ALWAYS_ON;
+}
+
+static int
+qca8k_cled_blink_set(struct led_classdev *ldev,
+ unsigned long *delay_on,
+ unsigned long *delay_off)
+{
+ struct qca8k_led *led = container_of(ldev, struct qca8k_led, cdev);
+ u32 mask, val = QCA8K_LED_ALWAYS_BLINK_4HZ;
+ struct qca8k_led_pattern_en reg_info;
+ struct qca8k_priv *priv = led->priv;
+
+ if (*delay_on == 0 && *delay_off == 0) {
+ *delay_on = 125;
+ *delay_off = 125;
+ }
+
+ if (*delay_on != 125 || *delay_off != 125) {
+ /* The hardware only supports blinking at 4Hz. Fall back
+ * to software implementation in other cases.
+ */
+ return -EINVAL;
+ }
+
+ qca8k_get_enable_led_reg(led->port_num, led->led_num, ®_info);
+
+ if (led->port_num == 0 || led->port_num == 4) {
+ mask = QCA8K_LED_PATTERN_EN_MASK;
+ val <<= QCA8K_LED_PATTERN_EN_SHIFT;
+ } else {
+ mask = QCA8K_LED_PHY123_PATTERN_EN_MASK;
+ }
+
+ regmap_update_bits(priv->regmap, reg_info.reg, mask << reg_info.shift,
+ val << reg_info.shift);
+
+ return 0;
+}
+
+static int
+qca8k_parse_port_leds(struct qca8k_priv *priv, struct fwnode_handle *port, int port_num)
+{
+ struct fwnode_handle *led = NULL, *leds = NULL;
+ struct led_init_data init_data = { };
+ struct dsa_switch *ds = priv->ds;
+ enum led_default_state state;
+ struct qca8k_led *port_led;
+ int led_num, led_index;
+ int ret;
+
+ leds = fwnode_get_named_child_node(port, "leds");
+ if (!leds) {
+ dev_dbg(priv->dev, "No Leds node specified in device tree for port %d!\n",
+ port_num);
+ return 0;
+ }
+
+ fwnode_for_each_child_node(leds, led) {
+ /* Reg represent the led number of the port.
+ * Each port can have at most 3 leds attached
+ * Commonly:
+ * 1. is gigabit led
+ * 2. is mbit led
+ * 3. additional status led
+ */
+ if (fwnode_property_read_u32(led, "reg", &led_num))
+ continue;
+
+ if (led_num >= QCA8K_LED_PORT_COUNT) {
+ dev_warn(priv->dev, "Invalid LED reg %d defined for port %d",
+ led_num, port_num);
+ continue;
+ }
+
+ led_index = QCA8K_LED_PORT_INDEX(port_num, led_num);
+
+ port_led = &priv->ports_led[led_index];
+ port_led->port_num = port_num;
+ port_led->led_num = led_num;
+ port_led->priv = priv;
+
+ state = led_init_default_state_get(led);
+ switch (state) {
+ case LEDS_DEFSTATE_ON:
+ port_led->cdev.brightness = 1;
+ qca8k_led_brightness_set(port_led, 1);
+ break;
+ case LEDS_DEFSTATE_KEEP:
+ port_led->cdev.brightness =
+ qca8k_led_brightness_get(port_led);
+ break;
+ default:
+ port_led->cdev.brightness = 0;
+ qca8k_led_brightness_set(port_led, 0);
+ }
+
+ port_led->cdev.max_brightness = 1;
+ port_led->cdev.brightness_set_blocking = qca8k_cled_brightness_set_blocking;
+ port_led->cdev.blink_set = qca8k_cled_blink_set;
+ init_data.default_label = ":port";
+ init_data.fwnode = led;
+ init_data.devname_mandatory = true;
+ init_data.devicename = kasprintf(GFP_KERNEL, "%s:0%d", ds->slave_mii_bus->id,
+ port_num);
+ if (!init_data.devicename)
+ return -ENOMEM;
+
+ ret = devm_led_classdev_register_ext(priv->dev, &port_led->cdev, &init_data);
+ if (ret)
+ dev_warn(priv->dev, "Failed to init LED %d for port %d", led_num, port_num);
+
+ kfree(init_data.devicename);
+ }
+
+ return 0;
+}
+
+int
+qca8k_setup_led_ctrl(struct qca8k_priv *priv)
+{
+ struct fwnode_handle *ports, *port;
+ int port_num;
+ int ret;
+
+ ports = device_get_named_child_node(priv->dev, "ports");
+ if (!ports) {
+ dev_info(priv->dev, "No ports node specified in device tree!");
+ return 0;
+ }
+
+ fwnode_for_each_child_node(ports, port) {
+ if (fwnode_property_read_u32(port, "reg", &port_num))
+ continue;
+
+ /* Skip checking for CPU port 0 and CPU port 6 as not supported */
+ if (port_num == 0 || port_num == 6)
+ continue;
+
+ /* Each port can have at most 3 different leds attached.
+ * Switch port starts from 0 to 6, but port 0 and 6 are CPU
+ * port. The port index needs to be decreased by one to identify
+ * the correct port for LED setup.
+ */
+ ret = qca8k_parse_port_leds(priv, port, qca8k_port_to_phy(port_num));
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/drivers/net/dsa/qca/qca8k.h b/drivers/net/dsa/qca/qca8k.h
index 7996975..c5cc8a17 100644
--- a/drivers/net/dsa/qca/qca8k.h
+++ b/drivers/net/dsa/qca/qca8k.h
@@ -11,6 +11,7 @@
#include <linux/delay.h>
#include <linux/regmap.h>
#include <linux/gpio.h>
+#include <linux/leds.h>
#include <linux/dsa/tag_qca.h>
#define QCA8K_ETHERNET_MDIO_PRIORITY 7
@@ -85,6 +86,51 @@
#define QCA8K_MDIO_MASTER_DATA(x) FIELD_PREP(QCA8K_MDIO_MASTER_DATA_MASK, x)
#define QCA8K_MDIO_MASTER_MAX_PORTS 5
#define QCA8K_MDIO_MASTER_MAX_REG 32
+
+/* LED control register */
+#define QCA8K_LED_PORT_COUNT 3
+#define QCA8K_LED_COUNT ((QCA8K_NUM_PORTS - QCA8K_NUM_CPU_PORTS) * QCA8K_LED_PORT_COUNT)
+#define QCA8K_LED_RULE_COUNT 6
+#define QCA8K_LED_RULE_MAX 11
+#define QCA8K_LED_PORT_INDEX(_phy, _led) (((_phy) * QCA8K_LED_PORT_COUNT) + (_led))
+
+#define QCA8K_LED_PHY123_PATTERN_EN_SHIFT(_phy, _led) ((((_phy) - 1) * 6) + 8 + (2 * (_led)))
+#define QCA8K_LED_PHY123_PATTERN_EN_MASK GENMASK(1, 0)
+
+#define QCA8K_LED_PHY0123_CONTROL_RULE_SHIFT 0
+#define QCA8K_LED_PHY4_CONTROL_RULE_SHIFT 16
+
+#define QCA8K_LED_CTRL_REG(_i) (0x050 + (_i) * 4)
+#define QCA8K_LED_CTRL0_REG 0x50
+#define QCA8K_LED_CTRL1_REG 0x54
+#define QCA8K_LED_CTRL2_REG 0x58
+#define QCA8K_LED_CTRL3_REG 0x5C
+#define QCA8K_LED_CTRL_SHIFT(_i) (((_i) % 2) * 16)
+#define QCA8K_LED_CTRL_MASK GENMASK(15, 0)
+#define QCA8K_LED_RULE_MASK GENMASK(13, 0)
+#define QCA8K_LED_BLINK_FREQ_MASK GENMASK(1, 0)
+#define QCA8K_LED_BLINK_FREQ_SHITF 0
+#define QCA8K_LED_BLINK_2HZ 0
+#define QCA8K_LED_BLINK_4HZ 1
+#define QCA8K_LED_BLINK_8HZ 2
+#define QCA8K_LED_BLINK_AUTO 3
+#define QCA8K_LED_LINKUP_OVER_MASK BIT(2)
+#define QCA8K_LED_TX_BLINK_MASK BIT(4)
+#define QCA8K_LED_RX_BLINK_MASK BIT(5)
+#define QCA8K_LED_COL_BLINK_MASK BIT(7)
+#define QCA8K_LED_LINK_10M_EN_MASK BIT(8)
+#define QCA8K_LED_LINK_100M_EN_MASK BIT(9)
+#define QCA8K_LED_LINK_1000M_EN_MASK BIT(10)
+#define QCA8K_LED_POWER_ON_LIGHT_MASK BIT(11)
+#define QCA8K_LED_HALF_DUPLEX_MASK BIT(12)
+#define QCA8K_LED_FULL_DUPLEX_MASK BIT(13)
+#define QCA8K_LED_PATTERN_EN_MASK GENMASK(15, 14)
+#define QCA8K_LED_PATTERN_EN_SHIFT 14
+#define QCA8K_LED_ALWAYS_OFF 0
+#define QCA8K_LED_ALWAYS_BLINK_4HZ 1
+#define QCA8K_LED_ALWAYS_ON 2
+#define QCA8K_LED_RULE_CONTROLLED 3
+
#define QCA8K_GOL_MAC_ADDR0 0x60
#define QCA8K_GOL_MAC_ADDR1 0x64
#define QCA8K_MAX_FRAME_SIZE 0x78
@@ -382,6 +428,19 @@ struct qca8k_pcs {
int port;
};
+struct qca8k_led_pattern_en {
+ u32 reg;
+ u8 shift;
+};
+
+struct qca8k_led {
+ u8 port_num;
+ u8 led_num;
+ u16 old_rule;
+ struct qca8k_priv *priv;
+ struct led_classdev cdev;
+};
+
struct qca8k_priv {
u8 switch_id;
u8 switch_revision;
@@ -406,6 +465,7 @@ struct qca8k_priv {
struct qca8k_pcs pcs_port_0;
struct qca8k_pcs pcs_port_6;
const struct qca8k_match_data *info;
+ struct qca8k_led ports_led[QCA8K_LED_COUNT];
};
struct qca8k_mib_desc {
@@ -421,6 +481,20 @@ struct qca8k_fdb {
u8 mac[6];
};
+static inline u32 qca8k_port_to_phy(int port)
+{
+ /* From Andrew Lunn:
+ * Port 0 has no internal phy.
+ * Port 1 has an internal PHY at MDIO address 0.
+ * Port 2 has an internal PHY at MDIO address 1.
+ * ...
+ * Port 5 has an internal PHY at MDIO address 4.
+ * Port 6 has no internal PHY.
+ */
+
+ return port - 1;
+}
+
/* Common setup function */
extern const struct qca8k_mib_desc ar8327_mib[];
extern const struct regmap_access_table qca8k_readable_table;
diff --git a/drivers/net/dsa/qca/qca8k_leds.h b/drivers/net/dsa/qca/qca8k_leds.h
new file mode 100644
index 0000000..ab367f0
--- /dev/null
+++ b/drivers/net/dsa/qca/qca8k_leds.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __QCA8K_LEDS_H
+#define __QCA8K_LEDS_H
+
+/* Leds Support function */
+#ifdef CONFIG_NET_DSA_QCA8K_LEDS_SUPPORT
+int qca8k_setup_led_ctrl(struct qca8k_priv *priv);
+#else
+static inline int qca8k_setup_led_ctrl(struct qca8k_priv *priv)
+{
+ return 0;
+}
+#endif
+
+#endif /* __QCA8K_LEDS_H */
diff --git a/drivers/net/ethernet/amd/Kconfig b/drivers/net/ethernet/amd/Kconfig
index ab42f75..235fcac 100644
--- a/drivers/net/ethernet/amd/Kconfig
+++ b/drivers/net/ethernet/amd/Kconfig
@@ -186,4 +186,16 @@
bool
default n
+config PDS_CORE
+ tristate "AMD/Pensando Data Systems Core Device Support"
+ depends on 64BIT && PCI
+ help
+ This enables the support for the AMD/Pensando Core device family of
+ adapters. More specific information on this driver can be
+ found in
+ <file:Documentation/networking/device_drivers/ethernet/amd/pds_core.rst>.
+
+ To compile this driver as a module, choose M here. The module
+ will be called pds_core.
+
endif # NET_VENDOR_AMD
diff --git a/drivers/net/ethernet/amd/Makefile b/drivers/net/ethernet/amd/Makefile
index 42742af..2dcfb847 100644
--- a/drivers/net/ethernet/amd/Makefile
+++ b/drivers/net/ethernet/amd/Makefile
@@ -17,3 +17,4 @@
obj-$(CONFIG_SUN3LANCE) += sun3lance.o
obj-$(CONFIG_SUNLANCE) += sunlance.o
obj-$(CONFIG_AMD_XGBE) += xgbe/
+obj-$(CONFIG_PDS_CORE) += pds_core/
diff --git a/drivers/net/ethernet/amd/pds_core/Makefile b/drivers/net/ethernet/amd/pds_core/Makefile
new file mode 100644
index 0000000..0abc33c
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/Makefile
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Advanced Micro Devices, Inc.
+
+obj-$(CONFIG_PDS_CORE) := pds_core.o
+
+pds_core-y := main.o \
+ devlink.o \
+ auxbus.o \
+ dev.o \
+ adminq.o \
+ core.o \
+ fw.o
+
+pds_core-$(CONFIG_DEBUG_FS) += debugfs.o
diff --git a/drivers/net/ethernet/amd/pds_core/adminq.c b/drivers/net/ethernet/amd/pds_core/adminq.c
new file mode 100644
index 0000000..045fe13
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/adminq.c
@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/dynamic_debug.h>
+
+#include "core.h"
+
+struct pdsc_wait_context {
+ struct pdsc_qcq *qcq;
+ struct completion wait_completion;
+};
+
+static int pdsc_process_notifyq(struct pdsc_qcq *qcq)
+{
+ union pds_core_notifyq_comp *comp;
+ struct pdsc *pdsc = qcq->pdsc;
+ struct pdsc_cq *cq = &qcq->cq;
+ struct pdsc_cq_info *cq_info;
+ int nq_work = 0;
+ u64 eid;
+
+ cq_info = &cq->info[cq->tail_idx];
+ comp = cq_info->comp;
+ eid = le64_to_cpu(comp->event.eid);
+ while (eid > pdsc->last_eid) {
+ u16 ecode = le16_to_cpu(comp->event.ecode);
+
+ switch (ecode) {
+ case PDS_EVENT_LINK_CHANGE:
+ dev_info(pdsc->dev, "NotifyQ LINK_CHANGE ecode %d eid %lld\n",
+ ecode, eid);
+ pdsc_notify(PDS_EVENT_LINK_CHANGE, comp);
+ break;
+
+ case PDS_EVENT_RESET:
+ dev_info(pdsc->dev, "NotifyQ RESET ecode %d eid %lld\n",
+ ecode, eid);
+ pdsc_notify(PDS_EVENT_RESET, comp);
+ break;
+
+ case PDS_EVENT_XCVR:
+ dev_info(pdsc->dev, "NotifyQ XCVR ecode %d eid %lld\n",
+ ecode, eid);
+ break;
+
+ default:
+ dev_info(pdsc->dev, "NotifyQ ecode %d eid %lld\n",
+ ecode, eid);
+ break;
+ }
+
+ pdsc->last_eid = eid;
+ cq->tail_idx = (cq->tail_idx + 1) & (cq->num_descs - 1);
+ cq_info = &cq->info[cq->tail_idx];
+ comp = cq_info->comp;
+ eid = le64_to_cpu(comp->event.eid);
+
+ nq_work++;
+ }
+
+ qcq->accum_work += nq_work;
+
+ return nq_work;
+}
+
+void pdsc_process_adminq(struct pdsc_qcq *qcq)
+{
+ union pds_core_adminq_comp *comp;
+ struct pdsc_queue *q = &qcq->q;
+ struct pdsc *pdsc = qcq->pdsc;
+ struct pdsc_cq *cq = &qcq->cq;
+ struct pdsc_q_info *q_info;
+ unsigned long irqflags;
+ int nq_work = 0;
+ int aq_work = 0;
+ int credits;
+
+ /* Don't process AdminQ when shutting down */
+ if (pdsc->state & BIT_ULL(PDSC_S_STOPPING_DRIVER)) {
+ dev_err(pdsc->dev, "%s: called while PDSC_S_STOPPING_DRIVER\n",
+ __func__);
+ return;
+ }
+
+ /* Check for NotifyQ event */
+ nq_work = pdsc_process_notifyq(&pdsc->notifyqcq);
+
+ /* Check for empty queue, which can happen if the interrupt was
+ * for a NotifyQ event and there are no new AdminQ completions.
+ */
+ if (q->tail_idx == q->head_idx)
+ goto credits;
+
+ /* Find the first completion to clean,
+ * run the callback in the related q_info,
+ * and continue while we still match done color
+ */
+ spin_lock_irqsave(&pdsc->adminq_lock, irqflags);
+ comp = cq->info[cq->tail_idx].comp;
+ while (pdsc_color_match(comp->color, cq->done_color)) {
+ q_info = &q->info[q->tail_idx];
+ q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1);
+
+ /* Copy out the completion data */
+ memcpy(q_info->dest, comp, sizeof(*comp));
+
+ complete_all(&q_info->wc->wait_completion);
+
+ if (cq->tail_idx == cq->num_descs - 1)
+ cq->done_color = !cq->done_color;
+ cq->tail_idx = (cq->tail_idx + 1) & (cq->num_descs - 1);
+ comp = cq->info[cq->tail_idx].comp;
+
+ aq_work++;
+ }
+ spin_unlock_irqrestore(&pdsc->adminq_lock, irqflags);
+
+ qcq->accum_work += aq_work;
+
+credits:
+ /* Return the interrupt credits, one for each completion */
+ credits = nq_work + aq_work;
+ if (credits)
+ pds_core_intr_credits(&pdsc->intr_ctrl[qcq->intx],
+ credits,
+ PDS_CORE_INTR_CRED_REARM);
+}
+
+void pdsc_work_thread(struct work_struct *work)
+{
+ struct pdsc_qcq *qcq = container_of(work, struct pdsc_qcq, work);
+
+ pdsc_process_adminq(qcq);
+}
+
+irqreturn_t pdsc_adminq_isr(int irq, void *data)
+{
+ struct pdsc_qcq *qcq = data;
+ struct pdsc *pdsc = qcq->pdsc;
+
+ /* Don't process AdminQ when shutting down */
+ if (pdsc->state & BIT_ULL(PDSC_S_STOPPING_DRIVER)) {
+ dev_err(pdsc->dev, "%s: called while PDSC_S_STOPPING_DRIVER\n",
+ __func__);
+ return IRQ_HANDLED;
+ }
+
+ queue_work(pdsc->wq, &qcq->work);
+ pds_core_intr_mask(&pdsc->intr_ctrl[irq], PDS_CORE_INTR_MASK_CLEAR);
+
+ return IRQ_HANDLED;
+}
+
+static int __pdsc_adminq_post(struct pdsc *pdsc,
+ struct pdsc_qcq *qcq,
+ union pds_core_adminq_cmd *cmd,
+ union pds_core_adminq_comp *comp,
+ struct pdsc_wait_context *wc)
+{
+ struct pdsc_queue *q = &qcq->q;
+ struct pdsc_q_info *q_info;
+ unsigned long irqflags;
+ unsigned int avail;
+ int index;
+ int ret;
+
+ spin_lock_irqsave(&pdsc->adminq_lock, irqflags);
+
+ /* Check for space in the queue */
+ avail = q->tail_idx;
+ if (q->head_idx >= avail)
+ avail += q->num_descs - q->head_idx - 1;
+ else
+ avail -= q->head_idx + 1;
+ if (!avail) {
+ ret = -ENOSPC;
+ goto err_out_unlock;
+ }
+
+ /* Check that the FW is running */
+ if (!pdsc_is_fw_running(pdsc)) {
+ u8 fw_status = ioread8(&pdsc->info_regs->fw_status);
+
+ dev_info(pdsc->dev, "%s: post failed - fw not running %#02x:\n",
+ __func__, fw_status);
+ ret = -ENXIO;
+
+ goto err_out_unlock;
+ }
+
+ /* Post the request */
+ index = q->head_idx;
+ q_info = &q->info[index];
+ q_info->wc = wc;
+ q_info->dest = comp;
+ memcpy(q_info->desc, cmd, sizeof(*cmd));
+
+ dev_dbg(pdsc->dev, "head_idx %d tail_idx %d\n",
+ q->head_idx, q->tail_idx);
+ dev_dbg(pdsc->dev, "post admin queue command:\n");
+ dynamic_hex_dump("cmd ", DUMP_PREFIX_OFFSET, 16, 1,
+ cmd, sizeof(*cmd), true);
+
+ q->head_idx = (q->head_idx + 1) & (q->num_descs - 1);
+
+ pds_core_dbell_ring(pdsc->kern_dbpage,
+ q->hw_type, q->dbval | q->head_idx);
+ ret = index;
+
+err_out_unlock:
+ spin_unlock_irqrestore(&pdsc->adminq_lock, irqflags);
+ return ret;
+}
+
+int pdsc_adminq_post(struct pdsc *pdsc,
+ union pds_core_adminq_cmd *cmd,
+ union pds_core_adminq_comp *comp,
+ bool fast_poll)
+{
+ struct pdsc_wait_context wc = {
+ .wait_completion =
+ COMPLETION_INITIALIZER_ONSTACK(wc.wait_completion),
+ };
+ unsigned long poll_interval = 1;
+ unsigned long poll_jiffies;
+ unsigned long time_limit;
+ unsigned long time_start;
+ unsigned long time_done;
+ unsigned long remaining;
+ int err = 0;
+ int index;
+
+ wc.qcq = &pdsc->adminqcq;
+ index = __pdsc_adminq_post(pdsc, &pdsc->adminqcq, cmd, comp, &wc);
+ if (index < 0) {
+ err = index;
+ goto err_out;
+ }
+
+ time_start = jiffies;
+ time_limit = time_start + HZ * pdsc->devcmd_timeout;
+ do {
+ /* Timeslice the actual wait to catch IO errors etc early */
+ poll_jiffies = msecs_to_jiffies(poll_interval);
+ remaining = wait_for_completion_timeout(&wc.wait_completion,
+ poll_jiffies);
+ if (remaining)
+ break;
+
+ if (!pdsc_is_fw_running(pdsc)) {
+ u8 fw_status = ioread8(&pdsc->info_regs->fw_status);
+
+ dev_dbg(pdsc->dev, "%s: post wait failed - fw not running %#02x:\n",
+ __func__, fw_status);
+ err = -ENXIO;
+ break;
+ }
+
+ /* When fast_poll is not requested, prevent aggressive polling
+ * on failures due to timeouts by doing exponential back off.
+ */
+ if (!fast_poll && poll_interval < PDSC_ADMINQ_MAX_POLL_INTERVAL)
+ poll_interval <<= 1;
+ } while (time_before(jiffies, time_limit));
+ time_done = jiffies;
+ dev_dbg(pdsc->dev, "%s: elapsed %d msecs\n",
+ __func__, jiffies_to_msecs(time_done - time_start));
+
+ /* Check the results */
+ if (time_after_eq(time_done, time_limit))
+ err = -ETIMEDOUT;
+
+ dev_dbg(pdsc->dev, "read admin queue completion idx %d:\n", index);
+ dynamic_hex_dump("comp ", DUMP_PREFIX_OFFSET, 16, 1,
+ comp, sizeof(*comp), true);
+
+ if (remaining && comp->status)
+ err = pdsc_err_to_errno(comp->status);
+
+err_out:
+ if (err) {
+ dev_dbg(pdsc->dev, "%s: opcode %d status %d err %pe\n",
+ __func__, cmd->opcode, comp->status, ERR_PTR(err));
+ if (err == -ENXIO || err == -ETIMEDOUT)
+ queue_work(pdsc->wq, &pdsc->health_work);
+ }
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(pdsc_adminq_post);
diff --git a/drivers/net/ethernet/amd/pds_core/auxbus.c b/drivers/net/ethernet/amd/pds_core/auxbus.c
new file mode 100644
index 0000000..561af8e
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/auxbus.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/pci.h>
+
+#include "core.h"
+#include <linux/pds/pds_auxbus.h>
+
+/**
+ * pds_client_register - Link the client to the firmware
+ * @pf_pdev: ptr to the PF driver struct
+ * @devname: name that includes service into, e.g. pds_core.vDPA
+ *
+ * Return: 0 on success, or
+ * negative for error
+ */
+int pds_client_register(struct pci_dev *pf_pdev, char *devname)
+{
+ union pds_core_adminq_comp comp = {};
+ union pds_core_adminq_cmd cmd = {};
+ struct pdsc *pf;
+ int err;
+ u16 ci;
+
+ pf = pci_get_drvdata(pf_pdev);
+ if (pf->state)
+ return -ENXIO;
+
+ cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
+ strscpy(cmd.client_reg.devname, devname,
+ sizeof(cmd.client_reg.devname));
+
+ err = pdsc_adminq_post(pf, &cmd, &comp, false);
+ if (err) {
+ dev_info(pf->dev, "register dev_name %s with DSC failed, status %d: %pe\n",
+ devname, comp.status, ERR_PTR(err));
+ return err;
+ }
+
+ ci = le16_to_cpu(comp.client_reg.client_id);
+ if (!ci) {
+ dev_err(pf->dev, "%s: device returned null client_id\n",
+ __func__);
+ return -EIO;
+ }
+
+ dev_dbg(pf->dev, "%s: device returned client_id %d for %s\n",
+ __func__, ci, devname);
+
+ return ci;
+}
+EXPORT_SYMBOL_GPL(pds_client_register);
+
+/**
+ * pds_client_unregister - Unlink the client from the firmware
+ * @pf_pdev: ptr to the PF driver struct
+ * @client_id: id returned from pds_client_register()
+ *
+ * Return: 0 on success, or
+ * negative for error
+ */
+int pds_client_unregister(struct pci_dev *pf_pdev, u16 client_id)
+{
+ union pds_core_adminq_comp comp = {};
+ union pds_core_adminq_cmd cmd = {};
+ struct pdsc *pf;
+ int err;
+
+ pf = pci_get_drvdata(pf_pdev);
+ if (pf->state)
+ return -ENXIO;
+
+ cmd.client_unreg.opcode = PDS_AQ_CMD_CLIENT_UNREG;
+ cmd.client_unreg.client_id = cpu_to_le16(client_id);
+
+ err = pdsc_adminq_post(pf, &cmd, &comp, false);
+ if (err)
+ dev_info(pf->dev, "unregister client_id %d failed, status %d: %pe\n",
+ client_id, comp.status, ERR_PTR(err));
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(pds_client_unregister);
+
+/**
+ * pds_client_adminq_cmd - Process an adminq request for the client
+ * @padev: ptr to the client device
+ * @req: ptr to buffer with request
+ * @req_len: length of actual struct used for request
+ * @resp: ptr to buffer where answer is to be copied
+ * @flags: optional flags from pds_core_adminq_flags
+ *
+ * Return: 0 on success, or
+ * negative for error
+ *
+ * Client sends pointers to request and response buffers
+ * Core copies request data into pds_core_client_request_cmd
+ * Core sets other fields as needed
+ * Core posts to AdminQ
+ * Core copies completion data into response buffer
+ */
+int pds_client_adminq_cmd(struct pds_auxiliary_dev *padev,
+ union pds_core_adminq_cmd *req,
+ size_t req_len,
+ union pds_core_adminq_comp *resp,
+ u64 flags)
+{
+ union pds_core_adminq_cmd cmd = {};
+ struct pci_dev *pf_pdev;
+ struct pdsc *pf;
+ size_t cp_len;
+ int err;
+
+ pf_pdev = pci_physfn(padev->vf_pdev);
+ pf = pci_get_drvdata(pf_pdev);
+
+ dev_dbg(pf->dev, "%s: %s opcode %d\n",
+ __func__, dev_name(&padev->aux_dev.dev), req->opcode);
+
+ if (pf->state)
+ return -ENXIO;
+
+ /* Wrap the client's request */
+ cmd.client_request.opcode = PDS_AQ_CMD_CLIENT_CMD;
+ cmd.client_request.client_id = cpu_to_le16(padev->client_id);
+ cp_len = min_t(size_t, req_len, sizeof(cmd.client_request.client_cmd));
+ memcpy(cmd.client_request.client_cmd, req, cp_len);
+
+ err = pdsc_adminq_post(pf, &cmd, resp,
+ !!(flags & PDS_AQ_FLAG_FASTPOLL));
+ if (err && err != -EAGAIN)
+ dev_info(pf->dev, "client admin cmd failed: %pe\n",
+ ERR_PTR(err));
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(pds_client_adminq_cmd);
+
+static void pdsc_auxbus_dev_release(struct device *dev)
+{
+ struct pds_auxiliary_dev *padev =
+ container_of(dev, struct pds_auxiliary_dev, aux_dev.dev);
+
+ kfree(padev);
+}
+
+static struct pds_auxiliary_dev *pdsc_auxbus_dev_register(struct pdsc *cf,
+ struct pdsc *pf,
+ u16 client_id,
+ char *name)
+{
+ struct auxiliary_device *aux_dev;
+ struct pds_auxiliary_dev *padev;
+ int err;
+
+ padev = kzalloc(sizeof(*padev), GFP_KERNEL);
+ if (!padev)
+ return ERR_PTR(-ENOMEM);
+
+ padev->vf_pdev = cf->pdev;
+ padev->client_id = client_id;
+
+ aux_dev = &padev->aux_dev;
+ aux_dev->name = name;
+ aux_dev->id = cf->uid;
+ aux_dev->dev.parent = cf->dev;
+ aux_dev->dev.release = pdsc_auxbus_dev_release;
+
+ err = auxiliary_device_init(aux_dev);
+ if (err < 0) {
+ dev_warn(cf->dev, "auxiliary_device_init of %s failed: %pe\n",
+ name, ERR_PTR(err));
+ goto err_out;
+ }
+
+ err = auxiliary_device_add(aux_dev);
+ if (err) {
+ dev_warn(cf->dev, "auxiliary_device_add of %s failed: %pe\n",
+ name, ERR_PTR(err));
+ goto err_out_uninit;
+ }
+
+ return padev;
+
+err_out_uninit:
+ auxiliary_device_uninit(aux_dev);
+err_out:
+ kfree(padev);
+ return ERR_PTR(err);
+}
+
+int pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf)
+{
+ struct pds_auxiliary_dev *padev;
+ int err = 0;
+
+ mutex_lock(&pf->config_lock);
+
+ padev = pf->vfs[cf->vf_id].padev;
+ if (padev) {
+ pds_client_unregister(pf->pdev, padev->client_id);
+ auxiliary_device_delete(&padev->aux_dev);
+ auxiliary_device_uninit(&padev->aux_dev);
+ padev->client_id = 0;
+ }
+ pf->vfs[cf->vf_id].padev = NULL;
+
+ mutex_unlock(&pf->config_lock);
+ return err;
+}
+
+int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf)
+{
+ struct pds_auxiliary_dev *padev;
+ enum pds_core_vif_types vt;
+ char devname[PDS_DEVNAME_LEN];
+ u16 vt_support;
+ int client_id;
+ int err = 0;
+
+ mutex_lock(&pf->config_lock);
+
+ /* We only support vDPA so far, so it is the only one to
+ * be verified that it is available in the Core device and
+ * enabled in the devlink param. In the future this might
+ * become a loop for several VIF types.
+ */
+
+ /* Verify that the type is supported and enabled. It is not
+ * an error if there is no auxbus device support for this
+ * VF, it just means something else needs to happen with it.
+ */
+ vt = PDS_DEV_TYPE_VDPA;
+ vt_support = !!le16_to_cpu(pf->dev_ident.vif_types[vt]);
+ if (!(vt_support &&
+ pf->viftype_status[vt].supported &&
+ pf->viftype_status[vt].enabled))
+ goto out_unlock;
+
+ /* Need to register with FW and get the client_id before
+ * creating the aux device so that the aux client can run
+ * adminq commands as part its probe
+ */
+ snprintf(devname, sizeof(devname), "%s.%s.%d",
+ PDS_CORE_DRV_NAME, pf->viftype_status[vt].name, cf->uid);
+ client_id = pds_client_register(pf->pdev, devname);
+ if (client_id < 0) {
+ err = client_id;
+ goto out_unlock;
+ }
+
+ padev = pdsc_auxbus_dev_register(cf, pf, client_id,
+ pf->viftype_status[vt].name);
+ if (IS_ERR(padev)) {
+ pds_client_unregister(pf->pdev, client_id);
+ err = PTR_ERR(padev);
+ goto out_unlock;
+ }
+ pf->vfs[cf->vf_id].padev = padev;
+
+out_unlock:
+ mutex_unlock(&pf->config_lock);
+ return err;
+}
diff --git a/drivers/net/ethernet/amd/pds_core/core.c b/drivers/net/ethernet/amd/pds_core/core.c
new file mode 100644
index 0000000..483a070
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/core.c
@@ -0,0 +1,597 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/pci.h>
+#include <linux/vmalloc.h>
+
+#include "core.h"
+
+static BLOCKING_NOTIFIER_HEAD(pds_notify_chain);
+
+int pdsc_register_notify(struct notifier_block *nb)
+{
+ return blocking_notifier_chain_register(&pds_notify_chain, nb);
+}
+EXPORT_SYMBOL_GPL(pdsc_register_notify);
+
+void pdsc_unregister_notify(struct notifier_block *nb)
+{
+ blocking_notifier_chain_unregister(&pds_notify_chain, nb);
+}
+EXPORT_SYMBOL_GPL(pdsc_unregister_notify);
+
+void pdsc_notify(unsigned long event, void *data)
+{
+ blocking_notifier_call_chain(&pds_notify_chain, event, data);
+}
+
+void pdsc_intr_free(struct pdsc *pdsc, int index)
+{
+ struct pdsc_intr_info *intr_info;
+
+ if (index >= pdsc->nintrs || index < 0) {
+ WARN(true, "bad intr index %d\n", index);
+ return;
+ }
+
+ intr_info = &pdsc->intr_info[index];
+ if (!intr_info->vector)
+ return;
+ dev_dbg(pdsc->dev, "%s: idx %d vec %d name %s\n",
+ __func__, index, intr_info->vector, intr_info->name);
+
+ pds_core_intr_mask(&pdsc->intr_ctrl[index], PDS_CORE_INTR_MASK_SET);
+ pds_core_intr_clean(&pdsc->intr_ctrl[index]);
+
+ free_irq(intr_info->vector, intr_info->data);
+
+ memset(intr_info, 0, sizeof(*intr_info));
+}
+
+int pdsc_intr_alloc(struct pdsc *pdsc, char *name,
+ irq_handler_t handler, void *data)
+{
+ struct pdsc_intr_info *intr_info;
+ unsigned int index;
+ int err;
+
+ /* Find the first available interrupt */
+ for (index = 0; index < pdsc->nintrs; index++)
+ if (!pdsc->intr_info[index].vector)
+ break;
+ if (index >= pdsc->nintrs) {
+ dev_warn(pdsc->dev, "%s: no intr, index=%d nintrs=%d\n",
+ __func__, index, pdsc->nintrs);
+ return -ENOSPC;
+ }
+
+ pds_core_intr_clean_flags(&pdsc->intr_ctrl[index],
+ PDS_CORE_INTR_CRED_RESET_COALESCE);
+
+ intr_info = &pdsc->intr_info[index];
+
+ intr_info->index = index;
+ intr_info->data = data;
+ strscpy(intr_info->name, name, sizeof(intr_info->name));
+
+ /* Get the OS vector number for the interrupt */
+ err = pci_irq_vector(pdsc->pdev, index);
+ if (err < 0) {
+ dev_err(pdsc->dev, "failed to get intr vector index %d: %pe\n",
+ index, ERR_PTR(err));
+ goto err_out_free_intr;
+ }
+ intr_info->vector = err;
+
+ /* Init the device's intr mask */
+ pds_core_intr_clean(&pdsc->intr_ctrl[index]);
+ pds_core_intr_mask_assert(&pdsc->intr_ctrl[index], 1);
+ pds_core_intr_mask(&pdsc->intr_ctrl[index], PDS_CORE_INTR_MASK_SET);
+
+ /* Register the isr with a name */
+ err = request_irq(intr_info->vector, handler, 0, intr_info->name, data);
+ if (err) {
+ dev_err(pdsc->dev, "failed to get intr irq vector %d: %pe\n",
+ intr_info->vector, ERR_PTR(err));
+ goto err_out_free_intr;
+ }
+
+ return index;
+
+err_out_free_intr:
+ pdsc_intr_free(pdsc, index);
+ return err;
+}
+
+static void pdsc_qcq_intr_free(struct pdsc *pdsc, struct pdsc_qcq *qcq)
+{
+ if (!(qcq->flags & PDS_CORE_QCQ_F_INTR) ||
+ qcq->intx == PDS_CORE_INTR_INDEX_NOT_ASSIGNED)
+ return;
+
+ pdsc_intr_free(pdsc, qcq->intx);
+ qcq->intx = PDS_CORE_INTR_INDEX_NOT_ASSIGNED;
+}
+
+static int pdsc_qcq_intr_alloc(struct pdsc *pdsc, struct pdsc_qcq *qcq)
+{
+ char name[PDSC_INTR_NAME_MAX_SZ];
+ int index;
+
+ if (!(qcq->flags & PDS_CORE_QCQ_F_INTR)) {
+ qcq->intx = PDS_CORE_INTR_INDEX_NOT_ASSIGNED;
+ return 0;
+ }
+
+ snprintf(name, sizeof(name), "%s-%d-%s",
+ PDS_CORE_DRV_NAME, pdsc->pdev->bus->number, qcq->q.name);
+ index = pdsc_intr_alloc(pdsc, name, pdsc_adminq_isr, qcq);
+ if (index < 0)
+ return index;
+ qcq->intx = index;
+
+ return 0;
+}
+
+void pdsc_qcq_free(struct pdsc *pdsc, struct pdsc_qcq *qcq)
+{
+ struct device *dev = pdsc->dev;
+
+ if (!(qcq && qcq->pdsc))
+ return;
+
+ pdsc_debugfs_del_qcq(qcq);
+
+ pdsc_qcq_intr_free(pdsc, qcq);
+
+ if (qcq->q_base)
+ dma_free_coherent(dev, qcq->q_size,
+ qcq->q_base, qcq->q_base_pa);
+
+ if (qcq->cq_base)
+ dma_free_coherent(dev, qcq->cq_size,
+ qcq->cq_base, qcq->cq_base_pa);
+
+ if (qcq->cq.info)
+ vfree(qcq->cq.info);
+
+ if (qcq->q.info)
+ vfree(qcq->q.info);
+
+ memset(qcq, 0, sizeof(*qcq));
+}
+
+static void pdsc_q_map(struct pdsc_queue *q, void *base, dma_addr_t base_pa)
+{
+ struct pdsc_q_info *cur;
+ unsigned int i;
+
+ q->base = base;
+ q->base_pa = base_pa;
+
+ for (i = 0, cur = q->info; i < q->num_descs; i++, cur++)
+ cur->desc = base + (i * q->desc_size);
+}
+
+static void pdsc_cq_map(struct pdsc_cq *cq, void *base, dma_addr_t base_pa)
+{
+ struct pdsc_cq_info *cur;
+ unsigned int i;
+
+ cq->base = base;
+ cq->base_pa = base_pa;
+
+ for (i = 0, cur = cq->info; i < cq->num_descs; i++, cur++)
+ cur->comp = base + (i * cq->desc_size);
+}
+
+int pdsc_qcq_alloc(struct pdsc *pdsc, unsigned int type, unsigned int index,
+ const char *name, unsigned int flags, unsigned int num_descs,
+ unsigned int desc_size, unsigned int cq_desc_size,
+ unsigned int pid, struct pdsc_qcq *qcq)
+{
+ struct device *dev = pdsc->dev;
+ void *q_base, *cq_base;
+ dma_addr_t cq_base_pa;
+ dma_addr_t q_base_pa;
+ int err;
+
+ qcq->q.info = vzalloc(num_descs * sizeof(*qcq->q.info));
+ if (!qcq->q.info) {
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ qcq->pdsc = pdsc;
+ qcq->flags = flags;
+ INIT_WORK(&qcq->work, pdsc_work_thread);
+
+ qcq->q.type = type;
+ qcq->q.index = index;
+ qcq->q.num_descs = num_descs;
+ qcq->q.desc_size = desc_size;
+ qcq->q.tail_idx = 0;
+ qcq->q.head_idx = 0;
+ qcq->q.pid = pid;
+ snprintf(qcq->q.name, sizeof(qcq->q.name), "%s%u", name, index);
+
+ err = pdsc_qcq_intr_alloc(pdsc, qcq);
+ if (err)
+ goto err_out_free_q_info;
+
+ qcq->cq.info = vzalloc(num_descs * sizeof(*qcq->cq.info));
+ if (!qcq->cq.info) {
+ err = -ENOMEM;
+ goto err_out_free_irq;
+ }
+
+ qcq->cq.bound_intr = &pdsc->intr_info[qcq->intx];
+ qcq->cq.num_descs = num_descs;
+ qcq->cq.desc_size = cq_desc_size;
+ qcq->cq.tail_idx = 0;
+ qcq->cq.done_color = 1;
+
+ if (flags & PDS_CORE_QCQ_F_NOTIFYQ) {
+ /* q & cq need to be contiguous in case of notifyq */
+ qcq->q_size = PDS_PAGE_SIZE +
+ ALIGN(num_descs * desc_size, PDS_PAGE_SIZE) +
+ ALIGN(num_descs * cq_desc_size, PDS_PAGE_SIZE);
+ qcq->q_base = dma_alloc_coherent(dev,
+ qcq->q_size + qcq->cq_size,
+ &qcq->q_base_pa,
+ GFP_KERNEL);
+ if (!qcq->q_base) {
+ err = -ENOMEM;
+ goto err_out_free_cq_info;
+ }
+ q_base = PTR_ALIGN(qcq->q_base, PDS_PAGE_SIZE);
+ q_base_pa = ALIGN(qcq->q_base_pa, PDS_PAGE_SIZE);
+ pdsc_q_map(&qcq->q, q_base, q_base_pa);
+
+ cq_base = PTR_ALIGN(q_base +
+ ALIGN(num_descs * desc_size, PDS_PAGE_SIZE),
+ PDS_PAGE_SIZE);
+ cq_base_pa = ALIGN(qcq->q_base_pa +
+ ALIGN(num_descs * desc_size, PDS_PAGE_SIZE),
+ PDS_PAGE_SIZE);
+
+ } else {
+ /* q DMA descriptors */
+ qcq->q_size = PDS_PAGE_SIZE + (num_descs * desc_size);
+ qcq->q_base = dma_alloc_coherent(dev, qcq->q_size,
+ &qcq->q_base_pa,
+ GFP_KERNEL);
+ if (!qcq->q_base) {
+ err = -ENOMEM;
+ goto err_out_free_cq_info;
+ }
+ q_base = PTR_ALIGN(qcq->q_base, PDS_PAGE_SIZE);
+ q_base_pa = ALIGN(qcq->q_base_pa, PDS_PAGE_SIZE);
+ pdsc_q_map(&qcq->q, q_base, q_base_pa);
+
+ /* cq DMA descriptors */
+ qcq->cq_size = PDS_PAGE_SIZE + (num_descs * cq_desc_size);
+ qcq->cq_base = dma_alloc_coherent(dev, qcq->cq_size,
+ &qcq->cq_base_pa,
+ GFP_KERNEL);
+ if (!qcq->cq_base) {
+ err = -ENOMEM;
+ goto err_out_free_q;
+ }
+ cq_base = PTR_ALIGN(qcq->cq_base, PDS_PAGE_SIZE);
+ cq_base_pa = ALIGN(qcq->cq_base_pa, PDS_PAGE_SIZE);
+ }
+
+ pdsc_cq_map(&qcq->cq, cq_base, cq_base_pa);
+ qcq->cq.bound_q = &qcq->q;
+
+ pdsc_debugfs_add_qcq(pdsc, qcq);
+
+ return 0;
+
+err_out_free_q:
+ dma_free_coherent(dev, qcq->q_size, qcq->q_base, qcq->q_base_pa);
+err_out_free_cq_info:
+ vfree(qcq->cq.info);
+err_out_free_irq:
+ pdsc_qcq_intr_free(pdsc, qcq);
+err_out_free_q_info:
+ vfree(qcq->q.info);
+ memset(qcq, 0, sizeof(*qcq));
+err_out:
+ dev_err(dev, "qcq alloc of %s%d failed %d\n", name, index, err);
+ return err;
+}
+
+static int pdsc_core_init(struct pdsc *pdsc)
+{
+ union pds_core_dev_comp comp = {};
+ union pds_core_dev_cmd cmd = {
+ .init.opcode = PDS_CORE_CMD_INIT,
+ };
+ struct pds_core_dev_init_data_out cido;
+ struct pds_core_dev_init_data_in cidi;
+ u32 dbid_count;
+ u32 dbpage_num;
+ size_t sz;
+ int err;
+
+ cidi.adminq_q_base = cpu_to_le64(pdsc->adminqcq.q_base_pa);
+ cidi.adminq_cq_base = cpu_to_le64(pdsc->adminqcq.cq_base_pa);
+ cidi.notifyq_cq_base = cpu_to_le64(pdsc->notifyqcq.cq.base_pa);
+ cidi.flags = cpu_to_le32(PDS_CORE_QINIT_F_IRQ | PDS_CORE_QINIT_F_ENA);
+ cidi.intr_index = cpu_to_le16(pdsc->adminqcq.intx);
+ cidi.adminq_ring_size = ilog2(pdsc->adminqcq.q.num_descs);
+ cidi.notifyq_ring_size = ilog2(pdsc->notifyqcq.q.num_descs);
+
+ mutex_lock(&pdsc->devcmd_lock);
+
+ sz = min_t(size_t, sizeof(cidi), sizeof(pdsc->cmd_regs->data));
+ memcpy_toio(&pdsc->cmd_regs->data, &cidi, sz);
+
+ err = pdsc_devcmd_locked(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+ if (!err) {
+ sz = min_t(size_t, sizeof(cido), sizeof(pdsc->cmd_regs->data));
+ memcpy_fromio(&cido, &pdsc->cmd_regs->data, sz);
+ }
+
+ mutex_unlock(&pdsc->devcmd_lock);
+ if (err) {
+ dev_err(pdsc->dev, "Device init command failed: %pe\n",
+ ERR_PTR(err));
+ return err;
+ }
+
+ pdsc->hw_index = le32_to_cpu(cido.core_hw_index);
+
+ dbid_count = le32_to_cpu(pdsc->dev_ident.ndbpgs_per_lif);
+ dbpage_num = pdsc->hw_index * dbid_count;
+ pdsc->kern_dbpage = pdsc_map_dbpage(pdsc, dbpage_num);
+ if (!pdsc->kern_dbpage) {
+ dev_err(pdsc->dev, "Cannot map dbpage, aborting\n");
+ return -ENOMEM;
+ }
+
+ pdsc->adminqcq.q.hw_type = cido.adminq_hw_type;
+ pdsc->adminqcq.q.hw_index = le32_to_cpu(cido.adminq_hw_index);
+ pdsc->adminqcq.q.dbval = PDS_CORE_DBELL_QID(pdsc->adminqcq.q.hw_index);
+
+ pdsc->notifyqcq.q.hw_type = cido.notifyq_hw_type;
+ pdsc->notifyqcq.q.hw_index = le32_to_cpu(cido.notifyq_hw_index);
+ pdsc->notifyqcq.q.dbval = PDS_CORE_DBELL_QID(pdsc->notifyqcq.q.hw_index);
+
+ pdsc->last_eid = 0;
+
+ return err;
+}
+
+static struct pdsc_viftype pdsc_viftype_defaults[] = {
+ [PDS_DEV_TYPE_VDPA] = { .name = PDS_DEV_TYPE_VDPA_STR,
+ .vif_id = PDS_DEV_TYPE_VDPA,
+ .dl_id = DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET },
+ [PDS_DEV_TYPE_MAX] = {}
+};
+
+static int pdsc_viftypes_init(struct pdsc *pdsc)
+{
+ enum pds_core_vif_types vt;
+
+ pdsc->viftype_status = kzalloc(sizeof(pdsc_viftype_defaults),
+ GFP_KERNEL);
+ if (!pdsc->viftype_status)
+ return -ENOMEM;
+
+ for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++) {
+ bool vt_support;
+
+ if (!pdsc_viftype_defaults[vt].name)
+ continue;
+
+ /* Grab the defaults */
+ pdsc->viftype_status[vt] = pdsc_viftype_defaults[vt];
+
+ /* See what the Core device has for support */
+ vt_support = !!le16_to_cpu(pdsc->dev_ident.vif_types[vt]);
+ dev_dbg(pdsc->dev, "VIF %s is %ssupported\n",
+ pdsc->viftype_status[vt].name,
+ vt_support ? "" : "not ");
+
+ pdsc->viftype_status[vt].supported = vt_support;
+ }
+
+ return 0;
+}
+
+int pdsc_setup(struct pdsc *pdsc, bool init)
+{
+ int numdescs;
+ int err;
+
+ if (init)
+ err = pdsc_dev_init(pdsc);
+ else
+ err = pdsc_dev_reinit(pdsc);
+ if (err)
+ return err;
+
+ /* Scale the descriptor ring length based on number of CPUs and VFs */
+ numdescs = max_t(int, PDSC_ADMINQ_MIN_LENGTH, num_online_cpus());
+ numdescs += 2 * pci_sriov_get_totalvfs(pdsc->pdev);
+ numdescs = roundup_pow_of_two(numdescs);
+ err = pdsc_qcq_alloc(pdsc, PDS_CORE_QTYPE_ADMINQ, 0, "adminq",
+ PDS_CORE_QCQ_F_CORE | PDS_CORE_QCQ_F_INTR,
+ numdescs,
+ sizeof(union pds_core_adminq_cmd),
+ sizeof(union pds_core_adminq_comp),
+ 0, &pdsc->adminqcq);
+ if (err)
+ goto err_out_teardown;
+
+ err = pdsc_qcq_alloc(pdsc, PDS_CORE_QTYPE_NOTIFYQ, 0, "notifyq",
+ PDS_CORE_QCQ_F_NOTIFYQ,
+ PDSC_NOTIFYQ_LENGTH,
+ sizeof(struct pds_core_notifyq_cmd),
+ sizeof(union pds_core_notifyq_comp),
+ 0, &pdsc->notifyqcq);
+ if (err)
+ goto err_out_teardown;
+
+ /* NotifyQ rides on the AdminQ interrupt */
+ pdsc->notifyqcq.intx = pdsc->adminqcq.intx;
+
+ /* Set up the Core with the AdminQ and NotifyQ info */
+ err = pdsc_core_init(pdsc);
+ if (err)
+ goto err_out_teardown;
+
+ /* Set up the VIFs */
+ err = pdsc_viftypes_init(pdsc);
+ if (err)
+ goto err_out_teardown;
+
+ if (init)
+ pdsc_debugfs_add_viftype(pdsc);
+
+ clear_bit(PDSC_S_FW_DEAD, &pdsc->state);
+ return 0;
+
+err_out_teardown:
+ pdsc_teardown(pdsc, init);
+ return err;
+}
+
+void pdsc_teardown(struct pdsc *pdsc, bool removing)
+{
+ int i;
+
+ pdsc_devcmd_reset(pdsc);
+ pdsc_qcq_free(pdsc, &pdsc->notifyqcq);
+ pdsc_qcq_free(pdsc, &pdsc->adminqcq);
+
+ kfree(pdsc->viftype_status);
+ pdsc->viftype_status = NULL;
+
+ if (pdsc->intr_info) {
+ for (i = 0; i < pdsc->nintrs; i++)
+ pdsc_intr_free(pdsc, i);
+
+ if (removing) {
+ kfree(pdsc->intr_info);
+ pdsc->intr_info = NULL;
+ }
+ }
+
+ if (pdsc->kern_dbpage) {
+ iounmap(pdsc->kern_dbpage);
+ pdsc->kern_dbpage = NULL;
+ }
+
+ set_bit(PDSC_S_FW_DEAD, &pdsc->state);
+}
+
+int pdsc_start(struct pdsc *pdsc)
+{
+ pds_core_intr_mask(&pdsc->intr_ctrl[pdsc->adminqcq.intx],
+ PDS_CORE_INTR_MASK_CLEAR);
+
+ return 0;
+}
+
+void pdsc_stop(struct pdsc *pdsc)
+{
+ int i;
+
+ if (!pdsc->intr_info)
+ return;
+
+ /* Mask interrupts that are in use */
+ for (i = 0; i < pdsc->nintrs; i++)
+ if (pdsc->intr_info[i].vector)
+ pds_core_intr_mask(&pdsc->intr_ctrl[i],
+ PDS_CORE_INTR_MASK_SET);
+}
+
+static void pdsc_fw_down(struct pdsc *pdsc)
+{
+ union pds_core_notifyq_comp reset_event = {
+ .reset.ecode = cpu_to_le16(PDS_EVENT_RESET),
+ .reset.state = 0,
+ };
+
+ if (test_and_set_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+ dev_err(pdsc->dev, "%s: already happening\n", __func__);
+ return;
+ }
+
+ /* Notify clients of fw_down */
+ devlink_health_report(pdsc->fw_reporter, "FW down reported", pdsc);
+ pdsc_notify(PDS_EVENT_RESET, &reset_event);
+
+ pdsc_stop(pdsc);
+ pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
+}
+
+static void pdsc_fw_up(struct pdsc *pdsc)
+{
+ union pds_core_notifyq_comp reset_event = {
+ .reset.ecode = cpu_to_le16(PDS_EVENT_RESET),
+ .reset.state = 1,
+ };
+ int err;
+
+ if (!test_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+ dev_err(pdsc->dev, "%s: fw not dead\n", __func__);
+ return;
+ }
+
+ err = pdsc_setup(pdsc, PDSC_SETUP_RECOVERY);
+ if (err)
+ goto err_out;
+
+ err = pdsc_start(pdsc);
+ if (err)
+ goto err_out;
+
+ /* Notify clients of fw_up */
+ pdsc->fw_recoveries++;
+ devlink_health_reporter_state_update(pdsc->fw_reporter,
+ DEVLINK_HEALTH_REPORTER_STATE_HEALTHY);
+ pdsc_notify(PDS_EVENT_RESET, &reset_event);
+
+ return;
+
+err_out:
+ pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
+}
+
+void pdsc_health_thread(struct work_struct *work)
+{
+ struct pdsc *pdsc = container_of(work, struct pdsc, health_work);
+ unsigned long mask;
+ bool healthy;
+
+ mutex_lock(&pdsc->config_lock);
+
+ /* Don't do a check when in a transition state */
+ mask = BIT_ULL(PDSC_S_INITING_DRIVER) |
+ BIT_ULL(PDSC_S_STOPPING_DRIVER);
+ if (pdsc->state & mask)
+ goto out_unlock;
+
+ healthy = pdsc_is_fw_good(pdsc);
+ dev_dbg(pdsc->dev, "%s: health %d fw_status %#02x fw_heartbeat %d\n",
+ __func__, healthy, pdsc->fw_status, pdsc->last_hb);
+
+ if (test_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+ if (healthy)
+ pdsc_fw_up(pdsc);
+ } else {
+ if (!healthy)
+ pdsc_fw_down(pdsc);
+ }
+
+ pdsc->fw_generation = pdsc->fw_status & PDS_CORE_FW_STS_F_GENERATION;
+
+out_unlock:
+ mutex_unlock(&pdsc->config_lock);
+}
diff --git a/drivers/net/ethernet/amd/pds_core/core.h b/drivers/net/ethernet/amd/pds_core/core.h
new file mode 100644
index 0000000..e545faf
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/core.h
@@ -0,0 +1,312 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#ifndef _PDSC_H_
+#define _PDSC_H_
+
+#include <linux/debugfs.h>
+#include <net/devlink.h>
+
+#include <linux/pds/pds_common.h>
+#include <linux/pds/pds_core_if.h>
+#include <linux/pds/pds_adminq.h>
+#include <linux/pds/pds_intr.h>
+
+#define PDSC_DRV_DESCRIPTION "AMD/Pensando Core Driver"
+
+#define PDSC_WATCHDOG_SECS 5
+#define PDSC_QUEUE_NAME_MAX_SZ 32
+#define PDSC_ADMINQ_MIN_LENGTH 16 /* must be a power of two */
+#define PDSC_NOTIFYQ_LENGTH 64 /* must be a power of two */
+#define PDSC_TEARDOWN_RECOVERY false
+#define PDSC_TEARDOWN_REMOVING true
+#define PDSC_SETUP_RECOVERY false
+#define PDSC_SETUP_INIT true
+
+struct pdsc_dev_bar {
+ void __iomem *vaddr;
+ phys_addr_t bus_addr;
+ unsigned long len;
+ int res_index;
+};
+
+struct pdsc;
+
+struct pdsc_vf {
+ struct pds_auxiliary_dev *padev;
+ struct pdsc *vf;
+ u16 index;
+ __le16 vif_types[PDS_DEV_TYPE_MAX];
+};
+
+struct pdsc_devinfo {
+ u8 asic_type;
+ u8 asic_rev;
+ char fw_version[PDS_CORE_DEVINFO_FWVERS_BUFLEN + 1];
+ char serial_num[PDS_CORE_DEVINFO_SERIAL_BUFLEN + 1];
+};
+
+struct pdsc_queue {
+ struct pdsc_q_info *info;
+ u64 dbval;
+ u16 head_idx;
+ u16 tail_idx;
+ u8 hw_type;
+ unsigned int index;
+ unsigned int num_descs;
+ u64 dbell_count;
+ u64 features;
+ unsigned int type;
+ unsigned int hw_index;
+ union {
+ void *base;
+ struct pds_core_admin_cmd *adminq;
+ };
+ dma_addr_t base_pa; /* must be page aligned */
+ unsigned int desc_size;
+ unsigned int pid;
+ char name[PDSC_QUEUE_NAME_MAX_SZ];
+};
+
+#define PDSC_INTR_NAME_MAX_SZ 32
+
+struct pdsc_intr_info {
+ char name[PDSC_INTR_NAME_MAX_SZ];
+ unsigned int index;
+ unsigned int vector;
+ void *data;
+};
+
+struct pdsc_cq_info {
+ void *comp;
+};
+
+struct pdsc_buf_info {
+ struct page *page;
+ dma_addr_t dma_addr;
+ u32 page_offset;
+ u32 len;
+};
+
+struct pdsc_q_info {
+ union {
+ void *desc;
+ struct pdsc_admin_cmd *adminq_desc;
+ };
+ unsigned int bytes;
+ unsigned int nbufs;
+ struct pdsc_buf_info bufs[PDS_CORE_MAX_FRAGS];
+ struct pdsc_wait_context *wc;
+ void *dest;
+};
+
+struct pdsc_cq {
+ struct pdsc_cq_info *info;
+ struct pdsc_queue *bound_q;
+ struct pdsc_intr_info *bound_intr;
+ u16 tail_idx;
+ bool done_color;
+ unsigned int num_descs;
+ unsigned int desc_size;
+ void *base;
+ dma_addr_t base_pa; /* must be page aligned */
+} ____cacheline_aligned_in_smp;
+
+struct pdsc_qcq {
+ struct pdsc *pdsc;
+ void *q_base;
+ dma_addr_t q_base_pa; /* might not be page aligned */
+ void *cq_base;
+ dma_addr_t cq_base_pa; /* might not be page aligned */
+ u32 q_size;
+ u32 cq_size;
+ bool armed;
+ unsigned int flags;
+
+ struct work_struct work;
+ struct pdsc_queue q;
+ struct pdsc_cq cq;
+ int intx;
+
+ u32 accum_work;
+ struct dentry *dentry;
+};
+
+struct pdsc_viftype {
+ char *name;
+ bool supported;
+ bool enabled;
+ int dl_id;
+ int vif_id;
+ struct pds_auxiliary_dev *padev;
+};
+
+/* No state flags set means we are in a steady running state */
+enum pdsc_state_flags {
+ PDSC_S_FW_DEAD, /* stopped, wait on startup or recovery */
+ PDSC_S_INITING_DRIVER, /* initial startup from probe */
+ PDSC_S_STOPPING_DRIVER, /* driver remove */
+
+ /* leave this as last */
+ PDSC_S_STATE_SIZE
+};
+
+struct pdsc {
+ struct pci_dev *pdev;
+ struct dentry *dentry;
+ struct device *dev;
+ struct pdsc_dev_bar bars[PDS_CORE_BARS_MAX];
+ struct pdsc_vf *vfs;
+ int num_vfs;
+ int vf_id;
+ int hw_index;
+ int uid;
+
+ unsigned long state;
+ u8 fw_status;
+ u8 fw_generation;
+ unsigned long last_fw_time;
+ u32 last_hb;
+ struct timer_list wdtimer;
+ unsigned int wdtimer_period;
+ struct work_struct health_work;
+ struct devlink_health_reporter *fw_reporter;
+ u32 fw_recoveries;
+
+ struct pdsc_devinfo dev_info;
+ struct pds_core_dev_identity dev_ident;
+ unsigned int nintrs;
+ struct pdsc_intr_info *intr_info; /* array of nintrs elements */
+
+ struct workqueue_struct *wq;
+
+ unsigned int devcmd_timeout;
+ struct mutex devcmd_lock; /* lock for dev_cmd operations */
+ struct mutex config_lock; /* lock for configuration operations */
+ spinlock_t adminq_lock; /* lock for adminq operations */
+ struct pds_core_dev_info_regs __iomem *info_regs;
+ struct pds_core_dev_cmd_regs __iomem *cmd_regs;
+ struct pds_core_intr __iomem *intr_ctrl;
+ u64 __iomem *intr_status;
+ u64 __iomem *db_pages;
+ dma_addr_t phy_db_pages;
+ u64 __iomem *kern_dbpage;
+
+ struct pdsc_qcq adminqcq;
+ struct pdsc_qcq notifyqcq;
+ u64 last_eid;
+ struct pdsc_viftype *viftype_status;
+};
+
+/** enum pds_core_dbell_bits - bitwise composition of dbell values.
+ *
+ * @PDS_CORE_DBELL_QID_MASK: unshifted mask of valid queue id bits.
+ * @PDS_CORE_DBELL_QID_SHIFT: queue id shift amount in dbell value.
+ * @PDS_CORE_DBELL_QID: macro to build QID component of dbell value.
+ *
+ * @PDS_CORE_DBELL_RING_MASK: unshifted mask of valid ring bits.
+ * @PDS_CORE_DBELL_RING_SHIFT: ring shift amount in dbell value.
+ * @PDS_CORE_DBELL_RING: macro to build ring component of dbell value.
+ *
+ * @PDS_CORE_DBELL_RING_0: ring zero dbell component value.
+ * @PDS_CORE_DBELL_RING_1: ring one dbell component value.
+ * @PDS_CORE_DBELL_RING_2: ring two dbell component value.
+ * @PDS_CORE_DBELL_RING_3: ring three dbell component value.
+ *
+ * @PDS_CORE_DBELL_INDEX_MASK: bit mask of valid index bits, no shift needed.
+ */
+enum pds_core_dbell_bits {
+ PDS_CORE_DBELL_QID_MASK = 0xffffff,
+ PDS_CORE_DBELL_QID_SHIFT = 24,
+
+#define PDS_CORE_DBELL_QID(n) \
+ (((u64)(n) & PDS_CORE_DBELL_QID_MASK) << PDS_CORE_DBELL_QID_SHIFT)
+
+ PDS_CORE_DBELL_RING_MASK = 0x7,
+ PDS_CORE_DBELL_RING_SHIFT = 16,
+
+#define PDS_CORE_DBELL_RING(n) \
+ (((u64)(n) & PDS_CORE_DBELL_RING_MASK) << PDS_CORE_DBELL_RING_SHIFT)
+
+ PDS_CORE_DBELL_RING_0 = 0,
+ PDS_CORE_DBELL_RING_1 = PDS_CORE_DBELL_RING(1),
+ PDS_CORE_DBELL_RING_2 = PDS_CORE_DBELL_RING(2),
+ PDS_CORE_DBELL_RING_3 = PDS_CORE_DBELL_RING(3),
+
+ PDS_CORE_DBELL_INDEX_MASK = 0xffff,
+};
+
+static inline void pds_core_dbell_ring(u64 __iomem *db_page,
+ enum pds_core_logical_qtype qtype,
+ u64 val)
+{
+ writeq(val, &db_page[qtype]);
+}
+
+int pdsc_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
+ struct devlink_fmsg *fmsg,
+ struct netlink_ext_ack *extack);
+int pdsc_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+ struct netlink_ext_ack *extack);
+int pdsc_dl_flash_update(struct devlink *dl,
+ struct devlink_flash_update_params *params,
+ struct netlink_ext_ack *extack);
+int pdsc_dl_enable_get(struct devlink *dl, u32 id,
+ struct devlink_param_gset_ctx *ctx);
+int pdsc_dl_enable_set(struct devlink *dl, u32 id,
+ struct devlink_param_gset_ctx *ctx);
+int pdsc_dl_enable_validate(struct devlink *dl, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack);
+
+void __iomem *pdsc_map_dbpage(struct pdsc *pdsc, int page_num);
+
+void pdsc_debugfs_create(void);
+void pdsc_debugfs_destroy(void);
+void pdsc_debugfs_add_dev(struct pdsc *pdsc);
+void pdsc_debugfs_del_dev(struct pdsc *pdsc);
+void pdsc_debugfs_add_ident(struct pdsc *pdsc);
+void pdsc_debugfs_add_viftype(struct pdsc *pdsc);
+void pdsc_debugfs_add_irqs(struct pdsc *pdsc);
+void pdsc_debugfs_add_qcq(struct pdsc *pdsc, struct pdsc_qcq *qcq);
+void pdsc_debugfs_del_qcq(struct pdsc_qcq *qcq);
+
+int pdsc_err_to_errno(enum pds_core_status_code code);
+bool pdsc_is_fw_running(struct pdsc *pdsc);
+bool pdsc_is_fw_good(struct pdsc *pdsc);
+int pdsc_devcmd(struct pdsc *pdsc, union pds_core_dev_cmd *cmd,
+ union pds_core_dev_comp *comp, int max_seconds);
+int pdsc_devcmd_locked(struct pdsc *pdsc, union pds_core_dev_cmd *cmd,
+ union pds_core_dev_comp *comp, int max_seconds);
+int pdsc_devcmd_init(struct pdsc *pdsc);
+int pdsc_devcmd_reset(struct pdsc *pdsc);
+int pdsc_dev_reinit(struct pdsc *pdsc);
+int pdsc_dev_init(struct pdsc *pdsc);
+
+int pdsc_intr_alloc(struct pdsc *pdsc, char *name,
+ irq_handler_t handler, void *data);
+void pdsc_intr_free(struct pdsc *pdsc, int index);
+void pdsc_qcq_free(struct pdsc *pdsc, struct pdsc_qcq *qcq);
+int pdsc_qcq_alloc(struct pdsc *pdsc, unsigned int type, unsigned int index,
+ const char *name, unsigned int flags, unsigned int num_descs,
+ unsigned int desc_size, unsigned int cq_desc_size,
+ unsigned int pid, struct pdsc_qcq *qcq);
+int pdsc_setup(struct pdsc *pdsc, bool init);
+void pdsc_teardown(struct pdsc *pdsc, bool removing);
+int pdsc_start(struct pdsc *pdsc);
+void pdsc_stop(struct pdsc *pdsc);
+void pdsc_health_thread(struct work_struct *work);
+
+int pdsc_register_notify(struct notifier_block *nb);
+void pdsc_unregister_notify(struct notifier_block *nb);
+void pdsc_notify(unsigned long event, void *data);
+int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf);
+int pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf);
+
+void pdsc_process_adminq(struct pdsc_qcq *qcq);
+void pdsc_work_thread(struct work_struct *work);
+irqreturn_t pdsc_adminq_isr(int irq, void *data);
+
+int pdsc_firmware_update(struct pdsc *pdsc, const struct firmware *fw,
+ struct netlink_ext_ack *extack);
+#endif /* _PDSC_H_ */
diff --git a/drivers/net/ethernet/amd/pds_core/debugfs.c b/drivers/net/ethernet/amd/pds_core/debugfs.c
new file mode 100644
index 0000000..8ec3922
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/debugfs.c
@@ -0,0 +1,170 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/pci.h>
+
+#include "core.h"
+
+static struct dentry *pdsc_dir;
+
+void pdsc_debugfs_create(void)
+{
+ pdsc_dir = debugfs_create_dir(PDS_CORE_DRV_NAME, NULL);
+}
+
+void pdsc_debugfs_destroy(void)
+{
+ debugfs_remove_recursive(pdsc_dir);
+}
+
+void pdsc_debugfs_add_dev(struct pdsc *pdsc)
+{
+ pdsc->dentry = debugfs_create_dir(pci_name(pdsc->pdev), pdsc_dir);
+
+ debugfs_create_ulong("state", 0400, pdsc->dentry, &pdsc->state);
+}
+
+void pdsc_debugfs_del_dev(struct pdsc *pdsc)
+{
+ debugfs_remove_recursive(pdsc->dentry);
+ pdsc->dentry = NULL;
+}
+
+static int identity_show(struct seq_file *seq, void *v)
+{
+ struct pdsc *pdsc = seq->private;
+ struct pds_core_dev_identity *ident;
+ int vt;
+
+ ident = &pdsc->dev_ident;
+
+ seq_printf(seq, "fw_heartbeat: 0x%x\n",
+ ioread32(&pdsc->info_regs->fw_heartbeat));
+
+ seq_printf(seq, "nlifs: %d\n",
+ le32_to_cpu(ident->nlifs));
+ seq_printf(seq, "nintrs: %d\n",
+ le32_to_cpu(ident->nintrs));
+ seq_printf(seq, "ndbpgs_per_lif: %d\n",
+ le32_to_cpu(ident->ndbpgs_per_lif));
+ seq_printf(seq, "intr_coal_mult: %d\n",
+ le32_to_cpu(ident->intr_coal_mult));
+ seq_printf(seq, "intr_coal_div: %d\n",
+ le32_to_cpu(ident->intr_coal_div));
+
+ seq_puts(seq, "vif_types: ");
+ for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++)
+ seq_printf(seq, "%d ",
+ le16_to_cpu(pdsc->dev_ident.vif_types[vt]));
+ seq_puts(seq, "\n");
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(identity);
+
+void pdsc_debugfs_add_ident(struct pdsc *pdsc)
+{
+ debugfs_create_file("identity", 0400, pdsc->dentry,
+ pdsc, &identity_fops);
+}
+
+static int viftype_show(struct seq_file *seq, void *v)
+{
+ struct pdsc *pdsc = seq->private;
+ int vt;
+
+ for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++) {
+ if (!pdsc->viftype_status[vt].name)
+ continue;
+
+ seq_printf(seq, "%s\t%d supported %d enabled\n",
+ pdsc->viftype_status[vt].name,
+ pdsc->viftype_status[vt].supported,
+ pdsc->viftype_status[vt].enabled);
+ }
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(viftype);
+
+void pdsc_debugfs_add_viftype(struct pdsc *pdsc)
+{
+ debugfs_create_file("viftypes", 0400, pdsc->dentry,
+ pdsc, &viftype_fops);
+}
+
+static const struct debugfs_reg32 intr_ctrl_regs[] = {
+ { .name = "coal_init", .offset = 0, },
+ { .name = "mask", .offset = 4, },
+ { .name = "credits", .offset = 8, },
+ { .name = "mask_on_assert", .offset = 12, },
+ { .name = "coal_timer", .offset = 16, },
+};
+
+void pdsc_debugfs_add_qcq(struct pdsc *pdsc, struct pdsc_qcq *qcq)
+{
+ struct dentry *qcq_dentry, *q_dentry, *cq_dentry;
+ struct dentry *intr_dentry;
+ struct debugfs_regset32 *intr_ctrl_regset;
+ struct pdsc_intr_info *intr = &pdsc->intr_info[qcq->intx];
+ struct pdsc_queue *q = &qcq->q;
+ struct pdsc_cq *cq = &qcq->cq;
+
+ qcq_dentry = debugfs_create_dir(q->name, pdsc->dentry);
+ if (IS_ERR_OR_NULL(qcq_dentry))
+ return;
+ qcq->dentry = qcq_dentry;
+
+ debugfs_create_x64("q_base_pa", 0400, qcq_dentry, &qcq->q_base_pa);
+ debugfs_create_x32("q_size", 0400, qcq_dentry, &qcq->q_size);
+ debugfs_create_x64("cq_base_pa", 0400, qcq_dentry, &qcq->cq_base_pa);
+ debugfs_create_x32("cq_size", 0400, qcq_dentry, &qcq->cq_size);
+ debugfs_create_x32("accum_work", 0400, qcq_dentry, &qcq->accum_work);
+
+ q_dentry = debugfs_create_dir("q", qcq->dentry);
+ if (IS_ERR_OR_NULL(q_dentry))
+ return;
+
+ debugfs_create_u32("index", 0400, q_dentry, &q->index);
+ debugfs_create_u32("num_descs", 0400, q_dentry, &q->num_descs);
+ debugfs_create_u32("desc_size", 0400, q_dentry, &q->desc_size);
+ debugfs_create_u32("pid", 0400, q_dentry, &q->pid);
+
+ debugfs_create_u16("tail", 0400, q_dentry, &q->tail_idx);
+ debugfs_create_u16("head", 0400, q_dentry, &q->head_idx);
+
+ cq_dentry = debugfs_create_dir("cq", qcq->dentry);
+ if (IS_ERR_OR_NULL(cq_dentry))
+ return;
+
+ debugfs_create_x64("base_pa", 0400, cq_dentry, &cq->base_pa);
+ debugfs_create_u32("num_descs", 0400, cq_dentry, &cq->num_descs);
+ debugfs_create_u32("desc_size", 0400, cq_dentry, &cq->desc_size);
+ debugfs_create_bool("done_color", 0400, cq_dentry, &cq->done_color);
+ debugfs_create_u16("tail", 0400, cq_dentry, &cq->tail_idx);
+
+ if (qcq->flags & PDS_CORE_QCQ_F_INTR) {
+ intr_dentry = debugfs_create_dir("intr", qcq->dentry);
+ if (IS_ERR_OR_NULL(intr_dentry))
+ return;
+
+ debugfs_create_u32("index", 0400, intr_dentry, &intr->index);
+ debugfs_create_u32("vector", 0400, intr_dentry, &intr->vector);
+
+ intr_ctrl_regset = kzalloc(sizeof(*intr_ctrl_regset),
+ GFP_KERNEL);
+ if (!intr_ctrl_regset)
+ return;
+ intr_ctrl_regset->regs = intr_ctrl_regs;
+ intr_ctrl_regset->nregs = ARRAY_SIZE(intr_ctrl_regs);
+ intr_ctrl_regset->base = &pdsc->intr_ctrl[intr->index];
+
+ debugfs_create_regset32("intr_ctrl", 0400, intr_dentry,
+ intr_ctrl_regset);
+ }
+};
+
+void pdsc_debugfs_del_qcq(struct pdsc_qcq *qcq)
+{
+ debugfs_remove_recursive(qcq->dentry);
+ qcq->dentry = NULL;
+}
diff --git a/drivers/net/ethernet/amd/pds_core/dev.c b/drivers/net/ethernet/amd/pds_core/dev.c
new file mode 100644
index 0000000..f7c597e
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/dev.c
@@ -0,0 +1,351 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/utsname.h>
+
+#include "core.h"
+
+int pdsc_err_to_errno(enum pds_core_status_code code)
+{
+ switch (code) {
+ case PDS_RC_SUCCESS:
+ return 0;
+ case PDS_RC_EVERSION:
+ case PDS_RC_EQTYPE:
+ case PDS_RC_EQID:
+ case PDS_RC_EINVAL:
+ case PDS_RC_ENOSUPP:
+ return -EINVAL;
+ case PDS_RC_EPERM:
+ return -EPERM;
+ case PDS_RC_ENOENT:
+ return -ENOENT;
+ case PDS_RC_EAGAIN:
+ return -EAGAIN;
+ case PDS_RC_ENOMEM:
+ return -ENOMEM;
+ case PDS_RC_EFAULT:
+ return -EFAULT;
+ case PDS_RC_EBUSY:
+ return -EBUSY;
+ case PDS_RC_EEXIST:
+ return -EEXIST;
+ case PDS_RC_EVFID:
+ return -ENODEV;
+ case PDS_RC_ECLIENT:
+ return -ECHILD;
+ case PDS_RC_ENOSPC:
+ return -ENOSPC;
+ case PDS_RC_ERANGE:
+ return -ERANGE;
+ case PDS_RC_BAD_ADDR:
+ return -EFAULT;
+ case PDS_RC_EOPCODE:
+ case PDS_RC_EINTR:
+ case PDS_RC_DEV_CMD:
+ case PDS_RC_ERROR:
+ case PDS_RC_ERDMA:
+ case PDS_RC_EIO:
+ default:
+ return -EIO;
+ }
+}
+
+bool pdsc_is_fw_running(struct pdsc *pdsc)
+{
+ pdsc->fw_status = ioread8(&pdsc->info_regs->fw_status);
+ pdsc->last_fw_time = jiffies;
+ pdsc->last_hb = ioread32(&pdsc->info_regs->fw_heartbeat);
+
+ /* Firmware is useful only if the running bit is set and
+ * fw_status != 0xff (bad PCI read)
+ */
+ return (pdsc->fw_status != 0xff) &&
+ (pdsc->fw_status & PDS_CORE_FW_STS_F_RUNNING);
+}
+
+bool pdsc_is_fw_good(struct pdsc *pdsc)
+{
+ u8 gen = pdsc->fw_status & PDS_CORE_FW_STS_F_GENERATION;
+
+ return pdsc_is_fw_running(pdsc) && gen == pdsc->fw_generation;
+}
+
+static u8 pdsc_devcmd_status(struct pdsc *pdsc)
+{
+ return ioread8(&pdsc->cmd_regs->comp.status);
+}
+
+static bool pdsc_devcmd_done(struct pdsc *pdsc)
+{
+ return ioread32(&pdsc->cmd_regs->done) & PDS_CORE_DEV_CMD_DONE;
+}
+
+static void pdsc_devcmd_dbell(struct pdsc *pdsc)
+{
+ iowrite32(0, &pdsc->cmd_regs->done);
+ iowrite32(1, &pdsc->cmd_regs->doorbell);
+}
+
+static void pdsc_devcmd_clean(struct pdsc *pdsc)
+{
+ iowrite32(0, &pdsc->cmd_regs->doorbell);
+ memset_io(&pdsc->cmd_regs->cmd, 0, sizeof(pdsc->cmd_regs->cmd));
+}
+
+static const char *pdsc_devcmd_str(int opcode)
+{
+ switch (opcode) {
+ case PDS_CORE_CMD_NOP:
+ return "PDS_CORE_CMD_NOP";
+ case PDS_CORE_CMD_IDENTIFY:
+ return "PDS_CORE_CMD_IDENTIFY";
+ case PDS_CORE_CMD_RESET:
+ return "PDS_CORE_CMD_RESET";
+ case PDS_CORE_CMD_INIT:
+ return "PDS_CORE_CMD_INIT";
+ case PDS_CORE_CMD_FW_DOWNLOAD:
+ return "PDS_CORE_CMD_FW_DOWNLOAD";
+ case PDS_CORE_CMD_FW_CONTROL:
+ return "PDS_CORE_CMD_FW_CONTROL";
+ default:
+ return "PDS_CORE_CMD_UNKNOWN";
+ }
+}
+
+static int pdsc_devcmd_wait(struct pdsc *pdsc, int max_seconds)
+{
+ struct device *dev = pdsc->dev;
+ unsigned long start_time;
+ unsigned long max_wait;
+ unsigned long duration;
+ int timeout = 0;
+ int done = 0;
+ int err = 0;
+ int status;
+ int opcode;
+
+ opcode = ioread8(&pdsc->cmd_regs->cmd.opcode);
+
+ start_time = jiffies;
+ max_wait = start_time + (max_seconds * HZ);
+
+ while (!done && !timeout) {
+ done = pdsc_devcmd_done(pdsc);
+ if (done)
+ break;
+
+ timeout = time_after(jiffies, max_wait);
+ if (timeout)
+ break;
+
+ usleep_range(100, 200);
+ }
+ duration = jiffies - start_time;
+
+ if (done && duration > HZ)
+ dev_dbg(dev, "DEVCMD %d %s after %ld secs\n",
+ opcode, pdsc_devcmd_str(opcode), duration / HZ);
+
+ if (!done || timeout) {
+ dev_err(dev, "DEVCMD %d %s timeout, done %d timeout %d max_seconds=%d\n",
+ opcode, pdsc_devcmd_str(opcode), done, timeout,
+ max_seconds);
+ err = -ETIMEDOUT;
+ pdsc_devcmd_clean(pdsc);
+ }
+
+ status = pdsc_devcmd_status(pdsc);
+ err = pdsc_err_to_errno(status);
+ if (err && err != -EAGAIN)
+ dev_err(dev, "DEVCMD %d %s failed, status=%d err %d %pe\n",
+ opcode, pdsc_devcmd_str(opcode), status, err,
+ ERR_PTR(err));
+
+ return err;
+}
+
+int pdsc_devcmd_locked(struct pdsc *pdsc, union pds_core_dev_cmd *cmd,
+ union pds_core_dev_comp *comp, int max_seconds)
+{
+ int err;
+
+ memcpy_toio(&pdsc->cmd_regs->cmd, cmd, sizeof(*cmd));
+ pdsc_devcmd_dbell(pdsc);
+ err = pdsc_devcmd_wait(pdsc, max_seconds);
+ memcpy_fromio(comp, &pdsc->cmd_regs->comp, sizeof(*comp));
+
+ if (err == -ENXIO || err == -ETIMEDOUT)
+ queue_work(pdsc->wq, &pdsc->health_work);
+
+ return err;
+}
+
+int pdsc_devcmd(struct pdsc *pdsc, union pds_core_dev_cmd *cmd,
+ union pds_core_dev_comp *comp, int max_seconds)
+{
+ int err;
+
+ mutex_lock(&pdsc->devcmd_lock);
+ err = pdsc_devcmd_locked(pdsc, cmd, comp, max_seconds);
+ mutex_unlock(&pdsc->devcmd_lock);
+
+ return err;
+}
+
+int pdsc_devcmd_init(struct pdsc *pdsc)
+{
+ union pds_core_dev_comp comp = {};
+ union pds_core_dev_cmd cmd = {
+ .opcode = PDS_CORE_CMD_INIT,
+ };
+
+ return pdsc_devcmd(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+}
+
+int pdsc_devcmd_reset(struct pdsc *pdsc)
+{
+ union pds_core_dev_comp comp = {};
+ union pds_core_dev_cmd cmd = {
+ .reset.opcode = PDS_CORE_CMD_RESET,
+ };
+
+ return pdsc_devcmd(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+}
+
+static int pdsc_devcmd_identify_locked(struct pdsc *pdsc)
+{
+ union pds_core_dev_comp comp = {};
+ union pds_core_dev_cmd cmd = {
+ .identify.opcode = PDS_CORE_CMD_IDENTIFY,
+ .identify.ver = PDS_CORE_IDENTITY_VERSION_1,
+ };
+
+ return pdsc_devcmd_locked(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+}
+
+static void pdsc_init_devinfo(struct pdsc *pdsc)
+{
+ pdsc->dev_info.asic_type = ioread8(&pdsc->info_regs->asic_type);
+ pdsc->dev_info.asic_rev = ioread8(&pdsc->info_regs->asic_rev);
+ pdsc->fw_generation = PDS_CORE_FW_STS_F_GENERATION &
+ ioread8(&pdsc->info_regs->fw_status);
+
+ memcpy_fromio(pdsc->dev_info.fw_version,
+ pdsc->info_regs->fw_version,
+ PDS_CORE_DEVINFO_FWVERS_BUFLEN);
+ pdsc->dev_info.fw_version[PDS_CORE_DEVINFO_FWVERS_BUFLEN] = 0;
+
+ memcpy_fromio(pdsc->dev_info.serial_num,
+ pdsc->info_regs->serial_num,
+ PDS_CORE_DEVINFO_SERIAL_BUFLEN);
+ pdsc->dev_info.serial_num[PDS_CORE_DEVINFO_SERIAL_BUFLEN] = 0;
+
+ dev_dbg(pdsc->dev, "fw_version %s\n", pdsc->dev_info.fw_version);
+}
+
+static int pdsc_identify(struct pdsc *pdsc)
+{
+ struct pds_core_drv_identity drv = {};
+ size_t sz;
+ int err;
+
+ drv.drv_type = cpu_to_le32(PDS_DRIVER_LINUX);
+ snprintf(drv.driver_ver_str, sizeof(drv.driver_ver_str),
+ "%s %s", PDS_CORE_DRV_NAME, utsname()->release);
+
+ /* Next let's get some info about the device
+ * We use the devcmd_lock at this level in order to
+ * get safe access to the cmd_regs->data before anyone
+ * else can mess it up
+ */
+ mutex_lock(&pdsc->devcmd_lock);
+
+ sz = min_t(size_t, sizeof(drv), sizeof(pdsc->cmd_regs->data));
+ memcpy_toio(&pdsc->cmd_regs->data, &drv, sz);
+
+ err = pdsc_devcmd_identify_locked(pdsc);
+ if (!err) {
+ sz = min_t(size_t, sizeof(pdsc->dev_ident),
+ sizeof(pdsc->cmd_regs->data));
+ memcpy_fromio(&pdsc->dev_ident, &pdsc->cmd_regs->data, sz);
+ }
+ mutex_unlock(&pdsc->devcmd_lock);
+
+ if (err) {
+ dev_err(pdsc->dev, "Cannot identify device: %pe\n",
+ ERR_PTR(err));
+ return err;
+ }
+
+ if (isprint(pdsc->dev_info.fw_version[0]) &&
+ isascii(pdsc->dev_info.fw_version[0]))
+ dev_info(pdsc->dev, "FW: %.*s\n",
+ (int)(sizeof(pdsc->dev_info.fw_version) - 1),
+ pdsc->dev_info.fw_version);
+ else
+ dev_info(pdsc->dev, "FW: (invalid string) 0x%02x 0x%02x 0x%02x 0x%02x ...\n",
+ (u8)pdsc->dev_info.fw_version[0],
+ (u8)pdsc->dev_info.fw_version[1],
+ (u8)pdsc->dev_info.fw_version[2],
+ (u8)pdsc->dev_info.fw_version[3]);
+
+ return 0;
+}
+
+int pdsc_dev_reinit(struct pdsc *pdsc)
+{
+ pdsc_init_devinfo(pdsc);
+
+ return pdsc_identify(pdsc);
+}
+
+int pdsc_dev_init(struct pdsc *pdsc)
+{
+ unsigned int nintrs;
+ int err;
+
+ /* Initial init and reset of device */
+ pdsc_init_devinfo(pdsc);
+ pdsc->devcmd_timeout = PDS_CORE_DEVCMD_TIMEOUT;
+
+ err = pdsc_devcmd_reset(pdsc);
+ if (err)
+ return err;
+
+ err = pdsc_identify(pdsc);
+ if (err)
+ return err;
+
+ pdsc_debugfs_add_ident(pdsc);
+
+ /* Now we can reserve interrupts */
+ nintrs = le32_to_cpu(pdsc->dev_ident.nintrs);
+ nintrs = min_t(unsigned int, num_online_cpus(), nintrs);
+
+ /* Get intr_info struct array for tracking */
+ pdsc->intr_info = kcalloc(nintrs, sizeof(*pdsc->intr_info), GFP_KERNEL);
+ if (!pdsc->intr_info) {
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ err = pci_alloc_irq_vectors(pdsc->pdev, nintrs, nintrs, PCI_IRQ_MSIX);
+ if (err != nintrs) {
+ dev_err(pdsc->dev, "Can't get %d intrs from OS: %pe\n",
+ nintrs, ERR_PTR(err));
+ err = -ENOSPC;
+ goto err_out;
+ }
+ pdsc->nintrs = nintrs;
+
+ return 0;
+
+err_out:
+ kfree(pdsc->intr_info);
+ pdsc->intr_info = NULL;
+
+ return err;
+}
diff --git a/drivers/net/ethernet/amd/pds_core/devlink.c b/drivers/net/ethernet/amd/pds_core/devlink.c
new file mode 100644
index 0000000..9c6b365
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/devlink.c
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include "core.h"
+#include <linux/pds/pds_auxbus.h>
+
+static struct
+pdsc_viftype *pdsc_dl_find_viftype_by_id(struct pdsc *pdsc,
+ enum devlink_param_type dl_id)
+{
+ int vt;
+
+ for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++) {
+ if (pdsc->viftype_status[vt].dl_id == dl_id)
+ return &pdsc->viftype_status[vt];
+ }
+
+ return NULL;
+}
+
+int pdsc_dl_enable_get(struct devlink *dl, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct pdsc *pdsc = devlink_priv(dl);
+ struct pdsc_viftype *vt_entry;
+
+ vt_entry = pdsc_dl_find_viftype_by_id(pdsc, id);
+ if (!vt_entry)
+ return -ENOENT;
+
+ ctx->val.vbool = vt_entry->enabled;
+
+ return 0;
+}
+
+int pdsc_dl_enable_set(struct devlink *dl, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct pdsc *pdsc = devlink_priv(dl);
+ struct pdsc_viftype *vt_entry;
+ int err = 0;
+ int vf_id;
+
+ vt_entry = pdsc_dl_find_viftype_by_id(pdsc, id);
+ if (!vt_entry || !vt_entry->supported)
+ return -EOPNOTSUPP;
+
+ if (vt_entry->enabled == ctx->val.vbool)
+ return 0;
+
+ vt_entry->enabled = ctx->val.vbool;
+ for (vf_id = 0; vf_id < pdsc->num_vfs; vf_id++) {
+ struct pdsc *vf = pdsc->vfs[vf_id].vf;
+
+ err = ctx->val.vbool ? pdsc_auxbus_dev_add(vf, pdsc) :
+ pdsc_auxbus_dev_del(vf, pdsc);
+ }
+
+ return err;
+}
+
+int pdsc_dl_enable_validate(struct devlink *dl, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ struct pdsc *pdsc = devlink_priv(dl);
+ struct pdsc_viftype *vt_entry;
+
+ vt_entry = pdsc_dl_find_viftype_by_id(pdsc, id);
+ if (!vt_entry || !vt_entry->supported)
+ return -EOPNOTSUPP;
+
+ if (!pdsc->viftype_status[vt_entry->vif_id].supported)
+ return -ENODEV;
+
+ return 0;
+}
+
+int pdsc_dl_flash_update(struct devlink *dl,
+ struct devlink_flash_update_params *params,
+ struct netlink_ext_ack *extack)
+{
+ struct pdsc *pdsc = devlink_priv(dl);
+
+ return pdsc_firmware_update(pdsc, params->fw, extack);
+}
+
+static char *fw_slotnames[] = {
+ "fw.goldfw",
+ "fw.mainfwa",
+ "fw.mainfwb",
+};
+
+int pdsc_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+ struct netlink_ext_ack *extack)
+{
+ union pds_core_dev_cmd cmd = {
+ .fw_control.opcode = PDS_CORE_CMD_FW_CONTROL,
+ .fw_control.oper = PDS_CORE_FW_GET_LIST,
+ };
+ struct pds_core_fw_list_info fw_list;
+ struct pdsc *pdsc = devlink_priv(dl);
+ union pds_core_dev_comp comp;
+ char buf[16];
+ int listlen;
+ int err;
+ int i;
+
+ mutex_lock(&pdsc->devcmd_lock);
+ err = pdsc_devcmd_locked(pdsc, &cmd, &comp, pdsc->devcmd_timeout * 2);
+ memcpy_fromio(&fw_list, pdsc->cmd_regs->data, sizeof(fw_list));
+ mutex_unlock(&pdsc->devcmd_lock);
+ if (err && err != -EIO)
+ return err;
+
+ listlen = fw_list.num_fw_slots;
+ for (i = 0; i < listlen; i++) {
+ if (i < ARRAY_SIZE(fw_slotnames))
+ strscpy(buf, fw_slotnames[i], sizeof(buf));
+ else
+ snprintf(buf, sizeof(buf), "fw.slot_%d", i);
+ err = devlink_info_version_stored_put(req, buf,
+ fw_list.fw_names[i].fw_version);
+ }
+
+ err = devlink_info_version_running_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_FW,
+ pdsc->dev_info.fw_version);
+ if (err)
+ return err;
+
+ snprintf(buf, sizeof(buf), "0x%x", pdsc->dev_info.asic_type);
+ err = devlink_info_version_fixed_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_ASIC_ID,
+ buf);
+ if (err)
+ return err;
+
+ snprintf(buf, sizeof(buf), "0x%x", pdsc->dev_info.asic_rev);
+ err = devlink_info_version_fixed_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_ASIC_REV,
+ buf);
+ if (err)
+ return err;
+
+ return devlink_info_serial_number_put(req, pdsc->dev_info.serial_num);
+}
+
+int pdsc_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
+ struct devlink_fmsg *fmsg,
+ struct netlink_ext_ack *extack)
+{
+ struct pdsc *pdsc = devlink_health_reporter_priv(reporter);
+ int err;
+
+ mutex_lock(&pdsc->config_lock);
+
+ if (test_bit(PDSC_S_FW_DEAD, &pdsc->state))
+ err = devlink_fmsg_string_pair_put(fmsg, "Status", "dead");
+ else if (!pdsc_is_fw_good(pdsc))
+ err = devlink_fmsg_string_pair_put(fmsg, "Status", "unhealthy");
+ else
+ err = devlink_fmsg_string_pair_put(fmsg, "Status", "healthy");
+
+ mutex_unlock(&pdsc->config_lock);
+
+ if (err)
+ return err;
+
+ err = devlink_fmsg_u32_pair_put(fmsg, "State",
+ pdsc->fw_status &
+ ~PDS_CORE_FW_STS_F_GENERATION);
+ if (err)
+ return err;
+
+ err = devlink_fmsg_u32_pair_put(fmsg, "Generation",
+ pdsc->fw_generation >> 4);
+ if (err)
+ return err;
+
+ return devlink_fmsg_u32_pair_put(fmsg, "Recoveries",
+ pdsc->fw_recoveries);
+}
diff --git a/drivers/net/ethernet/amd/pds_core/fw.c b/drivers/net/ethernet/amd/pds_core/fw.c
new file mode 100644
index 0000000..90a811f
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/fw.c
@@ -0,0 +1,194 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include "core.h"
+
+/* The worst case wait for the install activity is about 25 minutes when
+ * installing a new CPLD, which is very seldom. Normal is about 30-35
+ * seconds. Since the driver can't tell if a CPLD update will happen we
+ * set the timeout for the ugly case.
+ */
+#define PDSC_FW_INSTALL_TIMEOUT (25 * 60)
+#define PDSC_FW_SELECT_TIMEOUT 30
+
+/* Number of periodic log updates during fw file download */
+#define PDSC_FW_INTERVAL_FRACTION 32
+
+static int pdsc_devcmd_fw_download_locked(struct pdsc *pdsc, u64 addr,
+ u32 offset, u32 length)
+{
+ union pds_core_dev_cmd cmd = {
+ .fw_download.opcode = PDS_CORE_CMD_FW_DOWNLOAD,
+ .fw_download.offset = cpu_to_le32(offset),
+ .fw_download.addr = cpu_to_le64(addr),
+ .fw_download.length = cpu_to_le32(length),
+ };
+ union pds_core_dev_comp comp;
+
+ return pdsc_devcmd_locked(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+}
+
+static int pdsc_devcmd_fw_install(struct pdsc *pdsc)
+{
+ union pds_core_dev_cmd cmd = {
+ .fw_control.opcode = PDS_CORE_CMD_FW_CONTROL,
+ .fw_control.oper = PDS_CORE_FW_INSTALL_ASYNC
+ };
+ union pds_core_dev_comp comp;
+ int err;
+
+ err = pdsc_devcmd(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+ if (err < 0)
+ return err;
+
+ return comp.fw_control.slot;
+}
+
+static int pdsc_devcmd_fw_activate(struct pdsc *pdsc,
+ enum pds_core_fw_slot slot)
+{
+ union pds_core_dev_cmd cmd = {
+ .fw_control.opcode = PDS_CORE_CMD_FW_CONTROL,
+ .fw_control.oper = PDS_CORE_FW_ACTIVATE_ASYNC,
+ .fw_control.slot = slot
+ };
+ union pds_core_dev_comp comp;
+
+ return pdsc_devcmd(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+}
+
+static int pdsc_fw_status_long_wait(struct pdsc *pdsc,
+ const char *label,
+ unsigned long timeout,
+ u8 fw_cmd,
+ struct netlink_ext_ack *extack)
+{
+ union pds_core_dev_cmd cmd = {
+ .fw_control.opcode = PDS_CORE_CMD_FW_CONTROL,
+ .fw_control.oper = fw_cmd,
+ };
+ union pds_core_dev_comp comp;
+ unsigned long start_time;
+ unsigned long end_time;
+ int err;
+
+ /* Ping on the status of the long running async install
+ * command. We get EAGAIN while the command is still
+ * running, else we get the final command status.
+ */
+ start_time = jiffies;
+ end_time = start_time + (timeout * HZ);
+ do {
+ err = pdsc_devcmd(pdsc, &cmd, &comp, pdsc->devcmd_timeout);
+ msleep(20);
+ } while (time_before(jiffies, end_time) &&
+ (err == -EAGAIN || err == -ETIMEDOUT));
+
+ if (err == -EAGAIN || err == -ETIMEDOUT) {
+ NL_SET_ERR_MSG_MOD(extack, "Firmware wait timed out");
+ dev_err(pdsc->dev, "DEV_CMD firmware wait %s timed out\n",
+ label);
+ } else if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Firmware wait failed");
+ }
+
+ return err;
+}
+
+int pdsc_firmware_update(struct pdsc *pdsc, const struct firmware *fw,
+ struct netlink_ext_ack *extack)
+{
+ u32 buf_sz, copy_sz, offset;
+ struct devlink *dl;
+ int next_interval;
+ u64 data_addr;
+ int err = 0;
+ int fw_slot;
+
+ dev_info(pdsc->dev, "Installing firmware\n");
+
+ dl = priv_to_devlink(pdsc);
+ devlink_flash_update_status_notify(dl, "Preparing to flash",
+ NULL, 0, 0);
+
+ buf_sz = sizeof(pdsc->cmd_regs->data);
+
+ dev_dbg(pdsc->dev,
+ "downloading firmware - size %d part_sz %d nparts %lu\n",
+ (int)fw->size, buf_sz, DIV_ROUND_UP(fw->size, buf_sz));
+
+ offset = 0;
+ next_interval = 0;
+ data_addr = offsetof(struct pds_core_dev_cmd_regs, data);
+ while (offset < fw->size) {
+ if (offset >= next_interval) {
+ devlink_flash_update_status_notify(dl, "Downloading",
+ NULL, offset,
+ fw->size);
+ next_interval = offset +
+ (fw->size / PDSC_FW_INTERVAL_FRACTION);
+ }
+
+ copy_sz = min_t(unsigned int, buf_sz, fw->size - offset);
+ mutex_lock(&pdsc->devcmd_lock);
+ memcpy_toio(&pdsc->cmd_regs->data, fw->data + offset, copy_sz);
+ err = pdsc_devcmd_fw_download_locked(pdsc, data_addr,
+ offset, copy_sz);
+ mutex_unlock(&pdsc->devcmd_lock);
+ if (err) {
+ dev_err(pdsc->dev,
+ "download failed offset 0x%x addr 0x%llx len 0x%x: %pe\n",
+ offset, data_addr, copy_sz, ERR_PTR(err));
+ NL_SET_ERR_MSG_MOD(extack, "Segment download failed");
+ goto err_out;
+ }
+ offset += copy_sz;
+ }
+ devlink_flash_update_status_notify(dl, "Downloading", NULL,
+ fw->size, fw->size);
+
+ devlink_flash_update_timeout_notify(dl, "Installing", NULL,
+ PDSC_FW_INSTALL_TIMEOUT);
+
+ fw_slot = pdsc_devcmd_fw_install(pdsc);
+ if (fw_slot < 0) {
+ err = fw_slot;
+ dev_err(pdsc->dev, "install failed: %pe\n", ERR_PTR(err));
+ NL_SET_ERR_MSG_MOD(extack, "Failed to start firmware install");
+ goto err_out;
+ }
+
+ err = pdsc_fw_status_long_wait(pdsc, "Installing",
+ PDSC_FW_INSTALL_TIMEOUT,
+ PDS_CORE_FW_INSTALL_STATUS,
+ extack);
+ if (err)
+ goto err_out;
+
+ devlink_flash_update_timeout_notify(dl, "Selecting", NULL,
+ PDSC_FW_SELECT_TIMEOUT);
+
+ err = pdsc_devcmd_fw_activate(pdsc, fw_slot);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Failed to start firmware select");
+ goto err_out;
+ }
+
+ err = pdsc_fw_status_long_wait(pdsc, "Selecting",
+ PDSC_FW_SELECT_TIMEOUT,
+ PDS_CORE_FW_ACTIVATE_STATUS,
+ extack);
+ if (err)
+ goto err_out;
+
+ dev_info(pdsc->dev, "Firmware update completed, slot %d\n", fw_slot);
+
+err_out:
+ if (err)
+ devlink_flash_update_status_notify(dl, "Flash failed",
+ NULL, 0, 0);
+ else
+ devlink_flash_update_status_notify(dl, "Flash done",
+ NULL, 0, 0);
+ return err;
+}
diff --git a/drivers/net/ethernet/amd/pds_core/main.c b/drivers/net/ethernet/amd/pds_core/main.c
new file mode 100644
index 0000000..e2d14b1
--- /dev/null
+++ b/drivers/net/ethernet/amd/pds_core/main.c
@@ -0,0 +1,475 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/pci.h>
+
+#include <linux/pds/pds_common.h>
+
+#include "core.h"
+
+MODULE_DESCRIPTION(PDSC_DRV_DESCRIPTION);
+MODULE_AUTHOR("Advanced Micro Devices, Inc");
+MODULE_LICENSE("GPL");
+
+/* Supported devices */
+static const struct pci_device_id pdsc_id_table[] = {
+ { PCI_VDEVICE(PENSANDO, PCI_DEVICE_ID_PENSANDO_CORE_PF) },
+ { PCI_VDEVICE(PENSANDO, PCI_DEVICE_ID_PENSANDO_VDPA_VF) },
+ { 0, } /* end of table */
+};
+MODULE_DEVICE_TABLE(pci, pdsc_id_table);
+
+static void pdsc_wdtimer_cb(struct timer_list *t)
+{
+ struct pdsc *pdsc = from_timer(pdsc, t, wdtimer);
+
+ dev_dbg(pdsc->dev, "%s: jiffies %ld\n", __func__, jiffies);
+ mod_timer(&pdsc->wdtimer,
+ round_jiffies(jiffies + pdsc->wdtimer_period));
+
+ queue_work(pdsc->wq, &pdsc->health_work);
+}
+
+static void pdsc_unmap_bars(struct pdsc *pdsc)
+{
+ struct pdsc_dev_bar *bars = pdsc->bars;
+ unsigned int i;
+
+ for (i = 0; i < PDS_CORE_BARS_MAX; i++) {
+ if (bars[i].vaddr)
+ pci_iounmap(pdsc->pdev, bars[i].vaddr);
+ }
+}
+
+static int pdsc_map_bars(struct pdsc *pdsc)
+{
+ struct pdsc_dev_bar *bar = pdsc->bars;
+ struct pci_dev *pdev = pdsc->pdev;
+ struct device *dev = pdsc->dev;
+ struct pdsc_dev_bar *bars;
+ unsigned int i, j;
+ int num_bars = 0;
+ int err;
+ u32 sig;
+
+ bars = pdsc->bars;
+
+ /* Since the PCI interface in the hardware is configurable,
+ * we need to poke into all the bars to find the set we're
+ * expecting.
+ */
+ for (i = 0, j = 0; i < PDS_CORE_BARS_MAX; i++) {
+ if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM))
+ continue;
+
+ bars[j].len = pci_resource_len(pdev, i);
+ bars[j].bus_addr = pci_resource_start(pdev, i);
+ bars[j].res_index = i;
+
+ /* only map the whole bar 0 */
+ if (j > 0) {
+ bars[j].vaddr = NULL;
+ } else {
+ bars[j].vaddr = pci_iomap(pdev, i, bars[j].len);
+ if (!bars[j].vaddr) {
+ dev_err(dev, "Cannot map BAR %d, aborting\n", i);
+ return -ENODEV;
+ }
+ }
+
+ j++;
+ }
+ num_bars = j;
+
+ /* BAR0: dev_cmd and interrupts */
+ if (num_bars < 1) {
+ dev_err(dev, "No bars found\n");
+ err = -EFAULT;
+ goto err_out;
+ }
+
+ if (bar->len < PDS_CORE_BAR0_SIZE) {
+ dev_err(dev, "Resource bar size %lu too small\n", bar->len);
+ err = -EFAULT;
+ goto err_out;
+ }
+
+ pdsc->info_regs = bar->vaddr + PDS_CORE_BAR0_DEV_INFO_REGS_OFFSET;
+ pdsc->cmd_regs = bar->vaddr + PDS_CORE_BAR0_DEV_CMD_REGS_OFFSET;
+ pdsc->intr_status = bar->vaddr + PDS_CORE_BAR0_INTR_STATUS_OFFSET;
+ pdsc->intr_ctrl = bar->vaddr + PDS_CORE_BAR0_INTR_CTRL_OFFSET;
+
+ sig = ioread32(&pdsc->info_regs->signature);
+ if (sig != PDS_CORE_DEV_INFO_SIGNATURE) {
+ dev_err(dev, "Incompatible firmware signature %x", sig);
+ err = -EFAULT;
+ goto err_out;
+ }
+
+ /* BAR1: doorbells */
+ bar++;
+ if (num_bars < 2) {
+ dev_err(dev, "Doorbell bar missing\n");
+ err = -EFAULT;
+ goto err_out;
+ }
+
+ pdsc->db_pages = bar->vaddr;
+ pdsc->phy_db_pages = bar->bus_addr;
+
+ return 0;
+
+err_out:
+ pdsc_unmap_bars(pdsc);
+ return err;
+}
+
+void __iomem *pdsc_map_dbpage(struct pdsc *pdsc, int page_num)
+{
+ return pci_iomap_range(pdsc->pdev,
+ pdsc->bars[PDS_CORE_PCI_BAR_DBELL].res_index,
+ (u64)page_num << PAGE_SHIFT, PAGE_SIZE);
+}
+
+static int pdsc_sriov_configure(struct pci_dev *pdev, int num_vfs)
+{
+ struct pdsc *pdsc = pci_get_drvdata(pdev);
+ struct device *dev = pdsc->dev;
+ int ret = 0;
+
+ if (num_vfs > 0) {
+ pdsc->vfs = kcalloc(num_vfs, sizeof(struct pdsc_vf),
+ GFP_KERNEL);
+ if (!pdsc->vfs)
+ return -ENOMEM;
+ pdsc->num_vfs = num_vfs;
+
+ ret = pci_enable_sriov(pdev, num_vfs);
+ if (ret) {
+ dev_err(dev, "Cannot enable SRIOV: %pe\n",
+ ERR_PTR(ret));
+ goto no_vfs;
+ }
+
+ return num_vfs;
+ }
+
+no_vfs:
+ pci_disable_sriov(pdev);
+
+ kfree(pdsc->vfs);
+ pdsc->vfs = NULL;
+ pdsc->num_vfs = 0;
+
+ return ret;
+}
+
+static int pdsc_init_vf(struct pdsc *vf)
+{
+ struct devlink *dl;
+ struct pdsc *pf;
+ int err;
+
+ pf = pdsc_get_pf_struct(vf->pdev);
+ if (IS_ERR_OR_NULL(pf))
+ return PTR_ERR(pf) ?: -1;
+
+ vf->vf_id = pci_iov_vf_id(vf->pdev);
+
+ dl = priv_to_devlink(vf);
+ devl_lock(dl);
+ devl_register(dl);
+ devl_unlock(dl);
+
+ pf->vfs[vf->vf_id].vf = vf;
+ err = pdsc_auxbus_dev_add(vf, pf);
+ if (err) {
+ devl_lock(dl);
+ devl_unregister(dl);
+ devl_unlock(dl);
+ }
+
+ return err;
+}
+
+static const struct devlink_health_reporter_ops pdsc_fw_reporter_ops = {
+ .name = "fw",
+ .diagnose = pdsc_fw_reporter_diagnose,
+};
+
+static const struct devlink_param pdsc_dl_params[] = {
+ DEVLINK_PARAM_GENERIC(ENABLE_VNET,
+ BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ pdsc_dl_enable_get,
+ pdsc_dl_enable_set,
+ pdsc_dl_enable_validate),
+};
+
+#define PDSC_WQ_NAME_LEN 24
+
+static int pdsc_init_pf(struct pdsc *pdsc)
+{
+ struct devlink_health_reporter *hr;
+ char wq_name[PDSC_WQ_NAME_LEN];
+ struct devlink *dl;
+ int err;
+
+ pcie_print_link_status(pdsc->pdev);
+
+ err = pci_request_regions(pdsc->pdev, PDS_CORE_DRV_NAME);
+ if (err) {
+ dev_err(pdsc->dev, "Cannot request PCI regions: %pe\n",
+ ERR_PTR(err));
+ return err;
+ }
+
+ err = pdsc_map_bars(pdsc);
+ if (err)
+ goto err_out_release_regions;
+
+ /* General workqueue and timer, but don't start timer yet */
+ snprintf(wq_name, sizeof(wq_name), "%s.%d", PDS_CORE_DRV_NAME, pdsc->uid);
+ pdsc->wq = create_singlethread_workqueue(wq_name);
+ INIT_WORK(&pdsc->health_work, pdsc_health_thread);
+ timer_setup(&pdsc->wdtimer, pdsc_wdtimer_cb, 0);
+ pdsc->wdtimer_period = PDSC_WATCHDOG_SECS * HZ;
+
+ mutex_init(&pdsc->devcmd_lock);
+ mutex_init(&pdsc->config_lock);
+ spin_lock_init(&pdsc->adminq_lock);
+
+ mutex_lock(&pdsc->config_lock);
+ set_bit(PDSC_S_FW_DEAD, &pdsc->state);
+
+ err = pdsc_setup(pdsc, PDSC_SETUP_INIT);
+ if (err)
+ goto err_out_unmap_bars;
+ err = pdsc_start(pdsc);
+ if (err)
+ goto err_out_teardown;
+
+ mutex_unlock(&pdsc->config_lock);
+
+ dl = priv_to_devlink(pdsc);
+ devl_lock(dl);
+ err = devl_params_register(dl, pdsc_dl_params,
+ ARRAY_SIZE(pdsc_dl_params));
+ if (err) {
+ dev_warn(pdsc->dev, "Failed to register devlink params: %pe\n",
+ ERR_PTR(err));
+ goto err_out_unlock_dl;
+ }
+
+ hr = devl_health_reporter_create(dl, &pdsc_fw_reporter_ops, 0, pdsc);
+ if (IS_ERR(hr)) {
+ dev_warn(pdsc->dev, "Failed to create fw reporter: %pe\n", hr);
+ err = PTR_ERR(hr);
+ goto err_out_unreg_params;
+ }
+ pdsc->fw_reporter = hr;
+
+ devl_register(dl);
+ devl_unlock(dl);
+
+ /* Lastly, start the health check timer */
+ mod_timer(&pdsc->wdtimer, round_jiffies(jiffies + pdsc->wdtimer_period));
+
+ return 0;
+
+err_out_unreg_params:
+ devl_params_unregister(dl, pdsc_dl_params,
+ ARRAY_SIZE(pdsc_dl_params));
+err_out_unlock_dl:
+ devl_unlock(dl);
+ pdsc_stop(pdsc);
+err_out_teardown:
+ pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
+err_out_unmap_bars:
+ mutex_unlock(&pdsc->config_lock);
+ del_timer_sync(&pdsc->wdtimer);
+ if (pdsc->wq)
+ destroy_workqueue(pdsc->wq);
+ mutex_destroy(&pdsc->config_lock);
+ mutex_destroy(&pdsc->devcmd_lock);
+ pci_free_irq_vectors(pdsc->pdev);
+ pdsc_unmap_bars(pdsc);
+err_out_release_regions:
+ pci_release_regions(pdsc->pdev);
+
+ return err;
+}
+
+static const struct devlink_ops pdsc_dl_ops = {
+ .info_get = pdsc_dl_info_get,
+ .flash_update = pdsc_dl_flash_update,
+};
+
+static const struct devlink_ops pdsc_dl_vf_ops = {
+};
+
+static DEFINE_IDA(pdsc_ida);
+
+static int pdsc_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ struct device *dev = &pdev->dev;
+ const struct devlink_ops *ops;
+ struct devlink *dl;
+ struct pdsc *pdsc;
+ bool is_pf;
+ int err;
+
+ is_pf = !pdev->is_virtfn;
+ ops = is_pf ? &pdsc_dl_ops : &pdsc_dl_vf_ops;
+ dl = devlink_alloc(ops, sizeof(struct pdsc), dev);
+ if (!dl)
+ return -ENOMEM;
+ pdsc = devlink_priv(dl);
+
+ pdsc->pdev = pdev;
+ pdsc->dev = &pdev->dev;
+ set_bit(PDSC_S_INITING_DRIVER, &pdsc->state);
+ pci_set_drvdata(pdev, pdsc);
+ pdsc_debugfs_add_dev(pdsc);
+
+ err = ida_alloc(&pdsc_ida, GFP_KERNEL);
+ if (err < 0) {
+ dev_err(pdsc->dev, "%s: id alloc failed: %pe\n",
+ __func__, ERR_PTR(err));
+ goto err_out_free_devlink;
+ }
+ pdsc->uid = err;
+
+ /* Query system for DMA addressing limitation for the device. */
+ err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(PDS_CORE_ADDR_LEN));
+ if (err) {
+ dev_err(dev, "Unable to obtain 64-bit DMA for consistent allocations, aborting: %pe\n",
+ ERR_PTR(err));
+ goto err_out_free_ida;
+ }
+
+ err = pci_enable_device(pdev);
+ if (err) {
+ dev_err(dev, "Cannot enable PCI device: %pe\n", ERR_PTR(err));
+ goto err_out_free_ida;
+ }
+ pci_set_master(pdev);
+
+ if (is_pf)
+ err = pdsc_init_pf(pdsc);
+ else
+ err = pdsc_init_vf(pdsc);
+ if (err) {
+ dev_err(dev, "Cannot init device: %pe\n", ERR_PTR(err));
+ goto err_out_clear_master;
+ }
+
+ clear_bit(PDSC_S_INITING_DRIVER, &pdsc->state);
+ return 0;
+
+err_out_clear_master:
+ pci_clear_master(pdev);
+ pci_disable_device(pdev);
+err_out_free_ida:
+ ida_free(&pdsc_ida, pdsc->uid);
+err_out_free_devlink:
+ pdsc_debugfs_del_dev(pdsc);
+ devlink_free(dl);
+
+ return err;
+}
+
+static void pdsc_remove(struct pci_dev *pdev)
+{
+ struct pdsc *pdsc = pci_get_drvdata(pdev);
+ struct devlink *dl;
+
+ /* Unhook the registrations first to be sure there
+ * are no requests while we're stopping.
+ */
+ dl = priv_to_devlink(pdsc);
+ devl_lock(dl);
+ devl_unregister(dl);
+ if (!pdev->is_virtfn) {
+ if (pdsc->fw_reporter) {
+ devl_health_reporter_destroy(pdsc->fw_reporter);
+ pdsc->fw_reporter = NULL;
+ }
+ devl_params_unregister(dl, pdsc_dl_params,
+ ARRAY_SIZE(pdsc_dl_params));
+ }
+ devl_unlock(dl);
+
+ if (pdev->is_virtfn) {
+ struct pdsc *pf;
+
+ pf = pdsc_get_pf_struct(pdsc->pdev);
+ if (!IS_ERR(pf)) {
+ pdsc_auxbus_dev_del(pdsc, pf);
+ pf->vfs[pdsc->vf_id].vf = NULL;
+ }
+ } else {
+ /* Remove the VFs and their aux_bus connections before other
+ * cleanup so that the clients can use the AdminQ to cleanly
+ * shut themselves down.
+ */
+ pdsc_sriov_configure(pdev, 0);
+
+ del_timer_sync(&pdsc->wdtimer);
+ if (pdsc->wq)
+ destroy_workqueue(pdsc->wq);
+
+ mutex_lock(&pdsc->config_lock);
+ set_bit(PDSC_S_STOPPING_DRIVER, &pdsc->state);
+
+ pdsc_stop(pdsc);
+ pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
+ mutex_unlock(&pdsc->config_lock);
+ mutex_destroy(&pdsc->config_lock);
+ mutex_destroy(&pdsc->devcmd_lock);
+
+ pci_free_irq_vectors(pdev);
+ pdsc_unmap_bars(pdsc);
+ pci_release_regions(pdev);
+ }
+
+ pci_clear_master(pdev);
+ pci_disable_device(pdev);
+
+ ida_free(&pdsc_ida, pdsc->uid);
+ pdsc_debugfs_del_dev(pdsc);
+ devlink_free(dl);
+}
+
+static struct pci_driver pdsc_driver = {
+ .name = PDS_CORE_DRV_NAME,
+ .id_table = pdsc_id_table,
+ .probe = pdsc_probe,
+ .remove = pdsc_remove,
+ .sriov_configure = pdsc_sriov_configure,
+};
+
+void *pdsc_get_pf_struct(struct pci_dev *vf_pdev)
+{
+ return pci_iov_get_pf_drvdata(vf_pdev, &pdsc_driver);
+}
+EXPORT_SYMBOL_GPL(pdsc_get_pf_struct);
+
+static int __init pdsc_init_module(void)
+{
+ if (strcmp(KBUILD_MODNAME, PDS_CORE_DRV_NAME))
+ return -EINVAL;
+
+ pdsc_debugfs_create();
+ return pci_register_driver(&pdsc_driver);
+}
+
+static void __exit pdsc_cleanup_module(void)
+{
+ pci_unregister_driver(&pdsc_driver);
+ pdsc_debugfs_destroy();
+}
+
+module_init(pdsc_init_module);
+module_exit(pdsc_cleanup_module);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 92289ab..dcd9367 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2361,7 +2361,7 @@ static int bnxt_async_event_process(struct bnxt *bp,
case ASYNC_EVENT_CMPL_EVENT_ID_PHC_UPDATE: {
switch (BNXT_EVENT_PHC_EVENT_TYPE(data1)) {
case ASYNC_EVENT_CMPL_PHC_UPDATE_EVENT_DATA1_FLAGS_PHC_RTC_UPDATE:
- if (bp->fw_cap & BNXT_FW_CAP_PTP_RTC) {
+ if (BNXT_PTP_USE_RTC(bp)) {
struct bnxt_ptp_cfg *ptp = bp->ptp_cfg;
u64 ns;
@@ -3211,6 +3211,7 @@ static int bnxt_alloc_rx_page_pool(struct bnxt *bp,
pp.pool_size = bp->rx_ring_size;
pp.nid = dev_to_node(&bp->pdev->dev);
+ pp.napi = &rxr->bnapi->napi;
pp.dev = &bp->pdev->dev;
pp.dma_dir = DMA_BIDIRECTIONAL;
@@ -7600,7 +7601,7 @@ static int __bnxt_hwrm_ptp_qcfg(struct bnxt *bp)
u8 flags;
int rc;
- if (bp->hwrm_spec_code < 0x10801) {
+ if (bp->hwrm_spec_code < 0x10801 || !BNXT_CHIP_P5_THOR(bp)) {
rc = -ENODEV;
goto no_ptp;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index e7b5e28..852eb44 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -304,7 +304,7 @@ void bnxt_rdma_aux_device_uninit(struct bnxt *bp)
struct auxiliary_device *adev;
/* Skip if no auxiliary device init was done. */
- if (!(bp->flags & BNXT_FLAG_ROCE_CAP))
+ if (!bp->aux_priv)
return;
aux_priv = bp->aux_priv;
@@ -324,6 +324,7 @@ static void bnxt_aux_dev_release(struct device *dev)
bp->edev = NULL;
kfree(aux_priv->edev);
kfree(aux_priv);
+ bp->aux_priv = NULL;
}
static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp)
@@ -359,19 +360,18 @@ void bnxt_rdma_aux_device_init(struct bnxt *bp)
if (!(bp->flags & BNXT_FLAG_ROCE_CAP))
return;
- bp->aux_priv = kzalloc(sizeof(*bp->aux_priv), GFP_KERNEL);
- if (!bp->aux_priv)
+ aux_priv = kzalloc(sizeof(*bp->aux_priv), GFP_KERNEL);
+ if (!aux_priv)
goto exit;
- bp->aux_priv->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL);
- if (bp->aux_priv->id < 0) {
+ aux_priv->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL);
+ if (aux_priv->id < 0) {
netdev_warn(bp->dev,
"ida alloc failed for ROCE auxiliary device\n");
- kfree(bp->aux_priv);
+ kfree(aux_priv);
goto exit;
}
- aux_priv = bp->aux_priv;
aux_dev = &aux_priv->aux_dev;
aux_dev->id = aux_priv->id;
aux_dev->name = "rdma";
@@ -380,10 +380,11 @@ void bnxt_rdma_aux_device_init(struct bnxt *bp)
rc = auxiliary_device_init(aux_dev);
if (rc) {
- ida_free(&bnxt_aux_dev_ids, bp->aux_priv->id);
- kfree(bp->aux_priv);
+ ida_free(&bnxt_aux_dev_ids, aux_priv->id);
+ kfree(aux_priv);
goto exit;
}
+ bp->aux_priv = aux_priv;
/* From this point, all cleanup will happen via the .release callback &
* any error unwinding will need to include a call to
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index c1fc91c..cfbdd002 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -95,6 +95,8 @@
#define GEM_SA4B 0x00A0 /* Specific4 Bottom */
#define GEM_SA4T 0x00A4 /* Specific4 Top */
#define GEM_WOL 0x00b8 /* Wake on LAN */
+#define GEM_RXPTPUNI 0x00D4 /* PTP RX Unicast address */
+#define GEM_TXPTPUNI 0x00D8 /* PTP TX Unicast address */
#define GEM_EFTSH 0x00e8 /* PTP Event Frame Transmitted Seconds Register 47:32 */
#define GEM_EFRSH 0x00ec /* PTP Event Frame Received Seconds Register 47:32 */
#define GEM_PEFTSH 0x00f0 /* PTP Peer Event Frame Transmitted Seconds Register 47:32 */
@@ -245,6 +247,8 @@
#define MACB_TZQ_OFFSET 12 /* Transmit zero quantum pause frame */
#define MACB_TZQ_SIZE 1
#define MACB_SRTSM_OFFSET 15 /* Store Receive Timestamp to Memory */
+#define MACB_PTPUNI_OFFSET 20 /* PTP Unicast packet enable */
+#define MACB_PTPUNI_SIZE 1
#define MACB_OSSMODE_OFFSET 24 /* Enable One Step Synchro Mode */
#define MACB_OSSMODE_SIZE 1
#define MACB_MIIONRGMII_OFFSET 28 /* MII Usage on RGMII Interface */
@@ -1363,7 +1367,7 @@ static inline bool macb_is_gem(struct macb *bp)
static inline bool gem_has_ptp(struct macb *bp)
{
- return !!(bp->caps & MACB_CAPS_GEM_HAS_PTP);
+ return IS_ENABLED(CONFIG_MACB_USE_HWSTAMP) && (bp->caps & MACB_CAPS_GEM_HAS_PTP);
}
/**
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 541e4dd..29a1199d 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -287,6 +287,11 @@ static void macb_set_hwaddr(struct macb *bp)
top = cpu_to_le16(*((u16 *)(bp->dev->dev_addr + 4)));
macb_or_gem_writel(bp, SA1T, top);
+ if (gem_has_ptp(bp)) {
+ gem_writel(bp, RXPTPUNI, bottom);
+ gem_writel(bp, TXPTPUNI, bottom);
+ }
+
/* Clear unused address register sets */
macb_or_gem_writel(bp, SA2B, 0);
macb_or_gem_writel(bp, SA2T, 0);
@@ -773,8 +778,12 @@ static void macb_mac_link_up(struct phylink_config *config,
spin_unlock_irqrestore(&bp->lock, flags);
- /* Enable Rx and Tx */
- macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(RE) | MACB_BIT(TE));
+ /* Enable Rx and Tx; Enable PTP unicast */
+ ctrl = macb_readl(bp, NCR);
+ if (gem_has_ptp(bp))
+ ctrl |= MACB_BIT(PTPUNI);
+
+ macb_writel(bp, NCR, ctrl | MACB_BIT(RE) | MACB_BIT(TE));
netif_tx_wake_all_queues(ndev);
}
@@ -3893,17 +3902,17 @@ static void macb_configure_caps(struct macb *bp,
dcfg = gem_readl(bp, DCFG2);
if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0)
bp->caps |= MACB_CAPS_FIFO_MODE;
-#ifdef CONFIG_MACB_USE_HWSTAMP
if (gem_has_ptp(bp)) {
if (!GEM_BFEXT(TSU, gem_readl(bp, DCFG5)))
dev_err(&bp->pdev->dev,
"GEM doesn't support hardware ptp.\n");
else {
+#ifdef CONFIG_MACB_USE_HWSTAMP
bp->hw_dma_cap |= HW_DMA_CAP_PTP;
bp->ptp_info = &gem_ptp_info;
+#endif
}
}
-#endif
}
dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps);
diff --git a/drivers/net/ethernet/cadence/macb_ptp.c b/drivers/net/ethernet/cadence/macb_ptp.c
index f962a95..51d26fa 100644
--- a/drivers/net/ethernet/cadence/macb_ptp.c
+++ b/drivers/net/ethernet/cadence/macb_ptp.c
@@ -258,6 +258,8 @@ static int gem_hw_timestamp(struct macb *bp, u32 dma_desc_ts_1,
*/
gem_tsu_get_time(&bp->ptp_clock_info, &tsu, NULL);
+ ts->tv_sec |= ((~GEM_DMA_SEC_MASK) & tsu.tv_sec);
+
/* If the top bit is set in the timestamp,
* but not in 1588 timer, it has rolled over,
* so subtract max size
@@ -266,8 +268,6 @@ static int gem_hw_timestamp(struct macb *bp, u32 dma_desc_ts_1,
!(tsu.tv_sec & (GEM_DMA_SEC_TOP >> 1)))
ts->tv_sec -= GEM_DMA_SEC_TOP;
- ts->tv_sec += ((~GEM_DMA_SEC_MASK) & tsu.tv_sec);
-
return 0;
}
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
index dd9be22..d354115 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
@@ -1135,7 +1135,7 @@ void cxgb4_cleanup_tc_flower(struct adapter *adap)
return;
if (adap->flower_stats_timer.function)
- del_timer_sync(&adap->flower_stats_timer);
+ timer_shutdown_sync(&adap->flower_stats_timer);
cancel_work_sync(&adap->flower_stats_work);
rhashtable_destroy(&adap->flower_tbl);
adap->tc_flower_initialized = false;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index 2fc712b..3c4fa26 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -25,6 +25,13 @@ void enetc_port_mac_wr(struct enetc_si *si, u32 reg, u32 val)
}
EXPORT_SYMBOL_GPL(enetc_port_mac_wr);
+static void enetc_change_preemptible_tcs(struct enetc_ndev_priv *priv,
+ u8 preemptible_tcs)
+{
+ priv->preemptible_tcs = preemptible_tcs;
+ enetc_mm_commit_preemptible_tcs(priv);
+}
+
static int enetc_num_stack_tx_queues(struct enetc_ndev_priv *priv)
{
int num_tx_rings = priv->num_tx_rings;
@@ -2640,16 +2647,19 @@ static void enetc_reset_tc_mqprio(struct net_device *ndev)
}
enetc_debug_tx_ring_prios(priv);
+
+ enetc_change_preemptible_tcs(priv, 0);
}
int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
{
+ struct tc_mqprio_qopt_offload *mqprio = type_data;
struct enetc_ndev_priv *priv = netdev_priv(ndev);
- struct tc_mqprio_qopt *mqprio = type_data;
+ struct tc_mqprio_qopt *qopt = &mqprio->qopt;
struct enetc_hw *hw = &priv->si->hw;
int num_stack_tx_queues = 0;
- u8 num_tc = mqprio->num_tc;
struct enetc_bdr *tx_ring;
+ u8 num_tc = qopt->num_tc;
int offset, count;
int err, tc, q;
@@ -2663,8 +2673,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
return err;
for (tc = 0; tc < num_tc; tc++) {
- offset = mqprio->offset[tc];
- count = mqprio->count[tc];
+ offset = qopt->offset[tc];
+ count = qopt->count[tc];
num_stack_tx_queues += count;
err = netdev_set_tc_queue(ndev, tc, count, offset);
@@ -2693,6 +2703,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
enetc_debug_tx_ring_prios(priv);
+ enetc_change_preemptible_tcs(priv, mqprio->preemptible_tcs);
+
return 0;
err_reset_tc:
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h
index 8010f31..c97a8e3 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc.h
@@ -355,6 +355,9 @@ struct enetc_ndev_priv {
u16 rx_bd_count, tx_bd_count;
u16 msg_enable;
+
+ u8 preemptible_tcs;
+
enum enetc_active_offloads active_offloads;
u32 speed; /* store speed for compare update pspeed */
@@ -433,6 +436,7 @@ int enetc_xdp_xmit(struct net_device *ndev, int num_frames,
/* ethtool */
void enetc_set_ethtool_ops(struct net_device *ndev);
void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link);
+void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv);
/* control buffer descriptor ring (CBDR) */
int enetc_setup_cbdr(struct device *dev, struct enetc_hw *hw, int bd_count,
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
index 838750a..e993ed0 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
@@ -32,6 +32,12 @@ static const u32 enetc_port_regs[] = {
ENETC_PM0_CMD_CFG, ENETC_PM0_MAXFRM, ENETC_PM0_IF_MODE
};
+static const u32 enetc_port_mm_regs[] = {
+ ENETC_MMCSR, ENETC_PFPMR, ENETC_PTCFPR(0), ENETC_PTCFPR(1),
+ ENETC_PTCFPR(2), ENETC_PTCFPR(3), ENETC_PTCFPR(4), ENETC_PTCFPR(5),
+ ENETC_PTCFPR(6), ENETC_PTCFPR(7),
+};
+
static int enetc_get_reglen(struct net_device *ndev)
{
struct enetc_ndev_priv *priv = netdev_priv(ndev);
@@ -45,6 +51,9 @@ static int enetc_get_reglen(struct net_device *ndev)
if (hw->port)
len += ARRAY_SIZE(enetc_port_regs);
+ if (hw->port && !!(priv->si->hw_features & ENETC_SI_F_QBU))
+ len += ARRAY_SIZE(enetc_port_mm_regs);
+
len *= sizeof(u32) * 2; /* store 2 entries per reg: addr and value */
return len;
@@ -90,6 +99,14 @@ static void enetc_get_regs(struct net_device *ndev, struct ethtool_regs *regs,
*buf++ = addr;
*buf++ = enetc_rd(hw, addr);
}
+
+ if (priv->si->hw_features & ENETC_SI_F_QBU) {
+ for (i = 0; i < ARRAY_SIZE(enetc_port_mm_regs); i++) {
+ addr = ENETC_PORT_BASE + enetc_port_mm_regs[i];
+ *buf++ = addr;
+ *buf++ = enetc_rd(hw, addr);
+ }
+ }
}
static const struct {
@@ -976,7 +993,9 @@ static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state)
lafs = ENETC_MMCSR_GET_LAFS(val);
state->rx_min_frag_size = ethtool_mm_frag_size_add_to_min(lafs);
state->tx_enabled = !!(val & ENETC_MMCSR_LPE); /* mirror of MMCSR_ME */
- state->tx_active = !!(val & ENETC_MMCSR_LPA);
+ state->tx_active = state->tx_enabled &&
+ (state->verify_status == ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED ||
+ state->verify_status == ETHTOOL_MM_VERIFY_STATUS_DISABLED);
state->verify_enabled = !(val & ENETC_MMCSR_VDIS);
state->verify_time = ENETC_MMCSR_GET_VT(val);
/* A verifyTime of 128 ms would exceed the 7 bit width
@@ -989,6 +1008,64 @@ static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state)
return 0;
}
+static int enetc_mm_wait_tx_active(struct enetc_hw *hw, int verify_time)
+{
+ int timeout = verify_time * USEC_PER_MSEC * ENETC_MM_VERIFY_RETRIES;
+ u32 val;
+
+ /* This will time out after the standard value of 3 verification
+ * attempts. To not sleep forever, it relies on a non-zero verify_time,
+ * guarantee which is provided by the ethtool nlattr policy.
+ */
+ return read_poll_timeout(enetc_port_rd, val,
+ ENETC_MMCSR_GET_VSTS(val) == 3,
+ ENETC_MM_VERIFY_SLEEP_US, timeout,
+ true, hw, ENETC_MMCSR);
+}
+
+static void enetc_set_ptcfpr(struct enetc_hw *hw, u8 preemptible_tcs)
+{
+ u32 val;
+ int tc;
+
+ for (tc = 0; tc < 8; tc++) {
+ val = enetc_port_rd(hw, ENETC_PTCFPR(tc));
+
+ if (preemptible_tcs & BIT(tc))
+ val |= ENETC_PTCFPR_FPE;
+ else
+ val &= ~ENETC_PTCFPR_FPE;
+
+ enetc_port_wr(hw, ENETC_PTCFPR(tc), val);
+ }
+}
+
+/* ENETC does not have an IRQ to notify changes to the MAC Merge TX status
+ * (active/inactive), but the preemptible traffic classes should only be
+ * committed to hardware once TX is active. Resort to polling.
+ */
+void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv)
+{
+ struct enetc_hw *hw = &priv->si->hw;
+ u8 preemptible_tcs = 0;
+ u32 val;
+ int err;
+
+ val = enetc_port_rd(hw, ENETC_MMCSR);
+ if (!(val & ENETC_MMCSR_ME))
+ goto out;
+
+ if (!(val & ENETC_MMCSR_VDIS)) {
+ err = enetc_mm_wait_tx_active(hw, ENETC_MMCSR_GET_VT(val));
+ if (err)
+ goto out;
+ }
+
+ preemptible_tcs = priv->preemptible_tcs;
+out:
+ enetc_set_ptcfpr(hw, preemptible_tcs);
+}
+
/* FIXME: Workaround for the link partner's verification failing if ENETC
* priorly received too much express traffic. The documentation doesn't
* suggest this is needed.
@@ -1041,10 +1118,13 @@ static int enetc_set_mm(struct net_device *ndev, struct ethtool_mm_cfg *cfg,
else
priv->active_offloads &= ~ENETC_F_QBU;
- /* If link is up, enable MAC Merge right away */
- if (!!(priv->active_offloads & ENETC_F_QBU) &&
- !(val & ENETC_MMCSR_LINK_FAIL))
- val |= ENETC_MMCSR_ME;
+ /* If link is up, enable/disable MAC Merge right away */
+ if (!(val & ENETC_MMCSR_LINK_FAIL)) {
+ if (!!(priv->active_offloads & ENETC_F_QBU))
+ val |= ENETC_MMCSR_ME;
+ else
+ val &= ~ENETC_MMCSR_ME;
+ }
val &= ~ENETC_MMCSR_VT_MASK;
val |= ENETC_MMCSR_VT(cfg->verify_time);
@@ -1056,6 +1136,8 @@ static int enetc_set_mm(struct net_device *ndev, struct ethtool_mm_cfg *cfg,
enetc_restart_emac_rx(priv->si);
+ enetc_mm_commit_preemptible_tcs(priv);
+
mutex_unlock(&priv->mm_lock);
return 0;
@@ -1089,6 +1171,8 @@ void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link)
enetc_port_wr(hw, ENETC_MMCSR, val);
+ enetc_mm_commit_preemptible_tcs(priv);
+
mutex_unlock(&priv->mm_lock);
}
EXPORT_SYMBOL_GPL(enetc_mm_link_state_update);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h
index de2e0ee..1619943 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h
@@ -3,6 +3,9 @@
#include <linux/bitops.h>
+#define ENETC_MM_VERIFY_SLEEP_US USEC_PER_MSEC
+#define ENETC_MM_VERIFY_RETRIES 3
+
/* ENETC device IDs */
#define ENETC_DEV_ID_PF 0xe100
#define ENETC_DEV_ID_VF 0xef00
@@ -965,6 +968,10 @@ static inline u32 enetc_usecs_to_cycles(u32 usecs)
return (u32)div_u64(usecs * ENETC_CLK, 1000000ULL);
}
+/* Port traffic class frame preemption register */
+#define ENETC_PTCFPR(n) (0x1910 + (n) * 4) /* n = [0 ..7] */
+#define ENETC_PTCFPR_FPE BIT(31)
+
/* port time gating control register */
#define ENETC_PTGCR 0x11a00
#define ENETC_PTGCR_TGE BIT(31)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 6f5c16a..bd7ef59 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -5287,31 +5287,6 @@ static void e1000_watchdog_task(struct work_struct *work)
ew32(TARC(0), tarc0);
}
- /* disable TSO for pcie and 10/100 speeds, to avoid
- * some hardware issues
- */
- if (!(adapter->flags & FLAG_TSO_FORCE)) {
- switch (adapter->link_speed) {
- case SPEED_10:
- case SPEED_100:
- e_info("10/100 speed: disabling TSO\n");
- netdev->features &= ~NETIF_F_TSO;
- netdev->features &= ~NETIF_F_TSO6;
- break;
- case SPEED_1000:
- netdev->features |= NETIF_F_TSO;
- netdev->features |= NETIF_F_TSO6;
- break;
- default:
- /* oops */
- break;
- }
- if (hw->mac.type == e1000_pch_spt) {
- netdev->features &= ~NETIF_F_TSO;
- netdev->features &= ~NETIF_F_TSO6;
- }
- }
-
/* enable transmits in the hardware, need to do this
* after setting TARC(0)
*/
@@ -7525,6 +7500,32 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
NETIF_F_RXCSUM |
NETIF_F_HW_CSUM);
+ /* disable TSO for pcie and 10/100 speeds to avoid
+ * some hardware issues and for i219 to fix transfer
+ * speed being capped at 60%
+ */
+ if (!(adapter->flags & FLAG_TSO_FORCE)) {
+ switch (adapter->link_speed) {
+ case SPEED_10:
+ case SPEED_100:
+ e_info("10/100 speed: disabling TSO\n");
+ netdev->features &= ~NETIF_F_TSO;
+ netdev->features &= ~NETIF_F_TSO6;
+ break;
+ case SPEED_1000:
+ netdev->features |= NETIF_F_TSO;
+ netdev->features |= NETIF_F_TSO6;
+ break;
+ default:
+ /* oops */
+ break;
+ }
+ if (hw->mac.type == e1000_pch_spt) {
+ netdev->features &= ~NETIF_F_TSO;
+ netdev->features &= ~NETIF_F_TSO6;
+ }
+ }
+
/* Set user-changeable features (subset of all device features) */
netdev->hw_features = netdev->features;
netdev->hw_features |= NETIF_F_RXFCS;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index c8ff567..b847bd1 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -11072,8 +11072,11 @@ static void i40e_rebuild(struct i40e_pf *pf, bool reinit, bool lock_acquired)
pf->hw.aq.asq_last_status));
}
/* reinit the misc interrupt */
- if (pf->flags & I40E_FLAG_MSIX_ENABLED)
+ if (pf->flags & I40E_FLAG_MSIX_ENABLED) {
ret = i40e_setup_misc_vector(pf);
+ if (ret)
+ goto end_unlock;
+ }
/* Add a filter to drop all Flow control frames from any VSI from being
* transmitted. By doing so we stop a malicious VF from sending out
@@ -14147,15 +14150,15 @@ static int i40e_add_vsi(struct i40e_vsi *vsi)
vsi->id = ctxt.vsi_number;
}
- vsi->active_filters = 0;
- clear_bit(__I40E_VSI_OVERFLOW_PROMISC, vsi->state);
spin_lock_bh(&vsi->mac_filter_hash_lock);
+ vsi->active_filters = 0;
/* If macvlan filters already exist, force them to get loaded */
hash_for_each_safe(vsi->mac_filter_hash, bkt, h, f, hlist) {
f->state = I40E_FILTER_NEW;
f_count++;
}
spin_unlock_bh(&vsi->mac_filter_hash_lock);
+ clear_bit(__I40E_VSI_OVERFLOW_PROMISC, vsi->state);
if (f_count) {
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 6c2f1d4..ca3c66c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -75,7 +75,7 @@
esw/acl/egress_lgcy.o esw/acl/egress_ofld.o \
esw/acl/ingress_lgcy.o esw/acl/ingress_ofld.o
-mlx5_core-$(CONFIG_MLX5_BRIDGE) += esw/bridge.o en/rep/bridge.o
+mlx5_core-$(CONFIG_MLX5_BRIDGE) += esw/bridge.o esw/bridge_mcast.o en/rep/bridge.o
mlx5_core-$(CONFIG_THERMAL) += thermal.o
mlx5_core-$(CONFIG_MLX5_MPFS) += lib/mpfs.o
@@ -112,8 +112,8 @@
steering/dr_ste_v2.o \
steering/dr_cmd.o steering/dr_fw.o \
steering/dr_action.o steering/fs_dr.o \
- steering/dr_definer.o \
- steering/dr_dbg.o lib/smfs.o
+ steering/dr_definer.o steering/dr_ptrn.o \
+ steering/dr_arg.o steering/dr_dbg.o lib/smfs.o
#
# SF device
#
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
index b00e33e..d53de39 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -1802,7 +1802,7 @@ static struct mlx5_cmd_msg *alloc_msg(struct mlx5_core_dev *dev, int in_size,
if (in_size <= 16)
goto cache_miss;
- for (i = 0; i < MLX5_NUM_COMMAND_CACHES; i++) {
+ for (i = 0; i < dev->profile.num_cmd_caches; i++) {
ch = &cmd->cache[i];
if (in_size > ch->max_inbox_size)
continue;
@@ -2097,7 +2097,7 @@ static void destroy_msg_cache(struct mlx5_core_dev *dev)
struct mlx5_cmd_msg *n;
int i;
- for (i = 0; i < MLX5_NUM_COMMAND_CACHES; i++) {
+ for (i = 0; i < dev->profile.num_cmd_caches; i++) {
ch = &dev->cmd.cache[i];
list_for_each_entry_safe(msg, n, &ch->head, list) {
list_del(&msg->list);
@@ -2127,7 +2127,7 @@ static void create_msg_cache(struct mlx5_core_dev *dev)
int k;
/* Initialize and fill the caches with initial entries */
- for (k = 0; k < MLX5_NUM_COMMAND_CACHES; k++) {
+ for (k = 0; k < dev->profile.num_cmd_caches; k++) {
ch = &cmd->cache[k];
spin_lock_init(&ch->lock);
INIT_LIST_HEAD(&ch->head);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c
index e7739ac..1b33533 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c
@@ -60,9 +60,6 @@ bool mlx5_eth_supported(struct mlx5_core_dev *dev)
if (!IS_ENABLED(CONFIG_MLX5_CORE_EN))
return false;
- if (mlx5_core_is_management_pf(dev))
- return false;
-
if (MLX5_CAP_GEN(dev, port_type) != MLX5_CAP_PORT_TYPE_ETH)
return false;
@@ -191,9 +188,6 @@ bool mlx5_rdma_supported(struct mlx5_core_dev *dev)
if (!IS_ENABLED(CONFIG_MLX5_INFINIBAND))
return false;
- if (mlx5_core_is_management_pf(dev))
- return false;
-
if (dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_IB_ADEV)
return false;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
index 7c9c4e4..d000236 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
@@ -75,10 +75,6 @@ int mlx5_ec_init(struct mlx5_core_dev *dev)
if (!mlx5_core_is_ecpf(dev))
return 0;
- /* Management PF don't have a peer PF */
- if (mlx5_core_is_management_pf(dev))
- return 0;
-
return mlx5_host_pf_init(dev);
}
@@ -89,10 +85,6 @@ void mlx5_ec_cleanup(struct mlx5_core_dev *dev)
if (!mlx5_core_is_ecpf(dev))
return;
- /* Management PF don't have a peer PF */
- if (mlx5_core_is_management_pf(dev))
- return;
-
mlx5_host_pf_cleanup(dev);
err = mlx5_wait_for_pages(dev, &dev->priv.page_counters[MLX5_HOST_PF]);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index ba615b7..b8987a4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -475,59 +475,18 @@ struct mlx5e_txqsq {
cqe_ts_to_ns ptp_cyc2time;
} ____cacheline_aligned_in_smp;
-/* XDP packets can be transmitted in different ways. On completion, we need to
- * distinguish between them to clean up things in a proper way.
- */
-enum mlx5e_xdp_xmit_mode {
- /* An xdp_frame was transmitted due to either XDP_REDIRECT from another
- * device or XDP_TX from an XSK RQ. The frame has to be unmapped and
- * returned.
- */
- MLX5E_XDP_XMIT_MODE_FRAME,
-
- /* The xdp_frame was created in place as a result of XDP_TX from a
- * regular RQ. No DMA remapping happened, and the page belongs to us.
- */
- MLX5E_XDP_XMIT_MODE_PAGE,
-
- /* No xdp_frame was created at all, the transmit happened from a UMEM
- * page. The UMEM Completion Ring producer pointer has to be increased.
- */
- MLX5E_XDP_XMIT_MODE_XSK,
-};
-
-struct mlx5e_xdp_info {
- enum mlx5e_xdp_xmit_mode mode;
- union {
- struct {
- struct xdp_frame *xdpf;
- dma_addr_t dma_addr;
- } frame;
- struct {
- struct mlx5e_rq *rq;
- struct page *page;
- } page;
- };
-};
-
-struct mlx5e_xmit_data {
- dma_addr_t dma_addr;
- void *data;
- u32 len;
-};
-
struct mlx5e_xdp_info_fifo {
- struct mlx5e_xdp_info *xi;
+ union mlx5e_xdp_info *xi;
u32 *cc;
u32 *pc;
u32 mask;
};
struct mlx5e_xdpsq;
+struct mlx5e_xmit_data;
typedef int (*mlx5e_fp_xmit_xdp_frame_check)(struct mlx5e_xdpsq *);
typedef bool (*mlx5e_fp_xmit_xdp_frame)(struct mlx5e_xdpsq *,
struct mlx5e_xmit_data *,
- struct skb_shared_info *,
int);
struct mlx5e_xdpsq {
@@ -628,6 +587,7 @@ union mlx5e_alloc_units {
struct mlx5e_mpw_info {
u16 consumed_strides;
DECLARE_BITMAP(skip_release_bitmap, MLX5_MPWRQ_MAX_PAGES_PER_WQE);
+ struct mlx5e_frag_page linear_page;
union mlx5e_alloc_units alloc_units;
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index 31f3c6e..ef546ed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -253,17 +253,20 @@ static u32 mlx5e_rx_get_linear_stride_sz(struct mlx5_core_dev *mdev,
struct mlx5e_xsk_param *xsk,
bool mpwqe)
{
+ u32 sz;
+
/* XSK frames are mapped as individual pages, because frames may come in
* an arbitrary order from random locations in the UMEM.
*/
if (xsk)
return mpwqe ? 1 << mlx5e_mpwrq_page_shift(mdev, xsk) : PAGE_SIZE;
- /* XDP in mlx5e doesn't support multiple packets per page. */
- if (params->xdp_prog)
- return PAGE_SIZE;
+ sz = roundup_pow_of_two(mlx5e_rx_get_linear_sz_skb(params, false));
- return roundup_pow_of_two(mlx5e_rx_get_linear_sz_skb(params, false));
+ /* XDP in mlx5e doesn't support multiple packets per page.
+ * Do not assume sz <= PAGE_SIZE if params->xdp_prog is set.
+ */
+ return params->xdp_prog && sz < PAGE_SIZE ? PAGE_SIZE : sz;
}
static u8 mlx5e_mpwqe_log_pkts_per_wqe(struct mlx5_core_dev *mdev,
@@ -320,6 +323,20 @@ static bool mlx5e_verify_rx_mpwqe_strides(struct mlx5_core_dev *mdev,
return log_num_strides >= MLX5_MPWQE_LOG_NUM_STRIDES_BASE;
}
+bool mlx5e_verify_params_rx_mpwqe_strides(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params,
+ struct mlx5e_xsk_param *xsk)
+{
+ u8 log_wqe_num_of_strides = mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk);
+ u8 log_wqe_stride_size = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk);
+ enum mlx5e_mpwrq_umr_mode umr_mode = mlx5e_mpwrq_umr_mode(mdev, xsk);
+ u8 page_shift = mlx5e_mpwrq_page_shift(mdev, xsk);
+
+ return mlx5e_verify_rx_mpwqe_strides(mdev, log_wqe_stride_size,
+ log_wqe_num_of_strides,
+ page_shift, umr_mode);
+}
+
bool mlx5e_rx_mpwqe_is_linear_skb(struct mlx5_core_dev *mdev,
struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk)
@@ -402,6 +419,10 @@ u8 mlx5e_mpwqe_get_log_stride_size(struct mlx5_core_dev *mdev,
if (mlx5e_rx_mpwqe_is_linear_skb(mdev, params, xsk))
return order_base_2(mlx5e_rx_get_linear_stride_sz(mdev, params, xsk, true));
+ /* XDP in mlx5e doesn't support multiple packets per page. */
+ if (params->xdp_prog)
+ return PAGE_SHIFT;
+
return MLX5_MPWRQ_DEF_LOG_STRIDE_SZ(mdev);
}
@@ -572,9 +593,6 @@ int mlx5e_mpwrq_validate_regular(struct mlx5_core_dev *mdev, struct mlx5e_params
if (!mlx5e_check_fragmented_striding_rq_cap(mdev, page_shift, umr_mode))
return -EOPNOTSUPP;
- if (params->xdp_prog && !mlx5e_rx_mpwqe_is_linear_skb(mdev, params, NULL))
- return -EINVAL;
-
return 0;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
index c9be6eb..a5d20f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
@@ -153,6 +153,9 @@ int mlx5e_build_channel_param(struct mlx5_core_dev *mdev,
u16 mlx5e_calc_sq_stop_room(struct mlx5_core_dev *mdev, struct mlx5e_params *params);
int mlx5e_validate_params(struct mlx5_core_dev *mdev, struct mlx5e_params *params);
+bool mlx5e_verify_params_rx_mpwqe_strides(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params,
+ struct mlx5e_xsk_param *xsk);
static inline void mlx5e_params_print_info(struct mlx5_core_dev *mdev,
struct mlx5e_params *params)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
index ce85b48..fd19192 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
@@ -220,6 +220,7 @@ mlx5_esw_bridge_port_obj_add(struct net_device *dev,
struct netlink_ext_ack *extack = switchdev_notifier_info_to_extack(&port_obj_info->info);
const struct switchdev_obj *obj = port_obj_info->obj;
const struct switchdev_obj_port_vlan *vlan;
+ const struct switchdev_obj_port_mdb *mdb;
u16 vport_num, esw_owner_vhca_id;
int err;
@@ -235,6 +236,11 @@ mlx5_esw_bridge_port_obj_add(struct net_device *dev,
err = mlx5_esw_bridge_port_vlan_add(vport_num, esw_owner_vhca_id, vlan->vid,
vlan->flags, br_offloads, extack);
break;
+ case SWITCHDEV_OBJ_ID_PORT_MDB:
+ mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
+ err = mlx5_esw_bridge_port_mdb_add(dev, vport_num, esw_owner_vhca_id, mdb->addr,
+ mdb->vid, br_offloads, extack);
+ break;
default:
return -EOPNOTSUPP;
}
@@ -248,6 +254,7 @@ mlx5_esw_bridge_port_obj_del(struct net_device *dev,
{
const struct switchdev_obj *obj = port_obj_info->obj;
const struct switchdev_obj_port_vlan *vlan;
+ const struct switchdev_obj_port_mdb *mdb;
u16 vport_num, esw_owner_vhca_id;
if (!mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, br_offloads->esw, &vport_num,
@@ -261,6 +268,11 @@ mlx5_esw_bridge_port_obj_del(struct net_device *dev,
vlan = SWITCHDEV_OBJ_PORT_VLAN(obj);
mlx5_esw_bridge_port_vlan_del(vport_num, esw_owner_vhca_id, vlan->vid, br_offloads);
break;
+ case SWITCHDEV_OBJ_ID_PORT_MDB:
+ mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
+ mlx5_esw_bridge_port_mdb_del(dev, vport_num, esw_owner_vhca_id, mdb->addr, mdb->vid,
+ br_offloads);
+ break;
default:
return -EOPNOTSUPP;
}
@@ -306,6 +318,10 @@ mlx5_esw_bridge_port_obj_attr_set(struct net_device *dev,
attr->u.vlan_protocol,
br_offloads);
break;
+ case SWITCHDEV_ATTR_ID_BRIDGE_MC_DISABLED:
+ err = mlx5_esw_bridge_mcast_set(vport_num, esw_owner_vhca_id,
+ !attr->u.mc_disabled, br_offloads);
+ break;
default:
err = -EOPNOTSUPP;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index 651be7a..47381e9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -77,6 +77,19 @@ static inline bool mlx5e_rx_hw_stamp(struct hwtstamp_config *config)
}
/* TX */
+struct mlx5e_xmit_data {
+ dma_addr_t dma_addr;
+ void *data;
+ u32 len : 31;
+ u32 has_frags : 1;
+};
+
+struct mlx5e_xmit_data_frags {
+ struct mlx5e_xmit_data xd;
+ struct skb_shared_info *sinfo;
+ dma_addr_t *dma_arr;
+};
+
netdev_tx_t mlx5e_xmit(struct sk_buff *skb, struct net_device *dev);
bool mlx5e_poll_tx_cq(struct mlx5e_cq *cq, int napi_budget);
void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
index c8b532c..f0e6095 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
@@ -61,9 +61,8 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
struct xdp_buff *xdp)
{
struct page *page = virt_to_page(xdp->data);
- struct skb_shared_info *sinfo = NULL;
- struct mlx5e_xmit_data xdptxd;
- struct mlx5e_xdp_info xdpi;
+ struct mlx5e_xmit_data_frags xdptxdf = {};
+ struct mlx5e_xmit_data *xdptxd;
struct xdp_frame *xdpf;
dma_addr_t dma_addr;
int i;
@@ -72,8 +71,10 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
if (unlikely(!xdpf))
return false;
- xdptxd.data = xdpf->data;
- xdptxd.len = xdpf->len;
+ xdptxd = &xdptxdf.xd;
+ xdptxd->data = xdpf->data;
+ xdptxd->len = xdpf->len;
+ xdptxd->has_frags = xdp_frame_has_frags(xdpf);
if (xdp->rxq->mem.type == MEM_TYPE_XSK_BUFF_POOL) {
/* The xdp_buff was in the UMEM and was copied into a newly
@@ -88,24 +89,29 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
*/
__set_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags); /* non-atomic */
- xdpi.mode = MLX5E_XDP_XMIT_MODE_FRAME;
+ if (unlikely(xdptxd->has_frags))
+ return false;
- dma_addr = dma_map_single(sq->pdev, xdptxd.data, xdptxd.len,
+ dma_addr = dma_map_single(sq->pdev, xdptxd->data, xdptxd->len,
DMA_TO_DEVICE);
if (dma_mapping_error(sq->pdev, dma_addr)) {
xdp_return_frame(xdpf);
return false;
}
- xdptxd.dma_addr = dma_addr;
- xdpi.frame.xdpf = xdpf;
- xdpi.frame.dma_addr = dma_addr;
+ xdptxd->dma_addr = dma_addr;
if (unlikely(!INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
- mlx5e_xmit_xdp_frame, sq, &xdptxd, NULL, 0)))
+ mlx5e_xmit_xdp_frame, sq, xdptxd, 0)))
return false;
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, &xdpi);
+ /* xmit_mode == MLX5E_XDP_XMIT_MODE_FRAME */
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .mode = MLX5E_XDP_XMIT_MODE_FRAME });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .frame.xdpf = xdpf });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .frame.dma_addr = dma_addr });
return true;
}
@@ -115,17 +121,15 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
* mode.
*/
- xdpi.mode = MLX5E_XDP_XMIT_MODE_PAGE;
- xdpi.page.rq = rq;
-
dma_addr = page_pool_get_dma_addr(page) + (xdpf->data - (void *)xdpf);
- dma_sync_single_for_device(sq->pdev, dma_addr, xdptxd.len, DMA_BIDIRECTIONAL);
+ dma_sync_single_for_device(sq->pdev, dma_addr, xdptxd->len, DMA_BIDIRECTIONAL);
- if (unlikely(xdp_frame_has_frags(xdpf))) {
- sinfo = xdp_get_shared_info_from_frame(xdpf);
+ if (xdptxd->has_frags) {
+ xdptxdf.sinfo = xdp_get_shared_info_from_frame(xdpf);
+ xdptxdf.dma_arr = NULL;
- for (i = 0; i < sinfo->nr_frags; i++) {
- skb_frag_t *frag = &sinfo->frags[i];
+ for (i = 0; i < xdptxdf.sinfo->nr_frags; i++) {
+ skb_frag_t *frag = &xdptxdf.sinfo->frags[i];
dma_addr_t addr;
u32 len;
@@ -137,22 +141,34 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
}
}
- xdptxd.dma_addr = dma_addr;
+ xdptxd->dma_addr = dma_addr;
if (unlikely(!INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
- mlx5e_xmit_xdp_frame, sq, &xdptxd, sinfo, 0)))
+ mlx5e_xmit_xdp_frame, sq, xdptxd, 0)))
return false;
- xdpi.page.page = page;
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, &xdpi);
+ /* xmit_mode == MLX5E_XDP_XMIT_MODE_PAGE */
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .mode = MLX5E_XDP_XMIT_MODE_PAGE });
- if (unlikely(xdp_frame_has_frags(xdpf))) {
- for (i = 0; i < sinfo->nr_frags; i++) {
- skb_frag_t *frag = &sinfo->frags[i];
+ if (xdptxd->has_frags) {
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info)
+ { .page.num = 1 + xdptxdf.sinfo->nr_frags });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .page.page = page });
+ for (i = 0; i < xdptxdf.sinfo->nr_frags; i++) {
+ skb_frag_t *frag = &xdptxdf.sinfo->frags[i];
- xdpi.page.page = skb_frag_page(frag);
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, &xdpi);
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info)
+ { .page.page = skb_frag_page(frag) });
}
+ } else {
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .page.num = 1 });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .page.page = page });
}
return true;
@@ -381,26 +397,43 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq
INDIRECT_CALLABLE_SCOPE bool
mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd,
- struct skb_shared_info *sinfo, int check_result);
+ int check_result);
INDIRECT_CALLABLE_SCOPE bool
mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd,
- struct skb_shared_info *sinfo, int check_result)
+ int check_result)
{
struct mlx5e_tx_mpwqe *session = &sq->mpwqe;
struct mlx5e_xdpsq_stats *stats = sq->stats;
+ struct mlx5e_xmit_data *p = xdptxd;
+ struct mlx5e_xmit_data tmp;
- if (unlikely(sinfo)) {
- /* MPWQE is enabled, but a multi-buffer packet is queued for
- * transmission. MPWQE can't send fragmented packets, so close
- * the current session and fall back to a regular WQE.
- */
- if (unlikely(sq->mpwqe.wqe))
- mlx5e_xdp_mpwqe_complete(sq);
- return mlx5e_xmit_xdp_frame(sq, xdptxd, sinfo, 0);
+ if (xdptxd->has_frags) {
+ struct mlx5e_xmit_data_frags *xdptxdf =
+ container_of(xdptxd, struct mlx5e_xmit_data_frags, xd);
+
+ if (!!xdptxd->len + xdptxdf->sinfo->nr_frags > 1) {
+ /* MPWQE is enabled, but a multi-buffer packet is queued for
+ * transmission. MPWQE can't send fragmented packets, so close
+ * the current session and fall back to a regular WQE.
+ */
+ if (unlikely(sq->mpwqe.wqe))
+ mlx5e_xdp_mpwqe_complete(sq);
+ return mlx5e_xmit_xdp_frame(sq, xdptxd, 0);
+ }
+ if (!xdptxd->len) {
+ skb_frag_t *frag = &xdptxdf->sinfo->frags[0];
+
+ tmp.data = skb_frag_address(frag);
+ tmp.len = skb_frag_size(frag);
+ tmp.dma_addr = xdptxdf->dma_arr ? xdptxdf->dma_arr[0] :
+ page_pool_get_dma_addr(skb_frag_page(frag)) +
+ skb_frag_off(frag);
+ p = &tmp;
+ }
}
- if (unlikely(xdptxd->len > sq->hw_mtu)) {
+ if (unlikely(p->len > sq->hw_mtu)) {
stats->err++;
return false;
}
@@ -418,7 +451,7 @@ mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptx
mlx5e_xdp_mpwqe_session_start(sq);
}
- mlx5e_xdp_mpwqe_add_dseg(sq, xdptxd, stats);
+ mlx5e_xdp_mpwqe_add_dseg(sq, p, stats);
if (unlikely(mlx5e_xdp_mpwqe_is_full(session, sq->max_sq_mpw_wqebbs)))
mlx5e_xdp_mpwqe_complete(sq);
@@ -446,8 +479,10 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)
INDIRECT_CALLABLE_SCOPE bool
mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd,
- struct skb_shared_info *sinfo, int check_result)
+ int check_result)
{
+ struct mlx5e_xmit_data_frags *xdptxdf =
+ container_of(xdptxd, struct mlx5e_xmit_data_frags, xd);
struct mlx5_wq_cyc *wq = &sq->wq;
struct mlx5_wqe_ctrl_seg *cseg;
struct mlx5_wqe_data_seg *dseg;
@@ -459,26 +494,34 @@ mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd,
u16 ds_cnt, inline_hdr_sz;
u8 num_wqebbs = 1;
int num_frags = 0;
+ bool inline_ok;
+ bool linear;
u16 pi;
struct mlx5e_xdpsq_stats *stats = sq->stats;
- if (unlikely(dma_len < MLX5E_XDP_MIN_INLINE || sq->hw_mtu < dma_len)) {
+ inline_ok = sq->min_inline_mode == MLX5_INLINE_MODE_NONE ||
+ dma_len >= MLX5E_XDP_MIN_INLINE;
+
+ if (unlikely(!inline_ok || sq->hw_mtu < dma_len)) {
stats->err++;
return false;
}
- ds_cnt = MLX5E_TX_WQE_EMPTY_DS_COUNT + 1;
+ inline_hdr_sz = 0;
if (sq->min_inline_mode != MLX5_INLINE_MODE_NONE)
- ds_cnt++;
+ inline_hdr_sz = MLX5E_XDP_MIN_INLINE;
+
+ linear = !!(dma_len - inline_hdr_sz);
+ ds_cnt = MLX5E_TX_WQE_EMPTY_DS_COUNT + linear + !!inline_hdr_sz;
/* check_result must be 0 if sinfo is passed. */
if (!check_result) {
int stop_room = 1;
- if (unlikely(sinfo)) {
- ds_cnt += sinfo->nr_frags;
- num_frags = sinfo->nr_frags;
+ if (xdptxd->has_frags) {
+ ds_cnt += xdptxdf->sinfo->nr_frags;
+ num_frags = xdptxdf->sinfo->nr_frags;
num_wqebbs = DIV_ROUND_UP(ds_cnt, MLX5_SEND_WQEBB_NUM_DS);
/* Assuming MLX5_CAP_GEN(mdev, max_wqe_sz_sq) is big
* enough to hold all fragments.
@@ -499,53 +542,53 @@ mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd,
eseg = &wqe->eth;
dseg = wqe->data;
- inline_hdr_sz = 0;
-
/* copy the inline part if required */
- if (sq->min_inline_mode != MLX5_INLINE_MODE_NONE) {
+ if (inline_hdr_sz) {
memcpy(eseg->inline_hdr.start, xdptxd->data, sizeof(eseg->inline_hdr.start));
memcpy(dseg, xdptxd->data + sizeof(eseg->inline_hdr.start),
- MLX5E_XDP_MIN_INLINE - sizeof(eseg->inline_hdr.start));
- dma_len -= MLX5E_XDP_MIN_INLINE;
- dma_addr += MLX5E_XDP_MIN_INLINE;
- inline_hdr_sz = MLX5E_XDP_MIN_INLINE;
+ inline_hdr_sz - sizeof(eseg->inline_hdr.start));
+ dma_len -= inline_hdr_sz;
+ dma_addr += inline_hdr_sz;
dseg++;
}
/* write the dma part */
- dseg->addr = cpu_to_be64(dma_addr);
- dseg->byte_count = cpu_to_be32(dma_len);
+ if (linear) {
+ dseg->addr = cpu_to_be64(dma_addr);
+ dseg->byte_count = cpu_to_be32(dma_len);
+ dseg->lkey = sq->mkey_be;
+ dseg++;
+ }
cseg->opmod_idx_opcode = cpu_to_be32((sq->pc << 8) | MLX5_OPCODE_SEND);
- if (unlikely(test_bit(MLX5E_SQ_STATE_XDP_MULTIBUF, &sq->state))) {
- u8 num_pkts = 1 + num_frags;
+ if (test_bit(MLX5E_SQ_STATE_XDP_MULTIBUF, &sq->state)) {
int i;
memset(&cseg->trailer, 0, sizeof(cseg->trailer));
memset(eseg, 0, sizeof(*eseg) - sizeof(eseg->trailer));
eseg->inline_hdr.sz = cpu_to_be16(inline_hdr_sz);
- dseg->lkey = sq->mkey_be;
for (i = 0; i < num_frags; i++) {
- skb_frag_t *frag = &sinfo->frags[i];
+ skb_frag_t *frag = &xdptxdf->sinfo->frags[i];
dma_addr_t addr;
- addr = page_pool_get_dma_addr(skb_frag_page(frag)) +
+ addr = xdptxdf->dma_arr ? xdptxdf->dma_arr[i] :
+ page_pool_get_dma_addr(skb_frag_page(frag)) +
skb_frag_off(frag);
- dseg++;
dseg->addr = cpu_to_be64(addr);
dseg->byte_count = cpu_to_be32(skb_frag_size(frag));
dseg->lkey = sq->mkey_be;
+ dseg++;
}
cseg->qpn_ds = cpu_to_be32((sq->sqn << 8) | ds_cnt);
sq->db.wqe_info[pi] = (struct mlx5e_xdp_wqe_info) {
.num_wqebbs = num_wqebbs,
- .num_pkts = num_pkts,
+ .num_pkts = 1,
};
sq->pc += num_wqebbs;
@@ -570,20 +613,61 @@ static void mlx5e_free_xdpsq_desc(struct mlx5e_xdpsq *sq,
u16 i;
for (i = 0; i < wi->num_pkts; i++) {
- struct mlx5e_xdp_info xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ union mlx5e_xdp_info xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
switch (xdpi.mode) {
- case MLX5E_XDP_XMIT_MODE_FRAME:
+ case MLX5E_XDP_XMIT_MODE_FRAME: {
/* XDP_TX from the XSK RQ and XDP_REDIRECT */
- dma_unmap_single(sq->pdev, xdpi.frame.dma_addr,
- xdpi.frame.xdpf->len, DMA_TO_DEVICE);
- xdp_return_frame_bulk(xdpi.frame.xdpf, bq);
+ struct xdp_frame *xdpf;
+ dma_addr_t dma_addr;
+
+ xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ xdpf = xdpi.frame.xdpf;
+ xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ dma_addr = xdpi.frame.dma_addr;
+
+ dma_unmap_single(sq->pdev, dma_addr,
+ xdpf->len, DMA_TO_DEVICE);
+ if (xdp_frame_has_frags(xdpf)) {
+ struct skb_shared_info *sinfo;
+ int j;
+
+ sinfo = xdp_get_shared_info_from_frame(xdpf);
+ for (j = 0; j < sinfo->nr_frags; j++) {
+ skb_frag_t *frag = &sinfo->frags[j];
+
+ xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ dma_addr = xdpi.frame.dma_addr;
+
+ dma_unmap_single(sq->pdev, dma_addr,
+ skb_frag_size(frag), DMA_TO_DEVICE);
+ }
+ }
+ xdp_return_frame_bulk(xdpf, bq);
break;
- case MLX5E_XDP_XMIT_MODE_PAGE:
+ }
+ case MLX5E_XDP_XMIT_MODE_PAGE: {
/* XDP_TX from the regular RQ */
- page_pool_put_defragged_page(xdpi.page.rq->page_pool,
- xdpi.page.page, -1, true);
+ u8 num, n = 0;
+
+ xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ num = xdpi.page.num;
+
+ do {
+ struct page *page;
+
+ xdpi = mlx5e_xdpi_fifo_pop(xdpi_fifo);
+ page = xdpi.page.page;
+
+ /* No need to check ((page->pp_magic & ~0x3UL) == PP_SIGNATURE)
+ * as we know this is a page_pool page.
+ */
+ page_pool_put_defragged_page(page->pp,
+ page, -1, true);
+ } while (++n < num);
+
break;
+ }
case MLX5E_XDP_XMIT_MODE_XSK:
/* AF_XDP send */
(*xsk_frames)++;
@@ -717,34 +801,79 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,
sq = &priv->channels.c[sq_num]->xdpsq;
for (i = 0; i < n; i++) {
+ struct mlx5e_xmit_data_frags xdptxdf = {};
struct xdp_frame *xdpf = frames[i];
- struct mlx5e_xmit_data xdptxd;
- struct mlx5e_xdp_info xdpi;
+ dma_addr_t dma_arr[MAX_SKB_FRAGS];
+ struct mlx5e_xmit_data *xdptxd;
bool ret;
- xdptxd.data = xdpf->data;
- xdptxd.len = xdpf->len;
- xdptxd.dma_addr = dma_map_single(sq->pdev, xdptxd.data,
- xdptxd.len, DMA_TO_DEVICE);
+ xdptxd = &xdptxdf.xd;
+ xdptxd->data = xdpf->data;
+ xdptxd->len = xdpf->len;
+ xdptxd->has_frags = xdp_frame_has_frags(xdpf);
+ xdptxd->dma_addr = dma_map_single(sq->pdev, xdptxd->data,
+ xdptxd->len, DMA_TO_DEVICE);
- if (unlikely(dma_mapping_error(sq->pdev, xdptxd.dma_addr)))
+ if (unlikely(dma_mapping_error(sq->pdev, xdptxd->dma_addr)))
break;
- xdpi.mode = MLX5E_XDP_XMIT_MODE_FRAME;
- xdpi.frame.xdpf = xdpf;
- xdpi.frame.dma_addr = xdptxd.dma_addr;
+ if (xdptxd->has_frags) {
+ int j;
+
+ xdptxdf.sinfo = xdp_get_shared_info_from_frame(xdpf);
+ xdptxdf.dma_arr = dma_arr;
+ for (j = 0; j < xdptxdf.sinfo->nr_frags; j++) {
+ skb_frag_t *frag = &xdptxdf.sinfo->frags[j];
+
+ dma_arr[j] = dma_map_single(sq->pdev, skb_frag_address(frag),
+ skb_frag_size(frag), DMA_TO_DEVICE);
+
+ if (!dma_mapping_error(sq->pdev, dma_arr[j]))
+ continue;
+ /* mapping error */
+ while (--j >= 0)
+ dma_unmap_single(sq->pdev, dma_arr[j],
+ skb_frag_size(&xdptxdf.sinfo->frags[j]),
+ DMA_TO_DEVICE);
+ goto out;
+ }
+ }
ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
- mlx5e_xmit_xdp_frame, sq, &xdptxd, NULL, 0);
+ mlx5e_xmit_xdp_frame, sq, xdptxd, 0);
if (unlikely(!ret)) {
- dma_unmap_single(sq->pdev, xdptxd.dma_addr,
- xdptxd.len, DMA_TO_DEVICE);
+ int j;
+
+ dma_unmap_single(sq->pdev, xdptxd->dma_addr,
+ xdptxd->len, DMA_TO_DEVICE);
+ if (!xdptxd->has_frags)
+ break;
+ for (j = 0; j < xdptxdf.sinfo->nr_frags; j++)
+ dma_unmap_single(sq->pdev, dma_arr[j],
+ skb_frag_size(&xdptxdf.sinfo->frags[j]),
+ DMA_TO_DEVICE);
break;
}
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, &xdpi);
+
+ /* xmit_mode == MLX5E_XDP_XMIT_MODE_FRAME */
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .mode = MLX5E_XDP_XMIT_MODE_FRAME });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .frame.xdpf = xdpf });
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info) { .frame.dma_addr = xdptxd->dma_addr });
+ if (xdptxd->has_frags) {
+ int j;
+
+ for (j = 0; j < xdptxdf.sinfo->nr_frags; j++)
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo,
+ (union mlx5e_xdp_info)
+ { .frame.dma_addr = dma_arr[j] });
+ }
nxmit++;
}
+out:
if (flags & XDP_XMIT_FLUSH) {
if (sq->mpwqe.wqe)
mlx5e_xdp_mpwqe_complete(sq);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
index 10bcfa6..9e8e618 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
@@ -50,6 +50,53 @@ struct mlx5e_xdp_buff {
struct mlx5e_rq *rq;
};
+/* XDP packets can be transmitted in different ways. On completion, we need to
+ * distinguish between them to clean up things in a proper way.
+ */
+enum mlx5e_xdp_xmit_mode {
+ /* An xdp_frame was transmitted due to either XDP_REDIRECT from another
+ * device or XDP_TX from an XSK RQ. The frame has to be unmapped and
+ * returned.
+ */
+ MLX5E_XDP_XMIT_MODE_FRAME,
+
+ /* The xdp_frame was created in place as a result of XDP_TX from a
+ * regular RQ. No DMA remapping happened, and the page belongs to us.
+ */
+ MLX5E_XDP_XMIT_MODE_PAGE,
+
+ /* No xdp_frame was created at all, the transmit happened from a UMEM
+ * page. The UMEM Completion Ring producer pointer has to be increased.
+ */
+ MLX5E_XDP_XMIT_MODE_XSK,
+};
+
+/* xmit_mode entry is pushed to the fifo per packet, followed by multiple
+ * entries, as follows:
+ *
+ * MLX5E_XDP_XMIT_MODE_FRAME:
+ * xdpf, dma_addr_1, dma_addr_2, ... , dma_addr_num.
+ * 'num' is derived from xdpf.
+ *
+ * MLX5E_XDP_XMIT_MODE_PAGE:
+ * num, page_1, page_2, ... , page_num.
+ *
+ * MLX5E_XDP_XMIT_MODE_XSK:
+ * none.
+ */
+union mlx5e_xdp_info {
+ enum mlx5e_xdp_xmit_mode mode;
+ union {
+ struct xdp_frame *xdpf;
+ dma_addr_t dma_addr;
+ } frame;
+ union {
+ struct mlx5e_rq *rq;
+ u8 num;
+ struct page *page;
+ } page;
+};
+
struct mlx5e_xsk_param;
int mlx5e_xdp_max_mtu(struct mlx5e_params *params, struct mlx5e_xsk_param *xsk);
bool mlx5e_xdp_handle(struct mlx5e_rq *rq,
@@ -66,11 +113,9 @@ extern const struct xdp_metadata_ops mlx5e_xdp_metadata_ops;
INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq,
struct mlx5e_xmit_data *xdptxd,
- struct skb_shared_info *sinfo,
int check_result));
INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq,
struct mlx5e_xmit_data *xdptxd,
- struct skb_shared_info *sinfo,
int check_result));
INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq));
INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq));
@@ -179,14 +224,14 @@ mlx5e_xdp_mpwqe_add_dseg(struct mlx5e_xdpsq *sq,
static inline void
mlx5e_xdpi_fifo_push(struct mlx5e_xdp_info_fifo *fifo,
- struct mlx5e_xdp_info *xi)
+ union mlx5e_xdp_info xi)
{
u32 i = (*fifo->pc)++ & fifo->mask;
- fifo->xi[i] = *xi;
+ fifo->xi[i] = xi;
}
-static inline struct mlx5e_xdp_info
+static inline union mlx5e_xdp_info
mlx5e_xdpi_fifo_pop(struct mlx5e_xdp_info_fifo *fifo)
{
return fifo->xi[(*fifo->cc)++ & fifo->mask];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
index 367a950..597f319 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
@@ -44,7 +44,7 @@ int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags)
* same.
*/
static void mlx5e_xsk_tx_post_err(struct mlx5e_xdpsq *sq,
- struct mlx5e_xdp_info *xdpi)
+ union mlx5e_xdp_info *xdpi)
{
u16 pi = mlx5_wq_cyc_ctr2ix(&sq->wq, sq->pc);
struct mlx5e_xdp_wqe_info *wi = &sq->db.wqe_info[pi];
@@ -54,15 +54,14 @@ static void mlx5e_xsk_tx_post_err(struct mlx5e_xdpsq *sq,
wi->num_pkts = 1;
nopwqe = mlx5e_post_nop(&sq->wq, sq->sqn, &sq->pc);
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, xdpi);
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, *xdpi);
sq->doorbell_cseg = &nopwqe->ctrl;
}
bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
{
struct xsk_buff_pool *pool = sq->xsk_pool;
- struct mlx5e_xmit_data xdptxd;
- struct mlx5e_xdp_info xdpi;
+ union mlx5e_xdp_info xdpi;
bool work_done = true;
bool flush = false;
@@ -73,6 +72,7 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
mlx5e_xmit_xdp_frame_check_mpwqe,
mlx5e_xmit_xdp_frame_check,
sq);
+ struct mlx5e_xmit_data xdptxd = {};
struct xdp_desc desc;
bool ret;
@@ -97,7 +97,7 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
xsk_buff_raw_dma_sync_for_device(pool, xdptxd.dma_addr, xdptxd.len);
ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
- mlx5e_xmit_xdp_frame, sq, &xdptxd, NULL,
+ mlx5e_xmit_xdp_frame, sq, &xdptxd,
check_result);
if (unlikely(!ret)) {
if (sq->mpwqe.wqe)
@@ -105,7 +105,7 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
mlx5e_xsk_tx_post_err(sq, &xdpi);
} else {
- mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, &xdpi);
+ mlx5e_xdpi_fifo_push(&sq->db.xdpi_fifo, xdpi);
}
flush = true;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index def01bf..55b3854 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -35,12 +35,14 @@
#include <crypto/aead.h>
#include <linux/inetdevice.h>
#include <linux/netdevice.h>
+#include <net/netevent.h>
#include "en.h"
#include "ipsec.h"
#include "ipsec_rxtx.h"
#define MLX5_IPSEC_RESCHED msecs_to_jiffies(1000)
+#define MLX5E_IPSEC_TUNNEL_SA XA_MARK_1
static struct mlx5e_ipsec_sa_entry *to_ipsec_sa_entry(struct xfrm_state *x)
{
@@ -242,6 +244,54 @@ static void mlx5e_ipsec_init_limits(struct mlx5e_ipsec_sa_entry *sa_entry,
attrs->lft.numb_rounds_soft = (u64)n;
}
+static void mlx5e_ipsec_init_macs(struct mlx5e_ipsec_sa_entry *sa_entry,
+ struct mlx5_accel_esp_xfrm_attrs *attrs)
+{
+ struct mlx5_core_dev *mdev = mlx5e_ipsec_sa2dev(sa_entry);
+ struct xfrm_state *x = sa_entry->x;
+ struct net_device *netdev;
+ struct neighbour *n;
+ u8 addr[ETH_ALEN];
+ const void *pkey;
+ u8 *dst, *src;
+
+ if (attrs->mode != XFRM_MODE_TUNNEL ||
+ attrs->type != XFRM_DEV_OFFLOAD_PACKET)
+ return;
+
+ netdev = x->xso.real_dev;
+
+ mlx5_query_mac_address(mdev, addr);
+ switch (attrs->dir) {
+ case XFRM_DEV_OFFLOAD_IN:
+ src = attrs->dmac;
+ dst = attrs->smac;
+ pkey = &attrs->saddr.a4;
+ break;
+ case XFRM_DEV_OFFLOAD_OUT:
+ src = attrs->smac;
+ dst = attrs->dmac;
+ pkey = &attrs->daddr.a4;
+ break;
+ default:
+ return;
+ }
+
+ ether_addr_copy(src, addr);
+ n = neigh_lookup(&arp_tbl, pkey, netdev);
+ if (!n) {
+ n = neigh_create(&arp_tbl, pkey, netdev);
+ if (IS_ERR(n))
+ return;
+ neigh_event_send(n, NULL);
+ attrs->drop = true;
+ } else {
+ neigh_ha_snapshot(addr, n, netdev);
+ ether_addr_copy(dst, addr);
+ }
+ neigh_release(n);
+}
+
void mlx5e_ipsec_build_accel_xfrm_attrs(struct mlx5e_ipsec_sa_entry *sa_entry,
struct mlx5_accel_esp_xfrm_attrs *attrs)
{
@@ -297,8 +347,10 @@ void mlx5e_ipsec_build_accel_xfrm_attrs(struct mlx5e_ipsec_sa_entry *sa_entry,
attrs->upspec.sport = ntohs(x->sel.sport);
attrs->upspec.sport_mask = ntohs(x->sel.sport_mask);
attrs->upspec.proto = x->sel.proto;
+ attrs->mode = x->props.mode;
mlx5e_ipsec_init_limits(sa_entry, attrs);
+ mlx5e_ipsec_init_macs(sa_entry, attrs);
}
static int mlx5e_xfrm_validate_state(struct mlx5_core_dev *mdev,
@@ -367,6 +419,11 @@ static int mlx5e_xfrm_validate_state(struct mlx5_core_dev *mdev,
return -EINVAL;
}
+ if (x->props.mode != XFRM_MODE_TRANSPORT && x->props.mode != XFRM_MODE_TUNNEL) {
+ NL_SET_ERR_MSG_MOD(extack, "Only transport and tunnel xfrm states may be offloaded");
+ return -EINVAL;
+ }
+
switch (x->xso.type) {
case XFRM_DEV_OFFLOAD_CRYPTO:
if (!(mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_CRYPTO)) {
@@ -374,11 +431,6 @@ static int mlx5e_xfrm_validate_state(struct mlx5_core_dev *mdev,
return -EINVAL;
}
- if (x->props.mode != XFRM_MODE_TRANSPORT &&
- x->props.mode != XFRM_MODE_TUNNEL) {
- NL_SET_ERR_MSG_MOD(extack, "Only transport and tunnel xfrm states may be offloaded");
- return -EINVAL;
- }
break;
case XFRM_DEV_OFFLOAD_PACKET:
if (!(mlx5_ipsec_device_caps(mdev) &
@@ -387,8 +439,9 @@ static int mlx5e_xfrm_validate_state(struct mlx5_core_dev *mdev,
return -EINVAL;
}
- if (x->props.mode != XFRM_MODE_TRANSPORT) {
- NL_SET_ERR_MSG_MOD(extack, "Only transport xfrm states may be offloaded in packet mode");
+ if (x->props.mode == XFRM_MODE_TUNNEL &&
+ !(mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_TUNNEL)) {
+ NL_SET_ERR_MSG_MOD(extack, "Packet offload is not supported for tunnel mode");
return -EINVAL;
}
@@ -458,34 +511,81 @@ static void mlx5e_ipsec_set_esn_ops(struct mlx5e_ipsec_sa_entry *sa_entry)
sa_entry->set_iv_op = mlx5e_ipsec_set_iv;
}
+static void mlx5e_ipsec_handle_netdev_event(struct work_struct *_work)
+{
+ struct mlx5e_ipsec_work *work =
+ container_of(_work, struct mlx5e_ipsec_work, work);
+ struct mlx5e_ipsec_sa_entry *sa_entry = work->sa_entry;
+ struct mlx5e_ipsec_netevent_data *data = work->data;
+ struct mlx5_accel_esp_xfrm_attrs *attrs;
+
+ attrs = &sa_entry->attrs;
+
+ switch (attrs->dir) {
+ case XFRM_DEV_OFFLOAD_IN:
+ ether_addr_copy(attrs->smac, data->addr);
+ break;
+ case XFRM_DEV_OFFLOAD_OUT:
+ ether_addr_copy(attrs->dmac, data->addr);
+ break;
+ default:
+ WARN_ON_ONCE(true);
+ }
+ attrs->drop = false;
+ mlx5e_accel_ipsec_fs_modify(sa_entry);
+}
+
static int mlx5_ipsec_create_work(struct mlx5e_ipsec_sa_entry *sa_entry)
{
struct xfrm_state *x = sa_entry->x;
struct mlx5e_ipsec_work *work;
+ void *data = NULL;
switch (x->xso.type) {
case XFRM_DEV_OFFLOAD_CRYPTO:
if (!(x->props.flags & XFRM_STATE_ESN))
return 0;
break;
+ case XFRM_DEV_OFFLOAD_PACKET:
+ if (x->props.mode != XFRM_MODE_TUNNEL)
+ return 0;
+ break;
default:
- return 0;
+ break;
}
work = kzalloc(sizeof(*work), GFP_KERNEL);
if (!work)
return -ENOMEM;
- work->data = kzalloc(sizeof(*sa_entry), GFP_KERNEL);
- if (!work->data) {
- kfree(work);
- return -ENOMEM;
+ switch (x->xso.type) {
+ case XFRM_DEV_OFFLOAD_CRYPTO:
+ data = kzalloc(sizeof(*sa_entry), GFP_KERNEL);
+ if (!data)
+ goto free_work;
+
+ INIT_WORK(&work->work, mlx5e_ipsec_modify_state);
+ break;
+ case XFRM_DEV_OFFLOAD_PACKET:
+ data = kzalloc(sizeof(struct mlx5e_ipsec_netevent_data),
+ GFP_KERNEL);
+ if (!data)
+ goto free_work;
+
+ INIT_WORK(&work->work, mlx5e_ipsec_handle_netdev_event);
+ break;
+ default:
+ break;
}
- INIT_WORK(&work->work, mlx5e_ipsec_modify_state);
+ work->data = data;
work->sa_entry = sa_entry;
sa_entry->work = work;
return 0;
+
+free_work:
+ kfree(work);
+ return -ENOMEM;
}
static int mlx5e_ipsec_create_dwork(struct mlx5e_ipsec_sa_entry *sa_entry)
@@ -566,6 +666,14 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x,
if (err)
goto err_hw_ctx;
+ if (x->props.mode == XFRM_MODE_TUNNEL &&
+ x->xso.type == XFRM_DEV_OFFLOAD_PACKET &&
+ !mlx5e_ipsec_fs_tunnel_enabled(sa_entry)) {
+ NL_SET_ERR_MSG_MOD(extack, "Packet offload tunnel mode is disabled due to encap settings");
+ err = -EINVAL;
+ goto err_add_rule;
+ }
+
/* We use *_bh() variant because xfrm_timer_handler(), which runs
* in softirq context, can reach our state delete logic and we need
* xa_erase_bh() there.
@@ -580,6 +688,12 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x,
if (sa_entry->dwork)
queue_delayed_work(ipsec->wq, &sa_entry->dwork->dwork,
MLX5_IPSEC_RESCHED);
+
+ if (x->xso.type == XFRM_DEV_OFFLOAD_PACKET &&
+ x->props.mode == XFRM_MODE_TUNNEL)
+ xa_set_mark(&ipsec->sadb, sa_entry->ipsec_obj_id,
+ MLX5E_IPSEC_TUNNEL_SA);
+
out:
x->xso.offload_handle = (unsigned long)sa_entry;
return 0;
@@ -591,17 +705,19 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x,
release_dwork:
kfree(sa_entry->dwork);
release_work:
- kfree(sa_entry->work->data);
+ if (sa_entry->work)
+ kfree(sa_entry->work->data);
kfree(sa_entry->work);
err_xfrm:
kfree(sa_entry);
- NL_SET_ERR_MSG_MOD(extack, "Device failed to offload this policy");
+ NL_SET_ERR_MSG_WEAK_MOD(extack, "Device failed to offload this state");
return err;
}
static void mlx5e_xfrm_del_state(struct xfrm_state *x)
{
struct mlx5e_ipsec_sa_entry *sa_entry = to_ipsec_sa_entry(x);
+ struct mlx5_accel_esp_xfrm_attrs *attrs = &sa_entry->attrs;
struct mlx5e_ipsec *ipsec = sa_entry->ipsec;
struct mlx5e_ipsec_sa_entry *old;
@@ -610,6 +726,12 @@ static void mlx5e_xfrm_del_state(struct xfrm_state *x)
old = xa_erase_bh(&ipsec->sadb, sa_entry->ipsec_obj_id);
WARN_ON(old != sa_entry);
+
+ if (attrs->mode == XFRM_MODE_TUNNEL &&
+ attrs->type == XFRM_DEV_OFFLOAD_PACKET)
+ /* Make sure that no ARP requests are running in parallel */
+ flush_workqueue(ipsec->wq);
+
}
static void mlx5e_xfrm_free_state(struct xfrm_state *x)
@@ -628,12 +750,53 @@ static void mlx5e_xfrm_free_state(struct xfrm_state *x)
mlx5e_accel_ipsec_fs_del_rule(sa_entry);
mlx5_ipsec_free_sa_ctx(sa_entry);
kfree(sa_entry->dwork);
- kfree(sa_entry->work->data);
+ if (sa_entry->work)
+ kfree(sa_entry->work->data);
kfree(sa_entry->work);
sa_entry_free:
kfree(sa_entry);
}
+static int mlx5e_ipsec_netevent_event(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct mlx5_accel_esp_xfrm_attrs *attrs;
+ struct mlx5e_ipsec_netevent_data *data;
+ struct mlx5e_ipsec_sa_entry *sa_entry;
+ struct mlx5e_ipsec *ipsec;
+ struct neighbour *n = ptr;
+ struct net_device *netdev;
+ struct xfrm_state *x;
+ unsigned long idx;
+
+ if (event != NETEVENT_NEIGH_UPDATE || !(n->nud_state & NUD_VALID))
+ return NOTIFY_DONE;
+
+ ipsec = container_of(nb, struct mlx5e_ipsec, netevent_nb);
+ xa_for_each_marked(&ipsec->sadb, idx, sa_entry, MLX5E_IPSEC_TUNNEL_SA) {
+ attrs = &sa_entry->attrs;
+
+ if (attrs->family == AF_INET) {
+ if (!neigh_key_eq32(n, &attrs->saddr.a4) &&
+ !neigh_key_eq32(n, &attrs->daddr.a4))
+ continue;
+ } else {
+ if (!neigh_key_eq128(n, &attrs->saddr.a4) &&
+ !neigh_key_eq128(n, &attrs->daddr.a4))
+ continue;
+ }
+
+ x = sa_entry->x;
+ netdev = x->xso.real_dev;
+ data = sa_entry->work->data;
+
+ neigh_ha_snapshot(data->addr, n, netdev);
+ queue_work(ipsec->wq, &sa_entry->work->work);
+ }
+
+ return NOTIFY_DONE;
+}
+
void mlx5e_ipsec_init(struct mlx5e_priv *priv)
{
struct mlx5e_ipsec *ipsec;
@@ -662,6 +825,13 @@ void mlx5e_ipsec_init(struct mlx5e_priv *priv)
goto err_aso;
}
+ if (mlx5_ipsec_device_caps(priv->mdev) & MLX5_IPSEC_CAP_TUNNEL) {
+ ipsec->netevent_nb.notifier_call = mlx5e_ipsec_netevent_event;
+ ret = register_netevent_notifier(&ipsec->netevent_nb);
+ if (ret)
+ goto clear_aso;
+ }
+
ret = mlx5e_accel_ipsec_fs_init(ipsec);
if (ret)
goto err_fs_init;
@@ -672,6 +842,9 @@ void mlx5e_ipsec_init(struct mlx5e_priv *priv)
return;
err_fs_init:
+ if (mlx5_ipsec_device_caps(priv->mdev) & MLX5_IPSEC_CAP_TUNNEL)
+ unregister_netevent_notifier(&ipsec->netevent_nb);
+clear_aso:
if (mlx5_ipsec_device_caps(priv->mdev) & MLX5_IPSEC_CAP_PACKET_OFFLOAD)
mlx5e_ipsec_aso_cleanup(ipsec);
err_aso:
@@ -690,6 +863,8 @@ void mlx5e_ipsec_cleanup(struct mlx5e_priv *priv)
return;
mlx5e_accel_ipsec_fs_cleanup(ipsec);
+ if (mlx5_ipsec_device_caps(priv->mdev) & MLX5_IPSEC_CAP_TUNNEL)
+ unregister_netevent_notifier(&ipsec->netevent_nb);
if (mlx5_ipsec_device_caps(priv->mdev) & MLX5_IPSEC_CAP_PACKET_OFFLOAD)
mlx5e_ipsec_aso_cleanup(ipsec);
destroy_workqueue(ipsec->wq);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
index 52890d7d..4e98871 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
@@ -77,7 +77,7 @@ struct mlx5_replay_esn {
struct mlx5_accel_esp_xfrm_attrs {
u32 spi;
- u32 flags;
+ u32 mode;
struct aes_gcm_keymat aes_gcm;
union {
@@ -99,6 +99,8 @@ struct mlx5_accel_esp_xfrm_attrs {
u32 authsize;
u32 reqid;
struct mlx5_ipsec_lft lft;
+ u8 smac[ETH_ALEN];
+ u8 dmac[ETH_ALEN];
};
enum mlx5_ipsec_cap {
@@ -107,6 +109,7 @@ enum mlx5_ipsec_cap {
MLX5_IPSEC_CAP_PACKET_OFFLOAD = 1 << 2,
MLX5_IPSEC_CAP_ROCE = 1 << 3,
MLX5_IPSEC_CAP_PRIO = 1 << 4,
+ MLX5_IPSEC_CAP_TUNNEL = 1 << 5,
};
struct mlx5e_priv;
@@ -141,6 +144,10 @@ struct mlx5e_ipsec_work {
void *data;
};
+struct mlx5e_ipsec_netevent_data {
+ u8 addr[ETH_ALEN];
+};
+
struct mlx5e_ipsec_dwork {
struct delayed_work dwork;
struct mlx5e_ipsec_sa_entry *sa_entry;
@@ -166,6 +173,7 @@ struct mlx5e_ipsec {
struct mlx5e_ipsec_tx *tx;
struct mlx5e_ipsec_aso *aso;
struct notifier_block nb;
+ struct notifier_block netevent_nb;
struct mlx5_ipsec_fs *roce;
};
@@ -243,6 +251,7 @@ void mlx5e_accel_ipsec_fs_del_rule(struct mlx5e_ipsec_sa_entry *sa_entry);
int mlx5e_accel_ipsec_fs_add_pol(struct mlx5e_ipsec_pol_entry *pol_entry);
void mlx5e_accel_ipsec_fs_del_pol(struct mlx5e_ipsec_pol_entry *pol_entry);
void mlx5e_accel_ipsec_fs_modify(struct mlx5e_ipsec_sa_entry *sa_entry);
+bool mlx5e_ipsec_fs_tunnel_enabled(struct mlx5e_ipsec_sa_entry *sa_entry);
int mlx5_ipsec_create_sa_ctx(struct mlx5e_ipsec_sa_entry *sa_entry);
void mlx5_ipsec_free_sa_ctx(struct mlx5e_ipsec_sa_entry *sa_entry);
@@ -278,7 +287,7 @@ static inline bool addr6_all_zero(__be32 *addr6)
{
static const __be32 zaddr6[4] = {};
- return !memcmp(addr6, zaddr6, sizeof(*zaddr6));
+ return !memcmp(addr6, zaddr6, sizeof(zaddr6));
}
#else
static inline void mlx5e_ipsec_init(struct mlx5e_priv *priv)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
index b47794d..dbe87bf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
@@ -4,12 +4,15 @@
#include <linux/netdevice.h>
#include "en.h"
#include "en/fs.h"
+#include "eswitch.h"
#include "ipsec.h"
#include "fs_core.h"
#include "lib/ipsec_fs_roce.h"
#include "lib/fs_chains.h"
#define NUM_IPSEC_FTE BIT(15)
+#define MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_SIZE 16
+#define IPSEC_TUNNEL_DEFAULT_TTL 0x40
struct mlx5e_ipsec_fc {
struct mlx5_fc *cnt;
@@ -36,6 +39,7 @@ struct mlx5e_ipsec_rx {
struct mlx5e_ipsec_rule status;
struct mlx5e_ipsec_fc *fc;
struct mlx5_fs_chains *chains;
+ u8 allow_tunnel_mode : 1;
};
struct mlx5e_ipsec_tx {
@@ -45,6 +49,7 @@ struct mlx5e_ipsec_tx {
struct mlx5_flow_namespace *ns;
struct mlx5e_ipsec_fc *fc;
struct mlx5_fs_chains *chains;
+ u8 allow_tunnel_mode : 1;
};
/* IPsec RX flow steering */
@@ -118,7 +123,7 @@ static void ipsec_chains_put_table(struct mlx5_fs_chains *chains, u32 prio)
static struct mlx5_flow_table *ipsec_ft_create(struct mlx5_flow_namespace *ns,
int level, int prio,
- int max_num_groups)
+ int max_num_groups, u32 flags)
{
struct mlx5_flow_table_attr ft_attr = {};
@@ -127,6 +132,7 @@ static struct mlx5_flow_table *ipsec_ft_create(struct mlx5_flow_namespace *ns,
ft_attr.max_fte = NUM_IPSEC_FTE;
ft_attr.level = level;
ft_attr.prio = prio;
+ ft_attr.flags = flags;
return mlx5_create_auto_grouped_flow_table(ns, &ft_attr);
}
@@ -251,7 +257,8 @@ static void rx_destroy(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
mlx5_del_flow_rules(rx->sa.rule);
mlx5_destroy_flow_group(rx->sa.group);
mlx5_destroy_flow_table(rx->ft.sa);
-
+ if (rx->allow_tunnel_mode)
+ mlx5_eswitch_unblock_encap(mdev);
mlx5_del_flow_rules(rx->status.rule);
mlx5_modify_header_dealloc(mdev, rx->status.modify_hdr);
mlx5_destroy_flow_table(rx->ft.status);
@@ -267,6 +274,7 @@ static int rx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
struct mlx5_flow_destination default_dest;
struct mlx5_flow_destination dest[2];
struct mlx5_flow_table *ft;
+ u32 flags = 0;
int err;
default_dest = mlx5_ttc_get_default_dest(ttc, family2tt(family));
@@ -277,7 +285,7 @@ static int rx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
return err;
ft = ipsec_ft_create(ns, MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL,
- MLX5E_NIC_PRIO, 1);
+ MLX5E_NIC_PRIO, 1, 0);
if (IS_ERR(ft)) {
err = PTR_ERR(ft);
goto err_fs_ft_status;
@@ -300,8 +308,12 @@ static int rx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
goto err_add;
/* Create FT */
- ft = ipsec_ft_create(ns, MLX5E_ACCEL_FS_ESP_FT_LEVEL, MLX5E_NIC_PRIO,
- 2);
+ if (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_TUNNEL)
+ rx->allow_tunnel_mode = mlx5_eswitch_block_encap(mdev);
+ if (rx->allow_tunnel_mode)
+ flags = MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
+ ft = ipsec_ft_create(ns, MLX5E_ACCEL_FS_ESP_FT_LEVEL, MLX5E_NIC_PRIO, 2,
+ flags);
if (IS_ERR(ft)) {
err = PTR_ERR(ft);
goto err_fs_ft;
@@ -327,7 +339,7 @@ static int rx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
}
ft = ipsec_ft_create(ns, MLX5E_ACCEL_FS_POL_FT_LEVEL, MLX5E_NIC_PRIO,
- 2);
+ 2, 0);
if (IS_ERR(ft)) {
err = PTR_ERR(ft);
goto err_pol_ft;
@@ -356,6 +368,8 @@ static int rx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
err_fs:
mlx5_destroy_flow_table(rx->ft.sa);
err_fs_ft:
+ if (rx->allow_tunnel_mode)
+ mlx5_eswitch_unblock_encap(mdev);
mlx5_del_flow_rules(rx->status.rule);
mlx5_modify_header_dealloc(mdev, rx->status.modify_hdr);
err_add:
@@ -490,7 +504,8 @@ static int ipsec_counter_rule_tx(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_
}
/* IPsec TX flow steering */
-static void tx_destroy(struct mlx5e_ipsec_tx *tx, struct mlx5_ipsec_fs *roce)
+static void tx_destroy(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_tx *tx,
+ struct mlx5_ipsec_fs *roce)
{
mlx5_ipsec_fs_roce_tx_destroy(roce);
if (tx->chains) {
@@ -502,6 +517,8 @@ static void tx_destroy(struct mlx5e_ipsec_tx *tx, struct mlx5_ipsec_fs *roce)
}
mlx5_destroy_flow_table(tx->ft.sa);
+ if (tx->allow_tunnel_mode)
+ mlx5_eswitch_unblock_encap(mdev);
mlx5_del_flow_rules(tx->status.rule);
mlx5_destroy_flow_table(tx->ft.status);
}
@@ -511,9 +528,10 @@ static int tx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_tx *tx,
{
struct mlx5_flow_destination dest = {};
struct mlx5_flow_table *ft;
+ u32 flags = 0;
int err;
- ft = ipsec_ft_create(tx->ns, 2, 0, 1);
+ ft = ipsec_ft_create(tx->ns, 2, 0, 1, 0);
if (IS_ERR(ft))
return PTR_ERR(ft);
tx->ft.status = ft;
@@ -522,7 +540,11 @@ static int tx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_tx *tx,
if (err)
goto err_status_rule;
- ft = ipsec_ft_create(tx->ns, 1, 0, 4);
+ if (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_TUNNEL)
+ tx->allow_tunnel_mode = mlx5_eswitch_block_encap(mdev);
+ if (tx->allow_tunnel_mode)
+ flags = MLX5_FLOW_TABLE_TUNNEL_EN_REFORMAT;
+ ft = ipsec_ft_create(tx->ns, 1, 0, 4, flags);
if (IS_ERR(ft)) {
err = PTR_ERR(ft);
goto err_sa_ft;
@@ -541,7 +563,7 @@ static int tx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_tx *tx,
goto connect_roce;
}
- ft = ipsec_ft_create(tx->ns, 0, 0, 2);
+ ft = ipsec_ft_create(tx->ns, 0, 0, 2, 0);
if (IS_ERR(ft)) {
err = PTR_ERR(ft);
goto err_pol_ft;
@@ -572,6 +594,8 @@ static int tx_create(struct mlx5_core_dev *mdev, struct mlx5e_ipsec_tx *tx,
err_pol_ft:
mlx5_destroy_flow_table(tx->ft.sa);
err_sa_ft:
+ if (tx->allow_tunnel_mode)
+ mlx5_eswitch_unblock_encap(mdev);
mlx5_del_flow_rules(tx->status.rule);
err_status_rule:
mlx5_destroy_flow_table(tx->ft.status);
@@ -600,7 +624,7 @@ static void tx_put(struct mlx5e_ipsec *ipsec, struct mlx5e_ipsec_tx *tx)
if (--tx->ft.refcnt)
return;
- tx_destroy(tx, ipsec->roce);
+ tx_destroy(ipsec->mdev, tx, ipsec->roce);
}
static struct mlx5_flow_table *tx_ft_get_policy(struct mlx5_core_dev *mdev,
@@ -829,40 +853,181 @@ static int setup_modify_header(struct mlx5_core_dev *mdev, u32 val, u8 dir,
return 0;
}
+static int
+setup_pkt_tunnel_reformat(struct mlx5_core_dev *mdev,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ struct mlx5_pkt_reformat_params *reformat_params)
+{
+ struct ip_esp_hdr *esp_hdr;
+ struct ipv6hdr *ipv6hdr;
+ struct ethhdr *eth_hdr;
+ struct iphdr *iphdr;
+ char *reformatbf;
+ size_t bfflen;
+ void *hdr;
+
+ bfflen = sizeof(*eth_hdr);
+
+ if (attrs->dir == XFRM_DEV_OFFLOAD_OUT) {
+ bfflen += sizeof(*esp_hdr) + 8;
+
+ switch (attrs->family) {
+ case AF_INET:
+ bfflen += sizeof(*iphdr);
+ break;
+ case AF_INET6:
+ bfflen += sizeof(*ipv6hdr);
+ break;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ reformatbf = kzalloc(bfflen, GFP_KERNEL);
+ if (!reformatbf)
+ return -ENOMEM;
+
+ eth_hdr = (struct ethhdr *)reformatbf;
+ switch (attrs->family) {
+ case AF_INET:
+ eth_hdr->h_proto = htons(ETH_P_IP);
+ break;
+ case AF_INET6:
+ eth_hdr->h_proto = htons(ETH_P_IPV6);
+ break;
+ default:
+ goto free_reformatbf;
+ }
+
+ ether_addr_copy(eth_hdr->h_dest, attrs->dmac);
+ ether_addr_copy(eth_hdr->h_source, attrs->smac);
+
+ switch (attrs->dir) {
+ case XFRM_DEV_OFFLOAD_IN:
+ reformat_params->type = MLX5_REFORMAT_TYPE_L3_ESP_TUNNEL_TO_L2;
+ break;
+ case XFRM_DEV_OFFLOAD_OUT:
+ reformat_params->type = MLX5_REFORMAT_TYPE_L2_TO_L3_ESP_TUNNEL;
+ reformat_params->param_0 = attrs->authsize;
+
+ hdr = reformatbf + sizeof(*eth_hdr);
+ switch (attrs->family) {
+ case AF_INET:
+ iphdr = (struct iphdr *)hdr;
+ memcpy(&iphdr->saddr, &attrs->saddr.a4, 4);
+ memcpy(&iphdr->daddr, &attrs->daddr.a4, 4);
+ iphdr->version = 4;
+ iphdr->ihl = 5;
+ iphdr->ttl = IPSEC_TUNNEL_DEFAULT_TTL;
+ iphdr->protocol = IPPROTO_ESP;
+ hdr += sizeof(*iphdr);
+ break;
+ case AF_INET6:
+ ipv6hdr = (struct ipv6hdr *)hdr;
+ memcpy(&ipv6hdr->saddr, &attrs->saddr.a6, 16);
+ memcpy(&ipv6hdr->daddr, &attrs->daddr.a6, 16);
+ ipv6hdr->nexthdr = IPPROTO_ESP;
+ ipv6hdr->version = 6;
+ ipv6hdr->hop_limit = IPSEC_TUNNEL_DEFAULT_TTL;
+ hdr += sizeof(*ipv6hdr);
+ break;
+ default:
+ goto free_reformatbf;
+ }
+
+ esp_hdr = (struct ip_esp_hdr *)hdr;
+ esp_hdr->spi = htonl(attrs->spi);
+ break;
+ default:
+ goto free_reformatbf;
+ }
+
+ reformat_params->size = bfflen;
+ reformat_params->data = reformatbf;
+ return 0;
+
+free_reformatbf:
+ kfree(reformatbf);
+ return -EINVAL;
+}
+
+static int
+setup_pkt_transport_reformat(struct mlx5_accel_esp_xfrm_attrs *attrs,
+ struct mlx5_pkt_reformat_params *reformat_params)
+{
+ u8 *reformatbf;
+ __be32 spi;
+
+ switch (attrs->dir) {
+ case XFRM_DEV_OFFLOAD_IN:
+ reformat_params->type = MLX5_REFORMAT_TYPE_DEL_ESP_TRANSPORT;
+ break;
+ case XFRM_DEV_OFFLOAD_OUT:
+ if (attrs->family == AF_INET)
+ reformat_params->type =
+ MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV4;
+ else
+ reformat_params->type =
+ MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV6;
+
+ reformatbf = kzalloc(MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_SIZE,
+ GFP_KERNEL);
+ if (!reformatbf)
+ return -ENOMEM;
+
+ /* convert to network format */
+ spi = htonl(attrs->spi);
+ memcpy(reformatbf, &spi, sizeof(spi));
+
+ reformat_params->param_0 = attrs->authsize;
+ reformat_params->size =
+ MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_SIZE;
+ reformat_params->data = reformatbf;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int setup_pkt_reformat(struct mlx5_core_dev *mdev,
struct mlx5_accel_esp_xfrm_attrs *attrs,
struct mlx5_flow_act *flow_act)
{
- enum mlx5_flow_namespace_type ns_type = MLX5_FLOW_NAMESPACE_EGRESS;
struct mlx5_pkt_reformat_params reformat_params = {};
struct mlx5_pkt_reformat *pkt_reformat;
- u8 reformatbf[16] = {};
- __be32 spi;
+ enum mlx5_flow_namespace_type ns_type;
+ int ret;
- if (attrs->dir == XFRM_DEV_OFFLOAD_IN) {
- reformat_params.type = MLX5_REFORMAT_TYPE_DEL_ESP_TRANSPORT;
+ switch (attrs->dir) {
+ case XFRM_DEV_OFFLOAD_IN:
ns_type = MLX5_FLOW_NAMESPACE_KERNEL;
- goto cmd;
+ break;
+ case XFRM_DEV_OFFLOAD_OUT:
+ ns_type = MLX5_FLOW_NAMESPACE_EGRESS;
+ break;
+ default:
+ return -EINVAL;
}
- if (attrs->family == AF_INET)
- reformat_params.type =
- MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV4;
- else
- reformat_params.type =
- MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV6;
+ switch (attrs->mode) {
+ case XFRM_MODE_TRANSPORT:
+ ret = setup_pkt_transport_reformat(attrs, &reformat_params);
+ break;
+ case XFRM_MODE_TUNNEL:
+ ret = setup_pkt_tunnel_reformat(mdev, attrs, &reformat_params);
+ break;
+ default:
+ ret = -EINVAL;
+ }
- /* convert to network format */
- spi = htonl(attrs->spi);
- memcpy(reformatbf, &spi, 4);
+ if (ret)
+ return ret;
- reformat_params.param_0 = attrs->authsize;
- reformat_params.size = sizeof(reformatbf);
- reformat_params.data = &reformatbf;
-
-cmd:
pkt_reformat =
mlx5_packet_reformat_alloc(mdev, &reformat_params, ns_type);
+ kfree(reformat_params.data);
if (IS_ERR(pkt_reformat))
return PTR_ERR(pkt_reformat);
@@ -1087,16 +1252,16 @@ static int tx_add_policy(struct mlx5e_ipsec_pol_entry *pol_entry)
setup_fte_no_frags(spec);
setup_fte_upper_proto_match(spec, &attrs->upspec);
- if (attrs->reqid) {
+ switch (attrs->action) {
+ case XFRM_POLICY_ALLOW:
+ flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
+ if (!attrs->reqid)
+ break;
+
err = setup_modify_header(mdev, attrs->reqid,
XFRM_DEV_OFFLOAD_OUT, &flow_act);
if (err)
goto err_mod_header;
- }
-
- switch (attrs->action) {
- case XFRM_POLICY_ALLOW:
- flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
break;
case XFRM_POLICY_BLOCK:
flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_DROP |
@@ -1108,7 +1273,7 @@ static int tx_add_policy(struct mlx5e_ipsec_pol_entry *pol_entry)
default:
WARN_ON(true);
err = -EINVAL;
- goto err_action;
+ goto err_mod_header;
}
flow_act.flags |= FLOW_ACT_NO_APPEND;
@@ -1128,7 +1293,7 @@ static int tx_add_policy(struct mlx5e_ipsec_pol_entry *pol_entry)
return 0;
err_action:
- if (attrs->reqid)
+ if (flow_act.modify_hdr)
mlx5_modify_header_dealloc(mdev, flow_act.modify_hdr);
err_mod_header:
kvfree(spec);
@@ -1453,3 +1618,15 @@ void mlx5e_accel_ipsec_fs_modify(struct mlx5e_ipsec_sa_entry *sa_entry)
mlx5e_accel_ipsec_fs_del_rule(sa_entry);
memcpy(sa_entry, &sa_entry_shadow, sizeof(*sa_entry));
}
+
+bool mlx5e_ipsec_fs_tunnel_enabled(struct mlx5e_ipsec_sa_entry *sa_entry)
+{
+ struct mlx5e_ipsec_rx *rx =
+ ipsec_rx(sa_entry->ipsec, sa_entry->attrs.family);
+ struct mlx5e_ipsec_tx *tx = sa_entry->ipsec->tx;
+
+ if (sa_entry->attrs.dir == XFRM_DEV_OFFLOAD_OUT)
+ return tx->allow_tunnel_mode;
+
+ return rx->allow_tunnel_mode;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
index 5fddb86..df90e19 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
@@ -48,6 +48,12 @@ u32 mlx5_ipsec_device_caps(struct mlx5_core_dev *mdev)
if (MLX5_CAP_FLOWTABLE_NIC_TX(mdev, ignore_flow_level) &&
MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ignore_flow_level))
caps |= MLX5_IPSEC_CAP_PRIO;
+
+ if (MLX5_CAP_FLOWTABLE_NIC_TX(mdev,
+ reformat_l2_to_l3_esp_tunnel) &&
+ MLX5_CAP_FLOWTABLE_NIC_RX(mdev,
+ reformat_l3_esp_tunnel_to_l2))
+ caps |= MLX5_IPSEC_CAP_TUNNEL;
}
if (mlx5_get_roce_state(mdev) &&
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
index 33b3620..f3428db 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
@@ -4,6 +4,7 @@
#include <linux/mlx5/device.h>
#include <linux/mlx5/mlx5_ifc.h>
#include <linux/xarray.h>
+#include <linux/if_vlan.h>
#include "en.h"
#include "lib/aso.h"
@@ -348,12 +349,21 @@ static void mlx5e_macsec_cleanup_sa(struct mlx5e_macsec *macsec,
sa->macsec_rule = NULL;
}
+static struct mlx5e_priv *macsec_netdev_priv(const struct net_device *dev)
+{
+#if IS_ENABLED(CONFIG_VLAN_8021Q)
+ if (is_vlan_dev(dev))
+ return netdev_priv(vlan_dev_priv(dev)->real_dev);
+#endif
+ return netdev_priv(dev);
+}
+
static int mlx5e_macsec_init_sa(struct macsec_context *ctx,
struct mlx5e_macsec_sa *sa,
bool encrypt,
bool is_tx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec *macsec = priv->macsec;
struct mlx5_macsec_rule_attrs rule_attrs;
struct mlx5_core_dev *mdev = priv->mdev;
@@ -427,7 +437,7 @@ static int macsec_rx_sa_active_update(struct macsec_context *ctx,
struct mlx5e_macsec_sa *rx_sa,
bool active)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec *macsec = priv->macsec;
int err = 0;
@@ -508,9 +518,9 @@ static void update_macsec_epn(struct mlx5e_macsec_sa *sa, const struct macsec_ke
static int mlx5e_macsec_add_txsa(struct macsec_context *ctx)
{
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc;
const struct macsec_tx_sa *ctx_tx_sa = ctx->sa.tx_sa;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
const struct macsec_secy *secy = ctx->secy;
struct mlx5e_macsec_device *macsec_device;
struct mlx5_core_dev *mdev = priv->mdev;
@@ -583,9 +593,9 @@ static int mlx5e_macsec_add_txsa(struct macsec_context *ctx)
static int mlx5e_macsec_upd_txsa(struct macsec_context *ctx)
{
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc;
const struct macsec_tx_sa *ctx_tx_sa = ctx->sa.tx_sa;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
u8 assoc_num = ctx->sa.assoc_num;
struct mlx5e_macsec_sa *tx_sa;
@@ -645,7 +655,7 @@ static int mlx5e_macsec_upd_txsa(struct macsec_context *ctx)
static int mlx5e_macsec_del_txsa(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
u8 assoc_num = ctx->sa.assoc_num;
struct mlx5e_macsec_sa *tx_sa;
@@ -696,7 +706,7 @@ static u32 mlx5e_macsec_get_sa_from_hashtable(struct rhashtable *sci_hash, sci_t
static int mlx5e_macsec_add_rxsc(struct macsec_context *ctx)
{
struct mlx5e_macsec_rx_sc_xarray_element *sc_xarray_element;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_rx_sc *ctx_rx_sc = ctx->rx_sc;
struct mlx5e_macsec_device *macsec_device;
struct mlx5e_macsec_rx_sc *rx_sc;
@@ -776,7 +786,7 @@ static int mlx5e_macsec_add_rxsc(struct macsec_context *ctx)
static int mlx5e_macsec_upd_rxsc(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_rx_sc *ctx_rx_sc = ctx->rx_sc;
struct mlx5e_macsec_device *macsec_device;
struct mlx5e_macsec_rx_sc *rx_sc;
@@ -854,7 +864,7 @@ static void macsec_del_rxsc_ctx(struct mlx5e_macsec *macsec, struct mlx5e_macsec
static int mlx5e_macsec_del_rxsc(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
struct mlx5e_macsec_rx_sc *rx_sc;
struct mlx5e_macsec *macsec;
@@ -890,8 +900,8 @@ static int mlx5e_macsec_del_rxsc(struct macsec_context *ctx)
static int mlx5e_macsec_add_rxsa(struct macsec_context *ctx)
{
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_rx_sa *ctx_rx_sa = ctx->sa.rx_sa;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
struct mlx5_core_dev *mdev = priv->mdev;
u8 assoc_num = ctx->sa.assoc_num;
@@ -976,8 +986,8 @@ static int mlx5e_macsec_add_rxsa(struct macsec_context *ctx)
static int mlx5e_macsec_upd_rxsa(struct macsec_context *ctx)
{
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_rx_sa *ctx_rx_sa = ctx->sa.rx_sa;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
u8 assoc_num = ctx->sa.assoc_num;
struct mlx5e_macsec_rx_sc *rx_sc;
@@ -1033,7 +1043,7 @@ static int mlx5e_macsec_upd_rxsa(struct macsec_context *ctx)
static int mlx5e_macsec_del_rxsa(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
sci_t sci = ctx->sa.rx_sa->sc->sci;
struct mlx5e_macsec_rx_sc *rx_sc;
@@ -1085,7 +1095,7 @@ static int mlx5e_macsec_del_rxsa(struct macsec_context *ctx)
static int mlx5e_macsec_add_secy(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct net_device *dev = ctx->secy->netdev;
const struct net_device *netdev = ctx->netdev;
struct mlx5e_macsec_device *macsec_device;
@@ -1137,7 +1147,7 @@ static int mlx5e_macsec_add_secy(struct macsec_context *ctx)
static int macsec_upd_secy_hw_address(struct macsec_context *ctx,
struct mlx5e_macsec_device *macsec_device)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct net_device *dev = ctx->secy->netdev;
struct mlx5e_macsec *macsec = priv->macsec;
struct mlx5e_macsec_rx_sc *rx_sc, *tmp;
@@ -1184,8 +1194,8 @@ static int macsec_upd_secy_hw_address(struct macsec_context *ctx,
*/
static int mlx5e_macsec_upd_secy(struct macsec_context *ctx)
{
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc;
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
const struct net_device *dev = ctx->secy->netdev;
struct mlx5e_macsec_device *macsec_device;
struct mlx5e_macsec_sa *tx_sa;
@@ -1240,7 +1250,7 @@ static int mlx5e_macsec_upd_secy(struct macsec_context *ctx)
static int mlx5e_macsec_del_secy(struct macsec_context *ctx)
{
- struct mlx5e_priv *priv = netdev_priv(ctx->netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(ctx->netdev);
struct mlx5e_macsec_device *macsec_device;
struct mlx5e_macsec_rx_sc *rx_sc, *tmp;
struct mlx5e_macsec_sa *tx_sa;
@@ -1741,7 +1751,7 @@ void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev,
{
struct mlx5e_macsec_rx_sc_xarray_element *sc_xarray_element;
u32 macsec_meta_data = be32_to_cpu(cqe->ft_metadata);
- struct mlx5e_priv *priv = netdev_priv(netdev);
+ struct mlx5e_priv *priv = macsec_netdev_priv(netdev);
struct mlx5e_macsec_rx_sc *rx_sc;
struct mlx5e_macsec *macsec;
u32 fs_id;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c
index 9173b67..7fc901a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec_fs.c
@@ -4,6 +4,7 @@
#include <net/macsec.h>
#include <linux/netdevice.h>
#include <linux/mlx5/qp.h>
+#include <linux/if_vlan.h>
#include "fs_core.h"
#include "en/fs.h"
#include "en_accel/macsec_fs.h"
@@ -508,6 +509,8 @@ static void macsec_fs_tx_del_rule(struct mlx5e_macsec_fs *macsec_fs,
macsec_fs_tx_ft_put(macsec_fs);
}
+#define MLX5_REFORMAT_PARAM_ADD_MACSEC_OFFSET_4_BYTES 1
+
static union mlx5e_macsec_rule *
macsec_fs_tx_add_rule(struct mlx5e_macsec_fs *macsec_fs,
const struct macsec_context *macsec_ctx,
@@ -553,6 +556,10 @@ macsec_fs_tx_add_rule(struct mlx5e_macsec_fs *macsec_fs,
reformat_params.type = MLX5_REFORMAT_TYPE_ADD_MACSEC;
reformat_params.size = reformat_size;
reformat_params.data = reformatbf;
+
+ if (is_vlan_dev(macsec_ctx->netdev))
+ reformat_params.param_0 = MLX5_REFORMAT_PARAM_ADD_MACSEC_OFFSET_4_BYTES;
+
flow_act.pkt_reformat = mlx5_packet_reformat_alloc(macsec_fs->mdev,
&reformat_params,
MLX5_FLOW_NAMESPACE_EGRESS_MACSEC);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index ec72743b..2fda738 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -803,6 +803,9 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params,
pool_size = rq->mpwqe.pages_per_wqe <<
mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk);
+ if (!mlx5e_rx_mpwqe_is_linear_skb(mdev, params, xsk) && params->xdp_prog)
+ pool_size *= 2; /* additional page per packet for the linear part */
+
rq->mpwqe.log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk);
rq->mpwqe.num_strides =
BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk));
@@ -1300,17 +1303,19 @@ static int mlx5e_alloc_xdpsq_fifo(struct mlx5e_xdpsq *sq, int numa)
{
struct mlx5e_xdp_info_fifo *xdpi_fifo = &sq->db.xdpi_fifo;
int wq_sz = mlx5_wq_cyc_get_size(&sq->wq);
- int dsegs_per_wq = wq_sz * MLX5_SEND_WQEBB_NUM_DS;
+ int entries = wq_sz * MLX5_SEND_WQEBB_NUM_DS * 2; /* upper bound for maximum num of
+ * entries of all xmit_modes.
+ */
size_t size;
- size = array_size(sizeof(*xdpi_fifo->xi), dsegs_per_wq);
+ size = array_size(sizeof(*xdpi_fifo->xi), entries);
xdpi_fifo->xi = kvzalloc_node(size, GFP_KERNEL, numa);
if (!xdpi_fifo->xi)
return -ENOMEM;
xdpi_fifo->pc = &sq->xdpi_fifo_pc;
xdpi_fifo->cc = &sq->xdpi_fifo_cc;
- xdpi_fifo->mask = dsegs_per_wq - 1;
+ xdpi_fifo->mask = entries - 1;
return 0;
}
@@ -1860,11 +1865,7 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
csp.min_inline_mode = sq->min_inline_mode;
set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
- /* Don't enable multi buffer on XDP_REDIRECT SQ, as it's not yet
- * supported by upstream, and there is no defined trigger to allow
- * transmitting redirected multi-buffer frames.
- */
- if (param->is_xdp_mb && !is_redirect)
+ if (param->is_xdp_mb)
set_bit(MLX5E_SQ_STATE_XDP_MULTIBUF, &sq->state);
err = mlx5e_create_sq_rdy(c->mdev, param, &csp, 0, &sq->sqn);
@@ -1888,7 +1889,6 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
struct mlx5e_tx_wqe *wqe = mlx5_wq_cyc_get_wqe(&sq->wq, i);
struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl;
struct mlx5_wqe_eth_seg *eseg = &wqe->eth;
- struct mlx5_wqe_data_seg *dseg;
sq->db.wqe_info[i] = (struct mlx5e_xdp_wqe_info) {
.num_wqebbs = 1,
@@ -1897,9 +1897,6 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
cseg->qpn_ds = cpu_to_be32((sq->sqn << 8) | ds_cnt);
eseg->inline_hdr.sz = cpu_to_be16(inline_hdr_sz);
-
- dseg = (struct mlx5_wqe_data_seg *)cseg + (ds_cnt - 1);
- dseg->lkey = sq->mkey_be;
}
}
@@ -4066,9 +4063,9 @@ void mlx5e_set_xdp_feature(struct net_device *netdev)
val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
NETDEV_XDP_ACT_XSK_ZEROCOPY |
- NETDEV_XDP_ACT_NDO_XMIT;
- if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC)
- val |= NETDEV_XDP_ACT_RX_SG;
+ NETDEV_XDP_ACT_RX_SG |
+ NETDEV_XDP_ACT_NDO_XMIT |
+ NETDEV_XDP_ACT_NDO_XMIT_SG;
xdp_set_features_flag(netdev, val);
}
@@ -4262,19 +4259,24 @@ static bool mlx5e_params_validate_xdp(struct net_device *netdev,
/* No XSK params: AF_XDP can't be enabled yet at the point of setting
* the XDP program.
*/
- is_linear = mlx5e_rx_is_linear_skb(mdev, params, NULL);
+ is_linear = params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC ?
+ mlx5e_rx_is_linear_skb(mdev, params, NULL) :
+ mlx5e_rx_mpwqe_is_linear_skb(mdev, params, NULL);
- if (!is_linear && params->rq_wq_type != MLX5_WQ_TYPE_CYCLIC) {
- netdev_warn(netdev, "XDP is not allowed with striding RQ and MTU(%d) > %d\n",
- params->sw_mtu,
- mlx5e_xdp_max_mtu(params, NULL));
- return false;
- }
- if (!is_linear && !params->xdp_prog->aux->xdp_has_frags) {
- netdev_warn(netdev, "MTU(%d) > %d, too big for an XDP program not aware of multi buffer\n",
- params->sw_mtu,
- mlx5e_xdp_max_mtu(params, NULL));
- return false;
+ if (!is_linear) {
+ if (!params->xdp_prog->aux->xdp_has_frags) {
+ netdev_warn(netdev, "MTU(%d) > %d, too big for an XDP program not aware of multi buffer\n",
+ params->sw_mtu,
+ mlx5e_xdp_max_mtu(params, NULL));
+ return false;
+ }
+ if (params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ &&
+ !mlx5e_verify_params_rx_mpwqe_strides(mdev, params, NULL)) {
+ netdev_warn(netdev, "XDP is not allowed with striding RQ and MTU(%d) > %d\n",
+ params->sw_mtu,
+ mlx5e_xdp_max_mtu(params, NULL));
+ return false;
+ }
}
return true;
@@ -4766,20 +4768,15 @@ static void mlx5e_tx_timeout(struct net_device *dev, unsigned int txqueue)
queue_work(priv->wq, &priv->tx_timeout_work);
}
-static int mlx5e_xdp_allowed(struct mlx5e_priv *priv, struct bpf_prog *prog)
+static int mlx5e_xdp_allowed(struct net_device *netdev, struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params)
{
- struct net_device *netdev = priv->netdev;
- struct mlx5e_params new_params;
-
- if (priv->channels.params.packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
+ if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
netdev_warn(netdev, "can't set XDP while HW-GRO/LRO is on, disable them first\n");
return -EINVAL;
}
- new_params = priv->channels.params;
- new_params.xdp_prog = prog;
-
- if (!mlx5e_params_validate_xdp(netdev, priv->mdev, &new_params))
+ if (!mlx5e_params_validate_xdp(netdev, mdev, params))
return -EINVAL;
return 0;
@@ -4806,8 +4803,11 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
mutex_lock(&priv->state_lock);
+ new_params = priv->channels.params;
+ new_params.xdp_prog = prog;
+
if (prog) {
- err = mlx5e_xdp_allowed(priv, prog);
+ err = mlx5e_xdp_allowed(netdev, priv->mdev, &new_params);
if (err)
goto unlock;
}
@@ -4815,22 +4815,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
/* no need for full reset when exchanging programs */
reset = (!priv->channels.params.xdp_prog || !prog);
- new_params = priv->channels.params;
- new_params.xdp_prog = prog;
-
- /* XDP affects striding RQ parameters. Block XDP if striding RQ won't be
- * supported with the new parameters: if PAGE_SIZE is bigger than
- * MLX5_MPWQE_LOG_STRIDE_SZ_MAX, striding RQ can't be used, even though
- * the MTU is small enough for the linear mode, because XDP uses strides
- * of PAGE_SIZE on regular RQs.
- */
- if (reset && MLX5E_GET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ)) {
- /* Checking for regular RQs here; XSK RQs were checked on XSK bind. */
- err = mlx5e_mpwrq_validate_regular(priv->mdev, &new_params);
- if (err)
- goto unlock;
- }
-
old_prog = priv->channels.params.xdp_prog;
err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, reset);
@@ -5125,6 +5109,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
netdev->vlan_features |= NETIF_F_SG;
netdev->vlan_features |= NETIF_F_HW_CSUM;
+ netdev->vlan_features |= NETIF_F_HW_MACSEC;
netdev->vlan_features |= NETIF_F_GRO;
netdev->vlan_features |= NETIF_F_TSO;
netdev->vlan_features |= NETIF_F_TSO6;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 1049805..a8c2ae3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -471,6 +471,35 @@ static int mlx5e_refill_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk)
return i;
}
+static void
+mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinfo,
+ struct xdp_buff *xdp, struct mlx5e_frag_page *frag_page,
+ u32 frag_offset, u32 len)
+{
+ skb_frag_t *frag;
+
+ dma_addr_t addr = page_pool_get_dma_addr(frag_page->page);
+
+ dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_dir);
+ if (!xdp_buff_has_frags(xdp)) {
+ /* Init on the first fragment to avoid cold cache access
+ * when possible.
+ */
+ sinfo->nr_frags = 0;
+ sinfo->xdp_frags_size = 0;
+ xdp_buff_set_frags_flag(xdp);
+ }
+
+ frag = &sinfo->frags[sinfo->nr_frags++];
+ __skb_frag_set_page(frag, frag_page->page);
+ skb_frag_off_set(frag, frag_offset);
+ skb_frag_size_set(frag, len);
+
+ if (page_is_pfmemalloc(frag_page->page))
+ xdp_buff_set_frag_pfmemalloc(xdp);
+ sinfo->xdp_frags_size += len;
+}
+
static inline void
mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buff *skb,
struct page *page, u32 frag_offset, u32 len,
@@ -1601,10 +1630,10 @@ struct sk_buff *mlx5e_build_linear_skb(struct mlx5e_rq *rq, void *va,
}
static void mlx5e_fill_mxbuf(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
- void *va, u16 headroom, u32 len,
+ void *va, u16 headroom, u32 frame_sz, u32 len,
struct mlx5e_xdp_buff *mxbuf)
{
- xdp_init_buff(&mxbuf->xdp, rq->buff.frame0_sz, &rq->xdp_rxq);
+ xdp_init_buff(&mxbuf->xdp, frame_sz, &rq->xdp_rxq);
xdp_prepare_buff(&mxbuf->xdp, va, headroom, len, true);
mxbuf->cqe = cqe;
mxbuf->rq = rq;
@@ -1637,7 +1666,8 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi,
struct mlx5e_xdp_buff mxbuf;
net_prefetchw(va); /* xdp_frame data area */
- mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, cqe_bcnt, &mxbuf);
+ mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz,
+ cqe_bcnt, &mxbuf);
if (mlx5e_xdp_handle(rq, prog, &mxbuf))
return NULL; /* page/packet was consumed by XDP */
@@ -1685,7 +1715,8 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi
net_prefetchw(va); /* xdp_frame data area */
net_prefetch(va + rx_headroom);
- mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, frag_consumed_bytes, &mxbuf);
+ mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz,
+ frag_consumed_bytes, &mxbuf);
sinfo = xdp_get_shared_info_from_buff(&mxbuf.xdp);
truesize = 0;
@@ -1694,35 +1725,12 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi
wi++;
while (cqe_bcnt) {
- skb_frag_t *frag;
-
frag_page = wi->frag_page;
frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt);
- addr = page_pool_get_dma_addr(frag_page->page);
- dma_sync_single_for_cpu(rq->pdev, addr + wi->offset,
- frag_consumed_bytes, rq->buff.map_dir);
-
- if (!xdp_buff_has_frags(&mxbuf.xdp)) {
- /* Init on the first fragment to avoid cold cache access
- * when possible.
- */
- sinfo->nr_frags = 0;
- sinfo->xdp_frags_size = 0;
- xdp_buff_set_frags_flag(&mxbuf.xdp);
- }
-
- frag = &sinfo->frags[sinfo->nr_frags++];
-
- __skb_frag_set_page(frag, frag_page->page);
- skb_frag_off_set(frag, wi->offset);
- skb_frag_size_set(frag, frag_consumed_bytes);
-
- if (page_is_pfmemalloc(frag_page->page))
- xdp_buff_set_frag_pfmemalloc(&mxbuf.xdp);
-
- sinfo->xdp_frags_size += frag_consumed_bytes;
+ mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf.xdp, frag_page,
+ wi->offset, frag_consumed_bytes);
truesize += frag_info->frag_stride;
cqe_bcnt -= frag_consumed_bytes;
@@ -1969,35 +1977,139 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
struct mlx5e_frag_page *frag_page = &wi->alloc_units.frag_pages[page_idx];
u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt);
struct mlx5e_frag_page *head_page = frag_page;
- u32 frag_offset = head_offset + headlen;
- u32 byte_cnt = cqe_bcnt - headlen;
+ u32 frag_offset = head_offset;
+ u32 byte_cnt = cqe_bcnt;
+ struct skb_shared_info *sinfo;
+ struct mlx5e_xdp_buff mxbuf;
+ unsigned int truesize = 0;
+ struct bpf_prog *prog;
struct sk_buff *skb;
- dma_addr_t addr;
+ u32 linear_frame_sz;
+ u16 linear_data_len;
+ u16 linear_hr;
+ void *va;
- skb = napi_alloc_skb(rq->cq.napi,
- ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
- if (unlikely(!skb)) {
- rq->stats->buff_alloc_err++;
- return NULL;
+ prog = rcu_dereference(rq->xdp_prog);
+
+ if (prog) {
+ /* area for bpf_xdp_[store|load]_bytes */
+ net_prefetchw(page_address(frag_page->page) + frag_offset);
+ if (unlikely(mlx5e_page_alloc_fragmented(rq, &wi->linear_page))) {
+ rq->stats->buff_alloc_err++;
+ return NULL;
+ }
+ va = page_address(wi->linear_page.page);
+ net_prefetchw(va); /* xdp_frame data area */
+ linear_hr = XDP_PACKET_HEADROOM;
+ linear_data_len = 0;
+ linear_frame_sz = MLX5_SKB_FRAG_SZ(linear_hr + MLX5E_RX_MAX_HEAD);
+ } else {
+ skb = napi_alloc_skb(rq->cq.napi,
+ ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
+ if (unlikely(!skb)) {
+ rq->stats->buff_alloc_err++;
+ return NULL;
+ }
+ skb_mark_for_recycle(skb);
+ va = skb->head;
+ net_prefetchw(va); /* xdp_frame data area */
+ net_prefetchw(skb->data);
+
+ frag_offset += headlen;
+ byte_cnt -= headlen;
+ linear_hr = skb_headroom(skb);
+ linear_data_len = headlen;
+ linear_frame_sz = MLX5_SKB_FRAG_SZ(skb_end_offset(skb));
+ if (unlikely(frag_offset >= PAGE_SIZE)) {
+ frag_page++;
+ frag_offset -= PAGE_SIZE;
+ }
}
- net_prefetchw(skb->data);
+ mlx5e_fill_mxbuf(rq, cqe, va, linear_hr, linear_frame_sz, linear_data_len, &mxbuf);
- /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */
- if (unlikely(frag_offset >= PAGE_SIZE)) {
+ sinfo = xdp_get_shared_info_from_buff(&mxbuf.xdp);
+
+ while (byte_cnt) {
+ /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */
+ u32 pg_consumed_bytes = min_t(u32, PAGE_SIZE - frag_offset, byte_cnt);
+
+ if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state))
+ truesize += pg_consumed_bytes;
+ else
+ truesize += ALIGN(pg_consumed_bytes, BIT(rq->mpwqe.log_stride_sz));
+
+ mlx5e_add_skb_shared_info_frag(rq, sinfo, &mxbuf.xdp, frag_page, frag_offset,
+ pg_consumed_bytes);
+ byte_cnt -= pg_consumed_bytes;
+ frag_offset = 0;
frag_page++;
- frag_offset -= PAGE_SIZE;
}
- skb_mark_for_recycle(skb);
- mlx5e_fill_skb_data(skb, rq, frag_page, byte_cnt, frag_offset);
- /* copy header */
- addr = page_pool_get_dma_addr(head_page->page);
- mlx5e_copy_skb_header(rq, skb, head_page->page, addr,
- head_offset, head_offset, headlen);
- /* skb linear part was allocated with headlen and aligned to long */
- skb->tail += headlen;
- skb->len += headlen;
+ if (prog) {
+ if (mlx5e_xdp_handle(rq, prog, &mxbuf)) {
+ if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) {
+ int i;
+
+ for (i = 0; i < sinfo->nr_frags; i++)
+ /* non-atomic */
+ __set_bit(page_idx + i, wi->skip_release_bitmap);
+ return NULL;
+ }
+ mlx5e_page_release_fragmented(rq, &wi->linear_page);
+ return NULL; /* page/packet was consumed by XDP */
+ }
+
+ skb = mlx5e_build_linear_skb(rq, mxbuf.xdp.data_hard_start,
+ linear_frame_sz,
+ mxbuf.xdp.data - mxbuf.xdp.data_hard_start, 0,
+ mxbuf.xdp.data - mxbuf.xdp.data_meta);
+ if (unlikely(!skb)) {
+ mlx5e_page_release_fragmented(rq, &wi->linear_page);
+ return NULL;
+ }
+
+ skb_mark_for_recycle(skb);
+ wi->linear_page.frags++;
+ mlx5e_page_release_fragmented(rq, &wi->linear_page);
+
+ if (xdp_buff_has_frags(&mxbuf.xdp)) {
+ struct mlx5e_frag_page *pagep;
+
+ /* sinfo->nr_frags is reset by build_skb, calculate again. */
+ xdp_update_skb_shared_info(skb, frag_page - head_page,
+ sinfo->xdp_frags_size, truesize,
+ xdp_buff_is_frag_pfmemalloc(&mxbuf.xdp));
+
+ pagep = head_page;
+ do
+ pagep->frags++;
+ while (++pagep < frag_page);
+ }
+ __pskb_pull_tail(skb, headlen);
+ } else {
+ dma_addr_t addr;
+
+ if (xdp_buff_has_frags(&mxbuf.xdp)) {
+ struct mlx5e_frag_page *pagep;
+
+ xdp_update_skb_shared_info(skb, sinfo->nr_frags,
+ sinfo->xdp_frags_size, truesize,
+ xdp_buff_is_frag_pfmemalloc(&mxbuf.xdp));
+
+ pagep = frag_page - sinfo->nr_frags;
+ do
+ pagep->frags++;
+ while (++pagep < frag_page);
+ }
+ /* copy header */
+ addr = page_pool_get_dma_addr(head_page->page);
+ mlx5e_copy_skb_header(rq, skb, head_page->page, addr,
+ head_offset, head_offset, headlen);
+ /* skb linear part was allocated with headlen and aligned to long */
+ skb->tail += headlen;
+ skb->len += headlen;
+ }
return skb;
}
@@ -2036,7 +2148,8 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
struct mlx5e_xdp_buff mxbuf;
net_prefetchw(va); /* xdp_frame data area */
- mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, cqe_bcnt, &mxbuf);
+ mlx5e_fill_mxbuf(rq, cqe, va, rx_headroom, rq->buff.frame0_sz,
+ cqe_bcnt, &mxbuf);
if (mlx5e_xdp_handle(rq, prog, &mxbuf)) {
if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags))
__set_bit(page_idx, wi->skip_release_bitmap); /* non-atomic */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
index eb41f0a..1c35d721 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
@@ -1070,10 +1070,11 @@ mlx5_comp_irq_get_affinity_mask(struct mlx5_core_dev *dev, int vector)
list_for_each_entry(eq, &table->comp_eqs_list, list) {
if (i++ == vector)
- break;
+ return mlx5_irq_get_affinity_mask(eq->core.irq);
}
- return mlx5_irq_get_affinity_mask(eq->core.irq);
+ WARN_ON_ONCE(1);
+ return NULL;
}
EXPORT_SYMBOL(mlx5_comp_irq_get_affinity_mask);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
index 3cdcb0e..1ba03e2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
@@ -13,66 +13,6 @@
#define CREATE_TRACE_POINTS
#include "diag/bridge_tracepoint.h"
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE 12000
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_UNTAGGED_GRP_SIZE 16000
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_FROM 0
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_INGRESS_TABLE_UNTAGGED_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_INGRESS_TABLE_SIZE \
- (MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_TO + 1)
-static_assert(MLX5_ESW_BRIDGE_INGRESS_TABLE_SIZE == 64000);
-
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE 16000
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_SIZE (32000 - 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_FROM 0
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_TO \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_FROM + \
- MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_SIZE - 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_FROM \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_TO + 1)
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_TO \
- MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_FROM
-#define MLX5_ESW_BRIDGE_EGRESS_TABLE_SIZE \
- (MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_TO + 1)
-static_assert(MLX5_ESW_BRIDGE_EGRESS_TABLE_SIZE == 64000);
-
-#define MLX5_ESW_BRIDGE_SKIP_TABLE_SIZE 0
-
-enum {
- MLX5_ESW_BRIDGE_LEVEL_INGRESS_TABLE,
- MLX5_ESW_BRIDGE_LEVEL_EGRESS_TABLE,
- MLX5_ESW_BRIDGE_LEVEL_SKIP_TABLE,
-};
-
static const struct rhashtable_params fdb_ht_params = {
.key_offset = offsetof(struct mlx5_esw_bridge_fdb_entry, key),
.key_len = sizeof(struct mlx5_esw_bridge_fdb_key),
@@ -80,31 +20,6 @@ static const struct rhashtable_params fdb_ht_params = {
.automatic_shrinking = true,
};
-enum {
- MLX5_ESW_BRIDGE_VLAN_FILTERING_FLAG = BIT(0),
-};
-
-struct mlx5_esw_bridge {
- int ifindex;
- int refcnt;
- struct list_head list;
- struct mlx5_esw_bridge_offloads *br_offloads;
-
- struct list_head fdb_list;
- struct rhashtable fdb_ht;
-
- struct mlx5_flow_table *egress_ft;
- struct mlx5_flow_group *egress_vlan_fg;
- struct mlx5_flow_group *egress_qinq_fg;
- struct mlx5_flow_group *egress_mac_fg;
- struct mlx5_flow_group *egress_miss_fg;
- struct mlx5_pkt_reformat *egress_miss_pkt_reformat;
- struct mlx5_flow_handle *egress_miss_handle;
- unsigned long ageing_time;
- u32 flags;
- u16 vlan_proto;
-};
-
static void
mlx5_esw_bridge_fdb_offload_notify(struct net_device *dev, const unsigned char *addr, u16 vid,
unsigned long val)
@@ -146,7 +61,7 @@ mlx5_esw_bridge_pkt_reformat_vlan_pop_create(struct mlx5_eswitch *esw)
return mlx5_packet_reformat_alloc(esw->dev, &reformat_params, MLX5_FLOW_NAMESPACE_FDB);
}
-static struct mlx5_flow_table *
+struct mlx5_flow_table *
mlx5_esw_bridge_table_create(int max_fte, u32 level, struct mlx5_eswitch *esw)
{
struct mlx5_flow_table_attr ft_attr = {};
@@ -925,6 +840,10 @@ static struct mlx5_esw_bridge *mlx5_esw_bridge_create(int ifindex,
if (err)
goto err_fdb_ht;
+ err = mlx5_esw_bridge_mdb_init(bridge);
+ if (err)
+ goto err_mdb_ht;
+
INIT_LIST_HEAD(&bridge->fdb_list);
bridge->ifindex = ifindex;
bridge->refcnt = 1;
@@ -934,6 +853,8 @@ static struct mlx5_esw_bridge *mlx5_esw_bridge_create(int ifindex,
return bridge;
+err_mdb_ht:
+ rhashtable_destroy(&bridge->fdb_ht);
err_fdb_ht:
mlx5_esw_bridge_egress_table_cleanup(bridge);
err_egress_tbl:
@@ -953,7 +874,9 @@ static void mlx5_esw_bridge_put(struct mlx5_esw_bridge_offloads *br_offloads,
return;
mlx5_esw_bridge_egress_table_cleanup(bridge);
+ mlx5_esw_bridge_mcast_disable(bridge);
list_del(&bridge->list);
+ mlx5_esw_bridge_mdb_cleanup(bridge);
rhashtable_destroy(&bridge->fdb_ht);
kvfree(bridge);
@@ -993,7 +916,7 @@ static unsigned long mlx5_esw_bridge_port_key_from_data(u16 vport_num, u16 esw_o
return vport_num | (unsigned long)esw_owner_vhca_id << sizeof(vport_num) * BITS_PER_BYTE;
}
-static unsigned long mlx5_esw_bridge_port_key(struct mlx5_esw_bridge_port *port)
+unsigned long mlx5_esw_bridge_port_key(struct mlx5_esw_bridge_port *port)
{
return mlx5_esw_bridge_port_key_from_data(port->vport_num, port->esw_owner_vhca_id);
}
@@ -1018,6 +941,19 @@ static void mlx5_esw_bridge_port_erase(struct mlx5_esw_bridge_port *port,
xa_erase(&br_offloads->ports, mlx5_esw_bridge_port_key(port));
}
+static struct mlx5_esw_bridge *
+mlx5_esw_bridge_from_port_lookup(u16 vport_num, u16 esw_owner_vhca_id,
+ struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_esw_bridge_port *port;
+
+ port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!port)
+ return NULL;
+
+ return port->bridge;
+}
+
static void mlx5_esw_bridge_fdb_entry_refresh(struct mlx5_esw_bridge_fdb_entry *entry)
{
trace_mlx5_esw_bridge_fdb_entry_refresh(entry);
@@ -1166,8 +1102,21 @@ mlx5_esw_bridge_vlan_push_mark_cleanup(struct mlx5_esw_bridge_vlan *vlan, struct
}
static int
-mlx5_esw_bridge_vlan_push_pop_create(u16 vlan_proto, u16 flags, struct mlx5_esw_bridge_vlan *vlan,
- struct mlx5_eswitch *esw)
+mlx5_esw_bridge_vlan_push_pop_fhs_create(u16 vlan_proto, struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan)
+{
+ return mlx5_esw_bridge_vlan_mcast_init(vlan_proto, port, vlan);
+}
+
+static void
+mlx5_esw_bridge_vlan_push_pop_fhs_cleanup(struct mlx5_esw_bridge_vlan *vlan)
+{
+ mlx5_esw_bridge_vlan_mcast_cleanup(vlan);
+}
+
+static int
+mlx5_esw_bridge_vlan_push_pop_create(u16 vlan_proto, u16 flags, struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan, struct mlx5_eswitch *esw)
{
int err;
@@ -1185,10 +1134,16 @@ mlx5_esw_bridge_vlan_push_pop_create(u16 vlan_proto, u16 flags, struct mlx5_esw_
err = mlx5_esw_bridge_vlan_pop_create(vlan, esw);
if (err)
goto err_vlan_pop;
+
+ err = mlx5_esw_bridge_vlan_push_pop_fhs_create(vlan_proto, port, vlan);
+ if (err)
+ goto err_vlan_pop_fhs;
}
return 0;
+err_vlan_pop_fhs:
+ mlx5_esw_bridge_vlan_pop_cleanup(vlan, esw);
err_vlan_pop:
if (vlan->pkt_mod_hdr_push_mark)
mlx5_esw_bridge_vlan_push_mark_cleanup(vlan, esw);
@@ -1213,7 +1168,7 @@ mlx5_esw_bridge_vlan_create(u16 vlan_proto, u16 vid, u16 flags, struct mlx5_esw_
vlan->flags = flags;
INIT_LIST_HEAD(&vlan->fdb_list);
- err = mlx5_esw_bridge_vlan_push_pop_create(vlan_proto, flags, vlan, esw);
+ err = mlx5_esw_bridge_vlan_push_pop_create(vlan_proto, flags, port, vlan, esw);
if (err)
goto err_vlan_push_pop;
@@ -1225,6 +1180,8 @@ mlx5_esw_bridge_vlan_create(u16 vlan_proto, u16 vid, u16 flags, struct mlx5_esw_
return vlan;
err_xa_insert:
+ if (vlan->mcast_handle)
+ mlx5_esw_bridge_vlan_push_pop_fhs_cleanup(vlan);
if (vlan->pkt_reformat_pop)
mlx5_esw_bridge_vlan_pop_cleanup(vlan, esw);
if (vlan->pkt_mod_hdr_push_mark)
@@ -1242,7 +1199,8 @@ static void mlx5_esw_bridge_vlan_erase(struct mlx5_esw_bridge_port *port,
xa_erase(&port->vlans, vlan->vid);
}
-static void mlx5_esw_bridge_vlan_flush(struct mlx5_esw_bridge_vlan *vlan,
+static void mlx5_esw_bridge_vlan_flush(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan,
struct mlx5_esw_bridge *bridge)
{
struct mlx5_eswitch *esw = bridge->br_offloads->esw;
@@ -1250,7 +1208,10 @@ static void mlx5_esw_bridge_vlan_flush(struct mlx5_esw_bridge_vlan *vlan,
list_for_each_entry_safe(entry, tmp, &vlan->fdb_list, vlan_list)
mlx5_esw_bridge_fdb_entry_notify_and_cleanup(entry, bridge);
+ mlx5_esw_bridge_port_mdb_vlan_flush(port, vlan);
+ if (vlan->mcast_handle)
+ mlx5_esw_bridge_vlan_push_pop_fhs_cleanup(vlan);
if (vlan->pkt_reformat_pop)
mlx5_esw_bridge_vlan_pop_cleanup(vlan, esw);
if (vlan->pkt_mod_hdr_push_mark)
@@ -1264,7 +1225,7 @@ static void mlx5_esw_bridge_vlan_cleanup(struct mlx5_esw_bridge_port *port,
struct mlx5_esw_bridge *bridge)
{
trace_mlx5_esw_bridge_vlan_cleanup(vlan);
- mlx5_esw_bridge_vlan_flush(vlan, bridge);
+ mlx5_esw_bridge_vlan_flush(port, vlan, bridge);
mlx5_esw_bridge_vlan_erase(port, vlan);
kvfree(vlan);
}
@@ -1288,9 +1249,9 @@ static int mlx5_esw_bridge_port_vlans_recreate(struct mlx5_esw_bridge_port *port
int err;
xa_for_each(&port->vlans, i, vlan) {
- mlx5_esw_bridge_vlan_flush(vlan, bridge);
- err = mlx5_esw_bridge_vlan_push_pop_create(bridge->vlan_proto, vlan->flags, vlan,
- br_offloads->esw);
+ mlx5_esw_bridge_vlan_flush(port, vlan, bridge);
+ err = mlx5_esw_bridge_vlan_push_pop_create(bridge->vlan_proto, vlan->flags, port,
+ vlan, br_offloads->esw);
if (err) {
esw_warn(br_offloads->esw->dev,
"Failed to create VLAN=%u(proto=%x) push/pop actions (vport=%u,err=%d)\n",
@@ -1473,33 +1434,32 @@ mlx5_esw_bridge_fdb_entry_init(struct net_device *dev, u16 vport_num, u16 esw_ow
int mlx5_esw_bridge_ageing_time_set(u16 vport_num, u16 esw_owner_vhca_id, unsigned long ageing_time,
struct mlx5_esw_bridge_offloads *br_offloads)
{
- struct mlx5_esw_bridge_port *port;
+ struct mlx5_esw_bridge *bridge;
- port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
- if (!port)
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!bridge)
return -EINVAL;
- port->bridge->ageing_time = clock_t_to_jiffies(ageing_time);
+ bridge->ageing_time = clock_t_to_jiffies(ageing_time);
return 0;
}
int mlx5_esw_bridge_vlan_filtering_set(u16 vport_num, u16 esw_owner_vhca_id, bool enable,
struct mlx5_esw_bridge_offloads *br_offloads)
{
- struct mlx5_esw_bridge_port *port;
struct mlx5_esw_bridge *bridge;
bool filtering;
- port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
- if (!port)
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!bridge)
return -EINVAL;
- bridge = port->bridge;
filtering = bridge->flags & MLX5_ESW_BRIDGE_VLAN_FILTERING_FLAG;
if (filtering == enable)
return 0;
mlx5_esw_bridge_fdb_flush(bridge);
+ mlx5_esw_bridge_mdb_flush(bridge);
if (enable)
bridge->flags |= MLX5_ESW_BRIDGE_VLAN_FILTERING_FLAG;
else
@@ -1511,15 +1471,13 @@ int mlx5_esw_bridge_vlan_filtering_set(u16 vport_num, u16 esw_owner_vhca_id, boo
int mlx5_esw_bridge_vlan_proto_set(u16 vport_num, u16 esw_owner_vhca_id, u16 proto,
struct mlx5_esw_bridge_offloads *br_offloads)
{
- struct mlx5_esw_bridge_port *port;
struct mlx5_esw_bridge *bridge;
- port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id,
- br_offloads);
- if (!port)
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id,
+ br_offloads);
+ if (!bridge)
return -EINVAL;
- bridge = port->bridge;
if (bridge->vlan_proto == proto)
return 0;
if (proto != ETH_P_8021Q && proto != ETH_P_8021AD) {
@@ -1528,12 +1486,43 @@ int mlx5_esw_bridge_vlan_proto_set(u16 vport_num, u16 esw_owner_vhca_id, u16 pro
}
mlx5_esw_bridge_fdb_flush(bridge);
+ mlx5_esw_bridge_mdb_flush(bridge);
bridge->vlan_proto = proto;
mlx5_esw_bridge_vlans_recreate(bridge);
return 0;
}
+int mlx5_esw_bridge_mcast_set(u16 vport_num, u16 esw_owner_vhca_id, bool enable,
+ struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_eswitch *esw = br_offloads->esw;
+ struct mlx5_esw_bridge *bridge;
+ int err = 0;
+ bool mcast;
+
+ if (!(MLX5_CAP_ESW_FLOWTABLE((esw)->dev, fdb_multi_path_any_table) ||
+ MLX5_CAP_ESW_FLOWTABLE((esw)->dev, fdb_multi_path_any_table_limit_regc)) ||
+ !MLX5_CAP_ESW_FLOWTABLE((esw)->dev, fdb_uplink_hairpin) ||
+ !MLX5_CAP_ESW_FLOWTABLE_FDB((esw)->dev, ignore_flow_level))
+ return -EOPNOTSUPP;
+
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!bridge)
+ return -EINVAL;
+
+ mcast = bridge->flags & MLX5_ESW_BRIDGE_MCAST_FLAG;
+ if (mcast == enable)
+ return 0;
+
+ if (enable)
+ err = mlx5_esw_bridge_mcast_enable(bridge);
+ else
+ mlx5_esw_bridge_mcast_disable(bridge);
+
+ return err;
+}
+
static int mlx5_esw_bridge_vport_init(u16 vport_num, u16 esw_owner_vhca_id, u16 flags,
struct mlx5_esw_bridge_offloads *br_offloads,
struct mlx5_esw_bridge *bridge)
@@ -1551,6 +1540,15 @@ static int mlx5_esw_bridge_vport_init(u16 vport_num, u16 esw_owner_vhca_id, u16
port->bridge = bridge;
port->flags |= flags;
xa_init(&port->vlans);
+
+ err = mlx5_esw_bridge_port_mcast_init(port);
+ if (err) {
+ esw_warn(esw->dev,
+ "Failed to initialize port multicast (vport=%u,esw_owner_vhca_id=%u,err=%d)\n",
+ port->vport_num, port->esw_owner_vhca_id, err);
+ goto err_port_mcast;
+ }
+
err = mlx5_esw_bridge_port_insert(port, br_offloads);
if (err) {
esw_warn(esw->dev,
@@ -1563,6 +1561,8 @@ static int mlx5_esw_bridge_vport_init(u16 vport_num, u16 esw_owner_vhca_id, u16
return 0;
err_port_insert:
+ mlx5_esw_bridge_port_mcast_cleanup(port);
+err_port_mcast:
kvfree(port);
return err;
}
@@ -1580,6 +1580,7 @@ static int mlx5_esw_bridge_vport_cleanup(struct mlx5_esw_bridge_offloads *br_off
trace_mlx5_esw_bridge_vport_cleanup(port);
mlx5_esw_bridge_port_vlans_flush(port, bridge);
+ mlx5_esw_bridge_port_mcast_cleanup(port);
mlx5_esw_bridge_port_erase(port, br_offloads);
kvfree(port);
mlx5_esw_bridge_put(br_offloads, bridge);
@@ -1711,14 +1712,12 @@ void mlx5_esw_bridge_fdb_update_used(struct net_device *dev, u16 vport_num, u16
struct switchdev_notifier_fdb_info *fdb_info)
{
struct mlx5_esw_bridge_fdb_entry *entry;
- struct mlx5_esw_bridge_port *port;
struct mlx5_esw_bridge *bridge;
- port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
- if (!port)
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!bridge)
return;
- bridge = port->bridge;
entry = mlx5_esw_bridge_fdb_lookup(bridge, fdb_info->addr, fdb_info->vid);
if (!entry) {
esw_debug(br_offloads->esw->dev,
@@ -1765,14 +1764,12 @@ void mlx5_esw_bridge_fdb_remove(struct net_device *dev, u16 vport_num, u16 esw_o
{
struct mlx5_eswitch *esw = br_offloads->esw;
struct mlx5_esw_bridge_fdb_entry *entry;
- struct mlx5_esw_bridge_port *port;
struct mlx5_esw_bridge *bridge;
- port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
- if (!port)
+ bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!bridge)
return;
- bridge = port->bridge;
entry = mlx5_esw_bridge_fdb_lookup(bridge, fdb_info->addr, fdb_info->vid);
if (!entry) {
esw_debug(esw->dev,
@@ -1806,6 +1803,64 @@ void mlx5_esw_bridge_update(struct mlx5_esw_bridge_offloads *br_offloads)
}
}
+int mlx5_esw_bridge_port_mdb_add(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id,
+ const unsigned char *addr, u16 vid,
+ struct mlx5_esw_bridge_offloads *br_offloads,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_esw_bridge_vlan *vlan;
+ struct mlx5_esw_bridge_port *port;
+ struct mlx5_esw_bridge *bridge;
+ int err;
+
+ port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!port) {
+ esw_warn(br_offloads->esw->dev,
+ "Failed to lookup bridge port to add MDB (MAC=%pM,vport=%u)\n",
+ addr, vport_num);
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "Failed to lookup bridge port to add MDB (MAC=%pM,vport=%u)\n",
+ addr, vport_num);
+ return -EINVAL;
+ }
+
+ bridge = port->bridge;
+ if (bridge->flags & MLX5_ESW_BRIDGE_VLAN_FILTERING_FLAG && vid) {
+ vlan = mlx5_esw_bridge_vlan_lookup(vid, port);
+ if (!vlan) {
+ esw_warn(br_offloads->esw->dev,
+ "Failed to lookup bridge port vlan metadata to create MDB (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, vport_num);
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "Failed to lookup bridge port vlan metadata to create MDB (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, vport_num);
+ return -EINVAL;
+ }
+ }
+
+ err = mlx5_esw_bridge_port_mdb_attach(dev, port, addr, vid);
+ if (err) {
+ NL_SET_ERR_MSG_FMT_MOD(extack, "Failed to add MDB (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, vport_num);
+ return err;
+ }
+
+ return 0;
+}
+
+void mlx5_esw_bridge_port_mdb_del(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id,
+ const unsigned char *addr, u16 vid,
+ struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_esw_bridge_port *port;
+
+ port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+ if (!port)
+ return;
+
+ mlx5_esw_bridge_port_mdb_detach(dev, port, addr, vid);
+}
+
static void mlx5_esw_bridge_flush(struct mlx5_esw_bridge_offloads *br_offloads)
{
struct mlx5_esw_bridge_port *port;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
index 10851a5..a9dd18c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
@@ -25,12 +25,19 @@ struct mlx5_esw_bridge_offloads {
struct delayed_work update_work;
struct mlx5_flow_table *ingress_ft;
+ struct mlx5_flow_group *ingress_igmp_fg;
+ struct mlx5_flow_group *ingress_mld_fg;
struct mlx5_flow_group *ingress_vlan_fg;
struct mlx5_flow_group *ingress_vlan_filter_fg;
struct mlx5_flow_group *ingress_qinq_fg;
struct mlx5_flow_group *ingress_qinq_filter_fg;
struct mlx5_flow_group *ingress_mac_fg;
+ struct mlx5_flow_handle *igmp_handle;
+ struct mlx5_flow_handle *mld_query_handle;
+ struct mlx5_flow_handle *mld_report_handle;
+ struct mlx5_flow_handle *mld_done_handle;
+
struct mlx5_flow_table *skip_ft;
};
@@ -64,10 +71,20 @@ int mlx5_esw_bridge_vlan_filtering_set(u16 vport_num, u16 esw_owner_vhca_id, boo
struct mlx5_esw_bridge_offloads *br_offloads);
int mlx5_esw_bridge_vlan_proto_set(u16 vport_num, u16 esw_owner_vhca_id, u16 proto,
struct mlx5_esw_bridge_offloads *br_offloads);
+int mlx5_esw_bridge_mcast_set(u16 vport_num, u16 esw_owner_vhca_id, bool enable,
+ struct mlx5_esw_bridge_offloads *br_offloads);
int mlx5_esw_bridge_port_vlan_add(u16 vport_num, u16 esw_owner_vhca_id, u16 vid, u16 flags,
struct mlx5_esw_bridge_offloads *br_offloads,
struct netlink_ext_ack *extack);
void mlx5_esw_bridge_port_vlan_del(u16 vport_num, u16 esw_owner_vhca_id, u16 vid,
struct mlx5_esw_bridge_offloads *br_offloads);
+int mlx5_esw_bridge_port_mdb_add(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id,
+ const unsigned char *addr, u16 vid,
+ struct mlx5_esw_bridge_offloads *br_offloads,
+ struct netlink_ext_ack *extack);
+void mlx5_esw_bridge_port_mdb_del(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id,
+ const unsigned char *addr, u16 vid,
+ struct mlx5_esw_bridge_offloads *br_offloads);
+
#endif /* __MLX5_ESW_BRIDGE_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c
new file mode 100644
index 0000000..2eae594
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c
@@ -0,0 +1,1126 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include "lib/devcom.h"
+#include "bridge.h"
+#include "eswitch.h"
+#include "bridge_priv.h"
+#include "diag/bridge_tracepoint.h"
+
+static const struct rhashtable_params mdb_ht_params = {
+ .key_offset = offsetof(struct mlx5_esw_bridge_mdb_entry, key),
+ .key_len = sizeof(struct mlx5_esw_bridge_mdb_key),
+ .head_offset = offsetof(struct mlx5_esw_bridge_mdb_entry, ht_node),
+ .automatic_shrinking = true,
+};
+
+int mlx5_esw_bridge_mdb_init(struct mlx5_esw_bridge *bridge)
+{
+ INIT_LIST_HEAD(&bridge->mdb_list);
+ return rhashtable_init(&bridge->mdb_ht, &mdb_ht_params);
+}
+
+void mlx5_esw_bridge_mdb_cleanup(struct mlx5_esw_bridge *bridge)
+{
+ rhashtable_destroy(&bridge->mdb_ht);
+}
+
+static struct mlx5_esw_bridge_port *
+mlx5_esw_bridge_mdb_port_lookup(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ return xa_load(&entry->ports, mlx5_esw_bridge_port_key(port));
+}
+
+static int mlx5_esw_bridge_mdb_port_insert(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ int err = xa_insert(&entry->ports, mlx5_esw_bridge_port_key(port), port, GFP_KERNEL);
+
+ if (!err)
+ entry->num_ports++;
+ return err;
+}
+
+static void mlx5_esw_bridge_mdb_port_remove(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ xa_erase(&entry->ports, mlx5_esw_bridge_port_key(port));
+ entry->num_ports--;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mdb_flow_create(u16 esw_owner_vhca_id, struct mlx5_esw_bridge_mdb_entry *entry,
+ struct mlx5_esw_bridge *bridge)
+{
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,
+ .flags = FLOW_ACT_NO_APPEND | FLOW_ACT_IGNORE_FLOW_LEVEL,
+ };
+ int num_dests = entry->num_ports, i = 0;
+ struct mlx5_flow_destination *dests;
+ struct mlx5_esw_bridge_port *port;
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+ u8 *dmac_v, *dmac_c;
+ unsigned long idx;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ dests = kvcalloc(num_dests, sizeof(*dests), GFP_KERNEL);
+ if (!dests) {
+ kvfree(rule_spec);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ xa_for_each(&entry->ports, idx, port) {
+ dests[i].type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
+ dests[i].ft = port->mcast.ft;
+ i++;
+ }
+
+ rule_spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS;
+ dmac_v = MLX5_ADDR_OF(fte_match_param, rule_spec->match_value, outer_headers.dmac_47_16);
+ ether_addr_copy(dmac_v, entry->key.addr);
+ dmac_c = MLX5_ADDR_OF(fte_match_param, rule_spec->match_criteria, outer_headers.dmac_47_16);
+ eth_broadcast_addr(dmac_c);
+
+ if (entry->key.vid) {
+ if (bridge->vlan_proto == ETH_P_8021Q) {
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria,
+ outer_headers.cvlan_tag);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_value,
+ outer_headers.cvlan_tag);
+ } else if (bridge->vlan_proto == ETH_P_8021AD) {
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria,
+ outer_headers.svlan_tag);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_value,
+ outer_headers.svlan_tag);
+ }
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria,
+ outer_headers.first_vid);
+ MLX5_SET(fte_match_param, rule_spec->match_value, outer_headers.first_vid,
+ entry->key.vid);
+ }
+
+ handle = mlx5_add_flow_rules(bridge->egress_ft, rule_spec, &flow_act, dests, num_dests);
+
+ kvfree(dests);
+ kvfree(rule_spec);
+ return handle;
+}
+
+static int
+mlx5_esw_bridge_port_mdb_offload(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ struct mlx5_flow_handle *handle;
+
+ handle = mlx5_esw_bridge_mdb_flow_create(port->esw_owner_vhca_id, entry, port->bridge);
+ if (entry->egress_handle) {
+ mlx5_del_flow_rules(entry->egress_handle);
+ entry->egress_handle = NULL;
+ }
+ if (IS_ERR(handle))
+ return PTR_ERR(handle);
+
+ entry->egress_handle = handle;
+ return 0;
+}
+
+static struct mlx5_esw_bridge_mdb_entry *
+mlx5_esw_bridge_mdb_lookup(struct mlx5_esw_bridge *bridge,
+ const unsigned char *addr, u16 vid)
+{
+ struct mlx5_esw_bridge_mdb_key key = {};
+
+ ether_addr_copy(key.addr, addr);
+ key.vid = vid;
+ return rhashtable_lookup_fast(&bridge->mdb_ht, &key, mdb_ht_params);
+}
+
+static struct mlx5_esw_bridge_mdb_entry *
+mlx5_esw_bridge_port_mdb_entry_init(struct mlx5_esw_bridge_port *port,
+ const unsigned char *addr, u16 vid)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_esw_bridge_mdb_entry *entry;
+ int err;
+
+ entry = kvzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return ERR_PTR(-ENOMEM);
+
+ ether_addr_copy(entry->key.addr, addr);
+ entry->key.vid = vid;
+ xa_init(&entry->ports);
+ err = rhashtable_insert_fast(&bridge->mdb_ht, &entry->ht_node, mdb_ht_params);
+ if (err)
+ goto err_ht_insert;
+
+ list_add(&entry->list, &bridge->mdb_list);
+
+ return entry;
+
+err_ht_insert:
+ xa_destroy(&entry->ports);
+ kvfree(entry);
+ return ERR_PTR(err);
+}
+
+static void mlx5_esw_bridge_port_mdb_entry_cleanup(struct mlx5_esw_bridge *bridge,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ if (entry->egress_handle)
+ mlx5_del_flow_rules(entry->egress_handle);
+ list_del(&entry->list);
+ rhashtable_remove_fast(&bridge->mdb_ht, &entry->ht_node, mdb_ht_params);
+ xa_destroy(&entry->ports);
+ kvfree(entry);
+}
+
+int mlx5_esw_bridge_port_mdb_attach(struct net_device *dev, struct mlx5_esw_bridge_port *port,
+ const unsigned char *addr, u16 vid)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_esw_bridge_mdb_entry *entry;
+ int err;
+
+ if (!(bridge->flags & MLX5_ESW_BRIDGE_MCAST_FLAG))
+ return -EOPNOTSUPP;
+
+ entry = mlx5_esw_bridge_mdb_lookup(bridge, addr, vid);
+ if (entry) {
+ if (mlx5_esw_bridge_mdb_port_lookup(port, entry)) {
+ esw_warn(bridge->br_offloads->esw->dev, "MDB attach entry is already attached to port (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, port->vport_num);
+ return 0;
+ }
+ } else {
+ entry = mlx5_esw_bridge_port_mdb_entry_init(port, addr, vid);
+ if (IS_ERR(entry)) {
+ err = PTR_ERR(entry);
+ esw_warn(bridge->br_offloads->esw->dev, "MDB attach failed to init entry (MAC=%pM,vid=%u,vport=%u,err=%d)\n",
+ addr, vid, port->vport_num, err);
+ return err;
+ }
+ }
+
+ err = mlx5_esw_bridge_mdb_port_insert(port, entry);
+ if (err) {
+ if (!entry->num_ports)
+ mlx5_esw_bridge_port_mdb_entry_cleanup(bridge, entry); /* new mdb entry */
+ esw_warn(bridge->br_offloads->esw->dev,
+ "MDB attach failed to insert port (MAC=%pM,vid=%u,vport=%u,err=%d)\n",
+ addr, vid, port->vport_num, err);
+ return err;
+ }
+
+ err = mlx5_esw_bridge_port_mdb_offload(port, entry);
+ if (err)
+ /* Single mdb can be used by multiple ports, so just log the
+ * error and continue.
+ */
+ esw_warn(bridge->br_offloads->esw->dev, "MDB attach failed to offload (MAC=%pM,vid=%u,vport=%u,err=%d)\n",
+ addr, vid, port->vport_num, err);
+
+ trace_mlx5_esw_bridge_port_mdb_attach(dev, entry);
+ return 0;
+}
+
+static void mlx5_esw_bridge_port_mdb_entry_detach(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_mdb_entry *entry)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ int err;
+
+ mlx5_esw_bridge_mdb_port_remove(port, entry);
+ if (!entry->num_ports) {
+ mlx5_esw_bridge_port_mdb_entry_cleanup(bridge, entry);
+ return;
+ }
+
+ err = mlx5_esw_bridge_port_mdb_offload(port, entry);
+ if (err)
+ /* Single mdb can be used by multiple ports, so just log the
+ * error and continue.
+ */
+ esw_warn(bridge->br_offloads->esw->dev, "MDB detach failed to offload (MAC=%pM,vid=%u,vport=%u)\n",
+ entry->key.addr, entry->key.vid, port->vport_num);
+}
+
+void mlx5_esw_bridge_port_mdb_detach(struct net_device *dev, struct mlx5_esw_bridge_port *port,
+ const unsigned char *addr, u16 vid)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_esw_bridge_mdb_entry *entry;
+
+ entry = mlx5_esw_bridge_mdb_lookup(bridge, addr, vid);
+ if (!entry) {
+ esw_debug(bridge->br_offloads->esw->dev,
+ "MDB detach entry not found (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, port->vport_num);
+ return;
+ }
+
+ if (!mlx5_esw_bridge_mdb_port_lookup(port, entry)) {
+ esw_debug(bridge->br_offloads->esw->dev,
+ "MDB detach entry not attached to the port (MAC=%pM,vid=%u,vport=%u)\n",
+ addr, vid, port->vport_num);
+ return;
+ }
+
+ trace_mlx5_esw_bridge_port_mdb_detach(dev, entry);
+ mlx5_esw_bridge_port_mdb_entry_detach(port, entry);
+}
+
+void mlx5_esw_bridge_port_mdb_vlan_flush(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_esw_bridge_mdb_entry *entry, *tmp;
+
+ list_for_each_entry_safe(entry, tmp, &bridge->mdb_list, list)
+ if (entry->key.vid == vlan->vid && mlx5_esw_bridge_mdb_port_lookup(port, entry))
+ mlx5_esw_bridge_port_mdb_entry_detach(port, entry);
+}
+
+static void mlx5_esw_bridge_port_mdb_flush(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_esw_bridge_mdb_entry *entry, *tmp;
+
+ list_for_each_entry_safe(entry, tmp, &bridge->mdb_list, list)
+ if (mlx5_esw_bridge_mdb_port_lookup(port, entry))
+ mlx5_esw_bridge_port_mdb_entry_detach(port, entry);
+}
+
+void mlx5_esw_bridge_mdb_flush(struct mlx5_esw_bridge *bridge)
+{
+ struct mlx5_esw_bridge_mdb_entry *entry, *tmp;
+
+ list_for_each_entry_safe(entry, tmp, &bridge->mdb_list, list)
+ mlx5_esw_bridge_port_mdb_entry_cleanup(bridge, entry);
+}
+static int mlx5_esw_bridge_port_mcast_fts_init(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge *bridge)
+{
+ struct mlx5_eswitch *esw = bridge->br_offloads->esw;
+ struct mlx5_flow_table *mcast_ft;
+
+ mcast_ft = mlx5_esw_bridge_table_create(MLX5_ESW_BRIDGE_MCAST_TABLE_SIZE,
+ MLX5_ESW_BRIDGE_LEVEL_MCAST_TABLE,
+ esw);
+ if (IS_ERR(mcast_ft))
+ return PTR_ERR(mcast_ft);
+
+ port->mcast.ft = mcast_ft;
+ return 0;
+}
+
+static void mlx5_esw_bridge_port_mcast_fts_cleanup(struct mlx5_esw_bridge_port *port)
+{
+ if (port->mcast.ft)
+ mlx5_destroy_flow_table(port->mcast.ft);
+ port->mcast.ft = NULL;
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_mcast_filter_fg_create(struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *mcast_ft)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_group *fg;
+ u32 *in, *match;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return ERR_PTR(-ENOMEM);
+
+ MLX5_SET(create_flow_group_in, in, match_criteria_enable, MLX5_MATCH_MISC_PARAMETERS_2);
+ match = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria);
+
+ MLX5_SET(fte_match_param, match, misc_parameters_2.metadata_reg_c_0,
+ mlx5_eswitch_get_vport_metadata_mask());
+
+ MLX5_SET(create_flow_group_in, in, start_flow_index,
+ MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_IDX_FROM);
+ MLX5_SET(create_flow_group_in, in, end_flow_index,
+ MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_IDX_TO);
+
+ fg = mlx5_create_flow_group(mcast_ft, in);
+ kvfree(in);
+ if (IS_ERR(fg))
+ esw_warn(esw->dev,
+ "Failed to create filter flow group for bridge mcast table (err=%pe)\n",
+ fg);
+
+ return fg;
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_mcast_vlan_proto_fg_create(unsigned int from, unsigned int to, u16 vlan_proto,
+ struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *mcast_ft)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_group *fg;
+ u32 *in, *match;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return ERR_PTR(-ENOMEM);
+
+ MLX5_SET(create_flow_group_in, in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS);
+ match = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria);
+
+ if (vlan_proto == ETH_P_8021Q)
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.cvlan_tag);
+ else if (vlan_proto == ETH_P_8021AD)
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.svlan_tag);
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.first_vid);
+
+ MLX5_SET(create_flow_group_in, in, start_flow_index, from);
+ MLX5_SET(create_flow_group_in, in, end_flow_index, to);
+
+ fg = mlx5_create_flow_group(mcast_ft, in);
+ kvfree(in);
+ if (IS_ERR(fg))
+ esw_warn(esw->dev,
+ "Failed to create VLAN(proto=%x) flow group for bridge mcast table (err=%pe)\n",
+ vlan_proto, fg);
+
+ return fg;
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_mcast_vlan_fg_create(struct mlx5_eswitch *esw, struct mlx5_flow_table *mcast_ft)
+{
+ unsigned int from = MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_FROM;
+ unsigned int to = MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_TO;
+
+ return mlx5_esw_bridge_mcast_vlan_proto_fg_create(from, to, ETH_P_8021Q, esw, mcast_ft);
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_mcast_qinq_fg_create(struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *mcast_ft)
+{
+ unsigned int from = MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_FROM;
+ unsigned int to = MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_TO;
+
+ return mlx5_esw_bridge_mcast_vlan_proto_fg_create(from, to, ETH_P_8021AD, esw, mcast_ft);
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_mcast_fwd_fg_create(struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *mcast_ft)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_group *fg;
+ u32 *in;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return ERR_PTR(-ENOMEM);
+
+ MLX5_SET(create_flow_group_in, in, start_flow_index,
+ MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_FROM);
+ MLX5_SET(create_flow_group_in, in, end_flow_index,
+ MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_TO);
+
+ fg = mlx5_create_flow_group(mcast_ft, in);
+ kvfree(in);
+ if (IS_ERR(fg))
+ esw_warn(esw->dev,
+ "Failed to create forward flow group for bridge mcast table (err=%pe)\n",
+ fg);
+
+ return fg;
+}
+
+static int mlx5_esw_bridge_port_mcast_fgs_init(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_flow_group *fwd_fg, *qinq_fg, *vlan_fg, *filter_fg;
+ struct mlx5_eswitch *esw = port->bridge->br_offloads->esw;
+ struct mlx5_flow_table *mcast_ft = port->mcast.ft;
+ int err;
+
+ filter_fg = mlx5_esw_bridge_mcast_filter_fg_create(esw, mcast_ft);
+ if (IS_ERR(filter_fg))
+ return PTR_ERR(filter_fg);
+
+ vlan_fg = mlx5_esw_bridge_mcast_vlan_fg_create(esw, mcast_ft);
+ if (IS_ERR(vlan_fg)) {
+ err = PTR_ERR(vlan_fg);
+ goto err_vlan_fg;
+ }
+
+ qinq_fg = mlx5_esw_bridge_mcast_qinq_fg_create(esw, mcast_ft);
+ if (IS_ERR(qinq_fg)) {
+ err = PTR_ERR(qinq_fg);
+ goto err_qinq_fg;
+ }
+
+ fwd_fg = mlx5_esw_bridge_mcast_fwd_fg_create(esw, mcast_ft);
+ if (IS_ERR(fwd_fg)) {
+ err = PTR_ERR(fwd_fg);
+ goto err_fwd_fg;
+ }
+
+ port->mcast.filter_fg = filter_fg;
+ port->mcast.vlan_fg = vlan_fg;
+ port->mcast.qinq_fg = qinq_fg;
+ port->mcast.fwd_fg = fwd_fg;
+
+ return 0;
+
+err_fwd_fg:
+ mlx5_destroy_flow_group(qinq_fg);
+err_qinq_fg:
+ mlx5_destroy_flow_group(vlan_fg);
+err_vlan_fg:
+ mlx5_destroy_flow_group(filter_fg);
+ return err;
+}
+
+static void mlx5_esw_bridge_port_mcast_fgs_cleanup(struct mlx5_esw_bridge_port *port)
+{
+ if (port->mcast.fwd_fg)
+ mlx5_destroy_flow_group(port->mcast.fwd_fg);
+ port->mcast.fwd_fg = NULL;
+ if (port->mcast.qinq_fg)
+ mlx5_destroy_flow_group(port->mcast.qinq_fg);
+ port->mcast.qinq_fg = NULL;
+ if (port->mcast.vlan_fg)
+ mlx5_destroy_flow_group(port->mcast.vlan_fg);
+ port->mcast.vlan_fg = NULL;
+ if (port->mcast.filter_fg)
+ mlx5_destroy_flow_group(port->mcast.filter_fg);
+ port->mcast.filter_fg = NULL;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mcast_flow_with_esw_create(struct mlx5_esw_bridge_port *port,
+ struct mlx5_eswitch *esw)
+{
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_DROP,
+ .flags = FLOW_ACT_NO_APPEND,
+ };
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ rule_spec->match_criteria_enable = MLX5_MATCH_MISC_PARAMETERS_2;
+
+ MLX5_SET(fte_match_param, rule_spec->match_criteria,
+ misc_parameters_2.metadata_reg_c_0, mlx5_eswitch_get_vport_metadata_mask());
+ MLX5_SET(fte_match_param, rule_spec->match_value, misc_parameters_2.metadata_reg_c_0,
+ mlx5_eswitch_get_vport_metadata_for_match(esw, port->vport_num));
+
+ handle = mlx5_add_flow_rules(port->mcast.ft, rule_spec, &flow_act, NULL, 0);
+
+ kvfree(rule_spec);
+ return handle;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mcast_filter_flow_create(struct mlx5_esw_bridge_port *port)
+{
+ return mlx5_esw_bridge_mcast_flow_with_esw_create(port, port->bridge->br_offloads->esw);
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mcast_filter_flow_peer_create(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_devcom *devcom = port->bridge->br_offloads->esw->dev->priv.devcom;
+ static struct mlx5_flow_handle *handle;
+ struct mlx5_eswitch *peer_esw;
+
+ peer_esw = mlx5_devcom_get_peer_data(devcom, MLX5_DEVCOM_ESW_OFFLOADS);
+ if (!peer_esw)
+ return ERR_PTR(-ENODEV);
+
+ handle = mlx5_esw_bridge_mcast_flow_with_esw_create(port, peer_esw);
+
+ mlx5_devcom_release_peer_data(devcom, MLX5_DEVCOM_ESW_OFFLOADS);
+ return handle;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mcast_vlan_flow_create(u16 vlan_proto, struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan)
+{
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,
+ .flags = FLOW_ACT_NO_APPEND,
+ };
+ struct mlx5_flow_destination dest = {
+ .type = MLX5_FLOW_DESTINATION_TYPE_VPORT,
+ .vport.num = port->vport_num,
+ };
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ if (MLX5_CAP_ESW_FLOWTABLE(bridge->br_offloads->esw->dev, flow_source) &&
+ port->vport_num == MLX5_VPORT_UPLINK)
+ rule_spec->flow_context.flow_source =
+ MLX5_FLOW_CONTEXT_FLOW_SOURCE_LOCAL_VPORT;
+ rule_spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS;
+
+ flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
+ flow_act.pkt_reformat = vlan->pkt_reformat_pop;
+
+ if (vlan_proto == ETH_P_8021Q) {
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria,
+ outer_headers.cvlan_tag);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_value,
+ outer_headers.cvlan_tag);
+ } else if (vlan_proto == ETH_P_8021AD) {
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria,
+ outer_headers.svlan_tag);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_value,
+ outer_headers.svlan_tag);
+ }
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria, outer_headers.first_vid);
+ MLX5_SET(fte_match_param, rule_spec->match_value, outer_headers.first_vid, vlan->vid);
+
+ if (MLX5_CAP_ESW(bridge->br_offloads->esw->dev, merged_eswitch)) {
+ dest.vport.flags = MLX5_FLOW_DEST_VPORT_VHCA_ID;
+ dest.vport.vhca_id = port->esw_owner_vhca_id;
+ }
+ handle = mlx5_add_flow_rules(port->mcast.ft, rule_spec, &flow_act, &dest, 1);
+
+ kvfree(rule_spec);
+ return handle;
+}
+
+int mlx5_esw_bridge_vlan_mcast_init(u16 vlan_proto, struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan)
+{
+ struct mlx5_flow_handle *handle;
+
+ if (!(port->bridge->flags & MLX5_ESW_BRIDGE_MCAST_FLAG))
+ return 0;
+
+ handle = mlx5_esw_bridge_mcast_vlan_flow_create(vlan_proto, port, vlan);
+ if (IS_ERR(handle))
+ return PTR_ERR(handle);
+
+ vlan->mcast_handle = handle;
+ return 0;
+}
+
+void mlx5_esw_bridge_vlan_mcast_cleanup(struct mlx5_esw_bridge_vlan *vlan)
+{
+ if (vlan->mcast_handle)
+ mlx5_del_flow_rules(vlan->mcast_handle);
+ vlan->mcast_handle = NULL;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_mcast_fwd_flow_create(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,
+ .flags = FLOW_ACT_NO_APPEND,
+ };
+ struct mlx5_flow_destination dest = {
+ .type = MLX5_FLOW_DESTINATION_TYPE_VPORT,
+ .vport.num = port->vport_num,
+ };
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ if (MLX5_CAP_ESW_FLOWTABLE(bridge->br_offloads->esw->dev, flow_source) &&
+ port->vport_num == MLX5_VPORT_UPLINK)
+ rule_spec->flow_context.flow_source =
+ MLX5_FLOW_CONTEXT_FLOW_SOURCE_LOCAL_VPORT;
+
+ if (MLX5_CAP_ESW(bridge->br_offloads->esw->dev, merged_eswitch)) {
+ dest.vport.flags = MLX5_FLOW_DEST_VPORT_VHCA_ID;
+ dest.vport.vhca_id = port->esw_owner_vhca_id;
+ }
+ handle = mlx5_add_flow_rules(port->mcast.ft, rule_spec, &flow_act, &dest, 1);
+
+ kvfree(rule_spec);
+ return handle;
+}
+
+static int mlx5_esw_bridge_port_mcast_fhs_init(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_flow_handle *filter_handle, *fwd_handle;
+ struct mlx5_esw_bridge_vlan *vlan, *failed;
+ unsigned long index;
+ int err;
+
+
+ filter_handle = (port->flags & MLX5_ESW_BRIDGE_PORT_FLAG_PEER) ?
+ mlx5_esw_bridge_mcast_filter_flow_peer_create(port) :
+ mlx5_esw_bridge_mcast_filter_flow_create(port);
+ if (IS_ERR(filter_handle))
+ return PTR_ERR(filter_handle);
+
+ fwd_handle = mlx5_esw_bridge_mcast_fwd_flow_create(port);
+ if (IS_ERR(fwd_handle)) {
+ err = PTR_ERR(fwd_handle);
+ goto err_fwd;
+ }
+
+ xa_for_each(&port->vlans, index, vlan) {
+ err = mlx5_esw_bridge_vlan_mcast_init(port->bridge->vlan_proto, port, vlan);
+ if (err) {
+ failed = vlan;
+ goto err_vlan;
+ }
+ }
+
+ port->mcast.filter_handle = filter_handle;
+ port->mcast.fwd_handle = fwd_handle;
+
+ return 0;
+
+err_vlan:
+ xa_for_each(&port->vlans, index, vlan) {
+ if (vlan == failed)
+ break;
+
+ mlx5_esw_bridge_vlan_mcast_cleanup(vlan);
+ }
+ mlx5_del_flow_rules(fwd_handle);
+err_fwd:
+ mlx5_del_flow_rules(filter_handle);
+ return err;
+}
+
+static void mlx5_esw_bridge_port_mcast_fhs_cleanup(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_esw_bridge_vlan *vlan;
+ unsigned long index;
+
+ xa_for_each(&port->vlans, index, vlan)
+ mlx5_esw_bridge_vlan_mcast_cleanup(vlan);
+
+ if (port->mcast.fwd_handle)
+ mlx5_del_flow_rules(port->mcast.fwd_handle);
+ port->mcast.fwd_handle = NULL;
+ if (port->mcast.filter_handle)
+ mlx5_del_flow_rules(port->mcast.filter_handle);
+ port->mcast.filter_handle = NULL;
+}
+
+int mlx5_esw_bridge_port_mcast_init(struct mlx5_esw_bridge_port *port)
+{
+ struct mlx5_esw_bridge *bridge = port->bridge;
+ int err;
+
+ if (!(bridge->flags & MLX5_ESW_BRIDGE_MCAST_FLAG))
+ return 0;
+
+ err = mlx5_esw_bridge_port_mcast_fts_init(port, bridge);
+ if (err)
+ return err;
+
+ err = mlx5_esw_bridge_port_mcast_fgs_init(port);
+ if (err)
+ goto err_fgs;
+
+ err = mlx5_esw_bridge_port_mcast_fhs_init(port);
+ if (err)
+ goto err_fhs;
+ return err;
+
+err_fhs:
+ mlx5_esw_bridge_port_mcast_fgs_cleanup(port);
+err_fgs:
+ mlx5_esw_bridge_port_mcast_fts_cleanup(port);
+ return err;
+}
+
+void mlx5_esw_bridge_port_mcast_cleanup(struct mlx5_esw_bridge_port *port)
+{
+ mlx5_esw_bridge_port_mdb_flush(port);
+ mlx5_esw_bridge_port_mcast_fhs_cleanup(port);
+ mlx5_esw_bridge_port_mcast_fgs_cleanup(port);
+ mlx5_esw_bridge_port_mcast_fts_cleanup(port);
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_ingress_igmp_fg_create(struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *ingress_ft)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_group *fg;
+ u32 *in, *match;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return ERR_PTR(-ENOMEM);
+
+ MLX5_SET(create_flow_group_in, in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS);
+ match = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria);
+
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.ip_version);
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.ip_protocol);
+
+ MLX5_SET(create_flow_group_in, in, start_flow_index,
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_IDX_FROM);
+ MLX5_SET(create_flow_group_in, in, end_flow_index,
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_IDX_TO);
+
+ fg = mlx5_create_flow_group(ingress_ft, in);
+ kvfree(in);
+ if (IS_ERR(fg))
+ esw_warn(esw->dev,
+ "Failed to create IGMP flow group for bridge ingress table (err=%pe)\n",
+ fg);
+
+ return fg;
+}
+
+static struct mlx5_flow_group *
+mlx5_esw_bridge_ingress_mld_fg_create(struct mlx5_eswitch *esw,
+ struct mlx5_flow_table *ingress_ft)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_group *fg;
+ u32 *in, *match;
+
+ if (!(MLX5_CAP_GEN(esw->dev, flex_parser_protocols) & MLX5_FLEX_PROTO_ICMPV6)) {
+ esw_warn(esw->dev,
+ "Can't create MLD flow group due to missing hardware ICMPv6 parsing support\n");
+ return NULL;
+ }
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return ERR_PTR(-ENOMEM);
+
+ MLX5_SET(create_flow_group_in, in, match_criteria_enable,
+ MLX5_MATCH_OUTER_HEADERS | MLX5_MATCH_MISC_PARAMETERS_3);
+ match = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria);
+
+ MLX5_SET_TO_ONES(fte_match_param, match, outer_headers.ip_version);
+ MLX5_SET_TO_ONES(fte_match_param, match, misc_parameters_3.icmpv6_type);
+
+ MLX5_SET(create_flow_group_in, in, start_flow_index,
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_FROM);
+ MLX5_SET(create_flow_group_in, in, end_flow_index,
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_TO);
+
+ fg = mlx5_create_flow_group(ingress_ft, in);
+ kvfree(in);
+ if (IS_ERR(fg))
+ esw_warn(esw->dev,
+ "Failed to create MLD flow group for bridge ingress table (err=%pe)\n",
+ fg);
+
+ return fg;
+}
+
+static int
+mlx5_esw_bridge_ingress_mcast_fgs_init(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_flow_table *ingress_ft = br_offloads->ingress_ft;
+ struct mlx5_eswitch *esw = br_offloads->esw;
+ struct mlx5_flow_group *igmp_fg, *mld_fg;
+
+ igmp_fg = mlx5_esw_bridge_ingress_igmp_fg_create(esw, ingress_ft);
+ if (IS_ERR(igmp_fg))
+ return PTR_ERR(igmp_fg);
+
+ mld_fg = mlx5_esw_bridge_ingress_mld_fg_create(esw, ingress_ft);
+ if (IS_ERR(mld_fg)) {
+ mlx5_destroy_flow_group(igmp_fg);
+ return PTR_ERR(mld_fg);
+ }
+
+ br_offloads->ingress_igmp_fg = igmp_fg;
+ br_offloads->ingress_mld_fg = mld_fg;
+ return 0;
+}
+
+static void
+mlx5_esw_bridge_ingress_mcast_fgs_cleanup(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ if (br_offloads->ingress_mld_fg)
+ mlx5_destroy_flow_group(br_offloads->ingress_mld_fg);
+ br_offloads->ingress_mld_fg = NULL;
+ if (br_offloads->ingress_igmp_fg)
+ mlx5_destroy_flow_group(br_offloads->ingress_igmp_fg);
+ br_offloads->ingress_igmp_fg = NULL;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_ingress_igmp_fh_create(struct mlx5_flow_table *ingress_ft,
+ struct mlx5_flow_table *skip_ft)
+{
+ struct mlx5_flow_destination dest = {
+ .type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE,
+ .ft = skip_ft,
+ };
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,
+ .flags = FLOW_ACT_NO_APPEND,
+ };
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ rule_spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS;
+
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria, outer_headers.ip_version);
+ MLX5_SET(fte_match_param, rule_spec->match_value, outer_headers.ip_version, 4);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria, outer_headers.ip_protocol);
+ MLX5_SET(fte_match_param, rule_spec->match_value, outer_headers.ip_protocol, IPPROTO_IGMP);
+
+ handle = mlx5_add_flow_rules(ingress_ft, rule_spec, &flow_act, &dest, 1);
+
+ kvfree(rule_spec);
+ return handle;
+}
+
+static struct mlx5_flow_handle *
+mlx5_esw_bridge_ingress_mld_fh_create(u8 type, struct mlx5_flow_table *ingress_ft,
+ struct mlx5_flow_table *skip_ft)
+{
+ struct mlx5_flow_destination dest = {
+ .type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE,
+ .ft = skip_ft,
+ };
+ struct mlx5_flow_act flow_act = {
+ .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,
+ .flags = FLOW_ACT_NO_APPEND,
+ };
+ struct mlx5_flow_spec *rule_spec;
+ struct mlx5_flow_handle *handle;
+
+ rule_spec = kvzalloc(sizeof(*rule_spec), GFP_KERNEL);
+ if (!rule_spec)
+ return ERR_PTR(-ENOMEM);
+
+ rule_spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS | MLX5_MATCH_MISC_PARAMETERS_3;
+
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria, outer_headers.ip_version);
+ MLX5_SET(fte_match_param, rule_spec->match_value, outer_headers.ip_version, 6);
+ MLX5_SET_TO_ONES(fte_match_param, rule_spec->match_criteria, misc_parameters_3.icmpv6_type);
+ MLX5_SET(fte_match_param, rule_spec->match_value, misc_parameters_3.icmpv6_type, type);
+
+ handle = mlx5_add_flow_rules(ingress_ft, rule_spec, &flow_act, &dest, 1);
+
+ kvfree(rule_spec);
+ return handle;
+}
+
+static int
+mlx5_esw_bridge_ingress_mcast_fhs_create(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_flow_handle *igmp_handle, *mld_query_handle, *mld_report_handle,
+ *mld_done_handle;
+ struct mlx5_flow_table *ingress_ft = br_offloads->ingress_ft,
+ *skip_ft = br_offloads->skip_ft;
+ int err;
+
+ igmp_handle = mlx5_esw_bridge_ingress_igmp_fh_create(ingress_ft, skip_ft);
+ if (IS_ERR(igmp_handle))
+ return PTR_ERR(igmp_handle);
+
+ if (br_offloads->ingress_mld_fg) {
+ mld_query_handle = mlx5_esw_bridge_ingress_mld_fh_create(ICMPV6_MGM_QUERY,
+ ingress_ft,
+ skip_ft);
+ if (IS_ERR(mld_query_handle)) {
+ err = PTR_ERR(mld_query_handle);
+ goto err_mld_query;
+ }
+
+ mld_report_handle = mlx5_esw_bridge_ingress_mld_fh_create(ICMPV6_MGM_REPORT,
+ ingress_ft,
+ skip_ft);
+ if (IS_ERR(mld_report_handle)) {
+ err = PTR_ERR(mld_report_handle);
+ goto err_mld_report;
+ }
+
+ mld_done_handle = mlx5_esw_bridge_ingress_mld_fh_create(ICMPV6_MGM_REDUCTION,
+ ingress_ft,
+ skip_ft);
+ if (IS_ERR(mld_done_handle)) {
+ err = PTR_ERR(mld_done_handle);
+ goto err_mld_done;
+ }
+ } else {
+ mld_query_handle = NULL;
+ mld_report_handle = NULL;
+ mld_done_handle = NULL;
+ }
+
+ br_offloads->igmp_handle = igmp_handle;
+ br_offloads->mld_query_handle = mld_query_handle;
+ br_offloads->mld_report_handle = mld_report_handle;
+ br_offloads->mld_done_handle = mld_done_handle;
+
+ return 0;
+
+err_mld_done:
+ mlx5_del_flow_rules(mld_report_handle);
+err_mld_report:
+ mlx5_del_flow_rules(mld_query_handle);
+err_mld_query:
+ mlx5_del_flow_rules(igmp_handle);
+ return err;
+}
+
+static void
+mlx5_esw_bridge_ingress_mcast_fhs_cleanup(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ if (br_offloads->mld_done_handle)
+ mlx5_del_flow_rules(br_offloads->mld_done_handle);
+ br_offloads->mld_done_handle = NULL;
+ if (br_offloads->mld_report_handle)
+ mlx5_del_flow_rules(br_offloads->mld_report_handle);
+ br_offloads->mld_report_handle = NULL;
+ if (br_offloads->mld_query_handle)
+ mlx5_del_flow_rules(br_offloads->mld_query_handle);
+ br_offloads->mld_query_handle = NULL;
+ if (br_offloads->igmp_handle)
+ mlx5_del_flow_rules(br_offloads->igmp_handle);
+ br_offloads->igmp_handle = NULL;
+}
+
+static int mlx5_esw_brige_mcast_init(struct mlx5_esw_bridge *bridge)
+{
+ struct mlx5_esw_bridge_offloads *br_offloads = bridge->br_offloads;
+ struct mlx5_esw_bridge_port *port, *failed;
+ unsigned long i;
+ int err;
+
+ xa_for_each(&br_offloads->ports, i, port) {
+ if (port->bridge != bridge)
+ continue;
+
+ err = mlx5_esw_bridge_port_mcast_init(port);
+ if (err) {
+ failed = port;
+ goto err_port;
+ }
+ }
+ return 0;
+
+err_port:
+ xa_for_each(&br_offloads->ports, i, port) {
+ if (port == failed)
+ break;
+ if (port->bridge != bridge)
+ continue;
+
+ mlx5_esw_bridge_port_mcast_cleanup(port);
+ }
+ return err;
+}
+
+static void mlx5_esw_brige_mcast_cleanup(struct mlx5_esw_bridge *bridge)
+{
+ struct mlx5_esw_bridge_offloads *br_offloads = bridge->br_offloads;
+ struct mlx5_esw_bridge_port *port;
+ unsigned long i;
+
+ xa_for_each(&br_offloads->ports, i, port) {
+ if (port->bridge != bridge)
+ continue;
+
+ mlx5_esw_bridge_port_mcast_cleanup(port);
+ }
+}
+
+static int mlx5_esw_brige_mcast_global_enable(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ int err;
+
+ if (br_offloads->ingress_igmp_fg)
+ return 0; /* already enabled by another bridge */
+
+ err = mlx5_esw_bridge_ingress_mcast_fgs_init(br_offloads);
+ if (err) {
+ esw_warn(br_offloads->esw->dev,
+ "Failed to create global multicast flow groups (err=%d)\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_esw_bridge_ingress_mcast_fhs_create(br_offloads);
+ if (err) {
+ esw_warn(br_offloads->esw->dev,
+ "Failed to create global multicast flows (err=%d)\n",
+ err);
+ goto err_fhs;
+ }
+
+ return 0;
+
+err_fhs:
+ mlx5_esw_bridge_ingress_mcast_fgs_cleanup(br_offloads);
+ return err;
+}
+
+static void mlx5_esw_brige_mcast_global_disable(struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ struct mlx5_esw_bridge *br;
+
+ list_for_each_entry(br, &br_offloads->bridges, list) {
+ /* Ingress table is global, so only disable snooping when all
+ * bridges on esw have multicast disabled.
+ */
+ if (br->flags & MLX5_ESW_BRIDGE_MCAST_FLAG)
+ return;
+ }
+
+ mlx5_esw_bridge_ingress_mcast_fhs_cleanup(br_offloads);
+ mlx5_esw_bridge_ingress_mcast_fgs_cleanup(br_offloads);
+}
+
+int mlx5_esw_bridge_mcast_enable(struct mlx5_esw_bridge *bridge)
+{
+ int err;
+
+ err = mlx5_esw_brige_mcast_global_enable(bridge->br_offloads);
+ if (err)
+ return err;
+
+ bridge->flags |= MLX5_ESW_BRIDGE_MCAST_FLAG;
+
+ err = mlx5_esw_brige_mcast_init(bridge);
+ if (err) {
+ esw_warn(bridge->br_offloads->esw->dev, "Failed to enable multicast (err=%d)\n",
+ err);
+ bridge->flags &= ~MLX5_ESW_BRIDGE_MCAST_FLAG;
+ mlx5_esw_brige_mcast_global_disable(bridge->br_offloads);
+ }
+ return err;
+}
+
+void mlx5_esw_bridge_mcast_disable(struct mlx5_esw_bridge *bridge)
+{
+ mlx5_esw_brige_mcast_cleanup(bridge);
+ bridge->flags &= ~MLX5_ESW_BRIDGE_MCAST_FLAG;
+ mlx5_esw_brige_mcast_global_disable(bridge->br_offloads);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h
index 878311f..c959580 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h
@@ -12,11 +12,124 @@
#include <linux/xarray.h>
#include "fs_core.h"
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_SIZE 1
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_SIZE 3
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE 131072
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_UNTAGGED_GRP_SIZE \
+ (524288 - MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_SIZE - \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_SIZE)
+
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_IDX_FROM 0
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_IGMP_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_MLD_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_FILTER_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_QINQ_FILTER_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_INGRESS_TABLE_UNTAGGED_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_INGRESS_TABLE_SIZE \
+ (MLX5_ESW_BRIDGE_INGRESS_TABLE_MAC_GRP_IDX_TO + 1)
+static_assert(MLX5_ESW_BRIDGE_INGRESS_TABLE_SIZE == 1048576);
+
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE 131072
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_SIZE (262144 - 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_FROM 0
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_EGRESS_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_QINQ_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_MAC_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_TO \
+ MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_FROM
+#define MLX5_ESW_BRIDGE_EGRESS_TABLE_SIZE \
+ (MLX5_ESW_BRIDGE_EGRESS_TABLE_MISS_GRP_IDX_TO + 1)
+static_assert(MLX5_ESW_BRIDGE_EGRESS_TABLE_SIZE == 524288);
+
+#define MLX5_ESW_BRIDGE_SKIP_TABLE_SIZE 0
+
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_SIZE 1
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_SIZE 1
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_SIZE 4095
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_SIZE MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_SIZE
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_IDX_FROM 0
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_FILTER_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_VLAN_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_SIZE - 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_FROM \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_QINQ_GRP_IDX_TO + 1)
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_TO \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_FROM + \
+ MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_SIZE - 1)
+
+#define MLX5_ESW_BRIDGE_MCAST_TABLE_SIZE \
+ (MLX5_ESW_BRIDGE_MCAST_TABLE_FWD_GRP_IDX_TO + 1)
+static_assert(MLX5_ESW_BRIDGE_MCAST_TABLE_SIZE == 8192);
+
+enum {
+ MLX5_ESW_BRIDGE_LEVEL_INGRESS_TABLE,
+ MLX5_ESW_BRIDGE_LEVEL_EGRESS_TABLE,
+ MLX5_ESW_BRIDGE_LEVEL_MCAST_TABLE,
+ MLX5_ESW_BRIDGE_LEVEL_SKIP_TABLE,
+};
+
+enum {
+ MLX5_ESW_BRIDGE_VLAN_FILTERING_FLAG = BIT(0),
+ MLX5_ESW_BRIDGE_MCAST_FLAG = BIT(1),
+};
+
struct mlx5_esw_bridge_fdb_key {
unsigned char addr[ETH_ALEN];
u16 vid;
};
+struct mlx5_esw_bridge_mdb_key {
+ unsigned char addr[ETH_ALEN];
+ u16 vid;
+};
+
enum {
MLX5_ESW_BRIDGE_FLAG_ADDED_BY_USER = BIT(0),
MLX5_ESW_BRIDGE_FLAG_PEER = BIT(1),
@@ -43,6 +156,16 @@ struct mlx5_esw_bridge_fdb_entry {
struct mlx5_flow_handle *filter_handle;
};
+struct mlx5_esw_bridge_mdb_entry {
+ struct mlx5_esw_bridge_mdb_key key;
+ struct rhash_head ht_node;
+ struct list_head list;
+ struct xarray ports;
+ int num_ports;
+
+ struct mlx5_flow_handle *egress_handle;
+};
+
struct mlx5_esw_bridge_vlan {
u16 vid;
u16 flags;
@@ -50,6 +173,7 @@ struct mlx5_esw_bridge_vlan {
struct mlx5_pkt_reformat *pkt_reformat_push;
struct mlx5_pkt_reformat *pkt_reformat_pop;
struct mlx5_modify_hdr *pkt_mod_hdr_push_mark;
+ struct mlx5_flow_handle *mcast_handle;
};
struct mlx5_esw_bridge_port {
@@ -58,6 +182,63 @@ struct mlx5_esw_bridge_port {
u16 flags;
struct mlx5_esw_bridge *bridge;
struct xarray vlans;
+ struct {
+ struct mlx5_flow_table *ft;
+ struct mlx5_flow_group *filter_fg;
+ struct mlx5_flow_group *vlan_fg;
+ struct mlx5_flow_group *qinq_fg;
+ struct mlx5_flow_group *fwd_fg;
+
+ struct mlx5_flow_handle *filter_handle;
+ struct mlx5_flow_handle *fwd_handle;
+ } mcast;
};
+struct mlx5_esw_bridge {
+ int ifindex;
+ int refcnt;
+ struct list_head list;
+ struct mlx5_esw_bridge_offloads *br_offloads;
+
+ struct list_head fdb_list;
+ struct rhashtable fdb_ht;
+
+ struct list_head mdb_list;
+ struct rhashtable mdb_ht;
+
+ struct mlx5_flow_table *egress_ft;
+ struct mlx5_flow_group *egress_vlan_fg;
+ struct mlx5_flow_group *egress_qinq_fg;
+ struct mlx5_flow_group *egress_mac_fg;
+ struct mlx5_flow_group *egress_miss_fg;
+ struct mlx5_pkt_reformat *egress_miss_pkt_reformat;
+ struct mlx5_flow_handle *egress_miss_handle;
+ unsigned long ageing_time;
+ u32 flags;
+ u16 vlan_proto;
+};
+
+struct mlx5_flow_table *mlx5_esw_bridge_table_create(int max_fte, u32 level,
+ struct mlx5_eswitch *esw);
+unsigned long mlx5_esw_bridge_port_key(struct mlx5_esw_bridge_port *port);
+
+int mlx5_esw_bridge_port_mcast_init(struct mlx5_esw_bridge_port *port);
+void mlx5_esw_bridge_port_mcast_cleanup(struct mlx5_esw_bridge_port *port);
+int mlx5_esw_bridge_vlan_mcast_init(u16 vlan_proto, struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan);
+void mlx5_esw_bridge_vlan_mcast_cleanup(struct mlx5_esw_bridge_vlan *vlan);
+
+int mlx5_esw_bridge_mcast_enable(struct mlx5_esw_bridge *bridge);
+void mlx5_esw_bridge_mcast_disable(struct mlx5_esw_bridge *bridge);
+
+int mlx5_esw_bridge_mdb_init(struct mlx5_esw_bridge *bridge);
+void mlx5_esw_bridge_mdb_cleanup(struct mlx5_esw_bridge *bridge);
+int mlx5_esw_bridge_port_mdb_attach(struct net_device *dev, struct mlx5_esw_bridge_port *port,
+ const unsigned char *addr, u16 vid);
+void mlx5_esw_bridge_port_mdb_detach(struct net_device *dev, struct mlx5_esw_bridge_port *port,
+ const unsigned char *addr, u16 vid);
+void mlx5_esw_bridge_port_mdb_vlan_flush(struct mlx5_esw_bridge_port *port,
+ struct mlx5_esw_bridge_vlan *vlan);
+void mlx5_esw_bridge_mdb_flush(struct mlx5_esw_bridge *bridge);
+
#endif /* _MLX5_ESW_BRIDGE_PRIVATE_ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/diag/bridge_tracepoint.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/diag/bridge_tracepoint.h
index 51ac24e..1808da2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/diag/bridge_tracepoint.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/diag/bridge_tracepoint.h
@@ -110,6 +110,41 @@ DEFINE_EVENT(mlx5_esw_bridge_port_template,
TP_ARGS(port)
);
+DECLARE_EVENT_CLASS(mlx5_esw_bridge_mdb_port_change_template,
+ TP_PROTO(const struct net_device *dev,
+ const struct mlx5_esw_bridge_mdb_entry *mdb),
+ TP_ARGS(dev, mdb),
+ TP_STRUCT__entry(
+ __array(char, dev_name, IFNAMSIZ)
+ __array(unsigned char, addr, ETH_ALEN)
+ __field(u16, vid)
+ __field(int, num_ports)
+ __field(bool, offloaded)),
+ TP_fast_assign(
+ strscpy(__entry->dev_name, netdev_name(dev), IFNAMSIZ);
+ memcpy(__entry->addr, mdb->key.addr, ETH_ALEN);
+ __entry->vid = mdb->key.vid;
+ __entry->num_ports = mdb->num_ports;
+ __entry->offloaded = mdb->egress_handle;),
+ TP_printk("net_device=%s addr=%pM vid=%u num_ports=%d offloaded=%d",
+ __entry->dev_name,
+ __entry->addr,
+ __entry->vid,
+ __entry->num_ports,
+ __entry->offloaded));
+
+DEFINE_EVENT(mlx5_esw_bridge_mdb_port_change_template,
+ mlx5_esw_bridge_port_mdb_attach,
+ TP_PROTO(const struct net_device *dev,
+ const struct mlx5_esw_bridge_mdb_entry *mdb),
+ TP_ARGS(dev, mdb));
+
+DEFINE_EVENT(mlx5_esw_bridge_mdb_port_change_template,
+ mlx5_esw_bridge_port_mdb_detach,
+ TP_PROTO(const struct net_device *dev,
+ const struct mlx5_esw_bridge_mdb_entry *mdb),
+ TP_ARGS(dev, mdb));
+
#endif
/* This part must be outside protection */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 8bdf287..19fed51 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1488,7 +1488,7 @@ int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *
void *hca_caps;
int err;
- if (!mlx5_core_is_ecpf(dev) || mlx5_core_is_management_pf(dev)) {
+ if (!mlx5_core_is_ecpf(dev)) {
*max_sfs = 0;
return 0;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 19e9a77..e9d68fd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -263,6 +263,7 @@ struct mlx5_esw_offload {
const struct mlx5_eswitch_rep_ops *rep_ops[NUM_REP_TYPES];
u8 inline_mode;
atomic64_t num_flows;
+ u64 num_block_encap;
enum devlink_eswitch_encap_mode encap;
struct ida vport_metadata_ida;
unsigned int host_number; /* ECPF supports one external host */
@@ -748,6 +749,9 @@ void mlx5_eswitch_offloads_destroy_single_fdb(struct mlx5_eswitch *master_esw,
struct mlx5_eswitch *slave_esw);
int mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw);
+bool mlx5_eswitch_block_encap(struct mlx5_core_dev *dev);
+void mlx5_eswitch_unblock_encap(struct mlx5_core_dev *dev);
+
static inline int mlx5_eswitch_num_vfs(struct mlx5_eswitch *esw)
{
if (mlx5_esw_allowed(esw))
@@ -761,6 +765,7 @@ mlx5_eswitch_get_slow_fdb(struct mlx5_eswitch *esw)
{
return esw->fdb_table.offloads.slow_fdb;
}
+
#else /* CONFIG_MLX5_ESWITCH */
/* eswitch API stubs */
static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
@@ -805,6 +810,15 @@ mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw)
{
return 0;
}
+
+static inline bool mlx5_eswitch_block_encap(struct mlx5_core_dev *dev)
+{
+ return true;
+}
+
+static inline void mlx5_eswitch_unblock_encap(struct mlx5_core_dev *dev)
+{
+}
#endif /* CONFIG_MLX5_ESWITCH */
#endif /* __MLX5_ESWITCH_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 48036df..b6e2709 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3586,6 +3586,47 @@ int mlx5_devlink_eswitch_inline_mode_get(struct devlink *devlink, u8 *mode)
return err;
}
+bool mlx5_eswitch_block_encap(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ struct mlx5_eswitch *esw;
+
+ devl_lock(devlink);
+ esw = mlx5_devlink_eswitch_get(devlink);
+ if (IS_ERR(esw)) {
+ devl_unlock(devlink);
+ /* Failure means no eswitch => not possible to change encap */
+ return true;
+ }
+
+ down_write(&esw->mode_lock);
+ if (esw->mode != MLX5_ESWITCH_LEGACY &&
+ esw->offloads.encap != DEVLINK_ESWITCH_ENCAP_MODE_NONE) {
+ up_write(&esw->mode_lock);
+ devl_unlock(devlink);
+ return false;
+ }
+
+ esw->offloads.num_block_encap++;
+ up_write(&esw->mode_lock);
+ devl_unlock(devlink);
+ return true;
+}
+
+void mlx5_eswitch_unblock_encap(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ struct mlx5_eswitch *esw;
+
+ esw = mlx5_devlink_eswitch_get(devlink);
+ if (IS_ERR(esw))
+ return;
+
+ down_write(&esw->mode_lock);
+ esw->offloads.num_block_encap--;
+ up_write(&esw->mode_lock);
+}
+
int mlx5_devlink_eswitch_encap_mode_set(struct devlink *devlink,
enum devlink_eswitch_encap_mode encap,
struct netlink_ext_ack *extack)
@@ -3627,6 +3668,13 @@ int mlx5_devlink_eswitch_encap_mode_set(struct devlink *devlink,
goto unlock;
}
+ if (esw->offloads.num_block_encap) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Can't set encapsulation when IPsec SA and/or policies are configured");
+ err = -EOPNOTSUPP;
+ goto unlock;
+ }
+
esw_destroy_offloads_fdb_tables(esw);
esw->offloads.encap = encap;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
index 8e3da9d..19da02c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
@@ -2997,7 +2997,7 @@ static int init_fdb_root_ns(struct mlx5_flow_steering *steering)
goto out_err;
}
- maj_prio = fs_create_prio(&steering->fdb_root_ns->ns, FDB_BR_OFFLOAD, 3);
+ maj_prio = fs_create_prio(&steering->fdb_root_ns->ns, FDB_BR_OFFLOAD, 4);
if (IS_ERR(maj_prio)) {
err = PTR_ERR(maj_prio);
goto out_err;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index f95df73..a95d121 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -100,15 +100,19 @@ enum {
static struct mlx5_profile profile[] = {
[0] = {
.mask = 0,
+ .num_cmd_caches = MLX5_NUM_COMMAND_CACHES,
},
[1] = {
.mask = MLX5_PROF_MASK_QP_SIZE,
.log_max_qp = 12,
+ .num_cmd_caches = MLX5_NUM_COMMAND_CACHES,
+
},
[2] = {
.mask = MLX5_PROF_MASK_QP_SIZE |
MLX5_PROF_MASK_MR_CACHE,
.log_max_qp = LOG_MAX_SUPPORTED_QPS,
+ .num_cmd_caches = MLX5_NUM_COMMAND_CACHES,
.mr_cache[0] = {
.size = 500,
.limit = 250
@@ -174,6 +178,11 @@ static struct mlx5_profile profile[] = {
.limit = 4
},
},
+ [3] = {
+ .mask = MLX5_PROF_MASK_QP_SIZE,
+ .log_max_qp = LOG_MAX_SUPPORTED_QPS,
+ .num_cmd_caches = 0,
+ },
};
static int wait_fw_init(struct mlx5_core_dev *dev, u32 max_wait_mili,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index be0785f..5eaab99 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -142,6 +142,7 @@ enum mlx5_semaphore_space_address {
};
#define MLX5_DEFAULT_PROF 2
+#define MLX5_SF_PROF 3
static inline int mlx5_flexible_inlen(struct mlx5_core_dev *dev, size_t fixed,
size_t item_size, size_t num_items,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c
index a737761..e2f26d0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c
@@ -28,7 +28,7 @@ static int mlx5_sf_dev_probe(struct auxiliary_device *adev, const struct auxilia
mdev->priv.adev_idx = adev->id;
sf_dev->mdev = mdev;
- err = mlx5_mdev_init(mdev, MLX5_DEFAULT_PROF);
+ err = mlx5_mdev_init(mdev, MLX5_SF_PROF);
if (err) {
mlx5_core_warn(mdev, "mlx5_mdev_init on err=%d\n", err);
goto mdev_err;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
index ee104cf..0eb9a8d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
@@ -819,14 +819,34 @@ int mlx5dr_actions_build_ste_arr(struct mlx5dr_matcher *matcher,
case DR_ACTION_TYP_TNL_L2_TO_L2:
break;
case DR_ACTION_TYP_TNL_L3_TO_L2:
- attr.decap_index = action->rewrite->index;
- attr.decap_actions = action->rewrite->num_of_actions;
- attr.decap_with_vlan =
- attr.decap_actions == WITH_VLAN_NUM_HW_ACTIONS;
+ if (action->rewrite->ptrn && action->rewrite->arg) {
+ attr.decap_index = mlx5dr_arg_get_obj_id(action->rewrite->arg);
+ attr.decap_actions = action->rewrite->ptrn->num_of_actions;
+ attr.decap_pat_idx = action->rewrite->ptrn->index;
+ } else {
+ attr.decap_index = action->rewrite->index;
+ attr.decap_actions = action->rewrite->num_of_actions;
+ attr.decap_with_vlan =
+ attr.decap_actions == WITH_VLAN_NUM_HW_ACTIONS;
+ attr.decap_pat_idx = MLX5DR_INVALID_PATTERN_INDEX;
+ }
break;
case DR_ACTION_TYP_MODIFY_HDR:
- attr.modify_index = action->rewrite->index;
- attr.modify_actions = action->rewrite->num_of_actions;
+ if (action->rewrite->single_action_opt) {
+ attr.modify_actions = action->rewrite->num_of_actions;
+ attr.single_modify_action = action->rewrite->data;
+ } else {
+ if (action->rewrite->ptrn && action->rewrite->arg) {
+ attr.modify_index =
+ mlx5dr_arg_get_obj_id(action->rewrite->arg);
+ attr.modify_actions = action->rewrite->ptrn->num_of_actions;
+ attr.modify_pat_idx = action->rewrite->ptrn->index;
+ } else {
+ attr.modify_index = action->rewrite->index;
+ attr.modify_actions = action->rewrite->num_of_actions;
+ attr.modify_pat_idx = MLX5DR_INVALID_PATTERN_INDEX;
+ }
+ }
if (action->rewrite->modify_ttl)
dr_action_modify_ttl_adjust(dmn, &attr, rx_rule,
&recalc_cs_required);
@@ -1365,8 +1385,6 @@ dr_action_verify_reformat_params(enum mlx5dr_action_type reformat_type,
return -EINVAL;
}
-#define ACTION_CACHE_LINE_SIZE 64
-
static int
dr_action_create_reformat_action(struct mlx5dr_domain *dmn,
u8 reformat_param_0, u8 reformat_param_1,
@@ -1403,36 +1421,25 @@ dr_action_create_reformat_action(struct mlx5dr_domain *dmn,
}
case DR_ACTION_TYP_TNL_L3_TO_L2:
{
- u8 hw_actions[ACTION_CACHE_LINE_SIZE] = {};
+ u8 hw_actions[DR_ACTION_CACHE_LINE_SIZE] = {};
int ret;
ret = mlx5dr_ste_set_action_decap_l3_list(dmn->ste_ctx,
data, data_sz,
hw_actions,
- ACTION_CACHE_LINE_SIZE,
+ DR_ACTION_CACHE_LINE_SIZE,
&action->rewrite->num_of_actions);
if (ret) {
mlx5dr_dbg(dmn, "Failed creating decap l3 action list\n");
return ret;
}
- action->rewrite->chunk = mlx5dr_icm_alloc_chunk(dmn->action_icm_pool,
- DR_CHUNK_SIZE_8);
- if (!action->rewrite->chunk) {
- mlx5dr_dbg(dmn, "Failed allocating modify header chunk\n");
- return -ENOMEM;
- }
+ action->rewrite->data = hw_actions;
+ action->rewrite->dmn = dmn;
- action->rewrite->data = (void *)hw_actions;
- action->rewrite->index = (mlx5dr_icm_pool_get_chunk_icm_addr
- (action->rewrite->chunk) -
- dmn->info.caps.hdr_modify_icm_addr) /
- ACTION_CACHE_LINE_SIZE;
-
- ret = mlx5dr_send_postsend_action(dmn, action);
+ ret = mlx5dr_ste_alloc_modify_hdr(action);
if (ret) {
- mlx5dr_dbg(dmn, "Writing decap l3 actions to ICM failed\n");
- mlx5dr_icm_free_chunk(action->rewrite->chunk);
+ mlx5dr_dbg(dmn, "Failed preparing reformat data\n");
return ret;
}
return 0;
@@ -1963,7 +1970,6 @@ static int dr_action_create_modify_action(struct mlx5dr_domain *dmn,
__be64 actions[],
struct mlx5dr_action *action)
{
- struct mlx5dr_icm_chunk *chunk;
u32 max_hw_actions;
u32 num_hw_actions;
u32 num_sw_actions;
@@ -1980,15 +1986,9 @@ static int dr_action_create_modify_action(struct mlx5dr_domain *dmn,
return -EINVAL;
}
- chunk = mlx5dr_icm_alloc_chunk(dmn->action_icm_pool, DR_CHUNK_SIZE_16);
- if (!chunk)
- return -ENOMEM;
-
hw_actions = kcalloc(1, max_hw_actions * DR_MODIFY_ACTION_SIZE, GFP_KERNEL);
- if (!hw_actions) {
- ret = -ENOMEM;
- goto free_chunk;
- }
+ if (!hw_actions)
+ return -ENOMEM;
ret = dr_actions_convert_modify_header(action,
max_hw_actions,
@@ -2000,24 +2000,24 @@ static int dr_action_create_modify_action(struct mlx5dr_domain *dmn,
if (ret)
goto free_hw_actions;
- action->rewrite->chunk = chunk;
action->rewrite->modify_ttl = modify_ttl;
action->rewrite->data = (u8 *)hw_actions;
action->rewrite->num_of_actions = num_hw_actions;
- action->rewrite->index = (mlx5dr_icm_pool_get_chunk_icm_addr(chunk) -
- dmn->info.caps.hdr_modify_icm_addr) /
- ACTION_CACHE_LINE_SIZE;
- ret = mlx5dr_send_postsend_action(dmn, action);
- if (ret)
- goto free_hw_actions;
+ if (num_hw_actions == 1 &&
+ dmn->info.caps.sw_format_ver >= MLX5_STEERING_FORMAT_CONNECTX_6DX) {
+ action->rewrite->single_action_opt = true;
+ } else {
+ action->rewrite->single_action_opt = false;
+ ret = mlx5dr_ste_alloc_modify_hdr(action);
+ if (ret)
+ goto free_hw_actions;
+ }
return 0;
free_hw_actions:
kfree(hw_actions);
-free_chunk:
- mlx5dr_icm_free_chunk(chunk);
return ret;
}
@@ -2162,7 +2162,8 @@ int mlx5dr_action_destroy(struct mlx5dr_action *action)
refcount_dec(&action->reformat->dmn->refcount);
break;
case DR_ACTION_TYP_TNL_L3_TO_L2:
- mlx5dr_icm_free_chunk(action->rewrite->chunk);
+ mlx5dr_ste_free_modify_hdr(action);
+ kfree(action->rewrite->data);
refcount_dec(&action->rewrite->dmn->refcount);
break;
case DR_ACTION_TYP_L2_TO_TNL_L2:
@@ -2173,7 +2174,8 @@ int mlx5dr_action_destroy(struct mlx5dr_action *action)
refcount_dec(&action->reformat->dmn->refcount);
break;
case DR_ACTION_TYP_MODIFY_HDR:
- mlx5dr_icm_free_chunk(action->rewrite->chunk);
+ if (!action->rewrite->single_action_opt)
+ mlx5dr_ste_free_modify_hdr(action);
kfree(action->rewrite->data);
refcount_dec(&action->rewrite->dmn->refcount);
break;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_arg.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_arg.c
new file mode 100644
index 0000000..01ed644
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_arg.c
@@ -0,0 +1,273 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "dr_types.h"
+
+#define DR_ICM_MODIFY_HDR_GRANULARITY_4K 12
+
+/* modify-header arg pool */
+enum dr_arg_chunk_size {
+ DR_ARG_CHUNK_SIZE_1,
+ DR_ARG_CHUNK_SIZE_MIN = DR_ARG_CHUNK_SIZE_1, /* keep updated when changing */
+ DR_ARG_CHUNK_SIZE_2,
+ DR_ARG_CHUNK_SIZE_3,
+ DR_ARG_CHUNK_SIZE_4,
+ DR_ARG_CHUNK_SIZE_MAX,
+};
+
+/* argument pool area */
+struct dr_arg_pool {
+ enum dr_arg_chunk_size log_chunk_size;
+ struct mlx5dr_domain *dmn;
+ struct list_head free_list;
+ struct mutex mutex; /* protect arg pool */
+};
+
+struct mlx5dr_arg_mgr {
+ struct mlx5dr_domain *dmn;
+ struct dr_arg_pool *pools[DR_ARG_CHUNK_SIZE_MAX];
+};
+
+static int dr_arg_pool_alloc_objs(struct dr_arg_pool *pool)
+{
+ struct mlx5dr_arg_obj *arg_obj, *tmp_arg;
+ struct list_head cur_list;
+ u16 object_range;
+ int num_of_objects;
+ u32 obj_id = 0;
+ int i, ret;
+
+ INIT_LIST_HEAD(&cur_list);
+
+ object_range =
+ pool->dmn->info.caps.log_header_modify_argument_granularity;
+
+ object_range =
+ max_t(u32, pool->dmn->info.caps.log_header_modify_argument_granularity,
+ DR_ICM_MODIFY_HDR_GRANULARITY_4K);
+ object_range =
+ min_t(u32, pool->dmn->info.caps.log_header_modify_argument_max_alloc,
+ object_range);
+
+ if (pool->log_chunk_size > object_range) {
+ mlx5dr_err(pool->dmn, "Required chunk size (%d) is not supported\n",
+ pool->log_chunk_size);
+ return -ENOMEM;
+ }
+
+ num_of_objects = (1 << (object_range - pool->log_chunk_size));
+ /* Only one devx object per range */
+ ret = mlx5dr_cmd_create_modify_header_arg(pool->dmn->mdev,
+ object_range,
+ pool->dmn->pdn,
+ &obj_id);
+ if (ret) {
+ mlx5dr_err(pool->dmn, "failed allocating object with range: %d:\n",
+ object_range);
+ return -EAGAIN;
+ }
+
+ for (i = 0; i < num_of_objects; i++) {
+ arg_obj = kzalloc(sizeof(*arg_obj), GFP_KERNEL);
+ if (!arg_obj) {
+ ret = -ENOMEM;
+ goto clean_arg_obj;
+ }
+
+ arg_obj->log_chunk_size = pool->log_chunk_size;
+
+ list_add_tail(&arg_obj->list_node, &cur_list);
+
+ arg_obj->obj_id = obj_id;
+ arg_obj->obj_offset = i * (1 << pool->log_chunk_size);
+ }
+ list_splice_tail_init(&cur_list, &pool->free_list);
+
+ return 0;
+
+clean_arg_obj:
+ mlx5dr_cmd_destroy_modify_header_arg(pool->dmn->mdev, obj_id);
+ list_for_each_entry_safe(arg_obj, tmp_arg, &cur_list, list_node) {
+ list_del(&arg_obj->list_node);
+ kfree(arg_obj);
+ }
+ return ret;
+}
+
+static struct mlx5dr_arg_obj *dr_arg_pool_get_arg_obj(struct dr_arg_pool *pool)
+{
+ struct mlx5dr_arg_obj *arg_obj = NULL;
+ int ret;
+
+ mutex_lock(&pool->mutex);
+ if (list_empty(&pool->free_list)) {
+ ret = dr_arg_pool_alloc_objs(pool);
+ if (ret)
+ goto out;
+ }
+
+ arg_obj = list_first_entry_or_null(&pool->free_list,
+ struct mlx5dr_arg_obj,
+ list_node);
+ WARN(!arg_obj, "couldn't get dr arg obj from pool");
+
+ if (arg_obj)
+ list_del_init(&arg_obj->list_node);
+
+out:
+ mutex_unlock(&pool->mutex);
+ return arg_obj;
+}
+
+static void dr_arg_pool_put_arg_obj(struct dr_arg_pool *pool,
+ struct mlx5dr_arg_obj *arg_obj)
+{
+ mutex_lock(&pool->mutex);
+ list_add(&arg_obj->list_node, &pool->free_list);
+ mutex_unlock(&pool->mutex);
+}
+
+static struct dr_arg_pool *dr_arg_pool_create(struct mlx5dr_domain *dmn,
+ enum dr_arg_chunk_size chunk_size)
+{
+ struct dr_arg_pool *pool;
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->dmn = dmn;
+
+ INIT_LIST_HEAD(&pool->free_list);
+ mutex_init(&pool->mutex);
+
+ pool->log_chunk_size = chunk_size;
+ if (dr_arg_pool_alloc_objs(pool))
+ goto free_pool;
+
+ return pool;
+
+free_pool:
+ kfree(pool);
+
+ return NULL;
+}
+
+static void dr_arg_pool_destroy(struct dr_arg_pool *pool)
+{
+ struct mlx5dr_arg_obj *arg_obj, *tmp_arg;
+
+ list_for_each_entry_safe(arg_obj, tmp_arg, &pool->free_list, list_node) {
+ list_del(&arg_obj->list_node);
+ if (!arg_obj->obj_offset) /* the first in range */
+ mlx5dr_cmd_destroy_modify_header_arg(pool->dmn->mdev, arg_obj->obj_id);
+ kfree(arg_obj);
+ }
+
+ mutex_destroy(&pool->mutex);
+ kfree(pool);
+}
+
+static enum dr_arg_chunk_size dr_arg_get_chunk_size(u16 num_of_actions)
+{
+ if (num_of_actions <= 8)
+ return DR_ARG_CHUNK_SIZE_1;
+ if (num_of_actions <= 16)
+ return DR_ARG_CHUNK_SIZE_2;
+ if (num_of_actions <= 32)
+ return DR_ARG_CHUNK_SIZE_3;
+ if (num_of_actions <= 64)
+ return DR_ARG_CHUNK_SIZE_4;
+
+ return DR_ARG_CHUNK_SIZE_MAX;
+}
+
+u32 mlx5dr_arg_get_obj_id(struct mlx5dr_arg_obj *arg_obj)
+{
+ return (arg_obj->obj_id + arg_obj->obj_offset);
+}
+
+struct mlx5dr_arg_obj *mlx5dr_arg_get_obj(struct mlx5dr_arg_mgr *mgr,
+ u16 num_of_actions,
+ u8 *data)
+{
+ u32 size = dr_arg_get_chunk_size(num_of_actions);
+ struct mlx5dr_arg_obj *arg_obj;
+ int ret;
+
+ if (size >= DR_ARG_CHUNK_SIZE_MAX)
+ return NULL;
+
+ arg_obj = dr_arg_pool_get_arg_obj(mgr->pools[size]);
+ if (!arg_obj) {
+ mlx5dr_err(mgr->dmn, "Failed allocating args object for modify header\n");
+ return NULL;
+ }
+
+ /* write it into the hw */
+ ret = mlx5dr_send_postsend_args(mgr->dmn,
+ mlx5dr_arg_get_obj_id(arg_obj),
+ num_of_actions, data);
+ if (ret) {
+ mlx5dr_err(mgr->dmn, "Failed writing args object\n");
+ goto put_obj;
+ }
+
+ return arg_obj;
+
+put_obj:
+ mlx5dr_arg_put_obj(mgr, arg_obj);
+ return NULL;
+}
+
+void mlx5dr_arg_put_obj(struct mlx5dr_arg_mgr *mgr,
+ struct mlx5dr_arg_obj *arg_obj)
+{
+ dr_arg_pool_put_arg_obj(mgr->pools[arg_obj->log_chunk_size], arg_obj);
+}
+
+struct mlx5dr_arg_mgr*
+mlx5dr_arg_mgr_create(struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_arg_mgr *pool_mgr;
+ int i;
+
+ if (!mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return NULL;
+
+ pool_mgr = kzalloc(sizeof(*pool_mgr), GFP_KERNEL);
+ if (!pool_mgr)
+ return NULL;
+
+ pool_mgr->dmn = dmn;
+
+ for (i = 0; i < DR_ARG_CHUNK_SIZE_MAX; i++) {
+ pool_mgr->pools[i] = dr_arg_pool_create(dmn, i);
+ if (!pool_mgr->pools[i])
+ goto clean_pools;
+ }
+
+ return pool_mgr;
+
+clean_pools:
+ for (i--; i >= 0; i--)
+ dr_arg_pool_destroy(pool_mgr->pools[i]);
+
+ kfree(pool_mgr);
+ return NULL;
+}
+
+void mlx5dr_arg_mgr_destroy(struct mlx5dr_arg_mgr *mgr)
+{
+ struct dr_arg_pool **pools;
+ int i;
+
+ if (!mgr)
+ return;
+
+ pools = mgr->pools;
+ for (i = 0; i < DR_ARG_CHUNK_SIZE_MAX; i++)
+ dr_arg_pool_destroy(pools[i]);
+
+ kfree(mgr);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
index 07b6a6d..3835ba3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
@@ -132,6 +132,17 @@ int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
caps->isolate_vl_tc = MLX5_CAP_GEN(mdev, isolate_vl_tc_new);
+ caps->support_modify_argument =
+ MLX5_CAP_GEN_64(mdev, general_obj_types) &
+ MLX5_GENERAL_OBJ_TYPES_CAP_HEADER_MODIFY_ARGUMENT;
+
+ if (caps->support_modify_argument) {
+ caps->log_header_modify_argument_granularity =
+ MLX5_CAP_GEN(mdev, log_header_modify_argument_granularity);
+ caps->log_header_modify_argument_max_alloc =
+ MLX5_CAP_GEN(mdev, log_header_modify_argument_max_alloc);
+ }
+
/* geneve_tlv_option_0_exist is the indication of
* STE support for lookup type flex_parser_ok
*/
@@ -200,6 +211,12 @@ int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
caps->hdr_modify_icm_addr =
MLX5_CAP64_DEV_MEM(mdev, header_modify_sw_icm_start_address);
+ caps->log_modify_pattern_icm_size =
+ MLX5_CAP_DEV_MEM(mdev, log_header_modify_pattern_sw_icm_size);
+
+ caps->hdr_modify_pattern_icm_addr =
+ MLX5_CAP64_DEV_MEM(mdev, header_modify_pattern_sw_icm_start_address);
+
caps->roce_min_src_udp = MLX5_CAP_ROCE(mdev, r_roce_min_src_udp_port);
caps->is_ecpf = mlx5_core_is_ecpf_esw_manager(mdev);
@@ -676,6 +693,49 @@ int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num,
return 0;
}
+int mlx5dr_cmd_create_modify_header_arg(struct mlx5_core_dev *dev,
+ u16 log_obj_range, u32 pd,
+ u32 *obj_id)
+{
+ u32 in[MLX5_ST_SZ_DW(create_modify_header_arg_in)] = {};
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
+ void *attr;
+ int ret;
+
+ attr = MLX5_ADDR_OF(create_modify_header_arg_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, attr, opcode,
+ MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, attr, obj_type,
+ MLX5_OBJ_TYPE_HEADER_MODIFY_ARGUMENT);
+ MLX5_SET(general_obj_in_cmd_hdr, attr,
+ op_param.create.log_obj_range, log_obj_range);
+
+ attr = MLX5_ADDR_OF(create_modify_header_arg_in, in, arg);
+ MLX5_SET(modify_header_arg, attr, access_pd, pd);
+
+ ret = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+ if (ret)
+ return ret;
+
+ *obj_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return 0;
+}
+
+void mlx5dr_cmd_destroy_modify_header_arg(struct mlx5_core_dev *dev,
+ u32 obj_id)
+{
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
+ u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
+
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+ MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+ MLX5_OBJ_TYPE_HEADER_MODIFY_ARGUMENT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, obj_id);
+
+ mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+}
+
static int mlx5dr_cmd_set_extended_dest(struct mlx5_core_dev *dev,
struct mlx5dr_cmd_fte_info *fte,
bool *extended_dest)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c
index db81d88..1ff8bde 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_dbg.c
@@ -140,10 +140,31 @@ dr_dump_rule_action_mem(struct seq_file *file, const u64 rule_id,
action->flow_tag->flow_tag);
break;
case DR_ACTION_TYP_MODIFY_HDR:
- seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
+ {
+ struct mlx5dr_ptrn_obj *ptrn = action->rewrite->ptrn;
+ struct mlx5dr_arg_obj *arg = action->rewrite->arg;
+ u8 *rewrite_data = action->rewrite->data;
+ bool ptrn_arg;
+ int i;
+
+ ptrn_arg = !action->rewrite->single_action_opt && ptrn && arg;
+
+ seq_printf(file, "%d,0x%llx,0x%llx,0x%x,%d,0x%x,0x%x,0x%x",
DR_DUMP_REC_TYPE_ACTION_MODIFY_HDR, action_id,
- rule_id, action->rewrite->index);
+ rule_id, action->rewrite->index,
+ action->rewrite->single_action_opt,
+ action->rewrite->num_of_actions,
+ ptrn_arg ? ptrn->index : 0,
+ ptrn_arg ? mlx5dr_arg_get_obj_id(arg) : 0);
+
+ for (i = 0; i < action->rewrite->num_of_actions; i++) {
+ seq_printf(file, ",0x%016llx",
+ be64_to_cpu(((__be64 *)rewrite_data)[i]));
+ }
+
+ seq_puts(file, "\n");
break;
+ }
case DR_ACTION_TYP_VPORT:
seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
DR_DUMP_REC_TYPE_ACTION_VPORT, action_id, rule_id,
@@ -157,7 +178,10 @@ dr_dump_rule_action_mem(struct seq_file *file, const u64 rule_id,
case DR_ACTION_TYP_TNL_L3_TO_L2:
seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
DR_DUMP_REC_TYPE_ACTION_DECAP_L3, action_id,
- rule_id, action->rewrite->index);
+ rule_id,
+ (action->rewrite->ptrn && action->rewrite->arg) ?
+ mlx5dr_arg_get_obj_id(action->rewrite->arg) :
+ action->rewrite->index);
break;
case DR_ACTION_TYP_L2_TO_TNL_L2:
seq_printf(file, "%d,0x%llx,0x%llx,0x%x\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
index 5b8bb2c..9a2dfe6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
@@ -10,6 +10,46 @@
((dmn)->info.caps.dmn_type##_sw_owner_v2 && \
(dmn)->info.caps.sw_format_ver <= MLX5_STEERING_FORMAT_CONNECTX_7))
+bool mlx5dr_domain_is_support_ptrn_arg(struct mlx5dr_domain *dmn)
+{
+ return dmn->info.caps.sw_format_ver >= MLX5_STEERING_FORMAT_CONNECTX_6DX &&
+ dmn->info.caps.support_modify_argument;
+}
+
+static int dr_domain_init_modify_header_resources(struct mlx5dr_domain *dmn)
+{
+ if (!mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return 0;
+
+ dmn->ptrn_mgr = mlx5dr_ptrn_mgr_create(dmn);
+ if (!dmn->ptrn_mgr) {
+ mlx5dr_err(dmn, "Couldn't create ptrn_mgr\n");
+ return -ENOMEM;
+ }
+
+ /* create argument pool */
+ dmn->arg_mgr = mlx5dr_arg_mgr_create(dmn);
+ if (!dmn->arg_mgr) {
+ mlx5dr_err(dmn, "Couldn't create arg_mgr\n");
+ goto free_modify_header_pattern;
+ }
+
+ return 0;
+
+free_modify_header_pattern:
+ mlx5dr_ptrn_mgr_destroy(dmn->ptrn_mgr);
+ return -ENOMEM;
+}
+
+static void dr_domain_destroy_modify_header_resources(struct mlx5dr_domain *dmn)
+{
+ if (!mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return;
+
+ mlx5dr_arg_mgr_destroy(dmn->arg_mgr);
+ mlx5dr_ptrn_mgr_destroy(dmn->ptrn_mgr);
+}
+
static void dr_domain_init_csum_recalc_fts(struct mlx5dr_domain *dmn)
{
/* Per vport cached FW FT for checksum recalculation, this
@@ -149,14 +189,22 @@ static int dr_domain_init_resources(struct mlx5dr_domain *dmn)
goto clean_uar;
}
+ ret = dr_domain_init_modify_header_resources(dmn);
+ if (ret) {
+ mlx5dr_err(dmn, "Couldn't create modify-header-resources\n");
+ goto clean_mem_resources;
+ }
+
ret = mlx5dr_send_ring_alloc(dmn);
if (ret) {
mlx5dr_err(dmn, "Couldn't create send-ring\n");
- goto clean_mem_resources;
+ goto clean_modify_hdr;
}
return 0;
+clean_modify_hdr:
+ dr_domain_destroy_modify_header_resources(dmn);
clean_mem_resources:
dr_domain_uninit_mem_resources(dmn);
clean_uar:
@@ -170,6 +218,7 @@ static int dr_domain_init_resources(struct mlx5dr_domain *dmn)
static void dr_domain_uninit_resources(struct mlx5dr_domain *dmn)
{
mlx5dr_send_ring_free(dmn, dmn->send_ring);
+ dr_domain_destroy_modify_header_resources(dmn);
dr_domain_uninit_mem_resources(dmn);
mlx5_put_uars_page(dmn->mdev, dmn->uar);
mlx5_core_dealloc_pd(dmn->mdev, dmn->pdn);
@@ -215,7 +264,7 @@ static int dr_domain_query_vport(struct mlx5dr_domain *dmn,
return 0;
}
-static int dr_domain_query_esw_mngr(struct mlx5dr_domain *dmn)
+static int dr_domain_query_esw_mgr(struct mlx5dr_domain *dmn)
{
return dr_domain_query_vport(dmn, 0, false,
&dmn->info.caps.vports.esw_manager_caps);
@@ -321,7 +370,7 @@ static int dr_domain_query_fdb_caps(struct mlx5_core_dev *mdev,
* vports (vport 0, VFs and SFs) will be queried dynamically.
*/
- ret = dr_domain_query_esw_mngr(dmn);
+ ret = dr_domain_query_esw_mgr(dmn);
if (ret) {
mlx5dr_err(dmn, "Failed to query eswitch manager vport caps (err: %d)", ret);
goto free_vports_caps_xa;
@@ -435,6 +484,9 @@ mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type)
dmn->info.max_log_action_icm_sz = DR_CHUNK_SIZE_4K;
dmn->info.max_log_sw_icm_sz = min_t(u32, DR_CHUNK_SIZE_1024K,
dmn->info.caps.log_icm_size);
+ dmn->info.max_log_modify_hdr_pattern_icm_sz =
+ min_t(u32, DR_CHUNK_SIZE_4K,
+ dmn->info.caps.log_modify_pattern_icm_size);
if (!dmn->info.supp_sw_steering) {
mlx5dr_err(dmn, "SW steering is not supported\n");
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c
index 3eb6719..04fc170 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c
@@ -107,9 +107,9 @@ static struct mlx5dr_icm_mr *
dr_icm_pool_mr_create(struct mlx5dr_icm_pool *pool)
{
struct mlx5_core_dev *mdev = pool->dmn->mdev;
- enum mlx5_sw_icm_type dm_type;
+ enum mlx5_sw_icm_type dm_type = 0;
struct mlx5dr_icm_mr *icm_mr;
- size_t log_align_base;
+ size_t log_align_base = 0;
int err;
icm_mr = kvzalloc(sizeof(*icm_mr), GFP_KERNEL);
@@ -121,14 +121,25 @@ dr_icm_pool_mr_create(struct mlx5dr_icm_pool *pool)
icm_mr->dm.length = mlx5dr_icm_pool_chunk_size_to_byte(pool->max_log_chunk_sz,
pool->icm_type);
- if (pool->icm_type == DR_ICM_TYPE_STE) {
+ switch (pool->icm_type) {
+ case DR_ICM_TYPE_STE:
dm_type = MLX5_SW_ICM_TYPE_STEERING;
log_align_base = ilog2(icm_mr->dm.length);
- } else {
+ break;
+ case DR_ICM_TYPE_MODIFY_ACTION:
dm_type = MLX5_SW_ICM_TYPE_HEADER_MODIFY;
/* Align base is 64B */
log_align_base = ilog2(DR_ICM_MODIFY_HDR_ALIGN_BASE);
+ break;
+ case DR_ICM_TYPE_MODIFY_HDR_PTRN:
+ dm_type = MLX5_SW_ICM_TYPE_HEADER_MODIFY_PATTERN;
+ /* Align base is 64B */
+ log_align_base = ilog2(DR_ICM_MODIFY_HDR_ALIGN_BASE);
+ break;
+ default:
+ WARN_ON(pool->icm_type);
}
+
icm_mr->dm.type = dm_type;
err = mlx5_dm_sw_icm_alloc(mdev, icm_mr->dm.type, icm_mr->dm.length,
@@ -493,27 +504,33 @@ struct mlx5dr_icm_pool *mlx5dr_icm_pool_create(struct mlx5dr_domain *dmn,
enum mlx5dr_icm_type icm_type)
{
u32 num_of_chunks, entry_size, max_hot_size;
- enum mlx5dr_icm_chunk_size max_log_chunk_sz;
struct mlx5dr_icm_pool *pool;
- if (icm_type == DR_ICM_TYPE_STE)
- max_log_chunk_sz = dmn->info.max_log_sw_icm_sz;
- else
- max_log_chunk_sz = dmn->info.max_log_action_icm_sz;
-
pool = kvzalloc(sizeof(*pool), GFP_KERNEL);
if (!pool)
return NULL;
pool->dmn = dmn;
pool->icm_type = icm_type;
- pool->max_log_chunk_sz = max_log_chunk_sz;
pool->chunks_kmem_cache = dmn->chunks_kmem_cache;
INIT_LIST_HEAD(&pool->buddy_mem_list);
-
mutex_init(&pool->mutex);
+ switch (icm_type) {
+ case DR_ICM_TYPE_STE:
+ pool->max_log_chunk_sz = dmn->info.max_log_sw_icm_sz;
+ break;
+ case DR_ICM_TYPE_MODIFY_ACTION:
+ pool->max_log_chunk_sz = dmn->info.max_log_action_icm_sz;
+ break;
+ case DR_ICM_TYPE_MODIFY_HDR_PTRN:
+ pool->max_log_chunk_sz = dmn->info.max_log_modify_hdr_pattern_icm_sz;
+ break;
+ default:
+ WARN_ON(icm_type);
+ }
+
entry_size = mlx5dr_icm_pool_dm_type_to_entry_size(pool->icm_type);
max_hot_size = mlx5dr_icm_pool_chunk_size_to_byte(pool->max_log_chunk_sz,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ptrn.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ptrn.c
new file mode 100644
index 0000000..13e06a6
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ptrn.c
@@ -0,0 +1,241 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include "dr_types.h"
+#include "mlx5_ifc_dr_ste_v1.h"
+
+enum dr_ptrn_modify_hdr_action_id {
+ DR_PTRN_MODIFY_HDR_ACTION_ID_NOP = 0x00,
+ DR_PTRN_MODIFY_HDR_ACTION_ID_COPY = 0x05,
+ DR_PTRN_MODIFY_HDR_ACTION_ID_SET = 0x06,
+ DR_PTRN_MODIFY_HDR_ACTION_ID_ADD = 0x07,
+ DR_PTRN_MODIFY_HDR_ACTION_ID_INSERT_INLINE = 0x0a,
+};
+
+struct mlx5dr_ptrn_mgr {
+ struct mlx5dr_domain *dmn;
+ struct mlx5dr_icm_pool *ptrn_icm_pool;
+ /* cache for modify_header ptrn */
+ struct list_head ptrn_list;
+ struct mutex modify_hdr_mutex; /* protect the pattern cache */
+};
+
+/* Cache structure and functions */
+static bool dr_ptrn_compare_modify_hdr(size_t cur_num_of_actions,
+ __be64 cur_hw_actions[],
+ size_t num_of_actions,
+ __be64 hw_actions[])
+{
+ int i;
+
+ if (cur_num_of_actions != num_of_actions)
+ return false;
+
+ for (i = 0; i < num_of_actions; i++) {
+ u8 action_id =
+ MLX5_GET(ste_double_action_set_v1, &hw_actions[i], action_id);
+
+ if (action_id == DR_PTRN_MODIFY_HDR_ACTION_ID_COPY) {
+ if (hw_actions[i] != cur_hw_actions[i])
+ return false;
+ } else {
+ if ((__force __be32)hw_actions[i] !=
+ (__force __be32)cur_hw_actions[i])
+ return false;
+ }
+ }
+
+ return true;
+}
+
+static struct mlx5dr_ptrn_obj *
+dr_ptrn_find_cached_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ size_t num_of_actions,
+ __be64 hw_actions[])
+{
+ struct mlx5dr_ptrn_obj *cached_pattern;
+ struct mlx5dr_ptrn_obj *tmp;
+
+ list_for_each_entry_safe(cached_pattern, tmp, &mgr->ptrn_list, list) {
+ if (dr_ptrn_compare_modify_hdr(cached_pattern->num_of_actions,
+ (__be64 *)cached_pattern->data,
+ num_of_actions,
+ hw_actions)) {
+ /* Put this pattern in the head of the list,
+ * as we will probably use it more.
+ */
+ list_del_init(&cached_pattern->list);
+ list_add(&cached_pattern->list, &mgr->ptrn_list);
+ return cached_pattern;
+ }
+ }
+
+ return NULL;
+}
+
+static struct mlx5dr_ptrn_obj *
+dr_ptrn_alloc_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ u16 num_of_actions, u8 *data)
+{
+ struct mlx5dr_ptrn_obj *pattern;
+ struct mlx5dr_icm_chunk *chunk;
+ u32 chunk_size;
+ u32 index;
+
+ chunk_size = ilog2(num_of_actions);
+ /* HW modify action index granularity is at least 64B */
+ chunk_size = max_t(u32, chunk_size, DR_CHUNK_SIZE_8);
+
+ chunk = mlx5dr_icm_alloc_chunk(mgr->ptrn_icm_pool, chunk_size);
+ if (!chunk)
+ return NULL;
+
+ index = (mlx5dr_icm_pool_get_chunk_icm_addr(chunk) -
+ mgr->dmn->info.caps.hdr_modify_pattern_icm_addr) /
+ DR_ACTION_CACHE_LINE_SIZE;
+
+ pattern = kzalloc(sizeof(*pattern), GFP_KERNEL);
+ if (!pattern)
+ goto free_chunk;
+
+ pattern->data = kzalloc(num_of_actions * DR_MODIFY_ACTION_SIZE *
+ sizeof(*pattern->data), GFP_KERNEL);
+ if (!pattern->data)
+ goto free_pattern;
+
+ memcpy(pattern->data, data, num_of_actions * DR_MODIFY_ACTION_SIZE);
+ pattern->chunk = chunk;
+ pattern->index = index;
+ pattern->num_of_actions = num_of_actions;
+
+ list_add(&pattern->list, &mgr->ptrn_list);
+ refcount_set(&pattern->refcount, 1);
+
+ return pattern;
+
+free_pattern:
+ kfree(pattern);
+free_chunk:
+ mlx5dr_icm_free_chunk(chunk);
+ return NULL;
+}
+
+static void
+dr_ptrn_free_pattern(struct mlx5dr_ptrn_obj *pattern)
+{
+ list_del(&pattern->list);
+ mlx5dr_icm_free_chunk(pattern->chunk);
+ kfree(pattern->data);
+ kfree(pattern);
+}
+
+struct mlx5dr_ptrn_obj *
+mlx5dr_ptrn_cache_get_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ u16 num_of_actions,
+ u8 *data)
+{
+ struct mlx5dr_ptrn_obj *pattern;
+ u64 *hw_actions;
+ u8 action_id;
+ int i;
+
+ mutex_lock(&mgr->modify_hdr_mutex);
+ pattern = dr_ptrn_find_cached_pattern(mgr,
+ num_of_actions,
+ (__be64 *)data);
+ if (!pattern) {
+ /* Alloc and add new pattern to cache */
+ pattern = dr_ptrn_alloc_pattern(mgr, num_of_actions, data);
+ if (!pattern)
+ goto out_unlock;
+
+ hw_actions = (u64 *)pattern->data;
+ /* Here we mask the pattern data to create a valid pattern
+ * since we do an OR operation between the arg and pattern
+ */
+ for (i = 0; i < num_of_actions; i++) {
+ action_id = MLX5_GET(ste_double_action_set_v1, &hw_actions[i], action_id);
+
+ if (action_id == DR_PTRN_MODIFY_HDR_ACTION_ID_SET ||
+ action_id == DR_PTRN_MODIFY_HDR_ACTION_ID_ADD ||
+ action_id == DR_PTRN_MODIFY_HDR_ACTION_ID_INSERT_INLINE)
+ MLX5_SET(ste_double_action_set_v1, &hw_actions[i], inline_data, 0);
+ }
+
+ if (mlx5dr_send_postsend_pattern(mgr->dmn, pattern->chunk,
+ num_of_actions, pattern->data)) {
+ refcount_dec(&pattern->refcount);
+ goto free_pattern;
+ }
+ } else {
+ refcount_inc(&pattern->refcount);
+ }
+
+ mutex_unlock(&mgr->modify_hdr_mutex);
+
+ return pattern;
+
+free_pattern:
+ dr_ptrn_free_pattern(pattern);
+out_unlock:
+ mutex_unlock(&mgr->modify_hdr_mutex);
+ return NULL;
+}
+
+void
+mlx5dr_ptrn_cache_put_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ struct mlx5dr_ptrn_obj *pattern)
+{
+ mutex_lock(&mgr->modify_hdr_mutex);
+
+ if (refcount_dec_and_test(&pattern->refcount))
+ dr_ptrn_free_pattern(pattern);
+
+ mutex_unlock(&mgr->modify_hdr_mutex);
+}
+
+struct mlx5dr_ptrn_mgr *mlx5dr_ptrn_mgr_create(struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_ptrn_mgr *mgr;
+
+ if (!mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return NULL;
+
+ mgr = kzalloc(sizeof(*mgr), GFP_KERNEL);
+ if (!mgr)
+ return NULL;
+
+ mgr->dmn = dmn;
+ mgr->ptrn_icm_pool = mlx5dr_icm_pool_create(dmn, DR_ICM_TYPE_MODIFY_HDR_PTRN);
+ if (!mgr->ptrn_icm_pool) {
+ mlx5dr_err(dmn, "Couldn't get modify-header-pattern memory\n");
+ goto free_mgr;
+ }
+
+ INIT_LIST_HEAD(&mgr->ptrn_list);
+ return mgr;
+
+free_mgr:
+ kfree(mgr);
+ return NULL;
+}
+
+void mlx5dr_ptrn_mgr_destroy(struct mlx5dr_ptrn_mgr *mgr)
+{
+ struct mlx5dr_ptrn_obj *pattern;
+ struct mlx5dr_ptrn_obj *tmp;
+
+ if (!mgr)
+ return;
+
+ WARN_ON(!list_empty(&mgr->ptrn_list));
+
+ list_for_each_entry_safe(pattern, tmp, &mgr->ptrn_list, list) {
+ list_del(&pattern->list);
+ kfree(pattern->data);
+ kfree(pattern);
+ }
+
+ mlx5dr_icm_pool_destroy(mgr->ptrn_icm_pool);
+ kfree(mgr);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
index fd2d31c..4a5ae86 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
@@ -18,7 +18,13 @@ struct dr_data_seg {
unsigned int send_flags;
};
+enum send_info_type {
+ WRITE_ICM = 0,
+ GTA_ARG = 1,
+};
+
struct postsend_info {
+ enum send_info_type type;
struct dr_data_seg write;
struct dr_data_seg read;
u64 remote_addr;
@@ -261,9 +267,10 @@ static struct mlx5dr_qp *dr_create_rc_qp(struct mlx5_core_dev *mdev,
dr_qp->rq.pc = 0;
dr_qp->rq.cc = 0;
- dr_qp->rq.wqe_cnt = 4;
+ dr_qp->rq.wqe_cnt = 256;
dr_qp->sq.pc = 0;
dr_qp->sq.cc = 0;
+ dr_qp->sq.head = 0;
dr_qp->sq.wqe_cnt = roundup_pow_of_two(attr->max_send_wr);
MLX5_SET(qpc, temp_qpc, log_rq_stride, ilog2(MLX5_SEND_WQE_DS) - 4);
@@ -362,39 +369,113 @@ static void dr_cmd_notify_hw(struct mlx5dr_qp *dr_qp, void *ctrl)
mlx5_write64(ctrl, dr_qp->uar->map + MLX5_BF_OFFSET);
}
-static void dr_rdma_segments(struct mlx5dr_qp *dr_qp, u64 remote_addr,
- u32 rkey, struct dr_data_seg *data_seg,
- u32 opcode, bool notify_hw)
+static void
+dr_rdma_handle_flow_access_arg_segments(struct mlx5_wqe_ctrl_seg *wq_ctrl,
+ u32 remote_addr,
+ struct dr_data_seg *data_seg,
+ int *size)
+{
+ struct mlx5_wqe_header_modify_argument_update_seg *wq_arg_seg;
+ struct mlx5_wqe_flow_update_ctrl_seg *wq_flow_seg;
+
+ wq_ctrl->general_id = cpu_to_be32(remote_addr);
+ wq_flow_seg = (void *)(wq_ctrl + 1);
+
+ /* mlx5_wqe_flow_update_ctrl_seg - all reserved */
+ memset(wq_flow_seg, 0, sizeof(*wq_flow_seg));
+ wq_arg_seg = (void *)(wq_flow_seg + 1);
+
+ memcpy(wq_arg_seg->argument_list,
+ (void *)(uintptr_t)data_seg->addr,
+ data_seg->length);
+
+ *size = (sizeof(*wq_ctrl) + /* WQE ctrl segment */
+ sizeof(*wq_flow_seg) + /* WQE flow update ctrl seg - reserved */
+ sizeof(*wq_arg_seg)) / /* WQE hdr modify arg seg - data */
+ MLX5_SEND_WQE_DS;
+}
+
+static void
+dr_rdma_handle_icm_write_segments(struct mlx5_wqe_ctrl_seg *wq_ctrl,
+ u64 remote_addr,
+ u32 rkey,
+ struct dr_data_seg *data_seg,
+ unsigned int *size)
{
struct mlx5_wqe_raddr_seg *wq_raddr;
- struct mlx5_wqe_ctrl_seg *wq_ctrl;
struct mlx5_wqe_data_seg *wq_dseg;
- unsigned int size;
- unsigned int idx;
- size = sizeof(*wq_ctrl) / 16 + sizeof(*wq_dseg) / 16 +
- sizeof(*wq_raddr) / 16;
-
- idx = dr_qp->sq.pc & (dr_qp->sq.wqe_cnt - 1);
-
- wq_ctrl = mlx5_wq_cyc_get_wqe(&dr_qp->wq.sq, idx);
- wq_ctrl->imm = 0;
- wq_ctrl->fm_ce_se = (data_seg->send_flags) ?
- MLX5_WQE_CTRL_CQ_UPDATE : 0;
- wq_ctrl->opmod_idx_opcode = cpu_to_be32(((dr_qp->sq.pc & 0xffff) << 8) |
- opcode);
- wq_ctrl->qpn_ds = cpu_to_be32(size | dr_qp->qpn << 8);
wq_raddr = (void *)(wq_ctrl + 1);
+
wq_raddr->raddr = cpu_to_be64(remote_addr);
wq_raddr->rkey = cpu_to_be32(rkey);
wq_raddr->reserved = 0;
wq_dseg = (void *)(wq_raddr + 1);
+
wq_dseg->byte_count = cpu_to_be32(data_seg->length);
wq_dseg->lkey = cpu_to_be32(data_seg->lkey);
wq_dseg->addr = cpu_to_be64(data_seg->addr);
- dr_qp->sq.wqe_head[idx] = dr_qp->sq.pc++;
+ *size = (sizeof(*wq_ctrl) + /* WQE ctrl segment */
+ sizeof(*wq_dseg) + /* WQE data segment */
+ sizeof(*wq_raddr)) / /* WQE remote addr segment */
+ MLX5_SEND_WQE_DS;
+}
+
+static void dr_set_ctrl_seg(struct mlx5_wqe_ctrl_seg *wq_ctrl,
+ struct dr_data_seg *data_seg)
+{
+ wq_ctrl->signature = 0;
+ wq_ctrl->rsvd[0] = 0;
+ wq_ctrl->rsvd[1] = 0;
+ wq_ctrl->fm_ce_se = data_seg->send_flags & IB_SEND_SIGNALED ?
+ MLX5_WQE_CTRL_CQ_UPDATE : 0;
+ wq_ctrl->imm = 0;
+}
+
+static void dr_rdma_segments(struct mlx5dr_qp *dr_qp, u64 remote_addr,
+ u32 rkey, struct dr_data_seg *data_seg,
+ u32 opcode, bool notify_hw)
+{
+ struct mlx5_wqe_ctrl_seg *wq_ctrl;
+ int opcode_mod = 0;
+ unsigned int size;
+ unsigned int idx;
+
+ idx = dr_qp->sq.pc & (dr_qp->sq.wqe_cnt - 1);
+
+ wq_ctrl = mlx5_wq_cyc_get_wqe(&dr_qp->wq.sq, idx);
+ dr_set_ctrl_seg(wq_ctrl, data_seg);
+
+ switch (opcode) {
+ case MLX5_OPCODE_RDMA_READ:
+ case MLX5_OPCODE_RDMA_WRITE:
+ dr_rdma_handle_icm_write_segments(wq_ctrl, remote_addr,
+ rkey, data_seg, &size);
+ break;
+ case MLX5_OPCODE_FLOW_TBL_ACCESS:
+ opcode_mod = MLX5_CMD_OP_MOD_UPDATE_HEADER_MODIFY_ARGUMENT;
+ dr_rdma_handle_flow_access_arg_segments(wq_ctrl, remote_addr,
+ data_seg, &size);
+ break;
+ default:
+ WARN(true, "illegal opcode %d", opcode);
+ return;
+ }
+
+ /* --------------------------------------------------------
+ * |opcode_mod (8 bit)|wqe_index (16 bits)| opcod (8 bits)|
+ * --------------------------------------------------------
+ */
+ wq_ctrl->opmod_idx_opcode =
+ cpu_to_be32((opcode_mod << 24) |
+ ((dr_qp->sq.pc & 0xffff) << 8) |
+ opcode);
+ wq_ctrl->qpn_ds = cpu_to_be32(size | dr_qp->qpn << 8);
+
+ dr_qp->sq.pc += DIV_ROUND_UP(size * 16, MLX5_SEND_WQE_BB);
+ dr_qp->sq.wqe_head[idx] = dr_qp->sq.head++;
if (notify_hw)
dr_cmd_notify_hw(dr_qp, wq_ctrl);
@@ -402,10 +483,16 @@ static void dr_rdma_segments(struct mlx5dr_qp *dr_qp, u64 remote_addr,
static void dr_post_send(struct mlx5dr_qp *dr_qp, struct postsend_info *send_info)
{
- dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
- &send_info->write, MLX5_OPCODE_RDMA_WRITE, false);
- dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
- &send_info->read, MLX5_OPCODE_RDMA_READ, true);
+ if (send_info->type == WRITE_ICM) {
+ dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
+ &send_info->write, MLX5_OPCODE_RDMA_WRITE, false);
+ dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
+ &send_info->read, MLX5_OPCODE_RDMA_READ, true);
+ } else { /* GTA_ARG */
+ dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
+ &send_info->write, MLX5_OPCODE_FLOW_TBL_ACCESS, true);
+ }
+
}
/**
@@ -471,24 +558,54 @@ static int dr_handle_pending_wc(struct mlx5dr_domain *dmn,
} else if (ne == 1) {
send_ring->pending_wqe -= send_ring->signal_th;
}
- } while (is_drain && send_ring->pending_wqe);
+ } while (ne == 1 ||
+ (is_drain && send_ring->pending_wqe >= send_ring->signal_th));
return 0;
}
-static void dr_fill_data_segs(struct mlx5dr_send_ring *send_ring,
- struct postsend_info *send_info)
+static void dr_fill_write_args_segs(struct mlx5dr_send_ring *send_ring,
+ struct postsend_info *send_info)
{
send_ring->pending_wqe++;
if (send_ring->pending_wqe % send_ring->signal_th == 0)
send_info->write.send_flags |= IB_SEND_SIGNALED;
+ else
+ send_info->write.send_flags = 0;
+}
+
+static void dr_fill_write_icm_segs(struct mlx5dr_domain *dmn,
+ struct mlx5dr_send_ring *send_ring,
+ struct postsend_info *send_info)
+{
+ u32 buff_offset;
+
+ if (send_info->write.length > dmn->info.max_inline_size) {
+ buff_offset = (send_ring->tx_head &
+ (dmn->send_ring->signal_th - 1)) *
+ send_ring->max_post_send_size;
+ /* Copy to ring mr */
+ memcpy(send_ring->buf + buff_offset,
+ (void *)(uintptr_t)send_info->write.addr,
+ send_info->write.length);
+ send_info->write.addr = (uintptr_t)send_ring->mr->dma_addr + buff_offset;
+ send_info->write.lkey = send_ring->mr->mkey;
+
+ send_ring->tx_head++;
+ }
+
+ send_ring->pending_wqe++;
+
+ if (send_ring->pending_wqe % send_ring->signal_th == 0)
+ send_info->write.send_flags |= IB_SEND_SIGNALED;
send_ring->pending_wqe++;
send_info->read.length = send_info->write.length;
- /* Read into the same write area */
- send_info->read.addr = (uintptr_t)send_info->write.addr;
- send_info->read.lkey = send_ring->mr->mkey;
+
+ /* Read into dedicated sync buffer */
+ send_info->read.addr = (uintptr_t)send_ring->sync_mr->dma_addr;
+ send_info->read.lkey = send_ring->sync_mr->mkey;
if (send_ring->pending_wqe % send_ring->signal_th == 0)
send_info->read.send_flags = IB_SEND_SIGNALED;
@@ -496,11 +613,20 @@ static void dr_fill_data_segs(struct mlx5dr_send_ring *send_ring,
send_info->read.send_flags = 0;
}
+static void dr_fill_data_segs(struct mlx5dr_domain *dmn,
+ struct mlx5dr_send_ring *send_ring,
+ struct postsend_info *send_info)
+{
+ if (send_info->type == WRITE_ICM)
+ dr_fill_write_icm_segs(dmn, send_ring, send_info);
+ else /* args */
+ dr_fill_write_args_segs(send_ring, send_info);
+}
+
static int dr_postsend_icm_data(struct mlx5dr_domain *dmn,
struct postsend_info *send_info)
{
struct mlx5dr_send_ring *send_ring = dmn->send_ring;
- u32 buff_offset;
int ret;
if (unlikely(dmn->mdev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR ||
@@ -517,20 +643,7 @@ static int dr_postsend_icm_data(struct mlx5dr_domain *dmn,
if (ret)
goto out_unlock;
- if (send_info->write.length > dmn->info.max_inline_size) {
- buff_offset = (send_ring->tx_head &
- (dmn->send_ring->signal_th - 1)) *
- send_ring->max_post_send_size;
- /* Copy to ring mr */
- memcpy(send_ring->buf + buff_offset,
- (void *)(uintptr_t)send_info->write.addr,
- send_info->write.length);
- send_info->write.addr = (uintptr_t)send_ring->mr->dma_addr + buff_offset;
- send_info->write.lkey = send_ring->mr->mkey;
- }
-
- send_ring->tx_head++;
- dr_fill_data_segs(send_ring, send_info);
+ dr_fill_data_segs(dmn, send_ring, send_info);
dr_post_send(send_ring->qp, send_info);
out_unlock:
@@ -736,6 +849,59 @@ int mlx5dr_send_postsend_action(struct mlx5dr_domain *dmn,
return dr_postsend_icm_data(dmn, &send_info);
}
+int mlx5dr_send_postsend_pattern(struct mlx5dr_domain *dmn,
+ struct mlx5dr_icm_chunk *chunk,
+ u16 num_of_actions,
+ u8 *data)
+{
+ struct postsend_info send_info = {};
+ int ret;
+
+ send_info.write.addr = (uintptr_t)data;
+ send_info.write.length = num_of_actions * DR_MODIFY_ACTION_SIZE;
+ send_info.remote_addr = mlx5dr_icm_pool_get_chunk_mr_addr(chunk);
+ send_info.rkey = mlx5dr_icm_pool_get_chunk_rkey(chunk);
+
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+int mlx5dr_send_postsend_args(struct mlx5dr_domain *dmn, u64 arg_id,
+ u16 num_of_actions, u8 *actions_data)
+{
+ int data_len, iter = 0, cur_sent;
+ u64 addr;
+ int ret;
+
+ addr = (uintptr_t)actions_data;
+ data_len = num_of_actions * DR_MODIFY_ACTION_SIZE;
+
+ do {
+ struct postsend_info send_info = {};
+
+ send_info.type = GTA_ARG;
+ send_info.write.addr = addr;
+ cur_sent = min_t(u32, data_len, DR_ACTION_CACHE_LINE_SIZE);
+ send_info.write.length = cur_sent;
+ send_info.write.lkey = 0;
+ send_info.remote_addr = arg_id + iter;
+
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ if (ret)
+ goto out;
+
+ iter++;
+ addr += cur_sent;
+ data_len -= cur_sent;
+ } while (data_len > 0);
+
+out:
+ return ret;
+}
+
static int dr_modify_qp_rst2init(struct mlx5_core_dev *mdev,
struct mlx5dr_qp *dr_qp,
int port)
@@ -1123,16 +1289,25 @@ int mlx5dr_send_ring_alloc(struct mlx5dr_domain *dmn)
goto free_mem;
}
- dmn->send_ring->sync_mr = dr_reg_mr(dmn->mdev,
- dmn->pdn, dmn->send_ring->sync_buff,
- MIN_READ_SYNC);
- if (!dmn->send_ring->sync_mr) {
+ dmn->send_ring->sync_buff = kzalloc(dmn->send_ring->max_post_send_size,
+ GFP_KERNEL);
+ if (!dmn->send_ring->sync_buff) {
ret = -ENOMEM;
goto clean_mr;
}
+ dmn->send_ring->sync_mr = dr_reg_mr(dmn->mdev,
+ dmn->pdn, dmn->send_ring->sync_buff,
+ dmn->send_ring->max_post_send_size);
+ if (!dmn->send_ring->sync_mr) {
+ ret = -ENOMEM;
+ goto free_sync_mem;
+ }
+
return 0;
+free_sync_mem:
+ kfree(dmn->send_ring->sync_buff);
clean_mr:
dr_dereg_mr(dmn->mdev, dmn->send_ring->mr);
free_mem:
@@ -1155,6 +1330,7 @@ void mlx5dr_send_ring_free(struct mlx5dr_domain *dmn,
dr_dereg_mr(dmn->mdev, send_ring->sync_mr);
dr_dereg_mr(dmn->mdev, send_ring->mr);
kfree(send_ring->buf);
+ kfree(send_ring->sync_buff);
kfree(send_ring);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
index 1e15f60..9413aaf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
@@ -633,6 +633,63 @@ int mlx5dr_ste_set_action_decap_l3_list(struct mlx5dr_ste_ctx *ste_ctx,
used_hw_action_num);
}
+static int
+dr_ste_alloc_modify_hdr_chunk(struct mlx5dr_action *action)
+{
+ struct mlx5dr_domain *dmn = action->rewrite->dmn;
+ u32 chunk_size;
+ int ret;
+
+ chunk_size = ilog2(roundup_pow_of_two(action->rewrite->num_of_actions));
+
+ /* HW modify action index granularity is at least 64B */
+ chunk_size = max_t(u32, chunk_size, DR_CHUNK_SIZE_8);
+
+ action->rewrite->chunk = mlx5dr_icm_alloc_chunk(dmn->action_icm_pool,
+ chunk_size);
+ if (!action->rewrite->chunk)
+ return -ENOMEM;
+
+ action->rewrite->index = (mlx5dr_icm_pool_get_chunk_icm_addr(action->rewrite->chunk) -
+ dmn->info.caps.hdr_modify_icm_addr) /
+ DR_ACTION_CACHE_LINE_SIZE;
+
+ ret = mlx5dr_send_postsend_action(action->rewrite->dmn, action);
+ if (ret)
+ goto free_chunk;
+
+ return 0;
+
+free_chunk:
+ mlx5dr_icm_free_chunk(action->rewrite->chunk);
+ return -ENOMEM;
+}
+
+static void dr_ste_free_modify_hdr_chunk(struct mlx5dr_action *action)
+{
+ mlx5dr_icm_free_chunk(action->rewrite->chunk);
+}
+
+int mlx5dr_ste_alloc_modify_hdr(struct mlx5dr_action *action)
+{
+ struct mlx5dr_domain *dmn = action->rewrite->dmn;
+
+ if (mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return dmn->ste_ctx->alloc_modify_hdr_chunk(action);
+
+ return dr_ste_alloc_modify_hdr_chunk(action);
+}
+
+void mlx5dr_ste_free_modify_hdr(struct mlx5dr_action *action)
+{
+ struct mlx5dr_domain *dmn = action->rewrite->dmn;
+
+ if (mlx5dr_domain_is_support_ptrn_arg(dmn))
+ return dmn->ste_ctx->dealloc_modify_hdr_chunk(action);
+
+ return dr_ste_free_modify_hdr_chunk(action);
+}
+
static int dr_ste_build_pre_check_spec(struct mlx5dr_domain *dmn,
struct mlx5dr_match_spec *spec)
{
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
index 7075142..54a6619 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h
@@ -195,6 +195,8 @@ struct mlx5dr_ste_ctx {
u8 *hw_action,
u32 hw_action_sz,
u16 *used_hw_action_num);
+ int (*alloc_modify_hdr_chunk)(struct mlx5dr_action *action);
+ void (*dealloc_modify_hdr_chunk)(struct mlx5dr_action *action);
/* Send */
void (*prepare_for_postsend)(u8 *hw_ste_p, u32 ste_size);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
index 084145f..4c0704a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c
@@ -495,21 +495,66 @@ static void dr_ste_v1_set_rx_decap(u8 *hw_ste_p, u8 *s_action)
dr_ste_v1_set_reparse(hw_ste_p);
}
-static void dr_ste_v1_set_rewrite_actions(u8 *hw_ste_p,
- u8 *s_action,
- u16 num_of_actions,
- u32 re_write_index)
+static void dr_ste_v1_set_accelerated_rewrite_actions(u8 *hw_ste_p,
+ u8 *d_action,
+ u16 num_of_actions,
+ u32 rewrite_pattern,
+ u32 rewrite_args,
+ u8 *action_data)
+{
+ if (action_data) {
+ memcpy(d_action, action_data, DR_MODIFY_ACTION_SIZE);
+ } else {
+ MLX5_SET(ste_double_action_accelerated_modify_action_list_v1, d_action,
+ action_id, DR_STE_V1_ACTION_ID_ACCELERATED_LIST);
+ MLX5_SET(ste_double_action_accelerated_modify_action_list_v1, d_action,
+ modify_actions_pattern_pointer, rewrite_pattern);
+ MLX5_SET(ste_double_action_accelerated_modify_action_list_v1, d_action,
+ number_of_modify_actions, num_of_actions);
+ MLX5_SET(ste_double_action_accelerated_modify_action_list_v1, d_action,
+ modify_actions_argument_pointer, rewrite_args);
+ }
+
+ dr_ste_v1_set_reparse(hw_ste_p);
+}
+
+static void dr_ste_v1_set_basic_rewrite_actions(u8 *hw_ste_p,
+ u8 *s_action,
+ u16 num_of_actions,
+ u32 rewrite_index)
{
MLX5_SET(ste_single_action_modify_list_v1, s_action, action_id,
DR_STE_V1_ACTION_ID_MODIFY_LIST);
MLX5_SET(ste_single_action_modify_list_v1, s_action, num_of_modify_actions,
num_of_actions);
MLX5_SET(ste_single_action_modify_list_v1, s_action, modify_actions_ptr,
- re_write_index);
+ rewrite_index);
dr_ste_v1_set_reparse(hw_ste_p);
}
+static void dr_ste_v1_set_rewrite_actions(u8 *hw_ste_p,
+ u8 *action,
+ u16 num_of_actions,
+ u32 rewrite_pattern,
+ u32 rewrite_args,
+ u8 *action_data)
+{
+ if (rewrite_pattern != MLX5DR_INVALID_PATTERN_INDEX)
+ return dr_ste_v1_set_accelerated_rewrite_actions(hw_ste_p,
+ action,
+ num_of_actions,
+ rewrite_pattern,
+ rewrite_args,
+ action_data);
+
+ /* fall back to the code that doesn't support accelerated modify header */
+ return dr_ste_v1_set_basic_rewrite_actions(hw_ste_p,
+ action,
+ num_of_actions,
+ rewrite_args);
+}
+
static void dr_ste_v1_set_aso_flow_meter(u8 *d_action,
u32 object_id,
u32 offset,
@@ -604,9 +649,6 @@ void dr_ste_v1_set_actions_tx(struct mlx5dr_domain *dmn,
allow_modify_hdr = false;
}
- if (action_type_set[DR_ACTION_TYP_CTR])
- dr_ste_v1_set_counter_id(last_ste, attr->ctr_id);
-
if (action_type_set[DR_ACTION_TYP_MODIFY_HDR]) {
if (!allow_modify_hdr || action_sz < DR_STE_ACTION_DOUBLE_SZ) {
dr_ste_v1_arr_init_next_match(&last_ste, added_stes,
@@ -617,7 +659,9 @@ void dr_ste_v1_set_actions_tx(struct mlx5dr_domain *dmn,
}
dr_ste_v1_set_rewrite_actions(last_ste, action,
attr->modify_actions,
- attr->modify_index);
+ attr->modify_pat_idx,
+ attr->modify_index,
+ attr->single_modify_action);
action_sz -= DR_STE_ACTION_DOUBLE_SZ;
action += DR_STE_ACTION_DOUBLE_SZ;
allow_encap = false;
@@ -724,6 +768,10 @@ void dr_ste_v1_set_actions_tx(struct mlx5dr_domain *dmn,
attr->range.max);
}
+ /* set counter ID on the last STE to adhere to DMFS behavior */
+ if (action_type_set[DR_ACTION_TYP_CTR])
+ dr_ste_v1_set_counter_id(last_ste, attr->ctr_id);
+
dr_ste_v1_set_hit_gvmi(last_ste, attr->hit_gvmi);
dr_ste_v1_set_hit_addr(last_ste, attr->final_icm_addr, 1);
}
@@ -743,7 +791,9 @@ void dr_ste_v1_set_actions_rx(struct mlx5dr_domain *dmn,
if (action_type_set[DR_ACTION_TYP_TNL_L3_TO_L2]) {
dr_ste_v1_set_rewrite_actions(last_ste, action,
attr->decap_actions,
- attr->decap_index);
+ attr->decap_pat_idx,
+ attr->decap_index,
+ NULL);
action_sz -= DR_STE_ACTION_DOUBLE_SZ;
action += DR_STE_ACTION_DOUBLE_SZ;
allow_modify_hdr = false;
@@ -798,7 +848,9 @@ void dr_ste_v1_set_actions_rx(struct mlx5dr_domain *dmn,
}
dr_ste_v1_set_rewrite_actions(last_ste, action,
attr->modify_actions,
- attr->modify_index);
+ attr->modify_pat_idx,
+ attr->modify_index,
+ attr->single_modify_action);
action_sz -= DR_STE_ACTION_DOUBLE_SZ;
action += DR_STE_ACTION_DOUBLE_SZ;
}
@@ -2175,6 +2227,49 @@ dr_ste_v1_build_tnl_gtpu_flex_parser_1_init(struct mlx5dr_ste_build *sb,
sb->ste_build_tag_func = &dr_ste_v1_build_tnl_gtpu_flex_parser_1_tag;
}
+int dr_ste_v1_alloc_modify_hdr_ptrn_arg(struct mlx5dr_action *action)
+{
+ struct mlx5dr_ptrn_mgr *ptrn_mgr;
+ int ret;
+
+ ptrn_mgr = action->rewrite->dmn->ptrn_mgr;
+ if (!ptrn_mgr)
+ return -EOPNOTSUPP;
+
+ action->rewrite->arg = mlx5dr_arg_get_obj(action->rewrite->dmn->arg_mgr,
+ action->rewrite->num_of_actions,
+ action->rewrite->data);
+ if (!action->rewrite->arg) {
+ mlx5dr_err(action->rewrite->dmn, "Failed allocating args for modify header\n");
+ return -EAGAIN;
+ }
+
+ action->rewrite->ptrn =
+ mlx5dr_ptrn_cache_get_pattern(ptrn_mgr,
+ action->rewrite->num_of_actions,
+ action->rewrite->data);
+ if (!action->rewrite->ptrn) {
+ mlx5dr_err(action->rewrite->dmn, "Failed to get pattern\n");
+ ret = -EAGAIN;
+ goto put_arg;
+ }
+
+ return 0;
+
+put_arg:
+ mlx5dr_arg_put_obj(action->rewrite->dmn->arg_mgr,
+ action->rewrite->arg);
+ return ret;
+}
+
+void dr_ste_v1_free_modify_hdr_ptrn_arg(struct mlx5dr_action *action)
+{
+ mlx5dr_ptrn_cache_put_pattern(action->rewrite->dmn->ptrn_mgr,
+ action->rewrite->ptrn);
+ mlx5dr_arg_put_obj(action->rewrite->dmn->arg_mgr,
+ action->rewrite->arg);
+}
+
static struct mlx5dr_ste_ctx ste_ctx_v1 = {
/* Builders */
.build_eth_l2_src_dst_init = &dr_ste_v1_build_eth_l2_src_dst_init,
@@ -2231,6 +2326,9 @@ static struct mlx5dr_ste_ctx ste_ctx_v1 = {
.set_action_add = &dr_ste_v1_set_action_add,
.set_action_copy = &dr_ste_v1_set_action_copy,
.set_action_decap_l3_list = &dr_ste_v1_set_action_decap_l3_list,
+ .alloc_modify_hdr_chunk = &dr_ste_v1_alloc_modify_hdr_ptrn_arg,
+ .dealloc_modify_hdr_chunk = &dr_ste_v1_free_modify_hdr_ptrn_arg,
+
/* Send */
.prepare_for_postsend = &dr_ste_v1_prepare_for_postsend,
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.h
index b5c0f0f..e2fc698 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.h
@@ -31,6 +31,8 @@ void dr_ste_v1_set_action_copy(u8 *d_action, u8 dst_hw_field, u8 dst_shifter,
u8 dst_len, u8 src_hw_field, u8 src_shifter);
int dr_ste_v1_set_action_decap_l3_list(void *data, u32 data_sz, u8 *hw_action,
u32 hw_action_sz, u16 *used_hw_action_num);
+int dr_ste_v1_alloc_modify_hdr_ptrn_arg(struct mlx5dr_action *action);
+void dr_ste_v1_free_modify_hdr_ptrn_arg(struct mlx5dr_action *action);
void dr_ste_v1_build_eth_l2_src_dst_init(struct mlx5dr_ste_build *sb,
struct mlx5dr_match_param *mask);
void dr_ste_v1_build_eth_l3_ipv6_dst_init(struct mlx5dr_ste_build *sb,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v2.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v2.c
index cf1a3c9..808b013 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v2.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v2.c
@@ -221,6 +221,8 @@ static struct mlx5dr_ste_ctx ste_ctx_v2 = {
.set_action_add = &dr_ste_v1_set_action_add,
.set_action_copy = &dr_ste_v1_set_action_copy,
.set_action_decap_l3_list = &dr_ste_v1_set_action_decap_l3_list,
+ .alloc_modify_hdr_chunk = &dr_ste_v1_alloc_modify_hdr_ptrn_arg,
+ .dealloc_modify_hdr_chunk = &dr_ste_v1_free_modify_hdr_ptrn_arg,
/* Send */
.prepare_for_postsend = &dr_ste_v1_prepare_for_postsend,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
index 2b769dc..37b7b1a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
@@ -21,11 +21,16 @@
#define DR_NUM_OF_FLEX_PARSERS 8
#define DR_STE_MAX_FLEX_0_ID 3
#define DR_STE_MAX_FLEX_1_ID 7
+#define DR_ACTION_CACHE_LINE_SIZE 64
#define mlx5dr_err(dmn, arg...) mlx5_core_err((dmn)->mdev, ##arg)
#define mlx5dr_info(dmn, arg...) mlx5_core_info((dmn)->mdev, ##arg)
#define mlx5dr_dbg(dmn, arg...) mlx5_core_dbg((dmn)->mdev, ##arg)
+struct mlx5dr_ptrn_mgr;
+struct mlx5dr_arg_mgr;
+struct mlx5dr_arg_obj;
+
static inline bool dr_is_flex_parser_0_id(u8 parser_id)
{
return parser_id <= DR_STE_MAX_FLEX_0_ID;
@@ -66,6 +71,7 @@ enum mlx5dr_icm_chunk_size {
enum mlx5dr_icm_type {
DR_ICM_TYPE_STE,
DR_ICM_TYPE_MODIFY_ACTION,
+ DR_ICM_TYPE_MODIFY_HDR_PTRN,
};
static inline enum mlx5dr_icm_chunk_size
@@ -255,11 +261,15 @@ u64 mlx5dr_ste_get_mr_addr(struct mlx5dr_ste *ste);
struct list_head *mlx5dr_ste_get_miss_list(struct mlx5dr_ste *ste);
#define MLX5DR_MAX_VLANS 2
+#define MLX5DR_INVALID_PATTERN_INDEX 0xffffffff
struct mlx5dr_ste_actions_attr {
u32 modify_index;
+ u32 modify_pat_idx;
u16 modify_actions;
+ u8 *single_modify_action;
u32 decap_index;
+ u32 decap_pat_idx;
u16 decap_actions;
u8 decap_with_vlan:1;
u64 final_icm_addr;
@@ -331,6 +341,8 @@ int mlx5dr_ste_set_action_decap_l3_list(struct mlx5dr_ste_ctx *ste_ctx,
u8 *hw_action,
u32 hw_action_sz,
u16 *used_hw_action_num);
+int mlx5dr_ste_alloc_modify_hdr(struct mlx5dr_action *action);
+void mlx5dr_ste_free_modify_hdr(struct mlx5dr_action *action);
const struct mlx5dr_ste_action_modify_field *
mlx5dr_ste_conv_modify_hdr_sw_field(struct mlx5dr_ste_ctx *ste_ctx, u16 sw_field);
@@ -861,6 +873,8 @@ struct mlx5dr_cmd_caps {
u64 esw_tx_drop_address;
u32 log_icm_size;
u64 hdr_modify_icm_addr;
+ u32 log_modify_pattern_icm_size;
+ u64 hdr_modify_pattern_icm_addr;
u32 flex_protocols;
u8 flex_parser_id_icmp_dw0;
u8 flex_parser_id_icmp_dw1;
@@ -888,6 +902,9 @@ struct mlx5dr_cmd_caps {
struct mlx5dr_vports vports;
bool prio_tag_required;
struct mlx5dr_roce_cap roce_caps;
+ u16 log_header_modify_argument_granularity;
+ u16 log_header_modify_argument_max_alloc;
+ bool support_modify_argument;
u8 is_ecpf:1;
u8 isolate_vl_tc:1;
};
@@ -910,6 +927,7 @@ struct mlx5dr_domain_info {
u32 max_send_wr;
u32 max_log_sw_icm_sz;
u32 max_log_action_icm_sz;
+ u32 max_log_modify_hdr_pattern_icm_sz;
struct mlx5dr_domain_rx_tx rx;
struct mlx5dr_domain_rx_tx tx;
struct mlx5dr_cmd_caps caps;
@@ -928,6 +946,8 @@ struct mlx5dr_domain {
struct mlx5dr_send_info_pool *send_info_pool_tx;
struct kmem_cache *chunks_kmem_cache;
struct kmem_cache *htbls_kmem_cache;
+ struct mlx5dr_ptrn_mgr *ptrn_mgr;
+ struct mlx5dr_arg_mgr *arg_mgr;
struct mlx5dr_send_ring *send_ring;
struct mlx5dr_domain_info info;
struct xarray csum_fts_xa;
@@ -994,15 +1014,34 @@ struct mlx5dr_ste_action_modify_field {
u8 l4_type;
};
+struct mlx5dr_ptrn_obj {
+ struct mlx5dr_icm_chunk *chunk;
+ u8 *data;
+ u16 num_of_actions;
+ u32 index;
+ refcount_t refcount;
+ struct list_head list;
+};
+
+struct mlx5dr_arg_obj {
+ u32 obj_id;
+ u32 obj_offset;
+ struct list_head list_node;
+ u32 log_chunk_size;
+};
+
struct mlx5dr_action_rewrite {
struct mlx5dr_domain *dmn;
struct mlx5dr_icm_chunk *chunk;
u8 *data;
u16 num_of_actions;
u32 index;
+ u8 single_action_opt:1;
u8 allow_rx:1;
u8 allow_tx:1;
u8 modify_ttl:1;
+ struct mlx5dr_ptrn_obj *ptrn;
+ struct mlx5dr_arg_obj *arg;
};
struct mlx5dr_action_reformat {
@@ -1334,6 +1373,12 @@ struct mlx5dr_cmd_gid_attr {
int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num,
u16 index, struct mlx5dr_cmd_gid_attr *attr);
+int mlx5dr_cmd_create_modify_header_arg(struct mlx5_core_dev *dev,
+ u16 log_obj_range, u32 pd,
+ u32 *obj_id);
+void mlx5dr_cmd_destroy_modify_header_arg(struct mlx5_core_dev *dev,
+ u32 obj_id);
+
struct mlx5dr_icm_pool *mlx5dr_icm_pool_create(struct mlx5dr_domain *dmn,
enum mlx5dr_icm_type icm_type);
void mlx5dr_icm_pool_destroy(struct mlx5dr_icm_pool *pool);
@@ -1368,6 +1413,7 @@ struct mlx5dr_qp {
struct mlx5_wq_ctrl wq_ctrl;
u32 qpn;
struct {
+ unsigned int head;
unsigned int pc;
unsigned int cc;
unsigned int size;
@@ -1399,9 +1445,6 @@ struct mlx5dr_mr {
size_t size;
};
-#define MAX_SEND_CQE 64
-#define MIN_READ_SYNC 64
-
struct mlx5dr_send_ring {
struct mlx5dr_cq *cq;
struct mlx5dr_qp *qp;
@@ -1416,7 +1459,7 @@ struct mlx5dr_send_ring {
u32 tx_head;
void *buf;
u32 buf_size;
- u8 sync_buff[MIN_READ_SYNC];
+ u8 *sync_buff;
struct mlx5dr_mr *sync_mr;
spinlock_t lock; /* Protect the data path of the send ring */
bool err_state; /* send_ring is not usable in err state */
@@ -1440,6 +1483,12 @@ int mlx5dr_send_postsend_formatted_htbl(struct mlx5dr_domain *dmn,
bool update_hw_ste);
int mlx5dr_send_postsend_action(struct mlx5dr_domain *dmn,
struct mlx5dr_action *action);
+int mlx5dr_send_postsend_pattern(struct mlx5dr_domain *dmn,
+ struct mlx5dr_icm_chunk *chunk,
+ u16 num_of_actions,
+ u8 *data);
+int mlx5dr_send_postsend_args(struct mlx5dr_domain *dmn, u64 arg_id,
+ u16 num_of_actions, u8 *actions_data);
int mlx5dr_send_info_pool_create(struct mlx5dr_domain *dmn);
void mlx5dr_send_info_pool_destroy(struct mlx5dr_domain *dmn);
@@ -1526,4 +1575,20 @@ static inline bool mlx5dr_supp_match_ranges(struct mlx5_core_dev *dev)
(1ULL << MLX5_IFC_DEFINER_FORMAT_ID_SELECT));
}
+bool mlx5dr_domain_is_support_ptrn_arg(struct mlx5dr_domain *dmn);
+struct mlx5dr_ptrn_mgr *mlx5dr_ptrn_mgr_create(struct mlx5dr_domain *dmn);
+void mlx5dr_ptrn_mgr_destroy(struct mlx5dr_ptrn_mgr *mgr);
+struct mlx5dr_ptrn_obj *mlx5dr_ptrn_cache_get_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ u16 num_of_actions, u8 *data);
+void mlx5dr_ptrn_cache_put_pattern(struct mlx5dr_ptrn_mgr *mgr,
+ struct mlx5dr_ptrn_obj *pattern);
+struct mlx5dr_arg_mgr *mlx5dr_arg_mgr_create(struct mlx5dr_domain *dmn);
+void mlx5dr_arg_mgr_destroy(struct mlx5dr_arg_mgr *mgr);
+struct mlx5dr_arg_obj *mlx5dr_arg_get_obj(struct mlx5dr_arg_mgr *mgr,
+ u16 num_of_actions,
+ u8 *data);
+void mlx5dr_arg_put_obj(struct mlx5dr_arg_mgr *mgr,
+ struct mlx5dr_arg_obj *arg_obj);
+u32 mlx5dr_arg_get_obj_id(struct mlx5dr_arg_obj *arg_obj);
+
#endif /* _DR_TYPES_H_ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr_ste_v1.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr_ste_v1.h
index 790a17d..ca3b0f1 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr_ste_v1.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr_ste_v1.h
@@ -100,7 +100,7 @@ struct mlx5_ifc_ste_double_action_insert_with_ptr_v1_bits {
u8 pointer[0x20];
};
-struct mlx5_ifc_ste_double_action_modify_action_list_v1_bits {
+struct mlx5_ifc_ste_double_action_accelerated_modify_action_list_v1_bits {
u8 action_id[0x8];
u8 modify_actions_pattern_pointer[0x18];
diff --git a/drivers/net/ethernet/mellanox/mlxfw/mlxfw_mfa2_tlv_multi.c b/drivers/net/ethernet/mellanox/mlxfw/mlxfw_mfa2_tlv_multi.c
index 017d68f..972c571 100644
--- a/drivers/net/ethernet/mellanox/mlxfw/mlxfw_mfa2_tlv_multi.c
+++ b/drivers/net/ethernet/mellanox/mlxfw/mlxfw_mfa2_tlv_multi.c
@@ -31,6 +31,8 @@ mlxfw_mfa2_tlv_next(const struct mlxfw_mfa2_file *mfa2_file,
if (tlv->type == MLXFW_MFA2_TLV_MULTI_PART) {
multi = mlxfw_mfa2_tlv_multi_get(mfa2_file, tlv);
+ if (!multi)
+ return NULL;
tlv_len = NLA_ALIGN(tlv_len + be16_to_cpu(multi->total_len));
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci_hw.h b/drivers/net/ethernet/mellanox/mlxsw/pci_hw.h
index 48dbfea..7cdf0ce 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/pci_hw.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/pci_hw.h
@@ -26,7 +26,7 @@
#define MLXSW_PCI_CIR_TIMEOUT_MSECS 1000
#define MLXSW_PCI_SW_RESET_TIMEOUT_MSECS 900000
-#define MLXSW_PCI_SW_RESET_WAIT_MSECS 200
+#define MLXSW_PCI_SW_RESET_WAIT_MSECS 400
#define MLXSW_PCI_FW_READY 0xA1844
#define MLXSW_PCI_FW_READY_MASK 0xFFFF
#define MLXSW_PCI_FW_READY_MAGIC 0x5E
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
index 9be6462..2b6e046 100644
--- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
@@ -605,7 +605,7 @@ static u64 lan966x_ifh_get(u8 *ifh, size_t pos, size_t length)
v = ifh[IFH_LEN_BYTES - (j / 8) - 1];
if (v & (1 << k))
- val |= (1 << i);
+ val |= (1ULL << i);
}
return val;
diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
index 3caea63..23b1521 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
@@ -133,12 +133,6 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
return act;
}
-static unsigned int mana_xdp_fraglen(unsigned int len)
-{
- return SKB_DATA_ALIGN(len) +
- SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
-}
-
struct bpf_prog *mana_xdp_get(struct mana_port_context *apc)
{
ASSERT_RTNL();
@@ -179,17 +173,18 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
{
struct mana_port_context *apc = netdev_priv(ndev);
struct bpf_prog *old_prog;
- int buf_max;
+ struct gdma_context *gc;
+
+ gc = apc->ac->gdma_dev->gdma_context;
old_prog = mana_xdp_get(apc);
if (!old_prog && !prog)
return 0;
- buf_max = XDP_PACKET_HEADROOM + mana_xdp_fraglen(ndev->mtu + ETH_HLEN);
- if (prog && buf_max > PAGE_SIZE) {
- netdev_err(ndev, "XDP: mtu:%u too large, buf_max:%u\n",
- ndev->mtu, buf_max);
+ if (prog && ndev->mtu > MANA_XDP_MTU_MAX) {
+ netdev_err(ndev, "XDP: mtu:%u too large, mtu_max:%lu\n",
+ ndev->mtu, MANA_XDP_MTU_MAX);
NL_SET_ERR_MSG_MOD(extack, "XDP: mtu too large");
return -EOPNOTSUPP;
@@ -206,6 +201,11 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
if (apc->port_is_up)
mana_chn_setxdp(apc, prog);
+ if (prog)
+ ndev->max_mtu = MANA_XDP_MTU_MAX;
+ else
+ ndev->max_mtu = gc->adapter_mtu - ETH_HLEN;
+
return 0;
}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 492474b..cabecbf 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -427,6 +427,192 @@ static u16 mana_select_queue(struct net_device *ndev, struct sk_buff *skb,
return txq;
}
+/* Release pre-allocated RX buffers */
+static void mana_pre_dealloc_rxbufs(struct mana_port_context *mpc)
+{
+ struct device *dev;
+ int i;
+
+ dev = mpc->ac->gdma_dev->gdma_context->dev;
+
+ if (!mpc->rxbufs_pre)
+ goto out1;
+
+ if (!mpc->das_pre)
+ goto out2;
+
+ while (mpc->rxbpre_total) {
+ i = --mpc->rxbpre_total;
+ dma_unmap_single(dev, mpc->das_pre[i], mpc->rxbpre_datasize,
+ DMA_FROM_DEVICE);
+ put_page(virt_to_head_page(mpc->rxbufs_pre[i]));
+ }
+
+ kfree(mpc->das_pre);
+ mpc->das_pre = NULL;
+
+out2:
+ kfree(mpc->rxbufs_pre);
+ mpc->rxbufs_pre = NULL;
+
+out1:
+ mpc->rxbpre_datasize = 0;
+ mpc->rxbpre_alloc_size = 0;
+ mpc->rxbpre_headroom = 0;
+}
+
+/* Get a buffer from the pre-allocated RX buffers */
+static void *mana_get_rxbuf_pre(struct mana_rxq *rxq, dma_addr_t *da)
+{
+ struct net_device *ndev = rxq->ndev;
+ struct mana_port_context *mpc;
+ void *va;
+
+ mpc = netdev_priv(ndev);
+
+ if (!mpc->rxbufs_pre || !mpc->das_pre || !mpc->rxbpre_total) {
+ netdev_err(ndev, "No RX pre-allocated bufs\n");
+ return NULL;
+ }
+
+ /* Check sizes to catch unexpected coding error */
+ if (mpc->rxbpre_datasize != rxq->datasize) {
+ netdev_err(ndev, "rxbpre_datasize mismatch: %u: %u\n",
+ mpc->rxbpre_datasize, rxq->datasize);
+ return NULL;
+ }
+
+ if (mpc->rxbpre_alloc_size != rxq->alloc_size) {
+ netdev_err(ndev, "rxbpre_alloc_size mismatch: %u: %u\n",
+ mpc->rxbpre_alloc_size, rxq->alloc_size);
+ return NULL;
+ }
+
+ if (mpc->rxbpre_headroom != rxq->headroom) {
+ netdev_err(ndev, "rxbpre_headroom mismatch: %u: %u\n",
+ mpc->rxbpre_headroom, rxq->headroom);
+ return NULL;
+ }
+
+ mpc->rxbpre_total--;
+
+ *da = mpc->das_pre[mpc->rxbpre_total];
+ va = mpc->rxbufs_pre[mpc->rxbpre_total];
+ mpc->rxbufs_pre[mpc->rxbpre_total] = NULL;
+
+ /* Deallocate the array after all buffers are gone */
+ if (!mpc->rxbpre_total)
+ mana_pre_dealloc_rxbufs(mpc);
+
+ return va;
+}
+
+/* Get RX buffer's data size, alloc size, XDP headroom based on MTU */
+static void mana_get_rxbuf_cfg(int mtu, u32 *datasize, u32 *alloc_size,
+ u32 *headroom)
+{
+ if (mtu > MANA_XDP_MTU_MAX)
+ *headroom = 0; /* no support for XDP */
+ else
+ *headroom = XDP_PACKET_HEADROOM;
+
+ *alloc_size = mtu + MANA_RXBUF_PAD + *headroom;
+
+ *datasize = ALIGN(mtu + ETH_HLEN, MANA_RX_DATA_ALIGN);
+}
+
+static int mana_pre_alloc_rxbufs(struct mana_port_context *mpc, int new_mtu)
+{
+ struct device *dev;
+ struct page *page;
+ dma_addr_t da;
+ int num_rxb;
+ void *va;
+ int i;
+
+ mana_get_rxbuf_cfg(new_mtu, &mpc->rxbpre_datasize,
+ &mpc->rxbpre_alloc_size, &mpc->rxbpre_headroom);
+
+ dev = mpc->ac->gdma_dev->gdma_context->dev;
+
+ num_rxb = mpc->num_queues * RX_BUFFERS_PER_QUEUE;
+
+ WARN(mpc->rxbufs_pre, "mana rxbufs_pre exists\n");
+ mpc->rxbufs_pre = kmalloc_array(num_rxb, sizeof(void *), GFP_KERNEL);
+ if (!mpc->rxbufs_pre)
+ goto error;
+
+ mpc->das_pre = kmalloc_array(num_rxb, sizeof(dma_addr_t), GFP_KERNEL);
+ if (!mpc->das_pre)
+ goto error;
+
+ mpc->rxbpre_total = 0;
+
+ for (i = 0; i < num_rxb; i++) {
+ if (mpc->rxbpre_alloc_size > PAGE_SIZE) {
+ va = netdev_alloc_frag(mpc->rxbpre_alloc_size);
+ if (!va)
+ goto error;
+ } else {
+ page = dev_alloc_page();
+ if (!page)
+ goto error;
+
+ va = page_to_virt(page);
+ }
+
+ da = dma_map_single(dev, va + mpc->rxbpre_headroom,
+ mpc->rxbpre_datasize, DMA_FROM_DEVICE);
+
+ if (dma_mapping_error(dev, da)) {
+ put_page(virt_to_head_page(va));
+ goto error;
+ }
+
+ mpc->rxbufs_pre[i] = va;
+ mpc->das_pre[i] = da;
+ mpc->rxbpre_total = i + 1;
+ }
+
+ return 0;
+
+error:
+ mana_pre_dealloc_rxbufs(mpc);
+ return -ENOMEM;
+}
+
+static int mana_change_mtu(struct net_device *ndev, int new_mtu)
+{
+ struct mana_port_context *mpc = netdev_priv(ndev);
+ unsigned int old_mtu = ndev->mtu;
+ int err;
+
+ /* Pre-allocate buffers to prevent failure in mana_attach later */
+ err = mana_pre_alloc_rxbufs(mpc, new_mtu);
+ if (err) {
+ netdev_err(ndev, "Insufficient memory for new MTU\n");
+ return err;
+ }
+
+ err = mana_detach(ndev, false);
+ if (err) {
+ netdev_err(ndev, "mana_detach failed: %d\n", err);
+ goto out;
+ }
+
+ ndev->mtu = new_mtu;
+
+ err = mana_attach(ndev);
+ if (err) {
+ netdev_err(ndev, "mana_attach failed: %d\n", err);
+ ndev->mtu = old_mtu;
+ }
+
+out:
+ mana_pre_dealloc_rxbufs(mpc);
+ return err;
+}
+
static const struct net_device_ops mana_devops = {
.ndo_open = mana_open,
.ndo_stop = mana_close,
@@ -436,6 +622,7 @@ static const struct net_device_ops mana_devops = {
.ndo_get_stats64 = mana_get_stats64,
.ndo_bpf = mana_bpf,
.ndo_xdp_xmit = mana_xdp_xmit,
+ .ndo_change_mtu = mana_change_mtu,
};
static void mana_cleanup_port_context(struct mana_port_context *apc)
@@ -625,6 +812,9 @@ static int mana_query_device_cfg(struct mana_context *ac, u32 proto_major_ver,
mana_gd_init_req_hdr(&req.hdr, MANA_QUERY_DEV_CONFIG,
sizeof(req), sizeof(resp));
+
+ req.hdr.resp.msg_version = GDMA_MESSAGE_V2;
+
req.proto_major_ver = proto_major_ver;
req.proto_minor_ver = proto_minor_ver;
req.proto_micro_ver = proto_micro_ver;
@@ -647,6 +837,11 @@ static int mana_query_device_cfg(struct mana_context *ac, u32 proto_major_ver,
*max_num_vports = resp.max_num_vports;
+ if (resp.hdr.response.msg_version == GDMA_MESSAGE_V2)
+ gc->adapter_mtu = resp.adapter_mtu;
+ else
+ gc->adapter_mtu = ETH_FRAME_LEN;
+
return 0;
}
@@ -1185,10 +1380,10 @@ static void mana_post_pkt_rxq(struct mana_rxq *rxq)
WARN_ON_ONCE(recv_buf_oob->wqe_inf.wqe_size_in_bu != 1);
}
-static struct sk_buff *mana_build_skb(void *buf_va, uint pkt_len,
- struct xdp_buff *xdp)
+static struct sk_buff *mana_build_skb(struct mana_rxq *rxq, void *buf_va,
+ uint pkt_len, struct xdp_buff *xdp)
{
- struct sk_buff *skb = build_skb(buf_va, PAGE_SIZE);
+ struct sk_buff *skb = napi_build_skb(buf_va, rxq->alloc_size);
if (!skb)
return NULL;
@@ -1196,11 +1391,12 @@ static struct sk_buff *mana_build_skb(void *buf_va, uint pkt_len,
if (xdp->data_hard_start) {
skb_reserve(skb, xdp->data - xdp->data_hard_start);
skb_put(skb, xdp->data_end - xdp->data);
- } else {
- skb_reserve(skb, XDP_PACKET_HEADROOM);
- skb_put(skb, pkt_len);
+ return skb;
}
+ skb_reserve(skb, rxq->headroom);
+ skb_put(skb, pkt_len);
+
return skb;
}
@@ -1233,7 +1429,7 @@ static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
if (act != XDP_PASS && act != XDP_TX)
goto drop_xdp;
- skb = mana_build_skb(buf_va, pkt_len, &xdp);
+ skb = mana_build_skb(rxq, buf_va, pkt_len, &xdp);
if (!skb)
goto drop;
@@ -1282,14 +1478,72 @@ static void mana_rx_skb(void *buf_va, struct mana_rxcomp_oob *cqe,
u64_stats_update_end(&rx_stats->syncp);
drop:
- WARN_ON_ONCE(rxq->xdp_save_page);
- rxq->xdp_save_page = virt_to_page(buf_va);
+ WARN_ON_ONCE(rxq->xdp_save_va);
+ /* Save for reuse */
+ rxq->xdp_save_va = buf_va;
++ndev->stats.rx_dropped;
return;
}
+static void *mana_get_rxfrag(struct mana_rxq *rxq, struct device *dev,
+ dma_addr_t *da, bool is_napi)
+{
+ struct page *page;
+ void *va;
+
+ /* Reuse XDP dropped page if available */
+ if (rxq->xdp_save_va) {
+ va = rxq->xdp_save_va;
+ rxq->xdp_save_va = NULL;
+ } else if (rxq->alloc_size > PAGE_SIZE) {
+ if (is_napi)
+ va = napi_alloc_frag(rxq->alloc_size);
+ else
+ va = netdev_alloc_frag(rxq->alloc_size);
+
+ if (!va)
+ return NULL;
+ } else {
+ page = dev_alloc_page();
+ if (!page)
+ return NULL;
+
+ va = page_to_virt(page);
+ }
+
+ *da = dma_map_single(dev, va + rxq->headroom, rxq->datasize,
+ DMA_FROM_DEVICE);
+
+ if (dma_mapping_error(dev, *da)) {
+ put_page(virt_to_head_page(va));
+ return NULL;
+ }
+
+ return va;
+}
+
+/* Allocate frag for rx buffer, and save the old buf */
+static void mana_refill_rxoob(struct device *dev, struct mana_rxq *rxq,
+ struct mana_recv_buf_oob *rxoob, void **old_buf)
+{
+ dma_addr_t da;
+ void *va;
+
+ va = mana_get_rxfrag(rxq, dev, &da, true);
+
+ if (!va)
+ return;
+
+ dma_unmap_single(dev, rxoob->sgl[0].address, rxq->datasize,
+ DMA_FROM_DEVICE);
+ *old_buf = rxoob->buf_va;
+
+ rxoob->buf_va = va;
+ rxoob->sgl[0].address = da;
+}
+
static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
struct gdma_comp *cqe)
{
@@ -1299,10 +1553,8 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
struct mana_recv_buf_oob *rxbuf_oob;
struct mana_port_context *apc;
struct device *dev = gc->dev;
- void *new_buf, *old_buf;
- struct page *new_page;
+ void *old_buf = NULL;
u32 curr, pktlen;
- dma_addr_t da;
apc = netdev_priv(ndev);
@@ -1345,40 +1597,11 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
rxbuf_oob = &rxq->rx_oobs[curr];
WARN_ON_ONCE(rxbuf_oob->wqe_inf.wqe_size_in_bu != 1);
- /* Reuse XDP dropped page if available */
- if (rxq->xdp_save_page) {
- new_page = rxq->xdp_save_page;
- rxq->xdp_save_page = NULL;
- } else {
- new_page = alloc_page(GFP_ATOMIC);
- }
+ mana_refill_rxoob(dev, rxq, rxbuf_oob, &old_buf);
- if (new_page) {
- da = dma_map_page(dev, new_page, XDP_PACKET_HEADROOM, rxq->datasize,
- DMA_FROM_DEVICE);
-
- if (dma_mapping_error(dev, da)) {
- __free_page(new_page);
- new_page = NULL;
- }
- }
-
- new_buf = new_page ? page_to_virt(new_page) : NULL;
-
- if (new_buf) {
- dma_unmap_page(dev, rxbuf_oob->buf_dma_addr, rxq->datasize,
- DMA_FROM_DEVICE);
-
- old_buf = rxbuf_oob->buf_va;
-
- /* refresh the rxbuf_oob with the new page */
- rxbuf_oob->buf_va = new_buf;
- rxbuf_oob->buf_dma_addr = da;
- rxbuf_oob->sgl[0].address = rxbuf_oob->buf_dma_addr;
- } else {
- old_buf = NULL; /* drop the packet if no memory */
- }
-
+ /* Unsuccessful refill will have old_buf == NULL.
+ * In this case, mana_rx_skb() will drop the packet.
+ */
mana_rx_skb(old_buf, oob, rxq);
drop:
@@ -1659,8 +1882,8 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
mana_deinit_cq(apc, &rxq->rx_cq);
- if (rxq->xdp_save_page)
- __free_page(rxq->xdp_save_page);
+ if (rxq->xdp_save_va)
+ put_page(virt_to_head_page(rxq->xdp_save_va));
for (i = 0; i < rxq->num_rx_buf; i++) {
rx_oob = &rxq->rx_oobs[i];
@@ -1668,10 +1891,10 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
if (!rx_oob->buf_va)
continue;
- dma_unmap_page(dev, rx_oob->buf_dma_addr, rxq->datasize,
- DMA_FROM_DEVICE);
+ dma_unmap_single(dev, rx_oob->sgl[0].address,
+ rx_oob->sgl[0].size, DMA_FROM_DEVICE);
- free_page((unsigned long)rx_oob->buf_va);
+ put_page(virt_to_head_page(rx_oob->buf_va));
rx_oob->buf_va = NULL;
}
@@ -1681,6 +1904,30 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
kfree(rxq);
}
+static int mana_fill_rx_oob(struct mana_recv_buf_oob *rx_oob, u32 mem_key,
+ struct mana_rxq *rxq, struct device *dev)
+{
+ struct mana_port_context *mpc = netdev_priv(rxq->ndev);
+ dma_addr_t da;
+ void *va;
+
+ if (mpc->rxbufs_pre)
+ va = mana_get_rxbuf_pre(rxq, &da);
+ else
+ va = mana_get_rxfrag(rxq, dev, &da, false);
+
+ if (!va)
+ return -ENOMEM;
+
+ rx_oob->buf_va = va;
+
+ rx_oob->sgl[0].address = da;
+ rx_oob->sgl[0].size = rxq->datasize;
+ rx_oob->sgl[0].mem_key = mem_key;
+
+ return 0;
+}
+
#define MANA_WQE_HEADER_SIZE 16
#define MANA_WQE_SGE_SIZE 16
@@ -1690,11 +1937,10 @@ static int mana_alloc_rx_wqe(struct mana_port_context *apc,
struct gdma_context *gc = apc->ac->gdma_dev->gdma_context;
struct mana_recv_buf_oob *rx_oob;
struct device *dev = gc->dev;
- struct page *page;
- dma_addr_t da;
u32 buf_idx;
+ int ret;
- WARN_ON(rxq->datasize == 0 || rxq->datasize > PAGE_SIZE);
+ WARN_ON(rxq->datasize == 0);
*rxq_size = 0;
*cq_size = 0;
@@ -1703,25 +1949,12 @@ static int mana_alloc_rx_wqe(struct mana_port_context *apc,
rx_oob = &rxq->rx_oobs[buf_idx];
memset(rx_oob, 0, sizeof(*rx_oob));
- page = alloc_page(GFP_KERNEL);
- if (!page)
- return -ENOMEM;
-
- da = dma_map_page(dev, page, XDP_PACKET_HEADROOM, rxq->datasize,
- DMA_FROM_DEVICE);
-
- if (dma_mapping_error(dev, da)) {
- __free_page(page);
- return -ENOMEM;
- }
-
- rx_oob->buf_va = page_to_virt(page);
- rx_oob->buf_dma_addr = da;
-
rx_oob->num_sge = 1;
- rx_oob->sgl[0].address = rx_oob->buf_dma_addr;
- rx_oob->sgl[0].size = rxq->datasize;
- rx_oob->sgl[0].mem_key = apc->ac->gdma_dev->gpa_mkey;
+
+ ret = mana_fill_rx_oob(rx_oob, apc->ac->gdma_dev->gpa_mkey, rxq,
+ dev);
+ if (ret)
+ return ret;
rx_oob->wqe_req.sgl = rx_oob->sgl;
rx_oob->wqe_req.num_sge = rx_oob->num_sge;
@@ -1780,9 +2013,11 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
rxq->ndev = ndev;
rxq->num_rx_buf = RX_BUFFERS_PER_QUEUE;
rxq->rxq_idx = rxq_idx;
- rxq->datasize = ALIGN(MAX_FRAME_SIZE, 64);
rxq->rxobj = INVALID_MANA_HANDLE;
+ mana_get_rxbuf_cfg(ndev->mtu, &rxq->datasize, &rxq->alloc_size,
+ &rxq->headroom);
+
err = mana_alloc_rx_wqe(apc, rxq, &rq_size, &cq_size);
if (err)
goto out;
@@ -2194,8 +2429,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
ndev->netdev_ops = &mana_devops;
ndev->ethtool_ops = &mana_ethtool_ops;
ndev->mtu = ETH_DATA_LEN;
- ndev->max_mtu = ndev->mtu;
- ndev->min_mtu = ndev->mtu;
+ ndev->max_mtu = gc->adapter_mtu - ETH_HLEN;
+ ndev->min_mtu = ETH_MIN_MTU;
ndev->needed_headroom = MANA_HEADROOM;
ndev->dev_port = port_idx;
SET_NETDEV_DEV(ndev, gc->dev);
diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 1502bb2..1f5f00b 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -8,6 +8,7 @@
#include <linux/if_bridge.h>
#include <linux/iopoll.h>
#include <linux/phy/phy.h>
+#include <net/pkt_sched.h>
#include <soc/mscc/ocelot_hsio.h>
#include <soc/mscc/ocelot_vcap.h>
#include "ocelot.h"
@@ -1005,7 +1006,12 @@ void ocelot_phylink_mac_link_up(struct ocelot *ocelot, int port,
*/
if (ocelot->ops->cut_through_fwd) {
mutex_lock(&ocelot->fwd_domain_lock);
- ocelot->ops->cut_through_fwd(ocelot);
+ /* Workaround for hardware bug - FP doesn't work
+ * at all link speeds for all PHY modes. The function
+ * below also calls ocelot->ops->cut_through_fwd(),
+ * so we don't need to do it twice.
+ */
+ ocelot_port_update_active_preemptible_tcs(ocelot, port);
mutex_unlock(&ocelot->fwd_domain_lock);
}
@@ -2699,6 +2705,58 @@ void ocelot_port_mirror_del(struct ocelot *ocelot, int from, bool ingress)
}
EXPORT_SYMBOL_GPL(ocelot_port_mirror_del);
+static void ocelot_port_reset_mqprio(struct ocelot *ocelot, int port)
+{
+ struct net_device *dev = ocelot->ops->port_to_netdev(ocelot, port);
+
+ netdev_reset_tc(dev);
+ ocelot_port_change_fp(ocelot, port, 0);
+}
+
+int ocelot_port_mqprio(struct ocelot *ocelot, int port,
+ struct tc_mqprio_qopt_offload *mqprio)
+{
+ struct net_device *dev = ocelot->ops->port_to_netdev(ocelot, port);
+ struct netlink_ext_ack *extack = mqprio->extack;
+ struct tc_mqprio_qopt *qopt = &mqprio->qopt;
+ int num_tc = qopt->num_tc;
+ int tc, err;
+
+ if (!num_tc) {
+ ocelot_port_reset_mqprio(ocelot, port);
+ return 0;
+ }
+
+ err = netdev_set_num_tc(dev, num_tc);
+ if (err)
+ return err;
+
+ for (tc = 0; tc < num_tc; tc++) {
+ if (qopt->count[tc] != 1) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Only one TXQ per TC supported");
+ return -EINVAL;
+ }
+
+ err = netdev_set_tc_queue(dev, tc, 1, qopt->offset[tc]);
+ if (err)
+ goto err_reset_tc;
+ }
+
+ err = netif_set_real_num_tx_queues(dev, num_tc);
+ if (err)
+ goto err_reset_tc;
+
+ ocelot_port_change_fp(ocelot, port, mqprio->preemptible_tcs);
+
+ return 0;
+
+err_reset_tc:
+ ocelot_port_reset_mqprio(ocelot, port);
+ return err;
+}
+EXPORT_SYMBOL_GPL(ocelot_port_mqprio);
+
void ocelot_init_port(struct ocelot *ocelot, int port)
{
struct ocelot_port *ocelot_port = ocelot->ports[port];
diff --git a/drivers/net/ethernet/mscc/ocelot.h b/drivers/net/ethernet/mscc/ocelot.h
index e9a0179..87f2055 100644
--- a/drivers/net/ethernet/mscc/ocelot.h
+++ b/drivers/net/ethernet/mscc/ocelot.h
@@ -74,6 +74,15 @@ struct ocelot_multicast {
struct ocelot_pgid *pgid;
};
+static inline void ocelot_reg_to_target_addr(struct ocelot *ocelot,
+ enum ocelot_reg reg,
+ enum ocelot_target *target,
+ u32 *addr)
+{
+ *target = reg >> TARGET_OFFSET;
+ *addr = ocelot->map[*target][reg & REG_MASK];
+}
+
int ocelot_bridge_num_find(struct ocelot *ocelot,
const struct net_device *bridge);
@@ -85,9 +94,6 @@ int ocelot_mact_forget(struct ocelot *ocelot,
struct net_device *ocelot_port_to_netdev(struct ocelot *ocelot, int port);
int ocelot_netdev_to_port(struct net_device *dev);
-u32 ocelot_port_readl(struct ocelot_port *port, u32 reg);
-void ocelot_port_writel(struct ocelot_port *port, u32 val, u32 reg);
-
int ocelot_probe_port(struct ocelot *ocelot, int port, struct regmap *target,
struct device_node *portnp);
void ocelot_release_port(struct ocelot_port *ocelot_port);
@@ -110,6 +116,9 @@ int ocelot_stats_init(struct ocelot *ocelot);
void ocelot_stats_deinit(struct ocelot *ocelot);
int ocelot_mm_init(struct ocelot *ocelot);
+void ocelot_port_change_fp(struct ocelot *ocelot, int port,
+ unsigned long preemptible_tcs);
+void ocelot_port_update_active_preemptible_tcs(struct ocelot *ocelot, int port);
extern struct notifier_block ocelot_netdevice_nb;
extern struct notifier_block ocelot_switchdev_nb;
diff --git a/drivers/net/ethernet/mscc/ocelot_io.c b/drivers/net/ethernet/mscc/ocelot_io.c
index 2067382..3aa7dc2 100644
--- a/drivers/net/ethernet/mscc/ocelot_io.c
+++ b/drivers/net/ethernet/mscc/ocelot_io.c
@@ -10,57 +10,60 @@
#include "ocelot.h"
-int __ocelot_bulk_read_ix(struct ocelot *ocelot, u32 reg, u32 offset, void *buf,
- int count)
+int __ocelot_bulk_read_ix(struct ocelot *ocelot, enum ocelot_reg reg,
+ u32 offset, void *buf, int count)
{
- u16 target = reg >> TARGET_OFFSET;
+ enum ocelot_target target;
+ u32 addr;
+ ocelot_reg_to_target_addr(ocelot, reg, &target, &addr);
WARN_ON(!target);
- return regmap_bulk_read(ocelot->targets[target],
- ocelot->map[target][reg & REG_MASK] + offset,
+ return regmap_bulk_read(ocelot->targets[target], addr + offset,
buf, count);
}
EXPORT_SYMBOL_GPL(__ocelot_bulk_read_ix);
-u32 __ocelot_read_ix(struct ocelot *ocelot, u32 reg, u32 offset)
+u32 __ocelot_read_ix(struct ocelot *ocelot, enum ocelot_reg reg, u32 offset)
{
- u16 target = reg >> TARGET_OFFSET;
- u32 val;
+ enum ocelot_target target;
+ u32 addr, val;
+ ocelot_reg_to_target_addr(ocelot, reg, &target, &addr);
WARN_ON(!target);
- regmap_read(ocelot->targets[target],
- ocelot->map[target][reg & REG_MASK] + offset, &val);
+ regmap_read(ocelot->targets[target], addr + offset, &val);
return val;
}
EXPORT_SYMBOL_GPL(__ocelot_read_ix);
-void __ocelot_write_ix(struct ocelot *ocelot, u32 val, u32 reg, u32 offset)
+void __ocelot_write_ix(struct ocelot *ocelot, u32 val, enum ocelot_reg reg,
+ u32 offset)
{
- u16 target = reg >> TARGET_OFFSET;
+ enum ocelot_target target;
+ u32 addr;
+ ocelot_reg_to_target_addr(ocelot, reg, &target, &addr);
WARN_ON(!target);
- regmap_write(ocelot->targets[target],
- ocelot->map[target][reg & REG_MASK] + offset, val);
+ regmap_write(ocelot->targets[target], addr + offset, val);
}
EXPORT_SYMBOL_GPL(__ocelot_write_ix);
-void __ocelot_rmw_ix(struct ocelot *ocelot, u32 val, u32 mask, u32 reg,
- u32 offset)
+void __ocelot_rmw_ix(struct ocelot *ocelot, u32 val, u32 mask,
+ enum ocelot_reg reg, u32 offset)
{
- u16 target = reg >> TARGET_OFFSET;
+ enum ocelot_target target;
+ u32 addr;
+ ocelot_reg_to_target_addr(ocelot, reg, &target, &addr);
WARN_ON(!target);
- regmap_update_bits(ocelot->targets[target],
- ocelot->map[target][reg & REG_MASK] + offset,
- mask, val);
+ regmap_update_bits(ocelot->targets[target], addr + offset, mask, val);
}
EXPORT_SYMBOL_GPL(__ocelot_rmw_ix);
-u32 ocelot_port_readl(struct ocelot_port *port, u32 reg)
+u32 ocelot_port_readl(struct ocelot_port *port, enum ocelot_reg reg)
{
struct ocelot *ocelot = port->ocelot;
u16 target = reg >> TARGET_OFFSET;
@@ -73,7 +76,7 @@ u32 ocelot_port_readl(struct ocelot_port *port, u32 reg)
}
EXPORT_SYMBOL_GPL(ocelot_port_readl);
-void ocelot_port_writel(struct ocelot_port *port, u32 val, u32 reg)
+void ocelot_port_writel(struct ocelot_port *port, u32 val, enum ocelot_reg reg)
{
struct ocelot *ocelot = port->ocelot;
u16 target = reg >> TARGET_OFFSET;
@@ -84,7 +87,8 @@ void ocelot_port_writel(struct ocelot_port *port, u32 val, u32 reg)
}
EXPORT_SYMBOL_GPL(ocelot_port_writel);
-void ocelot_port_rmwl(struct ocelot_port *port, u32 val, u32 mask, u32 reg)
+void ocelot_port_rmwl(struct ocelot_port *port, u32 val, u32 mask,
+ enum ocelot_reg reg)
{
u32 cur = ocelot_port_readl(port, reg);
diff --git a/drivers/net/ethernet/mscc/ocelot_mm.c b/drivers/net/ethernet/mscc/ocelot_mm.c
index 0a8f21a..fb31451 100644
--- a/drivers/net/ethernet/mscc/ocelot_mm.c
+++ b/drivers/net/ethernet/mscc/ocelot_mm.c
@@ -49,14 +49,68 @@ static enum ethtool_mm_verify_status ocelot_mm_verify_status(u32 val)
}
}
-void ocelot_port_mm_irq(struct ocelot *ocelot, int port)
+void ocelot_port_update_active_preemptible_tcs(struct ocelot *ocelot, int port)
+{
+ struct ocelot_port *ocelot_port = ocelot->ports[port];
+ struct ocelot_mm_state *mm = &ocelot->mm[port];
+ u32 val = 0;
+
+ lockdep_assert_held(&ocelot->fwd_domain_lock);
+
+ /* Only commit preemptible TCs when MAC Merge is active.
+ * On NXP LS1028A, when using QSGMII, the port hangs if transmitting
+ * preemptible frames at any other link speed than gigabit, so avoid
+ * preemption at lower speeds in this PHY mode.
+ */
+ if ((ocelot_port->phy_mode != PHY_INTERFACE_MODE_QSGMII ||
+ ocelot_port->speed == SPEED_1000) && mm->tx_active)
+ val = mm->preemptible_tcs;
+
+ /* Cut through switching doesn't work for preemptible priorities,
+ * so first make sure it is disabled.
+ */
+ mm->active_preemptible_tcs = val;
+ ocelot->ops->cut_through_fwd(ocelot);
+
+ dev_dbg(ocelot->dev,
+ "port %d %s/%s, MM TX %s, preemptible TCs 0x%x, active 0x%x\n",
+ port, phy_modes(ocelot_port->phy_mode),
+ phy_speed_to_str(ocelot_port->speed),
+ mm->tx_active ? "active" : "inactive", mm->preemptible_tcs,
+ mm->active_preemptible_tcs);
+
+ ocelot_rmw_rix(ocelot, QSYS_PREEMPTION_CFG_P_QUEUES(val),
+ QSYS_PREEMPTION_CFG_P_QUEUES_M,
+ QSYS_PREEMPTION_CFG, port);
+}
+
+void ocelot_port_change_fp(struct ocelot *ocelot, int port,
+ unsigned long preemptible_tcs)
+{
+ struct ocelot_mm_state *mm = &ocelot->mm[port];
+
+ mutex_lock(&ocelot->fwd_domain_lock);
+
+ if (mm->preemptible_tcs == preemptible_tcs)
+ goto out_unlock;
+
+ mm->preemptible_tcs = preemptible_tcs;
+
+ ocelot_port_update_active_preemptible_tcs(ocelot, port);
+
+out_unlock:
+ mutex_unlock(&ocelot->fwd_domain_lock);
+}
+
+static void ocelot_mm_update_port_status(struct ocelot *ocelot, int port)
{
struct ocelot_port *ocelot_port = ocelot->ports[port];
struct ocelot_mm_state *mm = &ocelot->mm[port];
enum ethtool_mm_verify_status verify_status;
- u32 val;
+ u32 val, ack = 0;
- mutex_lock(&mm->lock);
+ if (!mm->tx_enabled)
+ return;
val = ocelot_port_readl(ocelot_port, DEV_MM_STATUS);
@@ -73,25 +127,43 @@ void ocelot_port_mm_irq(struct ocelot *ocelot, int port)
dev_dbg(ocelot->dev, "Port %d TX preemption %s\n",
port, mm->tx_active ? "active" : "inactive");
+ ocelot_port_update_active_preemptible_tcs(ocelot, port);
+
+ ack |= DEV_MM_STAT_MM_STATUS_PRMPT_ACTIVE_STICKY;
}
if (val & DEV_MM_STAT_MM_STATUS_UNEXP_RX_PFRM_STICKY) {
dev_err(ocelot->dev,
"Unexpected P-frame received on port %d while verification was unsuccessful or not yet verified\n",
port);
+
+ ack |= DEV_MM_STAT_MM_STATUS_UNEXP_RX_PFRM_STICKY;
}
if (val & DEV_MM_STAT_MM_STATUS_UNEXP_TX_PFRM_STICKY) {
dev_err(ocelot->dev,
"Unexpected P-frame requested to be transmitted on port %d while verification was unsuccessful or not yet verified, or MM_TX_ENA=0\n",
port);
+
+ ack |= DEV_MM_STAT_MM_STATUS_UNEXP_TX_PFRM_STICKY;
}
- ocelot_port_writel(ocelot_port, val, DEV_MM_STATUS);
-
- mutex_unlock(&mm->lock);
+ if (ack)
+ ocelot_port_writel(ocelot_port, ack, DEV_MM_STATUS);
}
-EXPORT_SYMBOL_GPL(ocelot_port_mm_irq);
+
+void ocelot_mm_irq(struct ocelot *ocelot)
+{
+ int port;
+
+ mutex_lock(&ocelot->fwd_domain_lock);
+
+ for (port = 0; port < ocelot->num_phys_ports; port++)
+ ocelot_mm_update_port_status(ocelot, port);
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
+}
+EXPORT_SYMBOL_GPL(ocelot_mm_irq);
int ocelot_port_set_mm(struct ocelot *ocelot, int port,
struct ethtool_mm_cfg *cfg,
@@ -121,7 +193,7 @@ int ocelot_port_set_mm(struct ocelot *ocelot, int port,
if (!cfg->verify_enabled)
verify_disable = DEV_MM_CONFIG_VERIF_CONFIG_PRM_VERIFY_DIS;
- mutex_lock(&mm->lock);
+ mutex_lock(&ocelot->fwd_domain_lock);
ocelot_port_rmwl(ocelot_port, mm_enable,
DEV_MM_CONFIG_ENABLE_CONFIG_MM_TX_ENA |
@@ -140,7 +212,20 @@ int ocelot_port_set_mm(struct ocelot *ocelot, int port,
QSYS_PREEMPTION_CFG,
port);
- mutex_unlock(&mm->lock);
+ /* The switch will emit an IRQ when TX is disabled, to notify that it
+ * has become inactive. We optimize ocelot_mm_update_port_status() to
+ * not bother processing MM IRQs at all for ports with TX disabled,
+ * but we need to ACK this IRQ now, while mm->tx_enabled is still set,
+ * otherwise we get an IRQ storm.
+ */
+ if (mm->tx_enabled && !cfg->tx_enabled) {
+ ocelot_mm_update_port_status(ocelot, port);
+ WARN_ON(mm->tx_active);
+ }
+
+ mm->tx_enabled = cfg->tx_enabled;
+
+ mutex_unlock(&ocelot->fwd_domain_lock);
return 0;
}
@@ -158,7 +243,7 @@ int ocelot_port_get_mm(struct ocelot *ocelot, int port,
mm = &ocelot->mm[port];
- mutex_lock(&mm->lock);
+ mutex_lock(&ocelot->fwd_domain_lock);
val = ocelot_port_readl(ocelot_port, DEV_MM_ENABLE_CONFIG);
state->pmac_enabled = !!(val & DEV_MM_CONFIG_ENABLE_CONFIG_MM_RX_ENA);
@@ -174,10 +259,11 @@ int ocelot_port_get_mm(struct ocelot *ocelot, int port,
state->tx_min_frag_size = ethtool_mm_frag_size_add_to_min(add_frag_size);
state->rx_min_frag_size = ETH_ZLEN;
+ ocelot_mm_update_port_status(ocelot, port);
state->verify_status = mm->verify_status;
state->tx_active = mm->tx_active;
- mutex_unlock(&mm->lock);
+ mutex_unlock(&ocelot->fwd_domain_lock);
return 0;
}
@@ -201,7 +287,6 @@ int ocelot_mm_init(struct ocelot *ocelot)
u32 val;
mm = &ocelot->mm[port];
- mutex_init(&mm->lock);
ocelot_port = ocelot->ports[port];
/* Update initial status variable for the
diff --git a/drivers/net/ethernet/mscc/ocelot_stats.c b/drivers/net/ethernet/mscc/ocelot_stats.c
index d0e6cd8..5c55197c7 100644
--- a/drivers/net/ethernet/mscc/ocelot_stats.c
+++ b/drivers/net/ethernet/mscc/ocelot_stats.c
@@ -145,7 +145,7 @@ enum ocelot_stat {
};
struct ocelot_stat_layout {
- u32 reg;
+ enum ocelot_reg reg;
char name[ETH_GSTRING_LEN];
};
@@ -257,7 +257,7 @@ struct ocelot_stat_layout {
struct ocelot_stats_region {
struct list_head node;
- u32 base;
+ enum ocelot_reg base;
enum ocelot_stat first_stat;
int count;
u32 *buf;
@@ -395,7 +395,7 @@ static void ocelot_check_stats_work(struct work_struct *work)
void ocelot_get_strings(struct ocelot *ocelot, int port, u32 sset, u8 *data)
{
const struct ocelot_stat_layout *layout;
- int i;
+ enum ocelot_stat i;
if (sset != ETH_SS_STATS)
return;
@@ -442,7 +442,8 @@ static void ocelot_port_stats_run(struct ocelot *ocelot, int port, void *priv,
int ocelot_get_sset_count(struct ocelot *ocelot, int port, int sset)
{
const struct ocelot_stat_layout *layout;
- int i, num_stats = 0;
+ enum ocelot_stat i;
+ int num_stats = 0;
if (sset != ETH_SS_STATS)
return -EOPNOTSUPP;
@@ -461,8 +462,8 @@ static void ocelot_port_ethtool_stats_cb(struct ocelot *ocelot, int port,
void *priv)
{
const struct ocelot_stat_layout *layout;
+ enum ocelot_stat i;
u64 *data = priv;
- int i;
layout = ocelot_get_stats_layout(ocelot);
@@ -889,8 +890,8 @@ static int ocelot_prepare_stats_regions(struct ocelot *ocelot)
{
struct ocelot_stats_region *region = NULL;
const struct ocelot_stat_layout *layout;
- unsigned int last = 0;
- int i;
+ enum ocelot_reg last = 0;
+ enum ocelot_stat i;
INIT_LIST_HEAD(&ocelot->stats_regions);
@@ -900,6 +901,17 @@ static int ocelot_prepare_stats_regions(struct ocelot *ocelot)
if (!layout[i].reg)
continue;
+ /* enum ocelot_stat must be kept sorted in the same order
+ * as the addresses behind layout[i].reg in order to have
+ * efficient bulking
+ */
+ if (last) {
+ WARN(ocelot->map[SYS][last & REG_MASK] >= ocelot->map[SYS][layout[i].reg & REG_MASK],
+ "reg 0x%x had address 0x%x but reg 0x%x has address 0x%x, bulking broken!",
+ last, ocelot->map[SYS][last & REG_MASK],
+ layout[i].reg, ocelot->map[SYS][layout[i].reg & REG_MASK]);
+ }
+
if (region && ocelot->map[SYS][layout[i].reg & REG_MASK] ==
ocelot->map[SYS][last & REG_MASK] + 4) {
region->count++;
@@ -909,12 +921,6 @@ static int ocelot_prepare_stats_regions(struct ocelot *ocelot)
if (!region)
return -ENOMEM;
- /* enum ocelot_stat must be kept sorted in the same
- * order as layout[i].reg in order to have efficient
- * bulking
- */
- WARN_ON(last >= layout[i].reg);
-
region->base = layout[i].reg;
region->first_stat = i;
region->count = 1;
@@ -925,6 +931,15 @@ static int ocelot_prepare_stats_regions(struct ocelot *ocelot)
}
list_for_each_entry(region, &ocelot->stats_regions, node) {
+ enum ocelot_target target;
+ u32 addr;
+
+ ocelot_reg_to_target_addr(ocelot, region->base, &target,
+ &addr);
+
+ dev_dbg(ocelot->dev,
+ "region of %d contiguous counters starting with SYS:STAT:CNT[0x%03x]\n",
+ region->count, addr / 4);
region->buf = devm_kcalloc(ocelot->dev, region->count,
sizeof(*region->buf), GFP_KERNEL);
if (!region->buf)
@@ -972,4 +987,3 @@ void ocelot_stats_deinit(struct ocelot *ocelot)
cancel_delayed_work(&ocelot->stats_work);
destroy_workqueue(ocelot->stats_queue);
}
-
diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index 9f8357b..a7e376e 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -30,6 +30,7 @@
#include <linux/ipv6.h>
#include <asm/unaligned.h>
#include <net/ip6_checksum.h>
+#include <net/netdev_queues.h>
#include "r8169.h"
#include "r8169_firmware.h"
@@ -68,6 +69,8 @@
#define NUM_RX_DESC 256 /* Number of Rx descriptor registers */
#define R8169_TX_RING_BYTES (NUM_TX_DESC * sizeof(struct TxDesc))
#define R8169_RX_RING_BYTES (NUM_RX_DESC * sizeof(struct RxDesc))
+#define R8169_TX_STOP_THRS (MAX_SKB_FRAGS + 1)
+#define R8169_TX_START_THRS (2 * R8169_TX_STOP_THRS)
#define OCP_STD_PHY_BASE 0xa400
@@ -4162,13 +4165,9 @@ static bool rtl8169_tso_csum_v2(struct rtl8169_private *tp,
return true;
}
-static bool rtl_tx_slots_avail(struct rtl8169_private *tp)
+static unsigned int rtl_tx_slots_avail(struct rtl8169_private *tp)
{
- unsigned int slots_avail = READ_ONCE(tp->dirty_tx) + NUM_TX_DESC
- - READ_ONCE(tp->cur_tx);
-
- /* A skbuff with nr_frags needs nr_frags+1 entries in the tx queue */
- return slots_avail > MAX_SKB_FRAGS;
+ return READ_ONCE(tp->dirty_tx) + NUM_TX_DESC - READ_ONCE(tp->cur_tx);
}
/* Versions RTL8102e and from RTL8168c onwards support csum_v2 */
@@ -4245,27 +4244,10 @@ static netdev_tx_t rtl8169_start_xmit(struct sk_buff *skb,
WRITE_ONCE(tp->cur_tx, tp->cur_tx + frags + 1);
- stop_queue = !rtl_tx_slots_avail(tp);
- if (unlikely(stop_queue)) {
- /* Avoid wrongly optimistic queue wake-up: rtl_tx thread must
- * not miss a ring update when it notices a stopped queue.
- */
- smp_wmb();
- netif_stop_queue(dev);
- /* Sync with rtl_tx:
- * - publish queue status and cur_tx ring index (write barrier)
- * - refresh dirty_tx ring index (read barrier).
- * May the current thread have a pessimistic view of the ring
- * status and forget to wake up queue, a racing rtl_tx thread
- * can't.
- */
- smp_mb__after_atomic();
- if (rtl_tx_slots_avail(tp))
- netif_start_queue(dev);
- door_bell = true;
- }
-
- if (door_bell)
+ stop_queue = !netif_subqueue_maybe_stop(dev, 0, rtl_tx_slots_avail(tp),
+ R8169_TX_STOP_THRS,
+ R8169_TX_START_THRS);
+ if (door_bell || stop_queue)
rtl8169_doorbell(tp);
return NETDEV_TX_OK;
@@ -4389,19 +4371,12 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp,
}
if (tp->dirty_tx != dirty_tx) {
- netdev_completed_queue(dev, pkts_compl, bytes_compl);
dev_sw_netstats_tx_add(dev, pkts_compl, bytes_compl);
+ WRITE_ONCE(tp->dirty_tx, dirty_tx);
- /* Sync with rtl8169_start_xmit:
- * - publish dirty_tx ring index (write barrier)
- * - refresh cur_tx ring index and queue status (read barrier)
- * May the current thread miss the stopped queue condition,
- * a racing xmit thread can only have a right view of the
- * ring status.
- */
- smp_store_mb(tp->dirty_tx, dirty_tx);
- if (netif_queue_stopped(dev) && rtl_tx_slots_avail(tp))
- netif_wake_queue(dev);
+ netif_subqueue_completed_wake(dev, 0, pkts_compl, bytes_compl,
+ rtl_tx_slots_avail(tp),
+ R8169_TX_START_THRS);
/*
* 8168 hack: TxPoll requests are lost when the Tx packets are
* too close. Let's kick an extra TxPoll request when a burst
diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c
index 746fd91..a4f22d8 100644
--- a/drivers/net/ethernet/sfc/efx.c
+++ b/drivers/net/ethernet/sfc/efx.c
@@ -540,7 +540,6 @@ int efx_net_open(struct net_device *net_dev)
else
efx->state = STATE_NET_UP;
- efx_selftest_async_start(efx);
return 0;
}
diff --git a/drivers/net/ethernet/sfc/efx_common.c b/drivers/net/ethernet/sfc/efx_common.c
index cc30524..361687d 100644
--- a/drivers/net/ethernet/sfc/efx_common.c
+++ b/drivers/net/ethernet/sfc/efx_common.c
@@ -544,6 +544,8 @@ void efx_start_all(struct efx_nic *efx)
/* Start the hardware monitor if there is one */
efx_start_monitor(efx);
+ efx_selftest_async_start(efx);
+
/* Link state detection is normally event-driven; we have
* to poll now because we could have missed a change
*/
diff --git a/drivers/net/ethernet/stmicro/stmmac/Kconfig b/drivers/net/ethernet/stmicro/stmmac/Kconfig
index f77511f..5f5a997 100644
--- a/drivers/net/ethernet/stmicro/stmmac/Kconfig
+++ b/drivers/net/ethernet/stmicro/stmmac/Kconfig
@@ -165,6 +165,18 @@
for the stmmac device driver. This driver is used for
arria5 and cyclone5 FPGA SoCs.
+config DWMAC_STARFIVE
+ tristate "StarFive dwmac support"
+ depends on OF && (ARCH_STARFIVE || COMPILE_TEST)
+ select MFD_SYSCON
+ default m if ARCH_STARFIVE
+ help
+ Support for ethernet controllers on StarFive RISC-V SoCs
+
+ This selects the StarFive platform specific glue layer support for
+ the stmmac device driver. This driver is used for StarFive JH7110
+ ethernet controller.
+
config DWMAC_STI
tristate "STi GMAC support"
default ARCH_STI
diff --git a/drivers/net/ethernet/stmicro/stmmac/Makefile b/drivers/net/ethernet/stmicro/stmmac/Makefile
index 057e4ba..8738fdb 100644
--- a/drivers/net/ethernet/stmicro/stmmac/Makefile
+++ b/drivers/net/ethernet/stmicro/stmmac/Makefile
@@ -23,6 +23,7 @@
obj-$(CONFIG_DWMAC_QCOM_ETHQOS) += dwmac-qcom-ethqos.o
obj-$(CONFIG_DWMAC_ROCKCHIP) += dwmac-rk.o
obj-$(CONFIG_DWMAC_SOCFPGA) += dwmac-altr-socfpga.o
+obj-$(CONFIG_DWMAC_STARFIVE) += dwmac-starfive.o
obj-$(CONFIG_DWMAC_STI) += dwmac-sti.o
obj-$(CONFIG_DWMAC_STM32) += dwmac-stm32.o
obj-$(CONFIG_DWMAC_SUNXI) += dwmac-sunxi.o
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
index e8b507f..f6754e3 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
@@ -263,6 +263,11 @@ static int meson_axg_set_phy_mode(struct meson8b_dwmac *dwmac)
return 0;
}
+static void meson8b_clk_disable_unprepare(void *data)
+{
+ clk_disable_unprepare(data);
+}
+
static int meson8b_devm_clk_prepare_enable(struct meson8b_dwmac *dwmac,
struct clk *clk)
{
@@ -273,8 +278,7 @@ static int meson8b_devm_clk_prepare_enable(struct meson8b_dwmac *dwmac,
return ret;
return devm_add_action_or_reset(dwmac->dev,
- (void(*)(void *))clk_disable_unprepare,
- clk);
+ meson8b_clk_disable_unprepare, clk);
}
static int meson8b_init_rgmii_delays(struct meson8b_dwmac *dwmac)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c
new file mode 100644
index 0000000..4f51a78
--- /dev/null
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c
@@ -0,0 +1,171 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * StarFive DWMAC platform driver
+ *
+ * Copyright (C) 2021 Emil Renner Berthing <kernel@esmil.dk>
+ * Copyright (C) 2022 StarFive Technology Co., Ltd.
+ *
+ */
+
+#include <linux/mfd/syscon.h>
+#include <linux/of_device.h>
+#include <linux/regmap.h>
+
+#include "stmmac_platform.h"
+
+#define STARFIVE_DWMAC_PHY_INFT_RGMII 0x1
+#define STARFIVE_DWMAC_PHY_INFT_RMII 0x4
+#define STARFIVE_DWMAC_PHY_INFT_FIELD 0x7U
+
+struct starfive_dwmac {
+ struct device *dev;
+ struct clk *clk_tx;
+};
+
+static void starfive_dwmac_fix_mac_speed(void *priv, unsigned int speed)
+{
+ struct starfive_dwmac *dwmac = priv;
+ unsigned long rate;
+ int err;
+
+ rate = clk_get_rate(dwmac->clk_tx);
+
+ switch (speed) {
+ case SPEED_1000:
+ rate = 125000000;
+ break;
+ case SPEED_100:
+ rate = 25000000;
+ break;
+ case SPEED_10:
+ rate = 2500000;
+ break;
+ default:
+ dev_err(dwmac->dev, "invalid speed %u\n", speed);
+ break;
+ }
+
+ err = clk_set_rate(dwmac->clk_tx, rate);
+ if (err)
+ dev_err(dwmac->dev, "failed to set tx rate %lu\n", rate);
+}
+
+static int starfive_dwmac_set_mode(struct plat_stmmacenet_data *plat_dat)
+{
+ struct starfive_dwmac *dwmac = plat_dat->bsp_priv;
+ struct regmap *regmap;
+ unsigned int args[2];
+ unsigned int mode;
+ int err;
+
+ switch (plat_dat->interface) {
+ case PHY_INTERFACE_MODE_RMII:
+ mode = STARFIVE_DWMAC_PHY_INFT_RMII;
+ break;
+
+ case PHY_INTERFACE_MODE_RGMII:
+ case PHY_INTERFACE_MODE_RGMII_ID:
+ mode = STARFIVE_DWMAC_PHY_INFT_RGMII;
+ break;
+
+ default:
+ dev_err(dwmac->dev, "unsupported interface %d\n",
+ plat_dat->interface);
+ return -EINVAL;
+ }
+
+ regmap = syscon_regmap_lookup_by_phandle_args(dwmac->dev->of_node,
+ "starfive,syscon",
+ 2, args);
+ if (IS_ERR(regmap))
+ return dev_err_probe(dwmac->dev, PTR_ERR(regmap), "getting the regmap failed\n");
+
+ /* args[0]:offset args[1]: shift */
+ err = regmap_update_bits(regmap, args[0],
+ STARFIVE_DWMAC_PHY_INFT_FIELD << args[1],
+ mode << args[1]);
+ if (err)
+ return dev_err_probe(dwmac->dev, err, "error setting phy mode\n");
+
+ return 0;
+}
+
+static int starfive_dwmac_probe(struct platform_device *pdev)
+{
+ struct plat_stmmacenet_data *plat_dat;
+ struct stmmac_resources stmmac_res;
+ struct starfive_dwmac *dwmac;
+ struct clk *clk_gtx;
+ int err;
+
+ err = stmmac_get_platform_resources(pdev, &stmmac_res);
+ if (err)
+ return dev_err_probe(&pdev->dev, err,
+ "failed to get resources\n");
+
+ plat_dat = stmmac_probe_config_dt(pdev, stmmac_res.mac);
+ if (IS_ERR(plat_dat))
+ return dev_err_probe(&pdev->dev, PTR_ERR(plat_dat),
+ "dt configuration failed\n");
+
+ dwmac = devm_kzalloc(&pdev->dev, sizeof(*dwmac), GFP_KERNEL);
+ if (!dwmac)
+ return -ENOMEM;
+
+ dwmac->clk_tx = devm_clk_get_enabled(&pdev->dev, "tx");
+ if (IS_ERR(dwmac->clk_tx))
+ return dev_err_probe(&pdev->dev, PTR_ERR(dwmac->clk_tx),
+ "error getting tx clock\n");
+
+ clk_gtx = devm_clk_get_enabled(&pdev->dev, "gtx");
+ if (IS_ERR(clk_gtx))
+ return dev_err_probe(&pdev->dev, PTR_ERR(clk_gtx),
+ "error getting gtx clock\n");
+
+ /* Generally, the rgmii_tx clock is provided by the internal clock,
+ * which needs to match the corresponding clock frequency according
+ * to different speeds. If the rgmii_tx clock is provided by the
+ * external rgmii_rxin, there is no need to configure the clock
+ * internally, because rgmii_rxin will be adaptively adjusted.
+ */
+ if (!device_property_read_bool(&pdev->dev, "starfive,tx-use-rgmii-clk"))
+ plat_dat->fix_mac_speed = starfive_dwmac_fix_mac_speed;
+
+ dwmac->dev = &pdev->dev;
+ plat_dat->bsp_priv = dwmac;
+ plat_dat->dma_cfg->dche = true;
+
+ err = starfive_dwmac_set_mode(plat_dat);
+ if (err)
+ return err;
+
+ err = stmmac_dvr_probe(&pdev->dev, plat_dat, &stmmac_res);
+ if (err) {
+ stmmac_remove_config_dt(pdev, plat_dat);
+ return err;
+ }
+
+ return 0;
+}
+
+static const struct of_device_id starfive_dwmac_match[] = {
+ { .compatible = "starfive,jh7110-dwmac" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, starfive_dwmac_match);
+
+static struct platform_driver starfive_dwmac_driver = {
+ .probe = starfive_dwmac_probe,
+ .remove = stmmac_pltfr_remove,
+ .driver = {
+ .name = "starfive-dwmac",
+ .pm = &stmmac_pltfr_pm_ops,
+ .of_match_table = starfive_dwmac_match,
+ },
+};
+module_platform_driver(starfive_dwmac_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("StarFive DWMAC platform driver");
+MODULE_AUTHOR("Emil Renner Berthing <kernel@esmil.dk>");
+MODULE_AUTHOR("Samin Guo <samin.guo@starfivetech.com>");
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-sti.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-sti.c
index be3b1eb..465ce66 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-sti.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-sti.c
@@ -35,7 +35,7 @@
#define IS_PHY_IF_MODE_GBIT(iface) (IS_PHY_IF_MODE_RGMII(iface) || \
iface == PHY_INTERFACE_MODE_GMII)
-/* STiH4xx register definitions (STiH415/STiH416/STiH407/STiH410 families)
+/* STiH4xx register definitions (STiH407/STiH410 families)
*
* Below table summarizes the clock requirement and clock sources for
* supported phy interface modes with link speeds.
@@ -75,27 +75,6 @@
#define STIH4XX_ETH_SEL_INTERNAL_NOTEXT_PHYCLK BIT(7)
#define STIH4XX_ETH_SEL_TXCLK_NOT_CLK125 BIT(6)
-/* STiD127 register definitions
- *-----------------------
- * src |BIT(6)| BIT(7)|
- *-----------------------
- * MII | 1 | n/a |
- *-----------------------
- * RMII | n/a | 1 |
- * clkgen| | |
- *-----------------------
- * RMII | n/a | 0 |
- * phyclk| | |
- *-----------------------
- * RGMII | 1 | n/a |
- * clkgen| | |
- *-----------------------
- */
-
-#define STID127_RETIME_SRC_MASK GENMASK(7, 6)
-#define STID127_ETH_SEL_INTERNAL_NOTEXT_PHYCLK BIT(7)
-#define STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK BIT(6)
-
#define ENMII_MASK GENMASK(5, 5)
#define ENMII BIT(5)
#define EN_MASK GENMASK(1, 1)
@@ -194,36 +173,6 @@ static void stih4xx_fix_retime_src(void *priv, u32 spd)
stih4xx_tx_retime_val[src]);
}
-static void stid127_fix_retime_src(void *priv, u32 spd)
-{
- struct sti_dwmac *dwmac = priv;
- u32 reg = dwmac->ctrl_reg;
- u32 freq = 0;
- u32 val = 0;
-
- if (dwmac->interface == PHY_INTERFACE_MODE_MII) {
- val = STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK;
- } else if (dwmac->interface == PHY_INTERFACE_MODE_RMII) {
- if (!dwmac->ext_phyclk) {
- val = STID127_ETH_SEL_INTERNAL_NOTEXT_PHYCLK;
- freq = DWMAC_50MHZ;
- }
- } else if (IS_PHY_IF_MODE_RGMII(dwmac->interface)) {
- val = STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK;
- if (spd == SPEED_1000)
- freq = DWMAC_125MHZ;
- else if (spd == SPEED_100)
- freq = DWMAC_25MHZ;
- else if (spd == SPEED_10)
- freq = DWMAC_2_5MHZ;
- }
-
- if (freq)
- clk_set_rate(dwmac->clk, freq);
-
- regmap_update_bits(dwmac->regmap, reg, STID127_RETIME_SRC_MASK, val);
-}
-
static int sti_dwmac_set_mode(struct sti_dwmac *dwmac)
{
struct regmap *regmap = dwmac->regmap;
@@ -408,14 +357,7 @@ static const struct sti_dwmac_of_data stih4xx_dwmac_data = {
.fix_retime_src = stih4xx_fix_retime_src,
};
-static const struct sti_dwmac_of_data stid127_dwmac_data = {
- .fix_retime_src = stid127_fix_retime_src,
-};
-
static const struct of_device_id sti_dwmac_match[] = {
- { .compatible = "st,stih415-dwmac", .data = &stih4xx_dwmac_data},
- { .compatible = "st,stih416-dwmac", .data = &stih4xx_dwmac_data},
- { .compatible = "st,stid127-dwmac", .data = &stid127_dwmac_data},
{ .compatible = "st,stih407-dwmac", .data = &stih4xx_dwmac_data},
{ }
};
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
index 3d15e1e..07ea5ab 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
@@ -92,6 +92,13 @@ struct stmmac_rx_buffer {
dma_addr_t sec_addr;
};
+struct stmmac_xdp_buff {
+ struct xdp_buff xdp;
+ struct stmmac_priv *priv;
+ struct dma_desc *desc;
+ struct dma_desc *ndesc;
+};
+
struct stmmac_rx_queue {
u32 rx_count_frames;
u32 queue_index;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index d7fcab0..f116e4a 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -1614,6 +1614,12 @@ static int stmmac_alloc_rx_buffers_zc(struct stmmac_priv *priv,
struct stmmac_rx_queue *rx_q = &dma_conf->rx_queue[queue];
int i;
+ /* struct stmmac_xdp_buff is using cb field (maximum size of 24 bytes)
+ * in struct xdp_buff_xsk to stash driver specific information. Thus,
+ * use this macro to make sure no size violations.
+ */
+ XSK_CHECK_PRIV_TYPE(struct stmmac_xdp_buff);
+
for (i = 0; i < dma_conf->dma_rx_size; i++) {
struct stmmac_rx_buffer *buf;
dma_addr_t dma_addr;
@@ -4998,6 +5004,16 @@ static bool stmmac_rx_refill_zc(struct stmmac_priv *priv, u32 queue, u32 budget)
return ret;
}
+static struct stmmac_xdp_buff *xsk_buff_to_stmmac_ctx(struct xdp_buff *xdp)
+{
+ /* In XDP zero copy data path, xdp field in struct xdp_buff_xsk is used
+ * to represent incoming packet, whereas cb field in the same structure
+ * is used to store driver specific info. Thus, struct stmmac_xdp_buff
+ * is laid on top of xdp and cb fields of struct xdp_buff_xsk.
+ */
+ return (struct stmmac_xdp_buff *)xdp;
+}
+
static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue)
{
struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[queue];
@@ -5027,6 +5043,7 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue)
}
while (count < limit) {
struct stmmac_rx_buffer *buf;
+ struct stmmac_xdp_buff *ctx;
unsigned int buf1_len = 0;
struct dma_desc *np, *p;
int entry;
@@ -5112,6 +5129,11 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue)
goto read_again;
}
+ ctx = xsk_buff_to_stmmac_ctx(buf->xdp);
+ ctx->priv = priv;
+ ctx->desc = p;
+ ctx->ndesc = np;
+
/* XDP ZC Frame only support primary buffers for now */
buf1_len = stmmac_rx_buf1_len(priv, p, status, len);
len += buf1_len;
@@ -5190,7 +5212,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
enum dma_data_direction dma_dir;
unsigned int desc_size;
struct sk_buff *skb = NULL;
- struct xdp_buff xdp;
+ struct stmmac_xdp_buff ctx;
int xdp_status = 0;
int buf_sz;
@@ -5311,17 +5333,22 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
dma_sync_single_for_cpu(priv->device, buf->addr,
buf1_len, dma_dir);
- xdp_init_buff(&xdp, buf_sz, &rx_q->xdp_rxq);
- xdp_prepare_buff(&xdp, page_address(buf->page),
- buf->page_offset, buf1_len, false);
+ xdp_init_buff(&ctx.xdp, buf_sz, &rx_q->xdp_rxq);
+ xdp_prepare_buff(&ctx.xdp, page_address(buf->page),
+ buf->page_offset, buf1_len, true);
- pre_len = xdp.data_end - xdp.data_hard_start -
+ pre_len = ctx.xdp.data_end - ctx.xdp.data_hard_start -
buf->page_offset;
- skb = stmmac_xdp_run_prog(priv, &xdp);
+
+ ctx.priv = priv;
+ ctx.desc = p;
+ ctx.ndesc = np;
+
+ skb = stmmac_xdp_run_prog(priv, &ctx.xdp);
/* Due xdp_adjust_tail: DMA sync for_device
* cover max len CPU touch
*/
- sync_len = xdp.data_end - xdp.data_hard_start -
+ sync_len = ctx.xdp.data_end - ctx.xdp.data_hard_start -
buf->page_offset;
sync_len = max(sync_len, pre_len);
@@ -5331,7 +5358,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
if (xdp_res & STMMAC_XDP_CONSUMED) {
page_pool_put_page(rx_q->page_pool,
- virt_to_head_page(xdp.data),
+ virt_to_head_page(ctx.xdp.data),
sync_len, true);
buf->page = NULL;
priv->dev->stats.rx_dropped++;
@@ -5359,7 +5386,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
if (!skb) {
/* XDP program may expand or reduce tail */
- buf1_len = xdp.data_end - xdp.data;
+ buf1_len = ctx.xdp.data_end - ctx.xdp.data;
skb = napi_alloc_skb(&ch->rx_napi, buf1_len);
if (!skb) {
@@ -5369,7 +5396,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
}
/* XDP program may adjust header */
- skb_copy_to_linear_data(skb, xdp.data, buf1_len);
+ skb_copy_to_linear_data(skb, ctx.xdp.data, buf1_len);
skb_put(skb, buf1_len);
/* Data payload copied into SKB, page ready for recycle */
@@ -6350,6 +6377,10 @@ static int stmmac_vlan_rx_add_vid(struct net_device *ndev, __be16 proto, u16 vid
bool is_double = false;
int ret;
+ ret = pm_runtime_resume_and_get(priv->device);
+ if (ret < 0)
+ return ret;
+
if (be16_to_cpu(proto) == ETH_P_8021AD)
is_double = true;
@@ -6357,16 +6388,18 @@ static int stmmac_vlan_rx_add_vid(struct net_device *ndev, __be16 proto, u16 vid
ret = stmmac_vlan_update(priv, is_double);
if (ret) {
clear_bit(vid, priv->active_vlans);
- return ret;
+ goto err_pm_put;
}
if (priv->hw->num_vlan) {
ret = stmmac_add_hw_vlan_rx_fltr(priv, ndev, priv->hw, proto, vid);
if (ret)
- return ret;
+ goto err_pm_put;
}
+err_pm_put:
+ pm_runtime_put(priv->device);
- return 0;
+ return ret;
}
static int stmmac_vlan_rx_kill_vid(struct net_device *ndev, __be16 proto, u16 vid)
@@ -7060,6 +7093,37 @@ void stmmac_fpe_handshake(struct stmmac_priv *priv, bool enable)
}
}
+static int stmmac_xdp_rx_timestamp(const struct xdp_md *_ctx, u64 *timestamp)
+{
+ const struct stmmac_xdp_buff *ctx = (void *)_ctx;
+ struct dma_desc *desc_contains_ts = ctx->desc;
+ struct stmmac_priv *priv = ctx->priv;
+ struct dma_desc *ndesc = ctx->ndesc;
+ struct dma_desc *desc = ctx->desc;
+ u64 ns = 0;
+
+ if (!priv->hwts_rx_en)
+ return -ENODATA;
+
+ /* For GMAC4, the valid timestamp is from CTX next desc. */
+ if (priv->plat->has_gmac4 || priv->plat->has_xgmac)
+ desc_contains_ts = ndesc;
+
+ /* Check if timestamp is available */
+ if (stmmac_get_rx_timestamp_status(priv, desc, ndesc, priv->adv_ts)) {
+ stmmac_get_timestamp(priv, desc_contains_ts, priv->adv_ts, &ns);
+ ns -= priv->plat->cdc_error_adj;
+ *timestamp = ns_to_ktime(ns);
+ return 0;
+ }
+
+ return -ENODATA;
+}
+
+static const struct xdp_metadata_ops stmmac_xdp_metadata_ops = {
+ .xmo_rx_timestamp = stmmac_xdp_rx_timestamp,
+};
+
/**
* stmmac_dvr_probe
* @device: device pointer
@@ -7167,6 +7231,8 @@ int stmmac_dvr_probe(struct device *device,
ndev->netdev_ops = &stmmac_netdev_ops;
+ ndev->xdp_metadata_ops = &stmmac_xdp_metadata_ops;
+
ndev->hw_features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
NETIF_F_RXCSUM;
ndev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
@@ -7253,6 +7319,10 @@ int stmmac_dvr_probe(struct device *device,
if (priv->dma_cap.rssen && priv->plat->rss_en)
ndev->features |= NETIF_F_RXHASH;
+ ndev->vlan_features |= ndev->features;
+ /* TSO doesn't work on VLANs yet */
+ ndev->vlan_features &= ~NETIF_F_TSO;
+
/* MTU range: 46 - hw-specific max */
ndev->min_mtu = ETH_ZLEN - ETH_HLEN;
if (priv->plat->has_xgmac)
@@ -7275,6 +7345,8 @@ int stmmac_dvr_probe(struct device *device,
if (flow_ctrl)
priv->flow_ctrl = FLOW_AUTO; /* RX/TX pause on */
+ ndev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+
/* Setup channels NAPI */
stmmac_napi_add(ndev);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
index 067a40f..eb0b289 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
@@ -519,7 +519,8 @@ stmmac_probe_config_dt(struct platform_device *pdev, u8 *mac)
if (of_device_is_compatible(np, "snps,dwmac-4.00") ||
of_device_is_compatible(np, "snps,dwmac-4.10a") ||
of_device_is_compatible(np, "snps,dwmac-4.20a") ||
- of_device_is_compatible(np, "snps,dwmac-5.10a")) {
+ of_device_is_compatible(np, "snps,dwmac-5.10a") ||
+ of_device_is_compatible(np, "snps,dwmac-5.20")) {
plat->has_gmac4 = 1;
plat->has_gmac = 0;
plat->pmt = 1;
diff --git a/drivers/net/ethernet/wangxun/libwx/wx_lib.c b/drivers/net/ethernet/wangxun/libwx/wx_lib.c
index eb89a274..1e8d8b7 100644
--- a/drivers/net/ethernet/wangxun/libwx/wx_lib.c
+++ b/drivers/net/ethernet/wangxun/libwx/wx_lib.c
@@ -1798,10 +1798,13 @@ static int wx_setup_rx_resources(struct wx_ring *rx_ring)
ret = wx_alloc_page_pool(rx_ring);
if (ret < 0) {
dev_err(rx_ring->dev, "Page pool creation failed: %d\n", ret);
- goto err;
+ goto err_desc;
}
return 0;
+
+err_desc:
+ dma_free_coherent(dev, rx_ring->size, rx_ring->desc, rx_ring->dma);
err:
kvfree(rx_ring->rx_buffer_info);
rx_ring->rx_buffer_info = NULL;
diff --git a/drivers/net/hamradio/Kconfig b/drivers/net/hamradio/Kconfig
index a9c44f0..a94c7bd5d 100644
--- a/drivers/net/hamradio/Kconfig
+++ b/drivers/net/hamradio/Kconfig
@@ -47,7 +47,7 @@
config SCC
tristate "Z8530 SCC driver"
- depends on ISA && AX25 && ISA_DMA_API
+ depends on ISA && AX25
help
These cards are used to connect your Linux box to an amateur radio
in order to communicate with other computers. If you want to use
diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c
index 2561624..3427993 100644
--- a/drivers/net/macsec.c
+++ b/drivers/net/macsec.c
@@ -1021,8 +1021,12 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb)
* the SecTAG, so we have to deduce which port to deliver to.
*/
if (macsec_is_offloaded(macsec) && netif_running(ndev)) {
- if (md_dst && md_dst->type == METADATA_MACSEC &&
- (!find_rx_sc(&macsec->secy, md_dst->u.macsec_info.sci)))
+ struct macsec_rx_sc *rx_sc = NULL;
+
+ if (md_dst && md_dst->type == METADATA_MACSEC)
+ rx_sc = find_rx_sc(&macsec->secy, md_dst->u.macsec_info.sci);
+
+ if (md_dst && md_dst->type == METADATA_MACSEC && !rx_sc)
continue;
if (ether_addr_equal_64bits(hdr->h_dest,
@@ -1047,7 +1051,13 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb)
nskb->pkt_type = PACKET_MULTICAST;
__netif_rx(nskb);
+ } else if (rx_sc || ndev->flags & IFF_PROMISC) {
+ skb->dev = ndev;
+ skb->pkt_type = PACKET_HOST;
+ ret = RX_HANDLER_ANOTHER;
+ goto out;
}
+
continue;
}
diff --git a/drivers/net/phy/Kconfig b/drivers/net/phy/Kconfig
index 6b9525d..bcfba07 100644
--- a/drivers/net/phy/Kconfig
+++ b/drivers/net/phy/Kconfig
@@ -44,6 +44,14 @@
<Speed in megabits>Mbps OR <Speed in gigabits>Gbps OR link
for any speed known to the PHY.
+config PHYLIB_LEDS
+ bool "Support probing LEDs from device tree"
+ depends on LEDS_CLASS=y || LEDS_CLASS=PHYLIB
+ depends on OF
+ default y
+ help
+ When LED class support is enabled, phylib can automatically
+ probe LED setting from device tree.
config FIXED_PHY
tristate "MDIO Bus/PHY emulation with fixed speed/link PHYs"
@@ -265,6 +273,12 @@
help
Currently supports the DP83865 PHY.
+config NXP_CBTX_PHY
+ tristate "NXP 100BASE-TX PHYs"
+ help
+ Support the 100BASE-TX PHY integrated on the SJA1110 automotive
+ switch family.
+
config NXP_C45_TJA11XX_PHY
tristate "NXP C45 TJA11XX PHYs"
depends on PTP_1588_CLOCK_OPTIONAL
diff --git a/drivers/net/phy/Makefile b/drivers/net/phy/Makefile
index b513806..ae11bf2 100644
--- a/drivers/net/phy/Makefile
+++ b/drivers/net/phy/Makefile
@@ -79,6 +79,7 @@
obj-$(CONFIG_NATIONAL_PHY) += national.o
obj-$(CONFIG_NCN26000_PHY) += ncn26000.o
obj-$(CONFIG_NXP_C45_TJA11XX_PHY) += nxp-c45-tja11xx.o
+obj-$(CONFIG_NXP_CBTX_PHY) += nxp-cbtx.o
obj-$(CONFIG_NXP_TJA11XX_PHY) += nxp-tja11xx.o
obj-$(CONFIG_QSEMI_PHY) += qsemi.o
obj-$(CONFIG_REALTEK_PHY) += realtek.o
diff --git a/drivers/net/phy/marvell.c b/drivers/net/phy/marvell.c
index 24853e9a..cd5d0ed 100644
--- a/drivers/net/phy/marvell.c
+++ b/drivers/net/phy/marvell.c
@@ -144,11 +144,15 @@
/* WOL Event Interrupt Enable */
#define MII_88E1318S_PHY_CSIER_WOL_EIE BIT(7)
-/* LED Timer Control Register */
-#define MII_88E1318S_PHY_LED_TCR 0x12
-#define MII_88E1318S_PHY_LED_TCR_FORCE_INT BIT(15)
-#define MII_88E1318S_PHY_LED_TCR_INTn_ENABLE BIT(7)
-#define MII_88E1318S_PHY_LED_TCR_INT_ACTIVE_LOW BIT(11)
+#define MII_88E1318S_PHY_LED_FUNC 0x10
+#define MII_88E1318S_PHY_LED_FUNC_OFF (0x8)
+#define MII_88E1318S_PHY_LED_FUNC_ON (0x9)
+#define MII_88E1318S_PHY_LED_FUNC_HI_Z (0xa)
+#define MII_88E1318S_PHY_LED_FUNC_BLINK (0xb)
+#define MII_88E1318S_PHY_LED_TCR 0x12
+#define MII_88E1318S_PHY_LED_TCR_FORCE_INT BIT(15)
+#define MII_88E1318S_PHY_LED_TCR_INTn_ENABLE BIT(7)
+#define MII_88E1318S_PHY_LED_TCR_INT_ACTIVE_LOW BIT(11)
/* Magic Packet MAC address registers */
#define MII_88E1318S_PHY_MAGIC_PACKET_WORD2 0x17
@@ -2832,6 +2836,63 @@ static int marvell_hwmon_probe(struct phy_device *phydev)
}
#endif
+static int m88e1318_led_brightness_set(struct phy_device *phydev,
+ u8 index, enum led_brightness value)
+{
+ int reg;
+
+ reg = phy_read_paged(phydev, MII_MARVELL_LED_PAGE,
+ MII_88E1318S_PHY_LED_FUNC);
+ if (reg < 0)
+ return reg;
+
+ switch (index) {
+ case 0:
+ case 1:
+ case 2:
+ reg &= ~(0xf << (4 * index));
+ if (value == LED_OFF)
+ reg |= MII_88E1318S_PHY_LED_FUNC_OFF << (4 * index);
+ else
+ reg |= MII_88E1318S_PHY_LED_FUNC_ON << (4 * index);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return phy_write_paged(phydev, MII_MARVELL_LED_PAGE,
+ MII_88E1318S_PHY_LED_FUNC, reg);
+}
+
+static int m88e1318_led_blink_set(struct phy_device *phydev, u8 index,
+ unsigned long *delay_on,
+ unsigned long *delay_off)
+{
+ int reg;
+
+ reg = phy_read_paged(phydev, MII_MARVELL_LED_PAGE,
+ MII_88E1318S_PHY_LED_FUNC);
+ if (reg < 0)
+ return reg;
+
+ switch (index) {
+ case 0:
+ case 1:
+ case 2:
+ reg &= ~(0xf << (4 * index));
+ reg |= MII_88E1318S_PHY_LED_FUNC_BLINK << (4 * index);
+ /* Reset default is 84ms */
+ *delay_on = 84 / 2;
+ *delay_off = 84 / 2;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return phy_write_paged(phydev, MII_MARVELL_LED_PAGE,
+ MII_88E1318S_PHY_LED_FUNC, reg);
+}
+
static int marvell_probe(struct phy_device *phydev)
{
struct marvell_priv *priv;
@@ -3081,6 +3142,8 @@ static struct phy_driver marvell_drivers[] = {
.get_sset_count = marvell_get_sset_count,
.get_strings = marvell_get_strings,
.get_stats = marvell_get_stats,
+ .led_brightness_set = m88e1318_led_brightness_set,
+ .led_blink_set = m88e1318_led_blink_set,
},
{
.phy_id = MARVELL_PHY_ID_88E1145,
@@ -3187,6 +3250,8 @@ static struct phy_driver marvell_drivers[] = {
.cable_test_start = marvell_vct7_cable_test_start,
.cable_test_tdr_start = marvell_vct5_cable_test_tdr_start,
.cable_test_get_status = marvell_vct7_cable_test_get_status,
+ .led_brightness_set = m88e1318_led_brightness_set,
+ .led_blink_set = m88e1318_led_blink_set,
},
{
.phy_id = MARVELL_PHY_ID_88E1540,
@@ -3213,6 +3278,8 @@ static struct phy_driver marvell_drivers[] = {
.cable_test_start = marvell_vct7_cable_test_start,
.cable_test_tdr_start = marvell_vct5_cable_test_tdr_start,
.cable_test_get_status = marvell_vct7_cable_test_get_status,
+ .led_brightness_set = m88e1318_led_brightness_set,
+ .led_blink_set = m88e1318_led_blink_set,
},
{
.phy_id = MARVELL_PHY_ID_88E1545,
@@ -3239,6 +3306,8 @@ static struct phy_driver marvell_drivers[] = {
.cable_test_start = marvell_vct7_cable_test_start,
.cable_test_tdr_start = marvell_vct5_cable_test_tdr_start,
.cable_test_get_status = marvell_vct7_cable_test_get_status,
+ .led_brightness_set = m88e1318_led_brightness_set,
+ .led_blink_set = m88e1318_led_blink_set,
},
{
.phy_id = MARVELL_PHY_ID_88E3016,
@@ -3380,6 +3449,8 @@ static struct phy_driver marvell_drivers[] = {
.get_stats = marvell_get_stats,
.get_tunable = m88e1540_get_tunable,
.set_tunable = m88e1540_set_tunable,
+ .led_brightness_set = m88e1318_led_brightness_set,
+ .led_blink_set = m88e1318_led_blink_set,
},
};
diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c
index 2184b1e..3f81bb8 100644
--- a/drivers/net/phy/micrel.c
+++ b/drivers/net/phy/micrel.c
@@ -10,13 +10,13 @@
* Copyright (c) 2014 Johan Hovold <johan@kernel.org>
*
* Support : Micrel Phys:
- * Giga phys: ksz9021, ksz9031, ksz9131
+ * Giga phys: ksz9021, ksz9031, ksz9131, lan8841, lan8814
* 100/10 Phys : ksz8001, ksz8721, ksz8737, ksz8041
* ksz8021, ksz8031, ksz8051,
* ksz8081, ksz8091,
* ksz8061,
* Switch : ksz8873, ksz886x
- * ksz9477
+ * ksz9477, lan8804
*/
#include <linux/bitfield.h>
@@ -3761,7 +3761,7 @@ static int lan8841_ptp_update_target(struct kszphy_ptp_priv *ptp_priv,
const struct timespec64 *ts)
{
return lan8841_ptp_set_target(ptp_priv, LAN8841_EVENT_A,
- ts->tv_sec + LAN8841_BUFFER_TIME, ts->tv_nsec);
+ ts->tv_sec + LAN8841_BUFFER_TIME, 0);
}
#define LAN8841_PTP_LTC_TARGET_RELOAD_SEC_HI(event) ((event) == LAN8841_EVENT_A ? 282 : 292)
diff --git a/drivers/net/phy/nxp-cbtx.c b/drivers/net/phy/nxp-cbtx.c
new file mode 100644
index 0000000..145703f
--- /dev/null
+++ b/drivers/net/phy/nxp-cbtx.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Driver for 100BASE-TX PHY embedded into NXP SJA1110 switch
+ *
+ * Copyright 2022-2023 NXP
+ */
+
+#include <linux/kernel.h>
+#include <linux/mii.h>
+#include <linux/module.h>
+#include <linux/phy.h>
+
+#define PHY_ID_CBTX_SJA1110 0x001bb020
+
+/* Registers */
+#define CBTX_MODE_CTRL_STAT 0x11
+#define CBTX_PDOWN_CTRL 0x18
+#define CBTX_RX_ERR_COUNTER 0x1a
+#define CBTX_IRQ_STAT 0x1d
+#define CBTX_IRQ_ENABLE 0x1e
+
+/* Fields */
+#define CBTX_MODE_CTRL_STAT_AUTO_MDIX_EN BIT(7)
+#define CBTX_MODE_CTRL_STAT_MDIX_MODE BIT(6)
+
+#define CBTX_PDOWN_CTL_TRUE_PDOWN BIT(0)
+
+#define CBTX_IRQ_ENERGYON BIT(7)
+#define CBTX_IRQ_AN_COMPLETE BIT(6)
+#define CBTX_IRQ_REM_FAULT BIT(5)
+#define CBTX_IRQ_LINK_DOWN BIT(4)
+#define CBTX_IRQ_AN_LP_ACK BIT(3)
+#define CBTX_IRQ_PARALLEL_DETECT_FAULT BIT(2)
+#define CBTX_IRQ_AN_PAGE_RECV BIT(1)
+
+static int cbtx_soft_reset(struct phy_device *phydev)
+{
+ int ret;
+
+ /* Can't soft reset unless we remove PHY from true power down mode */
+ ret = phy_clear_bits(phydev, CBTX_PDOWN_CTRL,
+ CBTX_PDOWN_CTL_TRUE_PDOWN);
+ if (ret)
+ return ret;
+
+ return genphy_soft_reset(phydev);
+}
+
+static int cbtx_config_init(struct phy_device *phydev)
+{
+ /* Wait for cbtx_config_aneg() to kick in and apply this */
+ phydev->mdix_ctrl = ETH_TP_MDI_AUTO;
+
+ return 0;
+}
+
+static int cbtx_mdix_status(struct phy_device *phydev)
+{
+ int ret;
+
+ ret = phy_read(phydev, CBTX_MODE_CTRL_STAT);
+ if (ret < 0)
+ return ret;
+
+ if (ret & CBTX_MODE_CTRL_STAT_MDIX_MODE)
+ phydev->mdix = ETH_TP_MDI_X;
+ else
+ phydev->mdix = ETH_TP_MDI;
+
+ return 0;
+}
+
+static int cbtx_read_status(struct phy_device *phydev)
+{
+ int ret;
+
+ ret = cbtx_mdix_status(phydev);
+ if (ret)
+ return ret;
+
+ return genphy_read_status(phydev);
+}
+
+static int cbtx_mdix_config(struct phy_device *phydev)
+{
+ int ret;
+
+ switch (phydev->mdix_ctrl) {
+ case ETH_TP_MDI_AUTO:
+ return phy_set_bits(phydev, CBTX_MODE_CTRL_STAT,
+ CBTX_MODE_CTRL_STAT_AUTO_MDIX_EN);
+ case ETH_TP_MDI:
+ ret = phy_clear_bits(phydev, CBTX_MODE_CTRL_STAT,
+ CBTX_MODE_CTRL_STAT_AUTO_MDIX_EN);
+ if (ret)
+ return ret;
+
+ return phy_clear_bits(phydev, CBTX_MODE_CTRL_STAT,
+ CBTX_MODE_CTRL_STAT_MDIX_MODE);
+ case ETH_TP_MDI_X:
+ ret = phy_clear_bits(phydev, CBTX_MODE_CTRL_STAT,
+ CBTX_MODE_CTRL_STAT_AUTO_MDIX_EN);
+ if (ret)
+ return ret;
+
+ return phy_set_bits(phydev, CBTX_MODE_CTRL_STAT,
+ CBTX_MODE_CTRL_STAT_MDIX_MODE);
+ }
+
+ return 0;
+}
+
+static int cbtx_config_aneg(struct phy_device *phydev)
+{
+ int ret;
+
+ ret = cbtx_mdix_config(phydev);
+ if (ret)
+ return ret;
+
+ return genphy_config_aneg(phydev);
+}
+
+static int cbtx_ack_interrupts(struct phy_device *phydev)
+{
+ return phy_read(phydev, CBTX_IRQ_STAT);
+}
+
+static int cbtx_config_intr(struct phy_device *phydev)
+{
+ int ret;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ ret = cbtx_ack_interrupts(phydev);
+ if (ret < 0)
+ return ret;
+
+ ret = phy_write(phydev, CBTX_IRQ_ENABLE, CBTX_IRQ_LINK_DOWN |
+ CBTX_IRQ_AN_COMPLETE | CBTX_IRQ_ENERGYON);
+ if (ret)
+ return ret;
+ } else {
+ ret = phy_write(phydev, CBTX_IRQ_ENABLE, 0);
+ if (ret)
+ return ret;
+
+ ret = cbtx_ack_interrupts(phydev);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static irqreturn_t cbtx_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_stat, irq_enabled;
+
+ irq_stat = cbtx_ack_interrupts(phydev);
+ if (irq_stat < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ irq_enabled = phy_read(phydev, CBTX_IRQ_ENABLE);
+ if (irq_enabled < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_enabled & irq_stat))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
+static int cbtx_get_sset_count(struct phy_device *phydev)
+{
+ return 1;
+}
+
+static void cbtx_get_strings(struct phy_device *phydev, u8 *data)
+{
+ strncpy(data, "100btx_rx_err", ETH_GSTRING_LEN);
+}
+
+static void cbtx_get_stats(struct phy_device *phydev,
+ struct ethtool_stats *stats, u64 *data)
+{
+ int ret;
+
+ ret = phy_read(phydev, CBTX_RX_ERR_COUNTER);
+ data[0] = (ret < 0) ? U64_MAX : ret;
+}
+
+static struct phy_driver cbtx_driver[] = {
+ {
+ PHY_ID_MATCH_MODEL(PHY_ID_CBTX_SJA1110),
+ .name = "NXP CBTX (SJA1110)",
+ /* PHY_BASIC_FEATURES */
+ .soft_reset = cbtx_soft_reset,
+ .config_init = cbtx_config_init,
+ .suspend = genphy_suspend,
+ .resume = genphy_resume,
+ .config_intr = cbtx_config_intr,
+ .handle_interrupt = cbtx_handle_interrupt,
+ .read_status = cbtx_read_status,
+ .config_aneg = cbtx_config_aneg,
+ .get_sset_count = cbtx_get_sset_count,
+ .get_strings = cbtx_get_strings,
+ .get_stats = cbtx_get_stats,
+ },
+};
+
+module_phy_driver(cbtx_driver);
+
+static struct mdio_device_id __maybe_unused cbtx_tbl[] = {
+ { PHY_ID_MATCH_MODEL(PHY_ID_CBTX_SJA1110) },
+ { },
+};
+
+MODULE_DEVICE_TABLE(mdio, cbtx_tbl);
+
+MODULE_AUTHOR("Vladimir Oltean <vladimir.oltean@nxp.com>");
+MODULE_DESCRIPTION("NXP CBTX PHY driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c
index 917ba84..d373446 100644
--- a/drivers/net/phy/phy_device.c
+++ b/drivers/net/phy/phy_device.c
@@ -19,10 +19,12 @@
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel.h>
+#include <linux/list.h>
#include <linux/mdio.h>
#include <linux/mii.h>
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/of.h>
#include <linux/netdevice.h>
#include <linux/phy.h>
#include <linux/phy_led_triggers.h>
@@ -674,6 +676,7 @@ struct phy_device *phy_device_create(struct mii_bus *bus, int addr, u32 phy_id,
device_initialize(&mdiodev->dev);
dev->state = PHY_DOWN;
+ INIT_LIST_HEAD(&dev->leds);
mutex_init(&dev->lock);
INIT_DELAYED_WORK(&dev->state_queue, phy_state_machine);
@@ -2988,6 +2991,101 @@ static bool phy_drv_supports_irq(struct phy_driver *phydrv)
return phydrv->config_intr && phydrv->handle_interrupt;
}
+static int phy_led_set_brightness(struct led_classdev *led_cdev,
+ enum led_brightness value)
+{
+ struct phy_led *phyled = to_phy_led(led_cdev);
+ struct phy_device *phydev = phyled->phydev;
+ int err;
+
+ mutex_lock(&phydev->lock);
+ err = phydev->drv->led_brightness_set(phydev, phyled->index, value);
+ mutex_unlock(&phydev->lock);
+
+ return err;
+}
+
+static int phy_led_blink_set(struct led_classdev *led_cdev,
+ unsigned long *delay_on,
+ unsigned long *delay_off)
+{
+ struct phy_led *phyled = to_phy_led(led_cdev);
+ struct phy_device *phydev = phyled->phydev;
+ int err;
+
+ mutex_lock(&phydev->lock);
+ err = phydev->drv->led_blink_set(phydev, phyled->index,
+ delay_on, delay_off);
+ mutex_unlock(&phydev->lock);
+
+ return err;
+}
+
+static int of_phy_led(struct phy_device *phydev,
+ struct device_node *led)
+{
+ struct device *dev = &phydev->mdio.dev;
+ struct led_init_data init_data = {};
+ struct led_classdev *cdev;
+ struct phy_led *phyled;
+ int err;
+
+ phyled = devm_kzalloc(dev, sizeof(*phyled), GFP_KERNEL);
+ if (!phyled)
+ return -ENOMEM;
+
+ cdev = &phyled->led_cdev;
+ phyled->phydev = phydev;
+
+ err = of_property_read_u8(led, "reg", &phyled->index);
+ if (err)
+ return err;
+
+ if (phydev->drv->led_brightness_set)
+ cdev->brightness_set_blocking = phy_led_set_brightness;
+ if (phydev->drv->led_blink_set)
+ cdev->blink_set = phy_led_blink_set;
+ cdev->max_brightness = 1;
+ init_data.devicename = dev_name(&phydev->mdio.dev);
+ init_data.fwnode = of_fwnode_handle(led);
+ init_data.devname_mandatory = true;
+
+ err = devm_led_classdev_register_ext(dev, cdev, &init_data);
+ if (err)
+ return err;
+
+ list_add(&phyled->list, &phydev->leds);
+
+ return 0;
+}
+
+static int of_phy_leds(struct phy_device *phydev)
+{
+ struct device_node *node = phydev->mdio.dev.of_node;
+ struct device_node *leds, *led;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_OF_MDIO))
+ return 0;
+
+ if (!node)
+ return 0;
+
+ leds = of_get_child_by_name(node, "leds");
+ if (!leds)
+ return 0;
+
+ for_each_available_child_of_node(leds, led) {
+ err = of_phy_led(phydev, led);
+ if (err) {
+ of_node_put(led);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
/**
* fwnode_mdio_find_device - Given a fwnode, find the mdio_device
* @fwnode: pointer to the mdio_device's fwnode
@@ -3183,6 +3281,12 @@ static int phy_probe(struct device *dev)
/* Set the state to READY by default */
phydev->state = PHY_READY;
+ /* Get the LEDs from the device tree, and instantiate standard
+ * LEDs for them.
+ */
+ if (IS_ENABLED(CONFIG_PHYLIB_LEDS))
+ err = of_phy_leds(phydev);
+
out:
/* Re-assert the reset signal on error */
if (err)
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index e1b38fb..4b3c664 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1262,11 +1262,12 @@ static void veth_set_xdp_features(struct net_device *dev)
peer = rtnl_dereference(priv->peer);
if (peer && peer->real_num_tx_queues <= dev->real_num_rx_queues) {
+ struct veth_priv *priv_peer = netdev_priv(peer);
xdp_features_t val = NETDEV_XDP_ACT_BASIC |
NETDEV_XDP_ACT_REDIRECT |
NETDEV_XDP_ACT_RX_SG;
- if (priv->_xdp_prog || veth_gro_requested(dev))
+ if (priv_peer->_xdp_prog || veth_gro_requested(peer))
val |= NETDEV_XDP_ACT_NDO_XMIT |
NETDEV_XDP_ACT_NDO_XMIT_SG;
xdp_set_features_flag(dev, val);
@@ -1504,19 +1505,23 @@ static int veth_set_features(struct net_device *dev,
{
netdev_features_t changed = features ^ dev->features;
struct veth_priv *priv = netdev_priv(dev);
+ struct net_device *peer;
int err;
if (!(changed & NETIF_F_GRO) || !(dev->flags & IFF_UP) || priv->_xdp_prog)
return 0;
+ peer = rtnl_dereference(priv->peer);
if (features & NETIF_F_GRO) {
err = veth_napi_enable(dev);
if (err)
return err;
- xdp_features_set_redirect_target(dev, true);
+ if (peer)
+ xdp_features_set_redirect_target(peer, true);
} else {
- xdp_features_clear_redirect_target(dev);
+ if (peer)
+ xdp_features_clear_redirect_target(peer);
veth_napi_del(dev);
}
return 0;
@@ -1598,13 +1603,13 @@ static int veth_xdp_set(struct net_device *dev, struct bpf_prog *prog,
peer->max_mtu = max_mtu;
}
- xdp_features_set_redirect_target(dev, true);
+ xdp_features_set_redirect_target(peer, true);
}
if (old_prog) {
if (!prog) {
- if (!veth_gro_requested(dev))
- xdp_features_clear_redirect_target(dev);
+ if (peer && !veth_gro_requested(dev))
+ xdp_features_clear_redirect_target(peer);
if (dev->flags & IFF_UP)
veth_disable_xdp(dev);
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index e2560b6..8d80385 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -815,8 +815,13 @@ static struct page *xdp_linearize_page(struct receive_queue *rq,
int page_off,
unsigned int *len)
{
- struct page *page = alloc_page(GFP_ATOMIC);
+ int tailroom = SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ struct page *page;
+ if (page_off + *len + tailroom > PAGE_SIZE)
+ return NULL;
+
+ page = alloc_page(GFP_ATOMIC);
if (!page)
return NULL;
@@ -824,7 +829,6 @@ static struct page *xdp_linearize_page(struct receive_queue *rq,
page_off += *len;
while (--*num_buf) {
- int tailroom = SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
unsigned int buflen;
void *buf;
int off;
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index da488cbb..f2b76ee 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -1504,7 +1504,7 @@ vmxnet3_rq_rx_complete(struct vmxnet3_rx_queue *rq,
goto rcd_done;
}
- if (rxDataRingUsed) {
+ if (rxDataRingUsed && adapter->rxdataring_enabled) {
size_t sz;
BUG_ON(rcd->len > rq->data_ring.desc_size);
diff --git a/drivers/net/wireless/ath/ath10k/ce.c b/drivers/net/wireless/ath/ath10k/ce.c
index b656cfc..c27b820 100644
--- a/drivers/net/wireless/ath/ath10k/ce.c
+++ b/drivers/net/wireless/ath/ath10k/ce.c
@@ -84,13 +84,6 @@ ath10k_set_ring_byte(unsigned int offset,
return ((offset << addr_map->lsb) & addr_map->mask);
}
-static inline unsigned int
-ath10k_get_ring_byte(unsigned int offset,
- struct ath10k_hw_ce_regs_addr_map *addr_map)
-{
- return ((offset & addr_map->mask) >> (addr_map->lsb));
-}
-
static inline u32 ath10k_ce_read32(struct ath10k *ar, u32 offset)
{
struct ath10k_ce *ce = ath10k_ce_priv(ar);
diff --git a/drivers/net/wireless/ath/ath10k/pci.c b/drivers/net/wireless/ath/ath10k/pci.c
index 728d607..a7f44f6 100644
--- a/drivers/net/wireless/ath/ath10k/pci.c
+++ b/drivers/net/wireless/ath/ath10k/pci.c
@@ -3406,15 +3406,12 @@ static int ath10k_pci_claim(struct ath10k *ar)
if (!ar_pci->mem) {
ath10k_err(ar, "failed to iomap BAR%d\n", BAR_NUM);
ret = -EIO;
- goto err_master;
+ goto err_region;
}
ath10k_dbg(ar, ATH10K_DBG_BOOT, "boot pci_mem 0x%pK\n", ar_pci->mem);
return 0;
-err_master:
- pci_clear_master(pdev);
-
err_region:
pci_release_region(pdev, BAR_NUM);
@@ -3431,7 +3428,6 @@ static void ath10k_pci_release(struct ath10k *ar)
pci_iounmap(pdev, ar_pci->mem);
pci_release_region(pdev, BAR_NUM);
- pci_clear_master(pdev);
pci_disable_device(pdev);
}
diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c
index b549576..5cbba9a 100644
--- a/drivers/net/wireless/ath/ath11k/ahb.c
+++ b/drivers/net/wireless/ath/ath11k/ahb.c
@@ -1078,6 +1078,12 @@ static int ath11k_ahb_fw_resource_deinit(struct ath11k_base *ab)
struct iommu_domain *iommu;
size_t unmapped_size;
+ /* Chipsets not requiring MSA would have not initialized
+ * MSA resources, return success in such cases.
+ */
+ if (!ab->hw_params.fixed_fw_mem)
+ return 0;
+
if (ab_ahb->fw.use_tz)
return 0;
diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c
index 75fdbe4..b1b90bd 100644
--- a/drivers/net/wireless/ath/ath11k/core.c
+++ b/drivers/net/wireless/ath/ath11k/core.c
@@ -116,7 +116,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tcl_ring_retry = true,
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
- .ftm_responder = true,
},
{
.hw_rev = ATH11K_HW_IPQ6018_HW10,
@@ -199,7 +198,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = false,
- .ftm_responder = true,
},
{
.name = "qca6390 hw2.0",
@@ -284,7 +282,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = true,
- .ftm_responder = false,
},
{
.name = "qcn9074 hw1.0",
@@ -366,7 +363,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = false,
- .ftm_responder = true,
},
{
.name = "wcn6855 hw2.0",
@@ -451,7 +447,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = true,
- .ftm_responder = false,
},
{
.name = "wcn6855 hw2.1",
@@ -534,7 +529,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = true,
- .ftm_responder = false,
},
{
.name = "wcn6750 hw1.0",
@@ -599,7 +593,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.current_cc_support = true,
.dbr_debug_support = false,
.global_reset = false,
- .bios_sar_capa = NULL,
+ .bios_sar_capa = &ath11k_hw_sar_capa_wcn6855,
.m3_fw_support = false,
.fixed_bdf_addr = false,
.fixed_mem_region = false,
@@ -615,7 +609,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE_WCN6750,
.smp2p_wow_exit = true,
.support_fw_mac_sequence = true,
- .ftm_responder = false,
},
{
.hw_rev = ATH11K_HW_IPQ5018_HW10,
@@ -695,7 +688,6 @@ static const struct ath11k_hw_params ath11k_hw_params[] = {
.tx_ring_size = DP_TCL_DATA_RING_SIZE,
.smp2p_wow_exit = false,
.support_fw_mac_sequence = false,
- .ftm_responder = true,
},
};
diff --git a/drivers/net/wireless/ath/ath11k/dbring.c b/drivers/net/wireless/ath/ath11k/dbring.c
index 2107ec0..5536e86 100644
--- a/drivers/net/wireless/ath/ath11k/dbring.c
+++ b/drivers/net/wireless/ath/ath11k/dbring.c
@@ -26,13 +26,13 @@ int ath11k_dbring_validate_buffer(struct ath11k *ar, void *buffer, u32 size)
static void ath11k_dbring_fill_magic_value(struct ath11k *ar,
void *buffer, u32 size)
{
- u32 *temp;
- int idx;
+ /* memset32 function fills buffer payload with the ATH11K_DB_MAGIC_VALUE
+ * and the variable size is expected to be the number of u32 values
+ * to be stored, not the number of bytes.
+ */
+ size = size / sizeof(u32);
- size = size >> 2;
-
- for (idx = 0, temp = buffer; idx < size; idx++, temp++)
- *temp++ = ATH11K_DB_MAGIC_VALUE;
+ memset32(buffer, ATH11K_DB_MAGIC_VALUE, size);
}
static int ath11k_dbring_bufs_replenish(struct ath11k *ar,
diff --git a/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h b/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h
index 2b97cbb..0bbd58a 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h
+++ b/drivers/net/wireless/ath/ath11k/debugfs_htt_stats.h
@@ -143,7 +143,8 @@ enum htt_tx_pdev_underrun_enum {
/* Bytes stored in little endian order */
/* Length should be multiple of DWORD */
struct htt_stats_string_tlv {
- u32 data[0]; /* Can be variable length */
+ /* Can be variable length */
+ DECLARE_FLEX_ARRAY(u32, data);
} __packed;
#define HTT_STATS_MAC_ID GENMASK(7, 0)
@@ -205,27 +206,32 @@ struct htt_tx_pdev_stats_cmn_tlv {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_pdev_stats_urrn_tlv_v {
- u32 urrn_stats[0]; /* HTT_TX_PDEV_MAX_URRN_STATS */
+ /* HTT_TX_PDEV_MAX_URRN_STATS */
+ DECLARE_FLEX_ARRAY(u32, urrn_stats);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_pdev_stats_flush_tlv_v {
- u32 flush_errs[0]; /* HTT_TX_PDEV_MAX_FLUSH_REASON_STATS */
+ /* HTT_TX_PDEV_MAX_FLUSH_REASON_STATS */
+ DECLARE_FLEX_ARRAY(u32, flush_errs);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_pdev_stats_sifs_tlv_v {
- u32 sifs_status[0]; /* HTT_TX_PDEV_MAX_SIFS_BURST_STATS */
+ /* HTT_TX_PDEV_MAX_SIFS_BURST_STATS */
+ DECLARE_FLEX_ARRAY(u32, sifs_status);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_pdev_stats_phy_err_tlv_v {
- u32 phy_errs[0]; /* HTT_TX_PDEV_MAX_PHY_ERR_STATS */
+ /* HTT_TX_PDEV_MAX_PHY_ERR_STATS */
+ DECLARE_FLEX_ARRAY(u32, phy_errs);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_pdev_stats_sifs_hist_tlv_v {
- u32 sifs_hist_status[0]; /* HTT_TX_PDEV_SIFS_BURST_HIST_STATS */
+ /* HTT_TX_PDEV_SIFS_BURST_HIST_STATS */
+ DECLARE_FLEX_ARRAY(u32, sifs_hist_status);
};
struct htt_tx_pdev_stats_tx_ppdu_stats_tlv_v {
@@ -590,20 +596,20 @@ struct htt_tx_hwq_difs_latency_stats_tlv_v {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_hwq_cmd_result_stats_tlv_v {
- /* Histogram of sched cmd result */
- u32 cmd_result[0]; /* HTT_TX_HWQ_MAX_CMD_RESULT_STATS */
+ /* Histogram of sched cmd result, HTT_TX_HWQ_MAX_CMD_RESULT_STATS */
+ DECLARE_FLEX_ARRAY(u32, cmd_result);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_hwq_cmd_stall_stats_tlv_v {
- /* Histogram of various pause conitions */
- u32 cmd_stall_status[0]; /* HTT_TX_HWQ_MAX_CMD_STALL_STATS */
+ /* Histogram of various pause conitions, HTT_TX_HWQ_MAX_CMD_STALL_STATS */
+ DECLARE_FLEX_ARRAY(u32, cmd_stall_status);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_hwq_fes_result_stats_tlv_v {
- /* Histogram of number of user fes result */
- u32 fes_result[0]; /* HTT_TX_HWQ_MAX_FES_RESULT_STATS */
+ /* Histogram of number of user fes result, HTT_TX_HWQ_MAX_FES_RESULT_STATS */
+ DECLARE_FLEX_ARRAY(u32, fes_result);
};
/* NOTE: Variable length TLV, use length spec to infer array size
@@ -635,8 +641,8 @@ struct htt_tx_hwq_tried_mpdu_cnt_hist_tlv_v {
* #define WAL_TXOP_USED_HISTOGRAM_INTERVAL 1000 ( 1 ms )
*/
struct htt_tx_hwq_txop_used_cnt_hist_tlv_v {
- /* Histogram of txop used cnt */
- u32 txop_used_cnt_hist[0]; /* HTT_TX_HWQ_TXOP_USED_CNT_HIST */
+ /* Histogram of txop used cnt, HTT_TX_HWQ_TXOP_USED_CNT_HIST */
+ DECLARE_FLEX_ARRAY(u32, txop_used_cnt_hist);
};
/* == TX SELFGEN STATS == */
@@ -804,17 +810,20 @@ struct htt_tx_pdev_mpdu_stats_tlv {
/* == TX SCHED STATS == */
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_sched_txq_cmd_posted_tlv_v {
- u32 sched_cmd_posted[0]; /* HTT_TX_PDEV_SCHED_TX_MODE_MAX */
+ /* HTT_TX_PDEV_SCHED_TX_MODE_MAX */
+ DECLARE_FLEX_ARRAY(u32, sched_cmd_posted);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_sched_txq_cmd_reaped_tlv_v {
- u32 sched_cmd_reaped[0]; /* HTT_TX_PDEV_SCHED_TX_MODE_MAX */
+ /* HTT_TX_PDEV_SCHED_TX_MODE_MAX */
+ DECLARE_FLEX_ARRAY(u32, sched_cmd_reaped);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_sched_txq_sched_order_su_tlv_v {
- u32 sched_order_su[0]; /* HTT_TX_PDEV_NUM_SCHED_ORDER_LOG */
+ /* HTT_TX_PDEV_NUM_SCHED_ORDER_LOG */
+ DECLARE_FLEX_ARRAY(u32, sched_order_su);
};
enum htt_sched_txq_sched_ineligibility_tlv_enum {
@@ -842,7 +851,7 @@ enum htt_sched_txq_sched_ineligibility_tlv_enum {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_sched_txq_sched_ineligibility_tlv_v {
/* indexed by htt_sched_txq_sched_ineligibility_tlv_enum */
- u32 sched_ineligibility[0];
+ DECLARE_FLEX_ARRAY(u32, sched_ineligibility);
};
#define HTT_TX_PDEV_STATS_SCHED_PER_TXQ_MAC_ID GENMASK(7, 0)
@@ -888,18 +897,20 @@ struct htt_stats_tx_sched_cmn_tlv {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_tqm_gen_mpdu_stats_tlv_v {
- u32 gen_mpdu_end_reason[0]; /* HTT_TX_TQM_MAX_GEN_MPDU_END_REASON */
+ /* HTT_TX_TQM_MAX_GEN_MPDU_END_REASON */
+ DECLARE_FLEX_ARRAY(u32, gen_mpdu_end_reason);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_tqm_list_mpdu_stats_tlv_v {
- u32 list_mpdu_end_reason[0]; /* HTT_TX_TQM_MAX_LIST_MPDU_END_REASON */
+ /* HTT_TX_TQM_MAX_LIST_MPDU_END_REASON */
+ DECLARE_FLEX_ARRAY(u32, list_mpdu_end_reason);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_tx_tqm_list_mpdu_cnt_tlv_v {
- u32 list_mpdu_cnt_hist[0];
- /* HTT_TX_TQM_MAX_LIST_MPDU_CNT_HISTOGRAM_BINS */
+ /* HTT_TX_TQM_MAX_LIST_MPDU_CNT_HISTOGRAM_BINS */
+ DECLARE_FLEX_ARRAY(u32, list_mpdu_cnt_hist);
};
struct htt_tx_tqm_pdev_stats_tlv_v {
@@ -1098,7 +1109,7 @@ struct htt_tx_de_compl_stats_tlv {
* ENTRIES_PER_BIN_COUNT)
*/
struct htt_tx_de_fw2wbm_ring_full_hist_tlv {
- u32 fw2wbm_ring_full_hist[0];
+ DECLARE_FLEX_ARRAY(u32, fw2wbm_ring_full_hist);
};
struct htt_tx_de_cmn_stats_tlv {
@@ -1151,7 +1162,7 @@ struct htt_ring_if_cmn_tlv {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_sfm_client_user_tlv_v {
/* Number of DWORDS used per user and per client */
- u32 dwords_used_by_user_n[0];
+ DECLARE_FLEX_ARRAY(u32, dwords_used_by_user_n);
};
struct htt_sfm_client_tlv {
@@ -1436,12 +1447,14 @@ struct htt_rx_soc_fw_stats_tlv {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_soc_fw_refill_ring_empty_tlv_v {
- u32 refill_ring_empty_cnt[0]; /* HTT_RX_STATS_REFILL_MAX_RING */
+ /* HTT_RX_STATS_REFILL_MAX_RING */
+ DECLARE_FLEX_ARRAY(u32, refill_ring_empty_cnt);
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_soc_fw_refill_ring_num_refill_tlv_v {
- u32 refill_ring_num_refill[0]; /* HTT_RX_STATS_REFILL_MAX_RING */
+ /* HTT_RX_STATS_REFILL_MAX_RING */
+ DECLARE_FLEX_ARRAY(u32, refill_ring_num_refill);
};
/* RXDMA error code from WBM released packets */
@@ -1473,7 +1486,7 @@ enum htt_rx_rxdma_error_code_enum {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_soc_fw_refill_ring_num_rxdma_err_tlv_v {
- u32 rxdma_err[0]; /* HTT_RX_RXDMA_MAX_ERR_CODE */
+ DECLARE_FLEX_ARRAY(u32, rxdma_err); /* HTT_RX_RXDMA_MAX_ERR_CODE */
};
/* REO error code from WBM released packets */
@@ -1505,7 +1518,7 @@ enum htt_rx_reo_error_code_enum {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_soc_fw_refill_ring_num_reo_err_tlv_v {
- u32 reo_err[0]; /* HTT_RX_REO_MAX_ERR_CODE */
+ DECLARE_FLEX_ARRAY(u32, reo_err); /* HTT_RX_REO_MAX_ERR_CODE */
};
/* == RX PDEV STATS == */
@@ -1622,13 +1635,13 @@ struct htt_rx_pdev_fw_stats_phy_err_tlv {
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_pdev_fw_ring_mpdu_err_tlv_v {
/* Num error MPDU for each RxDMA error type */
- u32 fw_ring_mpdu_err[0]; /* HTT_RX_STATS_RXDMA_MAX_ERR */
+ DECLARE_FLEX_ARRAY(u32, fw_ring_mpdu_err); /* HTT_RX_STATS_RXDMA_MAX_ERR */
};
/* NOTE: Variable length TLV, use length spec to infer array size */
struct htt_rx_pdev_fw_mpdu_drop_tlv_v {
/* Num MPDU dropped */
- u32 fw_mpdu_drop[0]; /* HTT_RX_STATS_FW_DROP_REASON_MAX */
+ DECLARE_FLEX_ARRAY(u32, fw_mpdu_drop); /* HTT_RX_STATS_FW_DROP_REASON_MAX */
};
#define HTT_PDEV_CCA_STATS_TX_FRAME_INFO_PRESENT (0x1)
diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c
index f5156a7..d070bcb 100644
--- a/drivers/net/wireless/ath/ath11k/dp.c
+++ b/drivers/net/wireless/ath/ath11k/dp.c
@@ -36,6 +36,7 @@ void ath11k_dp_peer_cleanup(struct ath11k *ar, int vdev_id, const u8 *addr)
}
ath11k_peer_rx_tid_cleanup(ar, peer);
+ peer->dp_setup_done = false;
crypto_free_shash(peer->tfm_mmic);
spin_unlock_bh(&ab->base_lock);
}
@@ -72,7 +73,8 @@ int ath11k_dp_peer_setup(struct ath11k *ar, int vdev_id, const u8 *addr)
ret = ath11k_peer_rx_frag_setup(ar, addr, vdev_id);
if (ret) {
ath11k_warn(ab, "failed to setup rx defrag context\n");
- return ret;
+ tid--;
+ goto peer_clean;
}
/* TODO: Setup other peer specific resource used in data path */
diff --git a/drivers/net/wireless/ath/ath11k/dp.h b/drivers/net/wireless/ath/ath11k/dp.h
index be9eafc..d04f78a 100644
--- a/drivers/net/wireless/ath/ath11k/dp.h
+++ b/drivers/net/wireless/ath/ath11k/dp.h
@@ -214,7 +214,7 @@ struct ath11k_pdev_dp {
#define DP_REO_REINJECT_RING_SIZE 32
#define DP_RX_RELEASE_RING_SIZE 1024
#define DP_REO_EXCEPTION_RING_SIZE 128
-#define DP_REO_CMD_RING_SIZE 128
+#define DP_REO_CMD_RING_SIZE 256
#define DP_REO_STATUS_RING_SIZE 2048
#define DP_RXDMA_BUF_RING_SIZE 4096
#define DP_RXDMA_REFILL_RING_SIZE 2048
@@ -303,12 +303,16 @@ struct ath11k_dp {
#define HTT_TX_WBM_COMP_STATUS_OFFSET 8
+#define HTT_INVALID_PEER_ID 0xffff
+
/* HTT tx completion is overlaid in wbm_release_ring */
#define HTT_TX_WBM_COMP_INFO0_STATUS GENMASK(12, 9)
#define HTT_TX_WBM_COMP_INFO0_REINJECT_REASON GENMASK(16, 13)
#define HTT_TX_WBM_COMP_INFO0_REINJECT_REASON GENMASK(16, 13)
#define HTT_TX_WBM_COMP_INFO1_ACK_RSSI GENMASK(31, 24)
+#define HTT_TX_WBM_COMP_INFO2_SW_PEER_ID GENMASK(15, 0)
+#define HTT_TX_WBM_COMP_INFO2_VALID BIT(21)
struct htt_tx_wbm_completion {
u32 info0;
diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c
index b65a84a..f67ce62 100644
--- a/drivers/net/wireless/ath/ath11k/dp_rx.c
+++ b/drivers/net/wireless/ath/ath11k/dp_rx.c
@@ -389,10 +389,10 @@ int ath11k_dp_rxbufs_replenish(struct ath11k_base *ab, int mac_id,
goto fail_free_skb;
spin_lock_bh(&rx_ring->idr_lock);
- buf_id = idr_alloc(&rx_ring->bufs_idr, skb, 0,
- rx_ring->bufs_max * 3, GFP_ATOMIC);
+ buf_id = idr_alloc(&rx_ring->bufs_idr, skb, 1,
+ (rx_ring->bufs_max * 3) + 1, GFP_ATOMIC);
spin_unlock_bh(&rx_ring->idr_lock);
- if (buf_id < 0)
+ if (buf_id <= 0)
goto fail_dma_unmap;
desc = ath11k_hal_srng_src_get_next_entry(ab, srng);
@@ -435,7 +435,6 @@ int ath11k_dp_rxbufs_replenish(struct ath11k_base *ab, int mac_id,
static int ath11k_dp_rxdma_buf_ring_free(struct ath11k *ar,
struct dp_rxdma_ring *rx_ring)
{
- struct ath11k_pdev_dp *dp = &ar->dp;
struct sk_buff *skb;
int buf_id;
@@ -453,28 +452,6 @@ static int ath11k_dp_rxdma_buf_ring_free(struct ath11k *ar,
idr_destroy(&rx_ring->bufs_idr);
spin_unlock_bh(&rx_ring->idr_lock);
- /* if rxdma1_enable is false, mon_status_refill_ring
- * isn't setup, so don't clean.
- */
- if (!ar->ab->hw_params.rxdma1_enable)
- return 0;
-
- rx_ring = &dp->rx_mon_status_refill_ring[0];
-
- spin_lock_bh(&rx_ring->idr_lock);
- idr_for_each_entry(&rx_ring->bufs_idr, skb, buf_id) {
- idr_remove(&rx_ring->bufs_idr, buf_id);
- /* XXX: Understand where internal driver does this dma_unmap
- * of rxdma_buffer.
- */
- dma_unmap_single(ar->ab->dev, ATH11K_SKB_RXCB(skb)->paddr,
- skb->len + skb_tailroom(skb), DMA_BIDIRECTIONAL);
- dev_kfree_skb_any(skb);
- }
-
- idr_destroy(&rx_ring->bufs_idr);
- spin_unlock_bh(&rx_ring->idr_lock);
-
return 0;
}
@@ -691,13 +668,18 @@ void ath11k_dp_reo_cmd_list_cleanup(struct ath11k_base *ab)
struct ath11k_dp *dp = &ab->dp;
struct dp_reo_cmd *cmd, *tmp;
struct dp_reo_cache_flush_elem *cmd_cache, *tmp_cache;
+ struct dp_rx_tid *rx_tid;
spin_lock_bh(&dp->reo_cmd_lock);
list_for_each_entry_safe(cmd, tmp, &dp->reo_cmd_list, list) {
list_del(&cmd->list);
- dma_unmap_single(ab->dev, cmd->data.paddr,
- cmd->data.size, DMA_BIDIRECTIONAL);
- kfree(cmd->data.vaddr);
+ rx_tid = &cmd->data;
+ if (rx_tid->vaddr) {
+ dma_unmap_single(ab->dev, rx_tid->paddr,
+ rx_tid->size, DMA_BIDIRECTIONAL);
+ kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
+ }
kfree(cmd);
}
@@ -705,9 +687,13 @@ void ath11k_dp_reo_cmd_list_cleanup(struct ath11k_base *ab)
&dp->reo_cmd_cache_flush_list, list) {
list_del(&cmd_cache->list);
dp->reo_cmd_cache_flush_count--;
- dma_unmap_single(ab->dev, cmd_cache->data.paddr,
- cmd_cache->data.size, DMA_BIDIRECTIONAL);
- kfree(cmd_cache->data.vaddr);
+ rx_tid = &cmd_cache->data;
+ if (rx_tid->vaddr) {
+ dma_unmap_single(ab->dev, rx_tid->paddr,
+ rx_tid->size, DMA_BIDIRECTIONAL);
+ kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
+ }
kfree(cmd_cache);
}
spin_unlock_bh(&dp->reo_cmd_lock);
@@ -721,10 +707,12 @@ static void ath11k_dp_reo_cmd_free(struct ath11k_dp *dp, void *ctx,
if (status != HAL_REO_CMD_SUCCESS)
ath11k_warn(dp->ab, "failed to flush rx tid hw desc, tid %d status %d\n",
rx_tid->tid, status);
-
- dma_unmap_single(dp->ab->dev, rx_tid->paddr, rx_tid->size,
- DMA_BIDIRECTIONAL);
- kfree(rx_tid->vaddr);
+ if (rx_tid->vaddr) {
+ dma_unmap_single(dp->ab->dev, rx_tid->paddr, rx_tid->size,
+ DMA_BIDIRECTIONAL);
+ kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
+ }
}
static void ath11k_dp_reo_cache_flush(struct ath11k_base *ab,
@@ -763,6 +751,7 @@ static void ath11k_dp_reo_cache_flush(struct ath11k_base *ab,
dma_unmap_single(ab->dev, rx_tid->paddr, rx_tid->size,
DMA_BIDIRECTIONAL);
kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
}
}
@@ -815,6 +804,7 @@ static void ath11k_dp_rx_tid_del_func(struct ath11k_dp *dp, void *ctx,
dma_unmap_single(ab->dev, rx_tid->paddr, rx_tid->size,
DMA_BIDIRECTIONAL);
kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
}
void ath11k_peer_rx_tid_delete(struct ath11k *ar,
@@ -827,6 +817,8 @@ void ath11k_peer_rx_tid_delete(struct ath11k *ar,
if (!rx_tid->active)
return;
+ rx_tid->active = false;
+
cmd.flag = HAL_REO_CMD_FLG_NEED_STATUS;
cmd.addr_lo = lower_32_bits(rx_tid->paddr);
cmd.addr_hi = upper_32_bits(rx_tid->paddr);
@@ -841,9 +833,11 @@ void ath11k_peer_rx_tid_delete(struct ath11k *ar,
dma_unmap_single(ar->ab->dev, rx_tid->paddr, rx_tid->size,
DMA_BIDIRECTIONAL);
kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
}
- rx_tid->active = false;
+ rx_tid->paddr = 0;
+ rx_tid->size = 0;
}
static int ath11k_dp_rx_link_desc_return(struct ath11k_base *ab,
@@ -990,6 +984,7 @@ static void ath11k_dp_rx_tid_mem_free(struct ath11k_base *ab,
dma_unmap_single(ab->dev, rx_tid->paddr, rx_tid->size,
DMA_BIDIRECTIONAL);
kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
rx_tid->active = false;
@@ -1014,7 +1009,8 @@ int ath11k_peer_rx_tid_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id,
peer = ath11k_peer_find(ab, vdev_id, peer_mac);
if (!peer) {
- ath11k_warn(ab, "failed to find the peer to set up rx tid\n");
+ ath11k_warn(ab, "failed to find the peer %pM to set up rx tid\n",
+ peer_mac);
spin_unlock_bh(&ab->base_lock);
return -ENOENT;
}
@@ -1027,7 +1023,8 @@ int ath11k_peer_rx_tid_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id,
ba_win_sz, ssn, true);
spin_unlock_bh(&ab->base_lock);
if (ret) {
- ath11k_warn(ab, "failed to update reo for rx tid %d\n", tid);
+ ath11k_warn(ab, "failed to update reo for peer %pM rx tid %d\n: %d",
+ peer_mac, tid, ret);
return ret;
}
@@ -1035,8 +1032,8 @@ int ath11k_peer_rx_tid_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id,
peer_mac, paddr,
tid, 1, ba_win_sz);
if (ret)
- ath11k_warn(ab, "failed to send wmi command to update rx reorder queue, tid :%d (%d)\n",
- tid, ret);
+ ath11k_warn(ab, "failed to send wmi rx reorder queue for peer %pM tid %d: %d\n",
+ peer_mac, tid, ret);
return ret;
}
@@ -1069,6 +1066,8 @@ int ath11k_peer_rx_tid_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id,
ret = dma_mapping_error(ab->dev, paddr);
if (ret) {
spin_unlock_bh(&ab->base_lock);
+ ath11k_warn(ab, "failed to setup dma map for peer %pM rx tid %d: %d\n",
+ peer_mac, tid, ret);
goto err_mem_free;
}
@@ -1082,15 +1081,16 @@ int ath11k_peer_rx_tid_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id,
ret = ath11k_wmi_peer_rx_reorder_queue_setup(ar, vdev_id, peer_mac,
paddr, tid, 1, ba_win_sz);
if (ret) {
- ath11k_warn(ar->ab, "failed to setup rx reorder queue, tid :%d (%d)\n",
- tid, ret);
+ ath11k_warn(ar->ab, "failed to setup rx reorder queue for peer %pM tid %d: %d\n",
+ peer_mac, tid, ret);
ath11k_dp_rx_tid_mem_free(ab, peer_mac, vdev_id, tid);
}
return ret;
err_mem_free:
- kfree(vaddr);
+ kfree(rx_tid->vaddr);
+ rx_tid->vaddr = NULL;
return ret;
}
@@ -2665,6 +2665,9 @@ int ath11k_dp_process_rx(struct ath11k_base *ab, int ring_id,
cookie);
mac_id = FIELD_GET(DP_RXDMA_BUF_COOKIE_PDEV_ID, cookie);
+ if (unlikely(buf_id == 0))
+ continue;
+
ar = ab->pdevs[mac_id].ar;
rx_ring = &ar->dp.rx_refill_buf_ring;
spin_lock_bh(&rx_ring->idr_lock);
@@ -3029,39 +3032,51 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id,
spin_lock_bh(&rx_ring->idr_lock);
skb = idr_find(&rx_ring->bufs_idr, buf_id);
+ spin_unlock_bh(&rx_ring->idr_lock);
+
if (!skb) {
ath11k_warn(ab, "rx monitor status with invalid buf_id %d\n",
buf_id);
- spin_unlock_bh(&rx_ring->idr_lock);
pmon->buf_state = DP_MON_STATUS_REPLINISH;
goto move_next;
}
- idr_remove(&rx_ring->bufs_idr, buf_id);
- spin_unlock_bh(&rx_ring->idr_lock);
-
rxcb = ATH11K_SKB_RXCB(skb);
- dma_unmap_single(ab->dev, rxcb->paddr,
- skb->len + skb_tailroom(skb),
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ab->dev, rxcb->paddr,
+ skb->len + skb_tailroom(skb),
+ DMA_FROM_DEVICE);
tlv = (struct hal_tlv_hdr *)skb->data;
if (FIELD_GET(HAL_TLV_HDR_TAG, tlv->tl) !=
HAL_RX_STATUS_BUFFER_DONE) {
- ath11k_warn(ab, "mon status DONE not set %lx\n",
+ ath11k_warn(ab, "mon status DONE not set %lx, buf_id %d\n",
FIELD_GET(HAL_TLV_HDR_TAG,
- tlv->tl));
- dev_kfree_skb_any(skb);
+ tlv->tl), buf_id);
+ /* If done status is missing, hold onto status
+ * ring until status is done for this status
+ * ring buffer.
+ * Keep HP in mon_status_ring unchanged,
+ * and break from here.
+ * Check status for same buffer for next time
+ */
pmon->buf_state = DP_MON_STATUS_NO_DMA;
- goto move_next;
+ break;
}
+ spin_lock_bh(&rx_ring->idr_lock);
+ idr_remove(&rx_ring->bufs_idr, buf_id);
+ spin_unlock_bh(&rx_ring->idr_lock);
if (ab->hw_params.full_monitor_mode) {
ath11k_dp_rx_mon_update_status_buf_state(pmon, tlv);
if (paddr == pmon->mon_status_paddr)
pmon->buf_state = DP_MON_STATUS_MATCH;
}
+
+ dma_unmap_single(ab->dev, rxcb->paddr,
+ skb->len + skb_tailroom(skb),
+ DMA_FROM_DEVICE);
+
__skb_queue_tail(skb_list, skb);
} else {
pmon->buf_state = DP_MON_STATUS_REPLINISH;
@@ -3117,8 +3132,11 @@ int ath11k_peer_rx_frag_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id
int i;
tfm = crypto_alloc_shash("michael_mic", 0, 0);
- if (IS_ERR(tfm))
+ if (IS_ERR(tfm)) {
+ ath11k_warn(ab, "failed to allocate michael_mic shash: %ld\n",
+ PTR_ERR(tfm));
return PTR_ERR(tfm);
+ }
spin_lock_bh(&ab->base_lock);
@@ -3138,6 +3156,7 @@ int ath11k_peer_rx_frag_setup(struct ath11k *ar, const u8 *peer_mac, int vdev_id
}
peer->tfm_mmic = tfm;
+ peer->dp_setup_done = true;
spin_unlock_bh(&ab->base_lock);
return 0;
@@ -3583,6 +3602,13 @@ static int ath11k_dp_rx_frag_h_mpdu(struct ath11k *ar,
ret = -ENOENT;
goto out_unlock;
}
+ if (!peer->dp_setup_done) {
+ ath11k_warn(ab, "The peer %pM [%d] has uninitialized datapath\n",
+ peer->addr, peer_id);
+ ret = -ENOENT;
+ goto out_unlock;
+ }
+
rx_tid = &peer->rx_tid[tid];
if ((!skb_queue_empty(&rx_tid->rx_frags) && seqno != rx_tid->cur_sn) ||
@@ -3598,7 +3624,7 @@ static int ath11k_dp_rx_frag_h_mpdu(struct ath11k *ar,
goto out_unlock;
}
- if (frag_no > __fls(rx_tid->rx_frag_bitmap))
+ if (!rx_tid->rx_frag_bitmap || (frag_no > __fls(rx_tid->rx_frag_bitmap)))
__skb_queue_tail(&rx_tid->rx_frags, msdu);
else
ath11k_dp_rx_h_sort_frags(ar, &rx_tid->rx_frags, msdu);
diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.c b/drivers/net/wireless/ath/ath11k/dp_tx.c
index 8afbba2..08a2846 100644
--- a/drivers/net/wireless/ath/ath11k/dp_tx.c
+++ b/drivers/net/wireless/ath/ath11k/dp_tx.c
@@ -316,10 +316,12 @@ ath11k_dp_tx_htt_tx_complete_buf(struct ath11k_base *ab,
struct dp_tx_ring *tx_ring,
struct ath11k_dp_htt_wbm_tx_status *ts)
{
+ struct ieee80211_tx_status status = { 0 };
struct sk_buff *msdu;
struct ieee80211_tx_info *info;
struct ath11k_skb_cb *skb_cb;
struct ath11k *ar;
+ struct ath11k_peer *peer;
spin_lock(&tx_ring->tx_idr_lock);
msdu = idr_remove(&tx_ring->txbuf_idr, ts->msdu_id);
@@ -341,6 +343,11 @@ ath11k_dp_tx_htt_tx_complete_buf(struct ath11k_base *ab,
dma_unmap_single(ab->dev, skb_cb->paddr, msdu->len, DMA_TO_DEVICE);
+ if (!skb_cb->vif) {
+ dev_kfree_skb_any(msdu);
+ return;
+ }
+
memset(&info->status, 0, sizeof(info->status));
if (ts->acked) {
@@ -355,7 +362,23 @@ ath11k_dp_tx_htt_tx_complete_buf(struct ath11k_base *ab,
}
}
- ieee80211_tx_status(ar->hw, msdu);
+ spin_lock_bh(&ab->base_lock);
+ peer = ath11k_peer_find_by_id(ab, ts->peer_id);
+ if (!peer || !peer->sta) {
+ ath11k_dbg(ab, ATH11K_DBG_DATA,
+ "dp_tx: failed to find the peer with peer_id %d\n",
+ ts->peer_id);
+ spin_unlock_bh(&ab->base_lock);
+ dev_kfree_skb_any(msdu);
+ return;
+ }
+ spin_unlock_bh(&ab->base_lock);
+
+ status.sta = peer->sta;
+ status.info = info;
+ status.skb = msdu;
+
+ ieee80211_tx_status_ext(ar->hw, &status);
}
static void
@@ -379,7 +402,15 @@ ath11k_dp_tx_process_htt_tx_complete(struct ath11k_base *ab,
ts.msdu_id = msdu_id;
ts.ack_rssi = FIELD_GET(HTT_TX_WBM_COMP_INFO1_ACK_RSSI,
status_desc->info1);
+
+ if (FIELD_GET(HTT_TX_WBM_COMP_INFO2_VALID, status_desc->info2))
+ ts.peer_id = FIELD_GET(HTT_TX_WBM_COMP_INFO2_SW_PEER_ID,
+ status_desc->info2);
+ else
+ ts.peer_id = HTT_INVALID_PEER_ID;
+
ath11k_dp_tx_htt_tx_complete_buf(ab, tx_ring, &ts);
+
break;
case HAL_WBM_REL_HTT_TX_COMP_STATUS_REINJ:
case HAL_WBM_REL_HTT_TX_COMP_STATUS_INSPECT:
diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.h b/drivers/net/wireless/ath/ath11k/dp_tx.h
index e87d65b..68a21ea 100644
--- a/drivers/net/wireless/ath/ath11k/dp_tx.h
+++ b/drivers/net/wireless/ath/ath11k/dp_tx.h
@@ -13,6 +13,7 @@ struct ath11k_dp_htt_wbm_tx_status {
u32 msdu_id;
bool acked;
int ack_rssi;
+ u16 peer_id;
};
void ath11k_dp_tx_update_txcompl(struct ath11k *ar, struct hal_tx_status *ts);
diff --git a/drivers/net/wireless/ath/ath11k/hal_rx.c b/drivers/net/wireless/ath/ath11k/hal_rx.c
index 7f39c6f..bb1d400 100644
--- a/drivers/net/wireless/ath/ath11k/hal_rx.c
+++ b/drivers/net/wireless/ath/ath11k/hal_rx.c
@@ -865,6 +865,12 @@ ath11k_hal_rx_populate_mu_user_info(void *rx_tlv, struct hal_rx_mon_ppdu_info *p
ath11k_hal_rx_populate_byte_count(rx_tlv, ppdu_info, rx_user_status);
}
+static u16 ath11k_hal_rx_mpduinfo_get_peerid(struct ath11k_base *ab,
+ struct hal_rx_mpdu_info *mpdu_info)
+{
+ return ab->hw_params.hw_ops->mpdu_info_get_peerid(mpdu_info);
+}
+
static enum hal_rx_mon_status
ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
struct hal_rx_mon_ppdu_info *ppdu_info,
@@ -1023,7 +1029,7 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
info1 = __le32_to_cpu(vht_sig->info1);
ppdu_info->ldpc = FIELD_GET(HAL_RX_VHT_SIG_A_INFO_INFO1_SU_MU_CODING,
- info0);
+ info1);
ppdu_info->mcs = FIELD_GET(HAL_RX_VHT_SIG_A_INFO_INFO1_MCS,
info1);
gi_setting = FIELD_GET(HAL_RX_VHT_SIG_A_INFO_INFO1_GI_SETTING,
@@ -1446,7 +1452,7 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
* PHYRX_OTHER_RECEIVE_INFO TLV.
*/
ppdu_info->rssi_comb =
- FIELD_GET(HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO1_RSSI_COMB,
+ FIELD_GET(HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO0_RSSI_COMB,
__le32_to_cpu(rssi->info0));
if (db2dbm) {
@@ -1459,9 +1465,11 @@ ath11k_hal_rx_parse_mon_status_tlv(struct ath11k_base *ab,
break;
}
case HAL_RX_MPDU_START: {
+ struct hal_rx_mpdu_info *mpdu_info =
+ (struct hal_rx_mpdu_info *)tlv_data;
u16 peer_id;
- peer_id = ab->hw_params.hw_ops->mpdu_info_get_peerid(tlv_data);
+ peer_id = ath11k_hal_rx_mpduinfo_get_peerid(ab, mpdu_info);
if (peer_id)
ppdu_info->peer_id = peer_id;
break;
diff --git a/drivers/net/wireless/ath/ath11k/hal_rx.h b/drivers/net/wireless/ath/ath11k/hal_rx.h
index f6bae07..61bd841 100644
--- a/drivers/net/wireless/ath/ath11k/hal_rx.h
+++ b/drivers/net/wireless/ath/ath11k/hal_rx.h
@@ -385,7 +385,7 @@ struct hal_rx_he_sig_b2_ofdma_info {
__le32 info0;
} __packed;
-#define HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO1_RSSI_COMB GENMASK(15, 8)
+#define HAL_RX_PHYRX_RSSI_LEGACY_INFO_INFO0_RSSI_COMB GENMASK(15, 8)
#define HAL_RX_PHYRX_RSSI_PREAMBLE_PRI20 GENMASK(7, 0)
@@ -405,7 +405,7 @@ struct hal_rx_phyrx_rssi_legacy_info {
#define HAL_RX_MPDU_INFO_INFO0_PEERID_WCN6855 GENMASK(15, 0)
#define HAL_RX_MPDU_INFO_INFO1_MPDU_LEN GENMASK(13, 0)
-struct hal_rx_mpdu_info {
+struct hal_rx_mpdu_info_ipq8074 {
__le32 rsvd0;
__le32 info0;
__le32 rsvd1[11];
@@ -413,12 +413,28 @@ struct hal_rx_mpdu_info {
__le32 rsvd2[9];
} __packed;
+struct hal_rx_mpdu_info_qcn9074 {
+ __le32 rsvd0[10];
+ __le32 info0;
+ __le32 rsvd1[2];
+ __le32 info1;
+ __le32 rsvd2[9];
+} __packed;
+
struct hal_rx_mpdu_info_wcn6855 {
__le32 rsvd0[8];
__le32 info0;
__le32 rsvd1[14];
} __packed;
+struct hal_rx_mpdu_info {
+ union {
+ struct hal_rx_mpdu_info_ipq8074 ipq8074;
+ struct hal_rx_mpdu_info_qcn9074 qcn9074;
+ struct hal_rx_mpdu_info_wcn6855 wcn6855;
+ } u;
+} __packed;
+
#define HAL_RX_PPDU_END_DURATION GENMASK(23, 0)
struct hal_rx_ppdu_end_duration {
__le32 rsvd0[9];
diff --git a/drivers/net/wireless/ath/ath11k/hw.c b/drivers/net/wireless/ath/ath11k/hw.c
index 60ac215..eb995f9 100644
--- a/drivers/net/wireless/ath/ath11k/hw.c
+++ b/drivers/net/wireless/ath/ath11k/hw.c
@@ -835,26 +835,35 @@ static void ath11k_hw_ipq5018_reo_setup(struct ath11k_base *ab)
ring_hash_map);
}
-static u16 ath11k_hw_ipq8074_mpdu_info_get_peerid(u8 *tlv_data)
+static u16
+ath11k_hw_ipq8074_mpdu_info_get_peerid(struct hal_rx_mpdu_info *mpdu_info)
{
u16 peer_id = 0;
- struct hal_rx_mpdu_info *mpdu_info =
- (struct hal_rx_mpdu_info *)tlv_data;
peer_id = FIELD_GET(HAL_RX_MPDU_INFO_INFO0_PEERID,
- __le32_to_cpu(mpdu_info->info0));
+ __le32_to_cpu(mpdu_info->u.ipq8074.info0));
return peer_id;
}
-static u16 ath11k_hw_wcn6855_mpdu_info_get_peerid(u8 *tlv_data)
+static u16
+ath11k_hw_qcn9074_mpdu_info_get_peerid(struct hal_rx_mpdu_info *mpdu_info)
{
u16 peer_id = 0;
- struct hal_rx_mpdu_info_wcn6855 *mpdu_info =
- (struct hal_rx_mpdu_info_wcn6855 *)tlv_data;
+
+ peer_id = FIELD_GET(HAL_RX_MPDU_INFO_INFO0_PEERID,
+ __le32_to_cpu(mpdu_info->u.qcn9074.info0));
+
+ return peer_id;
+}
+
+static u16
+ath11k_hw_wcn6855_mpdu_info_get_peerid(struct hal_rx_mpdu_info *mpdu_info)
+{
+ u16 peer_id = 0;
peer_id = FIELD_GET(HAL_RX_MPDU_INFO_INFO0_PEERID_WCN6855,
- __le32_to_cpu(mpdu_info->info0));
+ __le32_to_cpu(mpdu_info->u.wcn6855.info0));
return peer_id;
}
@@ -1042,7 +1051,7 @@ const struct ath11k_hw_ops qcn9074_ops = {
.rx_desc_get_attention = ath11k_hw_qcn9074_rx_desc_get_attention,
.rx_desc_get_msdu_payload = ath11k_hw_qcn9074_rx_desc_get_msdu_payload,
.reo_setup = ath11k_hw_ipq8074_reo_setup,
- .mpdu_info_get_peerid = ath11k_hw_ipq8074_mpdu_info_get_peerid,
+ .mpdu_info_get_peerid = ath11k_hw_qcn9074_mpdu_info_get_peerid,
.rx_desc_mac_addr2_valid = ath11k_hw_ipq9074_rx_desc_mac_addr2_valid,
.rx_desc_mpdu_start_addr2 = ath11k_hw_ipq9074_rx_desc_mpdu_start_addr2,
.get_ring_selector = ath11k_hw_ipq8074_get_tcl_ring_selector,
@@ -1224,6 +1233,7 @@ const struct ath11k_hw_ring_mask ath11k_hw_ring_mask_ipq8074 = {
ATH11K_RX_WBM_REL_RING_MASK_0,
},
.reo_status = {
+ 0, 0, 0,
ATH11K_REO_STATUS_RING_MASK_0,
},
.rxdma2host = {
diff --git a/drivers/net/wireless/ath/ath11k/hw.h b/drivers/net/wireless/ath/ath11k/hw.h
index 0be4e12..6a5dd2d 100644
--- a/drivers/net/wireless/ath/ath11k/hw.h
+++ b/drivers/net/wireless/ath/ath11k/hw.h
@@ -224,7 +224,6 @@ struct ath11k_hw_params {
u32 tx_ring_size;
bool smp2p_wow_exit;
bool support_fw_mac_sequence;
- bool ftm_responder;
};
struct ath11k_hw_ops {
@@ -264,7 +263,7 @@ struct ath11k_hw_ops {
struct rx_attention *(*rx_desc_get_attention)(struct hal_rx_desc *desc);
u8 *(*rx_desc_get_msdu_payload)(struct hal_rx_desc *desc);
void (*reo_setup)(struct ath11k_base *ab);
- u16 (*mpdu_info_get_peerid)(u8 *tlv_data);
+ u16 (*mpdu_info_get_peerid)(struct hal_rx_mpdu_info *mpdu_info);
bool (*rx_desc_mac_addr2_valid)(struct hal_rx_desc *desc);
u8* (*rx_desc_mpdu_start_addr2)(struct hal_rx_desc *desc);
u32 (*get_ring_selector)(struct sk_buff *skb);
diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c
index cad832e..1c93f1a 100644
--- a/drivers/net/wireless/ath/ath11k/mac.c
+++ b/drivers/net/wireless/ath/ath11k/mac.c
@@ -3538,7 +3538,7 @@ static void ath11k_mac_op_bss_info_changed(struct ieee80211_hw *hw,
if (changed & BSS_CHANGED_FTM_RESPONDER &&
arvif->ftm_responder != info->ftm_responder &&
- ar->ab->hw_params.ftm_responder &&
+ test_bit(WMI_TLV_SERVICE_RTT, ar->ab->wmi_ab.svc_map) &&
(vif->type == NL80211_IFTYPE_AP ||
vif->type == NL80211_IFTYPE_MESH_POINT)) {
arvif->ftm_responder = info->ftm_responder;
@@ -3755,6 +3755,18 @@ static int ath11k_mac_op_hw_scan(struct ieee80211_hw *hw,
int i;
u32 scan_timeout;
+ /* Firmwares advertising the support of triggering 11D algorithm
+ * on the scan results of a regular scan expects driver to send
+ * WMI_11D_SCAN_START_CMDID before sending WMI_START_SCAN_CMDID.
+ * With this feature, separate 11D scan can be avoided since
+ * regdomain can be determined with the scan results of the
+ * regular scan.
+ */
+ if (ar->state_11d == ATH11K_11D_PREPARING &&
+ test_bit(WMI_TLV_SERVICE_SUPPORT_11D_FOR_HOST_SCAN,
+ ar->ab->wmi_ab.svc_map))
+ ath11k_mac_11d_scan_start(ar, arvif->vdev_id);
+
mutex_lock(&ar->conf_mutex);
spin_lock_bh(&ar->data_lock);
@@ -3819,8 +3831,29 @@ static int ath11k_mac_op_hw_scan(struct ieee80211_hw *hw,
goto exit;
}
- for (i = 0; i < arg->num_chan; i++)
- arg->chan_list[i] = req->channels[i]->center_freq;
+ for (i = 0; i < arg->num_chan; i++) {
+ if (test_bit(WMI_TLV_SERVICE_SCAN_CONFIG_PER_CHANNEL,
+ ar->ab->wmi_ab.svc_map)) {
+ arg->chan_list[i] =
+ u32_encode_bits(req->channels[i]->center_freq,
+ WMI_SCAN_CONFIG_PER_CHANNEL_MASK);
+
+ /* If NL80211_SCAN_FLAG_COLOCATED_6GHZ is set in scan
+ * flags, then scan all PSC channels in 6 GHz band and
+ * those non-PSC channels where RNR IE is found during
+ * the legacy 2.4/5 GHz scan.
+ * If NL80211_SCAN_FLAG_COLOCATED_6GHZ is not set,
+ * then all channels in 6 GHz will be scanned.
+ */
+ if (req->channels[i]->band == NL80211_BAND_6GHZ &&
+ req->flags & NL80211_SCAN_FLAG_COLOCATED_6GHZ &&
+ !cfg80211_channel_is_psc(req->channels[i]))
+ arg->chan_list[i] |=
+ WMI_SCAN_CH_FLAG_SCAN_ONLY_IF_RNR_FOUND;
+ } else {
+ arg->chan_list[i] = req->channels[i]->center_freq;
+ }
+ }
}
if (req->flags & NL80211_SCAN_FLAG_RANDOM_ADDR) {
@@ -5552,10 +5585,6 @@ static int ath11k_mac_copy_he_cap(struct ath11k *ar,
he_cap_elem->mac_cap_info[1] &=
IEEE80211_HE_MAC_CAP1_TF_MAC_PAD_DUR_MASK;
- he_cap_elem->phy_cap_info[0] &=
- ~IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G;
- he_cap_elem->phy_cap_info[0] &=
- ~IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_80PLUS80_MHZ_IN_5G;
he_cap_elem->phy_cap_info[5] &=
~IEEE80211_HE_PHY_CAP5_BEAMFORMEE_NUM_SND_DIM_UNDER_80MHZ_MASK;
@@ -6652,6 +6681,11 @@ static void ath11k_mac_op_remove_interface(struct ieee80211_hw *hw,
ath11k_dbg(ab, ATH11K_DBG_MAC, "mac remove interface (vdev %d)\n",
arvif->vdev_id);
+ ret = ath11k_spectral_vif_stop(arvif);
+ if (ret)
+ ath11k_warn(ab, "failed to stop spectral for vdev %i: %d\n",
+ arvif->vdev_id, ret);
+
if (arvif->vdev_type == WMI_VDEV_TYPE_STA)
ath11k_mac_11d_scan_stop(ar);
@@ -9213,7 +9247,7 @@ static int __ath11k_mac_register(struct ath11k *ar)
wiphy_ext_feature_set(ar->hw->wiphy,
NL80211_EXT_FEATURE_SET_SCAN_DWELL);
- if (ab->hw_params.ftm_responder)
+ if (test_bit(WMI_TLV_SERVICE_RTT, ar->ab->wmi_ab.svc_map))
wiphy_ext_feature_set(ar->hw->wiphy,
NL80211_EXT_FEATURE_ENABLE_FTM_RESPONDER);
diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c
index 0aeef29..7b33731 100644
--- a/drivers/net/wireless/ath/ath11k/pci.c
+++ b/drivers/net/wireless/ath/ath11k/pci.c
@@ -540,7 +540,7 @@ static int ath11k_pci_claim(struct ath11k_pci *ab_pci, struct pci_dev *pdev)
if (!ab->mem) {
ath11k_err(ab, "failed to map pci bar %d\n", ATH11K_PCI_BAR_NUM);
ret = -EIO;
- goto clear_master;
+ goto release_region;
}
ab->mem_ce = ab->mem;
@@ -548,8 +548,6 @@ static int ath11k_pci_claim(struct ath11k_pci *ab_pci, struct pci_dev *pdev)
ath11k_dbg(ab, ATH11K_DBG_BOOT, "boot pci_mem 0x%pK\n", ab->mem);
return 0;
-clear_master:
- pci_clear_master(pdev);
release_region:
pci_release_region(pdev, ATH11K_PCI_BAR_NUM);
disable_device:
@@ -565,7 +563,6 @@ static void ath11k_pci_free_region(struct ath11k_pci *ab_pci)
pci_iounmap(pci_dev, ab->mem);
ab->mem = NULL;
- pci_clear_master(pci_dev);
pci_release_region(pci_dev, ATH11K_PCI_BAR_NUM);
if (pci_is_enabled(pci_dev))
pci_disable_device(pci_dev);
@@ -1039,7 +1036,8 @@ module_exit(ath11k_pci_exit);
MODULE_DESCRIPTION("Driver support for Qualcomm Technologies 802.11ax WLAN PCIe devices");
MODULE_LICENSE("Dual BSD/GPL");
-/* QCA639x 2.0 firmware files */
-MODULE_FIRMWARE(ATH11K_FW_DIR "/QCA6390/hw2.0/" ATH11K_BOARD_API2_FILE);
-MODULE_FIRMWARE(ATH11K_FW_DIR "/QCA6390/hw2.0/" ATH11K_AMSS_FILE);
-MODULE_FIRMWARE(ATH11K_FW_DIR "/QCA6390/hw2.0/" ATH11K_M3_FILE);
+/* firmware files */
+MODULE_FIRMWARE(ATH11K_FW_DIR "/QCA6390/hw2.0/*");
+MODULE_FIRMWARE(ATH11K_FW_DIR "/QCN9074/hw1.0/*");
+MODULE_FIRMWARE(ATH11K_FW_DIR "/WCN6855/hw2.0/*");
+MODULE_FIRMWARE(ATH11K_FW_DIR "/WCN6855/hw2.1/*");
diff --git a/drivers/net/wireless/ath/ath11k/peer.h b/drivers/net/wireless/ath/ath11k/peer.h
index 6dd17ba..9bd385d 100644
--- a/drivers/net/wireless/ath/ath11k/peer.h
+++ b/drivers/net/wireless/ath/ath11k/peer.h
@@ -35,6 +35,7 @@ struct ath11k_peer {
u16 sec_type;
u16 sec_type_grp;
bool is_authorized;
+ bool dp_setup_done;
};
void ath11k_peer_unmap_event(struct ath11k_base *ab, u16 peer_id);
diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c
index 27f3fce..d0b59bc 100644
--- a/drivers/net/wireless/ath/ath11k/wmi.c
+++ b/drivers/net/wireless/ath/ath11k/wmi.c
@@ -82,6 +82,12 @@ struct wmi_tlv_fw_stats_parse {
bool chain_rssi_done;
};
+struct wmi_tlv_mgmt_rx_parse {
+ const struct wmi_mgmt_rx_hdr *fixed;
+ const u8 *frame_buf;
+ bool frame_buf_done;
+};
+
static const struct wmi_tlv_policy wmi_tlv_policies[] = {
[WMI_TAG_ARRAY_BYTE]
= { .min_len = 0 },
@@ -865,7 +871,8 @@ static void ath11k_wmi_put_wmi_channel(struct wmi_channel *chan,
chan->band_center_freq2 = arg->channel.band_center_freq1;
- } else if (arg->channel.mode == MODE_11AC_VHT80_80) {
+ } else if ((arg->channel.mode == MODE_11AC_VHT80_80) ||
+ (arg->channel.mode == MODE_11AX_HE80_80)) {
chan->band_center_freq2 = arg->channel.band_center_freq2;
} else {
chan->band_center_freq2 = 0;
@@ -5633,28 +5640,49 @@ static int ath11k_pull_vdev_stopped_param_tlv(struct ath11k_base *ab, struct sk_
return 0;
}
+static int ath11k_wmi_tlv_mgmt_rx_parse(struct ath11k_base *ab,
+ u16 tag, u16 len,
+ const void *ptr, void *data)
+{
+ struct wmi_tlv_mgmt_rx_parse *parse = data;
+
+ switch (tag) {
+ case WMI_TAG_MGMT_RX_HDR:
+ parse->fixed = ptr;
+ break;
+ case WMI_TAG_ARRAY_BYTE:
+ if (!parse->frame_buf_done) {
+ parse->frame_buf = ptr;
+ parse->frame_buf_done = true;
+ }
+ break;
+ }
+ return 0;
+}
+
static int ath11k_pull_mgmt_rx_params_tlv(struct ath11k_base *ab,
struct sk_buff *skb,
struct mgmt_rx_event_params *hdr)
{
- const void **tb;
+ struct wmi_tlv_mgmt_rx_parse parse = { };
const struct wmi_mgmt_rx_hdr *ev;
const u8 *frame;
int ret;
- tb = ath11k_wmi_tlv_parse_alloc(ab, skb->data, skb->len, GFP_ATOMIC);
- if (IS_ERR(tb)) {
- ret = PTR_ERR(tb);
- ath11k_warn(ab, "failed to parse tlv: %d\n", ret);
+ ret = ath11k_wmi_tlv_iter(ab, skb->data, skb->len,
+ ath11k_wmi_tlv_mgmt_rx_parse,
+ &parse);
+ if (ret) {
+ ath11k_warn(ab, "failed to parse mgmt rx tlv %d\n",
+ ret);
return ret;
}
- ev = tb[WMI_TAG_MGMT_RX_HDR];
- frame = tb[WMI_TAG_ARRAY_BYTE];
+ ev = parse.fixed;
+ frame = parse.frame_buf;
if (!ev || !frame) {
ath11k_warn(ab, "failed to fetch mgmt rx hdr");
- kfree(tb);
return -EPROTO;
}
@@ -5673,7 +5701,6 @@ static int ath11k_pull_mgmt_rx_params_tlv(struct ath11k_base *ab,
if (skb->len < (frame - skb->data) + hdr->buf_len) {
ath11k_warn(ab, "invalid length in mgmt rx hdr ev");
- kfree(tb);
return -EPROTO;
}
@@ -5685,7 +5712,6 @@ static int ath11k_pull_mgmt_rx_params_tlv(struct ath11k_base *ab,
ath11k_ce_byte_swap(skb->data, hdr->buf_len);
- kfree(tb);
return 0;
}
diff --git a/drivers/net/wireless/ath/ath11k/wmi.h b/drivers/net/wireless/ath/ath11k/wmi.h
index b23b7a22..92fddb7 100644
--- a/drivers/net/wireless/ath/ath11k/wmi.h
+++ b/drivers/net/wireless/ath/ath11k/wmi.h
@@ -2100,8 +2100,10 @@ enum wmi_tlv_service {
/* The second 128 bits */
WMI_MAX_EXT_SERVICE = 256,
+ WMI_TLV_SERVICE_SCAN_CONFIG_PER_CHANNEL = 265,
WMI_TLV_SERVICE_REG_CC_EXT_EVENT_SUPPORT = 281,
WMI_TLV_SERVICE_BIOS_SAR_SUPPORT = 326,
+ WMI_TLV_SERVICE_SUPPORT_11D_FOR_HOST_SCAN = 357,
/* The third 128 bits */
WMI_MAX_EXT2_SERVICE = 384
@@ -3249,6 +3251,9 @@ struct wmi_start_scan_cmd {
#define WMI_SCAN_DWELL_MODE_SHIFT 21
#define WMI_SCAN_FLAG_EXT_PASSIVE_SCAN_START_TIME_ENHANCE 0x00000800
+#define WMI_SCAN_CONFIG_PER_CHANNEL_MASK GENMASK(19, 0)
+#define WMI_SCAN_CH_FLAG_SCAN_ONLY_IF_RNR_FOUND BIT(20)
+
enum {
WMI_SCAN_DWELL_MODE_DEFAULT = 0,
WMI_SCAN_DWELL_MODE_CONSERVATIVE = 1,
diff --git a/drivers/net/wireless/ath/ath12k/core.h b/drivers/net/wireless/ath/ath12k/core.h
index dffa687..9439052 100644
--- a/drivers/net/wireless/ath/ath12k/core.h
+++ b/drivers/net/wireless/ath/ath12k/core.h
@@ -395,6 +395,7 @@ struct ath12k_sta {
u8 rssi_comb;
struct ath12k_rx_peer_stats *rx_stats;
struct ath12k_wbm_tx_stats *wbm_tx_stats;
+ u32 bw_prev;
};
#define ATH12K_MIN_5G_FREQ 4150
diff --git a/drivers/net/wireless/ath/ath12k/dp_rx.c b/drivers/net/wireless/ath/ath12k/dp_rx.c
index 0adcbcf..e78478a 100644
--- a/drivers/net/wireless/ath/ath12k/dp_rx.c
+++ b/drivers/net/wireless/ath/ath12k/dp_rx.c
@@ -196,7 +196,8 @@ static void ath12k_dp_rxdesc_set_msdu_len(struct ath12k_base *ab,
static bool ath12k_dp_rx_h_is_mcbc(struct ath12k_base *ab,
struct hal_rx_desc *desc)
{
- return ab->hw_params->hal_ops->rx_desc_is_mcbc(desc);
+ return (ath12k_dp_rx_h_first_msdu(ab, desc) &&
+ ab->hw_params->hal_ops->rx_desc_is_mcbc(desc));
}
static bool ath12k_dp_rxdesc_mac_addr2_valid(struct ath12k_base *ab,
@@ -3047,10 +3048,14 @@ static int ath12k_dp_rx_h_defrag_reo_reinject(struct ath12k *ar,
reo_ent_ring->rx_mpdu_info.peer_meta_data =
reo_dest_ring->rx_mpdu_info.peer_meta_data;
- reo_ent_ring->queue_addr_lo = cpu_to_le32(lower_32_bits(rx_tid->paddr));
- reo_ent_ring->info0 = le32_encode_bits(upper_32_bits(rx_tid->paddr),
- HAL_REO_ENTR_RING_INFO0_QUEUE_ADDR_HI) |
- le32_encode_bits(dst_ind, HAL_REO_ENTR_RING_INFO0_DEST_IND);
+ /* Firmware expects physical address to be filled in queue_addr_lo in
+ * the MLO scenario and in case of non MLO peer meta data needs to be
+ * filled.
+ * TODO: Need to handle for MLO scenario.
+ */
+ reo_ent_ring->queue_addr_lo = reo_dest_ring->rx_mpdu_info.peer_meta_data;
+ reo_ent_ring->info0 = le32_encode_bits(dst_ind,
+ HAL_REO_ENTR_RING_INFO0_DEST_IND);
reo_ent_ring->info1 = le32_encode_bits(rx_tid->cur_sn,
HAL_REO_ENTR_RING_INFO1_MPDU_SEQ_NUM);
diff --git a/drivers/net/wireless/ath/ath12k/dp_tx.c b/drivers/net/wireless/ath/ath12k/dp_tx.c
index fd8d850..d3c7c76 100644
--- a/drivers/net/wireless/ath/ath12k/dp_tx.c
+++ b/drivers/net/wireless/ath/ath12k/dp_tx.c
@@ -13,6 +13,10 @@ static enum hal_tcl_encap_type
ath12k_dp_tx_get_encap_type(struct ath12k_vif *arvif, struct sk_buff *skb)
{
struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb);
+ struct ath12k_base *ab = arvif->ar->ab;
+
+ if (test_bit(ATH12K_FLAG_RAW_MODE, &ab->dev_flags))
+ return HAL_TCL_ENCAP_TYPE_RAW;
if (tx_info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP)
return HAL_TCL_ENCAP_TYPE_ETHERNET;
diff --git a/drivers/net/wireless/ath/ath12k/hw.c b/drivers/net/wireless/ath/ath12k/hw.c
index 91d576f..1ffac7e 100644
--- a/drivers/net/wireless/ath/ath12k/hw.c
+++ b/drivers/net/wireless/ath/ath12k/hw.c
@@ -944,7 +944,7 @@ static const struct ath12k_hw_params ath12k_hw_params[] = {
.interface_modes = BIT(NL80211_IFTYPE_STATION),
.supports_monitor = false,
- .idle_ps = false,
+ .idle_ps = true,
.download_calib = false,
.supports_suspend = false,
.tcl_ring_retry = false,
diff --git a/drivers/net/wireless/ath/ath12k/mac.c b/drivers/net/wireless/ath/ath12k/mac.c
index bf7e5b6..ee79282 100644
--- a/drivers/net/wireless/ath/ath12k/mac.c
+++ b/drivers/net/wireless/ath/ath12k/mac.c
@@ -3220,10 +3220,11 @@ static void ath12k_sta_rc_update_wk(struct work_struct *wk)
enum nl80211_band band;
const u8 *ht_mcs_mask;
const u16 *vht_mcs_mask;
- u32 changed, bw, nss, smps;
+ u32 changed, bw, nss, smps, bw_prev;
int err, num_vht_rates;
const struct cfg80211_bitrate_mask *mask;
struct ath12k_wmi_peer_assoc_arg peer_arg;
+ enum wmi_phy_mode peer_phymode;
arsta = container_of(wk, struct ath12k_sta, update_wk);
sta = container_of((void *)arsta, struct ieee80211_sta, drv_priv);
@@ -3243,6 +3244,7 @@ static void ath12k_sta_rc_update_wk(struct work_struct *wk)
arsta->changed = 0;
bw = arsta->bw;
+ bw_prev = arsta->bw_prev;
nss = arsta->nss;
smps = arsta->smps;
@@ -3255,11 +3257,53 @@ static void ath12k_sta_rc_update_wk(struct work_struct *wk)
ath12k_mac_max_vht_nss(vht_mcs_mask)));
if (changed & IEEE80211_RC_BW_CHANGED) {
- err = ath12k_wmi_set_peer_param(ar, sta->addr, arvif->vdev_id,
- WMI_PEER_CHWIDTH, bw);
- if (err)
- ath12k_warn(ar->ab, "failed to update STA %pM peer bw %d: %d\n",
- sta->addr, bw, err);
+ ath12k_peer_assoc_h_phymode(ar, arvif->vif, sta, &peer_arg);
+ peer_phymode = peer_arg.peer_phymode;
+
+ if (bw > bw_prev) {
+ /* Phymode shows maximum supported channel width, if we
+ * upgrade bandwidth then due to sanity check of firmware,
+ * we have to send WMI_PEER_PHYMODE followed by
+ * WMI_PEER_CHWIDTH
+ */
+ ath12k_dbg(ar->ab, ATH12K_DBG_MAC, "mac bandwidth upgrade for sta %pM new %d old %d\n",
+ sta->addr, bw, bw_prev);
+ err = ath12k_wmi_set_peer_param(ar, sta->addr,
+ arvif->vdev_id, WMI_PEER_PHYMODE,
+ peer_phymode);
+ if (err) {
+ ath12k_warn(ar->ab, "failed to update STA %pM to peer phymode %d: %d\n",
+ sta->addr, peer_phymode, err);
+ goto err_rc_bw_changed;
+ }
+ err = ath12k_wmi_set_peer_param(ar, sta->addr,
+ arvif->vdev_id, WMI_PEER_CHWIDTH,
+ bw);
+ if (err)
+ ath12k_warn(ar->ab, "failed to update STA %pM to peer bandwidth %d: %d\n",
+ sta->addr, bw, err);
+ } else {
+ /* When we downgrade bandwidth this will conflict with phymode
+ * and cause to trigger firmware crash. In this case we send
+ * WMI_PEER_CHWIDTH followed by WMI_PEER_PHYMODE
+ */
+ ath12k_dbg(ar->ab, ATH12K_DBG_MAC, "mac bandwidth downgrade for sta %pM new %d old %d\n",
+ sta->addr, bw, bw_prev);
+ err = ath12k_wmi_set_peer_param(ar, sta->addr,
+ arvif->vdev_id, WMI_PEER_CHWIDTH,
+ bw);
+ if (err) {
+ ath12k_warn(ar->ab, "failed to update STA %pM peer to bandwidth %d: %d\n",
+ sta->addr, bw, err);
+ goto err_rc_bw_changed;
+ }
+ err = ath12k_wmi_set_peer_param(ar, sta->addr,
+ arvif->vdev_id, WMI_PEER_PHYMODE,
+ peer_phymode);
+ if (err)
+ ath12k_warn(ar->ab, "failed to update STA %pM to peer phymode %d: %d\n",
+ sta->addr, peer_phymode, err);
+ }
}
if (changed & IEEE80211_RC_NSS_CHANGED) {
@@ -3321,7 +3365,7 @@ static void ath12k_sta_rc_update_wk(struct work_struct *wk)
sta->addr, arvif->vdev_id);
}
}
-
+err_rc_bw_changed:
mutex_unlock(&ar->conf_mutex);
}
@@ -3433,6 +3477,34 @@ static int ath12k_mac_station_add(struct ath12k *ar,
return ret;
}
+static u32 ath12k_mac_ieee80211_sta_bw_to_wmi(struct ath12k *ar,
+ struct ieee80211_sta *sta)
+{
+ u32 bw = WMI_PEER_CHWIDTH_20MHZ;
+
+ switch (sta->deflink.bandwidth) {
+ case IEEE80211_STA_RX_BW_20:
+ bw = WMI_PEER_CHWIDTH_20MHZ;
+ break;
+ case IEEE80211_STA_RX_BW_40:
+ bw = WMI_PEER_CHWIDTH_40MHZ;
+ break;
+ case IEEE80211_STA_RX_BW_80:
+ bw = WMI_PEER_CHWIDTH_80MHZ;
+ break;
+ case IEEE80211_STA_RX_BW_160:
+ bw = WMI_PEER_CHWIDTH_160MHZ;
+ break;
+ default:
+ ath12k_warn(ar->ab, "Invalid bandwidth %d in rc update for %pM\n",
+ sta->deflink.bandwidth, sta->addr);
+ bw = WMI_PEER_CHWIDTH_20MHZ;
+ break;
+ }
+
+ return bw;
+}
+
static int ath12k_mac_op_sta_state(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
@@ -3498,6 +3570,13 @@ static int ath12k_mac_op_sta_state(struct ieee80211_hw *hw,
if (ret)
ath12k_warn(ar->ab, "Failed to associate station: %pM\n",
sta->addr);
+
+ spin_lock_bh(&ar->data_lock);
+
+ arsta->bw = ath12k_mac_ieee80211_sta_bw_to_wmi(ar, sta);
+ arsta->bw_prev = sta->deflink.bandwidth;
+
+ spin_unlock_bh(&ar->data_lock);
} else if (old_state == IEEE80211_STA_ASSOC &&
new_state == IEEE80211_STA_AUTHORIZED) {
spin_lock_bh(&ar->ab->base_lock);
@@ -3607,28 +3686,8 @@ static void ath12k_mac_op_sta_rc_update(struct ieee80211_hw *hw,
spin_lock_bh(&ar->data_lock);
if (changed & IEEE80211_RC_BW_CHANGED) {
- bw = WMI_PEER_CHWIDTH_20MHZ;
-
- switch (sta->deflink.bandwidth) {
- case IEEE80211_STA_RX_BW_20:
- bw = WMI_PEER_CHWIDTH_20MHZ;
- break;
- case IEEE80211_STA_RX_BW_40:
- bw = WMI_PEER_CHWIDTH_40MHZ;
- break;
- case IEEE80211_STA_RX_BW_80:
- bw = WMI_PEER_CHWIDTH_80MHZ;
- break;
- case IEEE80211_STA_RX_BW_160:
- bw = WMI_PEER_CHWIDTH_160MHZ;
- break;
- default:
- ath12k_warn(ar->ab, "Invalid bandwidth %d in rc update for %pM\n",
- sta->deflink.bandwidth, sta->addr);
- bw = WMI_PEER_CHWIDTH_20MHZ;
- break;
- }
-
+ bw = ath12k_mac_ieee80211_sta_bw_to_wmi(ar, sta);
+ arsta->bw_prev = arsta->bw;
arsta->bw = bw;
}
diff --git a/drivers/net/wireless/ath/ath12k/pci.c b/drivers/net/wireless/ath/ath12k/pci.c
index d32637b..9f174da 100644
--- a/drivers/net/wireless/ath/ath12k/pci.c
+++ b/drivers/net/wireless/ath/ath12k/pci.c
@@ -755,14 +755,12 @@ static int ath12k_pci_claim(struct ath12k_pci *ab_pci, struct pci_dev *pdev)
if (!ab->mem) {
ath12k_err(ab, "failed to map pci bar %d\n", ATH12K_PCI_BAR_NUM);
ret = -EIO;
- goto clear_master;
+ goto release_region;
}
ath12k_dbg(ab, ATH12K_DBG_BOOT, "boot pci_mem 0x%pK\n", ab->mem);
return 0;
-clear_master:
- pci_clear_master(pdev);
release_region:
pci_release_region(pdev, ATH12K_PCI_BAR_NUM);
disable_device:
@@ -778,7 +776,6 @@ static void ath12k_pci_free_region(struct ath12k_pci *ab_pci)
pci_iounmap(pci_dev, ab->mem);
ab->mem = NULL;
- pci_clear_master(pci_dev);
pci_release_region(pci_dev, ATH12K_PCI_BAR_NUM);
if (pci_is_enabled(pci_dev))
pci_disable_device(pci_dev);
@@ -1223,7 +1220,8 @@ static int ath12k_pci_probe(struct pci_dev *pdev,
dev_err(&pdev->dev,
"Unknown hardware version found for QCN9274: 0x%x\n",
soc_hw_version_major);
- return -EOPNOTSUPP;
+ ret = -EOPNOTSUPP;
+ goto err_pci_free_region;
}
break;
case WCN7850_DEVICE_ID:
diff --git a/drivers/net/wireless/ath/ath12k/qmi.c b/drivers/net/wireless/ath/ath12k/qmi.c
index 979a63f..03ba245 100644
--- a/drivers/net/wireless/ath/ath12k/qmi.c
+++ b/drivers/net/wireless/ath/ath12k/qmi.c
@@ -2991,7 +2991,7 @@ static void ath12k_qmi_driver_event_work(struct work_struct *work)
spin_unlock(&qmi->event_lock);
if (test_bit(ATH12K_FLAG_UNREGISTERING, &ab->dev_flags))
- return;
+ goto skip;
switch (event->type) {
case ATH12K_QMI_EVENT_SERVER_ARRIVE:
@@ -3032,6 +3032,8 @@ static void ath12k_qmi_driver_event_work(struct work_struct *work)
ath12k_warn(ab, "invalid event type: %d", event->type);
break;
}
+
+skip:
kfree(event);
spin_lock(&qmi->event_lock);
}
diff --git a/drivers/net/wireless/ath/ath12k/wmi.c b/drivers/net/wireless/ath/ath12k/wmi.c
index 3e69911..7ae0bb78 100644
--- a/drivers/net/wireless/ath/ath12k/wmi.c
+++ b/drivers/net/wireless/ath/ath12k/wmi.c
@@ -2438,6 +2438,9 @@ int ath12k_wmi_send_scan_chan_list_cmd(struct ath12k *ar,
if (channel_arg->psc_channel)
chan_info->info |= cpu_to_le32(WMI_CHAN_INFO_PSC);
+ if (channel_arg->dfs_set)
+ chan_info->info |= cpu_to_le32(WMI_CHAN_INFO_DFS);
+
chan_info->info |= le32_encode_bits(channel_arg->phy_mode,
WMI_CHAN_INFO_MODE);
*reg1 |= le32_encode_bits(channel_arg->minpower,
@@ -4934,6 +4937,9 @@ static int freq_to_idx(struct ath12k *ar, int freq)
int band, ch, idx = 0;
for (band = NL80211_BAND_2GHZ; band < NUM_NL80211_BANDS; band++) {
+ if (!ar->mac.sbands[band].channels)
+ continue;
+
sband = ar->hw->wiphy->bands[band];
if (!sband)
continue;
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
index e0130be..27ff1ca 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.c
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
@@ -42,8 +42,6 @@ static const struct usb_device_id ath9k_hif_usb_ids[] = {
{ USB_DEVICE(0x0cf3, 0x7015),
.driver_info = AR9287_USB }, /* Atheros */
- { USB_DEVICE(0x1668, 0x1200),
- .driver_info = AR9287_USB }, /* Verizon */
{ USB_DEVICE(0x0cf3, 0x7010),
.driver_info = AR9280_USB }, /* Atheros */
diff --git a/drivers/net/wireless/ath/ath9k/mci.c b/drivers/net/wireless/ath/ath9k/mci.c
index 3363fc4..a084500 100644
--- a/drivers/net/wireless/ath/ath9k/mci.c
+++ b/drivers/net/wireless/ath/ath9k/mci.c
@@ -646,9 +646,7 @@ void ath9k_mci_update_wlan_channels(struct ath_softc *sc, bool allow_all)
struct ath_hw *ah = sc->sc_ah;
struct ath9k_hw_mci *mci = &ah->btcoex_hw.mci;
struct ath9k_channel *chan = ah->curchan;
- static const u32 channelmap[] = {
- 0x00000000, 0xffff0000, 0xffffffff, 0x7fffffff
- };
+ u32 channelmap[] = {0x00000000, 0xffff0000, 0xffffffff, 0x7fffffff};
int i;
s16 chan_start, chan_end;
u16 wlan_chan;
diff --git a/drivers/net/wireless/ath/ath9k/xmit.c b/drivers/net/wireless/ath/ath9k/xmit.c
index ef9a8e0..f6f2ab7 100644
--- a/drivers/net/wireless/ath/ath9k/xmit.c
+++ b/drivers/net/wireless/ath/ath9k/xmit.c
@@ -34,6 +34,12 @@
#define NUM_SYMBOLS_PER_USEC(_usec) (_usec >> 2)
#define NUM_SYMBOLS_PER_USEC_HALFGI(_usec) (((_usec*5)-4)/18)
+/* Shifts in ar5008_phy.c and ar9003_phy.c are equal for all revisions */
+#define ATH9K_PWRTBL_11NA_OFDM_SHIFT 0
+#define ATH9K_PWRTBL_11NG_OFDM_SHIFT 4
+#define ATH9K_PWRTBL_11NA_HT_SHIFT 8
+#define ATH9K_PWRTBL_11NG_HT_SHIFT 12
+
static u16 bits_per_symbol[][2] = {
/* 20MHz 40MHz */
@@ -1169,13 +1175,14 @@ void ath_update_max_aggr_framelen(struct ath_softc *sc, int queue, int txop)
}
static u8 ath_get_rate_txpower(struct ath_softc *sc, struct ath_buf *bf,
- u8 rateidx, bool is_40, bool is_cck)
+ u8 rateidx, bool is_40, bool is_cck, bool is_mcs)
{
u8 max_power;
struct sk_buff *skb;
struct ath_frame_info *fi;
struct ieee80211_tx_info *info;
struct ath_hw *ah = sc->sc_ah;
+ bool is_2ghz, is_5ghz, use_stbc;
if (sc->tx99_state || !ah->tpc_enabled)
return MAX_RATE_POWER;
@@ -1184,6 +1191,19 @@ static u8 ath_get_rate_txpower(struct ath_softc *sc, struct ath_buf *bf,
fi = get_frame_info(skb);
info = IEEE80211_SKB_CB(skb);
+ is_2ghz = info->band == NL80211_BAND_2GHZ;
+ is_5ghz = info->band == NL80211_BAND_5GHZ;
+ use_stbc = is_mcs && rateidx < 8 && (info->flags &
+ IEEE80211_TX_CTL_STBC);
+
+ if (is_mcs)
+ rateidx += is_5ghz ? ATH9K_PWRTBL_11NA_HT_SHIFT
+ : ATH9K_PWRTBL_11NG_HT_SHIFT;
+ else if (is_2ghz && !is_cck)
+ rateidx += ATH9K_PWRTBL_11NG_OFDM_SHIFT;
+ else
+ rateidx += ATH9K_PWRTBL_11NA_OFDM_SHIFT;
+
if (!AR_SREV_9300_20_OR_LATER(ah)) {
int txpower = fi->tx_power;
@@ -1193,10 +1213,8 @@ static u8 ath_get_rate_txpower(struct ath_softc *sc, struct ath_buf *bf,
u16 eeprom_rev = ah->eep_ops->get_eeprom_rev(ah);
if (eeprom_rev >= AR5416_EEP_MINOR_VER_2) {
- bool is_2ghz;
struct modal_eep_header *pmodal;
- is_2ghz = info->band == NL80211_BAND_2GHZ;
pmodal = &eep->modalHeader[is_2ghz];
power_ht40delta = pmodal->ht40PowerIncForPdadc;
} else {
@@ -1229,7 +1247,7 @@ static u8 ath_get_rate_txpower(struct ath_softc *sc, struct ath_buf *bf,
if (!max_power && !AR_SREV_9280_20_OR_LATER(ah))
max_power = 1;
} else if (!bf->bf_state.bfs_paprd) {
- if (rateidx < 8 && (info->flags & IEEE80211_TX_CTL_STBC))
+ if (use_stbc)
max_power = min_t(u8, ah->tx_power_stbc[rateidx],
fi->tx_power);
else
@@ -1319,7 +1337,7 @@ static void ath_buf_set_rate(struct ath_softc *sc, struct ath_buf *bf,
}
info->txpower[i] = ath_get_rate_txpower(sc, bf, rix,
- is_40, false);
+ is_40, false, true);
continue;
}
@@ -1350,7 +1368,7 @@ static void ath_buf_set_rate(struct ath_softc *sc, struct ath_buf *bf,
is_cck = IS_CCK_RATE(info->rates[i].Rate);
info->txpower[i] = ath_get_rate_txpower(sc, bf, rix, false,
- is_cck);
+ is_cck, false);
}
/* For AR5416 - RTS cannot be followed by a frame larger than 8K */
diff --git a/drivers/net/wireless/ath/carl9170/cmd.c b/drivers/net/wireless/ath/carl9170/cmd.c
index f2b4f53..b8ed193 100644
--- a/drivers/net/wireless/ath/carl9170/cmd.c
+++ b/drivers/net/wireless/ath/carl9170/cmd.c
@@ -120,7 +120,7 @@ struct carl9170_cmd *carl9170_cmd_buf(struct ar9170 *ar,
{
struct carl9170_cmd *tmp;
- tmp = kzalloc(sizeof(struct carl9170_cmd_head) + len, GFP_ATOMIC);
+ tmp = kzalloc(sizeof(*tmp), GFP_ATOMIC);
if (tmp) {
tmp->hdr.cmd = cmd;
tmp->hdr.len = len;
diff --git a/drivers/net/wireless/ath/carl9170/fwcmd.h b/drivers/net/wireless/ath/carl9170/fwcmd.h
index ff4b3b5..e5bcc36 100644
--- a/drivers/net/wireless/ath/carl9170/fwcmd.h
+++ b/drivers/net/wireless/ath/carl9170/fwcmd.h
@@ -320,9 +320,9 @@ struct carl9170_rsp {
struct carl9170_u32_list rreg_res;
struct carl9170_u32_list echo;
#ifdef __CARL9170FW__
- struct carl9170_tx_status tx_status[0];
+ DECLARE_FLEX_ARRAY(struct carl9170_tx_status, tx_status);
#endif /* __CARL9170FW__ */
- struct _carl9170_tx_status _tx_status[0];
+ DECLARE_FLEX_ARRAY(struct _carl9170_tx_status, _tx_status);
struct carl9170_gpio gpio;
struct carl9170_tsf_rsp tsf;
struct carl9170_psm psm;
diff --git a/drivers/net/wireless/ath/wcn36xx/dxe.c b/drivers/net/wireless/ath/wcn36xx/dxe.c
index 4e9e139..9013f05 100644
--- a/drivers/net/wireless/ath/wcn36xx/dxe.c
+++ b/drivers/net/wireless/ath/wcn36xx/dxe.c
@@ -112,8 +112,8 @@ int wcn36xx_dxe_alloc_ctl_blks(struct wcn36xx *wcn)
wcn->dxe_rx_l_ch.desc_num = WCN36XX_DXE_CH_DESC_NUMB_RX_L;
wcn->dxe_rx_h_ch.desc_num = WCN36XX_DXE_CH_DESC_NUMB_RX_H;
- wcn->dxe_tx_l_ch.dxe_wq = WCN36XX_DXE_WQ_TX_L;
- wcn->dxe_tx_h_ch.dxe_wq = WCN36XX_DXE_WQ_TX_H;
+ wcn->dxe_tx_l_ch.dxe_wq = WCN36XX_DXE_WQ_TX_L(wcn);
+ wcn->dxe_tx_h_ch.dxe_wq = WCN36XX_DXE_WQ_TX_H(wcn);
wcn->dxe_tx_l_ch.ctrl_bd = WCN36XX_DXE_CTRL_TX_L_BD;
wcn->dxe_tx_h_ch.ctrl_bd = WCN36XX_DXE_CTRL_TX_H_BD;
@@ -165,8 +165,9 @@ void wcn36xx_dxe_free_ctl_blks(struct wcn36xx *wcn)
wcn36xx_dxe_free_ctl_block(&wcn->dxe_rx_h_ch);
}
-static int wcn36xx_dxe_init_descs(struct device *dev, struct wcn36xx_dxe_ch *wcn_ch)
+static int wcn36xx_dxe_init_descs(struct wcn36xx *wcn, struct wcn36xx_dxe_ch *wcn_ch)
{
+ struct device *dev = wcn->dev;
struct wcn36xx_dxe_desc *cur_dxe = NULL;
struct wcn36xx_dxe_desc *prev_dxe = NULL;
struct wcn36xx_dxe_ctl *cur_ctl = NULL;
@@ -190,11 +191,11 @@ static int wcn36xx_dxe_init_descs(struct device *dev, struct wcn36xx_dxe_ch *wcn
switch (wcn_ch->ch_type) {
case WCN36XX_DXE_CH_TX_L:
cur_dxe->ctrl = WCN36XX_DXE_CTRL_TX_L;
- cur_dxe->dst_addr_l = WCN36XX_DXE_WQ_TX_L;
+ cur_dxe->dst_addr_l = WCN36XX_DXE_WQ_TX_L(wcn);
break;
case WCN36XX_DXE_CH_TX_H:
cur_dxe->ctrl = WCN36XX_DXE_CTRL_TX_H;
- cur_dxe->dst_addr_l = WCN36XX_DXE_WQ_TX_H;
+ cur_dxe->dst_addr_l = WCN36XX_DXE_WQ_TX_H(wcn);
break;
case WCN36XX_DXE_CH_RX_L:
cur_dxe->ctrl = WCN36XX_DXE_CTRL_RX_L;
@@ -914,7 +915,7 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
/***************************************/
/* Init descriptors for TX LOW channel */
/***************************************/
- ret = wcn36xx_dxe_init_descs(wcn->dev, &wcn->dxe_tx_l_ch);
+ ret = wcn36xx_dxe_init_descs(wcn, &wcn->dxe_tx_l_ch);
if (ret) {
dev_err(wcn->dev, "Error allocating descriptor\n");
return ret;
@@ -928,14 +929,14 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
/* Program DMA destination addr for TX LOW */
wcn36xx_dxe_write_register(wcn,
WCN36XX_DXE_CH_DEST_ADDR_TX_L,
- WCN36XX_DXE_WQ_TX_L);
+ WCN36XX_DXE_WQ_TX_L(wcn));
wcn36xx_dxe_read_register(wcn, WCN36XX_DXE_REG_CH_EN, ®_data);
/***************************************/
/* Init descriptors for TX HIGH channel */
/***************************************/
- ret = wcn36xx_dxe_init_descs(wcn->dev, &wcn->dxe_tx_h_ch);
+ ret = wcn36xx_dxe_init_descs(wcn, &wcn->dxe_tx_h_ch);
if (ret) {
dev_err(wcn->dev, "Error allocating descriptor\n");
goto out_err_txh_ch;
@@ -950,14 +951,14 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
/* Program DMA destination addr for TX HIGH */
wcn36xx_dxe_write_register(wcn,
WCN36XX_DXE_CH_DEST_ADDR_TX_H,
- WCN36XX_DXE_WQ_TX_H);
+ WCN36XX_DXE_WQ_TX_H(wcn));
wcn36xx_dxe_read_register(wcn, WCN36XX_DXE_REG_CH_EN, ®_data);
/***************************************/
/* Init descriptors for RX LOW channel */
/***************************************/
- ret = wcn36xx_dxe_init_descs(wcn->dev, &wcn->dxe_rx_l_ch);
+ ret = wcn36xx_dxe_init_descs(wcn, &wcn->dxe_rx_l_ch);
if (ret) {
dev_err(wcn->dev, "Error allocating descriptor\n");
goto out_err_rxl_ch;
@@ -988,7 +989,7 @@ int wcn36xx_dxe_init(struct wcn36xx *wcn)
/***************************************/
/* Init descriptors for RX HIGH channel */
/***************************************/
- ret = wcn36xx_dxe_init_descs(wcn->dev, &wcn->dxe_rx_h_ch);
+ ret = wcn36xx_dxe_init_descs(wcn, &wcn->dxe_rx_h_ch);
if (ret) {
dev_err(wcn->dev, "Error allocating descriptor\n");
goto out_err_rxh_ch;
diff --git a/drivers/net/wireless/ath/wcn36xx/dxe.h b/drivers/net/wireless/ath/wcn36xx/dxe.h
index 26a31ed..dd8c684a3 100644
--- a/drivers/net/wireless/ath/wcn36xx/dxe.h
+++ b/drivers/net/wireless/ath/wcn36xx/dxe.h
@@ -135,8 +135,8 @@ H2H_TEST_RX_TX = DMA2
WCN36xx_DXE_CTRL_ENDIANNESS)
/* TODO This must calculated properly but not hardcoded */
-#define WCN36XX_DXE_WQ_TX_L 0x17
-#define WCN36XX_DXE_WQ_TX_H 0x17
+#define WCN36XX_DXE_WQ_TX_L(wcn) ((wcn)->is_pronto_v3 ? 0x6 : 0x17)
+#define WCN36XX_DXE_WQ_TX_H(wcn) ((wcn)->is_pronto_v3 ? 0x6 : 0x17)
#define WCN36XX_DXE_WQ_RX_L 0xB
#define WCN36XX_DXE_WQ_RX_H 0x4
diff --git a/drivers/net/wireless/ath/wcn36xx/main.c b/drivers/net/wireless/ath/wcn36xx/main.c
index 3b79cc1..8dbd115 100644
--- a/drivers/net/wireless/ath/wcn36xx/main.c
+++ b/drivers/net/wireless/ath/wcn36xx/main.c
@@ -1508,6 +1508,7 @@ static int wcn36xx_platform_get_resources(struct wcn36xx *wcn,
}
wcn->is_pronto = !!of_device_is_compatible(mmio_node, "qcom,pronto");
+ wcn->is_pronto_v3 = !!of_device_is_compatible(mmio_node, "qcom,pronto-v3-pil");
/* Map the CCU memory */
index = of_property_match_string(mmio_node, "reg-names", "ccu");
diff --git a/drivers/net/wireless/ath/wcn36xx/wcn36xx.h b/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
index 9aa08b6..ff4a8e5 100644
--- a/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
+++ b/drivers/net/wireless/ath/wcn36xx/wcn36xx.h
@@ -217,6 +217,7 @@ struct wcn36xx {
u8 fw_major;
u32 fw_feat_caps[WCN36XX_HAL_CAPS_SIZE];
bool is_pronto;
+ bool is_pronto_v3;
/* extra byte for the NULL termination */
u8 crm_version[WCN36XX_HAL_VERSION_LENGTH + 1];
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
index 65d4799..ff710b0 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
@@ -965,6 +965,12 @@ int brcmf_sdiod_probe(struct brcmf_sdio_dev *sdiodev)
.driver_data = BRCMF_FWVENDOR_ ## fw_vend \
}
+#define CYW_SDIO_DEVICE(dev_id, fw_vend) \
+ { \
+ SDIO_DEVICE(SDIO_VENDOR_ID_CYPRESS, dev_id), \
+ .driver_data = BRCMF_FWVENDOR_ ## fw_vend \
+ }
+
/* devices we support, null terminated */
static const struct sdio_device_id brcmf_sdmmc_ids[] = {
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43143, WCC),
@@ -979,6 +985,7 @@ static const struct sdio_device_id brcmf_sdmmc_ids[] = {
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4335_4339, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4339, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43430, WCC),
+ BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43439, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4345, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43455, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4354, WCC),
@@ -986,9 +993,9 @@ static const struct sdio_device_id brcmf_sdmmc_ids[] = {
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4359, WCC),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373, CYW),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43012, CYW),
- BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439, CYW),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752, CYW),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_89359, CYW),
+ CYW_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439, CYW),
{ /* end: all zeroes */ }
};
MODULE_DEVICE_TABLE(sdio, brcmf_sdmmc_ids);
diff --git a/drivers/net/wireless/cisco/Kconfig b/drivers/net/wireless/cisco/Kconfig
index 681bfc2..b40ee25 100644
--- a/drivers/net/wireless/cisco/Kconfig
+++ b/drivers/net/wireless/cisco/Kconfig
@@ -14,7 +14,7 @@
config AIRO
tristate "Cisco/Aironet 34X/35X/4500/4800 ISA and PCI cards"
- depends on CFG80211 && ISA_DMA_API && (PCI || BROKEN)
+ depends on CFG80211 && (PCI || BROKEN)
select WIRELESS_EXT
select CRYPTO
select CRYPTO_SKCIPHER
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
index 0b10b34..b6f8251 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
@@ -10,7 +10,7 @@
#include "fw/api/txq.h"
/* Highest firmware API version supported */
-#define IWL_22000_UCODE_API_MAX 75
+#define IWL_22000_UCODE_API_MAX 78
/* Lowest firmware API version supported */
#define IWL_22000_UCODE_API_MIN 39
@@ -50,7 +50,13 @@
#define IWL_MA_A_GF4_A_FW_PRE "iwlwifi-ma-a0-gf4-a0-"
#define IWL_MA_A_MR_A_FW_PRE "iwlwifi-ma-a0-mr-a0-"
#define IWL_MA_A_FM_A_FW_PRE "iwlwifi-ma-a0-fm-a0-"
+#define IWL_MA_B_HR_B_FW_PRE "iwlwifi-ma-b0-hr-b0-"
+#define IWL_MA_B_GF_A_FW_PRE "iwlwifi-ma-b0-gf-a0-"
+#define IWL_MA_B_GF4_A_FW_PRE "iwlwifi-ma-b0-gf4-a0-"
+#define IWL_MA_B_MR_A_FW_PRE "iwlwifi-ma-b0-mr-a0-"
+#define IWL_MA_B_FM_A_FW_PRE "iwlwifi-ma-b0-fm-a0-"
#define IWL_SNJ_A_MR_A_FW_PRE "iwlwifi-SoSnj-a0-mr-a0-"
+#define IWL_BZ_A_HR_A_FW_PRE "iwlwifi-bz-a0-hr-b0-"
#define IWL_BZ_A_HR_B_FW_PRE "iwlwifi-bz-a0-hr-b0-"
#define IWL_BZ_A_GF_A_FW_PRE "iwlwifi-bz-a0-gf-a0-"
#define IWL_BZ_A_GF4_A_FW_PRE "iwlwifi-bz-a0-gf4-a0-"
@@ -69,7 +75,9 @@
#define IWL_BNJ_B_GF_A_FW_PRE "iwlwifi-BzBnj-b0-gf-a0-"
#define IWL_BNJ_A_GF4_A_FW_PRE "iwlwifi-BzBnj-a0-gf4-a0-"
#define IWL_BNJ_B_GF4_A_FW_PRE "iwlwifi-BzBnj-b0-gf4-a0-"
+#define IWL_BNJ_A_HR_A_FW_PRE "iwlwifi-BzBnj-a0-hr-b0-"
#define IWL_BNJ_A_HR_B_FW_PRE "iwlwifi-BzBnj-a0-hr-b0-"
+#define IWL_BNJ_B_HR_A_FW_PRE "iwlwifi-BzBnj-b0-hr-b0-"
#define IWL_BNJ_B_HR_B_FW_PRE "iwlwifi-BzBnj-b0-hr-b0-"
#define IWL_BNJ_B_FM_B_FW_PRE "iwlwifi-BzBnj-b0-fm-b0-"
@@ -116,8 +124,20 @@
IWL_MA_A_MR_A_FW_PRE __stringify(api) ".ucode"
#define IWL_MA_A_FM_A_FW_MODULE_FIRMWARE(api) \
IWL_MA_A_FM_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_MA_B_HR_B_FW_MODULE_FIRMWARE(api) \
+ IWL_MA_B_HR_B_FW_PRE __stringify(api) ".ucode"
+#define IWL_MA_B_GF_A_FW_MODULE_FIRMWARE(api) \
+ IWL_MA_B_GF_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_MA_B_GF4_A_FW_MODULE_FIRMWARE(api) \
+ IWL_MA_B_GF4_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_MA_B_MR_A_FW_MODULE_FIRMWARE(api) \
+ IWL_MA_B_MR_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_MA_B_FM_A_FW_MODULE_FIRMWARE(api) \
+ IWL_MA_B_FM_A_FW_PRE __stringify(api) ".ucode"
#define IWL_SNJ_A_MR_A_MODULE_FIRMWARE(api) \
IWL_SNJ_A_MR_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BZ_A_HR_A_MODULE_FIRMWARE(api) \
+ IWL_BZ_A_HR_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_HR_B_MODULE_FIRMWARE(api) \
IWL_BZ_A_HR_B_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_GF_A_MODULE_FIRMWARE(api) \
@@ -127,17 +147,17 @@
#define IWL_BZ_A_MR_A_MODULE_FIRMWARE(api) \
IWL_BZ_A_MR_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_FM_A_MODULE_FIRMWARE(api) \
- IWL_BZ_A_FM_A_FW_PRE __stringify(api) ".ucode"
+ IWL_BZ_A_FM_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_FM4_A_MODULE_FIRMWARE(api) \
- IWL_BZ_A_FM4_A_FW_PRE __stringify(api) ".ucode"
+ IWL_BZ_A_FM4_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_FM_B_MODULE_FIRMWARE(api) \
- IWL_BZ_A_FM_B_FW_PRE __stringify(api) ".ucode"
+ IWL_BZ_A_FM_B_FW_PRE __stringify(api) ".ucode"
#define IWL_BZ_A_FM4_B_MODULE_FIRMWARE(api) \
- IWL_BZ_A_FM4_B_FW_PRE __stringify(api) ".ucode"
+ IWL_BZ_A_FM4_B_FW_PRE __stringify(api) ".ucode"
#define IWL_GL_A_FM_A_MODULE_FIRMWARE(api) \
- IWL_GL_A_FM_A_FW_PRE __stringify(api) ".ucode"
+ IWL_GL_A_FM_A_FW_PRE __stringify(api) ".ucode"
#define IWL_GL_B_FM_B_MODULE_FIRMWARE(api) \
- IWL_GL_B_FM_B_FW_PRE __stringify(api) ".ucode"
+ IWL_GL_B_FM_B_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_A_FM_A_MODULE_FIRMWARE(api) \
IWL_BNJ_A_FM_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_A_FM4_A_MODULE_FIRMWARE(api) \
@@ -152,8 +172,12 @@
IWL_BNJ_A_GF4_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_B_GF4_A_MODULE_FIRMWARE(api) \
IWL_BNJ_B_GF4_A_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_A_HR_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_A_HR_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_A_HR_B_MODULE_FIRMWARE(api) \
IWL_BNJ_A_HR_B_FW_PRE __stringify(api) ".ucode"
+#define IWL_BNJ_B_HR_A_MODULE_FIRMWARE(api) \
+ IWL_BNJ_B_HR_A_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_B_HR_B_MODULE_FIRMWARE(api) \
IWL_BNJ_B_HR_B_FW_PRE __stringify(api) ".ucode"
#define IWL_BNJ_B_FM_B_MODULE_FIRMWARE(api) \
@@ -882,6 +906,41 @@ const struct iwl_cfg iwl_cfg_ma_a0_ms_a0 = {
.num_rbds = IWL_NUM_RBDS_AX210_HE,
};
+const struct iwl_cfg iwl_cfg_ma_b0_fm_a0 = {
+ .fw_name_pre = IWL_MA_B_FM_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_AX210,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_ma_b0_hr_b0 = {
+ .fw_name_pre = IWL_MA_B_HR_B_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_AX210,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_ma_b0_gf_a0 = {
+ .fw_name_pre = IWL_MA_B_GF_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_AX210,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_ma_b0_gf4_a0 = {
+ .fw_name_pre = IWL_MA_B_GF4_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_AX210,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
+const struct iwl_cfg iwl_cfg_ma_b0_mr_a0 = {
+ .fw_name_pre = IWL_MA_B_MR_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_AX210,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
const struct iwl_cfg iwl_cfg_so_a0_ms_a0 = {
.fw_name_pre = IWL_SO_A_MR_A_FW_PRE,
.uhb_supported = false,
@@ -928,6 +987,14 @@ const struct iwl_cfg iwl_cfg_quz_a0_hr_b0 = {
.num_rbds = IWL_NUM_RBDS_22000_HE,
};
+const struct iwl_cfg iwl_cfg_bz_a0_hr_a0 = {
+ .fw_name_pre = IWL_BZ_A_HR_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .features = IWL_TX_CSUM_NETIF_FLAGS_BZ | NETIF_F_RXCSUM,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
const struct iwl_cfg iwl_cfg_bz_a0_hr_b0 = {
.fw_name_pre = IWL_BZ_A_HR_B_FW_PRE,
.uhb_supported = true,
@@ -1072,6 +1139,14 @@ const struct iwl_cfg iwl_cfg_bnj_b0_gf4_a0 = {
.num_rbds = IWL_NUM_RBDS_AX210_HE,
};
+const struct iwl_cfg iwl_cfg_bnj_a0_hr_a0 = {
+ .fw_name_pre = IWL_BNJ_A_HR_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .features = IWL_TX_CSUM_NETIF_FLAGS | NETIF_F_RXCSUM,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0 = {
.fw_name_pre = IWL_BNJ_A_HR_B_FW_PRE,
.uhb_supported = true,
@@ -1080,6 +1155,14 @@ const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0 = {
.num_rbds = IWL_NUM_RBDS_AX210_HE,
};
+const struct iwl_cfg iwl_cfg_bnj_b0_hr_a0 = {
+ .fw_name_pre = IWL_BNJ_B_HR_A_FW_PRE,
+ .uhb_supported = true,
+ IWL_DEVICE_BZ,
+ .features = IWL_TX_CSUM_NETIF_FLAGS | NETIF_F_RXCSUM,
+ .num_rbds = IWL_NUM_RBDS_AX210_HE,
+};
+
const struct iwl_cfg iwl_cfg_bnj_b0_hr_b0 = {
.fw_name_pre = IWL_BNJ_B_HR_B_FW_PRE,
.uhb_supported = true,
@@ -1116,7 +1199,13 @@ MODULE_FIRMWARE(IWL_MA_A_GF_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_MA_A_GF4_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_MA_A_MR_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_MA_A_FM_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_MA_B_HR_B_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_MA_B_GF_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_MA_B_GF4_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_MA_B_MR_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_MA_B_FM_A_FW_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_SNJ_A_MR_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BZ_A_HR_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_GF_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
@@ -1132,6 +1221,7 @@ MODULE_FIRMWARE(IWL_BNJ_B_GF_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BNJ_A_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BNJ_B_GF4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BNJ_A_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_BNJ_B_HR_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BNJ_B_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_FM4_A_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_BZ_A_FM4_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/sta.c b/drivers/net/wireless/intel/iwlwifi/dvm/sta.c
index cef43cf..8b01ab9 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/sta.c
@@ -1081,6 +1081,7 @@ static int iwlagn_send_sta_key(struct iwl_priv *priv,
{
__le16 key_flags;
struct iwl_addsta_cmd sta_cmd;
+ size_t to_copy;
int i;
spin_lock_bh(&priv->sta_lock);
@@ -1100,7 +1101,9 @@ static int iwlagn_send_sta_key(struct iwl_priv *priv,
sta_cmd.key.tkip_rx_tsc_byte2 = tkip_iv32;
for (i = 0; i < 5; i++)
sta_cmd.key.tkip_rx_ttak[i] = cpu_to_le16(tkip_p1k[i]);
- memcpy(sta_cmd.key.key, keyconf->key, keyconf->keylen);
+ /* keyconf may contain MIC rx/tx keys which iwl does not use */
+ to_copy = min_t(size_t, sizeof(sta_cmd.key.key), keyconf->keylen);
+ memcpy(sta_cmd.key.key, keyconf->key, to_copy);
break;
case WLAN_CIPHER_SUITE_WEP104:
key_flags |= STA_KEY_FLG_KEY_SIZE_MSK;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
index a02e5a6..5f4a513 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c
@@ -1006,8 +1006,10 @@ int iwl_acpi_get_ppag_table(struct iwl_fw_runtime *fwrt)
union acpi_object *wifi_pkg, *data, *flags;
int i, j, ret, tbl_rev, num_sub_bands = 0;
int idx = 2;
+ u8 cmd_ver;
fwrt->ppag_flags = 0;
+ fwrt->ppag_table_valid = false;
data = iwl_acpi_get_object(fwrt->dev, ACPI_PPAG_METHOD);
if (IS_ERR(data))
@@ -1054,8 +1056,15 @@ int iwl_acpi_get_ppag_table(struct iwl_fw_runtime *fwrt)
}
fwrt->ppag_flags = flags->integer.value & ACPI_PPAG_MASK;
-
- if (!fwrt->ppag_flags) {
+ cmd_ver = iwl_fw_lookup_cmd_ver(fwrt->fw,
+ WIDE_ID(PHY_OPS_GROUP,
+ PER_PLATFORM_ANT_GAIN_CMD),
+ IWL_FW_CMD_VER_UNKNOWN);
+ if (cmd_ver == IWL_FW_CMD_VER_UNKNOWN) {
+ ret = -EINVAL;
+ goto out_free;
+ }
+ if (!fwrt->ppag_flags && cmd_ver <= 3) {
ret = 0;
goto out_free;
}
@@ -1076,21 +1085,22 @@ int iwl_acpi_get_ppag_table(struct iwl_fw_runtime *fwrt)
}
fwrt->ppag_chains[i].subbands[j] = ent->integer.value;
-
+ /* from ver 4 the fw deals with out of range values */
+ if (cmd_ver >= 4)
+ continue;
if ((j == 0 &&
(fwrt->ppag_chains[i].subbands[j] > ACPI_PPAG_MAX_LB ||
fwrt->ppag_chains[i].subbands[j] < ACPI_PPAG_MIN_LB)) ||
(j != 0 &&
(fwrt->ppag_chains[i].subbands[j] > ACPI_PPAG_MAX_HB ||
fwrt->ppag_chains[i].subbands[j] < ACPI_PPAG_MIN_HB))) {
- fwrt->ppag_flags = 0;
ret = -EINVAL;
goto out_free;
}
}
}
-
+ fwrt->ppag_table_valid = true;
ret = 0;
out_free:
@@ -1115,19 +1125,22 @@ int iwl_read_ppag_table(struct iwl_fw_runtime *fwrt, union iwl_ppag_table_cmd *c
IWL_DEBUG_RADIO(fwrt,
"PPAG capability not supported by FW, command not sent.\n");
return -EINVAL;
- }
- if (!fwrt->ppag_flags) {
- IWL_DEBUG_RADIO(fwrt, "PPAG not enabled, command not sent.\n");
- return -EINVAL;
- }
+ }
+
+ cmd_ver = iwl_fw_lookup_cmd_ver(fwrt->fw,
+ WIDE_ID(PHY_OPS_GROUP,
+ PER_PLATFORM_ANT_GAIN_CMD),
+ IWL_FW_CMD_VER_UNKNOWN);
+ if (!fwrt->ppag_table_valid || (cmd_ver <= 3 && !fwrt->ppag_flags)) {
+ IWL_DEBUG_RADIO(fwrt, "PPAG not enabled, command not sent.\n");
+ return -EINVAL;
+ }
/* The 'flags' field is the same in v1 and in v2 so we can just
* use v1 to access it.
*/
cmd->v1.flags = cpu_to_le32(fwrt->ppag_flags);
- cmd_ver = iwl_fw_lookup_cmd_ver(fwrt->fw,
- WIDE_ID(PHY_OPS_GROUP, PER_PLATFORM_ANT_GAIN_CMD),
- IWL_FW_CMD_VER_UNKNOWN);
+
if (cmd_ver == 1) {
num_sub_bands = IWL_NUM_SUB_BANDS_V1;
gain = cmd->v1.gain[0];
@@ -1138,7 +1151,7 @@ int iwl_read_ppag_table(struct iwl_fw_runtime *fwrt, union iwl_ppag_table_cmd *c
fwrt->ppag_ver);
cmd->v1.flags &= cpu_to_le32(IWL_PPAG_ETSI_MASK);
}
- } else if (cmd_ver == 2 || cmd_ver == 3) {
+ } else if (cmd_ver >= 2 && cmd_ver <= 4) {
num_sub_bands = IWL_NUM_SUB_BANDS_V2;
gain = cmd->v2.gain[0];
*cmd_size = sizeof(cmd->v2);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
index df08338..8a613e1 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
@@ -767,7 +767,7 @@ struct iwl_wowlan_status_v12 {
} __packed; /* WOWLAN_STATUSES_RSP_API_S_VER_12 */
/**
- * struct iwl_wowlan_info_notif - WoWLAN information notification
+ * struct iwl_wowlan_info_notif_v1 - WoWLAN information notification
* @gtk: GTK data
* @igtk: IGTK data
* @replay_ctr: GTK rekey replay counter
@@ -785,7 +785,7 @@ struct iwl_wowlan_status_v12 {
* @station_id: station id
* @reserved2: reserved
*/
-struct iwl_wowlan_info_notif {
+struct iwl_wowlan_info_notif_v1 {
struct iwl_wowlan_gtk_status_v3 gtk[WOWLAN_GTK_KEYS_NUM];
struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM];
__le64 replay_ctr;
@@ -804,6 +804,39 @@ struct iwl_wowlan_info_notif {
} __packed; /* WOWLAN_INFO_NTFY_API_S_VER_1 */
/**
+ * struct iwl_wowlan_info_notif - WoWLAN information notification
+ * @gtk: GTK data
+ * @igtk: IGTK data
+ * @replay_ctr: GTK rekey replay counter
+ * @pattern_number: number of the matched patterns
+ * @reserved1: reserved
+ * @qos_seq_ctr: QoS sequence counters to use next
+ * @wakeup_reasons: wakeup reasons, see &enum iwl_wowlan_wakeup_reason
+ * @num_of_gtk_rekeys: number of GTK rekeys
+ * @transmitted_ndps: number of transmitted neighbor discovery packets
+ * @received_beacons: number of received beacons
+ * @tid_tear_down: bit mask of tids whose BA sessions were closed
+ * in suspend state
+ * @station_id: station id
+ * @reserved2: reserved
+ */
+struct iwl_wowlan_info_notif {
+ struct iwl_wowlan_gtk_status_v3 gtk[WOWLAN_GTK_KEYS_NUM];
+ struct iwl_wowlan_igtk_status igtk[WOWLAN_IGTK_KEYS_NUM];
+ __le64 replay_ctr;
+ __le16 pattern_number;
+ __le16 reserved1;
+ __le16 qos_seq_ctr[8];
+ __le32 wakeup_reasons;
+ __le32 num_of_gtk_rekeys;
+ __le32 transmitted_ndps;
+ __le32 received_beacons;
+ u8 tid_tear_down;
+ u8 station_id;
+ u8 reserved2[2];
+} __packed; /* WOWLAN_INFO_NTFY_API_S_VER_2 */
+
+/**
* struct iwl_wowlan_wake_pkt_notif - WoWLAN wake packet notification
* @wake_packet_length: wakeup packet length
* @station_id: station id
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/mac-cfg.h b/drivers/net/wireless/intel/iwlwifi/fw/api/mac-cfg.h
index e236d1b..74f2efb 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/mac-cfg.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/mac-cfg.h
@@ -224,15 +224,6 @@ struct iwl_mac_client_data {
} __packed; /* MAC_CONTEXT_CONFIG_CLIENT_DATA_API_S_VER_1 */
/**
- * struct iwl_mac_go_ibss_data - configuration data for GO and IBSS MAC context
- *
- * @beacon_template: beacon template ID
- */
-struct iwl_mac_go_ibss_data {
- __le32 beacon_template;
-} __packed; /* MAC_CONTEXT_CONFIG_GO_IBSS_DATA_API_S_VER_1 */
-
-/**
* struct iwl_mac_p2p_dev_data - configuration data for P2P device MAC context
*
* @is_disc_extended: if set to true, P2P Device discoverability is enabled on
@@ -278,6 +269,7 @@ enum iwl_mac_config_filter_flags {
* @reserved_for_local_mld_addr: reserved
* @filter_flags: combination of &enum iwl_mac_config_filter_flags
* @he_support: does this MAC support HE
+ * @he_ap_support: HE AP enabled, "pseudo HE", no trigger frame handling
* @eht_support: does this MAC support EHT. Requires he_support
* @nic_not_ack_enabled: mark that the NIC doesn't support receiving
* ACK-enabled AGG, (i.e. both BACK and non-BACK frames in single AGG).
@@ -296,13 +288,13 @@ struct iwl_mac_config_cmd {
u8 local_mld_addr[6];
__le16 reserved_for_local_mld_addr;
__le32 filter_flags;
- __le32 he_support;
+ __le16 he_support;
+ __le16 he_ap_support;
__le32 eht_support;
__le32 nic_not_ack_enabled;
/* MAC_CONTEXT_CONFIG_SPECIFIC_DATA_API_U_VER_1 */
union {
struct iwl_mac_client_data client;
- struct iwl_mac_go_ibss_data go_ibss;
struct iwl_mac_p2p_dev_data p2p_dev;
};
} __packed; /* MAC_CONTEXT_CONFIG_CMD_API_S_VER_1 */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
index 7ba0e34..ec96ba0 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
@@ -709,10 +709,13 @@ enum iwl_umac_scan_general_flags_v2 {
* should be aware of a P2P GO operation on the 2GHz band.
* @IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_HB: scan event scheduling
* should be aware of a P2P GO operation on the 5GHz or 6GHz band.
+ * @IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_DONT_TOGGLE_ANT: don't toggle between
+ * valid antennas, and use the same antenna as in previous scan
*/
enum iwl_umac_scan_general_params_flags2 {
IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_LB = BIT(0),
IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_HB = BIT(1),
+ IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_DONT_TOGGLE_ANT = BIT(2),
};
/**
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
index ca97f2f..d9faaae 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
@@ -1038,7 +1038,7 @@ iwl_dump_ini_prph_mac_iter(struct iwl_fw_runtime *fwrt,
range->range_data_size = reg->dev_addr.size;
for (i = 0; i < le32_to_cpu(reg->dev_addr.size); i += 4) {
prph_val = iwl_read_prph(fwrt->trans, addr + i);
- if (prph_val == 0x5a5a5a5a)
+ if ((prph_val & ~0xf) == 0xa5a5a5a0)
return -EBUSY;
*val++ = cpu_to_le32(prph_val);
}
@@ -1388,13 +1388,13 @@ static void iwl_ini_get_rxf_data(struct iwl_fw_runtime *fwrt,
if (!data)
return;
+ memset(data, 0, sizeof(*data));
+
/* make sure only one bit is set in only one fid */
if (WARN_ONCE(hweight_long(fid1) + hweight_long(fid2) != 1,
"fid1=%x, fid2=%x\n", fid1, fid2))
return;
- memset(data, 0, sizeof(*data));
-
if (fid1) {
fifo_idx = ffs(fid1) - 1;
if (WARN_ONCE(fifo_idx >= MAX_NUM_LMAC, "fifo_idx=%d\n",
@@ -1562,7 +1562,7 @@ iwl_dump_ini_dbgi_sram_iter(struct iwl_fw_runtime *fwrt,
prph_data = iwl_read_prph_no_grab(fwrt->trans, (i % 2) ?
DBGI_SRAM_TARGET_ACCESS_RDATA_MSB :
DBGI_SRAM_TARGET_ACCESS_RDATA_LSB);
- if (prph_data == 0x5a5a5a5a) {
+ if ((prph_data & ~0xf) == 0xa5a5a5a0) {
iwl_trans_release_nic_access(fwrt->trans);
return -EBUSY;
}
@@ -2345,6 +2345,8 @@ static u32 iwl_dump_ini_file_name_info(struct iwl_fw_runtime *fwrt,
/* add the dump file name extension tlv to the list */
list_add_tail(&entry->list, list);
+ fwrt->trans->dbg.dump_file_name_ext_valid = false;
+
return entry->size;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c b/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c
index 43e9972..607e07e 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c
@@ -317,8 +317,10 @@ static void *iwl_dbgfs_fw_info_seq_next(struct seq_file *seq,
const struct iwl_fw *fw = priv->fwrt->fw;
*pos = ++state->pos;
- if (*pos >= fw->ucode_capa.n_cmd_versions)
+ if (*pos >= fw->ucode_capa.n_cmd_versions) {
+ kfree(state);
return NULL;
+ }
return state;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dump.c b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
index 59ed321..f86f7b4 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dump.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
@@ -484,6 +484,9 @@ static void iwl_fwrt_dump_fseq_regs(struct iwl_fw_runtime *fwrt)
void iwl_fwrt_dump_error_logs(struct iwl_fw_runtime *fwrt)
{
+ struct iwl_pc_data *pc_data;
+ u8 count;
+
if (!test_bit(STATUS_DEVICE_ENABLED, &fwrt->trans->status)) {
IWL_ERR(fwrt,
"DEVICE_ENABLED bit is not set. Aborting dump.\n");
@@ -502,6 +505,14 @@ void iwl_fwrt_dump_error_logs(struct iwl_fw_runtime *fwrt)
iwl_fwrt_dump_rcm_error_log(fwrt, 1);
iwl_fwrt_dump_iml_error_log(fwrt);
iwl_fwrt_dump_fseq_regs(fwrt);
+ if (fwrt->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000) {
+ pc_data = fwrt->trans->dbg.pc_data;
+ for (count = 0; count < fwrt->trans->dbg.num_pc;
+ count++, pc_data++)
+ IWL_ERR(fwrt, "%s: 0x%x\n",
+ pc_data->pc_name,
+ pc_data->pc_address);
+ }
if (fwrt->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
u32 scratch = iwl_read32(fwrt->trans, CSR_FUNC_SCRATCH);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/file.h b/drivers/net/wireless/intel/iwlwifi/fw/file.h
index f2eefca..cddf09d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/file.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/file.h
@@ -101,8 +101,10 @@ enum iwl_ucode_tlv_type {
IWL_UCODE_TLV_SEC_TABLE_ADDR = 66,
IWL_UCODE_TLV_D3_KEK_KCK_ADDR = 67,
+ IWL_UCODE_TLV_CURRENT_PC = 68,
IWL_UCODE_TLV_FW_NUM_STATIONS = IWL_UCODE_TLV_CONST_BASE + 0,
+ IWL_UCODE_TLV_FW_NUM_BEACONS = IWL_UCODE_TLV_CONST_BASE + 2,
IWL_UCODE_TLV_TYPE_DEBUG_INFO = IWL_UCODE_TLV_DEBUG_BASE + 0,
IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION = IWL_UCODE_TLV_DEBUG_BASE + 1,
@@ -458,6 +460,8 @@ enum iwl_ucode_tlv_capa {
IWL_UCODE_TLV_CAPA_SYNCED_TIME = (__force iwl_ucode_tlv_capa_t)106,
IWL_UCODE_TLV_CAPA_TIME_SYNC_BOTH_FTM_TM = (__force iwl_ucode_tlv_capa_t)108,
IWL_UCODE_TLV_CAPA_BIGTK_TX_SUPPORT = (__force iwl_ucode_tlv_capa_t)109,
+ IWL_UCODE_TLV_CAPA_MLD_API_SUPPORT = (__force iwl_ucode_tlv_capa_t)110,
+ IWL_UCODE_TLV_CAPA_SCAN_DONT_TOGGLE_ANT = (__force iwl_ucode_tlv_capa_t)111,
#ifdef __CHECKER__
/* sparse says it cannot increment the previous enum member */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/img.h b/drivers/net/wireless/intel/iwlwifi/fw/img.h
index f5c4d93d..8d0d58d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/img.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/img.h
@@ -51,6 +51,7 @@ struct iwl_ucode_capabilities {
u32 error_log_addr;
u32 error_log_size;
u32 num_stations;
+ u32 num_beacons;
unsigned long _api[BITS_TO_LONGS(NUM_IWL_UCODE_TLV_API)];
unsigned long _capa[BITS_TO_LONGS(NUM_IWL_UCODE_TLV_CAPA)];
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
index a59cf4d..df689a9 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
@@ -165,6 +165,7 @@ struct iwl_fw_runtime {
struct iwl_ppag_chain ppag_chains[IWL_NUM_CHAIN_LIMITS];
u32 ppag_flags;
u32 ppag_ver;
+ bool ppag_table_valid;
struct iwl_sar_offset_mapping_cmd sgom_table;
bool sgom_enabled;
u8 reduced_power_flags;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/uefi.c b/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
index 0b6f694..01afea3 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/uefi.c
@@ -222,7 +222,7 @@ void *iwl_uefi_get_reduced_power(struct iwl_trans *trans, size_t *len)
return ERR_PTR(-ENOMEM);
status = efi.get_variable(IWL_UEFI_REDUCED_POWER_NAME, &IWL_EFI_VAR_GUID,
- NULL, &package_size, data);
+ NULL, &package_size, package);
if (status != EFI_SUCCESS) {
IWL_DEBUG_FW(trans,
"Reduced Power UEFI variable not found 0x%lx (len %lu)\n",
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
index 9b7b6fc..411b7d4 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
@@ -469,6 +469,7 @@ struct iwl_dev_info {
u16 mac_type;
u16 rf_type;
u8 mac_step;
+ u8 rf_step;
u8 rf_id;
u8 no_160;
u8 cores;
@@ -639,11 +640,17 @@ extern const struct iwl_cfg iwl_cfg_ma_a0_gf4_a0;
extern const struct iwl_cfg iwl_cfg_ma_a0_mr_a0;
extern const struct iwl_cfg iwl_cfg_ma_a0_ms_a0;
extern const struct iwl_cfg iwl_cfg_ma_a0_fm_a0;
+extern const struct iwl_cfg iwl_cfg_ma_b0_hr_b0;
+extern const struct iwl_cfg iwl_cfg_ma_b0_gf_a0;
+extern const struct iwl_cfg iwl_cfg_ma_b0_gf4_a0;
+extern const struct iwl_cfg iwl_cfg_ma_b0_mr_a0;
+extern const struct iwl_cfg iwl_cfg_ma_b0_fm_a0;
extern const struct iwl_cfg iwl_cfg_snj_a0_mr_a0;
extern const struct iwl_cfg iwl_cfg_snj_a0_ms_a0;
extern const struct iwl_cfg iwl_cfg_so_a0_hr_a0;
extern const struct iwl_cfg iwl_cfg_so_a0_ms_a0;
extern const struct iwl_cfg iwl_cfg_quz_a0_hr_b0;
+extern const struct iwl_cfg iwl_cfg_bz_a0_hr_a0;
extern const struct iwl_cfg iwl_cfg_bz_a0_hr_b0;
extern const struct iwl_cfg iwl_cfg_bz_a0_gf_a0;
extern const struct iwl_cfg iwl_cfg_bz_a0_gf4_a0;
@@ -661,7 +668,9 @@ extern const struct iwl_cfg iwl_cfg_bnj_a0_gf_a0;
extern const struct iwl_cfg iwl_cfg_bnj_b0_gf_a0;
extern const struct iwl_cfg iwl_cfg_bnj_a0_gf4_a0;
extern const struct iwl_cfg iwl_cfg_bnj_b0_gf4_a0;
+extern const struct iwl_cfg iwl_cfg_bnj_a0_hr_a0;
extern const struct iwl_cfg iwl_cfg_bnj_a0_hr_b0;
+extern const struct iwl_cfg iwl_cfg_bnj_b0_hr_a0;
extern const struct iwl_cfg iwl_cfg_bnj_b0_hr_b0;
extern const struct iwl_cfg iwl_cfg_bnj_b0_fm_b0;
extern const struct iwl_cfg iwl_cfg_bnj_b0_fm4_b0;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
index bece76b..587368a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
@@ -102,6 +102,8 @@
#define CSR_LTR_LONG_VAL_AD_SNOOP_VAL 0x000003ff
#define CSR_LTR_LONG_VAL_AD_SCALE_USEC 2
+#define CSR_LTR_LAST_MSG (CSR_BASE + 0x0DC)
+
/* GIO Chicken Bits (PCI Express bus link power management) */
#define CSR_GIO_CHICKEN_BITS (CSR_BASE+0x100)
@@ -309,6 +311,8 @@ enum {
SILICON_A_STEP = 0,
SILICON_B_STEP,
SILICON_C_STEP,
+ SILICON_D_STEP,
+ SILICON_E_STEP,
SILICON_Z_STEP = 0xf,
};
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
index 87366b7..898d5dc 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
@@ -138,6 +138,12 @@ static int iwl_dbg_tlv_alloc_buf_alloc(struct iwl_trans *trans,
alloc_id != IWL_FW_INI_ALLOCATION_ID_DBGC1)
goto err;
+ if (buf_location == IWL_FW_INI_LOCATION_DRAM_PATH &&
+ alloc->req_size == 0) {
+ IWL_ERR(trans, "WRT: Invalid DRAM buffer allocation requested size (0)\n");
+ return -EINVAL;
+ }
+
trans->dbg.fw_mon_cfg[alloc_id] = *alloc;
return 0;
@@ -797,7 +803,7 @@ static void iwl_dbg_tlv_update_drams(struct iwl_fw_runtime *fwrt)
if (!ret)
dram_alloc = true;
else
- IWL_WARN(fwrt,
+ IWL_INFO(fwrt,
"WRT: Failed to set DRAM buffer for alloc id %d, ret=%d\n",
i, ret);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-debug.c b/drivers/net/wireless/intel/iwlwifi/iwl-debug.c
index ae4c2a3..3a3c13a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-debug.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-debug.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2005-2011, 2021 Intel Corporation
+ * Copyright (C) 2005-2011, 2021-2022 Intel Corporation
*/
#include <linux/device.h>
#include <linux/interrupt.h>
@@ -57,6 +57,7 @@ void __iwl_err(struct device *dev, enum iwl_err_mode mode, const char *fmt, ...)
default:
break;
}
+ vaf.va = &args;
trace_iwlwifi_err(&vaf);
va_end(args);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
index 4c977ba..34feb4d 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
@@ -127,6 +127,7 @@ static void iwl_dealloc_ucode(struct iwl_drv *drv)
kfree(drv->fw.iml);
kfree(drv->fw.ucode_capa.cmd_versions);
kfree(drv->fw.phy_integration_ver);
+ kfree(drv->trans->dbg.pc_data);
for (i = 0; i < IWL_UCODE_TYPE_MAX; i++)
iwl_free_fw_img(drv, drv->fw.img + i);
@@ -894,7 +895,7 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
drv->fw.img[IWL_UCODE_WOWLAN].is_dual_cpus =
true;
} else if ((num_of_cpus > 2) || (num_of_cpus < 1)) {
- IWL_ERR(drv, "Driver support upto 2 CPUs\n");
+ IWL_ERR(drv, "Driver support up to 2 CPUs\n");
return -EINVAL;
}
break;
@@ -1154,6 +1155,12 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
capa->num_stations =
le32_to_cpup((const __le32 *)tlv_data);
break;
+ case IWL_UCODE_TLV_FW_NUM_BEACONS:
+ if (tlv_len != sizeof(u32))
+ goto invalid_tlv_len;
+ capa->num_beacons =
+ le32_to_cpup((const __le32 *)tlv_data);
+ break;
case IWL_UCODE_TLV_UMAC_DEBUG_ADDRS: {
const struct iwl_umac_debug_addrs *dbg_ptrs =
(const void *)tlv_data;
@@ -1232,6 +1239,14 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
iwl_drv_set_dump_exclude(drv, tlv_type,
tlv_data, tlv_len);
break;
+ case IWL_UCODE_TLV_CURRENT_PC:
+ if (tlv_len < sizeof(struct iwl_pc_data))
+ goto invalid_tlv_len;
+ drv->trans->dbg.num_pc =
+ tlv_len / sizeof(struct iwl_pc_data);
+ drv->trans->dbg.pc_data =
+ kmemdup(tlv_data, tlv_len, GFP_KERNEL);
+ break;
default:
IWL_DEBUG_INFO(drv, "unknown TLV: %d\n", tlv_type);
break;
@@ -1406,6 +1421,7 @@ static void iwl_req_fw_callback(const struct firmware *ucode_raw, void *context)
IWL_DEFAULT_STANDARD_PHY_CALIBRATE_TBL_SIZE;
fw->ucode_capa.n_scan_channels = IWL_DEFAULT_SCAN_CHANNELS;
fw->ucode_capa.num_stations = IWL_MVM_STATION_COUNT_MAX;
+ fw->ucode_capa.num_beacons = 1;
/* dump all fw memory areas by default */
fw->dbg.dump_mask = 0xffffffff;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
index baa6433..0e8ca76 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
@@ -47,13 +47,12 @@ struct iwl_nvm_data {
struct ieee80211_supported_band bands[NUM_NL80211_BANDS];
/*
- * iftype data for low (2.4 GHz) and high (5 and 6 GHz) bands,
- * we can use the same for 5 and 6 GHz bands because they have
- * the same data
+ * iftype data for low (2.4 GHz) high (5 GHz) and uhb (6 GHz) bands
*/
struct {
struct ieee80211_sband_iftype_data low[2];
struct ieee80211_sband_iftype_data high[2];
+ struct ieee80211_sband_iftype_data uhb[2];
} iftd;
struct ieee80211_channel channels[];
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
index 923bbfc..7dcb1c3 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
@@ -860,7 +860,10 @@ iwl_nvm_fixup_sband_iftd(struct iwl_trans *trans,
/* Advertise an A-MPDU exponent extension based on
* operating band
*/
- if (sband->band != NL80211_BAND_2GHZ)
+ if (sband->band == NL80211_BAND_6GHZ && iftype_data->eht_cap.has_eht)
+ iftype_data->he_cap.he_cap_elem.mac_cap_info[3] |=
+ IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_2;
+ else if (sband->band != NL80211_BAND_2GHZ)
iftype_data->he_cap.he_cap_elem.mac_cap_info[3] |=
IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_EXT_1;
else
@@ -876,16 +879,13 @@ iwl_nvm_fixup_sband_iftd(struct iwl_trans *trans,
IEEE80211_EHT_MAC_CAP0_MAX_MPDU_LEN_MASK);
break;
case NL80211_BAND_6GHZ:
- if (!is_ap || iwlwifi_mod_params.nvm_file)
- iftype_data->eht_cap.eht_cap_elem.phy_cap_info[0] |=
- IEEE80211_EHT_PHY_CAP0_320MHZ_IN_6GHZ;
+ iftype_data->eht_cap.eht_cap_elem.phy_cap_info[0] |=
+ IEEE80211_EHT_PHY_CAP0_320MHZ_IN_6GHZ;
fallthrough;
case NL80211_BAND_5GHZ:
iftype_data->he_cap.he_cap_elem.phy_cap_info[0] |=
- IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_80MHZ_IN_5G;
- if (!is_ap || iwlwifi_mod_params.nvm_file)
- iftype_data->he_cap.he_cap_elem.phy_cap_info[0] |=
- IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G;
+ IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_40MHZ_80MHZ_IN_5G |
+ IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_160MHZ_IN_5G;
break;
default:
WARN_ON(1);
@@ -938,6 +938,10 @@ iwl_nvm_fixup_sband_iftd(struct iwl_trans *trans,
}
}
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210 && !is_ap)
+ iftype_data->he_cap.he_cap_elem.phy_cap_info[2] |=
+ IEEE80211_HE_PHY_CAP2_UL_MU_FULL_MU_MIMO;
+
switch (CSR_HW_RFID_TYPE(trans->hw_rf_id)) {
case IWL_CFG_RF_TYPE_GF:
case IWL_CFG_RF_TYPE_MR:
@@ -999,15 +1003,18 @@ static void iwl_init_he_hw_capab(struct iwl_trans *trans,
BUILD_BUG_ON(sizeof(data->iftd.low) != sizeof(iwl_he_eht_capa));
BUILD_BUG_ON(sizeof(data->iftd.high) != sizeof(iwl_he_eht_capa));
+ BUILD_BUG_ON(sizeof(data->iftd.uhb) != sizeof(iwl_he_eht_capa));
switch (sband->band) {
case NL80211_BAND_2GHZ:
iftype_data = data->iftd.low;
break;
case NL80211_BAND_5GHZ:
- case NL80211_BAND_6GHZ:
iftype_data = data->iftd.high;
break;
+ case NL80211_BAND_6GHZ:
+ iftype_data = data->iftd.uhb;
+ break;
default:
WARN_ON(1);
return;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
index 62ce116..0dfe00e 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
@@ -350,6 +350,11 @@
#define WFPM_OTP_CFG1_ADDR 0x00a03098
#define WFPM_OTP_CFG1_IS_JACKET_BIT BIT(4)
#define WFPM_OTP_CFG1_IS_CDB_BIT BIT(5)
+#define WFPM_OTP_BZ_BNJ_JACKET_BIT 5
+#define WFPM_OTP_BZ_BNJ_CDB_BIT 4
+#define WFPM_OTP_CFG1_IS_JACKET(_val) (((_val) & 0x00000020) >> WFPM_OTP_BZ_BNJ_JACKET_BIT)
+#define WFPM_OTP_CFG1_IS_CDB(_val) (((_val) & 0x00000010) >> WFPM_OTP_BZ_BNJ_CDB_BIT)
+
#define WFPM_GP2 0xA030B4
@@ -445,6 +450,8 @@ enum {
#define REG_CRF_ID_TYPE_GF_TC 0xF08
#define REG_CRF_ID_TYPE_MR 0x810
#define REG_CRF_ID_TYPE_FM 0x910
+#define REG_CRF_ID_TYPE_FMI 0x930
+#define REG_CRF_ID_TYPE_FMR 0x900
#define HPM_DEBUG 0xA03440
#define PERSISTENCE_BIT BIT(12)
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
index dd277a4..9f1228b 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
@@ -748,6 +748,18 @@ struct iwl_imr_data {
__le64 imr_base_addr;
};
+#define IWL_TRANS_CURRENT_PC_NAME_MAX_BYTES 32
+
+/**
+ * struct iwl_pc_data - program counter details
+ * @pc_name: cpu name
+ * @pc_address: cpu program counter
+ */
+struct iwl_pc_data {
+ u8 pc_name[IWL_TRANS_CURRENT_PC_NAME_MAX_BYTES];
+ u32 pc_address;
+};
+
/**
* struct iwl_trans_debug - transport debug related data
*
@@ -777,6 +789,8 @@ struct iwl_imr_data {
* @ucode_preset: preset based on ucode
* @dump_file_name_ext: dump file name extension
* @dump_file_name_ext_valid: dump file name extension if valid or not
+ * @num_pc: number of program counter for cpu
+ * @pc_data: details of the program counter
*/
struct iwl_trans_debug {
u8 n_dest_reg;
@@ -817,6 +831,8 @@ struct iwl_trans_debug {
struct iwl_imr_data imr_data;
u8 dump_file_name_ext[IWL_FW_INI_MAX_NAME];
bool dump_file_name_ext_valid;
+ u32 num_pc;
+ struct iwl_pc_data *pc_data;
};
struct iwl_dma_ptr {
@@ -981,7 +997,7 @@ struct iwl_trans_txqs {
* 0 indicates that frag SKBs (NETIF_F_SG) aren't supported.
* @hw_rf_id a u32 with the device RF ID
* @hw_crf_id a u32 with the device CRF ID
- * @hw_cdb_id a u32 with the device CDB ID
+ * @hw_wfpm_id a u32 with the device wfpm ID
* @hw_id: a u32 with the ID of the device / sub-device.
* Set during transport allocation.
* @hw_id_str: a string with info about HW ID. Set during transport allocation.
@@ -1024,7 +1040,8 @@ struct iwl_trans {
u32 hw_rev_step;
u32 hw_rf_id;
u32 hw_crf_id;
- u32 hw_cdb_id;
+ u32 hw_cnv_id;
+ u32 hw_wfpm_id;
u32 hw_id;
char hw_id_str[52];
u32 sku_id[3];
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h
index ae66192..655d95d 100644
--- a/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h
+++ b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
- * Copyright (C) 2021 Intel Corporation
+ * Copyright (C) 2021 - 2022 Intel Corporation
*/
#ifndef __iwl_mei_h__
@@ -301,7 +301,7 @@ struct iwl_mei_colloc_info {
struct iwl_mei_ops {
void (*me_conn_status)(void *priv,
const struct iwl_mei_conn_info *conn_info);
- void (*rfkill)(void *priv, bool blocked);
+ void (*rfkill)(void *priv, bool blocked, bool csme_taking_ownership);
void (*roaming_forbidden)(void *priv, bool forbidden);
void (*sap_connected)(void *priv);
void (*nic_stolen)(void *priv);
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/main.c b/drivers/net/wireless/intel/iwlwifi/mei/main.c
index 67dfb77..0a29fb0 100644
--- a/drivers/net/wireless/intel/iwlwifi/mei/main.c
+++ b/drivers/net/wireless/intel/iwlwifi/mei/main.c
@@ -31,6 +31,11 @@ MODULE_LICENSE("GPL");
#define MEI_WLAN_UUID UUID_LE(0x13280904, 0x7792, 0x4fcb, \
0xa1, 0xaa, 0x5e, 0x70, 0xcb, 0xb1, 0xe8, 0x65)
+/* After CSME takes ownership, it won't release it for 60 seconds to avoid
+ * frequent ownership transitions.
+ */
+#define MEI_OWNERSHIP_RETAKE_TIMEOUT_MS msecs_to_jiffies(60000)
+
/*
* Since iwlwifi calls iwlmei without any context, hold a pointer to the
* mei_cl_device structure here.
@@ -156,6 +161,8 @@ struct iwl_mei_filters {
* accessed without the mutex.
* @netdev_work: used to defer registering and unregistering of the netdev to
* avoid taking the rtnl lock in the SAP messages handlers.
+ * @ownership_dwork: used to re-ask for NIC ownership after ownership was taken
+ * by CSME or when a previous ownership request failed.
* @sap_seq_no: the sequence number for the SAP messages
* @seq_no: the sequence number for the SAP messages
* @dbgfs_dir: the debugfs dir entry
@@ -179,6 +186,7 @@ struct iwl_mei {
bool pldr_active;
spinlock_t data_q_lock;
struct work_struct netdev_work;
+ struct delayed_work ownership_dwork;
atomic_t sap_seq_no;
atomic_t seq_no;
@@ -716,7 +724,7 @@ iwl_mei_handle_conn_status(struct mei_cl_device *cldev,
status->link_prot_state);
else
iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv,
- status->link_prot_state);
+ status->link_prot_state, false);
}
static void iwl_mei_set_init_conf(struct iwl_mei *mei)
@@ -788,7 +796,7 @@ static void iwl_mei_handle_amt_state(struct mei_cl_device *cldev,
if (mei->amt_enabled)
iwl_mei_set_init_conf(mei);
else if (iwl_mei_cache.ops)
- iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false, false);
schedule_work(&mei->netdev_work);
@@ -829,10 +837,12 @@ static void iwl_mei_handle_csme_taking_ownership(struct mei_cl_device *cldev,
*/
mei->csme_taking_ownership = true;
- iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, true);
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, true, true);
} else {
iwl_mei_send_sap_msg(cldev,
SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED);
+ schedule_delayed_work(&mei->ownership_dwork,
+ MEI_OWNERSHIP_RETAKE_TIMEOUT_MS);
}
}
@@ -882,7 +892,7 @@ static void iwl_mei_handle_rx_host_own_req(struct mei_cl_device *cldev,
/* We can now start the connection, unblock rfkill */
if (iwl_mei_cache.ops)
- iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false, false);
}
static void iwl_mei_handle_pldr_ack(struct mei_cl_device *cldev,
@@ -1447,7 +1457,13 @@ int iwl_mei_get_ownership(void)
ret = wait_event_timeout(mei->get_ownership_wq,
mei->got_ownership, HZ / 2);
- return (!ret) ? -ETIMEDOUT : 0;
+ if (!ret) {
+ schedule_delayed_work(&mei->ownership_dwork,
+ MEI_OWNERSHIP_RETAKE_TIMEOUT_MS);
+ return -ETIMEDOUT;
+ }
+
+ return 0;
out:
mutex_unlock(&iwl_mei_mutex);
return ret;
@@ -1738,6 +1754,8 @@ void iwl_mei_device_state(bool up)
iwl_mei_send_sap_msg(mei->cldev,
SAP_MSG_NOTIF_CSME_OWNERSHIP_CONFIRMED);
mei->csme_taking_ownership = false;
+ schedule_delayed_work(&mei->ownership_dwork,
+ MEI_OWNERSHIP_RETAKE_TIMEOUT_MS);
out:
mutex_unlock(&iwl_mei_mutex);
}
@@ -1773,7 +1791,8 @@ int iwl_mei_register(void *priv, const struct iwl_mei_ops *ops)
if (iwl_mei_is_connected()) {
if (mei->amt_enabled)
iwl_mei_send_sap_msg(mei->cldev,
- SAP_MSG_NOTIF_WIFIDR_UP);
+ SAP_MSG_NOTIF_WIFIDR_UP,
+ false);
ops->rfkill(priv, mei->link_prot_state);
}
}
@@ -1894,6 +1913,11 @@ static void iwl_mei_dbgfs_unregister(struct iwl_mei *mei) {}
#endif /* CONFIG_DEBUG_FS */
+static void iwl_mei_ownership_dwork(struct work_struct *wk)
+{
+ iwl_mei_get_ownership();
+}
+
#define ALLOC_SHARED_MEM_RETRY_MAX_NUM 3
/*
@@ -1923,6 +1947,7 @@ static int iwl_mei_probe(struct mei_cl_device *cldev,
init_waitqueue_head(&mei->pldr_wq);
spin_lock_init(&mei->data_q_lock);
INIT_WORK(&mei->netdev_work, iwl_mei_netdev_work);
+ INIT_DELAYED_WORK(&mei->ownership_dwork, iwl_mei_ownership_dwork);
mei_cldev_set_drvdata(cldev, mei);
mei->cldev = cldev;
@@ -2087,7 +2112,7 @@ static void iwl_mei_remove(struct mei_cl_device *cldev)
spin_unlock_bh(&mei->data_q_lock);
if (iwl_mei_cache.ops)
- iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false);
+ iwl_mei_cache.ops->rfkill(iwl_mei_cache.priv, false, false);
/*
* mei_cldev_disable will return only after all the MEI Rx is done.
@@ -2105,6 +2130,7 @@ static void iwl_mei_remove(struct mei_cl_device *cldev)
cancel_work_sync(&mei->send_csa_msg_wk);
cancel_delayed_work_sync(&mei->csa_throttle_end_wk);
cancel_work_sync(&mei->netdev_work);
+ cancel_delayed_work_sync(&mei->ownership_dwork);
/*
* If someone waits for the ownership, let him know that we are going
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
index 40adf78..37aa467 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
@@ -564,6 +564,7 @@ static void iwl_mvm_wowlan_get_tkip_data(struct ieee80211_hw *hw,
}
for (i = 0; i < IWL_NUM_RSC; i++) {
+ ieee80211_get_key_rx_seq(key, i, &seq);
/* wrapping isn't allowed, AP must rekey */
if (seq.tkip.iv32 > cur_rx_iv32)
cur_rx_iv32 = seq.tkip.iv32;
@@ -2017,6 +2018,12 @@ static void iwl_mvm_parse_wowlan_info_notif(struct iwl_mvm *mvm,
{
u32 i;
+ if (!data) {
+ IWL_ERR(mvm, "iwl_wowlan_info_notif data is NULL\n");
+ status = NULL;
+ return;
+ }
+
if (len < sizeof(*data)) {
IWL_ERR(mvm, "Invalid WoWLAN info notification!\n");
status = NULL;
@@ -2705,10 +2712,15 @@ static bool iwl_mvm_wait_d3_notif(struct iwl_notif_wait_data *notif_wait,
struct iwl_d3_data *d3_data = data;
u32 len;
int ret;
+ int wowlan_info_ver = iwl_fw_lookup_notif_ver(mvm->fw,
+ PROT_OFFLOAD_GROUP,
+ WOWLAN_INFO_NOTIFICATION,
+ IWL_FW_CMD_VER_UNKNOWN);
+
switch (WIDE_ID(pkt->hdr.group_id, pkt->hdr.cmd)) {
case WIDE_ID(PROT_OFFLOAD_GROUP, WOWLAN_INFO_NOTIFICATION): {
- struct iwl_wowlan_info_notif *notif = (void *)pkt->data;
+ struct iwl_wowlan_info_notif *notif;
if (d3_data->notif_received & IWL_D3_NOTIF_WOWLAN_INFO) {
/* We might get two notifications due to dual bss */
@@ -2717,10 +2729,32 @@ static bool iwl_mvm_wait_d3_notif(struct iwl_notif_wait_data *notif_wait,
break;
}
+ if (wowlan_info_ver < 2) {
+ struct iwl_wowlan_info_notif_v1 *notif_v1 = (void *)pkt->data;
+
+ notif = kmemdup(notif_v1,
+ offsetofend(struct iwl_wowlan_info_notif,
+ received_beacons),
+ GFP_ATOMIC);
+
+ if (!notif)
+ return false;
+
+ notif->tid_tear_down = notif_v1->tid_tear_down;
+ notif->station_id = notif_v1->station_id;
+
+ } else {
+ notif = (void *)pkt->data;
+ }
+
d3_data->notif_received |= IWL_D3_NOTIF_WOWLAN_INFO;
len = iwl_rx_packet_payload_len(pkt);
iwl_mvm_parse_wowlan_info_notif(mvm, notif, d3_data->status,
len);
+
+ if (wowlan_info_ver < 2)
+ kfree(notif);
+
if (d3_data->status &&
d3_data->status->wakeup_reasons & IWL_WOWLAN_WAKEUP_REASON_HAS_WAKEUP_PKT)
/* We are supposed to get also wake packet notif */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
index 527daaf..84a4885 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
@@ -340,6 +340,26 @@ static ssize_t iwl_dbgfs_sar_geo_profile_read(struct file *file,
return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
}
+
+static ssize_t iwl_dbgfs_wifi_6e_enable_read(struct file *file,
+ char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct iwl_mvm *mvm = file->private_data;
+ int err, pos;
+ char buf[12];
+ u32 value;
+
+ err = iwl_acpi_get_dsm_u32(mvm->fwrt.dev, 0,
+ DSM_FUNC_ENABLE_6E,
+ &iwl_guid, &value);
+ if (err)
+ return err;
+
+ pos = sprintf(buf, "0x%08x\n", value);
+
+ return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
+}
#endif
static ssize_t iwl_dbgfs_stations_read(struct file *file, char __user *user_buf,
@@ -1898,6 +1918,7 @@ MVM_DEBUGFS_READ_FILE_OPS(uapsd_noagg_bssids);
#ifdef CONFIG_ACPI
MVM_DEBUGFS_READ_FILE_OPS(sar_geo_profile);
+MVM_DEBUGFS_READ_FILE_OPS(wifi_6e_enable);
#endif
MVM_DEBUGFS_READ_WRITE_STA_FILE_OPS(amsdu_len, 16);
@@ -1940,6 +1961,11 @@ static ssize_t iwl_dbgfs_mem_read(struct file *file, char __user *user_buf,
if (ret < 0)
return ret;
+ if (iwl_rx_packet_payload_len(hcmd.resp_pkt) < sizeof(*rsp)) {
+ ret = -EIO;
+ goto out;
+ }
+
rsp = (void *)hcmd.resp_pkt->data;
if (le32_to_cpu(rsp->status) != DEBUG_MEM_STATUS_SUCCESS) {
ret = -ENXIO;
@@ -2016,6 +2042,11 @@ static ssize_t iwl_dbgfs_mem_write(struct file *file,
if (ret < 0)
return ret;
+ if (iwl_rx_packet_payload_len(hcmd.resp_pkt) < sizeof(*rsp)) {
+ ret = -EIO;
+ goto out;
+ }
+
rsp = (void *)hcmd.resp_pkt->data;
if (rsp->status != DEBUG_MEM_STATUS_SUCCESS) {
ret = -ENXIO;
@@ -2092,6 +2123,7 @@ void iwl_mvm_dbgfs_register(struct iwl_mvm *mvm)
MVM_DEBUGFS_ADD_FILE(tas_get_status, mvm->debugfs_dir, 0400);
#ifdef CONFIG_ACPI
MVM_DEBUGFS_ADD_FILE(sar_geo_profile, mvm->debugfs_dir, 0400);
+ MVM_DEBUGFS_ADD_FILE(wifi_6e_enable, mvm->debugfs_dir, 0400);
#endif
MVM_DEBUGFS_ADD_FILE(he_sniffer_params, mvm->debugfs_dir, 0600);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
index 379da4b..3963a0d 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c
@@ -25,6 +25,10 @@ struct iwl_mvm_smooth_entry {
u64 host_time;
};
+enum iwl_mvm_pasn_flags {
+ IWL_MVM_PASN_FLAG_HAS_HLTK = BIT(0),
+};
+
struct iwl_mvm_ftm_pasn_entry {
struct list_head list;
u8 addr[ETH_ALEN];
@@ -33,6 +37,7 @@ struct iwl_mvm_ftm_pasn_entry {
u8 cipher;
u8 tx_pn[IEEE80211_CCMP_PN_LEN];
u8 rx_pn[IEEE80211_CCMP_PN_LEN];
+ u32 flags;
};
int iwl_mvm_ftm_add_pasn_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
@@ -79,14 +84,24 @@ int iwl_mvm_ftm_add_pasn_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
rcu_read_unlock();
}
- if (tk_len != expected_tk_len || hltk_len != sizeof(pasn->hltk)) {
+ if (tk_len != expected_tk_len ||
+ (hltk_len && hltk_len != sizeof(pasn->hltk))) {
IWL_ERR(mvm, "Invalid key length: tk_len=%u hltk_len=%u\n",
tk_len, hltk_len);
goto out;
}
+ if (!expected_tk_len && !hltk_len) {
+ IWL_ERR(mvm, "TK and HLTK not set\n");
+ goto out;
+ }
+
memcpy(pasn->addr, addr, sizeof(pasn->addr));
- memcpy(pasn->hltk, hltk, sizeof(pasn->hltk));
+
+ if (hltk_len) {
+ memcpy(pasn->hltk, hltk, sizeof(pasn->hltk));
+ pasn->flags |= IWL_MVM_PASN_FLAG_HAS_HLTK;
+ }
if (tk && tk_len)
memcpy(pasn->tk, tk, sizeof(pasn->tk));
@@ -691,7 +706,11 @@ iwl_mvm_ftm_set_secured_ranging(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
continue;
target->cipher = entry->cipher;
- memcpy(target->hltk, entry->hltk, sizeof(target->hltk));
+
+ if (entry->flags & IWL_MVM_PASN_FLAG_HAS_HLTK)
+ memcpy(target->hltk, entry->hltk, sizeof(target->hltk));
+ else
+ memset(target->hltk, 0, sizeof(target->hltk));
if (vif->cfg.assoc &&
!memcmp(vif->bss_conf.bssid, target->bssid,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c
index c37d793..1b6fb73 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c
@@ -317,6 +317,8 @@ int iwl_mvm_ftm_respoder_add_pasn_sta(struct iwl_mvm *mvm,
.addr = addr,
.hltk = hltk,
};
+ struct iwl_mvm_pasn_hltk_data *hltk_data_ptr = NULL;
+
u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw,
WIDE_ID(LOCATION_GROUP, TOF_RESPONDER_DYN_CONFIG_CMD),
2);
@@ -328,12 +330,21 @@ int iwl_mvm_ftm_respoder_add_pasn_sta(struct iwl_mvm *mvm,
return -ENOTSUPP;
}
- hltk_data.cipher = iwl_mvm_cipher_to_location_cipher(cipher);
- if (hltk_data.cipher == IWL_LOCATION_CIPHER_INVALID) {
- IWL_ERR(mvm, "invalid cipher: %u\n", cipher);
+ if ((!hltk || !hltk_len) && (!tk || !tk_len)) {
+ IWL_ERR(mvm, "TK and HLTK not set\n");
return -EINVAL;
}
+ if (hltk && hltk_len) {
+ hltk_data.cipher = iwl_mvm_cipher_to_location_cipher(cipher);
+ if (hltk_data.cipher == IWL_LOCATION_CIPHER_INVALID) {
+ IWL_ERR(mvm, "invalid cipher: %u\n", cipher);
+ return -EINVAL;
+ }
+
+ hltk_data_ptr = &hltk_data;
+ }
+
if (tk && tk_len) {
sta = kzalloc(sizeof(*sta), GFP_KERNEL);
if (!sta)
@@ -350,7 +361,7 @@ int iwl_mvm_ftm_respoder_add_pasn_sta(struct iwl_mvm *mvm,
list_add_tail(&sta->list, &mvm->resp_pasn_list);
}
- ret = iwl_mvm_ftm_responder_dyn_cfg_v3(mvm, vif, NULL, &hltk_data);
+ ret = iwl_mvm_ftm_responder_dyn_cfg_v3(mvm, vif, NULL, hltk_data_ptr);
if (ret && sta)
iwl_mvm_resp_del_pasn_sta(mvm, vif, sta);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
index 7fe733d..b35c96c 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
@@ -321,6 +321,8 @@ static int iwl_mvm_load_ucode_wait_alive(struct iwl_mvm *mvm,
static const u16 alive_cmd[] = { UCODE_ALIVE_NTFY };
bool run_in_rfkill =
ucode_type == IWL_UCODE_INIT || iwl_mvm_has_unified_ucode(mvm);
+ u8 count;
+ struct iwl_pc_data *pc_data;
if (ucode_type == IWL_UCODE_REGULAR &&
iwl_fw_dbg_conf_usniffer(mvm->fw, FW_DBG_START_FROM_ALIVE) &&
@@ -393,6 +395,14 @@ static int iwl_mvm_load_ucode_wait_alive(struct iwl_mvm *mvm,
/* LMAC/UMAC PC info */
if (trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22000) {
+ pc_data = trans->dbg.pc_data;
+ for (count = 0; count < trans->dbg.num_pc;
+ count++, pc_data++)
+ IWL_ERR(mvm, "%s: 0x%x\n",
+ pc_data->pc_name,
+ pc_data->pc_address);
+ } else if (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_9000) {
IWL_ERR(mvm, "UMAC PC: 0x%x\n",
iwl_read_umac_prph(trans,
@@ -467,111 +477,6 @@ static int iwl_mvm_load_ucode_wait_alive(struct iwl_mvm *mvm,
return 0;
}
-static int iwl_run_unified_mvm_ucode(struct iwl_mvm *mvm)
-{
- struct iwl_notification_wait init_wait;
- struct iwl_nvm_access_complete_cmd nvm_complete = {};
- struct iwl_init_extended_cfg_cmd init_cfg = {
- .init_flags = cpu_to_le32(BIT(IWL_INIT_NVM)),
- };
- static const u16 init_complete[] = {
- INIT_COMPLETE_NOTIF,
- };
- int ret;
-
- if (mvm->trans->cfg->tx_with_siso_diversity)
- init_cfg.init_flags |= cpu_to_le32(BIT(IWL_INIT_PHY));
-
- lockdep_assert_held(&mvm->mutex);
-
- mvm->rfkill_safe_init_done = false;
-
- iwl_init_notification_wait(&mvm->notif_wait,
- &init_wait,
- init_complete,
- ARRAY_SIZE(init_complete),
- iwl_wait_init_complete,
- NULL);
-
- iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_EARLY, NULL);
-
- /* Will also start the device */
- ret = iwl_mvm_load_ucode_wait_alive(mvm, IWL_UCODE_REGULAR);
- if (ret) {
- IWL_ERR(mvm, "Failed to start RT ucode: %d\n", ret);
- goto error;
- }
- iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_AFTER_ALIVE,
- NULL);
-
- /* Send init config command to mark that we are sending NVM access
- * commands
- */
- ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(SYSTEM_GROUP,
- INIT_EXTENDED_CFG_CMD),
- CMD_SEND_IN_RFKILL,
- sizeof(init_cfg), &init_cfg);
- if (ret) {
- IWL_ERR(mvm, "Failed to run init config command: %d\n",
- ret);
- goto error;
- }
-
- /* Load NVM to NIC if needed */
- if (mvm->nvm_file_name) {
- ret = iwl_read_external_nvm(mvm->trans, mvm->nvm_file_name,
- mvm->nvm_sections);
- if (ret)
- goto error;
- ret = iwl_mvm_load_nvm_to_nic(mvm);
- if (ret)
- goto error;
- }
-
- if (IWL_MVM_PARSE_NVM && !mvm->nvm_data) {
- ret = iwl_nvm_init(mvm);
- if (ret) {
- IWL_ERR(mvm, "Failed to read NVM: %d\n", ret);
- goto error;
- }
- }
-
- ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(REGULATORY_AND_NVM_GROUP,
- NVM_ACCESS_COMPLETE),
- CMD_SEND_IN_RFKILL,
- sizeof(nvm_complete), &nvm_complete);
- if (ret) {
- IWL_ERR(mvm, "Failed to run complete NVM access: %d\n",
- ret);
- goto error;
- }
-
- /* We wait for the INIT complete notification */
- ret = iwl_wait_notification(&mvm->notif_wait, &init_wait,
- MVM_UCODE_ALIVE_TIMEOUT);
- if (ret)
- return ret;
-
- /* Read the NVM only at driver load time, no need to do this twice */
- if (!IWL_MVM_PARSE_NVM && !mvm->nvm_data) {
- mvm->nvm_data = iwl_get_nvm(mvm->trans, mvm->fw);
- if (IS_ERR(mvm->nvm_data)) {
- ret = PTR_ERR(mvm->nvm_data);
- mvm->nvm_data = NULL;
- IWL_ERR(mvm, "Failed to read NVM: %d\n", ret);
- return ret;
- }
- }
-
- mvm->rfkill_safe_init_done = true;
-
- return 0;
-
-error:
- iwl_remove_notification(&mvm->notif_wait, &init_wait);
- return ret;
-}
-
#ifdef CONFIG_ACPI
static void iwl_mvm_phy_filter_init(struct iwl_mvm *mvm,
struct iwl_phy_specific_cfg *phy_filters)
@@ -698,6 +603,118 @@ static int iwl_send_phy_cfg_cmd(struct iwl_mvm *mvm)
return iwl_mvm_send_cmd_pdu(mvm, cmd_id, 0, cmd_size, &phy_cfg_cmd);
}
+static int iwl_run_unified_mvm_ucode(struct iwl_mvm *mvm)
+{
+ struct iwl_notification_wait init_wait;
+ struct iwl_nvm_access_complete_cmd nvm_complete = {};
+ struct iwl_init_extended_cfg_cmd init_cfg = {
+ .init_flags = cpu_to_le32(BIT(IWL_INIT_NVM)),
+ };
+ static const u16 init_complete[] = {
+ INIT_COMPLETE_NOTIF,
+ };
+ int ret;
+
+ if (mvm->trans->cfg->tx_with_siso_diversity)
+ init_cfg.init_flags |= cpu_to_le32(BIT(IWL_INIT_PHY));
+
+ lockdep_assert_held(&mvm->mutex);
+
+ mvm->rfkill_safe_init_done = false;
+
+ iwl_init_notification_wait(&mvm->notif_wait,
+ &init_wait,
+ init_complete,
+ ARRAY_SIZE(init_complete),
+ iwl_wait_init_complete,
+ NULL);
+
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_EARLY, NULL);
+
+ /* Will also start the device */
+ ret = iwl_mvm_load_ucode_wait_alive(mvm, IWL_UCODE_REGULAR);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to start RT ucode: %d\n", ret);
+ goto error;
+ }
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_AFTER_ALIVE,
+ NULL);
+
+ /* Send init config command to mark that we are sending NVM access
+ * commands
+ */
+ ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(SYSTEM_GROUP,
+ INIT_EXTENDED_CFG_CMD),
+ CMD_SEND_IN_RFKILL,
+ sizeof(init_cfg), &init_cfg);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to run init config command: %d\n",
+ ret);
+ goto error;
+ }
+
+ /* Load NVM to NIC if needed */
+ if (mvm->nvm_file_name) {
+ ret = iwl_read_external_nvm(mvm->trans, mvm->nvm_file_name,
+ mvm->nvm_sections);
+ if (ret)
+ goto error;
+ ret = iwl_mvm_load_nvm_to_nic(mvm);
+ if (ret)
+ goto error;
+ }
+
+ if (IWL_MVM_PARSE_NVM && !mvm->nvm_data) {
+ ret = iwl_nvm_init(mvm);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to read NVM: %d\n", ret);
+ goto error;
+ }
+ }
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(REGULATORY_AND_NVM_GROUP,
+ NVM_ACCESS_COMPLETE),
+ CMD_SEND_IN_RFKILL,
+ sizeof(nvm_complete), &nvm_complete);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to run complete NVM access: %d\n",
+ ret);
+ goto error;
+ }
+
+ ret = iwl_send_phy_cfg_cmd(mvm);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to run PHY configuration: %d\n",
+ ret);
+ goto error;
+ }
+
+ /* We wait for the INIT complete notification */
+ ret = iwl_wait_notification(&mvm->notif_wait, &init_wait,
+ MVM_UCODE_ALIVE_TIMEOUT);
+ if (ret)
+ return ret;
+
+ /* Read the NVM only at driver load time, no need to do this twice */
+ if (!IWL_MVM_PARSE_NVM && !mvm->nvm_data) {
+ mvm->nvm_data = iwl_get_nvm(mvm->trans, mvm->fw);
+ if (IS_ERR(mvm->nvm_data)) {
+ ret = PTR_ERR(mvm->nvm_data);
+ mvm->nvm_data = NULL;
+ IWL_ERR(mvm, "Failed to read NVM: %d\n", ret);
+ return ret;
+ }
+ }
+
+ mvm->rfkill_safe_init_done = true;
+
+ return 0;
+
+error:
+ iwl_remove_notification(&mvm->notif_wait, &init_wait);
+ return ret;
+}
+
int iwl_run_init_mvm_ucode(struct iwl_mvm *mvm)
{
struct iwl_notification_wait calib_wait;
@@ -1538,12 +1555,11 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
ret = iwl_send_phy_db_data(mvm->phy_db);
if (ret)
goto error;
+ ret = iwl_send_phy_cfg_cmd(mvm);
+ if (ret)
+ goto error;
}
- ret = iwl_send_phy_cfg_cmd(mvm);
- if (ret)
- goto error;
-
ret = iwl_mvm_send_bt_init_conf(mvm);
if (ret)
goto error;
@@ -1711,8 +1727,6 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
iwl_mvm_tas_init(mvm);
iwl_mvm_leds_sync(mvm);
- iwl_mvm_ftm_initiator_smooth_config(mvm);
-
if (fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_RFIM_SUPPORT)) {
if (iwl_mvm_eval_dsm_rfi(mvm) == DSM_VALUE_RFI_ENABLE)
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
index 82fad04..cc90f28 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
@@ -225,16 +225,20 @@ int iwl_mvm_mac_ctxt_init(struct iwl_mvm *mvm, struct ieee80211_vif *vif)
* that we should share it with another interface.
*/
- /* Currently, MAC ID 0 should be used only for the managed/IBSS vif */
- switch (vif->type) {
- case NL80211_IFTYPE_ADHOC:
- break;
- case NL80211_IFTYPE_STATION:
- if (!vif->p2p)
+ /* MAC ID 0 should be used only for the managed/IBSS vif with non-MLO
+ * FW API
+ */
+ if (!mvm->mld_api_is_used) {
+ switch (vif->type) {
+ case NL80211_IFTYPE_ADHOC:
break;
- fallthrough;
- default:
- __clear_bit(0, data.available_mac_ids);
+ case NL80211_IFTYPE_STATION:
+ if (!vif->p2p)
+ break;
+ fallthrough;
+ default:
+ __clear_bit(0, data.available_mac_ids);
+ }
}
ieee80211_iterate_active_interfaces_atomic(
@@ -870,17 +874,44 @@ static u32 iwl_mvm_find_ie_offset(u8 *beacon, u8 eid, u32 frame_size)
return ie - beacon;
}
-static u8 iwl_mvm_mac_ctxt_get_lowest_rate(struct iwl_mvm *mvm,
- struct ieee80211_tx_info *info,
- struct ieee80211_vif *vif)
+u8 iwl_mvm_mac_ctxt_get_lowest_rate(struct iwl_mvm *mvm,
+ struct ieee80211_tx_info *info,
+ struct ieee80211_vif *vif)
{
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
struct ieee80211_supported_band *sband;
unsigned long basic = vif->bss_conf.basic_rates;
u16 lowest_cck = IWL_RATE_COUNT, lowest_ofdm = IWL_RATE_COUNT;
+ u32 link_id = u32_get_bits(info->control.flags,
+ IEEE80211_TX_CTRL_MLO_LINK);
+ u8 band = info->band;
u8 rate;
u32 i;
- sband = mvm->hw->wiphy->bands[info->band];
+ if (link_id == IEEE80211_LINK_UNSPECIFIED && vif->valid_links) {
+ for (i = 0; i < ARRAY_SIZE(mvmvif->link); i++) {
+ if (!mvmvif->link[i])
+ continue;
+ /* shouldn't do this when >1 link is active */
+ WARN_ON_ONCE(link_id != IEEE80211_LINK_UNSPECIFIED);
+ link_id = i;
+ }
+ }
+
+ if (link_id < IEEE80211_LINK_UNSPECIFIED) {
+ struct ieee80211_bss_conf *link_conf;
+
+ rcu_read_lock();
+ link_conf = rcu_dereference(vif->link_conf[link_id]);
+ if (link_conf) {
+ basic = link_conf->basic_rates;
+ if (link_conf->chandef.chan)
+ band = link_conf->chandef.chan->band;
+ }
+ rcu_read_unlock();
+ }
+
+ sband = mvm->hw->wiphy->bands[band];
for_each_set_bit(i, &basic, BITS_PER_LONG) {
u16 hw = sband->bitrates[i].hw_value;
@@ -892,7 +923,9 @@ static u8 iwl_mvm_mac_ctxt_get_lowest_rate(struct iwl_mvm *mvm,
}
}
- if (info->band == NL80211_BAND_2GHZ && !vif->p2p) {
+ if (band == NL80211_BAND_2GHZ && !vif->p2p &&
+ vif->type != NL80211_IFTYPE_P2P_DEVICE &&
+ !(info->flags & IEEE80211_TX_CTL_NO_CCK_RATE)) {
if (lowest_cck != IWL_RATE_COUNT)
rate = lowest_cck;
else if (lowest_ofdm != IWL_RATE_COUNT)
@@ -1102,10 +1135,10 @@ static int iwl_mvm_mac_ctxt_send_beacon_v9(struct iwl_mvm *mvm,
sizeof(beacon_cmd));
}
-int iwl_mvm_mac_ctxt_send_beacon(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct sk_buff *beacon,
- struct ieee80211_bss_conf *link_conf)
+static int iwl_mvm_mac_ctxt_send_beacon(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct sk_buff *beacon,
+ struct ieee80211_bss_conf *link_conf)
{
if (WARN_ON(!beacon))
return -EINVAL;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
index aaa7e3c..0f01b62 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
@@ -227,14 +227,18 @@ int iwl_mvm_init_fw_regd(struct iwl_mvm *mvm)
static const u8 he_if_types_ext_capa_sta[] = {
[0] = WLAN_EXT_CAPA1_EXT_CHANNEL_SWITCHING,
[2] = WLAN_EXT_CAPA3_MULTI_BSSID_SUPPORT,
- [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF,
+ [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF |
+ WLAN_EXT_CAPA8_MAX_MSDU_IN_AMSDU_LSB,
+ [8] = WLAN_EXT_CAPA9_MAX_MSDU_IN_AMSDU_MSB,
};
static const u8 tm_if_types_ext_capa_sta[] = {
[0] = WLAN_EXT_CAPA1_EXT_CHANNEL_SWITCHING,
[2] = WLAN_EXT_CAPA3_MULTI_BSSID_SUPPORT |
WLAN_EXT_CAPA3_TIMING_MEASUREMENT_SUPPORT,
- [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF,
+ [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF |
+ WLAN_EXT_CAPA8_MAX_MSDU_IN_AMSDU_LSB,
+ [8] = WLAN_EXT_CAPA9_MAX_MSDU_IN_AMSDU_MSB,
[9] = WLAN_EXT_CAPA10_TWT_REQUESTER_SUPPORT,
};
@@ -301,6 +305,11 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
ieee80211_hw_set(hw, BUFF_MMPDU_TXQ);
ieee80211_hw_set(hw, STA_MMPDU_TXQ);
+ /* Set this early since we need to have it for the check below */
+ if (mvm->mld_api_is_used &&
+ mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ)
+ hw->wiphy->flags |= WIPHY_FLAG_SUPPORTS_MLO;
+
/* With MLD FW API, it tracks timing by itself,
* no need for any timing from the host
*/
@@ -409,6 +418,8 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
wiphy_ext_feature_set(hw->wiphy,
NL80211_EXT_FEATURE_BEACON_RATE_LEGACY);
+ wiphy_ext_feature_set(hw->wiphy,
+ NL80211_EXT_FEATURE_SCAN_MIN_PREQ_CONTENT);
if (fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_FTM_CALIBRATED)) {
@@ -1061,6 +1072,9 @@ static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm)
mvm->rx_ba_sessions = 0;
mvm->fwrt.dump.conf = FW_DBG_INVALID;
mvm->monitor_on = false;
+#ifdef CONFIG_IWLWIFI_DEBUGFS
+ mvm->beacon_inject_active = false;
+#endif
/* keep statistics ticking */
iwl_mvm_accu_radio_stats(mvm);
@@ -3305,8 +3319,8 @@ void iwl_mvm_sta_pre_rcu_remove(struct ieee80211_hw *hw,
lockdep_is_held(&mvm->mutex));
sta_id = link_sta->sta_id;
if (sta == rcu_access_pointer(mvm->fw_id_to_mac_id[sta_id])) {
- rcu_assign_pointer(mvm->fw_id_to_mac_id[sta_id],
- ERR_PTR(-ENOENT));
+ RCU_INIT_POINTER(mvm->fw_id_to_mac_id[sta_id],
+ ERR_PTR(-ENOENT));
RCU_INIT_POINTER(mvm->fw_id_to_link_sta[sta_id], NULL);
}
}
@@ -3568,24 +3582,22 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
*/
static void iwl_mvm_rs_rate_init_all_links(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
- struct ieee80211_sta *sta,
- bool update)
+ struct ieee80211_sta *sta)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
unsigned int link_id;
for_each_mvm_vif_valid_link(mvmvif, link_id) {
struct ieee80211_bss_conf *conf =
- link_conf_dereference_protected(vif, link_id);
+ link_conf_dereference_check(vif, link_id);
struct ieee80211_link_sta *link_sta =
- link_sta_dereference_protected(sta, link_id);
+ link_sta_dereference_check(sta, link_id);
if (!conf || !link_sta || !mvmvif->link[link_id]->phy_ctxt)
continue;
- iwl_mvm_rs_rate_init(mvm, sta, conf, link_sta,
- mvmvif->link[link_id]->phy_ctxt->channel->band,
- update);
+ iwl_mvm_rs_rate_init(mvm, vif, sta, conf, link_sta,
+ mvmvif->link[link_id]->phy_ctxt->channel->band);
}
}
@@ -3662,6 +3674,7 @@ iwl_mvm_sta_state_notexist_to_none(struct iwl_mvm *mvm,
struct ieee80211_sta *sta,
struct iwl_mvm_sta_state_ops *callbacks)
{
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
unsigned int i;
int ret;
@@ -3697,6 +3710,9 @@ iwl_mvm_sta_state_notexist_to_none(struct iwl_mvm *mvm,
}
ieee80211_sta_recalc_aggregates(sta);
+ if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls)
+ mvmvif->ap_sta = sta;
+
return 0;
}
@@ -3753,7 +3769,7 @@ iwl_mvm_sta_state_auth_to_assoc(struct ieee80211_hw *hw,
}
out:
- iwl_mvm_rs_rate_init_all_links(mvm, vif, sta, false);
+ iwl_mvm_rs_rate_init_all_links(mvm, vif, sta);
return callbacks->update_sta(mvm, vif, sta);
}
@@ -3786,7 +3802,9 @@ iwl_mvm_sta_state_assoc_to_authorized(struct iwl_mvm *mvm,
iwl_mvm_mei_host_associated(mvm, vif, mvm_sta);
}
- iwl_mvm_rs_rate_init_all_links(mvm, vif, sta, true);
+ mvm_sta->authorized = true;
+
+ iwl_mvm_rs_rate_init_all_links(mvm, vif, sta);
return 0;
}
@@ -3798,14 +3816,17 @@ iwl_mvm_sta_state_authorized_to_assoc(struct iwl_mvm *mvm,
struct iwl_mvm_sta_state_ops *callbacks)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
int ret;
lockdep_assert_held(&mvm->mutex);
+ mvmsta->authorized = false;
+
/* once we move into assoc state, need to update rate scale to
* disable using wide bandwidth
*/
- iwl_mvm_rs_rate_init_all_links(mvm, vif, sta, false);
+ iwl_mvm_rs_rate_init_all_links(mvm, vif, sta);
if (!sta->tdls) {
/* Set this but don't call iwl_mvm_mac_ctxt_changed()
@@ -3924,8 +3945,10 @@ int iwl_mvm_mac_sta_state_common(struct ieee80211_hw *hw,
ret = 0;
} else if (old_state == IEEE80211_STA_NONE &&
new_state == IEEE80211_STA_NOTEXIST) {
- if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls)
+ if (vif->type == NL80211_IFTYPE_STATION && !sta->tdls) {
iwl_mvm_stop_session_protection(mvm, vif);
+ mvmvif->ap_sta = NULL;
+ }
ret = callbacks->rm_sta(mvm, vif, sta);
if (sta->tdls) {
iwl_mvm_recalc_tdls_state(mvm, vif, false);
@@ -3968,15 +3991,11 @@ void iwl_mvm_sta_rc_update(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, u32 changed)
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
if (changed & (IEEE80211_RC_BW_CHANGED |
IEEE80211_RC_SUPP_RATES_CHANGED |
IEEE80211_RC_NSS_CHANGED))
- iwl_mvm_rs_rate_init(mvm, sta,
- &vif->bss_conf, &sta->deflink,
- mvmvif->deflink.phy_ctxt->channel->band,
- true);
+ iwl_mvm_rs_rate_init_all_links(mvm, vif, sta);
if (vif->type == NL80211_IFTYPE_STATION &&
changed & IEEE80211_RC_NSS_CHANGED)
@@ -4094,7 +4113,7 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
struct iwl_mvm_sta *mvmsta = NULL;
- struct iwl_mvm_key_pn *ptk_pn;
+ struct iwl_mvm_key_pn *ptk_pn = NULL;
int keyidx = key->keyidx;
u32 sec_key_id = WIDE_ID(DATA_PATH_GROUP, SEC_KEY_CMD);
u8 sec_key_ver = iwl_fw_lookup_cmd_ver(mvm->fw, sec_key_id, 0);
@@ -4252,6 +4271,10 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
if (ret) {
IWL_WARN(mvm, "set key failed\n");
key->hw_key_idx = STA_KEY_IDX_INVALID;
+ if (ptk_pn) {
+ RCU_INIT_POINTER(mvmsta->ptk_pn[keyidx], NULL);
+ kfree(ptk_pn);
+ }
/*
* can't add key for RX, but we don't need it
* in the device for TX so still return 0,
@@ -5588,6 +5611,7 @@ void iwl_mvm_mac_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct iwl_mvm_vif *mvmvif;
struct iwl_mvm_sta *mvmsta;
struct ieee80211_sta *sta;
+ bool ap_sta_done = false;
int i;
u32 msk = 0;
@@ -5616,8 +5640,14 @@ void iwl_mvm_mac_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
if (mvmsta->vif != vif)
continue;
+ if (sta == mvmvif->ap_sta) {
+ if (ap_sta_done)
+ continue;
+ ap_sta_done = true;
+ }
+
/* make sure only TDLS peers or the AP are flushed */
- WARN_ON_ONCE(i != mvmvif->deflink.ap_sta_id && !sta->tdls);
+ WARN_ON_ONCE(sta != mvmvif->ap_sta && !sta->tdls);
if (drop) {
if (iwl_mvm_flush_sta(mvm, mvmsta, false))
@@ -6129,7 +6159,7 @@ static bool iwl_mvm_mac_can_aggregate(struct ieee80211_hw *hw,
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
- if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ)
+ if (iwl_mvm_has_new_tx_csum(mvm))
return iwl_mvm_tx_csum_bz(mvm, head, true) ==
iwl_mvm_tx_csum_bz(mvm, skb, true);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-key.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-key.c
index f4785c0..8853821 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-key.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-key.c
@@ -14,23 +14,41 @@ static u32 iwl_mvm_get_sec_sta_mask(struct iwl_mvm *mvm,
struct ieee80211_key_conf *keyconf)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ struct iwl_mvm_vif_link_info *link_info = &mvmvif->deflink;
- if (vif->type == NL80211_IFTYPE_AP &&
- !(keyconf->flags & IEEE80211_KEY_FLAG_PAIRWISE))
- return BIT(mvmvif->deflink.mcast_sta.sta_id);
+ lockdep_assert_held(&mvm->mutex);
- if (sta) {
- struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
-
- return BIT(mvmsta->deflink.sta_id);
+ if (keyconf->link_id >= 0) {
+ link_info = mvmvif->link[keyconf->link_id];
+ if (!link_info)
+ return 0;
}
- if (vif->type == NL80211_IFTYPE_STATION &&
- mvmvif->deflink.ap_sta_id != IWL_MVM_INVALID_STA)
- return BIT(mvmvif->deflink.ap_sta_id);
+ /* AP group keys are per link and should be on the mcast STA */
+ if (vif->type == NL80211_IFTYPE_AP &&
+ !(keyconf->flags & IEEE80211_KEY_FLAG_PAIRWISE))
+ return BIT(link_info->mcast_sta.sta_id);
- /* invalid */
- return 0;
+ /* for client mode use the AP STA also for group keys */
+ if (!sta && vif->type == NL80211_IFTYPE_STATION)
+ sta = mvmvif->ap_sta;
+
+ /* During remove the STA was removed and the group keys come later
+ * (which sounds like a bad sequence, but remember that to mac80211 the
+ * group keys have no sta pointer), so we don't have a STA now.
+ * Since this happens for group keys only, just use the link_info as
+ * the group keys are per link; make sure that is the case by checking
+ * we do have a link_id or are not doing MLO.
+ * Of course the same can be done during add as well, but we must do
+ * it during remove, since we don't have the mvmvif->ap_sta pointer.
+ */
+ if (!sta && (keyconf->link_id >= 0 || !vif->valid_links))
+ return BIT(link_info->ap_sta_id);
+
+ /* STA should be non-NULL now, but iwl_mvm_sta_fw_id_mask() checks */
+
+ /* pass link_id to filter by it if not -1 (GTK on client) */
+ return iwl_mvm_sta_fw_id_mask(mvm, sta, keyconf->link_id);
}
static u32 iwl_mvm_get_sec_flags(struct iwl_mvm *mvm,
@@ -41,6 +59,8 @@ static u32 iwl_mvm_get_sec_flags(struct iwl_mvm *mvm,
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
u32 flags = 0;
+ lockdep_assert_held(&mvm->mutex);
+
if (!(keyconf->flags & IEEE80211_KEY_FLAG_PAIRWISE))
flags |= IWL_SEC_KEY_FLAG_MCAST_KEY;
@@ -68,22 +88,68 @@ static u32 iwl_mvm_get_sec_flags(struct iwl_mvm *mvm,
break;
}
- rcu_read_lock();
- if (!sta && vif->type == NL80211_IFTYPE_STATION &&
- mvmvif->deflink.ap_sta_id != IWL_MVM_INVALID_STA) {
- u8 sta_id = mvmvif->deflink.ap_sta_id;
-
- sta = rcu_dereference_check(mvm->fw_id_to_mac_id[sta_id],
- lockdep_is_held(&mvm->mutex));
- }
+ if (!sta && vif->type == NL80211_IFTYPE_STATION)
+ sta = mvmvif->ap_sta;
if (!IS_ERR_OR_NULL(sta) && sta->mfp)
flags |= IWL_SEC_KEY_FLAG_MFP;
- rcu_read_unlock();
return flags;
}
+struct iwl_mvm_sta_key_update_data {
+ struct ieee80211_sta *sta;
+ u32 old_sta_mask;
+ u32 new_sta_mask;
+ int err;
+};
+
+static void iwl_mvm_mld_update_sta_key(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ struct ieee80211_key_conf *key,
+ void *_data)
+{
+ u32 cmd_id = WIDE_ID(DATA_PATH_GROUP, SEC_KEY_CMD);
+ struct iwl_mvm_sta_key_update_data *data = _data;
+ struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
+ struct iwl_sec_key_cmd cmd = {
+ .action = cpu_to_le32(FW_CTXT_ACTION_MODIFY),
+ .u.modify.old_sta_mask = cpu_to_le32(data->old_sta_mask),
+ .u.modify.new_sta_mask = cpu_to_le32(data->new_sta_mask),
+ .u.modify.key_id = cpu_to_le32(key->keyidx),
+ .u.modify.key_flags =
+ cpu_to_le32(iwl_mvm_get_sec_flags(mvm, vif, sta, key)),
+ };
+ int err;
+
+ /* only need to do this for pairwise keys (link_id == -1) */
+ if (sta != data->sta || key->link_id >= 0)
+ return;
+
+ err = iwl_mvm_send_cmd_pdu(mvm, cmd_id, CMD_ASYNC, sizeof(cmd), &cmd);
+
+ if (err)
+ data->err = err;
+}
+
+int iwl_mvm_mld_update_sta_keys(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ u32 old_sta_mask,
+ u32 new_sta_mask)
+{
+ struct iwl_mvm_sta_key_update_data data = {
+ .sta = sta,
+ .old_sta_mask = old_sta_mask,
+ .new_sta_mask = new_sta_mask,
+ };
+
+ ieee80211_iter_keys_rcu(mvm->hw, vif, iwl_mvm_mld_update_sta_key,
+ &data);
+ return data.err;
+}
+
static int __iwl_mvm_sec_key_del(struct iwl_mvm *mvm, u32 sta_mask,
u32 key_flags, u32 keyidx, u32 flags)
{
@@ -118,6 +184,9 @@ int iwl_mvm_sec_key_add(struct iwl_mvm *mvm,
if (WARN_ON(keyconf->keylen > sizeof(cmd.u.add.key)))
return -EINVAL;
+ if (WARN_ON(!sta_mask))
+ return -EINVAL;
+
if (keyconf->cipher == WLAN_CIPHER_SUITE_WEP40 ||
keyconf->cipher == WLAN_CIPHER_SUITE_WEP104)
memcpy(cmd.u.add.key + IWL_SEC_WEP_KEY_OFFSET, keyconf->key,
@@ -164,6 +233,9 @@ static int _iwl_mvm_sec_key_del(struct iwl_mvm *mvm,
u32 key_flags = iwl_mvm_get_sec_flags(mvm, vif, sta, keyconf);
int ret;
+ if (WARN_ON(!sta_mask))
+ return -EINVAL;
+
ret = __iwl_mvm_sec_key_del(mvm, sta_mask, key_flags, keyconf->keyidx,
flags);
if (ret)
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac.c
index ab0ba85..1717fb5 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac.c
@@ -4,6 +4,16 @@
*/
#include "mvm.h"
+static void iwl_mvm_mld_set_he_support(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct iwl_mac_config_cmd *cmd)
+{
+ if (vif->type == NL80211_IFTYPE_AP)
+ cmd->he_ap_support = cpu_to_le16(1);
+ else
+ cmd->he_support = cpu_to_le16(1);
+}
+
static void iwl_mvm_mld_mac_ctxt_cmd_common(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct iwl_mac_config_cmd *cmd,
@@ -41,7 +51,7 @@ static void iwl_mvm_mld_mac_ctxt_cmd_common(struct iwl_mvm *mvm,
* and enable both when we have MLO.
*/
if (vif->valid_links) {
- cmd->he_support = cpu_to_le32(1);
+ iwl_mvm_mld_set_he_support(mvm, vif, cmd);
cmd->eht_support = cpu_to_le32(1);
return;
}
@@ -53,7 +63,7 @@ static void iwl_mvm_mld_mac_ctxt_cmd_common(struct iwl_mvm *mvm,
continue;
if (link_conf->he_support)
- cmd->he_support = cpu_to_le32(1);
+ iwl_mvm_mld_set_he_support(mvm, vif, cmd);
/* it's not reasonable to have EHT without HE and FW API doesn't
* support it. Ignore EHT in this case.
@@ -157,7 +167,6 @@ static int iwl_mvm_mld_mac_ctxt_cmd_ibss(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
u32 action)
{
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
struct iwl_mac_config_cmd cmd = {};
WARN_ON(vif->type != NL80211_IFTYPE_ADHOC);
@@ -168,9 +177,6 @@ static int iwl_mvm_mld_mac_ctxt_cmd_ibss(struct iwl_mvm *mvm,
MAC_CFG_FILTER_ACCEPT_PROBE_REQ |
MAC_CFG_FILTER_ACCEPT_GRP);
- /* TODO: Assumes that the beacon id == mac context id */
- cmd.go_ibss.beacon_template = cpu_to_le32(mvmvif->id);
-
return iwl_mvm_mld_mac_ctxt_send_cmd(mvm, &cmd);
}
@@ -210,9 +216,6 @@ static int iwl_mvm_mld_mac_ctxt_cmd_ap_go(struct iwl_mvm *mvm,
MAC_CFG_FILTER_ACCEPT_PROBE_REQ,
MAC_CFG_FILTER_ACCEPT_BEACON);
- /* TODO: Assume that the beacon id == mac context id */
- cmd.go_ibss.beacon_template = cpu_to_le32(mvmvif->id);
-
return iwl_mvm_mld_mac_ctxt_send_cmd(mvm, &cmd);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c
index 203f251..fbc2d5e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c
@@ -256,6 +256,30 @@ __iwl_mvm_mld_assign_vif_chanctx(struct iwl_mvm *mvm,
if (ret)
goto out;
+ /* Initialize rate control for the AP station, since we might be
+ * doing a link switch here - we cannot initialize it before since
+ * this needs the phy context assigned (and in FW?), and we cannot
+ * do it later because it needs to be initialized as soon as we're
+ * able to TX on the link, i.e. when active.
+ *
+ * Firmware restart isn't quite correct yet for MLO, but we don't
+ * need to do it in that case anyway since it will happen from the
+ * normal station state callback.
+ */
+ if (mvmvif->ap_sta &&
+ !test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status)) {
+ struct ieee80211_link_sta *link_sta;
+
+ rcu_read_lock();
+ link_sta = rcu_dereference(mvmvif->ap_sta->link[link_id]);
+
+ if (!WARN_ON_ONCE(!link_sta))
+ iwl_mvm_rs_rate_init(mvm, vif, mvmvif->ap_sta,
+ link_conf, link_sta,
+ phy_ctxt->channel->band);
+ rcu_read_unlock();
+ }
+
/* then activate */
ret = iwl_mvm_link_changed(mvm, vif, link_conf,
LINK_CONTEXT_MODIFY_ACTIVE |
@@ -882,7 +906,10 @@ iwl_mvm_mld_change_vif_links(struct ieee80211_hw *hw,
n_active++;
}
- if (n_active > 1)
+ if (vif->type == NL80211_IFTYPE_AP &&
+ n_active > mvm->fw->ucode_capa.num_beacons)
+ return -EOPNOTSUPP;
+ else if (n_active > 1)
return -EOPNOTSUPP;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c
index 78d4f18..0bfdf44 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c
@@ -4,6 +4,43 @@
*/
#include "mvm.h"
#include "time-sync.h"
+#include "sta.h"
+
+u32 iwl_mvm_sta_fw_id_mask(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
+ int filter_link_id)
+{
+ struct iwl_mvm_sta *mvmsta;
+ unsigned int link_id;
+ u32 result = 0;
+
+ if (!sta)
+ return 0;
+
+ mvmsta = iwl_mvm_sta_from_mac80211(sta);
+
+ /* it's easy when the STA is not an MLD */
+ if (!sta->valid_links)
+ return BIT(mvmsta->deflink.sta_id);
+
+ /* but if it is an MLD, get the mask of all the FW STAs it has ... */
+ for (link_id = 0; link_id < ARRAY_SIZE(mvmsta->link); link_id++) {
+ struct iwl_mvm_link_sta *link_sta;
+
+ /* unless we have a specific link in mind */
+ if (filter_link_id >= 0 && link_id != filter_link_id)
+ continue;
+
+ link_sta =
+ rcu_dereference_check(mvmsta->link[link_id],
+ lockdep_is_held(&mvm->mutex));
+ if (!link_sta)
+ continue;
+
+ result |= BIT(link_sta->sta_id);
+ }
+
+ return result;
+}
static int iwl_mvm_mld_send_sta_cmd(struct iwl_mvm *mvm,
struct iwl_mvm_sta_cfg_cmd *cmd)
@@ -262,19 +299,22 @@ int iwl_mvm_mld_add_aux_sta(struct iwl_mvm *mvm, u32 lmac_id)
IWL_MAX_TID_COUNT, NULL);
}
-static int iwl_mvm_mld_disable_txq(struct iwl_mvm *mvm, int sta_id,
+static int iwl_mvm_mld_disable_txq(struct iwl_mvm *mvm, u32 sta_mask,
u16 *queueptr, u8 tid)
{
int queue = *queueptr;
int ret = 0;
+ if (tid == IWL_MAX_TID_COUNT)
+ tid = IWL_MGMT_TID;
+
if (mvm->sta_remove_requires_queue_remove) {
u32 cmd_id = WIDE_ID(DATA_PATH_GROUP,
SCD_QUEUE_CONFIG_CMD);
struct iwl_scd_queue_cfg_cmd remove_cmd = {
.operation = cpu_to_le32(IWL_SCD_QUEUE_REMOVE),
.u.remove.tid = cpu_to_le32(tid),
- .u.remove.sta_mask = cpu_to_le32(BIT(sta_id)),
+ .u.remove.sta_mask = cpu_to_le32(sta_mask),
};
ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, 0,
@@ -304,7 +344,7 @@ static int iwl_mvm_mld_rm_int_sta(struct iwl_mvm *mvm,
if (flush)
iwl_mvm_flush_sta(mvm, int_sta, true);
- iwl_mvm_mld_disable_txq(mvm, int_sta->sta_id, queuptr, tid);
+ iwl_mvm_mld_disable_txq(mvm, BIT(int_sta->sta_id), queuptr, tid);
ret = iwl_mvm_mld_rm_sta_from_fw(mvm, int_sta->sta_id);
if (ret)
@@ -387,7 +427,6 @@ static int iwl_mvm_mld_cfg_sta(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
struct iwl_mvm_sta_cfg_cmd cmd = {
.sta_id = cpu_to_le32(mvm_link_sta->sta_id),
.station_type = cpu_to_le32(mvm_sta->sta_type),
- .mfp = cpu_to_le32(sta->mfp),
};
u32 agg_size = 0, mpdu_dens = 0;
@@ -720,6 +759,7 @@ static void iwl_mvm_mld_disable_sta_queues(struct iwl_mvm *mvm,
struct ieee80211_sta *sta)
{
struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
+ u32 sta_mask = iwl_mvm_sta_fw_id_mask(mvm, sta, -1);
int i;
lockdep_assert_held(&mvm->mutex);
@@ -728,7 +768,7 @@ static void iwl_mvm_mld_disable_sta_queues(struct iwl_mvm *mvm,
if (mvm_sta->tid_data[i].txq_id == IWL_MVM_INVALID_QUEUE)
continue;
- iwl_mvm_mld_disable_txq(mvm, mvm_sta->deflink.sta_id,
+ iwl_mvm_mld_disable_txq(mvm, sta_mask,
&mvm_sta->tid_data[i].txq_id, i);
mvm_sta->tid_data[i].txq_id = IWL_MVM_INVALID_QUEUE;
}
@@ -870,11 +910,12 @@ void iwl_mvm_mld_modify_all_sta_disable_tx(struct iwl_mvm *mvm,
rcu_read_unlock();
}
-static int iwl_mvm_mld_update_sta_queue(struct iwl_mvm *mvm,
- struct iwl_mvm_sta *mvm_sta,
- u32 old_sta_mask,
- u32 new_sta_mask)
+static int iwl_mvm_mld_update_sta_queues(struct iwl_mvm *mvm,
+ struct ieee80211_sta *sta,
+ u32 old_sta_mask,
+ u32 new_sta_mask)
{
+ struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
struct iwl_scd_queue_cfg_cmd cmd = {
.operation = cpu_to_le32(IWL_SCD_QUEUE_MODIFY),
.u.modify.old_sta_mask = cpu_to_le32(old_sta_mask),
@@ -910,6 +951,70 @@ static int iwl_mvm_mld_update_sta_queue(struct iwl_mvm *mvm,
return 0;
}
+static int iwl_mvm_mld_update_sta_baids(struct iwl_mvm *mvm,
+ u32 old_sta_mask,
+ u32 new_sta_mask)
+{
+ struct iwl_rx_baid_cfg_cmd cmd = {
+ .action = cpu_to_le32(IWL_RX_BAID_ACTION_MODIFY),
+ .modify.old_sta_id_mask = cpu_to_le32(old_sta_mask),
+ .modify.new_sta_id_mask = cpu_to_le32(new_sta_mask),
+ };
+ u32 cmd_id = WIDE_ID(DATA_PATH_GROUP, RX_BAID_ALLOCATION_CONFIG_CMD);
+ int baid;
+
+ BUILD_BUG_ON(sizeof(struct iwl_rx_baid_cfg_resp) != sizeof(baid));
+
+ for (baid = 0; baid < ARRAY_SIZE(mvm->baid_map); baid++) {
+ struct iwl_mvm_baid_data *data;
+ int ret;
+
+ data = rcu_dereference_protected(mvm->baid_map[baid],
+ lockdep_is_held(&mvm->mutex));
+ if (!data)
+ continue;
+
+ if (!(data->sta_mask & old_sta_mask))
+ continue;
+
+ WARN_ONCE(data->sta_mask != old_sta_mask,
+ "BAID data for %d corrupted - expected 0x%x found 0x%x\n",
+ baid, old_sta_mask, data->sta_mask);
+
+ cmd.modify.tid = cpu_to_le32(data->tid);
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, 0, sizeof(cmd), &cmd);
+ data->sta_mask = new_sta_mask;
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int iwl_mvm_mld_update_sta_resources(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ u32 old_sta_mask,
+ u32 new_sta_mask)
+{
+ int ret;
+
+ ret = iwl_mvm_mld_update_sta_queues(mvm, sta,
+ old_sta_mask,
+ new_sta_mask);
+ if (ret)
+ return ret;
+
+ ret = iwl_mvm_mld_update_sta_keys(mvm, vif, sta,
+ old_sta_mask,
+ new_sta_mask);
+ if (ret)
+ return ret;
+
+ return iwl_mvm_mld_update_sta_baids(mvm, old_sta_mask, new_sta_mask);
+}
+
int iwl_mvm_mld_update_sta_links(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
@@ -946,9 +1051,10 @@ int iwl_mvm_mld_update_sta_links(struct iwl_mvm *mvm,
}
if (sta_mask_to_rem) {
- ret = iwl_mvm_mld_update_sta_queue(mvm, mvm_sta,
- current_sta_mask,
- current_sta_mask & ~sta_mask_to_rem);
+ ret = iwl_mvm_mld_update_sta_resources(mvm, vif, sta,
+ current_sta_mask,
+ current_sta_mask &
+ ~sta_mask_to_rem);
if (WARN_ON(ret))
goto err;
@@ -1020,12 +1126,15 @@ int iwl_mvm_mld_update_sta_links(struct iwl_mvm *mvm,
goto err;
link_sta_added_to_fw |= BIT(link_id);
+
+ iwl_mvm_rs_add_sta_link(mvm, mvm_sta_link);
}
if (sta_mask_added) {
- ret = iwl_mvm_mld_update_sta_queue(mvm, mvm_sta,
- current_sta_mask,
- current_sta_mask | sta_mask_added);
+ ret = iwl_mvm_mld_update_sta_resources(mvm, vif, sta,
+ current_sta_mask,
+ current_sta_mask |
+ sta_mask_added);
if (WARN_ON(ret))
goto err;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
index e32ce87..6e7470d 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
@@ -434,6 +434,8 @@ struct iwl_mvm_vif {
/* TCP Checksum Offload */
netdev_features_t features;
+ struct ieee80211_sta *ap_sta;
+
/* we can only have 2 GTK + 2 IGTK active at a time */
struct ieee80211_key_conf *ap_early_keys[4];
@@ -678,7 +680,7 @@ __aligned(roundup_pow_of_two(sizeof(struct _iwl_mvm_reorder_buf_entry)))
/**
* struct iwl_mvm_baid_data - BA session data
- * @sta_id: station id
+ * @sta_mask: current station mask for the BAID
* @tid: tid of the session
* @baid baid of the session
* @timeout: the timeout set in the addba request
@@ -692,7 +694,7 @@ __aligned(roundup_pow_of_two(sizeof(struct _iwl_mvm_reorder_buf_entry)))
*/
struct iwl_mvm_baid_data {
struct rcu_head rcu_head;
- u8 sta_id;
+ u32 sta_mask;
u8 tid;
u8 baid;
u16 timeout;
@@ -824,6 +826,12 @@ struct iwl_time_sync_data {
bool active;
};
+struct iwl_mei_scan_filter {
+ bool is_mei_limited_scan;
+ struct sk_buff_head scan_res;
+ struct work_struct scan_work;
+};
+
struct iwl_mvm {
/* for logger access */
struct device *dev;
@@ -1175,6 +1183,8 @@ struct iwl_mvm {
bool pldr_sync;
struct iwl_time_sync_data time_sync;
+
+ struct iwl_mei_scan_filter mei_scan_filter;
};
/* Extract MVM priv from op_mode and _hw */
@@ -1401,24 +1411,8 @@ static inline bool iwl_mvm_has_new_rx_api(struct iwl_mvm *mvm)
static inline bool iwl_mvm_has_mld_api(const struct iwl_fw *fw)
{
- return (iwl_fw_lookup_cmd_ver(fw, LINK_CONFIG_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN) &&
- (iwl_fw_lookup_cmd_ver(fw, MAC_CONFIG_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN) &&
- (iwl_fw_lookup_cmd_ver(fw, STA_CONFIG_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN) &&
- (iwl_fw_lookup_cmd_ver(fw, AUX_STA_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN) &&
- (iwl_fw_lookup_cmd_ver(fw, STA_REMOVE_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN) &&
- (iwl_fw_lookup_cmd_ver(fw, STA_DISABLE_TX_CMD,
- IWL_FW_CMD_VER_UNKNOWN) !=
- IWL_FW_CMD_VER_UNKNOWN);
+ return fw_has_capa(&fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_MLD_API_SUPPORT);
}
static inline bool iwl_mvm_has_new_tx_api(struct iwl_mvm *mvm)
@@ -1522,6 +1516,19 @@ static inline bool iwl_mvm_is_ctdp_supported(struct iwl_mvm *mvm)
IWL_UCODE_TLV_CAPA_CTDP_SUPPORT);
}
+static inline bool iwl_mvm_has_new_tx_csum(struct iwl_mvm *mvm)
+{
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ)
+ return false;
+
+ if (mvm->trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_BZ &&
+ CSR_HW_REV_TYPE(mvm->trans->hw_rev) == IWL_CFG_MAC_TYPE_GL &&
+ mvm->trans->hw_rev_step <= SILICON_B_STEP)
+ return false;
+
+ return true;
+}
+
extern const u8 iwl_mvm_ac_to_tx_fifo[];
extern const u8 iwl_mvm_ac_to_gen2_tx_fifo[];
@@ -1781,16 +1788,15 @@ int iwl_mvm_mac_ctxt_remove(struct iwl_mvm *mvm, struct ieee80211_vif *vif);
int iwl_mvm_mac_ctxt_beacon_changed(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct ieee80211_bss_conf *link_conf);
-int iwl_mvm_mac_ctxt_send_beacon(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct sk_buff *beacon,
- struct ieee80211_bss_conf *link_conf);
int iwl_mvm_mac_ctxt_send_beacon_cmd(struct iwl_mvm *mvm,
struct sk_buff *beacon,
void *data, int len);
u8 iwl_mvm_mac_ctxt_get_beacon_rate(struct iwl_mvm *mvm,
struct ieee80211_tx_info *info,
struct ieee80211_vif *vif);
+u8 iwl_mvm_mac_ctxt_get_lowest_rate(struct iwl_mvm *mvm,
+ struct ieee80211_tx_info *info,
+ struct ieee80211_vif *vif);
u16 iwl_mvm_mac_ctxt_get_beacon_flags(const struct iwl_fw *fw,
u8 rate_idx);
void iwl_mvm_mac_ctxt_set_tim(struct iwl_mvm *mvm,
@@ -2306,6 +2312,7 @@ void iwl_mvm_event_frame_timeout_callback(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
const struct ieee80211_sta *sta,
u16 tid);
+void iwl_mvm_mei_scan_filter_init(struct iwl_mei_scan_filter *mei_scan_filter);
void iwl_mvm_ptp_init(struct iwl_mvm *mvm);
void iwl_mvm_ptp_remove(struct iwl_mvm *mvm);
@@ -2334,6 +2341,11 @@ void iwl_mvm_sec_key_remove_ap(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct iwl_mvm_vif_link_info *link,
unsigned int link_id);
+int iwl_mvm_mld_update_sta_keys(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ u32 old_sta_mask,
+ u32 new_sta_mask);
int iwl_rfi_send_config_cmd(struct iwl_mvm *mvm,
struct iwl_rfi_lut_entry *rfi_table);
@@ -2512,6 +2524,22 @@ static inline void iwl_mvm_mei_set_sw_rfkill_state(struct iwl_mvm *mvm)
sw_rfkill);
}
+static inline bool iwl_mvm_mei_filter_scan(struct iwl_mvm *mvm,
+ struct sk_buff *skb)
+{
+ struct ieee80211_mgmt *mgmt = (void *)skb->data;
+
+ if (mvm->mei_scan_filter.is_mei_limited_scan &&
+ (ieee80211_is_probe_resp(mgmt->frame_control) ||
+ ieee80211_is_beacon(mgmt->frame_control))) {
+ skb_queue_tail(&mvm->mei_scan_filter.scan_res, skb);
+ schedule_work(&mvm->mei_scan_filter.scan_work);
+ return true;
+ }
+
+ return false;
+}
+
void iwl_mvm_send_roaming_forbidden_event(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
bool forbidden);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
index 56a4e9d..32625bf 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
@@ -472,8 +472,8 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = {
HCMD_NAME(SCAN_OFFLOAD_PROFILES_QUERY_CMD),
HCMD_NAME(BT_COEX_UPDATE_REDUCED_TXP),
HCMD_NAME(BT_COEX_CI),
- HCMD_NAME(WNM_80211V_TIMING_MEASUREMENT_CONFIRM_NOTIFICATION),
HCMD_NAME(WNM_80211V_TIMING_MEASUREMENT_NOTIFICATION),
+ HCMD_NAME(WNM_80211V_TIMING_MEASUREMENT_CONFIRM_NOTIFICATION),
HCMD_NAME(PHY_CONFIGURATION_CMD),
HCMD_NAME(CALIB_RES_NOTIF_PHY_DB),
HCMD_NAME(PHY_DB_CMD),
@@ -1020,10 +1020,14 @@ static void iwl_mvm_me_conn_status(void *priv, const struct iwl_mei_conn_info *c
kfree_rcu(prev_conn_info, rcu_head);
}
-static void iwl_mvm_mei_rfkill(void *priv, bool blocked)
+static void iwl_mvm_mei_rfkill(void *priv, bool blocked,
+ bool csme_taking_ownership)
{
struct iwl_mvm *mvm = priv;
+ if (blocked && !csme_taking_ownership)
+ return;
+
mvm->mei_rfkill_blocked = blocked;
if (!mvm->hw_registered)
return;
@@ -1367,12 +1371,16 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
else
memset(&mvm->rx_stats, 0, sizeof(struct mvm_statistics_rx));
+ iwl_mvm_ftm_initiator_smooth_config(mvm);
+
iwl_mvm_init_time_sync(&mvm->time_sync);
mvm->debugfs_dir = dbgfs_dir;
mvm->mei_registered = !iwl_mei_register(mvm, &mei_ops);
+ iwl_mvm_mei_scan_filter_init(&mvm->mei_scan_filter);
+
if (iwl_mvm_start_get_nvm(mvm)) {
/*
* Getting NVM failed while CSME is the owner, but we are
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
index c8ba2fe..c3a00bf 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
@@ -61,12 +61,14 @@ static u8 rs_fw_sgi_cw_support(struct ieee80211_link_sta *link_sta)
}
static u16 rs_fw_get_config_flags(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
struct ieee80211_link_sta *link_sta,
struct ieee80211_supported_band *sband)
{
struct ieee80211_sta_ht_cap *ht_cap = &link_sta->ht_cap;
struct ieee80211_sta_vht_cap *vht_cap = &link_sta->vht_cap;
struct ieee80211_sta_he_cap *he_cap = &link_sta->he_cap;
+ const struct ieee80211_sta_he_cap *sband_he_cap;
bool vht_ena = vht_cap->vht_supported;
u16 flags = 0;
@@ -92,17 +94,19 @@ static u16 rs_fw_get_config_flags(struct iwl_mvm *mvm,
IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD))
flags |= IWL_TLC_MNG_CFG_FLAGS_LDPC_MSK;
- if (sband->iftype_data && sband->iftype_data->he_cap.has_he &&
- !(sband->iftype_data->he_cap.he_cap_elem.phy_cap_info[1] &
- IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD))
+ sband_he_cap = ieee80211_get_he_iftype_cap(sband,
+ ieee80211_vif_type_p2p(vif));
+ if (sband_he_cap &&
+ !(sband_he_cap->he_cap_elem.phy_cap_info[1] &
+ IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD))
flags &= ~IWL_TLC_MNG_CFG_FLAGS_LDPC_MSK;
if (he_cap->has_he &&
(he_cap->he_cap_elem.phy_cap_info[3] &
IEEE80211_HE_PHY_CAP3_DCM_MAX_CONST_RX_MASK &&
- sband->iftype_data &&
- sband->iftype_data->he_cap.he_cap_elem.phy_cap_info[3] &
- IEEE80211_HE_PHY_CAP3_DCM_MAX_CONST_TX_MASK))
+ sband_he_cap &&
+ sband_he_cap->he_cap_elem.phy_cap_info[3] &
+ IEEE80211_HE_PHY_CAP3_DCM_MAX_CONST_TX_MASK))
flags |= IWL_TLC_MNG_CFG_FLAGS_HE_DCM_NSS_1_MSK;
return flags;
@@ -283,7 +287,8 @@ rs_fw_rs_mcs2eht_mcs(enum IWL_TLC_MCS_PER_BW bw,
}
static void
-rs_fw_eht_set_enabled_rates(const struct ieee80211_link_sta *link_sta,
+rs_fw_eht_set_enabled_rates(struct ieee80211_vif *vif,
+ const struct ieee80211_link_sta *link_sta,
struct ieee80211_supported_band *sband,
struct iwl_tlc_config_cmd_v4 *cmd)
{
@@ -299,7 +304,8 @@ rs_fw_eht_set_enabled_rates(const struct ieee80211_link_sta *link_sta,
struct ieee80211_eht_mcs_nss_supp_20mhz_only mcs_tx_20;
/* peer is 20Mhz only */
- if (!(link_sta->he_cap.he_cap_elem.phy_cap_info[0] &
+ if (vif->type == NL80211_IFTYPE_AP &&
+ !(link_sta->he_cap.he_cap_elem.phy_cap_info[0] &
IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_MASK_ALL)) {
mcs_rx_20 = eht_rx_mcs->only_20mhz;
} else {
@@ -361,7 +367,8 @@ rs_fw_eht_set_enabled_rates(const struct ieee80211_link_sta *link_sta,
sizeof(cmd->ht_rates[IWL_TLC_NSS_2]));
}
-static void rs_fw_set_supp_rates(struct ieee80211_link_sta *link_sta,
+static void rs_fw_set_supp_rates(struct ieee80211_vif *vif,
+ struct ieee80211_link_sta *link_sta,
struct ieee80211_supported_band *sband,
struct iwl_tlc_config_cmd_v4 *cmd)
{
@@ -383,7 +390,7 @@ static void rs_fw_set_supp_rates(struct ieee80211_link_sta *link_sta,
/* HT/VHT rates */
if (link_sta->eht_cap.has_eht) {
cmd->mode = IWL_TLC_MNG_MODE_EHT;
- rs_fw_eht_set_enabled_rates(link_sta, sband, cmd);
+ rs_fw_eht_set_enabled_rates(vif, link_sta, sband, cmd);
} else if (he_cap->has_he) {
cmd->mode = IWL_TLC_MNG_MODE_HE;
rs_fw_he_set_enabled_rates(link_sta, sband, cmd);
@@ -557,10 +564,12 @@ u16 rs_fw_get_max_amsdu_len(struct ieee80211_sta *sta,
return 0;
}
-void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct ieee80211_bss_conf *link_conf,
- struct ieee80211_link_sta *link_sta,
- enum nl80211_band band, bool update)
+void iwl_mvm_rs_fw_rate_init(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ struct ieee80211_bss_conf *link_conf,
+ struct ieee80211_link_sta *link_sta,
+ enum nl80211_band band)
{
struct ieee80211_hw *hw = mvm->hw;
struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
@@ -570,10 +579,9 @@ void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
struct iwl_mvm_link_sta *mvm_link_sta;
struct iwl_lq_sta_rs_fw *lq_sta;
struct iwl_tlc_config_cmd_v4 cfg_cmd = {
- .max_ch_width = update ?
- rs_fw_bw_from_sta_bw(link_sta) :
- IWL_TLC_MNG_CH_WIDTH_20MHZ,
- .flags = cpu_to_le16(rs_fw_get_config_flags(mvm, link_sta,
+ .max_ch_width = mvmsta->authorized ?
+ rs_fw_bw_from_sta_bw(link_sta) : IWL_TLC_MNG_CH_WIDTH_20MHZ,
+ .flags = cpu_to_le16(rs_fw_get_config_flags(mvm, vif, link_sta,
sband)),
.chains = rs_fw_set_active_chains(iwl_mvm_get_valid_tx_ant(mvm)),
.sgi_ch_width_supp = rs_fw_sgi_cw_support(link_sta),
@@ -601,7 +609,7 @@ void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
#ifdef CONFIG_IWLWIFI_DEBUGFS
iwl_mvm_reset_frame_stats(mvm);
#endif
- rs_fw_set_supp_rates(link_sta, sband, &cfg_cmd);
+ rs_fw_set_supp_rates(vif, link_sta, sband, &cfg_cmd);
/*
* since TLC offload works with one mode we can assume
@@ -671,6 +679,26 @@ int rs_fw_tx_protection(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta,
return 0;
}
+void iwl_mvm_rs_add_sta_link(struct iwl_mvm *mvm,
+ struct iwl_mvm_link_sta *link_sta)
+{
+ struct iwl_lq_sta_rs_fw *lq_sta;
+
+ lq_sta = &link_sta->lq_sta.rs_fw;
+
+ lq_sta->pers.drv = mvm;
+ lq_sta->pers.sta_id = link_sta->sta_id;
+ lq_sta->pers.chains = 0;
+ memset(lq_sta->pers.chain_signal, 0,
+ sizeof(lq_sta->pers.chain_signal));
+ lq_sta->pers.last_rssi = S8_MIN;
+ lq_sta->last_rate_n_flags = 0;
+
+#ifdef CONFIG_MAC80211_DEBUGFS
+ lq_sta->pers.dbg_fixed_rate = 0;
+#endif
+}
+
void iwl_mvm_rs_add_sta(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta)
{
unsigned int link_id;
@@ -678,25 +706,12 @@ void iwl_mvm_rs_add_sta(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta)
IWL_DEBUG_RATE(mvm, "create station rate scale window\n");
for (link_id = 0; link_id < ARRAY_SIZE(mvmsta->link); link_id++) {
- struct iwl_lq_sta_rs_fw *lq_sta;
struct iwl_mvm_link_sta *link =
rcu_dereference_protected(mvmsta->link[link_id],
lockdep_is_held(&mvm->mutex));
if (!link)
continue;
- lq_sta = &link->lq_sta.rs_fw;
-
- lq_sta->pers.drv = mvm;
- lq_sta->pers.sta_id = link->sta_id;
- lq_sta->pers.chains = 0;
- memset(lq_sta->pers.chain_signal, 0,
- sizeof(lq_sta->pers.chain_signal));
- lq_sta->pers.last_rssi = S8_MIN;
- lq_sta->last_rate_n_flags = 0;
-
-#ifdef CONFIG_MAC80211_DEBUGFS
- lq_sta->pers.dbg_fixed_rate = 0;
-#endif
+ iwl_mvm_rs_add_sta_link(mvm, link);
}
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
index ab82965..a4c1e3b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
@@ -3015,9 +3015,9 @@ static void rs_drv_rate_update(void *mvm_r,
for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++)
ieee80211_stop_tx_ba_session(sta, tid);
- iwl_mvm_rs_rate_init(mvm, sta,
+ iwl_mvm_rs_rate_init(mvm, mvmsta->vif, sta,
&mvmsta->vif->bss_conf, &sta->deflink,
- sband->band, true);
+ sband->band);
}
static void __iwl_mvm_rs_tx_status(struct iwl_mvm *mvm,
@@ -4101,13 +4101,16 @@ static const struct rate_control_ops rs_mvm_ops_drv = {
.capa = RATE_CTRL_CAPA_VHT_EXT_NSS_BW,
};
-void iwl_mvm_rs_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
+void iwl_mvm_rs_rate_init(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
struct ieee80211_bss_conf *link_conf,
struct ieee80211_link_sta *link_sta,
- enum nl80211_band band, bool update)
+ enum nl80211_band band)
{
if (iwl_mvm_has_tlc_offload(mvm)) {
- rs_fw_rate_init(mvm, sta, link_conf, link_sta, band, update);
+ iwl_mvm_rs_fw_rate_init(mvm, vif, sta, link_conf,
+ link_sta, band);
} else {
struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs.h b/drivers/net/wireless/intel/iwlwifi/mvm/rs.h
index f99603b..1ca375a 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs.h
@@ -394,10 +394,12 @@ struct iwl_lq_sta {
((_c) << RS_DRV_DATA_LQ_COLOR_POS)))
/* Initialize station's rate scaling information after adding station */
-void iwl_mvm_rs_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
+void iwl_mvm_rs_rate_init(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
struct ieee80211_bss_conf *link_conf,
struct ieee80211_link_sta *link_sta,
- enum nl80211_band band, bool update);
+ enum nl80211_band band);
/* Notify RS about Tx status */
void iwl_mvm_rs_tx_status(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
@@ -432,11 +434,18 @@ int iwl_mvm_tx_protection(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta,
void iwl_mvm_reset_frame_stats(struct iwl_mvm *mvm);
#endif
+struct iwl_mvm_link_sta;
+
void iwl_mvm_rs_add_sta(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta);
-void rs_fw_rate_init(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct ieee80211_bss_conf *link_conf,
- struct ieee80211_link_sta *link_sta,
- enum nl80211_band band, bool update);
+void iwl_mvm_rs_add_sta_link(struct iwl_mvm *mvm,
+ struct iwl_mvm_link_sta *link_sta);
+
+void iwl_mvm_rs_fw_rate_init(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ struct ieee80211_bss_conf *link_conf,
+ struct ieee80211_link_sta *link_sta,
+ enum nl80211_band band);
int rs_fw_tx_protection(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvmsta,
bool enable);
void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
index e08dca8..b38b242 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
@@ -383,9 +383,10 @@ void iwl_mvm_rx_rx_mpdu(struct iwl_mvm *mvm, struct napi_struct *napi,
* Don't even try to decrypt a MCAST frame that was received
* before the managed vif is authorized, we'd fail anyway.
*/
- if (vif->type == NL80211_IFTYPE_STATION &&
+ if (is_multicast_ether_addr(hdr->addr1) &&
+ vif->type == NL80211_IFTYPE_STATION &&
!mvmvif->authorized &&
- is_multicast_ether_addr(hdr->addr1)) {
+ ieee80211_has_protected(hdr->frame_control)) {
IWL_DEBUG_DROP(mvm, "MCAST before the vif is authorized\n");
kfree_skb(skb);
rcu_read_unlock();
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
index 5d803e5..e1d02c2 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
@@ -106,28 +106,12 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
/*
* For non monitor interface strip the bytes the RADA might not have
- * removed. As monitor interface cannot exist with other interfaces
- * this removal is safe.
+ * removed (it might be disabled, e.g. for mgmt frames). As a monitor
+ * interface cannot exist with other interfaces, this removal is safe
+ * and sufficient, in monitor mode there's no decryption being done.
*/
- if (mic_crc_len && !ieee80211_hw_check(mvm->hw, RX_INCLUDES_FCS)) {
- u32 pkt_flags = le32_to_cpu(pkt->len_n_flags);
-
- /*
- * If RADA was not enabled then decryption was not performed so
- * the MIC cannot be removed.
- */
- if (!(pkt_flags & FH_RSCSR_RADA_EN)) {
- if (WARN_ON(crypt_len > mic_crc_len))
- return -EINVAL;
-
- mic_crc_len -= crypt_len;
- }
-
- if (WARN_ON(mic_crc_len > len))
- return -EINVAL;
-
+ if (len > mic_crc_len && !ieee80211_hw_check(mvm->hw, RX_INCLUDES_FCS))
len -= mic_crc_len;
- }
/* If frame is small enough to fit in skb->head, pull it completely.
* If not, only pull ieee80211_hdr (including crypto if present, and
@@ -172,8 +156,7 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
* Starting from Bz hardware, it calculates starting directly after
* the MAC header, so that matches mac80211's expectation.
*/
- if (skb->ip_summed == CHECKSUM_COMPLETE &&
- mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ) {
+ if (skb->ip_summed == CHECKSUM_COMPLETE) {
struct {
u8 hdr[6];
__be16 type;
@@ -188,7 +171,7 @@ static int iwl_mvm_create_skb(struct iwl_mvm *mvm, struct sk_buff *skb,
shdr->type != htons(ETH_P_PAE) &&
shdr->type != htons(ETH_P_TDLS))))
skb->ip_summed = CHECKSUM_NONE;
- else
+ else if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ)
/* mac80211 assumes full CSUM including SNAP header */
skb_postpush_rcsum(skb, shdr, sizeof(*shdr));
}
@@ -412,9 +395,7 @@ static int iwl_mvm_rx_crypto(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
if (!(status & IWL_RX_MPDU_STATUS_MIC_OK))
return -1;
- stats->flag |= RX_FLAG_DECRYPTED;
- if (pkt_flags & FH_RSCSR_RADA_EN)
- stats->flag |= RX_FLAG_MIC_STRIPPED;
+ stats->flag |= RX_FLAG_DECRYPTED | RX_FLAG_MIC_STRIPPED;
*crypt_len = IEEE80211_CCMP_HDR_LEN;
return 0;
case IWL_RX_MPDU_STATUS_SEC_TKIP:
@@ -706,7 +687,7 @@ void iwl_mvm_reorder_timer_expired(struct timer_list *t)
if (expired) {
struct ieee80211_sta *sta;
struct iwl_mvm_sta *mvmsta;
- u8 sta_id = baid_data->sta_id;
+ u8 sta_id = ffs(baid_data->sta_mask) - 1;
rcu_read_lock();
sta = rcu_dereference(buf->mvm->fw_id_to_mac_id[sta_id]);
@@ -741,6 +722,7 @@ static void iwl_mvm_del_ba(struct iwl_mvm *mvm, int queue,
struct ieee80211_sta *sta;
struct iwl_mvm_reorder_buffer *reorder_buf;
u8 baid = data->baid;
+ u32 sta_id;
if (WARN_ONCE(baid >= IWL_MAX_BAID, "invalid BAID: %x\n", baid))
return;
@@ -751,7 +733,9 @@ static void iwl_mvm_del_ba(struct iwl_mvm *mvm, int queue,
if (WARN_ON_ONCE(!ba_data))
goto out;
- sta = rcu_dereference(mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ /* pick any STA ID to find the pointer */
+ sta_id = ffs(ba_data->sta_mask) - 1;
+ sta = rcu_dereference(mvm->fw_id_to_mac_id[sta_id]);
if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta)))
goto out;
@@ -778,6 +762,7 @@ static void iwl_mvm_release_frames_from_notif(struct iwl_mvm *mvm,
struct ieee80211_sta *sta;
struct iwl_mvm_reorder_buffer *reorder_buf;
struct iwl_mvm_baid_data *ba_data;
+ u32 sta_id;
IWL_DEBUG_HT(mvm, "Frame release notification for BAID %u, NSSN %d\n",
baid, nssn);
@@ -795,7 +780,9 @@ static void iwl_mvm_release_frames_from_notif(struct iwl_mvm *mvm,
goto out;
}
- sta = rcu_dereference(mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ /* pick any STA ID to find the pointer */
+ sta_id = ffs(ba_data->sta_mask) - 1;
+ sta = rcu_dereference(mvm->fw_id_to_mac_id[sta_id]);
if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta)))
goto out;
@@ -936,7 +923,6 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
{
struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb);
struct ieee80211_hdr *hdr = (void *)skb_mac_header(skb);
- struct iwl_mvm_sta *mvm_sta;
struct iwl_mvm_baid_data *baid_data;
struct iwl_mvm_reorder_buffer *buffer;
struct sk_buff *tail;
@@ -948,6 +934,7 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
u8 sub_frame_idx = desc->amsdu_info &
IWL_RX_MPDU_AMSDU_SUBFRAME_IDX_MASK;
struct iwl_mvm_reorder_buf_entry *entries;
+ u32 sta_mask;
int index;
u16 nssn, sn;
u8 baid;
@@ -970,8 +957,6 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
"Got valid BAID without a valid station assigned\n"))
return false;
- mvm_sta = iwl_mvm_sta_from_mac80211(sta);
-
/* not a data packet or a bar */
if (!ieee80211_is_back_req(hdr->frame_control) &&
(!ieee80211_is_data_qos(hdr->frame_control) ||
@@ -989,11 +974,14 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
return false;
}
+ rcu_read_lock();
+ sta_mask = iwl_mvm_sta_fw_id_mask(mvm, sta, -1);
+ rcu_read_unlock();
+
if (WARN(tid != baid_data->tid ||
- mvm_sta->deflink.sta_id != baid_data->sta_id,
- "baid 0x%x is mapped to sta:%d tid:%d, but was received for sta:%d tid:%d\n",
- baid, baid_data->sta_id, baid_data->tid, mvm_sta->deflink.sta_id,
- tid))
+ !(sta_mask & baid_data->sta_mask),
+ "baid 0x%x is mapped to sta_mask:0x%x tid:%d, but was received for sta_mask:0x%x tid:%d\n",
+ baid, baid_data->sta_mask, baid_data->tid, sta_mask, tid))
return false;
nssn = reorder & IWL_RX_MPDU_REORDER_NSSN_MASK;
@@ -2600,10 +2588,10 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
}
if (!iwl_mvm_reorder(mvm, napi, queue, sta, skb, desc) &&
- likely(!iwl_mvm_time_sync_frame(mvm, skb, hdr->addr2)))
+ likely(!iwl_mvm_time_sync_frame(mvm, skb, hdr->addr2)) &&
+ likely(!iwl_mvm_mei_filter_scan(mvm, skb)))
iwl_mvm_pass_packet_to_mac80211(mvm, napi, skb, queue, sta,
link_sta);
-
out:
rcu_read_unlock();
}
@@ -2777,9 +2765,10 @@ void iwl_mvm_rx_bar_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
goto out;
}
- if (WARN(tid != baid_data->tid || sta_id != baid_data->sta_id,
- "baid 0x%x is mapped to sta:%d tid:%d, but BAR release received for sta:%d tid:%d\n",
- baid, baid_data->sta_id, baid_data->tid, sta_id,
+ if (WARN(tid != baid_data->tid || sta_id > IWL_MVM_STATION_COUNT_MAX ||
+ !(baid_data->sta_mask & BIT(sta_id)),
+ "baid 0x%x is mapped to sta_mask:0x%x tid:%d, but BAR release received for sta:%d tid:%d\n",
+ baid, baid_data->sta_mask, baid_data->tid, sta_id,
tid))
goto out;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
index 0704509..1756157 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
@@ -45,6 +45,9 @@
/* minimal number of 2GHz and 5GHz channels in the regular scan request */
#define IWL_MVM_6GHZ_PASSIVE_SCAN_MIN_CHANS 4
+/* Number of iterations on the channel for mei filtered scan */
+#define IWL_MEI_SCAN_NUM_ITER 5U
+
struct iwl_mvm_scan_timing_params {
u32 suspend_time;
u32 max_out_time;
@@ -98,6 +101,7 @@ struct iwl_mvm_scan_params {
bool scan_6ghz;
bool enable_6ghz_passive;
bool respect_p2p_go, respect_p2p_go_hb;
+ u8 bssid[ETH_ALEN] __aligned(2);
};
static inline void *iwl_mvm_get_scan_req_umac_data(struct iwl_mvm *mvm)
@@ -240,8 +244,9 @@ iwl_mvm_scan_type _iwl_mvm_get_scan_type(struct iwl_mvm *mvm,
* set all scan requests as fast-balance scan
*/
if (vif && vif->type == NL80211_IFTYPE_STATION &&
- vif->bss_conf.dtim_period < 220 &&
- data.is_dcm_with_p2p_go)
+ data.is_dcm_with_p2p_go &&
+ ((vif->bss_conf.beacon_int *
+ vif->bss_conf.dtim_period) < 220))
return IWL_SCAN_TYPE_FAST_BALANCE;
}
@@ -759,7 +764,7 @@ iwl_mvm_build_scan_probe(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
frame->frame_control = cpu_to_le16(IEEE80211_STYPE_PROBE_REQ);
eth_broadcast_addr(frame->da);
- eth_broadcast_addr(frame->bssid);
+ ether_addr_copy(frame->bssid, params->bssid);
frame->seq_ctrl = 0;
pos = frame->u.probe_req.variable;
@@ -2081,6 +2086,11 @@ static u8 iwl_mvm_scan_umac_flags2(struct iwl_mvm *mvm,
IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_RESPECT_P2P_GO_HB;
}
+ if (params->scan_6ghz &&
+ fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_SCAN_DONT_TOGGLE_ANT))
+ flags |= IWL_UMAC_SCAN_GEN_PARAMS_FLAGS2_DONT_TOGGLE_ANT;
+
return flags;
}
@@ -2292,7 +2302,7 @@ iwl_mvm_scan_umac_fill_general_p_v11(struct iwl_mvm *mvm,
iwl_mvm_scan_umac_dwell_v11(mvm, gp, params);
- IWL_DEBUG_SCAN(mvm, "Gerenal: flags=0x%x, flags2=0x%x\n",
+ IWL_DEBUG_SCAN(mvm, "General: flags=0x%x, flags2=0x%x\n",
gen_flags, gen_flags2);
gp->flags = cpu_to_le16(gen_flags);
@@ -2617,6 +2627,89 @@ static const struct iwl_scan_umac_handler iwl_scan_umac_handlers[] = {
IWL_SCAN_UMAC_HANDLER(12),
};
+static void iwl_mvm_mei_scan_work(struct work_struct *wk)
+{
+ struct iwl_mei_scan_filter *scan_filter =
+ container_of(wk, struct iwl_mei_scan_filter, scan_work);
+ struct iwl_mvm *mvm =
+ container_of(scan_filter, struct iwl_mvm, mei_scan_filter);
+ struct iwl_mvm_csme_conn_info *info;
+ struct sk_buff *skb;
+ u8 bssid[ETH_ALEN];
+
+ mutex_lock(&mvm->mutex);
+ info = iwl_mvm_get_csme_conn_info(mvm);
+ memcpy(bssid, info->conn_info.bssid, ETH_ALEN);
+ mutex_unlock(&mvm->mutex);
+
+ while ((skb = skb_dequeue(&scan_filter->scan_res))) {
+ struct ieee80211_mgmt *mgmt = (void *)skb->data;
+
+ if (!memcmp(mgmt->bssid, bssid, ETH_ALEN))
+ ieee80211_rx_irqsafe(mvm->hw, skb);
+ else
+ kfree_skb(skb);
+ }
+}
+
+void iwl_mvm_mei_scan_filter_init(struct iwl_mei_scan_filter *mei_scan_filter)
+{
+ skb_queue_head_init(&mei_scan_filter->scan_res);
+ INIT_WORK(&mei_scan_filter->scan_work, iwl_mvm_mei_scan_work);
+}
+
+/* In case CSME is connected and has link protection set, this function will
+ * override the scan request to scan only the associated channel and only for
+ * the associated SSID.
+ */
+static void iwl_mvm_mei_limited_scan(struct iwl_mvm *mvm,
+ struct iwl_mvm_scan_params *params)
+{
+ struct iwl_mvm_csme_conn_info *info = iwl_mvm_get_csme_conn_info(mvm);
+ struct iwl_mei_conn_info *conn_info;
+ struct ieee80211_channel *chan;
+ int scan_iters, i;
+
+ if (!info) {
+ IWL_DEBUG_SCAN(mvm, "mei_limited_scan: no connection info\n");
+ return;
+ }
+
+ conn_info = &info->conn_info;
+ if (!info->conn_info.lp_state || !info->conn_info.ssid_len)
+ return;
+
+ if (!params->n_channels || !params->n_ssids)
+ return;
+
+ mvm->mei_scan_filter.is_mei_limited_scan = true;
+
+ chan = ieee80211_get_channel(mvm->hw->wiphy,
+ ieee80211_channel_to_frequency(conn_info->channel,
+ conn_info->band));
+ if (!chan) {
+ IWL_DEBUG_SCAN(mvm,
+ "Failed to get CSME channel (chan=%u band=%u)\n",
+ conn_info->channel, conn_info->band);
+ return;
+ }
+
+ /* The mei filtered scan must find the AP, otherwise CSME will
+ * take the NIC ownership. Add several iterations on the channel to
+ * make the scan more robust.
+ */
+ scan_iters = min(IWL_MEI_SCAN_NUM_ITER, params->n_channels);
+ params->n_channels = scan_iters;
+ for (i = 0; i < scan_iters; i++)
+ params->channels[i] = chan;
+
+ IWL_DEBUG_SCAN(mvm, "Mei scan: num iterations=%u\n", scan_iters);
+
+ params->n_ssids = 1;
+ params->ssids[0].ssid_len = conn_info->ssid_len;
+ memcpy(params->ssids[0].ssid, conn_info->ssid, conn_info->ssid_len);
+}
+
static int iwl_mvm_build_scan_cmd(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct iwl_host_cmd *hcmd,
@@ -2629,6 +2722,8 @@ static int iwl_mvm_build_scan_cmd(struct iwl_mvm *mvm,
lockdep_assert_held(&mvm->mutex);
memset(mvm->scan_cmd, 0, mvm->scan_cmd_size);
+ iwl_mvm_mei_limited_scan(mvm, params);
+
if (!fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_UMAC_SCAN)) {
hcmd->id = SCAN_OFFLOAD_REQUEST_CMD;
@@ -2795,6 +2890,7 @@ int iwl_mvm_reg_scan_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
params.pass_all = true;
params.n_match_sets = 0;
params.match_sets = NULL;
+ ether_addr_copy(params.bssid, req->bssid);
params.scan_plans = &scan_plan;
params.n_scan_plans = 1;
@@ -2888,6 +2984,7 @@ int iwl_mvm_sched_scan_start(struct iwl_mvm *mvm,
params.pass_all = iwl_mvm_scan_pass_all(mvm, req);
params.n_match_sets = req->n_match_sets;
params.match_sets = req->match_sets;
+ eth_broadcast_addr(params.bssid);
if (!req->n_scan_plans)
return -EINVAL;
@@ -2983,6 +3080,8 @@ void iwl_mvm_rx_umac_scan_complete_notif(struct iwl_mvm *mvm,
u32 uid = __le32_to_cpu(notif->uid);
bool aborted = (notif->status == IWL_SCAN_OFFLOAD_ABORTED);
+ mvm->mei_scan_filter.is_mei_limited_scan = false;
+
if (WARN_ON(!(mvm->scan_uid_status[uid] & mvm->scan_status)))
return;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sf.c b/drivers/net/wireless/intel/iwlwifi/mvm/sf.c
index 7c5f41e..98f330f 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sf.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sf.c
@@ -8,7 +8,7 @@
/* For counting bound interfaces */
struct iwl_mvm_active_iface_iterator_data {
struct ieee80211_vif *ignore_vif;
- u8 sta_vif_ap_sta_id;
+ struct ieee80211_sta *sta_vif_ap_sta;
enum iwl_sf_state sta_vif_state;
u32 num_active_macs;
};
@@ -30,7 +30,7 @@ static void iwl_mvm_bound_iface_iterator(void *_data, u8 *mac,
data->num_active_macs++;
if (vif->type == NL80211_IFTYPE_STATION) {
- data->sta_vif_ap_sta_id = mvmvif->deflink.ap_sta_id;
+ data->sta_vif_ap_sta = mvmvif->ap_sta;
if (vif->cfg.assoc)
data->sta_vif_state = SF_FULL_ON;
else
@@ -172,13 +172,12 @@ static void iwl_mvm_fill_sf_command(struct iwl_mvm *mvm,
}
}
-static int iwl_mvm_sf_config(struct iwl_mvm *mvm, u8 sta_id,
+static int iwl_mvm_sf_config(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
enum iwl_sf_state new_state)
{
struct iwl_sf_cfg_cmd sf_cmd = {
.state = cpu_to_le32(new_state),
};
- struct ieee80211_sta *sta;
int ret = 0;
if (mvm->cfg->disable_dummy_notification)
@@ -196,20 +195,12 @@ static int iwl_mvm_sf_config(struct iwl_mvm *mvm, u8 sta_id,
iwl_mvm_fill_sf_command(mvm, &sf_cmd, NULL);
break;
case SF_FULL_ON:
- if (sta_id == IWL_MVM_INVALID_STA) {
+ if (!sta) {
IWL_ERR(mvm,
"No station: Cannot switch SF to FULL_ON\n");
return -EINVAL;
}
- rcu_read_lock();
- sta = rcu_dereference(mvm->fw_id_to_mac_id[sta_id]);
- if (IS_ERR_OR_NULL(sta)) {
- IWL_ERR(mvm, "Invalid station id\n");
- rcu_read_unlock();
- return -EINVAL;
- }
iwl_mvm_fill_sf_command(mvm, &sf_cmd, sta);
- rcu_read_unlock();
break;
case SF_INIT_OFF:
iwl_mvm_fill_sf_command(mvm, &sf_cmd, NULL);
@@ -237,13 +228,12 @@ int iwl_mvm_sf_update(struct iwl_mvm *mvm, struct ieee80211_vif *changed_vif,
bool remove_vif)
{
enum iwl_sf_state new_state;
- u8 sta_id = IWL_MVM_INVALID_STA;
struct iwl_mvm_vif *mvmvif = NULL;
struct iwl_mvm_active_iface_iterator_data data = {
.ignore_vif = changed_vif,
.sta_vif_state = SF_UNINIT,
- .sta_vif_ap_sta_id = IWL_MVM_INVALID_STA,
};
+ struct ieee80211_sta *sta = NULL;
/*
* Ignore the call if we are in HW Restart flow, or if the handled
@@ -273,7 +263,7 @@ int iwl_mvm_sf_update(struct iwl_mvm *mvm, struct ieee80211_vif *changed_vif,
* and we filled the relevant data during iteration
*/
new_state = data.sta_vif_state;
- sta_id = data.sta_vif_ap_sta_id;
+ sta = data.sta_vif_ap_sta;
} else {
if (WARN_ON(!changed_vif))
return -EINVAL;
@@ -282,7 +272,7 @@ int iwl_mvm_sf_update(struct iwl_mvm *mvm, struct ieee80211_vif *changed_vif,
} else if (changed_vif->cfg.assoc &&
changed_vif->bss_conf.dtim_period) {
mvmvif = iwl_mvm_vif_from_mac80211(changed_vif);
- sta_id = mvmvif->deflink.ap_sta_id;
+ sta = mvmvif->ap_sta;
new_state = SF_FULL_ON;
} else {
new_state = SF_INIT_OFF;
@@ -294,8 +284,5 @@ int iwl_mvm_sf_update(struct iwl_mvm *mvm, struct ieee80211_vif *changed_vif,
new_state = SF_UNINIT;
}
- /* For MLO it's ok to use deflink->sta_id as it's needed only to get
- * a pointer to mac80211 sta
- */
- return iwl_mvm_sf_config(mvm, sta_id, new_state);
+ return iwl_mvm_sf_config(mvm, sta, new_state);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
index 50e2248..5469d63 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
@@ -251,6 +251,7 @@ static void iwl_mvm_rx_agg_session_expired(struct timer_list *t)
struct ieee80211_sta *sta;
struct iwl_mvm_sta *mvm_sta;
unsigned long timeout;
+ unsigned int sta_id;
rcu_read_lock();
@@ -269,7 +270,8 @@ static void iwl_mvm_rx_agg_session_expired(struct timer_list *t)
}
/* Timer expired */
- sta = rcu_dereference(ba_data->mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ sta_id = ffs(ba_data->sta_mask) - 1; /* don't care which one */
+ sta = rcu_dereference(ba_data->mvm->fw_id_to_mac_id[sta_id]);
/*
* sta should be valid unless the following happens:
@@ -356,10 +358,14 @@ static int iwl_mvm_disable_txq(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
SCD_QUEUE_CONFIG_CMD);
struct iwl_scd_queue_cfg_cmd remove_cmd = {
.operation = cpu_to_le32(IWL_SCD_QUEUE_REMOVE),
- .u.remove.tid = cpu_to_le32(tid),
.u.remove.sta_mask = cpu_to_le32(BIT(sta_id)),
};
+ if (tid == IWL_MAX_TID_COUNT)
+ tid = IWL_MGMT_TID;
+
+ remove_cmd.u.remove.tid = cpu_to_le32(tid);
+
ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, 0,
sizeof(remove_cmd),
&remove_cmd);
@@ -2146,7 +2152,7 @@ int iwl_mvm_allocate_int_sta(struct iwl_mvm *mvm,
sta->type = type;
/* put a non-NULL value so iterating over the stations won't stop */
- rcu_assign_pointer(mvm->fw_id_to_mac_id[sta->sta_id], ERR_PTR(-EINVAL));
+ RCU_INIT_POINTER(mvm->fw_id_to_mac_id[sta->sta_id], ERR_PTR(-EINVAL));
return 0;
}
@@ -2227,11 +2233,13 @@ static int iwl_mvm_add_int_sta_with_queue(struct iwl_mvm *mvm, int macidx,
int iwl_mvm_add_aux_sta(struct iwl_mvm *mvm, u32 lmac_id)
{
int ret;
+ u32 qmask = mvm->aux_queue == IWL_MVM_INVALID_QUEUE ? 0 :
+ BIT(mvm->aux_queue);
lockdep_assert_held(&mvm->mutex);
/* Allocate aux station and assign to it the aux queue */
- ret = iwl_mvm_allocate_int_sta(mvm, &mvm->aux_sta, BIT(mvm->aux_queue),
+ ret = iwl_mvm_allocate_int_sta(mvm, &mvm->aux_sta, qmask,
NL80211_IFTYPE_UNSPECIFIED,
IWL_STA_AUX_ACTIVITY);
if (ret)
@@ -2750,10 +2758,11 @@ static void iwl_mvm_init_reorder_buffer(struct iwl_mvm *mvm,
}
static int iwl_mvm_fw_baid_op_sta(struct iwl_mvm *mvm,
- struct iwl_mvm_sta *mvm_sta,
+ struct ieee80211_sta *sta,
bool start, int tid, u16 ssn,
u16 buf_size)
{
+ struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
struct iwl_mvm_add_sta_cmd cmd = {
.mac_id_n_color = cpu_to_le32(mvm_sta->mac_id_n_color),
.sta_id = mvm_sta->deflink.sta_id,
@@ -2798,7 +2807,7 @@ static int iwl_mvm_fw_baid_op_sta(struct iwl_mvm *mvm,
}
static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm,
- struct iwl_mvm_sta *mvm_sta,
+ struct ieee80211_sta *sta,
bool start, int tid, u16 ssn,
u16 buf_size, int baid)
{
@@ -2813,7 +2822,7 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm,
if (start) {
cmd.alloc.sta_id_mask =
- cpu_to_le32(BIT(mvm_sta->deflink.sta_id));
+ cpu_to_le32(iwl_mvm_sta_fw_id_mask(mvm, sta, -1));
cmd.alloc.tid = tid;
cmd.alloc.ssn = cpu_to_le16(ssn);
cmd.alloc.win_size = cpu_to_le16(buf_size);
@@ -2823,7 +2832,7 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm,
BUILD_BUG_ON(sizeof(cmd.remove_v1) > sizeof(cmd.remove));
} else {
cmd.remove.sta_id_mask =
- cpu_to_le32(BIT(mvm_sta->deflink.sta_id));
+ cpu_to_le32(iwl_mvm_sta_fw_id_mask(mvm, sta, -1));
cmd.remove.tid = cpu_to_le32(tid);
}
@@ -2846,16 +2855,16 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm,
return baid;
}
-static int iwl_mvm_fw_baid_op(struct iwl_mvm *mvm, struct iwl_mvm_sta *mvm_sta,
+static int iwl_mvm_fw_baid_op(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
bool start, int tid, u16 ssn, u16 buf_size,
int baid)
{
if (fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_BAID_ML_SUPPORT))
- return iwl_mvm_fw_baid_op_cmd(mvm, mvm_sta, start,
+ return iwl_mvm_fw_baid_op_cmd(mvm, sta, start,
tid, ssn, buf_size, baid);
- return iwl_mvm_fw_baid_op_sta(mvm, mvm_sta, start,
+ return iwl_mvm_fw_baid_op_sta(mvm, sta, start,
tid, ssn, buf_size);
}
@@ -2925,7 +2934,7 @@ int iwl_mvm_sta_rx_agg(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
/* Don't send command to remove (start=0) BAID during restart */
if (start || !test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))
- baid = iwl_mvm_fw_baid_op(mvm, mvm_sta, start, tid, ssn, buf_size,
+ baid = iwl_mvm_fw_baid_op(mvm, sta, start, tid, ssn, buf_size,
baid);
if (baid < 0) {
@@ -2947,7 +2956,7 @@ int iwl_mvm_sta_rx_agg(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
iwl_mvm_rx_agg_session_expired, 0);
baid_data->mvm = mvm;
baid_data->tid = tid;
- baid_data->sta_id = mvm_sta->deflink.sta_id;
+ baid_data->sta_mask = iwl_mvm_sta_fw_id_mask(mvm, sta, -1);
mvm_sta->tid_to_baid[tid] = baid;
if (timeout)
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
index 7b9e919..a61d4f8 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
@@ -356,6 +356,7 @@ struct iwl_mvm_link_sta {
* @tid_disable_agg: bitmap: if bit(tid) is set, the fw won't send ampdus for
* tid.
* @sta_type: station type
+ * @authorized: indicates station is authorized
* @sta_state: station state according to enum %ieee80211_sta_state
* @bt_reduced_txpower: is reduced tx power enabled for this station
* @next_status_eosp: the next reclaimed packet is a PS-Poll response and
@@ -409,6 +410,7 @@ struct iwl_mvm_sta {
enum ieee80211_sta_state sta_state;
bool bt_reduced_txpower;
bool next_status_eosp;
+ bool authorized;
spinlock_t lock;
struct iwl_mvm_tid_data tid_data[IWL_MAX_TID_COUNT + 1];
u8 tid_to_baid[IWL_MAX_TID_COUNT];
@@ -642,6 +644,8 @@ int iwl_mvm_mld_update_sta_links(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
u16 old_links, u16 new_links);
+u32 iwl_mvm_sta_fw_id_mask(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
+ int filter_link_id);
/* Queues */
void iwl_mvm_mld_modify_all_sta_disable_tx(struct iwl_mvm *mvm,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
index 2c84293..10d7178 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
@@ -184,10 +184,7 @@ static u32 iwl_mvm_tx_csum(struct iwl_mvm *mvm, struct sk_buff *skb,
struct ieee80211_tx_info *info,
bool amsdu)
{
- if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_BZ ||
- (mvm->trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_BZ &&
- CSR_HW_REV_TYPE(mvm->trans->hw_rev) == IWL_CFG_MAC_TYPE_GL &&
- mvm->trans->hw_rev_step == SILICON_A_STEP))
+ if (!iwl_mvm_has_new_tx_csum(mvm))
return iwl_mvm_tx_csum_pre_bz(mvm, skb, info, amsdu);
return iwl_mvm_tx_csum_bz(mvm, skb, amsdu);
}
@@ -332,22 +329,23 @@ static u32 iwl_mvm_get_tx_rate(struct iwl_mvm *mvm,
sta ? iwl_mvm_sta_from_mac80211(sta)->sta_state : -1);
rate_idx = info->control.rates[0].idx;
+
+ /* For non 2 GHZ band, remap mac80211 rate indices into driver
+ * indices.
+ */
+ if (info->band != NL80211_BAND_2GHZ ||
+ (info->flags & IEEE80211_TX_CTL_NO_CCK_RATE))
+ rate_idx += IWL_FIRST_OFDM_RATE;
+
+ /* For 2.4 GHZ band, check that there is no need to remap */
+ BUILD_BUG_ON(IWL_FIRST_CCK_RATE != 0);
}
/* if the rate isn't a well known legacy rate, take the lowest one */
if (rate_idx < 0 || rate_idx >= IWL_RATE_COUNT_LEGACY)
- rate_idx = rate_lowest_index(
- &mvm->nvm_data->bands[info->band], sta);
-
- /*
- * For non 2 GHZ band, remap mac80211 rate
- * indices into driver indices
- */
- if (info->band != NL80211_BAND_2GHZ)
- rate_idx += IWL_FIRST_OFDM_RATE;
-
- /* For 2.4 GHZ band, check that there is no need to remap */
- BUILD_BUG_ON(IWL_FIRST_CCK_RATE != 0);
+ rate_idx = iwl_mvm_mac_ctxt_get_lowest_rate(mvm,
+ info,
+ info->control.vif);
/* Get PLCP rate for tx_cmd->rate_n_flags */
rate_plcp = iwl_mvm_mac80211_idx_to_hwrate(mvm->fw, rate_idx);
@@ -606,8 +604,9 @@ static void iwl_mvm_skb_prepare_status(struct sk_buff *skb,
static int iwl_mvm_get_ctrl_vif_queue(struct iwl_mvm *mvm,
struct iwl_mvm_vif_link_info *link,
struct ieee80211_tx_info *info,
- struct ieee80211_hdr *hdr)
+ struct sk_buff *skb)
{
+ struct ieee80211_hdr *hdr = (void *)skb->data;
__le16 fc = hdr->frame_control;
switch (info->control.vif->type) {
@@ -624,7 +623,7 @@ static int iwl_mvm_get_ctrl_vif_queue(struct iwl_mvm *mvm,
* reason 7 ("Class 3 frame received from nonassociated STA").
*/
if (ieee80211_is_mgmt(fc) &&
- (!ieee80211_is_bufferable_mmpdu(fc) ||
+ (!ieee80211_is_bufferable_mmpdu(skb) ||
ieee80211_is_deauth(fc) || ieee80211_is_disassoc(fc)))
return link->mgmt_queue;
@@ -757,7 +756,7 @@ int iwl_mvm_tx_skb_non_sta(struct iwl_mvm *mvm, struct sk_buff *skb)
sta_id = link->mcast_sta.sta_id;
queue = iwl_mvm_get_ctrl_vif_queue(mvm, link, &info,
- hdr);
+ skb);
} else if (info.control.vif->type == NL80211_IFTYPE_MONITOR) {
queue = mvm->snif_queue;
sta_id = mvm->snif_sta.sta_id;
@@ -805,10 +804,11 @@ unsigned int iwl_mvm_max_amsdu_size(struct iwl_mvm *mvm,
struct ieee80211_sta *sta, unsigned int tid)
{
struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
- enum nl80211_band band = mvmsta->vif->bss_conf.chandef.chan->band;
u8 ac = tid_to_mac80211_ac[tid];
+ enum nl80211_band band;
unsigned int txf;
- int lmac = iwl_mvm_get_lmac_id(mvm->fw, band);
+ unsigned int val;
+ int lmac;
/* For HE redirect to trigger based fifos */
if (sta->deflink.he_cap.has_he && !WARN_ON(!iwl_mvm_has_new_tx_api(mvm)))
@@ -822,7 +822,37 @@ unsigned int iwl_mvm_max_amsdu_size(struct iwl_mvm *mvm,
* We also want to have the start of the next packet inside the
* fifo to be able to send bursts.
*/
- return min_t(unsigned int, mvmsta->max_amsdu_len,
+ val = mvmsta->max_amsdu_len;
+
+ if (hweight16(sta->valid_links) <= 1) {
+ if (sta->valid_links) {
+ struct ieee80211_bss_conf *link_conf;
+ unsigned int link = ffs(sta->valid_links) - 1;
+
+ rcu_read_lock();
+ link_conf = rcu_dereference(mvmsta->vif->link_conf[link]);
+ if (WARN_ON(!link_conf))
+ band = NL80211_BAND_2GHZ;
+ else
+ band = link_conf->chandef.chan->band;
+ rcu_read_unlock();
+ } else {
+ band = mvmsta->vif->bss_conf.chandef.chan->band;
+ }
+
+ lmac = iwl_mvm_get_lmac_id(mvm->fw, band);
+ } else if (fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_CDB_SUPPORT)) {
+ /* for real MLO restrict to both LMACs if they exist */
+ lmac = IWL_LMAC_5G_INDEX;
+ val = min_t(unsigned int, val,
+ mvm->fwrt.smem_cfg.lmac[lmac].txfifo_size[txf] - 256);
+ lmac = IWL_LMAC_24G_INDEX;
+ } else {
+ lmac = IWL_LMAC_24G_INDEX;
+ }
+
+ return min_t(unsigned int, val,
mvm->fwrt.smem_cfg.lmac[lmac].txfifo_size[txf] - 256);
}
@@ -1256,8 +1286,7 @@ int iwl_mvm_tx_skb_sta(struct iwl_mvm *mvm, struct sk_buff *skb,
if (ret)
return ret;
- if (WARN_ON(skb_queue_empty(&mpdus_skbs)))
- return ret;
+ WARN_ON(skb_queue_empty(&mpdus_skbs));
while (!skb_queue_empty(&mpdus_skbs)) {
skb = __skb_dequeue(&mpdus_skbs);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
index 6c935d7..dba1123 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
@@ -504,6 +504,7 @@ static const struct pci_device_id iwl_hw_card_ids[] = {
/* Bz devices */
{IWL_PCI_DEVICE(0x2727, PCI_ANY_ID, iwl_bz_trans_cfg)},
+ {IWL_PCI_DEVICE(0x272b, PCI_ANY_ID, iwl_bz_trans_cfg)},
{IWL_PCI_DEVICE(0xA840, PCI_ANY_ID, iwl_bz_trans_cfg)},
{IWL_PCI_DEVICE(0x7740, PCI_ANY_ID, iwl_bz_trans_cfg)},
#endif /* CONFIG_IWLMVM */
@@ -513,16 +514,17 @@ static const struct pci_device_id iwl_hw_card_ids[] = {
MODULE_DEVICE_TABLE(pci, iwl_hw_card_ids);
#define _IWL_DEV_INFO(_device, _subdevice, _mac_type, _mac_step, _rf_type, \
- _rf_id, _no_160, _cores, _cdb, _jacket, _cfg, _name) \
- { .device = (_device), .subdevice = (_subdevice), .cfg = &(_cfg), \
- .name = _name, .mac_type = _mac_type, .rf_type = _rf_type, \
- .no_160 = _no_160, .cores = _cores, .rf_id = _rf_id, \
+ _rf_id, _rf_step, _no_160, _cores, _cdb, _jacket, _cfg, \
+ _name) \
+ { .device = (_device), .subdevice = (_subdevice), .cfg = &(_cfg), \
+ .name = _name, .mac_type = _mac_type, .rf_type = _rf_type, .rf_step = _rf_step, \
+ .no_160 = _no_160, .cores = _cores, .rf_id = _rf_id, \
.mac_step = _mac_step, .cdb = _cdb, .jacket = _jacket }
#define IWL_DEV_INFO(_device, _subdevice, _cfg, _name) \
- _IWL_DEV_INFO(_device, _subdevice, IWL_CFG_ANY, IWL_CFG_ANY, \
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, \
- IWL_CFG_ANY, IWL_CFG_ANY, _cfg, _name)
+ _IWL_DEV_INFO(_device, _subdevice, IWL_CFG_ANY, IWL_CFG_ANY, \
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, \
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, _cfg, _name)
static const struct iwl_dev_info iwl_dev_info_table[] = {
#if IS_ENABLED(CONFIG_IWLMVM)
@@ -565,7 +567,6 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
IWL_DEV_INFO(0x43F0, 0x1652, killer1650i_2ax_cfg_qu_b0_hr_b0, iwl_ax201_killer_1650i_name),
IWL_DEV_INFO(0x43F0, 0x2074, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0x43F0, 0x4070, iwl_ax201_cfg_qu_hr, NULL),
- IWL_DEV_INFO(0x43F0, 0x1651, killer1650s_2ax_cfg_qu_b0_hr_b0, iwl_ax201_killer_1650s_name),
IWL_DEV_INFO(0xA0F0, 0x0070, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0xA0F0, 0x0074, iwl_ax201_cfg_qu_hr, NULL),
IWL_DEV_INFO(0xA0F0, 0x0078, iwl_ax201_cfg_qu_hr, NULL),
@@ -694,87 +695,87 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PU, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_2ac_cfg_soc, iwl9560_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9461_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9461_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9462_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_PNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9462_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9270_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT_GNSS, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9270_name),
_IWL_DEV_INFO(0x271B, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9162_160_name),
_IWL_DEV_INFO(0x271B, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9162_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9260_160_name),
_IWL_DEV_INFO(0x2526, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_TH, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_TH, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9260_2ac_cfg, iwl9260_name),
@@ -782,176 +783,176 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
/* Qu B step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_b0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* Qu C step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qu_c0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* QuZ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_quz_a0_jf_b0_cfg, iwl9560_killer_1550i_name),
/* QnJ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1551,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_killer_1550s_name),
_IWL_DEV_INFO(IWL_CFG_ANY, 0x1552,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl9560_qnj_b0_jf_b0_cfg, iwl9560_killer_1550i_name),
@@ -959,397 +960,408 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
/* Qu B step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_b0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_b0_hr_b0, iwl_ax203_name),
/* Qu C step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_c0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_c0_hr_b0, iwl_ax203_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QU, SILICON_C_STEP,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qu_c0_hr_b0, iwl_ax201_name),
/* QuZ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_quz_a0_hr1_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QUZ, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_quz_a0_hr_b0, iwl_ax203_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_QUZ, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_quz_a0_hr_b0, iwl_ax201_name),
/* QnJ with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_QNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_qnj_b0_hr_b0_cfg, iwl_ax201_name),
/* SnJ with Jf */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9462_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_jf_b0, iwl9560_name),
/* SnJ with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_hr_b0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_hr_b0, iwl_ax201_name),
/* Ma */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_hr_b0, iwl_ax201_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_gf_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_gf4_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_mr_a0, iwl_ax221_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_fm_a0, iwl_ax231_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_mr_a0, iwl_ax221_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_ma_b0_hr_b0, iwl_ax201_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_ma_b0_gf_a0, iwl_ax211_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
+ iwl_cfg_ma_b0_gf4_a0, iwl_ax211_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_ma_b0_mr_a0, iwl_ax221_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_MA, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_ma_b0_fm_a0, iwl_ax231_name),
/* So with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax203_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
/* So-F with Hr */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax203_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_NO_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax101_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_hr_a0, iwl_ax201_name),
/* So-F with Gf */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwlax411_2ax_cfg_so_gf4_a0, iwl_ax411_name),
/* Bz */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_bz_a0_hr_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_hr_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_gf_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_gf4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MR, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_mr_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_BZ, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_A_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_a0_fm_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_A_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_NO_JACKET,
iwl_cfg_bz_a0_fm4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_B_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bz_a0_fm_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_B_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bz_a0_fm4_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_A_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET,
iwl_cfg_gl_a0_fm_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_B_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET,
iwl_cfg_gl_b0_fm_b0, iwl_bz_name),
/* BZ Z step */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_BZ, SILICON_Z_STEP,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bz_z0_gf_a0, iwl_bz_name),
/* BNJ */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_A_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_a0_fm_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_B_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_b0_fm_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_A_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_a0_fm4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, SILICON_B_STEP,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_b0_fm4_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_a0_gf_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_b0_gf_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_a0_gf4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_IS_JACKET,
iwl_cfg_bnj_b0_gf4_a0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_IS_JACKET,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_bnj_a0_hr_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, SILICON_A_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bnj_a0_hr_b0, iwl_bz_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
- IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY,
- IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_NO_JACKET,
+ IWL_CFG_RF_TYPE_HR1, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
+ iwl_cfg_bnj_b0_hr_a0, iwl_bz_name),
+ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_MAC_TYPE_GL, SILICON_B_STEP,
+ IWL_CFG_RF_TYPE_HR2, IWL_CFG_ANY, IWL_CFG_ANY,
+ IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_bnj_b0_hr_b0, iwl_bz_name),
/* SoF with JF2 */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
/* SoF with JF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9462_name),
-
-/* SoF with JF2 */
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
-
-/* SoF with JF */
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
- IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
- IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
- iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
- _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
- IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_name),
/* So with GF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax211_2ax_cfg_so_gf_a0, iwl_ax211_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_GF, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_CDB, IWL_CFG_ANY,
iwlax411_2ax_cfg_so_gf4_a0, iwl_ax411_name),
/* So with JF2 */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF,
+ IWL_CFG_RF_TYPE_JF2, IWL_CFG_RF_ID_JF, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9560_name),
/* So with JF */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_160_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9461_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV,
+ IWL_CFG_RF_TYPE_JF1, IWL_CFG_RF_ID_JF1_DIV, IWL_CFG_ANY,
IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwlax210_2ax_cfg_so_jf_b0, iwl9462_name),
@@ -1357,22 +1369,22 @@ static const struct iwl_dev_info iwl_dev_info_table[] = {
/* For now we use the same FW as MR, but this will change in the future. */
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SO, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_ms_a0, iwl_ax204_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_so_a0_ms_a0, iwl_ax204_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_MA, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_ma_a0_ms_a0, iwl_ax204_name),
_IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_MAC_TYPE_SNJ, IWL_CFG_ANY,
- IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY,
+ IWL_CFG_RF_TYPE_MS, IWL_CFG_ANY, IWL_CFG_ANY,
IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_NO_CDB, IWL_CFG_ANY,
iwl_cfg_snj_a0_ms_a0, iwl_ax204_name)
@@ -1407,8 +1419,16 @@ static int get_crf_id(struct iwl_trans *iwl_trans)
/* Read crf info */
iwl_trans->hw_crf_id = iwl_read_prph_no_grab(iwl_trans, sd_reg_ver_addr);
+ /* Read cnv info */
+ iwl_trans->hw_cnv_id =
+ iwl_read_prph_no_grab(iwl_trans, CNVI_AUX_MISC_CHIP);
+
/* Read cdb info (also contains the jacket info if needed in the future */
- iwl_trans->hw_cdb_id = iwl_read_umac_prph_no_grab(iwl_trans, WFPM_OTP_CFG1_ADDR);
+ iwl_trans->hw_wfpm_id =
+ iwl_read_umac_prph_no_grab(iwl_trans, WFPM_OTP_CFG1_ADDR);
+ IWL_INFO(iwl_trans, "Detected crf-id 0x%x, cnv-id 0x%x wfpm id 0x%x\n",
+ iwl_trans->hw_crf_id, iwl_trans->hw_cnv_id,
+ iwl_trans->hw_wfpm_id);
iwl_trans_release_nic_access(iwl_trans);
@@ -1424,7 +1444,11 @@ static int map_crf_id(struct iwl_trans *iwl_trans)
{
int ret = 0;
u32 val = iwl_trans->hw_crf_id;
- u32 cdb = iwl_trans->hw_cdb_id;
+ u32 step_id = REG_CRF_ID_STEP(val);
+ u32 slave_id = REG_CRF_ID_SLAVE(val);
+ u32 jacket_id_cnv = REG_CRF_ID_SLAVE(iwl_trans->hw_cnv_id);
+ u32 jacket_id_wfpm = WFPM_OTP_CFG1_IS_JACKET(iwl_trans->hw_wfpm_id);
+ u32 cdb_id_wfpm = WFPM_OTP_CFG1_IS_CDB(iwl_trans->hw_wfpm_id);
/* Map between crf id to rf id */
switch (REG_CRF_ID_TYPE(val)) {
@@ -1434,9 +1458,12 @@ static int map_crf_id(struct iwl_trans *iwl_trans)
case REG_CRF_ID_TYPE_JF_2:
iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_JF2 << 12);
break;
- case REG_CRF_ID_TYPE_HR_NONE_CDB:
+ case REG_CRF_ID_TYPE_HR_NONE_CDB_1X1:
iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_HR1 << 12);
break;
+ case REG_CRF_ID_TYPE_HR_NONE_CDB:
+ iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_HR2 << 12);
+ break;
case REG_CRF_ID_TYPE_HR_CDB:
iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_HR2 << 12);
break;
@@ -1446,27 +1473,43 @@ static int map_crf_id(struct iwl_trans *iwl_trans)
case REG_CRF_ID_TYPE_MR:
iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_MR << 12);
break;
- case REG_CRF_ID_TYPE_FM:
- iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_FM << 12);
- break;
+ case REG_CRF_ID_TYPE_FM:
+ case REG_CRF_ID_TYPE_FMI:
+ case REG_CRF_ID_TYPE_FMR:
+ iwl_trans->hw_rf_id = (IWL_CFG_RF_TYPE_FM << 12);
+ break;
default:
ret = -EIO;
IWL_ERR(iwl_trans,
- "Can find a correct rfid for crf id 0x%x\n",
+ "Can't find a correct rfid for crf id 0x%x\n",
REG_CRF_ID_TYPE(val));
goto out;
}
+ /* Set Step-id */
+ iwl_trans->hw_rf_id |= (step_id << 8);
+
/* Set CDB capabilities */
- if (cdb & BIT(4)) {
+ if (cdb_id_wfpm || slave_id) {
iwl_trans->hw_rf_id += BIT(28);
IWL_INFO(iwl_trans, "Adding cdb to rf id\n");
}
- IWL_INFO(iwl_trans, "Detected RF 0x%x from crf id 0x%x\n",
- iwl_trans->hw_rf_id, REG_CRF_ID_TYPE(val));
+ /* Set Jacket capabilities */
+ if (jacket_id_wfpm || jacket_id_cnv) {
+ iwl_trans->hw_rf_id += BIT(29);
+ IWL_INFO(iwl_trans, "Adding jacket to rf id\n");
+ }
+ IWL_INFO(iwl_trans,
+ "Detected rf-type 0x%x step-id 0x%x slave-id 0x%x from crf id 0x%x\n",
+ REG_CRF_ID_TYPE(val), step_id, slave_id, iwl_trans->hw_rf_id);
+ IWL_INFO(iwl_trans,
+ "Detected cdb-id 0x%x jacket-id 0x%x from wfpm id 0x%x\n",
+ cdb_id_wfpm, jacket_id_wfpm, iwl_trans->hw_wfpm_id);
+ IWL_INFO(iwl_trans, "Detected jacket-id 0x%x from cnvi id 0x%x\n",
+ jacket_id_cnv, iwl_trans->hw_cnv_id);
out:
return ret;
@@ -1477,8 +1520,8 @@ static int map_crf_id(struct iwl_trans *iwl_trans)
static const struct iwl_dev_info *
iwl_pci_find_dev_info(u16 device, u16 subsystem_device,
- u16 mac_type, u8 mac_step,
- u16 rf_type, u8 cdb, u8 jacket, u8 rf_id, u8 no_160, u8 cores)
+ u16 mac_type, u8 mac_step, u16 rf_type, u8 cdb,
+ u8 jacket, u8 rf_id, u8 no_160, u8 cores, u8 rf_step)
{
int num_devices = ARRAY_SIZE(iwl_dev_info_table);
int i;
@@ -1529,6 +1572,10 @@ iwl_pci_find_dev_info(u16 device, u16 subsystem_device,
dev_info->cores != cores)
continue;
+ if (dev_info->rf_step != (u8)IWL_CFG_ANY &&
+ dev_info->rf_step != rf_step)
+ continue;
+
return dev_info;
}
@@ -1600,6 +1647,10 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto out_free_trans;
}
+ IWL_INFO(iwl_trans, "PCI dev %04x/%04x, rev=0x%x, rfid=0x%x\n",
+ pdev->device, pdev->subsystem_device,
+ iwl_trans->hw_rev, iwl_trans->hw_rf_id);
+
dev_info = iwl_pci_find_dev_info(pdev->device, pdev->subsystem_device,
CSR_HW_REV_TYPE(iwl_trans->hw_rev),
iwl_trans->hw_rev_step,
@@ -1608,8 +1659,8 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
CSR_HW_RFID_IS_JACKET(iwl_trans->hw_rf_id),
IWL_SUBDEVICE_RF_ID(pdev->subsystem_device),
IWL_SUBDEVICE_NO_160(pdev->subsystem_device),
- IWL_SUBDEVICE_CORES(pdev->subsystem_device));
-
+ IWL_SUBDEVICE_CORES(pdev->subsystem_device),
+ CSR_HW_RFID_STEP(iwl_trans->hw_rf_id));
if (dev_info) {
iwl_trans->cfg = dev_info->cfg;
iwl_trans->name = dev_info->name;
@@ -1729,6 +1780,9 @@ static void iwl_pci_remove(struct pci_dev *pdev)
{
struct iwl_trans *trans = pci_get_drvdata(pdev);
+ if (!trans)
+ return;
+
iwl_drv_stop(trans->drv);
iwl_trans_pcie_free(trans);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
index f7e4f86..69b95ad 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
@@ -497,6 +497,7 @@ int iwl_pcie_rx_stop(struct iwl_trans *trans);
void iwl_pcie_rx_free(struct iwl_trans *trans);
void iwl_pcie_free_rbs_pool(struct iwl_trans *trans);
void iwl_pcie_rx_init_rxb_lists(struct iwl_rxq *rxq);
+void iwl_pcie_rx_napi_sync(struct iwl_trans *trans);
void iwl_pcie_rxq_alloc_rbs(struct iwl_trans *trans, gfp_t priority,
struct iwl_rxq *rxq);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
index 9c9f87f..0d7890f 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2003-2014, 2018-2022 Intel Corporation
+ * Copyright (C) 2003-2014, 2018-2023 Intel Corporation
* Copyright (C) 2013-2015 Intel Mobile Communications GmbH
* Copyright (C) 2016-2017 Intel Deutschland GmbH
*/
@@ -1053,6 +1053,22 @@ static int iwl_pcie_napi_poll_msix(struct napi_struct *napi, int budget)
return ret;
}
+void iwl_pcie_rx_napi_sync(struct iwl_trans *trans)
+{
+ struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
+ int i;
+
+ if (unlikely(!trans_pcie->rxq))
+ return;
+
+ for (i = 0; i < trans->num_rx_queues; i++) {
+ struct iwl_rxq *rxq = &trans_pcie->rxq[i];
+
+ if (rxq && rxq->napi.poll)
+ napi_synchronize(&rxq->napi);
+ }
+}
+
static int _iwl_pcie_rx_init(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
index 1e26315..73b3958 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
@@ -156,6 +156,7 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
if (test_and_clear_bit(STATUS_DEVICE_ENABLED, &trans->status)) {
IWL_DEBUG_INFO(trans,
"DEVICE_ENABLED bit was set and is now cleared\n");
+ iwl_pcie_rx_napi_sync(trans);
iwl_txq_gen2_tx_free(trans);
iwl_pcie_rx_stop(trans);
}
@@ -350,7 +351,7 @@ void iwl_trans_pcie_gen2_fw_alive(struct iwl_trans *trans, u32 scd_addr)
mutex_unlock(&trans_pcie->mutex);
}
-static void iwl_pcie_set_ltr(struct iwl_trans *trans)
+static bool iwl_pcie_set_ltr(struct iwl_trans *trans)
{
u32 ltr_val = CSR_LTR_LONG_VAL_AD_NO_SNOOP_REQ |
u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
@@ -371,18 +372,77 @@ static void iwl_pcie_set_ltr(struct iwl_trans *trans)
trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) &&
!trans->trans_cfg->integrated) {
iwl_write32(trans, CSR_LTR_LONG_VAL_AD, ltr_val);
- } else if (trans->trans_cfg->integrated &&
- trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) {
+ return true;
+ }
+
+ if (trans->trans_cfg->integrated &&
+ trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22000) {
iwl_write_prph(trans, HPM_MAC_LTR_CSR, HPM_MAC_LRT_ENABLE_ALL);
iwl_write_prph(trans, HPM_UMAC_LTR, ltr_val);
+ return true;
}
+
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210) {
+ /* First clear the interrupt, just in case */
+ iwl_write32(trans, CSR_MSIX_HW_INT_CAUSES_AD,
+ MSIX_HW_INT_CAUSES_REG_IML);
+ /* In this case, unfortunately the same ROM bug exists in the
+ * device (not setting LTR correctly), but we don't have control
+ * over the settings from the host due to some hardware security
+ * features. The only workaround we've been able to come up with
+ * so far is to try to keep the CPU and device busy by polling
+ * it and the IML (image loader) completed interrupt.
+ */
+ return false;
+ }
+
+ /* nothing needs to be done on other devices */
+ return true;
+}
+
+static void iwl_pcie_spin_for_iml(struct iwl_trans *trans)
+{
+/* in practice, this seems to complete in around 20-30ms at most, wait 100 */
+#define IML_WAIT_TIMEOUT (HZ / 10)
+ struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
+ unsigned long end_time = jiffies + IML_WAIT_TIMEOUT;
+ u32 value, loops = 0;
+ bool irq = false;
+
+ if (WARN_ON(!trans_pcie->iml))
+ return;
+
+ value = iwl_read32(trans, CSR_LTR_LAST_MSG);
+ IWL_DEBUG_INFO(trans, "Polling for IML load - CSR_LTR_LAST_MSG=0x%x\n",
+ value);
+
+ while (time_before(jiffies, end_time)) {
+ if (iwl_read32(trans, CSR_MSIX_HW_INT_CAUSES_AD) &
+ MSIX_HW_INT_CAUSES_REG_IML) {
+ irq = true;
+ break;
+ }
+ /* Keep the CPU and device busy. */
+ value = iwl_read32(trans, CSR_LTR_LAST_MSG);
+ loops++;
+ }
+
+ IWL_DEBUG_INFO(trans,
+ "Polled for IML load: irq=%d, loops=%d, CSR_LTR_LAST_MSG=0x%x\n",
+ irq, loops, value);
+
+ /* We don't fail here even if we timed out - maybe we get lucky and the
+ * interrupt comes in later (and we get alive from firmware) and then
+ * we're all happy - but if not we'll fail on alive timeout or get some
+ * other error out.
+ */
}
int iwl_trans_pcie_gen2_start_fw(struct iwl_trans *trans,
const struct fw_img *fw, bool run_in_rfkill)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- bool hw_rfkill;
+ bool hw_rfkill, keep_ram_busy;
int ret;
/* This may fail if AMT took ownership of the device */
@@ -443,7 +503,7 @@ int iwl_trans_pcie_gen2_start_fw(struct iwl_trans *trans,
if (ret)
goto out;
- iwl_pcie_set_ltr(trans);
+ keep_ram_busy = !iwl_pcie_set_ltr(trans);
if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
iwl_write32(trans, CSR_FUNC_SCRATCH, CSR_FUNC_SCRATCH_INIT_VALUE);
@@ -455,6 +515,9 @@ int iwl_trans_pcie_gen2_start_fw(struct iwl_trans *trans,
iwl_write_prph(trans, UREG_CPU_INIT_RUN, 1);
}
+ if (keep_ram_busy)
+ iwl_pcie_spin_for_iml(trans);
+
/* re-check RF-Kill state since we may have missed the interrupt */
hw_rfkill = iwl_pcie_check_hw_rf_kill(trans);
if (hw_rfkill && !run_in_rfkill)
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
index 0a9af1a..b281850 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
@@ -599,7 +599,6 @@ static int iwl_pcie_set_hw_ready(struct iwl_trans *trans)
int iwl_pcie_prepare_card_hw(struct iwl_trans *trans)
{
int ret;
- int t = 0;
int iter;
IWL_DEBUG_INFO(trans, "iwl_trans_prepare_card_hw enter\n");
@@ -616,6 +615,8 @@ int iwl_pcie_prepare_card_hw(struct iwl_trans *trans)
usleep_range(1000, 2000);
for (iter = 0; iter < 10; iter++) {
+ int t = 0;
+
/* If HW is not ready, prepare the conditions to check again */
iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG,
CSR_HW_IF_CONFIG_REG_PREPARE);
@@ -1260,6 +1261,7 @@ static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans)
if (test_and_clear_bit(STATUS_DEVICE_ENABLED, &trans->status)) {
IWL_DEBUG_INFO(trans,
"DEVICE_ENABLED bit was set and is now cleared\n");
+ iwl_pcie_rx_napi_sync(trans);
iwl_pcie_tx_stop(trans);
iwl_pcie_rx_stop(trans);
@@ -1522,19 +1524,16 @@ static int iwl_pcie_d3_handshake(struct iwl_trans *trans, bool suspend)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int ret;
- if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210) {
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
suspend ? UREG_DOORBELL_TO_ISR6_SUSPEND :
UREG_DOORBELL_TO_ISR6_RESUME);
- } else if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ) {
+ else if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_BZ)
iwl_write32(trans, CSR_IPC_SLEEP_CONTROL,
suspend ? CSR_IPC_SLEEP_CONTROL_SUSPEND :
CSR_IPC_SLEEP_CONTROL_RESUME);
- iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
- UREG_DOORBELL_TO_ISR6_SLEEP_CTRL);
- } else {
+ else
return 0;
- }
ret = wait_event_timeout(trans_pcie->sx_waitq,
trans_pcie->sx_complete, 2 * HZ);
@@ -2863,7 +2862,7 @@ static bool iwl_write_to_user_buf(char __user *user_buf, ssize_t count,
void *buf, ssize_t *size,
ssize_t *bytes_copied)
{
- int buf_size_left = count - *bytes_copied;
+ ssize_t buf_size_left = count - *bytes_copied;
buf_size_left = buf_size_left - (buf_size_left % sizeof(u32));
if (*size > buf_size_left)
diff --git a/drivers/net/wireless/intel/iwlwifi/queue/tx.c b/drivers/net/wireless/intel/iwlwifi/queue/tx.c
index 726185d..d1c39c2 100644
--- a/drivers/net/wireless/intel/iwlwifi/queue/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/queue/tx.c
@@ -1554,14 +1554,18 @@ void iwl_txq_reclaim(struct iwl_trans *trans, int txq_id, int ssn,
struct sk_buff_head *skbs)
{
struct iwl_txq *txq = trans->txqs.txq[txq_id];
- int tfd_num = iwl_txq_get_cmd_index(txq, ssn);
- int read_ptr = iwl_txq_get_cmd_index(txq, txq->read_ptr);
- int last_to_free;
+ int tfd_num, read_ptr, last_to_free;
/* This function is not meant to release cmd queue*/
if (WARN_ON(txq_id == trans->txqs.cmd.q_id))
return;
+ if (WARN_ON(!txq))
+ return;
+
+ tfd_num = iwl_txq_get_cmd_index(txq, ssn);
+ read_ptr = iwl_txq_get_cmd_index(txq, txq->read_ptr);
+
spin_lock_bh(&txq->lock);
if (!test_bit(txq_id, trans->txqs.queue_used)) {
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index da281cd..465190e 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -402,8 +402,8 @@ mt76_dma_get_buf(struct mt76_dev *dev, struct mt76_queue *q, int idx,
*info = le32_to_cpu(desc->info);
if (mt76_queue_is_wed_rx(q)) {
- u32 token = FIELD_GET(MT_DMA_CTL_TOKEN,
- le32_to_cpu(desc->buf1));
+ u32 buf1 = le32_to_cpu(desc->buf1);
+ u32 token = FIELD_GET(MT_DMA_CTL_TOKEN, buf1);
struct mt76_txwi_cache *t = mt76_rx_token_release(dev, token);
if (!t)
@@ -424,6 +424,8 @@ mt76_dma_get_buf(struct mt76_dev *dev, struct mt76_queue *q, int idx,
*drop = !!(ctrl & (MT_DMA_CTL_TO_HOST_A |
MT_DMA_CTL_DROP));
+
+ *drop |= !!(buf1 & MT_DMA_CTL_WO_DROP);
}
} else {
buf = e->buf;
@@ -576,7 +578,9 @@ mt76_dma_tx_queue_skb(struct mt76_dev *dev, struct mt76_queue *q,
free_skb:
status.skb = tx_info.skb;
hw = mt76_tx_status_get_hw(dev, tx_info.skb);
+ spin_lock_bh(&dev->rx_lock);
ieee80211_tx_status_ext(hw, &status);
+ spin_unlock_bh(&dev->rx_lock);
return ret;
}
@@ -849,7 +853,7 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
!(dev->drv->rx_check(dev, data, len)))
goto free_frag;
- skb = build_skb(data, q->buf_size);
+ skb = napi_build_skb(data, q->buf_size);
if (!skb)
goto free_frag;
diff --git a/drivers/net/wireless/mediatek/mt76/dma.h b/drivers/net/wireless/mediatek/mt76/dma.h
index 4b9bc7f..1b090d7 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.h
+++ b/drivers/net/wireless/mediatek/mt76/dma.h
@@ -19,6 +19,7 @@
#define MT_DMA_CTL_TO_HOST_A BIT(12)
#define MT_DMA_CTL_DROP BIT(14)
#define MT_DMA_CTL_TOKEN GENMASK(31, 16)
+#define MT_DMA_CTL_WO_DROP BIT(8)
#define MT_DMA_PPE_CPU_REASON GENMASK(15, 11)
#define MT_DMA_PPE_ENTRY GENMASK(30, 16)
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index 34abf70..467afef 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -418,7 +418,8 @@ mt76_phy_init(struct mt76_phy *phy, struct ieee80211_hw *hw)
SET_IEEE80211_DEV(hw, dev->dev);
SET_IEEE80211_PERM_ADDR(hw, phy->macaddr);
- wiphy->features |= NL80211_FEATURE_ACTIVE_MONITOR;
+ wiphy->features |= NL80211_FEATURE_ACTIVE_MONITOR |
+ NL80211_FEATURE_AP_MODE_CHAN_WIDTH_CHANGE;
wiphy->flags |= WIPHY_FLAG_HAS_CHANNEL_SWITCH |
WIPHY_FLAG_SUPPORTS_TDLS |
WIPHY_FLAG_AP_UAPSD;
@@ -1066,9 +1067,14 @@ mt76_rx_convert(struct mt76_dev *dev, struct sk_buff *skb,
status->enc_flags = mstat.enc_flags;
status->encoding = mstat.encoding;
status->bw = mstat.bw;
- status->he_ru = mstat.he_ru;
- status->he_gi = mstat.he_gi;
- status->he_dcm = mstat.he_dcm;
+ if (status->encoding == RX_ENC_EHT) {
+ status->eht.ru = mstat.eht.ru;
+ status->eht.gi = mstat.eht.gi;
+ } else {
+ status->he_ru = mstat.he_ru;
+ status->he_gi = mstat.he_gi;
+ status->he_dcm = mstat.he_dcm;
+ }
status->rate_idx = mstat.rate_idx;
status->nss = mstat.nss;
status->band = mstat.band;
@@ -1303,7 +1309,8 @@ mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
if (ps)
set_bit(MT_WCID_FLAG_PS, &wcid->flags);
- dev->drv->sta_ps(dev, sta, ps);
+ if (dev->drv->sta_ps)
+ dev->drv->sta_ps(dev, sta, ps);
if (!ps)
clear_bit(MT_WCID_FLAG_PS, &wcid->flags);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 183b0fc..6b07b8f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -621,12 +621,22 @@ struct mt76_rx_status {
u16 freq;
u32 flag;
u8 enc_flags;
- u8 encoding:2, bw:3, he_ru:3;
- u8 he_gi:2, he_dcm:1;
+ u8 encoding:3, bw:4;
+ union {
+ struct {
+ u8 he_ru:3;
+ u8 he_gi:2;
+ u8 he_dcm:1;
+ };
+ struct {
+ u8 ru:4;
+ u8 gi:2;
+ } eht;
+ };
+
u8 amsdu:1, first_amsdu:1, last_amsdu:1;
u8 rate_idx;
- u8 nss;
- u8 band;
+ u8 nss:5, band:3;
s8 signal;
u8 chains;
s8 chain_signal[IEEE80211_MAX_CHAINS];
@@ -778,6 +788,7 @@ struct mt76_dev {
spinlock_t rx_lock;
struct napi_struct napi[__MT_RXQ_MAX];
struct sk_buff_head rx_skb[__MT_RXQ_MAX];
+ struct tasklet_struct irq_tasklet;
struct list_head txwi_cache;
struct list_head rxwi_cache;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
index 70a7f84..12e0af5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
@@ -1279,8 +1279,11 @@ void mt7603_mac_add_txs(struct mt7603_dev *dev, void *data)
if (wcidx >= MT7603_WTBL_STA || !sta)
goto out;
- if (mt7603_fill_txs(dev, msta, &info, txs_data))
+ if (mt7603_fill_txs(dev, msta, &info, txs_data)) {
+ spin_lock_bh(&dev->mt76.rx_lock);
ieee80211_tx_status_noskb(mt76_hw(dev), sta, &info);
+ spin_unlock_bh(&dev->mt76.rx_lock);
+ }
out:
rcu_read_unlock();
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
index f191443..0ce01cc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
@@ -76,7 +76,8 @@ static int mt7615_poll_tx(struct napi_struct *napi, int budget)
mt76_queue_tx_cleanup(dev, dev->mt76.q_mcu[MT_MCUQ_WM], false);
if (napi_complete(napi))
- mt7615_irq_enable(dev, mt7615_tx_mcu_int_mask(dev));
+ mt76_connac_irq_enable(&dev->mt76,
+ mt7615_tx_mcu_int_mask(dev));
mt76_connac_pm_unref(&dev->mphy, &dev->pm);
@@ -297,7 +298,7 @@ int mt7615_dma_init(struct mt7615_dev *dev)
else
mask |= MT_INT_MCU_CMD;
- mt7615_irq_enable(dev, mask);
+ mt76_connac_irq_enable(&dev->mt76, mask);
mt7615_dma_start(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
index 6dbaaf9..68e8822 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
@@ -47,6 +47,9 @@ static int mt7615_efuse_init(struct mt7615_dev *dev, u32 base)
void *buf;
u32 val;
+ if (is_mt7663(&dev->mt76))
+ len = MT7663_EEPROM_SIZE;
+
val = mt76_rr(dev, base + MT_EFUSE_BASE_CTRL);
if (val & MT_EFUSE_BASE_CTRL_EMPTY)
return 0;
@@ -72,6 +75,8 @@ static int mt7615_eeprom_load(struct mt7615_dev *dev, u32 addr)
{
int ret;
+ BUILD_BUG_ON(MT7615_EEPROM_FULL_SIZE < MT7663_EEPROM_SIZE);
+
ret = mt76_eeprom_init(&dev->mt76, MT7615_EEPROM_FULL_SIZE);
if (ret < 0)
return ret;
@@ -336,7 +341,7 @@ int mt7615_eeprom_init(struct mt7615_dev *dev, u32 addr)
ret = mt7615_check_eeprom(&dev->mt76);
if (ret && dev->mt76.otp.data) {
memcpy(dev->mt76.eeprom.data, dev->mt76.otp.data,
- MT7615_EEPROM_SIZE);
+ dev->mt76.otp.size);
} else {
dev->flash_eeprom = true;
mt7615_cal_free_data(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.h b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.h
index a024dee..a67fbb90f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.h
@@ -46,7 +46,7 @@ enum mt7615_eeprom_field {
MT7615_EE_MAX = 0x3bf,
MT7622_EE_MAX = 0x3db,
- MT7663_EE_MAX = 0x400,
+ MT7663_EE_MAX = 0x600,
};
#define MT_EE_RATE_POWER_MASK GENMASK(5, 0)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/init.c b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
index 5fa6f09..621e69f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
@@ -396,6 +396,7 @@ mt7615_init_wiphy(struct ieee80211_hw *hw)
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_SET_SCAN_DWELL);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_VHT_IBSS);
+ wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
ieee80211_hw_set(hw, SINGLE_SCAN_ON_ALL_BANDS);
ieee80211_hw_set(hw, TX_STATUS_NO_AMPDU_LEN);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
index 51a968a..da1d17b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
@@ -655,11 +655,6 @@ static int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb)
return 0;
}
-void mt7615_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps)
-{
-}
-EXPORT_SYMBOL_GPL(mt7615_sta_ps);
-
static u16
mt7615_mac_tx_rate_val(struct mt7615_dev *dev,
struct mt76_phy *mphy,
@@ -1530,8 +1525,11 @@ static void mt7615_mac_add_txs(struct mt7615_dev *dev, void *data)
if (wcid->phy_idx && dev->mt76.phys[MT_BAND1])
mphy = dev->mt76.phys[MT_BAND1];
- if (mt7615_fill_txs(dev, msta, &info, txs_data))
+ if (mt7615_fill_txs(dev, msta, &info, txs_data)) {
+ spin_lock_bh(&dev->mt76.rx_lock);
ieee80211_tx_status_noskb(mphy->hw, sta, &info);
+ spin_unlock_bh(&dev->mt76.rx_lock);
+ }
out:
rcu_read_unlock();
@@ -2352,7 +2350,7 @@ void mt7615_coredump_work(struct work_struct *work)
break;
skb_pull(skb, sizeof(struct mt7615_mcu_rxd));
- if (data + skb->len - dump > MT76_CONNAC_COREDUMP_SZ) {
+ if (!dump || data + skb->len - dump > MT76_CONNAC_COREDUMP_SZ) {
dev_kfree_skb(skb);
continue;
}
@@ -2362,6 +2360,8 @@ void mt7615_coredump_work(struct work_struct *work)
dev_kfree_skb(skb);
}
- dev_coredumpv(dev->mt76.dev, dump, MT76_CONNAC_COREDUMP_SZ,
- GFP_KERNEL);
+
+ if (dump)
+ dev_coredumpv(dev->mt76.dev, dump, MT76_CONNAC_COREDUMP_SZ,
+ GFP_KERNEL);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
index 880c9f7..d08fbe6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
@@ -19,18 +19,6 @@
#define MT_RXD0_NORMAL_GROUP_3 BIT(27)
#define MT_RXD0_NORMAL_GROUP_4 BIT(28)
-enum rx_pkt_type {
- PKT_TYPE_TXS,
- PKT_TYPE_TXRXV,
- PKT_TYPE_NORMAL,
- PKT_TYPE_RX_DUP_RFB,
- PKT_TYPE_RX_TMR,
- PKT_TYPE_RETRIEVE,
- PKT_TYPE_TXRX_NOTIFY,
- PKT_TYPE_RX_EVENT,
- PKT_TYPE_NORMAL_MCU,
-};
-
#define MT_RXD1_NORMAL_BSSID GENMASK(31, 26)
#define MT_RXD1_NORMAL_PAYLOAD_FORMAT GENMASK(25, 24)
#define MT_RXD1_FIRST_AMSDU_FRAME GENMASK(1, 0)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
index eea398c..8d745c9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
@@ -163,16 +163,16 @@ int mt7615_mcu_parse_response(struct mt76_dev *mdev, int cmd,
cmd == MCU_UNI_CMD(HIF_CTRL) ||
cmd == MCU_UNI_CMD(OFFLOAD) ||
cmd == MCU_UNI_CMD(SUSPEND)) {
- struct mt7615_mcu_uni_event *event;
+ struct mt76_connac_mcu_uni_event *event;
skb_pull(skb, sizeof(*rxd));
- event = (struct mt7615_mcu_uni_event *)skb->data;
+ event = (struct mt76_connac_mcu_uni_event *)skb->data;
ret = le32_to_cpu(event->status);
} else if (cmd == MCU_CE_QUERY(REG_READ)) {
- struct mt7615_mcu_reg_event *event;
+ struct mt76_connac_mcu_reg_event *event;
skb_pull(skb, sizeof(*rxd));
- event = (struct mt7615_mcu_reg_event *)skb->data;
+ event = (struct mt76_connac_mcu_reg_event *)skb->data;
ret = (int)le32_to_cpu(event->val);
}
@@ -861,7 +861,8 @@ mt7615_mcu_wtbl_sta_add(struct mt7615_phy *phy, struct ieee80211_vif *vif,
else
mvif->sta_added = true;
}
- mt76_connac_mcu_sta_basic_tlv(sskb, vif, sta, enable, new_entry);
+ mt76_connac_mcu_sta_basic_tlv(&dev->mt76, sskb, vif, sta, enable,
+ new_entry);
if (enable && sta)
mt76_connac_mcu_sta_tlv(phy->mt76, sskb, sta, vif, 0,
MT76_STA_INFO_STATE_ASSOC);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
index 615956a..8e9604b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
@@ -206,17 +206,6 @@ enum {
MCU_ATE_SET_TX_POWER_CONTROL = 0x15,
};
-struct mt7615_mcu_uni_event {
- u8 cid;
- u8 pad[3];
- __le32 status; /* 0: success, others: fail */
-} __packed;
-
-struct mt7615_mcu_reg_event {
- __le32 reg;
- __le32 val;
-} __packed;
-
struct mt7615_roc_tlv {
u8 bss_idx;
u8 token;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
index 83173ef..ac036a0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mmio.c
@@ -66,9 +66,7 @@ const u32 mt7663e_reg_map[] = {
static void
mt7615_rx_poll_complete(struct mt76_dev *mdev, enum mt76_rxq_id q)
{
- struct mt7615_dev *dev = container_of(mdev, struct mt7615_dev, mt76);
-
- mt7615_irq_enable(dev, MT_INT_RX_DONE(q));
+ mt76_connac_irq_enable(mdev, MT_INT_RX_DONE(q));
}
static irqreturn_t mt7615_irq_handler(int irq, void *dev_instance)
@@ -80,14 +78,14 @@ static irqreturn_t mt7615_irq_handler(int irq, void *dev_instance)
if (!test_bit(MT76_STATE_INITIALIZED, &dev->mphy.state))
return IRQ_NONE;
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
return IRQ_HANDLED;
}
static void mt7615_irq_tasklet(struct tasklet_struct *t)
{
- struct mt7615_dev *dev = from_tasklet(dev, t, irq_tasklet);
+ struct mt7615_dev *dev = from_tasklet(dev, t, mt76.irq_tasklet);
u32 intr, mask = 0, tx_mcu_mask = mt7615_tx_mcu_int_mask(dev);
u32 mcu_int;
@@ -181,7 +179,6 @@ int mt7615_mmio_probe(struct device *pdev, void __iomem *mem_base,
.rx_check = mt7615_rx_check,
.rx_skb = mt7615_queue_rx_skb,
.rx_poll_complete = mt7615_rx_poll_complete,
- .sta_ps = mt7615_sta_ps,
.sta_add = mt7615_mac_sta_add,
.sta_remove = mt7615_mac_sta_remove,
.update_survey = mt7615_update_channel,
@@ -202,7 +199,7 @@ int mt7615_mmio_probe(struct device *pdev, void __iomem *mem_base,
dev = container_of(mdev, struct mt7615_dev, mt76);
mt76_mmio_init(&dev->mt76, mem_base);
- tasklet_setup(&dev->irq_tasklet, mt7615_irq_tasklet);
+ tasklet_setup(&mdev->irq_tasklet, mt7615_irq_tasklet);
dev->reg_map = map;
dev->ops = ops;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
index 9e58f69..582d1b5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
@@ -51,6 +51,7 @@
#define MT7663_FIRMWARE_N9 "mediatek/mt7663_n9_rebb.bin"
#define MT7615_EEPROM_SIZE 1024
+#define MT7663_EEPROM_SIZE 1536
#define MT7615_TOKEN_SIZE 4096
#define MT_FRAC_SCALE 12
@@ -245,8 +246,6 @@ struct mt7615_dev {
};
const struct mt76_bus_ops *bus_ops;
- struct tasklet_struct irq_tasklet;
-
struct mt7615_phy phy;
u64 omac_mask;
@@ -412,13 +411,6 @@ void mt7615_mcu_rx_event(struct mt7615_dev *dev, struct sk_buff *skb);
int mt7615_mcu_rdd_send_pattern(struct mt7615_dev *dev);
int mt7615_mcu_fw_log_2_host(struct mt7615_dev *dev, u8 ctrl);
-static inline void mt7615_irq_enable(struct mt7615_dev *dev, u32 mask)
-{
- mt76_set_irq_mask(&dev->mt76, 0, 0, mask);
-
- tasklet_schedule(&dev->irq_tasklet);
-}
-
static inline bool mt7615_firmware_offload(struct mt7615_dev *dev)
{
return dev->fw_ver > MT7615_FIRMWARE_V2;
@@ -518,7 +510,6 @@ void mt7615_tx_token_put(struct mt7615_dev *dev);
bool mt7615_rx_check(struct mt76_dev *mdev, void *data, int len);
void mt7615_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb, u32 *info);
-void mt7615_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps);
int mt7615_mac_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
void mt7615_mac_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/pci.c b/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
index b808248..9f43e67 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
@@ -94,7 +94,7 @@ static int mt7615_pci_suspend(struct pci_dev *pdev, pm_message_t state)
mt76_for_each_q_rx(mdev, i) {
napi_disable(&mdev->napi[i]);
}
- tasklet_kill(&dev->irq_tasklet);
+ tasklet_kill(&mdev->irq_tasklet);
mt7615_dma_reset(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
index 0680e00..f607eee 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
@@ -122,7 +122,7 @@ void mt7615_unregister_device(struct mt7615_dev *dev)
mt7615_tx_token_put(dev);
mt7615_dma_cleanup(dev);
- tasklet_disable(&dev->irq_tasklet);
+ tasklet_disable(&dev->mt76.irq_tasklet);
mt76_free_device(&dev->mt76);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c b/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c
index 304212f..fc547a0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/sdio.c
@@ -84,7 +84,6 @@ static int mt7663s_probe(struct sdio_func *func,
.tx_status_data = mt7663_usb_sdio_tx_status_data,
.rx_skb = mt7615_queue_rx_skb,
.rx_check = mt7615_rx_check,
- .sta_ps = mt7615_sta_ps,
.sta_add = mt7615_mac_sta_add,
.sta_remove = mt7615_mac_sta_remove,
.update_survey = mt7615_update_channel,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/usb.c b/drivers/net/wireless/mediatek/mt76/mt7615/usb.c
index f2d651d..04963b9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/usb.c
@@ -120,7 +120,6 @@ static int mt7663u_probe(struct usb_interface *usb_intf,
.tx_status_data = mt7663_usb_sdio_tx_status_data,
.rx_skb = mt7615_queue_rx_skb,
.rx_check = mt7615_rx_check,
- .sta_ps = mt7615_sta_ps,
.sta_add = mt7615_mac_sta_add,
.sta_remove = mt7615_mac_sta_remove,
.update_survey = mt7615_update_channel,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac.h
index b339c50..15653b2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac.h
@@ -6,6 +6,20 @@
#include "mt76.h"
+enum rx_pkt_type {
+ PKT_TYPE_TXS,
+ PKT_TYPE_TXRXV,
+ PKT_TYPE_NORMAL,
+ PKT_TYPE_RX_DUP_RFB,
+ PKT_TYPE_RX_TMR,
+ PKT_TYPE_RETRIEVE,
+ PKT_TYPE_TXRX_NOTIFY,
+ PKT_TYPE_RX_EVENT,
+ PKT_TYPE_NORMAL_MCU,
+ PKT_TYPE_RX_FW_MONITOR = 0x0c,
+ PKT_TYPE_TXRX_NOTIFY_V0 = 0x18,
+};
+
#define MT76_CONNAC_SCAN_IE_LEN 600
#define MT76_CONNAC_MAX_NUM_SCHED_SCAN_INTERVAL 10
#define MT76_CONNAC_MAX_TIME_SCHED_SCAN_INTERVAL U16_MAX
@@ -279,6 +293,12 @@ static inline u8 mt76_connac_spe_idx(u8 antenna_mask)
return ant_to_spe[antenna_mask];
}
+static inline void mt76_connac_irq_enable(struct mt76_dev *dev, u32 mask)
+{
+ mt76_set_irq_mask(dev, 0, 0, mask);
+ tasklet_schedule(&dev->irq_tasklet);
+}
+
int mt76_connac_pm_wake(struct mt76_phy *phy, struct mt76_connac_pm *pm);
void mt76_connac_power_save_sched(struct mt76_phy *phy,
struct mt76_connac_pm *pm);
@@ -353,6 +373,7 @@ mt76_connac_mutex_release(struct mt76_dev *dev, struct mt76_connac_pm *pm)
mutex_unlock(&dev->mutex);
}
+void mt76_connac_gen_ppe_thresh(u8 *he_ppet, int nss);
int mt76_connac_init_tx_queues(struct mt76_phy *phy, int idx, int n_desc,
int ring_base, u32 flags);
void mt76_connac_write_hw_txp(struct mt76_dev *dev,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h
index f33171b..a5ec0f6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac2_mac.h
@@ -32,6 +32,16 @@ enum {
MT_LMAC_PSMP0,
};
+#define MT_TX_FREE_MSDU_CNT GENMASK(9, 0)
+#define MT_TX_FREE_WLAN_ID GENMASK(23, 14)
+#define MT_TX_FREE_LATENCY GENMASK(12, 0)
+/* 0: success, others: dropped */
+#define MT_TX_FREE_STATUS GENMASK(14, 13)
+#define MT_TX_FREE_MSDU_ID GENMASK(30, 16)
+#define MT_TX_FREE_PAIR BIT(31)
+/* will support this field in further revision */
+#define MT_TX_FREE_RATE GENMASK(13, 0)
+
#define MT_TXD0_Q_IDX GENMASK(31, 25)
#define MT_TXD0_PKT_FMT GENMASK(24, 23)
#define MT_TXD0_ETH_TYPE_OFFSET GENMASK(22, 16)
@@ -166,6 +176,15 @@ enum {
#define MT_TXS7_MPDU_RETRY_CNT GENMASK(31, 23)
+/* RXD DW0 */
+#define MT_RXD0_LENGTH GENMASK(15, 0)
+#define MT_RXD0_PKT_FLAG GENMASK(19, 16)
+#define MT_RXD0_PKT_TYPE GENMASK(31, 27)
+
+#define MT_RXD0_NORMAL_ETH_TYPE_OFS GENMASK(22, 16)
+#define MT_RXD0_NORMAL_IP_SUM BIT(23)
+#define MT_RXD0_NORMAL_UDP_TCP_SUM BIT(24)
+
/* RXD DW1 */
#define MT_RXD1_NORMAL_WLAN_IDX GENMASK(9, 0)
#define MT_RXD1_NORMAL_GROUP_1 BIT(11)
@@ -308,6 +327,9 @@ enum {
#define MT_CRXV_FOE_HI GENMASK(6, 0)
#define MT_CRXV_FOE_SHIFT 13
+#define MT_CT_PARSE_LEN 72
+#define MT_CT_DMA_BUF_NUM 2
+
#define MT_CT_INFO_APPLY_TXD BIT(0)
#define MT_CT_INFO_COPY_HOST_TXD_ALL BIT(1)
#define MT_CT_INFO_MGMT_FRAME BIT(2)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
index aed4ee9..ee0fbfcd 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
@@ -9,6 +9,27 @@
#define HE_PREP(f, m, v) le16_encode_bits(le32_get_bits(v, MT_CRXV_HE_##m),\
IEEE80211_RADIOTAP_HE_##f)
+void mt76_connac_gen_ppe_thresh(u8 *he_ppet, int nss)
+{
+ static const u8 ppet16_ppet8_ru3_ru0[] = { 0x1c, 0xc7, 0x71 };
+ u8 i, ppet_bits, ppet_size, ru_bit_mask = 0x7; /* HE80 */
+
+ he_ppet[0] = FIELD_PREP(IEEE80211_PPE_THRES_NSS_MASK, nss - 1) |
+ FIELD_PREP(IEEE80211_PPE_THRES_RU_INDEX_BITMASK_MASK,
+ ru_bit_mask);
+
+ ppet_bits = IEEE80211_PPE_THRES_INFO_PPET_SIZE *
+ nss * hweight8(ru_bit_mask) * 2;
+ ppet_size = DIV_ROUND_UP(ppet_bits, 8);
+
+ for (i = 0; i < ppet_size - 1; i++)
+ he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3];
+
+ he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3] &
+ (0xff >> (8 - (ppet_bits - 1) % 8));
+}
+EXPORT_SYMBOL_GPL(mt76_connac_gen_ppe_thresh);
+
int mt76_connac_pm_wake(struct mt76_phy *phy, struct mt76_connac_pm *pm)
{
struct mt76_dev *dev = phy->dev;
@@ -267,11 +288,29 @@ int mt76_connac_init_tx_queues(struct mt76_phy *phy, int idx, int n_desc,
}
EXPORT_SYMBOL_GPL(mt76_connac_init_tx_queues);
+#define __bitrate_mask_check(_mcs, _mode) \
+({ \
+ u8 i = 0; \
+ for (nss = 0; i < ARRAY_SIZE(mask->control[band]._mcs); i++) { \
+ if (!mask->control[band]._mcs[i]) \
+ continue; \
+ if (hweight16(mask->control[band]._mcs[i]) == 1) { \
+ mode = MT_PHY_TYPE_##_mode; \
+ rateidx = ffs(mask->control[band]._mcs[i]) - 1; \
+ if (mode == MT_PHY_TYPE_HT) \
+ rateidx += 8 * i; \
+ else \
+ nss = i + 1; \
+ goto out; \
+ } \
+ } \
+})
+
u16 mt76_connac2_mac_tx_rate_val(struct mt76_phy *mphy,
struct ieee80211_vif *vif,
bool beacon, bool mcast)
{
- u8 mode = 0, band = mphy->chandef.chan->band;
+ u8 nss = 0, mode = 0, band = mphy->chandef.chan->band;
int rateidx = 0, mcast_rate;
if (!vif)
@@ -286,19 +325,12 @@ u16 mt76_connac2_mac_tx_rate_val(struct mt76_phy *mphy,
struct cfg80211_bitrate_mask *mask;
mask = &vif->bss_conf.beacon_tx_rate;
- if (hweight16(mask->control[band].he_mcs[0]) == 1) {
- rateidx = ffs(mask->control[band].he_mcs[0]) - 1;
- mode = MT_PHY_TYPE_HE_SU;
- goto out;
- } else if (hweight16(mask->control[band].vht_mcs[0]) == 1) {
- rateidx = ffs(mask->control[band].vht_mcs[0]) - 1;
- mode = MT_PHY_TYPE_VHT;
- goto out;
- } else if (hweight8(mask->control[band].ht_mcs[0]) == 1) {
- rateidx = ffs(mask->control[band].ht_mcs[0]) - 1;
- mode = MT_PHY_TYPE_HT;
- goto out;
- } else if (hweight32(mask->control[band].legacy) == 1) {
+
+ __bitrate_mask_check(he_mcs, HE_SU);
+ __bitrate_mask_check(vht_mcs, VHT);
+ __bitrate_mask_check(ht_mcs, HT);
+
+ if (hweight32(mask->control[band].legacy) == 1) {
rateidx = ffs(mask->control[band].legacy) - 1;
goto legacy;
}
@@ -314,9 +346,9 @@ u16 mt76_connac2_mac_tx_rate_val(struct mt76_phy *mphy,
rateidx = mt76_calculate_default_rate(mphy, rateidx);
mode = rateidx >> 8;
rateidx &= GENMASK(7, 0);
-
out:
- return FIELD_PREP(MT_TX_RATE_IDX, rateidx) |
+ return FIELD_PREP(MT_TX_RATE_NSS, nss) |
+ FIELD_PREP(MT_TX_RATE_IDX, rateidx) |
FIELD_PREP(MT_TX_RATE_MODE, mode);
}
EXPORT_SYMBOL_GPL(mt76_connac2_mac_tx_rate_val);
@@ -537,7 +569,8 @@ void mt76_connac2_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi,
if (txwi[2] & cpu_to_le32(MT_TXD2_FIX_RATE)) {
/* Fixed rata is available just for 802.11 txd */
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
- bool multicast = is_multicast_ether_addr(hdr->addr1);
+ bool multicast = ieee80211_is_data(hdr->frame_control) &&
+ is_multicast_ether_addr(hdr->addr1);
u16 rate = mt76_connac2_mac_tx_rate_val(mphy, vif, beacon,
multicast);
u32 val = MT_TXD6_FIXED_BW;
@@ -582,6 +615,17 @@ bool mt76_connac2_mac_fill_txs(struct mt76_dev *dev, struct mt76_wcid *wcid,
le32_get_bits(txs_data[6], MT_TXS6_MPDU_FAIL_CNT);
stats->tx_retries +=
le32_get_bits(txs_data[7], MT_TXS7_MPDU_RETRY_CNT);
+
+ if (wcid->sta) {
+ struct ieee80211_sta *sta;
+ u8 tid;
+
+ sta = container_of((void *)wcid, struct ieee80211_sta,
+ drv_priv);
+ tid = FIELD_GET(MT_TXS0_TID, txs);
+
+ ieee80211_refresh_tx_agg_session_timer(sta, tid);
+ }
}
txrate = FIELD_GET(MT_TXS0_TX_RATE, txs);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
index 008ece1..0f0a519 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c
@@ -363,7 +363,7 @@ void mt76_connac_mcu_bss_omac_tlv(struct sk_buff *skb,
}
EXPORT_SYMBOL_GPL(mt76_connac_mcu_bss_omac_tlv);
-void mt76_connac_mcu_sta_basic_tlv(struct sk_buff *skb,
+void mt76_connac_mcu_sta_basic_tlv(struct mt76_dev *dev, struct sk_buff *skb,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
bool enable, bool newly)
@@ -394,7 +394,7 @@ void mt76_connac_mcu_sta_basic_tlv(struct sk_buff *skb,
switch (vif->type) {
case NL80211_IFTYPE_MESH_POINT:
case NL80211_IFTYPE_AP:
- if (vif->p2p)
+ if (vif->p2p && !is_mt7921(dev))
conn_type = CONNECTION_P2P_GC;
else
conn_type = CONNECTION_INFRA_STA;
@@ -402,7 +402,7 @@ void mt76_connac_mcu_sta_basic_tlv(struct sk_buff *skb,
basic->aid = cpu_to_le16(sta->aid);
break;
case NL80211_IFTYPE_STATION:
- if (vif->p2p)
+ if (vif->p2p && !is_mt7921(dev))
conn_type = CONNECTION_P2P_GO;
else
conn_type = CONNECTION_INFRA_AP;
@@ -1029,7 +1029,7 @@ int mt76_connac_mcu_sta_cmd(struct mt76_phy *phy,
return PTR_ERR(skb);
if (info->sta || !info->offload_fw)
- mt76_connac_mcu_sta_basic_tlv(skb, info->vif, info->sta,
+ mt76_connac_mcu_sta_basic_tlv(dev, skb, info->vif, info->sta,
info->enable, info->newly);
if (info->sta && info->enable)
mt76_connac_mcu_sta_tlv(phy, skb, info->sta,
@@ -1678,8 +1678,16 @@ int mt76_connac_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif,
req->channel_min_dwell_time = cpu_to_le16(duration);
req->channel_dwell_time = cpu_to_le16(duration);
- req->channels_num = min_t(u8, sreq->n_channels, 32);
- req->ext_channels_num = min_t(u8, ext_channels_num, 32);
+ if (sreq->n_channels == 0 || sreq->n_channels > 64) {
+ req->channel_type = 0;
+ req->channels_num = 0;
+ req->ext_channels_num = 0;
+ } else {
+ req->channel_type = 4;
+ req->channels_num = min_t(u8, sreq->n_channels, 32);
+ req->ext_channels_num = min_t(u8, ext_channels_num, 32);
+ }
+
for (i = 0; i < req->channels_num + req->ext_channels_num; i++) {
if (i >= 32)
chan = &req->ext_channels[i - 32];
@@ -1699,7 +1707,6 @@ int mt76_connac_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif,
}
chan->channel_num = scan_list[i]->hw_value;
}
- req->channel_type = sreq->n_channels ? 4 : 0;
if (sreq->ie_len > 0) {
memcpy(req->ies, sreq->ie, sreq->ie_len);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
index a5e6ee4..ca1ce97 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h
@@ -127,7 +127,7 @@ struct mt76_connac2_mcu_rxd {
u8 rsv1[2];
u8 s2d_index;
- u8 tlv[0];
+ u8 tlv[];
};
struct mt76_connac2_patch_hdr {
@@ -967,9 +967,6 @@ enum {
DEV_INFO_MAX_NUM
};
-#define MCU_UNI_CMD_EVENT BIT(1)
-#define MCU_UNI_CMD_UNSOLICITED_EVENT BIT(2)
-
/* event table */
enum {
MCU_EVENT_TARGET_ADDRESS_LEN = 0x01,
@@ -1224,6 +1221,7 @@ enum {
MCU_UNI_CMD_VOW = 0x37,
MCU_UNI_CMD_RRO = 0x57,
MCU_UNI_CMD_OFFCH_SCAN_CTRL = 0x58,
+ MCU_UNI_CMD_ASSERT_DUMP = 0x6f,
};
enum {
@@ -1692,6 +1690,17 @@ struct mt76_connac_config {
u8 data[320];
} __packed;
+struct mt76_connac_mcu_uni_event {
+ u8 cid;
+ u8 pad[3];
+ __le32 status; /* 0: success, others: fail */
+} __packed;
+
+struct mt76_connac_mcu_reg_event {
+ __le32 reg;
+ __le32 val;
+} __packed;
+
static inline enum mcu_cipher_type
mt76_connac_mcu_get_cipher(int cipher)
{
@@ -1779,7 +1788,7 @@ mt76_connac_mcu_add_tlv(struct sk_buff *skb, int tag, int len)
int mt76_connac_mcu_set_channel_domain(struct mt76_phy *phy);
int mt76_connac_mcu_set_vif_ps(struct mt76_dev *dev, struct ieee80211_vif *vif);
-void mt76_connac_mcu_sta_basic_tlv(struct sk_buff *skb,
+void mt76_connac_mcu_sta_basic_tlv(struct mt76_dev *dev, struct sk_buff *skb,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool enable,
bool newly);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
index d3f7447..3e41d80 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
@@ -631,8 +631,11 @@ void mt76x02_send_tx_status(struct mt76x02_dev *dev,
mt76_tx_status_unlock(mdev, &list);
- if (!status.skb)
+ if (!status.skb) {
+ spin_lock_bh(&dev->mt76.rx_lock);
ieee80211_tx_status_ext(mt76_hw(dev), &status);
+ spin_unlock_bh(&dev->mt76.rx_lock);
+ }
if (!len)
goto out;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
index 5a46813..879884e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c
@@ -958,10 +958,10 @@ mt7915_xmit_queues_show(struct seq_file *file, void *data)
DEFINE_SHOW_ATTRIBUTE(mt7915_xmit_queues);
-#define mt7915_txpower_puts(prefix, rate) \
+#define mt7915_txpower_puts(rate) \
({ \
- len += scnprintf(buf + len, sz - len, "%-16s:", #prefix " (tmac)"); \
- for (i = 0; i < mt7915_sku_group_len[rate]; i++, offs++) \
+ len += scnprintf(buf + len, sz - len, "%-16s:", #rate " (TMAC)"); \
+ for (i = 0; i < mt7915_sku_group_len[SKU_##rate]; i++, offs++) \
len += scnprintf(buf + len, sz - len, " %6d", txpwr[offs]); \
len += scnprintf(buf + len, sz - len, "\n"); \
})
@@ -1004,41 +1004,41 @@ mt7915_rate_txpower_get(struct file *file, char __user *user_buf,
phy != &dev->phy, phy->mt76->chandef.chan->hw_value);
len += scnprintf(buf + len, sz - len, "%-16s %6s %6s %6s %6s\n",
" ", "1m", "2m", "5m", "11m");
- mt7915_txpower_puts(CCK, SKU_CCK);
+ mt7915_txpower_puts(CCK);
len += scnprintf(buf + len, sz - len,
"%-16s %6s %6s %6s %6s %6s %6s %6s %6s\n",
" ", "6m", "9m", "12m", "18m", "24m", "36m", "48m",
"54m");
- mt7915_txpower_puts(OFDM, SKU_OFDM);
+ mt7915_txpower_puts(OFDM);
len += scnprintf(buf + len, sz - len,
"%-16s %6s %6s %6s %6s %6s %6s %6s %6s\n",
" ", "mcs0", "mcs1", "mcs2", "mcs3", "mcs4",
"mcs5", "mcs6", "mcs7");
- mt7915_txpower_puts(HT20, SKU_HT_BW20);
+ mt7915_txpower_puts(HT_BW20);
len += scnprintf(buf + len, sz - len,
"%-16s %6s %6s %6s %6s %6s %6s %6s %6s %6s\n",
" ", "mcs0", "mcs1", "mcs2", "mcs3", "mcs4", "mcs5",
"mcs6", "mcs7", "mcs32");
- mt7915_txpower_puts(HT40, SKU_HT_BW40);
+ mt7915_txpower_puts(HT_BW40);
len += scnprintf(buf + len, sz - len,
"%-16s %6s %6s %6s %6s %6s %6s %6s %6s %6s %6s %6s %6s\n",
" ", "mcs0", "mcs1", "mcs2", "mcs3", "mcs4", "mcs5",
"mcs6", "mcs7", "mcs8", "mcs9", "mcs10", "mcs11");
- mt7915_txpower_puts(VHT20, SKU_VHT_BW20);
- mt7915_txpower_puts(VHT40, SKU_VHT_BW40);
- mt7915_txpower_puts(VHT80, SKU_VHT_BW80);
- mt7915_txpower_puts(VHT160, SKU_VHT_BW160);
- mt7915_txpower_puts(HE26, SKU_HE_RU26);
- mt7915_txpower_puts(HE52, SKU_HE_RU52);
- mt7915_txpower_puts(HE106, SKU_HE_RU106);
- mt7915_txpower_puts(HE242, SKU_HE_RU242);
- mt7915_txpower_puts(HE484, SKU_HE_RU484);
- mt7915_txpower_puts(HE996, SKU_HE_RU996);
- mt7915_txpower_puts(HE996x2, SKU_HE_RU2x996);
+ mt7915_txpower_puts(VHT_BW20);
+ mt7915_txpower_puts(VHT_BW40);
+ mt7915_txpower_puts(VHT_BW80);
+ mt7915_txpower_puts(VHT_BW160);
+ mt7915_txpower_puts(HE_RU26);
+ mt7915_txpower_puts(HE_RU52);
+ mt7915_txpower_puts(HE_RU106);
+ mt7915_txpower_puts(HE_RU242);
+ mt7915_txpower_puts(HE_RU484);
+ mt7915_txpower_puts(HE_RU996);
+ mt7915_txpower_puts(HE_RU2x996);
reg = is_mt7915(&dev->mt76) ? MT_WF_PHY_TPC_CTRL_STAT(band) :
MT_WF_PHY_TPC_CTRL_STAT_MT7916(band);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/dma.c b/drivers/net/wireless/mediatek/mt76/mt7915/dma.c
index abe17da..43a5456 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/dma.c
@@ -87,8 +87,14 @@ static void mt7915_dma_config(struct mt7915_dev *dev)
MT7916_RXQ_BAND0);
RXQ_CONFIG(MT_RXQ_MCU_WA, WFDMA0, MT_INT_WED_RX_DONE_WA_MT7916,
MT7916_RXQ_MCU_WA);
- RXQ_CONFIG(MT_RXQ_BAND1, WFDMA0, MT_INT_WED_RX_DONE_BAND1_MT7916,
- MT7916_RXQ_BAND1);
+ if (dev->hif2)
+ RXQ_CONFIG(MT_RXQ_BAND1, WFDMA0,
+ MT_INT_RX_DONE_BAND1_MT7916,
+ MT7916_RXQ_BAND1);
+ else
+ RXQ_CONFIG(MT_RXQ_BAND1, WFDMA0,
+ MT_INT_WED_RX_DONE_BAND1_MT7916,
+ MT7916_RXQ_BAND1);
RXQ_CONFIG(MT_RXQ_MAIN_WA, WFDMA0, MT_INT_WED_RX_DONE_WA_MAIN_MT7916,
MT7916_RXQ_MCU_WA_MAIN);
TXQ_CONFIG(0, WFDMA0, MT_INT_WED_TX_DONE_BAND0,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/init.c b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
index 5e28811..ac2049f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
@@ -89,6 +89,7 @@ static ssize_t mt7915_thermal_temp_store(struct device *dev,
val < phy->throttle_temp[MT7915_CRIT_TEMP_IDX])) {
dev_err(phy->dev->mt76.dev,
"temp1_max shall be greater than temp1_crit.");
+ mutex_unlock(&phy->dev->mt76.mutex);
return -EINVAL;
}
@@ -202,6 +203,10 @@ static int mt7915_thermal_init(struct mt7915_phy *phy)
phy->cdev = cdev;
}
+ /* initialize critical/maximum high temperature */
+ phy->throttle_temp[MT7915_CRIT_TEMP_IDX] = MT7915_CRIT_TEMP;
+ phy->throttle_temp[MT7915_MAX_TEMP_IDX] = MT7915_MAX_TEMP;
+
if (!IS_REACHABLE(CONFIG_HWMON))
return 0;
@@ -210,10 +215,6 @@ static int mt7915_thermal_init(struct mt7915_phy *phy)
if (IS_ERR(hwmon))
return PTR_ERR(hwmon);
- /* initialize critical/maximum high temperature */
- phy->throttle_temp[MT7915_CRIT_TEMP_IDX] = MT7915_CRIT_TEMP;
- phy->throttle_temp[MT7915_MAX_TEMP_IDX] = MT7915_MAX_TEMP;
-
return 0;
}
@@ -368,6 +369,7 @@ mt7915_init_wiphy(struct mt7915_phy *phy)
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_UNSOL_BCAST_PROBE_RESP);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_FILS_DISCOVERY);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_ACK_SIGNAL_SUPPORT);
+ wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
if (!is_mt7915(&dev->mt76))
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_STA_TX_PWR);
@@ -930,27 +932,6 @@ mt7915_set_stream_he_txbf_caps(struct mt7915_phy *phy,
}
}
-static void
-mt7915_gen_ppe_thresh(u8 *he_ppet, int nss)
-{
- u8 i, ppet_bits, ppet_size, ru_bit_mask = 0x7; /* HE80 */
- static const u8 ppet16_ppet8_ru3_ru0[] = {0x1c, 0xc7, 0x71};
-
- he_ppet[0] = FIELD_PREP(IEEE80211_PPE_THRES_NSS_MASK, nss - 1) |
- FIELD_PREP(IEEE80211_PPE_THRES_RU_INDEX_BITMASK_MASK,
- ru_bit_mask);
-
- ppet_bits = IEEE80211_PPE_THRES_INFO_PPET_SIZE *
- nss * hweight8(ru_bit_mask) * 2;
- ppet_size = DIV_ROUND_UP(ppet_bits, 8);
-
- for (i = 0; i < ppet_size - 1; i++)
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3];
-
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3] &
- (0xff >> (8 - (ppet_bits - 1) % 8));
-}
-
static int
mt7915_init_he_caps(struct mt7915_phy *phy, enum nl80211_band band,
struct ieee80211_sband_iftype_data *data)
@@ -1100,7 +1081,7 @@ mt7915_init_he_caps(struct mt7915_phy *phy, enum nl80211_band band,
memset(he_cap->ppe_thres, 0, sizeof(he_cap->ppe_thres));
if (he_cap_elem->phy_cap_info[6] &
IEEE80211_HE_PHY_CAP6_PPE_THRESHOLD_PRESENT) {
- mt7915_gen_ppe_thresh(he_cap->ppe_thres, nss);
+ mt76_connac_gen_ppe_thresh(he_cap->ppe_thres, nss);
} else {
he_cap_elem->phy_cap_info[9] |=
u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
@@ -1179,7 +1160,7 @@ static void mt7915_stop_hardware(struct mt7915_dev *dev)
mt7915_mcu_exit(dev);
mt7915_tx_token_put(dev);
mt7915_dma_cleanup(dev);
- tasklet_disable(&dev->irq_tasklet);
+ tasklet_disable(&dev->mt76.irq_tasklet);
if (is_mt7986(&dev->mt76))
mt7986_wmac_disable(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
index 97ca55d..7df8d95 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
@@ -73,10 +73,6 @@ static struct mt76_wcid *mt7915_rx_get_wcid(struct mt7915_dev *dev,
return &sta->vif->sta.wcid;
}
-void mt7915_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps)
-{
-}
-
bool mt7915_mac_wtbl_update(struct mt7915_dev *dev, int idx, u32 mask)
{
mt76_rmw(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_WLAN_IDX,
@@ -1627,7 +1623,7 @@ void mt7915_mac_reset_work(struct work_struct *work)
}
local_bh_enable();
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
mt76_wr(dev, MT_MCU_INT_EVENT, MT_MCU_INT_EVENT_RESET_DONE);
mt7915_wait_reset_state(dev, MT_MCU_CMD_NORMAL_STATE);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.h b/drivers/net/wireless/mediatek/mt76/mt7915/mac.h
index 6fa9c79..ce94f87 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.h
@@ -6,43 +6,12 @@
#include "../mt76_connac2_mac.h"
-#define MT_CT_PARSE_LEN 72
-#define MT_CT_DMA_BUF_NUM 2
-
-#define MT_RXD0_LENGTH GENMASK(15, 0)
-#define MT_RXD0_PKT_TYPE GENMASK(31, 27)
-
-#define MT_RXD0_NORMAL_ETH_TYPE_OFS GENMASK(22, 16)
-#define MT_RXD0_NORMAL_IP_SUM BIT(23)
-#define MT_RXD0_NORMAL_UDP_TCP_SUM BIT(24)
-
-enum rx_pkt_type {
- PKT_TYPE_TXS,
- PKT_TYPE_TXRXV,
- PKT_TYPE_NORMAL,
- PKT_TYPE_RX_DUP_RFB,
- PKT_TYPE_RX_TMR,
- PKT_TYPE_RETRIEVE,
- PKT_TYPE_TXRX_NOTIFY,
- PKT_TYPE_RX_EVENT,
- PKT_TYPE_RX_FW_MONITOR = 0x0c,
- PKT_TYPE_TXRX_NOTIFY_V0 = 0x18,
-};
-
#define MT_TX_FREE_VER GENMASK(18, 16)
-#define MT_TX_FREE_MSDU_CNT GENMASK(9, 0)
-#define MT_TX_FREE_MSDU_CNT_V0 GENMASK(6, 0)
-#define MT_TX_FREE_WLAN_ID GENMASK(23, 14)
-#define MT_TX_FREE_LATENCY GENMASK(12, 0)
+#define MT_TX_FREE_MSDU_CNT_V0 GENMASK(6, 0)
/* 0: success, others: dropped */
-#define MT_TX_FREE_MSDU_ID GENMASK(30, 16)
-#define MT_TX_FREE_PAIR BIT(31)
#define MT_TX_FREE_MPDU_HEADER BIT(30)
#define MT_TX_FREE_MSDU_ID_V3 GENMASK(14, 0)
-/* will support this field in further revision */
-#define MT_TX_FREE_RATE GENMASK(13, 0)
-
#define MT_TXS5_F0_FINAL_MPDU BIT(31)
#define MT_TXS5_F0_QOS BIT(30)
#define MT_TXS5_F0_TX_COUNT GENMASK(29, 25)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
index 784191e..1b36119 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
@@ -269,7 +269,6 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
vif->offload_flags |= IEEE80211_OFFLOAD_ENCAP_4ADDR;
mt7915_init_bitrate_mask(vif);
- memset(&mvif->cap, -1, sizeof(mvif->cap));
mt7915_mcu_add_bss_info(phy, vif, true);
mt7915_mcu_add_sta(dev, vif, NULL, true);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
index 5545a8b..9fcb22f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
@@ -706,7 +706,6 @@ static void
mt7915_mcu_sta_he_tlv(struct sk_buff *skb, struct ieee80211_sta *sta,
struct ieee80211_vif *vif)
{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct ieee80211_he_cap_elem *elem = &sta->deflink.he_cap.he_cap_elem;
struct ieee80211_he_mcs_nss_supp mcs_map;
struct sta_rec_he *he;
@@ -740,7 +739,7 @@ mt7915_mcu_sta_he_tlv(struct sk_buff *skb, struct ieee80211_sta *sta,
IEEE80211_HE_PHY_CAP0_CHANNEL_WIDTH_SET_RU_MAPPING_IN_5G))
cap |= STA_REC_HE_CAP_BW20_RU242_SUPPORT;
- if (mvif->cap.he_ldpc &&
+ if (vif->bss_conf.he_ldpc &&
(elem->phy_cap_info[1] &
IEEE80211_HE_PHY_CAP1_LDPC_CODING_IN_PAYLOAD))
cap |= STA_REC_HE_CAP_LDPC;
@@ -849,7 +848,6 @@ static void
mt7915_mcu_sta_muru_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
struct ieee80211_sta *sta, struct ieee80211_vif *vif)
{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
struct ieee80211_he_cap_elem *elem = &sta->deflink.he_cap.he_cap_elem;
struct sta_rec_muru *muru;
struct tlv *tlv;
@@ -862,9 +860,9 @@ mt7915_mcu_sta_muru_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
muru = (struct sta_rec_muru *)tlv;
- muru->cfg.mimo_dl_en = mvif->cap.he_mu_ebfer ||
- mvif->cap.vht_mu_ebfer ||
- mvif->cap.vht_mu_ebfee;
+ muru->cfg.mimo_dl_en = vif->bss_conf.he_mu_beamformer ||
+ vif->bss_conf.vht_mu_beamformer ||
+ vif->bss_conf.vht_mu_beamformee;
if (!is_mt7915(&dev->mt76))
muru->cfg.mimo_ul_en = true;
muru->cfg.ofdma_dl_en = true;
@@ -997,8 +995,8 @@ mt7915_mcu_sta_wtbl_tlv(struct mt7915_dev *dev, struct sk_buff *skb,
mt76_connac_mcu_wtbl_hdr_trans_tlv(skb, vif, wcid, tlv, wtbl_hdr);
if (sta)
mt76_connac_mcu_wtbl_ht_tlv(&dev->mt76, skb, sta, tlv,
- wtbl_hdr, mvif->cap.ht_ldpc,
- mvif->cap.vht_ldpc);
+ wtbl_hdr, vif->bss_conf.ht_ldpc,
+ vif->bss_conf.vht_ldpc);
return 0;
}
@@ -1007,7 +1005,6 @@ static inline bool
mt7915_is_ebf_supported(struct mt7915_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool bfee)
{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
int tx_ant = hweight8(phy->mt76->chainmask) - 1;
if (vif->type != NL80211_IFTYPE_STATION &&
@@ -1021,10 +1018,10 @@ mt7915_is_ebf_supported(struct mt7915_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_he_cap_elem *pe = &sta->deflink.he_cap.he_cap_elem;
if (bfee)
- return mvif->cap.he_su_ebfee &&
+ return vif->bss_conf.he_su_beamformee &&
HE_PHY(CAP3_SU_BEAMFORMER, pe->phy_cap_info[3]);
else
- return mvif->cap.he_su_ebfer &&
+ return vif->bss_conf.he_su_beamformer &&
HE_PHY(CAP4_SU_BEAMFORMEE, pe->phy_cap_info[4]);
}
@@ -1032,10 +1029,10 @@ mt7915_is_ebf_supported(struct mt7915_phy *phy, struct ieee80211_vif *vif,
u32 cap = sta->deflink.vht_cap.cap;
if (bfee)
- return mvif->cap.vht_su_ebfee &&
+ return vif->bss_conf.vht_su_beamformee &&
(cap & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE);
else
- return mvif->cap.vht_su_ebfer &&
+ return vif->bss_conf.vht_su_beamformer &&
(cap & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE);
}
@@ -1530,7 +1527,7 @@ mt7915_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7915_dev *dev,
cap |= STA_CAP_TX_STBC;
if (sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_RX_STBC)
cap |= STA_CAP_RX_STBC;
- if (mvif->cap.ht_ldpc &&
+ if (vif->bss_conf.ht_ldpc &&
(sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_LDPC_CODING))
cap |= STA_CAP_LDPC;
@@ -1556,7 +1553,7 @@ mt7915_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7915_dev *dev,
cap |= STA_CAP_VHT_TX_STBC;
if (sta->deflink.vht_cap.cap & IEEE80211_VHT_CAP_RXSTBC_1)
cap |= STA_CAP_VHT_RX_STBC;
- if (mvif->cap.vht_ldpc &&
+ if (vif->bss_conf.vht_ldpc &&
(sta->deflink.vht_cap.cap & IEEE80211_VHT_CAP_RXLDPC))
cap |= STA_CAP_VHT_LDPC;
@@ -1657,8 +1654,8 @@ int mt7915_mcu_add_sta(struct mt7915_dev *dev, struct ieee80211_vif *vif,
return PTR_ERR(skb);
/* starec basic */
- mt76_connac_mcu_sta_basic_tlv(skb, vif, sta, enable,
- !rcu_access_pointer(dev->mt76.wcid[msta->wcid.idx]));
+ mt76_connac_mcu_sta_basic_tlv(&dev->mt76, skb, vif, sta, enable,
+ !rcu_access_pointer(dev->mt76.wcid[msta->wcid.idx]));
if (!enable)
goto out;
@@ -1876,84 +1873,6 @@ mt7915_mcu_beacon_cont(struct mt7915_dev *dev, struct ieee80211_vif *vif,
}
static void
-mt7915_mcu_beacon_check_caps(struct mt7915_phy *phy, struct ieee80211_vif *vif,
- struct sk_buff *skb)
-{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- struct mt7915_vif_cap *vc = &mvif->cap;
- const struct ieee80211_he_cap_elem *he;
- const struct ieee80211_vht_cap *vht;
- const struct ieee80211_ht_cap *ht;
- struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)skb->data;
- const u8 *ie;
- u32 len, bc;
-
- /* Check missing configuration options to allow AP mode in mac80211
- * to remain in sync with hostapd settings, and get a subset of
- * beacon and hardware capabilities.
- */
- if (WARN_ON_ONCE(skb->len <= (mgmt->u.beacon.variable - skb->data)))
- return;
-
- memset(vc, 0, sizeof(*vc));
-
- len = skb->len - (mgmt->u.beacon.variable - skb->data);
-
- ie = cfg80211_find_ie(WLAN_EID_HT_CAPABILITY, mgmt->u.beacon.variable,
- len);
- if (ie && ie[1] >= sizeof(*ht)) {
- ht = (void *)(ie + 2);
- vc->ht_ldpc = !!(le16_to_cpu(ht->cap_info) &
- IEEE80211_HT_CAP_LDPC_CODING);
- }
-
- ie = cfg80211_find_ie(WLAN_EID_VHT_CAPABILITY, mgmt->u.beacon.variable,
- len);
- if (ie && ie[1] >= sizeof(*vht)) {
- u32 pc = phy->mt76->sband_5g.sband.vht_cap.cap;
-
- vht = (void *)(ie + 2);
- bc = le32_to_cpu(vht->vht_cap_info);
-
- vc->vht_ldpc = !!(bc & IEEE80211_VHT_CAP_RXLDPC);
- vc->vht_su_ebfer =
- (bc & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE);
- vc->vht_su_ebfee =
- (bc & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE);
- vc->vht_mu_ebfer =
- (bc & IEEE80211_VHT_CAP_MU_BEAMFORMER_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_MU_BEAMFORMER_CAPABLE);
- vc->vht_mu_ebfee =
- (bc & IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE);
- }
-
- ie = cfg80211_find_ext_ie(WLAN_EID_EXT_HE_CAPABILITY,
- mgmt->u.beacon.variable, len);
- if (ie && ie[1] >= sizeof(*he) + 1) {
- const struct ieee80211_sta_he_cap *pc =
- mt76_connac_get_he_phy_cap(phy->mt76, vif);
- const struct ieee80211_he_cap_elem *pe = &pc->he_cap_elem;
-
- he = (void *)(ie + 3);
-
- vc->he_ldpc =
- HE_PHY(CAP1_LDPC_CODING_IN_PAYLOAD, pe->phy_cap_info[1]);
- vc->he_su_ebfer =
- HE_PHY(CAP3_SU_BEAMFORMER, he->phy_cap_info[3]) &&
- HE_PHY(CAP3_SU_BEAMFORMER, pe->phy_cap_info[3]);
- vc->he_su_ebfee =
- HE_PHY(CAP4_SU_BEAMFORMEE, he->phy_cap_info[4]) &&
- HE_PHY(CAP4_SU_BEAMFORMEE, pe->phy_cap_info[4]);
- vc->he_mu_ebfer =
- HE_PHY(CAP4_MU_BEAMFORMER, he->phy_cap_info[4]) &&
- HE_PHY(CAP4_MU_BEAMFORMER, pe->phy_cap_info[4]);
- }
-}
-
-static void
mt7915_mcu_beacon_inband_discov(struct mt7915_dev *dev, struct ieee80211_vif *vif,
struct sk_buff *rskb, struct bss_info_bcn *bcn,
u32 changed)
@@ -2063,8 +1982,6 @@ int mt7915_mcu_add_beacon(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
info = IEEE80211_SKB_CB(skb);
info->hw_queue = FIELD_PREP(MT_TX_HW_QUEUE_PHY, ext_phy);
- mt7915_mcu_beacon_check_caps(phy, vif, skb);
-
mt7915_mcu_beacon_cntdwn(vif, rskb, skb, bcn, &offs);
mt7915_mcu_beacon_mbss(rskb, skb, vif, bcn, &offs);
mt7915_mcu_beacon_cont(dev, vif, rskb, skb, bcn, &offs);
@@ -2370,7 +2287,9 @@ int mt7915_mcu_init_firmware(struct mt7915_dev *dev)
if (ret)
return ret;
- if (mtk_wed_device_active(&dev->mt76.mmio.wed) && is_mt7915(&dev->mt76))
+ if ((mtk_wed_device_active(&dev->mt76.mmio.wed) &&
+ is_mt7915(&dev->mt76)) ||
+ !mtk_wed_get_rx_capa(&dev->mt76.mmio.wed))
mt7915_mcu_wa_cmd(dev, MCU_WA_PARAM_CMD(CAPABILITY), 0, 0, 0);
ret = mt7915_mcu_set_mwds(dev, 1);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c
index 225a196..45f3558b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mmio.c
@@ -916,7 +916,7 @@ static void mt7915_rx_poll_complete(struct mt76_dev *mdev,
/* TODO: support 2/4/6/8 MSI-X vectors */
static void mt7915_irq_tasklet(struct tasklet_struct *t)
{
- struct mt7915_dev *dev = from_tasklet(dev, t, irq_tasklet);
+ struct mt7915_dev *dev = from_tasklet(dev, t, mt76.irq_tasklet);
struct mtk_wed_device *wed = &dev->mt76.mmio.wed;
u32 intr, intr1, mask;
@@ -989,18 +989,18 @@ irqreturn_t mt7915_irq_handler(int irq, void *dev_instance)
struct mt7915_dev *dev = dev_instance;
struct mtk_wed_device *wed = &dev->mt76.mmio.wed;
- if (mtk_wed_device_active(wed)) {
+ if (mtk_wed_device_active(wed))
mtk_wed_device_irq_set_mask(wed, 0);
- } else {
+ else
mt76_wr(dev, MT_INT_MASK_CSR, 0);
- if (dev->hif2)
- mt76_wr(dev, MT_INT1_MASK_CSR, 0);
- }
+
+ if (dev->hif2)
+ mt76_wr(dev, MT_INT1_MASK_CSR, 0);
if (!test_bit(MT76_STATE_INITIALIZED, &dev->mphy.state))
return IRQ_NONE;
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
return IRQ_HANDLED;
}
@@ -1022,7 +1022,6 @@ struct mt7915_dev *mt7915_mmio_probe(struct device *pdev,
.rx_skb = mt7915_queue_rx_skb,
.rx_check = mt7915_rx_check,
.rx_poll_complete = mt7915_rx_poll_complete,
- .sta_ps = mt7915_sta_ps,
.sta_add = mt7915_mac_sta_add,
.sta_remove = mt7915_mac_sta_remove,
.update_survey = mt7915_update_channel,
@@ -1041,7 +1040,7 @@ struct mt7915_dev *mt7915_mmio_probe(struct device *pdev,
if (ret)
goto error;
- tasklet_setup(&dev->irq_tasklet, mt7915_irq_tasklet);
+ tasklet_setup(&mdev->irq_tasklet, mt7915_irq_tasklet);
return dev;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
index 3cbfb9b..b3ead35 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
@@ -147,23 +147,9 @@ struct mt7915_sta {
} twt;
};
-struct mt7915_vif_cap {
- bool ht_ldpc:1;
- bool vht_ldpc:1;
- bool he_ldpc:1;
- bool vht_su_ebfer:1;
- bool vht_su_ebfee:1;
- bool vht_mu_ebfer:1;
- bool vht_mu_ebfee:1;
- bool he_su_ebfer:1;
- bool he_su_ebfee:1;
- bool he_mu_ebfer:1;
-};
-
struct mt7915_vif {
struct mt76_vif mt76; /* must be first */
- struct mt7915_vif_cap cap;
struct mt7915_sta sta;
struct mt7915_phy *phy;
@@ -308,7 +294,6 @@ struct mt7915_dev {
u32 wfdma_mask;
const struct mt76_bus_ops *bus_ops;
- struct tasklet_struct irq_tasklet;
struct mt7915_phy phy;
/* monitor rx chain configured channel */
@@ -581,7 +566,7 @@ static inline void mt7915_irq_enable(struct mt7915_dev *dev, u32 mask)
else
mt76_set_irq_mask(&dev->mt76, 0, 0, mask);
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
}
static inline void mt7915_irq_disable(struct mt7915_dev *dev, u32 mask)
@@ -631,7 +616,6 @@ void mt7915_tx_token_put(struct mt7915_dev *dev);
void mt7915_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb, u32 *info);
bool mt7915_rx_check(struct mt76_dev *mdev, void *data, int len);
-void mt7915_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps);
void mt7915_stats_work(struct work_struct *work);
int mt76_dfs_start_rdd(struct mt7915_dev *dev, bool force);
int mt7915_dfs_init_radar_detector(struct mt7915_phy *phy);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/soc.c b/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
index 2ac0a0f..32c1370 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
@@ -1239,6 +1239,8 @@ static const struct of_device_id mt7986_wmac_of_match[] = {
{},
};
+MODULE_DEVICE_TABLE(of, mt7986_wmac_of_match);
+
struct platform_driver mt7986_wmac_driver = {
.driver = {
.name = "mt7986-wmac",
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.h b/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.h
index 35268b0..6f2c4a57 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/acpi_sar.h
@@ -24,7 +24,7 @@ struct mt7921_asar_dyn {
u8 names[4];
u8 enable;
u8 nr_tbl;
- struct mt7921_asar_dyn_limit tbl[0];
+ DECLARE_FLEX_ARRAY(struct mt7921_asar_dyn_limit, tbl);
} __packed;
struct mt7921_asar_dyn_limit_v2 {
@@ -37,7 +37,7 @@ struct mt7921_asar_dyn_v2 {
u8 enable;
u8 rsvd;
u8 nr_tbl;
- struct mt7921_asar_dyn_limit_v2 tbl[0];
+ DECLARE_FLEX_ARRAY(struct mt7921_asar_dyn_limit_v2, tbl);
} __packed;
struct mt7921_asar_geo_band {
@@ -55,7 +55,7 @@ struct mt7921_asar_geo {
u8 names[4];
u8 version;
u8 nr_tbl;
- struct mt7921_asar_geo_limit tbl[0];
+ DECLARE_FLEX_ARRAY(struct mt7921_asar_geo_limit, tbl);
} __packed;
struct mt7921_asar_geo_limit_v2 {
@@ -69,7 +69,7 @@ struct mt7921_asar_geo_v2 {
u8 version;
u8 rsvd;
u8 nr_tbl;
- struct mt7921_asar_geo_limit_v2 tbl[0];
+ DECLARE_FLEX_ARRAY(struct mt7921_asar_geo_limit_v2, tbl);
} __packed;
struct mt7921_asar_cl {
@@ -85,7 +85,7 @@ struct mt7921_asar_fg {
u8 rsvd;
u8 nr_flag;
u8 rsvd1;
- u8 flag[0];
+ u8 flag[];
} __packed;
struct mt7921_acpi_sar {
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
index 29d8883..d6b6edb 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c
@@ -2,7 +2,6 @@
/* Copyright (C) 2020 MediaTek Inc. */
#include "mt7921.h"
-#include "eeprom.h"
static int
mt7921_reg_set(void *data, u64 val)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/dma.c b/drivers/net/wireless/mediatek/mt76/mt7921/dma.c
index d1f10f6..f0a80c2b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/dma.c
@@ -3,7 +3,7 @@
#include "mt7921.h"
#include "../dma.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
static int mt7921_poll_tx(struct napi_struct *napi, int budget)
{
@@ -19,7 +19,7 @@ static int mt7921_poll_tx(struct napi_struct *napi, int budget)
mt76_connac_tx_cleanup(&dev->mt76);
if (napi_complete(napi))
- mt7921_irq_enable(dev, MT_INT_TX_DONE_ALL);
+ mt76_connac_irq_enable(&dev->mt76, MT_INT_TX_DONE_ALL);
mt76_connac_pm_unref(&dev->mphy, &dev->pm);
return 0;
@@ -66,6 +66,24 @@ static void mt7921_dma_prefetch(struct mt7921_dev *dev)
static int mt7921_dma_disable(struct mt7921_dev *dev, bool force)
{
+ /* disable WFDMA0 */
+ mt76_clear(dev, MT_WFDMA0_GLO_CFG,
+ MT_WFDMA0_GLO_CFG_TX_DMA_EN | MT_WFDMA0_GLO_CFG_RX_DMA_EN |
+ MT_WFDMA0_GLO_CFG_CSR_DISP_BASE_PTR_CHAIN_EN |
+ MT_WFDMA0_GLO_CFG_OMIT_TX_INFO |
+ MT_WFDMA0_GLO_CFG_OMIT_RX_INFO |
+ MT_WFDMA0_GLO_CFG_OMIT_RX_INFO_PFET2);
+
+ if (!mt76_poll_msec_tick(dev, MT_WFDMA0_GLO_CFG,
+ MT_WFDMA0_GLO_CFG_TX_DMA_BUSY |
+ MT_WFDMA0_GLO_CFG_RX_DMA_BUSY, 0, 100, 1))
+ return -ETIMEDOUT;
+
+ /* disable dmashdl */
+ mt76_clear(dev, MT_WFDMA0_GLO_CFG_EXT0,
+ MT_WFDMA0_CSR_TX_DMASHDL_ENABLE);
+ mt76_set(dev, MT_DMASHDL_SW_CONTROL, MT_DMASHDL_DMASHDL_BYPASS);
+
if (force) {
/* reset */
mt76_clear(dev, MT_WFDMA0_RST,
@@ -77,24 +95,6 @@ static int mt7921_dma_disable(struct mt7921_dev *dev, bool force)
MT_WFDMA0_RST_LOGIC_RST);
}
- /* disable dmashdl */
- mt76_clear(dev, MT_WFDMA0_GLO_CFG_EXT0,
- MT_WFDMA0_CSR_TX_DMASHDL_ENABLE);
- mt76_set(dev, MT_DMASHDL_SW_CONTROL, MT_DMASHDL_DMASHDL_BYPASS);
-
- /* disable WFDMA0 */
- mt76_clear(dev, MT_WFDMA0_GLO_CFG,
- MT_WFDMA0_GLO_CFG_TX_DMA_EN | MT_WFDMA0_GLO_CFG_RX_DMA_EN |
- MT_WFDMA0_GLO_CFG_CSR_DISP_BASE_PTR_CHAIN_EN |
- MT_WFDMA0_GLO_CFG_OMIT_TX_INFO |
- MT_WFDMA0_GLO_CFG_OMIT_RX_INFO |
- MT_WFDMA0_GLO_CFG_OMIT_RX_INFO_PFET2);
-
- if (!mt76_poll(dev, MT_WFDMA0_GLO_CFG,
- MT_WFDMA0_GLO_CFG_TX_DMA_BUSY |
- MT_WFDMA0_GLO_CFG_RX_DMA_BUSY, 0, 1000))
- return -ETIMEDOUT;
-
return 0;
}
@@ -123,9 +123,9 @@ static int mt7921_dma_enable(struct mt7921_dev *dev)
mt76_set(dev, MT_WFDMA_DUMMY_CR, MT_WFDMA_NEED_REINIT);
/* enable interrupts for TX/RX rings */
- mt7921_irq_enable(dev,
- MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_ALL |
- MT_INT_MCU_CMD);
+ mt76_connac_irq_enable(&dev->mt76,
+ MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_ALL |
+ MT_INT_MCU_CMD);
mt76_set(dev, MT_MCU2HOST_SW_INT_ENA, MT_MCU_CMD_WAKE_RX_PCIE);
return 0;
@@ -301,6 +301,10 @@ void mt7921_dma_cleanup(struct mt7921_dev *dev)
MT_WFDMA0_GLO_CFG_OMIT_RX_INFO |
MT_WFDMA0_GLO_CFG_OMIT_RX_INFO_PFET2);
+ mt76_poll_msec_tick(dev, MT_WFDMA0_GLO_CFG,
+ MT_WFDMA0_GLO_CFG_TX_DMA_BUSY |
+ MT_WFDMA0_GLO_CFG_RX_DMA_BUSY, 0, 100, 1);
+
/* reset */
mt76_clear(dev, MT_WFDMA0_RST,
MT_WFDMA0_RST_DMASHDL_ALL_RST |
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h b/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h
deleted file mode 100644
index 4b64727..0000000
--- a/drivers/net/wireless/mediatek/mt76/mt7921/eeprom.h
+++ /dev/null
@@ -1,30 +0,0 @@
-/* SPDX-License-Identifier: ISC */
-/* Copyright (C) 2020 MediaTek Inc. */
-
-#ifndef __MT7921_EEPROM_H
-#define __MT7921_EEPROM_H
-
-#include "mt7921.h"
-
-enum mt7921_eeprom_field {
- MT_EE_CHIP_ID = 0x000,
- MT_EE_VERSION = 0x002,
- MT_EE_MAC_ADDR = 0x004,
- MT_EE_WIFI_CONF = 0x07c,
- MT_EE_HW_TYPE = 0x55b,
- __MT_EE_MAX = 0x9ff
-};
-
-#define MT_EE_WIFI_CONF_TX_MASK BIT(0)
-#define MT_EE_WIFI_CONF_BAND_SEL GENMASK(3, 2)
-
-#define MT_EE_HW_TYPE_ENCAP BIT(0)
-
-enum mt7921_eeprom_band {
- MT_EE_NA,
- MT_EE_5GHZ,
- MT_EE_2GHZ,
- MT_EE_DUAL_BAND,
-};
-
-#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
index cc94531..bf1da9f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
@@ -4,9 +4,8 @@
#include <linux/etherdevice.h>
#include <linux/firmware.h>
#include "mt7921.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "mcu.h"
-#include "eeprom.h"
static const struct ieee80211_iface_limit if_limits[] = {
{
@@ -32,11 +31,13 @@ static const struct ieee80211_iface_combination if_comb[] = {
static const struct ieee80211_iface_limit if_limits_chanctx[] = {
{
.max = 2,
- .types = BIT(NL80211_IFTYPE_STATION),
+ .types = BIT(NL80211_IFTYPE_STATION) |
+ BIT(NL80211_IFTYPE_P2P_CLIENT)
},
{
.max = 1,
- .types = BIT(NL80211_IFTYPE_AP),
+ .types = BIT(NL80211_IFTYPE_AP) |
+ BIT(NL80211_IFTYPE_P2P_GO)
}
};
@@ -100,7 +101,9 @@ mt7921_init_wiphy(struct ieee80211_hw *hw)
wiphy->flags &= ~(WIPHY_FLAG_IBSS_RSN | WIPHY_FLAG_4ADDR_AP |
WIPHY_FLAG_4ADDR_STATION);
wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) |
- BIT(NL80211_IFTYPE_AP);
+ BIT(NL80211_IFTYPE_AP) |
+ BIT(NL80211_IFTYPE_P2P_CLIENT) |
+ BIT(NL80211_IFTYPE_P2P_GO);
wiphy->max_remain_on_channel_duration = 5000;
wiphy->max_scan_ie_len = MT76_CONNAC_SCAN_IE_LEN;
wiphy->max_scan_ssids = 4;
@@ -121,6 +124,7 @@ mt7921_init_wiphy(struct ieee80211_hw *hw)
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_BEACON_RATE_VHT);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_BEACON_RATE_HE);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_ACK_SIGNAL_SUPPORT);
+ wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
ieee80211_hw_set(hw, SINGLE_SCAN_ON_ALL_BANDS);
ieee80211_hw_set(hw, HAS_RATE_CONTROL);
@@ -169,7 +173,8 @@ mt7921_mac_init_band(struct mt7921_dev *dev, u8 band)
mt76_rmw(dev, MT_WTBLOFF_TOP_RSCR(band), mask, set);
}
-u8 mt7921_check_offload_capability(struct device *dev, const char *fw_wm)
+static u8
+mt7921_get_offload_capability(struct device *dev, const char *fw_wm)
{
const struct mt76_connac2_fw_trailer *hdr;
struct mt7921_realease_info *rel_info;
@@ -223,7 +228,31 @@ u8 mt7921_check_offload_capability(struct device *dev, const char *fw_wm)
return offload_caps;
}
-EXPORT_SYMBOL_GPL(mt7921_check_offload_capability);
+
+struct ieee80211_ops *
+mt7921_get_mac80211_ops(struct device *dev, void *drv_data, u8 *fw_features)
+{
+ struct ieee80211_ops *ops;
+
+ ops = devm_kmemdup(dev, &mt7921_ops, sizeof(mt7921_ops), GFP_KERNEL);
+ if (!ops)
+ return NULL;
+
+ *fw_features = mt7921_get_offload_capability(dev, drv_data);
+ if (!(*fw_features & MT7921_FW_CAP_CNM)) {
+ ops->remain_on_channel = NULL;
+ ops->cancel_remain_on_channel = NULL;
+ ops->add_chanctx = NULL;
+ ops->remove_chanctx = NULL;
+ ops->change_chanctx = NULL;
+ ops->assign_vif_chanctx = NULL;
+ ops->unassign_vif_chanctx = NULL;
+ ops->mgd_prepare_tx = NULL;
+ ops->mgd_complete_tx = NULL;
+ }
+ return ops;
+}
+EXPORT_SYMBOL_GPL(mt7921_get_mac80211_ops);
int mt7921_mac_init(struct mt7921_dev *dev)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
index 557c201..1675bf5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
@@ -6,9 +6,20 @@
#include <linux/timekeeping.h>
#include "mt7921.h"
#include "../dma.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "mcu.h"
+#define MT_WTBL_TXRX_CAP_RATE_OFFSET 7
+#define MT_WTBL_TXRX_RATE_G2_HE 24
+#define MT_WTBL_TXRX_RATE_G2 12
+
+#define MT_WTBL_AC0_CTT_OFFSET 20
+
+static u32 mt7921_mac_wtbl_lmac_addr(int idx, u8 offset)
+{
+ return MT_WTBL_LMAC_OFFS(idx, 0) + offset * 4;
+}
+
static struct mt76_wcid *mt7921_rx_get_wcid(struct mt7921_dev *dev,
u16 idx, bool unicast)
{
@@ -32,11 +43,6 @@ static struct mt76_wcid *mt7921_rx_get_wcid(struct mt7921_dev *dev,
return &sta->vif->sta.wcid;
}
-void mt7921_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps)
-{
-}
-EXPORT_SYMBOL_GPL(mt7921_sta_ps);
-
bool mt7921_mac_wtbl_update(struct mt7921_dev *dev, int idx, u32 mask)
{
mt76_rmw(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_WLAN_IDX,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.h b/drivers/net/wireless/mediatek/mt76/mt7921/mac.h
deleted file mode 100644
index 8afec60..0000000
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/* SPDX-License-Identifier: ISC */
-/* Copyright (C) 2020 MediaTek Inc. */
-
-#ifndef __MT7921_MAC_H
-#define __MT7921_MAC_H
-
-#include "../mt76_connac2_mac.h"
-
-#define MT_CT_PARSE_LEN 72
-#define MT_CT_DMA_BUF_NUM 2
-
-#define MT_RXD0_LENGTH GENMASK(15, 0)
-#define MT_RXD0_PKT_FLAG GENMASK(19, 16)
-#define MT_RXD0_PKT_TYPE GENMASK(31, 27)
-
-#define MT_RXD0_NORMAL_ETH_TYPE_OFS GENMASK(22, 16)
-#define MT_RXD0_NORMAL_IP_SUM BIT(23)
-#define MT_RXD0_NORMAL_UDP_TCP_SUM BIT(24)
-
-enum rx_pkt_type {
- PKT_TYPE_TXS,
- PKT_TYPE_TXRXV,
- PKT_TYPE_NORMAL,
- PKT_TYPE_RX_DUP_RFB,
- PKT_TYPE_RX_TMR,
- PKT_TYPE_RETRIEVE,
- PKT_TYPE_TXRX_NOTIFY,
- PKT_TYPE_RX_EVENT,
- PKT_TYPE_NORMAL_MCU,
-};
-
-#define MT_TX_FREE_MSDU_CNT GENMASK(9, 0)
-#define MT_TX_FREE_WLAN_ID GENMASK(23, 14)
-#define MT_TX_FREE_LATENCY GENMASK(12, 0)
-/* 0: success, others: dropped */
-#define MT_TX_FREE_STATUS GENMASK(14, 13)
-#define MT_TX_FREE_MSDU_ID GENMASK(30, 16)
-#define MT_TX_FREE_PAIR BIT(31)
-/* will support this field in further revision */
-#define MT_TX_FREE_RATE GENMASK(13, 0)
-
-#define MT_WTBL_TXRX_CAP_RATE_OFFSET 7
-#define MT_WTBL_TXRX_RATE_G2_HE 24
-#define MT_WTBL_TXRX_RATE_G2 12
-
-#define MT_WTBL_AC0_CTT_OFFSET 20
-
-static inline u32 mt7921_mac_wtbl_lmac_addr(int idx, u8 offset)
-{
- return MT_WTBL_LMAC_OFFS(idx, 0) + offset * 4;
-}
-
-#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index 42933a6..3b6adb2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -9,27 +9,6 @@
#include "mt7921.h"
#include "mcu.h"
-static void
-mt7921_gen_ppe_thresh(u8 *he_ppet, int nss)
-{
- u8 i, ppet_bits, ppet_size, ru_bit_mask = 0x7; /* HE80 */
- static const u8 ppet16_ppet8_ru3_ru0[] = {0x1c, 0xc7, 0x71};
-
- he_ppet[0] = FIELD_PREP(IEEE80211_PPE_THRES_NSS_MASK, nss - 1) |
- FIELD_PREP(IEEE80211_PPE_THRES_RU_INDEX_BITMASK_MASK,
- ru_bit_mask);
-
- ppet_bits = IEEE80211_PPE_THRES_INFO_PPET_SIZE *
- nss * hweight8(ru_bit_mask) * 2;
- ppet_size = DIV_ROUND_UP(ppet_bits, 8);
-
- for (i = 0; i < ppet_size - 1; i++)
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3];
-
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3] &
- (0xff >> (8 - (ppet_bits - 1) % 8));
-}
-
static int
mt7921_init_he_caps(struct mt7921_phy *phy, enum nl80211_band band,
struct ieee80211_sband_iftype_data *data)
@@ -168,7 +147,7 @@ mt7921_init_he_caps(struct mt7921_phy *phy, enum nl80211_band band,
memset(he_cap->ppe_thres, 0, sizeof(he_cap->ppe_thres));
if (he_cap_elem->phy_cap_info[6] &
IEEE80211_HE_PHY_CAP6_PPE_THRESHOLD_PRESENT) {
- mt7921_gen_ppe_thresh(he_cap->ppe_thres, nss);
+ mt76_connac_gen_ppe_thresh(he_cap->ppe_thres, nss);
} else {
he_cap_elem->phy_cap_info[9] |=
u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
@@ -702,10 +681,25 @@ static void mt7921_configure_filter(struct ieee80211_hw *hw,
unsigned int *total_flags,
u64 multicast)
{
+#define MT7921_FILTER_FCSFAIL BIT(2)
+#define MT7921_FILTER_CONTROL BIT(5)
+#define MT7921_FILTER_OTHER_BSS BIT(6)
+#define MT7921_FILTER_ENABLE BIT(31)
+
struct mt7921_dev *dev = mt7921_hw_dev(hw);
+ u32 flags = MT7921_FILTER_ENABLE;
+
+#define MT7921_FILTER(_fif, _type) do { \
+ if (*total_flags & (_fif)) \
+ flags |= MT7921_FILTER_##_type; \
+ } while (0)
+
+ MT7921_FILTER(FIF_FCSFAIL, FCSFAIL);
+ MT7921_FILTER(FIF_CONTROL, CONTROL);
+ MT7921_FILTER(FIF_OTHER_BSS, OTHER_BSS);
mt7921_mutex_acquire(dev);
- mt7921_mcu_set_rxfilter(dev, *total_flags, 0, 0);
+ mt7921_mcu_set_rxfilter(dev, flags, 0, 0);
mt7921_mutex_release(dev);
*total_flags &= (FIF_OTHER_BSS | FIF_FCSFAIL | FIF_CONTROL);
@@ -1694,7 +1688,7 @@ static void mt7921_ctx_iter(void *priv, u8 *mac,
if (ctx != mvif->ctx)
return;
- if (vif->type & NL80211_IFTYPE_MONITOR)
+ if (vif->type == NL80211_IFTYPE_MONITOR)
mt7921_mcu_config_sniffer(mvif, ctx);
else
mt76_connac_mcu_uni_set_chctx(mvif->phy->mt76, &mvif->mt76, ctx);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
index c5e7ad0..c69ce6d 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
@@ -5,9 +5,8 @@
#include <linux/firmware.h>
#include "mt7921.h"
#include "mt7921_trace.h"
-#include "eeprom.h"
#include "mcu.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#define MT_STA_BFER BIT(0)
#define MT_STA_BFEE BIT(1)
@@ -16,24 +15,6 @@ static bool mt7921_disable_clc;
module_param_named(disable_clc, mt7921_disable_clc, bool, 0644);
MODULE_PARM_DESC(disable_clc, "disable CLC support");
-static int
-mt7921_mcu_parse_eeprom(struct mt76_dev *dev, struct sk_buff *skb)
-{
- struct mt7921_mcu_eeprom_info *res;
- u8 *buf;
-
- if (!skb)
- return -EINVAL;
-
- skb_pull(skb, sizeof(struct mt76_connac2_mcu_rxd));
-
- res = (struct mt7921_mcu_eeprom_info *)skb->data;
- buf = dev->eeprom.data + le32_to_cpu(res->addr);
- memcpy(buf, res->data, 16);
-
- return 0;
-}
-
int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd,
struct sk_buff *skb, int seq)
{
@@ -60,27 +41,25 @@ int mt7921_mcu_parse_response(struct mt76_dev *mdev, int cmd,
} else if (cmd == MCU_EXT_CMD(THERMAL_CTRL)) {
skb_pull(skb, sizeof(*rxd) + 4);
ret = le32_to_cpu(*(__le32 *)skb->data);
- } else if (cmd == MCU_EXT_CMD(EFUSE_ACCESS)) {
- ret = mt7921_mcu_parse_eeprom(mdev, skb);
} else if (cmd == MCU_UNI_CMD(DEV_INFO_UPDATE) ||
cmd == MCU_UNI_CMD(BSS_INFO_UPDATE) ||
cmd == MCU_UNI_CMD(STA_REC_UPDATE) ||
cmd == MCU_UNI_CMD(HIF_CTRL) ||
cmd == MCU_UNI_CMD(OFFLOAD) ||
cmd == MCU_UNI_CMD(SUSPEND)) {
- struct mt7921_mcu_uni_event *event;
+ struct mt76_connac_mcu_uni_event *event;
skb_pull(skb, sizeof(*rxd));
- event = (struct mt7921_mcu_uni_event *)skb->data;
+ event = (struct mt76_connac_mcu_uni_event *)skb->data;
ret = le32_to_cpu(event->status);
/* skip invalid event */
if (mcu_cmd != event->cid)
ret = -EAGAIN;
} else if (cmd == MCU_CE_QUERY(REG_READ)) {
- struct mt7921_mcu_reg_event *event;
+ struct mt76_connac_mcu_reg_event *event;
skb_pull(skb, sizeof(*rxd));
- event = (struct mt7921_mcu_reg_event *)skb->data;
+ event = (struct mt76_connac_mcu_reg_event *)skb->data;
ret = (int)le32_to_cpu(event->val);
} else {
skb_pull(skb, sizeof(struct mt76_connac2_mcu_rxd));
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
index 96dc870..9b0aa3b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.h
@@ -50,22 +50,11 @@ struct mt7921_mcu_eeprom_info {
#define MT_RA_RATE_DCM_EN BIT(4)
#define MT_RA_RATE_BW GENMASK(14, 13)
-struct mt7921_mcu_uni_event {
- u8 cid;
- u8 pad[3];
- __le32 status; /* 0: success, others: fail */
-} __packed;
-
enum {
MT_EBF = BIT(0), /* explicit beamforming */
MT_IBF = BIT(1) /* implicit beamforming */
};
-struct mt7921_mcu_reg_event {
- __le32 reg;
- __le32 val;
-} __packed;
-
struct mt7921_mcu_ant_id_config {
u8 ant_id[4];
} __packed;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
index 1af70da..149acb1 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
@@ -266,6 +266,17 @@ struct mt7921_phy {
bool roc_grant;
};
+enum mt7921_eeprom_field {
+ MT_EE_CHIP_ID = 0x000,
+ MT_EE_VERSION = 0x002,
+ MT_EE_MAC_ADDR = 0x004,
+ MT_EE_WIFI_CONF = 0x07c,
+ MT_EE_HW_TYPE = 0x55b,
+ __MT_EE_MAX = 0x9ff
+};
+
+#define MT_EE_HW_TYPE_ENCAP BIT(0)
+
#define mt7921_init_reset(dev) ((dev)->hif_ops->init_reset(dev))
#define mt7921_dev_reset(dev) ((dev)->hif_ops->reset(dev))
#define mt7921_mcu_init(dev) ((dev)->hif_ops->mcu_init(dev))
@@ -287,7 +298,6 @@ struct mt7921_dev {
const struct mt76_bus_ops *bus_ops;
struct mt7921_phy phy;
- struct tasklet_struct irq_tasklet;
struct work_struct reset_work;
bool hw_full_reset:1;
@@ -391,13 +401,6 @@ void mt7921_mcu_rx_event(struct mt7921_dev *dev, struct sk_buff *skb);
int mt7921_mcu_set_rxfilter(struct mt7921_dev *dev, u32 fif,
u8 bit_op, u32 bit_map);
-static inline void mt7921_irq_enable(struct mt7921_dev *dev, u32 mask)
-{
- mt76_set_irq_mask(&dev->mt76, 0, 0, mask);
-
- tasklet_schedule(&dev->irq_tasklet);
-}
-
static inline u32
mt7921_reg_map_l1(struct mt7921_dev *dev, u32 addr)
{
@@ -478,7 +481,6 @@ void mt7921_tx_token_put(struct mt7921_dev *dev);
bool mt7921_rx_check(struct mt76_dev *mdev, void *data, int len);
void mt7921_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb, u32 *info);
-void mt7921_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps);
void mt7921_stats_work(struct work_struct *work);
void mt7921_set_stream_he_caps(struct mt7921_phy *phy);
void mt7921_update_channel(struct mt76_phy *mphy);
@@ -593,5 +595,6 @@ int mt7921_mcu_set_roc(struct mt7921_phy *phy, struct mt7921_vif *vif,
enum mt7921_roc_req type, u8 token_id);
int mt7921_mcu_abort_roc(struct mt7921_phy *phy, struct mt7921_vif *vif,
u8 token_id);
-u8 mt7921_check_offload_capability(struct device *dev, const char *fw_wm);
+struct ieee80211_ops *mt7921_get_mac80211_ops(struct device *dev,
+ void *drv_data, u8 *fw_features);
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
index 5c23c82..ddb1fa4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c
@@ -8,7 +8,7 @@
#include <linux/pci.h>
#include "mt7921.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "mcu.h"
#include "../trace.h"
@@ -31,14 +31,12 @@ MODULE_PARM_DESC(disable_aspm, "disable PCI ASPM support");
static void
mt7921_rx_poll_complete(struct mt76_dev *mdev, enum mt76_rxq_id q)
{
- struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76);
-
if (q == MT_RXQ_MAIN)
- mt7921_irq_enable(dev, MT_INT_RX_DONE_DATA);
+ mt76_connac_irq_enable(mdev, MT_INT_RX_DONE_DATA);
else if (q == MT_RXQ_MCU_WA)
- mt7921_irq_enable(dev, MT_INT_RX_DONE_WM2);
+ mt76_connac_irq_enable(mdev, MT_INT_RX_DONE_WM2);
else
- mt7921_irq_enable(dev, MT_INT_RX_DONE_WM);
+ mt76_connac_irq_enable(mdev, MT_INT_RX_DONE_WM);
}
static irqreturn_t mt7921_irq_handler(int irq, void *dev_instance)
@@ -50,7 +48,7 @@ static irqreturn_t mt7921_irq_handler(int irq, void *dev_instance)
if (!test_bit(MT76_STATE_INITIALIZED, &dev->mphy.state))
return IRQ_NONE;
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
return IRQ_HANDLED;
}
@@ -115,14 +113,15 @@ static void mt7921e_unregister_device(struct mt7921_dev *dev)
napi_disable(&dev->mt76.napi[i]);
cancel_delayed_work_sync(&pm->ps_work);
cancel_work_sync(&pm->wake_work);
+ cancel_work_sync(&dev->reset_work);
mt7921_tx_token_put(dev);
- mt7921_mcu_drv_pmctrl(dev);
+ __mt7921_mcu_drv_pmctrl(dev);
mt7921_dma_cleanup(dev);
mt7921_wfsys_reset(dev);
skb_queue_purge(&dev->mt76.mcu.res_q);
- tasklet_disable(&dev->irq_tasklet);
+ tasklet_disable(&dev->mt76.irq_tasklet);
}
static u32 __mt7921_reg_addr(struct mt7921_dev *dev, u32 addr)
@@ -243,7 +242,6 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
.rx_check = mt7921_rx_check,
.rx_skb = mt7921_queue_rx_skb,
.rx_poll_complete = mt7921_rx_poll_complete,
- .sta_ps = mt7921_sta_ps,
.sta_add = mt7921_mac_sta_add,
.sta_assoc = mt7921_mac_sta_assoc,
.sta_remove = mt7921_mac_sta_remove,
@@ -256,13 +254,13 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
.drv_own = mt7921e_mcu_drv_pmctrl,
.fw_own = mt7921e_mcu_fw_pmctrl,
};
-
struct ieee80211_ops *ops;
struct mt76_bus_ops *bus_ops;
struct mt7921_dev *dev;
struct mt76_dev *mdev;
u8 features;
int ret;
+ u16 cmd;
ret = pcim_enable_device(pdev);
if (ret)
@@ -272,6 +270,11 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
if (ret)
return ret;
+ pci_read_config_word(pdev, PCI_COMMAND, &cmd);
+ if (!(cmd & PCI_COMMAND_MEMORY)) {
+ cmd |= PCI_COMMAND_MEMORY;
+ pci_write_config_word(pdev, PCI_COMMAND, cmd);
+ }
pci_set_master(pdev);
ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES);
@@ -285,27 +288,13 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
if (mt7921_disable_aspm)
mt76_pci_disable_aspm(pdev);
- features = mt7921_check_offload_capability(&pdev->dev, (const char *)
- id->driver_data);
- ops = devm_kmemdup(&pdev->dev, &mt7921_ops, sizeof(mt7921_ops),
- GFP_KERNEL);
+ ops = mt7921_get_mac80211_ops(&pdev->dev, (void *)id->driver_data,
+ &features);
if (!ops) {
ret = -ENOMEM;
goto err_free_pci_vec;
}
- if (!(features & MT7921_FW_CAP_CNM)) {
- ops->remain_on_channel = NULL;
- ops->cancel_remain_on_channel = NULL;
- ops->add_chanctx = NULL;
- ops->remove_chanctx = NULL;
- ops->change_chanctx = NULL;
- ops->assign_vif_chanctx = NULL;
- ops->unassign_vif_chanctx = NULL;
- ops->mgd_prepare_tx = NULL;
- ops->mgd_complete_tx = NULL;
- }
-
mdev = mt76_alloc_device(&pdev->dev, sizeof(*dev), ops, &drv_ops);
if (!mdev) {
ret = -ENOMEM;
@@ -318,7 +307,7 @@ static int mt7921_pci_probe(struct pci_dev *pdev,
dev->fw_features = features;
dev->hif_ops = &mt7921_pcie_ops;
mt76_mmio_init(&dev->mt76, pcim_iomap_table(pdev)[0]);
- tasklet_init(&dev->irq_tasklet, mt7921_irq_tasklet, (unsigned long)dev);
+ tasklet_init(&mdev->irq_tasklet, mt7921_irq_tasklet, (unsigned long)dev);
dev->phy.dev = dev;
dev->phy.mt76 = &dev->mt76.phy;
@@ -430,7 +419,7 @@ static int mt7921_pci_suspend(struct device *device)
mt76_wr(dev, MT_WFDMA0_HOST_INT_ENA, 0);
mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0x0);
synchronize_irq(pdev->irq);
- tasklet_kill(&dev->irq_tasklet);
+ tasklet_kill(&mdev->irq_tasklet);
err = mt7921_mcu_fw_pmctrl(dev);
if (err)
@@ -474,8 +463,9 @@ static int mt7921_pci_resume(struct device *device)
/* enable interrupt */
mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff);
- mt7921_irq_enable(dev, MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_ALL |
- MT_INT_MCU_CMD);
+ mt76_connac_irq_enable(&dev->mt76,
+ MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_ALL |
+ MT_INT_MCU_CMD);
mt76_set(dev, MT_MCU2HOST_SW_INT_ENA, MT_MCU_CMD_WAKE_RX_PCIE);
/* put dma enabled */
@@ -509,17 +499,7 @@ static int mt7921_pci_resume(struct device *device)
static void mt7921_pci_shutdown(struct pci_dev *pdev)
{
- struct mt76_dev *mdev = pci_get_drvdata(pdev);
- struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76);
- struct mt76_connac_pm *pm = &dev->pm;
-
- cancel_delayed_work_sync(&pm->ps_work);
- cancel_work_sync(&pm->wake_work);
-
- /* chip cleanup before reboot */
- mt7921_mcu_drv_pmctrl(dev);
- mt7921_dma_cleanup(dev);
- mt7921_wfsys_reset(dev);
+ mt7921_pci_remove(pdev);
}
static DEFINE_SIMPLE_DEV_PM_OPS(mt7921_pm_ops, mt7921_pci_suspend, mt7921_pci_resume);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
index 8dd6040..6053a25 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci_mac.c
@@ -3,7 +3,7 @@
#include "mt7921.h"
#include "../dma.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
int mt7921e_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
enum mt76_txq_id qid, struct mt76_wcid *wcid,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
index 8ce4252..a77a309 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio.c
@@ -13,7 +13,7 @@
#include "mt7921.h"
#include "../sdio.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "mcu.h"
static const struct sdio_device_id mt7921s_table[] = {
@@ -99,7 +99,6 @@ static int mt7921s_probe(struct sdio_func *func,
.tx_status_data = mt7921_usb_sdio_tx_status_data,
.rx_skb = mt7921_queue_rx_skb,
.rx_check = mt7921_rx_check,
- .sta_ps = mt7921_sta_ps,
.sta_add = mt7921_mac_sta_add,
.sta_assoc = mt7921_mac_sta_assoc,
.sta_remove = mt7921_mac_sta_remove,
@@ -122,33 +121,17 @@ static int mt7921s_probe(struct sdio_func *func,
.drv_own = mt7921s_mcu_drv_pmctrl,
.fw_own = mt7921s_mcu_fw_pmctrl,
};
-
struct ieee80211_ops *ops;
struct mt7921_dev *dev;
struct mt76_dev *mdev;
u8 features;
int ret;
- features = mt7921_check_offload_capability(&func->dev, (const char *)
- id->driver_data);
-
- ops = devm_kmemdup(&func->dev, &mt7921_ops, sizeof(mt7921_ops),
- GFP_KERNEL);
+ ops = mt7921_get_mac80211_ops(&func->dev, (void *)id->driver_data,
+ &features);
if (!ops)
return -ENOMEM;
- if (!(features & MT7921_FW_CAP_CNM)) {
- ops->remain_on_channel = NULL;
- ops->cancel_remain_on_channel = NULL;
- ops->add_chanctx = NULL;
- ops->remove_chanctx = NULL;
- ops->change_chanctx = NULL;
- ops->assign_vif_chanctx = NULL;
- ops->unassign_vif_chanctx = NULL;
- ops->mgd_prepare_tx = NULL;
- ops->mgd_complete_tx = NULL;
- }
-
mdev = mt76_alloc_device(&func->dev, sizeof(*dev), ops, &drv_ops);
if (!mdev)
return -ENOMEM;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
index 1b3adb3..cff9925 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mac.c
@@ -4,7 +4,7 @@
#include <linux/iopoll.h>
#include <linux/mmc/sdio_func.h>
#include "mt7921.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "../sdio.h"
static void mt7921s_enable_irq(struct mt76_dev *dev)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
index 5c14897..177679c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/sdio_mcu.c
@@ -8,7 +8,7 @@
#include "mt7921.h"
#include "../sdio.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
#include "mcu.h"
#include "regs.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/usb.c b/drivers/net/wireless/mediatek/mt76/mt7921/usb.c
index 8fef09e..1f302c4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/usb.c
@@ -10,7 +10,7 @@
#include "mt7921.h"
#include "mcu.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
static const struct usb_device_id mt7921u_device_table[] = {
{ USB_DEVICE_AND_INTERFACE_INFO(0x0e8d, 0x7961, 0xff, 0xff, 0xff),
@@ -18,6 +18,9 @@ static const struct usb_device_id mt7921u_device_table[] = {
/* Comfast CF-952AX */
{ USB_DEVICE_AND_INTERFACE_INFO(0x3574, 0x6211, 0xff, 0xff, 0xff),
.driver_info = (kernel_ulong_t)MT7921_FIRMWARE_WM },
+ /* Netgear, Inc. [A8000,AXE3000] */
+ { USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0x9060, 0xff, 0xff, 0xff),
+ .driver_info = (kernel_ulong_t)MT7921_FIRMWARE_WM },
{ },
};
@@ -183,7 +186,6 @@ static int mt7921u_probe(struct usb_interface *usb_intf,
.tx_status_data = mt7921_usb_sdio_tx_status_data,
.rx_skb = mt7921_queue_rx_skb,
.rx_check = mt7921_rx_check,
- .sta_ps = mt7921_sta_ps,
.sta_add = mt7921_mac_sta_add,
.sta_assoc = mt7921_mac_sta_assoc,
.sta_remove = mt7921_mac_sta_remove,
@@ -210,27 +212,12 @@ static int mt7921u_probe(struct usb_interface *usb_intf,
u8 features;
int ret;
- features = mt7921_check_offload_capability(&usb_intf->dev, (const char *)
- id->driver_info);
- ops = devm_kmemdup(&usb_intf->dev, &mt7921_ops, sizeof(mt7921_ops),
- GFP_KERNEL);
+ ops = mt7921_get_mac80211_ops(&usb_intf->dev, (void *)id->driver_info,
+ &features);
if (!ops)
return -ENOMEM;
- if (!(features & MT7921_FW_CAP_CNM)) {
- ops->remain_on_channel = NULL;
- ops->cancel_remain_on_channel = NULL;
- ops->add_chanctx = NULL;
- ops->remove_chanctx = NULL;
- ops->change_chanctx = NULL;
- ops->assign_vif_chanctx = NULL;
- ops->unassign_vif_chanctx = NULL;
- ops->mgd_prepare_tx = NULL;
- ops->mgd_complete_tx = NULL;
- }
-
ops->stop = mt7921u_stop;
-
mdev = mt76_alloc_device(&usb_intf->dev, sizeof(*dev), ops, &drv_ops);
if (!mdev)
return -ENOMEM;
@@ -272,7 +259,7 @@ static int mt7921u_probe(struct usb_interface *usb_intf,
ret = mt7921u_dma_init(dev, false);
if (ret)
- return ret;
+ goto error;
hw = mt76_hw(dev);
/* check hw sg support in order to enable AMSDU */
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/usb_mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/usb_mac.c
index efbd3954..50eb6e7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/usb_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/usb_mac.c
@@ -10,7 +10,7 @@
#include "mt7921.h"
#include "mcu.h"
-#include "mac.h"
+#include "../mt76_connac2_mac.h"
static u32 mt7921u_uhw_rr(struct mt76_dev *dev, u32 addr)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/Kconfig b/drivers/net/wireless/mediatek/mt76/mt7996/Kconfig
index 79fb47a..1afa2f6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/Kconfig
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/Kconfig
@@ -2,6 +2,7 @@
config MT7996E
tristate "MediaTek MT7996 (PCIe) support"
select MT76_CONNAC_LIB
+ select WANT_DEV_COREDUMP
select RELAY
depends on MAC80211
depends on PCI
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/Makefile b/drivers/net/wireless/mediatek/mt76/mt7996/Makefile
index bcb9a3c..07c8b55 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/Makefile
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/Makefile
@@ -4,3 +4,5 @@
mt7996e-y := pci.o init.o dma.o eeprom.o main.o mcu.o mac.o \
debugfs.o mmio.o
+
+mt7996e-$(CONFIG_DEV_COREDUMP) += coredump.o
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/coredump.c b/drivers/net/wireless/mediatek/mt76/mt7996/coredump.c
new file mode 100644
index 0000000..ccab0d7
--- /dev/null
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/coredump.c
@@ -0,0 +1,268 @@
+// SPDX-License-Identifier: ISC
+/* Copyright (C) 2023 MediaTek Inc. */
+
+#include <linux/devcoredump.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/utsname.h>
+#include "coredump.h"
+
+static bool coredump_memdump;
+module_param(coredump_memdump, bool, 0644);
+MODULE_PARM_DESC(coredump_memdump, "Optional ability to dump firmware memory");
+
+static const struct mt7996_mem_region mt7996_mem_regions[] = {
+ {
+ .start = 0x00800000,
+ .len = 0x0004ffff,
+ .name = "ULM0",
+ },
+ {
+ .start = 0x00900000,
+ .len = 0x00037fff,
+ .name = "ULM1",
+ },
+ {
+ .start = 0x02200000,
+ .len = 0x0003ffff,
+ .name = "ULM2",
+ },
+ {
+ .start = 0x00400000,
+ .len = 0x00067fff,
+ .name = "SRAM",
+ },
+ {
+ .start = 0xe0000000,
+ .len = 0x0015ffff,
+ .name = "CRAM0",
+ },
+ {
+ .start = 0xe0160000,
+ .len = 0x0011bfff,
+ .name = "CRAM1",
+ },
+};
+
+const struct mt7996_mem_region*
+mt7996_coredump_get_mem_layout(struct mt7996_dev *dev, u32 *num)
+{
+ switch (mt76_chip(&dev->mt76)) {
+ case 0x7990:
+ case 0x7991:
+ *num = ARRAY_SIZE(mt7996_mem_regions);
+ return &mt7996_mem_regions[0];
+ default:
+ return NULL;
+ }
+}
+
+static int mt7996_coredump_get_mem_size(struct mt7996_dev *dev)
+{
+ const struct mt7996_mem_region *mem_region;
+ size_t size = 0;
+ u32 num;
+ int i;
+
+ mem_region = mt7996_coredump_get_mem_layout(dev, &num);
+ if (!mem_region)
+ return 0;
+
+ for (i = 0; i < num; i++) {
+ size += mem_region->len;
+ mem_region++;
+ }
+
+ /* reserve space for the headers */
+ size += num * sizeof(struct mt7996_mem_hdr);
+ /* make sure it is aligned 4 bytes for debug message print out */
+ size = ALIGN(size, 4);
+
+ return size;
+}
+
+struct mt7996_crash_data *mt7996_coredump_new(struct mt7996_dev *dev)
+{
+ struct mt7996_crash_data *crash_data = dev->coredump.crash_data;
+
+ lockdep_assert_held(&dev->dump_mutex);
+
+ if (coredump_memdump &&
+ !mt76_poll_msec(dev, MT_FW_DUMP_STATE, 0x3, 0x2, 500))
+ return NULL;
+
+ guid_gen(&crash_data->guid);
+ ktime_get_real_ts64(&crash_data->timestamp);
+
+ return crash_data;
+}
+
+static void
+mt7996_coredump_fw_state(struct mt7996_dev *dev, struct mt7996_coredump *dump,
+ bool *exception)
+{
+ u32 count;
+
+ count = mt76_rr(dev, MT_FW_ASSERT_CNT);
+
+ /* normal mode: driver can manually trigger assert for detail info */
+ if (!count)
+ strscpy(dump->fw_state, "normal", sizeof(dump->fw_state));
+ else
+ strscpy(dump->fw_state, "exception", sizeof(dump->fw_state));
+
+ *exception = !!count;
+}
+
+static void
+mt7996_coredump_fw_stack(struct mt7996_dev *dev, struct mt7996_coredump *dump,
+ bool exception)
+{
+ u32 oldest, i, idx;
+
+ strscpy(dump->pc_current, "program counter", sizeof(dump->pc_current));
+
+ /* 0: WM PC log output */
+ mt76_wr(dev, MT_CONN_DBG_CTL_OUT_SEL, 0);
+ /* choose 33th PC log buffer to read current PC index */
+ mt76_wr(dev, MT_CONN_DBG_CTL_PC_LOG_SEL, 0x3f);
+
+ /* read current PC */
+ dump->pc_stack[0] = mt76_rr(dev, MT_CONN_DBG_CTL_PC_LOG);
+
+ /* stop call stack record */
+ if (!exception) {
+ mt76_clear(dev, MT_MCU_WM_EXCP_PC_CTRL, BIT(0));
+ mt76_clear(dev, MT_MCU_WM_EXCP_LR_CTRL, BIT(0));
+ }
+
+ oldest = (u32)mt76_get_field(dev, MT_MCU_WM_EXCP_PC_CTRL,
+ GENMASK(20, 16)) + 2;
+ for (i = 0; i < 16; i++) {
+ idx = ((oldest + 2 * i + 1) % 32);
+ dump->pc_stack[i + 1] =
+ mt76_rr(dev, MT_MCU_WM_EXCP_PC_LOG + idx * 4);
+ }
+
+ oldest = (u32)mt76_get_field(dev, MT_MCU_WM_EXCP_LR_CTRL,
+ GENMASK(20, 16)) + 2;
+ for (i = 0; i < 16; i++) {
+ idx = ((oldest + 2 * i + 1) % 32);
+ dump->lr_stack[i] =
+ mt76_rr(dev, MT_MCU_WM_EXCP_LR_LOG + idx * 4);
+ }
+
+ /* start call stack record */
+ if (!exception) {
+ mt76_set(dev, MT_MCU_WM_EXCP_PC_CTRL, BIT(0));
+ mt76_set(dev, MT_MCU_WM_EXCP_LR_CTRL, BIT(0));
+ }
+}
+
+static struct mt7996_coredump *mt7996_coredump_build(struct mt7996_dev *dev)
+{
+ struct mt7996_crash_data *crash_data = dev->coredump.crash_data;
+ struct mt7996_coredump *dump;
+ struct mt7996_coredump_mem *dump_mem;
+ size_t len, sofar = 0, hdr_len = sizeof(*dump);
+ unsigned char *buf;
+ bool exception;
+
+ len = hdr_len;
+
+ if (coredump_memdump && crash_data->memdump_buf_len)
+ len += sizeof(*dump_mem) + crash_data->memdump_buf_len;
+
+ sofar += hdr_len;
+
+ /* this is going to get big when we start dumping memory and such,
+ * so go ahead and use vmalloc.
+ */
+ buf = vzalloc(len);
+ if (!buf)
+ return NULL;
+
+ mutex_lock(&dev->dump_mutex);
+
+ dump = (struct mt7996_coredump *)(buf);
+ dump->len = len;
+
+ /* plain text */
+ strscpy(dump->magic, "mt76-crash-dump", sizeof(dump->magic));
+ strscpy(dump->kernel, init_utsname()->release, sizeof(dump->kernel));
+ strscpy(dump->fw_ver, dev->mt76.hw->wiphy->fw_version,
+ sizeof(dump->fw_ver));
+
+ guid_copy(&dump->guid, &crash_data->guid);
+ dump->tv_sec = crash_data->timestamp.tv_sec;
+ dump->tv_nsec = crash_data->timestamp.tv_nsec;
+ dump->device_id = mt76_chip(&dev->mt76);
+
+ mt7996_coredump_fw_state(dev, dump, &exception);
+ mt7996_coredump_fw_stack(dev, dump, exception);
+
+ /* gather memory content */
+ dump_mem = (struct mt7996_coredump_mem *)(buf + sofar);
+ dump_mem->len = crash_data->memdump_buf_len;
+ if (coredump_memdump && crash_data->memdump_buf_len)
+ memcpy(dump_mem->data, crash_data->memdump_buf,
+ crash_data->memdump_buf_len);
+
+ mutex_unlock(&dev->dump_mutex);
+
+ return dump;
+}
+
+int mt7996_coredump_submit(struct mt7996_dev *dev)
+{
+ struct mt7996_coredump *dump;
+
+ dump = mt7996_coredump_build(dev);
+ if (!dump) {
+ dev_warn(dev->mt76.dev, "no crash dump data found\n");
+ return -ENODATA;
+ }
+
+ dev_coredumpv(dev->mt76.dev, dump, dump->len, GFP_KERNEL);
+
+ return 0;
+}
+
+int mt7996_coredump_register(struct mt7996_dev *dev)
+{
+ struct mt7996_crash_data *crash_data;
+
+ crash_data = vzalloc(sizeof(*dev->coredump.crash_data));
+ if (!crash_data)
+ return -ENOMEM;
+
+ dev->coredump.crash_data = crash_data;
+
+ if (coredump_memdump) {
+ crash_data->memdump_buf_len = mt7996_coredump_get_mem_size(dev);
+ if (!crash_data->memdump_buf_len)
+ /* no memory content */
+ return 0;
+
+ crash_data->memdump_buf = vzalloc(crash_data->memdump_buf_len);
+ if (!crash_data->memdump_buf) {
+ vfree(crash_data);
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+void mt7996_coredump_unregister(struct mt7996_dev *dev)
+{
+ if (dev->coredump.crash_data->memdump_buf) {
+ vfree(dev->coredump.crash_data->memdump_buf);
+ dev->coredump.crash_data->memdump_buf = NULL;
+ dev->coredump.crash_data->memdump_buf_len = 0;
+ }
+
+ vfree(dev->coredump.crash_data);
+ dev->coredump.crash_data = NULL;
+}
+
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/coredump.h b/drivers/net/wireless/mediatek/mt76/mt7996/coredump.h
new file mode 100644
index 0000000..af2ba21
--- /dev/null
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/coredump.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: ISC */
+/* Copyright (C) 2023 MediaTek Inc. */
+
+#ifndef _COREDUMP_H_
+#define _COREDUMP_H_
+
+#include "mt7996.h"
+
+struct mt7996_coredump {
+ char magic[16];
+
+ u32 len;
+
+ guid_t guid;
+
+ /* time-of-day stamp */
+ u64 tv_sec;
+ /* time-of-day stamp, nano-seconds */
+ u64 tv_nsec;
+ /* kernel version */
+ char kernel[64];
+ /* firmware version */
+ char fw_ver[ETHTOOL_FWVERS_LEN];
+
+ u32 device_id;
+
+ /* exception state */
+ char fw_state[12];
+
+ /* program counters */
+ char pc_current[16];
+ u32 pc_stack[17];
+ /* link registers */
+ u32 lr_stack[16];
+
+ /* memory content */
+ u8 data[];
+} __packed;
+
+struct mt7996_coredump_mem {
+ u32 len;
+ u8 data[];
+} __packed;
+
+struct mt7996_mem_hdr {
+ u32 start;
+ u32 len;
+ u8 data[];
+};
+
+struct mt7996_mem_region {
+ u32 start;
+ size_t len;
+
+ const char *name;
+};
+
+#ifdef CONFIG_DEV_COREDUMP
+
+const struct mt7996_mem_region *
+mt7996_coredump_get_mem_layout(struct mt7996_dev *dev, u32 *num);
+struct mt7996_crash_data *mt7996_coredump_new(struct mt7996_dev *dev);
+int mt7996_coredump_submit(struct mt7996_dev *dev);
+int mt7996_coredump_register(struct mt7996_dev *dev);
+void mt7996_coredump_unregister(struct mt7996_dev *dev);
+
+#else /* CONFIG_DEV_COREDUMP */
+
+static inline const struct mt7996_mem_region *
+mt7996_coredump_get_mem_layout(struct mt7996_dev *dev, u32 *num)
+{
+ return NULL;
+}
+
+static inline int mt7996_coredump_submit(struct mt7996_dev *dev)
+{
+ return 0;
+}
+
+static inline struct
+mt7996_crash_data *mt7996_coredump_new(struct mt7996_dev *dev)
+{
+ return NULL;
+}
+
+static inline int mt7996_coredump_register(struct mt7996_dev *dev)
+{
+ return 0;
+}
+
+static inline void mt7996_coredump_unregister(struct mt7996_dev *dev)
+{
+}
+
+#endif /* CONFIG_DEV_COREDUMP */
+
+#endif /* _COREDUMP_H_ */
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7996/debugfs.c
index 9c5e9ac..513ab4b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/debugfs.c
@@ -48,12 +48,12 @@ DEFINE_DEBUGFS_ATTRIBUTE(fops_implicit_txbf, mt7996_implicit_txbf_get,
/* test knob of system error recovery */
static ssize_t
-mt7996_fw_ser_set(struct file *file, const char __user *user_buf,
- size_t count, loff_t *ppos)
+mt7996_sys_recovery_set(struct file *file, const char __user *user_buf,
+ size_t count, loff_t *ppos)
{
struct mt7996_phy *phy = file->private_data;
struct mt7996_dev *dev = phy->dev;
- u8 band_idx = phy->mt76->band_idx;
+ bool band = phy->mt76->band_idx;
char buf[16];
int ret = 0;
u16 val;
@@ -73,17 +73,47 @@ mt7996_fw_ser_set(struct file *file, const char __user *user_buf,
return -EINVAL;
switch (val) {
- case SER_SET_RECOVER_L1:
- case SER_SET_RECOVER_L2:
- case SER_SET_RECOVER_L3_RX_ABORT:
- case SER_SET_RECOVER_L3_TX_ABORT:
- case SER_SET_RECOVER_L3_TX_DISABLE:
- case SER_SET_RECOVER_L3_BF:
- ret = mt7996_mcu_set_ser(dev, SER_ENABLE, BIT(val), band_idx);
+ /*
+ * 0: grab firmware current SER state.
+ * 1: trigger & enable system error L1 recovery.
+ * 2: trigger & enable system error L2 recovery.
+ * 3: trigger & enable system error L3 rx abort.
+ * 4: trigger & enable system error L3 tx abort
+ * 5: trigger & enable system error L3 tx disable.
+ * 6: trigger & enable system error L3 bf recovery.
+ * 7: trigger & enable system error L4 mdp recovery.
+ * 8: trigger & enable system error full recovery.
+ * 9: trigger firmware crash.
+ */
+ case UNI_CMD_SER_QUERY:
+ ret = mt7996_mcu_set_ser(dev, UNI_CMD_SER_QUERY, 0, band);
+ break;
+ case UNI_CMD_SER_SET_RECOVER_L1:
+ case UNI_CMD_SER_SET_RECOVER_L2:
+ case UNI_CMD_SER_SET_RECOVER_L3_RX_ABORT:
+ case UNI_CMD_SER_SET_RECOVER_L3_TX_ABORT:
+ case UNI_CMD_SER_SET_RECOVER_L3_TX_DISABLE:
+ case UNI_CMD_SER_SET_RECOVER_L3_BF:
+ case UNI_CMD_SER_SET_RECOVER_L4_MDP:
+ ret = mt7996_mcu_set_ser(dev, UNI_CMD_SER_SET, BIT(val), band);
if (ret)
return ret;
- ret = mt7996_mcu_set_ser(dev, SER_RECOVER, val, band_idx);
+ ret = mt7996_mcu_set_ser(dev, UNI_CMD_SER_TRIGGER, val, band);
+ break;
+
+ /* enable full chip reset */
+ case UNI_CMD_SER_SET_RECOVER_FULL:
+ mt76_set(dev, MT_WFDMA0_MCU_HOST_INT_ENA, MT_MCU_CMD_WDT_MASK);
+ dev->recovery.state |= MT_MCU_CMD_WDT_MASK;
+ mt7996_reset(dev);
+ break;
+
+ /* WARNING: trigger firmware crash */
+ case UNI_CMD_SER_SET_SYSTEM_ASSERT:
+ ret = mt7996_mcu_trigger_assert(dev);
+ if (ret)
+ return ret;
break;
default:
break;
@@ -92,9 +122,97 @@ mt7996_fw_ser_set(struct file *file, const char __user *user_buf,
return ret ? ret : count;
}
-static const struct file_operations mt7996_fw_ser_ops = {
- .write = mt7996_fw_ser_set,
- /* TODO: ser read */
+static ssize_t
+mt7996_sys_recovery_get(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct mt7996_phy *phy = file->private_data;
+ struct mt7996_dev *dev = phy->dev;
+ char *buff;
+ int desc = 0;
+ ssize_t ret;
+ static const size_t bufsz = 1024;
+
+ buff = kmalloc(bufsz, GFP_KERNEL);
+ if (!buff)
+ return -ENOMEM;
+
+ /* HELP */
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "Please echo the correct value ...\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "0: grab firmware transient SER state\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "1: trigger system error L1 recovery\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "2: trigger system error L2 recovery\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "3: trigger system error L3 rx abort\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "4: trigger system error L3 tx abort\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "5: trigger system error L3 tx disable\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "6: trigger system error L3 bf recovery\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "7: trigger system error L4 mdp recovery\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "8: trigger system error full recovery\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "9: trigger firmware crash\n");
+
+ /* SER statistics */
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "\nlet's dump firmware SER statistics...\n");
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_STATUS = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_SER_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_PLE_ERR = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_PLE_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_PLE_ERR_1 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_PLE1_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_PLE_ERR_AMSDU = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_PLE_AMSDU_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_PSE_ERR = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_PSE_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_PSE_ERR_1 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_PSE1_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR6_B0 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR6_BN0_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR6_B1 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR6_BN1_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR6_B2 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR6_BN2_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR7_B0 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR7_BN0_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR7_B1 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR7_BN1_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "::E R , SER_LMAC_WISR7_B2 = 0x%08x\n",
+ mt76_rr(dev, MT_SWDEF_LAMC_WISR7_BN2_STATS));
+ desc += scnprintf(buff + desc, bufsz - desc,
+ "\nSYS_RESET_COUNT: WM %d, WA %d\n",
+ dev->recovery.wm_reset_count,
+ dev->recovery.wa_reset_count);
+
+ ret = simple_read_from_buffer(user_buf, count, ppos, buff, desc);
+ kfree(buff);
+ return ret;
+}
+
+static const struct file_operations mt7996_sys_recovery_ops = {
+ .write = mt7996_sys_recovery_set,
+ .read = mt7996_sys_recovery_get,
.open = simple_open,
.llseek = default_llseek,
};
@@ -674,6 +792,8 @@ int mt7996_init_debugfs(struct mt7996_phy *phy)
debugfs_create_file("xmit-queues", 0400, dir, phy,
&mt7996_xmit_queues_fops);
debugfs_create_file("tx_stats", 0400, dir, phy, &mt7996_tx_stats_fops);
+ debugfs_create_file("sys_recovery", 0600, dir, phy,
+ &mt7996_sys_recovery_ops);
debugfs_create_file("fw_debug_wm", 0600, dir, dev, &fops_fw_debug_wm);
debugfs_create_file("fw_debug_wa", 0600, dir, dev, &fops_fw_debug_wa);
debugfs_create_file("fw_debug_bin", 0600, dir, dev, &fops_fw_debug_bin);
@@ -684,7 +804,6 @@ int mt7996_init_debugfs(struct mt7996_phy *phy)
&fops_implicit_txbf);
debugfs_create_devm_seqfile(dev->mt76.dev, "twt_stats", dir,
mt7996_twt_stats);
- debugfs_create_file("fw_ser", 0600, dir, phy, &mt7996_fw_ser_ops);
debugfs_create_file("rf_regval", 0600, dir, dev, &fops_rf_regval);
if (phy->mt76->cap.has_5ghz) {
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/dma.c b/drivers/net/wireless/mediatek/mt76/mt7996/dma.c
index c09fe42..5341434 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/dma.c
@@ -352,6 +352,70 @@ int mt7996_dma_init(struct mt7996_dev *dev)
return 0;
}
+void mt7996_dma_reset(struct mt7996_dev *dev, bool force)
+{
+ struct mt76_phy *phy2 = dev->mt76.phys[MT_BAND1];
+ struct mt76_phy *phy3 = dev->mt76.phys[MT_BAND2];
+ u32 hif1_ofs = MT_WFDMA0_PCIE1(0) - MT_WFDMA0(0);
+ int i;
+
+ mt76_clear(dev, MT_WFDMA0_GLO_CFG,
+ MT_WFDMA0_GLO_CFG_TX_DMA_EN |
+ MT_WFDMA0_GLO_CFG_RX_DMA_EN);
+
+ if (dev->hif2)
+ mt76_clear(dev, MT_WFDMA0_GLO_CFG + hif1_ofs,
+ MT_WFDMA0_GLO_CFG_TX_DMA_EN |
+ MT_WFDMA0_GLO_CFG_RX_DMA_EN);
+
+ usleep_range(1000, 2000);
+
+ for (i = 0; i < __MT_TXQ_MAX; i++) {
+ mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[i], true);
+ if (phy2)
+ mt76_queue_tx_cleanup(dev, phy2->q_tx[i], true);
+ if (phy3)
+ mt76_queue_tx_cleanup(dev, phy3->q_tx[i], true);
+ }
+
+ for (i = 0; i < __MT_MCUQ_MAX; i++)
+ mt76_queue_tx_cleanup(dev, dev->mt76.q_mcu[i], true);
+
+ mt76_for_each_q_rx(&dev->mt76, i)
+ mt76_queue_rx_cleanup(dev, &dev->mt76.q_rx[i]);
+
+ mt76_tx_status_check(&dev->mt76, true);
+
+ /* reset wfsys */
+ if (force)
+ mt7996_wfsys_reset(dev);
+
+ mt7996_dma_disable(dev, force);
+
+ /* reset hw queues */
+ for (i = 0; i < __MT_TXQ_MAX; i++) {
+ mt76_queue_reset(dev, dev->mphy.q_tx[i]);
+ if (phy2)
+ mt76_queue_reset(dev, phy2->q_tx[i]);
+ if (phy3)
+ mt76_queue_reset(dev, phy3->q_tx[i]);
+ }
+
+ for (i = 0; i < __MT_MCUQ_MAX; i++)
+ mt76_queue_reset(dev, dev->mt76.q_mcu[i]);
+
+ mt76_for_each_q_rx(&dev->mt76, i) {
+ mt76_queue_reset(dev, &dev->mt76.q_rx[i]);
+ }
+
+ mt76_tx_status_check(&dev->mt76, true);
+
+ mt76_for_each_q_rx(&dev->mt76, i)
+ mt76_queue_rx_reset(dev, i);
+
+ mt7996_dma_enable(dev);
+}
+
void mt7996_dma_cleanup(struct mt7996_dev *dev)
{
mt7996_dma_disable(dev, true);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
index 2e48c5a..544b6c6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
@@ -138,10 +138,6 @@ static int mt7996_eeprom_parse_band_config(struct mt7996_phy *phy)
case MT_EE_BAND_SEL_6GHZ:
phy->mt76->cap.has_6ghz = true;
break;
- case MT_EE_BAND_SEL_5GHZ_6GHZ:
- phy->mt76->cap.has_5ghz = true;
- phy->mt76->cap.has_6ghz = true;
- break;
default:
ret = -EINVAL;
break;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.h b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.h
index 8da599e..0c74977 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.h
@@ -31,11 +31,11 @@ enum mt7996_eeprom_field {
#define MT_EE_WIFI_CONF2_BAND_SEL GENMASK(2, 0)
#define MT_EE_WIFI_CONF1_TX_PATH_BAND0 GENMASK(5, 3)
-#define MT_EE_WIFI_CONF2_TX_PATH_BAND1 GENMASK(5, 3)
-#define MT_EE_WIFI_CONF2_TX_PATH_BAND2 GENMASK(2, 0)
+#define MT_EE_WIFI_CONF2_TX_PATH_BAND1 GENMASK(2, 0)
+#define MT_EE_WIFI_CONF2_TX_PATH_BAND2 GENMASK(5, 3)
#define MT_EE_WIFI_CONF4_STREAM_NUM_BAND0 GENMASK(5, 3)
-#define MT_EE_WIFI_CONF5_STREAM_NUM_BAND1 GENMASK(5, 3)
-#define MT_EE_WIFI_CONF5_STREAM_NUM_BAND2 GENMASK(2, 0)
+#define MT_EE_WIFI_CONF5_STREAM_NUM_BAND1 GENMASK(2, 0)
+#define MT_EE_WIFI_CONF5_STREAM_NUM_BAND2 GENMASK(5, 3)
#define MT_EE_RATE_DELTA_MASK GENMASK(5, 0)
#define MT_EE_RATE_DELTA_SIGN BIT(6)
@@ -46,7 +46,6 @@ enum mt7996_eeprom_band {
MT_EE_BAND_SEL_2GHZ,
MT_EE_BAND_SEL_5GHZ,
MT_EE_BAND_SEL_6GHZ,
- MT_EE_BAND_SEL_5GHZ_6GHZ,
};
static inline int
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/init.c b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
index 946da93..f1b48cd 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
@@ -8,6 +8,7 @@
#include "mt7996.h"
#include "mac.h"
#include "mcu.h"
+#include "coredump.h"
#include "eeprom.h"
static const struct ieee80211_iface_limit if_limits[] = {
@@ -99,9 +100,8 @@ static void mt7996_led_set_brightness(struct led_classdev *led_cdev,
mt7996_led_set_config(led_cdev, 0xff, 0);
}
-static void
-mt7996_init_txpower(struct mt7996_dev *dev,
- struct ieee80211_supported_band *sband)
+void mt7996_init_txpower(struct mt7996_dev *dev,
+ struct ieee80211_supported_band *sband)
{
int i, nss = hweight8(dev->mphy.antenna_mask);
int nss_delta = mt76_tx_power_nss_delta(nss);
@@ -182,6 +182,7 @@ mt7996_init_wiphy(struct ieee80211_hw *hw)
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_UNSOL_BCAST_PROBE_RESP);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_FILS_DISCOVERY);
wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_ACK_SIGNAL_SUPPORT);
+ wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
if (!mdev->dev->of_node ||
!of_property_read_bool(mdev->dev->of_node,
@@ -196,10 +197,13 @@ mt7996_init_wiphy(struct ieee80211_hw *hw)
hw->max_tx_fragments = 4;
- if (phy->mt76->cap.has_2ghz)
+ if (phy->mt76->cap.has_2ghz) {
phy->mt76->sband_2g.sband.ht_cap.cap |=
IEEE80211_HT_CAP_LDPC_CODING |
IEEE80211_HT_CAP_MAX_AMSDU;
+ phy->mt76->sband_2g.sband.ht_cap.ampdu_density =
+ IEEE80211_HT_MPDU_DENSITY_2;
+ }
if (phy->mt76->cap.has_5ghz) {
phy->mt76->sband_5g.sband.ht_cap.cap |=
@@ -211,6 +215,8 @@ mt7996_init_wiphy(struct ieee80211_hw *hw)
IEEE80211_VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_MASK |
IEEE80211_VHT_CAP_SHORT_GI_160 |
IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160MHZ;
+ phy->mt76->sband_5g.sband.ht_cap.ampdu_density =
+ IEEE80211_HT_MPDU_DENSITY_1;
}
mt76_set_stream_caps(phy->mt76, true);
@@ -250,7 +256,21 @@ mt7996_mac_init_band(struct mt7996_dev *dev, u8 band)
mt76_rmw(dev, MT_WTBLOFF_RSCR(band), mask, set);
}
-static void mt7996_mac_init(struct mt7996_dev *dev)
+static void mt7996_mac_init_basic_rates(struct mt7996_dev *dev)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(mt76_rates); i++) {
+ u16 rate = mt76_rates[i].hw_value;
+ u16 idx = MT7996_BASIC_RATES_TBL + i;
+
+ rate = FIELD_PREP(MT_TX_RATE_MODE, rate >> 8) |
+ FIELD_PREP(MT_TX_RATE_IDX, rate & GENMASK(7, 0));
+ mt7996_mac_set_fixed_rate_table(dev, idx, rate);
+ }
+}
+
+void mt7996_mac_init(struct mt7996_dev *dev)
{
#define HIF_TXD_V2_1 4
int i;
@@ -282,9 +302,11 @@ static void mt7996_mac_init(struct mt7996_dev *dev)
for (i = MT_BAND0; i <= MT_BAND2; i++)
mt7996_mac_init_band(dev, i);
+
+ mt7996_mac_init_basic_rates(dev);
}
-static int mt7996_txbf_init(struct mt7996_dev *dev)
+int mt7996_txbf_init(struct mt7996_dev *dev)
{
int ret;
@@ -553,27 +575,6 @@ mt7996_set_stream_he_txbf_caps(struct mt7996_phy *phy,
}
static void
-mt7996_gen_ppe_thresh(u8 *he_ppet, int nss)
-{
- u8 i, ppet_bits, ppet_size, ru_bit_mask = 0x7; /* HE80 */
- static const u8 ppet16_ppet8_ru3_ru0[] = {0x1c, 0xc7, 0x71};
-
- he_ppet[0] = FIELD_PREP(IEEE80211_PPE_THRES_NSS_MASK, nss - 1) |
- FIELD_PREP(IEEE80211_PPE_THRES_RU_INDEX_BITMASK_MASK,
- ru_bit_mask);
-
- ppet_bits = IEEE80211_PPE_THRES_INFO_PPET_SIZE *
- nss * hweight8(ru_bit_mask) * 2;
- ppet_size = DIV_ROUND_UP(ppet_bits, 8);
-
- for (i = 0; i < ppet_size - 1; i++)
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3];
-
- he_ppet[i + 1] = ppet16_ppet8_ru3_ru0[i % 3] &
- (0xff >> (8 - (ppet_bits - 1) % 8));
-}
-
-static void
mt7996_init_he_caps(struct mt7996_phy *phy, enum nl80211_band band,
struct ieee80211_sband_iftype_data *data,
enum nl80211_iftype iftype)
@@ -678,7 +679,7 @@ mt7996_init_he_caps(struct mt7996_phy *phy, enum nl80211_band band,
memset(he_cap->ppe_thres, 0, sizeof(he_cap->ppe_thres));
if (he_cap_elem->phy_cap_info[6] &
IEEE80211_HE_PHY_CAP6_PPE_THRESHOLD_PRESENT) {
- mt7996_gen_ppe_thresh(he_cap->ppe_thres, nss);
+ mt76_connac_gen_ppe_thresh(he_cap->ppe_thres, nss);
} else {
he_cap_elem->phy_cap_info[9] |=
u8_encode_bits(IEEE80211_HE_PHY_CAP9_NOMINAL_PKT_PADDING_16US,
@@ -689,7 +690,7 @@ mt7996_init_he_caps(struct mt7996_phy *phy, enum nl80211_band band,
u16 cap = IEEE80211_HE_6GHZ_CAP_TX_ANTPAT_CONS |
IEEE80211_HE_6GHZ_CAP_RX_ANTPAT_CONS;
- cap |= u16_encode_bits(IEEE80211_HT_MPDU_DENSITY_2,
+ cap |= u16_encode_bits(IEEE80211_HT_MPDU_DENSITY_0_5,
IEEE80211_HE_6GHZ_CAP_MIN_MPDU_START) |
u16_encode_bits(IEEE80211_VHT_MAX_AMPDU_1024K,
IEEE80211_HE_6GHZ_CAP_MAX_AMPDU_LEN_EXP) |
@@ -858,6 +859,8 @@ int mt7996_register_device(struct mt7996_dev *dev)
init_waitqueue_head(&dev->reset_wait);
INIT_WORK(&dev->reset_work, mt7996_mac_reset_work);
+ INIT_WORK(&dev->dump_work, mt7996_mac_dump_work);
+ mutex_init(&dev->dump_mutex);
ret = mt7996_init_hardware(dev);
if (ret)
@@ -886,18 +889,25 @@ int mt7996_register_device(struct mt7996_dev *dev)
if (ret)
return ret;
- return mt7996_init_debugfs(&dev->phy);
+ dev->recovery.hw_init_done = true;
+
+ ret = mt7996_init_debugfs(&dev->phy);
+ if (ret)
+ return ret;
+
+ return mt7996_coredump_register(dev);
}
void mt7996_unregister_device(struct mt7996_dev *dev)
{
mt7996_unregister_phy(mt7996_phy3(dev), MT_BAND2);
mt7996_unregister_phy(mt7996_phy2(dev), MT_BAND1);
+ mt7996_coredump_unregister(dev);
mt76_unregister_device(&dev->mt76);
mt7996_mcu_exit(dev);
mt7996_tx_token_put(dev);
mt7996_dma_cleanup(dev);
- tasklet_disable(&dev->irq_tasklet);
+ tasklet_disable(&dev->mt76.irq_tasklet);
mt76_free_device(&dev->mt76);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
index c9a9f0e..130eb7b4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
@@ -5,6 +5,7 @@
#include <linux/etherdevice.h>
#include <linux/timekeeping.h>
+#include "coredump.h"
#include "mt7996.h"
#include "../dma.h"
#include "mac.h"
@@ -78,10 +79,6 @@ static struct mt76_wcid *mt7996_rx_get_wcid(struct mt7996_dev *dev,
return &sta->vif->sta.wcid;
}
-void mt7996_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps)
-{
-}
-
bool mt7996_mac_wtbl_update(struct mt7996_dev *dev, int idx, u32 mask)
{
mt76_rmw(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_WLAN_IDX,
@@ -255,17 +252,25 @@ void mt7996_mac_enable_rtscts(struct mt7996_dev *dev,
mt76_clear(dev, addr, BIT(5));
}
+void mt7996_mac_set_fixed_rate_table(struct mt7996_dev *dev,
+ u8 tbl_idx, u16 rate_idx)
+{
+ u32 ctrl = MT_WTBL_ITCR_WR | MT_WTBL_ITCR_EXEC | tbl_idx;
+
+ mt76_wr(dev, MT_WTBL_ITDR0, rate_idx);
+ /* use wtbl spe idx */
+ mt76_wr(dev, MT_WTBL_ITDR1, MT_WTBL_SPE_IDX_SEL);
+ mt76_wr(dev, MT_WTBL_ITCR, ctrl);
+}
+
static void
mt7996_mac_decode_he_radiotap_ru(struct mt76_rx_status *status,
struct ieee80211_radiotap_he *he,
__le32 *rxv)
{
- u32 ru_h, ru_l;
- u8 ru, offs = 0;
+ u32 ru, offs = 0;
- ru_l = le32_get_bits(rxv[0], MT_PRXV_HE_RU_ALLOC_L);
- ru_h = le32_get_bits(rxv[1], MT_PRXV_HE_RU_ALLOC_H);
- ru = (u8)(ru_l | ru_h << 4);
+ ru = le32_get_bits(rxv[0], MT_PRXV_HE_RU_ALLOC);
status->bw = RATE_INFO_BW_HE_RU;
@@ -330,18 +335,23 @@ mt7996_mac_decode_he_mu_radiotap(struct sk_buff *skb, __le32 *rxv)
he_mu->flags2 |= MU_PREP(FLAGS2_BW_FROM_SIG_A_BW, status->bw) |
MU_PREP(FLAGS2_SIG_B_SYMS_USERS,
- le32_get_bits(rxv[2], MT_CRXV_HE_NUM_USER));
+ le32_get_bits(rxv[4], MT_CRXV_HE_NUM_USER));
- he_mu->ru_ch1[0] = le32_get_bits(rxv[3], MT_CRXV_HE_RU0);
+ he_mu->ru_ch1[0] = le32_get_bits(rxv[16], MT_CRXV_HE_RU0) & 0xff;
if (status->bw >= RATE_INFO_BW_40) {
he_mu->flags1 |= HE_BITS(MU_FLAGS1_CH2_RU_KNOWN);
- he_mu->ru_ch2[0] = le32_get_bits(rxv[3], MT_CRXV_HE_RU1);
+ he_mu->ru_ch2[0] = le32_get_bits(rxv[16], MT_CRXV_HE_RU1) & 0xff;
}
if (status->bw >= RATE_INFO_BW_80) {
- he_mu->ru_ch1[1] = le32_get_bits(rxv[3], MT_CRXV_HE_RU2);
- he_mu->ru_ch2[1] = le32_get_bits(rxv[3], MT_CRXV_HE_RU3);
+ u32 ru_h, ru_l;
+
+ he_mu->ru_ch1[1] = le32_get_bits(rxv[16], MT_CRXV_HE_RU2) & 0xff;
+
+ ru_l = le32_get_bits(rxv[16], MT_CRXV_HE_RU3_L);
+ ru_h = le32_get_bits(rxv[17], MT_CRXV_HE_RU3_H) & 0x7;
+ he_mu->ru_ch2[1] = (u8)(ru_l | ru_h << 4);
}
}
@@ -364,23 +374,23 @@ mt7996_mac_decode_he_radiotap(struct sk_buff *skb, __le32 *rxv, u8 mode)
HE_BITS(DATA2_TXOP_KNOWN),
};
struct ieee80211_radiotap_he *he = NULL;
- u32 ltf_size = le32_get_bits(rxv[2], MT_CRXV_HE_LTF_SIZE) + 1;
+ u32 ltf_size = le32_get_bits(rxv[4], MT_CRXV_HE_LTF_SIZE) + 1;
status->flag |= RX_FLAG_RADIOTAP_HE;
he = skb_push(skb, sizeof(known));
memcpy(he, &known, sizeof(known));
- he->data3 = HE_PREP(DATA3_BSS_COLOR, BSS_COLOR, rxv[14]) |
- HE_PREP(DATA3_LDPC_XSYMSEG, LDPC_EXT_SYM, rxv[2]);
- he->data4 = HE_PREP(DATA4_SU_MU_SPTL_REUSE, SR_MASK, rxv[11]);
- he->data5 = HE_PREP(DATA5_PE_DISAMBIG, PE_DISAMBIG, rxv[2]) |
+ he->data3 = HE_PREP(DATA3_BSS_COLOR, BSS_COLOR, rxv[9]) |
+ HE_PREP(DATA3_LDPC_XSYMSEG, LDPC_EXT_SYM, rxv[4]);
+ he->data4 = HE_PREP(DATA4_SU_MU_SPTL_REUSE, SR_MASK, rxv[13]);
+ he->data5 = HE_PREP(DATA5_PE_DISAMBIG, PE_DISAMBIG, rxv[5]) |
le16_encode_bits(ltf_size,
IEEE80211_RADIOTAP_HE_DATA5_LTF_SIZE);
if (le32_to_cpu(rxv[0]) & MT_PRXV_TXBF)
he->data5 |= HE_BITS(DATA5_TXBF);
- he->data6 = HE_PREP(DATA6_TXOP, TXOP_DUR, rxv[14]) |
- HE_PREP(DATA6_DOPPLER, DOPPLER, rxv[14]);
+ he->data6 = HE_PREP(DATA6_TXOP, TXOP_DUR, rxv[9]) |
+ HE_PREP(DATA6_DOPPLER, DOPPLER, rxv[9]);
switch (mode) {
case MT_PHY_TYPE_HE_SU:
@@ -389,22 +399,22 @@ mt7996_mac_decode_he_radiotap(struct sk_buff *skb, __le32 *rxv, u8 mode)
HE_BITS(DATA1_BEAM_CHANGE_KNOWN) |
HE_BITS(DATA1_BW_RU_ALLOC_KNOWN);
- he->data3 |= HE_PREP(DATA3_BEAM_CHANGE, BEAM_CHNG, rxv[14]) |
- HE_PREP(DATA3_UL_DL, UPLINK, rxv[2]);
+ he->data3 |= HE_PREP(DATA3_BEAM_CHANGE, BEAM_CHNG, rxv[8]) |
+ HE_PREP(DATA3_UL_DL, UPLINK, rxv[5]);
break;
case MT_PHY_TYPE_HE_EXT_SU:
he->data1 |= HE_BITS(DATA1_FORMAT_EXT_SU) |
HE_BITS(DATA1_UL_DL_KNOWN) |
HE_BITS(DATA1_BW_RU_ALLOC_KNOWN);
- he->data3 |= HE_PREP(DATA3_UL_DL, UPLINK, rxv[2]);
+ he->data3 |= HE_PREP(DATA3_UL_DL, UPLINK, rxv[5]);
break;
case MT_PHY_TYPE_HE_MU:
he->data1 |= HE_BITS(DATA1_FORMAT_MU) |
HE_BITS(DATA1_UL_DL_KNOWN);
- he->data3 |= HE_PREP(DATA3_UL_DL, UPLINK, rxv[2]);
- he->data4 |= HE_PREP(DATA4_MU_STA_ID, MU_AID, rxv[7]);
+ he->data3 |= HE_PREP(DATA3_UL_DL, UPLINK, rxv[5]);
+ he->data4 |= HE_PREP(DATA4_MU_STA_ID, MU_AID, rxv[8]);
mt7996_mac_decode_he_radiotap_ru(status, he, rxv);
mt7996_mac_decode_he_mu_radiotap(skb, rxv);
@@ -415,10 +425,10 @@ mt7996_mac_decode_he_radiotap(struct sk_buff *skb, __le32 *rxv, u8 mode)
HE_BITS(DATA1_SPTL_REUSE3_KNOWN) |
HE_BITS(DATA1_SPTL_REUSE4_KNOWN);
- he->data4 |= HE_PREP(DATA4_TB_SPTL_REUSE1, SR_MASK, rxv[11]) |
- HE_PREP(DATA4_TB_SPTL_REUSE2, SR1_MASK, rxv[11]) |
- HE_PREP(DATA4_TB_SPTL_REUSE3, SR2_MASK, rxv[11]) |
- HE_PREP(DATA4_TB_SPTL_REUSE4, SR3_MASK, rxv[11]);
+ he->data4 |= HE_PREP(DATA4_TB_SPTL_REUSE1, SR_MASK, rxv[13]) |
+ HE_PREP(DATA4_TB_SPTL_REUSE2, SR1_MASK, rxv[13]) |
+ HE_PREP(DATA4_TB_SPTL_REUSE3, SR2_MASK, rxv[13]) |
+ HE_PREP(DATA4_TB_SPTL_REUSE4, SR3_MASK, rxv[13]);
mt7996_mac_decode_he_radiotap_ru(status, he, rxv);
break;
@@ -570,11 +580,12 @@ mt7996_mac_fill_rx_rate(struct mt7996_dev *dev,
case MT_PHY_TYPE_EHT_SU:
case MT_PHY_TYPE_EHT_TRIG:
case MT_PHY_TYPE_EHT_MU:
- /* TODO: currently report rx rate with HE rate */
status->nss = nss;
- status->encoding = RX_ENC_HE;
- bw = min_t(int, bw, IEEE80211_STA_RX_BW_160);
- i = min_t(int, i & 0xf, 11);
+ status->encoding = RX_ENC_EHT;
+ i &= GENMASK(3, 0);
+
+ if (gi <= NL80211_RATE_INFO_EHT_GI_3_2)
+ status->eht.gi = gi;
break;
default:
return -EINVAL;
@@ -630,6 +641,8 @@ mt7996_mac_fill_rx(struct mt7996_dev *dev, struct sk_buff *skb)
u32 rxd4 = le32_to_cpu(rxd[4]);
u32 csum_mask = MT_RXD0_NORMAL_IP_SUM | MT_RXD0_NORMAL_UDP_TCP_SUM;
u32 csum_status = *(u32 *)skb->cb;
+ u32 mesh_mask = MT_RXD0_MESH | MT_RXD0_MHCP;
+ bool is_mesh = (rxd0 & mesh_mask) == mesh_mask;
bool unicast, insert_ccmp_hdr = false;
u8 remove_pad, amsdu_info, band_idx;
u8 mode = 0, qos_ctl = 0;
@@ -821,19 +834,16 @@ mt7996_mac_fill_rx(struct mt7996_dev *dev, struct sk_buff *skb)
int pad_start = 0;
skb_pull(skb, hdr_gap);
- if (!hdr_trans && status->amsdu) {
+ if (!hdr_trans && status->amsdu && !(ieee80211_has_a4(fc) && is_mesh)) {
pad_start = ieee80211_get_hdrlen_from_skb(skb);
- } else if (hdr_trans && (rxd2 & MT_RXD2_NORMAL_HDR_TRANS_ERROR)) {
+ } else if (hdr_trans && (rxd2 & MT_RXD2_NORMAL_HDR_TRANS_ERROR) &&
+ get_unaligned_be16(skb->data + pad_start) == ETH_P_8021Q) {
/* When header translation failure is indicated,
* the hardware will insert an extra 2-byte field
* containing the data length after the protocol
* type field.
*/
- pad_start = 12;
- if (get_unaligned_be16(skb->data + pad_start) == ETH_P_8021Q)
- pad_start += 4;
- else
- pad_start = 0;
+ pad_start = 16;
}
if (pad_start) {
@@ -854,8 +864,17 @@ mt7996_mac_fill_rx(struct mt7996_dev *dev, struct sk_buff *skb)
hdr = mt76_skb_get_hdr(skb);
fc = hdr->frame_control;
if (ieee80211_is_data_qos(fc)) {
+ u8 *qos = ieee80211_get_qos_ctl(hdr);
+
seq_ctrl = le16_to_cpu(hdr->seq_ctrl);
- qos_ctl = *ieee80211_get_qos_ctl(hdr);
+ qos_ctl = *qos;
+
+ /* Mesh DA/SA/Length will be stripped after hardware
+ * de-amsdu, so here needs to clear amsdu present bit
+ * to mark it as a normal mesh frame.
+ */
+ if (ieee80211_has_a4(fc) && is_mesh && status->amsdu)
+ *qos &= ~IEEE80211_QOS_CTL_A_MSDU_PRESENT;
}
} else {
status->flag |= RX_FLAG_8023;
@@ -979,12 +998,13 @@ mt7996_mac_write_txwi_80211(struct mt7996_dev *dev, __le32 *txwi,
}
void mt7996_mac_write_txwi(struct mt7996_dev *dev, __le32 *txwi,
- struct sk_buff *skb, struct mt76_wcid *wcid, int pid,
- struct ieee80211_key_conf *key, u32 changed)
+ struct sk_buff *skb, struct mt76_wcid *wcid,
+ struct ieee80211_key_conf *key, int pid,
+ enum mt76_txq_id qid, u32 changed)
{
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct ieee80211_vif *vif = info->control.vif;
- struct mt76_phy *mphy = &dev->mphy;
+ struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
u8 band_idx = (info->hw_queue & MT_TX_HW_QUEUE_PHY) >> 2;
u8 p_fmt, q_idx, omac_idx = 0, wmm_idx = 0;
bool is_8023 = info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP;
@@ -996,22 +1016,18 @@ void mt7996_mac_write_txwi(struct mt7996_dev *dev, __le32 *txwi,
BSS_CHANGED_FILS_DISCOVERY));
if (vif) {
- struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
-
omac_idx = mvif->mt76.omac_idx;
wmm_idx = mvif->mt76.wmm_idx;
band_idx = mvif->mt76.band_idx;
}
- mphy = mt76_dev_phy(&dev->mt76, band_idx);
-
if (inband_disc) {
p_fmt = MT_TX_TYPE_FW;
q_idx = MT_LMAC_ALTX0;
} else if (beacon) {
p_fmt = MT_TX_TYPE_FW;
q_idx = MT_LMAC_BCN0;
- } else if (skb_get_queue_mapping(skb) >= MT_TXQ_PSD) {
+ } else if (qid >= MT_TXQ_PSD) {
p_fmt = MT_TX_TYPE_CT;
q_idx = MT_LMAC_ALTX0;
} else {
@@ -1062,18 +1078,17 @@ void mt7996_mac_write_txwi(struct mt7996_dev *dev, __le32 *txwi,
mt7996_mac_write_txwi_80211(dev, txwi, skb, key);
if (txwi[1] & cpu_to_le32(MT_TXD1_FIXED_RATE)) {
- /* Fixed rata is available just for 802.11 txd */
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
- bool multicast = is_multicast_ether_addr(hdr->addr1);
- u16 rate = mt76_connac2_mac_tx_rate_val(mphy, vif, beacon,
- multicast);
+ bool mcast = ieee80211_is_data(hdr->frame_control) &&
+ is_multicast_ether_addr(hdr->addr1);
+ u8 idx = mvif->basic_rates_idx;
- /* fix to bw 20 */
- val = MT_TXD6_FIXED_BW |
- FIELD_PREP(MT_TXD6_BW, 0) |
- FIELD_PREP(MT_TXD6_TX_RATE, rate);
+ if (mcast && mvif->mcast_rates_idx)
+ idx = mvif->mcast_rates_idx;
+ else if (beacon && mvif->beacon_rates_idx)
+ idx = mvif->beacon_rates_idx;
- txwi[6] |= cpu_to_le32(val);
+ txwi[6] |= FIELD_PREP(MT_TXD6_TX_RATE, idx);
txwi[3] |= cpu_to_le32(MT_TXD3_BA_DISABLE);
}
}
@@ -1117,11 +1132,8 @@ int mt7996_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
return id;
pid = mt76_tx_status_skb_add(mdev, wcid, tx_info->skb);
- memset(txwi_ptr, 0, MT_TXD_SIZE);
- /* Transmit non qos data by 802.11 header and need to fill txd by host*/
- if (!is_8023 || pid >= MT_PACKET_ID_FIRST)
- mt7996_mac_write_txwi(dev, txwi_ptr, tx_info->skb, wcid, pid,
- key, 0);
+ mt7996_mac_write_txwi(dev, txwi_ptr, tx_info->skb, wcid, key,
+ pid, qid, 0);
txp = (struct mt76_connac_txp_common *)(txwi + MT_TXD_SIZE);
for (i = 0; i < nbuf; i++) {
@@ -1130,10 +1142,8 @@ int mt7996_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
}
txp->fw.nbuf = nbuf;
- txp->fw.flags = cpu_to_le16(MT_CT_INFO_FROM_HOST);
-
- if (!is_8023 || pid >= MT_PACKET_ID_FIRST)
- txp->fw.flags |= cpu_to_le16(MT_CT_INFO_APPLY_TXD);
+ txp->fw.flags =
+ cpu_to_le16(MT_CT_INFO_FROM_HOST | MT_CT_INFO_APPLY_TXD);
if (!key)
txp->fw.flags |= cpu_to_le16(MT_CT_INFO_NONE_CIPHER_FRAME);
@@ -1704,7 +1714,7 @@ mt7996_wait_reset_state(struct mt7996_dev *dev, u32 state)
bool ret;
ret = wait_event_timeout(dev->reset_wait,
- (READ_ONCE(dev->reset_state) & state),
+ (READ_ONCE(dev->recovery.state) & state),
MT7996_RESET_TIMEOUT);
WARN(!ret, "Timeout waiting for MCU reset state %x\n", state);
@@ -1753,53 +1763,6 @@ mt7996_update_beacons(struct mt7996_dev *dev)
mt7996_update_vif_beacon, phy3->hw);
}
-static void
-mt7996_dma_reset(struct mt7996_dev *dev)
-{
- struct mt76_phy *phy2 = dev->mt76.phys[MT_BAND1];
- struct mt76_phy *phy3 = dev->mt76.phys[MT_BAND2];
- u32 hif1_ofs = MT_WFDMA0_PCIE1(0) - MT_WFDMA0(0);
- int i;
-
- mt76_clear(dev, MT_WFDMA0_GLO_CFG,
- MT_WFDMA0_GLO_CFG_TX_DMA_EN |
- MT_WFDMA0_GLO_CFG_RX_DMA_EN);
-
- if (dev->hif2)
- mt76_clear(dev, MT_WFDMA0_GLO_CFG + hif1_ofs,
- MT_WFDMA0_GLO_CFG_TX_DMA_EN |
- MT_WFDMA0_GLO_CFG_RX_DMA_EN);
-
- usleep_range(1000, 2000);
-
- for (i = 0; i < __MT_TXQ_MAX; i++) {
- mt76_queue_tx_cleanup(dev, dev->mphy.q_tx[i], true);
- if (phy2)
- mt76_queue_tx_cleanup(dev, phy2->q_tx[i], true);
- if (phy3)
- mt76_queue_tx_cleanup(dev, phy3->q_tx[i], true);
- }
-
- for (i = 0; i < __MT_MCUQ_MAX; i++)
- mt76_queue_tx_cleanup(dev, dev->mt76.q_mcu[i], true);
-
- mt76_for_each_q_rx(&dev->mt76, i)
- mt76_queue_rx_reset(dev, i);
-
- mt76_tx_status_check(&dev->mt76, true);
-
- /* re-init prefetch settings after reset */
- mt7996_dma_prefetch(dev);
-
- mt76_set(dev, MT_WFDMA0_GLO_CFG,
- MT_WFDMA0_GLO_CFG_TX_DMA_EN | MT_WFDMA0_GLO_CFG_RX_DMA_EN);
-
- if (dev->hif2)
- mt76_set(dev, MT_WFDMA0_GLO_CFG + hif1_ofs,
- MT_WFDMA0_GLO_CFG_TX_DMA_EN |
- MT_WFDMA0_GLO_CFG_RX_DMA_EN);
-}
-
void mt7996_tx_token_put(struct mt7996_dev *dev)
{
struct mt76_txwi_cache *txwi;
@@ -1814,7 +1777,193 @@ void mt7996_tx_token_put(struct mt7996_dev *dev)
idr_destroy(&dev->mt76.token);
}
-/* system error recovery */
+static int
+mt7996_mac_restart(struct mt7996_dev *dev)
+{
+ struct mt7996_phy *phy2, *phy3;
+ struct mt76_dev *mdev = &dev->mt76;
+ int i, ret;
+
+ phy2 = mt7996_phy2(dev);
+ phy3 = mt7996_phy3(dev);
+
+ if (dev->hif2) {
+ mt76_wr(dev, MT_INT1_MASK_CSR, 0x0);
+ mt76_wr(dev, MT_INT1_SOURCE_CSR, ~0);
+ }
+
+ if (dev_is_pci(mdev->dev)) {
+ mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0x0);
+ if (dev->hif2)
+ mt76_wr(dev, MT_PCIE1_MAC_INT_ENABLE, 0x0);
+ }
+
+ set_bit(MT76_RESET, &dev->mphy.state);
+ set_bit(MT76_MCU_RESET, &dev->mphy.state);
+ wake_up(&dev->mt76.mcu.wait);
+ if (phy2) {
+ set_bit(MT76_RESET, &phy2->mt76->state);
+ set_bit(MT76_MCU_RESET, &phy2->mt76->state);
+ }
+ if (phy3) {
+ set_bit(MT76_RESET, &phy3->mt76->state);
+ set_bit(MT76_MCU_RESET, &phy3->mt76->state);
+ }
+
+ /* lock/unlock all queues to ensure that no tx is pending */
+ mt76_txq_schedule_all(&dev->mphy);
+ if (phy2)
+ mt76_txq_schedule_all(phy2->mt76);
+ if (phy3)
+ mt76_txq_schedule_all(phy3->mt76);
+
+ /* disable all tx/rx napi */
+ mt76_worker_disable(&dev->mt76.tx_worker);
+ mt76_for_each_q_rx(mdev, i) {
+ if (mdev->q_rx[i].ndesc)
+ napi_disable(&dev->mt76.napi[i]);
+ }
+ napi_disable(&dev->mt76.tx_napi);
+
+ /* token reinit */
+ mt7996_tx_token_put(dev);
+ idr_init(&dev->mt76.token);
+
+ mt7996_dma_reset(dev, true);
+
+ local_bh_disable();
+ mt76_for_each_q_rx(mdev, i) {
+ if (mdev->q_rx[i].ndesc) {
+ napi_enable(&dev->mt76.napi[i]);
+ napi_schedule(&dev->mt76.napi[i]);
+ }
+ }
+ local_bh_enable();
+ clear_bit(MT76_MCU_RESET, &dev->mphy.state);
+ clear_bit(MT76_STATE_MCU_RUNNING, &dev->mphy.state);
+
+ mt76_wr(dev, MT_INT_MASK_CSR, dev->mt76.mmio.irqmask);
+ mt76_wr(dev, MT_INT_SOURCE_CSR, ~0);
+ if (dev->hif2) {
+ mt76_wr(dev, MT_INT1_MASK_CSR, dev->mt76.mmio.irqmask);
+ mt76_wr(dev, MT_INT1_SOURCE_CSR, ~0);
+ }
+ if (dev_is_pci(mdev->dev)) {
+ mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff);
+ if (dev->hif2)
+ mt76_wr(dev, MT_PCIE1_MAC_INT_ENABLE, 0xff);
+ }
+
+ /* load firmware */
+ ret = mt7996_mcu_init_firmware(dev);
+ if (ret)
+ goto out;
+
+ /* set the necessary init items */
+ ret = mt7996_mcu_set_eeprom(dev);
+ if (ret)
+ goto out;
+
+ mt7996_mac_init(dev);
+ mt7996_init_txpower(dev, &dev->mphy.sband_2g.sband);
+ mt7996_init_txpower(dev, &dev->mphy.sband_5g.sband);
+ mt7996_init_txpower(dev, &dev->mphy.sband_6g.sband);
+ ret = mt7996_txbf_init(dev);
+
+ if (test_bit(MT76_STATE_RUNNING, &dev->mphy.state)) {
+ ret = mt7996_run(dev->mphy.hw);
+ if (ret)
+ goto out;
+ }
+
+ if (phy2 && test_bit(MT76_STATE_RUNNING, &phy2->mt76->state)) {
+ ret = mt7996_run(phy2->mt76->hw);
+ if (ret)
+ goto out;
+ }
+
+ if (phy3 && test_bit(MT76_STATE_RUNNING, &phy3->mt76->state)) {
+ ret = mt7996_run(phy3->mt76->hw);
+ if (ret)
+ goto out;
+ }
+
+out:
+ /* reset done */
+ clear_bit(MT76_RESET, &dev->mphy.state);
+ if (phy2)
+ clear_bit(MT76_RESET, &phy2->mt76->state);
+ if (phy3)
+ clear_bit(MT76_RESET, &phy3->mt76->state);
+
+ local_bh_disable();
+ napi_enable(&dev->mt76.tx_napi);
+ napi_schedule(&dev->mt76.tx_napi);
+ local_bh_enable();
+
+ mt76_worker_enable(&dev->mt76.tx_worker);
+ return ret;
+}
+
+static void
+mt7996_mac_full_reset(struct mt7996_dev *dev)
+{
+ struct mt7996_phy *phy2, *phy3;
+ int i;
+
+ phy2 = mt7996_phy2(dev);
+ phy3 = mt7996_phy3(dev);
+ dev->recovery.hw_full_reset = true;
+
+ wake_up(&dev->mt76.mcu.wait);
+ ieee80211_stop_queues(mt76_hw(dev));
+ if (phy2)
+ ieee80211_stop_queues(phy2->mt76->hw);
+ if (phy3)
+ ieee80211_stop_queues(phy3->mt76->hw);
+
+ cancel_delayed_work_sync(&dev->mphy.mac_work);
+ if (phy2)
+ cancel_delayed_work_sync(&phy2->mt76->mac_work);
+ if (phy3)
+ cancel_delayed_work_sync(&phy3->mt76->mac_work);
+
+ mutex_lock(&dev->mt76.mutex);
+ for (i = 0; i < 10; i++) {
+ if (!mt7996_mac_restart(dev))
+ break;
+ }
+ mutex_unlock(&dev->mt76.mutex);
+
+ if (i == 10)
+ dev_err(dev->mt76.dev, "chip full reset failed\n");
+
+ ieee80211_restart_hw(mt76_hw(dev));
+ if (phy2)
+ ieee80211_restart_hw(phy2->mt76->hw);
+ if (phy3)
+ ieee80211_restart_hw(phy3->mt76->hw);
+
+ ieee80211_wake_queues(mt76_hw(dev));
+ if (phy2)
+ ieee80211_wake_queues(phy2->mt76->hw);
+ if (phy3)
+ ieee80211_wake_queues(phy3->mt76->hw);
+
+ dev->recovery.hw_full_reset = false;
+ ieee80211_queue_delayed_work(mt76_hw(dev),
+ &dev->mphy.mac_work,
+ MT7996_WATCHDOG_TIME);
+ if (phy2)
+ ieee80211_queue_delayed_work(phy2->mt76->hw,
+ &phy2->mt76->mac_work,
+ MT7996_WATCHDOG_TIME);
+ if (phy3)
+ ieee80211_queue_delayed_work(phy3->mt76->hw,
+ &phy3->mt76->mac_work,
+ MT7996_WATCHDOG_TIME);
+}
+
void mt7996_mac_reset_work(struct work_struct *work)
{
struct mt7996_phy *phy2, *phy3;
@@ -1825,9 +1974,36 @@ void mt7996_mac_reset_work(struct work_struct *work)
phy2 = mt7996_phy2(dev);
phy3 = mt7996_phy3(dev);
- if (!(READ_ONCE(dev->reset_state) & MT_MCU_CMD_STOP_DMA))
+ /* chip full reset */
+ if (dev->recovery.restart) {
+ /* disable WA/WM WDT */
+ mt76_clear(dev, MT_WFDMA0_MCU_HOST_INT_ENA,
+ MT_MCU_CMD_WDT_MASK);
+
+ if (READ_ONCE(dev->recovery.state) & MT_MCU_CMD_WA_WDT)
+ dev->recovery.wa_reset_count++;
+ else
+ dev->recovery.wm_reset_count++;
+
+ mt7996_mac_full_reset(dev);
+
+ /* enable mcu irq */
+ mt7996_irq_enable(dev, MT_INT_MCU_CMD);
+ mt7996_irq_disable(dev, 0);
+
+ /* enable WA/WM WDT */
+ mt76_set(dev, MT_WFDMA0_MCU_HOST_INT_ENA, MT_MCU_CMD_WDT_MASK);
+
+ dev->recovery.state = MT_MCU_CMD_NORMAL_STATE;
+ dev->recovery.restart = false;
+ return;
+ }
+
+ if (!(READ_ONCE(dev->recovery.state) & MT_MCU_CMD_STOP_DMA))
return;
+ dev_info(dev->mt76.dev,"\n%s L1 SER recovery start.",
+ wiphy_name(dev->mt76.hw->wiphy));
ieee80211_stop_queues(mt76_hw(dev));
if (phy2)
ieee80211_stop_queues(phy2->mt76->hw);
@@ -1856,7 +2032,7 @@ void mt7996_mac_reset_work(struct work_struct *work)
mt76_wr(dev, MT_MCU_INT_EVENT, MT_MCU_INT_EVENT_DMA_STOPPED);
if (mt7996_wait_reset_state(dev, MT_MCU_CMD_RESET_DONE)) {
- mt7996_dma_reset(dev);
+ mt7996_dma_reset(dev, false);
mt7996_tx_token_put(dev);
idr_init(&dev->mt76.token);
@@ -1879,7 +2055,7 @@ void mt7996_mac_reset_work(struct work_struct *work)
}
local_bh_enable();
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
mt76_wr(dev, MT_MCU_INT_EVENT, MT_MCU_INT_EVENT_RESET_DONE);
mt7996_wait_reset_state(dev, MT_MCU_CMD_NORMAL_STATE);
@@ -1911,6 +2087,101 @@ void mt7996_mac_reset_work(struct work_struct *work)
ieee80211_queue_delayed_work(phy3->mt76->hw,
&phy3->mt76->mac_work,
MT7996_WATCHDOG_TIME);
+ dev_info(dev->mt76.dev,"\n%s L1 SER recovery completed.",
+ wiphy_name(dev->mt76.hw->wiphy));
+}
+
+/* firmware coredump */
+void mt7996_mac_dump_work(struct work_struct *work)
+{
+ const struct mt7996_mem_region *mem_region;
+ struct mt7996_crash_data *crash_data;
+ struct mt7996_dev *dev;
+ struct mt7996_mem_hdr *hdr;
+ size_t buf_len;
+ int i;
+ u32 num;
+ u8 *buf;
+
+ dev = container_of(work, struct mt7996_dev, dump_work);
+
+ mutex_lock(&dev->dump_mutex);
+
+ crash_data = mt7996_coredump_new(dev);
+ if (!crash_data) {
+ mutex_unlock(&dev->dump_mutex);
+ goto skip_coredump;
+ }
+
+ mem_region = mt7996_coredump_get_mem_layout(dev, &num);
+ if (!mem_region || !crash_data->memdump_buf_len) {
+ mutex_unlock(&dev->dump_mutex);
+ goto skip_memdump;
+ }
+
+ buf = crash_data->memdump_buf;
+ buf_len = crash_data->memdump_buf_len;
+
+ /* dumping memory content... */
+ memset(buf, 0, buf_len);
+ for (i = 0; i < num; i++) {
+ if (mem_region->len > buf_len) {
+ dev_warn(dev->mt76.dev, "%s len %zu is too large\n",
+ mem_region->name, mem_region->len);
+ break;
+ }
+
+ /* reserve space for the header */
+ hdr = (void *)buf;
+ buf += sizeof(*hdr);
+ buf_len -= sizeof(*hdr);
+
+ mt7996_memcpy_fromio(dev, buf, mem_region->start,
+ mem_region->len);
+
+ hdr->start = mem_region->start;
+ hdr->len = mem_region->len;
+
+ if (!mem_region->len)
+ /* note: the header remains, just with zero length */
+ break;
+
+ buf += mem_region->len;
+ buf_len -= mem_region->len;
+
+ mem_region++;
+ }
+
+ mutex_unlock(&dev->dump_mutex);
+
+skip_memdump:
+ mt7996_coredump_submit(dev);
+skip_coredump:
+ queue_work(dev->mt76.wq, &dev->reset_work);
+}
+
+void mt7996_reset(struct mt7996_dev *dev)
+{
+ if (!dev->recovery.hw_init_done)
+ return;
+
+ if (dev->recovery.hw_full_reset)
+ return;
+
+ /* wm/wa exception: do full recovery */
+ if (READ_ONCE(dev->recovery.state) & MT_MCU_CMD_WDT_MASK) {
+ dev->recovery.restart = true;
+ dev_info(dev->mt76.dev,
+ "%s indicated firmware crash, attempting recovery\n",
+ wiphy_name(dev->mt76.hw->wiphy));
+
+ mt7996_irq_disable(dev, MT_INT_MCU_CMD);
+ queue_work(dev->mt76.wq, &dev->dump_work);
+ return;
+ }
+
+ queue_work(dev->mt76.wq, &dev->reset_work);
+ wake_up(&dev->reset_wait);
}
void mt7996_mac_update_stats(struct mt7996_phy *phy)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mac.h b/drivers/net/wireless/mediatek/mt76/mt7996/mac.h
index 27184cb..bc4e6c5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mac.h
@@ -12,6 +12,8 @@
#define MT_RXD0_LENGTH GENMASK(15, 0)
#define MT_RXD0_PKT_TYPE GENMASK(31, 27)
+#define MT_RXD0_MESH BIT(18)
+#define MT_RXD0_MHCP BIT(19)
#define MT_RXD0_NORMAL_ETH_TYPE_OFS GENMASK(22, 16)
#define MT_RXD0_NORMAL_IP_SUM BIT(23)
#define MT_RXD0_NORMAL_UDP_TCP_SUM BIT(24)
@@ -20,18 +22,6 @@
#define MT_RXD0_SW_PKT_TYPE_MAP 0x380F
#define MT_RXD0_SW_PKT_TYPE_FRAME 0x3801
-enum rx_pkt_type {
- PKT_TYPE_TXS,
- PKT_TYPE_TXRXV,
- PKT_TYPE_NORMAL,
- PKT_TYPE_RX_DUP_RFB,
- PKT_TYPE_RX_TMR,
- PKT_TYPE_RETRIEVE,
- PKT_TYPE_TXRX_NOTIFY,
- PKT_TYPE_RX_EVENT,
- PKT_TYPE_RX_FW_MONITOR = 0x0c,
-};
-
/* RXD DW1 */
#define MT_RXD1_NORMAL_WLAN_IDX GENMASK(11, 0)
#define MT_RXD1_NORMAL_GROUP_1 BIT(16)
@@ -102,8 +92,7 @@ enum rx_pkt_type {
#define MT_PRXV_NSTS GENMASK(10, 7)
#define MT_PRXV_TXBF BIT(11)
#define MT_PRXV_HT_AD_CODE BIT(12)
-#define MT_PRXV_HE_RU_ALLOC_L GENMASK(31, 28)
-#define MT_PRXV_HE_RU_ALLOC_H GENMASK(3, 0)
+#define MT_PRXV_HE_RU_ALLOC GENMASK(30, 22)
#define MT_PRXV_RCPI3 GENMASK(31, 24)
#define MT_PRXV_RCPI2 GENMASK(23, 16)
#define MT_PRXV_RCPI1 GENMASK(15, 8)
@@ -113,34 +102,32 @@ enum rx_pkt_type {
#define MT_PRXV_TX_MODE GENMASK(14, 11)
#define MT_PRXV_FRAME_MODE GENMASK(2, 0)
#define MT_PRXV_DCM BIT(5)
-#define MT_PRXV_NUM_RX BIT(8, 6)
/* C-RXV */
-#define MT_CRXV_HT_STBC GENMASK(1, 0)
-#define MT_CRXV_TX_MODE GENMASK(7, 4)
-#define MT_CRXV_FRAME_MODE GENMASK(10, 8)
-#define MT_CRXV_HT_SHORT_GI GENMASK(14, 13)
-#define MT_CRXV_HE_LTF_SIZE GENMASK(18, 17)
-#define MT_CRXV_HE_LDPC_EXT_SYM BIT(20)
-#define MT_CRXV_HE_PE_DISAMBIG BIT(23)
-#define MT_CRXV_HE_NUM_USER GENMASK(30, 24)
-#define MT_CRXV_HE_UPLINK BIT(31)
-#define MT_CRXV_HE_RU0 GENMASK(7, 0)
-#define MT_CRXV_HE_RU1 GENMASK(15, 8)
-#define MT_CRXV_HE_RU2 GENMASK(23, 16)
-#define MT_CRXV_HE_RU3 GENMASK(31, 24)
+#define MT_CRXV_HE_NUM_USER GENMASK(26, 20)
+#define MT_CRXV_HE_LTF_SIZE GENMASK(28, 27)
+#define MT_CRXV_HE_LDPC_EXT_SYM BIT(30)
-#define MT_CRXV_HE_MU_AID GENMASK(30, 20)
+#define MT_CRXV_HE_PE_DISAMBIG BIT(1)
+#define MT_CRXV_HE_UPLINK BIT(2)
+
+#define MT_CRXV_HE_MU_AID GENMASK(27, 17)
+#define MT_CRXV_HE_BEAM_CHNG BIT(29)
+
+#define MT_CRXV_HE_DOPPLER BIT(0)
+#define MT_CRXV_HE_BSS_COLOR GENMASK(15, 10)
+#define MT_CRXV_HE_TXOP_DUR GENMASK(19, 17)
#define MT_CRXV_HE_SR_MASK GENMASK(11, 8)
#define MT_CRXV_HE_SR1_MASK GENMASK(16, 12)
#define MT_CRXV_HE_SR2_MASK GENMASK(20, 17)
#define MT_CRXV_HE_SR3_MASK GENMASK(24, 21)
-#define MT_CRXV_HE_BSS_COLOR GENMASK(5, 0)
-#define MT_CRXV_HE_TXOP_DUR GENMASK(12, 6)
-#define MT_CRXV_HE_BEAM_CHNG BIT(13)
-#define MT_CRXV_HE_DOPPLER BIT(16)
+#define MT_CRXV_HE_RU0 GENMASK(8, 0)
+#define MT_CRXV_HE_RU1 GENMASK(17, 9)
+#define MT_CRXV_HE_RU2 GENMASK(26, 18)
+#define MT_CRXV_HE_RU3_L GENMASK(31, 27)
+#define MT_CRXV_HE_RU3_H GENMASK(3, 0)
enum tx_header_format {
MT_HDR_FORMAT_802_3,
@@ -239,14 +226,11 @@ enum tx_mgnt_type {
#define MT_TXD6_TX_SRC GENMASK(31, 30)
#define MT_TXD6_VTA BIT(28)
-#define MT_TXD6_FIXED_BW BIT(25)
-#define MT_TXD6_BW GENMASK(24, 22)
+#define MT_TXD6_BW GENMASK(25, 22)
#define MT_TXD6_TX_RATE GENMASK(21, 16)
#define MT_TXD6_TIMESTAMP_OFS_EN BIT(15)
#define MT_TXD6_TIMESTAMP_OFS_IDX GENMASK(14, 10)
#define MT_TXD6_MSDU_CNT GENMASK(9, 4)
-#define MT_TXD6_SPE_ID_IDX BIT(10)
-#define MT_TXD6_ANT_ID GENMASK(7, 4)
#define MT_TXD6_DIS_MAT BIT(3)
#define MT_TXD6_DAS BIT(2)
#define MT_TXD6_AMSDU_CAP BIT(1)
@@ -260,7 +244,7 @@ enum tx_mgnt_type {
#define MT_TXD7_UDP_TCP_SUM BIT(15)
#define MT_TXD7_TX_TIME GENMASK(9, 0)
-#define MT_TX_RATE_STBC BIT(13)
+#define MT_TX_RATE_STBC BIT(14)
#define MT_TX_RATE_NSS GENMASK(13, 10)
#define MT_TX_RATE_MODE GENMASK(9, 6)
#define MT_TX_RATE_SU_EXT_TONE BIT(5)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/main.c b/drivers/net/wireless/mediatek/mt76/mt7996/main.c
index 1ba22d1..f306e9c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/main.c
@@ -5,6 +5,7 @@
#include "mt7996.h"
#include "mcu.h"
+#include "mac.h"
static bool mt7996_dev_running(struct mt7996_dev *dev)
{
@@ -22,17 +23,13 @@ static bool mt7996_dev_running(struct mt7996_dev *dev)
return phy && test_bit(MT76_STATE_RUNNING, &phy->mt76->state);
}
-static int mt7996_start(struct ieee80211_hw *hw)
+int mt7996_run(struct ieee80211_hw *hw)
{
struct mt7996_dev *dev = mt7996_hw_dev(hw);
struct mt7996_phy *phy = mt7996_hw_phy(hw);
bool running;
int ret;
- flush_work(&dev->init_work);
-
- mutex_lock(&dev->mt76.mutex);
-
running = mt7996_dev_running(dev);
if (!running) {
ret = mt7996_mcu_set_hdr_trans(dev, true);
@@ -52,10 +49,6 @@ static int mt7996_start(struct ieee80211_hw *hw)
set_bit(MT76_STATE_RUNNING, &phy->mt76->state);
- ieee80211_iterate_interfaces(dev->mt76.hw,
- IEEE80211_IFACE_ITER_RESUME_ALL,
- mt7996_mcu_set_pm, dev->mt76.hw);
-
ieee80211_queue_delayed_work(hw, &phy->mt76->mac_work,
MT7996_WATCHDOG_TIME);
@@ -63,6 +56,18 @@ static int mt7996_start(struct ieee80211_hw *hw)
mt7996_mac_reset_counters(phy);
out:
+ return ret;
+}
+
+static int mt7996_start(struct ieee80211_hw *hw)
+{
+ struct mt7996_dev *dev = mt7996_hw_dev(hw);
+ int ret;
+
+ flush_work(&dev->init_work);
+
+ mutex_lock(&dev->mt76.mutex);
+ ret = mt7996_run(hw);
mutex_unlock(&dev->mt76.mutex);
return ret;
@@ -79,10 +84,6 @@ static void mt7996_stop(struct ieee80211_hw *hw)
clear_bit(MT76_STATE_RUNNING, &phy->mt76->state);
- ieee80211_iterate_interfaces(dev->mt76.hw,
- IEEE80211_IFACE_ITER_RESUME_ALL,
- mt7996_mcu_set_pm, dev->mt76.hw);
-
mutex_unlock(&dev->mt76.mutex);
}
@@ -219,8 +220,12 @@ static int mt7996_add_interface(struct ieee80211_hw *hw,
vif->offload_flags = 0;
vif->offload_flags |= IEEE80211_OFFLOAD_ENCAP_4ADDR;
+ if (phy->mt76->chandef.chan->band != NL80211_BAND_2GHZ)
+ mvif->basic_rates_idx = MT7996_BASIC_RATES_TBL + 4;
+ else
+ mvif->basic_rates_idx = MT7996_BASIC_RATES_TBL;
+
mt7996_init_bitrate_mask(vif);
- memset(&mvif->cap, -1, sizeof(mvif->cap));
mt7996_mcu_add_bss_info(phy, vif, true);
mt7996_mcu_add_sta(dev, vif, NULL, true);
@@ -496,11 +501,41 @@ mt7996_update_bss_color(struct ieee80211_hw *hw,
}
}
+static u8
+mt7996_get_rates_table(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ bool beacon, bool mcast)
+{
+ struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
+ struct mt76_phy *mphy = hw->priv;
+ u16 rate;
+ u8 i, idx, ht;
+
+ rate = mt76_connac2_mac_tx_rate_val(mphy, vif, beacon, mcast);
+ ht = FIELD_GET(MT_TX_RATE_MODE, rate) > MT_PHY_TYPE_OFDM;
+
+ if (beacon && ht) {
+ struct mt7996_dev *dev = mt7996_hw_dev(hw);
+
+ /* must odd index */
+ idx = MT7996_BEACON_RATES_TBL + 2 * (mvif->mt76.idx % 20);
+ mt7996_mac_set_fixed_rate_table(dev, idx, rate);
+ return idx;
+ }
+
+ idx = FIELD_GET(MT_TX_RATE_IDX, rate);
+ for (i = 0; i < ARRAY_SIZE(mt76_rates); i++)
+ if ((mt76_rates[i].hw_value & GENMASK(7, 0)) == idx)
+ return MT7996_BASIC_RATES_TBL + i;
+
+ return mvif->basic_rates_idx;
+}
+
static void mt7996_bss_info_changed(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_bss_conf *info,
u64 changed)
{
+ struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
struct mt7996_phy *phy = mt7996_hw_phy(hw);
struct mt7996_dev *dev = mt7996_hw_dev(hw);
@@ -532,6 +567,14 @@ static void mt7996_bss_info_changed(struct ieee80211_hw *hw,
}
}
+ if (changed & BSS_CHANGED_MCAST_RATE)
+ mvif->mcast_rates_idx =
+ mt7996_get_rates_table(hw, vif, false, true);
+
+ if (changed & BSS_CHANGED_BASIC_RATES)
+ mvif->basic_rates_idx =
+ mt7996_get_rates_table(hw, vif, false, false);
+
if (changed & BSS_CHANGED_BEACON_ENABLED && info->enable_beacon) {
mt7996_mcu_add_bss_info(phy, vif, true);
mt7996_mcu_add_sta(dev, vif, NULL, true);
@@ -548,8 +591,12 @@ static void mt7996_bss_info_changed(struct ieee80211_hw *hw,
mt7996_update_bss_color(hw, vif, &info->he_bss_color);
if (changed & (BSS_CHANGED_BEACON |
- BSS_CHANGED_BEACON_ENABLED))
+ BSS_CHANGED_BEACON_ENABLED)) {
+ mvif->beacon_rates_idx =
+ mt7996_get_rates_table(hw, vif, true, false);
+
mt7996_mcu_add_beacon(hw, vif, info->enable_beacon);
+ }
if (changed & BSS_CHANGED_UNSOL_BCAST_PROBE_RESP ||
changed & BSS_CHANGED_FILS_DISCOVERY)
@@ -891,6 +938,7 @@ mt7996_set_antenna(struct ieee80211_hw *hw, u32 tx_ant, u32 rx_ant)
mt7996_set_stream_vht_txbf_caps(phy);
mt7996_set_stream_he_eht_caps(phy);
+ /* TODO: update bmc_wtbl spe_idx when antenna changes */
mutex_unlock(&dev->mt76.mutex);
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
index dbe3083..88e2f9d 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
@@ -422,7 +422,8 @@ mt7996_mcu_ie_countdown(struct mt7996_dev *dev, struct sk_buff *skb)
if (hdr->band && dev->mt76.phys[hdr->band])
mphy = dev->mt76.phys[hdr->band];
- tail = skb->data + le16_to_cpu(rxd->len);
+ tail = skb->data + skb->len;
+ data += sizeof(struct header);
while (data + sizeof(struct tlv) < tail && le16_to_cpu(tlv->len)) {
switch (le16_to_cpu(tlv->tag)) {
case UNI_EVENT_IE_COUNTDOWN_CSA:
@@ -596,25 +597,24 @@ mt7996_mcu_bss_he_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
}
static void
-mt7996_mcu_bss_bmc_tlv(struct sk_buff *skb, struct mt7996_phy *phy)
+mt7996_mcu_bss_bmc_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
+ struct mt7996_phy *phy)
{
+ struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
struct bss_rate_tlv *bmc;
struct cfg80211_chan_def *chandef = &phy->mt76->chandef;
enum nl80211_band band = chandef->chan->band;
struct tlv *tlv;
+ u8 idx = mvif->mcast_rates_idx ?
+ mvif->mcast_rates_idx : mvif->basic_rates_idx;
tlv = mt7996_mcu_add_uni_tlv(skb, UNI_BSS_INFO_RATE, sizeof(*bmc));
bmc = (struct bss_rate_tlv *)tlv;
- if (band == NL80211_BAND_2GHZ) {
- bmc->short_preamble = true;
- } else {
- bmc->bc_trans = cpu_to_le16(0x8080);
- bmc->mc_trans = cpu_to_le16(0x8080);
- bmc->bc_fixed_rate = 1;
- bmc->mc_fixed_rate = 1;
- bmc->short_preamble = 1;
- }
+
+ bmc->short_preamble = (band == NL80211_BAND_2GHZ);
+ bmc->bc_fixed_rate = idx;
+ bmc->mc_fixed_rate = idx;
}
static void
@@ -822,7 +822,7 @@ int mt7996_mcu_add_bss_info(struct mt7996_phy *phy,
if (enable) {
mt7996_mcu_bss_rfch_tlv(skb, vif, phy);
- mt7996_mcu_bss_bmc_tlv(skb, phy);
+ mt7996_mcu_bss_bmc_tlv(skb, vif, phy);
mt7996_mcu_bss_ra_tlv(skb, vif, phy);
mt7996_mcu_bss_txcmd_tlv(skb, true);
@@ -1022,6 +1022,7 @@ mt7996_mcu_sta_amsdu_tlv(struct mt7996_dev *dev, struct sk_buff *skb,
struct tlv *tlv;
if (vif->type != NL80211_IFTYPE_STATION &&
+ vif->type != NL80211_IFTYPE_MESH_POINT &&
vif->type != NL80211_IFTYPE_AP)
return;
@@ -1053,7 +1054,6 @@ static inline bool
mt7996_is_ebf_supported(struct mt7996_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, bool bfee)
{
- struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
int sts = hweight16(phy->mt76->chainmask);
if (vif->type != NL80211_IFTYPE_STATION &&
@@ -1068,10 +1068,10 @@ mt7996_is_ebf_supported(struct mt7996_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_eht_cap_elem_fixed *pe = &pc->eht_cap_elem;
if (bfee)
- return mvif->cap.eht_su_ebfee &&
+ return vif->bss_conf.eht_su_beamformee &&
EHT_PHY(CAP0_SU_BEAMFORMEE, pe->phy_cap_info[0]);
else
- return mvif->cap.eht_su_ebfer &&
+ return vif->bss_conf.eht_su_beamformer &&
EHT_PHY(CAP0_SU_BEAMFORMER, pe->phy_cap_info[0]);
}
@@ -1079,10 +1079,10 @@ mt7996_is_ebf_supported(struct mt7996_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_he_cap_elem *pe = &sta->deflink.he_cap.he_cap_elem;
if (bfee)
- return mvif->cap.he_su_ebfee &&
+ return vif->bss_conf.he_su_beamformee &&
HE_PHY(CAP3_SU_BEAMFORMER, pe->phy_cap_info[3]);
else
- return mvif->cap.he_su_ebfer &&
+ return vif->bss_conf.he_su_beamformer &&
HE_PHY(CAP4_SU_BEAMFORMEE, pe->phy_cap_info[4]);
}
@@ -1090,10 +1090,10 @@ mt7996_is_ebf_supported(struct mt7996_phy *phy, struct ieee80211_vif *vif,
u32 cap = sta->deflink.vht_cap.cap;
if (bfee)
- return mvif->cap.vht_su_ebfee &&
+ return vif->bss_conf.vht_su_beamformee &&
(cap & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE);
else
- return mvif->cap.vht_su_ebfer &&
+ return vif->bss_conf.vht_su_beamformer &&
(cap & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE);
}
@@ -1471,6 +1471,12 @@ mt7996_mcu_sta_hdr_trans_tlv(struct mt7996_dev *dev, struct sk_buff *skb,
hdr_trans->to_ds = true;
hdr_trans->from_ds = true;
}
+
+ if (vif->type == NL80211_IFTYPE_MESH_POINT) {
+ hdr_trans->to_ds = true;
+ hdr_trans->from_ds = true;
+ hdr_trans->mesh = true;
+ }
}
static enum mcu_mmps_mode
@@ -1572,7 +1578,7 @@ mt7996_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7996_dev *dev,
cap |= STA_CAP_TX_STBC;
if (sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_RX_STBC)
cap |= STA_CAP_RX_STBC;
- if (mvif->cap.ht_ldpc &&
+ if (vif->bss_conf.ht_ldpc &&
(sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_LDPC_CODING))
cap |= STA_CAP_LDPC;
@@ -1598,7 +1604,7 @@ mt7996_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7996_dev *dev,
cap |= STA_CAP_VHT_TX_STBC;
if (sta->deflink.vht_cap.cap & IEEE80211_VHT_CAP_RXSTBC_1)
cap |= STA_CAP_VHT_RX_STBC;
- if (mvif->cap.vht_ldpc &&
+ if (vif->bss_conf.vht_ldpc &&
(sta->deflink.vht_cap.cap & IEEE80211_VHT_CAP_RXLDPC))
cap |= STA_CAP_VHT_LDPC;
@@ -1694,8 +1700,8 @@ int mt7996_mcu_add_sta(struct mt7996_dev *dev, struct ieee80211_vif *vif,
return PTR_ERR(skb);
/* starec basic */
- mt76_connac_mcu_sta_basic_tlv(skb, vif, sta, enable,
- !rcu_access_pointer(dev->mt76.wcid[msta->wcid.idx]));
+ mt76_connac_mcu_sta_basic_tlv(&dev->mt76, skb, vif, sta, enable,
+ !rcu_access_pointer(dev->mt76.wcid[msta->wcid.idx]));
if (!enable)
goto out;
@@ -1906,107 +1912,12 @@ mt7996_mcu_beacon_cont(struct mt7996_dev *dev, struct ieee80211_vif *vif,
}
buf = (u8 *)bcn + sizeof(*bcn) - MAX_BEACON_SIZE;
- mt7996_mac_write_txwi(dev, (__le32 *)buf, skb, wcid, 0, NULL,
+ mt7996_mac_write_txwi(dev, (__le32 *)buf, skb, wcid, NULL, 0, 0,
BSS_CHANGED_BEACON);
+
memcpy(buf + MT_TXD_SIZE, skb->data, skb->len);
}
-static void
-mt7996_mcu_beacon_check_caps(struct mt7996_phy *phy, struct ieee80211_vif *vif,
- struct sk_buff *skb)
-{
- struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
- struct mt7996_vif_cap *vc = &mvif->cap;
- const struct ieee80211_eht_cap_elem_fixed *eht;
- const struct ieee80211_he_cap_elem *he;
- const struct ieee80211_vht_cap *vht;
- const struct ieee80211_ht_cap *ht;
- struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)skb->data;
- const u8 *ie;
- u32 len, bc;
-
- /* Check missing configuration options to allow AP mode in mac80211
- * to remain in sync with hostapd settings, and get a subset of
- * beacon and hardware capabilities.
- */
- if (WARN_ON_ONCE(skb->len <= (mgmt->u.beacon.variable - skb->data)))
- return;
-
- memset(vc, 0, sizeof(*vc));
-
- len = skb->len - (mgmt->u.beacon.variable - skb->data);
-
- ie = cfg80211_find_ie(WLAN_EID_HT_CAPABILITY, mgmt->u.beacon.variable,
- len);
- if (ie && ie[1] >= sizeof(*ht)) {
- ht = (void *)(ie + 2);
- vc->ht_ldpc |= !!(le16_to_cpu(ht->cap_info) &
- IEEE80211_HT_CAP_LDPC_CODING);
- }
-
- ie = cfg80211_find_ie(WLAN_EID_VHT_CAPABILITY, mgmt->u.beacon.variable,
- len);
- if (ie && ie[1] >= sizeof(*vht)) {
- u32 pc = phy->mt76->sband_5g.sband.vht_cap.cap;
-
- vht = (void *)(ie + 2);
- bc = le32_to_cpu(vht->vht_cap_info);
-
- vc->vht_ldpc |= !!(bc & IEEE80211_VHT_CAP_RXLDPC);
- vc->vht_su_ebfer =
- (bc & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE);
- vc->vht_su_ebfee =
- (bc & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE);
- vc->vht_mu_ebfer =
- (bc & IEEE80211_VHT_CAP_MU_BEAMFORMER_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_MU_BEAMFORMER_CAPABLE);
- vc->vht_mu_ebfee =
- (bc & IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE) &&
- (pc & IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE);
- }
-
- ie = cfg80211_find_ext_ie(WLAN_EID_EXT_HE_CAPABILITY,
- mgmt->u.beacon.variable, len);
- if (ie && ie[1] >= sizeof(*he) + 1) {
- const struct ieee80211_sta_he_cap *pc =
- mt76_connac_get_he_phy_cap(phy->mt76, vif);
- const struct ieee80211_he_cap_elem *pe = &pc->he_cap_elem;
-
- he = (void *)(ie + 3);
-
- vc->he_ldpc =
- HE_PHY(CAP1_LDPC_CODING_IN_PAYLOAD, pe->phy_cap_info[1]);
- vc->he_su_ebfer =
- HE_PHY(CAP3_SU_BEAMFORMER, he->phy_cap_info[3]) &&
- HE_PHY(CAP3_SU_BEAMFORMER, pe->phy_cap_info[3]);
- vc->he_su_ebfee =
- HE_PHY(CAP4_SU_BEAMFORMEE, he->phy_cap_info[4]) &&
- HE_PHY(CAP4_SU_BEAMFORMEE, pe->phy_cap_info[4]);
- vc->he_mu_ebfer =
- HE_PHY(CAP4_MU_BEAMFORMER, he->phy_cap_info[4]) &&
- HE_PHY(CAP4_MU_BEAMFORMER, pe->phy_cap_info[4]);
- }
-
- ie = cfg80211_find_ext_ie(WLAN_EID_EXT_EHT_CAPABILITY,
- mgmt->u.beacon.variable, len);
- if (ie && ie[1] >= sizeof(*eht) + 1) {
- const struct ieee80211_sta_eht_cap *pc =
- mt76_connac_get_eht_phy_cap(phy->mt76, vif);
- const struct ieee80211_eht_cap_elem_fixed *pe = &pc->eht_cap_elem;
-
- eht = (void *)(ie + 3);
-
- vc->eht_su_ebfer =
- EHT_PHY(CAP0_SU_BEAMFORMER, eht->phy_cap_info[0]) &&
- EHT_PHY(CAP0_SU_BEAMFORMER, pe->phy_cap_info[0]);
- vc->eht_su_ebfee =
- EHT_PHY(CAP0_SU_BEAMFORMEE, eht->phy_cap_info[0]) &&
- EHT_PHY(CAP0_SU_BEAMFORMEE, pe->phy_cap_info[0]);
- }
-}
-
int mt7996_mcu_add_beacon(struct ieee80211_hw *hw,
struct ieee80211_vif *vif, int en)
{
@@ -2045,8 +1956,6 @@ int mt7996_mcu_add_beacon(struct ieee80211_hw *hw,
info = IEEE80211_SKB_CB(skb);
info->hw_queue |= FIELD_PREP(MT_TX_HW_QUEUE_PHY, phy->mt76->band_idx);
- mt7996_mcu_beacon_check_caps(phy, vif, skb);
-
mt7996_mcu_beacon_cont(dev, vif, rskb, skb, bcn, &offs);
/* TODO: subtag - 11v MBSSID */
mt7996_mcu_beacon_cntdwn(vif, rskb, skb, &offs);
@@ -2115,8 +2024,7 @@ int mt7996_mcu_beacon_inband_discov(struct mt7996_dev *dev,
buf = (u8 *)tlv + sizeof(*discov) - MAX_INBAND_FRAME_SIZE;
- mt7996_mac_write_txwi(dev, (__le32 *)buf, skb, wcid, 0, NULL,
- changed);
+ mt7996_mac_write_txwi(dev, (__le32 *)buf, skb, wcid, NULL, 0, 0, changed);
memcpy(buf + MT_TXD_SIZE, skb->data, skb->len);
@@ -2523,17 +2431,10 @@ mt7996_mcu_init_rx_airtime(struct mt7996_dev *dev)
MCU_WM_UNI_CMD(VOW), true);
}
-int mt7996_mcu_init(struct mt7996_dev *dev)
+int mt7996_mcu_init_firmware(struct mt7996_dev *dev)
{
- static const struct mt76_mcu_ops mt7996_mcu_ops = {
- .headroom = sizeof(struct mt76_connac2_mcu_txd), /* reuse */
- .mcu_skb_send_msg = mt7996_mcu_send_message,
- .mcu_parse_response = mt7996_mcu_parse_response,
- };
int ret;
- dev->mt76.mcu_ops = &mt7996_mcu_ops;
-
/* force firmware operation mode into normal state,
* which should be set before firmware download stage.
*/
@@ -2574,6 +2475,19 @@ int mt7996_mcu_init(struct mt7996_dev *dev)
MCU_WA_PARAM_RED, 0, 0);
}
+int mt7996_mcu_init(struct mt7996_dev *dev)
+{
+ static const struct mt76_mcu_ops mt7996_mcu_ops = {
+ .headroom = sizeof(struct mt76_connac2_mcu_txd), /* reuse */
+ .mcu_skb_send_msg = mt7996_mcu_send_message,
+ .mcu_parse_response = mt7996_mcu_parse_response,
+ };
+
+ dev->mt76.mcu_ops = &mt7996_mcu_ops;
+
+ return mt7996_mcu_init_firmware(dev);
+}
+
void mt7996_mcu_exit(struct mt7996_dev *dev)
{
mt7996_mcu_restart(&dev->mt76);
@@ -3133,7 +3047,7 @@ int mt7996_mcu_get_chip_config(struct mt7996_dev *dev, u32 *cap)
break;
default:
break;
- };
+ }
buf += le16_to_cpu(tlv->len);
}
@@ -3576,32 +3490,6 @@ int mt7996_mcu_twt_agrt_update(struct mt7996_dev *dev,
&req, sizeof(req), true);
}
-void mt7996_mcu_set_pm(void *priv, u8 *mac, struct ieee80211_vif *vif)
-{
-#define EXIT_PM_STATE 0
-#define ENTER_PM_STATE 1
- struct ieee80211_hw *hw = priv;
- struct mt7996_dev *dev = mt7996_hw_dev(hw);
- struct mt7996_phy *phy = mt7996_hw_phy(hw);
- struct mt7996_vif *mvif = (struct mt7996_vif *)vif->drv_priv;
- struct bss_power_save *ps;
- struct sk_buff *skb;
- struct tlv *tlv;
- bool running = test_bit(MT76_STATE_RUNNING, &phy->mt76->state);
-
- skb = __mt7996_mcu_alloc_bss_req(&dev->mt76, &mvif->mt76,
- MT7996_BSS_UPDATE_MAX_SIZE);
- if (IS_ERR(skb))
- return;
-
- tlv = mt7996_mcu_add_uni_tlv(skb, UNI_BSS_INFO_PS, sizeof(*ps));
- ps = (struct bss_power_save *)tlv;
- ps->profile = running ? EXIT_PM_STATE : ENTER_PM_STATE;
-
- mt76_mcu_skb_send_msg(&dev->mt76, skb,
- MCU_WMWA_UNI_CMD(BSS_INFO_UPDATE), true);
-}
-
int mt7996_mcu_set_rts_thresh(struct mt7996_phy *phy, u32 val)
{
struct {
@@ -3733,6 +3621,22 @@ int mt7996_mcu_rf_regval(struct mt7996_dev *dev, u32 regidx, u32 *val, bool set)
return 0;
}
+int mt7996_mcu_trigger_assert(struct mt7996_dev *dev)
+{
+ struct {
+ __le16 tag;
+ __le16 len;
+ u8 enable;
+ u8 rsv[3];
+ } __packed req = {
+ .len = cpu_to_le16(sizeof(req) - 4),
+ .enable = true,
+ };
+
+ return mt76_mcu_send_msg(&dev->mt76, MCU_WM_UNI_CMD(ASSERT_DUMP),
+ &req, sizeof(req), false);
+}
+
int mt7996_mcu_set_rro(struct mt7996_dev *dev, u16 tag, u8 val)
{
struct {
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
index dd0c5ac..d7075a4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
@@ -396,7 +396,7 @@ struct sta_rec_hdr_trans {
u8 from_ds;
u8 to_ds;
u8 dis_rx_hdr_tran;
- u8 rsv;
+ u8 mesh;
} __packed;
struct hdr_trans_en {
@@ -648,23 +648,21 @@ enum {
};
enum {
- UNI_CMD_SER_QUERY = 0x0,
- UNI_CMD_SER_SET = 0x2,
- UNI_CMD_SER_TRIGGER = 0x3,
-};
-
-enum {
- SER_QUERY,
+ UNI_CMD_SER_QUERY,
/* recovery */
- SER_SET_RECOVER_L1,
- SER_SET_RECOVER_L2,
- SER_SET_RECOVER_L3_RX_ABORT,
- SER_SET_RECOVER_L3_TX_ABORT,
- SER_SET_RECOVER_L3_TX_DISABLE,
- SER_SET_RECOVER_L3_BF,
+ UNI_CMD_SER_SET_RECOVER_L1,
+ UNI_CMD_SER_SET_RECOVER_L2,
+ UNI_CMD_SER_SET_RECOVER_L3_RX_ABORT,
+ UNI_CMD_SER_SET_RECOVER_L3_TX_ABORT,
+ UNI_CMD_SER_SET_RECOVER_L3_TX_DISABLE,
+ UNI_CMD_SER_SET_RECOVER_L3_BF,
+ UNI_CMD_SER_SET_RECOVER_L4_MDP,
+ UNI_CMD_SER_SET_RECOVER_FULL,
+ UNI_CMD_SER_SET_SYSTEM_ASSERT,
/* action */
- SER_ENABLE = 2,
- SER_RECOVER
+ UNI_CMD_SER_ENABLE = 1,
+ UNI_CMD_SER_SET,
+ UNI_CMD_SER_TRIGGER
};
enum {
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
index 902370a..3a591a7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
@@ -162,6 +162,14 @@ static u32 __mt7996_reg_addr(struct mt7996_dev *dev, u32 addr)
return mt7996_reg_map_l2(dev, addr);
}
+void mt7996_memcpy_fromio(struct mt7996_dev *dev, void *buf, u32 offset,
+ size_t len)
+{
+ u32 addr = __mt7996_reg_addr(dev, offset);
+
+ memcpy_fromio(buf, dev->mt76.mmio.regs + addr, len);
+}
+
static u32 mt7996_rr(struct mt76_dev *mdev, u32 offset)
{
struct mt7996_dev *dev = container_of(mdev, struct mt7996_dev, mt76);
@@ -251,7 +259,7 @@ static void mt7996_rx_poll_complete(struct mt76_dev *mdev,
/* TODO: support 2/4/6/8 MSI-X vectors */
static void mt7996_irq_tasklet(struct tasklet_struct *t)
{
- struct mt7996_dev *dev = from_tasklet(dev, t, irq_tasklet);
+ struct mt7996_dev *dev = from_tasklet(dev, t, mt76.irq_tasklet);
u32 i, intr, mask, intr1;
mt76_wr(dev, MT_INT_MASK_CSR, 0);
@@ -289,10 +297,9 @@ static void mt7996_irq_tasklet(struct tasklet_struct *t)
u32 val = mt76_rr(dev, MT_MCU_CMD);
mt76_wr(dev, MT_MCU_CMD, val);
- if (val & MT_MCU_CMD_ERROR_MASK) {
- dev->reset_state = val;
- ieee80211_queue_work(mt76_hw(dev), &dev->reset_work);
- wake_up(&dev->reset_wait);
+ if (val & (MT_MCU_CMD_ERROR_MASK | MT_MCU_CMD_WDT_MASK)) {
+ dev->recovery.state = val;
+ mt7996_reset(dev);
}
}
}
@@ -308,7 +315,7 @@ irqreturn_t mt7996_irq_handler(int irq, void *dev_instance)
if (!test_bit(MT76_STATE_INITIALIZED, &dev->mphy.state))
return IRQ_NONE;
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
return IRQ_HANDLED;
}
@@ -320,6 +327,7 @@ struct mt7996_dev *mt7996_mmio_probe(struct device *pdev,
/* txwi_size = txd size + txp size */
.txwi_size = MT_TXD_SIZE + sizeof(struct mt76_connac_fw_txp),
.drv_flags = MT_DRV_TXWI_NO_FREE |
+ MT_DRV_AMSDU_OFFLOAD |
MT_DRV_HW_MGMT_TXQ,
.survey_flags = SURVEY_INFO_TIME_TX |
SURVEY_INFO_TIME_RX |
@@ -330,7 +338,6 @@ struct mt7996_dev *mt7996_mmio_probe(struct device *pdev,
.rx_skb = mt7996_queue_rx_skb,
.rx_check = mt7996_rx_check,
.rx_poll_complete = mt7996_rx_poll_complete,
- .sta_ps = mt7996_sta_ps,
.sta_add = mt7996_mac_sta_add,
.sta_remove = mt7996_mac_sta_remove,
.update_survey = mt7996_update_channel,
@@ -349,7 +356,7 @@ struct mt7996_dev *mt7996_mmio_probe(struct device *pdev,
if (ret)
goto error;
- tasklet_setup(&dev->irq_tasklet, mt7996_irq_tasklet);
+ tasklet_setup(&mdev->irq_tasklet, mt7996_irq_tasklet);
mt76_wr(dev, MT_INT_MASK_CSR, 0);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mt7996.h b/drivers/net/wireless/mediatek/mt76/mt7996/mt7996.h
index 018dfd2..4d7dcb9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mt7996.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mt7996.h
@@ -43,6 +43,10 @@
#define MT7996_MAX_STA_TWT_AGRT 8
#define MT7996_MAX_QUEUE (__MT_RXQ_MAX + __MT_MCUQ_MAX + 3)
+/* NOTE: used to map mt76_rates. idx may change if firmware expands table */
+#define MT7996_BASIC_RATES_TBL 11
+#define MT7996_BEACON_RATES_TBL 25
+
struct mt7996_vif;
struct mt7996_sta;
struct mt7996_dfs_pulse;
@@ -112,30 +116,18 @@ struct mt7996_sta {
} twt;
};
-struct mt7996_vif_cap {
- bool ht_ldpc:1;
- bool vht_ldpc:1;
- bool he_ldpc:1;
- bool vht_su_ebfer:1;
- bool vht_su_ebfee:1;
- bool vht_mu_ebfer:1;
- bool vht_mu_ebfee:1;
- bool he_su_ebfer:1;
- bool he_su_ebfee:1;
- bool he_mu_ebfer:1;
- bool eht_su_ebfer:1;
- bool eht_su_ebfee:1;
-};
-
struct mt7996_vif {
struct mt76_vif mt76; /* must be first */
- struct mt7996_vif_cap cap;
struct mt7996_sta sta;
struct mt7996_phy *phy;
struct ieee80211_tx_queue_params queue_params[IEEE80211_NUM_ACS];
struct cfg80211_bitrate_mask bitrate_mask;
+
+ u8 basic_rates_idx;
+ u8 mcast_rates_idx;
+ u8 beacon_rates_idx;
};
/* per-phy stats. */
@@ -192,6 +184,15 @@ struct mib_stats {
u32 tx_amsdu_cnt;
};
+/* crash-dump */
+struct mt7996_crash_data {
+ guid_t guid;
+ struct timespec64 timestamp;
+
+ u8 *memdump_buf;
+ size_t memdump_buf_len;
+};
+
struct mt7996_hif {
struct list_head list;
@@ -238,7 +239,6 @@ struct mt7996_dev {
u32 q_wfdma_mask;
const struct mt76_bus_ops *bus_ops;
- struct tasklet_struct irq_tasklet;
struct mt7996_phy phy;
/* monitor rx chain configured channel */
@@ -251,9 +251,25 @@ struct mt7996_dev {
struct work_struct init_work;
struct work_struct rc_work;
+ struct work_struct dump_work;
struct work_struct reset_work;
wait_queue_head_t reset_wait;
- u32 reset_state;
+ struct {
+ u32 state;
+ u32 wa_reset_count;
+ u32 wm_reset_count;
+ bool hw_full_reset:1;
+ bool hw_init_done:1;
+ bool restart:1;
+ } recovery;
+
+ /* protects coredump data */
+ struct mutex dump_mutex;
+#ifdef CONFIG_DEV_COREDUMP
+ struct {
+ struct mt7996_crash_data *crash_data;
+ } coredump;
+#endif
struct list_head sta_rc_list;
struct list_head sta_poll_list;
@@ -386,9 +402,16 @@ int mt7996_eeprom_get_target_power(struct mt7996_dev *dev,
struct ieee80211_channel *chan);
s8 mt7996_eeprom_get_power_delta(struct mt7996_dev *dev, int band);
int mt7996_dma_init(struct mt7996_dev *dev);
+void mt7996_dma_reset(struct mt7996_dev *dev, bool force);
void mt7996_dma_prefetch(struct mt7996_dev *dev);
void mt7996_dma_cleanup(struct mt7996_dev *dev);
+void mt7996_init_txpower(struct mt7996_dev *dev,
+ struct ieee80211_supported_band *sband);
+int mt7996_txbf_init(struct mt7996_dev *dev);
+void mt7996_reset(struct mt7996_dev *dev);
+int mt7996_run(struct ieee80211_hw *hw);
int mt7996_mcu_init(struct mt7996_dev *dev);
+int mt7996_mcu_init_firmware(struct mt7996_dev *dev);
int mt7996_mcu_twt_agrt_update(struct mt7996_dev *dev,
struct mt7996_vif *mvif,
struct mt7996_twt_flow *flow,
@@ -432,7 +455,6 @@ int mt7996_mcu_set_pulse_th(struct mt7996_dev *dev,
int mt7996_mcu_set_radar_th(struct mt7996_dev *dev, int index,
const struct mt7996_dfs_pattern *pattern);
int mt7996_mcu_set_radio_en(struct mt7996_phy *phy, bool enable);
-void mt7996_mcu_set_pm(void *priv, u8 *mac, struct ieee80211_vif *vif);
int mt7996_mcu_set_rts_thresh(struct mt7996_phy *phy, u32 val);
int mt7996_mcu_get_chan_mib_info(struct mt7996_phy *phy, bool chan_switch);
int mt7996_mcu_rdd_cmd(struct mt7996_dev *dev, int cmd, u8 index,
@@ -445,6 +467,7 @@ int mt7996_mcu_set_rro(struct mt7996_dev *dev, u16 tag, u8 val);
int mt7996_mcu_wa_cmd(struct mt7996_dev *dev, int cmd, u32 a1, u32 a2, u32 a3);
int mt7996_mcu_fw_log_2_host(struct mt7996_dev *dev, u8 type, u8 ctrl);
int mt7996_mcu_fw_dbg_ctrl(struct mt7996_dev *dev, u32 module, u8 level);
+int mt7996_mcu_trigger_assert(struct mt7996_dev *dev);
void mt7996_mcu_rx_event(struct mt7996_dev *dev, struct sk_buff *skb);
void mt7996_mcu_exit(struct mt7996_dev *dev);
@@ -468,7 +491,7 @@ static inline void mt7996_irq_enable(struct mt7996_dev *dev, u32 mask)
else
mt76_set_irq_mask(&dev->mt76, 0, 0, mask);
- tasklet_schedule(&dev->irq_tasklet);
+ tasklet_schedule(&dev->mt76.irq_tasklet);
}
static inline void mt7996_irq_disable(struct mt7996_dev *dev, u32 mask)
@@ -479,6 +502,10 @@ static inline void mt7996_irq_disable(struct mt7996_dev *dev, u32 mask)
mt76_set_irq_mask(&dev->mt76, MT_INT_MASK_CSR, mask, 0);
}
+void mt7996_memcpy_fromio(struct mt7996_dev *dev, void *buf, u32 offset,
+ size_t len);
+
+void mt7996_mac_init(struct mt7996_dev *dev);
u32 mt7996_mac_wtbl_lmac_addr(struct mt7996_dev *dev, u16 wcid, u8 dw);
bool mt7996_mac_wtbl_update(struct mt7996_dev *dev, int idx, u32 mask);
void mt7996_mac_reset_counters(struct mt7996_phy *phy);
@@ -486,9 +513,12 @@ void mt7996_mac_cca_stats_reset(struct mt7996_phy *phy);
void mt7996_mac_enable_nf(struct mt7996_dev *dev, u8 band);
void mt7996_mac_enable_rtscts(struct mt7996_dev *dev,
struct ieee80211_vif *vif, bool enable);
+void mt7996_mac_set_fixed_rate_table(struct mt7996_dev *dev,
+ u8 tbl_idx, u16 rate_idx);
void mt7996_mac_write_txwi(struct mt7996_dev *dev, __le32 *txwi,
- struct sk_buff *skb, struct mt76_wcid *wcid, int pid,
- struct ieee80211_key_conf *key, u32 changed);
+ struct sk_buff *skb, struct mt76_wcid *wcid,
+ struct ieee80211_key_conf *key, int pid,
+ enum mt76_txq_id qid, u32 changed);
void mt7996_mac_set_timing(struct mt7996_phy *phy);
int mt7996_mac_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
@@ -496,6 +526,7 @@ void mt7996_mac_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
void mt7996_mac_work(struct work_struct *work);
void mt7996_mac_reset_work(struct work_struct *work);
+void mt7996_mac_dump_work(struct work_struct *work);
void mt7996_mac_sta_rc_work(struct work_struct *work);
void mt7996_mac_update_stats(struct mt7996_phy *phy);
void mt7996_mac_twt_teardown_flow(struct mt7996_dev *dev,
@@ -512,7 +543,6 @@ void mt7996_tx_token_put(struct mt7996_dev *dev);
void mt7996_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
struct sk_buff *skb, u32 *info);
bool mt7996_rx_check(struct mt76_dev *mdev, void *data, int len);
-void mt7996_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps);
void mt7996_stats_work(struct work_struct *work);
int mt76_dfs_start_rdd(struct mt7996_dev *dev, bool force);
int mt7996_dfs_init_radar_detector(struct mt7996_phy *phy);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/regs.h b/drivers/net/wireless/mediatek/mt76/mt7996/regs.h
index 7a28cae..d1d3d15 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/regs.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/regs.h
@@ -228,6 +228,13 @@ enum base_rev {
#define MT_WTBL_UPDATE_ADM_COUNT_CLEAR BIT(14)
#define MT_WTBL_UPDATE_BUSY BIT(31)
+#define MT_WTBL_ITCR MT_WTBLON_TOP(0x3b0)
+#define MT_WTBL_ITCR_WR BIT(16)
+#define MT_WTBL_ITCR_EXEC BIT(31)
+#define MT_WTBL_ITDR0 MT_WTBLON_TOP(0x3b8)
+#define MT_WTBL_ITDR1 MT_WTBLON_TOP(0x3bc)
+#define MT_WTBL_SPE_IDX_SEL BIT(6)
+
/* WTBL */
#define MT_WTBL_BASE 0x820d8000
#define MT_WTBL_LMAC_ID GENMASK(14, 8)
@@ -317,6 +324,8 @@ enum base_rev {
#define MT_WFDMA0_RX_INT_PCIE_SEL MT_WFDMA0(0x154)
#define MT_WFDMA0_RX_INT_SEL_RING3 BIT(3)
+#define MT_WFDMA0_MCU_HOST_INT_ENA MT_WFDMA0(0x1f4)
+
#define MT_WFDMA0_GLO_CFG MT_WFDMA0(0x208)
#define MT_WFDMA0_GLO_CFG_TX_DMA_EN BIT(0)
#define MT_WFDMA0_GLO_CFG_RX_DMA_EN BIT(2)
@@ -444,6 +453,10 @@ enum base_rev {
#define MT_MCU_CMD_NORMAL_STATE BIT(5)
#define MT_MCU_CMD_ERROR_MASK GENMASK(5, 1)
+#define MT_MCU_CMD_WA_WDT BIT(31)
+#define MT_MCU_CMD_WM_WDT BIT(30)
+#define MT_MCU_CMD_WDT_MASK GENMASK(31, 30)
+
/* l1/l2 remap */
#define MT_HIF_REMAP_L1 0x155024
#define MT_HIF_REMAP_L1_MASK GENMASK(31, 16)
@@ -468,9 +481,28 @@ enum base_rev {
#define MT_INFRA_MCU_END 0x7c3fffff
/* FW MODE SYNC */
-#define MT_SWDEF_MODE 0x9143c
+#define MT_FW_ASSERT_CNT 0x02208274
+#define MT_FW_DUMP_STATE 0x02209e90
+
+#define MT_SWDEF_BASE 0x00401400
+
+#define MT_SWDEF(ofs) (MT_SWDEF_BASE + (ofs))
+#define MT_SWDEF_MODE MT_SWDEF(0x3c)
#define MT_SWDEF_NORMAL_MODE 0
+#define MT_SWDEF_SER_STATS MT_SWDEF(0x040)
+#define MT_SWDEF_PLE_STATS MT_SWDEF(0x044)
+#define MT_SWDEF_PLE1_STATS MT_SWDEF(0x048)
+#define MT_SWDEF_PLE_AMSDU_STATS MT_SWDEF(0x04c)
+#define MT_SWDEF_PSE_STATS MT_SWDEF(0x050)
+#define MT_SWDEF_PSE1_STATS MT_SWDEF(0x054)
+#define MT_SWDEF_LAMC_WISR6_BN0_STATS MT_SWDEF(0x058)
+#define MT_SWDEF_LAMC_WISR6_BN1_STATS MT_SWDEF(0x05c)
+#define MT_SWDEF_LAMC_WISR6_BN2_STATS MT_SWDEF(0x060)
+#define MT_SWDEF_LAMC_WISR7_BN0_STATS MT_SWDEF(0x064)
+#define MT_SWDEF_LAMC_WISR7_BN1_STATS MT_SWDEF(0x068)
+#define MT_SWDEF_LAMC_WISR7_BN2_STATS MT_SWDEF(0x06c)
+
/* LED */
#define MT_LED_TOP_BASE 0x18013000
#define MT_LED_PHYS(_n) (MT_LED_TOP_BASE + (_n))
@@ -486,6 +518,13 @@ enum base_rev {
#define MT_LED_EN(_n) MT_LED_PHYS(0x40 + ((_n) * 4))
+/* CONN DBG */
+#define MT_CONN_DBG_CTL_BASE 0x18023000
+#define MT_CONN_DBG_CTL(ofs) (MT_CONN_DBG_CTL_BASE + (ofs))
+#define MT_CONN_DBG_CTL_OUT_SEL MT_CONN_DBG_CTL(0x604)
+#define MT_CONN_DBG_CTL_PC_LOG_SEL MT_CONN_DBG_CTL(0x60c)
+#define MT_CONN_DBG_CTL_PC_LOG MT_CONN_DBG_CTL(0x610)
+
#define MT_LED_GPIO_MUX2 0x70005058 /* GPIO 18 */
#define MT_LED_GPIO_MUX3 0x7000505C /* GPIO 26 */
#define MT_LED_GPIO_SEL_MASK GENMASK(11, 8)
@@ -506,7 +545,7 @@ enum base_rev {
#define MT_TOP_MISC_FW_STATE GENMASK(2, 0)
#define MT_HW_REV 0x70010204
-#define MT_WF_SUBSYS_RST 0x70002600
+#define MT_WF_SUBSYS_RST 0x70028600
/* PCIE MAC */
#define MT_PCIE_MAC_BASE 0x74030000
@@ -539,4 +578,12 @@ enum base_rev {
#define MT_WF_PHYRX_CSD_BAND_RXTD12_IRPI_SW_CLR_ONLY BIT(18)
#define MT_WF_PHYRX_CSD_BAND_RXTD12_IRPI_SW_CLR BIT(29)
+/* CONN MCU EXCP CON */
+#define MT_MCU_WM_EXCP_BASE 0x89050000
+#define MT_MCU_WM_EXCP(ofs) (MT_MCU_WM_EXCP_BASE + (ofs))
+#define MT_MCU_WM_EXCP_PC_CTRL MT_MCU_WM_EXCP(0x100)
+#define MT_MCU_WM_EXCP_PC_LOG MT_MCU_WM_EXCP(0x104)
+#define MT_MCU_WM_EXCP_LR_CTRL MT_MCU_WM_EXCP(0x200)
+#define MT_MCU_WM_EXCP_LR_LOG MT_MCU_WM_EXCP(0x204)
+
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c
index 1f309d0..72b3ec7 100644
--- a/drivers/net/wireless/mediatek/mt76/tx.c
+++ b/drivers/net/wireless/mediatek/mt76/tx.c
@@ -77,7 +77,9 @@ mt76_tx_status_unlock(struct mt76_dev *dev, struct sk_buff_head *list)
}
hw = mt76_tx_status_get_hw(dev, skb);
+ spin_lock_bh(&dev->rx_lock);
ieee80211_tx_status_ext(hw, &status);
+ spin_unlock_bh(&dev->rx_lock);
}
rcu_read_unlock();
}
@@ -263,7 +265,9 @@ void __mt76_tx_complete_skb(struct mt76_dev *dev, u16 wcid_idx, struct sk_buff *
if (cb->pktid < MT_PACKET_ID_FIRST) {
hw = mt76_tx_status_get_hw(dev, skb);
status.sta = wcid_to_sta(wcid);
+ spin_lock_bh(&dev->rx_lock);
ieee80211_tx_status_ext(hw, &status);
+ spin_unlock_bh(&dev->rx_lock);
goto out;
}
@@ -330,7 +334,7 @@ mt76_tx(struct mt76_phy *phy, struct ieee80211_sta *sta,
if ((dev->drv->drv_flags & MT_DRV_HW_MGMT_TXQ) &&
!(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) &&
!ieee80211_is_data(hdr->frame_control) &&
- !ieee80211_is_bufferable_mmpdu(hdr->frame_control)) {
+ !ieee80211_is_bufferable_mmpdu(skb)) {
qid = MT_TXQ_PSD;
}
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
index 9d48c69f..8eafbf1 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
@@ -27,7 +27,7 @@
#define RTL8XXXU_MAX_REG_POLL 500
#define USB_INTR_CONTENT_LENGTH 56
-#define RTL8XXXU_OUT_ENDPOINTS 4
+#define RTL8XXXU_OUT_ENDPOINTS 6
#define REALTEK_USB_READ 0xc0
#define REALTEK_USB_WRITE 0x40
@@ -1923,6 +1923,11 @@ struct rtl8xxxu_fileops {
u8 has_tx_report:1;
u8 gen2_thermal_meter:1;
u8 needs_full_init:1;
+ u8 init_reg_rxfltmap:1;
+ u8 init_reg_pkt_life_time:1;
+ u8 init_reg_hmtfr:1;
+ u8 ampdu_max_time;
+ u8 ustime_tsf_edca;
u32 adda_1t_init;
u32 adda_1t_path_on;
u32 adda_2t_path_on_a;
@@ -1948,10 +1953,22 @@ u32 rtl8xxxu_read32(struct rtl8xxxu_priv *priv, u16 addr);
int rtl8xxxu_write8(struct rtl8xxxu_priv *priv, u16 addr, u8 val);
int rtl8xxxu_write16(struct rtl8xxxu_priv *priv, u16 addr, u16 val);
int rtl8xxxu_write32(struct rtl8xxxu_priv *priv, u16 addr, u32 val);
+int rtl8xxxu_write8_set(struct rtl8xxxu_priv *priv, u16 addr, u8 bits);
+int rtl8xxxu_write8_clear(struct rtl8xxxu_priv *priv, u16 addr, u8 bits);
+int rtl8xxxu_write16_set(struct rtl8xxxu_priv *priv, u16 addr, u16 bits);
+int rtl8xxxu_write16_clear(struct rtl8xxxu_priv *priv, u16 addr, u16 bits);
+int rtl8xxxu_write32_set(struct rtl8xxxu_priv *priv, u16 addr, u32 bits);
+int rtl8xxxu_write32_clear(struct rtl8xxxu_priv *priv, u16 addr, u32 bits);
+int rtl8xxxu_write32_mask(struct rtl8xxxu_priv *priv, u16 addr,
+ u32 mask, u32 val);
+
u32 rtl8xxxu_read_rfreg(struct rtl8xxxu_priv *priv,
enum rtl8xxxu_rfpath path, u8 reg);
int rtl8xxxu_write_rfreg(struct rtl8xxxu_priv *priv,
enum rtl8xxxu_rfpath path, u8 reg, u32 data);
+int rtl8xxxu_write_rfreg_mask(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg,
+ u32 mask, u32 val);
void rtl8xxxu_save_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
u32 *backup, int count);
void rtl8xxxu_restore_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188e.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188e.c
index 6a82ec4..8986783 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188e.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188e.c
@@ -568,10 +568,6 @@ static int rtl8188eu_parse_efuse(struct rtl8xxxu_priv *priv)
priv->default_crystal_cap = efuse->xtal_k & 0x3f;
- dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.11s\n", efuse->device_name);
- dev_info(&priv->udev->dev, "Serial: %.11s\n", efuse->serial);
-
return 0;
}
@@ -1883,6 +1879,7 @@ struct rtl8xxxu_fileops rtl8188eu_fops = {
.rx_desc_size = sizeof(struct rtl8xxxu_rxdesc16),
.tx_desc_size = sizeof(struct rtl8xxxu_txdesc32),
.has_tx_report = 1,
+ .init_reg_pkt_life_time = 1,
.gen2_thermal_meter = 1,
.adda_1t_init = 0x0b1b25a0,
.adda_1t_path_on = 0x0bdb25a0,
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188f.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188f.c
index 82dee1f..dbdfd77 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188f.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8188f.c
@@ -734,9 +734,6 @@ static int rtl8188fu_parse_efuse(struct rtl8xxxu_priv *priv)
priv->default_crystal_cap = efuse->xtal_k & 0x3f;
- dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.7s\n", efuse->device_name);
-
return 0;
}
@@ -1746,6 +1743,11 @@ struct rtl8xxxu_fileops rtl8188fu_fops = {
.has_tx_report = 1,
.gen2_thermal_meter = 1,
.needs_full_init = 1,
+ .init_reg_rxfltmap = 1,
+ .init_reg_pkt_life_time = 1,
+ .init_reg_hmtfr = 1,
+ .ampdu_max_time = 0x70,
+ .ustime_tsf_edca = 0x28,
.adda_1t_init = 0x03c00014,
.adda_1t_path_on = 0x03c00014,
.trxff_boundary = 0x3f7f,
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c
index caeba56..b30a9a5 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c
@@ -441,11 +441,6 @@ static int rtl8192cu_parse_efuse(struct rtl8xxxu_priv *priv)
efuse->ht20_max_power_offset,
sizeof(efuse->ht20_max_power_offset));
- dev_info(&priv->udev->dev, "Vendor: %.7s\n",
- efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.20s\n",
- efuse->device_name);
-
priv->power_base = &rtl8192c_power_base;
if (efuse->rf_regulatory & 0x20) {
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c
index 4498748..fcc2926 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c
@@ -601,43 +601,9 @@ rtl8192e_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
}
}
-static void rtl8192eu_log_next_device_info(struct rtl8xxxu_priv *priv,
- char *record_name,
- char *device_info,
- unsigned int *record_offset)
-{
- char *record = device_info + *record_offset;
-
- /* A record is [ total length | 0x03 | value ] */
- unsigned char l = record[0];
-
- /*
- * The whole device info section seems to be 80 characters, make sure
- * we don't read further.
- */
- if (*record_offset + l > 80) {
- dev_warn(&priv->udev->dev,
- "invalid record length %d while parsing \"%s\" at offset %u.\n",
- l, record_name, *record_offset);
- return;
- }
-
- if (l >= 2) {
- char value[80];
-
- memcpy(value, &record[2], l - 2);
- value[l - 2] = '\0';
- dev_info(&priv->udev->dev, "%s: %s\n", record_name, value);
- *record_offset = *record_offset + l;
- } else {
- dev_info(&priv->udev->dev, "%s not available.\n", record_name);
- }
-}
-
static int rtl8192eu_parse_efuse(struct rtl8xxxu_priv *priv)
{
struct rtl8192eu_efuse *efuse = &priv->efuse_wifi.efuse8192eu;
- unsigned int record_offset;
int i;
if (efuse->rtl_id != cpu_to_le16(0x8129))
@@ -684,26 +650,6 @@ static int rtl8192eu_parse_efuse(struct rtl8xxxu_priv *priv)
priv->default_crystal_cap = priv->efuse_wifi.efuse8192eu.xtal_k & 0x3f;
- /*
- * device_info section seems to be laid out as records
- * [ total length | 0x03 | value ] so:
- * - vendor length + 2
- * - 0x03
- * - vendor string (not null terminated)
- * - product length + 2
- * - 0x03
- * - product string (not null terminated)
- * Then there is one or 2 0x00 on all the 4 devices I own or found
- * dumped online.
- * As previous version of the code handled an optional serial
- * string, I now assume there may be a third record if the
- * length is not 0.
- */
- record_offset = 0;
- rtl8192eu_log_next_device_info(priv, "Vendor", efuse->device_info, &record_offset);
- rtl8192eu_log_next_device_info(priv, "Product", efuse->device_info, &record_offset);
- rtl8192eu_log_next_device_info(priv, "Serial", efuse->device_info, &record_offset);
-
return 0;
}
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8710b.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8710b.c
index 920466e..22d4704 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8710b.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8710b.c
@@ -1865,6 +1865,15 @@ struct rtl8xxxu_fileops rtl8710bu_fops = {
.has_tx_report = 1,
.gen2_thermal_meter = 1,
.needs_full_init = 1,
+ .init_reg_rxfltmap = 1,
+ .init_reg_pkt_life_time = 1,
+ .init_reg_hmtfr = 1,
+ .ampdu_max_time = 0x5e,
+ /*
+ * The RTL8710BU vendor driver uses 0x50 here and it works fine,
+ * but in rtl8xxxu 0x50 causes slow upload and random packet loss. Why?
+ */
+ .ustime_tsf_edca = 0x28,
.adda_1t_init = 0x03c00016,
.adda_1t_path_on = 0x03c00016,
.trxff_boundary = 0x3f7f,
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c
index d219be1..15a30e4 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c
@@ -222,10 +222,6 @@ static int rtl8723au_parse_efuse(struct rtl8xxxu_priv *priv)
priv->power_base = &rtl8723a_power_base;
- dev_info(&priv->udev->dev, "Vendor: %.7s\n",
- efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.41s\n",
- efuse->device_name);
return 0;
}
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c
index d99538e..abc56c7 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c
@@ -494,9 +494,6 @@ static int rtl8723bu_parse_efuse(struct rtl8xxxu_priv *priv)
priv->default_crystal_cap = priv->efuse_wifi.efuse8723bu.xtal_k & 0x3f;
- dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.41s\n", efuse->device_name);
-
return 0;
}
@@ -1741,6 +1738,9 @@ struct rtl8xxxu_fileops rtl8723bu_fops = {
.has_tx_report = 1,
.gen2_thermal_meter = 1,
.needs_full_init = 1,
+ .init_reg_hmtfr = 1,
+ .ampdu_max_time = 0x5e,
+ .ustime_tsf_edca = 0x50,
.adda_1t_init = 0x01c00014,
.adda_1t_path_on = 0x01c00014,
.adda_2t_path_on_a = 0x01c00014,
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
index c152b22..fd8c8c6 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
@@ -786,6 +786,85 @@ int rtl8xxxu_write32(struct rtl8xxxu_priv *priv, u16 addr, u32 val)
return ret;
}
+int rtl8xxxu_write8_set(struct rtl8xxxu_priv *priv, u16 addr, u8 bits)
+{
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, addr);
+ val8 |= bits;
+ return rtl8xxxu_write8(priv, addr, val8);
+}
+
+int rtl8xxxu_write8_clear(struct rtl8xxxu_priv *priv, u16 addr, u8 bits)
+{
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, addr);
+ val8 &= ~bits;
+ return rtl8xxxu_write8(priv, addr, val8);
+}
+
+int rtl8xxxu_write16_set(struct rtl8xxxu_priv *priv, u16 addr, u16 bits)
+{
+ u16 val16;
+
+ val16 = rtl8xxxu_read16(priv, addr);
+ val16 |= bits;
+ return rtl8xxxu_write16(priv, addr, val16);
+}
+
+int rtl8xxxu_write16_clear(struct rtl8xxxu_priv *priv, u16 addr, u16 bits)
+{
+ u16 val16;
+
+ val16 = rtl8xxxu_read16(priv, addr);
+ val16 &= ~bits;
+ return rtl8xxxu_write16(priv, addr, val16);
+}
+
+int rtl8xxxu_write32_set(struct rtl8xxxu_priv *priv, u16 addr, u32 bits)
+{
+ u32 val32;
+
+ val32 = rtl8xxxu_read32(priv, addr);
+ val32 |= bits;
+ return rtl8xxxu_write32(priv, addr, val32);
+}
+
+int rtl8xxxu_write32_clear(struct rtl8xxxu_priv *priv, u16 addr, u32 bits)
+{
+ u32 val32;
+
+ val32 = rtl8xxxu_read32(priv, addr);
+ val32 &= ~bits;
+ return rtl8xxxu_write32(priv, addr, val32);
+}
+
+int rtl8xxxu_write32_mask(struct rtl8xxxu_priv *priv, u16 addr,
+ u32 mask, u32 val)
+{
+ u32 orig, new, shift;
+
+ shift = __ffs(mask);
+
+ orig = rtl8xxxu_read32(priv, addr);
+ new = (orig & ~mask) | ((val << shift) & mask);
+ return rtl8xxxu_write32(priv, addr, new);
+}
+
+int rtl8xxxu_write_rfreg_mask(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg,
+ u32 mask, u32 val)
+{
+ u32 orig, new, shift;
+
+ shift = __ffs(mask);
+
+ orig = rtl8xxxu_read_rfreg(priv, path, reg);
+ new = (orig & ~mask) | ((val << shift) & mask);
+ return rtl8xxxu_write_rfreg(priv, path, reg, new);
+}
+
static int
rtl8xxxu_writeN(struct rtl8xxxu_priv *priv, u16 addr, u8 *buf, u16 len)
{
@@ -1663,6 +1742,8 @@ int rtl8xxxu_config_endpoints_no_sie(struct rtl8xxxu_priv *priv)
struct device *dev = &priv->udev->dev;
switch (priv->nr_out_eps) {
+ case 6:
+ case 5:
case 4:
case 3:
priv->ep_tx_low_queue = 1;
@@ -1916,7 +1997,7 @@ static int rtl8xxxu_start_firmware(struct rtl8xxxu_priv *priv)
/*
* Init H2C command
*/
- if (priv->rtl_chip == RTL8723B || priv->rtl_chip == RTL8188F || priv->rtl_chip == RTL8710B)
+ if (priv->fops->init_reg_hmtfr)
rtl8xxxu_write8(priv, REG_HMTFR, 0x0f);
exit:
return ret;
@@ -3864,11 +3945,8 @@ void rtl8xxxu_init_burst(struct rtl8xxxu_priv *priv)
rtl8xxxu_write8(priv, REG_HT_SINGLE_AMPDU_8723B, val8);
rtl8xxxu_write16(priv, REG_MAX_AGGR_NUM, 0x0c14);
- if (priv->rtl_chip == RTL8723B || priv->rtl_chip == RTL8710B)
- val8 = 0x5e;
- else if (priv->rtl_chip == RTL8188F)
- val8 = 0x70; /* 0x5e would make it very slow */
- rtl8xxxu_write8(priv, REG_AMPDU_MAX_TIME_8723B, val8);
+ rtl8xxxu_write8(priv, REG_AMPDU_MAX_TIME_8723B,
+ priv->fops->ampdu_max_time);
rtl8xxxu_write32(priv, REG_AGGLEN_LMT, 0xffffffff);
rtl8xxxu_write8(priv, REG_RX_PKT_LIMIT, 0x18);
rtl8xxxu_write8(priv, REG_PIFS, 0x00);
@@ -3876,16 +3954,8 @@ void rtl8xxxu_init_burst(struct rtl8xxxu_priv *priv)
rtl8xxxu_write8(priv, REG_FWHW_TXQ_CTRL, FWHW_TXQ_CTRL_AMPDU_RETRY);
rtl8xxxu_write32(priv, REG_FAST_EDCA_CTRL, 0x03086666);
}
- /*
- * The RTL8710BU vendor driver uses 0x50 here and it works fine,
- * but in rtl8xxxu 0x50 causes slow upload and random packet loss. Why?
- */
- if (priv->rtl_chip == RTL8723B)
- val8 = 0x50;
- else if (priv->rtl_chip == RTL8188F || priv->rtl_chip == RTL8710B)
- val8 = 0x28; /* 0x50 would make the upload slow */
- rtl8xxxu_write8(priv, REG_USTIME_TSF_8723B, val8);
- rtl8xxxu_write8(priv, REG_USTIME_EDCA, val8);
+ rtl8xxxu_write8(priv, REG_USTIME_TSF_8723B, priv->fops->ustime_tsf_edca);
+ rtl8xxxu_write8(priv, REG_USTIME_EDCA, priv->fops->ustime_tsf_edca);
/* to prevent mac is reseted by bus. */
val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
@@ -4102,7 +4172,7 @@ static int rtl8xxxu_init_device(struct ieee80211_hw *hw)
RCR_APPEND_PHYSTAT | RCR_APPEND_ICV | RCR_APPEND_MIC;
rtl8xxxu_write32(priv, REG_RCR, val32);
- if (priv->rtl_chip == RTL8188F || priv->rtl_chip == RTL8710B) {
+ if (fops->init_reg_rxfltmap) {
/* Accept all data frames */
rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0xffff);
@@ -4187,8 +4257,7 @@ static int rtl8xxxu_init_device(struct ieee80211_hw *hw)
if (fops->init_aggregation)
fops->init_aggregation(priv);
- if (priv->rtl_chip == RTL8188F || priv->rtl_chip == RTL8188E ||
- priv->rtl_chip == RTL8710B) {
+ if (fops->init_reg_pkt_life_time) {
rtl8xxxu_write16(priv, REG_PKT_VO_VI_LIFE_TIME, 0x0400); /* unit: 256us. 256ms */
rtl8xxxu_write16(priv, REG_PKT_BE_BK_LIFE_TIME, 0x0400); /* unit: 256us. 256ms */
}
@@ -6965,10 +7034,8 @@ static int rtl8xxxu_start(struct ieee80211_hw *hw)
rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0xffff);
rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0xffff);
- if (priv->rtl_chip == RTL8188E)
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, 0x6955341e);
- else
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, 0x6954341e);
+ rtl8xxxu_write32_mask(priv, REG_OFDM0_XA_AGC_CORE1,
+ OFDM0_X_AGC_CORE1_IGI_MASK, 0x1e);
return ret;
diff --git a/drivers/net/wireless/realtek/rtw88/Kconfig b/drivers/net/wireless/realtek/rtw88/Kconfig
index 651ab56..29eb2f8 100644
--- a/drivers/net/wireless/realtek/rtw88/Kconfig
+++ b/drivers/net/wireless/realtek/rtw88/Kconfig
@@ -16,6 +16,9 @@
config RTW88_PCI
tristate
+config RTW88_SDIO
+ tristate
+
config RTW88_USB
tristate
@@ -42,6 +45,17 @@
802.11ac PCIe wireless network adapter
+config RTW88_8822BS
+ tristate "Realtek 8822BS SDIO wireless network adapter"
+ depends on MMC
+ select RTW88_CORE
+ select RTW88_SDIO
+ select RTW88_8822B
+ help
+ Select this option will enable support for 8822BS chipset
+
+ 802.11ac SDIO wireless network adapter
+
config RTW88_8822BU
tristate "Realtek 8822BU USB wireless network adapter"
depends on USB
@@ -64,6 +78,17 @@
802.11ac PCIe wireless network adapter
+config RTW88_8822CS
+ tristate "Realtek 8822CS SDIO wireless network adapter"
+ depends on MMC
+ select RTW88_CORE
+ select RTW88_SDIO
+ select RTW88_8822C
+ help
+ Select this option will enable support for 8822CS chipset
+
+ 802.11ac SDIO wireless network adapter
+
config RTW88_8822CU
tristate "Realtek 8822CU USB wireless network adapter"
depends on USB
@@ -108,6 +133,17 @@
802.11ac PCIe wireless network adapter
+config RTW88_8821CS
+ tristate "Realtek 8821CS SDIO wireless network adapter"
+ depends on MMC
+ select RTW88_CORE
+ select RTW88_SDIO
+ select RTW88_8821C
+ help
+ Select this option will enable support for 8821CS chipset
+
+ 802.11ac SDIO wireless network adapter
+
config RTW88_8821CU
tristate "Realtek 8821CU USB wireless network adapter"
depends on USB
diff --git a/drivers/net/wireless/realtek/rtw88/Makefile b/drivers/net/wireless/realtek/rtw88/Makefile
index fe7293e..82979b3 100644
--- a/drivers/net/wireless/realtek/rtw88/Makefile
+++ b/drivers/net/wireless/realtek/rtw88/Makefile
@@ -26,6 +26,9 @@
obj-$(CONFIG_RTW88_8822BE) += rtw88_8822be.o
rtw88_8822be-objs := rtw8822be.o
+obj-$(CONFIG_RTW88_8822BS) += rtw88_8822bs.o
+rtw88_8822bs-objs := rtw8822bs.o
+
obj-$(CONFIG_RTW88_8822BU) += rtw88_8822bu.o
rtw88_8822bu-objs := rtw8822bu.o
@@ -35,6 +38,9 @@
obj-$(CONFIG_RTW88_8822CE) += rtw88_8822ce.o
rtw88_8822ce-objs := rtw8822ce.o
+obj-$(CONFIG_RTW88_8822CS) += rtw88_8822cs.o
+rtw88_8822cs-objs := rtw8822cs.o
+
obj-$(CONFIG_RTW88_8822CU) += rtw88_8822cu.o
rtw88_8822cu-objs := rtw8822cu.o
@@ -53,11 +59,17 @@
obj-$(CONFIG_RTW88_8821CE) += rtw88_8821ce.o
rtw88_8821ce-objs := rtw8821ce.o
+obj-$(CONFIG_RTW88_8821CS) += rtw88_8821cs.o
+rtw88_8821cs-objs := rtw8821cs.o
+
obj-$(CONFIG_RTW88_8821CU) += rtw88_8821cu.o
rtw88_8821cu-objs := rtw8821cu.o
obj-$(CONFIG_RTW88_PCI) += rtw88_pci.o
rtw88_pci-objs := pci.o
+obj-$(CONFIG_RTW88_SDIO) += rtw88_sdio.o
+rtw88_sdio-objs := sdio.o
+
obj-$(CONFIG_RTW88_USB) += rtw88_usb.o
rtw88_usb-objs := usb.o
diff --git a/drivers/net/wireless/realtek/rtw88/debug.h b/drivers/net/wireless/realtek/rtw88/debug.h
index 066792d..a9149c6 100644
--- a/drivers/net/wireless/realtek/rtw88/debug.h
+++ b/drivers/net/wireless/realtek/rtw88/debug.h
@@ -24,6 +24,7 @@ enum rtw_debug_mask {
RTW_DBG_ADAPTIVITY = 0x00008000,
RTW_DBG_HW_SCAN = 0x00010000,
RTW_DBG_STATE = 0x00020000,
+ RTW_DBG_SDIO = 0x00040000,
RTW_DBG_ALL = 0xffffffff
};
diff --git a/drivers/net/wireless/realtek/rtw88/fw.c b/drivers/net/wireless/realtek/rtw88/fw.c
index 82295ac..2a8ccc8 100644
--- a/drivers/net/wireless/realtek/rtw88/fw.c
+++ b/drivers/net/wireless/realtek/rtw88/fw.c
@@ -1393,6 +1393,10 @@ static void rtw_build_rsvd_page_iter(void *data, u8 *mac,
struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
struct rtw_rsvd_page *rsvd_pkt;
+ /* AP not yet started, don't gather its rsvd pages */
+ if (vif->type == NL80211_IFTYPE_AP && !rtwdev->ap_active)
+ return;
+
list_for_each_entry(rsvd_pkt, &rtwvif->rsvd_page_list, vif_list) {
if (rsvd_pkt->type == RSVD_BEACON)
list_add(&rsvd_pkt->build_list,
@@ -1614,6 +1618,7 @@ void rtw_fw_update_beacon_work(struct work_struct *work)
mutex_lock(&rtwdev->mutex);
rtw_fw_download_rsvd_page(rtwdev);
+ rtw_send_rsvd_page_h2c(rtwdev);
mutex_unlock(&rtwdev->mutex);
}
@@ -2155,11 +2160,19 @@ int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
}
rtw_fw_set_scan_offload(rtwdev, &cs_option, rtwvif, &chan_list);
out:
+ if (rtwdev->ap_active) {
+ ret = rtw_download_beacon(rtwdev);
+ if (ret)
+ rtw_err(rtwdev, "HW scan download beacon failed\n");
+ }
+
return ret;
}
-void rtw_hw_scan_abort(struct rtw_dev *rtwdev, struct ieee80211_vif *vif)
+void rtw_hw_scan_abort(struct rtw_dev *rtwdev)
{
+ struct ieee80211_vif *vif = rtwdev->scan_info.scanning_vif;
+
if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_SCAN_OFFLOAD))
return;
@@ -2244,6 +2257,7 @@ void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb)
if (rtw_is_op_chan(rtwdev, chan)) {
rtw_store_op_chan(rtwdev, false);
ieee80211_wake_queues(rtwdev->hw);
+ rtw_core_enable_beacon(rtwdev, true);
}
} else if (id == RTW_SCAN_NOTIFY_ID_PRESWITCH) {
if (IS_CH_5G_BAND(chan)) {
@@ -2262,8 +2276,10 @@ void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb)
* if next channel is non-op channel.
*/
if (!rtw_is_op_chan(rtwdev, chan) &&
- rtw_is_op_chan(rtwdev, hal->current_channel))
+ rtw_is_op_chan(rtwdev, hal->current_channel)) {
+ rtw_core_enable_beacon(rtwdev, false);
ieee80211_stop_queues(rtwdev->hw);
+ }
}
rtw_dbg(rtwdev, RTW_DBG_HW_SCAN,
diff --git a/drivers/net/wireless/realtek/rtw88/fw.h b/drivers/net/wireless/realtek/rtw88/fw.h
index 0a386e6..397cbc3 100644
--- a/drivers/net/wireless/realtek/rtw88/fw.h
+++ b/drivers/net/wireless/realtek/rtw88/fw.h
@@ -868,5 +868,5 @@ int rtw_hw_scan_offload(struct rtw_dev *rtwdev, struct ieee80211_vif *vif,
bool enable);
void rtw_hw_scan_status_report(struct rtw_dev *rtwdev, struct sk_buff *skb);
void rtw_hw_scan_chan_switch(struct rtw_dev *rtwdev, struct sk_buff *skb);
-void rtw_hw_scan_abort(struct rtw_dev *rtwdev, struct ieee80211_vif *vif);
+void rtw_hw_scan_abort(struct rtw_dev *rtwdev);
#endif
diff --git a/drivers/net/wireless/realtek/rtw88/mac.c b/drivers/net/wireless/realtek/rtw88/mac.c
index f3a566c..a168f36 100644
--- a/drivers/net/wireless/realtek/rtw88/mac.c
+++ b/drivers/net/wireless/realtek/rtw88/mac.c
@@ -7,6 +7,7 @@
#include "reg.h"
#include "fw.h"
#include "debug.h"
+#include "sdio.h"
void rtw_set_channel_mac(struct rtw_dev *rtwdev, u8 channel, u8 bw,
u8 primary_ch_idx)
@@ -60,6 +61,7 @@ EXPORT_SYMBOL(rtw_set_channel_mac);
static int rtw_mac_pre_system_cfg(struct rtw_dev *rtwdev)
{
+ unsigned int retry;
u32 value32;
u8 value8;
@@ -77,6 +79,28 @@ static int rtw_mac_pre_system_cfg(struct rtw_dev *rtwdev)
case RTW_HCI_TYPE_PCIE:
rtw_write32_set(rtwdev, REG_HCI_OPT_CTRL, BIT_USB_SUS_DIS);
break;
+ case RTW_HCI_TYPE_SDIO:
+ rtw_write8_clr(rtwdev, REG_SDIO_HSUS_CTRL, BIT_HCI_SUS_REQ);
+
+ for (retry = 0; retry < RTW_PWR_POLLING_CNT; retry++) {
+ if (rtw_read8(rtwdev, REG_SDIO_HSUS_CTRL) & BIT_HCI_RESUME_RDY)
+ break;
+
+ usleep_range(10, 50);
+ }
+
+ if (retry == RTW_PWR_POLLING_CNT) {
+ rtw_err(rtwdev, "failed to poll REG_SDIO_HSUS_CTRL[1]");
+ return -ETIMEDOUT;
+ }
+
+ if (rtw_sdio_is_sdio30_supported(rtwdev))
+ rtw_write8_set(rtwdev, REG_HCI_OPT_CTRL + 2,
+ BIT_SDIO_PAD_E5 >> 16);
+ else
+ rtw_write8_clr(rtwdev, REG_HCI_OPT_CTRL + 2,
+ BIT_SDIO_PAD_E5 >> 16);
+ break;
case RTW_HCI_TYPE_USB:
break;
default:
@@ -248,6 +272,7 @@ static int rtw_mac_power_switch(struct rtw_dev *rtwdev, bool pwr_on)
{
const struct rtw_chip_info *chip = rtwdev->chip;
const struct rtw_pwr_seq_cmd **pwr_seq;
+ u32 imr = 0;
u8 rpwm;
bool cur_pwr;
int ret;
@@ -273,17 +298,24 @@ static int rtw_mac_power_switch(struct rtw_dev *rtwdev, bool pwr_on)
if (pwr_on == cur_pwr)
return -EALREADY;
- pwr_seq = pwr_on ? chip->pwr_on_seq : chip->pwr_off_seq;
- ret = rtw_pwr_seq_parser(rtwdev, pwr_seq);
- if (ret)
- return ret;
+ if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_SDIO) {
+ imr = rtw_read32(rtwdev, REG_SDIO_HIMR);
+ rtw_write32(rtwdev, REG_SDIO_HIMR, 0);
+ }
- if (pwr_on)
- set_bit(RTW_FLAG_POWERON, rtwdev->flags);
- else
+ if (!pwr_on)
clear_bit(RTW_FLAG_POWERON, rtwdev->flags);
- return 0;
+ pwr_seq = pwr_on ? chip->pwr_on_seq : chip->pwr_off_seq;
+ ret = rtw_pwr_seq_parser(rtwdev, pwr_seq);
+
+ if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_SDIO)
+ rtw_write32(rtwdev, REG_SDIO_HIMR, imr);
+
+ if (!ret && pwr_on)
+ set_bit(RTW_FLAG_POWERON, rtwdev->flags);
+
+ return ret;
}
static int __rtw_mac_init_system_cfg(struct rtw_dev *rtwdev)
@@ -454,6 +486,9 @@ static void download_firmware_reg_backup(struct rtw_dev *rtwdev,
rtw_write16(rtwdev, REG_FIFOPAGE_INFO_1, 0x200);
rtw_write32(rtwdev, REG_RQPN_CTRL_2, bckp[bckp_idx - 1].val);
+ if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_SDIO)
+ rtw_read32(rtwdev, REG_SDIO_FREE_TXPG);
+
/* Disable beacon related functions */
tmp = rtw_read8(rtwdev, REG_BCN_CTRL);
bckp[bckp_idx].len = 1;
@@ -1066,8 +1101,12 @@ static int txdma_queue_mapping(struct rtw_dev *rtwdev)
if (rtw_chip_wcpu_11ac(rtwdev))
rtw_write32(rtwdev, REG_H2CQ_CSR, BIT_H2CQ_FULL);
- if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_USB)
+ if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_SDIO) {
+ rtw_read32(rtwdev, REG_SDIO_FREE_TXPG);
+ rtw_write32(rtwdev, REG_SDIO_TX_CTRL, 0);
+ } else if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_USB) {
rtw_write8_set(rtwdev, REG_TXDMA_PQ_MAP, BIT_RXDMA_ARBBW_EN);
+ }
return 0;
}
@@ -1080,7 +1119,7 @@ static int set_trx_fifo_info(struct rtw_dev *rtwdev)
u8 csi_buf_pg_num = chip->csi_buf_pg_num;
/* config rsvd page num */
- fifo->rsvd_drv_pg_num = 8;
+ fifo->rsvd_drv_pg_num = chip->rsvd_drv_pg_num;
fifo->txff_pg_num = chip->txff_size >> 7;
if (rtw_chip_wcpu_11n(rtwdev))
fifo->rsvd_pg_num = fifo->rsvd_drv_pg_num;
diff --git a/drivers/net/wireless/realtek/rtw88/mac.h b/drivers/net/wireless/realtek/rtw88/mac.h
index 3172aa5..58c3dcc 100644
--- a/drivers/net/wireless/realtek/rtw88/mac.h
+++ b/drivers/net/wireless/realtek/rtw88/mac.h
@@ -7,7 +7,6 @@
#define RTW_HW_PORT_NUM 5
#define cut_version_to_mask(cut) (0x1 << ((cut) + 1))
-#define SDIO_LOCAL_OFFSET 0x10250000
#define DDMA_POLLING_COUNT 1000
#define C2H_PKT_BUF 256
#define REPORT_BUF 128
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 3b92ac6..7aa6eda 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -155,25 +155,30 @@ static int rtw_ops_add_interface(struct ieee80211_hw *hw,
struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
enum rtw_net_type net_type;
u32 config = 0;
- u8 port = 0;
+ u8 port;
u8 bcn_ctrl = 0;
if (rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_BCN_FILTER))
vif->driver_flags |= IEEE80211_VIF_BEACON_FILTER |
IEEE80211_VIF_SUPPORTS_CQM_RSSI;
- rtwvif->port = port;
rtwvif->stats.tx_unicast = 0;
rtwvif->stats.rx_unicast = 0;
rtwvif->stats.tx_cnt = 0;
rtwvif->stats.rx_cnt = 0;
rtwvif->scan_req = NULL;
memset(&rtwvif->bfee, 0, sizeof(struct rtw_bfee));
- rtwvif->conf = &rtw_vif_port[port];
rtw_txq_init(rtwdev, vif->txq);
INIT_LIST_HEAD(&rtwvif->rsvd_page_list);
mutex_lock(&rtwdev->mutex);
+ port = find_first_zero_bit(rtwdev->hw_port, RTW_PORT_NUM);
+ if (port >= RTW_PORT_NUM)
+ return -EINVAL;
+ set_bit(port, rtwdev->hw_port);
+
+ rtwvif->port = port;
+ rtwvif->conf = &rtw_vif_port[port];
rtw_leave_lps_deep(rtwdev);
switch (vif->type) {
@@ -195,6 +200,7 @@ static int rtw_ops_add_interface(struct ieee80211_hw *hw,
break;
default:
WARN_ON(1);
+ clear_bit(rtwvif->port, rtwdev->hw_port);
mutex_unlock(&rtwdev->mutex);
return -EINVAL;
}
@@ -206,6 +212,7 @@ static int rtw_ops_add_interface(struct ieee80211_hw *hw,
rtwvif->bcn_ctrl = bcn_ctrl;
config |= PORT_SET_BCN_CTRL;
rtw_vif_port_config(rtwdev, rtwvif, config);
+ rtw_core_port_switch(rtwdev, vif);
mutex_unlock(&rtwdev->mutex);
@@ -236,6 +243,7 @@ static void rtw_ops_remove_interface(struct ieee80211_hw *hw,
rtwvif->bcn_ctrl = 0;
config |= PORT_SET_BCN_CTRL;
rtw_vif_port_config(rtwdev, rtwvif, config);
+ clear_bit(rtwvif->port, rtwdev->hw_port);
mutex_unlock(&rtwdev->mutex);
}
@@ -385,7 +393,8 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw,
* when disconnected by peer
*/
if (test_bit(RTW_FLAG_SCANNING, rtwdev->flags))
- rtw_hw_scan_abort(rtwdev, vif);
+ rtw_hw_scan_abort(rtwdev);
+
}
config |= PORT_SET_NET_TYPE;
@@ -395,7 +404,7 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw,
if (changed & BSS_CHANGED_BSSID) {
ether_addr_copy(rtwvif->bssid, conf->bssid);
config |= PORT_SET_BSSID;
- if (is_zero_ether_addr(rtwvif->bssid))
+ if (!rtw_core_check_sta_active(rtwdev))
rtw_clear_op_chan(rtwdev);
else
rtw_store_op_chan(rtwdev, true);
@@ -409,6 +418,7 @@ static void rtw_ops_bss_info_changed(struct ieee80211_hw *hw,
if (changed & BSS_CHANGED_BEACON) {
rtw_set_dtim_period(rtwdev, conf->dtim_period);
rtw_fw_download_rsvd_page(rtwdev);
+ rtw_send_rsvd_page_h2c(rtwdev);
}
if (changed & BSS_CHANGED_BEACON_ENABLED) {
@@ -441,12 +451,27 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw,
const struct rtw_chip_info *chip = rtwdev->chip;
mutex_lock(&rtwdev->mutex);
+ rtwdev->ap_active = true;
+ rtw_store_op_chan(rtwdev, true);
chip->ops->phy_calibration(rtwdev);
mutex_unlock(&rtwdev->mutex);
return 0;
}
+static void rtw_ops_stop_ap(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct ieee80211_bss_conf *link_conf)
+{
+ struct rtw_dev *rtwdev = hw->priv;
+
+ mutex_lock(&rtwdev->mutex);
+ rtwdev->ap_active = false;
+ if (!rtw_core_check_sta_active(rtwdev))
+ rtw_clear_op_chan(rtwdev);
+ mutex_unlock(&rtwdev->mutex);
+}
+
static int rtw_ops_conf_tx(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
unsigned int link_id, u16 ac,
@@ -849,7 +874,7 @@ static int rtw_ops_hw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
rtw_hw_scan_start(rtwdev, vif, req);
ret = rtw_hw_scan_offload(rtwdev, vif, true);
if (ret) {
- rtw_hw_scan_abort(rtwdev, vif);
+ rtw_hw_scan_abort(rtwdev);
rtw_err(rtwdev, "HW scan failed with status: %d\n", ret);
}
mutex_unlock(&rtwdev->mutex);
@@ -869,7 +894,7 @@ static void rtw_ops_cancel_hw_scan(struct ieee80211_hw *hw,
return;
mutex_lock(&rtwdev->mutex);
- rtw_hw_scan_abort(rtwdev, vif);
+ rtw_hw_scan_abort(rtwdev);
mutex_unlock(&rtwdev->mutex);
}
@@ -908,6 +933,7 @@ const struct ieee80211_ops rtw_ops = {
.configure_filter = rtw_ops_configure_filter,
.bss_info_changed = rtw_ops_bss_info_changed,
.start_ap = rtw_ops_start_ap,
+ .stop_ap = rtw_ops_stop_ap,
.conf_tx = rtw_ops_conf_tx,
.sta_add = rtw_ops_sta_add,
.sta_remove = rtw_ops_sta_remove,
diff --git a/drivers/net/wireless/realtek/rtw88/main.c b/drivers/net/wireless/realtek/rtw88/main.c
index b2e7873..5bf6b45 100644
--- a/drivers/net/wireless/realtek/rtw88/main.c
+++ b/drivers/net/wireless/realtek/rtw88/main.c
@@ -18,6 +18,7 @@
#include "debug.h"
#include "bf.h"
#include "sar.h"
+#include "sdio.h"
bool rtw_disable_lps_deep_mode;
EXPORT_SYMBOL(rtw_disable_lps_deep_mode);
@@ -102,6 +103,26 @@ static struct ieee80211_rate rtw_ratetable[] = {
{.bitrate = 540, .hw_value = 0x0b,},
};
+static const struct ieee80211_iface_limit rtw_iface_limits[] = {
+ {
+ .max = 1,
+ .types = BIT(NL80211_IFTYPE_STATION),
+ },
+ {
+ .max = 1,
+ .types = BIT(NL80211_IFTYPE_AP),
+ }
+};
+
+static const struct ieee80211_iface_combination rtw_iface_combs[] = {
+ {
+ .limits = rtw_iface_limits,
+ .n_limits = ARRAY_SIZE(rtw_iface_limits),
+ .max_interfaces = 2,
+ .num_different_channels = 1,
+ }
+};
+
u16 rtw_desc_to_bitrate(u8 desc_rate)
{
struct ieee80211_rate rate;
@@ -256,7 +277,7 @@ static void rtw_watch_dog_work(struct work_struct *work)
* threshold.
*/
if (rtwdev->ps_enabled && data.rtwvif && !ps_active &&
- !rtwdev->beacon_loss)
+ !rtwdev->beacon_loss && !rtwdev->ap_active)
rtw_enter_lps(rtwdev, data.rtwvif->port);
rtwdev->watch_dog_cnt++;
@@ -609,6 +630,7 @@ static void __fw_recovery_work(struct rtw_dev *rtwdev)
rcu_read_unlock();
rtw_iterate_stas_atomic(rtwdev, rtw_reset_sta_iter, rtwdev);
rtw_iterate_vifs_atomic(rtwdev, rtw_reset_vif_iter, rtwdev);
+ bitmap_zero(rtwdev->hw_port, RTW_PORT_NUM);
rtw_enter_ips(rtwdev);
}
@@ -828,6 +850,9 @@ void rtw_set_channel(struct rtw_dev *rtwdev)
rtw_update_channel(rtwdev, center_chan, primary_chan, band, bandwidth);
+ if (rtwdev->scan_info.op_chan)
+ rtw_store_op_chan(rtwdev, true);
+
chip->ops->set_channel(rtwdev, center_chan, bandwidth,
hal->current_primary_channel_index);
@@ -1785,6 +1810,10 @@ static int rtw_chip_parameter_setup(struct rtw_dev *rtwdev)
rtwdev->hci.rpwm_addr = 0x03d9;
rtwdev->hci.cpwm_addr = 0x03da;
break;
+ case RTW_HCI_TYPE_SDIO:
+ rtwdev->hci.rpwm_addr = REG_SDIO_HRPWM1;
+ rtwdev->hci.cpwm_addr = REG_SDIO_HCPWM1_V2;
+ break;
case RTW_HCI_TYPE_USB:
rtwdev->hci.rpwm_addr = 0xfe58;
rtwdev->hci.cpwm_addr = 0xfe57;
@@ -1979,7 +2008,7 @@ static int rtw_chip_board_info_setup(struct rtw_dev *rtwdev)
if (!rfe_def)
return -ENODEV;
- rtw_phy_setup_phy_cond(rtwdev, 0);
+ rtw_phy_setup_phy_cond(rtwdev, hal->pkg_type);
rtw_phy_init_tx_power(rtwdev);
if (rfe_def->agc_btg_tbl)
@@ -2158,9 +2187,11 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
int max_tx_headroom = 0;
int ret;
- /* TODO: USB & SDIO may need extra room? */
max_tx_headroom = rtwdev->chip->tx_pkt_desc_sz;
+ if (rtw_hci_type(rtwdev) == RTW_HCI_TYPE_SDIO)
+ max_tx_headroom += RTW_SDIO_DATA_PTR_ALIGN;
+
hw->extra_tx_headroom = max_tx_headroom;
hw->queues = IEEE80211_NUM_ACS;
hw->txq_data_size = sizeof(struct rtw_txq);
@@ -2194,6 +2225,11 @@ int rtw_register_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
hw->wiphy->max_scan_ssids = RTW_SCAN_MAX_SSIDS;
hw->wiphy->max_scan_ie_len = rtw_get_max_scan_ie_len(rtwdev);
+ if (rtwdev->chip->id == RTW_CHIP_TYPE_8822C) {
+ hw->wiphy->iface_combinations = rtw_iface_combs;
+ hw->wiphy->n_iface_combinations = ARRAY_SIZE(rtw_iface_combs);
+ }
+
wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_SCAN_RANDOM_SN);
wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_SET_SCAN_DWELL);
@@ -2243,6 +2279,121 @@ void rtw_unregister_hw(struct rtw_dev *rtwdev, struct ieee80211_hw *hw)
}
EXPORT_SYMBOL(rtw_unregister_hw);
+static
+void rtw_swap_reg_nbytes(struct rtw_dev *rtwdev, const struct rtw_hw_reg *reg1,
+ const struct rtw_hw_reg *reg2, u8 nbytes)
+{
+ u8 i;
+
+ for (i = 0; i < nbytes; i++) {
+ u8 v1 = rtw_read8(rtwdev, reg1->addr + i);
+ u8 v2 = rtw_read8(rtwdev, reg2->addr + i);
+
+ rtw_write8(rtwdev, reg1->addr + i, v2);
+ rtw_write8(rtwdev, reg2->addr + i, v1);
+ }
+}
+
+static
+void rtw_swap_reg_mask(struct rtw_dev *rtwdev, const struct rtw_hw_reg *reg1,
+ const struct rtw_hw_reg *reg2)
+{
+ u32 v1, v2;
+
+ v1 = rtw_read32_mask(rtwdev, reg1->addr, reg1->mask);
+ v2 = rtw_read32_mask(rtwdev, reg2->addr, reg2->mask);
+ rtw_write32_mask(rtwdev, reg2->addr, reg2->mask, v1);
+ rtw_write32_mask(rtwdev, reg1->addr, reg1->mask, v2);
+}
+
+struct rtw_iter_port_switch_data {
+ struct rtw_dev *rtwdev;
+ struct rtw_vif *rtwvif_ap;
+};
+
+static void rtw_port_switch_iter(void *data, u8 *mac, struct ieee80211_vif *vif)
+{
+ struct rtw_iter_port_switch_data *iter_data = data;
+ struct rtw_dev *rtwdev = iter_data->rtwdev;
+ struct rtw_vif *rtwvif_target = (struct rtw_vif *)vif->drv_priv;
+ struct rtw_vif *rtwvif_ap = iter_data->rtwvif_ap;
+ const struct rtw_hw_reg *reg1, *reg2;
+
+ if (rtwvif_target->port != RTW_PORT_0)
+ return;
+
+ rtw_dbg(rtwdev, RTW_DBG_STATE, "AP port switch from %d -> %d\n",
+ rtwvif_ap->port, rtwvif_target->port);
+
+ reg1 = &rtwvif_ap->conf->net_type;
+ reg2 = &rtwvif_target->conf->net_type;
+ rtw_swap_reg_mask(rtwdev, reg1, reg2);
+
+ reg1 = &rtwvif_ap->conf->mac_addr;
+ reg2 = &rtwvif_target->conf->mac_addr;
+ rtw_swap_reg_nbytes(rtwdev, reg1, reg2, ETH_ALEN);
+
+ reg1 = &rtwvif_ap->conf->bssid;
+ reg2 = &rtwvif_target->conf->bssid;
+ rtw_swap_reg_nbytes(rtwdev, reg1, reg2, ETH_ALEN);
+
+ reg1 = &rtwvif_ap->conf->bcn_ctrl;
+ reg2 = &rtwvif_target->conf->bcn_ctrl;
+ rtw_swap_reg_nbytes(rtwdev, reg1, reg2, 1);
+
+ swap(rtwvif_target->port, rtwvif_ap->port);
+ swap(rtwvif_target->conf, rtwvif_ap->conf);
+}
+
+void rtw_core_port_switch(struct rtw_dev *rtwdev, struct ieee80211_vif *vif)
+{
+ struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
+ struct rtw_iter_port_switch_data iter_data;
+
+ if (vif->type != NL80211_IFTYPE_AP || rtwvif->port == RTW_PORT_0)
+ return;
+
+ iter_data.rtwdev = rtwdev;
+ iter_data.rtwvif_ap = rtwvif;
+ rtw_iterate_vifs(rtwdev, rtw_port_switch_iter, &iter_data);
+}
+
+static void rtw_check_sta_active_iter(void *data, u8 *mac,
+ struct ieee80211_vif *vif)
+{
+ struct rtw_vif *rtwvif = (struct rtw_vif *)vif->drv_priv;
+ bool *active = data;
+
+ if (*active)
+ return;
+
+ if (vif->type != NL80211_IFTYPE_STATION)
+ return;
+
+ if (vif->cfg.assoc || !is_zero_ether_addr(rtwvif->bssid))
+ *active = true;
+}
+
+bool rtw_core_check_sta_active(struct rtw_dev *rtwdev)
+{
+ bool sta_active = false;
+
+ rtw_iterate_vifs(rtwdev, rtw_check_sta_active_iter, &sta_active);
+
+ return rtwdev->ap_active || sta_active;
+}
+
+void rtw_core_enable_beacon(struct rtw_dev *rtwdev, bool enable)
+{
+ if (!rtwdev->ap_active)
+ return;
+
+ if (enable)
+ rtw_write32_set(rtwdev, REG_BCN_CTRL, BIT_EN_BCN_FUNCTION);
+ else
+ rtw_write32_clr(rtwdev, REG_BCN_CTRL, BIT_EN_BCN_FUNCTION);
+}
+
MODULE_AUTHOR("Realtek Corporation");
MODULE_DESCRIPTION("Realtek 802.11ac wireless core module");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw88/main.h b/drivers/net/wireless/realtek/rtw88/main.h
index d4a53d5..a563285 100644
--- a/drivers/net/wireless/realtek/rtw88/main.h
+++ b/drivers/net/wireless/realtek/rtw88/main.h
@@ -88,7 +88,7 @@ enum rtw_supported_band {
RTW_BAND_60G = BIT(NL80211_BAND_60GHZ),
};
-/* now, support upto 80M bw */
+/* now, support up to 80M bw */
#define RTW_MAX_CHANNEL_WIDTH RTW_CHANNEL_WIDTH_80
enum rtw_bandwidth {
@@ -395,6 +395,15 @@ enum rtw_snr {
RTW_SNR_NUM
};
+enum rtw_port {
+ RTW_PORT_0 = 0,
+ RTW_PORT_1 = 1,
+ RTW_PORT_2 = 2,
+ RTW_PORT_3 = 3,
+ RTW_PORT_4 = 4,
+ RTW_PORT_NUM
+};
+
enum rtw_wow_flags {
RTW_WOW_FLAG_EN_MAGIC_PKT,
RTW_WOW_FLAG_EN_REKEY_PKT,
@@ -1168,6 +1177,7 @@ struct rtw_chip_info {
u32 txff_size;
u32 rxff_size;
u32 fw_rxff_size;
+ u16 rsvd_drv_pg_num;
u8 band;
u8 page_size;
u8 csi_buf_pg_num;
@@ -1871,7 +1881,7 @@ enum rtw_sar_bands {
RTW_SAR_BAND_NR,
};
-/* the union is reserved for other knids of SAR sources
+/* the union is reserved for other kinds of SAR sources
* which might not re-use same format with array common.
*/
union rtw_sar_cfg {
@@ -1890,7 +1900,9 @@ struct rtw_hal {
u8 cut_version;
u8 mp_chip;
u8 oem_id;
+ u8 pkg_type;
struct rtw_phy_cond phy_cond;
+ bool rfe_btg;
u8 ps_mode;
u8 current_channel;
@@ -2020,7 +2032,7 @@ struct rtw_dev {
struct rtw_tx_report tx_report;
struct {
- /* incicate the mail box to use with fw */
+ /* indicate the mail box to use with fw */
u8 last_box_num;
u32 seq;
} h2c;
@@ -2036,6 +2048,7 @@ struct rtw_dev {
u8 sta_cnt;
u32 rts_threshold;
+ DECLARE_BITMAP(hw_port, RTW_PORT_NUM);
DECLARE_BITMAP(mac_id_map, RTW_MAX_MAC_ID_NUM);
DECLARE_BITMAP(flags, NUM_OF_RTW_FLAGS);
@@ -2047,6 +2060,7 @@ struct rtw_dev {
bool need_rfk;
struct completion fw_scan_density;
+ bool ap_active;
/* hci related data, must be last */
u8 priv[] __aligned(sizeof(void *));
@@ -2188,4 +2202,7 @@ void rtw_set_txrx_1ss(struct rtw_dev *rtwdev, bool config_1ss);
void rtw_update_channel(struct rtw_dev *rtwdev, u8 center_channel,
u8 primary_channel, enum rtw_supported_band band,
enum rtw_bandwidth bandwidth);
+void rtw_core_port_switch(struct rtw_dev *rtwdev, struct ieee80211_vif *vif);
+bool rtw_core_check_sta_active(struct rtw_dev *rtwdev);
+void rtw_core_enable_beacon(struct rtw_dev *rtwdev, bool enable);
#endif
diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h
index 8852b24..2a2ae20 100644
--- a/drivers/net/wireless/realtek/rtw88/reg.h
+++ b/drivers/net/wireless/realtek/rtw88/reg.h
@@ -87,6 +87,7 @@
#define BIT_LTE_MUX_CTRL_PATH BIT(26)
#define REG_HCI_OPT_CTRL 0x0074
#define BIT_USB_SUS_DIS BIT(8)
+#define BIT_SDIO_PAD_E5 BIT(18)
#define REG_AFE_CTRL_4 0x0078
#define BIT_CK320M_AFE_EN BIT(4)
@@ -185,6 +186,9 @@
(((x) & BIT_MASK_TXDMA_VIQ_MAP) << BIT_SHIFT_TXDMA_VIQ_MAP)
#define REG_TXDMA_PQ_MAP 0x010C
#define BIT_RXDMA_ARBBW_EN BIT(0)
+#define BIT_RXSHFT_EN BIT(1)
+#define BIT_RXDMA_AGG_EN BIT(2)
+#define BIT_TXDMA_BW_EN BIT(3)
#define BIT_SHIFT_TXDMA_BEQ_MAP 8
#define BIT_MASK_TXDMA_BEQ_MAP 0x3
#define BIT_TXDMA_BEQ_MAP(x) \
@@ -283,10 +287,18 @@
#define REG_H2C_TAIL 0x0248
#define REG_H2C_READ_ADDR 0x024C
#define REG_H2C_INFO 0x0254
+#define REG_RXDMA_AGG_PG_TH 0x0280
+#define BIT_RXDMA_AGG_PG_TH GENMASK(7, 0)
+#define BIT_DMA_AGG_TO_V1 GENMASK(15, 8)
+#define BIT_EN_PRE_CALC BIT(29)
#define REG_RXPKT_NUM 0x0284
#define BIT_RXDMA_REQ BIT(19)
#define BIT_RW_RELEASE BIT(18)
#define BIT_RXDMA_IDLE BIT(17)
+#define REG_RXDMA_STATUS 0x0288
+#define REG_RXDMA_DPR 0x028C
+#define REG_RXDMA_MODE 0x0290
+#define BIT_DMA_MODE BIT(1)
#define REG_RXPKTNUM 0x02B0
#define REG_INT_MIG 0x0304
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8723d.c b/drivers/net/wireless/realtek/rtw88/rtw8723d.c
index 2d2f768..06e7454 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8723d.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8723d.c
@@ -2743,6 +2743,7 @@ const struct rtw_chip_info rtw8723d_hw_spec = {
.ptct_efuse_size = 96 + 1,
.txff_size = 32768,
.rxff_size = 16384,
+ .rsvd_drv_pg_num = 8,
.txgi_factor = 1,
.is_pwr_by_rate_dec = true,
.max_power_index = 0x3f,
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8821c.c b/drivers/net/wireless/realtek/rtw88/rtw8821c.c
index 7ae0541..adf2246 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8821c.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8821c.c
@@ -47,13 +47,14 @@ enum rtw8821ce_rf_set {
static int rtw8821c_read_efuse(struct rtw_dev *rtwdev, u8 *log_map)
{
+ struct rtw_hal *hal = &rtwdev->hal;
struct rtw_efuse *efuse = &rtwdev->efuse;
struct rtw8821c_efuse *map;
int i;
map = (struct rtw8821c_efuse *)log_map;
- efuse->rfe_option = map->rfe_option;
+ efuse->rfe_option = map->rfe_option & 0x1f;
efuse->rf_board_option = map->rf_board_option;
efuse->crystal_cap = map->xtal_k;
efuse->pa_type_2g = map->pa_type;
@@ -70,6 +71,19 @@ static int rtw8821c_read_efuse(struct rtw_dev *rtwdev, u8 *log_map)
efuse->tx_bb_swing_setting_2g = map->tx_bb_swing_setting_2g;
efuse->tx_bb_swing_setting_5g = map->tx_bb_swing_setting_5g;
+ hal->pkg_type = map->rfe_option & BIT(5) ? 1 : 0;
+
+ switch (efuse->rfe_option) {
+ case 0x2:
+ case 0x4:
+ case 0x7:
+ case 0xa:
+ case 0xc:
+ case 0xf:
+ hal->rfe_btg = true;
+ break;
+ }
+
for (i = 0; i < 4; i++)
efuse->txpwr_idx_table[i] = map->txpwr_idx_table[i];
@@ -295,6 +309,7 @@ static void rtw8821c_switch_rf_set(struct rtw_dev *rtwdev, u8 rf_set)
static void rtw8821c_set_channel_rf(struct rtw_dev *rtwdev, u8 channel, u8 bw)
{
+ struct rtw_hal *hal = &rtwdev->hal;
u32 rf_reg18;
rf_reg18 = rtw_read_rf(rtwdev, RF_PATH_A, 0x18, RFREG_MASK);
@@ -326,11 +341,10 @@ static void rtw8821c_set_channel_rf(struct rtw_dev *rtwdev, u8 channel, u8 bw)
}
if (channel <= 14) {
- if (rtwdev->efuse.rfe_option == 0)
- rtw8821c_switch_rf_set(rtwdev, SWITCH_TO_WLG);
- else if (rtwdev->efuse.rfe_option == 2 ||
- rtwdev->efuse.rfe_option == 4)
+ if (hal->rfe_btg)
rtw8821c_switch_rf_set(rtwdev, SWITCH_TO_BTG);
+ else
+ rtw8821c_switch_rf_set(rtwdev, SWITCH_TO_WLG);
rtw_write_rf(rtwdev, RF_PATH_A, RF_LUTDBG, BIT(6), 0x1);
rtw_write_rf(rtwdev, RF_PATH_A, 0x64, 0xf, 0xf);
} else {
@@ -1546,7 +1560,6 @@ static const struct rtw_rfe_def rtw8821c_rfe_defs[] = {
[2] = RTW_DEF_RFE_EXT(8821c, 0, 0, 2),
[4] = RTW_DEF_RFE_EXT(8821c, 0, 0, 2),
[6] = RTW_DEF_RFE(8821c, 0, 0),
- [34] = RTW_DEF_RFE(8821c, 0, 0),
};
static struct rtw_hw_reg rtw8821c_dig[] = {
@@ -1920,6 +1933,7 @@ const struct rtw_chip_info rtw8821c_hw_spec = {
.ptct_efuse_size = 96,
.txff_size = 65536,
.rxff_size = 16384,
+ .rsvd_drv_pg_num = 8,
.txgi_factor = 1,
.is_pwr_by_rate_dec = true,
.max_power_index = 0x3f,
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8821cs.c b/drivers/net/wireless/realtek/rtw88/rtw8821cs.c
new file mode 100644
index 0000000..a359413
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/rtw8821cs.c
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+ */
+
+#include <linux/mmc/sdio_func.h>
+#include <linux/mmc/sdio_ids.h>
+#include <linux/module.h>
+#include "main.h"
+#include "rtw8821c.h"
+#include "sdio.h"
+
+static const struct sdio_device_id rtw_8821cs_id_table[] = {
+ {
+ SDIO_DEVICE(SDIO_VENDOR_ID_REALTEK,
+ SDIO_DEVICE_ID_REALTEK_RTW8821CS),
+ .driver_data = (kernel_ulong_t)&rtw8821c_hw_spec,
+ },
+ {}
+};
+MODULE_DEVICE_TABLE(sdio, rtw_8821cs_id_table);
+
+static struct sdio_driver rtw_8821cs_driver = {
+ .name = "rtw_8821cs",
+ .probe = rtw_sdio_probe,
+ .remove = rtw_sdio_remove,
+ .id_table = rtw_8821cs_id_table,
+ .drv = {
+ .pm = &rtw_sdio_pm_ops,
+ .shutdown = rtw_sdio_shutdown,
+ }
+};
+module_sdio_driver(rtw_8821cs_driver);
+
+MODULE_AUTHOR("Martin Blumenstingl <martin.blumenstingl@googlemail.com>");
+MODULE_DESCRIPTION("Realtek 802.11ac wireless 8821cs driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822b.c b/drivers/net/wireless/realtek/rtw88/rtw8822b.c
index 531b677..3017a97 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8822b.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822b.c
@@ -2540,6 +2540,7 @@ const struct rtw_chip_info rtw8822b_hw_spec = {
.txff_size = 262144,
.rxff_size = 24576,
.fw_rxff_size = 12288,
+ .rsvd_drv_pg_num = 8,
.txgi_factor = 1,
.is_pwr_by_rate_dec = true,
.max_power_index = 0x3f,
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822bs.c b/drivers/net/wireless/realtek/rtw88/rtw8822bs.c
new file mode 100644
index 0000000..31d8645
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822bs.c
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) Jernej Skrabec <jernej.skrabec@gmail.com>
+ */
+
+#include <linux/mmc/sdio_func.h>
+#include <linux/mmc/sdio_ids.h>
+#include <linux/module.h>
+#include "main.h"
+#include "rtw8822b.h"
+#include "sdio.h"
+
+static const struct sdio_device_id rtw_8822bs_id_table[] = {
+ {
+ SDIO_DEVICE(SDIO_VENDOR_ID_REALTEK,
+ SDIO_DEVICE_ID_REALTEK_RTW8822BS),
+ .driver_data = (kernel_ulong_t)&rtw8822b_hw_spec,
+ },
+ {}
+};
+MODULE_DEVICE_TABLE(sdio, rtw_8822bs_id_table);
+
+static struct sdio_driver rtw_8822bs_driver = {
+ .name = "rtw_8822bs",
+ .probe = rtw_sdio_probe,
+ .remove = rtw_sdio_remove,
+ .id_table = rtw_8822bs_id_table,
+ .drv = {
+ .pm = &rtw_sdio_pm_ops,
+ .shutdown = rtw_sdio_shutdown,
+ }
+};
+module_sdio_driver(rtw_8822bs_driver);
+
+MODULE_AUTHOR("Jernej Skrabec <jernej.skrabec@gmail.com>");
+MODULE_DESCRIPTION("Realtek 802.11ac wireless 8822bs driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822c.c b/drivers/net/wireless/realtek/rtw88/rtw8822c.c
index 5a2c004b..cd965ed 100644
--- a/drivers/net/wireless/realtek/rtw88/rtw8822c.c
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822c.c
@@ -5358,6 +5358,7 @@ const struct rtw_chip_info rtw8822c_hw_spec = {
.txff_size = 262144,
.rxff_size = 24576,
.fw_rxff_size = 12288,
+ .rsvd_drv_pg_num = 16,
.txgi_factor = 2,
.is_pwr_by_rate_dec = false,
.max_power_index = 0x7f,
diff --git a/drivers/net/wireless/realtek/rtw88/rtw8822cs.c b/drivers/net/wireless/realtek/rtw88/rtw8822cs.c
new file mode 100644
index 0000000..975e81c
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/rtw8822cs.c
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+ */
+
+#include <linux/mmc/sdio_func.h>
+#include <linux/mmc/sdio_ids.h>
+#include <linux/module.h>
+#include "main.h"
+#include "rtw8822c.h"
+#include "sdio.h"
+
+static const struct sdio_device_id rtw_8822cs_id_table[] = {
+ {
+ SDIO_DEVICE(SDIO_VENDOR_ID_REALTEK,
+ SDIO_DEVICE_ID_REALTEK_RTW8822CS),
+ .driver_data = (kernel_ulong_t)&rtw8822c_hw_spec,
+ },
+ {}
+};
+MODULE_DEVICE_TABLE(sdio, rtw_8822cs_id_table);
+
+static struct sdio_driver rtw_8822cs_driver = {
+ .name = "rtw_8822cs",
+ .probe = rtw_sdio_probe,
+ .remove = rtw_sdio_remove,
+ .id_table = rtw_8822cs_id_table,
+ .drv = {
+ .pm = &rtw_sdio_pm_ops,
+ .shutdown = rtw_sdio_shutdown,
+ }
+};
+module_sdio_driver(rtw_8822cs_driver);
+
+MODULE_AUTHOR("Martin Blumenstingl <martin.blumenstingl@googlemail.com>");
+MODULE_DESCRIPTION("Realtek 802.11ac wireless 8822cs driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw88/sdio.c b/drivers/net/wireless/realtek/rtw88/sdio.c
new file mode 100644
index 0000000..af0459a
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/sdio.c
@@ -0,0 +1,1394 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright (C) 2021 Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+ * Copyright (C) 2021 Jernej Skrabec <jernej.skrabec@gmail.com>
+ *
+ * Based on rtw88/pci.c:
+ * Copyright(c) 2018-2019 Realtek Corporation
+ */
+
+#include <linux/module.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/sdio_func.h>
+#include "main.h"
+#include "debug.h"
+#include "fw.h"
+#include "ps.h"
+#include "reg.h"
+#include "rx.h"
+#include "sdio.h"
+#include "tx.h"
+
+#define RTW_SDIO_INDIRECT_RW_RETRIES 50
+
+static bool rtw_sdio_is_bus_addr(u32 addr)
+{
+ return !!(addr & RTW_SDIO_BUS_MSK);
+}
+
+static bool rtw_sdio_bus_claim_needed(struct rtw_sdio *rtwsdio)
+{
+ return !rtwsdio->irq_thread ||
+ rtwsdio->irq_thread != current;
+}
+
+static u32 rtw_sdio_to_bus_offset(struct rtw_dev *rtwdev, u32 addr)
+{
+ switch (addr & RTW_SDIO_BUS_MSK) {
+ case WLAN_IOREG_OFFSET:
+ addr &= WLAN_IOREG_REG_MSK;
+ addr |= FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_MAC_REG);
+ break;
+ case SDIO_LOCAL_OFFSET:
+ addr &= SDIO_LOCAL_REG_MSK;
+ addr |= FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_SDIO_REG);
+ break;
+ default:
+ rtw_warn(rtwdev, "Cannot convert addr 0x%08x to bus offset",
+ addr);
+ }
+
+ return addr;
+}
+
+static bool rtw_sdio_use_memcpy_io(struct rtw_dev *rtwdev, u32 addr,
+ u8 alignment)
+{
+ return IS_ALIGNED(addr, alignment) &&
+ test_bit(RTW_FLAG_POWERON, rtwdev->flags);
+}
+
+static void rtw_sdio_writel(struct rtw_dev *rtwdev, u32 val, u32 addr,
+ int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u8 buf[4];
+ int i;
+
+ if (rtw_sdio_use_memcpy_io(rtwdev, addr, 4)) {
+ sdio_writel(rtwsdio->sdio_func, val, addr, err_ret);
+ return;
+ }
+
+ *(__le32 *)buf = cpu_to_le32(val);
+
+ for (i = 0; i < 4; i++) {
+ sdio_writeb(rtwsdio->sdio_func, buf[i], addr + i, err_ret);
+ if (*err_ret)
+ return;
+ }
+}
+
+static void rtw_sdio_writew(struct rtw_dev *rtwdev, u16 val, u32 addr,
+ int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u8 buf[2];
+ int i;
+
+ if (rtw_sdio_use_memcpy_io(rtwdev, addr, 2)) {
+ sdio_writew(rtwsdio->sdio_func, val, addr, err_ret);
+ return;
+ }
+
+ *(__le16 *)buf = cpu_to_le16(val);
+
+ for (i = 0; i < 2; i++) {
+ sdio_writeb(rtwsdio->sdio_func, buf[i], addr + i, err_ret);
+ if (*err_ret)
+ return;
+ }
+}
+
+static u32 rtw_sdio_readl(struct rtw_dev *rtwdev, u32 addr, int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u8 buf[4];
+ int i;
+
+ if (rtw_sdio_use_memcpy_io(rtwdev, addr, 4))
+ return sdio_readl(rtwsdio->sdio_func, addr, err_ret);
+
+ for (i = 0; i < 4; i++) {
+ buf[i] = sdio_readb(rtwsdio->sdio_func, addr + i, err_ret);
+ if (*err_ret)
+ return 0;
+ }
+
+ return le32_to_cpu(*(__le32 *)buf);
+}
+
+static u16 rtw_sdio_readw(struct rtw_dev *rtwdev, u32 addr, int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u8 buf[2];
+ int i;
+
+ if (rtw_sdio_use_memcpy_io(rtwdev, addr, 2))
+ return sdio_readw(rtwsdio->sdio_func, addr, err_ret);
+
+ for (i = 0; i < 2; i++) {
+ buf[i] = sdio_readb(rtwsdio->sdio_func, addr + i, err_ret);
+ if (*err_ret)
+ return 0;
+ }
+
+ return le16_to_cpu(*(__le16 *)buf);
+}
+
+static u32 rtw_sdio_to_io_address(struct rtw_dev *rtwdev, u32 addr,
+ bool direct)
+{
+ if (!direct)
+ return addr;
+
+ if (!rtw_sdio_is_bus_addr(addr))
+ addr |= WLAN_IOREG_OFFSET;
+
+ return rtw_sdio_to_bus_offset(rtwdev, addr);
+}
+
+static bool rtw_sdio_use_direct_io(struct rtw_dev *rtwdev, u32 addr)
+{
+ return !rtw_sdio_is_sdio30_supported(rtwdev) ||
+ rtw_sdio_is_bus_addr(addr);
+}
+
+static int rtw_sdio_indirect_reg_cfg(struct rtw_dev *rtwdev, u32 addr, u32 cfg)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ unsigned int retry;
+ u32 reg_cfg;
+ int ret;
+ u8 tmp;
+
+ reg_cfg = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_CFG);
+
+ rtw_sdio_writel(rtwdev, addr | cfg | BIT_SDIO_INDIRECT_REG_CFG_UNK20,
+ reg_cfg, &ret);
+ if (ret)
+ return ret;
+
+ for (retry = 0; retry < RTW_SDIO_INDIRECT_RW_RETRIES; retry++) {
+ tmp = sdio_readb(rtwsdio->sdio_func, reg_cfg + 2, &ret);
+ if (!ret && (tmp & BIT(4)))
+ return 0;
+ }
+
+ return -ETIMEDOUT;
+}
+
+static u8 rtw_sdio_indirect_read8(struct rtw_dev *rtwdev, u32 addr,
+ int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u32 reg_data;
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_READ);
+ if (*err_ret)
+ return 0;
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ return sdio_readb(rtwsdio->sdio_func, reg_data, err_ret);
+}
+
+static int rtw_sdio_indirect_read_bytes(struct rtw_dev *rtwdev, u32 addr,
+ u8 *buf, int count)
+{
+ int i, ret = 0;
+
+ for (i = 0; i < count; i++) {
+ buf[i] = rtw_sdio_indirect_read8(rtwdev, addr + i, &ret);
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
+
+static u16 rtw_sdio_indirect_read16(struct rtw_dev *rtwdev, u32 addr,
+ int *err_ret)
+{
+ u32 reg_data;
+ u8 buf[2];
+
+ if (!IS_ALIGNED(addr, 2)) {
+ *err_ret = rtw_sdio_indirect_read_bytes(rtwdev, addr, buf, 2);
+ if (*err_ret)
+ return 0;
+
+ return le16_to_cpu(*(__le16 *)buf);
+ }
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_READ);
+ if (*err_ret)
+ return 0;
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ return rtw_sdio_readw(rtwdev, reg_data, err_ret);
+}
+
+static u32 rtw_sdio_indirect_read32(struct rtw_dev *rtwdev, u32 addr,
+ int *err_ret)
+{
+ u32 reg_data;
+ u8 buf[4];
+
+ if (!IS_ALIGNED(addr, 4)) {
+ *err_ret = rtw_sdio_indirect_read_bytes(rtwdev, addr, buf, 4);
+ if (*err_ret)
+ return 0;
+
+ return le32_to_cpu(*(__le32 *)buf);
+ }
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_READ);
+ if (*err_ret)
+ return 0;
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ return rtw_sdio_readl(rtwdev, reg_data, err_ret);
+}
+
+static u8 rtw_sdio_read8(struct rtw_dev *rtwdev, u32 addr)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ int ret;
+ u8 val;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ val = sdio_readb(rtwsdio->sdio_func, addr, &ret);
+ else
+ val = rtw_sdio_indirect_read8(rtwdev, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio read8 failed (0x%x): %d", addr, ret);
+
+ return val;
+}
+
+static u16 rtw_sdio_read16(struct rtw_dev *rtwdev, u32 addr)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ int ret;
+ u16 val;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ val = rtw_sdio_readw(rtwdev, addr, &ret);
+ else
+ val = rtw_sdio_indirect_read16(rtwdev, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio read16 failed (0x%x): %d", addr, ret);
+
+ return val;
+}
+
+static u32 rtw_sdio_read32(struct rtw_dev *rtwdev, u32 addr)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ u32 val;
+ int ret;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ val = rtw_sdio_readl(rtwdev, addr, &ret);
+ else
+ val = rtw_sdio_indirect_read32(rtwdev, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio read32 failed (0x%x): %d", addr, ret);
+
+ return val;
+}
+
+static void rtw_sdio_indirect_write8(struct rtw_dev *rtwdev, u8 val, u32 addr,
+ int *err_ret)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ u32 reg_data;
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ sdio_writeb(rtwsdio->sdio_func, val, reg_data, err_ret);
+ if (*err_ret)
+ return;
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_WRITE);
+}
+
+static void rtw_sdio_indirect_write16(struct rtw_dev *rtwdev, u16 val, u32 addr,
+ int *err_ret)
+{
+ u32 reg_data;
+
+ if (!IS_ALIGNED(addr, 2)) {
+ addr = rtw_sdio_to_io_address(rtwdev, addr, true);
+ rtw_sdio_writew(rtwdev, val, addr, err_ret);
+ return;
+ }
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ rtw_sdio_writew(rtwdev, val, reg_data, err_ret);
+ if (*err_ret)
+ return;
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_WRITE |
+ BIT_SDIO_INDIRECT_REG_CFG_WORD);
+}
+
+static void rtw_sdio_indirect_write32(struct rtw_dev *rtwdev, u32 val,
+ u32 addr, int *err_ret)
+{
+ u32 reg_data;
+
+ if (!IS_ALIGNED(addr, 4)) {
+ addr = rtw_sdio_to_io_address(rtwdev, addr, true);
+ rtw_sdio_writel(rtwdev, val, addr, err_ret);
+ return;
+ }
+
+ reg_data = rtw_sdio_to_bus_offset(rtwdev, REG_SDIO_INDIRECT_REG_DATA);
+ rtw_sdio_writel(rtwdev, val, reg_data, err_ret);
+
+ *err_ret = rtw_sdio_indirect_reg_cfg(rtwdev, addr,
+ BIT_SDIO_INDIRECT_REG_CFG_WRITE |
+ BIT_SDIO_INDIRECT_REG_CFG_DWORD);
+}
+
+static void rtw_sdio_write8(struct rtw_dev *rtwdev, u32 addr, u8 val)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ int ret;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ sdio_writeb(rtwsdio->sdio_func, val, addr, &ret);
+ else
+ rtw_sdio_indirect_write8(rtwdev, val, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio write8 failed (0x%x): %d", addr, ret);
+}
+
+static void rtw_sdio_write16(struct rtw_dev *rtwdev, u32 addr, u16 val)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ int ret;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ rtw_sdio_writew(rtwdev, val, addr, &ret);
+ else
+ rtw_sdio_indirect_write16(rtwdev, val, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio write16 failed (0x%x): %d", addr, ret);
+}
+
+static void rtw_sdio_write32(struct rtw_dev *rtwdev, u32 addr, u32 val)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool direct, bus_claim;
+ int ret;
+
+ direct = rtw_sdio_use_direct_io(rtwdev, addr);
+ addr = rtw_sdio_to_io_address(rtwdev, addr, direct);
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ if (direct)
+ rtw_sdio_writel(rtwdev, val, addr, &ret);
+ else
+ rtw_sdio_indirect_write32(rtwdev, val, addr, &ret);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev, "sdio write32 failed (0x%x): %d", addr, ret);
+}
+
+static u32 rtw_sdio_get_tx_addr(struct rtw_dev *rtwdev, size_t size,
+ enum rtw_tx_queue_type queue)
+{
+ u32 txaddr;
+
+ switch (queue) {
+ case RTW_TX_QUEUE_BCN:
+ case RTW_TX_QUEUE_H2C:
+ case RTW_TX_QUEUE_HI0:
+ txaddr = FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_TXFF_HIGH);
+ break;
+ case RTW_TX_QUEUE_VI:
+ case RTW_TX_QUEUE_VO:
+ txaddr = FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_TXFF_NORMAL);
+ break;
+ case RTW_TX_QUEUE_BE:
+ case RTW_TX_QUEUE_BK:
+ txaddr = FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_TXFF_LOW);
+ break;
+ case RTW_TX_QUEUE_MGMT:
+ txaddr = FIELD_PREP(REG_SDIO_CMD_ADDR_MSK,
+ REG_SDIO_CMD_ADDR_TXFF_EXTRA);
+ break;
+ default:
+ rtw_warn(rtwdev, "Unsupported queue for TX addr: 0x%02x\n",
+ queue);
+ return 0;
+ }
+
+ txaddr += DIV_ROUND_UP(size, 4);
+
+ return txaddr;
+};
+
+static int rtw_sdio_read_port(struct rtw_dev *rtwdev, u8 *buf, size_t count)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+ u32 rxaddr = rtwsdio->rx_addr++;
+ int ret;
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ ret = sdio_memcpy_fromio(rtwsdio->sdio_func, buf,
+ RTW_SDIO_ADDR_RX_RX0FF_GEN(rxaddr), count);
+ if (ret)
+ rtw_warn(rtwdev,
+ "Failed to read %zu byte(s) from SDIO port 0x%08x",
+ count, rxaddr);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ return ret;
+}
+
+static int rtw_sdio_check_free_txpg(struct rtw_dev *rtwdev, u8 queue,
+ size_t count)
+{
+ unsigned int pages_free, pages_needed;
+
+ if (rtw_chip_wcpu_11n(rtwdev)) {
+ u32 free_txpg;
+
+ free_txpg = rtw_sdio_read32(rtwdev, REG_SDIO_FREE_TXPG);
+
+ switch (queue) {
+ case RTW_TX_QUEUE_BCN:
+ case RTW_TX_QUEUE_H2C:
+ case RTW_TX_QUEUE_HI0:
+ case RTW_TX_QUEUE_MGMT:
+ /* high */
+ pages_free = free_txpg & 0xff;
+ break;
+ case RTW_TX_QUEUE_VI:
+ case RTW_TX_QUEUE_VO:
+ /* normal */
+ pages_free = (free_txpg >> 8) & 0xff;
+ break;
+ case RTW_TX_QUEUE_BE:
+ case RTW_TX_QUEUE_BK:
+ /* low */
+ pages_free = (free_txpg >> 16) & 0xff;
+ break;
+ default:
+ rtw_warn(rtwdev, "Unknown mapping for queue %u\n", queue);
+ return -EINVAL;
+ }
+
+ /* add the pages from the public queue */
+ pages_free += (free_txpg >> 24) & 0xff;
+ } else {
+ u32 free_txpg[3];
+
+ free_txpg[0] = rtw_sdio_read32(rtwdev, REG_SDIO_FREE_TXPG);
+ free_txpg[1] = rtw_sdio_read32(rtwdev, REG_SDIO_FREE_TXPG + 4);
+ free_txpg[2] = rtw_sdio_read32(rtwdev, REG_SDIO_FREE_TXPG + 8);
+
+ switch (queue) {
+ case RTW_TX_QUEUE_BCN:
+ case RTW_TX_QUEUE_H2C:
+ case RTW_TX_QUEUE_HI0:
+ /* high */
+ pages_free = free_txpg[0] & 0xfff;
+ break;
+ case RTW_TX_QUEUE_VI:
+ case RTW_TX_QUEUE_VO:
+ /* normal */
+ pages_free = (free_txpg[0] >> 16) & 0xfff;
+ break;
+ case RTW_TX_QUEUE_BE:
+ case RTW_TX_QUEUE_BK:
+ /* low */
+ pages_free = free_txpg[1] & 0xfff;
+ break;
+ case RTW_TX_QUEUE_MGMT:
+ /* extra */
+ pages_free = free_txpg[2] & 0xfff;
+ break;
+ default:
+ rtw_warn(rtwdev, "Unknown mapping for queue %u\n", queue);
+ return -EINVAL;
+ }
+
+ /* add the pages from the public queue */
+ pages_free += (free_txpg[1] >> 16) & 0xfff;
+ }
+
+ pages_needed = DIV_ROUND_UP(count, rtwdev->chip->page_size);
+
+ if (pages_needed > pages_free) {
+ rtw_dbg(rtwdev, RTW_DBG_SDIO,
+ "Not enough free pages (%u needed, %u free) in queue %u for %zu bytes\n",
+ pages_needed, pages_free, queue, count);
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
+static int rtw_sdio_write_port(struct rtw_dev *rtwdev, struct sk_buff *skb,
+ enum rtw_tx_queue_type queue)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool bus_claim;
+ size_t txsize;
+ u32 txaddr;
+ int ret;
+
+ txaddr = rtw_sdio_get_tx_addr(rtwdev, skb->len, queue);
+ if (!txaddr)
+ return -EINVAL;
+
+ txsize = sdio_align_size(rtwsdio->sdio_func, skb->len);
+
+ ret = rtw_sdio_check_free_txpg(rtwdev, queue, txsize);
+ if (ret)
+ return ret;
+
+ if (!IS_ALIGNED((unsigned long)skb->data, RTW_SDIO_DATA_PTR_ALIGN))
+ rtw_warn(rtwdev, "Got unaligned SKB in %s() for queue %u\n",
+ __func__, queue);
+
+ bus_claim = rtw_sdio_bus_claim_needed(rtwsdio);
+
+ if (bus_claim)
+ sdio_claim_host(rtwsdio->sdio_func);
+
+ ret = sdio_memcpy_toio(rtwsdio->sdio_func, txaddr, skb->data, txsize);
+
+ if (bus_claim)
+ sdio_release_host(rtwsdio->sdio_func);
+
+ if (ret)
+ rtw_warn(rtwdev,
+ "Failed to write %zu byte(s) to SDIO port 0x%08x",
+ txsize, txaddr);
+
+ return ret;
+}
+
+static void rtw_sdio_init(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ rtwsdio->irq_mask = REG_SDIO_HIMR_RX_REQUEST | REG_SDIO_HIMR_CPWM1;
+}
+
+static void rtw_sdio_enable_rx_aggregation(struct rtw_dev *rtwdev)
+{
+ u8 size, timeout;
+
+ if (rtw_chip_wcpu_11n(rtwdev)) {
+ size = 0x6;
+ timeout = 0x6;
+ } else {
+ size = 0xff;
+ timeout = 0x1;
+ }
+
+ /* Make the firmware honor the size limit configured below */
+ rtw_write32_set(rtwdev, REG_RXDMA_AGG_PG_TH, BIT_EN_PRE_CALC);
+
+ rtw_write8_set(rtwdev, REG_TXDMA_PQ_MAP, BIT_RXDMA_AGG_EN);
+
+ rtw_write16(rtwdev, REG_RXDMA_AGG_PG_TH,
+ FIELD_PREP(BIT_RXDMA_AGG_PG_TH, size) |
+ FIELD_PREP(BIT_DMA_AGG_TO_V1, timeout));
+
+ rtw_write8_set(rtwdev, REG_RXDMA_MODE, BIT_DMA_MODE);
+}
+
+static void rtw_sdio_enable_interrupt(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ rtw_write32(rtwdev, REG_SDIO_HIMR, rtwsdio->irq_mask);
+}
+
+static void rtw_sdio_disable_interrupt(struct rtw_dev *rtwdev)
+{
+ rtw_write32(rtwdev, REG_SDIO_HIMR, 0x0);
+}
+
+static u8 rtw_sdio_get_tx_qsel(struct rtw_dev *rtwdev, struct sk_buff *skb,
+ u8 queue)
+{
+ switch (queue) {
+ case RTW_TX_QUEUE_BCN:
+ return TX_DESC_QSEL_BEACON;
+ case RTW_TX_QUEUE_H2C:
+ return TX_DESC_QSEL_H2C;
+ case RTW_TX_QUEUE_MGMT:
+ if (rtw_chip_wcpu_11n(rtwdev))
+ return TX_DESC_QSEL_HIGH;
+ else
+ return TX_DESC_QSEL_MGMT;
+ case RTW_TX_QUEUE_HI0:
+ return TX_DESC_QSEL_HIGH;
+ default:
+ return skb->priority;
+ }
+}
+
+static int rtw_sdio_setup(struct rtw_dev *rtwdev)
+{
+ /* nothing to do */
+ return 0;
+}
+
+static int rtw_sdio_start(struct rtw_dev *rtwdev)
+{
+ rtw_sdio_enable_rx_aggregation(rtwdev);
+ rtw_sdio_enable_interrupt(rtwdev);
+
+ return 0;
+}
+
+static void rtw_sdio_stop(struct rtw_dev *rtwdev)
+{
+ rtw_sdio_disable_interrupt(rtwdev);
+}
+
+static void rtw_sdio_deep_ps_enter(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ bool tx_empty = true;
+ u8 queue;
+
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_TX_WAKE)) {
+ /* Deep PS state is not allowed to TX-DMA */
+ for (queue = 0; queue < RTK_MAX_TX_QUEUE_NUM; queue++) {
+ /* BCN queue is rsvd page, does not have DMA interrupt
+ * H2C queue is managed by firmware
+ */
+ if (queue == RTW_TX_QUEUE_BCN ||
+ queue == RTW_TX_QUEUE_H2C)
+ continue;
+
+ /* check if there is any skb DMAing */
+ if (skb_queue_len(&rtwsdio->tx_queue[queue])) {
+ tx_empty = false;
+ break;
+ }
+ }
+ }
+
+ if (!tx_empty) {
+ rtw_dbg(rtwdev, RTW_DBG_PS,
+ "TX path not empty, cannot enter deep power save state\n");
+ return;
+ }
+
+ set_bit(RTW_FLAG_LEISURE_PS_DEEP, rtwdev->flags);
+ rtw_power_mode_change(rtwdev, true);
+}
+
+static void rtw_sdio_deep_ps_leave(struct rtw_dev *rtwdev)
+{
+ if (test_and_clear_bit(RTW_FLAG_LEISURE_PS_DEEP, rtwdev->flags))
+ rtw_power_mode_change(rtwdev, false);
+}
+
+static void rtw_sdio_deep_ps(struct rtw_dev *rtwdev, bool enter)
+{
+ if (enter && !test_bit(RTW_FLAG_LEISURE_PS_DEEP, rtwdev->flags))
+ rtw_sdio_deep_ps_enter(rtwdev);
+
+ if (!enter && test_bit(RTW_FLAG_LEISURE_PS_DEEP, rtwdev->flags))
+ rtw_sdio_deep_ps_leave(rtwdev);
+}
+
+static void rtw_sdio_tx_kick_off(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ queue_work(rtwsdio->txwq, &rtwsdio->tx_handler_data->work);
+}
+
+static void rtw_sdio_link_ps(struct rtw_dev *rtwdev, bool enter)
+{
+ /* nothing to do */
+}
+
+static void rtw_sdio_interface_cfg(struct rtw_dev *rtwdev)
+{
+ u32 val;
+
+ rtw_read32(rtwdev, REG_SDIO_FREE_TXPG);
+
+ val = rtw_read32(rtwdev, REG_SDIO_TX_CTRL);
+ val &= 0xfff8;
+ rtw_write32(rtwdev, REG_SDIO_TX_CTRL, val);
+}
+
+static struct rtw_sdio_tx_data *rtw_sdio_get_tx_data(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+
+ BUILD_BUG_ON(sizeof(struct rtw_sdio_tx_data) >
+ sizeof(info->status.status_driver_data));
+
+ return (struct rtw_sdio_tx_data *)info->status.status_driver_data;
+}
+
+static void rtw_sdio_tx_skb_prepare(struct rtw_dev *rtwdev,
+ struct rtw_tx_pkt_info *pkt_info,
+ struct sk_buff *skb,
+ enum rtw_tx_queue_type queue)
+{
+ const struct rtw_chip_info *chip = rtwdev->chip;
+ unsigned long data_addr, aligned_addr;
+ size_t offset;
+ u8 *pkt_desc;
+
+ pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz);
+
+ data_addr = (unsigned long)pkt_desc;
+ aligned_addr = ALIGN(data_addr, RTW_SDIO_DATA_PTR_ALIGN);
+
+ if (data_addr != aligned_addr) {
+ /* Ensure that the start of the pkt_desc is always aligned at
+ * RTW_SDIO_DATA_PTR_ALIGN.
+ */
+ offset = RTW_SDIO_DATA_PTR_ALIGN - (aligned_addr - data_addr);
+
+ pkt_desc = skb_push(skb, offset);
+
+ /* By inserting padding to align the start of the pkt_desc we
+ * need to inform the firmware that the actual data starts at
+ * a different offset than normal.
+ */
+ pkt_info->offset += offset;
+ }
+
+ memset(pkt_desc, 0, chip->tx_pkt_desc_sz);
+
+ pkt_info->qsel = rtw_sdio_get_tx_qsel(rtwdev, skb, queue);
+
+ rtw_tx_fill_tx_desc(pkt_info, skb);
+ rtw_tx_fill_txdesc_checksum(rtwdev, pkt_info, pkt_desc);
+}
+
+static int rtw_sdio_write_data(struct rtw_dev *rtwdev,
+ struct rtw_tx_pkt_info *pkt_info,
+ struct sk_buff *skb,
+ enum rtw_tx_queue_type queue)
+{
+ int ret;
+
+ rtw_sdio_tx_skb_prepare(rtwdev, pkt_info, skb, queue);
+
+ ret = rtw_sdio_write_port(rtwdev, skb, queue);
+ dev_kfree_skb_any(skb);
+
+ return ret;
+}
+
+static int rtw_sdio_write_data_rsvd_page(struct rtw_dev *rtwdev, u8 *buf,
+ u32 size)
+{
+ struct rtw_tx_pkt_info pkt_info = {};
+ struct sk_buff *skb;
+
+ skb = rtw_tx_write_data_rsvd_page_get(rtwdev, &pkt_info, buf, size);
+ if (!skb)
+ return -ENOMEM;
+
+ return rtw_sdio_write_data(rtwdev, &pkt_info, skb, RTW_TX_QUEUE_BCN);
+}
+
+static int rtw_sdio_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size)
+{
+ struct rtw_tx_pkt_info pkt_info = {};
+ struct sk_buff *skb;
+
+ skb = rtw_tx_write_data_h2c_get(rtwdev, &pkt_info, buf, size);
+ if (!skb)
+ return -ENOMEM;
+
+ return rtw_sdio_write_data(rtwdev, &pkt_info, skb, RTW_TX_QUEUE_H2C);
+}
+
+static int rtw_sdio_tx_write(struct rtw_dev *rtwdev,
+ struct rtw_tx_pkt_info *pkt_info,
+ struct sk_buff *skb)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ enum rtw_tx_queue_type queue = rtw_tx_queue_mapping(skb);
+ struct rtw_sdio_tx_data *tx_data;
+
+ rtw_sdio_tx_skb_prepare(rtwdev, pkt_info, skb, queue);
+
+ tx_data = rtw_sdio_get_tx_data(skb);
+ tx_data->sn = pkt_info->sn;
+
+ skb_queue_tail(&rtwsdio->tx_queue[queue], skb);
+
+ return 0;
+}
+
+static void rtw_sdio_tx_err_isr(struct rtw_dev *rtwdev)
+{
+ u32 val = rtw_read32(rtwdev, REG_TXDMA_STATUS);
+
+ rtw_write32(rtwdev, REG_TXDMA_STATUS, val);
+}
+
+static void rtw_sdio_rx_skb(struct rtw_dev *rtwdev, struct sk_buff *skb,
+ u32 pkt_offset, struct rtw_rx_pkt_stat *pkt_stat,
+ struct ieee80211_rx_status *rx_status)
+{
+ *IEEE80211_SKB_RXCB(skb) = *rx_status;
+
+ if (pkt_stat->is_c2h) {
+ skb_put(skb, pkt_stat->pkt_len + pkt_offset);
+ rtw_fw_c2h_cmd_rx_irqsafe(rtwdev, pkt_offset, skb);
+ return;
+ }
+
+ skb_put(skb, pkt_stat->pkt_len);
+ skb_reserve(skb, pkt_offset);
+
+ rtw_rx_stats(rtwdev, pkt_stat->vif, skb);
+
+ ieee80211_rx_irqsafe(rtwdev->hw, skb);
+}
+
+static void rtw_sdio_rxfifo_recv(struct rtw_dev *rtwdev, u32 rx_len)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ const struct rtw_chip_info *chip = rtwdev->chip;
+ u32 pkt_desc_sz = chip->rx_pkt_desc_sz;
+ struct ieee80211_rx_status rx_status;
+ struct rtw_rx_pkt_stat pkt_stat;
+ struct sk_buff *skb, *split_skb;
+ u32 pkt_offset, curr_pkt_len;
+ size_t bufsz;
+ u8 *rx_desc;
+ int ret;
+
+ bufsz = sdio_align_size(rtwsdio->sdio_func, rx_len);
+
+ skb = dev_alloc_skb(bufsz);
+ if (!skb)
+ return;
+
+ ret = rtw_sdio_read_port(rtwdev, skb->data, bufsz);
+ if (ret) {
+ dev_kfree_skb_any(skb);
+ return;
+ }
+
+ while (true) {
+ rx_desc = skb->data;
+ chip->ops->query_rx_desc(rtwdev, rx_desc, &pkt_stat,
+ &rx_status);
+ pkt_offset = pkt_desc_sz + pkt_stat.drv_info_sz +
+ pkt_stat.shift;
+
+ curr_pkt_len = ALIGN(pkt_offset + pkt_stat.pkt_len,
+ RTW_SDIO_DATA_PTR_ALIGN);
+
+ if ((curr_pkt_len + pkt_desc_sz) >= rx_len) {
+ /* Use the original skb (with it's adjusted offset)
+ * when processing the last (or even the only) entry to
+ * have it's memory freed automatically.
+ */
+ rtw_sdio_rx_skb(rtwdev, skb, pkt_offset, &pkt_stat,
+ &rx_status);
+ break;
+ }
+
+ split_skb = dev_alloc_skb(curr_pkt_len);
+ if (!split_skb) {
+ rtw_sdio_rx_skb(rtwdev, skb, pkt_offset, &pkt_stat,
+ &rx_status);
+ break;
+ }
+
+ skb_copy_header(split_skb, skb);
+ memcpy(split_skb->data, skb->data, curr_pkt_len);
+
+ rtw_sdio_rx_skb(rtwdev, split_skb, pkt_offset, &pkt_stat,
+ &rx_status);
+
+ /* Move to the start of the next RX descriptor */
+ skb_reserve(skb, curr_pkt_len);
+ rx_len -= curr_pkt_len;
+ }
+}
+
+static void rtw_sdio_rx_isr(struct rtw_dev *rtwdev)
+{
+ u32 rx_len, total_rx_bytes = 0;
+
+ while (total_rx_bytes < SZ_64K) {
+ if (rtw_chip_wcpu_11n(rtwdev))
+ rx_len = rtw_read16(rtwdev, REG_SDIO_RX0_REQ_LEN);
+ else
+ rx_len = rtw_read32(rtwdev, REG_SDIO_RX0_REQ_LEN);
+
+ if (!rx_len)
+ break;
+
+ rtw_sdio_rxfifo_recv(rtwdev, rx_len);
+
+ total_rx_bytes += rx_len;
+ }
+}
+
+static void rtw_sdio_handle_interrupt(struct sdio_func *sdio_func)
+{
+ struct ieee80211_hw *hw = sdio_get_drvdata(sdio_func);
+ struct rtw_sdio *rtwsdio;
+ struct rtw_dev *rtwdev;
+ u32 hisr;
+
+ rtwdev = hw->priv;
+ rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ rtwsdio->irq_thread = current;
+
+ hisr = rtw_read32(rtwdev, REG_SDIO_HISR);
+
+ if (hisr & REG_SDIO_HISR_TXERR)
+ rtw_sdio_tx_err_isr(rtwdev);
+ if (hisr & REG_SDIO_HISR_RX_REQUEST) {
+ hisr &= ~REG_SDIO_HISR_RX_REQUEST;
+ rtw_sdio_rx_isr(rtwdev);
+ }
+
+ rtw_write32(rtwdev, REG_SDIO_HISR, hisr);
+
+ rtwsdio->irq_thread = NULL;
+}
+
+static int __maybe_unused rtw_sdio_suspend(struct device *dev)
+{
+ struct sdio_func *func = dev_to_sdio_func(dev);
+ struct ieee80211_hw *hw = dev_get_drvdata(dev);
+ struct rtw_dev *rtwdev = hw->priv;
+ int ret;
+
+ ret = sdio_set_host_pm_flags(func, MMC_PM_KEEP_POWER);
+ if (ret)
+ rtw_err(rtwdev, "Failed to host PM flag MMC_PM_KEEP_POWER");
+
+ return ret;
+}
+
+static int __maybe_unused rtw_sdio_resume(struct device *dev)
+{
+ return 0;
+}
+
+SIMPLE_DEV_PM_OPS(rtw_sdio_pm_ops, rtw_sdio_suspend, rtw_sdio_resume);
+EXPORT_SYMBOL(rtw_sdio_pm_ops);
+
+static int rtw_sdio_claim(struct rtw_dev *rtwdev, struct sdio_func *sdio_func)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ int ret;
+
+ sdio_claim_host(sdio_func);
+
+ ret = sdio_enable_func(sdio_func);
+ if (ret) {
+ rtw_err(rtwdev, "Failed to enable SDIO func");
+ goto err_release_host;
+ }
+
+ ret = sdio_set_block_size(sdio_func, RTW_SDIO_BLOCK_SIZE);
+ if (ret) {
+ rtw_err(rtwdev, "Failed to set SDIO block size to 512");
+ goto err_disable_func;
+ }
+
+ rtwsdio->sdio_func = sdio_func;
+
+ rtwsdio->sdio3_bus_mode = mmc_card_uhs(sdio_func->card);
+
+ sdio_set_drvdata(sdio_func, rtwdev->hw);
+ SET_IEEE80211_DEV(rtwdev->hw, &sdio_func->dev);
+
+ sdio_release_host(sdio_func);
+
+ return 0;
+
+err_disable_func:
+ sdio_disable_func(sdio_func);
+err_release_host:
+ sdio_release_host(sdio_func);
+ return ret;
+}
+
+static void rtw_sdio_declaim(struct rtw_dev *rtwdev,
+ struct sdio_func *sdio_func)
+{
+ sdio_claim_host(sdio_func);
+ sdio_disable_func(sdio_func);
+ sdio_release_host(sdio_func);
+}
+
+static struct rtw_hci_ops rtw_sdio_ops = {
+ .tx_write = rtw_sdio_tx_write,
+ .tx_kick_off = rtw_sdio_tx_kick_off,
+ .setup = rtw_sdio_setup,
+ .start = rtw_sdio_start,
+ .stop = rtw_sdio_stop,
+ .deep_ps = rtw_sdio_deep_ps,
+ .link_ps = rtw_sdio_link_ps,
+ .interface_cfg = rtw_sdio_interface_cfg,
+
+ .read8 = rtw_sdio_read8,
+ .read16 = rtw_sdio_read16,
+ .read32 = rtw_sdio_read32,
+ .write8 = rtw_sdio_write8,
+ .write16 = rtw_sdio_write16,
+ .write32 = rtw_sdio_write32,
+ .write_data_rsvd_page = rtw_sdio_write_data_rsvd_page,
+ .write_data_h2c = rtw_sdio_write_data_h2c,
+};
+
+static int rtw_sdio_request_irq(struct rtw_dev *rtwdev,
+ struct sdio_func *sdio_func)
+{
+ int ret;
+
+ sdio_claim_host(sdio_func);
+ ret = sdio_claim_irq(sdio_func, &rtw_sdio_handle_interrupt);
+ sdio_release_host(sdio_func);
+
+ if (ret) {
+ rtw_err(rtwdev, "failed to claim SDIO IRQ");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void rtw_sdio_indicate_tx_status(struct rtw_dev *rtwdev,
+ struct sk_buff *skb)
+{
+ struct rtw_sdio_tx_data *tx_data = rtw_sdio_get_tx_data(skb);
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct ieee80211_hw *hw = rtwdev->hw;
+
+ /* enqueue to wait for tx report */
+ if (info->flags & IEEE80211_TX_CTL_REQ_TX_STATUS) {
+ rtw_tx_report_enqueue(rtwdev, skb, tx_data->sn);
+ return;
+ }
+
+ /* always ACK for others, then they won't be marked as drop */
+ ieee80211_tx_info_clear_status(info);
+ if (info->flags & IEEE80211_TX_CTL_NO_ACK)
+ info->flags |= IEEE80211_TX_STAT_NOACK_TRANSMITTED;
+ else
+ info->flags |= IEEE80211_TX_STAT_ACK;
+
+ ieee80211_tx_status_irqsafe(hw, skb);
+}
+
+static void rtw_sdio_process_tx_queue(struct rtw_dev *rtwdev,
+ enum rtw_tx_queue_type queue)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = skb_dequeue(&rtwsdio->tx_queue[queue]);
+ if (!skb)
+ return;
+
+ ret = rtw_sdio_write_port(rtwdev, skb, queue);
+ if (ret) {
+ skb_queue_head(&rtwsdio->tx_queue[queue], skb);
+ return;
+ }
+
+ if (queue <= RTW_TX_QUEUE_VO)
+ rtw_sdio_indicate_tx_status(rtwdev, skb);
+ else
+ dev_kfree_skb_any(skb);
+}
+
+static void rtw_sdio_tx_handler(struct work_struct *work)
+{
+ struct rtw_sdio_work_data *work_data =
+ container_of(work, struct rtw_sdio_work_data, work);
+ struct rtw_sdio *rtwsdio;
+ struct rtw_dev *rtwdev;
+ int limit, queue;
+
+ rtwdev = work_data->rtwdev;
+ rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ if (!rtw_fw_feature_check(&rtwdev->fw, FW_FEATURE_TX_WAKE))
+ rtw_sdio_deep_ps_leave(rtwdev);
+
+ for (queue = RTK_MAX_TX_QUEUE_NUM - 1; queue >= 0; queue--) {
+ for (limit = 0; limit < 1000; limit++) {
+ rtw_sdio_process_tx_queue(rtwdev, queue);
+
+ if (skb_queue_empty(&rtwsdio->tx_queue[queue]))
+ break;
+ }
+ }
+}
+
+static void rtw_sdio_free_irq(struct rtw_dev *rtwdev,
+ struct sdio_func *sdio_func)
+{
+ sdio_claim_host(sdio_func);
+ sdio_release_irq(sdio_func);
+ sdio_release_host(sdio_func);
+}
+
+static int rtw_sdio_init_tx(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ int i;
+
+ rtwsdio->txwq = create_singlethread_workqueue("rtw88_sdio: tx wq");
+ if (!rtwsdio->txwq) {
+ rtw_err(rtwdev, "failed to create TX work queue\n");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < RTK_MAX_TX_QUEUE_NUM; i++)
+ skb_queue_head_init(&rtwsdio->tx_queue[i]);
+ rtwsdio->tx_handler_data = kmalloc(sizeof(*rtwsdio->tx_handler_data),
+ GFP_KERNEL);
+ if (!rtwsdio->tx_handler_data)
+ goto err_destroy_wq;
+
+ rtwsdio->tx_handler_data->rtwdev = rtwdev;
+ INIT_WORK(&rtwsdio->tx_handler_data->work, rtw_sdio_tx_handler);
+
+ return 0;
+
+err_destroy_wq:
+ destroy_workqueue(rtwsdio->txwq);
+ return -ENOMEM;
+}
+
+static void rtw_sdio_deinit_tx(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+ int i;
+
+ for (i = 0; i < RTK_MAX_TX_QUEUE_NUM; i++)
+ skb_queue_purge(&rtwsdio->tx_queue[i]);
+
+ flush_workqueue(rtwsdio->txwq);
+ destroy_workqueue(rtwsdio->txwq);
+ kfree(rtwsdio->tx_handler_data);
+}
+
+int rtw_sdio_probe(struct sdio_func *sdio_func,
+ const struct sdio_device_id *id)
+{
+ struct ieee80211_hw *hw;
+ struct rtw_dev *rtwdev;
+ int drv_data_size;
+ int ret;
+
+ drv_data_size = sizeof(struct rtw_dev) + sizeof(struct rtw_sdio);
+ hw = ieee80211_alloc_hw(drv_data_size, &rtw_ops);
+ if (!hw) {
+ dev_err(&sdio_func->dev, "failed to allocate hw");
+ return -ENOMEM;
+ }
+
+ rtwdev = hw->priv;
+ rtwdev->hw = hw;
+ rtwdev->dev = &sdio_func->dev;
+ rtwdev->chip = (struct rtw_chip_info *)id->driver_data;
+ rtwdev->hci.ops = &rtw_sdio_ops;
+ rtwdev->hci.type = RTW_HCI_TYPE_SDIO;
+
+ ret = rtw_core_init(rtwdev);
+ if (ret)
+ goto err_release_hw;
+
+ rtw_dbg(rtwdev, RTW_DBG_SDIO,
+ "rtw88 SDIO probe: vendor=0x%04x device=%04x class=%02x",
+ id->vendor, id->device, id->class);
+
+ ret = rtw_sdio_claim(rtwdev, sdio_func);
+ if (ret) {
+ rtw_err(rtwdev, "failed to claim SDIO device");
+ goto err_deinit_core;
+ }
+
+ rtw_sdio_init(rtwdev);
+
+ ret = rtw_sdio_init_tx(rtwdev);
+ if (ret) {
+ rtw_err(rtwdev, "failed to init SDIO TX queue\n");
+ goto err_sdio_declaim;
+ }
+
+ ret = rtw_chip_info_setup(rtwdev);
+ if (ret) {
+ rtw_err(rtwdev, "failed to setup chip information");
+ goto err_destroy_txwq;
+ }
+
+ ret = rtw_sdio_request_irq(rtwdev, sdio_func);
+ if (ret)
+ goto err_destroy_txwq;
+
+ ret = rtw_register_hw(rtwdev, hw);
+ if (ret) {
+ rtw_err(rtwdev, "failed to register hw");
+ goto err_free_irq;
+ }
+
+ return 0;
+
+err_free_irq:
+ rtw_sdio_free_irq(rtwdev, sdio_func);
+err_destroy_txwq:
+ rtw_sdio_deinit_tx(rtwdev);
+err_sdio_declaim:
+ rtw_sdio_declaim(rtwdev, sdio_func);
+err_deinit_core:
+ rtw_core_deinit(rtwdev);
+err_release_hw:
+ ieee80211_free_hw(hw);
+
+ return ret;
+}
+EXPORT_SYMBOL(rtw_sdio_probe);
+
+void rtw_sdio_remove(struct sdio_func *sdio_func)
+{
+ struct ieee80211_hw *hw = sdio_get_drvdata(sdio_func);
+ struct rtw_dev *rtwdev;
+
+ if (!hw)
+ return;
+
+ rtwdev = hw->priv;
+
+ rtw_unregister_hw(rtwdev, hw);
+ rtw_sdio_disable_interrupt(rtwdev);
+ rtw_sdio_free_irq(rtwdev, sdio_func);
+ rtw_sdio_declaim(rtwdev, sdio_func);
+ rtw_sdio_deinit_tx(rtwdev);
+ rtw_core_deinit(rtwdev);
+ ieee80211_free_hw(hw);
+}
+EXPORT_SYMBOL(rtw_sdio_remove);
+
+void rtw_sdio_shutdown(struct device *dev)
+{
+ struct sdio_func *sdio_func = dev_to_sdio_func(dev);
+ const struct rtw_chip_info *chip;
+ struct ieee80211_hw *hw;
+ struct rtw_dev *rtwdev;
+
+ hw = sdio_get_drvdata(sdio_func);
+ if (!hw)
+ return;
+
+ rtwdev = hw->priv;
+ chip = rtwdev->chip;
+
+ if (chip->ops->shutdown)
+ chip->ops->shutdown(rtwdev);
+}
+EXPORT_SYMBOL(rtw_sdio_shutdown);
+
+MODULE_AUTHOR("Martin Blumenstingl");
+MODULE_AUTHOR("Jernej Skrabec");
+MODULE_DESCRIPTION("Realtek 802.11ac wireless SDIO driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw88/sdio.h b/drivers/net/wireless/realtek/rtw88/sdio.h
new file mode 100644
index 0000000..3c659ed
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw88/sdio.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/* Copyright (C) 2021 Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+ * Copyright (C) 2021 Jernej Skrabec <jernej.skrabec@gmail.com>
+ */
+
+#ifndef __REG_SDIO_H_
+#define __REG_SDIO_H_
+
+/* I/O bus domain address mapping */
+#define SDIO_LOCAL_OFFSET 0x10250000
+#define WLAN_IOREG_OFFSET 0x10260000
+#define FIRMWARE_FIFO_OFFSET 0x10270000
+#define TX_HIQ_OFFSET 0x10310000
+#define TX_MIQ_OFFSET 0x10320000
+#define TX_LOQ_OFFSET 0x10330000
+#define TX_EPQ_OFFSET 0x10350000
+#define RX_RX0FF_OFFSET 0x10340000
+
+#define RTW_SDIO_BUS_MSK 0xffff0000
+#define SDIO_LOCAL_REG_MSK 0x00000fff
+#define WLAN_IOREG_REG_MSK 0x0000ffff
+
+/* SDIO Tx Control */
+#define REG_SDIO_TX_CTRL (SDIO_LOCAL_OFFSET + 0x0000)
+
+/*SDIO status timeout*/
+#define REG_SDIO_TIMEOUT (SDIO_LOCAL_OFFSET + 0x0002)
+
+/* SDIO Host Interrupt Mask */
+#define REG_SDIO_HIMR (SDIO_LOCAL_OFFSET + 0x0014)
+#define REG_SDIO_HIMR_RX_REQUEST BIT(0)
+#define REG_SDIO_HIMR_AVAL BIT(1)
+#define REG_SDIO_HIMR_TXERR BIT(2)
+#define REG_SDIO_HIMR_RXERR BIT(3)
+#define REG_SDIO_HIMR_TXFOVW BIT(4)
+#define REG_SDIO_HIMR_RXFOVW BIT(5)
+#define REG_SDIO_HIMR_TXBCNOK BIT(6)
+#define REG_SDIO_HIMR_TXBCNERR BIT(7)
+#define REG_SDIO_HIMR_BCNERLY_INT BIT(16)
+#define REG_SDIO_HIMR_C2HCMD BIT(17)
+#define REG_SDIO_HIMR_CPWM1 BIT(18)
+#define REG_SDIO_HIMR_CPWM2 BIT(19)
+#define REG_SDIO_HIMR_HSISR_IND BIT(20)
+#define REG_SDIO_HIMR_GTINT3_IND BIT(21)
+#define REG_SDIO_HIMR_GTINT4_IND BIT(22)
+#define REG_SDIO_HIMR_PSTIMEOUT BIT(23)
+#define REG_SDIO_HIMR_OCPINT BIT(24)
+#define REG_SDIO_HIMR_ATIMEND BIT(25)
+#define REG_SDIO_HIMR_ATIMEND_E BIT(26)
+#define REG_SDIO_HIMR_CTWEND BIT(27)
+/* the following two are RTL8188 SDIO Specific */
+#define REG_SDIO_HIMR_MCU_ERR BIT(28)
+#define REG_SDIO_HIMR_TSF_BIT32_TOGGLE BIT(29)
+
+/* SDIO Host Interrupt Service Routine */
+#define REG_SDIO_HISR (SDIO_LOCAL_OFFSET + 0x0018)
+#define REG_SDIO_HISR_RX_REQUEST BIT(0)
+#define REG_SDIO_HISR_AVAL BIT(1)
+#define REG_SDIO_HISR_TXERR BIT(2)
+#define REG_SDIO_HISR_RXERR BIT(3)
+#define REG_SDIO_HISR_TXFOVW BIT(4)
+#define REG_SDIO_HISR_RXFOVW BIT(5)
+#define REG_SDIO_HISR_TXBCNOK BIT(6)
+#define REG_SDIO_HISR_TXBCNERR BIT(7)
+#define REG_SDIO_HISR_BCNERLY_INT BIT(16)
+#define REG_SDIO_HISR_C2HCMD BIT(17)
+#define REG_SDIO_HISR_CPWM1 BIT(18)
+#define REG_SDIO_HISR_CPWM2 BIT(19)
+#define REG_SDIO_HISR_HSISR_IND BIT(20)
+#define REG_SDIO_HISR_GTINT3_IND BIT(21)
+#define REG_SDIO_HISR_GTINT4_IND BIT(22)
+#define REG_SDIO_HISR_PSTIMEOUT BIT(23)
+#define REG_SDIO_HISR_OCPINT BIT(24)
+#define REG_SDIO_HISR_ATIMEND BIT(25)
+#define REG_SDIO_HISR_ATIMEND_E BIT(26)
+#define REG_SDIO_HISR_CTWEND BIT(27)
+/* the following two are RTL8188 SDIO Specific */
+#define REG_SDIO_HISR_MCU_ERR BIT(28)
+#define REG_SDIO_HISR_TSF_BIT32_TOGGLE BIT(29)
+
+/* HCI Current Power Mode */
+#define REG_SDIO_HCPWM (SDIO_LOCAL_OFFSET + 0x0019)
+/* RXDMA Request Length */
+#define REG_SDIO_RX0_REQ_LEN (SDIO_LOCAL_OFFSET + 0x001C)
+/* OQT Free Page */
+#define REG_SDIO_OQT_FREE_PG (SDIO_LOCAL_OFFSET + 0x001E)
+/* Free Tx Buffer Page */
+#define REG_SDIO_FREE_TXPG (SDIO_LOCAL_OFFSET + 0x0020)
+/* HCI Current Power Mode 1 */
+#define REG_SDIO_HCPWM1 (SDIO_LOCAL_OFFSET + 0x0024)
+/* HCI Current Power Mode 2 */
+#define REG_SDIO_HCPWM2 (SDIO_LOCAL_OFFSET + 0x0026)
+/* Free Tx Page Sequence */
+#define REG_SDIO_FREE_TXPG_SEQ (SDIO_LOCAL_OFFSET + 0x0028)
+/* HTSF Information */
+#define REG_SDIO_HTSFR_INFO (SDIO_LOCAL_OFFSET + 0x0030)
+#define REG_SDIO_HCPWM1_V2 (SDIO_LOCAL_OFFSET + 0x0038)
+/* H2C */
+#define REG_SDIO_H2C (SDIO_LOCAL_OFFSET + 0x0060)
+/* HCI Request Power Mode 1 */
+#define REG_SDIO_HRPWM1 (SDIO_LOCAL_OFFSET + 0x0080)
+/* HCI Request Power Mode 2 */
+#define REG_SDIO_HRPWM2 (SDIO_LOCAL_OFFSET + 0x0082)
+/* HCI Power Save Clock */
+#define REG_SDIO_HPS_CLKR (SDIO_LOCAL_OFFSET + 0x0084)
+/* SDIO HCI Suspend Control */
+#define REG_SDIO_HSUS_CTRL (SDIO_LOCAL_OFFSET + 0x0086)
+#define BIT_HCI_SUS_REQ BIT(0)
+#define BIT_HCI_RESUME_RDY BIT(1)
+/* SDIO Host Extension Interrupt Mask Always */
+#define REG_SDIO_HIMR_ON (SDIO_LOCAL_OFFSET + 0x0090)
+/* SDIO Host Extension Interrupt Status Always */
+#define REG_SDIO_HISR_ON (SDIO_LOCAL_OFFSET + 0x0091)
+
+#define REG_SDIO_INDIRECT_REG_CFG (SDIO_LOCAL_OFFSET + 0x0040)
+#define BIT_SDIO_INDIRECT_REG_CFG_WORD BIT(16)
+#define BIT_SDIO_INDIRECT_REG_CFG_DWORD BIT(17)
+#define BIT_SDIO_INDIRECT_REG_CFG_WRITE BIT(18)
+#define BIT_SDIO_INDIRECT_REG_CFG_READ BIT(19)
+#define BIT_SDIO_INDIRECT_REG_CFG_UNK20 BIT(20)
+#define REG_SDIO_INDIRECT_REG_DATA (SDIO_LOCAL_OFFSET + 0x0044)
+
+/* Sdio Address for SDIO Local Reg, TRX FIFO, MAC Reg */
+#define REG_SDIO_CMD_ADDR_MSK GENMASK(16, 13)
+#define REG_SDIO_CMD_ADDR_SDIO_REG 0
+#define REG_SDIO_CMD_ADDR_MAC_REG 8
+#define REG_SDIO_CMD_ADDR_TXFF_HIGH 4
+#define REG_SDIO_CMD_ADDR_TXFF_LOW 6
+#define REG_SDIO_CMD_ADDR_TXFF_NORMAL 5
+#define REG_SDIO_CMD_ADDR_TXFF_EXTRA 7
+#define REG_SDIO_CMD_ADDR_RXFF 7
+
+#define RTW_SDIO_BLOCK_SIZE 512
+#define RTW_SDIO_ADDR_RX_RX0FF_GEN(_id) (0x0e000 | ((_id) & 0x3))
+
+#define RTW_SDIO_DATA_PTR_ALIGN 8
+
+struct sdio_func;
+struct sdio_device_id;
+
+struct rtw_sdio_tx_data {
+ u8 sn;
+};
+
+struct rtw_sdio_work_data {
+ struct work_struct work;
+ struct rtw_dev *rtwdev;
+};
+
+struct rtw_sdio {
+ struct sdio_func *sdio_func;
+
+ u32 irq_mask;
+ u8 rx_addr;
+ bool sdio3_bus_mode;
+
+ void *irq_thread;
+
+ struct workqueue_struct *txwq;
+ struct rtw_sdio_work_data *tx_handler_data;
+ struct sk_buff_head tx_queue[RTK_MAX_TX_QUEUE_NUM];
+};
+
+extern const struct dev_pm_ops rtw_sdio_pm_ops;
+
+int rtw_sdio_probe(struct sdio_func *sdio_func,
+ const struct sdio_device_id *id);
+void rtw_sdio_remove(struct sdio_func *sdio_func);
+void rtw_sdio_shutdown(struct device *dev);
+
+static inline bool rtw_sdio_is_sdio30_supported(struct rtw_dev *rtwdev)
+{
+ struct rtw_sdio *rtwsdio = (struct rtw_sdio *)rtwdev->priv;
+
+ return rtwsdio->sdio3_bus_mode;
+}
+
+#endif
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index 68e1b782d..44a5faf 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -118,6 +118,22 @@ static void rtw_usb_write32(struct rtw_dev *rtwdev, u32 addr, u32 val)
rtw_usb_write(rtwdev, addr, val, 4);
}
+static int dma_mapping_to_ep(enum rtw_dma_mapping dma_mapping)
+{
+ switch (dma_mapping) {
+ case RTW_DMA_MAPPING_HIGH:
+ return 0;
+ case RTW_DMA_MAPPING_NORMAL:
+ return 1;
+ case RTW_DMA_MAPPING_LOW:
+ return 2;
+ case RTW_DMA_MAPPING_EXTRA:
+ return 3;
+ default:
+ return -EINVAL;
+ }
+}
+
static int rtw_usb_parse(struct rtw_dev *rtwdev,
struct usb_interface *interface)
{
@@ -129,6 +145,8 @@ static int rtw_usb_parse(struct rtw_dev *rtwdev,
int num_out_pipes = 0;
int i;
u8 num;
+ const struct rtw_chip_info *chip = rtwdev->chip;
+ const struct rtw_rqpn *rqpn;
for (i = 0; i < interface_desc->bNumEndpoints; i++) {
endpoint = &host_interface->endpoint[i].desc;
@@ -183,31 +201,34 @@ static int rtw_usb_parse(struct rtw_dev *rtwdev,
rtwdev->hci.bulkout_num = num_out_pipes;
- switch (num_out_pipes) {
- case 4:
- case 3:
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID0] = 2;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID1] = 2;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID2] = 2;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID3] = 2;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID4] = 1;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID5] = 1;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID6] = 0;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID7] = 0;
- break;
- case 2:
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID0] = 1;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID1] = 1;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID2] = 1;
- rtwusb->qsel_to_ep[TX_DESC_QSEL_TID3] = 1;
- break;
- case 1:
- break;
- default:
- rtw_err(rtwdev, "failed to get out_pipes(%d)\n", num_out_pipes);
+ if (num_out_pipes < 1 || num_out_pipes > 4) {
+ rtw_err(rtwdev, "invalid number of endpoints %d\n", num_out_pipes);
return -EINVAL;
}
+ rqpn = &chip->rqpn_table[num_out_pipes];
+
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID0] = dma_mapping_to_ep(rqpn->dma_map_be);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID1] = dma_mapping_to_ep(rqpn->dma_map_bk);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID2] = dma_mapping_to_ep(rqpn->dma_map_bk);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID3] = dma_mapping_to_ep(rqpn->dma_map_be);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID4] = dma_mapping_to_ep(rqpn->dma_map_vi);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID5] = dma_mapping_to_ep(rqpn->dma_map_vi);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID6] = dma_mapping_to_ep(rqpn->dma_map_vo);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID7] = dma_mapping_to_ep(rqpn->dma_map_vo);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID8] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID9] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID10] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID11] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID12] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID13] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID14] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_TID15] = -EINVAL;
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_BEACON] = dma_mapping_to_ep(rqpn->dma_map_hi);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_HIGH] = dma_mapping_to_ep(rqpn->dma_map_hi);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_MGMT] = dma_mapping_to_ep(rqpn->dma_map_mg);
+ rtwusb->qsel_to_ep[TX_DESC_QSEL_H2C] = dma_mapping_to_ep(rqpn->dma_map_hi);
+
return 0;
}
@@ -250,7 +271,7 @@ static void rtw_usb_write_port_tx_complete(struct urb *urb)
static int qsel_to_ep(struct rtw_usb *rtwusb, unsigned int qsel)
{
if (qsel >= ARRAY_SIZE(rtwusb->qsel_to_ep))
- return 0;
+ return -EINVAL;
return rtwusb->qsel_to_ep[qsel];
}
@@ -265,6 +286,9 @@ static int rtw_usb_write_port(struct rtw_dev *rtwdev, u8 qsel, struct sk_buff *s
int ret;
int ep = qsel_to_ep(rtwusb, qsel);
+ if (ep < 0)
+ return ep;
+
pipe = usb_sndbulkpipe(usbd, rtwusb->out_ep[ep]);
urb = usb_alloc_urb(0, GFP_ATOMIC);
if (!urb)
@@ -780,6 +804,7 @@ static void rtw_usb_intf_deinit(struct rtw_dev *rtwdev,
struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev);
usb_put_dev(rtwusb->udev);
+ kfree(rtwusb->usb_data);
usb_set_intfdata(intf, NULL);
}
diff --git a/drivers/net/wireless/realtek/rtw89/chan.c b/drivers/net/wireless/realtek/rtw89/chan.c
index 9059680..4663db4 100644
--- a/drivers/net/wireless/realtek/rtw89/chan.c
+++ b/drivers/net/wireless/realtek/rtw89/chan.c
@@ -141,6 +141,38 @@ void rtw89_config_entity_chandef(struct rtw89_dev *rtwdev,
__rtw89_config_entity_chandef(rtwdev, idx, chandef, true);
}
+void rtw89_config_roc_chandef(struct rtw89_dev *rtwdev,
+ enum rtw89_sub_entity_idx idx,
+ const struct cfg80211_chan_def *chandef)
+{
+ struct rtw89_hal *hal = &rtwdev->hal;
+ enum rtw89_sub_entity_idx cur;
+
+ if (chandef) {
+ cur = atomic_cmpxchg(&hal->roc_entity_idx,
+ RTW89_SUB_ENTITY_IDLE, idx);
+ if (cur != RTW89_SUB_ENTITY_IDLE) {
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX,
+ "ROC still processing on entity %d\n", idx);
+ return;
+ }
+
+ hal->roc_chandef = *chandef;
+ } else {
+ cur = atomic_cmpxchg(&hal->roc_entity_idx, idx,
+ RTW89_SUB_ENTITY_IDLE);
+ if (cur == idx)
+ return;
+
+ if (cur == RTW89_SUB_ENTITY_IDLE)
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX,
+ "ROC already finished on entity %d\n", idx);
+ else
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX,
+ "ROC is processing on entity %d\n", cur);
+ }
+}
+
static void rtw89_config_default_chandef(struct rtw89_dev *rtwdev)
{
struct cfg80211_chan_def chandef = {0};
@@ -154,6 +186,7 @@ void rtw89_entity_init(struct rtw89_dev *rtwdev)
struct rtw89_hal *hal = &rtwdev->hal;
bitmap_zero(hal->entity_map, NUM_OF_RTW89_SUB_ENTITY);
+ atomic_set(&hal->roc_entity_idx, RTW89_SUB_ENTITY_IDLE);
rtw89_config_default_chandef(rtwdev);
}
@@ -229,6 +262,8 @@ void rtw89_chanctx_ops_remove(struct rtw89_dev *rtwdev,
rtwvif->sub_entity_idx = RTW89_SUB_ENTITY_0;
}
+ atomic_cmpxchg(&hal->roc_entity_idx, roll, RTW89_SUB_ENTITY_0);
+
drop = roll;
out:
diff --git a/drivers/net/wireless/realtek/rtw89/chan.h b/drivers/net/wireless/realtek/rtw89/chan.h
index ecbd450..bdf369d 100644
--- a/drivers/net/wireless/realtek/rtw89/chan.h
+++ b/drivers/net/wireless/realtek/rtw89/chan.h
@@ -45,6 +45,9 @@ bool rtw89_assign_entity_chan(struct rtw89_dev *rtwdev,
void rtw89_config_entity_chandef(struct rtw89_dev *rtwdev,
enum rtw89_sub_entity_idx idx,
const struct cfg80211_chan_def *chandef);
+void rtw89_config_roc_chandef(struct rtw89_dev *rtwdev,
+ enum rtw89_sub_entity_idx idx,
+ const struct cfg80211_chan_def *chandef);
void rtw89_entity_init(struct rtw89_dev *rtwdev);
enum rtw89_entity_mode rtw89_entity_recalc(struct rtw89_dev *rtwdev);
int rtw89_chanctx_ops_add(struct rtw89_dev *rtwdev,
diff --git a/drivers/net/wireless/realtek/rtw89/core.c b/drivers/net/wireless/realtek/rtw89/core.c
index 56a13be..7fc0a26 100644
--- a/drivers/net/wireless/realtek/rtw89/core.c
+++ b/drivers/net/wireless/realtek/rtw89/core.c
@@ -156,6 +156,28 @@ static struct ieee80211_rate rtw89_bitrates[] = {
{ .bitrate = 540, .hw_value = 0x0b, },
};
+static const struct ieee80211_iface_limit rtw89_iface_limits[] = {
+ {
+ .max = 1,
+ .types = BIT(NL80211_IFTYPE_STATION),
+ },
+ {
+ .max = 1,
+ .types = BIT(NL80211_IFTYPE_P2P_CLIENT) |
+ BIT(NL80211_IFTYPE_P2P_GO) |
+ BIT(NL80211_IFTYPE_AP),
+ },
+};
+
+static const struct ieee80211_iface_combination rtw89_iface_combs[] = {
+ {
+ .limits = rtw89_iface_limits,
+ .n_limits = ARRAY_SIZE(rtw89_iface_limits),
+ .max_interfaces = 2,
+ .num_different_channels = 1,
+ }
+};
+
bool rtw89_ra_report_to_bitrate(struct rtw89_dev *rtwdev, u8 rpt_rate, u16 *bitrate)
{
struct ieee80211_rate rate;
@@ -360,6 +382,15 @@ void rtw89_set_channel(struct rtw89_dev *rtwdev)
rtw89_set_entity_state(rtwdev, true);
}
+void rtw89_get_channel(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ struct rtw89_chan *chan)
+{
+ const struct cfg80211_chan_def *chandef;
+
+ chandef = rtw89_chandef_get(rtwdev, rtwvif->sub_entity_idx);
+ rtw89_get_channel_params(chandef, chan);
+}
+
static enum rtw89_core_tx_type
rtw89_core_get_tx_type(struct rtw89_dev *rtwdev,
struct sk_buff *skb)
@@ -707,7 +738,7 @@ static u16 rtw89_core_get_data_rate(struct rtw89_dev *rtwdev,
else
lowest_rate = RTW89_HW_RATE_OFDM6;
- if (!sta->deflink.supp_rates[chan->band_type])
+ if (!sta || !sta->deflink.supp_rates[chan->band_type])
return lowest_rate;
return __ffs(sta->deflink.supp_rates[chan->band_type]) + lowest_rate;
@@ -867,6 +898,37 @@ void rtw89_core_tx_kick_off(struct rtw89_dev *rtwdev, u8 qsel)
rtw89_hci_tx_kick_off(rtwdev, ch_dma);
}
+int rtw89_core_tx_kick_off_and_wait(struct rtw89_dev *rtwdev, struct sk_buff *skb,
+ int qsel, unsigned int timeout)
+{
+ struct rtw89_tx_skb_data *skb_data = RTW89_TX_SKB_CB(skb);
+ struct rtw89_tx_wait_info *wait;
+ unsigned long time_left;
+ int ret = 0;
+
+ wait = kzalloc(sizeof(*wait), GFP_KERNEL);
+ if (!wait) {
+ rtw89_core_tx_kick_off(rtwdev, qsel);
+ return 0;
+ }
+
+ init_completion(&wait->completion);
+ rcu_assign_pointer(skb_data->wait, wait);
+
+ rtw89_core_tx_kick_off(rtwdev, qsel);
+ time_left = wait_for_completion_timeout(&wait->completion,
+ msecs_to_jiffies(timeout));
+ if (time_left == 0)
+ ret = -ETIMEDOUT;
+ else if (!wait->tx_done)
+ ret = -EAGAIN;
+
+ rcu_assign_pointer(skb_data->wait, NULL);
+ kfree_rcu(wait, rcu_head);
+
+ return ret;
+}
+
int rtw89_h2c_tx(struct rtw89_dev *rtwdev,
struct sk_buff *skb, bool fwdl)
{
@@ -1457,6 +1519,7 @@ static void rtw89_vif_rx_stats_iter(void *data, u8 *mac,
struct rtw89_rx_desc_info *desc_info = iter_data->desc_info;
struct sk_buff *skb = iter_data->skb;
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct rtw89_rx_phy_ppdu *phy_ppdu = iter_data->phy_ppdu;
const u8 *bssid = iter_data->bssid;
if (rtwdev->scanning &&
@@ -1475,8 +1538,11 @@ static void rtw89_vif_rx_stats_iter(void *data, u8 *mac,
if (!ether_addr_equal(vif->bss_conf.bssid, bssid))
return;
- if (ieee80211_is_beacon(hdr->frame_control))
+ if (ieee80211_is_beacon(hdr->frame_control)) {
+ if (vif->type == NL80211_IFTYPE_STATION)
+ rtw89_fw_h2c_rssi_offload(rtwdev, phy_ppdu);
pkt_stat->beacon_nr++;
+ }
if (!ether_addr_equal(vif->addr, hdr->addr1))
return;
@@ -1979,6 +2045,18 @@ static void rtw89_core_free_sta_pending_forbid_ba(struct rtw89_dev *rtwdev,
spin_unlock_bh(&rtwdev->ba_lock);
}
+static void rtw89_core_free_sta_pending_roc_tx(struct rtw89_dev *rtwdev,
+ struct ieee80211_sta *sta)
+{
+ struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv;
+ struct sk_buff *skb, *tmp;
+
+ skb_queue_walk_safe(&rtwsta->roc_queue, skb, tmp) {
+ skb_unlink(skb, &rtwsta->roc_queue);
+ dev_kfree_skb_any(skb);
+ }
+}
+
static void rtw89_core_stop_tx_ba_session(struct rtw89_dev *rtwdev,
struct rtw89_txq *rtwtxq)
{
@@ -2118,6 +2196,7 @@ static void rtw89_core_txq_schedule(struct rtw89_dev *rtwdev, u8 ac, bool *reinv
{
struct ieee80211_hw *hw = rtwdev->hw;
struct ieee80211_txq *txq;
+ struct rtw89_vif *rtwvif;
struct rtw89_txq *rtwtxq;
unsigned long frame_cnt;
unsigned long byte_cnt;
@@ -2127,6 +2206,12 @@ static void rtw89_core_txq_schedule(struct rtw89_dev *rtwdev, u8 ac, bool *reinv
ieee80211_txq_schedule_start(hw, ac);
while ((txq = ieee80211_next_txq(hw, ac))) {
rtwtxq = (struct rtw89_txq *)txq->drv_priv;
+ rtwvif = (struct rtw89_vif *)txq->vif->drv_priv;
+
+ if (rtwvif->offchan) {
+ ieee80211_return_txq(hw, txq, true);
+ continue;
+ }
tx_resource = rtw89_check_and_reclaim_tx_resource(rtwdev, txq->tid);
sched_txq = false;
@@ -2153,8 +2238,7 @@ static void rtw89_ips_work(struct work_struct *work)
struct rtw89_dev *rtwdev = container_of(work, struct rtw89_dev,
ips_work);
mutex_lock(&rtwdev->mutex);
- if (rtwdev->hw->conf.flags & IEEE80211_CONF_IDLE)
- rtw89_enter_ips(rtwdev);
+ rtw89_enter_ips_by_hwflags(rtwdev);
mutex_unlock(&rtwdev->mutex);
}
@@ -2195,6 +2279,187 @@ static void rtw89_forbid_ba_work(struct work_struct *w)
spin_unlock_bh(&rtwdev->ba_lock);
}
+static void rtw89_core_sta_pending_tx_iter(void *data,
+ struct ieee80211_sta *sta)
+{
+ struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv;
+ struct rtw89_vif *rtwvif_target = data, *rtwvif = rtwsta->rtwvif;
+ struct rtw89_dev *rtwdev = rtwvif->rtwdev;
+ struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif);
+ struct sk_buff *skb, *tmp;
+ int qsel, ret;
+
+ if (rtwvif->sub_entity_idx != rtwvif_target->sub_entity_idx)
+ return;
+
+ if (skb_queue_len(&rtwsta->roc_queue) == 0)
+ return;
+
+ skb_queue_walk_safe(&rtwsta->roc_queue, skb, tmp) {
+ skb_unlink(skb, &rtwsta->roc_queue);
+
+ ret = rtw89_core_tx_write(rtwdev, vif, sta, skb, &qsel);
+ if (ret) {
+ rtw89_warn(rtwdev, "pending tx failed with %d\n", ret);
+ dev_kfree_skb_any(skb);
+ } else {
+ rtw89_core_tx_kick_off(rtwdev, qsel);
+ }
+ }
+}
+
+static void rtw89_core_handle_sta_pending_tx(struct rtw89_dev *rtwdev,
+ struct rtw89_vif *rtwvif)
+{
+ ieee80211_iterate_stations_atomic(rtwdev->hw,
+ rtw89_core_sta_pending_tx_iter,
+ rtwvif);
+}
+
+static int rtw89_core_send_nullfunc(struct rtw89_dev *rtwdev,
+ struct rtw89_vif *rtwvif, bool qos, bool ps)
+{
+ struct ieee80211_vif *vif = rtwvif_to_vif(rtwvif);
+ struct ieee80211_sta *sta;
+ struct ieee80211_hdr *hdr;
+ struct sk_buff *skb;
+ int ret, qsel;
+
+ if (vif->type != NL80211_IFTYPE_STATION || !vif->cfg.assoc)
+ return 0;
+
+ rcu_read_lock();
+ sta = ieee80211_find_sta(vif, vif->bss_conf.bssid);
+ if (!sta) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ skb = ieee80211_nullfunc_get(rtwdev->hw, vif, -1, qos);
+ if (!skb) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ hdr = (struct ieee80211_hdr *)skb->data;
+ if (ps)
+ hdr->frame_control |= cpu_to_le16(IEEE80211_FCTL_PM);
+
+ ret = rtw89_core_tx_write(rtwdev, vif, sta, skb, &qsel);
+ if (ret) {
+ rtw89_warn(rtwdev, "nullfunc transmit failed: %d\n", ret);
+ dev_kfree_skb_any(skb);
+ goto out;
+ }
+
+ rcu_read_unlock();
+
+ return rtw89_core_tx_kick_off_and_wait(rtwdev, skb, qsel,
+ RTW89_ROC_TX_TIMEOUT);
+out:
+ rcu_read_unlock();
+
+ return ret;
+}
+
+void rtw89_roc_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
+{
+ struct ieee80211_hw *hw = rtwdev->hw;
+ struct rtw89_roc *roc = &rtwvif->roc;
+ struct cfg80211_chan_def roc_chan;
+ struct rtw89_vif *tmp;
+ int ret;
+
+ lockdep_assert_held(&rtwdev->mutex);
+
+ ieee80211_queue_delayed_work(hw, &rtwvif->roc.roc_work,
+ msecs_to_jiffies(rtwvif->roc.duration));
+
+ rtw89_leave_ips_by_hwflags(rtwdev);
+ rtw89_leave_lps(rtwdev);
+
+ ret = rtw89_core_send_nullfunc(rtwdev, rtwvif, true, true);
+ if (ret)
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX,
+ "roc send null-1 failed: %d\n", ret);
+
+ rtw89_for_each_rtwvif(rtwdev, tmp)
+ if (tmp->sub_entity_idx == rtwvif->sub_entity_idx)
+ tmp->offchan = true;
+
+ cfg80211_chandef_create(&roc_chan, &roc->chan, NL80211_CHAN_NO_HT);
+ rtw89_config_roc_chandef(rtwdev, rtwvif->sub_entity_idx, &roc_chan);
+ rtw89_set_channel(rtwdev);
+ rtw89_write32_clr(rtwdev,
+ rtw89_mac_reg_by_idx(R_AX_RX_FLTR_OPT, RTW89_MAC_0),
+ B_AX_A_UC_CAM_MATCH | B_AX_A_BC_CAM_MATCH);
+
+ ieee80211_ready_on_channel(hw);
+}
+
+void rtw89_roc_end(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
+{
+ struct ieee80211_hw *hw = rtwdev->hw;
+ struct rtw89_roc *roc = &rtwvif->roc;
+ struct rtw89_vif *tmp;
+ int ret;
+
+ lockdep_assert_held(&rtwdev->mutex);
+
+ ieee80211_remain_on_channel_expired(hw);
+
+ rtw89_leave_ips_by_hwflags(rtwdev);
+ rtw89_leave_lps(rtwdev);
+
+ rtw89_write32_mask(rtwdev,
+ rtw89_mac_reg_by_idx(R_AX_RX_FLTR_OPT, RTW89_MAC_0),
+ B_AX_RX_FLTR_CFG_MASK,
+ rtwdev->hal.rx_fltr);
+
+ roc->state = RTW89_ROC_IDLE;
+ rtw89_config_roc_chandef(rtwdev, rtwvif->sub_entity_idx, NULL);
+ rtw89_set_channel(rtwdev);
+ ret = rtw89_core_send_nullfunc(rtwdev, rtwvif, true, false);
+ if (ret)
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX,
+ "roc send null-0 failed: %d\n", ret);
+
+ rtw89_for_each_rtwvif(rtwdev, tmp)
+ if (tmp->sub_entity_idx == rtwvif->sub_entity_idx)
+ tmp->offchan = false;
+
+ rtw89_core_handle_sta_pending_tx(rtwdev, rtwvif);
+ queue_work(rtwdev->txq_wq, &rtwdev->txq_work);
+
+ if (hw->conf.flags & IEEE80211_CONF_IDLE)
+ ieee80211_queue_delayed_work(hw, &roc->roc_work,
+ RTW89_ROC_IDLE_TIMEOUT);
+}
+
+void rtw89_roc_work(struct work_struct *work)
+{
+ struct rtw89_vif *rtwvif = container_of(work, struct rtw89_vif,
+ roc.roc_work.work);
+ struct rtw89_dev *rtwdev = rtwvif->rtwdev;
+ struct rtw89_roc *roc = &rtwvif->roc;
+
+ mutex_lock(&rtwdev->mutex);
+
+ switch (roc->state) {
+ case RTW89_ROC_IDLE:
+ rtw89_enter_ips_by_hwflags(rtwdev);
+ break;
+ case RTW89_ROC_MGMT:
+ case RTW89_ROC_NORMAL:
+ rtw89_roc_end(rtwdev, rtwvif);
+ break;
+ default:
+ break;
+ }
+
+ mutex_unlock(&rtwdev->mutex);
+}
+
static enum rtw89_tfc_lv rtw89_get_traffic_level(struct rtw89_dev *rtwdev,
u32 throughput, u64 cnt)
{
@@ -2251,8 +2516,10 @@ static bool rtw89_traffic_stats_track(struct rtw89_dev *rtwdev)
bool tfc_changed;
tfc_changed = rtw89_traffic_stats_calc(rtwdev, &rtwdev->stats);
- rtw89_for_each_rtwvif(rtwdev, rtwvif)
+ rtw89_for_each_rtwvif(rtwdev, rtwvif) {
rtw89_traffic_stats_calc(rtwdev, &rtwvif->stats);
+ rtw89_fw_h2c_tp_offload(rtwdev, rtwvif);
+ }
return tfc_changed;
}
@@ -2264,9 +2531,15 @@ static void rtw89_vif_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwv
rtwvif->tdls_peer)
return;
+ if (rtwdev->total_sta_assoc > 1)
+ return;
+
+ if (rtwvif->offchan)
+ return;
+
if (rtwvif->stats.tx_tfc_lv == RTW89_TFC_IDLE &&
rtwvif->stats.rx_tfc_lv == RTW89_TFC_IDLE)
- rtw89_enter_lps(rtwdev, rtwvif);
+ rtw89_enter_lps(rtwdev, rtwvif, true);
}
static void rtw89_enter_lps_track(struct rtw89_dev *rtwdev)
@@ -2493,6 +2766,7 @@ int rtw89_core_sta_add(struct rtw89_dev *rtwdev,
rtwsta->rtwvif = rtwvif;
rtwsta->prev_rssi = 0;
INIT_LIST_HEAD(&rtwsta->ba_cam_list);
+ skb_queue_head_init(&rtwsta->roc_queue);
for (i = 0; i < ARRAY_SIZE(sta->txq); i++)
rtw89_core_txq_init(rtwdev, sta->txq[i]);
@@ -2539,6 +2813,9 @@ int rtw89_core_sta_disassoc(struct rtw89_dev *rtwdev,
struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv;
+ if (vif->type == NL80211_IFTYPE_STATION)
+ rtw89_fw_h2c_set_bcn_fltr_cfg(rtwdev, vif, false);
+
rtwdev->total_sta_assoc--;
if (sta->tdls)
rtwvif->tdls_peer--;
@@ -2559,6 +2836,8 @@ int rtw89_core_sta_disconnect(struct rtw89_dev *rtwdev,
rtw89_mac_bf_disassoc(rtwdev, vif, sta);
rtw89_core_free_sta_pending_ba(rtwdev, sta);
rtw89_core_free_sta_pending_forbid_ba(rtwdev, sta);
+ rtw89_core_free_sta_pending_roc_tx(rtwdev, sta);
+
if (vif->type == NL80211_IFTYPE_AP || sta->tdls)
rtw89_cam_deinit_addr_cam(rtwdev, &rtwsta->addr_cam);
if (sta->tdls)
@@ -3180,7 +3459,6 @@ void rtw89_core_stop(struct rtw89_dev *rtwdev)
int rtw89_core_init(struct rtw89_dev *rtwdev)
{
struct rtw89_btc *btc = &rtwdev->btc;
- int ret;
u8 band;
INIT_LIST_HEAD(&rtwdev->ba_list);
@@ -3214,6 +3492,8 @@ int rtw89_core_init(struct rtw89_dev *rtwdev)
INIT_WORK(&rtwdev->c2h_work, rtw89_fw_c2h_work);
INIT_WORK(&rtwdev->ips_work, rtw89_ips_work);
+ INIT_WORK(&rtwdev->load_firmware_work, rtw89_load_firmware_work);
+
skb_queue_head_init(&rtwdev->c2h_queue);
rtw89_core_ppdu_sts_init(rtwdev);
rtw89_traffic_stats_init(rtwdev, &rtwdev->stats);
@@ -3225,12 +3505,10 @@ int rtw89_core_init(struct rtw89_dev *rtwdev)
INIT_WORK(&btc->dhcp_notify_work, rtw89_btc_ntfy_dhcp_packet_work);
INIT_WORK(&btc->icmp_notify_work, rtw89_btc_ntfy_icmp_packet_work);
- ret = rtw89_load_firmware(rtwdev);
- if (ret) {
- rtw89_warn(rtwdev, "no firmware loaded\n");
- destroy_workqueue(rtwdev->txq_wq);
- return ret;
- }
+ init_completion(&rtwdev->fw.req.completion);
+
+ schedule_work(&rtwdev->load_firmware_work);
+
rtw89_ser_init(rtwdev);
rtw89_entity_init(rtwdev);
@@ -3257,8 +3535,8 @@ void rtw89_core_scan_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
rtwdev->scanning = true;
rtw89_leave_lps(rtwdev);
- if (hw_scan && (rtwdev->hw->conf.flags & IEEE80211_CONF_IDLE))
- rtw89_leave_ips(rtwdev);
+ if (hw_scan)
+ rtw89_leave_ips_by_hwflags(rtwdev);
ether_addr_copy(rtwvif->mac_addr, mac_addr);
rtw89_btc_ntfy_scan_start(rtwdev, RTW89_PHY_0, chan->band_type);
@@ -3293,6 +3571,8 @@ void rtw89_core_scan_complete(struct rtw89_dev *rtwdev,
static void rtw89_read_chip_ver(struct rtw89_dev *rtwdev)
{
const struct rtw89_chip_info *chip = rtwdev->chip;
+ int ret;
+ u8 val;
u8 cv;
cv = rtw89_read32_mask(rtwdev, R_AX_SYS_CFG1, B_AX_CHIP_VER_MASK);
@@ -3304,6 +3584,14 @@ static void rtw89_read_chip_ver(struct rtw89_dev *rtwdev)
}
rtwdev->hal.cv = cv;
+
+ if (chip->chip_id == RTL8852B || chip->chip_id == RTL8851B) {
+ ret = rtw89_mac_read_xtal_si(rtwdev, XTAL_SI_CV, &val);
+ if (!ret)
+ return;
+
+ rtwdev->hal.acv = u8_get_bits(val, XTAL_SI_ACV_MASK);
+ }
}
static void rtw89_core_setup_phycap(struct rtw89_dev *rtwdev)
@@ -3315,6 +3603,28 @@ static void rtw89_core_setup_phycap(struct rtw89_dev *rtwdev)
rtwdev->chip->chip_id == RTL8852A && rtwdev->hal.cv <= CHIP_CBV;
}
+static void rtw89_core_setup_rfe_parms(struct rtw89_dev *rtwdev)
+{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
+ const struct rtw89_rfe_parms_conf *conf = chip->rfe_parms_conf;
+ struct rtw89_efuse *efuse = &rtwdev->efuse;
+ u8 rfe_type = efuse->rfe_type;
+
+ if (!conf)
+ goto out;
+
+ while (conf->rfe_parms) {
+ if (rfe_type == conf->rfe_type) {
+ rtwdev->rfe_parms = conf->rfe_parms;
+ return;
+ }
+ conf++;
+ }
+
+out:
+ rtwdev->rfe_parms = chip->dflt_parms;
+}
+
static int rtw89_chip_efuse_info_setup(struct rtw89_dev *rtwdev)
{
int ret;
@@ -3336,6 +3646,7 @@ static int rtw89_chip_efuse_info_setup(struct rtw89_dev *rtwdev)
return ret;
rtw89_core_setup_phycap(rtwdev);
+ rtw89_core_setup_rfe_parms(rtwdev);
rtw89_mac_pwr_off(rtwdev);
@@ -3415,6 +3726,8 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev)
ieee80211_hw_set(hw, SINGLE_SCAN_ON_ALL_BANDS);
ieee80211_hw_set(hw, SUPPORTS_MULTI_BSSID);
ieee80211_hw_set(hw, WANT_MONITOR_VIF);
+ if (RTW89_CHK_FW_FEATURE(BEACON_FILTER, &rtwdev->fw))
+ ieee80211_hw_set(hw, CONNECTION_MONITOR);
hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_AP) |
@@ -3440,6 +3753,7 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev)
hw->wiphy->tid_config_support.peer |= BIT(NL80211_TID_CONFIG_ATTR_AMPDU_CTRL);
hw->wiphy->tid_config_support.vif |= BIT(NL80211_TID_CONFIG_ATTR_AMSDU_CTRL);
hw->wiphy->tid_config_support.peer |= BIT(NL80211_TID_CONFIG_ATTR_AMSDU_CTRL);
+ hw->wiphy->max_remain_on_channel_duration = 1000;
wiphy_ext_feature_set(hw->wiphy, NL80211_EXT_FEATURE_CAN_REPLACE_PTK0);
@@ -3508,22 +3822,24 @@ struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device,
u32 bus_data_size,
const struct rtw89_chip_info *chip)
{
+ struct rtw89_fw_info early_fw = {};
const struct firmware *firmware;
struct ieee80211_hw *hw;
struct rtw89_dev *rtwdev;
struct ieee80211_ops *ops;
u32 driver_data_size;
- u32 early_feat_map = 0;
+ int fw_format = -1;
bool no_chanctx;
- firmware = rtw89_early_fw_feature_recognize(device, chip, &early_feat_map);
+ firmware = rtw89_early_fw_feature_recognize(device, chip, &early_fw, &fw_format);
ops = kmemdup(&rtw89_ops, sizeof(rtw89_ops), GFP_KERNEL);
if (!ops)
goto err;
no_chanctx = chip->support_chanctx_num == 0 ||
- !(early_feat_map & BIT(RTW89_FW_FEATURE_SCAN_OFFLOAD));
+ !RTW89_CHK_FW_FEATURE(SCAN_OFFLOAD, &early_fw) ||
+ !RTW89_CHK_FW_FEATURE(BEACON_FILTER, &early_fw);
if (no_chanctx) {
ops->add_chanctx = NULL;
@@ -3531,6 +3847,8 @@ struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device,
ops->change_chanctx = NULL;
ops->assign_vif_chanctx = NULL;
ops->unassign_vif_chanctx = NULL;
+ ops->remain_on_channel = NULL;
+ ops->cancel_remain_on_channel = NULL;
}
driver_data_size = sizeof(struct rtw89_dev) + bus_data_size;
@@ -3538,12 +3856,16 @@ struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device,
if (!hw)
goto err;
+ hw->wiphy->iface_combinations = rtw89_iface_combs;
+ hw->wiphy->n_iface_combinations = ARRAY_SIZE(rtw89_iface_combs);
+
rtwdev = hw->priv;
rtwdev->hw = hw;
rtwdev->dev = device;
rtwdev->ops = ops;
rtwdev->chip = chip;
- rtwdev->fw.firmware = firmware;
+ rtwdev->fw.req.firmware = firmware;
+ rtwdev->fw.fw_format = fw_format;
rtw89_debug(rtwdev, RTW89_DBG_FW, "probe driver %s chanctx\n",
no_chanctx ? "without" : "with");
@@ -3560,7 +3882,7 @@ EXPORT_SYMBOL(rtw89_alloc_ieee80211_hw);
void rtw89_free_ieee80211_hw(struct rtw89_dev *rtwdev)
{
kfree(rtwdev->ops);
- release_firmware(rtwdev->fw.firmware);
+ release_firmware(rtwdev->fw.req.firmware);
ieee80211_free_hw(rtwdev->hw);
}
EXPORT_SYMBOL(rtw89_free_ieee80211_hw);
diff --git a/drivers/net/wireless/realtek/rtw89/core.h b/drivers/net/wireless/realtek/rtw89/core.h
index 40fb18b..6df386a 100644
--- a/drivers/net/wireless/realtek/rtw89/core.h
+++ b/drivers/net/wireless/realtek/rtw89/core.h
@@ -108,6 +108,7 @@ enum rtw89_core_chip_id {
RTL8852A,
RTL8852B,
RTL8852C,
+ RTL8851B,
};
enum rtw89_cv {
@@ -569,6 +570,7 @@ enum rtw89_sub_entity_idx {
RTW89_SUB_ENTITY_0 = 0,
NUM_OF_RTW89_SUB_ENTITY,
+ RTW89_SUB_ENTITY_IDLE = NUM_OF_RTW89_SUB_ENTITY,
};
enum rtw89_rf_path {
@@ -957,6 +959,8 @@ struct rtw89_btc_ant_info {
u8 single_pos: 1;/* Single antenna at S0 or S1 */
u8 diversity: 1;
+ u8 btg_pos: 2;
+ u8 stream_cnt: 4;
};
enum rtw89_tfc_dir {
@@ -1413,8 +1417,9 @@ struct rtw89_btc_module {
u8 bt_solo: 1;
u8 bt_pos: 1;
u8 switch_type: 1;
+ u8 wa_type: 3;
- u8 rsvd;
+ u8 kt_ver_adie;
};
#define RTW89_BTC_DM_MAXSTEP 30
@@ -2597,6 +2602,7 @@ struct rtw89_sta {
struct rtw89_addr_cam_entry addr_cam; /* AP mode or TDLS peer only */
struct rtw89_bssid_cam_entry bssid_cam; /* TDLS peer only */
struct list_head ba_cam_list;
+ struct sk_buff_head roc_queue;
bool use_cfg_mask;
struct cfg80211_bitrate_mask mask;
@@ -2623,11 +2629,39 @@ struct rtw89_phy_rate_pattern {
bool enable;
};
+struct rtw89_tx_wait_info {
+ struct rcu_head rcu_head;
+ struct completion completion;
+ bool tx_done;
+};
+
+struct rtw89_tx_skb_data {
+ struct rtw89_tx_wait_info __rcu *wait;
+ u8 hci_priv[];
+};
+
+#define RTW89_ROC_IDLE_TIMEOUT 500
+#define RTW89_ROC_TX_TIMEOUT 30
+enum rtw89_roc_state {
+ RTW89_ROC_IDLE,
+ RTW89_ROC_NORMAL,
+ RTW89_ROC_MGMT,
+};
+
+struct rtw89_roc {
+ struct ieee80211_channel chan;
+ struct delayed_work roc_work;
+ enum ieee80211_roc_type type;
+ enum rtw89_roc_state state;
+ int duration;
+};
+
#define RTW89_P2P_MAX_NOA_NUM 2
struct rtw89_vif {
struct list_head list;
struct rtw89_dev *rtwdev;
+ struct rtw89_roc roc;
enum rtw89_sub_entity_idx sub_entity_idx;
u8 mac_id;
@@ -2643,6 +2677,7 @@ struct rtw89_vif {
u8 bcn_hit_cond;
u8 hit_rule;
u8 last_noa_nr;
+ bool offchan;
bool trigger;
bool lsig_txop;
u8 tgt_ind;
@@ -2959,6 +2994,41 @@ struct rtw89_txpwr_table {
const struct rtw89_txpwr_table *tbl);
};
+struct rtw89_txpwr_rule_2ghz {
+ const s8 (*lmt)[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
+ const s8 (*lmt_ru)[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
+};
+
+struct rtw89_txpwr_rule_5ghz {
+ const s8 (*lmt)[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
+ const s8 (*lmt_ru)[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
+};
+
+struct rtw89_txpwr_rule_6ghz {
+ const s8 (*lmt)[RTW89_6G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
+ const s8 (*lmt_ru)[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
+};
+
+struct rtw89_rfe_parms {
+ struct rtw89_txpwr_rule_2ghz rule_2ghz;
+ struct rtw89_txpwr_rule_5ghz rule_5ghz;
+ struct rtw89_txpwr_rule_6ghz rule_6ghz;
+};
+
+struct rtw89_rfe_parms_conf {
+ const struct rtw89_rfe_parms *rfe_parms;
+ u8 rfe_type;
+};
+
struct rtw89_page_regs {
u32 hci_fc_ctrl;
u32 ch_page_ctrl;
@@ -3049,7 +3119,8 @@ struct rtw89_phy_ul_tb_info {
struct rtw89_chip_info {
enum rtw89_core_chip_id chip_id;
const struct rtw89_chip_ops *ops;
- const char *fw_name;
+ const char *fw_basename;
+ u8 fw_format_max;
bool try_ce_fw;
u32 fifo_size;
u32 dle_scc_rsvd_size;
@@ -3095,21 +3166,10 @@ struct rtw89_chip_info {
const struct rtw89_phy_dig_gain_table *dig_table;
const struct rtw89_dig_regs *dig_regs;
const struct rtw89_phy_tssi_dbw_table *tssi_dbw_table;
- const s8 (*txpwr_lmt_2g)[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
- const s8 (*txpwr_lmt_5g)[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
- const s8 (*txpwr_lmt_6g)[RTW89_6G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
- const s8 (*txpwr_lmt_ru_2g)[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
- const s8 (*txpwr_lmt_ru_5g)[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
- const s8 (*txpwr_lmt_ru_6g)[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
+
+ /* NULL if no rfe-specific, or a null-terminated array by rfe_parms */
+ const struct rtw89_rfe_parms_conf *rfe_parms_conf;
+ const struct rtw89_rfe_parms *dflt_parms;
u8 txpwr_factor_rf;
u8 txpwr_factor_mac;
@@ -3146,6 +3206,7 @@ struct rtw89_chip_info {
struct rtw89_reg_def c2h_counter_reg;
const struct rtw89_page_regs *page_regs;
bool cfo_src_fd;
+ bool cfo_hw_comp;
const struct rtw89_reg_def *dcfo_comp;
u8 dcfo_comp_sft;
const struct rtw89_imr_info *imr_info;
@@ -3231,6 +3292,7 @@ enum rtw89_fw_feature {
RTW89_FW_FEATURE_NO_PACKET_DROP,
RTW89_FW_FEATURE_NO_DEEP_PS,
RTW89_FW_FEATURE_NO_LPS_PG,
+ RTW89_FW_FEATURE_BEACON_FILTER,
};
struct rtw89_fw_suit {
@@ -3265,10 +3327,14 @@ struct rtw89_fw_suit {
GET_FW_HDR_SUBVERSION(fw_hdr), \
GET_FW_HDR_SUBINDEX(fw_hdr))
-struct rtw89_fw_info {
+struct rtw89_fw_req_info {
const struct firmware *firmware;
- struct rtw89_dev *rtwdev;
struct completion completion;
+};
+
+struct rtw89_fw_info {
+ struct rtw89_fw_req_info req;
+ int fw_format;
u8 h2c_seq;
u8 rec_seq;
u8 h2c_counter;
@@ -3350,6 +3416,7 @@ struct rtw89_sub_entity {
struct rtw89_hal {
u32 rx_fltr;
u8 cv;
+ u8 acv;
u32 sw_amsdu_max_size;
u32 antenna_tx;
u32 antenna_rx;
@@ -3358,9 +3425,11 @@ struct rtw89_hal {
bool tx_path_diversity;
bool support_cckpd;
bool support_igi;
+ atomic_t roc_entity_idx;
DECLARE_BITMAP(entity_map, NUM_OF_RTW89_SUB_ENTITY);
struct rtw89_sub_entity sub[NUM_OF_RTW89_SUB_ENTITY];
+ struct cfg80211_chan_def roc_chandef;
bool entity_active;
enum rtw89_entity_mode entity_mode;
@@ -3621,6 +3690,8 @@ struct rtw89_cfo_tracking_info {
s32 cfo_avg_pre;
s32 cfo_avg[CFO_TRACK_MAX_USER];
s32 pre_cfo_avg[CFO_TRACK_MAX_USER];
+ s32 dcfo_avg;
+ s32 dcfo_avg_pre;
u32 packet_count;
u32 packet_count_pre;
s32 residual_cfo_acc;
@@ -3865,10 +3936,7 @@ struct rtw89_early_h2c {
struct rtw89_hw_scan_info {
struct ieee80211_vif *scanning_vif;
struct list_head pkt_list[NUM_NL80211_BANDS];
- u8 op_pri_ch;
- u8 op_chan;
- u8 op_bw;
- u8 op_band;
+ struct rtw89_chan op_chan;
u32 last_chan_idx;
};
@@ -3953,6 +4021,7 @@ struct rtw89_dev {
struct rtw89_hw_scan_info scan_info;
const struct rtw89_chip_info *chip;
const struct rtw89_pci_info *pci_info;
+ const struct rtw89_rfe_parms *rfe_parms;
struct rtw89_hal hal;
struct rtw89_mcc_info mcc;
struct rtw89_mac_info mac;
@@ -3984,6 +4053,7 @@ struct rtw89_dev {
struct sk_buff_head c2h_queue;
struct work_struct c2h_work;
struct work_struct ips_work;
+ struct work_struct load_firmware_work;
struct list_head early_h2c_list;
@@ -4023,6 +4093,7 @@ struct rtw89_dev {
struct delayed_work coex_rfk_chk_work;
struct delayed_work cfo_track_work;
struct delayed_work forbid_ba_work;
+ struct delayed_work roc_work;
struct rtw89_ppdu_sts_info ppdu_sts;
u8 total_sta_assoc;
bool scanning;
@@ -4178,6 +4249,14 @@ static inline void rtw89_hci_clear(struct rtw89_dev *rtwdev, struct pci_dev *pde
rtwdev->hci.ops->clear(rtwdev, pdev);
}
+static inline
+struct rtw89_tx_skb_data *RTW89_TX_SKB_CB(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+
+ return (struct rtw89_tx_skb_data *)info->status.status_driver_data;
+}
+
static inline u8 rtw89_read8(struct rtw89_dev *rtwdev, u32 addr)
{
return rtwdev->hci.ops->read8(rtwdev, addr);
@@ -4530,6 +4609,10 @@ const struct cfg80211_chan_def *rtw89_chandef_get(struct rtw89_dev *rtwdev,
enum rtw89_sub_entity_idx idx)
{
struct rtw89_hal *hal = &rtwdev->hal;
+ enum rtw89_sub_entity_idx roc_idx = atomic_read(&hal->roc_entity_idx);
+
+ if (roc_idx == idx)
+ return &hal->roc_chandef;
return &hal->sub[idx].chandef;
}
@@ -4821,11 +4904,32 @@ static inline struct sk_buff *rtw89_alloc_skb_for_rx(struct rtw89_dev *rtwdev,
return dev_alloc_skb(length);
}
+static inline void rtw89_core_tx_wait_complete(struct rtw89_dev *rtwdev,
+ struct rtw89_tx_skb_data *skb_data,
+ bool tx_done)
+{
+ struct rtw89_tx_wait_info *wait;
+
+ rcu_read_lock();
+
+ wait = rcu_dereference(skb_data->wait);
+ if (!wait)
+ goto out;
+
+ wait->tx_done = tx_done;
+ complete(&wait->completion);
+
+out:
+ rcu_read_unlock();
+}
+
int rtw89_core_tx_write(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, struct sk_buff *skb, int *qsel);
int rtw89_h2c_tx(struct rtw89_dev *rtwdev,
struct sk_buff *skb, bool fwdl);
void rtw89_core_tx_kick_off(struct rtw89_dev *rtwdev, u8 qsel);
+int rtw89_core_tx_kick_off_and_wait(struct rtw89_dev *rtwdev, struct sk_buff *skb,
+ int qsel, unsigned int timeout);
void rtw89_core_fill_txdesc(struct rtw89_dev *rtwdev,
struct rtw89_tx_desc_info *desc_info,
void *txdesc);
@@ -4874,6 +4978,8 @@ void rtw89_free_ieee80211_hw(struct rtw89_dev *rtwdev);
void rtw89_core_set_chip_txpwr(struct rtw89_dev *rtwdev);
void rtw89_get_default_chandef(struct cfg80211_chan_def *chandef);
void rtw89_set_channel(struct rtw89_dev *rtwdev);
+void rtw89_get_channel(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ struct rtw89_chan *chan);
u8 rtw89_core_acquire_bit_map(unsigned long *addr, unsigned long size);
void rtw89_core_release_bit_map(unsigned long *addr, u8 bit);
void rtw89_core_release_all_bits_map(unsigned long *addr, unsigned int nbits);
@@ -4895,6 +5001,9 @@ void rtw89_complete_cond(struct rtw89_wait_info *wait, unsigned int cond,
int rtw89_core_start(struct rtw89_dev *rtwdev);
void rtw89_core_stop(struct rtw89_dev *rtwdev);
void rtw89_core_update_beacon_work(struct work_struct *work);
+void rtw89_roc_work(struct work_struct *work);
+void rtw89_roc_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
+void rtw89_roc_end(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
void rtw89_core_scan_start(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
const u8 *mac_addr, bool hw_scan);
void rtw89_core_scan_complete(struct rtw89_dev *rtwdev,
diff --git a/drivers/net/wireless/realtek/rtw89/debug.c b/drivers/net/wireless/realtek/rtw89/debug.c
index 0e0e148..1e5b7a9 100644
--- a/drivers/net/wireless/realtek/rtw89/debug.c
+++ b/drivers/net/wireless/realtek/rtw89/debug.c
@@ -3069,18 +3069,13 @@ static int rtw89_dbg_trigger_ctrl_error(struct rtw89_dev *rtwdev)
{
struct rtw89_cpuio_ctrl ctrl_para = {0};
u16 pkt_id;
+ int ret;
rtw89_leave_ps_mode(rtwdev);
- pkt_id = rtw89_mac_dle_buf_req(rtwdev, 0x20, true);
- switch (pkt_id) {
- case 0xffff:
- return -ETIMEDOUT;
- case 0xfff:
- return -ENOMEM;
- default:
- break;
- }
+ ret = rtw89_mac_dle_buf_req(rtwdev, 0x20, true, &pkt_id);
+ if (ret)
+ return ret;
/* intentionally, enqueue two pkt, but has only one pkt id */
ctrl_para.cmd_type = CPUIO_OP_CMD_ENQ_TO_HEAD;
diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c
index 5fa6863d..b9b675b 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.c
+++ b/drivers/net/wireless/realtek/rtw89/fw.c
@@ -155,8 +155,9 @@ int rtw89_mfw_recognize(struct rtw89_dev *rtwdev, enum rtw89_fw_type type,
struct rtw89_fw_suit *fw_suit, bool nowarn)
{
struct rtw89_fw_info *fw_info = &rtwdev->fw;
- const u8 *mfw = fw_info->firmware->data;
- u32 mfw_len = fw_info->firmware->size;
+ const struct firmware *firmware = fw_info->req.firmware;
+ const u8 *mfw = firmware->data;
+ u32 mfw_len = firmware->size;
const struct rtw89_mfw_hdr *mfw_hdr = (const struct rtw89_mfw_hdr *)mfw;
const struct rtw89_mfw_info *mfw_info;
int i;
@@ -266,40 +267,51 @@ static const struct __fw_feat_cfg fw_feat_tbl[] = {
__CFG_FW_FEAT(RTL8852C, ge, 0, 27, 34, 0, TX_WAKE),
__CFG_FW_FEAT(RTL8852C, ge, 0, 27, 36, 0, SCAN_OFFLOAD),
__CFG_FW_FEAT(RTL8852C, ge, 0, 27, 40, 0, CRASH_TRIGGER),
+ __CFG_FW_FEAT(RTL8852C, ge, 0, 27, 56, 10, BEACON_FILTER),
};
+static void rtw89_fw_iterate_feature_cfg(struct rtw89_fw_info *fw,
+ const struct rtw89_chip_info *chip,
+ u32 ver_code)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(fw_feat_tbl); i++) {
+ const struct __fw_feat_cfg *ent = &fw_feat_tbl[i];
+
+ if (chip->chip_id != ent->chip_id)
+ continue;
+
+ if (ent->cond(ver_code, ent->ver_code))
+ RTW89_SET_FW_FEATURE(ent->feature, fw);
+ }
+}
+
static void rtw89_fw_recognize_features(struct rtw89_dev *rtwdev)
{
const struct rtw89_chip_info *chip = rtwdev->chip;
- const struct __fw_feat_cfg *ent;
const struct rtw89_fw_suit *fw_suit;
u32 suit_ver_code;
- int i;
fw_suit = rtw89_fw_suit_get(rtwdev, RTW89_FW_NORMAL);
suit_ver_code = RTW89_FW_SUIT_VER_CODE(fw_suit);
- for (i = 0; i < ARRAY_SIZE(fw_feat_tbl); i++) {
- ent = &fw_feat_tbl[i];
- if (chip->chip_id != ent->chip_id)
- continue;
-
- if (ent->cond(suit_ver_code, ent->ver_code))
- RTW89_SET_FW_FEATURE(ent->feature, &rtwdev->fw);
- }
+ rtw89_fw_iterate_feature_cfg(&rtwdev->fw, chip, suit_ver_code);
}
const struct firmware *
rtw89_early_fw_feature_recognize(struct device *device,
const struct rtw89_chip_info *chip,
- u32 *early_feat_map)
+ struct rtw89_fw_info *early_fw,
+ int *used_fw_format)
{
union rtw89_compat_fw_hdr buf = {};
const struct firmware *firmware;
bool full_req = false;
+ char fw_name[64];
+ int fw_format;
u32 ver_code;
int ret;
- int i;
/* If SECURITY_LOADPIN_ENFORCE is enabled, reading partial files will
* be denied (-EPERM). Then, we don't get right firmware things as
@@ -308,12 +320,22 @@ rtw89_early_fw_feature_recognize(struct device *device,
if (IS_ENABLED(CONFIG_SECURITY_LOADPIN_ENFORCE))
full_req = true;
- if (full_req)
- ret = request_firmware(&firmware, chip->fw_name, device);
- else
- ret = request_partial_firmware_into_buf(&firmware, chip->fw_name,
- device, &buf, sizeof(buf),
- 0);
+ for (fw_format = chip->fw_format_max; fw_format >= 0; fw_format--) {
+ rtw89_fw_get_filename(fw_name, sizeof(fw_name),
+ chip->fw_basename, fw_format);
+
+ if (full_req)
+ ret = request_firmware(&firmware, fw_name, device);
+ else
+ ret = request_partial_firmware_into_buf(&firmware, fw_name,
+ device, &buf, sizeof(buf),
+ 0);
+ if (!ret) {
+ dev_info(device, "loaded firmware %s\n", fw_name);
+ *used_fw_format = fw_format;
+ break;
+ }
+ }
if (ret) {
dev_err(device, "failed to early request firmware: %d\n", ret);
@@ -328,15 +350,7 @@ rtw89_early_fw_feature_recognize(struct device *device,
if (!ver_code)
goto out;
- for (i = 0; i < ARRAY_SIZE(fw_feat_tbl); i++) {
- const struct __fw_feat_cfg *ent = &fw_feat_tbl[i];
-
- if (chip->chip_id != ent->chip_id)
- continue;
-
- if (ent->cond(ver_code, ent->ver_code))
- *early_feat_map |= BIT(ent->feature);
- }
+ rtw89_fw_iterate_feature_cfg(early_fw, chip, ver_code);
out:
if (full_req)
@@ -631,67 +645,62 @@ int rtw89_wait_firmware_completion(struct rtw89_dev *rtwdev)
{
struct rtw89_fw_info *fw = &rtwdev->fw;
- wait_for_completion(&fw->completion);
- if (!fw->firmware)
+ wait_for_completion(&fw->req.completion);
+ if (!fw->req.firmware)
return -EINVAL;
return 0;
}
-static void rtw89_load_firmware_cb(const struct firmware *firmware, void *context)
+static int rtw89_load_firmware_req(struct rtw89_dev *rtwdev,
+ struct rtw89_fw_req_info *req,
+ const char *fw_name, bool nowarn)
{
- struct rtw89_fw_info *fw = context;
- struct rtw89_dev *rtwdev = fw->rtwdev;
-
- if (!firmware || !firmware->data) {
- rtw89_err(rtwdev, "failed to request firmware\n");
- complete_all(&fw->completion);
- return;
- }
-
- fw->firmware = firmware;
- complete_all(&fw->completion);
-}
-
-int rtw89_load_firmware(struct rtw89_dev *rtwdev)
-{
- struct rtw89_fw_info *fw = &rtwdev->fw;
- const char *fw_name = rtwdev->chip->fw_name;
int ret;
- fw->rtwdev = rtwdev;
- init_completion(&fw->completion);
-
- if (fw->firmware) {
+ if (req->firmware) {
rtw89_debug(rtwdev, RTW89_DBG_FW,
"full firmware has been early requested\n");
- complete_all(&fw->completion);
+ complete_all(&req->completion);
return 0;
}
- ret = request_firmware_nowait(THIS_MODULE, true, fw_name, rtwdev->dev,
- GFP_KERNEL, fw, rtw89_load_firmware_cb);
- if (ret) {
- rtw89_err(rtwdev, "failed to async firmware request\n");
- return ret;
- }
+ if (nowarn)
+ ret = firmware_request_nowarn(&req->firmware, fw_name, rtwdev->dev);
+ else
+ ret = request_firmware(&req->firmware, fw_name, rtwdev->dev);
- return 0;
+ complete_all(&req->completion);
+
+ return ret;
+}
+
+void rtw89_load_firmware_work(struct work_struct *work)
+{
+ struct rtw89_dev *rtwdev =
+ container_of(work, struct rtw89_dev, load_firmware_work);
+ const struct rtw89_chip_info *chip = rtwdev->chip;
+ char fw_name[64];
+
+ rtw89_fw_get_filename(fw_name, sizeof(fw_name),
+ chip->fw_basename, rtwdev->fw.fw_format);
+
+ rtw89_load_firmware_req(rtwdev, &rtwdev->fw.req, fw_name, false);
}
void rtw89_unload_firmware(struct rtw89_dev *rtwdev)
{
struct rtw89_fw_info *fw = &rtwdev->fw;
- rtw89_wait_firmware_completion(rtwdev);
+ cancel_work_sync(&rtwdev->load_firmware_work);
- if (fw->firmware) {
- release_firmware(fw->firmware);
+ if (fw->req.firmware) {
+ release_firmware(fw->req.firmware);
/* assign NULL back in case rtw89_free_ieee80211_hw()
* try to release the same one again.
*/
- fw->firmware = NULL;
+ fw->req.firmware = NULL;
}
}
@@ -1152,9 +1161,18 @@ int rtw89_fw_h2c_p2p_act(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
static void __rtw89_fw_h2c_set_tx_path(struct rtw89_dev *rtwdev,
struct sk_buff *skb)
{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
struct rtw89_hal *hal = &rtwdev->hal;
- u8 ntx_path = hal->antenna_tx ? hal->antenna_tx : RF_B;
- u8 map_b = hal->antenna_tx == RF_AB ? 1 : 0;
+ u8 ntx_path;
+ u8 map_b;
+
+ if (chip->rf_path_num == 1) {
+ ntx_path = RF_A;
+ map_b = 0;
+ } else {
+ ntx_path = hal->antenna_tx ? hal->antenna_tx : RF_B;
+ map_b = hal->antenna_tx == RF_AB ? 1 : 0;
+ }
SET_CMC_TBL_NTX_PATH_EN(skb->data, ntx_path);
SET_CMC_TBL_PATH_MAP_A(skb->data, 0);
@@ -1737,6 +1755,147 @@ int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev)
return ret;
}
+int rtw89_fw_h2c_set_bcn_fltr_cfg(struct rtw89_dev *rtwdev,
+ struct ieee80211_vif *vif,
+ bool connect)
+{
+ struct rtw89_vif *rtwvif = vif_to_rtwvif_safe(vif);
+ struct ieee80211_bss_conf *bss_conf = vif ? &vif->bss_conf : NULL;
+ struct rtw89_h2c_bcnfltr *h2c;
+ u32 len = sizeof(*h2c);
+ struct sk_buff *skb;
+ int ret;
+
+ if (!RTW89_CHK_FW_FEATURE(BEACON_FILTER, &rtwdev->fw))
+ return -EINVAL;
+
+ if (!rtwvif || !bss_conf || rtwvif->net_type != RTW89_NET_TYPE_INFRA)
+ return -EINVAL;
+
+ skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len);
+ if (!skb) {
+ rtw89_err(rtwdev, "failed to alloc skb for h2c bcn filter\n");
+ return -ENOMEM;
+ }
+
+ skb_put(skb, len);
+ h2c = (struct rtw89_h2c_bcnfltr *)skb->data;
+
+ h2c->w0 = le32_encode_bits(connect, RTW89_H2C_BCNFLTR_W0_MON_RSSI) |
+ le32_encode_bits(connect, RTW89_H2C_BCNFLTR_W0_MON_BCN) |
+ le32_encode_bits(connect, RTW89_H2C_BCNFLTR_W0_MON_EN) |
+ le32_encode_bits(RTW89_BCN_FLTR_OFFLOAD_MODE_DEFAULT,
+ RTW89_H2C_BCNFLTR_W0_MODE) |
+ le32_encode_bits(RTW89_BCN_LOSS_CNT, RTW89_H2C_BCNFLTR_W0_BCN_LOSS_CNT) |
+ le32_encode_bits(bss_conf->cqm_rssi_hyst, RTW89_H2C_BCNFLTR_W0_RSSI_HYST) |
+ le32_encode_bits(bss_conf->cqm_rssi_thold + MAX_RSSI,
+ RTW89_H2C_BCNFLTR_W0_RSSI_THRESHOLD) |
+ le32_encode_bits(rtwvif->mac_id, RTW89_H2C_BCNFLTR_W0_MAC_ID);
+
+ rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
+ H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD,
+ H2C_FUNC_CFG_BCNFLTR, 0, 1, len);
+
+ ret = rtw89_h2c_tx(rtwdev, skb, false);
+ if (ret) {
+ rtw89_err(rtwdev, "failed to send h2c\n");
+ goto fail;
+ }
+
+ return 0;
+fail:
+ dev_kfree_skb_any(skb);
+
+ return ret;
+}
+
+int rtw89_fw_h2c_rssi_offload(struct rtw89_dev *rtwdev,
+ struct rtw89_rx_phy_ppdu *phy_ppdu)
+{
+ struct rtw89_h2c_ofld_rssi *h2c;
+ u32 len = sizeof(*h2c);
+ struct sk_buff *skb;
+ s8 rssi;
+ int ret;
+
+ if (!RTW89_CHK_FW_FEATURE(BEACON_FILTER, &rtwdev->fw))
+ return -EINVAL;
+
+ if (!phy_ppdu)
+ return -EINVAL;
+
+ skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len);
+ if (!skb) {
+ rtw89_err(rtwdev, "failed to alloc skb for h2c rssi\n");
+ return -ENOMEM;
+ }
+
+ rssi = phy_ppdu->rssi_avg >> RSSI_FACTOR;
+ skb_put(skb, len);
+ h2c = (struct rtw89_h2c_ofld_rssi *)skb->data;
+
+ h2c->w0 = le32_encode_bits(phy_ppdu->mac_id, RTW89_H2C_OFLD_RSSI_W0_MACID) |
+ le32_encode_bits(1, RTW89_H2C_OFLD_RSSI_W0_NUM);
+ h2c->w1 = le32_encode_bits(rssi, RTW89_H2C_OFLD_RSSI_W1_VAL);
+
+ rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
+ H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD,
+ H2C_FUNC_OFLD_RSSI, 0, 1, len);
+
+ ret = rtw89_h2c_tx(rtwdev, skb, false);
+ if (ret) {
+ rtw89_err(rtwdev, "failed to send h2c\n");
+ goto fail;
+ }
+
+ return 0;
+fail:
+ dev_kfree_skb_any(skb);
+
+ return ret;
+}
+
+int rtw89_fw_h2c_tp_offload(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
+{
+ struct rtw89_traffic_stats *stats = &rtwvif->stats;
+ struct rtw89_h2c_ofld *h2c;
+ u32 len = sizeof(*h2c);
+ struct sk_buff *skb;
+ int ret;
+
+ if (rtwvif->net_type != RTW89_NET_TYPE_INFRA)
+ return -EINVAL;
+
+ skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len);
+ if (!skb) {
+ rtw89_err(rtwdev, "failed to alloc skb for h2c tp\n");
+ return -ENOMEM;
+ }
+
+ skb_put(skb, len);
+ h2c = (struct rtw89_h2c_ofld *)skb->data;
+
+ h2c->w0 = le32_encode_bits(rtwvif->mac_id, RTW89_H2C_OFLD_W0_MAC_ID) |
+ le32_encode_bits(stats->tx_throughput, RTW89_H2C_OFLD_W0_TX_TP) |
+ le32_encode_bits(stats->rx_throughput, RTW89_H2C_OFLD_W0_RX_TP);
+
+ rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
+ H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD,
+ H2C_FUNC_OFLD_TP, 0, 1, len);
+
+ ret = rtw89_h2c_tx(rtwdev, skb, false);
+ if (ret) {
+ rtw89_err(rtwdev, "failed to send h2c\n");
+ goto fail;
+ }
+
+ return 0;
+fail:
+ dev_kfree_skb_any(skb);
+
+ return ret;
+}
+
#define H2C_RA_LEN 16
int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi)
{
@@ -1806,8 +1965,6 @@ int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi
return ret;
}
-#define H2C_LEN_CXDRVHDR 2
-#define H2C_LEN_CXDRVINFO_INIT (12 + H2C_LEN_CXDRVHDR)
int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev)
{
struct rtw89_btc *btc = &rtwdev->btc;
@@ -1815,44 +1972,52 @@ int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev)
struct rtw89_btc_init_info *init_info = &dm->init_info;
struct rtw89_btc_module *module = &init_info->module;
struct rtw89_btc_ant_info *ant = &module->ant;
+ struct rtw89_h2c_cxinit *h2c;
+ u32 len = sizeof(*h2c);
struct sk_buff *skb;
- u8 *cmd;
int ret;
- skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_CXDRVINFO_INIT);
+ skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len);
if (!skb) {
rtw89_err(rtwdev, "failed to alloc skb for h2c cxdrv_init\n");
return -ENOMEM;
}
- skb_put(skb, H2C_LEN_CXDRVINFO_INIT);
- cmd = skb->data;
+ skb_put(skb, len);
+ h2c = (struct rtw89_h2c_cxinit *)skb->data;
- RTW89_SET_FWCMD_CXHDR_TYPE(cmd, CXDRVINFO_INIT);
- RTW89_SET_FWCMD_CXHDR_LEN(cmd, H2C_LEN_CXDRVINFO_INIT - H2C_LEN_CXDRVHDR);
+ h2c->hdr.type = CXDRVINFO_INIT;
+ h2c->hdr.len = len - H2C_LEN_CXDRVHDR;
- RTW89_SET_FWCMD_CXINIT_ANT_TYPE(cmd, ant->type);
- RTW89_SET_FWCMD_CXINIT_ANT_NUM(cmd, ant->num);
- RTW89_SET_FWCMD_CXINIT_ANT_ISO(cmd, ant->isolation);
- RTW89_SET_FWCMD_CXINIT_ANT_POS(cmd, ant->single_pos);
- RTW89_SET_FWCMD_CXINIT_ANT_DIVERSITY(cmd, ant->diversity);
+ h2c->ant_type = ant->type;
+ h2c->ant_num = ant->num;
+ h2c->ant_iso = ant->isolation;
+ h2c->ant_info =
+ u8_encode_bits(ant->single_pos, RTW89_H2C_CXINIT_ANT_INFO_POS) |
+ u8_encode_bits(ant->diversity, RTW89_H2C_CXINIT_ANT_INFO_DIVERSITY) |
+ u8_encode_bits(ant->btg_pos, RTW89_H2C_CXINIT_ANT_INFO_BTG_POS) |
+ u8_encode_bits(ant->stream_cnt, RTW89_H2C_CXINIT_ANT_INFO_STREAM_CNT);
- RTW89_SET_FWCMD_CXINIT_MOD_RFE(cmd, module->rfe_type);
- RTW89_SET_FWCMD_CXINIT_MOD_CV(cmd, module->cv);
- RTW89_SET_FWCMD_CXINIT_MOD_BT_SOLO(cmd, module->bt_solo);
- RTW89_SET_FWCMD_CXINIT_MOD_BT_POS(cmd, module->bt_pos);
- RTW89_SET_FWCMD_CXINIT_MOD_SW_TYPE(cmd, module->switch_type);
+ h2c->mod_rfe = module->rfe_type;
+ h2c->mod_cv = module->cv;
+ h2c->mod_info =
+ u8_encode_bits(module->bt_solo, RTW89_H2C_CXINIT_MOD_INFO_BT_SOLO) |
+ u8_encode_bits(module->bt_pos, RTW89_H2C_CXINIT_MOD_INFO_BT_POS) |
+ u8_encode_bits(module->switch_type, RTW89_H2C_CXINIT_MOD_INFO_SW_TYPE) |
+ u8_encode_bits(module->wa_type, RTW89_H2C_CXINIT_MOD_INFO_WA_TYPE);
+ h2c->mod_adie_kt = module->kt_ver_adie;
+ h2c->wl_gch = init_info->wl_guard_ch;
- RTW89_SET_FWCMD_CXINIT_WL_GCH(cmd, init_info->wl_guard_ch);
- RTW89_SET_FWCMD_CXINIT_WL_ONLY(cmd, init_info->wl_only);
- RTW89_SET_FWCMD_CXINIT_WL_INITOK(cmd, init_info->wl_init_ok);
- RTW89_SET_FWCMD_CXINIT_DBCC_EN(cmd, init_info->dbcc_en);
- RTW89_SET_FWCMD_CXINIT_CX_OTHER(cmd, init_info->cx_other);
- RTW89_SET_FWCMD_CXINIT_BT_ONLY(cmd, init_info->bt_only);
+ h2c->info =
+ u8_encode_bits(init_info->wl_only, RTW89_H2C_CXINIT_INFO_WL_ONLY) |
+ u8_encode_bits(init_info->wl_init_ok, RTW89_H2C_CXINIT_INFO_WL_INITOK) |
+ u8_encode_bits(init_info->dbcc_en, RTW89_H2C_CXINIT_INFO_DBCC_EN) |
+ u8_encode_bits(init_info->cx_other, RTW89_H2C_CXINIT_INFO_CX_OTHER) |
+ u8_encode_bits(init_info->bt_only, RTW89_H2C_CXINIT_INFO_BT_ONLY);
rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
H2C_CAT_OUTSRC, BTFC_SET,
SET_DRV_INFO, 0, 0,
- H2C_LEN_CXDRVINFO_INIT);
+ len);
ret = rtw89_h2c_tx(rtwdev, skb, false);
if (ret) {
@@ -2422,46 +2587,51 @@ int rtw89_fw_h2c_scan_list_offload(struct rtw89_dev *rtwdev, int len,
return ret;
}
-#define H2C_LEN_SCAN_OFFLOAD 28
int rtw89_fw_h2c_scan_offload(struct rtw89_dev *rtwdev,
struct rtw89_scan_option *option,
struct rtw89_vif *rtwvif)
{
- struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info;
+ struct rtw89_chan *op = &rtwdev->scan_info.op_chan;
+ struct rtw89_h2c_scanofld *h2c;
+ u32 len = sizeof(*h2c);
struct sk_buff *skb;
- u8 *cmd;
int ret;
- skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, H2C_LEN_SCAN_OFFLOAD);
+ skb = rtw89_fw_h2c_alloc_skb_with_hdr(rtwdev, len);
if (!skb) {
rtw89_err(rtwdev, "failed to alloc skb for h2c scan offload\n");
return -ENOMEM;
}
- skb_put(skb, H2C_LEN_SCAN_OFFLOAD);
- cmd = skb->data;
+ skb_put(skb, len);
+ h2c = (struct rtw89_h2c_scanofld *)skb->data;
- RTW89_SET_FWCMD_SCANOFLD_MACID(cmd, rtwvif->mac_id);
- RTW89_SET_FWCMD_SCANOFLD_PORT_ID(cmd, rtwvif->port);
- RTW89_SET_FWCMD_SCANOFLD_BAND(cmd, RTW89_PHY_0);
- RTW89_SET_FWCMD_SCANOFLD_OPERATION(cmd, option->enable);
- RTW89_SET_FWCMD_SCANOFLD_NOTIFY_END(cmd, true);
- RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_MODE(cmd, option->target_ch_mode);
- RTW89_SET_FWCMD_SCANOFLD_START_MODE(cmd, RTW89_SCAN_IMMEDIATE);
- RTW89_SET_FWCMD_SCANOFLD_SCAN_TYPE(cmd, RTW89_SCAN_ONCE);
+ h2c->w0 = le32_encode_bits(rtwvif->mac_id, RTW89_H2C_SCANOFLD_W0_MACID) |
+ le32_encode_bits(rtwvif->port, RTW89_H2C_SCANOFLD_W0_PORT_ID) |
+ le32_encode_bits(RTW89_PHY_0, RTW89_H2C_SCANOFLD_W0_BAND) |
+ le32_encode_bits(option->enable, RTW89_H2C_SCANOFLD_W0_OPERATION);
+
+ h2c->w1 = le32_encode_bits(true, RTW89_H2C_SCANOFLD_W1_NOTIFY_END) |
+ le32_encode_bits(option->target_ch_mode,
+ RTW89_H2C_SCANOFLD_W1_TARGET_CH_MODE) |
+ le32_encode_bits(RTW89_SCAN_IMMEDIATE,
+ RTW89_H2C_SCANOFLD_W1_START_MODE) |
+ le32_encode_bits(RTW89_SCAN_ONCE, RTW89_H2C_SCANOFLD_W1_SCAN_TYPE);
+
if (option->target_ch_mode) {
- RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_BW(cmd, scan_info->op_bw);
- RTW89_SET_FWCMD_SCANOFLD_TARGET_PRI_CH(cmd,
- scan_info->op_pri_ch);
- RTW89_SET_FWCMD_SCANOFLD_TARGET_CENTRAL_CH(cmd,
- scan_info->op_chan);
- RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_BAND(cmd,
- scan_info->op_band);
+ h2c->w1 |= le32_encode_bits(op->band_width,
+ RTW89_H2C_SCANOFLD_W1_TARGET_CH_BW) |
+ le32_encode_bits(op->primary_channel,
+ RTW89_H2C_SCANOFLD_W1_TARGET_PRI_CH) |
+ le32_encode_bits(op->channel,
+ RTW89_H2C_SCANOFLD_W1_TARGET_CENTRAL_CH);
+ h2c->w0 |= le32_encode_bits(op->band_type,
+ RTW89_H2C_SCANOFLD_W0_TARGET_CH_BAND);
}
rtw89_h2c_pkt_set_hdr(rtwdev, skb, FWCMD_TYPE_H2C,
H2C_CAT_MAC, H2C_CL_MAC_FW_OFLD,
H2C_FUNC_SCANOFLD, 1, 1,
- H2C_LEN_SCAN_OFFLOAD);
+ len);
ret = rtw89_h2c_tx(rtwdev, skb, false);
if (ret) {
@@ -3034,6 +3204,7 @@ static void rtw89_hw_scan_add_chan(struct rtw89_dev *rtwdev, int chan_type,
struct ieee80211_vif *vif = rtwdev->scan_info.scanning_vif;
struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
struct cfg80211_scan_request *req = rtwvif->scan_req;
+ struct rtw89_chan *op = &rtwdev->scan_info.op_chan;
struct rtw89_pktofld_info *info;
u8 band, probe_count = 0;
int ret;
@@ -3077,10 +3248,10 @@ static void rtw89_hw_scan_add_chan(struct rtw89_dev *rtwdev, int chan_type,
switch (chan_type) {
case RTW89_CHAN_OPERATE:
- ch_info->central_ch = scan_info->op_chan;
- ch_info->pri_ch = scan_info->op_pri_ch;
- ch_info->ch_band = scan_info->op_band;
- ch_info->bw = scan_info->op_bw;
+ ch_info->central_ch = op->channel;
+ ch_info->pri_ch = op->primary_channel;
+ ch_info->ch_band = op->band_type;
+ ch_info->bw = op->band_width;
ch_info->tx_null = true;
ch_info->num_pkt = 0;
break;
@@ -3098,7 +3269,7 @@ static void rtw89_hw_scan_add_chan(struct rtw89_dev *rtwdev, int chan_type,
}
static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev,
- struct rtw89_vif *rtwvif)
+ struct rtw89_vif *rtwvif, bool connected)
{
struct cfg80211_scan_request *req = rtwvif->scan_req;
struct rtw89_mac_chinfo *ch_info, *tmp;
@@ -3142,7 +3313,7 @@ static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev,
type = RTW89_CHAN_ACTIVE;
rtw89_hw_scan_add_chan(rtwdev, type, req->n_ssids, ch_info);
- if (rtwvif->net_type != RTW89_NET_TYPE_NO_LINK &&
+ if (connected &&
off_chan_time + ch_info->period > RTW89_OFF_CHAN_TIME) {
tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
if (!tmp) {
@@ -3175,7 +3346,7 @@ static int rtw89_hw_scan_add_chan_list(struct rtw89_dev *rtwdev,
}
static int rtw89_hw_scan_prehandle(struct rtw89_dev *rtwdev,
- struct rtw89_vif *rtwvif)
+ struct rtw89_vif *rtwvif, bool connected)
{
int ret;
@@ -3184,7 +3355,7 @@ static int rtw89_hw_scan_prehandle(struct rtw89_dev *rtwdev,
rtw89_err(rtwdev, "Update probe request failed\n");
goto out;
}
- ret = rtw89_hw_scan_add_chan_list(rtwdev, rtwvif);
+ ret = rtw89_hw_scan_add_chan_list(rtwdev, rtwvif, connected);
out:
return ret;
}
@@ -3197,6 +3368,7 @@ void rtw89_hw_scan_start(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
u32 rx_fltr = rtwdev->hal.rx_fltr;
u8 mac_addr[ETH_ALEN];
+ rtw89_get_channel(rtwdev, rtwvif, &rtwdev->scan_info.op_chan);
rtwdev->scan_info.scanning_vif = vif;
rtwdev->scan_info.last_chan_idx = 0;
rtwvif->scan_ies = &scan_req->ies;
@@ -3222,6 +3394,7 @@ void rtw89_hw_scan_start(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
bool aborted)
{
+ struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info;
struct cfg80211_scan_info info = {
.aborted = aborted,
};
@@ -3243,11 +3416,9 @@ void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
rtwvif = (struct rtw89_vif *)vif->drv_priv;
rtwvif->scan_req = NULL;
rtwvif->scan_ies = NULL;
- rtwdev->scan_info.last_chan_idx = 0;
- rtwdev->scan_info.scanning_vif = NULL;
+ scan_info->last_chan_idx = 0;
+ scan_info->scanning_vif = NULL;
- if (rtwvif->net_type != RTW89_NET_TYPE_NO_LINK)
- rtw89_store_op_chan(rtwdev, false);
rtw89_set_channel(rtwdev);
}
@@ -3262,16 +3433,19 @@ int rtw89_hw_scan_offload(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
{
struct rtw89_scan_option opt = {0};
struct rtw89_vif *rtwvif;
+ bool connected;
int ret = 0;
rtwvif = vif ? (struct rtw89_vif *)vif->drv_priv : NULL;
if (!rtwvif)
return -EINVAL;
+ /* This variable implies connected or during attempt to connect */
+ connected = !is_zero_ether_addr(rtwvif->bssid);
opt.enable = enable;
- opt.target_ch_mode = rtwvif->net_type != RTW89_NET_TYPE_NO_LINK;
+ opt.target_ch_mode = connected;
if (enable) {
- ret = rtw89_hw_scan_prehandle(rtwdev, rtwvif);
+ ret = rtw89_hw_scan_prehandle(rtwdev, rtwvif, connected);
if (ret)
goto out;
}
@@ -3280,24 +3454,6 @@ int rtw89_hw_scan_offload(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
return ret;
}
-void rtw89_store_op_chan(struct rtw89_dev *rtwdev, bool backup)
-{
- struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info;
- const struct rtw89_chan *cur = rtw89_chan_get(rtwdev, RTW89_SUB_ENTITY_0);
- struct rtw89_chan new;
-
- if (backup) {
- scan_info->op_pri_ch = cur->primary_channel;
- scan_info->op_chan = cur->channel;
- scan_info->op_bw = cur->band_width;
- scan_info->op_band = cur->band_type;
- } else {
- rtw89_chan_create(&new, scan_info->op_chan, scan_info->op_pri_ch,
- scan_info->op_band, scan_info->op_bw);
- rtw89_assign_entity_chan(rtwdev, RTW89_SUB_ENTITY_0, &new);
- }
-}
-
#define H2C_FW_CPU_EXCEPTION_LEN 4
#define H2C_FW_CPU_EXCEPTION_TYPE_DEF 0x5566
int rtw89_fw_h2c_trigger_cpu_exception(struct rtw89_dev *rtwdev)
diff --git a/drivers/net/wireless/realtek/rtw89/fw.h b/drivers/net/wireless/realtek/rtw89/fw.h
index c3c67dd..675f85c 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.h
+++ b/drivers/net/wireless/realtek/rtw89/fw.h
@@ -162,6 +162,27 @@ enum rtw89_p2pps_action {
RTW89_P2P_ACT_TERMINATE = 3,
};
+enum rtw89_bcn_fltr_offload_mode {
+ RTW89_BCN_FLTR_OFFLOAD_MODE_0 = 0,
+ RTW89_BCN_FLTR_OFFLOAD_MODE_1,
+ RTW89_BCN_FLTR_OFFLOAD_MODE_2,
+ RTW89_BCN_FLTR_OFFLOAD_MODE_3,
+
+ RTW89_BCN_FLTR_OFFLOAD_MODE_DEFAULT = RTW89_BCN_FLTR_OFFLOAD_MODE_0,
+};
+
+enum rtw89_bcn_fltr_type {
+ RTW89_BCN_FLTR_BEACON_LOSS,
+ RTW89_BCN_FLTR_RSSI,
+ RTW89_BCN_FLTR_NOTIFY,
+};
+
+enum rtw89_bcn_fltr_rssi_event {
+ RTW89_BCN_FLTR_RSSI_NOT_CHANGED,
+ RTW89_BCN_FLTR_RSSI_HIGH,
+ RTW89_BCN_FLTR_RSSI_LOW,
+};
+
#define FWDL_SECTION_MAX_NUM 10
#define FWDL_SECTION_CHKSUM_LEN 8
#define FWDL_SECTION_PER_PKT_LEN 2020
@@ -216,6 +237,8 @@ struct rtw89_h2creg_sch_tx_en {
#define RTW89_SCAN_LIST_LIMIT \
((RTW89_H2C_MAX_SIZE / RTW89_MAC_CHINFO_SIZE) - RTW89_SCAN_LIST_GUARD)
+#define RTW89_BCN_LOSS_CNT 10
+
struct rtw89_mac_chinfo {
u8 period;
u8 dwell_time;
@@ -2174,85 +2197,44 @@ static inline void RTW89_SET_FWCMD_CXHDR_LEN(void *cmd, u8 val)
u8p_replace_bits((u8 *)(cmd) + 1, val, GENMASK(7, 0));
}
-static inline void RTW89_SET_FWCMD_CXINIT_ANT_TYPE(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 2, val, GENMASK(7, 0));
-}
+struct rtw89_h2c_cxhdr {
+ u8 type;
+ u8 len;
+} __packed;
-static inline void RTW89_SET_FWCMD_CXINIT_ANT_NUM(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 3, val, GENMASK(7, 0));
-}
+#define H2C_LEN_CXDRVHDR sizeof(struct rtw89_h2c_cxhdr)
-static inline void RTW89_SET_FWCMD_CXINIT_ANT_ISO(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 4, val, GENMASK(7, 0));
-}
+struct rtw89_h2c_cxinit {
+ struct rtw89_h2c_cxhdr hdr;
+ u8 ant_type;
+ u8 ant_num;
+ u8 ant_iso;
+ u8 ant_info;
+ u8 mod_rfe;
+ u8 mod_cv;
+ u8 mod_info;
+ u8 mod_adie_kt;
+ u8 wl_gch;
+ u8 info;
+ u8 rsvd;
+ u8 rsvd1;
+} __packed;
-static inline void RTW89_SET_FWCMD_CXINIT_ANT_POS(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(0));
-}
+#define RTW89_H2C_CXINIT_ANT_INFO_POS BIT(0)
+#define RTW89_H2C_CXINIT_ANT_INFO_DIVERSITY BIT(1)
+#define RTW89_H2C_CXINIT_ANT_INFO_BTG_POS GENMASK(3, 2)
+#define RTW89_H2C_CXINIT_ANT_INFO_STREAM_CNT GENMASK(7, 4)
-static inline void RTW89_SET_FWCMD_CXINIT_ANT_DIVERSITY(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 5, val, BIT(1));
-}
+#define RTW89_H2C_CXINIT_MOD_INFO_BT_SOLO BIT(0)
+#define RTW89_H2C_CXINIT_MOD_INFO_BT_POS BIT(1)
+#define RTW89_H2C_CXINIT_MOD_INFO_SW_TYPE BIT(2)
+#define RTW89_H2C_CXINIT_MOD_INFO_WA_TYPE GENMASK(5, 3)
-static inline void RTW89_SET_FWCMD_CXINIT_MOD_RFE(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 6, val, GENMASK(7, 0));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_MOD_CV(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 7, val, GENMASK(7, 0));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_MOD_BT_SOLO(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(0));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_MOD_BT_POS(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(1));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_MOD_SW_TYPE(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 8, val, BIT(2));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_WL_GCH(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 10, val, GENMASK(7, 0));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_WL_ONLY(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(0));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_WL_INITOK(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(1));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_DBCC_EN(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(2));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_CX_OTHER(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(3));
-}
-
-static inline void RTW89_SET_FWCMD_CXINIT_BT_ONLY(void *cmd, u8 val)
-{
- u8p_replace_bits((u8 *)(cmd) + 11, val, BIT(4));
-}
+#define RTW89_H2C_CXINIT_INFO_WL_ONLY BIT(0)
+#define RTW89_H2C_CXINIT_INFO_WL_INITOK BIT(1)
+#define RTW89_H2C_CXINIT_INFO_DBCC_EN BIT(2)
+#define RTW89_H2C_CXINIT_INFO_CX_OTHER BIT(3)
+#define RTW89_H2C_CXINIT_INFO_BT_ONLY BIT(4)
static inline void RTW89_SET_FWCMD_CXROLE_CONNECT_CNT(void *cmd, u8 val)
{
@@ -2749,96 +2731,32 @@ static inline void RTW89_SET_FWCMD_CHINFO_POWER_IDX(void *cmd, u32 val)
le32p_replace_bits((__le32 *)((u8 *)(cmd) + 16), val, GENMASK(15, 0));
}
-static inline void RTW89_SET_FWCMD_SCANOFLD_MACID(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, GENMASK(7, 0));
-}
+struct rtw89_h2c_scanofld {
+ __le32 w0;
+ __le32 w1;
+ __le32 w2;
+ __le32 tsf_high;
+ __le32 tsf_low;
+ __le32 w5;
+ __le32 w6;
+} __packed;
-static inline void RTW89_SET_FWCMD_SCANOFLD_NORM_CY(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, GENMASK(15, 8));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_PORT_ID(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, GENMASK(18, 16));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_BAND(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, BIT(19));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_OPERATION(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, GENMASK(21, 20));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_BAND(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd)), val, GENMASK(23, 22));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_NOTIFY_END(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, BIT(0));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_MODE(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, BIT(1));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_START_MODE(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, BIT(2));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_SCAN_TYPE(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, GENMASK(4, 3));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TARGET_CH_BW(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, GENMASK(7, 5));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TARGET_PRI_CH(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, GENMASK(15, 8));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TARGET_CENTRAL_CH(void *cmd,
- u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, GENMASK(23, 16));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_PROBE_REQ_PKT_ID(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 4), val, GENMASK(31, 24));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_NORM_PD(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 8), val, GENMASK(15, 0));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_SLOW_PD(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 8), val, GENMASK(23, 16));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TSF_HIGH(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 12), val, GENMASK(31, 0));
-}
-
-static inline void RTW89_SET_FWCMD_SCANOFLD_TSF_SLOW(void *cmd, u32 val)
-{
- le32p_replace_bits((__le32 *)((u8 *)(cmd) + 16), val, GENMASK(31, 0));
-}
+#define RTW89_H2C_SCANOFLD_W0_MACID GENMASK(7, 0)
+#define RTW89_H2C_SCANOFLD_W0_NORM_CY GENMASK(15, 8)
+#define RTW89_H2C_SCANOFLD_W0_PORT_ID GENMASK(18, 16)
+#define RTW89_H2C_SCANOFLD_W0_BAND BIT(19)
+#define RTW89_H2C_SCANOFLD_W0_OPERATION GENMASK(21, 20)
+#define RTW89_H2C_SCANOFLD_W0_TARGET_CH_BAND GENMASK(23, 22)
+#define RTW89_H2C_SCANOFLD_W1_NOTIFY_END BIT(0)
+#define RTW89_H2C_SCANOFLD_W1_TARGET_CH_MODE BIT(1)
+#define RTW89_H2C_SCANOFLD_W1_START_MODE BIT(2)
+#define RTW89_H2C_SCANOFLD_W1_SCAN_TYPE GENMASK(4, 3)
+#define RTW89_H2C_SCANOFLD_W1_TARGET_CH_BW GENMASK(7, 5)
+#define RTW89_H2C_SCANOFLD_W1_TARGET_PRI_CH GENMASK(15, 8)
+#define RTW89_H2C_SCANOFLD_W1_TARGET_CENTRAL_CH GENMASK(23, 16)
+#define RTW89_H2C_SCANOFLD_W1_PROBE_REQ_PKT_ID GENMASK(31, 24)
+#define RTW89_H2C_SCANOFLD_W2_NORM_PD GENMASK(15, 0)
+#define RTW89_H2C_SCANOFLD_W2_SLOW_PD GENMASK(23, 16)
static inline void RTW89_SET_FWCMD_P2P_MACID(void *cmd, u32 val)
{
@@ -3317,6 +3235,17 @@ static inline struct rtw89_fw_c2h_attr *RTW89_SKB_C2H_CB(struct sk_buff *skb)
#define RTW89_GET_MAC_C2H_REV_ACK_H2C_SEQ(c2h) \
le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(23, 16))
+struct rtw89_c2h_mac_bcnfltr_rpt {
+ __le32 w0;
+ __le32 w1;
+ __le32 w2;
+} __packed;
+
+#define RTW89_C2H_MAC_BCNFLTR_RPT_W2_MACID GENMASK(7, 0)
+#define RTW89_C2H_MAC_BCNFLTR_RPT_W2_TYPE GENMASK(9, 8)
+#define RTW89_C2H_MAC_BCNFLTR_RPT_W2_EVENT GENMASK(11, 10)
+#define RTW89_C2H_MAC_BCNFLTR_RPT_W2_MA GENMASK(23, 16)
+
#define RTW89_GET_PHY_C2H_RA_RPT_MACID(c2h) \
le32_get_bits(*((const __le32 *)(c2h) + 2), GENMASK(15, 0))
#define RTW89_GET_PHY_C2H_RA_RPT_RETRY_RATIO(c2h) \
@@ -3410,6 +3339,36 @@ static_assert(sizeof(struct rtw89_mac_mcc_tsf_rpt) <= RTW89_COMPLETION_BUF_SIZE)
#define RTW89_GET_MAC_C2H_MCC_STATUS_RPT_TSF_HIGH(c2h) \
le32_get_bits(*((const __le32 *)(c2h) + 4), GENMASK(31, 0))
+struct rtw89_h2c_bcnfltr {
+ __le32 w0;
+} __packed;
+
+#define RTW89_H2C_BCNFLTR_W0_MON_RSSI BIT(0)
+#define RTW89_H2C_BCNFLTR_W0_MON_BCN BIT(1)
+#define RTW89_H2C_BCNFLTR_W0_MON_EN BIT(2)
+#define RTW89_H2C_BCNFLTR_W0_MODE GENMASK(4, 3)
+#define RTW89_H2C_BCNFLTR_W0_BCN_LOSS_CNT GENMASK(11, 8)
+#define RTW89_H2C_BCNFLTR_W0_RSSI_HYST GENMASK(15, 12)
+#define RTW89_H2C_BCNFLTR_W0_RSSI_THRESHOLD GENMASK(23, 16)
+#define RTW89_H2C_BCNFLTR_W0_MAC_ID GENMASK(31, 24)
+
+struct rtw89_h2c_ofld_rssi {
+ __le32 w0;
+ __le32 w1;
+} __packed;
+
+#define RTW89_H2C_OFLD_RSSI_W0_MACID GENMASK(7, 0)
+#define RTW89_H2C_OFLD_RSSI_W0_NUM GENMASK(15, 8)
+#define RTW89_H2C_OFLD_RSSI_W1_VAL GENMASK(7, 0)
+
+struct rtw89_h2c_ofld {
+ __le32 w0;
+} __packed;
+
+#define RTW89_H2C_OFLD_W0_MAC_ID GENMASK(7, 0)
+#define RTW89_H2C_OFLD_W0_TX_TP GENMASK(17, 8)
+#define RTW89_H2C_OFLD_W0_RX_TP GENMASK(27, 18)
+
#define RTW89_FW_HDR_SIZE 32
#define RTW89_FW_SECTION_HDR_SIZE 16
@@ -3459,6 +3418,15 @@ static inline u32 rtw89_compat_fw_hdr_ver_code(const void *fw_buf)
return RTW89_FW_HDR_VER_CODE(&compat->fw_hdr);
}
+static inline void rtw89_fw_get_filename(char *buf, size_t size,
+ const char *fw_basename, int fw_format)
+{
+ if (fw_format <= 0)
+ snprintf(buf, size, "%s.bin", fw_basename);
+ else
+ snprintf(buf, size, "%s-%d.bin", fw_basename, fw_format);
+}
+
#define RTW89_H2C_RF_PAGE_SIZE 500
#define RTW89_H2C_RF_PAGE_NUM 3
struct rtw89_fw_h2c_rf_reg_info {
@@ -3537,6 +3505,9 @@ struct rtw89_fw_h2c_rf_reg_info {
#define H2C_FUNC_ADD_SCANOFLD_CH 0x16
#define H2C_FUNC_SCANOFLD 0x17
#define H2C_FUNC_PKT_DROP 0x1b
+#define H2C_FUNC_CFG_BCNFLTR 0x1e
+#define H2C_FUNC_OFLD_RSSI 0x1f
+#define H2C_FUNC_OFLD_TP 0x20
/* CLASS 10 - Security CAM */
#define H2C_CL_MAC_SEC_CAM 0xa
@@ -3600,9 +3571,10 @@ int rtw89_fw_recognize(struct rtw89_dev *rtwdev);
const struct firmware *
rtw89_early_fw_feature_recognize(struct device *device,
const struct rtw89_chip_info *chip,
- u32 *early_feat_map);
+ struct rtw89_fw_info *early_fw,
+ int *used_fw_format);
int rtw89_fw_download(struct rtw89_dev *rtwdev, enum rtw89_fw_type type);
-int rtw89_load_firmware(struct rtw89_dev *rtwdev);
+void rtw89_load_firmware_work(struct work_struct *work);
void rtw89_unload_firmware(struct rtw89_dev *rtwdev);
int rtw89_wait_firmware_completion(struct rtw89_dev *rtwdev);
void rtw89_h2c_pkt_set_hdr(struct rtw89_dev *rtwdev, struct sk_buff *skb,
@@ -3637,6 +3609,12 @@ int rtw89_fw_h2c_macid_pause(struct rtw89_dev *rtwdev, u8 sh, u8 grp,
int rtw89_fw_h2c_set_edca(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
u8 ac, u32 val);
int rtw89_fw_h2c_set_ofld_cfg(struct rtw89_dev *rtwdev);
+int rtw89_fw_h2c_set_bcn_fltr_cfg(struct rtw89_dev *rtwdev,
+ struct ieee80211_vif *vif,
+ bool connect);
+int rtw89_fw_h2c_rssi_offload(struct rtw89_dev *rtwdev,
+ struct rtw89_rx_phy_ppdu *phy_ppdu);
+int rtw89_fw_h2c_tp_offload(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
int rtw89_fw_h2c_ra(struct rtw89_dev *rtwdev, struct rtw89_ra_info *ra, bool csi);
int rtw89_fw_h2c_cxdrv_init(struct rtw89_dev *rtwdev);
int rtw89_fw_h2c_cxdrv_role(struct rtw89_dev *rtwdev);
@@ -3681,7 +3659,6 @@ int rtw89_fw_msg_reg(struct rtw89_dev *rtwdev,
struct rtw89_mac_c2h_info *c2h_info);
int rtw89_fw_h2c_fw_log(struct rtw89_dev *rtwdev, bool enable);
void rtw89_fw_st_dbg_dump(struct rtw89_dev *rtwdev);
-void rtw89_store_op_chan(struct rtw89_dev *rtwdev, bool backup);
void rtw89_hw_scan_start(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
struct ieee80211_scan_request *req);
void rtw89_hw_scan_complete(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c
index d0e138f..b8019cf 100644
--- a/drivers/net/wireless/realtek/rtw89/mac.c
+++ b/drivers/net/wireless/realtek/rtw89/mac.c
@@ -1473,6 +1473,8 @@ const struct rtw89_mac_size_set rtw89_mac_size = {
.ple_qt58 = {147, 0, 16, 20, 157, 13, 229, 0, 172, 14, 24, 0,},
/* 8852A PCIE WOW */
.ple_qt_52a_wow = {264, 0, 32, 20, 64, 13, 1005, 0, 64, 128, 120,},
+ /* 8852B PCIE WOW */
+ .ple_qt_52b_wow = {147, 0, 16, 20, 157, 13, 133, 0, 172, 14, 24, 0,},
};
EXPORT_SYMBOL(rtw89_mac_size);
@@ -1584,12 +1586,15 @@ static void dle_func_en(struct rtw89_dev *rtwdev, bool enable)
static void dle_clk_en(struct rtw89_dev *rtwdev, bool enable)
{
- if (enable)
- rtw89_write32_set(rtwdev, R_AX_DMAC_CLK_EN,
- B_AX_DLE_WDE_CLK_EN | B_AX_DLE_PLE_CLK_EN);
- else
- rtw89_write32_clr(rtwdev, R_AX_DMAC_CLK_EN,
- B_AX_DLE_WDE_CLK_EN | B_AX_DLE_PLE_CLK_EN);
+ u32 val = B_AX_DLE_WDE_CLK_EN | B_AX_DLE_PLE_CLK_EN;
+
+ if (enable) {
+ if (rtwdev->chip->chip_id == RTL8851B)
+ val |= B_AX_AXIDMA_CLK_EN;
+ rtw89_write32_set(rtwdev, R_AX_DMAC_CLK_EN, val);
+ } else {
+ rtw89_write32_clr(rtwdev, R_AX_DMAC_CLK_EN, val);
+ }
}
static int dle_mix_cfg(struct rtw89_dev *rtwdev, const struct rtw89_dle_mem *cfg)
@@ -1854,7 +1859,8 @@ static int preload_init(struct rtw89_dev *rtwdev, enum rtw89_mac_idx mac_idx,
{
const struct rtw89_chip_info *chip = rtwdev->chip;
- if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B || !is_qta_poh(rtwdev))
+ if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B ||
+ chip->chip_id == RTL8851B || !is_qta_poh(rtwdev))
return 0;
return preload_init_set(rtwdev, mac_idx, mode);
@@ -1890,7 +1896,8 @@ static void _patch_ss2f_path(struct rtw89_dev *rtwdev)
{
const struct rtw89_chip_info *chip = rtwdev->chip;
- if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B)
+ if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B ||
+ chip->chip_id == RTL8851B)
return;
rtw89_write32_mask(rtwdev, R_AX_SS2FINFO_PATH, B_AX_SS_DEST_QUEUE_MASK,
@@ -1959,7 +1966,8 @@ static int sec_eng_init(struct rtw89_dev *rtwdev)
/* init TX encryption */
val |= (B_AX_SEC_TX_ENC | B_AX_SEC_RX_DEC);
val |= (B_AX_MC_DEC | B_AX_BC_DEC);
- if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B)
+ if (chip->chip_id == RTL8852A || chip->chip_id == RTL8852B ||
+ chip->chip_id == RTL8851B)
val &= ~B_AX_TX_PARTIAL_MODE;
rtw89_write32(rtwdev, R_AX_SEC_ENG_CTRL, val);
@@ -2065,7 +2073,7 @@ static int scheduler_init(struct rtw89_dev *rtwdev, u8 mac_idx)
rtw89_write32_mask(rtwdev, reg, B_AX_SIFS_MACTXEN_T1_MASK,
SIFS_MACTXEN_T1);
- if (rtwdev->chip->chip_id == RTL8852B) {
+ if (rtwdev->chip->chip_id == RTL8852B || rtwdev->chip->chip_id == RTL8851B) {
reg = rtw89_mac_reg_by_idx(R_AX_SCH_EXT_CTRL, mac_idx);
rtw89_write32_set(rtwdev, reg, B_AX_PORT_RST_TSF_ADV);
}
@@ -2805,7 +2813,7 @@ int rtw89_mac_resume_sch_tx_v1(struct rtw89_dev *rtwdev, u8 mac_idx, u32 tx_en)
}
EXPORT_SYMBOL(rtw89_mac_resume_sch_tx_v1);
-u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd)
+int rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd, u16 *pkt_id)
{
u32 val, reg;
int ret;
@@ -2820,9 +2828,13 @@ u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd)
ret = read_poll_timeout(rtw89_read32, val, val & B_AX_WD_BUF_STAT_DONE,
1, 2000, false, rtwdev, reg);
if (ret)
- return 0xffff;
+ return ret;
- return FIELD_GET(B_AX_WD_BUF_STAT_PKTID_MASK, val);
+ *pkt_id = FIELD_GET(B_AX_WD_BUF_STAT_PKTID_MASK, val);
+ if (*pkt_id == S_WD_BUF_STAT_PKTID_INVALID)
+ return -ENOENT;
+
+ return 0;
}
int rtw89_mac_set_cpuio(struct rtw89_dev *rtwdev,
@@ -2899,10 +2911,10 @@ static int dle_quota_change(struct rtw89_dev *rtwdev, enum rtw89_qta_mode mode)
dle_quota_cfg(rtwdev, cfg, INVALID_QT_WCPU);
- pkt_id = rtw89_mac_dle_buf_req(rtwdev, 0x20, true);
- if (pkt_id == 0xffff) {
+ ret = rtw89_mac_dle_buf_req(rtwdev, 0x20, true, &pkt_id);
+ if (ret) {
rtw89_err(rtwdev, "[ERR]WDE DLE buf req\n");
- return -ENOMEM;
+ return ret;
}
ctrl_para.cmd_type = CPUIO_OP_CMD_ENQ_TO_HEAD;
@@ -2917,10 +2929,10 @@ static int dle_quota_change(struct rtw89_dev *rtwdev, enum rtw89_qta_mode mode)
return -EFAULT;
}
- pkt_id = rtw89_mac_dle_buf_req(rtwdev, 0x20, false);
- if (pkt_id == 0xffff) {
+ ret = rtw89_mac_dle_buf_req(rtwdev, 0x20, false, &pkt_id);
+ if (ret) {
rtw89_err(rtwdev, "[ERR]PLE DLE buf req\n");
- return -ENOMEM;
+ return ret;
}
ctrl_para.cmd_type = CPUIO_OP_CMD_ENQ_TO_HEAD;
@@ -3364,8 +3376,15 @@ static int rtw89_mac_trx_init(struct rtw89_dev *rtwdev)
static void rtw89_disable_fw_watchdog(struct rtw89_dev *rtwdev)
{
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
u32 val32;
+ if (chip_id == RTL8852B || chip_id == RTL8851B) {
+ rtw89_write32_clr(rtwdev, R_AX_PLATFORM_ENABLE, B_AX_APB_WRAP_EN);
+ rtw89_write32_set(rtwdev, R_AX_PLATFORM_ENABLE, B_AX_APB_WRAP_EN);
+ return;
+ }
+
rtw89_mac_mem_write(rtwdev, R_AX_WDT_CTRL,
WDT_CTRL_ALL_DIS, RTW89_MAC_MEM_CPU_LOCAL);
@@ -3450,7 +3469,10 @@ static int rtw89_mac_dmac_pre_init(struct rtw89_dev *rtwdev)
B_AX_PKT_BUF_EN;
rtw89_write32(rtwdev, R_AX_DMAC_FUNC_EN, val);
- val = B_AX_DISPATCHER_CLK_EN;
+ if (chip_id == RTL8851B)
+ val = B_AX_DISPATCHER_CLK_EN | B_AX_AXIDMA_CLK_EN;
+ else
+ val = B_AX_DISPATCHER_CLK_EN;
rtw89_write32(rtwdev, R_AX_DMAC_CLK_EN, val);
if (chip_id != RTL8852C)
@@ -4176,9 +4198,9 @@ rtw89_mac_c2h_macid_pause(struct rtw89_dev *rtwdev, struct sk_buff *c2h, u32 len
static bool rtw89_is_op_chan(struct rtw89_dev *rtwdev, u8 band, u8 channel)
{
- struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info;
+ const struct rtw89_chan *op = &rtwdev->scan_info.op_chan;
- return band == scan_info->op_band && channel == scan_info->op_pri_ch;
+ return band == op->band_type && channel == op->primary_channel;
}
static void
@@ -4193,6 +4215,9 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h,
u16 chan;
int ret;
+ if (!rtwvif)
+ return;
+
tx_fail = RTW89_GET_MAC_C2H_SCANOFLD_TX_FAIL(c2h->data);
status = RTW89_GET_MAC_C2H_SCANOFLD_STATUS(c2h->data);
chan = RTW89_GET_MAC_C2H_SCANOFLD_PRI_CH(c2h->data);
@@ -4225,11 +4250,15 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h,
}
break;
case RTW89_SCAN_ENTER_CH_NOTIFY:
- rtw89_chan_create(&new, chan, chan, band, RTW89_CHANNEL_WIDTH_20);
- rtw89_assign_entity_chan(rtwdev, RTW89_SUB_ENTITY_0, &new);
if (rtw89_is_op_chan(rtwdev, band, chan)) {
- rtw89_store_op_chan(rtwdev, false);
+ rtw89_assign_entity_chan(rtwdev, rtwvif->sub_entity_idx,
+ &rtwdev->scan_info.op_chan);
ieee80211_wake_queues(rtwdev->hw);
+ } else {
+ rtw89_chan_create(&new, chan, chan, band,
+ RTW89_CHANNEL_WIDTH_20);
+ rtw89_assign_entity_chan(rtwdev, rtwvif->sub_entity_idx,
+ &new);
}
break;
default:
@@ -4238,6 +4267,64 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *c2h,
}
static void
+rtw89_mac_bcn_fltr_rpt(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ struct sk_buff *skb)
+{
+ struct ieee80211_vif *vif = rtwvif_to_vif_safe(rtwvif);
+ enum nl80211_cqm_rssi_threshold_event nl_event;
+ const struct rtw89_c2h_mac_bcnfltr_rpt *c2h =
+ (const struct rtw89_c2h_mac_bcnfltr_rpt *)skb->data;
+ u8 type, event, mac_id;
+ s8 sig;
+
+ type = le32_get_bits(c2h->w2, RTW89_C2H_MAC_BCNFLTR_RPT_W2_TYPE);
+ sig = le32_get_bits(c2h->w2, RTW89_C2H_MAC_BCNFLTR_RPT_W2_MA) - MAX_RSSI;
+ event = le32_get_bits(c2h->w2, RTW89_C2H_MAC_BCNFLTR_RPT_W2_EVENT);
+ mac_id = le32_get_bits(c2h->w2, RTW89_C2H_MAC_BCNFLTR_RPT_W2_MACID);
+
+ if (mac_id != rtwvif->mac_id)
+ return;
+
+ rtw89_debug(rtwdev, RTW89_DBG_FW,
+ "C2H bcnfltr rpt macid: %d, type: %d, ma: %d, event: %d\n",
+ mac_id, type, sig, event);
+
+ switch (type) {
+ case RTW89_BCN_FLTR_BEACON_LOSS:
+ if (!rtwdev->scanning && !rtwvif->offchan)
+ ieee80211_connection_loss(vif);
+ else
+ rtw89_fw_h2c_set_bcn_fltr_cfg(rtwdev, vif, true);
+ return;
+ case RTW89_BCN_FLTR_NOTIFY:
+ nl_event = NL80211_CQM_RSSI_THRESHOLD_EVENT_HIGH;
+ break;
+ case RTW89_BCN_FLTR_RSSI:
+ if (event == RTW89_BCN_FLTR_RSSI_LOW)
+ nl_event = NL80211_CQM_RSSI_THRESHOLD_EVENT_LOW;
+ else if (event == RTW89_BCN_FLTR_RSSI_HIGH)
+ nl_event = NL80211_CQM_RSSI_THRESHOLD_EVENT_HIGH;
+ else
+ return;
+ break;
+ default:
+ return;
+ }
+
+ ieee80211_cqm_rssi_notify(vif, nl_event, sig, GFP_KERNEL);
+}
+
+static void
+rtw89_mac_c2h_bcn_fltr_rpt(struct rtw89_dev *rtwdev, struct sk_buff *c2h,
+ u32 len)
+{
+ struct rtw89_vif *rtwvif;
+
+ rtw89_for_each_rtwvif(rtwdev, rtwvif)
+ rtw89_mac_bcn_fltr_rpt(rtwdev, rtwvif, c2h);
+}
+
+static void
rtw89_mac_c2h_rec_ack(struct rtw89_dev *rtwdev, struct sk_buff *c2h, u32 len)
{
rtw89_debug(rtwdev, RTW89_DBG_FW,
@@ -4457,6 +4544,7 @@ void (* const rtw89_mac_c2h_ofld_handler[])(struct rtw89_dev *rtwdev,
[RTW89_MAC_C2H_FUNC_MACID_PAUSE] = rtw89_mac_c2h_macid_pause,
[RTW89_MAC_C2H_FUNC_SCANOFLD_RSP] = rtw89_mac_c2h_scanofld_rsp,
[RTW89_MAC_C2H_FUNC_TSF32_TOGL_RPT] = rtw89_mac_c2h_tsf32_toggle_rpt,
+ [RTW89_MAC_C2H_FUNC_BCNFLTR_RPT] = rtw89_mac_c2h_bcn_fltr_rpt,
};
static
@@ -4629,11 +4717,13 @@ int rtw89_mac_coex_init(struct rtw89_dev *rtwdev, const struct rtw89_mac_ax_coex
int ret;
rtw89_write8_set(rtwdev, R_AX_GPIO_MUXCFG, B_AX_ENBT);
- rtw89_write8_set(rtwdev, R_AX_BTC_FUNC_EN, B_AX_PTA_WL_TX_EN);
+ if (rtwdev->chip->chip_id != RTL8851B)
+ rtw89_write8_set(rtwdev, R_AX_BTC_FUNC_EN, B_AX_PTA_WL_TX_EN);
rtw89_write8_set(rtwdev, R_AX_BT_COEX_CFG_2 + 1, B_AX_GNT_BT_POLARITY >> 8);
rtw89_write8_set(rtwdev, R_AX_CSR_MODE, B_AX_STATIS_BT_EN | B_AX_WL_ACT_MSK);
rtw89_write8_set(rtwdev, R_AX_CSR_MODE + 2, B_AX_BT_CNT_RST >> 16);
- rtw89_write8_clr(rtwdev, R_AX_TRXPTCL_RESP_0 + 3, B_AX_RSP_CHK_BTCCA >> 24);
+ if (rtwdev->chip->chip_id != RTL8851B)
+ rtw89_write8_clr(rtwdev, R_AX_TRXPTCL_RESP_0 + 3, B_AX_RSP_CHK_BTCCA >> 24);
val16 = rtw89_read16(rtwdev, R_AX_CCA_CFG_0);
val16 = (val16 | B_AX_BTCCA_EN) & ~B_AX_BTCCA_BRK_TXOP_EN;
diff --git a/drivers/net/wireless/realtek/rtw89/mac.h b/drivers/net/wireless/realtek/rtw89/mac.h
index 8064d39..a8d9847 100644
--- a/drivers/net/wireless/realtek/rtw89/mac.h
+++ b/drivers/net/wireless/realtek/rtw89/mac.h
@@ -359,6 +359,7 @@ enum rtw89_mac_c2h_ofld_func {
RTW89_MAC_C2H_FUNC_MACID_PAUSE,
RTW89_MAC_C2H_FUNC_TSF32_TOGL_RPT = 0x6,
RTW89_MAC_C2H_FUNC_SCANOFLD_RSP = 0x9,
+ RTW89_MAC_C2H_FUNC_BCNFLTR_RPT = 0xd,
RTW89_MAC_C2H_FUNC_OFLD_MAX,
};
@@ -815,6 +816,7 @@ struct rtw89_mac_size_set {
const struct rtw89_ple_quota ple_qt47;
const struct rtw89_ple_quota ple_qt58;
const struct rtw89_ple_quota ple_qt_52a_wow;
+ const struct rtw89_ple_quota ple_qt_52b_wow;
};
extern const struct rtw89_mac_size_set rtw89_mac_size;
@@ -1116,6 +1118,7 @@ enum rtw89_mac_xtal_si_offset {
XTAL_SI_XTAL_XMD_4 = 0x26,
#define XTAL_SI_LPS_CAP GENMASK(3, 0)
XTAL_SI_CV = 0x41,
+#define XTAL_SI_ACV_MASK GENMASK(3, 0)
XTAL_SI_LOW_ADDR = 0x62,
#define XTAL_SI_LOW_ADDR_MASK GENMASK(7, 0)
XTAL_SI_CTRL = 0x63,
@@ -1146,7 +1149,7 @@ enum rtw89_mac_xtal_si_offset {
int rtw89_mac_write_xtal_si(struct rtw89_dev *rtwdev, u8 offset, u8 val, u8 mask);
int rtw89_mac_read_xtal_si(struct rtw89_dev *rtwdev, u8 offset, u8 *val);
void rtw89_mac_pkt_drop_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
-u16 rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd);
+int rtw89_mac_dle_buf_req(struct rtw89_dev *rtwdev, u16 buf_len, bool wd, u16 *pkt_id);
int rtw89_mac_set_cpuio(struct rtw89_dev *rtwdev,
struct rtw89_cpuio_ctrl *ctrl_para, bool wd);
int rtw89_mac_typ_fltr_opt(struct rtw89_dev *rtwdev,
diff --git a/drivers/net/wireless/realtek/rtw89/mac80211.c b/drivers/net/wireless/realtek/rtw89/mac80211.c
index 367a7bf..ee4588b 100644
--- a/drivers/net/wireless/realtek/rtw89/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw89/mac80211.c
@@ -23,9 +23,19 @@ static void rtw89_ops_tx(struct ieee80211_hw *hw,
struct rtw89_dev *rtwdev = hw->priv;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct ieee80211_vif *vif = info->control.vif;
+ struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv;
struct ieee80211_sta *sta = control->sta;
+ u32 flags = IEEE80211_SKB_CB(skb)->flags;
int ret, qsel;
+ if (rtwvif->offchan && !(flags & IEEE80211_TX_CTL_TX_OFFCHAN) && sta) {
+ struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv;
+
+ rtw89_debug(rtwdev, RTW89_DBG_TXRX, "ops_tx during offchan\n");
+ skb_queue_tail(&rtwsta->roc_queue, skb);
+ return;
+ }
+
ret = rtw89_core_tx_write(rtwdev, vif, sta, skb, &qsel);
if (ret) {
rtw89_err(rtwdev, "failed to transmit skb: %d\n", ret);
@@ -95,7 +105,8 @@ static int rtw89_ops_config(struct ieee80211_hw *hw, u32 changed)
}
if ((changed & IEEE80211_CONF_CHANGE_IDLE) &&
- (hw->conf.flags & IEEE80211_CONF_IDLE))
+ (hw->conf.flags & IEEE80211_CONF_IDLE) &&
+ !rtwdev->scanning)
rtw89_enter_ips(rtwdev);
mutex_unlock(&rtwdev->mutex);
@@ -114,9 +125,19 @@ static int rtw89_ops_add_interface(struct ieee80211_hw *hw,
vif->addr, vif->type, vif->p2p);
mutex_lock(&rtwdev->mutex);
+
+ rtw89_leave_ips_by_hwflags(rtwdev);
+
+ if (RTW89_CHK_FW_FEATURE(BEACON_FILTER, &rtwdev->fw))
+ vif->driver_flags |= IEEE80211_VIF_BEACON_FILTER |
+ IEEE80211_VIF_SUPPORTS_CQM_RSSI;
+
rtwvif->rtwdev = rtwdev;
+ rtwvif->roc.state = RTW89_ROC_IDLE;
+ rtwvif->offchan = false;
list_add_tail(&rtwvif->list, &rtwdev->rtwvifs_list);
INIT_WORK(&rtwvif->update_beacon_work, rtw89_core_update_beacon_work);
+ INIT_DELAYED_WORK(&rtwvif->roc.roc_work, rtw89_roc_work);
rtw89_leave_ps_mode(rtwdev);
rtw89_traffic_stats_init(rtwdev, &rtwvif->stats);
@@ -163,6 +184,7 @@ static void rtw89_ops_remove_interface(struct ieee80211_hw *hw,
vif->addr, vif->type, vif->p2p);
cancel_work_sync(&rtwvif->update_beacon_work);
+ cancel_delayed_work_sync(&rtwvif->roc.roc_work);
mutex_lock(&rtwdev->mutex);
rtw89_leave_ps_mode(rtwdev);
@@ -170,6 +192,8 @@ static void rtw89_ops_remove_interface(struct ieee80211_hw *hw,
rtw89_mac_remove_vif(rtwdev, rtwvif);
rtw89_core_release_bit_map(rtwdev->hw_port, rtwvif->port);
list_del_init(&rtwvif->list);
+ rtw89_enter_ips_by_hwflags(rtwdev);
+
mutex_unlock(&rtwdev->mutex);
}
@@ -394,7 +418,6 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw,
rtw89_chip_cfg_txpwr_ul_tb_offset(rtwdev, vif);
rtw89_mac_port_update(rtwdev, rtwvif);
rtw89_mac_set_he_obss_narrow_bw_ru(rtwdev, vif);
- rtw89_store_op_chan(rtwdev, true);
} else {
/* Abort ongoing scan if cancel_scan isn't issued
* when disconnected by peer
@@ -425,6 +448,9 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw,
if (changed & BSS_CHANGED_P2P_PS)
rtw89_process_p2p_ps(rtwdev, vif);
+ if (changed & BSS_CHANGED_CQM)
+ rtw89_fw_h2c_set_bcn_fltr_cfg(rtwdev, vif, true);
+
mutex_unlock(&rtwdev->mutex);
}
@@ -795,12 +821,13 @@ static int rtw89_ops_hw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct ieee80211_scan_request *req)
{
struct rtw89_dev *rtwdev = hw->priv;
+ struct rtw89_vif *rtwvif = vif_to_rtwvif_safe(vif);
int ret = 0;
if (!RTW89_CHK_FW_FEATURE(SCAN_OFFLOAD, &rtwdev->fw))
return 1;
- if (rtwdev->scanning)
+ if (rtwdev->scanning || rtwvif->offchan)
return -EBUSY;
mutex_lock(&rtwdev->mutex);
@@ -903,6 +930,63 @@ static void rtw89_ops_unassign_vif_chanctx(struct ieee80211_hw *hw,
mutex_unlock(&rtwdev->mutex);
}
+static int rtw89_ops_remain_on_channel(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct ieee80211_channel *chan,
+ int duration,
+ enum ieee80211_roc_type type)
+{
+ struct rtw89_dev *rtwdev = hw->priv;
+ struct rtw89_vif *rtwvif = vif_to_rtwvif_safe(vif);
+ struct rtw89_roc *roc = &rtwvif->roc;
+
+ if (!vif)
+ return -EINVAL;
+
+ mutex_lock(&rtwdev->mutex);
+
+ if (roc->state != RTW89_ROC_IDLE) {
+ mutex_unlock(&rtwdev->mutex);
+ return -EBUSY;
+ }
+
+ if (rtwdev->scanning)
+ rtw89_hw_scan_abort(rtwdev, vif);
+
+ if (type == IEEE80211_ROC_TYPE_MGMT_TX)
+ roc->state = RTW89_ROC_MGMT;
+ else
+ roc->state = RTW89_ROC_NORMAL;
+
+ roc->duration = duration;
+ roc->chan = *chan;
+ roc->type = type;
+
+ rtw89_roc_start(rtwdev, rtwvif);
+
+ mutex_unlock(&rtwdev->mutex);
+
+ return 0;
+}
+
+static int rtw89_ops_cancel_remain_on_channel(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct rtw89_dev *rtwdev = hw->priv;
+ struct rtw89_vif *rtwvif = vif_to_rtwvif_safe(vif);
+
+ if (!rtwvif)
+ return -EINVAL;
+
+ cancel_delayed_work_sync(&rtwvif->roc.roc_work);
+
+ mutex_lock(&rtwdev->mutex);
+ rtw89_roc_end(rtwdev, rtwvif);
+ mutex_unlock(&rtwdev->mutex);
+
+ return 0;
+}
+
static void rtw89_set_tid_config_iter(void *data, struct ieee80211_sta *sta)
{
struct cfg80211_tid_config *tid_config = data;
@@ -1014,6 +1098,8 @@ const struct ieee80211_ops rtw89_ops = {
.change_chanctx = rtw89_ops_change_chanctx,
.assign_vif_chanctx = rtw89_ops_assign_vif_chanctx,
.unassign_vif_chanctx = rtw89_ops_unassign_vif_chanctx,
+ .remain_on_channel = rtw89_ops_remain_on_channel,
+ .cancel_remain_on_channel = rtw89_ops_cancel_remain_on_channel,
.set_sar_specs = rtw89_ops_set_sar_specs,
.sta_rc_update = rtw89_ops_sta_rc_update,
.set_tid_config = rtw89_ops_set_tid_config,
diff --git a/drivers/net/wireless/realtek/rtw89/pci.c b/drivers/net/wireless/realtek/rtw89/pci.c
index 68f0fed..70b4754 100644
--- a/drivers/net/wireless/realtek/rtw89/pci.c
+++ b/drivers/net/wireless/realtek/rtw89/pci.c
@@ -364,8 +364,11 @@ static void rtw89_pci_tx_status(struct rtw89_dev *rtwdev,
struct rtw89_pci_tx_ring *tx_ring,
struct sk_buff *skb, u8 tx_status)
{
+ struct rtw89_tx_skb_data *skb_data = RTW89_TX_SKB_CB(skb);
struct ieee80211_tx_info *info;
+ rtw89_core_tx_wait_complete(rtwdev, skb_data, tx_status == RTW89_TX_DONE);
+
info = IEEE80211_SKB_CB(skb);
ieee80211_tx_info_clear_status(info);
@@ -1203,6 +1206,7 @@ static int rtw89_pci_txwd_submit(struct rtw89_dev *rtwdev,
struct pci_dev *pdev = rtwpci->pdev;
struct sk_buff *skb = tx_req->skb;
struct rtw89_pci_tx_data *tx_data = RTW89_PCI_TX_SKB_CB(skb);
+ struct rtw89_tx_skb_data *skb_data = RTW89_TX_SKB_CB(skb);
bool en_wd_info = desc_info->en_wd_info;
u32 txwd_len;
u32 txwp_len;
@@ -1218,6 +1222,7 @@ static int rtw89_pci_txwd_submit(struct rtw89_dev *rtwdev,
}
tx_data->dma = dma;
+ rcu_assign_pointer(skb_data->wait, NULL);
txwp_len = sizeof(*txwp_info);
txwd_len = chip->txwd_body_size;
@@ -1912,9 +1917,10 @@ __get_target(struct rtw89_dev *rtwdev, u16 *target, enum rtw89_pcie_phy phy_rate
static int rtw89_pci_autok_x(struct rtw89_dev *rtwdev)
{
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
int ret;
- if (rtwdev->chip->chip_id != RTL8852B)
+ if (chip_id != RTL8852B && chip_id != RTL8851B)
return 0;
ret = rtw89_write16_mdio_mask(rtwdev, RAC_REG_FLD_0, BAC_AUTOK_N_MASK,
@@ -1924,13 +1930,14 @@ static int rtw89_pci_autok_x(struct rtw89_dev *rtwdev)
static int rtw89_pci_auto_refclk_cal(struct rtw89_dev *rtwdev, bool autook_en)
{
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
enum rtw89_pcie_phy phy_rate;
u16 val16, mgn_set, div_set, tar;
u8 val8, bdr_ori;
bool l1_flag = false;
int ret = 0;
- if (rtwdev->chip->chip_id != RTL8852B)
+ if (chip_id != RTL8852B && chip_id != RTL8851B)
return 0;
ret = rtw89_pci_read_config_byte(rtwdev, RTW89_PCIE_PHY_RATE, &val8);
@@ -2107,7 +2114,9 @@ static void rtw89_pci_rxdma_prefth(struct rtw89_dev *rtwdev)
static void rtw89_pci_l1off_pwroff(struct rtw89_dev *rtwdev)
{
- if (rtwdev->chip->chip_id != RTL8852A && rtwdev->chip->chip_id != RTL8852B)
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
+
+ if (chip_id != RTL8852A && chip_id != RTL8852B && chip_id != RTL8851B)
return;
rtw89_write32_clr(rtwdev, R_AX_PCIE_PS_CTRL, B_AX_L1OFF_PWR_OFF_EN);
@@ -2135,7 +2144,9 @@ static u32 rtw89_pci_l2_rxen_lat(struct rtw89_dev *rtwdev)
static void rtw89_pci_aphy_pwrcut(struct rtw89_dev *rtwdev)
{
- if (rtwdev->chip->chip_id != RTL8852A && rtwdev->chip->chip_id != RTL8852B)
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
+
+ if (chip_id != RTL8852A && chip_id != RTL8852B && chip_id != RTL8851B)
return;
rtw89_write32_clr(rtwdev, R_AX_SYS_PW_CTRL, B_AX_PSUS_OFF_CAPC_EN);
@@ -2143,8 +2154,9 @@ static void rtw89_pci_aphy_pwrcut(struct rtw89_dev *rtwdev)
static void rtw89_pci_hci_ldo(struct rtw89_dev *rtwdev)
{
- if (rtwdev->chip->chip_id == RTL8852A ||
- rtwdev->chip->chip_id == RTL8852B) {
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
+
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
rtw89_write32_set(rtwdev, R_AX_SYS_SDIO_CTRL,
B_AX_PCIE_DIS_L2_CTRL_LDO_HCI);
rtw89_write32_clr(rtwdev, R_AX_SYS_SDIO_CTRL,
@@ -2157,7 +2169,9 @@ static void rtw89_pci_hci_ldo(struct rtw89_dev *rtwdev)
static int rtw89_pci_dphy_delay(struct rtw89_dev *rtwdev)
{
- if (rtwdev->chip->chip_id != RTL8852B)
+ enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
+
+ if (chip_id != RTL8852B && chip_id != RTL8851B)
return 0;
return rtw89_write16_mdio_mask(rtwdev, RAC_REG_REV2, BAC_CMU_EN_DLY_MASK,
@@ -3397,7 +3411,7 @@ static void rtw89_pci_clkreq_set(struct rtw89_dev *rtwdev, bool enable)
if (ret)
rtw89_err(rtwdev, "failed to set CLKREQ Delay\n");
- if (chip_id == RTL8852A || chip_id == RTL8852B) {
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
if (enable)
ret = rtw89_pci_config_byte_set(rtwdev,
RTW89_PCIE_L1_CTRL,
@@ -3442,7 +3456,7 @@ static void rtw89_pci_aspm_set(struct rtw89_dev *rtwdev, bool enable)
if (ret)
rtw89_err(rtwdev, "failed to read ASPM Delay\n");
- if (chip_id == RTL8852A || chip_id == RTL8852B) {
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
if (enable)
ret = rtw89_pci_config_byte_set(rtwdev,
RTW89_PCIE_L1_CTRL,
@@ -3522,7 +3536,7 @@ static void rtw89_pci_l1ss_set(struct rtw89_dev *rtwdev, bool enable)
enum rtw89_core_chip_id chip_id = rtwdev->chip->chip_id;
int ret;
- if (chip_id == RTL8852A || chip_id == RTL8852B) {
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
if (enable)
ret = rtw89_pci_config_byte_set(rtwdev,
RTW89_PCIE_TIMER_CTRL,
@@ -3721,7 +3735,7 @@ static int __maybe_unused rtw89_pci_suspend(struct device *dev)
rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6);
rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_R_DIS_PRST);
rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6);
- if (chip_id == RTL8852A || chip_id == RTL8852B) {
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
rtw89_write32_clr(rtwdev, R_AX_SYS_SDIO_CTRL,
B_AX_PCIE_DIS_L2_CTRL_LDO_HCI);
rtw89_write32_set(rtwdev, R_AX_PCIE_INIT_CFG1,
@@ -3755,7 +3769,7 @@ static int __maybe_unused rtw89_pci_resume(struct device *dev)
rtw89_write32_set(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6);
rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_R_DIS_PRST);
rtw89_write32_clr(rtwdev, R_AX_RSV_CTRL, B_AX_WLOCK_1C_BIT6);
- if (chip_id == RTL8852A || chip_id == RTL8852B) {
+ if (chip_id == RTL8852A || chip_id == RTL8852B || chip_id == RTL8851B) {
rtw89_write32_set(rtwdev, R_AX_SYS_SDIO_CTRL,
B_AX_PCIE_DIS_L2_CTRL_LDO_HCI);
rtw89_write32_clr(rtwdev, R_AX_PCIE_INIT_CFG1,
diff --git a/drivers/net/wireless/realtek/rtw89/pci.h b/drivers/net/wireless/realtek/rtw89/pci.h
index 1e19740..0e4bd21 100644
--- a/drivers/net/wireless/realtek/rtw89/pci.h
+++ b/drivers/net/wireless/realtek/rtw89/pci.h
@@ -1004,9 +1004,9 @@ rtw89_pci_rxbd_increase(struct rtw89_pci_rx_ring *rx_ring, u32 cnt)
static inline struct rtw89_pci_tx_data *RTW89_PCI_TX_SKB_CB(struct sk_buff *skb)
{
- struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct rtw89_tx_skb_data *data = RTW89_TX_SKB_CB(skb);
- return (struct rtw89_pci_tx_data *)info->status.status_driver_data;
+ return (struct rtw89_pci_tx_data *)data->hci_priv;
}
static inline struct rtw89_pci_tx_bd_32 *
diff --git a/drivers/net/wireless/realtek/rtw89/phy.c b/drivers/net/wireless/realtek/rtw89/phy.c
index cb0f6cc..c7e9061 100644
--- a/drivers/net/wireless/realtek/rtw89/phy.c
+++ b/drivers/net/wireless/realtek/rtw89/phy.c
@@ -1617,29 +1617,35 @@ static u8 rtw89_channel_to_idx(struct rtw89_dev *rtwdev, u8 band, u8 channel)
s8 rtw89_phy_read_txpwr_limit(struct rtw89_dev *rtwdev, u8 band,
u8 bw, u8 ntx, u8 rs, u8 bf, u8 ch)
{
- const struct rtw89_chip_info *chip = rtwdev->chip;
+ const struct rtw89_rfe_parms *rfe_parms = rtwdev->rfe_parms;
+ const struct rtw89_txpwr_rule_2ghz *rule_2ghz = &rfe_parms->rule_2ghz;
+ const struct rtw89_txpwr_rule_5ghz *rule_5ghz = &rfe_parms->rule_5ghz;
+ const struct rtw89_txpwr_rule_6ghz *rule_6ghz = &rfe_parms->rule_6ghz;
u8 ch_idx = rtw89_channel_to_idx(rtwdev, band, ch);
u8 regd = rtw89_regd_get(rtwdev, band);
s8 lmt = 0, sar;
switch (band) {
case RTW89_BAND_2G:
- lmt = (*chip->txpwr_lmt_2g)[bw][ntx][rs][bf][regd][ch_idx];
- if (!lmt)
- lmt = (*chip->txpwr_lmt_2g)[bw][ntx][rs][bf]
- [RTW89_WW][ch_idx];
+ lmt = (*rule_2ghz->lmt)[bw][ntx][rs][bf][regd][ch_idx];
+ if (lmt)
+ break;
+
+ lmt = (*rule_2ghz->lmt)[bw][ntx][rs][bf][RTW89_WW][ch_idx];
break;
case RTW89_BAND_5G:
- lmt = (*chip->txpwr_lmt_5g)[bw][ntx][rs][bf][regd][ch_idx];
- if (!lmt)
- lmt = (*chip->txpwr_lmt_5g)[bw][ntx][rs][bf]
- [RTW89_WW][ch_idx];
+ lmt = (*rule_5ghz->lmt)[bw][ntx][rs][bf][regd][ch_idx];
+ if (lmt)
+ break;
+
+ lmt = (*rule_5ghz->lmt)[bw][ntx][rs][bf][RTW89_WW][ch_idx];
break;
case RTW89_BAND_6G:
- lmt = (*chip->txpwr_lmt_6g)[bw][ntx][rs][bf][regd][ch_idx];
- if (!lmt)
- lmt = (*chip->txpwr_lmt_6g)[bw][ntx][rs][bf]
- [RTW89_WW][ch_idx];
+ lmt = (*rule_6ghz->lmt)[bw][ntx][rs][bf][regd][ch_idx];
+ if (lmt)
+ break;
+
+ lmt = (*rule_6ghz->lmt)[bw][ntx][rs][bf][RTW89_WW][ch_idx];
break;
default:
rtw89_warn(rtwdev, "unknown band type: %d\n", band);
@@ -1862,29 +1868,35 @@ void rtw89_phy_fill_txpwr_limit(struct rtw89_dev *rtwdev,
static s8 rtw89_phy_read_txpwr_limit_ru(struct rtw89_dev *rtwdev, u8 band,
u8 ru, u8 ntx, u8 ch)
{
- const struct rtw89_chip_info *chip = rtwdev->chip;
+ const struct rtw89_rfe_parms *rfe_parms = rtwdev->rfe_parms;
+ const struct rtw89_txpwr_rule_2ghz *rule_2ghz = &rfe_parms->rule_2ghz;
+ const struct rtw89_txpwr_rule_5ghz *rule_5ghz = &rfe_parms->rule_5ghz;
+ const struct rtw89_txpwr_rule_6ghz *rule_6ghz = &rfe_parms->rule_6ghz;
u8 ch_idx = rtw89_channel_to_idx(rtwdev, band, ch);
u8 regd = rtw89_regd_get(rtwdev, band);
s8 lmt_ru = 0, sar;
switch (band) {
case RTW89_BAND_2G:
- lmt_ru = (*chip->txpwr_lmt_ru_2g)[ru][ntx][regd][ch_idx];
- if (!lmt_ru)
- lmt_ru = (*chip->txpwr_lmt_ru_2g)[ru][ntx]
- [RTW89_WW][ch_idx];
+ lmt_ru = (*rule_2ghz->lmt_ru)[ru][ntx][regd][ch_idx];
+ if (lmt_ru)
+ break;
+
+ lmt_ru = (*rule_2ghz->lmt_ru)[ru][ntx][RTW89_WW][ch_idx];
break;
case RTW89_BAND_5G:
- lmt_ru = (*chip->txpwr_lmt_ru_5g)[ru][ntx][regd][ch_idx];
- if (!lmt_ru)
- lmt_ru = (*chip->txpwr_lmt_ru_5g)[ru][ntx]
- [RTW89_WW][ch_idx];
+ lmt_ru = (*rule_5ghz->lmt_ru)[ru][ntx][regd][ch_idx];
+ if (lmt_ru)
+ break;
+
+ lmt_ru = (*rule_5ghz->lmt_ru)[ru][ntx][RTW89_WW][ch_idx];
break;
case RTW89_BAND_6G:
- lmt_ru = (*chip->txpwr_lmt_ru_6g)[ru][ntx][regd][ch_idx];
- if (!lmt_ru)
- lmt_ru = (*chip->txpwr_lmt_ru_6g)[ru][ntx]
- [RTW89_WW][ch_idx];
+ lmt_ru = (*rule_6ghz->lmt_ru)[ru][ntx][regd][ch_idx];
+ if (lmt_ru)
+ break;
+
+ lmt_ru = (*rule_6ghz->lmt_ru)[ru][ntx][RTW89_WW][ch_idx];
break;
default:
rtw89_warn(rtwdev, "unknown band type: %d\n", band);
@@ -2405,7 +2417,6 @@ static void rtw89_dcfo_comp(struct rtw89_dev *rtwdev, s32 curr_cfo)
bool is_linked = rtwdev->total_sta_assoc > 0;
s32 cfo_avg_312;
s32 dcfo_comp_val;
- u8 dcfo_comp_sft = rtwdev->chip->dcfo_comp_sft;
int sign;
if (!is_linked) {
@@ -2418,8 +2429,8 @@ static void rtw89_dcfo_comp(struct rtw89_dev *rtwdev, s32 curr_cfo)
return;
dcfo_comp_val = rtw89_phy_read32_mask(rtwdev, R_DCFO, B_DCFO);
sign = curr_cfo > 0 ? 1 : -1;
- cfo_avg_312 = (curr_cfo << dcfo_comp_sft) / 5 + sign * dcfo_comp_val;
- rtw89_debug(rtwdev, RTW89_DBG_CFO, "DCFO: avg_cfo=%d\n", cfo_avg_312);
+ cfo_avg_312 = curr_cfo / 625 + sign * dcfo_comp_val;
+ rtw89_debug(rtwdev, RTW89_DBG_CFO, "avg_cfo_312=%d step\n", cfo_avg_312);
if (rtwdev->chip->chip_id == RTL8852A && rtwdev->hal.cv == CHIP_CBV)
cfo_avg_312 = -cfo_avg_312;
rtw89_phy_set_phy_regs(rtwdev, dcfo_comp->addr, dcfo_comp->mask,
@@ -2428,9 +2439,16 @@ static void rtw89_dcfo_comp(struct rtw89_dev *rtwdev, s32 curr_cfo)
static void rtw89_dcfo_comp_init(struct rtw89_dev *rtwdev)
{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
+
rtw89_phy_set_phy_regs(rtwdev, R_DCFO_OPT, B_DCFO_OPT_EN, 1);
rtw89_phy_set_phy_regs(rtwdev, R_DCFO_WEIGHT, B_DCFO_WEIGHT_MSK, 8);
- rtw89_write32_clr(rtwdev, R_AX_PWR_UL_CTRL2, B_AX_PWR_UL_CFO_MASK);
+
+ if (chip->cfo_hw_comp)
+ rtw89_write32_mask(rtwdev, R_AX_PWR_UL_CTRL2,
+ B_AX_PWR_UL_CFO_MASK, 0x6);
+ else
+ rtw89_write32_clr(rtwdev, R_AX_PWR_UL_CTRL2, B_AX_PWR_UL_CFO_MASK);
}
static void rtw89_phy_cfo_init(struct rtw89_dev *rtwdev)
@@ -2500,6 +2518,7 @@ static void rtw89_phy_cfo_crystal_cap_adjust(struct rtw89_dev *rtwdev,
static s32 rtw89_phy_average_cfo_calc(struct rtw89_dev *rtwdev)
{
+ const struct rtw89_chip_info *chip = rtwdev->chip;
struct rtw89_cfo_tracking_info *cfo = &rtwdev->cfo_tracking;
s32 cfo_khz_all = 0;
s32 cfo_cnt_all = 0;
@@ -2516,6 +2535,8 @@ static s32 rtw89_phy_average_cfo_calc(struct rtw89_dev *rtwdev)
cfo_cnt_all += cfo->cfo_cnt[i];
cfo_all_avg = phy_div(cfo_khz_all, cfo_cnt_all);
cfo->pre_cfo_avg[i] = cfo->cfo_avg[i];
+ cfo->dcfo_avg = phy_div(cfo_khz_all << chip->dcfo_comp_sft,
+ cfo_cnt_all);
}
rtw89_debug(rtwdev, RTW89_DBG_CFO,
"CFO track for macid = %d\n", i);
@@ -2642,7 +2663,9 @@ static void rtw89_phy_cfo_dm(struct rtw89_dev *rtwdev)
s32 new_cfo = 0;
bool x_cap_update = false;
u8 pre_x_cap = cfo->crystal_cap;
+ u8 dcfo_comp_sft = rtwdev->chip->dcfo_comp_sft;
+ cfo->dcfo_avg = 0;
rtw89_debug(rtwdev, RTW89_DBG_CFO, "CFO:total_sta_assoc=%d\n",
rtwdev->total_sta_assoc);
if (rtwdev->total_sta_assoc == 0) {
@@ -2684,18 +2707,19 @@ static void rtw89_phy_cfo_dm(struct rtw89_dev *rtwdev)
rtw89_phy_cfo_crystal_cap_adjust(rtwdev, new_cfo);
cfo->cfo_avg_pre = new_cfo;
+ cfo->dcfo_avg_pre = cfo->dcfo_avg;
x_cap_update = cfo->crystal_cap != pre_x_cap;
rtw89_debug(rtwdev, RTW89_DBG_CFO, "Xcap_up=%d\n", x_cap_update);
rtw89_debug(rtwdev, RTW89_DBG_CFO, "Xcap: D:%x C:%x->%x, ofst=%d\n",
cfo->def_x_cap, pre_x_cap, cfo->crystal_cap,
cfo->x_cap_ofst);
if (x_cap_update) {
- if (new_cfo > 0)
- new_cfo -= CFO_SW_COMP_FINE_TUNE;
+ if (cfo->dcfo_avg > 0)
+ cfo->dcfo_avg -= CFO_SW_COMP_FINE_TUNE << dcfo_comp_sft;
else
- new_cfo += CFO_SW_COMP_FINE_TUNE;
+ cfo->dcfo_avg += CFO_SW_COMP_FINE_TUNE << dcfo_comp_sft;
}
- rtw89_dcfo_comp(rtwdev, new_cfo);
+ rtw89_dcfo_comp(rtwdev, cfo->dcfo_avg);
rtw89_phy_cfo_statistics_reset(rtwdev);
}
diff --git a/drivers/net/wireless/realtek/rtw89/ps.c b/drivers/net/wireless/realtek/rtw89/ps.c
index 4049881..fa94335 100644
--- a/drivers/net/wireless/realtek/rtw89/ps.c
+++ b/drivers/net/wireless/realtek/rtw89/ps.c
@@ -114,7 +114,8 @@ void rtw89_leave_ps_mode(struct rtw89_dev *rtwdev)
__rtw89_leave_ps_mode(rtwdev);
}
-void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
+void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ bool ps_mode)
{
lockdep_assert_held(&rtwdev->mutex);
@@ -122,7 +123,8 @@ void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
return;
__rtw89_enter_lps(rtwdev, rtwvif->mac_id);
- __rtw89_enter_ps_mode(rtwdev, rtwvif);
+ if (ps_mode)
+ __rtw89_enter_ps_mode(rtwdev, rtwvif);
}
static void rtw89_leave_lps_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif)
@@ -155,6 +157,9 @@ void rtw89_enter_ips(struct rtw89_dev *rtwdev)
set_bit(RTW89_FLAG_INACTIVE_PS, rtwdev->flags);
+ if (!test_bit(RTW89_FLAG_POWERON, rtwdev->flags))
+ return;
+
rtw89_for_each_rtwvif(rtwdev, rtwvif)
rtw89_mac_vif_deinit(rtwdev, rtwvif);
@@ -166,6 +171,9 @@ void rtw89_leave_ips(struct rtw89_dev *rtwdev)
struct rtw89_vif *rtwvif;
int ret;
+ if (test_bit(RTW89_FLAG_POWERON, rtwdev->flags))
+ return;
+
ret = rtw89_core_start(rtwdev);
if (ret)
rtw89_err(rtwdev, "failed to leave idle state\n");
diff --git a/drivers/net/wireless/realtek/rtw89/ps.h b/drivers/net/wireless/realtek/rtw89/ps.h
index 6ac1f7ea..73c008db 100644
--- a/drivers/net/wireless/realtek/rtw89/ps.h
+++ b/drivers/net/wireless/realtek/rtw89/ps.h
@@ -5,7 +5,8 @@
#ifndef __RTW89_PS_H_
#define __RTW89_PS_H_
-void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
+void rtw89_enter_lps(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif,
+ bool ps_mode);
void rtw89_leave_lps(struct rtw89_dev *rtwdev);
void __rtw89_leave_ps_mode(struct rtw89_dev *rtwdev);
void __rtw89_enter_ps_mode(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif);
@@ -15,4 +16,20 @@ void rtw89_leave_ips(struct rtw89_dev *rtwdev);
void rtw89_set_coex_ctrl_lps(struct rtw89_dev *rtwdev, bool btc_ctrl);
void rtw89_process_p2p_ps(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif);
+static inline void rtw89_leave_ips_by_hwflags(struct rtw89_dev *rtwdev)
+{
+ struct ieee80211_hw *hw = rtwdev->hw;
+
+ if (hw->conf.flags & IEEE80211_CONF_IDLE)
+ rtw89_leave_ips(rtwdev);
+}
+
+static inline void rtw89_enter_ips_by_hwflags(struct rtw89_dev *rtwdev)
+{
+ struct ieee80211_hw *hw = rtwdev->hw;
+
+ if (hw->conf.flags & IEEE80211_CONF_IDLE)
+ rtw89_enter_ips(rtwdev);
+}
+
#endif
diff --git a/drivers/net/wireless/realtek/rtw89/reg.h b/drivers/net/wireless/realtek/rtw89/reg.h
index 37a1777..266e423 100644
--- a/drivers/net/wireless/realtek/rtw89/reg.h
+++ b/drivers/net/wireless/realtek/rtw89/reg.h
@@ -129,6 +129,7 @@
#define R_AX_PLATFORM_ENABLE 0x0088
#define B_AX_AXIDMA_EN BIT(3)
+#define B_AX_APB_WRAP_EN BIT(2)
#define B_AX_WCPU_EN BIT(1)
#define B_AX_PLATFORM_EN BIT(0)
@@ -488,6 +489,7 @@
#define B_AX_DISPATCHER_CLK_EN BIT(18)
#define B_AX_BBRPT_CLK_EN BIT(17)
#define B_AX_MAC_SEC_CLK_EN BIT(16)
+#define B_AX_AXIDMA_CLK_EN BIT(9)
#define PCI_LTR_IDLE_TIMER_1US 0
#define PCI_LTR_IDLE_TIMER_10US 1
@@ -1586,6 +1588,7 @@
#define R_AX_PL_BUF_STATUS 0x9824
#define B_AX_WD_BUF_STAT_DONE BIT(31)
#define B_AX_WD_BUF_STAT_PKTID_MASK GENMASK(11, 0)
+#define S_WD_BUF_STAT_PKTID_INVALID GENMASK(11, 0)
#define R_AX_WD_CPUQ_OP_0 0x9810
#define R_AX_PL_CPUQ_OP_0 0x9830
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.c b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.c
new file mode 100644
index 0000000..0abf797
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.c
@@ -0,0 +1,534 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) 2022-2023 Realtek Corporation
+ */
+
+#include "rtw8851b_rfk_table.h"
+
+static const struct rtw89_reg5_def rtw8851b_dadck_setup_defs[] = {
+ RTW89_DECL_RFK_WM(0xc210, 0x003fc000, 0x80),
+ RTW89_DECL_RFK_WM(0xc224, 0x003fc000, 0x80),
+ RTW89_DECL_RFK_WM(0xc0f8, 0x30000000, 0x3),
+ RTW89_DECL_RFK_WM(0x12b8, BIT(30), 0x1),
+ RTW89_DECL_RFK_WM(0x030c, 0x1f000000, 0x1f),
+ RTW89_DECL_RFK_WM(0x032c, 0xc0000000, 0x0),
+ RTW89_DECL_RFK_WM(0x032c, BIT(22), 0x0),
+ RTW89_DECL_RFK_WM(0x032c, BIT(22), 0x1),
+ RTW89_DECL_RFK_WM(0x032c, BIT(16), 0x0),
+ RTW89_DECL_RFK_WM(0x032c, BIT(20), 0x1),
+ RTW89_DECL_RFK_WM(0x030c, 0x0f000000, 0x3),
+ RTW89_DECL_RFK_WM(0xc0f4, BIT(2), 0x0),
+ RTW89_DECL_RFK_WM(0xc0f4, BIT(4), 0x0),
+ RTW89_DECL_RFK_WM(0xc0f4, BIT(11), 0x1),
+ RTW89_DECL_RFK_WM(0xc0f4, BIT(11), 0x0),
+ RTW89_DECL_RFK_DELAY(1),
+ RTW89_DECL_RFK_WM(0xc0f4, 0x300, 0x1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_dadck_setup_defs);
+
+static const struct rtw89_reg5_def rtw8851b_dadck_post_defs[] = {
+ RTW89_DECL_RFK_WM(0x032c, BIT(16), 0x1),
+ RTW89_DECL_RFK_WM(0x032c, BIT(20), 0x0),
+ RTW89_DECL_RFK_WM(0x030c, 0x1f000000, 0xc),
+ RTW89_DECL_RFK_WM(0x032c, 0xc0000000, 0x1),
+ RTW89_DECL_RFK_WM(0x12b8, BIT(30), 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_dadck_post_defs);
+
+static const struct rtw89_reg5_def rtw8851b_dack_s0_1_defs[] = {
+ RTW89_DECL_RFK_WM(0x12a0, BIT(15), 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x7000, 0x3),
+ RTW89_DECL_RFK_WM(0x12b8, BIT(30), 0x1),
+ RTW89_DECL_RFK_WM(0x030c, BIT(28), 0x1),
+ RTW89_DECL_RFK_WM(0x032c, 0x80000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_dack_s0_1_defs);
+
+static const struct rtw89_reg5_def rtw8851b_dack_s0_2_defs[] = {
+ RTW89_DECL_RFK_WM(0xc004, BIT(0), 0x0),
+ RTW89_DECL_RFK_WM(0x12a0, BIT(15), 0x0),
+ RTW89_DECL_RFK_WM(0x12a0, 0x7000, 0x7),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_dack_s0_2_defs);
+
+static const struct rtw89_reg5_def rtw8851b_dack_manual_off_defs[] = {
+ RTW89_DECL_RFK_WM(0xc0f8, 0x30000000, 0x0),
+ RTW89_DECL_RFK_WM(0xc210, BIT(0), 0x0),
+ RTW89_DECL_RFK_WM(0xc224, BIT(0), 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_dack_manual_off_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_rxclk_80_defs[] = {
+ RTW89_DECL_RFK_WM(0x20fc, 0xffff0000, 0x0101),
+ RTW89_DECL_RFK_WM(0x5670, 0x00002000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00080000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00070000, 0x2),
+ RTW89_DECL_RFK_WM(0x5670, 0x60000000, 0x1),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00000780, 0x8),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00007800, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x0c000000, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d8, 0x000001e0, 0x5),
+ RTW89_DECL_RFK_WM(0xc0c4, 0x003e0000, 0xf),
+ RTW89_DECL_RFK_WM(0xc0ec, 0x00006000, 0x0),
+ RTW89_DECL_RFK_WM(0x12b8, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x0f),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x03),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0001),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0041),
+ RTW89_DECL_RFK_WM(0x20fc, 0xffff0000, 0x1101),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_rxclk_80_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_rxclk_others_defs[] = {
+ RTW89_DECL_RFK_WM(0x20fc, 0xffff0000, 0x0101),
+ RTW89_DECL_RFK_WM(0x5670, 0x00002000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00080000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00070000, 0x2),
+ RTW89_DECL_RFK_WM(0x5670, 0x60000000, 0x0),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00000780, 0x8),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00007800, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x0c000000, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d8, 0x000001e0, 0x5),
+ RTW89_DECL_RFK_WM(0xc0c4, 0x003e0000, 0xf),
+ RTW89_DECL_RFK_WM(0xc0ec, 0x00006000, 0x2),
+ RTW89_DECL_RFK_WM(0x12b8, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x0f),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x03),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0001),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0041),
+ RTW89_DECL_RFK_WM(0x20fc, 0xffff0000, 0x1101),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_rxclk_others_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_txk_2ghz_defs[] = {
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x51, 0x80000, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x51, 0x00800, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x52, 0x00800, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x55, 0x0001f, 0x4),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0xef, 0x00004, 0x1),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x00, 0xffff0, 0x403e),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x00003, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x00070, 0x6),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x1f000, 0x10),
+ RTW89_DECL_RFK_DELAY(1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_txk_2ghz_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_txk_5ghz_defs[] = {
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x60, 0x00007, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x55, 0x0001f, 0x4),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0xef, 0x00004, 0x1),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x00, 0xffff0, 0x403e),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x00003, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x00070, 0x7),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x11, 0x1f000, 0x7),
+ RTW89_DECL_RFK_DELAY(1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_txk_5ghz_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_afebb_restore_defs[] = {
+ RTW89_DECL_RFK_WM(0x12b8, 0x40000000, 0x0),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00010000, 0x1),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00100000, 0x0),
+ RTW89_DECL_RFK_WM(0x20fc, 0x01000000, 0x1),
+ RTW89_DECL_RFK_WM(0x20fc, 0x10000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5670, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x12a0, 0x000ff000, 0x00),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00010000, 0x0),
+ RTW89_DECL_RFK_WM(0x20fc, 0x01000000, 0x0),
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x10005, 0x00001, 0x1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_afebb_restore_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_macbb_defs[] = {
+ RTW89_DECL_RFK_WRF(RF_PATH_A, 0x10005, 0x00001, 0x0),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00010000, 0x1),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00100000, 0x0),
+ RTW89_DECL_RFK_WM(0x20fc, 0x01000000, 0x1),
+ RTW89_DECL_RFK_WM(0x20fc, 0x10000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5670, MASKDWORD, 0xf801fffd),
+ RTW89_DECL_RFK_WM(0x5670, 0x00004000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00008000, 0x1),
+ RTW89_DECL_RFK_WM(0x5670, 0x80000000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00007000, 0x7),
+ RTW89_DECL_RFK_WM(0x5670, 0x00002000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00080000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00070000, 0x3),
+ RTW89_DECL_RFK_WM(0x5670, 0x60000000, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00000780, 0x9),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00007800, 0x1),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x0c000000, 0x0),
+ RTW89_DECL_RFK_WM(0xc0d8, 0x000001e0, 0x3),
+ RTW89_DECL_RFK_WM(0xc0c4, 0x003e0000, 0xa),
+ RTW89_DECL_RFK_WM(0xc0ec, 0x00006000, 0x0),
+ RTW89_DECL_RFK_WM(0xc0e8, 0x00000040, 0x1),
+ RTW89_DECL_RFK_WM(0x12b8, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x1f),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x13),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0001),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0041),
+ RTW89_DECL_RFK_WM(0x20fc, 0x00100000, 0x1),
+ RTW89_DECL_RFK_WM(0x20fc, 0x10000000, 0x1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_macbb_defs);
+
+static const struct rtw89_reg5_def rtw8851b_iqk_bb_afe_defs[] = {
+ RTW89_DECL_RFK_WM(0x5670, 0x00004000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00008000, 0x1),
+ RTW89_DECL_RFK_WM(0x5670, 0x80000000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00007000, 0x7),
+ RTW89_DECL_RFK_WM(0x5670, 0x00002000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00080000, 0x1),
+ RTW89_DECL_RFK_WM(0x12a0, 0x00070000, 0x3),
+ RTW89_DECL_RFK_WM(0x5670, 0x60000000, 0x2),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00000780, 0x9),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x00007800, 0x1),
+ RTW89_DECL_RFK_WM(0xc0d4, 0x0c000000, 0x0),
+ RTW89_DECL_RFK_WM(0xc0d8, 0x000001e0, 0x3),
+ RTW89_DECL_RFK_WM(0xc0c4, 0x003e0000, 0xa),
+ RTW89_DECL_RFK_WM(0xc0ec, 0x00006000, 0x0),
+ RTW89_DECL_RFK_WM(0xc0e8, 0x00000040, 0x1),
+ RTW89_DECL_RFK_WM(0x12b8, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x030c, MASKBYTE3, 0x1f),
+ RTW89_DECL_RFK_WM(0x030c, MASKBYTE3, 0x13),
+ RTW89_DECL_RFK_WM(0x032c, MASKHWORD, 0x0001),
+ RTW89_DECL_RFK_WM(0x032c, MASKHWORD, 0x0041),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_iqk_bb_afe_defs);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_sys_defs[] = {
+ RTW89_DECL_RFK_WM(0x12bc, 0x000ffff0, 0xb5b5),
+ RTW89_DECL_RFK_WM(0x32bc, 0x000ffff0, 0xb5b5),
+ RTW89_DECL_RFK_WM(0x0300, 0xff000000, 0x16),
+ RTW89_DECL_RFK_WM(0x0304, 0x0000ffff, 0x1f19),
+ RTW89_DECL_RFK_WM(0x0308, 0xff000000, 0x1c),
+ RTW89_DECL_RFK_WM(0x0314, 0xffff0000, 0x2041),
+ RTW89_DECL_RFK_WM(0x0318, 0xffffffff, 0x20012041),
+ RTW89_DECL_RFK_WM(0x0324, 0xffff0000, 0x2001),
+ RTW89_DECL_RFK_WM(0x0020, 0x00006000, 0x3),
+ RTW89_DECL_RFK_WM(0x0024, 0x00006000, 0x3),
+ RTW89_DECL_RFK_WM(0x0704, 0xffff0000, 0x601e),
+ RTW89_DECL_RFK_WM(0x2704, 0xffff0000, 0x601e),
+ RTW89_DECL_RFK_WM(0x0700, 0xf0000000, 0x4),
+ RTW89_DECL_RFK_WM(0x2700, 0xf0000000, 0x4),
+ RTW89_DECL_RFK_WM(0x0650, 0x3c000000, 0x0),
+ RTW89_DECL_RFK_WM(0x2650, 0x3c000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_sys_defs);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_sys_a_defs_2g[] = {
+ RTW89_DECL_RFK_WM(0x120c, 0x000000ff, 0x33),
+ RTW89_DECL_RFK_WM(0x12c0, 0x0ff00000, 0x33),
+ RTW89_DECL_RFK_WM(0x58f8, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5814, 0x20000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_sys_a_defs_2g);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_sys_a_defs_5g[] = {
+ RTW89_DECL_RFK_WM(0x120c, 0x000000ff, 0x44),
+ RTW89_DECL_RFK_WM(0x12c0, 0x0ff00000, 0x44),
+ RTW89_DECL_RFK_WM(0x58f8, 0x40000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5814, 0x20000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_sys_a_defs_5g);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_init_txpwr_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x566c, 0x00001000, 0x0),
+ RTW89_DECL_RFK_WM(0x5800, 0xffffffff, 0x003f807f),
+ RTW89_DECL_RFK_WM(0x580c, 0x0000007f, 0x40),
+ RTW89_DECL_RFK_WM(0x580c, 0x0fffff00, 0x00040),
+ RTW89_DECL_RFK_WM(0x5810, 0xffffffff, 0x59010000),
+ RTW89_DECL_RFK_WM(0x5814, 0x01ffffff, 0x026d000),
+ RTW89_DECL_RFK_WM(0x5814, 0xf8000000, 0x00),
+ RTW89_DECL_RFK_WM(0x5818, 0x00ffffff, 0x2c18e8),
+ RTW89_DECL_RFK_WM(0x5818, 0x07000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5818, 0xf0000000, 0x0),
+ RTW89_DECL_RFK_WM(0x581c, 0x3fffffff, 0x3dc80280),
+ RTW89_DECL_RFK_WM(0x5820, 0xffffffff, 0x00000080),
+ RTW89_DECL_RFK_WM(0x58e8, 0x0000003f, 0x04),
+ RTW89_DECL_RFK_WM(0x580c, 0x10000000, 0x1),
+ RTW89_DECL_RFK_WM(0x580c, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5834, 0x3fffffff, 0x000115f2),
+ RTW89_DECL_RFK_WM(0x5838, 0x7fffffff, 0x0000121),
+ RTW89_DECL_RFK_WM(0x5854, 0x3fffffff, 0x000115f2),
+ RTW89_DECL_RFK_WM(0x5858, 0x7fffffff, 0x0000121),
+ RTW89_DECL_RFK_WM(0x5860, 0x80000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5864, 0x07ffffff, 0x00801ff),
+ RTW89_DECL_RFK_WM(0x5898, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x589c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x58a4, 0x000000ff, 0x16),
+ RTW89_DECL_RFK_WM(0x58b0, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x58b4, 0x7fffffff, 0x0a002000),
+ RTW89_DECL_RFK_WM(0x58b8, 0x7fffffff, 0x00007628),
+ RTW89_DECL_RFK_WM(0x58bc, 0x07ffffff, 0x7a7807f),
+ RTW89_DECL_RFK_WM(0x58c0, 0xfffe0000, 0x003f),
+ RTW89_DECL_RFK_WM(0x58c4, 0xffffffff, 0x0003ffff),
+ RTW89_DECL_RFK_WM(0x58c8, 0x00ffffff, 0x000000),
+ RTW89_DECL_RFK_WM(0x58c8, 0xf0000000, 0x0),
+ RTW89_DECL_RFK_WM(0x58cc, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x58d0, 0x07ffffff, 0x2008101),
+ RTW89_DECL_RFK_WM(0x58d4, 0x000000ff, 0x00),
+ RTW89_DECL_RFK_WM(0x58d4, 0x0003fe00, 0x0ff),
+ RTW89_DECL_RFK_WM(0x58d4, 0x07fc0000, 0x100),
+ RTW89_DECL_RFK_WM(0x58d8, 0xffffffff, 0x8008016c),
+ RTW89_DECL_RFK_WM(0x58dc, 0x0001ffff, 0x0807f),
+ RTW89_DECL_RFK_WM(0x58dc, 0xfff00000, 0x800),
+ RTW89_DECL_RFK_WM(0x58f0, 0x0003ffff, 0x001ff),
+ RTW89_DECL_RFK_WM(0x58f4, 0x000fffff, 0x00000),
+ RTW89_DECL_RFK_WM(0x58f8, 0x000fffff, 0x00000),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_init_txpwr_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_init_txpwr_he_tb_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x58a0, MASKDWORD, 0x000000fe),
+ RTW89_DECL_RFK_WM(0x58e4, 0x0000007f, 0x1f),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_init_txpwr_he_tb_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_dck_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x580c, 0x0fff0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5814, 0x00001000, 0x1),
+ RTW89_DECL_RFK_WM(0x5814, 0x00002000, 0x1),
+ RTW89_DECL_RFK_WM(0x5814, 0x00004000, 0x1),
+ RTW89_DECL_RFK_WM(0x5814, 0x00038000, 0x3),
+ RTW89_DECL_RFK_WM(0x5814, 0x003c0000, 0x5),
+ RTW89_DECL_RFK_WM(0x5814, 0x18000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_dck_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_dac_gain_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x58b0, 0x00000fff, 0x000),
+ RTW89_DECL_RFK_WM(0x5a00, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a04, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a08, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a0c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a10, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a14, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a18, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a1c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a20, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a24, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a28, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a2c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a30, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a34, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a38, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a3c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a40, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a44, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a48, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a4c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a50, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a54, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a58, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a5c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a60, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a64, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a68, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a6c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a70, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a74, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a78, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a7c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a80, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a84, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a88, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a8c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a90, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a94, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a98, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5a9c, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5aa0, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5aa4, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5aa8, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5aac, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5ab0, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5ab4, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5ab8, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5abc, MASKDWORD, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5ac0, MASKDWORD, 0x00000000),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_dac_gain_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_slope_a_defs_2g[] = {
+ RTW89_DECL_RFK_WM(0x5608, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x560c, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x5610, 0x07ffffff, 0x0200e08),
+ RTW89_DECL_RFK_WM(0x5614, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x5618, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x561c, 0x000001ff, 0x007),
+ RTW89_DECL_RFK_WM(0x561c, 0xffff0000, 0x0808),
+ RTW89_DECL_RFK_WM(0x5620, 0xffffffff, 0x08080808),
+ RTW89_DECL_RFK_WM(0x5624, 0xffffffff, 0x08080808),
+ RTW89_DECL_RFK_WM(0x5628, 0xffffffff, 0x08080808),
+ RTW89_DECL_RFK_WM(0x562c, 0x0000ffff, 0x0808),
+ RTW89_DECL_RFK_WM(0x581c, 0x00100000, 0x1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_slope_a_defs_2g);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_slope_a_defs_5g[] = {
+ RTW89_DECL_RFK_WM(0x5608, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x560c, 0x07ffffff, 0x0341a08),
+ RTW89_DECL_RFK_WM(0x5610, 0x07ffffff, 0x0201417),
+ RTW89_DECL_RFK_WM(0x5614, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x5618, 0x07ffffff, 0x0201008),
+ RTW89_DECL_RFK_WM(0x561c, 0x000001ff, 0x008),
+ RTW89_DECL_RFK_WM(0x561c, 0xffff0000, 0x0808),
+ RTW89_DECL_RFK_WM(0x5620, 0xffffffff, 0x0e0e0808),
+ RTW89_DECL_RFK_WM(0x5624, 0xffffffff, 0x08080d18),
+ RTW89_DECL_RFK_WM(0x5628, 0xffffffff, 0x08080808),
+ RTW89_DECL_RFK_WM(0x562c, 0x0000ffff, 0x0808),
+ RTW89_DECL_RFK_WM(0x581c, 0x00100000, 0x1),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_slope_a_defs_5g);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_align_a_2g_defs[] = {
+ RTW89_DECL_RFK_WM(0x5604, 0x80000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5600, 0x3fffffff, 0x000000),
+ RTW89_DECL_RFK_WM(0x5604, 0x003fffff, 0x2d2400),
+ RTW89_DECL_RFK_WM(0x5630, 0x3fffffff, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5634, 0x000003ff, 0x000),
+ RTW89_DECL_RFK_WM(0x5634, 0x000ffc00, 0x000),
+ RTW89_DECL_RFK_WM(0x5634, 0x3ff00000, 0x3fa),
+ RTW89_DECL_RFK_WM(0x5638, 0x000003ff, 0x02e),
+ RTW89_DECL_RFK_WM(0x5638, 0x000ffc00, 0x09c),
+ RTW89_DECL_RFK_WM(0x563c, 0x3fffffff, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5640, 0x3fffffff, 0x3fb00000),
+ RTW89_DECL_RFK_WM(0x5644, 0x000003ff, 0x02f),
+ RTW89_DECL_RFK_WM(0x5644, 0x000ffc00, 0x09c),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_align_a_2g_defs);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_align_a_5g_defs[] = {
+ RTW89_DECL_RFK_WM(0x5604, 0x80000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5600, 0x3fffffff, 0x000000),
+ RTW89_DECL_RFK_WM(0x5604, 0x003fffff, 0x3b2d24),
+ RTW89_DECL_RFK_WM(0x5630, 0x3fffffff, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5634, 0x000003ff, 0x000),
+ RTW89_DECL_RFK_WM(0x5634, 0x000ffc00, 0x3cb),
+ RTW89_DECL_RFK_WM(0x5634, 0x3ff00000, 0x030),
+ RTW89_DECL_RFK_WM(0x5638, 0x000003ff, 0x73),
+ RTW89_DECL_RFK_WM(0x5638, 0x000ffc00, 0xd4),
+ RTW89_DECL_RFK_WM(0x563c, 0x3fffffff, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5640, 0x3fffffff, 0x00000000),
+ RTW89_DECL_RFK_WM(0x5644, 0x000fffff, 0x00000),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_align_a_5g_defs);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_slope_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x5820, 0x80000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5818, 0x10000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5814, 0x00000800, 0x1),
+ RTW89_DECL_RFK_WM(0x581c, 0x20000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5820, 0x0000f000, 0xf),
+ RTW89_DECL_RFK_WM(0x581c, 0x000003ff, 0x280),
+ RTW89_DECL_RFK_WM(0x581c, 0x000ffc00, 0x200),
+ RTW89_DECL_RFK_WM(0x58b8, 0x007f0000, 0x00),
+ RTW89_DECL_RFK_WM(0x58b8, 0x7f000000, 0x00),
+ RTW89_DECL_RFK_WM(0x58b4, 0x7f000000, 0x0a),
+ RTW89_DECL_RFK_WM(0x58b8, 0x0000007f, 0x28),
+ RTW89_DECL_RFK_WM(0x58b8, 0x00007f00, 0x76),
+ RTW89_DECL_RFK_WM(0x5810, 0x20000000, 0x0),
+ RTW89_DECL_RFK_WM(0x580c, 0x10000000, 0x1),
+ RTW89_DECL_RFK_WM(0x580c, 0x40000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5834, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x5834, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5838, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5838, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x5854, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x5854, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5858, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5858, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x5824, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x5824, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5828, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5828, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x582c, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x582c, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5830, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5830, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x583c, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x583c, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5840, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5840, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x5844, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x5844, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5848, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5848, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x584c, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x584c, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5850, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5850, 0x003ff000, 0x000),
+ RTW89_DECL_RFK_WM(0x585c, 0x0003ffff, 0x115f2),
+ RTW89_DECL_RFK_WM(0x585c, 0x3ffc0000, 0x000),
+ RTW89_DECL_RFK_WM(0x5860, 0x00000fff, 0x121),
+ RTW89_DECL_RFK_WM(0x5860, 0x003ff000, 0x000),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_slope_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_track_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x5820, 0x80000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5818, 0x10000000, 0x0),
+ RTW89_DECL_RFK_WM(0x5814, 0x00000800, 0x0),
+ RTW89_DECL_RFK_WM(0x581c, 0x20000000, 0x1),
+ RTW89_DECL_RFK_WM(0x5864, 0x000003ff, 0x1ff),
+ RTW89_DECL_RFK_WM(0x5864, 0x000ffc00, 0x200),
+ RTW89_DECL_RFK_WM(0x5820, 0x00000fff, 0x080),
+ RTW89_DECL_RFK_WM(0x5814, 0x01000000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_track_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_tssi_mv_avg_defs_a[] = {
+ RTW89_DECL_RFK_WM(0x58e4, 0x00003800, 0x1),
+ RTW89_DECL_RFK_WM(0x58e4, 0x00004000, 0x0),
+ RTW89_DECL_RFK_WM(0x58e4, 0x00008000, 0x1),
+ RTW89_DECL_RFK_WM(0x58e4, 0x000f0000, 0x0),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_tssi_mv_avg_defs_a);
+
+static const struct rtw89_reg5_def rtw8851b_nctl_post_defs[] = {
+ RTW89_DECL_RFK_WM(0x5864, 0x18000000, 0x3),
+ RTW89_DECL_RFK_WM(0x7864, 0x18000000, 0x3),
+ RTW89_DECL_RFK_WM(0x030c, 0xff000000, 0x13),
+ RTW89_DECL_RFK_WM(0x032c, 0xffff0000, 0x0041),
+ RTW89_DECL_RFK_WM(0x12b8, 0x10000000, 0x1),
+ RTW89_DECL_RFK_WM(0x2008, 0x01ffffff, 0x00fffff),
+ RTW89_DECL_RFK_WM(0x0c60, 0x00000003, 0x3),
+ RTW89_DECL_RFK_WM(0x0c6c, 0x00000001, 0x1),
+ RTW89_DECL_RFK_WM(0x58ac, 0x08000000, 0x1),
+ RTW89_DECL_RFK_WM(0x78ac, 0x08000000, 0x1),
+ RTW89_DECL_RFK_WM(0x0730, 0x00003800, 0x7),
+ RTW89_DECL_RFK_WM(0x2730, 0x00003800, 0x7),
+ RTW89_DECL_RFK_WM(0x0c7c, 0x00e00000, 0x1),
+ RTW89_DECL_RFK_WM(0x58c0, 0x0001ffff, 0x00000),
+ RTW89_DECL_RFK_WM(0x78c0, 0x0001ffff, 0x00000),
+ RTW89_DECL_RFK_WM(0x58fc, 0x3f000000, 0x00),
+ RTW89_DECL_RFK_WM(0x78fc, 0x3f000000, 0x00),
+};
+
+RTW89_DECLARE_RFK_TBL(rtw8851b_nctl_post_defs);
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.h b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.h
new file mode 100644
index 0000000..febfbec
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk_table.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/* Copyright(c) 2022-2023 Realtek Corporation
+ */
+
+#ifndef __RTW89_8851B_RFK_TABLE_H__
+#define __RTW89_8851B_RFK_TABLE_H__
+
+#include "phy.h"
+
+extern const struct rtw89_rfk_tbl rtw8851b_dadck_setup_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_dadck_post_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_dack_s0_1_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_dack_s0_2_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_dack_manual_off_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_rxclk_80_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_rxclk_others_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_txk_2ghz_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_txk_5ghz_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_afebb_restore_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_bb_afe_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_iqk_macbb_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_sys_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_sys_a_defs_2g_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_sys_a_defs_5g_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_init_txpwr_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_init_txpwr_he_tb_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_dck_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_dac_gain_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_slope_a_defs_2g_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_slope_a_defs_5g_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_align_a_2g_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_align_a_5g_defs_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_slope_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_track_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_tssi_mv_avg_defs_a_tbl;
+extern const struct rtw89_rfk_tbl rtw8851b_nctl_post_defs_tbl;
+
+#endif
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b_table.c b/drivers/net/wireless/realtek/rtw89/rtw8851b_table.c
new file mode 100644
index 0000000..bb72414
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw89/rtw8851b_table.c
@@ -0,0 +1,14824 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright(c) 2022-2023 Realtek Corporation
+ */
+
+#include "phy.h"
+#include "reg.h"
+#include "rtw8851b_table.h"
+
+static const struct rtw89_reg2_def rtw89_8851b_phy_bb_regs[] = {
+ {0x704, 0x601E0500},
+ {0x4000, 0x00000000},
+ {0x4004, 0xCA014000},
+ {0x4008, 0xC751D4F0},
+ {0x400C, 0x44511475},
+ {0x4010, 0x00000000},
+ {0x4014, 0x00000000},
+ {0x47BC, 0x00000380},
+ {0x4018, 0x4F4C084B},
+ {0x401C, 0x084A4E52},
+ {0x4020, 0x4D504E4B},
+ {0x4024, 0x4F4C0849},
+ {0x4028, 0x08484C50},
+ {0x402C, 0x4C50504C},
+ {0x4030, 0x5454084A},
+ {0x4034, 0x084B5654},
+ {0x4038, 0x6A6C605A},
+ {0x403C, 0x4C4C084C},
+ {0x4040, 0x084B4E4D},
+ {0x4044, 0x4E4C4B4B},
+ {0x4048, 0x4B4B084A},
+ {0x404C, 0x084A4E4C},
+ {0x4050, 0x514F4C4A},
+ {0x4054, 0x524E084A},
+ {0x4058, 0x084A5154},
+ {0x405C, 0x53555554},
+ {0x4060, 0x45450845},
+ {0x4064, 0x08454144},
+ {0x4068, 0x40434445},
+ {0x406C, 0x44450845},
+ {0x4070, 0x08444043},
+ {0x4074, 0x42434444},
+ {0x4078, 0x46450844},
+ {0x407C, 0x08444843},
+ {0x4080, 0x4B4E4A47},
+ {0x4084, 0x4F4C084B},
+ {0x4088, 0x084A4E52},
+ {0x408C, 0x4D504E4B},
+ {0x4090, 0x4F4C0849},
+ {0x4094, 0x08484C50},
+ {0x4098, 0x4C50504C},
+ {0x409C, 0x5454084A},
+ {0x40A0, 0x084B5654},
+ {0x40A4, 0x6A6C605A},
+ {0x40A8, 0x4C4C084C},
+ {0x40AC, 0x084B4E4D},
+ {0x40B0, 0x4E4C4B4B},
+ {0x40B4, 0x4B4B084A},
+ {0x40B8, 0x084A4E4C},
+ {0x40BC, 0x514F4C4A},
+ {0x40C0, 0x524E084A},
+ {0x40C4, 0x084A5154},
+ {0x40C8, 0x53555554},
+ {0x40CC, 0x45450845},
+ {0x40D0, 0x08454144},
+ {0x40D4, 0x40434445},
+ {0x40D8, 0x44450845},
+ {0x40DC, 0x08444043},
+ {0x40E0, 0x42434444},
+ {0x40E4, 0x46450844},
+ {0x40E8, 0x08444843},
+ {0x40EC, 0x4B4E4A47},
+ {0x40F0, 0x00000000},
+ {0x4A38, 0x00000000},
+ {0x40F4, 0x00000006},
+ {0x40F8, 0x00000000},
+ {0x40FC, 0x8C30C30C},
+ {0x4100, 0x4C30C30C},
+ {0x4104, 0x0C30C30C},
+ {0x4108, 0x0C30C30C},
+ {0x410C, 0x0C30C30C},
+ {0x4110, 0x0C30C30C},
+ {0x4114, 0x28A28A28},
+ {0x4118, 0x28A28A28},
+ {0x411C, 0x28A28A28},
+ {0x4120, 0x28A28A28},
+ {0x4124, 0x28A28A28},
+ {0x4128, 0x28A28A28},
+ {0x412C, 0x06666666},
+ {0x4130, 0x33333333},
+ {0x4134, 0x33333333},
+ {0x4138, 0x33333333},
+ {0x413C, 0x00000031},
+ {0x4140, 0x5100600A},
+ {0x4144, 0x18363113},
+ {0x4148, 0x1D976DDC},
+ {0x414C, 0x1C072DD7},
+ {0x4150, 0x1127CDF4},
+ {0x4154, 0x1E37BDF1},
+ {0x4158, 0x1FB7F1D6},
+ {0x415C, 0x1EA7DDF9},
+ {0x4160, 0x1FE445DD},
+ {0x4164, 0x1F97F1FE},
+ {0x4168, 0x1FF781ED},
+ {0x416C, 0x1FA7F5FE},
+ {0x4170, 0x1E07B913},
+ {0x4174, 0x1FD7FDFF},
+ {0x4178, 0x1E17B9FA},
+ {0x417C, 0x19A66914},
+ {0x4180, 0x10F65598},
+ {0x4184, 0x14A5A111},
+ {0x4188, 0x1D3765DB},
+ {0x418C, 0x17C685CA},
+ {0x4190, 0x1107C5F3},
+ {0x4194, 0x1B5785EB},
+ {0x4198, 0x1F97ED8F},
+ {0x419C, 0x1BC7A5F3},
+ {0x41A0, 0x1FE43595},
+ {0x41A4, 0x1EB7D9FC},
+ {0x41A8, 0x1FE65DBE},
+ {0x41AC, 0x1EC7D9FC},
+ {0x41B0, 0x1976FCFF},
+ {0x41B4, 0x1F77F5FF},
+ {0x41B8, 0x1976FDEC},
+ {0x41BC, 0x198664EF},
+ {0x41C0, 0x11062D93},
+ {0x41C4, 0x10C4E910},
+ {0x41C8, 0x1CA759DB},
+ {0x41CC, 0x1335A9B5},
+ {0x41D0, 0x1097B9F3},
+ {0x41D4, 0x17B72DE1},
+ {0x41D8, 0x1F67ED42},
+ {0x41DC, 0x18074DE9},
+ {0x41E0, 0x1FD40547},
+ {0x41E4, 0x1D57ADF9},
+ {0x41E8, 0x1FE52182},
+ {0x41EC, 0x1D67B1F9},
+ {0x41F0, 0x14860CE1},
+ {0x41F4, 0x1EC7E9FE},
+ {0x41F8, 0x14860DD6},
+ {0x41FC, 0x195664C7},
+ {0x4200, 0x0005E58A},
+ {0x4204, 0x00000000},
+ {0x4208, 0x00000000},
+ {0x420C, 0x7A000000},
+ {0x4210, 0x0F9F3D7A},
+ {0x4214, 0x0040817C},
+ {0x4218, 0x00E10204},
+ {0x421C, 0x227D94CD},
+ {0x4220, 0x08028A28},
+ {0x4224, 0x00000200},
+ {0x4228, 0x04688000},
+ {0x47C0, 0x00000001},
+ {0x4A48, 0x00000002},
+ {0x4B04, 0x00000000},
+ {0x4B08, 0x00000000},
+ {0x422C, 0x0060B002},
+ {0x4230, 0x9A8249A8},
+ {0x4234, 0x26A1469E},
+ {0x4238, 0x2099A824},
+ {0x423C, 0x2359461C},
+ {0x4240, 0x1631A675},
+ {0x4244, 0x2C6B1D63},
+ {0x4248, 0x0000000E},
+ {0x424C, 0x00000001},
+ {0x4250, 0x00000001},
+ {0x4254, 0x00000000},
+ {0x4258, 0x00000000},
+ {0x425C, 0x00000000},
+ {0x4260, 0x0020000C},
+ {0x4A30, 0x00000000},
+ {0x4264, 0x00000000},
+ {0x4268, 0x00000000},
+ {0x426C, 0x0418317C},
+ {0x4270, 0x2B33135C},
+ {0x4274, 0x00000002},
+ {0x4278, 0x00000000},
+ {0x427C, 0x00000000},
+ {0x4280, 0x00000000},
+ {0x4284, 0x00000000},
+ {0x4288, 0x00000000},
+ {0x428C, 0x00000000},
+ {0x4290, 0x00000000},
+ {0x4294, 0x00000000},
+ {0x4298, 0x00000000},
+ {0x429C, 0x84026000},
+ {0x42A0, 0x0051AC20},
+ {0x4A24, 0x0010C040},
+ {0x42A4, 0x02024008},
+ {0x42A8, 0x00000000},
+ {0x42AC, 0x00000000},
+ {0x42B0, 0x22CE803C},
+ {0x42B4, 0xD8000000},
+ {0x42B8, 0x596FD67E},
+ {0x42BC, 0x7D67D67D},
+ {0x42C0, 0x7D67D65B},
+ {0x42C4, 0x28029F59},
+ {0x42C8, 0x00280280},
+ {0x4AF4, 0x00000000},
+ {0x42CC, 0x00000000},
+ {0x42D0, 0x00000000},
+ {0x42D4, 0x00000003},
+ {0x4AF8, 0x00280000},
+ {0x42D8, 0x00000001},
+ {0x42DC, 0x69AEC800},
+ {0x42E0, 0x8B4CD3D1},
+ {0x42E4, 0xC514534F},
+ {0x42E8, 0x85145145},
+ {0x42EC, 0x45145145},
+ {0x42F0, 0x05145145},
+ {0x42F4, 0x05145145},
+ {0x42F8, 0x05145145},
+ {0x42FC, 0x17659145},
+ {0x4300, 0x176DD5D9},
+ {0x4304, 0x0F65765B},
+ {0x4308, 0x0F3CF3CF},
+ {0x430C, 0x0F3CF3CF},
+ {0x4310, 0x0F3CF3CF},
+ {0x4314, 0x0F3CF3CF},
+ {0x4318, 0x0F3CF3CF},
+ {0x431C, 0x0F3CF3CF},
+ {0x4320, 0x0F3CF3CF},
+ {0x4324, 0x0F44F351},
+ {0x4328, 0x192D7547},
+ {0x432C, 0x0F5CF5CF},
+ {0x4330, 0x051593D9},
+ {0x4334, 0x05145145},
+ {0x4338, 0x05145145},
+ {0x433C, 0x05145145},
+ {0x4340, 0x05145145},
+ {0x4344, 0x05145145},
+ {0x4348, 0x19545145},
+ {0x434C, 0x1B65B5DB},
+ {0x4350, 0x1965965B},
+ {0x4354, 0x0F3CF3CF},
+ {0x4358, 0x0F3CF3CF},
+ {0x435C, 0x0F3CF1CF},
+ {0x4360, 0x0F3CF3CF},
+ {0x4364, 0x0F3CF3CF},
+ {0x4368, 0x0F3CF3CF},
+ {0x436C, 0x0F3CF3CF},
+ {0x4370, 0x0934D2CF},
+ {0x4374, 0x112CB3CF},
+ {0x4378, 0x9777A777},
+ {0x437C, 0xBB7BAC95},
+ {0x4380, 0xB667B889},
+ {0x4384, 0x7B9B8899},
+ {0x4388, 0x7A5567C8},
+ {0x438C, 0x2278CCCC},
+ {0x4390, 0x7C222222},
+ {0x4394, 0x0000029B},
+ {0x4398, 0x001CCCCC},
+ {0x4AAC, 0xCCCCC88C},
+ {0x4AB0, 0x0000AACC},
+ {0x439C, 0x00000000},
+ {0x43A0, 0x00000008},
+ {0x43A4, 0x00000000},
+ {0x43A8, 0x00000000},
+ {0x43AC, 0x00000000},
+ {0x43B0, 0x10000000},
+ {0x43B4, 0x00401001},
+ {0x43B8, 0x00061003},
+ {0x43BC, 0x000024D8},
+ {0x43C0, 0x00000000},
+ {0x43C4, 0x10000020},
+ {0x43C8, 0x20000200},
+ {0x43CC, 0x00000000},
+ {0x43D0, 0x04000000},
+ {0x43D4, 0x44000100},
+ {0x43D8, 0x60804060},
+ {0x43DC, 0x44204210},
+ {0x43E0, 0x82108082},
+ {0x43E4, 0x82108402},
+ {0x43E8, 0xC8082108},
+ {0x43EC, 0xC8202084},
+ {0x43F0, 0x44208208},
+ {0x43F4, 0x84108204},
+ {0x43F8, 0xD0108104},
+ {0x43FC, 0xF8210108},
+ {0x4400, 0x6431E930},
+ {0x4404, 0x02309468},
+ {0x4408, 0x10C61C22},
+ {0x440C, 0x02109469},
+ {0x4410, 0x10C61C22},
+ {0x4414, 0x00041049},
+ {0x4A4C, 0x00060581},
+ {0x4418, 0x00000000},
+ {0x441C, 0x00000000},
+ {0x4420, 0xEC000000},
+ {0x4424, 0xB0200020},
+ {0x4428, 0x00001FF0},
+ {0x4AC8, 0x00000001},
+ {0x4B0C, 0x00000000},
+ {0x4CDC, 0x00000000},
+ {0x442C, 0x00000000},
+ {0x4430, 0x00000000},
+ {0x4434, 0x00000000},
+ {0x4438, 0x00000000},
+ {0x443C, 0x190642D0},
+ {0x4440, 0xA80668A0},
+ {0x4444, 0x60900820},
+ {0x4448, 0x9F28518C},
+ {0x444C, 0x32488A62},
+ {0x4450, 0x9C6E36DC},
+ {0x4454, 0x0000F52B},
+ {0x4A34, 0x00000007},
+ {0x4CE0, 0x68120000},
+ {0x4CE4, 0x1A0681E0},
+ {0x4CE8, 0x94060180},
+ {0x4CEC, 0x000603FF},
+ {0x4CF0, 0xA0502000},
+ {0x4CF4, 0x00001000},
+ {0x4D00, 0x00000044},
+ {0x4B14, 0x00000000},
+ {0x4458, 0x00000000},
+ {0x445C, 0x4801442E},
+ {0x4460, 0x0051A0FA},
+ {0x4B18, 0x0000011F},
+ {0x4B1C, 0x0000011F},
+ {0x4464, 0x00000000},
+ {0x4468, 0x00000000},
+ {0x446C, 0x00000000},
+ {0x4470, 0x00000000},
+ {0x4474, 0x00000000},
+ {0x4478, 0x00000000},
+ {0x447C, 0x00000000},
+ {0x4480, 0x2A0A6040},
+ {0x4484, 0x0A0A6829},
+ {0x4488, 0x00000004},
+ {0x448C, 0x00000000},
+ {0x4490, 0x80000000},
+ {0x4494, 0x10000000},
+ {0x4498, 0xE0000000},
+ {0x4A28, 0x000ED877},
+ {0x4AB4, 0x00000000},
+ {0x4B20, 0x00000000},
+ {0x4B24, 0x00000000},
+ {0x4B28, 0x00000000},
+ {0x4B2C, 0x00000000},
+ {0x449C, 0x0000001E},
+ {0x44A0, 0x02B2C394},
+ {0x44A4, 0x00000400},
+ {0x4A2C, 0x0050240E},
+ {0x4B30, 0x7FFFFD20},
+ {0x4B34, 0x920823FF},
+ {0x4B38, 0x7FFFFFFF},
+ {0x4B3C, 0x01773773},
+ {0x44A8, 0x00000001},
+ {0x44AC, 0x000190C0},
+ {0x44B0, 0x00000000},
+ {0x44B4, 0x00000000},
+ {0x44B8, 0x00000000},
+ {0x44BC, 0x00000000},
+ {0x44C0, 0x00000000},
+ {0x44C4, 0x00000000},
+ {0x44C8, 0x00000000},
+ {0x44CC, 0x00000000},
+ {0x44D0, 0x00000000},
+ {0x44D4, 0x00000000},
+ {0x44D8, 0x00000000},
+ {0x44DC, 0x00000000},
+ {0x44E0, 0x00000000},
+ {0x44E4, 0x00000000},
+ {0x44E8, 0x00000000},
+ {0x44EC, 0x00000000},
+ {0x44F0, 0x00000000},
+ {0x44F4, 0x00000000},
+ {0x44F8, 0x00000000},
+ {0x44FC, 0x00000000},
+ {0x4500, 0x00000000},
+ {0x4504, 0x00000000},
+ {0x4508, 0x00000000},
+ {0x450C, 0x00000000},
+ {0x4510, 0x00000000},
+ {0x4514, 0x00000000},
+ {0x4518, 0x00000000},
+ {0x451C, 0x00000000},
+ {0x4520, 0x00000000},
+ {0x4524, 0x00000000},
+ {0x4528, 0x00000000},
+ {0x452C, 0x00000000},
+ {0x4530, 0x4E830171},
+ {0x4534, 0x00000870},
+ {0x4538, 0x000000FF},
+ {0x453C, 0x00000000},
+ {0x4540, 0x00000000},
+ {0x4544, 0x00000000},
+ {0x4548, 0x00000000},
+ {0x454C, 0x00000000},
+ {0x4550, 0x00000000},
+ {0x4554, 0x00000000},
+ {0x4558, 0x00000000},
+ {0x455C, 0x00000000},
+ {0x4560, 0x40000000},
+ {0x4564, 0x40000000},
+ {0x4568, 0x00000000},
+ {0x456C, 0x20000000},
+ {0x4570, 0x04F040BB},
+ {0x4574, 0x000E53FF},
+ {0x4578, 0x000205CB},
+ {0x457C, 0x00200000},
+ {0x4580, 0x00000040},
+ {0x4584, 0x00000000},
+ {0x4588, 0x00000017},
+ {0x458C, 0x30000000},
+ {0x4590, 0x00000000},
+ {0x4594, 0x00000000},
+ {0x4598, 0x00000001},
+ {0x459C, 0x0003FE00},
+ {0x45A0, 0x00000086},
+ {0x45A4, 0x00000000},
+ {0x45A8, 0xC00001C0},
+ {0x45AC, 0x78038000},
+ {0x45B0, 0x8000004A},
+ {0x45B4, 0x04094800},
+ {0x45B8, 0x00280002},
+ {0x45BC, 0x06748790},
+ {0x45C0, 0x80000000},
+ {0x45C4, 0x00000000},
+ {0x45C8, 0x00000000},
+ {0x45CC, 0x00558670},
+ {0x45D0, 0x002883F0},
+ {0x45D4, 0x00090120},
+ {0x45D8, 0x00000000},
+ {0x45E0, 0xA3A6D3C4},
+ {0x45E4, 0xAB27B126},
+ {0x45E8, 0x00006778},
+ {0x45F4, 0x000001B5},
+ {0x45EC, 0x11110F0A},
+ {0x45F0, 0x00000003},
+ {0x4A0C, 0x0000000A},
+ {0x45F8, 0x0058BC3F},
+ {0x45FC, 0x00000003},
+ {0x462C, 0x00000020},
+ {0x4600, 0x000003D9},
+ {0x45F0, 0x00000004},
+ {0x4604, 0x002B1CB0},
+ {0x4A50, 0xC0000000},
+ {0x4A54, 0x00001000},
+ {0x4A58, 0x00000000},
+ {0x4A18, 0x00000024},
+ {0x4608, 0x00000001},
+ {0x460C, 0x00000000},
+ {0x4A10, 0x00000001},
+ {0x4610, 0x00000001},
+ {0x4614, 0x16E5298F},
+ {0x4618, 0x18C6294A},
+ {0x461C, 0x0E06318A},
+ {0x4620, 0x0E539CE5},
+ {0x4624, 0x00019287},
+ {0x4A14, 0x000000BF},
+ {0x4628, 0x00000001},
+ {0x4630, 0x000001AA},
+ {0x4A18, 0x00001900},
+ {0x4A1C, 0x000002A6},
+ {0x4634, 0x000000A3},
+ {0x4A20, 0x00000086},
+ {0x4638, 0x00045656},
+ {0x49F8, 0x00000000},
+ {0x463C, 0x00000000},
+ {0x4640, 0x00000000},
+ {0x4644, 0x00C8CC00},
+ {0x4648, 0xC400B6B6},
+ {0x464C, 0xDC400FC0},
+ {0x4A44, 0x00000000},
+ {0x4A8C, 0x00000110},
+ {0x4BC4, 0x00000001},
+ {0x4650, 0x08882550},
+ {0x4654, 0x08CC2660},
+ {0x4658, 0x09102660},
+ {0x465C, 0x00000154},
+ {0x45DC, 0xC39E38E8},
+ {0x4660, 0x452607E6},
+ {0x4664, 0x6750DC65},
+ {0x4668, 0xF3F0F1ED},
+ {0x466C, 0x30141506},
+ {0x4670, 0x2C2B2B2B},
+ {0x4674, 0x2C2C2C2C},
+ {0x4678, 0xDDB738E8},
+ {0x467C, 0x543618FB},
+ {0x4680, 0x4F31DC6F},
+ {0x4684, 0xFBEBDA00},
+ {0x4688, 0x1A10FF04},
+ {0x468C, 0x282A3000},
+ {0x4690, 0x2A29292A},
+ {0x4694, 0x04FA2A2A},
+ {0x4698, 0xEE0F04D1},
+ {0x469C, 0x99E91436},
+ {0x46A0, 0x0701E79E},
+ {0x46A4, 0x08D77CFF},
+ {0x46A8, 0x321AFF14},
+ {0x46AC, 0x60313447},
+ {0x46B0, 0x63666666},
+ {0x46B4, 0x35374425},
+ {0x46B8, 0x35883042},
+ {0x46BC, 0x5177C252},
+ {0x4720, 0x7FFFFD63},
+ {0x4724, 0xB58D11FF},
+ {0x4728, 0x47FFFFFF},
+ {0x472C, 0x0E7893B6},
+ {0x4730, 0xE0391201},
+ {0x4734, 0x00000020},
+ {0x4738, 0x8325C500},
+ {0x473C, 0x00000B7F},
+ {0x46C0, 0x00000000},
+ {0x46C4, 0x00000000},
+ {0x46C8, 0x00000219},
+ {0x4BDC, 0x00002020},
+ {0x46CC, 0x00000000},
+ {0x46D0, 0x00000000},
+ {0x4A3C, 0x00000002},
+ {0x46D4, 0x00000001},
+ {0x46D8, 0x00000001},
+ {0x46DC, 0x00000000},
+ {0x46E0, 0x00000000},
+ {0x46E4, 0x00000151},
+ {0x46E8, 0x00000498},
+ {0x46EC, 0x00000498},
+ {0x46F0, 0x00000000},
+ {0x46F4, 0x00000000},
+ {0x46F8, 0x00001146},
+ {0x46FC, 0x00000000},
+ {0x4700, 0x00000000},
+ {0x4704, 0x00C8CC00},
+ {0x4708, 0xC400B6B6},
+ {0x470C, 0xDC400FC0},
+ {0x4A90, 0x00000110},
+ {0x4B10, 0x00000000},
+ {0x4BE0, 0x00000001},
+ {0x4710, 0x08882550},
+ {0x4714, 0x08CC2660},
+ {0x4718, 0x09102660},
+ {0x471C, 0x00000154},
+ {0x4740, 0xC69F38E8},
+ {0x4744, 0x462709E9},
+ {0x4748, 0x6750DC67},
+ {0x474C, 0xF3F0F1ED},
+ {0x4750, 0x30141506},
+ {0x4754, 0x2C2B2B2B},
+ {0x4758, 0x2C2C2C2C},
+ {0x475C, 0xE0B738E8},
+ {0x4760, 0x52381BFE},
+ {0x4764, 0x5031DC6C},
+ {0x4768, 0xFBEBDA00},
+ {0x476C, 0x1A10FF04},
+ {0x4770, 0x282A3000},
+ {0x4774, 0x2A29292A},
+ {0x4778, 0x04FA2A2A},
+ {0x477C, 0xEE0F04D1},
+ {0x47C4, 0x00000000},
+ {0x47C8, 0xA32103FE},
+ {0x47CC, 0xB20A5328},
+ {0x47D0, 0xC686314F},
+ {0x47D4, 0x000004D7},
+ {0x4BFC, 0x00000000},
+ {0x4C00, 0x0C442416},
+ {0x4C04, 0x00000000},
+ {0x47D8, 0x009B902A},
+ {0x47DC, 0x009B902A},
+ {0x47E0, 0x98682C18},
+ {0x47E4, 0x6318C4C1},
+ {0x47E8, 0x6248C631},
+ {0x47EC, 0x922A8253},
+ {0x47F0, 0x00000005},
+ {0x47F4, 0x00001759},
+ {0x47F8, 0x4BB01800},
+ {0x47FC, 0x831408BE},
+ {0x4A84, 0x000000E9},
+ {0x4C08, 0x0F801404},
+ {0x4C0C, 0x00A2B404},
+ {0x4800, 0x9ABBCACB},
+ {0x4804, 0x56867578},
+ {0x4808, 0xBCCBBB13},
+ {0x480C, 0x7889989B},
+ {0x4810, 0xBBB0F455},
+ {0x4814, 0x777BBBBB},
+ {0x4818, 0x15277777},
+ {0x481C, 0x27039CE9},
+ {0x4820, 0x42424432},
+ {0x4824, 0x36058342},
+ {0x4828, 0x00000006},
+ {0x482C, 0x00000005},
+ {0x4830, 0x00000005},
+ {0x4834, 0xC7013016},
+ {0x4838, 0x84413016},
+ {0x483C, 0x84413016},
+ {0x4840, 0x8C413016},
+ {0x4844, 0x8C40B028},
+ {0x4848, 0x3140B028},
+ {0x484C, 0x2940B028},
+ {0x4850, 0x8440B028},
+ {0x4854, 0x2318C610},
+ {0x4858, 0x45344753},
+ {0x485C, 0x236A6A88},
+ {0x4860, 0xAC8DF814},
+ {0x4864, 0x08877ACB},
+ {0x4868, 0x000107AA},
+ {0x4A94, 0x00000000},
+ {0x486C, 0xBCEB4A14},
+ {0x4870, 0x000A3A4A},
+ {0x4874, 0xBCEB4A14},
+ {0x4878, 0x000A3A4A},
+ {0x487C, 0xBCBDBD85},
+ {0x4880, 0x0CABB99A},
+ {0x4884, 0x38384242},
+ {0x4888, 0x0086102E},
+ {0x488C, 0xCA24C82A},
+ {0x4AFC, 0x00000000},
+ {0x4C14, 0x0000349D},
+ {0x4CF8, 0x00000007},
+ {0x4890, 0x00008A62},
+ {0x4894, 0x00000008},
+ {0x4958, 0x80040000},
+ {0x495C, 0x80040000},
+ {0x4960, 0xFE800000},
+ {0x4964, 0x834C0000},
+ {0x4968, 0x00000000},
+ {0x496C, 0x00000000},
+ {0x4970, 0x00000000},
+ {0x4974, 0x00000000},
+ {0x4978, 0x00000000},
+ {0x497C, 0x00000000},
+ {0x4980, 0x40000000},
+ {0x4984, 0x00000000},
+ {0x4988, 0x00000000},
+ {0x498C, 0x00000000},
+ {0x4990, 0x00000000},
+ {0x4994, 0x04065800},
+ {0x4998, 0x02004080},
+ {0x499C, 0x0E1E3E05},
+ {0x49A0, 0x0A163068},
+ {0x49A4, 0x00206040},
+ {0x49A8, 0x02020202},
+ {0x49AC, 0x00002020},
+ {0x49B0, 0xF8F8F418},
+ {0x49B4, 0xF8E8F8F8},
+ {0x49B8, 0xF80808E8},
+ {0x4A00, 0xF8F8FA00},
+ {0x4A04, 0xFAFAFAF8},
+ {0x4A08, 0xFAFAFAFA},
+ {0x49BC, 0x00000000},
+ {0x49C0, 0x800C562D},
+ {0x49C4, 0x00000101},
+ {0x49C8, 0x00000000},
+ {0x49CC, 0x00000000},
+ {0x49D0, 0x00000000},
+ {0x49D4, 0x00000000},
+ {0x49D8, 0x00000000},
+ {0x49DC, 0x00000000},
+ {0x49E0, 0x00000000},
+ {0x49E4, 0x00000000},
+ {0x49E8, 0x00000000},
+ {0x49EC, 0x00000000},
+ {0x4C28, 0x00000000},
+ {0x4C2C, 0x00000000},
+ {0x4C30, 0x00000000},
+ {0x4C34, 0x00000000},
+ {0x4C38, 0x00000000},
+ {0x4C3C, 0x00000000},
+ {0x4C40, 0x00000000},
+ {0x4C44, 0x01C0C832},
+ {0x4C48, 0x03207032},
+ {0x4C4C, 0x0320701C},
+ {0x4C50, 0x03207032},
+ {0x4C54, 0x01C0C81C},
+ {0x4C58, 0x00A0281C},
+ {0x4C5C, 0x0320C80A},
+ {0x4C60, 0x00A0C832},
+ {0x4C64, 0x01C0C832},
+ {0x4C68, 0x03207032},
+ {0x4C6C, 0x0320701C},
+ {0x4C70, 0x03207032},
+ {0x4C74, 0x01C0C81C},
+ {0x4C78, 0x00A0281C},
+ {0x4C7C, 0x0321A80A},
+ {0x4C80, 0x0320C86A},
+ {0x4C84, 0x12B02832},
+ {0x4C88, 0x12B3292B},
+ {0x4C8C, 0x0CA4ACCA},
+ {0x4C90, 0x12B4AC6A},
+ {0x4C94, 0x0CA4ACCA},
+ {0x4C98, 0x06A3292B},
+ {0x4C9C, 0x06A0280A},
+ {0x4CA0, 0x0CA0286A},
+ {0x4CA4, 0x0CA1A8CA},
+ {0x4CA8, 0x06A3286A},
+ {0x4CAC, 0x0000000A},
+ {0x4CB0, 0x01209C27},
+ {0x4CB4, 0x02704800},
+ {0x4CB8, 0x02704812},
+ {0x4CBC, 0x00004827},
+ {0x4CC0, 0x01209C12},
+ {0x4CC4, 0x00000012},
+ {0x4CC8, 0x02718000},
+ {0x4CCC, 0x02709C60},
+ {0x4CD0, 0x00000027},
+ {0x4CD4, 0x00000000},
+ {0x4CD8, 0x0000014A},
+ {0x994, 0x00000010},
+ {0x904, 0x00000005},
+ {0x708, 0x00000000},
+ {0x884, 0x0043F01D},
+ {0x710, 0xEF810000},
+ {0x718, 0x1333233F},
+ {0x604, 0x041E1E1E},
+ {0x714, 0x00010000},
+ {0x586C, 0x000000F0},
+ {0x586C, 0x000000E0},
+ {0x586C, 0x000000D0},
+ {0x586C, 0x000000C0},
+ {0x586C, 0x000000B0},
+ {0x586C, 0x000000A0},
+ {0x586C, 0x00000090},
+ {0x586C, 0x00000080},
+ {0x586C, 0x00000070},
+ {0x586C, 0x00000060},
+ {0x586C, 0x00000050},
+ {0x586C, 0x00000040},
+ {0x586C, 0x00000030},
+ {0x586C, 0x00000020},
+ {0x586C, 0x00000010},
+ {0x586C, 0x00000000},
+ {0xC0D4, 0xABA41460},
+ {0xC0D8, 0xC43A7E87},
+ {0xC0DC, 0x30C194B8},
+ {0xC0E0, 0x75008138},
+ {0xC0E4, 0x0000272B},
+ {0xC0E8, 0x000A0C81},
+ {0xC0EC, 0x00030003},
+ {0xC0F0, 0x00000024},
+ {0xC0C4, 0x005E3A00},
+ {0xC004, 0x45800000},
+ {0xC024, 0x45800000},
+ {0x334, 0xFFFFFFFF},
+ {0x33C, 0x55000000},
+ {0x340, 0x00005555},
+ {0x724, 0x00111200},
+ {0x5868, 0xA9550000},
+ {0x5870, 0x33221100},
+ {0x5874, 0x77665544},
+ {0x5878, 0xBBAA9988},
+ {0x587C, 0xFFEEDDCC},
+ {0x5880, 0x76543210},
+ {0x5884, 0xFEDCBA98},
+ {0x5888, 0x00000000},
+ {0x588C, 0x00000000},
+ {0x5894, 0x00000008},
+ {0x650, 0x00200888},
+ {0x710, 0xF3810000},
+ {0x020, 0x0000F381},
+ {0x024, 0x0000F381},
+ {0x000, 0xC580801E},
+ {0x980, 0x10002250},
+ {0x988, 0x3C3C4107},
+ {0x994, 0x00000010},
+ {0x000, 0x0580801F},
+ {0x240C, 0x00000000},
+ {0x640, 0x210A141E},
+ {0x640, 0x2114141E},
+ {0x640, 0x2114141E},
+ {0x644, 0x3414283C},
+ {0x644, 0x3425283C},
+ {0x644, 0x3426283C},
+ {0x2640, 0x140A141E},
+ {0x2640, 0x1414141E},
+ {0x2640, 0x1414141E},
+ {0x2644, 0x3414283C},
+ {0x2644, 0x3425283C},
+ {0x2644, 0x3425183C},
+ {0x2300, 0x02748790},
+ {0x2304, 0x00558670},
+ {0x2308, 0x002883F0},
+ {0x230C, 0x00090120},
+ {0x2310, 0x00000000},
+ {0x2314, 0x06000000},
+ {0x2318, 0x00000000},
+ {0x231C, 0x00000000},
+ {0x2320, 0x03020100},
+ {0x2324, 0x07060504},
+ {0x2328, 0x0B0A0908},
+ {0x232C, 0x0F0E0D0C},
+ {0x2330, 0x13121110},
+ {0x2334, 0x17161514},
+ {0x2338, 0x0C700022},
+ {0x233C, 0x0A0529D0},
+ {0x2340, 0x000529D0},
+ {0x2344, 0x0006318A},
+ {0x2348, 0xB7E6318A},
+ {0x234C, 0x80039C00},
+ {0x2350, 0x80039C00},
+ {0x2354, 0x0005298F},
+ {0x2358, 0x0015296E},
+ {0x235C, 0x0C07FC31},
+ {0x2360, 0x0219AAAE},
+ {0x2364, 0xE4F624C3},
+ {0x2368, 0x53626F15},
+ {0x236C, 0x48000000},
+ {0x2370, 0x48000000},
+ {0x2374, 0x07540000},
+ {0x2378, 0x202401B9},
+ {0x237C, 0x00F7000E},
+ {0x2380, 0x0F0A1111},
+ {0x2384, 0x30D9000F},
+ {0x2388, 0x0200EA02},
+ {0x238C, 0x003CB061},
+ {0x2390, 0x69C00000},
+ {0x2394, 0x00000000},
+ {0x2398, 0x000000F0},
+ {0x239C, 0x0001FFFF},
+ {0x23A0, 0x00C80064},
+ {0x23A4, 0x0190012C},
+ {0x23A8, 0x001917BE},
+ {0x23AC, 0x0B30880C},
+ {0x23B0, 0x9281CE00},
+ {0x23B4, 0x7F027C00},
+ {0x704, 0x601E0502},
+ {0x5600, 0x00000000},
+ {0x5604, 0x802D2721},
+ {0x5610, 0x00201020},
+ {0x5618, 0x00801008},
+ {0x5624, 0x0808081E},
+ {0x562C, 0x0000081D},
+ {0x5634, 0x3D2EE000},
+ {0x5638, 0x0001AC42},
+ {0x5640, 0x3D6EF000},
+ {0x5644, 0x0001AC3E},
+ {0x566C, 0x00210005},
+ {0x5680, 0x20500010},
+ {0x5684, 0x00020001},
+ {0x56A0, 0x0034C000},
+ {0x56BC, 0x04000000},
+ {0x56C0, 0x00000688},
+ {0x56C4, 0x00000010},
+ {0x56C8, 0x0E800400},
+ {0x56CC, 0x01E400FF},
+ {0x5800, 0x003F807F},
+ {0x5810, 0x59008400},
+ {0x5814, 0x201AF000},
+ {0x5818, 0x182C18E8},
+ {0x581C, 0x3DD80280},
+ {0x5820, 0x80000080},
+ {0x5828, 0x023F8121},
+ {0x5830, 0x023F8121},
+ {0x5838, 0x003F8121},
+ {0x5840, 0x023F8121},
+ {0x5848, 0x023F8121},
+ {0x5850, 0x023F8121},
+ {0x5858, 0x003F7121},
+ {0x5860, 0x023F7121},
+ {0x5864, 0x1A1801FF},
+ {0x5868, 0xA9A90002},
+ {0x5880, 0x77777777},
+ {0x5884, 0x77777777},
+ {0x5894, 0x01080604},
+ {0x5898, 0x00000000},
+ {0x589C, 0x00000000},
+ {0x58A0, 0x000000FE},
+ {0x58B0, 0x00000800},
+ {0x58BC, 0x07A7807F},
+ {0x58C0, 0x007E0000},
+ {0x58C4, 0x0003FFFF},
+ {0x58D4, 0x7401FE00},
+ {0x58D8, 0x8008016C},
+ {0x58DC, 0xC000807F},
+ {0x58E4, 0x3000881F},
+ {0x58E8, 0x00000003},
+ {0x58F0, 0x400401FF},
+ {0x58F4, 0x80000000},
+ {0x58F8, 0xC0000000},
+ {0x58FC, 0x00000000},
+ {0x700, 0x40000030},
+ {0x704, 0x601E0502},
+ {0x704, 0x601E0500},
+ {0x704, 0x601E0502},
+ {0x20FC, 0x00000000},
+ {0x20F8, 0x00000000},
+ {0x20F0, 0x00000000},
+ {0x9C0, 0x00000001},
+ {0x9C0, 0x00000000},
+ {0x9C0, 0x00000001},
+ {0x9C0, 0x00000000},
+ {0x4AE8, 0x00000744},
+ {0x4AD4, 0x00000040},
+ {0x4AE4, 0x0079E99E},
+ {0x4BC8, 0xFBD5B89F},
+ {0x4BCC, 0x99563918},
+ {0x4BD0, 0x12EED5B8},
+ {0x4BD4, 0x6F7D542F},
+ {0x4BD8, 0x0000001D},
+ {0x300, 0xF30CE31C},
+ {0x304, 0x13EF1F19},
+ {0x308, 0x0C0CF3F3},
+ {0x30C, 0x0CE30C0C},
+ {0x310, 0x80496000},
+ {0x314, 0x0041E000},
+ {0x318, 0x20022042},
+ {0x31C, 0x20448009},
+ {0x320, 0x00010031},
+ {0x324, 0xE000E000},
+ {0x328, 0xE000E000},
+ {0x32C, 0xE0008049},
+ {0x12BC, 0x10104041},
+ {0x12C0, 0x13311111},
+ {0x12E4, 0x30D52A68},
+ {0x010, 0x0005FFFF},
+ {0x028, 0x0000F381},
+ {0x02C, 0x0000F381},
+ {0x620, 0x00141230},
+ {0x70C, 0x00000020},
+ {0x720, 0x20000000},
+ {0x730, 0x00000002},
+ {0x738, 0x004100C0},
+ {0x73C, 0x00000002},
+ {0x748, 0x01000002},
+ {0x74C, 0x00000001},
+ {0xA08, 0x00007800},
+ {0xC14, 0x25010000},
+ {0xC3C, 0x2840E1BF},
+ {0xC40, 0x00000000},
+ {0xC44, 0x00000007},
+ {0xC48, 0x410E4000},
+ {0xC54, 0x1EE14368},
+ {0xC58, 0x41000000},
+ {0xC5C, 0x80558000},
+ {0xC60, 0x017FFFF2},
+ {0xC64, 0x0010A130},
+ {0xC68, 0x90000050},
+ {0xC6C, 0x10201021},
+ {0xC70, 0x071B0660},
+ {0xC74, 0x00000000},
+ {0xC78, 0x80000000},
+ {0xC7C, 0x0020BFE0},
+ {0xC88, 0xC2AC8000},
+ {0xC8C, 0x02F2FC08},
+ {0xD00, 0x77777777},
+ {0xD04, 0xBBBBBBBB},
+ {0xD08, 0xBBBBBBBB},
+ {0xD0C, 0x000B2070},
+ {0xD10, 0x20110FFF},
+ {0xD18, 0x50009800},
+ {0xD20, 0x01900000},
+ {0xD30, 0x03FF8000},
+ {0xD40, 0xF64FA0F7},
+ {0xD44, 0x0401463F},
+ {0xD48, 0x0003FF7F},
+ {0xD4C, 0x00000000},
+ {0xD50, 0xF64FA0F7},
+ {0xD54, 0x04100437},
+ {0xD58, 0x0000FF7F},
+ {0xD5C, 0x00000000},
+ {0xD60, 0x00000000},
+ {0xD64, 0x00000000},
+ {0xD70, 0x00000015},
+ {0xD78, 0x00000001},
+ {0xD7C, 0x001D050E},
+ {0xD80, 0x00000100},
+ {0xD84, 0x00006607},
+ {0xD90, 0x000003FF},
+ {0xD94, 0x00000000},
+ {0xD98, 0x0000003F},
+ {0xD9C, 0x00000000},
+ {0xDA0, 0x000003FE},
+ {0xDA4, 0x00000000},
+ {0xDA8, 0x0000003F},
+ {0xDAC, 0x00000000},
+ {0xDD4, 0x00000000},
+ {0x1010, 0x00000000},
+ {0x2000, 0x50BBBF04},
+ {0x2008, 0x000FFFFF},
+ {0x5800, 0x03FF807F},
+ {0x5804, 0x04237040},
+ {0x5808, 0x04237040},
+ {0x5818, 0x082C1800},
+ {0x624, 0x0101030A},
+ {0x241C, 0x00000001},
+ {0xC0F8, 0x00000001},
+ {0x35C, 0x000004C4},
+ {0x1200, 0x00010142},
+ {0x120C, 0x00012233},
+ {0x1210, 0x8049E304},
+ {0x12A0, 0x49107056},
+ {0x12A8, 0x33337025},
+ {0x12AC, 0x12333121},
+ {0x12B8, 0x30020000},
+ {0x0F0, 0x00000001},
+ {0x0F4, 0x00000011},
+ {0x0F8, 0x20230307},
+};
+
+static const struct rtw89_reg2_def rtw89_8851b_phy_bb_reg_gain[] = {
+ {0xF00100FF, 0x00000000},
+ {0xF00200FF, 0x00000001},
+ {0xF00300FF, 0x00000002},
+ {0xF00400FF, 0x00000003},
+ {0xF00500FF, 0x00000004},
+ {0xF00600FF, 0x00000005},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x13F6D7B6},
+ {0x001, 0x00725132},
+ {0x002, 0x00005A38},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x13F6D7B6},
+ {0x001, 0x00725132},
+ {0x002, 0x00005A38},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x13F6D7B6},
+ {0x001, 0x00725132},
+ {0x002, 0x00005A38},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x19FADCBC},
+ {0x001, 0x007A5A3A},
+ {0x002, 0x00005838},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x19FADCBC},
+ {0x001, 0x007A5A3A},
+ {0x002, 0x00005838},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x000, 0x19FADCBC},
+ {0x001, 0x007A5A3A},
+ {0x002, 0x00005838},
+ {0xA0000000, 0x00000000},
+ {0x000, 0x13F6D7B6},
+ {0x001, 0x00725132},
+ {0x002, 0x00005A38},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0xA0000000, 0x00000000},
+ {0x100, 0x1BFEE0B7},
+ {0x101, 0x006C5238},
+ {0x102, 0x00005031},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x19F8D8C1},
+ {0x10001, 0x006F4F31},
+ {0x10002, 0x00006F58},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x19F8D8C1},
+ {0x10001, 0x006F4F31},
+ {0x10002, 0x00006F58},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x19F8D8C1},
+ {0x10001, 0x006F4F31},
+ {0x10002, 0x00006F58},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x1DF8DAC1},
+ {0x10001, 0x00755437},
+ {0x10002, 0x00007058},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x1DF8DAC1},
+ {0x10001, 0x00755437},
+ {0x10002, 0x00007058},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10000, 0x1DF8DAC1},
+ {0x10001, 0x00755437},
+ {0x10002, 0x00007058},
+ {0xA0000000, 0x00000000},
+ {0x10000, 0x19F8D8C1},
+ {0x10001, 0x006F4F31},
+ {0x10002, 0x00006F58},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0xA0000000, 0x00000000},
+ {0x10100, 0x09E9C69F},
+ {0x10101, 0x00674627},
+ {0x10102, 0x00006750},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1AF0D2B8},
+ {0x20001, 0x00755334},
+ {0x20002, 0x00006F58},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1AF0D2B8},
+ {0x20001, 0x00755334},
+ {0x20002, 0x00006F58},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1AF0D2B8},
+ {0x20001, 0x00755334},
+ {0x20002, 0x00006F58},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1D00E2C8},
+ {0x20001, 0x00775336},
+ {0x20002, 0x00006D58},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1D00E2C8},
+ {0x20001, 0x00775336},
+ {0x20002, 0x00006D58},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20000, 0x1D00E2C8},
+ {0x20001, 0x00775336},
+ {0x20002, 0x00006D58},
+ {0xA0000000, 0x00000000},
+ {0x20000, 0x1AF0D2B8},
+ {0x20001, 0x00755334},
+ {0x20002, 0x00006F58},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0xA0000000, 0x00000000},
+ {0x20100, 0x07E9C6A0},
+ {0x20101, 0x00674728},
+ {0x20102, 0x00006850},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x15EED2B6},
+ {0x30001, 0x006F4D2F},
+ {0x30002, 0x00006F58},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x15EED2B6},
+ {0x30001, 0x006F4D2F},
+ {0x30002, 0x00006F58},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x15EED2B6},
+ {0x30001, 0x006F4D2F},
+ {0x30002, 0x00006F58},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x1F00E2C6},
+ {0x30001, 0x00795739},
+ {0x30002, 0x00006F58},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x1F00E2C6},
+ {0x30001, 0x00795739},
+ {0x30002, 0x00006F58},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30000, 0x1F00E2C6},
+ {0x30001, 0x00795739},
+ {0x30002, 0x00006F58},
+ {0xA0000000, 0x00000000},
+ {0x30000, 0x15EED2B6},
+ {0x30001, 0x006F4D2F},
+ {0x30002, 0x00006F58},
+ {0xB0000000, 0x00000000},
+ {0x800100ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0x900200ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0x900300ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0x900400ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0x900500ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0x900600ff, 0x00000000}, {0x40000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0xA0000000, 0x00000000},
+ {0x30100, 0x06E9C69F},
+ {0x30101, 0x00654527},
+ {0x30102, 0x00006750},
+ {0xB0000000, 0x00000000},
+ {0x1000000, 0x000000F4},
+ {0x1000010, 0x000000F8},
+ {0x1000011, 0x0000F8F8},
+ {0x1000100, 0x000000F8},
+ {0x1000110, 0x00000000},
+ {0x1000111, 0x00000000},
+ {0x1010000, 0x000000F4},
+ {0x1010010, 0x000000F8},
+ {0x1010011, 0x0000F8F8},
+ {0x1010020, 0x000000F8},
+ {0x1010021, 0x0808E8E8},
+ {0x1010029, 0x0000F8F8},
+ {0x1010100, 0x000000F4},
+ {0x1010110, 0x000000F8},
+ {0x1010111, 0x0000F8F8},
+ {0x1010120, 0x000000F8},
+ {0x1010121, 0x0808E8E8},
+ {0x1010129, 0x0000F8F8},
+ {0x1020000, 0x000000F4},
+ {0x1020010, 0x000000F8},
+ {0x1020011, 0x0000F8F8},
+ {0x1020020, 0x000000F8},
+ {0x1020021, 0x0808E8E8},
+ {0x1020029, 0x0000F8F8},
+ {0x1020100, 0x000000F4},
+ {0x1020110, 0x000000F8},
+ {0x1020111, 0x0000F8F8},
+ {0x1020120, 0x000000F8},
+ {0x1020121, 0x0808E8E8},
+ {0x1020129, 0x0000F8F8},
+ {0x1030000, 0x000000F4},
+ {0x1030010, 0x000000F8},
+ {0x1030011, 0x0000F8F8},
+ {0x1030020, 0x000000F8},
+ {0x1030021, 0x0808E8E8},
+ {0x1030029, 0x0000F8F8},
+ {0x1030100, 0x000000F4},
+ {0x1030110, 0x000000F8},
+ {0x1030111, 0x0000F8F8},
+ {0x1030120, 0x000000F8},
+ {0x1030121, 0x0808E8E8},
+ {0x1030129, 0x0000F8F8},
+};
+
+static const struct rtw89_reg2_def rtw89_8851b_phy_radioa_regs[] = {
+ {0xF0010000, 0x00000000},
+ {0xF0020000, 0x00000001},
+ {0xF0030000, 0x00000002},
+ {0x000, 0x00030000},
+ {0x018, 0x00013124},
+ {0x0EF, 0x00080000},
+ {0x033, 0x00000008},
+ {0x03E, 0x00000110},
+ {0x03F, 0x0000D39C},
+ {0x033, 0x0000000C},
+ {0x03E, 0x00000110},
+ {0x03F, 0x0000F79E},
+ {0x0EF, 0x00000000},
+ {0x01B, 0x00003A40},
+ {0x08F, 0x000C170E},
+ {0x08E, 0x00005160},
+ {0x002, 0x00000600},
+ {0x0EE, 0x00000002},
+ {0x033, 0x00000002},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000003},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000004},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000005},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000006},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000007},
+ {0x03F, 0x0000003F},
+ {0x033, 0x00000008},
+ {0x03F, 0x0000003F},
+ {0x033, 0x0000000C},
+ {0x03F, 0x0000003F},
+ {0x033, 0x0000000D},
+ {0x03F, 0x0000003F},
+ {0x033, 0x0000000E},
+ {0x03F, 0x0000003F},
+ {0x0EE, 0x00000000},
+ {0x0EF, 0x00004000},
+ {0x033, 0x00000007},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00000707},
+ {0x033, 0x00000006},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00000704},
+ {0x033, 0x00000005},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00020500},
+ {0x033, 0x00000004},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00010404},
+ {0x033, 0x00000003},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00099B04},
+ {0x033, 0x00000002},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00092B04},
+ {0x033, 0x00000001},
+ {0x03E, 0x00000000},
+ {0x03F, 0x000B3204},
+ {0x033, 0x00000000},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00003000},
+ {0x033, 0x00000017},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00000787},
+ {0x033, 0x00000016},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00000784},
+ {0x033, 0x00000015},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00020580},
+ {0x033, 0x00000014},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00010484},
+ {0x033, 0x00000013},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00099B84},
+ {0x033, 0x00000012},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00092B84},
+ {0x033, 0x00000011},
+ {0x03E, 0x00000000},
+ {0x03F, 0x000B3284},
+ {0x033, 0x00000010},
+ {0x03E, 0x00000000},
+ {0x03F, 0x00003080},
+ {0x0EF, 0x00000000},
+ {0x0EE, 0x00000010},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000003},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000003},
+ {0x0EE, 0x00000000},
+ {0x0EF, 0x00001000},
+ {0x033, 0x00000000},
+ {0x03F, 0x00000034},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000037},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000034},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000024},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000037},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000027},
+ {0x0EF, 0x00000000},
+ {0x0EC, 0x00000400},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000022},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000022},
+ {0x033, 0x00000009},
+ {0x03F, 0x00000022},
+ {0x0EC, 0x00000000},
+ {0x0EC, 0x00000004},
+ {0x033, 0x00000000},
+ {0x03F, 0x000000AE},
+ {0x033, 0x00000001},
+ {0x03F, 0x0000008C},
+ {0x033, 0x00000002},
+ {0x03F, 0x0000006A},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000048},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000026},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000004},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000002},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000000},
+ {0x0EC, 0x00000000},
+ {0x0EF, 0x00008000},
+ {0x033, 0x00000007},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000006},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000005},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x00000004},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x00000003},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001BB0},
+ {0x033, 0x00000002},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001AB0},
+ {0x033, 0x00000001},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D9BC},
+ {0x033, 0x00000000},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D4BC},
+ {0x033, 0x00000017},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000016},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000015},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x00000014},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x00000013},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001BB0},
+ {0x033, 0x00000012},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001AB0},
+ {0x033, 0x00000011},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D9BC},
+ {0x033, 0x00000010},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D4BC},
+ {0x033, 0x00000027},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000026},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x00000025},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x00000024},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x00000023},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001BB0},
+ {0x033, 0x00000022},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001AB0},
+ {0x033, 0x00000021},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D9BC},
+ {0x033, 0x00000020},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D4BC},
+ {0x033, 0x0000000E},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x0000000D},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x0000000C},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x0000000B},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00091BB0},
+ {0x033, 0x0000000A},
+ {0x03E, 0x00000003},
+ {0x03F, 0x000A9AB0},
+ {0x033, 0x00000009},
+ {0x03E, 0x00000003},
+ {0x03F, 0x000BD9BC},
+ {0x033, 0x00000008},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0009D4BC},
+ {0x033, 0x0000001E},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x0000001D},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x0000001C},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x0000001B},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00091BB0},
+ {0x033, 0x0000001A},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00099AB0},
+ {0x033, 0x00000019},
+ {0x03E, 0x00000003},
+ {0x03F, 0x000AD9BC},
+ {0x033, 0x00000018},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0009D4BC},
+ {0x033, 0x0000002E},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001FB0},
+ {0x033, 0x0000002D},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001DB0},
+ {0x033, 0x0000002C},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001CB0},
+ {0x033, 0x0000002B},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00001BB0},
+ {0x033, 0x0000002A},
+ {0x03E, 0x00000003},
+ {0x03F, 0x00009AB0},
+ {0x033, 0x00000029},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0009D9BC},
+ {0x033, 0x00000028},
+ {0x03E, 0x00000003},
+ {0x03F, 0x0000D4BC},
+ {0x0EF, 0x00000000},
+ {0x0EF, 0x00002000},
+ {0x033, 0x00000000},
+ {0x03F, 0x00000005},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000004},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000004},
+ {0x033, 0x00000004},
+ {0x03F, 0x0000000C},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000003},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000003},
+ {0x0EF, 0x00000000},
+ {0x06C, 0x00038085},
+ {0x06D, 0x00000D6B},
+ {0x06E, 0x0001FB89},
+ {0x06F, 0x00097B99},
+ {0x069, 0x00008040},
+ {0x0EF, 0x00000200},
+ {0x033, 0x00000004},
+ {0x03F, 0x000008FF},
+ {0x033, 0x00000005},
+ {0x03F, 0x000004F2},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000217},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000131},
+ {0x0EF, 0x00000000},
+ {0x0EF, 0x00000400},
+ {0x033, 0x00000004},
+ {0x03F, 0x000004F7},
+ {0x033, 0x00000005},
+ {0x03F, 0x000004F7},
+ {0x033, 0x00000006},
+ {0x03F, 0x000004F2},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000117},
+ {0x0EF, 0x00000000},
+ {0x043, 0x00005000},
+ {0x036, 0x000147D0},
+ {0x0B0, 0x0008677C},
+ {0x0B1, 0x00012920},
+ {0x0BB, 0x000EF000},
+ {0x0CB, 0x000A9594},
+ {0x0CC, 0x000C36D2},
+ {0x0CD, 0x00024923},
+ {0x0CE, 0x00020180},
+ {0x0CF, 0x00000000},
+ {0x0D5, 0x0006E27A},
+ {0x0D8, 0x00000044},
+ {0x0D9, 0x00000007},
+ {0x0DD, 0x00000020},
+ {0x0E3, 0x0000002C},
+ {0x0B7, 0x0000000C},
+ {0x0E1, 0x000080C0},
+ {0x0E4, 0x00000380},
+ {0x0ED, 0x00002000},
+ {0x033, 0x00000001},
+ {0x03D, 0x000A6094},
+ {0x03E, 0x00003449},
+ {0x03F, 0x00000001},
+ {0x033, 0x00000003},
+ {0x03D, 0x000AA094},
+ {0x03E, 0x00003449},
+ {0x03F, 0x00000001},
+ {0x0ED, 0x00000000},
+ {0x0ED, 0x00000100},
+ {0x033, 0x00000000},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000001},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000002},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000003},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000004},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000005},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000007},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000008},
+ {0x03F, 0x0000007F},
+ {0x033, 0x00000009},
+ {0x03F, 0x0000007F},
+ {0x0ED, 0x00000000},
+ {0x0ED, 0x00000080},
+ {0x033, 0x00000000},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x033, 0x00000010},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x033, 0x00000030},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x033, 0x00000040},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x033, 0x00000050},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x033, 0x00000070},
+ {0x03E, 0x000007E1},
+ {0x03F, 0x0001F87F},
+ {0x0ED, 0x00000000},
+ {0x0ED, 0x00000004},
+ {0x033, 0x00000000},
+ {0x03F, 0x00008420},
+ {0x0ED, 0x00000000},
+ {0x018, 0x00011108},
+ {0x0B9, 0x00000000},
+ {0x0B9, 0x00000000},
+ {0x0B9, 0x00000200},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0FF, 0x00000000},
+ {0x0B9, 0x00000000},
+ {0x018, 0x00013124},
+ {0x05A, 0x0006808F},
+ {0x0ED, 0x00000008},
+ {0x033, 0x00000001},
+ {0x03F, 0x0000000F},
+ {0x0ED, 0x00000000},
+ {0x000, 0x00020000},
+ {0x018, 0x00010124},
+ {0x0EE, 0x00000800},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000002},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000003},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000006},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000007},
+ {0x0EE, 0x00000000},
+ {0x0EE, 0x00001000},
+ {0x033, 0x00000008},
+ {0x03F, 0x00000000},
+ {0x033, 0x00000009},
+ {0x03F, 0x00000001},
+ {0x033, 0x0000000A},
+ {0x03F, 0x00000003},
+ {0x033, 0x0000000B},
+ {0x03F, 0x00000103},
+ {0x033, 0x0000000C},
+ {0x03F, 0x00000107},
+ {0x033, 0x0000000D},
+ {0x03F, 0x00000207},
+ {0x033, 0x0000000E},
+ {0x03F, 0x00000307},
+ {0x033, 0x0000000F},
+ {0x03F, 0x00000307},
+ {0x0EE, 0x00000000},
+ {0x0EE, 0x00000200},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000000},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000001},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000002},
+ {0x033, 0x00000007},
+ {0x03F, 0x00000003},
+ {0x0EE, 0x00000000},
+ {0x011, 0x00014062},
+ {0x0EF, 0x00000010},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000DF3},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000DF3},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000A83},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000A83},
+ {0x033, 0x00000005},
+ {0x03F, 0x00000643},
+ {0x033, 0x00000006},
+ {0x03F, 0x00000643},
+ {0x0EF, 0x00000000},
+ {0x0EF, 0x00000100},
+ {0x033, 0x00000001},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000002},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000003},
+ {0x03F, 0x0001B5A9},
+ {0x033, 0x00000004},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000005},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000006},
+ {0x03F, 0x0001B589},
+ {0x033, 0x00000007},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000008},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000009},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000000A},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000000B},
+ {0x03F, 0x0001B5A9},
+ {0x033, 0x0000000C},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000000D},
+ {0x03F, 0x0001B5A9},
+ {0x033, 0x0000000E},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000000F},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000010},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000011},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000012},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000013},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000014},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000015},
+ {0x03F, 0x0001B589},
+ {0x033, 0x00000016},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000017},
+ {0x03F, 0x0001B5A9},
+ {0x033, 0x00000018},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000019},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001A},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001B},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001C},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001D},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001E},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x0000001F},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000020},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000021},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000022},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000023},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000024},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000025},
+ {0x03F, 0x0001B5A8},
+ {0x033, 0x00000022},
+ {0x03F, 0x0001B5A8},
+ {0x0EF, 0x00000000},
+ {0x0EF, 0x00000040},
+ {0x033, 0x00000001},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000002},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000003},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000004},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000005},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000006},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000007},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000008},
+ {0x03F, 0x000002C5},
+ {0x033, 0x00000009},
+ {0x03F, 0x000002C5},
+ {0x033, 0x0000000A},
+ {0x03F, 0x000002C5},
+ {0x033, 0x0000000B},
+ {0x03F, 0x000002C5},
+ {0x0EF, 0x00000000},
+ {0x059, 0x00050033},
+ {0x061, 0x0005F48A},
+ {0x062, 0x00077435},
+ {0x063, 0x000F80A2},
+ {0x065, 0x00018F22},
+ {0x067, 0x00008060},
+ {0x07E, 0x0009780B},
+ {0x0EE, 0x00000004},
+ {0x033, 0x0000000B},
+ {0x03F, 0x0000000B},
+ {0x033, 0x0000000C},
+ {0x03F, 0x00000012},
+ {0x033, 0x0000000D},
+ {0x03F, 0x00000019},
+ {0x033, 0x0000000F},
+ {0x03F, 0x0000000B},
+ {0x033, 0x00000010},
+ {0x03F, 0x00000012},
+ {0x033, 0x00000011},
+ {0x03F, 0x00000019},
+ {0x03F, 0x00000000},
+ {0x0EE, 0x00000000},
+ {0x0EE, 0x00000800},
+ {0x033, 0x00000000},
+ {0x03F, 0x00000001},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000002},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000003},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000007},
+ {0x0EE, 0x00000000},
+ {0x0EE, 0x00001000},
+ {0x033, 0x00000000},
+ {0x03F, 0x00003000},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000000},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000001},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000003},
+ {0x033, 0x00000004},
+ {0x03F, 0x00000007},
+ {0x033, 0x00000005},
+ {0x03F, 0x0000000F},
+ {0x033, 0x00000006},
+ {0x03F, 0x0000010F},
+ {0x033, 0x00000007},
+ {0x03F, 0x0000030F},
+ {0x0EE, 0x00000000},
+ {0x0EE, 0x00000200},
+ {0x033, 0x00000000},
+ {0x03F, 0x00000004},
+ {0x033, 0x00000001},
+ {0x03F, 0x00000005},
+ {0x033, 0x00000002},
+ {0x03F, 0x00000006},
+ {0x033, 0x00000003},
+ {0x03F, 0x00000007},
+ {0x0EE, 0x00000000},
+ {0x0EF, 0x00000080},
+ {0x033, 0x00000004},
+ {0x03E, 0x0000001D},
+ {0x03F, 0x0001A241},
+ {0x033, 0x00000005},
+ {0x03E, 0x0000001D},
+ {0x03F, 0x0001A241},
+ {0x033, 0x00000006},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000007},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000008},
+ {0x03E, 0x0000001D},
+ {0x03F, 0x0001A241},
+ {0x033, 0x00000009},
+ {0x03E, 0x0001A241},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000000A},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000000B},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000000C},
+ {0x03E, 0x0000001D},
+ {0x03F, 0x0001A241},
+ {0x033, 0x0000000D},
+ {0x03E, 0x0000001D},
+ {0x03F, 0x0001A241},
+ {0x033, 0x0000000E},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000000F},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000010},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000011},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000012},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000013},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000014},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000015},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000016},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000017},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000018},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000019},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x0000001A},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000001B},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000001C},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x0000001D},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x0000001E},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000001F},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000020},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000021},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x000199C1},
+ {0x033, 0x00000022},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000023},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000024},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000025},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000026},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000027},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000028},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000029},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x0000002A},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000002B},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000002C},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x0000002D},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x0000002E},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000002F},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000030},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000031},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000032},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000033},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000034},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000035},
+ {0x03E, 0x0000001C},
+ {0x03F, 0x0001E141},
+ {0x033, 0x00000036},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000037},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000038},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x00000039},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C1},
+ {0x033, 0x0000003A},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x033, 0x0000003B},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x033, 0x0000003C},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x033, 0x0000003D},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x033, 0x0000003E},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x033, 0x0000003F},
+ {0x03E, 0x0000001B},
+ {0x03F, 0x0001C3C3},
+ {0x0EF, 0x00000000},
+ {0x051, 0x0003D368},
+ {0x052, 0x000A3338},
+ {0x053, 0x000688AF},
+ {0x054, 0x00012C04},
+ {0x058, 0x00084221},
+ {0x05B, 0x000EB000},
+ {0x100EE, 0x00002000},
+ {0x10030, 0x000000F9},
+ {0x10030, 0x000004F6},
+ {0x10030, 0x000008F3},
+ {0x10030, 0x00000CF0},
+ {0x10030, 0x000010ED},
+ {0x10030, 0x000014EA},
+ {0x10030, 0x000018E7},
+ {0x10030, 0x00001CE4},
+ {0x10030, 0x000020E1},
+ {0x10030, 0x000024A4},
+ {0x10030, 0x000028A1},
+ {0x10030, 0x00002C9E},
+ {0x10030, 0x0000309B},
+ {0x10030, 0x0000341E},
+ {0x10030, 0x0000381B},
+ {0x10030, 0x00003C18},
+ {0x10030, 0x00004015},
+ {0x10030, 0x000200BC},
+ {0x10030, 0x000204B9},
+ {0x10030, 0x000208B6},
+ {0x10030, 0x00020CB3},
+ {0x10030, 0x000210B0},
+ {0x10030, 0x000214AD},
+ {0x10030, 0x0002186C},
+ {0x10030, 0x00021C69},
+ {0x10030, 0x00022066},
+ {0x10030, 0x00022426},
+ {0x10030, 0x00022823},
+ {0x10030, 0x00022C20},
+ {0x10030, 0x0002301D},
+ {0x10030, 0x0002341A},
+ {0x10030, 0x00023817},
+ {0x10030, 0x00023C14},
+ {0x10030, 0x00024011},
+ {0x10030, 0x000280BC},
+ {0x10030, 0x000284B9},
+ {0x10030, 0x000288B6},
+ {0x10030, 0x00028CB3},
+ {0x10030, 0x000290B0},
+ {0x10030, 0x000294AD},
+ {0x10030, 0x0002986C},
+ {0x10030, 0x00029C69},
+ {0x10030, 0x0002A066},
+ {0x10030, 0x0002A426},
+ {0x10030, 0x0002A823},
+ {0x10030, 0x0002AC20},
+ {0x10030, 0x0002B01D},
+ {0x10030, 0x0002B41A},
+ {0x10030, 0x0002B817},
+ {0x10030, 0x0002BC14},
+ {0x10030, 0x0002C011},
+ {0x10030, 0x000300BC},
+ {0x10030, 0x000304B9},
+ {0x10030, 0x000308B6},
+ {0x10030, 0x00030CB3},
+ {0x10030, 0x000310B0},
+ {0x10030, 0x000314AD},
+ {0x10030, 0x0003186C},
+ {0x10030, 0x00031C69},
+ {0x10030, 0x00032066},
+ {0x10030, 0x00032426},
+ {0x10030, 0x00032823},
+ {0x10030, 0x00032C20},
+ {0x10030, 0x0003301D},
+ {0x10030, 0x0003341A},
+ {0x10030, 0x00033817},
+ {0x10030, 0x00033C14},
+ {0x10030, 0x00034011},
+ {0x100EE, 0x00000000},
+ {0x100EE, 0x00004000},
+ {0x10030, 0x000201EF},
+ {0x10030, 0x000205E9},
+ {0x10030, 0x000209E3},
+ {0x10030, 0x00020DDD},
+ {0x10030, 0x000211D7},
+ {0x10030, 0x000215D1},
+ {0x10030, 0x00021919},
+ {0x10030, 0x00021D13},
+ {0x10030, 0x000220D9},
+ {0x10030, 0x000224D3},
+ {0x10030, 0x00022899},
+ {0x10030, 0x00022C93},
+ {0x10030, 0x00023059},
+ {0x10030, 0x00023453},
+ {0x10030, 0x00023819},
+ {0x10030, 0x00023C13},
+ {0x10030, 0x0002400D},
+ {0x10030, 0x00024407},
+ {0x10030, 0x000281EF},
+ {0x10030, 0x000285E9},
+ {0x10030, 0x000289E3},
+ {0x10030, 0x00028DDD},
+ {0x10030, 0x000291D7},
+ {0x10030, 0x000295D1},
+ {0x10030, 0x00029919},
+ {0x10030, 0x00029D13},
+ {0x10030, 0x0002A0D9},
+ {0x10030, 0x0002A4D3},
+ {0x10030, 0x0002A899},
+ {0x10030, 0x0002AC93},
+ {0x10030, 0x0002B059},
+ {0x10030, 0x0002B453},
+ {0x10030, 0x0002B819},
+ {0x10030, 0x0002BC13},
+ {0x10030, 0x0002C00D},
+ {0x10030, 0x0002C407},
+ {0x10030, 0x000301EF},
+ {0x10030, 0x000305E9},
+ {0x10030, 0x000309E3},
+ {0x10030, 0x00030DDD},
+ {0x10030, 0x000311D7},
+ {0x10030, 0x000315D1},
+ {0x10030, 0x00031919},
+ {0x10030, 0x00031D13},
+ {0x10030, 0x000320D9},
+ {0x10030, 0x000324D3},
+ {0x10030, 0x00032899},
+ {0x10030, 0x00032C93},
+ {0x10030, 0x00033059},
+ {0x10030, 0x00033453},
+ {0x10030, 0x00033819},
+ {0x10030, 0x00033C13},
+ {0x10030, 0x0003400D},
+ {0x10030, 0x00034407},
+ {0x100EE, 0x00000000},
+ {0x100EE, 0x00004000},
+ {0x10030, 0x000001EF},
+ {0x10030, 0x000005E9},
+ {0x10030, 0x000009E3},
+ {0x10030, 0x00000DDD},
+ {0x10030, 0x000011A5},
+ {0x10030, 0x0000159F},
+ {0x10030, 0x00001965},
+ {0x10030, 0x00001D5F},
+ {0x10030, 0x00002125},
+ {0x10030, 0x0000251F},
+ {0x10030, 0x000028E5},
+ {0x10030, 0x00002CDF},
+ {0x10030, 0x000030A5},
+ {0x10030, 0x0000349F},
+ {0x10030, 0x00003865},
+ {0x10030, 0x00003C5F},
+ {0x10030, 0x00004025},
+ {0x10030, 0x0000441F},
+ {0x100EE, 0x00000000},
+ {0x0EF, 0x00000008},
+ {0x033, 0x00000000},
+ {0x03F, 0x00000004},
+ {0x0EF, 0x00000000},
+ {0x005, 0x00000001},
+ {0x10005, 0x00000001},
+ {0x0FE, 0x00000022},
+};
+
+static const struct rtw89_reg2_def rtw89_8851b_phy_nctl_regs[] = {
+ {0x8000, 0x00000008},
+ {0x8008, 0x00000000},
+ {0x8004, 0xe8862b66},
+ {0x800c, 0x78000000},
+ {0x8010, 0x88015000},
+ {0x8014, 0x80010100},
+ {0x8018, 0x10010100},
+ {0x801c, 0xa210bc00},
+ {0x8020, 0x000403e0},
+ {0x8024, 0x00072160},
+ {0x8028, 0x00180e00},
+ {0x8030, 0x400000c0},
+ {0x8034, 0x11000830},
+ {0x8038, 0x40000000},
+ {0x803c, 0x00000008},
+ {0x8040, 0x00000046},
+ {0x8044, 0x0010001f},
+ {0x8048, 0x00000003},
+ {0x804c, 0x420840e0},
+ {0x8050, 0xce08cce0},
+ {0x8054, 0x420840e0},
+ {0x8058, 0xce08cce0},
+ {0x805c, 0x150c0b02},
+ {0x8060, 0x150c0b02},
+ {0x8064, 0x2aa00047},
+ {0x8074, 0x80000000},
+ {0x807c, 0x000000ee},
+ {0x8088, 0x80000000},
+ {0x808c, 0x00000000},
+ {0x80b0, 0x00000000},
+ {0x80cc, 0x00000000},
+ {0x80d0, 0x00000000},
+ {0x80ec, 0x00000002},
+ {0x8098, 0x0000ff00},
+ {0x8070, 0x00e80000},
+ {0x80b0, 0xffe00fff},
+ {0x809c, 0x0000001f},
+ {0x80b8, 0x00002000},
+ {0x80bc, 0x00050033},
+ {0xa400, 0x00000000},
+ {0xa404, 0x00000180},
+ {0xa408, 0x000001af},
+ {0xa40c, 0x000001e3},
+ {0xa410, 0x00000220},
+ {0xa414, 0x00000262},
+ {0xa418, 0x000002ac},
+ {0xa41c, 0x0000035e},
+ {0xa420, 0x000003c7},
+ {0xa424, 0x0000043d},
+ {0xa428, 0x000004c1},
+ {0xa42c, 0x00000556},
+ {0xa430, 0x000005fc},
+ {0xa434, 0x000006b7},
+ {0xa438, 0x00000789},
+ {0xa43c, 0x00000875},
+ {0xa440, 0x0000011f},
+ {0x8104, 0x00000000},
+ {0x810c, 0x00000000},
+ {0x8110, 0x00000000},
+ {0x8114, 0x00000000},
+ {0x8120, 0x10010000},
+ {0x8124, 0x00000000},
+ {0x8128, 0x00000200},
+ {0x812c, 0x0000c000},
+ {0x8130, 0x40000000},
+ {0x8138, 0x40000000},
+ {0x813c, 0x40000000},
+ {0x8140, 0x00000000},
+ {0x8144, 0x0b040b03},
+ {0x8148, 0x07020b04},
+ {0x814c, 0x07020b04},
+ {0x8150, 0xe4e40000},
+ {0x8158, 0xffffffff},
+ {0x815c, 0xffffffff},
+ {0x8160, 0xffffffff},
+ {0x8164, 0xffffffff},
+ {0x8168, 0xffffffff},
+ {0x816c, 0x1fffffff},
+ {0x81cc, 0x00000000},
+ {0x81dc, 0x00000002},
+ {0x81e0, 0x00000000},
+ {0x81e4, 0x00000001},
+ {0x81a0, 0x00000000},
+ {0x81ac, 0x3fc20400},
+ {0x81b0, 0x3f914100},
+ {0x81bc, 0x0000005b},
+ {0x81c0, 0x0000005b},
+ {0x81b4, 0x01e0f078},
+ {0x81b8, 0x01e0f078},
+ {0x81f0, 0x0000f078},
+ {0x81d8, 0x00000001},
+ {0x9500, 0x00000000},
+ {0x9504, 0x00000000},
+ {0x9508, 0x00000000},
+ {0x950c, 0x00000000},
+ {0x9510, 0x00000000},
+ {0x9514, 0x00000000},
+ {0x9518, 0x00000000},
+ {0x951c, 0x00000000},
+ {0x9520, 0x00000000},
+ {0x9524, 0x00000000},
+ {0x9528, 0x00000000},
+ {0x952c, 0x00000000},
+ {0x9530, 0x00000000},
+ {0x9534, 0x00000000},
+ {0x9538, 0x00000000},
+ {0x953c, 0x00000000},
+ {0x9540, 0x04000000},
+ {0x9544, 0x00000000},
+ {0x9548, 0x00000000},
+ {0x954c, 0x00000000},
+ {0x9550, 0x00000000},
+ {0x9554, 0x00000000},
+ {0x9558, 0x00000000},
+ {0x955c, 0x00000000},
+ {0x9560, 0x00000000},
+ {0x9564, 0x00000000},
+ {0x9568, 0x00000000},
+ {0x956c, 0x00000000},
+ {0x9570, 0x00000000},
+ {0x9574, 0x00000000},
+ {0x9578, 0x00000000},
+ {0x957c, 0x00000000},
+ {0x9580, 0x00000000},
+ {0x9584, 0x04000000},
+ {0x9588, 0x00000000},
+ {0x958c, 0x00000000},
+ {0x9590, 0x00000000},
+ {0x9594, 0x00000000},
+ {0x9598, 0x00000000},
+ {0x959c, 0x00000000},
+ {0x95a0, 0x00000000},
+ {0x95a4, 0x00000000},
+ {0x95a8, 0x00000000},
+ {0x95ac, 0x00000000},
+ {0x95b0, 0x00000000},
+ {0x95b4, 0x00000000},
+ {0x95b8, 0x00000000},
+ {0x95bc, 0x00000000},
+ {0x95c0, 0x00000000},
+ {0x95c4, 0x00000000},
+ {0x95c8, 0x04000000},
+ {0x95cc, 0x00000000},
+ {0x95d0, 0x00000000},
+ {0x95d4, 0x00000000},
+ {0x95d8, 0x00000000},
+ {0x95dc, 0x00000000},
+ {0x95e0, 0x00000000},
+ {0x95e4, 0x00000000},
+ {0x95e8, 0x00000000},
+ {0x95ec, 0x00000000},
+ {0x95f0, 0x00000000},
+ {0x95f4, 0x00000000},
+ {0x95f8, 0x00000000},
+ {0x95fc, 0x00000000},
+ {0x9600, 0x00000000},
+ {0x9604, 0x00000000},
+ {0x9608, 0x00000000},
+ {0x960c, 0x04000000},
+ {0x9610, 0x00000000},
+ {0x9614, 0x00000000},
+ {0x9618, 0x00000000},
+ {0x961c, 0x00000000},
+ {0x9620, 0x00000000},
+ {0x9624, 0x00000000},
+ {0x9628, 0x00000000},
+ {0x962c, 0x00000000},
+ {0x9630, 0x00000000},
+ {0x9634, 0x00000000},
+ {0x9638, 0x00000000},
+ {0x963c, 0x00000000},
+ {0x9640, 0x00000000},
+ {0x9644, 0x00000000},
+ {0x9648, 0x00000000},
+ {0x964c, 0x00000000},
+ {0x9650, 0x04000000},
+ {0x9654, 0x00000000},
+ {0x9658, 0x00000000},
+ {0x965c, 0x00000000},
+ {0x9660, 0x00000000},
+ {0x9664, 0x00000000},
+ {0x9668, 0x00000000},
+ {0x966c, 0x00000000},
+ {0x9670, 0x00000000},
+ {0x9674, 0x00000000},
+ {0x9678, 0x00000000},
+ {0x967c, 0x00000000},
+ {0x9680, 0x00000000},
+ {0x9684, 0x00000000},
+ {0x9688, 0x00000000},
+ {0x968c, 0x00000000},
+ {0x9690, 0x00000000},
+ {0x9694, 0x04000000},
+ {0x9698, 0x00000000},
+ {0x969c, 0x00000000},
+ {0x96a0, 0x00000000},
+ {0x96a4, 0x00000000},
+ {0x96a8, 0x00000000},
+ {0x96ac, 0x00000000},
+ {0x96b0, 0x00000000},
+ {0x96b4, 0x00000000},
+ {0x96b8, 0x00000000},
+ {0x96bc, 0x00000000},
+ {0x96c0, 0x00000000},
+ {0x96c4, 0x00000000},
+ {0x96c8, 0x00000000},
+ {0x96cc, 0x00000000},
+ {0x96d0, 0x00000000},
+ {0x96d4, 0x00000000},
+ {0x96d8, 0x04000000},
+ {0x96dc, 0x00000000},
+ {0x96e0, 0x00000000},
+ {0x96e4, 0x00000000},
+ {0x96e8, 0x00000000},
+ {0x96ec, 0x00000000},
+ {0x96f0, 0x00000000},
+ {0x96f4, 0x00000000},
+ {0x96f8, 0x00000000},
+ {0x96fc, 0x00000000},
+ {0x9700, 0x00000000},
+ {0x9704, 0x00000000},
+ {0x9708, 0x00000000},
+ {0x970c, 0x00000000},
+ {0x9710, 0x00000000},
+ {0x9714, 0x00000000},
+ {0x9718, 0x00000000},
+ {0x971c, 0x04000000},
+ {0x9720, 0x00000000},
+ {0x9724, 0x00000000},
+ {0x9728, 0x00000000},
+ {0x972c, 0x00000000},
+ {0x9730, 0x00000000},
+ {0x9734, 0x00000000},
+ {0x9738, 0x00000000},
+ {0x973c, 0x00000000},
+ {0x9740, 0x00000000},
+ {0x9744, 0x00000000},
+ {0x9748, 0x00000000},
+ {0x974c, 0x00000000},
+ {0x9750, 0x00000000},
+ {0x9754, 0x00000000},
+ {0x9758, 0x00000000},
+ {0x975c, 0x00000000},
+ {0x9760, 0x04000000},
+ {0x9764, 0x00000000},
+ {0x9768, 0x00000000},
+ {0x976c, 0x00000000},
+ {0x9770, 0x00000000},
+ {0x9774, 0x00000000},
+ {0x9778, 0x00000000},
+ {0x977c, 0x00000000},
+ {0x9780, 0x00000000},
+ {0x9784, 0x00000000},
+ {0x9788, 0x00000000},
+ {0x978c, 0x00000000},
+ {0x9790, 0x00000000},
+ {0x9794, 0x00000000},
+ {0x9798, 0x00000000},
+ {0x979c, 0x00000000},
+ {0x97a0, 0x00000000},
+ {0x97a4, 0x04000000},
+ {0x97a8, 0x00000000},
+ {0x97ac, 0x00000000},
+ {0x97b0, 0x00000000},
+ {0x97b4, 0x00000000},
+ {0x97b8, 0x00000000},
+ {0x97bc, 0x00000000},
+ {0x97c0, 0x00000000},
+ {0x97c4, 0x00000000},
+ {0x97c8, 0x00000000},
+ {0x97cc, 0x00000000},
+ {0x97d0, 0x00000000},
+ {0x97d4, 0x00000000},
+ {0x97d8, 0x00000000},
+ {0x97dc, 0x00000000},
+ {0x97e0, 0x00000000},
+ {0x97e4, 0x00000000},
+ {0x97e8, 0x04000000},
+ {0x97ec, 0x00000000},
+ {0x97f0, 0x00000000},
+ {0x97f4, 0x00000000},
+ {0x97f8, 0x00000000},
+ {0x97fc, 0x00000000},
+ {0x9800, 0x00000000},
+ {0x9804, 0x00000000},
+ {0x9808, 0x00000000},
+ {0x980c, 0x00000000},
+ {0x9810, 0x00000000},
+ {0x9814, 0x00000000},
+ {0x9818, 0x00000000},
+ {0x981c, 0x00000000},
+ {0x9820, 0x00000000},
+ {0x9824, 0x00000000},
+ {0x9828, 0x00000000},
+ {0x982c, 0x04000000},
+ {0x81d8, 0x00000000},
+ {0xb104, 0x2b251f19},
+ {0xb108, 0x433d3731},
+ {0xb10c, 0x5b554f49},
+ {0xb110, 0x736d6761},
+ {0xb114, 0x7f7f7f79},
+ {0xb118, 0x120f7f7f},
+ {0xb11c, 0x1e1b1815},
+ {0xb120, 0x2a272421},
+ {0xb124, 0x3633302d},
+ {0xb128, 0x3f3f3c39},
+ {0xb12c, 0x3f3f3f3f},
+ {0x8088, 0x00000110},
+ {0x8000, 0x00000008},
+ {0x8080, 0x00000005},
+ {0x8500, 0x80000008},
+ {0x8504, 0x43000004},
+ {0x8508, 0x4b044a00},
+ {0x850c, 0x40098604},
+ {0x8510, 0x0004e01f},
+ {0x8514, 0x74104b00},
+ {0x8518, 0x000021e0},
+ {0x851c, 0x74301658},
+ {0x8520, 0x43800004},
+ {0x8524, 0x4c000007},
+ {0x8528, 0x43000004},
+ {0x852c, 0x56030007},
+ {0x8530, 0x57000004},
+ {0x8534, 0x400042fe},
+ {0x8538, 0x50554200},
+ {0x853c, 0xb4183000},
+ {0x8540, 0xe537a50f},
+ {0x8544, 0xf12bf02b},
+ {0x8548, 0xf32bf22b},
+ {0x854c, 0xf62bf42b},
+ {0x8550, 0xf82bf72b},
+ {0x8554, 0xfa2bf92b},
+ {0x8558, 0xfd2bfc2b},
+ {0x855c, 0xe537fe2b},
+ {0x8560, 0xf12af02a},
+ {0x8564, 0xf32af22a},
+ {0x8568, 0xf52af42a},
+ {0x856c, 0x000bf62a},
+ {0x8570, 0xf028a511},
+ {0x8574, 0xf228f128},
+ {0x8578, 0xf428f328},
+ {0x857c, 0xf628f528},
+ {0x8580, 0xf828f728},
+ {0x8584, 0xfa28f928},
+ {0x8588, 0xfc28fb28},
+ {0x858c, 0xfe28fd28},
+ {0x8590, 0xf028ff28},
+ {0x8594, 0xf228f128},
+ {0x8598, 0x30750001},
+ {0x859c, 0x30753075},
+ {0x85a0, 0x30b63097},
+ {0x85a4, 0x30be30bb},
+ {0x85a8, 0x30d930cc},
+ {0x85ac, 0x316d30e6},
+ {0x85b0, 0x3189317f},
+ {0x85b4, 0x31d23193},
+ {0x85b8, 0x31e43210},
+ {0x85bc, 0x31e831dd},
+ {0x85c0, 0x322831e1},
+ {0x85c4, 0x323c3232},
+ {0x85c8, 0x32503246},
+ {0x85cc, 0x3264325a},
+ {0x85d0, 0x3278326e},
+ {0x85d4, 0x32983285},
+ {0x85d8, 0x32aa32a6},
+ {0x85dc, 0x330b32f3},
+ {0x85e0, 0x333f330c},
+ {0x85e4, 0x334c3341},
+ {0x85e8, 0xe35e0001},
+ {0x85ec, 0x20887410},
+ {0x85f0, 0x140f0200},
+ {0x85f4, 0x02002098},
+ {0x85f8, 0x7430140f},
+ {0x85fc, 0x5b10e39c},
+ {0x8600, 0x20807410},
+ {0x8604, 0x140f0000},
+ {0x8608, 0x56015507},
+ {0x860c, 0x7410e382},
+ {0x8610, 0x02002088},
+ {0x8614, 0x5517140f},
+ {0x8618, 0xe34ee382},
+ {0x861c, 0x468e7508},
+ {0x8620, 0xe0ace38c},
+ {0x8624, 0x5500f0e2},
+ {0x8628, 0x5501e37e},
+ {0x862c, 0x5b10f1de},
+ {0x8630, 0x20907410},
+ {0x8634, 0x140f0000},
+ {0x8638, 0xe3825507},
+ {0x863c, 0x20987410},
+ {0x8640, 0x140f0200},
+ {0x8644, 0xe3825517},
+ {0x8648, 0x46967509},
+ {0x864c, 0xe0ace38c},
+ {0x8650, 0xe37e5500},
+ {0x8654, 0x00015501},
+ {0x8658, 0x4d000007},
+ {0x865c, 0x74200004},
+ {0x8660, 0x57005710},
+ {0x8664, 0x9700140f},
+ {0x8668, 0x00017430},
+ {0x866c, 0xe39ce35e},
+ {0x8670, 0xe52a0bbd},
+ {0x8674, 0xe36a0001},
+ {0x8678, 0x0001e3c4},
+ {0x867c, 0x55005b30},
+ {0x8680, 0x46500005},
+ {0x8684, 0x74000004},
+ {0x8688, 0x1658e37e},
+ {0x868c, 0x74305501},
+ {0x8690, 0x46100005},
+ {0x8694, 0x00010004},
+ {0x8698, 0x30f8e35e},
+ {0x869c, 0xe52a0023},
+ {0x86a0, 0x54ed0002},
+ {0x86a4, 0x00230baa},
+ {0x86a8, 0x0002e52a},
+ {0x86ac, 0xe356e3e4},
+ {0x86b0, 0xe35e0001},
+ {0x86b4, 0x002230f3},
+ {0x86b8, 0x0002e52a},
+ {0x86bc, 0x0baa54ec},
+ {0x86c0, 0xe52a0022},
+ {0x86c4, 0xe3e40002},
+ {0x86c8, 0x0001e356},
+ {0x86cc, 0x0baae35e},
+ {0x86d0, 0xe3e430ec},
+ {0x86d4, 0x0001e356},
+ {0x86d8, 0x6d0f6c67},
+ {0x86dc, 0xe52ae39c},
+ {0x86e0, 0xe39c6c8b},
+ {0x86e4, 0x0bace52a},
+ {0x86e8, 0x6d0f6cb3},
+ {0x86ec, 0xe52ae39c},
+ {0x86f0, 0x6cdb0bad},
+ {0x86f4, 0xe39c6d0f},
+ {0x86f8, 0x6cf5e52a},
+ {0x86fc, 0xe39c6d0f},
+ {0x8700, 0x6c0be52a},
+ {0x8704, 0xe39c6d00},
+ {0x8708, 0x6c25e52a},
+ {0x870c, 0xe52ae39c},
+ {0x8710, 0x6c4df8c6},
+ {0x8714, 0xe52ae39c},
+ {0x8718, 0x6c75f9cf},
+ {0x871c, 0xe52ae39c},
+ {0x8720, 0xe39c6c99},
+ {0x8724, 0xfad6e52a},
+ {0x8728, 0x21e87410},
+ {0x872c, 0x6e670009},
+ {0x8730, 0xe3c46f0f},
+ {0x8734, 0x7410e52f},
+ {0x8738, 0x000b21e8},
+ {0x873c, 0xe3c46e8b},
+ {0x8740, 0x7410e52f},
+ {0x8744, 0x000d21e8},
+ {0x8748, 0x6f0f6eb3},
+ {0x874c, 0xe52fe3c4},
+ {0x8750, 0xfe07ff08},
+ {0x8754, 0x21e87410},
+ {0x8758, 0x6ec7000e},
+ {0x875c, 0xe52fe3c4},
+ {0x8760, 0x21e87410},
+ {0x8764, 0x6edb000f},
+ {0x8768, 0xe3c46f0f},
+ {0x876c, 0x7410e52f},
+ {0x8770, 0x001021e8},
+ {0x8774, 0xe3c46eef},
+ {0x8778, 0xff03e52f},
+ {0x877c, 0xe52ffe02},
+ {0x8780, 0x21e87410},
+ {0x8784, 0x6e110013},
+ {0x8788, 0xe3c46f00},
+ {0x878c, 0xff03e52f},
+ {0x8790, 0xe52ffe02},
+ {0x8794, 0x21e87410},
+ {0x8798, 0x6e250014},
+ {0x879c, 0xe52fe3c4},
+ {0x87a0, 0xff08fc24},
+ {0x87a4, 0x7410fe07},
+ {0x87a8, 0x001521e8},
+ {0x87ac, 0xe3c46e39},
+ {0x87b0, 0x7410e52f},
+ {0x87b4, 0x001621e8},
+ {0x87b8, 0xe3c46e4d},
+ {0x87bc, 0xfd27e52f},
+ {0x87c0, 0x21e87410},
+ {0x87c4, 0x6e750018},
+ {0x87c8, 0xe52fe3c4},
+ {0x87cc, 0x21e87410},
+ {0x87d0, 0x6e99001a},
+ {0x87d4, 0xe52fe3c4},
+ {0x87d8, 0xe36afe24},
+ {0x87dc, 0x63404380},
+ {0x87e0, 0x43006880},
+ {0x87e4, 0x31300bac},
+ {0x87e8, 0xe52f0022},
+ {0x87ec, 0x54ec0002},
+ {0x87f0, 0x00220baa},
+ {0x87f4, 0x0002e52f},
+ {0x87f8, 0xe362e3e4},
+ {0x87fc, 0xe36a0001},
+ {0x8800, 0x63404380},
+ {0x8804, 0x43006881},
+ {0x8808, 0x31210baa},
+ {0x880c, 0xe362e3e4},
+ {0x8810, 0xe36a0001},
+ {0x8814, 0x63414380},
+ {0x8818, 0x43006882},
+ {0x881c, 0x31140baa},
+ {0x8820, 0xe362e3e4},
+ {0x8824, 0x00040001},
+ {0x8828, 0x000742fc},
+ {0x882c, 0x00046001},
+ {0x8830, 0x00074200},
+ {0x8834, 0x62006220},
+ {0x8838, 0x55010004},
+ {0x883c, 0x66055b40},
+ {0x8840, 0x62000007},
+ {0x8844, 0xe40e6300},
+ {0x8848, 0x09000004},
+ {0x884c, 0x0b400a01},
+ {0x8850, 0x0e010d00},
+ {0x8854, 0x00040032},
+ {0x8858, 0x42fb950b},
+ {0x885c, 0x4d040007},
+ {0x8860, 0x42000004},
+ {0x8864, 0x00074380},
+ {0x8868, 0x00044d01},
+ {0x886c, 0x00074300},
+ {0x8870, 0x05a30562},
+ {0x8874, 0xe40e961f},
+ {0x8878, 0xe37e0004},
+ {0x887c, 0x06a20007},
+ {0x8880, 0xe40e07a3},
+ {0x8884, 0xe37e0004},
+ {0x8888, 0x0002e3fe},
+ {0x888c, 0x4380e406},
+ {0x8890, 0x4d000007},
+ {0x8894, 0x43000004},
+ {0x8898, 0x000742fe},
+ {0x889c, 0x00044d00},
+ {0x88a0, 0x00014200},
+ {0x88a4, 0x42fc0004},
+ {0x88a8, 0x60030007},
+ {0x88ac, 0x42000004},
+ {0x88b0, 0x00073199},
+ {0x88b4, 0x07a306a2},
+ {0x88b8, 0xe1eb31c5},
+ {0x88bc, 0xe1fee1f9},
+ {0x88c0, 0xe1eb0001},
+ {0x88c4, 0x0001e1fe},
+ {0x88c8, 0xe1f9e1f2},
+ {0x88cc, 0x0001e1fe},
+ {0x88d0, 0xe1fee1f2},
+ {0x88d4, 0x00040001},
+ {0x88d8, 0x000742fc},
+ {0x88dc, 0x00046003},
+ {0x88e0, 0x00014200},
+ {0x88e4, 0x42fc0004},
+ {0x88e8, 0x60010007},
+ {0x88ec, 0x42000004},
+ {0x88f0, 0x00070001},
+ {0x88f4, 0x62006220},
+ {0x88f8, 0x0001e406},
+ {0x88fc, 0x63000007},
+ {0x8900, 0x09000004},
+ {0x8904, 0x0e010a00},
+ {0x8908, 0x00070032},
+ {0x890c, 0xe40e06a2},
+ {0x8910, 0x0002e41a},
+ {0x8914, 0x000742fe},
+ {0x8918, 0x00044d00},
+ {0x891c, 0x00014200},
+ {0x8920, 0x77000005},
+ {0x8924, 0x52000007},
+ {0x8928, 0x42fe0004},
+ {0x892c, 0x60000007},
+ {0x8930, 0x42000004},
+ {0x8934, 0x60004380},
+ {0x8938, 0x62016100},
+ {0x893c, 0x68046310},
+ {0x8940, 0x41000005},
+ {0x8944, 0x00075500},
+ {0x8948, 0x00045c02},
+ {0x894c, 0x00014300},
+ {0x8950, 0x6c060005},
+ {0x8954, 0xe2aae298},
+ {0x8958, 0xe42ae285},
+ {0x895c, 0xe432e2f3},
+ {0x8960, 0x0001e30c},
+ {0x8964, 0x0005e285},
+ {0x8968, 0xe2986c06},
+ {0x896c, 0xe42ae4a9},
+ {0x8970, 0xe432e2f3},
+ {0x8974, 0x0001e30c},
+ {0x8978, 0x6c000005},
+ {0x897c, 0xe2aae298},
+ {0x8980, 0xe445e285},
+ {0x8984, 0xe44de2f3},
+ {0x8988, 0x0001e30c},
+ {0x898c, 0x0005e285},
+ {0x8990, 0xe2986c00},
+ {0x8994, 0xe445e4a9},
+ {0x8998, 0xe44de2f3},
+ {0x899c, 0x0001e30c},
+ {0x89a0, 0x6c040005},
+ {0x89a4, 0xe2aae298},
+ {0x89a8, 0xe460e285},
+ {0x89ac, 0xe468e2f3},
+ {0x89b0, 0x0001e30c},
+ {0x89b4, 0x0005e285},
+ {0x89b8, 0xe2986c04},
+ {0x89bc, 0xe460e4a9},
+ {0x89c0, 0xe468e2f3},
+ {0x89c4, 0x0001e30c},
+ {0x89c8, 0x6c020005},
+ {0x89cc, 0xe2aae298},
+ {0x89d0, 0xe47be285},
+ {0x89d4, 0xe483e2f3},
+ {0x89d8, 0x0001e30c},
+ {0x89dc, 0x0005e285},
+ {0x89e0, 0xe2986c02},
+ {0x89e4, 0xe47be4a9},
+ {0x89e8, 0xe483e2f3},
+ {0x89ec, 0x0001e30c},
+ {0x89f0, 0x43800004},
+ {0x89f4, 0x610a6008},
+ {0x89f8, 0x63ce6200},
+ {0x89fc, 0x60800006},
+ {0x8a00, 0x00047f00},
+ {0x8a04, 0xe4e04300},
+ {0x8a08, 0x00070001},
+ {0x8a0c, 0x4d015500},
+ {0x8a10, 0x74200004},
+ {0x8a14, 0x57107711},
+ {0x8a18, 0x140f5700},
+ {0x8a1c, 0x00077430},
+ {0x8a20, 0x00044d00},
+ {0x8a24, 0x00074380},
+ {0x8a28, 0x00047200},
+ {0x8a2c, 0x00014300},
+ {0x8a30, 0x74200004},
+ {0x8a34, 0x77000005},
+ {0x8a38, 0x73887e07},
+ {0x8a3c, 0x8f007380},
+ {0x8a40, 0x0004140f},
+ {0x8a44, 0x00057430},
+ {0x8a48, 0x00017300},
+ {0x8a4c, 0x0005e496},
+ {0x8a50, 0x00017300},
+ {0x8a54, 0x43800004},
+ {0x8a58, 0x0006b103},
+ {0x8a5c, 0x91037cdb},
+ {0x8a60, 0x40db0007},
+ {0x8a64, 0x43000004},
+ {0x8a68, 0x0005e496},
+ {0x8a6c, 0x00067380},
+ {0x8a70, 0x60025d01},
+ {0x8a74, 0xe4ba6200},
+ {0x8a78, 0x73000005},
+ {0x8a7c, 0x76080007},
+ {0x8a80, 0x00047578},
+ {0x8a84, 0x00074380},
+ {0x8a88, 0x5e005e01},
+ {0x8a8c, 0x0006140a},
+ {0x8a90, 0x7f006380},
+ {0x8a94, 0x00076080},
+ {0x8a98, 0x4e204c3f},
+ {0x8a9c, 0x73047280},
+ {0x8aa0, 0x140a7300},
+ {0x8aa4, 0x00044d20},
+ {0x8aa8, 0x00064300},
+ {0x8aac, 0x00077402},
+ {0x8ab0, 0x40004001},
+ {0x8ab4, 0x0006ab00},
+ {0x8ab8, 0x00077404},
+ {0x8abc, 0x40004001},
+ {0x8ac0, 0x140aab00},
+ {0x8ac4, 0x43800004},
+ {0x8ac8, 0x52800007},
+ {0x8acc, 0x140a5200},
+ {0x8ad0, 0x4d004c00},
+ {0x8ad4, 0x00064e00},
+ {0x8ad8, 0x63006080},
+ {0x8adc, 0x43000004},
+ {0x8ae0, 0x76000007},
+ {0x8ae4, 0x00040001},
+ {0x8ae8, 0xb1034380},
+ {0x8aec, 0x7cdb0006},
+ {0x8af0, 0x00079103},
+ {0x8af4, 0x000440db},
+ {0x8af8, 0xe4964300},
+ {0x8afc, 0xe4ba7e03},
+ {0x8b00, 0x43800004},
+ {0x8b04, 0x0006b103},
+ {0x8b08, 0x91037c5b},
+ {0x8b0c, 0x405b0007},
+ {0x8b10, 0x43000004},
+ {0x8b14, 0x00010001},
+ {0x8b18, 0x43800004},
+ {0x8b1c, 0x4e200007},
+ {0x8b20, 0x63800006},
+ {0x8b24, 0x5f807cdb},
+ {0x8b28, 0x43000004},
+ {0x8b2c, 0x76080007},
+ {0x8b30, 0x00057560},
+ {0x8b34, 0x00047380},
+ {0x8b38, 0x0005420e},
+ {0x8b3c, 0x14c86c01},
+ {0x8b40, 0x6c001432},
+ {0x8b44, 0x42000004},
+ {0x8b48, 0x43800004},
+ {0x8b4c, 0x5f000006},
+ {0x8b50, 0x73010007},
+ {0x8b54, 0x00047300},
+ {0x8b58, 0x0007420f},
+ {0x8b5c, 0x52005280},
+ {0x8b60, 0x0004140a},
+ {0x8b64, 0x00064200},
+ {0x8b68, 0x7c5b6300},
+ {0x8b6c, 0x4e000007},
+ {0x8b70, 0x43000004},
+ {0x8b74, 0x73000005},
+ {0x8b78, 0x76000007},
+ {0x8b7c, 0xe4c30001},
+ {0x8b80, 0x00040001},
+ {0x8b84, 0x60004380},
+ {0x8b88, 0x62016100},
+ {0x8b8c, 0x00066310},
+ {0x8b90, 0x00046000},
+ {0x8b94, 0x00014300},
+ {0x8b98, 0x0001e4e0},
+ {0x8b9c, 0x4e004f02},
+ {0x8ba0, 0x52015302},
+ {0x8ba4, 0x140f0001},
+ {0x8ba8, 0x00019700},
+ {0x8bac, 0x65014380},
+ {0x8bb0, 0x79007800},
+ {0x8bb4, 0x7b407a00},
+ {0x8bb8, 0x00014300},
+ {0x8bbc, 0x65004380},
+ {0x8bc0, 0x00014300},
+ {0x8bc4, 0x64014380},
+ {0x8bc8, 0x7d007c00},
+ {0x8bcc, 0x7f407e00},
+ {0x8bd0, 0x00014300},
+ {0x8bd4, 0x64004380},
+ {0x8bd8, 0x00014300},
+ {0x8bdc, 0x7b004380},
+ {0x8be0, 0x79007a04},
+ {0x8be4, 0x43007802},
+ {0x8be8, 0x33825509},
+ {0x8bec, 0x43800001},
+ {0x8bf0, 0x7a007b40},
+ {0x8bf4, 0x55194300},
+ {0x8bf8, 0x00013382},
+ {0x8bfc, 0x74007401},
+ {0x8c00, 0x00018e00},
+ {0x8c04, 0x52300007},
+ {0x8c08, 0x74310004},
+ {0x8c0c, 0x8e007430},
+ {0x8c10, 0x52200007},
+ {0x8c14, 0x00010004},
+ {0x8c18, 0x57005702},
+ {0x8c1c, 0x00018e00},
+ {0x8c20, 0x57425740},
+ {0x8c24, 0x8e005740},
+ {0x8c28, 0x00015700},
+ {0x8c2c, 0x561042ef},
+ {0x8c30, 0x42005600},
+ {0x8c34, 0x00018c00},
+ {0x8c38, 0xe3a75b20},
+ {0x8c3c, 0x54005480},
+ {0x8c40, 0x54005481},
+ {0x8c44, 0x54005482},
+ {0x8c48, 0xbf1ae3ac},
+ {0x8c4c, 0xe36e300b},
+ {0x8c50, 0xe390e377},
+ {0x8c54, 0x0001e523},
+ {0x8c58, 0x54c054bf},
+ {0x8c5c, 0x54c154a3},
+ {0x8c60, 0x4c1854a4},
+ {0x8c64, 0xbf091402},
+ {0x8c68, 0x54a454c2},
+ {0x8c6c, 0xbf051402},
+ {0x8c70, 0x54a354c1},
+ {0x8c74, 0xbf011402},
+ {0x8c78, 0x54dfe534},
+ {0x8c7c, 0x54bf0001},
+ {0x8c80, 0x050a54e5},
+ {0x8c84, 0x000154df},
+ {0x8c88, 0x00071657},
+ {0x8c8c, 0x00044c80},
+ {0x8c90, 0x43807430},
+ {0x8c94, 0x7e007f40},
+ {0x8c98, 0x7c027d00},
+ {0x8c9c, 0x5b404300},
+ {0x8ca0, 0x5c015501},
+ {0x8ca4, 0x5480e396},
+ {0x8ca8, 0x54815400},
+ {0x8cac, 0x54825400},
+ {0x8cb0, 0x00075400},
+ {0x8cb4, 0x00044c00},
+ {0x8cb8, 0xe3ac7410},
+ {0x8cbc, 0x300bbfe1},
+ {0x8cc0, 0x56005610},
+ {0x8cc4, 0x00018c00},
+ {0x8cc8, 0x57005704},
+ {0x8ccc, 0xa7038e00},
+ {0x8cd0, 0x33f0aff7},
+ {0x8cd4, 0xaf034019},
+ {0x8cd8, 0x33f0402b},
+ {0x8cdc, 0x33df402b},
+ {0x8ce0, 0x57005708},
+ {0x8ce4, 0x57818e00},
+ {0x8ce8, 0x8e005780},
+ {0x8cec, 0x00074380},
+ {0x8cf0, 0x5c005c01},
+ {0x8cf4, 0x00041403},
+ {0x8cf8, 0x00014300},
+ {0x8cfc, 0x0007427f},
+ {0x8d00, 0x62006280},
+ {0x8d04, 0x00049200},
+ {0x8d08, 0x00014200},
+ {0x8d0c, 0x0007427f},
+ {0x8d10, 0x63146394},
+ {0x8d14, 0x00049200},
+ {0x8d18, 0x00014200},
+ {0x8d1c, 0x42fe0004},
+ {0x8d20, 0x4d010007},
+ {0x8d24, 0x42000004},
+ {0x8d28, 0x140f7420},
+ {0x8d2c, 0x57005710},
+ {0x8d30, 0x0001141f},
+ {0x8d34, 0x42fe0004},
+ {0x8d38, 0x4d010007},
+ {0x8d3c, 0x42000004},
+ {0x8d40, 0x140f7420},
+ {0x8d44, 0x000742bf},
+ {0x8d48, 0x62006240},
+ {0x8d4c, 0x0004141f},
+ {0x8d50, 0x00014200},
+ {0x8d54, 0x5d060006},
+ {0x8d58, 0x61046003},
+ {0x8d5c, 0x00056201},
+ {0x8d60, 0x00017310},
+ {0x8d64, 0x43800004},
+ {0x8d68, 0x5e010007},
+ {0x8d6c, 0x140a5e00},
+ {0x8d70, 0x0006b103},
+ {0x8d74, 0x91037f07},
+ {0x8d78, 0x43070007},
+ {0x8d7c, 0x5c000006},
+ {0x8d80, 0x5e035d02},
+ {0x8d84, 0x43000004},
+ {0x8d88, 0x00060001},
+ {0x8d8c, 0x60005d04},
+ {0x8d90, 0x62016104},
+ {0x8d94, 0x73100005},
+ {0x8d98, 0x00040001},
+ {0x8d9c, 0x00074380},
+ {0x8da0, 0x5e005e01},
+ {0x8da4, 0xb103140a},
+ {0x8da8, 0x7fc60006},
+ {0x8dac, 0x00079103},
+ {0x8db0, 0x000643c6},
+ {0x8db4, 0x5d025c00},
+ {0x8db8, 0x00045e03},
+ {0x8dbc, 0x00014300},
+ {0x8dc0, 0x5d040006},
+ {0x8dc4, 0x61046000},
+ {0x8dc8, 0x00056201},
+ {0x8dcc, 0x00017310},
+ {0x8dd0, 0x43800004},
+ {0x8dd4, 0x5e010007},
+ {0x8dd8, 0x140a5e00},
+ {0x8ddc, 0x0006b103},
+ {0x8de0, 0x91037fc6},
+ {0x8de4, 0x43c60007},
+ {0x8de8, 0x5c000006},
+ {0x8dec, 0x5e035d02},
+ {0x8df0, 0x43000004},
+ {0x8df4, 0x00060001},
+ {0x8df8, 0x60025d00},
+ {0x8dfc, 0x62016100},
+ {0x8e00, 0x73000005},
+ {0x8e04, 0x00040001},
+ {0x8e08, 0x00074380},
+ {0x8e0c, 0x5e005e01},
+ {0x8e10, 0xb103140a},
+ {0x8e14, 0x7fc00006},
+ {0x8e18, 0x00079103},
+ {0x8e1c, 0x000643c0},
+ {0x8e20, 0x5d025c00},
+ {0x8e24, 0x00045e03},
+ {0x8e28, 0x00014300},
+ {0x8e2c, 0x7e020005},
+ {0x8e30, 0x42f70004},
+ {0x8e34, 0x6c080005},
+ {0x8e38, 0x42700004},
+ {0x8e3c, 0x73810005},
+ {0x8e40, 0x93007380},
+ {0x8e44, 0x42f70004},
+ {0x8e48, 0x6c000005},
+ {0x8e4c, 0x42000004},
+ {0x8e50, 0x00040001},
+ {0x8e54, 0x00074380},
+ {0x8e58, 0x73007304},
+ {0x8e5c, 0x72401405},
+ {0x8e60, 0x43000004},
+ {0x8e64, 0x74040006},
+ {0x8e68, 0x40010007},
+ {0x8e6c, 0xab004000},
+ {0x8e70, 0x0001140f},
+ {0x8e74, 0x140ae517},
+ {0x8e78, 0x140ae4c3},
+ {0x8e7c, 0x0001e51e},
+ {0x8e80, 0xe4c3e517},
+ {0x8e84, 0x00040001},
+ {0x8e88, 0x00047410},
+ {0x8e8c, 0x42f04380},
+ {0x8e90, 0x62080007},
+ {0x8e94, 0x24206301},
+ {0x8e98, 0x14c80000},
+ {0x8e9c, 0x00002428},
+ {0x8ea0, 0x1a4215f4},
+ {0x8ea4, 0x6300000b},
+ {0x8ea8, 0x42000004},
+ {0x8eac, 0x74304300},
+ {0x8eb0, 0x4380140f},
+ {0x8eb4, 0x73080007},
+ {0x8eb8, 0x00047300},
+ {0x8ebc, 0x00014300},
+ {0x8ec0, 0x4bf00007},
+ {0x8ec4, 0x490b4a8f},
+ {0x8ec8, 0x4a8e48f1},
+ {0x8ecc, 0x48a5490a},
+ {0x8ed0, 0x49094a8d},
+ {0x8ed4, 0x4a8c487d},
+ {0x8ed8, 0x48754908},
+ {0x8edc, 0x49074a8b},
+ {0x8ee0, 0x4a8a4889},
+ {0x8ee4, 0x48b74906},
+ {0x8ee8, 0x49054a89},
+ {0x8eec, 0x4a8848fc},
+ {0x8ef0, 0x48564905},
+ {0x8ef4, 0x49044a87},
+ {0x8ef8, 0x4a8648c1},
+ {0x8efc, 0x483d4904},
+ {0x8f00, 0x49034a85},
+ {0x8f04, 0x4a8448c7},
+ {0x8f08, 0x485e4903},
+ {0x8f0c, 0x49024a83},
+ {0x8f10, 0x4a8248ac},
+ {0x8f14, 0x48624902},
+ {0x8f18, 0x49024a81},
+ {0x8f1c, 0x4a804820},
+ {0x8f20, 0x48004900},
+ {0x8f24, 0x49014a90},
+ {0x8f28, 0x4a10481f},
+ {0x8f2c, 0x00060001},
+ {0x8f30, 0x5f005f80},
+ {0x8f34, 0x00059900},
+ {0x8f38, 0x00017300},
+ {0x8f3c, 0x63800006},
+ {0x8f40, 0x98006300},
+ {0x8f44, 0x549f0001},
+ {0x8f48, 0x5c015400},
+ {0x8f4c, 0x540054df},
+ {0x8f50, 0x00015c02},
+ {0x8f54, 0x07145c01},
+ {0x8f58, 0x5c025400},
+ {0x8f5c, 0x5c020001},
+ {0x8f60, 0x54000714},
+ {0x8f64, 0x00015c01},
+ {0x8f68, 0x4c184c98},
+ {0x8f6c, 0x00080001},
+ {0x8f70, 0x5c020004},
+ {0x8f74, 0x09017430},
+ {0x8f78, 0x0ba60c01},
+ {0x8f7c, 0x77800005},
+ {0x8f80, 0x52200007},
+ {0x8f84, 0x43800004},
+ {0x8f88, 0x610a6008},
+ {0x8f8c, 0x63c26200},
+ {0x8f90, 0x5c000007},
+ {0x8f94, 0x43000004},
+ {0x8f98, 0x00000001},
+ {0x8080, 0x00000004},
+ {0x8080, 0x00000000},
+ {0x8088, 0x00000000},
+};
+
+static const struct rtw89_txpwr_byrate_cfg rtw89_8851b_txpwr_byrate[] = {
+ { 0, 0, 0, 0, 4, 0x50505050, },
+ { 0, 0, 1, 0, 4, 0x54585858, },
+ { 0, 0, 1, 4, 4, 0x44484c50, },
+ { 0, 0, 2, 0, 4, 0x50545858, },
+ { 0, 0, 2, 4, 4, 0x4044484c, },
+ { 0, 0, 2, 8, 4, 0x3034383c, },
+ { 0, 0, 3, 0, 4, 0x50505050, },
+ { 0, 1, 2, 0, 4, 0x50545858, },
+ { 0, 1, 2, 4, 4, 0x4044484c, },
+ { 0, 1, 2, 8, 4, 0x3034383c, },
+ { 0, 1, 3, 0, 4, 0x50505050, },
+ { 0, 0, 4, 1, 4, 0x00000000, },
+ { 0, 0, 4, 0, 1, 0x00000000, },
+ { 1, 0, 1, 0, 4, 0x58585858, },
+ { 1, 0, 1, 4, 4, 0x484c5054, },
+ { 1, 0, 2, 0, 4, 0x54585858, },
+ { 1, 0, 2, 4, 4, 0x44484c50, },
+ { 1, 0, 2, 8, 4, 0x34383c40, },
+ { 1, 0, 3, 0, 4, 0x40404040, },
+ { 1, 1, 2, 0, 4, 0x54585858, },
+ { 1, 1, 2, 4, 4, 0x44484c50, },
+ { 1, 1, 2, 8, 4, 0x34383c40, },
+ { 1, 1, 3, 0, 4, 0x48484848, },
+ { 1, 0, 4, 0, 4, 0x00000000, },
+ { 2, 0, 1, 0, 4, 0x40404040, },
+ { 2, 0, 1, 4, 4, 0x383c4040, },
+ { 2, 0, 2, 0, 4, 0x40404040, },
+ { 2, 0, 2, 4, 4, 0x34383c40, },
+ { 2, 0, 2, 8, 4, 0x24282c30, },
+ { 2, 0, 3, 0, 4, 0x40404040, },
+ { 2, 1, 2, 0, 4, 0x40404040, },
+ { 2, 1, 2, 4, 4, 0x34383c40, },
+ { 2, 1, 2, 8, 4, 0x24282c30, },
+ { 2, 1, 3, 0, 4, 0x40404040, },
+ { 2, 0, 4, 0, 4, 0x00000000, },
+};
+
+static const s8 _txpwr_track_delta_swingidx_5ga_n[][DELTA_SWINGIDX_SIZE] = {
+ {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1},
+ {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,
+ 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4},
+};
+
+static const s8 _txpwr_track_delta_swingidx_5ga_p[][DELTA_SWINGIDX_SIZE] = {
+ {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ {0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,
+ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2},
+};
+
+static const s8 _txpwr_track_delta_swingidx_2ga_n[] = {
+ 0, 0, 0, 0, -1, -1, -1, -2, -2, -2, -2, -3, -3, -3, -3, -3,
+ -4, -4, -4, -4, -4, -5, -5, -5, -5, -5, -5, -6, -6, -6
+};
+
+static const s8 _txpwr_track_delta_swingidx_2ga_p[] = {
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4
+};
+
+static const s8 _txpwr_track_delta_swingidx_2g_cck_a_n[] = {
+ 0, 0, 0, 0, -1, -1, -1, -2, -2, -2, -2, -3, -3, -3, -3, -3,
+ -4, -4, -4, -4, -4, -5, -5, -5, -5, -5, -5, -6, -6, -6
+};
+
+static const s8 _txpwr_track_delta_swingidx_2g_cck_a_p[] = {
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4
+};
+
+const u8 rtw89_8851b_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
+ [RTW89_REGD_NUM] = {
+ [0][0][RTW89_ACMA] = 0,
+ [0][0][RTW89_CN] = 0,
+ [0][0][RTW89_ETSI] = 0,
+ [0][0][RTW89_FCC] = 1,
+ [0][0][RTW89_IC] = 1,
+ [0][0][RTW89_KCC] = 0,
+ [0][0][RTW89_MKK] = 0,
+ [0][0][RTW89_UK] = 0,
+ [0][1][RTW89_ACMA] = 0,
+ [0][1][RTW89_CN] = 0,
+ [0][1][RTW89_ETSI] = 0,
+ [0][1][RTW89_FCC] = 3,
+ [0][1][RTW89_IC] = 3,
+ [0][1][RTW89_KCC] = 0,
+ [0][1][RTW89_MKK] = 0,
+ [0][1][RTW89_UK] = 0,
+ [1][1][RTW89_ACMA] = 0,
+ [1][1][RTW89_CN] = 0,
+ [1][1][RTW89_ETSI] = 0,
+ [1][1][RTW89_FCC] = 3,
+ [1][1][RTW89_IC] = 3,
+ [1][1][RTW89_KCC] = 0,
+ [1][1][RTW89_MKK] = 0,
+ [1][1][RTW89_UK] = 0,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
+ [0][0][0][0][RTW89_WW][0] = 58,
+ [0][0][0][0][RTW89_WW][1] = 58,
+ [0][0][0][0][RTW89_WW][2] = 58,
+ [0][0][0][0][RTW89_WW][3] = 58,
+ [0][0][0][0][RTW89_WW][4] = 58,
+ [0][0][0][0][RTW89_WW][5] = 58,
+ [0][0][0][0][RTW89_WW][6] = 58,
+ [0][0][0][0][RTW89_WW][7] = 58,
+ [0][0][0][0][RTW89_WW][8] = 58,
+ [0][0][0][0][RTW89_WW][9] = 58,
+ [0][0][0][0][RTW89_WW][10] = 58,
+ [0][0][0][0][RTW89_WW][11] = 58,
+ [0][0][0][0][RTW89_WW][12] = 52,
+ [0][0][0][0][RTW89_WW][13] = 76,
+ [0][1][0][0][RTW89_WW][0] = 0,
+ [0][1][0][0][RTW89_WW][1] = 0,
+ [0][1][0][0][RTW89_WW][2] = 0,
+ [0][1][0][0][RTW89_WW][3] = 0,
+ [0][1][0][0][RTW89_WW][4] = 0,
+ [0][1][0][0][RTW89_WW][5] = 0,
+ [0][1][0][0][RTW89_WW][6] = 0,
+ [0][1][0][0][RTW89_WW][7] = 0,
+ [0][1][0][0][RTW89_WW][8] = 0,
+ [0][1][0][0][RTW89_WW][9] = 0,
+ [0][1][0][0][RTW89_WW][10] = 0,
+ [0][1][0][0][RTW89_WW][11] = 0,
+ [0][1][0][0][RTW89_WW][12] = 0,
+ [0][1][0][0][RTW89_WW][13] = 0,
+ [1][0][0][0][RTW89_WW][0] = 0,
+ [1][0][0][0][RTW89_WW][1] = 0,
+ [1][0][0][0][RTW89_WW][2] = 58,
+ [1][0][0][0][RTW89_WW][3] = 58,
+ [1][0][0][0][RTW89_WW][4] = 58,
+ [1][0][0][0][RTW89_WW][5] = 58,
+ [1][0][0][0][RTW89_WW][6] = 58,
+ [1][0][0][0][RTW89_WW][7] = 58,
+ [1][0][0][0][RTW89_WW][8] = 58,
+ [1][0][0][0][RTW89_WW][9] = 58,
+ [1][0][0][0][RTW89_WW][10] = 58,
+ [1][0][0][0][RTW89_WW][11] = 0,
+ [1][0][0][0][RTW89_WW][12] = 0,
+ [1][0][0][0][RTW89_WW][13] = 0,
+ [1][1][0][0][RTW89_WW][0] = 0,
+ [1][1][0][0][RTW89_WW][1] = 0,
+ [1][1][0][0][RTW89_WW][2] = 0,
+ [1][1][0][0][RTW89_WW][3] = 0,
+ [1][1][0][0][RTW89_WW][4] = 0,
+ [1][1][0][0][RTW89_WW][5] = 0,
+ [1][1][0][0][RTW89_WW][6] = 0,
+ [1][1][0][0][RTW89_WW][7] = 0,
+ [1][1][0][0][RTW89_WW][8] = 0,
+ [1][1][0][0][RTW89_WW][9] = 0,
+ [1][1][0][0][RTW89_WW][10] = 0,
+ [1][1][0][0][RTW89_WW][11] = 0,
+ [1][1][0][0][RTW89_WW][12] = 0,
+ [1][1][0][0][RTW89_WW][13] = 0,
+ [0][0][1][0][RTW89_WW][0] = 58,
+ [0][0][1][0][RTW89_WW][1] = 60,
+ [0][0][1][0][RTW89_WW][2] = 60,
+ [0][0][1][0][RTW89_WW][3] = 60,
+ [0][0][1][0][RTW89_WW][4] = 60,
+ [0][0][1][0][RTW89_WW][5] = 60,
+ [0][0][1][0][RTW89_WW][6] = 60,
+ [0][0][1][0][RTW89_WW][7] = 60,
+ [0][0][1][0][RTW89_WW][8] = 60,
+ [0][0][1][0][RTW89_WW][9] = 60,
+ [0][0][1][0][RTW89_WW][10] = 60,
+ [0][0][1][0][RTW89_WW][11] = 60,
+ [0][0][1][0][RTW89_WW][12] = 58,
+ [0][0][1][0][RTW89_WW][13] = 0,
+ [0][1][1][0][RTW89_WW][0] = 0,
+ [0][1][1][0][RTW89_WW][1] = 0,
+ [0][1][1][0][RTW89_WW][2] = 0,
+ [0][1][1][0][RTW89_WW][3] = 0,
+ [0][1][1][0][RTW89_WW][4] = 0,
+ [0][1][1][0][RTW89_WW][5] = 0,
+ [0][1][1][0][RTW89_WW][6] = 0,
+ [0][1][1][0][RTW89_WW][7] = 0,
+ [0][1][1][0][RTW89_WW][8] = 0,
+ [0][1][1][0][RTW89_WW][9] = 0,
+ [0][1][1][0][RTW89_WW][10] = 0,
+ [0][1][1][0][RTW89_WW][11] = 0,
+ [0][1][1][0][RTW89_WW][12] = 0,
+ [0][1][1][0][RTW89_WW][13] = 0,
+ [0][0][2][0][RTW89_WW][0] = 60,
+ [0][0][2][0][RTW89_WW][1] = 60,
+ [0][0][2][0][RTW89_WW][2] = 60,
+ [0][0][2][0][RTW89_WW][3] = 60,
+ [0][0][2][0][RTW89_WW][4] = 60,
+ [0][0][2][0][RTW89_WW][5] = 60,
+ [0][0][2][0][RTW89_WW][6] = 60,
+ [0][0][2][0][RTW89_WW][7] = 60,
+ [0][0][2][0][RTW89_WW][8] = 60,
+ [0][0][2][0][RTW89_WW][9] = 60,
+ [0][0][2][0][RTW89_WW][10] = 60,
+ [0][0][2][0][RTW89_WW][11] = 60,
+ [0][0][2][0][RTW89_WW][12] = 60,
+ [0][0][2][0][RTW89_WW][13] = 0,
+ [0][1][2][0][RTW89_WW][0] = 0,
+ [0][1][2][0][RTW89_WW][1] = 0,
+ [0][1][2][0][RTW89_WW][2] = 0,
+ [0][1][2][0][RTW89_WW][3] = 0,
+ [0][1][2][0][RTW89_WW][4] = 0,
+ [0][1][2][0][RTW89_WW][5] = 0,
+ [0][1][2][0][RTW89_WW][6] = 0,
+ [0][1][2][0][RTW89_WW][7] = 0,
+ [0][1][2][0][RTW89_WW][8] = 0,
+ [0][1][2][0][RTW89_WW][9] = 0,
+ [0][1][2][0][RTW89_WW][10] = 0,
+ [0][1][2][0][RTW89_WW][11] = 0,
+ [0][1][2][0][RTW89_WW][12] = 0,
+ [0][1][2][0][RTW89_WW][13] = 0,
+ [0][1][2][1][RTW89_WW][0] = 0,
+ [0][1][2][1][RTW89_WW][1] = 0,
+ [0][1][2][1][RTW89_WW][2] = 0,
+ [0][1][2][1][RTW89_WW][3] = 0,
+ [0][1][2][1][RTW89_WW][4] = 0,
+ [0][1][2][1][RTW89_WW][5] = 0,
+ [0][1][2][1][RTW89_WW][6] = 0,
+ [0][1][2][1][RTW89_WW][7] = 0,
+ [0][1][2][1][RTW89_WW][8] = 0,
+ [0][1][2][1][RTW89_WW][9] = 0,
+ [0][1][2][1][RTW89_WW][10] = 0,
+ [0][1][2][1][RTW89_WW][11] = 0,
+ [0][1][2][1][RTW89_WW][12] = 0,
+ [0][1][2][1][RTW89_WW][13] = 0,
+ [1][0][2][0][RTW89_WW][0] = 0,
+ [1][0][2][0][RTW89_WW][1] = 0,
+ [1][0][2][0][RTW89_WW][2] = 58,
+ [1][0][2][0][RTW89_WW][3] = 58,
+ [1][0][2][0][RTW89_WW][4] = 58,
+ [1][0][2][0][RTW89_WW][5] = 58,
+ [1][0][2][0][RTW89_WW][6] = 58,
+ [1][0][2][0][RTW89_WW][7] = 58,
+ [1][0][2][0][RTW89_WW][8] = 58,
+ [1][0][2][0][RTW89_WW][9] = 58,
+ [1][0][2][0][RTW89_WW][10] = 58,
+ [1][0][2][0][RTW89_WW][11] = 0,
+ [1][0][2][0][RTW89_WW][12] = 0,
+ [1][0][2][0][RTW89_WW][13] = 0,
+ [1][1][2][0][RTW89_WW][0] = 0,
+ [1][1][2][0][RTW89_WW][1] = 0,
+ [1][1][2][0][RTW89_WW][2] = 0,
+ [1][1][2][0][RTW89_WW][3] = 0,
+ [1][1][2][0][RTW89_WW][4] = 0,
+ [1][1][2][0][RTW89_WW][5] = 0,
+ [1][1][2][0][RTW89_WW][6] = 0,
+ [1][1][2][0][RTW89_WW][7] = 0,
+ [1][1][2][0][RTW89_WW][8] = 0,
+ [1][1][2][0][RTW89_WW][9] = 0,
+ [1][1][2][0][RTW89_WW][10] = 0,
+ [1][1][2][0][RTW89_WW][11] = 0,
+ [1][1][2][0][RTW89_WW][12] = 0,
+ [1][1][2][0][RTW89_WW][13] = 0,
+ [1][1][2][1][RTW89_WW][0] = 0,
+ [1][1][2][1][RTW89_WW][1] = 0,
+ [1][1][2][1][RTW89_WW][2] = 0,
+ [1][1][2][1][RTW89_WW][3] = 0,
+ [1][1][2][1][RTW89_WW][4] = 0,
+ [1][1][2][1][RTW89_WW][5] = 0,
+ [1][1][2][1][RTW89_WW][6] = 0,
+ [1][1][2][1][RTW89_WW][7] = 0,
+ [1][1][2][1][RTW89_WW][8] = 0,
+ [1][1][2][1][RTW89_WW][9] = 0,
+ [1][1][2][1][RTW89_WW][10] = 0,
+ [1][1][2][1][RTW89_WW][11] = 0,
+ [1][1][2][1][RTW89_WW][12] = 0,
+ [1][1][2][1][RTW89_WW][13] = 0,
+ [0][0][0][0][RTW89_FCC][0] = 84,
+ [0][0][0][0][RTW89_ETSI][0] = 58,
+ [0][0][0][0][RTW89_MKK][0] = 68,
+ [0][0][0][0][RTW89_IC][0] = 84,
+ [0][0][0][0][RTW89_KCC][0] = 68,
+ [0][0][0][0][RTW89_ACMA][0] = 58,
+ [0][0][0][0][RTW89_CN][0] = 60,
+ [0][0][0][0][RTW89_UK][0] = 58,
+ [0][0][0][0][RTW89_FCC][1] = 84,
+ [0][0][0][0][RTW89_ETSI][1] = 58,
+ [0][0][0][0][RTW89_MKK][1] = 68,
+ [0][0][0][0][RTW89_IC][1] = 84,
+ [0][0][0][0][RTW89_KCC][1] = 68,
+ [0][0][0][0][RTW89_ACMA][1] = 58,
+ [0][0][0][0][RTW89_CN][1] = 60,
+ [0][0][0][0][RTW89_UK][1] = 58,
+ [0][0][0][0][RTW89_FCC][2] = 84,
+ [0][0][0][0][RTW89_ETSI][2] = 58,
+ [0][0][0][0][RTW89_MKK][2] = 68,
+ [0][0][0][0][RTW89_IC][2] = 84,
+ [0][0][0][0][RTW89_KCC][2] = 68,
+ [0][0][0][0][RTW89_ACMA][2] = 58,
+ [0][0][0][0][RTW89_CN][2] = 60,
+ [0][0][0][0][RTW89_UK][2] = 58,
+ [0][0][0][0][RTW89_FCC][3] = 84,
+ [0][0][0][0][RTW89_ETSI][3] = 58,
+ [0][0][0][0][RTW89_MKK][3] = 68,
+ [0][0][0][0][RTW89_IC][3] = 84,
+ [0][0][0][0][RTW89_KCC][3] = 68,
+ [0][0][0][0][RTW89_ACMA][3] = 58,
+ [0][0][0][0][RTW89_CN][3] = 60,
+ [0][0][0][0][RTW89_UK][3] = 58,
+ [0][0][0][0][RTW89_FCC][4] = 84,
+ [0][0][0][0][RTW89_ETSI][4] = 58,
+ [0][0][0][0][RTW89_MKK][4] = 68,
+ [0][0][0][0][RTW89_IC][4] = 84,
+ [0][0][0][0][RTW89_KCC][4] = 68,
+ [0][0][0][0][RTW89_ACMA][4] = 58,
+ [0][0][0][0][RTW89_CN][4] = 60,
+ [0][0][0][0][RTW89_UK][4] = 58,
+ [0][0][0][0][RTW89_FCC][5] = 84,
+ [0][0][0][0][RTW89_ETSI][5] = 58,
+ [0][0][0][0][RTW89_MKK][5] = 68,
+ [0][0][0][0][RTW89_IC][5] = 84,
+ [0][0][0][0][RTW89_KCC][5] = 68,
+ [0][0][0][0][RTW89_ACMA][5] = 58,
+ [0][0][0][0][RTW89_CN][5] = 60,
+ [0][0][0][0][RTW89_UK][5] = 58,
+ [0][0][0][0][RTW89_FCC][6] = 84,
+ [0][0][0][0][RTW89_ETSI][6] = 58,
+ [0][0][0][0][RTW89_MKK][6] = 68,
+ [0][0][0][0][RTW89_IC][6] = 84,
+ [0][0][0][0][RTW89_KCC][6] = 68,
+ [0][0][0][0][RTW89_ACMA][6] = 58,
+ [0][0][0][0][RTW89_CN][6] = 60,
+ [0][0][0][0][RTW89_UK][6] = 58,
+ [0][0][0][0][RTW89_FCC][7] = 84,
+ [0][0][0][0][RTW89_ETSI][7] = 58,
+ [0][0][0][0][RTW89_MKK][7] = 68,
+ [0][0][0][0][RTW89_IC][7] = 84,
+ [0][0][0][0][RTW89_KCC][7] = 68,
+ [0][0][0][0][RTW89_ACMA][7] = 58,
+ [0][0][0][0][RTW89_CN][7] = 60,
+ [0][0][0][0][RTW89_UK][7] = 58,
+ [0][0][0][0][RTW89_FCC][8] = 84,
+ [0][0][0][0][RTW89_ETSI][8] = 58,
+ [0][0][0][0][RTW89_MKK][8] = 68,
+ [0][0][0][0][RTW89_IC][8] = 84,
+ [0][0][0][0][RTW89_KCC][8] = 68,
+ [0][0][0][0][RTW89_ACMA][8] = 58,
+ [0][0][0][0][RTW89_CN][8] = 60,
+ [0][0][0][0][RTW89_UK][8] = 58,
+ [0][0][0][0][RTW89_FCC][9] = 84,
+ [0][0][0][0][RTW89_ETSI][9] = 58,
+ [0][0][0][0][RTW89_MKK][9] = 68,
+ [0][0][0][0][RTW89_IC][9] = 84,
+ [0][0][0][0][RTW89_KCC][9] = 68,
+ [0][0][0][0][RTW89_ACMA][9] = 58,
+ [0][0][0][0][RTW89_CN][9] = 60,
+ [0][0][0][0][RTW89_UK][9] = 58,
+ [0][0][0][0][RTW89_FCC][10] = 82,
+ [0][0][0][0][RTW89_ETSI][10] = 58,
+ [0][0][0][0][RTW89_MKK][10] = 68,
+ [0][0][0][0][RTW89_IC][10] = 82,
+ [0][0][0][0][RTW89_KCC][10] = 68,
+ [0][0][0][0][RTW89_ACMA][10] = 58,
+ [0][0][0][0][RTW89_CN][10] = 60,
+ [0][0][0][0][RTW89_UK][10] = 58,
+ [0][0][0][0][RTW89_FCC][11] = 62,
+ [0][0][0][0][RTW89_ETSI][11] = 58,
+ [0][0][0][0][RTW89_MKK][11] = 68,
+ [0][0][0][0][RTW89_IC][11] = 62,
+ [0][0][0][0][RTW89_KCC][11] = 68,
+ [0][0][0][0][RTW89_ACMA][11] = 58,
+ [0][0][0][0][RTW89_CN][11] = 60,
+ [0][0][0][0][RTW89_UK][11] = 58,
+ [0][0][0][0][RTW89_FCC][12] = 52,
+ [0][0][0][0][RTW89_ETSI][12] = 58,
+ [0][0][0][0][RTW89_MKK][12] = 68,
+ [0][0][0][0][RTW89_IC][12] = 52,
+ [0][0][0][0][RTW89_KCC][12] = 68,
+ [0][0][0][0][RTW89_ACMA][12] = 58,
+ [0][0][0][0][RTW89_CN][12] = 60,
+ [0][0][0][0][RTW89_UK][12] = 58,
+ [0][0][0][0][RTW89_FCC][13] = 127,
+ [0][0][0][0][RTW89_ETSI][13] = 127,
+ [0][0][0][0][RTW89_MKK][13] = 76,
+ [0][0][0][0][RTW89_IC][13] = 127,
+ [0][0][0][0][RTW89_KCC][13] = 127,
+ [0][0][0][0][RTW89_ACMA][13] = 127,
+ [0][0][0][0][RTW89_CN][13] = 127,
+ [0][0][0][0][RTW89_UK][13] = 127,
+ [0][1][0][0][RTW89_FCC][0] = 127,
+ [0][1][0][0][RTW89_ETSI][0] = 127,
+ [0][1][0][0][RTW89_MKK][0] = 127,
+ [0][1][0][0][RTW89_IC][0] = 127,
+ [0][1][0][0][RTW89_KCC][0] = 127,
+ [0][1][0][0][RTW89_ACMA][0] = 127,
+ [0][1][0][0][RTW89_CN][0] = 127,
+ [0][1][0][0][RTW89_UK][0] = 127,
+ [0][1][0][0][RTW89_FCC][1] = 127,
+ [0][1][0][0][RTW89_ETSI][1] = 127,
+ [0][1][0][0][RTW89_MKK][1] = 127,
+ [0][1][0][0][RTW89_IC][1] = 127,
+ [0][1][0][0][RTW89_KCC][1] = 127,
+ [0][1][0][0][RTW89_ACMA][1] = 127,
+ [0][1][0][0][RTW89_CN][1] = 127,
+ [0][1][0][0][RTW89_UK][1] = 127,
+ [0][1][0][0][RTW89_FCC][2] = 127,
+ [0][1][0][0][RTW89_ETSI][2] = 127,
+ [0][1][0][0][RTW89_MKK][2] = 127,
+ [0][1][0][0][RTW89_IC][2] = 127,
+ [0][1][0][0][RTW89_KCC][2] = 127,
+ [0][1][0][0][RTW89_ACMA][2] = 127,
+ [0][1][0][0][RTW89_CN][2] = 127,
+ [0][1][0][0][RTW89_UK][2] = 127,
+ [0][1][0][0][RTW89_FCC][3] = 127,
+ [0][1][0][0][RTW89_ETSI][3] = 127,
+ [0][1][0][0][RTW89_MKK][3] = 127,
+ [0][1][0][0][RTW89_IC][3] = 127,
+ [0][1][0][0][RTW89_KCC][3] = 127,
+ [0][1][0][0][RTW89_ACMA][3] = 127,
+ [0][1][0][0][RTW89_CN][3] = 127,
+ [0][1][0][0][RTW89_UK][3] = 127,
+ [0][1][0][0][RTW89_FCC][4] = 127,
+ [0][1][0][0][RTW89_ETSI][4] = 127,
+ [0][1][0][0][RTW89_MKK][4] = 127,
+ [0][1][0][0][RTW89_IC][4] = 127,
+ [0][1][0][0][RTW89_KCC][4] = 127,
+ [0][1][0][0][RTW89_ACMA][4] = 127,
+ [0][1][0][0][RTW89_CN][4] = 127,
+ [0][1][0][0][RTW89_UK][4] = 127,
+ [0][1][0][0][RTW89_FCC][5] = 127,
+ [0][1][0][0][RTW89_ETSI][5] = 127,
+ [0][1][0][0][RTW89_MKK][5] = 127,
+ [0][1][0][0][RTW89_IC][5] = 127,
+ [0][1][0][0][RTW89_KCC][5] = 127,
+ [0][1][0][0][RTW89_ACMA][5] = 127,
+ [0][1][0][0][RTW89_CN][5] = 127,
+ [0][1][0][0][RTW89_UK][5] = 127,
+ [0][1][0][0][RTW89_FCC][6] = 127,
+ [0][1][0][0][RTW89_ETSI][6] = 127,
+ [0][1][0][0][RTW89_MKK][6] = 127,
+ [0][1][0][0][RTW89_IC][6] = 127,
+ [0][1][0][0][RTW89_KCC][6] = 127,
+ [0][1][0][0][RTW89_ACMA][6] = 127,
+ [0][1][0][0][RTW89_CN][6] = 127,
+ [0][1][0][0][RTW89_UK][6] = 127,
+ [0][1][0][0][RTW89_FCC][7] = 127,
+ [0][1][0][0][RTW89_ETSI][7] = 127,
+ [0][1][0][0][RTW89_MKK][7] = 127,
+ [0][1][0][0][RTW89_IC][7] = 127,
+ [0][1][0][0][RTW89_KCC][7] = 127,
+ [0][1][0][0][RTW89_ACMA][7] = 127,
+ [0][1][0][0][RTW89_CN][7] = 127,
+ [0][1][0][0][RTW89_UK][7] = 127,
+ [0][1][0][0][RTW89_FCC][8] = 127,
+ [0][1][0][0][RTW89_ETSI][8] = 127,
+ [0][1][0][0][RTW89_MKK][8] = 127,
+ [0][1][0][0][RTW89_IC][8] = 127,
+ [0][1][0][0][RTW89_KCC][8] = 127,
+ [0][1][0][0][RTW89_ACMA][8] = 127,
+ [0][1][0][0][RTW89_CN][8] = 127,
+ [0][1][0][0][RTW89_UK][8] = 127,
+ [0][1][0][0][RTW89_FCC][9] = 127,
+ [0][1][0][0][RTW89_ETSI][9] = 127,
+ [0][1][0][0][RTW89_MKK][9] = 127,
+ [0][1][0][0][RTW89_IC][9] = 127,
+ [0][1][0][0][RTW89_KCC][9] = 127,
+ [0][1][0][0][RTW89_ACMA][9] = 127,
+ [0][1][0][0][RTW89_CN][9] = 127,
+ [0][1][0][0][RTW89_UK][9] = 127,
+ [0][1][0][0][RTW89_FCC][10] = 127,
+ [0][1][0][0][RTW89_ETSI][10] = 127,
+ [0][1][0][0][RTW89_MKK][10] = 127,
+ [0][1][0][0][RTW89_IC][10] = 127,
+ [0][1][0][0][RTW89_KCC][10] = 127,
+ [0][1][0][0][RTW89_ACMA][10] = 127,
+ [0][1][0][0][RTW89_CN][10] = 127,
+ [0][1][0][0][RTW89_UK][10] = 127,
+ [0][1][0][0][RTW89_FCC][11] = 127,
+ [0][1][0][0][RTW89_ETSI][11] = 127,
+ [0][1][0][0][RTW89_MKK][11] = 127,
+ [0][1][0][0][RTW89_IC][11] = 127,
+ [0][1][0][0][RTW89_KCC][11] = 127,
+ [0][1][0][0][RTW89_ACMA][11] = 127,
+ [0][1][0][0][RTW89_CN][11] = 127,
+ [0][1][0][0][RTW89_UK][11] = 127,
+ [0][1][0][0][RTW89_FCC][12] = 127,
+ [0][1][0][0][RTW89_ETSI][12] = 127,
+ [0][1][0][0][RTW89_MKK][12] = 127,
+ [0][1][0][0][RTW89_IC][12] = 127,
+ [0][1][0][0][RTW89_KCC][12] = 127,
+ [0][1][0][0][RTW89_ACMA][12] = 127,
+ [0][1][0][0][RTW89_CN][12] = 127,
+ [0][1][0][0][RTW89_UK][12] = 127,
+ [0][1][0][0][RTW89_FCC][13] = 127,
+ [0][1][0][0][RTW89_ETSI][13] = 127,
+ [0][1][0][0][RTW89_MKK][13] = 127,
+ [0][1][0][0][RTW89_IC][13] = 127,
+ [0][1][0][0][RTW89_KCC][13] = 127,
+ [0][1][0][0][RTW89_ACMA][13] = 127,
+ [0][1][0][0][RTW89_CN][13] = 127,
+ [0][1][0][0][RTW89_UK][13] = 127,
+ [1][0][0][0][RTW89_FCC][0] = 127,
+ [1][0][0][0][RTW89_ETSI][0] = 127,
+ [1][0][0][0][RTW89_MKK][0] = 127,
+ [1][0][0][0][RTW89_IC][0] = 127,
+ [1][0][0][0][RTW89_KCC][0] = 127,
+ [1][0][0][0][RTW89_ACMA][0] = 127,
+ [1][0][0][0][RTW89_CN][0] = 127,
+ [1][0][0][0][RTW89_UK][0] = 127,
+ [1][0][0][0][RTW89_FCC][1] = 127,
+ [1][0][0][0][RTW89_ETSI][1] = 127,
+ [1][0][0][0][RTW89_MKK][1] = 127,
+ [1][0][0][0][RTW89_IC][1] = 127,
+ [1][0][0][0][RTW89_KCC][1] = 127,
+ [1][0][0][0][RTW89_ACMA][1] = 127,
+ [1][0][0][0][RTW89_CN][1] = 127,
+ [1][0][0][0][RTW89_UK][1] = 127,
+ [1][0][0][0][RTW89_FCC][2] = 127,
+ [1][0][0][0][RTW89_ETSI][2] = 58,
+ [1][0][0][0][RTW89_MKK][2] = 70,
+ [1][0][0][0][RTW89_IC][2] = 127,
+ [1][0][0][0][RTW89_KCC][2] = 68,
+ [1][0][0][0][RTW89_ACMA][2] = 58,
+ [1][0][0][0][RTW89_CN][2] = 60,
+ [1][0][0][0][RTW89_UK][2] = 58,
+ [1][0][0][0][RTW89_FCC][3] = 127,
+ [1][0][0][0][RTW89_ETSI][3] = 58,
+ [1][0][0][0][RTW89_MKK][3] = 76,
+ [1][0][0][0][RTW89_IC][3] = 127,
+ [1][0][0][0][RTW89_KCC][3] = 68,
+ [1][0][0][0][RTW89_ACMA][3] = 58,
+ [1][0][0][0][RTW89_CN][3] = 60,
+ [1][0][0][0][RTW89_UK][3] = 58,
+ [1][0][0][0][RTW89_FCC][4] = 127,
+ [1][0][0][0][RTW89_ETSI][4] = 58,
+ [1][0][0][0][RTW89_MKK][4] = 76,
+ [1][0][0][0][RTW89_IC][4] = 127,
+ [1][0][0][0][RTW89_KCC][4] = 68,
+ [1][0][0][0][RTW89_ACMA][4] = 58,
+ [1][0][0][0][RTW89_CN][4] = 60,
+ [1][0][0][0][RTW89_UK][4] = 58,
+ [1][0][0][0][RTW89_FCC][5] = 127,
+ [1][0][0][0][RTW89_ETSI][5] = 58,
+ [1][0][0][0][RTW89_MKK][5] = 76,
+ [1][0][0][0][RTW89_IC][5] = 127,
+ [1][0][0][0][RTW89_KCC][5] = 68,
+ [1][0][0][0][RTW89_ACMA][5] = 58,
+ [1][0][0][0][RTW89_CN][5] = 60,
+ [1][0][0][0][RTW89_UK][5] = 58,
+ [1][0][0][0][RTW89_FCC][6] = 127,
+ [1][0][0][0][RTW89_ETSI][6] = 58,
+ [1][0][0][0][RTW89_MKK][6] = 76,
+ [1][0][0][0][RTW89_IC][6] = 127,
+ [1][0][0][0][RTW89_KCC][6] = 68,
+ [1][0][0][0][RTW89_ACMA][6] = 58,
+ [1][0][0][0][RTW89_CN][6] = 60,
+ [1][0][0][0][RTW89_UK][6] = 58,
+ [1][0][0][0][RTW89_FCC][7] = 127,
+ [1][0][0][0][RTW89_ETSI][7] = 58,
+ [1][0][0][0][RTW89_MKK][7] = 76,
+ [1][0][0][0][RTW89_IC][7] = 127,
+ [1][0][0][0][RTW89_KCC][7] = 68,
+ [1][0][0][0][RTW89_ACMA][7] = 58,
+ [1][0][0][0][RTW89_CN][7] = 60,
+ [1][0][0][0][RTW89_UK][7] = 58,
+ [1][0][0][0][RTW89_FCC][8] = 127,
+ [1][0][0][0][RTW89_ETSI][8] = 58,
+ [1][0][0][0][RTW89_MKK][8] = 76,
+ [1][0][0][0][RTW89_IC][8] = 127,
+ [1][0][0][0][RTW89_KCC][8] = 68,
+ [1][0][0][0][RTW89_ACMA][8] = 58,
+ [1][0][0][0][RTW89_CN][8] = 60,
+ [1][0][0][0][RTW89_UK][8] = 58,
+ [1][0][0][0][RTW89_FCC][9] = 127,
+ [1][0][0][0][RTW89_ETSI][9] = 58,
+ [1][0][0][0][RTW89_MKK][9] = 76,
+ [1][0][0][0][RTW89_IC][9] = 127,
+ [1][0][0][0][RTW89_KCC][9] = 68,
+ [1][0][0][0][RTW89_ACMA][9] = 58,
+ [1][0][0][0][RTW89_CN][9] = 60,
+ [1][0][0][0][RTW89_UK][9] = 58,
+ [1][0][0][0][RTW89_FCC][10] = 127,
+ [1][0][0][0][RTW89_ETSI][10] = 58,
+ [1][0][0][0][RTW89_MKK][10] = 66,
+ [1][0][0][0][RTW89_IC][10] = 127,
+ [1][0][0][0][RTW89_KCC][10] = 68,
+ [1][0][0][0][RTW89_ACMA][10] = 58,
+ [1][0][0][0][RTW89_CN][10] = 60,
+ [1][0][0][0][RTW89_UK][10] = 58,
+ [1][0][0][0][RTW89_FCC][11] = 127,
+ [1][0][0][0][RTW89_ETSI][11] = 127,
+ [1][0][0][0][RTW89_MKK][11] = 127,
+ [1][0][0][0][RTW89_IC][11] = 127,
+ [1][0][0][0][RTW89_KCC][11] = 127,
+ [1][0][0][0][RTW89_ACMA][11] = 127,
+ [1][0][0][0][RTW89_CN][11] = 127,
+ [1][0][0][0][RTW89_UK][11] = 127,
+ [1][0][0][0][RTW89_FCC][12] = 127,
+ [1][0][0][0][RTW89_ETSI][12] = 127,
+ [1][0][0][0][RTW89_MKK][12] = 127,
+ [1][0][0][0][RTW89_IC][12] = 127,
+ [1][0][0][0][RTW89_KCC][12] = 127,
+ [1][0][0][0][RTW89_ACMA][12] = 127,
+ [1][0][0][0][RTW89_CN][12] = 127,
+ [1][0][0][0][RTW89_UK][12] = 127,
+ [1][0][0][0][RTW89_FCC][13] = 127,
+ [1][0][0][0][RTW89_ETSI][13] = 127,
+ [1][0][0][0][RTW89_MKK][13] = 127,
+ [1][0][0][0][RTW89_IC][13] = 127,
+ [1][0][0][0][RTW89_KCC][13] = 127,
+ [1][0][0][0][RTW89_ACMA][13] = 127,
+ [1][0][0][0][RTW89_CN][13] = 127,
+ [1][0][0][0][RTW89_UK][13] = 127,
+ [1][1][0][0][RTW89_FCC][0] = 127,
+ [1][1][0][0][RTW89_ETSI][0] = 127,
+ [1][1][0][0][RTW89_MKK][0] = 127,
+ [1][1][0][0][RTW89_IC][0] = 127,
+ [1][1][0][0][RTW89_KCC][0] = 127,
+ [1][1][0][0][RTW89_ACMA][0] = 127,
+ [1][1][0][0][RTW89_CN][0] = 127,
+ [1][1][0][0][RTW89_UK][0] = 127,
+ [1][1][0][0][RTW89_FCC][1] = 127,
+ [1][1][0][0][RTW89_ETSI][1] = 127,
+ [1][1][0][0][RTW89_MKK][1] = 127,
+ [1][1][0][0][RTW89_IC][1] = 127,
+ [1][1][0][0][RTW89_KCC][1] = 127,
+ [1][1][0][0][RTW89_ACMA][1] = 127,
+ [1][1][0][0][RTW89_CN][1] = 127,
+ [1][1][0][0][RTW89_UK][1] = 127,
+ [1][1][0][0][RTW89_FCC][2] = 127,
+ [1][1][0][0][RTW89_ETSI][2] = 127,
+ [1][1][0][0][RTW89_MKK][2] = 127,
+ [1][1][0][0][RTW89_IC][2] = 127,
+ [1][1][0][0][RTW89_KCC][2] = 127,
+ [1][1][0][0][RTW89_ACMA][2] = 127,
+ [1][1][0][0][RTW89_CN][2] = 127,
+ [1][1][0][0][RTW89_UK][2] = 127,
+ [1][1][0][0][RTW89_FCC][3] = 127,
+ [1][1][0][0][RTW89_ETSI][3] = 127,
+ [1][1][0][0][RTW89_MKK][3] = 127,
+ [1][1][0][0][RTW89_IC][3] = 127,
+ [1][1][0][0][RTW89_KCC][3] = 127,
+ [1][1][0][0][RTW89_ACMA][3] = 127,
+ [1][1][0][0][RTW89_CN][3] = 127,
+ [1][1][0][0][RTW89_UK][3] = 127,
+ [1][1][0][0][RTW89_FCC][4] = 127,
+ [1][1][0][0][RTW89_ETSI][4] = 127,
+ [1][1][0][0][RTW89_MKK][4] = 127,
+ [1][1][0][0][RTW89_IC][4] = 127,
+ [1][1][0][0][RTW89_KCC][4] = 127,
+ [1][1][0][0][RTW89_ACMA][4] = 127,
+ [1][1][0][0][RTW89_CN][4] = 127,
+ [1][1][0][0][RTW89_UK][4] = 127,
+ [1][1][0][0][RTW89_FCC][5] = 127,
+ [1][1][0][0][RTW89_ETSI][5] = 127,
+ [1][1][0][0][RTW89_MKK][5] = 127,
+ [1][1][0][0][RTW89_IC][5] = 127,
+ [1][1][0][0][RTW89_KCC][5] = 127,
+ [1][1][0][0][RTW89_ACMA][5] = 127,
+ [1][1][0][0][RTW89_CN][5] = 127,
+ [1][1][0][0][RTW89_UK][5] = 127,
+ [1][1][0][0][RTW89_FCC][6] = 127,
+ [1][1][0][0][RTW89_ETSI][6] = 127,
+ [1][1][0][0][RTW89_MKK][6] = 127,
+ [1][1][0][0][RTW89_IC][6] = 127,
+ [1][1][0][0][RTW89_KCC][6] = 127,
+ [1][1][0][0][RTW89_ACMA][6] = 127,
+ [1][1][0][0][RTW89_CN][6] = 127,
+ [1][1][0][0][RTW89_UK][6] = 127,
+ [1][1][0][0][RTW89_FCC][7] = 127,
+ [1][1][0][0][RTW89_ETSI][7] = 127,
+ [1][1][0][0][RTW89_MKK][7] = 127,
+ [1][1][0][0][RTW89_IC][7] = 127,
+ [1][1][0][0][RTW89_KCC][7] = 127,
+ [1][1][0][0][RTW89_ACMA][7] = 127,
+ [1][1][0][0][RTW89_CN][7] = 127,
+ [1][1][0][0][RTW89_UK][7] = 127,
+ [1][1][0][0][RTW89_FCC][8] = 127,
+ [1][1][0][0][RTW89_ETSI][8] = 127,
+ [1][1][0][0][RTW89_MKK][8] = 127,
+ [1][1][0][0][RTW89_IC][8] = 127,
+ [1][1][0][0][RTW89_KCC][8] = 127,
+ [1][1][0][0][RTW89_ACMA][8] = 127,
+ [1][1][0][0][RTW89_CN][8] = 127,
+ [1][1][0][0][RTW89_UK][8] = 127,
+ [1][1][0][0][RTW89_FCC][9] = 127,
+ [1][1][0][0][RTW89_ETSI][9] = 127,
+ [1][1][0][0][RTW89_MKK][9] = 127,
+ [1][1][0][0][RTW89_IC][9] = 127,
+ [1][1][0][0][RTW89_KCC][9] = 127,
+ [1][1][0][0][RTW89_ACMA][9] = 127,
+ [1][1][0][0][RTW89_CN][9] = 127,
+ [1][1][0][0][RTW89_UK][9] = 127,
+ [1][1][0][0][RTW89_FCC][10] = 127,
+ [1][1][0][0][RTW89_ETSI][10] = 127,
+ [1][1][0][0][RTW89_MKK][10] = 127,
+ [1][1][0][0][RTW89_IC][10] = 127,
+ [1][1][0][0][RTW89_KCC][10] = 127,
+ [1][1][0][0][RTW89_ACMA][10] = 127,
+ [1][1][0][0][RTW89_CN][10] = 127,
+ [1][1][0][0][RTW89_UK][10] = 127,
+ [1][1][0][0][RTW89_FCC][11] = 127,
+ [1][1][0][0][RTW89_ETSI][11] = 127,
+ [1][1][0][0][RTW89_MKK][11] = 127,
+ [1][1][0][0][RTW89_IC][11] = 127,
+ [1][1][0][0][RTW89_KCC][11] = 127,
+ [1][1][0][0][RTW89_ACMA][11] = 127,
+ [1][1][0][0][RTW89_CN][11] = 127,
+ [1][1][0][0][RTW89_UK][11] = 127,
+ [1][1][0][0][RTW89_FCC][12] = 127,
+ [1][1][0][0][RTW89_ETSI][12] = 127,
+ [1][1][0][0][RTW89_MKK][12] = 127,
+ [1][1][0][0][RTW89_IC][12] = 127,
+ [1][1][0][0][RTW89_KCC][12] = 127,
+ [1][1][0][0][RTW89_ACMA][12] = 127,
+ [1][1][0][0][RTW89_CN][12] = 127,
+ [1][1][0][0][RTW89_UK][12] = 127,
+ [1][1][0][0][RTW89_FCC][13] = 127,
+ [1][1][0][0][RTW89_ETSI][13] = 127,
+ [1][1][0][0][RTW89_MKK][13] = 127,
+ [1][1][0][0][RTW89_IC][13] = 127,
+ [1][1][0][0][RTW89_KCC][13] = 127,
+ [1][1][0][0][RTW89_ACMA][13] = 127,
+ [1][1][0][0][RTW89_CN][13] = 127,
+ [1][1][0][0][RTW89_UK][13] = 127,
+ [0][0][1][0][RTW89_FCC][0] = 80,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 72,
+ [0][0][1][0][RTW89_IC][0] = 80,
+ [0][0][1][0][RTW89_KCC][0] = 78,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CN][0] = 60,
+ [0][0][1][0][RTW89_UK][0] = 58,
+ [0][0][1][0][RTW89_FCC][1] = 80,
+ [0][0][1][0][RTW89_ETSI][1] = 60,
+ [0][0][1][0][RTW89_MKK][1] = 74,
+ [0][0][1][0][RTW89_IC][1] = 80,
+ [0][0][1][0][RTW89_KCC][1] = 78,
+ [0][0][1][0][RTW89_ACMA][1] = 60,
+ [0][0][1][0][RTW89_CN][1] = 60,
+ [0][0][1][0][RTW89_UK][1] = 60,
+ [0][0][1][0][RTW89_FCC][2] = 84,
+ [0][0][1][0][RTW89_ETSI][2] = 60,
+ [0][0][1][0][RTW89_MKK][2] = 74,
+ [0][0][1][0][RTW89_IC][2] = 84,
+ [0][0][1][0][RTW89_KCC][2] = 78,
+ [0][0][1][0][RTW89_ACMA][2] = 60,
+ [0][0][1][0][RTW89_CN][2] = 60,
+ [0][0][1][0][RTW89_UK][2] = 60,
+ [0][0][1][0][RTW89_FCC][3] = 84,
+ [0][0][1][0][RTW89_ETSI][3] = 60,
+ [0][0][1][0][RTW89_MKK][3] = 74,
+ [0][0][1][0][RTW89_IC][3] = 84,
+ [0][0][1][0][RTW89_KCC][3] = 78,
+ [0][0][1][0][RTW89_ACMA][3] = 60,
+ [0][0][1][0][RTW89_CN][3] = 60,
+ [0][0][1][0][RTW89_UK][3] = 60,
+ [0][0][1][0][RTW89_FCC][4] = 84,
+ [0][0][1][0][RTW89_ETSI][4] = 60,
+ [0][0][1][0][RTW89_MKK][4] = 74,
+ [0][0][1][0][RTW89_IC][4] = 84,
+ [0][0][1][0][RTW89_KCC][4] = 76,
+ [0][0][1][0][RTW89_ACMA][4] = 60,
+ [0][0][1][0][RTW89_CN][4] = 60,
+ [0][0][1][0][RTW89_UK][4] = 60,
+ [0][0][1][0][RTW89_FCC][5] = 84,
+ [0][0][1][0][RTW89_ETSI][5] = 60,
+ [0][0][1][0][RTW89_MKK][5] = 74,
+ [0][0][1][0][RTW89_IC][5] = 84,
+ [0][0][1][0][RTW89_KCC][5] = 76,
+ [0][0][1][0][RTW89_ACMA][5] = 60,
+ [0][0][1][0][RTW89_CN][5] = 60,
+ [0][0][1][0][RTW89_UK][5] = 60,
+ [0][0][1][0][RTW89_FCC][6] = 84,
+ [0][0][1][0][RTW89_ETSI][6] = 60,
+ [0][0][1][0][RTW89_MKK][6] = 74,
+ [0][0][1][0][RTW89_IC][6] = 84,
+ [0][0][1][0][RTW89_KCC][6] = 76,
+ [0][0][1][0][RTW89_ACMA][6] = 60,
+ [0][0][1][0][RTW89_CN][6] = 60,
+ [0][0][1][0][RTW89_UK][6] = 60,
+ [0][0][1][0][RTW89_FCC][7] = 84,
+ [0][0][1][0][RTW89_ETSI][7] = 60,
+ [0][0][1][0][RTW89_MKK][7] = 74,
+ [0][0][1][0][RTW89_IC][7] = 84,
+ [0][0][1][0][RTW89_KCC][7] = 76,
+ [0][0][1][0][RTW89_ACMA][7] = 60,
+ [0][0][1][0][RTW89_CN][7] = 60,
+ [0][0][1][0][RTW89_UK][7] = 60,
+ [0][0][1][0][RTW89_FCC][8] = 80,
+ [0][0][1][0][RTW89_ETSI][8] = 60,
+ [0][0][1][0][RTW89_MKK][8] = 74,
+ [0][0][1][0][RTW89_IC][8] = 80,
+ [0][0][1][0][RTW89_KCC][8] = 76,
+ [0][0][1][0][RTW89_ACMA][8] = 60,
+ [0][0][1][0][RTW89_CN][8] = 60,
+ [0][0][1][0][RTW89_UK][8] = 60,
+ [0][0][1][0][RTW89_FCC][9] = 76,
+ [0][0][1][0][RTW89_ETSI][9] = 60,
+ [0][0][1][0][RTW89_MKK][9] = 74,
+ [0][0][1][0][RTW89_IC][9] = 76,
+ [0][0][1][0][RTW89_KCC][9] = 74,
+ [0][0][1][0][RTW89_ACMA][9] = 60,
+ [0][0][1][0][RTW89_CN][9] = 60,
+ [0][0][1][0][RTW89_UK][9] = 60,
+ [0][0][1][0][RTW89_FCC][10] = 76,
+ [0][0][1][0][RTW89_ETSI][10] = 60,
+ [0][0][1][0][RTW89_MKK][10] = 74,
+ [0][0][1][0][RTW89_IC][10] = 76,
+ [0][0][1][0][RTW89_KCC][10] = 74,
+ [0][0][1][0][RTW89_ACMA][10] = 60,
+ [0][0][1][0][RTW89_CN][10] = 60,
+ [0][0][1][0][RTW89_UK][10] = 60,
+ [0][0][1][0][RTW89_FCC][11] = 68,
+ [0][0][1][0][RTW89_ETSI][11] = 60,
+ [0][0][1][0][RTW89_MKK][11] = 74,
+ [0][0][1][0][RTW89_IC][11] = 68,
+ [0][0][1][0][RTW89_KCC][11] = 74,
+ [0][0][1][0][RTW89_ACMA][11] = 60,
+ [0][0][1][0][RTW89_CN][11] = 60,
+ [0][0][1][0][RTW89_UK][11] = 60,
+ [0][0][1][0][RTW89_FCC][12] = 64,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 70,
+ [0][0][1][0][RTW89_IC][12] = 64,
+ [0][0][1][0][RTW89_KCC][12] = 74,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CN][12] = 60,
+ [0][0][1][0][RTW89_UK][12] = 58,
+ [0][0][1][0][RTW89_FCC][13] = 127,
+ [0][0][1][0][RTW89_ETSI][13] = 127,
+ [0][0][1][0][RTW89_MKK][13] = 127,
+ [0][0][1][0][RTW89_IC][13] = 127,
+ [0][0][1][0][RTW89_KCC][13] = 127,
+ [0][0][1][0][RTW89_ACMA][13] = 127,
+ [0][0][1][0][RTW89_CN][13] = 127,
+ [0][0][1][0][RTW89_UK][13] = 127,
+ [0][1][1][0][RTW89_FCC][0] = 127,
+ [0][1][1][0][RTW89_ETSI][0] = 127,
+ [0][1][1][0][RTW89_MKK][0] = 127,
+ [0][1][1][0][RTW89_IC][0] = 127,
+ [0][1][1][0][RTW89_KCC][0] = 127,
+ [0][1][1][0][RTW89_ACMA][0] = 127,
+ [0][1][1][0][RTW89_CN][0] = 127,
+ [0][1][1][0][RTW89_UK][0] = 127,
+ [0][1][1][0][RTW89_FCC][1] = 127,
+ [0][1][1][0][RTW89_ETSI][1] = 127,
+ [0][1][1][0][RTW89_MKK][1] = 127,
+ [0][1][1][0][RTW89_IC][1] = 127,
+ [0][1][1][0][RTW89_KCC][1] = 127,
+ [0][1][1][0][RTW89_ACMA][1] = 127,
+ [0][1][1][0][RTW89_CN][1] = 127,
+ [0][1][1][0][RTW89_UK][1] = 127,
+ [0][1][1][0][RTW89_FCC][2] = 127,
+ [0][1][1][0][RTW89_ETSI][2] = 127,
+ [0][1][1][0][RTW89_MKK][2] = 127,
+ [0][1][1][0][RTW89_IC][2] = 127,
+ [0][1][1][0][RTW89_KCC][2] = 127,
+ [0][1][1][0][RTW89_ACMA][2] = 127,
+ [0][1][1][0][RTW89_CN][2] = 127,
+ [0][1][1][0][RTW89_UK][2] = 127,
+ [0][1][1][0][RTW89_FCC][3] = 127,
+ [0][1][1][0][RTW89_ETSI][3] = 127,
+ [0][1][1][0][RTW89_MKK][3] = 127,
+ [0][1][1][0][RTW89_IC][3] = 127,
+ [0][1][1][0][RTW89_KCC][3] = 127,
+ [0][1][1][0][RTW89_ACMA][3] = 127,
+ [0][1][1][0][RTW89_CN][3] = 127,
+ [0][1][1][0][RTW89_UK][3] = 127,
+ [0][1][1][0][RTW89_FCC][4] = 127,
+ [0][1][1][0][RTW89_ETSI][4] = 127,
+ [0][1][1][0][RTW89_MKK][4] = 127,
+ [0][1][1][0][RTW89_IC][4] = 127,
+ [0][1][1][0][RTW89_KCC][4] = 127,
+ [0][1][1][0][RTW89_ACMA][4] = 127,
+ [0][1][1][0][RTW89_CN][4] = 127,
+ [0][1][1][0][RTW89_UK][4] = 127,
+ [0][1][1][0][RTW89_FCC][5] = 127,
+ [0][1][1][0][RTW89_ETSI][5] = 127,
+ [0][1][1][0][RTW89_MKK][5] = 127,
+ [0][1][1][0][RTW89_IC][5] = 127,
+ [0][1][1][0][RTW89_KCC][5] = 127,
+ [0][1][1][0][RTW89_ACMA][5] = 127,
+ [0][1][1][0][RTW89_CN][5] = 127,
+ [0][1][1][0][RTW89_UK][5] = 127,
+ [0][1][1][0][RTW89_FCC][6] = 127,
+ [0][1][1][0][RTW89_ETSI][6] = 127,
+ [0][1][1][0][RTW89_MKK][6] = 127,
+ [0][1][1][0][RTW89_IC][6] = 127,
+ [0][1][1][0][RTW89_KCC][6] = 127,
+ [0][1][1][0][RTW89_ACMA][6] = 127,
+ [0][1][1][0][RTW89_CN][6] = 127,
+ [0][1][1][0][RTW89_UK][6] = 127,
+ [0][1][1][0][RTW89_FCC][7] = 127,
+ [0][1][1][0][RTW89_ETSI][7] = 127,
+ [0][1][1][0][RTW89_MKK][7] = 127,
+ [0][1][1][0][RTW89_IC][7] = 127,
+ [0][1][1][0][RTW89_KCC][7] = 127,
+ [0][1][1][0][RTW89_ACMA][7] = 127,
+ [0][1][1][0][RTW89_CN][7] = 127,
+ [0][1][1][0][RTW89_UK][7] = 127,
+ [0][1][1][0][RTW89_FCC][8] = 127,
+ [0][1][1][0][RTW89_ETSI][8] = 127,
+ [0][1][1][0][RTW89_MKK][8] = 127,
+ [0][1][1][0][RTW89_IC][8] = 127,
+ [0][1][1][0][RTW89_KCC][8] = 127,
+ [0][1][1][0][RTW89_ACMA][8] = 127,
+ [0][1][1][0][RTW89_CN][8] = 127,
+ [0][1][1][0][RTW89_UK][8] = 127,
+ [0][1][1][0][RTW89_FCC][9] = 127,
+ [0][1][1][0][RTW89_ETSI][9] = 127,
+ [0][1][1][0][RTW89_MKK][9] = 127,
+ [0][1][1][0][RTW89_IC][9] = 127,
+ [0][1][1][0][RTW89_KCC][9] = 127,
+ [0][1][1][0][RTW89_ACMA][9] = 127,
+ [0][1][1][0][RTW89_CN][9] = 127,
+ [0][1][1][0][RTW89_UK][9] = 127,
+ [0][1][1][0][RTW89_FCC][10] = 127,
+ [0][1][1][0][RTW89_ETSI][10] = 127,
+ [0][1][1][0][RTW89_MKK][10] = 127,
+ [0][1][1][0][RTW89_IC][10] = 127,
+ [0][1][1][0][RTW89_KCC][10] = 127,
+ [0][1][1][0][RTW89_ACMA][10] = 127,
+ [0][1][1][0][RTW89_CN][10] = 127,
+ [0][1][1][0][RTW89_UK][10] = 127,
+ [0][1][1][0][RTW89_FCC][11] = 127,
+ [0][1][1][0][RTW89_ETSI][11] = 127,
+ [0][1][1][0][RTW89_MKK][11] = 127,
+ [0][1][1][0][RTW89_IC][11] = 127,
+ [0][1][1][0][RTW89_KCC][11] = 127,
+ [0][1][1][0][RTW89_ACMA][11] = 127,
+ [0][1][1][0][RTW89_CN][11] = 127,
+ [0][1][1][0][RTW89_UK][11] = 127,
+ [0][1][1][0][RTW89_FCC][12] = 127,
+ [0][1][1][0][RTW89_ETSI][12] = 127,
+ [0][1][1][0][RTW89_MKK][12] = 127,
+ [0][1][1][0][RTW89_IC][12] = 127,
+ [0][1][1][0][RTW89_KCC][12] = 127,
+ [0][1][1][0][RTW89_ACMA][12] = 127,
+ [0][1][1][0][RTW89_CN][12] = 127,
+ [0][1][1][0][RTW89_UK][12] = 127,
+ [0][1][1][0][RTW89_FCC][13] = 127,
+ [0][1][1][0][RTW89_ETSI][13] = 127,
+ [0][1][1][0][RTW89_MKK][13] = 127,
+ [0][1][1][0][RTW89_IC][13] = 127,
+ [0][1][1][0][RTW89_KCC][13] = 127,
+ [0][1][1][0][RTW89_ACMA][13] = 127,
+ [0][1][1][0][RTW89_CN][13] = 127,
+ [0][1][1][0][RTW89_UK][13] = 127,
+ [0][0][2][0][RTW89_FCC][0] = 78,
+ [0][0][2][0][RTW89_ETSI][0] = 60,
+ [0][0][2][0][RTW89_MKK][0] = 72,
+ [0][0][2][0][RTW89_IC][0] = 78,
+ [0][0][2][0][RTW89_KCC][0] = 78,
+ [0][0][2][0][RTW89_ACMA][0] = 60,
+ [0][0][2][0][RTW89_CN][0] = 60,
+ [0][0][2][0][RTW89_UK][0] = 60,
+ [0][0][2][0][RTW89_FCC][1] = 78,
+ [0][0][2][0][RTW89_ETSI][1] = 60,
+ [0][0][2][0][RTW89_MKK][1] = 78,
+ [0][0][2][0][RTW89_IC][1] = 78,
+ [0][0][2][0][RTW89_KCC][1] = 78,
+ [0][0][2][0][RTW89_ACMA][1] = 60,
+ [0][0][2][0][RTW89_CN][1] = 60,
+ [0][0][2][0][RTW89_UK][1] = 60,
+ [0][0][2][0][RTW89_FCC][2] = 82,
+ [0][0][2][0][RTW89_ETSI][2] = 60,
+ [0][0][2][0][RTW89_MKK][2] = 78,
+ [0][0][2][0][RTW89_IC][2] = 82,
+ [0][0][2][0][RTW89_KCC][2] = 78,
+ [0][0][2][0][RTW89_ACMA][2] = 60,
+ [0][0][2][0][RTW89_CN][2] = 60,
+ [0][0][2][0][RTW89_UK][2] = 60,
+ [0][0][2][0][RTW89_FCC][3] = 82,
+ [0][0][2][0][RTW89_ETSI][3] = 60,
+ [0][0][2][0][RTW89_MKK][3] = 78,
+ [0][0][2][0][RTW89_IC][3] = 82,
+ [0][0][2][0][RTW89_KCC][3] = 78,
+ [0][0][2][0][RTW89_ACMA][3] = 60,
+ [0][0][2][0][RTW89_CN][3] = 60,
+ [0][0][2][0][RTW89_UK][3] = 60,
+ [0][0][2][0][RTW89_FCC][4] = 82,
+ [0][0][2][0][RTW89_ETSI][4] = 60,
+ [0][0][2][0][RTW89_MKK][4] = 78,
+ [0][0][2][0][RTW89_IC][4] = 82,
+ [0][0][2][0][RTW89_KCC][4] = 78,
+ [0][0][2][0][RTW89_ACMA][4] = 60,
+ [0][0][2][0][RTW89_CN][4] = 60,
+ [0][0][2][0][RTW89_UK][4] = 60,
+ [0][0][2][0][RTW89_FCC][5] = 82,
+ [0][0][2][0][RTW89_ETSI][5] = 60,
+ [0][0][2][0][RTW89_MKK][5] = 78,
+ [0][0][2][0][RTW89_IC][5] = 82,
+ [0][0][2][0][RTW89_KCC][5] = 78,
+ [0][0][2][0][RTW89_ACMA][5] = 60,
+ [0][0][2][0][RTW89_CN][5] = 60,
+ [0][0][2][0][RTW89_UK][5] = 60,
+ [0][0][2][0][RTW89_FCC][6] = 82,
+ [0][0][2][0][RTW89_ETSI][6] = 60,
+ [0][0][2][0][RTW89_MKK][6] = 78,
+ [0][0][2][0][RTW89_IC][6] = 82,
+ [0][0][2][0][RTW89_KCC][6] = 78,
+ [0][0][2][0][RTW89_ACMA][6] = 60,
+ [0][0][2][0][RTW89_CN][6] = 60,
+ [0][0][2][0][RTW89_UK][6] = 60,
+ [0][0][2][0][RTW89_FCC][7] = 82,
+ [0][0][2][0][RTW89_ETSI][7] = 60,
+ [0][0][2][0][RTW89_MKK][7] = 78,
+ [0][0][2][0][RTW89_IC][7] = 82,
+ [0][0][2][0][RTW89_KCC][7] = 78,
+ [0][0][2][0][RTW89_ACMA][7] = 60,
+ [0][0][2][0][RTW89_CN][7] = 60,
+ [0][0][2][0][RTW89_UK][7] = 60,
+ [0][0][2][0][RTW89_FCC][8] = 80,
+ [0][0][2][0][RTW89_ETSI][8] = 60,
+ [0][0][2][0][RTW89_MKK][8] = 78,
+ [0][0][2][0][RTW89_IC][8] = 80,
+ [0][0][2][0][RTW89_KCC][8] = 78,
+ [0][0][2][0][RTW89_ACMA][8] = 60,
+ [0][0][2][0][RTW89_CN][8] = 60,
+ [0][0][2][0][RTW89_UK][8] = 60,
+ [0][0][2][0][RTW89_FCC][9] = 76,
+ [0][0][2][0][RTW89_ETSI][9] = 60,
+ [0][0][2][0][RTW89_MKK][9] = 78,
+ [0][0][2][0][RTW89_IC][9] = 76,
+ [0][0][2][0][RTW89_KCC][9] = 78,
+ [0][0][2][0][RTW89_ACMA][9] = 60,
+ [0][0][2][0][RTW89_CN][9] = 60,
+ [0][0][2][0][RTW89_UK][9] = 60,
+ [0][0][2][0][RTW89_FCC][10] = 76,
+ [0][0][2][0][RTW89_ETSI][10] = 60,
+ [0][0][2][0][RTW89_MKK][10] = 78,
+ [0][0][2][0][RTW89_IC][10] = 76,
+ [0][0][2][0][RTW89_KCC][10] = 78,
+ [0][0][2][0][RTW89_ACMA][10] = 60,
+ [0][0][2][0][RTW89_CN][10] = 60,
+ [0][0][2][0][RTW89_UK][10] = 60,
+ [0][0][2][0][RTW89_FCC][11] = 70,
+ [0][0][2][0][RTW89_ETSI][11] = 60,
+ [0][0][2][0][RTW89_MKK][11] = 78,
+ [0][0][2][0][RTW89_IC][11] = 70,
+ [0][0][2][0][RTW89_KCC][11] = 78,
+ [0][0][2][0][RTW89_ACMA][11] = 60,
+ [0][0][2][0][RTW89_CN][11] = 60,
+ [0][0][2][0][RTW89_UK][11] = 60,
+ [0][0][2][0][RTW89_FCC][12] = 70,
+ [0][0][2][0][RTW89_ETSI][12] = 60,
+ [0][0][2][0][RTW89_MKK][12] = 70,
+ [0][0][2][0][RTW89_IC][12] = 70,
+ [0][0][2][0][RTW89_KCC][12] = 78,
+ [0][0][2][0][RTW89_ACMA][12] = 60,
+ [0][0][2][0][RTW89_CN][12] = 60,
+ [0][0][2][0][RTW89_UK][12] = 60,
+ [0][0][2][0][RTW89_FCC][13] = 127,
+ [0][0][2][0][RTW89_ETSI][13] = 127,
+ [0][0][2][0][RTW89_MKK][13] = 127,
+ [0][0][2][0][RTW89_IC][13] = 127,
+ [0][0][2][0][RTW89_KCC][13] = 127,
+ [0][0][2][0][RTW89_ACMA][13] = 127,
+ [0][0][2][0][RTW89_CN][13] = 127,
+ [0][0][2][0][RTW89_UK][13] = 127,
+ [0][1][2][0][RTW89_FCC][0] = 127,
+ [0][1][2][0][RTW89_ETSI][0] = 127,
+ [0][1][2][0][RTW89_MKK][0] = 127,
+ [0][1][2][0][RTW89_IC][0] = 127,
+ [0][1][2][0][RTW89_KCC][0] = 127,
+ [0][1][2][0][RTW89_ACMA][0] = 127,
+ [0][1][2][0][RTW89_CN][0] = 127,
+ [0][1][2][0][RTW89_UK][0] = 127,
+ [0][1][2][0][RTW89_FCC][1] = 127,
+ [0][1][2][0][RTW89_ETSI][1] = 127,
+ [0][1][2][0][RTW89_MKK][1] = 127,
+ [0][1][2][0][RTW89_IC][1] = 127,
+ [0][1][2][0][RTW89_KCC][1] = 127,
+ [0][1][2][0][RTW89_ACMA][1] = 127,
+ [0][1][2][0][RTW89_CN][1] = 127,
+ [0][1][2][0][RTW89_UK][1] = 127,
+ [0][1][2][0][RTW89_FCC][2] = 127,
+ [0][1][2][0][RTW89_ETSI][2] = 127,
+ [0][1][2][0][RTW89_MKK][2] = 127,
+ [0][1][2][0][RTW89_IC][2] = 127,
+ [0][1][2][0][RTW89_KCC][2] = 127,
+ [0][1][2][0][RTW89_ACMA][2] = 127,
+ [0][1][2][0][RTW89_CN][2] = 127,
+ [0][1][2][0][RTW89_UK][2] = 127,
+ [0][1][2][0][RTW89_FCC][3] = 127,
+ [0][1][2][0][RTW89_ETSI][3] = 127,
+ [0][1][2][0][RTW89_MKK][3] = 127,
+ [0][1][2][0][RTW89_IC][3] = 127,
+ [0][1][2][0][RTW89_KCC][3] = 127,
+ [0][1][2][0][RTW89_ACMA][3] = 127,
+ [0][1][2][0][RTW89_CN][3] = 127,
+ [0][1][2][0][RTW89_UK][3] = 127,
+ [0][1][2][0][RTW89_FCC][4] = 127,
+ [0][1][2][0][RTW89_ETSI][4] = 127,
+ [0][1][2][0][RTW89_MKK][4] = 127,
+ [0][1][2][0][RTW89_IC][4] = 127,
+ [0][1][2][0][RTW89_KCC][4] = 127,
+ [0][1][2][0][RTW89_ACMA][4] = 127,
+ [0][1][2][0][RTW89_CN][4] = 127,
+ [0][1][2][0][RTW89_UK][4] = 127,
+ [0][1][2][0][RTW89_FCC][5] = 127,
+ [0][1][2][0][RTW89_ETSI][5] = 127,
+ [0][1][2][0][RTW89_MKK][5] = 127,
+ [0][1][2][0][RTW89_IC][5] = 127,
+ [0][1][2][0][RTW89_KCC][5] = 127,
+ [0][1][2][0][RTW89_ACMA][5] = 127,
+ [0][1][2][0][RTW89_CN][5] = 127,
+ [0][1][2][0][RTW89_UK][5] = 127,
+ [0][1][2][0][RTW89_FCC][6] = 127,
+ [0][1][2][0][RTW89_ETSI][6] = 127,
+ [0][1][2][0][RTW89_MKK][6] = 127,
+ [0][1][2][0][RTW89_IC][6] = 127,
+ [0][1][2][0][RTW89_KCC][6] = 127,
+ [0][1][2][0][RTW89_ACMA][6] = 127,
+ [0][1][2][0][RTW89_CN][6] = 127,
+ [0][1][2][0][RTW89_UK][6] = 127,
+ [0][1][2][0][RTW89_FCC][7] = 127,
+ [0][1][2][0][RTW89_ETSI][7] = 127,
+ [0][1][2][0][RTW89_MKK][7] = 127,
+ [0][1][2][0][RTW89_IC][7] = 127,
+ [0][1][2][0][RTW89_KCC][7] = 127,
+ [0][1][2][0][RTW89_ACMA][7] = 127,
+ [0][1][2][0][RTW89_CN][7] = 127,
+ [0][1][2][0][RTW89_UK][7] = 127,
+ [0][1][2][0][RTW89_FCC][8] = 127,
+ [0][1][2][0][RTW89_ETSI][8] = 127,
+ [0][1][2][0][RTW89_MKK][8] = 127,
+ [0][1][2][0][RTW89_IC][8] = 127,
+ [0][1][2][0][RTW89_KCC][8] = 127,
+ [0][1][2][0][RTW89_ACMA][8] = 127,
+ [0][1][2][0][RTW89_CN][8] = 127,
+ [0][1][2][0][RTW89_UK][8] = 127,
+ [0][1][2][0][RTW89_FCC][9] = 127,
+ [0][1][2][0][RTW89_ETSI][9] = 127,
+ [0][1][2][0][RTW89_MKK][9] = 127,
+ [0][1][2][0][RTW89_IC][9] = 127,
+ [0][1][2][0][RTW89_KCC][9] = 127,
+ [0][1][2][0][RTW89_ACMA][9] = 127,
+ [0][1][2][0][RTW89_CN][9] = 127,
+ [0][1][2][0][RTW89_UK][9] = 127,
+ [0][1][2][0][RTW89_FCC][10] = 127,
+ [0][1][2][0][RTW89_ETSI][10] = 127,
+ [0][1][2][0][RTW89_MKK][10] = 127,
+ [0][1][2][0][RTW89_IC][10] = 127,
+ [0][1][2][0][RTW89_KCC][10] = 127,
+ [0][1][2][0][RTW89_ACMA][10] = 127,
+ [0][1][2][0][RTW89_CN][10] = 127,
+ [0][1][2][0][RTW89_UK][10] = 127,
+ [0][1][2][0][RTW89_FCC][11] = 127,
+ [0][1][2][0][RTW89_ETSI][11] = 127,
+ [0][1][2][0][RTW89_MKK][11] = 127,
+ [0][1][2][0][RTW89_IC][11] = 127,
+ [0][1][2][0][RTW89_KCC][11] = 127,
+ [0][1][2][0][RTW89_ACMA][11] = 127,
+ [0][1][2][0][RTW89_CN][11] = 127,
+ [0][1][2][0][RTW89_UK][11] = 127,
+ [0][1][2][0][RTW89_FCC][12] = 127,
+ [0][1][2][0][RTW89_ETSI][12] = 127,
+ [0][1][2][0][RTW89_MKK][12] = 127,
+ [0][1][2][0][RTW89_IC][12] = 127,
+ [0][1][2][0][RTW89_KCC][12] = 127,
+ [0][1][2][0][RTW89_ACMA][12] = 127,
+ [0][1][2][0][RTW89_CN][12] = 127,
+ [0][1][2][0][RTW89_UK][12] = 127,
+ [0][1][2][0][RTW89_FCC][13] = 127,
+ [0][1][2][0][RTW89_ETSI][13] = 127,
+ [0][1][2][0][RTW89_MKK][13] = 127,
+ [0][1][2][0][RTW89_IC][13] = 127,
+ [0][1][2][0][RTW89_KCC][13] = 127,
+ [0][1][2][0][RTW89_ACMA][13] = 127,
+ [0][1][2][0][RTW89_CN][13] = 127,
+ [0][1][2][0][RTW89_UK][13] = 127,
+ [0][1][2][1][RTW89_FCC][0] = 127,
+ [0][1][2][1][RTW89_ETSI][0] = 127,
+ [0][1][2][1][RTW89_MKK][0] = 127,
+ [0][1][2][1][RTW89_IC][0] = 127,
+ [0][1][2][1][RTW89_KCC][0] = 127,
+ [0][1][2][1][RTW89_ACMA][0] = 127,
+ [0][1][2][1][RTW89_CN][0] = 127,
+ [0][1][2][1][RTW89_UK][0] = 127,
+ [0][1][2][1][RTW89_FCC][1] = 127,
+ [0][1][2][1][RTW89_ETSI][1] = 127,
+ [0][1][2][1][RTW89_MKK][1] = 127,
+ [0][1][2][1][RTW89_IC][1] = 127,
+ [0][1][2][1][RTW89_KCC][1] = 127,
+ [0][1][2][1][RTW89_ACMA][1] = 127,
+ [0][1][2][1][RTW89_CN][1] = 127,
+ [0][1][2][1][RTW89_UK][1] = 127,
+ [0][1][2][1][RTW89_FCC][2] = 127,
+ [0][1][2][1][RTW89_ETSI][2] = 127,
+ [0][1][2][1][RTW89_MKK][2] = 127,
+ [0][1][2][1][RTW89_IC][2] = 127,
+ [0][1][2][1][RTW89_KCC][2] = 127,
+ [0][1][2][1][RTW89_ACMA][2] = 127,
+ [0][1][2][1][RTW89_CN][2] = 127,
+ [0][1][2][1][RTW89_UK][2] = 127,
+ [0][1][2][1][RTW89_FCC][3] = 127,
+ [0][1][2][1][RTW89_ETSI][3] = 127,
+ [0][1][2][1][RTW89_MKK][3] = 127,
+ [0][1][2][1][RTW89_IC][3] = 127,
+ [0][1][2][1][RTW89_KCC][3] = 127,
+ [0][1][2][1][RTW89_ACMA][3] = 127,
+ [0][1][2][1][RTW89_CN][3] = 127,
+ [0][1][2][1][RTW89_UK][3] = 127,
+ [0][1][2][1][RTW89_FCC][4] = 127,
+ [0][1][2][1][RTW89_ETSI][4] = 127,
+ [0][1][2][1][RTW89_MKK][4] = 127,
+ [0][1][2][1][RTW89_IC][4] = 127,
+ [0][1][2][1][RTW89_KCC][4] = 127,
+ [0][1][2][1][RTW89_ACMA][4] = 127,
+ [0][1][2][1][RTW89_CN][4] = 127,
+ [0][1][2][1][RTW89_UK][4] = 127,
+ [0][1][2][1][RTW89_FCC][5] = 127,
+ [0][1][2][1][RTW89_ETSI][5] = 127,
+ [0][1][2][1][RTW89_MKK][5] = 127,
+ [0][1][2][1][RTW89_IC][5] = 127,
+ [0][1][2][1][RTW89_KCC][5] = 127,
+ [0][1][2][1][RTW89_ACMA][5] = 127,
+ [0][1][2][1][RTW89_CN][5] = 127,
+ [0][1][2][1][RTW89_UK][5] = 127,
+ [0][1][2][1][RTW89_FCC][6] = 127,
+ [0][1][2][1][RTW89_ETSI][6] = 127,
+ [0][1][2][1][RTW89_MKK][6] = 127,
+ [0][1][2][1][RTW89_IC][6] = 127,
+ [0][1][2][1][RTW89_KCC][6] = 127,
+ [0][1][2][1][RTW89_ACMA][6] = 127,
+ [0][1][2][1][RTW89_CN][6] = 127,
+ [0][1][2][1][RTW89_UK][6] = 127,
+ [0][1][2][1][RTW89_FCC][7] = 127,
+ [0][1][2][1][RTW89_ETSI][7] = 127,
+ [0][1][2][1][RTW89_MKK][7] = 127,
+ [0][1][2][1][RTW89_IC][7] = 127,
+ [0][1][2][1][RTW89_KCC][7] = 127,
+ [0][1][2][1][RTW89_ACMA][7] = 127,
+ [0][1][2][1][RTW89_CN][7] = 127,
+ [0][1][2][1][RTW89_UK][7] = 127,
+ [0][1][2][1][RTW89_FCC][8] = 127,
+ [0][1][2][1][RTW89_ETSI][8] = 127,
+ [0][1][2][1][RTW89_MKK][8] = 127,
+ [0][1][2][1][RTW89_IC][8] = 127,
+ [0][1][2][1][RTW89_KCC][8] = 127,
+ [0][1][2][1][RTW89_ACMA][8] = 127,
+ [0][1][2][1][RTW89_CN][8] = 127,
+ [0][1][2][1][RTW89_UK][8] = 127,
+ [0][1][2][1][RTW89_FCC][9] = 127,
+ [0][1][2][1][RTW89_ETSI][9] = 127,
+ [0][1][2][1][RTW89_MKK][9] = 127,
+ [0][1][2][1][RTW89_IC][9] = 127,
+ [0][1][2][1][RTW89_KCC][9] = 127,
+ [0][1][2][1][RTW89_ACMA][9] = 127,
+ [0][1][2][1][RTW89_CN][9] = 127,
+ [0][1][2][1][RTW89_UK][9] = 127,
+ [0][1][2][1][RTW89_FCC][10] = 127,
+ [0][1][2][1][RTW89_ETSI][10] = 127,
+ [0][1][2][1][RTW89_MKK][10] = 127,
+ [0][1][2][1][RTW89_IC][10] = 127,
+ [0][1][2][1][RTW89_KCC][10] = 127,
+ [0][1][2][1][RTW89_ACMA][10] = 127,
+ [0][1][2][1][RTW89_CN][10] = 127,
+ [0][1][2][1][RTW89_UK][10] = 127,
+ [0][1][2][1][RTW89_FCC][11] = 127,
+ [0][1][2][1][RTW89_ETSI][11] = 127,
+ [0][1][2][1][RTW89_MKK][11] = 127,
+ [0][1][2][1][RTW89_IC][11] = 127,
+ [0][1][2][1][RTW89_KCC][11] = 127,
+ [0][1][2][1][RTW89_ACMA][11] = 127,
+ [0][1][2][1][RTW89_CN][11] = 127,
+ [0][1][2][1][RTW89_UK][11] = 127,
+ [0][1][2][1][RTW89_FCC][12] = 127,
+ [0][1][2][1][RTW89_ETSI][12] = 127,
+ [0][1][2][1][RTW89_MKK][12] = 127,
+ [0][1][2][1][RTW89_IC][12] = 127,
+ [0][1][2][1][RTW89_KCC][12] = 127,
+ [0][1][2][1][RTW89_ACMA][12] = 127,
+ [0][1][2][1][RTW89_CN][12] = 127,
+ [0][1][2][1][RTW89_UK][12] = 127,
+ [0][1][2][1][RTW89_FCC][13] = 127,
+ [0][1][2][1][RTW89_ETSI][13] = 127,
+ [0][1][2][1][RTW89_MKK][13] = 127,
+ [0][1][2][1][RTW89_IC][13] = 127,
+ [0][1][2][1][RTW89_KCC][13] = 127,
+ [0][1][2][1][RTW89_ACMA][13] = 127,
+ [0][1][2][1][RTW89_CN][13] = 127,
+ [0][1][2][1][RTW89_UK][13] = 127,
+ [1][0][2][0][RTW89_FCC][0] = 127,
+ [1][0][2][0][RTW89_ETSI][0] = 127,
+ [1][0][2][0][RTW89_MKK][0] = 127,
+ [1][0][2][0][RTW89_IC][0] = 127,
+ [1][0][2][0][RTW89_KCC][0] = 127,
+ [1][0][2][0][RTW89_ACMA][0] = 127,
+ [1][0][2][0][RTW89_CN][0] = 127,
+ [1][0][2][0][RTW89_UK][0] = 127,
+ [1][0][2][0][RTW89_FCC][1] = 127,
+ [1][0][2][0][RTW89_ETSI][1] = 127,
+ [1][0][2][0][RTW89_MKK][1] = 127,
+ [1][0][2][0][RTW89_IC][1] = 127,
+ [1][0][2][0][RTW89_KCC][1] = 127,
+ [1][0][2][0][RTW89_ACMA][1] = 127,
+ [1][0][2][0][RTW89_CN][1] = 127,
+ [1][0][2][0][RTW89_UK][1] = 127,
+ [1][0][2][0][RTW89_FCC][2] = 72,
+ [1][0][2][0][RTW89_ETSI][2] = 58,
+ [1][0][2][0][RTW89_MKK][2] = 80,
+ [1][0][2][0][RTW89_IC][2] = 72,
+ [1][0][2][0][RTW89_KCC][2] = 80,
+ [1][0][2][0][RTW89_ACMA][2] = 58,
+ [1][0][2][0][RTW89_CN][2] = 60,
+ [1][0][2][0][RTW89_UK][2] = 58,
+ [1][0][2][0][RTW89_FCC][3] = 72,
+ [1][0][2][0][RTW89_ETSI][3] = 58,
+ [1][0][2][0][RTW89_MKK][3] = 80,
+ [1][0][2][0][RTW89_IC][3] = 72,
+ [1][0][2][0][RTW89_KCC][3] = 80,
+ [1][0][2][0][RTW89_ACMA][3] = 58,
+ [1][0][2][0][RTW89_CN][3] = 60,
+ [1][0][2][0][RTW89_UK][3] = 58,
+ [1][0][2][0][RTW89_FCC][4] = 76,
+ [1][0][2][0][RTW89_ETSI][4] = 58,
+ [1][0][2][0][RTW89_MKK][4] = 80,
+ [1][0][2][0][RTW89_IC][4] = 76,
+ [1][0][2][0][RTW89_KCC][4] = 80,
+ [1][0][2][0][RTW89_ACMA][4] = 58,
+ [1][0][2][0][RTW89_CN][4] = 60,
+ [1][0][2][0][RTW89_UK][4] = 58,
+ [1][0][2][0][RTW89_FCC][5] = 78,
+ [1][0][2][0][RTW89_ETSI][5] = 58,
+ [1][0][2][0][RTW89_MKK][5] = 80,
+ [1][0][2][0][RTW89_IC][5] = 78,
+ [1][0][2][0][RTW89_KCC][5] = 80,
+ [1][0][2][0][RTW89_ACMA][5] = 58,
+ [1][0][2][0][RTW89_CN][5] = 60,
+ [1][0][2][0][RTW89_UK][5] = 58,
+ [1][0][2][0][RTW89_FCC][6] = 78,
+ [1][0][2][0][RTW89_ETSI][6] = 58,
+ [1][0][2][0][RTW89_MKK][6] = 78,
+ [1][0][2][0][RTW89_IC][6] = 78,
+ [1][0][2][0][RTW89_KCC][6] = 80,
+ [1][0][2][0][RTW89_ACMA][6] = 58,
+ [1][0][2][0][RTW89_CN][6] = 60,
+ [1][0][2][0][RTW89_UK][6] = 58,
+ [1][0][2][0][RTW89_FCC][7] = 78,
+ [1][0][2][0][RTW89_ETSI][7] = 58,
+ [1][0][2][0][RTW89_MKK][7] = 80,
+ [1][0][2][0][RTW89_IC][7] = 78,
+ [1][0][2][0][RTW89_KCC][7] = 80,
+ [1][0][2][0][RTW89_ACMA][7] = 58,
+ [1][0][2][0][RTW89_CN][7] = 60,
+ [1][0][2][0][RTW89_UK][7] = 58,
+ [1][0][2][0][RTW89_FCC][8] = 78,
+ [1][0][2][0][RTW89_ETSI][8] = 58,
+ [1][0][2][0][RTW89_MKK][8] = 80,
+ [1][0][2][0][RTW89_IC][8] = 78,
+ [1][0][2][0][RTW89_KCC][8] = 78,
+ [1][0][2][0][RTW89_ACMA][8] = 58,
+ [1][0][2][0][RTW89_CN][8] = 60,
+ [1][0][2][0][RTW89_UK][8] = 58,
+ [1][0][2][0][RTW89_FCC][9] = 76,
+ [1][0][2][0][RTW89_ETSI][9] = 58,
+ [1][0][2][0][RTW89_MKK][9] = 80,
+ [1][0][2][0][RTW89_IC][9] = 76,
+ [1][0][2][0][RTW89_KCC][9] = 78,
+ [1][0][2][0][RTW89_ACMA][9] = 58,
+ [1][0][2][0][RTW89_CN][9] = 60,
+ [1][0][2][0][RTW89_UK][9] = 58,
+ [1][0][2][0][RTW89_FCC][10] = 70,
+ [1][0][2][0][RTW89_ETSI][10] = 58,
+ [1][0][2][0][RTW89_MKK][10] = 78,
+ [1][0][2][0][RTW89_IC][10] = 70,
+ [1][0][2][0][RTW89_KCC][10] = 78,
+ [1][0][2][0][RTW89_ACMA][10] = 58,
+ [1][0][2][0][RTW89_CN][10] = 60,
+ [1][0][2][0][RTW89_UK][10] = 58,
+ [1][0][2][0][RTW89_FCC][11] = 127,
+ [1][0][2][0][RTW89_ETSI][11] = 127,
+ [1][0][2][0][RTW89_MKK][11] = 127,
+ [1][0][2][0][RTW89_IC][11] = 127,
+ [1][0][2][0][RTW89_KCC][11] = 127,
+ [1][0][2][0][RTW89_ACMA][11] = 127,
+ [1][0][2][0][RTW89_CN][11] = 127,
+ [1][0][2][0][RTW89_UK][11] = 127,
+ [1][0][2][0][RTW89_FCC][12] = 127,
+ [1][0][2][0][RTW89_ETSI][12] = 127,
+ [1][0][2][0][RTW89_MKK][12] = 127,
+ [1][0][2][0][RTW89_IC][12] = 127,
+ [1][0][2][0][RTW89_KCC][12] = 127,
+ [1][0][2][0][RTW89_ACMA][12] = 127,
+ [1][0][2][0][RTW89_CN][12] = 127,
+ [1][0][2][0][RTW89_UK][12] = 127,
+ [1][0][2][0][RTW89_FCC][13] = 127,
+ [1][0][2][0][RTW89_ETSI][13] = 127,
+ [1][0][2][0][RTW89_MKK][13] = 127,
+ [1][0][2][0][RTW89_IC][13] = 127,
+ [1][0][2][0][RTW89_KCC][13] = 127,
+ [1][0][2][0][RTW89_ACMA][13] = 127,
+ [1][0][2][0][RTW89_CN][13] = 127,
+ [1][0][2][0][RTW89_UK][13] = 127,
+ [1][1][2][0][RTW89_FCC][0] = 127,
+ [1][1][2][0][RTW89_ETSI][0] = 127,
+ [1][1][2][0][RTW89_MKK][0] = 127,
+ [1][1][2][0][RTW89_IC][0] = 127,
+ [1][1][2][0][RTW89_KCC][0] = 127,
+ [1][1][2][0][RTW89_ACMA][0] = 127,
+ [1][1][2][0][RTW89_CN][0] = 127,
+ [1][1][2][0][RTW89_UK][0] = 127,
+ [1][1][2][0][RTW89_FCC][1] = 127,
+ [1][1][2][0][RTW89_ETSI][1] = 127,
+ [1][1][2][0][RTW89_MKK][1] = 127,
+ [1][1][2][0][RTW89_IC][1] = 127,
+ [1][1][2][0][RTW89_KCC][1] = 127,
+ [1][1][2][0][RTW89_ACMA][1] = 127,
+ [1][1][2][0][RTW89_CN][1] = 127,
+ [1][1][2][0][RTW89_UK][1] = 127,
+ [1][1][2][0][RTW89_FCC][2] = 127,
+ [1][1][2][0][RTW89_ETSI][2] = 127,
+ [1][1][2][0][RTW89_MKK][2] = 127,
+ [1][1][2][0][RTW89_IC][2] = 127,
+ [1][1][2][0][RTW89_KCC][2] = 127,
+ [1][1][2][0][RTW89_ACMA][2] = 127,
+ [1][1][2][0][RTW89_CN][2] = 127,
+ [1][1][2][0][RTW89_UK][2] = 127,
+ [1][1][2][0][RTW89_FCC][3] = 127,
+ [1][1][2][0][RTW89_ETSI][3] = 127,
+ [1][1][2][0][RTW89_MKK][3] = 127,
+ [1][1][2][0][RTW89_IC][3] = 127,
+ [1][1][2][0][RTW89_KCC][3] = 127,
+ [1][1][2][0][RTW89_ACMA][3] = 127,
+ [1][1][2][0][RTW89_CN][3] = 127,
+ [1][1][2][0][RTW89_UK][3] = 127,
+ [1][1][2][0][RTW89_FCC][4] = 127,
+ [1][1][2][0][RTW89_ETSI][4] = 127,
+ [1][1][2][0][RTW89_MKK][4] = 127,
+ [1][1][2][0][RTW89_IC][4] = 127,
+ [1][1][2][0][RTW89_KCC][4] = 127,
+ [1][1][2][0][RTW89_ACMA][4] = 127,
+ [1][1][2][0][RTW89_CN][4] = 127,
+ [1][1][2][0][RTW89_UK][4] = 127,
+ [1][1][2][0][RTW89_FCC][5] = 127,
+ [1][1][2][0][RTW89_ETSI][5] = 127,
+ [1][1][2][0][RTW89_MKK][5] = 127,
+ [1][1][2][0][RTW89_IC][5] = 127,
+ [1][1][2][0][RTW89_KCC][5] = 127,
+ [1][1][2][0][RTW89_ACMA][5] = 127,
+ [1][1][2][0][RTW89_CN][5] = 127,
+ [1][1][2][0][RTW89_UK][5] = 127,
+ [1][1][2][0][RTW89_FCC][6] = 127,
+ [1][1][2][0][RTW89_ETSI][6] = 127,
+ [1][1][2][0][RTW89_MKK][6] = 127,
+ [1][1][2][0][RTW89_IC][6] = 127,
+ [1][1][2][0][RTW89_KCC][6] = 127,
+ [1][1][2][0][RTW89_ACMA][6] = 127,
+ [1][1][2][0][RTW89_CN][6] = 127,
+ [1][1][2][0][RTW89_UK][6] = 127,
+ [1][1][2][0][RTW89_FCC][7] = 127,
+ [1][1][2][0][RTW89_ETSI][7] = 127,
+ [1][1][2][0][RTW89_MKK][7] = 127,
+ [1][1][2][0][RTW89_IC][7] = 127,
+ [1][1][2][0][RTW89_KCC][7] = 127,
+ [1][1][2][0][RTW89_ACMA][7] = 127,
+ [1][1][2][0][RTW89_CN][7] = 127,
+ [1][1][2][0][RTW89_UK][7] = 127,
+ [1][1][2][0][RTW89_FCC][8] = 127,
+ [1][1][2][0][RTW89_ETSI][8] = 127,
+ [1][1][2][0][RTW89_MKK][8] = 127,
+ [1][1][2][0][RTW89_IC][8] = 127,
+ [1][1][2][0][RTW89_KCC][8] = 127,
+ [1][1][2][0][RTW89_ACMA][8] = 127,
+ [1][1][2][0][RTW89_CN][8] = 127,
+ [1][1][2][0][RTW89_UK][8] = 127,
+ [1][1][2][0][RTW89_FCC][9] = 127,
+ [1][1][2][0][RTW89_ETSI][9] = 127,
+ [1][1][2][0][RTW89_MKK][9] = 127,
+ [1][1][2][0][RTW89_IC][9] = 127,
+ [1][1][2][0][RTW89_KCC][9] = 127,
+ [1][1][2][0][RTW89_ACMA][9] = 127,
+ [1][1][2][0][RTW89_CN][9] = 127,
+ [1][1][2][0][RTW89_UK][9] = 127,
+ [1][1][2][0][RTW89_FCC][10] = 127,
+ [1][1][2][0][RTW89_ETSI][10] = 127,
+ [1][1][2][0][RTW89_MKK][10] = 127,
+ [1][1][2][0][RTW89_IC][10] = 127,
+ [1][1][2][0][RTW89_KCC][10] = 127,
+ [1][1][2][0][RTW89_ACMA][10] = 127,
+ [1][1][2][0][RTW89_CN][10] = 127,
+ [1][1][2][0][RTW89_UK][10] = 127,
+ [1][1][2][0][RTW89_FCC][11] = 127,
+ [1][1][2][0][RTW89_ETSI][11] = 127,
+ [1][1][2][0][RTW89_MKK][11] = 127,
+ [1][1][2][0][RTW89_IC][11] = 127,
+ [1][1][2][0][RTW89_KCC][11] = 127,
+ [1][1][2][0][RTW89_ACMA][11] = 127,
+ [1][1][2][0][RTW89_CN][11] = 127,
+ [1][1][2][0][RTW89_UK][11] = 127,
+ [1][1][2][0][RTW89_FCC][12] = 127,
+ [1][1][2][0][RTW89_ETSI][12] = 127,
+ [1][1][2][0][RTW89_MKK][12] = 127,
+ [1][1][2][0][RTW89_IC][12] = 127,
+ [1][1][2][0][RTW89_KCC][12] = 127,
+ [1][1][2][0][RTW89_ACMA][12] = 127,
+ [1][1][2][0][RTW89_CN][12] = 127,
+ [1][1][2][0][RTW89_UK][12] = 127,
+ [1][1][2][0][RTW89_FCC][13] = 127,
+ [1][1][2][0][RTW89_ETSI][13] = 127,
+ [1][1][2][0][RTW89_MKK][13] = 127,
+ [1][1][2][0][RTW89_IC][13] = 127,
+ [1][1][2][0][RTW89_KCC][13] = 127,
+ [1][1][2][0][RTW89_ACMA][13] = 127,
+ [1][1][2][0][RTW89_CN][13] = 127,
+ [1][1][2][0][RTW89_UK][13] = 127,
+ [1][1][2][1][RTW89_FCC][0] = 127,
+ [1][1][2][1][RTW89_ETSI][0] = 127,
+ [1][1][2][1][RTW89_MKK][0] = 127,
+ [1][1][2][1][RTW89_IC][0] = 127,
+ [1][1][2][1][RTW89_KCC][0] = 127,
+ [1][1][2][1][RTW89_ACMA][0] = 127,
+ [1][1][2][1][RTW89_CN][0] = 127,
+ [1][1][2][1][RTW89_UK][0] = 127,
+ [1][1][2][1][RTW89_FCC][1] = 127,
+ [1][1][2][1][RTW89_ETSI][1] = 127,
+ [1][1][2][1][RTW89_MKK][1] = 127,
+ [1][1][2][1][RTW89_IC][1] = 127,
+ [1][1][2][1][RTW89_KCC][1] = 127,
+ [1][1][2][1][RTW89_ACMA][1] = 127,
+ [1][1][2][1][RTW89_CN][1] = 127,
+ [1][1][2][1][RTW89_UK][1] = 127,
+ [1][1][2][1][RTW89_FCC][2] = 127,
+ [1][1][2][1][RTW89_ETSI][2] = 127,
+ [1][1][2][1][RTW89_MKK][2] = 127,
+ [1][1][2][1][RTW89_IC][2] = 127,
+ [1][1][2][1][RTW89_KCC][2] = 127,
+ [1][1][2][1][RTW89_ACMA][2] = 127,
+ [1][1][2][1][RTW89_CN][2] = 127,
+ [1][1][2][1][RTW89_UK][2] = 127,
+ [1][1][2][1][RTW89_FCC][3] = 127,
+ [1][1][2][1][RTW89_ETSI][3] = 127,
+ [1][1][2][1][RTW89_MKK][3] = 127,
+ [1][1][2][1][RTW89_IC][3] = 127,
+ [1][1][2][1][RTW89_KCC][3] = 127,
+ [1][1][2][1][RTW89_ACMA][3] = 127,
+ [1][1][2][1][RTW89_CN][3] = 127,
+ [1][1][2][1][RTW89_UK][3] = 127,
+ [1][1][2][1][RTW89_FCC][4] = 127,
+ [1][1][2][1][RTW89_ETSI][4] = 127,
+ [1][1][2][1][RTW89_MKK][4] = 127,
+ [1][1][2][1][RTW89_IC][4] = 127,
+ [1][1][2][1][RTW89_KCC][4] = 127,
+ [1][1][2][1][RTW89_ACMA][4] = 127,
+ [1][1][2][1][RTW89_CN][4] = 127,
+ [1][1][2][1][RTW89_UK][4] = 127,
+ [1][1][2][1][RTW89_FCC][5] = 127,
+ [1][1][2][1][RTW89_ETSI][5] = 127,
+ [1][1][2][1][RTW89_MKK][5] = 127,
+ [1][1][2][1][RTW89_IC][5] = 127,
+ [1][1][2][1][RTW89_KCC][5] = 127,
+ [1][1][2][1][RTW89_ACMA][5] = 127,
+ [1][1][2][1][RTW89_CN][5] = 127,
+ [1][1][2][1][RTW89_UK][5] = 127,
+ [1][1][2][1][RTW89_FCC][6] = 127,
+ [1][1][2][1][RTW89_ETSI][6] = 127,
+ [1][1][2][1][RTW89_MKK][6] = 127,
+ [1][1][2][1][RTW89_IC][6] = 127,
+ [1][1][2][1][RTW89_KCC][6] = 127,
+ [1][1][2][1][RTW89_ACMA][6] = 127,
+ [1][1][2][1][RTW89_CN][6] = 127,
+ [1][1][2][1][RTW89_UK][6] = 127,
+ [1][1][2][1][RTW89_FCC][7] = 127,
+ [1][1][2][1][RTW89_ETSI][7] = 127,
+ [1][1][2][1][RTW89_MKK][7] = 127,
+ [1][1][2][1][RTW89_IC][7] = 127,
+ [1][1][2][1][RTW89_KCC][7] = 127,
+ [1][1][2][1][RTW89_ACMA][7] = 127,
+ [1][1][2][1][RTW89_CN][7] = 127,
+ [1][1][2][1][RTW89_UK][7] = 127,
+ [1][1][2][1][RTW89_FCC][8] = 127,
+ [1][1][2][1][RTW89_ETSI][8] = 127,
+ [1][1][2][1][RTW89_MKK][8] = 127,
+ [1][1][2][1][RTW89_IC][8] = 127,
+ [1][1][2][1][RTW89_KCC][8] = 127,
+ [1][1][2][1][RTW89_ACMA][8] = 127,
+ [1][1][2][1][RTW89_CN][8] = 127,
+ [1][1][2][1][RTW89_UK][8] = 127,
+ [1][1][2][1][RTW89_FCC][9] = 127,
+ [1][1][2][1][RTW89_ETSI][9] = 127,
+ [1][1][2][1][RTW89_MKK][9] = 127,
+ [1][1][2][1][RTW89_IC][9] = 127,
+ [1][1][2][1][RTW89_KCC][9] = 127,
+ [1][1][2][1][RTW89_ACMA][9] = 127,
+ [1][1][2][1][RTW89_CN][9] = 127,
+ [1][1][2][1][RTW89_UK][9] = 127,
+ [1][1][2][1][RTW89_FCC][10] = 127,
+ [1][1][2][1][RTW89_ETSI][10] = 127,
+ [1][1][2][1][RTW89_MKK][10] = 127,
+ [1][1][2][1][RTW89_IC][10] = 127,
+ [1][1][2][1][RTW89_KCC][10] = 127,
+ [1][1][2][1][RTW89_ACMA][10] = 127,
+ [1][1][2][1][RTW89_CN][10] = 127,
+ [1][1][2][1][RTW89_UK][10] = 127,
+ [1][1][2][1][RTW89_FCC][11] = 127,
+ [1][1][2][1][RTW89_ETSI][11] = 127,
+ [1][1][2][1][RTW89_MKK][11] = 127,
+ [1][1][2][1][RTW89_IC][11] = 127,
+ [1][1][2][1][RTW89_KCC][11] = 127,
+ [1][1][2][1][RTW89_ACMA][11] = 127,
+ [1][1][2][1][RTW89_CN][11] = 127,
+ [1][1][2][1][RTW89_UK][11] = 127,
+ [1][1][2][1][RTW89_FCC][12] = 127,
+ [1][1][2][1][RTW89_ETSI][12] = 127,
+ [1][1][2][1][RTW89_MKK][12] = 127,
+ [1][1][2][1][RTW89_IC][12] = 127,
+ [1][1][2][1][RTW89_KCC][12] = 127,
+ [1][1][2][1][RTW89_ACMA][12] = 127,
+ [1][1][2][1][RTW89_CN][12] = 127,
+ [1][1][2][1][RTW89_UK][12] = 127,
+ [1][1][2][1][RTW89_FCC][13] = 127,
+ [1][1][2][1][RTW89_ETSI][13] = 127,
+ [1][1][2][1][RTW89_MKK][13] = 127,
+ [1][1][2][1][RTW89_IC][13] = 127,
+ [1][1][2][1][RTW89_KCC][13] = 127,
+ [1][1][2][1][RTW89_ACMA][13] = 127,
+ [1][1][2][1][RTW89_CN][13] = 127,
+ [1][1][2][1][RTW89_UK][13] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
+ [0][0][1][0][RTW89_WW][0] = 58,
+ [0][0][1][0][RTW89_WW][2] = 58,
+ [0][0][1][0][RTW89_WW][4] = 58,
+ [0][0][1][0][RTW89_WW][6] = 50,
+ [0][0][1][0][RTW89_WW][8] = 58,
+ [0][0][1][0][RTW89_WW][10] = 58,
+ [0][0][1][0][RTW89_WW][12] = 58,
+ [0][0][1][0][RTW89_WW][14] = 58,
+ [0][0][1][0][RTW89_WW][15] = 58,
+ [0][0][1][0][RTW89_WW][17] = 60,
+ [0][0][1][0][RTW89_WW][19] = 60,
+ [0][0][1][0][RTW89_WW][21] = 60,
+ [0][0][1][0][RTW89_WW][23] = 60,
+ [0][0][1][0][RTW89_WW][25] = 60,
+ [0][0][1][0][RTW89_WW][27] = 60,
+ [0][0][1][0][RTW89_WW][29] = 60,
+ [0][0][1][0][RTW89_WW][31] = 60,
+ [0][0][1][0][RTW89_WW][33] = 60,
+ [0][0][1][0][RTW89_WW][35] = 60,
+ [0][0][1][0][RTW89_WW][37] = 74,
+ [0][0][1][0][RTW89_WW][38] = 30,
+ [0][0][1][0][RTW89_WW][40] = 30,
+ [0][0][1][0][RTW89_WW][42] = 30,
+ [0][0][1][0][RTW89_WW][44] = 30,
+ [0][0][1][0][RTW89_WW][46] = 30,
+ [0][0][1][0][RTW89_WW][48] = 72,
+ [0][0][1][0][RTW89_WW][50] = 72,
+ [0][0][1][0][RTW89_WW][52] = 72,
+ [0][1][1][0][RTW89_WW][0] = 0,
+ [0][1][1][0][RTW89_WW][2] = 0,
+ [0][1][1][0][RTW89_WW][4] = 0,
+ [0][1][1][0][RTW89_WW][6] = 0,
+ [0][1][1][0][RTW89_WW][8] = 0,
+ [0][1][1][0][RTW89_WW][10] = 0,
+ [0][1][1][0][RTW89_WW][12] = 0,
+ [0][1][1][0][RTW89_WW][14] = 0,
+ [0][1][1][0][RTW89_WW][15] = 0,
+ [0][1][1][0][RTW89_WW][17] = 0,
+ [0][1][1][0][RTW89_WW][19] = 0,
+ [0][1][1][0][RTW89_WW][21] = 0,
+ [0][1][1][0][RTW89_WW][23] = 0,
+ [0][1][1][0][RTW89_WW][25] = 0,
+ [0][1][1][0][RTW89_WW][27] = 0,
+ [0][1][1][0][RTW89_WW][29] = 0,
+ [0][1][1][0][RTW89_WW][31] = 0,
+ [0][1][1][0][RTW89_WW][33] = 0,
+ [0][1][1][0][RTW89_WW][35] = 0,
+ [0][1][1][0][RTW89_WW][37] = 0,
+ [0][1][1][0][RTW89_WW][38] = 0,
+ [0][1][1][0][RTW89_WW][40] = 0,
+ [0][1][1][0][RTW89_WW][42] = 0,
+ [0][1][1][0][RTW89_WW][44] = 0,
+ [0][1][1][0][RTW89_WW][46] = 0,
+ [0][1][1][0][RTW89_WW][48] = 0,
+ [0][1][1][0][RTW89_WW][50] = 0,
+ [0][1][1][0][RTW89_WW][52] = 0,
+ [0][0][2][0][RTW89_WW][0] = 62,
+ [0][0][2][0][RTW89_WW][2] = 62,
+ [0][0][2][0][RTW89_WW][4] = 62,
+ [0][0][2][0][RTW89_WW][6] = 54,
+ [0][0][2][0][RTW89_WW][8] = 62,
+ [0][0][2][0][RTW89_WW][10] = 62,
+ [0][0][2][0][RTW89_WW][12] = 62,
+ [0][0][2][0][RTW89_WW][14] = 62,
+ [0][0][2][0][RTW89_WW][15] = 60,
+ [0][0][2][0][RTW89_WW][17] = 62,
+ [0][0][2][0][RTW89_WW][19] = 62,
+ [0][0][2][0][RTW89_WW][21] = 62,
+ [0][0][2][0][RTW89_WW][23] = 62,
+ [0][0][2][0][RTW89_WW][25] = 62,
+ [0][0][2][0][RTW89_WW][27] = 62,
+ [0][0][2][0][RTW89_WW][29] = 62,
+ [0][0][2][0][RTW89_WW][31] = 62,
+ [0][0][2][0][RTW89_WW][33] = 62,
+ [0][0][2][0][RTW89_WW][35] = 62,
+ [0][0][2][0][RTW89_WW][37] = 76,
+ [0][0][2][0][RTW89_WW][38] = 30,
+ [0][0][2][0][RTW89_WW][40] = 30,
+ [0][0][2][0][RTW89_WW][42] = 30,
+ [0][0][2][0][RTW89_WW][44] = 30,
+ [0][0][2][0][RTW89_WW][46] = 30,
+ [0][0][2][0][RTW89_WW][48] = 74,
+ [0][0][2][0][RTW89_WW][50] = 76,
+ [0][0][2][0][RTW89_WW][52] = 76,
+ [0][1][2][0][RTW89_WW][0] = 0,
+ [0][1][2][0][RTW89_WW][2] = 0,
+ [0][1][2][0][RTW89_WW][4] = 0,
+ [0][1][2][0][RTW89_WW][6] = 0,
+ [0][1][2][0][RTW89_WW][8] = 0,
+ [0][1][2][0][RTW89_WW][10] = 0,
+ [0][1][2][0][RTW89_WW][12] = 0,
+ [0][1][2][0][RTW89_WW][14] = 0,
+ [0][1][2][0][RTW89_WW][15] = 0,
+ [0][1][2][0][RTW89_WW][17] = 0,
+ [0][1][2][0][RTW89_WW][19] = 0,
+ [0][1][2][0][RTW89_WW][21] = 0,
+ [0][1][2][0][RTW89_WW][23] = 0,
+ [0][1][2][0][RTW89_WW][25] = 0,
+ [0][1][2][0][RTW89_WW][27] = 0,
+ [0][1][2][0][RTW89_WW][29] = 0,
+ [0][1][2][0][RTW89_WW][31] = 0,
+ [0][1][2][0][RTW89_WW][33] = 0,
+ [0][1][2][0][RTW89_WW][35] = 0,
+ [0][1][2][0][RTW89_WW][37] = 0,
+ [0][1][2][0][RTW89_WW][38] = 0,
+ [0][1][2][0][RTW89_WW][40] = 0,
+ [0][1][2][0][RTW89_WW][42] = 0,
+ [0][1][2][0][RTW89_WW][44] = 0,
+ [0][1][2][0][RTW89_WW][46] = 0,
+ [0][1][2][0][RTW89_WW][48] = 0,
+ [0][1][2][0][RTW89_WW][50] = 0,
+ [0][1][2][0][RTW89_WW][52] = 0,
+ [0][1][2][1][RTW89_WW][0] = 0,
+ [0][1][2][1][RTW89_WW][2] = 0,
+ [0][1][2][1][RTW89_WW][4] = 0,
+ [0][1][2][1][RTW89_WW][6] = 0,
+ [0][1][2][1][RTW89_WW][8] = 0,
+ [0][1][2][1][RTW89_WW][10] = 0,
+ [0][1][2][1][RTW89_WW][12] = 0,
+ [0][1][2][1][RTW89_WW][14] = 0,
+ [0][1][2][1][RTW89_WW][15] = 0,
+ [0][1][2][1][RTW89_WW][17] = 0,
+ [0][1][2][1][RTW89_WW][19] = 0,
+ [0][1][2][1][RTW89_WW][21] = 0,
+ [0][1][2][1][RTW89_WW][23] = 0,
+ [0][1][2][1][RTW89_WW][25] = 0,
+ [0][1][2][1][RTW89_WW][27] = 0,
+ [0][1][2][1][RTW89_WW][29] = 0,
+ [0][1][2][1][RTW89_WW][31] = 0,
+ [0][1][2][1][RTW89_WW][33] = 0,
+ [0][1][2][1][RTW89_WW][35] = 0,
+ [0][1][2][1][RTW89_WW][37] = 0,
+ [0][1][2][1][RTW89_WW][38] = 0,
+ [0][1][2][1][RTW89_WW][40] = 0,
+ [0][1][2][1][RTW89_WW][42] = 0,
+ [0][1][2][1][RTW89_WW][44] = 0,
+ [0][1][2][1][RTW89_WW][46] = 0,
+ [0][1][2][1][RTW89_WW][48] = 0,
+ [0][1][2][1][RTW89_WW][50] = 0,
+ [0][1][2][1][RTW89_WW][52] = 0,
+ [1][0][2][0][RTW89_WW][1] = 64,
+ [1][0][2][0][RTW89_WW][5] = 62,
+ [1][0][2][0][RTW89_WW][9] = 64,
+ [1][0][2][0][RTW89_WW][13] = 64,
+ [1][0][2][0][RTW89_WW][16] = 66,
+ [1][0][2][0][RTW89_WW][20] = 66,
+ [1][0][2][0][RTW89_WW][24] = 66,
+ [1][0][2][0][RTW89_WW][28] = 66,
+ [1][0][2][0][RTW89_WW][32] = 66,
+ [1][0][2][0][RTW89_WW][36] = 76,
+ [1][0][2][0][RTW89_WW][39] = 30,
+ [1][0][2][0][RTW89_WW][43] = 30,
+ [1][0][2][0][RTW89_WW][47] = 84,
+ [1][0][2][0][RTW89_WW][51] = 84,
+ [1][1][2][0][RTW89_WW][1] = 0,
+ [1][1][2][0][RTW89_WW][5] = 0,
+ [1][1][2][0][RTW89_WW][9] = 0,
+ [1][1][2][0][RTW89_WW][13] = 0,
+ [1][1][2][0][RTW89_WW][16] = 0,
+ [1][1][2][0][RTW89_WW][20] = 0,
+ [1][1][2][0][RTW89_WW][24] = 0,
+ [1][1][2][0][RTW89_WW][28] = 0,
+ [1][1][2][0][RTW89_WW][32] = 0,
+ [1][1][2][0][RTW89_WW][36] = 0,
+ [1][1][2][0][RTW89_WW][39] = 0,
+ [1][1][2][0][RTW89_WW][43] = 0,
+ [1][1][2][0][RTW89_WW][47] = 0,
+ [1][1][2][0][RTW89_WW][51] = 0,
+ [1][1][2][1][RTW89_WW][1] = 0,
+ [1][1][2][1][RTW89_WW][5] = 0,
+ [1][1][2][1][RTW89_WW][9] = 0,
+ [1][1][2][1][RTW89_WW][13] = 0,
+ [1][1][2][1][RTW89_WW][16] = 0,
+ [1][1][2][1][RTW89_WW][20] = 0,
+ [1][1][2][1][RTW89_WW][24] = 0,
+ [1][1][2][1][RTW89_WW][28] = 0,
+ [1][1][2][1][RTW89_WW][32] = 0,
+ [1][1][2][1][RTW89_WW][36] = 0,
+ [1][1][2][1][RTW89_WW][39] = 0,
+ [1][1][2][1][RTW89_WW][43] = 0,
+ [1][1][2][1][RTW89_WW][47] = 0,
+ [1][1][2][1][RTW89_WW][51] = 0,
+ [2][0][2][0][RTW89_WW][3] = 62,
+ [2][0][2][0][RTW89_WW][11] = 62,
+ [2][0][2][0][RTW89_WW][18] = 64,
+ [2][0][2][0][RTW89_WW][26] = 64,
+ [2][0][2][0][RTW89_WW][34] = 72,
+ [2][0][2][0][RTW89_WW][41] = 30,
+ [2][0][2][0][RTW89_WW][49] = 74,
+ [2][1][2][0][RTW89_WW][3] = 0,
+ [2][1][2][0][RTW89_WW][11] = 0,
+ [2][1][2][0][RTW89_WW][18] = 0,
+ [2][1][2][0][RTW89_WW][26] = 0,
+ [2][1][2][0][RTW89_WW][34] = 0,
+ [2][1][2][0][RTW89_WW][41] = 0,
+ [2][1][2][0][RTW89_WW][49] = 0,
+ [2][1][2][1][RTW89_WW][3] = 0,
+ [2][1][2][1][RTW89_WW][11] = 0,
+ [2][1][2][1][RTW89_WW][18] = 0,
+ [2][1][2][1][RTW89_WW][26] = 0,
+ [2][1][2][1][RTW89_WW][34] = 0,
+ [2][1][2][1][RTW89_WW][41] = 0,
+ [2][1][2][1][RTW89_WW][49] = 0,
+ [3][0][2][0][RTW89_WW][7] = 58,
+ [3][0][2][0][RTW89_WW][22] = 58,
+ [3][0][2][0][RTW89_WW][45] = 0,
+ [3][1][2][0][RTW89_WW][7] = 0,
+ [3][1][2][0][RTW89_WW][22] = 0,
+ [3][1][2][0][RTW89_WW][45] = 0,
+ [3][1][2][1][RTW89_WW][7] = 0,
+ [3][1][2][1][RTW89_WW][22] = 0,
+ [3][1][2][1][RTW89_WW][45] = 0,
+ [0][0][1][0][RTW89_FCC][0] = 80,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 60,
+ [0][0][1][0][RTW89_IC][0] = 62,
+ [0][0][1][0][RTW89_KCC][0] = 74,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CN][0] = 60,
+ [0][0][1][0][RTW89_UK][0] = 58,
+ [0][0][1][0][RTW89_FCC][2] = 82,
+ [0][0][1][0][RTW89_ETSI][2] = 58,
+ [0][0][1][0][RTW89_MKK][2] = 60,
+ [0][0][1][0][RTW89_IC][2] = 62,
+ [0][0][1][0][RTW89_KCC][2] = 74,
+ [0][0][1][0][RTW89_ACMA][2] = 58,
+ [0][0][1][0][RTW89_CN][2] = 60,
+ [0][0][1][0][RTW89_UK][2] = 58,
+ [0][0][1][0][RTW89_FCC][4] = 82,
+ [0][0][1][0][RTW89_ETSI][4] = 58,
+ [0][0][1][0][RTW89_MKK][4] = 60,
+ [0][0][1][0][RTW89_IC][4] = 62,
+ [0][0][1][0][RTW89_KCC][4] = 74,
+ [0][0][1][0][RTW89_ACMA][4] = 58,
+ [0][0][1][0][RTW89_CN][4] = 60,
+ [0][0][1][0][RTW89_UK][4] = 58,
+ [0][0][1][0][RTW89_FCC][6] = 82,
+ [0][0][1][0][RTW89_ETSI][6] = 58,
+ [0][0][1][0][RTW89_MKK][6] = 60,
+ [0][0][1][0][RTW89_IC][6] = 62,
+ [0][0][1][0][RTW89_KCC][6] = 50,
+ [0][0][1][0][RTW89_ACMA][6] = 58,
+ [0][0][1][0][RTW89_CN][6] = 60,
+ [0][0][1][0][RTW89_UK][6] = 58,
+ [0][0][1][0][RTW89_FCC][8] = 82,
+ [0][0][1][0][RTW89_ETSI][8] = 58,
+ [0][0][1][0][RTW89_MKK][8] = 60,
+ [0][0][1][0][RTW89_IC][8] = 64,
+ [0][0][1][0][RTW89_KCC][8] = 74,
+ [0][0][1][0][RTW89_ACMA][8] = 58,
+ [0][0][1][0][RTW89_CN][8] = 60,
+ [0][0][1][0][RTW89_UK][8] = 58,
+ [0][0][1][0][RTW89_FCC][10] = 82,
+ [0][0][1][0][RTW89_ETSI][10] = 58,
+ [0][0][1][0][RTW89_MKK][10] = 60,
+ [0][0][1][0][RTW89_IC][10] = 64,
+ [0][0][1][0][RTW89_KCC][10] = 74,
+ [0][0][1][0][RTW89_ACMA][10] = 58,
+ [0][0][1][0][RTW89_CN][10] = 60,
+ [0][0][1][0][RTW89_UK][10] = 58,
+ [0][0][1][0][RTW89_FCC][12] = 82,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 60,
+ [0][0][1][0][RTW89_IC][12] = 64,
+ [0][0][1][0][RTW89_KCC][12] = 76,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CN][12] = 60,
+ [0][0][1][0][RTW89_UK][12] = 58,
+ [0][0][1][0][RTW89_FCC][14] = 78,
+ [0][0][1][0][RTW89_ETSI][14] = 58,
+ [0][0][1][0][RTW89_MKK][14] = 60,
+ [0][0][1][0][RTW89_IC][14] = 64,
+ [0][0][1][0][RTW89_KCC][14] = 76,
+ [0][0][1][0][RTW89_ACMA][14] = 58,
+ [0][0][1][0][RTW89_CN][14] = 60,
+ [0][0][1][0][RTW89_UK][14] = 58,
+ [0][0][1][0][RTW89_FCC][15] = 78,
+ [0][0][1][0][RTW89_ETSI][15] = 58,
+ [0][0][1][0][RTW89_MKK][15] = 78,
+ [0][0][1][0][RTW89_IC][15] = 78,
+ [0][0][1][0][RTW89_KCC][15] = 78,
+ [0][0][1][0][RTW89_ACMA][15] = 58,
+ [0][0][1][0][RTW89_CN][15] = 127,
+ [0][0][1][0][RTW89_UK][15] = 58,
+ [0][0][1][0][RTW89_FCC][17] = 82,
+ [0][0][1][0][RTW89_ETSI][17] = 60,
+ [0][0][1][0][RTW89_MKK][17] = 78,
+ [0][0][1][0][RTW89_IC][17] = 82,
+ [0][0][1][0][RTW89_KCC][17] = 78,
+ [0][0][1][0][RTW89_ACMA][17] = 60,
+ [0][0][1][0][RTW89_CN][17] = 127,
+ [0][0][1][0][RTW89_UK][17] = 60,
+ [0][0][1][0][RTW89_FCC][19] = 82,
+ [0][0][1][0][RTW89_ETSI][19] = 60,
+ [0][0][1][0][RTW89_MKK][19] = 78,
+ [0][0][1][0][RTW89_IC][19] = 82,
+ [0][0][1][0][RTW89_KCC][19] = 78,
+ [0][0][1][0][RTW89_ACMA][19] = 60,
+ [0][0][1][0][RTW89_CN][19] = 127,
+ [0][0][1][0][RTW89_UK][19] = 60,
+ [0][0][1][0][RTW89_FCC][21] = 82,
+ [0][0][1][0][RTW89_ETSI][21] = 60,
+ [0][0][1][0][RTW89_MKK][21] = 78,
+ [0][0][1][0][RTW89_IC][21] = 82,
+ [0][0][1][0][RTW89_KCC][21] = 78,
+ [0][0][1][0][RTW89_ACMA][21] = 60,
+ [0][0][1][0][RTW89_CN][21] = 127,
+ [0][0][1][0][RTW89_UK][21] = 60,
+ [0][0][1][0][RTW89_FCC][23] = 82,
+ [0][0][1][0][RTW89_ETSI][23] = 60,
+ [0][0][1][0][RTW89_MKK][23] = 78,
+ [0][0][1][0][RTW89_IC][23] = 82,
+ [0][0][1][0][RTW89_KCC][23] = 78,
+ [0][0][1][0][RTW89_ACMA][23] = 60,
+ [0][0][1][0][RTW89_CN][23] = 127,
+ [0][0][1][0][RTW89_UK][23] = 60,
+ [0][0][1][0][RTW89_FCC][25] = 82,
+ [0][0][1][0][RTW89_ETSI][25] = 60,
+ [0][0][1][0][RTW89_MKK][25] = 78,
+ [0][0][1][0][RTW89_IC][25] = 127,
+ [0][0][1][0][RTW89_KCC][25] = 78,
+ [0][0][1][0][RTW89_ACMA][25] = 127,
+ [0][0][1][0][RTW89_CN][25] = 127,
+ [0][0][1][0][RTW89_UK][25] = 60,
+ [0][0][1][0][RTW89_FCC][27] = 82,
+ [0][0][1][0][RTW89_ETSI][27] = 60,
+ [0][0][1][0][RTW89_MKK][27] = 78,
+ [0][0][1][0][RTW89_IC][27] = 127,
+ [0][0][1][0][RTW89_KCC][27] = 78,
+ [0][0][1][0][RTW89_ACMA][27] = 127,
+ [0][0][1][0][RTW89_CN][27] = 127,
+ [0][0][1][0][RTW89_UK][27] = 60,
+ [0][0][1][0][RTW89_FCC][29] = 82,
+ [0][0][1][0][RTW89_ETSI][29] = 60,
+ [0][0][1][0][RTW89_MKK][29] = 78,
+ [0][0][1][0][RTW89_IC][29] = 127,
+ [0][0][1][0][RTW89_KCC][29] = 78,
+ [0][0][1][0][RTW89_ACMA][29] = 127,
+ [0][0][1][0][RTW89_CN][29] = 127,
+ [0][0][1][0][RTW89_UK][29] = 60,
+ [0][0][1][0][RTW89_FCC][31] = 82,
+ [0][0][1][0][RTW89_ETSI][31] = 60,
+ [0][0][1][0][RTW89_MKK][31] = 78,
+ [0][0][1][0][RTW89_IC][31] = 82,
+ [0][0][1][0][RTW89_KCC][31] = 74,
+ [0][0][1][0][RTW89_ACMA][31] = 60,
+ [0][0][1][0][RTW89_CN][31] = 127,
+ [0][0][1][0][RTW89_UK][31] = 60,
+ [0][0][1][0][RTW89_FCC][33] = 82,
+ [0][0][1][0][RTW89_ETSI][33] = 60,
+ [0][0][1][0][RTW89_MKK][33] = 78,
+ [0][0][1][0][RTW89_IC][33] = 82,
+ [0][0][1][0][RTW89_KCC][33] = 74,
+ [0][0][1][0][RTW89_ACMA][33] = 60,
+ [0][0][1][0][RTW89_CN][33] = 127,
+ [0][0][1][0][RTW89_UK][33] = 60,
+ [0][0][1][0][RTW89_FCC][35] = 72,
+ [0][0][1][0][RTW89_ETSI][35] = 60,
+ [0][0][1][0][RTW89_MKK][35] = 78,
+ [0][0][1][0][RTW89_IC][35] = 72,
+ [0][0][1][0][RTW89_KCC][35] = 74,
+ [0][0][1][0][RTW89_ACMA][35] = 60,
+ [0][0][1][0][RTW89_CN][35] = 127,
+ [0][0][1][0][RTW89_UK][35] = 60,
+ [0][0][1][0][RTW89_FCC][37] = 82,
+ [0][0][1][0][RTW89_ETSI][37] = 127,
+ [0][0][1][0][RTW89_MKK][37] = 78,
+ [0][0][1][0][RTW89_IC][37] = 82,
+ [0][0][1][0][RTW89_KCC][37] = 74,
+ [0][0][1][0][RTW89_ACMA][37] = 78,
+ [0][0][1][0][RTW89_CN][37] = 127,
+ [0][0][1][0][RTW89_UK][37] = 78,
+ [0][0][1][0][RTW89_FCC][38] = 82,
+ [0][0][1][0][RTW89_ETSI][38] = 30,
+ [0][0][1][0][RTW89_MKK][38] = 127,
+ [0][0][1][0][RTW89_IC][38] = 82,
+ [0][0][1][0][RTW89_KCC][38] = 70,
+ [0][0][1][0][RTW89_ACMA][38] = 78,
+ [0][0][1][0][RTW89_CN][38] = 78,
+ [0][0][1][0][RTW89_UK][38] = 58,
+ [0][0][1][0][RTW89_FCC][40] = 82,
+ [0][0][1][0][RTW89_ETSI][40] = 30,
+ [0][0][1][0][RTW89_MKK][40] = 127,
+ [0][0][1][0][RTW89_IC][40] = 82,
+ [0][0][1][0][RTW89_KCC][40] = 76,
+ [0][0][1][0][RTW89_ACMA][40] = 78,
+ [0][0][1][0][RTW89_CN][40] = 78,
+ [0][0][1][0][RTW89_UK][40] = 58,
+ [0][0][1][0][RTW89_FCC][42] = 82,
+ [0][0][1][0][RTW89_ETSI][42] = 30,
+ [0][0][1][0][RTW89_MKK][42] = 127,
+ [0][0][1][0][RTW89_IC][42] = 82,
+ [0][0][1][0][RTW89_KCC][42] = 76,
+ [0][0][1][0][RTW89_ACMA][42] = 78,
+ [0][0][1][0][RTW89_CN][42] = 78,
+ [0][0][1][0][RTW89_UK][42] = 58,
+ [0][0][1][0][RTW89_FCC][44] = 82,
+ [0][0][1][0][RTW89_ETSI][44] = 30,
+ [0][0][1][0][RTW89_MKK][44] = 127,
+ [0][0][1][0][RTW89_IC][44] = 82,
+ [0][0][1][0][RTW89_KCC][44] = 76,
+ [0][0][1][0][RTW89_ACMA][44] = 78,
+ [0][0][1][0][RTW89_CN][44] = 78,
+ [0][0][1][0][RTW89_UK][44] = 58,
+ [0][0][1][0][RTW89_FCC][46] = 82,
+ [0][0][1][0][RTW89_ETSI][46] = 30,
+ [0][0][1][0][RTW89_MKK][46] = 127,
+ [0][0][1][0][RTW89_IC][46] = 82,
+ [0][0][1][0][RTW89_KCC][46] = 76,
+ [0][0][1][0][RTW89_ACMA][46] = 78,
+ [0][0][1][0][RTW89_CN][46] = 78,
+ [0][0][1][0][RTW89_UK][46] = 58,
+ [0][0][1][0][RTW89_FCC][48] = 72,
+ [0][0][1][0][RTW89_ETSI][48] = 127,
+ [0][0][1][0][RTW89_MKK][48] = 127,
+ [0][0][1][0][RTW89_IC][48] = 127,
+ [0][0][1][0][RTW89_KCC][48] = 127,
+ [0][0][1][0][RTW89_ACMA][48] = 127,
+ [0][0][1][0][RTW89_CN][48] = 127,
+ [0][0][1][0][RTW89_UK][48] = 127,
+ [0][0][1][0][RTW89_FCC][50] = 72,
+ [0][0][1][0][RTW89_ETSI][50] = 127,
+ [0][0][1][0][RTW89_MKK][50] = 127,
+ [0][0][1][0][RTW89_IC][50] = 127,
+ [0][0][1][0][RTW89_KCC][50] = 127,
+ [0][0][1][0][RTW89_ACMA][50] = 127,
+ [0][0][1][0][RTW89_CN][50] = 127,
+ [0][0][1][0][RTW89_UK][50] = 127,
+ [0][0][1][0][RTW89_FCC][52] = 72,
+ [0][0][1][0][RTW89_ETSI][52] = 127,
+ [0][0][1][0][RTW89_MKK][52] = 127,
+ [0][0][1][0][RTW89_IC][52] = 127,
+ [0][0][1][0][RTW89_KCC][52] = 127,
+ [0][0][1][0][RTW89_ACMA][52] = 127,
+ [0][0][1][0][RTW89_CN][52] = 127,
+ [0][0][1][0][RTW89_UK][52] = 127,
+ [0][1][1][0][RTW89_FCC][0] = 127,
+ [0][1][1][0][RTW89_ETSI][0] = 127,
+ [0][1][1][0][RTW89_MKK][0] = 127,
+ [0][1][1][0][RTW89_IC][0] = 127,
+ [0][1][1][0][RTW89_KCC][0] = 127,
+ [0][1][1][0][RTW89_ACMA][0] = 127,
+ [0][1][1][0][RTW89_CN][0] = 127,
+ [0][1][1][0][RTW89_UK][0] = 127,
+ [0][1][1][0][RTW89_FCC][2] = 127,
+ [0][1][1][0][RTW89_ETSI][2] = 127,
+ [0][1][1][0][RTW89_MKK][2] = 127,
+ [0][1][1][0][RTW89_IC][2] = 127,
+ [0][1][1][0][RTW89_KCC][2] = 127,
+ [0][1][1][0][RTW89_ACMA][2] = 127,
+ [0][1][1][0][RTW89_CN][2] = 127,
+ [0][1][1][0][RTW89_UK][2] = 127,
+ [0][1][1][0][RTW89_FCC][4] = 127,
+ [0][1][1][0][RTW89_ETSI][4] = 127,
+ [0][1][1][0][RTW89_MKK][4] = 127,
+ [0][1][1][0][RTW89_IC][4] = 127,
+ [0][1][1][0][RTW89_KCC][4] = 127,
+ [0][1][1][0][RTW89_ACMA][4] = 127,
+ [0][1][1][0][RTW89_CN][4] = 127,
+ [0][1][1][0][RTW89_UK][4] = 127,
+ [0][1][1][0][RTW89_FCC][6] = 127,
+ [0][1][1][0][RTW89_ETSI][6] = 127,
+ [0][1][1][0][RTW89_MKK][6] = 127,
+ [0][1][1][0][RTW89_IC][6] = 127,
+ [0][1][1][0][RTW89_KCC][6] = 127,
+ [0][1][1][0][RTW89_ACMA][6] = 127,
+ [0][1][1][0][RTW89_CN][6] = 127,
+ [0][1][1][0][RTW89_UK][6] = 127,
+ [0][1][1][0][RTW89_FCC][8] = 127,
+ [0][1][1][0][RTW89_ETSI][8] = 127,
+ [0][1][1][0][RTW89_MKK][8] = 127,
+ [0][1][1][0][RTW89_IC][8] = 127,
+ [0][1][1][0][RTW89_KCC][8] = 127,
+ [0][1][1][0][RTW89_ACMA][8] = 127,
+ [0][1][1][0][RTW89_CN][8] = 127,
+ [0][1][1][0][RTW89_UK][8] = 127,
+ [0][1][1][0][RTW89_FCC][10] = 127,
+ [0][1][1][0][RTW89_ETSI][10] = 127,
+ [0][1][1][0][RTW89_MKK][10] = 127,
+ [0][1][1][0][RTW89_IC][10] = 127,
+ [0][1][1][0][RTW89_KCC][10] = 127,
+ [0][1][1][0][RTW89_ACMA][10] = 127,
+ [0][1][1][0][RTW89_CN][10] = 127,
+ [0][1][1][0][RTW89_UK][10] = 127,
+ [0][1][1][0][RTW89_FCC][12] = 127,
+ [0][1][1][0][RTW89_ETSI][12] = 127,
+ [0][1][1][0][RTW89_MKK][12] = 127,
+ [0][1][1][0][RTW89_IC][12] = 127,
+ [0][1][1][0][RTW89_KCC][12] = 127,
+ [0][1][1][0][RTW89_ACMA][12] = 127,
+ [0][1][1][0][RTW89_CN][12] = 127,
+ [0][1][1][0][RTW89_UK][12] = 127,
+ [0][1][1][0][RTW89_FCC][14] = 127,
+ [0][1][1][0][RTW89_ETSI][14] = 127,
+ [0][1][1][0][RTW89_MKK][14] = 127,
+ [0][1][1][0][RTW89_IC][14] = 127,
+ [0][1][1][0][RTW89_KCC][14] = 127,
+ [0][1][1][0][RTW89_ACMA][14] = 127,
+ [0][1][1][0][RTW89_CN][14] = 127,
+ [0][1][1][0][RTW89_UK][14] = 127,
+ [0][1][1][0][RTW89_FCC][15] = 127,
+ [0][1][1][0][RTW89_ETSI][15] = 127,
+ [0][1][1][0][RTW89_MKK][15] = 127,
+ [0][1][1][0][RTW89_IC][15] = 127,
+ [0][1][1][0][RTW89_KCC][15] = 127,
+ [0][1][1][0][RTW89_ACMA][15] = 127,
+ [0][1][1][0][RTW89_CN][15] = 127,
+ [0][1][1][0][RTW89_UK][15] = 127,
+ [0][1][1][0][RTW89_FCC][17] = 127,
+ [0][1][1][0][RTW89_ETSI][17] = 127,
+ [0][1][1][0][RTW89_MKK][17] = 127,
+ [0][1][1][0][RTW89_IC][17] = 127,
+ [0][1][1][0][RTW89_KCC][17] = 127,
+ [0][1][1][0][RTW89_ACMA][17] = 127,
+ [0][1][1][0][RTW89_CN][17] = 127,
+ [0][1][1][0][RTW89_UK][17] = 127,
+ [0][1][1][0][RTW89_FCC][19] = 127,
+ [0][1][1][0][RTW89_ETSI][19] = 127,
+ [0][1][1][0][RTW89_MKK][19] = 127,
+ [0][1][1][0][RTW89_IC][19] = 127,
+ [0][1][1][0][RTW89_KCC][19] = 127,
+ [0][1][1][0][RTW89_ACMA][19] = 127,
+ [0][1][1][0][RTW89_CN][19] = 127,
+ [0][1][1][0][RTW89_UK][19] = 127,
+ [0][1][1][0][RTW89_FCC][21] = 127,
+ [0][1][1][0][RTW89_ETSI][21] = 127,
+ [0][1][1][0][RTW89_MKK][21] = 127,
+ [0][1][1][0][RTW89_IC][21] = 127,
+ [0][1][1][0][RTW89_KCC][21] = 127,
+ [0][1][1][0][RTW89_ACMA][21] = 127,
+ [0][1][1][0][RTW89_CN][21] = 127,
+ [0][1][1][0][RTW89_UK][21] = 127,
+ [0][1][1][0][RTW89_FCC][23] = 127,
+ [0][1][1][0][RTW89_ETSI][23] = 127,
+ [0][1][1][0][RTW89_MKK][23] = 127,
+ [0][1][1][0][RTW89_IC][23] = 127,
+ [0][1][1][0][RTW89_KCC][23] = 127,
+ [0][1][1][0][RTW89_ACMA][23] = 127,
+ [0][1][1][0][RTW89_CN][23] = 127,
+ [0][1][1][0][RTW89_UK][23] = 127,
+ [0][1][1][0][RTW89_FCC][25] = 127,
+ [0][1][1][0][RTW89_ETSI][25] = 127,
+ [0][1][1][0][RTW89_MKK][25] = 127,
+ [0][1][1][0][RTW89_IC][25] = 127,
+ [0][1][1][0][RTW89_KCC][25] = 127,
+ [0][1][1][0][RTW89_ACMA][25] = 127,
+ [0][1][1][0][RTW89_CN][25] = 127,
+ [0][1][1][0][RTW89_UK][25] = 127,
+ [0][1][1][0][RTW89_FCC][27] = 127,
+ [0][1][1][0][RTW89_ETSI][27] = 127,
+ [0][1][1][0][RTW89_MKK][27] = 127,
+ [0][1][1][0][RTW89_IC][27] = 127,
+ [0][1][1][0][RTW89_KCC][27] = 127,
+ [0][1][1][0][RTW89_ACMA][27] = 127,
+ [0][1][1][0][RTW89_CN][27] = 127,
+ [0][1][1][0][RTW89_UK][27] = 127,
+ [0][1][1][0][RTW89_FCC][29] = 127,
+ [0][1][1][0][RTW89_ETSI][29] = 127,
+ [0][1][1][0][RTW89_MKK][29] = 127,
+ [0][1][1][0][RTW89_IC][29] = 127,
+ [0][1][1][0][RTW89_KCC][29] = 127,
+ [0][1][1][0][RTW89_ACMA][29] = 127,
+ [0][1][1][0][RTW89_CN][29] = 127,
+ [0][1][1][0][RTW89_UK][29] = 127,
+ [0][1][1][0][RTW89_FCC][31] = 127,
+ [0][1][1][0][RTW89_ETSI][31] = 127,
+ [0][1][1][0][RTW89_MKK][31] = 127,
+ [0][1][1][0][RTW89_IC][31] = 127,
+ [0][1][1][0][RTW89_KCC][31] = 127,
+ [0][1][1][0][RTW89_ACMA][31] = 127,
+ [0][1][1][0][RTW89_CN][31] = 127,
+ [0][1][1][0][RTW89_UK][31] = 127,
+ [0][1][1][0][RTW89_FCC][33] = 127,
+ [0][1][1][0][RTW89_ETSI][33] = 127,
+ [0][1][1][0][RTW89_MKK][33] = 127,
+ [0][1][1][0][RTW89_IC][33] = 127,
+ [0][1][1][0][RTW89_KCC][33] = 127,
+ [0][1][1][0][RTW89_ACMA][33] = 127,
+ [0][1][1][0][RTW89_CN][33] = 127,
+ [0][1][1][0][RTW89_UK][33] = 127,
+ [0][1][1][0][RTW89_FCC][35] = 127,
+ [0][1][1][0][RTW89_ETSI][35] = 127,
+ [0][1][1][0][RTW89_MKK][35] = 127,
+ [0][1][1][0][RTW89_IC][35] = 127,
+ [0][1][1][0][RTW89_KCC][35] = 127,
+ [0][1][1][0][RTW89_ACMA][35] = 127,
+ [0][1][1][0][RTW89_CN][35] = 127,
+ [0][1][1][0][RTW89_UK][35] = 127,
+ [0][1][1][0][RTW89_FCC][37] = 127,
+ [0][1][1][0][RTW89_ETSI][37] = 127,
+ [0][1][1][0][RTW89_MKK][37] = 127,
+ [0][1][1][0][RTW89_IC][37] = 127,
+ [0][1][1][0][RTW89_KCC][37] = 127,
+ [0][1][1][0][RTW89_ACMA][37] = 127,
+ [0][1][1][0][RTW89_CN][37] = 127,
+ [0][1][1][0][RTW89_UK][37] = 127,
+ [0][1][1][0][RTW89_FCC][38] = 127,
+ [0][1][1][0][RTW89_ETSI][38] = 127,
+ [0][1][1][0][RTW89_MKK][38] = 127,
+ [0][1][1][0][RTW89_IC][38] = 127,
+ [0][1][1][0][RTW89_KCC][38] = 127,
+ [0][1][1][0][RTW89_ACMA][38] = 127,
+ [0][1][1][0][RTW89_CN][38] = 127,
+ [0][1][1][0][RTW89_UK][38] = 127,
+ [0][1][1][0][RTW89_FCC][40] = 127,
+ [0][1][1][0][RTW89_ETSI][40] = 127,
+ [0][1][1][0][RTW89_MKK][40] = 127,
+ [0][1][1][0][RTW89_IC][40] = 127,
+ [0][1][1][0][RTW89_KCC][40] = 127,
+ [0][1][1][0][RTW89_ACMA][40] = 127,
+ [0][1][1][0][RTW89_CN][40] = 127,
+ [0][1][1][0][RTW89_UK][40] = 127,
+ [0][1][1][0][RTW89_FCC][42] = 127,
+ [0][1][1][0][RTW89_ETSI][42] = 127,
+ [0][1][1][0][RTW89_MKK][42] = 127,
+ [0][1][1][0][RTW89_IC][42] = 127,
+ [0][1][1][0][RTW89_KCC][42] = 127,
+ [0][1][1][0][RTW89_ACMA][42] = 127,
+ [0][1][1][0][RTW89_CN][42] = 127,
+ [0][1][1][0][RTW89_UK][42] = 127,
+ [0][1][1][0][RTW89_FCC][44] = 127,
+ [0][1][1][0][RTW89_ETSI][44] = 127,
+ [0][1][1][0][RTW89_MKK][44] = 127,
+ [0][1][1][0][RTW89_IC][44] = 127,
+ [0][1][1][0][RTW89_KCC][44] = 127,
+ [0][1][1][0][RTW89_ACMA][44] = 127,
+ [0][1][1][0][RTW89_CN][44] = 127,
+ [0][1][1][0][RTW89_UK][44] = 127,
+ [0][1][1][0][RTW89_FCC][46] = 127,
+ [0][1][1][0][RTW89_ETSI][46] = 127,
+ [0][1][1][0][RTW89_MKK][46] = 127,
+ [0][1][1][0][RTW89_IC][46] = 127,
+ [0][1][1][0][RTW89_KCC][46] = 127,
+ [0][1][1][0][RTW89_ACMA][46] = 127,
+ [0][1][1][0][RTW89_CN][46] = 127,
+ [0][1][1][0][RTW89_UK][46] = 127,
+ [0][1][1][0][RTW89_FCC][48] = 127,
+ [0][1][1][0][RTW89_ETSI][48] = 127,
+ [0][1][1][0][RTW89_MKK][48] = 127,
+ [0][1][1][0][RTW89_IC][48] = 127,
+ [0][1][1][0][RTW89_KCC][48] = 127,
+ [0][1][1][0][RTW89_ACMA][48] = 127,
+ [0][1][1][0][RTW89_CN][48] = 127,
+ [0][1][1][0][RTW89_UK][48] = 127,
+ [0][1][1][0][RTW89_FCC][50] = 127,
+ [0][1][1][0][RTW89_ETSI][50] = 127,
+ [0][1][1][0][RTW89_MKK][50] = 127,
+ [0][1][1][0][RTW89_IC][50] = 127,
+ [0][1][1][0][RTW89_KCC][50] = 127,
+ [0][1][1][0][RTW89_ACMA][50] = 127,
+ [0][1][1][0][RTW89_CN][50] = 127,
+ [0][1][1][0][RTW89_UK][50] = 127,
+ [0][1][1][0][RTW89_FCC][52] = 127,
+ [0][1][1][0][RTW89_ETSI][52] = 127,
+ [0][1][1][0][RTW89_MKK][52] = 127,
+ [0][1][1][0][RTW89_IC][52] = 127,
+ [0][1][1][0][RTW89_KCC][52] = 127,
+ [0][1][1][0][RTW89_ACMA][52] = 127,
+ [0][1][1][0][RTW89_CN][52] = 127,
+ [0][1][1][0][RTW89_UK][52] = 127,
+ [0][0][2][0][RTW89_FCC][0] = 78,
+ [0][0][2][0][RTW89_ETSI][0] = 62,
+ [0][0][2][0][RTW89_MKK][0] = 62,
+ [0][0][2][0][RTW89_IC][0] = 64,
+ [0][0][2][0][RTW89_KCC][0] = 76,
+ [0][0][2][0][RTW89_ACMA][0] = 62,
+ [0][0][2][0][RTW89_CN][0] = 62,
+ [0][0][2][0][RTW89_UK][0] = 62,
+ [0][0][2][0][RTW89_FCC][2] = 82,
+ [0][0][2][0][RTW89_ETSI][2] = 62,
+ [0][0][2][0][RTW89_MKK][2] = 62,
+ [0][0][2][0][RTW89_IC][2] = 64,
+ [0][0][2][0][RTW89_KCC][2] = 76,
+ [0][0][2][0][RTW89_ACMA][2] = 62,
+ [0][0][2][0][RTW89_CN][2] = 62,
+ [0][0][2][0][RTW89_UK][2] = 62,
+ [0][0][2][0][RTW89_FCC][4] = 82,
+ [0][0][2][0][RTW89_ETSI][4] = 62,
+ [0][0][2][0][RTW89_MKK][4] = 62,
+ [0][0][2][0][RTW89_IC][4] = 64,
+ [0][0][2][0][RTW89_KCC][4] = 76,
+ [0][0][2][0][RTW89_ACMA][4] = 62,
+ [0][0][2][0][RTW89_CN][4] = 62,
+ [0][0][2][0][RTW89_UK][4] = 62,
+ [0][0][2][0][RTW89_FCC][6] = 82,
+ [0][0][2][0][RTW89_ETSI][6] = 62,
+ [0][0][2][0][RTW89_MKK][6] = 62,
+ [0][0][2][0][RTW89_IC][6] = 64,
+ [0][0][2][0][RTW89_KCC][6] = 54,
+ [0][0][2][0][RTW89_ACMA][6] = 62,
+ [0][0][2][0][RTW89_CN][6] = 62,
+ [0][0][2][0][RTW89_UK][6] = 62,
+ [0][0][2][0][RTW89_FCC][8] = 82,
+ [0][0][2][0][RTW89_ETSI][8] = 62,
+ [0][0][2][0][RTW89_MKK][8] = 62,
+ [0][0][2][0][RTW89_IC][8] = 64,
+ [0][0][2][0][RTW89_KCC][8] = 76,
+ [0][0][2][0][RTW89_ACMA][8] = 62,
+ [0][0][2][0][RTW89_CN][8] = 62,
+ [0][0][2][0][RTW89_UK][8] = 62,
+ [0][0][2][0][RTW89_FCC][10] = 82,
+ [0][0][2][0][RTW89_ETSI][10] = 62,
+ [0][0][2][0][RTW89_MKK][10] = 62,
+ [0][0][2][0][RTW89_IC][10] = 64,
+ [0][0][2][0][RTW89_KCC][10] = 76,
+ [0][0][2][0][RTW89_ACMA][10] = 62,
+ [0][0][2][0][RTW89_CN][10] = 62,
+ [0][0][2][0][RTW89_UK][10] = 62,
+ [0][0][2][0][RTW89_FCC][12] = 82,
+ [0][0][2][0][RTW89_ETSI][12] = 62,
+ [0][0][2][0][RTW89_MKK][12] = 62,
+ [0][0][2][0][RTW89_IC][12] = 64,
+ [0][0][2][0][RTW89_KCC][12] = 78,
+ [0][0][2][0][RTW89_ACMA][12] = 62,
+ [0][0][2][0][RTW89_CN][12] = 62,
+ [0][0][2][0][RTW89_UK][12] = 62,
+ [0][0][2][0][RTW89_FCC][14] = 76,
+ [0][0][2][0][RTW89_ETSI][14] = 62,
+ [0][0][2][0][RTW89_MKK][14] = 62,
+ [0][0][2][0][RTW89_IC][14] = 64,
+ [0][0][2][0][RTW89_KCC][14] = 78,
+ [0][0][2][0][RTW89_ACMA][14] = 62,
+ [0][0][2][0][RTW89_CN][14] = 62,
+ [0][0][2][0][RTW89_UK][14] = 62,
+ [0][0][2][0][RTW89_FCC][15] = 76,
+ [0][0][2][0][RTW89_ETSI][15] = 60,
+ [0][0][2][0][RTW89_MKK][15] = 78,
+ [0][0][2][0][RTW89_IC][15] = 76,
+ [0][0][2][0][RTW89_KCC][15] = 78,
+ [0][0][2][0][RTW89_ACMA][15] = 60,
+ [0][0][2][0][RTW89_CN][15] = 127,
+ [0][0][2][0][RTW89_UK][15] = 60,
+ [0][0][2][0][RTW89_FCC][17] = 82,
+ [0][0][2][0][RTW89_ETSI][17] = 62,
+ [0][0][2][0][RTW89_MKK][17] = 78,
+ [0][0][2][0][RTW89_IC][17] = 82,
+ [0][0][2][0][RTW89_KCC][17] = 78,
+ [0][0][2][0][RTW89_ACMA][17] = 62,
+ [0][0][2][0][RTW89_CN][17] = 127,
+ [0][0][2][0][RTW89_UK][17] = 62,
+ [0][0][2][0][RTW89_FCC][19] = 82,
+ [0][0][2][0][RTW89_ETSI][19] = 62,
+ [0][0][2][0][RTW89_MKK][19] = 78,
+ [0][0][2][0][RTW89_IC][19] = 82,
+ [0][0][2][0][RTW89_KCC][19] = 78,
+ [0][0][2][0][RTW89_ACMA][19] = 62,
+ [0][0][2][0][RTW89_CN][19] = 127,
+ [0][0][2][0][RTW89_UK][19] = 62,
+ [0][0][2][0][RTW89_FCC][21] = 82,
+ [0][0][2][0][RTW89_ETSI][21] = 62,
+ [0][0][2][0][RTW89_MKK][21] = 78,
+ [0][0][2][0][RTW89_IC][21] = 82,
+ [0][0][2][0][RTW89_KCC][21] = 78,
+ [0][0][2][0][RTW89_ACMA][21] = 62,
+ [0][0][2][0][RTW89_CN][21] = 127,
+ [0][0][2][0][RTW89_UK][21] = 62,
+ [0][0][2][0][RTW89_FCC][23] = 82,
+ [0][0][2][0][RTW89_ETSI][23] = 62,
+ [0][0][2][0][RTW89_MKK][23] = 78,
+ [0][0][2][0][RTW89_IC][23] = 82,
+ [0][0][2][0][RTW89_KCC][23] = 78,
+ [0][0][2][0][RTW89_ACMA][23] = 62,
+ [0][0][2][0][RTW89_CN][23] = 127,
+ [0][0][2][0][RTW89_UK][23] = 62,
+ [0][0][2][0][RTW89_FCC][25] = 82,
+ [0][0][2][0][RTW89_ETSI][25] = 62,
+ [0][0][2][0][RTW89_MKK][25] = 78,
+ [0][0][2][0][RTW89_IC][25] = 127,
+ [0][0][2][0][RTW89_KCC][25] = 78,
+ [0][0][2][0][RTW89_ACMA][25] = 127,
+ [0][0][2][0][RTW89_CN][25] = 127,
+ [0][0][2][0][RTW89_UK][25] = 62,
+ [0][0][2][0][RTW89_FCC][27] = 82,
+ [0][0][2][0][RTW89_ETSI][27] = 62,
+ [0][0][2][0][RTW89_MKK][27] = 78,
+ [0][0][2][0][RTW89_IC][27] = 127,
+ [0][0][2][0][RTW89_KCC][27] = 78,
+ [0][0][2][0][RTW89_ACMA][27] = 127,
+ [0][0][2][0][RTW89_CN][27] = 127,
+ [0][0][2][0][RTW89_UK][27] = 62,
+ [0][0][2][0][RTW89_FCC][29] = 82,
+ [0][0][2][0][RTW89_ETSI][29] = 62,
+ [0][0][2][0][RTW89_MKK][29] = 78,
+ [0][0][2][0][RTW89_IC][29] = 127,
+ [0][0][2][0][RTW89_KCC][29] = 78,
+ [0][0][2][0][RTW89_ACMA][29] = 127,
+ [0][0][2][0][RTW89_CN][29] = 127,
+ [0][0][2][0][RTW89_UK][29] = 62,
+ [0][0][2][0][RTW89_FCC][31] = 82,
+ [0][0][2][0][RTW89_ETSI][31] = 62,
+ [0][0][2][0][RTW89_MKK][31] = 78,
+ [0][0][2][0][RTW89_IC][31] = 82,
+ [0][0][2][0][RTW89_KCC][31] = 74,
+ [0][0][2][0][RTW89_ACMA][31] = 62,
+ [0][0][2][0][RTW89_CN][31] = 127,
+ [0][0][2][0][RTW89_UK][31] = 62,
+ [0][0][2][0][RTW89_FCC][33] = 82,
+ [0][0][2][0][RTW89_ETSI][33] = 62,
+ [0][0][2][0][RTW89_MKK][33] = 78,
+ [0][0][2][0][RTW89_IC][33] = 82,
+ [0][0][2][0][RTW89_KCC][33] = 74,
+ [0][0][2][0][RTW89_ACMA][33] = 62,
+ [0][0][2][0][RTW89_CN][33] = 127,
+ [0][0][2][0][RTW89_UK][33] = 62,
+ [0][0][2][0][RTW89_FCC][35] = 72,
+ [0][0][2][0][RTW89_ETSI][35] = 62,
+ [0][0][2][0][RTW89_MKK][35] = 78,
+ [0][0][2][0][RTW89_IC][35] = 72,
+ [0][0][2][0][RTW89_KCC][35] = 74,
+ [0][0][2][0][RTW89_ACMA][35] = 62,
+ [0][0][2][0][RTW89_CN][35] = 127,
+ [0][0][2][0][RTW89_UK][35] = 62,
+ [0][0][2][0][RTW89_FCC][37] = 82,
+ [0][0][2][0][RTW89_ETSI][37] = 127,
+ [0][0][2][0][RTW89_MKK][37] = 78,
+ [0][0][2][0][RTW89_IC][37] = 82,
+ [0][0][2][0][RTW89_KCC][37] = 76,
+ [0][0][2][0][RTW89_ACMA][37] = 78,
+ [0][0][2][0][RTW89_CN][37] = 127,
+ [0][0][2][0][RTW89_UK][37] = 78,
+ [0][0][2][0][RTW89_FCC][38] = 82,
+ [0][0][2][0][RTW89_ETSI][38] = 30,
+ [0][0][2][0][RTW89_MKK][38] = 127,
+ [0][0][2][0][RTW89_IC][38] = 82,
+ [0][0][2][0][RTW89_KCC][38] = 66,
+ [0][0][2][0][RTW89_ACMA][38] = 78,
+ [0][0][2][0][RTW89_CN][38] = 78,
+ [0][0][2][0][RTW89_UK][38] = 60,
+ [0][0][2][0][RTW89_FCC][40] = 82,
+ [0][0][2][0][RTW89_ETSI][40] = 30,
+ [0][0][2][0][RTW89_MKK][40] = 127,
+ [0][0][2][0][RTW89_IC][40] = 82,
+ [0][0][2][0][RTW89_KCC][40] = 74,
+ [0][0][2][0][RTW89_ACMA][40] = 78,
+ [0][0][2][0][RTW89_CN][40] = 78,
+ [0][0][2][0][RTW89_UK][40] = 60,
+ [0][0][2][0][RTW89_FCC][42] = 82,
+ [0][0][2][0][RTW89_ETSI][42] = 30,
+ [0][0][2][0][RTW89_MKK][42] = 127,
+ [0][0][2][0][RTW89_IC][42] = 82,
+ [0][0][2][0][RTW89_KCC][42] = 74,
+ [0][0][2][0][RTW89_ACMA][42] = 78,
+ [0][0][2][0][RTW89_CN][42] = 78,
+ [0][0][2][0][RTW89_UK][42] = 60,
+ [0][0][2][0][RTW89_FCC][44] = 82,
+ [0][0][2][0][RTW89_ETSI][44] = 30,
+ [0][0][2][0][RTW89_MKK][44] = 127,
+ [0][0][2][0][RTW89_IC][44] = 82,
+ [0][0][2][0][RTW89_KCC][44] = 74,
+ [0][0][2][0][RTW89_ACMA][44] = 78,
+ [0][0][2][0][RTW89_CN][44] = 78,
+ [0][0][2][0][RTW89_UK][44] = 60,
+ [0][0][2][0][RTW89_FCC][46] = 82,
+ [0][0][2][0][RTW89_ETSI][46] = 30,
+ [0][0][2][0][RTW89_MKK][46] = 127,
+ [0][0][2][0][RTW89_IC][46] = 82,
+ [0][0][2][0][RTW89_KCC][46] = 74,
+ [0][0][2][0][RTW89_ACMA][46] = 78,
+ [0][0][2][0][RTW89_CN][46] = 78,
+ [0][0][2][0][RTW89_UK][46] = 60,
+ [0][0][2][0][RTW89_FCC][48] = 74,
+ [0][0][2][0][RTW89_ETSI][48] = 127,
+ [0][0][2][0][RTW89_MKK][48] = 127,
+ [0][0][2][0][RTW89_IC][48] = 127,
+ [0][0][2][0][RTW89_KCC][48] = 127,
+ [0][0][2][0][RTW89_ACMA][48] = 127,
+ [0][0][2][0][RTW89_CN][48] = 127,
+ [0][0][2][0][RTW89_UK][48] = 127,
+ [0][0][2][0][RTW89_FCC][50] = 76,
+ [0][0][2][0][RTW89_ETSI][50] = 127,
+ [0][0][2][0][RTW89_MKK][50] = 127,
+ [0][0][2][0][RTW89_IC][50] = 127,
+ [0][0][2][0][RTW89_KCC][50] = 127,
+ [0][0][2][0][RTW89_ACMA][50] = 127,
+ [0][0][2][0][RTW89_CN][50] = 127,
+ [0][0][2][0][RTW89_UK][50] = 127,
+ [0][0][2][0][RTW89_FCC][52] = 76,
+ [0][0][2][0][RTW89_ETSI][52] = 127,
+ [0][0][2][0][RTW89_MKK][52] = 127,
+ [0][0][2][0][RTW89_IC][52] = 127,
+ [0][0][2][0][RTW89_KCC][52] = 127,
+ [0][0][2][0][RTW89_ACMA][52] = 127,
+ [0][0][2][0][RTW89_CN][52] = 127,
+ [0][0][2][0][RTW89_UK][52] = 127,
+ [0][1][2][0][RTW89_FCC][0] = 127,
+ [0][1][2][0][RTW89_ETSI][0] = 127,
+ [0][1][2][0][RTW89_MKK][0] = 127,
+ [0][1][2][0][RTW89_IC][0] = 127,
+ [0][1][2][0][RTW89_KCC][0] = 127,
+ [0][1][2][0][RTW89_ACMA][0] = 127,
+ [0][1][2][0][RTW89_CN][0] = 127,
+ [0][1][2][0][RTW89_UK][0] = 127,
+ [0][1][2][0][RTW89_FCC][2] = 127,
+ [0][1][2][0][RTW89_ETSI][2] = 127,
+ [0][1][2][0][RTW89_MKK][2] = 127,
+ [0][1][2][0][RTW89_IC][2] = 127,
+ [0][1][2][0][RTW89_KCC][2] = 127,
+ [0][1][2][0][RTW89_ACMA][2] = 127,
+ [0][1][2][0][RTW89_CN][2] = 127,
+ [0][1][2][0][RTW89_UK][2] = 127,
+ [0][1][2][0][RTW89_FCC][4] = 127,
+ [0][1][2][0][RTW89_ETSI][4] = 127,
+ [0][1][2][0][RTW89_MKK][4] = 127,
+ [0][1][2][0][RTW89_IC][4] = 127,
+ [0][1][2][0][RTW89_KCC][4] = 127,
+ [0][1][2][0][RTW89_ACMA][4] = 127,
+ [0][1][2][0][RTW89_CN][4] = 127,
+ [0][1][2][0][RTW89_UK][4] = 127,
+ [0][1][2][0][RTW89_FCC][6] = 127,
+ [0][1][2][0][RTW89_ETSI][6] = 127,
+ [0][1][2][0][RTW89_MKK][6] = 127,
+ [0][1][2][0][RTW89_IC][6] = 127,
+ [0][1][2][0][RTW89_KCC][6] = 127,
+ [0][1][2][0][RTW89_ACMA][6] = 127,
+ [0][1][2][0][RTW89_CN][6] = 127,
+ [0][1][2][0][RTW89_UK][6] = 127,
+ [0][1][2][0][RTW89_FCC][8] = 127,
+ [0][1][2][0][RTW89_ETSI][8] = 127,
+ [0][1][2][0][RTW89_MKK][8] = 127,
+ [0][1][2][0][RTW89_IC][8] = 127,
+ [0][1][2][0][RTW89_KCC][8] = 127,
+ [0][1][2][0][RTW89_ACMA][8] = 127,
+ [0][1][2][0][RTW89_CN][8] = 127,
+ [0][1][2][0][RTW89_UK][8] = 127,
+ [0][1][2][0][RTW89_FCC][10] = 127,
+ [0][1][2][0][RTW89_ETSI][10] = 127,
+ [0][1][2][0][RTW89_MKK][10] = 127,
+ [0][1][2][0][RTW89_IC][10] = 127,
+ [0][1][2][0][RTW89_KCC][10] = 127,
+ [0][1][2][0][RTW89_ACMA][10] = 127,
+ [0][1][2][0][RTW89_CN][10] = 127,
+ [0][1][2][0][RTW89_UK][10] = 127,
+ [0][1][2][0][RTW89_FCC][12] = 127,
+ [0][1][2][0][RTW89_ETSI][12] = 127,
+ [0][1][2][0][RTW89_MKK][12] = 127,
+ [0][1][2][0][RTW89_IC][12] = 127,
+ [0][1][2][0][RTW89_KCC][12] = 127,
+ [0][1][2][0][RTW89_ACMA][12] = 127,
+ [0][1][2][0][RTW89_CN][12] = 127,
+ [0][1][2][0][RTW89_UK][12] = 127,
+ [0][1][2][0][RTW89_FCC][14] = 127,
+ [0][1][2][0][RTW89_ETSI][14] = 127,
+ [0][1][2][0][RTW89_MKK][14] = 127,
+ [0][1][2][0][RTW89_IC][14] = 127,
+ [0][1][2][0][RTW89_KCC][14] = 127,
+ [0][1][2][0][RTW89_ACMA][14] = 127,
+ [0][1][2][0][RTW89_CN][14] = 127,
+ [0][1][2][0][RTW89_UK][14] = 127,
+ [0][1][2][0][RTW89_FCC][15] = 127,
+ [0][1][2][0][RTW89_ETSI][15] = 127,
+ [0][1][2][0][RTW89_MKK][15] = 127,
+ [0][1][2][0][RTW89_IC][15] = 127,
+ [0][1][2][0][RTW89_KCC][15] = 127,
+ [0][1][2][0][RTW89_ACMA][15] = 127,
+ [0][1][2][0][RTW89_CN][15] = 127,
+ [0][1][2][0][RTW89_UK][15] = 127,
+ [0][1][2][0][RTW89_FCC][17] = 127,
+ [0][1][2][0][RTW89_ETSI][17] = 127,
+ [0][1][2][0][RTW89_MKK][17] = 127,
+ [0][1][2][0][RTW89_IC][17] = 127,
+ [0][1][2][0][RTW89_KCC][17] = 127,
+ [0][1][2][0][RTW89_ACMA][17] = 127,
+ [0][1][2][0][RTW89_CN][17] = 127,
+ [0][1][2][0][RTW89_UK][17] = 127,
+ [0][1][2][0][RTW89_FCC][19] = 127,
+ [0][1][2][0][RTW89_ETSI][19] = 127,
+ [0][1][2][0][RTW89_MKK][19] = 127,
+ [0][1][2][0][RTW89_IC][19] = 127,
+ [0][1][2][0][RTW89_KCC][19] = 127,
+ [0][1][2][0][RTW89_ACMA][19] = 127,
+ [0][1][2][0][RTW89_CN][19] = 127,
+ [0][1][2][0][RTW89_UK][19] = 127,
+ [0][1][2][0][RTW89_FCC][21] = 127,
+ [0][1][2][0][RTW89_ETSI][21] = 127,
+ [0][1][2][0][RTW89_MKK][21] = 127,
+ [0][1][2][0][RTW89_IC][21] = 127,
+ [0][1][2][0][RTW89_KCC][21] = 127,
+ [0][1][2][0][RTW89_ACMA][21] = 127,
+ [0][1][2][0][RTW89_CN][21] = 127,
+ [0][1][2][0][RTW89_UK][21] = 127,
+ [0][1][2][0][RTW89_FCC][23] = 127,
+ [0][1][2][0][RTW89_ETSI][23] = 127,
+ [0][1][2][0][RTW89_MKK][23] = 127,
+ [0][1][2][0][RTW89_IC][23] = 127,
+ [0][1][2][0][RTW89_KCC][23] = 127,
+ [0][1][2][0][RTW89_ACMA][23] = 127,
+ [0][1][2][0][RTW89_CN][23] = 127,
+ [0][1][2][0][RTW89_UK][23] = 127,
+ [0][1][2][0][RTW89_FCC][25] = 127,
+ [0][1][2][0][RTW89_ETSI][25] = 127,
+ [0][1][2][0][RTW89_MKK][25] = 127,
+ [0][1][2][0][RTW89_IC][25] = 127,
+ [0][1][2][0][RTW89_KCC][25] = 127,
+ [0][1][2][0][RTW89_ACMA][25] = 127,
+ [0][1][2][0][RTW89_CN][25] = 127,
+ [0][1][2][0][RTW89_UK][25] = 127,
+ [0][1][2][0][RTW89_FCC][27] = 127,
+ [0][1][2][0][RTW89_ETSI][27] = 127,
+ [0][1][2][0][RTW89_MKK][27] = 127,
+ [0][1][2][0][RTW89_IC][27] = 127,
+ [0][1][2][0][RTW89_KCC][27] = 127,
+ [0][1][2][0][RTW89_ACMA][27] = 127,
+ [0][1][2][0][RTW89_CN][27] = 127,
+ [0][1][2][0][RTW89_UK][27] = 127,
+ [0][1][2][0][RTW89_FCC][29] = 127,
+ [0][1][2][0][RTW89_ETSI][29] = 127,
+ [0][1][2][0][RTW89_MKK][29] = 127,
+ [0][1][2][0][RTW89_IC][29] = 127,
+ [0][1][2][0][RTW89_KCC][29] = 127,
+ [0][1][2][0][RTW89_ACMA][29] = 127,
+ [0][1][2][0][RTW89_CN][29] = 127,
+ [0][1][2][0][RTW89_UK][29] = 127,
+ [0][1][2][0][RTW89_FCC][31] = 127,
+ [0][1][2][0][RTW89_ETSI][31] = 127,
+ [0][1][2][0][RTW89_MKK][31] = 127,
+ [0][1][2][0][RTW89_IC][31] = 127,
+ [0][1][2][0][RTW89_KCC][31] = 127,
+ [0][1][2][0][RTW89_ACMA][31] = 127,
+ [0][1][2][0][RTW89_CN][31] = 127,
+ [0][1][2][0][RTW89_UK][31] = 127,
+ [0][1][2][0][RTW89_FCC][33] = 127,
+ [0][1][2][0][RTW89_ETSI][33] = 127,
+ [0][1][2][0][RTW89_MKK][33] = 127,
+ [0][1][2][0][RTW89_IC][33] = 127,
+ [0][1][2][0][RTW89_KCC][33] = 127,
+ [0][1][2][0][RTW89_ACMA][33] = 127,
+ [0][1][2][0][RTW89_CN][33] = 127,
+ [0][1][2][0][RTW89_UK][33] = 127,
+ [0][1][2][0][RTW89_FCC][35] = 127,
+ [0][1][2][0][RTW89_ETSI][35] = 127,
+ [0][1][2][0][RTW89_MKK][35] = 127,
+ [0][1][2][0][RTW89_IC][35] = 127,
+ [0][1][2][0][RTW89_KCC][35] = 127,
+ [0][1][2][0][RTW89_ACMA][35] = 127,
+ [0][1][2][0][RTW89_CN][35] = 127,
+ [0][1][2][0][RTW89_UK][35] = 127,
+ [0][1][2][0][RTW89_FCC][37] = 127,
+ [0][1][2][0][RTW89_ETSI][37] = 127,
+ [0][1][2][0][RTW89_MKK][37] = 127,
+ [0][1][2][0][RTW89_IC][37] = 127,
+ [0][1][2][0][RTW89_KCC][37] = 127,
+ [0][1][2][0][RTW89_ACMA][37] = 127,
+ [0][1][2][0][RTW89_CN][37] = 127,
+ [0][1][2][0][RTW89_UK][37] = 127,
+ [0][1][2][0][RTW89_FCC][38] = 127,
+ [0][1][2][0][RTW89_ETSI][38] = 127,
+ [0][1][2][0][RTW89_MKK][38] = 127,
+ [0][1][2][0][RTW89_IC][38] = 127,
+ [0][1][2][0][RTW89_KCC][38] = 127,
+ [0][1][2][0][RTW89_ACMA][38] = 127,
+ [0][1][2][0][RTW89_CN][38] = 127,
+ [0][1][2][0][RTW89_UK][38] = 127,
+ [0][1][2][0][RTW89_FCC][40] = 127,
+ [0][1][2][0][RTW89_ETSI][40] = 127,
+ [0][1][2][0][RTW89_MKK][40] = 127,
+ [0][1][2][0][RTW89_IC][40] = 127,
+ [0][1][2][0][RTW89_KCC][40] = 127,
+ [0][1][2][0][RTW89_ACMA][40] = 127,
+ [0][1][2][0][RTW89_CN][40] = 127,
+ [0][1][2][0][RTW89_UK][40] = 127,
+ [0][1][2][0][RTW89_FCC][42] = 127,
+ [0][1][2][0][RTW89_ETSI][42] = 127,
+ [0][1][2][0][RTW89_MKK][42] = 127,
+ [0][1][2][0][RTW89_IC][42] = 127,
+ [0][1][2][0][RTW89_KCC][42] = 127,
+ [0][1][2][0][RTW89_ACMA][42] = 127,
+ [0][1][2][0][RTW89_CN][42] = 127,
+ [0][1][2][0][RTW89_UK][42] = 127,
+ [0][1][2][0][RTW89_FCC][44] = 127,
+ [0][1][2][0][RTW89_ETSI][44] = 127,
+ [0][1][2][0][RTW89_MKK][44] = 127,
+ [0][1][2][0][RTW89_IC][44] = 127,
+ [0][1][2][0][RTW89_KCC][44] = 127,
+ [0][1][2][0][RTW89_ACMA][44] = 127,
+ [0][1][2][0][RTW89_CN][44] = 127,
+ [0][1][2][0][RTW89_UK][44] = 127,
+ [0][1][2][0][RTW89_FCC][46] = 127,
+ [0][1][2][0][RTW89_ETSI][46] = 127,
+ [0][1][2][0][RTW89_MKK][46] = 127,
+ [0][1][2][0][RTW89_IC][46] = 127,
+ [0][1][2][0][RTW89_KCC][46] = 127,
+ [0][1][2][0][RTW89_ACMA][46] = 127,
+ [0][1][2][0][RTW89_CN][46] = 127,
+ [0][1][2][0][RTW89_UK][46] = 127,
+ [0][1][2][0][RTW89_FCC][48] = 127,
+ [0][1][2][0][RTW89_ETSI][48] = 127,
+ [0][1][2][0][RTW89_MKK][48] = 127,
+ [0][1][2][0][RTW89_IC][48] = 127,
+ [0][1][2][0][RTW89_KCC][48] = 127,
+ [0][1][2][0][RTW89_ACMA][48] = 127,
+ [0][1][2][0][RTW89_CN][48] = 127,
+ [0][1][2][0][RTW89_UK][48] = 127,
+ [0][1][2][0][RTW89_FCC][50] = 127,
+ [0][1][2][0][RTW89_ETSI][50] = 127,
+ [0][1][2][0][RTW89_MKK][50] = 127,
+ [0][1][2][0][RTW89_IC][50] = 127,
+ [0][1][2][0][RTW89_KCC][50] = 127,
+ [0][1][2][0][RTW89_ACMA][50] = 127,
+ [0][1][2][0][RTW89_CN][50] = 127,
+ [0][1][2][0][RTW89_UK][50] = 127,
+ [0][1][2][0][RTW89_FCC][52] = 127,
+ [0][1][2][0][RTW89_ETSI][52] = 127,
+ [0][1][2][0][RTW89_MKK][52] = 127,
+ [0][1][2][0][RTW89_IC][52] = 127,
+ [0][1][2][0][RTW89_KCC][52] = 127,
+ [0][1][2][0][RTW89_ACMA][52] = 127,
+ [0][1][2][0][RTW89_CN][52] = 127,
+ [0][1][2][0][RTW89_UK][52] = 127,
+ [0][1][2][1][RTW89_FCC][0] = 127,
+ [0][1][2][1][RTW89_ETSI][0] = 127,
+ [0][1][2][1][RTW89_MKK][0] = 127,
+ [0][1][2][1][RTW89_IC][0] = 127,
+ [0][1][2][1][RTW89_KCC][0] = 127,
+ [0][1][2][1][RTW89_ACMA][0] = 127,
+ [0][1][2][1][RTW89_CN][0] = 127,
+ [0][1][2][1][RTW89_UK][0] = 127,
+ [0][1][2][1][RTW89_FCC][2] = 127,
+ [0][1][2][1][RTW89_ETSI][2] = 127,
+ [0][1][2][1][RTW89_MKK][2] = 127,
+ [0][1][2][1][RTW89_IC][2] = 127,
+ [0][1][2][1][RTW89_KCC][2] = 127,
+ [0][1][2][1][RTW89_ACMA][2] = 127,
+ [0][1][2][1][RTW89_CN][2] = 127,
+ [0][1][2][1][RTW89_UK][2] = 127,
+ [0][1][2][1][RTW89_FCC][4] = 127,
+ [0][1][2][1][RTW89_ETSI][4] = 127,
+ [0][1][2][1][RTW89_MKK][4] = 127,
+ [0][1][2][1][RTW89_IC][4] = 127,
+ [0][1][2][1][RTW89_KCC][4] = 127,
+ [0][1][2][1][RTW89_ACMA][4] = 127,
+ [0][1][2][1][RTW89_CN][4] = 127,
+ [0][1][2][1][RTW89_UK][4] = 127,
+ [0][1][2][1][RTW89_FCC][6] = 127,
+ [0][1][2][1][RTW89_ETSI][6] = 127,
+ [0][1][2][1][RTW89_MKK][6] = 127,
+ [0][1][2][1][RTW89_IC][6] = 127,
+ [0][1][2][1][RTW89_KCC][6] = 127,
+ [0][1][2][1][RTW89_ACMA][6] = 127,
+ [0][1][2][1][RTW89_CN][6] = 127,
+ [0][1][2][1][RTW89_UK][6] = 127,
+ [0][1][2][1][RTW89_FCC][8] = 127,
+ [0][1][2][1][RTW89_ETSI][8] = 127,
+ [0][1][2][1][RTW89_MKK][8] = 127,
+ [0][1][2][1][RTW89_IC][8] = 127,
+ [0][1][2][1][RTW89_KCC][8] = 127,
+ [0][1][2][1][RTW89_ACMA][8] = 127,
+ [0][1][2][1][RTW89_CN][8] = 127,
+ [0][1][2][1][RTW89_UK][8] = 127,
+ [0][1][2][1][RTW89_FCC][10] = 127,
+ [0][1][2][1][RTW89_ETSI][10] = 127,
+ [0][1][2][1][RTW89_MKK][10] = 127,
+ [0][1][2][1][RTW89_IC][10] = 127,
+ [0][1][2][1][RTW89_KCC][10] = 127,
+ [0][1][2][1][RTW89_ACMA][10] = 127,
+ [0][1][2][1][RTW89_CN][10] = 127,
+ [0][1][2][1][RTW89_UK][10] = 127,
+ [0][1][2][1][RTW89_FCC][12] = 127,
+ [0][1][2][1][RTW89_ETSI][12] = 127,
+ [0][1][2][1][RTW89_MKK][12] = 127,
+ [0][1][2][1][RTW89_IC][12] = 127,
+ [0][1][2][1][RTW89_KCC][12] = 127,
+ [0][1][2][1][RTW89_ACMA][12] = 127,
+ [0][1][2][1][RTW89_CN][12] = 127,
+ [0][1][2][1][RTW89_UK][12] = 127,
+ [0][1][2][1][RTW89_FCC][14] = 127,
+ [0][1][2][1][RTW89_ETSI][14] = 127,
+ [0][1][2][1][RTW89_MKK][14] = 127,
+ [0][1][2][1][RTW89_IC][14] = 127,
+ [0][1][2][1][RTW89_KCC][14] = 127,
+ [0][1][2][1][RTW89_ACMA][14] = 127,
+ [0][1][2][1][RTW89_CN][14] = 127,
+ [0][1][2][1][RTW89_UK][14] = 127,
+ [0][1][2][1][RTW89_FCC][15] = 127,
+ [0][1][2][1][RTW89_ETSI][15] = 127,
+ [0][1][2][1][RTW89_MKK][15] = 127,
+ [0][1][2][1][RTW89_IC][15] = 127,
+ [0][1][2][1][RTW89_KCC][15] = 127,
+ [0][1][2][1][RTW89_ACMA][15] = 127,
+ [0][1][2][1][RTW89_CN][15] = 127,
+ [0][1][2][1][RTW89_UK][15] = 127,
+ [0][1][2][1][RTW89_FCC][17] = 127,
+ [0][1][2][1][RTW89_ETSI][17] = 127,
+ [0][1][2][1][RTW89_MKK][17] = 127,
+ [0][1][2][1][RTW89_IC][17] = 127,
+ [0][1][2][1][RTW89_KCC][17] = 127,
+ [0][1][2][1][RTW89_ACMA][17] = 127,
+ [0][1][2][1][RTW89_CN][17] = 127,
+ [0][1][2][1][RTW89_UK][17] = 127,
+ [0][1][2][1][RTW89_FCC][19] = 127,
+ [0][1][2][1][RTW89_ETSI][19] = 127,
+ [0][1][2][1][RTW89_MKK][19] = 127,
+ [0][1][2][1][RTW89_IC][19] = 127,
+ [0][1][2][1][RTW89_KCC][19] = 127,
+ [0][1][2][1][RTW89_ACMA][19] = 127,
+ [0][1][2][1][RTW89_CN][19] = 127,
+ [0][1][2][1][RTW89_UK][19] = 127,
+ [0][1][2][1][RTW89_FCC][21] = 127,
+ [0][1][2][1][RTW89_ETSI][21] = 127,
+ [0][1][2][1][RTW89_MKK][21] = 127,
+ [0][1][2][1][RTW89_IC][21] = 127,
+ [0][1][2][1][RTW89_KCC][21] = 127,
+ [0][1][2][1][RTW89_ACMA][21] = 127,
+ [0][1][2][1][RTW89_CN][21] = 127,
+ [0][1][2][1][RTW89_UK][21] = 127,
+ [0][1][2][1][RTW89_FCC][23] = 127,
+ [0][1][2][1][RTW89_ETSI][23] = 127,
+ [0][1][2][1][RTW89_MKK][23] = 127,
+ [0][1][2][1][RTW89_IC][23] = 127,
+ [0][1][2][1][RTW89_KCC][23] = 127,
+ [0][1][2][1][RTW89_ACMA][23] = 127,
+ [0][1][2][1][RTW89_CN][23] = 127,
+ [0][1][2][1][RTW89_UK][23] = 127,
+ [0][1][2][1][RTW89_FCC][25] = 127,
+ [0][1][2][1][RTW89_ETSI][25] = 127,
+ [0][1][2][1][RTW89_MKK][25] = 127,
+ [0][1][2][1][RTW89_IC][25] = 127,
+ [0][1][2][1][RTW89_KCC][25] = 127,
+ [0][1][2][1][RTW89_ACMA][25] = 127,
+ [0][1][2][1][RTW89_CN][25] = 127,
+ [0][1][2][1][RTW89_UK][25] = 127,
+ [0][1][2][1][RTW89_FCC][27] = 127,
+ [0][1][2][1][RTW89_ETSI][27] = 127,
+ [0][1][2][1][RTW89_MKK][27] = 127,
+ [0][1][2][1][RTW89_IC][27] = 127,
+ [0][1][2][1][RTW89_KCC][27] = 127,
+ [0][1][2][1][RTW89_ACMA][27] = 127,
+ [0][1][2][1][RTW89_CN][27] = 127,
+ [0][1][2][1][RTW89_UK][27] = 127,
+ [0][1][2][1][RTW89_FCC][29] = 127,
+ [0][1][2][1][RTW89_ETSI][29] = 127,
+ [0][1][2][1][RTW89_MKK][29] = 127,
+ [0][1][2][1][RTW89_IC][29] = 127,
+ [0][1][2][1][RTW89_KCC][29] = 127,
+ [0][1][2][1][RTW89_ACMA][29] = 127,
+ [0][1][2][1][RTW89_CN][29] = 127,
+ [0][1][2][1][RTW89_UK][29] = 127,
+ [0][1][2][1][RTW89_FCC][31] = 127,
+ [0][1][2][1][RTW89_ETSI][31] = 127,
+ [0][1][2][1][RTW89_MKK][31] = 127,
+ [0][1][2][1][RTW89_IC][31] = 127,
+ [0][1][2][1][RTW89_KCC][31] = 127,
+ [0][1][2][1][RTW89_ACMA][31] = 127,
+ [0][1][2][1][RTW89_CN][31] = 127,
+ [0][1][2][1][RTW89_UK][31] = 127,
+ [0][1][2][1][RTW89_FCC][33] = 127,
+ [0][1][2][1][RTW89_ETSI][33] = 127,
+ [0][1][2][1][RTW89_MKK][33] = 127,
+ [0][1][2][1][RTW89_IC][33] = 127,
+ [0][1][2][1][RTW89_KCC][33] = 127,
+ [0][1][2][1][RTW89_ACMA][33] = 127,
+ [0][1][2][1][RTW89_CN][33] = 127,
+ [0][1][2][1][RTW89_UK][33] = 127,
+ [0][1][2][1][RTW89_FCC][35] = 127,
+ [0][1][2][1][RTW89_ETSI][35] = 127,
+ [0][1][2][1][RTW89_MKK][35] = 127,
+ [0][1][2][1][RTW89_IC][35] = 127,
+ [0][1][2][1][RTW89_KCC][35] = 127,
+ [0][1][2][1][RTW89_ACMA][35] = 127,
+ [0][1][2][1][RTW89_CN][35] = 127,
+ [0][1][2][1][RTW89_UK][35] = 127,
+ [0][1][2][1][RTW89_FCC][37] = 127,
+ [0][1][2][1][RTW89_ETSI][37] = 127,
+ [0][1][2][1][RTW89_MKK][37] = 127,
+ [0][1][2][1][RTW89_IC][37] = 127,
+ [0][1][2][1][RTW89_KCC][37] = 127,
+ [0][1][2][1][RTW89_ACMA][37] = 127,
+ [0][1][2][1][RTW89_CN][37] = 127,
+ [0][1][2][1][RTW89_UK][37] = 127,
+ [0][1][2][1][RTW89_FCC][38] = 127,
+ [0][1][2][1][RTW89_ETSI][38] = 127,
+ [0][1][2][1][RTW89_MKK][38] = 127,
+ [0][1][2][1][RTW89_IC][38] = 127,
+ [0][1][2][1][RTW89_KCC][38] = 127,
+ [0][1][2][1][RTW89_ACMA][38] = 127,
+ [0][1][2][1][RTW89_CN][38] = 127,
+ [0][1][2][1][RTW89_UK][38] = 127,
+ [0][1][2][1][RTW89_FCC][40] = 127,
+ [0][1][2][1][RTW89_ETSI][40] = 127,
+ [0][1][2][1][RTW89_MKK][40] = 127,
+ [0][1][2][1][RTW89_IC][40] = 127,
+ [0][1][2][1][RTW89_KCC][40] = 127,
+ [0][1][2][1][RTW89_ACMA][40] = 127,
+ [0][1][2][1][RTW89_CN][40] = 127,
+ [0][1][2][1][RTW89_UK][40] = 127,
+ [0][1][2][1][RTW89_FCC][42] = 127,
+ [0][1][2][1][RTW89_ETSI][42] = 127,
+ [0][1][2][1][RTW89_MKK][42] = 127,
+ [0][1][2][1][RTW89_IC][42] = 127,
+ [0][1][2][1][RTW89_KCC][42] = 127,
+ [0][1][2][1][RTW89_ACMA][42] = 127,
+ [0][1][2][1][RTW89_CN][42] = 127,
+ [0][1][2][1][RTW89_UK][42] = 127,
+ [0][1][2][1][RTW89_FCC][44] = 127,
+ [0][1][2][1][RTW89_ETSI][44] = 127,
+ [0][1][2][1][RTW89_MKK][44] = 127,
+ [0][1][2][1][RTW89_IC][44] = 127,
+ [0][1][2][1][RTW89_KCC][44] = 127,
+ [0][1][2][1][RTW89_ACMA][44] = 127,
+ [0][1][2][1][RTW89_CN][44] = 127,
+ [0][1][2][1][RTW89_UK][44] = 127,
+ [0][1][2][1][RTW89_FCC][46] = 127,
+ [0][1][2][1][RTW89_ETSI][46] = 127,
+ [0][1][2][1][RTW89_MKK][46] = 127,
+ [0][1][2][1][RTW89_IC][46] = 127,
+ [0][1][2][1][RTW89_KCC][46] = 127,
+ [0][1][2][1][RTW89_ACMA][46] = 127,
+ [0][1][2][1][RTW89_CN][46] = 127,
+ [0][1][2][1][RTW89_UK][46] = 127,
+ [0][1][2][1][RTW89_FCC][48] = 127,
+ [0][1][2][1][RTW89_ETSI][48] = 127,
+ [0][1][2][1][RTW89_MKK][48] = 127,
+ [0][1][2][1][RTW89_IC][48] = 127,
+ [0][1][2][1][RTW89_KCC][48] = 127,
+ [0][1][2][1][RTW89_ACMA][48] = 127,
+ [0][1][2][1][RTW89_CN][48] = 127,
+ [0][1][2][1][RTW89_UK][48] = 127,
+ [0][1][2][1][RTW89_FCC][50] = 127,
+ [0][1][2][1][RTW89_ETSI][50] = 127,
+ [0][1][2][1][RTW89_MKK][50] = 127,
+ [0][1][2][1][RTW89_IC][50] = 127,
+ [0][1][2][1][RTW89_KCC][50] = 127,
+ [0][1][2][1][RTW89_ACMA][50] = 127,
+ [0][1][2][1][RTW89_CN][50] = 127,
+ [0][1][2][1][RTW89_UK][50] = 127,
+ [0][1][2][1][RTW89_FCC][52] = 127,
+ [0][1][2][1][RTW89_ETSI][52] = 127,
+ [0][1][2][1][RTW89_MKK][52] = 127,
+ [0][1][2][1][RTW89_IC][52] = 127,
+ [0][1][2][1][RTW89_KCC][52] = 127,
+ [0][1][2][1][RTW89_ACMA][52] = 127,
+ [0][1][2][1][RTW89_CN][52] = 127,
+ [0][1][2][1][RTW89_UK][52] = 127,
+ [1][0][2][0][RTW89_FCC][1] = 68,
+ [1][0][2][0][RTW89_ETSI][1] = 64,
+ [1][0][2][0][RTW89_MKK][1] = 64,
+ [1][0][2][0][RTW89_IC][1] = 64,
+ [1][0][2][0][RTW89_KCC][1] = 74,
+ [1][0][2][0][RTW89_ACMA][1] = 64,
+ [1][0][2][0][RTW89_CN][1] = 64,
+ [1][0][2][0][RTW89_UK][1] = 64,
+ [1][0][2][0][RTW89_FCC][5] = 82,
+ [1][0][2][0][RTW89_ETSI][5] = 64,
+ [1][0][2][0][RTW89_MKK][5] = 62,
+ [1][0][2][0][RTW89_IC][5] = 64,
+ [1][0][2][0][RTW89_KCC][5] = 66,
+ [1][0][2][0][RTW89_ACMA][5] = 64,
+ [1][0][2][0][RTW89_CN][5] = 64,
+ [1][0][2][0][RTW89_UK][5] = 64,
+ [1][0][2][0][RTW89_FCC][9] = 82,
+ [1][0][2][0][RTW89_ETSI][9] = 64,
+ [1][0][2][0][RTW89_MKK][9] = 64,
+ [1][0][2][0][RTW89_IC][9] = 64,
+ [1][0][2][0][RTW89_KCC][9] = 78,
+ [1][0][2][0][RTW89_ACMA][9] = 64,
+ [1][0][2][0][RTW89_CN][9] = 64,
+ [1][0][2][0][RTW89_UK][9] = 64,
+ [1][0][2][0][RTW89_FCC][13] = 66,
+ [1][0][2][0][RTW89_ETSI][13] = 64,
+ [1][0][2][0][RTW89_MKK][13] = 64,
+ [1][0][2][0][RTW89_IC][13] = 64,
+ [1][0][2][0][RTW89_KCC][13] = 72,
+ [1][0][2][0][RTW89_ACMA][13] = 64,
+ [1][0][2][0][RTW89_CN][13] = 64,
+ [1][0][2][0][RTW89_UK][13] = 64,
+ [1][0][2][0][RTW89_FCC][16] = 66,
+ [1][0][2][0][RTW89_ETSI][16] = 66,
+ [1][0][2][0][RTW89_MKK][16] = 80,
+ [1][0][2][0][RTW89_IC][16] = 66,
+ [1][0][2][0][RTW89_KCC][16] = 74,
+ [1][0][2][0][RTW89_ACMA][16] = 66,
+ [1][0][2][0][RTW89_CN][16] = 127,
+ [1][0][2][0][RTW89_UK][16] = 66,
+ [1][0][2][0][RTW89_FCC][20] = 80,
+ [1][0][2][0][RTW89_ETSI][20] = 66,
+ [1][0][2][0][RTW89_MKK][20] = 80,
+ [1][0][2][0][RTW89_IC][20] = 80,
+ [1][0][2][0][RTW89_KCC][20] = 74,
+ [1][0][2][0][RTW89_ACMA][20] = 66,
+ [1][0][2][0][RTW89_CN][20] = 127,
+ [1][0][2][0][RTW89_UK][20] = 66,
+ [1][0][2][0][RTW89_FCC][24] = 80,
+ [1][0][2][0][RTW89_ETSI][24] = 66,
+ [1][0][2][0][RTW89_MKK][24] = 80,
+ [1][0][2][0][RTW89_IC][24] = 127,
+ [1][0][2][0][RTW89_KCC][24] = 74,
+ [1][0][2][0][RTW89_ACMA][24] = 127,
+ [1][0][2][0][RTW89_CN][24] = 127,
+ [1][0][2][0][RTW89_UK][24] = 66,
+ [1][0][2][0][RTW89_FCC][28] = 80,
+ [1][0][2][0][RTW89_ETSI][28] = 66,
+ [1][0][2][0][RTW89_MKK][28] = 80,
+ [1][0][2][0][RTW89_IC][28] = 127,
+ [1][0][2][0][RTW89_KCC][28] = 74,
+ [1][0][2][0][RTW89_ACMA][28] = 127,
+ [1][0][2][0][RTW89_CN][28] = 127,
+ [1][0][2][0][RTW89_UK][28] = 66,
+ [1][0][2][0][RTW89_FCC][32] = 76,
+ [1][0][2][0][RTW89_ETSI][32] = 66,
+ [1][0][2][0][RTW89_MKK][32] = 80,
+ [1][0][2][0][RTW89_IC][32] = 76,
+ [1][0][2][0][RTW89_KCC][32] = 78,
+ [1][0][2][0][RTW89_ACMA][32] = 66,
+ [1][0][2][0][RTW89_CN][32] = 127,
+ [1][0][2][0][RTW89_UK][32] = 66,
+ [1][0][2][0][RTW89_FCC][36] = 80,
+ [1][0][2][0][RTW89_ETSI][36] = 127,
+ [1][0][2][0][RTW89_MKK][36] = 80,
+ [1][0][2][0][RTW89_IC][36] = 80,
+ [1][0][2][0][RTW89_KCC][36] = 76,
+ [1][0][2][0][RTW89_ACMA][36] = 78,
+ [1][0][2][0][RTW89_CN][36] = 127,
+ [1][0][2][0][RTW89_UK][36] = 80,
+ [1][0][2][0][RTW89_FCC][39] = 84,
+ [1][0][2][0][RTW89_ETSI][39] = 30,
+ [1][0][2][0][RTW89_MKK][39] = 127,
+ [1][0][2][0][RTW89_IC][39] = 84,
+ [1][0][2][0][RTW89_KCC][39] = 68,
+ [1][0][2][0][RTW89_ACMA][39] = 80,
+ [1][0][2][0][RTW89_CN][39] = 70,
+ [1][0][2][0][RTW89_UK][39] = 64,
+ [1][0][2][0][RTW89_FCC][43] = 84,
+ [1][0][2][0][RTW89_ETSI][43] = 30,
+ [1][0][2][0][RTW89_MKK][43] = 127,
+ [1][0][2][0][RTW89_IC][43] = 84,
+ [1][0][2][0][RTW89_KCC][43] = 78,
+ [1][0][2][0][RTW89_ACMA][43] = 80,
+ [1][0][2][0][RTW89_CN][43] = 80,
+ [1][0][2][0][RTW89_UK][43] = 64,
+ [1][0][2][0][RTW89_FCC][47] = 84,
+ [1][0][2][0][RTW89_ETSI][47] = 127,
+ [1][0][2][0][RTW89_MKK][47] = 127,
+ [1][0][2][0][RTW89_IC][47] = 127,
+ [1][0][2][0][RTW89_KCC][47] = 127,
+ [1][0][2][0][RTW89_ACMA][47] = 127,
+ [1][0][2][0][RTW89_CN][47] = 127,
+ [1][0][2][0][RTW89_UK][47] = 127,
+ [1][0][2][0][RTW89_FCC][51] = 84,
+ [1][0][2][0][RTW89_ETSI][51] = 127,
+ [1][0][2][0][RTW89_MKK][51] = 127,
+ [1][0][2][0][RTW89_IC][51] = 127,
+ [1][0][2][0][RTW89_KCC][51] = 127,
+ [1][0][2][0][RTW89_ACMA][51] = 127,
+ [1][0][2][0][RTW89_CN][51] = 127,
+ [1][0][2][0][RTW89_UK][51] = 127,
+ [1][1][2][0][RTW89_FCC][1] = 127,
+ [1][1][2][0][RTW89_ETSI][1] = 127,
+ [1][1][2][0][RTW89_MKK][1] = 127,
+ [1][1][2][0][RTW89_IC][1] = 127,
+ [1][1][2][0][RTW89_KCC][1] = 127,
+ [1][1][2][0][RTW89_ACMA][1] = 127,
+ [1][1][2][0][RTW89_CN][1] = 127,
+ [1][1][2][0][RTW89_UK][1] = 127,
+ [1][1][2][0][RTW89_FCC][5] = 127,
+ [1][1][2][0][RTW89_ETSI][5] = 127,
+ [1][1][2][0][RTW89_MKK][5] = 127,
+ [1][1][2][0][RTW89_IC][5] = 127,
+ [1][1][2][0][RTW89_KCC][5] = 127,
+ [1][1][2][0][RTW89_ACMA][5] = 127,
+ [1][1][2][0][RTW89_CN][5] = 127,
+ [1][1][2][0][RTW89_UK][5] = 127,
+ [1][1][2][0][RTW89_FCC][9] = 127,
+ [1][1][2][0][RTW89_ETSI][9] = 127,
+ [1][1][2][0][RTW89_MKK][9] = 127,
+ [1][1][2][0][RTW89_IC][9] = 127,
+ [1][1][2][0][RTW89_KCC][9] = 127,
+ [1][1][2][0][RTW89_ACMA][9] = 127,
+ [1][1][2][0][RTW89_CN][9] = 127,
+ [1][1][2][0][RTW89_UK][9] = 127,
+ [1][1][2][0][RTW89_FCC][13] = 127,
+ [1][1][2][0][RTW89_ETSI][13] = 127,
+ [1][1][2][0][RTW89_MKK][13] = 127,
+ [1][1][2][0][RTW89_IC][13] = 127,
+ [1][1][2][0][RTW89_KCC][13] = 127,
+ [1][1][2][0][RTW89_ACMA][13] = 127,
+ [1][1][2][0][RTW89_CN][13] = 127,
+ [1][1][2][0][RTW89_UK][13] = 127,
+ [1][1][2][0][RTW89_FCC][16] = 127,
+ [1][1][2][0][RTW89_ETSI][16] = 127,
+ [1][1][2][0][RTW89_MKK][16] = 127,
+ [1][1][2][0][RTW89_IC][16] = 127,
+ [1][1][2][0][RTW89_KCC][16] = 127,
+ [1][1][2][0][RTW89_ACMA][16] = 127,
+ [1][1][2][0][RTW89_CN][16] = 127,
+ [1][1][2][0][RTW89_UK][16] = 127,
+ [1][1][2][0][RTW89_FCC][20] = 127,
+ [1][1][2][0][RTW89_ETSI][20] = 127,
+ [1][1][2][0][RTW89_MKK][20] = 127,
+ [1][1][2][0][RTW89_IC][20] = 127,
+ [1][1][2][0][RTW89_KCC][20] = 127,
+ [1][1][2][0][RTW89_ACMA][20] = 127,
+ [1][1][2][0][RTW89_CN][20] = 127,
+ [1][1][2][0][RTW89_UK][20] = 127,
+ [1][1][2][0][RTW89_FCC][24] = 127,
+ [1][1][2][0][RTW89_ETSI][24] = 127,
+ [1][1][2][0][RTW89_MKK][24] = 127,
+ [1][1][2][0][RTW89_IC][24] = 127,
+ [1][1][2][0][RTW89_KCC][24] = 127,
+ [1][1][2][0][RTW89_ACMA][24] = 127,
+ [1][1][2][0][RTW89_CN][24] = 127,
+ [1][1][2][0][RTW89_UK][24] = 127,
+ [1][1][2][0][RTW89_FCC][28] = 127,
+ [1][1][2][0][RTW89_ETSI][28] = 127,
+ [1][1][2][0][RTW89_MKK][28] = 127,
+ [1][1][2][0][RTW89_IC][28] = 127,
+ [1][1][2][0][RTW89_KCC][28] = 127,
+ [1][1][2][0][RTW89_ACMA][28] = 127,
+ [1][1][2][0][RTW89_CN][28] = 127,
+ [1][1][2][0][RTW89_UK][28] = 127,
+ [1][1][2][0][RTW89_FCC][32] = 127,
+ [1][1][2][0][RTW89_ETSI][32] = 127,
+ [1][1][2][0][RTW89_MKK][32] = 127,
+ [1][1][2][0][RTW89_IC][32] = 127,
+ [1][1][2][0][RTW89_KCC][32] = 127,
+ [1][1][2][0][RTW89_ACMA][32] = 127,
+ [1][1][2][0][RTW89_CN][32] = 127,
+ [1][1][2][0][RTW89_UK][32] = 127,
+ [1][1][2][0][RTW89_FCC][36] = 127,
+ [1][1][2][0][RTW89_ETSI][36] = 127,
+ [1][1][2][0][RTW89_MKK][36] = 127,
+ [1][1][2][0][RTW89_IC][36] = 127,
+ [1][1][2][0][RTW89_KCC][36] = 127,
+ [1][1][2][0][RTW89_ACMA][36] = 127,
+ [1][1][2][0][RTW89_CN][36] = 127,
+ [1][1][2][0][RTW89_UK][36] = 127,
+ [1][1][2][0][RTW89_FCC][39] = 127,
+ [1][1][2][0][RTW89_ETSI][39] = 127,
+ [1][1][2][0][RTW89_MKK][39] = 127,
+ [1][1][2][0][RTW89_IC][39] = 127,
+ [1][1][2][0][RTW89_KCC][39] = 127,
+ [1][1][2][0][RTW89_ACMA][39] = 127,
+ [1][1][2][0][RTW89_CN][39] = 127,
+ [1][1][2][0][RTW89_UK][39] = 127,
+ [1][1][2][0][RTW89_FCC][43] = 127,
+ [1][1][2][0][RTW89_ETSI][43] = 127,
+ [1][1][2][0][RTW89_MKK][43] = 127,
+ [1][1][2][0][RTW89_IC][43] = 127,
+ [1][1][2][0][RTW89_KCC][43] = 127,
+ [1][1][2][0][RTW89_ACMA][43] = 127,
+ [1][1][2][0][RTW89_CN][43] = 127,
+ [1][1][2][0][RTW89_UK][43] = 127,
+ [1][1][2][0][RTW89_FCC][47] = 127,
+ [1][1][2][0][RTW89_ETSI][47] = 127,
+ [1][1][2][0][RTW89_MKK][47] = 127,
+ [1][1][2][0][RTW89_IC][47] = 127,
+ [1][1][2][0][RTW89_KCC][47] = 127,
+ [1][1][2][0][RTW89_ACMA][47] = 127,
+ [1][1][2][0][RTW89_CN][47] = 127,
+ [1][1][2][0][RTW89_UK][47] = 127,
+ [1][1][2][0][RTW89_FCC][51] = 127,
+ [1][1][2][0][RTW89_ETSI][51] = 127,
+ [1][1][2][0][RTW89_MKK][51] = 127,
+ [1][1][2][0][RTW89_IC][51] = 127,
+ [1][1][2][0][RTW89_KCC][51] = 127,
+ [1][1][2][0][RTW89_ACMA][51] = 127,
+ [1][1][2][0][RTW89_CN][51] = 127,
+ [1][1][2][0][RTW89_UK][51] = 127,
+ [1][1][2][1][RTW89_FCC][1] = 127,
+ [1][1][2][1][RTW89_ETSI][1] = 127,
+ [1][1][2][1][RTW89_MKK][1] = 127,
+ [1][1][2][1][RTW89_IC][1] = 127,
+ [1][1][2][1][RTW89_KCC][1] = 127,
+ [1][1][2][1][RTW89_ACMA][1] = 127,
+ [1][1][2][1][RTW89_CN][1] = 127,
+ [1][1][2][1][RTW89_UK][1] = 127,
+ [1][1][2][1][RTW89_FCC][5] = 127,
+ [1][1][2][1][RTW89_ETSI][5] = 127,
+ [1][1][2][1][RTW89_MKK][5] = 127,
+ [1][1][2][1][RTW89_IC][5] = 127,
+ [1][1][2][1][RTW89_KCC][5] = 127,
+ [1][1][2][1][RTW89_ACMA][5] = 127,
+ [1][1][2][1][RTW89_CN][5] = 127,
+ [1][1][2][1][RTW89_UK][5] = 127,
+ [1][1][2][1][RTW89_FCC][9] = 127,
+ [1][1][2][1][RTW89_ETSI][9] = 127,
+ [1][1][2][1][RTW89_MKK][9] = 127,
+ [1][1][2][1][RTW89_IC][9] = 127,
+ [1][1][2][1][RTW89_KCC][9] = 127,
+ [1][1][2][1][RTW89_ACMA][9] = 127,
+ [1][1][2][1][RTW89_CN][9] = 127,
+ [1][1][2][1][RTW89_UK][9] = 127,
+ [1][1][2][1][RTW89_FCC][13] = 127,
+ [1][1][2][1][RTW89_ETSI][13] = 127,
+ [1][1][2][1][RTW89_MKK][13] = 127,
+ [1][1][2][1][RTW89_IC][13] = 127,
+ [1][1][2][1][RTW89_KCC][13] = 127,
+ [1][1][2][1][RTW89_ACMA][13] = 127,
+ [1][1][2][1][RTW89_CN][13] = 127,
+ [1][1][2][1][RTW89_UK][13] = 127,
+ [1][1][2][1][RTW89_FCC][16] = 127,
+ [1][1][2][1][RTW89_ETSI][16] = 127,
+ [1][1][2][1][RTW89_MKK][16] = 127,
+ [1][1][2][1][RTW89_IC][16] = 127,
+ [1][1][2][1][RTW89_KCC][16] = 127,
+ [1][1][2][1][RTW89_ACMA][16] = 127,
+ [1][1][2][1][RTW89_CN][16] = 127,
+ [1][1][2][1][RTW89_UK][16] = 127,
+ [1][1][2][1][RTW89_FCC][20] = 127,
+ [1][1][2][1][RTW89_ETSI][20] = 127,
+ [1][1][2][1][RTW89_MKK][20] = 127,
+ [1][1][2][1][RTW89_IC][20] = 127,
+ [1][1][2][1][RTW89_KCC][20] = 127,
+ [1][1][2][1][RTW89_ACMA][20] = 127,
+ [1][1][2][1][RTW89_CN][20] = 127,
+ [1][1][2][1][RTW89_UK][20] = 127,
+ [1][1][2][1][RTW89_FCC][24] = 127,
+ [1][1][2][1][RTW89_ETSI][24] = 127,
+ [1][1][2][1][RTW89_MKK][24] = 127,
+ [1][1][2][1][RTW89_IC][24] = 127,
+ [1][1][2][1][RTW89_KCC][24] = 127,
+ [1][1][2][1][RTW89_ACMA][24] = 127,
+ [1][1][2][1][RTW89_CN][24] = 127,
+ [1][1][2][1][RTW89_UK][24] = 127,
+ [1][1][2][1][RTW89_FCC][28] = 127,
+ [1][1][2][1][RTW89_ETSI][28] = 127,
+ [1][1][2][1][RTW89_MKK][28] = 127,
+ [1][1][2][1][RTW89_IC][28] = 127,
+ [1][1][2][1][RTW89_KCC][28] = 127,
+ [1][1][2][1][RTW89_ACMA][28] = 127,
+ [1][1][2][1][RTW89_CN][28] = 127,
+ [1][1][2][1][RTW89_UK][28] = 127,
+ [1][1][2][1][RTW89_FCC][32] = 127,
+ [1][1][2][1][RTW89_ETSI][32] = 127,
+ [1][1][2][1][RTW89_MKK][32] = 127,
+ [1][1][2][1][RTW89_IC][32] = 127,
+ [1][1][2][1][RTW89_KCC][32] = 127,
+ [1][1][2][1][RTW89_ACMA][32] = 127,
+ [1][1][2][1][RTW89_CN][32] = 127,
+ [1][1][2][1][RTW89_UK][32] = 127,
+ [1][1][2][1][RTW89_FCC][36] = 127,
+ [1][1][2][1][RTW89_ETSI][36] = 127,
+ [1][1][2][1][RTW89_MKK][36] = 127,
+ [1][1][2][1][RTW89_IC][36] = 127,
+ [1][1][2][1][RTW89_KCC][36] = 127,
+ [1][1][2][1][RTW89_ACMA][36] = 127,
+ [1][1][2][1][RTW89_CN][36] = 127,
+ [1][1][2][1][RTW89_UK][36] = 127,
+ [1][1][2][1][RTW89_FCC][39] = 127,
+ [1][1][2][1][RTW89_ETSI][39] = 127,
+ [1][1][2][1][RTW89_MKK][39] = 127,
+ [1][1][2][1][RTW89_IC][39] = 127,
+ [1][1][2][1][RTW89_KCC][39] = 127,
+ [1][1][2][1][RTW89_ACMA][39] = 127,
+ [1][1][2][1][RTW89_CN][39] = 127,
+ [1][1][2][1][RTW89_UK][39] = 127,
+ [1][1][2][1][RTW89_FCC][43] = 127,
+ [1][1][2][1][RTW89_ETSI][43] = 127,
+ [1][1][2][1][RTW89_MKK][43] = 127,
+ [1][1][2][1][RTW89_IC][43] = 127,
+ [1][1][2][1][RTW89_KCC][43] = 127,
+ [1][1][2][1][RTW89_ACMA][43] = 127,
+ [1][1][2][1][RTW89_CN][43] = 127,
+ [1][1][2][1][RTW89_UK][43] = 127,
+ [1][1][2][1][RTW89_FCC][47] = 127,
+ [1][1][2][1][RTW89_ETSI][47] = 127,
+ [1][1][2][1][RTW89_MKK][47] = 127,
+ [1][1][2][1][RTW89_IC][47] = 127,
+ [1][1][2][1][RTW89_KCC][47] = 127,
+ [1][1][2][1][RTW89_ACMA][47] = 127,
+ [1][1][2][1][RTW89_CN][47] = 127,
+ [1][1][2][1][RTW89_UK][47] = 127,
+ [1][1][2][1][RTW89_FCC][51] = 127,
+ [1][1][2][1][RTW89_ETSI][51] = 127,
+ [1][1][2][1][RTW89_MKK][51] = 127,
+ [1][1][2][1][RTW89_IC][51] = 127,
+ [1][1][2][1][RTW89_KCC][51] = 127,
+ [1][1][2][1][RTW89_ACMA][51] = 127,
+ [1][1][2][1][RTW89_CN][51] = 127,
+ [1][1][2][1][RTW89_UK][51] = 127,
+ [2][0][2][0][RTW89_FCC][3] = 76,
+ [2][0][2][0][RTW89_ETSI][3] = 64,
+ [2][0][2][0][RTW89_MKK][3] = 62,
+ [2][0][2][0][RTW89_IC][3] = 64,
+ [2][0][2][0][RTW89_KCC][3] = 72,
+ [2][0][2][0][RTW89_ACMA][3] = 64,
+ [2][0][2][0][RTW89_CN][3] = 64,
+ [2][0][2][0][RTW89_UK][3] = 64,
+ [2][0][2][0][RTW89_FCC][11] = 64,
+ [2][0][2][0][RTW89_ETSI][11] = 64,
+ [2][0][2][0][RTW89_MKK][11] = 64,
+ [2][0][2][0][RTW89_IC][11] = 62,
+ [2][0][2][0][RTW89_KCC][11] = 72,
+ [2][0][2][0][RTW89_ACMA][11] = 64,
+ [2][0][2][0][RTW89_CN][11] = 64,
+ [2][0][2][0][RTW89_UK][11] = 64,
+ [2][0][2][0][RTW89_FCC][18] = 66,
+ [2][0][2][0][RTW89_ETSI][18] = 64,
+ [2][0][2][0][RTW89_MKK][18] = 72,
+ [2][0][2][0][RTW89_IC][18] = 66,
+ [2][0][2][0][RTW89_KCC][18] = 72,
+ [2][0][2][0][RTW89_ACMA][18] = 64,
+ [2][0][2][0][RTW89_CN][18] = 127,
+ [2][0][2][0][RTW89_UK][18] = 64,
+ [2][0][2][0][RTW89_FCC][26] = 76,
+ [2][0][2][0][RTW89_ETSI][26] = 64,
+ [2][0][2][0][RTW89_MKK][26] = 72,
+ [2][0][2][0][RTW89_IC][26] = 127,
+ [2][0][2][0][RTW89_KCC][26] = 72,
+ [2][0][2][0][RTW89_ACMA][26] = 127,
+ [2][0][2][0][RTW89_CN][26] = 127,
+ [2][0][2][0][RTW89_UK][26] = 64,
+ [2][0][2][0][RTW89_FCC][34] = 76,
+ [2][0][2][0][RTW89_ETSI][34] = 127,
+ [2][0][2][0][RTW89_MKK][34] = 72,
+ [2][0][2][0][RTW89_IC][34] = 76,
+ [2][0][2][0][RTW89_KCC][34] = 72,
+ [2][0][2][0][RTW89_ACMA][34] = 72,
+ [2][0][2][0][RTW89_CN][34] = 127,
+ [2][0][2][0][RTW89_UK][34] = 72,
+ [2][0][2][0][RTW89_FCC][41] = 76,
+ [2][0][2][0][RTW89_ETSI][41] = 30,
+ [2][0][2][0][RTW89_MKK][41] = 127,
+ [2][0][2][0][RTW89_IC][41] = 76,
+ [2][0][2][0][RTW89_KCC][41] = 64,
+ [2][0][2][0][RTW89_ACMA][41] = 72,
+ [2][0][2][0][RTW89_CN][41] = 72,
+ [2][0][2][0][RTW89_UK][41] = 64,
+ [2][0][2][0][RTW89_FCC][49] = 74,
+ [2][0][2][0][RTW89_ETSI][49] = 127,
+ [2][0][2][0][RTW89_MKK][49] = 127,
+ [2][0][2][0][RTW89_IC][49] = 127,
+ [2][0][2][0][RTW89_KCC][49] = 127,
+ [2][0][2][0][RTW89_ACMA][49] = 127,
+ [2][0][2][0][RTW89_CN][49] = 127,
+ [2][0][2][0][RTW89_UK][49] = 127,
+ [2][1][2][0][RTW89_FCC][3] = 127,
+ [2][1][2][0][RTW89_ETSI][3] = 127,
+ [2][1][2][0][RTW89_MKK][3] = 127,
+ [2][1][2][0][RTW89_IC][3] = 127,
+ [2][1][2][0][RTW89_KCC][3] = 127,
+ [2][1][2][0][RTW89_ACMA][3] = 127,
+ [2][1][2][0][RTW89_CN][3] = 127,
+ [2][1][2][0][RTW89_UK][3] = 127,
+ [2][1][2][0][RTW89_FCC][11] = 127,
+ [2][1][2][0][RTW89_ETSI][11] = 127,
+ [2][1][2][0][RTW89_MKK][11] = 127,
+ [2][1][2][0][RTW89_IC][11] = 127,
+ [2][1][2][0][RTW89_KCC][11] = 127,
+ [2][1][2][0][RTW89_ACMA][11] = 127,
+ [2][1][2][0][RTW89_CN][11] = 127,
+ [2][1][2][0][RTW89_UK][11] = 127,
+ [2][1][2][0][RTW89_FCC][18] = 127,
+ [2][1][2][0][RTW89_ETSI][18] = 127,
+ [2][1][2][0][RTW89_MKK][18] = 127,
+ [2][1][2][0][RTW89_IC][18] = 127,
+ [2][1][2][0][RTW89_KCC][18] = 127,
+ [2][1][2][0][RTW89_ACMA][18] = 127,
+ [2][1][2][0][RTW89_CN][18] = 127,
+ [2][1][2][0][RTW89_UK][18] = 127,
+ [2][1][2][0][RTW89_FCC][26] = 127,
+ [2][1][2][0][RTW89_ETSI][26] = 127,
+ [2][1][2][0][RTW89_MKK][26] = 127,
+ [2][1][2][0][RTW89_IC][26] = 127,
+ [2][1][2][0][RTW89_KCC][26] = 127,
+ [2][1][2][0][RTW89_ACMA][26] = 127,
+ [2][1][2][0][RTW89_CN][26] = 127,
+ [2][1][2][0][RTW89_UK][26] = 127,
+ [2][1][2][0][RTW89_FCC][34] = 127,
+ [2][1][2][0][RTW89_ETSI][34] = 127,
+ [2][1][2][0][RTW89_MKK][34] = 127,
+ [2][1][2][0][RTW89_IC][34] = 127,
+ [2][1][2][0][RTW89_KCC][34] = 127,
+ [2][1][2][0][RTW89_ACMA][34] = 127,
+ [2][1][2][0][RTW89_CN][34] = 127,
+ [2][1][2][0][RTW89_UK][34] = 127,
+ [2][1][2][0][RTW89_FCC][41] = 127,
+ [2][1][2][0][RTW89_ETSI][41] = 127,
+ [2][1][2][0][RTW89_MKK][41] = 127,
+ [2][1][2][0][RTW89_IC][41] = 127,
+ [2][1][2][0][RTW89_KCC][41] = 127,
+ [2][1][2][0][RTW89_ACMA][41] = 127,
+ [2][1][2][0][RTW89_CN][41] = 127,
+ [2][1][2][0][RTW89_UK][41] = 127,
+ [2][1][2][0][RTW89_FCC][49] = 127,
+ [2][1][2][0][RTW89_ETSI][49] = 127,
+ [2][1][2][0][RTW89_MKK][49] = 127,
+ [2][1][2][0][RTW89_IC][49] = 127,
+ [2][1][2][0][RTW89_KCC][49] = 127,
+ [2][1][2][0][RTW89_ACMA][49] = 127,
+ [2][1][2][0][RTW89_CN][49] = 127,
+ [2][1][2][0][RTW89_UK][49] = 127,
+ [2][1][2][1][RTW89_FCC][3] = 127,
+ [2][1][2][1][RTW89_ETSI][3] = 127,
+ [2][1][2][1][RTW89_MKK][3] = 127,
+ [2][1][2][1][RTW89_IC][3] = 127,
+ [2][1][2][1][RTW89_KCC][3] = 127,
+ [2][1][2][1][RTW89_ACMA][3] = 127,
+ [2][1][2][1][RTW89_CN][3] = 127,
+ [2][1][2][1][RTW89_UK][3] = 127,
+ [2][1][2][1][RTW89_FCC][11] = 127,
+ [2][1][2][1][RTW89_ETSI][11] = 127,
+ [2][1][2][1][RTW89_MKK][11] = 127,
+ [2][1][2][1][RTW89_IC][11] = 127,
+ [2][1][2][1][RTW89_KCC][11] = 127,
+ [2][1][2][1][RTW89_ACMA][11] = 127,
+ [2][1][2][1][RTW89_CN][11] = 127,
+ [2][1][2][1][RTW89_UK][11] = 127,
+ [2][1][2][1][RTW89_FCC][18] = 127,
+ [2][1][2][1][RTW89_ETSI][18] = 127,
+ [2][1][2][1][RTW89_MKK][18] = 127,
+ [2][1][2][1][RTW89_IC][18] = 127,
+ [2][1][2][1][RTW89_KCC][18] = 127,
+ [2][1][2][1][RTW89_ACMA][18] = 127,
+ [2][1][2][1][RTW89_CN][18] = 127,
+ [2][1][2][1][RTW89_UK][18] = 127,
+ [2][1][2][1][RTW89_FCC][26] = 127,
+ [2][1][2][1][RTW89_ETSI][26] = 127,
+ [2][1][2][1][RTW89_MKK][26] = 127,
+ [2][1][2][1][RTW89_IC][26] = 127,
+ [2][1][2][1][RTW89_KCC][26] = 127,
+ [2][1][2][1][RTW89_ACMA][26] = 127,
+ [2][1][2][1][RTW89_CN][26] = 127,
+ [2][1][2][1][RTW89_UK][26] = 127,
+ [2][1][2][1][RTW89_FCC][34] = 127,
+ [2][1][2][1][RTW89_ETSI][34] = 127,
+ [2][1][2][1][RTW89_MKK][34] = 127,
+ [2][1][2][1][RTW89_IC][34] = 127,
+ [2][1][2][1][RTW89_KCC][34] = 127,
+ [2][1][2][1][RTW89_ACMA][34] = 127,
+ [2][1][2][1][RTW89_CN][34] = 127,
+ [2][1][2][1][RTW89_UK][34] = 127,
+ [2][1][2][1][RTW89_FCC][41] = 127,
+ [2][1][2][1][RTW89_ETSI][41] = 127,
+ [2][1][2][1][RTW89_MKK][41] = 127,
+ [2][1][2][1][RTW89_IC][41] = 127,
+ [2][1][2][1][RTW89_KCC][41] = 127,
+ [2][1][2][1][RTW89_ACMA][41] = 127,
+ [2][1][2][1][RTW89_CN][41] = 127,
+ [2][1][2][1][RTW89_UK][41] = 127,
+ [2][1][2][1][RTW89_FCC][49] = 127,
+ [2][1][2][1][RTW89_ETSI][49] = 127,
+ [2][1][2][1][RTW89_MKK][49] = 127,
+ [2][1][2][1][RTW89_IC][49] = 127,
+ [2][1][2][1][RTW89_KCC][49] = 127,
+ [2][1][2][1][RTW89_ACMA][49] = 127,
+ [2][1][2][1][RTW89_CN][49] = 127,
+ [2][1][2][1][RTW89_UK][49] = 127,
+ [3][0][2][0][RTW89_FCC][7] = 127,
+ [3][0][2][0][RTW89_ETSI][7] = 127,
+ [3][0][2][0][RTW89_MKK][7] = 127,
+ [3][0][2][0][RTW89_IC][7] = 127,
+ [3][0][2][0][RTW89_KCC][7] = 127,
+ [3][0][2][0][RTW89_ACMA][7] = 127,
+ [3][0][2][0][RTW89_CN][7] = 58,
+ [3][0][2][0][RTW89_UK][7] = 127,
+ [3][0][2][0][RTW89_FCC][22] = 127,
+ [3][0][2][0][RTW89_ETSI][22] = 127,
+ [3][0][2][0][RTW89_MKK][22] = 127,
+ [3][0][2][0][RTW89_IC][22] = 127,
+ [3][0][2][0][RTW89_KCC][22] = 127,
+ [3][0][2][0][RTW89_ACMA][22] = 127,
+ [3][0][2][0][RTW89_CN][22] = 58,
+ [3][0][2][0][RTW89_UK][22] = 127,
+ [3][0][2][0][RTW89_FCC][45] = 127,
+ [3][0][2][0][RTW89_ETSI][45] = 127,
+ [3][0][2][0][RTW89_MKK][45] = 127,
+ [3][0][2][0][RTW89_IC][45] = 127,
+ [3][0][2][0][RTW89_KCC][45] = 127,
+ [3][0][2][0][RTW89_ACMA][45] = 127,
+ [3][0][2][0][RTW89_CN][45] = 127,
+ [3][0][2][0][RTW89_UK][45] = 127,
+ [3][1][2][0][RTW89_FCC][7] = 127,
+ [3][1][2][0][RTW89_ETSI][7] = 127,
+ [3][1][2][0][RTW89_MKK][7] = 127,
+ [3][1][2][0][RTW89_IC][7] = 127,
+ [3][1][2][0][RTW89_KCC][7] = 127,
+ [3][1][2][0][RTW89_ACMA][7] = 127,
+ [3][1][2][0][RTW89_CN][7] = 127,
+ [3][1][2][0][RTW89_UK][7] = 127,
+ [3][1][2][0][RTW89_FCC][22] = 127,
+ [3][1][2][0][RTW89_ETSI][22] = 127,
+ [3][1][2][0][RTW89_MKK][22] = 127,
+ [3][1][2][0][RTW89_IC][22] = 127,
+ [3][1][2][0][RTW89_KCC][22] = 127,
+ [3][1][2][0][RTW89_ACMA][22] = 127,
+ [3][1][2][0][RTW89_CN][22] = 127,
+ [3][1][2][0][RTW89_UK][22] = 127,
+ [3][1][2][0][RTW89_FCC][45] = 127,
+ [3][1][2][0][RTW89_ETSI][45] = 127,
+ [3][1][2][0][RTW89_MKK][45] = 127,
+ [3][1][2][0][RTW89_IC][45] = 127,
+ [3][1][2][0][RTW89_KCC][45] = 127,
+ [3][1][2][0][RTW89_ACMA][45] = 127,
+ [3][1][2][0][RTW89_CN][45] = 127,
+ [3][1][2][0][RTW89_UK][45] = 127,
+ [3][1][2][1][RTW89_FCC][7] = 127,
+ [3][1][2][1][RTW89_ETSI][7] = 127,
+ [3][1][2][1][RTW89_MKK][7] = 127,
+ [3][1][2][1][RTW89_IC][7] = 127,
+ [3][1][2][1][RTW89_KCC][7] = 127,
+ [3][1][2][1][RTW89_ACMA][7] = 127,
+ [3][1][2][1][RTW89_CN][7] = 127,
+ [3][1][2][1][RTW89_UK][7] = 127,
+ [3][1][2][1][RTW89_FCC][22] = 127,
+ [3][1][2][1][RTW89_ETSI][22] = 127,
+ [3][1][2][1][RTW89_MKK][22] = 127,
+ [3][1][2][1][RTW89_IC][22] = 127,
+ [3][1][2][1][RTW89_KCC][22] = 127,
+ [3][1][2][1][RTW89_ACMA][22] = 127,
+ [3][1][2][1][RTW89_CN][22] = 127,
+ [3][1][2][1][RTW89_UK][22] = 127,
+ [3][1][2][1][RTW89_FCC][45] = 127,
+ [3][1][2][1][RTW89_ETSI][45] = 127,
+ [3][1][2][1][RTW89_MKK][45] = 127,
+ [3][1][2][1][RTW89_IC][45] = 127,
+ [3][1][2][1][RTW89_KCC][45] = 127,
+ [3][1][2][1][RTW89_ACMA][45] = 127,
+ [3][1][2][1][RTW89_CN][45] = 127,
+ [3][1][2][1][RTW89_UK][45] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
+ [0][0][RTW89_WW][0] = 30,
+ [0][0][RTW89_WW][1] = 30,
+ [0][0][RTW89_WW][2] = 30,
+ [0][0][RTW89_WW][3] = 30,
+ [0][0][RTW89_WW][4] = 30,
+ [0][0][RTW89_WW][5] = 30,
+ [0][0][RTW89_WW][6] = 30,
+ [0][0][RTW89_WW][7] = 30,
+ [0][0][RTW89_WW][8] = 30,
+ [0][0][RTW89_WW][9] = 30,
+ [0][0][RTW89_WW][10] = 30,
+ [0][0][RTW89_WW][11] = 30,
+ [0][0][RTW89_WW][12] = 30,
+ [0][0][RTW89_WW][13] = 0,
+ [0][1][RTW89_WW][0] = 20,
+ [0][1][RTW89_WW][1] = 22,
+ [0][1][RTW89_WW][2] = 22,
+ [0][1][RTW89_WW][3] = 22,
+ [0][1][RTW89_WW][4] = 22,
+ [0][1][RTW89_WW][5] = 22,
+ [0][1][RTW89_WW][6] = 22,
+ [0][1][RTW89_WW][7] = 22,
+ [0][1][RTW89_WW][8] = 22,
+ [0][1][RTW89_WW][9] = 22,
+ [0][1][RTW89_WW][10] = 22,
+ [0][1][RTW89_WW][11] = 22,
+ [0][1][RTW89_WW][12] = 20,
+ [0][1][RTW89_WW][13] = 0,
+ [1][0][RTW89_WW][0] = 42,
+ [1][0][RTW89_WW][1] = 42,
+ [1][0][RTW89_WW][2] = 42,
+ [1][0][RTW89_WW][3] = 42,
+ [1][0][RTW89_WW][4] = 42,
+ [1][0][RTW89_WW][5] = 42,
+ [1][0][RTW89_WW][6] = 42,
+ [1][0][RTW89_WW][7] = 42,
+ [1][0][RTW89_WW][8] = 42,
+ [1][0][RTW89_WW][9] = 42,
+ [1][0][RTW89_WW][10] = 42,
+ [1][0][RTW89_WW][11] = 42,
+ [1][0][RTW89_WW][12] = 34,
+ [1][0][RTW89_WW][13] = 0,
+ [1][1][RTW89_WW][0] = 32,
+ [1][1][RTW89_WW][1] = 32,
+ [1][1][RTW89_WW][2] = 32,
+ [1][1][RTW89_WW][3] = 32,
+ [1][1][RTW89_WW][4] = 32,
+ [1][1][RTW89_WW][5] = 32,
+ [1][1][RTW89_WW][6] = 32,
+ [1][1][RTW89_WW][7] = 32,
+ [1][1][RTW89_WW][8] = 32,
+ [1][1][RTW89_WW][9] = 32,
+ [1][1][RTW89_WW][10] = 32,
+ [1][1][RTW89_WW][11] = 32,
+ [1][1][RTW89_WW][12] = 32,
+ [1][1][RTW89_WW][13] = 0,
+ [2][0][RTW89_WW][0] = 54,
+ [2][0][RTW89_WW][1] = 54,
+ [2][0][RTW89_WW][2] = 54,
+ [2][0][RTW89_WW][3] = 54,
+ [2][0][RTW89_WW][4] = 54,
+ [2][0][RTW89_WW][5] = 54,
+ [2][0][RTW89_WW][6] = 54,
+ [2][0][RTW89_WW][7] = 54,
+ [2][0][RTW89_WW][8] = 54,
+ [2][0][RTW89_WW][9] = 54,
+ [2][0][RTW89_WW][10] = 54,
+ [2][0][RTW89_WW][11] = 54,
+ [2][0][RTW89_WW][12] = 34,
+ [2][0][RTW89_WW][13] = 0,
+ [2][1][RTW89_WW][0] = 44,
+ [2][1][RTW89_WW][1] = 44,
+ [2][1][RTW89_WW][2] = 44,
+ [2][1][RTW89_WW][3] = 44,
+ [2][1][RTW89_WW][4] = 44,
+ [2][1][RTW89_WW][5] = 44,
+ [2][1][RTW89_WW][6] = 44,
+ [2][1][RTW89_WW][7] = 44,
+ [2][1][RTW89_WW][8] = 44,
+ [2][1][RTW89_WW][9] = 44,
+ [2][1][RTW89_WW][10] = 44,
+ [2][1][RTW89_WW][11] = 44,
+ [2][1][RTW89_WW][12] = 42,
+ [2][1][RTW89_WW][13] = 0,
+ [0][0][RTW89_FCC][0] = 62,
+ [0][0][RTW89_ETSI][0] = 30,
+ [0][0][RTW89_MKK][0] = 40,
+ [0][0][RTW89_IC][0] = 62,
+ [0][0][RTW89_KCC][0] = 46,
+ [0][0][RTW89_ACMA][0] = 30,
+ [0][0][RTW89_CN][0] = 32,
+ [0][0][RTW89_UK][0] = 30,
+ [0][0][RTW89_FCC][1] = 62,
+ [0][0][RTW89_ETSI][1] = 30,
+ [0][0][RTW89_MKK][1] = 44,
+ [0][0][RTW89_IC][1] = 62,
+ [0][0][RTW89_KCC][1] = 46,
+ [0][0][RTW89_ACMA][1] = 30,
+ [0][0][RTW89_CN][1] = 32,
+ [0][0][RTW89_UK][1] = 30,
+ [0][0][RTW89_FCC][2] = 66,
+ [0][0][RTW89_ETSI][2] = 30,
+ [0][0][RTW89_MKK][2] = 44,
+ [0][0][RTW89_IC][2] = 66,
+ [0][0][RTW89_KCC][2] = 46,
+ [0][0][RTW89_ACMA][2] = 30,
+ [0][0][RTW89_CN][2] = 32,
+ [0][0][RTW89_UK][2] = 30,
+ [0][0][RTW89_FCC][3] = 70,
+ [0][0][RTW89_ETSI][3] = 30,
+ [0][0][RTW89_MKK][3] = 44,
+ [0][0][RTW89_IC][3] = 70,
+ [0][0][RTW89_KCC][3] = 46,
+ [0][0][RTW89_ACMA][3] = 30,
+ [0][0][RTW89_CN][3] = 32,
+ [0][0][RTW89_UK][3] = 30,
+ [0][0][RTW89_FCC][4] = 70,
+ [0][0][RTW89_ETSI][4] = 30,
+ [0][0][RTW89_MKK][4] = 44,
+ [0][0][RTW89_IC][4] = 70,
+ [0][0][RTW89_KCC][4] = 48,
+ [0][0][RTW89_ACMA][4] = 30,
+ [0][0][RTW89_CN][4] = 32,
+ [0][0][RTW89_UK][4] = 30,
+ [0][0][RTW89_FCC][5] = 84,
+ [0][0][RTW89_ETSI][5] = 30,
+ [0][0][RTW89_MKK][5] = 44,
+ [0][0][RTW89_IC][5] = 84,
+ [0][0][RTW89_KCC][5] = 48,
+ [0][0][RTW89_ACMA][5] = 30,
+ [0][0][RTW89_CN][5] = 32,
+ [0][0][RTW89_UK][5] = 30,
+ [0][0][RTW89_FCC][6] = 66,
+ [0][0][RTW89_ETSI][6] = 30,
+ [0][0][RTW89_MKK][6] = 44,
+ [0][0][RTW89_IC][6] = 66,
+ [0][0][RTW89_KCC][6] = 48,
+ [0][0][RTW89_ACMA][6] = 30,
+ [0][0][RTW89_CN][6] = 32,
+ [0][0][RTW89_UK][6] = 30,
+ [0][0][RTW89_FCC][7] = 66,
+ [0][0][RTW89_ETSI][7] = 30,
+ [0][0][RTW89_MKK][7] = 44,
+ [0][0][RTW89_IC][7] = 66,
+ [0][0][RTW89_KCC][7] = 48,
+ [0][0][RTW89_ACMA][7] = 30,
+ [0][0][RTW89_CN][7] = 32,
+ [0][0][RTW89_UK][7] = 30,
+ [0][0][RTW89_FCC][8] = 62,
+ [0][0][RTW89_ETSI][8] = 30,
+ [0][0][RTW89_MKK][8] = 44,
+ [0][0][RTW89_IC][8] = 62,
+ [0][0][RTW89_KCC][8] = 48,
+ [0][0][RTW89_ACMA][8] = 30,
+ [0][0][RTW89_CN][8] = 32,
+ [0][0][RTW89_UK][8] = 30,
+ [0][0][RTW89_FCC][9] = 58,
+ [0][0][RTW89_ETSI][9] = 30,
+ [0][0][RTW89_MKK][9] = 44,
+ [0][0][RTW89_IC][9] = 58,
+ [0][0][RTW89_KCC][9] = 44,
+ [0][0][RTW89_ACMA][9] = 30,
+ [0][0][RTW89_CN][9] = 32,
+ [0][0][RTW89_UK][9] = 30,
+ [0][0][RTW89_FCC][10] = 58,
+ [0][0][RTW89_ETSI][10] = 30,
+ [0][0][RTW89_MKK][10] = 44,
+ [0][0][RTW89_IC][10] = 58,
+ [0][0][RTW89_KCC][10] = 44,
+ [0][0][RTW89_ACMA][10] = 30,
+ [0][0][RTW89_CN][10] = 32,
+ [0][0][RTW89_UK][10] = 30,
+ [0][0][RTW89_FCC][11] = 54,
+ [0][0][RTW89_ETSI][11] = 30,
+ [0][0][RTW89_MKK][11] = 44,
+ [0][0][RTW89_IC][11] = 54,
+ [0][0][RTW89_KCC][11] = 44,
+ [0][0][RTW89_ACMA][11] = 30,
+ [0][0][RTW89_CN][11] = 32,
+ [0][0][RTW89_UK][11] = 30,
+ [0][0][RTW89_FCC][12] = 36,
+ [0][0][RTW89_ETSI][12] = 30,
+ [0][0][RTW89_MKK][12] = 40,
+ [0][0][RTW89_IC][12] = 36,
+ [0][0][RTW89_KCC][12] = 44,
+ [0][0][RTW89_ACMA][12] = 30,
+ [0][0][RTW89_CN][12] = 32,
+ [0][0][RTW89_UK][12] = 30,
+ [0][0][RTW89_FCC][13] = 127,
+ [0][0][RTW89_ETSI][13] = 127,
+ [0][0][RTW89_MKK][13] = 127,
+ [0][0][RTW89_IC][13] = 127,
+ [0][0][RTW89_KCC][13] = 127,
+ [0][0][RTW89_ACMA][13] = 127,
+ [0][0][RTW89_CN][13] = 127,
+ [0][0][RTW89_UK][13] = 127,
+ [0][1][RTW89_FCC][0] = 127,
+ [0][1][RTW89_ETSI][0] = 127,
+ [0][1][RTW89_MKK][0] = 127,
+ [0][1][RTW89_IC][0] = 127,
+ [0][1][RTW89_KCC][0] = 127,
+ [0][1][RTW89_ACMA][0] = 127,
+ [0][1][RTW89_CN][0] = 20,
+ [0][1][RTW89_UK][0] = 127,
+ [0][1][RTW89_FCC][1] = 127,
+ [0][1][RTW89_ETSI][1] = 127,
+ [0][1][RTW89_MKK][1] = 127,
+ [0][1][RTW89_IC][1] = 127,
+ [0][1][RTW89_KCC][1] = 127,
+ [0][1][RTW89_ACMA][1] = 127,
+ [0][1][RTW89_CN][1] = 22,
+ [0][1][RTW89_UK][1] = 127,
+ [0][1][RTW89_FCC][2] = 127,
+ [0][1][RTW89_ETSI][2] = 127,
+ [0][1][RTW89_MKK][2] = 127,
+ [0][1][RTW89_IC][2] = 127,
+ [0][1][RTW89_KCC][2] = 127,
+ [0][1][RTW89_ACMA][2] = 127,
+ [0][1][RTW89_CN][2] = 22,
+ [0][1][RTW89_UK][2] = 127,
+ [0][1][RTW89_FCC][3] = 127,
+ [0][1][RTW89_ETSI][3] = 127,
+ [0][1][RTW89_MKK][3] = 127,
+ [0][1][RTW89_IC][3] = 127,
+ [0][1][RTW89_KCC][3] = 127,
+ [0][1][RTW89_ACMA][3] = 127,
+ [0][1][RTW89_CN][3] = 22,
+ [0][1][RTW89_UK][3] = 127,
+ [0][1][RTW89_FCC][4] = 127,
+ [0][1][RTW89_ETSI][4] = 127,
+ [0][1][RTW89_MKK][4] = 127,
+ [0][1][RTW89_IC][4] = 127,
+ [0][1][RTW89_KCC][4] = 127,
+ [0][1][RTW89_ACMA][4] = 127,
+ [0][1][RTW89_CN][4] = 22,
+ [0][1][RTW89_UK][4] = 127,
+ [0][1][RTW89_FCC][5] = 127,
+ [0][1][RTW89_ETSI][5] = 127,
+ [0][1][RTW89_MKK][5] = 127,
+ [0][1][RTW89_IC][5] = 127,
+ [0][1][RTW89_KCC][5] = 127,
+ [0][1][RTW89_ACMA][5] = 127,
+ [0][1][RTW89_CN][5] = 22,
+ [0][1][RTW89_UK][5] = 127,
+ [0][1][RTW89_FCC][6] = 127,
+ [0][1][RTW89_ETSI][6] = 127,
+ [0][1][RTW89_MKK][6] = 127,
+ [0][1][RTW89_IC][6] = 127,
+ [0][1][RTW89_KCC][6] = 127,
+ [0][1][RTW89_ACMA][6] = 127,
+ [0][1][RTW89_CN][6] = 22,
+ [0][1][RTW89_UK][6] = 127,
+ [0][1][RTW89_FCC][7] = 127,
+ [0][1][RTW89_ETSI][7] = 127,
+ [0][1][RTW89_MKK][7] = 127,
+ [0][1][RTW89_IC][7] = 127,
+ [0][1][RTW89_KCC][7] = 127,
+ [0][1][RTW89_ACMA][7] = 127,
+ [0][1][RTW89_CN][7] = 22,
+ [0][1][RTW89_UK][7] = 127,
+ [0][1][RTW89_FCC][8] = 127,
+ [0][1][RTW89_ETSI][8] = 127,
+ [0][1][RTW89_MKK][8] = 127,
+ [0][1][RTW89_IC][8] = 127,
+ [0][1][RTW89_KCC][8] = 127,
+ [0][1][RTW89_ACMA][8] = 127,
+ [0][1][RTW89_CN][8] = 22,
+ [0][1][RTW89_UK][8] = 127,
+ [0][1][RTW89_FCC][9] = 127,
+ [0][1][RTW89_ETSI][9] = 127,
+ [0][1][RTW89_MKK][9] = 127,
+ [0][1][RTW89_IC][9] = 127,
+ [0][1][RTW89_KCC][9] = 127,
+ [0][1][RTW89_ACMA][9] = 127,
+ [0][1][RTW89_CN][9] = 22,
+ [0][1][RTW89_UK][9] = 127,
+ [0][1][RTW89_FCC][10] = 127,
+ [0][1][RTW89_ETSI][10] = 127,
+ [0][1][RTW89_MKK][10] = 127,
+ [0][1][RTW89_IC][10] = 127,
+ [0][1][RTW89_KCC][10] = 127,
+ [0][1][RTW89_ACMA][10] = 127,
+ [0][1][RTW89_CN][10] = 22,
+ [0][1][RTW89_UK][10] = 127,
+ [0][1][RTW89_FCC][11] = 127,
+ [0][1][RTW89_ETSI][11] = 127,
+ [0][1][RTW89_MKK][11] = 127,
+ [0][1][RTW89_IC][11] = 127,
+ [0][1][RTW89_KCC][11] = 127,
+ [0][1][RTW89_ACMA][11] = 127,
+ [0][1][RTW89_CN][11] = 22,
+ [0][1][RTW89_UK][11] = 127,
+ [0][1][RTW89_FCC][12] = 127,
+ [0][1][RTW89_ETSI][12] = 127,
+ [0][1][RTW89_MKK][12] = 127,
+ [0][1][RTW89_IC][12] = 127,
+ [0][1][RTW89_KCC][12] = 127,
+ [0][1][RTW89_ACMA][12] = 127,
+ [0][1][RTW89_CN][12] = 20,
+ [0][1][RTW89_UK][12] = 127,
+ [0][1][RTW89_FCC][13] = 127,
+ [0][1][RTW89_ETSI][13] = 127,
+ [0][1][RTW89_MKK][13] = 127,
+ [0][1][RTW89_IC][13] = 127,
+ [0][1][RTW89_KCC][13] = 127,
+ [0][1][RTW89_ACMA][13] = 127,
+ [0][1][RTW89_CN][13] = 127,
+ [0][1][RTW89_UK][13] = 127,
+ [1][0][RTW89_FCC][0] = 70,
+ [1][0][RTW89_ETSI][0] = 42,
+ [1][0][RTW89_MKK][0] = 52,
+ [1][0][RTW89_IC][0] = 70,
+ [1][0][RTW89_KCC][0] = 56,
+ [1][0][RTW89_ACMA][0] = 42,
+ [1][0][RTW89_CN][0] = 42,
+ [1][0][RTW89_UK][0] = 42,
+ [1][0][RTW89_FCC][1] = 70,
+ [1][0][RTW89_ETSI][1] = 42,
+ [1][0][RTW89_MKK][1] = 52,
+ [1][0][RTW89_IC][1] = 70,
+ [1][0][RTW89_KCC][1] = 56,
+ [1][0][RTW89_ACMA][1] = 42,
+ [1][0][RTW89_CN][1] = 44,
+ [1][0][RTW89_UK][1] = 42,
+ [1][0][RTW89_FCC][2] = 74,
+ [1][0][RTW89_ETSI][2] = 42,
+ [1][0][RTW89_MKK][2] = 52,
+ [1][0][RTW89_IC][2] = 74,
+ [1][0][RTW89_KCC][2] = 56,
+ [1][0][RTW89_ACMA][2] = 42,
+ [1][0][RTW89_CN][2] = 44,
+ [1][0][RTW89_UK][2] = 42,
+ [1][0][RTW89_FCC][3] = 76,
+ [1][0][RTW89_ETSI][3] = 42,
+ [1][0][RTW89_MKK][3] = 52,
+ [1][0][RTW89_IC][3] = 76,
+ [1][0][RTW89_KCC][3] = 56,
+ [1][0][RTW89_ACMA][3] = 42,
+ [1][0][RTW89_CN][3] = 44,
+ [1][0][RTW89_UK][3] = 42,
+ [1][0][RTW89_FCC][4] = 76,
+ [1][0][RTW89_ETSI][4] = 42,
+ [1][0][RTW89_MKK][4] = 52,
+ [1][0][RTW89_IC][4] = 76,
+ [1][0][RTW89_KCC][4] = 56,
+ [1][0][RTW89_ACMA][4] = 42,
+ [1][0][RTW89_CN][4] = 44,
+ [1][0][RTW89_UK][4] = 42,
+ [1][0][RTW89_FCC][5] = 82,
+ [1][0][RTW89_ETSI][5] = 42,
+ [1][0][RTW89_MKK][5] = 52,
+ [1][0][RTW89_IC][5] = 82,
+ [1][0][RTW89_KCC][5] = 56,
+ [1][0][RTW89_ACMA][5] = 42,
+ [1][0][RTW89_CN][5] = 44,
+ [1][0][RTW89_UK][5] = 42,
+ [1][0][RTW89_FCC][6] = 74,
+ [1][0][RTW89_ETSI][6] = 42,
+ [1][0][RTW89_MKK][6] = 52,
+ [1][0][RTW89_IC][6] = 74,
+ [1][0][RTW89_KCC][6] = 56,
+ [1][0][RTW89_ACMA][6] = 42,
+ [1][0][RTW89_CN][6] = 44,
+ [1][0][RTW89_UK][6] = 42,
+ [1][0][RTW89_FCC][7] = 74,
+ [1][0][RTW89_ETSI][7] = 42,
+ [1][0][RTW89_MKK][7] = 52,
+ [1][0][RTW89_IC][7] = 74,
+ [1][0][RTW89_KCC][7] = 56,
+ [1][0][RTW89_ACMA][7] = 42,
+ [1][0][RTW89_CN][7] = 44,
+ [1][0][RTW89_UK][7] = 42,
+ [1][0][RTW89_FCC][8] = 74,
+ [1][0][RTW89_ETSI][8] = 42,
+ [1][0][RTW89_MKK][8] = 52,
+ [1][0][RTW89_IC][8] = 74,
+ [1][0][RTW89_KCC][8] = 56,
+ [1][0][RTW89_ACMA][8] = 42,
+ [1][0][RTW89_CN][8] = 44,
+ [1][0][RTW89_UK][8] = 42,
+ [1][0][RTW89_FCC][9] = 70,
+ [1][0][RTW89_ETSI][9] = 42,
+ [1][0][RTW89_MKK][9] = 52,
+ [1][0][RTW89_IC][9] = 70,
+ [1][0][RTW89_KCC][9] = 58,
+ [1][0][RTW89_ACMA][9] = 42,
+ [1][0][RTW89_CN][9] = 44,
+ [1][0][RTW89_UK][9] = 42,
+ [1][0][RTW89_FCC][10] = 70,
+ [1][0][RTW89_ETSI][10] = 42,
+ [1][0][RTW89_MKK][10] = 52,
+ [1][0][RTW89_IC][10] = 70,
+ [1][0][RTW89_KCC][10] = 58,
+ [1][0][RTW89_ACMA][10] = 42,
+ [1][0][RTW89_CN][10] = 44,
+ [1][0][RTW89_UK][10] = 42,
+ [1][0][RTW89_FCC][11] = 66,
+ [1][0][RTW89_ETSI][11] = 42,
+ [1][0][RTW89_MKK][11] = 52,
+ [1][0][RTW89_IC][11] = 66,
+ [1][0][RTW89_KCC][11] = 58,
+ [1][0][RTW89_ACMA][11] = 42,
+ [1][0][RTW89_CN][11] = 44,
+ [1][0][RTW89_UK][11] = 42,
+ [1][0][RTW89_FCC][12] = 34,
+ [1][0][RTW89_ETSI][12] = 42,
+ [1][0][RTW89_MKK][12] = 52,
+ [1][0][RTW89_IC][12] = 34,
+ [1][0][RTW89_KCC][12] = 58,
+ [1][0][RTW89_ACMA][12] = 42,
+ [1][0][RTW89_CN][12] = 42,
+ [1][0][RTW89_UK][12] = 42,
+ [1][0][RTW89_FCC][13] = 127,
+ [1][0][RTW89_ETSI][13] = 127,
+ [1][0][RTW89_MKK][13] = 127,
+ [1][0][RTW89_IC][13] = 127,
+ [1][0][RTW89_KCC][13] = 127,
+ [1][0][RTW89_ACMA][13] = 127,
+ [1][0][RTW89_CN][13] = 127,
+ [1][0][RTW89_UK][13] = 127,
+ [1][1][RTW89_FCC][0] = 127,
+ [1][1][RTW89_ETSI][0] = 127,
+ [1][1][RTW89_MKK][0] = 127,
+ [1][1][RTW89_IC][0] = 127,
+ [1][1][RTW89_KCC][0] = 127,
+ [1][1][RTW89_ACMA][0] = 127,
+ [1][1][RTW89_CN][0] = 32,
+ [1][1][RTW89_UK][0] = 127,
+ [1][1][RTW89_FCC][1] = 127,
+ [1][1][RTW89_ETSI][1] = 127,
+ [1][1][RTW89_MKK][1] = 127,
+ [1][1][RTW89_IC][1] = 127,
+ [1][1][RTW89_KCC][1] = 127,
+ [1][1][RTW89_ACMA][1] = 127,
+ [1][1][RTW89_CN][1] = 32,
+ [1][1][RTW89_UK][1] = 127,
+ [1][1][RTW89_FCC][2] = 127,
+ [1][1][RTW89_ETSI][2] = 127,
+ [1][1][RTW89_MKK][2] = 127,
+ [1][1][RTW89_IC][2] = 127,
+ [1][1][RTW89_KCC][2] = 127,
+ [1][1][RTW89_ACMA][2] = 127,
+ [1][1][RTW89_CN][2] = 32,
+ [1][1][RTW89_UK][2] = 127,
+ [1][1][RTW89_FCC][3] = 127,
+ [1][1][RTW89_ETSI][3] = 127,
+ [1][1][RTW89_MKK][3] = 127,
+ [1][1][RTW89_IC][3] = 127,
+ [1][1][RTW89_KCC][3] = 127,
+ [1][1][RTW89_ACMA][3] = 127,
+ [1][1][RTW89_CN][3] = 32,
+ [1][1][RTW89_UK][3] = 127,
+ [1][1][RTW89_FCC][4] = 127,
+ [1][1][RTW89_ETSI][4] = 127,
+ [1][1][RTW89_MKK][4] = 127,
+ [1][1][RTW89_IC][4] = 127,
+ [1][1][RTW89_KCC][4] = 127,
+ [1][1][RTW89_ACMA][4] = 127,
+ [1][1][RTW89_CN][4] = 32,
+ [1][1][RTW89_UK][4] = 127,
+ [1][1][RTW89_FCC][5] = 127,
+ [1][1][RTW89_ETSI][5] = 127,
+ [1][1][RTW89_MKK][5] = 127,
+ [1][1][RTW89_IC][5] = 127,
+ [1][1][RTW89_KCC][5] = 127,
+ [1][1][RTW89_ACMA][5] = 127,
+ [1][1][RTW89_CN][5] = 32,
+ [1][1][RTW89_UK][5] = 127,
+ [1][1][RTW89_FCC][6] = 127,
+ [1][1][RTW89_ETSI][6] = 127,
+ [1][1][RTW89_MKK][6] = 127,
+ [1][1][RTW89_IC][6] = 127,
+ [1][1][RTW89_KCC][6] = 127,
+ [1][1][RTW89_ACMA][6] = 127,
+ [1][1][RTW89_CN][6] = 32,
+ [1][1][RTW89_UK][6] = 127,
+ [1][1][RTW89_FCC][7] = 127,
+ [1][1][RTW89_ETSI][7] = 127,
+ [1][1][RTW89_MKK][7] = 127,
+ [1][1][RTW89_IC][7] = 127,
+ [1][1][RTW89_KCC][7] = 127,
+ [1][1][RTW89_ACMA][7] = 127,
+ [1][1][RTW89_CN][7] = 32,
+ [1][1][RTW89_UK][7] = 127,
+ [1][1][RTW89_FCC][8] = 127,
+ [1][1][RTW89_ETSI][8] = 127,
+ [1][1][RTW89_MKK][8] = 127,
+ [1][1][RTW89_IC][8] = 127,
+ [1][1][RTW89_KCC][8] = 127,
+ [1][1][RTW89_ACMA][8] = 127,
+ [1][1][RTW89_CN][8] = 32,
+ [1][1][RTW89_UK][8] = 127,
+ [1][1][RTW89_FCC][9] = 127,
+ [1][1][RTW89_ETSI][9] = 127,
+ [1][1][RTW89_MKK][9] = 127,
+ [1][1][RTW89_IC][9] = 127,
+ [1][1][RTW89_KCC][9] = 127,
+ [1][1][RTW89_ACMA][9] = 127,
+ [1][1][RTW89_CN][9] = 32,
+ [1][1][RTW89_UK][9] = 127,
+ [1][1][RTW89_FCC][10] = 127,
+ [1][1][RTW89_ETSI][10] = 127,
+ [1][1][RTW89_MKK][10] = 127,
+ [1][1][RTW89_IC][10] = 127,
+ [1][1][RTW89_KCC][10] = 127,
+ [1][1][RTW89_ACMA][10] = 127,
+ [1][1][RTW89_CN][10] = 32,
+ [1][1][RTW89_UK][10] = 127,
+ [1][1][RTW89_FCC][11] = 127,
+ [1][1][RTW89_ETSI][11] = 127,
+ [1][1][RTW89_MKK][11] = 127,
+ [1][1][RTW89_IC][11] = 127,
+ [1][1][RTW89_KCC][11] = 127,
+ [1][1][RTW89_ACMA][11] = 127,
+ [1][1][RTW89_CN][11] = 32,
+ [1][1][RTW89_UK][11] = 127,
+ [1][1][RTW89_FCC][12] = 127,
+ [1][1][RTW89_ETSI][12] = 127,
+ [1][1][RTW89_MKK][12] = 127,
+ [1][1][RTW89_IC][12] = 127,
+ [1][1][RTW89_KCC][12] = 127,
+ [1][1][RTW89_ACMA][12] = 127,
+ [1][1][RTW89_CN][12] = 32,
+ [1][1][RTW89_UK][12] = 127,
+ [1][1][RTW89_FCC][13] = 127,
+ [1][1][RTW89_ETSI][13] = 127,
+ [1][1][RTW89_MKK][13] = 127,
+ [1][1][RTW89_IC][13] = 127,
+ [1][1][RTW89_KCC][13] = 127,
+ [1][1][RTW89_ACMA][13] = 127,
+ [1][1][RTW89_CN][13] = 127,
+ [1][1][RTW89_UK][13] = 127,
+ [2][0][RTW89_FCC][0] = 76,
+ [2][0][RTW89_ETSI][0] = 54,
+ [2][0][RTW89_MKK][0] = 64,
+ [2][0][RTW89_IC][0] = 76,
+ [2][0][RTW89_KCC][0] = 68,
+ [2][0][RTW89_ACMA][0] = 54,
+ [2][0][RTW89_CN][0] = 56,
+ [2][0][RTW89_UK][0] = 54,
+ [2][0][RTW89_FCC][1] = 76,
+ [2][0][RTW89_ETSI][1] = 54,
+ [2][0][RTW89_MKK][1] = 64,
+ [2][0][RTW89_IC][1] = 76,
+ [2][0][RTW89_KCC][1] = 68,
+ [2][0][RTW89_ACMA][1] = 54,
+ [2][0][RTW89_CN][1] = 56,
+ [2][0][RTW89_UK][1] = 54,
+ [2][0][RTW89_FCC][2] = 78,
+ [2][0][RTW89_ETSI][2] = 54,
+ [2][0][RTW89_MKK][2] = 64,
+ [2][0][RTW89_IC][2] = 78,
+ [2][0][RTW89_KCC][2] = 68,
+ [2][0][RTW89_ACMA][2] = 54,
+ [2][0][RTW89_CN][2] = 56,
+ [2][0][RTW89_UK][2] = 54,
+ [2][0][RTW89_FCC][3] = 78,
+ [2][0][RTW89_ETSI][3] = 54,
+ [2][0][RTW89_MKK][3] = 64,
+ [2][0][RTW89_IC][3] = 78,
+ [2][0][RTW89_KCC][3] = 68,
+ [2][0][RTW89_ACMA][3] = 54,
+ [2][0][RTW89_CN][3] = 56,
+ [2][0][RTW89_UK][3] = 54,
+ [2][0][RTW89_FCC][4] = 78,
+ [2][0][RTW89_ETSI][4] = 54,
+ [2][0][RTW89_MKK][4] = 64,
+ [2][0][RTW89_IC][4] = 78,
+ [2][0][RTW89_KCC][4] = 68,
+ [2][0][RTW89_ACMA][4] = 54,
+ [2][0][RTW89_CN][4] = 56,
+ [2][0][RTW89_UK][4] = 54,
+ [2][0][RTW89_FCC][5] = 82,
+ [2][0][RTW89_ETSI][5] = 54,
+ [2][0][RTW89_MKK][5] = 64,
+ [2][0][RTW89_IC][5] = 82,
+ [2][0][RTW89_KCC][5] = 68,
+ [2][0][RTW89_ACMA][5] = 54,
+ [2][0][RTW89_CN][5] = 56,
+ [2][0][RTW89_UK][5] = 54,
+ [2][0][RTW89_FCC][6] = 74,
+ [2][0][RTW89_ETSI][6] = 54,
+ [2][0][RTW89_MKK][6] = 64,
+ [2][0][RTW89_IC][6] = 74,
+ [2][0][RTW89_KCC][6] = 68,
+ [2][0][RTW89_ACMA][6] = 54,
+ [2][0][RTW89_CN][6] = 56,
+ [2][0][RTW89_UK][6] = 54,
+ [2][0][RTW89_FCC][7] = 74,
+ [2][0][RTW89_ETSI][7] = 54,
+ [2][0][RTW89_MKK][7] = 64,
+ [2][0][RTW89_IC][7] = 74,
+ [2][0][RTW89_KCC][7] = 68,
+ [2][0][RTW89_ACMA][7] = 54,
+ [2][0][RTW89_CN][7] = 56,
+ [2][0][RTW89_UK][7] = 54,
+ [2][0][RTW89_FCC][8] = 74,
+ [2][0][RTW89_ETSI][8] = 54,
+ [2][0][RTW89_MKK][8] = 64,
+ [2][0][RTW89_IC][8] = 74,
+ [2][0][RTW89_KCC][8] = 68,
+ [2][0][RTW89_ACMA][8] = 54,
+ [2][0][RTW89_CN][8] = 56,
+ [2][0][RTW89_UK][8] = 54,
+ [2][0][RTW89_FCC][9] = 72,
+ [2][0][RTW89_ETSI][9] = 54,
+ [2][0][RTW89_MKK][9] = 64,
+ [2][0][RTW89_IC][9] = 72,
+ [2][0][RTW89_KCC][9] = 68,
+ [2][0][RTW89_ACMA][9] = 54,
+ [2][0][RTW89_CN][9] = 56,
+ [2][0][RTW89_UK][9] = 54,
+ [2][0][RTW89_FCC][10] = 72,
+ [2][0][RTW89_ETSI][10] = 54,
+ [2][0][RTW89_MKK][10] = 64,
+ [2][0][RTW89_IC][10] = 72,
+ [2][0][RTW89_KCC][10] = 68,
+ [2][0][RTW89_ACMA][10] = 54,
+ [2][0][RTW89_CN][10] = 56,
+ [2][0][RTW89_UK][10] = 54,
+ [2][0][RTW89_FCC][11] = 64,
+ [2][0][RTW89_ETSI][11] = 54,
+ [2][0][RTW89_MKK][11] = 64,
+ [2][0][RTW89_IC][11] = 64,
+ [2][0][RTW89_KCC][11] = 68,
+ [2][0][RTW89_ACMA][11] = 54,
+ [2][0][RTW89_CN][11] = 56,
+ [2][0][RTW89_UK][11] = 54,
+ [2][0][RTW89_FCC][12] = 34,
+ [2][0][RTW89_ETSI][12] = 54,
+ [2][0][RTW89_MKK][12] = 64,
+ [2][0][RTW89_IC][12] = 34,
+ [2][0][RTW89_KCC][12] = 68,
+ [2][0][RTW89_ACMA][12] = 54,
+ [2][0][RTW89_CN][12] = 56,
+ [2][0][RTW89_UK][12] = 54,
+ [2][0][RTW89_FCC][13] = 127,
+ [2][0][RTW89_ETSI][13] = 127,
+ [2][0][RTW89_MKK][13] = 127,
+ [2][0][RTW89_IC][13] = 127,
+ [2][0][RTW89_KCC][13] = 127,
+ [2][0][RTW89_ACMA][13] = 127,
+ [2][0][RTW89_CN][13] = 127,
+ [2][0][RTW89_UK][13] = 127,
+ [2][1][RTW89_FCC][0] = 127,
+ [2][1][RTW89_ETSI][0] = 127,
+ [2][1][RTW89_MKK][0] = 127,
+ [2][1][RTW89_IC][0] = 127,
+ [2][1][RTW89_KCC][0] = 127,
+ [2][1][RTW89_ACMA][0] = 127,
+ [2][1][RTW89_CN][0] = 44,
+ [2][1][RTW89_UK][0] = 127,
+ [2][1][RTW89_FCC][1] = 127,
+ [2][1][RTW89_ETSI][1] = 127,
+ [2][1][RTW89_MKK][1] = 127,
+ [2][1][RTW89_IC][1] = 127,
+ [2][1][RTW89_KCC][1] = 127,
+ [2][1][RTW89_ACMA][1] = 127,
+ [2][1][RTW89_CN][1] = 44,
+ [2][1][RTW89_UK][1] = 127,
+ [2][1][RTW89_FCC][2] = 127,
+ [2][1][RTW89_ETSI][2] = 127,
+ [2][1][RTW89_MKK][2] = 127,
+ [2][1][RTW89_IC][2] = 127,
+ [2][1][RTW89_KCC][2] = 127,
+ [2][1][RTW89_ACMA][2] = 127,
+ [2][1][RTW89_CN][2] = 44,
+ [2][1][RTW89_UK][2] = 127,
+ [2][1][RTW89_FCC][3] = 127,
+ [2][1][RTW89_ETSI][3] = 127,
+ [2][1][RTW89_MKK][3] = 127,
+ [2][1][RTW89_IC][3] = 127,
+ [2][1][RTW89_KCC][3] = 127,
+ [2][1][RTW89_ACMA][3] = 127,
+ [2][1][RTW89_CN][3] = 44,
+ [2][1][RTW89_UK][3] = 127,
+ [2][1][RTW89_FCC][4] = 127,
+ [2][1][RTW89_ETSI][4] = 127,
+ [2][1][RTW89_MKK][4] = 127,
+ [2][1][RTW89_IC][4] = 127,
+ [2][1][RTW89_KCC][4] = 127,
+ [2][1][RTW89_ACMA][4] = 127,
+ [2][1][RTW89_CN][4] = 44,
+ [2][1][RTW89_UK][4] = 127,
+ [2][1][RTW89_FCC][5] = 127,
+ [2][1][RTW89_ETSI][5] = 127,
+ [2][1][RTW89_MKK][5] = 127,
+ [2][1][RTW89_IC][5] = 127,
+ [2][1][RTW89_KCC][5] = 127,
+ [2][1][RTW89_ACMA][5] = 127,
+ [2][1][RTW89_CN][5] = 44,
+ [2][1][RTW89_UK][5] = 127,
+ [2][1][RTW89_FCC][6] = 127,
+ [2][1][RTW89_ETSI][6] = 127,
+ [2][1][RTW89_MKK][6] = 127,
+ [2][1][RTW89_IC][6] = 127,
+ [2][1][RTW89_KCC][6] = 127,
+ [2][1][RTW89_ACMA][6] = 127,
+ [2][1][RTW89_CN][6] = 44,
+ [2][1][RTW89_UK][6] = 127,
+ [2][1][RTW89_FCC][7] = 127,
+ [2][1][RTW89_ETSI][7] = 127,
+ [2][1][RTW89_MKK][7] = 127,
+ [2][1][RTW89_IC][7] = 127,
+ [2][1][RTW89_KCC][7] = 127,
+ [2][1][RTW89_ACMA][7] = 127,
+ [2][1][RTW89_CN][7] = 44,
+ [2][1][RTW89_UK][7] = 127,
+ [2][1][RTW89_FCC][8] = 127,
+ [2][1][RTW89_ETSI][8] = 127,
+ [2][1][RTW89_MKK][8] = 127,
+ [2][1][RTW89_IC][8] = 127,
+ [2][1][RTW89_KCC][8] = 127,
+ [2][1][RTW89_ACMA][8] = 127,
+ [2][1][RTW89_CN][8] = 44,
+ [2][1][RTW89_UK][8] = 127,
+ [2][1][RTW89_FCC][9] = 127,
+ [2][1][RTW89_ETSI][9] = 127,
+ [2][1][RTW89_MKK][9] = 127,
+ [2][1][RTW89_IC][9] = 127,
+ [2][1][RTW89_KCC][9] = 127,
+ [2][1][RTW89_ACMA][9] = 127,
+ [2][1][RTW89_CN][9] = 44,
+ [2][1][RTW89_UK][9] = 127,
+ [2][1][RTW89_FCC][10] = 127,
+ [2][1][RTW89_ETSI][10] = 127,
+ [2][1][RTW89_MKK][10] = 127,
+ [2][1][RTW89_IC][10] = 127,
+ [2][1][RTW89_KCC][10] = 127,
+ [2][1][RTW89_ACMA][10] = 127,
+ [2][1][RTW89_CN][10] = 44,
+ [2][1][RTW89_UK][10] = 127,
+ [2][1][RTW89_FCC][11] = 127,
+ [2][1][RTW89_ETSI][11] = 127,
+ [2][1][RTW89_MKK][11] = 127,
+ [2][1][RTW89_IC][11] = 127,
+ [2][1][RTW89_KCC][11] = 127,
+ [2][1][RTW89_ACMA][11] = 127,
+ [2][1][RTW89_CN][11] = 44,
+ [2][1][RTW89_UK][11] = 127,
+ [2][1][RTW89_FCC][12] = 127,
+ [2][1][RTW89_ETSI][12] = 127,
+ [2][1][RTW89_MKK][12] = 127,
+ [2][1][RTW89_IC][12] = 127,
+ [2][1][RTW89_KCC][12] = 127,
+ [2][1][RTW89_ACMA][12] = 127,
+ [2][1][RTW89_CN][12] = 42,
+ [2][1][RTW89_UK][12] = 127,
+ [2][1][RTW89_FCC][13] = 127,
+ [2][1][RTW89_ETSI][13] = 127,
+ [2][1][RTW89_MKK][13] = 127,
+ [2][1][RTW89_IC][13] = 127,
+ [2][1][RTW89_KCC][13] = 127,
+ [2][1][RTW89_ACMA][13] = 127,
+ [2][1][RTW89_CN][13] = 127,
+ [2][1][RTW89_UK][13] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
+ [0][0][RTW89_WW][0] = 16,
+ [0][0][RTW89_WW][2] = 16,
+ [0][0][RTW89_WW][4] = 16,
+ [0][0][RTW89_WW][6] = 16,
+ [0][0][RTW89_WW][8] = 16,
+ [0][0][RTW89_WW][10] = 16,
+ [0][0][RTW89_WW][12] = 16,
+ [0][0][RTW89_WW][14] = 16,
+ [0][0][RTW89_WW][15] = 24,
+ [0][0][RTW89_WW][17] = 24,
+ [0][0][RTW89_WW][19] = 24,
+ [0][0][RTW89_WW][21] = 24,
+ [0][0][RTW89_WW][23] = 24,
+ [0][0][RTW89_WW][25] = 24,
+ [0][0][RTW89_WW][27] = 24,
+ [0][0][RTW89_WW][29] = 24,
+ [0][0][RTW89_WW][31] = 24,
+ [0][0][RTW89_WW][33] = 24,
+ [0][0][RTW89_WW][35] = 24,
+ [0][0][RTW89_WW][37] = 44,
+ [0][0][RTW89_WW][38] = 24,
+ [0][0][RTW89_WW][40] = 24,
+ [0][0][RTW89_WW][42] = 24,
+ [0][0][RTW89_WW][44] = 24,
+ [0][0][RTW89_WW][46] = 24,
+ [0][0][RTW89_WW][48] = 42,
+ [0][0][RTW89_WW][50] = 42,
+ [0][0][RTW89_WW][52] = 40,
+ [0][1][RTW89_WW][0] = 4,
+ [0][1][RTW89_WW][2] = 4,
+ [0][1][RTW89_WW][4] = 4,
+ [0][1][RTW89_WW][6] = 4,
+ [0][1][RTW89_WW][8] = 4,
+ [0][1][RTW89_WW][10] = 4,
+ [0][1][RTW89_WW][12] = 4,
+ [0][1][RTW89_WW][14] = 4,
+ [0][1][RTW89_WW][15] = 0,
+ [0][1][RTW89_WW][17] = 0,
+ [0][1][RTW89_WW][19] = 0,
+ [0][1][RTW89_WW][21] = 0,
+ [0][1][RTW89_WW][23] = 0,
+ [0][1][RTW89_WW][25] = 0,
+ [0][1][RTW89_WW][27] = 0,
+ [0][1][RTW89_WW][29] = 0,
+ [0][1][RTW89_WW][31] = 0,
+ [0][1][RTW89_WW][33] = 0,
+ [0][1][RTW89_WW][35] = 0,
+ [0][1][RTW89_WW][37] = 0,
+ [0][1][RTW89_WW][38] = 42,
+ [0][1][RTW89_WW][40] = 42,
+ [0][1][RTW89_WW][42] = 42,
+ [0][1][RTW89_WW][44] = 42,
+ [0][1][RTW89_WW][46] = 42,
+ [0][1][RTW89_WW][48] = 0,
+ [0][1][RTW89_WW][50] = 0,
+ [0][1][RTW89_WW][52] = 0,
+ [1][0][RTW89_WW][0] = 26,
+ [1][0][RTW89_WW][2] = 26,
+ [1][0][RTW89_WW][4] = 26,
+ [1][0][RTW89_WW][6] = 26,
+ [1][0][RTW89_WW][8] = 26,
+ [1][0][RTW89_WW][10] = 26,
+ [1][0][RTW89_WW][12] = 26,
+ [1][0][RTW89_WW][14] = 26,
+ [1][0][RTW89_WW][15] = 34,
+ [1][0][RTW89_WW][17] = 34,
+ [1][0][RTW89_WW][19] = 34,
+ [1][0][RTW89_WW][21] = 34,
+ [1][0][RTW89_WW][23] = 34,
+ [1][0][RTW89_WW][25] = 34,
+ [1][0][RTW89_WW][27] = 34,
+ [1][0][RTW89_WW][29] = 34,
+ [1][0][RTW89_WW][31] = 34,
+ [1][0][RTW89_WW][33] = 34,
+ [1][0][RTW89_WW][35] = 34,
+ [1][0][RTW89_WW][37] = 54,
+ [1][0][RTW89_WW][38] = 28,
+ [1][0][RTW89_WW][40] = 28,
+ [1][0][RTW89_WW][42] = 28,
+ [1][0][RTW89_WW][44] = 28,
+ [1][0][RTW89_WW][46] = 28,
+ [1][0][RTW89_WW][48] = 52,
+ [1][0][RTW89_WW][50] = 52,
+ [1][0][RTW89_WW][52] = 52,
+ [1][1][RTW89_WW][0] = 14,
+ [1][1][RTW89_WW][2] = 14,
+ [1][1][RTW89_WW][4] = 14,
+ [1][1][RTW89_WW][6] = 14,
+ [1][1][RTW89_WW][8] = 14,
+ [1][1][RTW89_WW][10] = 14,
+ [1][1][RTW89_WW][12] = 14,
+ [1][1][RTW89_WW][14] = 14,
+ [1][1][RTW89_WW][15] = 0,
+ [1][1][RTW89_WW][17] = 0,
+ [1][1][RTW89_WW][19] = 0,
+ [1][1][RTW89_WW][21] = 0,
+ [1][1][RTW89_WW][23] = 0,
+ [1][1][RTW89_WW][25] = 0,
+ [1][1][RTW89_WW][27] = 0,
+ [1][1][RTW89_WW][29] = 0,
+ [1][1][RTW89_WW][31] = 0,
+ [1][1][RTW89_WW][33] = 0,
+ [1][1][RTW89_WW][35] = 0,
+ [1][1][RTW89_WW][37] = 0,
+ [1][1][RTW89_WW][38] = 54,
+ [1][1][RTW89_WW][40] = 54,
+ [1][1][RTW89_WW][42] = 54,
+ [1][1][RTW89_WW][44] = 54,
+ [1][1][RTW89_WW][46] = 54,
+ [1][1][RTW89_WW][48] = 0,
+ [1][1][RTW89_WW][50] = 0,
+ [1][1][RTW89_WW][52] = 0,
+ [2][0][RTW89_WW][0] = 40,
+ [2][0][RTW89_WW][2] = 40,
+ [2][0][RTW89_WW][4] = 40,
+ [2][0][RTW89_WW][6] = 40,
+ [2][0][RTW89_WW][8] = 40,
+ [2][0][RTW89_WW][10] = 40,
+ [2][0][RTW89_WW][12] = 40,
+ [2][0][RTW89_WW][14] = 40,
+ [2][0][RTW89_WW][15] = 46,
+ [2][0][RTW89_WW][17] = 46,
+ [2][0][RTW89_WW][19] = 46,
+ [2][0][RTW89_WW][21] = 46,
+ [2][0][RTW89_WW][23] = 46,
+ [2][0][RTW89_WW][25] = 46,
+ [2][0][RTW89_WW][27] = 46,
+ [2][0][RTW89_WW][29] = 46,
+ [2][0][RTW89_WW][31] = 46,
+ [2][0][RTW89_WW][33] = 46,
+ [2][0][RTW89_WW][35] = 46,
+ [2][0][RTW89_WW][37] = 66,
+ [2][0][RTW89_WW][38] = 28,
+ [2][0][RTW89_WW][40] = 28,
+ [2][0][RTW89_WW][42] = 28,
+ [2][0][RTW89_WW][44] = 28,
+ [2][0][RTW89_WW][46] = 28,
+ [2][0][RTW89_WW][48] = 64,
+ [2][0][RTW89_WW][50] = 64,
+ [2][0][RTW89_WW][52] = 60,
+ [2][1][RTW89_WW][0] = 28,
+ [2][1][RTW89_WW][2] = 28,
+ [2][1][RTW89_WW][4] = 28,
+ [2][1][RTW89_WW][6] = 28,
+ [2][1][RTW89_WW][8] = 28,
+ [2][1][RTW89_WW][10] = 28,
+ [2][1][RTW89_WW][12] = 28,
+ [2][1][RTW89_WW][14] = 28,
+ [2][1][RTW89_WW][15] = 0,
+ [2][1][RTW89_WW][17] = 0,
+ [2][1][RTW89_WW][19] = 0,
+ [2][1][RTW89_WW][21] = 0,
+ [2][1][RTW89_WW][23] = 0,
+ [2][1][RTW89_WW][25] = 0,
+ [2][1][RTW89_WW][27] = 0,
+ [2][1][RTW89_WW][29] = 0,
+ [2][1][RTW89_WW][31] = 0,
+ [2][1][RTW89_WW][33] = 0,
+ [2][1][RTW89_WW][35] = 0,
+ [2][1][RTW89_WW][37] = 0,
+ [2][1][RTW89_WW][38] = 56,
+ [2][1][RTW89_WW][40] = 56,
+ [2][1][RTW89_WW][42] = 56,
+ [2][1][RTW89_WW][44] = 56,
+ [2][1][RTW89_WW][46] = 56,
+ [2][1][RTW89_WW][48] = 0,
+ [2][1][RTW89_WW][50] = 0,
+ [2][1][RTW89_WW][52] = 0,
+ [0][0][RTW89_FCC][0] = 52,
+ [0][0][RTW89_ETSI][0] = 24,
+ [0][0][RTW89_MKK][0] = 26,
+ [0][0][RTW89_IC][0] = 28,
+ [0][0][RTW89_KCC][0] = 42,
+ [0][0][RTW89_ACMA][0] = 24,
+ [0][0][RTW89_CN][0] = 16,
+ [0][0][RTW89_UK][0] = 24,
+ [0][0][RTW89_FCC][2] = 54,
+ [0][0][RTW89_ETSI][2] = 24,
+ [0][0][RTW89_MKK][2] = 26,
+ [0][0][RTW89_IC][2] = 28,
+ [0][0][RTW89_KCC][2] = 42,
+ [0][0][RTW89_ACMA][2] = 24,
+ [0][0][RTW89_CN][2] = 16,
+ [0][0][RTW89_UK][2] = 24,
+ [0][0][RTW89_FCC][4] = 52,
+ [0][0][RTW89_ETSI][4] = 24,
+ [0][0][RTW89_MKK][4] = 26,
+ [0][0][RTW89_IC][4] = 28,
+ [0][0][RTW89_KCC][4] = 42,
+ [0][0][RTW89_ACMA][4] = 24,
+ [0][0][RTW89_CN][4] = 16,
+ [0][0][RTW89_UK][4] = 24,
+ [0][0][RTW89_FCC][6] = 52,
+ [0][0][RTW89_ETSI][6] = 24,
+ [0][0][RTW89_MKK][6] = 26,
+ [0][0][RTW89_IC][6] = 28,
+ [0][0][RTW89_KCC][6] = 18,
+ [0][0][RTW89_ACMA][6] = 24,
+ [0][0][RTW89_CN][6] = 16,
+ [0][0][RTW89_UK][6] = 24,
+ [0][0][RTW89_FCC][8] = 52,
+ [0][0][RTW89_ETSI][8] = 24,
+ [0][0][RTW89_MKK][8] = 26,
+ [0][0][RTW89_IC][8] = 52,
+ [0][0][RTW89_KCC][8] = 42,
+ [0][0][RTW89_ACMA][8] = 24,
+ [0][0][RTW89_CN][8] = 16,
+ [0][0][RTW89_UK][8] = 24,
+ [0][0][RTW89_FCC][10] = 52,
+ [0][0][RTW89_ETSI][10] = 24,
+ [0][0][RTW89_MKK][10] = 26,
+ [0][0][RTW89_IC][10] = 52,
+ [0][0][RTW89_KCC][10] = 42,
+ [0][0][RTW89_ACMA][10] = 24,
+ [0][0][RTW89_CN][10] = 16,
+ [0][0][RTW89_UK][10] = 24,
+ [0][0][RTW89_FCC][12] = 56,
+ [0][0][RTW89_ETSI][12] = 24,
+ [0][0][RTW89_MKK][12] = 26,
+ [0][0][RTW89_IC][12] = 56,
+ [0][0][RTW89_KCC][12] = 44,
+ [0][0][RTW89_ACMA][12] = 24,
+ [0][0][RTW89_CN][12] = 16,
+ [0][0][RTW89_UK][12] = 24,
+ [0][0][RTW89_FCC][14] = 56,
+ [0][0][RTW89_ETSI][14] = 24,
+ [0][0][RTW89_MKK][14] = 26,
+ [0][0][RTW89_IC][14] = 56,
+ [0][0][RTW89_KCC][14] = 44,
+ [0][0][RTW89_ACMA][14] = 24,
+ [0][0][RTW89_CN][14] = 16,
+ [0][0][RTW89_UK][14] = 24,
+ [0][0][RTW89_FCC][15] = 54,
+ [0][0][RTW89_ETSI][15] = 24,
+ [0][0][RTW89_MKK][15] = 46,
+ [0][0][RTW89_IC][15] = 54,
+ [0][0][RTW89_KCC][15] = 44,
+ [0][0][RTW89_ACMA][15] = 24,
+ [0][0][RTW89_CN][15] = 127,
+ [0][0][RTW89_UK][15] = 24,
+ [0][0][RTW89_FCC][17] = 54,
+ [0][0][RTW89_ETSI][17] = 24,
+ [0][0][RTW89_MKK][17] = 50,
+ [0][0][RTW89_IC][17] = 54,
+ [0][0][RTW89_KCC][17] = 44,
+ [0][0][RTW89_ACMA][17] = 24,
+ [0][0][RTW89_CN][17] = 127,
+ [0][0][RTW89_UK][17] = 24,
+ [0][0][RTW89_FCC][19] = 54,
+ [0][0][RTW89_ETSI][19] = 24,
+ [0][0][RTW89_MKK][19] = 50,
+ [0][0][RTW89_IC][19] = 54,
+ [0][0][RTW89_KCC][19] = 44,
+ [0][0][RTW89_ACMA][19] = 24,
+ [0][0][RTW89_CN][19] = 127,
+ [0][0][RTW89_UK][19] = 24,
+ [0][0][RTW89_FCC][21] = 54,
+ [0][0][RTW89_ETSI][21] = 24,
+ [0][0][RTW89_MKK][21] = 50,
+ [0][0][RTW89_IC][21] = 54,
+ [0][0][RTW89_KCC][21] = 44,
+ [0][0][RTW89_ACMA][21] = 24,
+ [0][0][RTW89_CN][21] = 127,
+ [0][0][RTW89_UK][21] = 24,
+ [0][0][RTW89_FCC][23] = 54,
+ [0][0][RTW89_ETSI][23] = 24,
+ [0][0][RTW89_MKK][23] = 50,
+ [0][0][RTW89_IC][23] = 54,
+ [0][0][RTW89_KCC][23] = 44,
+ [0][0][RTW89_ACMA][23] = 24,
+ [0][0][RTW89_CN][23] = 127,
+ [0][0][RTW89_UK][23] = 24,
+ [0][0][RTW89_FCC][25] = 54,
+ [0][0][RTW89_ETSI][25] = 24,
+ [0][0][RTW89_MKK][25] = 50,
+ [0][0][RTW89_IC][25] = 127,
+ [0][0][RTW89_KCC][25] = 44,
+ [0][0][RTW89_ACMA][25] = 127,
+ [0][0][RTW89_CN][25] = 127,
+ [0][0][RTW89_UK][25] = 24,
+ [0][0][RTW89_FCC][27] = 54,
+ [0][0][RTW89_ETSI][27] = 24,
+ [0][0][RTW89_MKK][27] = 50,
+ [0][0][RTW89_IC][27] = 127,
+ [0][0][RTW89_KCC][27] = 42,
+ [0][0][RTW89_ACMA][27] = 127,
+ [0][0][RTW89_CN][27] = 127,
+ [0][0][RTW89_UK][27] = 24,
+ [0][0][RTW89_FCC][29] = 54,
+ [0][0][RTW89_ETSI][29] = 24,
+ [0][0][RTW89_MKK][29] = 50,
+ [0][0][RTW89_IC][29] = 127,
+ [0][0][RTW89_KCC][29] = 42,
+ [0][0][RTW89_ACMA][29] = 127,
+ [0][0][RTW89_CN][29] = 127,
+ [0][0][RTW89_UK][29] = 24,
+ [0][0][RTW89_FCC][31] = 54,
+ [0][0][RTW89_ETSI][31] = 24,
+ [0][0][RTW89_MKK][31] = 50,
+ [0][0][RTW89_IC][31] = 56,
+ [0][0][RTW89_KCC][31] = 42,
+ [0][0][RTW89_ACMA][31] = 24,
+ [0][0][RTW89_CN][31] = 127,
+ [0][0][RTW89_UK][31] = 24,
+ [0][0][RTW89_FCC][33] = 56,
+ [0][0][RTW89_ETSI][33] = 24,
+ [0][0][RTW89_MKK][33] = 50,
+ [0][0][RTW89_IC][33] = 56,
+ [0][0][RTW89_KCC][33] = 42,
+ [0][0][RTW89_ACMA][33] = 24,
+ [0][0][RTW89_CN][33] = 127,
+ [0][0][RTW89_UK][33] = 24,
+ [0][0][RTW89_FCC][35] = 56,
+ [0][0][RTW89_ETSI][35] = 24,
+ [0][0][RTW89_MKK][35] = 50,
+ [0][0][RTW89_IC][35] = 56,
+ [0][0][RTW89_KCC][35] = 42,
+ [0][0][RTW89_ACMA][35] = 24,
+ [0][0][RTW89_CN][35] = 127,
+ [0][0][RTW89_UK][35] = 24,
+ [0][0][RTW89_FCC][37] = 86,
+ [0][0][RTW89_ETSI][37] = 127,
+ [0][0][RTW89_MKK][37] = 46,
+ [0][0][RTW89_IC][37] = 86,
+ [0][0][RTW89_KCC][37] = 44,
+ [0][0][RTW89_ACMA][37] = 50,
+ [0][0][RTW89_CN][37] = 127,
+ [0][0][RTW89_UK][37] = 52,
+ [0][0][RTW89_FCC][38] = 68,
+ [0][0][RTW89_ETSI][38] = 28,
+ [0][0][RTW89_MKK][38] = 127,
+ [0][0][RTW89_IC][38] = 68,
+ [0][0][RTW89_KCC][38] = 44,
+ [0][0][RTW89_ACMA][38] = 84,
+ [0][0][RTW89_CN][38] = 54,
+ [0][0][RTW89_UK][38] = 24,
+ [0][0][RTW89_FCC][40] = 68,
+ [0][0][RTW89_ETSI][40] = 28,
+ [0][0][RTW89_MKK][40] = 127,
+ [0][0][RTW89_IC][40] = 68,
+ [0][0][RTW89_KCC][40] = 44,
+ [0][0][RTW89_ACMA][40] = 84,
+ [0][0][RTW89_CN][40] = 54,
+ [0][0][RTW89_UK][40] = 24,
+ [0][0][RTW89_FCC][42] = 70,
+ [0][0][RTW89_ETSI][42] = 28,
+ [0][0][RTW89_MKK][42] = 127,
+ [0][0][RTW89_IC][42] = 70,
+ [0][0][RTW89_KCC][42] = 44,
+ [0][0][RTW89_ACMA][42] = 84,
+ [0][0][RTW89_CN][42] = 54,
+ [0][0][RTW89_UK][42] = 24,
+ [0][0][RTW89_FCC][44] = 62,
+ [0][0][RTW89_ETSI][44] = 28,
+ [0][0][RTW89_MKK][44] = 127,
+ [0][0][RTW89_IC][44] = 62,
+ [0][0][RTW89_KCC][44] = 44,
+ [0][0][RTW89_ACMA][44] = 84,
+ [0][0][RTW89_CN][44] = 54,
+ [0][0][RTW89_UK][44] = 24,
+ [0][0][RTW89_FCC][46] = 62,
+ [0][0][RTW89_ETSI][46] = 28,
+ [0][0][RTW89_MKK][46] = 127,
+ [0][0][RTW89_IC][46] = 62,
+ [0][0][RTW89_KCC][46] = 44,
+ [0][0][RTW89_ACMA][46] = 84,
+ [0][0][RTW89_CN][46] = 54,
+ [0][0][RTW89_UK][46] = 24,
+ [0][0][RTW89_FCC][48] = 42,
+ [0][0][RTW89_ETSI][48] = 127,
+ [0][0][RTW89_MKK][48] = 127,
+ [0][0][RTW89_IC][48] = 127,
+ [0][0][RTW89_KCC][48] = 127,
+ [0][0][RTW89_ACMA][48] = 127,
+ [0][0][RTW89_CN][48] = 127,
+ [0][0][RTW89_UK][48] = 127,
+ [0][0][RTW89_FCC][50] = 42,
+ [0][0][RTW89_ETSI][50] = 127,
+ [0][0][RTW89_MKK][50] = 127,
+ [0][0][RTW89_IC][50] = 127,
+ [0][0][RTW89_KCC][50] = 127,
+ [0][0][RTW89_ACMA][50] = 127,
+ [0][0][RTW89_CN][50] = 127,
+ [0][0][RTW89_UK][50] = 127,
+ [0][0][RTW89_FCC][52] = 40,
+ [0][0][RTW89_ETSI][52] = 127,
+ [0][0][RTW89_MKK][52] = 127,
+ [0][0][RTW89_IC][52] = 127,
+ [0][0][RTW89_KCC][52] = 127,
+ [0][0][RTW89_ACMA][52] = 127,
+ [0][0][RTW89_CN][52] = 127,
+ [0][0][RTW89_UK][52] = 127,
+ [0][1][RTW89_FCC][0] = 127,
+ [0][1][RTW89_ETSI][0] = 127,
+ [0][1][RTW89_MKK][0] = 127,
+ [0][1][RTW89_IC][0] = 127,
+ [0][1][RTW89_KCC][0] = 127,
+ [0][1][RTW89_ACMA][0] = 127,
+ [0][1][RTW89_CN][0] = 4,
+ [0][1][RTW89_UK][0] = 127,
+ [0][1][RTW89_FCC][2] = 127,
+ [0][1][RTW89_ETSI][2] = 127,
+ [0][1][RTW89_MKK][2] = 127,
+ [0][1][RTW89_IC][2] = 127,
+ [0][1][RTW89_KCC][2] = 127,
+ [0][1][RTW89_ACMA][2] = 127,
+ [0][1][RTW89_CN][2] = 4,
+ [0][1][RTW89_UK][2] = 127,
+ [0][1][RTW89_FCC][4] = 127,
+ [0][1][RTW89_ETSI][4] = 127,
+ [0][1][RTW89_MKK][4] = 127,
+ [0][1][RTW89_IC][4] = 127,
+ [0][1][RTW89_KCC][4] = 127,
+ [0][1][RTW89_ACMA][4] = 127,
+ [0][1][RTW89_CN][4] = 4,
+ [0][1][RTW89_UK][4] = 127,
+ [0][1][RTW89_FCC][6] = 127,
+ [0][1][RTW89_ETSI][6] = 127,
+ [0][1][RTW89_MKK][6] = 127,
+ [0][1][RTW89_IC][6] = 127,
+ [0][1][RTW89_KCC][6] = 127,
+ [0][1][RTW89_ACMA][6] = 127,
+ [0][1][RTW89_CN][6] = 4,
+ [0][1][RTW89_UK][6] = 127,
+ [0][1][RTW89_FCC][8] = 127,
+ [0][1][RTW89_ETSI][8] = 127,
+ [0][1][RTW89_MKK][8] = 127,
+ [0][1][RTW89_IC][8] = 127,
+ [0][1][RTW89_KCC][8] = 127,
+ [0][1][RTW89_ACMA][8] = 127,
+ [0][1][RTW89_CN][8] = 4,
+ [0][1][RTW89_UK][8] = 127,
+ [0][1][RTW89_FCC][10] = 127,
+ [0][1][RTW89_ETSI][10] = 127,
+ [0][1][RTW89_MKK][10] = 127,
+ [0][1][RTW89_IC][10] = 127,
+ [0][1][RTW89_KCC][10] = 127,
+ [0][1][RTW89_ACMA][10] = 127,
+ [0][1][RTW89_CN][10] = 4,
+ [0][1][RTW89_UK][10] = 127,
+ [0][1][RTW89_FCC][12] = 127,
+ [0][1][RTW89_ETSI][12] = 127,
+ [0][1][RTW89_MKK][12] = 127,
+ [0][1][RTW89_IC][12] = 127,
+ [0][1][RTW89_KCC][12] = 127,
+ [0][1][RTW89_ACMA][12] = 127,
+ [0][1][RTW89_CN][12] = 4,
+ [0][1][RTW89_UK][12] = 127,
+ [0][1][RTW89_FCC][14] = 127,
+ [0][1][RTW89_ETSI][14] = 127,
+ [0][1][RTW89_MKK][14] = 127,
+ [0][1][RTW89_IC][14] = 127,
+ [0][1][RTW89_KCC][14] = 127,
+ [0][1][RTW89_ACMA][14] = 127,
+ [0][1][RTW89_CN][14] = 4,
+ [0][1][RTW89_UK][14] = 127,
+ [0][1][RTW89_FCC][15] = 127,
+ [0][1][RTW89_ETSI][15] = 127,
+ [0][1][RTW89_MKK][15] = 127,
+ [0][1][RTW89_IC][15] = 127,
+ [0][1][RTW89_KCC][15] = 127,
+ [0][1][RTW89_ACMA][15] = 127,
+ [0][1][RTW89_CN][15] = 127,
+ [0][1][RTW89_UK][15] = 127,
+ [0][1][RTW89_FCC][17] = 127,
+ [0][1][RTW89_ETSI][17] = 127,
+ [0][1][RTW89_MKK][17] = 127,
+ [0][1][RTW89_IC][17] = 127,
+ [0][1][RTW89_KCC][17] = 127,
+ [0][1][RTW89_ACMA][17] = 127,
+ [0][1][RTW89_CN][17] = 127,
+ [0][1][RTW89_UK][17] = 127,
+ [0][1][RTW89_FCC][19] = 127,
+ [0][1][RTW89_ETSI][19] = 127,
+ [0][1][RTW89_MKK][19] = 127,
+ [0][1][RTW89_IC][19] = 127,
+ [0][1][RTW89_KCC][19] = 127,
+ [0][1][RTW89_ACMA][19] = 127,
+ [0][1][RTW89_CN][19] = 127,
+ [0][1][RTW89_UK][19] = 127,
+ [0][1][RTW89_FCC][21] = 127,
+ [0][1][RTW89_ETSI][21] = 127,
+ [0][1][RTW89_MKK][21] = 127,
+ [0][1][RTW89_IC][21] = 127,
+ [0][1][RTW89_KCC][21] = 127,
+ [0][1][RTW89_ACMA][21] = 127,
+ [0][1][RTW89_CN][21] = 127,
+ [0][1][RTW89_UK][21] = 127,
+ [0][1][RTW89_FCC][23] = 127,
+ [0][1][RTW89_ETSI][23] = 127,
+ [0][1][RTW89_MKK][23] = 127,
+ [0][1][RTW89_IC][23] = 127,
+ [0][1][RTW89_KCC][23] = 127,
+ [0][1][RTW89_ACMA][23] = 127,
+ [0][1][RTW89_CN][23] = 127,
+ [0][1][RTW89_UK][23] = 127,
+ [0][1][RTW89_FCC][25] = 127,
+ [0][1][RTW89_ETSI][25] = 127,
+ [0][1][RTW89_MKK][25] = 127,
+ [0][1][RTW89_IC][25] = 127,
+ [0][1][RTW89_KCC][25] = 127,
+ [0][1][RTW89_ACMA][25] = 127,
+ [0][1][RTW89_CN][25] = 127,
+ [0][1][RTW89_UK][25] = 127,
+ [0][1][RTW89_FCC][27] = 127,
+ [0][1][RTW89_ETSI][27] = 127,
+ [0][1][RTW89_MKK][27] = 127,
+ [0][1][RTW89_IC][27] = 127,
+ [0][1][RTW89_KCC][27] = 127,
+ [0][1][RTW89_ACMA][27] = 127,
+ [0][1][RTW89_CN][27] = 127,
+ [0][1][RTW89_UK][27] = 127,
+ [0][1][RTW89_FCC][29] = 127,
+ [0][1][RTW89_ETSI][29] = 127,
+ [0][1][RTW89_MKK][29] = 127,
+ [0][1][RTW89_IC][29] = 127,
+ [0][1][RTW89_KCC][29] = 127,
+ [0][1][RTW89_ACMA][29] = 127,
+ [0][1][RTW89_CN][29] = 127,
+ [0][1][RTW89_UK][29] = 127,
+ [0][1][RTW89_FCC][31] = 127,
+ [0][1][RTW89_ETSI][31] = 127,
+ [0][1][RTW89_MKK][31] = 127,
+ [0][1][RTW89_IC][31] = 127,
+ [0][1][RTW89_KCC][31] = 127,
+ [0][1][RTW89_ACMA][31] = 127,
+ [0][1][RTW89_CN][31] = 127,
+ [0][1][RTW89_UK][31] = 127,
+ [0][1][RTW89_FCC][33] = 127,
+ [0][1][RTW89_ETSI][33] = 127,
+ [0][1][RTW89_MKK][33] = 127,
+ [0][1][RTW89_IC][33] = 127,
+ [0][1][RTW89_KCC][33] = 127,
+ [0][1][RTW89_ACMA][33] = 127,
+ [0][1][RTW89_CN][33] = 127,
+ [0][1][RTW89_UK][33] = 127,
+ [0][1][RTW89_FCC][35] = 127,
+ [0][1][RTW89_ETSI][35] = 127,
+ [0][1][RTW89_MKK][35] = 127,
+ [0][1][RTW89_IC][35] = 127,
+ [0][1][RTW89_KCC][35] = 127,
+ [0][1][RTW89_ACMA][35] = 127,
+ [0][1][RTW89_CN][35] = 127,
+ [0][1][RTW89_UK][35] = 127,
+ [0][1][RTW89_FCC][37] = 127,
+ [0][1][RTW89_ETSI][37] = 127,
+ [0][1][RTW89_MKK][37] = 127,
+ [0][1][RTW89_IC][37] = 127,
+ [0][1][RTW89_KCC][37] = 127,
+ [0][1][RTW89_ACMA][37] = 127,
+ [0][1][RTW89_CN][37] = 127,
+ [0][1][RTW89_UK][37] = 127,
+ [0][1][RTW89_FCC][38] = 127,
+ [0][1][RTW89_ETSI][38] = 127,
+ [0][1][RTW89_MKK][38] = 127,
+ [0][1][RTW89_IC][38] = 127,
+ [0][1][RTW89_KCC][38] = 127,
+ [0][1][RTW89_ACMA][38] = 127,
+ [0][1][RTW89_CN][38] = 42,
+ [0][1][RTW89_UK][38] = 127,
+ [0][1][RTW89_FCC][40] = 127,
+ [0][1][RTW89_ETSI][40] = 127,
+ [0][1][RTW89_MKK][40] = 127,
+ [0][1][RTW89_IC][40] = 127,
+ [0][1][RTW89_KCC][40] = 127,
+ [0][1][RTW89_ACMA][40] = 127,
+ [0][1][RTW89_CN][40] = 42,
+ [0][1][RTW89_UK][40] = 127,
+ [0][1][RTW89_FCC][42] = 127,
+ [0][1][RTW89_ETSI][42] = 127,
+ [0][1][RTW89_MKK][42] = 127,
+ [0][1][RTW89_IC][42] = 127,
+ [0][1][RTW89_KCC][42] = 127,
+ [0][1][RTW89_ACMA][42] = 127,
+ [0][1][RTW89_CN][42] = 42,
+ [0][1][RTW89_UK][42] = 127,
+ [0][1][RTW89_FCC][44] = 127,
+ [0][1][RTW89_ETSI][44] = 127,
+ [0][1][RTW89_MKK][44] = 127,
+ [0][1][RTW89_IC][44] = 127,
+ [0][1][RTW89_KCC][44] = 127,
+ [0][1][RTW89_ACMA][44] = 127,
+ [0][1][RTW89_CN][44] = 42,
+ [0][1][RTW89_UK][44] = 127,
+ [0][1][RTW89_FCC][46] = 127,
+ [0][1][RTW89_ETSI][46] = 127,
+ [0][1][RTW89_MKK][46] = 127,
+ [0][1][RTW89_IC][46] = 127,
+ [0][1][RTW89_KCC][46] = 127,
+ [0][1][RTW89_ACMA][46] = 127,
+ [0][1][RTW89_CN][46] = 42,
+ [0][1][RTW89_UK][46] = 127,
+ [0][1][RTW89_FCC][48] = 127,
+ [0][1][RTW89_ETSI][48] = 127,
+ [0][1][RTW89_MKK][48] = 127,
+ [0][1][RTW89_IC][48] = 127,
+ [0][1][RTW89_KCC][48] = 127,
+ [0][1][RTW89_ACMA][48] = 127,
+ [0][1][RTW89_CN][48] = 127,
+ [0][1][RTW89_UK][48] = 127,
+ [0][1][RTW89_FCC][50] = 127,
+ [0][1][RTW89_ETSI][50] = 127,
+ [0][1][RTW89_MKK][50] = 127,
+ [0][1][RTW89_IC][50] = 127,
+ [0][1][RTW89_KCC][50] = 127,
+ [0][1][RTW89_ACMA][50] = 127,
+ [0][1][RTW89_CN][50] = 127,
+ [0][1][RTW89_UK][50] = 127,
+ [0][1][RTW89_FCC][52] = 127,
+ [0][1][RTW89_ETSI][52] = 127,
+ [0][1][RTW89_MKK][52] = 127,
+ [0][1][RTW89_IC][52] = 127,
+ [0][1][RTW89_KCC][52] = 127,
+ [0][1][RTW89_ACMA][52] = 127,
+ [0][1][RTW89_CN][52] = 127,
+ [0][1][RTW89_UK][52] = 127,
+ [1][0][RTW89_FCC][0] = 64,
+ [1][0][RTW89_ETSI][0] = 34,
+ [1][0][RTW89_MKK][0] = 38,
+ [1][0][RTW89_IC][0] = 38,
+ [1][0][RTW89_KCC][0] = 52,
+ [1][0][RTW89_ACMA][0] = 34,
+ [1][0][RTW89_CN][0] = 26,
+ [1][0][RTW89_UK][0] = 34,
+ [1][0][RTW89_FCC][2] = 66,
+ [1][0][RTW89_ETSI][2] = 34,
+ [1][0][RTW89_MKK][2] = 38,
+ [1][0][RTW89_IC][2] = 38,
+ [1][0][RTW89_KCC][2] = 52,
+ [1][0][RTW89_ACMA][2] = 34,
+ [1][0][RTW89_CN][2] = 26,
+ [1][0][RTW89_UK][2] = 34,
+ [1][0][RTW89_FCC][4] = 62,
+ [1][0][RTW89_ETSI][4] = 34,
+ [1][0][RTW89_MKK][4] = 36,
+ [1][0][RTW89_IC][4] = 38,
+ [1][0][RTW89_KCC][4] = 52,
+ [1][0][RTW89_ACMA][4] = 34,
+ [1][0][RTW89_CN][4] = 26,
+ [1][0][RTW89_UK][4] = 34,
+ [1][0][RTW89_FCC][6] = 62,
+ [1][0][RTW89_ETSI][6] = 34,
+ [1][0][RTW89_MKK][6] = 36,
+ [1][0][RTW89_IC][6] = 38,
+ [1][0][RTW89_KCC][6] = 32,
+ [1][0][RTW89_ACMA][6] = 34,
+ [1][0][RTW89_CN][6] = 26,
+ [1][0][RTW89_UK][6] = 34,
+ [1][0][RTW89_FCC][8] = 62,
+ [1][0][RTW89_ETSI][8] = 34,
+ [1][0][RTW89_MKK][8] = 38,
+ [1][0][RTW89_IC][8] = 62,
+ [1][0][RTW89_KCC][8] = 52,
+ [1][0][RTW89_ACMA][8] = 34,
+ [1][0][RTW89_CN][8] = 26,
+ [1][0][RTW89_UK][8] = 34,
+ [1][0][RTW89_FCC][10] = 62,
+ [1][0][RTW89_ETSI][10] = 34,
+ [1][0][RTW89_MKK][10] = 38,
+ [1][0][RTW89_IC][10] = 62,
+ [1][0][RTW89_KCC][10] = 52,
+ [1][0][RTW89_ACMA][10] = 34,
+ [1][0][RTW89_CN][10] = 26,
+ [1][0][RTW89_UK][10] = 34,
+ [1][0][RTW89_FCC][12] = 62,
+ [1][0][RTW89_ETSI][12] = 34,
+ [1][0][RTW89_MKK][12] = 38,
+ [1][0][RTW89_IC][12] = 62,
+ [1][0][RTW89_KCC][12] = 54,
+ [1][0][RTW89_ACMA][12] = 34,
+ [1][0][RTW89_CN][12] = 26,
+ [1][0][RTW89_UK][12] = 34,
+ [1][0][RTW89_FCC][14] = 64,
+ [1][0][RTW89_ETSI][14] = 34,
+ [1][0][RTW89_MKK][14] = 38,
+ [1][0][RTW89_IC][14] = 64,
+ [1][0][RTW89_KCC][14] = 54,
+ [1][0][RTW89_ACMA][14] = 34,
+ [1][0][RTW89_CN][14] = 26,
+ [1][0][RTW89_UK][14] = 34,
+ [1][0][RTW89_FCC][15] = 62,
+ [1][0][RTW89_ETSI][15] = 34,
+ [1][0][RTW89_MKK][15] = 58,
+ [1][0][RTW89_IC][15] = 62,
+ [1][0][RTW89_KCC][15] = 54,
+ [1][0][RTW89_ACMA][15] = 34,
+ [1][0][RTW89_CN][15] = 127,
+ [1][0][RTW89_UK][15] = 34,
+ [1][0][RTW89_FCC][17] = 62,
+ [1][0][RTW89_ETSI][17] = 34,
+ [1][0][RTW89_MKK][17] = 58,
+ [1][0][RTW89_IC][17] = 62,
+ [1][0][RTW89_KCC][17] = 54,
+ [1][0][RTW89_ACMA][17] = 34,
+ [1][0][RTW89_CN][17] = 127,
+ [1][0][RTW89_UK][17] = 34,
+ [1][0][RTW89_FCC][19] = 64,
+ [1][0][RTW89_ETSI][19] = 34,
+ [1][0][RTW89_MKK][19] = 58,
+ [1][0][RTW89_IC][19] = 64,
+ [1][0][RTW89_KCC][19] = 54,
+ [1][0][RTW89_ACMA][19] = 34,
+ [1][0][RTW89_CN][19] = 127,
+ [1][0][RTW89_UK][19] = 34,
+ [1][0][RTW89_FCC][21] = 64,
+ [1][0][RTW89_ETSI][21] = 34,
+ [1][0][RTW89_MKK][21] = 58,
+ [1][0][RTW89_IC][21] = 64,
+ [1][0][RTW89_KCC][21] = 54,
+ [1][0][RTW89_ACMA][21] = 34,
+ [1][0][RTW89_CN][21] = 127,
+ [1][0][RTW89_UK][21] = 34,
+ [1][0][RTW89_FCC][23] = 64,
+ [1][0][RTW89_ETSI][23] = 34,
+ [1][0][RTW89_MKK][23] = 58,
+ [1][0][RTW89_IC][23] = 64,
+ [1][0][RTW89_KCC][23] = 54,
+ [1][0][RTW89_ACMA][23] = 34,
+ [1][0][RTW89_CN][23] = 127,
+ [1][0][RTW89_UK][23] = 34,
+ [1][0][RTW89_FCC][25] = 64,
+ [1][0][RTW89_ETSI][25] = 34,
+ [1][0][RTW89_MKK][25] = 58,
+ [1][0][RTW89_IC][25] = 127,
+ [1][0][RTW89_KCC][25] = 54,
+ [1][0][RTW89_ACMA][25] = 127,
+ [1][0][RTW89_CN][25] = 127,
+ [1][0][RTW89_UK][25] = 34,
+ [1][0][RTW89_FCC][27] = 64,
+ [1][0][RTW89_ETSI][27] = 34,
+ [1][0][RTW89_MKK][27] = 58,
+ [1][0][RTW89_IC][27] = 127,
+ [1][0][RTW89_KCC][27] = 54,
+ [1][0][RTW89_ACMA][27] = 127,
+ [1][0][RTW89_CN][27] = 127,
+ [1][0][RTW89_UK][27] = 34,
+ [1][0][RTW89_FCC][29] = 64,
+ [1][0][RTW89_ETSI][29] = 34,
+ [1][0][RTW89_MKK][29] = 58,
+ [1][0][RTW89_IC][29] = 127,
+ [1][0][RTW89_KCC][29] = 54,
+ [1][0][RTW89_ACMA][29] = 127,
+ [1][0][RTW89_CN][29] = 127,
+ [1][0][RTW89_UK][29] = 34,
+ [1][0][RTW89_FCC][31] = 64,
+ [1][0][RTW89_ETSI][31] = 34,
+ [1][0][RTW89_MKK][31] = 58,
+ [1][0][RTW89_IC][31] = 64,
+ [1][0][RTW89_KCC][31] = 54,
+ [1][0][RTW89_ACMA][31] = 34,
+ [1][0][RTW89_CN][31] = 127,
+ [1][0][RTW89_UK][31] = 34,
+ [1][0][RTW89_FCC][33] = 64,
+ [1][0][RTW89_ETSI][33] = 34,
+ [1][0][RTW89_MKK][33] = 58,
+ [1][0][RTW89_IC][33] = 64,
+ [1][0][RTW89_KCC][33] = 54,
+ [1][0][RTW89_ACMA][33] = 34,
+ [1][0][RTW89_CN][33] = 127,
+ [1][0][RTW89_UK][33] = 34,
+ [1][0][RTW89_FCC][35] = 64,
+ [1][0][RTW89_ETSI][35] = 34,
+ [1][0][RTW89_MKK][35] = 58,
+ [1][0][RTW89_IC][35] = 64,
+ [1][0][RTW89_KCC][35] = 54,
+ [1][0][RTW89_ACMA][35] = 34,
+ [1][0][RTW89_CN][35] = 127,
+ [1][0][RTW89_UK][35] = 34,
+ [1][0][RTW89_FCC][37] = 78,
+ [1][0][RTW89_ETSI][37] = 127,
+ [1][0][RTW89_MKK][37] = 56,
+ [1][0][RTW89_IC][37] = 78,
+ [1][0][RTW89_KCC][37] = 54,
+ [1][0][RTW89_ACMA][37] = 62,
+ [1][0][RTW89_CN][37] = 127,
+ [1][0][RTW89_UK][37] = 62,
+ [1][0][RTW89_FCC][38] = 82,
+ [1][0][RTW89_ETSI][38] = 28,
+ [1][0][RTW89_MKK][38] = 127,
+ [1][0][RTW89_IC][38] = 82,
+ [1][0][RTW89_KCC][38] = 54,
+ [1][0][RTW89_ACMA][38] = 84,
+ [1][0][RTW89_CN][38] = 66,
+ [1][0][RTW89_UK][38] = 34,
+ [1][0][RTW89_FCC][40] = 82,
+ [1][0][RTW89_ETSI][40] = 28,
+ [1][0][RTW89_MKK][40] = 127,
+ [1][0][RTW89_IC][40] = 82,
+ [1][0][RTW89_KCC][40] = 54,
+ [1][0][RTW89_ACMA][40] = 84,
+ [1][0][RTW89_CN][40] = 66,
+ [1][0][RTW89_UK][40] = 34,
+ [1][0][RTW89_FCC][42] = 78,
+ [1][0][RTW89_ETSI][42] = 28,
+ [1][0][RTW89_MKK][42] = 127,
+ [1][0][RTW89_IC][42] = 78,
+ [1][0][RTW89_KCC][42] = 54,
+ [1][0][RTW89_ACMA][42] = 84,
+ [1][0][RTW89_CN][42] = 66,
+ [1][0][RTW89_UK][42] = 34,
+ [1][0][RTW89_FCC][44] = 82,
+ [1][0][RTW89_ETSI][44] = 28,
+ [1][0][RTW89_MKK][44] = 127,
+ [1][0][RTW89_IC][44] = 82,
+ [1][0][RTW89_KCC][44] = 54,
+ [1][0][RTW89_ACMA][44] = 84,
+ [1][0][RTW89_CN][44] = 66,
+ [1][0][RTW89_UK][44] = 34,
+ [1][0][RTW89_FCC][46] = 82,
+ [1][0][RTW89_ETSI][46] = 28,
+ [1][0][RTW89_MKK][46] = 127,
+ [1][0][RTW89_IC][46] = 82,
+ [1][0][RTW89_KCC][46] = 54,
+ [1][0][RTW89_ACMA][46] = 84,
+ [1][0][RTW89_CN][46] = 66,
+ [1][0][RTW89_UK][46] = 34,
+ [1][0][RTW89_FCC][48] = 52,
+ [1][0][RTW89_ETSI][48] = 127,
+ [1][0][RTW89_MKK][48] = 127,
+ [1][0][RTW89_IC][48] = 127,
+ [1][0][RTW89_KCC][48] = 127,
+ [1][0][RTW89_ACMA][48] = 127,
+ [1][0][RTW89_CN][48] = 127,
+ [1][0][RTW89_UK][48] = 127,
+ [1][0][RTW89_FCC][50] = 52,
+ [1][0][RTW89_ETSI][50] = 127,
+ [1][0][RTW89_MKK][50] = 127,
+ [1][0][RTW89_IC][50] = 127,
+ [1][0][RTW89_KCC][50] = 127,
+ [1][0][RTW89_ACMA][50] = 127,
+ [1][0][RTW89_CN][50] = 127,
+ [1][0][RTW89_UK][50] = 127,
+ [1][0][RTW89_FCC][52] = 52,
+ [1][0][RTW89_ETSI][52] = 127,
+ [1][0][RTW89_MKK][52] = 127,
+ [1][0][RTW89_IC][52] = 127,
+ [1][0][RTW89_KCC][52] = 127,
+ [1][0][RTW89_ACMA][52] = 127,
+ [1][0][RTW89_CN][52] = 127,
+ [1][0][RTW89_UK][52] = 127,
+ [1][1][RTW89_FCC][0] = 127,
+ [1][1][RTW89_ETSI][0] = 127,
+ [1][1][RTW89_MKK][0] = 127,
+ [1][1][RTW89_IC][0] = 127,
+ [1][1][RTW89_KCC][0] = 127,
+ [1][1][RTW89_ACMA][0] = 127,
+ [1][1][RTW89_CN][0] = 14,
+ [1][1][RTW89_UK][0] = 127,
+ [1][1][RTW89_FCC][2] = 127,
+ [1][1][RTW89_ETSI][2] = 127,
+ [1][1][RTW89_MKK][2] = 127,
+ [1][1][RTW89_IC][2] = 127,
+ [1][1][RTW89_KCC][2] = 127,
+ [1][1][RTW89_ACMA][2] = 127,
+ [1][1][RTW89_CN][2] = 14,
+ [1][1][RTW89_UK][2] = 127,
+ [1][1][RTW89_FCC][4] = 127,
+ [1][1][RTW89_ETSI][4] = 127,
+ [1][1][RTW89_MKK][4] = 127,
+ [1][1][RTW89_IC][4] = 127,
+ [1][1][RTW89_KCC][4] = 127,
+ [1][1][RTW89_ACMA][4] = 127,
+ [1][1][RTW89_CN][4] = 14,
+ [1][1][RTW89_UK][4] = 127,
+ [1][1][RTW89_FCC][6] = 127,
+ [1][1][RTW89_ETSI][6] = 127,
+ [1][1][RTW89_MKK][6] = 127,
+ [1][1][RTW89_IC][6] = 127,
+ [1][1][RTW89_KCC][6] = 127,
+ [1][1][RTW89_ACMA][6] = 127,
+ [1][1][RTW89_CN][6] = 14,
+ [1][1][RTW89_UK][6] = 127,
+ [1][1][RTW89_FCC][8] = 127,
+ [1][1][RTW89_ETSI][8] = 127,
+ [1][1][RTW89_MKK][8] = 127,
+ [1][1][RTW89_IC][8] = 127,
+ [1][1][RTW89_KCC][8] = 127,
+ [1][1][RTW89_ACMA][8] = 127,
+ [1][1][RTW89_CN][8] = 14,
+ [1][1][RTW89_UK][8] = 127,
+ [1][1][RTW89_FCC][10] = 127,
+ [1][1][RTW89_ETSI][10] = 127,
+ [1][1][RTW89_MKK][10] = 127,
+ [1][1][RTW89_IC][10] = 127,
+ [1][1][RTW89_KCC][10] = 127,
+ [1][1][RTW89_ACMA][10] = 127,
+ [1][1][RTW89_CN][10] = 14,
+ [1][1][RTW89_UK][10] = 127,
+ [1][1][RTW89_FCC][12] = 127,
+ [1][1][RTW89_ETSI][12] = 127,
+ [1][1][RTW89_MKK][12] = 127,
+ [1][1][RTW89_IC][12] = 127,
+ [1][1][RTW89_KCC][12] = 127,
+ [1][1][RTW89_ACMA][12] = 127,
+ [1][1][RTW89_CN][12] = 14,
+ [1][1][RTW89_UK][12] = 127,
+ [1][1][RTW89_FCC][14] = 127,
+ [1][1][RTW89_ETSI][14] = 127,
+ [1][1][RTW89_MKK][14] = 127,
+ [1][1][RTW89_IC][14] = 127,
+ [1][1][RTW89_KCC][14] = 127,
+ [1][1][RTW89_ACMA][14] = 127,
+ [1][1][RTW89_CN][14] = 14,
+ [1][1][RTW89_UK][14] = 127,
+ [1][1][RTW89_FCC][15] = 127,
+ [1][1][RTW89_ETSI][15] = 127,
+ [1][1][RTW89_MKK][15] = 127,
+ [1][1][RTW89_IC][15] = 127,
+ [1][1][RTW89_KCC][15] = 127,
+ [1][1][RTW89_ACMA][15] = 127,
+ [1][1][RTW89_CN][15] = 127,
+ [1][1][RTW89_UK][15] = 127,
+ [1][1][RTW89_FCC][17] = 127,
+ [1][1][RTW89_ETSI][17] = 127,
+ [1][1][RTW89_MKK][17] = 127,
+ [1][1][RTW89_IC][17] = 127,
+ [1][1][RTW89_KCC][17] = 127,
+ [1][1][RTW89_ACMA][17] = 127,
+ [1][1][RTW89_CN][17] = 127,
+ [1][1][RTW89_UK][17] = 127,
+ [1][1][RTW89_FCC][19] = 127,
+ [1][1][RTW89_ETSI][19] = 127,
+ [1][1][RTW89_MKK][19] = 127,
+ [1][1][RTW89_IC][19] = 127,
+ [1][1][RTW89_KCC][19] = 127,
+ [1][1][RTW89_ACMA][19] = 127,
+ [1][1][RTW89_CN][19] = 127,
+ [1][1][RTW89_UK][19] = 127,
+ [1][1][RTW89_FCC][21] = 127,
+ [1][1][RTW89_ETSI][21] = 127,
+ [1][1][RTW89_MKK][21] = 127,
+ [1][1][RTW89_IC][21] = 127,
+ [1][1][RTW89_KCC][21] = 127,
+ [1][1][RTW89_ACMA][21] = 127,
+ [1][1][RTW89_CN][21] = 127,
+ [1][1][RTW89_UK][21] = 127,
+ [1][1][RTW89_FCC][23] = 127,
+ [1][1][RTW89_ETSI][23] = 127,
+ [1][1][RTW89_MKK][23] = 127,
+ [1][1][RTW89_IC][23] = 127,
+ [1][1][RTW89_KCC][23] = 127,
+ [1][1][RTW89_ACMA][23] = 127,
+ [1][1][RTW89_CN][23] = 127,
+ [1][1][RTW89_UK][23] = 127,
+ [1][1][RTW89_FCC][25] = 127,
+ [1][1][RTW89_ETSI][25] = 127,
+ [1][1][RTW89_MKK][25] = 127,
+ [1][1][RTW89_IC][25] = 127,
+ [1][1][RTW89_KCC][25] = 127,
+ [1][1][RTW89_ACMA][25] = 127,
+ [1][1][RTW89_CN][25] = 127,
+ [1][1][RTW89_UK][25] = 127,
+ [1][1][RTW89_FCC][27] = 127,
+ [1][1][RTW89_ETSI][27] = 127,
+ [1][1][RTW89_MKK][27] = 127,
+ [1][1][RTW89_IC][27] = 127,
+ [1][1][RTW89_KCC][27] = 127,
+ [1][1][RTW89_ACMA][27] = 127,
+ [1][1][RTW89_CN][27] = 127,
+ [1][1][RTW89_UK][27] = 127,
+ [1][1][RTW89_FCC][29] = 127,
+ [1][1][RTW89_ETSI][29] = 127,
+ [1][1][RTW89_MKK][29] = 127,
+ [1][1][RTW89_IC][29] = 127,
+ [1][1][RTW89_KCC][29] = 127,
+ [1][1][RTW89_ACMA][29] = 127,
+ [1][1][RTW89_CN][29] = 127,
+ [1][1][RTW89_UK][29] = 127,
+ [1][1][RTW89_FCC][31] = 127,
+ [1][1][RTW89_ETSI][31] = 127,
+ [1][1][RTW89_MKK][31] = 127,
+ [1][1][RTW89_IC][31] = 127,
+ [1][1][RTW89_KCC][31] = 127,
+ [1][1][RTW89_ACMA][31] = 127,
+ [1][1][RTW89_CN][31] = 127,
+ [1][1][RTW89_UK][31] = 127,
+ [1][1][RTW89_FCC][33] = 127,
+ [1][1][RTW89_ETSI][33] = 127,
+ [1][1][RTW89_MKK][33] = 127,
+ [1][1][RTW89_IC][33] = 127,
+ [1][1][RTW89_KCC][33] = 127,
+ [1][1][RTW89_ACMA][33] = 127,
+ [1][1][RTW89_CN][33] = 127,
+ [1][1][RTW89_UK][33] = 127,
+ [1][1][RTW89_FCC][35] = 127,
+ [1][1][RTW89_ETSI][35] = 127,
+ [1][1][RTW89_MKK][35] = 127,
+ [1][1][RTW89_IC][35] = 127,
+ [1][1][RTW89_KCC][35] = 127,
+ [1][1][RTW89_ACMA][35] = 127,
+ [1][1][RTW89_CN][35] = 127,
+ [1][1][RTW89_UK][35] = 127,
+ [1][1][RTW89_FCC][37] = 127,
+ [1][1][RTW89_ETSI][37] = 127,
+ [1][1][RTW89_MKK][37] = 127,
+ [1][1][RTW89_IC][37] = 127,
+ [1][1][RTW89_KCC][37] = 127,
+ [1][1][RTW89_ACMA][37] = 127,
+ [1][1][RTW89_CN][37] = 127,
+ [1][1][RTW89_UK][37] = 127,
+ [1][1][RTW89_FCC][38] = 127,
+ [1][1][RTW89_ETSI][38] = 127,
+ [1][1][RTW89_MKK][38] = 127,
+ [1][1][RTW89_IC][38] = 127,
+ [1][1][RTW89_KCC][38] = 127,
+ [1][1][RTW89_ACMA][38] = 127,
+ [1][1][RTW89_CN][38] = 54,
+ [1][1][RTW89_UK][38] = 127,
+ [1][1][RTW89_FCC][40] = 127,
+ [1][1][RTW89_ETSI][40] = 127,
+ [1][1][RTW89_MKK][40] = 127,
+ [1][1][RTW89_IC][40] = 127,
+ [1][1][RTW89_KCC][40] = 127,
+ [1][1][RTW89_ACMA][40] = 127,
+ [1][1][RTW89_CN][40] = 54,
+ [1][1][RTW89_UK][40] = 127,
+ [1][1][RTW89_FCC][42] = 127,
+ [1][1][RTW89_ETSI][42] = 127,
+ [1][1][RTW89_MKK][42] = 127,
+ [1][1][RTW89_IC][42] = 127,
+ [1][1][RTW89_KCC][42] = 127,
+ [1][1][RTW89_ACMA][42] = 127,
+ [1][1][RTW89_CN][42] = 54,
+ [1][1][RTW89_UK][42] = 127,
+ [1][1][RTW89_FCC][44] = 127,
+ [1][1][RTW89_ETSI][44] = 127,
+ [1][1][RTW89_MKK][44] = 127,
+ [1][1][RTW89_IC][44] = 127,
+ [1][1][RTW89_KCC][44] = 127,
+ [1][1][RTW89_ACMA][44] = 127,
+ [1][1][RTW89_CN][44] = 54,
+ [1][1][RTW89_UK][44] = 127,
+ [1][1][RTW89_FCC][46] = 127,
+ [1][1][RTW89_ETSI][46] = 127,
+ [1][1][RTW89_MKK][46] = 127,
+ [1][1][RTW89_IC][46] = 127,
+ [1][1][RTW89_KCC][46] = 127,
+ [1][1][RTW89_ACMA][46] = 127,
+ [1][1][RTW89_CN][46] = 54,
+ [1][1][RTW89_UK][46] = 127,
+ [1][1][RTW89_FCC][48] = 127,
+ [1][1][RTW89_ETSI][48] = 127,
+ [1][1][RTW89_MKK][48] = 127,
+ [1][1][RTW89_IC][48] = 127,
+ [1][1][RTW89_KCC][48] = 127,
+ [1][1][RTW89_ACMA][48] = 127,
+ [1][1][RTW89_CN][48] = 127,
+ [1][1][RTW89_UK][48] = 127,
+ [1][1][RTW89_FCC][50] = 127,
+ [1][1][RTW89_ETSI][50] = 127,
+ [1][1][RTW89_MKK][50] = 127,
+ [1][1][RTW89_IC][50] = 127,
+ [1][1][RTW89_KCC][50] = 127,
+ [1][1][RTW89_ACMA][50] = 127,
+ [1][1][RTW89_CN][50] = 127,
+ [1][1][RTW89_UK][50] = 127,
+ [1][1][RTW89_FCC][52] = 127,
+ [1][1][RTW89_ETSI][52] = 127,
+ [1][1][RTW89_MKK][52] = 127,
+ [1][1][RTW89_IC][52] = 127,
+ [1][1][RTW89_KCC][52] = 127,
+ [1][1][RTW89_ACMA][52] = 127,
+ [1][1][RTW89_CN][52] = 127,
+ [1][1][RTW89_UK][52] = 127,
+ [2][0][RTW89_FCC][0] = 78,
+ [2][0][RTW89_ETSI][0] = 46,
+ [2][0][RTW89_MKK][0] = 48,
+ [2][0][RTW89_IC][0] = 50,
+ [2][0][RTW89_KCC][0] = 64,
+ [2][0][RTW89_ACMA][0] = 46,
+ [2][0][RTW89_CN][0] = 40,
+ [2][0][RTW89_UK][0] = 46,
+ [2][0][RTW89_FCC][2] = 74,
+ [2][0][RTW89_ETSI][2] = 46,
+ [2][0][RTW89_MKK][2] = 48,
+ [2][0][RTW89_IC][2] = 48,
+ [2][0][RTW89_KCC][2] = 64,
+ [2][0][RTW89_ACMA][2] = 46,
+ [2][0][RTW89_CN][2] = 40,
+ [2][0][RTW89_UK][2] = 46,
+ [2][0][RTW89_FCC][4] = 74,
+ [2][0][RTW89_ETSI][4] = 46,
+ [2][0][RTW89_MKK][4] = 48,
+ [2][0][RTW89_IC][4] = 48,
+ [2][0][RTW89_KCC][4] = 64,
+ [2][0][RTW89_ACMA][4] = 46,
+ [2][0][RTW89_CN][4] = 40,
+ [2][0][RTW89_UK][4] = 46,
+ [2][0][RTW89_FCC][6] = 74,
+ [2][0][RTW89_ETSI][6] = 46,
+ [2][0][RTW89_MKK][6] = 48,
+ [2][0][RTW89_IC][6] = 48,
+ [2][0][RTW89_KCC][6] = 40,
+ [2][0][RTW89_ACMA][6] = 46,
+ [2][0][RTW89_CN][6] = 40,
+ [2][0][RTW89_UK][6] = 46,
+ [2][0][RTW89_FCC][8] = 74,
+ [2][0][RTW89_ETSI][8] = 46,
+ [2][0][RTW89_MKK][8] = 48,
+ [2][0][RTW89_IC][8] = 64,
+ [2][0][RTW89_KCC][8] = 66,
+ [2][0][RTW89_ACMA][8] = 46,
+ [2][0][RTW89_CN][8] = 40,
+ [2][0][RTW89_UK][8] = 46,
+ [2][0][RTW89_FCC][10] = 74,
+ [2][0][RTW89_ETSI][10] = 46,
+ [2][0][RTW89_MKK][10] = 48,
+ [2][0][RTW89_IC][10] = 64,
+ [2][0][RTW89_KCC][10] = 66,
+ [2][0][RTW89_ACMA][10] = 46,
+ [2][0][RTW89_CN][10] = 40,
+ [2][0][RTW89_UK][10] = 46,
+ [2][0][RTW89_FCC][12] = 74,
+ [2][0][RTW89_ETSI][12] = 46,
+ [2][0][RTW89_MKK][12] = 48,
+ [2][0][RTW89_IC][12] = 64,
+ [2][0][RTW89_KCC][12] = 64,
+ [2][0][RTW89_ACMA][12] = 46,
+ [2][0][RTW89_CN][12] = 40,
+ [2][0][RTW89_UK][12] = 46,
+ [2][0][RTW89_FCC][14] = 80,
+ [2][0][RTW89_ETSI][14] = 46,
+ [2][0][RTW89_MKK][14] = 48,
+ [2][0][RTW89_IC][14] = 64,
+ [2][0][RTW89_KCC][14] = 64,
+ [2][0][RTW89_ACMA][14] = 46,
+ [2][0][RTW89_CN][14] = 40,
+ [2][0][RTW89_UK][14] = 46,
+ [2][0][RTW89_FCC][15] = 72,
+ [2][0][RTW89_ETSI][15] = 46,
+ [2][0][RTW89_MKK][15] = 70,
+ [2][0][RTW89_IC][15] = 72,
+ [2][0][RTW89_KCC][15] = 66,
+ [2][0][RTW89_ACMA][15] = 46,
+ [2][0][RTW89_CN][15] = 127,
+ [2][0][RTW89_UK][15] = 46,
+ [2][0][RTW89_FCC][17] = 72,
+ [2][0][RTW89_ETSI][17] = 46,
+ [2][0][RTW89_MKK][17] = 70,
+ [2][0][RTW89_IC][17] = 72,
+ [2][0][RTW89_KCC][17] = 66,
+ [2][0][RTW89_ACMA][17] = 46,
+ [2][0][RTW89_CN][17] = 127,
+ [2][0][RTW89_UK][17] = 46,
+ [2][0][RTW89_FCC][19] = 70,
+ [2][0][RTW89_ETSI][19] = 46,
+ [2][0][RTW89_MKK][19] = 70,
+ [2][0][RTW89_IC][19] = 70,
+ [2][0][RTW89_KCC][19] = 66,
+ [2][0][RTW89_ACMA][19] = 46,
+ [2][0][RTW89_CN][19] = 127,
+ [2][0][RTW89_UK][19] = 46,
+ [2][0][RTW89_FCC][21] = 70,
+ [2][0][RTW89_ETSI][21] = 46,
+ [2][0][RTW89_MKK][21] = 70,
+ [2][0][RTW89_IC][21] = 70,
+ [2][0][RTW89_KCC][21] = 66,
+ [2][0][RTW89_ACMA][21] = 46,
+ [2][0][RTW89_CN][21] = 127,
+ [2][0][RTW89_UK][21] = 46,
+ [2][0][RTW89_FCC][23] = 70,
+ [2][0][RTW89_ETSI][23] = 46,
+ [2][0][RTW89_MKK][23] = 70,
+ [2][0][RTW89_IC][23] = 70,
+ [2][0][RTW89_KCC][23] = 66,
+ [2][0][RTW89_ACMA][23] = 46,
+ [2][0][RTW89_CN][23] = 127,
+ [2][0][RTW89_UK][23] = 46,
+ [2][0][RTW89_FCC][25] = 70,
+ [2][0][RTW89_ETSI][25] = 46,
+ [2][0][RTW89_MKK][25] = 70,
+ [2][0][RTW89_IC][25] = 127,
+ [2][0][RTW89_KCC][25] = 66,
+ [2][0][RTW89_ACMA][25] = 127,
+ [2][0][RTW89_CN][25] = 127,
+ [2][0][RTW89_UK][25] = 46,
+ [2][0][RTW89_FCC][27] = 70,
+ [2][0][RTW89_ETSI][27] = 46,
+ [2][0][RTW89_MKK][27] = 70,
+ [2][0][RTW89_IC][27] = 127,
+ [2][0][RTW89_KCC][27] = 64,
+ [2][0][RTW89_ACMA][27] = 127,
+ [2][0][RTW89_CN][27] = 127,
+ [2][0][RTW89_UK][27] = 46,
+ [2][0][RTW89_FCC][29] = 70,
+ [2][0][RTW89_ETSI][29] = 46,
+ [2][0][RTW89_MKK][29] = 70,
+ [2][0][RTW89_IC][29] = 127,
+ [2][0][RTW89_KCC][29] = 64,
+ [2][0][RTW89_ACMA][29] = 127,
+ [2][0][RTW89_CN][29] = 127,
+ [2][0][RTW89_UK][29] = 46,
+ [2][0][RTW89_FCC][31] = 70,
+ [2][0][RTW89_ETSI][31] = 46,
+ [2][0][RTW89_MKK][31] = 70,
+ [2][0][RTW89_IC][31] = 70,
+ [2][0][RTW89_KCC][31] = 64,
+ [2][0][RTW89_ACMA][31] = 46,
+ [2][0][RTW89_CN][31] = 127,
+ [2][0][RTW89_UK][31] = 46,
+ [2][0][RTW89_FCC][33] = 70,
+ [2][0][RTW89_ETSI][33] = 46,
+ [2][0][RTW89_MKK][33] = 70,
+ [2][0][RTW89_IC][33] = 70,
+ [2][0][RTW89_KCC][33] = 64,
+ [2][0][RTW89_ACMA][33] = 46,
+ [2][0][RTW89_CN][33] = 127,
+ [2][0][RTW89_UK][33] = 46,
+ [2][0][RTW89_FCC][35] = 70,
+ [2][0][RTW89_ETSI][35] = 46,
+ [2][0][RTW89_MKK][35] = 70,
+ [2][0][RTW89_IC][35] = 70,
+ [2][0][RTW89_KCC][35] = 64,
+ [2][0][RTW89_ACMA][35] = 46,
+ [2][0][RTW89_CN][35] = 127,
+ [2][0][RTW89_UK][35] = 46,
+ [2][0][RTW89_FCC][37] = 84,
+ [2][0][RTW89_ETSI][37] = 127,
+ [2][0][RTW89_MKK][37] = 68,
+ [2][0][RTW89_IC][37] = 84,
+ [2][0][RTW89_KCC][37] = 66,
+ [2][0][RTW89_ACMA][37] = 74,
+ [2][0][RTW89_CN][37] = 127,
+ [2][0][RTW89_UK][37] = 74,
+ [2][0][RTW89_FCC][38] = 84,
+ [2][0][RTW89_ETSI][38] = 28,
+ [2][0][RTW89_MKK][38] = 127,
+ [2][0][RTW89_IC][38] = 84,
+ [2][0][RTW89_KCC][38] = 64,
+ [2][0][RTW89_ACMA][38] = 84,
+ [2][0][RTW89_CN][38] = 68,
+ [2][0][RTW89_UK][38] = 46,
+ [2][0][RTW89_FCC][40] = 84,
+ [2][0][RTW89_ETSI][40] = 28,
+ [2][0][RTW89_MKK][40] = 127,
+ [2][0][RTW89_IC][40] = 84,
+ [2][0][RTW89_KCC][40] = 64,
+ [2][0][RTW89_ACMA][40] = 84,
+ [2][0][RTW89_CN][40] = 68,
+ [2][0][RTW89_UK][40] = 46,
+ [2][0][RTW89_FCC][42] = 80,
+ [2][0][RTW89_ETSI][42] = 28,
+ [2][0][RTW89_MKK][42] = 127,
+ [2][0][RTW89_IC][42] = 80,
+ [2][0][RTW89_KCC][42] = 66,
+ [2][0][RTW89_ACMA][42] = 84,
+ [2][0][RTW89_CN][42] = 68,
+ [2][0][RTW89_UK][42] = 46,
+ [2][0][RTW89_FCC][44] = 82,
+ [2][0][RTW89_ETSI][44] = 28,
+ [2][0][RTW89_MKK][44] = 127,
+ [2][0][RTW89_IC][44] = 82,
+ [2][0][RTW89_KCC][44] = 66,
+ [2][0][RTW89_ACMA][44] = 84,
+ [2][0][RTW89_CN][44] = 68,
+ [2][0][RTW89_UK][44] = 46,
+ [2][0][RTW89_FCC][46] = 82,
+ [2][0][RTW89_ETSI][46] = 28,
+ [2][0][RTW89_MKK][46] = 127,
+ [2][0][RTW89_IC][46] = 82,
+ [2][0][RTW89_KCC][46] = 66,
+ [2][0][RTW89_ACMA][46] = 84,
+ [2][0][RTW89_CN][46] = 68,
+ [2][0][RTW89_UK][46] = 46,
+ [2][0][RTW89_FCC][48] = 64,
+ [2][0][RTW89_ETSI][48] = 127,
+ [2][0][RTW89_MKK][48] = 127,
+ [2][0][RTW89_IC][48] = 127,
+ [2][0][RTW89_KCC][48] = 127,
+ [2][0][RTW89_ACMA][48] = 127,
+ [2][0][RTW89_CN][48] = 127,
+ [2][0][RTW89_UK][48] = 127,
+ [2][0][RTW89_FCC][50] = 64,
+ [2][0][RTW89_ETSI][50] = 127,
+ [2][0][RTW89_MKK][50] = 127,
+ [2][0][RTW89_IC][50] = 127,
+ [2][0][RTW89_KCC][50] = 127,
+ [2][0][RTW89_ACMA][50] = 127,
+ [2][0][RTW89_CN][50] = 127,
+ [2][0][RTW89_UK][50] = 127,
+ [2][0][RTW89_FCC][52] = 60,
+ [2][0][RTW89_ETSI][52] = 127,
+ [2][0][RTW89_MKK][52] = 127,
+ [2][0][RTW89_IC][52] = 127,
+ [2][0][RTW89_KCC][52] = 127,
+ [2][0][RTW89_ACMA][52] = 127,
+ [2][0][RTW89_CN][52] = 127,
+ [2][0][RTW89_UK][52] = 127,
+ [2][1][RTW89_FCC][0] = 127,
+ [2][1][RTW89_ETSI][0] = 127,
+ [2][1][RTW89_MKK][0] = 127,
+ [2][1][RTW89_IC][0] = 127,
+ [2][1][RTW89_KCC][0] = 127,
+ [2][1][RTW89_ACMA][0] = 127,
+ [2][1][RTW89_CN][0] = 28,
+ [2][1][RTW89_UK][0] = 127,
+ [2][1][RTW89_FCC][2] = 127,
+ [2][1][RTW89_ETSI][2] = 127,
+ [2][1][RTW89_MKK][2] = 127,
+ [2][1][RTW89_IC][2] = 127,
+ [2][1][RTW89_KCC][2] = 127,
+ [2][1][RTW89_ACMA][2] = 127,
+ [2][1][RTW89_CN][2] = 28,
+ [2][1][RTW89_UK][2] = 127,
+ [2][1][RTW89_FCC][4] = 127,
+ [2][1][RTW89_ETSI][4] = 127,
+ [2][1][RTW89_MKK][4] = 127,
+ [2][1][RTW89_IC][4] = 127,
+ [2][1][RTW89_KCC][4] = 127,
+ [2][1][RTW89_ACMA][4] = 127,
+ [2][1][RTW89_CN][4] = 28,
+ [2][1][RTW89_UK][4] = 127,
+ [2][1][RTW89_FCC][6] = 127,
+ [2][1][RTW89_ETSI][6] = 127,
+ [2][1][RTW89_MKK][6] = 127,
+ [2][1][RTW89_IC][6] = 127,
+ [2][1][RTW89_KCC][6] = 127,
+ [2][1][RTW89_ACMA][6] = 127,
+ [2][1][RTW89_CN][6] = 28,
+ [2][1][RTW89_UK][6] = 127,
+ [2][1][RTW89_FCC][8] = 127,
+ [2][1][RTW89_ETSI][8] = 127,
+ [2][1][RTW89_MKK][8] = 127,
+ [2][1][RTW89_IC][8] = 127,
+ [2][1][RTW89_KCC][8] = 127,
+ [2][1][RTW89_ACMA][8] = 127,
+ [2][1][RTW89_CN][8] = 28,
+ [2][1][RTW89_UK][8] = 127,
+ [2][1][RTW89_FCC][10] = 127,
+ [2][1][RTW89_ETSI][10] = 127,
+ [2][1][RTW89_MKK][10] = 127,
+ [2][1][RTW89_IC][10] = 127,
+ [2][1][RTW89_KCC][10] = 127,
+ [2][1][RTW89_ACMA][10] = 127,
+ [2][1][RTW89_CN][10] = 28,
+ [2][1][RTW89_UK][10] = 127,
+ [2][1][RTW89_FCC][12] = 127,
+ [2][1][RTW89_ETSI][12] = 127,
+ [2][1][RTW89_MKK][12] = 127,
+ [2][1][RTW89_IC][12] = 127,
+ [2][1][RTW89_KCC][12] = 127,
+ [2][1][RTW89_ACMA][12] = 127,
+ [2][1][RTW89_CN][12] = 28,
+ [2][1][RTW89_UK][12] = 127,
+ [2][1][RTW89_FCC][14] = 127,
+ [2][1][RTW89_ETSI][14] = 127,
+ [2][1][RTW89_MKK][14] = 127,
+ [2][1][RTW89_IC][14] = 127,
+ [2][1][RTW89_KCC][14] = 127,
+ [2][1][RTW89_ACMA][14] = 127,
+ [2][1][RTW89_CN][14] = 28,
+ [2][1][RTW89_UK][14] = 127,
+ [2][1][RTW89_FCC][15] = 127,
+ [2][1][RTW89_ETSI][15] = 127,
+ [2][1][RTW89_MKK][15] = 127,
+ [2][1][RTW89_IC][15] = 127,
+ [2][1][RTW89_KCC][15] = 127,
+ [2][1][RTW89_ACMA][15] = 127,
+ [2][1][RTW89_CN][15] = 127,
+ [2][1][RTW89_UK][15] = 127,
+ [2][1][RTW89_FCC][17] = 127,
+ [2][1][RTW89_ETSI][17] = 127,
+ [2][1][RTW89_MKK][17] = 127,
+ [2][1][RTW89_IC][17] = 127,
+ [2][1][RTW89_KCC][17] = 127,
+ [2][1][RTW89_ACMA][17] = 127,
+ [2][1][RTW89_CN][17] = 127,
+ [2][1][RTW89_UK][17] = 127,
+ [2][1][RTW89_FCC][19] = 127,
+ [2][1][RTW89_ETSI][19] = 127,
+ [2][1][RTW89_MKK][19] = 127,
+ [2][1][RTW89_IC][19] = 127,
+ [2][1][RTW89_KCC][19] = 127,
+ [2][1][RTW89_ACMA][19] = 127,
+ [2][1][RTW89_CN][19] = 127,
+ [2][1][RTW89_UK][19] = 127,
+ [2][1][RTW89_FCC][21] = 127,
+ [2][1][RTW89_ETSI][21] = 127,
+ [2][1][RTW89_MKK][21] = 127,
+ [2][1][RTW89_IC][21] = 127,
+ [2][1][RTW89_KCC][21] = 127,
+ [2][1][RTW89_ACMA][21] = 127,
+ [2][1][RTW89_CN][21] = 127,
+ [2][1][RTW89_UK][21] = 127,
+ [2][1][RTW89_FCC][23] = 127,
+ [2][1][RTW89_ETSI][23] = 127,
+ [2][1][RTW89_MKK][23] = 127,
+ [2][1][RTW89_IC][23] = 127,
+ [2][1][RTW89_KCC][23] = 127,
+ [2][1][RTW89_ACMA][23] = 127,
+ [2][1][RTW89_CN][23] = 127,
+ [2][1][RTW89_UK][23] = 127,
+ [2][1][RTW89_FCC][25] = 127,
+ [2][1][RTW89_ETSI][25] = 127,
+ [2][1][RTW89_MKK][25] = 127,
+ [2][1][RTW89_IC][25] = 127,
+ [2][1][RTW89_KCC][25] = 127,
+ [2][1][RTW89_ACMA][25] = 127,
+ [2][1][RTW89_CN][25] = 127,
+ [2][1][RTW89_UK][25] = 127,
+ [2][1][RTW89_FCC][27] = 127,
+ [2][1][RTW89_ETSI][27] = 127,
+ [2][1][RTW89_MKK][27] = 127,
+ [2][1][RTW89_IC][27] = 127,
+ [2][1][RTW89_KCC][27] = 127,
+ [2][1][RTW89_ACMA][27] = 127,
+ [2][1][RTW89_CN][27] = 127,
+ [2][1][RTW89_UK][27] = 127,
+ [2][1][RTW89_FCC][29] = 127,
+ [2][1][RTW89_ETSI][29] = 127,
+ [2][1][RTW89_MKK][29] = 127,
+ [2][1][RTW89_IC][29] = 127,
+ [2][1][RTW89_KCC][29] = 127,
+ [2][1][RTW89_ACMA][29] = 127,
+ [2][1][RTW89_CN][29] = 127,
+ [2][1][RTW89_UK][29] = 127,
+ [2][1][RTW89_FCC][31] = 127,
+ [2][1][RTW89_ETSI][31] = 127,
+ [2][1][RTW89_MKK][31] = 127,
+ [2][1][RTW89_IC][31] = 127,
+ [2][1][RTW89_KCC][31] = 127,
+ [2][1][RTW89_ACMA][31] = 127,
+ [2][1][RTW89_CN][31] = 127,
+ [2][1][RTW89_UK][31] = 127,
+ [2][1][RTW89_FCC][33] = 127,
+ [2][1][RTW89_ETSI][33] = 127,
+ [2][1][RTW89_MKK][33] = 127,
+ [2][1][RTW89_IC][33] = 127,
+ [2][1][RTW89_KCC][33] = 127,
+ [2][1][RTW89_ACMA][33] = 127,
+ [2][1][RTW89_CN][33] = 127,
+ [2][1][RTW89_UK][33] = 127,
+ [2][1][RTW89_FCC][35] = 127,
+ [2][1][RTW89_ETSI][35] = 127,
+ [2][1][RTW89_MKK][35] = 127,
+ [2][1][RTW89_IC][35] = 127,
+ [2][1][RTW89_KCC][35] = 127,
+ [2][1][RTW89_ACMA][35] = 127,
+ [2][1][RTW89_CN][35] = 127,
+ [2][1][RTW89_UK][35] = 127,
+ [2][1][RTW89_FCC][37] = 127,
+ [2][1][RTW89_ETSI][37] = 127,
+ [2][1][RTW89_MKK][37] = 127,
+ [2][1][RTW89_IC][37] = 127,
+ [2][1][RTW89_KCC][37] = 127,
+ [2][1][RTW89_ACMA][37] = 127,
+ [2][1][RTW89_CN][37] = 127,
+ [2][1][RTW89_UK][37] = 127,
+ [2][1][RTW89_FCC][38] = 127,
+ [2][1][RTW89_ETSI][38] = 127,
+ [2][1][RTW89_MKK][38] = 127,
+ [2][1][RTW89_IC][38] = 127,
+ [2][1][RTW89_KCC][38] = 127,
+ [2][1][RTW89_ACMA][38] = 127,
+ [2][1][RTW89_CN][38] = 56,
+ [2][1][RTW89_UK][38] = 127,
+ [2][1][RTW89_FCC][40] = 127,
+ [2][1][RTW89_ETSI][40] = 127,
+ [2][1][RTW89_MKK][40] = 127,
+ [2][1][RTW89_IC][40] = 127,
+ [2][1][RTW89_KCC][40] = 127,
+ [2][1][RTW89_ACMA][40] = 127,
+ [2][1][RTW89_CN][40] = 56,
+ [2][1][RTW89_UK][40] = 127,
+ [2][1][RTW89_FCC][42] = 127,
+ [2][1][RTW89_ETSI][42] = 127,
+ [2][1][RTW89_MKK][42] = 127,
+ [2][1][RTW89_IC][42] = 127,
+ [2][1][RTW89_KCC][42] = 127,
+ [2][1][RTW89_ACMA][42] = 127,
+ [2][1][RTW89_CN][42] = 56,
+ [2][1][RTW89_UK][42] = 127,
+ [2][1][RTW89_FCC][44] = 127,
+ [2][1][RTW89_ETSI][44] = 127,
+ [2][1][RTW89_MKK][44] = 127,
+ [2][1][RTW89_IC][44] = 127,
+ [2][1][RTW89_KCC][44] = 127,
+ [2][1][RTW89_ACMA][44] = 127,
+ [2][1][RTW89_CN][44] = 56,
+ [2][1][RTW89_UK][44] = 127,
+ [2][1][RTW89_FCC][46] = 127,
+ [2][1][RTW89_ETSI][46] = 127,
+ [2][1][RTW89_MKK][46] = 127,
+ [2][1][RTW89_IC][46] = 127,
+ [2][1][RTW89_KCC][46] = 127,
+ [2][1][RTW89_ACMA][46] = 127,
+ [2][1][RTW89_CN][46] = 56,
+ [2][1][RTW89_UK][46] = 127,
+ [2][1][RTW89_FCC][48] = 127,
+ [2][1][RTW89_ETSI][48] = 127,
+ [2][1][RTW89_MKK][48] = 127,
+ [2][1][RTW89_IC][48] = 127,
+ [2][1][RTW89_KCC][48] = 127,
+ [2][1][RTW89_ACMA][48] = 127,
+ [2][1][RTW89_CN][48] = 127,
+ [2][1][RTW89_UK][48] = 127,
+ [2][1][RTW89_FCC][50] = 127,
+ [2][1][RTW89_ETSI][50] = 127,
+ [2][1][RTW89_MKK][50] = 127,
+ [2][1][RTW89_IC][50] = 127,
+ [2][1][RTW89_KCC][50] = 127,
+ [2][1][RTW89_ACMA][50] = 127,
+ [2][1][RTW89_CN][50] = 127,
+ [2][1][RTW89_UK][50] = 127,
+ [2][1][RTW89_FCC][52] = 127,
+ [2][1][RTW89_ETSI][52] = 127,
+ [2][1][RTW89_MKK][52] = 127,
+ [2][1][RTW89_IC][52] = 127,
+ [2][1][RTW89_KCC][52] = 127,
+ [2][1][RTW89_ACMA][52] = 127,
+ [2][1][RTW89_CN][52] = 127,
+ [2][1][RTW89_UK][52] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_2g_type2[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
+ [0][0][0][0][RTW89_WW][0] = 58,
+ [0][0][0][0][RTW89_WW][1] = 58,
+ [0][0][0][0][RTW89_WW][2] = 58,
+ [0][0][0][0][RTW89_WW][3] = 58,
+ [0][0][0][0][RTW89_WW][4] = 58,
+ [0][0][0][0][RTW89_WW][5] = 58,
+ [0][0][0][0][RTW89_WW][6] = 58,
+ [0][0][0][0][RTW89_WW][7] = 58,
+ [0][0][0][0][RTW89_WW][8] = 58,
+ [0][0][0][0][RTW89_WW][9] = 58,
+ [0][0][0][0][RTW89_WW][10] = 58,
+ [0][0][0][0][RTW89_WW][11] = 58,
+ [0][0][0][0][RTW89_WW][12] = 52,
+ [0][0][0][0][RTW89_WW][13] = 76,
+ [0][1][0][0][RTW89_WW][0] = 0,
+ [0][1][0][0][RTW89_WW][1] = 0,
+ [0][1][0][0][RTW89_WW][2] = 0,
+ [0][1][0][0][RTW89_WW][3] = 0,
+ [0][1][0][0][RTW89_WW][4] = 0,
+ [0][1][0][0][RTW89_WW][5] = 0,
+ [0][1][0][0][RTW89_WW][6] = 0,
+ [0][1][0][0][RTW89_WW][7] = 0,
+ [0][1][0][0][RTW89_WW][8] = 0,
+ [0][1][0][0][RTW89_WW][9] = 0,
+ [0][1][0][0][RTW89_WW][10] = 0,
+ [0][1][0][0][RTW89_WW][11] = 0,
+ [0][1][0][0][RTW89_WW][12] = 0,
+ [0][1][0][0][RTW89_WW][13] = 0,
+ [1][0][0][0][RTW89_WW][0] = 0,
+ [1][0][0][0][RTW89_WW][1] = 0,
+ [1][0][0][0][RTW89_WW][2] = 58,
+ [1][0][0][0][RTW89_WW][3] = 58,
+ [1][0][0][0][RTW89_WW][4] = 58,
+ [1][0][0][0][RTW89_WW][5] = 58,
+ [1][0][0][0][RTW89_WW][6] = 58,
+ [1][0][0][0][RTW89_WW][7] = 58,
+ [1][0][0][0][RTW89_WW][8] = 58,
+ [1][0][0][0][RTW89_WW][9] = 58,
+ [1][0][0][0][RTW89_WW][10] = 58,
+ [1][0][0][0][RTW89_WW][11] = 0,
+ [1][0][0][0][RTW89_WW][12] = 0,
+ [1][0][0][0][RTW89_WW][13] = 0,
+ [1][1][0][0][RTW89_WW][0] = 0,
+ [1][1][0][0][RTW89_WW][1] = 0,
+ [1][1][0][0][RTW89_WW][2] = 0,
+ [1][1][0][0][RTW89_WW][3] = 0,
+ [1][1][0][0][RTW89_WW][4] = 0,
+ [1][1][0][0][RTW89_WW][5] = 0,
+ [1][1][0][0][RTW89_WW][6] = 0,
+ [1][1][0][0][RTW89_WW][7] = 0,
+ [1][1][0][0][RTW89_WW][8] = 0,
+ [1][1][0][0][RTW89_WW][9] = 0,
+ [1][1][0][0][RTW89_WW][10] = 0,
+ [1][1][0][0][RTW89_WW][11] = 0,
+ [1][1][0][0][RTW89_WW][12] = 0,
+ [1][1][0][0][RTW89_WW][13] = 0,
+ [0][0][1][0][RTW89_WW][0] = 58,
+ [0][0][1][0][RTW89_WW][1] = 60,
+ [0][0][1][0][RTW89_WW][2] = 60,
+ [0][0][1][0][RTW89_WW][3] = 60,
+ [0][0][1][0][RTW89_WW][4] = 60,
+ [0][0][1][0][RTW89_WW][5] = 60,
+ [0][0][1][0][RTW89_WW][6] = 60,
+ [0][0][1][0][RTW89_WW][7] = 60,
+ [0][0][1][0][RTW89_WW][8] = 60,
+ [0][0][1][0][RTW89_WW][9] = 60,
+ [0][0][1][0][RTW89_WW][10] = 60,
+ [0][0][1][0][RTW89_WW][11] = 60,
+ [0][0][1][0][RTW89_WW][12] = 58,
+ [0][0][1][0][RTW89_WW][13] = 0,
+ [0][1][1][0][RTW89_WW][0] = 0,
+ [0][1][1][0][RTW89_WW][1] = 0,
+ [0][1][1][0][RTW89_WW][2] = 0,
+ [0][1][1][0][RTW89_WW][3] = 0,
+ [0][1][1][0][RTW89_WW][4] = 0,
+ [0][1][1][0][RTW89_WW][5] = 0,
+ [0][1][1][0][RTW89_WW][6] = 0,
+ [0][1][1][0][RTW89_WW][7] = 0,
+ [0][1][1][0][RTW89_WW][8] = 0,
+ [0][1][1][0][RTW89_WW][9] = 0,
+ [0][1][1][0][RTW89_WW][10] = 0,
+ [0][1][1][0][RTW89_WW][11] = 0,
+ [0][1][1][0][RTW89_WW][12] = 0,
+ [0][1][1][0][RTW89_WW][13] = 0,
+ [0][0][2][0][RTW89_WW][0] = 60,
+ [0][0][2][0][RTW89_WW][1] = 60,
+ [0][0][2][0][RTW89_WW][2] = 60,
+ [0][0][2][0][RTW89_WW][3] = 60,
+ [0][0][2][0][RTW89_WW][4] = 60,
+ [0][0][2][0][RTW89_WW][5] = 60,
+ [0][0][2][0][RTW89_WW][6] = 60,
+ [0][0][2][0][RTW89_WW][7] = 60,
+ [0][0][2][0][RTW89_WW][8] = 60,
+ [0][0][2][0][RTW89_WW][9] = 60,
+ [0][0][2][0][RTW89_WW][10] = 60,
+ [0][0][2][0][RTW89_WW][11] = 60,
+ [0][0][2][0][RTW89_WW][12] = 60,
+ [0][0][2][0][RTW89_WW][13] = 0,
+ [0][1][2][0][RTW89_WW][0] = 0,
+ [0][1][2][0][RTW89_WW][1] = 0,
+ [0][1][2][0][RTW89_WW][2] = 0,
+ [0][1][2][0][RTW89_WW][3] = 0,
+ [0][1][2][0][RTW89_WW][4] = 0,
+ [0][1][2][0][RTW89_WW][5] = 0,
+ [0][1][2][0][RTW89_WW][6] = 0,
+ [0][1][2][0][RTW89_WW][7] = 0,
+ [0][1][2][0][RTW89_WW][8] = 0,
+ [0][1][2][0][RTW89_WW][9] = 0,
+ [0][1][2][0][RTW89_WW][10] = 0,
+ [0][1][2][0][RTW89_WW][11] = 0,
+ [0][1][2][0][RTW89_WW][12] = 0,
+ [0][1][2][0][RTW89_WW][13] = 0,
+ [0][1][2][1][RTW89_WW][0] = 0,
+ [0][1][2][1][RTW89_WW][1] = 0,
+ [0][1][2][1][RTW89_WW][2] = 0,
+ [0][1][2][1][RTW89_WW][3] = 0,
+ [0][1][2][1][RTW89_WW][4] = 0,
+ [0][1][2][1][RTW89_WW][5] = 0,
+ [0][1][2][1][RTW89_WW][6] = 0,
+ [0][1][2][1][RTW89_WW][7] = 0,
+ [0][1][2][1][RTW89_WW][8] = 0,
+ [0][1][2][1][RTW89_WW][9] = 0,
+ [0][1][2][1][RTW89_WW][10] = 0,
+ [0][1][2][1][RTW89_WW][11] = 0,
+ [0][1][2][1][RTW89_WW][12] = 0,
+ [0][1][2][1][RTW89_WW][13] = 0,
+ [1][0][2][0][RTW89_WW][0] = 0,
+ [1][0][2][0][RTW89_WW][1] = 0,
+ [1][0][2][0][RTW89_WW][2] = 58,
+ [1][0][2][0][RTW89_WW][3] = 58,
+ [1][0][2][0][RTW89_WW][4] = 58,
+ [1][0][2][0][RTW89_WW][5] = 58,
+ [1][0][2][0][RTW89_WW][6] = 58,
+ [1][0][2][0][RTW89_WW][7] = 58,
+ [1][0][2][0][RTW89_WW][8] = 58,
+ [1][0][2][0][RTW89_WW][9] = 58,
+ [1][0][2][0][RTW89_WW][10] = 58,
+ [1][0][2][0][RTW89_WW][11] = 0,
+ [1][0][2][0][RTW89_WW][12] = 0,
+ [1][0][2][0][RTW89_WW][13] = 0,
+ [1][1][2][0][RTW89_WW][0] = 0,
+ [1][1][2][0][RTW89_WW][1] = 0,
+ [1][1][2][0][RTW89_WW][2] = 0,
+ [1][1][2][0][RTW89_WW][3] = 0,
+ [1][1][2][0][RTW89_WW][4] = 0,
+ [1][1][2][0][RTW89_WW][5] = 0,
+ [1][1][2][0][RTW89_WW][6] = 0,
+ [1][1][2][0][RTW89_WW][7] = 0,
+ [1][1][2][0][RTW89_WW][8] = 0,
+ [1][1][2][0][RTW89_WW][9] = 0,
+ [1][1][2][0][RTW89_WW][10] = 0,
+ [1][1][2][0][RTW89_WW][11] = 0,
+ [1][1][2][0][RTW89_WW][12] = 0,
+ [1][1][2][0][RTW89_WW][13] = 0,
+ [1][1][2][1][RTW89_WW][0] = 0,
+ [1][1][2][1][RTW89_WW][1] = 0,
+ [1][1][2][1][RTW89_WW][2] = 0,
+ [1][1][2][1][RTW89_WW][3] = 0,
+ [1][1][2][1][RTW89_WW][4] = 0,
+ [1][1][2][1][RTW89_WW][5] = 0,
+ [1][1][2][1][RTW89_WW][6] = 0,
+ [1][1][2][1][RTW89_WW][7] = 0,
+ [1][1][2][1][RTW89_WW][8] = 0,
+ [1][1][2][1][RTW89_WW][9] = 0,
+ [1][1][2][1][RTW89_WW][10] = 0,
+ [1][1][2][1][RTW89_WW][11] = 0,
+ [1][1][2][1][RTW89_WW][12] = 0,
+ [1][1][2][1][RTW89_WW][13] = 0,
+ [0][0][0][0][RTW89_FCC][0] = 82,
+ [0][0][0][0][RTW89_ETSI][0] = 58,
+ [0][0][0][0][RTW89_MKK][0] = 68,
+ [0][0][0][0][RTW89_IC][0] = 82,
+ [0][0][0][0][RTW89_KCC][0] = 68,
+ [0][0][0][0][RTW89_ACMA][0] = 58,
+ [0][0][0][0][RTW89_CN][0] = 60,
+ [0][0][0][0][RTW89_UK][0] = 58,
+ [0][0][0][0][RTW89_FCC][1] = 82,
+ [0][0][0][0][RTW89_ETSI][1] = 58,
+ [0][0][0][0][RTW89_MKK][1] = 68,
+ [0][0][0][0][RTW89_IC][1] = 82,
+ [0][0][0][0][RTW89_KCC][1] = 68,
+ [0][0][0][0][RTW89_ACMA][1] = 58,
+ [0][0][0][0][RTW89_CN][1] = 60,
+ [0][0][0][0][RTW89_UK][1] = 58,
+ [0][0][0][0][RTW89_FCC][2] = 82,
+ [0][0][0][0][RTW89_ETSI][2] = 58,
+ [0][0][0][0][RTW89_MKK][2] = 68,
+ [0][0][0][0][RTW89_IC][2] = 82,
+ [0][0][0][0][RTW89_KCC][2] = 68,
+ [0][0][0][0][RTW89_ACMA][2] = 58,
+ [0][0][0][0][RTW89_CN][2] = 60,
+ [0][0][0][0][RTW89_UK][2] = 58,
+ [0][0][0][0][RTW89_FCC][3] = 82,
+ [0][0][0][0][RTW89_ETSI][3] = 58,
+ [0][0][0][0][RTW89_MKK][3] = 68,
+ [0][0][0][0][RTW89_IC][3] = 82,
+ [0][0][0][0][RTW89_KCC][3] = 68,
+ [0][0][0][0][RTW89_ACMA][3] = 58,
+ [0][0][0][0][RTW89_CN][3] = 60,
+ [0][0][0][0][RTW89_UK][3] = 58,
+ [0][0][0][0][RTW89_FCC][4] = 82,
+ [0][0][0][0][RTW89_ETSI][4] = 58,
+ [0][0][0][0][RTW89_MKK][4] = 68,
+ [0][0][0][0][RTW89_IC][4] = 82,
+ [0][0][0][0][RTW89_KCC][4] = 68,
+ [0][0][0][0][RTW89_ACMA][4] = 58,
+ [0][0][0][0][RTW89_CN][4] = 60,
+ [0][0][0][0][RTW89_UK][4] = 58,
+ [0][0][0][0][RTW89_FCC][5] = 82,
+ [0][0][0][0][RTW89_ETSI][5] = 58,
+ [0][0][0][0][RTW89_MKK][5] = 68,
+ [0][0][0][0][RTW89_IC][5] = 82,
+ [0][0][0][0][RTW89_KCC][5] = 68,
+ [0][0][0][0][RTW89_ACMA][5] = 58,
+ [0][0][0][0][RTW89_CN][5] = 60,
+ [0][0][0][0][RTW89_UK][5] = 58,
+ [0][0][0][0][RTW89_FCC][6] = 82,
+ [0][0][0][0][RTW89_ETSI][6] = 58,
+ [0][0][0][0][RTW89_MKK][6] = 68,
+ [0][0][0][0][RTW89_IC][6] = 82,
+ [0][0][0][0][RTW89_KCC][6] = 68,
+ [0][0][0][0][RTW89_ACMA][6] = 58,
+ [0][0][0][0][RTW89_CN][6] = 60,
+ [0][0][0][0][RTW89_UK][6] = 58,
+ [0][0][0][0][RTW89_FCC][7] = 82,
+ [0][0][0][0][RTW89_ETSI][7] = 58,
+ [0][0][0][0][RTW89_MKK][7] = 68,
+ [0][0][0][0][RTW89_IC][7] = 82,
+ [0][0][0][0][RTW89_KCC][7] = 68,
+ [0][0][0][0][RTW89_ACMA][7] = 58,
+ [0][0][0][0][RTW89_CN][7] = 60,
+ [0][0][0][0][RTW89_UK][7] = 58,
+ [0][0][0][0][RTW89_FCC][8] = 82,
+ [0][0][0][0][RTW89_ETSI][8] = 58,
+ [0][0][0][0][RTW89_MKK][8] = 68,
+ [0][0][0][0][RTW89_IC][8] = 82,
+ [0][0][0][0][RTW89_KCC][8] = 68,
+ [0][0][0][0][RTW89_ACMA][8] = 58,
+ [0][0][0][0][RTW89_CN][8] = 60,
+ [0][0][0][0][RTW89_UK][8] = 58,
+ [0][0][0][0][RTW89_FCC][9] = 82,
+ [0][0][0][0][RTW89_ETSI][9] = 58,
+ [0][0][0][0][RTW89_MKK][9] = 68,
+ [0][0][0][0][RTW89_IC][9] = 82,
+ [0][0][0][0][RTW89_KCC][9] = 68,
+ [0][0][0][0][RTW89_ACMA][9] = 58,
+ [0][0][0][0][RTW89_CN][9] = 60,
+ [0][0][0][0][RTW89_UK][9] = 58,
+ [0][0][0][0][RTW89_FCC][10] = 80,
+ [0][0][0][0][RTW89_ETSI][10] = 58,
+ [0][0][0][0][RTW89_MKK][10] = 68,
+ [0][0][0][0][RTW89_IC][10] = 80,
+ [0][0][0][0][RTW89_KCC][10] = 68,
+ [0][0][0][0][RTW89_ACMA][10] = 58,
+ [0][0][0][0][RTW89_CN][10] = 60,
+ [0][0][0][0][RTW89_UK][10] = 58,
+ [0][0][0][0][RTW89_FCC][11] = 60,
+ [0][0][0][0][RTW89_ETSI][11] = 58,
+ [0][0][0][0][RTW89_MKK][11] = 68,
+ [0][0][0][0][RTW89_IC][11] = 60,
+ [0][0][0][0][RTW89_KCC][11] = 68,
+ [0][0][0][0][RTW89_ACMA][11] = 58,
+ [0][0][0][0][RTW89_CN][11] = 60,
+ [0][0][0][0][RTW89_UK][11] = 58,
+ [0][0][0][0][RTW89_FCC][12] = 52,
+ [0][0][0][0][RTW89_ETSI][12] = 58,
+ [0][0][0][0][RTW89_MKK][12] = 68,
+ [0][0][0][0][RTW89_IC][12] = 52,
+ [0][0][0][0][RTW89_KCC][12] = 68,
+ [0][0][0][0][RTW89_ACMA][12] = 58,
+ [0][0][0][0][RTW89_CN][12] = 60,
+ [0][0][0][0][RTW89_UK][12] = 58,
+ [0][0][0][0][RTW89_FCC][13] = 127,
+ [0][0][0][0][RTW89_ETSI][13] = 127,
+ [0][0][0][0][RTW89_MKK][13] = 76,
+ [0][0][0][0][RTW89_IC][13] = 127,
+ [0][0][0][0][RTW89_KCC][13] = 127,
+ [0][0][0][0][RTW89_ACMA][13] = 127,
+ [0][0][0][0][RTW89_CN][13] = 127,
+ [0][0][0][0][RTW89_UK][13] = 127,
+ [0][1][0][0][RTW89_FCC][0] = 127,
+ [0][1][0][0][RTW89_ETSI][0] = 127,
+ [0][1][0][0][RTW89_MKK][0] = 127,
+ [0][1][0][0][RTW89_IC][0] = 127,
+ [0][1][0][0][RTW89_KCC][0] = 127,
+ [0][1][0][0][RTW89_ACMA][0] = 127,
+ [0][1][0][0][RTW89_CN][0] = 127,
+ [0][1][0][0][RTW89_UK][0] = 127,
+ [0][1][0][0][RTW89_FCC][1] = 127,
+ [0][1][0][0][RTW89_ETSI][1] = 127,
+ [0][1][0][0][RTW89_MKK][1] = 127,
+ [0][1][0][0][RTW89_IC][1] = 127,
+ [0][1][0][0][RTW89_KCC][1] = 127,
+ [0][1][0][0][RTW89_ACMA][1] = 127,
+ [0][1][0][0][RTW89_CN][1] = 127,
+ [0][1][0][0][RTW89_UK][1] = 127,
+ [0][1][0][0][RTW89_FCC][2] = 127,
+ [0][1][0][0][RTW89_ETSI][2] = 127,
+ [0][1][0][0][RTW89_MKK][2] = 127,
+ [0][1][0][0][RTW89_IC][2] = 127,
+ [0][1][0][0][RTW89_KCC][2] = 127,
+ [0][1][0][0][RTW89_ACMA][2] = 127,
+ [0][1][0][0][RTW89_CN][2] = 127,
+ [0][1][0][0][RTW89_UK][2] = 127,
+ [0][1][0][0][RTW89_FCC][3] = 127,
+ [0][1][0][0][RTW89_ETSI][3] = 127,
+ [0][1][0][0][RTW89_MKK][3] = 127,
+ [0][1][0][0][RTW89_IC][3] = 127,
+ [0][1][0][0][RTW89_KCC][3] = 127,
+ [0][1][0][0][RTW89_ACMA][3] = 127,
+ [0][1][0][0][RTW89_CN][3] = 127,
+ [0][1][0][0][RTW89_UK][3] = 127,
+ [0][1][0][0][RTW89_FCC][4] = 127,
+ [0][1][0][0][RTW89_ETSI][4] = 127,
+ [0][1][0][0][RTW89_MKK][4] = 127,
+ [0][1][0][0][RTW89_IC][4] = 127,
+ [0][1][0][0][RTW89_KCC][4] = 127,
+ [0][1][0][0][RTW89_ACMA][4] = 127,
+ [0][1][0][0][RTW89_CN][4] = 127,
+ [0][1][0][0][RTW89_UK][4] = 127,
+ [0][1][0][0][RTW89_FCC][5] = 127,
+ [0][1][0][0][RTW89_ETSI][5] = 127,
+ [0][1][0][0][RTW89_MKK][5] = 127,
+ [0][1][0][0][RTW89_IC][5] = 127,
+ [0][1][0][0][RTW89_KCC][5] = 127,
+ [0][1][0][0][RTW89_ACMA][5] = 127,
+ [0][1][0][0][RTW89_CN][5] = 127,
+ [0][1][0][0][RTW89_UK][5] = 127,
+ [0][1][0][0][RTW89_FCC][6] = 127,
+ [0][1][0][0][RTW89_ETSI][6] = 127,
+ [0][1][0][0][RTW89_MKK][6] = 127,
+ [0][1][0][0][RTW89_IC][6] = 127,
+ [0][1][0][0][RTW89_KCC][6] = 127,
+ [0][1][0][0][RTW89_ACMA][6] = 127,
+ [0][1][0][0][RTW89_CN][6] = 127,
+ [0][1][0][0][RTW89_UK][6] = 127,
+ [0][1][0][0][RTW89_FCC][7] = 127,
+ [0][1][0][0][RTW89_ETSI][7] = 127,
+ [0][1][0][0][RTW89_MKK][7] = 127,
+ [0][1][0][0][RTW89_IC][7] = 127,
+ [0][1][0][0][RTW89_KCC][7] = 127,
+ [0][1][0][0][RTW89_ACMA][7] = 127,
+ [0][1][0][0][RTW89_CN][7] = 127,
+ [0][1][0][0][RTW89_UK][7] = 127,
+ [0][1][0][0][RTW89_FCC][8] = 127,
+ [0][1][0][0][RTW89_ETSI][8] = 127,
+ [0][1][0][0][RTW89_MKK][8] = 127,
+ [0][1][0][0][RTW89_IC][8] = 127,
+ [0][1][0][0][RTW89_KCC][8] = 127,
+ [0][1][0][0][RTW89_ACMA][8] = 127,
+ [0][1][0][0][RTW89_CN][8] = 127,
+ [0][1][0][0][RTW89_UK][8] = 127,
+ [0][1][0][0][RTW89_FCC][9] = 127,
+ [0][1][0][0][RTW89_ETSI][9] = 127,
+ [0][1][0][0][RTW89_MKK][9] = 127,
+ [0][1][0][0][RTW89_IC][9] = 127,
+ [0][1][0][0][RTW89_KCC][9] = 127,
+ [0][1][0][0][RTW89_ACMA][9] = 127,
+ [0][1][0][0][RTW89_CN][9] = 127,
+ [0][1][0][0][RTW89_UK][9] = 127,
+ [0][1][0][0][RTW89_FCC][10] = 127,
+ [0][1][0][0][RTW89_ETSI][10] = 127,
+ [0][1][0][0][RTW89_MKK][10] = 127,
+ [0][1][0][0][RTW89_IC][10] = 127,
+ [0][1][0][0][RTW89_KCC][10] = 127,
+ [0][1][0][0][RTW89_ACMA][10] = 127,
+ [0][1][0][0][RTW89_CN][10] = 127,
+ [0][1][0][0][RTW89_UK][10] = 127,
+ [0][1][0][0][RTW89_FCC][11] = 127,
+ [0][1][0][0][RTW89_ETSI][11] = 127,
+ [0][1][0][0][RTW89_MKK][11] = 127,
+ [0][1][0][0][RTW89_IC][11] = 127,
+ [0][1][0][0][RTW89_KCC][11] = 127,
+ [0][1][0][0][RTW89_ACMA][11] = 127,
+ [0][1][0][0][RTW89_CN][11] = 127,
+ [0][1][0][0][RTW89_UK][11] = 127,
+ [0][1][0][0][RTW89_FCC][12] = 127,
+ [0][1][0][0][RTW89_ETSI][12] = 127,
+ [0][1][0][0][RTW89_MKK][12] = 127,
+ [0][1][0][0][RTW89_IC][12] = 127,
+ [0][1][0][0][RTW89_KCC][12] = 127,
+ [0][1][0][0][RTW89_ACMA][12] = 127,
+ [0][1][0][0][RTW89_CN][12] = 127,
+ [0][1][0][0][RTW89_UK][12] = 127,
+ [0][1][0][0][RTW89_FCC][13] = 127,
+ [0][1][0][0][RTW89_ETSI][13] = 127,
+ [0][1][0][0][RTW89_MKK][13] = 127,
+ [0][1][0][0][RTW89_IC][13] = 127,
+ [0][1][0][0][RTW89_KCC][13] = 127,
+ [0][1][0][0][RTW89_ACMA][13] = 127,
+ [0][1][0][0][RTW89_CN][13] = 127,
+ [0][1][0][0][RTW89_UK][13] = 127,
+ [1][0][0][0][RTW89_FCC][0] = 127,
+ [1][0][0][0][RTW89_ETSI][0] = 127,
+ [1][0][0][0][RTW89_MKK][0] = 127,
+ [1][0][0][0][RTW89_IC][0] = 127,
+ [1][0][0][0][RTW89_KCC][0] = 127,
+ [1][0][0][0][RTW89_ACMA][0] = 127,
+ [1][0][0][0][RTW89_CN][0] = 127,
+ [1][0][0][0][RTW89_UK][0] = 127,
+ [1][0][0][0][RTW89_FCC][1] = 127,
+ [1][0][0][0][RTW89_ETSI][1] = 127,
+ [1][0][0][0][RTW89_MKK][1] = 127,
+ [1][0][0][0][RTW89_IC][1] = 127,
+ [1][0][0][0][RTW89_KCC][1] = 127,
+ [1][0][0][0][RTW89_ACMA][1] = 127,
+ [1][0][0][0][RTW89_CN][1] = 127,
+ [1][0][0][0][RTW89_UK][1] = 127,
+ [1][0][0][0][RTW89_FCC][2] = 127,
+ [1][0][0][0][RTW89_ETSI][2] = 58,
+ [1][0][0][0][RTW89_MKK][2] = 70,
+ [1][0][0][0][RTW89_IC][2] = 127,
+ [1][0][0][0][RTW89_KCC][2] = 68,
+ [1][0][0][0][RTW89_ACMA][2] = 58,
+ [1][0][0][0][RTW89_CN][2] = 60,
+ [1][0][0][0][RTW89_UK][2] = 58,
+ [1][0][0][0][RTW89_FCC][3] = 127,
+ [1][0][0][0][RTW89_ETSI][3] = 58,
+ [1][0][0][0][RTW89_MKK][3] = 76,
+ [1][0][0][0][RTW89_IC][3] = 127,
+ [1][0][0][0][RTW89_KCC][3] = 68,
+ [1][0][0][0][RTW89_ACMA][3] = 58,
+ [1][0][0][0][RTW89_CN][3] = 60,
+ [1][0][0][0][RTW89_UK][3] = 58,
+ [1][0][0][0][RTW89_FCC][4] = 127,
+ [1][0][0][0][RTW89_ETSI][4] = 58,
+ [1][0][0][0][RTW89_MKK][4] = 76,
+ [1][0][0][0][RTW89_IC][4] = 127,
+ [1][0][0][0][RTW89_KCC][4] = 68,
+ [1][0][0][0][RTW89_ACMA][4] = 58,
+ [1][0][0][0][RTW89_CN][4] = 60,
+ [1][0][0][0][RTW89_UK][4] = 58,
+ [1][0][0][0][RTW89_FCC][5] = 127,
+ [1][0][0][0][RTW89_ETSI][5] = 58,
+ [1][0][0][0][RTW89_MKK][5] = 76,
+ [1][0][0][0][RTW89_IC][5] = 127,
+ [1][0][0][0][RTW89_KCC][5] = 68,
+ [1][0][0][0][RTW89_ACMA][5] = 58,
+ [1][0][0][0][RTW89_CN][5] = 60,
+ [1][0][0][0][RTW89_UK][5] = 58,
+ [1][0][0][0][RTW89_FCC][6] = 127,
+ [1][0][0][0][RTW89_ETSI][6] = 58,
+ [1][0][0][0][RTW89_MKK][6] = 76,
+ [1][0][0][0][RTW89_IC][6] = 127,
+ [1][0][0][0][RTW89_KCC][6] = 68,
+ [1][0][0][0][RTW89_ACMA][6] = 58,
+ [1][0][0][0][RTW89_CN][6] = 60,
+ [1][0][0][0][RTW89_UK][6] = 58,
+ [1][0][0][0][RTW89_FCC][7] = 127,
+ [1][0][0][0][RTW89_ETSI][7] = 58,
+ [1][0][0][0][RTW89_MKK][7] = 76,
+ [1][0][0][0][RTW89_IC][7] = 127,
+ [1][0][0][0][RTW89_KCC][7] = 68,
+ [1][0][0][0][RTW89_ACMA][7] = 58,
+ [1][0][0][0][RTW89_CN][7] = 60,
+ [1][0][0][0][RTW89_UK][7] = 58,
+ [1][0][0][0][RTW89_FCC][8] = 127,
+ [1][0][0][0][RTW89_ETSI][8] = 58,
+ [1][0][0][0][RTW89_MKK][8] = 76,
+ [1][0][0][0][RTW89_IC][8] = 127,
+ [1][0][0][0][RTW89_KCC][8] = 68,
+ [1][0][0][0][RTW89_ACMA][8] = 58,
+ [1][0][0][0][RTW89_CN][8] = 60,
+ [1][0][0][0][RTW89_UK][8] = 58,
+ [1][0][0][0][RTW89_FCC][9] = 127,
+ [1][0][0][0][RTW89_ETSI][9] = 58,
+ [1][0][0][0][RTW89_MKK][9] = 76,
+ [1][0][0][0][RTW89_IC][9] = 127,
+ [1][0][0][0][RTW89_KCC][9] = 68,
+ [1][0][0][0][RTW89_ACMA][9] = 58,
+ [1][0][0][0][RTW89_CN][9] = 60,
+ [1][0][0][0][RTW89_UK][9] = 58,
+ [1][0][0][0][RTW89_FCC][10] = 127,
+ [1][0][0][0][RTW89_ETSI][10] = 58,
+ [1][0][0][0][RTW89_MKK][10] = 66,
+ [1][0][0][0][RTW89_IC][10] = 127,
+ [1][0][0][0][RTW89_KCC][10] = 68,
+ [1][0][0][0][RTW89_ACMA][10] = 58,
+ [1][0][0][0][RTW89_CN][10] = 60,
+ [1][0][0][0][RTW89_UK][10] = 58,
+ [1][0][0][0][RTW89_FCC][11] = 127,
+ [1][0][0][0][RTW89_ETSI][11] = 127,
+ [1][0][0][0][RTW89_MKK][11] = 127,
+ [1][0][0][0][RTW89_IC][11] = 127,
+ [1][0][0][0][RTW89_KCC][11] = 127,
+ [1][0][0][0][RTW89_ACMA][11] = 127,
+ [1][0][0][0][RTW89_CN][11] = 127,
+ [1][0][0][0][RTW89_UK][11] = 127,
+ [1][0][0][0][RTW89_FCC][12] = 127,
+ [1][0][0][0][RTW89_ETSI][12] = 127,
+ [1][0][0][0][RTW89_MKK][12] = 127,
+ [1][0][0][0][RTW89_IC][12] = 127,
+ [1][0][0][0][RTW89_KCC][12] = 127,
+ [1][0][0][0][RTW89_ACMA][12] = 127,
+ [1][0][0][0][RTW89_CN][12] = 127,
+ [1][0][0][0][RTW89_UK][12] = 127,
+ [1][0][0][0][RTW89_FCC][13] = 127,
+ [1][0][0][0][RTW89_ETSI][13] = 127,
+ [1][0][0][0][RTW89_MKK][13] = 127,
+ [1][0][0][0][RTW89_IC][13] = 127,
+ [1][0][0][0][RTW89_KCC][13] = 127,
+ [1][0][0][0][RTW89_ACMA][13] = 127,
+ [1][0][0][0][RTW89_CN][13] = 127,
+ [1][0][0][0][RTW89_UK][13] = 127,
+ [1][1][0][0][RTW89_FCC][0] = 127,
+ [1][1][0][0][RTW89_ETSI][0] = 127,
+ [1][1][0][0][RTW89_MKK][0] = 127,
+ [1][1][0][0][RTW89_IC][0] = 127,
+ [1][1][0][0][RTW89_KCC][0] = 127,
+ [1][1][0][0][RTW89_ACMA][0] = 127,
+ [1][1][0][0][RTW89_CN][0] = 127,
+ [1][1][0][0][RTW89_UK][0] = 127,
+ [1][1][0][0][RTW89_FCC][1] = 127,
+ [1][1][0][0][RTW89_ETSI][1] = 127,
+ [1][1][0][0][RTW89_MKK][1] = 127,
+ [1][1][0][0][RTW89_IC][1] = 127,
+ [1][1][0][0][RTW89_KCC][1] = 127,
+ [1][1][0][0][RTW89_ACMA][1] = 127,
+ [1][1][0][0][RTW89_CN][1] = 127,
+ [1][1][0][0][RTW89_UK][1] = 127,
+ [1][1][0][0][RTW89_FCC][2] = 127,
+ [1][1][0][0][RTW89_ETSI][2] = 127,
+ [1][1][0][0][RTW89_MKK][2] = 127,
+ [1][1][0][0][RTW89_IC][2] = 127,
+ [1][1][0][0][RTW89_KCC][2] = 127,
+ [1][1][0][0][RTW89_ACMA][2] = 127,
+ [1][1][0][0][RTW89_CN][2] = 127,
+ [1][1][0][0][RTW89_UK][2] = 127,
+ [1][1][0][0][RTW89_FCC][3] = 127,
+ [1][1][0][0][RTW89_ETSI][3] = 127,
+ [1][1][0][0][RTW89_MKK][3] = 127,
+ [1][1][0][0][RTW89_IC][3] = 127,
+ [1][1][0][0][RTW89_KCC][3] = 127,
+ [1][1][0][0][RTW89_ACMA][3] = 127,
+ [1][1][0][0][RTW89_CN][3] = 127,
+ [1][1][0][0][RTW89_UK][3] = 127,
+ [1][1][0][0][RTW89_FCC][4] = 127,
+ [1][1][0][0][RTW89_ETSI][4] = 127,
+ [1][1][0][0][RTW89_MKK][4] = 127,
+ [1][1][0][0][RTW89_IC][4] = 127,
+ [1][1][0][0][RTW89_KCC][4] = 127,
+ [1][1][0][0][RTW89_ACMA][4] = 127,
+ [1][1][0][0][RTW89_CN][4] = 127,
+ [1][1][0][0][RTW89_UK][4] = 127,
+ [1][1][0][0][RTW89_FCC][5] = 127,
+ [1][1][0][0][RTW89_ETSI][5] = 127,
+ [1][1][0][0][RTW89_MKK][5] = 127,
+ [1][1][0][0][RTW89_IC][5] = 127,
+ [1][1][0][0][RTW89_KCC][5] = 127,
+ [1][1][0][0][RTW89_ACMA][5] = 127,
+ [1][1][0][0][RTW89_CN][5] = 127,
+ [1][1][0][0][RTW89_UK][5] = 127,
+ [1][1][0][0][RTW89_FCC][6] = 127,
+ [1][1][0][0][RTW89_ETSI][6] = 127,
+ [1][1][0][0][RTW89_MKK][6] = 127,
+ [1][1][0][0][RTW89_IC][6] = 127,
+ [1][1][0][0][RTW89_KCC][6] = 127,
+ [1][1][0][0][RTW89_ACMA][6] = 127,
+ [1][1][0][0][RTW89_CN][6] = 127,
+ [1][1][0][0][RTW89_UK][6] = 127,
+ [1][1][0][0][RTW89_FCC][7] = 127,
+ [1][1][0][0][RTW89_ETSI][7] = 127,
+ [1][1][0][0][RTW89_MKK][7] = 127,
+ [1][1][0][0][RTW89_IC][7] = 127,
+ [1][1][0][0][RTW89_KCC][7] = 127,
+ [1][1][0][0][RTW89_ACMA][7] = 127,
+ [1][1][0][0][RTW89_CN][7] = 127,
+ [1][1][0][0][RTW89_UK][7] = 127,
+ [1][1][0][0][RTW89_FCC][8] = 127,
+ [1][1][0][0][RTW89_ETSI][8] = 127,
+ [1][1][0][0][RTW89_MKK][8] = 127,
+ [1][1][0][0][RTW89_IC][8] = 127,
+ [1][1][0][0][RTW89_KCC][8] = 127,
+ [1][1][0][0][RTW89_ACMA][8] = 127,
+ [1][1][0][0][RTW89_CN][8] = 127,
+ [1][1][0][0][RTW89_UK][8] = 127,
+ [1][1][0][0][RTW89_FCC][9] = 127,
+ [1][1][0][0][RTW89_ETSI][9] = 127,
+ [1][1][0][0][RTW89_MKK][9] = 127,
+ [1][1][0][0][RTW89_IC][9] = 127,
+ [1][1][0][0][RTW89_KCC][9] = 127,
+ [1][1][0][0][RTW89_ACMA][9] = 127,
+ [1][1][0][0][RTW89_CN][9] = 127,
+ [1][1][0][0][RTW89_UK][9] = 127,
+ [1][1][0][0][RTW89_FCC][10] = 127,
+ [1][1][0][0][RTW89_ETSI][10] = 127,
+ [1][1][0][0][RTW89_MKK][10] = 127,
+ [1][1][0][0][RTW89_IC][10] = 127,
+ [1][1][0][0][RTW89_KCC][10] = 127,
+ [1][1][0][0][RTW89_ACMA][10] = 127,
+ [1][1][0][0][RTW89_CN][10] = 127,
+ [1][1][0][0][RTW89_UK][10] = 127,
+ [1][1][0][0][RTW89_FCC][11] = 127,
+ [1][1][0][0][RTW89_ETSI][11] = 127,
+ [1][1][0][0][RTW89_MKK][11] = 127,
+ [1][1][0][0][RTW89_IC][11] = 127,
+ [1][1][0][0][RTW89_KCC][11] = 127,
+ [1][1][0][0][RTW89_ACMA][11] = 127,
+ [1][1][0][0][RTW89_CN][11] = 127,
+ [1][1][0][0][RTW89_UK][11] = 127,
+ [1][1][0][0][RTW89_FCC][12] = 127,
+ [1][1][0][0][RTW89_ETSI][12] = 127,
+ [1][1][0][0][RTW89_MKK][12] = 127,
+ [1][1][0][0][RTW89_IC][12] = 127,
+ [1][1][0][0][RTW89_KCC][12] = 127,
+ [1][1][0][0][RTW89_ACMA][12] = 127,
+ [1][1][0][0][RTW89_CN][12] = 127,
+ [1][1][0][0][RTW89_UK][12] = 127,
+ [1][1][0][0][RTW89_FCC][13] = 127,
+ [1][1][0][0][RTW89_ETSI][13] = 127,
+ [1][1][0][0][RTW89_MKK][13] = 127,
+ [1][1][0][0][RTW89_IC][13] = 127,
+ [1][1][0][0][RTW89_KCC][13] = 127,
+ [1][1][0][0][RTW89_ACMA][13] = 127,
+ [1][1][0][0][RTW89_CN][13] = 127,
+ [1][1][0][0][RTW89_UK][13] = 127,
+ [0][0][1][0][RTW89_FCC][0] = 78,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 72,
+ [0][0][1][0][RTW89_IC][0] = 78,
+ [0][0][1][0][RTW89_KCC][0] = 76,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CN][0] = 60,
+ [0][0][1][0][RTW89_UK][0] = 58,
+ [0][0][1][0][RTW89_FCC][1] = 78,
+ [0][0][1][0][RTW89_ETSI][1] = 60,
+ [0][0][1][0][RTW89_MKK][1] = 74,
+ [0][0][1][0][RTW89_IC][1] = 78,
+ [0][0][1][0][RTW89_KCC][1] = 76,
+ [0][0][1][0][RTW89_ACMA][1] = 60,
+ [0][0][1][0][RTW89_CN][1] = 60,
+ [0][0][1][0][RTW89_UK][1] = 60,
+ [0][0][1][0][RTW89_FCC][2] = 80,
+ [0][0][1][0][RTW89_ETSI][2] = 60,
+ [0][0][1][0][RTW89_MKK][2] = 74,
+ [0][0][1][0][RTW89_IC][2] = 80,
+ [0][0][1][0][RTW89_KCC][2] = 76,
+ [0][0][1][0][RTW89_ACMA][2] = 60,
+ [0][0][1][0][RTW89_CN][2] = 60,
+ [0][0][1][0][RTW89_UK][2] = 60,
+ [0][0][1][0][RTW89_FCC][3] = 80,
+ [0][0][1][0][RTW89_ETSI][3] = 60,
+ [0][0][1][0][RTW89_MKK][3] = 74,
+ [0][0][1][0][RTW89_IC][3] = 80,
+ [0][0][1][0][RTW89_KCC][3] = 76,
+ [0][0][1][0][RTW89_ACMA][3] = 60,
+ [0][0][1][0][RTW89_CN][3] = 60,
+ [0][0][1][0][RTW89_UK][3] = 60,
+ [0][0][1][0][RTW89_FCC][4] = 80,
+ [0][0][1][0][RTW89_ETSI][4] = 60,
+ [0][0][1][0][RTW89_MKK][4] = 74,
+ [0][0][1][0][RTW89_IC][4] = 80,
+ [0][0][1][0][RTW89_KCC][4] = 76,
+ [0][0][1][0][RTW89_ACMA][4] = 60,
+ [0][0][1][0][RTW89_CN][4] = 60,
+ [0][0][1][0][RTW89_UK][4] = 60,
+ [0][0][1][0][RTW89_FCC][5] = 80,
+ [0][0][1][0][RTW89_ETSI][5] = 60,
+ [0][0][1][0][RTW89_MKK][5] = 74,
+ [0][0][1][0][RTW89_IC][5] = 80,
+ [0][0][1][0][RTW89_KCC][5] = 76,
+ [0][0][1][0][RTW89_ACMA][5] = 60,
+ [0][0][1][0][RTW89_CN][5] = 60,
+ [0][0][1][0][RTW89_UK][5] = 60,
+ [0][0][1][0][RTW89_FCC][6] = 80,
+ [0][0][1][0][RTW89_ETSI][6] = 60,
+ [0][0][1][0][RTW89_MKK][6] = 74,
+ [0][0][1][0][RTW89_IC][6] = 80,
+ [0][0][1][0][RTW89_KCC][6] = 76,
+ [0][0][1][0][RTW89_ACMA][6] = 60,
+ [0][0][1][0][RTW89_CN][6] = 60,
+ [0][0][1][0][RTW89_UK][6] = 60,
+ [0][0][1][0][RTW89_FCC][7] = 80,
+ [0][0][1][0][RTW89_ETSI][7] = 60,
+ [0][0][1][0][RTW89_MKK][7] = 74,
+ [0][0][1][0][RTW89_IC][7] = 80,
+ [0][0][1][0][RTW89_KCC][7] = 76,
+ [0][0][1][0][RTW89_ACMA][7] = 60,
+ [0][0][1][0][RTW89_CN][7] = 60,
+ [0][0][1][0][RTW89_UK][7] = 60,
+ [0][0][1][0][RTW89_FCC][8] = 80,
+ [0][0][1][0][RTW89_ETSI][8] = 60,
+ [0][0][1][0][RTW89_MKK][8] = 74,
+ [0][0][1][0][RTW89_IC][8] = 80,
+ [0][0][1][0][RTW89_KCC][8] = 76,
+ [0][0][1][0][RTW89_ACMA][8] = 60,
+ [0][0][1][0][RTW89_CN][8] = 60,
+ [0][0][1][0][RTW89_UK][8] = 60,
+ [0][0][1][0][RTW89_FCC][9] = 76,
+ [0][0][1][0][RTW89_ETSI][9] = 60,
+ [0][0][1][0][RTW89_MKK][9] = 74,
+ [0][0][1][0][RTW89_IC][9] = 76,
+ [0][0][1][0][RTW89_KCC][9] = 74,
+ [0][0][1][0][RTW89_ACMA][9] = 60,
+ [0][0][1][0][RTW89_CN][9] = 60,
+ [0][0][1][0][RTW89_UK][9] = 60,
+ [0][0][1][0][RTW89_FCC][10] = 76,
+ [0][0][1][0][RTW89_ETSI][10] = 60,
+ [0][0][1][0][RTW89_MKK][10] = 74,
+ [0][0][1][0][RTW89_IC][10] = 76,
+ [0][0][1][0][RTW89_KCC][10] = 74,
+ [0][0][1][0][RTW89_ACMA][10] = 60,
+ [0][0][1][0][RTW89_CN][10] = 60,
+ [0][0][1][0][RTW89_UK][10] = 60,
+ [0][0][1][0][RTW89_FCC][11] = 68,
+ [0][0][1][0][RTW89_ETSI][11] = 60,
+ [0][0][1][0][RTW89_MKK][11] = 74,
+ [0][0][1][0][RTW89_IC][11] = 68,
+ [0][0][1][0][RTW89_KCC][11] = 74,
+ [0][0][1][0][RTW89_ACMA][11] = 60,
+ [0][0][1][0][RTW89_CN][11] = 60,
+ [0][0][1][0][RTW89_UK][11] = 60,
+ [0][0][1][0][RTW89_FCC][12] = 64,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 70,
+ [0][0][1][0][RTW89_IC][12] = 64,
+ [0][0][1][0][RTW89_KCC][12] = 74,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CN][12] = 60,
+ [0][0][1][0][RTW89_UK][12] = 58,
+ [0][0][1][0][RTW89_FCC][13] = 127,
+ [0][0][1][0][RTW89_ETSI][13] = 127,
+ [0][0][1][0][RTW89_MKK][13] = 127,
+ [0][0][1][0][RTW89_IC][13] = 127,
+ [0][0][1][0][RTW89_KCC][13] = 127,
+ [0][0][1][0][RTW89_ACMA][13] = 127,
+ [0][0][1][0][RTW89_CN][13] = 127,
+ [0][0][1][0][RTW89_UK][13] = 127,
+ [0][1][1][0][RTW89_FCC][0] = 127,
+ [0][1][1][0][RTW89_ETSI][0] = 127,
+ [0][1][1][0][RTW89_MKK][0] = 127,
+ [0][1][1][0][RTW89_IC][0] = 127,
+ [0][1][1][0][RTW89_KCC][0] = 127,
+ [0][1][1][0][RTW89_ACMA][0] = 127,
+ [0][1][1][0][RTW89_CN][0] = 127,
+ [0][1][1][0][RTW89_UK][0] = 127,
+ [0][1][1][0][RTW89_FCC][1] = 127,
+ [0][1][1][0][RTW89_ETSI][1] = 127,
+ [0][1][1][0][RTW89_MKK][1] = 127,
+ [0][1][1][0][RTW89_IC][1] = 127,
+ [0][1][1][0][RTW89_KCC][1] = 127,
+ [0][1][1][0][RTW89_ACMA][1] = 127,
+ [0][1][1][0][RTW89_CN][1] = 127,
+ [0][1][1][0][RTW89_UK][1] = 127,
+ [0][1][1][0][RTW89_FCC][2] = 127,
+ [0][1][1][0][RTW89_ETSI][2] = 127,
+ [0][1][1][0][RTW89_MKK][2] = 127,
+ [0][1][1][0][RTW89_IC][2] = 127,
+ [0][1][1][0][RTW89_KCC][2] = 127,
+ [0][1][1][0][RTW89_ACMA][2] = 127,
+ [0][1][1][0][RTW89_CN][2] = 127,
+ [0][1][1][0][RTW89_UK][2] = 127,
+ [0][1][1][0][RTW89_FCC][3] = 127,
+ [0][1][1][0][RTW89_ETSI][3] = 127,
+ [0][1][1][0][RTW89_MKK][3] = 127,
+ [0][1][1][0][RTW89_IC][3] = 127,
+ [0][1][1][0][RTW89_KCC][3] = 127,
+ [0][1][1][0][RTW89_ACMA][3] = 127,
+ [0][1][1][0][RTW89_CN][3] = 127,
+ [0][1][1][0][RTW89_UK][3] = 127,
+ [0][1][1][0][RTW89_FCC][4] = 127,
+ [0][1][1][0][RTW89_ETSI][4] = 127,
+ [0][1][1][0][RTW89_MKK][4] = 127,
+ [0][1][1][0][RTW89_IC][4] = 127,
+ [0][1][1][0][RTW89_KCC][4] = 127,
+ [0][1][1][0][RTW89_ACMA][4] = 127,
+ [0][1][1][0][RTW89_CN][4] = 127,
+ [0][1][1][0][RTW89_UK][4] = 127,
+ [0][1][1][0][RTW89_FCC][5] = 127,
+ [0][1][1][0][RTW89_ETSI][5] = 127,
+ [0][1][1][0][RTW89_MKK][5] = 127,
+ [0][1][1][0][RTW89_IC][5] = 127,
+ [0][1][1][0][RTW89_KCC][5] = 127,
+ [0][1][1][0][RTW89_ACMA][5] = 127,
+ [0][1][1][0][RTW89_CN][5] = 127,
+ [0][1][1][0][RTW89_UK][5] = 127,
+ [0][1][1][0][RTW89_FCC][6] = 127,
+ [0][1][1][0][RTW89_ETSI][6] = 127,
+ [0][1][1][0][RTW89_MKK][6] = 127,
+ [0][1][1][0][RTW89_IC][6] = 127,
+ [0][1][1][0][RTW89_KCC][6] = 127,
+ [0][1][1][0][RTW89_ACMA][6] = 127,
+ [0][1][1][0][RTW89_CN][6] = 127,
+ [0][1][1][0][RTW89_UK][6] = 127,
+ [0][1][1][0][RTW89_FCC][7] = 127,
+ [0][1][1][0][RTW89_ETSI][7] = 127,
+ [0][1][1][0][RTW89_MKK][7] = 127,
+ [0][1][1][0][RTW89_IC][7] = 127,
+ [0][1][1][0][RTW89_KCC][7] = 127,
+ [0][1][1][0][RTW89_ACMA][7] = 127,
+ [0][1][1][0][RTW89_CN][7] = 127,
+ [0][1][1][0][RTW89_UK][7] = 127,
+ [0][1][1][0][RTW89_FCC][8] = 127,
+ [0][1][1][0][RTW89_ETSI][8] = 127,
+ [0][1][1][0][RTW89_MKK][8] = 127,
+ [0][1][1][0][RTW89_IC][8] = 127,
+ [0][1][1][0][RTW89_KCC][8] = 127,
+ [0][1][1][0][RTW89_ACMA][8] = 127,
+ [0][1][1][0][RTW89_CN][8] = 127,
+ [0][1][1][0][RTW89_UK][8] = 127,
+ [0][1][1][0][RTW89_FCC][9] = 127,
+ [0][1][1][0][RTW89_ETSI][9] = 127,
+ [0][1][1][0][RTW89_MKK][9] = 127,
+ [0][1][1][0][RTW89_IC][9] = 127,
+ [0][1][1][0][RTW89_KCC][9] = 127,
+ [0][1][1][0][RTW89_ACMA][9] = 127,
+ [0][1][1][0][RTW89_CN][9] = 127,
+ [0][1][1][0][RTW89_UK][9] = 127,
+ [0][1][1][0][RTW89_FCC][10] = 127,
+ [0][1][1][0][RTW89_ETSI][10] = 127,
+ [0][1][1][0][RTW89_MKK][10] = 127,
+ [0][1][1][0][RTW89_IC][10] = 127,
+ [0][1][1][0][RTW89_KCC][10] = 127,
+ [0][1][1][0][RTW89_ACMA][10] = 127,
+ [0][1][1][0][RTW89_CN][10] = 127,
+ [0][1][1][0][RTW89_UK][10] = 127,
+ [0][1][1][0][RTW89_FCC][11] = 127,
+ [0][1][1][0][RTW89_ETSI][11] = 127,
+ [0][1][1][0][RTW89_MKK][11] = 127,
+ [0][1][1][0][RTW89_IC][11] = 127,
+ [0][1][1][0][RTW89_KCC][11] = 127,
+ [0][1][1][0][RTW89_ACMA][11] = 127,
+ [0][1][1][0][RTW89_CN][11] = 127,
+ [0][1][1][0][RTW89_UK][11] = 127,
+ [0][1][1][0][RTW89_FCC][12] = 127,
+ [0][1][1][0][RTW89_ETSI][12] = 127,
+ [0][1][1][0][RTW89_MKK][12] = 127,
+ [0][1][1][0][RTW89_IC][12] = 127,
+ [0][1][1][0][RTW89_KCC][12] = 127,
+ [0][1][1][0][RTW89_ACMA][12] = 127,
+ [0][1][1][0][RTW89_CN][12] = 127,
+ [0][1][1][0][RTW89_UK][12] = 127,
+ [0][1][1][0][RTW89_FCC][13] = 127,
+ [0][1][1][0][RTW89_ETSI][13] = 127,
+ [0][1][1][0][RTW89_MKK][13] = 127,
+ [0][1][1][0][RTW89_IC][13] = 127,
+ [0][1][1][0][RTW89_KCC][13] = 127,
+ [0][1][1][0][RTW89_ACMA][13] = 127,
+ [0][1][1][0][RTW89_CN][13] = 127,
+ [0][1][1][0][RTW89_UK][13] = 127,
+ [0][0][2][0][RTW89_FCC][0] = 78,
+ [0][0][2][0][RTW89_ETSI][0] = 60,
+ [0][0][2][0][RTW89_MKK][0] = 72,
+ [0][0][2][0][RTW89_IC][0] = 78,
+ [0][0][2][0][RTW89_KCC][0] = 76,
+ [0][0][2][0][RTW89_ACMA][0] = 60,
+ [0][0][2][0][RTW89_CN][0] = 60,
+ [0][0][2][0][RTW89_UK][0] = 60,
+ [0][0][2][0][RTW89_FCC][1] = 78,
+ [0][0][2][0][RTW89_ETSI][1] = 60,
+ [0][0][2][0][RTW89_MKK][1] = 76,
+ [0][0][2][0][RTW89_IC][1] = 78,
+ [0][0][2][0][RTW89_KCC][1] = 76,
+ [0][0][2][0][RTW89_ACMA][1] = 60,
+ [0][0][2][0][RTW89_CN][1] = 60,
+ [0][0][2][0][RTW89_UK][1] = 60,
+ [0][0][2][0][RTW89_FCC][2] = 80,
+ [0][0][2][0][RTW89_ETSI][2] = 60,
+ [0][0][2][0][RTW89_MKK][2] = 76,
+ [0][0][2][0][RTW89_IC][2] = 80,
+ [0][0][2][0][RTW89_KCC][2] = 76,
+ [0][0][2][0][RTW89_ACMA][2] = 60,
+ [0][0][2][0][RTW89_CN][2] = 60,
+ [0][0][2][0][RTW89_UK][2] = 60,
+ [0][0][2][0][RTW89_FCC][3] = 80,
+ [0][0][2][0][RTW89_ETSI][3] = 60,
+ [0][0][2][0][RTW89_MKK][3] = 76,
+ [0][0][2][0][RTW89_IC][3] = 80,
+ [0][0][2][0][RTW89_KCC][3] = 76,
+ [0][0][2][0][RTW89_ACMA][3] = 60,
+ [0][0][2][0][RTW89_CN][3] = 60,
+ [0][0][2][0][RTW89_UK][3] = 60,
+ [0][0][2][0][RTW89_FCC][4] = 80,
+ [0][0][2][0][RTW89_ETSI][4] = 60,
+ [0][0][2][0][RTW89_MKK][4] = 76,
+ [0][0][2][0][RTW89_IC][4] = 80,
+ [0][0][2][0][RTW89_KCC][4] = 76,
+ [0][0][2][0][RTW89_ACMA][4] = 60,
+ [0][0][2][0][RTW89_CN][4] = 60,
+ [0][0][2][0][RTW89_UK][4] = 60,
+ [0][0][2][0][RTW89_FCC][5] = 80,
+ [0][0][2][0][RTW89_ETSI][5] = 60,
+ [0][0][2][0][RTW89_MKK][5] = 76,
+ [0][0][2][0][RTW89_IC][5] = 80,
+ [0][0][2][0][RTW89_KCC][5] = 76,
+ [0][0][2][0][RTW89_ACMA][5] = 60,
+ [0][0][2][0][RTW89_CN][5] = 60,
+ [0][0][2][0][RTW89_UK][5] = 60,
+ [0][0][2][0][RTW89_FCC][6] = 80,
+ [0][0][2][0][RTW89_ETSI][6] = 60,
+ [0][0][2][0][RTW89_MKK][6] = 76,
+ [0][0][2][0][RTW89_IC][6] = 80,
+ [0][0][2][0][RTW89_KCC][6] = 76,
+ [0][0][2][0][RTW89_ACMA][6] = 60,
+ [0][0][2][0][RTW89_CN][6] = 60,
+ [0][0][2][0][RTW89_UK][6] = 60,
+ [0][0][2][0][RTW89_FCC][7] = 80,
+ [0][0][2][0][RTW89_ETSI][7] = 60,
+ [0][0][2][0][RTW89_MKK][7] = 76,
+ [0][0][2][0][RTW89_IC][7] = 80,
+ [0][0][2][0][RTW89_KCC][7] = 76,
+ [0][0][2][0][RTW89_ACMA][7] = 60,
+ [0][0][2][0][RTW89_CN][7] = 60,
+ [0][0][2][0][RTW89_UK][7] = 60,
+ [0][0][2][0][RTW89_FCC][8] = 78,
+ [0][0][2][0][RTW89_ETSI][8] = 60,
+ [0][0][2][0][RTW89_MKK][8] = 76,
+ [0][0][2][0][RTW89_IC][8] = 78,
+ [0][0][2][0][RTW89_KCC][8] = 76,
+ [0][0][2][0][RTW89_ACMA][8] = 60,
+ [0][0][2][0][RTW89_CN][8] = 60,
+ [0][0][2][0][RTW89_UK][8] = 60,
+ [0][0][2][0][RTW89_FCC][9] = 74,
+ [0][0][2][0][RTW89_ETSI][9] = 60,
+ [0][0][2][0][RTW89_MKK][9] = 76,
+ [0][0][2][0][RTW89_IC][9] = 74,
+ [0][0][2][0][RTW89_KCC][9] = 76,
+ [0][0][2][0][RTW89_ACMA][9] = 60,
+ [0][0][2][0][RTW89_CN][9] = 60,
+ [0][0][2][0][RTW89_UK][9] = 60,
+ [0][0][2][0][RTW89_FCC][10] = 74,
+ [0][0][2][0][RTW89_ETSI][10] = 60,
+ [0][0][2][0][RTW89_MKK][10] = 76,
+ [0][0][2][0][RTW89_IC][10] = 74,
+ [0][0][2][0][RTW89_KCC][10] = 76,
+ [0][0][2][0][RTW89_ACMA][10] = 60,
+ [0][0][2][0][RTW89_CN][10] = 60,
+ [0][0][2][0][RTW89_UK][10] = 60,
+ [0][0][2][0][RTW89_FCC][11] = 68,
+ [0][0][2][0][RTW89_ETSI][11] = 60,
+ [0][0][2][0][RTW89_MKK][11] = 76,
+ [0][0][2][0][RTW89_IC][11] = 68,
+ [0][0][2][0][RTW89_KCC][11] = 76,
+ [0][0][2][0][RTW89_ACMA][11] = 60,
+ [0][0][2][0][RTW89_CN][11] = 60,
+ [0][0][2][0][RTW89_UK][11] = 60,
+ [0][0][2][0][RTW89_FCC][12] = 68,
+ [0][0][2][0][RTW89_ETSI][12] = 60,
+ [0][0][2][0][RTW89_MKK][12] = 70,
+ [0][0][2][0][RTW89_IC][12] = 68,
+ [0][0][2][0][RTW89_KCC][12] = 76,
+ [0][0][2][0][RTW89_ACMA][12] = 60,
+ [0][0][2][0][RTW89_CN][12] = 60,
+ [0][0][2][0][RTW89_UK][12] = 60,
+ [0][0][2][0][RTW89_FCC][13] = 127,
+ [0][0][2][0][RTW89_ETSI][13] = 127,
+ [0][0][2][0][RTW89_MKK][13] = 127,
+ [0][0][2][0][RTW89_IC][13] = 127,
+ [0][0][2][0][RTW89_KCC][13] = 127,
+ [0][0][2][0][RTW89_ACMA][13] = 127,
+ [0][0][2][0][RTW89_CN][13] = 127,
+ [0][0][2][0][RTW89_UK][13] = 127,
+ [0][1][2][0][RTW89_FCC][0] = 127,
+ [0][1][2][0][RTW89_ETSI][0] = 127,
+ [0][1][2][0][RTW89_MKK][0] = 127,
+ [0][1][2][0][RTW89_IC][0] = 127,
+ [0][1][2][0][RTW89_KCC][0] = 127,
+ [0][1][2][0][RTW89_ACMA][0] = 127,
+ [0][1][2][0][RTW89_CN][0] = 127,
+ [0][1][2][0][RTW89_UK][0] = 127,
+ [0][1][2][0][RTW89_FCC][1] = 127,
+ [0][1][2][0][RTW89_ETSI][1] = 127,
+ [0][1][2][0][RTW89_MKK][1] = 127,
+ [0][1][2][0][RTW89_IC][1] = 127,
+ [0][1][2][0][RTW89_KCC][1] = 127,
+ [0][1][2][0][RTW89_ACMA][1] = 127,
+ [0][1][2][0][RTW89_CN][1] = 127,
+ [0][1][2][0][RTW89_UK][1] = 127,
+ [0][1][2][0][RTW89_FCC][2] = 127,
+ [0][1][2][0][RTW89_ETSI][2] = 127,
+ [0][1][2][0][RTW89_MKK][2] = 127,
+ [0][1][2][0][RTW89_IC][2] = 127,
+ [0][1][2][0][RTW89_KCC][2] = 127,
+ [0][1][2][0][RTW89_ACMA][2] = 127,
+ [0][1][2][0][RTW89_CN][2] = 127,
+ [0][1][2][0][RTW89_UK][2] = 127,
+ [0][1][2][0][RTW89_FCC][3] = 127,
+ [0][1][2][0][RTW89_ETSI][3] = 127,
+ [0][1][2][0][RTW89_MKK][3] = 127,
+ [0][1][2][0][RTW89_IC][3] = 127,
+ [0][1][2][0][RTW89_KCC][3] = 127,
+ [0][1][2][0][RTW89_ACMA][3] = 127,
+ [0][1][2][0][RTW89_CN][3] = 127,
+ [0][1][2][0][RTW89_UK][3] = 127,
+ [0][1][2][0][RTW89_FCC][4] = 127,
+ [0][1][2][0][RTW89_ETSI][4] = 127,
+ [0][1][2][0][RTW89_MKK][4] = 127,
+ [0][1][2][0][RTW89_IC][4] = 127,
+ [0][1][2][0][RTW89_KCC][4] = 127,
+ [0][1][2][0][RTW89_ACMA][4] = 127,
+ [0][1][2][0][RTW89_CN][4] = 127,
+ [0][1][2][0][RTW89_UK][4] = 127,
+ [0][1][2][0][RTW89_FCC][5] = 127,
+ [0][1][2][0][RTW89_ETSI][5] = 127,
+ [0][1][2][0][RTW89_MKK][5] = 127,
+ [0][1][2][0][RTW89_IC][5] = 127,
+ [0][1][2][0][RTW89_KCC][5] = 127,
+ [0][1][2][0][RTW89_ACMA][5] = 127,
+ [0][1][2][0][RTW89_CN][5] = 127,
+ [0][1][2][0][RTW89_UK][5] = 127,
+ [0][1][2][0][RTW89_FCC][6] = 127,
+ [0][1][2][0][RTW89_ETSI][6] = 127,
+ [0][1][2][0][RTW89_MKK][6] = 127,
+ [0][1][2][0][RTW89_IC][6] = 127,
+ [0][1][2][0][RTW89_KCC][6] = 127,
+ [0][1][2][0][RTW89_ACMA][6] = 127,
+ [0][1][2][0][RTW89_CN][6] = 127,
+ [0][1][2][0][RTW89_UK][6] = 127,
+ [0][1][2][0][RTW89_FCC][7] = 127,
+ [0][1][2][0][RTW89_ETSI][7] = 127,
+ [0][1][2][0][RTW89_MKK][7] = 127,
+ [0][1][2][0][RTW89_IC][7] = 127,
+ [0][1][2][0][RTW89_KCC][7] = 127,
+ [0][1][2][0][RTW89_ACMA][7] = 127,
+ [0][1][2][0][RTW89_CN][7] = 127,
+ [0][1][2][0][RTW89_UK][7] = 127,
+ [0][1][2][0][RTW89_FCC][8] = 127,
+ [0][1][2][0][RTW89_ETSI][8] = 127,
+ [0][1][2][0][RTW89_MKK][8] = 127,
+ [0][1][2][0][RTW89_IC][8] = 127,
+ [0][1][2][0][RTW89_KCC][8] = 127,
+ [0][1][2][0][RTW89_ACMA][8] = 127,
+ [0][1][2][0][RTW89_CN][8] = 127,
+ [0][1][2][0][RTW89_UK][8] = 127,
+ [0][1][2][0][RTW89_FCC][9] = 127,
+ [0][1][2][0][RTW89_ETSI][9] = 127,
+ [0][1][2][0][RTW89_MKK][9] = 127,
+ [0][1][2][0][RTW89_IC][9] = 127,
+ [0][1][2][0][RTW89_KCC][9] = 127,
+ [0][1][2][0][RTW89_ACMA][9] = 127,
+ [0][1][2][0][RTW89_CN][9] = 127,
+ [0][1][2][0][RTW89_UK][9] = 127,
+ [0][1][2][0][RTW89_FCC][10] = 127,
+ [0][1][2][0][RTW89_ETSI][10] = 127,
+ [0][1][2][0][RTW89_MKK][10] = 127,
+ [0][1][2][0][RTW89_IC][10] = 127,
+ [0][1][2][0][RTW89_KCC][10] = 127,
+ [0][1][2][0][RTW89_ACMA][10] = 127,
+ [0][1][2][0][RTW89_CN][10] = 127,
+ [0][1][2][0][RTW89_UK][10] = 127,
+ [0][1][2][0][RTW89_FCC][11] = 127,
+ [0][1][2][0][RTW89_ETSI][11] = 127,
+ [0][1][2][0][RTW89_MKK][11] = 127,
+ [0][1][2][0][RTW89_IC][11] = 127,
+ [0][1][2][0][RTW89_KCC][11] = 127,
+ [0][1][2][0][RTW89_ACMA][11] = 127,
+ [0][1][2][0][RTW89_CN][11] = 127,
+ [0][1][2][0][RTW89_UK][11] = 127,
+ [0][1][2][0][RTW89_FCC][12] = 127,
+ [0][1][2][0][RTW89_ETSI][12] = 127,
+ [0][1][2][0][RTW89_MKK][12] = 127,
+ [0][1][2][0][RTW89_IC][12] = 127,
+ [0][1][2][0][RTW89_KCC][12] = 127,
+ [0][1][2][0][RTW89_ACMA][12] = 127,
+ [0][1][2][0][RTW89_CN][12] = 127,
+ [0][1][2][0][RTW89_UK][12] = 127,
+ [0][1][2][0][RTW89_FCC][13] = 127,
+ [0][1][2][0][RTW89_ETSI][13] = 127,
+ [0][1][2][0][RTW89_MKK][13] = 127,
+ [0][1][2][0][RTW89_IC][13] = 127,
+ [0][1][2][0][RTW89_KCC][13] = 127,
+ [0][1][2][0][RTW89_ACMA][13] = 127,
+ [0][1][2][0][RTW89_CN][13] = 127,
+ [0][1][2][0][RTW89_UK][13] = 127,
+ [0][1][2][1][RTW89_FCC][0] = 127,
+ [0][1][2][1][RTW89_ETSI][0] = 127,
+ [0][1][2][1][RTW89_MKK][0] = 127,
+ [0][1][2][1][RTW89_IC][0] = 127,
+ [0][1][2][1][RTW89_KCC][0] = 127,
+ [0][1][2][1][RTW89_ACMA][0] = 127,
+ [0][1][2][1][RTW89_CN][0] = 127,
+ [0][1][2][1][RTW89_UK][0] = 127,
+ [0][1][2][1][RTW89_FCC][1] = 127,
+ [0][1][2][1][RTW89_ETSI][1] = 127,
+ [0][1][2][1][RTW89_MKK][1] = 127,
+ [0][1][2][1][RTW89_IC][1] = 127,
+ [0][1][2][1][RTW89_KCC][1] = 127,
+ [0][1][2][1][RTW89_ACMA][1] = 127,
+ [0][1][2][1][RTW89_CN][1] = 127,
+ [0][1][2][1][RTW89_UK][1] = 127,
+ [0][1][2][1][RTW89_FCC][2] = 127,
+ [0][1][2][1][RTW89_ETSI][2] = 127,
+ [0][1][2][1][RTW89_MKK][2] = 127,
+ [0][1][2][1][RTW89_IC][2] = 127,
+ [0][1][2][1][RTW89_KCC][2] = 127,
+ [0][1][2][1][RTW89_ACMA][2] = 127,
+ [0][1][2][1][RTW89_CN][2] = 127,
+ [0][1][2][1][RTW89_UK][2] = 127,
+ [0][1][2][1][RTW89_FCC][3] = 127,
+ [0][1][2][1][RTW89_ETSI][3] = 127,
+ [0][1][2][1][RTW89_MKK][3] = 127,
+ [0][1][2][1][RTW89_IC][3] = 127,
+ [0][1][2][1][RTW89_KCC][3] = 127,
+ [0][1][2][1][RTW89_ACMA][3] = 127,
+ [0][1][2][1][RTW89_CN][3] = 127,
+ [0][1][2][1][RTW89_UK][3] = 127,
+ [0][1][2][1][RTW89_FCC][4] = 127,
+ [0][1][2][1][RTW89_ETSI][4] = 127,
+ [0][1][2][1][RTW89_MKK][4] = 127,
+ [0][1][2][1][RTW89_IC][4] = 127,
+ [0][1][2][1][RTW89_KCC][4] = 127,
+ [0][1][2][1][RTW89_ACMA][4] = 127,
+ [0][1][2][1][RTW89_CN][4] = 127,
+ [0][1][2][1][RTW89_UK][4] = 127,
+ [0][1][2][1][RTW89_FCC][5] = 127,
+ [0][1][2][1][RTW89_ETSI][5] = 127,
+ [0][1][2][1][RTW89_MKK][5] = 127,
+ [0][1][2][1][RTW89_IC][5] = 127,
+ [0][1][2][1][RTW89_KCC][5] = 127,
+ [0][1][2][1][RTW89_ACMA][5] = 127,
+ [0][1][2][1][RTW89_CN][5] = 127,
+ [0][1][2][1][RTW89_UK][5] = 127,
+ [0][1][2][1][RTW89_FCC][6] = 127,
+ [0][1][2][1][RTW89_ETSI][6] = 127,
+ [0][1][2][1][RTW89_MKK][6] = 127,
+ [0][1][2][1][RTW89_IC][6] = 127,
+ [0][1][2][1][RTW89_KCC][6] = 127,
+ [0][1][2][1][RTW89_ACMA][6] = 127,
+ [0][1][2][1][RTW89_CN][6] = 127,
+ [0][1][2][1][RTW89_UK][6] = 127,
+ [0][1][2][1][RTW89_FCC][7] = 127,
+ [0][1][2][1][RTW89_ETSI][7] = 127,
+ [0][1][2][1][RTW89_MKK][7] = 127,
+ [0][1][2][1][RTW89_IC][7] = 127,
+ [0][1][2][1][RTW89_KCC][7] = 127,
+ [0][1][2][1][RTW89_ACMA][7] = 127,
+ [0][1][2][1][RTW89_CN][7] = 127,
+ [0][1][2][1][RTW89_UK][7] = 127,
+ [0][1][2][1][RTW89_FCC][8] = 127,
+ [0][1][2][1][RTW89_ETSI][8] = 127,
+ [0][1][2][1][RTW89_MKK][8] = 127,
+ [0][1][2][1][RTW89_IC][8] = 127,
+ [0][1][2][1][RTW89_KCC][8] = 127,
+ [0][1][2][1][RTW89_ACMA][8] = 127,
+ [0][1][2][1][RTW89_CN][8] = 127,
+ [0][1][2][1][RTW89_UK][8] = 127,
+ [0][1][2][1][RTW89_FCC][9] = 127,
+ [0][1][2][1][RTW89_ETSI][9] = 127,
+ [0][1][2][1][RTW89_MKK][9] = 127,
+ [0][1][2][1][RTW89_IC][9] = 127,
+ [0][1][2][1][RTW89_KCC][9] = 127,
+ [0][1][2][1][RTW89_ACMA][9] = 127,
+ [0][1][2][1][RTW89_CN][9] = 127,
+ [0][1][2][1][RTW89_UK][9] = 127,
+ [0][1][2][1][RTW89_FCC][10] = 127,
+ [0][1][2][1][RTW89_ETSI][10] = 127,
+ [0][1][2][1][RTW89_MKK][10] = 127,
+ [0][1][2][1][RTW89_IC][10] = 127,
+ [0][1][2][1][RTW89_KCC][10] = 127,
+ [0][1][2][1][RTW89_ACMA][10] = 127,
+ [0][1][2][1][RTW89_CN][10] = 127,
+ [0][1][2][1][RTW89_UK][10] = 127,
+ [0][1][2][1][RTW89_FCC][11] = 127,
+ [0][1][2][1][RTW89_ETSI][11] = 127,
+ [0][1][2][1][RTW89_MKK][11] = 127,
+ [0][1][2][1][RTW89_IC][11] = 127,
+ [0][1][2][1][RTW89_KCC][11] = 127,
+ [0][1][2][1][RTW89_ACMA][11] = 127,
+ [0][1][2][1][RTW89_CN][11] = 127,
+ [0][1][2][1][RTW89_UK][11] = 127,
+ [0][1][2][1][RTW89_FCC][12] = 127,
+ [0][1][2][1][RTW89_ETSI][12] = 127,
+ [0][1][2][1][RTW89_MKK][12] = 127,
+ [0][1][2][1][RTW89_IC][12] = 127,
+ [0][1][2][1][RTW89_KCC][12] = 127,
+ [0][1][2][1][RTW89_ACMA][12] = 127,
+ [0][1][2][1][RTW89_CN][12] = 127,
+ [0][1][2][1][RTW89_UK][12] = 127,
+ [0][1][2][1][RTW89_FCC][13] = 127,
+ [0][1][2][1][RTW89_ETSI][13] = 127,
+ [0][1][2][1][RTW89_MKK][13] = 127,
+ [0][1][2][1][RTW89_IC][13] = 127,
+ [0][1][2][1][RTW89_KCC][13] = 127,
+ [0][1][2][1][RTW89_ACMA][13] = 127,
+ [0][1][2][1][RTW89_CN][13] = 127,
+ [0][1][2][1][RTW89_UK][13] = 127,
+ [1][0][2][0][RTW89_FCC][0] = 127,
+ [1][0][2][0][RTW89_ETSI][0] = 127,
+ [1][0][2][0][RTW89_MKK][0] = 127,
+ [1][0][2][0][RTW89_IC][0] = 127,
+ [1][0][2][0][RTW89_KCC][0] = 127,
+ [1][0][2][0][RTW89_ACMA][0] = 127,
+ [1][0][2][0][RTW89_CN][0] = 127,
+ [1][0][2][0][RTW89_UK][0] = 127,
+ [1][0][2][0][RTW89_FCC][1] = 127,
+ [1][0][2][0][RTW89_ETSI][1] = 127,
+ [1][0][2][0][RTW89_MKK][1] = 127,
+ [1][0][2][0][RTW89_IC][1] = 127,
+ [1][0][2][0][RTW89_KCC][1] = 127,
+ [1][0][2][0][RTW89_ACMA][1] = 127,
+ [1][0][2][0][RTW89_CN][1] = 127,
+ [1][0][2][0][RTW89_UK][1] = 127,
+ [1][0][2][0][RTW89_FCC][2] = 70,
+ [1][0][2][0][RTW89_ETSI][2] = 58,
+ [1][0][2][0][RTW89_MKK][2] = 76,
+ [1][0][2][0][RTW89_IC][2] = 70,
+ [1][0][2][0][RTW89_KCC][2] = 76,
+ [1][0][2][0][RTW89_ACMA][2] = 58,
+ [1][0][2][0][RTW89_CN][2] = 60,
+ [1][0][2][0][RTW89_UK][2] = 58,
+ [1][0][2][0][RTW89_FCC][3] = 70,
+ [1][0][2][0][RTW89_ETSI][3] = 58,
+ [1][0][2][0][RTW89_MKK][3] = 76,
+ [1][0][2][0][RTW89_IC][3] = 70,
+ [1][0][2][0][RTW89_KCC][3] = 76,
+ [1][0][2][0][RTW89_ACMA][3] = 58,
+ [1][0][2][0][RTW89_CN][3] = 60,
+ [1][0][2][0][RTW89_UK][3] = 58,
+ [1][0][2][0][RTW89_FCC][4] = 74,
+ [1][0][2][0][RTW89_ETSI][4] = 58,
+ [1][0][2][0][RTW89_MKK][4] = 76,
+ [1][0][2][0][RTW89_IC][4] = 74,
+ [1][0][2][0][RTW89_KCC][4] = 76,
+ [1][0][2][0][RTW89_ACMA][4] = 58,
+ [1][0][2][0][RTW89_CN][4] = 60,
+ [1][0][2][0][RTW89_UK][4] = 58,
+ [1][0][2][0][RTW89_FCC][5] = 76,
+ [1][0][2][0][RTW89_ETSI][5] = 58,
+ [1][0][2][0][RTW89_MKK][5] = 76,
+ [1][0][2][0][RTW89_IC][5] = 76,
+ [1][0][2][0][RTW89_KCC][5] = 76,
+ [1][0][2][0][RTW89_ACMA][5] = 58,
+ [1][0][2][0][RTW89_CN][5] = 60,
+ [1][0][2][0][RTW89_UK][5] = 58,
+ [1][0][2][0][RTW89_FCC][6] = 76,
+ [1][0][2][0][RTW89_ETSI][6] = 58,
+ [1][0][2][0][RTW89_MKK][6] = 76,
+ [1][0][2][0][RTW89_IC][6] = 76,
+ [1][0][2][0][RTW89_KCC][6] = 76,
+ [1][0][2][0][RTW89_ACMA][6] = 58,
+ [1][0][2][0][RTW89_CN][6] = 60,
+ [1][0][2][0][RTW89_UK][6] = 58,
+ [1][0][2][0][RTW89_FCC][7] = 76,
+ [1][0][2][0][RTW89_ETSI][7] = 58,
+ [1][0][2][0][RTW89_MKK][7] = 76,
+ [1][0][2][0][RTW89_IC][7] = 76,
+ [1][0][2][0][RTW89_KCC][7] = 76,
+ [1][0][2][0][RTW89_ACMA][7] = 58,
+ [1][0][2][0][RTW89_CN][7] = 60,
+ [1][0][2][0][RTW89_UK][7] = 58,
+ [1][0][2][0][RTW89_FCC][8] = 78,
+ [1][0][2][0][RTW89_ETSI][8] = 58,
+ [1][0][2][0][RTW89_MKK][8] = 76,
+ [1][0][2][0][RTW89_IC][8] = 78,
+ [1][0][2][0][RTW89_KCC][8] = 76,
+ [1][0][2][0][RTW89_ACMA][8] = 58,
+ [1][0][2][0][RTW89_CN][8] = 60,
+ [1][0][2][0][RTW89_UK][8] = 58,
+ [1][0][2][0][RTW89_FCC][9] = 74,
+ [1][0][2][0][RTW89_ETSI][9] = 58,
+ [1][0][2][0][RTW89_MKK][9] = 76,
+ [1][0][2][0][RTW89_IC][9] = 74,
+ [1][0][2][0][RTW89_KCC][9] = 76,
+ [1][0][2][0][RTW89_ACMA][9] = 58,
+ [1][0][2][0][RTW89_CN][9] = 60,
+ [1][0][2][0][RTW89_UK][9] = 58,
+ [1][0][2][0][RTW89_FCC][10] = 68,
+ [1][0][2][0][RTW89_ETSI][10] = 58,
+ [1][0][2][0][RTW89_MKK][10] = 76,
+ [1][0][2][0][RTW89_IC][10] = 68,
+ [1][0][2][0][RTW89_KCC][10] = 76,
+ [1][0][2][0][RTW89_ACMA][10] = 58,
+ [1][0][2][0][RTW89_CN][10] = 60,
+ [1][0][2][0][RTW89_UK][10] = 58,
+ [1][0][2][0][RTW89_FCC][11] = 127,
+ [1][0][2][0][RTW89_ETSI][11] = 127,
+ [1][0][2][0][RTW89_MKK][11] = 127,
+ [1][0][2][0][RTW89_IC][11] = 127,
+ [1][0][2][0][RTW89_KCC][11] = 127,
+ [1][0][2][0][RTW89_ACMA][11] = 127,
+ [1][0][2][0][RTW89_CN][11] = 127,
+ [1][0][2][0][RTW89_UK][11] = 127,
+ [1][0][2][0][RTW89_FCC][12] = 127,
+ [1][0][2][0][RTW89_ETSI][12] = 127,
+ [1][0][2][0][RTW89_MKK][12] = 127,
+ [1][0][2][0][RTW89_IC][12] = 127,
+ [1][0][2][0][RTW89_KCC][12] = 127,
+ [1][0][2][0][RTW89_ACMA][12] = 127,
+ [1][0][2][0][RTW89_CN][12] = 127,
+ [1][0][2][0][RTW89_UK][12] = 127,
+ [1][0][2][0][RTW89_FCC][13] = 127,
+ [1][0][2][0][RTW89_ETSI][13] = 127,
+ [1][0][2][0][RTW89_MKK][13] = 127,
+ [1][0][2][0][RTW89_IC][13] = 127,
+ [1][0][2][0][RTW89_KCC][13] = 127,
+ [1][0][2][0][RTW89_ACMA][13] = 127,
+ [1][0][2][0][RTW89_CN][13] = 127,
+ [1][0][2][0][RTW89_UK][13] = 127,
+ [1][1][2][0][RTW89_FCC][0] = 127,
+ [1][1][2][0][RTW89_ETSI][0] = 127,
+ [1][1][2][0][RTW89_MKK][0] = 127,
+ [1][1][2][0][RTW89_IC][0] = 127,
+ [1][1][2][0][RTW89_KCC][0] = 127,
+ [1][1][2][0][RTW89_ACMA][0] = 127,
+ [1][1][2][0][RTW89_CN][0] = 127,
+ [1][1][2][0][RTW89_UK][0] = 127,
+ [1][1][2][0][RTW89_FCC][1] = 127,
+ [1][1][2][0][RTW89_ETSI][1] = 127,
+ [1][1][2][0][RTW89_MKK][1] = 127,
+ [1][1][2][0][RTW89_IC][1] = 127,
+ [1][1][2][0][RTW89_KCC][1] = 127,
+ [1][1][2][0][RTW89_ACMA][1] = 127,
+ [1][1][2][0][RTW89_CN][1] = 127,
+ [1][1][2][0][RTW89_UK][1] = 127,
+ [1][1][2][0][RTW89_FCC][2] = 127,
+ [1][1][2][0][RTW89_ETSI][2] = 127,
+ [1][1][2][0][RTW89_MKK][2] = 127,
+ [1][1][2][0][RTW89_IC][2] = 127,
+ [1][1][2][0][RTW89_KCC][2] = 127,
+ [1][1][2][0][RTW89_ACMA][2] = 127,
+ [1][1][2][0][RTW89_CN][2] = 127,
+ [1][1][2][0][RTW89_UK][2] = 127,
+ [1][1][2][0][RTW89_FCC][3] = 127,
+ [1][1][2][0][RTW89_ETSI][3] = 127,
+ [1][1][2][0][RTW89_MKK][3] = 127,
+ [1][1][2][0][RTW89_IC][3] = 127,
+ [1][1][2][0][RTW89_KCC][3] = 127,
+ [1][1][2][0][RTW89_ACMA][3] = 127,
+ [1][1][2][0][RTW89_CN][3] = 127,
+ [1][1][2][0][RTW89_UK][3] = 127,
+ [1][1][2][0][RTW89_FCC][4] = 127,
+ [1][1][2][0][RTW89_ETSI][4] = 127,
+ [1][1][2][0][RTW89_MKK][4] = 127,
+ [1][1][2][0][RTW89_IC][4] = 127,
+ [1][1][2][0][RTW89_KCC][4] = 127,
+ [1][1][2][0][RTW89_ACMA][4] = 127,
+ [1][1][2][0][RTW89_CN][4] = 127,
+ [1][1][2][0][RTW89_UK][4] = 127,
+ [1][1][2][0][RTW89_FCC][5] = 127,
+ [1][1][2][0][RTW89_ETSI][5] = 127,
+ [1][1][2][0][RTW89_MKK][5] = 127,
+ [1][1][2][0][RTW89_IC][5] = 127,
+ [1][1][2][0][RTW89_KCC][5] = 127,
+ [1][1][2][0][RTW89_ACMA][5] = 127,
+ [1][1][2][0][RTW89_CN][5] = 127,
+ [1][1][2][0][RTW89_UK][5] = 127,
+ [1][1][2][0][RTW89_FCC][6] = 127,
+ [1][1][2][0][RTW89_ETSI][6] = 127,
+ [1][1][2][0][RTW89_MKK][6] = 127,
+ [1][1][2][0][RTW89_IC][6] = 127,
+ [1][1][2][0][RTW89_KCC][6] = 127,
+ [1][1][2][0][RTW89_ACMA][6] = 127,
+ [1][1][2][0][RTW89_CN][6] = 127,
+ [1][1][2][0][RTW89_UK][6] = 127,
+ [1][1][2][0][RTW89_FCC][7] = 127,
+ [1][1][2][0][RTW89_ETSI][7] = 127,
+ [1][1][2][0][RTW89_MKK][7] = 127,
+ [1][1][2][0][RTW89_IC][7] = 127,
+ [1][1][2][0][RTW89_KCC][7] = 127,
+ [1][1][2][0][RTW89_ACMA][7] = 127,
+ [1][1][2][0][RTW89_CN][7] = 127,
+ [1][1][2][0][RTW89_UK][7] = 127,
+ [1][1][2][0][RTW89_FCC][8] = 127,
+ [1][1][2][0][RTW89_ETSI][8] = 127,
+ [1][1][2][0][RTW89_MKK][8] = 127,
+ [1][1][2][0][RTW89_IC][8] = 127,
+ [1][1][2][0][RTW89_KCC][8] = 127,
+ [1][1][2][0][RTW89_ACMA][8] = 127,
+ [1][1][2][0][RTW89_CN][8] = 127,
+ [1][1][2][0][RTW89_UK][8] = 127,
+ [1][1][2][0][RTW89_FCC][9] = 127,
+ [1][1][2][0][RTW89_ETSI][9] = 127,
+ [1][1][2][0][RTW89_MKK][9] = 127,
+ [1][1][2][0][RTW89_IC][9] = 127,
+ [1][1][2][0][RTW89_KCC][9] = 127,
+ [1][1][2][0][RTW89_ACMA][9] = 127,
+ [1][1][2][0][RTW89_CN][9] = 127,
+ [1][1][2][0][RTW89_UK][9] = 127,
+ [1][1][2][0][RTW89_FCC][10] = 127,
+ [1][1][2][0][RTW89_ETSI][10] = 127,
+ [1][1][2][0][RTW89_MKK][10] = 127,
+ [1][1][2][0][RTW89_IC][10] = 127,
+ [1][1][2][0][RTW89_KCC][10] = 127,
+ [1][1][2][0][RTW89_ACMA][10] = 127,
+ [1][1][2][0][RTW89_CN][10] = 127,
+ [1][1][2][0][RTW89_UK][10] = 127,
+ [1][1][2][0][RTW89_FCC][11] = 127,
+ [1][1][2][0][RTW89_ETSI][11] = 127,
+ [1][1][2][0][RTW89_MKK][11] = 127,
+ [1][1][2][0][RTW89_IC][11] = 127,
+ [1][1][2][0][RTW89_KCC][11] = 127,
+ [1][1][2][0][RTW89_ACMA][11] = 127,
+ [1][1][2][0][RTW89_CN][11] = 127,
+ [1][1][2][0][RTW89_UK][11] = 127,
+ [1][1][2][0][RTW89_FCC][12] = 127,
+ [1][1][2][0][RTW89_ETSI][12] = 127,
+ [1][1][2][0][RTW89_MKK][12] = 127,
+ [1][1][2][0][RTW89_IC][12] = 127,
+ [1][1][2][0][RTW89_KCC][12] = 127,
+ [1][1][2][0][RTW89_ACMA][12] = 127,
+ [1][1][2][0][RTW89_CN][12] = 127,
+ [1][1][2][0][RTW89_UK][12] = 127,
+ [1][1][2][0][RTW89_FCC][13] = 127,
+ [1][1][2][0][RTW89_ETSI][13] = 127,
+ [1][1][2][0][RTW89_MKK][13] = 127,
+ [1][1][2][0][RTW89_IC][13] = 127,
+ [1][1][2][0][RTW89_KCC][13] = 127,
+ [1][1][2][0][RTW89_ACMA][13] = 127,
+ [1][1][2][0][RTW89_CN][13] = 127,
+ [1][1][2][0][RTW89_UK][13] = 127,
+ [1][1][2][1][RTW89_FCC][0] = 127,
+ [1][1][2][1][RTW89_ETSI][0] = 127,
+ [1][1][2][1][RTW89_MKK][0] = 127,
+ [1][1][2][1][RTW89_IC][0] = 127,
+ [1][1][2][1][RTW89_KCC][0] = 127,
+ [1][1][2][1][RTW89_ACMA][0] = 127,
+ [1][1][2][1][RTW89_CN][0] = 127,
+ [1][1][2][1][RTW89_UK][0] = 127,
+ [1][1][2][1][RTW89_FCC][1] = 127,
+ [1][1][2][1][RTW89_ETSI][1] = 127,
+ [1][1][2][1][RTW89_MKK][1] = 127,
+ [1][1][2][1][RTW89_IC][1] = 127,
+ [1][1][2][1][RTW89_KCC][1] = 127,
+ [1][1][2][1][RTW89_ACMA][1] = 127,
+ [1][1][2][1][RTW89_CN][1] = 127,
+ [1][1][2][1][RTW89_UK][1] = 127,
+ [1][1][2][1][RTW89_FCC][2] = 127,
+ [1][1][2][1][RTW89_ETSI][2] = 127,
+ [1][1][2][1][RTW89_MKK][2] = 127,
+ [1][1][2][1][RTW89_IC][2] = 127,
+ [1][1][2][1][RTW89_KCC][2] = 127,
+ [1][1][2][1][RTW89_ACMA][2] = 127,
+ [1][1][2][1][RTW89_CN][2] = 127,
+ [1][1][2][1][RTW89_UK][2] = 127,
+ [1][1][2][1][RTW89_FCC][3] = 127,
+ [1][1][2][1][RTW89_ETSI][3] = 127,
+ [1][1][2][1][RTW89_MKK][3] = 127,
+ [1][1][2][1][RTW89_IC][3] = 127,
+ [1][1][2][1][RTW89_KCC][3] = 127,
+ [1][1][2][1][RTW89_ACMA][3] = 127,
+ [1][1][2][1][RTW89_CN][3] = 127,
+ [1][1][2][1][RTW89_UK][3] = 127,
+ [1][1][2][1][RTW89_FCC][4] = 127,
+ [1][1][2][1][RTW89_ETSI][4] = 127,
+ [1][1][2][1][RTW89_MKK][4] = 127,
+ [1][1][2][1][RTW89_IC][4] = 127,
+ [1][1][2][1][RTW89_KCC][4] = 127,
+ [1][1][2][1][RTW89_ACMA][4] = 127,
+ [1][1][2][1][RTW89_CN][4] = 127,
+ [1][1][2][1][RTW89_UK][4] = 127,
+ [1][1][2][1][RTW89_FCC][5] = 127,
+ [1][1][2][1][RTW89_ETSI][5] = 127,
+ [1][1][2][1][RTW89_MKK][5] = 127,
+ [1][1][2][1][RTW89_IC][5] = 127,
+ [1][1][2][1][RTW89_KCC][5] = 127,
+ [1][1][2][1][RTW89_ACMA][5] = 127,
+ [1][1][2][1][RTW89_CN][5] = 127,
+ [1][1][2][1][RTW89_UK][5] = 127,
+ [1][1][2][1][RTW89_FCC][6] = 127,
+ [1][1][2][1][RTW89_ETSI][6] = 127,
+ [1][1][2][1][RTW89_MKK][6] = 127,
+ [1][1][2][1][RTW89_IC][6] = 127,
+ [1][1][2][1][RTW89_KCC][6] = 127,
+ [1][1][2][1][RTW89_ACMA][6] = 127,
+ [1][1][2][1][RTW89_CN][6] = 127,
+ [1][1][2][1][RTW89_UK][6] = 127,
+ [1][1][2][1][RTW89_FCC][7] = 127,
+ [1][1][2][1][RTW89_ETSI][7] = 127,
+ [1][1][2][1][RTW89_MKK][7] = 127,
+ [1][1][2][1][RTW89_IC][7] = 127,
+ [1][1][2][1][RTW89_KCC][7] = 127,
+ [1][1][2][1][RTW89_ACMA][7] = 127,
+ [1][1][2][1][RTW89_CN][7] = 127,
+ [1][1][2][1][RTW89_UK][7] = 127,
+ [1][1][2][1][RTW89_FCC][8] = 127,
+ [1][1][2][1][RTW89_ETSI][8] = 127,
+ [1][1][2][1][RTW89_MKK][8] = 127,
+ [1][1][2][1][RTW89_IC][8] = 127,
+ [1][1][2][1][RTW89_KCC][8] = 127,
+ [1][1][2][1][RTW89_ACMA][8] = 127,
+ [1][1][2][1][RTW89_CN][8] = 127,
+ [1][1][2][1][RTW89_UK][8] = 127,
+ [1][1][2][1][RTW89_FCC][9] = 127,
+ [1][1][2][1][RTW89_ETSI][9] = 127,
+ [1][1][2][1][RTW89_MKK][9] = 127,
+ [1][1][2][1][RTW89_IC][9] = 127,
+ [1][1][2][1][RTW89_KCC][9] = 127,
+ [1][1][2][1][RTW89_ACMA][9] = 127,
+ [1][1][2][1][RTW89_CN][9] = 127,
+ [1][1][2][1][RTW89_UK][9] = 127,
+ [1][1][2][1][RTW89_FCC][10] = 127,
+ [1][1][2][1][RTW89_ETSI][10] = 127,
+ [1][1][2][1][RTW89_MKK][10] = 127,
+ [1][1][2][1][RTW89_IC][10] = 127,
+ [1][1][2][1][RTW89_KCC][10] = 127,
+ [1][1][2][1][RTW89_ACMA][10] = 127,
+ [1][1][2][1][RTW89_CN][10] = 127,
+ [1][1][2][1][RTW89_UK][10] = 127,
+ [1][1][2][1][RTW89_FCC][11] = 127,
+ [1][1][2][1][RTW89_ETSI][11] = 127,
+ [1][1][2][1][RTW89_MKK][11] = 127,
+ [1][1][2][1][RTW89_IC][11] = 127,
+ [1][1][2][1][RTW89_KCC][11] = 127,
+ [1][1][2][1][RTW89_ACMA][11] = 127,
+ [1][1][2][1][RTW89_CN][11] = 127,
+ [1][1][2][1][RTW89_UK][11] = 127,
+ [1][1][2][1][RTW89_FCC][12] = 127,
+ [1][1][2][1][RTW89_ETSI][12] = 127,
+ [1][1][2][1][RTW89_MKK][12] = 127,
+ [1][1][2][1][RTW89_IC][12] = 127,
+ [1][1][2][1][RTW89_KCC][12] = 127,
+ [1][1][2][1][RTW89_ACMA][12] = 127,
+ [1][1][2][1][RTW89_CN][12] = 127,
+ [1][1][2][1][RTW89_UK][12] = 127,
+ [1][1][2][1][RTW89_FCC][13] = 127,
+ [1][1][2][1][RTW89_ETSI][13] = 127,
+ [1][1][2][1][RTW89_MKK][13] = 127,
+ [1][1][2][1][RTW89_IC][13] = 127,
+ [1][1][2][1][RTW89_KCC][13] = 127,
+ [1][1][2][1][RTW89_ACMA][13] = 127,
+ [1][1][2][1][RTW89_CN][13] = 127,
+ [1][1][2][1][RTW89_UK][13] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_5g_type2[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
+ [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
+ [0][0][1][0][RTW89_WW][0] = 58,
+ [0][0][1][0][RTW89_WW][2] = 58,
+ [0][0][1][0][RTW89_WW][4] = 58,
+ [0][0][1][0][RTW89_WW][6] = 50,
+ [0][0][1][0][RTW89_WW][8] = 58,
+ [0][0][1][0][RTW89_WW][10] = 58,
+ [0][0][1][0][RTW89_WW][12] = 58,
+ [0][0][1][0][RTW89_WW][14] = 58,
+ [0][0][1][0][RTW89_WW][15] = 58,
+ [0][0][1][0][RTW89_WW][17] = 60,
+ [0][0][1][0][RTW89_WW][19] = 60,
+ [0][0][1][0][RTW89_WW][21] = 60,
+ [0][0][1][0][RTW89_WW][23] = 60,
+ [0][0][1][0][RTW89_WW][25] = 60,
+ [0][0][1][0][RTW89_WW][27] = 60,
+ [0][0][1][0][RTW89_WW][29] = 60,
+ [0][0][1][0][RTW89_WW][31] = 60,
+ [0][0][1][0][RTW89_WW][33] = 60,
+ [0][0][1][0][RTW89_WW][35] = 60,
+ [0][0][1][0][RTW89_WW][37] = 74,
+ [0][0][1][0][RTW89_WW][38] = 30,
+ [0][0][1][0][RTW89_WW][40] = 30,
+ [0][0][1][0][RTW89_WW][42] = 30,
+ [0][0][1][0][RTW89_WW][44] = 30,
+ [0][0][1][0][RTW89_WW][46] = 30,
+ [0][0][1][0][RTW89_WW][48] = 72,
+ [0][0][1][0][RTW89_WW][50] = 72,
+ [0][0][1][0][RTW89_WW][52] = 72,
+ [0][1][1][0][RTW89_WW][0] = 0,
+ [0][1][1][0][RTW89_WW][2] = 0,
+ [0][1][1][0][RTW89_WW][4] = 0,
+ [0][1][1][0][RTW89_WW][6] = 0,
+ [0][1][1][0][RTW89_WW][8] = 0,
+ [0][1][1][0][RTW89_WW][10] = 0,
+ [0][1][1][0][RTW89_WW][12] = 0,
+ [0][1][1][0][RTW89_WW][14] = 0,
+ [0][1][1][0][RTW89_WW][15] = 0,
+ [0][1][1][0][RTW89_WW][17] = 0,
+ [0][1][1][0][RTW89_WW][19] = 0,
+ [0][1][1][0][RTW89_WW][21] = 0,
+ [0][1][1][0][RTW89_WW][23] = 0,
+ [0][1][1][0][RTW89_WW][25] = 0,
+ [0][1][1][0][RTW89_WW][27] = 0,
+ [0][1][1][0][RTW89_WW][29] = 0,
+ [0][1][1][0][RTW89_WW][31] = 0,
+ [0][1][1][0][RTW89_WW][33] = 0,
+ [0][1][1][0][RTW89_WW][35] = 0,
+ [0][1][1][0][RTW89_WW][37] = 0,
+ [0][1][1][0][RTW89_WW][38] = 0,
+ [0][1][1][0][RTW89_WW][40] = 0,
+ [0][1][1][0][RTW89_WW][42] = 0,
+ [0][1][1][0][RTW89_WW][44] = 0,
+ [0][1][1][0][RTW89_WW][46] = 0,
+ [0][1][1][0][RTW89_WW][48] = 0,
+ [0][1][1][0][RTW89_WW][50] = 0,
+ [0][1][1][0][RTW89_WW][52] = 0,
+ [0][0][2][0][RTW89_WW][0] = 62,
+ [0][0][2][0][RTW89_WW][2] = 62,
+ [0][0][2][0][RTW89_WW][4] = 62,
+ [0][0][2][0][RTW89_WW][6] = 54,
+ [0][0][2][0][RTW89_WW][8] = 62,
+ [0][0][2][0][RTW89_WW][10] = 62,
+ [0][0][2][0][RTW89_WW][12] = 62,
+ [0][0][2][0][RTW89_WW][14] = 62,
+ [0][0][2][0][RTW89_WW][15] = 60,
+ [0][0][2][0][RTW89_WW][17] = 62,
+ [0][0][2][0][RTW89_WW][19] = 62,
+ [0][0][2][0][RTW89_WW][21] = 62,
+ [0][0][2][0][RTW89_WW][23] = 62,
+ [0][0][2][0][RTW89_WW][25] = 62,
+ [0][0][2][0][RTW89_WW][27] = 62,
+ [0][0][2][0][RTW89_WW][29] = 62,
+ [0][0][2][0][RTW89_WW][31] = 62,
+ [0][0][2][0][RTW89_WW][33] = 62,
+ [0][0][2][0][RTW89_WW][35] = 62,
+ [0][0][2][0][RTW89_WW][37] = 74,
+ [0][0][2][0][RTW89_WW][38] = 30,
+ [0][0][2][0][RTW89_WW][40] = 30,
+ [0][0][2][0][RTW89_WW][42] = 30,
+ [0][0][2][0][RTW89_WW][44] = 30,
+ [0][0][2][0][RTW89_WW][46] = 30,
+ [0][0][2][0][RTW89_WW][48] = 74,
+ [0][0][2][0][RTW89_WW][50] = 74,
+ [0][0][2][0][RTW89_WW][52] = 74,
+ [0][1][2][0][RTW89_WW][0] = 0,
+ [0][1][2][0][RTW89_WW][2] = 0,
+ [0][1][2][0][RTW89_WW][4] = 0,
+ [0][1][2][0][RTW89_WW][6] = 0,
+ [0][1][2][0][RTW89_WW][8] = 0,
+ [0][1][2][0][RTW89_WW][10] = 0,
+ [0][1][2][0][RTW89_WW][12] = 0,
+ [0][1][2][0][RTW89_WW][14] = 0,
+ [0][1][2][0][RTW89_WW][15] = 0,
+ [0][1][2][0][RTW89_WW][17] = 0,
+ [0][1][2][0][RTW89_WW][19] = 0,
+ [0][1][2][0][RTW89_WW][21] = 0,
+ [0][1][2][0][RTW89_WW][23] = 0,
+ [0][1][2][0][RTW89_WW][25] = 0,
+ [0][1][2][0][RTW89_WW][27] = 0,
+ [0][1][2][0][RTW89_WW][29] = 0,
+ [0][1][2][0][RTW89_WW][31] = 0,
+ [0][1][2][0][RTW89_WW][33] = 0,
+ [0][1][2][0][RTW89_WW][35] = 0,
+ [0][1][2][0][RTW89_WW][37] = 0,
+ [0][1][2][0][RTW89_WW][38] = 0,
+ [0][1][2][0][RTW89_WW][40] = 0,
+ [0][1][2][0][RTW89_WW][42] = 0,
+ [0][1][2][0][RTW89_WW][44] = 0,
+ [0][1][2][0][RTW89_WW][46] = 0,
+ [0][1][2][0][RTW89_WW][48] = 0,
+ [0][1][2][0][RTW89_WW][50] = 0,
+ [0][1][2][0][RTW89_WW][52] = 0,
+ [0][1][2][1][RTW89_WW][0] = 0,
+ [0][1][2][1][RTW89_WW][2] = 0,
+ [0][1][2][1][RTW89_WW][4] = 0,
+ [0][1][2][1][RTW89_WW][6] = 0,
+ [0][1][2][1][RTW89_WW][8] = 0,
+ [0][1][2][1][RTW89_WW][10] = 0,
+ [0][1][2][1][RTW89_WW][12] = 0,
+ [0][1][2][1][RTW89_WW][14] = 0,
+ [0][1][2][1][RTW89_WW][15] = 0,
+ [0][1][2][1][RTW89_WW][17] = 0,
+ [0][1][2][1][RTW89_WW][19] = 0,
+ [0][1][2][1][RTW89_WW][21] = 0,
+ [0][1][2][1][RTW89_WW][23] = 0,
+ [0][1][2][1][RTW89_WW][25] = 0,
+ [0][1][2][1][RTW89_WW][27] = 0,
+ [0][1][2][1][RTW89_WW][29] = 0,
+ [0][1][2][1][RTW89_WW][31] = 0,
+ [0][1][2][1][RTW89_WW][33] = 0,
+ [0][1][2][1][RTW89_WW][35] = 0,
+ [0][1][2][1][RTW89_WW][37] = 0,
+ [0][1][2][1][RTW89_WW][38] = 0,
+ [0][1][2][1][RTW89_WW][40] = 0,
+ [0][1][2][1][RTW89_WW][42] = 0,
+ [0][1][2][1][RTW89_WW][44] = 0,
+ [0][1][2][1][RTW89_WW][46] = 0,
+ [0][1][2][1][RTW89_WW][48] = 0,
+ [0][1][2][1][RTW89_WW][50] = 0,
+ [0][1][2][1][RTW89_WW][52] = 0,
+ [1][0][2][0][RTW89_WW][1] = 64,
+ [1][0][2][0][RTW89_WW][5] = 62,
+ [1][0][2][0][RTW89_WW][9] = 64,
+ [1][0][2][0][RTW89_WW][13] = 64,
+ [1][0][2][0][RTW89_WW][16] = 66,
+ [1][0][2][0][RTW89_WW][20] = 66,
+ [1][0][2][0][RTW89_WW][24] = 66,
+ [1][0][2][0][RTW89_WW][28] = 66,
+ [1][0][2][0][RTW89_WW][32] = 66,
+ [1][0][2][0][RTW89_WW][36] = 76,
+ [1][0][2][0][RTW89_WW][39] = 30,
+ [1][0][2][0][RTW89_WW][43] = 30,
+ [1][0][2][0][RTW89_WW][47] = 80,
+ [1][0][2][0][RTW89_WW][51] = 80,
+ [1][1][2][0][RTW89_WW][1] = 0,
+ [1][1][2][0][RTW89_WW][5] = 0,
+ [1][1][2][0][RTW89_WW][9] = 0,
+ [1][1][2][0][RTW89_WW][13] = 0,
+ [1][1][2][0][RTW89_WW][16] = 0,
+ [1][1][2][0][RTW89_WW][20] = 0,
+ [1][1][2][0][RTW89_WW][24] = 0,
+ [1][1][2][0][RTW89_WW][28] = 0,
+ [1][1][2][0][RTW89_WW][32] = 0,
+ [1][1][2][0][RTW89_WW][36] = 0,
+ [1][1][2][0][RTW89_WW][39] = 0,
+ [1][1][2][0][RTW89_WW][43] = 0,
+ [1][1][2][0][RTW89_WW][47] = 0,
+ [1][1][2][0][RTW89_WW][51] = 0,
+ [1][1][2][1][RTW89_WW][1] = 0,
+ [1][1][2][1][RTW89_WW][5] = 0,
+ [1][1][2][1][RTW89_WW][9] = 0,
+ [1][1][2][1][RTW89_WW][13] = 0,
+ [1][1][2][1][RTW89_WW][16] = 0,
+ [1][1][2][1][RTW89_WW][20] = 0,
+ [1][1][2][1][RTW89_WW][24] = 0,
+ [1][1][2][1][RTW89_WW][28] = 0,
+ [1][1][2][1][RTW89_WW][32] = 0,
+ [1][1][2][1][RTW89_WW][36] = 0,
+ [1][1][2][1][RTW89_WW][39] = 0,
+ [1][1][2][1][RTW89_WW][43] = 0,
+ [1][1][2][1][RTW89_WW][47] = 0,
+ [1][1][2][1][RTW89_WW][51] = 0,
+ [2][0][2][0][RTW89_WW][3] = 62,
+ [2][0][2][0][RTW89_WW][11] = 62,
+ [2][0][2][0][RTW89_WW][18] = 64,
+ [2][0][2][0][RTW89_WW][26] = 64,
+ [2][0][2][0][RTW89_WW][34] = 68,
+ [2][0][2][0][RTW89_WW][41] = 30,
+ [2][0][2][0][RTW89_WW][49] = 72,
+ [2][1][2][0][RTW89_WW][3] = 0,
+ [2][1][2][0][RTW89_WW][11] = 0,
+ [2][1][2][0][RTW89_WW][18] = 0,
+ [2][1][2][0][RTW89_WW][26] = 0,
+ [2][1][2][0][RTW89_WW][34] = 0,
+ [2][1][2][0][RTW89_WW][41] = 0,
+ [2][1][2][0][RTW89_WW][49] = 0,
+ [2][1][2][1][RTW89_WW][3] = 0,
+ [2][1][2][1][RTW89_WW][11] = 0,
+ [2][1][2][1][RTW89_WW][18] = 0,
+ [2][1][2][1][RTW89_WW][26] = 0,
+ [2][1][2][1][RTW89_WW][34] = 0,
+ [2][1][2][1][RTW89_WW][41] = 0,
+ [2][1][2][1][RTW89_WW][49] = 0,
+ [3][0][2][0][RTW89_WW][7] = 58,
+ [3][0][2][0][RTW89_WW][22] = 58,
+ [3][0][2][0][RTW89_WW][45] = 0,
+ [3][1][2][0][RTW89_WW][7] = 0,
+ [3][1][2][0][RTW89_WW][22] = 0,
+ [3][1][2][0][RTW89_WW][45] = 0,
+ [3][1][2][1][RTW89_WW][7] = 0,
+ [3][1][2][1][RTW89_WW][22] = 0,
+ [3][1][2][1][RTW89_WW][45] = 0,
+ [0][0][1][0][RTW89_FCC][0] = 78,
+ [0][0][1][0][RTW89_ETSI][0] = 58,
+ [0][0][1][0][RTW89_MKK][0] = 60,
+ [0][0][1][0][RTW89_IC][0] = 62,
+ [0][0][1][0][RTW89_KCC][0] = 74,
+ [0][0][1][0][RTW89_ACMA][0] = 58,
+ [0][0][1][0][RTW89_CN][0] = 60,
+ [0][0][1][0][RTW89_UK][0] = 58,
+ [0][0][1][0][RTW89_FCC][2] = 78,
+ [0][0][1][0][RTW89_ETSI][2] = 58,
+ [0][0][1][0][RTW89_MKK][2] = 60,
+ [0][0][1][0][RTW89_IC][2] = 62,
+ [0][0][1][0][RTW89_KCC][2] = 74,
+ [0][0][1][0][RTW89_ACMA][2] = 58,
+ [0][0][1][0][RTW89_CN][2] = 60,
+ [0][0][1][0][RTW89_UK][2] = 58,
+ [0][0][1][0][RTW89_FCC][4] = 78,
+ [0][0][1][0][RTW89_ETSI][4] = 58,
+ [0][0][1][0][RTW89_MKK][4] = 60,
+ [0][0][1][0][RTW89_IC][4] = 62,
+ [0][0][1][0][RTW89_KCC][4] = 74,
+ [0][0][1][0][RTW89_ACMA][4] = 58,
+ [0][0][1][0][RTW89_CN][4] = 60,
+ [0][0][1][0][RTW89_UK][4] = 58,
+ [0][0][1][0][RTW89_FCC][6] = 78,
+ [0][0][1][0][RTW89_ETSI][6] = 58,
+ [0][0][1][0][RTW89_MKK][6] = 60,
+ [0][0][1][0][RTW89_IC][6] = 62,
+ [0][0][1][0][RTW89_KCC][6] = 50,
+ [0][0][1][0][RTW89_ACMA][6] = 58,
+ [0][0][1][0][RTW89_CN][6] = 60,
+ [0][0][1][0][RTW89_UK][6] = 58,
+ [0][0][1][0][RTW89_FCC][8] = 78,
+ [0][0][1][0][RTW89_ETSI][8] = 58,
+ [0][0][1][0][RTW89_MKK][8] = 60,
+ [0][0][1][0][RTW89_IC][8] = 62,
+ [0][0][1][0][RTW89_KCC][8] = 74,
+ [0][0][1][0][RTW89_ACMA][8] = 58,
+ [0][0][1][0][RTW89_CN][8] = 60,
+ [0][0][1][0][RTW89_UK][8] = 58,
+ [0][0][1][0][RTW89_FCC][10] = 78,
+ [0][0][1][0][RTW89_ETSI][10] = 58,
+ [0][0][1][0][RTW89_MKK][10] = 60,
+ [0][0][1][0][RTW89_IC][10] = 64,
+ [0][0][1][0][RTW89_KCC][10] = 74,
+ [0][0][1][0][RTW89_ACMA][10] = 58,
+ [0][0][1][0][RTW89_CN][10] = 60,
+ [0][0][1][0][RTW89_UK][10] = 58,
+ [0][0][1][0][RTW89_FCC][12] = 78,
+ [0][0][1][0][RTW89_ETSI][12] = 58,
+ [0][0][1][0][RTW89_MKK][12] = 60,
+ [0][0][1][0][RTW89_IC][12] = 64,
+ [0][0][1][0][RTW89_KCC][12] = 74,
+ [0][0][1][0][RTW89_ACMA][12] = 58,
+ [0][0][1][0][RTW89_CN][12] = 60,
+ [0][0][1][0][RTW89_UK][12] = 58,
+ [0][0][1][0][RTW89_FCC][14] = 76,
+ [0][0][1][0][RTW89_ETSI][14] = 58,
+ [0][0][1][0][RTW89_MKK][14] = 60,
+ [0][0][1][0][RTW89_IC][14] = 62,
+ [0][0][1][0][RTW89_KCC][14] = 74,
+ [0][0][1][0][RTW89_ACMA][14] = 58,
+ [0][0][1][0][RTW89_CN][14] = 60,
+ [0][0][1][0][RTW89_UK][14] = 58,
+ [0][0][1][0][RTW89_FCC][15] = 76,
+ [0][0][1][0][RTW89_ETSI][15] = 58,
+ [0][0][1][0][RTW89_MKK][15] = 74,
+ [0][0][1][0][RTW89_IC][15] = 76,
+ [0][0][1][0][RTW89_KCC][15] = 74,
+ [0][0][1][0][RTW89_ACMA][15] = 58,
+ [0][0][1][0][RTW89_CN][15] = 127,
+ [0][0][1][0][RTW89_UK][15] = 58,
+ [0][0][1][0][RTW89_FCC][17] = 78,
+ [0][0][1][0][RTW89_ETSI][17] = 60,
+ [0][0][1][0][RTW89_MKK][17] = 74,
+ [0][0][1][0][RTW89_IC][17] = 78,
+ [0][0][1][0][RTW89_KCC][17] = 74,
+ [0][0][1][0][RTW89_ACMA][17] = 60,
+ [0][0][1][0][RTW89_CN][17] = 127,
+ [0][0][1][0][RTW89_UK][17] = 60,
+ [0][0][1][0][RTW89_FCC][19] = 78,
+ [0][0][1][0][RTW89_ETSI][19] = 60,
+ [0][0][1][0][RTW89_MKK][19] = 74,
+ [0][0][1][0][RTW89_IC][19] = 78,
+ [0][0][1][0][RTW89_KCC][19] = 74,
+ [0][0][1][0][RTW89_ACMA][19] = 60,
+ [0][0][1][0][RTW89_CN][19] = 127,
+ [0][0][1][0][RTW89_UK][19] = 60,
+ [0][0][1][0][RTW89_FCC][21] = 78,
+ [0][0][1][0][RTW89_ETSI][21] = 60,
+ [0][0][1][0][RTW89_MKK][21] = 74,
+ [0][0][1][0][RTW89_IC][21] = 78,
+ [0][0][1][0][RTW89_KCC][21] = 74,
+ [0][0][1][0][RTW89_ACMA][21] = 60,
+ [0][0][1][0][RTW89_CN][21] = 127,
+ [0][0][1][0][RTW89_UK][21] = 60,
+ [0][0][1][0][RTW89_FCC][23] = 78,
+ [0][0][1][0][RTW89_ETSI][23] = 60,
+ [0][0][1][0][RTW89_MKK][23] = 74,
+ [0][0][1][0][RTW89_IC][23] = 78,
+ [0][0][1][0][RTW89_KCC][23] = 74,
+ [0][0][1][0][RTW89_ACMA][23] = 60,
+ [0][0][1][0][RTW89_CN][23] = 127,
+ [0][0][1][0][RTW89_UK][23] = 60,
+ [0][0][1][0][RTW89_FCC][25] = 78,
+ [0][0][1][0][RTW89_ETSI][25] = 60,
+ [0][0][1][0][RTW89_MKK][25] = 74,
+ [0][0][1][0][RTW89_IC][25] = 127,
+ [0][0][1][0][RTW89_KCC][25] = 74,
+ [0][0][1][0][RTW89_ACMA][25] = 127,
+ [0][0][1][0][RTW89_CN][25] = 127,
+ [0][0][1][0][RTW89_UK][25] = 60,
+ [0][0][1][0][RTW89_FCC][27] = 78,
+ [0][0][1][0][RTW89_ETSI][27] = 60,
+ [0][0][1][0][RTW89_MKK][27] = 74,
+ [0][0][1][0][RTW89_IC][27] = 127,
+ [0][0][1][0][RTW89_KCC][27] = 74,
+ [0][0][1][0][RTW89_ACMA][27] = 127,
+ [0][0][1][0][RTW89_CN][27] = 127,
+ [0][0][1][0][RTW89_UK][27] = 60,
+ [0][0][1][0][RTW89_FCC][29] = 78,
+ [0][0][1][0][RTW89_ETSI][29] = 60,
+ [0][0][1][0][RTW89_MKK][29] = 74,
+ [0][0][1][0][RTW89_IC][29] = 127,
+ [0][0][1][0][RTW89_KCC][29] = 74,
+ [0][0][1][0][RTW89_ACMA][29] = 127,
+ [0][0][1][0][RTW89_CN][29] = 127,
+ [0][0][1][0][RTW89_UK][29] = 60,
+ [0][0][1][0][RTW89_FCC][31] = 78,
+ [0][0][1][0][RTW89_ETSI][31] = 60,
+ [0][0][1][0][RTW89_MKK][31] = 74,
+ [0][0][1][0][RTW89_IC][31] = 78,
+ [0][0][1][0][RTW89_KCC][31] = 74,
+ [0][0][1][0][RTW89_ACMA][31] = 60,
+ [0][0][1][0][RTW89_CN][31] = 127,
+ [0][0][1][0][RTW89_UK][31] = 60,
+ [0][0][1][0][RTW89_FCC][33] = 78,
+ [0][0][1][0][RTW89_ETSI][33] = 60,
+ [0][0][1][0][RTW89_MKK][33] = 74,
+ [0][0][1][0][RTW89_IC][33] = 78,
+ [0][0][1][0][RTW89_KCC][33] = 74,
+ [0][0][1][0][RTW89_ACMA][33] = 60,
+ [0][0][1][0][RTW89_CN][33] = 127,
+ [0][0][1][0][RTW89_UK][33] = 60,
+ [0][0][1][0][RTW89_FCC][35] = 70,
+ [0][0][1][0][RTW89_ETSI][35] = 60,
+ [0][0][1][0][RTW89_MKK][35] = 74,
+ [0][0][1][0][RTW89_IC][35] = 70,
+ [0][0][1][0][RTW89_KCC][35] = 74,
+ [0][0][1][0][RTW89_ACMA][35] = 60,
+ [0][0][1][0][RTW89_CN][35] = 127,
+ [0][0][1][0][RTW89_UK][35] = 60,
+ [0][0][1][0][RTW89_FCC][37] = 78,
+ [0][0][1][0][RTW89_ETSI][37] = 127,
+ [0][0][1][0][RTW89_MKK][37] = 74,
+ [0][0][1][0][RTW89_IC][37] = 78,
+ [0][0][1][0][RTW89_KCC][37] = 74,
+ [0][0][1][0][RTW89_ACMA][37] = 74,
+ [0][0][1][0][RTW89_CN][37] = 127,
+ [0][0][1][0][RTW89_UK][37] = 74,
+ [0][0][1][0][RTW89_FCC][38] = 78,
+ [0][0][1][0][RTW89_ETSI][38] = 30,
+ [0][0][1][0][RTW89_MKK][38] = 127,
+ [0][0][1][0][RTW89_IC][38] = 78,
+ [0][0][1][0][RTW89_KCC][38] = 70,
+ [0][0][1][0][RTW89_ACMA][38] = 74,
+ [0][0][1][0][RTW89_CN][38] = 74,
+ [0][0][1][0][RTW89_UK][38] = 58,
+ [0][0][1][0][RTW89_FCC][40] = 78,
+ [0][0][1][0][RTW89_ETSI][40] = 30,
+ [0][0][1][0][RTW89_MKK][40] = 127,
+ [0][0][1][0][RTW89_IC][40] = 78,
+ [0][0][1][0][RTW89_KCC][40] = 74,
+ [0][0][1][0][RTW89_ACMA][40] = 74,
+ [0][0][1][0][RTW89_CN][40] = 74,
+ [0][0][1][0][RTW89_UK][40] = 58,
+ [0][0][1][0][RTW89_FCC][42] = 78,
+ [0][0][1][0][RTW89_ETSI][42] = 30,
+ [0][0][1][0][RTW89_MKK][42] = 127,
+ [0][0][1][0][RTW89_IC][42] = 78,
+ [0][0][1][0][RTW89_KCC][42] = 74,
+ [0][0][1][0][RTW89_ACMA][42] = 74,
+ [0][0][1][0][RTW89_CN][42] = 74,
+ [0][0][1][0][RTW89_UK][42] = 58,
+ [0][0][1][0][RTW89_FCC][44] = 78,
+ [0][0][1][0][RTW89_ETSI][44] = 30,
+ [0][0][1][0][RTW89_MKK][44] = 127,
+ [0][0][1][0][RTW89_IC][44] = 78,
+ [0][0][1][0][RTW89_KCC][44] = 74,
+ [0][0][1][0][RTW89_ACMA][44] = 74,
+ [0][0][1][0][RTW89_CN][44] = 74,
+ [0][0][1][0][RTW89_UK][44] = 58,
+ [0][0][1][0][RTW89_FCC][46] = 78,
+ [0][0][1][0][RTW89_ETSI][46] = 30,
+ [0][0][1][0][RTW89_MKK][46] = 127,
+ [0][0][1][0][RTW89_IC][46] = 78,
+ [0][0][1][0][RTW89_KCC][46] = 74,
+ [0][0][1][0][RTW89_ACMA][46] = 74,
+ [0][0][1][0][RTW89_CN][46] = 74,
+ [0][0][1][0][RTW89_UK][46] = 58,
+ [0][0][1][0][RTW89_FCC][48] = 72,
+ [0][0][1][0][RTW89_ETSI][48] = 127,
+ [0][0][1][0][RTW89_MKK][48] = 127,
+ [0][0][1][0][RTW89_IC][48] = 127,
+ [0][0][1][0][RTW89_KCC][48] = 127,
+ [0][0][1][0][RTW89_ACMA][48] = 127,
+ [0][0][1][0][RTW89_CN][48] = 127,
+ [0][0][1][0][RTW89_UK][48] = 127,
+ [0][0][1][0][RTW89_FCC][50] = 72,
+ [0][0][1][0][RTW89_ETSI][50] = 127,
+ [0][0][1][0][RTW89_MKK][50] = 127,
+ [0][0][1][0][RTW89_IC][50] = 127,
+ [0][0][1][0][RTW89_KCC][50] = 127,
+ [0][0][1][0][RTW89_ACMA][50] = 127,
+ [0][0][1][0][RTW89_CN][50] = 127,
+ [0][0][1][0][RTW89_UK][50] = 127,
+ [0][0][1][0][RTW89_FCC][52] = 72,
+ [0][0][1][0][RTW89_ETSI][52] = 127,
+ [0][0][1][0][RTW89_MKK][52] = 127,
+ [0][0][1][0][RTW89_IC][52] = 127,
+ [0][0][1][0][RTW89_KCC][52] = 127,
+ [0][0][1][0][RTW89_ACMA][52] = 127,
+ [0][0][1][0][RTW89_CN][52] = 127,
+ [0][0][1][0][RTW89_UK][52] = 127,
+ [0][1][1][0][RTW89_FCC][0] = 127,
+ [0][1][1][0][RTW89_ETSI][0] = 127,
+ [0][1][1][0][RTW89_MKK][0] = 127,
+ [0][1][1][0][RTW89_IC][0] = 127,
+ [0][1][1][0][RTW89_KCC][0] = 127,
+ [0][1][1][0][RTW89_ACMA][0] = 127,
+ [0][1][1][0][RTW89_CN][0] = 127,
+ [0][1][1][0][RTW89_UK][0] = 127,
+ [0][1][1][0][RTW89_FCC][2] = 127,
+ [0][1][1][0][RTW89_ETSI][2] = 127,
+ [0][1][1][0][RTW89_MKK][2] = 127,
+ [0][1][1][0][RTW89_IC][2] = 127,
+ [0][1][1][0][RTW89_KCC][2] = 127,
+ [0][1][1][0][RTW89_ACMA][2] = 127,
+ [0][1][1][0][RTW89_CN][2] = 127,
+ [0][1][1][0][RTW89_UK][2] = 127,
+ [0][1][1][0][RTW89_FCC][4] = 127,
+ [0][1][1][0][RTW89_ETSI][4] = 127,
+ [0][1][1][0][RTW89_MKK][4] = 127,
+ [0][1][1][0][RTW89_IC][4] = 127,
+ [0][1][1][0][RTW89_KCC][4] = 127,
+ [0][1][1][0][RTW89_ACMA][4] = 127,
+ [0][1][1][0][RTW89_CN][4] = 127,
+ [0][1][1][0][RTW89_UK][4] = 127,
+ [0][1][1][0][RTW89_FCC][6] = 127,
+ [0][1][1][0][RTW89_ETSI][6] = 127,
+ [0][1][1][0][RTW89_MKK][6] = 127,
+ [0][1][1][0][RTW89_IC][6] = 127,
+ [0][1][1][0][RTW89_KCC][6] = 127,
+ [0][1][1][0][RTW89_ACMA][6] = 127,
+ [0][1][1][0][RTW89_CN][6] = 127,
+ [0][1][1][0][RTW89_UK][6] = 127,
+ [0][1][1][0][RTW89_FCC][8] = 127,
+ [0][1][1][0][RTW89_ETSI][8] = 127,
+ [0][1][1][0][RTW89_MKK][8] = 127,
+ [0][1][1][0][RTW89_IC][8] = 127,
+ [0][1][1][0][RTW89_KCC][8] = 127,
+ [0][1][1][0][RTW89_ACMA][8] = 127,
+ [0][1][1][0][RTW89_CN][8] = 127,
+ [0][1][1][0][RTW89_UK][8] = 127,
+ [0][1][1][0][RTW89_FCC][10] = 127,
+ [0][1][1][0][RTW89_ETSI][10] = 127,
+ [0][1][1][0][RTW89_MKK][10] = 127,
+ [0][1][1][0][RTW89_IC][10] = 127,
+ [0][1][1][0][RTW89_KCC][10] = 127,
+ [0][1][1][0][RTW89_ACMA][10] = 127,
+ [0][1][1][0][RTW89_CN][10] = 127,
+ [0][1][1][0][RTW89_UK][10] = 127,
+ [0][1][1][0][RTW89_FCC][12] = 127,
+ [0][1][1][0][RTW89_ETSI][12] = 127,
+ [0][1][1][0][RTW89_MKK][12] = 127,
+ [0][1][1][0][RTW89_IC][12] = 127,
+ [0][1][1][0][RTW89_KCC][12] = 127,
+ [0][1][1][0][RTW89_ACMA][12] = 127,
+ [0][1][1][0][RTW89_CN][12] = 127,
+ [0][1][1][0][RTW89_UK][12] = 127,
+ [0][1][1][0][RTW89_FCC][14] = 127,
+ [0][1][1][0][RTW89_ETSI][14] = 127,
+ [0][1][1][0][RTW89_MKK][14] = 127,
+ [0][1][1][0][RTW89_IC][14] = 127,
+ [0][1][1][0][RTW89_KCC][14] = 127,
+ [0][1][1][0][RTW89_ACMA][14] = 127,
+ [0][1][1][0][RTW89_CN][14] = 127,
+ [0][1][1][0][RTW89_UK][14] = 127,
+ [0][1][1][0][RTW89_FCC][15] = 127,
+ [0][1][1][0][RTW89_ETSI][15] = 127,
+ [0][1][1][0][RTW89_MKK][15] = 127,
+ [0][1][1][0][RTW89_IC][15] = 127,
+ [0][1][1][0][RTW89_KCC][15] = 127,
+ [0][1][1][0][RTW89_ACMA][15] = 127,
+ [0][1][1][0][RTW89_CN][15] = 127,
+ [0][1][1][0][RTW89_UK][15] = 127,
+ [0][1][1][0][RTW89_FCC][17] = 127,
+ [0][1][1][0][RTW89_ETSI][17] = 127,
+ [0][1][1][0][RTW89_MKK][17] = 127,
+ [0][1][1][0][RTW89_IC][17] = 127,
+ [0][1][1][0][RTW89_KCC][17] = 127,
+ [0][1][1][0][RTW89_ACMA][17] = 127,
+ [0][1][1][0][RTW89_CN][17] = 127,
+ [0][1][1][0][RTW89_UK][17] = 127,
+ [0][1][1][0][RTW89_FCC][19] = 127,
+ [0][1][1][0][RTW89_ETSI][19] = 127,
+ [0][1][1][0][RTW89_MKK][19] = 127,
+ [0][1][1][0][RTW89_IC][19] = 127,
+ [0][1][1][0][RTW89_KCC][19] = 127,
+ [0][1][1][0][RTW89_ACMA][19] = 127,
+ [0][1][1][0][RTW89_CN][19] = 127,
+ [0][1][1][0][RTW89_UK][19] = 127,
+ [0][1][1][0][RTW89_FCC][21] = 127,
+ [0][1][1][0][RTW89_ETSI][21] = 127,
+ [0][1][1][0][RTW89_MKK][21] = 127,
+ [0][1][1][0][RTW89_IC][21] = 127,
+ [0][1][1][0][RTW89_KCC][21] = 127,
+ [0][1][1][0][RTW89_ACMA][21] = 127,
+ [0][1][1][0][RTW89_CN][21] = 127,
+ [0][1][1][0][RTW89_UK][21] = 127,
+ [0][1][1][0][RTW89_FCC][23] = 127,
+ [0][1][1][0][RTW89_ETSI][23] = 127,
+ [0][1][1][0][RTW89_MKK][23] = 127,
+ [0][1][1][0][RTW89_IC][23] = 127,
+ [0][1][1][0][RTW89_KCC][23] = 127,
+ [0][1][1][0][RTW89_ACMA][23] = 127,
+ [0][1][1][0][RTW89_CN][23] = 127,
+ [0][1][1][0][RTW89_UK][23] = 127,
+ [0][1][1][0][RTW89_FCC][25] = 127,
+ [0][1][1][0][RTW89_ETSI][25] = 127,
+ [0][1][1][0][RTW89_MKK][25] = 127,
+ [0][1][1][0][RTW89_IC][25] = 127,
+ [0][1][1][0][RTW89_KCC][25] = 127,
+ [0][1][1][0][RTW89_ACMA][25] = 127,
+ [0][1][1][0][RTW89_CN][25] = 127,
+ [0][1][1][0][RTW89_UK][25] = 127,
+ [0][1][1][0][RTW89_FCC][27] = 127,
+ [0][1][1][0][RTW89_ETSI][27] = 127,
+ [0][1][1][0][RTW89_MKK][27] = 127,
+ [0][1][1][0][RTW89_IC][27] = 127,
+ [0][1][1][0][RTW89_KCC][27] = 127,
+ [0][1][1][0][RTW89_ACMA][27] = 127,
+ [0][1][1][0][RTW89_CN][27] = 127,
+ [0][1][1][0][RTW89_UK][27] = 127,
+ [0][1][1][0][RTW89_FCC][29] = 127,
+ [0][1][1][0][RTW89_ETSI][29] = 127,
+ [0][1][1][0][RTW89_MKK][29] = 127,
+ [0][1][1][0][RTW89_IC][29] = 127,
+ [0][1][1][0][RTW89_KCC][29] = 127,
+ [0][1][1][0][RTW89_ACMA][29] = 127,
+ [0][1][1][0][RTW89_CN][29] = 127,
+ [0][1][1][0][RTW89_UK][29] = 127,
+ [0][1][1][0][RTW89_FCC][31] = 127,
+ [0][1][1][0][RTW89_ETSI][31] = 127,
+ [0][1][1][0][RTW89_MKK][31] = 127,
+ [0][1][1][0][RTW89_IC][31] = 127,
+ [0][1][1][0][RTW89_KCC][31] = 127,
+ [0][1][1][0][RTW89_ACMA][31] = 127,
+ [0][1][1][0][RTW89_CN][31] = 127,
+ [0][1][1][0][RTW89_UK][31] = 127,
+ [0][1][1][0][RTW89_FCC][33] = 127,
+ [0][1][1][0][RTW89_ETSI][33] = 127,
+ [0][1][1][0][RTW89_MKK][33] = 127,
+ [0][1][1][0][RTW89_IC][33] = 127,
+ [0][1][1][0][RTW89_KCC][33] = 127,
+ [0][1][1][0][RTW89_ACMA][33] = 127,
+ [0][1][1][0][RTW89_CN][33] = 127,
+ [0][1][1][0][RTW89_UK][33] = 127,
+ [0][1][1][0][RTW89_FCC][35] = 127,
+ [0][1][1][0][RTW89_ETSI][35] = 127,
+ [0][1][1][0][RTW89_MKK][35] = 127,
+ [0][1][1][0][RTW89_IC][35] = 127,
+ [0][1][1][0][RTW89_KCC][35] = 127,
+ [0][1][1][0][RTW89_ACMA][35] = 127,
+ [0][1][1][0][RTW89_CN][35] = 127,
+ [0][1][1][0][RTW89_UK][35] = 127,
+ [0][1][1][0][RTW89_FCC][37] = 127,
+ [0][1][1][0][RTW89_ETSI][37] = 127,
+ [0][1][1][0][RTW89_MKK][37] = 127,
+ [0][1][1][0][RTW89_IC][37] = 127,
+ [0][1][1][0][RTW89_KCC][37] = 127,
+ [0][1][1][0][RTW89_ACMA][37] = 127,
+ [0][1][1][0][RTW89_CN][37] = 127,
+ [0][1][1][0][RTW89_UK][37] = 127,
+ [0][1][1][0][RTW89_FCC][38] = 127,
+ [0][1][1][0][RTW89_ETSI][38] = 127,
+ [0][1][1][0][RTW89_MKK][38] = 127,
+ [0][1][1][0][RTW89_IC][38] = 127,
+ [0][1][1][0][RTW89_KCC][38] = 127,
+ [0][1][1][0][RTW89_ACMA][38] = 127,
+ [0][1][1][0][RTW89_CN][38] = 127,
+ [0][1][1][0][RTW89_UK][38] = 127,
+ [0][1][1][0][RTW89_FCC][40] = 127,
+ [0][1][1][0][RTW89_ETSI][40] = 127,
+ [0][1][1][0][RTW89_MKK][40] = 127,
+ [0][1][1][0][RTW89_IC][40] = 127,
+ [0][1][1][0][RTW89_KCC][40] = 127,
+ [0][1][1][0][RTW89_ACMA][40] = 127,
+ [0][1][1][0][RTW89_CN][40] = 127,
+ [0][1][1][0][RTW89_UK][40] = 127,
+ [0][1][1][0][RTW89_FCC][42] = 127,
+ [0][1][1][0][RTW89_ETSI][42] = 127,
+ [0][1][1][0][RTW89_MKK][42] = 127,
+ [0][1][1][0][RTW89_IC][42] = 127,
+ [0][1][1][0][RTW89_KCC][42] = 127,
+ [0][1][1][0][RTW89_ACMA][42] = 127,
+ [0][1][1][0][RTW89_CN][42] = 127,
+ [0][1][1][0][RTW89_UK][42] = 127,
+ [0][1][1][0][RTW89_FCC][44] = 127,
+ [0][1][1][0][RTW89_ETSI][44] = 127,
+ [0][1][1][0][RTW89_MKK][44] = 127,
+ [0][1][1][0][RTW89_IC][44] = 127,
+ [0][1][1][0][RTW89_KCC][44] = 127,
+ [0][1][1][0][RTW89_ACMA][44] = 127,
+ [0][1][1][0][RTW89_CN][44] = 127,
+ [0][1][1][0][RTW89_UK][44] = 127,
+ [0][1][1][0][RTW89_FCC][46] = 127,
+ [0][1][1][0][RTW89_ETSI][46] = 127,
+ [0][1][1][0][RTW89_MKK][46] = 127,
+ [0][1][1][0][RTW89_IC][46] = 127,
+ [0][1][1][0][RTW89_KCC][46] = 127,
+ [0][1][1][0][RTW89_ACMA][46] = 127,
+ [0][1][1][0][RTW89_CN][46] = 127,
+ [0][1][1][0][RTW89_UK][46] = 127,
+ [0][1][1][0][RTW89_FCC][48] = 127,
+ [0][1][1][0][RTW89_ETSI][48] = 127,
+ [0][1][1][0][RTW89_MKK][48] = 127,
+ [0][1][1][0][RTW89_IC][48] = 127,
+ [0][1][1][0][RTW89_KCC][48] = 127,
+ [0][1][1][0][RTW89_ACMA][48] = 127,
+ [0][1][1][0][RTW89_CN][48] = 127,
+ [0][1][1][0][RTW89_UK][48] = 127,
+ [0][1][1][0][RTW89_FCC][50] = 127,
+ [0][1][1][0][RTW89_ETSI][50] = 127,
+ [0][1][1][0][RTW89_MKK][50] = 127,
+ [0][1][1][0][RTW89_IC][50] = 127,
+ [0][1][1][0][RTW89_KCC][50] = 127,
+ [0][1][1][0][RTW89_ACMA][50] = 127,
+ [0][1][1][0][RTW89_CN][50] = 127,
+ [0][1][1][0][RTW89_UK][50] = 127,
+ [0][1][1][0][RTW89_FCC][52] = 127,
+ [0][1][1][0][RTW89_ETSI][52] = 127,
+ [0][1][1][0][RTW89_MKK][52] = 127,
+ [0][1][1][0][RTW89_IC][52] = 127,
+ [0][1][1][0][RTW89_KCC][52] = 127,
+ [0][1][1][0][RTW89_ACMA][52] = 127,
+ [0][1][1][0][RTW89_CN][52] = 127,
+ [0][1][1][0][RTW89_UK][52] = 127,
+ [0][0][2][0][RTW89_FCC][0] = 76,
+ [0][0][2][0][RTW89_ETSI][0] = 62,
+ [0][0][2][0][RTW89_MKK][0] = 62,
+ [0][0][2][0][RTW89_IC][0] = 64,
+ [0][0][2][0][RTW89_KCC][0] = 74,
+ [0][0][2][0][RTW89_ACMA][0] = 62,
+ [0][0][2][0][RTW89_CN][0] = 62,
+ [0][0][2][0][RTW89_UK][0] = 62,
+ [0][0][2][0][RTW89_FCC][2] = 78,
+ [0][0][2][0][RTW89_ETSI][2] = 62,
+ [0][0][2][0][RTW89_MKK][2] = 62,
+ [0][0][2][0][RTW89_IC][2] = 64,
+ [0][0][2][0][RTW89_KCC][2] = 74,
+ [0][0][2][0][RTW89_ACMA][2] = 62,
+ [0][0][2][0][RTW89_CN][2] = 62,
+ [0][0][2][0][RTW89_UK][2] = 62,
+ [0][0][2][0][RTW89_FCC][4] = 78,
+ [0][0][2][0][RTW89_ETSI][4] = 62,
+ [0][0][2][0][RTW89_MKK][4] = 62,
+ [0][0][2][0][RTW89_IC][4] = 64,
+ [0][0][2][0][RTW89_KCC][4] = 74,
+ [0][0][2][0][RTW89_ACMA][4] = 62,
+ [0][0][2][0][RTW89_CN][4] = 62,
+ [0][0][2][0][RTW89_UK][4] = 62,
+ [0][0][2][0][RTW89_FCC][6] = 78,
+ [0][0][2][0][RTW89_ETSI][6] = 62,
+ [0][0][2][0][RTW89_MKK][6] = 62,
+ [0][0][2][0][RTW89_IC][6] = 64,
+ [0][0][2][0][RTW89_KCC][6] = 54,
+ [0][0][2][0][RTW89_ACMA][6] = 62,
+ [0][0][2][0][RTW89_CN][6] = 62,
+ [0][0][2][0][RTW89_UK][6] = 62,
+ [0][0][2][0][RTW89_FCC][8] = 78,
+ [0][0][2][0][RTW89_ETSI][8] = 62,
+ [0][0][2][0][RTW89_MKK][8] = 62,
+ [0][0][2][0][RTW89_IC][8] = 64,
+ [0][0][2][0][RTW89_KCC][8] = 74,
+ [0][0][2][0][RTW89_ACMA][8] = 62,
+ [0][0][2][0][RTW89_CN][8] = 62,
+ [0][0][2][0][RTW89_UK][8] = 62,
+ [0][0][2][0][RTW89_FCC][10] = 78,
+ [0][0][2][0][RTW89_ETSI][10] = 62,
+ [0][0][2][0][RTW89_MKK][10] = 62,
+ [0][0][2][0][RTW89_IC][10] = 64,
+ [0][0][2][0][RTW89_KCC][10] = 74,
+ [0][0][2][0][RTW89_ACMA][10] = 62,
+ [0][0][2][0][RTW89_CN][10] = 62,
+ [0][0][2][0][RTW89_UK][10] = 62,
+ [0][0][2][0][RTW89_FCC][12] = 78,
+ [0][0][2][0][RTW89_ETSI][12] = 62,
+ [0][0][2][0][RTW89_MKK][12] = 62,
+ [0][0][2][0][RTW89_IC][12] = 64,
+ [0][0][2][0][RTW89_KCC][12] = 74,
+ [0][0][2][0][RTW89_ACMA][12] = 62,
+ [0][0][2][0][RTW89_CN][12] = 62,
+ [0][0][2][0][RTW89_UK][12] = 62,
+ [0][0][2][0][RTW89_FCC][14] = 74,
+ [0][0][2][0][RTW89_ETSI][14] = 62,
+ [0][0][2][0][RTW89_MKK][14] = 62,
+ [0][0][2][0][RTW89_IC][14] = 64,
+ [0][0][2][0][RTW89_KCC][14] = 74,
+ [0][0][2][0][RTW89_ACMA][14] = 62,
+ [0][0][2][0][RTW89_CN][14] = 62,
+ [0][0][2][0][RTW89_UK][14] = 62,
+ [0][0][2][0][RTW89_FCC][15] = 74,
+ [0][0][2][0][RTW89_ETSI][15] = 60,
+ [0][0][2][0][RTW89_MKK][15] = 74,
+ [0][0][2][0][RTW89_IC][15] = 74,
+ [0][0][2][0][RTW89_KCC][15] = 74,
+ [0][0][2][0][RTW89_ACMA][15] = 60,
+ [0][0][2][0][RTW89_CN][15] = 127,
+ [0][0][2][0][RTW89_UK][15] = 60,
+ [0][0][2][0][RTW89_FCC][17] = 78,
+ [0][0][2][0][RTW89_ETSI][17] = 62,
+ [0][0][2][0][RTW89_MKK][17] = 74,
+ [0][0][2][0][RTW89_IC][17] = 78,
+ [0][0][2][0][RTW89_KCC][17] = 74,
+ [0][0][2][0][RTW89_ACMA][17] = 62,
+ [0][0][2][0][RTW89_CN][17] = 127,
+ [0][0][2][0][RTW89_UK][17] = 62,
+ [0][0][2][0][RTW89_FCC][19] = 78,
+ [0][0][2][0][RTW89_ETSI][19] = 62,
+ [0][0][2][0][RTW89_MKK][19] = 74,
+ [0][0][2][0][RTW89_IC][19] = 78,
+ [0][0][2][0][RTW89_KCC][19] = 74,
+ [0][0][2][0][RTW89_ACMA][19] = 62,
+ [0][0][2][0][RTW89_CN][19] = 127,
+ [0][0][2][0][RTW89_UK][19] = 62,
+ [0][0][2][0][RTW89_FCC][21] = 78,
+ [0][0][2][0][RTW89_ETSI][21] = 62,
+ [0][0][2][0][RTW89_MKK][21] = 74,
+ [0][0][2][0][RTW89_IC][21] = 78,
+ [0][0][2][0][RTW89_KCC][21] = 74,
+ [0][0][2][0][RTW89_ACMA][21] = 62,
+ [0][0][2][0][RTW89_CN][21] = 127,
+ [0][0][2][0][RTW89_UK][21] = 62,
+ [0][0][2][0][RTW89_FCC][23] = 78,
+ [0][0][2][0][RTW89_ETSI][23] = 62,
+ [0][0][2][0][RTW89_MKK][23] = 74,
+ [0][0][2][0][RTW89_IC][23] = 78,
+ [0][0][2][0][RTW89_KCC][23] = 74,
+ [0][0][2][0][RTW89_ACMA][23] = 62,
+ [0][0][2][0][RTW89_CN][23] = 127,
+ [0][0][2][0][RTW89_UK][23] = 62,
+ [0][0][2][0][RTW89_FCC][25] = 78,
+ [0][0][2][0][RTW89_ETSI][25] = 62,
+ [0][0][2][0][RTW89_MKK][25] = 74,
+ [0][0][2][0][RTW89_IC][25] = 127,
+ [0][0][2][0][RTW89_KCC][25] = 74,
+ [0][0][2][0][RTW89_ACMA][25] = 127,
+ [0][0][2][0][RTW89_CN][25] = 127,
+ [0][0][2][0][RTW89_UK][25] = 62,
+ [0][0][2][0][RTW89_FCC][27] = 78,
+ [0][0][2][0][RTW89_ETSI][27] = 62,
+ [0][0][2][0][RTW89_MKK][27] = 74,
+ [0][0][2][0][RTW89_IC][27] = 127,
+ [0][0][2][0][RTW89_KCC][27] = 74,
+ [0][0][2][0][RTW89_ACMA][27] = 127,
+ [0][0][2][0][RTW89_CN][27] = 127,
+ [0][0][2][0][RTW89_UK][27] = 62,
+ [0][0][2][0][RTW89_FCC][29] = 78,
+ [0][0][2][0][RTW89_ETSI][29] = 62,
+ [0][0][2][0][RTW89_MKK][29] = 74,
+ [0][0][2][0][RTW89_IC][29] = 127,
+ [0][0][2][0][RTW89_KCC][29] = 74,
+ [0][0][2][0][RTW89_ACMA][29] = 127,
+ [0][0][2][0][RTW89_CN][29] = 127,
+ [0][0][2][0][RTW89_UK][29] = 62,
+ [0][0][2][0][RTW89_FCC][31] = 78,
+ [0][0][2][0][RTW89_ETSI][31] = 62,
+ [0][0][2][0][RTW89_MKK][31] = 74,
+ [0][0][2][0][RTW89_IC][31] = 78,
+ [0][0][2][0][RTW89_KCC][31] = 74,
+ [0][0][2][0][RTW89_ACMA][31] = 62,
+ [0][0][2][0][RTW89_CN][31] = 127,
+ [0][0][2][0][RTW89_UK][31] = 62,
+ [0][0][2][0][RTW89_FCC][33] = 78,
+ [0][0][2][0][RTW89_ETSI][33] = 62,
+ [0][0][2][0][RTW89_MKK][33] = 74,
+ [0][0][2][0][RTW89_IC][33] = 78,
+ [0][0][2][0][RTW89_KCC][33] = 74,
+ [0][0][2][0][RTW89_ACMA][33] = 62,
+ [0][0][2][0][RTW89_CN][33] = 127,
+ [0][0][2][0][RTW89_UK][33] = 62,
+ [0][0][2][0][RTW89_FCC][35] = 72,
+ [0][0][2][0][RTW89_ETSI][35] = 62,
+ [0][0][2][0][RTW89_MKK][35] = 74,
+ [0][0][2][0][RTW89_IC][35] = 72,
+ [0][0][2][0][RTW89_KCC][35] = 74,
+ [0][0][2][0][RTW89_ACMA][35] = 62,
+ [0][0][2][0][RTW89_CN][35] = 127,
+ [0][0][2][0][RTW89_UK][35] = 62,
+ [0][0][2][0][RTW89_FCC][37] = 78,
+ [0][0][2][0][RTW89_ETSI][37] = 127,
+ [0][0][2][0][RTW89_MKK][37] = 74,
+ [0][0][2][0][RTW89_IC][37] = 78,
+ [0][0][2][0][RTW89_KCC][37] = 74,
+ [0][0][2][0][RTW89_ACMA][37] = 74,
+ [0][0][2][0][RTW89_CN][37] = 127,
+ [0][0][2][0][RTW89_UK][37] = 74,
+ [0][0][2][0][RTW89_FCC][38] = 78,
+ [0][0][2][0][RTW89_ETSI][38] = 30,
+ [0][0][2][0][RTW89_MKK][38] = 127,
+ [0][0][2][0][RTW89_IC][38] = 78,
+ [0][0][2][0][RTW89_KCC][38] = 66,
+ [0][0][2][0][RTW89_ACMA][38] = 74,
+ [0][0][2][0][RTW89_CN][38] = 74,
+ [0][0][2][0][RTW89_UK][38] = 60,
+ [0][0][2][0][RTW89_FCC][40] = 78,
+ [0][0][2][0][RTW89_ETSI][40] = 30,
+ [0][0][2][0][RTW89_MKK][40] = 127,
+ [0][0][2][0][RTW89_IC][40] = 78,
+ [0][0][2][0][RTW89_KCC][40] = 74,
+ [0][0][2][0][RTW89_ACMA][40] = 74,
+ [0][0][2][0][RTW89_CN][40] = 74,
+ [0][0][2][0][RTW89_UK][40] = 60,
+ [0][0][2][0][RTW89_FCC][42] = 78,
+ [0][0][2][0][RTW89_ETSI][42] = 30,
+ [0][0][2][0][RTW89_MKK][42] = 127,
+ [0][0][2][0][RTW89_IC][42] = 78,
+ [0][0][2][0][RTW89_KCC][42] = 74,
+ [0][0][2][0][RTW89_ACMA][42] = 74,
+ [0][0][2][0][RTW89_CN][42] = 74,
+ [0][0][2][0][RTW89_UK][42] = 60,
+ [0][0][2][0][RTW89_FCC][44] = 78,
+ [0][0][2][0][RTW89_ETSI][44] = 30,
+ [0][0][2][0][RTW89_MKK][44] = 127,
+ [0][0][2][0][RTW89_IC][44] = 78,
+ [0][0][2][0][RTW89_KCC][44] = 74,
+ [0][0][2][0][RTW89_ACMA][44] = 74,
+ [0][0][2][0][RTW89_CN][44] = 74,
+ [0][0][2][0][RTW89_UK][44] = 60,
+ [0][0][2][0][RTW89_FCC][46] = 78,
+ [0][0][2][0][RTW89_ETSI][46] = 30,
+ [0][0][2][0][RTW89_MKK][46] = 127,
+ [0][0][2][0][RTW89_IC][46] = 78,
+ [0][0][2][0][RTW89_KCC][46] = 74,
+ [0][0][2][0][RTW89_ACMA][46] = 74,
+ [0][0][2][0][RTW89_CN][46] = 74,
+ [0][0][2][0][RTW89_UK][46] = 60,
+ [0][0][2][0][RTW89_FCC][48] = 74,
+ [0][0][2][0][RTW89_ETSI][48] = 127,
+ [0][0][2][0][RTW89_MKK][48] = 127,
+ [0][0][2][0][RTW89_IC][48] = 127,
+ [0][0][2][0][RTW89_KCC][48] = 127,
+ [0][0][2][0][RTW89_ACMA][48] = 127,
+ [0][0][2][0][RTW89_CN][48] = 127,
+ [0][0][2][0][RTW89_UK][48] = 127,
+ [0][0][2][0][RTW89_FCC][50] = 74,
+ [0][0][2][0][RTW89_ETSI][50] = 127,
+ [0][0][2][0][RTW89_MKK][50] = 127,
+ [0][0][2][0][RTW89_IC][50] = 127,
+ [0][0][2][0][RTW89_KCC][50] = 127,
+ [0][0][2][0][RTW89_ACMA][50] = 127,
+ [0][0][2][0][RTW89_CN][50] = 127,
+ [0][0][2][0][RTW89_UK][50] = 127,
+ [0][0][2][0][RTW89_FCC][52] = 74,
+ [0][0][2][0][RTW89_ETSI][52] = 127,
+ [0][0][2][0][RTW89_MKK][52] = 127,
+ [0][0][2][0][RTW89_IC][52] = 127,
+ [0][0][2][0][RTW89_KCC][52] = 127,
+ [0][0][2][0][RTW89_ACMA][52] = 127,
+ [0][0][2][0][RTW89_CN][52] = 127,
+ [0][0][2][0][RTW89_UK][52] = 127,
+ [0][1][2][0][RTW89_FCC][0] = 127,
+ [0][1][2][0][RTW89_ETSI][0] = 127,
+ [0][1][2][0][RTW89_MKK][0] = 127,
+ [0][1][2][0][RTW89_IC][0] = 127,
+ [0][1][2][0][RTW89_KCC][0] = 127,
+ [0][1][2][0][RTW89_ACMA][0] = 127,
+ [0][1][2][0][RTW89_CN][0] = 127,
+ [0][1][2][0][RTW89_UK][0] = 127,
+ [0][1][2][0][RTW89_FCC][2] = 127,
+ [0][1][2][0][RTW89_ETSI][2] = 127,
+ [0][1][2][0][RTW89_MKK][2] = 127,
+ [0][1][2][0][RTW89_IC][2] = 127,
+ [0][1][2][0][RTW89_KCC][2] = 127,
+ [0][1][2][0][RTW89_ACMA][2] = 127,
+ [0][1][2][0][RTW89_CN][2] = 127,
+ [0][1][2][0][RTW89_UK][2] = 127,
+ [0][1][2][0][RTW89_FCC][4] = 127,
+ [0][1][2][0][RTW89_ETSI][4] = 127,
+ [0][1][2][0][RTW89_MKK][4] = 127,
+ [0][1][2][0][RTW89_IC][4] = 127,
+ [0][1][2][0][RTW89_KCC][4] = 127,
+ [0][1][2][0][RTW89_ACMA][4] = 127,
+ [0][1][2][0][RTW89_CN][4] = 127,
+ [0][1][2][0][RTW89_UK][4] = 127,
+ [0][1][2][0][RTW89_FCC][6] = 127,
+ [0][1][2][0][RTW89_ETSI][6] = 127,
+ [0][1][2][0][RTW89_MKK][6] = 127,
+ [0][1][2][0][RTW89_IC][6] = 127,
+ [0][1][2][0][RTW89_KCC][6] = 127,
+ [0][1][2][0][RTW89_ACMA][6] = 127,
+ [0][1][2][0][RTW89_CN][6] = 127,
+ [0][1][2][0][RTW89_UK][6] = 127,
+ [0][1][2][0][RTW89_FCC][8] = 127,
+ [0][1][2][0][RTW89_ETSI][8] = 127,
+ [0][1][2][0][RTW89_MKK][8] = 127,
+ [0][1][2][0][RTW89_IC][8] = 127,
+ [0][1][2][0][RTW89_KCC][8] = 127,
+ [0][1][2][0][RTW89_ACMA][8] = 127,
+ [0][1][2][0][RTW89_CN][8] = 127,
+ [0][1][2][0][RTW89_UK][8] = 127,
+ [0][1][2][0][RTW89_FCC][10] = 127,
+ [0][1][2][0][RTW89_ETSI][10] = 127,
+ [0][1][2][0][RTW89_MKK][10] = 127,
+ [0][1][2][0][RTW89_IC][10] = 127,
+ [0][1][2][0][RTW89_KCC][10] = 127,
+ [0][1][2][0][RTW89_ACMA][10] = 127,
+ [0][1][2][0][RTW89_CN][10] = 127,
+ [0][1][2][0][RTW89_UK][10] = 127,
+ [0][1][2][0][RTW89_FCC][12] = 127,
+ [0][1][2][0][RTW89_ETSI][12] = 127,
+ [0][1][2][0][RTW89_MKK][12] = 127,
+ [0][1][2][0][RTW89_IC][12] = 127,
+ [0][1][2][0][RTW89_KCC][12] = 127,
+ [0][1][2][0][RTW89_ACMA][12] = 127,
+ [0][1][2][0][RTW89_CN][12] = 127,
+ [0][1][2][0][RTW89_UK][12] = 127,
+ [0][1][2][0][RTW89_FCC][14] = 127,
+ [0][1][2][0][RTW89_ETSI][14] = 127,
+ [0][1][2][0][RTW89_MKK][14] = 127,
+ [0][1][2][0][RTW89_IC][14] = 127,
+ [0][1][2][0][RTW89_KCC][14] = 127,
+ [0][1][2][0][RTW89_ACMA][14] = 127,
+ [0][1][2][0][RTW89_CN][14] = 127,
+ [0][1][2][0][RTW89_UK][14] = 127,
+ [0][1][2][0][RTW89_FCC][15] = 127,
+ [0][1][2][0][RTW89_ETSI][15] = 127,
+ [0][1][2][0][RTW89_MKK][15] = 127,
+ [0][1][2][0][RTW89_IC][15] = 127,
+ [0][1][2][0][RTW89_KCC][15] = 127,
+ [0][1][2][0][RTW89_ACMA][15] = 127,
+ [0][1][2][0][RTW89_CN][15] = 127,
+ [0][1][2][0][RTW89_UK][15] = 127,
+ [0][1][2][0][RTW89_FCC][17] = 127,
+ [0][1][2][0][RTW89_ETSI][17] = 127,
+ [0][1][2][0][RTW89_MKK][17] = 127,
+ [0][1][2][0][RTW89_IC][17] = 127,
+ [0][1][2][0][RTW89_KCC][17] = 127,
+ [0][1][2][0][RTW89_ACMA][17] = 127,
+ [0][1][2][0][RTW89_CN][17] = 127,
+ [0][1][2][0][RTW89_UK][17] = 127,
+ [0][1][2][0][RTW89_FCC][19] = 127,
+ [0][1][2][0][RTW89_ETSI][19] = 127,
+ [0][1][2][0][RTW89_MKK][19] = 127,
+ [0][1][2][0][RTW89_IC][19] = 127,
+ [0][1][2][0][RTW89_KCC][19] = 127,
+ [0][1][2][0][RTW89_ACMA][19] = 127,
+ [0][1][2][0][RTW89_CN][19] = 127,
+ [0][1][2][0][RTW89_UK][19] = 127,
+ [0][1][2][0][RTW89_FCC][21] = 127,
+ [0][1][2][0][RTW89_ETSI][21] = 127,
+ [0][1][2][0][RTW89_MKK][21] = 127,
+ [0][1][2][0][RTW89_IC][21] = 127,
+ [0][1][2][0][RTW89_KCC][21] = 127,
+ [0][1][2][0][RTW89_ACMA][21] = 127,
+ [0][1][2][0][RTW89_CN][21] = 127,
+ [0][1][2][0][RTW89_UK][21] = 127,
+ [0][1][2][0][RTW89_FCC][23] = 127,
+ [0][1][2][0][RTW89_ETSI][23] = 127,
+ [0][1][2][0][RTW89_MKK][23] = 127,
+ [0][1][2][0][RTW89_IC][23] = 127,
+ [0][1][2][0][RTW89_KCC][23] = 127,
+ [0][1][2][0][RTW89_ACMA][23] = 127,
+ [0][1][2][0][RTW89_CN][23] = 127,
+ [0][1][2][0][RTW89_UK][23] = 127,
+ [0][1][2][0][RTW89_FCC][25] = 127,
+ [0][1][2][0][RTW89_ETSI][25] = 127,
+ [0][1][2][0][RTW89_MKK][25] = 127,
+ [0][1][2][0][RTW89_IC][25] = 127,
+ [0][1][2][0][RTW89_KCC][25] = 127,
+ [0][1][2][0][RTW89_ACMA][25] = 127,
+ [0][1][2][0][RTW89_CN][25] = 127,
+ [0][1][2][0][RTW89_UK][25] = 127,
+ [0][1][2][0][RTW89_FCC][27] = 127,
+ [0][1][2][0][RTW89_ETSI][27] = 127,
+ [0][1][2][0][RTW89_MKK][27] = 127,
+ [0][1][2][0][RTW89_IC][27] = 127,
+ [0][1][2][0][RTW89_KCC][27] = 127,
+ [0][1][2][0][RTW89_ACMA][27] = 127,
+ [0][1][2][0][RTW89_CN][27] = 127,
+ [0][1][2][0][RTW89_UK][27] = 127,
+ [0][1][2][0][RTW89_FCC][29] = 127,
+ [0][1][2][0][RTW89_ETSI][29] = 127,
+ [0][1][2][0][RTW89_MKK][29] = 127,
+ [0][1][2][0][RTW89_IC][29] = 127,
+ [0][1][2][0][RTW89_KCC][29] = 127,
+ [0][1][2][0][RTW89_ACMA][29] = 127,
+ [0][1][2][0][RTW89_CN][29] = 127,
+ [0][1][2][0][RTW89_UK][29] = 127,
+ [0][1][2][0][RTW89_FCC][31] = 127,
+ [0][1][2][0][RTW89_ETSI][31] = 127,
+ [0][1][2][0][RTW89_MKK][31] = 127,
+ [0][1][2][0][RTW89_IC][31] = 127,
+ [0][1][2][0][RTW89_KCC][31] = 127,
+ [0][1][2][0][RTW89_ACMA][31] = 127,
+ [0][1][2][0][RTW89_CN][31] = 127,
+ [0][1][2][0][RTW89_UK][31] = 127,
+ [0][1][2][0][RTW89_FCC][33] = 127,
+ [0][1][2][0][RTW89_ETSI][33] = 127,
+ [0][1][2][0][RTW89_MKK][33] = 127,
+ [0][1][2][0][RTW89_IC][33] = 127,
+ [0][1][2][0][RTW89_KCC][33] = 127,
+ [0][1][2][0][RTW89_ACMA][33] = 127,
+ [0][1][2][0][RTW89_CN][33] = 127,
+ [0][1][2][0][RTW89_UK][33] = 127,
+ [0][1][2][0][RTW89_FCC][35] = 127,
+ [0][1][2][0][RTW89_ETSI][35] = 127,
+ [0][1][2][0][RTW89_MKK][35] = 127,
+ [0][1][2][0][RTW89_IC][35] = 127,
+ [0][1][2][0][RTW89_KCC][35] = 127,
+ [0][1][2][0][RTW89_ACMA][35] = 127,
+ [0][1][2][0][RTW89_CN][35] = 127,
+ [0][1][2][0][RTW89_UK][35] = 127,
+ [0][1][2][0][RTW89_FCC][37] = 127,
+ [0][1][2][0][RTW89_ETSI][37] = 127,
+ [0][1][2][0][RTW89_MKK][37] = 127,
+ [0][1][2][0][RTW89_IC][37] = 127,
+ [0][1][2][0][RTW89_KCC][37] = 127,
+ [0][1][2][0][RTW89_ACMA][37] = 127,
+ [0][1][2][0][RTW89_CN][37] = 127,
+ [0][1][2][0][RTW89_UK][37] = 127,
+ [0][1][2][0][RTW89_FCC][38] = 127,
+ [0][1][2][0][RTW89_ETSI][38] = 127,
+ [0][1][2][0][RTW89_MKK][38] = 127,
+ [0][1][2][0][RTW89_IC][38] = 127,
+ [0][1][2][0][RTW89_KCC][38] = 127,
+ [0][1][2][0][RTW89_ACMA][38] = 127,
+ [0][1][2][0][RTW89_CN][38] = 127,
+ [0][1][2][0][RTW89_UK][38] = 127,
+ [0][1][2][0][RTW89_FCC][40] = 127,
+ [0][1][2][0][RTW89_ETSI][40] = 127,
+ [0][1][2][0][RTW89_MKK][40] = 127,
+ [0][1][2][0][RTW89_IC][40] = 127,
+ [0][1][2][0][RTW89_KCC][40] = 127,
+ [0][1][2][0][RTW89_ACMA][40] = 127,
+ [0][1][2][0][RTW89_CN][40] = 127,
+ [0][1][2][0][RTW89_UK][40] = 127,
+ [0][1][2][0][RTW89_FCC][42] = 127,
+ [0][1][2][0][RTW89_ETSI][42] = 127,
+ [0][1][2][0][RTW89_MKK][42] = 127,
+ [0][1][2][0][RTW89_IC][42] = 127,
+ [0][1][2][0][RTW89_KCC][42] = 127,
+ [0][1][2][0][RTW89_ACMA][42] = 127,
+ [0][1][2][0][RTW89_CN][42] = 127,
+ [0][1][2][0][RTW89_UK][42] = 127,
+ [0][1][2][0][RTW89_FCC][44] = 127,
+ [0][1][2][0][RTW89_ETSI][44] = 127,
+ [0][1][2][0][RTW89_MKK][44] = 127,
+ [0][1][2][0][RTW89_IC][44] = 127,
+ [0][1][2][0][RTW89_KCC][44] = 127,
+ [0][1][2][0][RTW89_ACMA][44] = 127,
+ [0][1][2][0][RTW89_CN][44] = 127,
+ [0][1][2][0][RTW89_UK][44] = 127,
+ [0][1][2][0][RTW89_FCC][46] = 127,
+ [0][1][2][0][RTW89_ETSI][46] = 127,
+ [0][1][2][0][RTW89_MKK][46] = 127,
+ [0][1][2][0][RTW89_IC][46] = 127,
+ [0][1][2][0][RTW89_KCC][46] = 127,
+ [0][1][2][0][RTW89_ACMA][46] = 127,
+ [0][1][2][0][RTW89_CN][46] = 127,
+ [0][1][2][0][RTW89_UK][46] = 127,
+ [0][1][2][0][RTW89_FCC][48] = 127,
+ [0][1][2][0][RTW89_ETSI][48] = 127,
+ [0][1][2][0][RTW89_MKK][48] = 127,
+ [0][1][2][0][RTW89_IC][48] = 127,
+ [0][1][2][0][RTW89_KCC][48] = 127,
+ [0][1][2][0][RTW89_ACMA][48] = 127,
+ [0][1][2][0][RTW89_CN][48] = 127,
+ [0][1][2][0][RTW89_UK][48] = 127,
+ [0][1][2][0][RTW89_FCC][50] = 127,
+ [0][1][2][0][RTW89_ETSI][50] = 127,
+ [0][1][2][0][RTW89_MKK][50] = 127,
+ [0][1][2][0][RTW89_IC][50] = 127,
+ [0][1][2][0][RTW89_KCC][50] = 127,
+ [0][1][2][0][RTW89_ACMA][50] = 127,
+ [0][1][2][0][RTW89_CN][50] = 127,
+ [0][1][2][0][RTW89_UK][50] = 127,
+ [0][1][2][0][RTW89_FCC][52] = 127,
+ [0][1][2][0][RTW89_ETSI][52] = 127,
+ [0][1][2][0][RTW89_MKK][52] = 127,
+ [0][1][2][0][RTW89_IC][52] = 127,
+ [0][1][2][0][RTW89_KCC][52] = 127,
+ [0][1][2][0][RTW89_ACMA][52] = 127,
+ [0][1][2][0][RTW89_CN][52] = 127,
+ [0][1][2][0][RTW89_UK][52] = 127,
+ [0][1][2][1][RTW89_FCC][0] = 127,
+ [0][1][2][1][RTW89_ETSI][0] = 127,
+ [0][1][2][1][RTW89_MKK][0] = 127,
+ [0][1][2][1][RTW89_IC][0] = 127,
+ [0][1][2][1][RTW89_KCC][0] = 127,
+ [0][1][2][1][RTW89_ACMA][0] = 127,
+ [0][1][2][1][RTW89_CN][0] = 127,
+ [0][1][2][1][RTW89_UK][0] = 127,
+ [0][1][2][1][RTW89_FCC][2] = 127,
+ [0][1][2][1][RTW89_ETSI][2] = 127,
+ [0][1][2][1][RTW89_MKK][2] = 127,
+ [0][1][2][1][RTW89_IC][2] = 127,
+ [0][1][2][1][RTW89_KCC][2] = 127,
+ [0][1][2][1][RTW89_ACMA][2] = 127,
+ [0][1][2][1][RTW89_CN][2] = 127,
+ [0][1][2][1][RTW89_UK][2] = 127,
+ [0][1][2][1][RTW89_FCC][4] = 127,
+ [0][1][2][1][RTW89_ETSI][4] = 127,
+ [0][1][2][1][RTW89_MKK][4] = 127,
+ [0][1][2][1][RTW89_IC][4] = 127,
+ [0][1][2][1][RTW89_KCC][4] = 127,
+ [0][1][2][1][RTW89_ACMA][4] = 127,
+ [0][1][2][1][RTW89_CN][4] = 127,
+ [0][1][2][1][RTW89_UK][4] = 127,
+ [0][1][2][1][RTW89_FCC][6] = 127,
+ [0][1][2][1][RTW89_ETSI][6] = 127,
+ [0][1][2][1][RTW89_MKK][6] = 127,
+ [0][1][2][1][RTW89_IC][6] = 127,
+ [0][1][2][1][RTW89_KCC][6] = 127,
+ [0][1][2][1][RTW89_ACMA][6] = 127,
+ [0][1][2][1][RTW89_CN][6] = 127,
+ [0][1][2][1][RTW89_UK][6] = 127,
+ [0][1][2][1][RTW89_FCC][8] = 127,
+ [0][1][2][1][RTW89_ETSI][8] = 127,
+ [0][1][2][1][RTW89_MKK][8] = 127,
+ [0][1][2][1][RTW89_IC][8] = 127,
+ [0][1][2][1][RTW89_KCC][8] = 127,
+ [0][1][2][1][RTW89_ACMA][8] = 127,
+ [0][1][2][1][RTW89_CN][8] = 127,
+ [0][1][2][1][RTW89_UK][8] = 127,
+ [0][1][2][1][RTW89_FCC][10] = 127,
+ [0][1][2][1][RTW89_ETSI][10] = 127,
+ [0][1][2][1][RTW89_MKK][10] = 127,
+ [0][1][2][1][RTW89_IC][10] = 127,
+ [0][1][2][1][RTW89_KCC][10] = 127,
+ [0][1][2][1][RTW89_ACMA][10] = 127,
+ [0][1][2][1][RTW89_CN][10] = 127,
+ [0][1][2][1][RTW89_UK][10] = 127,
+ [0][1][2][1][RTW89_FCC][12] = 127,
+ [0][1][2][1][RTW89_ETSI][12] = 127,
+ [0][1][2][1][RTW89_MKK][12] = 127,
+ [0][1][2][1][RTW89_IC][12] = 127,
+ [0][1][2][1][RTW89_KCC][12] = 127,
+ [0][1][2][1][RTW89_ACMA][12] = 127,
+ [0][1][2][1][RTW89_CN][12] = 127,
+ [0][1][2][1][RTW89_UK][12] = 127,
+ [0][1][2][1][RTW89_FCC][14] = 127,
+ [0][1][2][1][RTW89_ETSI][14] = 127,
+ [0][1][2][1][RTW89_MKK][14] = 127,
+ [0][1][2][1][RTW89_IC][14] = 127,
+ [0][1][2][1][RTW89_KCC][14] = 127,
+ [0][1][2][1][RTW89_ACMA][14] = 127,
+ [0][1][2][1][RTW89_CN][14] = 127,
+ [0][1][2][1][RTW89_UK][14] = 127,
+ [0][1][2][1][RTW89_FCC][15] = 127,
+ [0][1][2][1][RTW89_ETSI][15] = 127,
+ [0][1][2][1][RTW89_MKK][15] = 127,
+ [0][1][2][1][RTW89_IC][15] = 127,
+ [0][1][2][1][RTW89_KCC][15] = 127,
+ [0][1][2][1][RTW89_ACMA][15] = 127,
+ [0][1][2][1][RTW89_CN][15] = 127,
+ [0][1][2][1][RTW89_UK][15] = 127,
+ [0][1][2][1][RTW89_FCC][17] = 127,
+ [0][1][2][1][RTW89_ETSI][17] = 127,
+ [0][1][2][1][RTW89_MKK][17] = 127,
+ [0][1][2][1][RTW89_IC][17] = 127,
+ [0][1][2][1][RTW89_KCC][17] = 127,
+ [0][1][2][1][RTW89_ACMA][17] = 127,
+ [0][1][2][1][RTW89_CN][17] = 127,
+ [0][1][2][1][RTW89_UK][17] = 127,
+ [0][1][2][1][RTW89_FCC][19] = 127,
+ [0][1][2][1][RTW89_ETSI][19] = 127,
+ [0][1][2][1][RTW89_MKK][19] = 127,
+ [0][1][2][1][RTW89_IC][19] = 127,
+ [0][1][2][1][RTW89_KCC][19] = 127,
+ [0][1][2][1][RTW89_ACMA][19] = 127,
+ [0][1][2][1][RTW89_CN][19] = 127,
+ [0][1][2][1][RTW89_UK][19] = 127,
+ [0][1][2][1][RTW89_FCC][21] = 127,
+ [0][1][2][1][RTW89_ETSI][21] = 127,
+ [0][1][2][1][RTW89_MKK][21] = 127,
+ [0][1][2][1][RTW89_IC][21] = 127,
+ [0][1][2][1][RTW89_KCC][21] = 127,
+ [0][1][2][1][RTW89_ACMA][21] = 127,
+ [0][1][2][1][RTW89_CN][21] = 127,
+ [0][1][2][1][RTW89_UK][21] = 127,
+ [0][1][2][1][RTW89_FCC][23] = 127,
+ [0][1][2][1][RTW89_ETSI][23] = 127,
+ [0][1][2][1][RTW89_MKK][23] = 127,
+ [0][1][2][1][RTW89_IC][23] = 127,
+ [0][1][2][1][RTW89_KCC][23] = 127,
+ [0][1][2][1][RTW89_ACMA][23] = 127,
+ [0][1][2][1][RTW89_CN][23] = 127,
+ [0][1][2][1][RTW89_UK][23] = 127,
+ [0][1][2][1][RTW89_FCC][25] = 127,
+ [0][1][2][1][RTW89_ETSI][25] = 127,
+ [0][1][2][1][RTW89_MKK][25] = 127,
+ [0][1][2][1][RTW89_IC][25] = 127,
+ [0][1][2][1][RTW89_KCC][25] = 127,
+ [0][1][2][1][RTW89_ACMA][25] = 127,
+ [0][1][2][1][RTW89_CN][25] = 127,
+ [0][1][2][1][RTW89_UK][25] = 127,
+ [0][1][2][1][RTW89_FCC][27] = 127,
+ [0][1][2][1][RTW89_ETSI][27] = 127,
+ [0][1][2][1][RTW89_MKK][27] = 127,
+ [0][1][2][1][RTW89_IC][27] = 127,
+ [0][1][2][1][RTW89_KCC][27] = 127,
+ [0][1][2][1][RTW89_ACMA][27] = 127,
+ [0][1][2][1][RTW89_CN][27] = 127,
+ [0][1][2][1][RTW89_UK][27] = 127,
+ [0][1][2][1][RTW89_FCC][29] = 127,
+ [0][1][2][1][RTW89_ETSI][29] = 127,
+ [0][1][2][1][RTW89_MKK][29] = 127,
+ [0][1][2][1][RTW89_IC][29] = 127,
+ [0][1][2][1][RTW89_KCC][29] = 127,
+ [0][1][2][1][RTW89_ACMA][29] = 127,
+ [0][1][2][1][RTW89_CN][29] = 127,
+ [0][1][2][1][RTW89_UK][29] = 127,
+ [0][1][2][1][RTW89_FCC][31] = 127,
+ [0][1][2][1][RTW89_ETSI][31] = 127,
+ [0][1][2][1][RTW89_MKK][31] = 127,
+ [0][1][2][1][RTW89_IC][31] = 127,
+ [0][1][2][1][RTW89_KCC][31] = 127,
+ [0][1][2][1][RTW89_ACMA][31] = 127,
+ [0][1][2][1][RTW89_CN][31] = 127,
+ [0][1][2][1][RTW89_UK][31] = 127,
+ [0][1][2][1][RTW89_FCC][33] = 127,
+ [0][1][2][1][RTW89_ETSI][33] = 127,
+ [0][1][2][1][RTW89_MKK][33] = 127,
+ [0][1][2][1][RTW89_IC][33] = 127,
+ [0][1][2][1][RTW89_KCC][33] = 127,
+ [0][1][2][1][RTW89_ACMA][33] = 127,
+ [0][1][2][1][RTW89_CN][33] = 127,
+ [0][1][2][1][RTW89_UK][33] = 127,
+ [0][1][2][1][RTW89_FCC][35] = 127,
+ [0][1][2][1][RTW89_ETSI][35] = 127,
+ [0][1][2][1][RTW89_MKK][35] = 127,
+ [0][1][2][1][RTW89_IC][35] = 127,
+ [0][1][2][1][RTW89_KCC][35] = 127,
+ [0][1][2][1][RTW89_ACMA][35] = 127,
+ [0][1][2][1][RTW89_CN][35] = 127,
+ [0][1][2][1][RTW89_UK][35] = 127,
+ [0][1][2][1][RTW89_FCC][37] = 127,
+ [0][1][2][1][RTW89_ETSI][37] = 127,
+ [0][1][2][1][RTW89_MKK][37] = 127,
+ [0][1][2][1][RTW89_IC][37] = 127,
+ [0][1][2][1][RTW89_KCC][37] = 127,
+ [0][1][2][1][RTW89_ACMA][37] = 127,
+ [0][1][2][1][RTW89_CN][37] = 127,
+ [0][1][2][1][RTW89_UK][37] = 127,
+ [0][1][2][1][RTW89_FCC][38] = 127,
+ [0][1][2][1][RTW89_ETSI][38] = 127,
+ [0][1][2][1][RTW89_MKK][38] = 127,
+ [0][1][2][1][RTW89_IC][38] = 127,
+ [0][1][2][1][RTW89_KCC][38] = 127,
+ [0][1][2][1][RTW89_ACMA][38] = 127,
+ [0][1][2][1][RTW89_CN][38] = 127,
+ [0][1][2][1][RTW89_UK][38] = 127,
+ [0][1][2][1][RTW89_FCC][40] = 127,
+ [0][1][2][1][RTW89_ETSI][40] = 127,
+ [0][1][2][1][RTW89_MKK][40] = 127,
+ [0][1][2][1][RTW89_IC][40] = 127,
+ [0][1][2][1][RTW89_KCC][40] = 127,
+ [0][1][2][1][RTW89_ACMA][40] = 127,
+ [0][1][2][1][RTW89_CN][40] = 127,
+ [0][1][2][1][RTW89_UK][40] = 127,
+ [0][1][2][1][RTW89_FCC][42] = 127,
+ [0][1][2][1][RTW89_ETSI][42] = 127,
+ [0][1][2][1][RTW89_MKK][42] = 127,
+ [0][1][2][1][RTW89_IC][42] = 127,
+ [0][1][2][1][RTW89_KCC][42] = 127,
+ [0][1][2][1][RTW89_ACMA][42] = 127,
+ [0][1][2][1][RTW89_CN][42] = 127,
+ [0][1][2][1][RTW89_UK][42] = 127,
+ [0][1][2][1][RTW89_FCC][44] = 127,
+ [0][1][2][1][RTW89_ETSI][44] = 127,
+ [0][1][2][1][RTW89_MKK][44] = 127,
+ [0][1][2][1][RTW89_IC][44] = 127,
+ [0][1][2][1][RTW89_KCC][44] = 127,
+ [0][1][2][1][RTW89_ACMA][44] = 127,
+ [0][1][2][1][RTW89_CN][44] = 127,
+ [0][1][2][1][RTW89_UK][44] = 127,
+ [0][1][2][1][RTW89_FCC][46] = 127,
+ [0][1][2][1][RTW89_ETSI][46] = 127,
+ [0][1][2][1][RTW89_MKK][46] = 127,
+ [0][1][2][1][RTW89_IC][46] = 127,
+ [0][1][2][1][RTW89_KCC][46] = 127,
+ [0][1][2][1][RTW89_ACMA][46] = 127,
+ [0][1][2][1][RTW89_CN][46] = 127,
+ [0][1][2][1][RTW89_UK][46] = 127,
+ [0][1][2][1][RTW89_FCC][48] = 127,
+ [0][1][2][1][RTW89_ETSI][48] = 127,
+ [0][1][2][1][RTW89_MKK][48] = 127,
+ [0][1][2][1][RTW89_IC][48] = 127,
+ [0][1][2][1][RTW89_KCC][48] = 127,
+ [0][1][2][1][RTW89_ACMA][48] = 127,
+ [0][1][2][1][RTW89_CN][48] = 127,
+ [0][1][2][1][RTW89_UK][48] = 127,
+ [0][1][2][1][RTW89_FCC][50] = 127,
+ [0][1][2][1][RTW89_ETSI][50] = 127,
+ [0][1][2][1][RTW89_MKK][50] = 127,
+ [0][1][2][1][RTW89_IC][50] = 127,
+ [0][1][2][1][RTW89_KCC][50] = 127,
+ [0][1][2][1][RTW89_ACMA][50] = 127,
+ [0][1][2][1][RTW89_CN][50] = 127,
+ [0][1][2][1][RTW89_UK][50] = 127,
+ [0][1][2][1][RTW89_FCC][52] = 127,
+ [0][1][2][1][RTW89_ETSI][52] = 127,
+ [0][1][2][1][RTW89_MKK][52] = 127,
+ [0][1][2][1][RTW89_IC][52] = 127,
+ [0][1][2][1][RTW89_KCC][52] = 127,
+ [0][1][2][1][RTW89_ACMA][52] = 127,
+ [0][1][2][1][RTW89_CN][52] = 127,
+ [0][1][2][1][RTW89_UK][52] = 127,
+ [1][0][2][0][RTW89_FCC][1] = 66,
+ [1][0][2][0][RTW89_ETSI][1] = 64,
+ [1][0][2][0][RTW89_MKK][1] = 64,
+ [1][0][2][0][RTW89_IC][1] = 64,
+ [1][0][2][0][RTW89_KCC][1] = 74,
+ [1][0][2][0][RTW89_ACMA][1] = 64,
+ [1][0][2][0][RTW89_CN][1] = 64,
+ [1][0][2][0][RTW89_UK][1] = 64,
+ [1][0][2][0][RTW89_FCC][5] = 80,
+ [1][0][2][0][RTW89_ETSI][5] = 64,
+ [1][0][2][0][RTW89_MKK][5] = 62,
+ [1][0][2][0][RTW89_IC][5] = 64,
+ [1][0][2][0][RTW89_KCC][5] = 66,
+ [1][0][2][0][RTW89_ACMA][5] = 64,
+ [1][0][2][0][RTW89_CN][5] = 64,
+ [1][0][2][0][RTW89_UK][5] = 64,
+ [1][0][2][0][RTW89_FCC][9] = 80,
+ [1][0][2][0][RTW89_ETSI][9] = 64,
+ [1][0][2][0][RTW89_MKK][9] = 64,
+ [1][0][2][0][RTW89_IC][9] = 64,
+ [1][0][2][0][RTW89_KCC][9] = 76,
+ [1][0][2][0][RTW89_ACMA][9] = 64,
+ [1][0][2][0][RTW89_CN][9] = 64,
+ [1][0][2][0][RTW89_UK][9] = 64,
+ [1][0][2][0][RTW89_FCC][13] = 64,
+ [1][0][2][0][RTW89_ETSI][13] = 64,
+ [1][0][2][0][RTW89_MKK][13] = 64,
+ [1][0][2][0][RTW89_IC][13] = 64,
+ [1][0][2][0][RTW89_KCC][13] = 72,
+ [1][0][2][0][RTW89_ACMA][13] = 64,
+ [1][0][2][0][RTW89_CN][13] = 64,
+ [1][0][2][0][RTW89_UK][13] = 64,
+ [1][0][2][0][RTW89_FCC][16] = 66,
+ [1][0][2][0][RTW89_ETSI][16] = 66,
+ [1][0][2][0][RTW89_MKK][16] = 76,
+ [1][0][2][0][RTW89_IC][16] = 66,
+ [1][0][2][0][RTW89_KCC][16] = 74,
+ [1][0][2][0][RTW89_ACMA][16] = 66,
+ [1][0][2][0][RTW89_CN][16] = 127,
+ [1][0][2][0][RTW89_UK][16] = 66,
+ [1][0][2][0][RTW89_FCC][20] = 80,
+ [1][0][2][0][RTW89_ETSI][20] = 66,
+ [1][0][2][0][RTW89_MKK][20] = 76,
+ [1][0][2][0][RTW89_IC][20] = 80,
+ [1][0][2][0][RTW89_KCC][20] = 74,
+ [1][0][2][0][RTW89_ACMA][20] = 66,
+ [1][0][2][0][RTW89_CN][20] = 127,
+ [1][0][2][0][RTW89_UK][20] = 66,
+ [1][0][2][0][RTW89_FCC][24] = 80,
+ [1][0][2][0][RTW89_ETSI][24] = 66,
+ [1][0][2][0][RTW89_MKK][24] = 76,
+ [1][0][2][0][RTW89_IC][24] = 127,
+ [1][0][2][0][RTW89_KCC][24] = 74,
+ [1][0][2][0][RTW89_ACMA][24] = 127,
+ [1][0][2][0][RTW89_CN][24] = 127,
+ [1][0][2][0][RTW89_UK][24] = 66,
+ [1][0][2][0][RTW89_FCC][28] = 80,
+ [1][0][2][0][RTW89_ETSI][28] = 66,
+ [1][0][2][0][RTW89_MKK][28] = 76,
+ [1][0][2][0][RTW89_IC][28] = 127,
+ [1][0][2][0][RTW89_KCC][28] = 74,
+ [1][0][2][0][RTW89_ACMA][28] = 127,
+ [1][0][2][0][RTW89_CN][28] = 127,
+ [1][0][2][0][RTW89_UK][28] = 66,
+ [1][0][2][0][RTW89_FCC][32] = 74,
+ [1][0][2][0][RTW89_ETSI][32] = 66,
+ [1][0][2][0][RTW89_MKK][32] = 76,
+ [1][0][2][0][RTW89_IC][32] = 74,
+ [1][0][2][0][RTW89_KCC][32] = 76,
+ [1][0][2][0][RTW89_ACMA][32] = 66,
+ [1][0][2][0][RTW89_CN][32] = 127,
+ [1][0][2][0][RTW89_UK][32] = 66,
+ [1][0][2][0][RTW89_FCC][36] = 78,
+ [1][0][2][0][RTW89_ETSI][36] = 127,
+ [1][0][2][0][RTW89_MKK][36] = 76,
+ [1][0][2][0][RTW89_IC][36] = 78,
+ [1][0][2][0][RTW89_KCC][36] = 76,
+ [1][0][2][0][RTW89_ACMA][36] = 76,
+ [1][0][2][0][RTW89_CN][36] = 127,
+ [1][0][2][0][RTW89_UK][36] = 76,
+ [1][0][2][0][RTW89_FCC][39] = 80,
+ [1][0][2][0][RTW89_ETSI][39] = 30,
+ [1][0][2][0][RTW89_MKK][39] = 127,
+ [1][0][2][0][RTW89_IC][39] = 80,
+ [1][0][2][0][RTW89_KCC][39] = 68,
+ [1][0][2][0][RTW89_ACMA][39] = 76,
+ [1][0][2][0][RTW89_CN][39] = 70,
+ [1][0][2][0][RTW89_UK][39] = 64,
+ [1][0][2][0][RTW89_FCC][43] = 80,
+ [1][0][2][0][RTW89_ETSI][43] = 30,
+ [1][0][2][0][RTW89_MKK][43] = 127,
+ [1][0][2][0][RTW89_IC][43] = 80,
+ [1][0][2][0][RTW89_KCC][43] = 76,
+ [1][0][2][0][RTW89_ACMA][43] = 76,
+ [1][0][2][0][RTW89_CN][43] = 76,
+ [1][0][2][0][RTW89_UK][43] = 64,
+ [1][0][2][0][RTW89_FCC][47] = 80,
+ [1][0][2][0][RTW89_ETSI][47] = 127,
+ [1][0][2][0][RTW89_MKK][47] = 127,
+ [1][0][2][0][RTW89_IC][47] = 127,
+ [1][0][2][0][RTW89_KCC][47] = 127,
+ [1][0][2][0][RTW89_ACMA][47] = 127,
+ [1][0][2][0][RTW89_CN][47] = 127,
+ [1][0][2][0][RTW89_UK][47] = 127,
+ [1][0][2][0][RTW89_FCC][51] = 80,
+ [1][0][2][0][RTW89_ETSI][51] = 127,
+ [1][0][2][0][RTW89_MKK][51] = 127,
+ [1][0][2][0][RTW89_IC][51] = 127,
+ [1][0][2][0][RTW89_KCC][51] = 127,
+ [1][0][2][0][RTW89_ACMA][51] = 127,
+ [1][0][2][0][RTW89_CN][51] = 127,
+ [1][0][2][0][RTW89_UK][51] = 127,
+ [1][1][2][0][RTW89_FCC][1] = 127,
+ [1][1][2][0][RTW89_ETSI][1] = 127,
+ [1][1][2][0][RTW89_MKK][1] = 127,
+ [1][1][2][0][RTW89_IC][1] = 127,
+ [1][1][2][0][RTW89_KCC][1] = 127,
+ [1][1][2][0][RTW89_ACMA][1] = 127,
+ [1][1][2][0][RTW89_CN][1] = 127,
+ [1][1][2][0][RTW89_UK][1] = 127,
+ [1][1][2][0][RTW89_FCC][5] = 127,
+ [1][1][2][0][RTW89_ETSI][5] = 127,
+ [1][1][2][0][RTW89_MKK][5] = 127,
+ [1][1][2][0][RTW89_IC][5] = 127,
+ [1][1][2][0][RTW89_KCC][5] = 127,
+ [1][1][2][0][RTW89_ACMA][5] = 127,
+ [1][1][2][0][RTW89_CN][5] = 127,
+ [1][1][2][0][RTW89_UK][5] = 127,
+ [1][1][2][0][RTW89_FCC][9] = 127,
+ [1][1][2][0][RTW89_ETSI][9] = 127,
+ [1][1][2][0][RTW89_MKK][9] = 127,
+ [1][1][2][0][RTW89_IC][9] = 127,
+ [1][1][2][0][RTW89_KCC][9] = 127,
+ [1][1][2][0][RTW89_ACMA][9] = 127,
+ [1][1][2][0][RTW89_CN][9] = 127,
+ [1][1][2][0][RTW89_UK][9] = 127,
+ [1][1][2][0][RTW89_FCC][13] = 127,
+ [1][1][2][0][RTW89_ETSI][13] = 127,
+ [1][1][2][0][RTW89_MKK][13] = 127,
+ [1][1][2][0][RTW89_IC][13] = 127,
+ [1][1][2][0][RTW89_KCC][13] = 127,
+ [1][1][2][0][RTW89_ACMA][13] = 127,
+ [1][1][2][0][RTW89_CN][13] = 127,
+ [1][1][2][0][RTW89_UK][13] = 127,
+ [1][1][2][0][RTW89_FCC][16] = 127,
+ [1][1][2][0][RTW89_ETSI][16] = 127,
+ [1][1][2][0][RTW89_MKK][16] = 127,
+ [1][1][2][0][RTW89_IC][16] = 127,
+ [1][1][2][0][RTW89_KCC][16] = 127,
+ [1][1][2][0][RTW89_ACMA][16] = 127,
+ [1][1][2][0][RTW89_CN][16] = 127,
+ [1][1][2][0][RTW89_UK][16] = 127,
+ [1][1][2][0][RTW89_FCC][20] = 127,
+ [1][1][2][0][RTW89_ETSI][20] = 127,
+ [1][1][2][0][RTW89_MKK][20] = 127,
+ [1][1][2][0][RTW89_IC][20] = 127,
+ [1][1][2][0][RTW89_KCC][20] = 127,
+ [1][1][2][0][RTW89_ACMA][20] = 127,
+ [1][1][2][0][RTW89_CN][20] = 127,
+ [1][1][2][0][RTW89_UK][20] = 127,
+ [1][1][2][0][RTW89_FCC][24] = 127,
+ [1][1][2][0][RTW89_ETSI][24] = 127,
+ [1][1][2][0][RTW89_MKK][24] = 127,
+ [1][1][2][0][RTW89_IC][24] = 127,
+ [1][1][2][0][RTW89_KCC][24] = 127,
+ [1][1][2][0][RTW89_ACMA][24] = 127,
+ [1][1][2][0][RTW89_CN][24] = 127,
+ [1][1][2][0][RTW89_UK][24] = 127,
+ [1][1][2][0][RTW89_FCC][28] = 127,
+ [1][1][2][0][RTW89_ETSI][28] = 127,
+ [1][1][2][0][RTW89_MKK][28] = 127,
+ [1][1][2][0][RTW89_IC][28] = 127,
+ [1][1][2][0][RTW89_KCC][28] = 127,
+ [1][1][2][0][RTW89_ACMA][28] = 127,
+ [1][1][2][0][RTW89_CN][28] = 127,
+ [1][1][2][0][RTW89_UK][28] = 127,
+ [1][1][2][0][RTW89_FCC][32] = 127,
+ [1][1][2][0][RTW89_ETSI][32] = 127,
+ [1][1][2][0][RTW89_MKK][32] = 127,
+ [1][1][2][0][RTW89_IC][32] = 127,
+ [1][1][2][0][RTW89_KCC][32] = 127,
+ [1][1][2][0][RTW89_ACMA][32] = 127,
+ [1][1][2][0][RTW89_CN][32] = 127,
+ [1][1][2][0][RTW89_UK][32] = 127,
+ [1][1][2][0][RTW89_FCC][36] = 127,
+ [1][1][2][0][RTW89_ETSI][36] = 127,
+ [1][1][2][0][RTW89_MKK][36] = 127,
+ [1][1][2][0][RTW89_IC][36] = 127,
+ [1][1][2][0][RTW89_KCC][36] = 127,
+ [1][1][2][0][RTW89_ACMA][36] = 127,
+ [1][1][2][0][RTW89_CN][36] = 127,
+ [1][1][2][0][RTW89_UK][36] = 127,
+ [1][1][2][0][RTW89_FCC][39] = 127,
+ [1][1][2][0][RTW89_ETSI][39] = 127,
+ [1][1][2][0][RTW89_MKK][39] = 127,
+ [1][1][2][0][RTW89_IC][39] = 127,
+ [1][1][2][0][RTW89_KCC][39] = 127,
+ [1][1][2][0][RTW89_ACMA][39] = 127,
+ [1][1][2][0][RTW89_CN][39] = 127,
+ [1][1][2][0][RTW89_UK][39] = 127,
+ [1][1][2][0][RTW89_FCC][43] = 127,
+ [1][1][2][0][RTW89_ETSI][43] = 127,
+ [1][1][2][0][RTW89_MKK][43] = 127,
+ [1][1][2][0][RTW89_IC][43] = 127,
+ [1][1][2][0][RTW89_KCC][43] = 127,
+ [1][1][2][0][RTW89_ACMA][43] = 127,
+ [1][1][2][0][RTW89_CN][43] = 127,
+ [1][1][2][0][RTW89_UK][43] = 127,
+ [1][1][2][0][RTW89_FCC][47] = 127,
+ [1][1][2][0][RTW89_ETSI][47] = 127,
+ [1][1][2][0][RTW89_MKK][47] = 127,
+ [1][1][2][0][RTW89_IC][47] = 127,
+ [1][1][2][0][RTW89_KCC][47] = 127,
+ [1][1][2][0][RTW89_ACMA][47] = 127,
+ [1][1][2][0][RTW89_CN][47] = 127,
+ [1][1][2][0][RTW89_UK][47] = 127,
+ [1][1][2][0][RTW89_FCC][51] = 127,
+ [1][1][2][0][RTW89_ETSI][51] = 127,
+ [1][1][2][0][RTW89_MKK][51] = 127,
+ [1][1][2][0][RTW89_IC][51] = 127,
+ [1][1][2][0][RTW89_KCC][51] = 127,
+ [1][1][2][0][RTW89_ACMA][51] = 127,
+ [1][1][2][0][RTW89_CN][51] = 127,
+ [1][1][2][0][RTW89_UK][51] = 127,
+ [1][1][2][1][RTW89_FCC][1] = 127,
+ [1][1][2][1][RTW89_ETSI][1] = 127,
+ [1][1][2][1][RTW89_MKK][1] = 127,
+ [1][1][2][1][RTW89_IC][1] = 127,
+ [1][1][2][1][RTW89_KCC][1] = 127,
+ [1][1][2][1][RTW89_ACMA][1] = 127,
+ [1][1][2][1][RTW89_CN][1] = 127,
+ [1][1][2][1][RTW89_UK][1] = 127,
+ [1][1][2][1][RTW89_FCC][5] = 127,
+ [1][1][2][1][RTW89_ETSI][5] = 127,
+ [1][1][2][1][RTW89_MKK][5] = 127,
+ [1][1][2][1][RTW89_IC][5] = 127,
+ [1][1][2][1][RTW89_KCC][5] = 127,
+ [1][1][2][1][RTW89_ACMA][5] = 127,
+ [1][1][2][1][RTW89_CN][5] = 127,
+ [1][1][2][1][RTW89_UK][5] = 127,
+ [1][1][2][1][RTW89_FCC][9] = 127,
+ [1][1][2][1][RTW89_ETSI][9] = 127,
+ [1][1][2][1][RTW89_MKK][9] = 127,
+ [1][1][2][1][RTW89_IC][9] = 127,
+ [1][1][2][1][RTW89_KCC][9] = 127,
+ [1][1][2][1][RTW89_ACMA][9] = 127,
+ [1][1][2][1][RTW89_CN][9] = 127,
+ [1][1][2][1][RTW89_UK][9] = 127,
+ [1][1][2][1][RTW89_FCC][13] = 127,
+ [1][1][2][1][RTW89_ETSI][13] = 127,
+ [1][1][2][1][RTW89_MKK][13] = 127,
+ [1][1][2][1][RTW89_IC][13] = 127,
+ [1][1][2][1][RTW89_KCC][13] = 127,
+ [1][1][2][1][RTW89_ACMA][13] = 127,
+ [1][1][2][1][RTW89_CN][13] = 127,
+ [1][1][2][1][RTW89_UK][13] = 127,
+ [1][1][2][1][RTW89_FCC][16] = 127,
+ [1][1][2][1][RTW89_ETSI][16] = 127,
+ [1][1][2][1][RTW89_MKK][16] = 127,
+ [1][1][2][1][RTW89_IC][16] = 127,
+ [1][1][2][1][RTW89_KCC][16] = 127,
+ [1][1][2][1][RTW89_ACMA][16] = 127,
+ [1][1][2][1][RTW89_CN][16] = 127,
+ [1][1][2][1][RTW89_UK][16] = 127,
+ [1][1][2][1][RTW89_FCC][20] = 127,
+ [1][1][2][1][RTW89_ETSI][20] = 127,
+ [1][1][2][1][RTW89_MKK][20] = 127,
+ [1][1][2][1][RTW89_IC][20] = 127,
+ [1][1][2][1][RTW89_KCC][20] = 127,
+ [1][1][2][1][RTW89_ACMA][20] = 127,
+ [1][1][2][1][RTW89_CN][20] = 127,
+ [1][1][2][1][RTW89_UK][20] = 127,
+ [1][1][2][1][RTW89_FCC][24] = 127,
+ [1][1][2][1][RTW89_ETSI][24] = 127,
+ [1][1][2][1][RTW89_MKK][24] = 127,
+ [1][1][2][1][RTW89_IC][24] = 127,
+ [1][1][2][1][RTW89_KCC][24] = 127,
+ [1][1][2][1][RTW89_ACMA][24] = 127,
+ [1][1][2][1][RTW89_CN][24] = 127,
+ [1][1][2][1][RTW89_UK][24] = 127,
+ [1][1][2][1][RTW89_FCC][28] = 127,
+ [1][1][2][1][RTW89_ETSI][28] = 127,
+ [1][1][2][1][RTW89_MKK][28] = 127,
+ [1][1][2][1][RTW89_IC][28] = 127,
+ [1][1][2][1][RTW89_KCC][28] = 127,
+ [1][1][2][1][RTW89_ACMA][28] = 127,
+ [1][1][2][1][RTW89_CN][28] = 127,
+ [1][1][2][1][RTW89_UK][28] = 127,
+ [1][1][2][1][RTW89_FCC][32] = 127,
+ [1][1][2][1][RTW89_ETSI][32] = 127,
+ [1][1][2][1][RTW89_MKK][32] = 127,
+ [1][1][2][1][RTW89_IC][32] = 127,
+ [1][1][2][1][RTW89_KCC][32] = 127,
+ [1][1][2][1][RTW89_ACMA][32] = 127,
+ [1][1][2][1][RTW89_CN][32] = 127,
+ [1][1][2][1][RTW89_UK][32] = 127,
+ [1][1][2][1][RTW89_FCC][36] = 127,
+ [1][1][2][1][RTW89_ETSI][36] = 127,
+ [1][1][2][1][RTW89_MKK][36] = 127,
+ [1][1][2][1][RTW89_IC][36] = 127,
+ [1][1][2][1][RTW89_KCC][36] = 127,
+ [1][1][2][1][RTW89_ACMA][36] = 127,
+ [1][1][2][1][RTW89_CN][36] = 127,
+ [1][1][2][1][RTW89_UK][36] = 127,
+ [1][1][2][1][RTW89_FCC][39] = 127,
+ [1][1][2][1][RTW89_ETSI][39] = 127,
+ [1][1][2][1][RTW89_MKK][39] = 127,
+ [1][1][2][1][RTW89_IC][39] = 127,
+ [1][1][2][1][RTW89_KCC][39] = 127,
+ [1][1][2][1][RTW89_ACMA][39] = 127,
+ [1][1][2][1][RTW89_CN][39] = 127,
+ [1][1][2][1][RTW89_UK][39] = 127,
+ [1][1][2][1][RTW89_FCC][43] = 127,
+ [1][1][2][1][RTW89_ETSI][43] = 127,
+ [1][1][2][1][RTW89_MKK][43] = 127,
+ [1][1][2][1][RTW89_IC][43] = 127,
+ [1][1][2][1][RTW89_KCC][43] = 127,
+ [1][1][2][1][RTW89_ACMA][43] = 127,
+ [1][1][2][1][RTW89_CN][43] = 127,
+ [1][1][2][1][RTW89_UK][43] = 127,
+ [1][1][2][1][RTW89_FCC][47] = 127,
+ [1][1][2][1][RTW89_ETSI][47] = 127,
+ [1][1][2][1][RTW89_MKK][47] = 127,
+ [1][1][2][1][RTW89_IC][47] = 127,
+ [1][1][2][1][RTW89_KCC][47] = 127,
+ [1][1][2][1][RTW89_ACMA][47] = 127,
+ [1][1][2][1][RTW89_CN][47] = 127,
+ [1][1][2][1][RTW89_UK][47] = 127,
+ [1][1][2][1][RTW89_FCC][51] = 127,
+ [1][1][2][1][RTW89_ETSI][51] = 127,
+ [1][1][2][1][RTW89_MKK][51] = 127,
+ [1][1][2][1][RTW89_IC][51] = 127,
+ [1][1][2][1][RTW89_KCC][51] = 127,
+ [1][1][2][1][RTW89_ACMA][51] = 127,
+ [1][1][2][1][RTW89_CN][51] = 127,
+ [1][1][2][1][RTW89_UK][51] = 127,
+ [2][0][2][0][RTW89_FCC][3] = 72,
+ [2][0][2][0][RTW89_ETSI][3] = 64,
+ [2][0][2][0][RTW89_MKK][3] = 62,
+ [2][0][2][0][RTW89_IC][3] = 64,
+ [2][0][2][0][RTW89_KCC][3] = 68,
+ [2][0][2][0][RTW89_ACMA][3] = 64,
+ [2][0][2][0][RTW89_CN][3] = 64,
+ [2][0][2][0][RTW89_UK][3] = 64,
+ [2][0][2][0][RTW89_FCC][11] = 62,
+ [2][0][2][0][RTW89_ETSI][11] = 64,
+ [2][0][2][0][RTW89_MKK][11] = 64,
+ [2][0][2][0][RTW89_IC][11] = 62,
+ [2][0][2][0][RTW89_KCC][11] = 68,
+ [2][0][2][0][RTW89_ACMA][11] = 64,
+ [2][0][2][0][RTW89_CN][11] = 64,
+ [2][0][2][0][RTW89_UK][11] = 64,
+ [2][0][2][0][RTW89_FCC][18] = 66,
+ [2][0][2][0][RTW89_ETSI][18] = 64,
+ [2][0][2][0][RTW89_MKK][18] = 68,
+ [2][0][2][0][RTW89_IC][18] = 66,
+ [2][0][2][0][RTW89_KCC][18] = 68,
+ [2][0][2][0][RTW89_ACMA][18] = 64,
+ [2][0][2][0][RTW89_CN][18] = 127,
+ [2][0][2][0][RTW89_UK][18] = 64,
+ [2][0][2][0][RTW89_FCC][26] = 72,
+ [2][0][2][0][RTW89_ETSI][26] = 64,
+ [2][0][2][0][RTW89_MKK][26] = 68,
+ [2][0][2][0][RTW89_IC][26] = 127,
+ [2][0][2][0][RTW89_KCC][26] = 68,
+ [2][0][2][0][RTW89_ACMA][26] = 127,
+ [2][0][2][0][RTW89_CN][26] = 127,
+ [2][0][2][0][RTW89_UK][26] = 64,
+ [2][0][2][0][RTW89_FCC][34] = 72,
+ [2][0][2][0][RTW89_ETSI][34] = 127,
+ [2][0][2][0][RTW89_MKK][34] = 68,
+ [2][0][2][0][RTW89_IC][34] = 72,
+ [2][0][2][0][RTW89_KCC][34] = 68,
+ [2][0][2][0][RTW89_ACMA][34] = 68,
+ [2][0][2][0][RTW89_CN][34] = 127,
+ [2][0][2][0][RTW89_UK][34] = 68,
+ [2][0][2][0][RTW89_FCC][41] = 72,
+ [2][0][2][0][RTW89_ETSI][41] = 30,
+ [2][0][2][0][RTW89_MKK][41] = 127,
+ [2][0][2][0][RTW89_IC][41] = 72,
+ [2][0][2][0][RTW89_KCC][41] = 64,
+ [2][0][2][0][RTW89_ACMA][41] = 68,
+ [2][0][2][0][RTW89_CN][41] = 68,
+ [2][0][2][0][RTW89_UK][41] = 64,
+ [2][0][2][0][RTW89_FCC][49] = 72,
+ [2][0][2][0][RTW89_ETSI][49] = 127,
+ [2][0][2][0][RTW89_MKK][49] = 127,
+ [2][0][2][0][RTW89_IC][49] = 127,
+ [2][0][2][0][RTW89_KCC][49] = 127,
+ [2][0][2][0][RTW89_ACMA][49] = 127,
+ [2][0][2][0][RTW89_CN][49] = 127,
+ [2][0][2][0][RTW89_UK][49] = 127,
+ [2][1][2][0][RTW89_FCC][3] = 127,
+ [2][1][2][0][RTW89_ETSI][3] = 127,
+ [2][1][2][0][RTW89_MKK][3] = 127,
+ [2][1][2][0][RTW89_IC][3] = 127,
+ [2][1][2][0][RTW89_KCC][3] = 127,
+ [2][1][2][0][RTW89_ACMA][3] = 127,
+ [2][1][2][0][RTW89_CN][3] = 127,
+ [2][1][2][0][RTW89_UK][3] = 127,
+ [2][1][2][0][RTW89_FCC][11] = 127,
+ [2][1][2][0][RTW89_ETSI][11] = 127,
+ [2][1][2][0][RTW89_MKK][11] = 127,
+ [2][1][2][0][RTW89_IC][11] = 127,
+ [2][1][2][0][RTW89_KCC][11] = 127,
+ [2][1][2][0][RTW89_ACMA][11] = 127,
+ [2][1][2][0][RTW89_CN][11] = 127,
+ [2][1][2][0][RTW89_UK][11] = 127,
+ [2][1][2][0][RTW89_FCC][18] = 127,
+ [2][1][2][0][RTW89_ETSI][18] = 127,
+ [2][1][2][0][RTW89_MKK][18] = 127,
+ [2][1][2][0][RTW89_IC][18] = 127,
+ [2][1][2][0][RTW89_KCC][18] = 127,
+ [2][1][2][0][RTW89_ACMA][18] = 127,
+ [2][1][2][0][RTW89_CN][18] = 127,
+ [2][1][2][0][RTW89_UK][18] = 127,
+ [2][1][2][0][RTW89_FCC][26] = 127,
+ [2][1][2][0][RTW89_ETSI][26] = 127,
+ [2][1][2][0][RTW89_MKK][26] = 127,
+ [2][1][2][0][RTW89_IC][26] = 127,
+ [2][1][2][0][RTW89_KCC][26] = 127,
+ [2][1][2][0][RTW89_ACMA][26] = 127,
+ [2][1][2][0][RTW89_CN][26] = 127,
+ [2][1][2][0][RTW89_UK][26] = 127,
+ [2][1][2][0][RTW89_FCC][34] = 127,
+ [2][1][2][0][RTW89_ETSI][34] = 127,
+ [2][1][2][0][RTW89_MKK][34] = 127,
+ [2][1][2][0][RTW89_IC][34] = 127,
+ [2][1][2][0][RTW89_KCC][34] = 127,
+ [2][1][2][0][RTW89_ACMA][34] = 127,
+ [2][1][2][0][RTW89_CN][34] = 127,
+ [2][1][2][0][RTW89_UK][34] = 127,
+ [2][1][2][0][RTW89_FCC][41] = 127,
+ [2][1][2][0][RTW89_ETSI][41] = 127,
+ [2][1][2][0][RTW89_MKK][41] = 127,
+ [2][1][2][0][RTW89_IC][41] = 127,
+ [2][1][2][0][RTW89_KCC][41] = 127,
+ [2][1][2][0][RTW89_ACMA][41] = 127,
+ [2][1][2][0][RTW89_CN][41] = 127,
+ [2][1][2][0][RTW89_UK][41] = 127,
+ [2][1][2][0][RTW89_FCC][49] = 127,
+ [2][1][2][0][RTW89_ETSI][49] = 127,
+ [2][1][2][0][RTW89_MKK][49] = 127,
+ [2][1][2][0][RTW89_IC][49] = 127,
+ [2][1][2][0][RTW89_KCC][49] = 127,
+ [2][1][2][0][RTW89_ACMA][49] = 127,
+ [2][1][2][0][RTW89_CN][49] = 127,
+ [2][1][2][0][RTW89_UK][49] = 127,
+ [2][1][2][1][RTW89_FCC][3] = 127,
+ [2][1][2][1][RTW89_ETSI][3] = 127,
+ [2][1][2][1][RTW89_MKK][3] = 127,
+ [2][1][2][1][RTW89_IC][3] = 127,
+ [2][1][2][1][RTW89_KCC][3] = 127,
+ [2][1][2][1][RTW89_ACMA][3] = 127,
+ [2][1][2][1][RTW89_CN][3] = 127,
+ [2][1][2][1][RTW89_UK][3] = 127,
+ [2][1][2][1][RTW89_FCC][11] = 127,
+ [2][1][2][1][RTW89_ETSI][11] = 127,
+ [2][1][2][1][RTW89_MKK][11] = 127,
+ [2][1][2][1][RTW89_IC][11] = 127,
+ [2][1][2][1][RTW89_KCC][11] = 127,
+ [2][1][2][1][RTW89_ACMA][11] = 127,
+ [2][1][2][1][RTW89_CN][11] = 127,
+ [2][1][2][1][RTW89_UK][11] = 127,
+ [2][1][2][1][RTW89_FCC][18] = 127,
+ [2][1][2][1][RTW89_ETSI][18] = 127,
+ [2][1][2][1][RTW89_MKK][18] = 127,
+ [2][1][2][1][RTW89_IC][18] = 127,
+ [2][1][2][1][RTW89_KCC][18] = 127,
+ [2][1][2][1][RTW89_ACMA][18] = 127,
+ [2][1][2][1][RTW89_CN][18] = 127,
+ [2][1][2][1][RTW89_UK][18] = 127,
+ [2][1][2][1][RTW89_FCC][26] = 127,
+ [2][1][2][1][RTW89_ETSI][26] = 127,
+ [2][1][2][1][RTW89_MKK][26] = 127,
+ [2][1][2][1][RTW89_IC][26] = 127,
+ [2][1][2][1][RTW89_KCC][26] = 127,
+ [2][1][2][1][RTW89_ACMA][26] = 127,
+ [2][1][2][1][RTW89_CN][26] = 127,
+ [2][1][2][1][RTW89_UK][26] = 127,
+ [2][1][2][1][RTW89_FCC][34] = 127,
+ [2][1][2][1][RTW89_ETSI][34] = 127,
+ [2][1][2][1][RTW89_MKK][34] = 127,
+ [2][1][2][1][RTW89_IC][34] = 127,
+ [2][1][2][1][RTW89_KCC][34] = 127,
+ [2][1][2][1][RTW89_ACMA][34] = 127,
+ [2][1][2][1][RTW89_CN][34] = 127,
+ [2][1][2][1][RTW89_UK][34] = 127,
+ [2][1][2][1][RTW89_FCC][41] = 127,
+ [2][1][2][1][RTW89_ETSI][41] = 127,
+ [2][1][2][1][RTW89_MKK][41] = 127,
+ [2][1][2][1][RTW89_IC][41] = 127,
+ [2][1][2][1][RTW89_KCC][41] = 127,
+ [2][1][2][1][RTW89_ACMA][41] = 127,
+ [2][1][2][1][RTW89_CN][41] = 127,
+ [2][1][2][1][RTW89_UK][41] = 127,
+ [2][1][2][1][RTW89_FCC][49] = 127,
+ [2][1][2][1][RTW89_ETSI][49] = 127,
+ [2][1][2][1][RTW89_MKK][49] = 127,
+ [2][1][2][1][RTW89_IC][49] = 127,
+ [2][1][2][1][RTW89_KCC][49] = 127,
+ [2][1][2][1][RTW89_ACMA][49] = 127,
+ [2][1][2][1][RTW89_CN][49] = 127,
+ [2][1][2][1][RTW89_UK][49] = 127,
+ [3][0][2][0][RTW89_FCC][7] = 127,
+ [3][0][2][0][RTW89_ETSI][7] = 127,
+ [3][0][2][0][RTW89_MKK][7] = 127,
+ [3][0][2][0][RTW89_IC][7] = 127,
+ [3][0][2][0][RTW89_KCC][7] = 127,
+ [3][0][2][0][RTW89_ACMA][7] = 127,
+ [3][0][2][0][RTW89_CN][7] = 58,
+ [3][0][2][0][RTW89_UK][7] = 127,
+ [3][0][2][0][RTW89_FCC][22] = 127,
+ [3][0][2][0][RTW89_ETSI][22] = 127,
+ [3][0][2][0][RTW89_MKK][22] = 127,
+ [3][0][2][0][RTW89_IC][22] = 127,
+ [3][0][2][0][RTW89_KCC][22] = 127,
+ [3][0][2][0][RTW89_ACMA][22] = 127,
+ [3][0][2][0][RTW89_CN][22] = 58,
+ [3][0][2][0][RTW89_UK][22] = 127,
+ [3][0][2][0][RTW89_FCC][45] = 127,
+ [3][0][2][0][RTW89_ETSI][45] = 127,
+ [3][0][2][0][RTW89_MKK][45] = 127,
+ [3][0][2][0][RTW89_IC][45] = 127,
+ [3][0][2][0][RTW89_KCC][45] = 127,
+ [3][0][2][0][RTW89_ACMA][45] = 127,
+ [3][0][2][0][RTW89_CN][45] = 127,
+ [3][0][2][0][RTW89_UK][45] = 127,
+ [3][1][2][0][RTW89_FCC][7] = 127,
+ [3][1][2][0][RTW89_ETSI][7] = 127,
+ [3][1][2][0][RTW89_MKK][7] = 127,
+ [3][1][2][0][RTW89_IC][7] = 127,
+ [3][1][2][0][RTW89_KCC][7] = 127,
+ [3][1][2][0][RTW89_ACMA][7] = 127,
+ [3][1][2][0][RTW89_CN][7] = 127,
+ [3][1][2][0][RTW89_UK][7] = 127,
+ [3][1][2][0][RTW89_FCC][22] = 127,
+ [3][1][2][0][RTW89_ETSI][22] = 127,
+ [3][1][2][0][RTW89_MKK][22] = 127,
+ [3][1][2][0][RTW89_IC][22] = 127,
+ [3][1][2][0][RTW89_KCC][22] = 127,
+ [3][1][2][0][RTW89_ACMA][22] = 127,
+ [3][1][2][0][RTW89_CN][22] = 127,
+ [3][1][2][0][RTW89_UK][22] = 127,
+ [3][1][2][0][RTW89_FCC][45] = 127,
+ [3][1][2][0][RTW89_ETSI][45] = 127,
+ [3][1][2][0][RTW89_MKK][45] = 127,
+ [3][1][2][0][RTW89_IC][45] = 127,
+ [3][1][2][0][RTW89_KCC][45] = 127,
+ [3][1][2][0][RTW89_ACMA][45] = 127,
+ [3][1][2][0][RTW89_CN][45] = 127,
+ [3][1][2][0][RTW89_UK][45] = 127,
+ [3][1][2][1][RTW89_FCC][7] = 127,
+ [3][1][2][1][RTW89_ETSI][7] = 127,
+ [3][1][2][1][RTW89_MKK][7] = 127,
+ [3][1][2][1][RTW89_IC][7] = 127,
+ [3][1][2][1][RTW89_KCC][7] = 127,
+ [3][1][2][1][RTW89_ACMA][7] = 127,
+ [3][1][2][1][RTW89_CN][7] = 127,
+ [3][1][2][1][RTW89_UK][7] = 127,
+ [3][1][2][1][RTW89_FCC][22] = 127,
+ [3][1][2][1][RTW89_ETSI][22] = 127,
+ [3][1][2][1][RTW89_MKK][22] = 127,
+ [3][1][2][1][RTW89_IC][22] = 127,
+ [3][1][2][1][RTW89_KCC][22] = 127,
+ [3][1][2][1][RTW89_ACMA][22] = 127,
+ [3][1][2][1][RTW89_CN][22] = 127,
+ [3][1][2][1][RTW89_UK][22] = 127,
+ [3][1][2][1][RTW89_FCC][45] = 127,
+ [3][1][2][1][RTW89_ETSI][45] = 127,
+ [3][1][2][1][RTW89_MKK][45] = 127,
+ [3][1][2][1][RTW89_IC][45] = 127,
+ [3][1][2][1][RTW89_KCC][45] = 127,
+ [3][1][2][1][RTW89_ACMA][45] = 127,
+ [3][1][2][1][RTW89_CN][45] = 127,
+ [3][1][2][1][RTW89_UK][45] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_ru_2g_type2[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
+ [0][0][RTW89_WW][0] = 30,
+ [0][0][RTW89_WW][1] = 30,
+ [0][0][RTW89_WW][2] = 30,
+ [0][0][RTW89_WW][3] = 30,
+ [0][0][RTW89_WW][4] = 30,
+ [0][0][RTW89_WW][5] = 30,
+ [0][0][RTW89_WW][6] = 30,
+ [0][0][RTW89_WW][7] = 30,
+ [0][0][RTW89_WW][8] = 30,
+ [0][0][RTW89_WW][9] = 30,
+ [0][0][RTW89_WW][10] = 30,
+ [0][0][RTW89_WW][11] = 30,
+ [0][0][RTW89_WW][12] = 30,
+ [0][0][RTW89_WW][13] = 0,
+ [0][1][RTW89_WW][0] = 20,
+ [0][1][RTW89_WW][1] = 22,
+ [0][1][RTW89_WW][2] = 22,
+ [0][1][RTW89_WW][3] = 22,
+ [0][1][RTW89_WW][4] = 22,
+ [0][1][RTW89_WW][5] = 22,
+ [0][1][RTW89_WW][6] = 22,
+ [0][1][RTW89_WW][7] = 22,
+ [0][1][RTW89_WW][8] = 22,
+ [0][1][RTW89_WW][9] = 22,
+ [0][1][RTW89_WW][10] = 22,
+ [0][1][RTW89_WW][11] = 22,
+ [0][1][RTW89_WW][12] = 20,
+ [0][1][RTW89_WW][13] = 0,
+ [1][0][RTW89_WW][0] = 42,
+ [1][0][RTW89_WW][1] = 42,
+ [1][0][RTW89_WW][2] = 42,
+ [1][0][RTW89_WW][3] = 42,
+ [1][0][RTW89_WW][4] = 42,
+ [1][0][RTW89_WW][5] = 42,
+ [1][0][RTW89_WW][6] = 42,
+ [1][0][RTW89_WW][7] = 42,
+ [1][0][RTW89_WW][8] = 42,
+ [1][0][RTW89_WW][9] = 42,
+ [1][0][RTW89_WW][10] = 42,
+ [1][0][RTW89_WW][11] = 42,
+ [1][0][RTW89_WW][12] = 34,
+ [1][0][RTW89_WW][13] = 0,
+ [1][1][RTW89_WW][0] = 32,
+ [1][1][RTW89_WW][1] = 32,
+ [1][1][RTW89_WW][2] = 32,
+ [1][1][RTW89_WW][3] = 32,
+ [1][1][RTW89_WW][4] = 32,
+ [1][1][RTW89_WW][5] = 32,
+ [1][1][RTW89_WW][6] = 32,
+ [1][1][RTW89_WW][7] = 32,
+ [1][1][RTW89_WW][8] = 32,
+ [1][1][RTW89_WW][9] = 32,
+ [1][1][RTW89_WW][10] = 32,
+ [1][1][RTW89_WW][11] = 32,
+ [1][1][RTW89_WW][12] = 32,
+ [1][1][RTW89_WW][13] = 0,
+ [2][0][RTW89_WW][0] = 54,
+ [2][0][RTW89_WW][1] = 54,
+ [2][0][RTW89_WW][2] = 54,
+ [2][0][RTW89_WW][3] = 54,
+ [2][0][RTW89_WW][4] = 54,
+ [2][0][RTW89_WW][5] = 54,
+ [2][0][RTW89_WW][6] = 54,
+ [2][0][RTW89_WW][7] = 54,
+ [2][0][RTW89_WW][8] = 54,
+ [2][0][RTW89_WW][9] = 54,
+ [2][0][RTW89_WW][10] = 54,
+ [2][0][RTW89_WW][11] = 54,
+ [2][0][RTW89_WW][12] = 34,
+ [2][0][RTW89_WW][13] = 0,
+ [2][1][RTW89_WW][0] = 44,
+ [2][1][RTW89_WW][1] = 44,
+ [2][1][RTW89_WW][2] = 44,
+ [2][1][RTW89_WW][3] = 44,
+ [2][1][RTW89_WW][4] = 44,
+ [2][1][RTW89_WW][5] = 44,
+ [2][1][RTW89_WW][6] = 44,
+ [2][1][RTW89_WW][7] = 44,
+ [2][1][RTW89_WW][8] = 44,
+ [2][1][RTW89_WW][9] = 44,
+ [2][1][RTW89_WW][10] = 44,
+ [2][1][RTW89_WW][11] = 44,
+ [2][1][RTW89_WW][12] = 42,
+ [2][1][RTW89_WW][13] = 0,
+ [0][0][RTW89_FCC][0] = 60,
+ [0][0][RTW89_ETSI][0] = 30,
+ [0][0][RTW89_MKK][0] = 40,
+ [0][0][RTW89_IC][0] = 60,
+ [0][0][RTW89_KCC][0] = 46,
+ [0][0][RTW89_ACMA][0] = 30,
+ [0][0][RTW89_CN][0] = 32,
+ [0][0][RTW89_UK][0] = 30,
+ [0][0][RTW89_FCC][1] = 60,
+ [0][0][RTW89_ETSI][1] = 30,
+ [0][0][RTW89_MKK][1] = 44,
+ [0][0][RTW89_IC][1] = 60,
+ [0][0][RTW89_KCC][1] = 46,
+ [0][0][RTW89_ACMA][1] = 30,
+ [0][0][RTW89_CN][1] = 32,
+ [0][0][RTW89_UK][1] = 30,
+ [0][0][RTW89_FCC][2] = 64,
+ [0][0][RTW89_ETSI][2] = 30,
+ [0][0][RTW89_MKK][2] = 44,
+ [0][0][RTW89_IC][2] = 64,
+ [0][0][RTW89_KCC][2] = 46,
+ [0][0][RTW89_ACMA][2] = 30,
+ [0][0][RTW89_CN][2] = 32,
+ [0][0][RTW89_UK][2] = 30,
+ [0][0][RTW89_FCC][3] = 68,
+ [0][0][RTW89_ETSI][3] = 30,
+ [0][0][RTW89_MKK][3] = 44,
+ [0][0][RTW89_IC][3] = 68,
+ [0][0][RTW89_KCC][3] = 46,
+ [0][0][RTW89_ACMA][3] = 30,
+ [0][0][RTW89_CN][3] = 32,
+ [0][0][RTW89_UK][3] = 30,
+ [0][0][RTW89_FCC][4] = 68,
+ [0][0][RTW89_ETSI][4] = 30,
+ [0][0][RTW89_MKK][4] = 44,
+ [0][0][RTW89_IC][4] = 68,
+ [0][0][RTW89_KCC][4] = 48,
+ [0][0][RTW89_ACMA][4] = 30,
+ [0][0][RTW89_CN][4] = 32,
+ [0][0][RTW89_UK][4] = 30,
+ [0][0][RTW89_FCC][5] = 82,
+ [0][0][RTW89_ETSI][5] = 30,
+ [0][0][RTW89_MKK][5] = 44,
+ [0][0][RTW89_IC][5] = 82,
+ [0][0][RTW89_KCC][5] = 48,
+ [0][0][RTW89_ACMA][5] = 30,
+ [0][0][RTW89_CN][5] = 32,
+ [0][0][RTW89_UK][5] = 30,
+ [0][0][RTW89_FCC][6] = 64,
+ [0][0][RTW89_ETSI][6] = 30,
+ [0][0][RTW89_MKK][6] = 44,
+ [0][0][RTW89_IC][6] = 64,
+ [0][0][RTW89_KCC][6] = 48,
+ [0][0][RTW89_ACMA][6] = 30,
+ [0][0][RTW89_CN][6] = 32,
+ [0][0][RTW89_UK][6] = 30,
+ [0][0][RTW89_FCC][7] = 64,
+ [0][0][RTW89_ETSI][7] = 30,
+ [0][0][RTW89_MKK][7] = 44,
+ [0][0][RTW89_IC][7] = 64,
+ [0][0][RTW89_KCC][7] = 48,
+ [0][0][RTW89_ACMA][7] = 30,
+ [0][0][RTW89_CN][7] = 32,
+ [0][0][RTW89_UK][7] = 30,
+ [0][0][RTW89_FCC][8] = 60,
+ [0][0][RTW89_ETSI][8] = 30,
+ [0][0][RTW89_MKK][8] = 44,
+ [0][0][RTW89_IC][8] = 60,
+ [0][0][RTW89_KCC][8] = 48,
+ [0][0][RTW89_ACMA][8] = 30,
+ [0][0][RTW89_CN][8] = 32,
+ [0][0][RTW89_UK][8] = 30,
+ [0][0][RTW89_FCC][9] = 56,
+ [0][0][RTW89_ETSI][9] = 30,
+ [0][0][RTW89_MKK][9] = 44,
+ [0][0][RTW89_IC][9] = 56,
+ [0][0][RTW89_KCC][9] = 44,
+ [0][0][RTW89_ACMA][9] = 30,
+ [0][0][RTW89_CN][9] = 32,
+ [0][0][RTW89_UK][9] = 30,
+ [0][0][RTW89_FCC][10] = 56,
+ [0][0][RTW89_ETSI][10] = 30,
+ [0][0][RTW89_MKK][10] = 44,
+ [0][0][RTW89_IC][10] = 56,
+ [0][0][RTW89_KCC][10] = 44,
+ [0][0][RTW89_ACMA][10] = 30,
+ [0][0][RTW89_CN][10] = 32,
+ [0][0][RTW89_UK][10] = 30,
+ [0][0][RTW89_FCC][11] = 54,
+ [0][0][RTW89_ETSI][11] = 30,
+ [0][0][RTW89_MKK][11] = 44,
+ [0][0][RTW89_IC][11] = 54,
+ [0][0][RTW89_KCC][11] = 44,
+ [0][0][RTW89_ACMA][11] = 30,
+ [0][0][RTW89_CN][11] = 32,
+ [0][0][RTW89_UK][11] = 30,
+ [0][0][RTW89_FCC][12] = 34,
+ [0][0][RTW89_ETSI][12] = 30,
+ [0][0][RTW89_MKK][12] = 40,
+ [0][0][RTW89_IC][12] = 34,
+ [0][0][RTW89_KCC][12] = 44,
+ [0][0][RTW89_ACMA][12] = 30,
+ [0][0][RTW89_CN][12] = 32,
+ [0][0][RTW89_UK][12] = 30,
+ [0][0][RTW89_FCC][13] = 127,
+ [0][0][RTW89_ETSI][13] = 127,
+ [0][0][RTW89_MKK][13] = 127,
+ [0][0][RTW89_IC][13] = 127,
+ [0][0][RTW89_KCC][13] = 127,
+ [0][0][RTW89_ACMA][13] = 127,
+ [0][0][RTW89_CN][13] = 127,
+ [0][0][RTW89_UK][13] = 127,
+ [0][1][RTW89_FCC][0] = 127,
+ [0][1][RTW89_ETSI][0] = 127,
+ [0][1][RTW89_MKK][0] = 127,
+ [0][1][RTW89_IC][0] = 127,
+ [0][1][RTW89_KCC][0] = 127,
+ [0][1][RTW89_ACMA][0] = 127,
+ [0][1][RTW89_CN][0] = 20,
+ [0][1][RTW89_UK][0] = 127,
+ [0][1][RTW89_FCC][1] = 127,
+ [0][1][RTW89_ETSI][1] = 127,
+ [0][1][RTW89_MKK][1] = 127,
+ [0][1][RTW89_IC][1] = 127,
+ [0][1][RTW89_KCC][1] = 127,
+ [0][1][RTW89_ACMA][1] = 127,
+ [0][1][RTW89_CN][1] = 22,
+ [0][1][RTW89_UK][1] = 127,
+ [0][1][RTW89_FCC][2] = 127,
+ [0][1][RTW89_ETSI][2] = 127,
+ [0][1][RTW89_MKK][2] = 127,
+ [0][1][RTW89_IC][2] = 127,
+ [0][1][RTW89_KCC][2] = 127,
+ [0][1][RTW89_ACMA][2] = 127,
+ [0][1][RTW89_CN][2] = 22,
+ [0][1][RTW89_UK][2] = 127,
+ [0][1][RTW89_FCC][3] = 127,
+ [0][1][RTW89_ETSI][3] = 127,
+ [0][1][RTW89_MKK][3] = 127,
+ [0][1][RTW89_IC][3] = 127,
+ [0][1][RTW89_KCC][3] = 127,
+ [0][1][RTW89_ACMA][3] = 127,
+ [0][1][RTW89_CN][3] = 22,
+ [0][1][RTW89_UK][3] = 127,
+ [0][1][RTW89_FCC][4] = 127,
+ [0][1][RTW89_ETSI][4] = 127,
+ [0][1][RTW89_MKK][4] = 127,
+ [0][1][RTW89_IC][4] = 127,
+ [0][1][RTW89_KCC][4] = 127,
+ [0][1][RTW89_ACMA][4] = 127,
+ [0][1][RTW89_CN][4] = 22,
+ [0][1][RTW89_UK][4] = 127,
+ [0][1][RTW89_FCC][5] = 127,
+ [0][1][RTW89_ETSI][5] = 127,
+ [0][1][RTW89_MKK][5] = 127,
+ [0][1][RTW89_IC][5] = 127,
+ [0][1][RTW89_KCC][5] = 127,
+ [0][1][RTW89_ACMA][5] = 127,
+ [0][1][RTW89_CN][5] = 22,
+ [0][1][RTW89_UK][5] = 127,
+ [0][1][RTW89_FCC][6] = 127,
+ [0][1][RTW89_ETSI][6] = 127,
+ [0][1][RTW89_MKK][6] = 127,
+ [0][1][RTW89_IC][6] = 127,
+ [0][1][RTW89_KCC][6] = 127,
+ [0][1][RTW89_ACMA][6] = 127,
+ [0][1][RTW89_CN][6] = 22,
+ [0][1][RTW89_UK][6] = 127,
+ [0][1][RTW89_FCC][7] = 127,
+ [0][1][RTW89_ETSI][7] = 127,
+ [0][1][RTW89_MKK][7] = 127,
+ [0][1][RTW89_IC][7] = 127,
+ [0][1][RTW89_KCC][7] = 127,
+ [0][1][RTW89_ACMA][7] = 127,
+ [0][1][RTW89_CN][7] = 22,
+ [0][1][RTW89_UK][7] = 127,
+ [0][1][RTW89_FCC][8] = 127,
+ [0][1][RTW89_ETSI][8] = 127,
+ [0][1][RTW89_MKK][8] = 127,
+ [0][1][RTW89_IC][8] = 127,
+ [0][1][RTW89_KCC][8] = 127,
+ [0][1][RTW89_ACMA][8] = 127,
+ [0][1][RTW89_CN][8] = 22,
+ [0][1][RTW89_UK][8] = 127,
+ [0][1][RTW89_FCC][9] = 127,
+ [0][1][RTW89_ETSI][9] = 127,
+ [0][1][RTW89_MKK][9] = 127,
+ [0][1][RTW89_IC][9] = 127,
+ [0][1][RTW89_KCC][9] = 127,
+ [0][1][RTW89_ACMA][9] = 127,
+ [0][1][RTW89_CN][9] = 22,
+ [0][1][RTW89_UK][9] = 127,
+ [0][1][RTW89_FCC][10] = 127,
+ [0][1][RTW89_ETSI][10] = 127,
+ [0][1][RTW89_MKK][10] = 127,
+ [0][1][RTW89_IC][10] = 127,
+ [0][1][RTW89_KCC][10] = 127,
+ [0][1][RTW89_ACMA][10] = 127,
+ [0][1][RTW89_CN][10] = 22,
+ [0][1][RTW89_UK][10] = 127,
+ [0][1][RTW89_FCC][11] = 127,
+ [0][1][RTW89_ETSI][11] = 127,
+ [0][1][RTW89_MKK][11] = 127,
+ [0][1][RTW89_IC][11] = 127,
+ [0][1][RTW89_KCC][11] = 127,
+ [0][1][RTW89_ACMA][11] = 127,
+ [0][1][RTW89_CN][11] = 22,
+ [0][1][RTW89_UK][11] = 127,
+ [0][1][RTW89_FCC][12] = 127,
+ [0][1][RTW89_ETSI][12] = 127,
+ [0][1][RTW89_MKK][12] = 127,
+ [0][1][RTW89_IC][12] = 127,
+ [0][1][RTW89_KCC][12] = 127,
+ [0][1][RTW89_ACMA][12] = 127,
+ [0][1][RTW89_CN][12] = 20,
+ [0][1][RTW89_UK][12] = 127,
+ [0][1][RTW89_FCC][13] = 127,
+ [0][1][RTW89_ETSI][13] = 127,
+ [0][1][RTW89_MKK][13] = 127,
+ [0][1][RTW89_IC][13] = 127,
+ [0][1][RTW89_KCC][13] = 127,
+ [0][1][RTW89_ACMA][13] = 127,
+ [0][1][RTW89_CN][13] = 127,
+ [0][1][RTW89_UK][13] = 127,
+ [1][0][RTW89_FCC][0] = 70,
+ [1][0][RTW89_ETSI][0] = 42,
+ [1][0][RTW89_MKK][0] = 52,
+ [1][0][RTW89_IC][0] = 70,
+ [1][0][RTW89_KCC][0] = 56,
+ [1][0][RTW89_ACMA][0] = 42,
+ [1][0][RTW89_CN][0] = 42,
+ [1][0][RTW89_UK][0] = 42,
+ [1][0][RTW89_FCC][1] = 70,
+ [1][0][RTW89_ETSI][1] = 42,
+ [1][0][RTW89_MKK][1] = 52,
+ [1][0][RTW89_IC][1] = 70,
+ [1][0][RTW89_KCC][1] = 56,
+ [1][0][RTW89_ACMA][1] = 42,
+ [1][0][RTW89_CN][1] = 44,
+ [1][0][RTW89_UK][1] = 42,
+ [1][0][RTW89_FCC][2] = 74,
+ [1][0][RTW89_ETSI][2] = 42,
+ [1][0][RTW89_MKK][2] = 52,
+ [1][0][RTW89_IC][2] = 74,
+ [1][0][RTW89_KCC][2] = 56,
+ [1][0][RTW89_ACMA][2] = 42,
+ [1][0][RTW89_CN][2] = 44,
+ [1][0][RTW89_UK][2] = 42,
+ [1][0][RTW89_FCC][3] = 76,
+ [1][0][RTW89_ETSI][3] = 42,
+ [1][0][RTW89_MKK][3] = 52,
+ [1][0][RTW89_IC][3] = 76,
+ [1][0][RTW89_KCC][3] = 56,
+ [1][0][RTW89_ACMA][3] = 42,
+ [1][0][RTW89_CN][3] = 44,
+ [1][0][RTW89_UK][3] = 42,
+ [1][0][RTW89_FCC][4] = 76,
+ [1][0][RTW89_ETSI][4] = 42,
+ [1][0][RTW89_MKK][4] = 52,
+ [1][0][RTW89_IC][4] = 76,
+ [1][0][RTW89_KCC][4] = 56,
+ [1][0][RTW89_ACMA][4] = 42,
+ [1][0][RTW89_CN][4] = 44,
+ [1][0][RTW89_UK][4] = 42,
+ [1][0][RTW89_FCC][5] = 82,
+ [1][0][RTW89_ETSI][5] = 42,
+ [1][0][RTW89_MKK][5] = 52,
+ [1][0][RTW89_IC][5] = 82,
+ [1][0][RTW89_KCC][5] = 56,
+ [1][0][RTW89_ACMA][5] = 42,
+ [1][0][RTW89_CN][5] = 44,
+ [1][0][RTW89_UK][5] = 42,
+ [1][0][RTW89_FCC][6] = 72,
+ [1][0][RTW89_ETSI][6] = 42,
+ [1][0][RTW89_MKK][6] = 52,
+ [1][0][RTW89_IC][6] = 72,
+ [1][0][RTW89_KCC][6] = 56,
+ [1][0][RTW89_ACMA][6] = 42,
+ [1][0][RTW89_CN][6] = 44,
+ [1][0][RTW89_UK][6] = 42,
+ [1][0][RTW89_FCC][7] = 72,
+ [1][0][RTW89_ETSI][7] = 42,
+ [1][0][RTW89_MKK][7] = 52,
+ [1][0][RTW89_IC][7] = 72,
+ [1][0][RTW89_KCC][7] = 56,
+ [1][0][RTW89_ACMA][7] = 42,
+ [1][0][RTW89_CN][7] = 44,
+ [1][0][RTW89_UK][7] = 42,
+ [1][0][RTW89_FCC][8] = 72,
+ [1][0][RTW89_ETSI][8] = 42,
+ [1][0][RTW89_MKK][8] = 52,
+ [1][0][RTW89_IC][8] = 72,
+ [1][0][RTW89_KCC][8] = 56,
+ [1][0][RTW89_ACMA][8] = 42,
+ [1][0][RTW89_CN][8] = 44,
+ [1][0][RTW89_UK][8] = 42,
+ [1][0][RTW89_FCC][9] = 68,
+ [1][0][RTW89_ETSI][9] = 42,
+ [1][0][RTW89_MKK][9] = 52,
+ [1][0][RTW89_IC][9] = 68,
+ [1][0][RTW89_KCC][9] = 58,
+ [1][0][RTW89_ACMA][9] = 42,
+ [1][0][RTW89_CN][9] = 44,
+ [1][0][RTW89_UK][9] = 42,
+ [1][0][RTW89_FCC][10] = 68,
+ [1][0][RTW89_ETSI][10] = 42,
+ [1][0][RTW89_MKK][10] = 52,
+ [1][0][RTW89_IC][10] = 68,
+ [1][0][RTW89_KCC][10] = 58,
+ [1][0][RTW89_ACMA][10] = 42,
+ [1][0][RTW89_CN][10] = 44,
+ [1][0][RTW89_UK][10] = 42,
+ [1][0][RTW89_FCC][11] = 66,
+ [1][0][RTW89_ETSI][11] = 42,
+ [1][0][RTW89_MKK][11] = 52,
+ [1][0][RTW89_IC][11] = 66,
+ [1][0][RTW89_KCC][11] = 58,
+ [1][0][RTW89_ACMA][11] = 42,
+ [1][0][RTW89_CN][11] = 44,
+ [1][0][RTW89_UK][11] = 42,
+ [1][0][RTW89_FCC][12] = 34,
+ [1][0][RTW89_ETSI][12] = 42,
+ [1][0][RTW89_MKK][12] = 52,
+ [1][0][RTW89_IC][12] = 34,
+ [1][0][RTW89_KCC][12] = 58,
+ [1][0][RTW89_ACMA][12] = 42,
+ [1][0][RTW89_CN][12] = 42,
+ [1][0][RTW89_UK][12] = 42,
+ [1][0][RTW89_FCC][13] = 127,
+ [1][0][RTW89_ETSI][13] = 127,
+ [1][0][RTW89_MKK][13] = 127,
+ [1][0][RTW89_IC][13] = 127,
+ [1][0][RTW89_KCC][13] = 127,
+ [1][0][RTW89_ACMA][13] = 127,
+ [1][0][RTW89_CN][13] = 127,
+ [1][0][RTW89_UK][13] = 127,
+ [1][1][RTW89_FCC][0] = 127,
+ [1][1][RTW89_ETSI][0] = 127,
+ [1][1][RTW89_MKK][0] = 127,
+ [1][1][RTW89_IC][0] = 127,
+ [1][1][RTW89_KCC][0] = 127,
+ [1][1][RTW89_ACMA][0] = 127,
+ [1][1][RTW89_CN][0] = 32,
+ [1][1][RTW89_UK][0] = 127,
+ [1][1][RTW89_FCC][1] = 127,
+ [1][1][RTW89_ETSI][1] = 127,
+ [1][1][RTW89_MKK][1] = 127,
+ [1][1][RTW89_IC][1] = 127,
+ [1][1][RTW89_KCC][1] = 127,
+ [1][1][RTW89_ACMA][1] = 127,
+ [1][1][RTW89_CN][1] = 32,
+ [1][1][RTW89_UK][1] = 127,
+ [1][1][RTW89_FCC][2] = 127,
+ [1][1][RTW89_ETSI][2] = 127,
+ [1][1][RTW89_MKK][2] = 127,
+ [1][1][RTW89_IC][2] = 127,
+ [1][1][RTW89_KCC][2] = 127,
+ [1][1][RTW89_ACMA][2] = 127,
+ [1][1][RTW89_CN][2] = 32,
+ [1][1][RTW89_UK][2] = 127,
+ [1][1][RTW89_FCC][3] = 127,
+ [1][1][RTW89_ETSI][3] = 127,
+ [1][1][RTW89_MKK][3] = 127,
+ [1][1][RTW89_IC][3] = 127,
+ [1][1][RTW89_KCC][3] = 127,
+ [1][1][RTW89_ACMA][3] = 127,
+ [1][1][RTW89_CN][3] = 32,
+ [1][1][RTW89_UK][3] = 127,
+ [1][1][RTW89_FCC][4] = 127,
+ [1][1][RTW89_ETSI][4] = 127,
+ [1][1][RTW89_MKK][4] = 127,
+ [1][1][RTW89_IC][4] = 127,
+ [1][1][RTW89_KCC][4] = 127,
+ [1][1][RTW89_ACMA][4] = 127,
+ [1][1][RTW89_CN][4] = 32,
+ [1][1][RTW89_UK][4] = 127,
+ [1][1][RTW89_FCC][5] = 127,
+ [1][1][RTW89_ETSI][5] = 127,
+ [1][1][RTW89_MKK][5] = 127,
+ [1][1][RTW89_IC][5] = 127,
+ [1][1][RTW89_KCC][5] = 127,
+ [1][1][RTW89_ACMA][5] = 127,
+ [1][1][RTW89_CN][5] = 32,
+ [1][1][RTW89_UK][5] = 127,
+ [1][1][RTW89_FCC][6] = 127,
+ [1][1][RTW89_ETSI][6] = 127,
+ [1][1][RTW89_MKK][6] = 127,
+ [1][1][RTW89_IC][6] = 127,
+ [1][1][RTW89_KCC][6] = 127,
+ [1][1][RTW89_ACMA][6] = 127,
+ [1][1][RTW89_CN][6] = 32,
+ [1][1][RTW89_UK][6] = 127,
+ [1][1][RTW89_FCC][7] = 127,
+ [1][1][RTW89_ETSI][7] = 127,
+ [1][1][RTW89_MKK][7] = 127,
+ [1][1][RTW89_IC][7] = 127,
+ [1][1][RTW89_KCC][7] = 127,
+ [1][1][RTW89_ACMA][7] = 127,
+ [1][1][RTW89_CN][7] = 32,
+ [1][1][RTW89_UK][7] = 127,
+ [1][1][RTW89_FCC][8] = 127,
+ [1][1][RTW89_ETSI][8] = 127,
+ [1][1][RTW89_MKK][8] = 127,
+ [1][1][RTW89_IC][8] = 127,
+ [1][1][RTW89_KCC][8] = 127,
+ [1][1][RTW89_ACMA][8] = 127,
+ [1][1][RTW89_CN][8] = 32,
+ [1][1][RTW89_UK][8] = 127,
+ [1][1][RTW89_FCC][9] = 127,
+ [1][1][RTW89_ETSI][9] = 127,
+ [1][1][RTW89_MKK][9] = 127,
+ [1][1][RTW89_IC][9] = 127,
+ [1][1][RTW89_KCC][9] = 127,
+ [1][1][RTW89_ACMA][9] = 127,
+ [1][1][RTW89_CN][9] = 32,
+ [1][1][RTW89_UK][9] = 127,
+ [1][1][RTW89_FCC][10] = 127,
+ [1][1][RTW89_ETSI][10] = 127,
+ [1][1][RTW89_MKK][10] = 127,
+ [1][1][RTW89_IC][10] = 127,
+ [1][1][RTW89_KCC][10] = 127,
+ [1][1][RTW89_ACMA][10] = 127,
+ [1][1][RTW89_CN][10] = 32,
+ [1][1][RTW89_UK][10] = 127,
+ [1][1][RTW89_FCC][11] = 127,
+ [1][1][RTW89_ETSI][11] = 127,
+ [1][1][RTW89_MKK][11] = 127,
+ [1][1][RTW89_IC][11] = 127,
+ [1][1][RTW89_KCC][11] = 127,
+ [1][1][RTW89_ACMA][11] = 127,
+ [1][1][RTW89_CN][11] = 32,
+ [1][1][RTW89_UK][11] = 127,
+ [1][1][RTW89_FCC][12] = 127,
+ [1][1][RTW89_ETSI][12] = 127,
+ [1][1][RTW89_MKK][12] = 127,
+ [1][1][RTW89_IC][12] = 127,
+ [1][1][RTW89_KCC][12] = 127,
+ [1][1][RTW89_ACMA][12] = 127,
+ [1][1][RTW89_CN][12] = 32,
+ [1][1][RTW89_UK][12] = 127,
+ [1][1][RTW89_FCC][13] = 127,
+ [1][1][RTW89_ETSI][13] = 127,
+ [1][1][RTW89_MKK][13] = 127,
+ [1][1][RTW89_IC][13] = 127,
+ [1][1][RTW89_KCC][13] = 127,
+ [1][1][RTW89_ACMA][13] = 127,
+ [1][1][RTW89_CN][13] = 127,
+ [1][1][RTW89_UK][13] = 127,
+ [2][0][RTW89_FCC][0] = 74,
+ [2][0][RTW89_ETSI][0] = 54,
+ [2][0][RTW89_MKK][0] = 64,
+ [2][0][RTW89_IC][0] = 74,
+ [2][0][RTW89_KCC][0] = 68,
+ [2][0][RTW89_ACMA][0] = 54,
+ [2][0][RTW89_CN][0] = 56,
+ [2][0][RTW89_UK][0] = 54,
+ [2][0][RTW89_FCC][1] = 74,
+ [2][0][RTW89_ETSI][1] = 54,
+ [2][0][RTW89_MKK][1] = 64,
+ [2][0][RTW89_IC][1] = 74,
+ [2][0][RTW89_KCC][1] = 68,
+ [2][0][RTW89_ACMA][1] = 54,
+ [2][0][RTW89_CN][1] = 56,
+ [2][0][RTW89_UK][1] = 54,
+ [2][0][RTW89_FCC][2] = 76,
+ [2][0][RTW89_ETSI][2] = 54,
+ [2][0][RTW89_MKK][2] = 64,
+ [2][0][RTW89_IC][2] = 76,
+ [2][0][RTW89_KCC][2] = 68,
+ [2][0][RTW89_ACMA][2] = 54,
+ [2][0][RTW89_CN][2] = 56,
+ [2][0][RTW89_UK][2] = 54,
+ [2][0][RTW89_FCC][3] = 76,
+ [2][0][RTW89_ETSI][3] = 54,
+ [2][0][RTW89_MKK][3] = 64,
+ [2][0][RTW89_IC][3] = 76,
+ [2][0][RTW89_KCC][3] = 68,
+ [2][0][RTW89_ACMA][3] = 54,
+ [2][0][RTW89_CN][3] = 56,
+ [2][0][RTW89_UK][3] = 54,
+ [2][0][RTW89_FCC][4] = 76,
+ [2][0][RTW89_ETSI][4] = 54,
+ [2][0][RTW89_MKK][4] = 64,
+ [2][0][RTW89_IC][4] = 76,
+ [2][0][RTW89_KCC][4] = 68,
+ [2][0][RTW89_ACMA][4] = 54,
+ [2][0][RTW89_CN][4] = 56,
+ [2][0][RTW89_UK][4] = 54,
+ [2][0][RTW89_FCC][5] = 80,
+ [2][0][RTW89_ETSI][5] = 54,
+ [2][0][RTW89_MKK][5] = 64,
+ [2][0][RTW89_IC][5] = 80,
+ [2][0][RTW89_KCC][5] = 68,
+ [2][0][RTW89_ACMA][5] = 54,
+ [2][0][RTW89_CN][5] = 56,
+ [2][0][RTW89_UK][5] = 54,
+ [2][0][RTW89_FCC][6] = 72,
+ [2][0][RTW89_ETSI][6] = 54,
+ [2][0][RTW89_MKK][6] = 64,
+ [2][0][RTW89_IC][6] = 72,
+ [2][0][RTW89_KCC][6] = 68,
+ [2][0][RTW89_ACMA][6] = 54,
+ [2][0][RTW89_CN][6] = 56,
+ [2][0][RTW89_UK][6] = 54,
+ [2][0][RTW89_FCC][7] = 72,
+ [2][0][RTW89_ETSI][7] = 54,
+ [2][0][RTW89_MKK][7] = 64,
+ [2][0][RTW89_IC][7] = 72,
+ [2][0][RTW89_KCC][7] = 68,
+ [2][0][RTW89_ACMA][7] = 54,
+ [2][0][RTW89_CN][7] = 56,
+ [2][0][RTW89_UK][7] = 54,
+ [2][0][RTW89_FCC][8] = 72,
+ [2][0][RTW89_ETSI][8] = 54,
+ [2][0][RTW89_MKK][8] = 64,
+ [2][0][RTW89_IC][8] = 72,
+ [2][0][RTW89_KCC][8] = 68,
+ [2][0][RTW89_ACMA][8] = 54,
+ [2][0][RTW89_CN][8] = 56,
+ [2][0][RTW89_UK][8] = 54,
+ [2][0][RTW89_FCC][9] = 70,
+ [2][0][RTW89_ETSI][9] = 54,
+ [2][0][RTW89_MKK][9] = 64,
+ [2][0][RTW89_IC][9] = 70,
+ [2][0][RTW89_KCC][9] = 68,
+ [2][0][RTW89_ACMA][9] = 54,
+ [2][0][RTW89_CN][9] = 56,
+ [2][0][RTW89_UK][9] = 54,
+ [2][0][RTW89_FCC][10] = 70,
+ [2][0][RTW89_ETSI][10] = 54,
+ [2][0][RTW89_MKK][10] = 64,
+ [2][0][RTW89_IC][10] = 70,
+ [2][0][RTW89_KCC][10] = 68,
+ [2][0][RTW89_ACMA][10] = 54,
+ [2][0][RTW89_CN][10] = 56,
+ [2][0][RTW89_UK][10] = 54,
+ [2][0][RTW89_FCC][11] = 62,
+ [2][0][RTW89_ETSI][11] = 54,
+ [2][0][RTW89_MKK][11] = 64,
+ [2][0][RTW89_IC][11] = 62,
+ [2][0][RTW89_KCC][11] = 68,
+ [2][0][RTW89_ACMA][11] = 54,
+ [2][0][RTW89_CN][11] = 56,
+ [2][0][RTW89_UK][11] = 54,
+ [2][0][RTW89_FCC][12] = 34,
+ [2][0][RTW89_ETSI][12] = 54,
+ [2][0][RTW89_MKK][12] = 64,
+ [2][0][RTW89_IC][12] = 34,
+ [2][0][RTW89_KCC][12] = 68,
+ [2][0][RTW89_ACMA][12] = 54,
+ [2][0][RTW89_CN][12] = 56,
+ [2][0][RTW89_UK][12] = 54,
+ [2][0][RTW89_FCC][13] = 127,
+ [2][0][RTW89_ETSI][13] = 127,
+ [2][0][RTW89_MKK][13] = 127,
+ [2][0][RTW89_IC][13] = 127,
+ [2][0][RTW89_KCC][13] = 127,
+ [2][0][RTW89_ACMA][13] = 127,
+ [2][0][RTW89_CN][13] = 127,
+ [2][0][RTW89_UK][13] = 127,
+ [2][1][RTW89_FCC][0] = 127,
+ [2][1][RTW89_ETSI][0] = 127,
+ [2][1][RTW89_MKK][0] = 127,
+ [2][1][RTW89_IC][0] = 127,
+ [2][1][RTW89_KCC][0] = 127,
+ [2][1][RTW89_ACMA][0] = 127,
+ [2][1][RTW89_CN][0] = 44,
+ [2][1][RTW89_UK][0] = 127,
+ [2][1][RTW89_FCC][1] = 127,
+ [2][1][RTW89_ETSI][1] = 127,
+ [2][1][RTW89_MKK][1] = 127,
+ [2][1][RTW89_IC][1] = 127,
+ [2][1][RTW89_KCC][1] = 127,
+ [2][1][RTW89_ACMA][1] = 127,
+ [2][1][RTW89_CN][1] = 44,
+ [2][1][RTW89_UK][1] = 127,
+ [2][1][RTW89_FCC][2] = 127,
+ [2][1][RTW89_ETSI][2] = 127,
+ [2][1][RTW89_MKK][2] = 127,
+ [2][1][RTW89_IC][2] = 127,
+ [2][1][RTW89_KCC][2] = 127,
+ [2][1][RTW89_ACMA][2] = 127,
+ [2][1][RTW89_CN][2] = 44,
+ [2][1][RTW89_UK][2] = 127,
+ [2][1][RTW89_FCC][3] = 127,
+ [2][1][RTW89_ETSI][3] = 127,
+ [2][1][RTW89_MKK][3] = 127,
+ [2][1][RTW89_IC][3] = 127,
+ [2][1][RTW89_KCC][3] = 127,
+ [2][1][RTW89_ACMA][3] = 127,
+ [2][1][RTW89_CN][3] = 44,
+ [2][1][RTW89_UK][3] = 127,
+ [2][1][RTW89_FCC][4] = 127,
+ [2][1][RTW89_ETSI][4] = 127,
+ [2][1][RTW89_MKK][4] = 127,
+ [2][1][RTW89_IC][4] = 127,
+ [2][1][RTW89_KCC][4] = 127,
+ [2][1][RTW89_ACMA][4] = 127,
+ [2][1][RTW89_CN][4] = 44,
+ [2][1][RTW89_UK][4] = 127,
+ [2][1][RTW89_FCC][5] = 127,
+ [2][1][RTW89_ETSI][5] = 127,
+ [2][1][RTW89_MKK][5] = 127,
+ [2][1][RTW89_IC][5] = 127,
+ [2][1][RTW89_KCC][5] = 127,
+ [2][1][RTW89_ACMA][5] = 127,
+ [2][1][RTW89_CN][5] = 44,
+ [2][1][RTW89_UK][5] = 127,
+ [2][1][RTW89_FCC][6] = 127,
+ [2][1][RTW89_ETSI][6] = 127,
+ [2][1][RTW89_MKK][6] = 127,
+ [2][1][RTW89_IC][6] = 127,
+ [2][1][RTW89_KCC][6] = 127,
+ [2][1][RTW89_ACMA][6] = 127,
+ [2][1][RTW89_CN][6] = 44,
+ [2][1][RTW89_UK][6] = 127,
+ [2][1][RTW89_FCC][7] = 127,
+ [2][1][RTW89_ETSI][7] = 127,
+ [2][1][RTW89_MKK][7] = 127,
+ [2][1][RTW89_IC][7] = 127,
+ [2][1][RTW89_KCC][7] = 127,
+ [2][1][RTW89_ACMA][7] = 127,
+ [2][1][RTW89_CN][7] = 44,
+ [2][1][RTW89_UK][7] = 127,
+ [2][1][RTW89_FCC][8] = 127,
+ [2][1][RTW89_ETSI][8] = 127,
+ [2][1][RTW89_MKK][8] = 127,
+ [2][1][RTW89_IC][8] = 127,
+ [2][1][RTW89_KCC][8] = 127,
+ [2][1][RTW89_ACMA][8] = 127,
+ [2][1][RTW89_CN][8] = 44,
+ [2][1][RTW89_UK][8] = 127,
+ [2][1][RTW89_FCC][9] = 127,
+ [2][1][RTW89_ETSI][9] = 127,
+ [2][1][RTW89_MKK][9] = 127,
+ [2][1][RTW89_IC][9] = 127,
+ [2][1][RTW89_KCC][9] = 127,
+ [2][1][RTW89_ACMA][9] = 127,
+ [2][1][RTW89_CN][9] = 44,
+ [2][1][RTW89_UK][9] = 127,
+ [2][1][RTW89_FCC][10] = 127,
+ [2][1][RTW89_ETSI][10] = 127,
+ [2][1][RTW89_MKK][10] = 127,
+ [2][1][RTW89_IC][10] = 127,
+ [2][1][RTW89_KCC][10] = 127,
+ [2][1][RTW89_ACMA][10] = 127,
+ [2][1][RTW89_CN][10] = 44,
+ [2][1][RTW89_UK][10] = 127,
+ [2][1][RTW89_FCC][11] = 127,
+ [2][1][RTW89_ETSI][11] = 127,
+ [2][1][RTW89_MKK][11] = 127,
+ [2][1][RTW89_IC][11] = 127,
+ [2][1][RTW89_KCC][11] = 127,
+ [2][1][RTW89_ACMA][11] = 127,
+ [2][1][RTW89_CN][11] = 44,
+ [2][1][RTW89_UK][11] = 127,
+ [2][1][RTW89_FCC][12] = 127,
+ [2][1][RTW89_ETSI][12] = 127,
+ [2][1][RTW89_MKK][12] = 127,
+ [2][1][RTW89_IC][12] = 127,
+ [2][1][RTW89_KCC][12] = 127,
+ [2][1][RTW89_ACMA][12] = 127,
+ [2][1][RTW89_CN][12] = 42,
+ [2][1][RTW89_UK][12] = 127,
+ [2][1][RTW89_FCC][13] = 127,
+ [2][1][RTW89_ETSI][13] = 127,
+ [2][1][RTW89_MKK][13] = 127,
+ [2][1][RTW89_IC][13] = 127,
+ [2][1][RTW89_KCC][13] = 127,
+ [2][1][RTW89_ACMA][13] = 127,
+ [2][1][RTW89_CN][13] = 127,
+ [2][1][RTW89_UK][13] = 127,
+};
+
+static
+const s8 rtw89_8851b_txpwr_lmt_ru_5g_type2[RTW89_RU_NUM][RTW89_NTX_NUM]
+ [RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
+ [0][0][RTW89_WW][0] = 16,
+ [0][0][RTW89_WW][2] = 16,
+ [0][0][RTW89_WW][4] = 16,
+ [0][0][RTW89_WW][6] = 16,
+ [0][0][RTW89_WW][8] = 16,
+ [0][0][RTW89_WW][10] = 16,
+ [0][0][RTW89_WW][12] = 16,
+ [0][0][RTW89_WW][14] = 16,
+ [0][0][RTW89_WW][15] = 24,
+ [0][0][RTW89_WW][17] = 24,
+ [0][0][RTW89_WW][19] = 24,
+ [0][0][RTW89_WW][21] = 24,
+ [0][0][RTW89_WW][23] = 24,
+ [0][0][RTW89_WW][25] = 24,
+ [0][0][RTW89_WW][27] = 24,
+ [0][0][RTW89_WW][29] = 24,
+ [0][0][RTW89_WW][31] = 24,
+ [0][0][RTW89_WW][33] = 24,
+ [0][0][RTW89_WW][35] = 24,
+ [0][0][RTW89_WW][37] = 44,
+ [0][0][RTW89_WW][38] = 24,
+ [0][0][RTW89_WW][40] = 24,
+ [0][0][RTW89_WW][42] = 24,
+ [0][0][RTW89_WW][44] = 24,
+ [0][0][RTW89_WW][46] = 24,
+ [0][0][RTW89_WW][48] = 40,
+ [0][0][RTW89_WW][50] = 42,
+ [0][0][RTW89_WW][52] = 38,
+ [0][1][RTW89_WW][0] = 4,
+ [0][1][RTW89_WW][2] = 4,
+ [0][1][RTW89_WW][4] = 4,
+ [0][1][RTW89_WW][6] = 4,
+ [0][1][RTW89_WW][8] = 4,
+ [0][1][RTW89_WW][10] = 4,
+ [0][1][RTW89_WW][12] = 4,
+ [0][1][RTW89_WW][14] = 4,
+ [0][1][RTW89_WW][15] = 0,
+ [0][1][RTW89_WW][17] = 0,
+ [0][1][RTW89_WW][19] = 0,
+ [0][1][RTW89_WW][21] = 0,
+ [0][1][RTW89_WW][23] = 0,
+ [0][1][RTW89_WW][25] = 0,
+ [0][1][RTW89_WW][27] = 0,
+ [0][1][RTW89_WW][29] = 0,
+ [0][1][RTW89_WW][31] = 0,
+ [0][1][RTW89_WW][33] = 0,
+ [0][1][RTW89_WW][35] = 0,
+ [0][1][RTW89_WW][37] = 0,
+ [0][1][RTW89_WW][38] = 42,
+ [0][1][RTW89_WW][40] = 42,
+ [0][1][RTW89_WW][42] = 42,
+ [0][1][RTW89_WW][44] = 42,
+ [0][1][RTW89_WW][46] = 42,
+ [0][1][RTW89_WW][48] = 0,
+ [0][1][RTW89_WW][50] = 0,
+ [0][1][RTW89_WW][52] = 0,
+ [1][0][RTW89_WW][0] = 26,
+ [1][0][RTW89_WW][2] = 26,
+ [1][0][RTW89_WW][4] = 26,
+ [1][0][RTW89_WW][6] = 26,
+ [1][0][RTW89_WW][8] = 26,
+ [1][0][RTW89_WW][10] = 26,
+ [1][0][RTW89_WW][12] = 26,
+ [1][0][RTW89_WW][14] = 26,
+ [1][0][RTW89_WW][15] = 34,
+ [1][0][RTW89_WW][17] = 34,
+ [1][0][RTW89_WW][19] = 34,
+ [1][0][RTW89_WW][21] = 34,
+ [1][0][RTW89_WW][23] = 34,
+ [1][0][RTW89_WW][25] = 34,
+ [1][0][RTW89_WW][27] = 34,
+ [1][0][RTW89_WW][29] = 34,
+ [1][0][RTW89_WW][31] = 34,
+ [1][0][RTW89_WW][33] = 34,
+ [1][0][RTW89_WW][35] = 34,
+ [1][0][RTW89_WW][37] = 54,
+ [1][0][RTW89_WW][38] = 28,
+ [1][0][RTW89_WW][40] = 28,
+ [1][0][RTW89_WW][42] = 28,
+ [1][0][RTW89_WW][44] = 28,
+ [1][0][RTW89_WW][46] = 28,
+ [1][0][RTW89_WW][48] = 52,
+ [1][0][RTW89_WW][50] = 52,
+ [1][0][RTW89_WW][52] = 50,
+ [1][1][RTW89_WW][0] = 14,
+ [1][1][RTW89_WW][2] = 14,
+ [1][1][RTW89_WW][4] = 14,
+ [1][1][RTW89_WW][6] = 14,
+ [1][1][RTW89_WW][8] = 14,
+ [1][1][RTW89_WW][10] = 14,
+ [1][1][RTW89_WW][12] = 14,
+ [1][1][RTW89_WW][14] = 14,
+ [1][1][RTW89_WW][15] = 0,
+ [1][1][RTW89_WW][17] = 0,
+ [1][1][RTW89_WW][19] = 0,
+ [1][1][RTW89_WW][21] = 0,
+ [1][1][RTW89_WW][23] = 0,
+ [1][1][RTW89_WW][25] = 0,
+ [1][1][RTW89_WW][27] = 0,
+ [1][1][RTW89_WW][29] = 0,
+ [1][1][RTW89_WW][31] = 0,
+ [1][1][RTW89_WW][33] = 0,
+ [1][1][RTW89_WW][35] = 0,
+ [1][1][RTW89_WW][37] = 0,
+ [1][1][RTW89_WW][38] = 54,
+ [1][1][RTW89_WW][40] = 54,
+ [1][1][RTW89_WW][42] = 54,
+ [1][1][RTW89_WW][44] = 54,
+ [1][1][RTW89_WW][46] = 54,
+ [1][1][RTW89_WW][48] = 0,
+ [1][1][RTW89_WW][50] = 0,
+ [1][1][RTW89_WW][52] = 0,
+ [2][0][RTW89_WW][0] = 40,
+ [2][0][RTW89_WW][2] = 40,
+ [2][0][RTW89_WW][4] = 40,
+ [2][0][RTW89_WW][6] = 40,
+ [2][0][RTW89_WW][8] = 40,
+ [2][0][RTW89_WW][10] = 40,
+ [2][0][RTW89_WW][12] = 40,
+ [2][0][RTW89_WW][14] = 40,
+ [2][0][RTW89_WW][15] = 46,
+ [2][0][RTW89_WW][17] = 46,
+ [2][0][RTW89_WW][19] = 46,
+ [2][0][RTW89_WW][21] = 46,
+ [2][0][RTW89_WW][23] = 46,
+ [2][0][RTW89_WW][25] = 46,
+ [2][0][RTW89_WW][27] = 46,
+ [2][0][RTW89_WW][29] = 46,
+ [2][0][RTW89_WW][31] = 46,
+ [2][0][RTW89_WW][33] = 46,
+ [2][0][RTW89_WW][35] = 46,
+ [2][0][RTW89_WW][37] = 66,
+ [2][0][RTW89_WW][38] = 28,
+ [2][0][RTW89_WW][40] = 28,
+ [2][0][RTW89_WW][42] = 28,
+ [2][0][RTW89_WW][44] = 28,
+ [2][0][RTW89_WW][46] = 28,
+ [2][0][RTW89_WW][48] = 62,
+ [2][0][RTW89_WW][50] = 62,
+ [2][0][RTW89_WW][52] = 60,
+ [2][1][RTW89_WW][0] = 28,
+ [2][1][RTW89_WW][2] = 28,
+ [2][1][RTW89_WW][4] = 28,
+ [2][1][RTW89_WW][6] = 28,
+ [2][1][RTW89_WW][8] = 28,
+ [2][1][RTW89_WW][10] = 28,
+ [2][1][RTW89_WW][12] = 28,
+ [2][1][RTW89_WW][14] = 28,
+ [2][1][RTW89_WW][15] = 0,
+ [2][1][RTW89_WW][17] = 0,
+ [2][1][RTW89_WW][19] = 0,
+ [2][1][RTW89_WW][21] = 0,
+ [2][1][RTW89_WW][23] = 0,
+ [2][1][RTW89_WW][25] = 0,
+ [2][1][RTW89_WW][27] = 0,
+ [2][1][RTW89_WW][29] = 0,
+ [2][1][RTW89_WW][31] = 0,
+ [2][1][RTW89_WW][33] = 0,
+ [2][1][RTW89_WW][35] = 0,
+ [2][1][RTW89_WW][37] = 0,
+ [2][1][RTW89_WW][38] = 56,
+ [2][1][RTW89_WW][40] = 56,
+ [2][1][RTW89_WW][42] = 56,
+ [2][1][RTW89_WW][44] = 56,
+ [2][1][RTW89_WW][46] = 56,
+ [2][1][RTW89_WW][48] = 0,
+ [2][1][RTW89_WW][50] = 0,
+ [2][1][RTW89_WW][52] = 0,
+ [0][0][RTW89_FCC][0] = 50,
+ [0][0][RTW89_ETSI][0] = 24,
+ [0][0][RTW89_MKK][0] = 26,
+ [0][0][RTW89_IC][0] = 28,
+ [0][0][RTW89_KCC][0] = 42,
+ [0][0][RTW89_ACMA][0] = 24,
+ [0][0][RTW89_CN][0] = 16,
+ [0][0][RTW89_UK][0] = 24,
+ [0][0][RTW89_FCC][2] = 54,
+ [0][0][RTW89_ETSI][2] = 24,
+ [0][0][RTW89_MKK][2] = 26,
+ [0][0][RTW89_IC][2] = 28,
+ [0][0][RTW89_KCC][2] = 42,
+ [0][0][RTW89_ACMA][2] = 24,
+ [0][0][RTW89_CN][2] = 16,
+ [0][0][RTW89_UK][2] = 24,
+ [0][0][RTW89_FCC][4] = 50,
+ [0][0][RTW89_ETSI][4] = 24,
+ [0][0][RTW89_MKK][4] = 26,
+ [0][0][RTW89_IC][4] = 28,
+ [0][0][RTW89_KCC][4] = 42,
+ [0][0][RTW89_ACMA][4] = 24,
+ [0][0][RTW89_CN][4] = 16,
+ [0][0][RTW89_UK][4] = 24,
+ [0][0][RTW89_FCC][6] = 50,
+ [0][0][RTW89_ETSI][6] = 24,
+ [0][0][RTW89_MKK][6] = 26,
+ [0][0][RTW89_IC][6] = 28,
+ [0][0][RTW89_KCC][6] = 18,
+ [0][0][RTW89_ACMA][6] = 24,
+ [0][0][RTW89_CN][6] = 16,
+ [0][0][RTW89_UK][6] = 24,
+ [0][0][RTW89_FCC][8] = 52,
+ [0][0][RTW89_ETSI][8] = 24,
+ [0][0][RTW89_MKK][8] = 26,
+ [0][0][RTW89_IC][8] = 52,
+ [0][0][RTW89_KCC][8] = 42,
+ [0][0][RTW89_ACMA][8] = 24,
+ [0][0][RTW89_CN][8] = 16,
+ [0][0][RTW89_UK][8] = 24,
+ [0][0][RTW89_FCC][10] = 52,
+ [0][0][RTW89_ETSI][10] = 24,
+ [0][0][RTW89_MKK][10] = 26,
+ [0][0][RTW89_IC][10] = 52,
+ [0][0][RTW89_KCC][10] = 42,
+ [0][0][RTW89_ACMA][10] = 24,
+ [0][0][RTW89_CN][10] = 16,
+ [0][0][RTW89_UK][10] = 24,
+ [0][0][RTW89_FCC][12] = 56,
+ [0][0][RTW89_ETSI][12] = 24,
+ [0][0][RTW89_MKK][12] = 26,
+ [0][0][RTW89_IC][12] = 56,
+ [0][0][RTW89_KCC][12] = 44,
+ [0][0][RTW89_ACMA][12] = 24,
+ [0][0][RTW89_CN][12] = 16,
+ [0][0][RTW89_UK][12] = 24,
+ [0][0][RTW89_FCC][14] = 56,
+ [0][0][RTW89_ETSI][14] = 24,
+ [0][0][RTW89_MKK][14] = 26,
+ [0][0][RTW89_IC][14] = 56,
+ [0][0][RTW89_KCC][14] = 44,
+ [0][0][RTW89_ACMA][14] = 24,
+ [0][0][RTW89_CN][14] = 16,
+ [0][0][RTW89_UK][14] = 24,
+ [0][0][RTW89_FCC][15] = 52,
+ [0][0][RTW89_ETSI][15] = 24,
+ [0][0][RTW89_MKK][15] = 46,
+ [0][0][RTW89_IC][15] = 52,
+ [0][0][RTW89_KCC][15] = 44,
+ [0][0][RTW89_ACMA][15] = 24,
+ [0][0][RTW89_CN][15] = 127,
+ [0][0][RTW89_UK][15] = 24,
+ [0][0][RTW89_FCC][17] = 52,
+ [0][0][RTW89_ETSI][17] = 24,
+ [0][0][RTW89_MKK][17] = 50,
+ [0][0][RTW89_IC][17] = 52,
+ [0][0][RTW89_KCC][17] = 44,
+ [0][0][RTW89_ACMA][17] = 24,
+ [0][0][RTW89_CN][17] = 127,
+ [0][0][RTW89_UK][17] = 24,
+ [0][0][RTW89_FCC][19] = 52,
+ [0][0][RTW89_ETSI][19] = 24,
+ [0][0][RTW89_MKK][19] = 50,
+ [0][0][RTW89_IC][19] = 52,
+ [0][0][RTW89_KCC][19] = 44,
+ [0][0][RTW89_ACMA][19] = 24,
+ [0][0][RTW89_CN][19] = 127,
+ [0][0][RTW89_UK][19] = 24,
+ [0][0][RTW89_FCC][21] = 52,
+ [0][0][RTW89_ETSI][21] = 24,
+ [0][0][RTW89_MKK][21] = 50,
+ [0][0][RTW89_IC][21] = 52,
+ [0][0][RTW89_KCC][21] = 44,
+ [0][0][RTW89_ACMA][21] = 24,
+ [0][0][RTW89_CN][21] = 127,
+ [0][0][RTW89_UK][21] = 24,
+ [0][0][RTW89_FCC][23] = 52,
+ [0][0][RTW89_ETSI][23] = 24,
+ [0][0][RTW89_MKK][23] = 50,
+ [0][0][RTW89_IC][23] = 52,
+ [0][0][RTW89_KCC][23] = 44,
+ [0][0][RTW89_ACMA][23] = 24,
+ [0][0][RTW89_CN][23] = 127,
+ [0][0][RTW89_UK][23] = 24,
+ [0][0][RTW89_FCC][25] = 52,
+ [0][0][RTW89_ETSI][25] = 24,
+ [0][0][RTW89_MKK][25] = 50,
+ [0][0][RTW89_IC][25] = 127,
+ [0][0][RTW89_KCC][25] = 44,
+ [0][0][RTW89_ACMA][25] = 127,
+ [0][0][RTW89_CN][25] = 127,
+ [0][0][RTW89_UK][25] = 24,
+ [0][0][RTW89_FCC][27] = 52,
+ [0][0][RTW89_ETSI][27] = 24,
+ [0][0][RTW89_MKK][27] = 50,
+ [0][0][RTW89_IC][27] = 127,
+ [0][0][RTW89_KCC][27] = 42,
+ [0][0][RTW89_ACMA][27] = 127,
+ [0][0][RTW89_CN][27] = 127,
+ [0][0][RTW89_UK][27] = 24,
+ [0][0][RTW89_FCC][29] = 52,
+ [0][0][RTW89_ETSI][29] = 24,
+ [0][0][RTW89_MKK][29] = 50,
+ [0][0][RTW89_IC][29] = 127,
+ [0][0][RTW89_KCC][29] = 42,
+ [0][0][RTW89_ACMA][29] = 127,
+ [0][0][RTW89_CN][29] = 127,
+ [0][0][RTW89_UK][29] = 24,
+ [0][0][RTW89_FCC][31] = 52,
+ [0][0][RTW89_ETSI][31] = 24,
+ [0][0][RTW89_MKK][31] = 50,
+ [0][0][RTW89_IC][31] = 56,
+ [0][0][RTW89_KCC][31] = 42,
+ [0][0][RTW89_ACMA][31] = 24,
+ [0][0][RTW89_CN][31] = 127,
+ [0][0][RTW89_UK][31] = 24,
+ [0][0][RTW89_FCC][33] = 56,
+ [0][0][RTW89_ETSI][33] = 24,
+ [0][0][RTW89_MKK][33] = 50,
+ [0][0][RTW89_IC][33] = 56,
+ [0][0][RTW89_KCC][33] = 42,
+ [0][0][RTW89_ACMA][33] = 24,
+ [0][0][RTW89_CN][33] = 127,
+ [0][0][RTW89_UK][33] = 24,
+ [0][0][RTW89_FCC][35] = 56,
+ [0][0][RTW89_ETSI][35] = 24,
+ [0][0][RTW89_MKK][35] = 50,
+ [0][0][RTW89_IC][35] = 56,
+ [0][0][RTW89_KCC][35] = 42,
+ [0][0][RTW89_ACMA][35] = 24,
+ [0][0][RTW89_CN][35] = 127,
+ [0][0][RTW89_UK][35] = 24,
+ [0][0][RTW89_FCC][37] = 84,
+ [0][0][RTW89_ETSI][37] = 127,
+ [0][0][RTW89_MKK][37] = 46,
+ [0][0][RTW89_IC][37] = 84,
+ [0][0][RTW89_KCC][37] = 44,
+ [0][0][RTW89_ACMA][37] = 50,
+ [0][0][RTW89_CN][37] = 127,
+ [0][0][RTW89_UK][37] = 52,
+ [0][0][RTW89_FCC][38] = 68,
+ [0][0][RTW89_ETSI][38] = 28,
+ [0][0][RTW89_MKK][38] = 127,
+ [0][0][RTW89_IC][38] = 68,
+ [0][0][RTW89_KCC][38] = 44,
+ [0][0][RTW89_ACMA][38] = 84,
+ [0][0][RTW89_CN][38] = 54,
+ [0][0][RTW89_UK][38] = 24,
+ [0][0][RTW89_FCC][40] = 68,
+ [0][0][RTW89_ETSI][40] = 28,
+ [0][0][RTW89_MKK][40] = 127,
+ [0][0][RTW89_IC][40] = 68,
+ [0][0][RTW89_KCC][40] = 44,
+ [0][0][RTW89_ACMA][40] = 84,
+ [0][0][RTW89_CN][40] = 54,
+ [0][0][RTW89_UK][40] = 24,
+ [0][0][RTW89_FCC][42] = 70,
+ [0][0][RTW89_ETSI][42] = 28,
+ [0][0][RTW89_MKK][42] = 127,
+ [0][0][RTW89_IC][42] = 70,
+ [0][0][RTW89_KCC][42] = 44,
+ [0][0][RTW89_ACMA][42] = 84,
+ [0][0][RTW89_CN][42] = 54,
+ [0][0][RTW89_UK][42] = 24,
+ [0][0][RTW89_FCC][44] = 62,
+ [0][0][RTW89_ETSI][44] = 28,
+ [0][0][RTW89_MKK][44] = 127,
+ [0][0][RTW89_IC][44] = 62,
+ [0][0][RTW89_KCC][44] = 44,
+ [0][0][RTW89_ACMA][44] = 84,
+ [0][0][RTW89_CN][44] = 54,
+ [0][0][RTW89_UK][44] = 24,
+ [0][0][RTW89_FCC][46] = 62,
+ [0][0][RTW89_ETSI][46] = 28,
+ [0][0][RTW89_MKK][46] = 127,
+ [0][0][RTW89_IC][46] = 62,
+ [0][0][RTW89_KCC][46] = 44,
+ [0][0][RTW89_ACMA][46] = 84,
+ [0][0][RTW89_CN][46] = 54,
+ [0][0][RTW89_UK][46] = 24,
+ [0][0][RTW89_FCC][48] = 40,
+ [0][0][RTW89_ETSI][48] = 127,
+ [0][0][RTW89_MKK][48] = 127,
+ [0][0][RTW89_IC][48] = 127,
+ [0][0][RTW89_KCC][48] = 127,
+ [0][0][RTW89_ACMA][48] = 127,
+ [0][0][RTW89_CN][48] = 127,
+ [0][0][RTW89_UK][48] = 127,
+ [0][0][RTW89_FCC][50] = 42,
+ [0][0][RTW89_ETSI][50] = 127,
+ [0][0][RTW89_MKK][50] = 127,
+ [0][0][RTW89_IC][50] = 127,
+ [0][0][RTW89_KCC][50] = 127,
+ [0][0][RTW89_ACMA][50] = 127,
+ [0][0][RTW89_CN][50] = 127,
+ [0][0][RTW89_UK][50] = 127,
+ [0][0][RTW89_FCC][52] = 38,
+ [0][0][RTW89_ETSI][52] = 127,
+ [0][0][RTW89_MKK][52] = 127,
+ [0][0][RTW89_IC][52] = 127,
+ [0][0][RTW89_KCC][52] = 127,
+ [0][0][RTW89_ACMA][52] = 127,
+ [0][0][RTW89_CN][52] = 127,
+ [0][0][RTW89_UK][52] = 127,
+ [0][1][RTW89_FCC][0] = 127,
+ [0][1][RTW89_ETSI][0] = 127,
+ [0][1][RTW89_MKK][0] = 127,
+ [0][1][RTW89_IC][0] = 127,
+ [0][1][RTW89_KCC][0] = 127,
+ [0][1][RTW89_ACMA][0] = 127,
+ [0][1][RTW89_CN][0] = 4,
+ [0][1][RTW89_UK][0] = 127,
+ [0][1][RTW89_FCC][2] = 127,
+ [0][1][RTW89_ETSI][2] = 127,
+ [0][1][RTW89_MKK][2] = 127,
+ [0][1][RTW89_IC][2] = 127,
+ [0][1][RTW89_KCC][2] = 127,
+ [0][1][RTW89_ACMA][2] = 127,
+ [0][1][RTW89_CN][2] = 4,
+ [0][1][RTW89_UK][2] = 127,
+ [0][1][RTW89_FCC][4] = 127,
+ [0][1][RTW89_ETSI][4] = 127,
+ [0][1][RTW89_MKK][4] = 127,
+ [0][1][RTW89_IC][4] = 127,
+ [0][1][RTW89_KCC][4] = 127,
+ [0][1][RTW89_ACMA][4] = 127,
+ [0][1][RTW89_CN][4] = 4,
+ [0][1][RTW89_UK][4] = 127,
+ [0][1][RTW89_FCC][6] = 127,
+ [0][1][RTW89_ETSI][6] = 127,
+ [0][1][RTW89_MKK][6] = 127,
+ [0][1][RTW89_IC][6] = 127,
+ [0][1][RTW89_KCC][6] = 127,
+ [0][1][RTW89_ACMA][6] = 127,
+ [0][1][RTW89_CN][6] = 4,
+ [0][1][RTW89_UK][6] = 127,
+ [0][1][RTW89_FCC][8] = 127,
+ [0][1][RTW89_ETSI][8] = 127,
+ [0][1][RTW89_MKK][8] = 127,
+ [0][1][RTW89_IC][8] = 127,
+ [0][1][RTW89_KCC][8] = 127,
+ [0][1][RTW89_ACMA][8] = 127,
+ [0][1][RTW89_CN][8] = 4,
+ [0][1][RTW89_UK][8] = 127,
+ [0][1][RTW89_FCC][10] = 127,
+ [0][1][RTW89_ETSI][10] = 127,
+ [0][1][RTW89_MKK][10] = 127,
+ [0][1][RTW89_IC][10] = 127,
+ [0][1][RTW89_KCC][10] = 127,
+ [0][1][RTW89_ACMA][10] = 127,
+ [0][1][RTW89_CN][10] = 4,
+ [0][1][RTW89_UK][10] = 127,
+ [0][1][RTW89_FCC][12] = 127,
+ [0][1][RTW89_ETSI][12] = 127,
+ [0][1][RTW89_MKK][12] = 127,
+ [0][1][RTW89_IC][12] = 127,
+ [0][1][RTW89_KCC][12] = 127,
+ [0][1][RTW89_ACMA][12] = 127,
+ [0][1][RTW89_CN][12] = 4,
+ [0][1][RTW89_UK][12] = 127,
+ [0][1][RTW89_FCC][14] = 127,
+ [0][1][RTW89_ETSI][14] = 127,
+ [0][1][RTW89_MKK][14] = 127,
+ [0][1][RTW89_IC][14] = 127,
+ [0][1][RTW89_KCC][14] = 127,
+ [0][1][RTW89_ACMA][14] = 127,
+ [0][1][RTW89_CN][14] = 4,
+ [0][1][RTW89_UK][14] = 127,
+ [0][1][RTW89_FCC][15] = 127,
+ [0][1][RTW89_ETSI][15] = 127,
+ [0][1][RTW89_MKK][15] = 127,
+ [0][1][RTW89_IC][15] = 127,
+ [0][1][RTW89_KCC][15] = 127,
+ [0][1][RTW89_ACMA][15] = 127,
+ [0][1][RTW89_CN][15] = 127,
+ [0][1][RTW89_UK][15] = 127,
+ [0][1][RTW89_FCC][17] = 127,
+ [0][1][RTW89_ETSI][17] = 127,
+ [0][1][RTW89_MKK][17] = 127,
+ [0][1][RTW89_IC][17] = 127,
+ [0][1][RTW89_KCC][17] = 127,
+ [0][1][RTW89_ACMA][17] = 127,
+ [0][1][RTW89_CN][17] = 127,
+ [0][1][RTW89_UK][17] = 127,
+ [0][1][RTW89_FCC][19] = 127,
+ [0][1][RTW89_ETSI][19] = 127,
+ [0][1][RTW89_MKK][19] = 127,
+ [0][1][RTW89_IC][19] = 127,
+ [0][1][RTW89_KCC][19] = 127,
+ [0][1][RTW89_ACMA][19] = 127,
+ [0][1][RTW89_CN][19] = 127,
+ [0][1][RTW89_UK][19] = 127,
+ [0][1][RTW89_FCC][21] = 127,
+ [0][1][RTW89_ETSI][21] = 127,
+ [0][1][RTW89_MKK][21] = 127,
+ [0][1][RTW89_IC][21] = 127,
+ [0][1][RTW89_KCC][21] = 127,
+ [0][1][RTW89_ACMA][21] = 127,
+ [0][1][RTW89_CN][21] = 127,
+ [0][1][RTW89_UK][21] = 127,
+ [0][1][RTW89_FCC][23] = 127,
+ [0][1][RTW89_ETSI][23] = 127,
+ [0][1][RTW89_MKK][23] = 127,
+ [0][1][RTW89_IC][23] = 127,
+ [0][1][RTW89_KCC][23] = 127,
+ [0][1][RTW89_ACMA][23] = 127,
+ [0][1][RTW89_CN][23] = 127,
+ [0][1][RTW89_UK][23] = 127,
+ [0][1][RTW89_FCC][25] = 127,
+ [0][1][RTW89_ETSI][25] = 127,
+ [0][1][RTW89_MKK][25] = 127,
+ [0][1][RTW89_IC][25] = 127,
+ [0][1][RTW89_KCC][25] = 127,
+ [0][1][RTW89_ACMA][25] = 127,
+ [0][1][RTW89_CN][25] = 127,
+ [0][1][RTW89_UK][25] = 127,
+ [0][1][RTW89_FCC][27] = 127,
+ [0][1][RTW89_ETSI][27] = 127,
+ [0][1][RTW89_MKK][27] = 127,
+ [0][1][RTW89_IC][27] = 127,
+ [0][1][RTW89_KCC][27] = 127,
+ [0][1][RTW89_ACMA][27] = 127,
+ [0][1][RTW89_CN][27] = 127,
+ [0][1][RTW89_UK][27] = 127,
+ [0][1][RTW89_FCC][29] = 127,
+ [0][1][RTW89_ETSI][29] = 127,
+ [0][1][RTW89_MKK][29] = 127,
+ [0][1][RTW89_IC][29] = 127,
+ [0][1][RTW89_KCC][29] = 127,
+ [0][1][RTW89_ACMA][29] = 127,
+ [0][1][RTW89_CN][29] = 127,
+ [0][1][RTW89_UK][29] = 127,
+ [0][1][RTW89_FCC][31] = 127,
+ [0][1][RTW89_ETSI][31] = 127,
+ [0][1][RTW89_MKK][31] = 127,
+ [0][1][RTW89_IC][31] = 127,
+ [0][1][RTW89_KCC][31] = 127,
+ [0][1][RTW89_ACMA][31] = 127,
+ [0][1][RTW89_CN][31] = 127,
+ [0][1][RTW89_UK][31] = 127,
+ [0][1][RTW89_FCC][33] = 127,
+ [0][1][RTW89_ETSI][33] = 127,
+ [0][1][RTW89_MKK][33] = 127,
+ [0][1][RTW89_IC][33] = 127,
+ [0][1][RTW89_KCC][33] = 127,
+ [0][1][RTW89_ACMA][33] = 127,
+ [0][1][RTW89_CN][33] = 127,
+ [0][1][RTW89_UK][33] = 127,
+ [0][1][RTW89_FCC][35] = 127,
+ [0][1][RTW89_ETSI][35] = 127,
+ [0][1][RTW89_MKK][35] = 127,
+ [0][1][RTW89_IC][35] = 127,
+ [0][1][RTW89_KCC][35] = 127,
+ [0][1][RTW89_ACMA][35] = 127,
+ [0][1][RTW89_CN][35] = 127,
+ [0][1][RTW89_UK][35] = 127,
+ [0][1][RTW89_FCC][37] = 127,
+ [0][1][RTW89_ETSI][37] = 127,
+ [0][1][RTW89_MKK][37] = 127,
+ [0][1][RTW89_IC][37] = 127,
+ [0][1][RTW89_KCC][37] = 127,
+ [0][1][RTW89_ACMA][37] = 127,
+ [0][1][RTW89_CN][37] = 127,
+ [0][1][RTW89_UK][37] = 127,
+ [0][1][RTW89_FCC][38] = 127,
+ [0][1][RTW89_ETSI][38] = 127,
+ [0][1][RTW89_MKK][38] = 127,
+ [0][1][RTW89_IC][38] = 127,
+ [0][1][RTW89_KCC][38] = 127,
+ [0][1][RTW89_ACMA][38] = 127,
+ [0][1][RTW89_CN][38] = 42,
+ [0][1][RTW89_UK][38] = 127,
+ [0][1][RTW89_FCC][40] = 127,
+ [0][1][RTW89_ETSI][40] = 127,
+ [0][1][RTW89_MKK][40] = 127,
+ [0][1][RTW89_IC][40] = 127,
+ [0][1][RTW89_KCC][40] = 127,
+ [0][1][RTW89_ACMA][40] = 127,
+ [0][1][RTW89_CN][40] = 42,
+ [0][1][RTW89_UK][40] = 127,
+ [0][1][RTW89_FCC][42] = 127,
+ [0][1][RTW89_ETSI][42] = 127,
+ [0][1][RTW89_MKK][42] = 127,
+ [0][1][RTW89_IC][42] = 127,
+ [0][1][RTW89_KCC][42] = 127,
+ [0][1][RTW89_ACMA][42] = 127,
+ [0][1][RTW89_CN][42] = 42,
+ [0][1][RTW89_UK][42] = 127,
+ [0][1][RTW89_FCC][44] = 127,
+ [0][1][RTW89_ETSI][44] = 127,
+ [0][1][RTW89_MKK][44] = 127,
+ [0][1][RTW89_IC][44] = 127,
+ [0][1][RTW89_KCC][44] = 127,
+ [0][1][RTW89_ACMA][44] = 127,
+ [0][1][RTW89_CN][44] = 42,
+ [0][1][RTW89_UK][44] = 127,
+ [0][1][RTW89_FCC][46] = 127,
+ [0][1][RTW89_ETSI][46] = 127,
+ [0][1][RTW89_MKK][46] = 127,
+ [0][1][RTW89_IC][46] = 127,
+ [0][1][RTW89_KCC][46] = 127,
+ [0][1][RTW89_ACMA][46] = 127,
+ [0][1][RTW89_CN][46] = 42,
+ [0][1][RTW89_UK][46] = 127,
+ [0][1][RTW89_FCC][48] = 127,
+ [0][1][RTW89_ETSI][48] = 127,
+ [0][1][RTW89_MKK][48] = 127,
+ [0][1][RTW89_IC][48] = 127,
+ [0][1][RTW89_KCC][48] = 127,
+ [0][1][RTW89_ACMA][48] = 127,
+ [0][1][RTW89_CN][48] = 127,
+ [0][1][RTW89_UK][48] = 127,
+ [0][1][RTW89_FCC][50] = 127,
+ [0][1][RTW89_ETSI][50] = 127,
+ [0][1][RTW89_MKK][50] = 127,
+ [0][1][RTW89_IC][50] = 127,
+ [0][1][RTW89_KCC][50] = 127,
+ [0][1][RTW89_ACMA][50] = 127,
+ [0][1][RTW89_CN][50] = 127,
+ [0][1][RTW89_UK][50] = 127,
+ [0][1][RTW89_FCC][52] = 127,
+ [0][1][RTW89_ETSI][52] = 127,
+ [0][1][RTW89_MKK][52] = 127,
+ [0][1][RTW89_IC][52] = 127,
+ [0][1][RTW89_KCC][52] = 127,
+ [0][1][RTW89_ACMA][52] = 127,
+ [0][1][RTW89_CN][52] = 127,
+ [0][1][RTW89_UK][52] = 127,
+ [1][0][RTW89_FCC][0] = 64,
+ [1][0][RTW89_ETSI][0] = 34,
+ [1][0][RTW89_MKK][0] = 38,
+ [1][0][RTW89_IC][0] = 38,
+ [1][0][RTW89_KCC][0] = 52,
+ [1][0][RTW89_ACMA][0] = 34,
+ [1][0][RTW89_CN][0] = 26,
+ [1][0][RTW89_UK][0] = 34,
+ [1][0][RTW89_FCC][2] = 66,
+ [1][0][RTW89_ETSI][2] = 34,
+ [1][0][RTW89_MKK][2] = 38,
+ [1][0][RTW89_IC][2] = 38,
+ [1][0][RTW89_KCC][2] = 52,
+ [1][0][RTW89_ACMA][2] = 34,
+ [1][0][RTW89_CN][2] = 26,
+ [1][0][RTW89_UK][2] = 34,
+ [1][0][RTW89_FCC][4] = 60,
+ [1][0][RTW89_ETSI][4] = 34,
+ [1][0][RTW89_MKK][4] = 36,
+ [1][0][RTW89_IC][4] = 38,
+ [1][0][RTW89_KCC][4] = 52,
+ [1][0][RTW89_ACMA][4] = 34,
+ [1][0][RTW89_CN][4] = 26,
+ [1][0][RTW89_UK][4] = 34,
+ [1][0][RTW89_FCC][6] = 60,
+ [1][0][RTW89_ETSI][6] = 34,
+ [1][0][RTW89_MKK][6] = 36,
+ [1][0][RTW89_IC][6] = 38,
+ [1][0][RTW89_KCC][6] = 32,
+ [1][0][RTW89_ACMA][6] = 34,
+ [1][0][RTW89_CN][6] = 26,
+ [1][0][RTW89_UK][6] = 34,
+ [1][0][RTW89_FCC][8] = 62,
+ [1][0][RTW89_ETSI][8] = 34,
+ [1][0][RTW89_MKK][8] = 38,
+ [1][0][RTW89_IC][8] = 62,
+ [1][0][RTW89_KCC][8] = 52,
+ [1][0][RTW89_ACMA][8] = 34,
+ [1][0][RTW89_CN][8] = 26,
+ [1][0][RTW89_UK][8] = 34,
+ [1][0][RTW89_FCC][10] = 62,
+ [1][0][RTW89_ETSI][10] = 34,
+ [1][0][RTW89_MKK][10] = 38,
+ [1][0][RTW89_IC][10] = 62,
+ [1][0][RTW89_KCC][10] = 52,
+ [1][0][RTW89_ACMA][10] = 34,
+ [1][0][RTW89_CN][10] = 26,
+ [1][0][RTW89_UK][10] = 34,
+ [1][0][RTW89_FCC][12] = 62,
+ [1][0][RTW89_ETSI][12] = 34,
+ [1][0][RTW89_MKK][12] = 38,
+ [1][0][RTW89_IC][12] = 62,
+ [1][0][RTW89_KCC][12] = 54,
+ [1][0][RTW89_ACMA][12] = 34,
+ [1][0][RTW89_CN][12] = 26,
+ [1][0][RTW89_UK][12] = 34,
+ [1][0][RTW89_FCC][14] = 62,
+ [1][0][RTW89_ETSI][14] = 34,
+ [1][0][RTW89_MKK][14] = 38,
+ [1][0][RTW89_IC][14] = 62,
+ [1][0][RTW89_KCC][14] = 54,
+ [1][0][RTW89_ACMA][14] = 34,
+ [1][0][RTW89_CN][14] = 26,
+ [1][0][RTW89_UK][14] = 34,
+ [1][0][RTW89_FCC][15] = 60,
+ [1][0][RTW89_ETSI][15] = 34,
+ [1][0][RTW89_MKK][15] = 58,
+ [1][0][RTW89_IC][15] = 60,
+ [1][0][RTW89_KCC][15] = 54,
+ [1][0][RTW89_ACMA][15] = 34,
+ [1][0][RTW89_CN][15] = 127,
+ [1][0][RTW89_UK][15] = 34,
+ [1][0][RTW89_FCC][17] = 60,
+ [1][0][RTW89_ETSI][17] = 34,
+ [1][0][RTW89_MKK][17] = 58,
+ [1][0][RTW89_IC][17] = 60,
+ [1][0][RTW89_KCC][17] = 54,
+ [1][0][RTW89_ACMA][17] = 34,
+ [1][0][RTW89_CN][17] = 127,
+ [1][0][RTW89_UK][17] = 34,
+ [1][0][RTW89_FCC][19] = 62,
+ [1][0][RTW89_ETSI][19] = 34,
+ [1][0][RTW89_MKK][19] = 58,
+ [1][0][RTW89_IC][19] = 62,
+ [1][0][RTW89_KCC][19] = 54,
+ [1][0][RTW89_ACMA][19] = 34,
+ [1][0][RTW89_CN][19] = 127,
+ [1][0][RTW89_UK][19] = 34,
+ [1][0][RTW89_FCC][21] = 62,
+ [1][0][RTW89_ETSI][21] = 34,
+ [1][0][RTW89_MKK][21] = 58,
+ [1][0][RTW89_IC][21] = 62,
+ [1][0][RTW89_KCC][21] = 54,
+ [1][0][RTW89_ACMA][21] = 34,
+ [1][0][RTW89_CN][21] = 127,
+ [1][0][RTW89_UK][21] = 34,
+ [1][0][RTW89_FCC][23] = 62,
+ [1][0][RTW89_ETSI][23] = 34,
+ [1][0][RTW89_MKK][23] = 58,
+ [1][0][RTW89_IC][23] = 62,
+ [1][0][RTW89_KCC][23] = 54,
+ [1][0][RTW89_ACMA][23] = 34,
+ [1][0][RTW89_CN][23] = 127,
+ [1][0][RTW89_UK][23] = 34,
+ [1][0][RTW89_FCC][25] = 62,
+ [1][0][RTW89_ETSI][25] = 34,
+ [1][0][RTW89_MKK][25] = 58,
+ [1][0][RTW89_IC][25] = 127,
+ [1][0][RTW89_KCC][25] = 54,
+ [1][0][RTW89_ACMA][25] = 127,
+ [1][0][RTW89_CN][25] = 127,
+ [1][0][RTW89_UK][25] = 34,
+ [1][0][RTW89_FCC][27] = 62,
+ [1][0][RTW89_ETSI][27] = 34,
+ [1][0][RTW89_MKK][27] = 58,
+ [1][0][RTW89_IC][27] = 127,
+ [1][0][RTW89_KCC][27] = 54,
+ [1][0][RTW89_ACMA][27] = 127,
+ [1][0][RTW89_CN][27] = 127,
+ [1][0][RTW89_UK][27] = 34,
+ [1][0][RTW89_FCC][29] = 62,
+ [1][0][RTW89_ETSI][29] = 34,
+ [1][0][RTW89_MKK][29] = 58,
+ [1][0][RTW89_IC][29] = 127,
+ [1][0][RTW89_KCC][29] = 54,
+ [1][0][RTW89_ACMA][29] = 127,
+ [1][0][RTW89_CN][29] = 127,
+ [1][0][RTW89_UK][29] = 34,
+ [1][0][RTW89_FCC][31] = 62,
+ [1][0][RTW89_ETSI][31] = 34,
+ [1][0][RTW89_MKK][31] = 58,
+ [1][0][RTW89_IC][31] = 64,
+ [1][0][RTW89_KCC][31] = 54,
+ [1][0][RTW89_ACMA][31] = 34,
+ [1][0][RTW89_CN][31] = 127,
+ [1][0][RTW89_UK][31] = 34,
+ [1][0][RTW89_FCC][33] = 64,
+ [1][0][RTW89_ETSI][33] = 34,
+ [1][0][RTW89_MKK][33] = 58,
+ [1][0][RTW89_IC][33] = 64,
+ [1][0][RTW89_KCC][33] = 54,
+ [1][0][RTW89_ACMA][33] = 34,
+ [1][0][RTW89_CN][33] = 127,
+ [1][0][RTW89_UK][33] = 34,
+ [1][0][RTW89_FCC][35] = 64,
+ [1][0][RTW89_ETSI][35] = 34,
+ [1][0][RTW89_MKK][35] = 58,
+ [1][0][RTW89_IC][35] = 64,
+ [1][0][RTW89_KCC][35] = 54,
+ [1][0][RTW89_ACMA][35] = 34,
+ [1][0][RTW89_CN][35] = 127,
+ [1][0][RTW89_UK][35] = 34,
+ [1][0][RTW89_FCC][37] = 76,
+ [1][0][RTW89_ETSI][37] = 127,
+ [1][0][RTW89_MKK][37] = 56,
+ [1][0][RTW89_IC][37] = 76,
+ [1][0][RTW89_KCC][37] = 54,
+ [1][0][RTW89_ACMA][37] = 62,
+ [1][0][RTW89_CN][37] = 127,
+ [1][0][RTW89_UK][37] = 62,
+ [1][0][RTW89_FCC][38] = 82,
+ [1][0][RTW89_ETSI][38] = 28,
+ [1][0][RTW89_MKK][38] = 127,
+ [1][0][RTW89_IC][38] = 82,
+ [1][0][RTW89_KCC][38] = 54,
+ [1][0][RTW89_ACMA][38] = 84,
+ [1][0][RTW89_CN][38] = 66,
+ [1][0][RTW89_UK][38] = 34,
+ [1][0][RTW89_FCC][40] = 82,
+ [1][0][RTW89_ETSI][40] = 28,
+ [1][0][RTW89_MKK][40] = 127,
+ [1][0][RTW89_IC][40] = 82,
+ [1][0][RTW89_KCC][40] = 54,
+ [1][0][RTW89_ACMA][40] = 84,
+ [1][0][RTW89_CN][40] = 66,
+ [1][0][RTW89_UK][40] = 34,
+ [1][0][RTW89_FCC][42] = 78,
+ [1][0][RTW89_ETSI][42] = 28,
+ [1][0][RTW89_MKK][42] = 127,
+ [1][0][RTW89_IC][42] = 78,
+ [1][0][RTW89_KCC][42] = 54,
+ [1][0][RTW89_ACMA][42] = 84,
+ [1][0][RTW89_CN][42] = 66,
+ [1][0][RTW89_UK][42] = 34,
+ [1][0][RTW89_FCC][44] = 82,
+ [1][0][RTW89_ETSI][44] = 28,
+ [1][0][RTW89_MKK][44] = 127,
+ [1][0][RTW89_IC][44] = 82,
+ [1][0][RTW89_KCC][44] = 54,
+ [1][0][RTW89_ACMA][44] = 84,
+ [1][0][RTW89_CN][44] = 66,
+ [1][0][RTW89_UK][44] = 34,
+ [1][0][RTW89_FCC][46] = 82,
+ [1][0][RTW89_ETSI][46] = 28,
+ [1][0][RTW89_MKK][46] = 127,
+ [1][0][RTW89_IC][46] = 82,
+ [1][0][RTW89_KCC][46] = 54,
+ [1][0][RTW89_ACMA][46] = 84,
+ [1][0][RTW89_CN][46] = 66,
+ [1][0][RTW89_UK][46] = 34,
+ [1][0][RTW89_FCC][48] = 52,
+ [1][0][RTW89_ETSI][48] = 127,
+ [1][0][RTW89_MKK][48] = 127,
+ [1][0][RTW89_IC][48] = 127,
+ [1][0][RTW89_KCC][48] = 127,
+ [1][0][RTW89_ACMA][48] = 127,
+ [1][0][RTW89_CN][48] = 127,
+ [1][0][RTW89_UK][48] = 127,
+ [1][0][RTW89_FCC][50] = 52,
+ [1][0][RTW89_ETSI][50] = 127,
+ [1][0][RTW89_MKK][50] = 127,
+ [1][0][RTW89_IC][50] = 127,
+ [1][0][RTW89_KCC][50] = 127,
+ [1][0][RTW89_ACMA][50] = 127,
+ [1][0][RTW89_CN][50] = 127,
+ [1][0][RTW89_UK][50] = 127,
+ [1][0][RTW89_FCC][52] = 50,
+ [1][0][RTW89_ETSI][52] = 127,
+ [1][0][RTW89_MKK][52] = 127,
+ [1][0][RTW89_IC][52] = 127,
+ [1][0][RTW89_KCC][52] = 127,
+ [1][0][RTW89_ACMA][52] = 127,
+ [1][0][RTW89_CN][52] = 127,
+ [1][0][RTW89_UK][52] = 127,
+ [1][1][RTW89_FCC][0] = 127,
+ [1][1][RTW89_ETSI][0] = 127,
+ [1][1][RTW89_MKK][0] = 127,
+ [1][1][RTW89_IC][0] = 127,
+ [1][1][RTW89_KCC][0] = 127,
+ [1][1][RTW89_ACMA][0] = 127,
+ [1][1][RTW89_CN][0] = 14,
+ [1][1][RTW89_UK][0] = 127,
+ [1][1][RTW89_FCC][2] = 127,
+ [1][1][RTW89_ETSI][2] = 127,
+ [1][1][RTW89_MKK][2] = 127,
+ [1][1][RTW89_IC][2] = 127,
+ [1][1][RTW89_KCC][2] = 127,
+ [1][1][RTW89_ACMA][2] = 127,
+ [1][1][RTW89_CN][2] = 14,
+ [1][1][RTW89_UK][2] = 127,
+ [1][1][RTW89_FCC][4] = 127,
+ [1][1][RTW89_ETSI][4] = 127,
+ [1][1][RTW89_MKK][4] = 127,
+ [1][1][RTW89_IC][4] = 127,
+ [1][1][RTW89_KCC][4] = 127,
+ [1][1][RTW89_ACMA][4] = 127,
+ [1][1][RTW89_CN][4] = 14,
+ [1][1][RTW89_UK][4] = 127,
+ [1][1][RTW89_FCC][6] = 127,
+ [1][1][RTW89_ETSI][6] = 127,
+ [1][1][RTW89_MKK][6] = 127,
+ [1][1][RTW89_IC][6] = 127,
+ [1][1][RTW89_KCC][6] = 127,
+ [1][1][RTW89_ACMA][6] = 127,
+ [1][1][RTW89_CN][6] = 14,
+ [1][1][RTW89_UK][6] = 127,
+ [1][1][RTW89_FCC][8] = 127,
+ [1][1][RTW89_ETSI][8] = 127,
+ [1][1][RTW89_MKK][8] = 127,
+ [1][1][RTW89_IC][8] = 127,
+ [1][1][RTW89_KCC][8] = 127,
+ [1][1][RTW89_ACMA][8] = 127,
+ [1][1][RTW89_CN][8] = 14,
+ [1][1][RTW89_UK][8] = 127,
+ [1][1][RTW89_FCC][10] = 127,
+ [1][1][RTW89_ETSI][10] = 127,
+ [1][1][RTW89_MKK][10] = 127,
+ [1][1][RTW89_IC][10] = 127,
+ [1][1][RTW89_KCC][10] = 127,
+ [1][1][RTW89_ACMA][10] = 127,
+ [1][1][RTW89_CN][10] = 14,
+ [1][1][RTW89_UK][10] = 127,
+ [1][1][RTW89_FCC][12] = 127,
+ [1][1][RTW89_ETSI][12] = 127,
+ [1][1][RTW89_MKK][12] = 127,
+ [1][1][RTW89_IC][12] = 127,
+ [1][1][RTW89_KCC][12] = 127,
+ [1][1][RTW89_ACMA][12] = 127,
+ [1][1][RTW89_CN][12] = 14,
+ [1][1][RTW89_UK][12] = 127,
+ [1][1][RTW89_FCC][14] = 127,
+ [1][1][RTW89_ETSI][14] = 127,
+ [1][1][RTW89_MKK][14] = 127,
+ [1][1][RTW89_IC][14] = 127,
+ [1][1][RTW89_KCC][14] = 127,
+ [1][1][RTW89_ACMA][14] = 127,
+ [1][1][RTW89_CN][14] = 14,
+ [1][1][RTW89_UK][14] = 127,
+ [1][1][RTW89_FCC][15] = 127,
+ [1][1][RTW89_ETSI][15] = 127,
+ [1][1][RTW89_MKK][15] = 127,
+ [1][1][RTW89_IC][15] = 127,
+ [1][1][RTW89_KCC][15] = 127,
+ [1][1][RTW89_ACMA][15] = 127,
+ [1][1][RTW89_CN][15] = 127,
+ [1][1][RTW89_UK][15] = 127,
+ [1][1][RTW89_FCC][17] = 127,
+ [1][1][RTW89_ETSI][17] = 127,
+ [1][1][RTW89_MKK][17] = 127,
+ [1][1][RTW89_IC][17] = 127,
+ [1][1][RTW89_KCC][17] = 127,
+ [1][1][RTW89_ACMA][17] = 127,
+ [1][1][RTW89_CN][17] = 127,
+ [1][1][RTW89_UK][17] = 127,
+ [1][1][RTW89_FCC][19] = 127,
+ [1][1][RTW89_ETSI][19] = 127,
+ [1][1][RTW89_MKK][19] = 127,
+ [1][1][RTW89_IC][19] = 127,
+ [1][1][RTW89_KCC][19] = 127,
+ [1][1][RTW89_ACMA][19] = 127,
+ [1][1][RTW89_CN][19] = 127,
+ [1][1][RTW89_UK][19] = 127,
+ [1][1][RTW89_FCC][21] = 127,
+ [1][1][RTW89_ETSI][21] = 127,
+ [1][1][RTW89_MKK][21] = 127,
+ [1][1][RTW89_IC][21] = 127,
+ [1][1][RTW89_KCC][21] = 127,
+ [1][1][RTW89_ACMA][21] = 127,
+ [1][1][RTW89_CN][21] = 127,
+ [1][1][RTW89_UK][21] = 127,
+ [1][1][RTW89_FCC][23] = 127,
+ [1][1][RTW89_ETSI][23] = 127,
+ [1][1][RTW89_MKK][23] = 127,
+ [1][1][RTW89_IC][23] = 127,
+ [1][1][RTW89_KCC][23] = 127,
+ [1][1][RTW89_ACMA][23] = 127,
+ [1][1][RTW89_CN][23] = 127,
+ [1][1][RTW89_UK][23] = 127,
+ [1][1][RTW89_FCC][25] = 127,
+ [1][1][RTW89_ETSI][25] = 127,
+ [1][1][RTW89_MKK][25] = 127,
+ [1][1][RTW89_IC][25] = 127,
+ [1][1][RTW89_KCC][25] = 127,
+ [1][1][RTW89_ACMA][25] = 127,
+ [1][1][RTW89_CN][25] = 127,
+ [1][1][RTW89_UK][25] = 127,
+ [1][1][RTW89_FCC][27] = 127,
+ [1][1][RTW89_ETSI][27] = 127,
+ [1][1][RTW89_MKK][27] = 127,
+ [1][1][RTW89_IC][27] = 127,
+ [1][1][RTW89_KCC][27] = 127,
+ [1][1][RTW89_ACMA][27] = 127,
+ [1][1][RTW89_CN][27] = 127,
+ [1][1][RTW89_UK][27] = 127,
+ [1][1][RTW89_FCC][29] = 127,
+ [1][1][RTW89_ETSI][29] = 127,
+ [1][1][RTW89_MKK][29] = 127,
+ [1][1][RTW89_IC][29] = 127,
+ [1][1][RTW89_KCC][29] = 127,
+ [1][1][RTW89_ACMA][29] = 127,
+ [1][1][RTW89_CN][29] = 127,
+ [1][1][RTW89_UK][29] = 127,
+ [1][1][RTW89_FCC][31] = 127,
+ [1][1][RTW89_ETSI][31] = 127,
+ [1][1][RTW89_MKK][31] = 127,
+ [1][1][RTW89_IC][31] = 127,
+ [1][1][RTW89_KCC][31] = 127,
+ [1][1][RTW89_ACMA][31] = 127,
+ [1][1][RTW89_CN][31] = 127,
+ [1][1][RTW89_UK][31] = 127,
+ [1][1][RTW89_FCC][33] = 127,
+ [1][1][RTW89_ETSI][33] = 127,
+ [1][1][RTW89_MKK][33] = 127,
+ [1][1][RTW89_IC][33] = 127,
+ [1][1][RTW89_KCC][33] = 127,
+ [1][1][RTW89_ACMA][33] = 127,
+ [1][1][RTW89_CN][33] = 127,
+ [1][1][RTW89_UK][33] = 127,
+ [1][1][RTW89_FCC][35] = 127,
+ [1][1][RTW89_ETSI][35] = 127,
+ [1][1][RTW89_MKK][35] = 127,
+ [1][1][RTW89_IC][35] = 127,
+ [1][1][RTW89_KCC][35] = 127,
+ [1][1][RTW89_ACMA][35] = 127,
+ [1][1][RTW89_CN][35] = 127,
+ [1][1][RTW89_UK][35] = 127,
+ [1][1][RTW89_FCC][37] = 127,
+ [1][1][RTW89_ETSI][37] = 127,
+ [1][1][RTW89_MKK][37] = 127,
+ [1][1][RTW89_IC][37] = 127,
+ [1][1][RTW89_KCC][37] = 127,
+ [1][1][RTW89_ACMA][37] = 127,
+ [1][1][RTW89_CN][37] = 127,
+ [1][1][RTW89_UK][37] = 127,
+ [1][1][RTW89_FCC][38] = 127,
+ [1][1][RTW89_ETSI][38] = 127,
+ [1][1][RTW89_MKK][38] = 127,
+ [1][1][RTW89_IC][38] = 127,
+ [1][1][RTW89_KCC][38] = 127,
+ [1][1][RTW89_ACMA][38] = 127,
+ [1][1][RTW89_CN][38] = 54,
+ [1][1][RTW89_UK][38] = 127,
+ [1][1][RTW89_FCC][40] = 127,
+ [1][1][RTW89_ETSI][40] = 127,
+ [1][1][RTW89_MKK][40] = 127,
+ [1][1][RTW89_IC][40] = 127,
+ [1][1][RTW89_KCC][40] = 127,
+ [1][1][RTW89_ACMA][40] = 127,
+ [1][1][RTW89_CN][40] = 54,
+ [1][1][RTW89_UK][40] = 127,
+ [1][1][RTW89_FCC][42] = 127,
+ [1][1][RTW89_ETSI][42] = 127,
+ [1][1][RTW89_MKK][42] = 127,
+ [1][1][RTW89_IC][42] = 127,
+ [1][1][RTW89_KCC][42] = 127,
+ [1][1][RTW89_ACMA][42] = 127,
+ [1][1][RTW89_CN][42] = 54,
+ [1][1][RTW89_UK][42] = 127,
+ [1][1][RTW89_FCC][44] = 127,
+ [1][1][RTW89_ETSI][44] = 127,
+ [1][1][RTW89_MKK][44] = 127,
+ [1][1][RTW89_IC][44] = 127,
+ [1][1][RTW89_KCC][44] = 127,
+ [1][1][RTW89_ACMA][44] = 127,
+ [1][1][RTW89_CN][44] = 54,
+ [1][1][RTW89_UK][44] = 127,
+ [1][1][RTW89_FCC][46] = 127,
+ [1][1][RTW89_ETSI][46] = 127,
+ [1][1][RTW89_MKK][46] = 127,
+ [1][1][RTW89_IC][46] = 127,
+ [1][1][RTW89_KCC][46] = 127,
+ [1][1][RTW89_ACMA][46] = 127,
+ [1][1][RTW89_CN][46] = 54,
+ [1][1][RTW89_UK][46] = 127,
+ [1][1][RTW89_FCC][48] = 127,
+ [1][1][RTW89_ETSI][48] = 127,
+ [1][1][RTW89_MKK][48] = 127,
+ [1][1][RTW89_IC][48] = 127,
+ [1][1][RTW89_KCC][48] = 127,
+ [1][1][RTW89_ACMA][48] = 127,
+ [1][1][RTW89_CN][48] = 127,
+ [1][1][RTW89_UK][48] = 127,
+ [1][1][RTW89_FCC][50] = 127,
+ [1][1][RTW89_ETSI][50] = 127,
+ [1][1][RTW89_MKK][50] = 127,
+ [1][1][RTW89_IC][50] = 127,
+ [1][1][RTW89_KCC][50] = 127,
+ [1][1][RTW89_ACMA][50] = 127,
+ [1][1][RTW89_CN][50] = 127,
+ [1][1][RTW89_UK][50] = 127,
+ [1][1][RTW89_FCC][52] = 127,
+ [1][1][RTW89_ETSI][52] = 127,
+ [1][1][RTW89_MKK][52] = 127,
+ [1][1][RTW89_IC][52] = 127,
+ [1][1][RTW89_KCC][52] = 127,
+ [1][1][RTW89_ACMA][52] = 127,
+ [1][1][RTW89_CN][52] = 127,
+ [1][1][RTW89_UK][52] = 127,
+ [2][0][RTW89_FCC][0] = 76,
+ [2][0][RTW89_ETSI][0] = 46,
+ [2][0][RTW89_MKK][0] = 48,
+ [2][0][RTW89_IC][0] = 50,
+ [2][0][RTW89_KCC][0] = 64,
+ [2][0][RTW89_ACMA][0] = 46,
+ [2][0][RTW89_CN][0] = 40,
+ [2][0][RTW89_UK][0] = 46,
+ [2][0][RTW89_FCC][2] = 72,
+ [2][0][RTW89_ETSI][2] = 46,
+ [2][0][RTW89_MKK][2] = 48,
+ [2][0][RTW89_IC][2] = 48,
+ [2][0][RTW89_KCC][2] = 64,
+ [2][0][RTW89_ACMA][2] = 46,
+ [2][0][RTW89_CN][2] = 40,
+ [2][0][RTW89_UK][2] = 46,
+ [2][0][RTW89_FCC][4] = 74,
+ [2][0][RTW89_ETSI][4] = 46,
+ [2][0][RTW89_MKK][4] = 48,
+ [2][0][RTW89_IC][4] = 48,
+ [2][0][RTW89_KCC][4] = 64,
+ [2][0][RTW89_ACMA][4] = 46,
+ [2][0][RTW89_CN][4] = 40,
+ [2][0][RTW89_UK][4] = 46,
+ [2][0][RTW89_FCC][6] = 74,
+ [2][0][RTW89_ETSI][6] = 46,
+ [2][0][RTW89_MKK][6] = 48,
+ [2][0][RTW89_IC][6] = 48,
+ [2][0][RTW89_KCC][6] = 40,
+ [2][0][RTW89_ACMA][6] = 46,
+ [2][0][RTW89_CN][6] = 40,
+ [2][0][RTW89_UK][6] = 46,
+ [2][0][RTW89_FCC][8] = 72,
+ [2][0][RTW89_ETSI][8] = 46,
+ [2][0][RTW89_MKK][8] = 48,
+ [2][0][RTW89_IC][8] = 64,
+ [2][0][RTW89_KCC][8] = 66,
+ [2][0][RTW89_ACMA][8] = 46,
+ [2][0][RTW89_CN][8] = 40,
+ [2][0][RTW89_UK][8] = 46,
+ [2][0][RTW89_FCC][10] = 72,
+ [2][0][RTW89_ETSI][10] = 46,
+ [2][0][RTW89_MKK][10] = 48,
+ [2][0][RTW89_IC][10] = 64,
+ [2][0][RTW89_KCC][10] = 66,
+ [2][0][RTW89_ACMA][10] = 46,
+ [2][0][RTW89_CN][10] = 40,
+ [2][0][RTW89_UK][10] = 46,
+ [2][0][RTW89_FCC][12] = 74,
+ [2][0][RTW89_ETSI][12] = 46,
+ [2][0][RTW89_MKK][12] = 48,
+ [2][0][RTW89_IC][12] = 64,
+ [2][0][RTW89_KCC][12] = 64,
+ [2][0][RTW89_ACMA][12] = 46,
+ [2][0][RTW89_CN][12] = 40,
+ [2][0][RTW89_UK][12] = 46,
+ [2][0][RTW89_FCC][14] = 80,
+ [2][0][RTW89_ETSI][14] = 46,
+ [2][0][RTW89_MKK][14] = 48,
+ [2][0][RTW89_IC][14] = 64,
+ [2][0][RTW89_KCC][14] = 64,
+ [2][0][RTW89_ACMA][14] = 46,
+ [2][0][RTW89_CN][14] = 40,
+ [2][0][RTW89_UK][14] = 46,
+ [2][0][RTW89_FCC][15] = 72,
+ [2][0][RTW89_ETSI][15] = 46,
+ [2][0][RTW89_MKK][15] = 70,
+ [2][0][RTW89_IC][15] = 72,
+ [2][0][RTW89_KCC][15] = 66,
+ [2][0][RTW89_ACMA][15] = 46,
+ [2][0][RTW89_CN][15] = 127,
+ [2][0][RTW89_UK][15] = 46,
+ [2][0][RTW89_FCC][17] = 72,
+ [2][0][RTW89_ETSI][17] = 46,
+ [2][0][RTW89_MKK][17] = 70,
+ [2][0][RTW89_IC][17] = 72,
+ [2][0][RTW89_KCC][17] = 66,
+ [2][0][RTW89_ACMA][17] = 46,
+ [2][0][RTW89_CN][17] = 127,
+ [2][0][RTW89_UK][17] = 46,
+ [2][0][RTW89_FCC][19] = 68,
+ [2][0][RTW89_ETSI][19] = 46,
+ [2][0][RTW89_MKK][19] = 70,
+ [2][0][RTW89_IC][19] = 68,
+ [2][0][RTW89_KCC][19] = 66,
+ [2][0][RTW89_ACMA][19] = 46,
+ [2][0][RTW89_CN][19] = 127,
+ [2][0][RTW89_UK][19] = 46,
+ [2][0][RTW89_FCC][21] = 68,
+ [2][0][RTW89_ETSI][21] = 46,
+ [2][0][RTW89_MKK][21] = 70,
+ [2][0][RTW89_IC][21] = 68,
+ [2][0][RTW89_KCC][21] = 66,
+ [2][0][RTW89_ACMA][21] = 46,
+ [2][0][RTW89_CN][21] = 127,
+ [2][0][RTW89_UK][21] = 46,
+ [2][0][RTW89_FCC][23] = 68,
+ [2][0][RTW89_ETSI][23] = 46,
+ [2][0][RTW89_MKK][23] = 70,
+ [2][0][RTW89_IC][23] = 68,
+ [2][0][RTW89_KCC][23] = 66,
+ [2][0][RTW89_ACMA][23] = 46,
+ [2][0][RTW89_CN][23] = 127,
+ [2][0][RTW89_UK][23] = 46,
+ [2][0][RTW89_FCC][25] = 68,
+ [2][0][RTW89_ETSI][25] = 46,
+ [2][0][RTW89_MKK][25] = 70,
+ [2][0][RTW89_IC][25] = 127,
+ [2][0][RTW89_KCC][25] = 66,
+ [2][0][RTW89_ACMA][25] = 127,
+ [2][0][RTW89_CN][25] = 127,
+ [2][0][RTW89_UK][25] = 46,
+ [2][0][RTW89_FCC][27] = 68,
+ [2][0][RTW89_ETSI][27] = 46,
+ [2][0][RTW89_MKK][27] = 70,
+ [2][0][RTW89_IC][27] = 127,
+ [2][0][RTW89_KCC][27] = 64,
+ [2][0][RTW89_ACMA][27] = 127,
+ [2][0][RTW89_CN][27] = 127,
+ [2][0][RTW89_UK][27] = 46,
+ [2][0][RTW89_FCC][29] = 68,
+ [2][0][RTW89_ETSI][29] = 46,
+ [2][0][RTW89_MKK][29] = 70,
+ [2][0][RTW89_IC][29] = 127,
+ [2][0][RTW89_KCC][29] = 64,
+ [2][0][RTW89_ACMA][29] = 127,
+ [2][0][RTW89_CN][29] = 127,
+ [2][0][RTW89_UK][29] = 46,
+ [2][0][RTW89_FCC][31] = 68,
+ [2][0][RTW89_ETSI][31] = 46,
+ [2][0][RTW89_MKK][31] = 70,
+ [2][0][RTW89_IC][31] = 70,
+ [2][0][RTW89_KCC][31] = 64,
+ [2][0][RTW89_ACMA][31] = 46,
+ [2][0][RTW89_CN][31] = 127,
+ [2][0][RTW89_UK][31] = 46,
+ [2][0][RTW89_FCC][33] = 70,
+ [2][0][RTW89_ETSI][33] = 46,
+ [2][0][RTW89_MKK][33] = 70,
+ [2][0][RTW89_IC][33] = 70,
+ [2][0][RTW89_KCC][33] = 64,
+ [2][0][RTW89_ACMA][33] = 46,
+ [2][0][RTW89_CN][33] = 127,
+ [2][0][RTW89_UK][33] = 46,
+ [2][0][RTW89_FCC][35] = 70,
+ [2][0][RTW89_ETSI][35] = 46,
+ [2][0][RTW89_MKK][35] = 70,
+ [2][0][RTW89_IC][35] = 70,
+ [2][0][RTW89_KCC][35] = 64,
+ [2][0][RTW89_ACMA][35] = 46,
+ [2][0][RTW89_CN][35] = 127,
+ [2][0][RTW89_UK][35] = 46,
+ [2][0][RTW89_FCC][37] = 84,
+ [2][0][RTW89_ETSI][37] = 127,
+ [2][0][RTW89_MKK][37] = 68,
+ [2][0][RTW89_IC][37] = 84,
+ [2][0][RTW89_KCC][37] = 66,
+ [2][0][RTW89_ACMA][37] = 74,
+ [2][0][RTW89_CN][37] = 127,
+ [2][0][RTW89_UK][37] = 74,
+ [2][0][RTW89_FCC][38] = 84,
+ [2][0][RTW89_ETSI][38] = 28,
+ [2][0][RTW89_MKK][38] = 127,
+ [2][0][RTW89_IC][38] = 84,
+ [2][0][RTW89_KCC][38] = 64,
+ [2][0][RTW89_ACMA][38] = 84,
+ [2][0][RTW89_CN][38] = 68,
+ [2][0][RTW89_UK][38] = 46,
+ [2][0][RTW89_FCC][40] = 84,
+ [2][0][RTW89_ETSI][40] = 28,
+ [2][0][RTW89_MKK][40] = 127,
+ [2][0][RTW89_IC][40] = 84,
+ [2][0][RTW89_KCC][40] = 64,
+ [2][0][RTW89_ACMA][40] = 84,
+ [2][0][RTW89_CN][40] = 68,
+ [2][0][RTW89_UK][40] = 46,
+ [2][0][RTW89_FCC][42] = 78,
+ [2][0][RTW89_ETSI][42] = 28,
+ [2][0][RTW89_MKK][42] = 127,
+ [2][0][RTW89_IC][42] = 78,
+ [2][0][RTW89_KCC][42] = 66,
+ [2][0][RTW89_ACMA][42] = 84,
+ [2][0][RTW89_CN][42] = 68,
+ [2][0][RTW89_UK][42] = 46,
+ [2][0][RTW89_FCC][44] = 80,
+ [2][0][RTW89_ETSI][44] = 28,
+ [2][0][RTW89_MKK][44] = 127,
+ [2][0][RTW89_IC][44] = 80,
+ [2][0][RTW89_KCC][44] = 66,
+ [2][0][RTW89_ACMA][44] = 84,
+ [2][0][RTW89_CN][44] = 68,
+ [2][0][RTW89_UK][44] = 46,
+ [2][0][RTW89_FCC][46] = 80,
+ [2][0][RTW89_ETSI][46] = 28,
+ [2][0][RTW89_MKK][46] = 127,
+ [2][0][RTW89_IC][46] = 80,
+ [2][0][RTW89_KCC][46] = 66,
+ [2][0][RTW89_ACMA][46] = 84,
+ [2][0][RTW89_CN][46] = 68,
+ [2][0][RTW89_UK][46] = 46,
+ [2][0][RTW89_FCC][48] = 62,
+ [2][0][RTW89_ETSI][48] = 127,
+ [2][0][RTW89_MKK][48] = 127,
+ [2][0][RTW89_IC][48] = 127,
+ [2][0][RTW89_KCC][48] = 127,
+ [2][0][RTW89_ACMA][48] = 127,
+ [2][0][RTW89_CN][48] = 127,
+ [2][0][RTW89_UK][48] = 127,
+ [2][0][RTW89_FCC][50] = 62,
+ [2][0][RTW89_ETSI][50] = 127,
+ [2][0][RTW89_MKK][50] = 127,
+ [2][0][RTW89_IC][50] = 127,
+ [2][0][RTW89_KCC][50] = 127,
+ [2][0][RTW89_ACMA][50] = 127,
+ [2][0][RTW89_CN][50] = 127,
+ [2][0][RTW89_UK][50] = 127,
+ [2][0][RTW89_FCC][52] = 60,
+ [2][0][RTW89_ETSI][52] = 127,
+ [2][0][RTW89_MKK][52] = 127,
+ [2][0][RTW89_IC][52] = 127,
+ [2][0][RTW89_KCC][52] = 127,
+ [2][0][RTW89_ACMA][52] = 127,
+ [2][0][RTW89_CN][52] = 127,
+ [2][0][RTW89_UK][52] = 127,
+ [2][1][RTW89_FCC][0] = 127,
+ [2][1][RTW89_ETSI][0] = 127,
+ [2][1][RTW89_MKK][0] = 127,
+ [2][1][RTW89_IC][0] = 127,
+ [2][1][RTW89_KCC][0] = 127,
+ [2][1][RTW89_ACMA][0] = 127,
+ [2][1][RTW89_CN][0] = 28,
+ [2][1][RTW89_UK][0] = 127,
+ [2][1][RTW89_FCC][2] = 127,
+ [2][1][RTW89_ETSI][2] = 127,
+ [2][1][RTW89_MKK][2] = 127,
+ [2][1][RTW89_IC][2] = 127,
+ [2][1][RTW89_KCC][2] = 127,
+ [2][1][RTW89_ACMA][2] = 127,
+ [2][1][RTW89_CN][2] = 28,
+ [2][1][RTW89_UK][2] = 127,
+ [2][1][RTW89_FCC][4] = 127,
+ [2][1][RTW89_ETSI][4] = 127,
+ [2][1][RTW89_MKK][4] = 127,
+ [2][1][RTW89_IC][4] = 127,
+ [2][1][RTW89_KCC][4] = 127,
+ [2][1][RTW89_ACMA][4] = 127,
+ [2][1][RTW89_CN][4] = 28,
+ [2][1][RTW89_UK][4] = 127,
+ [2][1][RTW89_FCC][6] = 127,
+ [2][1][RTW89_ETSI][6] = 127,
+ [2][1][RTW89_MKK][6] = 127,
+ [2][1][RTW89_IC][6] = 127,
+ [2][1][RTW89_KCC][6] = 127,
+ [2][1][RTW89_ACMA][6] = 127,
+ [2][1][RTW89_CN][6] = 28,
+ [2][1][RTW89_UK][6] = 127,
+ [2][1][RTW89_FCC][8] = 127,
+ [2][1][RTW89_ETSI][8] = 127,
+ [2][1][RTW89_MKK][8] = 127,
+ [2][1][RTW89_IC][8] = 127,
+ [2][1][RTW89_KCC][8] = 127,
+ [2][1][RTW89_ACMA][8] = 127,
+ [2][1][RTW89_CN][8] = 28,
+ [2][1][RTW89_UK][8] = 127,
+ [2][1][RTW89_FCC][10] = 127,
+ [2][1][RTW89_ETSI][10] = 127,
+ [2][1][RTW89_MKK][10] = 127,
+ [2][1][RTW89_IC][10] = 127,
+ [2][1][RTW89_KCC][10] = 127,
+ [2][1][RTW89_ACMA][10] = 127,
+ [2][1][RTW89_CN][10] = 28,
+ [2][1][RTW89_UK][10] = 127,
+ [2][1][RTW89_FCC][12] = 127,
+ [2][1][RTW89_ETSI][12] = 127,
+ [2][1][RTW89_MKK][12] = 127,
+ [2][1][RTW89_IC][12] = 127,
+ [2][1][RTW89_KCC][12] = 127,
+ [2][1][RTW89_ACMA][12] = 127,
+ [2][1][RTW89_CN][12] = 28,
+ [2][1][RTW89_UK][12] = 127,
+ [2][1][RTW89_FCC][14] = 127,
+ [2][1][RTW89_ETSI][14] = 127,
+ [2][1][RTW89_MKK][14] = 127,
+ [2][1][RTW89_IC][14] = 127,
+ [2][1][RTW89_KCC][14] = 127,
+ [2][1][RTW89_ACMA][14] = 127,
+ [2][1][RTW89_CN][14] = 28,
+ [2][1][RTW89_UK][14] = 127,
+ [2][1][RTW89_FCC][15] = 127,
+ [2][1][RTW89_ETSI][15] = 127,
+ [2][1][RTW89_MKK][15] = 127,
+ [2][1][RTW89_IC][15] = 127,
+ [2][1][RTW89_KCC][15] = 127,
+ [2][1][RTW89_ACMA][15] = 127,
+ [2][1][RTW89_CN][15] = 127,
+ [2][1][RTW89_UK][15] = 127,
+ [2][1][RTW89_FCC][17] = 127,
+ [2][1][RTW89_ETSI][17] = 127,
+ [2][1][RTW89_MKK][17] = 127,
+ [2][1][RTW89_IC][17] = 127,
+ [2][1][RTW89_KCC][17] = 127,
+ [2][1][RTW89_ACMA][17] = 127,
+ [2][1][RTW89_CN][17] = 127,
+ [2][1][RTW89_UK][17] = 127,
+ [2][1][RTW89_FCC][19] = 127,
+ [2][1][RTW89_ETSI][19] = 127,
+ [2][1][RTW89_MKK][19] = 127,
+ [2][1][RTW89_IC][19] = 127,
+ [2][1][RTW89_KCC][19] = 127,
+ [2][1][RTW89_ACMA][19] = 127,
+ [2][1][RTW89_CN][19] = 127,
+ [2][1][RTW89_UK][19] = 127,
+ [2][1][RTW89_FCC][21] = 127,
+ [2][1][RTW89_ETSI][21] = 127,
+ [2][1][RTW89_MKK][21] = 127,
+ [2][1][RTW89_IC][21] = 127,
+ [2][1][RTW89_KCC][21] = 127,
+ [2][1][RTW89_ACMA][21] = 127,
+ [2][1][RTW89_CN][21] = 127,
+ [2][1][RTW89_UK][21] = 127,
+ [2][1][RTW89_FCC][23] = 127,
+ [2][1][RTW89_ETSI][23] = 127,
+ [2][1][RTW89_MKK][23] = 127,
+ [2][1][RTW89_IC][23] = 127,
+ [2][1][RTW89_KCC][23] = 127,
+ [2][1][RTW89_ACMA][23] = 127,
+ [2][1][RTW89_CN][23] = 127,
+ [2][1][RTW89_UK][23] = 127,
+ [2][1][RTW89_FCC][25] = 127,
+ [2][1][RTW89_ETSI][25] = 127,
+ [2][1][RTW89_MKK][25] = 127,
+ [2][1][RTW89_IC][25] = 127,
+ [2][1][RTW89_KCC][25] = 127,
+ [2][1][RTW89_ACMA][25] = 127,
+ [2][1][RTW89_CN][25] = 127,
+ [2][1][RTW89_UK][25] = 127,
+ [2][1][RTW89_FCC][27] = 127,
+ [2][1][RTW89_ETSI][27] = 127,
+ [2][1][RTW89_MKK][27] = 127,
+ [2][1][RTW89_IC][27] = 127,
+ [2][1][RTW89_KCC][27] = 127,
+ [2][1][RTW89_ACMA][27] = 127,
+ [2][1][RTW89_CN][27] = 127,
+ [2][1][RTW89_UK][27] = 127,
+ [2][1][RTW89_FCC][29] = 127,
+ [2][1][RTW89_ETSI][29] = 127,
+ [2][1][RTW89_MKK][29] = 127,
+ [2][1][RTW89_IC][29] = 127,
+ [2][1][RTW89_KCC][29] = 127,
+ [2][1][RTW89_ACMA][29] = 127,
+ [2][1][RTW89_CN][29] = 127,
+ [2][1][RTW89_UK][29] = 127,
+ [2][1][RTW89_FCC][31] = 127,
+ [2][1][RTW89_ETSI][31] = 127,
+ [2][1][RTW89_MKK][31] = 127,
+ [2][1][RTW89_IC][31] = 127,
+ [2][1][RTW89_KCC][31] = 127,
+ [2][1][RTW89_ACMA][31] = 127,
+ [2][1][RTW89_CN][31] = 127,
+ [2][1][RTW89_UK][31] = 127,
+ [2][1][RTW89_FCC][33] = 127,
+ [2][1][RTW89_ETSI][33] = 127,
+ [2][1][RTW89_MKK][33] = 127,
+ [2][1][RTW89_IC][33] = 127,
+ [2][1][RTW89_KCC][33] = 127,
+ [2][1][RTW89_ACMA][33] = 127,
+ [2][1][RTW89_CN][33] = 127,
+ [2][1][RTW89_UK][33] = 127,
+ [2][1][RTW89_FCC][35] = 127,
+ [2][1][RTW89_ETSI][35] = 127,
+ [2][1][RTW89_MKK][35] = 127,
+ [2][1][RTW89_IC][35] = 127,
+ [2][1][RTW89_KCC][35] = 127,
+ [2][1][RTW89_ACMA][35] = 127,
+ [2][1][RTW89_CN][35] = 127,
+ [2][1][RTW89_UK][35] = 127,
+ [2][1][RTW89_FCC][37] = 127,
+ [2][1][RTW89_ETSI][37] = 127,
+ [2][1][RTW89_MKK][37] = 127,
+ [2][1][RTW89_IC][37] = 127,
+ [2][1][RTW89_KCC][37] = 127,
+ [2][1][RTW89_ACMA][37] = 127,
+ [2][1][RTW89_CN][37] = 127,
+ [2][1][RTW89_UK][37] = 127,
+ [2][1][RTW89_FCC][38] = 127,
+ [2][1][RTW89_ETSI][38] = 127,
+ [2][1][RTW89_MKK][38] = 127,
+ [2][1][RTW89_IC][38] = 127,
+ [2][1][RTW89_KCC][38] = 127,
+ [2][1][RTW89_ACMA][38] = 127,
+ [2][1][RTW89_CN][38] = 56,
+ [2][1][RTW89_UK][38] = 127,
+ [2][1][RTW89_FCC][40] = 127,
+ [2][1][RTW89_ETSI][40] = 127,
+ [2][1][RTW89_MKK][40] = 127,
+ [2][1][RTW89_IC][40] = 127,
+ [2][1][RTW89_KCC][40] = 127,
+ [2][1][RTW89_ACMA][40] = 127,
+ [2][1][RTW89_CN][40] = 56,
+ [2][1][RTW89_UK][40] = 127,
+ [2][1][RTW89_FCC][42] = 127,
+ [2][1][RTW89_ETSI][42] = 127,
+ [2][1][RTW89_MKK][42] = 127,
+ [2][1][RTW89_IC][42] = 127,
+ [2][1][RTW89_KCC][42] = 127,
+ [2][1][RTW89_ACMA][42] = 127,
+ [2][1][RTW89_CN][42] = 56,
+ [2][1][RTW89_UK][42] = 127,
+ [2][1][RTW89_FCC][44] = 127,
+ [2][1][RTW89_ETSI][44] = 127,
+ [2][1][RTW89_MKK][44] = 127,
+ [2][1][RTW89_IC][44] = 127,
+ [2][1][RTW89_KCC][44] = 127,
+ [2][1][RTW89_ACMA][44] = 127,
+ [2][1][RTW89_CN][44] = 56,
+ [2][1][RTW89_UK][44] = 127,
+ [2][1][RTW89_FCC][46] = 127,
+ [2][1][RTW89_ETSI][46] = 127,
+ [2][1][RTW89_MKK][46] = 127,
+ [2][1][RTW89_IC][46] = 127,
+ [2][1][RTW89_KCC][46] = 127,
+ [2][1][RTW89_ACMA][46] = 127,
+ [2][1][RTW89_CN][46] = 56,
+ [2][1][RTW89_UK][46] = 127,
+ [2][1][RTW89_FCC][48] = 127,
+ [2][1][RTW89_ETSI][48] = 127,
+ [2][1][RTW89_MKK][48] = 127,
+ [2][1][RTW89_IC][48] = 127,
+ [2][1][RTW89_KCC][48] = 127,
+ [2][1][RTW89_ACMA][48] = 127,
+ [2][1][RTW89_CN][48] = 127,
+ [2][1][RTW89_UK][48] = 127,
+ [2][1][RTW89_FCC][50] = 127,
+ [2][1][RTW89_ETSI][50] = 127,
+ [2][1][RTW89_MKK][50] = 127,
+ [2][1][RTW89_IC][50] = 127,
+ [2][1][RTW89_KCC][50] = 127,
+ [2][1][RTW89_ACMA][50] = 127,
+ [2][1][RTW89_CN][50] = 127,
+ [2][1][RTW89_UK][50] = 127,
+ [2][1][RTW89_FCC][52] = 127,
+ [2][1][RTW89_ETSI][52] = 127,
+ [2][1][RTW89_MKK][52] = 127,
+ [2][1][RTW89_IC][52] = 127,
+ [2][1][RTW89_KCC][52] = 127,
+ [2][1][RTW89_ACMA][52] = 127,
+ [2][1][RTW89_CN][52] = 127,
+ [2][1][RTW89_UK][52] = 127,
+};
+
+const struct rtw89_phy_table rtw89_8851b_phy_bb_table = {
+ .regs = rtw89_8851b_phy_bb_regs,
+ .n_regs = ARRAY_SIZE(rtw89_8851b_phy_bb_regs),
+ .rf_path = 0, /* don't care */
+};
+
+const struct rtw89_phy_table rtw89_8851b_phy_bb_gain_table = {
+ .regs = rtw89_8851b_phy_bb_reg_gain,
+ .n_regs = ARRAY_SIZE(rtw89_8851b_phy_bb_reg_gain),
+ .rf_path = 0, /* don't care */
+};
+
+const struct rtw89_phy_table rtw89_8851b_phy_radioa_table = {
+ .regs = rtw89_8851b_phy_radioa_regs,
+ .n_regs = ARRAY_SIZE(rtw89_8851b_phy_radioa_regs),
+ .rf_path = RF_PATH_A,
+ .config = rtw89_phy_config_rf_reg_v1,
+};
+
+const struct rtw89_phy_table rtw89_8851b_phy_nctl_table = {
+ .regs = rtw89_8851b_phy_nctl_regs,
+ .n_regs = ARRAY_SIZE(rtw89_8851b_phy_nctl_regs),
+ .rf_path = 0, /* don't care */
+};
+
+const struct rtw89_txpwr_table rtw89_8851b_byr_table = {
+ .data = rtw89_8851b_txpwr_byrate,
+ .size = ARRAY_SIZE(rtw89_8851b_txpwr_byrate),
+ .load = rtw89_phy_load_txpwr_byrate,
+};
+
+const struct rtw89_txpwr_track_cfg rtw89_8851b_trk_cfg = {
+ .delta_swingidx_5ga_n = _txpwr_track_delta_swingidx_5ga_n,
+ .delta_swingidx_5ga_p = _txpwr_track_delta_swingidx_5ga_p,
+ .delta_swingidx_2ga_n = _txpwr_track_delta_swingidx_2ga_n,
+ .delta_swingidx_2ga_p = _txpwr_track_delta_swingidx_2ga_p,
+ .delta_swingidx_2g_cck_a_n = _txpwr_track_delta_swingidx_2g_cck_a_n,
+ .delta_swingidx_2g_cck_a_p = _txpwr_track_delta_swingidx_2g_cck_a_p,
+};
+
+const struct rtw89_rfe_parms rtw89_8851b_dflt_parms = {
+ .rule_2ghz = {
+ .lmt = &rtw89_8851b_txpwr_lmt_2g,
+ .lmt_ru = &rtw89_8851b_txpwr_lmt_ru_2g,
+ },
+ .rule_5ghz = {
+ .lmt = &rtw89_8851b_txpwr_lmt_5g,
+ .lmt_ru = &rtw89_8851b_txpwr_lmt_ru_5g,
+ },
+};
+
+static const struct rtw89_rfe_parms rtw89_8851b_rfe_parms_type2 = {
+ .rule_2ghz = {
+ .lmt = &rtw89_8851b_txpwr_lmt_2g_type2,
+ .lmt_ru = &rtw89_8851b_txpwr_lmt_ru_2g_type2,
+ },
+ .rule_5ghz = {
+ .lmt = &rtw89_8851b_txpwr_lmt_5g_type2,
+ .lmt_ru = &rtw89_8851b_txpwr_lmt_ru_5g_type2,
+ },
+};
+
+const struct rtw89_rfe_parms_conf rtw89_8851b_rfe_parms_conf[] = {
+ {
+ .rfe_parms = &rtw89_8851b_rfe_parms_type2,
+ .rfe_type = 2,
+ },
+ {},
+};
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b_table.h b/drivers/net/wireless/realtek/rtw89/rtw8851b_table.h
new file mode 100644
index 0000000..f2e673b
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtw89/rtw8851b_table.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/* Copyright(c) 2022-2023 Realtek Corporation
+ */
+
+#ifndef __RTW89_8851B_TABLE_H__
+#define __RTW89_8851B_TABLE_H__
+
+#include "core.h"
+
+extern const struct rtw89_phy_table rtw89_8851b_phy_bb_table;
+extern const struct rtw89_phy_table rtw89_8851b_phy_bb_gain_table;
+extern const struct rtw89_phy_table rtw89_8851b_phy_radioa_table;
+extern const struct rtw89_phy_table rtw89_8851b_phy_nctl_table;
+extern const struct rtw89_txpwr_table rtw89_8851b_byr_table;
+extern const struct rtw89_txpwr_track_cfg rtw89_8851b_trk_cfg;
+extern const u8 rtw89_8851b_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
+ [RTW89_REGD_NUM];
+extern const struct rtw89_rfe_parms rtw89_8851b_dflt_parms;
+extern const struct rtw89_rfe_parms_conf rtw89_8851b_rfe_parms_conf[];
+
+#endif
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a.c b/drivers/net/wireless/realtek/rtw89/rtw8852a.c
index 4690774..d7930ef 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852a.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852a.c
@@ -12,6 +12,11 @@
#include "rtw8852a_table.h"
#include "txrx.h"
+#define RTW8852A_FW_FORMAT_MAX 0
+#define RTW8852A_FW_BASENAME "rtw89/rtw8852a_fw"
+#define RTW8852A_MODULE_FIRMWARE \
+ RTW8852A_FW_BASENAME ".bin"
+
static const struct rtw89_hfc_ch_cfg rtw8852a_hfc_chcfg_pcie[] = {
{128, 1896, grp_0}, /* ACH 0 */
{128, 1896, grp_0}, /* ACH 1 */
@@ -1827,7 +1832,8 @@ rtw8852a_btc_set_wl_txpwr_ctrl(struct rtw89_dev *rtwdev, u32 txpwr_val)
static
s8 rtw8852a_btc_get_bt_rssi(struct rtw89_dev *rtwdev, s8 val)
{
- return clamp_t(s8, val, -100, 0) + 100;
+ /* +6 for compensate offset */
+ return clamp_t(s8, val + 6, -100, 0) + 100;
}
static struct rtw89_btc_rf_trx_para rtw89_btc_8852a_rf_ul[] = {
@@ -2059,7 +2065,8 @@ static const struct rtw89_chip_ops rtw8852a_chip_ops = {
const struct rtw89_chip_info rtw8852a_chip_info = {
.chip_id = RTL8852A,
.ops = &rtw8852a_chip_ops,
- .fw_name = "rtw89/rtw8852a_fw.bin",
+ .fw_basename = RTW8852A_FW_BASENAME,
+ .fw_format_max = RTW8852A_FW_FORMAT_MAX,
.try_ce_fw = false,
.fifo_size = 458752,
.dle_scc_rsvd_size = 0,
@@ -2079,10 +2086,8 @@ const struct rtw89_chip_info rtw8852a_chip_info = {
&rtw89_8852a_phy_radiob_table,},
.nctl_table = &rtw89_8852a_phy_nctl_table,
.byr_table = &rtw89_8852a_byr_table,
- .txpwr_lmt_2g = &rtw89_8852a_txpwr_lmt_2g,
- .txpwr_lmt_5g = &rtw89_8852a_txpwr_lmt_5g,
- .txpwr_lmt_ru_2g = &rtw89_8852a_txpwr_lmt_ru_2g,
- .txpwr_lmt_ru_5g = &rtw89_8852a_txpwr_lmt_ru_5g,
+ .dflt_parms = &rtw89_8852a_dflt_parms,
+ .rfe_parms_conf = NULL,
.txpwr_factor_rf = 2,
.txpwr_factor_mac = 1,
.dig_table = &rtw89_8852a_phy_dig_table,
@@ -2143,8 +2148,9 @@ const struct rtw89_chip_info rtw8852a_chip_info = {
.c2h_counter_reg = {R_AX_UDM1 + 1, B_AX_UDM1_HALMAC_C2H_ENQ_CNT_MASK >> 8},
.page_regs = &rtw8852a_page_regs,
.cfo_src_fd = false,
+ .cfo_hw_comp = false,
.dcfo_comp = &rtw8852a_dcfo_comp,
- .dcfo_comp_sft = 3,
+ .dcfo_comp_sft = 10,
.imr_info = &rtw8852a_imr_info,
.rrsr_cfgs = &rtw8852a_rrsr_cfgs,
.bss_clr_map_reg = R_BSS_CLR_MAP,
@@ -2156,7 +2162,7 @@ const struct rtw89_chip_info rtw8852a_chip_info = {
};
EXPORT_SYMBOL(rtw8852a_chip_info);
-MODULE_FIRMWARE("rtw89/rtw8852a_fw.bin");
+MODULE_FIRMWARE(RTW8852A_MODULE_FIRMWARE);
MODULE_AUTHOR("Realtek Corporation");
MODULE_DESCRIPTION("Realtek 802.11ax wireless 8852A driver");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
index 320bcd4..be54194 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.c
@@ -43377,6 +43377,7 @@ static const s8 _txpwr_track_delta_swingidx_2g_cck_a_p[] = {
0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5,
6, 6, 6, 7, 7, 7, 8, 8, 8, 9, 9, 9, 10, 10};
+static
const s8 rtw89_8852a_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
@@ -45566,6 +45567,7 @@ const s8 rtw89_8852a_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[1][1][2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852a_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
@@ -47898,6 +47900,7 @@ const s8 rtw89_8852a_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[2][1][2][1][RTW89_UK][41] = 40,
};
+static
const s8 rtw89_8852a_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
[0][0][RTW89_WW][0] = 32,
@@ -48994,6 +48997,7 @@ const s8 rtw89_8852a_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852a_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
[0][0][RTW89_WW][0] = 22,
@@ -51043,3 +51047,14 @@ const struct rtw89_phy_dig_gain_table rtw89_8852a_phy_dig_table = {
.cfg_lna_a = &rtw89_8852a_lna_gain_a_table,
.cfg_tia_a = &rtw89_8852a_tia_gain_a_table
};
+
+const struct rtw89_rfe_parms rtw89_8852a_dflt_parms = {
+ .rule_2ghz = {
+ .lmt = &rtw89_8852a_txpwr_lmt_2g,
+ .lmt_ru = &rtw89_8852a_txpwr_lmt_ru_2g,
+ },
+ .rule_5ghz = {
+ .lmt = &rtw89_8852a_txpwr_lmt_5g,
+ .lmt_ru = &rtw89_8852a_txpwr_lmt_ru_5g,
+ },
+};
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.h b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.h
index 9137965..41c379b 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852a_table.h
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852a_table.h
@@ -14,15 +14,6 @@ extern const struct rtw89_phy_table rtw89_8852a_phy_nctl_table;
extern const struct rtw89_txpwr_table rtw89_8852a_byr_table;
extern const struct rtw89_phy_dig_gain_table rtw89_8852a_phy_dig_table;
extern const struct rtw89_txpwr_track_cfg rtw89_8852a_trk_cfg;
-extern const s8 rtw89_8852a_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852a_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
-extern const s8 rtw89_8852a_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852a_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
+extern const struct rtw89_rfe_parms rtw89_8852a_dflt_parms;
#endif
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852b.c b/drivers/net/wireless/realtek/rtw89/rtw8852b.c
index bae8098..eaa2ea05 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852b.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852b.c
@@ -12,6 +12,11 @@
#include "rtw8852b_table.h"
#include "txrx.h"
+#define RTW8852B_FW_FORMAT_MAX 1
+#define RTW8852B_FW_BASENAME "rtw89/rtw8852b_fw"
+#define RTW8852B_MODULE_FIRMWARE \
+ RTW8852B_FW_BASENAME "-" __stringify(RTW8852B_FW_FORMAT_MAX) ".bin"
+
static const struct rtw89_hfc_ch_cfg rtw8852b_hfc_chcfg_pcie[] = {
{5, 343, grp_0}, /* ACH 0 */
{5, 343, grp_0}, /* ACH 1 */
@@ -48,6 +53,10 @@ static const struct rtw89_dle_mem rtw8852b_dle_mem_pcie[] = {
&rtw89_mac_size.ple_size6, &rtw89_mac_size.wde_qt6,
&rtw89_mac_size.wde_qt6, &rtw89_mac_size.ple_qt18,
&rtw89_mac_size.ple_qt58},
+ [RTW89_QTA_WOW] = {RTW89_QTA_WOW, &rtw89_mac_size.wde_size6,
+ &rtw89_mac_size.ple_size6, &rtw89_mac_size.wde_qt6,
+ &rtw89_mac_size.wde_qt6, &rtw89_mac_size.ple_qt18,
+ &rtw89_mac_size.ple_qt_52b_wow},
[RTW89_QTA_DLFW] = {RTW89_QTA_DLFW, &rtw89_mac_size.wde_size9,
&rtw89_mac_size.ple_size8, &rtw89_mac_size.wde_qt4,
&rtw89_mac_size.wde_qt4, &rtw89_mac_size.ple_qt13,
@@ -323,7 +332,7 @@ static const struct rtw89_btc_rf_trx_para rtw89_btc_8852b_rf_ul[] = {
{255, 0, 0, 7}, /* 2 ->reserved for shared-antenna */
{255, 0, 0, 7}, /* 3- >reserved for shared-antenna */
{255, 0, 0, 7}, /* 4 ->reserved for shared-antenna */
- {255, 0, 0, 7}, /* the below id is for non-shared-antenna free-run */
+ {255, 1, 0, 7}, /* the below id is for non-shared-antenna free-run */
{6, 1, 0, 7},
{13, 1, 0, 7},
{13, 1, 0, 7}
@@ -335,7 +344,7 @@ static const struct rtw89_btc_rf_trx_para rtw89_btc_8852b_rf_dl[] = {
{255, 0, 0, 7}, /* 2 ->reserved for shared-antenna */
{255, 0, 0, 7}, /* 3- >reserved for shared-antenna */
{255, 0, 0, 7}, /* 4 ->reserved for shared-antenna */
- {255, 0, 0, 7}, /* the below id is for non-shared-antenna free-run */
+ {255, 1, 0, 7}, /* the below id is for non-shared-antenna free-run */
{255, 1, 0, 7},
{255, 1, 0, 7},
{255, 1, 0, 7}
@@ -355,7 +364,9 @@ static const struct rtw89_btc_fbtc_mreg rtw89_btc_8852b_mon_reg[] = {
RTW89_DEF_FBTC_MREG(REG_MAC, 4, 0xd200),
RTW89_DEF_FBTC_MREG(REG_MAC, 4, 0xd220),
RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x980),
- RTW89_DEF_FBTC_MREG(REG_BT_MODEM, 4, 0x178),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x4738),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x4688),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x4694),
};
static const u8 rtw89_btc_8852b_wl_rssi_thres[BTC_WL_RSSI_THMAX] = {70, 60, 50, 40};
@@ -1284,7 +1295,7 @@ static void rtw8852b_ctrl_cck_en(struct rtw89_dev *rtwdev, bool cck_en)
static void rtw8852b_5m_mask(struct rtw89_dev *rtwdev, const struct rtw89_chan *chan,
enum rtw89_phy_idx phy_idx)
{
- u8 pri_ch = chan->primary_channel;
+ u8 pri_ch = chan->pri_ch_idx;
bool mask_5m_low;
bool mask_5m_en;
@@ -1292,12 +1303,13 @@ static void rtw8852b_5m_mask(struct rtw89_dev *rtwdev, const struct rtw89_chan *
case RTW89_CHANNEL_WIDTH_40:
/* Prich=1: Mask 5M High, Prich=2: Mask 5M Low */
mask_5m_en = true;
- mask_5m_low = pri_ch == 2;
+ mask_5m_low = pri_ch == RTW89_SC_20_LOWER;
break;
case RTW89_CHANNEL_WIDTH_80:
/* Prich=3: Mask 5M High, Prich=4: Mask 5M Low, Else: Disable */
- mask_5m_en = pri_ch == 3 || pri_ch == 4;
- mask_5m_low = pri_ch == 4;
+ mask_5m_en = pri_ch == RTW89_SC_20_UPMOST ||
+ pri_ch == RTW89_SC_20_LOWEST;
+ mask_5m_low = pri_ch == RTW89_SC_20_LOWEST;
break;
default:
mask_5m_en = false;
@@ -2267,7 +2279,8 @@ do { \
static
s8 rtw8852b_btc_get_bt_rssi(struct rtw89_dev *rtwdev, s8 val)
{
- return clamp_t(s8, val, -100, 0) + 100;
+ /* +6 for compensate offset */
+ return clamp_t(s8, val + 6, -100, 0) + 100;
}
static
@@ -2477,10 +2490,20 @@ static const struct rtw89_chip_ops rtw8852b_chip_ops = {
.btc_set_policy = rtw89_btc_set_policy_v1,
};
+#ifdef CONFIG_PM
+static const struct wiphy_wowlan_support rtw_wowlan_stub_8852b = {
+ .flags = WIPHY_WOWLAN_MAGIC_PKT | WIPHY_WOWLAN_DISCONNECT,
+ .n_patterns = RTW89_MAX_PATTERN_NUM,
+ .pattern_max_len = RTW89_MAX_PATTERN_SIZE,
+ .pattern_min_len = 1,
+};
+#endif
+
const struct rtw89_chip_info rtw8852b_chip_info = {
.chip_id = RTL8852B,
.ops = &rtw8852b_chip_ops,
- .fw_name = "rtw89/rtw8852b_fw.bin",
+ .fw_basename = RTW8852B_FW_BASENAME,
+ .fw_format_max = RTW8852B_FW_FORMAT_MAX,
.try_ce_fw = true,
.fifo_size = 196608,
.dle_scc_rsvd_size = 98304,
@@ -2500,10 +2523,8 @@ const struct rtw89_chip_info rtw8852b_chip_info = {
&rtw89_8852b_phy_radiob_table,},
.nctl_table = &rtw89_8852b_phy_nctl_table,
.byr_table = &rtw89_8852b_byr_table,
- .txpwr_lmt_2g = &rtw89_8852b_txpwr_lmt_2g,
- .txpwr_lmt_5g = &rtw89_8852b_txpwr_lmt_5g,
- .txpwr_lmt_ru_2g = &rtw89_8852b_txpwr_lmt_ru_2g,
- .txpwr_lmt_ru_5g = &rtw89_8852b_txpwr_lmt_ru_5g,
+ .dflt_parms = &rtw89_8852b_dflt_parms,
+ .rfe_parms_conf = NULL,
.txpwr_factor_rf = 2,
.txpwr_factor_mac = 1,
.dig_table = NULL,
@@ -2564,8 +2585,9 @@ const struct rtw89_chip_info rtw8852b_chip_info = {
.c2h_regs = rtw8852b_c2h_regs,
.page_regs = &rtw8852b_page_regs,
.cfo_src_fd = true,
+ .cfo_hw_comp = true,
.dcfo_comp = &rtw8852b_dcfo_comp,
- .dcfo_comp_sft = 3,
+ .dcfo_comp_sft = 10,
.imr_info = &rtw8852b_imr_info,
.rrsr_cfgs = &rtw8852b_rrsr_cfgs,
.bss_clr_map_reg = R_BSS_CLR_MAP_V1,
@@ -2573,10 +2595,13 @@ const struct rtw89_chip_info rtw8852b_chip_info = {
BIT(RTW89_DMA_ACH6) | BIT(RTW89_DMA_ACH7) |
BIT(RTW89_DMA_B1MG) | BIT(RTW89_DMA_B1HI),
.edcca_lvl_reg = R_SEG0R_EDCCA_LVL_V1,
+#ifdef CONFIG_PM
+ .wowlan_stub = &rtw_wowlan_stub_8852b,
+#endif
};
EXPORT_SYMBOL(rtw8852b_chip_info);
-MODULE_FIRMWARE("rtw89/rtw8852b_fw.bin");
+MODULE_FIRMWARE(RTW8852B_MODULE_FIRMWARE);
MODULE_AUTHOR("Realtek Corporation");
MODULE_DESCRIPTION("Realtek 802.11ax wireless 8852B driver");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852b_table.c b/drivers/net/wireless/realtek/rtw89/rtw8852b_table.c
index a673496..904cdb9 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852b_table.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852b_table.c
@@ -14706,6 +14706,7 @@ const u8 rtw89_8852b_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
[1][1][RTW89_UKRAINE] = 0,
};
+static
const s8 rtw89_8852b_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
@@ -16895,6 +16896,7 @@ const s8 rtw89_8852b_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[1][1][2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852b_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
@@ -19539,6 +19541,7 @@ const s8 rtw89_8852b_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[2][1][2][1][RTW89_UK][49] = 127,
};
+static
const s8 rtw89_8852b_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
[0][0][RTW89_WW][0] = 32,
@@ -20635,6 +20638,7 @@ const s8 rtw89_8852b_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852b_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
[0][0][RTW89_WW][0] = 24,
@@ -22875,3 +22879,14 @@ const struct rtw89_txpwr_track_cfg rtw89_8852b_trk_cfg = {
.delta_swingidx_2g_cck_a_n = _txpwr_track_delta_swingidx_2g_cck_a_n,
.delta_swingidx_2g_cck_a_p = _txpwr_track_delta_swingidx_2g_cck_a_p,
};
+
+const struct rtw89_rfe_parms rtw89_8852b_dflt_parms = {
+ .rule_2ghz = {
+ .lmt = &rtw89_8852b_txpwr_lmt_2g,
+ .lmt_ru = &rtw89_8852b_txpwr_lmt_ru_2g,
+ },
+ .rule_5ghz = {
+ .lmt = &rtw89_8852b_txpwr_lmt_5g,
+ .lmt_ru = &rtw89_8852b_txpwr_lmt_ru_5g,
+ },
+};
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852b_table.h b/drivers/net/wireless/realtek/rtw89/rtw8852b_table.h
index 114337a..5f41614 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852b_table.h
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852b_table.h
@@ -16,15 +16,6 @@ extern const struct rtw89_txpwr_table rtw89_8852b_byr_table;
extern const struct rtw89_txpwr_track_cfg rtw89_8852b_trk_cfg;
extern const u8 rtw89_8852b_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
[RTW89_REGD_NUM];
-extern const s8 rtw89_8852b_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852b_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
-extern const s8 rtw89_8852b_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852b_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
+extern const struct rtw89_rfe_parms rtw89_8852b_dflt_parms;
#endif
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c.c b/drivers/net/wireless/realtek/rtw89/rtw8852c.c
index ba728fc..ceb819a 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852c.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852c.c
@@ -13,6 +13,11 @@
#include "rtw8852c_table.h"
#include "util.h"
+#define RTW8852C_FW_FORMAT_MAX 0
+#define RTW8852C_FW_BASENAME "rtw89/rtw8852c_fw"
+#define RTW8852C_MODULE_FIRMWARE \
+ RTW8852C_FW_BASENAME ".bin"
+
static const struct rtw89_hfc_ch_cfg rtw8852c_hfc_chcfg_pcie[] = {
{13, 1614, grp_0}, /* ACH 0 */
{13, 1614, grp_0}, /* ACH 1 */
@@ -1375,18 +1380,19 @@ static void rtw8852c_5m_mask(struct rtw89_dev *rtwdev,
const struct rtw89_chan *chan,
enum rtw89_phy_idx phy_idx)
{
- u8 pri_ch = chan->primary_channel;
+ u8 pri_ch = chan->pri_ch_idx;
bool mask_5m_low;
bool mask_5m_en;
switch (chan->band_width) {
case RTW89_CHANNEL_WIDTH_40:
mask_5m_en = true;
- mask_5m_low = pri_ch == 2;
+ mask_5m_low = pri_ch == RTW89_SC_20_LOWER;
break;
case RTW89_CHANNEL_WIDTH_80:
- mask_5m_en = ((pri_ch == 3) || (pri_ch == 4));
- mask_5m_low = pri_ch == 4;
+ mask_5m_en = pri_ch == RTW89_SC_20_UPMOST ||
+ pri_ch == RTW89_SC_20_LOWEST;
+ mask_5m_low = pri_ch == RTW89_SC_20_LOWEST;
break;
default:
mask_5m_en = false;
@@ -2527,7 +2533,8 @@ do { \
static
s8 rtw8852c_btc_get_bt_rssi(struct rtw89_dev *rtwdev, s8 val)
{
- return clamp_t(s8, val, -100, 0) + 100;
+ /* +6 for compensate offset */
+ return clamp_t(s8, val + 6, -100, 0) + 100;
}
static const struct rtw89_btc_rf_trx_para rtw89_btc_8852c_rf_ul[] = {
@@ -2536,7 +2543,7 @@ static const struct rtw89_btc_rf_trx_para rtw89_btc_8852c_rf_ul[] = {
{255, 0, 0, 7}, /* 2 ->reserved for shared-antenna */
{255, 0, 0, 7}, /* 3- >reserved for shared-antenna */
{255, 0, 0, 7}, /* 4 ->reserved for shared-antenna */
- {255, 0, 0, 7}, /* the below id is for non-shared-antenna free-run */
+ {255, 1, 0, 7}, /* the below id is for non-shared-antenna free-run */
{6, 1, 0, 7},
{13, 1, 0, 7},
{13, 1, 0, 7}
@@ -2548,7 +2555,7 @@ static const struct rtw89_btc_rf_trx_para rtw89_btc_8852c_rf_dl[] = {
{255, 0, 0, 7}, /* 2 ->reserved for shared-antenna */
{255, 0, 0, 7}, /* 3- >reserved for shared-antenna */
{255, 0, 0, 7}, /* 4 ->reserved for shared-antenna */
- {255, 0, 0, 7}, /* the below id is for non-shared-antenna free-run */
+ {255, 1, 0, 7}, /* the below id is for non-shared-antenna free-run */
{255, 1, 0, 7},
{255, 1, 0, 7},
{255, 1, 0, 7}
@@ -2570,6 +2577,9 @@ static const struct rtw89_btc_fbtc_mreg rtw89_btc_8852c_mon_reg[] = {
RTW89_DEF_FBTC_MREG(REG_MAC, 4, 0xd200),
RTW89_DEF_FBTC_MREG(REG_MAC, 4, 0xd220),
RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x980),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x4aa4),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x4778),
+ RTW89_DEF_FBTC_MREG(REG_BB, 4, 0x476c),
};
static
@@ -2791,7 +2801,8 @@ static const struct rtw89_chip_ops rtw8852c_chip_ops = {
const struct rtw89_chip_info rtw8852c_chip_info = {
.chip_id = RTL8852C,
.ops = &rtw8852c_chip_ops,
- .fw_name = "rtw89/rtw8852c_fw.bin",
+ .fw_basename = RTW8852C_FW_BASENAME,
+ .fw_format_max = RTW8852C_FW_FORMAT_MAX,
.try_ce_fw = false,
.fifo_size = 458752,
.dle_scc_rsvd_size = 0,
@@ -2811,12 +2822,8 @@ const struct rtw89_chip_info rtw8852c_chip_info = {
&rtw89_8852c_phy_radioa_table,},
.nctl_table = &rtw89_8852c_phy_nctl_table,
.byr_table = &rtw89_8852c_byr_table,
- .txpwr_lmt_2g = &rtw89_8852c_txpwr_lmt_2g,
- .txpwr_lmt_5g = &rtw89_8852c_txpwr_lmt_5g,
- .txpwr_lmt_6g = &rtw89_8852c_txpwr_lmt_6g,
- .txpwr_lmt_ru_2g = &rtw89_8852c_txpwr_lmt_ru_2g,
- .txpwr_lmt_ru_5g = &rtw89_8852c_txpwr_lmt_ru_5g,
- .txpwr_lmt_ru_6g = &rtw89_8852c_txpwr_lmt_ru_6g,
+ .dflt_parms = &rtw89_8852c_dflt_parms,
+ .rfe_parms_conf = NULL,
.txpwr_factor_rf = 2,
.txpwr_factor_mac = 1,
.dig_table = NULL,
@@ -2879,8 +2886,9 @@ const struct rtw89_chip_info rtw8852c_chip_info = {
.c2h_regs = rtw8852c_c2h_regs,
.page_regs = &rtw8852c_page_regs,
.cfo_src_fd = false,
+ .cfo_hw_comp = false,
.dcfo_comp = &rtw8852c_dcfo_comp,
- .dcfo_comp_sft = 5,
+ .dcfo_comp_sft = 12,
.imr_info = &rtw8852c_imr_info,
.rrsr_cfgs = &rtw8852c_rrsr_cfgs,
.bss_clr_map_reg = R_BSS_CLR_MAP,
@@ -2892,7 +2900,7 @@ const struct rtw89_chip_info rtw8852c_chip_info = {
};
EXPORT_SYMBOL(rtw8852c_chip_info);
-MODULE_FIRMWARE("rtw89/rtw8852c_fw.bin");
+MODULE_FIRMWARE(RTW8852C_MODULE_FIRMWARE);
MODULE_AUTHOR("Realtek Corporation");
MODULE_DESCRIPTION("Realtek 802.11ax wireless 8852C driver");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c
index 96c264a..7011e5a 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.c
@@ -28590,6 +28590,7 @@ const u8 rtw89_8852c_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
[2][1][RTW89_KCC] = 0,
};
+static
const s8 rtw89_8852c_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
@@ -30107,6 +30108,7 @@ const s8 rtw89_8852c_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
[1][1][2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852c_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
@@ -32020,6 +32022,7 @@ const s8 rtw89_8852c_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
[3][1][2][1][RTW89_UK][45] = 127,
};
+static
const s8 rtw89_8852c_txpwr_lmt_6g[RTW89_6G_BW_NUM][RTW89_NTX_NUM]
[RTW89_RS_LMT_NUM][RTW89_BF_NUM]
[RTW89_REGD_NUM][RTW89_6G_CH_NUM] = {
@@ -33977,6 +33980,7 @@ const s8 rtw89_8852c_txpwr_lmt_6g[RTW89_6G_BW_NUM][RTW89_NTX_NUM]
[3][1][2][1][RTW89_KCC][112] = 127,
};
+static
const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_2G_CH_NUM] = {
[0][0][RTW89_WW][0] = 32,
@@ -34737,6 +34741,7 @@ const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
[2][1][RTW89_UK][13] = 127,
};
+static
const s8 rtw89_8852c_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_5G_CH_NUM] = {
[0][0][RTW89_WW][0] = 16,
@@ -36253,6 +36258,7 @@ const s8 rtw89_8852c_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
[2][1][RTW89_UK][52] = 127,
};
+static
const s8 rtw89_8852c_txpwr_lmt_ru_6g[RTW89_RU_NUM][RTW89_NTX_NUM]
[RTW89_REGD_NUM][RTW89_6G_CH_NUM] = {
[0][0][RTW89_WW][0] = -16,
@@ -37472,3 +37478,18 @@ const struct rtw89_phy_tssi_dbw_table rtw89_8852c_tssi_dbw_table = {
.data[RTW89_TSSI_BANDEDGE_MID] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
.data[RTW89_TSSI_BANDEDGE_HIGH] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
};
+
+const struct rtw89_rfe_parms rtw89_8852c_dflt_parms = {
+ .rule_2ghz = {
+ .lmt = &rtw89_8852c_txpwr_lmt_2g,
+ .lmt_ru = &rtw89_8852c_txpwr_lmt_ru_2g,
+ },
+ .rule_5ghz = {
+ .lmt = &rtw89_8852c_txpwr_lmt_5g,
+ .lmt_ru = &rtw89_8852c_txpwr_lmt_ru_5g,
+ },
+ .rule_6ghz = {
+ .lmt = &rtw89_8852c_txpwr_lmt_6g,
+ .lmt_ru = &rtw89_8852c_txpwr_lmt_ru_6g,
+ },
+};
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.h b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.h
index 7d71a92..6da1849 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8852c_table.h
+++ b/drivers/net/wireless/realtek/rtw89/rtw8852c_table.h
@@ -17,20 +17,6 @@ extern const struct rtw89_phy_tssi_dbw_table rtw89_8852c_tssi_dbw_table;
extern const struct rtw89_txpwr_track_cfg rtw89_8852c_trk_cfg;
extern const u8 rtw89_8852c_tx_shape[RTW89_BAND_MAX][RTW89_RS_TX_SHAPE_NUM]
[RTW89_REGD_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_2g[RTW89_2G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_5g[RTW89_5G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_6g[RTW89_6G_BW_NUM][RTW89_NTX_NUM]
- [RTW89_RS_LMT_NUM][RTW89_BF_NUM]
- [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_ru_2g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_2G_CH_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_ru_5g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_5G_CH_NUM];
-extern const s8 rtw89_8852c_txpwr_lmt_ru_6g[RTW89_RU_NUM][RTW89_NTX_NUM]
- [RTW89_REGD_NUM][RTW89_6G_CH_NUM];
+extern const struct rtw89_rfe_parms rtw89_8852c_dflt_parms;
#endif
diff --git a/drivers/net/wireless/realtek/rtw89/wow.c b/drivers/net/wireless/realtek/rtw89/wow.c
index 7cff9c1..2ca8abb 100644
--- a/drivers/net/wireless/realtek/rtw89/wow.c
+++ b/drivers/net/wireless/realtek/rtw89/wow.c
@@ -30,7 +30,7 @@ static void rtw89_wow_enter_lps(struct rtw89_dev *rtwdev)
struct ieee80211_vif *wow_vif = rtwdev->wow.wow_vif;
struct rtw89_vif *rtwvif = (struct rtw89_vif *)wow_vif->drv_priv;
- rtw89_enter_lps(rtwdev, rtwvif);
+ rtw89_enter_lps(rtwdev, rtwvif, false);
}
static void rtw89_wow_leave_lps(struct rtw89_dev *rtwdev)
diff --git a/drivers/net/wireless/virtual/mac80211_hwsim.c b/drivers/net/wireless/virtual/mac80211_hwsim.c
index f446d8f..2211fa5 100644
--- a/drivers/net/wireless/virtual/mac80211_hwsim.c
+++ b/drivers/net/wireless/virtual/mac80211_hwsim.c
@@ -3760,6 +3760,9 @@ static int hwsim_pmsr_report_nl(struct sk_buff *msg, struct genl_info *info)
int err;
int rem;
+ if (!info->attrs[HWSIM_ATTR_ADDR_TRANSMITTER])
+ return -EINVAL;
+
src = nla_data(info->attrs[HWSIM_ATTR_ADDR_TRANSMITTER]);
data = get_hwsim_data_ref_from_addr(src);
if (!data)
diff --git a/drivers/net/wwan/rpmsg_wwan_ctrl.c b/drivers/net/wwan/rpmsg_wwan_ctrl.c
index 06f4b02..86b60aa 100644
--- a/drivers/net/wwan/rpmsg_wwan_ctrl.c
+++ b/drivers/net/wwan/rpmsg_wwan_ctrl.c
@@ -149,6 +149,7 @@ static const struct rpmsg_device_id rpmsg_wwan_ctrl_id_table[] = {
/* RPMSG channels for Qualcomm SoCs with integrated modem */
{ .name = "DATA5_CNTL", .driver_data = WWAN_PORT_QMI },
{ .name = "DATA4", .driver_data = WWAN_PORT_AT },
+ { .name = "DATA1", .driver_data = WWAN_PORT_AT },
{},
};
MODULE_DEVICE_TABLE(rpmsg, rpmsg_wwan_ctrl_id_table);
diff --git a/drivers/net/wwan/wwan_core.c b/drivers/net/wwan/wwan_core.c
index 2e1c01cf..aa54fa6 100644
--- a/drivers/net/wwan/wwan_core.c
+++ b/drivers/net/wwan/wwan_core.c
@@ -492,6 +492,7 @@ struct wwan_port *wwan_create_port(struct device *parent,
if (err)
goto error_put_device;
+ dev_info(&wwandev->dev, "port %s attached\n", dev_name(&port->dev));
return port;
error_put_device:
@@ -517,6 +518,8 @@ void wwan_remove_port(struct wwan_port *port)
skb_queue_purge(&port->rxq);
dev_set_drvdata(&port->dev, NULL);
+
+ dev_info(&wwandev->dev, "port %s disconnected\n", dev_name(&port->dev));
device_unregister(&port->dev);
/* Release related wwan device */
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 282d808..cd7873d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3443,6 +3443,8 @@ static const struct pci_device_id nvme_id_table[] = {
{ PCI_DEVICE(0x1d97, 0x2269), /* Lexar NM760 */
.driver_data = NVME_QUIRK_BOGUS_NID |
NVME_QUIRK_IGNORE_DEV_SUBNQN, },
+ { PCI_DEVICE(0x10ec, 0x5763), /* TEAMGROUP T-FORCE CARDEA ZERO Z330 SSD */
+ .driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0061),
.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0065),
diff --git a/drivers/perf/amlogic/meson_g12_ddr_pmu.c b/drivers/perf/amlogic/meson_g12_ddr_pmu.c
index a78fdb1..8b64388 100644
--- a/drivers/perf/amlogic/meson_g12_ddr_pmu.c
+++ b/drivers/perf/amlogic/meson_g12_ddr_pmu.c
@@ -21,23 +21,23 @@
#define DMC_QOS_IRQ BIT(30)
/* DMC bandwidth monitor register address offset */
-#define DMC_MON_G12_CTRL0 (0x20 << 2)
-#define DMC_MON_G12_CTRL1 (0x21 << 2)
-#define DMC_MON_G12_CTRL2 (0x22 << 2)
-#define DMC_MON_G12_CTRL3 (0x23 << 2)
-#define DMC_MON_G12_CTRL4 (0x24 << 2)
-#define DMC_MON_G12_CTRL5 (0x25 << 2)
-#define DMC_MON_G12_CTRL6 (0x26 << 2)
-#define DMC_MON_G12_CTRL7 (0x27 << 2)
-#define DMC_MON_G12_CTRL8 (0x28 << 2)
+#define DMC_MON_G12_CTRL0 (0x0 << 2)
+#define DMC_MON_G12_CTRL1 (0x1 << 2)
+#define DMC_MON_G12_CTRL2 (0x2 << 2)
+#define DMC_MON_G12_CTRL3 (0x3 << 2)
+#define DMC_MON_G12_CTRL4 (0x4 << 2)
+#define DMC_MON_G12_CTRL5 (0x5 << 2)
+#define DMC_MON_G12_CTRL6 (0x6 << 2)
+#define DMC_MON_G12_CTRL7 (0x7 << 2)
+#define DMC_MON_G12_CTRL8 (0x8 << 2)
-#define DMC_MON_G12_ALL_REQ_CNT (0x29 << 2)
-#define DMC_MON_G12_ALL_GRANT_CNT (0x2a << 2)
-#define DMC_MON_G12_ONE_GRANT_CNT (0x2b << 2)
-#define DMC_MON_G12_SEC_GRANT_CNT (0x2c << 2)
-#define DMC_MON_G12_THD_GRANT_CNT (0x2d << 2)
-#define DMC_MON_G12_FOR_GRANT_CNT (0x2e << 2)
-#define DMC_MON_G12_TIMER (0x2f << 2)
+#define DMC_MON_G12_ALL_REQ_CNT (0x9 << 2)
+#define DMC_MON_G12_ALL_GRANT_CNT (0xa << 2)
+#define DMC_MON_G12_ONE_GRANT_CNT (0xb << 2)
+#define DMC_MON_G12_SEC_GRANT_CNT (0xc << 2)
+#define DMC_MON_G12_THD_GRANT_CNT (0xd << 2)
+#define DMC_MON_G12_FOR_GRANT_CNT (0xe << 2)
+#define DMC_MON_G12_TIMER (0xf << 2)
/* Each bit represent a axi line */
PMU_FORMAT_ATTR(event, "config:0-7");
diff --git a/drivers/regulator/fan53555.c b/drivers/regulator/fan53555.c
index 529963a..41537c4 100644
--- a/drivers/regulator/fan53555.c
+++ b/drivers/regulator/fan53555.c
@@ -8,18 +8,19 @@
// Copyright (c) 2012 Marvell Technology Ltd.
// Yunfan Zhang <yfzhang@marvell.com>
-#include <linux/module.h>
-#include <linux/param.h>
+#include <linux/bits.h>
#include <linux/err.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/param.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include <linux/regulator/driver.h>
+#include <linux/regulator/fan53555.h>
#include <linux/regulator/machine.h>
#include <linux/regulator/of_regulator.h>
-#include <linux/of_device.h>
-#include <linux/i2c.h>
#include <linux/slab.h>
-#include <linux/regmap.h>
-#include <linux/regulator/fan53555.h>
/* Voltage setting */
#define FAN53555_VSEL0 0x00
@@ -60,7 +61,7 @@
#define TCS_VSEL1_MODE (1 << 6)
#define TCS_SLEW_SHIFT 3
-#define TCS_SLEW_MASK (0x3 < 3)
+#define TCS_SLEW_MASK GENMASK(4, 3)
enum fan53555_vendor {
FAN53526_VENDOR_FAIRCHILD = 0,
diff --git a/drivers/regulator/sm5703-regulator.c b/drivers/regulator/sm5703-regulator.c
index 05ad28f..229df71 100644
--- a/drivers/regulator/sm5703-regulator.c
+++ b/drivers/regulator/sm5703-regulator.c
@@ -42,6 +42,7 @@ static const int sm5703_buck_voltagemap[] = {
.type = REGULATOR_VOLTAGE, \
.id = SM5703_USBLDO ## _id, \
.ops = &sm5703_regulator_ops_fixed, \
+ .n_voltages = 1, \
.fixed_uV = SM5703_USBLDO_MICROVOLT, \
.enable_reg = SM5703_REG_USBLDO12, \
.enable_mask = SM5703_REG_EN_USBLDO ##_id, \
@@ -56,6 +57,7 @@ static const int sm5703_buck_voltagemap[] = {
.type = REGULATOR_VOLTAGE, \
.id = SM5703_VBUS, \
.ops = &sm5703_regulator_ops_fixed, \
+ .n_voltages = 1, \
.fixed_uV = SM5703_VBUS_MICROVOLT, \
.enable_reg = SM5703_REG_CNTL, \
.enable_mask = SM5703_OPERATION_MODE_MASK, \
diff --git a/drivers/scsi/ses.c b/drivers/scsi/ses.c
index b11a916..b54f2c6 100644
--- a/drivers/scsi/ses.c
+++ b/drivers/scsi/ses.c
@@ -509,9 +509,6 @@ static int ses_enclosure_find_by_addr(struct enclosure_device *edev,
int i;
struct ses_component *scomp;
- if (!edev->component[0].scratch)
- return 0;
-
for (i = 0; i < edev->components; i++) {
scomp = edev->component[i].scratch;
if (scomp->addr != efd->addr)
@@ -602,8 +599,10 @@ static void ses_enclosure_data_process(struct enclosure_device *edev,
components++,
type_ptr[0],
name);
- else
+ else if (components < edev->components)
ecomp = &edev->component[components++];
+ else
+ ecomp = ERR_PTR(-EINVAL);
if (!IS_ERR(ecomp)) {
if (addl_desc_ptr) {
@@ -734,11 +733,6 @@ static int ses_intf_add(struct device *cdev,
components += type_ptr[1];
}
- if (components == 0) {
- sdev_printk(KERN_WARNING, sdev, "enclosure has no enumerated components\n");
- goto err_free;
- }
-
ses_dev->page1 = buf;
ses_dev->page1_len = len;
buf = NULL;
@@ -780,9 +774,11 @@ static int ses_intf_add(struct device *cdev,
buf = NULL;
}
page2_not_supported:
- scomp = kcalloc(components, sizeof(struct ses_component), GFP_KERNEL);
- if (!scomp)
- goto err_free;
+ if (components > 0) {
+ scomp = kcalloc(components, sizeof(struct ses_component), GFP_KERNEL);
+ if (!scomp)
+ goto err_free;
+ }
edev = enclosure_register(cdev->parent, dev_name(&sdev->sdev_gendev),
components, &ses_enclosure_callbacks);
diff --git a/drivers/spi/spi-rockchip-sfc.c b/drivers/spi/spi-rockchip-sfc.c
index bd87d3c9..69347b6 100644
--- a/drivers/spi/spi-rockchip-sfc.c
+++ b/drivers/spi/spi-rockchip-sfc.c
@@ -632,7 +632,7 @@ static int rockchip_sfc_probe(struct platform_device *pdev)
if (ret) {
dev_err(dev, "Failed to request irq\n");
- return ret;
+ goto err_irq;
}
ret = rockchip_sfc_init(sfc);
diff --git a/drivers/tee/optee/call.c b/drivers/tee/optee/call.c
index 290b1bb..df5fb54 100644
--- a/drivers/tee/optee/call.c
+++ b/drivers/tee/optee/call.c
@@ -488,7 +488,7 @@ static bool is_normal_memory(pgprot_t p)
#elif defined(CONFIG_ARM64)
return (pgprot_val(p) & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL);
#else
-#error "Unuspported architecture"
+#error "Unsupported architecture"
#endif
}
diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c
index b1c6231..673cf03 100644
--- a/drivers/tee/tee_shm.c
+++ b/drivers/tee/tee_shm.c
@@ -32,7 +32,7 @@ static int shm_get_kernel_pages(unsigned long start, size_t page_count,
is_kmap_addr((void *)start)))
return -EINVAL;
- page = virt_to_page(start);
+ page = virt_to_page((void *)start);
for (n = 0; n < page_count; n++) {
pages[n] = page + n;
get_page(pages[n]);
diff --git a/drivers/thermal/intel/therm_throt.c b/drivers/thermal/intel/therm_throt.c
index 2e22bb8..e69868e 100644
--- a/drivers/thermal/intel/therm_throt.c
+++ b/drivers/thermal/intel/therm_throt.c
@@ -193,8 +193,67 @@ static const struct attribute_group thermal_attr_group = {
#define THERM_THROT_POLL_INTERVAL HZ
#define THERM_STATUS_PROCHOT_LOG BIT(1)
-#define THERM_STATUS_CLEAR_CORE_MASK (BIT(1) | BIT(3) | BIT(5) | BIT(7) | BIT(9) | BIT(11) | BIT(13) | BIT(15))
-#define THERM_STATUS_CLEAR_PKG_MASK (BIT(1) | BIT(3) | BIT(5) | BIT(7) | BIT(9) | BIT(11))
+static u64 therm_intr_core_clear_mask;
+static u64 therm_intr_pkg_clear_mask;
+
+static void thermal_intr_init_core_clear_mask(void)
+{
+ if (therm_intr_core_clear_mask)
+ return;
+
+ /*
+ * Reference: Intel SDM Volume 4
+ * "Table 2-2. IA-32 Architectural MSRs", MSR 0x19C
+ * IA32_THERM_STATUS.
+ */
+
+ /*
+ * Bit 1, 3, 5: CPUID.01H:EDX[22] = 1. This driver will not
+ * enable interrupts, when 0 as it checks for X86_FEATURE_ACPI.
+ */
+ therm_intr_core_clear_mask = (BIT(1) | BIT(3) | BIT(5));
+
+ /*
+ * Bit 7 and 9: Thermal Threshold #1 and #2 log
+ * If CPUID.01H:ECX[8] = 1
+ */
+ if (boot_cpu_has(X86_FEATURE_TM2))
+ therm_intr_core_clear_mask |= (BIT(7) | BIT(9));
+
+ /* Bit 11: Power Limitation log (R/WC0) If CPUID.06H:EAX[4] = 1 */
+ if (boot_cpu_has(X86_FEATURE_PLN))
+ therm_intr_core_clear_mask |= BIT(11);
+
+ /*
+ * Bit 13: Current Limit log (R/WC0) If CPUID.06H:EAX[7] = 1
+ * Bit 15: Cross Domain Limit log (R/WC0) If CPUID.06H:EAX[7] = 1
+ */
+ if (boot_cpu_has(X86_FEATURE_HWP))
+ therm_intr_core_clear_mask |= (BIT(13) | BIT(15));
+}
+
+static void thermal_intr_init_pkg_clear_mask(void)
+{
+ if (therm_intr_pkg_clear_mask)
+ return;
+
+ /*
+ * Reference: Intel SDM Volume 4
+ * "Table 2-2. IA-32 Architectural MSRs", MSR 0x1B1
+ * IA32_PACKAGE_THERM_STATUS.
+ */
+
+ /* All bits except BIT 26 depend on CPUID.06H: EAX[6] = 1 */
+ if (boot_cpu_has(X86_FEATURE_PTS))
+ therm_intr_pkg_clear_mask = (BIT(1) | BIT(3) | BIT(5) | BIT(7) | BIT(9) | BIT(11));
+
+ /*
+ * Intel SDM Volume 2A: Thermal and Power Management Leaf
+ * Bit 26: CPUID.06H: EAX[19] = 1
+ */
+ if (boot_cpu_has(X86_FEATURE_HFI))
+ therm_intr_pkg_clear_mask |= BIT(26);
+}
/*
* Clear the bits in package thermal status register for bit = 1
@@ -207,13 +266,10 @@ void thermal_clear_package_intr_status(int level, u64 bit_mask)
if (level == CORE_LEVEL) {
msr = MSR_IA32_THERM_STATUS;
- msr_val = THERM_STATUS_CLEAR_CORE_MASK;
+ msr_val = therm_intr_core_clear_mask;
} else {
msr = MSR_IA32_PACKAGE_THERM_STATUS;
- msr_val = THERM_STATUS_CLEAR_PKG_MASK;
- if (boot_cpu_has(X86_FEATURE_HFI))
- msr_val |= BIT(26);
-
+ msr_val = therm_intr_pkg_clear_mask;
}
msr_val &= ~bit_mask;
@@ -708,6 +764,9 @@ void intel_init_thermal(struct cpuinfo_x86 *c)
h = THERMAL_APIC_VECTOR | APIC_DM_FIXED | APIC_LVT_MASKED;
apic_write(APIC_LVTTHMR, h);
+ thermal_intr_init_core_clear_mask();
+ thermal_intr_init_pkg_clear_mask();
+
rdmsr(MSR_IA32_THERM_INTERRUPT, l, h);
if (cpu_has(c, X86_FEATURE_PLN) && !int_pln_enable)
wrmsr(MSR_IA32_THERM_INTERRUPT,
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index 2b92132..4245249 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -587,11 +587,15 @@ assemble_neg_contexts(struct smb2_negotiate_req *req,
}
+/* If invalid preauth context warn but use what we requested, SHA-512 */
static void decode_preauth_context(struct smb2_preauth_neg_context *ctxt)
{
unsigned int len = le16_to_cpu(ctxt->DataLength);
- /* If invalid preauth context warn but use what we requested, SHA-512 */
+ /*
+ * Caller checked that DataLength remains within SMB boundary. We still
+ * need to confirm that one HashAlgorithms member is accounted for.
+ */
if (len < MIN_PREAUTH_CTXT_DATA_LEN) {
pr_warn_once("server sent bad preauth context\n");
return;
@@ -610,7 +614,11 @@ static void decode_compress_ctx(struct TCP_Server_Info *server,
{
unsigned int len = le16_to_cpu(ctxt->DataLength);
- /* sizeof compress context is a one element compression capbility struct */
+ /*
+ * Caller checked that DataLength remains within SMB boundary. We still
+ * need to confirm that one CompressionAlgorithms member is accounted
+ * for.
+ */
if (len < 10) {
pr_warn_once("server sent bad compression cntxt\n");
return;
@@ -632,6 +640,11 @@ static int decode_encrypt_ctx(struct TCP_Server_Info *server,
unsigned int len = le16_to_cpu(ctxt->DataLength);
cifs_dbg(FYI, "decode SMB3.11 encryption neg context of len %d\n", len);
+ /*
+ * Caller checked that DataLength remains within SMB boundary. We still
+ * need to confirm that one Cipher flexible array member is accounted
+ * for.
+ */
if (len < MIN_ENCRYPT_CTXT_DATA_LEN) {
pr_warn_once("server sent bad crypto ctxt len\n");
return -EINVAL;
@@ -678,6 +691,11 @@ static void decode_signing_ctx(struct TCP_Server_Info *server,
{
unsigned int len = le16_to_cpu(pctxt->DataLength);
+ /*
+ * Caller checked that DataLength remains within SMB boundary. We still
+ * need to confirm that one SigningAlgorithms flexible array member is
+ * accounted for.
+ */
if ((len < 4) || (len > 16)) {
pr_warn_once("server sent bad signing negcontext\n");
return;
@@ -719,14 +737,19 @@ static int smb311_decode_neg_context(struct smb2_negotiate_rsp *rsp,
for (i = 0; i < ctxt_cnt; i++) {
int clen;
/* check that offset is not beyond end of SMB */
- if (len_of_ctxts == 0)
- break;
-
if (len_of_ctxts < sizeof(struct smb2_neg_context))
break;
pctx = (struct smb2_neg_context *)(offset + (char *)rsp);
- clen = le16_to_cpu(pctx->DataLength);
+ clen = sizeof(struct smb2_neg_context)
+ + le16_to_cpu(pctx->DataLength);
+ /*
+ * 2.2.4 SMB2 NEGOTIATE Response
+ * Subsequent negotiate contexts MUST appear at the first 8-byte
+ * aligned offset following the previous negotiate context.
+ */
+ if (i + 1 != ctxt_cnt)
+ clen = ALIGN(clen, 8);
if (clen > len_of_ctxts)
break;
@@ -747,12 +770,10 @@ static int smb311_decode_neg_context(struct smb2_negotiate_rsp *rsp,
else
cifs_server_dbg(VFS, "unknown negcontext of type %d ignored\n",
le16_to_cpu(pctx->ContextType));
-
if (rc)
break;
- /* offsets must be 8 byte aligned */
- clen = ALIGN(clen, 8);
- offset += clen + sizeof(struct smb2_neg_context);
+
+ offset += clen;
len_of_ctxts -= clen;
}
return rc;
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 195dc23..1db3e3c 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -978,6 +978,16 @@ static void bdi_split_work_to_wbs(struct backing_dev_info *bdi,
continue;
}
+ /*
+ * If wb_tryget fails, the wb has been shutdown, skip it.
+ *
+ * Pin @wb so that it stays on @bdi->wb_list. This allows
+ * continuing iteration from @wb after dropping and
+ * regrabbing rcu read lock.
+ */
+ if (!wb_tryget(wb))
+ continue;
+
/* alloc failed, execute synchronously using on-stack fallback */
work = &fallback_work;
*work = *base_work;
@@ -986,13 +996,6 @@ static void bdi_split_work_to_wbs(struct backing_dev_info *bdi,
work->done = &fallback_work_done;
wb_queue_work(wb, work);
-
- /*
- * Pin @wb so that it stays on @bdi->wb_list. This allows
- * continuing iteration from @wb after dropping and
- * regrabbing rcu read lock.
- */
- wb_get(wb);
last_wb = wb;
rcu_read_unlock();
diff --git a/fs/ksmbd/smb2pdu.c b/fs/ksmbd/smb2pdu.c
index 8af939a..67b7e76 100644
--- a/fs/ksmbd/smb2pdu.c
+++ b/fs/ksmbd/smb2pdu.c
@@ -876,17 +876,21 @@ static void assemble_neg_contexts(struct ksmbd_conn *conn,
}
static __le32 decode_preauth_ctxt(struct ksmbd_conn *conn,
- struct smb2_preauth_neg_context *pneg_ctxt)
+ struct smb2_preauth_neg_context *pneg_ctxt,
+ int len_of_ctxts)
{
- __le32 err = STATUS_NO_PREAUTH_INTEGRITY_HASH_OVERLAP;
+ /*
+ * sizeof(smb2_preauth_neg_context) assumes SMB311_SALT_SIZE Salt,
+ * which may not be present. Only check for used HashAlgorithms[1].
+ */
+ if (len_of_ctxts < MIN_PREAUTH_CTXT_DATA_LEN)
+ return STATUS_INVALID_PARAMETER;
- if (pneg_ctxt->HashAlgorithms == SMB2_PREAUTH_INTEGRITY_SHA512) {
- conn->preauth_info->Preauth_HashId =
- SMB2_PREAUTH_INTEGRITY_SHA512;
- err = STATUS_SUCCESS;
- }
+ if (pneg_ctxt->HashAlgorithms != SMB2_PREAUTH_INTEGRITY_SHA512)
+ return STATUS_NO_PREAUTH_INTEGRITY_HASH_OVERLAP;
- return err;
+ conn->preauth_info->Preauth_HashId = SMB2_PREAUTH_INTEGRITY_SHA512;
+ return STATUS_SUCCESS;
}
static void decode_encrypt_ctxt(struct ksmbd_conn *conn,
@@ -1014,7 +1018,8 @@ static __le32 deassemble_neg_contexts(struct ksmbd_conn *conn,
break;
status = decode_preauth_ctxt(conn,
- (struct smb2_preauth_neg_context *)pctx);
+ (struct smb2_preauth_neg_context *)pctx,
+ len_of_ctxts);
if (status != STATUS_SUCCESS)
break;
} else if (pctx->ContextType == SMB2_ENCRYPTION_CAPABILITIES) {
diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
index 6ad4139..2286596 100644
--- a/fs/nilfs2/segment.c
+++ b/fs/nilfs2/segment.c
@@ -430,6 +430,23 @@ static int nilfs_segctor_reset_segment_buffer(struct nilfs_sc_info *sci)
return 0;
}
+/**
+ * nilfs_segctor_zeropad_segsum - zero pad the rest of the segment summary area
+ * @sci: segment constructor object
+ *
+ * nilfs_segctor_zeropad_segsum() zero-fills unallocated space at the end of
+ * the current segment summary block.
+ */
+static void nilfs_segctor_zeropad_segsum(struct nilfs_sc_info *sci)
+{
+ struct nilfs_segsum_pointer *ssp;
+
+ ssp = sci->sc_blk_cnt > 0 ? &sci->sc_binfo_ptr : &sci->sc_finfo_ptr;
+ if (ssp->offset < ssp->bh->b_size)
+ memset(ssp->bh->b_data + ssp->offset, 0,
+ ssp->bh->b_size - ssp->offset);
+}
+
static int nilfs_segctor_feed_segment(struct nilfs_sc_info *sci)
{
sci->sc_nblk_this_inc += sci->sc_curseg->sb_sum.nblocks;
@@ -438,6 +455,7 @@ static int nilfs_segctor_feed_segment(struct nilfs_sc_info *sci)
* The current segment is filled up
* (internal code)
*/
+ nilfs_segctor_zeropad_segsum(sci);
sci->sc_curseg = NILFS_NEXT_SEGBUF(sci->sc_curseg);
return nilfs_segctor_reset_segment_buffer(sci);
}
@@ -542,6 +560,7 @@ static int nilfs_segctor_add_file_block(struct nilfs_sc_info *sci,
goto retry;
}
if (unlikely(required)) {
+ nilfs_segctor_zeropad_segsum(sci);
err = nilfs_segbuf_extend_segsum(segbuf);
if (unlikely(err))
goto failed;
@@ -1533,6 +1552,7 @@ static int nilfs_segctor_collect(struct nilfs_sc_info *sci,
nadd = min_t(int, nadd << 1, SC_MAX_SEGDELTA);
sci->sc_stage = prev_stage;
}
+ nilfs_segctor_zeropad_segsum(sci);
nilfs_segctor_truncate_segments(sci, sci->sc_curseg, nilfs->ns_sufile);
return 0;
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 44d1ee4..40f9e1a 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1955,8 +1955,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx,
ret = -EFAULT;
if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api)))
goto out;
- /* Ignore unsupported features (userspace built against newer kernel) */
- features = uffdio_api.features & UFFD_API_FEATURES;
+ features = uffdio_api.features;
+ ret = -EINVAL;
+ if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES))
+ goto err_out;
ret = -EPERM;
if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE))
goto err_out;
diff --git a/include/linux/ethtool.h b/include/linux/ethtool.h
index 798d358..62b6152 100644
--- a/include/linux/ethtool.h
+++ b/include/linux/ethtool.h
@@ -711,6 +711,7 @@ struct ethtool_mm_stats {
* @get_dump_data: Get dump data.
* @set_dump: Set dump specific flags to the device.
* @get_ts_info: Get the time stamping and PTP hardware clock capabilities.
+ * It may be called with RCU, or rtnl or reference on the device.
* Drivers supporting transmit time stamps in software should set this to
* ethtool_op_get_ts_info().
* @get_module_info: Get the size and type of the eeprom contained within
diff --git a/include/linux/ethtool_netlink.h b/include/linux/ethtool_netlink.h
index 17003b3..fae0dfb 100644
--- a/include/linux/ethtool_netlink.h
+++ b/include/linux/ethtool_netlink.h
@@ -39,6 +39,7 @@ void ethtool_aggregate_pause_stats(struct net_device *dev,
struct ethtool_pause_stats *pause_stats);
void ethtool_aggregate_rmon_stats(struct net_device *dev,
struct ethtool_rmon_stats *rmon_stats);
+bool ethtool_dev_mm_supported(struct net_device *dev);
#else
static inline int ethnl_cable_test_alloc(struct phy_device *phydev, u8 cmd)
@@ -112,5 +113,10 @@ ethtool_aggregate_rmon_stats(struct net_device *dev,
{
}
+static inline bool ethtool_dev_mm_supported(struct net_device *dev)
+{
+ return false;
+}
+
#endif /* IS_ENABLED(CONFIG_ETHTOOL_NETLINK) */
#endif /* _LINUX_ETHTOOL_NETLINK_H_ */
diff --git a/include/linux/ieee80211.h b/include/linux/ieee80211.h
index 2463bdd2..c4cf296 100644
--- a/include/linux/ieee80211.h
+++ b/include/linux/ieee80211.h
@@ -9,7 +9,7 @@
* Copyright (c) 2006, Michael Wu <flamingice@sourmilk.net>
* Copyright (c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright (c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright (c) 2018 - 2022 Intel Corporation
+ * Copyright (c) 2018 - 2023 Intel Corporation
*/
#ifndef LINUX_IEEE80211_H
@@ -783,20 +783,6 @@ static inline bool ieee80211_is_any_nullfunc(__le16 fc)
}
/**
- * ieee80211_is_bufferable_mmpdu - check if frame is bufferable MMPDU
- * @fc: frame control field in little-endian byteorder
- */
-static inline bool ieee80211_is_bufferable_mmpdu(__le16 fc)
-{
- /* IEEE 802.11-2012, definition of "bufferable management frame";
- * note that this ignores the IBSS special case. */
- return ieee80211_is_mgmt(fc) &&
- (ieee80211_is_action(fc) ||
- ieee80211_is_disassoc(fc) ||
- ieee80211_is_deauth(fc));
-}
-
-/**
* ieee80211_is_first_frag - check if IEEE80211_SCTL_FRAG is not set
* @seq_ctrl: frame sequence control bytes in little-endian byteorder
*/
@@ -3557,11 +3543,6 @@ enum ieee80211_unprotected_wnm_actioncode {
WLAN_UNPROTECTED_WNM_ACTION_TIMING_MEASUREMENT_RESPONSE = 1,
};
-/* Public action codes */
-enum ieee80211_public_actioncode {
- WLAN_PUBLIC_ACTION_FTM_RESPONSE = 33,
-};
-
/* Security key length */
enum ieee80211_key_len {
WLAN_KEY_LEN_WEP40 = 5,
@@ -3653,7 +3634,7 @@ enum ieee80211_pub_actioncode {
WLAN_PUB_ACTION_NETWORK_CHANNEL_CONTROL = 30,
WLAN_PUB_ACTION_WHITE_SPACE_MAP_ANN = 31,
WLAN_PUB_ACTION_FTM_REQUEST = 32,
- WLAN_PUB_ACTION_FTM = 33,
+ WLAN_PUB_ACTION_FTM_RESPONSE = 33,
WLAN_PUB_ACTION_FILS_DISCOVERY = 34,
};
@@ -4138,6 +4119,44 @@ static inline u8 *ieee80211_get_DA(struct ieee80211_hdr *hdr)
}
/**
+ * ieee80211_is_bufferable_mmpdu - check if frame is bufferable MMPDU
+ * @skb: the skb to check, starting with the 802.11 header
+ */
+static inline bool ieee80211_is_bufferable_mmpdu(struct sk_buff *skb)
+{
+ struct ieee80211_mgmt *mgmt = (void *)skb->data;
+ __le16 fc = mgmt->frame_control;
+
+ /*
+ * IEEE 802.11 REVme D2.0 definition of bufferable MMPDU;
+ * note that this ignores the IBSS special case.
+ */
+ if (!ieee80211_is_mgmt(fc))
+ return false;
+
+ if (ieee80211_is_disassoc(fc) || ieee80211_is_deauth(fc))
+ return true;
+
+ if (!ieee80211_is_action(fc))
+ return false;
+
+ if (skb->len < offsetofend(typeof(*mgmt), u.action.u.ftm.action_code))
+ return true;
+
+ /* action frame - additionally check for non-bufferable FTM */
+
+ if (mgmt->u.action.category != WLAN_CATEGORY_PUBLIC &&
+ mgmt->u.action.category != WLAN_CATEGORY_PROTECTED_DUAL_OF_ACTION)
+ return true;
+
+ if (mgmt->u.action.u.ftm.action_code == WLAN_PUB_ACTION_FTM_REQUEST ||
+ mgmt->u.action.u.ftm.action_code == WLAN_PUB_ACTION_FTM_RESPONSE)
+ return false;
+
+ return true;
+}
+
+/**
* _ieee80211_is_robust_mgmt_frame - check if frame is a robust management frame
* @hdr: the frame (buffer must include at least the first octet of payload)
*/
@@ -4383,7 +4402,7 @@ static inline bool ieee80211_is_ftm(struct sk_buff *skb)
return false;
if (mgmt->u.action.u.ftm.action_code ==
- WLAN_PUBLIC_ACTION_FTM_RESPONSE &&
+ WLAN_PUB_ACTION_FTM_RESPONSE &&
skb->len >= offsetofend(typeof(*mgmt), u.action.u.ftm))
return true;
diff --git a/include/linux/if_bridge.h b/include/linux/if_bridge.h
index 1668ac4..3ff96ae 100644
--- a/include/linux/if_bridge.h
+++ b/include/linux/if_bridge.h
@@ -60,6 +60,7 @@ struct br_ip_list {
#define BR_TX_FWD_OFFLOAD BIT(20)
#define BR_PORT_LOCKED BIT(21)
#define BR_PORT_MAB BIT(22)
+#define BR_NEIGH_VLAN_SUPPRESS BIT(23)
#define BR_DEFAULT_AGEING_TIME (300 * HZ)
diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
index e38ae3c..30b1764 100644
--- a/include/linux/kmsan.h
+++ b/include/linux/kmsan.h
@@ -134,11 +134,12 @@ void kmsan_kfree_large(const void *ptr);
* @page_shift: page_shift passed to vmap_range_noflush().
*
* KMSAN maps shadow and origin pages of @pages into contiguous ranges in
- * vmalloc metadata address range.
+ * vmalloc metadata address range. Returns 0 on success, callers must check
+ * for non-zero return value.
*/
-void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
- pgprot_t prot, struct page **pages,
- unsigned int page_shift);
+int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages,
+ unsigned int page_shift);
/**
* kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap.
@@ -159,11 +160,12 @@ void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end);
* @page_shift: page_shift argument passed to vmap_range_noflush().
*
* KMSAN creates new metadata pages for the physical pages mapped into the
- * virtual memory.
+ * virtual memory. Returns 0 on success, callers must check for non-zero return
+ * value.
*/
-void kmsan_ioremap_page_range(unsigned long addr, unsigned long end,
- phys_addr_t phys_addr, pgprot_t prot,
- unsigned int page_shift);
+int kmsan_ioremap_page_range(unsigned long addr, unsigned long end,
+ phys_addr_t phys_addr, pgprot_t prot,
+ unsigned int page_shift);
/**
* kmsan_iounmap_page_range() - Notify KMSAN about a iounmap_page_range() call.
@@ -281,12 +283,13 @@ static inline void kmsan_kfree_large(const void *ptr)
{
}
-static inline void kmsan_vmap_pages_range_noflush(unsigned long start,
- unsigned long end,
- pgprot_t prot,
- struct page **pages,
- unsigned int page_shift)
+static inline int kmsan_vmap_pages_range_noflush(unsigned long start,
+ unsigned long end,
+ pgprot_t prot,
+ struct page **pages,
+ unsigned int page_shift)
{
+ return 0;
}
static inline void kmsan_vunmap_range_noflush(unsigned long start,
@@ -294,12 +297,12 @@ static inline void kmsan_vunmap_range_noflush(unsigned long start,
{
}
-static inline void kmsan_ioremap_page_range(unsigned long start,
- unsigned long end,
- phys_addr_t phys_addr,
- pgprot_t prot,
- unsigned int page_shift)
+static inline int kmsan_ioremap_page_range(unsigned long start,
+ unsigned long end,
+ phys_addr_t phys_addr, pgprot_t prot,
+ unsigned int page_shift)
{
+ return 0;
}
static inline void kmsan_iounmap_page_range(unsigned long start,
diff --git a/include/linux/leds.h b/include/linux/leds.h
index d71201a..aa48e64 100644
--- a/include/linux/leds.h
+++ b/include/linux/leds.h
@@ -82,7 +82,15 @@ struct led_init_data {
bool devname_mandatory;
};
+#if IS_ENABLED(CONFIG_NEW_LEDS)
enum led_default_state led_init_default_state_get(struct fwnode_handle *fwnode);
+#else
+static inline enum led_default_state
+led_init_default_state_get(struct fwnode_handle *fwnode)
+{
+ return LEDS_DEFSTATE_OFF;
+}
+#endif
struct led_hw_trigger_type {
int dummy;
@@ -217,9 +225,19 @@ static inline int led_classdev_register(struct device *parent,
return led_classdev_register_ext(parent, led_cdev, NULL);
}
+#if IS_ENABLED(CONFIG_LEDS_CLASS)
int devm_led_classdev_register_ext(struct device *parent,
struct led_classdev *led_cdev,
struct led_init_data *init_data);
+#else
+static inline int
+devm_led_classdev_register_ext(struct device *parent,
+ struct led_classdev *led_cdev,
+ struct led_init_data *init_data)
+{
+ return 0;
+}
+#endif
static inline int devm_led_classdev_register(struct device *parent,
struct led_classdev *led_cdev)
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index 10ec6dc..c0af74e 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -443,6 +443,8 @@ enum {
MLX5_OPCODE_UMR = 0x25,
+ MLX5_OPCODE_FLOW_TBL_ACCESS = 0x2c,
+
MLX5_OPCODE_ACCESS_ASO = 0x2d,
};
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index f243bd1..4c3636d 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -751,6 +751,7 @@ enum {
struct mlx5_profile {
u64 mask;
u8 log_max_qp;
+ u8 num_cmd_caches;
struct {
int size;
int limit;
@@ -1214,11 +1215,6 @@ static inline bool mlx5_core_is_vf(const struct mlx5_core_dev *dev)
return dev->coredev_type == MLX5_COREDEV_VF;
}
-static inline bool mlx5_core_is_management_pf(const struct mlx5_core_dev *dev)
-{
- return MLX5_CAP_GEN(dev, num_ports) == 1 && !MLX5_CAP_GEN(dev, native_port_num);
-}
-
static inline bool mlx5_core_is_ecpf(const struct mlx5_core_dev *dev)
{
return dev->caps.embedded_cpu;
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index e47d6c5..20d00e0 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -78,12 +78,15 @@ enum {
enum {
MLX5_OBJ_TYPE_SW_ICM = 0x0008,
+ MLX5_OBJ_TYPE_HEADER_MODIFY_ARGUMENT = 0x23,
};
enum {
MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM = (1ULL << MLX5_OBJ_TYPE_SW_ICM),
MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT = (1ULL << 11),
MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q = (1ULL << 13),
+ MLX5_GENERAL_OBJ_TYPES_CAP_HEADER_MODIFY_ARGUMENT =
+ (1ULL << MLX5_OBJ_TYPE_HEADER_MODIFY_ARGUMENT),
MLX5_GENERAL_OBJ_TYPES_CAP_MACSEC_OFFLOAD = (1ULL << 39),
};
@@ -321,6 +324,10 @@ enum {
MLX5_FT_NIC_TX_RDMA_2_NIC_TX = BIT(1),
};
+enum {
+ MLX5_CMD_OP_MOD_UPDATE_HEADER_MODIFY_ARGUMENT = 0x1,
+};
+
struct mlx5_ifc_flow_table_fields_supported_bits {
u8 outer_dmac[0x1];
u8 outer_smac[0x1];
@@ -456,9 +463,11 @@ struct mlx5_ifc_flow_table_prop_layout_bits {
u8 max_ft_level[0x8];
u8 reformat_add_esp_trasport[0x1];
- u8 reserved_at_41[0x2];
+ u8 reformat_l2_to_l3_esp_tunnel[0x1];
+ u8 reserved_at_42[0x1];
u8 reformat_del_esp_trasport[0x1];
- u8 reserved_at_44[0x2];
+ u8 reformat_l3_esp_tunnel_to_l2[0x1];
+ u8 reserved_at_45[0x1];
u8 execute_aso[0x1];
u8 reserved_at_47[0x19];
@@ -880,7 +889,12 @@ enum {
struct mlx5_ifc_flow_table_eswitch_cap_bits {
u8 fdb_to_vport_reg_c_id[0x8];
- u8 reserved_at_8[0xd];
+ u8 reserved_at_8[0x5];
+ u8 fdb_uplink_hairpin[0x1];
+ u8 fdb_multi_path_any_table_limit_regc[0x1];
+ u8 reserved_at_f[0x3];
+ u8 fdb_multi_path_any_table[0x1];
+ u8 reserved_at_13[0x2];
u8 fdb_modify_header_fwd_to_table[0x1];
u8 fdb_ipv4_ttl_modify[0x1];
u8 flow_source[0x1];
@@ -1922,7 +1936,14 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 reserved_at_750[0x4];
u8 max_dynamic_vf_msix_table_size[0xc];
- u8 reserved_at_760[0x20];
+ u8 reserved_at_760[0x3];
+ u8 log_max_num_header_modify_argument[0x5];
+ u8 reserved_at_768[0x4];
+ u8 log_header_modify_argument_granularity[0x4];
+ u8 reserved_at_770[0x3];
+ u8 log_header_modify_argument_max_alloc[0x5];
+ u8 reserved_at_778[0x8];
+
u8 vhca_tunnel_commands[0x40];
u8 match_definer_format_supported[0x40];
};
@@ -6356,6 +6377,18 @@ struct mlx5_ifc_general_obj_out_cmd_hdr_bits {
u8 reserved_at_60[0x20];
};
+struct mlx5_ifc_modify_header_arg_bits {
+ u8 reserved_at_0[0x80];
+
+ u8 reserved_at_80[0x8];
+ u8 access_pd[0x18];
+};
+
+struct mlx5_ifc_create_modify_header_arg_in_bits {
+ struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr;
+ struct mlx5_ifc_modify_header_arg_bits arg;
+};
+
struct mlx5_ifc_create_match_definer_in_bits {
struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr;
@@ -6599,7 +6632,9 @@ enum mlx5_reformat_ctx_type {
MLX5_REFORMAT_TYPE_L3_TUNNEL_TO_L2 = 0x3,
MLX5_REFORMAT_TYPE_L2_TO_L3_TUNNEL = 0x4,
MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV4 = 0x5,
+ MLX5_REFORMAT_TYPE_L2_TO_L3_ESP_TUNNEL = 0x6,
MLX5_REFORMAT_TYPE_DEL_ESP_TRANSPORT = 0x8,
+ MLX5_REFORMAT_TYPE_L3_ESP_TUNNEL_TO_L2 = 0x9,
MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV6 = 0xb,
MLX5_REFORMAT_TYPE_INSERT_HDR = 0xf,
MLX5_REFORMAT_TYPE_REMOVE_HDR = 0x10,
diff --git a/include/linux/mlx5/qp.h b/include/linux/mlx5/qp.h
index df55fbb..bd53cf4 100644
--- a/include/linux/mlx5/qp.h
+++ b/include/linux/mlx5/qp.h
@@ -499,6 +499,16 @@ struct mlx5_stride_block_ctrl_seg {
__be16 num_entries;
};
+struct mlx5_wqe_flow_update_ctrl_seg {
+ __be32 flow_idx_update;
+ __be32 dest_handle;
+ u8 reserved0[40];
+};
+
+struct mlx5_wqe_header_modify_argument_update_seg {
+ u8 argument_list[64];
+};
+
struct mlx5_core_qp {
struct mlx5_core_rsc_common common; /* must be first */
void (*event) (struct mlx5_core_qp *, int);
diff --git a/include/linux/mmc/sdio_ids.h b/include/linux/mmc/sdio_ids.h
index 0e4ef9c..c653acc 100644
--- a/include/linux/mmc/sdio_ids.h
+++ b/include/linux/mmc/sdio_ids.h
@@ -74,10 +74,13 @@
#define SDIO_DEVICE_ID_BROADCOM_43362 0xa962
#define SDIO_DEVICE_ID_BROADCOM_43364 0xa9a4
#define SDIO_DEVICE_ID_BROADCOM_43430 0xa9a6
-#define SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439 0xa9af
+#define SDIO_DEVICE_ID_BROADCOM_43439 0xa9af
#define SDIO_DEVICE_ID_BROADCOM_43455 0xa9bf
#define SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752 0xaae8
+#define SDIO_VENDOR_ID_CYPRESS 0x04b4
+#define SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439 0xbd3d
+
#define SDIO_VENDOR_ID_MARVELL 0x02df
#define SDIO_DEVICE_ID_MARVELL_LIBERTAS 0x9103
#define SDIO_DEVICE_ID_MARVELL_8688_WLAN 0x9104
@@ -112,6 +115,15 @@
#define SDIO_VENDOR_ID_MICROCHIP_WILC 0x0296
#define SDIO_DEVICE_ID_MICROCHIP_WILC1000 0x5347
+#define SDIO_VENDOR_ID_REALTEK 0x024c
+#define SDIO_DEVICE_ID_REALTEK_RTW8723BS 0xb723
+#define SDIO_DEVICE_ID_REALTEK_RTW8821BS 0xb821
+#define SDIO_DEVICE_ID_REALTEK_RTW8822BS 0xb822
+#define SDIO_DEVICE_ID_REALTEK_RTW8821CS 0xc821
+#define SDIO_DEVICE_ID_REALTEK_RTW8822CS 0xc822
+#define SDIO_DEVICE_ID_REALTEK_RTW8723DS 0xd723
+#define SDIO_DEVICE_ID_REALTEK_RTW8821DS 0xd821
+
#define SDIO_VENDOR_ID_SIANO 0x039a
#define SDIO_DEVICE_ID_SIANO_NOVA_B0 0x0201
#define SDIO_DEVICE_ID_SIANO_NICE 0x0202
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 96d27d5..a6a3e94 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -52,7 +52,7 @@
#include <linux/rbtree.h>
#include <net/net_trackers.h>
#include <net/net_debug.h>
-#include <net/dropreason.h>
+#include <net/dropreason-core.h>
struct netpoll_info;
struct device;
@@ -360,8 +360,11 @@ struct napi_struct {
unsigned long gro_bitmask;
int (*poll)(struct napi_struct *, int);
#ifdef CONFIG_NETPOLL
+ /* CPU actively polling if netpoll is configured */
int poll_owner;
#endif
+ /* CPU on which NAPI has been scheduled for processing */
+ int list_owner;
struct net_device *dev;
struct gro_list gro_hash[GRO_HASH_BUCKETS];
struct sk_buff *skb;
diff --git a/include/linux/pds/pds_adminq.h b/include/linux/pds/pds_adminq.h
new file mode 100644
index 0000000..98a60ce
--- /dev/null
+++ b/include/linux/pds/pds_adminq.h
@@ -0,0 +1,647 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#ifndef _PDS_CORE_ADMINQ_H_
+#define _PDS_CORE_ADMINQ_H_
+
+#define PDSC_ADMINQ_MAX_POLL_INTERVAL 256
+
+enum pds_core_adminq_flags {
+ PDS_AQ_FLAG_FASTPOLL = BIT(1), /* completion poll at 1ms */
+};
+
+/*
+ * enum pds_core_adminq_opcode - AdminQ command opcodes
+ * These commands are only processed on AdminQ, not available in devcmd
+ */
+enum pds_core_adminq_opcode {
+ PDS_AQ_CMD_NOP = 0,
+
+ /* Client control */
+ PDS_AQ_CMD_CLIENT_REG = 6,
+ PDS_AQ_CMD_CLIENT_UNREG = 7,
+ PDS_AQ_CMD_CLIENT_CMD = 8,
+
+ /* LIF commands */
+ PDS_AQ_CMD_LIF_IDENTIFY = 20,
+ PDS_AQ_CMD_LIF_INIT = 21,
+ PDS_AQ_CMD_LIF_RESET = 22,
+ PDS_AQ_CMD_LIF_GETATTR = 23,
+ PDS_AQ_CMD_LIF_SETATTR = 24,
+ PDS_AQ_CMD_LIF_SETPHC = 25,
+
+ PDS_AQ_CMD_RX_MODE_SET = 30,
+ PDS_AQ_CMD_RX_FILTER_ADD = 31,
+ PDS_AQ_CMD_RX_FILTER_DEL = 32,
+
+ /* Queue commands */
+ PDS_AQ_CMD_Q_IDENTIFY = 39,
+ PDS_AQ_CMD_Q_INIT = 40,
+ PDS_AQ_CMD_Q_CONTROL = 41,
+
+ /* SR/IOV commands */
+ PDS_AQ_CMD_VF_GETATTR = 60,
+ PDS_AQ_CMD_VF_SETATTR = 61,
+};
+
+/*
+ * enum pds_core_notifyq_opcode - NotifyQ event codes
+ */
+enum pds_core_notifyq_opcode {
+ PDS_EVENT_LINK_CHANGE = 1,
+ PDS_EVENT_RESET = 2,
+ PDS_EVENT_XCVR = 5,
+ PDS_EVENT_CLIENT = 6,
+};
+
+#define PDS_COMP_COLOR_MASK 0x80
+
+/**
+ * struct pds_core_notifyq_event - Generic event reporting structure
+ * @eid: event number
+ * @ecode: event code
+ *
+ * This is the generic event report struct from which the other
+ * actual events will be formed.
+ */
+struct pds_core_notifyq_event {
+ __le64 eid;
+ __le16 ecode;
+};
+
+/**
+ * struct pds_core_link_change_event - Link change event notification
+ * @eid: event number
+ * @ecode: event code = PDS_EVENT_LINK_CHANGE
+ * @link_status: link up/down, with error bits
+ * @link_speed: speed of the network link
+ *
+ * Sent when the network link state changes between UP and DOWN
+ */
+struct pds_core_link_change_event {
+ __le64 eid;
+ __le16 ecode;
+ __le16 link_status;
+ __le32 link_speed; /* units of 1Mbps: e.g. 10000 = 10Gbps */
+};
+
+/**
+ * struct pds_core_reset_event - Reset event notification
+ * @eid: event number
+ * @ecode: event code = PDS_EVENT_RESET
+ * @reset_code: reset type
+ * @state: 0=pending, 1=complete, 2=error
+ *
+ * Sent when the NIC or some subsystem is going to be or
+ * has been reset.
+ */
+struct pds_core_reset_event {
+ __le64 eid;
+ __le16 ecode;
+ u8 reset_code;
+ u8 state;
+};
+
+/**
+ * struct pds_core_client_event - Client event notification
+ * @eid: event number
+ * @ecode: event code = PDS_EVENT_CLIENT
+ * @client_id: client to sent event to
+ * @client_event: wrapped event struct for the client
+ *
+ * Sent when an event needs to be passed on to a client
+ */
+struct pds_core_client_event {
+ __le64 eid;
+ __le16 ecode;
+ __le16 client_id;
+ u8 client_event[54];
+};
+
+/**
+ * struct pds_core_notifyq_cmd - Placeholder for building qcq
+ * @data: anonymous field for building the qcq
+ */
+struct pds_core_notifyq_cmd {
+ __le32 data; /* Not used but needed for qcq structure */
+};
+
+/*
+ * union pds_core_notifyq_comp - Overlay of notifyq event structures
+ */
+union pds_core_notifyq_comp {
+ struct {
+ __le64 eid;
+ __le16 ecode;
+ };
+ struct pds_core_notifyq_event event;
+ struct pds_core_link_change_event link_change;
+ struct pds_core_reset_event reset;
+ u8 data[64];
+};
+
+#define PDS_DEVNAME_LEN 32
+/**
+ * struct pds_core_client_reg_cmd - Register a new client with DSC
+ * @opcode: opcode PDS_AQ_CMD_CLIENT_REG
+ * @rsvd: word boundary padding
+ * @devname: text name of client device
+ * @vif_type: what type of device (enum pds_core_vif_types)
+ *
+ * Tell the DSC of the new client, and receive a client_id from DSC.
+ */
+struct pds_core_client_reg_cmd {
+ u8 opcode;
+ u8 rsvd[3];
+ char devname[PDS_DEVNAME_LEN];
+ u8 vif_type;
+};
+
+/**
+ * struct pds_core_client_reg_comp - Client registration completion
+ * @status: Status of the command (enum pdc_core_status_code)
+ * @rsvd: Word boundary padding
+ * @comp_index: Index in the descriptor ring for which this is the completion
+ * @client_id: New id assigned by DSC
+ * @rsvd1: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_client_reg_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ __le16 client_id;
+ u8 rsvd1[9];
+ u8 color;
+};
+
+/**
+ * struct pds_core_client_unreg_cmd - Unregister a client from DSC
+ * @opcode: opcode PDS_AQ_CMD_CLIENT_UNREG
+ * @rsvd: word boundary padding
+ * @client_id: id of client being removed
+ *
+ * Tell the DSC this client is going away and remove its context
+ * This uses the generic completion.
+ */
+struct pds_core_client_unreg_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 client_id;
+};
+
+/**
+ * struct pds_core_client_request_cmd - Pass along a wrapped client AdminQ cmd
+ * @opcode: opcode PDS_AQ_CMD_CLIENT_CMD
+ * @rsvd: word boundary padding
+ * @client_id: id of client being removed
+ * @client_cmd: the wrapped client command
+ *
+ * Proxy post an adminq command for the client.
+ * This uses the generic completion.
+ */
+struct pds_core_client_request_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 client_id;
+ u8 client_cmd[60];
+};
+
+#define PDS_CORE_MAX_FRAGS 16
+
+#define PDS_CORE_QCQ_F_INITED BIT(0)
+#define PDS_CORE_QCQ_F_SG BIT(1)
+#define PDS_CORE_QCQ_F_INTR BIT(2)
+#define PDS_CORE_QCQ_F_TX_STATS BIT(3)
+#define PDS_CORE_QCQ_F_RX_STATS BIT(4)
+#define PDS_CORE_QCQ_F_NOTIFYQ BIT(5)
+#define PDS_CORE_QCQ_F_CMB_RINGS BIT(6)
+#define PDS_CORE_QCQ_F_CORE BIT(7)
+
+enum pds_core_lif_type {
+ PDS_CORE_LIF_TYPE_DEFAULT = 0,
+};
+
+/**
+ * union pds_core_lif_config - LIF configuration
+ * @state: LIF state (enum pds_core_lif_state)
+ * @rsvd: Word boundary padding
+ * @name: LIF name
+ * @rsvd2: Word boundary padding
+ * @features: LIF features active (enum pds_core_hw_features)
+ * @queue_count: Queue counts per queue-type
+ * @words: Full union buffer size
+ */
+union pds_core_lif_config {
+ struct {
+ u8 state;
+ u8 rsvd[3];
+ char name[PDS_CORE_IFNAMSIZ];
+ u8 rsvd2[12];
+ __le64 features;
+ __le32 queue_count[PDS_CORE_QTYPE_MAX];
+ } __packed;
+ __le32 words[64];
+};
+
+/**
+ * struct pds_core_lif_status - LIF status register
+ * @eid: most recent NotifyQ event id
+ * @rsvd: full struct size
+ */
+struct pds_core_lif_status {
+ __le64 eid;
+ u8 rsvd[56];
+};
+
+/**
+ * struct pds_core_lif_info - LIF info structure
+ * @config: LIF configuration structure
+ * @status: LIF status structure
+ */
+struct pds_core_lif_info {
+ union pds_core_lif_config config;
+ struct pds_core_lif_status status;
+};
+
+/**
+ * struct pds_core_lif_identity - LIF identity information (type-specific)
+ * @features: LIF features (see enum pds_core_hw_features)
+ * @version: Identify structure version
+ * @hw_index: LIF hardware index
+ * @rsvd: Word boundary padding
+ * @max_nb_sessions: Maximum number of sessions supported
+ * @rsvd2: buffer padding
+ * @config: LIF config struct with features, q counts
+ */
+struct pds_core_lif_identity {
+ __le64 features;
+ u8 version;
+ u8 hw_index;
+ u8 rsvd[2];
+ __le32 max_nb_sessions;
+ u8 rsvd2[120];
+ union pds_core_lif_config config;
+};
+
+/**
+ * struct pds_core_lif_identify_cmd - Get LIF identity info command
+ * @opcode: Opcode PDS_AQ_CMD_LIF_IDENTIFY
+ * @type: LIF type (enum pds_core_lif_type)
+ * @client_id: Client identifier
+ * @ver: Version of identify returned by device
+ * @rsvd: Word boundary padding
+ * @ident_pa: DMA address to receive identity info
+ *
+ * Firmware will copy LIF identity data (struct pds_core_lif_identity)
+ * into the buffer address given.
+ */
+struct pds_core_lif_identify_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 client_id;
+ u8 ver;
+ u8 rsvd[3];
+ __le64 ident_pa;
+};
+
+/**
+ * struct pds_core_lif_identify_comp - LIF identify command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @ver: Version of identify returned by device
+ * @bytes: Bytes copied into the buffer
+ * @rsvd: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_lif_identify_comp {
+ u8 status;
+ u8 ver;
+ __le16 bytes;
+ u8 rsvd[11];
+ u8 color;
+};
+
+/**
+ * struct pds_core_lif_init_cmd - LIF init command
+ * @opcode: Opcode PDS_AQ_CMD_LIF_INIT
+ * @type: LIF type (enum pds_core_lif_type)
+ * @client_id: Client identifier
+ * @rsvd: Word boundary padding
+ * @info_pa: Destination address for LIF info (struct pds_core_lif_info)
+ */
+struct pds_core_lif_init_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 client_id;
+ __le32 rsvd;
+ __le64 info_pa;
+};
+
+/**
+ * struct pds_core_lif_init_comp - LIF init command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word boundary padding
+ * @hw_index: Hardware index of the initialized LIF
+ * @rsvd1: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_lif_init_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 hw_index;
+ u8 rsvd1[11];
+ u8 color;
+};
+
+/**
+ * struct pds_core_lif_reset_cmd - LIF reset command
+ * Will reset only the specified LIF.
+ * @opcode: Opcode PDS_AQ_CMD_LIF_RESET
+ * @rsvd: Word boundary padding
+ * @client_id: Client identifier
+ */
+struct pds_core_lif_reset_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 client_id;
+};
+
+/**
+ * enum pds_core_lif_attr - List of LIF attributes
+ * @PDS_CORE_LIF_ATTR_STATE: LIF state attribute
+ * @PDS_CORE_LIF_ATTR_NAME: LIF name attribute
+ * @PDS_CORE_LIF_ATTR_FEATURES: LIF features attribute
+ * @PDS_CORE_LIF_ATTR_STATS_CTRL: LIF statistics control attribute
+ */
+enum pds_core_lif_attr {
+ PDS_CORE_LIF_ATTR_STATE = 0,
+ PDS_CORE_LIF_ATTR_NAME = 1,
+ PDS_CORE_LIF_ATTR_FEATURES = 4,
+ PDS_CORE_LIF_ATTR_STATS_CTRL = 6,
+};
+
+/**
+ * struct pds_core_lif_setattr_cmd - Set LIF attributes on the NIC
+ * @opcode: Opcode PDS_AQ_CMD_LIF_SETATTR
+ * @attr: Attribute type (enum pds_core_lif_attr)
+ * @client_id: Client identifier
+ * @state: LIF state (enum pds_core_lif_state)
+ * @name: The name string, 0 terminated
+ * @features: Features (enum pds_core_hw_features)
+ * @stats_ctl: Stats control commands (enum pds_core_stats_ctl_cmd)
+ * @rsvd: Command Buffer padding
+ */
+struct pds_core_lif_setattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 client_id;
+ union {
+ u8 state;
+ char name[PDS_CORE_IFNAMSIZ];
+ __le64 features;
+ u8 stats_ctl;
+ u8 rsvd[60];
+ } __packed;
+};
+
+/**
+ * struct pds_core_lif_setattr_comp - LIF set attr command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word boundary padding
+ * @comp_index: Index in the descriptor ring for which this is the completion
+ * @features: Features (enum pds_core_hw_features)
+ * @rsvd2: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_lif_setattr_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ union {
+ __le64 features;
+ u8 rsvd2[11];
+ } __packed;
+ u8 color;
+};
+
+/**
+ * struct pds_core_lif_getattr_cmd - Get LIF attributes from the NIC
+ * @opcode: Opcode PDS_AQ_CMD_LIF_GETATTR
+ * @attr: Attribute type (enum pds_core_lif_attr)
+ * @client_id: Client identifier
+ */
+struct pds_core_lif_getattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 client_id;
+};
+
+/**
+ * struct pds_core_lif_getattr_comp - LIF get attr command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word boundary padding
+ * @comp_index: Index in the descriptor ring for which this is the completion
+ * @state: LIF state (enum pds_core_lif_state)
+ * @name: LIF name string, 0 terminated
+ * @features: Features (enum pds_core_hw_features)
+ * @rsvd2: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_lif_getattr_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ union {
+ u8 state;
+ __le64 features;
+ u8 rsvd2[11];
+ } __packed;
+ u8 color;
+};
+
+/**
+ * union pds_core_q_identity - Queue identity information
+ * @version: Queue type version that can be used with FW
+ * @supported: Bitfield of queue versions, first bit = ver 0
+ * @rsvd: Word boundary padding
+ * @features: Queue features
+ * @desc_sz: Descriptor size
+ * @comp_sz: Completion descriptor size
+ * @rsvd2: Word boundary padding
+ */
+struct pds_core_q_identity {
+ u8 version;
+ u8 supported;
+ u8 rsvd[6];
+#define PDS_CORE_QIDENT_F_CQ 0x01 /* queue has completion ring */
+ __le64 features;
+ __le16 desc_sz;
+ __le16 comp_sz;
+ u8 rsvd2[6];
+};
+
+/**
+ * struct pds_core_q_identify_cmd - queue identify command
+ * @opcode: Opcode PDS_AQ_CMD_Q_IDENTIFY
+ * @type: Logical queue type (enum pds_core_logical_qtype)
+ * @client_id: Client identifier
+ * @ver: Highest queue type version that the driver supports
+ * @rsvd: Word boundary padding
+ * @ident_pa: DMA address to receive the data (struct pds_core_q_identity)
+ */
+struct pds_core_q_identify_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 client_id;
+ u8 ver;
+ u8 rsvd[3];
+ __le64 ident_pa;
+};
+
+/**
+ * struct pds_core_q_identify_comp - queue identify command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word boundary padding
+ * @comp_index: Index in the descriptor ring for which this is the completion
+ * @ver: Queue type version that can be used with FW
+ * @rsvd1: Word boundary padding
+ * @color: Color bit
+ */
+struct pds_core_q_identify_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ u8 ver;
+ u8 rsvd1[10];
+ u8 color;
+};
+
+/**
+ * struct pds_core_q_init_cmd - Queue init command
+ * @opcode: Opcode PDS_AQ_CMD_Q_INIT
+ * @type: Logical queue type
+ * @client_id: Client identifier
+ * @ver: Queue type version
+ * @rsvd: Word boundary padding
+ * @index: (LIF, qtype) relative admin queue index
+ * @intr_index: Interrupt control register index, or Event queue index
+ * @pid: Process ID
+ * @flags:
+ * IRQ: Interrupt requested on completion
+ * ENA: Enable the queue. If ENA=0 the queue is initialized
+ * but remains disabled, to be later enabled with the
+ * Queue Enable command. If ENA=1, then queue is
+ * initialized and then enabled.
+ * @cos: Class of service for this queue
+ * @ring_size: Queue ring size, encoded as a log2(size), in
+ * number of descriptors. The actual ring size is
+ * (1 << ring_size). For example, to select a ring size
+ * of 64 descriptors write ring_size = 6. The minimum
+ * ring_size value is 2 for a ring of 4 descriptors.
+ * The maximum ring_size value is 12 for a ring of 4k
+ * descriptors. Values of ring_size <2 and >12 are
+ * reserved.
+ * @ring_base: Queue ring base address
+ * @cq_ring_base: Completion queue ring base address
+ */
+struct pds_core_q_init_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 client_id;
+ u8 ver;
+ u8 rsvd[3];
+ __le32 index;
+ __le16 pid;
+ __le16 intr_index;
+ __le16 flags;
+#define PDS_CORE_QINIT_F_IRQ 0x01 /* Request interrupt on completion */
+#define PDS_CORE_QINIT_F_ENA 0x02 /* Enable the queue */
+ u8 cos;
+#define PDS_CORE_QSIZE_MIN_LG2 2
+#define PDS_CORE_QSIZE_MAX_LG2 12
+ u8 ring_size;
+ __le64 ring_base;
+ __le64 cq_ring_base;
+} __packed;
+
+/**
+ * struct pds_core_q_init_comp - Queue init command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word boundary padding
+ * @comp_index: Index in the descriptor ring for which this is the completion
+ * @hw_index: Hardware Queue ID
+ * @hw_type: Hardware Queue type
+ * @rsvd2: Word boundary padding
+ * @color: Color
+ */
+struct pds_core_q_init_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ __le32 hw_index;
+ u8 hw_type;
+ u8 rsvd2[6];
+ u8 color;
+};
+
+union pds_core_adminq_cmd {
+ u8 opcode;
+ u8 bytes[64];
+
+ struct pds_core_client_reg_cmd client_reg;
+ struct pds_core_client_unreg_cmd client_unreg;
+ struct pds_core_client_request_cmd client_request;
+
+ struct pds_core_lif_identify_cmd lif_ident;
+ struct pds_core_lif_init_cmd lif_init;
+ struct pds_core_lif_reset_cmd lif_reset;
+ struct pds_core_lif_setattr_cmd lif_setattr;
+ struct pds_core_lif_getattr_cmd lif_getattr;
+
+ struct pds_core_q_identify_cmd q_ident;
+ struct pds_core_q_init_cmd q_init;
+};
+
+union pds_core_adminq_comp {
+ struct {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ u8 rsvd2[11];
+ u8 color;
+ };
+ u32 words[4];
+
+ struct pds_core_client_reg_comp client_reg;
+
+ struct pds_core_lif_identify_comp lif_ident;
+ struct pds_core_lif_init_comp lif_init;
+ struct pds_core_lif_setattr_comp lif_setattr;
+ struct pds_core_lif_getattr_comp lif_getattr;
+
+ struct pds_core_q_identify_comp q_ident;
+ struct pds_core_q_init_comp q_init;
+};
+
+#ifndef __CHECKER__
+static_assert(sizeof(union pds_core_adminq_cmd) == 64);
+static_assert(sizeof(union pds_core_adminq_comp) == 16);
+static_assert(sizeof(union pds_core_notifyq_comp) == 64);
+#endif /* __CHECKER__ */
+
+/* The color bit is a 'done' bit for the completion descriptors
+ * where the meaning alternates between '1' and '0' for alternating
+ * passes through the completion descriptor ring.
+ */
+static inline bool pdsc_color_match(u8 color, bool done_color)
+{
+ return (!!(color & PDS_COMP_COLOR_MASK)) == done_color;
+}
+
+struct pdsc;
+int pdsc_adminq_post(struct pdsc *pdsc,
+ union pds_core_adminq_cmd *cmd,
+ union pds_core_adminq_comp *comp,
+ bool fast_poll);
+
+#endif /* _PDS_CORE_ADMINQ_H_ */
diff --git a/include/linux/pds/pds_auxbus.h b/include/linux/pds/pds_auxbus.h
new file mode 100644
index 0000000..214ef12
--- /dev/null
+++ b/include/linux/pds/pds_auxbus.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#ifndef _PDSC_AUXBUS_H_
+#define _PDSC_AUXBUS_H_
+
+#include <linux/auxiliary_bus.h>
+
+struct pds_auxiliary_dev {
+ struct auxiliary_device aux_dev;
+ struct pci_dev *vf_pdev;
+ u16 client_id;
+};
+
+int pds_client_adminq_cmd(struct pds_auxiliary_dev *padev,
+ union pds_core_adminq_cmd *req,
+ size_t req_len,
+ union pds_core_adminq_comp *resp,
+ u64 flags);
+#endif /* _PDSC_AUXBUS_H_ */
diff --git a/include/linux/pds/pds_common.h b/include/linux/pds/pds_common.h
new file mode 100644
index 0000000..0603314
--- /dev/null
+++ b/include/linux/pds/pds_common.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) OR BSD-2-Clause */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _PDS_COMMON_H_
+#define _PDS_COMMON_H_
+
+#define PDS_CORE_DRV_NAME "pds_core"
+
+/* the device's internal addressing uses up to 52 bits */
+#define PDS_CORE_ADDR_LEN 52
+#define PDS_CORE_ADDR_MASK (BIT_ULL(PDS_ADDR_LEN) - 1)
+#define PDS_PAGE_SIZE 4096
+
+enum pds_core_driver_type {
+ PDS_DRIVER_LINUX = 1,
+ PDS_DRIVER_WIN = 2,
+ PDS_DRIVER_DPDK = 3,
+ PDS_DRIVER_FREEBSD = 4,
+ PDS_DRIVER_IPXE = 5,
+ PDS_DRIVER_ESXI = 6,
+};
+
+enum pds_core_vif_types {
+ PDS_DEV_TYPE_CORE = 0,
+ PDS_DEV_TYPE_VDPA = 1,
+ PDS_DEV_TYPE_VFIO = 2,
+ PDS_DEV_TYPE_ETH = 3,
+ PDS_DEV_TYPE_RDMA = 4,
+ PDS_DEV_TYPE_LM = 5,
+
+ /* new ones added before this line */
+ PDS_DEV_TYPE_MAX = 16 /* don't change - used in struct size */
+};
+
+#define PDS_DEV_TYPE_CORE_STR "Core"
+#define PDS_DEV_TYPE_VDPA_STR "vDPA"
+#define PDS_DEV_TYPE_VFIO_STR "VFio"
+#define PDS_DEV_TYPE_ETH_STR "Eth"
+#define PDS_DEV_TYPE_RDMA_STR "RDMA"
+#define PDS_DEV_TYPE_LM_STR "LM"
+
+#define PDS_CORE_IFNAMSIZ 16
+
+/**
+ * enum pds_core_logical_qtype - Logical Queue Types
+ * @PDS_CORE_QTYPE_ADMINQ: Administrative Queue
+ * @PDS_CORE_QTYPE_NOTIFYQ: Notify Queue
+ * @PDS_CORE_QTYPE_RXQ: Receive Queue
+ * @PDS_CORE_QTYPE_TXQ: Transmit Queue
+ * @PDS_CORE_QTYPE_EQ: Event Queue
+ * @PDS_CORE_QTYPE_MAX: Max queue type supported
+ */
+enum pds_core_logical_qtype {
+ PDS_CORE_QTYPE_ADMINQ = 0,
+ PDS_CORE_QTYPE_NOTIFYQ = 1,
+ PDS_CORE_QTYPE_RXQ = 2,
+ PDS_CORE_QTYPE_TXQ = 3,
+ PDS_CORE_QTYPE_EQ = 4,
+
+ PDS_CORE_QTYPE_MAX = 16 /* don't change - used in struct size */
+};
+
+int pdsc_register_notify(struct notifier_block *nb);
+void pdsc_unregister_notify(struct notifier_block *nb);
+void *pdsc_get_pf_struct(struct pci_dev *vf_pdev);
+int pds_client_register(struct pci_dev *pf_pdev, char *devname);
+int pds_client_unregister(struct pci_dev *pf_pdev, u16 client_id);
+#endif /* _PDS_COMMON_H_ */
diff --git a/include/linux/pds/pds_core_if.h b/include/linux/pds/pds_core_if.h
new file mode 100644
index 0000000..e838a2b
--- /dev/null
+++ b/include/linux/pds/pds_core_if.h
@@ -0,0 +1,571 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) OR BSD-2-Clause */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _PDS_CORE_IF_H_
+#define _PDS_CORE_IF_H_
+
+#define PCI_VENDOR_ID_PENSANDO 0x1dd8
+#define PCI_DEVICE_ID_PENSANDO_CORE_PF 0x100c
+#define PCI_DEVICE_ID_VIRTIO_NET_TRANS 0x1000
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF 0x1003
+#define PCI_DEVICE_ID_PENSANDO_VDPA_VF 0x100b
+#define PDS_CORE_BARS_MAX 4
+#define PDS_CORE_PCI_BAR_DBELL 1
+
+/* Bar0 */
+#define PDS_CORE_DEV_INFO_SIGNATURE 0x44455649 /* 'DEVI' */
+#define PDS_CORE_BAR0_SIZE 0x8000
+#define PDS_CORE_BAR0_DEV_INFO_REGS_OFFSET 0x0000
+#define PDS_CORE_BAR0_DEV_CMD_REGS_OFFSET 0x0800
+#define PDS_CORE_BAR0_DEV_CMD_DATA_REGS_OFFSET 0x0c00
+#define PDS_CORE_BAR0_INTR_STATUS_OFFSET 0x1000
+#define PDS_CORE_BAR0_INTR_CTRL_OFFSET 0x2000
+#define PDS_CORE_DEV_CMD_DONE 0x00000001
+
+#define PDS_CORE_DEVCMD_TIMEOUT 5
+
+#define PDS_CORE_CLIENT_ID 0
+#define PDS_CORE_ASIC_TYPE_CAPRI 0
+
+/*
+ * enum pds_core_cmd_opcode - Device commands
+ */
+enum pds_core_cmd_opcode {
+ /* Core init */
+ PDS_CORE_CMD_NOP = 0,
+ PDS_CORE_CMD_IDENTIFY = 1,
+ PDS_CORE_CMD_RESET = 2,
+ PDS_CORE_CMD_INIT = 3,
+
+ PDS_CORE_CMD_FW_DOWNLOAD = 4,
+ PDS_CORE_CMD_FW_CONTROL = 5,
+
+ /* SR/IOV commands */
+ PDS_CORE_CMD_VF_GETATTR = 60,
+ PDS_CORE_CMD_VF_SETATTR = 61,
+ PDS_CORE_CMD_VF_CTRL = 62,
+
+ /* Add commands before this line */
+ PDS_CORE_CMD_MAX,
+ PDS_CORE_CMD_COUNT
+};
+
+/*
+ * enum pds_core_status_code - Device command return codes
+ */
+enum pds_core_status_code {
+ PDS_RC_SUCCESS = 0, /* Success */
+ PDS_RC_EVERSION = 1, /* Incorrect version for request */
+ PDS_RC_EOPCODE = 2, /* Invalid cmd opcode */
+ PDS_RC_EIO = 3, /* I/O error */
+ PDS_RC_EPERM = 4, /* Permission denied */
+ PDS_RC_EQID = 5, /* Bad qid */
+ PDS_RC_EQTYPE = 6, /* Bad qtype */
+ PDS_RC_ENOENT = 7, /* No such element */
+ PDS_RC_EINTR = 8, /* operation interrupted */
+ PDS_RC_EAGAIN = 9, /* Try again */
+ PDS_RC_ENOMEM = 10, /* Out of memory */
+ PDS_RC_EFAULT = 11, /* Bad address */
+ PDS_RC_EBUSY = 12, /* Device or resource busy */
+ PDS_RC_EEXIST = 13, /* object already exists */
+ PDS_RC_EINVAL = 14, /* Invalid argument */
+ PDS_RC_ENOSPC = 15, /* No space left or alloc failure */
+ PDS_RC_ERANGE = 16, /* Parameter out of range */
+ PDS_RC_BAD_ADDR = 17, /* Descriptor contains a bad ptr */
+ PDS_RC_DEV_CMD = 18, /* Device cmd attempted on AdminQ */
+ PDS_RC_ENOSUPP = 19, /* Operation not supported */
+ PDS_RC_ERROR = 29, /* Generic error */
+ PDS_RC_ERDMA = 30, /* Generic RDMA error */
+ PDS_RC_EVFID = 31, /* VF ID does not exist */
+ PDS_RC_BAD_FW = 32, /* FW file is invalid or corrupted */
+ PDS_RC_ECLIENT = 33, /* No such client id */
+};
+
+/**
+ * struct pds_core_drv_identity - Driver identity information
+ * @drv_type: Driver type (enum pds_core_driver_type)
+ * @os_dist: OS distribution, numeric format
+ * @os_dist_str: OS distribution, string format
+ * @kernel_ver: Kernel version, numeric format
+ * @kernel_ver_str: Kernel version, string format
+ * @driver_ver_str: Driver version, string format
+ */
+struct pds_core_drv_identity {
+ __le32 drv_type;
+ __le32 os_dist;
+ char os_dist_str[128];
+ __le32 kernel_ver;
+ char kernel_ver_str[32];
+ char driver_ver_str[32];
+};
+
+#define PDS_DEV_TYPE_MAX 16
+/**
+ * struct pds_core_dev_identity - Device identity information
+ * @version: Version of device identify
+ * @type: Identify type (0 for now)
+ * @state: Device state
+ * @rsvd: Word boundary padding
+ * @nlifs: Number of LIFs provisioned
+ * @nintrs: Number of interrupts provisioned
+ * @ndbpgs_per_lif: Number of doorbell pages per LIF
+ * @intr_coal_mult: Interrupt coalescing multiplication factor
+ * Scale user-supplied interrupt coalescing
+ * value in usecs to device units using:
+ * device units = usecs * mult / div
+ * @intr_coal_div: Interrupt coalescing division factor
+ * Scale user-supplied interrupt coalescing
+ * value in usecs to device units using:
+ * device units = usecs * mult / div
+ * @vif_types: How many of each VIF device type is supported
+ */
+struct pds_core_dev_identity {
+ u8 version;
+ u8 type;
+ u8 state;
+ u8 rsvd;
+ __le32 nlifs;
+ __le32 nintrs;
+ __le32 ndbpgs_per_lif;
+ __le32 intr_coal_mult;
+ __le32 intr_coal_div;
+ __le16 vif_types[PDS_DEV_TYPE_MAX];
+};
+
+#define PDS_CORE_IDENTITY_VERSION_1 1
+
+/**
+ * struct pds_core_dev_identify_cmd - Driver/device identify command
+ * @opcode: Opcode PDS_CORE_CMD_IDENTIFY
+ * @ver: Highest version of identify supported by driver
+ *
+ * Expects to find driver identification info (struct pds_core_drv_identity)
+ * in cmd_regs->data. Driver should keep the devcmd interface locked
+ * while preparing the driver info.
+ */
+struct pds_core_dev_identify_cmd {
+ u8 opcode;
+ u8 ver;
+};
+
+/**
+ * struct pds_core_dev_identify_comp - Device identify command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @ver: Version of identify returned by device
+ *
+ * Device identification info (struct pds_core_dev_identity) can be found
+ * in cmd_regs->data. Driver should keep the devcmd interface locked
+ * while reading the results.
+ */
+struct pds_core_dev_identify_comp {
+ u8 status;
+ u8 ver;
+};
+
+/**
+ * struct pds_core_dev_reset_cmd - Device reset command
+ * @opcode: Opcode PDS_CORE_CMD_RESET
+ *
+ * Resets and clears all LIFs, VDevs, and VIFs on the device.
+ */
+struct pds_core_dev_reset_cmd {
+ u8 opcode;
+};
+
+/**
+ * struct pds_core_dev_reset_comp - Reset command completion
+ * @status: Status of the command (enum pds_core_status_code)
+ */
+struct pds_core_dev_reset_comp {
+ u8 status;
+};
+
+/*
+ * struct pds_core_dev_init_data - Pointers and info needed for the Core
+ * initialization PDS_CORE_CMD_INIT command. The in and out structs are
+ * overlays on the pds_core_dev_cmd_regs.data space for passing data down
+ * to the firmware on init, and then returning initialization results.
+ */
+struct pds_core_dev_init_data_in {
+ __le64 adminq_q_base;
+ __le64 adminq_cq_base;
+ __le64 notifyq_cq_base;
+ __le32 flags;
+ __le16 intr_index;
+ u8 adminq_ring_size;
+ u8 notifyq_ring_size;
+};
+
+struct pds_core_dev_init_data_out {
+ __le32 core_hw_index;
+ __le32 adminq_hw_index;
+ __le32 notifyq_hw_index;
+ u8 adminq_hw_type;
+ u8 notifyq_hw_type;
+};
+
+/**
+ * struct pds_core_dev_init_cmd - Core device initialize
+ * @opcode: opcode PDS_CORE_CMD_INIT
+ *
+ * Initializes the core device and sets up the AdminQ and NotifyQ.
+ * Expects to find initialization data (struct pds_core_dev_init_data_in)
+ * in cmd_regs->data. Driver should keep the devcmd interface locked
+ * while preparing the driver info.
+ */
+struct pds_core_dev_init_cmd {
+ u8 opcode;
+};
+
+/**
+ * struct pds_core_dev_init_comp - Core init completion
+ * @status: Status of the command (enum pds_core_status_code)
+ *
+ * Initialization result data (struct pds_core_dev_init_data_in)
+ * is found in cmd_regs->data.
+ */
+struct pds_core_dev_init_comp {
+ u8 status;
+};
+
+/**
+ * struct pds_core_fw_download_cmd - Firmware download command
+ * @opcode: opcode
+ * @rsvd: Word boundary padding
+ * @addr: DMA address of the firmware buffer
+ * @offset: offset of the firmware buffer within the full image
+ * @length: number of valid bytes in the firmware buffer
+ */
+struct pds_core_fw_download_cmd {
+ u8 opcode;
+ u8 rsvd[3];
+ __le32 offset;
+ __le64 addr;
+ __le32 length;
+};
+
+/**
+ * struct pds_core_fw_download_comp - Firmware download completion
+ * @status: Status of the command (enum pds_core_status_code)
+ */
+struct pds_core_fw_download_comp {
+ u8 status;
+};
+
+/**
+ * enum pds_core_fw_control_oper - FW control operations
+ * @PDS_CORE_FW_INSTALL_ASYNC: Install firmware asynchronously
+ * @PDS_CORE_FW_INSTALL_STATUS: Firmware installation status
+ * @PDS_CORE_FW_ACTIVATE_ASYNC: Activate firmware asynchronously
+ * @PDS_CORE_FW_ACTIVATE_STATUS: Firmware activate status
+ * @PDS_CORE_FW_UPDATE_CLEANUP: Cleanup any firmware update leftovers
+ * @PDS_CORE_FW_GET_BOOT: Return current active firmware slot
+ * @PDS_CORE_FW_SET_BOOT: Set active firmware slot for next boot
+ * @PDS_CORE_FW_GET_LIST: Return list of installed firmware images
+ */
+enum pds_core_fw_control_oper {
+ PDS_CORE_FW_INSTALL_ASYNC = 0,
+ PDS_CORE_FW_INSTALL_STATUS = 1,
+ PDS_CORE_FW_ACTIVATE_ASYNC = 2,
+ PDS_CORE_FW_ACTIVATE_STATUS = 3,
+ PDS_CORE_FW_UPDATE_CLEANUP = 4,
+ PDS_CORE_FW_GET_BOOT = 5,
+ PDS_CORE_FW_SET_BOOT = 6,
+ PDS_CORE_FW_GET_LIST = 7,
+};
+
+enum pds_core_fw_slot {
+ PDS_CORE_FW_SLOT_INVALID = 0,
+ PDS_CORE_FW_SLOT_A = 1,
+ PDS_CORE_FW_SLOT_B = 2,
+ PDS_CORE_FW_SLOT_GOLD = 3,
+};
+
+/**
+ * struct pds_core_fw_control_cmd - Firmware control command
+ * @opcode: opcode
+ * @rsvd: Word boundary padding
+ * @oper: firmware control operation (enum pds_core_fw_control_oper)
+ * @slot: slot to operate on (enum pds_core_fw_slot)
+ */
+struct pds_core_fw_control_cmd {
+ u8 opcode;
+ u8 rsvd[3];
+ u8 oper;
+ u8 slot;
+};
+
+/**
+ * struct pds_core_fw_control_comp - Firmware control copletion
+ * @status: Status of the command (enum pds_core_status_code)
+ * @rsvd: Word alignment space
+ * @slot: Slot number (enum pds_core_fw_slot)
+ * @rsvd1: Struct padding
+ * @color: Color bit
+ */
+struct pds_core_fw_control_comp {
+ u8 status;
+ u8 rsvd[3];
+ u8 slot;
+ u8 rsvd1[10];
+ u8 color;
+};
+
+struct pds_core_fw_name_info {
+#define PDS_CORE_FWSLOT_BUFLEN 8
+#define PDS_CORE_FWVERS_BUFLEN 32
+ char slotname[PDS_CORE_FWSLOT_BUFLEN];
+ char fw_version[PDS_CORE_FWVERS_BUFLEN];
+};
+
+struct pds_core_fw_list_info {
+#define PDS_CORE_FWVERS_LIST_LEN 16
+ u8 num_fw_slots;
+ struct pds_core_fw_name_info fw_names[PDS_CORE_FWVERS_LIST_LEN];
+} __packed;
+
+enum pds_core_vf_attr {
+ PDS_CORE_VF_ATTR_SPOOFCHK = 1,
+ PDS_CORE_VF_ATTR_TRUST = 2,
+ PDS_CORE_VF_ATTR_MAC = 3,
+ PDS_CORE_VF_ATTR_LINKSTATE = 4,
+ PDS_CORE_VF_ATTR_VLAN = 5,
+ PDS_CORE_VF_ATTR_RATE = 6,
+ PDS_CORE_VF_ATTR_STATSADDR = 7,
+};
+
+/**
+ * enum pds_core_vf_link_status - Virtual Function link status
+ * @PDS_CORE_VF_LINK_STATUS_AUTO: Use link state of the uplink
+ * @PDS_CORE_VF_LINK_STATUS_UP: Link always up
+ * @PDS_CORE_VF_LINK_STATUS_DOWN: Link always down
+ */
+enum pds_core_vf_link_status {
+ PDS_CORE_VF_LINK_STATUS_AUTO = 0,
+ PDS_CORE_VF_LINK_STATUS_UP = 1,
+ PDS_CORE_VF_LINK_STATUS_DOWN = 2,
+};
+
+/**
+ * struct pds_core_vf_setattr_cmd - Set VF attributes on the NIC
+ * @opcode: Opcode
+ * @attr: Attribute type (enum pds_core_vf_attr)
+ * @vf_index: VF index
+ * @macaddr: mac address
+ * @vlanid: vlan ID
+ * @maxrate: max Tx rate in Mbps
+ * @spoofchk: enable address spoof checking
+ * @trust: enable VF trust
+ * @linkstate: set link up or down
+ * @stats: stats addr struct
+ * @stats.pa: set DMA address for VF stats
+ * @stats.len: length of VF stats space
+ * @pad: force union to specific size
+ */
+struct pds_core_vf_setattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 vf_index;
+ union {
+ u8 macaddr[6];
+ __le16 vlanid;
+ __le32 maxrate;
+ u8 spoofchk;
+ u8 trust;
+ u8 linkstate;
+ struct {
+ __le64 pa;
+ __le32 len;
+ } stats;
+ u8 pad[60];
+ } __packed;
+};
+
+struct pds_core_vf_setattr_comp {
+ u8 status;
+ u8 attr;
+ __le16 vf_index;
+ __le16 comp_index;
+ u8 rsvd[9];
+ u8 color;
+};
+
+/**
+ * struct pds_core_vf_getattr_cmd - Get VF attributes from the NIC
+ * @opcode: Opcode
+ * @attr: Attribute type (enum pds_core_vf_attr)
+ * @vf_index: VF index
+ */
+struct pds_core_vf_getattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 vf_index;
+};
+
+struct pds_core_vf_getattr_comp {
+ u8 status;
+ u8 attr;
+ __le16 vf_index;
+ union {
+ u8 macaddr[6];
+ __le16 vlanid;
+ __le32 maxrate;
+ u8 spoofchk;
+ u8 trust;
+ u8 linkstate;
+ __le64 stats_pa;
+ u8 pad[11];
+ } __packed;
+ u8 color;
+};
+
+enum pds_core_vf_ctrl_opcode {
+ PDS_CORE_VF_CTRL_START_ALL = 0,
+ PDS_CORE_VF_CTRL_START = 1,
+};
+
+/**
+ * struct pds_core_vf_ctrl_cmd - VF control command
+ * @opcode: Opcode for the command
+ * @ctrl_opcode: VF control operation type
+ * @vf_index: VF Index. It is unused if op START_ALL is used.
+ */
+
+struct pds_core_vf_ctrl_cmd {
+ u8 opcode;
+ u8 ctrl_opcode;
+ __le16 vf_index;
+};
+
+/**
+ * struct pds_core_vf_ctrl_comp - VF_CTRL command completion.
+ * @status: Status of the command (enum pds_core_status_code)
+ */
+struct pds_core_vf_ctrl_comp {
+ u8 status;
+};
+
+/*
+ * union pds_core_dev_cmd - Overlay of core device command structures
+ */
+union pds_core_dev_cmd {
+ u8 opcode;
+ u32 words[16];
+
+ struct pds_core_dev_identify_cmd identify;
+ struct pds_core_dev_init_cmd init;
+ struct pds_core_dev_reset_cmd reset;
+ struct pds_core_fw_download_cmd fw_download;
+ struct pds_core_fw_control_cmd fw_control;
+
+ struct pds_core_vf_setattr_cmd vf_setattr;
+ struct pds_core_vf_getattr_cmd vf_getattr;
+ struct pds_core_vf_ctrl_cmd vf_ctrl;
+};
+
+/*
+ * union pds_core_dev_comp - Overlay of core device completion structures
+ */
+union pds_core_dev_comp {
+ u8 status;
+ u8 bytes[16];
+
+ struct pds_core_dev_identify_comp identify;
+ struct pds_core_dev_reset_comp reset;
+ struct pds_core_dev_init_comp init;
+ struct pds_core_fw_download_comp fw_download;
+ struct pds_core_fw_control_comp fw_control;
+
+ struct pds_core_vf_setattr_comp vf_setattr;
+ struct pds_core_vf_getattr_comp vf_getattr;
+ struct pds_core_vf_ctrl_comp vf_ctrl;
+};
+
+/**
+ * struct pds_core_dev_hwstamp_regs - Hardware current timestamp registers
+ * @tick_low: Low 32 bits of hardware timestamp
+ * @tick_high: High 32 bits of hardware timestamp
+ */
+struct pds_core_dev_hwstamp_regs {
+ u32 tick_low;
+ u32 tick_high;
+};
+
+/**
+ * struct pds_core_dev_info_regs - Device info register format (read-only)
+ * @signature: Signature value of 0x44455649 ('DEVI')
+ * @version: Current version of info
+ * @asic_type: Asic type
+ * @asic_rev: Asic revision
+ * @fw_status: Firmware status
+ * bit 0 - 1 = fw running
+ * bit 4-7 - 4 bit generation number, changes on fw restart
+ * @fw_heartbeat: Firmware heartbeat counter
+ * @serial_num: Serial number
+ * @fw_version: Firmware version
+ * @oprom_regs: oprom_regs to store oprom debug enable/disable and bmp
+ * @rsvd_pad1024: Struct padding
+ * @hwstamp: Hardware current timestamp registers
+ * @rsvd_pad2048: Struct padding
+ */
+struct pds_core_dev_info_regs {
+#define PDS_CORE_DEVINFO_FWVERS_BUFLEN 32
+#define PDS_CORE_DEVINFO_SERIAL_BUFLEN 32
+ u32 signature;
+ u8 version;
+ u8 asic_type;
+ u8 asic_rev;
+#define PDS_CORE_FW_STS_F_STOPPED 0x00
+#define PDS_CORE_FW_STS_F_RUNNING 0x01
+#define PDS_CORE_FW_STS_F_GENERATION 0xF0
+ u8 fw_status;
+ __le32 fw_heartbeat;
+ char fw_version[PDS_CORE_DEVINFO_FWVERS_BUFLEN];
+ char serial_num[PDS_CORE_DEVINFO_SERIAL_BUFLEN];
+ u8 oprom_regs[32]; /* reserved */
+ u8 rsvd_pad1024[916];
+ struct pds_core_dev_hwstamp_regs hwstamp; /* on 1k boundary */
+ u8 rsvd_pad2048[1016];
+} __packed;
+
+/**
+ * struct pds_core_dev_cmd_regs - Device command register format (read-write)
+ * @doorbell: Device Cmd Doorbell, write-only
+ * Write a 1 to signal device to process cmd
+ * @done: Command completed indicator, poll for completion
+ * bit 0 == 1 when command is complete
+ * @cmd: Opcode-specific command bytes
+ * @comp: Opcode-specific response bytes
+ * @rsvd: Struct padding
+ * @data: Opcode-specific side-data
+ */
+struct pds_core_dev_cmd_regs {
+ u32 doorbell;
+ u32 done;
+ union pds_core_dev_cmd cmd;
+ union pds_core_dev_comp comp;
+ u8 rsvd[48];
+ u32 data[478];
+} __packed;
+
+/**
+ * struct pds_core_dev_regs - Device register format for bar 0 page 0
+ * @info: Device info registers
+ * @devcmd: Device command registers
+ */
+struct pds_core_dev_regs {
+ struct pds_core_dev_info_regs info;
+ struct pds_core_dev_cmd_regs devcmd;
+} __packed;
+
+#ifndef __CHECKER__
+static_assert(sizeof(struct pds_core_drv_identity) <= 1912);
+static_assert(sizeof(struct pds_core_dev_identity) <= 1912);
+static_assert(sizeof(union pds_core_dev_cmd) == 64);
+static_assert(sizeof(union pds_core_dev_comp) == 16);
+static_assert(sizeof(struct pds_core_dev_info_regs) == 2048);
+static_assert(sizeof(struct pds_core_dev_cmd_regs) == 2048);
+static_assert(sizeof(struct pds_core_dev_regs) == 4096);
+#endif /* __CHECKER__ */
+
+#endif /* _PDS_CORE_IF_H_ */
diff --git a/include/linux/pds/pds_intr.h b/include/linux/pds/pds_intr.h
new file mode 100644
index 0000000..56277c3
--- /dev/null
+++ b/include/linux/pds/pds_intr.h
@@ -0,0 +1,163 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) OR BSD-2-Clause */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _PDS_INTR_H_
+#define _PDS_INTR_H_
+
+/*
+ * Interrupt control register
+ * @coal_init: Coalescing timer initial value, in
+ * device units. Use @identity->intr_coal_mult
+ * and @identity->intr_coal_div to convert from
+ * usecs to device units:
+ *
+ * coal_init = coal_usecs * coal_mutl / coal_div
+ *
+ * When an interrupt is sent the interrupt
+ * coalescing timer current value
+ * (@coalescing_curr) is initialized with this
+ * value and begins counting down. No more
+ * interrupts are sent until the coalescing
+ * timer reaches 0. When @coalescing_init=0
+ * interrupt coalescing is effectively disabled
+ * and every interrupt assert results in an
+ * interrupt. Reset value: 0
+ * @mask: Interrupt mask. When @mask=1 the interrupt
+ * resource will not send an interrupt. When
+ * @mask=0 the interrupt resource will send an
+ * interrupt if an interrupt event is pending
+ * or on the next interrupt assertion event.
+ * Reset value: 1
+ * @credits: Interrupt credits. This register indicates
+ * how many interrupt events the hardware has
+ * sent. When written by software this
+ * register atomically decrements @int_credits
+ * by the value written. When @int_credits
+ * becomes 0 then the "pending interrupt" bit
+ * in the Interrupt Status register is cleared
+ * by the hardware and any pending but unsent
+ * interrupts are cleared.
+ * !!!IMPORTANT!!! This is a signed register.
+ * @flags: Interrupt control flags
+ * @unmask -- When this bit is written with a 1
+ * the interrupt resource will set mask=0.
+ * @coal_timer_reset -- When this
+ * bit is written with a 1 the
+ * @coalescing_curr will be reloaded with
+ * @coalescing_init to reset the coalescing
+ * timer.
+ * @mask_on_assert: Automatically mask on assertion. When
+ * @mask_on_assert=1 the interrupt resource
+ * will set @mask=1 whenever an interrupt is
+ * sent. When using interrupts in Legacy
+ * Interrupt mode the driver must select
+ * @mask_on_assert=0 for proper interrupt
+ * operation.
+ * @coalescing_curr: Coalescing timer current value, in
+ * microseconds. When this value reaches 0
+ * the interrupt resource is again eligible to
+ * send an interrupt. If an interrupt event
+ * is already pending when @coalescing_curr
+ * reaches 0 the pending interrupt will be
+ * sent, otherwise an interrupt will be sent
+ * on the next interrupt assertion event.
+ */
+struct pds_core_intr {
+ u32 coal_init;
+ u32 mask;
+ u16 credits;
+ u16 flags;
+#define PDS_CORE_INTR_F_UNMASK 0x0001
+#define PDS_CORE_INTR_F_TIMER_RESET 0x0002
+ u32 mask_on_assert;
+ u32 coalescing_curr;
+ u32 rsvd6[3];
+};
+
+#ifndef __CHECKER__
+static_assert(sizeof(struct pds_core_intr) == 32);
+#endif /* __CHECKER__ */
+
+#define PDS_CORE_INTR_CTRL_REGS_MAX 2048
+#define PDS_CORE_INTR_CTRL_COAL_MAX 0x3F
+#define PDS_CORE_INTR_INDEX_NOT_ASSIGNED -1
+
+struct pds_core_intr_status {
+ u32 status[2];
+};
+
+/**
+ * enum pds_core_intr_mask_vals - valid values for mask and mask_assert.
+ * @PDS_CORE_INTR_MASK_CLEAR: unmask interrupt.
+ * @PDS_CORE_INTR_MASK_SET: mask interrupt.
+ */
+enum pds_core_intr_mask_vals {
+ PDS_CORE_INTR_MASK_CLEAR = 0,
+ PDS_CORE_INTR_MASK_SET = 1,
+};
+
+/**
+ * enum pds_core_intr_credits_bits - Bitwise composition of credits values.
+ * @PDS_CORE_INTR_CRED_COUNT: bit mask of credit count, no shift needed.
+ * @PDS_CORE_INTR_CRED_COUNT_SIGNED: bit mask of credit count, including sign bit.
+ * @PDS_CORE_INTR_CRED_UNMASK: unmask the interrupt.
+ * @PDS_CORE_INTR_CRED_RESET_COALESCE: reset the coalesce timer.
+ * @PDS_CORE_INTR_CRED_REARM: unmask the and reset the timer.
+ */
+enum pds_core_intr_credits_bits {
+ PDS_CORE_INTR_CRED_COUNT = 0x7fffu,
+ PDS_CORE_INTR_CRED_COUNT_SIGNED = 0xffffu,
+ PDS_CORE_INTR_CRED_UNMASK = 0x10000u,
+ PDS_CORE_INTR_CRED_RESET_COALESCE = 0x20000u,
+ PDS_CORE_INTR_CRED_REARM = (PDS_CORE_INTR_CRED_UNMASK |
+ PDS_CORE_INTR_CRED_RESET_COALESCE),
+};
+
+static inline void
+pds_core_intr_coal_init(struct pds_core_intr __iomem *intr_ctrl, u32 coal)
+{
+ iowrite32(coal, &intr_ctrl->coal_init);
+}
+
+static inline void
+pds_core_intr_mask(struct pds_core_intr __iomem *intr_ctrl, u32 mask)
+{
+ iowrite32(mask, &intr_ctrl->mask);
+}
+
+static inline void
+pds_core_intr_credits(struct pds_core_intr __iomem *intr_ctrl,
+ u32 cred, u32 flags)
+{
+ if (WARN_ON_ONCE(cred > PDS_CORE_INTR_CRED_COUNT)) {
+ cred = ioread32(&intr_ctrl->credits);
+ cred &= PDS_CORE_INTR_CRED_COUNT_SIGNED;
+ }
+
+ iowrite32(cred | flags, &intr_ctrl->credits);
+}
+
+static inline void
+pds_core_intr_clean_flags(struct pds_core_intr __iomem *intr_ctrl, u32 flags)
+{
+ u32 cred;
+
+ cred = ioread32(&intr_ctrl->credits);
+ cred &= PDS_CORE_INTR_CRED_COUNT_SIGNED;
+ cred |= flags;
+ iowrite32(cred, &intr_ctrl->credits);
+}
+
+static inline void
+pds_core_intr_clean(struct pds_core_intr __iomem *intr_ctrl)
+{
+ pds_core_intr_clean_flags(intr_ctrl, PDS_CORE_INTR_CRED_RESET_COALESCE);
+}
+
+static inline void
+pds_core_intr_mask_assert(struct pds_core_intr __iomem *intr_ctrl, u32 mask)
+{
+ iowrite32(mask, &intr_ctrl->mask_on_assert);
+}
+
+#endif /* _PDS_INTR_H_ */
diff --git a/include/linux/phy.h b/include/linux/phy.h
index 2f83cfc..c5a0dc8 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -14,6 +14,7 @@
#include <linux/compiler.h>
#include <linux/spinlock.h>
#include <linux/ethtool.h>
+#include <linux/leds.h>
#include <linux/linkmode.h>
#include <linux/netlink.h>
#include <linux/mdio.h>
@@ -600,6 +601,7 @@ struct macsec_ops;
* @phy_num_led_triggers: Number of triggers in @phy_led_triggers
* @led_link_trigger: LED trigger for link up/down
* @last_triggered: last LED trigger for link speed
+ * @leds: list of PHY LED structures
* @master_slave_set: User requested master/slave configuration
* @master_slave_get: Current master/slave advertisement
* @master_slave_state: Current master/slave configuration
@@ -699,6 +701,7 @@ struct phy_device {
struct phy_led_trigger *led_link_trigger;
#endif
+ struct list_head leds;
/*
* Interrupt number for this PHY
@@ -835,6 +838,23 @@ struct phy_plca_status {
};
/**
+ * struct phy_led: An LED driven by the PHY
+ *
+ * @list: List of LEDs
+ * @phydev: PHY this LED is attached to
+ * @led_cdev: Standard LED class structure
+ * @index: Number of the LED
+ */
+struct phy_led {
+ struct list_head list;
+ struct phy_device *phydev;
+ struct led_classdev led_cdev;
+ u8 index;
+};
+
+#define to_phy_led(d) container_of(d, struct phy_led, led_cdev)
+
+/**
* struct phy_driver - Driver structure for a particular PHY type
*
* @mdiodrv: Data common to all MDIO devices
@@ -1056,6 +1076,27 @@ struct phy_driver {
/** @get_plca_status: Return the current PLCA status info */
int (*get_plca_status)(struct phy_device *dev,
struct phy_plca_status *plca_st);
+
+ /**
+ * @led_brightness_set: Set a PHY LED brightness. Index
+ * indicates which of the PHYs led should be set. Value
+ * follows the standard LED class meaning, e.g. LED_OFF,
+ * LED_HALF, LED_FULL.
+ */
+ int (*led_brightness_set)(struct phy_device *dev,
+ u8 index, enum led_brightness value);
+
+ /**
+ * @led_blink_set: Set a PHY LED brightness. Index indicates
+ * which of the PHYs led should be configured to blink. Delays
+ * are in milliseconds and if both are zero then a sensible
+ * default should be chosen. The call should adjust the
+ * timings in that case and if it can't match the values
+ * specified exactly.
+ */
+ int (*led_blink_set)(struct phy_device *dev, u8 index,
+ unsigned long *delay_on,
+ unsigned long *delay_off);
};
#define to_phy_driver(d) container_of(to_mdio_common_driver(d), \
struct phy_driver, mdiodrv)
diff --git a/include/linux/sctp.h b/include/linux/sctp.h
index 358dc08..836a7e2 100644
--- a/include/linux/sctp.h
+++ b/include/linux/sctp.h
@@ -222,7 +222,7 @@ struct sctp_datahdr {
__be16 stream;
__be16 ssn;
__u32 ppid;
- __u8 payload[];
+ /* __u8 payload[]; */
};
struct sctp_data_chunk {
@@ -270,7 +270,7 @@ struct sctp_inithdr {
__be16 num_outbound_streams;
__be16 num_inbound_streams;
__be32 initial_tsn;
- __u8 params[];
+ /* __u8 params[]; */
};
struct sctp_init_chunk {
@@ -385,7 +385,7 @@ struct sctp_sackhdr {
__be32 a_rwnd;
__be16 num_gap_ack_blocks;
__be16 num_dup_tsns;
- union sctp_sack_variable variable[];
+ /* union sctp_sack_variable variable[]; */
};
struct sctp_sack_chunk {
@@ -443,7 +443,7 @@ struct sctp_shutdown_chunk {
struct sctp_errhdr {
__be16 cause;
__be16 length;
- __u8 variable[];
+ /* __u8 variable[]; */
};
struct sctp_operr_chunk {
@@ -603,7 +603,7 @@ struct sctp_fwdtsn_skip {
struct sctp_fwdtsn_hdr {
__be32 new_cum_tsn;
- struct sctp_fwdtsn_skip skip[];
+ /* struct sctp_fwdtsn_skip skip[]; */
};
struct sctp_fwdtsn_chunk {
@@ -620,7 +620,7 @@ struct sctp_ifwdtsn_skip {
struct sctp_ifwdtsn_hdr {
__be32 new_cum_tsn;
- struct sctp_ifwdtsn_skip skip[];
+ /* struct sctp_ifwdtsn_skip skip[]; */
};
struct sctp_ifwdtsn_chunk {
@@ -667,7 +667,7 @@ struct sctp_addip_param {
struct sctp_addiphdr {
__be32 serial;
- __u8 params[];
+ /* __u8 params[]; */
};
struct sctp_addip_chunk {
@@ -727,7 +727,7 @@ struct sctp_addip_chunk {
struct sctp_authhdr {
__be16 shkey_id;
__be16 hmac_id;
- __u8 hmac[];
+ /* __u8 hmac[]; */
};
struct sctp_auth_chunk {
@@ -742,7 +742,7 @@ struct sctp_infox {
struct sctp_reconf_chunk {
struct sctp_chunkhdr chunk_hdr;
- __u8 params[];
+ /* __u8 params[]; */
};
struct sctp_strreset_outreq {
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 9ff2e3d..738776a 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -37,7 +37,7 @@
#include <linux/netfilter/nf_conntrack_common.h>
#endif
#include <net/net_debug.h>
-#include <net/dropreason.h>
+#include <net/dropreason-core.h>
/**
* DOC: skb checksums
@@ -294,6 +294,7 @@ struct nf_bridge_info {
u8 pkt_otherhost:1;
u8 in_prerouting:1;
u8 bridged_dnat:1;
+ u8 sabotage_in_done:1;
__u16 frag_max_size;
struct net_device *physindev;
@@ -934,7 +935,7 @@ struct sk_buff {
/* public: */
__u8 pkt_type:3; /* see PKT_TYPE_MAX */
__u8 ignore_df:1;
- __u8 nf_trace:1;
+ __u8 dst_pending_confirm:1;
__u8 ip_summed:2;
__u8 ooo_okay:1;
@@ -949,12 +950,14 @@ struct sk_buff {
__u8 remcsum_offload:1;
__u8 csum_complete_sw:1;
__u8 csum_level:2;
- __u8 dst_pending_confirm:1;
+ __u8 inner_protocol_type:1;
__u8 l4_hash:1;
__u8 sw_hash:1;
+#ifdef CONFIG_WIRELESS
__u8 wifi_acked_valid:1;
__u8 wifi_acked:1;
+#endif
__u8 no_fcs:1;
/* Indicates the inner headers are valid in the skbuff. */
__u8 encapsulation:1;
@@ -964,8 +967,12 @@ struct sk_buff {
__u8 ndisc_nodetype:2;
#endif
+#if IS_ENABLED(CONFIG_IP_VS)
__u8 ipvs_property:1;
- __u8 inner_protocol_type:1;
+#endif
+#if IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) || IS_ENABLED(CONFIG_NF_TABLES)
+ __u8 nf_trace:1;
+#endif
#ifdef CONFIG_NET_SWITCHDEV
__u8 offload_fwd_mark:1;
__u8 offload_l3_fwd_mark:1;
@@ -981,12 +988,16 @@ struct sk_buff {
__u8 decrypted:1;
#endif
__u8 slow_gro:1;
+#if IS_ENABLED(CONFIG_IP_SCTP)
__u8 csum_not_inet:1;
+#endif
#ifdef CONFIG_NET_SCHED
__u16 tc_index; /* traffic control index */
#endif
+ u16 alloc_cpu;
+
union {
__wsum csum;
struct {
@@ -1010,7 +1021,6 @@ struct sk_buff {
unsigned int sender_cpu;
};
#endif
- u16 alloc_cpu;
#ifdef CONFIG_NETWORK_SECMARK
__u32 secmark;
#endif
@@ -1187,6 +1197,15 @@ static inline unsigned int skb_napi_id(const struct sk_buff *skb)
#endif
}
+static inline bool skb_wifi_acked_valid(const struct sk_buff *skb)
+{
+#ifdef CONFIG_WIRELESS
+ return skb->wifi_acked_valid;
+#else
+ return 0;
+#endif
+}
+
/**
* skb_unref - decrement the skb's reference count
* @skb: buffer
@@ -3234,7 +3253,7 @@ static inline struct sk_buff *napi_alloc_skb(struct napi_struct *napi,
void napi_consume_skb(struct sk_buff *skb, int budget);
void napi_skb_free_stolen_head(struct sk_buff *skb);
-void __kfree_skb_defer(struct sk_buff *skb);
+void __napi_kfree_skb(struct sk_buff *skb, enum skb_drop_reason reason);
/**
* __dev_alloc_pages - allocate page for network Rx
@@ -3386,6 +3405,18 @@ static inline void skb_frag_ref(struct sk_buff *skb, int f)
__skb_frag_ref(&skb_shinfo(skb)->frags[f]);
}
+static inline void
+napi_frag_unref(skb_frag_t *frag, bool recycle, bool napi_safe)
+{
+ struct page *page = skb_frag_page(frag);
+
+#ifdef CONFIG_PAGE_POOL
+ if (recycle && page_pool_return_skb_page(page, napi_safe))
+ return;
+#endif
+ put_page(page);
+}
+
/**
* __skb_frag_unref - release a reference on a paged fragment.
* @frag: the paged fragment
@@ -3396,13 +3427,7 @@ static inline void skb_frag_ref(struct sk_buff *skb, int f)
*/
static inline void __skb_frag_unref(skb_frag_t *frag, bool recycle)
{
- struct page *page = skb_frag_page(frag);
-
-#ifdef CONFIG_PAGE_POOL
- if (recycle && page_pool_return_skb_page(page))
- return;
-#endif
- put_page(page);
+ napi_frag_unref(frag, recycle, false);
}
/**
@@ -4704,7 +4729,7 @@ static inline void nf_reset_ct(struct sk_buff *skb)
static inline void nf_reset_trace(struct sk_buff *skb)
{
-#if IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) || defined(CONFIG_NF_TABLES)
+#if IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) || IS_ENABLED(CONFIG_NF_TABLES)
skb->nf_trace = 0;
#endif
}
@@ -4724,7 +4749,7 @@ static inline void __nf_copy(struct sk_buff *dst, const struct sk_buff *src,
dst->_nfct = src->_nfct;
nf_conntrack_get(skb_nfct(src));
#endif
-#if IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) || defined(CONFIG_NF_TABLES)
+#if IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) || IS_ENABLED(CONFIG_NF_TABLES)
if (copy)
dst->nf_trace = src->nf_trace;
#endif
@@ -5052,7 +5077,19 @@ static inline void skb_set_redirected_noclear(struct sk_buff *skb,
static inline bool skb_csum_is_sctp(struct sk_buff *skb)
{
+#if IS_ENABLED(CONFIG_IP_SCTP)
return skb->csum_not_inet;
+#else
+ return 0;
+#endif
+}
+
+static inline void skb_reset_csum_not_inet(struct sk_buff *skb)
+{
+ skb->ip_summed = CHECKSUM_NONE;
+#if IS_ENABLED(CONFIG_IP_SCTP)
+ skb->csum_not_inet = 0;
+#endif
}
static inline void skb_set_kcov_handle(struct sk_buff *skb,
diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
new file mode 100644
index 0000000..a2b953b
--- /dev/null
+++ b/include/net/dropreason-core.h
@@ -0,0 +1,370 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _LINUX_DROPREASON_CORE_H
+#define _LINUX_DROPREASON_CORE_H
+
+#define DEFINE_DROP_REASON(FN, FNe) \
+ FN(NOT_SPECIFIED) \
+ FN(NO_SOCKET) \
+ FN(PKT_TOO_SMALL) \
+ FN(TCP_CSUM) \
+ FN(SOCKET_FILTER) \
+ FN(UDP_CSUM) \
+ FN(NETFILTER_DROP) \
+ FN(OTHERHOST) \
+ FN(IP_CSUM) \
+ FN(IP_INHDR) \
+ FN(IP_RPFILTER) \
+ FN(UNICAST_IN_L2_MULTICAST) \
+ FN(XFRM_POLICY) \
+ FN(IP_NOPROTO) \
+ FN(SOCKET_RCVBUFF) \
+ FN(PROTO_MEM) \
+ FN(TCP_MD5NOTFOUND) \
+ FN(TCP_MD5UNEXPECTED) \
+ FN(TCP_MD5FAILURE) \
+ FN(SOCKET_BACKLOG) \
+ FN(TCP_FLAGS) \
+ FN(TCP_ZEROWINDOW) \
+ FN(TCP_OLD_DATA) \
+ FN(TCP_OVERWINDOW) \
+ FN(TCP_OFOMERGE) \
+ FN(TCP_RFC7323_PAWS) \
+ FN(TCP_INVALID_SEQUENCE) \
+ FN(TCP_RESET) \
+ FN(TCP_INVALID_SYN) \
+ FN(TCP_CLOSE) \
+ FN(TCP_FASTOPEN) \
+ FN(TCP_OLD_ACK) \
+ FN(TCP_TOO_OLD_ACK) \
+ FN(TCP_ACK_UNSENT_DATA) \
+ FN(TCP_OFO_QUEUE_PRUNE) \
+ FN(TCP_OFO_DROP) \
+ FN(IP_OUTNOROUTES) \
+ FN(BPF_CGROUP_EGRESS) \
+ FN(IPV6DISABLED) \
+ FN(NEIGH_CREATEFAIL) \
+ FN(NEIGH_FAILED) \
+ FN(NEIGH_QUEUEFULL) \
+ FN(NEIGH_DEAD) \
+ FN(TC_EGRESS) \
+ FN(QDISC_DROP) \
+ FN(CPU_BACKLOG) \
+ FN(XDP) \
+ FN(TC_INGRESS) \
+ FN(UNHANDLED_PROTO) \
+ FN(SKB_CSUM) \
+ FN(SKB_GSO_SEG) \
+ FN(SKB_UCOPY_FAULT) \
+ FN(DEV_HDR) \
+ FN(DEV_READY) \
+ FN(FULL_RING) \
+ FN(NOMEM) \
+ FN(HDR_TRUNC) \
+ FN(TAP_FILTER) \
+ FN(TAP_TXFILTER) \
+ FN(ICMP_CSUM) \
+ FN(INVALID_PROTO) \
+ FN(IP_INADDRERRORS) \
+ FN(IP_INNOROUTES) \
+ FN(PKT_TOO_BIG) \
+ FN(DUP_FRAG) \
+ FN(FRAG_REASM_TIMEOUT) \
+ FN(FRAG_TOO_FAR) \
+ FN(TCP_MINTTL) \
+ FN(IPV6_BAD_EXTHDR) \
+ FN(IPV6_NDISC_FRAG) \
+ FN(IPV6_NDISC_HOP_LIMIT) \
+ FN(IPV6_NDISC_BAD_CODE) \
+ FN(IPV6_NDISC_BAD_OPTIONS) \
+ FN(IPV6_NDISC_NS_OTHERHOST) \
+ FNe(MAX)
+
+/**
+ * enum skb_drop_reason - the reasons of skb drops
+ *
+ * The reason of skb drop, which is used in kfree_skb_reason().
+ */
+enum skb_drop_reason {
+ /**
+ * @SKB_NOT_DROPPED_YET: skb is not dropped yet (used for no-drop case)
+ */
+ SKB_NOT_DROPPED_YET = 0,
+ /** @SKB_CONSUMED: packet has been consumed */
+ SKB_CONSUMED,
+ /** @SKB_DROP_REASON_NOT_SPECIFIED: drop reason is not specified */
+ SKB_DROP_REASON_NOT_SPECIFIED,
+ /** @SKB_DROP_REASON_NO_SOCKET: socket not found */
+ SKB_DROP_REASON_NO_SOCKET,
+ /** @SKB_DROP_REASON_PKT_TOO_SMALL: packet size is too small */
+ SKB_DROP_REASON_PKT_TOO_SMALL,
+ /** @SKB_DROP_REASON_TCP_CSUM: TCP checksum error */
+ SKB_DROP_REASON_TCP_CSUM,
+ /** @SKB_DROP_REASON_SOCKET_FILTER: dropped by socket filter */
+ SKB_DROP_REASON_SOCKET_FILTER,
+ /** @SKB_DROP_REASON_UDP_CSUM: UDP checksum error */
+ SKB_DROP_REASON_UDP_CSUM,
+ /** @SKB_DROP_REASON_NETFILTER_DROP: dropped by netfilter */
+ SKB_DROP_REASON_NETFILTER_DROP,
+ /**
+ * @SKB_DROP_REASON_OTHERHOST: packet don't belong to current host
+ * (interface is in promisc mode)
+ */
+ SKB_DROP_REASON_OTHERHOST,
+ /** @SKB_DROP_REASON_IP_CSUM: IP checksum error */
+ SKB_DROP_REASON_IP_CSUM,
+ /**
+ * @SKB_DROP_REASON_IP_INHDR: there is something wrong with IP header (see
+ * IPSTATS_MIB_INHDRERRORS)
+ */
+ SKB_DROP_REASON_IP_INHDR,
+ /**
+ * @SKB_DROP_REASON_IP_RPFILTER: IP rpfilter validate failed. see the
+ * document for rp_filter in ip-sysctl.rst for more information
+ */
+ SKB_DROP_REASON_IP_RPFILTER,
+ /**
+ * @SKB_DROP_REASON_UNICAST_IN_L2_MULTICAST: destination address of L2 is
+ * multicast, but L3 is unicast.
+ */
+ SKB_DROP_REASON_UNICAST_IN_L2_MULTICAST,
+ /** @SKB_DROP_REASON_XFRM_POLICY: xfrm policy check failed */
+ SKB_DROP_REASON_XFRM_POLICY,
+ /** @SKB_DROP_REASON_IP_NOPROTO: no support for IP protocol */
+ SKB_DROP_REASON_IP_NOPROTO,
+ /** @SKB_DROP_REASON_SOCKET_RCVBUFF: socket receive buff is full */
+ SKB_DROP_REASON_SOCKET_RCVBUFF,
+ /**
+ * @SKB_DROP_REASON_PROTO_MEM: proto memory limition, such as udp packet
+ * drop out of udp_memory_allocated.
+ */
+ SKB_DROP_REASON_PROTO_MEM,
+ /**
+ * @SKB_DROP_REASON_TCP_MD5NOTFOUND: no MD5 hash and one expected,
+ * corresponding to LINUX_MIB_TCPMD5NOTFOUND
+ */
+ SKB_DROP_REASON_TCP_MD5NOTFOUND,
+ /**
+ * @SKB_DROP_REASON_TCP_MD5UNEXPECTED: MD5 hash and we're not expecting
+ * one, corresponding to LINUX_MIB_TCPMD5UNEXPECTED
+ */
+ SKB_DROP_REASON_TCP_MD5UNEXPECTED,
+ /**
+ * @SKB_DROP_REASON_TCP_MD5FAILURE: MD5 hash and its wrong, corresponding
+ * to LINUX_MIB_TCPMD5FAILURE
+ */
+ SKB_DROP_REASON_TCP_MD5FAILURE,
+ /**
+ * @SKB_DROP_REASON_SOCKET_BACKLOG: failed to add skb to socket backlog (
+ * see LINUX_MIB_TCPBACKLOGDROP)
+ */
+ SKB_DROP_REASON_SOCKET_BACKLOG,
+ /** @SKB_DROP_REASON_TCP_FLAGS: TCP flags invalid */
+ SKB_DROP_REASON_TCP_FLAGS,
+ /**
+ * @SKB_DROP_REASON_TCP_ZEROWINDOW: TCP receive window size is zero,
+ * see LINUX_MIB_TCPZEROWINDOWDROP
+ */
+ SKB_DROP_REASON_TCP_ZEROWINDOW,
+ /**
+ * @SKB_DROP_REASON_TCP_OLD_DATA: the TCP data reveived is already
+ * received before (spurious retrans may happened), see
+ * LINUX_MIB_DELAYEDACKLOST
+ */
+ SKB_DROP_REASON_TCP_OLD_DATA,
+ /**
+ * @SKB_DROP_REASON_TCP_OVERWINDOW: the TCP data is out of window,
+ * the seq of the first byte exceed the right edges of receive
+ * window
+ */
+ SKB_DROP_REASON_TCP_OVERWINDOW,
+ /**
+ * @SKB_DROP_REASON_TCP_OFOMERGE: the data of skb is already in the ofo
+ * queue, corresponding to LINUX_MIB_TCPOFOMERGE
+ */
+ SKB_DROP_REASON_TCP_OFOMERGE,
+ /**
+ * @SKB_DROP_REASON_TCP_RFC7323_PAWS: PAWS check, corresponding to
+ * LINUX_MIB_PAWSESTABREJECTED
+ */
+ SKB_DROP_REASON_TCP_RFC7323_PAWS,
+ /** @SKB_DROP_REASON_TCP_INVALID_SEQUENCE: Not acceptable SEQ field */
+ SKB_DROP_REASON_TCP_INVALID_SEQUENCE,
+ /** @SKB_DROP_REASON_TCP_RESET: Invalid RST packet */
+ SKB_DROP_REASON_TCP_RESET,
+ /**
+ * @SKB_DROP_REASON_TCP_INVALID_SYN: Incoming packet has unexpected
+ * SYN flag
+ */
+ SKB_DROP_REASON_TCP_INVALID_SYN,
+ /** @SKB_DROP_REASON_TCP_CLOSE: TCP socket in CLOSE state */
+ SKB_DROP_REASON_TCP_CLOSE,
+ /** @SKB_DROP_REASON_TCP_FASTOPEN: dropped by FASTOPEN request socket */
+ SKB_DROP_REASON_TCP_FASTOPEN,
+ /** @SKB_DROP_REASON_TCP_OLD_ACK: TCP ACK is old, but in window */
+ SKB_DROP_REASON_TCP_OLD_ACK,
+ /** @SKB_DROP_REASON_TCP_TOO_OLD_ACK: TCP ACK is too old */
+ SKB_DROP_REASON_TCP_TOO_OLD_ACK,
+ /**
+ * @SKB_DROP_REASON_TCP_ACK_UNSENT_DATA: TCP ACK for data we haven't
+ * sent yet
+ */
+ SKB_DROP_REASON_TCP_ACK_UNSENT_DATA,
+ /** @SKB_DROP_REASON_TCP_OFO_QUEUE_PRUNE: pruned from TCP OFO queue */
+ SKB_DROP_REASON_TCP_OFO_QUEUE_PRUNE,
+ /** @SKB_DROP_REASON_TCP_OFO_DROP: data already in receive queue */
+ SKB_DROP_REASON_TCP_OFO_DROP,
+ /** @SKB_DROP_REASON_IP_OUTNOROUTES: route lookup failed */
+ SKB_DROP_REASON_IP_OUTNOROUTES,
+ /**
+ * @SKB_DROP_REASON_BPF_CGROUP_EGRESS: dropped by BPF_PROG_TYPE_CGROUP_SKB
+ * eBPF program
+ */
+ SKB_DROP_REASON_BPF_CGROUP_EGRESS,
+ /** @SKB_DROP_REASON_IPV6DISABLED: IPv6 is disabled on the device */
+ SKB_DROP_REASON_IPV6DISABLED,
+ /** @SKB_DROP_REASON_NEIGH_CREATEFAIL: failed to create neigh entry */
+ SKB_DROP_REASON_NEIGH_CREATEFAIL,
+ /** @SKB_DROP_REASON_NEIGH_FAILED: neigh entry in failed state */
+ SKB_DROP_REASON_NEIGH_FAILED,
+ /** @SKB_DROP_REASON_NEIGH_QUEUEFULL: arp_queue for neigh entry is full */
+ SKB_DROP_REASON_NEIGH_QUEUEFULL,
+ /** @SKB_DROP_REASON_NEIGH_DEAD: neigh entry is dead */
+ SKB_DROP_REASON_NEIGH_DEAD,
+ /** @SKB_DROP_REASON_TC_EGRESS: dropped in TC egress HOOK */
+ SKB_DROP_REASON_TC_EGRESS,
+ /**
+ * @SKB_DROP_REASON_QDISC_DROP: dropped by qdisc when packet outputting (
+ * failed to enqueue to current qdisc)
+ */
+ SKB_DROP_REASON_QDISC_DROP,
+ /**
+ * @SKB_DROP_REASON_CPU_BACKLOG: failed to enqueue the skb to the per CPU
+ * backlog queue. This can be caused by backlog queue full (see
+ * netdev_max_backlog in net.rst) or RPS flow limit
+ */
+ SKB_DROP_REASON_CPU_BACKLOG,
+ /** @SKB_DROP_REASON_XDP: dropped by XDP in input path */
+ SKB_DROP_REASON_XDP,
+ /** @SKB_DROP_REASON_TC_INGRESS: dropped in TC ingress HOOK */
+ SKB_DROP_REASON_TC_INGRESS,
+ /** @SKB_DROP_REASON_UNHANDLED_PROTO: protocol not implemented or not supported */
+ SKB_DROP_REASON_UNHANDLED_PROTO,
+ /** @SKB_DROP_REASON_SKB_CSUM: sk_buff checksum computation error */
+ SKB_DROP_REASON_SKB_CSUM,
+ /** @SKB_DROP_REASON_SKB_GSO_SEG: gso segmentation error */
+ SKB_DROP_REASON_SKB_GSO_SEG,
+ /**
+ * @SKB_DROP_REASON_SKB_UCOPY_FAULT: failed to copy data from user space,
+ * e.g., via zerocopy_sg_from_iter() or skb_orphan_frags_rx()
+ */
+ SKB_DROP_REASON_SKB_UCOPY_FAULT,
+ /** @SKB_DROP_REASON_DEV_HDR: device driver specific header/metadata is invalid */
+ SKB_DROP_REASON_DEV_HDR,
+ /**
+ * @SKB_DROP_REASON_DEV_READY: the device is not ready to xmit/recv due to
+ * any of its data structure that is not up/ready/initialized,
+ * e.g., the IFF_UP is not set, or driver specific tun->tfiles[txq]
+ * is not initialized
+ */
+ SKB_DROP_REASON_DEV_READY,
+ /** @SKB_DROP_REASON_FULL_RING: ring buffer is full */
+ SKB_DROP_REASON_FULL_RING,
+ /** @SKB_DROP_REASON_NOMEM: error due to OOM */
+ SKB_DROP_REASON_NOMEM,
+ /**
+ * @SKB_DROP_REASON_HDR_TRUNC: failed to trunc/extract the header from
+ * networking data, e.g., failed to pull the protocol header from
+ * frags via pskb_may_pull()
+ */
+ SKB_DROP_REASON_HDR_TRUNC,
+ /**
+ * @SKB_DROP_REASON_TAP_FILTER: dropped by (ebpf) filter directly attached
+ * to tun/tap, e.g., via TUNSETFILTEREBPF
+ */
+ SKB_DROP_REASON_TAP_FILTER,
+ /**
+ * @SKB_DROP_REASON_TAP_TXFILTER: dropped by tx filter implemented at
+ * tun/tap, e.g., check_filter()
+ */
+ SKB_DROP_REASON_TAP_TXFILTER,
+ /** @SKB_DROP_REASON_ICMP_CSUM: ICMP checksum error */
+ SKB_DROP_REASON_ICMP_CSUM,
+ /**
+ * @SKB_DROP_REASON_INVALID_PROTO: the packet doesn't follow RFC 2211,
+ * such as a broadcasts ICMP_TIMESTAMP
+ */
+ SKB_DROP_REASON_INVALID_PROTO,
+ /**
+ * @SKB_DROP_REASON_IP_INADDRERRORS: host unreachable, corresponding to
+ * IPSTATS_MIB_INADDRERRORS
+ */
+ SKB_DROP_REASON_IP_INADDRERRORS,
+ /**
+ * @SKB_DROP_REASON_IP_INNOROUTES: network unreachable, corresponding to
+ * IPSTATS_MIB_INADDRERRORS
+ */
+ SKB_DROP_REASON_IP_INNOROUTES,
+ /**
+ * @SKB_DROP_REASON_PKT_TOO_BIG: packet size is too big (maybe exceed the
+ * MTU)
+ */
+ SKB_DROP_REASON_PKT_TOO_BIG,
+ /** @SKB_DROP_REASON_DUP_FRAG: duplicate fragment */
+ SKB_DROP_REASON_DUP_FRAG,
+ /** @SKB_DROP_REASON_FRAG_REASM_TIMEOUT: fragment reassembly timeout */
+ SKB_DROP_REASON_FRAG_REASM_TIMEOUT,
+ /**
+ * @SKB_DROP_REASON_FRAG_TOO_FAR: ipv4 fragment too far.
+ * (/proc/sys/net/ipv4/ipfrag_max_dist)
+ */
+ SKB_DROP_REASON_FRAG_TOO_FAR,
+ /**
+ * @SKB_DROP_REASON_TCP_MINTTL: ipv4 ttl or ipv6 hoplimit below
+ * the threshold (IP_MINTTL or IPV6_MINHOPCOUNT).
+ */
+ SKB_DROP_REASON_TCP_MINTTL,
+ /** @SKB_DROP_REASON_IPV6_BAD_EXTHDR: Bad IPv6 extension header. */
+ SKB_DROP_REASON_IPV6_BAD_EXTHDR,
+ /** @SKB_DROP_REASON_IPV6_NDISC_FRAG: invalid frag (suppress_frag_ndisc). */
+ SKB_DROP_REASON_IPV6_NDISC_FRAG,
+ /** @SKB_DROP_REASON_IPV6_NDISC_HOP_LIMIT: invalid hop limit. */
+ SKB_DROP_REASON_IPV6_NDISC_HOP_LIMIT,
+ /** @SKB_DROP_REASON_IPV6_NDISC_BAD_CODE: invalid NDISC icmp6 code. */
+ SKB_DROP_REASON_IPV6_NDISC_BAD_CODE,
+ /** @SKB_DROP_REASON_IPV6_NDISC_BAD_OPTIONS: invalid NDISC options. */
+ SKB_DROP_REASON_IPV6_NDISC_BAD_OPTIONS,
+ /**
+ * @SKB_DROP_REASON_IPV6_NDISC_NS_OTHERHOST: NEIGHBOUR SOLICITATION
+ * for another host.
+ */
+ SKB_DROP_REASON_IPV6_NDISC_NS_OTHERHOST,
+ /**
+ * @SKB_DROP_REASON_MAX: the maximum of core drop reasons, which
+ * shouldn't be used as a real 'reason' - only for tracing code gen
+ */
+ SKB_DROP_REASON_MAX,
+
+ /**
+ * @SKB_DROP_REASON_SUBSYS_MASK: subsystem mask in drop reasons,
+ * see &enum skb_drop_reason_subsys
+ */
+ SKB_DROP_REASON_SUBSYS_MASK = 0xffff0000,
+};
+
+#define SKB_DROP_REASON_SUBSYS_SHIFT 16
+
+#define SKB_DR_INIT(name, reason) \
+ enum skb_drop_reason name = SKB_DROP_REASON_##reason
+#define SKB_DR(name) \
+ SKB_DR_INIT(name, NOT_SPECIFIED)
+#define SKB_DR_SET(name, reason) \
+ (name = SKB_DROP_REASON_##reason)
+#define SKB_DR_OR(name, reason) \
+ do { \
+ if (name == SKB_DROP_REASON_NOT_SPECIFIED || \
+ name == SKB_NOT_DROPPED_YET) \
+ SKB_DR_SET(name, reason); \
+ } while (0)
+
+#endif
diff --git a/include/net/dropreason.h b/include/net/dropreason.h
index c0a3ea8..685fb37 100644
--- a/include/net/dropreason.h
+++ b/include/net/dropreason.h
@@ -2,362 +2,42 @@
#ifndef _LINUX_DROPREASON_H
#define _LINUX_DROPREASON_H
-
-#define DEFINE_DROP_REASON(FN, FNe) \
- FN(NOT_SPECIFIED) \
- FN(NO_SOCKET) \
- FN(PKT_TOO_SMALL) \
- FN(TCP_CSUM) \
- FN(SOCKET_FILTER) \
- FN(UDP_CSUM) \
- FN(NETFILTER_DROP) \
- FN(OTHERHOST) \
- FN(IP_CSUM) \
- FN(IP_INHDR) \
- FN(IP_RPFILTER) \
- FN(UNICAST_IN_L2_MULTICAST) \
- FN(XFRM_POLICY) \
- FN(IP_NOPROTO) \
- FN(SOCKET_RCVBUFF) \
- FN(PROTO_MEM) \
- FN(TCP_MD5NOTFOUND) \
- FN(TCP_MD5UNEXPECTED) \
- FN(TCP_MD5FAILURE) \
- FN(SOCKET_BACKLOG) \
- FN(TCP_FLAGS) \
- FN(TCP_ZEROWINDOW) \
- FN(TCP_OLD_DATA) \
- FN(TCP_OVERWINDOW) \
- FN(TCP_OFOMERGE) \
- FN(TCP_RFC7323_PAWS) \
- FN(TCP_INVALID_SEQUENCE) \
- FN(TCP_RESET) \
- FN(TCP_INVALID_SYN) \
- FN(TCP_CLOSE) \
- FN(TCP_FASTOPEN) \
- FN(TCP_OLD_ACK) \
- FN(TCP_TOO_OLD_ACK) \
- FN(TCP_ACK_UNSENT_DATA) \
- FN(TCP_OFO_QUEUE_PRUNE) \
- FN(TCP_OFO_DROP) \
- FN(IP_OUTNOROUTES) \
- FN(BPF_CGROUP_EGRESS) \
- FN(IPV6DISABLED) \
- FN(NEIGH_CREATEFAIL) \
- FN(NEIGH_FAILED) \
- FN(NEIGH_QUEUEFULL) \
- FN(NEIGH_DEAD) \
- FN(TC_EGRESS) \
- FN(QDISC_DROP) \
- FN(CPU_BACKLOG) \
- FN(XDP) \
- FN(TC_INGRESS) \
- FN(UNHANDLED_PROTO) \
- FN(SKB_CSUM) \
- FN(SKB_GSO_SEG) \
- FN(SKB_UCOPY_FAULT) \
- FN(DEV_HDR) \
- FN(DEV_READY) \
- FN(FULL_RING) \
- FN(NOMEM) \
- FN(HDR_TRUNC) \
- FN(TAP_FILTER) \
- FN(TAP_TXFILTER) \
- FN(ICMP_CSUM) \
- FN(INVALID_PROTO) \
- FN(IP_INADDRERRORS) \
- FN(IP_INNOROUTES) \
- FN(PKT_TOO_BIG) \
- FN(DUP_FRAG) \
- FN(FRAG_REASM_TIMEOUT) \
- FN(FRAG_TOO_FAR) \
- FN(TCP_MINTTL) \
- FN(IPV6_BAD_EXTHDR) \
- FN(IPV6_NDISC_FRAG) \
- FN(IPV6_NDISC_HOP_LIMIT) \
- FN(IPV6_NDISC_BAD_CODE) \
- FN(IPV6_NDISC_BAD_OPTIONS) \
- FN(IPV6_NDISC_NS_OTHERHOST) \
- FNe(MAX)
+#include <net/dropreason-core.h>
/**
- * enum skb_drop_reason - the reasons of skb drops
- *
- * The reason of skb drop, which is used in kfree_skb_reason().
+ * enum skb_drop_reason_subsys - subsystem tag for (extended) drop reasons
*/
-enum skb_drop_reason {
+enum skb_drop_reason_subsys {
+ /** @SKB_DROP_REASON_SUBSYS_CORE: core drop reasons defined above */
+ SKB_DROP_REASON_SUBSYS_CORE,
+
/**
- * @SKB_NOT_DROPPED_YET: skb is not dropped yet (used for no-drop case)
+ * @SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE: mac80211 drop reasons
+ * for unusable frames, see net/mac80211/drop.h
*/
- SKB_NOT_DROPPED_YET = 0,
- /** @SKB_CONSUMED: packet has been consumed */
- SKB_CONSUMED,
- /** @SKB_DROP_REASON_NOT_SPECIFIED: drop reason is not specified */
- SKB_DROP_REASON_NOT_SPECIFIED,
- /** @SKB_DROP_REASON_NO_SOCKET: socket not found */
- SKB_DROP_REASON_NO_SOCKET,
- /** @SKB_DROP_REASON_PKT_TOO_SMALL: packet size is too small */
- SKB_DROP_REASON_PKT_TOO_SMALL,
- /** @SKB_DROP_REASON_TCP_CSUM: TCP checksum error */
- SKB_DROP_REASON_TCP_CSUM,
- /** @SKB_DROP_REASON_SOCKET_FILTER: dropped by socket filter */
- SKB_DROP_REASON_SOCKET_FILTER,
- /** @SKB_DROP_REASON_UDP_CSUM: UDP checksum error */
- SKB_DROP_REASON_UDP_CSUM,
- /** @SKB_DROP_REASON_NETFILTER_DROP: dropped by netfilter */
- SKB_DROP_REASON_NETFILTER_DROP,
+ SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE,
+
/**
- * @SKB_DROP_REASON_OTHERHOST: packet don't belong to current host
- * (interface is in promisc mode)
+ * @SKB_DROP_REASON_SUBSYS_MAC80211_MONITOR: mac80211 drop reasons
+ * for frames still going to monitor, see net/mac80211/drop.h
*/
- SKB_DROP_REASON_OTHERHOST,
- /** @SKB_DROP_REASON_IP_CSUM: IP checksum error */
- SKB_DROP_REASON_IP_CSUM,
- /**
- * @SKB_DROP_REASON_IP_INHDR: there is something wrong with IP header (see
- * IPSTATS_MIB_INHDRERRORS)
- */
- SKB_DROP_REASON_IP_INHDR,
- /**
- * @SKB_DROP_REASON_IP_RPFILTER: IP rpfilter validate failed. see the
- * document for rp_filter in ip-sysctl.rst for more information
- */
- SKB_DROP_REASON_IP_RPFILTER,
- /**
- * @SKB_DROP_REASON_UNICAST_IN_L2_MULTICAST: destination address of L2 is
- * multicast, but L3 is unicast.
- */
- SKB_DROP_REASON_UNICAST_IN_L2_MULTICAST,
- /** @SKB_DROP_REASON_XFRM_POLICY: xfrm policy check failed */
- SKB_DROP_REASON_XFRM_POLICY,
- /** @SKB_DROP_REASON_IP_NOPROTO: no support for IP protocol */
- SKB_DROP_REASON_IP_NOPROTO,
- /** @SKB_DROP_REASON_SOCKET_RCVBUFF: socket receive buff is full */
- SKB_DROP_REASON_SOCKET_RCVBUFF,
- /**
- * @SKB_DROP_REASON_PROTO_MEM: proto memory limition, such as udp packet
- * drop out of udp_memory_allocated.
- */
- SKB_DROP_REASON_PROTO_MEM,
- /**
- * @SKB_DROP_REASON_TCP_MD5NOTFOUND: no MD5 hash and one expected,
- * corresponding to LINUX_MIB_TCPMD5NOTFOUND
- */
- SKB_DROP_REASON_TCP_MD5NOTFOUND,
- /**
- * @SKB_DROP_REASON_TCP_MD5UNEXPECTED: MD5 hash and we're not expecting
- * one, corresponding to LINUX_MIB_TCPMD5UNEXPECTED
- */
- SKB_DROP_REASON_TCP_MD5UNEXPECTED,
- /**
- * @SKB_DROP_REASON_TCP_MD5FAILURE: MD5 hash and its wrong, corresponding
- * to LINUX_MIB_TCPMD5FAILURE
- */
- SKB_DROP_REASON_TCP_MD5FAILURE,
- /**
- * @SKB_DROP_REASON_SOCKET_BACKLOG: failed to add skb to socket backlog (
- * see LINUX_MIB_TCPBACKLOGDROP)
- */
- SKB_DROP_REASON_SOCKET_BACKLOG,
- /** @SKB_DROP_REASON_TCP_FLAGS: TCP flags invalid */
- SKB_DROP_REASON_TCP_FLAGS,
- /**
- * @SKB_DROP_REASON_TCP_ZEROWINDOW: TCP receive window size is zero,
- * see LINUX_MIB_TCPZEROWINDOWDROP
- */
- SKB_DROP_REASON_TCP_ZEROWINDOW,
- /**
- * @SKB_DROP_REASON_TCP_OLD_DATA: the TCP data reveived is already
- * received before (spurious retrans may happened), see
- * LINUX_MIB_DELAYEDACKLOST
- */
- SKB_DROP_REASON_TCP_OLD_DATA,
- /**
- * @SKB_DROP_REASON_TCP_OVERWINDOW: the TCP data is out of window,
- * the seq of the first byte exceed the right edges of receive
- * window
- */
- SKB_DROP_REASON_TCP_OVERWINDOW,
- /**
- * @SKB_DROP_REASON_TCP_OFOMERGE: the data of skb is already in the ofo
- * queue, corresponding to LINUX_MIB_TCPOFOMERGE
- */
- SKB_DROP_REASON_TCP_OFOMERGE,
- /**
- * @SKB_DROP_REASON_TCP_RFC7323_PAWS: PAWS check, corresponding to
- * LINUX_MIB_PAWSESTABREJECTED
- */
- SKB_DROP_REASON_TCP_RFC7323_PAWS,
- /** @SKB_DROP_REASON_TCP_INVALID_SEQUENCE: Not acceptable SEQ field */
- SKB_DROP_REASON_TCP_INVALID_SEQUENCE,
- /** @SKB_DROP_REASON_TCP_RESET: Invalid RST packet */
- SKB_DROP_REASON_TCP_RESET,
- /**
- * @SKB_DROP_REASON_TCP_INVALID_SYN: Incoming packet has unexpected
- * SYN flag
- */
- SKB_DROP_REASON_TCP_INVALID_SYN,
- /** @SKB_DROP_REASON_TCP_CLOSE: TCP socket in CLOSE state */
- SKB_DROP_REASON_TCP_CLOSE,
- /** @SKB_DROP_REASON_TCP_FASTOPEN: dropped by FASTOPEN request socket */
- SKB_DROP_REASON_TCP_FASTOPEN,
- /** @SKB_DROP_REASON_TCP_OLD_ACK: TCP ACK is old, but in window */
- SKB_DROP_REASON_TCP_OLD_ACK,
- /** @SKB_DROP_REASON_TCP_TOO_OLD_ACK: TCP ACK is too old */
- SKB_DROP_REASON_TCP_TOO_OLD_ACK,
- /**
- * @SKB_DROP_REASON_TCP_ACK_UNSENT_DATA: TCP ACK for data we haven't
- * sent yet
- */
- SKB_DROP_REASON_TCP_ACK_UNSENT_DATA,
- /** @SKB_DROP_REASON_TCP_OFO_QUEUE_PRUNE: pruned from TCP OFO queue */
- SKB_DROP_REASON_TCP_OFO_QUEUE_PRUNE,
- /** @SKB_DROP_REASON_TCP_OFO_DROP: data already in receive queue */
- SKB_DROP_REASON_TCP_OFO_DROP,
- /** @SKB_DROP_REASON_IP_OUTNOROUTES: route lookup failed */
- SKB_DROP_REASON_IP_OUTNOROUTES,
- /**
- * @SKB_DROP_REASON_BPF_CGROUP_EGRESS: dropped by BPF_PROG_TYPE_CGROUP_SKB
- * eBPF program
- */
- SKB_DROP_REASON_BPF_CGROUP_EGRESS,
- /** @SKB_DROP_REASON_IPV6DISABLED: IPv6 is disabled on the device */
- SKB_DROP_REASON_IPV6DISABLED,
- /** @SKB_DROP_REASON_NEIGH_CREATEFAIL: failed to create neigh entry */
- SKB_DROP_REASON_NEIGH_CREATEFAIL,
- /** @SKB_DROP_REASON_NEIGH_FAILED: neigh entry in failed state */
- SKB_DROP_REASON_NEIGH_FAILED,
- /** @SKB_DROP_REASON_NEIGH_QUEUEFULL: arp_queue for neigh entry is full */
- SKB_DROP_REASON_NEIGH_QUEUEFULL,
- /** @SKB_DROP_REASON_NEIGH_DEAD: neigh entry is dead */
- SKB_DROP_REASON_NEIGH_DEAD,
- /** @SKB_DROP_REASON_TC_EGRESS: dropped in TC egress HOOK */
- SKB_DROP_REASON_TC_EGRESS,
- /**
- * @SKB_DROP_REASON_QDISC_DROP: dropped by qdisc when packet outputting (
- * failed to enqueue to current qdisc)
- */
- SKB_DROP_REASON_QDISC_DROP,
- /**
- * @SKB_DROP_REASON_CPU_BACKLOG: failed to enqueue the skb to the per CPU
- * backlog queue. This can be caused by backlog queue full (see
- * netdev_max_backlog in net.rst) or RPS flow limit
- */
- SKB_DROP_REASON_CPU_BACKLOG,
- /** @SKB_DROP_REASON_XDP: dropped by XDP in input path */
- SKB_DROP_REASON_XDP,
- /** @SKB_DROP_REASON_TC_INGRESS: dropped in TC ingress HOOK */
- SKB_DROP_REASON_TC_INGRESS,
- /** @SKB_DROP_REASON_UNHANDLED_PROTO: protocol not implemented or not supported */
- SKB_DROP_REASON_UNHANDLED_PROTO,
- /** @SKB_DROP_REASON_SKB_CSUM: sk_buff checksum computation error */
- SKB_DROP_REASON_SKB_CSUM,
- /** @SKB_DROP_REASON_SKB_GSO_SEG: gso segmentation error */
- SKB_DROP_REASON_SKB_GSO_SEG,
- /**
- * @SKB_DROP_REASON_SKB_UCOPY_FAULT: failed to copy data from user space,
- * e.g., via zerocopy_sg_from_iter() or skb_orphan_frags_rx()
- */
- SKB_DROP_REASON_SKB_UCOPY_FAULT,
- /** @SKB_DROP_REASON_DEV_HDR: device driver specific header/metadata is invalid */
- SKB_DROP_REASON_DEV_HDR,
- /**
- * @SKB_DROP_REASON_DEV_READY: the device is not ready to xmit/recv due to
- * any of its data structure that is not up/ready/initialized,
- * e.g., the IFF_UP is not set, or driver specific tun->tfiles[txq]
- * is not initialized
- */
- SKB_DROP_REASON_DEV_READY,
- /** @SKB_DROP_REASON_FULL_RING: ring buffer is full */
- SKB_DROP_REASON_FULL_RING,
- /** @SKB_DROP_REASON_NOMEM: error due to OOM */
- SKB_DROP_REASON_NOMEM,
- /**
- * @SKB_DROP_REASON_HDR_TRUNC: failed to trunc/extract the header from
- * networking data, e.g., failed to pull the protocol header from
- * frags via pskb_may_pull()
- */
- SKB_DROP_REASON_HDR_TRUNC,
- /**
- * @SKB_DROP_REASON_TAP_FILTER: dropped by (ebpf) filter directly attached
- * to tun/tap, e.g., via TUNSETFILTEREBPF
- */
- SKB_DROP_REASON_TAP_FILTER,
- /**
- * @SKB_DROP_REASON_TAP_TXFILTER: dropped by tx filter implemented at
- * tun/tap, e.g., check_filter()
- */
- SKB_DROP_REASON_TAP_TXFILTER,
- /** @SKB_DROP_REASON_ICMP_CSUM: ICMP checksum error */
- SKB_DROP_REASON_ICMP_CSUM,
- /**
- * @SKB_DROP_REASON_INVALID_PROTO: the packet doesn't follow RFC 2211,
- * such as a broadcasts ICMP_TIMESTAMP
- */
- SKB_DROP_REASON_INVALID_PROTO,
- /**
- * @SKB_DROP_REASON_IP_INADDRERRORS: host unreachable, corresponding to
- * IPSTATS_MIB_INADDRERRORS
- */
- SKB_DROP_REASON_IP_INADDRERRORS,
- /**
- * @SKB_DROP_REASON_IP_INNOROUTES: network unreachable, corresponding to
- * IPSTATS_MIB_INADDRERRORS
- */
- SKB_DROP_REASON_IP_INNOROUTES,
- /**
- * @SKB_DROP_REASON_PKT_TOO_BIG: packet size is too big (maybe exceed the
- * MTU)
- */
- SKB_DROP_REASON_PKT_TOO_BIG,
- /** @SKB_DROP_REASON_DUP_FRAG: duplicate fragment */
- SKB_DROP_REASON_DUP_FRAG,
- /** @SKB_DROP_REASON_FRAG_REASM_TIMEOUT: fragment reassembly timeout */
- SKB_DROP_REASON_FRAG_REASM_TIMEOUT,
- /**
- * @SKB_DROP_REASON_FRAG_TOO_FAR: ipv4 fragment too far.
- * (/proc/sys/net/ipv4/ipfrag_max_dist)
- */
- SKB_DROP_REASON_FRAG_TOO_FAR,
- /**
- * @SKB_DROP_REASON_TCP_MINTTL: ipv4 ttl or ipv6 hoplimit below
- * the threshold (IP_MINTTL or IPV6_MINHOPCOUNT).
- */
- SKB_DROP_REASON_TCP_MINTTL,
- /** @SKB_DROP_REASON_IPV6_BAD_EXTHDR: Bad IPv6 extension header. */
- SKB_DROP_REASON_IPV6_BAD_EXTHDR,
- /** @SKB_DROP_REASON_IPV6_NDISC_FRAG: invalid frag (suppress_frag_ndisc). */
- SKB_DROP_REASON_IPV6_NDISC_FRAG,
- /** @SKB_DROP_REASON_IPV6_NDISC_HOP_LIMIT: invalid hop limit. */
- SKB_DROP_REASON_IPV6_NDISC_HOP_LIMIT,
- /** @SKB_DROP_REASON_IPV6_NDISC_BAD_CODE: invalid NDISC icmp6 code. */
- SKB_DROP_REASON_IPV6_NDISC_BAD_CODE,
- /** @SKB_DROP_REASON_IPV6_NDISC_BAD_OPTIONS: invalid NDISC options. */
- SKB_DROP_REASON_IPV6_NDISC_BAD_OPTIONS,
- /** @SKB_DROP_REASON_IPV6_NDISC_NS_OTHERHOST: NEIGHBOUR SOLICITATION
- * for another host.
- */
- SKB_DROP_REASON_IPV6_NDISC_NS_OTHERHOST,
- /**
- * @SKB_DROP_REASON_MAX: the maximum of drop reason, which shouldn't be
- * used as a real 'reason'
- */
- SKB_DROP_REASON_MAX,
+ SKB_DROP_REASON_SUBSYS_MAC80211_MONITOR,
+
+ /** @SKB_DROP_REASON_SUBSYS_NUM: number of subsystems defined */
+ SKB_DROP_REASON_SUBSYS_NUM
};
-#define SKB_DR_INIT(name, reason) \
- enum skb_drop_reason name = SKB_DROP_REASON_##reason
-#define SKB_DR(name) \
- SKB_DR_INIT(name, NOT_SPECIFIED)
-#define SKB_DR_SET(name, reason) \
- (name = SKB_DROP_REASON_##reason)
-#define SKB_DR_OR(name, reason) \
- do { \
- if (name == SKB_DROP_REASON_NOT_SPECIFIED || \
- name == SKB_NOT_DROPPED_YET) \
- SKB_DR_SET(name, reason); \
- } while (0)
+struct drop_reason_list {
+ const char * const *reasons;
+ size_t n_reasons;
+};
-extern const char * const drop_reasons[];
+/* Note: due to dynamic registrations, access must be under RCU */
+extern const struct drop_reason_list __rcu *
+drop_reasons_by_subsys[SKB_DROP_REASON_SUBSYS_NUM];
+
+void drop_reasons_register_subsys(enum skb_drop_reason_subsys subsys,
+ const struct drop_reason_list *list);
+void drop_reasons_unregister_subsys(enum skb_drop_reason_subsys subsys);
#endif
diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h
index 5ccf52e..85b2281 100644
--- a/include/net/flow_dissector.h
+++ b/include/net/flow_dissector.h
@@ -14,7 +14,9 @@ struct sk_buff;
/**
* struct flow_dissector_key_control:
- * @thoff: Transport header offset
+ * @thoff: Transport header offset
+ * @addr_type: Type of key. One of FLOW_DISSECTOR_KEY_*
+ * @flags: Key flags. Any of FLOW_DIS_(IS_FRAGMENT|FIRST_FRAGENCAPSULATION)
*/
struct flow_dissector_key_control {
u16 thoff;
@@ -36,8 +38,9 @@ enum flow_dissect_ret {
/**
* struct flow_dissector_key_basic:
- * @n_proto: Network header protocol (eg. IPv4/IPv6)
+ * @n_proto: Network header protocol (eg. IPv4/IPv6)
* @ip_proto: Transport header protocol (eg. TCP/UDP)
+ * @padding: Unused
*/
struct flow_dissector_key_basic {
__be16 n_proto;
@@ -135,6 +138,7 @@ struct flow_dissector_key_tipc {
* struct flow_dissector_key_addrs:
* @v4addrs: IPv4 addresses
* @v6addrs: IPv6 addresses
+ * @tipckey: TIPC key
*/
struct flow_dissector_key_addrs {
union {
@@ -145,14 +149,12 @@ struct flow_dissector_key_addrs {
};
/**
- * flow_dissector_key_arp:
- * @ports: Operation, source and target addresses for an ARP header
- * for Ethernet hardware addresses and IPv4 protocol addresses
- * sip: Sender IP address
- * tip: Target IP address
- * op: Operation
- * sha: Sender hardware address
- * tpa: Target hardware address
+ * struct flow_dissector_key_arp:
+ * @sip: Sender IP address
+ * @tip: Target IP address
+ * @op: Operation
+ * @sha: Sender hardware address
+ * @tha: Target hardware address
*/
struct flow_dissector_key_arp {
__u32 sip;
@@ -163,10 +165,10 @@ struct flow_dissector_key_arp {
};
/**
- * flow_dissector_key_tp_ports:
- * @ports: port numbers of Transport header
- * src: source port number
- * dst: destination port number
+ * struct flow_dissector_key_ports:
+ * @ports: port numbers of Transport header
+ * @src: source port number
+ * @dst: destination port number
*/
struct flow_dissector_key_ports {
union {
@@ -195,10 +197,10 @@ struct flow_dissector_key_ports_range {
};
/**
- * flow_dissector_key_icmp:
- * type: ICMP type
- * code: ICMP code
- * id: session identifier
+ * struct flow_dissector_key_icmp:
+ * @type: ICMP type
+ * @code: ICMP code
+ * @id: Session identifier
*/
struct flow_dissector_key_icmp {
struct {
diff --git a/include/net/handshake.h b/include/net/handshake.h
new file mode 100644
index 0000000..3352b1a
--- /dev/null
+++ b/include/net/handshake.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Generic netlink HANDSHAKE service.
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2023, Oracle and/or its affiliates.
+ */
+
+#ifndef _NET_HANDSHAKE_H
+#define _NET_HANDSHAKE_H
+
+enum {
+ TLS_NO_KEYRING = 0,
+ TLS_NO_PEERID = 0,
+ TLS_NO_CERT = 0,
+ TLS_NO_PRIVKEY = 0,
+};
+
+typedef void (*tls_done_func_t)(void *data, int status,
+ key_serial_t peerid);
+
+struct tls_handshake_args {
+ struct socket *ta_sock;
+ tls_done_func_t ta_done;
+ void *ta_data;
+ unsigned int ta_timeout_ms;
+ key_serial_t ta_keyring;
+ key_serial_t ta_my_cert;
+ key_serial_t ta_my_privkey;
+ unsigned int ta_num_peerids;
+ key_serial_t ta_my_peerids[5];
+};
+
+int tls_client_hello_anon(const struct tls_handshake_args *args, gfp_t flags);
+int tls_client_hello_x509(const struct tls_handshake_args *args, gfp_t flags);
+int tls_client_hello_psk(const struct tls_handshake_args *args, gfp_t flags);
+int tls_server_hello_x509(const struct tls_handshake_args *args, gfp_t flags);
+int tls_server_hello_psk(const struct tls_handshake_args *args, gfp_t flags);
+
+bool tls_handshake_cancel(struct sock *sk);
+
+#endif /* _NET_HANDSHAKE_H */
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index b23ddec..325ad89 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -7,7 +7,7 @@
#include <linux/in6.h>
#include <linux/rbtree_types.h>
#include <linux/refcount.h>
-#include <net/dropreason.h>
+#include <net/dropreason-core.h>
/* Per netns frag queues directory */
struct fqdir {
diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index 679421d..ac0370e 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -3950,6 +3950,10 @@ struct ieee80211_prep_tx_info {
* Note that vif can be NULL.
* The callback can sleep.
*
+ * @flush_sta: Flush or drop all pending frames from the hardware queue(s) for
+ * the given station, as it's about to be removed.
+ * The callback can sleep.
+ *
* @channel_switch: Drivers that need (or want) to offload the channel
* switch operation for CSAs received from the AP may implement this
* callback. They must then call ieee80211_chswitch_done() to indicate
@@ -4415,6 +4419,8 @@ struct ieee80211_ops {
#endif
void (*flush)(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
u32 queues, bool drop);
+ void (*flush_sta)(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta);
void (*channel_switch)(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_channel_switch *ch_switch);
@@ -5210,26 +5216,6 @@ void ieee80211_tx_status_irqsafe(struct ieee80211_hw *hw,
struct sk_buff *skb);
/**
- * ieee80211_tx_status_8023 - transmit status callback for 802.3 frame format
- *
- * Call this function for all transmitted data frames after their transmit
- * completion. This callback should only be called for data frames which
- * are using driver's (or hardware's) offload capability of encap/decap
- * 802.11 frames.
- *
- * This function may not be called in IRQ context. Calls to this function
- * for a single hardware must be synchronized against each other and all
- * calls in the same tx status family.
- *
- * @hw: the hardware the frame was transmitted by
- * @vif: the interface for which the frame was transmitted
- * @skb: the frame that was transmitted, owned by mac80211 after this call
- */
-void ieee80211_tx_status_8023(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif,
- struct sk_buff *skb);
-
-/**
* ieee80211_report_low_ack - report non-responding station
*
* When operating in AP-mode, call this function to report a non-responding
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 56189e4..96c1201 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -145,6 +145,7 @@ struct gdma_general_req {
}; /* HW DATA */
#define GDMA_MESSAGE_V1 1
+#define GDMA_MESSAGE_V2 2
struct gdma_general_resp {
struct gdma_resp_hdr hdr;
@@ -354,6 +355,9 @@ struct gdma_context {
struct gdma_resource msix_resource;
struct gdma_irq_context *irq_contexts;
+ /* L2 MTU */
+ u16 adapter_mtu;
+
/* This maps a CQ index to the queue structure. */
unsigned int max_num_cqs;
struct gdma_queue **cq_table;
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index bb11a65..cd386aa 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -36,10 +36,8 @@ enum TRI_STATE {
#define COMP_ENTRY_SIZE 64
-#define ADAPTER_MTU_SIZE 1500
-#define MAX_FRAME_SIZE (ADAPTER_MTU_SIZE + 14)
-
#define RX_BUFFERS_PER_QUEUE 512
+#define MANA_RX_DATA_ALIGN 64
#define MAX_SEND_BUFFERS_PER_QUEUE 256
@@ -282,7 +280,6 @@ struct mana_recv_buf_oob {
struct gdma_wqe_request wqe_req;
void *buf_va;
- dma_addr_t buf_dma_addr;
/* SGL of the buffer going to be sent has part of the work request. */
u32 num_sge;
@@ -295,6 +292,11 @@ struct mana_recv_buf_oob {
struct gdma_posted_wqe_info wqe_inf;
};
+#define MANA_RXBUF_PAD (SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) \
+ + ETH_HLEN)
+
+#define MANA_XDP_MTU_MAX (PAGE_SIZE - MANA_RXBUF_PAD - XDP_PACKET_HEADROOM)
+
struct mana_rxq {
struct gdma_queue *gdma_rq;
/* Cache the gdma receive queue id */
@@ -304,6 +306,8 @@ struct mana_rxq {
u32 rxq_idx;
u32 datasize;
+ u32 alloc_size;
+ u32 headroom;
mana_handle_t rxobj;
@@ -322,7 +326,7 @@ struct mana_rxq {
struct bpf_prog __rcu *bpf_prog;
struct xdp_rxq_info xdp_rxq;
- struct page *xdp_save_page;
+ void *xdp_save_va; /* for reusing */
bool xdp_flush;
int xdp_rc; /* XDP redirect return code */
@@ -387,6 +391,14 @@ struct mana_port_context {
/* This points to an array of num_queues of RQ pointers. */
struct mana_rxq **rxqs;
+ /* pre-allocated rx buffer array */
+ void **rxbufs_pre;
+ dma_addr_t *das_pre;
+ int rxbpre_total;
+ u32 rxbpre_datasize;
+ u32 rxbpre_alloc_size;
+ u32 rxbpre_headroom;
+
struct bpf_prog *bpf_prog;
/* Create num_queues EQs, SQs, SQ-CQs, RQs and RQ-CQs, respectively. */
@@ -486,6 +498,11 @@ struct mana_query_device_cfg_resp {
u16 max_num_vports;
u16 reserved;
u32 max_num_eqs;
+
+ /* response v2: */
+ u16 adapter_mtu;
+ u16 reserved2;
+ u32 reserved3;
}; /* HW DATA */
/* Query vPort Configuration */
diff --git a/include/net/netdev_queues.h b/include/net/netdev_queues.h
index b26fdb4..d68b0a4 100644
--- a/include/net/netdev_queues.h
+++ b/include/net/netdev_queues.h
@@ -160,4 +160,14 @@ netdev_txq_completed_mb(struct netdev_queue *dev_queue,
netif_txq_maybe_stop(txq, get_desc, stop_thrs, start_thrs); \
})
+#define netif_subqueue_completed_wake(dev, idx, pkts, bytes, \
+ get_desc, start_thrs) \
+ ({ \
+ struct netdev_queue *txq; \
+ \
+ txq = netdev_get_tx_queue(dev, idx); \
+ netif_txq_completed_wake(txq, pkts, bytes, \
+ get_desc, start_thrs); \
+ })
+
#endif
diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h
index 9430128..1b8e305 100644
--- a/include/net/netfilter/nf_tables.h
+++ b/include/net/netfilter/nf_tables.h
@@ -1085,6 +1085,10 @@ struct nft_chain {
};
int nft_chain_validate(const struct nft_ctx *ctx, const struct nft_chain *chain);
+int nft_setelem_validate(const struct nft_ctx *ctx, struct nft_set *set,
+ const struct nft_set_iter *iter,
+ struct nft_set_elem *elem);
+int nft_set_catchall_validate(const struct nft_ctx *ctx, struct nft_set *set);
enum nft_chain_types {
NFT_CHAIN_T_DEFAULT = 0,
diff --git a/include/net/netns/ipv6.h b/include/net/netns/ipv6.h
index b4af483..3cceb3e 100644
--- a/include/net/netns/ipv6.h
+++ b/include/net/netns/ipv6.h
@@ -55,6 +55,7 @@ struct netns_sysctl_ipv6 {
u64 ioam6_id_wide;
bool skip_notify_on_dev_down;
u8 fib_notify_on_flag_change;
+ u8 icmpv6_error_anycast_as_unicast;
};
struct netns_ipv6 {
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index ddfa0b3..c8ec2f3 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -77,6 +77,7 @@ struct page_pool_params {
unsigned int pool_size;
int nid; /* Numa node id to allocate from pages from */
struct device *dev; /* device, for DMA pre-mapping purposes */
+ struct napi_struct *napi; /* Sole consumer of pages, otherwise NULL */
enum dma_data_direction dma_dir; /* DMA mapping direction */
unsigned int max_len; /* max DMA sync memory size */
unsigned int offset; /* DMA addr offset */
@@ -239,13 +240,14 @@ inline enum dma_data_direction page_pool_get_dma_dir(struct page_pool *pool)
return pool->p.dma_dir;
}
-bool page_pool_return_skb_page(struct page *page);
+bool page_pool_return_skb_page(struct page *page, bool napi_safe);
struct page_pool *page_pool_create(const struct page_pool_params *params);
struct xdp_mem_info;
#ifdef CONFIG_PAGE_POOL
+void page_pool_unlink_napi(struct page_pool *pool);
void page_pool_destroy(struct page_pool *pool);
void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
struct xdp_mem_info *mem);
@@ -253,6 +255,10 @@ void page_pool_release_page(struct page_pool *pool, struct page *page);
void page_pool_put_page_bulk(struct page_pool *pool, void **data,
int count);
#else
+static inline void page_pool_unlink_napi(struct page_pool *pool)
+{
+}
+
static inline void page_pool_destroy(struct page_pool *pool)
{
}
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index bb0bd69..f436688 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -166,11 +166,13 @@ struct tc_mqprio_caps {
struct tc_mqprio_qopt_offload {
/* struct tc_mqprio_qopt must always be the first element */
struct tc_mqprio_qopt qopt;
+ struct netlink_ext_ack *extack;
u16 mode;
u16 shaper;
u32 flags;
u64 min_rate[TC_QOPT_MAX_QUEUE];
u64 max_rate[TC_QOPT_MAX_QUEUE];
+ unsigned long preemptible_tcs;
};
struct tc_taprio_caps {
@@ -193,6 +195,7 @@ struct tc_taprio_sched_entry {
struct tc_taprio_qopt_offload {
struct tc_mqprio_qopt_offload mqprio;
+ struct netlink_ext_ack *extack;
u8 enable;
ktime_t base_time;
u64 cycle_time;
diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h
index c335dd0..2a67100 100644
--- a/include/net/sctp/sctp.h
+++ b/include/net/sctp/sctp.h
@@ -425,11 +425,11 @@ static inline bool sctp_chunk_pending(const struct sctp_chunk *chunk)
* the chunk length to indicate when to stop. Make sure
* there is room for a param header too.
*/
-#define sctp_walk_params(pos, chunk, member)\
-_sctp_walk_params((pos), (chunk), ntohs((chunk)->chunk_hdr.length), member)
+#define sctp_walk_params(pos, chunk)\
+_sctp_walk_params((pos), (chunk), ntohs((chunk)->chunk_hdr.length))
-#define _sctp_walk_params(pos, chunk, end, member)\
-for (pos.v = chunk->member;\
+#define _sctp_walk_params(pos, chunk, end)\
+for (pos.v = (u8 *)(chunk + 1);\
(pos.v + offsetof(struct sctp_paramhdr, length) + sizeof(pos.p->length) <=\
(void *)chunk + end) &&\
pos.v <= (void *)chunk + end - ntohs(pos.p->length) &&\
@@ -452,8 +452,8 @@ for (err = (struct sctp_errhdr *)((void *)chunk_hdr + \
_sctp_walk_fwdtsn((pos), (chunk), ntohs((chunk)->chunk_hdr->length) - sizeof(struct sctp_fwdtsn_chunk))
#define _sctp_walk_fwdtsn(pos, chunk, end)\
-for (pos = chunk->subh.fwdtsn_hdr->skip;\
- (void *)pos <= (void *)chunk->subh.fwdtsn_hdr->skip + end - sizeof(struct sctp_fwdtsn_skip);\
+for (pos = (void *)(chunk->subh.fwdtsn_hdr + 1);\
+ (void *)pos <= (void *)(chunk->subh.fwdtsn_hdr + 1) + end - sizeof(struct sctp_fwdtsn_skip);\
pos++)
/* External references. */
diff --git a/include/net/sctp/structs.h b/include/net/sctp/structs.h
index a0933ef..5c72d18 100644
--- a/include/net/sctp/structs.h
+++ b/include/net/sctp/structs.h
@@ -332,7 +332,7 @@ struct sctp_cookie {
* the association TCB is re-constructed from the cookie.
*/
__u32 raw_addr_list_len;
- struct sctp_init_chunk peer_init[];
+ /* struct sctp_init_chunk peer_init[]; */
};
@@ -1711,7 +1711,6 @@ struct sctp_association {
__u16 ecn_capable:1, /* Can peer do ECN? */
ipv4_address:1, /* Peer understands IPv4 addresses? */
ipv6_address:1, /* Peer understands IPv6 addresses? */
- hostname_address:1, /* Peer understands DNS addresses? */
asconf_capable:1, /* Does peer support ADDIP? */
prsctp_capable:1, /* Can peer do PR-SCTP? */
reconf_capable:1, /* Can peer do RE-CONFIG? */
diff --git a/include/net/sock.h b/include/net/sock.h
index 5edf003..8b7ed71 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2697,7 +2697,7 @@ sock_recv_timestamp(struct msghdr *msg, struct sock *sk, struct sk_buff *skb)
else
sock_write_timestamp(sk, kt);
- if (sock_flag(sk, SOCK_WIFI_STATUS) && skb->wifi_acked_valid)
+ if (sock_flag(sk, SOCK_WIFI_STATUS) && skb_wifi_acked_valid(skb))
__sock_recv_wifi_status(msg, sk, skb);
}
diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
index d757b5e..cb8fbb2 100644
--- a/include/soc/mscc/ocelot.h
+++ b/include/soc/mscc/ocelot.h
@@ -11,6 +11,8 @@
#include <linux/regmap.h>
#include <net/dsa.h>
+struct tc_mqprio_qopt_offload;
+
/* Port Group IDs (PGID) are masks of destination ports.
*
* For L2 forwarding, the switch performs 3 lookups in the PGID table for each
@@ -744,9 +746,11 @@ struct ocelot_mirror {
};
struct ocelot_mm_state {
- struct mutex lock;
enum ethtool_mm_verify_status verify_status;
+ bool tx_enabled;
bool tx_active;
+ u8 preemptible_tcs;
+ u8 active_preemptible_tcs;
};
struct ocelot_port;
@@ -940,15 +944,17 @@ struct ocelot_policer {
__ocelot_target_write_ix(ocelot, target, val, reg, 0)
/* I/O */
-u32 ocelot_port_readl(struct ocelot_port *port, u32 reg);
-void ocelot_port_writel(struct ocelot_port *port, u32 val, u32 reg);
-void ocelot_port_rmwl(struct ocelot_port *port, u32 val, u32 mask, u32 reg);
-int __ocelot_bulk_read_ix(struct ocelot *ocelot, u32 reg, u32 offset, void *buf,
- int count);
-u32 __ocelot_read_ix(struct ocelot *ocelot, u32 reg, u32 offset);
-void __ocelot_write_ix(struct ocelot *ocelot, u32 val, u32 reg, u32 offset);
-void __ocelot_rmw_ix(struct ocelot *ocelot, u32 val, u32 mask, u32 reg,
- u32 offset);
+u32 ocelot_port_readl(struct ocelot_port *port, enum ocelot_reg reg);
+void ocelot_port_writel(struct ocelot_port *port, u32 val, enum ocelot_reg reg);
+void ocelot_port_rmwl(struct ocelot_port *port, u32 val, u32 mask,
+ enum ocelot_reg reg);
+int __ocelot_bulk_read_ix(struct ocelot *ocelot, enum ocelot_reg reg,
+ u32 offset, void *buf, int count);
+u32 __ocelot_read_ix(struct ocelot *ocelot, enum ocelot_reg reg, u32 offset);
+void __ocelot_write_ix(struct ocelot *ocelot, u32 val, enum ocelot_reg reg,
+ u32 offset);
+void __ocelot_rmw_ix(struct ocelot *ocelot, u32 val, u32 mask,
+ enum ocelot_reg reg, u32 offset);
u32 __ocelot_target_read_ix(struct ocelot *ocelot, enum ocelot_target target,
u32 reg, u32 offset);
void __ocelot_target_write_ix(struct ocelot *ocelot, enum ocelot_target target,
@@ -1146,12 +1152,15 @@ int ocelot_vcap_policer_add(struct ocelot *ocelot, u32 pol_ix,
struct ocelot_policer *pol);
int ocelot_vcap_policer_del(struct ocelot *ocelot, u32 pol_ix);
-void ocelot_port_mm_irq(struct ocelot *ocelot, int port);
+void ocelot_mm_irq(struct ocelot *ocelot);
int ocelot_port_set_mm(struct ocelot *ocelot, int port,
struct ethtool_mm_cfg *cfg,
struct netlink_ext_ack *extack);
int ocelot_port_get_mm(struct ocelot *ocelot, int port,
struct ethtool_mm_state *state);
+int ocelot_port_mqprio(struct ocelot *ocelot, int port,
+ struct tc_mqprio_qopt_offload *mqprio);
+void ocelot_port_update_preemptible_tcs(struct ocelot *ocelot, int port);
#if IS_ENABLED(CONFIG_BRIDGE_MRP)
int ocelot_mrp_add(struct ocelot *ocelot, int port,
diff --git a/include/trace/events/handshake.h b/include/trace/events/handshake.h
new file mode 100644
index 0000000..8dadcab
--- /dev/null
+++ b/include/trace/events/handshake.h
@@ -0,0 +1,159 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM handshake
+
+#if !defined(_TRACE_HANDSHAKE_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HANDSHAKE_H
+
+#include <linux/net.h>
+#include <linux/tracepoint.h>
+
+DECLARE_EVENT_CLASS(handshake_event_class,
+ TP_PROTO(
+ const struct net *net,
+ const struct handshake_req *req,
+ const struct sock *sk
+ ),
+ TP_ARGS(net, req, sk),
+ TP_STRUCT__entry(
+ __field(const void *, req)
+ __field(const void *, sk)
+ __field(unsigned int, netns_ino)
+ ),
+ TP_fast_assign(
+ __entry->req = req;
+ __entry->sk = sk;
+ __entry->netns_ino = net->ns.inum;
+ ),
+ TP_printk("req=%p sk=%p",
+ __entry->req, __entry->sk
+ )
+);
+#define DEFINE_HANDSHAKE_EVENT(name) \
+ DEFINE_EVENT(handshake_event_class, name, \
+ TP_PROTO( \
+ const struct net *net, \
+ const struct handshake_req *req, \
+ const struct sock *sk \
+ ), \
+ TP_ARGS(net, req, sk))
+
+DECLARE_EVENT_CLASS(handshake_fd_class,
+ TP_PROTO(
+ const struct net *net,
+ const struct handshake_req *req,
+ const struct sock *sk,
+ int fd
+ ),
+ TP_ARGS(net, req, sk, fd),
+ TP_STRUCT__entry(
+ __field(const void *, req)
+ __field(const void *, sk)
+ __field(int, fd)
+ __field(unsigned int, netns_ino)
+ ),
+ TP_fast_assign(
+ __entry->req = req;
+ __entry->sk = req->hr_sk;
+ __entry->fd = fd;
+ __entry->netns_ino = net->ns.inum;
+ ),
+ TP_printk("req=%p sk=%p fd=%d",
+ __entry->req, __entry->sk, __entry->fd
+ )
+);
+#define DEFINE_HANDSHAKE_FD_EVENT(name) \
+ DEFINE_EVENT(handshake_fd_class, name, \
+ TP_PROTO( \
+ const struct net *net, \
+ const struct handshake_req *req, \
+ const struct sock *sk, \
+ int fd \
+ ), \
+ TP_ARGS(net, req, sk, fd))
+
+DECLARE_EVENT_CLASS(handshake_error_class,
+ TP_PROTO(
+ const struct net *net,
+ const struct handshake_req *req,
+ const struct sock *sk,
+ int err
+ ),
+ TP_ARGS(net, req, sk, err),
+ TP_STRUCT__entry(
+ __field(const void *, req)
+ __field(const void *, sk)
+ __field(int, err)
+ __field(unsigned int, netns_ino)
+ ),
+ TP_fast_assign(
+ __entry->req = req;
+ __entry->sk = sk;
+ __entry->err = err;
+ __entry->netns_ino = net->ns.inum;
+ ),
+ TP_printk("req=%p sk=%p err=%d",
+ __entry->req, __entry->sk, __entry->err
+ )
+);
+#define DEFINE_HANDSHAKE_ERROR(name) \
+ DEFINE_EVENT(handshake_error_class, name, \
+ TP_PROTO( \
+ const struct net *net, \
+ const struct handshake_req *req, \
+ const struct sock *sk, \
+ int err \
+ ), \
+ TP_ARGS(net, req, sk, err))
+
+
+/*
+ * Request lifetime events
+ */
+
+DEFINE_HANDSHAKE_EVENT(handshake_submit);
+DEFINE_HANDSHAKE_ERROR(handshake_submit_err);
+DEFINE_HANDSHAKE_EVENT(handshake_cancel);
+DEFINE_HANDSHAKE_EVENT(handshake_cancel_none);
+DEFINE_HANDSHAKE_EVENT(handshake_cancel_busy);
+DEFINE_HANDSHAKE_EVENT(handshake_destruct);
+
+
+TRACE_EVENT(handshake_complete,
+ TP_PROTO(
+ const struct net *net,
+ const struct handshake_req *req,
+ const struct sock *sk,
+ int status
+ ),
+ TP_ARGS(net, req, sk, status),
+ TP_STRUCT__entry(
+ __field(const void *, req)
+ __field(const void *, sk)
+ __field(int, status)
+ __field(unsigned int, netns_ino)
+ ),
+ TP_fast_assign(
+ __entry->req = req;
+ __entry->sk = sk;
+ __entry->status = status;
+ __entry->netns_ino = net->ns.inum;
+ ),
+ TP_printk("req=%p sk=%p status=%d",
+ __entry->req, __entry->sk, __entry->status
+ )
+);
+
+/*
+ * Netlink events
+ */
+
+DEFINE_HANDSHAKE_ERROR(handshake_notify_err);
+DEFINE_HANDSHAKE_FD_EVENT(handshake_cmd_accept);
+DEFINE_HANDSHAKE_ERROR(handshake_cmd_accept_err);
+DEFINE_HANDSHAKE_FD_EVENT(handshake_cmd_done);
+DEFINE_HANDSHAKE_ERROR(handshake_cmd_done_err);
+
+#endif /* _TRACE_HANDSHAKE_H */
+
+#include <trace/define_trace.h>
diff --git a/include/uapi/linux/handshake.h b/include/uapi/linux/handshake.h
new file mode 100644
index 0000000..1de4d0b
--- /dev/null
+++ b/include/uapi/linux/handshake.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/handshake.yaml */
+/* YNL-GEN uapi header */
+
+#ifndef _UAPI_LINUX_HANDSHAKE_H
+#define _UAPI_LINUX_HANDSHAKE_H
+
+#define HANDSHAKE_FAMILY_NAME "handshake"
+#define HANDSHAKE_FAMILY_VERSION 1
+
+enum handshake_handler_class {
+ HANDSHAKE_HANDLER_CLASS_NONE,
+ HANDSHAKE_HANDLER_CLASS_TLSHD,
+ HANDSHAKE_HANDLER_CLASS_MAX,
+};
+
+enum handshake_msg_type {
+ HANDSHAKE_MSG_TYPE_UNSPEC,
+ HANDSHAKE_MSG_TYPE_CLIENTHELLO,
+ HANDSHAKE_MSG_TYPE_SERVERHELLO,
+};
+
+enum handshake_auth {
+ HANDSHAKE_AUTH_UNSPEC,
+ HANDSHAKE_AUTH_UNAUTH,
+ HANDSHAKE_AUTH_PSK,
+ HANDSHAKE_AUTH_X509,
+};
+
+enum {
+ HANDSHAKE_A_X509_CERT = 1,
+ HANDSHAKE_A_X509_PRIVKEY,
+
+ __HANDSHAKE_A_X509_MAX,
+ HANDSHAKE_A_X509_MAX = (__HANDSHAKE_A_X509_MAX - 1)
+};
+
+enum {
+ HANDSHAKE_A_ACCEPT_SOCKFD = 1,
+ HANDSHAKE_A_ACCEPT_HANDLER_CLASS,
+ HANDSHAKE_A_ACCEPT_MESSAGE_TYPE,
+ HANDSHAKE_A_ACCEPT_TIMEOUT,
+ HANDSHAKE_A_ACCEPT_AUTH_MODE,
+ HANDSHAKE_A_ACCEPT_PEER_IDENTITY,
+ HANDSHAKE_A_ACCEPT_CERTIFICATE,
+
+ __HANDSHAKE_A_ACCEPT_MAX,
+ HANDSHAKE_A_ACCEPT_MAX = (__HANDSHAKE_A_ACCEPT_MAX - 1)
+};
+
+enum {
+ HANDSHAKE_A_DONE_STATUS = 1,
+ HANDSHAKE_A_DONE_SOCKFD,
+ HANDSHAKE_A_DONE_REMOTE_AUTH,
+
+ __HANDSHAKE_A_DONE_MAX,
+ HANDSHAKE_A_DONE_MAX = (__HANDSHAKE_A_DONE_MAX - 1)
+};
+
+enum {
+ HANDSHAKE_CMD_READY = 1,
+ HANDSHAKE_CMD_ACCEPT,
+ HANDSHAKE_CMD_DONE,
+
+ __HANDSHAKE_CMD_MAX,
+ HANDSHAKE_CMD_MAX = (__HANDSHAKE_CMD_MAX - 1)
+};
+
+#define HANDSHAKE_MCGRP_NONE "none"
+#define HANDSHAKE_MCGRP_TLSHD "tlshd"
+
+#endif /* _UAPI_LINUX_HANDSHAKE_H */
diff --git a/include/uapi/linux/if_bridge.h b/include/uapi/linux/if_bridge.h
index c9d624f..f95326f 100644
--- a/include/uapi/linux/if_bridge.h
+++ b/include/uapi/linux/if_bridge.h
@@ -525,6 +525,7 @@ enum {
BRIDGE_VLANDB_ENTRY_MCAST_ROUTER,
BRIDGE_VLANDB_ENTRY_MCAST_N_GROUPS,
BRIDGE_VLANDB_ENTRY_MCAST_MAX_GROUPS,
+ BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS,
__BRIDGE_VLANDB_ENTRY_MAX,
};
#define BRIDGE_VLANDB_ENTRY_MAX (__BRIDGE_VLANDB_ENTRY_MAX - 1)
diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h
index 8d67968..4ac1000 100644
--- a/include/uapi/linux/if_link.h
+++ b/include/uapi/linux/if_link.h
@@ -569,6 +569,7 @@ enum {
IFLA_BRPORT_MAB,
IFLA_BRPORT_MCAST_N_GROUPS,
IFLA_BRPORT_MCAST_MAX_GROUPS,
+ IFLA_BRPORT_NEIGH_VLAN_SUPPRESS,
__IFLA_BRPORT_MAX
};
#define IFLA_BRPORT_MAX (__IFLA_BRPORT_MAX - 1)
diff --git a/include/uapi/linux/if_packet.h b/include/uapi/linux/if_packet.h
index 78c981d..9efc423 100644
--- a/include/uapi/linux/if_packet.h
+++ b/include/uapi/linux/if_packet.h
@@ -59,6 +59,7 @@ struct sockaddr_ll {
#define PACKET_ROLLOVER_STATS 21
#define PACKET_FANOUT_DATA 22
#define PACKET_IGNORE_OUTGOING 23
+#define PACKET_VNET_HDR_SZ 24
#define PACKET_FANOUT_HASH 0
#define PACKET_FANOUT_LB 1
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 000eec1..51a7add 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -719,6 +719,11 @@ enum {
#define __TC_MQPRIO_SHAPER_MAX (__TC_MQPRIO_SHAPER_MAX - 1)
+enum {
+ TC_FP_EXPRESS = 1,
+ TC_FP_PREEMPTIBLE = 2,
+};
+
struct tc_mqprio_qopt {
__u8 num_tc;
__u8 prio_tc_map[TC_QOPT_BITMASK + 1];
@@ -733,11 +738,22 @@ struct tc_mqprio_qopt {
#define TC_MQPRIO_F_MAX_RATE 0x8
enum {
+ TCA_MQPRIO_TC_ENTRY_UNSPEC,
+ TCA_MQPRIO_TC_ENTRY_INDEX, /* u32 */
+ TCA_MQPRIO_TC_ENTRY_FP, /* u32 */
+
+ /* add new constants above here */
+ __TCA_MQPRIO_TC_ENTRY_CNT,
+ TCA_MQPRIO_TC_ENTRY_MAX = (__TCA_MQPRIO_TC_ENTRY_CNT - 1)
+};
+
+enum {
TCA_MQPRIO_UNSPEC,
TCA_MQPRIO_MODE,
TCA_MQPRIO_SHAPER,
TCA_MQPRIO_MIN_RATE64,
TCA_MQPRIO_MAX_RATE64,
+ TCA_MQPRIO_TC_ENTRY,
__TCA_MQPRIO_MAX,
};
@@ -1236,6 +1252,7 @@ enum {
TCA_TAPRIO_TC_ENTRY_UNSPEC,
TCA_TAPRIO_TC_ENTRY_INDEX, /* u32 */
TCA_TAPRIO_TC_ENTRY_MAX_SDU, /* u32 */
+ TCA_TAPRIO_TC_ENTRY_FP, /* u32 */
/* add new constants above here */
__TCA_TAPRIO_TC_ENTRY_CNT,
diff --git a/init/initramfs.c b/init/initramfs.c
index f6c112e..e7a01c2 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -60,15 +60,8 @@ static void __init error(char *x)
message = x;
}
-static void panic_show_mem(const char *fmt, ...)
-{
- va_list args;
-
- show_mem(0, NULL);
- va_start(args, fmt);
- panic(fmt, args);
- va_end(args);
-}
+#define panic_show_mem(fmt, ...) \
+ ({ show_mem(0, NULL); panic(fmt, ##__VA_ARGS__); })
/* link hash */
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 2a8b8c3..4a865f0 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -998,7 +998,7 @@ static void __io_req_complete_post(struct io_kiocb *req)
void io_req_complete_post(struct io_kiocb *req, unsigned issue_flags)
{
- if (req->ctx->task_complete && (issue_flags & IO_URING_F_IOWQ)) {
+ if (req->ctx->task_complete && req->ctx->submitter_task != current) {
req->io_task_work.func = io_req_task_complete;
io_req_task_work_add(req);
} else if (!(issue_flags & IO_URING_F_UNLOCKED) ||
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index fc7281d..0d73139 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3276,6 +3276,21 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx,
}
} else if (opcode == BPF_EXIT) {
return -ENOTSUPP;
+ } else if (BPF_SRC(insn->code) == BPF_X) {
+ if (!(*reg_mask & (dreg | sreg)))
+ return 0;
+ /* dreg <cond> sreg
+ * Both dreg and sreg need precision before
+ * this insn. If only sreg was marked precise
+ * before it would be equally necessary to
+ * propagate it to dreg.
+ */
+ *reg_mask |= (sreg | dreg);
+ /* else dreg <cond> K
+ * Only dreg still needs precision before
+ * this insn, so for the K-based conditional
+ * there is nothing new to be marked.
+ */
}
} else if (class == BPF_LD) {
if (!(*reg_mask & dreg))
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 636f1c6..505d86b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1513,7 +1513,7 @@ static int update_parent_subparts_cpumask(struct cpuset *cs, int cmd,
spin_unlock_irq(&callback_lock);
if (adding || deleting)
- update_tasks_cpumask(parent, tmp->new_cpus);
+ update_tasks_cpumask(parent, tmp->addmask);
/*
* Set or clear CS_SCHED_LOAD_BALANCE when partcmd_update, if necessary.
@@ -1770,10 +1770,13 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
/*
* Use the cpumasks in trialcs for tmpmasks when they are pointers
* to allocated cpumasks.
+ *
+ * Note that update_parent_subparts_cpumask() uses only addmask &
+ * delmask, but not new_cpus.
*/
tmp.addmask = trialcs->subparts_cpus;
tmp.delmask = trialcs->effective_cpus;
- tmp.new_cpus = trialcs->cpus_allowed;
+ tmp.new_cpus = NULL;
#endif
retval = validate_change(cs, trialcs);
@@ -1838,6 +1841,11 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
}
spin_unlock_irq(&callback_lock);
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ /* Now trialcs->cpus_allowed is available */
+ tmp.new_cpus = trialcs->cpus_allowed;
+#endif
+
/* effective_cpus will be updated here */
update_cpumasks_hier(cs, &tmp, false);
@@ -2445,6 +2453,20 @@ static int fmeter_getrate(struct fmeter *fmp)
static struct cpuset *cpuset_attach_old_cs;
+/*
+ * Check to see if a cpuset can accept a new task
+ * For v1, cpus_allowed and mems_allowed can't be empty.
+ * For v2, effective_cpus can't be empty.
+ * Note that in v1, effective_cpus = cpus_allowed.
+ */
+static int cpuset_can_attach_check(struct cpuset *cs)
+{
+ if (cpumask_empty(cs->effective_cpus) ||
+ (!is_in_v2_mode() && nodes_empty(cs->mems_allowed)))
+ return -ENOSPC;
+ return 0;
+}
+
/* Called by cgroups to determine if a cpuset is usable; cpuset_rwsem held */
static int cpuset_can_attach(struct cgroup_taskset *tset)
{
@@ -2459,16 +2481,9 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
percpu_down_write(&cpuset_rwsem);
- /* allow moving tasks into an empty cpuset if on default hierarchy */
- ret = -ENOSPC;
- if (!is_in_v2_mode() &&
- (cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed)))
- goto out_unlock;
-
- /*
- * Task cannot be moved to a cpuset with empty effective cpus.
- */
- if (cpumask_empty(cs->effective_cpus))
+ /* Check to see if task is allowed in the cpuset */
+ ret = cpuset_can_attach_check(cs);
+ if (ret)
goto out_unlock;
cgroup_taskset_for_each(task, css, tset) {
@@ -2485,7 +2500,6 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
* changes which zero cpus/mems_allowed.
*/
cs->attach_in_progress++;
- ret = 0;
out_unlock:
percpu_up_write(&cpuset_rwsem);
return ret;
@@ -2494,25 +2508,47 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
static void cpuset_cancel_attach(struct cgroup_taskset *tset)
{
struct cgroup_subsys_state *css;
+ struct cpuset *cs;
cgroup_taskset_first(tset, &css);
+ cs = css_cs(css);
percpu_down_write(&cpuset_rwsem);
- css_cs(css)->attach_in_progress--;
+ cs->attach_in_progress--;
+ if (!cs->attach_in_progress)
+ wake_up(&cpuset_attach_wq);
percpu_up_write(&cpuset_rwsem);
}
/*
- * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach()
+ * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach_task()
* but we can't allocate it dynamically there. Define it global and
* allocate from cpuset_init().
*/
static cpumask_var_t cpus_attach;
+static nodemask_t cpuset_attach_nodemask_to;
+
+static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task)
+{
+ percpu_rwsem_assert_held(&cpuset_rwsem);
+
+ if (cs != &top_cpuset)
+ guarantee_online_cpus(task, cpus_attach);
+ else
+ cpumask_andnot(cpus_attach, task_cpu_possible_mask(task),
+ cs->subparts_cpus);
+ /*
+ * can_attach beforehand should guarantee that this doesn't
+ * fail. TODO: have a better way to handle failure here
+ */
+ WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
+
+ cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to);
+ cpuset_update_task_spread_flags(cs, task);
+}
static void cpuset_attach(struct cgroup_taskset *tset)
{
- /* static buf protected by cpuset_rwsem */
- static nodemask_t cpuset_attach_nodemask_to;
struct task_struct *task;
struct task_struct *leader;
struct cgroup_subsys_state *css;
@@ -2543,20 +2579,8 @@ static void cpuset_attach(struct cgroup_taskset *tset)
guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
- cgroup_taskset_for_each(task, css, tset) {
- if (cs != &top_cpuset)
- guarantee_online_cpus(task, cpus_attach);
- else
- cpumask_copy(cpus_attach, task_cpu_possible_mask(task));
- /*
- * can_attach beforehand should guarantee that this doesn't
- * fail. TODO: have a better way to handle failure here
- */
- WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
-
- cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to);
- cpuset_update_task_spread_flags(cs, task);
- }
+ cgroup_taskset_for_each(task, css, tset)
+ cpuset_attach_task(cs, task);
/*
* Change mm for all threadgroup leaders. This is expensive and may
@@ -3248,17 +3272,101 @@ static void cpuset_bind(struct cgroup_subsys_state *root_css)
}
/*
+ * In case the child is cloned into a cpuset different from its parent,
+ * additional checks are done to see if the move is allowed.
+ */
+static int cpuset_can_fork(struct task_struct *task, struct css_set *cset)
+{
+ struct cpuset *cs = css_cs(cset->subsys[cpuset_cgrp_id]);
+ bool same_cs;
+ int ret;
+
+ rcu_read_lock();
+ same_cs = (cs == task_cs(current));
+ rcu_read_unlock();
+
+ if (same_cs)
+ return 0;
+
+ lockdep_assert_held(&cgroup_mutex);
+ percpu_down_write(&cpuset_rwsem);
+
+ /* Check to see if task is allowed in the cpuset */
+ ret = cpuset_can_attach_check(cs);
+ if (ret)
+ goto out_unlock;
+
+ ret = task_can_attach(task, cs->effective_cpus);
+ if (ret)
+ goto out_unlock;
+
+ ret = security_task_setscheduler(task);
+ if (ret)
+ goto out_unlock;
+
+ /*
+ * Mark attach is in progress. This makes validate_change() fail
+ * changes which zero cpus/mems_allowed.
+ */
+ cs->attach_in_progress++;
+out_unlock:
+ percpu_up_write(&cpuset_rwsem);
+ return ret;
+}
+
+static void cpuset_cancel_fork(struct task_struct *task, struct css_set *cset)
+{
+ struct cpuset *cs = css_cs(cset->subsys[cpuset_cgrp_id]);
+ bool same_cs;
+
+ rcu_read_lock();
+ same_cs = (cs == task_cs(current));
+ rcu_read_unlock();
+
+ if (same_cs)
+ return;
+
+ percpu_down_write(&cpuset_rwsem);
+ cs->attach_in_progress--;
+ if (!cs->attach_in_progress)
+ wake_up(&cpuset_attach_wq);
+ percpu_up_write(&cpuset_rwsem);
+}
+
+/*
* Make sure the new task conform to the current state of its parent,
* which could have been changed by cpuset just after it inherits the
* state from the parent and before it sits on the cgroup's task list.
*/
static void cpuset_fork(struct task_struct *task)
{
- if (task_css_is_root(task, cpuset_cgrp_id))
- return;
+ struct cpuset *cs;
+ bool same_cs;
- set_cpus_allowed_ptr(task, current->cpus_ptr);
- task->mems_allowed = current->mems_allowed;
+ rcu_read_lock();
+ cs = task_cs(task);
+ same_cs = (cs == task_cs(current));
+ rcu_read_unlock();
+
+ if (same_cs) {
+ if (cs == &top_cpuset)
+ return;
+
+ set_cpus_allowed_ptr(task, current->cpus_ptr);
+ task->mems_allowed = current->mems_allowed;
+ return;
+ }
+
+ /* CLONE_INTO_CGROUP */
+ percpu_down_write(&cpuset_rwsem);
+ guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
+ cpuset_attach_task(cs, task);
+
+ cs->attach_in_progress--;
+ if (!cs->attach_in_progress)
+ wake_up(&cpuset_attach_wq);
+
+ percpu_up_write(&cpuset_rwsem);
}
struct cgroup_subsys cpuset_cgrp_subsys = {
@@ -3271,6 +3379,8 @@ struct cgroup_subsys cpuset_cgrp_subsys = {
.attach = cpuset_attach,
.post_attach = cpuset_post_attach,
.bind = cpuset_bind,
+ .can_fork = cpuset_can_fork,
+ .cancel_fork = cpuset_cancel_fork,
.fork = cpuset_fork,
.legacy_cftypes = legacy_files,
.dfl_cftypes = dfl_files,
diff --git a/kernel/cgroup/legacy_freezer.c b/kernel/cgroup/legacy_freezer.c
index 1b6b218..9364732 100644
--- a/kernel/cgroup/legacy_freezer.c
+++ b/kernel/cgroup/legacy_freezer.c
@@ -22,6 +22,7 @@
#include <linux/freezer.h>
#include <linux/seq_file.h>
#include <linux/mutex.h>
+#include <linux/cpu.h>
/*
* A cgroup is freezing if any FREEZING flags are set. FREEZING_SELF is
@@ -350,7 +351,7 @@ static void freezer_apply_state(struct freezer *freezer, bool freeze,
if (freeze) {
if (!(freezer->state & CGROUP_FREEZING))
- static_branch_inc(&freezer_active);
+ static_branch_inc_cpuslocked(&freezer_active);
freezer->state |= state;
freeze_cgroup(freezer);
} else {
@@ -361,7 +362,7 @@ static void freezer_apply_state(struct freezer *freezer, bool freeze,
if (!(freezer->state & CGROUP_FREEZING)) {
freezer->state &= ~CGROUP_FROZEN;
if (was_freezing)
- static_branch_dec(&freezer_active);
+ static_branch_dec_cpuslocked(&freezer_active);
unfreeze_cgroup(freezer);
}
}
@@ -379,6 +380,7 @@ static void freezer_change_state(struct freezer *freezer, bool freeze)
{
struct cgroup_subsys_state *pos;
+ cpus_read_lock();
/*
* Update all its descendants in pre-order traversal. Each
* descendant will try to inherit its parent's FREEZING state as
@@ -407,6 +409,7 @@ static void freezer_change_state(struct freezer *freezer, bool freeze)
}
rcu_read_unlock();
mutex_unlock(&freezer_mutex);
+ cpus_read_unlock();
}
static ssize_t freezer_write(struct kernfs_open_file *of,
diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c
index 831f1f4..0a2b496 100644
--- a/kernel/cgroup/rstat.c
+++ b/kernel/cgroup/rstat.c
@@ -457,9 +457,7 @@ static void root_cgroup_cputime(struct cgroup_base_stat *bstat)
struct task_cputime *cputime = &bstat->cputime;
int i;
- cputime->stime = 0;
- cputime->utime = 0;
- cputime->sum_exec_runtime = 0;
+ memset(bstat, 0, sizeof(*bstat));
for_each_possible_cpu(i) {
struct kernel_cpustat kcpustat;
u64 *cpustat = kcpustat.cpustat;
diff --git a/kernel/fork.c b/kernel/fork.c
index 0c92f22..ea33231 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1174,6 +1174,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p,
fail_pcpu:
while (i > 0)
percpu_counter_destroy(&mm->rss_stat[--i]);
+ destroy_context(mm);
fail_nocontext:
mm_free_pgd(mm);
fail_nopgd:
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 6986ea3..5f6587d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10238,6 +10238,16 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s
sds->avg_load = (sds->total_load * SCHED_CAPACITY_SCALE) /
sds->total_capacity;
+
+ /*
+ * If the local group is more loaded than the average system
+ * load, don't try to pull any tasks.
+ */
+ if (local->avg_load >= sds->avg_load) {
+ env->imbalance = 0;
+ return;
+ }
+
}
/*
diff --git a/kernel/sys.c b/kernel/sys.c
index 495cd87..351de79 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -664,6 +664,7 @@ long __sys_setresuid(uid_t ruid, uid_t euid, uid_t suid)
struct cred *new;
int retval;
kuid_t kruid, keuid, ksuid;
+ bool ruid_new, euid_new, suid_new;
kruid = make_kuid(ns, ruid);
keuid = make_kuid(ns, euid);
@@ -678,25 +679,29 @@ long __sys_setresuid(uid_t ruid, uid_t euid, uid_t suid)
if ((suid != (uid_t) -1) && !uid_valid(ksuid))
return -EINVAL;
+ old = current_cred();
+
+ /* check for no-op */
+ if ((ruid == (uid_t) -1 || uid_eq(kruid, old->uid)) &&
+ (euid == (uid_t) -1 || (uid_eq(keuid, old->euid) &&
+ uid_eq(keuid, old->fsuid))) &&
+ (suid == (uid_t) -1 || uid_eq(ksuid, old->suid)))
+ return 0;
+
+ ruid_new = ruid != (uid_t) -1 && !uid_eq(kruid, old->uid) &&
+ !uid_eq(kruid, old->euid) && !uid_eq(kruid, old->suid);
+ euid_new = euid != (uid_t) -1 && !uid_eq(keuid, old->uid) &&
+ !uid_eq(keuid, old->euid) && !uid_eq(keuid, old->suid);
+ suid_new = suid != (uid_t) -1 && !uid_eq(ksuid, old->uid) &&
+ !uid_eq(ksuid, old->euid) && !uid_eq(ksuid, old->suid);
+ if ((ruid_new || euid_new || suid_new) &&
+ !ns_capable_setid(old->user_ns, CAP_SETUID))
+ return -EPERM;
+
new = prepare_creds();
if (!new)
return -ENOMEM;
- old = current_cred();
-
- retval = -EPERM;
- if (!ns_capable_setid(old->user_ns, CAP_SETUID)) {
- if (ruid != (uid_t) -1 && !uid_eq(kruid, old->uid) &&
- !uid_eq(kruid, old->euid) && !uid_eq(kruid, old->suid))
- goto error;
- if (euid != (uid_t) -1 && !uid_eq(keuid, old->uid) &&
- !uid_eq(keuid, old->euid) && !uid_eq(keuid, old->suid))
- goto error;
- if (suid != (uid_t) -1 && !uid_eq(ksuid, old->uid) &&
- !uid_eq(ksuid, old->euid) && !uid_eq(ksuid, old->suid))
- goto error;
- }
-
if (ruid != (uid_t) -1) {
new->uid = kruid;
if (!uid_eq(kruid, old->uid)) {
@@ -761,6 +766,7 @@ long __sys_setresgid(gid_t rgid, gid_t egid, gid_t sgid)
struct cred *new;
int retval;
kgid_t krgid, kegid, ksgid;
+ bool rgid_new, egid_new, sgid_new;
krgid = make_kgid(ns, rgid);
kegid = make_kgid(ns, egid);
@@ -773,23 +779,28 @@ long __sys_setresgid(gid_t rgid, gid_t egid, gid_t sgid)
if ((sgid != (gid_t) -1) && !gid_valid(ksgid))
return -EINVAL;
+ old = current_cred();
+
+ /* check for no-op */
+ if ((rgid == (gid_t) -1 || gid_eq(krgid, old->gid)) &&
+ (egid == (gid_t) -1 || (gid_eq(kegid, old->egid) &&
+ gid_eq(kegid, old->fsgid))) &&
+ (sgid == (gid_t) -1 || gid_eq(ksgid, old->sgid)))
+ return 0;
+
+ rgid_new = rgid != (gid_t) -1 && !gid_eq(krgid, old->gid) &&
+ !gid_eq(krgid, old->egid) && !gid_eq(krgid, old->sgid);
+ egid_new = egid != (gid_t) -1 && !gid_eq(kegid, old->gid) &&
+ !gid_eq(kegid, old->egid) && !gid_eq(kegid, old->sgid);
+ sgid_new = sgid != (gid_t) -1 && !gid_eq(ksgid, old->gid) &&
+ !gid_eq(ksgid, old->egid) && !gid_eq(ksgid, old->sgid);
+ if ((rgid_new || egid_new || sgid_new) &&
+ !ns_capable_setid(old->user_ns, CAP_SETGID))
+ return -EPERM;
+
new = prepare_creds();
if (!new)
return -ENOMEM;
- old = current_cred();
-
- retval = -EPERM;
- if (!ns_capable_setid(old->user_ns, CAP_SETGID)) {
- if (rgid != (gid_t) -1 && !gid_eq(krgid, old->gid) &&
- !gid_eq(krgid, old->egid) && !gid_eq(krgid, old->sgid))
- goto error;
- if (egid != (gid_t) -1 && !gid_eq(kegid, old->gid) &&
- !gid_eq(kegid, old->egid) && !gid_eq(kegid, old->sgid))
- goto error;
- if (sgid != (gid_t) -1 && !gid_eq(ksgid, old->gid) &&
- !gid_eq(ksgid, old->egid) && !gid_eq(ksgid, old->sgid))
- goto error;
- }
if (rgid != (gid_t) -1)
new->gid = krgid;
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index db60edb..1281a40 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -1303,26 +1303,21 @@ static inline void mas_alloc_nodes(struct ma_state *mas, gfp_t gfp)
node = mas->alloc;
node->request_count = 0;
while (requested) {
- max_req = MAPLE_ALLOC_SLOTS;
- if (node->node_count) {
- unsigned int offset = node->node_count;
-
- slots = (void **)&node->slot[offset];
- max_req -= offset;
- } else {
- slots = (void **)&node->slot;
- }
-
+ max_req = MAPLE_ALLOC_SLOTS - node->node_count;
+ slots = (void **)&node->slot[node->node_count];
max_req = min(requested, max_req);
count = mt_alloc_bulk(gfp, max_req, slots);
if (!count)
goto nomem_bulk;
+ if (node->node_count == 0) {
+ node->slot[0]->node_count = 0;
+ node->slot[0]->request_count = 0;
+ }
+
node->node_count += count;
allocated += count;
node = node->slot[0];
- node->node_count = 0;
- node->request_count = 0;
requested -= count;
}
mas->alloc->total = allocated;
@@ -4970,7 +4965,8 @@ static inline void *mas_prev_entry(struct ma_state *mas, unsigned long min)
* Return: True if found in a leaf, false otherwise.
*
*/
-static bool mas_rev_awalk(struct ma_state *mas, unsigned long size)
+static bool mas_rev_awalk(struct ma_state *mas, unsigned long size,
+ unsigned long *gap_min, unsigned long *gap_max)
{
enum maple_type type = mte_node_type(mas->node);
struct maple_node *node = mas_mn(mas);
@@ -5035,8 +5031,8 @@ static bool mas_rev_awalk(struct ma_state *mas, unsigned long size)
if (unlikely(ma_is_leaf(type))) {
mas->offset = offset;
- mas->min = min;
- mas->max = min + gap - 1;
+ *gap_min = min;
+ *gap_max = min + gap - 1;
return true;
}
@@ -5060,10 +5056,10 @@ static inline bool mas_anode_descend(struct ma_state *mas, unsigned long size)
{
enum maple_type type = mte_node_type(mas->node);
unsigned long pivot, min, gap = 0;
- unsigned char offset;
- unsigned long *gaps;
- unsigned long *pivots = ma_pivots(mas_mn(mas), type);
- void __rcu **slots = ma_slots(mas_mn(mas), type);
+ unsigned char offset, data_end;
+ unsigned long *gaps, *pivots;
+ void __rcu **slots;
+ struct maple_node *node;
bool found = false;
if (ma_is_dense(type)) {
@@ -5071,13 +5067,15 @@ static inline bool mas_anode_descend(struct ma_state *mas, unsigned long size)
return true;
}
- gaps = ma_gaps(mte_to_node(mas->node), type);
+ node = mas_mn(mas);
+ pivots = ma_pivots(node, type);
+ slots = ma_slots(node, type);
+ gaps = ma_gaps(node, type);
offset = mas->offset;
min = mas_safe_min(mas, pivots, offset);
- for (; offset < mt_slots[type]; offset++) {
- pivot = mas_safe_pivot(mas, pivots, offset, type);
- if (offset && !pivot)
- break;
+ data_end = ma_data_end(node, type, pivots, mas->max);
+ for (; offset <= data_end; offset++) {
+ pivot = mas_logical_pivot(mas, pivots, offset, type);
/* Not within lower bounds */
if (mas->index > pivot)
@@ -5312,6 +5310,9 @@ int mas_empty_area(struct ma_state *mas, unsigned long min,
unsigned long *pivots;
enum maple_type mt;
+ if (min >= max)
+ return -EINVAL;
+
if (mas_is_start(mas))
mas_start(mas);
else if (mas->offset >= 2)
@@ -5366,6 +5367,9 @@ int mas_empty_area_rev(struct ma_state *mas, unsigned long min,
{
struct maple_enode *last = mas->node;
+ if (min >= max)
+ return -EINVAL;
+
if (mas_is_start(mas)) {
mas_start(mas);
mas->offset = mas_data_end(mas);
@@ -5385,7 +5389,7 @@ int mas_empty_area_rev(struct ma_state *mas, unsigned long min,
mas->index = min;
mas->last = max;
- while (!mas_rev_awalk(mas, size)) {
+ while (!mas_rev_awalk(mas, size, &min, &max)) {
if (last == mas->node) {
if (!mas_rewind_node(mas))
return -EBUSY;
@@ -5400,17 +5404,9 @@ int mas_empty_area_rev(struct ma_state *mas, unsigned long min,
if (unlikely(mas->offset == MAPLE_NODE_SLOTS))
return -EBUSY;
- /*
- * mas_rev_awalk() has set mas->min and mas->max to the gap values. If
- * the maximum is outside the window we are searching, then use the last
- * location in the search.
- * mas->max and mas->min is the range of the gap.
- * mas->index and mas->last are currently set to the search range.
- */
-
/* Trim the upper limit to the max. */
- if (mas->max <= mas->last)
- mas->last = mas->max;
+ if (max <= mas->last)
+ mas->last = max;
mas->index = mas->last - size + 1;
return 0;
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index a53b936..30d2d03 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -507,6 +507,15 @@ static LIST_HEAD(offline_cgwbs);
static void cleanup_offline_cgwbs_workfn(struct work_struct *work);
static DECLARE_WORK(cleanup_offline_cgwbs_work, cleanup_offline_cgwbs_workfn);
+static void cgwb_free_rcu(struct rcu_head *rcu_head)
+{
+ struct bdi_writeback *wb = container_of(rcu_head,
+ struct bdi_writeback, rcu);
+
+ percpu_ref_exit(&wb->refcnt);
+ kfree(wb);
+}
+
static void cgwb_release_workfn(struct work_struct *work)
{
struct bdi_writeback *wb = container_of(work, struct bdi_writeback,
@@ -529,11 +538,10 @@ static void cgwb_release_workfn(struct work_struct *work)
list_del(&wb->offline_node);
spin_unlock_irq(&cgwb_lock);
- percpu_ref_exit(&wb->refcnt);
wb_exit(wb);
bdi_put(bdi);
WARN_ON_ONCE(!list_empty(&wb->b_attached));
- kfree_rcu(wb, rcu);
+ call_rcu(&wb->rcu, cgwb_free_rcu);
}
static void cgwb_release(struct percpu_ref *refcnt)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 032fb0e..3fae2d2 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1838,10 +1838,10 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (is_swap_pmd(*pmd)) {
swp_entry_t entry = pmd_to_swp_entry(*pmd);
struct page *page = pfn_swap_entry_to_page(entry);
+ pmd_t newpmd;
VM_BUG_ON(!is_pmd_migration_entry(*pmd));
if (is_writable_migration_entry(entry)) {
- pmd_t newpmd;
/*
* A protection check is difficult so
* just be safe and disable write
@@ -1855,8 +1855,16 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
newpmd = pmd_swp_mksoft_dirty(newpmd);
if (pmd_swp_uffd_wp(*pmd))
newpmd = pmd_swp_mkuffd_wp(newpmd);
- set_pmd_at(mm, addr, pmd, newpmd);
+ } else {
+ newpmd = *pmd;
}
+
+ if (uffd_wp)
+ newpmd = pmd_swp_mkuffd_wp(newpmd);
+ else if (uffd_wp_resolve)
+ newpmd = pmd_swp_clear_uffd_wp(newpmd);
+ if (!pmd_same(*pmd, newpmd))
+ set_pmd_at(mm, addr, pmd, newpmd);
goto unlock;
}
#endif
@@ -2657,9 +2665,10 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
is_hzp = is_huge_zero_page(&folio->page);
- VM_WARN_ON_ONCE_FOLIO(is_hzp, folio);
- if (is_hzp)
+ if (is_hzp) {
+ pr_warn_ratelimited("Called split_huge_page for huge zero page\n");
return -EBUSY;
+ }
if (folio_test_writeback(folio))
return -EBUSY;
@@ -3251,6 +3260,8 @@ int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
pmdswp = swp_entry_to_pmd(entry);
if (pmd_soft_dirty(pmdval))
pmdswp = pmd_swp_mksoft_dirty(pmdswp);
+ if (pmd_uffd_wp(pmdval))
+ pmdswp = pmd_swp_mkuffd_wp(pmdswp);
set_pmd_at(mm, address, pvmw->pmd, pmdswp);
page_remove_rmap(page, vma, true);
put_page(page);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 92e6f56..0ec69b9 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -572,6 +572,10 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma,
result = SCAN_PTE_NON_PRESENT;
goto out;
}
+ if (pte_uffd_wp(pteval)) {
+ result = SCAN_PTE_UFFD_WP;
+ goto out;
+ }
page = vm_normal_page(vma, address, pteval);
if (unlikely(!page) || unlikely(is_zone_device_page(page))) {
result = SCAN_PAGE_NULL;
diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
index 3807502..ec0da72 100644
--- a/mm/kmsan/hooks.c
+++ b/mm/kmsan/hooks.c
@@ -148,35 +148,74 @@ void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
* into the virtual memory. If those physical pages already had shadow/origin,
* those are ignored.
*/
-void kmsan_ioremap_page_range(unsigned long start, unsigned long end,
- phys_addr_t phys_addr, pgprot_t prot,
- unsigned int page_shift)
+int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
+ phys_addr_t phys_addr, pgprot_t prot,
+ unsigned int page_shift)
{
gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO;
struct page *shadow, *origin;
unsigned long off = 0;
- int nr;
+ int nr, err = 0, clean = 0, mapped;
if (!kmsan_enabled || kmsan_in_runtime())
- return;
+ return 0;
nr = (end - start) / PAGE_SIZE;
kmsan_enter_runtime();
- for (int i = 0; i < nr; i++, off += PAGE_SIZE) {
+ for (int i = 0; i < nr; i++, off += PAGE_SIZE, clean = i) {
shadow = alloc_pages(gfp_mask, 1);
origin = alloc_pages(gfp_mask, 1);
- __vmap_pages_range_noflush(
+ if (!shadow || !origin) {
+ err = -ENOMEM;
+ goto ret;
+ }
+ mapped = __vmap_pages_range_noflush(
vmalloc_shadow(start + off),
vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow,
PAGE_SHIFT);
- __vmap_pages_range_noflush(
+ if (mapped) {
+ err = mapped;
+ goto ret;
+ }
+ shadow = NULL;
+ mapped = __vmap_pages_range_noflush(
vmalloc_origin(start + off),
vmalloc_origin(start + off + PAGE_SIZE), prot, &origin,
PAGE_SHIFT);
+ if (mapped) {
+ __vunmap_range_noflush(
+ vmalloc_shadow(start + off),
+ vmalloc_shadow(start + off + PAGE_SIZE));
+ err = mapped;
+ goto ret;
+ }
+ origin = NULL;
+ }
+ /* Page mapping loop finished normally, nothing to clean up. */
+ clean = 0;
+
+ret:
+ if (clean > 0) {
+ /*
+ * Something went wrong. Clean up shadow/origin pages allocated
+ * on the last loop iteration, then delete mappings created
+ * during the previous iterations.
+ */
+ if (shadow)
+ __free_pages(shadow, 1);
+ if (origin)
+ __free_pages(origin, 1);
+ __vunmap_range_noflush(
+ vmalloc_shadow(start),
+ vmalloc_shadow(start + clean * PAGE_SIZE));
+ __vunmap_range_noflush(
+ vmalloc_origin(start),
+ vmalloc_origin(start + clean * PAGE_SIZE));
}
flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
kmsan_leave_runtime();
+ return err;
}
void kmsan_iounmap_page_range(unsigned long start, unsigned long end)
diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
index a787c04..b8bb95e 100644
--- a/mm/kmsan/shadow.c
+++ b/mm/kmsan/shadow.c
@@ -216,27 +216,29 @@ void kmsan_free_page(struct page *page, unsigned int order)
kmsan_leave_runtime();
}
-void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
- pgprot_t prot, struct page **pages,
- unsigned int page_shift)
+int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages,
+ unsigned int page_shift)
{
unsigned long shadow_start, origin_start, shadow_end, origin_end;
struct page **s_pages, **o_pages;
- int nr, mapped;
+ int nr, mapped, err = 0;
if (!kmsan_enabled)
- return;
+ return 0;
shadow_start = vmalloc_meta((void *)start, KMSAN_META_SHADOW);
shadow_end = vmalloc_meta((void *)end, KMSAN_META_SHADOW);
if (!shadow_start)
- return;
+ return 0;
nr = (end - start) / PAGE_SIZE;
s_pages = kcalloc(nr, sizeof(*s_pages), GFP_KERNEL);
o_pages = kcalloc(nr, sizeof(*o_pages), GFP_KERNEL);
- if (!s_pages || !o_pages)
+ if (!s_pages || !o_pages) {
+ err = -ENOMEM;
goto ret;
+ }
for (int i = 0; i < nr; i++) {
s_pages[i] = shadow_page_for(pages[i]);
o_pages[i] = origin_page_for(pages[i]);
@@ -249,10 +251,16 @@ void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
kmsan_enter_runtime();
mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot,
s_pages, page_shift);
- KMSAN_WARN_ON(mapped);
+ if (mapped) {
+ err = mapped;
+ goto ret;
+ }
mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot,
o_pages, page_shift);
- KMSAN_WARN_ON(mapped);
+ if (mapped) {
+ err = mapped;
+ goto ret;
+ }
kmsan_leave_runtime();
flush_tlb_kernel_range(shadow_start, shadow_end);
flush_tlb_kernel_range(origin_start, origin_end);
@@ -262,6 +270,7 @@ void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
ret:
kfree(s_pages);
kfree(o_pages);
+ return err;
}
/* Allocate metadata for pages allocated at boot time. */
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index a256a24..2068b59 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -790,61 +790,50 @@ static int vma_replace_policy(struct vm_area_struct *vma,
return err;
}
-/* Step 2: apply policy to a range and do splits. */
-static int mbind_range(struct mm_struct *mm, unsigned long start,
- unsigned long end, struct mempolicy *new_pol)
+/* Split or merge the VMA (if required) and apply the new policy */
+static int mbind_range(struct vma_iterator *vmi, struct vm_area_struct *vma,
+ struct vm_area_struct **prev, unsigned long start,
+ unsigned long end, struct mempolicy *new_pol)
{
- VMA_ITERATOR(vmi, mm, start);
- struct vm_area_struct *prev;
- struct vm_area_struct *vma;
- int err = 0;
+ struct vm_area_struct *merged;
+ unsigned long vmstart, vmend;
pgoff_t pgoff;
+ int err;
- prev = vma_prev(&vmi);
- vma = vma_find(&vmi, end);
- if (WARN_ON(!vma))
+ vmend = min(end, vma->vm_end);
+ if (start > vma->vm_start) {
+ *prev = vma;
+ vmstart = start;
+ } else {
+ vmstart = vma->vm_start;
+ }
+
+ if (mpol_equal(vma_policy(vma), new_pol))
return 0;
- if (start > vma->vm_start)
- prev = vma;
+ pgoff = vma->vm_pgoff + ((vmstart - vma->vm_start) >> PAGE_SHIFT);
+ merged = vma_merge(vmi, vma->vm_mm, *prev, vmstart, vmend, vma->vm_flags,
+ vma->anon_vma, vma->vm_file, pgoff, new_pol,
+ vma->vm_userfaultfd_ctx, anon_vma_name(vma));
+ if (merged) {
+ *prev = merged;
+ return vma_replace_policy(merged, new_pol);
+ }
- do {
- unsigned long vmstart = max(start, vma->vm_start);
- unsigned long vmend = min(end, vma->vm_end);
-
- if (mpol_equal(vma_policy(vma), new_pol))
- goto next;
-
- pgoff = vma->vm_pgoff +
- ((vmstart - vma->vm_start) >> PAGE_SHIFT);
- prev = vma_merge(&vmi, mm, prev, vmstart, vmend, vma->vm_flags,
- vma->anon_vma, vma->vm_file, pgoff,
- new_pol, vma->vm_userfaultfd_ctx,
- anon_vma_name(vma));
- if (prev) {
- vma = prev;
- goto replace;
- }
- if (vma->vm_start != vmstart) {
- err = split_vma(&vmi, vma, vmstart, 1);
- if (err)
- goto out;
- }
- if (vma->vm_end != vmend) {
- err = split_vma(&vmi, vma, vmend, 0);
- if (err)
- goto out;
- }
-replace:
- err = vma_replace_policy(vma, new_pol);
+ if (vma->vm_start != vmstart) {
+ err = split_vma(vmi, vma, vmstart, 1);
if (err)
- goto out;
-next:
- prev = vma;
- } for_each_vma_range(vmi, vma, end);
+ return err;
+ }
-out:
- return err;
+ if (vma->vm_end != vmend) {
+ err = split_vma(vmi, vma, vmend, 0);
+ if (err)
+ return err;
+ }
+
+ *prev = vma;
+ return vma_replace_policy(vma, new_pol);
}
/* Set the process memory policy */
@@ -1259,6 +1248,8 @@ static long do_mbind(unsigned long start, unsigned long len,
nodemask_t *nmask, unsigned long flags)
{
struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma, *prev;
+ struct vma_iterator vmi;
struct mempolicy *new;
unsigned long end;
int err;
@@ -1328,7 +1319,13 @@ static long do_mbind(unsigned long start, unsigned long len,
goto up_out;
}
- err = mbind_range(mm, start, end, new);
+ vma_iter_init(&vmi, mm, start);
+ prev = vma_prev(&vmi);
+ for_each_vma_range(vmi, vma, end) {
+ err = mbind_range(&vmi, vma, &prev, start, end, new);
+ if (err)
+ break;
+ }
if (!err) {
int nr_failed = 0;
@@ -1489,10 +1486,8 @@ SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, le
unsigned long, home_node, unsigned long, flags)
{
struct mm_struct *mm = current->mm;
- struct vm_area_struct *vma;
+ struct vm_area_struct *vma, *prev;
struct mempolicy *new, *old;
- unsigned long vmstart;
- unsigned long vmend;
unsigned long end;
int err = -ENOENT;
VMA_ITERATOR(vmi, mm, start);
@@ -1521,6 +1516,7 @@ SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, le
if (end == start)
return 0;
mmap_write_lock(mm);
+ prev = vma_prev(&vmi);
for_each_vma_range(vmi, vma, end) {
/*
* If any vma in the range got policy other than MPOL_BIND
@@ -1541,9 +1537,7 @@ SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, le
}
new->home_node = home_node;
- vmstart = max(start, vma->vm_start);
- vmend = min(end, vma->vm_end);
- err = mbind_range(mm, vmstart, vmend, new);
+ err = mbind_range(&vmi, vma, &prev, start, end, new);
mpol_put(new);
if (err)
break;
diff --git a/mm/mmap.c b/mm/mmap.c
index ff68a67..d5475fb 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1518,7 +1518,8 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
*/
static unsigned long unmapped_area(struct vm_unmapped_area_info *info)
{
- unsigned long length, gap;
+ unsigned long length, gap, low_limit;
+ struct vm_area_struct *tmp;
MA_STATE(mas, ¤t->mm->mm_mt, 0, 0);
@@ -1527,12 +1528,29 @@ static unsigned long unmapped_area(struct vm_unmapped_area_info *info)
if (length < info->length)
return -ENOMEM;
- if (mas_empty_area(&mas, info->low_limit, info->high_limit - 1,
- length))
+ low_limit = info->low_limit;
+retry:
+ if (mas_empty_area(&mas, low_limit, info->high_limit - 1, length))
return -ENOMEM;
gap = mas.index;
gap += (info->align_offset - gap) & info->align_mask;
+ tmp = mas_next(&mas, ULONG_MAX);
+ if (tmp && (tmp->vm_flags & VM_GROWSDOWN)) { /* Avoid prev check if possible */
+ if (vm_start_gap(tmp) < gap + length - 1) {
+ low_limit = tmp->vm_end;
+ mas_reset(&mas);
+ goto retry;
+ }
+ } else {
+ tmp = mas_prev(&mas, 0);
+ if (tmp && vm_end_gap(tmp) > gap) {
+ low_limit = vm_end_gap(tmp);
+ mas_reset(&mas);
+ goto retry;
+ }
+ }
+
return gap;
}
@@ -1548,7 +1566,8 @@ static unsigned long unmapped_area(struct vm_unmapped_area_info *info)
*/
static unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info)
{
- unsigned long length, gap;
+ unsigned long length, gap, high_limit, gap_end;
+ struct vm_area_struct *tmp;
MA_STATE(mas, ¤t->mm->mm_mt, 0, 0);
/* Adjust search length to account for worst case alignment overhead */
@@ -1556,12 +1575,31 @@ static unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info)
if (length < info->length)
return -ENOMEM;
- if (mas_empty_area_rev(&mas, info->low_limit, info->high_limit - 1,
+ high_limit = info->high_limit;
+retry:
+ if (mas_empty_area_rev(&mas, info->low_limit, high_limit - 1,
length))
return -ENOMEM;
gap = mas.last + 1 - info->length;
gap -= (gap - info->align_offset) & info->align_mask;
+ gap_end = mas.last;
+ tmp = mas_next(&mas, ULONG_MAX);
+ if (tmp && (tmp->vm_flags & VM_GROWSDOWN)) { /* Avoid prev check if possible */
+ if (vm_start_gap(tmp) <= gap_end) {
+ high_limit = vm_start_gap(tmp);
+ mas_reset(&mas);
+ goto retry;
+ }
+ } else {
+ tmp = mas_prev(&mas, 0);
+ if (tmp && vm_end_gap(tmp) > gap) {
+ high_limit = tmp->vm_start;
+ mas_reset(&mas);
+ goto retry;
+ }
+ }
+
return gap;
}
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 13e84d8..36351a0 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -838,7 +838,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
}
tlb_finish_mmu(&tlb);
- if (vma_iter_end(&vmi) < end)
+ if (!error && vma_iter_end(&vmi) < end)
error = -ENOMEM;
out:
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7136c36..8e39705 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6632,7 +6632,21 @@ static void __build_all_zonelists(void *data)
int nid;
int __maybe_unused cpu;
pg_data_t *self = data;
+ unsigned long flags;
+ /*
+ * Explicitly disable this CPU's interrupts before taking seqlock
+ * to prevent any IRQ handler from calling into the page allocator
+ * (e.g. GFP_ATOMIC) that could hit zonelist_iter_begin and livelock.
+ */
+ local_irq_save(flags);
+ /*
+ * Explicitly disable this CPU's synchronous printk() before taking
+ * seqlock to prevent any printk() from trying to hold port->lock, for
+ * tty_insert_flip_string_and_push_buffer() on other CPU might be
+ * calling kmalloc(GFP_ATOMIC | __GFP_NOWARN) with port->lock held.
+ */
+ printk_deferred_enter();
write_seqlock(&zonelist_update_seq);
#ifdef CONFIG_NUMA
@@ -6671,6 +6685,8 @@ static void __build_all_zonelists(void *data)
}
write_sequnlock(&zonelist_update_seq);
+ printk_deferred_exit();
+ local_irq_restore(flags);
}
static noinline void __init
@@ -9450,6 +9466,9 @@ static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn,
if (PageReserved(page))
return false;
+
+ if (PageHuge(page))
+ return false;
}
return true;
}
diff --git a/mm/swap.c b/mm/swap.c
index 57cb01b..423199e 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -222,7 +222,7 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)
if (lruvec)
unlock_page_lruvec_irqrestore(lruvec, flags);
folios_put(fbatch->folios, folio_batch_count(fbatch));
- folio_batch_init(fbatch);
+ folio_batch_reinit(fbatch);
}
static void folio_batch_add_and_move(struct folio_batch *fbatch,
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index a500720..31ff782 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -313,8 +313,8 @@ int ioremap_page_range(unsigned long addr, unsigned long end,
ioremap_max_page_shift);
flush_cache_vmap(addr, end);
if (!err)
- kmsan_ioremap_page_range(addr, end, phys_addr, prot,
- ioremap_max_page_shift);
+ err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
+ ioremap_max_page_shift);
return err;
}
@@ -605,7 +605,11 @@ int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
pgprot_t prot, struct page **pages, unsigned int page_shift)
{
- kmsan_vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
+ int ret = kmsan_vmap_pages_range_noflush(addr, end, prot, pages,
+ page_shift);
+
+ if (ret)
+ return ret;
return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
}
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index 5920544..870e493 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -26,6 +26,7 @@
#include <linux/ethtool.h>
#include <linux/phy.h>
#include <net/arp.h>
+#include <net/macsec.h>
#include "vlan.h"
#include "vlanproc.h"
@@ -572,6 +573,9 @@ static int vlan_dev_init(struct net_device *dev)
NETIF_F_HIGHDMA | NETIF_F_SCTP_CRC |
NETIF_F_ALL_FCOE;
+ if (real_dev->vlan_features & NETIF_F_HW_MACSEC)
+ dev->hw_features |= NETIF_F_HW_MACSEC;
+
dev->features |= dev->hw_features | NETIF_F_LLTX;
netif_inherit_tso_max(dev, real_dev);
if (dev->features & NETIF_F_VLAN_FEATURES)
@@ -803,6 +807,241 @@ static int vlan_dev_fill_forward_path(struct net_device_path_ctx *ctx,
return 0;
}
+#if IS_ENABLED(CONFIG_MACSEC)
+
+static const struct macsec_ops *vlan_get_macsec_ops(const struct macsec_context *ctx)
+{
+ return vlan_dev_priv(ctx->netdev)->real_dev->macsec_ops;
+}
+
+static int vlan_macsec_offload(int (* const func)(struct macsec_context *),
+ struct macsec_context *ctx)
+{
+ if (unlikely(!func))
+ return 0;
+
+ return (*func)(ctx);
+}
+
+static int vlan_macsec_dev_open(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_dev_open, ctx);
+}
+
+static int vlan_macsec_dev_stop(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_dev_stop, ctx);
+}
+
+static int vlan_macsec_add_secy(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_add_secy, ctx);
+}
+
+static int vlan_macsec_upd_secy(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_upd_secy, ctx);
+}
+
+static int vlan_macsec_del_secy(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_del_secy, ctx);
+}
+
+static int vlan_macsec_add_rxsc(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_add_rxsc, ctx);
+}
+
+static int vlan_macsec_upd_rxsc(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_upd_rxsc, ctx);
+}
+
+static int vlan_macsec_del_rxsc(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_del_rxsc, ctx);
+}
+
+static int vlan_macsec_add_rxsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_add_rxsa, ctx);
+}
+
+static int vlan_macsec_upd_rxsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_upd_rxsa, ctx);
+}
+
+static int vlan_macsec_del_rxsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_del_rxsa, ctx);
+}
+
+static int vlan_macsec_add_txsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_add_txsa, ctx);
+}
+
+static int vlan_macsec_upd_txsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_upd_txsa, ctx);
+}
+
+static int vlan_macsec_del_txsa(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_del_txsa, ctx);
+}
+
+static int vlan_macsec_get_dev_stats(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_get_dev_stats, ctx);
+}
+
+static int vlan_macsec_get_tx_sc_stats(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_get_tx_sc_stats, ctx);
+}
+
+static int vlan_macsec_get_tx_sa_stats(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_get_tx_sa_stats, ctx);
+}
+
+static int vlan_macsec_get_rx_sc_stats(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_get_rx_sc_stats, ctx);
+}
+
+static int vlan_macsec_get_rx_sa_stats(struct macsec_context *ctx)
+{
+ const struct macsec_ops *ops = vlan_get_macsec_ops(ctx);
+
+ if (!ops)
+ return -EOPNOTSUPP;
+
+ return vlan_macsec_offload(ops->mdo_get_rx_sa_stats, ctx);
+}
+
+static const struct macsec_ops macsec_offload_ops = {
+ /* Device wide */
+ .mdo_dev_open = vlan_macsec_dev_open,
+ .mdo_dev_stop = vlan_macsec_dev_stop,
+ /* SecY */
+ .mdo_add_secy = vlan_macsec_add_secy,
+ .mdo_upd_secy = vlan_macsec_upd_secy,
+ .mdo_del_secy = vlan_macsec_del_secy,
+ /* Security channels */
+ .mdo_add_rxsc = vlan_macsec_add_rxsc,
+ .mdo_upd_rxsc = vlan_macsec_upd_rxsc,
+ .mdo_del_rxsc = vlan_macsec_del_rxsc,
+ /* Security associations */
+ .mdo_add_rxsa = vlan_macsec_add_rxsa,
+ .mdo_upd_rxsa = vlan_macsec_upd_rxsa,
+ .mdo_del_rxsa = vlan_macsec_del_rxsa,
+ .mdo_add_txsa = vlan_macsec_add_txsa,
+ .mdo_upd_txsa = vlan_macsec_upd_txsa,
+ .mdo_del_txsa = vlan_macsec_del_txsa,
+ /* Statistics */
+ .mdo_get_dev_stats = vlan_macsec_get_dev_stats,
+ .mdo_get_tx_sc_stats = vlan_macsec_get_tx_sc_stats,
+ .mdo_get_tx_sa_stats = vlan_macsec_get_tx_sa_stats,
+ .mdo_get_rx_sc_stats = vlan_macsec_get_rx_sc_stats,
+ .mdo_get_rx_sa_stats = vlan_macsec_get_rx_sa_stats,
+};
+
+#endif
+
static const struct ethtool_ops vlan_ethtool_ops = {
.get_link_ksettings = vlan_ethtool_get_link_ksettings,
.get_drvinfo = vlan_ethtool_get_drvinfo,
@@ -869,6 +1108,9 @@ void vlan_setup(struct net_device *dev)
dev->priv_destructor = vlan_dev_free;
dev->ethtool_ops = &vlan_ethtool_ops;
+#if IS_ENABLED(CONFIG_MACSEC)
+ dev->macsec_ops = &macsec_offload_ops;
+#endif
dev->min_mtu = 0;
dev->max_mtu = ETH_MAX_MTU;
diff --git a/net/Kconfig b/net/Kconfig
index f806722..7d39c177 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -68,6 +68,26 @@
source "net/smc/Kconfig"
source "net/xdp/Kconfig"
+config NET_HANDSHAKE
+ bool
+ depends on SUNRPC || NVME_TARGET_TCP || NVME_TCP
+ default y
+
+config NET_HANDSHAKE_KUNIT_TEST
+ tristate "KUnit tests for the handshake upcall mechanism" if !KUNIT_ALL_TESTS
+ default KUNIT_ALL_TESTS
+ depends on KUNIT
+ help
+ This builds the KUnit tests for the handshake upcall mechanism.
+
+ KUnit tests run during boot and output the results to the debug
+ log in TAP format (https://testanything.org/). Only useful for
+ kernel devs running KUnit test harness and are not for inclusion
+ into a production build.
+
+ For more information on KUnit and unit tests in general, refer
+ to the KUnit documentation in Documentation/dev-tools/kunit/.
+
config INET
bool "TCP/IP networking"
help
diff --git a/net/Makefile b/net/Makefile
index 8759200..4c4dc53 100644
--- a/net/Makefile
+++ b/net/Makefile
@@ -79,3 +79,4 @@
obj-$(CONFIG_XDP_SOCKETS) += xdp/
obj-$(CONFIG_MPTCP) += mptcp/
obj-$(CONFIG_MCTP) += mctp/
+obj-$(CONFIG_NET_HANDSHAKE) += handshake/
diff --git a/net/bridge/br_arp_nd_proxy.c b/net/bridge/br_arp_nd_proxy.c
index b45c00c..c7869a2 100644
--- a/net/bridge/br_arp_nd_proxy.c
+++ b/net/bridge/br_arp_nd_proxy.c
@@ -30,7 +30,7 @@ void br_recalculate_neigh_suppress_enabled(struct net_bridge *br)
bool neigh_suppress = false;
list_for_each_entry(p, &br->port_list, list) {
- if (p->flags & BR_NEIGH_SUPPRESS) {
+ if (p->flags & (BR_NEIGH_SUPPRESS | BR_NEIGH_VLAN_SUPPRESS)) {
neigh_suppress = true;
break;
}
@@ -158,7 +158,7 @@ void br_do_proxy_suppress_arp(struct sk_buff *skb, struct net_bridge *br,
return;
if (br_opt_get(br, BROPT_NEIGH_SUPPRESS_ENABLED)) {
- if (p && (p->flags & BR_NEIGH_SUPPRESS))
+ if (br_is_neigh_suppress_enabled(p, vid))
return;
if (parp->ar_op != htons(ARPOP_RREQUEST) &&
parp->ar_op != htons(ARPOP_RREPLY) &&
@@ -202,8 +202,8 @@ void br_do_proxy_suppress_arp(struct sk_buff *skb, struct net_bridge *br,
bool replied = false;
if ((p && (p->flags & BR_PROXYARP)) ||
- (f->dst && (f->dst->flags & (BR_PROXYARP_WIFI |
- BR_NEIGH_SUPPRESS)))) {
+ (f->dst && (f->dst->flags & BR_PROXYARP_WIFI)) ||
+ br_is_neigh_suppress_enabled(f->dst, vid)) {
if (!vid)
br_arp_send(br, p, skb->dev, sip, tip,
sha, n->ha, sha, 0, 0);
@@ -407,7 +407,7 @@ void br_do_suppress_nd(struct sk_buff *skb, struct net_bridge *br,
BR_INPUT_SKB_CB(skb)->proxyarp_replied = 0;
- if (p && (p->flags & BR_NEIGH_SUPPRESS))
+ if (br_is_neigh_suppress_enabled(p, vid))
return;
if (msg->icmph.icmp6_type == NDISC_NEIGHBOUR_ADVERTISEMENT &&
@@ -461,7 +461,7 @@ void br_do_suppress_nd(struct sk_buff *skb, struct net_bridge *br,
if (f) {
bool replied = false;
- if (f->dst && (f->dst->flags & BR_NEIGH_SUPPRESS)) {
+ if (br_is_neigh_suppress_enabled(f->dst, vid)) {
if (vid != 0)
br_nd_send(br, p, skb, n,
skb->vlan_proto,
@@ -483,3 +483,24 @@ void br_do_suppress_nd(struct sk_buff *skb, struct net_bridge *br,
}
}
#endif
+
+bool br_is_neigh_suppress_enabled(const struct net_bridge_port *p, u16 vid)
+{
+ if (!p)
+ return false;
+
+ if (!vid)
+ return !!(p->flags & BR_NEIGH_SUPPRESS);
+
+ if (p->flags & BR_NEIGH_VLAN_SUPPRESS) {
+ struct net_bridge_vlan_group *vg = nbp_vlan_group_rcu(p);
+ struct net_bridge_vlan *v;
+
+ v = br_vlan_find(vg, vid);
+ if (!v)
+ return false;
+ return !!(v->priv_flags & BR_VLFLAG_NEIGH_SUPPRESS_ENABLED);
+ } else {
+ return !!(p->flags & BR_NEIGH_SUPPRESS);
+ }
+}
diff --git a/net/bridge/br_device.c b/net/bridge/br_device.c
index df47c87..8eca8a5 100644
--- a/net/bridge/br_device.c
+++ b/net/bridge/br_device.c
@@ -80,10 +80,10 @@ netdev_tx_t br_dev_xmit(struct sk_buff *skb, struct net_device *dev)
dest = eth_hdr(skb)->h_dest;
if (is_broadcast_ether_addr(dest)) {
- br_flood(br, skb, BR_PKT_BROADCAST, false, true);
+ br_flood(br, skb, BR_PKT_BROADCAST, false, true, vid);
} else if (is_multicast_ether_addr(dest)) {
if (unlikely(netpoll_tx_running(dev))) {
- br_flood(br, skb, BR_PKT_MULTICAST, false, true);
+ br_flood(br, skb, BR_PKT_MULTICAST, false, true, vid);
goto out;
}
if (br_multicast_rcv(&brmctx, &pmctx_null, vlan, skb, vid)) {
@@ -96,11 +96,11 @@ netdev_tx_t br_dev_xmit(struct sk_buff *skb, struct net_device *dev)
br_multicast_querier_exists(brmctx, eth_hdr(skb), mdst))
br_multicast_flood(mdst, skb, brmctx, false, true);
else
- br_flood(br, skb, BR_PKT_MULTICAST, false, true);
+ br_flood(br, skb, BR_PKT_MULTICAST, false, true, vid);
} else if ((dst = br_fdb_find_rcu(br, dest, vid)) != NULL) {
br_forward(dst->dst, skb, false, true);
} else {
- br_flood(br, skb, BR_PKT_UNICAST, false, true);
+ br_flood(br, skb, BR_PKT_UNICAST, false, true, vid);
}
out:
rcu_read_unlock();
diff --git a/net/bridge/br_forward.c b/net/bridge/br_forward.c
index 02bb620..5774470 100644
--- a/net/bridge/br_forward.c
+++ b/net/bridge/br_forward.c
@@ -197,7 +197,8 @@ static struct net_bridge_port *maybe_deliver(
/* called under rcu_read_lock */
void br_flood(struct net_bridge *br, struct sk_buff *skb,
- enum br_pkt_type pkt_type, bool local_rcv, bool local_orig)
+ enum br_pkt_type pkt_type, bool local_rcv, bool local_orig,
+ u16 vid)
{
struct net_bridge_port *prev = NULL;
struct net_bridge_port *p;
@@ -224,8 +225,9 @@ void br_flood(struct net_bridge *br, struct sk_buff *skb,
/* Do not flood to ports that enable proxy ARP */
if (p->flags & BR_PROXYARP)
continue;
- if ((p->flags & (BR_PROXYARP_WIFI | BR_NEIGH_SUPPRESS)) &&
- BR_INPUT_SKB_CB(skb)->proxyarp_replied)
+ if (BR_INPUT_SKB_CB(skb)->proxyarp_replied &&
+ ((p->flags & BR_PROXYARP_WIFI) ||
+ br_is_neigh_suppress_enabled(p, vid)))
continue;
prev = maybe_deliver(prev, p, skb, local_orig);
diff --git a/net/bridge/br_if.c b/net/bridge/br_if.c
index 24f01ff..3f04b40 100644
--- a/net/bridge/br_if.c
+++ b/net/bridge/br_if.c
@@ -759,7 +759,7 @@ void br_port_flags_change(struct net_bridge_port *p, unsigned long mask)
if (mask & BR_AUTO_MASK)
nbp_update_port_count(br);
- if (mask & BR_NEIGH_SUPPRESS)
+ if (mask & (BR_NEIGH_SUPPRESS | BR_NEIGH_VLAN_SUPPRESS))
br_recalculate_neigh_suppress_enabled(br);
}
diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c
index 3027e8f..fc17b9f 100644
--- a/net/bridge/br_input.c
+++ b/net/bridge/br_input.c
@@ -207,7 +207,7 @@ int br_handle_frame_finish(struct net *net, struct sock *sk, struct sk_buff *skb
br_forward(dst->dst, skb, local_rcv, false);
} else {
if (!mcast_hit)
- br_flood(br, skb, pkt_type, local_rcv, false);
+ br_flood(br, skb, pkt_type, local_rcv, false, vid);
else
br_multicast_flood(mdst, skb, brmctx, local_rcv, false);
}
diff --git a/net/bridge/br_netfilter_hooks.c b/net/bridge/br_netfilter_hooks.c
index 3e3065b..1a801fa 100644
--- a/net/bridge/br_netfilter_hooks.c
+++ b/net/bridge/br_netfilter_hooks.c
@@ -869,12 +869,17 @@ static unsigned int ip_sabotage_in(void *priv,
{
struct nf_bridge_info *nf_bridge = nf_bridge_info_get(skb);
- if (nf_bridge && !nf_bridge->in_prerouting &&
- !netif_is_l3_master(skb->dev) &&
- !netif_is_l3_slave(skb->dev)) {
- nf_bridge_info_free(skb);
- state->okfn(state->net, state->sk, skb);
- return NF_STOLEN;
+ if (nf_bridge) {
+ if (nf_bridge->sabotage_in_done)
+ return NF_ACCEPT;
+
+ if (!nf_bridge->in_prerouting &&
+ !netif_is_l3_master(skb->dev) &&
+ !netif_is_l3_slave(skb->dev)) {
+ nf_bridge->sabotage_in_done = 1;
+ state->okfn(state->net, state->sk, skb);
+ return NF_STOLEN;
+ }
}
return NF_ACCEPT;
diff --git a/net/bridge/br_netlink.c b/net/bridge/br_netlink.c
index fefb1c0..05c5863 100644
--- a/net/bridge/br_netlink.c
+++ b/net/bridge/br_netlink.c
@@ -189,6 +189,7 @@ static inline size_t br_port_info_size(void)
+ nla_total_size(1) /* IFLA_BRPORT_ISOLATED */
+ nla_total_size(1) /* IFLA_BRPORT_LOCKED */
+ nla_total_size(1) /* IFLA_BRPORT_MAB */
+ + nla_total_size(1) /* IFLA_BRPORT_NEIGH_VLAN_SUPPRESS */
+ nla_total_size(sizeof(struct ifla_bridge_id)) /* IFLA_BRPORT_ROOT_ID */
+ nla_total_size(sizeof(struct ifla_bridge_id)) /* IFLA_BRPORT_BRIDGE_ID */
+ nla_total_size(sizeof(u16)) /* IFLA_BRPORT_DESIGNATED_PORT */
@@ -278,7 +279,9 @@ static int br_port_fill_attrs(struct sk_buff *skb,
!!(p->flags & BR_MRP_LOST_IN_CONT)) ||
nla_put_u8(skb, IFLA_BRPORT_ISOLATED, !!(p->flags & BR_ISOLATED)) ||
nla_put_u8(skb, IFLA_BRPORT_LOCKED, !!(p->flags & BR_PORT_LOCKED)) ||
- nla_put_u8(skb, IFLA_BRPORT_MAB, !!(p->flags & BR_PORT_MAB)))
+ nla_put_u8(skb, IFLA_BRPORT_MAB, !!(p->flags & BR_PORT_MAB)) ||
+ nla_put_u8(skb, IFLA_BRPORT_NEIGH_VLAN_SUPPRESS,
+ !!(p->flags & BR_NEIGH_VLAN_SUPPRESS)))
return -EMSGSIZE;
timerval = br_timer_value(&p->message_age_timer);
@@ -891,6 +894,7 @@ static const struct nla_policy br_port_policy[IFLA_BRPORT_MAX + 1] = {
[IFLA_BRPORT_MCAST_EHT_HOSTS_LIMIT] = { .type = NLA_U32 },
[IFLA_BRPORT_MCAST_N_GROUPS] = { .type = NLA_REJECT },
[IFLA_BRPORT_MCAST_MAX_GROUPS] = { .type = NLA_U32 },
+ [IFLA_BRPORT_NEIGH_VLAN_SUPPRESS] = NLA_POLICY_MAX(NLA_U8, 1),
};
/* Change the state of the port and notify spanning tree */
@@ -957,6 +961,8 @@ static int br_setport(struct net_bridge_port *p, struct nlattr *tb[],
br_set_port_flag(p, tb, IFLA_BRPORT_ISOLATED, BR_ISOLATED);
br_set_port_flag(p, tb, IFLA_BRPORT_LOCKED, BR_PORT_LOCKED);
br_set_port_flag(p, tb, IFLA_BRPORT_MAB, BR_PORT_MAB);
+ br_set_port_flag(p, tb, IFLA_BRPORT_NEIGH_VLAN_SUPPRESS,
+ BR_NEIGH_VLAN_SUPPRESS);
if ((p->flags & BR_PORT_MAB) &&
(!(p->flags & BR_PORT_LOCKED) || !(p->flags & BR_LEARNING))) {
diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
index 7264fd4..2119729 100644
--- a/net/bridge/br_private.h
+++ b/net/bridge/br_private.h
@@ -178,6 +178,7 @@ enum {
BR_VLFLAG_ADDED_BY_SWITCHDEV = BIT(1),
BR_VLFLAG_MCAST_ENABLED = BIT(2),
BR_VLFLAG_GLOBAL_MCAST_ENABLED = BIT(3),
+ BR_VLFLAG_NEIGH_SUPPRESS_ENABLED = BIT(4),
};
/**
@@ -849,7 +850,8 @@ void br_forward(const struct net_bridge_port *to, struct sk_buff *skb,
bool local_rcv, bool local_orig);
int br_forward_finish(struct net *net, struct sock *sk, struct sk_buff *skb);
void br_flood(struct net_bridge *br, struct sk_buff *skb,
- enum br_pkt_type pkt_type, bool local_rcv, bool local_orig);
+ enum br_pkt_type pkt_type, bool local_rcv, bool local_orig,
+ u16 vid);
/* return true if both source port and dest port are isolated */
static inline bool br_skb_isolated(const struct net_bridge_port *to,
@@ -2218,4 +2220,5 @@ void br_do_proxy_suppress_arp(struct sk_buff *skb, struct net_bridge *br,
void br_do_suppress_nd(struct sk_buff *skb, struct net_bridge *br,
u16 vid, struct net_bridge_port *p, struct nd_msg *msg);
struct nd_msg *br_is_nd_neigh_msg(struct sk_buff *skb, struct nd_msg *m);
+bool br_is_neigh_suppress_enabled(const struct net_bridge_port *p, u16 vid);
#endif
diff --git a/net/bridge/br_switchdev.c b/net/bridge/br_switchdev.c
index de18e9c..ba95c4d 100644
--- a/net/bridge/br_switchdev.c
+++ b/net/bridge/br_switchdev.c
@@ -148,6 +148,17 @@ br_switchdev_fdb_notify(struct net_bridge *br,
if (test_bit(BR_FDB_LOCKED, &fdb->flags))
return;
+ /* Entries with these flags were created using ndm_state == NUD_REACHABLE,
+ * ndm_flags == NTF_MASTER( | NTF_STICKY), ext_flags == 0 by something
+ * equivalent to 'bridge fdb add ... master dynamic (sticky)'.
+ * Drivers don't know how to deal with these, so don't notify them to
+ * avoid confusing them.
+ */
+ if (test_bit(BR_FDB_ADDED_BY_USER, &fdb->flags) &&
+ !test_bit(BR_FDB_STATIC, &fdb->flags) &&
+ !test_bit(BR_FDB_ADDED_BY_EXT_LEARN, &fdb->flags))
+ return;
+
br_switchdev_fdb_populate(br, &item, fdb, NULL);
switch (type) {
diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c
index 8a3dbc0..15f44d0 100644
--- a/net/bridge/br_vlan.c
+++ b/net/bridge/br_vlan.c
@@ -2134,6 +2134,7 @@ static const struct nla_policy br_vlan_db_policy[BRIDGE_VLANDB_ENTRY_MAX + 1] =
[BRIDGE_VLANDB_ENTRY_MCAST_ROUTER] = { .type = NLA_U8 },
[BRIDGE_VLANDB_ENTRY_MCAST_N_GROUPS] = { .type = NLA_REJECT },
[BRIDGE_VLANDB_ENTRY_MCAST_MAX_GROUPS] = { .type = NLA_U32 },
+ [BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS] = NLA_POLICY_MAX(NLA_U8, 1),
};
static int br_vlan_rtm_process_one(struct net_device *dev,
diff --git a/net/bridge/br_vlan_options.c b/net/bridge/br_vlan_options.c
index e378c2f..8fa89b0 100644
--- a/net/bridge/br_vlan_options.c
+++ b/net/bridge/br_vlan_options.c
@@ -52,7 +52,9 @@ bool br_vlan_opts_fill(struct sk_buff *skb, const struct net_bridge_vlan *v,
const struct net_bridge_port *p)
{
if (nla_put_u8(skb, BRIDGE_VLANDB_ENTRY_STATE, br_vlan_get_state(v)) ||
- !__vlan_tun_put(skb, v))
+ !__vlan_tun_put(skb, v) ||
+ nla_put_u8(skb, BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS,
+ !!(v->priv_flags & BR_VLFLAG_NEIGH_SUPPRESS_ENABLED)))
return false;
#ifdef CONFIG_BRIDGE_IGMP_SNOOPING
@@ -80,6 +82,7 @@ size_t br_vlan_opts_nl_size(void)
+ nla_total_size(sizeof(u32)) /* BRIDGE_VLANDB_ENTRY_MCAST_N_GROUPS */
+ nla_total_size(sizeof(u32)) /* BRIDGE_VLANDB_ENTRY_MCAST_MAX_GROUPS */
#endif
+ + nla_total_size(sizeof(u8)) /* BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS */
+ 0;
}
@@ -239,6 +242,21 @@ static int br_vlan_process_one_opts(const struct net_bridge *br,
}
#endif
+ if (tb[BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS]) {
+ bool enabled = v->priv_flags & BR_VLFLAG_NEIGH_SUPPRESS_ENABLED;
+ bool val = nla_get_u8(tb[BRIDGE_VLANDB_ENTRY_NEIGH_SUPPRESS]);
+
+ if (!p) {
+ NL_SET_ERR_MSG_MOD(extack, "Can't set neigh_suppress for non-port vlans");
+ return -EINVAL;
+ }
+
+ if (val != enabled) {
+ v->priv_flags ^= BR_VLFLAG_NEIGH_SUPPRESS_ENABLED;
+ *changed = true;
+ }
+ }
+
return 0;
}
diff --git a/net/compat.c b/net/compat.c
index 161b7be..6564720 100644
--- a/net/compat.c
+++ b/net/compat.c
@@ -113,7 +113,7 @@ int get_compat_msghdr(struct msghdr *kmsg,
#define CMSG_COMPAT_FIRSTHDR(msg) \
(((msg)->msg_controllen) >= sizeof(struct compat_cmsghdr) ? \
- (struct compat_cmsghdr __user *)((msg)->msg_control) : \
+ (struct compat_cmsghdr __user *)((msg)->msg_control_user) : \
(struct compat_cmsghdr __user *)NULL)
#define CMSG_COMPAT_OK(ucmlen, ucmsg, mhdr) \
@@ -126,7 +126,7 @@ static inline struct compat_cmsghdr __user *cmsg_compat_nxthdr(struct msghdr *ms
struct compat_cmsghdr __user *cmsg, int cmsg_len)
{
char __user *ptr = (char __user *)cmsg + CMSG_COMPAT_ALIGN(cmsg_len);
- if ((unsigned long)(ptr + 1 - (char __user *)msg->msg_control) >
+ if ((unsigned long)(ptr + 1 - (char __user *)msg->msg_control_user) >
msg->msg_controllen)
return NULL;
return (struct compat_cmsghdr __user *)ptr;
@@ -211,6 +211,7 @@ int cmsghdr_from_user_compat_to_kern(struct msghdr *kmsg, struct sock *sk,
goto Einval;
/* Ok, looks like we made it. Hook it up and return success. */
+ kmsg->msg_control_is_user = false;
kmsg->msg_control = kcmsg_base;
kmsg->msg_controllen = kcmlen;
return 0;
@@ -225,7 +226,7 @@ int cmsghdr_from_user_compat_to_kern(struct msghdr *kmsg, struct sock *sk,
int put_cmsg_compat(struct msghdr *kmsg, int level, int type, int len, void *data)
{
- struct compat_cmsghdr __user *cm = (struct compat_cmsghdr __user *) kmsg->msg_control;
+ struct compat_cmsghdr __user *cm = (struct compat_cmsghdr __user *) kmsg->msg_control_user;
struct compat_cmsghdr cmhdr;
struct old_timeval32 ctv;
struct old_timespec32 cts[3];
@@ -274,7 +275,7 @@ int put_cmsg_compat(struct msghdr *kmsg, int level, int type, int len, void *dat
cmlen = CMSG_COMPAT_SPACE(len);
if (kmsg->msg_controllen < cmlen)
cmlen = kmsg->msg_controllen;
- kmsg->msg_control += cmlen;
+ kmsg->msg_control_user += cmlen;
kmsg->msg_controllen -= cmlen;
return 0;
}
@@ -289,7 +290,7 @@ static int scm_max_fds_compat(struct msghdr *msg)
void scm_detach_fds_compat(struct msghdr *msg, struct scm_cookie *scm)
{
struct compat_cmsghdr __user *cm =
- (struct compat_cmsghdr __user *)msg->msg_control;
+ (struct compat_cmsghdr __user *)msg->msg_control_user;
unsigned int o_flags = (msg->msg_flags & MSG_CMSG_CLOEXEC) ? O_CLOEXEC : 0;
int fdmax = min_t(int, scm_max_fds_compat(msg), scm->fp->count);
int __user *cmsg_data = CMSG_COMPAT_DATA(cm);
@@ -313,7 +314,7 @@ void scm_detach_fds_compat(struct msghdr *msg, struct scm_cookie *scm)
cmlen = CMSG_COMPAT_SPACE(i * sizeof(int));
if (msg->msg_controllen < cmlen)
cmlen = msg->msg_controllen;
- msg->msg_control += cmlen;
+ msg->msg_control_user += cmlen;
msg->msg_controllen -= cmlen;
}
}
diff --git a/net/core/dev.c b/net/core/dev.c
index c7f1374..1551aab 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3315,8 +3315,7 @@ int skb_crc32c_csum_help(struct sk_buff *skb)
skb->len - start, ~(__u32)0,
crc32c_csum_stub));
*(__le32 *)(skb->data + offset) = crc32c_csum;
- skb->ip_summed = CHECKSUM_NONE;
- skb->csum_not_inet = 0;
+ skb_reset_csum_not_inet(skb);
out:
return ret;
}
@@ -4359,6 +4358,7 @@ static inline void ____napi_schedule(struct softnet_data *sd,
}
list_add_tail(&napi->poll_list, &sd->poll_list);
+ WRITE_ONCE(napi->list_owner, smp_processor_id());
/* If not called from net_rx_action()
* we have to raise NET_RX_SOFTIRQ.
*/
@@ -5040,7 +5040,8 @@ static __latent_entropy void net_tx_action(struct softirq_action *h)
if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
__kfree_skb(skb);
else
- __kfree_skb_defer(skb);
+ __napi_kfree_skb(skb,
+ get_kfree_skb_cb(skb)->reason);
}
}
@@ -6069,6 +6070,7 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
list_del_init(&n->poll_list);
local_irq_restore(flags);
}
+ WRITE_ONCE(n->list_owner, -1);
val = READ_ONCE(n->state);
do {
@@ -6384,6 +6386,7 @@ void netif_napi_add_weight(struct net_device *dev, struct napi_struct *napi,
#ifdef CONFIG_NETPOLL
napi->poll_owner = -1;
#endif
+ napi->list_owner = -1;
set_bit(NAPI_STATE_SCHED, &napi->state);
set_bit(NAPI_STATE_NPSVC, &napi->state);
list_add_rcu(&napi->dev_list, &dev->napi_list);
diff --git a/net/core/drop_monitor.c b/net/core/drop_monitor.c
index 5a782d1..aff31cd9 100644
--- a/net/core/drop_monitor.c
+++ b/net/core/drop_monitor.c
@@ -21,6 +21,7 @@
#include <linux/workqueue.h>
#include <linux/netlink.h>
#include <linux/net_dropmon.h>
+#include <linux/bitfield.h>
#include <linux/percpu.h>
#include <linux/timer.h>
#include <linux/bitops.h>
@@ -29,6 +30,7 @@
#include <net/genetlink.h>
#include <net/netevent.h>
#include <net/flow_offload.h>
+#include <net/dropreason.h>
#include <net/devlink.h>
#include <trace/events/skb.h>
@@ -504,8 +506,6 @@ static void net_dm_packet_trace_kfree_skb_hit(void *ignore,
if (!nskb)
return;
- if (unlikely(reason >= SKB_DROP_REASON_MAX || reason <= 0))
- reason = SKB_DROP_REASON_NOT_SPECIFIED;
cb = NET_DM_SKB_CB(nskb);
cb->reason = reason;
cb->pc = location;
@@ -552,9 +552,9 @@ static size_t net_dm_in_port_size(void)
}
#define NET_DM_MAX_SYMBOL_LEN 40
+#define NET_DM_MAX_REASON_LEN 50
-static size_t net_dm_packet_report_size(size_t payload_len,
- enum skb_drop_reason reason)
+static size_t net_dm_packet_report_size(size_t payload_len)
{
size_t size;
@@ -576,7 +576,7 @@ static size_t net_dm_packet_report_size(size_t payload_len,
/* NET_DM_ATTR_PROTO */
nla_total_size(sizeof(u16)) +
/* NET_DM_ATTR_REASON */
- nla_total_size(strlen(drop_reasons[reason]) + 1) +
+ nla_total_size(NET_DM_MAX_REASON_LEN + 1) +
/* NET_DM_ATTR_PAYLOAD */
nla_total_size(payload_len);
}
@@ -610,6 +610,8 @@ static int net_dm_packet_report_fill(struct sk_buff *msg, struct sk_buff *skb,
size_t payload_len)
{
struct net_dm_skb_cb *cb = NET_DM_SKB_CB(skb);
+ const struct drop_reason_list *list = NULL;
+ unsigned int subsys, subsys_reason;
char buf[NET_DM_MAX_SYMBOL_LEN];
struct nlattr *attr;
void *hdr;
@@ -627,9 +629,24 @@ static int net_dm_packet_report_fill(struct sk_buff *msg, struct sk_buff *skb,
NET_DM_ATTR_PAD))
goto nla_put_failure;
+ rcu_read_lock();
+ subsys = u32_get_bits(cb->reason, SKB_DROP_REASON_SUBSYS_MASK);
+ if (subsys < SKB_DROP_REASON_SUBSYS_NUM)
+ list = rcu_dereference(drop_reasons_by_subsys[subsys]);
+ subsys_reason = cb->reason & ~SKB_DROP_REASON_SUBSYS_MASK;
+ if (!list ||
+ subsys_reason >= list->n_reasons ||
+ !list->reasons[subsys_reason] ||
+ strlen(list->reasons[subsys_reason]) > NET_DM_MAX_REASON_LEN) {
+ list = rcu_dereference(drop_reasons_by_subsys[SKB_DROP_REASON_SUBSYS_CORE]);
+ subsys_reason = SKB_DROP_REASON_NOT_SPECIFIED;
+ }
if (nla_put_string(msg, NET_DM_ATTR_REASON,
- drop_reasons[cb->reason]))
+ list->reasons[subsys_reason])) {
+ rcu_read_unlock();
goto nla_put_failure;
+ }
+ rcu_read_unlock();
snprintf(buf, sizeof(buf), "%pS", cb->pc);
if (nla_put_string(msg, NET_DM_ATTR_SYMBOL, buf))
@@ -687,9 +704,7 @@ static void net_dm_packet_report(struct sk_buff *skb)
if (net_dm_trunc_len)
payload_len = min_t(size_t, net_dm_trunc_len, payload_len);
- msg = nlmsg_new(net_dm_packet_report_size(payload_len,
- NET_DM_SKB_CB(skb)->reason),
- GFP_KERNEL);
+ msg = nlmsg_new(net_dm_packet_report_size(payload_len), GFP_KERNEL);
if (!msg)
goto out;
diff --git a/net/core/dst.c b/net/core/dst.c
index 3247e84..79d9306 100644
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -67,6 +67,7 @@ void dst_init(struct dst_entry *dst, struct dst_ops *ops,
#endif
dst->lwtstate = NULL;
rcuref_init(&dst->__rcuref, initial_ref);
+ INIT_LIST_HEAD(&dst->rt_uncached);
dst->__use = 0;
dst->lastuse = jiffies;
dst->flags = flags;
diff --git a/net/core/gro.c b/net/core/gro.c
index a606705..2d84165 100644
--- a/net/core/gro.c
+++ b/net/core/gro.c
@@ -633,7 +633,7 @@ static gro_result_t napi_skb_finish(struct napi_struct *napi,
else if (skb->fclone != SKB_FCLONE_UNAVAILABLE)
__kfree_skb(skb);
else
- __kfree_skb_defer(skb);
+ __napi_kfree_skb(skb, SKB_CONSUMED);
break;
case GRO_HELD:
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 193c187..e212e9d 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -19,6 +19,7 @@
#include <linux/mm.h> /* for put_page() */
#include <linux/poison.h>
#include <linux/ethtool.h>
+#include <linux/netdevice.h>
#include <trace/events/page_pool.h>
@@ -315,7 +316,8 @@ static bool page_pool_dma_map(struct page_pool *pool, struct page *page)
*/
dma = dma_map_page_attrs(pool->p.dev, page, 0,
(PAGE_SIZE << pool->p.order),
- pool->p.dma_dir, DMA_ATTR_SKIP_CPU_SYNC);
+ pool->p.dma_dir, DMA_ATTR_SKIP_CPU_SYNC |
+ DMA_ATTR_WEAK_ORDERING);
if (dma_mapping_error(pool->p.dev, dma))
return false;
@@ -483,7 +485,7 @@ void page_pool_release_page(struct page_pool *pool, struct page *page)
/* When page is unmapped, it cannot be returned to our pool */
dma_unmap_page_attrs(pool->p.dev, dma,
PAGE_SIZE << pool->p.order, pool->p.dma_dir,
- DMA_ATTR_SKIP_CPU_SYNC);
+ DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING);
page_pool_set_dma_addr(page, 0);
skip_dma_unmap:
page_pool_clear_pp_info(page);
@@ -837,6 +839,21 @@ void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *),
pool->xdp_mem_id = mem->id;
}
+void page_pool_unlink_napi(struct page_pool *pool)
+{
+ if (!pool->p.napi)
+ return;
+
+ /* To avoid races with recycling and additional barriers make sure
+ * pool and NAPI are unlinked when NAPI is disabled.
+ */
+ WARN_ON(!test_bit(NAPI_STATE_SCHED, &pool->p.napi->state) ||
+ READ_ONCE(pool->p.napi->list_owner) != -1);
+
+ WRITE_ONCE(pool->p.napi, NULL);
+}
+EXPORT_SYMBOL(page_pool_unlink_napi);
+
void page_pool_destroy(struct page_pool *pool)
{
if (!pool)
@@ -845,6 +862,7 @@ void page_pool_destroy(struct page_pool *pool)
if (!page_pool_put(pool))
return;
+ page_pool_unlink_napi(pool);
page_pool_free_frag(pool);
if (!page_pool_release(pool))
@@ -874,9 +892,11 @@ void page_pool_update_nid(struct page_pool *pool, int new_nid)
}
EXPORT_SYMBOL(page_pool_update_nid);
-bool page_pool_return_skb_page(struct page *page)
+bool page_pool_return_skb_page(struct page *page, bool napi_safe)
{
+ struct napi_struct *napi;
struct page_pool *pp;
+ bool allow_direct;
page = compound_head(page);
@@ -892,12 +912,20 @@ bool page_pool_return_skb_page(struct page *page)
pp = page->pp;
+ /* Allow direct recycle if we have reasons to believe that we are
+ * in the same context as the consumer would run, so there's
+ * no possible race.
+ */
+ napi = READ_ONCE(pp->p.napi);
+ allow_direct = napi_safe && napi &&
+ READ_ONCE(napi->list_owner) == smp_processor_id();
+
/* Driver set this to memory recycling info. Reset it on recycle.
* This will *not* work for NIC using a split-page memory model.
* The page will be returned to the pool here regardless of the
* 'flipped' fragment being in use or not.
*/
- page_pool_put_full_page(pp, page, false);
+ page_pool_put_full_page(pp, page, allow_direct);
return true;
}
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index e844d75..653901a 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -61,7 +61,7 @@
#include "dev.h"
#define RTNL_MAX_TYPE 50
-#define RTNL_SLAVE_MAX_TYPE 42
+#define RTNL_SLAVE_MAX_TYPE 43
struct rtnl_link {
rtnl_doit_func doit;
diff --git a/net/core/scm.c b/net/core/scm.c
index acb7d77..3cd7dd3 100644
--- a/net/core/scm.c
+++ b/net/core/scm.c
@@ -250,7 +250,10 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)
}
cmlen = min(CMSG_SPACE(len), msg->msg_controllen);
- msg->msg_control += cmlen;
+ if (msg->msg_control_is_user)
+ msg->msg_control_user += cmlen;
+ else
+ msg->msg_control += cmlen;
msg->msg_controllen -= cmlen;
return 0;
@@ -299,7 +302,7 @@ static int scm_max_fds(struct msghdr *msg)
void scm_detach_fds(struct msghdr *msg, struct scm_cookie *scm)
{
struct cmsghdr __user *cm =
- (__force struct cmsghdr __user *)msg->msg_control;
+ (__force struct cmsghdr __user *)msg->msg_control_user;
unsigned int o_flags = (msg->msg_flags & MSG_CMSG_CLOEXEC) ? O_CLOEXEC : 0;
int fdmax = min_t(int, scm_max_fds(msg), scm->fp->count);
int __user *cmsg_data = CMSG_USER_DATA(cm);
@@ -332,7 +335,7 @@ void scm_detach_fds(struct msghdr *msg, struct scm_cookie *scm)
cmlen = CMSG_SPACE(i * sizeof(int));
if (msg->msg_controllen < cmlen)
cmlen = msg->msg_controllen;
- msg->msg_control += cmlen;
+ msg->msg_control_user += cmlen;
msg->msg_controllen -= cmlen;
}
}
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 78238a1..0d99880 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -58,6 +58,7 @@
#include <linux/scatterlist.h>
#include <linux/errqueue.h>
#include <linux/prefetch.h>
+#include <linux/bitfield.h>
#include <linux/if_vlan.h>
#include <linux/mpls.h>
#include <linux/kcov.h>
@@ -72,6 +73,7 @@
#include <net/mptcp.h>
#include <net/mctp.h>
#include <net/page_pool.h>
+#include <net/dropreason.h>
#include <linux/uaccess.h>
#include <trace/events/skb.h>
@@ -122,11 +124,59 @@ EXPORT_SYMBOL(sysctl_max_skb_frags);
#undef FN
#define FN(reason) [SKB_DROP_REASON_##reason] = #reason,
-const char * const drop_reasons[] = {
+static const char * const drop_reasons[] = {
[SKB_CONSUMED] = "CONSUMED",
DEFINE_DROP_REASON(FN, FN)
};
-EXPORT_SYMBOL(drop_reasons);
+
+static const struct drop_reason_list drop_reasons_core = {
+ .reasons = drop_reasons,
+ .n_reasons = ARRAY_SIZE(drop_reasons),
+};
+
+const struct drop_reason_list __rcu *
+drop_reasons_by_subsys[SKB_DROP_REASON_SUBSYS_NUM] = {
+ [SKB_DROP_REASON_SUBSYS_CORE] = RCU_INITIALIZER(&drop_reasons_core),
+};
+EXPORT_SYMBOL(drop_reasons_by_subsys);
+
+/**
+ * drop_reasons_register_subsys - register another drop reason subsystem
+ * @subsys: the subsystem to register, must not be the core
+ * @list: the list of drop reasons within the subsystem, must point to
+ * a statically initialized list
+ */
+void drop_reasons_register_subsys(enum skb_drop_reason_subsys subsys,
+ const struct drop_reason_list *list)
+{
+ if (WARN(subsys <= SKB_DROP_REASON_SUBSYS_CORE ||
+ subsys >= ARRAY_SIZE(drop_reasons_by_subsys),
+ "invalid subsystem %d\n", subsys))
+ return;
+
+ /* must point to statically allocated memory, so INIT is OK */
+ RCU_INIT_POINTER(drop_reasons_by_subsys[subsys], list);
+}
+EXPORT_SYMBOL_GPL(drop_reasons_register_subsys);
+
+/**
+ * drop_reasons_unregister_subsys - unregister a drop reason subsystem
+ * @subsys: the subsystem to remove, must not be the core
+ *
+ * Note: This will synchronize_rcu() to ensure no users when it returns.
+ */
+void drop_reasons_unregister_subsys(enum skb_drop_reason_subsys subsys)
+{
+ if (WARN(subsys <= SKB_DROP_REASON_SUBSYS_CORE ||
+ subsys >= ARRAY_SIZE(drop_reasons_by_subsys),
+ "invalid subsystem %d\n", subsys))
+ return;
+
+ RCU_INIT_POINTER(drop_reasons_by_subsys[subsys], NULL);
+
+ synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(drop_reasons_unregister_subsys);
/**
* skb_panic - private function for out-of-line support
@@ -839,11 +889,11 @@ static void skb_clone_fraglist(struct sk_buff *skb)
skb_get(list);
}
-static bool skb_pp_recycle(struct sk_buff *skb, void *data)
+static bool skb_pp_recycle(struct sk_buff *skb, void *data, bool napi_safe)
{
if (!IS_ENABLED(CONFIG_PAGE_POOL) || !skb->pp_recycle)
return false;
- return page_pool_return_skb_page(virt_to_page(data));
+ return page_pool_return_skb_page(virt_to_page(data), napi_safe);
}
static void skb_kfree_head(void *head, unsigned int end_offset)
@@ -856,12 +906,12 @@ static void skb_kfree_head(void *head, unsigned int end_offset)
kfree(head);
}
-static void skb_free_head(struct sk_buff *skb)
+static void skb_free_head(struct sk_buff *skb, bool napi_safe)
{
unsigned char *head = skb->head;
if (skb->head_frag) {
- if (skb_pp_recycle(skb, head))
+ if (skb_pp_recycle(skb, head, napi_safe))
return;
skb_free_frag(head);
} else {
@@ -869,7 +919,8 @@ static void skb_free_head(struct sk_buff *skb)
}
}
-static void skb_release_data(struct sk_buff *skb, enum skb_drop_reason reason)
+static void skb_release_data(struct sk_buff *skb, enum skb_drop_reason reason,
+ bool napi_safe)
{
struct skb_shared_info *shinfo = skb_shinfo(skb);
int i;
@@ -888,13 +939,13 @@ static void skb_release_data(struct sk_buff *skb, enum skb_drop_reason reason)
}
for (i = 0; i < shinfo->nr_frags; i++)
- __skb_frag_unref(&shinfo->frags[i], skb->pp_recycle);
+ napi_frag_unref(&shinfo->frags[i], skb->pp_recycle, napi_safe);
free_head:
if (shinfo->frag_list)
kfree_skb_list_reason(shinfo->frag_list, reason);
- skb_free_head(skb);
+ skb_free_head(skb, napi_safe);
exit:
/* When we clone an SKB we copy the reycling bit. The pp_recycle
* bit is only set on the head though, so in order to avoid races
@@ -955,11 +1006,12 @@ void skb_release_head_state(struct sk_buff *skb)
}
/* Free everything but the sk_buff shell. */
-static void skb_release_all(struct sk_buff *skb, enum skb_drop_reason reason)
+static void skb_release_all(struct sk_buff *skb, enum skb_drop_reason reason,
+ bool napi_safe)
{
skb_release_head_state(skb);
if (likely(skb->head))
- skb_release_data(skb, reason);
+ skb_release_data(skb, reason, napi_safe);
}
/**
@@ -973,7 +1025,7 @@ static void skb_release_all(struct sk_buff *skb, enum skb_drop_reason reason)
void __kfree_skb(struct sk_buff *skb)
{
- skb_release_all(skb, SKB_DROP_REASON_NOT_SPECIFIED);
+ skb_release_all(skb, SKB_DROP_REASON_NOT_SPECIFIED, false);
kfree_skbmem(skb);
}
EXPORT_SYMBOL(__kfree_skb);
@@ -984,7 +1036,10 @@ bool __kfree_skb_reason(struct sk_buff *skb, enum skb_drop_reason reason)
if (unlikely(!skb_unref(skb)))
return false;
- DEBUG_NET_WARN_ON_ONCE(reason <= 0 || reason >= SKB_DROP_REASON_MAX);
+ DEBUG_NET_WARN_ON_ONCE(reason == SKB_NOT_DROPPED_YET ||
+ u32_get_bits(reason,
+ SKB_DROP_REASON_SUBSYS_MASK) >=
+ SKB_DROP_REASON_SUBSYS_NUM);
if (reason == SKB_CONSUMED)
trace_consume_skb(skb, __builtin_return_address(0));
@@ -1027,7 +1082,7 @@ static void kfree_skb_add_bulk(struct sk_buff *skb,
return;
}
- skb_release_all(skb, reason);
+ skb_release_all(skb, reason, false);
sa->skb_array[sa->skb_count++] = skb;
if (unlikely(sa->skb_count == KFREE_SKB_BULK_SIZE)) {
@@ -1201,7 +1256,7 @@ EXPORT_SYMBOL(consume_skb);
void __consume_stateless_skb(struct sk_buff *skb)
{
trace_consume_skb(skb, __builtin_return_address(0));
- skb_release_data(skb, SKB_CONSUMED);
+ skb_release_data(skb, SKB_CONSUMED, false);
kfree_skbmem(skb);
}
@@ -1224,9 +1279,9 @@ static void napi_skb_cache_put(struct sk_buff *skb)
}
}
-void __kfree_skb_defer(struct sk_buff *skb)
+void __napi_kfree_skb(struct sk_buff *skb, enum skb_drop_reason reason)
{
- skb_release_all(skb, SKB_DROP_REASON_NOT_SPECIFIED);
+ skb_release_all(skb, reason, true);
napi_skb_cache_put(skb);
}
@@ -1264,7 +1319,7 @@ void napi_consume_skb(struct sk_buff *skb, int budget)
return;
}
- skb_release_all(skb, SKB_CONSUMED);
+ skb_release_all(skb, SKB_CONSUMED, !!budget);
napi_skb_cache_put(skb);
}
EXPORT_SYMBOL(napi_consume_skb);
@@ -1395,7 +1450,7 @@ EXPORT_SYMBOL_GPL(alloc_skb_for_msg);
*/
struct sk_buff *skb_morph(struct sk_buff *dst, struct sk_buff *src)
{
- skb_release_all(dst, SKB_CONSUMED);
+ skb_release_all(dst, SKB_CONSUMED, false);
return __skb_clone(dst, src);
}
EXPORT_SYMBOL_GPL(skb_morph);
@@ -2018,9 +2073,9 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail,
if (skb_has_frag_list(skb))
skb_clone_fraglist(skb);
- skb_release_data(skb, SKB_CONSUMED);
+ skb_release_data(skb, SKB_CONSUMED, false);
} else {
- skb_free_head(skb);
+ skb_free_head(skb, false);
}
off = (data + nhead) - skb->head;
@@ -5187,6 +5242,7 @@ void skb_tstamp_tx(struct sk_buff *orig_skb,
}
EXPORT_SYMBOL_GPL(skb_tstamp_tx);
+#ifdef CONFIG_WIRELESS
void skb_complete_wifi_ack(struct sk_buff *skb, bool acked)
{
struct sock *sk = skb->sk;
@@ -5212,6 +5268,7 @@ void skb_complete_wifi_ack(struct sk_buff *skb, bool acked)
kfree_skb(skb);
}
EXPORT_SYMBOL_GPL(skb_complete_wifi_ack);
+#endif /* CONFIG_WIRELESS */
/**
* skb_partial_csum_set - set up and verify partial csum values for packet
@@ -6389,12 +6446,12 @@ static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off,
skb_frag_ref(skb, i);
if (skb_has_frag_list(skb))
skb_clone_fraglist(skb);
- skb_release_data(skb, SKB_CONSUMED);
+ skb_release_data(skb, SKB_CONSUMED, false);
} else {
/* we can reuse existing recount- all we did was
* relocate values
*/
- skb_free_head(skb);
+ skb_free_head(skb, false);
}
skb->head = data;
@@ -6529,7 +6586,7 @@ static int pskb_carve_inside_nonlinear(struct sk_buff *skb, const u32 off,
skb_kfree_head(data, size);
return -ENOMEM;
}
- skb_release_data(skb, SKB_CONSUMED);
+ skb_release_data(skb, SKB_CONSUMED, false);
skb->head = data;
skb->head_frag = 0;
diff --git a/net/ethtool/mm.c b/net/ethtool/mm.c
index fce3cc2..4058a55 100644
--- a/net/ethtool/mm.c
+++ b/net/ethtool/mm.c
@@ -214,6 +214,16 @@ static int ethnl_set_mm(struct ethnl_req_info *req_info, struct genl_info *info)
return -ERANGE;
}
+ if (cfg.verify_enabled && !cfg.tx_enabled) {
+ NL_SET_ERR_MSG(extack, "Verification requires TX enabled");
+ return -EINVAL;
+ }
+
+ if (cfg.tx_enabled && !cfg.pmac_enabled) {
+ NL_SET_ERR_MSG(extack, "TX enabled requires pMAC enabled");
+ return -EINVAL;
+ }
+
ret = dev->ethtool_ops->set_mm(dev, &cfg, extack);
return ret < 0 ? ret : 1;
}
@@ -249,3 +259,26 @@ bool __ethtool_dev_mm_supported(struct net_device *dev)
return !ret;
}
+
+bool ethtool_dev_mm_supported(struct net_device *dev)
+{
+ const struct ethtool_ops *ops = dev->ethtool_ops;
+ bool supported;
+ int ret;
+
+ ASSERT_RTNL();
+
+ if (!ops)
+ return false;
+
+ ret = ethnl_ops_begin(dev);
+ if (ret < 0)
+ return false;
+
+ supported = __ethtool_dev_mm_supported(dev);
+
+ ethnl_ops_complete(dev);
+
+ return supported;
+}
+EXPORT_SYMBOL_GPL(ethtool_dev_mm_supported);
diff --git a/net/handshake/.kunitconfig b/net/handshake/.kunitconfig
new file mode 100644
index 0000000..5c48cf4
--- /dev/null
+++ b/net/handshake/.kunitconfig
@@ -0,0 +1,11 @@
+CONFIG_KUNIT=y
+CONFIG_UBSAN=y
+CONFIG_STACKTRACE=y
+CONFIG_NET=y
+CONFIG_NETWORK_FILESYSTEMS=y
+CONFIG_INET=y
+CONFIG_MULTIUSER=y
+CONFIG_NFS_FS=y
+CONFIG_SUNRPC=y
+CONFIG_NET_HANDSHAKE=y
+CONFIG_NET_HANDSHAKE_KUNIT_TEST=y
diff --git a/net/handshake/Makefile b/net/handshake/Makefile
new file mode 100644
index 0000000..247d73c
--- /dev/null
+++ b/net/handshake/Makefile
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for the Generic HANDSHAKE service
+#
+# Author: Chuck Lever <chuck.lever@oracle.com>
+#
+# Copyright (c) 2023, Oracle and/or its affiliates.
+#
+
+obj-y += handshake.o
+handshake-y := genl.o netlink.o request.o tlshd.o trace.o
+
+obj-$(CONFIG_NET_HANDSHAKE_KUNIT_TEST) += handshake-test.o
diff --git a/net/handshake/genl.c b/net/handshake/genl.c
new file mode 100644
index 0000000..9f29efb
--- /dev/null
+++ b/net/handshake/genl.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/handshake.yaml */
+/* YNL-GEN kernel source */
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include "genl.h"
+
+#include <linux/handshake.h>
+
+/* HANDSHAKE_CMD_ACCEPT - do */
+static const struct nla_policy handshake_accept_nl_policy[HANDSHAKE_A_ACCEPT_HANDLER_CLASS + 1] = {
+ [HANDSHAKE_A_ACCEPT_HANDLER_CLASS] = NLA_POLICY_MAX(NLA_U32, 2),
+};
+
+/* HANDSHAKE_CMD_DONE - do */
+static const struct nla_policy handshake_done_nl_policy[HANDSHAKE_A_DONE_REMOTE_AUTH + 1] = {
+ [HANDSHAKE_A_DONE_STATUS] = { .type = NLA_U32, },
+ [HANDSHAKE_A_DONE_SOCKFD] = { .type = NLA_U32, },
+ [HANDSHAKE_A_DONE_REMOTE_AUTH] = { .type = NLA_U32, },
+};
+
+/* Ops table for handshake */
+static const struct genl_split_ops handshake_nl_ops[] = {
+ {
+ .cmd = HANDSHAKE_CMD_ACCEPT,
+ .doit = handshake_nl_accept_doit,
+ .policy = handshake_accept_nl_policy,
+ .maxattr = HANDSHAKE_A_ACCEPT_HANDLER_CLASS,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = HANDSHAKE_CMD_DONE,
+ .doit = handshake_nl_done_doit,
+ .policy = handshake_done_nl_policy,
+ .maxattr = HANDSHAKE_A_DONE_REMOTE_AUTH,
+ .flags = GENL_CMD_CAP_DO,
+ },
+};
+
+static const struct genl_multicast_group handshake_nl_mcgrps[] = {
+ [HANDSHAKE_NLGRP_NONE] = { "none", },
+ [HANDSHAKE_NLGRP_TLSHD] = { "tlshd", },
+};
+
+struct genl_family handshake_nl_family __ro_after_init = {
+ .name = HANDSHAKE_FAMILY_NAME,
+ .version = HANDSHAKE_FAMILY_VERSION,
+ .netnsok = true,
+ .parallel_ops = true,
+ .module = THIS_MODULE,
+ .split_ops = handshake_nl_ops,
+ .n_split_ops = ARRAY_SIZE(handshake_nl_ops),
+ .mcgrps = handshake_nl_mcgrps,
+ .n_mcgrps = ARRAY_SIZE(handshake_nl_mcgrps),
+};
diff --git a/net/handshake/genl.h b/net/handshake/genl.h
new file mode 100644
index 0000000..2c1f1aa
--- /dev/null
+++ b/net/handshake/genl.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/handshake.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_HANDSHAKE_GEN_H
+#define _LINUX_HANDSHAKE_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <linux/handshake.h>
+
+int handshake_nl_accept_doit(struct sk_buff *skb, struct genl_info *info);
+int handshake_nl_done_doit(struct sk_buff *skb, struct genl_info *info);
+
+enum {
+ HANDSHAKE_NLGRP_NONE,
+ HANDSHAKE_NLGRP_TLSHD,
+};
+
+extern struct genl_family handshake_nl_family;
+
+#endif /* _LINUX_HANDSHAKE_GEN_H */
diff --git a/net/handshake/handshake-test.c b/net/handshake/handshake-test.c
new file mode 100644
index 0000000..e6adc5d
--- /dev/null
+++ b/net/handshake/handshake-test.c
@@ -0,0 +1,523 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2023 Oracle and/or its affiliates.
+ *
+ * KUnit test of the handshake upcall mechanism.
+ */
+
+#include <kunit/test.h>
+#include <kunit/visibility.h>
+
+#include <linux/kernel.h>
+
+#include <net/sock.h>
+#include <net/genetlink.h>
+#include <net/netns/generic.h>
+
+#include <uapi/linux/handshake.h>
+#include "handshake.h"
+
+MODULE_IMPORT_NS(EXPORTED_FOR_KUNIT_TESTING);
+
+static int test_accept_func(struct handshake_req *req, struct genl_info *info,
+ int fd)
+{
+ return 0;
+}
+
+static void test_done_func(struct handshake_req *req, unsigned int status,
+ struct genl_info *info)
+{
+}
+
+struct handshake_req_alloc_test_param {
+ const char *desc;
+ struct handshake_proto *proto;
+ gfp_t gfp;
+ bool expect_success;
+};
+
+static struct handshake_proto handshake_req_alloc_proto_2 = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_NONE,
+};
+
+static struct handshake_proto handshake_req_alloc_proto_3 = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_MAX,
+};
+
+static struct handshake_proto handshake_req_alloc_proto_4 = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+};
+
+static struct handshake_proto handshake_req_alloc_proto_5 = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+ .hp_accept = test_accept_func,
+};
+
+static struct handshake_proto handshake_req_alloc_proto_6 = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+ .hp_privsize = UINT_MAX,
+ .hp_accept = test_accept_func,
+ .hp_done = test_done_func,
+};
+
+static struct handshake_proto handshake_req_alloc_proto_good = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+ .hp_accept = test_accept_func,
+ .hp_done = test_done_func,
+};
+
+static const
+struct handshake_req_alloc_test_param handshake_req_alloc_params[] = {
+ {
+ .desc = "handshake_req_alloc NULL proto",
+ .proto = NULL,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc CLASS_NONE",
+ .proto = &handshake_req_alloc_proto_2,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc CLASS_MAX",
+ .proto = &handshake_req_alloc_proto_3,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc no callbacks",
+ .proto = &handshake_req_alloc_proto_4,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc no done callback",
+ .proto = &handshake_req_alloc_proto_5,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc excessive privsize",
+ .proto = &handshake_req_alloc_proto_6,
+ .gfp = GFP_KERNEL,
+ .expect_success = false,
+ },
+ {
+ .desc = "handshake_req_alloc all good",
+ .proto = &handshake_req_alloc_proto_good,
+ .gfp = GFP_KERNEL,
+ .expect_success = true,
+ },
+};
+
+static void
+handshake_req_alloc_get_desc(const struct handshake_req_alloc_test_param *param,
+ char *desc)
+{
+ strscpy(desc, param->desc, KUNIT_PARAM_DESC_SIZE);
+}
+
+/* Creates the function handshake_req_alloc_gen_params */
+KUNIT_ARRAY_PARAM(handshake_req_alloc, handshake_req_alloc_params,
+ handshake_req_alloc_get_desc);
+
+static void handshake_req_alloc_case(struct kunit *test)
+{
+ const struct handshake_req_alloc_test_param *param = test->param_value;
+ struct handshake_req *result;
+
+ /* Arrange */
+
+ /* Act */
+ result = handshake_req_alloc(param->proto, param->gfp);
+
+ /* Assert */
+ if (param->expect_success)
+ KUNIT_EXPECT_NOT_NULL(test, result);
+ else
+ KUNIT_EXPECT_NULL(test, result);
+
+ kfree(result);
+}
+
+static void handshake_req_submit_test1(struct kunit *test)
+{
+ struct socket *sock;
+ int err, result;
+
+ /* Arrange */
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ /* Act */
+ result = handshake_req_submit(sock, NULL, GFP_KERNEL);
+
+ /* Assert */
+ KUNIT_EXPECT_EQ(test, result, -EINVAL);
+
+ sock_release(sock);
+}
+
+static void handshake_req_submit_test2(struct kunit *test)
+{
+ struct handshake_req *req;
+ int result;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ /* Act */
+ result = handshake_req_submit(NULL, req, GFP_KERNEL);
+
+ /* Assert */
+ KUNIT_EXPECT_EQ(test, result, -EINVAL);
+
+ /* handshake_req_submit() destroys @req on error */
+}
+
+static void handshake_req_submit_test3(struct kunit *test)
+{
+ struct handshake_req *req;
+ struct socket *sock;
+ int err, result;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ sock->file = NULL;
+
+ /* Act */
+ result = handshake_req_submit(sock, req, GFP_KERNEL);
+
+ /* Assert */
+ KUNIT_EXPECT_EQ(test, result, -EINVAL);
+
+ /* handshake_req_submit() destroys @req on error */
+ sock_release(sock);
+}
+
+static void handshake_req_submit_test4(struct kunit *test)
+{
+ struct handshake_req *req, *result;
+ struct socket *sock;
+ int err;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+ KUNIT_ASSERT_NOT_NULL(test, sock->sk);
+
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ /* Act */
+ result = handshake_req_hash_lookup(sock->sk);
+
+ /* Assert */
+ KUNIT_EXPECT_NOT_NULL(test, result);
+ KUNIT_EXPECT_PTR_EQ(test, req, result);
+
+ handshake_req_cancel(sock->sk);
+ sock_release(sock);
+}
+
+static void handshake_req_submit_test5(struct kunit *test)
+{
+ struct handshake_req *req;
+ struct handshake_net *hn;
+ struct socket *sock;
+ struct net *net;
+ int saved, err;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+ KUNIT_ASSERT_NOT_NULL(test, sock->sk);
+
+ net = sock_net(sock->sk);
+ hn = handshake_pernet(net);
+ KUNIT_ASSERT_NOT_NULL(test, hn);
+
+ saved = hn->hn_pending;
+ hn->hn_pending = hn->hn_pending_max + 1;
+
+ /* Act */
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+
+ /* Assert */
+ KUNIT_EXPECT_EQ(test, err, -EAGAIN);
+
+ sock_release(sock);
+ hn->hn_pending = saved;
+}
+
+static void handshake_req_submit_test6(struct kunit *test)
+{
+ struct handshake_req *req1, *req2;
+ struct socket *sock;
+ int err;
+
+ /* Arrange */
+ req1 = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req1);
+ req2 = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req2);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+ KUNIT_ASSERT_NOT_NULL(test, sock->sk);
+
+ /* Act */
+ err = handshake_req_submit(sock, req1, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+ err = handshake_req_submit(sock, req2, GFP_KERNEL);
+
+ /* Assert */
+ KUNIT_EXPECT_EQ(test, err, -EBUSY);
+
+ handshake_req_cancel(sock->sk);
+ sock_release(sock);
+}
+
+static void handshake_req_cancel_test1(struct kunit *test)
+{
+ struct handshake_req *req;
+ struct socket *sock;
+ bool result;
+ int err;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ /* NB: handshake_req hasn't been accepted */
+
+ /* Act */
+ result = handshake_req_cancel(sock->sk);
+
+ /* Assert */
+ KUNIT_EXPECT_TRUE(test, result);
+
+ sock_release(sock);
+}
+
+static void handshake_req_cancel_test2(struct kunit *test)
+{
+ struct handshake_req *req, *next;
+ struct handshake_net *hn;
+ struct socket *sock;
+ struct net *net;
+ bool result;
+ int err;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ net = sock_net(sock->sk);
+ hn = handshake_pernet(net);
+ KUNIT_ASSERT_NOT_NULL(test, hn);
+
+ /* Pretend to accept this request */
+ next = handshake_req_next(hn, HANDSHAKE_HANDLER_CLASS_TLSHD);
+ KUNIT_ASSERT_PTR_EQ(test, req, next);
+
+ /* Act */
+ result = handshake_req_cancel(sock->sk);
+
+ /* Assert */
+ KUNIT_EXPECT_TRUE(test, result);
+
+ sock_release(sock);
+}
+
+static void handshake_req_cancel_test3(struct kunit *test)
+{
+ struct handshake_req *req, *next;
+ struct handshake_net *hn;
+ struct socket *sock;
+ struct net *net;
+ bool result;
+ int err;
+
+ /* Arrange */
+ req = handshake_req_alloc(&handshake_req_alloc_proto_good, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ net = sock_net(sock->sk);
+ hn = handshake_pernet(net);
+ KUNIT_ASSERT_NOT_NULL(test, hn);
+
+ /* Pretend to accept this request */
+ next = handshake_req_next(hn, HANDSHAKE_HANDLER_CLASS_TLSHD);
+ KUNIT_ASSERT_PTR_EQ(test, req, next);
+
+ /* Pretend to complete this request */
+ handshake_complete(next, -ETIMEDOUT, NULL);
+
+ /* Act */
+ result = handshake_req_cancel(sock->sk);
+
+ /* Assert */
+ KUNIT_EXPECT_FALSE(test, result);
+
+ sock_release(sock);
+}
+
+static struct handshake_req *handshake_req_destroy_test;
+
+static void test_destroy_func(struct handshake_req *req)
+{
+ handshake_req_destroy_test = req;
+}
+
+static struct handshake_proto handshake_req_alloc_proto_destroy = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+ .hp_accept = test_accept_func,
+ .hp_done = test_done_func,
+ .hp_destroy = test_destroy_func,
+};
+
+static void handshake_req_destroy_test1(struct kunit *test)
+{
+ struct handshake_req *req;
+ struct socket *sock;
+ int err;
+
+ /* Arrange */
+ handshake_req_destroy_test = NULL;
+
+ req = handshake_req_alloc(&handshake_req_alloc_proto_destroy, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, req);
+
+ err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+ &sock, 1);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ sock->file = sock_alloc_file(sock, O_NONBLOCK, NULL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sock->file);
+
+ err = handshake_req_submit(sock, req, GFP_KERNEL);
+ KUNIT_ASSERT_EQ(test, err, 0);
+
+ handshake_req_cancel(sock->sk);
+
+ /* Act */
+ sock_release(sock);
+
+ /* Assert */
+ KUNIT_EXPECT_PTR_EQ(test, handshake_req_destroy_test, req);
+}
+
+static struct kunit_case handshake_api_test_cases[] = {
+ {
+ .name = "req_alloc API fuzzing",
+ .run_case = handshake_req_alloc_case,
+ .generate_params = handshake_req_alloc_gen_params,
+ },
+ {
+ .name = "req_submit NULL req arg",
+ .run_case = handshake_req_submit_test1,
+ },
+ {
+ .name = "req_submit NULL sock arg",
+ .run_case = handshake_req_submit_test2,
+ },
+ {
+ .name = "req_submit NULL sock->file",
+ .run_case = handshake_req_submit_test3,
+ },
+ {
+ .name = "req_lookup works",
+ .run_case = handshake_req_submit_test4,
+ },
+ {
+ .name = "req_submit max pending",
+ .run_case = handshake_req_submit_test5,
+ },
+ {
+ .name = "req_submit multiple",
+ .run_case = handshake_req_submit_test6,
+ },
+ {
+ .name = "req_cancel before accept",
+ .run_case = handshake_req_cancel_test1,
+ },
+ {
+ .name = "req_cancel after accept",
+ .run_case = handshake_req_cancel_test2,
+ },
+ {
+ .name = "req_cancel after done",
+ .run_case = handshake_req_cancel_test3,
+ },
+ {
+ .name = "req_destroy works",
+ .run_case = handshake_req_destroy_test1,
+ },
+ {}
+};
+
+static struct kunit_suite handshake_api_suite = {
+ .name = "Handshake API tests",
+ .test_cases = handshake_api_test_cases,
+};
+
+kunit_test_suites(&handshake_api_suite);
+
+MODULE_DESCRIPTION("Test handshake upcall API functions");
+MODULE_LICENSE("GPL");
diff --git a/net/handshake/handshake.h b/net/handshake/handshake.h
new file mode 100644
index 0000000..4dac965
--- /dev/null
+++ b/net/handshake/handshake.h
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Generic netlink handshake service
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2023, Oracle and/or its affiliates.
+ */
+
+#ifndef _INTERNAL_HANDSHAKE_H
+#define _INTERNAL_HANDSHAKE_H
+
+/* Per-net namespace context */
+struct handshake_net {
+ spinlock_t hn_lock; /* protects next 3 fields */
+ int hn_pending;
+ int hn_pending_max;
+ struct list_head hn_requests;
+
+ unsigned long hn_flags;
+};
+
+enum hn_flags_bits {
+ HANDSHAKE_F_NET_DRAINING,
+};
+
+struct handshake_proto;
+
+/* One handshake request */
+struct handshake_req {
+ struct list_head hr_list;
+ struct rhash_head hr_rhash;
+ unsigned long hr_flags;
+ const struct handshake_proto *hr_proto;
+ struct sock *hr_sk;
+ void (*hr_odestruct)(struct sock *sk);
+
+ /* Always the last field */
+ char hr_priv[];
+};
+
+enum hr_flags_bits {
+ HANDSHAKE_F_REQ_COMPLETED,
+};
+
+/* Invariants for all handshake requests for one transport layer
+ * security protocol
+ */
+struct handshake_proto {
+ int hp_handler_class;
+ size_t hp_privsize;
+ unsigned long hp_flags;
+
+ int (*hp_accept)(struct handshake_req *req,
+ struct genl_info *info, int fd);
+ void (*hp_done)(struct handshake_req *req,
+ unsigned int status,
+ struct genl_info *info);
+ void (*hp_destroy)(struct handshake_req *req);
+};
+
+enum hp_flags_bits {
+ HANDSHAKE_F_PROTO_NOTIFY,
+};
+
+/* netlink.c */
+int handshake_genl_notify(struct net *net, const struct handshake_proto *proto,
+ gfp_t flags);
+struct nlmsghdr *handshake_genl_put(struct sk_buff *msg,
+ struct genl_info *info);
+struct handshake_net *handshake_pernet(struct net *net);
+
+/* request.c */
+struct handshake_req *handshake_req_alloc(const struct handshake_proto *proto,
+ gfp_t flags);
+int handshake_req_hash_init(void);
+void handshake_req_hash_destroy(void);
+void *handshake_req_private(struct handshake_req *req);
+struct handshake_req *handshake_req_hash_lookup(struct sock *sk);
+struct handshake_req *handshake_req_next(struct handshake_net *hn, int class);
+int handshake_req_submit(struct socket *sock, struct handshake_req *req,
+ gfp_t flags);
+void handshake_complete(struct handshake_req *req, unsigned int status,
+ struct genl_info *info);
+bool handshake_req_cancel(struct sock *sk);
+
+#endif /* _INTERNAL_HANDSHAKE_H */
diff --git a/net/handshake/netlink.c b/net/handshake/netlink.c
new file mode 100644
index 0000000..35c9c44
--- /dev/null
+++ b/net/handshake/netlink.c
@@ -0,0 +1,319 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Generic netlink handshake service
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2023, Oracle and/or its affiliates.
+ */
+
+#include <linux/types.h>
+#include <linux/socket.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/mm.h>
+
+#include <net/sock.h>
+#include <net/genetlink.h>
+#include <net/netns/generic.h>
+
+#include <kunit/visibility.h>
+
+#include <uapi/linux/handshake.h>
+#include "handshake.h"
+#include "genl.h"
+
+#include <trace/events/handshake.h>
+
+/**
+ * handshake_genl_notify - Notify handlers that a request is waiting
+ * @net: target network namespace
+ * @proto: handshake protocol
+ * @flags: memory allocation control flags
+ *
+ * Returns zero on success or a negative errno if notification failed.
+ */
+int handshake_genl_notify(struct net *net, const struct handshake_proto *proto,
+ gfp_t flags)
+{
+ struct sk_buff *msg;
+ void *hdr;
+
+ /* Disable notifications during unit testing */
+ if (!test_bit(HANDSHAKE_F_PROTO_NOTIFY, &proto->hp_flags))
+ return 0;
+
+ if (!genl_has_listeners(&handshake_nl_family, net,
+ proto->hp_handler_class))
+ return -ESRCH;
+
+ msg = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
+ hdr = genlmsg_put(msg, 0, 0, &handshake_nl_family, 0,
+ HANDSHAKE_CMD_READY);
+ if (!hdr)
+ goto out_free;
+
+ if (nla_put_u32(msg, HANDSHAKE_A_ACCEPT_HANDLER_CLASS,
+ proto->hp_handler_class) < 0) {
+ genlmsg_cancel(msg, hdr);
+ goto out_free;
+ }
+
+ genlmsg_end(msg, hdr);
+ return genlmsg_multicast_netns(&handshake_nl_family, net, msg,
+ 0, proto->hp_handler_class, flags);
+
+out_free:
+ nlmsg_free(msg);
+ return -EMSGSIZE;
+}
+
+/**
+ * handshake_genl_put - Create a generic netlink message header
+ * @msg: buffer in which to create the header
+ * @info: generic netlink message context
+ *
+ * Returns a ready-to-use header, or NULL.
+ */
+struct nlmsghdr *handshake_genl_put(struct sk_buff *msg,
+ struct genl_info *info)
+{
+ return genlmsg_put(msg, info->snd_portid, info->snd_seq,
+ &handshake_nl_family, 0, info->genlhdr->cmd);
+}
+EXPORT_SYMBOL(handshake_genl_put);
+
+/*
+ * dup() a kernel socket for use as a user space file descriptor
+ * in the current process. The kernel socket must have an
+ * instatiated struct file.
+ *
+ * Implicit argument: "current()"
+ */
+static int handshake_dup(struct socket *sock)
+{
+ struct file *file;
+ int newfd;
+
+ if (!sock->file)
+ return -EBADF;
+
+ file = get_file(sock->file);
+ newfd = get_unused_fd_flags(O_CLOEXEC);
+ if (newfd < 0) {
+ fput(file);
+ return newfd;
+ }
+
+ fd_install(newfd, file);
+ return newfd;
+}
+
+int handshake_nl_accept_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ struct net *net = sock_net(skb->sk);
+ struct handshake_net *hn = handshake_pernet(net);
+ struct handshake_req *req = NULL;
+ struct socket *sock;
+ int class, fd, err;
+
+ err = -EOPNOTSUPP;
+ if (!hn)
+ goto out_status;
+
+ err = -EINVAL;
+ if (GENL_REQ_ATTR_CHECK(info, HANDSHAKE_A_ACCEPT_HANDLER_CLASS))
+ goto out_status;
+ class = nla_get_u32(info->attrs[HANDSHAKE_A_ACCEPT_HANDLER_CLASS]);
+
+ err = -EAGAIN;
+ req = handshake_req_next(hn, class);
+ if (!req)
+ goto out_status;
+
+ sock = req->hr_sk->sk_socket;
+ fd = handshake_dup(sock);
+ if (fd < 0) {
+ err = fd;
+ goto out_complete;
+ }
+ err = req->hr_proto->hp_accept(req, info, fd);
+ if (err)
+ goto out_complete;
+
+ trace_handshake_cmd_accept(net, req, req->hr_sk, fd);
+ return 0;
+
+out_complete:
+ handshake_complete(req, -EIO, NULL);
+ fput(sock->file);
+out_status:
+ trace_handshake_cmd_accept_err(net, req, NULL, err);
+ return err;
+}
+
+int handshake_nl_done_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ struct net *net = sock_net(skb->sk);
+ struct socket *sock = NULL;
+ struct handshake_req *req;
+ int fd, status, err;
+
+ if (GENL_REQ_ATTR_CHECK(info, HANDSHAKE_A_DONE_SOCKFD))
+ return -EINVAL;
+ fd = nla_get_u32(info->attrs[HANDSHAKE_A_DONE_SOCKFD]);
+
+ err = 0;
+ sock = sockfd_lookup(fd, &err);
+ if (err) {
+ err = -EBADF;
+ goto out_status;
+ }
+
+ req = handshake_req_hash_lookup(sock->sk);
+ if (!req) {
+ err = -EBUSY;
+ fput(sock->file);
+ goto out_status;
+ }
+
+ trace_handshake_cmd_done(net, req, sock->sk, fd);
+
+ status = -EIO;
+ if (info->attrs[HANDSHAKE_A_DONE_STATUS])
+ status = nla_get_u32(info->attrs[HANDSHAKE_A_DONE_STATUS]);
+
+ handshake_complete(req, status, info);
+ fput(sock->file);
+ return 0;
+
+out_status:
+ trace_handshake_cmd_done_err(net, req, sock->sk, err);
+ return err;
+}
+
+static unsigned int handshake_net_id;
+
+static int __net_init handshake_net_init(struct net *net)
+{
+ struct handshake_net *hn = net_generic(net, handshake_net_id);
+ unsigned long tmp;
+ struct sysinfo si;
+
+ /*
+ * Arbitrary limit to prevent handshakes that do not make
+ * progress from clogging up the system. The cap scales up
+ * with the amount of physical memory on the system.
+ */
+ si_meminfo(&si);
+ tmp = si.totalram / (25 * si.mem_unit);
+ hn->hn_pending_max = clamp(tmp, 3UL, 50UL);
+
+ spin_lock_init(&hn->hn_lock);
+ hn->hn_pending = 0;
+ hn->hn_flags = 0;
+ INIT_LIST_HEAD(&hn->hn_requests);
+ return 0;
+}
+
+static void __net_exit handshake_net_exit(struct net *net)
+{
+ struct handshake_net *hn = net_generic(net, handshake_net_id);
+ struct handshake_req *req;
+ LIST_HEAD(requests);
+
+ /*
+ * Drain the net's pending list. Requests that have been
+ * accepted and are in progress will be destroyed when
+ * the socket is closed.
+ */
+ spin_lock(&hn->hn_lock);
+ set_bit(HANDSHAKE_F_NET_DRAINING, &hn->hn_flags);
+ list_splice_init(&requests, &hn->hn_requests);
+ spin_unlock(&hn->hn_lock);
+
+ while (!list_empty(&requests)) {
+ req = list_first_entry(&requests, struct handshake_req, hr_list);
+ list_del(&req->hr_list);
+
+ /*
+ * Requests on this list have not yet been
+ * accepted, so they do not have an fd to put.
+ */
+
+ handshake_complete(req, -ETIMEDOUT, NULL);
+ }
+}
+
+static struct pernet_operations handshake_genl_net_ops = {
+ .init = handshake_net_init,
+ .exit = handshake_net_exit,
+ .id = &handshake_net_id,
+ .size = sizeof(struct handshake_net),
+};
+
+/**
+ * handshake_pernet - Get the handshake private per-net structure
+ * @net: network namespace
+ *
+ * Returns a pointer to the net's private per-net structure for the
+ * handshake module, or NULL if handshake_init() failed.
+ */
+struct handshake_net *handshake_pernet(struct net *net)
+{
+ return handshake_net_id ?
+ net_generic(net, handshake_net_id) : NULL;
+}
+EXPORT_SYMBOL_IF_KUNIT(handshake_pernet);
+
+static int __init handshake_init(void)
+{
+ int ret;
+
+ ret = handshake_req_hash_init();
+ if (ret) {
+ pr_warn("handshake: hash initialization failed (%d)\n", ret);
+ return ret;
+ }
+
+ ret = genl_register_family(&handshake_nl_family);
+ if (ret) {
+ pr_warn("handshake: netlink registration failed (%d)\n", ret);
+ handshake_req_hash_destroy();
+ return ret;
+ }
+
+ /*
+ * ORDER: register_pernet_subsys must be done last.
+ *
+ * If initialization does not make it past pernet_subsys
+ * registration, then handshake_net_id will remain 0. That
+ * shunts the handshake consumer API to return ENOTSUPP
+ * to prevent it from dereferencing something that hasn't
+ * been allocated.
+ */
+ ret = register_pernet_subsys(&handshake_genl_net_ops);
+ if (ret) {
+ pr_warn("handshake: pernet registration failed (%d)\n", ret);
+ genl_unregister_family(&handshake_nl_family);
+ handshake_req_hash_destroy();
+ }
+
+ return ret;
+}
+
+static void __exit handshake_exit(void)
+{
+ unregister_pernet_subsys(&handshake_genl_net_ops);
+ handshake_net_id = 0;
+
+ handshake_req_hash_destroy();
+ genl_unregister_family(&handshake_nl_family);
+}
+
+module_init(handshake_init);
+module_exit(handshake_exit);
diff --git a/net/handshake/request.c b/net/handshake/request.c
new file mode 100644
index 0000000..94d5cef
--- /dev/null
+++ b/net/handshake/request.c
@@ -0,0 +1,344 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Handshake request lifetime events
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2023, Oracle and/or its affiliates.
+ */
+
+#include <linux/types.h>
+#include <linux/socket.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/inet.h>
+#include <linux/fdtable.h>
+#include <linux/rhashtable.h>
+
+#include <net/sock.h>
+#include <net/genetlink.h>
+#include <net/netns/generic.h>
+
+#include <kunit/visibility.h>
+
+#include <uapi/linux/handshake.h>
+#include "handshake.h"
+
+#include <trace/events/handshake.h>
+
+/*
+ * We need both a handshake_req -> sock mapping, and a sock ->
+ * handshake_req mapping. Both are one-to-one.
+ *
+ * To avoid adding another pointer field to struct sock, net/handshake
+ * maintains a hash table, indexed by the memory address of @sock, to
+ * find the struct handshake_req outstanding for that socket. The
+ * reverse direction uses a simple pointer field in the handshake_req
+ * struct.
+ */
+
+static struct rhashtable handshake_rhashtbl ____cacheline_aligned_in_smp;
+
+static const struct rhashtable_params handshake_rhash_params = {
+ .key_len = sizeof_field(struct handshake_req, hr_sk),
+ .key_offset = offsetof(struct handshake_req, hr_sk),
+ .head_offset = offsetof(struct handshake_req, hr_rhash),
+ .automatic_shrinking = true,
+};
+
+int handshake_req_hash_init(void)
+{
+ return rhashtable_init(&handshake_rhashtbl, &handshake_rhash_params);
+}
+
+void handshake_req_hash_destroy(void)
+{
+ rhashtable_destroy(&handshake_rhashtbl);
+}
+
+struct handshake_req *handshake_req_hash_lookup(struct sock *sk)
+{
+ return rhashtable_lookup_fast(&handshake_rhashtbl, &sk,
+ handshake_rhash_params);
+}
+EXPORT_SYMBOL_IF_KUNIT(handshake_req_hash_lookup);
+
+static bool handshake_req_hash_add(struct handshake_req *req)
+{
+ int ret;
+
+ ret = rhashtable_lookup_insert_fast(&handshake_rhashtbl,
+ &req->hr_rhash,
+ handshake_rhash_params);
+ return ret == 0;
+}
+
+static void handshake_req_destroy(struct handshake_req *req)
+{
+ if (req->hr_proto->hp_destroy)
+ req->hr_proto->hp_destroy(req);
+ rhashtable_remove_fast(&handshake_rhashtbl, &req->hr_rhash,
+ handshake_rhash_params);
+ kfree(req);
+}
+
+static void handshake_sk_destruct(struct sock *sk)
+{
+ void (*sk_destruct)(struct sock *sk);
+ struct handshake_req *req;
+
+ req = handshake_req_hash_lookup(sk);
+ if (!req)
+ return;
+
+ trace_handshake_destruct(sock_net(sk), req, sk);
+ sk_destruct = req->hr_odestruct;
+ handshake_req_destroy(req);
+ if (sk_destruct)
+ sk_destruct(sk);
+}
+
+/**
+ * handshake_req_alloc - Allocate a handshake request
+ * @proto: security protocol
+ * @flags: memory allocation flags
+ *
+ * Returns an initialized handshake_req or NULL.
+ */
+struct handshake_req *handshake_req_alloc(const struct handshake_proto *proto,
+ gfp_t flags)
+{
+ struct handshake_req *req;
+
+ if (!proto)
+ return NULL;
+ if (proto->hp_handler_class <= HANDSHAKE_HANDLER_CLASS_NONE)
+ return NULL;
+ if (proto->hp_handler_class >= HANDSHAKE_HANDLER_CLASS_MAX)
+ return NULL;
+ if (!proto->hp_accept || !proto->hp_done)
+ return NULL;
+
+ req = kzalloc(struct_size(req, hr_priv, proto->hp_privsize), flags);
+ if (!req)
+ return NULL;
+
+ INIT_LIST_HEAD(&req->hr_list);
+ req->hr_proto = proto;
+ return req;
+}
+EXPORT_SYMBOL(handshake_req_alloc);
+
+/**
+ * handshake_req_private - Get per-handshake private data
+ * @req: handshake arguments
+ *
+ */
+void *handshake_req_private(struct handshake_req *req)
+{
+ return (void *)&req->hr_priv;
+}
+EXPORT_SYMBOL(handshake_req_private);
+
+static bool __add_pending_locked(struct handshake_net *hn,
+ struct handshake_req *req)
+{
+ if (WARN_ON_ONCE(!list_empty(&req->hr_list)))
+ return false;
+ hn->hn_pending++;
+ list_add_tail(&req->hr_list, &hn->hn_requests);
+ return true;
+}
+
+static void __remove_pending_locked(struct handshake_net *hn,
+ struct handshake_req *req)
+{
+ hn->hn_pending--;
+ list_del_init(&req->hr_list);
+}
+
+/*
+ * Returns %true if the request was found on @net's pending list,
+ * otherwise %false.
+ *
+ * If @req was on a pending list, it has not yet been accepted.
+ */
+static bool remove_pending(struct handshake_net *hn, struct handshake_req *req)
+{
+ bool ret = false;
+
+ spin_lock(&hn->hn_lock);
+ if (!list_empty(&req->hr_list)) {
+ __remove_pending_locked(hn, req);
+ ret = true;
+ }
+ spin_unlock(&hn->hn_lock);
+
+ return ret;
+}
+
+struct handshake_req *handshake_req_next(struct handshake_net *hn, int class)
+{
+ struct handshake_req *req, *pos;
+
+ req = NULL;
+ spin_lock(&hn->hn_lock);
+ list_for_each_entry(pos, &hn->hn_requests, hr_list) {
+ if (pos->hr_proto->hp_handler_class != class)
+ continue;
+ __remove_pending_locked(hn, pos);
+ req = pos;
+ break;
+ }
+ spin_unlock(&hn->hn_lock);
+
+ return req;
+}
+EXPORT_SYMBOL_IF_KUNIT(handshake_req_next);
+
+/**
+ * handshake_req_submit - Submit a handshake request
+ * @sock: open socket on which to perform the handshake
+ * @req: handshake arguments
+ * @flags: memory allocation flags
+ *
+ * Return values:
+ * %0: Request queued
+ * %-EINVAL: Invalid argument
+ * %-EBUSY: A handshake is already under way for this socket
+ * %-ESRCH: No handshake agent is available
+ * %-EAGAIN: Too many pending handshake requests
+ * %-ENOMEM: Failed to allocate memory
+ * %-EMSGSIZE: Failed to construct notification message
+ * %-EOPNOTSUPP: Handshake module not initialized
+ *
+ * A zero return value from handshake_req_submit() means that
+ * exactly one subsequent completion callback is guaranteed.
+ *
+ * A negative return value from handshake_req_submit() means that
+ * no completion callback will be done and that @req has been
+ * destroyed.
+ */
+int handshake_req_submit(struct socket *sock, struct handshake_req *req,
+ gfp_t flags)
+{
+ struct handshake_net *hn;
+ struct net *net;
+ int ret;
+
+ if (!sock || !req || !sock->file) {
+ kfree(req);
+ return -EINVAL;
+ }
+
+ req->hr_sk = sock->sk;
+ if (!req->hr_sk) {
+ kfree(req);
+ return -EINVAL;
+ }
+ req->hr_odestruct = req->hr_sk->sk_destruct;
+ req->hr_sk->sk_destruct = handshake_sk_destruct;
+
+ ret = -EOPNOTSUPP;
+ net = sock_net(req->hr_sk);
+ hn = handshake_pernet(net);
+ if (!hn)
+ goto out_err;
+
+ ret = -EAGAIN;
+ if (READ_ONCE(hn->hn_pending) >= hn->hn_pending_max)
+ goto out_err;
+
+ spin_lock(&hn->hn_lock);
+ ret = -EOPNOTSUPP;
+ if (test_bit(HANDSHAKE_F_NET_DRAINING, &hn->hn_flags))
+ goto out_unlock;
+ ret = -EBUSY;
+ if (!handshake_req_hash_add(req))
+ goto out_unlock;
+ if (!__add_pending_locked(hn, req))
+ goto out_unlock;
+ spin_unlock(&hn->hn_lock);
+
+ ret = handshake_genl_notify(net, req->hr_proto, flags);
+ if (ret) {
+ trace_handshake_notify_err(net, req, req->hr_sk, ret);
+ if (remove_pending(hn, req))
+ goto out_err;
+ }
+
+ /* Prevent socket release while a handshake request is pending */
+ sock_hold(req->hr_sk);
+
+ trace_handshake_submit(net, req, req->hr_sk);
+ return 0;
+
+out_unlock:
+ spin_unlock(&hn->hn_lock);
+out_err:
+ trace_handshake_submit_err(net, req, req->hr_sk, ret);
+ handshake_req_destroy(req);
+ return ret;
+}
+EXPORT_SYMBOL(handshake_req_submit);
+
+void handshake_complete(struct handshake_req *req, unsigned int status,
+ struct genl_info *info)
+{
+ struct sock *sk = req->hr_sk;
+ struct net *net = sock_net(sk);
+
+ if (!test_and_set_bit(HANDSHAKE_F_REQ_COMPLETED, &req->hr_flags)) {
+ trace_handshake_complete(net, req, sk, status);
+ req->hr_proto->hp_done(req, status, info);
+
+ /* Handshake request is no longer pending */
+ sock_put(sk);
+ }
+}
+EXPORT_SYMBOL_IF_KUNIT(handshake_complete);
+
+/**
+ * handshake_req_cancel - Cancel an in-progress handshake
+ * @sk: socket on which there is an ongoing handshake
+ *
+ * Request cancellation races with request completion. To determine
+ * who won, callers examine the return value from this function.
+ *
+ * Return values:
+ * %true - Uncompleted handshake request was canceled
+ * %false - Handshake request already completed or not found
+ */
+bool handshake_req_cancel(struct sock *sk)
+{
+ struct handshake_req *req;
+ struct handshake_net *hn;
+ struct net *net;
+
+ net = sock_net(sk);
+ req = handshake_req_hash_lookup(sk);
+ if (!req) {
+ trace_handshake_cancel_none(net, req, sk);
+ return false;
+ }
+
+ hn = handshake_pernet(net);
+ if (hn && remove_pending(hn, req)) {
+ /* Request hadn't been accepted */
+ goto out_true;
+ }
+ if (test_and_set_bit(HANDSHAKE_F_REQ_COMPLETED, &req->hr_flags)) {
+ /* Request already completed */
+ trace_handshake_cancel_busy(net, req, sk);
+ return false;
+ }
+
+out_true:
+ trace_handshake_cancel(net, req, sk);
+
+ /* Handshake request is no longer pending */
+ sock_put(sk);
+ return true;
+}
+EXPORT_SYMBOL(handshake_req_cancel);
diff --git a/net/handshake/tlshd.c b/net/handshake/tlshd.c
new file mode 100644
index 0000000..fcbeb63
--- /dev/null
+++ b/net/handshake/tlshd.c
@@ -0,0 +1,418 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Establish a TLS session for a kernel socket consumer
+ * using the tlshd user space handler.
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2021-2023, Oracle and/or its affiliates.
+ */
+
+#include <linux/types.h>
+#include <linux/socket.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/key.h>
+
+#include <net/sock.h>
+#include <net/handshake.h>
+#include <net/genetlink.h>
+
+#include <uapi/linux/keyctl.h>
+#include <uapi/linux/handshake.h>
+#include "handshake.h"
+
+struct tls_handshake_req {
+ void (*th_consumer_done)(void *data, int status,
+ key_serial_t peerid);
+ void *th_consumer_data;
+
+ int th_type;
+ unsigned int th_timeout_ms;
+ int th_auth_mode;
+ key_serial_t th_keyring;
+ key_serial_t th_certificate;
+ key_serial_t th_privkey;
+
+ unsigned int th_num_peerids;
+ key_serial_t th_peerid[5];
+};
+
+static struct tls_handshake_req *
+tls_handshake_req_init(struct handshake_req *req,
+ const struct tls_handshake_args *args)
+{
+ struct tls_handshake_req *treq = handshake_req_private(req);
+
+ treq->th_timeout_ms = args->ta_timeout_ms;
+ treq->th_consumer_done = args->ta_done;
+ treq->th_consumer_data = args->ta_data;
+ treq->th_keyring = args->ta_keyring;
+ treq->th_num_peerids = 0;
+ treq->th_certificate = TLS_NO_CERT;
+ treq->th_privkey = TLS_NO_PRIVKEY;
+ return treq;
+}
+
+static void tls_handshake_remote_peerids(struct tls_handshake_req *treq,
+ struct genl_info *info)
+{
+ struct nlattr *head = nlmsg_attrdata(info->nlhdr, GENL_HDRLEN);
+ int rem, len = nlmsg_attrlen(info->nlhdr, GENL_HDRLEN);
+ struct nlattr *nla;
+ unsigned int i;
+
+ i = 0;
+ nla_for_each_attr(nla, head, len, rem) {
+ if (nla_type(nla) == HANDSHAKE_A_DONE_REMOTE_AUTH)
+ i++;
+ }
+ if (!i)
+ return;
+ treq->th_num_peerids = min_t(unsigned int, i,
+ ARRAY_SIZE(treq->th_peerid));
+
+ i = 0;
+ nla_for_each_attr(nla, head, len, rem) {
+ if (nla_type(nla) == HANDSHAKE_A_DONE_REMOTE_AUTH)
+ treq->th_peerid[i++] = nla_get_u32(nla);
+ if (i >= treq->th_num_peerids)
+ break;
+ }
+}
+
+/**
+ * tls_handshake_done - callback to handle a CMD_DONE request
+ * @req: socket on which the handshake was performed
+ * @status: session status code
+ * @info: full results of session establishment
+ *
+ */
+static void tls_handshake_done(struct handshake_req *req,
+ unsigned int status, struct genl_info *info)
+{
+ struct tls_handshake_req *treq = handshake_req_private(req);
+
+ treq->th_peerid[0] = TLS_NO_PEERID;
+ if (info)
+ tls_handshake_remote_peerids(treq, info);
+
+ treq->th_consumer_done(treq->th_consumer_data, -status,
+ treq->th_peerid[0]);
+}
+
+#if IS_ENABLED(CONFIG_KEYS)
+static int tls_handshake_private_keyring(struct tls_handshake_req *treq)
+{
+ key_ref_t process_keyring_ref, keyring_ref;
+ int ret;
+
+ if (treq->th_keyring == TLS_NO_KEYRING)
+ return 0;
+
+ process_keyring_ref = lookup_user_key(KEY_SPEC_PROCESS_KEYRING,
+ KEY_LOOKUP_CREATE,
+ KEY_NEED_WRITE);
+ if (IS_ERR(process_keyring_ref)) {
+ ret = PTR_ERR(process_keyring_ref);
+ goto out;
+ }
+
+ keyring_ref = lookup_user_key(treq->th_keyring, KEY_LOOKUP_CREATE,
+ KEY_NEED_LINK);
+ if (IS_ERR(keyring_ref)) {
+ ret = PTR_ERR(keyring_ref);
+ goto out_put_key;
+ }
+
+ ret = key_link(key_ref_to_ptr(process_keyring_ref),
+ key_ref_to_ptr(keyring_ref));
+
+ key_ref_put(keyring_ref);
+out_put_key:
+ key_ref_put(process_keyring_ref);
+out:
+ return ret;
+}
+#else
+static int tls_handshake_private_keyring(struct tls_handshake_req *treq)
+{
+ return 0;
+}
+#endif
+
+static int tls_handshake_put_peer_identity(struct sk_buff *msg,
+ struct tls_handshake_req *treq)
+{
+ unsigned int i;
+
+ for (i = 0; i < treq->th_num_peerids; i++)
+ if (nla_put_u32(msg, HANDSHAKE_A_ACCEPT_PEER_IDENTITY,
+ treq->th_peerid[i]) < 0)
+ return -EMSGSIZE;
+ return 0;
+}
+
+static int tls_handshake_put_certificate(struct sk_buff *msg,
+ struct tls_handshake_req *treq)
+{
+ struct nlattr *entry_attr;
+
+ if (treq->th_certificate == TLS_NO_CERT &&
+ treq->th_privkey == TLS_NO_PRIVKEY)
+ return 0;
+
+ entry_attr = nla_nest_start(msg, HANDSHAKE_A_ACCEPT_CERTIFICATE);
+ if (!entry_attr)
+ return -EMSGSIZE;
+
+ if (nla_put_u32(msg, HANDSHAKE_A_X509_CERT,
+ treq->th_certificate) ||
+ nla_put_u32(msg, HANDSHAKE_A_X509_PRIVKEY,
+ treq->th_privkey)) {
+ nla_nest_cancel(msg, entry_attr);
+ return -EMSGSIZE;
+ }
+
+ nla_nest_end(msg, entry_attr);
+ return 0;
+}
+
+/**
+ * tls_handshake_accept - callback to construct a CMD_ACCEPT response
+ * @req: handshake parameters to return
+ * @info: generic netlink message context
+ * @fd: file descriptor to be returned
+ *
+ * Returns zero on success, or a negative errno on failure.
+ */
+static int tls_handshake_accept(struct handshake_req *req,
+ struct genl_info *info, int fd)
+{
+ struct tls_handshake_req *treq = handshake_req_private(req);
+ struct nlmsghdr *hdr;
+ struct sk_buff *msg;
+ int ret;
+
+ ret = tls_handshake_private_keyring(treq);
+ if (ret < 0)
+ goto out;
+
+ ret = -ENOMEM;
+ msg = genlmsg_new(GENLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ goto out;
+ hdr = handshake_genl_put(msg, info);
+ if (!hdr)
+ goto out_cancel;
+
+ ret = -EMSGSIZE;
+ ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_SOCKFD, fd);
+ if (ret < 0)
+ goto out_cancel;
+ ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_MESSAGE_TYPE, treq->th_type);
+ if (ret < 0)
+ goto out_cancel;
+ if (treq->th_timeout_ms) {
+ ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_TIMEOUT, treq->th_timeout_ms);
+ if (ret < 0)
+ goto out_cancel;
+ }
+
+ ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_AUTH_MODE,
+ treq->th_auth_mode);
+ if (ret < 0)
+ goto out_cancel;
+ switch (treq->th_auth_mode) {
+ case HANDSHAKE_AUTH_PSK:
+ ret = tls_handshake_put_peer_identity(msg, treq);
+ if (ret < 0)
+ goto out_cancel;
+ break;
+ case HANDSHAKE_AUTH_X509:
+ ret = tls_handshake_put_certificate(msg, treq);
+ if (ret < 0)
+ goto out_cancel;
+ break;
+ }
+
+ genlmsg_end(msg, hdr);
+ return genlmsg_reply(msg, info);
+
+out_cancel:
+ genlmsg_cancel(msg, hdr);
+out:
+ return ret;
+}
+
+static const struct handshake_proto tls_handshake_proto = {
+ .hp_handler_class = HANDSHAKE_HANDLER_CLASS_TLSHD,
+ .hp_privsize = sizeof(struct tls_handshake_req),
+ .hp_flags = BIT(HANDSHAKE_F_PROTO_NOTIFY),
+
+ .hp_accept = tls_handshake_accept,
+ .hp_done = tls_handshake_done,
+};
+
+/**
+ * tls_client_hello_anon - request an anonymous TLS handshake on a socket
+ * @args: socket and handshake parameters for this request
+ * @flags: memory allocation control flags
+ *
+ * Return values:
+ * %0: Handshake request enqueue; ->done will be called when complete
+ * %-ESRCH: No user agent is available
+ * %-ENOMEM: Memory allocation failed
+ */
+int tls_client_hello_anon(const struct tls_handshake_args *args, gfp_t flags)
+{
+ struct tls_handshake_req *treq;
+ struct handshake_req *req;
+
+ req = handshake_req_alloc(&tls_handshake_proto, flags);
+ if (!req)
+ return -ENOMEM;
+ treq = tls_handshake_req_init(req, args);
+ treq->th_type = HANDSHAKE_MSG_TYPE_CLIENTHELLO;
+ treq->th_auth_mode = HANDSHAKE_AUTH_UNAUTH;
+
+ return handshake_req_submit(args->ta_sock, req, flags);
+}
+EXPORT_SYMBOL(tls_client_hello_anon);
+
+/**
+ * tls_client_hello_x509 - request an x.509-based TLS handshake on a socket
+ * @args: socket and handshake parameters for this request
+ * @flags: memory allocation control flags
+ *
+ * Return values:
+ * %0: Handshake request enqueue; ->done will be called when complete
+ * %-ESRCH: No user agent is available
+ * %-ENOMEM: Memory allocation failed
+ */
+int tls_client_hello_x509(const struct tls_handshake_args *args, gfp_t flags)
+{
+ struct tls_handshake_req *treq;
+ struct handshake_req *req;
+
+ req = handshake_req_alloc(&tls_handshake_proto, flags);
+ if (!req)
+ return -ENOMEM;
+ treq = tls_handshake_req_init(req, args);
+ treq->th_type = HANDSHAKE_MSG_TYPE_CLIENTHELLO;
+ treq->th_auth_mode = HANDSHAKE_AUTH_X509;
+ treq->th_certificate = args->ta_my_cert;
+ treq->th_privkey = args->ta_my_privkey;
+
+ return handshake_req_submit(args->ta_sock, req, flags);
+}
+EXPORT_SYMBOL(tls_client_hello_x509);
+
+/**
+ * tls_client_hello_psk - request a PSK-based TLS handshake on a socket
+ * @args: socket and handshake parameters for this request
+ * @flags: memory allocation control flags
+ *
+ * Return values:
+ * %0: Handshake request enqueue; ->done will be called when complete
+ * %-EINVAL: Wrong number of local peer IDs
+ * %-ESRCH: No user agent is available
+ * %-ENOMEM: Memory allocation failed
+ */
+int tls_client_hello_psk(const struct tls_handshake_args *args, gfp_t flags)
+{
+ struct tls_handshake_req *treq;
+ struct handshake_req *req;
+ unsigned int i;
+
+ if (!args->ta_num_peerids ||
+ args->ta_num_peerids > ARRAY_SIZE(treq->th_peerid))
+ return -EINVAL;
+
+ req = handshake_req_alloc(&tls_handshake_proto, flags);
+ if (!req)
+ return -ENOMEM;
+ treq = tls_handshake_req_init(req, args);
+ treq->th_type = HANDSHAKE_MSG_TYPE_CLIENTHELLO;
+ treq->th_auth_mode = HANDSHAKE_AUTH_PSK;
+ treq->th_num_peerids = args->ta_num_peerids;
+ for (i = 0; i < args->ta_num_peerids; i++)
+ treq->th_peerid[i] = args->ta_my_peerids[i];
+
+ return handshake_req_submit(args->ta_sock, req, flags);
+}
+EXPORT_SYMBOL(tls_client_hello_psk);
+
+/**
+ * tls_server_hello_x509 - request a server TLS handshake on a socket
+ * @args: socket and handshake parameters for this request
+ * @flags: memory allocation control flags
+ *
+ * Return values:
+ * %0: Handshake request enqueue; ->done will be called when complete
+ * %-ESRCH: No user agent is available
+ * %-ENOMEM: Memory allocation failed
+ */
+int tls_server_hello_x509(const struct tls_handshake_args *args, gfp_t flags)
+{
+ struct tls_handshake_req *treq;
+ struct handshake_req *req;
+
+ req = handshake_req_alloc(&tls_handshake_proto, flags);
+ if (!req)
+ return -ENOMEM;
+ treq = tls_handshake_req_init(req, args);
+ treq->th_type = HANDSHAKE_MSG_TYPE_SERVERHELLO;
+ treq->th_auth_mode = HANDSHAKE_AUTH_X509;
+ treq->th_certificate = args->ta_my_cert;
+ treq->th_privkey = args->ta_my_privkey;
+
+ return handshake_req_submit(args->ta_sock, req, flags);
+}
+EXPORT_SYMBOL(tls_server_hello_x509);
+
+/**
+ * tls_server_hello_psk - request a server TLS handshake on a socket
+ * @args: socket and handshake parameters for this request
+ * @flags: memory allocation control flags
+ *
+ * Return values:
+ * %0: Handshake request enqueue; ->done will be called when complete
+ * %-ESRCH: No user agent is available
+ * %-ENOMEM: Memory allocation failed
+ */
+int tls_server_hello_psk(const struct tls_handshake_args *args, gfp_t flags)
+{
+ struct tls_handshake_req *treq;
+ struct handshake_req *req;
+
+ req = handshake_req_alloc(&tls_handshake_proto, flags);
+ if (!req)
+ return -ENOMEM;
+ treq = tls_handshake_req_init(req, args);
+ treq->th_type = HANDSHAKE_MSG_TYPE_SERVERHELLO;
+ treq->th_auth_mode = HANDSHAKE_AUTH_PSK;
+ treq->th_num_peerids = 1;
+ treq->th_peerid[0] = args->ta_my_peerids[0];
+
+ return handshake_req_submit(args->ta_sock, req, flags);
+}
+EXPORT_SYMBOL(tls_server_hello_psk);
+
+/**
+ * tls_handshake_cancel - cancel a pending handshake
+ * @sk: socket on which there is an ongoing handshake
+ *
+ * Request cancellation races with request completion. To determine
+ * who won, callers examine the return value from this function.
+ *
+ * Return values:
+ * %true - Uncompleted handshake request was canceled
+ * %false - Handshake request already completed or not found
+ */
+bool tls_handshake_cancel(struct sock *sk)
+{
+ return handshake_req_cancel(sk);
+}
+EXPORT_SYMBOL(tls_handshake_cancel);
diff --git a/net/handshake/trace.c b/net/handshake/trace.c
new file mode 100644
index 0000000..1c4d8e2
--- /dev/null
+++ b/net/handshake/trace.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Trace points for transport security layer handshakes.
+ *
+ * Author: Chuck Lever <chuck.lever@oracle.com>
+ *
+ * Copyright (c) 2023, Oracle and/or its affiliates.
+ */
+
+#include <linux/types.h>
+
+#include <net/sock.h>
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include "handshake.h"
+
+#define CREATE_TRACE_POINTS
+
+#include <trace/events/handshake.h>
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 2a3d14d9..98d7e6b 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -1644,7 +1644,6 @@ struct rtable *rt_dst_alloc(struct net_device *dev,
rt->rt_uses_gateway = 0;
rt->rt_gw_family = 0;
rt->rt_gw4 = 0;
- INIT_LIST_HEAD(&rt->dst.rt_uncached);
rt->dst.output = ip_output;
if (flags & RTCF_LOCAL)
@@ -1675,7 +1674,6 @@ struct rtable *rt_dst_clone(struct net_device *dev, struct rtable *rt)
new_rt->rt_gw4 = rt->rt_gw4;
else if (rt->rt_gw_family == AF_INET6)
new_rt->rt_gw6 = rt->rt_gw6;
- INIT_LIST_HEAD(&new_rt->dst.rt_uncached);
new_rt->dst.input = rt->dst.input;
new_rt->dst.output = rt->dst.output;
@@ -2858,8 +2856,6 @@ struct dst_entry *ipv4_blackhole_route(struct net *net, struct dst_entry *dst_or
rt->rt_gw4 = ort->rt_gw4;
else if (rt->rt_gw_family == AF_INET6)
rt->rt_gw6 = ort->rt_gw6;
-
- INIT_LIST_HEAD(&rt->dst.rt_uncached);
}
dst_release(dst_orig);
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index fd68d49..20db115c3 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2165,7 +2165,7 @@ static void tcp_zc_finalize_rx_tstamp(struct sock *sk,
struct msghdr cmsg_dummy;
msg_control_addr = (unsigned long)zc->msg_control;
- cmsg_dummy.msg_control = (void *)msg_control_addr;
+ cmsg_dummy.msg_control_user = (void __user *)msg_control_addr;
cmsg_dummy.msg_controllen =
(__kernel_size_t)zc->msg_controllen;
cmsg_dummy.msg_flags = in_compat_syscall()
@@ -2176,7 +2176,7 @@ static void tcp_zc_finalize_rx_tstamp(struct sock *sk,
zc->msg_controllen == cmsg_dummy.msg_controllen) {
tcp_recv_timestamp(&cmsg_dummy, sk, tss);
zc->msg_control = (__u64)
- ((uintptr_t)cmsg_dummy.msg_control);
+ ((uintptr_t)cmsg_dummy.msg_control_user);
zc->msg_controllen =
(__u64)cmsg_dummy.msg_controllen;
zc->msg_flags = (__u32)cmsg_dummy.msg_flags;
diff --git a/net/ipv4/xfrm4_policy.c b/net/ipv4/xfrm4_policy.c
index 47861c8..9403bba 100644
--- a/net/ipv4/xfrm4_policy.c
+++ b/net/ipv4/xfrm4_policy.c
@@ -91,7 +91,6 @@ static int xfrm4_fill_dst(struct xfrm_dst *xdst, struct net_device *dev,
xdst->u.rt.rt_gw6 = rt->rt_gw6;
xdst->u.rt.rt_pmtu = rt->rt_pmtu;
xdst->u.rt.rt_mtu_locked = rt->rt_mtu_locked;
- INIT_LIST_HEAD(&xdst->u.rt.dst.rt_uncached);
rt_add_uncached_list(&xdst->u.rt);
return 0;
@@ -121,8 +120,7 @@ static void xfrm4_dst_destroy(struct dst_entry *dst)
struct xfrm_dst *xdst = (struct xfrm_dst *)dst;
dst_destroy_metrics_generic(dst);
- if (xdst->u.rt.dst.rt_uncached_list)
- rt_del_uncached_list(&xdst->u.rt);
+ rt_del_uncached_list(&xdst->u.rt);
xfrm_dst_destroy(xdst);
}
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index e1b679a..2bbf132 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -952,6 +952,7 @@ static int __net_init inet6_net_init(struct net *net)
net->ipv6.sysctl.icmpv6_echo_ignore_all = 0;
net->ipv6.sysctl.icmpv6_echo_ignore_multicast = 0;
net->ipv6.sysctl.icmpv6_echo_ignore_anycast = 0;
+ net->ipv6.sysctl.icmpv6_error_anycast_as_unicast = 0;
/* By default, rate limit error messages.
* Except for pmtu discovery, it would break it.
diff --git a/net/ipv6/icmp.c b/net/ipv6/icmp.c
index 1f53f2a..9edf1f4 100644
--- a/net/ipv6/icmp.c
+++ b/net/ipv6/icmp.c
@@ -362,9 +362,10 @@ static struct dst_entry *icmpv6_route_lookup(struct net *net,
/*
* We won't send icmp if the destination is known
- * anycast.
+ * anycast unless we need to treat anycast as unicast.
*/
- if (ipv6_anycast_destination(dst, &fl6->daddr)) {
+ if (!READ_ONCE(net->ipv6.sysctl.icmpv6_error_anycast_as_unicast) &&
+ ipv6_anycast_destination(dst, &fl6->daddr)) {
net_dbg_ratelimited("icmp6_send: acast source\n");
dst_release(dst);
return ERR_PTR(-EINVAL);
@@ -1195,6 +1196,15 @@ static struct ctl_table ipv6_icmp_table_template[] = {
.mode = 0644,
.proc_handler = proc_do_large_bitmap,
},
+ {
+ .procname = "error_anycast_as_unicast",
+ .data = &init_net.ipv6.sysctl.icmpv6_error_anycast_as_unicast,
+ .maxlen = sizeof(u8),
+ .mode = 0644,
+ .proc_handler = proc_dou8vec_minmax,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
+ },
{ },
};
@@ -1212,6 +1222,7 @@ struct ctl_table * __net_init ipv6_icmp_sysctl_init(struct net *net)
table[2].data = &net->ipv6.sysctl.icmpv6_echo_ignore_multicast;
table[3].data = &net->ipv6.sysctl.icmpv6_echo_ignore_anycast;
table[4].data = &net->ipv6.sysctl.icmpv6_ratemask_ptr;
+ table[5].data = &net->ipv6.sysctl.icmpv6_error_anycast_as_unicast;
}
return table;
}
diff --git a/net/ipv6/ipv6_sockglue.c b/net/ipv6/ipv6_sockglue.c
index 2917dd8..ae818ff 100644
--- a/net/ipv6/ipv6_sockglue.c
+++ b/net/ipv6/ipv6_sockglue.c
@@ -716,6 +716,7 @@ int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
goto done;
msg.msg_controllen = optlen;
+ msg.msg_control_is_user = false;
msg.msg_control = (void *)(opt+1);
ipc6.opt = opt;
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 35085fc..e3aec46 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -334,7 +334,6 @@ static const struct rt6_info ip6_blk_hole_entry_template = {
static void rt6_info_init(struct rt6_info *rt)
{
memset_after(rt, 0, dst);
- INIT_LIST_HEAD(&rt->dst.rt_uncached);
}
/* allocate dst with ip6_dst_ops */
diff --git a/net/ipv6/rpl.c b/net/ipv6/rpl.c
index 488aec9..d1876f19 100644
--- a/net/ipv6/rpl.c
+++ b/net/ipv6/rpl.c
@@ -32,7 +32,8 @@ static void *ipv6_rpl_segdata_pos(const struct ipv6_rpl_sr_hdr *hdr, int i)
size_t ipv6_rpl_srh_size(unsigned char n, unsigned char cmpri,
unsigned char cmpre)
{
- return (n * IPV6_PFXTAIL_LEN(cmpri)) + IPV6_PFXTAIL_LEN(cmpre);
+ return sizeof(struct ipv6_rpl_sr_hdr) + (n * IPV6_PFXTAIL_LEN(cmpri)) +
+ IPV6_PFXTAIL_LEN(cmpre);
}
void ipv6_rpl_srh_decompress(struct ipv6_rpl_sr_hdr *outhdr,
diff --git a/net/ipv6/xfrm6_policy.c b/net/ipv6/xfrm6_policy.c
index 2b493f8..eecc5e5 100644
--- a/net/ipv6/xfrm6_policy.c
+++ b/net/ipv6/xfrm6_policy.c
@@ -89,7 +89,6 @@ static int xfrm6_fill_dst(struct xfrm_dst *xdst, struct net_device *dev,
xdst->u.rt6.rt6i_gateway = rt->rt6i_gateway;
xdst->u.rt6.rt6i_dst = rt->rt6i_dst;
xdst->u.rt6.rt6i_src = rt->rt6i_src;
- INIT_LIST_HEAD(&xdst->u.rt6.dst.rt_uncached);
rt6_uncached_list_add(&xdst->u.rt6);
return 0;
@@ -121,8 +120,7 @@ static void xfrm6_dst_destroy(struct dst_entry *dst)
if (likely(xdst->u.rt6.rt6i_idev))
in6_dev_put(xdst->u.rt6.rt6i_idev);
dst_destroy_metrics_generic(dst);
- if (xdst->u.rt6.dst.rt_uncached_list)
- rt6_uncached_list_del(&xdst->u.rt6);
+ rt6_uncached_list_del(&xdst->u.rt6);
xfrm_dst_destroy(xdst);
}
diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c
index 4739156..7317e4a 100644
--- a/net/mac80211/cfg.c
+++ b/net/mac80211/cfg.c
@@ -1342,6 +1342,10 @@ static int ieee80211_start_ap(struct wiphy *wiphy, struct net_device *dev,
}
if (params->eht_cap) {
+ if (!link_conf->he_support)
+ return -EOPNOTSUPP;
+
+ link_conf->eht_support = true;
link_conf->eht_puncturing = params->punct_bitmap;
changed |= BSS_CHANGED_EHT_PUNCTURING;
diff --git a/net/mac80211/debugfs.c b/net/mac80211/debugfs.c
index dfb9f55..207f772 100644
--- a/net/mac80211/debugfs.c
+++ b/net/mac80211/debugfs.c
@@ -673,10 +673,6 @@ void debugfs_hw_add(struct ieee80211_local *local)
statsd = debugfs_create_dir("statistics", phyd);
- /* if the dir failed, don't put all the other things into the root! */
- if (!statsd)
- return;
-
#ifdef CONFIG_MAC80211_DEBUG_COUNTERS
DEBUGFS_STATS_ADD(dot11TransmittedFragmentCount);
DEBUGFS_STATS_ADD(dot11MulticastTransmittedFrameCount);
diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h
index 0bf208f..45d3e53 100644
--- a/net/mac80211/driver-ops.h
+++ b/net/mac80211/driver-ops.h
@@ -649,6 +649,21 @@ static inline void drv_flush(struct ieee80211_local *local,
trace_drv_return_void(local);
}
+static inline void drv_flush_sta(struct ieee80211_local *local,
+ struct ieee80211_sub_if_data *sdata,
+ struct sta_info *sta)
+{
+ might_sleep();
+
+ if (sdata && !check_sdata_in_driver(sdata))
+ return;
+
+ trace_drv_flush_sta(local, sdata, &sta->sta);
+ if (local->ops->flush_sta)
+ local->ops->flush_sta(&local->hw, &sdata->vif, &sta->sta);
+ trace_drv_return_void(local);
+}
+
static inline void drv_channel_switch(struct ieee80211_local *local,
struct ieee80211_sub_if_data *sdata,
struct ieee80211_channel_switch *ch_switch)
diff --git a/net/mac80211/drop.h b/net/mac80211/drop.h
new file mode 100644
index 0000000..49dc809
--- /dev/null
+++ b/net/mac80211/drop.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * mac80211 drop reason list
+ *
+ * Copyright (C) 2023 Intel Corporation
+ */
+
+#ifndef MAC80211_DROP_H
+#define MAC80211_DROP_H
+#include <net/dropreason.h>
+
+typedef unsigned int __bitwise ieee80211_rx_result;
+
+#define MAC80211_DROP_REASONS_MONITOR(R) \
+ R(RX_DROP_M_UNEXPECTED_4ADDR_FRAME) \
+ R(RX_DROP_M_BAD_BCN_KEYIDX) \
+ R(RX_DROP_M_BAD_MGMT_KEYIDX) \
+/* this line for the trailing \ - add before this */
+
+#define MAC80211_DROP_REASONS_UNUSABLE(R) \
+ R(RX_DROP_U_MIC_FAIL) \
+ R(RX_DROP_U_REPLAY) \
+ R(RX_DROP_U_BAD_MMIE) \
+/* this line for the trailing \ - add before this */
+
+/* having two enums allows for checking ieee80211_rx_result use with sparse */
+enum ___mac80211_drop_reason {
+/* if we get to the end of handlers with RX_CONTINUE this will be the reason */
+ ___RX_CONTINUE = SKB_CONSUMED,
+
+/* this never gets used as an argument to kfree_skb_reason() */
+ ___RX_QUEUED = SKB_NOT_DROPPED_YET,
+
+#define ENUM(x) ___ ## x,
+ ___RX_DROP_MONITOR = SKB_DROP_REASON_SUBSYS_MAC80211_MONITOR <<
+ SKB_DROP_REASON_SUBSYS_SHIFT,
+ MAC80211_DROP_REASONS_MONITOR(ENUM)
+
+ ___RX_DROP_UNUSABLE = SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE <<
+ SKB_DROP_REASON_SUBSYS_SHIFT,
+ MAC80211_DROP_REASONS_UNUSABLE(ENUM)
+#undef ENUM
+};
+
+enum mac80211_drop_reason {
+ RX_CONTINUE = (__force ieee80211_rx_result)___RX_CONTINUE,
+ RX_QUEUED = (__force ieee80211_rx_result)___RX_QUEUED,
+ RX_DROP_MONITOR = (__force ieee80211_rx_result)___RX_DROP_MONITOR,
+ RX_DROP_UNUSABLE = (__force ieee80211_rx_result)___RX_DROP_UNUSABLE,
+#define DEF(x) x = (__force ieee80211_rx_result)___ ## x,
+ MAC80211_DROP_REASONS_MONITOR(DEF)
+ MAC80211_DROP_REASONS_UNUSABLE(DEF)
+#undef DEF
+};
+
+#endif /* MAC80211_DROP_H */
diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h
index 9b7e184..a0a7839 100644
--- a/net/mac80211/ieee80211_i.h
+++ b/net/mac80211/ieee80211_i.h
@@ -33,6 +33,7 @@
#include "key.h"
#include "sta_info.h"
#include "debug.h"
+#include "drop.h"
extern const struct cfg80211_ops mac80211_config_ops;
@@ -170,13 +171,6 @@ struct ieee80211_tx_data {
unsigned int flags;
};
-
-typedef unsigned __bitwise ieee80211_rx_result;
-#define RX_CONTINUE ((__force ieee80211_rx_result) 0u)
-#define RX_DROP_UNUSABLE ((__force ieee80211_rx_result) 1u)
-#define RX_DROP_MONITOR ((__force ieee80211_rx_result) 2u)
-#define RX_QUEUED ((__force ieee80211_rx_result) 3u)
-
/**
* enum ieee80211_packet_rx_flags - packet RX flags
* @IEEE80211_RX_AMSDU: a-MSDU packet
diff --git a/net/mac80211/main.c b/net/mac80211/main.c
index ddf2b78..55cdfae 100644
--- a/net/mac80211/main.c
+++ b/net/mac80211/main.c
@@ -22,6 +22,7 @@
#include <linux/bitmap.h>
#include <linux/inetdevice.h>
#include <net/net_namespace.h>
+#include <net/dropreason.h>
#include <net/cfg80211.h>
#include <net/addrconf.h>
@@ -1542,6 +1543,28 @@ void ieee80211_free_hw(struct ieee80211_hw *hw)
}
EXPORT_SYMBOL(ieee80211_free_hw);
+static const char * const drop_reasons_monitor[] = {
+#define V(x) #x,
+ [0] = "RX_DROP_MONITOR",
+ MAC80211_DROP_REASONS_MONITOR(V)
+};
+
+static struct drop_reason_list drop_reason_list_monitor = {
+ .reasons = drop_reasons_monitor,
+ .n_reasons = ARRAY_SIZE(drop_reasons_monitor),
+};
+
+static const char * const drop_reasons_unusable[] = {
+ [0] = "RX_DROP_UNUSABLE",
+ MAC80211_DROP_REASONS_UNUSABLE(V)
+#undef V
+};
+
+static struct drop_reason_list drop_reason_list_unusable = {
+ .reasons = drop_reasons_unusable,
+ .n_reasons = ARRAY_SIZE(drop_reasons_unusable),
+};
+
static int __init ieee80211_init(void)
{
struct sk_buff *skb;
@@ -1559,6 +1582,11 @@ static int __init ieee80211_init(void)
if (ret)
goto err_netdev;
+ drop_reasons_register_subsys(SKB_DROP_REASON_SUBSYS_MAC80211_MONITOR,
+ &drop_reason_list_monitor);
+ drop_reasons_register_subsys(SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE,
+ &drop_reason_list_unusable);
+
return 0;
err_netdev:
rc80211_minstrel_exit();
@@ -1574,6 +1602,9 @@ static void __exit ieee80211_exit(void)
ieee80211_iface_exit();
+ drop_reasons_unregister_subsys(SKB_DROP_REASON_SUBSYS_MAC80211_MONITOR);
+ drop_reasons_unregister_subsys(SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE);
+
rcu_barrier();
}
diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c
index db3451f..58222c0 100644
--- a/net/mac80211/rx.c
+++ b/net/mac80211/rx.c
@@ -1826,7 +1826,7 @@ ieee80211_rx_h_sta_process(struct ieee80211_rx_data *rx)
cfg80211_rx_unexpected_4addr_frame(
rx->sdata->dev, sta->sta.addr,
GFP_ATOMIC);
- return RX_DROP_MONITOR;
+ return RX_DROP_M_UNEXPECTED_4ADDR_FRAME;
}
/*
* Update counter and free packet here to avoid
@@ -1961,7 +1961,7 @@ ieee80211_rx_h_decrypt(struct ieee80211_rx_data *rx)
cfg80211_rx_unprot_mlme_mgmt(rx->sdata->dev,
skb->data,
skb->len);
- return RX_DROP_MONITOR; /* unexpected BIP keyidx */
+ return RX_DROP_M_BAD_BCN_KEYIDX;
}
rx->key = ieee80211_rx_get_bigtk(rx, mmie_keyidx);
@@ -1975,7 +1975,7 @@ ieee80211_rx_h_decrypt(struct ieee80211_rx_data *rx)
if (mmie_keyidx < NUM_DEFAULT_KEYS ||
mmie_keyidx >= NUM_DEFAULT_KEYS + NUM_DEFAULT_MGMT_KEYS)
- return RX_DROP_MONITOR; /* unexpected BIP keyidx */
+ return RX_DROP_M_BAD_MGMT_KEYIDX; /* unexpected BIP keyidx */
if (rx->link_sta) {
if (ieee80211_is_group_privacy_action(skb) &&
test_sta_flag(rx->sta, WLAN_STA_MFP))
@@ -3960,7 +3960,8 @@ ieee80211_rx_h_mgmt(struct ieee80211_rx_data *rx)
}
static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx,
- struct ieee80211_rate *rate)
+ struct ieee80211_rate *rate,
+ ieee80211_rx_result reason)
{
struct ieee80211_sub_if_data *sdata;
struct ieee80211_local *local = rx->local;
@@ -4024,42 +4025,38 @@ static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx,
}
out_free_skb:
- dev_kfree_skb(skb);
+ kfree_skb_reason(skb, (__force u32)reason);
}
static void ieee80211_rx_handlers_result(struct ieee80211_rx_data *rx,
ieee80211_rx_result res)
{
- switch (res) {
- case RX_DROP_MONITOR:
- I802_DEBUG_INC(rx->sdata->local->rx_handlers_drop);
- if (rx->sta)
- rx->link_sta->rx_stats.dropped++;
- fallthrough;
- case RX_CONTINUE: {
- struct ieee80211_rate *rate = NULL;
- struct ieee80211_supported_band *sband;
- struct ieee80211_rx_status *status;
+ struct ieee80211_rx_status *status = IEEE80211_SKB_RXCB(rx->skb);
+ struct ieee80211_supported_band *sband;
+ struct ieee80211_rate *rate = NULL;
- status = IEEE80211_SKB_RXCB((rx->skb));
-
- sband = rx->local->hw.wiphy->bands[status->band];
- if (status->encoding == RX_ENC_LEGACY)
- rate = &sband->bitrates[status->rate_idx];
-
- ieee80211_rx_cooked_monitor(rx, rate);
- break;
- }
- case RX_DROP_UNUSABLE:
- I802_DEBUG_INC(rx->sdata->local->rx_handlers_drop);
- if (rx->sta)
- rx->link_sta->rx_stats.dropped++;
- dev_kfree_skb(rx->skb);
- break;
- case RX_QUEUED:
+ if (res == RX_QUEUED) {
I802_DEBUG_INC(rx->sdata->local->rx_handlers_queued);
- break;
+ return;
}
+
+ if (res != RX_CONTINUE) {
+ I802_DEBUG_INC(rx->sdata->local->rx_handlers_drop);
+ if (rx->sta)
+ rx->link_sta->rx_stats.dropped++;
+ }
+
+ if (u32_get_bits((__force u32)res, SKB_DROP_REASON_SUBSYS_MASK) ==
+ SKB_DROP_REASON_SUBSYS_MAC80211_UNUSABLE) {
+ kfree_skb_reason(rx->skb, (__force u32)res);
+ return;
+ }
+
+ sband = rx->local->hw.wiphy->bands[status->band];
+ if (status->encoding == RX_ENC_LEGACY)
+ rate = &sband->bitrates[status->rate_idx];
+
+ ieee80211_rx_cooked_monitor(rx, rate, res);
}
static void ieee80211_rx_handlers(struct ieee80211_rx_data *rx,
diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c
index 941bda9..1400512 100644
--- a/net/mac80211/sta_info.c
+++ b/net/mac80211/sta_info.c
@@ -1294,6 +1294,18 @@ static void __sta_info_destroy_part2(struct sta_info *sta)
WARN_ON_ONCE(ret);
}
+ /* Flush queues before removing keys, as that might remove them
+ * from hardware, and then depending on the offload method, any
+ * frames sitting on hardware queues might be sent out without
+ * any encryption at all.
+ */
+ if (local->ops->set_key) {
+ if (local->ops->flush_sta)
+ drv_flush_sta(local, sta->sdata, sta);
+ else
+ ieee80211_flush_queues(local, sta->sdata, false);
+ }
+
/* now keys can no longer be reached */
ieee80211_free_sta_keys(local, sta);
diff --git a/net/mac80211/status.c b/net/mac80211/status.c
index 3f9ddd7..2b13a52 100644
--- a/net/mac80211/status.c
+++ b/net/mac80211/status.c
@@ -1244,30 +1244,6 @@ void ieee80211_tx_rate_update(struct ieee80211_hw *hw,
}
EXPORT_SYMBOL(ieee80211_tx_rate_update);
-void ieee80211_tx_status_8023(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif,
- struct sk_buff *skb)
-{
- struct ieee80211_sub_if_data *sdata;
- struct ieee80211_tx_status status = {
- .skb = skb,
- .info = IEEE80211_SKB_CB(skb),
- };
- struct sta_info *sta;
-
- sdata = vif_to_sdata(vif);
-
- rcu_read_lock();
-
- if (!ieee80211_lookup_ra_sta(sdata, skb, &sta) && !IS_ERR(sta))
- status.sta = &sta->sta;
-
- ieee80211_tx_status_ext(hw, &status);
-
- rcu_read_unlock();
-}
-EXPORT_SYMBOL(ieee80211_tx_status_8023);
-
void ieee80211_report_low_ack(struct ieee80211_sta *pubsta, u32 num_packets)
{
struct sta_info *sta = container_of(pubsta, struct sta_info, sta);
diff --git a/net/mac80211/trace.h b/net/mac80211/trace.h
index e0ccf5f..de5d69f 100644
--- a/net/mac80211/trace.h
+++ b/net/mac80211/trace.h
@@ -1177,6 +1177,13 @@ TRACE_EVENT(drv_flush,
)
);
+DEFINE_EVENT(sta_event, drv_flush_sta,
+ TP_PROTO(struct ieee80211_local *local,
+ struct ieee80211_sub_if_data *sdata,
+ struct ieee80211_sta *sta),
+ TP_ARGS(local, sdata, sta)
+);
+
TRACE_EVENT(drv_channel_switch,
TP_PROTO(struct ieee80211_local *local,
struct ieee80211_sub_if_data *sdata,
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index dfe6b9c..1a33274 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -488,7 +488,7 @@ ieee80211_tx_h_unicast_ps_buf(struct ieee80211_tx_data *tx)
int ac = skb_get_queue_mapping(tx->skb);
if (ieee80211_is_mgmt(hdr->frame_control) &&
- !ieee80211_is_bufferable_mmpdu(hdr->frame_control)) {
+ !ieee80211_is_bufferable_mmpdu(tx->skb)) {
info->flags |= IEEE80211_TX_CTL_NO_PS_BUFFER;
return TX_CONTINUE;
}
@@ -1323,7 +1323,7 @@ static struct txq_info *ieee80211_get_txq(struct ieee80211_local *local,
if (!(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) &&
unlikely(!ieee80211_is_data_present(hdr->frame_control))) {
if ((!ieee80211_is_mgmt(hdr->frame_control) ||
- ieee80211_is_bufferable_mmpdu(hdr->frame_control) ||
+ ieee80211_is_bufferable_mmpdu(skb) ||
vif->type == NL80211_IFTYPE_STATION) &&
sta && sta->uploaded) {
/*
diff --git a/net/mac80211/wpa.c b/net/mac80211/wpa.c
index 20f742b..4133496 100644
--- a/net/mac80211/wpa.c
+++ b/net/mac80211/wpa.c
@@ -550,7 +550,7 @@ ieee80211_crypto_ccmp_decrypt(struct ieee80211_rx_data *rx,
if (res < 0 ||
(!res && !(status->flag & RX_FLAG_ALLOW_SAME_PN))) {
key->u.ccmp.replays++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_REPLAY;
}
if (!(status->flag & RX_FLAG_DECRYPTED)) {
@@ -564,7 +564,7 @@ ieee80211_crypto_ccmp_decrypt(struct ieee80211_rx_data *rx,
skb->data + hdrlen + IEEE80211_CCMP_HDR_LEN,
data_len,
skb->data + skb->len - mic_len))
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_MIC_FAIL;
}
memcpy(key->u.ccmp.rx_pn[queue], pn, IEEE80211_CCMP_PN_LEN);
@@ -746,7 +746,7 @@ ieee80211_crypto_gcmp_decrypt(struct ieee80211_rx_data *rx)
if (res < 0 ||
(!res && !(status->flag & RX_FLAG_ALLOW_SAME_PN))) {
key->u.gcmp.replays++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_REPLAY;
}
if (!(status->flag & RX_FLAG_DECRYPTED)) {
@@ -761,7 +761,7 @@ ieee80211_crypto_gcmp_decrypt(struct ieee80211_rx_data *rx)
data_len,
skb->data + skb->len -
IEEE80211_GCMP_MIC_LEN))
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_MIC_FAIL;
}
memcpy(key->u.gcmp.rx_pn[queue], pn, IEEE80211_GCMP_PN_LEN);
@@ -930,13 +930,13 @@ ieee80211_crypto_aes_cmac_decrypt(struct ieee80211_rx_data *rx)
(skb->data + skb->len - sizeof(*mmie));
if (mmie->element_id != WLAN_EID_MMIE ||
mmie->length != sizeof(*mmie) - 2)
- return RX_DROP_UNUSABLE; /* Invalid MMIE */
+ return RX_DROP_U_BAD_MMIE; /* Invalid MMIE */
bip_ipn_swap(ipn, mmie->sequence_number);
if (memcmp(ipn, key->u.aes_cmac.rx_pn, 6) <= 0) {
key->u.aes_cmac.replays++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_REPLAY;
}
if (!(status->flag & RX_FLAG_DECRYPTED)) {
@@ -946,7 +946,7 @@ ieee80211_crypto_aes_cmac_decrypt(struct ieee80211_rx_data *rx)
skb->data + 24, skb->len - 24, mic);
if (crypto_memneq(mic, mmie->mic, sizeof(mmie->mic))) {
key->u.aes_cmac.icverrors++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_MIC_FAIL;
}
}
@@ -986,7 +986,7 @@ ieee80211_crypto_aes_cmac_256_decrypt(struct ieee80211_rx_data *rx)
if (memcmp(ipn, key->u.aes_cmac.rx_pn, 6) <= 0) {
key->u.aes_cmac.replays++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_REPLAY;
}
if (!(status->flag & RX_FLAG_DECRYPTED)) {
@@ -996,7 +996,7 @@ ieee80211_crypto_aes_cmac_256_decrypt(struct ieee80211_rx_data *rx)
skb->data + 24, skb->len - 24, mic);
if (crypto_memneq(mic, mmie->mic, sizeof(mmie->mic))) {
key->u.aes_cmac.icverrors++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_MIC_FAIL;
}
}
@@ -1079,13 +1079,13 @@ ieee80211_crypto_aes_gmac_decrypt(struct ieee80211_rx_data *rx)
(skb->data + skb->len - sizeof(*mmie));
if (mmie->element_id != WLAN_EID_MMIE ||
mmie->length != sizeof(*mmie) - 2)
- return RX_DROP_UNUSABLE; /* Invalid MMIE */
+ return RX_DROP_U_BAD_MMIE; /* Invalid MMIE */
bip_ipn_swap(ipn, mmie->sequence_number);
if (memcmp(ipn, key->u.aes_gmac.rx_pn, 6) <= 0) {
key->u.aes_gmac.replays++;
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_REPLAY;
}
if (!(status->flag & RX_FLAG_DECRYPTED)) {
@@ -1104,7 +1104,7 @@ ieee80211_crypto_aes_gmac_decrypt(struct ieee80211_rx_data *rx)
crypto_memneq(mic, mmie->mic, sizeof(mmie->mic))) {
key->u.aes_gmac.icverrors++;
kfree(mic);
- return RX_DROP_UNUSABLE;
+ return RX_DROP_U_MIC_FAIL;
}
kfree(mic);
}
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index 355f798d..19a01b6 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -442,7 +442,6 @@ static void clear_3rdack_retransmission(struct sock *sk)
static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
bool snd_data_fin_enable,
unsigned int *size,
- unsigned int remaining,
struct mptcp_out_options *opts)
{
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
@@ -556,7 +555,6 @@ static void mptcp_write_data_fin(struct mptcp_subflow_context *subflow,
static bool mptcp_established_options_dss(struct sock *sk, struct sk_buff *skb,
bool snd_data_fin_enable,
unsigned int *size,
- unsigned int remaining,
struct mptcp_out_options *opts)
{
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
@@ -580,7 +578,6 @@ static bool mptcp_established_options_dss(struct sock *sk, struct sk_buff *skb,
opts->ext_copy = *mpext;
}
- remaining -= map_size;
dss_size = map_size;
if (skb && snd_data_fin_enable)
mptcp_write_data_fin(subflow, skb, &opts->ext_copy);
@@ -851,9 +848,9 @@ bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
}
snd_data_fin = mptcp_data_fin_enabled(msk);
- if (mptcp_established_options_mp(sk, skb, snd_data_fin, &opt_size, remaining, opts))
+ if (mptcp_established_options_mp(sk, skb, snd_data_fin, &opt_size, opts))
ret = true;
- else if (mptcp_established_options_dss(sk, skb, snd_data_fin, &opt_size, remaining, opts)) {
+ else if (mptcp_established_options_dss(sk, skb, snd_data_fin, &opt_size, opts)) {
unsigned int mp_fail_size;
ret = true;
@@ -1001,7 +998,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
clear_3rdack_retransmission(ssk);
mptcp_pm_subflow_established(msk);
} else {
- mptcp_pm_fully_established(msk, ssk, GFP_ATOMIC);
+ mptcp_pm_fully_established(msk, ssk);
}
return true;
diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 70f0ced..78c9245 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -126,7 +126,7 @@ static bool mptcp_pm_schedule_work(struct mptcp_sock *msk,
return true;
}
-void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk, gfp_t gfp)
+void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk)
{
struct mptcp_pm_data *pm = &msk->pm;
bool announce = false;
@@ -150,7 +150,7 @@ void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk,
spin_unlock_bh(&pm->lock);
if (announce)
- mptcp_event(MPTCP_EVENT_ESTABLISHED, msk, ssk, gfp);
+ mptcp_event(MPTCP_EVENT_ESTABLISHED, msk, ssk, GFP_ATOMIC);
}
void mptcp_pm_connection_closed(struct mptcp_sock *msk)
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index 1c42beb..bc343da 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -1035,8 +1035,8 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk,
lock_sock(newsk);
ssock = __mptcp_nmpc_socket(mptcp_sk(newsk));
release_sock(newsk);
- if (!ssock)
- return -EINVAL;
+ if (IS_ERR(ssock))
+ return PTR_ERR(ssock);
mptcp_info2sockaddr(&entry->addr, &addr, entry->addr.family);
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
index a02d3cb..27a275805 100644
--- a/net/mptcp/pm_userspace.c
+++ b/net/mptcp/pm_userspace.c
@@ -25,8 +25,8 @@ void mptcp_free_local_addr_list(struct mptcp_sock *msk)
}
}
-int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
- struct mptcp_pm_addr_entry *entry)
+static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
+ struct mptcp_pm_addr_entry *entry)
{
DECLARE_BITMAP(id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
struct mptcp_pm_addr_entry *match = NULL;
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e6cb367..08dc53f 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -49,18 +49,6 @@ static void __mptcp_check_send_data_fin(struct sock *sk);
DEFINE_PER_CPU(struct mptcp_delegated_action, mptcp_delegated_actions);
static struct net_device mptcp_napi_dev;
-/* If msk has an initial subflow socket, and the MP_CAPABLE handshake has not
- * completed yet or has failed, return the subflow socket.
- * Otherwise return NULL.
- */
-struct socket *__mptcp_nmpc_socket(const struct mptcp_sock *msk)
-{
- if (!msk->subflow || READ_ONCE(msk->can_ack))
- return NULL;
-
- return msk->subflow;
-}
-
/* Returns end sequence number of the receiver's advertised window */
static u64 mptcp_wnd_end(const struct mptcp_sock *msk)
{
@@ -116,6 +104,31 @@ static int __mptcp_socket_create(struct mptcp_sock *msk)
return 0;
}
+/* If the MPC handshake is not started, returns the first subflow,
+ * eventually allocating it.
+ */
+struct socket *__mptcp_nmpc_socket(struct mptcp_sock *msk)
+{
+ struct sock *sk = (struct sock *)msk;
+ int ret;
+
+ if (!((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)))
+ return ERR_PTR(-EINVAL);
+
+ if (!msk->subflow) {
+ if (msk->first)
+ return ERR_PTR(-EINVAL);
+
+ ret = __mptcp_socket_create(msk);
+ if (ret)
+ return ERR_PTR(ret);
+
+ mptcp_sockopt_sync(msk, msk->first);
+ }
+
+ return msk->subflow;
+}
+
static void mptcp_drop(struct sock *sk, struct sk_buff *skb)
{
sk_drops_add(sk, skb);
@@ -1662,13 +1675,31 @@ static void mptcp_set_nospace(struct sock *sk)
static int mptcp_disconnect(struct sock *sk, int flags);
-static int mptcp_sendmsg_fastopen(struct sock *sk, struct sock *ssk, struct msghdr *msg,
+static int mptcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg,
size_t len, int *copied_syn)
{
unsigned int saved_flags = msg->msg_flags;
struct mptcp_sock *msk = mptcp_sk(sk);
+ struct socket *ssock;
+ struct sock *ssk;
int ret;
+ /* on flags based fastopen the mptcp is supposed to create the
+ * first subflow right now. Otherwise we are in the defer_connect
+ * path, and the first subflow must be already present.
+ * Since the defer_connect flag is cleared after the first succsful
+ * fastopen attempt, no need to check for additional subflow status.
+ */
+ if (msg->msg_flags & MSG_FASTOPEN) {
+ ssock = __mptcp_nmpc_socket(msk);
+ if (IS_ERR(ssock))
+ return PTR_ERR(ssock);
+ }
+ if (!msk->first)
+ return -EINVAL;
+
+ ssk = msk->first;
+
lock_sock(ssk);
msg->msg_flags |= MSG_DONTWAIT;
msk->connect_flags = O_NONBLOCK;
@@ -1691,6 +1722,7 @@ static int mptcp_sendmsg_fastopen(struct sock *sk, struct sock *ssk, struct msgh
} else if (ret && ret != -EINPROGRESS) {
mptcp_disconnect(sk, 0);
}
+ inet_sk(sk)->defer_connect = 0;
return ret;
}
@@ -1699,7 +1731,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
{
struct mptcp_sock *msk = mptcp_sk(sk);
struct page_frag *pfrag;
- struct socket *ssock;
size_t copied = 0;
int ret = 0;
long timeo;
@@ -1709,12 +1740,10 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
lock_sock(sk);
- ssock = __mptcp_nmpc_socket(msk);
- if (unlikely(ssock && (inet_sk(ssock->sk)->defer_connect ||
- msg->msg_flags & MSG_FASTOPEN))) {
+ if (unlikely(inet_sk(sk)->defer_connect || msg->msg_flags & MSG_FASTOPEN)) {
int copied_syn = 0;
- ret = mptcp_sendmsg_fastopen(sk, ssock->sk, msg, len, &copied_syn);
+ ret = mptcp_sendmsg_fastopen(sk, msg, len, &copied_syn);
copied += copied_syn;
if (ret == -EINPROGRESS && copied_syn > 0)
goto out;
@@ -2315,7 +2344,26 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
unsigned int flags)
{
struct mptcp_sock *msk = mptcp_sk(sk);
- bool need_push, dispose_it;
+ bool dispose_it, need_push = false;
+
+ /* If the first subflow moved to a close state before accept, e.g. due
+ * to an incoming reset, mptcp either:
+ * - if either the subflow or the msk are dead, destroy the context
+ * (the subflow socket is deleted by inet_child_forget) and the msk
+ * - otherwise do nothing at the moment and take action at accept and/or
+ * listener shutdown - user-space must be able to accept() the closed
+ * socket.
+ */
+ if (msk->in_accept_queue && msk->first == ssk) {
+ if (!sock_flag(sk, SOCK_DEAD) && !sock_flag(ssk, SOCK_DEAD))
+ return;
+
+ /* ensure later check in mptcp_worker() will dispose the msk */
+ sock_set_flag(sk, SOCK_DEAD);
+ lock_sock_nested(ssk, SINGLE_DEPTH_NESTING);
+ mptcp_subflow_drop_ctx(ssk);
+ goto out_release;
+ }
dispose_it = !msk->subflow || ssk != msk->subflow->sk;
if (dispose_it)
@@ -2351,28 +2399,22 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
if (!inet_csk(ssk)->icsk_ulp_ops) {
WARN_ON_ONCE(!sock_flag(ssk, SOCK_DEAD));
kfree_rcu(subflow, rcu);
- } else if (msk->in_accept_queue && msk->first == ssk) {
- /* if the first subflow moved to a close state, e.g. due to
- * incoming reset and we reach here before inet_child_forget()
- * the TCP stack could later try to close it via
- * inet_csk_listen_stop(), or deliver it to the user space via
- * accept().
- * We can't delete the subflow - or risk a double free - nor let
- * the msk survive - or will be leaked in the non accept scenario:
- * fallback and let TCP cope with the subflow cleanup.
- */
- WARN_ON_ONCE(sock_flag(ssk, SOCK_DEAD));
- mptcp_subflow_drop_ctx(ssk);
} else {
/* otherwise tcp will dispose of the ssk and subflow ctx */
- if (ssk->sk_state == TCP_LISTEN)
+ if (ssk->sk_state == TCP_LISTEN) {
+ tcp_set_state(ssk, TCP_CLOSE);
+ mptcp_subflow_queue_clean(sk, ssk);
+ inet_csk_listen_stop(ssk);
mptcp_event_pm_listener(ssk, MPTCP_EVENT_LISTENER_CLOSED);
+ }
__tcp_close(ssk, 0);
/* close acquired an extra ref */
__sock_put(ssk);
}
+
+out_release:
release_sock(ssk);
sock_put(ssk);
@@ -2427,21 +2469,14 @@ static void __mptcp_close_subflow(struct sock *sk)
mptcp_close_ssk(sk, ssk, subflow);
}
- /* if the MPC subflow has been closed before the msk is accepted,
- * msk will never be accept-ed, close it now
- */
- if (!msk->first && msk->in_accept_queue) {
- sock_set_flag(sk, SOCK_DEAD);
- inet_sk_state_store(sk, TCP_CLOSE);
- }
}
-static bool mptcp_check_close_timeout(const struct sock *sk)
+static bool mptcp_should_close(const struct sock *sk)
{
s32 delta = tcp_jiffies32 - inet_csk(sk)->icsk_mtup.probe_timestamp;
struct mptcp_subflow_context *subflow;
- if (delta >= TCP_TIMEWAIT_LEN)
+ if (delta >= TCP_TIMEWAIT_LEN || mptcp_sk(sk)->in_accept_queue)
return true;
/* if all subflows are in closed status don't bother with additional
@@ -2649,7 +2684,7 @@ static void mptcp_worker(struct work_struct *work)
* even if it is orphaned and in FIN_WAIT2 state
*/
if (sock_flag(sk, SOCK_DEAD)) {
- if (mptcp_check_close_timeout(sk)) {
+ if (mptcp_should_close(sk)) {
inet_sk_state_store(sk, TCP_CLOSE);
mptcp_do_fastclose(sk);
}
@@ -2728,10 +2763,6 @@ static int mptcp_init_sock(struct sock *sk)
if (unlikely(!net->mib.mptcp_statistics) && !mptcp_mib_alloc(net))
return -ENOMEM;
- ret = __mptcp_socket_create(mptcp_sk(sk));
- if (ret)
- return ret;
-
set_bit(SOCK_CUSTOM_SOCKOPT, &sk->sk_socket->flags);
/* fetch the ca name; do it outside __mptcp_init_sock(), so that clone will
@@ -2895,6 +2926,14 @@ static void __mptcp_destroy_sock(struct sock *sk)
sock_put(sk);
}
+void __mptcp_unaccepted_force_close(struct sock *sk)
+{
+ sock_set_flag(sk, SOCK_DEAD);
+ inet_sk_state_store(sk, TCP_CLOSE);
+ mptcp_do_fastclose(sk);
+ __mptcp_destroy_sock(sk);
+}
+
static __poll_t mptcp_check_readable(struct mptcp_sock *msk)
{
/* Concurrent splices from sk_receive_queue into receive_queue will
@@ -2928,10 +2967,13 @@ bool __mptcp_close(struct sock *sk, long timeout)
goto cleanup;
}
- if (mptcp_check_readable(msk)) {
- /* the msk has read data, do the MPTCP equivalent of TCP reset */
+ if (mptcp_check_readable(msk) || timeout < 0) {
+ /* If the msk has read data, or the caller explicitly ask it,
+ * do the MPTCP equivalent of TCP reset, aka MPTCP fastclose
+ */
inet_sk_state_store(sk, TCP_CLOSE);
mptcp_do_fastclose(sk);
+ timeout = 0;
} else if (mptcp_close_state(sk)) {
__mptcp_wr_shutdown(sk);
}
@@ -3143,7 +3185,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
struct socket *listener;
struct sock *newsk;
- listener = __mptcp_nmpc_socket(msk);
+ listener = msk->subflow;
if (WARN_ON_ONCE(!listener)) {
*err = -EINVAL;
return NULL;
@@ -3363,7 +3405,7 @@ static int mptcp_get_port(struct sock *sk, unsigned short snum)
struct mptcp_sock *msk = mptcp_sk(sk);
struct socket *ssock;
- ssock = __mptcp_nmpc_socket(msk);
+ ssock = msk->subflow;
pr_debug("msk=%p, subflow=%p", msk, ssock);
if (WARN_ON_ONCE(!ssock))
return -EINVAL;
@@ -3551,8 +3593,8 @@ static int mptcp_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)
int err = -EINVAL;
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock)
- return -EINVAL;
+ if (IS_ERR(ssock))
+ return PTR_ERR(ssock);
mptcp_token_destroy(msk);
inet_sk_state_store(sk, TCP_SYN_SENT);
@@ -3640,8 +3682,8 @@ static int mptcp_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
lock_sock(sock->sk);
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
- err = -EINVAL;
+ if (IS_ERR(ssock)) {
+ err = PTR_ERR(ssock);
goto unlock;
}
@@ -3677,8 +3719,8 @@ static int mptcp_listen(struct socket *sock, int backlog)
lock_sock(sk);
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
- err = -EINVAL;
+ if (IS_ERR(ssock)) {
+ err = PTR_ERR(ssock);
goto unlock;
}
@@ -3709,7 +3751,10 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock,
pr_debug("msk=%p", msk);
- ssock = __mptcp_nmpc_socket(msk);
+ /* buggy applications can call accept on socket states other then LISTEN
+ * but no need to allocate the first subflow just to error out.
+ */
+ ssock = msk->subflow;
if (!ssock)
return -EINVAL;
@@ -3733,6 +3778,18 @@ static int mptcp_stream_accept(struct socket *sock, struct socket *newsock,
if (!ssk->sk_socket)
mptcp_sock_graft(ssk, newsock);
}
+
+ /* Do late cleanup for the first subflow as necessary. Also
+ * deal with bad peers not doing a complete shutdown.
+ */
+ if (msk->first &&
+ unlikely(inet_sk_state_load(msk->first) == TCP_CLOSE)) {
+ __mptcp_close_ssk(newsk, msk->first,
+ mptcp_subflow_ctx(msk->first), 0);
+ if (unlikely(list_empty(&msk->conn_list)))
+ inet_sk_state_store(newsk, TCP_CLOSE);
+ }
+
release_sock(newsk);
}
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index e1310bc..2d7b2c8 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -626,10 +626,12 @@ void mptcp_close_ssk(struct sock *sk, struct sock *ssk,
struct mptcp_subflow_context *subflow);
void __mptcp_subflow_send_ack(struct sock *ssk);
void mptcp_subflow_reset(struct sock *ssk);
+void mptcp_subflow_queue_clean(struct sock *sk, struct sock *ssk);
void mptcp_sock_graft(struct sock *sk, struct socket *parent);
-struct socket *__mptcp_nmpc_socket(const struct mptcp_sock *msk);
+struct socket *__mptcp_nmpc_socket(struct mptcp_sock *msk);
bool __mptcp_close(struct sock *sk, long timeout);
void mptcp_cancel_work(struct sock *sk);
+void __mptcp_unaccepted_force_close(struct sock *sk);
void mptcp_set_owner_r(struct sk_buff *skb, struct sock *sk);
bool mptcp_addresses_equal(const struct mptcp_addr_info *a,
@@ -782,7 +784,7 @@ bool mptcp_pm_addr_families_match(const struct sock *sk,
void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
void mptcp_pm_nl_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
void mptcp_pm_new_connection(struct mptcp_sock *msk, const struct sock *ssk, int server_side);
-void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk, gfp_t gfp);
+void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk);
bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk);
void mptcp_pm_connection_closed(struct mptcp_sock *msk);
void mptcp_pm_subflow_established(struct mptcp_sock *msk);
@@ -830,8 +832,6 @@ int mptcp_pm_remove_subflow(struct mptcp_sock *msk, const struct mptcp_rm_list *
void mptcp_pm_remove_addrs_and_subflows(struct mptcp_sock *msk,
struct list_head *rm_list);
-int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
- struct mptcp_pm_addr_entry *entry);
void mptcp_free_local_addr_list(struct mptcp_sock *msk);
int mptcp_nl_cmd_announce(struct sk_buff *skb, struct genl_info *info);
int mptcp_nl_cmd_remove(struct sk_buff *skb, struct genl_info *info);
diff --git a/net/mptcp/sockopt.c b/net/mptcp/sockopt.c
index b655ceb..d425886 100644
--- a/net/mptcp/sockopt.c
+++ b/net/mptcp/sockopt.c
@@ -301,9 +301,9 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname,
case SO_BINDTOIFINDEX:
lock_sock(sk);
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
+ if (IS_ERR(ssock)) {
release_sock(sk);
- return -EINVAL;
+ return PTR_ERR(ssock);
}
ret = sock_setsockopt(ssock, SOL_SOCKET, optname, optval, optlen);
@@ -396,9 +396,9 @@ static int mptcp_setsockopt_v6(struct mptcp_sock *msk, int optname,
case IPV6_FREEBIND:
lock_sock(sk);
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
+ if (IS_ERR(ssock)) {
release_sock(sk);
- return -EINVAL;
+ return PTR_ERR(ssock);
}
ret = tcp_setsockopt(ssock->sk, SOL_IPV6, optname, optval, optlen);
@@ -693,9 +693,9 @@ static int mptcp_setsockopt_sol_ip_set_transparent(struct mptcp_sock *msk, int o
lock_sock(sk);
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock) {
+ if (IS_ERR(ssock)) {
release_sock(sk);
- return -EINVAL;
+ return PTR_ERR(ssock);
}
issk = inet_sk(ssock->sk);
@@ -762,13 +762,15 @@ static int mptcp_setsockopt_first_sf_only(struct mptcp_sock *msk, int level, int
{
struct sock *sk = (struct sock *)msk;
struct socket *sock;
- int ret = -EINVAL;
+ int ret;
/* Limit to first subflow, before the connection establishment */
lock_sock(sk);
sock = __mptcp_nmpc_socket(msk);
- if (!sock)
+ if (IS_ERR(sock)) {
+ ret = PTR_ERR(sock);
goto unlock;
+ }
ret = tcp_setsockopt(sock->sk, level, optname, optval, optlen);
@@ -861,7 +863,7 @@ static int mptcp_getsockopt_first_sf_only(struct mptcp_sock *msk, int level, int
{
struct sock *sk = (struct sock *)msk;
struct socket *ssock;
- int ret = -EINVAL;
+ int ret;
struct sock *ssk;
lock_sock(sk);
@@ -872,8 +874,10 @@ static int mptcp_getsockopt_first_sf_only(struct mptcp_sock *msk, int level, int
}
ssock = __mptcp_nmpc_socket(msk);
- if (!ssock)
+ if (IS_ERR(ssock)) {
+ ret = PTR_ERR(ssock);
goto out;
+ }
ret = tcp_getsockopt(ssock->sk, level, optname, optval, optlen);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index f46d8f6..ba065b6 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -715,9 +715,12 @@ void mptcp_subflow_drop_ctx(struct sock *ssk)
if (!ctx)
return;
- subflow_ulp_fallback(ssk, ctx);
- if (ctx->conn)
- sock_put(ctx->conn);
+ list_del(&mptcp_subflow_ctx(ssk)->node);
+ if (inet_csk(ssk)->icsk_ulp_ops) {
+ subflow_ulp_fallback(ssk, ctx);
+ if (ctx->conn)
+ sock_put(ctx->conn);
+ }
kfree_rcu(ctx, rcu);
}
@@ -850,7 +853,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
*/
if (mp_opt.suboptions & OPTION_MPTCP_MPC_ACK) {
mptcp_subflow_fully_established(ctx, &mp_opt);
- mptcp_pm_fully_established(owner, child, GFP_ATOMIC);
+ mptcp_pm_fully_established(owner, child);
ctx->pm_notified = 1;
}
} else if (ctx->mp_join) {
@@ -1802,6 +1805,77 @@ static void subflow_state_change(struct sock *sk)
}
}
+void mptcp_subflow_queue_clean(struct sock *listener_sk, struct sock *listener_ssk)
+{
+ struct request_sock_queue *queue = &inet_csk(listener_ssk)->icsk_accept_queue;
+ struct mptcp_sock *msk, *next, *head = NULL;
+ struct request_sock *req;
+ struct sock *sk;
+
+ /* build a list of all unaccepted mptcp sockets */
+ spin_lock_bh(&queue->rskq_lock);
+ for (req = queue->rskq_accept_head; req; req = req->dl_next) {
+ struct mptcp_subflow_context *subflow;
+ struct sock *ssk = req->sk;
+
+ if (!sk_is_mptcp(ssk))
+ continue;
+
+ subflow = mptcp_subflow_ctx(ssk);
+ if (!subflow || !subflow->conn)
+ continue;
+
+ /* skip if already in list */
+ sk = subflow->conn;
+ msk = mptcp_sk(sk);
+ if (msk->dl_next || msk == head)
+ continue;
+
+ sock_hold(sk);
+ msk->dl_next = head;
+ head = msk;
+ }
+ spin_unlock_bh(&queue->rskq_lock);
+ if (!head)
+ return;
+
+ /* can't acquire the msk socket lock under the subflow one,
+ * or will cause ABBA deadlock
+ */
+ release_sock(listener_ssk);
+
+ for (msk = head; msk; msk = next) {
+ sk = (struct sock *)msk;
+
+ lock_sock_nested(sk, SINGLE_DEPTH_NESTING);
+ next = msk->dl_next;
+ msk->dl_next = NULL;
+
+ __mptcp_unaccepted_force_close(sk);
+ release_sock(sk);
+
+ /* lockdep will report a false positive ABBA deadlock
+ * between cancel_work_sync and the listener socket.
+ * The involved locks belong to different sockets WRT
+ * the existing AB chain.
+ * Using a per socket key is problematic as key
+ * deregistration requires process context and must be
+ * performed at socket disposal time, in atomic
+ * context.
+ * Just tell lockdep to consider the listener socket
+ * released here.
+ */
+ mutex_release(&listener_sk->sk_lock.dep_map, _RET_IP_);
+ mptcp_cancel_work(sk);
+ mutex_acquire(&listener_sk->sk_lock.dep_map, 0, 0, _RET_IP_);
+
+ sock_put(sk);
+ }
+
+ /* we are still under the listener msk socket lock */
+ lock_sock_nested(listener_ssk, SINGLE_DEPTH_NESTING);
+}
+
static int subflow_ulp_init(struct sock *sk)
{
struct inet_connection_sock *icsk = inet_csk(sk);
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index 6004d4b..e48ab8d 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -3447,6 +3447,64 @@ static int nft_table_validate(struct net *net, const struct nft_table *table)
return 0;
}
+int nft_setelem_validate(const struct nft_ctx *ctx, struct nft_set *set,
+ const struct nft_set_iter *iter,
+ struct nft_set_elem *elem)
+{
+ const struct nft_set_ext *ext = nft_set_elem_ext(set, elem->priv);
+ struct nft_ctx *pctx = (struct nft_ctx *)ctx;
+ const struct nft_data *data;
+ int err;
+
+ if (nft_set_ext_exists(ext, NFT_SET_EXT_FLAGS) &&
+ *nft_set_ext_flags(ext) & NFT_SET_ELEM_INTERVAL_END)
+ return 0;
+
+ data = nft_set_ext_data(ext);
+ switch (data->verdict.code) {
+ case NFT_JUMP:
+ case NFT_GOTO:
+ pctx->level++;
+ err = nft_chain_validate(ctx, data->verdict.chain);
+ if (err < 0)
+ return err;
+ pctx->level--;
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+struct nft_set_elem_catchall {
+ struct list_head list;
+ struct rcu_head rcu;
+ void *elem;
+};
+
+int nft_set_catchall_validate(const struct nft_ctx *ctx, struct nft_set *set)
+{
+ u8 genmask = nft_genmask_next(ctx->net);
+ struct nft_set_elem_catchall *catchall;
+ struct nft_set_elem elem;
+ struct nft_set_ext *ext;
+ int ret = 0;
+
+ list_for_each_entry_rcu(catchall, &set->catchall_list, list) {
+ ext = nft_set_elem_ext(set, catchall->elem);
+ if (!nft_set_elem_active(ext, genmask))
+ continue;
+
+ elem.priv = catchall->elem;
+ ret = nft_setelem_validate(ctx, set, NULL, &elem);
+ if (ret < 0)
+ return ret;
+ }
+
+ return ret;
+}
+
static struct nft_rule *nft_rule_lookup_byid(const struct net *net,
const struct nft_chain *chain,
const struct nlattr *nla);
@@ -4759,12 +4817,6 @@ static int nf_tables_newset(struct sk_buff *skb, const struct nfnl_info *info,
return err;
}
-struct nft_set_elem_catchall {
- struct list_head list;
- struct rcu_head rcu;
- void *elem;
-};
-
static void nft_set_catchall_destroy(const struct nft_ctx *ctx,
struct nft_set *set)
{
@@ -6056,7 +6108,8 @@ static int nft_add_set_elem(struct nft_ctx *ctx, struct nft_set *set,
if (err < 0)
return err;
- if (!nla[NFTA_SET_ELEM_KEY] && !(flags & NFT_SET_ELEM_CATCHALL))
+ if (((flags & NFT_SET_ELEM_CATCHALL) && nla[NFTA_SET_ELEM_KEY]) ||
+ (!(flags & NFT_SET_ELEM_CATCHALL) && !nla[NFTA_SET_ELEM_KEY]))
return -EINVAL;
if (flags != 0) {
@@ -7052,7 +7105,7 @@ static int nf_tables_newobj(struct sk_buff *skb, const struct nfnl_info *info,
}
if (nla[NFTA_OBJ_USERDATA]) {
- obj->udata = nla_memdup(nla[NFTA_OBJ_USERDATA], GFP_KERNEL);
+ obj->udata = nla_memdup(nla[NFTA_OBJ_USERDATA], GFP_KERNEL_ACCOUNT);
if (obj->udata == NULL)
goto err_userdata;
diff --git a/net/netfilter/nft_lookup.c b/net/netfilter/nft_lookup.c
index cae5a67..cecf8ab 100644
--- a/net/netfilter/nft_lookup.c
+++ b/net/netfilter/nft_lookup.c
@@ -199,37 +199,6 @@ static int nft_lookup_dump(struct sk_buff *skb,
return -1;
}
-static int nft_lookup_validate_setelem(const struct nft_ctx *ctx,
- struct nft_set *set,
- const struct nft_set_iter *iter,
- struct nft_set_elem *elem)
-{
- const struct nft_set_ext *ext = nft_set_elem_ext(set, elem->priv);
- struct nft_ctx *pctx = (struct nft_ctx *)ctx;
- const struct nft_data *data;
- int err;
-
- if (nft_set_ext_exists(ext, NFT_SET_EXT_FLAGS) &&
- *nft_set_ext_flags(ext) & NFT_SET_ELEM_INTERVAL_END)
- return 0;
-
- data = nft_set_ext_data(ext);
- switch (data->verdict.code) {
- case NFT_JUMP:
- case NFT_GOTO:
- pctx->level++;
- err = nft_chain_validate(ctx, data->verdict.chain);
- if (err < 0)
- return err;
- pctx->level--;
- break;
- default:
- break;
- }
-
- return 0;
-}
-
static int nft_lookup_validate(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nft_data **d)
@@ -245,9 +214,12 @@ static int nft_lookup_validate(const struct nft_ctx *ctx,
iter.skip = 0;
iter.count = 0;
iter.err = 0;
- iter.fn = nft_lookup_validate_setelem;
+ iter.fn = nft_setelem_validate;
priv->set->ops->walk(ctx, priv->set, &iter);
+ if (!iter.err)
+ iter.err = nft_set_catchall_validate(ctx, priv->set);
+
if (iter.err < 0)
return iter.err;
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 568f8d7..6080c0d 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -2090,18 +2090,18 @@ static unsigned int run_filter(struct sk_buff *skb,
}
static int packet_rcv_vnet(struct msghdr *msg, const struct sk_buff *skb,
- size_t *len)
+ size_t *len, int vnet_hdr_sz)
{
- struct virtio_net_hdr vnet_hdr;
+ struct virtio_net_hdr_mrg_rxbuf vnet_hdr = { .num_buffers = 0 };
- if (*len < sizeof(vnet_hdr))
+ if (*len < vnet_hdr_sz)
return -EINVAL;
- *len -= sizeof(vnet_hdr);
+ *len -= vnet_hdr_sz;
- if (virtio_net_hdr_from_skb(skb, &vnet_hdr, vio_le(), true, 0))
+ if (virtio_net_hdr_from_skb(skb, (struct virtio_net_hdr *)&vnet_hdr, vio_le(), true, 0))
return -EINVAL;
- return memcpy_to_msg(msg, (void *)&vnet_hdr, sizeof(vnet_hdr));
+ return memcpy_to_msg(msg, (void *)&vnet_hdr, vnet_hdr_sz);
}
/*
@@ -2250,7 +2250,7 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
__u32 ts_status;
bool is_drop_n_account = false;
unsigned int slot_id = 0;
- bool do_vnet = false;
+ int vnet_hdr_sz = 0;
/* struct tpacket{2,3}_hdr is aligned to a multiple of TPACKET_ALIGNMENT.
* We may add members to them until current aligned size without forcing
@@ -2308,10 +2308,9 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
netoff = TPACKET_ALIGN(po->tp_hdrlen +
(maclen < 16 ? 16 : maclen)) +
po->tp_reserve;
- if (packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR)) {
- netoff += sizeof(struct virtio_net_hdr);
- do_vnet = true;
- }
+ vnet_hdr_sz = READ_ONCE(po->vnet_hdr_sz);
+ if (vnet_hdr_sz)
+ netoff += vnet_hdr_sz;
macoff = netoff - maclen;
}
if (netoff > USHRT_MAX) {
@@ -2337,7 +2336,7 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
snaplen = po->rx_ring.frame_size - macoff;
if ((int)snaplen < 0) {
snaplen = 0;
- do_vnet = false;
+ vnet_hdr_sz = 0;
}
}
} else if (unlikely(macoff + snaplen >
@@ -2351,7 +2350,7 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
if (unlikely((int)snaplen < 0)) {
snaplen = 0;
macoff = GET_PBDQC_FROM_RB(&po->rx_ring)->max_frame_len;
- do_vnet = false;
+ vnet_hdr_sz = 0;
}
}
spin_lock(&sk->sk_receive_queue.lock);
@@ -2367,7 +2366,7 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
__set_bit(slot_id, po->rx_ring.rx_owner_map);
}
- if (do_vnet &&
+ if (vnet_hdr_sz &&
virtio_net_hdr_from_skb(skb, h.raw + macoff -
sizeof(struct virtio_net_hdr),
vio_le(), true, 0)) {
@@ -2551,16 +2550,26 @@ static int __packet_snd_vnet_parse(struct virtio_net_hdr *vnet_hdr, size_t len)
}
static int packet_snd_vnet_parse(struct msghdr *msg, size_t *len,
- struct virtio_net_hdr *vnet_hdr)
+ struct virtio_net_hdr *vnet_hdr, int vnet_hdr_sz)
{
- if (*len < sizeof(*vnet_hdr))
+ int ret;
+
+ if (*len < vnet_hdr_sz)
return -EINVAL;
- *len -= sizeof(*vnet_hdr);
+ *len -= vnet_hdr_sz;
if (!copy_from_iter_full(vnet_hdr, sizeof(*vnet_hdr), &msg->msg_iter))
return -EFAULT;
- return __packet_snd_vnet_parse(vnet_hdr, *len);
+ ret = __packet_snd_vnet_parse(vnet_hdr, *len);
+ if (ret)
+ return ret;
+
+ /* move iter to point to the start of mac header */
+ if (vnet_hdr_sz != sizeof(struct virtio_net_hdr))
+ iov_iter_advance(&msg->msg_iter, vnet_hdr_sz - sizeof(struct virtio_net_hdr));
+
+ return 0;
}
static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb,
@@ -2722,6 +2731,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
void *ph;
DECLARE_SOCKADDR(struct sockaddr_ll *, saddr, msg->msg_name);
bool need_wait = !(msg->msg_flags & MSG_DONTWAIT);
+ int vnet_hdr_sz = READ_ONCE(po->vnet_hdr_sz);
unsigned char *addr = NULL;
int tp_len, size_max;
void *data;
@@ -2779,8 +2789,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
size_max = po->tx_ring.frame_size
- (po->tp_hdrlen - sizeof(struct sockaddr_ll));
- if ((size_max > dev->mtu + reserve + VLAN_HLEN) &&
- !packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR))
+ if ((size_max > dev->mtu + reserve + VLAN_HLEN) && !vnet_hdr_sz)
size_max = dev->mtu + reserve + VLAN_HLEN;
reinit_completion(&po->skb_completion);
@@ -2809,10 +2818,10 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
status = TP_STATUS_SEND_REQUEST;
hlen = LL_RESERVED_SPACE(dev);
tlen = dev->needed_tailroom;
- if (packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR)) {
+ if (vnet_hdr_sz) {
vnet_hdr = data;
- data += sizeof(*vnet_hdr);
- tp_len -= sizeof(*vnet_hdr);
+ data += vnet_hdr_sz;
+ tp_len -= vnet_hdr_sz;
if (tp_len < 0 ||
__packet_snd_vnet_parse(vnet_hdr, tp_len)) {
tp_len = -EINVAL;
@@ -2837,7 +2846,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
addr, hlen, copylen, &sockc);
if (likely(tp_len >= 0) &&
tp_len > dev->mtu + reserve &&
- !packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR) &&
+ !vnet_hdr_sz &&
!packet_extra_vlan_len_allowed(dev, skb))
tp_len = -EMSGSIZE;
@@ -2856,7 +2865,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
}
}
- if (packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR)) {
+ if (vnet_hdr_sz) {
if (virtio_net_hdr_to_skb(skb, vnet_hdr, vio_le())) {
tp_len = -EINVAL;
goto tpacket_error;
@@ -2946,7 +2955,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
struct virtio_net_hdr vnet_hdr = { 0 };
int offset = 0;
struct packet_sock *po = pkt_sk(sk);
- bool has_vnet_hdr = false;
+ int vnet_hdr_sz = READ_ONCE(po->vnet_hdr_sz);
int hlen, tlen, linear;
int extra_len = 0;
@@ -2990,11 +2999,10 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
if (sock->type == SOCK_RAW)
reserve = dev->hard_header_len;
- if (packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR)) {
- err = packet_snd_vnet_parse(msg, &len, &vnet_hdr);
+ if (vnet_hdr_sz) {
+ err = packet_snd_vnet_parse(msg, &len, &vnet_hdr, vnet_hdr_sz);
if (err)
goto out_unlock;
- has_vnet_hdr = true;
}
if (unlikely(sock_flag(sk, SOCK_NOFCS))) {
@@ -3064,11 +3072,11 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
packet_parse_headers(skb, sock);
- if (has_vnet_hdr) {
+ if (vnet_hdr_sz) {
err = virtio_net_hdr_to_skb(skb, &vnet_hdr, vio_le());
if (err)
goto out_free;
- len += sizeof(vnet_hdr);
+ len += vnet_hdr_sz;
virtio_net_hdr_set_proto(skb, &vnet_hdr);
}
@@ -3408,7 +3416,7 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
struct sock *sk = sock->sk;
struct sk_buff *skb;
int copied, err;
- int vnet_hdr_len = 0;
+ int vnet_hdr_len = READ_ONCE(pkt_sk(sk)->vnet_hdr_sz);
unsigned int origlen = 0;
err = -EINVAL;
@@ -3449,11 +3457,10 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
packet_rcv_try_clear_pressure(pkt_sk(sk));
- if (packet_sock_flag(pkt_sk(sk), PACKET_SOCK_HAS_VNET_HDR)) {
- err = packet_rcv_vnet(msg, skb, &len);
+ if (vnet_hdr_len) {
+ err = packet_rcv_vnet(msg, skb, &len, vnet_hdr_len);
if (err)
goto out_free;
- vnet_hdr_len = sizeof(struct virtio_net_hdr);
}
/* You lose any data beyond the buffer you gave. If it worries
@@ -3915,8 +3922,9 @@ packet_setsockopt(struct socket *sock, int level, int optname, sockptr_t optval,
return 0;
}
case PACKET_VNET_HDR:
+ case PACKET_VNET_HDR_SZ:
{
- int val;
+ int val, hdr_len;
if (sock->type != SOCK_RAW)
return -EINVAL;
@@ -3925,11 +3933,19 @@ packet_setsockopt(struct socket *sock, int level, int optname, sockptr_t optval,
if (copy_from_sockptr(&val, optval, sizeof(val)))
return -EFAULT;
+ if (optname == PACKET_VNET_HDR_SZ) {
+ if (val && val != sizeof(struct virtio_net_hdr) &&
+ val != sizeof(struct virtio_net_hdr_mrg_rxbuf))
+ return -EINVAL;
+ hdr_len = val;
+ } else {
+ hdr_len = val ? sizeof(struct virtio_net_hdr) : 0;
+ }
lock_sock(sk);
if (po->rx_ring.pg_vec || po->tx_ring.pg_vec) {
ret = -EBUSY;
} else {
- packet_sock_flag_set(po, PACKET_SOCK_HAS_VNET_HDR, val);
+ WRITE_ONCE(po->vnet_hdr_sz, hdr_len);
ret = 0;
}
release_sock(sk);
@@ -4062,7 +4078,10 @@ static int packet_getsockopt(struct socket *sock, int level, int optname,
val = packet_sock_flag(po, PACKET_SOCK_ORIGDEV);
break;
case PACKET_VNET_HDR:
- val = packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR);
+ val = !!READ_ONCE(po->vnet_hdr_sz);
+ break;
+ case PACKET_VNET_HDR_SZ:
+ val = READ_ONCE(po->vnet_hdr_sz);
break;
case PACKET_VERSION:
val = po->tp_version;
diff --git a/net/packet/diag.c b/net/packet/diag.c
index de4ced5..d0c4eda 100644
--- a/net/packet/diag.c
+++ b/net/packet/diag.c
@@ -27,7 +27,7 @@ static int pdiag_put_info(const struct packet_sock *po, struct sk_buff *nlskb)
pinfo.pdi_flags |= PDI_AUXDATA;
if (packet_sock_flag(po, PACKET_SOCK_ORIGDEV))
pinfo.pdi_flags |= PDI_ORIGDEV;
- if (packet_sock_flag(po, PACKET_SOCK_HAS_VNET_HDR))
+ if (READ_ONCE(po->vnet_hdr_sz))
pinfo.pdi_flags |= PDI_VNETHDR;
if (packet_sock_flag(po, PACKET_SOCK_TP_LOSS))
pinfo.pdi_flags |= PDI_LOSS;
diff --git a/net/packet/internal.h b/net/packet/internal.h
index 27930f6..63f4865 100644
--- a/net/packet/internal.h
+++ b/net/packet/internal.h
@@ -118,6 +118,7 @@ struct packet_sock {
struct mutex pg_vec_lock;
unsigned long flags;
int ifindex; /* bound device */
+ u8 vnet_hdr_sz;
__be16 num;
struct packet_rollover *rollover;
struct packet_mclist *mclist;
@@ -139,7 +140,6 @@ enum packet_sock_flags {
PACKET_SOCK_AUXDATA,
PACKET_SOCK_TX_HAS_OFF,
PACKET_SOCK_TP_LOSS,
- PACKET_SOCK_HAS_VNET_HDR,
PACKET_SOCK_RUNNING,
PACKET_SOCK_PRESSURE,
PACKET_SOCK_QDISC_BYPASS,
diff --git a/net/sched/act_csum.c b/net/sched/act_csum.c
index 95e9304..8ed2850 100644
--- a/net/sched/act_csum.c
+++ b/net/sched/act_csum.c
@@ -376,8 +376,7 @@ static int tcf_csum_sctp(struct sk_buff *skb, unsigned int ihl,
sctph->checksum = sctp_compute_cksum(skb,
skb_network_offset(skb) + ihl);
- skb->ip_summed = CHECKSUM_NONE;
- skb->csum_not_inet = 0;
+ skb_reset_csum_not_inet(skb);
return 1;
}
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 2a6b6be..35785a3 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -3235,6 +3235,9 @@ int tcf_exts_init_ex(struct tcf_exts *exts, struct net *net, int action,
err_miss_alloc:
tcf_exts_destroy(exts);
+#ifdef CONFIG_NET_CLS_ACT
+ exts->actions = NULL;
+#endif
return err;
}
EXPORT_SYMBOL(tcf_exts_init_ex);
diff --git a/net/sched/sch_mqprio.c b/net/sched/sch_mqprio.c
index fdd6a65..dc5a0ff 100644
--- a/net/sched/sch_mqprio.c
+++ b/net/sched/sch_mqprio.c
@@ -5,6 +5,7 @@
* Copyright (c) 2010 John Fastabend <john.r.fastabend@intel.com>
*/
+#include <linux/ethtool_netlink.h>
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/kernel.h>
@@ -27,15 +28,19 @@ struct mqprio_sched {
u32 flags;
u64 min_rate[TC_QOPT_MAX_QUEUE];
u64 max_rate[TC_QOPT_MAX_QUEUE];
+ u32 fp[TC_QOPT_MAX_QUEUE];
};
static int mqprio_enable_offload(struct Qdisc *sch,
const struct tc_mqprio_qopt *qopt,
struct netlink_ext_ack *extack)
{
- struct tc_mqprio_qopt_offload mqprio = {.qopt = *qopt};
struct mqprio_sched *priv = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
+ struct tc_mqprio_qopt_offload mqprio = {
+ .qopt = *qopt,
+ .extack = extack,
+ };
int err, i;
switch (priv->mode) {
@@ -60,6 +65,8 @@ static int mqprio_enable_offload(struct Qdisc *sch,
return -EINVAL;
}
+ mqprio_fp_to_offload(priv->fp, &mqprio);
+
err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_MQPRIO,
&mqprio);
if (err)
@@ -133,48 +140,131 @@ static int mqprio_parse_opt(struct net_device *dev, struct tc_mqprio_qopt *qopt,
/* If ndo_setup_tc is not present then hardware doesn't support offload
* and we should return an error.
*/
- if (qopt->hw && !dev->netdev_ops->ndo_setup_tc)
+ if (qopt->hw && !dev->netdev_ops->ndo_setup_tc) {
+ NL_SET_ERR_MSG(extack,
+ "Device does not support hardware offload");
return -EINVAL;
+ }
return 0;
}
+static const struct
+nla_policy mqprio_tc_entry_policy[TCA_MQPRIO_TC_ENTRY_MAX + 1] = {
+ [TCA_MQPRIO_TC_ENTRY_INDEX] = NLA_POLICY_MAX(NLA_U32,
+ TC_QOPT_MAX_QUEUE),
+ [TCA_MQPRIO_TC_ENTRY_FP] = NLA_POLICY_RANGE(NLA_U32,
+ TC_FP_EXPRESS,
+ TC_FP_PREEMPTIBLE),
+};
+
static const struct nla_policy mqprio_policy[TCA_MQPRIO_MAX + 1] = {
[TCA_MQPRIO_MODE] = { .len = sizeof(u16) },
[TCA_MQPRIO_SHAPER] = { .len = sizeof(u16) },
[TCA_MQPRIO_MIN_RATE64] = { .type = NLA_NESTED },
[TCA_MQPRIO_MAX_RATE64] = { .type = NLA_NESTED },
+ [TCA_MQPRIO_TC_ENTRY] = { .type = NLA_NESTED },
};
-static int parse_attr(struct nlattr *tb[], int maxtype, struct nlattr *nla,
- const struct nla_policy *policy, int len)
+static int mqprio_parse_tc_entry(u32 fp[TC_QOPT_MAX_QUEUE],
+ struct nlattr *opt,
+ unsigned long *seen_tcs,
+ struct netlink_ext_ack *extack)
{
- int nested_len = nla_len(nla) - NLA_ALIGN(len);
+ struct nlattr *tb[TCA_MQPRIO_TC_ENTRY_MAX + 1];
+ int err, tc;
- if (nested_len >= nla_attr_size(0))
- return nla_parse_deprecated(tb, maxtype,
- nla_data(nla) + NLA_ALIGN(len),
- nested_len, policy, NULL);
-
- memset(tb, 0, sizeof(struct nlattr *) * (maxtype + 1));
- return 0;
-}
-
-static int mqprio_parse_nlattr(struct Qdisc *sch, struct tc_mqprio_qopt *qopt,
- struct nlattr *opt)
-{
- struct mqprio_sched *priv = qdisc_priv(sch);
- struct nlattr *tb[TCA_MQPRIO_MAX + 1];
- struct nlattr *attr;
- int i, rem, err;
-
- err = parse_attr(tb, TCA_MQPRIO_MAX, opt, mqprio_policy,
- sizeof(*qopt));
+ err = nla_parse_nested(tb, TCA_MQPRIO_TC_ENTRY_MAX, opt,
+ mqprio_tc_entry_policy, extack);
if (err < 0)
return err;
- if (!qopt->hw)
+ if (NL_REQ_ATTR_CHECK(extack, opt, tb, TCA_MQPRIO_TC_ENTRY_INDEX)) {
+ NL_SET_ERR_MSG(extack, "TC entry index missing");
return -EINVAL;
+ }
+
+ tc = nla_get_u32(tb[TCA_MQPRIO_TC_ENTRY_INDEX]);
+ if (*seen_tcs & BIT(tc)) {
+ NL_SET_ERR_MSG_ATTR(extack, tb[TCA_MQPRIO_TC_ENTRY_INDEX],
+ "Duplicate tc entry");
+ return -EINVAL;
+ }
+
+ *seen_tcs |= BIT(tc);
+
+ if (tb[TCA_MQPRIO_TC_ENTRY_FP])
+ fp[tc] = nla_get_u32(tb[TCA_MQPRIO_TC_ENTRY_FP]);
+
+ return 0;
+}
+
+static int mqprio_parse_tc_entries(struct Qdisc *sch, struct nlattr *nlattr_opt,
+ int nlattr_opt_len,
+ struct netlink_ext_ack *extack)
+{
+ struct mqprio_sched *priv = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
+ bool have_preemption = false;
+ unsigned long seen_tcs = 0;
+ u32 fp[TC_QOPT_MAX_QUEUE];
+ struct nlattr *n;
+ int tc, rem;
+ int err = 0;
+
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ fp[tc] = priv->fp[tc];
+
+ nla_for_each_attr(n, nlattr_opt, nlattr_opt_len, rem) {
+ if (nla_type(n) != TCA_MQPRIO_TC_ENTRY)
+ continue;
+
+ err = mqprio_parse_tc_entry(fp, n, &seen_tcs, extack);
+ if (err)
+ goto out;
+ }
+
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++) {
+ priv->fp[tc] = fp[tc];
+ if (fp[tc] == TC_FP_PREEMPTIBLE)
+ have_preemption = true;
+ }
+
+ if (have_preemption && !ethtool_dev_mm_supported(dev)) {
+ NL_SET_ERR_MSG(extack, "Device does not support preemption");
+ return -EOPNOTSUPP;
+ }
+out:
+ return err;
+}
+
+/* Parse the other netlink attributes that represent the payload of
+ * TCA_OPTIONS, which are appended right after struct tc_mqprio_qopt.
+ */
+static int mqprio_parse_nlattr(struct Qdisc *sch, struct tc_mqprio_qopt *qopt,
+ struct nlattr *opt,
+ struct netlink_ext_ack *extack)
+{
+ struct nlattr *nlattr_opt = nla_data(opt) + NLA_ALIGN(sizeof(*qopt));
+ int nlattr_opt_len = nla_len(opt) - NLA_ALIGN(sizeof(*qopt));
+ struct mqprio_sched *priv = qdisc_priv(sch);
+ struct nlattr *tb[TCA_MQPRIO_MAX + 1] = {};
+ struct nlattr *attr;
+ int i, rem, err;
+
+ if (nlattr_opt_len >= nla_attr_size(0)) {
+ err = nla_parse_deprecated(tb, TCA_MQPRIO_MAX, nlattr_opt,
+ nlattr_opt_len, mqprio_policy,
+ NULL);
+ if (err < 0)
+ return err;
+ }
+
+ if (!qopt->hw) {
+ NL_SET_ERR_MSG(extack,
+ "mqprio TCA_OPTIONS can only contain netlink attributes in hardware mode");
+ return -EINVAL;
+ }
if (tb[TCA_MQPRIO_MODE]) {
priv->flags |= TC_MQPRIO_F_MODE;
@@ -187,13 +277,19 @@ static int mqprio_parse_nlattr(struct Qdisc *sch, struct tc_mqprio_qopt *qopt,
}
if (tb[TCA_MQPRIO_MIN_RATE64]) {
- if (priv->shaper != TC_MQPRIO_SHAPER_BW_RATE)
+ if (priv->shaper != TC_MQPRIO_SHAPER_BW_RATE) {
+ NL_SET_ERR_MSG_ATTR(extack, tb[TCA_MQPRIO_MIN_RATE64],
+ "min_rate accepted only when shaper is in bw_rlimit mode");
return -EINVAL;
+ }
i = 0;
nla_for_each_nested(attr, tb[TCA_MQPRIO_MIN_RATE64],
rem) {
- if (nla_type(attr) != TCA_MQPRIO_MIN_RATE64)
+ if (nla_type(attr) != TCA_MQPRIO_MIN_RATE64) {
+ NL_SET_ERR_MSG_ATTR(extack, attr,
+ "Attribute type expected to be TCA_MQPRIO_MIN_RATE64");
return -EINVAL;
+ }
if (i >= qopt->num_tc)
break;
priv->min_rate[i] = nla_get_u64(attr);
@@ -203,13 +299,19 @@ static int mqprio_parse_nlattr(struct Qdisc *sch, struct tc_mqprio_qopt *qopt,
}
if (tb[TCA_MQPRIO_MAX_RATE64]) {
- if (priv->shaper != TC_MQPRIO_SHAPER_BW_RATE)
+ if (priv->shaper != TC_MQPRIO_SHAPER_BW_RATE) {
+ NL_SET_ERR_MSG_ATTR(extack, tb[TCA_MQPRIO_MAX_RATE64],
+ "max_rate accepted only when shaper is in bw_rlimit mode");
return -EINVAL;
+ }
i = 0;
nla_for_each_nested(attr, tb[TCA_MQPRIO_MAX_RATE64],
rem) {
- if (nla_type(attr) != TCA_MQPRIO_MAX_RATE64)
+ if (nla_type(attr) != TCA_MQPRIO_MAX_RATE64) {
+ NL_SET_ERR_MSG_ATTR(extack, attr,
+ "Attribute type expected to be TCA_MQPRIO_MAX_RATE64");
return -EINVAL;
+ }
if (i >= qopt->num_tc)
break;
priv->max_rate[i] = nla_get_u64(attr);
@@ -218,6 +320,13 @@ static int mqprio_parse_nlattr(struct Qdisc *sch, struct tc_mqprio_qopt *qopt,
priv->flags |= TC_MQPRIO_F_MAX_RATE;
}
+ if (tb[TCA_MQPRIO_TC_ENTRY]) {
+ err = mqprio_parse_tc_entries(sch, nlattr_opt, nlattr_opt_len,
+ extack);
+ if (err)
+ return err;
+ }
+
return 0;
}
@@ -231,7 +340,7 @@ static int mqprio_init(struct Qdisc *sch, struct nlattr *opt,
int i, err = -EOPNOTSUPP;
struct tc_mqprio_qopt *qopt = NULL;
struct tc_mqprio_caps caps;
- int len;
+ int len, tc;
BUILD_BUG_ON(TC_MAX_QUEUE != TC_QOPT_MAX_QUEUE);
BUILD_BUG_ON(TC_BITMASK != TC_QOPT_BITMASK);
@@ -249,6 +358,9 @@ static int mqprio_init(struct Qdisc *sch, struct nlattr *opt,
if (!opt || nla_len(opt) < sizeof(*qopt))
return -EINVAL;
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ priv->fp[tc] = TC_FP_EXPRESS;
+
qdisc_offload_query_caps(dev, TC_SETUP_QDISC_MQPRIO,
&caps, sizeof(caps));
@@ -258,7 +370,7 @@ static int mqprio_init(struct Qdisc *sch, struct nlattr *opt,
len = nla_len(opt) - NLA_ALIGN(sizeof(*qopt));
if (len > 0) {
- err = mqprio_parse_nlattr(sch, qopt, opt);
+ err = mqprio_parse_nlattr(sch, qopt, opt, extack);
if (err)
return err;
}
@@ -399,6 +511,33 @@ static int dump_rates(struct mqprio_sched *priv,
return -1;
}
+static int mqprio_dump_tc_entries(struct mqprio_sched *priv,
+ struct sk_buff *skb)
+{
+ struct nlattr *n;
+ int tc;
+
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++) {
+ n = nla_nest_start(skb, TCA_MQPRIO_TC_ENTRY);
+ if (!n)
+ return -EMSGSIZE;
+
+ if (nla_put_u32(skb, TCA_MQPRIO_TC_ENTRY_INDEX, tc))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_MQPRIO_TC_ENTRY_FP, priv->fp[tc]))
+ goto nla_put_failure;
+
+ nla_nest_end(skb, n);
+ }
+
+ return 0;
+
+nla_put_failure:
+ nla_nest_cancel(skb, n);
+ return -EMSGSIZE;
+}
+
static int mqprio_dump(struct Qdisc *sch, struct sk_buff *skb)
{
struct net_device *dev = qdisc_dev(sch);
@@ -449,6 +588,9 @@ static int mqprio_dump(struct Qdisc *sch, struct sk_buff *skb)
(dump_rates(priv, &opt, skb) != 0))
goto nla_put_failure;
+ if (mqprio_dump_tc_entries(priv, skb))
+ goto nla_put_failure;
+
return nla_nest_end(skb, nla);
nla_put_failure:
nlmsg_trim(skb, nla);
diff --git a/net/sched/sch_mqprio_lib.c b/net/sched/sch_mqprio_lib.c
index c58a533..83b3793 100644
--- a/net/sched/sch_mqprio_lib.c
+++ b/net/sched/sch_mqprio_lib.c
@@ -114,4 +114,18 @@ void mqprio_qopt_reconstruct(struct net_device *dev, struct tc_mqprio_qopt *qopt
}
EXPORT_SYMBOL_GPL(mqprio_qopt_reconstruct);
+void mqprio_fp_to_offload(u32 fp[TC_QOPT_MAX_QUEUE],
+ struct tc_mqprio_qopt_offload *mqprio)
+{
+ unsigned long preemptible_tcs = 0;
+ int tc;
+
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ if (fp[tc] == TC_FP_PREEMPTIBLE)
+ preemptible_tcs |= BIT(tc);
+
+ mqprio->preemptible_tcs = preemptible_tcs;
+}
+EXPORT_SYMBOL_GPL(mqprio_fp_to_offload);
+
MODULE_LICENSE("GPL");
diff --git a/net/sched/sch_mqprio_lib.h b/net/sched/sch_mqprio_lib.h
index 63f725a..079f597 100644
--- a/net/sched/sch_mqprio_lib.h
+++ b/net/sched/sch_mqprio_lib.h
@@ -14,5 +14,7 @@ int mqprio_validate_qopt(struct net_device *dev, struct tc_mqprio_qopt *qopt,
struct netlink_ext_ack *extack);
void mqprio_qopt_reconstruct(struct net_device *dev,
struct tc_mqprio_qopt *qopt);
+void mqprio_fp_to_offload(u32 fp[TC_QOPT_MAX_QUEUE],
+ struct tc_mqprio_qopt_offload *mqprio);
#endif
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index cf5ebe4..02098a0 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -421,15 +421,16 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
} else
weight = 1;
- if (tb[TCA_QFQ_LMAX]) {
+ if (tb[TCA_QFQ_LMAX])
lmax = nla_get_u32(tb[TCA_QFQ_LMAX]);
- if (lmax < QFQ_MIN_LMAX || lmax > (1UL << QFQ_MTU_SHIFT)) {
- pr_notice("qfq: invalid max length %u\n", lmax);
- return -EINVAL;
- }
- } else
+ else
lmax = psched_mtu(qdisc_dev(sch));
+ if (lmax < QFQ_MIN_LMAX || lmax > (1UL << QFQ_MTU_SHIFT)) {
+ pr_notice("qfq: invalid max length %u\n", lmax);
+ return -EINVAL;
+ }
+
inv_w = ONE_FP / weight;
weight = ONE_FP / inv_w;
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 1f46986..76db9a1 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -7,6 +7,7 @@
*/
#include <linux/ethtool.h>
+#include <linux/ethtool_netlink.h>
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/kernel.h>
@@ -96,6 +97,7 @@ struct taprio_sched {
struct list_head taprio_list;
int cur_txq[TC_MAX_QUEUE];
u32 max_sdu[TC_MAX_QUEUE]; /* save info from the user */
+ u32 fp[TC_QOPT_MAX_QUEUE]; /* only for dump and offloading */
u32 txtime_delay;
};
@@ -1002,6 +1004,9 @@ static const struct nla_policy entry_policy[TCA_TAPRIO_SCHED_ENTRY_MAX + 1] = {
static const struct nla_policy taprio_tc_policy[TCA_TAPRIO_TC_ENTRY_MAX + 1] = {
[TCA_TAPRIO_TC_ENTRY_INDEX] = { .type = NLA_U32 },
[TCA_TAPRIO_TC_ENTRY_MAX_SDU] = { .type = NLA_U32 },
+ [TCA_TAPRIO_TC_ENTRY_FP] = NLA_POLICY_RANGE(NLA_U32,
+ TC_FP_EXPRESS,
+ TC_FP_PREEMPTIBLE),
};
static const struct nla_policy taprio_policy[TCA_TAPRIO_ATTR_MAX + 1] = {
@@ -1520,22 +1525,31 @@ static int taprio_enable_offload(struct net_device *dev,
return -ENOMEM;
}
offload->enable = 1;
+ offload->extack = extack;
mqprio_qopt_reconstruct(dev, &offload->mqprio.qopt);
+ offload->mqprio.extack = extack;
taprio_sched_to_offload(dev, sched, offload, &caps);
+ mqprio_fp_to_offload(q->fp, &offload->mqprio);
for (tc = 0; tc < TC_MAX_QUEUE; tc++)
offload->max_sdu[tc] = q->max_sdu[tc];
err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, offload);
if (err < 0) {
- NL_SET_ERR_MSG(extack,
- "Device failed to setup taprio offload");
+ NL_SET_ERR_MSG_WEAK(extack,
+ "Device failed to setup taprio offload");
goto done;
}
q->offloaded = true;
done:
+ /* The offload structure may linger around via a reference taken by the
+ * device driver, so clear up the netlink extack pointer so that the
+ * driver isn't tempted to dereference data which stopped being valid
+ */
+ offload->extack = NULL;
+ offload->mqprio.extack = NULL;
taprio_offload_free(offload);
return err;
@@ -1663,13 +1677,14 @@ static int taprio_parse_clockid(struct Qdisc *sch, struct nlattr **tb,
static int taprio_parse_tc_entry(struct Qdisc *sch,
struct nlattr *opt,
u32 max_sdu[TC_QOPT_MAX_QUEUE],
+ u32 fp[TC_QOPT_MAX_QUEUE],
unsigned long *seen_tcs,
struct netlink_ext_ack *extack)
{
struct nlattr *tb[TCA_TAPRIO_TC_ENTRY_MAX + 1] = { };
struct net_device *dev = qdisc_dev(sch);
- u32 val = 0;
int err, tc;
+ u32 val;
err = nla_parse_nested(tb, TCA_TAPRIO_TC_ENTRY_MAX, opt,
taprio_tc_policy, extack);
@@ -1694,15 +1709,18 @@ static int taprio_parse_tc_entry(struct Qdisc *sch,
*seen_tcs |= BIT(tc);
- if (tb[TCA_TAPRIO_TC_ENTRY_MAX_SDU])
+ if (tb[TCA_TAPRIO_TC_ENTRY_MAX_SDU]) {
val = nla_get_u32(tb[TCA_TAPRIO_TC_ENTRY_MAX_SDU]);
+ if (val > dev->max_mtu) {
+ NL_SET_ERR_MSG_MOD(extack, "TC max SDU exceeds device max MTU");
+ return -ERANGE;
+ }
- if (val > dev->max_mtu) {
- NL_SET_ERR_MSG_MOD(extack, "TC max SDU exceeds device max MTU");
- return -ERANGE;
+ max_sdu[tc] = val;
}
- max_sdu[tc] = val;
+ if (tb[TCA_TAPRIO_TC_ENTRY_FP])
+ fp[tc] = nla_get_u32(tb[TCA_TAPRIO_TC_ENTRY_FP]);
return 0;
}
@@ -1712,29 +1730,51 @@ static int taprio_parse_tc_entries(struct Qdisc *sch,
struct netlink_ext_ack *extack)
{
struct taprio_sched *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
u32 max_sdu[TC_QOPT_MAX_QUEUE];
+ bool have_preemption = false;
unsigned long seen_tcs = 0;
+ u32 fp[TC_QOPT_MAX_QUEUE];
struct nlattr *n;
int tc, rem;
int err = 0;
- for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++) {
max_sdu[tc] = q->max_sdu[tc];
+ fp[tc] = q->fp[tc];
+ }
nla_for_each_nested(n, opt, rem) {
if (nla_type(n) != TCA_TAPRIO_ATTR_TC_ENTRY)
continue;
- err = taprio_parse_tc_entry(sch, n, max_sdu, &seen_tcs,
+ err = taprio_parse_tc_entry(sch, n, max_sdu, fp, &seen_tcs,
extack);
if (err)
- goto out;
+ return err;
}
- for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++) {
q->max_sdu[tc] = max_sdu[tc];
+ q->fp[tc] = fp[tc];
+ if (fp[tc] != TC_FP_EXPRESS)
+ have_preemption = true;
+ }
-out:
+ if (have_preemption) {
+ if (!FULL_OFFLOAD_IS_ENABLED(q->flags)) {
+ NL_SET_ERR_MSG(extack,
+ "Preemption only supported with full offload");
+ return -EOPNOTSUPP;
+ }
+
+ if (!ethtool_dev_mm_supported(dev)) {
+ NL_SET_ERR_MSG(extack,
+ "Device does not support preemption");
+ return -EOPNOTSUPP;
+ }
+ }
+
return err;
}
@@ -2015,7 +2055,7 @@ static int taprio_init(struct Qdisc *sch, struct nlattr *opt,
{
struct taprio_sched *q = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
- int i;
+ int i, tc;
spin_lock_init(&q->current_entry_lock);
@@ -2072,6 +2112,9 @@ static int taprio_init(struct Qdisc *sch, struct nlattr *opt,
q->qdiscs[i] = qdisc;
}
+ for (tc = 0; tc < TC_QOPT_MAX_QUEUE; tc++)
+ q->fp[tc] = TC_FP_EXPRESS;
+
taprio_detect_broken_mqprio(q);
return taprio_change(sch, opt, extack);
@@ -2215,6 +2258,7 @@ static int dump_schedule(struct sk_buff *msg,
}
static int taprio_dump_tc_entries(struct sk_buff *skb,
+ struct taprio_sched *q,
struct sched_gate_list *sched)
{
struct nlattr *n;
@@ -2232,6 +2276,9 @@ static int taprio_dump_tc_entries(struct sk_buff *skb,
sched->max_sdu[tc]))
goto nla_put_failure;
+ if (nla_put_u32(skb, TCA_TAPRIO_TC_ENTRY_FP, q->fp[tc]))
+ goto nla_put_failure;
+
nla_nest_end(skb, n);
}
@@ -2273,7 +2320,7 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
nla_put_u32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
goto options_error;
- if (oper && taprio_dump_tc_entries(skb, oper))
+ if (oper && taprio_dump_tc_entries(skb, q, oper))
goto options_error;
if (oper && dump_schedule(skb, oper))
diff --git a/net/sctp/associola.c b/net/sctp/associola.c
index 63ba555..7965291 100644
--- a/net/sctp/associola.c
+++ b/net/sctp/associola.c
@@ -1597,9 +1597,10 @@ int sctp_assoc_set_bind_addr_from_cookie(struct sctp_association *asoc,
struct sctp_cookie *cookie,
gfp_t gfp)
{
- int var_size2 = ntohs(cookie->peer_init->chunk_hdr.length);
+ struct sctp_init_chunk *peer_init = (struct sctp_init_chunk *)(cookie + 1);
+ int var_size2 = ntohs(peer_init->chunk_hdr.length);
int var_size3 = cookie->raw_addr_list_len;
- __u8 *raw = (__u8 *)cookie->peer_init + var_size2;
+ __u8 *raw = (__u8 *)peer_init + var_size2;
return sctp_raw_to_bind_addrs(&asoc->base.bind_addr, raw, var_size3,
asoc->ep->base.bind_addr.port, gfp);
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 3496414..c58fffc 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -738,7 +738,7 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
tfm = asoc->ep->auth_hmacs[hmac_id];
- digest = auth->auth_hdr.hmac;
+ digest = (u8 *)(&auth->auth_hdr + 1);
if (crypto_shash_setkey(tfm, &asoc_key->data[0], asoc_key->len))
goto free;
diff --git a/net/sctp/input.c b/net/sctp/input.c
index 127bf28a..2613c4d 100644
--- a/net/sctp/input.c
+++ b/net/sctp/input.c
@@ -1150,7 +1150,7 @@ static struct sctp_association *__sctp_rcv_init_lookup(struct net *net,
init = (struct sctp_init_chunk *)skb->data;
/* Walk the parameters looking for embedded addresses. */
- sctp_walk_params(params, init, init_hdr.params) {
+ sctp_walk_params(params, init) {
/* Note: Ignoring hostname addresses. */
af = sctp_get_af_specific(param_type2af(params.p->type));
diff --git a/net/sctp/outqueue.c b/net/sctp/outqueue.c
index 2083107..0dc6b8a 100644
--- a/net/sctp/outqueue.c
+++ b/net/sctp/outqueue.c
@@ -1231,7 +1231,7 @@ static void sctp_sack_update_unack_data(struct sctp_association *assoc,
unack_data = assoc->next_tsn - assoc->ctsn_ack_point - 1;
- frags = sack->variable;
+ frags = (union sctp_sack_variable *)(sack + 1);
for (i = 0; i < ntohs(sack->num_gap_ack_blocks); i++) {
unack_data -= ((ntohs(frags[i].gab.end) -
ntohs(frags[i].gab.start) + 1));
@@ -1252,7 +1252,6 @@ int sctp_outq_sack(struct sctp_outq *q, struct sctp_chunk *chunk)
struct sctp_transport *transport;
struct sctp_chunk *tchunk = NULL;
struct list_head *lchunk, *transport_list, *temp;
- union sctp_sack_variable *frags = sack->variable;
__u32 sack_ctsn, ctsn, tsn;
__u32 highest_tsn, highest_new_tsn;
__u32 sack_a_rwnd;
@@ -1313,8 +1312,12 @@ int sctp_outq_sack(struct sctp_outq *q, struct sctp_chunk *chunk)
/* Get the highest TSN in the sack. */
highest_tsn = sack_ctsn;
- if (gap_ack_blocks)
+ if (gap_ack_blocks) {
+ union sctp_sack_variable *frags =
+ (union sctp_sack_variable *)(sack + 1);
+
highest_tsn += ntohs(frags[gap_ack_blocks - 1].gab.end);
+ }
if (TSN_lt(asoc->highest_sacked, highest_tsn))
asoc->highest_sacked = highest_tsn;
@@ -1789,7 +1792,7 @@ static int sctp_acked(struct sctp_sackhdr *sack, __u32 tsn)
* Block are assumed to have been received correctly.
*/
- frags = sack->variable;
+ frags = (union sctp_sack_variable *)(sack + 1);
blocks = ntohs(sack->num_gap_ack_blocks);
tsn_offset = tsn - ctsn;
for (i = 0; i < blocks; ++i) {
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index c7503fd..08527d8 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1707,11 +1707,11 @@ static struct sctp_cookie_param *sctp_pack_cookie(
ktime_get_real());
/* Copy the peer's init packet. */
- memcpy(&cookie->c.peer_init[0], init_chunk->chunk_hdr,
+ memcpy(cookie + 1, init_chunk->chunk_hdr,
ntohs(init_chunk->chunk_hdr->length));
/* Copy the raw local address list of the association. */
- memcpy((__u8 *)&cookie->c.peer_init[0] +
+ memcpy((__u8 *)(cookie + 1) +
ntohs(init_chunk->chunk_hdr->length), raw_addrs, addrs_len);
if (sctp_sk(ep->base.sk)->hmac) {
@@ -2207,7 +2207,7 @@ static enum sctp_ierror sctp_verify_param(struct net *net,
break;
case SCTP_PARAM_HOST_NAME_ADDRESS:
- /* Tell the peer, we won't support this param. */
+ /* This param has been Deprecated, send ABORT. */
sctp_process_hn_param(asoc, param, chunk, err_chunk);
retval = SCTP_IERROR_ABORT;
break;
@@ -2306,7 +2306,7 @@ int sctp_verify_init(struct net *net, const struct sctp_endpoint *ep,
ntohl(peer_init->init_hdr.a_rwnd) < SCTP_DEFAULT_MINWINDOW)
return sctp_process_inv_mandatory(asoc, chunk, errp);
- sctp_walk_params(param, peer_init, init_hdr.params) {
+ sctp_walk_params(param, peer_init) {
if (param.p->type == SCTP_PARAM_STATE_COOKIE)
has_cookie = true;
}
@@ -2329,7 +2329,7 @@ int sctp_verify_init(struct net *net, const struct sctp_endpoint *ep,
chunk, errp);
/* Verify all the variable length parameters */
- sctp_walk_params(param, peer_init, init_hdr.params) {
+ sctp_walk_params(param, peer_init) {
result = sctp_verify_param(net, ep, asoc, param, cid,
chunk, errp);
switch (result) {
@@ -2381,7 +2381,7 @@ int sctp_process_init(struct sctp_association *asoc, struct sctp_chunk *chunk,
src_match = 1;
/* Process the initialization parameters. */
- sctp_walk_params(param, peer_init, init_hdr.params) {
+ sctp_walk_params(param, peer_init) {
if (!src_match &&
(param.p->type == SCTP_PARAM_IPV4_ADDRESS ||
param.p->type == SCTP_PARAM_IPV6_ADDRESS)) {
@@ -2589,10 +2589,6 @@ static int sctp_process_param(struct sctp_association *asoc,
asoc->cookie_life = ktime_add_ms(asoc->cookie_life, stale);
break;
- case SCTP_PARAM_HOST_NAME_ADDRESS:
- pr_debug("%s: unimplemented SCTP_HOST_NAME_ADDRESS\n", __func__);
- break;
-
case SCTP_PARAM_SUPPORTED_ADDRESS_TYPES:
/* Turn off the default values first so we'll know which
* ones are really set by the peer.
@@ -2624,10 +2620,6 @@ static int sctp_process_param(struct sctp_association *asoc,
asoc->peer.ipv6_address = 1;
break;
- case SCTP_PARAM_HOST_NAME_ADDRESS:
- asoc->peer.hostname_address = 1;
- break;
-
default: /* Just ignore anything else. */
break;
}
@@ -3210,7 +3202,7 @@ bool sctp_verify_asconf(const struct sctp_association *asoc,
union sctp_params param;
addip = (struct sctp_addip_chunk *)chunk->chunk_hdr;
- sctp_walk_params(param, addip, addip_hdr.params) {
+ sctp_walk_params(param, addip) {
size_t length = ntohs(param.p->length);
*errp = param.p;
@@ -3223,14 +3215,14 @@ bool sctp_verify_asconf(const struct sctp_association *asoc,
/* ensure there is only one addr param and it's in the
* beginning of addip_hdr params, or we reject it.
*/
- if (param.v != addip->addip_hdr.params)
+ if (param.v != (addip + 1))
return false;
addr_param_seen = true;
break;
case SCTP_PARAM_IPV6_ADDRESS:
if (length != sizeof(struct sctp_ipv6addr_param))
return false;
- if (param.v != addip->addip_hdr.params)
+ if (param.v != (addip + 1))
return false;
addr_param_seen = true;
break;
@@ -3310,7 +3302,7 @@ struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc,
goto done;
/* Process the TLVs contained within the ASCONF chunk. */
- sctp_walk_params(param, addip, addip_hdr.params) {
+ sctp_walk_params(param, addip) {
/* Skip preceeding address parameters. */
if (param.p->type == SCTP_PARAM_IPV4_ADDRESS ||
param.p->type == SCTP_PARAM_IPV6_ADDRESS)
@@ -3644,7 +3636,7 @@ static struct sctp_chunk *sctp_make_reconf(const struct sctp_association *asoc,
return NULL;
reconf = (struct sctp_reconf_chunk *)retval->chunk_hdr;
- retval->param_hdr.v = reconf->params;
+ retval->param_hdr.v = (u8 *)(reconf + 1);
return retval;
}
@@ -3886,7 +3878,7 @@ bool sctp_verify_reconf(const struct sctp_association *asoc,
__u16 cnt = 0;
hdr = (struct sctp_reconf_chunk *)chunk->chunk_hdr;
- sctp_walk_params(param, hdr, params) {
+ sctp_walk_params(param, hdr) {
__u16 length = ntohs(param.p->length);
*errp = param.p;
diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c
index 463c4a5..7fbeb99 100644
--- a/net/sctp/sm_sideeffect.c
+++ b/net/sctp/sm_sideeffect.c
@@ -984,8 +984,7 @@ static void sctp_cmd_process_operr(struct sctp_cmd_seq *cmds,
{
struct sctp_chunkhdr *unk_chunk_hdr;
- unk_chunk_hdr = (struct sctp_chunkhdr *)
- err_hdr->variable;
+ unk_chunk_hdr = (struct sctp_chunkhdr *)(err_hdr + 1);
switch (unk_chunk_hdr->type) {
/* ADDIP 4.1 A9) If the peer responds to an ASCONF with
* an ERROR chunk reporting that it did not recognized
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index ce54261..97f1155 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -794,8 +794,7 @@ enum sctp_disposition sctp_sf_do_5_1D_ce(struct net *net,
/* This is a brand-new association, so these are not yet side
* effects--it is safe to run them here.
*/
- peer_init = &chunk->subh.cookie_hdr->c.peer_init[0];
-
+ peer_init = (struct sctp_init_chunk *)(chunk->subh.cookie_hdr + 1);
if (!sctp_process_init(new_asoc, chunk,
&chunk->subh.cookie_hdr->c.peer_addr,
peer_init, GFP_ATOMIC))
@@ -1337,7 +1336,7 @@ static int sctp_sf_send_restart_abort(struct net *net, union sctp_addr *ssa,
* throughout the code today.
*/
errhdr = (struct sctp_errhdr *)buffer;
- addrparm = (union sctp_addr_param *)errhdr->variable;
+ addrparm = (union sctp_addr_param *)(errhdr + 1);
/* Copy into a parm format. */
len = af->to_addr_param(ssa, addrparm);
@@ -1869,8 +1868,7 @@ static enum sctp_disposition sctp_sf_do_dupcook_a(
/* new_asoc is a brand-new association, so these are not yet
* side effects--it is safe to run them here.
*/
- peer_init = &chunk->subh.cookie_hdr->c.peer_init[0];
-
+ peer_init = (struct sctp_init_chunk *)(chunk->subh.cookie_hdr + 1);
if (!sctp_process_init(new_asoc, chunk, sctp_source(chunk), peer_init,
GFP_ATOMIC))
goto nomem;
@@ -1990,7 +1988,7 @@ static enum sctp_disposition sctp_sf_do_dupcook_b(
/* new_asoc is a brand-new association, so these are not yet
* side effects--it is safe to run them here.
*/
- peer_init = &chunk->subh.cookie_hdr->c.peer_init[0];
+ peer_init = (struct sctp_init_chunk *)(chunk->subh.cookie_hdr + 1);
if (!sctp_process_init(new_asoc, chunk, sctp_source(chunk), peer_init,
GFP_ATOMIC))
goto nomem;
@@ -4142,7 +4140,7 @@ enum sctp_disposition sctp_sf_do_reconf(struct net *net,
(void *)err_param, commands);
hdr = (struct sctp_reconf_chunk *)chunk->chunk_hdr;
- sctp_walk_params(param, hdr, params) {
+ sctp_walk_params(param, hdr) {
struct sctp_chunk *reply = NULL;
struct sctp_ulpevent *ev = NULL;
@@ -4393,7 +4391,7 @@ static enum sctp_ierror sctp_sf_authenticate(
* 3. Compute the new digest
* 4. Compare saved and new digests.
*/
- digest = auth_hdr->hmac;
+ digest = (u8 *)(auth_hdr + 1);
skb_pull(chunk->skb, sig_len);
save_digest = kmemdup(digest, sig_len, GFP_ATOMIC);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 218e098..cda8c28 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -5192,10 +5192,11 @@ int sctp_get_sctp_info(struct sock *sk, struct sctp_association *asoc,
info->sctpi_peer_rwnd = asoc->peer.rwnd;
info->sctpi_peer_tag = asoc->c.peer_vtag;
- mask = asoc->peer.ecn_capable << 1;
+ mask = asoc->peer.intl_capable << 1;
+ mask = (mask | asoc->peer.ecn_capable) << 1;
mask = (mask | asoc->peer.ipv4_address) << 1;
mask = (mask | asoc->peer.ipv6_address) << 1;
- mask = (mask | asoc->peer.hostname_address) << 1;
+ mask = (mask | asoc->peer.reconf_capable) << 1;
mask = (mask | asoc->peer.asconf_capable) << 1;
mask = (mask | asoc->peer.prsctp_capable) << 1;
mask = (mask | asoc->peer.auth_capable);
diff --git a/net/sctp/stream.c b/net/sctp/stream.c
index ee6514a..c241cc55 100644
--- a/net/sctp/stream.c
+++ b/net/sctp/stream.c
@@ -491,7 +491,7 @@ static struct sctp_paramhdr *sctp_chunk_lookup_strreset_param(
return NULL;
hdr = (struct sctp_reconf_chunk *)chunk->chunk_hdr;
- sctp_walk_params(param, hdr, params) {
+ sctp_walk_params(param, hdr) {
/* sctp_strreset_tsnreq is actually the basic structure
* of all stream reconf params, so it's safe to use it
* to access request_seq.
diff --git a/net/sctp/stream_interleave.c b/net/sctp/stream_interleave.c
index b046b11..840f240 100644
--- a/net/sctp/stream_interleave.c
+++ b/net/sctp/stream_interleave.c
@@ -1153,8 +1153,8 @@ static void sctp_generate_iftsn(struct sctp_outq *q, __u32 ctsn)
}
#define _sctp_walk_ifwdtsn(pos, chunk, end) \
- for (pos = chunk->subh.ifwdtsn_hdr->skip; \
- (void *)pos <= (void *)chunk->subh.ifwdtsn_hdr->skip + (end) - \
+ for (pos = (void *)(chunk->subh.ifwdtsn_hdr + 1); \
+ (void *)pos <= (void *)(chunk->subh.ifwdtsn_hdr + 1) + (end) - \
sizeof(struct sctp_ifwdtsn_skip); pos++)
#define sctp_walk_ifwdtsn(pos, ch) \
diff --git a/net/socket.c b/net/socket.c
index 73e493d..a7b4b37 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -957,6 +957,7 @@ void __sock_recv_timestamp(struct msghdr *msg, struct sock *sk,
}
EXPORT_SYMBOL_GPL(__sock_recv_timestamp);
+#ifdef CONFIG_WIRELESS
void __sock_recv_wifi_status(struct msghdr *msg, struct sock *sk,
struct sk_buff *skb)
{
@@ -972,6 +973,7 @@ void __sock_recv_wifi_status(struct msghdr *msg, struct sock *sk,
put_cmsg(msg, SOL_SOCKET, SCM_WIFI_STATUS, sizeof(ack), &ack);
}
EXPORT_SYMBOL_GPL(__sock_recv_wifi_status);
+#endif
static inline void sock_recv_drops(struct msghdr *msg, struct sock *sk,
struct sk_buff *skb)
diff --git a/net/sunrpc/auth_gss/gss_krb5_test.c b/net/sunrpc/auth_gss/gss_krb5_test.c
index ce0541e..95ca783 100644
--- a/net/sunrpc/auth_gss/gss_krb5_test.c
+++ b/net/sunrpc/auth_gss/gss_krb5_test.c
@@ -73,7 +73,6 @@ static void checksum_case(struct kunit *test)
{
const struct gss_krb5_test_param *param = test->param_value;
struct xdr_buf buf = {
- .head[0].iov_base = param->plaintext->data,
.head[0].iov_len = param->plaintext->len,
.len = param->plaintext->len,
};
@@ -99,6 +98,10 @@ static void checksum_case(struct kunit *test)
err = crypto_ahash_setkey(tfm, Kc.data, Kc.len);
KUNIT_ASSERT_EQ(test, err, 0);
+ buf.head[0].iov_base = kunit_kzalloc(test, buf.head[0].iov_len, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf.head[0].iov_base);
+ memcpy(buf.head[0].iov_base, param->plaintext->data, buf.head[0].iov_len);
+
checksum.len = gk5e->cksumlength;
checksum.data = kunit_kzalloc(test, checksum.len, GFP_KERNEL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, checksum.data);
@@ -1327,6 +1330,7 @@ static void rfc6803_encrypt_case(struct kunit *test)
if (!gk5e)
kunit_skip(test, "Encryption type is not available");
+ memset(usage_data, 0, sizeof(usage_data));
usage.data[3] = param->constant;
Ke.len = gk5e->Ke_length;
diff --git a/net/vmw_vsock/vsock_loopback.c b/net/vmw_vsock/vsock_loopback.c
index e3afc0c8..5c6360d 100644
--- a/net/vmw_vsock/vsock_loopback.c
+++ b/net/vmw_vsock/vsock_loopback.c
@@ -31,8 +31,7 @@ static int vsock_loopback_send_pkt(struct sk_buff *skb)
struct vsock_loopback *vsock = &the_vsock_loopback;
int len = skb->len;
- skb_queue_tail(&vsock->pkt_queue, skb);
-
+ virtio_vsock_skb_queue_tail(&vsock->pkt_queue, skb);
queue_work(vsock->workqueue, &vsock->pkt_work);
return len;
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index 436d296..39fb91f 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -231,9 +231,6 @@ static int xfrm4_remove_tunnel_encap(struct xfrm_state *x, struct sk_buff *skb)
{
int err = -EINVAL;
- if (XFRM_MODE_SKB_CB(skb)->protocol != IPPROTO_IPIP)
- goto out;
-
if (!pskb_may_pull(skb, sizeof(struct iphdr)))
goto out;
@@ -269,8 +266,6 @@ static int xfrm6_remove_tunnel_encap(struct xfrm_state *x, struct sk_buff *skb)
{
int err = -EINVAL;
- if (XFRM_MODE_SKB_CB(skb)->protocol != IPPROTO_IPV6)
- goto out;
if (!pskb_may_pull(skb, sizeof(struct ipv6hdr)))
goto out;
@@ -331,22 +326,26 @@ static int xfrm6_remove_beet_encap(struct xfrm_state *x, struct sk_buff *skb)
*/
static int
xfrm_inner_mode_encap_remove(struct xfrm_state *x,
- const struct xfrm_mode *inner_mode,
struct sk_buff *skb)
{
- switch (inner_mode->encap) {
+ switch (x->props.mode) {
case XFRM_MODE_BEET:
- if (inner_mode->family == AF_INET)
+ switch (XFRM_MODE_SKB_CB(skb)->protocol) {
+ case IPPROTO_IPIP:
+ case IPPROTO_BEETPH:
return xfrm4_remove_beet_encap(x, skb);
- if (inner_mode->family == AF_INET6)
+ case IPPROTO_IPV6:
return xfrm6_remove_beet_encap(x, skb);
+ }
break;
case XFRM_MODE_TUNNEL:
- if (inner_mode->family == AF_INET)
+ switch (XFRM_MODE_SKB_CB(skb)->protocol) {
+ case IPPROTO_IPIP:
return xfrm4_remove_tunnel_encap(x, skb);
- if (inner_mode->family == AF_INET6)
+ case IPPROTO_IPV6:
return xfrm6_remove_tunnel_encap(x, skb);
break;
+ }
}
WARN_ON_ONCE(1);
@@ -355,9 +354,7 @@ xfrm_inner_mode_encap_remove(struct xfrm_state *x,
static int xfrm_prepare_input(struct xfrm_state *x, struct sk_buff *skb)
{
- const struct xfrm_mode *inner_mode = &x->inner_mode;
-
- switch (x->outer_mode.family) {
+ switch (x->props.family) {
case AF_INET:
xfrm4_extract_header(skb);
break;
@@ -369,17 +366,12 @@ static int xfrm_prepare_input(struct xfrm_state *x, struct sk_buff *skb)
return -EAFNOSUPPORT;
}
- if (x->sel.family == AF_UNSPEC) {
- inner_mode = xfrm_ip2inner_mode(x, XFRM_MODE_SKB_CB(skb)->protocol);
- if (!inner_mode)
- return -EAFNOSUPPORT;
- }
-
- switch (inner_mode->family) {
- case AF_INET:
+ switch (XFRM_MODE_SKB_CB(skb)->protocol) {
+ case IPPROTO_IPIP:
+ case IPPROTO_BEETPH:
skb->protocol = htons(ETH_P_IP);
break;
- case AF_INET6:
+ case IPPROTO_IPV6:
skb->protocol = htons(ETH_P_IPV6);
break;
default:
@@ -387,7 +379,7 @@ static int xfrm_prepare_input(struct xfrm_state *x, struct sk_buff *skb)
break;
}
- return xfrm_inner_mode_encap_remove(x, inner_mode, skb);
+ return xfrm_inner_mode_encap_remove(x, skb);
}
/* Remove encapsulation header.
@@ -433,17 +425,16 @@ static int xfrm6_transport_input(struct xfrm_state *x, struct sk_buff *skb)
}
static int xfrm_inner_mode_input(struct xfrm_state *x,
- const struct xfrm_mode *inner_mode,
struct sk_buff *skb)
{
- switch (inner_mode->encap) {
+ switch (x->props.mode) {
case XFRM_MODE_BEET:
case XFRM_MODE_TUNNEL:
return xfrm_prepare_input(x, skb);
case XFRM_MODE_TRANSPORT:
- if (inner_mode->family == AF_INET)
+ if (x->props.family == AF_INET)
return xfrm4_transport_input(x, skb);
- if (inner_mode->family == AF_INET6)
+ if (x->props.family == AF_INET6)
return xfrm6_transport_input(x, skb);
break;
case XFRM_MODE_ROUTEOPTIMIZATION:
@@ -461,7 +452,6 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
{
const struct xfrm_state_afinfo *afinfo;
struct net *net = dev_net(skb->dev);
- const struct xfrm_mode *inner_mode;
int err;
__be32 seq;
__be32 seq_hi;
@@ -491,7 +481,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
goto drop;
}
- family = x->outer_mode.family;
+ family = x->props.family;
/* An encap_type of -1 indicates async resumption. */
if (encap_type == -1) {
@@ -676,17 +666,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
XFRM_MODE_SKB_CB(skb)->protocol = nexthdr;
- inner_mode = &x->inner_mode;
-
- if (x->sel.family == AF_UNSPEC) {
- inner_mode = xfrm_ip2inner_mode(x, XFRM_MODE_SKB_CB(skb)->protocol);
- if (inner_mode == NULL) {
- XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR);
- goto drop;
- }
- }
-
- if (xfrm_inner_mode_input(x, inner_mode, skb)) {
+ if (xfrm_inner_mode_input(x, skb)) {
XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR);
goto drop;
}
@@ -701,7 +681,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
* transport mode so the outer address is identical.
*/
daddr = &x->id.daddr;
- family = x->outer_mode.family;
+ family = x->props.family;
err = xfrm_parse_spi(skb, nexthdr, &spi, &seq);
if (err < 0) {
@@ -732,7 +712,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
err = -EAFNOSUPPORT;
rcu_read_lock();
- afinfo = xfrm_state_afinfo_get_rcu(x->inner_mode.family);
+ afinfo = xfrm_state_afinfo_get_rcu(x->props.family);
if (likely(afinfo))
err = afinfo->transport_finish(skb, xfrm_gro || async);
rcu_read_unlock();
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index ff114d6..369e5de 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -412,7 +412,7 @@ static int xfrm4_prepare_output(struct xfrm_state *x, struct sk_buff *skb)
IPCB(skb)->flags |= IPSKB_XFRM_TUNNEL_SIZE;
skb->protocol = htons(ETH_P_IP);
- switch (x->outer_mode.encap) {
+ switch (x->props.mode) {
case XFRM_MODE_BEET:
return xfrm4_beet_encap_add(x, skb);
case XFRM_MODE_TUNNEL:
@@ -435,7 +435,7 @@ static int xfrm6_prepare_output(struct xfrm_state *x, struct sk_buff *skb)
skb->ignore_df = 1;
skb->protocol = htons(ETH_P_IPV6);
- switch (x->outer_mode.encap) {
+ switch (x->props.mode) {
case XFRM_MODE_BEET:
return xfrm6_beet_encap_add(x, skb);
case XFRM_MODE_TUNNEL:
@@ -451,22 +451,22 @@ static int xfrm6_prepare_output(struct xfrm_state *x, struct sk_buff *skb)
static int xfrm_outer_mode_output(struct xfrm_state *x, struct sk_buff *skb)
{
- switch (x->outer_mode.encap) {
+ switch (x->props.mode) {
case XFRM_MODE_BEET:
case XFRM_MODE_TUNNEL:
- if (x->outer_mode.family == AF_INET)
+ if (x->props.family == AF_INET)
return xfrm4_prepare_output(x, skb);
- if (x->outer_mode.family == AF_INET6)
+ if (x->props.family == AF_INET6)
return xfrm6_prepare_output(x, skb);
break;
case XFRM_MODE_TRANSPORT:
- if (x->outer_mode.family == AF_INET)
+ if (x->props.family == AF_INET)
return xfrm4_transport_output(x, skb);
- if (x->outer_mode.family == AF_INET6)
+ if (x->props.family == AF_INET6)
return xfrm6_transport_output(x, skb);
break;
case XFRM_MODE_ROUTEOPTIMIZATION:
- if (x->outer_mode.family == AF_INET6)
+ if (x->props.family == AF_INET6)
return xfrm6_ro_output(x, skb);
WARN_ON_ONCE(1);
break;
@@ -875,21 +875,10 @@ static int xfrm6_extract_output(struct xfrm_state *x, struct sk_buff *skb)
static int xfrm_inner_extract_output(struct xfrm_state *x, struct sk_buff *skb)
{
- const struct xfrm_mode *inner_mode;
-
- if (x->sel.family == AF_UNSPEC)
- inner_mode = xfrm_ip2inner_mode(x,
- xfrm_af2proto(skb_dst(skb)->ops->family));
- else
- inner_mode = &x->inner_mode;
-
- if (inner_mode == NULL)
- return -EAFNOSUPPORT;
-
- switch (inner_mode->family) {
- case AF_INET:
+ switch (skb->protocol) {
+ case htons(ETH_P_IP):
return xfrm4_extract_output(x, skb);
- case AF_INET6:
+ case htons(ETH_P_IPV6):
return xfrm6_extract_output(x, skb);
}
diff --git a/rust/Makefile b/rust/Makefile
index f88d108..aef85e9 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -262,6 +262,20 @@
# some configurations, with new GCC versions, etc.
bindgen_extra_c_flags = -w --target=$(BINDGEN_TARGET)
+# Auto variable zero-initialization requires an additional special option with
+# clang that is going to be removed sometime in the future (likely in
+# clang-18), so make sure to pass this option only if clang supports it
+# (libclang major version < 16).
+#
+# https://github.com/llvm/llvm-project/issues/44842
+# https://github.com/llvm/llvm-project/blob/llvmorg-16.0.0-rc2/clang/docs/ReleaseNotes.rst#deprecated-compiler-flags
+ifdef CONFIG_INIT_STACK_ALL_ZERO
+libclang_maj_ver=$(shell $(BINDGEN) $(srctree)/scripts/rust_is_available_bindgen_libclang.h 2>&1 | sed -ne 's/.*clang version \([0-9]*\).*/\1/p')
+ifeq ($(shell expr $(libclang_maj_ver) \< 16), 1)
+bindgen_extra_c_flags += -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
+endif
+endif
+
bindgen_c_flags = $(filter-out $(bindgen_skip_c_flags), $(c_flags)) \
$(bindgen_extra_c_flags)
endif
@@ -283,7 +297,7 @@
$(bindgen_target_cflags) $(bindgen_target_extra)
$(obj)/bindings/bindings_generated.rs: private bindgen_target_flags = \
- $(shell grep -v '^\#\|^$$' $(srctree)/$(src)/bindgen_parameters)
+ $(shell grep -v '^#\|^$$' $(srctree)/$(src)/bindgen_parameters)
$(obj)/bindings/bindings_generated.rs: $(src)/bindings/bindings_helper.h \
$(src)/bindgen_parameters FORCE
$(call if_changed_dep,bindgen)
diff --git a/rust/kernel/print.rs b/rust/kernel/print.rs
index 3010332..8009184 100644
--- a/rust/kernel/print.rs
+++ b/rust/kernel/print.rs
@@ -18,7 +18,11 @@
// Called from `vsprintf` with format specifier `%pA`.
#[no_mangle]
-unsafe fn rust_fmt_argument(buf: *mut c_char, end: *mut c_char, ptr: *const c_void) -> *mut c_char {
+unsafe extern "C" fn rust_fmt_argument(
+ buf: *mut c_char,
+ end: *mut c_char,
+ ptr: *const c_void,
+) -> *mut c_char {
use fmt::Write;
// SAFETY: The C contract guarantees that `buf` is valid if it's less than `end`.
let mut w = unsafe { RawFormatter::from_ptrs(buf.cast(), end.cast()) };
diff --git a/rust/kernel/str.rs b/rust/kernel/str.rs
index b771310..cd3d2a6 100644
--- a/rust/kernel/str.rs
+++ b/rust/kernel/str.rs
@@ -408,7 +408,7 @@
/// If `pos` is less than `end`, then the region between `pos` (inclusive) and `end`
/// (exclusive) must be valid for writes for the lifetime of the returned [`RawFormatter`].
pub(crate) unsafe fn from_ptrs(pos: *mut u8, end: *mut u8) -> Self {
- // INVARIANT: The safety requierments guarantee the type invariants.
+ // INVARIANT: The safety requirements guarantee the type invariants.
Self {
beg: pos as _,
pos: pos as _,
diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index 61f72eb..4d90691 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -27,21 +27,6 @@
tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \
--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3)
-# tarball compression
-# ---------------------------------------------------------------------------
-
-%.tar.gz: %.tar
- $(call cmd,gzip)
-
-%.tar.bz2: %.tar
- $(call cmd,bzip2)
-
-%.tar.xz: %.tar
- $(call cmd,xzmisc)
-
-%.tar.zst: %.tar
- $(call cmd,zstd)
-
# Git
# ---------------------------------------------------------------------------
@@ -57,16 +42,24 @@
false; \
fi
+git-config-tar.gz = -c tar.tar.gz.command="$(KGZIP)"
+git-config-tar.bz2 = -c tar.tar.bz2.command="$(KBZIP2)"
+git-config-tar.xz = -c tar.tar.xz.command="$(XZ)"
+git-config-tar.zst = -c tar.tar.zst.command="$(ZSTD)"
+
+quiet_cmd_archive = ARCHIVE $@
+ cmd_archive = git -C $(srctree) $(git-config-tar$(suffix $@)) archive \
+ --output=$$(realpath $@) --prefix=$(basename $@)/ $(archive-args)
+
# Linux source tarball
# ---------------------------------------------------------------------------
-quiet_cmd_archive_linux = ARCHIVE $@
- cmd_archive_linux = \
- git -C $(srctree) archive --output=$$(realpath $@) --prefix=$(basename $@)/ $$(cat $<)
+linux-tarballs := $(addprefix linux, .tar.gz)
-targets += linux.tar
-linux.tar: .tmp_HEAD FORCE
- $(call if_changed,archive_linux)
+targets += $(linux-tarballs)
+$(linux-tarballs): archive-args = $$(cat $<)
+$(linux-tarballs): .tmp_HEAD FORCE
+ $(call if_changed,archive)
# rpm-pkg
# ---------------------------------------------------------------------------
@@ -94,7 +87,7 @@
$(UTS_MACHINE)-linux -bb $(objtree)/binkernel.spec
quiet_cmd_debianize = GEN $@
- cmd_debianize = $(srctree)/scripts/package/mkdebian
+ cmd_debianize = $(srctree)/scripts/package/mkdebian $(mkdebian-opts)
debian: FORCE
$(call cmd,debianize)
@@ -103,6 +96,7 @@
debian-orig: private source = $(shell dpkg-parsechangelog -S Source)
debian-orig: private version = $(shell dpkg-parsechangelog -S Version | sed 's/-[^-]*$$//')
debian-orig: private orig-name = $(source)_$(version).orig.tar.gz
+debian-orig: mkdebian-opts = --need-source
debian-orig: linux.tar.gz debian
$(Q)if [ "$(df --output=target .. 2>/dev/null)" = "$(df --output=target $< 2>/dev/null)" ]; then \
ln -f $< ../$(orig-name); \
@@ -145,10 +139,17 @@
$(Q)$(MAKE) -f $(srctree)/Makefile
+$(Q)$(srctree)/scripts/package/buildtar $@
-quiet_cmd_tar = TAR $@
- cmd_tar = cd $<; tar cf ../$@ --owner=root --group=root --sort=name *
+compress-tar.gz = -I "$(KGZIP)"
+compress-tar.bz2 = -I "$(KBZIP2)"
+compress-tar.xz = -I "$(XZ)"
+compress-tar.zst = -I "$(ZSTD)"
-linux-$(KERNELRELEASE)-$(ARCH).tar: tar-install
+quiet_cmd_tar = TAR $@
+ cmd_tar = cd $<; tar cf ../$@ $(compress-tar$(suffix $@)) --owner=root --group=root --sort=name *
+
+dir-tarballs := $(addprefix linux-$(KERNELRELEASE)-$(ARCH), .tar .tar.gz .tar.bz2 .tar.xz .tar.zst)
+
+$(dir-tarballs): tar-install
$(call cmd,tar)
PHONY += dir-pkg
@@ -180,16 +181,17 @@
.tmp_perf/PERF-VERSION-FILE: .tmp_HEAD $(srctree)/tools/perf/util/PERF-VERSION-GEN | .tmp_perf
$(call cmd,perf_version_file)
-quiet_cmd_archive_perf = ARCHIVE $@
- cmd_archive_perf = \
- git -C $(srctree) archive --output=$$(realpath $@) --prefix=$(basename $@)/ \
- --add-file=$$(realpath $(word 2, $^)) \
+perf-archive-args = --add-file=$$(realpath $(word 2, $^)) \
--add-file=$$(realpath $(word 3, $^)) \
$$(cat $(word 2, $^))^{tree} $$(cat $<)
-targets += perf-$(KERNELVERSION).tar
-perf-$(KERNELVERSION).tar: tools/perf/MANIFEST .tmp_perf/HEAD .tmp_perf/PERF-VERSION-FILE FORCE
- $(call if_changed,archive_perf)
+
+perf-tarballs := $(addprefix perf-$(KERNELVERSION), .tar .tar.gz .tar.bz2 .tar.xz .tar.zst)
+
+targets += $(perf-tarballs)
+$(perf-tarballs): archive-args = $(perf-archive-args)
+$(perf-tarballs): tools/perf/MANIFEST .tmp_perf/HEAD .tmp_perf/PERF-VERSION-FILE FORCE
+ $(call if_changed,archive)
PHONY += perf-tar-src-pkg
perf-tar-src-pkg: perf-$(KERNELVERSION).tar
diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index ecc7ea9..946e250 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -104,7 +104,10 @@
name = path.name.replace(".rs", "")
# Skip those that are not crate roots.
- if f"{name}.o" not in open(path.parent / "Makefile").read():
+ try:
+ if f"{name}.o" not in open(path.parent / "Makefile").read():
+ continue
+ except FileNotFoundError:
continue
logging.info("Adding %s", name)
diff --git a/scripts/is_rust_module.sh b/scripts/is_rust_module.sh
index 28b3831..464761a 100755
--- a/scripts/is_rust_module.sh
+++ b/scripts/is_rust_module.sh
@@ -13,4 +13,4 @@
#
# In the future, checking for the `.comment` section may be another
# option, see https://github.com/rust-lang/rust/pull/97550.
-${NM} "$*" | grep -qE '^[0-9a-fA-F]+ r _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'
+${NM} "$*" | grep -qE '^[0-9a-fA-F]+ [Rr] _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'
diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
index f842ab5..8a98b7b 100755
--- a/scripts/package/gen-diff-patch
+++ b/scripts/package/gen-diff-patch
@@ -1,44 +1,36 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-only
-diff_patch="${1}"
-untracked_patch="${2}"
-srctree=$(dirname $0)/../..
+diff_patch=$1
-rm -f ${diff_patch} ${untracked_patch}
+mkdir -p "$(dirname "${diff_patch}")"
-if ! ${srctree}/scripts/check-git; then
+git -C "${srctree:-.}" diff HEAD > "${diff_patch}"
+
+if [ ! -s "${diff_patch}" ] ||
+ [ -z "$(git -C "${srctree:-.}" ls-files --other --exclude-standard | head -n1)" ]; then
exit
fi
-mkdir -p "$(dirname ${diff_patch})" "$(dirname ${untracked_patch})"
-
-git -C "${srctree}" diff HEAD > "${diff_patch}"
-
-if [ ! -s "${diff_patch}" ]; then
- rm -f "${diff_patch}"
- exit
-fi
-
-git -C ${srctree} status --porcelain --untracked-files=all |
-while read stat path
-do
- if [ "${stat}" = '??' ]; then
-
- if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
- ! head -n1 .tmp_diff | grep -q "Binary files"; then
- {
- echo "--- /dev/null"
- echo "+++ linux/$path"
- cat .tmp_diff | tail -n +3
- } >> ${untracked_patch}
- fi
- fi
-done
-
-rm -f .tmp_diff
-
-if [ ! -s "${diff_patch}" ]; then
- rm -f "${diff_patch}"
- exit
-fi
+# The source tarball, which is generated by 'git archive', contains everything
+# you committed in the repository. If you have local diff ('git diff HEAD'),
+# it will go into ${diff_patch}. If untracked files are remaining, the resulting
+# source package may not be correct.
+#
+# Examples:
+# - You modified a source file to add #include "new-header.h"
+# but forgot to add new-header.h
+# - You modified a Makefile to add 'obj-$(CONFIG_FOO) += new-dirver.o'
+# but you forgot to add new-driver.c
+#
+# You need to commit them, or at least stage them by 'git add'.
+#
+# This script does not take care of untracked files because doing so would
+# introduce additional complexity. Instead, print a warning message here if
+# untracked files are found.
+# If all untracked files are just garbage, you can ignore this warning.
+echo >&2 "============================ WARNING ============================"
+echo >&2 "Your working tree has diff from HEAD, and also untracked file(s)."
+echo >&2 "Please make sure you did 'git add' for all new files you need in"
+echo >&2 "the source package."
+echo >&2 "================================================================="
diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index e20a2b5..a4c2c22 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -84,7 +84,66 @@
fi
}
+# Create debian/source/ if it is a source package build
+gen_source ()
+{
+ mkdir -p debian/source
+
+ echo "3.0 (quilt)" > debian/source/format
+
+ {
+ echo "diff-ignore"
+ echo "extend-diff-ignore = .*"
+ } > debian/source/local-options
+
+ # Add .config as a patch
+ mkdir -p debian/patches
+ {
+ echo "Subject: Add .config"
+ echo "Author: ${maintainer}"
+ echo
+ echo "--- /dev/null"
+ echo "+++ linux/.config"
+ diff -u /dev/null "${KCONFIG_CONFIG}" | tail -n +3
+ } > debian/patches/config.patch
+ echo config.patch > debian/patches/series
+
+ "${srctree}/scripts/package/gen-diff-patch" debian/patches/diff.patch
+ if [ -s debian/patches/diff.patch ]; then
+ sed -i "
+ 1iSubject: Add local diff
+ 1iAuthor: ${maintainer}
+ 1i
+ " debian/patches/diff.patch
+
+ echo diff.patch >> debian/patches/series
+ else
+ rm -f debian/patches/diff.patch
+ fi
+}
+
rm -rf debian
+mkdir debian
+
+email=${DEBEMAIL-$EMAIL}
+
+# use email string directly if it contains <email>
+if echo "${email}" | grep -q '<.*>'; then
+ maintainer=${email}
+else
+ # or construct the maintainer string
+ user=${KBUILD_BUILD_USER-$(id -nu)}
+ name=${DEBFULLNAME-${user}}
+ if [ -z "${email}" ]; then
+ buildhost=${KBUILD_BUILD_HOST-$(hostname -f 2>/dev/null || hostname)}
+ email="${user}@${buildhost}"
+ fi
+ maintainer="${name} <${email}>"
+fi
+
+if [ "$1" = --need-source ]; then
+ gen_source
+fi
# Some variables and settings used throughout the script
version=$KERNELRELEASE
@@ -104,22 +163,6 @@
debarch=
set_debarch
-email=${DEBEMAIL-$EMAIL}
-
-# use email string directly if it contains <email>
-if echo $email | grep -q '<.*>'; then
- maintainer=$email
-else
- # or construct the maintainer string
- user=${KBUILD_BUILD_USER-$(id -nu)}
- name=${DEBFULLNAME-$user}
- if [ -z "$email" ]; then
- buildhost=${KBUILD_BUILD_HOST-$(hostname -f 2>/dev/null || hostname)}
- email="$user@$buildhost"
- fi
- maintainer="$name <$email>"
-fi
-
# Try to determine distribution
if [ -n "$KDEB_CHANGELOG_DIST" ]; then
distribution=$KDEB_CHANGELOG_DIST
@@ -132,34 +175,6 @@
echo >&2 "Install lsb-release or set \$KDEB_CHANGELOG_DIST explicitly"
fi
-mkdir -p debian/source/
-echo "3.0 (quilt)" > debian/source/format
-
-{
- echo "diff-ignore"
- echo "extend-diff-ignore = .*"
-} > debian/source/local-options
-
-# Add .config as a patch
-mkdir -p debian/patches
-{
- echo "Subject: Add .config"
- echo "Author: ${maintainer}"
- echo
- echo "--- /dev/null"
- echo "+++ linux/.config"
- diff -u /dev/null "${KCONFIG_CONFIG}" | tail -n +3
-} > debian/patches/config
-echo config > debian/patches/series
-
-$(dirname $0)/gen-diff-patch debian/patches/diff.patch debian/patches/untracked.patch
-if [ -f debian/patches/diff.patch ]; then
- echo diff.patch >> debian/patches/series
-fi
-if [ -f debian/patches/untracked.patch ]; then
- echo untracked.patch >> debian/patches/series
-fi
-
echo $debarch > debian/arch
extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
extra_build_depends="$extra_build_depends, $(if_enabled_echo CONFIG_SYSTEM_TRUSTED_KEYRING libssl-dev:native)"
diff --git a/scripts/package/mkspec b/scripts/package/mkspec
index b7d1dc2..fc8ad3f 100755
--- a/scripts/package/mkspec
+++ b/scripts/package/mkspec
@@ -19,8 +19,7 @@
mkdir -p rpmbuild/SOURCES
cp linux.tar.gz rpmbuild/SOURCES
cp "${KCONFIG_CONFIG}" rpmbuild/SOURCES/config
- $(dirname $0)/gen-diff-patch rpmbuild/SOURCES/diff.patch rpmbuild/SOURCES/untracked.patch
- touch rpmbuild/SOURCES/diff.patch rpmbuild/SOURCES/untracked.patch
+ "${srctree}/scripts/package/gen-diff-patch" rpmbuild/SOURCES/diff.patch
fi
if grep -q CONFIG_MODULES=y include/config/auto.conf; then
@@ -56,7 +55,6 @@
$S Source0: linux.tar.gz
$S Source1: config
$S Source2: diff.patch
-$S Source3: untracked.patch
Provides: $PROVIDES
$S BuildRequires: bc binutils bison dwarves
$S BuildRequires: (elfutils-libelf-devel or libelf-devel) flex
@@ -94,12 +92,7 @@
$S %prep
$S %setup -q -n linux
$S cp %{SOURCE1} .config
-$S if [ -s %{SOURCE2} ]; then
-$S patch -p1 < %{SOURCE2}
-$S fi
-$S if [ -s %{SOURCE3} ]; then
-$S patch -p1 < %{SOURCE3}
-$S fi
+$S patch -p1 < %{SOURCE2}
$S
$S %build
$S $MAKE %{?_smp_mflags} KERNELRELEASE=$KERNELRELEASE KBUILD_BUILD_VERSION=%{release}
diff --git a/sound/firewire/tascam/tascam-stream.c b/sound/firewire/tascam/tascam-stream.c
index 53e094c..dfe783d 100644
--- a/sound/firewire/tascam/tascam-stream.c
+++ b/sound/firewire/tascam/tascam-stream.c
@@ -490,7 +490,7 @@ int snd_tscm_stream_start_duplex(struct snd_tscm *tscm, unsigned int rate)
// packet is important for media clock recovery.
err = amdtp_domain_start(&tscm->domain, tx_init_skip_cycles, true, true);
if (err < 0)
- return err;
+ goto error;
if (!amdtp_domain_wait_ready(&tscm->domain, READY_TIMEOUT_MS)) {
err = -ETIMEDOUT;
diff --git a/sound/i2c/cs8427.c b/sound/i2c/cs8427.c
index 65012af6..f58b14b 100644
--- a/sound/i2c/cs8427.c
+++ b/sound/i2c/cs8427.c
@@ -561,10 +561,13 @@ int snd_cs8427_iec958_active(struct snd_i2c_device *cs8427, int active)
if (snd_BUG_ON(!cs8427))
return -ENXIO;
chip = cs8427->private_data;
- if (active)
+ if (active) {
memcpy(chip->playback.pcm_status,
chip->playback.def_status, 24);
- chip->playback.pcm_ctl->vd[0].access &= ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
+ chip->playback.pcm_ctl->vd[0].access &= ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
+ } else {
+ chip->playback.pcm_ctl->vd[0].access |= SNDRV_CTL_ELEM_ACCESS_INACTIVE;
+ }
snd_ctl_notify(cs8427->bus->card,
SNDRV_CTL_EVENT_MASK_VALUE | SNDRV_CTL_EVENT_MASK_INFO,
&chip->playback.pcm_ctl->id);
diff --git a/sound/pci/emu10k1/emupcm.c b/sound/pci/emu10k1/emupcm.c
index 48af77a..6ec394f 100644
--- a/sound/pci/emu10k1/emupcm.c
+++ b/sound/pci/emu10k1/emupcm.c
@@ -1236,7 +1236,7 @@ static int snd_emu10k1_capture_mic_close(struct snd_pcm_substream *substream)
{
struct snd_emu10k1 *emu = snd_pcm_substream_chip(substream);
- emu->capture_interrupt = NULL;
+ emu->capture_mic_interrupt = NULL;
emu->pcm_capture_mic_substream = NULL;
return 0;
}
@@ -1344,7 +1344,7 @@ static int snd_emu10k1_capture_efx_close(struct snd_pcm_substream *substream)
{
struct snd_emu10k1 *emu = snd_pcm_substream_chip(substream);
- emu->capture_interrupt = NULL;
+ emu->capture_efx_interrupt = NULL;
emu->pcm_capture_efx_substream = NULL;
return 0;
}
@@ -1781,17 +1781,21 @@ int snd_emu10k1_pcm_efx(struct snd_emu10k1 *emu, int device)
struct snd_kcontrol *kctl;
int err;
- err = snd_pcm_new(emu->card, "emu10k1 efx", device, 8, 1, &pcm);
+ err = snd_pcm_new(emu->card, "emu10k1 efx", device, emu->audigy ? 0 : 8, 1, &pcm);
if (err < 0)
return err;
pcm->private_data = emu;
- snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK, &snd_emu10k1_fx8010_playback_ops);
+ if (!emu->audigy)
+ snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK, &snd_emu10k1_fx8010_playback_ops);
snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE, &snd_emu10k1_capture_efx_ops);
pcm->info_flags = 0;
- strcpy(pcm->name, "Multichannel Capture/PT Playback");
+ if (emu->audigy)
+ strcpy(pcm->name, "Multichannel Capture");
+ else
+ strcpy(pcm->name, "Multichannel Capture/PT Playback");
emu->pcm_efx = pcm;
/* EFX capture - record the "FXBUS2" channels, by default we connect the EXTINs
diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c
index 4ffa3a5..5c69803 100644
--- a/sound/pci/hda/patch_hdmi.c
+++ b/sound/pci/hda/patch_hdmi.c
@@ -4604,7 +4604,7 @@ HDA_CODEC_ENTRY(0x80862814, "DG1 HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x80862815, "Alderlake HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x80862816, "Rocketlake HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x80862818, "Raptorlake HDMI", patch_i915_tgl_hdmi),
-HDA_CODEC_ENTRY(0x80862819, "DG2 HDMI", patch_i915_adlp_hdmi),
+HDA_CODEC_ENTRY(0x80862819, "DG2 HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x8086281a, "Jasperlake HDMI", patch_i915_icl_hdmi),
HDA_CODEC_ENTRY(0x8086281b, "Elkhartlake HDMI", patch_i915_icl_hdmi),
HDA_CODEC_ENTRY(0x8086281c, "Alderlake-P HDMI", patch_i915_adlp_hdmi),
diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c
index 26187f5..3b9f077 100644
--- a/sound/pci/hda/patch_realtek.c
+++ b/sound/pci/hda/patch_realtek.c
@@ -6960,6 +6960,8 @@ enum {
ALC269_FIXUP_DELL_M101Z,
ALC269_FIXUP_SKU_IGNORE,
ALC269_FIXUP_ASUS_G73JW,
+ ALC269_FIXUP_ASUS_N7601ZM_PINS,
+ ALC269_FIXUP_ASUS_N7601ZM,
ALC269_FIXUP_LENOVO_EAPD,
ALC275_FIXUP_SONY_HWEQ,
ALC275_FIXUP_SONY_DISABLE_AAMIX,
@@ -7256,6 +7258,29 @@ static const struct hda_fixup alc269_fixups[] = {
{ }
}
},
+ [ALC269_FIXUP_ASUS_N7601ZM_PINS] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x19, 0x03A11050 },
+ { 0x1a, 0x03A11C30 },
+ { 0x21, 0x03211420 },
+ { }
+ }
+ },
+ [ALC269_FIXUP_ASUS_N7601ZM] = {
+ .type = HDA_FIXUP_VERBS,
+ .v.verbs = (const struct hda_verb[]) {
+ {0x20, AC_VERB_SET_COEF_INDEX, 0x62},
+ {0x20, AC_VERB_SET_PROC_COEF, 0xa007},
+ {0x20, AC_VERB_SET_COEF_INDEX, 0x10},
+ {0x20, AC_VERB_SET_PROC_COEF, 0x8420},
+ {0x20, AC_VERB_SET_COEF_INDEX, 0x0f},
+ {0x20, AC_VERB_SET_PROC_COEF, 0x7774},
+ { }
+ },
+ .chained = true,
+ .chain_id = ALC269_FIXUP_ASUS_N7601ZM_PINS,
+ },
[ALC269_FIXUP_LENOVO_EAPD] = {
.type = HDA_FIXUP_VERBS,
.v.verbs = (const struct hda_verb[]) {
@@ -9466,6 +9491,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1043, 0x1271, "ASUS X430UN", ALC256_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1043, 0x1290, "ASUS X441SA", ALC233_FIXUP_EAPD_COEF_AND_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1043, 0x12a0, "ASUS X441UV", ALC233_FIXUP_EAPD_COEF_AND_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1043, 0x12a3, "Asus N7691ZM", ALC269_FIXUP_ASUS_N7601ZM),
SND_PCI_QUIRK(0x1043, 0x12af, "ASUS UX582ZS", ALC245_FIXUP_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x12e0, "ASUS X541SA", ALC256_FIXUP_ASUS_MIC),
SND_PCI_QUIRK(0x1043, 0x12f0, "ASUS X541UV", ALC256_FIXUP_ASUS_MIC),
@@ -9663,6 +9689,9 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x17aa, 0x22f1, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x17aa, 0x22f2, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x17aa, 0x22f3, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x17aa, 0x2318, "Thinkpad Z13 Gen2", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x17aa, 0x2319, "Thinkpad Z16 Gen2", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x17aa, 0x231a, "Thinkpad Z16 Gen2", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x17aa, 0x30bb, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x30e2, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x310c, "ThinkCentre Station", ALC294_FIXUP_LENOVO_MIC_LOCATION),
diff --git a/sound/pci/hda/patch_sigmatel.c b/sound/pci/hda/patch_sigmatel.c
index a794a01..61258b0 100644
--- a/sound/pci/hda/patch_sigmatel.c
+++ b/sound/pci/hda/patch_sigmatel.c
@@ -1707,6 +1707,7 @@ static const struct snd_pci_quirk stac925x_fixup_tbl[] = {
};
static const struct hda_pintbl ref92hd73xx_pin_configs[] = {
+ // Port A-H
{ 0x0a, 0x02214030 },
{ 0x0b, 0x02a19040 },
{ 0x0c, 0x01a19020 },
@@ -1715,9 +1716,12 @@ static const struct hda_pintbl ref92hd73xx_pin_configs[] = {
{ 0x0f, 0x01014010 },
{ 0x10, 0x01014020 },
{ 0x11, 0x01014030 },
+ // CD in
{ 0x12, 0x02319040 },
+ // Digial Mic ins
{ 0x13, 0x90a000f0 },
{ 0x14, 0x90a000f0 },
+ // Digital outs
{ 0x22, 0x01452050 },
{ 0x23, 0x01452050 },
{}
@@ -1758,6 +1762,7 @@ static const struct hda_pintbl alienware_m17x_pin_configs[] = {
};
static const struct hda_pintbl intel_dg45id_pin_configs[] = {
+ // Analog outputs
{ 0x0a, 0x02214230 },
{ 0x0b, 0x02A19240 },
{ 0x0c, 0x01013214 },
@@ -1765,6 +1770,9 @@ static const struct hda_pintbl intel_dg45id_pin_configs[] = {
{ 0x0e, 0x01A19250 },
{ 0x0f, 0x01011212 },
{ 0x10, 0x01016211 },
+ // Digital output
+ { 0x22, 0x01451380 },
+ { 0x23, 0x40f000f0 },
{}
};
@@ -1955,6 +1963,8 @@ static const struct snd_pci_quirk stac92hd73xx_fixup_tbl[] = {
"DFI LanParty", STAC_92HD73XX_REF),
SND_PCI_QUIRK(PCI_VENDOR_ID_DFI, 0x3101,
"DFI LanParty", STAC_92HD73XX_REF),
+ SND_PCI_QUIRK(PCI_VENDOR_ID_INTEL, 0x5001,
+ "Intel DP45SG", STAC_92HD73XX_INTEL),
SND_PCI_QUIRK(PCI_VENDOR_ID_INTEL, 0x5002,
"Intel DG45ID", STAC_92HD73XX_INTEL),
SND_PCI_QUIRK(PCI_VENDOR_ID_INTEL, 0x5003,
diff --git a/tools/Makefile b/tools/Makefile
index e497875..37e9f68 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -39,7 +39,7 @@
@echo ' turbostat - Intel CPU idle stats and freq reporting tool'
@echo ' usb - USB testing tools'
@echo ' virtio - vhost test module'
- @echo ' vm - misc vm tools'
+ @echo ' mm - misc mm tools'
@echo ' wmi - WMI interface examples'
@echo ' x86_energy_perf_policy - Intel energy policy tool'
@echo ''
@@ -69,7 +69,7 @@
cpupower: FORCE
$(call descend,power/$@)
-cgroup counter firewire hv guest bootconfig spi usb virtio vm bpf iio gpio objtool leds wmi pci firmware debugging tracing: FORCE
+cgroup counter firewire hv guest bootconfig spi usb virtio mm bpf iio gpio objtool leds wmi pci firmware debugging tracing: FORCE
$(call descend,$@)
bpf/%: FORCE
@@ -118,7 +118,7 @@
all: acpi cgroup counter cpupower gpio hv firewire \
perf selftests bootconfig spi turbostat usb \
- virtio vm bpf x86_energy_perf_policy \
+ virtio mm bpf x86_energy_perf_policy \
tmon freefall iio objtool kvm_stat wmi \
pci debugging tracing thermal thermometer thermal-engine
@@ -128,7 +128,7 @@
cpupower_install:
$(call descend,power/$(@:_install=),install)
-cgroup_install counter_install firewire_install gpio_install hv_install iio_install perf_install bootconfig_install spi_install usb_install virtio_install vm_install bpf_install objtool_install wmi_install pci_install debugging_install tracing_install:
+cgroup_install counter_install firewire_install gpio_install hv_install iio_install perf_install bootconfig_install spi_install usb_install virtio_install mm_install bpf_install objtool_install wmi_install pci_install debugging_install tracing_install:
$(call descend,$(@:_install=),install)
selftests_install:
@@ -158,7 +158,7 @@
install: acpi_install cgroup_install counter_install cpupower_install gpio_install \
hv_install firewire_install iio_install \
perf_install selftests_install turbostat_install usb_install \
- virtio_install vm_install bpf_install x86_energy_perf_policy_install \
+ virtio_install mm_install bpf_install x86_energy_perf_policy_install \
tmon_install freefall_install objtool_install kvm_stat_install \
wmi_install pci_install debugging_install intel-speed-select_install \
tracing_install thermometer_install thermal-engine_install
@@ -169,7 +169,7 @@
cpupower_clean:
$(call descend,power/cpupower,clean)
-cgroup_clean counter_clean hv_clean firewire_clean bootconfig_clean spi_clean usb_clean virtio_clean vm_clean wmi_clean bpf_clean iio_clean gpio_clean objtool_clean leds_clean pci_clean firmware_clean debugging_clean tracing_clean:
+cgroup_clean counter_clean hv_clean firewire_clean bootconfig_clean spi_clean usb_clean virtio_clean mm_clean wmi_clean bpf_clean iio_clean gpio_clean objtool_clean leds_clean pci_clean firmware_clean debugging_clean tracing_clean:
$(call descend,$(@:_clean=),clean)
libapi_clean:
@@ -211,7 +211,7 @@
clean: acpi_clean cgroup_clean counter_clean cpupower_clean hv_clean firewire_clean \
perf_clean selftests_clean turbostat_clean bootconfig_clean spi_clean usb_clean virtio_clean \
- vm_clean bpf_clean iio_clean x86_energy_perf_policy_clean tmon_clean \
+ mm_clean bpf_clean iio_clean x86_energy_perf_policy_clean tmon_clean \
freefall_clean build_clean libbpf_clean libsubcmd_clean \
gpio_clean objtool_clean leds_clean wmi_clean pci_clean firmware_clean debugging_clean \
intel-speed-select_clean tracing_clean thermal_clean thermometer_clean thermal-engine_clean
diff --git a/tools/arch/loongarch/include/uapi/asm/bitsperlong.h b/tools/arch/loongarch/include/uapi/asm/bitsperlong.h
index d4e32b3..00b4ba1 100644
--- a/tools/arch/loongarch/include/uapi/asm/bitsperlong.h
+++ b/tools/arch/loongarch/include/uapi/asm/bitsperlong.h
@@ -2,7 +2,7 @@
#ifndef __ASM_LOONGARCH_BITSPERLONG_H
#define __ASM_LOONGARCH_BITSPERLONG_H
-#define __BITS_PER_LONG (__SIZEOF_POINTER__ * 8)
+#define __BITS_PER_LONG (__SIZEOF_LONG__ * 8)
#include <asm-generic/bitsperlong.h>
diff --git a/tools/mm/page_owner_sort.c b/tools/mm/page_owner_sort.c
index 7c2ac12..9979889 100644
--- a/tools/mm/page_owner_sort.c
+++ b/tools/mm/page_owner_sort.c
@@ -857,7 +857,7 @@ int main(int argc, char **argv)
if (cull & CULL_PID || filter & FILTER_PID)
fprintf(fout, ", PID %d", list[i].pid);
if (cull & CULL_TGID || filter & FILTER_TGID)
- fprintf(fout, ", TGID %d", list[i].pid);
+ fprintf(fout, ", TGID %d", list[i].tgid);
if (cull & CULL_COMM || filter & FILTER_COMM)
fprintf(fout, ", task_comm_name: %s", list[i].comm);
if (cull & CULL_ALLOCATOR) {
diff --git a/tools/net/ynl/ethtool b/tools/net/ynl/ethtool.py
similarity index 98%
rename from tools/net/ynl/ethtool
rename to tools/net/ynl/ethtool.py
index 5fb1d67..6c9f7e3 100755
--- a/tools/net/ynl/ethtool
+++ b/tools/net/ynl/ethtool.py
@@ -152,8 +152,8 @@
global args
args = parser.parse_args()
- spec = '/usr/local/google/home/sdf/src/linux/Documentation/netlink/specs/ethtool.yaml'
- schema = '/usr/local/google/home/sdf/src/linux/Documentation/netlink/genetlink-legacy.yaml'
+ spec = '../../../Documentation/netlink/specs/ethtool.yaml'
+ schema = '../../../Documentation/netlink/genetlink-legacy.yaml'
ynl = YnlFamily(spec, schema)
diff --git a/tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh b/tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh
index 3569ff4..88162b4 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh
@@ -18,7 +18,6 @@
NUM_NETIFS=0
source $lib_dir/lib.sh
source $lib_dir/devlink_lib.sh
-source qos_lib.sh
swp=$NETIF_NO_CABLE
@@ -371,7 +370,7 @@
tc qdisc delete dev $swp root
}
-bail_on_lldpad
+bail_on_lldpad "configure DCB" "configure Qdiscs"
trap cleanup EXIT
setup_wait
diff --git a/tools/testing/selftests/drivers/net/mlxsw/qos_lib.sh b/tools/testing/selftests/drivers/net/mlxsw/qos_lib.sh
index faa5101..5ad092b 100644
--- a/tools/testing/selftests/drivers/net/mlxsw/qos_lib.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/qos_lib.sh
@@ -54,31 +54,3 @@
echo $ir $er
return $ret
}
-
-bail_on_lldpad()
-{
- if systemctl is-active --quiet lldpad; then
-
- cat >/dev/stderr <<-EOF
- WARNING: lldpad is running
-
- lldpad will likely configure DCB, and this test will
- configure Qdiscs. mlxsw does not support both at the
- same time, one of them is arbitrarily going to overwrite
- the other. That will cause spurious failures (or,
- unlikely, passes) of this test.
- EOF
-
- if [[ -z $ALLOW_LLDPAD ]]; then
- cat >/dev/stderr <<-EOF
-
- If you want to run the test anyway, please set
- an environment variable ALLOW_LLDPAD to a
- non-empty string.
- EOF
- exit 1
- else
- return
- fi
- fi
-}
diff --git a/tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh b/tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh
index f9858e2..42ce602 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh
@@ -79,7 +79,6 @@
NUM_NETIFS=6
source $lib_dir/lib.sh
source $lib_dir/devlink_lib.sh
-source qos_lib.sh
_1KB=1000
_100KB=$((100 * _1KB))
@@ -393,7 +392,7 @@
log_test "PFC"
}
-bail_on_lldpad
+bail_on_lldpad "configure DCB" "configure Qdiscs"
trap cleanup EXIT
setup_prepare
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_ets.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_ets.sh
index ceaa76b..139175f 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_ets.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_ets.sh
@@ -5,7 +5,6 @@
lib_dir=$(dirname $0)/../../../net/forwarding
source $lib_dir/sch_ets_core.sh
source $lib_dir/devlink_lib.sh
-source qos_lib.sh
ALL_TESTS="
ping_ipv4
@@ -78,5 +77,5 @@
done
}
-bail_on_lldpad
+bail_on_lldpad "configure DCB" "configure Qdiscs"
ets_run
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_red_core.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_red_core.sh
index 45b41b8f..299e06a 100644
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_red_core.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_red_core.sh
@@ -74,7 +74,6 @@
source $lib_dir/lib.sh
source $lib_dir/devlink_lib.sh
source mlxsw_lib.sh
-source qos_lib.sh
ipaddr()
{
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh
index 0d01c7c..8ecddaf 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh
@@ -166,7 +166,7 @@
uninstall_qdisc
}
-bail_on_lldpad
+bail_on_lldpad "configure DCB" "configure Qdiscs"
trap cleanup EXIT
setup_prepare
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh
index 8602053..159108d 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh
@@ -73,7 +73,7 @@
uninstall_qdisc
}
-bail_on_lldpad
+bail_on_lldpad "configure DCB" "configure Qdiscs"
trap cleanup EXIT
setup_prepare
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_ets.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_ets.sh
index c6ce0b4..ecc3664 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_ets.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_ets.sh
@@ -1,8 +1,10 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
-source qos_lib.sh
-bail_on_lldpad
+sch_tbf_pre_hook()
+{
+ bail_on_lldpad "configure DCB" "configure Qdiscs"
+}
lib_dir=$(dirname $0)/../../../net/forwarding
TCFLAGS=skip_sw
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_prio.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_prio.sh
index 8d245f3..2e0a4ef 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_prio.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_prio.sh
@@ -1,8 +1,10 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
-source qos_lib.sh
-bail_on_lldpad
+sch_tbf_pre_hook()
+{
+ bail_on_lldpad "configure DCB" "configure Qdiscs"
+}
lib_dir=$(dirname $0)/../../../net/forwarding
TCFLAGS=skip_sw
diff --git a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_root.sh b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_root.sh
index 0138860..6679a33 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_root.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/sch_tbf_root.sh
@@ -1,8 +1,10 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
-source qos_lib.sh
-bail_on_lldpad
+sch_tbf_pre_hook()
+{
+ bail_on_lldpad "configure DCB" "configure Qdiscs"
+}
lib_dir=$(dirname $0)/../../../net/forwarding
TCFLAGS=skip_sw
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 1de34ec..c12df57 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -83,6 +83,7 @@
TEST_GEN_FILES += ip_local_port_range
TEST_GEN_FILES += bind_wildcard
TEST_PROGS += test_vxlan_mdb.sh
+TEST_PROGS += test_bridge_neigh_suppress.sh
TEST_FILES := settings
diff --git a/tools/testing/selftests/net/forwarding/Makefile b/tools/testing/selftests/net/forwarding/Makefile
index 236f6b7..a474c60 100644
--- a/tools/testing/selftests/net/forwarding/Makefile
+++ b/tools/testing/selftests/net/forwarding/Makefile
@@ -15,6 +15,7 @@
custom_multipath_hash.sh \
dual_vxlan_bridge.sh \
ethtool_extended_state.sh \
+ ethtool_mm.sh \
ethtool.sh \
gre_custom_multipath_hash.sh \
gre_inner_v4_multipath.sh \
diff --git a/tools/testing/selftests/net/forwarding/ethtool_mm.sh b/tools/testing/selftests/net/forwarding/ethtool_mm.sh
new file mode 100755
index 0000000..c580ad6
--- /dev/null
+++ b/tools/testing/selftests/net/forwarding/ethtool_mm.sh
@@ -0,0 +1,288 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+ALL_TESTS="
+ manual_with_verification_h1_to_h2
+ manual_with_verification_h2_to_h1
+ manual_without_verification_h1_to_h2
+ manual_without_verification_h2_to_h1
+ manual_failed_verification_h1_to_h2
+ manual_failed_verification_h2_to_h1
+ lldp
+"
+
+NUM_NETIFS=2
+REQUIRE_MZ=no
+PREEMPTIBLE_PRIO=0
+source lib.sh
+
+traffic_test()
+{
+ local if=$1; shift
+ local src=$1; shift
+ local num_pkts=10000
+ local before=
+ local after=
+ local delta=
+
+ before=$(ethtool_std_stats_get $if "eth-mac" "FramesTransmittedOK" $src)
+
+ $MZ $if -q -c $num_pkts -p 64 -b bcast -t ip -R $PREEMPTIBLE_PRIO
+
+ after=$(ethtool_std_stats_get $if "eth-mac" "FramesTransmittedOK" $src)
+
+ delta=$((after - before))
+
+ # Allow an extra 1% tolerance for random packets sent by the stack
+ [ $delta -ge $num_pkts ] && [ $delta -le $((num_pkts + 100)) ]
+}
+
+manual_with_verification()
+{
+ local tx=$1; shift
+ local rx=$1; shift
+
+ RET=0
+
+ # It isn't completely clear from IEEE 802.3-2018 Figure 99-5: Transmit
+ # Processing state diagram whether the "send_r" variable (send response
+ # to verification frame) should be taken into consideration while the
+ # MAC Merge TX direction is disabled. That being said, at least the
+ # NXP ENETC does not, and requires tx-enabled on in order to respond to
+ # the link partner's verification frames.
+ ethtool --set-mm $rx tx-enabled on
+ ethtool --set-mm $tx verify-enabled on tx-enabled on
+
+ # Wait for verification to finish
+ sleep 1
+
+ ethtool --json --show-mm $tx | jq -r '.[]."verify-status"' | \
+ grep -q 'SUCCEEDED'
+ check_err "$?" "Verification did not succeed"
+
+ ethtool --json --show-mm $tx | jq -r '.[]."tx-active"' | grep -q 'true'
+ check_err "$?" "pMAC TX is not active"
+
+ traffic_test $tx "pmac"
+ check_err "$?" "Traffic did not get sent through $tx's pMAC"
+
+ ethtool --set-mm $tx verify-enabled off tx-enabled off
+ ethtool --set-mm $rx tx-enabled off
+
+ log_test "Manual configuration with verification: $tx to $rx"
+}
+
+manual_with_verification_h1_to_h2()
+{
+ manual_with_verification $h1 $h2
+}
+
+manual_with_verification_h2_to_h1()
+{
+ manual_with_verification $h2 $h1
+}
+
+manual_without_verification()
+{
+ local tx=$1; shift
+ local rx=$1; shift
+
+ RET=0
+
+ ethtool --set-mm $tx verify-enabled off tx-enabled on
+
+ ethtool --json --show-mm $tx | jq -r '.[]."verify-status"' | \
+ grep -q 'DISABLED'
+ check_err "$?" "Verification is not disabled"
+
+ ethtool --json --show-mm $tx | jq -r '.[]."tx-active"' | grep -q 'true'
+ check_err "$?" "pMAC TX is not active"
+
+ traffic_test $tx "pmac"
+ check_err "$?" "Traffic did not get sent through $tx's pMAC"
+
+ ethtool --set-mm $tx verify-enabled off tx-enabled off
+
+ log_test "Manual configuration without verification: $tx to $rx"
+}
+
+manual_without_verification_h1_to_h2()
+{
+ manual_without_verification $h1 $h2
+}
+
+manual_without_verification_h2_to_h1()
+{
+ manual_without_verification $h2 $h1
+}
+
+manual_failed_verification()
+{
+ local tx=$1; shift
+ local rx=$1; shift
+
+ RET=0
+
+ ethtool --set-mm $rx pmac-enabled off
+ ethtool --set-mm $tx verify-enabled on tx-enabled on
+
+ # Wait for verification to time out
+ sleep 1
+
+ ethtool --json --show-mm $tx | jq -r '.[]."verify-status"' | \
+ grep -q 'SUCCEEDED'
+ check_fail "$?" "Verification succeeded when it shouldn't have"
+
+ ethtool --json --show-mm $tx | jq -r '.[]."tx-active"' | grep -q 'true'
+ check_fail "$?" "pMAC TX is active when it shouldn't have"
+
+ traffic_test $tx "emac"
+ check_err "$?" "Traffic did not get sent through $tx's eMAC"
+
+ ethtool --set-mm $tx verify-enabled off tx-enabled off
+ ethtool --set-mm $rx pmac-enabled on
+
+ log_test "Manual configuration with failed verification: $tx to $rx"
+}
+
+manual_failed_verification_h1_to_h2()
+{
+ manual_failed_verification $h1 $h2
+}
+
+manual_failed_verification_h2_to_h1()
+{
+ manual_failed_verification $h2 $h1
+}
+
+lldp_change_add_frag_size()
+{
+ local add_frag_size=$1
+
+ lldptool -T -i $h1 -V addEthCaps addFragSize=$add_frag_size >/dev/null
+ # Wait for TLVs to be received
+ sleep 2
+ lldptool -i $h2 -t -n -V addEthCaps | \
+ grep -q "Additional fragment size: $add_frag_size"
+}
+
+lldp()
+{
+ RET=0
+
+ systemctl start lldpad
+
+ # Configure the interfaces to receive and transmit LLDPDUs
+ lldptool -L -i $h1 adminStatus=rxtx >/dev/null
+ lldptool -L -i $h2 adminStatus=rxtx >/dev/null
+
+ # Enable the transmission of Additional Ethernet Capabilities TLV
+ lldptool -T -i $h1 -V addEthCaps enableTx=yes >/dev/null
+ lldptool -T -i $h2 -V addEthCaps enableTx=yes >/dev/null
+
+ # Wait for TLVs to be received
+ sleep 2
+
+ lldptool -i $h1 -t -n -V addEthCaps | \
+ grep -q "Preemption capability active"
+ check_err "$?" "$h1 pMAC TX is not active"
+
+ lldptool -i $h2 -t -n -V addEthCaps | \
+ grep -q "Preemption capability active"
+ check_err "$?" "$h2 pMAC TX is not active"
+
+ lldp_change_add_frag_size 3
+ check_err "$?" "addFragSize 3"
+
+ lldp_change_add_frag_size 2
+ check_err "$?" "addFragSize 2"
+
+ lldp_change_add_frag_size 1
+ check_err "$?" "addFragSize 1"
+
+ lldp_change_add_frag_size 0
+ check_err "$?" "addFragSize 0"
+
+ traffic_test $h1 "pmac"
+ check_err "$?" "Traffic did not get sent through $h1's pMAC"
+
+ traffic_test $h2 "pmac"
+ check_err "$?" "Traffic did not get sent through $h2's pMAC"
+
+ systemctl stop lldpad
+
+ log_test "LLDP"
+}
+
+h1_create()
+{
+ ip link set dev $h1 up
+
+ tc qdisc add dev $h1 root mqprio num_tc 4 map 0 1 2 3 \
+ queues 1@0 1@1 1@2 1@3 \
+ fp P E E E \
+ hw 1
+
+ ethtool --set-mm $h1 pmac-enabled on tx-enabled off verify-enabled off
+}
+
+h2_create()
+{
+ ip link set dev $h2 up
+
+ ethtool --set-mm $h2 pmac-enabled on tx-enabled off verify-enabled off
+
+ tc qdisc add dev $h2 root mqprio num_tc 4 map 0 1 2 3 \
+ queues 1@0 1@1 1@2 1@3 \
+ fp P E E E \
+ hw 1
+}
+
+h1_destroy()
+{
+ ethtool --set-mm $h1 pmac-enabled off tx-enabled off verify-enabled off
+
+ tc qdisc del dev $h1 root
+
+ ip link set dev $h1 down
+}
+
+h2_destroy()
+{
+ tc qdisc del dev $h2 root
+
+ ethtool --set-mm $h2 pmac-enabled off tx-enabled off verify-enabled off
+
+ ip link set dev $h2 down
+}
+
+setup_prepare()
+{
+ check_ethtool_mm_support
+ check_tc_fp_support
+ require_command lldptool
+ bail_on_lldpad "autoconfigure the MAC Merge layer" "configure it manually"
+
+ h1=${NETIFS[p1]}
+ h2=${NETIFS[p2]}
+
+ h1_create
+ h2_create
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ h2_destroy
+ h1_destroy
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
index d47499b..057c3d0 100755
--- a/tools/testing/selftests/net/forwarding/lib.sh
+++ b/tools/testing/selftests/net/forwarding/lib.sh
@@ -120,6 +120,15 @@
fi
}
+check_tc_fp_support()
+{
+ tc qdisc add dev lo mqprio help 2>&1 | grep -q "fp "
+ if [[ $? -ne 0 ]]; then
+ echo "SKIP: iproute2 too old; tc is missing frame preemption support"
+ exit $ksft_skip
+ fi
+}
+
check_ethtool_lanes_support()
{
ethtool --help 2>&1| grep lanes &> /dev/null
@@ -129,6 +138,15 @@
fi
}
+check_ethtool_mm_support()
+{
+ ethtool --help 2>&1| grep -- '--show-mm' &> /dev/null
+ if [[ $? -ne 0 ]]; then
+ echo "SKIP: ethtool too old; it is missing MAC Merge layer support"
+ exit $ksft_skip
+ fi
+}
+
check_locked_port_support()
{
if ! bridge -d link show | grep -q " locked"; then
@@ -787,6 +805,17 @@
ethtool -S $dev | grep "^ *$stat:" | head -n 1 | cut -d: -f2
}
+ethtool_std_stats_get()
+{
+ local dev=$1; shift
+ local grp=$1; shift
+ local name=$1; shift
+ local src=$1; shift
+
+ ethtool --json -S $dev --groups $grp -- --src $src | \
+ jq '.[]."'"$grp"'"."'$name'"'
+}
+
qdisc_stats_get()
{
local dev=$1; shift
@@ -1887,3 +1916,34 @@
payload_template_expand_checksum "$hbh$icmpv6" $checksum
}
+
+bail_on_lldpad()
+{
+ local reason1="$1"; shift
+ local reason2="$1"; shift
+
+ if systemctl is-active --quiet lldpad; then
+
+ cat >/dev/stderr <<-EOF
+ WARNING: lldpad is running
+
+ lldpad will likely $reason1, and this test will
+ $reason2. Both are not supported at the same time,
+ one of them is arbitrarily going to overwrite the
+ other. That will cause spurious failures (or, unlikely,
+ passes) of this test.
+ EOF
+
+ if [[ -z $ALLOW_LLDPAD ]]; then
+ cat >/dev/stderr <<-EOF
+
+ If you want to run the test anyway, please set
+ an environment variable ALLOW_LLDPAD to a
+ non-empty string.
+ EOF
+ exit 1
+ else
+ return
+ fi
+ fi
+}
diff --git a/tools/testing/selftests/net/forwarding/sch_tbf_etsprio.sh b/tools/testing/selftests/net/forwarding/sch_tbf_etsprio.sh
index 75a37c1..df9bcd6 100644
--- a/tools/testing/selftests/net/forwarding/sch_tbf_etsprio.sh
+++ b/tools/testing/selftests/net/forwarding/sch_tbf_etsprio.sh
@@ -57,6 +57,10 @@
tc qdisc del dev $swp2 root
}
+if type -t sch_tbf_pre_hook >/dev/null; then
+ sch_tbf_pre_hook
+fi
+
trap cleanup EXIT
setup_prepare
diff --git a/tools/testing/selftests/net/forwarding/sch_tbf_root.sh b/tools/testing/selftests/net/forwarding/sch_tbf_root.sh
index 72aa21b..96c997b 100755
--- a/tools/testing/selftests/net/forwarding/sch_tbf_root.sh
+++ b/tools/testing/selftests/net/forwarding/sch_tbf_root.sh
@@ -23,6 +23,10 @@
tc qdisc del dev $swp2 root
}
+if type -t sch_tbf_pre_hook >/dev/null; then
+ sch_tbf_pre_hook
+fi
+
trap cleanup EXIT
setup_prepare
diff --git a/tools/testing/selftests/net/mptcp/mptcp_connect.c b/tools/testing/selftests/net/mptcp/mptcp_connect.c
index b25a314..c7f9ebe 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_connect.c
+++ b/tools/testing/selftests/net/mptcp/mptcp_connect.c
@@ -106,8 +106,8 @@ static struct cfg_sockopt_types cfg_sockopt_types;
static void die_usage(void)
{
fprintf(stderr, "Usage: mptcp_connect [-6] [-c cmsg] [-f offset] [-i file] [-I num] [-j] [-l] "
- "[-m mode] [-M mark] [-o option] [-p port] [-P mode] [-j] [-l] [-r num] "
- "[-s MPTCP|TCP] [-S num] [-r num] [-t num] [-T num] [-u] [-w sec] connect_address\n");
+ "[-m mode] [-M mark] [-o option] [-p port] [-P mode] [-r num] [-R num] "
+ "[-s MPTCP|TCP] [-S num] [-t num] [-T num] [-w sec] connect_address\n");
fprintf(stderr, "\t-6 use ipv6\n");
fprintf(stderr, "\t-c cmsg -- test cmsg type <cmsg>\n");
fprintf(stderr, "\t-f offset -- stop the I/O after receiving and sending the specified amount "
@@ -126,13 +126,13 @@ static void die_usage(void)
fprintf(stderr, "\t-p num -- use port num\n");
fprintf(stderr,
"\t-P [saveWithPeek|saveAfterPeek] -- save data with/after MSG_PEEK form tcp socket\n");
- fprintf(stderr, "\t-t num -- set poll timeout to num\n");
- fprintf(stderr, "\t-T num -- set expected runtime to num ms\n");
fprintf(stderr, "\t-r num -- enable slow mode, limiting each write to num bytes "
"-- for remove addr tests\n");
fprintf(stderr, "\t-R num -- set SO_RCVBUF to num\n");
fprintf(stderr, "\t-s [MPTCP|TCP] -- use mptcp(default) or tcp sockets\n");
fprintf(stderr, "\t-S num -- set SO_SNDBUF to num\n");
+ fprintf(stderr, "\t-t num -- set poll timeout to num\n");
+ fprintf(stderr, "\t-T num -- set expected runtime to num ms\n");
fprintf(stderr, "\t-w num -- wait num sec before closing the socket\n");
exit(1);
}
diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index fafd19e..26310c1 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -6,6 +6,10 @@
# address all other issues detected by shellcheck.
#shellcheck disable=SC2086
+# ShellCheck incorrectly believes that most of the code here is unreachable
+# because it's invoked by variable name, see how the "tests" array is used
+#shellcheck disable=SC2317
+
ret=0
sin=""
sinfail=""
@@ -371,8 +375,9 @@
local line
if [ -n "$bytes" ]; then
+ local out_size
# when truncating we must check the size explicitly
- local out_size=$(wc -c $out | awk '{print $1}')
+ out_size=$(wc -c $out | awk '{print $1}')
if [ $out_size -ne $bytes ]; then
echo "[ FAIL ] $what output file has wrong size ($out_size, $bytes)"
fail_test
@@ -500,6 +505,7 @@
kill_tests_wait()
{
+ #shellcheck disable=SC2046
kill -SIGUSR1 $(ip netns pids $ns2) $(ip netns pids $ns1)
wait
}
@@ -1703,7 +1709,7 @@
cnt1=$(ss -N $ns1 -tOni | grep -c token)
cnt2=$(ss -N $ns2 -tOni | grep -c token)
- if [ "$cnt1" != "$subflow_nr" -o "$cnt2" != "$subflow_nr" ]; then
+ if [ "$cnt1" != "$subflow_nr" ] || [ "$cnt2" != "$subflow_nr" ]; then
echo "[fail] got $cnt1:$cnt2 subflows expected $subflow_nr"
fail_test
dump_stats=1
diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh b/tools/testing/selftests/net/openvswitch/openvswitch.sh
index 7ce4670..3117a4b 100755
--- a/tools/testing/selftests/net/openvswitch/openvswitch.sh
+++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh
@@ -11,7 +11,8 @@
TRACING=0
tests="
- netlink_checks ovsnl: validate netlink attrs and settings"
+ netlink_checks ovsnl: validate netlink attrs and settings
+ upcall_interfaces ovs: test the upcall interfaces"
info() {
[ $VERBOSE = 0 ] || echo $*
@@ -70,6 +71,62 @@
on_exit "ovs_sbx $sbxname python3 $ovs_base/ovs-dpctl.py del-dp $1;"
}
+ovs_add_if () {
+ info "Adding IF to DP: br:$2 if:$3"
+ if [ "$4" != "-u" ]; then
+ ovs_sbx "$1" python3 $ovs_base/ovs-dpctl.py add-if "$2" "$3" \
+ || return 1
+ else
+ python3 $ovs_base/ovs-dpctl.py add-if \
+ -u "$2" "$3" >$ovs_dir/$3.out 2>$ovs_dir/$3.err &
+ pid=$!
+ on_exit "ovs_sbx $1 kill -TERM $pid 2>/dev/null"
+ fi
+}
+
+ovs_del_if () {
+ info "Deleting IF from DP: br:$2 if:$3"
+ ovs_sbx "$1" python3 $ovs_base/ovs-dpctl.py del-if "$2" "$3" || return 1
+}
+
+ovs_netns_spawn_daemon() {
+ sbx=$1
+ shift
+ netns=$1
+ shift
+ info "spawning cmd: $*"
+ ip netns exec $netns $* >> $ovs_dir/stdout 2>> $ovs_dir/stderr &
+ pid=$!
+ ovs_sbx "$sbx" on_exit "kill -TERM $pid 2>/dev/null"
+}
+
+ovs_add_netns_and_veths () {
+ info "Adding netns attached: sbx:$1 dp:$2 {$3, $4, $5}"
+ ovs_sbx "$1" ip netns add "$3" || return 1
+ on_exit "ovs_sbx $1 ip netns del $3"
+ ovs_sbx "$1" ip link add "$4" type veth peer name "$5" || return 1
+ on_exit "ovs_sbx $1 ip link del $4 >/dev/null 2>&1"
+ ovs_sbx "$1" ip link set "$4" up || return 1
+ ovs_sbx "$1" ip link set "$5" netns "$3" || return 1
+ ovs_sbx "$1" ip netns exec "$3" ip link set "$5" up || return 1
+
+ if [ "$6" != "" ]; then
+ ovs_sbx "$1" ip netns exec "$3" ip addr add "$6" dev "$5" \
+ || return 1
+ fi
+
+ if [ "$7" != "-u" ]; then
+ ovs_add_if "$1" "$2" "$4" || return 1
+ else
+ ovs_add_if "$1" "$2" "$4" -u || return 1
+ fi
+
+ [ $TRACING -eq 1 ] && ovs_netns_spawn_daemon "$1" "$ns" \
+ tcpdump -i any -s 65535
+
+ return 0
+}
+
usage() {
echo
echo "$0 [OPTIONS] [TEST]..."
@@ -101,6 +158,36 @@
return 1
fi
+ ovs_add_netns_and_veths "test_netlink_checks" nv0 left left0 l0 || \
+ return 1
+ ovs_add_netns_and_veths "test_netlink_checks" nv0 right right0 r0 || \
+ return 1
+ [ $(python3 $ovs_base/ovs-dpctl.py show nv0 | grep port | \
+ wc -l) == 3 ] || \
+ return 1
+ ovs_del_if "test_netlink_checks" nv0 right0 || return 1
+ [ $(python3 $ovs_base/ovs-dpctl.py show nv0 | grep port | \
+ wc -l) == 2 ] || \
+ return 1
+
+ return 0
+}
+
+test_upcall_interfaces() {
+ sbx_add "test_upcall_interfaces" || return 1
+
+ info "setting up new DP"
+ ovs_add_dp "test_upcall_interfaces" ui0 -V 2:1 || return 1
+
+ ovs_add_netns_and_veths "test_upcall_interfaces" ui0 upc left0 l0 \
+ 172.31.110.1/24 -u || return 1
+
+ sleep 1
+ info "sending arping"
+ ip netns exec upc arping -I l0 172.31.110.20 -c 1 \
+ >$ovs_dir/arping.stdout 2>$ovs_dir/arping.stderr
+
+ grep -E "MISS upcall\[0/yes\]: .*arp\(sip=172.31.110.1,tip=172.31.110.20,op=1,sha=" $ovs_dir/left0.out >/dev/null 2>&1 || return 1
return 0
}
diff --git a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
index 5d467d1..1c8b36b 100644
--- a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
+++ b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
@@ -6,15 +6,23 @@
import argparse
import errno
+import ipaddress
+import logging
+import multiprocessing
+import struct
import sys
+import time
try:
from pyroute2 import NDB
+ from pyroute2.netlink import NLA_F_NESTED
from pyroute2.netlink import NLM_F_ACK
+ from pyroute2.netlink import NLM_F_DUMP
from pyroute2.netlink import NLM_F_REQUEST
from pyroute2.netlink import genlmsg
from pyroute2.netlink import nla
+ from pyroute2.netlink import nlmsg_atoms
from pyroute2.netlink.exceptions import NetlinkError
from pyroute2.netlink.generic import GenericNetlinkSocket
except ModuleNotFoundError:
@@ -40,6 +48,36 @@
OVS_VPORT_CMD_GET = 3
OVS_VPORT_CMD_SET = 4
+OVS_FLOW_CMD_NEW = 1
+OVS_FLOW_CMD_DEL = 2
+OVS_FLOW_CMD_GET = 3
+OVS_FLOW_CMD_SET = 4
+
+
+def macstr(mac):
+ outstr = ":".join(["%02X" % i for i in mac])
+ return outstr
+
+
+def convert_mac(mac_str, mask=False):
+ if mac_str is None or mac_str == "":
+ mac_str = "00:00:00:00:00:00"
+ if mask is True and mac_str != "00:00:00:00:00:00":
+ mac_str = "FF:FF:FF:FF:FF:FF"
+ mac_split = mac_str.split(":")
+ ret = bytearray([int(i, 16) for i in mac_split])
+ return bytes(ret)
+
+
+def convert_ipv4(ip, mask=False):
+ if ip is None:
+ ip = 0
+ if mask is True:
+ if ip != 0:
+ ip = int(ipaddress.IPv4Address(ip)) & 0xFFFFFFFF
+
+ return int(ipaddress.IPv4Address(ip))
+
class ovs_dp_msg(genlmsg):
# include the OVS version
@@ -49,8 +87,893 @@
fields = genlmsg.fields + (("dpifindex", "I"),)
-class OvsDatapath(GenericNetlinkSocket):
+class ovsactions(nla):
+ nla_flags = NLA_F_NESTED
+ nla_map = (
+ ("OVS_ACTION_ATTR_UNSPEC", "none"),
+ ("OVS_ACTION_ATTR_OUTPUT", "uint32"),
+ ("OVS_ACTION_ATTR_USERSPACE", "userspace"),
+ ("OVS_ACTION_ATTR_SET", "none"),
+ ("OVS_ACTION_ATTR_PUSH_VLAN", "none"),
+ ("OVS_ACTION_ATTR_POP_VLAN", "flag"),
+ ("OVS_ACTION_ATTR_SAMPLE", "none"),
+ ("OVS_ACTION_ATTR_RECIRC", "uint32"),
+ ("OVS_ACTION_ATTR_HASH", "none"),
+ ("OVS_ACTION_ATTR_PUSH_MPLS", "none"),
+ ("OVS_ACTION_ATTR_POP_MPLS", "flag"),
+ ("OVS_ACTION_ATTR_SET_MASKED", "none"),
+ ("OVS_ACTION_ATTR_CT", "ctact"),
+ ("OVS_ACTION_ATTR_TRUNC", "uint32"),
+ ("OVS_ACTION_ATTR_PUSH_ETH", "none"),
+ ("OVS_ACTION_ATTR_POP_ETH", "flag"),
+ ("OVS_ACTION_ATTR_CT_CLEAR", "flag"),
+ ("OVS_ACTION_ATTR_PUSH_NSH", "none"),
+ ("OVS_ACTION_ATTR_POP_NSH", "flag"),
+ ("OVS_ACTION_ATTR_METER", "none"),
+ ("OVS_ACTION_ATTR_CLONE", "none"),
+ ("OVS_ACTION_ATTR_CHECK_PKT_LEN", "none"),
+ ("OVS_ACTION_ATTR_ADD_MPLS", "none"),
+ ("OVS_ACTION_ATTR_DEC_TTL", "none"),
+ )
+
+ class ctact(nla):
+ nla_flags = NLA_F_NESTED
+
+ nla_map = (
+ ("OVS_CT_ATTR_NONE", "none"),
+ ("OVS_CT_ATTR_COMMIT", "flag"),
+ ("OVS_CT_ATTR_ZONE", "uint16"),
+ ("OVS_CT_ATTR_MARK", "none"),
+ ("OVS_CT_ATTR_LABELS", "none"),
+ ("OVS_CT_ATTR_HELPER", "asciiz"),
+ ("OVS_CT_ATTR_NAT", "natattr"),
+ ("OVS_CT_ATTR_FORCE_COMMIT", "flag"),
+ ("OVS_CT_ATTR_EVENTMASK", "uint32"),
+ ("OVS_CT_ATTR_TIMEOUT", "asciiz"),
+ )
+
+ class natattr(nla):
+ nla_flags = NLA_F_NESTED
+
+ nla_map = (
+ ("OVS_NAT_ATTR_NONE", "none"),
+ ("OVS_NAT_ATTR_SRC", "flag"),
+ ("OVS_NAT_ATTR_DST", "flag"),
+ ("OVS_NAT_ATTR_IP_MIN", "ipaddr"),
+ ("OVS_NAT_ATTR_IP_MAX", "ipaddr"),
+ ("OVS_NAT_ATTR_PROTO_MIN", "uint16"),
+ ("OVS_NAT_ATTR_PROTO_MAX", "uint16"),
+ ("OVS_NAT_ATTR_PERSISTENT", "flag"),
+ ("OVS_NAT_ATTR_PROTO_HASH", "flag"),
+ ("OVS_NAT_ATTR_PROTO_RANDOM", "flag"),
+ )
+
+ def dpstr(self, more=False):
+ print_str = "nat("
+
+ if self.get_attr("OVS_NAT_ATTR_SRC"):
+ print_str += "src"
+ elif self.get_attr("OVS_NAT_ATTR_DST"):
+ print_str += "dst"
+ else:
+ print_str += "XXX-unknown-nat"
+
+ if self.get_attr("OVS_NAT_ATTR_IP_MIN") or self.get_attr(
+ "OVS_NAT_ATTR_IP_MAX"
+ ):
+ if self.get_attr("OVS_NAT_ATTR_IP_MIN"):
+ print_str += "=%s," % str(
+ self.get_attr("OVS_NAT_ATTR_IP_MIN")
+ )
+
+ if self.get_attr("OVS_NAT_ATTR_IP_MAX"):
+ print_str += "-%s," % str(
+ self.get_attr("OVS_NAT_ATTR_IP_MAX")
+ )
+ else:
+ print_str += ","
+
+ if self.get_attr("OVS_NAT_ATTR_PROTO_MIN"):
+ print_str += "proto_min=%d," % self.get_attr(
+ "OVS_NAT_ATTR_PROTO_MIN"
+ )
+
+ if self.get_attr("OVS_NAT_ATTR_PROTO_MAX"):
+ print_str += "proto_max=%d," % self.get_attr(
+ "OVS_NAT_ATTR_PROTO_MAX"
+ )
+
+ if self.get_attr("OVS_NAT_ATTR_PERSISTENT"):
+ print_str += "persistent,"
+ if self.get_attr("OVS_NAT_ATTR_HASH"):
+ print_str += "hash,"
+ if self.get_attr("OVS_NAT_ATTR_RANDOM"):
+ print_str += "random"
+ print_str += ")"
+ return print_str
+
+ def dpstr(self, more=False):
+ print_str = "ct("
+
+ if self.get_attr("OVS_CT_ATTR_COMMIT") is not None:
+ print_str += "commit,"
+ if self.get_attr("OVS_CT_ATTR_ZONE") is not None:
+ print_str += "zone=%d," % self.get_attr("OVS_CT_ATTR_ZONE")
+ if self.get_attr("OVS_CT_ATTR_HELPER") is not None:
+ print_str += "helper=%s," % self.get_attr("OVS_CT_ATTR_HELPER")
+ if self.get_attr("OVS_CT_ATTR_NAT") is not None:
+ print_str += self.get_attr("OVS_CT_ATTR_NAT").dpstr(more)
+ print_str += ","
+ if self.get_attr("OVS_CT_ATTR_FORCE_COMMIT") is not None:
+ print_str += "force,"
+ if self.get_attr("OVS_CT_ATTR_EVENTMASK") is not None:
+ print_str += "emask=0x%X," % self.get_attr(
+ "OVS_CT_ATTR_EVENTMASK"
+ )
+ if self.get_attr("OVS_CT_ATTR_TIMEOUT") is not None:
+ print_str += "timeout=%s" % self.get_attr(
+ "OVS_CT_ATTR_TIMEOUT"
+ )
+ print_str += ")"
+ return print_str
+
+ class userspace(nla):
+ nla_flags = NLA_F_NESTED
+
+ nla_map = (
+ ("OVS_USERSPACE_ATTR_UNUSED", "none"),
+ ("OVS_USERSPACE_ATTR_PID", "uint32"),
+ ("OVS_USERSPACE_ATTR_USERDATA", "array(uint8)"),
+ ("OVS_USERSPACE_ATTR_EGRESS_TUN_PORT", "uint32"),
+ )
+
+ def dpstr(self, more=False):
+ print_str = "userspace("
+ if self.get_attr("OVS_USERSPACE_ATTR_PID") is not None:
+ print_str += "pid=%d," % self.get_attr(
+ "OVS_USERSPACE_ATTR_PID"
+ )
+ if self.get_attr("OVS_USERSPACE_ATTR_USERDATA") is not None:
+ print_str += "userdata="
+ for f in self.get_attr("OVS_USERSPACE_ATTR_USERDATA"):
+ print_str += "%x." % f
+ if self.get_attr("OVS_USERSPACE_ATTR_TUN_PORT") is not None:
+ print_str += "egress_tun_port=%d" % self.get_attr(
+ "OVS_USERSPACE_ATTR_TUN_PORT"
+ )
+ print_str += ")"
+ return print_str
+
+ def dpstr(self, more=False):
+ print_str = ""
+
+ for field in self.nla_map:
+ if field[1] == "none" or self.get_attr(field[0]) is None:
+ continue
+ if print_str != "":
+ print_str += ","
+
+ if field[1] == "uint32":
+ if field[0] == "OVS_ACTION_ATTR_OUTPUT":
+ print_str += "%d" % int(self.get_attr(field[0]))
+ elif field[0] == "OVS_ACTION_ATTR_RECIRC":
+ print_str += "recirc(0x%x)" % int(self.get_attr(field[0]))
+ elif field[0] == "OVS_ACTION_ATTR_TRUNC":
+ print_str += "trunc(%d)" % int(self.get_attr(field[0]))
+ elif field[1] == "flag":
+ if field[0] == "OVS_ACTION_ATTR_CT_CLEAR":
+ print_str += "ct_clear"
+ elif field[0] == "OVS_ACTION_ATTR_POP_VLAN":
+ print_str += "pop_vlan"
+ elif field[0] == "OVS_ACTION_ATTR_POP_ETH":
+ print_str += "pop_eth"
+ elif field[0] == "OVS_ACTION_ATTR_POP_NSH":
+ print_str += "pop_nsh"
+ elif field[0] == "OVS_ACTION_ATTR_POP_MPLS":
+ print_str += "pop_mpls"
+ else:
+ datum = self.get_attr(field[0])
+ print_str += datum.dpstr(more)
+
+ return print_str
+
+
+class ovskey(nla):
+ nla_flags = NLA_F_NESTED
+ nla_map = (
+ ("OVS_KEY_ATTR_UNSPEC", "none"),
+ ("OVS_KEY_ATTR_ENCAP", "none"),
+ ("OVS_KEY_ATTR_PRIORITY", "uint32"),
+ ("OVS_KEY_ATTR_IN_PORT", "uint32"),
+ ("OVS_KEY_ATTR_ETHERNET", "ethaddr"),
+ ("OVS_KEY_ATTR_VLAN", "uint16"),
+ ("OVS_KEY_ATTR_ETHERTYPE", "be16"),
+ ("OVS_KEY_ATTR_IPV4", "ovs_key_ipv4"),
+ ("OVS_KEY_ATTR_IPV6", "ovs_key_ipv6"),
+ ("OVS_KEY_ATTR_TCP", "ovs_key_tcp"),
+ ("OVS_KEY_ATTR_UDP", "ovs_key_udp"),
+ ("OVS_KEY_ATTR_ICMP", "ovs_key_icmp"),
+ ("OVS_KEY_ATTR_ICMPV6", "ovs_key_icmpv6"),
+ ("OVS_KEY_ATTR_ARP", "ovs_key_arp"),
+ ("OVS_KEY_ATTR_ND", "ovs_key_nd"),
+ ("OVS_KEY_ATTR_SKB_MARK", "uint32"),
+ ("OVS_KEY_ATTR_TUNNEL", "none"),
+ ("OVS_KEY_ATTR_SCTP", "ovs_key_sctp"),
+ ("OVS_KEY_ATTR_TCP_FLAGS", "be16"),
+ ("OVS_KEY_ATTR_DP_HASH", "uint32"),
+ ("OVS_KEY_ATTR_RECIRC_ID", "uint32"),
+ ("OVS_KEY_ATTR_MPLS", "array(ovs_key_mpls)"),
+ ("OVS_KEY_ATTR_CT_STATE", "uint32"),
+ ("OVS_KEY_ATTR_CT_ZONE", "uint16"),
+ ("OVS_KEY_ATTR_CT_MARK", "uint32"),
+ ("OVS_KEY_ATTR_CT_LABELS", "none"),
+ ("OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV4", "ovs_key_ct_tuple_ipv4"),
+ ("OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV6", "ovs_key_ct_tuple_ipv6"),
+ ("OVS_KEY_ATTR_NSH", "none"),
+ ("OVS_KEY_ATTR_PACKET_TYPE", "none"),
+ ("OVS_KEY_ATTR_ND_EXTENSIONS", "none"),
+ ("OVS_KEY_ATTR_TUNNEL_INFO", "none"),
+ ("OVS_KEY_ATTR_IPV6_EXTENSIONS", "none"),
+ )
+
+ class ovs_key_proto(nla):
+ fields = (
+ ("src", "!H"),
+ ("dst", "!H"),
+ )
+
+ fields_map = (
+ ("src", "src", "%d", lambda x: int(x) if x is not None else 0),
+ ("dst", "dst", "%d", lambda x: int(x) if x is not None else 0),
+ )
+
+ def __init__(
+ self,
+ protostr,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ self.proto_str = protostr
+ nla.__init__(
+ self,
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ def dpstr(self, masked=None, more=False):
+ outstr = self.proto_str + "("
+ first = False
+ for f in self.fields_map:
+ if first:
+ outstr += ","
+ if masked is None:
+ outstr += "%s=" % f[0]
+ if isinstance(f[2], str):
+ outstr += f[2] % self[f[1]]
+ else:
+ outstr += f[2](self[f[1]])
+ first = True
+ elif more or f[3](masked[f[1]]) != 0:
+ outstr += "%s=" % f[0]
+ if isinstance(f[2], str):
+ outstr += f[2] % self[f[1]]
+ else:
+ outstr += f[2](self[f[1]])
+ outstr += "/"
+ if isinstance(f[2], str):
+ outstr += f[2] % masked[f[1]]
+ else:
+ outstr += f[2](masked[f[1]])
+ first = True
+ outstr += ")"
+ return outstr
+
+ class ethaddr(ovs_key_proto):
+ fields = (
+ ("src", "!6s"),
+ ("dst", "!6s"),
+ )
+
+ fields_map = (
+ (
+ "src",
+ "src",
+ macstr,
+ lambda x: int.from_bytes(x, "big"),
+ convert_mac,
+ ),
+ (
+ "dst",
+ "dst",
+ macstr,
+ lambda x: int.from_bytes(x, "big"),
+ convert_mac,
+ ),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "eth",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_ipv4(ovs_key_proto):
+ fields = (
+ ("src", "!I"),
+ ("dst", "!I"),
+ ("proto", "B"),
+ ("tos", "B"),
+ ("ttl", "B"),
+ ("frag", "B"),
+ )
+
+ fields_map = (
+ (
+ "src",
+ "src",
+ lambda x: str(ipaddress.IPv4Address(x)),
+ int,
+ convert_ipv4,
+ ),
+ (
+ "dst",
+ "dst",
+ lambda x: str(ipaddress.IPv4Address(x)),
+ int,
+ convert_ipv4,
+ ),
+ ("proto", "proto", "%d", lambda x: int(x) if x is not None else 0),
+ ("tos", "tos", "%d", lambda x: int(x) if x is not None else 0),
+ ("ttl", "ttl", "%d", lambda x: int(x) if x is not None else 0),
+ ("frag", "frag", "%d", lambda x: int(x) if x is not None else 0),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "ipv4",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_ipv6(ovs_key_proto):
+ fields = (
+ ("src", "!16s"),
+ ("dst", "!16s"),
+ ("label", "!I"),
+ ("proto", "B"),
+ ("tclass", "B"),
+ ("hlimit", "B"),
+ ("frag", "B"),
+ )
+
+ fields_map = (
+ (
+ "src",
+ "src",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ lambda x: int.from_bytes(x, "big"),
+ lambda x: ipaddress.IPv6Address(x),
+ ),
+ (
+ "dst",
+ "dst",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ lambda x: int.from_bytes(x, "big"),
+ lambda x: ipaddress.IPv6Address(x),
+ ),
+ ("label", "label", "%d", int),
+ ("proto", "proto", "%d", int),
+ ("tclass", "tclass", "%d", int),
+ ("hlimit", "hlimit", "%d", int),
+ ("frag", "frag", "%d", int),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "ipv6",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_tcp(ovs_key_proto):
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "tcp",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_udp(ovs_key_proto):
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "udp",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_sctp(ovs_key_proto):
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "sctp",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_icmp(ovs_key_proto):
+ fields = (
+ ("type", "B"),
+ ("code", "B"),
+ )
+
+ fields_map = (
+ ("type", "type", "%d", int),
+ ("code", "code", "%d", int),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "icmp",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_icmpv6(ovs_key_icmp):
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "icmpv6",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_arp(ovs_key_proto):
+ fields = (
+ ("sip", "!I"),
+ ("tip", "!I"),
+ ("op", "!H"),
+ ("sha", "!6s"),
+ ("tha", "!6s"),
+ ("pad", "xx"),
+ )
+
+ fields_map = (
+ (
+ "sip",
+ "sip",
+ lambda x: str(ipaddress.IPv4Address(x)),
+ int,
+ convert_ipv4,
+ ),
+ (
+ "tip",
+ "tip",
+ lambda x: str(ipaddress.IPv4Address(x)),
+ int,
+ convert_ipv4,
+ ),
+ ("op", "op", "%d", lambda x: int(x) if x is not None else 0),
+ (
+ "sha",
+ "sha",
+ macstr,
+ lambda x: int.from_bytes(x, "big"),
+ convert_mac,
+ ),
+ (
+ "tha",
+ "tha",
+ macstr,
+ lambda x: int.from_bytes(x, "big"),
+ convert_mac,
+ ),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "arp",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_nd(ovs_key_proto):
+ fields = (
+ ("target", "!16s"),
+ ("sll", "!6s"),
+ ("tll", "!6s"),
+ )
+
+ fields_map = (
+ (
+ "target",
+ "target",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ lambda x: int.from_bytes(x, "big"),
+ ),
+ ("sll", "sll", macstr, lambda x: int.from_bytes(x, "big")),
+ ("tll", "tll", macstr, lambda x: int.from_bytes(x, "big")),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "nd",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_ct_tuple_ipv4(ovs_key_proto):
+ fields = (
+ ("src", "!I"),
+ ("dst", "!I"),
+ ("tp_src", "!H"),
+ ("tp_dst", "!H"),
+ ("proto", "B"),
+ )
+
+ fields_map = (
+ (
+ "src",
+ "src",
+ lambda x: str(ipaddress.IPv4Address(x)),
+ int,
+ ),
+ (
+ "dst",
+ "dst",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ int,
+ ),
+ ("tp_src", "tp_src", "%d", int),
+ ("tp_dst", "tp_dst", "%d", int),
+ ("proto", "proto", "%d", int),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "ct_tuple4",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_ct_tuple_ipv6(nla):
+ fields = (
+ ("src", "!16s"),
+ ("dst", "!16s"),
+ ("tp_src", "!H"),
+ ("tp_dst", "!H"),
+ ("proto", "B"),
+ )
+
+ fields_map = (
+ (
+ "src",
+ "src",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ lambda x: int.from_bytes(x, "big", convertmac),
+ ),
+ (
+ "dst",
+ "dst",
+ lambda x: str(ipaddress.IPv6Address(x)),
+ lambda x: int.from_bytes(x, "big"),
+ ),
+ ("tp_src", "tp_src", "%d", int),
+ ("tp_dst", "tp_dst", "%d", int),
+ ("proto", "proto", "%d", int),
+ )
+
+ def __init__(
+ self,
+ data=None,
+ offset=None,
+ parent=None,
+ length=None,
+ init=None,
+ ):
+ ovskey.ovs_key_proto.__init__(
+ self,
+ "ct_tuple6",
+ data=data,
+ offset=offset,
+ parent=parent,
+ length=length,
+ init=init,
+ )
+
+ class ovs_key_mpls(nla):
+ fields = (("lse", ">I"),)
+
+ def dpstr(self, mask=None, more=False):
+ print_str = ""
+
+ for field in (
+ (
+ "OVS_KEY_ATTR_PRIORITY",
+ "skb_priority",
+ "%d",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_SKB_MARK",
+ "skb_mark",
+ "%d",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_RECIRC_ID",
+ "recirc_id",
+ "0x%08X",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_DP_HASH",
+ "dp_hash",
+ "0x%08X",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_CT_STATE",
+ "ct_state",
+ "0x%04x",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_CT_ZONE",
+ "ct_zone",
+ "0x%04x",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_CT_MARK",
+ "ct_mark",
+ "0x%08x",
+ lambda x: False,
+ True,
+ ),
+ (
+ "OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV4",
+ None,
+ None,
+ False,
+ False,
+ ),
+ (
+ "OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV6",
+ None,
+ None,
+ False,
+ False,
+ ),
+ (
+ "OVS_KEY_ATTR_IN_PORT",
+ "in_port",
+ "%d",
+ lambda x: True,
+ True,
+ ),
+ ("OVS_KEY_ATTR_ETHERNET", None, None, False, False),
+ (
+ "OVS_KEY_ATTR_ETHERTYPE",
+ "eth_type",
+ "0x%04x",
+ lambda x: int(x) == 0xFFFF,
+ True,
+ ),
+ ("OVS_KEY_ATTR_IPV4", None, None, False, False),
+ ("OVS_KEY_ATTR_IPV6", None, None, False, False),
+ ("OVS_KEY_ATTR_ARP", None, None, False, False),
+ ("OVS_KEY_ATTR_TCP", None, None, False, False),
+ (
+ "OVS_KEY_ATTR_TCP_FLAGS",
+ "tcp_flags",
+ "0x%04x",
+ lambda x: False,
+ True,
+ ),
+ ("OVS_KEY_ATTR_UDP", None, None, False, False),
+ ("OVS_KEY_ATTR_SCTP", None, None, False, False),
+ ("OVS_KEY_ATTR_ICMP", None, None, False, False),
+ ("OVS_KEY_ATTR_ICMPV6", None, None, False, False),
+ ("OVS_KEY_ATTR_ND", None, None, False, False),
+ ):
+ v = self.get_attr(field[0])
+ if v is not None:
+ m = None if mask is None else mask.get_attr(field[0])
+ if field[4] is False:
+ print_str += v.dpstr(m, more)
+ print_str += ","
+ else:
+ if m is None or field[3](m):
+ print_str += field[1] + "("
+ print_str += field[2] % v
+ print_str += "),"
+ elif more or m != 0:
+ print_str += field[1] + "("
+ print_str += (field[2] % v) + "/" + (field[2] % m)
+ print_str += "),"
+
+ return print_str
+
+
+class OvsPacket(GenericNetlinkSocket):
+ OVS_PACKET_CMD_MISS = 1 # Flow table miss
+ OVS_PACKET_CMD_ACTION = 2 # USERSPACE action
+ OVS_PACKET_CMD_EXECUTE = 3 # Apply actions to packet
+
+ class ovs_packet_msg(ovs_dp_msg):
+ nla_map = (
+ ("OVS_PACKET_ATTR_UNSPEC", "none"),
+ ("OVS_PACKET_ATTR_PACKET", "array(uint8)"),
+ ("OVS_PACKET_ATTR_KEY", "ovskey"),
+ ("OVS_PACKET_ATTR_ACTIONS", "ovsactions"),
+ ("OVS_PACKET_ATTR_USERDATA", "none"),
+ ("OVS_PACKET_ATTR_EGRESS_TUN_KEY", "none"),
+ ("OVS_PACKET_ATTR_UNUSED1", "none"),
+ ("OVS_PACKET_ATTR_UNUSED2", "none"),
+ ("OVS_PACKET_ATTR_PROBE", "none"),
+ ("OVS_PACKET_ATTR_MRU", "uint16"),
+ ("OVS_PACKET_ATTR_LEN", "uint32"),
+ ("OVS_PACKET_ATTR_HASH", "uint64"),
+ )
+
+ def __init__(self):
+ GenericNetlinkSocket.__init__(self)
+ self.bind(OVS_PACKET_FAMILY, OvsPacket.ovs_packet_msg)
+
+ def upcall_handler(self, up=None):
+ print("listening on upcall packet handler:", self.epid)
+ while True:
+ try:
+ msgs = self.get()
+ for msg in msgs:
+ if not up:
+ continue
+ if msg["cmd"] == OvsPacket.OVS_PACKET_CMD_MISS:
+ up.miss(msg)
+ elif msg["cmd"] == OvsPacket.OVS_PACKET_CMD_ACTION:
+ up.action(msg)
+ elif msg["cmd"] == OvsPacket.OVS_PACKET_CMD_EXECUTE:
+ up.execute(msg)
+ else:
+ print("Unkonwn cmd: %d" % msg["cmd"])
+ except NetlinkError as ne:
+ raise ne
+
+
+class OvsDatapath(GenericNetlinkSocket):
OVS_DP_F_VPORT_PIDS = 1 << 1
OVS_DP_F_DISPATCH_UPCALL_PER_CPU = 1 << 3
@@ -113,7 +1036,9 @@
return reply
- def create(self, dpname, shouldUpcall=False, versionStr=None):
+ def create(
+ self, dpname, shouldUpcall=False, versionStr=None, p=OvsPacket()
+ ):
msg = OvsDatapath.dp_cmd_msg()
msg["cmd"] = OVS_DP_CMD_NEW
if versionStr is None:
@@ -128,11 +1053,18 @@
if versionStr is not None and versionStr.find(":") != -1:
dpfeatures = int(versionStr.split(":")[1], 0)
else:
- dpfeatures = OvsDatapath.OVS_DP_F_VPORT_PIDS
+ if versionStr is None or versionStr.find(":") == -1:
+ dpfeatures |= OvsDatapath.OVS_DP_F_DISPATCH_UPCALL_PER_CPU
+ dpfeatures &= ~OvsDatapath.OVS_DP_F_VPORT_PIDS
+ nproc = multiprocessing.cpu_count()
+ procarray = []
+ for i in range(1, nproc):
+ procarray += [int(p.epid)]
+ msg["attrs"].append(["OVS_DP_ATTR_UPCALL_PID", procarray])
msg["attrs"].append(["OVS_DP_ATTR_USER_FEATURES", dpfeatures])
if not shouldUpcall:
- msg["attrs"].append(["OVS_DP_ATTR_UPCALL_PID", 0])
+ msg["attrs"].append(["OVS_DP_ATTR_UPCALL_PID", [0]])
try:
reply = self.nlm_request(
@@ -170,6 +1102,12 @@
class OvsVport(GenericNetlinkSocket):
+ OVS_VPORT_TYPE_NETDEV = 1
+ OVS_VPORT_TYPE_INTERNAL = 2
+ OVS_VPORT_TYPE_GRE = 3
+ OVS_VPORT_TYPE_VXLAN = 4
+ OVS_VPORT_TYPE_GENEVE = 5
+
class ovs_vport_msg(ovs_dp_msg):
nla_map = (
("OVS_VPORT_ATTR_UNSPEC", "none"),
@@ -197,21 +1135,35 @@
)
def type_to_str(vport_type):
- if vport_type == 1:
+ if vport_type == OvsVport.OVS_VPORT_TYPE_NETDEV:
return "netdev"
- elif vport_type == 2:
+ elif vport_type == OvsVport.OVS_VPORT_TYPE_INTERNAL:
return "internal"
- elif vport_type == 3:
+ elif vport_type == OvsVport.OVS_VPORT_TYPE_GRE:
return "gre"
- elif vport_type == 4:
+ elif vport_type == OvsVport.OVS_VPORT_TYPE_VXLAN:
return "vxlan"
- elif vport_type == 5:
+ elif vport_type == OvsVport.OVS_VPORT_TYPE_GENEVE:
return "geneve"
- return "unknown:%d" % vport_type
+ raise ValueError("Unknown vport type:%d" % vport_type)
- def __init__(self):
+ def str_to_type(vport_type):
+ if vport_type == "netdev":
+ return OvsVport.OVS_VPORT_TYPE_NETDEV
+ elif vport_type == "internal":
+ return OvsVport.OVS_VPORT_TYPE_INTERNAL
+ elif vport_type == "gre":
+ return OvsVport.OVS_VPORT_TYPE_INTERNAL
+ elif vport_type == "vxlan":
+ return OvsVport.OVS_VPORT_TYPE_VXLAN
+ elif vport_type == "geneve":
+ return OvsVport.OVS_VPORT_TYPE_GENEVE
+ raise ValueError("Unknown vport type: '%s'" % vport_type)
+
+ def __init__(self, packet=OvsPacket()):
GenericNetlinkSocket.__init__(self)
self.bind(OVS_VPORT_FAMILY, OvsVport.ovs_vport_msg)
+ self.upcall_packet = packet
def info(self, vport_name, dpifindex=0, portno=None):
msg = OvsVport.ovs_vport_msg()
@@ -238,8 +1190,231 @@
raise ne
return reply
+ def attach(self, dpindex, vport_ifname, ptype):
+ msg = OvsVport.ovs_vport_msg()
-def print_ovsdp_full(dp_lookup_rep, ifindex, ndb=NDB()):
+ msg["cmd"] = OVS_VPORT_CMD_NEW
+ msg["version"] = OVS_DATAPATH_VERSION
+ msg["reserved"] = 0
+ msg["dpifindex"] = dpindex
+ port_type = OvsVport.str_to_type(ptype)
+
+ msg["attrs"].append(["OVS_VPORT_ATTR_TYPE", port_type])
+ msg["attrs"].append(["OVS_VPORT_ATTR_NAME", vport_ifname])
+ msg["attrs"].append(
+ ["OVS_VPORT_ATTR_UPCALL_PID", [self.upcall_packet.epid]]
+ )
+
+ try:
+ reply = self.nlm_request(
+ msg, msg_type=self.prid, msg_flags=NLM_F_REQUEST | NLM_F_ACK
+ )
+ reply = reply[0]
+ except NetlinkError as ne:
+ if ne.code == errno.EEXIST:
+ reply = None
+ else:
+ raise ne
+ return reply
+
+ def reset_upcall(self, dpindex, vport_ifname, p=None):
+ msg = OvsVport.ovs_vport_msg()
+
+ msg["cmd"] = OVS_VPORT_CMD_SET
+ msg["version"] = OVS_DATAPATH_VERSION
+ msg["reserved"] = 0
+ msg["dpifindex"] = dpindex
+ msg["attrs"].append(["OVS_VPORT_ATTR_NAME", vport_ifname])
+
+ if p == None:
+ p = self.upcall_packet
+ else:
+ self.upcall_packet = p
+
+ msg["attrs"].append(["OVS_VPORT_ATTR_UPCALL_PID", [p.epid]])
+
+ try:
+ reply = self.nlm_request(
+ msg, msg_type=self.prid, msg_flags=NLM_F_REQUEST | NLM_F_ACK
+ )
+ reply = reply[0]
+ except NetlinkError as ne:
+ raise ne
+ return reply
+
+ def detach(self, dpindex, vport_ifname):
+ msg = OvsVport.ovs_vport_msg()
+
+ msg["cmd"] = OVS_VPORT_CMD_DEL
+ msg["version"] = OVS_DATAPATH_VERSION
+ msg["reserved"] = 0
+ msg["dpifindex"] = dpindex
+ msg["attrs"].append(["OVS_VPORT_ATTR_NAME", vport_ifname])
+
+ try:
+ reply = self.nlm_request(
+ msg, msg_type=self.prid, msg_flags=NLM_F_REQUEST | NLM_F_ACK
+ )
+ reply = reply[0]
+ except NetlinkError as ne:
+ if ne.code == errno.ENODEV:
+ reply = None
+ else:
+ raise ne
+ return reply
+
+ def upcall_handler(self, handler=None):
+ self.upcall_packet.upcall_handler(handler)
+
+
+class OvsFlow(GenericNetlinkSocket):
+ class ovs_flow_msg(ovs_dp_msg):
+ nla_map = (
+ ("OVS_FLOW_ATTR_UNSPEC", "none"),
+ ("OVS_FLOW_ATTR_KEY", "ovskey"),
+ ("OVS_FLOW_ATTR_ACTIONS", "ovsactions"),
+ ("OVS_FLOW_ATTR_STATS", "flowstats"),
+ ("OVS_FLOW_ATTR_TCP_FLAGS", "uint8"),
+ ("OVS_FLOW_ATTR_USED", "uint64"),
+ ("OVS_FLOW_ATTR_CLEAR", "none"),
+ ("OVS_FLOW_ATTR_MASK", "ovskey"),
+ ("OVS_FLOW_ATTR_PROBE", "none"),
+ ("OVS_FLOW_ATTR_UFID", "array(uint32)"),
+ ("OVS_FLOW_ATTR_UFID_FLAGS", "uint32"),
+ )
+
+ class flowstats(nla):
+ fields = (
+ ("packets", "=Q"),
+ ("bytes", "=Q"),
+ )
+
+ def dpstr(self, more=False):
+ ufid = self.get_attr("OVS_FLOW_ATTR_UFID")
+ ufid_str = ""
+ if ufid is not None:
+ ufid_str = (
+ "ufid:{:08x}-{:04x}-{:04x}-{:04x}-{:04x}{:08x}".format(
+ ufid[0],
+ ufid[1] >> 16,
+ ufid[1] & 0xFFFF,
+ ufid[2] >> 16,
+ ufid[2] & 0,
+ ufid[3],
+ )
+ )
+
+ key_field = self.get_attr("OVS_FLOW_ATTR_KEY")
+ keymsg = None
+ if key_field is not None:
+ keymsg = key_field
+
+ mask_field = self.get_attr("OVS_FLOW_ATTR_MASK")
+ maskmsg = None
+ if mask_field is not None:
+ maskmsg = mask_field
+
+ acts_field = self.get_attr("OVS_FLOW_ATTR_ACTIONS")
+ actsmsg = None
+ if acts_field is not None:
+ actsmsg = acts_field
+
+ print_str = ""
+
+ if more:
+ print_str += ufid_str + ","
+
+ if keymsg is not None:
+ print_str += keymsg.dpstr(maskmsg, more)
+
+ stats = self.get_attr("OVS_FLOW_ATTR_STATS")
+ if stats is None:
+ print_str += " packets:0, bytes:0,"
+ else:
+ print_str += " packets:%d, bytes:%d," % (
+ stats["packets"],
+ stats["bytes"],
+ )
+
+ used = self.get_attr("OVS_FLOW_ATTR_USED")
+ print_str += " used:"
+ if used is None:
+ print_str += "never,"
+ else:
+ used_time = int(used)
+ cur_time_sec = time.clock_gettime(time.CLOCK_MONOTONIC)
+ used_time = (cur_time_sec * 1000) - used_time
+ print_str += "{}s,".format(used_time / 1000)
+
+ print_str += " actions:"
+ if (
+ actsmsg is None
+ or "attrs" not in actsmsg
+ or len(actsmsg["attrs"]) == 0
+ ):
+ print_str += "drop"
+ else:
+ print_str += actsmsg.dpstr(more)
+
+ return print_str
+
+ def __init__(self):
+ GenericNetlinkSocket.__init__(self)
+
+ self.bind(OVS_FLOW_FAMILY, OvsFlow.ovs_flow_msg)
+
+ def dump(self, dpifindex, flowspec=None):
+ """
+ Returns a list of messages containing flows.
+
+ dpifindex should be a valid datapath obtained by calling
+ into the OvsDatapath lookup
+
+ flowpsec is a string which represents a flow in the dpctl
+ format.
+ """
+ msg = OvsFlow.ovs_flow_msg()
+
+ msg["cmd"] = OVS_FLOW_CMD_GET
+ msg["version"] = OVS_DATAPATH_VERSION
+ msg["reserved"] = 0
+ msg["dpifindex"] = dpifindex
+
+ msg_flags = NLM_F_REQUEST | NLM_F_ACK
+ if flowspec is None:
+ msg_flags |= NLM_F_DUMP
+ rep = None
+
+ try:
+ rep = self.nlm_request(
+ msg,
+ msg_type=self.prid,
+ msg_flags=msg_flags,
+ )
+ except NetlinkError as ne:
+ raise ne
+ return rep
+
+ def miss(self, packetmsg):
+ seq = packetmsg["header"]["sequence_number"]
+ keystr = "(none)"
+ key_field = packetmsg.get_attr("OVS_PACKET_ATTR_KEY")
+ if key_field is not None:
+ keystr = key_field.dpstr(None, True)
+
+ pktdata = packetmsg.get_attr("OVS_PACKET_ATTR_PACKET")
+ pktpres = "yes" if pktdata is not None else "no"
+
+ print("MISS upcall[%d/%s]: %s" % (seq, pktpres, keystr), flush=True)
+
+ def execute(self, packetmsg):
+ print("userspace execute command")
+
+ def action(self, packetmsg):
+ print("userspace action command")
+
+
+def print_ovsdp_full(dp_lookup_rep, ifindex, ndb=NDB(), vpl=OvsVport()):
dp_name = dp_lookup_rep.get_attr("OVS_DP_ATTR_NAME")
base_stats = dp_lookup_rep.get_attr("OVS_DP_ATTR_STATS")
megaflow_stats = dp_lookup_rep.get_attr("OVS_DP_ATTR_MEGAFLOW_STATS")
@@ -265,7 +1440,6 @@
print(" features: 0x%X" % user_features)
# port print out
- vpl = OvsVport()
for iface in ndb.interfaces:
rep = vpl.info(iface.ifname, ifindex)
if rep is not None:
@@ -280,12 +1454,16 @@
def main(argv):
+ nlmsg_atoms.ovskey = ovskey
+ nlmsg_atoms.ovsactions = ovsactions
+
parser = argparse.ArgumentParser()
parser.add_argument(
"-v",
"--verbose",
action="count",
help="Increment 'verbose' output counter.",
+ default=0,
)
subparsers = parser.add_subparsers()
@@ -312,9 +1490,40 @@
deldpcmd = subparsers.add_parser("del-dp")
deldpcmd.add_argument("deldp", help="Datapath Name")
+ addifcmd = subparsers.add_parser("add-if")
+ addifcmd.add_argument("dpname", help="Datapath Name")
+ addifcmd.add_argument("addif", help="Interface name for adding")
+ addifcmd.add_argument(
+ "-u",
+ "--upcall",
+ action="store_true",
+ help="Leave open a reader for upcalls",
+ )
+ addifcmd.add_argument(
+ "-t",
+ "--ptype",
+ type=str,
+ default="netdev",
+ choices=["netdev", "internal"],
+ help="Interface type (default netdev)",
+ )
+ delifcmd = subparsers.add_parser("del-if")
+ delifcmd.add_argument("dpname", help="Datapath Name")
+ delifcmd.add_argument("delif", help="Interface name for adding")
+
+ dumpflcmd = subparsers.add_parser("dump-flows")
+ dumpflcmd.add_argument("dumpdp", help="Datapath Name")
+
args = parser.parse_args()
+ if args.verbose > 0:
+ if args.verbose > 1:
+ logging.basicConfig(level=logging.DEBUG)
+
+ ovspk = OvsPacket()
ovsdp = OvsDatapath()
+ ovsvp = OvsVport(ovspk)
+ ovsflow = OvsFlow()
ndb = NDB()
if hasattr(args, "showdp"):
@@ -328,7 +1537,7 @@
if rep is not None:
found = True
- print_ovsdp_full(rep, iface.index, ndb)
+ print_ovsdp_full(rep, iface.index, ndb, ovsvp)
if not found:
msg = "No DP found"
@@ -336,13 +1545,50 @@
msg += ":'%s'" % args.showdp
print(msg)
elif hasattr(args, "adddp"):
- rep = ovsdp.create(args.adddp, args.upcall, args.versioning)
+ rep = ovsdp.create(args.adddp, args.upcall, args.versioning, ovspk)
if rep is None:
print("DP '%s' already exists" % args.adddp)
else:
print("DP '%s' added" % args.adddp)
+ if args.upcall:
+ ovspk.upcall_handler(ovsflow)
elif hasattr(args, "deldp"):
ovsdp.destroy(args.deldp)
+ elif hasattr(args, "addif"):
+ rep = ovsdp.info(args.dpname, 0)
+ if rep is None:
+ print("DP '%s' not found." % args.dpname)
+ return 1
+ dpindex = rep["dpifindex"]
+ rep = ovsvp.attach(rep["dpifindex"], args.addif, args.ptype)
+ msg = "vport '%s'" % args.addif
+ if rep and rep["header"]["error"] is None:
+ msg += " added."
+ else:
+ msg += " failed to add."
+ if args.upcall:
+ if rep is None:
+ rep = ovsvp.reset_upcall(dpindex, args.addif, ovspk)
+ ovsvp.upcall_handler(ovsflow)
+ elif hasattr(args, "delif"):
+ rep = ovsdp.info(args.dpname, 0)
+ if rep is None:
+ print("DP '%s' not found." % args.dpname)
+ return 1
+ rep = ovsvp.detach(rep["dpifindex"], args.delif)
+ msg = "vport '%s'" % args.delif
+ if rep and rep["header"]["error"] is None:
+ msg += " removed."
+ else:
+ msg += " failed to remove."
+ elif hasattr(args, "dumpdp"):
+ rep = ovsdp.info(args.dumpdp, 0)
+ if rep is None:
+ print("DP '%s' not found." % args.dumpdp)
+ return 1
+ rep = ovsflow.dump(rep["dpifindex"])
+ for flow in rep:
+ print(flow.dpstr(True if args.verbose > 0 else False))
return 0
diff --git a/tools/testing/selftests/net/test_bridge_neigh_suppress.sh b/tools/testing/selftests/net/test_bridge_neigh_suppress.sh
new file mode 100755
index 0000000..d80f2cd
--- /dev/null
+++ b/tools/testing/selftests/net/test_bridge_neigh_suppress.sh
@@ -0,0 +1,862 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# This test is for checking bridge neighbor suppression functionality. The
+# topology consists of two bridges (VTEPs) connected using VXLAN. A single
+# host is connected to each bridge over multiple VLANs. The test checks that
+# ARP/NS messages from the first host are suppressed on the VXLAN port when
+# should.
+#
+# +-----------------------+ +------------------------+
+# | h1 | | h2 |
+# | | | |
+# | + eth0.10 | | + eth0.10 |
+# | | 192.0.2.1/28 | | | 192.0.2.2/28 |
+# | | 2001:db8:1::1/64 | | | 2001:db8:1::2/64 |
+# | | | | | |
+# | | + eth0.20 | | | + eth0.20 |
+# | \ | 192.0.2.17/28 | | \ | 192.0.2.18/28 |
+# | \ | 2001:db8:2::1/64 | | \ | 2001:db8:2::2/64 |
+# | \| | | \| |
+# | + eth0 | | + eth0 |
+# +----|------------------+ +----|-------------------+
+# | |
+# | |
+# +----|-------------------------------+ +----|-------------------------------+
+# | + swp1 + vx0 | | + swp1 + vx0 |
+# | | | | | | | |
+# | | br0 | | | | | |
+# | +------------+-----------+ | | +------------+-----------+ |
+# | | | | | |
+# | | | | | |
+# | +---+---+ | | +---+---+ |
+# | | | | | | | |
+# | | | | | | | |
+# | + + | | + + |
+# | br0.10 br0.20 | | br0.10 br0.20 |
+# | | | |
+# | 192.0.2.33 | | 192.0.2.34 |
+# | + lo | | + lo |
+# | | | |
+# | | | |
+# | 192.0.2.49/28 | | 192.0.2.50/28 |
+# | veth0 +-------+ veth0 |
+# | | | |
+# | sw1 | | sw2 |
+# +------------------------------------+ +------------------------------------+
+
+ret=0
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
+# All tests in this script. Can be overridden with -t option.
+TESTS="
+ neigh_suppress_arp
+ neigh_suppress_ns
+ neigh_vlan_suppress_arp
+ neigh_vlan_suppress_ns
+"
+VERBOSE=0
+PAUSE_ON_FAIL=no
+PAUSE=no
+
+################################################################################
+# Utilities
+
+log_test()
+{
+ local rc=$1
+ local expected=$2
+ local msg="$3"
+
+ if [ ${rc} -eq ${expected} ]; then
+ printf "TEST: %-60s [ OK ]\n" "${msg}"
+ nsuccess=$((nsuccess+1))
+ else
+ ret=1
+ nfail=$((nfail+1))
+ printf "TEST: %-60s [FAIL]\n" "${msg}"
+ if [ "$VERBOSE" = "1" ]; then
+ echo " rc=$rc, expected $expected"
+ fi
+
+ if [ "${PAUSE_ON_FAIL}" = "yes" ]; then
+ echo
+ echo "hit enter to continue, 'q' to quit"
+ read a
+ [ "$a" = "q" ] && exit 1
+ fi
+ fi
+
+ if [ "${PAUSE}" = "yes" ]; then
+ echo
+ echo "hit enter to continue, 'q' to quit"
+ read a
+ [ "$a" = "q" ] && exit 1
+ fi
+
+ [ "$VERBOSE" = "1" ] && echo
+}
+
+run_cmd()
+{
+ local cmd="$1"
+ local out
+ local stderr="2>/dev/null"
+
+ if [ "$VERBOSE" = "1" ]; then
+ printf "COMMAND: $cmd\n"
+ stderr=
+ fi
+
+ out=$(eval $cmd $stderr)
+ rc=$?
+ if [ "$VERBOSE" = "1" -a -n "$out" ]; then
+ echo " $out"
+ fi
+
+ return $rc
+}
+
+tc_check_packets()
+{
+ local ns=$1; shift
+ local id=$1; shift
+ local handle=$1; shift
+ local count=$1; shift
+ local pkts
+
+ sleep 0.1
+ pkts=$(tc -n $ns -j -s filter show $id \
+ | jq ".[] | select(.options.handle == $handle) | \
+ .options.actions[0].stats.packets")
+ [[ $pkts == $count ]]
+}
+
+################################################################################
+# Setup
+
+setup_topo_ns()
+{
+ local ns=$1; shift
+
+ ip netns add $ns
+ ip -n $ns link set dev lo up
+
+ ip netns exec $ns sysctl -qw net.ipv6.conf.all.keep_addr_on_down=1
+ ip netns exec $ns sysctl -qw net.ipv6.conf.default.ignore_routes_with_linkdown=1
+ ip netns exec $ns sysctl -qw net.ipv6.conf.all.accept_dad=0
+ ip netns exec $ns sysctl -qw net.ipv6.conf.default.accept_dad=0
+}
+
+setup_topo()
+{
+ local ns
+
+ for ns in h1 h2 sw1 sw2; do
+ setup_topo_ns $ns
+ done
+
+ ip link add name veth0 type veth peer name veth1
+ ip link set dev veth0 netns h1 name eth0
+ ip link set dev veth1 netns sw1 name swp1
+
+ ip link add name veth0 type veth peer name veth1
+ ip link set dev veth0 netns sw1 name veth0
+ ip link set dev veth1 netns sw2 name veth0
+
+ ip link add name veth0 type veth peer name veth1
+ ip link set dev veth0 netns h2 name eth0
+ ip link set dev veth1 netns sw2 name swp1
+}
+
+setup_host_common()
+{
+ local ns=$1; shift
+ local v4addr1=$1; shift
+ local v4addr2=$1; shift
+ local v6addr1=$1; shift
+ local v6addr2=$1; shift
+
+ ip -n $ns link set dev eth0 up
+ ip -n $ns link add link eth0 name eth0.10 up type vlan id 10
+ ip -n $ns link add link eth0 name eth0.20 up type vlan id 20
+
+ ip -n $ns address add $v4addr1 dev eth0.10
+ ip -n $ns address add $v4addr2 dev eth0.20
+ ip -n $ns address add $v6addr1 dev eth0.10
+ ip -n $ns address add $v6addr2 dev eth0.20
+}
+
+setup_h1()
+{
+ local ns=h1
+ local v4addr1=192.0.2.1/28
+ local v4addr2=192.0.2.17/28
+ local v6addr1=2001:db8:1::1/64
+ local v6addr2=2001:db8:2::1/64
+
+ setup_host_common $ns $v4addr1 $v4addr2 $v6addr1 $v6addr2
+}
+
+setup_h2()
+{
+ local ns=h2
+ local v4addr1=192.0.2.2/28
+ local v4addr2=192.0.2.18/28
+ local v6addr1=2001:db8:1::2/64
+ local v6addr2=2001:db8:2::2/64
+
+ setup_host_common $ns $v4addr1 $v4addr2 $v6addr1 $v6addr2
+}
+
+setup_sw_common()
+{
+ local ns=$1; shift
+ local local_addr=$1; shift
+ local remote_addr=$1; shift
+ local veth_addr=$1; shift
+ local gw_addr=$1; shift
+
+ ip -n $ns address add $local_addr/32 dev lo
+
+ ip -n $ns link set dev veth0 up
+ ip -n $ns address add $veth_addr/28 dev veth0
+ ip -n $ns route add default via $gw_addr
+
+ ip -n $ns link add name br0 up type bridge vlan_filtering 1 \
+ vlan_default_pvid 0 mcast_snooping 0
+
+ ip -n $ns link add link br0 name br0.10 up type vlan id 10
+ bridge -n $ns vlan add vid 10 dev br0 self
+
+ ip -n $ns link add link br0 name br0.20 up type vlan id 20
+ bridge -n $ns vlan add vid 20 dev br0 self
+
+ ip -n $ns link set dev swp1 up master br0
+ bridge -n $ns vlan add vid 10 dev swp1
+ bridge -n $ns vlan add vid 20 dev swp1
+
+ ip -n $ns link add name vx0 up master br0 type vxlan \
+ local $local_addr dstport 4789 nolearning external
+ bridge -n $ns fdb add 00:00:00:00:00:00 dev vx0 self static \
+ dst $remote_addr src_vni 10010
+ bridge -n $ns fdb add 00:00:00:00:00:00 dev vx0 self static \
+ dst $remote_addr src_vni 10020
+ bridge -n $ns link set dev vx0 vlan_tunnel on learning off
+
+ bridge -n $ns vlan add vid 10 dev vx0
+ bridge -n $ns vlan add vid 10 dev vx0 tunnel_info id 10010
+
+ bridge -n $ns vlan add vid 20 dev vx0
+ bridge -n $ns vlan add vid 20 dev vx0 tunnel_info id 10020
+}
+
+setup_sw1()
+{
+ local ns=sw1
+ local local_addr=192.0.2.33
+ local remote_addr=192.0.2.34
+ local veth_addr=192.0.2.49
+ local gw_addr=192.0.2.50
+
+ setup_sw_common $ns $local_addr $remote_addr $veth_addr $gw_addr
+}
+
+setup_sw2()
+{
+ local ns=sw2
+ local local_addr=192.0.2.34
+ local remote_addr=192.0.2.33
+ local veth_addr=192.0.2.50
+ local gw_addr=192.0.2.49
+
+ setup_sw_common $ns $local_addr $remote_addr $veth_addr $gw_addr
+}
+
+setup()
+{
+ set -e
+
+ setup_topo
+ setup_h1
+ setup_h2
+ setup_sw1
+ setup_sw2
+
+ sleep 5
+
+ set +e
+}
+
+cleanup()
+{
+ local ns
+
+ for ns in h1 h2 sw1 sw2; do
+ ip netns del $ns &> /dev/null
+ done
+}
+
+################################################################################
+# Tests
+
+neigh_suppress_arp_common()
+{
+ local vid=$1; shift
+ local sip=$1; shift
+ local tip=$1; shift
+ local h2_mac
+
+ echo
+ echo "Per-port ARP suppression - VLAN $vid"
+ echo "----------------------------------"
+
+ run_cmd "tc -n sw1 qdisc replace dev vx0 clsact"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 101 proto 0x0806 flower indev swp1 arp_tip $tip arp_sip $sip arp_op request action pass"
+
+ # Initial state - check that ARP requests are not suppressed and that
+ # ARP replies are received.
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "ARP suppression"
+
+ # Enable neighbor suppression and check that nothing changes compared
+ # to the initial state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "ARP suppression"
+
+ # Install an FDB entry for the remote host and check that nothing
+ # changes compared to the initial state.
+ h2_mac=$(ip -n h2 -j -p link show eth0.$vid | jq -r '.[]["address"]')
+ run_cmd "bridge -n sw1 fdb replace $h2_mac dev vx0 master static vlan $vid"
+ log_test $? 0 "FDB entry installation"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "ARP suppression"
+
+ # Install a neighbor on the matching SVI interface and check that ARP
+ # requests are suppressed.
+ run_cmd "ip -n sw1 neigh replace $tip lladdr $h2_mac nud permanent dev br0.$vid"
+ log_test $? 0 "Neighbor entry installation"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "ARP suppression"
+
+ # Take the second host down and check that ARP requests are suppressed
+ # and that ARP replies are received.
+ run_cmd "ip -n h2 link set dev eth0.$vid down"
+ log_test $? 0 "H2 down"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "ARP suppression"
+
+ run_cmd "ip -n h2 link set dev eth0.$vid up"
+ log_test $? 0 "H2 up"
+
+ # Disable neighbor suppression and check that ARP requests are no
+ # longer suppressed.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 0 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 4
+ log_test $? 0 "ARP suppression"
+
+ # Take the second host down and check that ARP requests are not
+ # suppressed and that ARP replies are not received.
+ run_cmd "ip -n h2 link set dev eth0.$vid down"
+ log_test $? 0 "H2 down"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip -I eth0.$vid $tip"
+ log_test $? 1 "arping"
+ tc_check_packets sw1 "dev vx0 egress" 101 5
+ log_test $? 0 "ARP suppression"
+}
+
+neigh_suppress_arp()
+{
+ local vid=10
+ local sip=192.0.2.1
+ local tip=192.0.2.2
+
+ neigh_suppress_arp_common $vid $sip $tip
+
+ vid=20
+ sip=192.0.2.17
+ tip=192.0.2.18
+ neigh_suppress_arp_common $vid $sip $tip
+}
+
+neigh_suppress_ns_common()
+{
+ local vid=$1; shift
+ local saddr=$1; shift
+ local daddr=$1; shift
+ local maddr=$1; shift
+ local h2_mac
+
+ echo
+ echo "Per-port NS suppression - VLAN $vid"
+ echo "---------------------------------"
+
+ run_cmd "tc -n sw1 qdisc replace dev vx0 clsact"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 101 proto ipv6 flower indev swp1 ip_proto icmpv6 dst_ip $maddr src_ip $saddr type 135 code 0 action pass"
+
+ # Initial state - check that NS messages are not suppressed and that ND
+ # messages are received.
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "NS suppression"
+
+ # Enable neighbor suppression and check that nothing changes compared
+ # to the initial state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "NS suppression"
+
+ # Install an FDB entry for the remote host and check that nothing
+ # changes compared to the initial state.
+ h2_mac=$(ip -n h2 -j -p link show eth0.$vid | jq -r '.[]["address"]')
+ run_cmd "bridge -n sw1 fdb replace $h2_mac dev vx0 master static vlan $vid"
+ log_test $? 0 "FDB entry installation"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "NS suppression"
+
+ # Install a neighbor on the matching SVI interface and check that NS
+ # messages are suppressed.
+ run_cmd "ip -n sw1 neigh replace $daddr lladdr $h2_mac nud permanent dev br0.$vid"
+ log_test $? 0 "Neighbor entry installation"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "NS suppression"
+
+ # Take the second host down and check that NS messages are suppressed
+ # and that ND messages are received.
+ run_cmd "ip -n h2 link set dev eth0.$vid down"
+ log_test $? 0 "H2 down"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 3
+ log_test $? 0 "NS suppression"
+
+ run_cmd "ip -n h2 link set dev eth0.$vid up"
+ log_test $? 0 "H2 up"
+
+ # Disable neighbor suppression and check that NS messages are no longer
+ # suppressed.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 0 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 4
+ log_test $? 0 "NS suppression"
+
+ # Take the second host down and check that NS messages are not
+ # suppressed and that ND messages are not received.
+ run_cmd "ip -n h2 link set dev eth0.$vid down"
+ log_test $? 0 "H2 down"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr -w 5000 $daddr eth0.$vid"
+ log_test $? 2 "ndisc6"
+ tc_check_packets sw1 "dev vx0 egress" 101 5
+ log_test $? 0 "NS suppression"
+}
+
+neigh_suppress_ns()
+{
+ local vid=10
+ local saddr=2001:db8:1::1
+ local daddr=2001:db8:1::2
+ local maddr=ff02::1:ff00:2
+
+ neigh_suppress_ns_common $vid $saddr $daddr $maddr
+
+ vid=20
+ saddr=2001:db8:2::1
+ daddr=2001:db8:2::2
+ maddr=ff02::1:ff00:2
+
+ neigh_suppress_ns_common $vid $saddr $daddr $maddr
+}
+
+neigh_vlan_suppress_arp()
+{
+ local vid1=10
+ local vid2=20
+ local sip1=192.0.2.1
+ local sip2=192.0.2.17
+ local tip1=192.0.2.2
+ local tip2=192.0.2.18
+ local h2_mac1
+ local h2_mac2
+
+ echo
+ echo "Per-{Port, VLAN} ARP suppression"
+ echo "--------------------------------"
+
+ run_cmd "tc -n sw1 qdisc replace dev vx0 clsact"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 101 proto 0x0806 flower indev swp1 arp_tip $tip1 arp_sip $sip1 arp_op request action pass"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 102 proto 0x0806 flower indev swp1 arp_tip $tip2 arp_sip $sip2 arp_op request action pass"
+
+ h2_mac1=$(ip -n h2 -j -p link show eth0.$vid1 | jq -r '.[]["address"]')
+ h2_mac2=$(ip -n h2 -j -p link show eth0.$vid2 | jq -r '.[]["address"]')
+ run_cmd "bridge -n sw1 fdb replace $h2_mac1 dev vx0 master static vlan $vid1"
+ run_cmd "bridge -n sw1 fdb replace $h2_mac2 dev vx0 master static vlan $vid2"
+ run_cmd "ip -n sw1 neigh replace $tip1 lladdr $h2_mac1 nud permanent dev br0.$vid1"
+ run_cmd "ip -n sw1 neigh replace $tip2 lladdr $h2_mac2 nud permanent dev br0.$vid2"
+
+ # Enable per-{Port, VLAN} neighbor suppression and check that ARP
+ # requests are not suppressed and that ARP replies are received.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_vlan_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_vlan_suppress on\""
+ log_test $? 0 "\"neigh_vlan_suppress\" is on"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 1
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+
+ # Enable neighbor suppression on VLAN 10 and check that only on this
+ # VLAN ARP requests are suppressed.
+ run_cmd "bridge -n sw1 vlan set vid $vid1 dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid1 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on (VLAN $vid1)"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid2 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off (VLAN $vid2)"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 2
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+
+ # Enable neighbor suppression on the port and check that it has no
+ # effect compared to previous state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 3
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+
+ # Disable neighbor suppression on the port and check that it has no
+ # effect compared to previous state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 4
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+
+ # Disable neighbor suppression on VLAN 10 and check that ARP requests
+ # are no longer suppressed on this VLAN.
+ run_cmd "bridge -n sw1 vlan set vid $vid1 dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid1 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off (VLAN $vid1)"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 5
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+
+ # Disable per-{Port, VLAN} neighbor suppression, enable neighbor
+ # suppression on the port and check that on both VLANs ARP requests are
+ # suppressed.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_vlan_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_vlan_suppress off\""
+ log_test $? 0 "\"neigh_vlan_suppress\" is off"
+
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip1 -I eth0.$vid1 $tip1"
+ log_test $? 0 "arping (VLAN $vid1)"
+ run_cmd "ip netns exec h1 arping -q -b -c 1 -w 5 -s $sip2 -I eth0.$vid2 $tip2"
+ log_test $? 0 "arping (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "ARP suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 5
+ log_test $? 0 "ARP suppression (VLAN $vid2)"
+}
+
+neigh_vlan_suppress_ns()
+{
+ local vid1=10
+ local vid2=20
+ local saddr1=2001:db8:1::1
+ local saddr2=2001:db8:2::1
+ local daddr1=2001:db8:1::2
+ local daddr2=2001:db8:2::2
+ local maddr=ff02::1:ff00:2
+ local h2_mac1
+ local h2_mac2
+
+ echo
+ echo "Per-{Port, VLAN} NS suppression"
+ echo "-------------------------------"
+
+ run_cmd "tc -n sw1 qdisc replace dev vx0 clsact"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 101 proto ipv6 flower indev swp1 ip_proto icmpv6 dst_ip $maddr src_ip $saddr1 type 135 code 0 action pass"
+ run_cmd "tc -n sw1 filter replace dev vx0 egress pref 1 handle 102 proto ipv6 flower indev swp1 ip_proto icmpv6 dst_ip $maddr src_ip $saddr2 type 135 code 0 action pass"
+
+ h2_mac1=$(ip -n h2 -j -p link show eth0.$vid1 | jq -r '.[]["address"]')
+ h2_mac2=$(ip -n h2 -j -p link show eth0.$vid2 | jq -r '.[]["address"]')
+ run_cmd "bridge -n sw1 fdb replace $h2_mac1 dev vx0 master static vlan $vid1"
+ run_cmd "bridge -n sw1 fdb replace $h2_mac2 dev vx0 master static vlan $vid2"
+ run_cmd "ip -n sw1 neigh replace $daddr1 lladdr $h2_mac1 nud permanent dev br0.$vid1"
+ run_cmd "ip -n sw1 neigh replace $daddr2 lladdr $h2_mac2 nud permanent dev br0.$vid2"
+
+ # Enable per-{Port, VLAN} neighbor suppression and check that NS
+ # messages are not suppressed and that ND messages are received.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_vlan_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_vlan_suppress on\""
+ log_test $? 0 "\"neigh_vlan_suppress\" is on"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 1
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+
+ # Enable neighbor suppression on VLAN 10 and check that only on this
+ # VLAN NS messages are suppressed.
+ run_cmd "bridge -n sw1 vlan set vid $vid1 dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid1 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on (VLAN $vid1)"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid2 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off (VLAN $vid2)"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 2
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+
+ # Enable neighbor suppression on the port and check that it has no
+ # effect compared to previous state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 3
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+
+ # Disable neighbor suppression on the port and check that it has no
+ # effect compared to previous state.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 1
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 4
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+
+ # Disable neighbor suppression on VLAN 10 and check that NS messages
+ # are no longer suppressed on this VLAN.
+ run_cmd "bridge -n sw1 vlan set vid $vid1 dev vx0 neigh_suppress off"
+ run_cmd "bridge -n sw1 -d vlan show dev vx0 vid $vid1 | grep \"neigh_suppress off\""
+ log_test $? 0 "\"neigh_suppress\" is off (VLAN $vid1)"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 5
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+
+ # Disable per-{Port, VLAN} neighbor suppression, enable neighbor
+ # suppression on the port and check that on both VLANs NS messages are
+ # suppressed.
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_vlan_suppress off"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_vlan_suppress off\""
+ log_test $? 0 "\"neigh_vlan_suppress\" is off"
+
+ run_cmd "bridge -n sw1 link set dev vx0 neigh_suppress on"
+ run_cmd "bridge -n sw1 -d link show dev vx0 | grep \"neigh_suppress on\""
+ log_test $? 0 "\"neigh_suppress\" is on"
+
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr1 -w 5000 $daddr1 eth0.$vid1"
+ log_test $? 0 "ndisc6 (VLAN $vid1)"
+ run_cmd "ip netns exec h1 ndisc6 -q -r 1 -s $saddr2 -w 5000 $daddr2 eth0.$vid2"
+ log_test $? 0 "ndisc6 (VLAN $vid2)"
+
+ tc_check_packets sw1 "dev vx0 egress" 101 2
+ log_test $? 0 "NS suppression (VLAN $vid1)"
+ tc_check_packets sw1 "dev vx0 egress" 102 5
+ log_test $? 0 "NS suppression (VLAN $vid2)"
+}
+
+################################################################################
+# Usage
+
+usage()
+{
+ cat <<EOF
+usage: ${0##*/} OPTS
+
+ -t <test> Test(s) to run (default: all)
+ (options: $TESTS)
+ -p Pause on fail
+ -P Pause after each test before cleanup
+ -v Verbose mode (show commands and output)
+EOF
+}
+
+################################################################################
+# Main
+
+trap cleanup EXIT
+
+while getopts ":t:pPvh" opt; do
+ case $opt in
+ t) TESTS=$OPTARG;;
+ p) PAUSE_ON_FAIL=yes;;
+ P) PAUSE=yes;;
+ v) VERBOSE=$(($VERBOSE + 1));;
+ h) usage; exit 0;;
+ *) usage; exit 1;;
+ esac
+done
+
+# Make sure we don't pause twice.
+[ "${PAUSE}" = "yes" ] && PAUSE_ON_FAIL=no
+
+if [ "$(id -u)" -ne 0 ];then
+ echo "SKIP: Need root privileges"
+ exit $ksft_skip;
+fi
+
+if [ ! -x "$(command -v ip)" ]; then
+ echo "SKIP: Could not run test without ip tool"
+ exit $ksft_skip
+fi
+
+if [ ! -x "$(command -v bridge)" ]; then
+ echo "SKIP: Could not run test without bridge tool"
+ exit $ksft_skip
+fi
+
+if [ ! -x "$(command -v tc)" ]; then
+ echo "SKIP: Could not run test without tc tool"
+ exit $ksft_skip
+fi
+
+if [ ! -x "$(command -v arping)" ]; then
+ echo "SKIP: Could not run test without arping tool"
+ exit $ksft_skip
+fi
+
+if [ ! -x "$(command -v ndisc6)" ]; then
+ echo "SKIP: Could not run test without ndisc6 tool"
+ exit $ksft_skip
+fi
+
+if [ ! -x "$(command -v jq)" ]; then
+ echo "SKIP: Could not run test without jq tool"
+ exit $ksft_skip
+fi
+
+bridge link help 2>&1 | grep -q "neigh_vlan_suppress"
+if [ $? -ne 0 ]; then
+ echo "SKIP: iproute2 bridge too old, missing per-VLAN neighbor suppression support"
+ exit $ksft_skip
+fi
+
+# Start clean.
+cleanup
+
+for t in $TESTS
+do
+ setup; $t; cleanup;
+done
+
+if [ "$TESTS" != "none" ]; then
+ printf "\nTests passed: %3d\n" ${nsuccess}
+ printf "Tests failed: %3d\n" ${nfail}
+fi
+
+exit $ret
diff --git a/usr/gen_init_cpio.c b/usr/gen_init_cpio.c
index ee01e40..6123053 100644
--- a/usr/gen_init_cpio.c
+++ b/usr/gen_init_cpio.c
@@ -353,6 +353,12 @@ static int cpio_mkfile(const char *name, const char *location,
buf.st_mtime = 0xffffffff;
}
+ if (buf.st_mtime < 0) {
+ fprintf(stderr, "%s: Timestamp negative, clipping.\n",
+ location);
+ buf.st_mtime = 0;
+ }
+
if (buf.st_size > 0xffffffff) {
fprintf(stderr, "%s: Size exceeds maximum cpio file size\n",
location);
@@ -602,10 +608,10 @@ int main (int argc, char *argv[])
/*
* Timestamps after 2106-02-07 06:28:15 UTC have an ascii hex time_t
* representation that exceeds 8 chars and breaks the cpio header
- * specification.
+ * specification. Negative timestamps similarly exceed 8 chars.
*/
- if (default_mtime > 0xffffffff) {
- fprintf(stderr, "ERROR: Timestamp too large for cpio format\n");
+ if (default_mtime > 0xffffffff || default_mtime < 0) {
+ fprintf(stderr, "ERROR: Timestamp out of range for cpio format\n");
exit(1);
}
