| ========================= |
| NXP SJA1105 switch driver |
| ========================= |
| |
| Overview |
| ======== |
| |
| The NXP SJA1105 is a family of 10 SPI-managed automotive switches: |
| |
| - SJA1105E: First generation, no TTEthernet |
| - SJA1105T: First generation, TTEthernet |
| - SJA1105P: Second generation, no TTEthernet, no SGMII |
| - SJA1105Q: Second generation, TTEthernet, no SGMII |
| - SJA1105R: Second generation, no TTEthernet, SGMII |
| - SJA1105S: Second generation, TTEthernet, SGMII |
| - SJA1110A: Third generation, TTEthernet, SGMII, integrated 100base-T1 and |
| 100base-TX PHYs |
| - SJA1110B: Third generation, TTEthernet, SGMII, 100base-T1, 100base-TX |
| - SJA1110C: Third generation, TTEthernet, SGMII, 100base-T1, 100base-TX |
| - SJA1110D: Third generation, TTEthernet, SGMII, 100base-T1 |
| |
| Being automotive parts, their configuration interface is geared towards |
| set-and-forget use, with minimal dynamic interaction at runtime. They |
| require a static configuration to be composed by software and packed |
| with CRC and table headers, and sent over SPI. |
| |
| The static configuration is composed of several configuration tables. Each |
| table takes a number of entries. Some configuration tables can be (partially) |
| reconfigured at runtime, some not. Some tables are mandatory, some not: |
| |
| ============================= ================== ============================= |
| Table Mandatory Reconfigurable |
| ============================= ================== ============================= |
| Schedule no no |
| Schedule entry points if Scheduling no |
| VL Lookup no no |
| VL Policing if VL Lookup no |
| VL Forwarding if VL Lookup no |
| L2 Lookup no no |
| L2 Policing yes no |
| VLAN Lookup yes yes |
| L2 Forwarding yes partially (fully on P/Q/R/S) |
| MAC Config yes partially (fully on P/Q/R/S) |
| Schedule Params if Scheduling no |
| Schedule Entry Points Params if Scheduling no |
| VL Forwarding Params if VL Forwarding no |
| L2 Lookup Params no partially (fully on P/Q/R/S) |
| L2 Forwarding Params yes no |
| Clock Sync Params no no |
| AVB Params no no |
| General Params yes partially |
| Retagging no yes |
| xMII Params yes no |
| SGMII no yes |
| ============================= ================== ============================= |
| |
| |
| Also the configuration is write-only (software cannot read it back from the |
| switch except for very few exceptions). |
| |
| The driver creates a static configuration at probe time, and keeps it at |
| all times in memory, as a shadow for the hardware state. When required to |
| change a hardware setting, the static configuration is also updated. |
| If that changed setting can be transmitted to the switch through the dynamic |
| reconfiguration interface, it is; otherwise the switch is reset and |
| reprogrammed with the updated static configuration. |
| |
| Switching features |
| ================== |
| |
| The driver supports the configuration of L2 forwarding rules in hardware for |
| port bridging. The forwarding, broadcast and flooding domain between ports can |
| be restricted through two methods: either at the L2 forwarding level (isolate |
| one bridge's ports from another's) or at the VLAN port membership level |
| (isolate ports within the same bridge). The final forwarding decision taken by |
| the hardware is a logical AND of these two sets of rules. |
| |
| The hardware tags all traffic internally with a port-based VLAN (pvid), or it |
| decodes the VLAN information from the 802.1Q tag. Advanced VLAN classification |
| is not possible. Once attributed a VLAN tag, frames are checked against the |
| port's membership rules and dropped at ingress if they don't match any VLAN. |
| This behavior is available when switch ports are enslaved to a bridge with |
| ``vlan_filtering 1``. |
| |
| Normally the hardware is not configurable with respect to VLAN awareness, but |
| by changing what TPID the switch searches 802.1Q tags for, the semantics of a |
| bridge with ``vlan_filtering 0`` can be kept (accept all traffic, tagged or |
| untagged), and therefore this mode is also supported. |
| |
| Segregating the switch ports in multiple bridges is supported (e.g. 2 + 2), but |
| all bridges should have the same level of VLAN awareness (either both have |
| ``vlan_filtering`` 0, or both 1). |
| |
| Topology and loop detection through STP is supported. |
| |
| Offloads |
| ======== |
| |
| Time-aware scheduling |
| --------------------- |
| |
| The switch supports a variation of the enhancements for scheduled traffic |
| specified in IEEE 802.1Q-2018 (formerly 802.1Qbv). This means it can be used to |
| ensure deterministic latency for priority traffic that is sent in-band with its |
| gate-open event in the network schedule. |
| |
| This capability can be managed through the tc-taprio offload ('flags 2'). The |
| difference compared to the software implementation of taprio is that the latter |
| would only be able to shape traffic originated from the CPU, but not |
| autonomously forwarded flows. |
| |
| The device has 8 traffic classes, and maps incoming frames to one of them based |
| on the VLAN PCP bits (if no VLAN is present, the port-based default is used). |
| As described in the previous sections, depending on the value of |
| ``vlan_filtering``, the EtherType recognized by the switch as being VLAN can |
| either be the typical 0x8100 or a custom value used internally by the driver |
| for tagging. Therefore, the switch ignores the VLAN PCP if used in standalone |
| or bridge mode with ``vlan_filtering=0``, as it will not recognize the 0x8100 |
| EtherType. In these modes, injecting into a particular TX queue can only be |
| done by the DSA net devices, which populate the PCP field of the tagging header |
| on egress. Using ``vlan_filtering=1``, the behavior is the other way around: |
| offloaded flows can be steered to TX queues based on the VLAN PCP, but the DSA |
| net devices are no longer able to do that. To inject frames into a hardware TX |
| queue with VLAN awareness active, it is necessary to create a VLAN |
| sub-interface on the DSA master port, and send normal (0x8100) VLAN-tagged |
| towards the switch, with the VLAN PCP bits set appropriately. |
| |
| Management traffic (having DMAC 01-80-C2-xx-xx-xx or 01-19-1B-xx-xx-xx) is the |
| notable exception: the switch always treats it with a fixed priority and |
| disregards any VLAN PCP bits even if present. The traffic class for management |
| traffic has a value of 7 (highest priority) at the moment, which is not |
| configurable in the driver. |
| |
| Below is an example of configuring a 500 us cyclic schedule on egress port |
| ``swp5``. The traffic class gate for management traffic (7) is open for 100 us, |
| and the gates for all other traffic classes are open for 400 us:: |
| |
| #!/bin/bash |
| |
| set -e -u -o pipefail |
| |
| NSEC_PER_SEC="1000000000" |
| |
| gatemask() { |
| local tc_list="$1" |
| local mask=0 |
| |
| for tc in ${tc_list}; do |
| mask=$((${mask} | (1 << ${tc}))) |
| done |
| |
| printf "%02x" ${mask} |
| } |
| |
| if ! systemctl is-active --quiet ptp4l; then |
| echo "Please start the ptp4l service" |
| exit |
| fi |
| |
| now=$(phc_ctl /dev/ptp1 get | gawk '/clock time is/ { print $5; }') |
| # Phase-align the base time to the start of the next second. |
| sec=$(echo "${now}" | gawk -F. '{ print $1; }') |
| base_time="$(((${sec} + 1) * ${NSEC_PER_SEC}))" |
| |
| tc qdisc add dev swp5 parent root handle 100 taprio \ |
| num_tc 8 \ |
| map 0 1 2 3 5 6 7 \ |
| queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \ |
| base-time ${base_time} \ |
| sched-entry S $(gatemask 7) 100000 \ |
| sched-entry S $(gatemask "0 1 2 3 4 5 6") 400000 \ |
| flags 2 |
| |
| It is possible to apply the tc-taprio offload on multiple egress ports. There |
| are hardware restrictions related to the fact that no gate event may trigger |
| simultaneously on two ports. The driver checks the consistency of the schedules |
| against this restriction and errors out when appropriate. Schedule analysis is |
| needed to avoid this, which is outside the scope of the document. |
| |
| Routing actions (redirect, trap, drop) |
| -------------------------------------- |
| |
| The switch is able to offload flow-based redirection of packets to a set of |
| destination ports specified by the user. Internally, this is implemented by |
| making use of Virtual Links, a TTEthernet concept. |
| |
| The driver supports 2 types of keys for Virtual Links: |
| |
| - VLAN-aware virtual links: these match on destination MAC address, VLAN ID and |
| VLAN PCP. |
| - VLAN-unaware virtual links: these match on destination MAC address only. |
| |
| The VLAN awareness state of the bridge (vlan_filtering) cannot be changed while |
| there are virtual link rules installed. |
| |
| Composing multiple actions inside the same rule is supported. When only routing |
| actions are requested, the driver creates a "non-critical" virtual link. When |
| the action list also contains tc-gate (more details below), the virtual link |
| becomes "time-critical" (draws frame buffers from a reserved memory partition, |
| etc). |
| |
| The 3 routing actions that are supported are "trap", "drop" and "redirect". |
| |
| Example 1: send frames received on swp2 with a DA of 42:be:24:9b:76:20 to the |
| CPU and to swp3. This type of key (DA only) when the port's VLAN awareness |
| state is off:: |
| |
| tc qdisc add dev swp2 clsact |
| tc filter add dev swp2 ingress flower skip_sw dst_mac 42:be:24:9b:76:20 \ |
| action mirred egress redirect dev swp3 \ |
| action trap |
| |
| Example 2: drop frames received on swp2 with a DA of 42:be:24:9b:76:20, a VID |
| of 100 and a PCP of 0:: |
| |
| tc filter add dev swp2 ingress protocol 802.1Q flower skip_sw \ |
| dst_mac 42:be:24:9b:76:20 vlan_id 100 vlan_prio 0 action drop |
| |
| Time-based ingress policing |
| --------------------------- |
| |
| The TTEthernet hardware abilities of the switch can be constrained to act |
| similarly to the Per-Stream Filtering and Policing (PSFP) clause specified in |
| IEEE 802.1Q-2018 (formerly 802.1Qci). This means it can be used to perform |
| tight timing-based admission control for up to 1024 flows (identified by a |
| tuple composed of destination MAC address, VLAN ID and VLAN PCP). Packets which |
| are received outside their expected reception window are dropped. |
| |
| This capability can be managed through the offload of the tc-gate action. As |
| routing actions are intrinsic to virtual links in TTEthernet (which performs |
| explicit routing of time-critical traffic and does not leave that in the hands |
| of the FDB, flooding etc), the tc-gate action may never appear alone when |
| asking sja1105 to offload it. One (or more) redirect or trap actions must also |
| follow along. |
| |
| Example: create a tc-taprio schedule that is phase-aligned with a tc-gate |
| schedule (the clocks must be synchronized by a 1588 application stack, which is |
| outside the scope of this document). No packet delivered by the sender will be |
| dropped. Note that the reception window is larger than the transmission window |
| (and much more so, in this example) to compensate for the packet propagation |
| delay of the link (which can be determined by the 1588 application stack). |
| |
| Receiver (sja1105):: |
| |
| tc qdisc add dev swp2 clsact |
| now=$(phc_ctl /dev/ptp1 get | awk '/clock time is/ {print $5}') && \ |
| sec=$(echo $now | awk -F. '{print $1}') && \ |
| base_time="$(((sec + 2) * 1000000000))" && \ |
| echo "base time ${base_time}" |
| tc filter add dev swp2 ingress flower skip_sw \ |
| dst_mac 42:be:24:9b:76:20 \ |
| action gate base-time ${base_time} \ |
| sched-entry OPEN 60000 -1 -1 \ |
| sched-entry CLOSE 40000 -1 -1 \ |
| action trap |
| |
| Sender:: |
| |
| now=$(phc_ctl /dev/ptp0 get | awk '/clock time is/ {print $5}') && \ |
| sec=$(echo $now | awk -F. '{print $1}') && \ |
| base_time="$(((sec + 2) * 1000000000))" && \ |
| echo "base time ${base_time}" |
| tc qdisc add dev eno0 parent root taprio \ |
| num_tc 8 \ |
| map 0 1 2 3 4 5 6 7 \ |
| queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \ |
| base-time ${base_time} \ |
| sched-entry S 01 50000 \ |
| sched-entry S 00 50000 \ |
| flags 2 |
| |
| The engine used to schedule the ingress gate operations is the same that the |
| one used for the tc-taprio offload. Therefore, the restrictions regarding the |
| fact that no two gate actions (either tc-gate or tc-taprio gates) may fire at |
| the same time (during the same 200 ns slot) still apply. |
| |
| To come in handy, it is possible to share time-triggered virtual links across |
| more than 1 ingress port, via flow blocks. In this case, the restriction of |
| firing at the same time does not apply because there is a single schedule in |
| the system, that of the shared virtual link:: |
| |
| tc qdisc add dev swp2 ingress_block 1 clsact |
| tc qdisc add dev swp3 ingress_block 1 clsact |
| tc filter add block 1 flower skip_sw dst_mac 42:be:24:9b:76:20 \ |
| action gate index 2 \ |
| base-time 0 \ |
| sched-entry OPEN 50000000 -1 -1 \ |
| sched-entry CLOSE 50000000 -1 -1 \ |
| action trap |
| |
| Hardware statistics for each flow are also available ("pkts" counts the number |
| of dropped frames, which is a sum of frames dropped due to timing violations, |
| lack of destination ports and MTU enforcement checks). Byte-level counters are |
| not available. |
| |
| Limitations |
| =========== |
| |
| The SJA1105 switch family always performs VLAN processing. When configured as |
| VLAN-unaware, frames carry a different VLAN tag internally, depending on |
| whether the port is standalone or under a VLAN-unaware bridge. |
| |
| The virtual link keys are always fixed at {MAC DA, VLAN ID, VLAN PCP}, but the |
| driver asks for the VLAN ID and VLAN PCP when the port is under a VLAN-aware |
| bridge. Otherwise, it fills in the VLAN ID and PCP automatically, based on |
| whether the port is standalone or in a VLAN-unaware bridge, and accepts only |
| "VLAN-unaware" tc-flower keys (MAC DA). |
| |
| The existing tc-flower keys that are offloaded using virtual links are no |
| longer operational after one of the following happens: |
| |
| - port was standalone and joins a bridge (VLAN-aware or VLAN-unaware) |
| - port is part of a bridge whose VLAN awareness state changes |
| - port was part of a bridge and becomes standalone |
| - port was standalone, but another port joins a VLAN-aware bridge and this |
| changes the global VLAN awareness state of the bridge |
| |
| The driver cannot veto all these operations, and it cannot update/remove the |
| existing tc-flower filters either. So for proper operation, the tc-flower |
| filters should be installed only after the forwarding configuration of the port |
| has been made, and removed by user space before making any changes to it. |
| |
| Device Tree bindings and board design |
| ===================================== |
| |
| This section references ``Documentation/devicetree/bindings/net/dsa/nxp,sja1105.yaml`` |
| and aims to showcase some potential switch caveats. |
| |
| RMII PHY role and out-of-band signaling |
| --------------------------------------- |
| |
| In the RMII spec, the 50 MHz clock signals are either driven by the MAC or by |
| an external oscillator (but not by the PHY). |
| But the spec is rather loose and devices go outside it in several ways. |
| Some PHYs go against the spec and may provide an output pin where they source |
| the 50 MHz clock themselves, in an attempt to be helpful. |
| On the other hand, the SJA1105 is only binary configurable - when in the RMII |
| MAC role it will also attempt to drive the clock signal. To prevent this from |
| happening it must be put in RMII PHY role. |
| But doing so has some unintended consequences. |
| In the RMII spec, the PHY can transmit extra out-of-band signals via RXD[1:0]. |
| These are practically some extra code words (/J/ and /K/) sent prior to the |
| preamble of each frame. The MAC does not have this out-of-band signaling |
| mechanism defined by the RMII spec. |
| So when the SJA1105 port is put in PHY role to avoid having 2 drivers on the |
| clock signal, inevitably an RMII PHY-to-PHY connection is created. The SJA1105 |
| emulates a PHY interface fully and generates the /J/ and /K/ symbols prior to |
| frame preambles, which the real PHY is not expected to understand. So the PHY |
| simply encodes the extra symbols received from the SJA1105-as-PHY onto the |
| 100Base-Tx wire. |
| On the other side of the wire, some link partners might discard these extra |
| symbols, while others might choke on them and discard the entire Ethernet |
| frames that follow along. This looks like packet loss with some link partners |
| but not with others. |
| The take-away is that in RMII mode, the SJA1105 must be let to drive the |
| reference clock if connected to a PHY. |
| |
| RGMII fixed-link and internal delays |
| ------------------------------------ |
| |
| As mentioned in the bindings document, the second generation of devices has |
| tunable delay lines as part of the MAC, which can be used to establish the |
| correct RGMII timing budget. |
| When powered up, these can shift the Rx and Tx clocks with a phase difference |
| between 73.8 and 101.7 degrees. |
| The catch is that the delay lines need to lock onto a clock signal with a |
| stable frequency. This means that there must be at least 2 microseconds of |
| silence between the clock at the old vs at the new frequency. Otherwise the |
| lock is lost and the delay lines must be reset (powered down and back up). |
| In RGMII the clock frequency changes with link speed (125 MHz at 1000 Mbps, 25 |
| MHz at 100 Mbps and 2.5 MHz at 10 Mbps), and link speed might change during the |
| AN process. |
| In the situation where the switch port is connected through an RGMII fixed-link |
| to a link partner whose link state life cycle is outside the control of Linux |
| (such as a different SoC), then the delay lines would remain unlocked (and |
| inactive) until there is manual intervention (ifdown/ifup on the switch port). |
| The take-away is that in RGMII mode, the switch's internal delays are only |
| reliable if the link partner never changes link speeds, or if it does, it does |
| so in a way that is coordinated with the switch port (practically, both ends of |
| the fixed-link are under control of the same Linux system). |
| As to why would a fixed-link interface ever change link speeds: there are |
| Ethernet controllers out there which come out of reset in 100 Mbps mode, and |
| their driver inevitably needs to change the speed and clock frequency if it's |
| required to work at gigabit. |
| |
| MDIO bus and PHY management |
| --------------------------- |
| |
| The SJA1105 does not have an MDIO bus and does not perform in-band AN either. |
| Therefore there is no link state notification coming from the switch device. |
| A board would need to hook up the PHYs connected to the switch to any other |
| MDIO bus available to Linux within the system (e.g. to the DSA master's MDIO |
| bus). Link state management then works by the driver manually keeping in sync |
| (over SPI commands) the MAC link speed with the settings negotiated by the PHY. |
| |
| By comparison, the SJA1110 supports an MDIO slave access point over which its |
| internal 100base-T1 PHYs can be accessed from the host. This is, however, not |
| used by the driver, instead the internal 100base-T1 and 100base-TX PHYs are |
| accessed through SPI commands, modeled in Linux as virtual MDIO buses. |
| |
| The microcontroller attached to the SJA1110 port 0 also has an MDIO controller |
| operating in master mode, however the driver does not support this either, |
| since the microcontroller gets disabled when the Linux driver operates. |
| Discrete PHYs connected to the switch ports should have their MDIO interface |
| attached to an MDIO controller from the host system and not to the switch, |
| similar to SJA1105. |
| |
| Port compatibility matrix |
| ------------------------- |
| |
| The SJA1105 port compatibility matrix is: |
| |
| ===== ============== ============== ============== |
| Port SJA1105E/T SJA1105P/Q SJA1105R/S |
| ===== ============== ============== ============== |
| 0 xMII xMII xMII |
| 1 xMII xMII xMII |
| 2 xMII xMII xMII |
| 3 xMII xMII xMII |
| 4 xMII xMII SGMII |
| ===== ============== ============== ============== |
| |
| |
| The SJA1110 port compatibility matrix is: |
| |
| ===== ============== ============== ============== ============== |
| Port SJA1110A SJA1110B SJA1110C SJA1110D |
| ===== ============== ============== ============== ============== |
| 0 RevMII (uC) RevMII (uC) RevMII (uC) RevMII (uC) |
| 1 100base-TX 100base-TX 100base-TX |
| or SGMII SGMII |
| 2 xMII xMII xMII xMII |
| or SGMII or SGMII |
| 3 xMII xMII xMII |
| or SGMII or SGMII SGMII |
| or 2500base-X or 2500base-X or 2500base-X |
| 4 SGMII SGMII SGMII SGMII |
| or 2500base-X or 2500base-X or 2500base-X or 2500base-X |
| 5 100base-T1 100base-T1 100base-T1 100base-T1 |
| 6 100base-T1 100base-T1 100base-T1 100base-T1 |
| 7 100base-T1 100base-T1 100base-T1 100base-T1 |
| 8 100base-T1 100base-T1 n/a n/a |
| 9 100base-T1 100base-T1 n/a n/a |
| 10 100base-T1 n/a n/a n/a |
| ===== ============== ============== ============== ============== |
