blob: a05e751363a02c40c88d6ee278c87d73204c62da [file] [log] [blame]
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -03001Introduction
2============
3
Ed Cashin5b8c4be2012-10-04 17:16:49 -07004ATA over Ethernet is a network protocol that provides simple access to
5block storage on the LAN.
6
7 http://support.coraid.com/documents/AoEr11.txt
8
Ed Cashineecdf222012-10-04 17:16:39 -07009The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ...
Linus Torvalds1da177e2005-04-16 15:20:36 -070010
Ed Cashineecdf222012-10-04 17:16:39 -070011 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html
Linus Torvalds1da177e2005-04-16 15:20:36 -070012
Ed Cashineecdf222012-10-04 17:16:39 -070013It has many tips and hints! Please see, especially, recommended
14tunings for virtual memory:
15
16 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19
Linus Torvalds1da177e2005-04-16 15:20:36 -070017
Ed L Cashin67d9f842005-04-29 10:24:18 -040018The aoetools are userland programs that are designed to work with this
19driver. The aoetools are on sourceforge.
20
21 http://aoetools.sourceforge.net/
22
Mauro Carvalho Chehab83bbf6e2019-06-27 12:59:40 -030023The scripts in this Documentation/admin-guide/aoe directory are intended to
Ed L Cashin67d9f842005-04-29 10:24:18 -040024document the use of the driver and are not necessary if you install
25the aoetools.
26
27
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030028Creating Device Nodes
29=====================
Linus Torvalds1da177e2005-04-16 15:20:36 -070030
31 Users of udev should find the block device nodes created
32 automatically, but to create all the necessary device nodes, use the
33 udev configuration rules provided in udev.txt (in this directory).
34
35 There is a udev-install.sh script that shows how to install these
36 rules on your system.
37
Linus Torvalds1da177e2005-04-16 15:20:36 -070038 There is also an autoload script that shows how to edit
Lucas De Marchi970e2482012-03-30 13:37:16 -070039 /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when
Ed Cashin5b8c4be2012-10-04 17:16:49 -070040 necessary. Preloading the aoe module is preferable to autoloading,
41 however, because AoE discovery takes a few seconds. It can be
42 confusing when an AoE device is not present the first time the a
43 command is run but appears a second later.
Linus Torvalds1da177e2005-04-16 15:20:36 -070044
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030045Using Device Nodes
46==================
Linus Torvalds1da177e2005-04-16 15:20:36 -070047
48 "cat /dev/etherd/err" blocks, waiting for error diagnostic output,
49 like any retransmitted packets.
50
51 "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to
52 limit ATA over Ethernet traffic to eth2 and eth4. AoE traffic from
Ed L Cashin67d9f842005-04-29 10:24:18 -040053 untrusted networks should be ignored as a matter of security. See
54 also the aoe_iflist driver option described below.
Linus Torvalds1da177e2005-04-16 15:20:36 -070055
56 "echo > /dev/etherd/discover" tells the driver to find out what AoE
57 devices are available.
58
Ed Cashin5b8c4be2012-10-04 17:16:49 -070059 In the future these character devices may disappear and be replaced
60 by sysfs counterparts. Using the commands in aoetools insulates
61 users from these implementation details.
Linus Torvalds1da177e2005-04-16 15:20:36 -070062
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030063 The block devices are named like this::
Linus Torvalds1da177e2005-04-16 15:20:36 -070064
65 e{shelf}.{slot}
66 e{shelf}.{slot}p{part}
67
68 ... so that "e0.2" is the third blade from the left (slot 2) in the
69 first shelf (shelf address zero). That's the whole disk. The first
70 partition on that disk would be "e0.2p1".
71
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030072Using sysfs
73===========
Linus Torvalds1da177e2005-04-16 15:20:36 -070074
75 Each aoe block device in /sys/block has the extra attributes of
76 state, mac, and netif. The state attribute is "up" when the device
77 is ready for I/O and "down" if detected but unusable. The
78 "down,closewait" state shows that the device is still open and
79 cannot come up again until it has been closed.
80
81 The mac attribute is the ethernet address of the remote AoE device.
82 The netif attribute is the network interface on the localhost
83 through which we are communicating with the remote AoE device.
84
Ed Cashin5b8c4be2012-10-04 17:16:49 -070085 There is a script in this directory that formats this information in
86 a convenient way. Users with aoetools should use the aoe-stat
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030087 command::
Linus Torvalds1da177e2005-04-16 15:20:36 -070088
Mauro Carvalho Chehab83bbf6e2019-06-27 12:59:40 -030089 root@makki root# sh Documentation/admin-guide/aoe/status.sh
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -030090 e10.0 eth3 up
91 e10.1 eth3 up
92 e10.2 eth3 up
93 e10.3 eth3 up
94 e10.4 eth3 up
95 e10.5 eth3 up
96 e10.6 eth3 up
97 e10.7 eth3 up
98 e10.8 eth3 up
99 e10.9 eth3 up
100 e4.0 eth1 up
101 e4.1 eth1 up
102 e4.2 eth1 up
103 e4.3 eth1 up
104 e4.4 eth1 up
105 e4.5 eth1 up
106 e4.6 eth1 up
107 e4.7 eth1 up
108 e4.8 eth1 up
109 e4.9 eth1 up
Ed L Cashin03c41c42005-04-29 10:24:03 -0400110
111 Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver
112 option discussed below) instead of /dev/etherd/interfaces to limit
113 AoE traffic to the network interfaces in the given
114 whitespace-separated list. Unlike the old character device, the
115 sysfs entry can be read from as well as written to.
116
117 It's helpful to trigger discovery after setting the list of allowed
Ed L Cashin67d9f842005-04-29 10:24:18 -0400118 interfaces. The aoetools package provides an aoe-discover script
119 for this purpose. You can also directly use the
120 /dev/etherd/discover special file described above.
Ed L Cashin03c41c42005-04-29 10:24:03 -0400121
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -0300122Driver Options
123==============
Ed L Cashin03c41c42005-04-29 10:24:03 -0400124
125 There is a boot option for the built-in aoe driver and a
126 corresponding module parameter, aoe_iflist. Without this option,
127 all network interfaces may be used for ATA over Ethernet. Here is a
Mauro Carvalho Chehab305a99eb2019-06-12 14:52:37 -0300128 usage example for the module parameter::
Ed L Cashin03c41c42005-04-29 10:24:03 -0400129
130 modprobe aoe_iflist="eth1 eth3"
Ed Cashin5b8c4be2012-10-04 17:16:49 -0700131
132 The aoe_deadsecs module parameter determines the maximum number of
133 seconds that the driver will wait for an AoE device to provide a
134 response to an AoE command. After aoe_deadsecs seconds have
Ed Cashinc450ba02012-12-17 16:04:14 -0800135 elapsed, the AoE device will be marked as "down". A value of zero
136 is supported for testing purposes and makes the aoe driver keep
137 trying AoE commands forever.
Ed Cashin5b8c4be2012-10-04 17:16:49 -0700138
139 The aoe_maxout module parameter has a default of 128. This is the
140 maximum number of unresponded packets that will be sent to an AoE
141 target at one time.
142
143 The aoe_dyndevs module parameter defaults to 1, meaning that the
144 driver will assign a block device minor number to a discovered AoE
145 target based on the order of its discovery. With dynamic minor
146 device numbers in use, a greater range of AoE shelf and slot
147 addresses can be supported. Users with udev will never have to
148 think about minor numbers. Using aoe_dyndevs=0 allows device nodes
149 to be pre-created using a static minor-number scheme with the
150 aoe-mkshelf script in the aoetools.