Back in August, I was looking for a
Homeserver replacement. During FrOSCon I was then reminded
of the Turris Omnia project by NIC.cz. The basic SoC (Marvel
Armada 38x) seemed to be nice hand have decent mainline support (and,
with the turris, users interested in keeping it working). Only I don't
want any WIFI and I wasn't sure the standard case would be all that
usefully. Fortunately, there's also a simple board available with the
same SoC called ClearFog and so I got one of these (the Base
version). With shipping and the SSD (the only 2242 M.2 SSD with 250
GiB I could find, a ADATA SP600) it slightly exceeds the budget but
well.
When installing the machine, the obvious goal was to use mainline FOSS
components only if possible. Fortunately there's mainline kernel
support for the device as well as mainline U-Boot. First attempts
to boot from a micro SD card did not work out at all, both with
mainline U-Boot and the vendor version though. Turns out the eMMC
version of the board does not support any micro SD cards at all, a
fact that is documented but others failed to notice as well.
U-Boot
As the board does not come with any loader on eMMC and booting
directly from M.2 requires removing some resistors from the board, the
easiest way is using UART for booting. The vendor wiki has some shell
script wrapping an included C fragment to feed U-Boot to the device
but all that is really needed is U-Boot's kwboot utility. For some
reason the SPL didn't properly detect UART booting on my device (wrong
magic number) but patching the if (in arch-mvebu's spl.c) and always
assume UART boot is an easy way around.
The plan then was to boot a Debian armhf rootfs with a
defconfig kernel from USB stick. and install U-Boot and the rootfs
to eMMC from within that system. Unfortunately U-Boot seems to be
unable to talk to the USB3 port so no kernel loading from there. One
could probably make UART loading work but switching between screen for
serial console and xmodem seemed somewhat fragile and I never got it
working. However ethernet can be made to work, though you need to set
eth1addr to eth3addr (or just the right one of these) in U-Boot,
saveenv and reboot. After that TFTP works (but is somewhat slow).
eMMC
There's one last step required to allow U-Boot and Linux to access the
eMMC. eMMC is wired to the same PINs as the SD card would be. However
the SD card has an additional indicator pin showing whether a card is
present. You might be lucky inserting a dummy card into the slot or go
the clean route and remove the pin specification from the device tree.
--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -306,7 +307,6 @@
sdhci@d8000 {
bus-width = <4>;
- cd-gpios = <&gpio0 20 GPIO_ACTIVE_LOW>;
no-1-8-v;
pinctrl-0 = <&clearfog_sdhci_pins
&clearfog_sdhci_cd_pins>;
Next Up is flashing the U-Boot to eMMC. This seems to
work with the vendor U-Boot but proves to be tricky with
mainline. The fun part boils down to the fact that the boot firmware
reads the first block from eMMC, but the second from SD card. If
you write the mainline U-Boot, which was written and tested for SD
card, to eMMC the SPL will try to load the main U-Boot starting from
it's second sector from flash -- obviously resulting in garbage. This
one took me several tries to figure out and made me read most of the
SPL code for the device. The fix however is trivial (apart from the
question on how to support all different variants from one codebase,
which I'll leave to the U-Boot developers):
--- a/include/configs/clearfog.h
+++ b/include/configs/clearfog.h
@@ -143,8 +143,7 @@
#define CONFIG_SPL_LIBDISK_SUPPORT
#define CONFIG_SYS_MMC_U_BOOT_OFFS (160 << 10)
#define CONFIG_SYS_U_BOOT_OFFS CONFIG_SYS_MMC_U_BOOT_OFFS
-#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR ((CONFIG_SYS_U_BOOT_OFFS / 512)\
- + 1)
+#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR (CONFIG_SYS_U_BOOT_OFFS / 512)
#define CONFIG_SYS_U_BOOT_MAX_SIZE_SECTORS ((512 << 10) / 512) /* 512KiB */
#ifdef CONFIG_SPL_BUILD
#define CONFIG_FIXED_SDHCI_ALIGNED_BUFFER 0x00180000 /* in SDRAM */
Linux
Now we have a System booting from eMMC with mainline U-Boot (which is
a most welcome speedup compared to the UART and TFTP combination from
the beginning). Getting to fine-tune linux on the device -- we want to
install the armmp Debian kernel and have it work. As all the drivers
are build as modules for that kernel this also means initrd
support. Funnily U-Boots bootz allows booting a plain vmlinux
kernel but I couldn't get it to boot a plain initrd. Passing a uImage
initrd and a normal kernel however works pretty well. Back when I
first tried there were some modules missing and ethernet didn't work
with the PHY driver built as a module. In the meantime the PHY problem
was fixed in the Debian kernel and almost all modules already
added. Ben then only added the USB3 module on my suggestion and as a
result, unstable's armhf armmp kernel should work perfectly well on
the device (you still need to patch the device tree similar to the
patch above). Still missing is an updated flash-kernel to
automatically generate the initrd uImage which is
work in progress but got stalled until I fixed the U-Boot on eMMC
problem and everything should be fine -- maybe get debian u-boot
builds for that board.
Pro versus Base
The main difference so far between the Pro and the Base version of the
ClearFog is the switch chip which is included on the Pro. The Base
instead "just" has two gigabit ethernet ports and a SFP. Both, linux'
and U-Boot's device tree are intended for the Pro version which makes
on of the ethernet ports unusable (it tries to find the switch behind
the ethernet port which isn't there). To get both ports working (or
the one you settled on earlier) there's a second patch to the device
tree (my version might be sub-optimal but works), U-Boot -- the
linux-kernel version is a trivial adaption:
--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -89,13 +89,10 @@
internal-regs {
ethernet@30000 {
mac-address = [00 50 43 02 02 02];
+ managed = "in-band-status";
+ phy = <&phy1>;
phy-mode = "sgmii";
status = "okay";
-
- fixed-link {
- speed = <1000>;
- full-duplex;
- };
};
ethernet@34000 {
@@ -227,6 +224,10 @@
pinctrl-0 = <&mdio_pins>;
pinctrl-names = "default";
+ phy1: ethernet-phy@1 { /* Marvell 88E1512 */
+ reg = <1>;
+ };
+
phy_dedicated: ethernet-phy@0 {
/*
* Annoyingly, the marvell phy driver
@@ -386,62 +386,6 @@
tx-fault-gpio = <&expander0 13 GPIO_ACTIVE_HIGH>;
};
- dsa@0 {
- compatible = "marvell,dsa";
- dsa,ethernet = <ð1>;
- dsa,mii-bus = <&mdio>;
- pinctrl-0 = <&clearfog_dsa0_clk_pins &clearfog_dsa0_pins>;
- pinctrl-names = "default";
- #address-cells = <2>;
- #size-cells = <0>;
-
- switch@0 {
- #address-cells = <1>;
- #size-cells = <0>;
- reg = <4 0>;
-
- port@0 {
- reg = <0>;
- label = "lan1";
- };
-
- port@1 {
- reg = <1>;
- label = "lan2";
- };
-
- port@2 {
- reg = <2>;
- label = "lan3";
- };
-
- port@3 {
- reg = <3>;
- label = "lan4";
- };
-
- port@4 {
- reg = <4>;
- label = "lan5";
- };
-
- port@5 {
- reg = <5>;
- label = "cpu";
- };
-
- port@6 {
- /* 88E1512 external phy */
- reg = <6>;
- label = "lan6";
- fixed-link {
- speed = <1000>;
- full-duplex;
- };
- };
- };
- };
-
gpio-keys {
compatible = "gpio-keys";
pinctrl-0 = <&rear_button_pins>;
Conclusion
Apart from the mess with eMMC this seems to be a pretty nice
device. It's now happily running with a M.2 SSD providing enough
storage for now and still has a mSATA/mPCIe plug left for future
journeys. It seems to be drawing around 5.5 Watts with SSD and one
Ethernet connected while mostly idle and can feed around 500 Mb/s from
disk over an encrypted ethernet connection which is, I guess, not too
bad. My plans now include helping to finish flash-kernel support,
creating a nice case and probably get it deployed. I might bring
it to FOSDEM first though.
Working on it was really quite some fun (apart from the frustrating
parts finding the one-block-offset ..) and people were really
helpful. Big thanks here to Debian's arm folks, Ben Hutchings the
kernel maintainer and U-Boot upstream (especially Tom Rini and Stefan
Roese)
I'm coming as well! Really looking forward to meet the people from Debconf9 again. Also people from the Games Team and the buildd and kfreebsd folks. And ideally there will be some more people interested in (Common) Lisp as well, we'll see
