Intro
Felt like a good time to take a breather from Eth2 for a day, so I started looking at some of the Whisper code that recently was contributed it to Nimbus - it’s a first cut so I thought it would be fun to take it for a spin - and it works surprisingly well!
Another thing I’d been meaning to do was to try out QT - never really gotten around to write an app in it, and given how they added QtQuick and claimed it was good for rapid prototyping, it seemed like a good match for the day.
Thus, without further ado, I’d like to present to you the outcome of the experiment: Stratus - one more desktop chat application (well, a viewer at least!) for the Status Network 
- obviously borne out of jealousy that @iurimatias & co have their own Status protocol implementation…
Technology
The stack is thankfully fairly simple: Nim for networking and logic with a C++-based UI framework that is customized through QML and JavaScript.
P2P layer
At the bottom we have nim-eth-p2p which handles the Ethereum devp2p protocol - serialization, encryption, discovery, multiplexing - that sort of stuff.
Incidentally, this is the part that many want to replace with libp2p eventually - though for our first beacon node demo in Nimbus, we’ll likely keep using it simply because the code is written already.
It’s generally a good thing that it’s getting replaced - for us it’s causing some trouble, such as the non-standard port that prevents our chat from being used on many public wifi’s, or the encryption that’s not really doing its job.
Whisper layer
nim-eth-p2p is also where the core Whisper implementation lives in Nimbus - raw Whisper packets are decoded and matched against available decryption keys, bloom filters are formed to shape bandwidth consumption and the database of live messages buzzing around in the system is kept here.
For example when you send a whisper message, the nim-eth-p2p code takes a structure like the following:
Envelope* = object
## What goes on the wire in the whisper protocol - a payload and some
## book-keeping
## Don't touch field order, there's lots of macro magic that depends on it
expiry*: uint32 ## Unix timestamp when message expires
ttl*: uint32 ## Time-to-live, seconds - message was created at (expiry - ttl)
topic*: Topic
data*: Bytes ## Payload, as given by user
nonce*: uint64 ## Nonce used for proof-of-work calculation
and turns it into RLP-encoded bytes for further processing.
Status layer
Ok - we arrive at the first half of the thing that was hacked together this time, with emphasis on hacked. Once a Whisper message arrives at the application layer, it has already been decrypted, the key of the sender is available and now it’s time to turn raw bytes into chat bubbles.
Status encodes messages using json in a peculiar shape, created by a particular library - I can’t really think of anything positive to write about it, so I’ll just leave a link.
try:
# Sample message:
# ["~#c4",["dcasdc","text/plain","~:public-group-user-message",
# 154604971756901,1546049717568,[
# "^ ","~:chat-id","nimbus-test","~:text","dcasdc"]]]
let
src =
if msg.decoded.src.isSome(): $msg.decoded.src.get()
else: ""
payload = cast[string](msg.decoded.payload)
data = parseJson(cast[string](msg.decoded.payload))
channel = data.elems[1].elems[5].elems[2].str
time = $fromUnix(data.elems[1].elems[4].num div 1000)
message = data.elems[1].elems[0].str
info "adding", full=(cast[string](msg.decoded.payload))
rootItem.add(channel, src[0..<8] & "..." & src[^8..^1], message, time)
except:
notice "no luck parsing", message=getCurrentExceptionMsg()
The message arrives and you have a nested list of heterogeneous items in it. You’ll immediately notice the shoddy hackathon-style code there - all based on the honorable practice of reverse engineering instead of reading documentation. I’m not sure which of those two time-looking numbers are actually the message time - close enough for now though 
. Status once had a funny bug where messages would be sorted by this field - meaning that anyone could craft a message that would get stuck at the beginning of the list and never scroll away. This code is no better - breaks a cardinal rule of internet applications: you can’t trust what the remote end is sending you… That cast looks fishy and going by index just like that… well… and so it goes on.
UI Layer
The second part of the hack, the UI, is done using the QtQuick framework. Really nice! The idea is that you write a mostly declarative UI description, annotate it with some JavaScript to handle visuals like animation and shuffling of data from widget to model, and then keep the application logic apart.
My initial plan for the day was to write the data model in C++, that being the native language for QT, and then take the Nim-C interop features out for an exercise to bring in the rest of Nimbus. Luck would have it however that someone’s written a QML wrapper for Nim, and I could stay in Nim-land throughout. Unusually, it even came with documentation and examples!
Let’s take a look at a snippet from the QML:
ListView {
id: messagesView
model: root.messageList
// model: sampleMessages
Layout.fillWidth: true
Layout.fillHeight: true
delegate: RowLayout {
width: messagesView.width
Text { text: "👾"; font.pointSize: 28 }
ColumnLayout {
Layout.fillWidth: true
RowLayout {
Text { text: channel; font.bold: true }
Text { text: " ("; }
Text { text: source; }
Text { text: ")"; }
Item { Layout.fillWidth: true }
Text { text: time; }
}
Text { Layout.fillWidth: true; text: message; wrapMode: Text.WordWrap }
}
}
}
Above we can see the bit that creates the layout for the rows of the message view, including the cute alien which in discuss looks more like a space invader.
First with have the List itself, and it’s annotated with parameters informing the resizable layout engine. You can also see the model which is how content is pulled in. Finally, the delegate is a template for what each row should look like.
To get the message content to show up, in Nim we fill out an object that has the right fields, and stuff it in the messageList:
proc add*(
self: MessageList, channel: string, source: string, message: string,
time: string) {.slot.} =
let sm = newStatusMessage(channel, source, message, time)
self.beginInsertRows(newQModelIndex(), self.messages.len, self.messages.len)
self.messages.add(sm)
self.endInsertRows()
There’s a bit of book-keeping going on there as well, to let the UI know it has to update the visuals for the new data.
In general, the QML approach has two killer features:
- you can churn something that looks decent in no time
- designers can work on the UX independently - the layout can done in a visual designer tool and is separate from the code
… and then the third half
Of course, like any software development story there’s a snag somewhere around the time when you actually have to build the thing, and this time it was no different. As commonly known, build system complexity grows exponentially with the number of frameworks and languages you throw into the mix…
With the app mostly done, my first thought was to just statically link Qt and not have to bother with versions. Turns out that Qt is distributed as a collection of dynamic libraries and comes with a custom build system on top. Plan B suddenly became much more appealing: create an AppImage - should be simple, just compile and zip it up that’s it. Haha.
I’m generally not a fan of Docker - it’s one of those smells that says that something else has become too complex to manage and now the least bad solution is to throw another piece of complexity at it - but whatever, it’s hack day (more night now) and we want results. Fortunately, I’m not the only one that has had this need and conveniently, there exists a docker image that has a dated Ubuntu with a fresh Qt on it (that’s the right mix for AppImages!). As a cherry on top, it comes with instructions how to use it to deploy a QT app on Android.
It’s ugly, and it works. Long story short, the repo now holds the magic bits to create a mostly reproducible AppImage build (some Nim parts still would need locking down - that’s a work in progress in the community).
That’s it
Wanna give it a try? Here’s what works so far:
- Connect to the status whisper nodes and listen for messages
- Subscribe to channels and look at messages
- Pure
nimbus! nogethinvolved
! - Only tested on Linux - in theory it should work elsewhere as well with the right magic incantations - even Android and iOS - though you know what they say about theory and practice…
From here, it should be fairly easy to add any of the following:
- Send messages
- View other whisper traffic - think wireshark, but for whisper - can’t do much with it but interesting for exploring the protocol
- work on UI - don’t even have to know how to code!
Yeah, the code is untested and all that, and I know there are some nasty crashes and races etc
You can get the AppImage from Swarm (for that warm, fuzzy, distributed feeling) - might need reseeding occasionally though, just let me know.
Download: Stratus-20181229-0.0.0.0.0.0.1-x86_64.AppImage
Code: https://github.com/status-im/nim-eth-p2p/tree/stratus/examples/stratus (note the branch)
I’ll be happy to walk you through it as well, should you want to get your hands dirty - it’s turned out quite educational
) , by basically making accretion easy (append an element to an array), and breaking changes hard, although possible (you are not supposed to modify any field, they ought to be considered immutable, schema-wise, append only), if a breaking change is needed a new name (i.e. tag) needs to be created