Wherefore Art Thou Mailserver - Treatise on Waku Terminology

samuel · May 12, 2020, 2:52pm

What is in a name?

What’s in a name? That which we call a rose by any other name would smell as sweet.

Though, calling a rose “swamp stench” would give the uninitiated the forgivable assumption that sweetness is far from its prime quality.

Waku Nomenclature

Waku has inherited a lot of things from Whisper and despite effort to bring clarity to the Waku specification there are still a number of vestigial quirks in its makeup. Listed here is the non-exhaustive findings of several weeks wading in the Waku weeds:

The deceptive Mailserver
The word broth of Message, Envelope, Batch, Messages
Synonym salad of Response, Confirmation, Acknowledge

I may be wrong, and misunderstand everything, so here’s to my inevitable humiliation and embarrassment.

Mailserver is a swamp stench

Waku doesn’t have a concept of “Mail”, neither does it have a concept of “Server”. Waku does have a concept of “Message” and a concept of “Node”, but “MessageNode” doesn’t describe what a Mailserver actually does, neither does “MailServer” BTW.

The current Waku specs’ abstract states:

Waku is a fork of the original Whisper protocol that enables better usability … through … (b) historic messages (with a mailserver)

The key to what a Mailserver does is explained in only 2 words, “historic messages”.

The term Mailserver is antiquated (at least in the Waku context) and laced with decades of exclusive association with email, and has all the connotations of what an email server is and does.

Proposal One

Rename “Mailserver” to a better more Wakuy name

Proposal Two

The new name for Mailserver should be History Node or Persistence Node

Confusing Concept Curry of Messages

What is a Waku Message?

According to the specs:

Use of RLPx transport protocol

All Waku messages are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See informal RLP spec and the Ethereum Yellow Paper, appendix B for more details on RLP.

Packet Codes

The message codes reserved for Waku protocol: 0 - 127.

Messages with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.

So far so good, “Messages” are the data packets that Waku uses to communicate various things between peers.

messages

This packet is used for sending the standard Waku envelopes.

Oh, so a data package message with the code of 1 represents Messages, and the definition of these Messages is Waku envelopes.

Ok. So a Message is a data packet that may or may not be of the packet code type 1 which represents Messages, which contains Envelopes. This is already a little confusing as the Waku term Messages does not represent multiple Message data packets, instead represents Envelopes.

What’s an Envelope?

Definitions

Envelope Messages sent and received by Waku nodes.

Oh, ok. So an Envelope is a Message ? But, doesn’t Message contain Messages that list Envelopes? Is an Envelope an infinite recursive loop of data packets?

I’m obviously being facetious here, I do know what an Envelope is, but I didn’t learn it from the specs, I learned it from the status-go code. The specs don’t formally define what an Envelope is, or what an Envelope is used for and its properties, the specs assumes that the reader knows this.

Proposal Three

Formally define and/or rename and make strict distinctions between the following concepts:

The name for raw data packets sent between Waku peers, referred to as Message
Packet code type 1 currently defined as Messages
The name for the communication the user sends, also referred to as a “message”.

Maybe use some of the old Whisper lingo Whisper Overview · ethereum/go-ethereum Wiki · GitHub

Proposal Four

Specify the purpose and usage of the Envelope. Perhaps copypasta of or reference to Whisper Overview · ethereum/go-ethereum Wiki · GitHub

Acknowledging the Confirmation of a Response

I split this apart from the Message, Messages, Envelope section because it is a distinct issue, but there is some overlap.

Message Confirmations update

Message confirmations tell a node that a message originating from it has been received by its peers, allowing a node to know whether a message has or has not been received.

A node MAY send a message confirmation for any batch of messages received with a packet Messages Code.

A message confirmation is sent using Batch Acknowledge packet or Message Response packet. The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data.

So, if a data packet has code 1, a node may send a Message Confirmation in response. The node can use data packet code 11 for Batch Acknowledge or code 12 for Message Response.

In which circumstance would a node send a Batch Acknowledge confirmation?
In which circumstance would a node send a Message Response confirmation?
Why does Message Response have a version number?
Is Message Response a type of Message Confirmation or visa versa?

Proposal Five

Define in which circumstance a node would send a Batch Acknowledge confirmation
Define in which circumstance a node would send a Message Response confirmation

Proposal Six

Determine if Message Response should have a version number.

Could Message Response version defer to the protocol version? Example waku/1
In which circumstance would Message Response make a version change that wouldn’t affect the protocol version?

decanus · May 12, 2020, 3:25pm

The renaming of Mailserver has been a subject for a long time. There are multiple threads on it. I disagree with changing it for various reasons but don’t really care to go into this discussion again too much. Mailservers are an ugly construct and have an ugly name, they should be removed in general rather than working on renaming.

cammellos · May 12, 2020, 3:26pm

Nice post, thanks!

I agree with most points (if not all, I don’t have a strong opinion on any, I am ok with mailservers as a name, but changing it is totally fine by me)

The only observation I would like to make is about point 6:

Determine if Message Response should have a version number.

I think if we can get away without having versioning within a waku version, that’s what we should strive for and defer to protocol as much as we can.
Having to deal with multiple versioning will increase complexity and leads to some subtle issues:

For example:

github.com

status-im/status-go/blob/a21f8a39bc4b786999011f88f519bf518fbfa299/waku/v1/peer.go#L276


      
          
          	return err
          }
          
          func (p *Peer) handleMessageResponseCode(packet p2p.Msg) error {
          	var resp MultiVersionResponse
          	if err := packet.Decode(&resp); err != nil {
          		common.EnvelopesRejectedCounter.WithLabelValues("failed_read").Inc()
          		return fmt.Errorf("invalid response message: %v", err)
          	}
          	if resp.Version != 1 {
          		p.logger.Info("received unsupported version of MultiVersionResponse for messageResponseCode packet", zap.Uint("version", resp.Version))
          		return nil
          	}
          
          	response, err := resp.DecodeResponse1()
          	if err != nil {
          		common.EnvelopesRejectedCounter.WithLabelValues("invalid_data").Inc()
          		return fmt.Errorf("failed to decode response message: %v", err)
          	}

means that we can’t really change version without deferring to waku versioning.

In this case we can’t change the version number without breaking, even if the format is identical, we should remove these checks as they make maintaining compatibility difficult, in case things change.
Not sure why we enforce a version number, but the same was done with the handshake previously.

In which circumstance would Message Response make a version change that wouldn’t affect the protocol version?

That’s a good question, I don’t see a reason for this, as we always have the option to just bump waku/x and have separate handling of it (we can always negotiate at a higher layer).

The only argument for having versioning at a lower level is mostly implementation dependent, as far as I can see, the argument would be:

We decided to have separate codebase for each waku version. One of the drawbacks is code duplication as we discussed.
There’s a part of the protocol that changes often (say Message Response is one of them)
Every change a lot of code duplication would be introduced (waku/2, waku/3, waku/4), and we would not decommission older versions fast enough.

That might not be acceptable I guess, but the protocol at this layer is fairly static, so I would not be worried too much about this (we can always change our minds at a later stage).

Another argument is that if the complexity of Message Response increases or it becomes something that makes sense on its own (can be used independently of waku/x), at some point it would make sense to version it separately.

I don’t find neither of these arguments very strong at this point, I would add versioning only if there’s a use case for it (MessageResponse has only version 1, I am not sure version 0 ever existed, so even in this case seems like is not really needed ) and assume Version 0 for unversioned.

samuel · May 12, 2020, 3:44pm

Thanks for your reply, could you point to these threads please?

What would you replace message persistence nodes with? User client node based persistence? IPFS archiving?

decanus · May 15, 2020, 10:27am

deme · May 18, 2020, 9:55am

Thanks for the post, some remarks regarding your proposals:

Mail server: As mentioned above, this has been discussed before. I don’t really care too much about its current name. To me it is more important to put effort in replacing the construct itself.
Message, Envelope, etc.: Yeah, usage of these is not consistent. This inconsistency is inherited from the Whisper EIP-627, and I think even from the RLPx specification, and it didn’t properly get corrected in the text (possibly with some copying also from the Status specs) for Waku. In EIP-627 & the RLPx specification Packet & Message are both used to (mostly) express the same. Envelope is quite clear I believe, but sometimes message is used for that too. And then there is the Messages Packet (or the Messages Message).
However, I don’t agree that envelope is nowhere specified in Waku, as it is in the ABNF spec similar as it is in EIP-627 (But perhaps it should be accompanied with some more explanatory text if it is not clear enough). This is actually one thing we chose specifically for the Waku spec, to use ABNF from now onwards, and for me it makes the spec more clear and consistent. In fact, in that ABNF spec we clearly differentiate names (packet, packet-code, etc.), so you should really base yourself on that. The accompanying text is less consistent, and this should be fixed (your proposal 3, I did an attempt here Be consistent in usage of terms packet, message and envelope by kdeme · Pull Request #123 · vacp2p/rfc · GitHub, I hope this clarifies things).
The old Whisper lingo for the message term that you link is a part that should not even be in the Waku spec imo. And this is also why it is moved to a separate file.
Message confirmations: This one bothers me more. When I first saw that packet hitting a Nimbus node, and I noticed it was nowhere documented I asked what it was used for. It got documented in the meanwhile, but it is still not fully clear to me today. The way I understand it now is that it is used for confirmations of messages send to mail servers (only used for those afaik, which makes sense, as it is rather useless for other nodes, and also an attack vector), and that this has to do with the UI response when sending a message. But I’d like to know if that is really still the case, and if we need this packet at all. And if so, it should be explained better (your proposal 5).

cammellos · May 18, 2020, 10:04am

But I’d like to know if that is really still the case, and if we need this packet at all.

I can answer this question, it is still used currently, messages will be shown as “Sent” only once a confirmation is received from a mailserver.

samuel · May 18, 2020, 2:02pm

Thank you everyone for taking the time to read and address my topic.

Thank you for this. I’ve re-read that section in the context of your comment, I understand much better.

I’ll open a PR making an explicit reference that Envelope is specified by ABNF. Resolving Proposal 4.

Thank you very much for making that PR. If I had the confidence of understanding the specs thoroughly I would have made these changes myself.

@cammellos

Do you know in what circumstances Batch Acknowledge vs Message Response would be used?

cammellos · May 18, 2020, 2:12pm

Do you know in what circumstances Batch Acknowledge vs Message Response would be used?

Not sure, looking at the code, it seems we always send both, that seems like some backward compatible code:

github.com

status-im/status-go/blob/cc9c352337e401ecab04aca18477ba6ac5606f33/waku/v1/peer.go#L382


      
          	}
          
          	var payload []byte
          	if err := packet.Decode(&payload); err != nil {
          		return fmt.Errorf("invalid p2p request complete message: %v", err)
          	}
          	return p.host.OnP2PRequestCompleted(payload, p)
          }
          
          // sendConfirmation sends messageResponseCode and batchAcknowledgedCode messages.
          func (p *Peer) sendConfirmation(data []byte, envelopeErrors []common.EnvelopeError) (err error) {
          	batchHash := crypto.Keccak256Hash(data)
          	err = p2p.Send(p.rw, messageResponseCode, NewMessagesResponse(batchHash, envelopeErrors))
          	if err != nil {
          		return
          	}
          	err = p2p.Send(p.rw, batchAcknowledgedCode, batchHash) // DEPRECATED
          	return
          }
          
          // handshake sends the protocol initiation status message to the remote peer and

seems like already in v1 we could pick one and stick with it, removing the old one

samuel · May 18, 2020, 3:24pm

Yeah, I saw that code and to me it looked like Message Response is preferred but no real distinction is made.

Perhaps we can add “Remove Batch Acknowledge” as another proposal for Waku V2. I’d imagine dropping Batch Ack would be another breaking change.