Log-based comms

Messenger
1

Note; a similar post from @adam that explores this question from a different angle is Status.app - check it out also!

Like many things in life, including glasses of water and black-and-white curvy symbols, the chat can be approached from different angles that that on the surface describe the same system, but underneath represent completely different world views, and consequently, different trade-offs, strengths and weaknesses.

Today at status, we treat the chat primarily as a messaging system. In this mental model, messages are pushed from the sender to the receiver, assuming that the receiver will be online and able to receive the message as it’s being sent. This system comes with several interesting properties that work well for live communications:

  • Conceptually matches the act of sending a message to someone
  • Low latency
  • Low overhead for sender
  • Well suited for synchronous, live communication

In the raw form of this model, messages are lost if the receiver is offline - the sender does not store them and there’s no one else around to do so either. Similarly, if a message is lost for networking reasons, it is lost forever, with no baked-in way of retrieving it. One could say that reliability and offline-capability are add-ons to the system. It is also the onus of the sender to prove that the message is useful - through proof of work or other means.

An alternate model for chat (or communications in general) is that of a distributed or replicated database. Here, every node is the holder of a database that can be replicated somehow between participants of the network - replication is the primary means of receiving a message - you append and synchronize, not send and receive.

This model comes with a different set of advantages:

  • Reliability is an direct effect of the design
  • Service degrades gracefully and naturally with limited connectivity and works well with alternate transports such as mesh networks, usb sticks or anything else that can get bits across.
  • In a way, the receiver decides on the usefulness of syncning a particular log, and the sender is free to do as they want.

In a system like this, you can further play with the topology and layout by splitting up the storage responsibilities. Traditionally, a chat history is viewed and saved as a series of messages for one or more participants - the store contains all messeges from all participants.

In a log-centric view, each participant (at an abstract level) can store their own messages only - synchronization means fetching the message of the other participant without affecting your own log and sending a message means appending a message to your own log only, and letting the recipient synchronize the new message when available - the full chat history can then be stitched together “on the fly”. Group chat? Add multiple logs together and use a graph to determine a faithful ordering.

What about storage? Well, you primarily store your own messages in a log on your own device. You can then choose to replicate them to any storage, and the mechanism for retrieving them is the same, regardless where they are stored - swarm, IPFS, USB stick. Likewise, you can choose other logs to replicate that interest you - such as the log created where your contact writes their side of the dialogue to you.

Equally interesting is the fact that there are no “special” participants required for this system to work. True to decentralization, every node is equipped to be self-sufficient in its capabilities. It works when connectivity is down, when there’s no server around.

To get a feeling for the technical implemenation of a distributed log, and how it works, check out The Log: What every software engineer should know about real-time data's unifying abstraction | LinkedIn Engineering, but as developers, we should already have a good feeling for it, because we use one almost every day - git.

To delve into the philosophical aspects and add security on top, and how to build a chat or a twitter specifically on top of this, a gold mine is the stories section of scuttlebutt.

The question to ask ourselves now is - which of these mental models best agrees with the product we’re building - a secure decentralized messenger for a device that often is offline and may potentially be used in a mesh setting?

The compendium holds links to more material on this topic as well, including several interesting messengers like Briar, Tox and SSB that in some shape of form make use of these principles.

7 Likes
2

It’s a great post! I really like this part:

Equally interesting is the fact that there are no “special” participants required for this system to work. True to decentralization, every node is equipped to be self-sufficient in its capabilities. It works when connectivity is down, when there’s no server around.

This seems like much better suited and true to the principles than our current model with special intermediaries like mail servers.

I started thinking how we can incorporate this model into our case. I took a look at IPFS and swarm and both of them lack of efficient pubsub mechanism (IPFS is a bit ahead as they got pubsub mechanism in alpha). Since we use Whisper as a primary transport, we can use it exactly for that.

A new message can be appended to the local log (there can be many logs, one per topic for example) and synchronized with other nodes at some point. Simultaneously, the message will be broadcasted within Whisper network. In this model, it’s also possible to choose whether to persist messages or not (still, someone else can persist messages if they listen to Whisper traffic).

Thanks to swarm Mutable Resource Updates and ENS, it’s possible to easily download logs of each contact. The resource IDs need to be exchanged beforehand but just once.

The tricky part is group chats, though. The idea of combining logs from different participants is good but the question is how to figure out all participants of the conversation?

This whole thing should also work without Whisper but it seems difficult to achieve low latency without some pubsub layer.

1 Like
3

indeed - nothing prevents from using helpers like this to make it work “better” or fill in gaps in the model - what matters is what you consider your core abstraction - the first principle from which you build the system. The better alignment you have in your technology, the better the outcome. With the “messaging” or push line of thought, you have to plug offline availability and spam - with “log” or pull, you have latency to deal with (etc)…

Thinking about expectations, it’s easy for me to understand that I miss a message when my phone is not connected, so even if we use whisper and the “push” notification gets lost, no user satisfaction is gone because no user expectation was broken.

Likewise, whisper could be one of the supported ways to do key exchange or group membership negotiation or a number of other “auxiliary” functions needed for a complete solution.

There’s this really neat parallel to the database world, where a key-value can be represented either as “state” or “events” - in the state-based model, it’s easy to find what the value is for any given key, but hard to know what a particular update contains - if you know you only care about the last two updates, you can still build a little diff cache as a patch, while retaining all the good properties of having the full state available. If instead you choose events, obviously the diff is easy but the current state is hard - again, you can build a cache for the most used keys to help some particular situation, but it remains diff-first.

What I worry about a little with this model, and what perhaps remains unsolved, is the darkness / secure messaging aspect - how do you avoid leaking meta-data?

indeed - we’d first have to be clear on what we mean by group chat, and what the desired feature set is - who manages membership? are there admins and users? moderators? is participation open? should new joiners be able to access history?

I had some interesting conversations with the swarm team on these topics during their conference - one idea is to use something like nucypher reencryption to grant access to a group chat key - but there remain open questions about how to do forward secrecy (assuming that is at all interesting in a group setting) and initial key exchange. They also mentioned access control lists and native swarm encryption as a possible route to go, but I haven’t dug into that solution.

Generally, for group chat, https://eprint.iacr.org/2017/666.pdf and https://git.matrix.org/git/olm/about/docs/megolm.rst are some interesting resources I’ve come across on the subject.

1 Like
4

I though about it a bit. Darkness might be actually a problem unless a mobile node is a part of swarm network as well. I am not familiar with swarm network setup but I imagine it’s difficult to figure out from where a file originated.

I don’t see any other metadata being leaked compared to recording Whisper envelopes, though. Actually, this flow can allow more secure communication if the log address is unknown and shared via Whisper initially. Later, Whisper won’t be even needed to communicate if increased latency is not an issue.

1 Like
5

I’m a big fan of this model, and logs / event sourcing in general, though I agree it has its challenges.

What do people suggest in terms of more concrete next steps here? For example: exploratory work needed, more concentrated effort to answer some questions around guarantees/possibilities, etc.

1 Like
6

I like the decentralized - “no special participants required” - nature of this approach, that’s definitely better long-term solution than current mailservers approach. However, since I first heard about this model, I had troubles internalizing it and accepting as a better model, and I think I understand now why.

To me, software systems should mirror real-world systems as much as possible. The better the mapping between the real world and our mental models (and, hence, the software we write), the better experience in using it, refactoring it to the new changes and adapting for new real-world use-cases. Let me elaborate on that.

The traditional communication model involves ephemeral information transfer – the same way how we speak to each other, transferring information using our voices via sound over the air. When we talk to each other, we send information from “sender” to “recepient(s)”. There is no explicit storage. One might argue, that we “store” logs of our communication in our brains, but that’s a separate topic.

The point I’m making here, that traditional communication model is based on every-day experience every human being possesses, and it works well for software messaging applications as well. It fits expectations, it’s natural, it’s easy to reason about and it’s the most fundamental layer of every kind of communication - sending information from sender to receiver.

The storage for our communication is the second level, it’s an additional, even optional feature to ephemeral communication. Storing copies of sent letters and archiving received letters is an example of that, but we can still see easily that those still are two separate actions – sending a message and storing/duplicating this message. You can read the letter and destroy it. Communication and storing communication are still fundamentally different in this model. Communication is first, storage is second.

Now, the log-based communication model turns this model over saying that storing is first, and communication is second. In fact, it makes communication a byproduct of storage replication mechanism. The benefits of this approach are well described above, but I want to address the fact that it doesn’t align well with how communication is happening in “real-world” and that imposes some perils:

  1. In the communication-first model, it’s possible to have “communication works, storage doesn’t”, and virtually impossible to have “communication doesn’t work, storage works” case. In storage-first it’s the opposite – we can easily imagine a case where “storage works, communication does not”.
  2. Storage-first approach shifts design focus on storage, forcing us to think about storage in the first place. The design we chose govern our technical decisions and we can easily lose an importance of making communication experience great, focusing more on the storage.
  3. More often than not, storage is a harder problem than communication (I believe that’s the main reason why this discussion here exists). It means, for some use cases - like “Revolution mode” - we might be willing to sacrifice the storage functionality, while still having communication. Again, I want to stress, it comes from the better mapping of this traditional model to the real world. In the real world, I want to communicate important information to my fellow(s) in a first place, even if there is no storage history option. “Storage-first” approach doesn’t allow that without building a special solution for that.
  4. There are many possible real-world scenarios where “storage-first” model doesn’t fit at all. For example, it’s a valid case where chat history might be different for sender and receiver. Sender and/or receiver might be willing to remove/hide some messages from the chat history – from sexting to sending private credit card information – anything you don’t want to stay in your history. “Storage-first” approach can probably handle it as a special case, but it’s not in the design.

So the key takeaway is that the design we build into our systems should reflect the real-world patterns and interactions. I’m not implying that we should copy offline communication designs – not at all. Digital communication is different in many ways, and darkness is a good example here where we have almost no good analogy in the offline world. But it also similar in many ways and I feel like “log-based communication” is less aligned with real-world communication, and poses some design challenges that we’ll need to address and prove that this misalignment really brings more benefits than challenges and complexities.

That said, I’d really love to see more elaboration of this approach and maybe seeking some middle-ground between the two models because the promises of the log-based communication model are appealing. I definitely welcome further research in this direction, but we should keep in mind that traditional approach also seems to have a decent decentralized solution to storage problem – it’s just waiting to be implemented.

3 Likes
7

It seems like this discussion is at the intersection of two equally important ideals in a p2p web.

  1. Privacy in communication
  2. Ownership and control of one’s data

@divan brings up some great points that show how these two things can rub against each other and infringe on one another.

I’m all for the idea that privacy is the default, and it can be eased at will by the user. For example, you can still have log-based comms and just don’t keep a log of your own stuff, or have them expire after a while. Everyone else’s stuff is plausibly deniable if they also don’t keep their own stuff.

I really like the log-based comms idea because it’s offline first, which to me is one of the important principles of a resilient p2p web.

One thing worth pointing out is that “log-based comms” vs “ephemeral messaging + mailserver” is barely distinguishable from a user standpoint.

As a user I care about getting my message to my friend or group. If we’re both online I want them to have it now. If we’re not, I want them to get it later.

The absolute only difference from a user’s point of view between “communication first” and “storage first” is that in the communication first model, the user will get an error in the UI if the message wasn’t delivered. Whereas the storage first model will happily continue pretending it’s sending messages by appending them to the user’s visible chat list.

No matter which model you use underneath, the UI could still do the same middle-ground: warn the user they’re offline and their messages won’t be delivered, and then let them send them anyway.

So in that way, from a UI perspective, both models appear as “storage first” to the user.

8

Communicate directly or store-forward? We need a server always up to do store and forward. For ethereum, it is not a problem. But there is no way to cummunicate directly right now since there is no presnce server. Instant Messaging or Email? Ethereum is very easy to do email type of cummunication but hard to do real time peer to peer communication.