Hello Staples - Swarm PSS/Feeds PoC for #buidlweek

Archive
,
1

Hello Staples

Source: GitHub - status-im/staples: Replicated chunks made out of robust zinc-stained steel in an ordered log..

image

Replicated chunks made out of robust zinc-stained steel in an ordered log.

For messaging, we need to store and pass things around between people. Swarm is a place to store and spread things. PSS is a way to do messaging on top of Swarm. Sometimes people are away, and when they come back they need a way to find out what they missed. Feeds is a place we can look to find thing that have changed. Staples is a proof of concept for messaging using Swarm, PSS and Feeds.

What it looks like

image
image2560Ă—1440 496 KB

You can see a demo of it here.

Instructions


# Install Swarm locally:

# https://swarm-guide.readthedocs.io/en/latest/installation.html

# In three separate terminals, run the following:

./scripts/run-alice

./scripts/run-bob

./scripts/run-charlie

# Send mesages from Alice and notice how it appears in Bob's window.

# If Bob disconnects and then reconnects, it fetches messages it missed since last time.

Rationale

Context: Status currently uses Whisper as a messaging protocol. For offline inboxing, and it uses Whisper mailservers in a cluster for offline inboxing. To get the latest messages, it currently assumes a given mailserver has received all latest messages and queries it upon startup. To do message ordering, it uses Lamport timestamps with real-time clock for hints., but largely relies on Whisper mailservers being highly available to provide this consistency.

  1. PSS is in some ways the spiritual successor to Whisper, and it has a very similar API. It provides better scalability due to its superior routing strategy, while maintaining the ability to be privacy-preserving. The proof of work used in Whisper is also a poor fit for heterogenerous devices. While Swap incentives, the DoS protection mechanism in PSS, isn’t implemented yet, it is arguably a more sound design. Whisper is also not actively developed, and several developers have moved on to working on Swarm/PSS. For a more detailed comparison, see here.

  2. Swarm is a distributed storage platform and content distribution network, which deals with things such as replication and fault tolerance in a rigorous way. This is unlike Whisper mailservers, which have high uptime requirements, and fails to enable reliable offline inboxing during events such as Chaos Unicorn Day. In Swarm content is spread out and replication across the network, it doesn’t require high uptime for any individual node. Research and prototyping of things such as SWAP accounting system, light nodes, erasure coding, proof of custody and storage insurance is also at an avanced stage. To read more about Swarm and how it works, see the documentation for Swarm POC3.

  3. Feeds provide a way to get mutable state in an immutable world. Since Feeds use Swarm, fault tolerance and availability is baked in. Conceptually, it is similar to ENS or DNS. For more on Feeds, see here.

  4. Message dependencies. By including hashes of previous message dependencies, we can build up a Distributed Acylic Graph (DAG) of messages. Messages that haven’t had its dependencies met are not delivered to the upper layer. This ensures high availaility while maintaining casual consistency (topologically ordered conversations). For more on rationale of this, see this post on Discuss and this data sync research thread.

How it works

  • Alice, Bob and Charlie are three local Swarm nodes. It uses the Go API (RPC and HTTP client).

  • A message consists of some text and parent message ids.

  • Alice sends messages to Bob via PSS, including parent messages hashes (if they exist).

  • When Alice sends a message, it also updates its feed with topic bob (and uploads chunk to Swarm).

  • When Bob comes online, it first checks Alice’s feed under topic bob.

  • Alice does not have to be online for Bob to receive messages from her.

  • If Bob sees a parent message it hasn’t seen before, it first fetches that chunk on Swarm.

  • Bob only sees the message once all messages dependencies have been delivered.

  • When Bob has synced up to the present state, it also receives live messages via PSS.

  • Charlie is a helper node to deal with some Kademlia/local network connectivity issues.

Shortcomings & Enhancements

Things that can be improved:

  1. Only supports Alice talking to Bob right now.

  • => Should be fairly straightforward to generalize this for 1:1 chat at least

  • Mapping of concepts to group chat and public chat not obvious.

  • => Different structure with a local log that requires different patterns, but it’s doable.

  1. Only CLI interface.

  • Integrate as API console-client, or make UI, or add to Status app.

  • Bad code with hardcoded endpoints, poor structure, exposed private keys, etc.

  • => Refactor and use best practices; integrate into existing code bases.

  1. Due to issue with stale/non-existant Feed reads, cheats a bit using local helper node.

  • likely cause: local network connectivity issues or poor Kademlia connectivity

  • => Kademlia connectivty can be checked by using health checks

  • => local network connectivity just requires a bit more debugging/log checking

  1. Lack of Swarm light client mode for mobile.

  • This is under active development

  • => Just try running it and see what happens, desktop should be fine, can fork with hacks

  1. Swarm is still under active development.

  • This means things like incentives, fault tolerance, connectivity etc are still issues

  • => Don’t make strong assumptions but use it optimistically

  • => Try it and file issues / help fix them

  1. Not integrated with current Status code bases (status-go/console-client) and protocol.

  2. Additionally, some small differences when it comes to what keys/identities etc are.

  • => Do it, fairly straightforward code-wise for basic infrastructure

  • => Write up upgrade path for Whisper->PSS and how to use e.g. feeds in parallel

  • => For protocol, write proposal to specs repo on what upgrade would look like

  • => Consider parallel support via feature flag for gradual move

  • => More research/thinking on properties and trade-offs

  1. Probably more.

Takeaways

  • It was easier than expected to get a basic PoC up and running, and swarm is really cool and underutilized
  • That said, there are a bunch of rough edges (API docs, connectivity) and missing features
  • We should spend more time seriously trying it out (both in terms of building light client, and using pss/feeds for app and in protocol work).
4 Likes
2

Thanks for the post, interesting experiment.
I fully agree that having swarm for offline storage is definitely superior to mailservers.

but largely relies on Whisper mailservers being highly available to provide this consistency.

if you are referring to casual ordering ( not consistency), which is what lamport timestamps are used for, this statement is not accurate, as it does not rely at all on mailservers, but on previous messages timestamps (which might or might not be fetched from a mailserver), with timestamp hinting for when a message might not be available. Briar I believe use the same method for ordering, and turning off mailserver will give you causally ( and totally) ordered messages, as per algorithm. It does not solve causal consistency off course as that’s not its purpose and you d have to use vector clocks to ensure that.

This ensures high availaility while maintaining casual consistency (topologically ordered conversations)

I believe we discussed this before, reliance on parent id for ordering or dependencies comes with pretty drastic implications (i.e you can’t post at point F until A - E is fetched), if you are going to enforce a parent id on each message, if you don’t the benefits will be limited and you still need something to ensure causal ordering, although it will be useful for fetching previous messages.

By enforcing this, you are trading off in availability ( as you can’t write, post a message, with partial history), so I think before calling it high availability it requires more accurate thinking.

Currently swarm makes no guarantees on storage (poc4), and it will likely need to be incentivised, so making the system resilient on message loss is paramount, by the way do you know what it will look like? any rumors or sneak previews :slight_smile: ?

Overall if we solve group / public chats it would be very interesting to plug it in, with or without pss, as it might just be an independent piece.

thanks for the post!

3

Thanks for the reply!

if you are referring to casual ordering ( not consistency), which is what lamport timestamps are used for, this statement is not accurate, as it does not rely at all on mailservers, but on previous messages timestamps (which might or might not be fetched from a mailserver), with timestamp hinting for when a message might not be available. Briar I believe use the same method for ordering, and turning off mailserver will give you causally ( and totally) ordered messages, as per algorithm. It does not solve causal consistency off course as that’s not its purpose and you d have to use vector clocks to ensure that.

This was sloppily written. I meant casual consistency here. If a mailsever is offline even for a short period of time (sans our duct taped cluster syncing), you’ll lose messages. It is true though that you know that you missed some messages, but you don’t have any direct way of access them, as far as I’m aware.

I believe we discussed this before, reliance on parent id for ordering or dependencies comes with pretty drastic implications (i.e you can’t post at point F until A - E is fetched), if you are going to enforce a parent id on each message, if you don’t the benefits will be limited and you still need something to ensure causal ordering, although it will be useful for fetching previous messages.

You can still post at point F, as that’s just appending to your local log. You just won’t see messages E until you fetched its dependencies A-D. Even that can be negotiable, since you for long message histories you probably want to do some kind of paging, or possibly even aggregation/snapshotting. How this should best be reflected in the UI is still not clear.

Currently swarm makes no guarantees on storage (poc4), and it will likely need to be incentivised, so making the system resilient on message loss is paramount, by the way do you know what it will look like? any rumors or sneak previews :slight_smile: ?

Agree, there are also some open questions regarding replication factor etc. No sneak peeks :stuck_out_tongue: My thinking is that even so it can be leveraged optimistically, without relying on it. I.e. actual data sync would be end to end, but Swarm would smooth things over and improve the situation in for mostly-offline devices.

Overall if we solve group / public chats it would be very interesting to plug it in, with or without pss, as it might just be an independent piece.

Agree! f([users], topic) => feed, solve for f!

2 Likes
4

Finally! Thanks for this. It still amazes me that status devs still didn’t integrate / contribute back to swarm so far, as it is the only sensible thing to do (since status ICO), even before thinking about a mobile app. Well, better late than never → keep up the good work :+1:t2:
To be honest I find this combination of mailserver+whisper such a dumb idea that only served to waste human capital, and it goes against even what status stands for, which afaik is a decentralised messaging system (if we were to use servers, we can just use FB in the first place).

1 Like