Chaos Unicorn Day, April 1 2019 (aka kill our cluster)

Status Mobile
1

One of the goals of Status is to be decentralized, censorship-resistant and continue to exists without our current contributors or centralized services, like our cluster and Infura. This is a goal that is widely shared in the Ethereum community, at least in theory. In practice however, too many projects, us included, rely on centralized services and take shortcuts to get a working app. We want to change that.

The following proposal was prompted by previous discussions and this twitter thread: https://twitter.com/jcksie/status/1089584345126334464

Proposal

Have a dedicated day, where we disable or turn off centralized services and see how we deal with it. Encourage other projects in the Ethereum space to do the same.

This includes things like:

  • Our cluster (including bootnodes, Whisper nodes, mailserver)
  • Infura and other web2 centralized gateways (Etherscan, etc)

Individual teams can choose to what extent they want to prepare vs get caught red-handed.

Next steps

  1. Discuss and get buy-in from core stakeholders
  2. Set a date and scope
  3. Setup non Status branded static site and encourage other teams to join

FAQ

Won’t this just break everything? What’s the point?

Very likely, yes. We’ll see exactly how though, as well as how possible workarounds work in practice. For example: setting up custom bootnodes, mailservers and LES nodes, then see how coordination works out of-band. E.g. sharing QR codes in Twitter, etc.

Individual teams can choose how much they prepare, e.g. dealing with bad paths. This depends on their priorities and how much shame they can handle. At an extreme end it might just be a single hack day, or it could be part of existing prios but reinforced.

Won’t this turn off our core user base?

We are still not in the app store, and our core user base is likely going to understand and be excited by the prospect, provided we communicate properly.

This is a perfect opportunity while we are still small and in beta. We don’t want to be “too big (and fragile) to fail”. This allows us to set the terms, as opposed to the terms being set for us in the future (Infura paywall, credit card for cluster blocked).

When?

TBD, but proposal is end of February or end of March. Some teams might want to do their homework beforehand to ensure it isn’t a complete failure (although that’d be valuable on its own), as well as ensure other projects in Ethereum can participate.

5 Likes
2

I like the idea, @igor suggestion to have special builds with our cluster disabled is probably also worth taking into consideration, in order to avoid disruption to other users.

1 Like
3

Love the idea to learn from workarounds.

Not sure about the Feb/March timing. As the inevitable outcomes will probably distract from delivering on the Whitepaper promises.

2 Likes
4

I don’t expect it will disturb stuff that much though… Sure, for Core Improvements it will mean some additional work before and after, but all the whitepaper promises. Hope the swarms aren’t using mailservers or Infura directly anywhere :stuck_out_tongue:

5

I actually wonder how we will disable Infura without making a special build though.

1 Like
6

You must have missed this @oskarth :wink:

Status.app

1 Like
7

I love the idea though since most if not all of the app won’t work, I am also rather in favor of a special build for now. It could use notetherscan.com instead of etherscan, a set of non-existent bootnodes and mailservers, wrong infura proxy, and so on…

2 Likes
8

If we’re dogfooding the app, we should be dogfooding it’s weaknesses too.

9

April 1st.
Chaos Unicorn day.

4 Likes
10

FY0iupz
FY0iupz596Ă—534 270 KB
is “Chaos Unicorn Day Mother Fuckers” too much? I can redo the image if you all don’t like this one.

6 Likes
11

April 1 and Chaos Unicorn day it is!

If we are going do this it should be for the whole app, not making exceptions. The point is to kill all centralized aspects of our app and see how we deal with it (I’ll likely personally prepare with my own bootnode, for example). We can use special builds to prepare beforehand, if we wish.

As to how specifically, I’m sure we can figure something out. Could be time based or based on (non) responsive from a ping, etc. Suggest we do it from midnight UTC April 1 and for 24h. April 1 is a Monday.

Next steps:

4 Likes
12

@oskarth I highly suggest you prepare some comms and response for all the people who are not expecting this nor totally understand why you are doing it and what to expect. I.E. when everything breaks and stops working, you have a response to those who’s first reaction is to just stop using status and walk away.

For those people not immediately aware of the technical details of the cluster (which is many), they will just assume Status is broken/sucks and stop using it.

We have already seen some negative feedback from the EthDenver community (and those are people in the larger ethereum community) and @Hutch and myself work very hard to onboard these people, nurture their experience, and educate them. To throw away that work for an experiment is disappointing.

2 Likes
13

@jonathan I agree! One idea is to show a small popup in the app with a link to the site explaining what is going on. We should also tweet something and let people know in #status a few days beforehand.

There’s a big difference between things randomly breaking and deliberating constraining ourselves to what we are supposed to deliver. Considering how few users we are, and how involved they generally are in Web3, the latter is hardly “throwing them away”. If anything, these people are more likely to pay attention to what we are trying to enable here - censorship-resistant and decentralized communication that we can’t shutdown.

I realize this might seem scary and like a step backwards, but really this is a great opportunity to:
(a) build what we are supposed to be building
(b) educate people on what we are building and why
(c) involve them in making sure we deliver on our promises (e.g. we could make a “how to survive Status cluster blackout” with links to dappnode guide, etc)

2 Likes
14

This is absolutely essential, yes. We’re already being kicked around (i.e. the average civilian’s “wtf if Status doing with its millions to deliver a simple app?” attitude) by various other wallets/dapp browsers/messengers that have a more centralized aspect but people don’t mind because they’re not aware of what can go wrong. If this somehow goes mainstream, it can be an annual chaos holiday where Infura, Etherscan, and other API providers willingly power down their free APIs for 24 hours as a push directed towards the entire ecosystem, essentially doing a yearly reset of people’s expectations. There can even be a special “chaos API” that apps could opt to use, and only those using those APIs would be subject to the shutdown.

There is no better time to do this than now(ish) - the userbase is small and the protocol is being researched and is in need of thoroughly investigated vulnerabilities. This would let us truly prioritize.

3 Likes
15

It’s almost time! Follow up issue covering TODOs on cluster/app/comms side: Chaos Unicorn Meta Issue · Issue #7726 · status-im/status-mobile · GitHub

Raw notes from call a few weeks ago Chaos Unicorn Day Call - CodiMD

Please keep in mind this is more of a fire drill, and should require minimal effort from most people, except on April 1 when it’s scramble time.

1 Like
16

The preparation epic.

https://www.pivotaltracker.com/epic/show/4218446

17

This Chaos Unicorn Day reminds me of the day we deleted slack, or rather the day @oskarth left slack and started the great migration. Here is the historical footage btw:

https://www.youtube.com/watch?v=jbq5dsQ-l9M

Anyway what really helped the transition for me was (and still is sometimes) Status.im People
Despite all the time that I had to start over with a clean db, I was always able to rely on this directory to find the address of the people I needed to talk to.

In the case of the Chaos Unicorn Day, I wonder if we could have an equivalent for bootnodes and mailservers. Maybe a way to share bootnodes? Even communicate a twitter hashtag like #status-bootnode where people share qr codes?

5 Likes
18

I like the hashtag idea!

19

At least I have chicken.

Agree, how about using the #chaosunicorn hashtag? It’s already communicated in the website and blog post.

20

In regard to https://www.pivotaltracker.com/n/projects/2232205/stories/164626713/ (Create an HTTP endpoint showing if Chaos Unicorn Day is active). I’ve talked about it with Igor, Pedro, and Roman, and suggested that using a HTTP endpoint is more work and less reliable. It would be simpler to just use a DNS TXT entry.

I’ve already created one for Roman to test when developing his App changes:

 $ dig +noall +answer chaos-unicorn-day-test.status.im TXT
chaos-unicorn-day-test.status.im. 300 IN TXT    "DO IT LIVE!"

When the day arrives I can simply switch it to be chaos-unicorn-day.status.im.