Oops, got confused and was primarily thinking about the three-letter name in the user interface when writing the above, but the same reasoning applies to topic selection - specially about spreading public key bits.
Do we intend for this post-mortem to live here in discussā¦ Iād rather it be a separate document so I can start a section of post-mortems in the docs to gather them in a single place of ālessons learnedā
Just wanted to add that the single topic was a progressive change.
Initially it was only used as a discovery topic, for contact requests, after which step a random topic was used.
There was a bug there which lead to a drop in reliability, but instead of being fixed the single discovery topic started to be used for everything.
After some time the random topics were removed from the code altogether, until it became partitioned.
it is more a āpre-mortemā right now to discuss.
the real post-mortem should be a separate document for sure
1 Like
UPDATE: We decided to NOT ENABLE partitioned topic for 0.11.0 and spend more time researching for better solutions.
Update: The conversation continues in Update partitioned topic derivation algorithm by rasom Ā· Pull Request #7981 Ā· status-im/status-mobile Ā· GitHub
Copy-pasting comment in Github so this doesnāt get lost in a specific PR:
What @adambabik said. Spoke with @mandrigin about this, but it should be a separate PR to specs repo with a proposal that takes into account upgradability, etc. It doesnāt have to build on the current spec WIP PR, but it can refer to it.
Relevant here would also be to mention:
- bandwidth usage and how it might change with launch and users
- how this would impact trade-off of potentially not having compatibility
- what a 100% or 99.9% compatibilty would look like (multicast vs device abstraction) and what it requires
- entertain option of what keep listening to discovery topic and send less over it through some form of toggle would look like, i.e. 99.9% compatibility case (similar to device sync, but perhaps corner case re recovered account way in future)
- upgrade path, both from old to new as well as from proposal to further (say we change topic partition again in future)
- summary of options with trade-offs
- possibly options for sacrificing compatibility in short term and regaining it later, e.g. with device data sync
- evaluation should have a big premium/bias towards keeping compatibility, for reasons mentioned in several previous core dev calls
I realize this sounds like a lot, and itād be easier to just hack it and implement it. But we are trying to create a protocol and spec that exists on its own, and canāt be changed ad hoc. This will only increase in importance going forward, especially as we get more clients. By doing this properly now we are changing the culture of what spec changes look like, and ensuring we know how to do it now rather than later. It is similar to Chaos Unicorn Day in that sense. It is cheap to learn this now, expensive to learn it later.
As a role model here I just want to point to @arnetheduck, who has a lot of experience on working on p2p protocols, including getting 70m install for a compatible reverse-engineered p2p app 10 years ago. @cammellos rightfully points out there are some differences in terms of protocol negotiation etc, and this is the kind of consideration we want to see in the spec proposal. Similarly, @mandrigin and others have pointed out that we do have real-world constraints, such as launching an app with non-sucky BW usage, which might lead to different trade-offs. This type of thing can too be reasoned about thoughtfully in a proposal which clearly outlines different options and trade-offs with rationale.
Once that proposal is in a better shape, letās bring it up in Core Dev call.
1 Like
the very first draft (will work on rationale, etc tomorrow): https://github.com/status-im/specs/pull/4