I’m really glad to see a push for clearer backend direction and long-term architectural thinking. Thank you for sharing it. There’s a lot to align on, and the direction outlined in your post opens the door for meaningful discussion. Which is really great to see.
For the sake of alignment vis-à-vis the actual implementation within status-go could we visit a few points just to balance out some observations in this post? This could be a good opportunity to polish the context as presented above and perhaps apply some minor tweaks for accuracy on the architectural reality.
As stated, I really support the mission of defining an ideal backend architecture. Maybe to help us work towards a practical roadmap I have noted a number of points that jumped out at me, I believe those not familiar with the status-go code base would benefit from understanding them.
The lack of documentation
Agreed. We have fits and spurts of motivation on this, but it always finds itself deprioritised after a time. Formalising specification is always a worthwhile goal, and I welcome efforts to improve it collaboratively. I understand this is a shared issue across many IFT teams so working together we can definitely benefit each other.
Chat protocols.
This is an area we certainly need to smarten up. As many will know in Vac’s absence Status CCs have actively worked on the chat protocol for years, though the work of encapsulating it has never been a priority.
You’ll likely remember that 5 years ago we chose to de-encapsulate the chat protocol by migrating the chat repo in to the main status-go repo, we can debate whether that was a mistake, I’d argue it was. For those that don’t remember see the details from the repo:
- status-go/protocol/README.md at develop · status-im/status-go · GitHub
- Move to monorepo structure by pedropombeiro · Pull Request #1684 · status-im/status-go · GitHub
- Remove submodules by adambabik · Pull Request #1835 · status-im/status-go · GitHub
“incorporation of new protocols and the development of new features and business cases for Status is slow and difficult to reason about”
This point is probably meant to be read with a pinch of salt, as the observation obviously doesn’t actually track with the experiences of Status devs. Those reading this literally can be assured that we have many many many examples where we’ve implemented new features and new protocols in rather impressive timeframes. A handful of examples include 
:
- Chat Identity protocol and user profile feature.
- Wallet on-ramping
- Wallet swaps feature
- Actually the entire wallet product is a testament to fast and easy to reason development.
- Local transfer, sync and pairing protocol and feature
- Double ratchet encryption algorithm implementation for group and 1:1 chat
- Hash ratchet encryption for communities encryption
- Communities history store, transport and restore.
- Push notifications and local notifications.
- etc etc
“no integration of Nimbus or verification proxy”
Yes, this is a critical and highly desirable feature, though the issue with implementing it within status-go has never been architectural.
For the benefit of those new to the project, we have in fact attempted 3 times with concerted effort to integrate Nimbus / Fluffy. The real obstacle has consistently been the extreme difficulty of integrating Nim-based components into a complex user-facing cross-platform Go/mobile stack.
This is a very large and technically complex subject, and the details of which are understandably occasionally forgotten. For the benefit of institutional knowledge I’ve already drafted a highly detailed analysis of the extensive and technically intricate issues we have experienced with this venture. And I’m currently consulting with our fellow CCs on each point, and I can share it once we are confident of the content.
The unavoidable reality is that the go language is near peerless for cross compilation over multiple demanding platforms, Nim regrettably is not. We need to work towards resolving this, I’d love to see Fluffy working within Status it would be amazing. In any case to move forward, any future plans for integration need to take on board the lessons learned on this matter.
“It has been difficult to abstract away key protocols”
The Chat SDK could be used within the current structure, but improvement to the architecture would certainly make this a better developer experience. There is always room for improvement, and currently there are some unfortunate tight couplings between the core products within Status(e.g. Messenger ↔ Waku ↔ Wallet), but they don’t present a current blocker to building a chat SDK.
Communities are relatively self-contained. Its current immaturity though is regrettable and due to a rushed implementation under pressure from a poor understanding of the complexities by a former program lead. Communities grew too fast, with baked in assumptions, overly opinionated, and wasn’t resourced properly to evolve.
“The community can’t build on Status”
This isn’t a technical or architectural limitation. It’s a product focus issue. The only real extensibility feature we had, the DApp browser, was removed under instruction by the same former program lead. Dapps were/are an exceptional way for us to encourage community building, we would write about it regularly. See here for examples Status app - Dapp integration news.
More broadly speaking 3rd party building hasn’t been prioritised as a use case. That’s a strategic call, arguably not an architectural limitation. On a product level we have not implemented the required hooks for 3rd party developers to build from.
That said, we are working right now on improving chat extensibility incrementally. It’s a product-driven effort, and like all things Status, it’s proof-oriented: build, test, validate. Architecture will follow function.
For more details see our first stepping stones to allow a more extensible chat
In Summary
I really love the initiative, and after reconciling it with what implementation work actually looks like, we’ll be able to make some great progress. I think you will find strong support of the idea within Status, we all want a future-facing status-go that can work cleanly with Logos technologies (personally I’m dying to see Codex able to be used within Status).
It will take real effort though as it needs to account for cross-platform constraints (desktop and mobile) as a first-class requirement. It’s one of the hardest and most constant forces shaping the Status codebase.