Cost related to Waku infrastructure

fryorcraken · August 26, 2024, 11:19am

As the discussion to integrate Codex [1] in the app has started, and as we are inching towards RLN readiness for integration [2] , I’d like to open the topic of infrastructure costs. This is specific to the context of Waku but can likely be extended to other infrastructures powering the app.

The intent here is to share knowledge and lay down possible paths.

There are existing writing on Waku cost and payment [3] [4].

Terminology: below, I refer as project an application that uses Waku. In this context, it can either be Status or a Status Community. A strict definition is not necessary as this article mainly target Status app. But it is important to keep in mind that a community could be seen as a project in itself with community owners the driving force behind such project.

Costs

There are 2 types of expenses when using Waku:

RLN membership price and related onchain gas cost, used to secure the network (similar to how gas or transaction fees secure a blockchain).
Hosting: the hardware and associated cost to run Waku services.

RLN

To publish a message on a DoS protected Waku network, one needs to hold an RLN membership.
We recently published a smart contract specs [5] as a starting point for RLN on mainnet.

An RLN membership has a rate limit over time. We are currently proposing up to 600 msgs per 10min [5], but this can changed and application developers should consider lower limits for users.

RLN memberships expire and need to have a cost. The deposit is refundable, after the 6 months expiry period. This is likely to change in the future.

Transactions to register a membership or extend it cost gas. As any transactions, gas costs depend on the chain used.

Hosting

Hosting covers a wide range of costs, from physical infrastructure (servers, datacentres, electricity, bandwidth, etc) to devops team running and configuring servers.

With Waku incentivisation [6] still a work in progress, the cost models are currently similar to any Web2.0 frontend/backend application.

Waku is a peer-to-peer protocol, so why do we need to host nodes? A few reasons:

bootstrapping in the network: some static infrastructure is needed to enable new nodes to connect to the network. Once connected, decentralised peer discovery can be used to find other peers
Supporting consumer devices: most users are expected to run Status app on mobile or desktop. Such devices can be offline for long period of time (or even mostly offline). As well has having constraints around bandwidth and CPU usage. To support such devices, Waku nodes provide several services:
- Store: to enable devices to retrieve messages sent while they were offline
- lightpush/filter/peer exchange (light client protocols): alternative protocols to the Waku relay peer-to-peer network, enabling mobile phones to send/receive messages and find new peers without consuming large amount of data.

You can learn about peer-to-peer network fundamentals in [7].

There are opportunities for end users to contribute to infrastructure costs:

lightpush/filter/peer exchange services are provided by Status desktop applications, servicing Status mobile users
Community owners could host own store node for their community, further work [8] is needed to make possible.

A couple of Waku milestones will change the picture above:

Store decentralisation: The building blocks to decentralise the store services are in place. We are starting dogfooding the last foundational protocol, store sync [9], which will enable store nodes to hold the same data. Once done, it should be possible to enable users share their own disk space to provide store services from Status Desktop.
Waku service incentivisation: we are working on defining an incentivisation protocol which would enable Waku nodes to be paid for providing services to the network. Coupled with a marketplace, this should provide more flexibility for projects and users to pay for the hosting of services.

Paying

I see two ways to handle costs:

Direct: User pays directly for the resources they consume
Indirect: Project subsidise the user by paying for their resources, assuming an expectation that the user brings value to the project making this investment worthwhile.

A subcategory of (1) can be qualified as altruist where the user provides resource or pay for them out of band. By sharing some local disk space or bandwidth to the network.

Incentivisation is likely to resolve (1). Waku service provision happens between a client and server, client can pay server for providing services. End user pays service node.

This makes (2) more difficult to handle, service credentials [10] could play a role to facilitate this. Where the subsidiser acquires credentials and give them to users, who can use them for services. The question is how to give credentials to users, while preventing abuse?

Removing friction

Great apps succeed by minimizing onboarding friction, allowing users to experience the app before registering. This enhances user journey and can boost retention & satisfaction.

In the Web3/Bitcoin ecosystem, including the Status app, an example is the required seed backup step being moved from onboarding, to a post-boarding notification or reminder.

In the context of the Status app and Waku, it is assumed there is a will to remove friction:
To enable users to load their social graph, chat with their friend and join a community with minimum friction.

We will see in this post that removing friction does increase complexity. Hence, it is best to assess this assumption and potentially invalidate it. Privileging a simple working system first, and remove onboarding obstacles once identified.
For example, is it more important for new users to try chat features for free, or for existing users to be able to cover fees for friends they invite?

Removing the onboarding friction of getting users to pay for their RLN membership and usage of the Waku network, raise the question on how costs should be covered.

In terms of RLN membership, there are several options on the table:

a. RLNaaS: services nodes attach RLN proof for light client for free.
b. Paymaster: User makes a transaction and a paymaster covers the fee. An interesting choice when considering the Status Network/L2.
c. Stealth commitment: Similar to a paymaster but without needing a paymaster, where RLN membership registration is fully done by a 3rd party, in a secure manner.

For hosting, until incentivisation is ready, the most straightforward manner is for the project to run the infra. This is what is being done by Status today, and should be feasible for community owners at some point [8].

Necessary friction

An important aspect when considering subsidising user cost is the need for friction. Every system exposed to the open internet has to have rate limiting, as no system has infinite resources. It is usually done with a combination of IP, captcha, email or mobile number or even governmental ids collection.

In the context of a privacy-minded and decentralised application, some PIIs (Personally Identifiable Information) may not be acceptable to collect.

Hence a careful review is needed to ensure the system is protected from spam and DDoS while maintaining the principles the applications strive for.

When considering blockchain accounts, the question arises: does a rate limit on any account work, without consideration for its history? History says no [11], as an account is just a cryptographic key pair.

Reducing RLN friction

If no-cost RLN registration is a must, then there are several way to go about it.

RLNaaS

With RLNaaS, service nodes have RLN memberships and light clients can borrow those one message at at time. with the service node handling the RLN requirements (attaching proofs to messages). It is an attractive solution to roll out RLN as it keeps RLN away from the end user. Enabling RLN roll out with minimal UX changes and product considerations.

However, it fundamentally changes the topology of a Waku network in that message publishers are not contributors to the network (i.e. do not run RLN relay nodes).

Moreover, the security it brings is limited, and some form of spam protection is still needed to ensure individual nodes are not DDoS’d.
Waku nodes do rate limits incoming requests, which protect them but creates a scarcity for users.
Abuse of the system may be possible: An attacker can attempt to continuously deplete the membership of the nodes. Potentially DoS’ing users by occupying all service nodes.
In this context, light push service nodes are being used as centralized REST API, which is not the intented design. It might work, but there is risk it may not.

While more complex solutions can be designed to mitigate risks, the easiest solution in this instance is for users to have their own RLN memberships to publish directly to the network.

Subsidising onchain

Both paymaster and stealth commitments are strategies where someone, such as the project, subsidies the membership registration for the user. Again, system design need to take in account potential abuse.
As previously stated, anyone can create an blockchain account for free as it only requires generating a keypair.
A solution is to let users in based on their past onchain activities. A downside is that users would need to dox their blockchain account to the subsidiser.
However, with both paymaster and stealth commitments, it would not be possible to link their account with the messages they publish using RLN relay.

Another solution is to encourage new blockchain activity. Rainbow wallet points, or SNT MP [12] are examples. In this case, the project can decide to only subsidies RLN memberships for active users. This could also act as a form of reward.

This solution assumes the project subsidies the user. Note there is an important security assumption here that a project may apply their own authentication mechanism prior to sponsoring a membership for a user. In other words, not all potential abuse have to be solved by Waku or RLN.

It is also possible for a user to subsidies another user. A referral model. With stealth commitments [13], Alice can invite Bob in by using a set of URL or QR codes. The final step being Alice registering Bob’s membership on chain, covering gas and membership cost for him.

Users can always buy their own membership. Sovereignty is always enabled, free as freedom but not free as complimentary.

Conclusion

DoS protection in the form of rate limit are needed for all systems exposed to the open Internet. RLN is the go to solution for Waku, it rates limits publishers. The friction to become a publisher is enabled by membership price and onchain gas.
For the sake of onboarding users, it is possible to remove this friction in several manners. However, a careful design is necessary to keep the system secure and DoS protected.