Extensions 3.0 Proposal

Browser
1

Extensions 1.0

Language: JavaScript
Runtime: JavaScript interpreter in Go (GitHub - robertkrimen/otto: A JavaScript interpreter in Go (golang))

What was good:

  • Programming language support (JS)

What was bad:

  • No extension code validation
  • Bad UI interface in extension’s code
  • Interpreter didn’t support all EcmaScript specification
  • Because of the bridge between App and extensions, UI was laggy

Extensions 2.0

Language: Clojure like Markup language
Runtime: Pluto engine (GitHub - status-im/pluto)

What was good:

  • Extension is run in the app (no bridge)
  • Extension’s code validation
  • Hooks
  • Queries
  • Events
  • Extensions db

What was bad:

- No programming language

What was wrong in both versions, we were trying to develop extensions engine without understanding what developers of extensions really need. Our focus was on command cli, but it’s too complex, and most devs didn’t understand how it works and development of extensions was complex

Proposal for Extensions 3.0

Language: Clojure
Runtime: Status app

Because clojure code is data , we can validate it easily, we could use clojure spec (Clojure - clojure.spec - Rationale and Overview) to validate extensions written in clojure

Also clojurescript can compile itself, that means we could run validated extensions’s code right in the app

And instead of cli we could give developers a blank screen and they could do whatever they want without complex rules etc, like dapp in browser but native in status app

MVP could be done in 1m / 1dev

4 Likes
2

Sounds like a project to put up on Assemble when in launches in two weeks :slightly_smiling_face:

3 Likes
3

Just out of curiosity, why not javascript?
It has likely a wider appeal, and in terms of security it’s identical, as static analysis can only go that far, so most of the security measure we’d take would be into making runtime execution safe (sandboxing, limit access to resources/api etc), and potentially we can leverage other tools (jslint for example, for syntax checking).

Not that I don’t like clojure of course, just wondering whether it should be taken into consideration javascript as well, as it might have some benefits.

1 Like
4

Looking around, there’s also a few people recommend eclipse plugin architecture as a good example, Notes on the Eclipse Plug-in Architecture might be worth taking inspiration from a successful architecture if we are solving a similar problem

5

I think because it will be simpler to incorporate extensions in the app in that case, and make them more native with smooth UI, I like how it works in 2.0 version , there you can use re-frame subscriptions and events, so extensions are like a part of the app but installable, also, extension’s devs could contribute to core, and core devs could create extension easily

in 1.0 (version with JS) we had a lot of issues , there was a problem with the UI, you need some syntax for UI elements, and you need somehow to handle events from the UI, it might be complex

Our experience with 2.0 shows that clojure is not a problem, the problem is the lack of a programming language

With clojure i can see how it can be done in a reasonable amount of time

1 Like
6

definitely speak with @zahary on this as he has alot of experience with plugin architecture and put forward some interesting proposal privately

1 Like
7

I’ve been trying to find time to write more about my ideas regarding extensions. Luckily, I’ve had few opportunities to discuss them in person with Jarrad, Yenda and Andrey, so I’ll try to summarize only the most important points as a start:

  1. If we think about Status as an operating system or a window into the Ethereum world, the extension system is our application platform for the developers who will create the killer apps for the OS.

    When you strive to create a development platform, you want to optimise for maximum developer inclusivity. This means selecting programming languages that are widely used and familiar. For me, the most reasonable choice here is JavaScript and WASM. Both of them are required for running the browser-based DApps and the Eth2 smart contracts and they come at no additional costs for the client.

    We should remember that creating a platform is all about the developers, developers, developers :slight_smile:

  2. We have an unique opportunity to create a new kind of DApp - the social one. We should not think about the extensions as merely a way to customise your own experience. Instead, the APIs should allow the creation of distributed apps that can interact with multiple users. Status brings to the table notions such as identity, contact list, different modes of communication and strong support for data synchronization through vacp2p. These capabilities could be a powerful attractor for developers trying to leverage the platform to create all kinds of social DApps, from simple ones such as splitting a bill among friends to complex marketplaces connecting sellers and buyers (e.g. Uber/LibreTaxi).

  3. All code is data (all languages can be represented as abstract syntax trees and run-time bytecode). We have decades of programming language research exploring different ways to achieve some level of “code validation” and “formal verification” through the use of type systems, symbolic execution, proof-carrying code and so on. Inventing our own way of doing this is no picnic, but there is a much better understood security model used by the extension systems of browsers and the modern mobile OSes. It’s the capability-based security model with on-demand permission screens. A DApp can state its required permissions in a manifest file or it can request them with code looking like this:

    var sendMessage = requireCapability("SendMessage")  
sendMessage(...)

    On extension activation or during use, the Status app asks the user to confirm the granting of the requested permissions. The DApp developers get to use their familiar powerful general purpose programming languages. We can tackle the halting problem with with the usual gas model employed by Ethereum.

  4. If we further think of Status as a standardized protocol with a reference client implementation, then the extension platform must be part of this standard. Otherwise, it won’t be possible to create interoperable clients capable of running all DApps targeting the users of the network.

    For this to work, the UI layer has to be more portable. But before we venture deeper into this, let’s make some observations. Since the data synchronisation layer is based on event sourcing, we can leverage that to create a strong separation of the following concerns:

    • The app model (this determines how the ordered events lead to changes to the agreed-upon application state).

    • The controller API (this determines the types of events that can be written to the synchronized event log)

    • The UI layer (this determines what gets displayed on the user screen and how the interactions lead to invocations of the controller APIs).

    Since the UI layer of most apps will be open-source, the users will be able to tweak and replace it with a different one that exploits the capabilities of their device better. I know that @yenda for one cares deeply about this and @jarradhope is currently working on bringing Status to feature phones.

    I think the platform should offer two types of UIs:

    A. Conversational ones that embed buttons, responses and graphics within the chat interface of Status. The APIs here will offer a limited choice of UI controls that offer great accessibility and are easy to implement within a more limited client. For example, the APIs for using an image in this context should also require a text caption that can be displayed in place of the image in the Status terminal client or within an audio-only client for the blind.

    B. Full-screen interactions. Think about ordering a taxi or playing a chess game with a friend within Status. Such apps will fill a large portion of the screen, leaving only a small area controlled by the Status app. The framing of this area is important in order to make it obvious to the user that a full-screen interaction is underway, so the extension cannot launch a phishing attack imitating part of the Status UI and tricking the user into entering sensitive data. The Status-controlled area allows the user to switch between the active interactions, to exit any misbehaving one and to safely return to a previous state.

    To gain portability here, I think the DApp manifest can feature multiple alternative “content types” describing the UI layer in different terms:

    a) Canvas - a simple drawing API offering a subset of the HTML5 canvas capabilities. The UI code can feature arbitrary drawing commands that populate the designated “full-screen” area and it can react to user interactions such as touches, clicks, hardware buttons and so on. The main idea here is that the reference ReactNative client will be able to map this API directly to a regular canvas element within React, but a more minimal client will be able to package a library like Skia to implement the API directly without relying on a HTML5 run-time.

    b) VDOM - The UI code is responsible for defining a function that will produce a DOM-like tree of elements that can use a subset of the CSS layout directives (for example, only flex-box). Again, the idea here is that the ReactNative client will have everything needed out of the box, but any other conforming client will just have to implement a subset of the CSS layout algorithms or perhaps even a simpler scene graph API that allows adding and removing visual rectangular elements that will be drawn in back-to-front order. The virtual DOM diffing engine and even the CSS layout algorithms can be shipped as a WASM library maintained by Status that any conforming client having a WASM run-time can use without much effort. Within this library, we can also ship a rich set of reusable interactive controls defined on top of the DOM that will provide a common look and feel for all Status extensions.

    c) Text wizard (conversation). This reuses the controls described in section A and it can act as a fallback mechanism describing the full-screen interaction in terms that can be implemented in a terminal client or a screen reader.

    d) HTML5 DApps - Regular web and web3 sites will be also able to detect and request access to the extra communication APIs offered by Status. This is not much different from the mechanism used by browser extensions such as MetaMask that insert additional JavaScript functions within the page.

    So, I believe that the popular DApps will strive to develop multiple alternative implementation of their UI layer, so that each client can be served with the best option. Certain frameworks may appear that make this easy. By requiring the UI code to be obtained from an URL having a certain content-type, the clients can allow the user to override the UI with personal code or community-developed tweaks for certain devices.

I’d love to see an ambitious plan like this being funded on Assemble. I think we can bring some additional external developers that can try to validate some of these ideas by extending the Status terminal client and Stratus (@arnetheduck’s toy Nim client).

5 Likes
8

My layperson opinion, after trying to hire for Clojure folks & watching bounty contributors onboard, is that we should go with JS for extensions.

Much more accessible. I also like what I’m reading about a much better understood security model used by the extension systems of browsers and the modern mobile OSes—as lack of security modeling is the main reason we tabled the extensions alpha.

@zahary @andrey on the topic of getting outside help to validate these ideas, I’m having conversations with a couple of exchange DApps who are interested in wallet integrations—today actually. Can I put the idea forward, or are there questions you would like me to ask?

9

In a way we already have this kind of js extensions with the much better understood security model used by the extension systems of browsers and the modern mobile OSes, that’s the Status browser and Status api with permissions.

A first step would be to extend the capabilities and have more fine grained permissions. A fiddle could help push dapps just like the one we had for extensions. One thing that would help is a better integration of browser windows, like the cards UI, or mixed in the home screen with chats and a quick filter. There should be an option to reset a dapp to its home since you can quickly navigate away from it in a browser scenario
That would provide the type B described by @zahary with I believe option a) and d) and work on iOS

10

What can extensions enable that dapps will never be able to do via the Status api?

11

The only thing that worries me about extensions, chiefly on the iOS side, is that it’s considered code execution and Apple might block app submissions because of it.

If we do implement this functionality, I’d say we should implement it behind a compilation flag that completely disables it for some platforms.

2 Likes
12

it depends which extensions we are talking about, but dapps are what’s I’m proposing, if we cover the whole web3 specs + some niceties like status chat & all that would be a good first step. Then like @andre suggest we can have something that will be more akin to code execution and banned on iOS but we have many options for Android. But we can also give a shot at using JSCore for run extension code

13

My proposal is more for extending Status app with more functionality, its about UX at first place , and deep intergration into status app, so i think it’s different directions with Zahary proposal.

I think status might be the first ever app with such powerful and simple extensions model, because of react , clojuescript and re-frame, it so simple to integrate hooks in every part of the app, because we already have this modular pure framework in status app itself, and any extension might be integrated in very simple way and be powerful with native UX.

Also it gives answers for two questions

What can extensions enable that dapps will never be able to do via the Status api?

UX. Dapp will never achieve native UX, try to work with dapps, usually its awful experience if we are talking about mobile, thats why some dapps have their own mobile apps, but we could give a powerful simple tool and they could use status app

The only thing that worries me about extensions, chiefly on the iOS side, is that it’s considered code execution and Apple might block app submissions because of it.

Because extensions are just the same code as status app we can include them into status app with status builds, so for example for ios there are only preinstalled extensions and for android you can install remote extensions as well

Why to have “preinstalled” extension, it’s also interesting model there users don’t need to deal with permissions, they could trust preinstalled extensions, why they still extensions and disabled by default, because we don’t know which exactly extensions user may want, and to do not make a monster app

Also we have huge experience with Extensions 2.0 and it’s proven that this model is simple and powerful

1 Like
14

This extension idea is brilliant! BRILLIANT!
Especially the guy that came up with Extensions 2.0 should get a nobel prize or something!!

7 Likes
15

Glad to see you here dude :smiley:

16

:joy: hiiii @dude-fake-mustache!

17

I think the most promising way to discover the required capabilities of the extension system is to try step into the shoes of the DApp creators. We can imagine some potential use cases and then try to see what are the APIs that will be needed for them. Here are some examples, I’ll be happy if somebody contributes more.

  1. I’d like to create a groupware product similar to Slack and Discord, but running within the Status network. I create a named organization and I send invites to people. Upon accepting the invite, the users are automatically connected to everyone in the org and they get access to pre-established rooms with history. Certain idiosyncratic practices of the organization are streamlined in particular channels and particular type of messages and actions.

  2. I have a dinner with friends. We all have Status, but I’ve just installed a new cool bill splitting extension. I use it to send a special message to everyone allowing them to follow some simple steps to accept splitting the bill.

  3. I want to create a marketplace connecting people physically. This could be the Status teller network or a ride-hailing app similar to LibreTaxi. Certain users have opted-in to share their location in a decentralized geospatial index and my extension is able to reach them via a set of messages guiding us through a streamlined exchange. Money changes hands in the process, but there is no privileged party that takes a cut out of every transaction. Ratings and sybil attacks are addressed through certain proofs of identity.

  4. I want to play a game with a group of friends. With the capabilities of Status, there could be something at stake here (i.e. a poker game) or it could be just a fun game such as Doodle or Die. I create a game instance and invite my friends with a message. Upon accepting it, a P2P network is established for carrying out the game communication. If the game is serious, we sign our moves and we use an external oracle providing a fair source of randomness for the game mechanics (e.g. the Eth2 randao reveals). The game result is submitted to the main chain.

  5. I want to enhance Status with new modes of communication (threaded messages, push to talk audio, turn-based discussions, etc) or new media types (3D models, screencasts, custom NFTs, collaborative documents, etc) that are relevant for a minority of users. Such users can opt-in to enable a specific extension giving them the extra capabilities.

Once we identify some directions as more promising, it would be easier to translate them to specific plans for the extensions API.

We are likely to see that the creators of many DApps would want to make them as broadly accessible as possible and they would prefer if Status is used only as an identity provider (i.e. the web3-equivalent of OAuth). Even for this goal, they would probably prefer something more open-ended such as ERC 725. With identity in place, the users may be onboarded into any kind of experience through a deep link that can be shared in any platform.

But if we take a very long view on the decentralized web, we can see that all kinds of applications should live there eventually. The challenge for their creators is to build the required infrastructure. Everyone will be solving the same data synchronization problem in the absence of central servers and we can see ourselves as vendors of this technology. In the current centralized world, platforms like Parse and Meteor are examples of similar SDKs allowing you to build applications without managing any servers. With VacP2P, distributed storage and a SDK for custom side-chains, we can offer an interesting platform that can save a lot of time for many DApp developers in the decentralized world and once Status gains popularity additional network effects will kick in. Maybe we should see these efforts as wider-reaching than the Status app itself, so the extension system is merely an unprivileged user of the underlying technologies (we’ll be talking about VacP2P apps instead of Status apps). It’s true that most of the suggested DApps above can be delivered in a WebView with links providing the bridge between chats and the full-screen UI, with the only downside that this slightly raises the bar on what it means to be a Status client.

Finally, I would also acknowledge that If the extensions are concerned only with scriptability and local UX customization, then they would be client-specific and a tighter integration with the app UI is certainly possible.

2 Likes
18

Some we know from actually building DApps for Status internally and externally.

Something that has been requested for sometime is the ability for a DApp to send a message. I think the user flow would be similar to how some web apps request to post something to twitter on your behalf where the user would need to confirm the action.