Here are just some of my thoughts about problems I see with the current extension design (GitHub - status-im/pluto):
Problematic points
-
No much thought about who will be extension writers actually and what’s the purpose of extensions. Are they primary meant for core developers to be able to make changes faster/more dynamically ? 3rd party Web3 devs ?
Empower non/less technical people to contribute directly by writing simple functionality ?
It’s quite important, as it has design implications and things which can make one group happy can make the other group angry and vice versa. -
Scripting (eq basic programming constructs) not supported with no clear plan how to introduce it later.
This has multiple implications, one of them is that we are many times re-inventing the wheel and introducing incomplete/buggy implementations while parsing extensions and generating views (lexical scopes, destructuring, etc.) second and most important is that extensions are almost unusable in current form for authors not having access tostatus-reactcodebase as well (where they can “cheat” and make specialised components later referenced in extension).
This could be of course addressed by slowly adding functionality (like logical operators, numeric operators,foroperator…) to the view “language” but we will spent a lot of time re-inventing the wheel to create a subset of Clojure with little bit altered syntax (which will feel super weird for any extension writer who already knows Clojure).
Ideas how to mitigate them
-
Discuss and think much more about who will be extensions writers :
Are we supposed to use them internally as a tool to boost productivity and/or ship (optional) functionality without going through traditional release cycle and app store installation ? In that case any non-standard syntax and unnecessary deviations from Clojure/Re-frame/Reagent conventions are problematic and not desired.
Will we target Web3 devs, most of them fluent in JS ? In that case we should probably re-evaluate the decision to push EDN (Clojure syntax) in extensions on them altogether. -
In case we decide that EDN syntax is indeed the best, I propose to go the full way and actually include restricted cljs execution environment for extensions.
Restricted cljs environment for extensions
The way how I imagine it could work would be that for certain extension contexts (like views/view-id, events/event-id) the forms would be evaluated in restricted cljs execution environment, with top-level bindings provided by us and every symbol restricted to either refer our top-level bindings or symbols from clojure.core namespace (we can fine tune that).
Due to the fact that code is data in lisp and awesome existing projects like clojure.tools.analyzer, we can run the form through analyser which will produce enriched AST like this:
clojure.tools.analyzer.jvm> (ast/children (analyze '(do 1 2 :foo)))
[{:op :const,
:id 0,
:type :number,
:val 1,
:form 1,
...}
{:op :const,
:id 1,
:type :number,
:val 2,
:form 2,
...}
{:op :const,
:id 3,
:type :keyword,
:val :foo,
:form :foo,
...}]
With this result, where each symbol is enriched with important information, we can trigger warning and reject the code every time non supported operation is used and basically restrict the code to functional, pure subset of clojure just producing data - no side effect calls, no platform interop calls, just pure functions from clojure.core namespaces + our provided functionality which will be injected into execution env as top level bindings.
Given that piece of code passes the security analysis, we can then safely evaluate and execute it.
So this in extensions:
views/preview
(let [{:keys [a b c]} params]
[react/text a])
Will be evaluated as
(fn [params]
custom-code-written-by-extension-writer) ;; params provided in top-level bindings
Advantages of such approach:
- No more reinventing the wheel by re-implementing lexical scoping, necessary operators, etc.
- For simple things, the custom code will stay exactly as simple as before, so writing simple view won’t be any more daunting or discouraging for the beginners compared to current solution.
But need to “cheat” and write custom components for extensions will be greatly reduced and with little bit time and practice completely eliminated (when we fine tune what bindings will be actually provided for the custom code) - Because we will evaluate just specific parts of the extension (like views, events and subscriptions) in the restricted cljs execution environment, we won’t loose most of the existing validation functionality, where we validate hook properties, etc.
Given thatclojure.tools.analyzeris super flexible and provides awesome data-rich output, we can easily do same analysis as before, eq checking if subscribing to subscription actually refers subscription exposed by host (capacities map), etc. - There will be no extra “scripting” resource to refer and going from simple extension with barely any logic to more advanced with custom events/subscriptions and/or view leveraging
mapfor repeating elements will feel natural. - Extension writing will be great experience even for experienced users (while less experienced users won’t be penalised), because it will be basically functional programmers paradise - writing pure functions, producing just data (view markup for views, effect map for events…) with host developers (core status devs) responsible for ensuring that such data are meaningfully translated into necessary side-effects.
