Went back to using the metro packager as I was running into other issues with Haul when deploying the release version.
Managed to progress with the Nix recipe that deploys the prebuilt node_modules as symlinks with only the required files as read-write files. However I’m now confronted with a loading error when gradle runs the following command:
Loading dependency graph, done.
error Unable to resolve module `@babel/runtime/helpers/interopRequireDefault` from `/home/pedro/src/github.com/status-im/nix-react-native-test/index.js`: Module `@babel/runtime/helpers/interopRequireDefault` does not exist in the Haste module map
This might be related to https://github.com/facebook/react-native/issues/4968
To resolve try the following:
1. Clear watchman watches: `watchman watch-del-all`.
2. Delete the `node_modules` folder: `rm -rf node_modules && npm install`.
3. Reset Metro Bundler cache: `rm -rf /tmp/metro-bundler-cache-*` or `npm start -- --reset-cache`.
4. Remove haste cache: `rm -rf /tmp/haste-map-react-native-packager-*`.. Run CLI with --verbose flag for more details.
There is indeed a folder there (no differences from non-Nix node_modules), so I need to check what else’s going on.
took a small detour to look into generating ubuntu-server and realm binaries in Nix. Using node2nix 1.6.1, managed to package the pkg node.js package in a way that doesn’t requiring downloads from the internet and is therefore compatible, and leverage the great work by an external contributor in #7759 to do the same for Realm. Will come back to this later.
The test app is able to build/deploy if I make node_modules fully writable. If I make only parts of node_modules/react-native writable, then I’m met with the Haste map error mentioned above when
is run as part of react-native run-android --variant=release. Tried switching from yarn2nix to node2nix with the same result. The challenge will be finding which file needs its permissions changed. Will be giving strace a shot.
Turns out that the issue is not so much read-only files, but symlinks. Babel doesn’t like symlinks, so we’ll need to either do full copies of node_modules (which take ~9 secs) or do hard-linking. In any case, the idea is to only do this once, and after that, only redo it if you change target platforms (between mobile and desktop);
In order to run the test project with --pure, we need to pre-download the Gradle dependencies, so I’ll be writing a script that’ll generate a Nix map that can be used to download them;
Looks like there’s a bug in node2nix 1.6.1 where a package is extracted multiple times, and the second time fails because the target already exists.
Great tool from Serokell to automatically format Nix code (should help us keep our Nix code tidy and consistent): GitHub - serokell/nixfmt: A formatter for Nix code. Works great when coupled with a plugin for VSCode such as Save and Run. Just add this to VSCode’s config:
The tool is not entirely done yet, but the formatting it applies is very consistent and improves overall readability although unattractive in many places. We’re working hard to make it produce good results by the end of the month. In the first place, we aim to reach a nixfmt that generates somewhat pretty code for almost everything. At that point, it will be considered widely usable. Then we’d like to work along with the NixOS community people to bring the project up to standard for an officially accepted code formatter.
managed to finish Gradle installArchives phase (meaning it comprises downloading/building NDK dependencies such as boost) using --pure environment.
will work on getting react-native run-android --variant=release to work in a pure environment and then test that the sample project is reproducible across build machines.
Started applying learnings from PoC project in status-react. Finished wrapping all RN/RN plugins maven dependencies in Nix.
Looking into downstream issues where RN plugins under node_modules are missing folders (such as react-native-firebase/dist or react-native-*/android/build). It’s as if npm install isn’t running the scripts for those packages.
As you can see, the installArchives phase is now producing reproducible paths (starting with /build), which should help immensely and maybe even avoid having to patch the logging library:
It looks like node2nix only runs the install scripts, but some other scripts need to run when building the release Android app (namely react-native-firebase’s prepare script). node2nix probably does this to prevent arbitrary code from running, which would most often break pure builds anyway, so we’ll likely need to manually replicate those scripts from our dependencies.
Also, it doesn’t seem to install devDependencies, even though we set production to false (this is because it only seems to care about the top-level package, not the dependencies).
After manually performing the prepare actions for react-native-firebase, and upgrading react-native-dialogs which contained a duplicate type definition (not sure why this doesn’t cause issues in develop), I was able to get to the point where gradle tries to deploy the app to the phone, so we’re getting pretty close!
Fixed apk crash. App built on branch with node2nix and Nix-wrapped gradle modules runs just fine on Android device now. The problem was that gradle was using the existing cache which was creating issues for the Nix-based build. Solution was to define a local cache for the project.
Checked Nix cache build on the test/nix-gradle branch and discovered that although successful, it generated a derivation that was named differently than the ones requested by clients. After investigation, it turned out to be due to 2 separate problems:
The Android build was depending on the whole status-go package, instead of just the Android build, causing the gradle/npm expression to have different inputs depending on the value of TARGET_OS;
The CI build is checking out sources to directories which are named based on the Ci config, instead of status-react. Since the src attribute of the Nix expression takes on the project directory name, this ultimately caused the Nix expression to have a different hash. Will investigate if it is possible to pin down the name of the src derivation, and if not, work with @jakubgs to standardize the checkout directory name to always be status-react.
Just ran the first successful Android build for the test/nix-gradle branch, and already it shaved off 4 minutes from the Android CI build (from 16 minutes to 12 minutes). This is the first benefit of many to come for this branch (e.g. better reproducible builds, more pure environment, etc.)
Investigated Nix cache builds hash mismatches (PR created by @jakubgs to fix it);
Tested new Android build environment in macOS;
Started adapting Nix iOS expressions to make use of node2nix as well (so that the dichotomy in how node_modules is instantiated happens between mobile and desktop platforms, instead of android vs everything else).
To give an overview of what will be changing with the gradle/node2nix branch:
Maven dependencies will be downloaded and hosted by Nix. The gradle installArchives build will be wrapped as a Nix expression and cached;
npm modules will be likewise wrapped in a Nix expression (using node2nix), and a (mostly) read-only version will be copied whenever necessary from the Nix store into the local repo. Normally package.json won’t need to be symlinked to the repo root anymore. scripts/prepare-for-platform.sh will only contain logic to switch to the desktop platform, as the mobile platform will be handled completely in Nix.
This means that when we need to add or update an npm package, we’ll need to make sure package.json is on the repo root, run the commands we need in npm, and then run a script that will regenerate the Nix expression from the updated package.json file.
Likewise for maven dependencies, although in that case some automation in order to find missing dependencies will be welcome (so far I’ve relied on running the build, massaging the output errors to find the missing packages, regenerate the Nix expression and repeat until there are no more errors).
The reasoning for a (mostly) read-only node_modules folder: ideally we’d symlink the folder directly to the Nix store so that we’re guaranteed to have a reproducible source. Unfortunately react-native wants to build some stuff after the fact and this is incompatible with the concept of a pure Nix package. Therefore we copy the whole source to the repo directory, allow writing only on the folders where it is absolutely required, and therefore we still keep some peace of mind that the rest of node_modules is unchanged the rest of the time.
Got make release-ios working in the test/nix-gradle branch with node2nix, although the react-native CLI tool doesn’t seem to be available on macOS, will need to check that.
Noticed that make react-native-android and make react-native-ios are broken in this branch. Will be looking into it:
Loading dependency graph, done.
error: bundling failed: ReferenceError: SHA-1 for file /nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/lib/polyfills/require.js (/nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/lib/polyfills/require.js) is not computed
at DependencyGraph.getSha1 (/nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/node-haste/DependencyGraph.js:259:13)
at /nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:204:26
at Generator.next (<anonymous>)
at asyncGeneratorStep (/nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:46:24)
at _next (/nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:66:9)
at /nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:71:7
at new Promise (<anonymous>)
at /nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:63:12
at Transformer.transformFile (/nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/DeltaBundler/Transformer.js:229:7)
at /nix/store/4yldv0x1ml5a7r3bsnmnr7mrn0x8n2xa-gradle-install-android-archives-and-patched-npm-modules/node_modules/metro/src/Bundler.js:83:34
BUNDLE [android, dev] ./index.android.js ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 100.0% (1320/1320)::ffff:127.0.0.1 - - [05/Jun/2019:21:54:22 +0000] "GET /index.android.bundle?platform=android&dev=true&minify=false HTTP/1.1" 500 - "-" "okhttp/3.12.1"
The issue seems to be that the nix store node_modules is being used instead of the live repo, so the Haste map has no idea about those files.
turning off minification by changing :optimizations to :simple solves that, and lets timbre log library macro calls differences show up;
a small difference in the header of classes.dex which I believe is due to the differences above.
Any assistance from the Clojure team to fix these remaining 2 issues (expecially the minification one) would be greatly welcome, as it is more up their alley.
For the second issue, I think I’ll bite the bullet and make a make assemble-android-release target which will do the same as the CI server, but all inside of Nix (with reproducible build paths).
I took a look again at the Timbre logging library sources, and it looks like we have 2 options to obtain deterministic code (other than switching libraries):
fork the library to remove the rand call used to calculate the callsite-id (this rand call is only used to distinguish among multiple calls to logging functions in the same line, so it’s not really required for us);
wrap all calls to Timbre so that we don’t use macros but rather functions (the downside is that we can’t really guarantee that someone won’t use Timbre directly in the code rather than our wrappers, except maybe with some automated code checker).
I’d appreciate input from the Clojure team at large about the preferred way to go about this.
UPDATE: The more I think about this, the more I lean towards the first approach, as it seems to be the only that allows to take full control of the calculation of the callsite-id value. Removing the rand call would give us reproducible builds on the same build path, but if the same commit was built on another machine with another build path, the full file path would still be used to compute the callsite-id, leading to a different number. Ideally we’d use only the relative path of the source file, or worst case, just the file name.
Forked the timbre logging library and integrated it with status-react. That problem is gone.
Added some post-processing to the build of status-go so that 2 headers containing version information are removed from the libraries. They are now identical when built on different machines;
There was a mismatch in the number of files under assets/js-modules (the local build sometimes had more files). Turned out that we weren’t cleaning build directories, so existing build artifacts from building other branches were getting integrated into the apk. This is yet another reason to try to have an end-to-end pure build for Android in Nix.
At the moment, there are 2 problems that subsist:
the minification that the Clojure compiler uses is not deterministic;
some resource xml files in res folder have a single byte that is different when building in different machines.
Im trying to build android with the custom status-go tag, and have this
building ‘/nix/store/spq4p8h9xv2vd73m9kxm2hdlh6cvdn6m-status-go-onb.v.0.1-android.drv’…
unpacking sources
unpacking source archive /nix/store/bhgd62k3n16hc4hrplqkjg6m1wk4v6d5-status-go-source
source root is status-go-source
patching sources
configuring
There are some required tools missing in the system:
Xcode 10.1
iPhone SDK
Please install Xcode 10.1 from the App Store.
builder for ‘/nix/store/spq4p8h9xv2vd73m9kxm2hdlh6cvdn6m-status-go-onb.v.0.1-android.drv’ failed with exit code 1
