Managed to build the Nimbus API with the Nix cross-compiler support (instead of resorting to Android NDK) so I could include the pcre dependency, but it still fails linking in status-go. I’ve removed -lpcre from the status-go code, but then run into:
# github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `raiseOutOfMem_mMRdr4sgmnykA9aWeM9aDZlw':
(.text+0x1c): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `raiseOutOfMem_mMRdr4sgmnykA9aWeM9aDZlw':
(.text+0x20): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `nimRegisterThreadLocalMarker':
(.text+0x3470): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `nimRegisterThreadLocalMarker':
(.text+0x3474): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `writeToStdErr_a2kDfqdSc1eYf0ZCWOGinQ':
(.text+0x5114): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o):(.text+0x5118): more undefined references to `stderr' follow
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `nimLoadLibraryError':
(.text+0x95c4): undefined reference to `__fwrite_chk'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `procAddrError':
(.text+0x961c): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `procAddrError':
(.text+0x9620): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `systemInit000':
(.text+0x11488): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_system.nim.c.o): In function `systemInit000':
(.text+0x1148c): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(_7vendor7nim-chronicles7chronicles7log__output.nim.c.o): In function `flushRecord_2TFyTWhDxeU5kGr2yqnA6w':
(.text+0xbcc): undefined reference to `stdout'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(_7vendor7nim-chronicles7chronicles7log__output.nim.c.o): In function `flushRecord_2TFyTWhDxeU5kGr2yqnA6w':
(.text+0xbdc): undefined reference to `stdout'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o): In function `secp256k1_context_create':
(.text+0xb50): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o): In function `secp256k1_context_create':
(.text+0xb54): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o): In function `secp256k1_context_create':
(.text+0xb74): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o): In function `secp256k1_context_create':
(.text+0xb78): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o): In function `default_illegal_callback_fn':
(.text+0xcd0): undefined reference to `stderr'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(secp256k1.c.o):(.text+0xcd4): more undefined references to `stderr' follow
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_io.nim.c.o): In function `readAll_mQf9blHFlTwuSvJjirhpY6g':
(.text+0xdec): undefined reference to `stdin'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_io.nim.c.o): In function `readAll_mQf9blHFlTwuSvJjirhpY6g':
(.text+0xdf0): undefined reference to `stdin'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_io.nim.c.o): In function `readFile_4PGnM9bWmsH0Nu7dnr3XzgA':
(.text+0xf78): undefined reference to `stdin'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(stdlib_io.nim.c.o): In function `readFile_4PGnM9bWmsH0Nu7dnr3XzgA':
(.text+0xf7c): undefined reference to `stdin'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
github.com/status-im/status-go/vendor/github.com/ethereum/go-ethereum/core/forkid
I’m a bit puzzled as to where this is coming from, since I’m not familiar with the Nimbus dependencies.
It would seem that gomobile isn’t linking the standard C library in, for whatever reason. I tried passing --passL:"-static -lstdc++" to the nim command to see if it would statically link the C stdlib, but that didn’t change anything according to objdump.
Thanks to @yuriy and @stefantalpalaru, who suggested increasing the target Android API from 18 to 23, that seems to have fixed the problem of the missing stdout and stderr symbols and unblocked the linking to Android status-go in status-react!
With that obstacle surmounted, Android builds started complaining about missing secp256k1 (since I’m localizing them with objcopy -L to fix duplicate symbol errors on desktop builds):
# github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(_7vendor7nim-eth7eth7keys.nim.c.o): In function `shutdownLibsecp256k1_B4oHFVQl9b4aMmNV3AIq9bFQ':
(.text+0x8): undefined reference to `secp256k1_context_destroy'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(_7vendor7nim-eth7eth7keys.nim.c.o): In function `newEthKeysContext_uA9a649aQkV8LOVyv7Gocwbw':
(.text+0x11c): undefined reference to `secp256k1_context_create'
go/src/github.com/status-im/status-go/vendor/github.com/status-im/status-go/eth-node/bridge/nimbus/libnimbus.a(_7vendor7nim-eth7eth7keys.nim.c.o): In function `newEthKeysContext_uA9a649aQkV8LOVyv7Gocwbw':
(.text+0x130): undefined reference to `secp256k1_context_set_illegal_callback'
So I tried not localizing the symbols on Android, but that gets me back to the duplicate symbols situation, so for now I’ll just add -extldflags=-Wl,--allow-multiple-definition to Android status-go builds so that the linker sticks to the first definition it finds and ignores the other.
Finally got Nimbus linked into status-react on Android. As of now it’s crashing because Nimbus doesn’t implement our InitKeystore endpoint, but at least now we have a starting point:
12-05 16:57:28.367 12917 12977 D StatusModule: initKeystore
12-05 16:57:28.368 12917 12976 D ReactNativeJS: DEBUG [status-im.utils.handlers:32] - Handling re-frame event: :hardwallet.callback/on-register-card-events
12-05 16:57:28.370 12917 12976 D ReactNativeJS: DEBUG [status-im.utils.handlers:32] - Handling re-frame event: :update-window-dimensions
12-05 16:57:28.371 12917 12976 I ReactNativeJS: Running "StatusIm" with {"rootTag":1}
12-05 16:57:28.373 12917 0 E Go : panic: InitKeystore
12-05 16:57:28.373 12917 0 E Go :
12-05 16:57:28.373 12917 0 E Go : goroutine 17 [running, locked to thread]:
12-05 16:57:28.373 12917 0 E Go : github.com/status-im/status-go/mobile.InitKeystore(...)
12-05 16:57:28.373 12917 0 E Go : /build/go/src/github.com/status-im/status-go/mobile/status.go:377
12-05 16:57:28.373 12917 0 E Go : main.proxystatusgo__InitKeystore(0x75c442ad60, 0x35, 0x0, 0x0)
12-05 16:57:28.373 12917 0 E Go : /build/gomobile-work/src/gobind/go_statusgomain.go:597 +0x4c
12-05 16:57:28.373 12917 12988 E GoLog : panic: InitKeystore
12-05 16:57:28.373 12917 0 E Go : main._cgoexpwrap_dc788809385e_proxystatusgo__InitKeystore(0x75c442ad60, 0x35, 0x0, 0x0)
12-05 16:57:28.373 12917 0 E Go : _cgo_gotypes.go:1278 +0x5c
12-05 16:57:28.373 12917 12988 E GoLog :
12-05 16:57:28.373 12917 13003 F libc : Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 13003 (pool-1-thread-2), pid 12917 (tus.ethereum.pr)
12-05 16:57:28.386 12917 12977 D StatusModule: openAccounts
Next step is to work with @kim to implement and test this endpoint.
Surveyed once again the endpoints required in the most recent version in order to have the MVP to create an account and send a message.
Started commenting out methods which aren’t absolutely required (e.g. saving account on disk after creation) and am now at the point where I can start the Nimbus node. Since @kim is on vacation today I won’t be able to make much progress here. Hoping to tackle it on Monday.
In order to avoid waiting for Nimbus implementation of account creation methods, I’ve started testing with existing account state. When calling nimbus_start, I’m getting a crash due to the new seccomp filter behavior in Android O. Trying to figure out what system call 186 represents:
12-09 18:00:38.784 8072 8072 F DEBUG : ABI: 'arm64'
12-09 18:00:38.784 8072 8072 F DEBUG : Happend: 'Mon Dec 9 18:00:38 2019
12-09 18:00:38.784 8072 8072 F DEBUG : '
12-09 18:00:38.784 8072 8072 F DEBUG : SYSVMTYPE: Art
12-09 18:00:38.784 8072 8072 F DEBUG : APPVMTYPE: Art
12-09 18:00:38.785 8072 8072 F DEBUG : pid: 7946, tid: 8010, name: mqt_native_modu >>> im.status.ethereum.pr <<<
12-09 18:00:38.785 8072 8072 F DEBUG : signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
12-09 18:00:38.785 8072 8072 F DEBUG : Cause: seccomp prevented call to disallowed arm64 system call 186
Decided to sidestep the seccomp filter issue for the time being since I can test the software on an emulator that has an older Android version. Spent the day figuring out the required toolchain/configuration changes and making initial tests. nimbus_start is now successful and I’m starting to go through the following calls from status-react (SelectAccount, etc.)
Survey of KeyStore types and functions required from Nimbus:
// Account represents an Ethereum account located at a specific location defined
// by the optional URL field.
// from github.com/status-im/go-ethereum/accounts/accounts.go
type Account struct {
Address types.Address `json:"address"` // Ethereum account address derived from the key
URL URL `json:"url"` // Optional resource locator within a backend
}
// ExtendedKey represents BIP44-compliant HD key
// from status-go/extkeys/hdkey.go
type ExtendedKey struct {
Version []byte // 4 bytes, mainnet: 0x0488B21E public, 0x0488ADE4 private; testnet: 0x043587CF public, 0x04358394 private
Depth uint8 // 1 byte, depth: 0x00 for master nodes, 0x01 for level-1 derived keys, ....
FingerPrint []byte // 4 bytes, fingerprint of the parent's key (0x00000000 if master key)
ChildNumber uint32 // 4 bytes, This is ser32(i) for i in xi = xpar/i, with xi the key being serialized. (0x00000000 if master key)
KeyData []byte // 33 bytes, the public key or private key data (serP(K) for public keys, 0x00 || ser256(k) for private keys)
ChainCode []byte // 32 bytes, the chain code
IsPrivate bool // (non-serialized) if false, this chain will only contain a public key and can only create a public key chain.
CachedPubKeyData []byte // (non-serialized) used for memoization of public key (calculated from a private key)
}
// from github.com/status-im/go-ethereum/accounts/keystore/key.go
type Key struct {
Id uuid.UUID // Version 4 "random" for unique id not derived from key data
// to simplify lookups we also store the address
Address common.Address
// we only store privkey as pubkey/address can be derived from it
// privkey in this struct is always in plaintext
PrivateKey *ecdsa.PrivateKey
// ExtendedKey is the extended key of the PrivateKey itself, and it's used
// to derive child keys.
ExtendedKey *extkeys.ExtendedKey
}
// ImportECDSA stores the given key into the key directory, encrypting it with the passphrase.
func ImportECDSA(priv *ecdsa.PrivateKey, passphrase string) (Account, error)
// ImportSingleExtendedKey imports an extended key setting it in both the PrivateKey and ExtendedKey fields
// of the Key struct.
// ImportExtendedKey is used in older version of Status where PrivateKey is set to be the BIP44 key at index 0,
// and ExtendedKey is the extended key of the BIP44 key at index 1.
func ImportSingleExtendedKey(extKey *extkeys.ExtendedKey, passphrase string) (Account, error) {
func DecryptKey(keyjson []byte, auth string) (*Key, error)
// AccountDecryptedKey returns decrypted key for account (provided that password is correct).
func AccountDecryptedKey(a accounts.Account, auth string) (Account, *Key, error) {
Making Nimbus stripped-down versions of services/shhext and node in status-go in order to be able to put the protocol Messenger and the RPC client in place.
Managed to get status-go to build with stripped-down versions of the Status services (still need to plug them in with Nimbus). Also need to implement the RPC client:
12-13 09:57:45.978 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error web3_clientVersion :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.981 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error mailservers_getMailserverTopics :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.982 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error mailservers_getChatRequestRanges :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.988 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error browsers_getBrowsers :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.990 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error mailservers_getMailservers :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.994 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error permissions_getDappPermissions :error JSON-RPC client is unavailable :params nil
12-13 09:57:45.997 2628 2654 W ReactNativeJS: WARN [status-im.ethereum.json-rpc:110] - :json-rpc/error settings_getConfigs :error JSON-RPC client is unavailable :params [["multiaccount" "current-network" "networks"]]
The stripped-down versions of the Status services are now mostly operational, but running into issues with Nimbus requirements to have nimbus_poll function be called periodically on the same thread that called nimbus_start.
RPC server/client services in status-go are operational.
Managed to implement a (ugly) workaround to allow initializing Nimbus and running the polling routine from the same OS thread in Go.
Ran into issues decoding RLP messages in Nimbus:
Failed rlp.read topics="rlpx" tid=17353 dataType=uint exception="The RLP contains a larger than expected Int value" peer=Node[167.99.19.148:443]
turned out to be due to Nimbus using the uint type for PoW requirement values, instead of uint64. @kim has a PR out to fix that.
Ran into issues with emulator device clock (we should implement a way to pass the NTP offset calculations we do in status-go to Nimbus so that it can apply that offset to get a much smaller time skew between the device clock and the atomic clock).
Running into an error from Nimbus: Message does not match node bloom filter. Will look into that next. UPDATE: turned out to be because I was sending messages from an old nightly build. Using a recent PR build I was able to receive the message with Nimbus.
