Status-go Database De-dithering

Archive Software Engineering
1

Intro

The main thrust of this topic is summarised in the below GitHub issue:

Also I can see there has been an effort to identify all databases that are used by Status. Please note the focus of my topic is not to document or change database usage, but to consolidate functionality that interacts with the databases.

My Proposal

Merge all the database logic, in status-go, into one coherent package with appropriate sub-packages. Removing from the code base the numerous independent implementations for database interaction.

Details

The status-go repository has a number of database related packages that are spread out across the code base and I’m looking to consult with the team about consolidating them. Packages I’ve identified are:

appdatabase

Handles SQLite connection, used for device based storage

appdatabase/migrations

Handles SQLite migrations for:

  • settings
  • accounts
  • browsers
  • browsers_history
  • dapps
  • permissions
  • transfers
  • blocks
  • blocks_ranges
  • mailservers
  • mailserver_request_gaps
  • mailserver_topics
  • mailserver_chat_request_ranges
  • tokens

db

Handles data persistence using leveldb

This package is used for the storage of chat messages.

This package also seems to partially implement storage for Transactions, but doesn’t appear to be consumed anywhere in repo.

multiaccounts

Contains:

  • Account data model
  • SQLite connection code
  • migrations for the accounts table
  • Functionality for getting and setting Account and Settings data

protocol/internal/sqlite

Migration for code protocol/sqlite

Source code states that the code was generated from

// 000001_init.down.db.sql
// 000001_init.up.db.sql

Possibly a redundant package.

protocol/migrations

migrations for:

  • chats
  • user_messages
  • contacts
  • raw_messages
  • messenger_transactions_to_validate

At this moment I am unsure if the migrations in this dir are supplementary to or instead of the migrations found in protocol/internal/sqlite.

protocol/sqlite

  • Code for opening a sql database
  • Code for triggering migrations

sqlite

  • Code for opening a SQLite db
  • Code for migrations
  • JSON blob handling

What do I want from you?

I’d like your opinion on:

  • If it makes sense to consolidate these packages.
  • How you’d consolidate these packages

I’ll give my approach in a following reply :smile: .

Notes

Because all the database code is scattered this may not be the full list of DB things. I looked but looking doesn’t mean that I didn’t miss something. If you are aware of any missing DB packages in status-go please report them here :police_car: .

1 Like