We’re currently using Nim 1.2 for all our projects - locking the version has served us well because it allowed us to take churn in Nim out of the equation in our quest for a stable baseline to code against, focusing instead on solving the problems within our domain that we actually need to be solving.
In the meantime, we’ve also been in close communication with the Nim developers to address things in the language and tooling around it, as well as shine a focusing light on the quality of the compiler itself. The result of all this work is Nim 1.6 which will bring several improvements to the language, libraries and infrastructure, and sets the stage for even more changes in the future, specially on the tooling front.
With Nimbus out the door and more projects relying on Nim at Status, the time is ripe to start thinking about how to safely move to a new version of Nim - a somewhat daunting task for several reasons, above all because we want to do so without regressing on either stability or quality, and maintain flexibility in case something unexpected comes up.
The plan we’re discussing broadly looks like the following:
- We begin by upgrading our CI procedures to test all projects with multiple Nim versions - we are stepping up tests to reach Nim 1.2, 1.4 and 1.6 - some projects also test Nim
devel - The increased testing sometimes leads to the discovery of regressions and backwards compatibility issues in the compiler and libraries surrounding it - the CI at that point is marked as allowed-to-fail - this allows us to continue targeting 1.2 and not disrupting our ordinary workflow, while still keeping an eye on regressions that need addressing
- We report any backwards compat issues to upstream, and they help us investigate them - the aim is to get as many fixes as possible into Nim before 1.6 is actually released
- Once a library supports a particular Nim version, we remove the allowed-to-fail flag and keep it compatible with newer versions as well
- Once all libraries have coverage for all Nim versions and are stable, we can decide library by library what we want to do:
- Continue supporting all versions - this offers the greatest flexibility for consumers of our libraries
- Support all versions, but use shims to access new features and degrade gracefully for older Nim versions
- No matter what, we do not start using 1.6 features until our 1.2 codebase is stable with 1.6, without modifications
- Drop support for older versions
The decision to drop support for older versions should not be taken lightly - libraries are more useful when they put fewer constraints on how and where they can be used - supporting only a narrow range of versions is one such constraint, as all applications using a library must follow suit.
On the other hand, applications are well served by locking the compiler version - this makes it easier to maintain quality over time, avoiding regressions and making debugging and troubleshooting easier. In applications, picking a particular version also doesn’t force that change onto others, allowing each application to upgrade at their own pace and avoiding an exponential number of combinations of potentially incompatible libraries to test against.
With this testing infrastructure in place, it will also be easier to support coming versions of Nim - both point releases and major ones - in the future, it should hopefully be less bloody in general - another corner stone of making upgrades less painful is to use stand-alone libraries as much as is reasonable, over the Nim standard libraries - stand-alone libraries are easier to upgrade outside of the Nim release cycle allowing more frequent upgrades when needed (for example to address security issues) and fewer moving parts when not (when upgrading the compiler, the surface area of collateral changes is smaller).