Rebase your CI
I’ve seen A LOT of continuous integration implementations, both while mentoring CI and during my day work.
One of the simplest but most important of the HOLY 11 CI COMMANDMENTS is:
Keep the build fast
And yet, the top mistake I keep seeing all over:
Wasting time
Things we keep doing wrong
- Long build times due to lack of simple caching
- Cumbersome “SAFE GATES” that involves humans (!)
- Not paralleling where possible
And the list goes on and on.
HOWEVER
Not all is lost, I’m going to share with you a simple small change of mindset that will improve your CI/CD builds.
Another good old nugget of wisdom is Kent Beck’s DRY design principle.
If we think about that in a CI point of view, you’ll see that typically , people are running the same tests on their feature branches as in master
(or for those who still do develop
as well).
This means that for the same set of code, without any changes, we violate the DRY principle by assuming that the build is mutable for the same set of changes.
HMM
Let’s rethink that, and what could cause such despicable mutable builds:
- There’s no way to actually determine that nothing has changed
- Our build system is mutable by design, each build might break due to weak dependencies or simply not having immutable infrastructure for your CI jobs (hello
Exception: out of disk space
my old friend).
Okkkk, well?
Whilst the second bullet is something harder to solve (or just move to one of the SAAS CI services out there), the first is incredibly easy.
NOTE: I’m going to assume that you’re part of this century and working with Git.
Enter Git Rebase
If you’re working with Git Rebase, and you should, there is no difference between the code you’re testing in your feature branch and master
.
Simply because it’s the same Git Commit SHA-1.
Git has already done all the hard work of uniquely identifying your set of changes.
WHOA
Thus, if your CI build takes X seconds, you can shave off the same X seconds from your master
build.
WHAT, I TOLD YOU IT WAS SIMPLE
To recap
By using Git Rebase, you can make your CI pipeline leaner, but simply not repeating yourself when not needed.
Here’s an example
The example below is using Travis CI, but it can just as easy be any other CI platform.
Before
|
|
This will run for whatever push you are doing, including pushing the same commit that was already tested to master.
After
|
|
This ensures that tests will only run on feature branches and not on master
or tags.
In addition, the cache is separated between developer dependencies and production dependencies to speed things up a bit more.
Gimme them numbers
NodeJS projects: Removed ~3 minutes from the master build.
Scala projects: Removed around ~14 minutes (!).
For simplicity, I did not include the entire scripts, but several other changes besides just not running tests were made in the same mindset:
- Don’t rebuild docker images, only add tag
- Don’t recreate artifacts when not needed