Block a PR from merge if e2e tests fail

Status Mobile
1

Hi all,

The topic is created for tracking progress and steps upon the workflow implementation

Steps:

  1. bugs in develop should be fixed for getting 100% passed tests:

  2. Each PR should be rebased on latest develop automatically in order to test integration part (@jakubgs can you help us with that?)

  3. Use PRs branch for running e2e tests instead of develop (@Anton will implement that)

  4. Implement feature which will block a PR (@pedro are you working on that? should I modify the report or do anything else in order to help you?)

4 Likes
2

@Anton I’m not yet doing anything in this area. I could add logic to Probot (similar to PRChecklist script) or if you’re dealing with GitHub API already, you could assign a status to the PR yourself when adding the report to the PR. That would probably be the most sensible solution. Let me know what you think.

3

You are right, I’ll do that since we are already dealing with the api for publishing the report

4

A relation question: Do we need to run builds and tests for all PR’s? It seems to add an extra burden by increasing queue times even for small updates that don’t touch the code. Perhaps we could add a skip-jenkins label?

edit: I see we have WIP [skip ci] use yarn instead of npm by jakubgs · Pull Request #5790 · status-im/status-mobile · GitHub but heard it doesn’t work

5

e2e tests are triggered by Probot and it looks like the logic to skip the tests should be implemented there
cc @pedro

6

I really loved how we’d solved this with TeamCity in @prevJob. We’d trigger a special TeamCity build, but build configuration allowed for a filter for unimportant files, so if all the changes were considered unimportant for a specific job, it wouldn’t run. So we’d keep the configuration where it belongs, at the CI job level, instead of a bot. Right now with Probot, it seems that the lowest hanging fruit is either the label or PR name prefix.

1 Like