RFC: Improve CI / GH Actions / Jenkins setup

The Neos Project Tools & Operations
1

During the Contribution Day in Berlin @davidspiola and I discussed our “CI setup” with @mroberts, who provided very helpful insights and an “outside view” that triggered quite some thoughts. This tries to summarize what we came up with and outline a few steps into the future…

Any errors and omissions are mine, feel free to correct and amend!

The following is based on this flipchart photo from Berlin, only slightly helpful… (2.1 MB) and my imagination.

Vision

Anyone in the Neos team can confidently release a Neos or Flow version without fear of breaking things or the need for a lot of knowledge.

The release process as well as any other automated processes are working as invisible as possible while being very transparent about how they work and what they, any results can be traced back to the “magician” easily.

Goals

The current release process is not as automated as we imagined it to be at some point (see Release process topic sprint, Hamburg, January 15th/16th 2019 and https://www.neos.io/blog/off-to-a-better-neos-release-process.html), but by now I would not follow that path any longer.

The part that is and should be automated is the actual branching and tagging, and I dare say it works fine by now. It has some issues, still, but those can be tackled “small-scale”.

We looked at what we have and found:

CI structure
CI structure487×552 13.3 KB

From there we came to the conclusion that other aspects need consideration and identified some things we could do, that lead to other things we could then do to…

  • ease understanding of what happens where in the project
  • make it easier for others to contribute to automation, lower entry barrier
  • lower the complexity by having less “moving parts” to ease maintenance and development
  • save time needed to keep infrastructure updated and secure
  • save resources & money

Next steps

Ultimately this lead us to the following steps:

Along the way we should look into the CircleCI setup used for the Neos UI to see if we can get rid of that, to have removed another tool, but that’s not urgent (as the service is “free” in monetary and maintenance.)

4 Likes
2

@ahaeslich :wave:

3

Thanks for the summary, as I was listing only for a short time before lunch I was not that involved.
The CircleCI Setup can be replaced, although I really like to ssh into the containers when something went wrong. The GH Actions are a more black box, and that makes it not easier to resolve issues in the pipeline.

But with GH Action, we can also resolve issues we currently have. The e2e pipeline, for instance, is not running for forked PRs as we cannot share credentials and then Sauce Labs is not running.
With GH we can make it work :slight_smile:

Thanks for the rethinking :slight_smile:
It would be nice to break the barriers in the pipelines, as the UI and the rest were totally different at some point.

4

We could keep (some of) the UI things in CircleCI, no problem. Since it’s something we don’t need to maintain it (on the low-level side of things), there’s only the “another tool for some things” factor.

But: Would be nice to have… so, let’s not rush it, but at some point I’d love to get a tour of what is actually done on CircleCI, and what the UI jobs on Jenkins do – and if there is a connection of some sort between those.

5

We also moved two CircleCI tests some month ago to GitHub actions.
Currently, the quick win would be to move also the PHP unit tests to GH action, but the 3 left steps are related than to execute the e2e tests.

It is also possible to move everything. But then we cannot debug the pipeline with ssh anymore.

Screenshot 2023-05-01 at 21.37.28
Screenshot 2023-05-01 at 21.37.281628×212 13.4 KB

6

Depending on what you need to do, there are ways of making it possible to debug using ssh either for the person who triggered the action or for a specified group of users.

In this case I would recommend that we evaluate:

  1. What are the typical actions which you would do via ssh when debugging?
  2. Are these actions which result in an output which could always be generated as an artifact for after the fact debugging
  3. Is the person debugging typically the person triggering the pipeline?