The Jenkins CI/CD project has long used hundreds of GitHub repositories to allow different contributors to work on plugins and Jenkins itself in a decoupled fashion, using the popular Apache Maven project definition and build tool to deploy binaries to the Artifactory repository manager. Deployed artifacts can not only be run as is, but used as dependencies when building and testing “downstream” plugins.
Traditionally the Maven “release” plugin was run by maintainers to deploy final versions. As complex features started to be developed which spanned multiple repositories and branches, and various initiatives started to need frequent “trunk” or experimental builds of different components, it became more critical to allow multiple developers and continuous integration (CI) systems to work with artifacts without the overhead of full releases. The Maven “snapshot” system allows unreleased code to be deployed, but with very poor traceability.
To address these problems, the speaker introduced a new system “Incrementals” which integrates deeply with Maven and its dependency tooling. Once a project has been prepared, a Git commit from any branch or pull request can be built and deployed like a true release, with a deterministic version number that increments in a natural way after follow-on commits and merges. Downstream changes, Docker packages, and more can then specify an exact upstream commit.
A key aspect of the installation was to make CI builds of participating components deploy securely to a dedicated Artifactory repository, without requiring developers to hold deployment credentials—indeed allowing external contributors to publish builds just by filing pull requests against public repositories.
This talk will discuss both the technical challenges involved in tracking changes across multiple repositories in Maven; and the interplay between security, openness, and convenience when managing CI on GitHub.