Artifact Management via Nexus3

For Artifact storage we are going to be leveraging Sonatype Nexus3 (https://www.sonatype.com/nexus/repository-oss). Nexus3 will serve as:

• Maven Repositories
• proxy for maven central and other maven repos
• hosting for our own maven artifacts in a private repo
• Docker Registry
• proxy for public docker registries
• hosting for our own docker images in a private registry
• APT Proxies used primarily for docker container apt-get installs
• Definitive Media Library (DML) to be able to host other artifacts (binaries or other files) we need for our infrastructure. Those are generally install files or packages of third party applications.

Another alternative is JFrog Artifactory (https://jfrog.com/open-source/) which also offers an Open Source edition.

Infrastructure Inception Problem

As we look at our core Infrastructure guidelines specific to Availability and Reproducibility, we run into the challenge of creating the first piece of our CI/CD ecosystem. As we do not have a DML available yet (we are creating it), how can we follow the guideline of minimizing external file downloads. Remember, the goal of our infrastructure is to remain as constrained to our own network as possible and allow full re-creation from our own artifacts.

A Definitive Media Library is a secure Information Technology repository in which an organization's definitive, authorized versions of software media are stored and protected.
-Wikipedia

For this specific purpose we are going through what is considered an Infrastructure Seed process consisting of:

1. A seed installation of Nexus3 with files downloaded from the web
2. Upload of the install files to the Nexus3 DML we just created
3. Reinstall DML-based Nexus3 with files downloaded from our Seed Nexus3 installed DML
4. Replace Seed Nexus3 container with DML installed Nexus3 container

In docker terms, we use the Seed Installation steps to create and populate the container volume that we can then connect to the newly re-created container. The end result is the ability to re-create even Nexus3 from our own artifacts (aka new nexus from running nexus).

Nexus3 Installation Steps

The full Nexus3 installation is grouped into the following steps:

1. Base Images - Ubuntu and JDK base images
2. Default Test Install - non-automated installation of Nexus3 for test purposes
3. Automated Seed Install - an ephemeral but fully automated installation of Nexus3 using groovy scripts for setup
4. Infrastructure Base Images - re-installed "infrastructure grade" Base images
5. Infrastructure Install - re-installed "infrastructure grade" Nexus3 image

While we are not making use of the default install, it serves the purpose of providing context on image versioning and general approach at image build and run management. In addition, the default install also represents a simple Docker Nexus3 install without dependencies on third party docker images or container registries.