Jenkins Headless Install

The headless installation of Jenkins builds upon the base image we had previously created by adding an additional initializer called init-02.groovy. This script will automatically configure Maven to ensure its using Nexus3, add a new pipeline based job, and start it automatically. The job in question here is based on a simple Jenkinsfile pipeline.

labimagesjenkins-job-pipeline

version: '3.3'

services:
  jenkins-job-pipeline:
    image: infra/jenkins-job-pipeline:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"

FROM infra/jenkins:1.0.0

ARG ARG_ART_URL
ARG user=jenkins

# to root so we can add our file
USER root

###############################################################
## Install maven by hand
###############################################################

RUN curl $ARG_ART_URL/repository/dml/docker/maven/apache-maven-3.8.6-bin.tar.gz -o /tmp/apache-maven-3.8.6-bin.tar.gz \
    && tar xf /tmp/apache-maven-*.tar.gz -C /opt \
    && ln -s /opt/apache-maven-3.8.6 /opt/maven

ENV M2_HOME /opt/maven
ENV MAVEN_HOME /opt/maven
ENV MAVEN_OPTS="-Xss10m -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"
ENV PATH ${M2_HOME}/bin:${PATH}

# copy settings.xml for maven to link to nexus
# this is an issue if we map jenkins home as drive SIGH
# may want to do this as part of the init groovy in the future
RUN mkdir $JENKINS_HOME/.m2
COPY settings.xml $JENKINS_HOME/.m2
RUN chown -R ${user}:${user} $JENKINS_HOME/.m2

###############################################################
## init
###############################################################

# plugins for this image
COPY plugins-02.txt /tmp/plugins-02.txt
RUN /usr/local/bin/jenkins-plugin-cli.sh -f /tmp/plugins-02.txt

COPY init-02.groovy /usr/share/jenkins/ref/init.groovy.d/init-02.groovy
RUN chmod +x /usr/share/jenkins/ref/init.groovy.d/init-02.groovy
COPY pipeline-job.xml /tmp/pipeline-job.xml

# back to jenkins
USER ${user}

import hudson.model.*;
import jenkins.model.*;

import hudson.tasks.Maven.MavenInstallation;
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.util.DescribableList;

import com.cloudbees.plugins.credentials.impl.*;
import com.cloudbees.plugins.credentials.*;
import com.cloudbees.plugins.credentials.domains.*;

Thread.start {
      
      println "--> starting init-02"

      def env = System.getenv()

      /*
       * Step 1 - setup maven properly inside Global Tools
       *          Manage Jenkins > Global Tools Configuration > Maven - Maven Installations
       */

      //https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/configMavenAutoInstaller.groovy
      def mavenDesc = jenkins.model.Jenkins.instance.getExtensionList(hudson.tasks.Maven.DescriptorImpl.class)[0]
      def proplist = new DescribableList<ToolProperty<?>, ToolPropertyDescriptor>()
      def installation = new MavenInstallation("M3", "/usr/share/maven", proplist)
      mavenDesc.setInstallations(installation)
      mavenDesc.save()

      println "--> added Maven Tool"

      /*
       * Step 2 - we use maven with artifact link, so make sure to properly edit the settings xml
       *          that we pushed into the docker image.
       */

      def artifactsPassword = new File('/run/secrets/jenkins-passwords/artifacts-pwd.txt').text

      def file = new File(env['JENKINS_HOME'] + '/.m2/settings.xml')
      def newConfig = file.text
            .replace('$artifacts.user', env['artifacts.user'])
            .replace('$artifacts.password', artifactsPassword)
            .replace('$artifacts.url.mavenCentral', env['artifacts.url.mavenCentral'])
      file.text = newConfig

      println "--> changed settings.xml from env and secrets"

      /*
       * Step 3 - add the default job
       *          To get the XML, create a job and use following URL:
       *                http://localhost:8080/job/default/config.xml
       *          with [default] being the job name.
       */

      
      def jobXmlFile = new File('/tmp/pipeline-job.xml')
      def jobName = "default"
      def configXml = jobXmlFile.text
            .replace('$pipeline.scm.url', env['pipeline.scm.url'])
            .replace('$pipeline.scm.credentials', env['pipeline.scm.credentials'])
            .replace('$pipeline.scriptPath', env['pipeline.scriptPath'])
            .replace('$pipeline.branch', env['pipeline.branch'])

      def xmlStream = new ByteArrayInputStream( configXml.getBytes() )

      Jenkins.instance.createProjectFromXML(jobName, xmlStream)

      println "--> added default job based on env"
      

      /*
       * Step 4 - start the job
       */

      def job = hudson.model.Hudson.instance.getJob("default")
      hudson.model.Hudson.instance.queue.schedule(job, 0)

      println "--> running job"

      sleep 10000

      println "--> Job output available at http://localhost:8080/job/default/1/consoleText"
      
}

<?xml version='1.1' encoding='UTF-8'?>
<flow-definition plugin="workflow-job@2.41">
  <actions>
    <org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobAction plugin="pipeline-model-definition@1.9.1"/>
    <org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction plugin="pipeline-model-definition@1.9.1">
      <jobProperties/>
      <triggers/>
      <parameters/>
      <options/>
    </org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction>
  </actions>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition" plugin="workflow-cps@2.93">
    <scm class="hudson.plugins.git.GitSCM" plugin="git@4.8.2">
      <configVersion>2</configVersion>
      <userRemoteConfigs>
        <hudson.plugins.git.UserRemoteConfig>
          <url>$pipeline.scm.url</url>
          <credentialsId>$pipeline.scm.credentials</credentialsId>
        </hudson.plugins.git.UserRemoteConfig>
      </userRemoteConfigs>
      <branches>
        <hudson.plugins.git.BranchSpec>
          <name>$pipeline.branch</name>
        </hudson.plugins.git.BranchSpec>
      </branches>
      <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
      <submoduleCfg class="empty-list"/>
      <extensions/>
    </scm>
    <scriptPath>$pipeline.scriptPath</scriptPath>
    <lightweight>true</lightweight>
  </definition>
  <triggers/>
  <disabled>false</disabled>
</flow-definition>

workflow-aggregator:2.6
git:4.8.2
pipeline-utility-steps:2.10.0
http_request:1.11
nexus-artifact-uploader:2.13

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

  <servers>
    <server>
      <id>artifacts</id>
      <username>$artifacts.user</username>
      <password>$artifacts.password</password>
    </server>
  </servers>

  <mirrors>
    <mirror>
      <id>artifacts</id>
	    <name>Maven Central Public Mirror</name>
	    <url>$artifacts.url.mavenCentral</url>
	    <mirrorOf>*</mirrorOf>
    </mirror>
  </mirrors>

</settings>

docker-compose.yml
Dockerfile
init-02.groovy
pipeline-job.xml
plugins-02.txt
settings.xml

The above Docker image represents an automated job setup based on the following:

New Pipeline Project

Simple Config to pull jenkins file from git

To obtain the XML script for a specific job, the typical process for development is as follows:

Load a Base Jenkins
Create Job based in Jenkins instance with the name default
Test Job, parameters and configs and adjust based on your needs
Obtain XML by visiting http://localhost:8080/job/default/config.xml
Parameterize XML and pair with groovy script

The current Jenkins setup relies on the raw Job XML created by Jenkins. While this offers the ability to fully automate job creation it comes at the drawback of being a native descriptor of Jenkins with direct dependency links to plugin versions and other tools. Any changes made to base image for dependencies and tools may require updates on the job itself.

Note: We are in the process of reviewing Jenkins plugins that allow job management via YAML instead of the native Jenkins XML.

Building and Running

Build the Pipeline Job project.

docker-compose -f images/jenkins-job-pipeline/docker-compose.yml build

#docker login docker-private.acme.com
images/docker-push.sh infra/jenkins-job-pipeline:1.0.0 docker-private.acme.com

For testing the new image, we will be relying on a Jenkinsfile and Source Code we created in the prior step for the test project.

labcomposed

admin123

admin123

version: '3.3'

services:
  jenkins-job-pipeline:
    image: infra/jenkins-job-pipeline:1.0.0
    ports:
      - "8080:8080"
    #volumes:
    #  - jenkins-mvn-repo:/var/jenkins_home/.m2/repository
    environment:
      - artifacts.id=nexus
      - artifacts.user=admin
      - scm.id=bitbucket
      - scm.user=admin
      - artifacts.url.mavenCentral=https://nexus.acme.com/repository/maven-public/
      - pipeline.scm.url=https://bitbucket.acme.com/scm/hey/helloworld-ops.git
      - pipeline.scm.credentials=bitbucket
      - pipeline.scriptPath=jenkins/Jenkinsfile
      #- pipeline.scriptPath=jenkins/Jenkinsfile-local
      - pipeline.branch=*/master
    secrets:
      - jenkins-passwords
    networks:
      - ops-network
    extra_hosts:
      - "bitbucket.acme.com:172.22.90.1"
      - "nexus.acme.com:172.22.90.1"
networks:
  ops-network:
    name: ops-network
volumes:
  jenkins-mvn-repo:
    name: jenkins-mvn-repo
secrets:
  jenkins-passwords:
    #artifacts-pwd.txt
    #scm-pwd.txt
    #ssh-passphrase.txt
    #ssh-private-key.txt
    file: secrets/jenkins-passwords

secrets/jenkins-passwords/artifacts-pwd.txt
secrets/jenkins-passwords/scm-pwd.txt
docker-compose-jenkins-job-pipeline.yml

Add secrets in case you skipped it on the previous base install.

echo 'admin123' > composed/secrets/jenkins-passwords/scm-pwd.txt
echo 'admin123' > composed/secrets/jenkins-passwords/artifacts-pwd.txt

We always want to make sure to force-recreate to ensure its a fresh new image.

docker-compose -f composed/docker-compose-jenkins-job-pipeline.yml up --force-recreate

Base Image To Job Image Dependencies

Both the Jenkins Base image and the Jenkins Job Pipeline image work hand-in-hand. Depending on the type of projects and usages for Jenkins you may want to create more generic base images and move aspects of configurations into the Job Pipeline image; or create multiple types of base images to be used for Jenkins use cases. We try to "draw the line" on a base image that is fully configured to allow Job Development to be done against. For this specific job image, our focus is building java projects using Maven so configuration for those specific tools are done here.

Local Build Runs

One of the benefits of the standalone headless Jenkins is the ability to run a full "Jenkins" build locally on your own machine. This can easily be accomplished with the Jenkinsfile-local running mvn clean install without any artifact uploads. Simply comment in the above docker-compose to use the Jenkinsfile-local. Another benefit, when building Java projects, especially polyrepo based projects with dependencies to other projects, is the ability to persist your local mvn repo for artifact references. For example:

Project 1 v1.1.0 - build and persist artifact to local maven repo (aka mvn clean install)
Project 2 v1.2.0 - depends on Project 1 v1.1.0 but now available in your local mvn repo persisted between runs
[...]

The above docker-compose has volumes already setup but commented out.

Headless Jenkins - Test Project

Headless Jenkins - Headless Install Docker Build