Sonarqube

The Sonarqube Dockerfile re-creation involved several additional steps and work arounds. First off, the Dockerfile at https://github.com/SonarSource/docker-sonarqube/blob/master/8/community/Dockerfile is quite similar to our approach by taking more control of the third party dependencies. With sonarqube being a code security focused company, we can understand a more stringent approach to ensure quality and operation is tested as a whole without the risk of underlying dependencies changing.

labimagessonarqube

version: '3.3'

services:
  sonarqube:
    image: infra/sonarqube:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"

FROM infra/java/openjdk-11:1.0.0

ARG ARG_ART_URL

ARG SONARQUBE_VERSION=9.6.0.59041
ARG SONARQUBE_ZIP_URL=$ARG_ART_URL/repository/dml/docker/sonarqube/sonarqube-${SONARQUBE_VERSION}.zip
ENV JAVA_HOME=/opt/java/openjdk \
    PATH="/opt/java/openjdk/bin:$PATH" \
    SONARQUBE_HOME=/opt/sonarqube \
    SONAR_VERSION="${SONARQUBE_VERSION}" \
    SQ_DATA_DIR="/opt/sonarqube/data" \
    SQ_EXTENSIONS_DIR="/opt/sonarqube/extensions" \
    SQ_LOGS_DIR="/opt/sonarqube/logs" \
    SQ_TEMP_DIR="/opt/sonarqube/temp"

RUN sed -e "s|APT_URL|${ARG_ART_URL}|" /etc/apt/sources.list.base > /etc/apt/sources.list \
    && apt-get update \
    && apt-get install -y unzip curl dos2unix gosu \
    && apt-get clean \
    && rm /etc/apt/sources.list

ARG CONTAINER_UID=1000
ARG CONTAINER_GID=1000
ARG CONTAINER_USER=sonarqube
ARG CONTAINER_GROUP=sonarqube

RUN groupadd -g ${CONTAINER_GID} ${CONTAINER_GROUP} \
    && useradd -d /home/${CONTAINER_USER} -u ${CONTAINER_UID} -g ${CONTAINER_GROUP} -m -s /bin/bash ${CONTAINER_USER}

#RUN set -eux; \
RUN set -eu; \
    mkdir --parents /opt; \
    cd /opt; \
    curl --fail --location --output sonarqube.zip --silent --show-error "${SONARQUBE_ZIP_URL}"; \
    unzip -q sonarqube.zip; \
    mv "sonarqube-${SONARQUBE_VERSION}" sonarqube; \
    rm sonarqube.zip*; \
    rm -rf ${SONARQUBE_HOME}/bin/*; \
    chown -R sonarqube:sonarqube ${SONARQUBE_HOME}; \
    # this 777 will be replaced by 700 at runtime (allows semi-arbitrary "--user" values)
    chmod -R 777 "${SQ_DATA_DIR}" "${SQ_EXTENSIONS_DIR}" "${SQ_LOGS_DIR}" "${SQ_TEMP_DIR}";
    
COPY --chown=sonarqube:sonarqube run.sh sonar.sh ${SONARQUBE_HOME}/bin/

RUN dos2unix ${SONARQUBE_HOME}/bin/run.sh \
	&& dos2unix ${SONARQUBE_HOME}/bin/sonar.sh

WORKDIR ${SONARQUBE_HOME}
ENTRYPOINT ["bin/run.sh"]
CMD ["bin/sonar.sh"]

#!/usr/bin/env bash

set -euo pipefail

declare -a sq_opts=()
set_prop_from_deprecated_env_var() {
  if [ "$2" ]; then
    sq_opts+=("-D$1=$2")
  fi
}

# if nothing is passed, assume we want to run sonarqube server
if [ "$#" == 0 ]; then
  set -- bin/sonar.sh
fi

# if first arg looks like a flag, assume we want to run sonarqube server with flags
if [ "${1:0:1}" = '-' ]; then
    set -- bin/sonar.sh "$@"
fi

if [[ "$1" = 'bin/sonar.sh' ]]; then
    chown -R "$(id -u):$(id -g)" "${SQ_DATA_DIR}" "${SQ_EXTENSIONS_DIR}" "${SQ_LOGS_DIR}" "${SQ_TEMP_DIR}" 2>/dev/null || :
    chmod -R 700 "${SQ_DATA_DIR}" "${SQ_EXTENSIONS_DIR}" "${SQ_LOGS_DIR}" "${SQ_TEMP_DIR}" 2>/dev/null || :

    # Allow the container to be started with `--user`
    if [[ "$(id -u)" = '0' ]]; then
        chown -R sonarqube:sonarqube "${SQ_DATA_DIR}" "${SQ_EXTENSIONS_DIR}" "${SQ_LOGS_DIR}" "${SQ_TEMP_DIR}"
        exec gosu sonarqube "$0" "$@"
    fi

    #
    # Deprecated way to pass settings to SonarQube that will be removed in future versions.
    # Please use environment variables (https://docs.sonarqube.org/latest/setup/environment-variables/)
    # instead to customize SonarQube.
    #
    while IFS='=' read -r envvar_key envvar_value
    do
        if [[ "$envvar_key" =~ sonar.* ]] || [[ "$envvar_key" =~ ldap.* ]]; then
            sq_opts+=("-D${envvar_key}=${envvar_value}")
        fi
    done < <(env)

    #
    # Deprecated environment variable mapping that will be removed in future versions.
    # Please use environment variables from https://docs.sonarqube.org/latest/setup/environment-variables/
    # instead of using these 4 environment variables below.
    #
    set_prop_from_deprecated_env_var "sonar.jdbc.username" "${SONARQUBE_JDBC_USERNAME:-}"
    set_prop_from_deprecated_env_var "sonar.jdbc.password" "${SONARQUBE_JDBC_PASSWORD:-}"
    set_prop_from_deprecated_env_var "sonar.jdbc.url" "${SONARQUBE_JDBC_URL:-}"
    set_prop_from_deprecated_env_var "sonar.web.javaAdditionalOpts" "${SONARQUBE_WEB_JVM_OPTS:-}"
    if [ ${#sq_opts[@]} -ne 0 ]; then
        set -- "$@" "${sq_opts[@]}"
    fi
fi

exec "$@"

#!/usr/bin/env bash
exec java -jar lib/sonar-application-"${SONAR_VERSION}".jar -Dsonar.log.console=true "$@"

docker-compose.yml
Dockerfile
run.sh
sonar.sh

While we are removing the signature verification, remember that we sha verify the content of the file during our DML upload. Also, since it is hidden inside the run.sh, we are using gosu instead of su-exec since su-exec is specific to Alpine.

Blue Box of Information: gosu

You can read more about https://github.com/tianon/gosu here, especially their Warning. In a nutshell, we may want to allow the container to be started with --user thus using gosu as a su docker equivalent when starting the process.

Installation

The automated installation of Sonarqube is quite similar to our Bitbucket installation regarding Database and application.

labcomposed

version: '3.3'

services:
  db-script-sqlserver:
    image: infra/db-script-sqlserver:1.0.0
    environment:
      - SQL_SERVER_HOST=d1i-doc-dbss01
      - SA_PASSWORD=yourStrong(!)Password
    volumes:
      - ./db/sonarqube-sqlserver-init.sql:/init/script.sql
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network


CREATE DATABASE sonar
GO
USE sonar
GO
ALTER DATABASE sonar SET ALLOW_SNAPSHOT_ISOLATION ON
GO
ALTER DATABASE sonar SET READ_COMMITTED_SNAPSHOT ON
GO
ALTER DATABASE sonar COLLATE SQL_Latin1_General_CP1_CS_AS
GO
SET NOCOUNT OFF
GO
USE master
GO
CREATE LOGIN sonaruser WITH PASSWORD=N'ZeCodeIzGood', DEFAULT_DATABASE=sonar, CHECK_EXPIRATION=OFF, CHECK_POLICY=OFF
GO
ALTER AUTHORIZATION ON DATABASE::sonar TO sonaruser
GO

docker-compose-sonarqube-sqlserver-init.yml
sonarqube-sqlserver-init.sql

The initial database setup is simply using our prebuild SQL Server script runner with a new initialization file to create the sonarqube database.

labcomposed

version: '3.3'

services:
  db-script-sqlserver:
    image: infra/db-script-sqlserver:1.0.0
    environment:
      - SQL_SERVER_HOST=d1i-doc-dbss01
      - SA_PASSWORD=yourStrong(!)Password
    volumes:
      - ./db/sonarqube-sqlserver-post-init.sql:/init/script.sql
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network

USE sonar
GO
update users set reset_password = 0, crypted_password = '$2a$12$wyLOlQaHXfTC0mfUExf0FuDHU/xURMYOhyRkKWB9bgca7vigVg.LG', hash_method = 'BCRYPT', salt = null where login = 'admin'
GO

docker-compose-sonarqube-sqlserver-post-init.yml
sonarqube-sqlserver-post-init.sql

After the initial installation of sonarqube we had to create a bit of a workaround to ensure headless interactions if so desired. By default Sonarqube creates a username admin with password admin. Further this password requires to be changed on first login. We initially attempted to simply change the reset password flag however the installation code was smart enough to revert the reset flag if the password remained "admin". A reference in their documentation at https://docs.sonarqube.org/latest/instance-administration/security/ provided a hint, describing how to reset the default password in case you forgot. See the below snippet:

update users
    set crypted_password = '&#36;2a&#36;12&#36;uCkkXmhW5ThVK8mpBvnXOOJRLd64LJeHTeCkSuB3lfaR2N0AYBaSi',
    salt=null,
    hash_method='BCRYPT',
    reset_password='true'
where login = 'admin'

Looking at the password, we can infer hash and rounds. So a 12 round BCRYPT for our admin123 password should allow us to ensure we can login without being forced to change the initial admin password and of course use the same admin:admin123 for our lab as we had done for all other apps thus far. Our version inside the sonarqube-sqlserver-post-init.sql is quite similar:

USE sonar
GO
update users
    set reset_password = 0,
    crypted_password = '&#36;2a&#36;12&#36;wyLOlQaHXfTC0mfUExf0FuDHU/xURMYOhyRkKWB9bgca7vigVg.LG',
    hash_method = 'BCRYPT',
    salt = null
where login = 'admin'
GO

Building and Running

First the sonarqube image.

docker-compose -f images/sonarqube/docker-compose.yml build

#docker login docker-private.acme.com
images/docker-push.sh infra/sonarqube:1.0.0 docker-private.acme.com

We will be reusing the same DB instance that is currently used by Bitbucket to create the sonarqube Database.

docker-compose -f composed/docker-compose-sonarqube-sqlserver-init.yml up

Now we can start sonarqube for the initial installation. This is done via the following compose:

labcomposed

version: '3.3'

services:
  sonarqube:
    image: infra/sonarqube:1.0.0
    ports:
      - 9000:9000
    hostname: d1i-doc-sonar01
    environment: 
      - SONAR_JDBC_URL=jdbc:sqlserver://d1i-doc-dbss01:1433;databaseName=sonar;encrypt=true;trustServerCertificate=true;
      - SONAR_JDBC_USERNAME=sonaruser
      - SONAR_JDBC_PASSWORD=ZeCodeIzGood
    volumes:
      - sonarqube-data:/opt/sonarqube/data
      - sonarqube-extensions:/opt/sonarqube/extensions
      - sonarqube-logs:/opt/sonarqube/logs
      - sonarqube-temp:/opt/sonarqube/temp
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network
volumes:
  sonarqube-data:
    name: sonarqube-data
  sonarqube-extensions:
    name: sonarqube-extensions
  sonarqube-logs:
    name: sonarqube-logs
  sonarqube-temp:
    name: sonarqube-temp

docker-compose-sonarqube.yml

Run it.

docker-compose -f composed/docker-compose-sonarqube.yml up

This process takes approx 3-4 minutes until the following message appears SonarQube is up. You can verify that it was installed by visiting http://localhost:9000. Once completed stop the process.

Startup Issues

Depending on your OS, your sonarqube process may exit during startup with the following error:

ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /opt/sonarqube/logs/sonarqube.log

The solution is dependent on where and how you run it. As we are focused here on docker desktop, for Windows do the following via powershell and restart docker desktop.

wsl -d docker-desktop
sysctl -w vm.max_map_count=262144

Important This needs to be redone for every startup cycle as it does not persist. https://github.com/docker/for-win/issues/5202

The final setup is our post installation credentials changes to ensure we have our default admin user with password admin123 for further automation.

docker-compose -f composed/docker-compose-sonarqube-sqlserver-post-init.yml up

As we will be running this behind our nginx as well alongside the other applications, we have to make some minor adjustments as we need to make it compatible with Sonarqube Scanner first.

Sonarqube Scanner with a Self Signed Certificate

While you can easily run the web interface behind our wild card self-signed certificate, running the sonarqube scanner against this certificate will bring a couple of challenges. As such we need to re-generate a new self signed certificate for Sonarqube specifically. This is done via another SSL generator as we need to generate the cert with the SAN extension.

Blue Box of Information: Sonarqube Self Signed Certificate Challenges

If you were to run the sonarqube scanner while using our wild card cert, you will get the following error.

Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.8.0.2131:sonar (default-cli) on project [project-name]:
Unable to execute SonarScanner analysis:
Fail to get bootstrap index from server:
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
-> [Help 1]

Sonarqube scanner uses their own jdk to ensure it is isolated from whatever jdk is installed on the host. Part of the JDK includes the common certificate file cacerts which does not include our self signed wildcard cert. However, when adding our own certificate to the cacerts and injecting it as a java option you will get the following error:

# -Djavax.net.ssl.trustStore=your-cacerts -Djavax.net.ssl.trustStorePassword=changeit

Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.8.0.2131:sonar (default-cli) on project [project-name]:
Unable to execute SonarScanner analysis:
Fail to get bootstrap index from server: Hostname sonarqube.acme.com not verified:
    certificate: sha256/[cert-sha]
    DN: CN=*.acme.com, O=XXXX, L=XXXX, ST=XXXX, C=XX
    subjectAltNames: []
-> [Help 1]

As a good engineer, you listen to the error message and wonder why its complaining, maybe lets use a named cert. So you re-generate a named cert, regenerate your cacerts and run it again and you end up with your original error eventhough you have your hostname in the CN or Common Name.

Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.8.0.2131:sonar (default-cli) on project infra-discovery:
Unable to execute SonarScanner analysis:
Fail to get bootstrap index from server: Hostname sonarqube.acme.com not verified:
    certificate: sha256/[cert-sha]
    DN: CN=sonarqube.acme.com, O=XXXX, L=XXXX, ST=XXXX, C=XX
    subjectAltNames: []
-> [Help 1]

At this point, you are thoroughly confused. Long story short, one of the scanners downstream dependencies OkHttp made a change in its verification process resulting in a much more complicated self-signed certificate process than you had anticipated.

Solution is creation of a self signed certificate that uses the san extensions, or Subject Alternate Name for the same host.

The ssl-cert-san image is doing the same as our ssl-cert image we used during the wildcard setups but adding the new san extension inside the init.sh.

labtoolsssl-cert-san

version: '3.3'

services:
  ssl-cert-san:
    image: tools/ssl-cert-san:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"

FROM infra/ubuntu/focal:1.0.0

ARG ARG_ART_URL

RUN sed -e "s|APT_URL|${ARG_ART_URL}|" /etc/apt/sources.list.base > /etc/apt/sources.list \
    && apt-get update \
    && apt-get install -y openssl dos2unix \
    && apt-get clean \
    && rm /etc/apt/sources.list

COPY init.sh /init/init.sh
RUN chmod +x /init/init.sh \
    && dos2unix /init/init.sh

ENTRYPOINT ["/init/init.sh"]

#!/bin/bash

# fail if anything errors
set -e

openssl req -x509 -newkey rsa:4096 \
    -subj "/C=XX/ST=XXXX/L=XXXX/O=XXXX/CN=${CERT_CN}" \
    -keyout "/ssl/${KEY_NAME}" \
    -out "/ssl/${CERT_NAME}" \
    -days 365 -nodes -sha256 \
    -extensions san \
    -config <( \
        echo "[req]"; \
        echo "distinguished_name=req"; \
        echo "[san]"; \
        echo "subjectAltName=DNS:${CERT_CN}")

docker-compose.yml
Dockerfile
init.sh

Next, we need to generate a new cacerts for the scanners java process to ensure our new certificate is included. For the purpose of this lab, we will be using our OpenJdks existing cacerts file and simply add ours. Alternatively, you can create a new keystore and only add our sonarqube certificate there.

labtoolsssl-cert-cacerts

version: '3.3'

services:
  ssl-cacerts:
    image: tools/ssl-cacerts:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"

FROM infra/java/openjdk-8:1.0.0

ARG ARG_ART_URL

RUN sed -e "s|APT_URL|${ARG_ART_URL}|" /etc/apt/sources.list.base > /etc/apt/sources.list \
    && apt-get update \
    && apt-get install -y dos2unix \
    && apt-get clean \
    && rm /etc/apt/sources.list

COPY init.sh /init/init.sh
RUN chmod +x /init/init.sh \
    && dos2unix /init/init.sh

ENTRYPOINT ["/init/init.sh"]

#!/bin/bash

# fail if anything errors
set -e

# get cacerts from latest openjdk (whichever we using in FROM)
# if you jsut want yours skip this and change from to base ubuntu
cp /opt/java/openjdk/jre/lib/security/cacerts /tmp

# add the one we injecting
cp /ssl/${CERT_NAME} /tmp

# add to keystore (either add if exists from cacerts or create if new one)
cd /tmp
keytool -keystore cacerts -storepass changeit -noprompt -trustcacerts -importcert -alias ${CACERT_ALIAS} -file /ssl/${CERT_NAME}

# bring it back external folder
cp cacerts /ssl/cacerts

docker-compose.yml
Dockerfile
init.sh

We run this via the following two composes.

labcomposed

version: '3.3'

services:
  ssl-cert-san:
    image: tools/ssl-cert-san:1.0.0
    volumes:
      - ./ssl:/ssl
    environment:
      - CERT_CN=sonarqube.acme.com
      - KEY_NAME=key-sonarqube.pem
      - CERT_NAME=cert-sonarqube.pem
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network

version: '3.3'

services:
  ssl-cacerts:
    image: tools/ssl-cacerts:1.0.0
    volumes:
      - ./ssl:/ssl
    environment:
      - CACERT_ALIAS=SonarQube
      - CERT_NAME=cert-sonarqube.pem
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network

docker-compose-ssl-cert-san-sonarqube.yml
docker-compose-ssl-cacerts-sonarqube.yml

Lets build and run those new SSL tools.

# actual builds and run
docker-compose -f tools/ssl-cert-san/docker-compose.yml build
docker-compose -f tools/ssl-cacerts/docker-compose.yml build

docker-compose -f composed/docker-compose-ssl-cert-san-sonarqube.yml up
docker-compose -f composed/docker-compose-ssl-cacerts-sonarqube.yml up

We are now ready to add Sonarqube behind our existing nginx alongside the other CI/CD "always-on" processes.

For the nginx, we will be adding the new certificates needed for sonar as well.

labcomposed

server {
    listen       *:443 ssl;
    server_name  "nexus.acme.com";

    ssl_certificate           /etc/nginx/external/cert.pem;
    ssl_certificate_key       /etc/nginx/external/key.pem;

    client_max_body_size 1G;

    location / {
        proxy_pass         http://d1i-doc-nexus01:8081;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
	    proxy_set_header   X-Forwarded-Proto https;
    }
}

server {
    listen       *:443 ssl;
    server_name  "docker.acme.com";

    ssl_certificate           /etc/nginx/external/cert.pem;
    ssl_certificate_key       /etc/nginx/external/key.pem;

    client_max_body_size 1G;

    location / {
        proxy_pass         http://d1i-doc-nexus01:8082;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
	    proxy_set_header   X-Forwarded-Proto https;
    }
}



server {
    listen       *:443 ssl;
    server_name  "docker-private.acme.com";

    ssl_certificate           /etc/nginx/external/cert.pem;
    ssl_certificate_key       /etc/nginx/external/key.pem;

    client_max_body_size 1G;

    location / {
        proxy_pass         http://d1i-doc-nexus01:8083;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
	    proxy_set_header   X-Forwarded-Proto https;
    }
}

server {
    listen       *:443 ssl;
    server_name  "bitbucket.acme.com";

    ssl_certificate           /etc/nginx/external/cert.pem;
    ssl_certificate_key       /etc/nginx/external/key.pem;

    client_max_body_size 1G;

    location / {
        proxy_pass         http://d1i-doc-bitbucket01:7990;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto https;
    }
}

server {
    listen       *:443 ssl;
    server_name  "sonarqube.acme.com";

    ssl_certificate           /etc/nginx/external/cert-sonarqube.pem;
    ssl_certificate_key       /etc/nginx/external/key-sonarqube.pem;

    client_max_body_size 1G;

    location / {
        proxy_pass         http://d1i-doc-sonar01:9000;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto https;
    }
}

version: '3.3'

services:
  nginx:
    image: infra/nginx/base:1.0.0
    volumes:
      - ./ssl/cert.pem:/etc/nginx/external/cert.pem
      - ./ssl/key.pem:/etc/nginx/external/key.pem
      - ./nginx/nexus.conf:/etc/nginx/conf.d/nexus.conf
      - ./nginx/bitbucket.conf:/etc/nginx/conf.d/bitbucket.conf
      - ./ssl/cert-sonarqube.pem:/etc/nginx/external/cert-sonarqube.pem
      - ./ssl/key-sonarqube.pem:/etc/nginx/external/key-sonarqube.pem
      - ./nginx/sonarqube.conf:/etc/nginx/conf.d/sonarqube.conf
    ports:
      - 443:443
    hostname: d1i-doc-ng01
    networks:
      ops-network:
        ipv4_address: 172.22.90.1
networks:
  ops-network:
    name: ops-network
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.22.90.0/16

nginx/nexus.conf
nginx/bitbucket.conf
nginx/sonarqube.conf
docker-compose-nginx-nexus-bitbucket-sonarqube.yml

Create, and start it.

# start up sonarqube
docker-compose -f composed/docker-compose-sonarqube.yml up --no-start
docker-compose -f composed/docker-compose-sonarqube.yml start

# replace nginx with sonarqube add on
docker-compose -f composed/docker-compose-nginx-nexus-bitbucket.yml stop
docker-compose -f composed/docker-compose-nginx-nexus-bitbucket-sonarqube.yml up --no-start
docker-compose -f composed/docker-compose-nginx-nexus-bitbucket-sonarqube.yml start

You can now login to https://sonarqube.acme.com with the admin:admin123 username and password.

Test Project

As we had previously created a test project for our Jenkins testing, we want to extend this same behavior for Sonarqube as well. Similar to the bitbucket project and repository creation, we have automated the sonarqube project setup and creation of the Sonarqube token that we want to pass into a new Jenkinsfile that will publish results. All this is done via the Sonarqube API on our installed and running container.

labtoolssonarqube-helloworld

version: '3.3'

services:
  sonarqube-helloworld:
    image: infra/tools/sonarqube-helloworld:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"
    volumes:
      # this is mounting into the root/composed folder
      - ./../../composed/secrets/jenkins-passwords:/init/pwd
    networks:
      - ops-network
networks:
  ops-network:
    name: ops-network

FROM infra/ubuntu/focal:1.0.0

ARG ARG_ART_URL

RUN sed -e "s|APT_URL|${ARG_ART_URL}|" /etc/apt/sources.list.base > /etc/apt/sources.list \
    && apt-get update \
    && apt-get install -y curl dos2unix jq \
    && apt-get clean \
    && rm /etc/apt/sources.list

COPY init.sh /init/init.sh
RUN chmod +x /init/init.sh \
    && dos2unix /init/init.sh

ENTRYPOINT ["/init/init.sh"]

#!/bin/bash

# fail if anything errors
set -e

##########################
## Create Projects
##########################

# we can re-run to delete it all
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/delete?project=com.acme:infra-discovery'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/delete?project=com.acme:infra-gateway'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/delete?project=com.acme:helloworld-api'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/delete?project=com.acme:helloworld-web'

# revoke token
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/user_tokens/revoke?name=com.acme'

# Create Project
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/create?project=com.acme:infra-discovery&name=com.acme:infra-discovery'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/create?project=com.acme:infra-gateway&name=com.acme:infra-gateway'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/create?project=com.acme:helloworld-api&name=com.acme:helloworld-api'
curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/projects/create?project=com.acme:helloworld-web&name=com.acme:helloworld-web'

# now generate a token
SONAR_TOKEN=$(curl -u admin:admin123 -X POST 'http://d1i-doc-sonar01:9000/api/user_tokens/generate?name=com.acme' | jq -r '.token')


##########################
## create sonar env file
##########################

echo "${SONAR_TOKEN}" > /init/pwd/sonar-token.txt

docker-compose.yml
Dockerfile
init.sh

Important Note: We are mounting this image directly into the secrets folder in our lab folder structure to be able to automatically add the sonar token to be used later in our jenkins auto-installation. This can also be accomplished manually by simply creating a local mount and then copying the file after the fact.

The sonar token is created via the sonarqube API inside the init.sh script. Alternatively, we can output this into another location and then relocate it on the host itself.

SONAR_TOKEN=$(curl -u admin:admin123 -X POST 'http://host.docker.internal:9000/api/user_tokens/generate?name=com.acme' | jq -r '.token')

echo "${SONAR_TOKEN}" > /init/pwd/sonar-token.txt

Build and run in.

docker-compose -f tools/sonarqube-helloworld/docker-compose.yml build
docker-compose -f tools/sonarqube-helloworld/docker-compose.yml up

Sonarqube Jenkins

As we isolate responsibilities for each Jenkins installation, we will be adding a sonar specific Jenkins image by extending the Job Pipeline image we have already created. We simply adding our sonar credentials here.

labimagesjenkins-job-pipeline-sonarqube

version: '3.3'

services:
  jenkins-job-pipeline-sonarqube:
    image: infra/jenkins-job-pipeline-sonarqube:1.0.0
    build:
      context: .
      network: host
      args:
        ARG_ART_URL: http://d1i-doc-ngbuild:3001
      extra_hosts:
        - "d1i-doc-ngbuild:172.22.90.2"

FROM infra/jenkins-job-pipeline:1.0.0

ARG user=jenkins

# to root so we can add our file
USER root

###############################################################
## init
###############################################################

COPY init-03.groovy /usr/share/jenkins/ref/init.groovy.d/init-03.groovy
RUN chmod +x /usr/share/jenkins/ref/init.groovy.d/init-03.groovy

# back to jenkins
USER ${user}

import hudson.model.*;
import jenkins.model.*;

import hudson.tasks.Maven.MavenInstallation;
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.util.DescribableList;

import com.cloudbees.plugins.credentials.impl.*;
import com.cloudbees.plugins.credentials.*;
import com.cloudbees.plugins.credentials.domains.*;

import hudson.util.Secret;
import org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl;

Thread.start {
      
      println "--> starting init-03"

      /*
       * Step 1 - add sonar token
       */

      def sonarToken = new File('/run/secrets/jenkins-passwords/sonar-token.txt').text;

      // may not need that credential here
      Credentials artifactCredentials = (Credentials) new StringCredentialsImpl(CredentialsScope.GLOBAL, "sonar-token", "sonar-token", Secret.fromString(sonarToken))
      SystemCredentialsProvider.getInstance().getStore().addCredentials(Domain.global(), artifactCredentials)
      
      println "--> added sonar string credentials"

      /*
       * Step 2 - add sonar credentials for api calls
       */

      def env = System.getenv()

      def sonarPassword = new File('/run/secrets/jenkins-passwords/sonar-pwd.txt').text;

      Credentials sonarCredentials = (Credentials) new UsernamePasswordCredentialsImpl(CredentialsScope.GLOBAL, "sonar", "sonar", env['sonar.user'], sonarPassword)
      SystemCredentialsProvider.getInstance().getStore().addCredentials(Domain.global(), sonarCredentials)

      println "--> added sonar credentials from env and secrets"

}

docker-compose.yml
Dockerfile
init-03.groovy

Now running with the new credentials mounted as secrets.

labcomposed

admin123

admin123

admin123

n/a

version: '3.3'

services:
  jenkins-job-pipeline-sonarqube:
    image: infra/jenkins-job-pipeline-sonarqube:1.0.0
    ports:
      - "8080:8080"
    volumes:
      - ./ssl/cacerts:/tmp/ssl/cacerts
      - ./local/:/var/jenkins_home/.m2/repository
    environment:
      - artifacts.id=nexus
      - artifacts.user=admin
      - scm.id=bitbucket
      - scm.user=admin
      - artifacts.url.mavenCentral=https://nexus.acme.com/repository/maven-public/
      - pipeline.scm.url=https://bitbucket.acme.com/scm/hey/helloworld-ops.git
      - pipeline.scm.credentials=bitbucket
      - pipeline.scriptPath=jenkins/Jenkinsfile-sonar
      - pipeline.branch=*/master
      - sonar.user=admin
      - SONAR_SCANNER_OPTS=-Djavax.net.ssl.trustStore=/tmp/ssl/cacerts -Djavax.net.ssl.trustStorePassword=changeit
    secrets:
      - jenkins-passwords
    networks:
      - ops-network
    extra_hosts:
      - "bitbucket.acme.com:172.22.90.1"
      - "nexus.acme.com:172.22.90.1"
      - "sonarqube.acme.com:172.22.90.1"
networks:
  ops-network:
    name: ops-network
volumes:
  jenkins-mvn-repo:
    name: jenkins-mvn-repo
secrets:
  jenkins-passwords:
    #artifacts-pwd.txt
    #scm-pwd.txt
    #ssh-passphrase.txt
    #ssh-private-key.txt
    #sonar-pwd.txt
    #sonar-token.txt
    file: secrets/jenkins-passwords

secrets/jenkins-passwords/artifacts-pwd.txt
secrets/jenkins-passwords/scm-pwd.txt
secrets/jenkins-passwords/sonar-password.txt
secrets/jenkins-passwords/sonar-token.txt
docker-compose-jenkins-job-pipeline-sonarqube.yml

docker-compose -f images/jenkins-job-pipeline-sonarqube/docker-compose.yml build

#docker login docker-private.acme.com
images/docker-push.sh infra/jenkins-job-pipeline-sonarqube:1.0.0 docker-private.acme.com

# make sure to also add the new pwd
echo 'admin123' > composed/secrets/jenkins-passwords/sonar-pwd.txt

# start it up
docker-compose -f composed/docker-compose-jenkins-job-pipeline-sonarqube.yml up --force-recreate

Once completed, you will have all test projects verified and measured via sonarqube.

Sonarqube applied to our Test Projects

Headless Jenkins - Headless Install Docker Build

Command Line Only