PHP CI with Jenkins and Docker (Part 1)

I’ve been developing in PHP now for longer than I haven’t.

Going from using PHP as a hammer to a nail, using it to allow forms to send emails, to operating popular open source projects, to leading a team of developers in a business enterprise.

One key advice I learned from running an open source project on the SourceForge platform was “release early, release often”.

This is a mantra that I’ve always tried to stick to and its always brought me good results. As I get into more and more complex projects, both in code structure and politically, I find myself turning to tools to solve problems. One of those tools is Continuous Integration.

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

• Thought Works

For me, since I’m very process driven and love automation, Continuous Integration hits the sweet spot. It’s about defining a process and automating it. With PHP, we’re quite fortunate because there’s a ton of software tools out there that help us.

For example, using phpStorm and leveraging PHP Code Sniffer (phpcs), you can always ensure your code is to a predefined coding style.

The inspection requires PHP Code Sniffer to be properly installed in PEAR home directory as described here: http://pear.php.net/package/PHP_CodeSniffer and set up in the IDE at Settings

PHP

Code Sniffer.

But that’s not all we need to do. The point of continuous integration isn’t to just get code into your master branch or mainline code, it’s about being able to maintain quality.

“Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.” — Martin Fowler, Chief Scientist, ThoughtWorks

In order to maintain quality, we need to know what’s expected from our developers. They need standards and guidelines.

If you follow the test-driven development (TDD) software development process, then you will no doubt have unit tests too as an effort to maintain quality.

Another thing to think about is documentation. Despite the fact that the agile manifesto states “working software over comprehensive documentation”, that does not mean no documentation at all.

In PHP, the “DocBlocks” have become as integral as the code. Without them in many cases you simply won’t be able to follow the code, making it more complex and less maintainable.

There’s more, we want to know about quality of the actual code, how do we know it’s getting better or worse? What’s the solution?

If you’re working with PHP, there’s a ton of services and software solutions out there that provide “CI”, such as:

• TeamCity
• Travis CI
• Scrutinizer
• Bamboo
• GoCD
• CruiseControl

And of course, there’s Jenkins.

I’ve been working with Jenkins for quite a few years now and Jenkins 2 has brought the Pipeline plugin enabled by default, allowing you to write build instructions in “Jenkinsfiles” written in Apache Groovy.

The ‘Jenkinsfiles’ are a game changer, it means that instead of using the UI or the cumbersome CLI interface, you can “write once, use many” your build pipeline.

Jenkins has been around for many years, it’s open source, written in Java and a plays a major part of that community, as a result, it’s super powerful and there’s lots of plugins.

Today I’m going to something new, setup a Jenkins PHP CI server for use, bare metal, Docker or Kubernetes.

My goals is to give PHP developers a local Jenkins setup that they can get started with relatively easily.

First things first, I’m going to create a Dockerfile that contains everything I need to setup the environment.

Historically I’ve based on either CentOS (because it’s what I know) or Alpine Linux (because it’s what Docker recommends).

As I’ve already hit a couple of blockers with PHP and Alpine, I’m going to go with CentOS.

Before I get started, I’m going to name this project “jenkins-php-ci”.

If you’re wondering why I’m doing this when there’s already lots of similar projects, it’s because this one’s mine using my ways.

Let’s get started.

Setup

All the PHP command line tools that we need to installed are:

• phpunit - For running unit tests
• phpcs - PHP_CodeSniffer is a development tool that ensures your code remains clean and consistent
• phploc - A tool for quickly measuring the size and analyzing the structure of a PHP project
• pdepend - PHP_Depend is a metrics analysis tool for software developed in PHP.
• phpmd - PHP Mess detector, looks for possible bugs, suboptimal code, overcomplicated expressions, unused parameters, methods, properties
• phpcpd - Copy/Paste Detector (CPD) for PHP code
• phpdox - Documentation generator for PHP Code using standard technology

Here’s how that looks as a docker file:

FROM centos:7
MAINTAINER James Wade <[email protected]>

ADD http://pkg.jenkins-ci.org/redhat/jenkins.repo /etc/yum.repos.d/jenkins.repo
RUN rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key && \
    yum install -y initscripts php-intl phpunit java jenkins ant wget php-pear git
RUN pear install PHP_CodeSniffer && \
    wget https://phar.phpunit.de/phploc.phar && chmod +x phploc.phar && mv phploc.phar /usr/local/bin/phploc && \
    wget https://static.pdepend.org/php/latest/pdepend.phar --no-check-certificate && chmod +x pdepend.phar && mv pdepend.phar /usr/local/bin/pdepend && \
    wget https://static.phpmd.org/php/latest/phpmd.phar --no-check-certificate && chmod +x phpmd.phar && mv phpmd.phar /usr/local/bin/phpmd && \
    wget https://phar.phpunit.de/phpcpd.phar && chmod +x phpcpd.phar && mv phpcpd.phar /usr/local/bin/phpcpd && \
    wget http://phpdox.de/releases/phpdox.phar && chmod +x phpdox.phar && mv phpdox.phar /usr/bin/phpdox

ADD setup.sh /setup.sh
ADD example.xml /example.xml
RUN sh /setup.sh

EXPOSE 8080

CMD service jenkins start && tail -f /var/log/jenkins/jenkins.log

We need to specify a few plugins to be installed, they are:

• git - This plugin allows use of Git as a build SCM
• checkstyle - For processing PHP_CodeSniffer log files in Checkstyle format
• cloverphp - For processing PHPUnit’s Clover XML logfile
• crap4j - For processing PHPUnit’s Crap4J XML logfile
• dry - For processing phpcpd logfiles in PMD-CPD format
• htmlpublisher - For publishing documentation generated by phpDox, for instance
• jdepend - For processing PHP_Depend log files in JDepend format
• plot - For processing phploc CSV output
• pmd - For processing PHPMD log files in PMD format
• violations - For processing various log files
• warnings - For processing PHP compiler warnings in the console log
• xunit - For processing PHPUnits JUnit XML log file
• workflow-aggregator - Allows for Jenkinsfiles

In the Dockerfile above, you’ll notice that we also run a setup.sh, which sets up Jenkins and installs the plugins we want.

#!/usr/bin/env bash
set -e
service jenkins start
JENKINS_HOME=/var/lib/jenkins
printf 'Waiting for Jenkins initialise'
until [ -f ${JENKINS_HOME}/config.xml ]; do printf '.' && sleep 5; done && echo .
printf 'Waiting for Jenkins to become available'
until [ -f jenkins-cli.jar ]; do wget -q http://localhost:8080/jnlpJars/jenkins-cli.jar && printf . && sleep 5; done && echo .
until [ -f ${JENKINS_HOME}/secrets/initialAdminPassword ]; do printf '.' && sleep 5; done && echo .
ADMIN_PASS=$(cat ${JENKINS_HOME}/secrets/initialAdminPassword)
MY_CRUMB=$(curl -s -u "admin:$ADMIN_PASS" 'http://localhost:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
curl -L https://updates.jenkins-ci.org/update-center.json | sed '1d;$d' | curl -X POST -u "admin:$ADMIN_PASS" -H 'Accept: application/json' -H "$MY_CRUMB" -d @- http://localhost:8080/updateCenter/byId/default/postBack
service jenkins restart
until [[ $(curl -s -w "%{http_code}" -u "admin:$ADMIN_PASS" http://localhost:8080 -o /dev/null) == "200" ]]; do printf '.' && sleep 5; done;
until curl --silent http://localhost:8080 -u "admin:$ADMIN_PASS" | grep -v "Please wait"; do printf '.' && sleep 5; done
java -jar jenkins-cli.jar -auth admin:$ADMIN_PASS -s http://localhost:8080 install-plugin ansicolor git checkstyle cloverphp crap4j dry htmlpublisher jdepend plot pmd violations warnings xunit workflow-aggregator
cat example.xml | java -jar jenkins-cli.jar -auth admin:$ADMIN_PASS -s http://localhost:8080 create-job php-template
java -jar jenkins-cli.jar -auth admin:$ADMIN_PASS -s http://localhost:8080 reload-configuration
java -jar jenkins-cli.jar -auth admin:$ADMIN_PASS -s http://localhost:8080 safe-restart

We can build this by running the following commands:

docker-machine ip || docker-machine start
eval $(docker-machine env)
docker build . -t jenkins-php-ci

Then finally, once that’s built, it’s ready to run:

docker-machine ip || docker-machine start
eval $(docker-machine env)
docker stop jenkins-php-ci
docker rm jenkins-php-ci
docker run -p 8080:8080 --name jenkins-php-ci -d jenkins-php-ci

Optionally, you can get into bash by running:

docker exec -it jenkins-php-ci bash

This will bring up Jenkins running on your docker-machine IP on port 8080, eg:

• http://192.168.99.100:8080/

Automation

The most important part about Continuous Integration is that the job is automated.

Historically, at this point you may have done your build automation using something like Apache Ant, but now with the power of Jenkinsfile, you can do it all using the built in ‘Groovy’ language.

You can still use Ant, but I would highly recommend using and learning how to use a Jenkinsfile.

Jenkinsfiles are a bit of a black art, you’ll find that there’s a serious lack of documentation, few code examples and limited ways to verify your code apart from actually running it.

Fortunately, most of the shortcomings have already been addressed across a number of articles, so I won’t go in too much depth here, but will perhaps revisit another time.

The first thing we’re going to get our Jenkinsfile to do is linting (php -l).

Using one of the examples, and modifying it so it uses the php lint command:

node {
    stage 'Checkout'
    checkout scm
    stage "Lint"
    sh "php -l"
}

Rewind though. We need some actual code to test…

To Setup a Jenkinsfile for an example PHP project, giving us our pipeline, we’ll want to do the following:

• Persist data between container stop/restart/deletion, so we’ll need to sort out volumes and permissions
• Allow our Jenkins installation to connect to github.com
• Fork an example such as Sebastian Bergmann’s Money project
• Create a Jenkinsfile in the forked project

We’ll put this all together in an example PHP project with a Jenkinsfile to get our pipelines kicked off.

Stay tuned for part 2…