Tag Archives: drone.io

Measuring pending tests for OSS drone.io v0.4

Number of executable tests at same time are as same as registered number of docker nodes. When exceeds the limit, rest of the test will be delayed. Measuring the load is important, because long task will easily block the other many tests.

$ sqlite3 /var/lib/drone/drone.sqlite "select count(*) from jobs where job_status = 'pending';"
2

if you want to get more detailed information,

select repo_owner, repo_name, repo_private, build_branch from jobs inner join builds on jobs.job_build_id = builds.build_id inner join repos on builds.build_repo_id = repos.repo_id where job_status = 'pending'

Job status list and schema are available on github.

I created the bot that is watching the pending tests, and notify to the hipchat.

Advertisement

OSS drone.io v0.3 behind reverse proxy

OSS drone is a open source platform version of drone.io

drone v0.4 has breaking changes. Without converting .drone.yml, all tests must fail. One of mitigation is a parallel run with Name Based Virtual Host.

Apache2 setting sample is below.

<VirtualHost *:443>
...snip...
  ProxyRequests Off # NO FORWARD PROXY
  ProxyPass /api/stream/ wss://localhost:8080/api/stream/
  ProxyPassReverse /api/stream/ wss://localhost:8080/api/stream/
  ProxyPass / https://localhost:8080/
  ProxyPassReverse / https://localhost:8080
  ProxyPreserveHost On
</VirtualHost>

drone v0.3 does not support the reverse proxy. X-Forwaeded-Host cannot be interpreted. ProxyReserveHost will pass the Host header from the incoming request to the drone. Authentication callback can be return to the proper url (not localhost).

drone uses the websocket. apache2 needs mod_proxy_wstunnel
if ubuntu

sudo a2enmod proxy_wstunnel

Restarting the apache, v0.3 and v0.4 run on same host.

Custom Docker Image for Open Source Drone.io

Have you tried open source edition of drone.io?
You can get your own private CI server through the use of the open source edition. It’s really awesome.

Drone provides a lot of docker images

# these two are base images. all Drone images are built on top of these
# these are BIG (~3GB) so make sure you have a FAST internet connection
docker pull bradrydzewski/ubuntu
docker pull bradrydzewski/base

# clojure images
docker pull bradrydzewski/lein             # image: lein

** snip **

# ruby images
docker pull bradrydzewski/ruby:2.0.0       # image: ruby2.0.0
docker pull bradrydzewski/ruby:1.9.3       # image: ruby1.9.3

# scala images
docker pull bradrydzewski/scala:2.10.3     # image: scala2.10.3
docker pull bradrydzewski/scala:2.9.3      # image: scala2.9.3

https://github.com/drone/drone/blob/master/README.md#images

However, there are many variations of image, drone supports a custom image.
For example, you can create the custom images what have any distribution, any language, any version, any databases, and any libraries you want to link. It also saves the test time, because drone doesn’t cache the test environment (except cache section on .drone.yml), build all from the scratch each test phase.

This article describe how to create a docker image for drone.

Setup Development Environment

Drone uses docker for building a sandbox for each test. While you are in Linux, install from the packager. If not use Boot2Docker. On this section, uses drone development environment on Vagrant.

Before you enter the setup process, install VirtualBox and Vagrant.

$ git clone https://github.com/drone/drone.git
$ cd drone
$ vagrant up default
$ vagrant ssh default

Take a coffee break to wait for building.

$ vagrant ssh default
vagrant@precise64: $ make run

Go http://localhost:8080/install If you see the installation page, drone works fine.

Building custom image from Dockerfile

Put a Dockerfile which is instructions to build an image like this.
This is an exmaple of ruby on CentOS 6

FROM centos:centos6
MAINTAINER Kazuki Hamasaki <ashphy@ashphy.com>

WORKDIR /root
USER root

# packages
RUN yum groupinstall -y "Development Tools"
RUN yum -y install openssl openssl-devel readline-devel readline libxml2-devel libxslt-devel libyaml-devel zlib zlib-devel git --enablerepo=centosplus

# rbenv
RUN git clone https://github.com/sstephenson/rbenv.git /usr/local/rbenv
RUN git clone https://github.com/sstephenson/ruby-build.git /usr/local/rbenv/plugins/ruby-build
ADD rbenv.sh /etc/drone.d/

# ruby install
RUN bash -lc 'rbenv install 2.1.2'

# bundler
RUN bash -lc 'rbenv global 2.1.2; gem install rbenv-rehash'
RUN bash -lc 'rbenv global 2.1.2; gem install bundler'
RUN bash -lc 'rbenv rehash'

Be aware of the difference of the image conditions between official and unofficial image.

the default user for all official Drone image is the “ubuntu” user, since all build images inherit from the ubuntu cloud ISO

All other images are assumed to use the root user.

https://github.com/drone/drone/blob/4f0585bee3ea49c3dc1871a2717b289938312444/pkg/build/build.go#L440

You can use the bradrydzewski/base for your base image, but it doesn’t work by this handling of unofficial images.

If you want to run the script before a test, put the script to /etc/drone.d
The drone will kick this. Drone’s shell doesn’t load /etc/profile, .bash_profile, and .bashrc.

Building image

I recommend setting the repository name with ‘-t’ option for easy reference.

$ sudo docker build --rm -t ashphy/centos6-ruby:2.1.2 .
Sending build context to Docker daemon 75.26 kB
Sending build context to Docker daemon
Step 0 : FROM ashphy/centos6-ruby:base
 ---> d793989a9b8a
Step 1 : MAINTAINER Kazuki Hamasaki <ashphy@ashphy.com>
 ---> Using cache
 ---> 30578dc14bd9
** snip **
Step 8 : RUN bash -lc 'rbenv rehash'
 ---> Using cache
 ---> 491d71171a33
Successfully built 491d71171a33
build success

Got a special image for you.

Testing your image on the drone

Now let’s check your image if it works well on the drone.

Prepare the test repository, you want to test. Set the image name you build at the image section on .drone.yml file like below.

image: ashphy/centos6-ruby:2.1.2
script:
  - bundle install
  - rspec

Test on the drone with following command.

$ /opt/go/src/github.com/drone/drone/bin/drone build .

Uploading your image to the docker hub

If the image specified by .drone.yml is missing, drone will do ‘docker pull’, This means that the image will be downloaded from docker hub.
So that your private image that cannot be upload to public is needed to build on the drone server.

Please register in advance at https://registry.hub.docker.com/.

$ sudo docker publish ashphy/centos6-ruby

CentOS6 + ruby image

I created custom images with CentOS6 + rbenv + ruby.
Put images to centos6-ruby. And Dockerfiles to https://github.com/ashphy/centos-ruby

I hope you find it informative.