OSS drone.io v0.3 behind reverse proxy

OSS drone is a open source platform version of drone.io

drone v0.4 has breaking changes. Without converting .drone.yml, all tests must fail. One of mitigation is a parallel run with Name Based Virtual Host.

Apache2 setting sample is below.

<VirtualHost *:443>
...snip...
  ProxyRequests Off # NO FORWARD PROXY
  ProxyPass /api/stream/ wss://localhost:8080/api/stream/
  ProxyPassReverse /api/stream/ wss://localhost:8080/api/stream/
  ProxyPass / https://localhost:8080/
  ProxyPassReverse / https://localhost:8080
  ProxyPreserveHost On
</VirtualHost>

drone v0.3 does not support the reverse proxy. X-Forwaeded-Host cannot be interpreted. ProxyReserveHost will pass the Host header from the incoming request to the drone. Authentication callback can be return to the proper url (not localhost).

drone uses the websocket. apache2 needs mod_proxy_wstunnel
if ubuntu

sudo a2enmod proxy_wstunnel

Restarting the apache, v0.3 and v0.4 run on same host.

Advertisement

Reset the webhooks for Drone – Open Source CI Server

Drone is a useful Open Source CI Server. You can host the private and dedicate CI environement.

Once you delete or modify the webhooks on the github (enterprise), never reset the setting from Web UI. NOT YET IMPLEMENTED.

But, we have APIs.

First of all, get the api key from https://your.ci.server/account/profile

my_profile

Let’s curl

$ curl -X POST https://your.ci.server/api/repos/github.com/ashphy/baku?access_token=<INSERT YOUR API KEY>

API Reference: http://readme.drone.io/api/repos/post/

Deactivate is also available.

Custom Docker Image for Open Source Drone.io

Have you tried open source edition of drone.io?
You can get your own private CI server through the use of the open source edition. It’s really awesome.

Drone provides a lot of docker images

# these two are base images. all Drone images are built on top of these
# these are BIG (~3GB) so make sure you have a FAST internet connection
docker pull bradrydzewski/ubuntu
docker pull bradrydzewski/base

# clojure images
docker pull bradrydzewski/lein             # image: lein

** snip **

# ruby images
docker pull bradrydzewski/ruby:2.0.0       # image: ruby2.0.0
docker pull bradrydzewski/ruby:1.9.3       # image: ruby1.9.3

# scala images
docker pull bradrydzewski/scala:2.10.3     # image: scala2.10.3
docker pull bradrydzewski/scala:2.9.3      # image: scala2.9.3

https://github.com/drone/drone/blob/master/README.md#images

However, there are many variations of image, drone supports a custom image.
For example, you can create the custom images what have any distribution, any language, any version, any databases, and any libraries you want to link. It also saves the test time, because drone doesn’t cache the test environment (except cache section on .drone.yml), build all from the scratch each test phase.

This article describe how to create a docker image for drone.

Setup Development Environment

Drone uses docker for building a sandbox for each test. While you are in Linux, install from the packager. If not use Boot2Docker. On this section, uses drone development environment on Vagrant.

Before you enter the setup process, install VirtualBox and Vagrant.

$ git clone https://github.com/drone/drone.git
$ cd drone
$ vagrant up default
$ vagrant ssh default

Take a coffee break to wait for building.

$ vagrant ssh default
vagrant@precise64: $ make run

Go http://localhost:8080/install If you see the installation page, drone works fine.

Building custom image from Dockerfile

Put a Dockerfile which is instructions to build an image like this.
This is an exmaple of ruby on CentOS 6

FROM centos:centos6
MAINTAINER Kazuki Hamasaki <ashphy@ashphy.com>

WORKDIR /root
USER root

# packages
RUN yum groupinstall -y "Development Tools"
RUN yum -y install openssl openssl-devel readline-devel readline libxml2-devel libxslt-devel libyaml-devel zlib zlib-devel git --enablerepo=centosplus

# rbenv
RUN git clone https://github.com/sstephenson/rbenv.git /usr/local/rbenv
RUN git clone https://github.com/sstephenson/ruby-build.git /usr/local/rbenv/plugins/ruby-build
ADD rbenv.sh /etc/drone.d/

# ruby install
RUN bash -lc 'rbenv install 2.1.2'

# bundler
RUN bash -lc 'rbenv global 2.1.2; gem install rbenv-rehash'
RUN bash -lc 'rbenv global 2.1.2; gem install bundler'
RUN bash -lc 'rbenv rehash'

Be aware of the difference of the image conditions between official and unofficial image.

the default user for all official Drone image is the “ubuntu” user, since all build images inherit from the ubuntu cloud ISO

All other images are assumed to use the root user.

https://github.com/drone/drone/blob/4f0585bee3ea49c3dc1871a2717b289938312444/pkg/build/build.go#L440

You can use the bradrydzewski/base for your base image, but it doesn’t work by this handling of unofficial images.

If you want to run the script before a test, put the script to /etc/drone.d
The drone will kick this. Drone’s shell doesn’t load /etc/profile, .bash_profile, and .bashrc.

Building image

I recommend setting the repository name with ‘-t’ option for easy reference.

$ sudo docker build --rm -t ashphy/centos6-ruby:2.1.2 .
Sending build context to Docker daemon 75.26 kB
Sending build context to Docker daemon
Step 0 : FROM ashphy/centos6-ruby:base
 ---> d793989a9b8a
Step 1 : MAINTAINER Kazuki Hamasaki <ashphy@ashphy.com>
 ---> Using cache
 ---> 30578dc14bd9
** snip **
Step 8 : RUN bash -lc 'rbenv rehash'
 ---> Using cache
 ---> 491d71171a33
Successfully built 491d71171a33
build success

Got a special image for you.

Testing your image on the drone

Now let’s check your image if it works well on the drone.

Prepare the test repository, you want to test. Set the image name you build at the image section on .drone.yml file like below.

image: ashphy/centos6-ruby:2.1.2
script:
  - bundle install
  - rspec

Test on the drone with following command.

$ /opt/go/src/github.com/drone/drone/bin/drone build .

Uploading your image to the docker hub

If the image specified by .drone.yml is missing, drone will do ‘docker pull’, This means that the image will be downloaded from docker hub.
So that your private image that cannot be upload to public is needed to build on the drone server.

Please register in advance at https://registry.hub.docker.com/.

$ sudo docker publish ashphy/centos6-ruby

CentOS6 + ruby image

I created custom images with CentOS6 + rbenv + ruby.
Put images to centos6-ruby. And Dockerfiles to https://github.com/ashphy/centos-ruby

I hope you find it informative.

Modern Pubsubhubbub Implementation

I modified pubsubhubbub implementation. No longer original impl does not work, because it has no update from 2011.

https://github.com/ashphy/pubsubhubbub

My forked impl will works fine with Python 2.7 and Google App Engine for Python 1.8.9.

I’m unfamiliar with Python and GAE. Any improvements are welcome.

How to Run the servers

$ git clone https://code.google.com/p/pubsubhubbub/

Hub

$ dev_appserver.py hub

Publisher

$ dev_appserver.py --port=8081 --admin_port=8001 publisher

Subscriber

$ dev_appserver.py --port=8082 --admin_port=8002 subscriber

[Vagrant] object failure: RPC_S_SERVER_UNAVAILABLE error on Windows7

If you see error like this on Vagrant 1.2.2 and VirtualBox 4.2.14, when you do ‘vagrant up’


C:\Users\kazuki-h>vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'hogehoge'...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["import", "C:/Users/kazuki-h/.vagrant.d/boxes/hogehoge/virtualbox/box.ovf"]

Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting C:\Users\kazuki-h\.vagrant.d\boxes\hogehoge\virtualbox\box.ovf...

OK.
0%...
Progress object failure: RPC_S_SERVER_UNAVAILABLE 0x800706BA
Downgrade VirtualBox to 4.2.12, it will works fine.
It could raise again on Mac OS X or Linux, because VirtualBox’s changes might be directly responsible for this error.

Refarence:

Edit

This problem has been fixed in the version 4.2.16.

rbenv does not work in aptana terminal for Mac OS X

rbenv is the best solution to to pick a ruby version for your specific environment.
However, even if rbenv has been installed successfully, the “Terminal” in Aptana Studio cannot stop choose the system default ruby version.

Because aptana terminal does not load ~/.bash_profile

 

Fix

My Environments

  • Mac OS X Mountain Lion
  • Aptana Studio 3
  • rbenv 0.4.0

Aptana terminal load the .aptanarc instead of .bashrc when start up. You can insert a path configurations for rbenv on it like this

$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.aptanarc
$ echo 'eval "$(rbenv init -)"' >> ~/.aptanarc

Enjoy, rubymongers!