19 Mar 2014
This weekend one of my colleagues at Bloomberg and I spent a few
hours at my apartment this weekend to deploy OpenStack. I made
the same attempt last year by myself and was barely able to get a
OpenStack service running. It was that hard. But this past year,
with the help of a vibrant community, have made the process for
deploying a cluster a hell of a lot easier.
Using Fuel from Mirantis, and with a little configuration of my
managed switch, I had a two node OpenStack cluster built from eBay
servers that I had lying around. One controller node and one compute
node we were ready to rock and roll. Even the managed switch
configuration for VLAN support went incredibly smooth.
The ultimate goal is to have a relatively portable, simple datacenter
so that distributed systems can be easily demonstrated at meetups. Chaz
and I will be working on making this process as transparent and automatic
as we can.
Even though my cluster is still in pieces on the floor I am hoping to
use it to demonstrate some Chef development at future
Scale DC meetups. Chaz will be presenting the work that he has
done setting up Fuel and his home cloud which puts my completely
to shame. After ChefConf 2014 I am hoping to introduce some of
the work that the new Web Operations team at Bloomberg have been
hacking away at.
12 Mar 2014
For the past year I've been working on a grassroots effort building reusable
patterns for infrastructure at Bloomberg Government. Over the course of
a year my team was able to stabalize and automate datacenter operations.
Because of legacy infrastructure our focus was primarily on the applications
rather than the machines themselves.
Of course making application deployments deterministic and improving the
continuous delivery pipeline were likely the low hanging fruit at BGOV.
But the larger issue needed to be managed at orchestration and provisioning
of new virtual machines. The more that we dug into the problem we realized
that BGOV was not the only organization at Bloomberg that was suffering
As of two weeks ago I've been given a tiger team of engineers throughout
research and development to work with our cloud infrastructure and solve
these problems for all of our web verticals. We were given the charge to
automate the datacenter operations - including orchestration, provisioning and
continuous delivery - and integrate with the enterprise systems that we use
throughout our firm.
This goes hand-in-hand with implementing a culture of DevOps where the
firm promotes communication, collaboration and integration amongst all of the
stakeholders in a business unit. This includes the research and development
component of a product, but also the business and technology support teams.
The ultimate goal is to deliver a stable, higher quality product to our
27 Apr 2013
My good British friend Phil Sturgeon posted an awesome writeup
regarding Vagrant and Chef. I've been meaning to do a post
of my own. Over the last few months a side project of mine has got me
involved a lot with Vagrant, Chef and Veewee. But to
find resources on the Internet, especially tutorials, which combine
all that knowledge is definitely a task I would not wish on anyone
So I figured I'd put together a simple little blog post that gets you
started building out your own testing environment. Just a heads up: I
am going to assume you have the Vagrant dependencies installed. I
prefer installing these from the website and not through RubyGems, but
to each his own. The version I am using is 1.1.3. Everything else
should be picked up automatically from the appropriate files.
I am going to walk you through the steps that I used to build the
example application which is available as a Gist. Remember that
you can just run this using the
vagrant up command and it will be
available (in a few minutes) by accessing the 8080 port on your host
machine. You're still going to have to go through the whole
Veewee build of the base boxes. But after you add the base box
that you've built you can just start the application with
The first order of business is to checkout some base box
definitions. I tend to use Opscode bento repository which
contains Veewee definitions that pre-builds your machines with
Chef 11 already ready to rock. This obviously alleviates the need
to spend time on the nitty-gritty and just get down to the business.
$ git clone git://github.com/opscode/bento.git bento
$ cd bento
$ bundle install
Now that you have the bento repository checked out and all of the
dependencies installed using Bundler you can begin by building
your initial base box. I'm a fan of CentOS but most of the
open source cookbooks out there will work flawlessly with
Ubuntu. So let's start with using that.
$ bundle exec veewee vbox build 'ubuntu-12.04-i386' --nogui --force
$ bundle exec veewee vbox export 'ubuntu-12.04-i386'
$ vagrant box add 'ubuntu-12.04-i386' 'ubuntu-12.04-i386.box'
The first command that we executed there will take a little while to
finish. But it performs a headless installation of Ubuntu 12.04
32-bit and will overwrite any existing Vagrant VirtualBox
machine that previously built in this directory. They key off of the
ubuntu-12.04-i386 definition name. After that we export to the
Vagrant recognized format.
Because Veewee is using Vagrant to interact with
VirtualBox we actually have all of the flexibility with our
base box definitions. What does that mean exactly? Vagrant
has several plugins which allow us to use many different
providers for provisioning. For local testing it makes sense to use
VirtualBox but perhaps our integration machines are vanilla Linux
and we want to use KVM. Its very simple with Vagrant to
configure our project to have multiple providers for provisioning.
But once again for flexibility (and making this tutorial easy) let's
stick with the basics. Our project is very simple and we just want to
get a virtual machine ready for testing. This means we just want a
simple Ruby and Rack application since I'm more familiar
We need to modify our Vagrantfile that was created using the
vagrant init command. This is the file where we define the
directives necessary for provisioning your virtual machine. Once
again, we're going to use Ruby and Rack so your mileage
may vary with any other languages or frameworks.
You can clear out all of the default logic, or place this at the top
so that you have that for reference. Here's the basic changes we're
going to do:
- Set our base box name to the one that we exported using
- Forward HTTP traffic from the Rack application so it is
accessible to our host machine.
- Only use the amount of memory that we think we'll need.
- Provision using Chef 11 a Ruby and Rack
Now that we have our Vagrantfile setup we just need to get all of our
cookbook dependencies. I've kept this a little light so that you can
easily work through everything. Using the librarian-chef gem we can
define cookbook dependencies similar to using Bundler for gems.
$ bundle exec librarian-chef install
I have defined the path the Unicorn webserver to point to the current
working directory which has been mounted in the guest virtual machine
as /vagrant. This is defined in the rack_application Chef role. I am
not going to go into the Rack application itself, you can take a look
at the whole project's repository and check it out.
At this point assuming you've added the base box to Vagrant you can
vagrant up and see the application from your localhost
running on port 8080. Once the virtual machine is running you can use
vagrant reload if you are aking modifications to any recipes or
roles. I'm going to post some more in depth articles about writing
custom cookbooks in the near future, but I hope that this was
useful. If you have any additional questions feel free to hit me up on
27 Feb 2013
A few days ago I wrote a post about marrying Ruby and C and
provided an easy to follow example gist. After spending some more
days hacking on C/C++ integration with the Ruby virtual machine I
decided that finding some quality examples was surely a real big
pain. Many of the problems that we are facing are directly related to
objects being accessed from multiple threads at the same time in
the Ruby virtual machine.
While learning I found writing some example code definitely
helps. This new example is a little more complex, but pretty much
covers most of the bases in regards to basic object functionality. I
decided to leave out inheritance in this example as it was getting to
be pretty big. My goal was to be able to sit down and write the
example, plus a blog post, in a single night. If it took me longer
than a few hours its probably too complex of an example.
I am going to continue diving into embedding the virtual machine, and
with that I hope to be able to continue to document examples here and
on my github account. Feel free to drop me some patches if you
find errors in the code. Or leave a few comments.
My previous example takes some time and shows very basic Ruby
virtual machine integration with a C application. This example expands
on this and includes how you might wrap a C structure and integrate it
with normal functionality inside of Ruby. Some examples that you'll
see here include: instance and class variables, instance and class
methods, using the initialize method and yielding to the calling
Be sure to take some time and read the basic example so that
you're sure to understand how it all works. The script can easily be
changed in the main.c file if you want to
puts some information.
23 Feb 2013
For a few weeks now I've spent some time working on embedding the Ruby
virtual machine into an application service for a Celluloid
server spike. Specifically we took the trunk release of Ruby 1.9.3 and
started integrating our service framework utilizing some of our
tried and true libraries inside of our server code. This is a
similar approach that you would take when developing a native
extension - the commonly used mysql2 gem is an example of this -
where an interface API is written to expose functionality into the
We took a similar approach. The beauty of the Ruby language is
that it is very easy to digest, and even easier to interface
with. I've written up a small example gist which I have included
below that illustrates how easily you can embed the virtual machine
into an existing C application with very little effort.