Everything is a Freaking DNS problem

by @botchagalupe
on Virtualization, Open Source tools and DNS Problems

Here's how it started :

So I used to be a software developer, writing perl for the web, then C, then Java, then PHP, till I realized nobody ever configured my servers correctly and I changed trades becoming a system engineer, while teaching new developers the basics of their trade, whom grew into doing Infrastructure Architecture .. familiar story for much of the crowd at DevopsDays ... a crowd that wants to stopping the war between developers and system engineering , a crowd that wants to automate builds, integrate testing, deploy, deploy on very large scale, deploy in the cloud and much more.

So what do you get when you put together some of the experts on building software, organizing development teams , Agile geeks, Cloud infrastructure projects, and Automating guru's in 1 location for 2 days in Gent ? Exactly .. DevopsDays ..

The format was 2 days .. 3 kickass formal talks in the morning.. Open Space sessions in the afternoon. ... Friday featured talks on Non Functional Requirements, CucumberNagios and Monitoring in the Cloud with FlapJack and Building Agile Infrastructures with Puppet while discussing the James White Manifesto ..

which I had never heard of, but which apparently comes down to this

== Rules ==
 On Infrastructure
 -----------------
 There is one system, not a collection of systems.
 The desired state of the system should be a known quantity.
 The "known quantity" must be machine parseable.
 The actual state of the system must self-correct to the desired state.
 The only authoritative source for the actual state of the system is the system.
 The entire system must be deployable using source media and text files.
 
 On Buying Software
 -------------------
 Keep the components in the infrastructure simple so it will be better understood.
 All products must authenticate and authorize from external, configurable sources.
 Use small tools that interoperate well, not one "do everything poorly" product.
 Do not implement any product that no one in your organization has administered.
 "Administered" does not mean saw it in a rigged demo, online or otherwise.
 If you must deploy the product, hire someone who has implemented it before to do so.
 
 On Automation
 -------------
 Do not author any code you would not buy.
 Do not implement any product that does not provide an API.
 The provided API must have all functionality that the application provides.
 The provided API must be tailored to more than one language and platform.
 Source code counts as an API, and may be restricted to one language or platform.
 The API must include functional examples and not requre someone to be an expert on the product to use.
 Do not use any product with configurations that are not machine parseable and machine writeable.
 All data stored in the product must be machine readable and writeable by applications other than the product itself.
 Writing hacks around the deficiencies in a product should be less work than writing the product's functionality.
 
 In general 
 ----------
 Keep the disparity in your architecture to an absolute minimum.
 Use [http://en.wikipedia.org/wiki/Set_theory Set Theory] to accomplish this.
 Do not improve manual processes if you can automate them instead.
 Do not buy software that requires bare-metal.
 Manual data transfers and datastores maintained manually are to be avoided.

Much unlike the FAIL Manifesto

The openspaces tackled how to migrate from a totally unreproducable environment too a correctly bootstrapped infreaastructure, over the Ubuntu Enterprise Cloud , then dinner and off for beers to the Vooruit . The OpenQRM "crowd" stayed at my place so I didn't stay around too late ..

Saturday morning came early ... sadly I missed the first 10 minutes of a very interresting talk about Kanban in operations ... let's ee if we can convince some more people to try it out ...

The talk on Continuous integration, Build Pipelines and Continuous deployment was also really interresting with lots of stories from the real world.. . after the openqRM talk it was time again for OpenSpaces with e.g discussions on svn vs git and building a feature matrix of Cloud , with @botchagalupe, @openqrm and @maesjoch in the room and @diegomarino online .

Devopsdays ended too soon , with way to much interresting ideas to build on .. Let's hope we can all work them out !

As Teyo will be in Leuven (Belgium) next week for the training , some folks had the idea to meetup after the training for a Puppet Users Meetup.

So the plan is to meet for Drinks and Puppettalk, next week wednesday as of 20u00 in the Domus in Leuven

http://www.domusleuven.be/

I hope to meet a bunch of you there !

PS. Feel free to spread the news.
PPS. There is a chances I won't make it myselve which shouldn't keep you folks back :)

Next month I`ll be teaching a course on Virtualization at the KULAK.

Apart from my training there is also a Puppet training planned in Belgium.

As I assisted Luke in finding a location for the training I tought it might be a good idea to have a Puppet Users meetup while some people are already gathering in Leuven

Current plan is to meet up somewhere in Leuven on the evening of the 25th , more announcements later ..

When I tell people that the concept of copying VM's around as frequently done in the VMWare world is one of the most stupid ideas on this planet, I get the weirdest looks.

In my world it is, I want my infrastructure to be reproducible , I want to be able to throw any machine in my infrastructure out of the 10th floor of a building and be up and running again in no time. If I spread a bunch of VM copies around who knows what kind of life they start leading. Some will get upgrades, some won't ..
If I get an image from someone, how did he get there ? Nobody knows ..

To me Image Sprawl is more than not being able to to manage your Virtual Machines, it also matters for physical machines that are being deployed using a golden image.

Now rewind back about 4 something years.. back then I wrote a paper for LinuxKongress titled Automating Xen Virtual Machine Deployment which described a Hybrid way of Bootstrapping an infrastructure.
Quicly summarized, you use the benefits of images to quickly deploy a minimal image which
Luke today calls a Stem Cell then go on using centralized package management and a configuration management tool to keep them up to par. There are 2 things that changed in between,
we replaced CFEngine with Puppet , and the fact that today some people do care a bit more about the infrastructure side of the web, guess we have to thank Amazon and the Cloud Hype for that

But fundamentally .. not that much changed :)

Chef absolutely confuses me..

Luke is confused too ..

I’m clearly disappointed that someone who has been a high-profile user of Puppet but has never contributed much in the way of code (Ohloh claims 2 commits) would decide to start a whole new project rather than attempt to contribute to Puppet

Now , if you know me a bit you know that reinventing the wheel, or creating identical projects with no clear reasons is something I dislike .

When looking at Chef's FAQ there isn't really a clear reason listed why they wanted to create a new project.

I could understand if Chef were written in a total different language .. but hmm.. it's written in Ruby again .. I can only think of one other area where there are 2 major competing tools written in the same language and that is OTRS and RT, still wondering how that can happen.

One of the core values of an Open Source project is that you can contribute, adapt , and even fork.. why would you want to start over from scratch ?
So launching a competing open source project in that way therefore doesn't really seem like a smart thing to do,

Maybe one way to explain it is the European vs American style of Open Source Adoption ... , Luke has the more European approach (consultancy, build new features, support, train, evangelize, earn a good living) , where as OpsCode with Jesse Robins in charge might head for a more American style (Productize, Dual License , CashOut ).

So can the Chefs please explain why they didn't contribute to Puppet, or as their FAQ , well it doesn't really Answer any of the Questions

In case you haven't noticed them yet ..
MySQL Proxy has a new release and moved it's public repo to Launchpad.

(Still Launchpad isn't open source yet .. a matter of time .. but in the meanwhile Jira and Confluence are sadly gaining adoption in the market)

In the devministration area there is a new Puppet module that automagically populates puppet managed machines in Zenoss , I've seen different people using Puppet to populate their Nagios configs, but adding Zenoss to the list is new.. so when will we see the Zabbix and Hyperic plugins ?

Oh yes.. and then there is Chef

This was not a CloudCamp !

Don't get me wrong, it was a great event and I met lots of interesting people , but it was not a *camp.
The idea was there to have an unconference after the formal sessions, but the formal sessions ran out and there was no time because of food and bar duties.

The event was a mixture of regular Belgian Campers, Virtualization geeks, Open Source folks , obviously there were a couple of "lost" americans , and the crowd from up North :)

The location was weird to say the least, what if the boat hat floated off on the river :)

It's obvious the world doesn't have a fixed definition for "Cloud Computing" yet , Tarry really made a safe bet by cut and pasting the definition from WikiPedia but the thing that really worried me was that when Raph asked if the audience could define Open Source they couldn't either.

Given the audience it's really hard to understand why they couldn't explain what Open Source is .. they should be able to. As the biggest chunk of Cloud Infrastructure is based on Open Source , the audience of a CloudCamp should be able to define Open Source, but then again there was quite a number of suits around that weren't expected to understand what it is all about :)

The fact is that a the cloud today still is a bit of undefined, different marketeers are grabbing the opportunity to rebrand their longtime existing product as fresh and hot cloud.

The interesting part of the Cloud to me is the mix of Virtualization, Scalability, Automation , Large Scale Deployment , playing the puppetmaster, and High Availability ..

It's stuff I have been doing for ages , it's the stuff this blog has been covering since the beginning ... but I don't plan on renaming my blog .. as afterall the whole cloud issue is just a Freaking DNS Problem

Pictures of the event are here

In his CloudCafe 18 Podcast John talks about Puppet to Luke and they coin the idea of Devministration

I really like the terminology, so I`m a devministrator, and probably the bigger part of Inuits are Devministrators.

The first stage in becoming a devministrator is using version control, then bugtracking .. etc.
Coming from an era where I was the sysadmin pushing the developers to use version control this sounds really strange to me..
Yes I had to convince developers to use version control, while Luke thinks he needs to convince sysadmins to use version control.
Weird.. other continent, other habits, but the important part is we all use it.

But the big part is that we don't spend our time managing servers, but rather scripting the automation of the management. Learning machines how they should manage our configs and automate.

Like the old Google saying, you have to automate yourself out of a job every 18 months.

Guess that's also what ad Devministrator is.

Bart, this is not a poor man's ntp. This is a stupid man's Unix management :)

Do you plan on running that script from cron each day ? Or do you plan on eventually configuring ntp for the boxen ?

Your poor man's configuration management solution would be to use a tool such as dssh or Cluster SSH

However what you really want to do is look into a tool like Puppet and instruct puppet to "Configure NTP on host X-Y-Z"

There's a great article from chromatic about Puppet and Luke over at O'Reilly, you might want to check it out .