Everything is a Freaking DNS problem - chef

Lies, Damn Lies and Statistics, 2016 Edition

Kris Buytaert — Wed, 23 Mar 2016 21:28:36 +0000

When people sign up for Configuration Management Camp, we ask them what community room they are mostly interested in.
We ask this question because we have rooms in different sizes and we don't want to put communities with 20 people showing interest in a 120 seat room and we don't want to put a community with 200 people in a 60 seat room.

But it also gives us to opportunity to build some very interesting graph over the potential evolution of the communities.

So looking at the figures ... the overall community is obviously growing,From 350 to 420, to just short of 600 people registered now.

The Puppet Community is not the biggest anymore, that spot went to the Ansible Community room. And all but the CFengine communities are growing.

One more thing , The organisation team discussed several times if we should rebrand the event. We opted not to .. Infracoders.eu could have been an alternative name .. but we decided to stick with the name that already is known,
the content will evolve.. but Config Management Camp will stay the place where people that care about Infrastructure as Code and Infrastructure automation meet.

2014 vs 2015 interest in Open Source Configuration Management

Kris Buytaert — Mon, 09 Feb 2015 20:05:31 +0000

A couple of people asked me to results of the survey of the 2015 vs 2014 Configuration Management Camp room interrests.

This is a bunch of 350 last year and 420 people telling us what tools they are interested in so we can map the right roomsizes to the communities.

2014 :

2015:

Enjoy.. but remember there's Lies, Damn Lies and Statistics ..
PS. this is a mostly European Audience .

FOSDEM 2014 is coming

Kris Buytaert — Sun, 22 Dec 2013 20:04:04 +0000

and with that almost a full week of side events.
For those who don't know FOSDEM, (where have you been hiding for the past 13 years ? ) Fosdem is the annual Free and Open Source Developers European meeting. If you are into open source , you just can't mis this event where thousands of likeminded people will meet.

And if 2 days of FOSDEM madness isn't enough people organise events around it.

Last year I organised PuppetCamp in Gent, the days before Fosdem and a MonitoringLove Hackfest in our office the 2 days after FOSDEM This year another marathon is planned.

On Friday (31/1/2014) the CentOs community is hosting a Dojo in Brussels at the IBM Forum. (Free, but registration required by the venue)

After the success of PuppetCamp in Gent last year we decided to open up the discussion and get more Infrastructure as Code people involved in a CfgMgmtCamp.eu

The keynotes for CfgMgmtCamp will include the leaders of the 3 most popular tools around , both Mark Burgess, Luke Kanies and Adam Jacob will present at the event which will take place in Gent right after Fosdem. We expect people from all the major communities including, but not limited to , Ansible, Salt, Chef, Puppet, CFengine, Rudder, Foreman and Juju (Free but registration required for catering)

And because 3 events in one week isn't enough the RedHat Community is hosting their Infrastructure.next conference after CfgMgmtCamp at the same venue. (Free but registration required for catering)

cya in Belgium next year..

Vagrant and Drupal, a winning team

Kris Buytaert — Sat, 25 Aug 2012 21:51:31 +0000

While heading back home from DrupalCon Munich after 4 days of good interaction with lots of Drupal folks.
I realized to my big suprise that there are a lot of people using Vagrant to make sure that developers are not working on platforms they invented their own. Lots of people have realized that "It works on my computer" is not something they want to hear from a developer and are reaching out to give them viable solutions to work on shared and reproducible solutions.

There were 2 talks proposing solutions to the problem,

the first one was ..Fearless development with Drush, Vagrant and Aegir by Christopher Gervais He talked about Drush VAgrant Integraion and how extentions to Drush allow for easy vagrant integration , bridging this gap allows rupal developers to use a tool they are already familiar with

The second one was Jochen Lillich who explained how he us using Vagrant an Chef for this purpose his talk titled Use datacenter tools to make your dev life easier has been posted already.

During the Vagrant BOF , I briefly ran over @patrickdebois old slides on Vagrant after which people started discussing their use cases.. 2 other projects came up

First is Project Oscar which aims at providing developers with a default Drupal development environment in a Jiffy. they do this by providing a bunch of puppetmanifests that sets up a working environment.

And the second one is Ariadne which is a standardized virtual machine development evironment for easily developing Drupal sites in a local sandbox that is essentially identical to a fully-configured hosted solution. It attempts to emulate a dedicated Acquia/Pantheon server as closely as possible, with added development tools. Project Ariadne is just like the examples from Jochen Lillich based on Chef

With all of these tools and examples around , there should be no excuses anymore for Drupal developers to hack on their own machine and tell the systems people "It works on my machine" (let alone to hack in production).

Drupal and Configuration Mgmt, we're getting there ...

Kris Buytaert — Sun, 17 Jul 2011 20:39:27 +0000

For those who haven't noticed yet .. I`m into devops .. I`m also a little bit into Drupal, (blame my last name..) , so one of the frustrations I've been having with Drupal (an much other software) is the automation of deployment and upgrades of Drupal sites ...

So for the past couple of days I've been trying to catch up to the ongoing discussion regarding the results of the configuration mgmt sprint , I've been looking at it mainly from a systems point of view , being with the use of Puppet/ Chef or similar tools in mind .. I know I`m late to the discussion but hey , some people take holidays in this season :) So below you can read a bunch of my comments ... and thoughts on the topic ..

First of all , to me JSON looks like a valid option.
Initially there was the plan to wrap the JSON in a PHP header for "security" reasons, but that seems to be gone even while nobody mentioned the problems that would have been caused for external configuration management tools.
When thinking about external tools that should be capable of mangling the file plenty of them support JSON but won't be able to recognize a JSON file with a weird header ( thinking e.g about Augeas (augeas.net) , I`m not talking about IDE's , GUI's etc here, I`m talking about system level tools and libraries that are designed to mangle standard files. For Augeas we could create a separate lens to manage these files , but other tools might have bigger problems with the concept.

As catch suggest a clean .htaccess should be capable of preventing people to access the .json files There's other methods to figure out if files have been tampered with , not sure if this even fits within Drupal (I`m thinking about reusing existing CA setups rather than having yet another security setup to manage) ,

In general to me tools such as puppet should be capable of modifying config files , and then activating that config with no human interaction required , obviously drush is a good candidate here to trigger the system after the config files have been change, but unlike some people think having to browse to a web page to confirm the changes is not an acceptable solution. Just think about having to do this on multiple environments ... manual actions are error prone..

Apart from that I also think the storing of the certificates should not be part of the file. What about a meta file with the appropriate checksums ? (Also if I`m using Puppet or any other tool to manage my config files then the security , preventing to tamper these files, is already covered by the configuration management tools, I do understand that people want to build Drupal in the most secure way possible, but I don't think this belongs in any web application.

When I look at other similar discussions that wanted to provide a similar secure setup they ran into a lot of end user problems with these kind of setups, an alternative approach is to make this configurable and or plugable. The default approach should be to have it enable, but the more experienced users should have the opportunity to disable this, or replace it with another framework. Making it plugable upfront solves a lot of hassle later.

Someone in the discussion noted :
"One simple suggestion for enhancing security might be to make it possible to omit the secret key file and require the user to enter the key into the UI or drush in order to load configuration from disk."

Requiring the user to enter a key in the UI or drush would be counterproductive in the goal one wants to achieve, the last thing you want as a requirement is manual/human interaction when automating setups. therefore a feature like this should never be implemented

Luckily there seems to be new idea around that doesn't plan on using a raped json file
instead of storing the config files in a standard place, we store them in a directory that is named using a hash of your site's private key, like sites/default/config_723fd490de3fb7203c3a408abee8c0bf3c2d302392. The files in this directory would still be protected via .htaccess/web.config, but if that protection failed then the files would still be essentially impossible to find. This means we could store pure, native .json files everywhere instead, to still bring the benefits of JSON (human editable, syntax checkable, interoperability with external configuration management tools, native + speedy encoding/decoding functions), without the confusing and controversial PHP wrapper.

Figuring out the directory name for the configs from a configuration mgmt tool then could be done by something similar to

cd sites/default/conf/$(ls sites/default/conf|head -1)

In general I think the proposed setup looks acceptable , it definitely goes in the right direction of providing systems people with a way to automate the deployment of Drupal sites and applications at scale.

I`ll be keeping a eye on both the direction they are heading into and the evolution of the code !

Beyond Configuration Mgmt

Kris Buytaert — Wed, 25 May 2011 21:18:08 +0000

(This post has been sitting in the drafts folder for way to long, I decided to push the publish button anyhow .. some people might get ideas from it..)

We've all run in to the problem, you've puppetized, or euh .. cooked , about every part of your infrastructure and then there's this one service which has no config files, a broken api that doesn't allow you to configure antyhing, but a magnificent web gui to configure all aspects of the service. Magnificent for the eye , full of AJAX and other fancy stuff which wget isn't really keen on. Off course before it even starts working you need to set it's password , from that webgui.

Sometimes when you are lucky they store al their config in a database, which you can dump, parse and replace all the host specific parameters for other deployments, but is that an approach you like ? As for each new version you'll need to reanalyze the db layout. But no matter how you look at it ,dumping the DB and restoring it is an ugly hack you don't want.

Other alternatives like sniffing the traffic and replaying the POSTS etc were considered ... but fancy AJAX stuff and SSL make that less trivial than it seems

Wo while discussing with an upstream project they proposed to actually screenscrape their config webgui .

So screenscraping the config gui it is .. but how ... I started looking at tools that are typically used for testing rather than for automation, with the purpose of replaying the scenarios one needs to configure the services.

My first attempt was Selenium, it plugs into a browser , so it's easy to acraully record what it has to do, and it saves it's scenarios in a somewhat readable/ editable format.
Having found the export to perl function it alll looked promising. However the export to perl isn't really an export to perl as I epxected .. I assumed it would just generate the perl code to run the same scneario which would be awesome .. it however generates a perl script that instructs a selenium server to run the script.

One of the annoyancies I ran into with Selenium is that a browser
doesn't accept self signed certificates , and one can't preprovision a browser easyily with those freshly created certificates. (Yes Karl I already read about certutil ... )

I had heard good things about Cucumber so I was pretty eager to start testing it ... In short Cucumber lack documentation ,
I tried a couple of things but I couldn't get beyond testing if a certain string was on a page.. couldn't figure out how to fill in a form etc ...
Maybe if anyone could point me to some great documentation on how you should write recipe's here ... I didn't find the documentation all to easy to find ..
Bummer as it really really looks promisiung .. specially since it is so lightweight ..

IP played with JMeter and Sahi too .. but still

So apart from filing bugs to the upstream project/product and hoping they understand your problem and are willing to oopen up their API , what other options do you folks suggest ?

I gave a short talk about this at Puppetcamp in Amsterdam and the audience came up with a bunch of other potential projects to look at .

The main problem still is that all these are tools to automate testing , they don't provide you with a general purpose approach to solve the configuration mgmt problem, each time the upstream vendor modifies the layout of his page you hav e to do the work again and that .. really doesn't sound promising ..

Guest Post Season

Kris Buytaert — Sat, 18 Dec 2010 21:23:39 +0000

Apparently December is the month where everybody starts writing guest posts for other blogs.

Earlier this month I wrote an article with the title of this blog for Sysadvent ,

It's a sysadmin relative of the Perl Advent Calendar: One article for each day of December, ending on the 25th article. With the goals of of sharing, openness, and mentoring, we aim to provide great articles about systems administration topics written by fellow sysadmins

My article is here, but there's plenty more other articles written about a variety of topics, such as chef, tcpdump , how ls works, cucumber and Devops.

On the other side, Matthias over at Agile Web Development and Operations is hosting a series on Devops where lots of Devops Advocates and Evangelists are having their say about Devops ...

My entry about the Challenges the Devops Crowd faces was put online yesterday

@Beaker on #Devops

Kris Buytaert — Tue, 01 Jun 2010 21:10:29 +0000

Yesterday @beaker posted his ideas on the #devops movement ...

Apparently we haven't been stressing enough on the fact that it isn't just about Devs and Ops,
So let me repeat it's not just about Devs and Ops, it's about breaking silo's , about being good at our jobs, about getting conversation started, about talking to different stakeholders in the processes . We are absolutely trying to include all groups, not exclude some.

@beaker also seems to have seen many presentations where developers are shown to have evolved in practice and methodology, but operators (of all kinds) are described as being stuck in the dark ages. , is that a different point of view on another continent \, on this side of the Atlantic, it's mostly the Ops people that are already using agile methods spreading the word and it isn't about Devs talking about Deopvs yet. It's actually mostly the ops spreading the word because they feel most of the pain .

Hoff also wonders about routers switches firewall and all the other boxen where we aren't running puppet or chef on , the boxes that are left out of our fully automated environments .
Indeed, Puppetcamp Europe once again woke up the discussion on how to tackle these boxen, the lack of use of existing standards was covered .. and some mentioned that CIM and family are pretty much death or irrelevant for real life usage , both the Puppet and Chef communities are working on manifest, modules and recipes to solve the issues.

But the good thing is that we now have the security people involved too, maybe they'll figure out how to survive longer than 6 months in a CSO position if they talk to the others and come out of their Ivory towers :)

Linux Open Administration Days 2010

Kris Buytaert — Tue, 20 Apr 2010 19:43:05 +0000

So about 4 monts ago there was the crazy idea to start a new FOSS event in Belgium targeted at sysadmins.

What started out as an event for local people to meet local people with some local speakers actually ended up being a small local event with some top international speakers on onfiguration mananagement and system administration mixed with a bunch of good local ones !

I had the honour to open the conference with an extremely short version of the Devops talk I gave earlier last year.. extremely short as I knew that over the course of the weekend the topic would reoccur a lot.

We had the first european talk on Chef, by Joshua Timberman, and we had Puppet talks amongst by Dan Bode from Puppetlabs and CFengine talks , devops was a frequently dropped word,

We had a book raffle where we handed out O'Reilly's .. we had a great free pizza party (got the idea from the saturday pizza event at LCA 2005) , and we had some free beer. Sounds like a good combination for a geeky weekend.

Apart from the regular talks there were plenty of Open Spaces where interesting topics were discussed ... we had spaces on Open Source vs Open Core , strong voices were heard when we discussed what we should do with the Open Core companies that claim to value Open Source , some people think we should actually list the fauxpensource ones somewhere and make sure the world knows about them

We had an awesome configuration management discussion session discussing Chef vs Puppet vs CFengine . And much much more ...

Some people owe me plenty of Sushi as I had to do my MySQL HA talk before their Managing MySQL talk , but other than that .. things just went fine..

11 days till Loadays

Kris Buytaert — Tue, 30 Mar 2010 20:59:34 +0000

That's right .. only 11 more ...
The schedule looks promising, there will be some devops juice, some open spaces, some tutorials, som regular talks .. it really looks promising ... the schedule is packed ,

Apart from the talks, tutorials and open spaces there's also the
Pizza party and the Beer event on saturday ...

No need to register .. just show up ..