Sep 08 2005

Supporting an Infrastructure

I dream of a world where a customer logs a support call, or a ticket
is created by a network monitoring tool.

A support engineer accepts the call and looks at the problem.

(Off course the environment is fully documented in a structured way ,
e.g docbook style, and one has a logbook of all the events that happened
before the support call).

The engineer solves the call and documents his changes in the ticketing
system, these notes are automatically added to the infrastructures'

When an engineer has to modify substantial amount of documentation she
opens up the documentation site and start editing wiki style the
documentation , or uses the plaintext version that she has checked out
of a repository and commits it back afterwards.

Upon closing a call the customer gets an automatically generated update
of his documentation (generated in .rtf / .pdf or online)"

Uff course the engineer only has 1 place where he/she should point a
browser to .

I don't think it's that far away .. however someone needs to integrate these tools

In case you're wondering which tools

  • s/ticketing system/otrs/g;
  • s/network monitoring tool/nagios/g;
  • s/documented structured way/docbook sgml/g;
  • s/repository/cvs|svn/g;
  • s/documentation site/dookbookwiki/g;
Sep 01 2005

Desktop login failing

I`m struggling with a bad adsl connection latelty. So bad that my connection often dies for hours, It's supposed to be my internal telephone line which I've replaced by now so I`m hoping for the best.

Anyway .. we had short power outage yesterday night and my iMac (running ubuntu) got rebooted. When trying to log in, the screen went brown and nothing else happened.
Strange.. didn't see any errors in the logs , when switching to console mode one error catched my eye, some files couldn't be untarred because they were in the future.
While looking at the permissions of files in /tmp where gnome etc tend to throw some files , I noticed that a lot of files were dated 1 january 1904 (yes that's right 1904 !!).
Which meant that ntpdate had failed, it couldn't look up the hostname for it's ntp server (in this case because I had no ip connectivity).

So because a DNS lookup failed I could not log on to my desktop..

Aug 25 2005

10th floor test

The 10th floor test , is a term first used by Steve Traugott from It refers to the ability to take a random machine in your infrastructure , drop it from the 10th floor, and be able to restore your infrastructure to working within 5-10 minutes.

I initially toughtt it was Thomas Lange who invented the term (as told to me on #fai) but it seems he got it from Steve.

I tried to add this as a Wikipedia entry, but people are voting for deletion beacuse "Zero relevant google hits (highly unlikely for a true IT-related term)" and "Neologism"
Well, maybe 10 years from now ...

Aug 22 2005

Everything is a Fscking DNS Problem, the T-Shirt !

Get it Now! ;)

Ok so I`m not planning on making money with it , but as people keep missing the point , it is time for a broader campaing to spread the word :

Everything is a freaking DNS Problem

no really, it is .. and if it's not well...we have a list of other issues that might cause your problems but they are only worth looking at , AFTER we excluded the DNS issues.
So get those T-Shirts, and wear them as much as you can so that each time someone starts complaining about his slow ssh access to a machine, you just have to point him to the text on your chest.
After a while they'll realise your right :)

BTW: I only bought the original black & white T-Shirt with Text so far, but the quality seems really good so I hope the same for the other designs.

Aug 22 2005

Living in Structured Text files :)

Everybody knows that the average human brain isn't designed to be able to memorise all possible facts, phonenumbers, passwords, tips, tricks , command line options etc etc, so keeping stuff stored somewhere o n a safe location might be a wise thing to do
Living In Text Files might be a good idea at first but I prefer to live in a structured Text text file.

A structured text file (such as a docbook based xml or even sgml document) gives you a couple of extra benefits.
I structure my notes a chapter a day, then subsections for eacht topic I`m working on.

What I get back is an unlimted ammount of data that I will need afterwards. From my day to day notes I can easily build project documentation , I can grep trough my docs for important strings , and If I really don't know what string to look for I can at least go have a look based on the date I wrote it.. Mostly I have some idea when I did something.

The key thing is to document everything, from command line options you don't frequently use to errors in log files and how you solved them.
From meeting minutes to failure in things you tried. Trial and error needs to be documented in order not to make the same errors again :)

So Yes I Live in Text files, even a couple of big ones but they are structured

Aug 18 2005

Mom, Here's our new commercial

Yeah we finally figured it out , In todays Dilbert (desktop editition) , Dilbert silently hints at which company he is actually working for, and what a surprise , their HQ is in Redmond !

Aug 09 2005


A small bit of chaos since the building where the talks were
yesterday was closed, no signs or anything so after talking to some
people I figured out they moved to the basement of another building

The FAI talk , I`m interrested how this differs from SystemImager
etc , Thomas Lange even mentions a couple of
times. He also mentions the only good way when using imaging (golden
image etc) is when you automatically build the image (with FAI) ,
what can I say more.. (GMTA? :)) Off course I build the images
either with systeminstaller ,deboostrap or yum (--install=$PATH -y groupinstall Base Apart from installing Thomas explains how he
modifies config files using CFengine , he also mentions that you
should be using cfengine to modify existing config files but not create config files with them.

Thomas mentions future RPM distro support based on smartpm No Xen integration yet :) Sometimes talks are great, not because you learn something but because they totally confirm that the way you are doing things is the good way to do it.

Hands off , on the other way of automating Debian installs , Philip explains he needed a way to install Debian for people with no infrastructure, (people with no (reverse) dns knowledge etc)

It seems that started with a tradition that is spreading to the rest of the planet now, last conference day lunch == pizza fight.
As we have to be back in Dover at a reasonable hour I`ll be skipping the last talks in order to be able to check out or the hotel :)

Looking back the UKUUG Conference is one of the more interresting conferences around , a nice location combined with interresting topics , Alasdair did a great job organising the event, thnx a lot !

Aug 09 2005


Ian Pratt , Xen, a full 1.5 hour talk about Xen

Ian mentions lots of uses for Xen : Consolidate, Avoid Downtime ,
Dynamically re-balance workload to guarantee application SLA's and Enforce security policy

Multilevel secure systems , 1 virtual machine for browsing (back at home) , 1 virtual machine for corporate usage which can securely
connect back to the office.

The Xen 3.0 release is between 4 to 6 week away and will support new architectures Xen is moving stuff out of xen and let Linux do that
stuff for them, this way they can start using ACPI and AGP ,initially Xen was developed as a Server back end which means that
back then people didn't need X and ACPI, this will now change.

SMP virtual machines are now possible, (or how to fake a 32 CPU machine on a single CPU machine ..) Support for the new Intel VTx
extensions which will enable them to run proprietary platforms is also on the roadmap.

Marcus Hardt shows us some Use cases for Xen, since he doesn't get the budget to buy enough machines when teaching HPC he uses Xen to build a Grid in a box. He's also providing people with a Login Cluster which gives people an on demand distro based on their username

Marcus then went on to discuss the Benchmarking he with tools regarding Xen.
Ruediger Berlich explained what a grid really is (unlike Oracle's wannabe grid stuff) and how the EGEE project tries to solve the problem of different grid projects not being compatible with eachother by creating and operating a production grid and supporting the end -users by building gLite on top of best of breed of other grid projects.

After lunch I squeezed down my laptops resolution to 640x480 and gave a fast forward openMosix presentation, the 10 minutes left sign really came way to fast, which made me skip a couple of slides and probably made the talk even more chaoitic than it already is :( It probably also made the talk look like I didn't say anything more than the stuff already on the slides which was obviously not the goal.

Dropped back into the Linux on Power talk by Nigel, who pointed us to what he thinks is a better way of displaying/reporting system information

Back to the hotel to get some food and sleep :)

Aug 07 2005


I first went to the "Experiences of a large linux deployment in
education." talk of which the title already got changed to "a linux
deployment", no sign of the large part :( Which was rather
dissapointing for me. However the talk was interesting anyway. They
mainly discussed thin client solutions using LTSP. With 1 server
they serve about 35 users. (dual cpu 4Gb) Using blade servers (IBM)
only for standardisation reasons.

They had a relevant complaint about Memory utilisation of both
OpenOffice, Gnome and KDE, where's the time when we squezed everything into 4Kb :)

Moving to adopting a driver where Matthew Wilcox explained how you
can end up maintaining a subsystem and called for more volunteers :)

Linux and ACPI, a brave new world of power management, a good and
detailed explanation of why ACPI is so difficult, where's the time
that VGA mode was really VGA mode (just set a couple of registers and int 13h right ? )

System Tap, rerun of an OLS 2005 talk , it introduces a new tool to
diagnose complex system problem while not having to rebuild / reboot
your environment too much. It's based on the kprobes infrastructure
and not only aimed at kernel developers alone ;)

Lunch, great lunch, som curries with rice, nice .. reminds me of the
Lunches at LinuxKongress and the Pizza party at LCA.

BSD Talk about Mitigating Security , somehow this reminds me of the
RedHat security talk at Fudconf 2005 Linux, well, at least some of
the parts do :)

Missed Bdale's talk at .. and now I finally know what
bouncing of the moon is :) The most interresting part of the talk is
the reason why people have been working so hard on gnu radio,
because this way they prevent legislation to disallow them to
reverse engineer hdtv since there already is an open source version available that can display it.

On to , Steve Coast claims to be the new Wikipedia
for but then for Maps, just as wikipedia was after their first year
they have a good starting point that contains some usefull maps and
with the help of the community it will grow, maybe I should hook up a GPS to my car and start feeding them some info. We need more
detailed freely available maps of Antwerp and surroundings :)

Napi talk from Jamal Hadi Salim , with some benchmarking regardingNapi.

Now am I asleep or did I just see Christoph having to use Jamals
laptop cause his Powerbook doesn't wan't to work on the projector ?
Anyway we can sleep safe since Christoph just confirmed that he is
not a spy working for Sco (anymore):)

Seems like a hughe amount of presentations are having problems with
the projectors ;( There is some urgent need for people to fix this.

After the Conference Dinner, we had Dave Miller giving the Keynote ,
he gave a nice overview of Netconf, on who is doing what, what we'll
see in the new kernels. I've got no notes as I was trying to make sure Dave didn't fall over my feet while giving his talk, and I
usually don't run around with my laptop at conference dinners :)

Aug 05 2005


Arrived in Swansea around 1700 on thursday and found the University really easy.

Trying to get in .. seemed like a little problem as the building
that hosted the conference registration doors were locked, so after
walking around the building we were lucky enough to sneek back in
with some people leaving the building.

Decided on going out for dinner just the 2 of us , making sure that
I at least saw a bit of Swansea , conference is dinner tomorrow so :)