opensource

Mar 02 2009

FAT

Harald has a nice overview of the FUD that is being spread again by some company from RedMond ..

Now why are they targeting TomTom ? Surely because Canon and Sony and different others are to big to sue, Both my EOS400D, Ixus400 and K800i , are using a way to FAT filesystem to store files on,
I tried putting ext2 formatted disks in them but that failed :)
(Not running Linux or already having paid up might be another reason)

Harald notes that there is no difference between the Kernel.org FAT implementation and the TomTom implementation so why aren't they going for a bigger fish ?

I`m sure there are plenty of other companies shipping Linux based products that are allowing people to use their own Flashdisks hence using the FAT code.

TomTom is originally from The Netherlands, yet the case is
filed somewhere in the US. I wonder how the EC would react to such a suit.

Oh well.. interresting times ahead ...

Feb 06 2009

Image Sprawl , and the new cure ..

When I tell people that the concept of copying VM's around as frequently done in the VMWare world is one of the most stupid ideas on this planet, I get the weirdest looks.

In my world it is, I want my infrastructure to be reproducible , I want to be able to throw any machine in my infrastructure out of the 10th floor of a building and be up and running again in no time. If I spread a bunch of VM copies around who knows what kind of life they start leading. Some will get upgrades, some won't ..
If I get an image from someone, how did he get there ? Nobody knows ..

To me Image Sprawl is more than not being able to to manage your Virtual Machines, it also matters for physical machines that are being deployed using a golden image.

Now rewind back about 4 something years.. back then I wrote a paper for LinuxKongress titled Automating Xen Virtual Machine Deployment which described a Hybrid way of Bootstrapping an infrastructure.
Quicly summarized, you use the benefits of images to quickly deploy a minimal image which
Luke today calls a Stem Cell then go on using centralized package management and a configuration management tool to keep them up to par. There are 2 things that changed in between,
we replaced CFEngine with Puppet , and the fact that today some people do care a bit more about the infrastructure side of the web, guess we have to thank Amazon and the Cloud Hype for that

But fundamentally .. not that much changed :)

Feb 06 2009

On Monty Leaving Sun

When I read Monty's post on leaving this passage struck me the most.

The main reason for leaving was that I am not satisfied with the way the MySQL server has been developed, as can be seen on my previous blog post. In particular I would have like to see the server development to be moved to a true open development environment that would encourage outside participation and without any need of differentiation on the source code. Sun has been considering opening up the server development, but the pace has been too slow.

In short, Sun isn't open enough. I think I've said that enough, it's typically more Open Core than Open Source .. and for a growing amount of people.. that isn't good enough.

Reacting on that post we see Matt Asay trying to convince his CNet Audience that Open Source Ideals don't translate well into a big sofware business.

I think Tarus view I see it as just the opposite. Open source spells the end of big software, if big software is defined as companies that make billions of dollars from selling software licenses. is much more to the point.

And the way Sun has been working, with MySQL, Virtualbox and different others doesn't seem to work that well either.
I stopped counting the P companies but I think I`m allmost a point where I know mere ex-Sun/MySQL employees than current Sun/MySQL employees and it's not just on the MySQL side that this happens.

Maybe Monty leaving Sun will wake up the powers that be, maybe it won't.

Finishing off with another quote from Tarus

In my mind Monty is a role model and I wish him all the best.

Jan 29 2009

What does your BOFH want ? :)

Larry, I`m glad you asked ..

With the risk of receiving a flood of comments pointing me to already existing tools here's my go at what I as a sysadmin of often large deployments am looking for in Drupalland .

You suggest LDAP and syslog integration .. guess we already have that don't we ?

But what I haven't found yet ..
An rpm/deb repository of Drupal modules So we can do an apt-get install drupal-package , yes I know about Drush, but I want the files on my system to be in a package and clearly identified, it helps me keeping my system uncluttered.

apt-get update drupal-package , or yum install drupal-package would be a luxury, same for themes btw.

It would be lovely if the postinstalls of those package also trigger a database upgrade if needed.

Which brings me to the next issue. If I have a multisite setup and I update a module in sites/all/modules, I usually have to go trough each and every site hosted there to update the databases. No really something I like to do for 10+ euh 3+ sites.

Also think big, don't waste your time on desktop apps or guis .since as long as you only have 1 site to manage point an click is fine, for you , Think in terms of what if you have 10 sites, 100 sites, or more .. do you really want to do that kind of administration via a browser or gui ? Some wise man once said If your computer can't install it the installer is broken. A script should or automation tool should be able to interact with the sites, not a human operating a mouse ;) It's not just the RSI , but also the fact that to err is human, and if the computer fails a script you can patch it :)

Jan 26 2009

Stop stupid Software Patents

If you read this blog, you should probably sign the petition !

Jan 15 2009

How popular is an Open Source project ?

There is a really easy way to figure out ...

Look at the size of of the devrooms , if at all , a project gets at Fosdem
It's really interresting to see the Embedded room move to a 500 seat room which it really needed when the first openMoko talk was held there in a previous edition.

The FDO , Drupal, Mozilla, Centos/Fedora and Suse rooms stay in similar size rooms as last year.. But it seems lik the BSD and PostgreSQL room which was pretty crowded moved to a bigger location.

The Ruby room also seems to move to a bigger room. Fosdem has a couple of new rooms too it seems this freed up room for new groups such as the MySQL Crowd

Obviously these sizes aren't a real match to the size of a community, as the new rooms might need (and probably will need) a reshuffle for next year ;)

Different schedules for the devrooms are online .. you'll probably find me in the Debian , Fedora+CentOS, MySQL or Drupal rooms :)

Did I mention that :

and I`ll actually be speaking there again about Monitoring MySQL

Jan 14 2009

Contributing Back

A while ago Dries wondered about Contributing back to Drupal .

Now Inuits is not a Webshop, we are an Open Source shop, so you won't see a zillion Drupal modules being contributed by us in the near future (albeit there are a couple) , we are company assisting other organisations in their adoption of Open Source, and Drupal is amongst the projects we care for. You'll notice code from us in the different other Open Source projects, including the Linux Kernel and other core infrastructure. And not all of it was code, there is a lot of published documentation, methodologies, bughunting and also spreading the words, or talking about our experiences around different topics such as MySQL Cluster and Drupal, or different alternatives to Monitor , or to Monitor MySQL etc ..

But we've been contributing in different other ways to open source and we have always been benefiting from that. And we try to convince our customers to do so too.
So Dries is right in all his reasons why one should contribute back to open source, specially as an organisation that uses OpenSource for it's customers you just have to.

The lack of contributions however might have different reasons
I can imagine however that the moment a commercial Open Source company starts backing or distributing people start looking at that project in a different way.

The amount of contributions to a pure open source project has been traditionally lower than the amount of code contributed to an open core project.

And I`m pretty sure that most of the braindead box moving RedHat and other so called Value Added Resellers aren't contributing a single line of code and my fear is that with the current growth of open source adoption that more of those traditional IT shops will just resell support subscriptions as if it were just another software product.

Now the Drupal community is one with a lot of developers so things might be different there compared to a group of system integrators that are installing operating systems and don't know the difference between python and php.

I think we'll see more and more open source users that aren't planning to contribute back, (although it is easy ) , But do we honestly think te group of developers is infinite ? However the more Open Source users the more chance we have these users turn into developers, I just don't think the current percentages will stay the same.

Anyway .. we'll continue to contribute.. and we hope you do the same ...

Dec 19 2008

MythTV recover Lost+Found

My MythTV store lives on an LVM volume that is spread over 2 disks, one of them is an external USB disk. So the cleaninglady seems to have touched a cable and after coming back from holiday I had a read-only filesystem that afer a remount had about 350Gb in lost+found with irrelevant filenames.

  1. total 337407844
  2. drwx------ 2 tv tv 4096 Dec 17 22:47 .
  3. drwxrwxrwx 15 tv tv 4096 Dec 17 22:44 ..
  4. -rw-r--r-- 1 root root 423343556 Dec 14 07:10 I303109.RCN
  5. -rw-r--r-- 1 root root 2990538924 Dec 13 19:05 I303107.RCN
  6. -rw-r--r-- 1 root root 1023691768 Dec 13 08:10 I319494.RCN
  7. -rw-r--r-- 1 root root 1023622348 Dec 13 07:45 I327684.RCN
  8. -rw-r--r-- 1 root root 423735892 Dec 13 07:10 I327682.RCN
  9. -rw-r--r-- 1 root root 466749476 Dec 12 15:43 I135169.RCN
  10. -rw-r--r-- 1 root root 1023314212 Dec 12 07:45 I098309.RCN
  11. -rw-r--r-- 1 root root 1022459928 Dec 12 06:35 I098306.RCN
  12. -rw-r--r-- 1 root root 2458822948 Dec 9 22:50 I139264.RCN
  13. -rw-r--r-- 1 root root 2129683736 Dec 9 21:30 I323592.RCN
  14. -rw-r--r-- 1 root root 466735992 Dec 9 15:43 I323590.RCN
  15. -rw-r--r-- 1 root root 1022747296 Dec 9 07:45 I323588.RCN

Obviously I wanted to recover my data.
So I had files with a wrong filename on a filesystem but with a correct timestamp and probably the right filesize.
Luckily the mythconverg.recorded table also gives me lots of information about the files that mythtv had originally stored the content in.

  1. mysql> select basename,lastmodified,filesize from recorded limit 10;
  2. +---------------------------+---------------------+------------+
  3. | basename | lastmodified | filesize |
  4. +---------------------------+---------------------+------------+
  5. | 1003_20081003230000.mpg | 2008-10-07 21:53:49 | 6197765380 |
  6. | 1093_20080320232600.mpg | 2008-03-20 23:25:31 | 0 |
  7. | 1075_20060301191300.mpg | 2006-03-24 22:48:42 | 0 |
  8. | 1002_20080729160500.mpg | 2008-07-29 19:20:30 | 3679223940 |
  9. | 592251_20081217072000.mpg | 2008-12-17 07:20:02 | 0 |
  10. | 1002_20080911143500.mpg | 2008-09-11 16:41:44 | 3486101572 |
  11. | 1002_20080923143500.mpg | 2008-09-23 16:49:41 | 3679789684 |
  12. | 1033_20081110153500.mpg | 2008-11-10 15:47:12 | 338877000 |
  13. | 1002_20080922144000.mpg | 2008-09-22 16:47:38 | 3485505140 |
  14. | 1002_20080721160500.mpg | 2008-07-23 20:52:16 | 3679559444 |
  15. +---------------------------+---------------------+------------+
  16. 10 rows in set (0.00 sec)

My first idea was to use the mysql filesystem engine to query the filesytem and write me a simple query however I totally failed to build that engine :(
(Anyone else successfull here ? )

So I created a temp table

  1. CREATE TABLE `temp2` (
  2. `size` bigint(20) default NULL,
  3. `oldname` varchar(255) default NULL,
  4. `lastmod` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP
  5. ) ENGINE=MyISAM DEFAULT CHARSET=latin1

And parsed the content of my lost+found directory into a set of insert statements

  1. ls -l --time-style=long-iso | awk -F' ' '{print "insert into temp2 values (" $5 ",\"" $8"\",\""$6" "$7"\");"}'

From there is a matter of grabbing the matching filenames

  1. echo "select \"mv \" , oldname, basename from recorded, temp2 where temp2.size= recorded.filesize ;" | mysql mythconverg

And moving the actual files ... now all is back to normal ..

Dec 19 2008

MySQL 5.1 on Fedora 10

So I wanted a fresh MySQL 5.1 release on my Fedora 10 box.
Fedora comes with a mysql 5.0 package and the MySQL community packages for 5.1 conflict with that in different ways. (to start with MySQL vs mysql)

For some weird reason libmysqlclient (in mysql-libs) is required by redland, which is required by openoffice.org-core. I probaly never need the RDF funtionalities that redland provides me but I`m stuck with them anyhow.

So Fred pointed me to

Remi repo for mysql 5.1

  1. yum --enablerepo remi install mysql-server

Now gives me a very nice MySQL 5.1 to play around with.

Oct 23 2008

MythTV Broken again :(

The earlier posted fix , getting my data from mythportal seemed to be a temporary solution.

It also seems the solution that was mentioned at Kristof Vanhertum's blog , the agent fix doesn't work anymore either.

Somewhere in the comments of that post however people start referring to mc2xml a tool to get your info (even 14 days worth of it) from a really untrustworthy source and import it amongst other tools into MythTV

No source code to be found, just a binary .. something that always scares me .. But running the thing (first time as a dummy empty user, one never knows) actually seems to give me an XML file with relevant data , to be used by mythfilldatabase.

  1. ./mc2xml -c be -g 2610
  2. Loading ..... : mc2xml <mc2xml@gmail.com> (2008-09-15)
  3. Connecting .. : microsoft.com
  4.  
  5. Select lineup:
  6.  
  7. 0: TER Unmapped Analogue Terrestrial
  8. 1: TER Unmapped Digital Terrestrial
  9. 2: CAB Unmapped Analogue Cable
  10. 3: CAB Unmapped Digital Cable
  11. 4: SAT Unmapped Analogue Satellite
  12. 5: SAT Unmapped Digital Satellite
  13. 6: SAT TV Vlaanderen Digital Satellite
  14. 7: TER TV Vlaanderen Digital terrestrial
  15. 8: CAB Telenet Analog Kabel (132)
  16.  
  17. (0-8): 8
  18.  
  19. Downloading . : microsoft.com
  20. Downloaded .. : 353944 bytes
  21. Source ...... : BDS
  22. Headend ..... : 7200116
  23. Version ..... : 2693590756
  24. Start time .. : 2008-10-23T00:00:00 UTC
  25. End time .... : 2008-11-06T23:55:00 UTC
  26. Parsing ..... : Telenet Analog Kabel (132)
  27. Writing ..... : xmltv.xml
  28. Writing ..... : mc2xml.dat
  29. Completed ... : 5384193 bytes in 26s, 27 channels, 9409 programs scheduled.

Yes that's my actual postcode. Different postcode values will give you other alternatives such as Coditel/Integan .. so you better use the right one.

I`m not really satisfied however, it fails to recognize my already defined channels. So I seem to have something that at first sight looks like relevant data . Only my channel numbers are way way way off ..

I need to fix that asap. The only problem being that tomorrow it's Geekdinner and I`ll be spending the rest of the weekend at T-Dose hence I have no time to fix this anymore ...

Anyone got a quick fix for that ? Or will I just have to live with a weekend of non-recordings ? :)

Well.. maybe next week will be a productive one then :)