Kris Buytaert's blog

Dec 30 2008

Oops, Free beer for some ..

So yesterday around 2010 a bunch of us arrived at the Kulminator,

To our surprise it was closed :(
We opted to move to the closest available place , the Zeppos and
I called pretty much everybody who's phonenr I had handy that I knew could come. (much voicemails) I hope not too much people ended up standing in front of a closed door with no idea where to go (my phonenr is easy to find) but the ones that made it to the new place benefitted from Free Beer sponsored by Inuits

We chose the Kulminator because it was a well known place and different people had suggested it. And when Bram commented that the place wouldn't open till 2000 on mondays I figured it would be open for sure.

Big was our surprise when towards the end of the evening he pulled out a card from the Kulminator which indeed mentioned it's opening hours.

So let's not go there in the beginning of august either .. :)

Dec 29 2008

Joke of the Day

Overheard today in our technical workshop.

Q: Does anyone have a stable Debian with him ?
A: Yes
Q: Can you pass me the floppy.

Dec 23 2008

Planet Grep.be meetup

As I got a lot of positive reactions for the idea I posted last week, here is some more detail for the beerdrinking next monday.
29 december 2008 that is ..

The plan is to meet from 20:00 onwards in the Kulminator in Antwerpen.

I already got feedback from over a dozen PGB readers and posters that they will show up !

But feel free to abuse the comments to let us know you are coming too !

PS. As Bram commented .. no need to be early as Kulminator only opens at 20.00 on monday ..

Dec 19 2008

MythTV recover Lost+Found

My MythTV store lives on an LVM volume that is spread over 2 disks, one of them is an external USB disk. So the cleaninglady seems to have touched a cable and after coming back from holiday I had a read-only filesystem that afer a remount had about 350Gb in lost+found with irrelevant filenames.

  1. total 337407844
  2. drwx------ 2 tv tv 4096 Dec 17 22:47 .
  3. drwxrwxrwx 15 tv tv 4096 Dec 17 22:44 ..
  4. -rw-r--r-- 1 root root 423343556 Dec 14 07:10 I303109.RCN
  5. -rw-r--r-- 1 root root 2990538924 Dec 13 19:05 I303107.RCN
  6. -rw-r--r-- 1 root root 1023691768 Dec 13 08:10 I319494.RCN
  7. -rw-r--r-- 1 root root 1023622348 Dec 13 07:45 I327684.RCN
  8. -rw-r--r-- 1 root root 423735892 Dec 13 07:10 I327682.RCN
  9. -rw-r--r-- 1 root root 466749476 Dec 12 15:43 I135169.RCN
  10. -rw-r--r-- 1 root root 1023314212 Dec 12 07:45 I098309.RCN
  11. -rw-r--r-- 1 root root 1022459928 Dec 12 06:35 I098306.RCN
  12. -rw-r--r-- 1 root root 2458822948 Dec 9 22:50 I139264.RCN
  13. -rw-r--r-- 1 root root 2129683736 Dec 9 21:30 I323592.RCN
  14. -rw-r--r-- 1 root root 466735992 Dec 9 15:43 I323590.RCN
  15. -rw-r--r-- 1 root root 1022747296 Dec 9 07:45 I323588.RCN

Obviously I wanted to recover my data.
So I had files with a wrong filename on a filesystem but with a correct timestamp and probably the right filesize.
Luckily the mythconverg.recorded table also gives me lots of information about the files that mythtv had originally stored the content in.

  1. mysql> select basename,lastmodified,filesize from recorded limit 10;
  2. +---------------------------+---------------------+------------+
  3. | basename | lastmodified | filesize |
  4. +---------------------------+---------------------+------------+
  5. | 1003_20081003230000.mpg | 2008-10-07 21:53:49 | 6197765380 |
  6. | 1093_20080320232600.mpg | 2008-03-20 23:25:31 | 0 |
  7. | 1075_20060301191300.mpg | 2006-03-24 22:48:42 | 0 |
  8. | 1002_20080729160500.mpg | 2008-07-29 19:20:30 | 3679223940 |
  9. | 592251_20081217072000.mpg | 2008-12-17 07:20:02 | 0 |
  10. | 1002_20080911143500.mpg | 2008-09-11 16:41:44 | 3486101572 |
  11. | 1002_20080923143500.mpg | 2008-09-23 16:49:41 | 3679789684 |
  12. | 1033_20081110153500.mpg | 2008-11-10 15:47:12 | 338877000 |
  13. | 1002_20080922144000.mpg | 2008-09-22 16:47:38 | 3485505140 |
  14. | 1002_20080721160500.mpg | 2008-07-23 20:52:16 | 3679559444 |
  15. +---------------------------+---------------------+------------+
  16. 10 rows in set (0.00 sec)

My first idea was to use the mysql filesystem engine to query the filesytem and write me a simple query however I totally failed to build that engine :(
(Anyone else successfull here ? )

So I created a temp table

  1. CREATE TABLE `temp2` (
  2. `size` bigint(20) default NULL,
  3. `oldname` varchar(255) default NULL,
  4. `lastmod` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP
  5. ) ENGINE=MyISAM DEFAULT CHARSET=latin1

And parsed the content of my lost+found directory into a set of insert statements

  1. ls -l --time-style=long-iso | awk -F' ' '{print "insert into temp2 values (" $5 ",\"" $8"\",\""$6" "$7"\");"}'

From there is a matter of grabbing the matching filenames

  1. echo "select \"mv \" , oldname, basename from recorded, temp2 where temp2.size= recorded.filesize ;" | mysql mythconverg

And moving the actual files ... now all is back to normal ..

Dec 19 2008

MySQL 5.1 on Fedora 10

So I wanted a fresh MySQL 5.1 release on my Fedora 10 box.
Fedora comes with a mysql 5.0 package and the MySQL community packages for 5.1 conflict with that in different ways. (to start with MySQL vs mysql)

For some weird reason libmysqlclient (in mysql-libs) is required by redland, which is required by openoffice.org-core. I probaly never need the RDF funtionalities that redland provides me but I`m stuck with them anyhow.

So Fred pointed me to

Remi repo for mysql 5.1

  1. yum --enablerepo remi install mysql-server

Now gives me a very nice MySQL 5.1 to play around with.

Dec 19 2008

JoliCloud

Am I the only one who thinks the JoliCloud screenshot on TechCrunch really really looks like a default eeebuntu setup ?

Dec 19 2008

So True

So True

Dec 18 2008

Free Beer, planet.grep.be Meetup

planet.grep.be today has a lot of active open source users and contributors.
The weird thing is that we never meet apart from Fosdem.
Yes, we occasionally run into eachother at other events but there's not enough beer involved.

I've had different people ask me what and where are the Open Source gatherings in Belgium, and we must admit that apart
from Fosdem there isn't that much in our little country.

There were a couple of MySQL User Group events, some Drupal ones, some LUG had meetings altough I have the idea most of them have dried out :(

So I have this crazy idea of inviting you all to grab a beer, maybe even free beer :) on december 29 some in a pub in Antwerp

I`m open for suggestions on good locations.

Oh and everybody is welcome, both readers and writers :)

PS. Yes I know that some of you will be drinking in Berlin at that time .. we'll drink an extra one for you.

Dec 07 2008

CCK & Views, My First Experiences

and probably my last one ..

I mentioned a couple of weeks ago that I was looking into a way of converting my static page with published papers, articles and presentations in a more dynamic page where I could create RSS feeds from the updates and feed them into another site for everybody to use.

Some people suggested that I'd have a look at CCK & Views for this. While up till now when creating an app I had usually written my own Drupal module with my own database schema and manually written SQL Queries. This indeed looked like the perfect opportunity to dig into the CCK and Views thingie.

Now I must admit that I`m not really fond of "generated queries"

I've had nightmares before when having to debug 2Mb Hibernate queries that could be reduced to a single select value from table where query but Hibernate used a zillion join on tables I didn't need.
So I was a bit afraid of what Views would do ..

So here's what happened, I created 2 pages .. presentations. and publications. Personally I'd go for 1 table each with some smart indexes and then the only thing you want is SELECT * FROM table ORDER BY date

Now if I'd create a special content type for this I probably get some more benefits.
And I could get the data from the tables with a rather simple query like this :

  1. SELECT node.nid,
  2. content_type_presentation.field_datum_value,
  3. content_type_presentation.field_location_value,
  4. content_type_presentation.field_country_value ,
  5. content_type_presentation.field_event_link_title,
  6. content_type_presentation.field_event_link_url
  7. FRON content_type_presentation, node
  8. WHERE node.type="presentation" AND node.nid = content_type_presentation.nid
  9. ORDER BY content_type_presentation.field_datum_value desc;

However when using Views , this is what is being created.

  1. SELECT DISTINCT(node.nid) AS nid,
  2. node_data_field_country.field_datum_value AS node_data_field_country_field_datum_value,
  3. node_data_field_country.nid AS node_data_field_country_nid,
  4. node.type AS node_type,
  5. node_data_field_country.field_event_link_url AS node_data_field_country_field_event_link_url,
  6. node_data_field_country.field_event_link_title AS node_data_field_country_field_event_link_title,
  7. node_data_field_country.field_event_link_attributes AS node_data_field_country_field_event_link_attributes,
  8. node_data_field_country.field_location_value AS node_data_field_country_field_location_value,
  9. node_data_field_country.field_country_value AS node_data_field_country_field_country_value,
  10. node_data_field_link.field_link_url AS node_data_field_link_field_link_url,
  11. node_data_field_link.field_link_title AS node_data_field_link_field_link_title,
  12. node_data_field_link.field_link_attributes AS node_data_field_link_field_link_attributes,
  13. node_data_field_link.nid AS node_data_field_link_nid
  14. FROM node node
  15. LEFT JOIN content_type_presentation node_data_field_country ON node.vid = node_data_field_country.vid
  16. LEFT JOIN content_field_link node_data_field_link ON node.vid = node_data_field_link.vid
  17. WHERE UPPER(node_data_field_country.field_country_value) != UPPER('')
  18. ORDER BY node_data_field_country_field_datum_value DESC

So there's some joins, a zillion more fields that I don't really need being fetched from my database.
The explain also tells me some interesting things.. like the Using temporary AND Using Filesort

  1. | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
  2. +----+-------------+-------------------------+--------+---------------+---------+---------+-------------------------------------------+------+----------------------------------------------+
  3. | 1 | SIMPLE | node_data_field_country | ALL | PRIMARY | NULL | NULL | NULL | 33 | Using where; Using temporary; Using filesort |
  4. | 1 | SIMPLE | node | eq_ref |vid | vid | 4 |buytaert_blog.node_data_field_country.vid | 1 | |
  5. | 1 | SIMPLE | node_data_field_link | eq_ref | PRIMARY | PRIMARY | 4 | buytaert_blog.node.vid | 1 | |
  6. +----+-------------+-------------------------+--------+---------------+---------+---------+-------------------------------------------+------+----------------------------------------------+

Now my simpler query uses the same amount of rows to return the result but doesn't build a temporary table.

  1. +----+-------------+---------------------------+------+-----------------------+------+---------+---------------------------------------------+------+----------------+
  2. | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
  3. +----+-------------+---------------------------+------+-----------------------+------+---------+---------------------------------------------+------+----------------+
  4. | 1 | SIMPLE | content_type_presentation | ALL | nid | NULL | NULL | NULL | 33 | Using filesort |
  5. | 1 | SIMPLE | node | ref | PRIMARY,node_type,nid | nid | 4 | buytaert_blog.content_type_presentation.nid | 1 | Using where |
  6. +----+-------------+---------------------------+------+-----------------------+------+---------+---------------------------------------------+------+----------------+

In this small example there isn't much difference in performance to be noticed, but with bigger tables you will see other results. However I still prefer my personal query approach.

The other thing that bugs me is the excessive use of aliases. The MySQL internal experts can shed a light on the performance impact of using them, but if you are running queries remotely you at least have an increase in bandwidth usage. Not a problem now, but who knows later when you are big and famous :)

Now Drupal seems to do it pretty ok .. it's aliasing table.field to table_field which still makes sense. Not unlike some other tools that alias table.field to dynamically_generated_unreproducable_unreadable_key_word_with_no_relevant_meaning , try debugging that for a change. And yes I see valuable use for resultset aliasing, but I also se it is often overused.

But the short summary ... given my background I prefer writing the module and crafting my own queries over having Views create
a query for me.

Dec 07 2008

Puppy FUD

Dries commented on Seth reporting about the puppyfud some proprietary company not understanding open source was uttering.

It was a pretty lousy comparison actually .. they compared Open Source software to a Free Puppy.. it looks nice and cheap at first but it will cost you anyhow as it needs food and shots etc so it isn't really free after all.

Now the fact is that with the proprietary stuff they are trying to shove down their customers throats, their CMS's also need configuration, updates, and styling left and right, probably just as much as the Open Source alternative out there.
Now as they have a proprietary platform the chance that they have an equally big expert group around their product is small, people don't just buy software to learn it, however they will download a piece of interesting open source software to get aquanted with it.. There probably won't be that much free examples and tips and hints a round, or even free themes to base a site on ..

So no the purebred puppy really isn't the better alternative it's going to cost you equally in maintenance and you have to pay the high cost to actually be able to play with it .. or did you think it would be your software after you payed the pricetag ?

Now there's more bad news for the purebred software buyer. He isn't free to go to any vet, no , he has to go to a specialized one in his breed, actually if he doesn't do so once.. he risks never to be helped by another one in the future either.
And no choosing which food to buy for the puppy, no ther's just 1 brand that works, or maybe 2 .. but the prices have been set .. and it won't be cheap.

So what do you want .. the freedom to go to any vet you like, and to buy dogfood where you want , or even give them some of your own leftovers. Or the heavy pricetag, AND the limited choice in vets and food.