Everything is a Freaking DNS problem - monitoringlove http://127.0.0.1:8080/blog/taxonomy/term/1491/0 en Love, MonitoringLove http://127.0.0.1:8080/blog/love-monitoringlove <p>Last year we were pretty negative about Monitoring, We shouted out that MonitoringSucked ... A year has passed and a lot has changed ... most importantly our new found love for monitoring, thanks to an inspirational Ignite talk by <a href="https://twitter.com/ulfmansson" rel="nofollow">Ulf Mansson</a> at devopsdays Rome.</p> <p>Right after Fosdem about 20 people showed up at the #monitoringlove hacksessions hosted at the <a href="http://www.inuits.eu/" rel="nofollow">Inuits.eu</a> offices to work on Open Source monitoring projects and exchange ideas. Some completely new people, some people with already a lot of experience.</p> <p>Amongst the projects that were worked on was Maciej working on Packaging graphite for Debian, Ohter people were fixing bugs in Puppet , I spent some time with a <a href="https://github.com/krisbuytaert/vagrant-sensu" rel="nofollow">vagrant box</a> to deploy Sensu using Puppet. Last time I was playing with Sensu was on the flight back from PuppetCon , I gave up the fight with<br /> RabbitMQ and SSL because I had no internet connection .. and now Ulf just pointed out that I could disable SSL at all, which resulted in having a POC up and running in no time.</p> <p>Patrick was hacking on the Chef counterpart of the vagrant-puppet sensu setup a part of <a href="https://github.com/monigusto" rel="nofollow">#monigusto</a>. Ulf Mansson was getting <a href="http://shopify.github.com/dashing/" rel="nofollow">dashing</a> to display on a Raspberry Pi ... pretty cool stuff<br /> And Jelle Smet was working on <a href="https://github.com/smetj/pyseps" rel="nofollow">Pyseps</a> a Python based Simple Event Processing Server framework that consume JSON docs from RabbitMQ and forwards them real time to other queues using MongoDB query syntax.</p> <p>One of the more interesting discussion was around the topic of alerting and modeling business rules and input from a lot of different sources<br /> in order to send the right alerts to the right people. </p> <p>We explored different ideas like using BPM tools such as Activity or Rules engines like Ruby Rools. There exist some Saas providers that try to solve this need like PagerDuty and friends but obviously there is still a lot of work that needs to be done in order to create a viable alerting system based on different input sources.</p> <p>The monitoring problem is not solved yet .. and it will stay around for a couple of years .. but with the advent of event such as <a href="http://monitorama.com/" rel="nofollow">Monitorama</a> its clear<br /> that an event like our #monitoring love hackessions is needed .. and is probably here to stay for a couple of years.</p> http://127.0.0.1:8080/blog/love-monitoringlove#comments devops infracoders monitoringlove monitoringsucks puppet sensu Wed, 13 Feb 2013 17:10:32 +0000 Kris Buytaert 1077 at http://127.0.0.1:8080/blog check_graphite http://127.0.0.1:8080/blog/checkgraphite <p>During my Puppetcamp Gent talk last week, I explained how to get alerts based on trends from graphite. A number of people asked ,e how to do that.</p> <p>First lets quickly explain why you might want to do that .<br /> Sometimes you don't care about the current value of a metric..as an example take a Queing system .. there is no problem if there are messages added to the queue, not even if there are a lot of messages on the queue, there might however be a problem if over a certain period the number of messages on a queue stays to high.</p> <p>In this example I`m monitoring the queue length of a hornetq setup which is exposed by JMX.<br /> On the server runnnig HornetQ I have an exported resource that tells the JMXTrans server to send the MessageCount to graphite<br /> (you could also do this using collectd plugins) </p> <p><div class="geshifilter"><pre class="text geshifilter-text" style="font-family:monospace;"><ol><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">@@jmxtrans::graphite {&quot;MessageCountMonitor-${::fqdn}&quot;:</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> jmxhost =&gt; hiera('hornetqserver'),</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> jmxport =&gt; &quot;5446&quot;,</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> objtype =&gt; 'org.hornetq:type=Queue,*',</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> attributes =&gt; '&quot;MessageCount&quot;,&quot;MessagesAdded&quot;,&quot;ConsrCount&quot;',</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> resultalias =&gt; &quot;hornetq&quot;,</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> typenames =&gt; &quot;name&quot;,</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> graphitehost =&gt; hiera('graphite'),</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal"> graphiteport =&gt; &quot;2003&quot;,</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">}</div></li></ol></pre></div></p> <p>This gives me a computable url on which I can get the graphite view </p> <p>The next step then is to configure a nagios check that verifies this data. For that I need to use the <a href="http://github.com/datacratic/check_graphite" rel="nofollow">check_graphite</a> plugin from Datacratic ..</p> <p>Which can work with an nrpe config like<br /> <div class="geshifilter"><pre class="text geshifilter-text" style="font-family:monospace;"><ol><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">### File managed with puppet ###</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">### Served by: '&lt;%= scope.lookupvar('::servername') %&gt;'</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">### Module: '&lt;%= scope.to_hash['module_name'] %&gt;'</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">### Template source: '&lt;%= template_source %&gt;'</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">&nbsp;</div></li><li style="font-family: monospace; font-weight: normal;"><div style="font-family: monospace; font-weight: normal; font-style: normal">command[check_hornetq]=/usr/lib64/nagios/plugins/check_graphite -u &quot;http://&lt;%= graphitehost%&gt;/render?target=servers.&lt;%= scope.lookupvar('::fqdn').gsub(/\./,'_')%&gt;_5446.hornetq.docstore_private_trigger_notification.MessageCount&amp;from=-30minutes&amp;rawData=true&quot; -w 2000 -c 20000</div></li></ol></pre></div></p> <p>I define this check on the host where HornetQ is running as it then will map to that host on Icinga/Nagios rather than throw a host error on an unrelated host.</p> http://127.0.0.1:8080/blog/checkgraphite#comments graphite icinga monitoringlove monitoringsucks puppet Tue, 05 Feb 2013 09:10:15 +0000 Kris Buytaert 1076 at http://127.0.0.1:8080/blog #devops Conference Season heating up... http://127.0.0.1:8080/blog/devops-conference-season-heating <p>My last post has been a while ... in that I announced that there would be another event right before FOSDEM ... I totally forgot to announce it here but I`m sure that most of you already know. Yes. PuppetCamp Europe is coming back to it's roots... it's coming back to the city where we hosted it for the first time on this side of the ocean.. Gent. (that's 31/1 and 1/2 ) </p> <p>There is still time to register for the event <a href="http://puppetcampghent2013.eventbrite.com/" rel="nofollow">http://puppetcampghent2013.eventbrite.com/</a> The schedule for the event will be published soonish (given that the selection was done on Friday evening and the speakers already received their feedback)</p> <p>Co-located with PuppetCamp there will another Build and Open Source cloud day<br /> <a href="http://buildacloud.org/about-cloudstack/cloudstack-events/viewevent/140-build-a-cloud-day-ghent-2013.html" rel="nofollow">Build a Cloud day</a> with interesting topics such as Cloudstack, Ceph, devops and a really interesting talk on how the Spotify crowd is using Cloudstack.</p> <p>So after those 2 days in Ghent, a lot of people will be warmed up for the open source event of the year FOSDEM.</p> <p>And right after FOSDEM a bunch of people will gather at the Inuits office for 2 days of discussing, hacking and evangelizing around #monitoringlove (see previous post)</p> <p>I almost forgot but even before the FOSDEM week-long there is the <a href="http://conference.phpbenelux.eu/2013/" rel="nofollow">2013 PHP Benelux Conference</a> where I`ll be running a fresh version of the 7 Tools for your devops stack </p> <p>There is a ****load of #DevopsDays events being planned this year .... the 2012 edition of New York will be taking place next week .<br /> Austin and London have been announced and have opened up their CFP and Registration but different groups are organizing themselves to host events in Berlin, Mountain View, Tokyo, Barcelona, Paris, Amsterdam , Australia , Atlanta and many more .. </p> <p>And there's even more to come .. April 6 and 7 will be the dates for the <a href="http://loadays.org/" rel="nofollow">Linux Open Administration Days</a> (Loadays 2013) in Antwerp again ... a nice small conference where people gather to discuss different interesting Linux topics .... Call For Presentations is still open ..<a href="http://loadays.org/pages/cfp-info.html" rel="nofollow">Submit here</a></p> <p>On the other side of the ocean there's <a href="http://portland2013.drupal.org/" rel="nofollow">DrupalCon Portland</a> which once again is featuring a #devops track , and also the folks over at <a href="http://agile2013.agilealliance.org/" rel="nofollow">Agile 2013</a> (Nashville)<br /> have a #devops track now. Both events are still looking for speakers .. </p> <p>So if by the end of this year you still don't know what devops is all about .. you probably don't care and shouldn't be in the IT industry anyhow.</p> <p>And those are only the events I`m somehow involved in for the next couple of months</p> http://127.0.0.1:8080/blog/devops-conference-season-heating#comments agile devops devopsdays drupal fosdem monitoringlove phpbenelux puppet puppetcamp puppetzie Sun, 13 Jan 2013 10:21:11 +0000 Kris Buytaert 1075 at http://127.0.0.1:8080/blog #monitoringlove hackfest http://127.0.0.1:8080/blog/monitoringlove-hackfest <p>The age of #monitoringsucks is over, we're now transitioning into a #monitoringlove period. </p> <p>That however doesn't mean al the work is done, we still need to do a lot of work and a lot of people are working on a lot of stuff.</p> <p>Therefore like last year we are opening up our offices again right after Fosdem for a #monitoringlove hackfest</p> <p>That's right on february 4 and 5 a bunch of people interrested to fix the problem will be meeting , discussing and hacking stuff together in Antwerp. In short a #monitoringlove hackathon</p> <p>Inuits is opening up their offices for everybody who wants to join the effort Please let us (@KrisBuytaert) know if you want to join us in Antwerp. We'll provide caffeine, wireless, chairs and some snacks. </p> <p>Please register upfront at : <a href="http://monitoringlove2013.eventbrite.com/" rel="nofollow">http://monitoringlove2013.eventbrite.com/</a></p> <p>Obviously if you can't make it to Antwerp you can join the effort on ##monitoringsucks on Freenode or on Twitter.</p> <p>The <a href="http://www.inuits.eu/contact" rel="nofollow">location will be Duboistraat 50 , Antwerp</a><br /> It is about 10 minutes walk from the Antwerp Central Trainstation<br /> Depending on Traffic Antwerp is about half an hour north of Brussels and there are hotels at walking distance from the venue.</p> <p>Plenty of parking space is available on the other side of the Park </p> <p>Read last years report <a href="http://www.krisbuytaert.be/blog/we-didnt-fix-it">http://www.krisbuytaert.be/blog/we-didnt-fix-it</a> to get an idea of what will happen...</p> <p>PS. Yes I`m trying to get another event of the ground the days before Fosdem but I`m still awaiting confirmation of the venue ..</p> http://127.0.0.1:8080/blog/monitoringlove-hackfest#comments fosdem inuits monitoringlove monitoringsucks opensource Tue, 13 Nov 2012 20:54:24 +0000 Kris Buytaert 1074 at http://127.0.0.1:8080/blog Linuxcon Europe, Barcelona http://127.0.0.1:8080/blog/linuxcon-europe-barcelona <p>I`m in Barcelona this week for LinuxCon Europe ..<br /> I`ll be hosting 3 sessions this week.</p> <p>Today I`ll be helping <a href="https://twitter.com/sebgoa" rel="nofollow">Sebastien Goasguen</a> with his session <a href="http://linuxconeurope2012.sched.org/event/8816c31538f6278779459b3337a45559#.UJi--tHtBrc" rel="nofollow">Building Foss Clouds</a>, I`ll be talking about how to use tools such as Puppet, Graphite and Icinga to monitor your clouds. Sebastian inherited the Building FOSS Clouds tutorial last week and asked me to help out. </p> <p>Tomorrow I`ll be giving my <a href="//linuxconeurope2012.sched.org/event/cd9a06295026ac8d17fb51cc26ff2ff2#.UJjD2NHtBrc" rel="nofollow">11 Open Source Tools for your Devops Stack </a> talking.</p> <p>And on thursday I will be giving a more general "What's this #devops thing anyhow" talk during the <a href="http://cloudstack-linuxconeu.eventbrite.com/" rel="nofollow">Cloudstack</a> event following LinuxCon.</p> http://127.0.0.1:8080/blog/linuxcon-europe-barcelona#comments devops linuxcon monitoringlove puppet Tue, 06 Nov 2012 08:12:51 +0000 Kris Buytaert 1073 at http://127.0.0.1:8080/blog