<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>ODM Technology &#187; Uncategorized</title>
	<atom:link href="http://blog.odmtech.com/category/uncategorized/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.odmtech.com</link>
	<description>Ord's Blog</description>
	<lastBuildDate>Sat, 04 Jul 2009 15:22:00 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>iPhone call forwarding indicator</title>
		<link>http://blog.odmtech.com/2009/07/04/iphone-call-forwarding-indicator/</link>
		<comments>http://blog.odmtech.com/2009/07/04/iphone-call-forwarding-indicator/#comments</comments>
		<pubDate>Sat, 04 Jul 2009 15:19:35 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[3.0]]></category>
		<category><![CDATA[3G]]></category>
		<category><![CDATA[call forwarding]]></category>
		<category><![CDATA[icon]]></category>
		<category><![CDATA[iPhone]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=100</guid>
		<description><![CDATA[Ever since I started using my iPhone, I wished it had a call forwarding indicator.  I typically give out my cell number as my main number when meeting people, and then forward the iPhone to wherever I am.  Sometimes, I forget to turn off the forwarding when I leave again, making me unreachable.
To [...]]]></description>
			<content:encoded><![CDATA[<p>Ever since I started using my iPhone, I wished it had a call forwarding indicator.  I typically give out my cell number as my main number when meeting people, and then forward the iPhone to wherever I am.  Sometimes, I forget to turn off the forwarding when I leave again, making me unreachable.</p>
<p>To my delight, I discovered that since upgrading the OS to 3.0 my iPhone 3G now indicates when the call forwarding is active with an icon on the status bar.</p>
<p>I haven&#8217;t seen this feature announced, so I am not sure if it is part of the overall 3.0 package, or if it is a feature specific to my carrier: Rogers Canada.    The screen shot below shows the phone with the indicator on.  <div id="attachment_104" class="wp-caption alignleft" style="width: 490px"><img src="http://blog.odmtech.com/wp-content/uploads/2009/07/cfd-on.jpg" alt="iPhone 3G on Rogers with call forwarding on" title="cfd on" width="480" height="236" class="size-full wp-image-104" /><p class="wp-caption-text">iPhone 3G on Rogers with call forwarding on</p></div></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/07/04/iphone-call-forwarding-indicator/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Riding with the iPhone</title>
		<link>http://blog.odmtech.com/2009/06/21/riding-with-the-iphone/</link>
		<comments>http://blog.odmtech.com/2009/06/21/riding-with-the-iphone/#comments</comments>
		<pubDate>Sun, 21 Jun 2009 20:01:39 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[google]]></category>
		<category><![CDATA[iPhone]]></category>
		<category><![CDATA[maps]]></category>
		<category><![CDATA[server]]></category>
		<category><![CDATA[ssh]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/2009/06/21/riding-with-the-iphone/</guid>
		<description><![CDATA[
I&#8217;m on a motorcycle trip this week, and left all technology behind&#8230; Except the iPhone.  It gives me the chance to get away without having to lug along a laptop and plan my trip around wifi hotspots. Before leaving, I subscribed to the Rogers data roaming plan. It&#8217;s still unreasonable, but it&#8217;s better that [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.flickr.com/photos/breakworld/2615136497/" title="V Star 1100 by Ord, on Flickr"><img src="http://farm4.static.flickr.com/3178/2615136497_c392b0bfd8.jpg" width="500" height="327" alt="V Star 1100" /></a></p>
<p>I&#8217;m on a motorcycle trip this week, and left all technology behind&#8230; Except the iPhone.  It gives me the chance to get away without having to lug along a laptop and plan my trip around wifi hotspots. Before leaving, I subscribed to the Rogers data roaming plan. It&#8217;s still unreasonable, but it&#8217;s better that the abuse I&#8217;d otherwise take for data roaming.  It makes email cheaper than text messaging while I am in the US.</p>
<p>It&#8217;s then end of the first day, and the iPhone has already proved itself.  First save came when one of my friend&#8217;s bikes broke down.  Google quickly gave us the location of the nearest Harley dealership, and their technician helped us with the diagnosis. A bit of tinkering and we were on our way.</p>
<p>A couple of hours later, I got a SMS telling me that there was a problem with one of my servers.  I launched the SSH client on my iPhone, logged into the server, found and fixed the problem. A few minutes later I was emailing my partner to let him know it was running.</p>
<p>Of course, the three other guys did make fun of me for not having the tip calculator app installed after lunch, but we were able to pay our bill anyway.</p>
<p>At stops for food and gas, I updated my facebook status so my wife could follow my progress.  Mapping is quicker and easier that it ever was with my handheld GPS.  The Wordpress app let&#8217;s me write and publish all this from my iPhone.  I even use it on occasion to make phone calls.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/06/21/riding-with-the-iphone/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>What&#8217;s right, now?</title>
		<link>http://blog.odmtech.com/2009/06/09/whats-right-now/</link>
		<comments>http://blog.odmtech.com/2009/06/09/whats-right-now/#comments</comments>
		<pubDate>Tue, 09 Jun 2009 14:12:14 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[information]]></category>
		<category><![CDATA[truth]]></category>
		<category><![CDATA[wikipedia]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=93</guid>
		<description><![CDATA[I recently heard that a study has shown that wikipedia was more accurate that Britanica as an encyclopedia. I don&#8217;t know the details of the study, but to me the important question is how do we define &#8220;right&#8221; when it comes to facts. 
Not so long ago it was a fact that the earth was [...]]]></description>
			<content:encoded><![CDATA[<p>I recently heard that a study has shown that wikipedia was more accurate that Britanica as an encyclopedia. I don&#8217;t know the details of the study, but to me the important question is how do we define &#8220;right&#8221; when it comes to facts. </p>
<p>Not so long ago it was a fact that the earth was the centre of the universe. Anyone who disagreed was considered wrong, or crazy, or both. At one time, it was &#8220;fact&#8221; that an atom was an indivisible component. Later we came to accept that electrons, protons and neutrons were the smallest particles, but we no longer believe that today.  In all cases, the scientific community and the world at large accepted these as truth. </p>
<p>At one time, a student who answered that light was composed of particles would have been told he was wrong. At other times those that said light was waves would be called wrong. The physics of light didn&#8217;t change, but our understanding did. </p>
<p>So how do we define right or true?  In some cases we can check for ourselves. I can count how many sides a cube has, and that becomes a fact to me, and absolute truth. However, in many practical matters we accept what the majority of the people we trust say to be the truth. As we are exposed to more and more information we increasingly encounter facts that we can&#8217;t check ourselves by simple observation.  The more specific or intricate the subject is, the less chance there is that someone I know and trust has direct knowledge of it. In these cases, truth becomes whatever the majority believes it to be (more specifically, those that have chosen to make their beliefs known through writing).  I may filter the information according to my perception of the reliability of the source, but in the end I am accepting it because the greatest number of credible people say so. </p>
<p>By defining truth this way, it isn&#8217;t surprising that a community edited encyclopedia proves to be more accurate than one edited by a limited staff. The one that conforms to the beliefs of the greatest number will be considered right by more people.   </p>
<p>This extends to the principle of using Twitter or other social networks as a search engine. If I define a relevant result for a search as the answer that most people who think like me believe, then I won&#8217;t be surprised that my network of contacts would give &#8220;better&#8221; results than an algorithm. The challenge will lie in building the right human networks for different queries. This isn&#8217;t a new concept &#8211; anyone who has asked a question on a discussion forum or newsgroup has used a human powered information system. The differences now are in the speed with which we can get feedback, and the number of people we can access. </p>
<p>Information is being updated and spread faster than ever. The &#8220;truth&#8221; as we understand it can change from day to day. I can no longer consider right and wrong as static conditions, facts need timestamps to be meaningful now.  </p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/06/09/whats-right-now/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Life without laptop</title>
		<link>http://blog.odmtech.com/2009/06/08/life-without-laptop/</link>
		<comments>http://blog.odmtech.com/2009/06/08/life-without-laptop/#comments</comments>
		<pubDate>Mon, 08 Jun 2009 19:51:33 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[iPhone]]></category>
		<category><![CDATA[ssh]]></category>
		<category><![CDATA[travel]]></category>
		<category><![CDATA[word press]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=92</guid>
		<description><![CDATA[I am on the road at a conference as I write this. Normally, I don&#8217;t leave home without a laptop &#8211; but since I was travelling by motorcycle and space is at a premium I left the notebook at home. This meant putting a lot of faith in my iPhone, and so far it has [...]]]></description>
			<content:encoded><![CDATA[<p>I am on the road at a conference as I write this. Normally, I don&#8217;t leave home without a laptop &#8211; but since I was travelling by motorcycle and space is at a premium I left the notebook at home. This meant putting a lot of faith in my iPhone, and so far it has paid off. </p>
<p>Of course, getting &#038; sending emails, voicemails and tweets is commonplace with the iPhone, but could I do any &#8220;real work&#8221; if needed?  Turns out the answer is yes. </p>
<p>A client contacted us about a change to an online project that was needed urgently. It wasn&#8217;t a big change but it did need to be done before I&#8217;d be back.  I quickly logged in to the server over ssh conection using TouchTerm (<a href="http://www.jbrink.net/touchterm/index.html">touchterm</a>) launched vi and edited the file. A quick check in the browser, and I was emailing the client back to say that the change had been made. </p>
<p>I&#8217;m still not sure if I&#8217;d be comfortable to be away for a few weeks without a laptop, but I won&#8217;t worry for a few days as long as I have the iPhone with me. </p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/06/08/life-without-laptop/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Open source sounds</title>
		<link>http://blog.odmtech.com/2009/03/21/open-source-sounds/</link>
		<comments>http://blog.odmtech.com/2009/03/21/open-source-sounds/#comments</comments>
		<pubDate>Sat, 21 Mar 2009 15:47:50 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=78</guid>
		<description><![CDATA[Sometimes I need a short sound clip for a software project &#8211; maybe a click when a user selects something, a buzzer or bell when a timer expires, or some special effect for opening and closing documents.  Making and recording sounds can be a real time sink, depending on the equipment and software that you [...]]]></description>
			<content:encoded><![CDATA[<p>Sometimes I need a short sound clip for a software project &#8211; maybe a click when a user selects something, a buzzer or bell when a timer expires, or some special effect for opening and closing documents.  Making and recording sounds can be a real time sink, depending on the equipment and software that you have available.</p>
<p>I will use <a href="http://flstudio.image-line.com/documents/what.html">FLStudio </a>with software synths to make some sounds or pieces of music.  The Sytrus hybrid synthesizer is usually my first choice, but recently I have been starting to use the Morphine additive synth.  Morphine works well for me if I have an existing sample or recording as a starting point.  If you are purchasing an image line product, consider using my <a href="http://affiliate.image-line.com/HCAJDB412">affiliate link</a> &#8211; you will save 10% on your purchase, and I can earn discounts on my future purchases.</p>
<p>Although I really enjoy working with sounds, I can easily spend hours playing around.  Sometimes, I&#8217;m better off just finding exactly what I want that someone else has already made.  For public domain sounds, <a href="http://www.freesound.org">FreeSound.org</a> has a great collection.  These are available for use under the creative commons license; in effect you need to attribute the work to the creator in a way that doesn&#8217;t imply an endorsement.</p>
<p>Recently I found that <a href="http://www.istockphoto.com">iStockPhoto</a> now also offers sound files in addition to images.  Here you can buy royalty free sound clips for use in any project.  I have used iStock for both buying and selling photos, and it looks like their interface for sound files is as good as the one for images.  So far their library isn&#8217;t as extensive as it could be, but it seems to be growing quickly.</p>
<p>For editing sound files and converting formats, I&#8217;ve grown pretty attached to <a href="http://audacity.sourceforge.net/">Audacity</a>.  This is an open source sound editor, that does pretty much everything I need or could imagine needing.  It handles multiple tracks, so it&#8217;s easy to layer sounds together to get a special combination.  It includes plenty of effects, from the standard amplitude and frequency changes to echoes, phasers and wah type effects.  The noise removal and normalizing functions work well, and make it very easy to clean up samples.</p>
<p>Using royalty free samples and open source editing software, I can now get sounds done in a few minutes.  It&#8217;s not as much fun as building them from scratch with softsyths, but the time savings usually makes up for that.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/03/21/open-source-sounds/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>iPhone sync with Google Calendar</title>
		<link>http://blog.odmtech.com/2009/03/08/iphone-sync-with-google-calendar/</link>
		<comments>http://blog.odmtech.com/2009/03/08/iphone-sync-with-google-calendar/#comments</comments>
		<pubDate>Sun, 08 Mar 2009 06:12:05 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=70</guid>
		<description><![CDATA[I had been looking at different iPhone syncing options for my contacts, and it seemed that to do anything with a PC was going to involve using Outlook, which I was reluctant to install.  In general, I try to use only open source applications, ones which run under Windows and Linux (and if there [...]]]></description>
			<content:encoded><![CDATA[<p>I had been looking at different iPhone syncing options for my contacts, and it seemed that to do anything with a PC was going to involve using Outlook, which I was reluctant to install.  In general, I try to use only open source applications, ones which run under Windows and Linux (and if there is a Mac port too, even better).  This was I don&#8217;t have to remember different ways of doing things when booting under the other OS, and files don&#8217;t need converting.</p>
<p>Instead, I found the Beta of <a href="http://www.google.com/mobile/apple/sync.html">Google Sync for iPhone</a>.  This works so well, I was amazed.  I have already used Google Calendar (my wife and I share one for keeping each other up to date with our appointments), but I never bothered syncing it with anything.  The iPhone sync is outstanding &#8211; changes made through the web appear in my contacts and calendar instantly, and vice versa.   Now there are several options for syncing Google Calendar and Contacts with Thunderbird, so if I want an offline solution for my laptop or desktop I can go that way too.</p>
<p>Apple&#8217;s Mobile Me looks interesting also, although again for the PC it seems to support only Outlook.  There is currently a <a href="http://www.apple.com/mobileme/">free 60 day trial</a> of Mobile Me available from Apple, so it doesn&#8217;t hurt to try that out too.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/03/08/iphone-sync-with-google-calendar/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>Geocoding in PHP with Microsoft&#8217;s API</title>
		<link>http://blog.odmtech.com/2009/02/24/geocoding-in-php-with-microsofts-api/</link>
		<comments>http://blog.odmtech.com/2009/02/24/geocoding-in-php-with-microsofts-api/#comments</comments>
		<pubDate>Tue, 24 Feb 2009 22:54:58 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=40</guid>
		<description><![CDATA[Yesterday I looked at using the Google Map API to geocode some addresses &#8211; today I examined the same using Microsoft&#8217;s API.
The Microsoft API is somewhat more complicated to use, as it is a SOAP API we can&#8217;t just fetch and parse HTTP as we could with Google.
To use these services with PHP, we need [...]]]></description>
			<content:encoded><![CDATA[<p>Yesterday I looked at using the Google Map API to geocode some addresses &#8211; today I examined the same using Microsoft&#8217;s API.</p>
<p>The Microsoft API is somewhat more complicated to use, as it is a SOAP API we can&#8217;t just fetch and parse HTTP as we could with Google.</p>
<p>To use these services with PHP, we need SOAP functions.  The server I had available to test had an old version of PHP (4.4), which doesn&#8217;t have this functionality built in.  It is easily added through the <a href="http://sourceforge.net/projects/nusoap/">nusoap package</a>.  Even with a later version of PHP that inclused SOAP functionality we might want to use nusoap anyway.</p>
<p>Microsoft has a <a href="http://msdn.microsoft.com/en-us/library/ms980207.aspx">technical article</a> showing exactly how to geocode an address using  their MapPoint web services in PHP.  I think it&#8217;s actually the only piece of PHP sample code in the whole Virtual Earth documentation, and it happens to be for Geocoding &#8211; lucky me.   The code as is runs fine, and it&#8217;s easy to understand.</p>
<p>Before doing any of this, you will need to get an evaluation developers account opened.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/02/24/geocoding-in-php-with-microsofts-api/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Twittering</title>
		<link>http://blog.odmtech.com/2009/02/22/twittering/</link>
		<comments>http://blog.odmtech.com/2009/02/22/twittering/#comments</comments>
		<pubDate>Sun, 22 Feb 2009 17:12:56 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=25</guid>
		<description><![CDATA[
Ok, so after hearing about twitter for a while, I decided to try it out.  As I&#8217;m not normally particularly interested in communicating with anyone, the idea of broadcasting my comings and goings to the world at large seemed a little odd.  I did however go ahead and set up an account (@ordmillar), and all [...]]]></description>
			<content:encoded><![CDATA[<p><a title="Two birds by Ord, on Flickr" href="http://www.flickr.com/photos/breakworld/77657535/"><img src="http://farm1.static.flickr.com/37/77657535_13a1be8265.jpg" alt="Two birds" width="500" height="333" /></a></p>
<p>Ok, so after hearing about twitter for a while, I decided to try it out.  As I&#8217;m not normally particularly interested in communicating with anyone, the idea of broadcasting my comings and goings to the world at large seemed a little odd.  I did however go ahead and set up an account (<a href="http://twitter.com/ordmillar">@ordmillar</a>), and all of a sudden there I was, tweeting away!</p>
<p>What makes it really interesting (apart from the people) is the <a href="http://apiwiki.twitter.com/">twitter api</a>.  Pretty much everything is exposed, so apps can do just about anything that a user could do through the web site.  I&#8217;ve been trying out some of the clients made with the api, but so far none of them has really impressed me.  The closest to what I want is one called <a href="http://www.alertthingy.com/">Alert Thingy</a>, which is nice in that it will show notices from twitter, facebook, flikr, RSS feeds and more.  It has worked well, execpt for one unexplained crash.  The flickr feed isn&#8217;t great &#8211; it sends a notice whenever anyone posts a picture, but doesn&#8217;t show any sort of preview so I am forced to visit the site if I want to see if I am interested in it.</p>
<p>With so many platforms out there, and so many social networking sites I can see that there is a need for the &#8220;grand unification tool&#8221;.  Almost like the grand unification theory of physics, execpt that the universe won&#8217;t end if we get it wrong.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/02/22/twittering/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>Scraping with Solvent</title>
		<link>http://blog.odmtech.com/2009/02/21/scraping-with-solvent/</link>
		<comments>http://blog.odmtech.com/2009/02/21/scraping-with-solvent/#comments</comments>
		<pubDate>Sat, 21 Feb 2009 22:57:30 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=23</guid>
		<description><![CDATA[The Solvent extension for FireFox lets us write  screen scrapers that turn a web page into RDF data.  The data in this format is intended to feel into their Piggy Bank package, but for my purposes I will be using the generated scraping script to covert data read by the Crowbar application into RDF.  From [...]]]></description>
			<content:encoded><![CDATA[<p>The <a href="http://simile.mit.edu/wiki/Solvent">Solvent extension</a> for FireFox lets us write  screen scrapers that turn a web page into RDF data.  The data in this format is intended to feel into their Piggy Bank package, but for my purposes I will be using the generated scraping script to covert data read by the Crowbar application into RDF.  From there, it becomes easy to convert it to any format I need and save it to the database.</p>
<p>Unfortunately Solvent requires Piggy Bank, which doesn&#8217;t work with FireFox 3.  Not being willing to remove my 3.0.6 installation, I needed a way to get a version 2 installed at the same time.  I found some simple instructions:  <a href="http://www.troyrutter.com/2008/06/18/firefox-2-and-3-living-together-in-harmony.html">Firefox 2 and 3 Living Together in Harmony</a>.  My install was slightly different than the one described because I wanted to keep my existing profile for version 3 and use the newly created one for version 2.   Also, I did this with the latest version of 2.x, which you can find at <a href="http://www.mozilla.com/en-US/firefox/all-older.html">www.mozilla.com/en-US/firefox/all-older.html</a>.</p>
<p>After making a test scraper and uloading to a web server, I tried it with <a href="http://simile.mit.edu/wiki/Crowbar">Crowbar</a> using the -mode=scrape option, and sure enough it returned an RDF/XML document.</p>
<p>There are a few bugs with Crowbar &#8211; the one that immediately caused me problems was non-encoded ampersand characters.  It&#8217;s otherwise such a useful tool that I will definitely be spending some time on it.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/02/21/scraping-with-solvent/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>HTML Parsers</title>
		<link>http://blog.odmtech.com/2009/02/20/html-parsers/</link>
		<comments>http://blog.odmtech.com/2009/02/20/html-parsers/#comments</comments>
		<pubDate>Fri, 20 Feb 2009 16:11:43 +0000</pubDate>
		<dc:creator>Ord</dc:creator>
				<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://blog.odmtech.com/?p=19</guid>
		<description><![CDATA[Once again, I find that I need to parse some HTML pages.  I&#8217;ve looked at difference parsers in the past, and always ended up writing some string or pattern matching routines to get the data I wanted off the page.  This time, the data I want isn&#8217;t actually on the HTML pages, it gets written [...]]]></description>
			<content:encoded><![CDATA[<p>Once again, I find that I need to parse some HTML pages.  I&#8217;ve looked at difference parsers in the past, and always ended up writing some string or pattern matching routines to get the data I wanted off the page.  This time, the data I want isn&#8217;t actually on the HTML pages, it gets written by a Javascript that is included from a third party.  So, I need a parser/fetcher that understands Javascript, can load files, and can wait for the scripts to finish running before giving me the output.</p>
<p>The <a href="http://lobobrowser.org/cobra.jsp">Cobra toolkit</a>, part of the <a href="http://lobobrowser.org/">Lobo Browser</a> project looked ideal.   The toolkit itself looked great, and was fairly easy to get set up and testing.  It generally fetched and parsed pages as I would expect, although often with a lot of warnings.  Unfortunately, the pages I was actually wanting to parse didn&#8217;t work.  I decided to try the whole Lobo Browser to see how it would display those pages, but the installer gave a permissions error at the beginning of the install.  I decided to put this aside for now.  When I have to start troubleshooting installers so that I can test a program to find out if a library can do what I want, I see a huge risk of lost time.</p>
<p>The Mozilla Parser looked like another good option.  This package is essentially a java interface to the engine that is used in the Firefox browser.  Since I knew that Firefox rendered the pages I wanted properly, this seemed like a good choice.   Again I had some issues getting this installed onto the 64 bit vista platform.  The documentation is limited, and the project seems to have not been touched in over a year.  Rather than spend a lot of time to find out why that wasn&#8217;t running, I looked at other options for using the Mozilla engine.</p>
<p><a href="https://developer.mozilla.org/en/Gecko">Gecko</a>, the engine behind Mozilla&#8217;s browsers has a lot of information about embedding it.   There is a whole SDK available, and if it were a bigger project I would probably take this route.  They also provide XULRunner, which allows running of XUL applications outside of the browser.  (Firefox 3 can also be used to run XUL on windows, by invoking with the -app parameter)</p>
<p>The <a href="http://simile.mit.edu/wiki/Crowbar">Crowbar</a> program is an XUL app that fetches and parses web pages, and as it uses the same Firefox engine, it does parse the pages I need without any problem.  This runs as a server, it accepts HTTP requests and returns the data.  I will do some work with Crowbar, it looks like just a few small changes will make it the ideal tool for this project.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.odmtech.com/2009/02/20/html-parsers/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>
