Opie now supported in Angstrom

Opie 1.2.3 is now supported in the Ångström Linux distribution. Check out the announcement.

Woot! :)

IntelliSync testing

I had a go with IntelliSync (the SL version) connecting to Opie on my Zaurus this evening. I found that despite being a little hard to find, IntelliSync is in fact still available for download, which is good because the CDs that came with my Zaurus are in Japanese only. Thankfully the installer does not require any licence keys or device presence in order to install.

The next thing was to get the USB network connection working in Windows. I plugged in my Zaurus, Windows popped up the usual dialogs and I fed it the usual linux.inf file which it accepted. Unfortunately though after a delay the result was the error "This device cannot start. (Code 10)". There are numerous references to this error message on the web, but I couldn't find a satisfactory solution amongst any of them. (Of course USB networking between my Z and my laptop when running Linux works perfectly.)

I decided to simply press on and try to get IntelliSync to connect to my Z over WiFi. Despite only having USB and Serial options for the connection type and it insisting on not allowing the IP address of my Z (192.168.0.5) I was finally able to coerce it into connecting by specifying an address of the same length that it would allow (eg. 192.168.1.1), and then editing the file NetStat.dat with a hex editor to change the address. If you then go to the connection settings and click OK the change should be permanent.

This is not enough to actually get it synchronising however - it's obviously connecting, however all you get is a quick flash of the sync screen on the Opie side and then an error with the suggestion to "check the application version" from IntelliSync. Presumably this means that Opie is not sending the responses that IntelliSync expects. I suppose I will have to analyse the traffic in order to fix this.

Opie

I've been doing a lot of work on Opie lately. So far I have managed to achieve the following:

* Fixed Konqueror-Embedded (albeit the 20070316 version) to work with the nice new GUI. I have issues building it from OE .dev though and SSL doesn't seem to be working, so there is work to be done yet. Getting the 20080319 version working would be nice (currently segfaults on startup). A lot of people have asked for this one so it's nice to make some progress.

* Improvements to the new opie-cardapplet. It should now manage removeable cards with ease and won't let you do anything silly like ejecting the card your system is running from :)

* Fixed *a lot* of minor bugs for the 1.2.4 release, which is close at hand.

* Fixed a few package building issues in OE, mostly for packages outside the core set built for images (including third-party applications). I still haven't got Portabase or QPDF2 building though, and I haven't even looked at kopi/kapi yet.

* Tested and pushed for Opie 1.2.3 to be supported in Angstrom. We're almost there, the "machine mentors" just need to get into gear and accept the already built images.

I also bought a Sharp Zaurus SL-C3000 from Japan on eBay, which has given me not only a new burst of inspiration but also some extra items for my task list (mainly to do with improving Opie's high-resolution support), and given me another machine to test Opie with. It's really quite a nice device - great screen, usable keyboard, CF, SD, and USB host. Built-in bluetooth and wifi would be nice, but at least you can get those with an additional CF card.

Next thing is to get back into sorting out the syncing situation. Opensync has moved on a little bit lately and I need to catch up. I would also like to test the copy of Intellisync that I received with my Zaurus - OK, it's commercial software, but lots of Zaurus users will still have it and no doubt expect it to work. With any luck it should just work out of the box.

LRL '08

So, LugRadio is ending :( but I'm going to LugRadio Live :)

Should be entertaining.

Hello from sunny England

So I've been in the UK now for about three and a half months. There's tons of things going on here at the moment and so far I'm quite happy with my decision to move. Of course I miss NZ, my family & friends etc., but this is my chance to see the world. On that front so far I've been to Wales for a few days (freezing cold) and Rome for a week (awesome), and I'm off to some mystery destination for a weekend soon. For longer holidays I'm limited by work for the next few months but I have a few places on my list for after that.

I stayed with friends and relatives for a bit initially, but before too long I needed to find a job and a place of my own. I found a job in Croydon (doing Delphi still) and a flat in Brockley (south east London) which was about the right price and convenient for getting the train to work. Public transport here is pretty good but it's damned expensive - more so than anywhere else in Europe apparently. I miss my car already :/ (although petrol is also damned expensive and shows no sign of getting any cheaper). At least I got paid in pounds for the first time the other day :)

The Big OE

In less than a week now I am leaving New Zealand to go to the UK, for what us Kiwis refer to as "the big OE" (OE being Overseas Experience). The UK makes a good base for taking trips to Europe, which is what I plan to do in addition to working (most likely in London) for the next 1-2 years. Several of my friends are already in the UK and I have plenty of relatives there so at least I won't arrive in the country knowing nobody. I'm sure I'll miss home - the nice weather, having my own car, and of course being able to see my family and friends. Still, you don't find adventure on your own doorstep as someone once told me. Very true.

This week is already shaping up to be a busy one at work, winding things down. In many ways it's sad to be leaving there - I've been there just over six years now, and they are like family to me. I would have liked to get to know several new team members who joined recently better as well. There are plenty of fine people there to pick up where I'm leaving off though, and I've left them with documentation that should suffice when I'm not there to answer questions :)

Being the procrastinatory individual that I am, I still have a list of things yet to do before I go. I haven't packed yet, although I have most of the things other than clothes that I plan to take with me (much less stuff than I expected, actually). Four days left in the week - should be plenty of time ;)

In case anyone is wondering I will still be involved in my various projects (Opie, OpenSync, etc.) - being that these are run over the internet I can still participate wherever I am in the world. In some ways it will make things easier - a lot of the open source / free software conferences are in Europe anyway.

Where did my collection go?

So I was trying to move my music collection in Amarok to a different machine. I simply moved all of the MP3 files from machine A to machine B, taking care to preserve the same path structure, and then copied the configuration files (~/.kde/share/config/amarokrc and ~/.kde/share/apps/amarok/* - for clarity, I'm using sqlite backend to store the collection). Then I opened Amarok on the second machine, only to find that my collection was empty. Now, this is not a situation that I'm unfamiliar with - the same thing happened one time when I upgraded Amarok, although somehow I managed to recover from that one. What confused me initially though was that I thought it might not be successfully reading the collection database file; however, after some searching and investigation I figured out what the real problem is and how to solve it.

Digging into the data structure of Amarok's collection.db (by running "sqlite3 ~/.kde/share/apps/amarok/collection.db"), it seems that Amarok records the device where each track is stored so that you can have tracks on removable devices, and they appear when you plug the removable device in and disappear when you unplug it. When running Amarok with the same collection data on another machine, the device that it thought contained all of my tracks (ie, one of my hard drive partitions) no longer existed and thus the collection was shown as empty.

Once you know what's going on here it's not hard to fix the problem, simply by fixing up the device links. In fact, in collections created in newer versions of Amarok (at least in 1.4.8 - my collection has been around for quite a few versions now) it seems to put the files against no device at all (device_id of -1). A few SQL statements later and I had simply cleared out the contents of the devices table and set all of the device_id values in other tables to -1 instead of a specific device, and voila - the next time I started Amarok my collection was restored :)

It seems to me though that there ought to be a nicer way of exporting your collection so you don't have to muck around like this. It might make a good summer of code project for someone... ;)

Angstrom Opie 1.2.3 testing

Testing images of the Ångström distribution containing Opie 1.2.3 are now available for many devices, see this page for details. If your device is listed, we would welcome your testing and feedback.

This has been a long time coming, and I'd like to thank everyone who helped make it possible. Of course this is just a step towards Opie becoming supported in Angstrom - we'll allow at least a month for people to test and then we'll evaluate the situation, and possibly after further tweaking be able to release final images, populate feeds and finally declare Opie as supported.

Meanwhile, over the holidays I have been making quite a few fixes in Opie CVS to be included in 1.2.4 - many of them fix bugs that have annoyed me and no doubt others for a while now, which is very satisfying. Included were changes to make VMemo actually usable, many small improvements in AdvancedFM, better alarm sound playback in the Clock, and some others. Once Opie 1.2.3 becomes supported in at least one distribution we can start to think about doing the 1.2.4 release.

LCA 2008

Kind of at the last minute I decided to go to linux.conf.au in Melbourne next January. I missed the last earlybird ticket by mere hours, but oh well. As usual there are plenty of interesting talks on, a few people to meet up with and a new city to check out as well. Flights, accommodation and the conference itself are all booked and paid for, so now all I need to do is wait. Should be an interesting trip :)

Wake me up before you go-go

I was playing around with the Opie clock today and I discovered to my dismay that alarm wakeup is not working at all on my iPAQ h2200 running a self-built Angstrom + Opie image (2.6.21-hh17 kernel). It turns out that most of the pieces of the system are working fine up until the point when opiealarm (which gets run just before suspend) tells the kernel to set the rtc wakeup time. According to the rtc info in /sys it is being set to a date somewhere in 2035, which is not particularly useful especially if you really do want to be woken up early for that meeting tomorrow morning.

I have looked at the code in opiealarm and it looks OK to me, and hasn't been changed in a long time. It is using the old ioctl() based method of setting the alarm but that really ought to work properly. I suspect this is a bug in the kernel, but with the handhelds.org kernel discussion mailing list being down I'm not sure where to report it. Setting the wakeup time through /sys does work so at least that's something - if it comes to it I suppose I could modify opiealarm to use that method if it is available.

CMake in OpenSync, Opie 1.2.5?

So I did some more work on OpenSync/Opie lately - fixed a few issues in Opie's sync authentication code, finally changed the todo list to show some reasonable icons, etc. The most interesting change for me though was to do with CMake - OpenSync has just moved over to it from scons (having previously moved from autotools to scons a couple of months ago). I was reasonably impressed with scons when I came to move the opie-sync plugin over to it, in that it removed a lot of the complexity of autotools, but CMake just blew me away. It's even simpler and requires even less files in my source tree, and switching over was a breeze. Build systems are often the bane of software developers, particularly in the free/open source software world, so kudos to Kitware for making a build system that doesn't make my head want to explode :) Thanks also to Daniel Gollub of SUSE/Novell for all his recent hard work on the OpenSync core, including the decision and the effort to move OpenSync over to CMake.

I've started some tentative plans for Opie 1.2.5. I know, I know, 1.2.4 isn't out yet and 1.2.3 isn't even available to a lot of people, but we're working on that too. I think it's important to be looking ahead. Take a look at the recently updated Opie roadmap for details.

Click-click-click-click... uh-oh...

I have not exactly had great luck with hard drives lately. First, a Seagate 320GB SATA drive in my main PC started acting up - fortunately only a few files were unreadable and I managed to copy the rest of the data off without issues. Then, just days later a Seagate 120GB IDE drive in my parents' PC (of which I am the de-facto maintainer) began to bite the dust - it was pretty much unreadable and making lots of nasty noises. After trying a few other options I attempted copying the entire drive using Acronis Disk Director, and following several hours of constant grinding to my amazement it seemed to have copied all of the data successfully, this time onto an IDE RAID 1 array to avoid further faulty hard drive hassles.

You would think that the story would end there. Unfortunately not. A week later Windows XP on my parents' PC begins to report corrupt files, so it seems that one of the drives or the cheap no-brand IDE RAID controller is faulty. If it is one of the drives then it obviously isn't serious enough to make the RAID controller fail the drive out of the array, because it helpfully reports that everything is OK. Should I mention that the two drives in the array are both brand new Seagate 160GB IDE drives?

Right, that's it - no more Seagate. I used to be a loyal Seagate buyer, but two confirmed failures and one possible failure in two weeks, plus at least four previous failed drives in the last seven years is just too much to bear. To replace the failing 320GB drive in my machine I've gone for two Western Digital Caviar "RAID Edition" 320GB SATA drives in a Linux software RAID-1 array. (The RAID Edition drives have some added reliability features, but also have a 5-year warranty and aren't all that much more expensive than the standard WD drives). Since I've just moved to a new PC I've given the old one to my parents, and tonight I have rebuilt it for them using two new WD RAID Edition 160GB SATA drives in RAID-1 using the onboard Promise RAID controller. Third time's a charm...

Earlier this year I read two interesting articles on hard drive reliability (which are summaries of two research papers) on the StorageMojo site:

Google’s Disk Failure Experience
Everything You Know About Disks Is Wrong

In my case, the comments regarding how useless SMART is were definitely borne out - SMART reported no errors at all before the problems started. In fact when I was attempting to recover the 120GB drive using SpinRite it was still reporting no errors in the SMART status as it was clicking away. Brilliant.

I guess the only lucky thing to come out all of this is that no significant amount of data was lost. It has also been a bit of a wakeup call in that I realised I hadn't done a backup in ages. Needless to say I have since rectified that oversight :)

OpenSync hacking

I've spent quite a bit of time over the last week or so getting the Opie plugin for OpenSync ported to the latest 0.3x branch (ie, trunk in svn). 99% of the job is done, all that is needed now is testing and a few fixes here and there. If you're brave, and make backups, you can check out OpenSync from svn and give it a try :)

In the process of porting I've also fixed a few things:

* scp support now actually uses scp/ssh instead of sftp, so that you don't have to install additional packages to have a secure connection (it works with dropbear that is included in most handheld distros by default). You will need to set up key-based authentication however - there's no mechanism to enter an ssh password at sync time.
* scp can now sync notes
* Improved error handling in a lot of places
* Lots of refactoring and code tidying

The 0.3x branch will eventually become the next stable version of OpenSync, 0.40, but there's still quite a bit of work to be done before then (see the TODO list). There are a few things I'd like to add to the Opie plugin, mostly along the lines of making it more robust, but also some minor features eg. supporting the Opie-Notes application. In the near future I'd like to also look at adding support for the SQLite backend in Opie, however at the moment the SQLite backend has no mechanism for retrieving data remotely for syncing purposes, so that will need to be added in Opie as well.

I've probably mentioned it before, but I have to say I'm pretty excited about OpenSync - it's a powerful syncing platform and one that has a huge amount of potential. We need to not only keep pushing the development forward, but we also make it more visible and accessible to people. KitchenSync with OpenSync as a backend is now in KDE 3.5.7, gnome-sync has just been released, and I think once KDE 4.x comes out with Akonadi and OpenSync support makes it into Conduit (which I understand is on the cards) we will have a bit more exposure.

Opie 1.2.3 is out!

So finally we've released Opie 1.2.3. It's been a long road, but a lot of people helped out (some of whom are new to Opie) and that made all the difference. It's nice to have something to show that the project is still alive :)

Next on the agenda: 1.2.4, naturally. I hope we can improve synchronisation, and that includes not just working on the OpenSync plugin, but also fixing up PIM bugs on the Opie side and likely fixing problems syncing with Qtopia Desktop as well. In general I'd also like to get a few of the long-standing annoying bugs fixed too. If there's something annoying that prevents you from using Opie on a daily basis, please report it - or even better, try to help us fix it.

KonqE & Opie 1.2.3 again

OK, so thanks to some work by Marek Vašut (Marex), Paul Sokolovsky (pfalcon) & polyonymous in fixing the OBEX regression, we are nearly ready to release Opie 1.2.3 - mid next week is the target. I hope the release will breathe some new life into Opie - certainly there has been quite a bit of activity over the past week or so.

Part of that activity was to do with getting the new version of Konqueror Embedded working (which is not actually part of Opie, although one might say it is the unofficial flagship browser of the platform). Thanks again to Marex it is now almost completely working:




















As you can see the window style needs to be fixed and a few other bits might need some work, but otherwise we've now got a working modern browser :) See the Opie wiki page for further details.

NetworkManager, Wireless cards and WPA

So, I was setting up a PC for my sister running Kubuntu 7.04 (Feisty Fawn) and she needed wireless to be able to connect to the internet. I had a spare NetGear MA311 PCI card, and I figured it's fairly old, it's been well supported for ages using the hostap driver - so it should just work out of the box, right?

Wrong. I couldn't even see any networks using KNetworkManager. After fiddling around I discovered that not one, not two, but three wireless drivers were dueling to access the card, and the one that won was the old linux-wlan-ng driver which doesn't support any wireless extensions that NetworkManager needs. After searching through the bug reports at launchpad.net and blacklisting the prism2_pci, prism2, hermes, orinoco, and orinoco_pci modules, I eventually got it so that hostap was the only driver in charge of the wireless card. I tested all of this on a spare machine at home and it connected to my wireless just fine.

I drive back over to my sister's place in full anticipation that it will now work with no hassles. Nope - I discover the network at her flat uses WPA, and inexplicably KNetworkManager can see the network but thinks it's only WEP. Unfortunately I had not tested making a WPA connection at home beforehand or I would have discovered this problem immediately. I tried connecting manually with KNetworkManager, I tried running wpa_supplicant directly (again, it could see the network but insisted it was WEP), and I tried putting the connection settings in /etc/network/interfaces; nothing worked.

I knew I could connect to the wireless network at my sister's flat just fine with no configuration from my laptop (running the same version of Kubuntu, but with built-in Intel wireless using the ipw2200 driver), so I knew it could work. So after looking through what was available for purchase here and checking up on Linux support, I bought two more PCI wireless cards - an Edimax EW-7128G and a Netgear WG311T.

At home I tested the Edimax card first. It was detected (using the rt61 driver) and again could see my now WPA-encrypted network, but again it told me that it was encrypted using WEP and not WPA, and would not connect to it. Sigh.

The Netgear card was another story altogether. The test machine I had at home wouldn't even get past the BIOS screen with it plugged in, which was disturbing. However, I tested it in another machine I had using a Kubuntu live CD and to my surprise it connected first time with no configuration. Next weekend I put it into my sister's PC and it connected out of the box. Wireless problems solved!

So, if you're looking for a PCI wireless card that should work out of the box with Ubuntu/Kubuntu Linux, then the Netgear WG311T could be the answer - assuming your PC will boot with it plugged in, that is. I'm sure there are other cards that work well, too, and with the introduction of the new wireless stack in the soon to be released 2.6.22 kernel, things will hopefully get a lot better for wireless under Linux in the next 6 months. Still, right now it can be a little frustrating :(

Kubuntu on a laptop

I just installed Kubuntu 7.04 ("Feisty Fawn" for those who like those codename thingies, which I don't) on my laptop, a Toshiba Satellite M50. I rarely use my laptop, but when I do use it I want it to "just work" (TM). I use Gentoo on my desktop but given the amount of messing about that requires it was obviously not an option for this machine.

I've been through the Kubuntu install process before and it was fairly smooth - the only hitch being that it told me it was unable to create the first partition layout I gave it, without telling me why. I chose a different setup and it continued. Once everything was installed the laptop booted into Kubuntu, complete with nice smooth startup screens. I was very impressed to see that I could plug in my ethernet cable and even switch on the wifi and it happily connected automatically to both of them with no problems. Plugged in a mouse, digital camera, USB stick - all detected and worked immediately. It even suspended and resumed with no hacking required, which is a first as far as this laptop is concerned.

However, I would be remiss if I did not mention the problems that I had after installation:

1. Setting up DVD playback using the instructions in the help documentation did not work. After a visit to Google and installing xine-ui the problem went away.
2. The version of vim installed by default, vim-tiny, is horrendously broken. Arrow keys and backspace don't work in insert mode - I mean, WTF?! Eventually I got sick of this and installed the proper "vim" package and it worked as expected.
3. Hibernate didn't work - when it booted back up it brought up a grey bar on the screen and froze up. Hibernate is something I can do without at the moment so I'm not too worried about this one. (I was able to recover simply by selecting the built-in recovery mode from the boot menu and then restarting).
   
4. The worst however was when completely on a whim I attempted to plug in an external monitor. After fiddling with the System Settings application I succeeded only in completely borking up my xorg configuration, without trying too hard I might add. After changing the settings and rebooting I was greeted with the primary screen being the external monitor (which was not what I set it to) and both screens stuck at 640x480, which on a laptop screen with a native resolution of 1280x768 does not look very good at all. So I thought I'd just log in and disable the second monitor - but no, as soon as I logged in the power manager applet crashed, and when I tried to run the System Settings - Monitor and Display settings, it also crashed. After a bit of searching it seems that this is a known bug - the xrandr library that both of these things use (which allows changing screen resolutions on the fly) just crashes when Xinerama (ie, multi-monitor support) is enabled. After much fiddling and cursing I found the only way to get back to a configuration where the laptop's widescreen video mode was detected and available was to restore a backup copy of the xorg.conf file, which fortunately the Monitor and Display settings applet had saved.
   

I'd have to say I'd still recommend Kubuntu to people looking for a nice desktop distribution, since apart from the problems mentioned above it seemed very well polished - for inexperienced users it certainly beats all of the other Linux distributions I have tried.

Opie 1.2.3, Konqueror Embedded: The Resurrection

The Opie 1.2.3 release has been delayed - we are currently looking into some issues around recently added Bluetooth support, namely that it's somewhat broken. Hopefully these issues will be sorted out soon since there's a fair bit of useful stuff in the release.

The version of Konqueror Embedded for Opie in Familiar is from 2003, and since it has sat unmaintained for so long I had figured that was it. It turns out that a company called basysKom, who you may well have heard of since it is staffed by several KDE developers, has released an updated version just recently including most interestingly a version backported to Qt 2.x. It's not quite compatible with Opie out of the box since they are targeting Qtopia 2.0, but it's close. I have started trying to hack it to work with Opie, and if you are interested you can keep up with progress by monitoring the KonquerorEmbedded2007 page on the Opie wiki (you can even subscribe to the page so you get emails about changes if you want). Better still, grab the source and the patch and give me a hand :)

I've just signed up to be a mentor in the Google Summer of Code for Opie (under handhelds.org). If you're a student interested in working on Opie or something else handhelds-related under the handhelds.org banner, please have a look at this page and start thinking about what you might like to do. There are a bunch of suggestions up there but we'd love to hear any new ideas so please let us know if you have any.

OpenMoko

A couple of months ago I got a surprise email asking if I would like to be part of the of the OpenMoko Neo1973 development phase 0. Naturally since this involved me being sent a free phone - and one of the first truly open phones at that - I graciously accepted. Thank you very much to FIC and the OpenMoko team!

A week ago my Neo arrived. It's a little larger than I expected, but then that's probably a good thing since it allows for a more readable screen. The software is in the very early stages and there are some issues with charging, but already you can start to see the potential of the device. I've nearly bricked it once already by updating the bootloader incorrectly, but fortunately the USB DFU (Device Firmware Update) was still working and I was able to flash a new working bootloader. Unfortunately though at the moment my desktop system is suffering from problems with USB devices which seems to be triggered by connecting the Neo. It's not nice to plug in a new device and then suddenly find that your mouse has powered down, and won't power up until you reboot. Mousekeys just aren't quite the same.

Hopefully I will be able to find a way to contribute back to OpenMoko soon - perhaps in the area of apps development, which is my forte, although I admit I would prefer to be developing with Qt than GTK. Still, that is the platform they have chosen (and not without good reason), and if it can be polished up to a state where it looks like the mockups on the website it should be quite a nice environment indeed.

Note that this does not affect my commitment to handhelds.org or Opie. In fact, on the latter front I hope that one day I will be able to run Opie on the Neo.

Perled tongues, the scientific method, and impending releases

This video is absolutely hilarious. Admittedly he is trying to do something that the Vista speech recognition wasn't designed for, but it still made me laugh so hard it started to hurt. It looks to me that under the right conditions and with the right training it would work well for writing letters and other documents, but I honestly still can't see speech recognition taking off in a huge way - a perfectly quiet environment is not something you usually find in the average office.

On Friday at our usual work lunch at Nando's we were discussing the difference in taste between Coke, Coke Zero, and Diet Coke, and the perceived taste difference between soft drinks out of a can and out of a bottle. After much discussion it was resolved that some blind testing had to be done to resolve the issue. Five of us did a blind test using four identical glasses of Coke, Diet Coke, Coke Zero all out of glass bottles, and Coke out of a can. We found that (a) almost all of us had no trouble discerning "real" Coke (with sugar) and the others; (b) Diet Coke and Coke Zero have a very similar taste but that the slightly unpleasant aftertaste that you get with Diet Coke wasn't as noticeable in Coke Zero, so they aren't quite the same; and (c) there was absolutely no difference between the coke out of a can and out of a bottle, so it must be just the drinking experience that influences the taste perception. Case closed :)

The Opie code is "frozen" at the moment in preparation for the Opie 1.2.3 release which should happen very soon. There's been the suggestion of another minor Familiar release including it (among other things) in the near future as well, which I would definitely support (and try to help with). Actually I'm looking forward to when the freeze period ends so that I can have a look at committing a few bits of code I've been sitting on - such as by external keyboard (kbdd) control stuff.