Jamie Strandboge: A bite out of the Apple

Being the “computer guy” for friends and family is both demanding and rewarding. On the one hand, I get to install Ubuntu on lots of computers. On the other hand, I end up having to deal with various hardware and devices. For many years I have successfully avoided Apple’s proprietary products, but recently I’ve had to deal with them. Let me first say that the hardware I’ve gotten my hands on is excellent, and let me then say that having worked with open source for so long, it is surprising at how hard Apple works at making sure you are locked into using only their stuff. Also, am I the only one who finds their faux-ubiquity in TV and movies annoying? I mean, how is it that virtually every laptop, monitor and computer has the Apple logo emblazoned on it? Clearly, their marketing is at least as excellent as their hardware, but I digress….

After an annoying foray into the world of the MacBook Pro for a friend (btw, with all the Ubuntu developers I see at UDS with Apple hardware, why isn’t it better supported in the default install? I mean, I felt like I stepped back 5 years with all the twiddling I had to do with the silly touchpad to get a halfway sane configuration — I’d fix it in Ubuntu myself, but the device is out of my hands now), I finally was confronted with trying to get a new, off-the-shelf iPod Touch to work with Ubuntu 10.10. Pretty and shiny as it was, let’s just say it did not work well with Ubuntu out of the box. Looking at https://help.ubuntu.com/community/PortableDevices/iPod and the video at http://www.libimobiledevice.org/ I thought it was going to be relatively easy, but there were several roadblocks:

1. Currently, you can’t just open a new iPod and use it with a non-iTunes system (ie, Windows and OS X — last I checked, the latest iTunes doesn’t work too well in Wine). As soon as you plug it into a USB port, there is a helpful graphic telling you that you need to connect to iTunes. This step seems to be required to download the latest firmware for the device and register it.
2. In addition to the above, you must do an initial music sync from iTunes to initialize the music databases on the device.
3. This new iPod Touch only does USB 2.0 and won’t fallback to 1.1 — a frustrating tactic that works quite well at driving revenue since people all of a sudden feel that their perfectly good computer yesterday is old and outdated after getting their shiny new iPod today. Ranting aside, qemu-kvm (my preferred virtualization technology) does not provide a USB 2.0 host controller.
4. The version of usbmuxd in Maverick (Ubuntu 10.10) is too old. Among other things, it installs a udev rule to start usbmuxd which is used to coordinate the communications between the device and Ubuntu (see the README file in /usr/share/doc/usbmuxd for a detailed explanation of an IMHO too complex protocol). The udev rule for Maverick did not catch the idProduct of my device (0x129e) and would not start usbmuxd.
5. While hand-editing /lib/udev/rules.d/85-usbmuxd.rules and running ‘sudo udevadm control –reload-rules’ did get udev to start usbmuxd, usbmuxd didn’t know how to deal with my device. I guess there was a reason the udev rule was so specific to begin with. ;)
6. Apple’s latest version of iOS (4.2.1) broke stuff for Linux users. AIUI, libimobiledevice is the (excellent) library that pulls all the disparate parts of dealing with iPods together and makes it so that music players like Rhythmbox work seamlessly with with the device. Upstream fixed the 4.2.1 problem quickly, but the version of libimobiledevice in Maverick is too old.
7. As of the time of this writing, DbVersion 5 is not supported by libgpod, so after drag and dropping files in Rhythmbox to the device, the iTunes database on the iPod is not updated and the files don’t show up. This is also being tracked in Ubuntu as LP: #601251.

See what I mean about Apple trying very hard to make sure that their devices work with only their stuff? Merry Christmas Jamie, you get to fiddle with several highly proprietary systems for hours on end! *Sigh*. Of course this isn’t Ubuntu’s fault; Apple doesn’t provide iTunes for Linux and they keep changing things in incompatible ways while the open source developers furiously try to keep up (admirably).

So what did I do to get the thing to work? Well, I didn’t because of the libgpod DbVersion 5 support. That said, I am to the point now where I resolved everything except this, so when the new libgpod comes out, I’ll be ready. Here is what I did to address each of the above:

• I didn’t have a Windows install handy, but do have a legal installation disk for Windows XP. I installed it via virt-manager, but ran into a bug where the domain xml used ‘kvm’ as the emulator instead of qemu-system-i386, so when I rebooted the machine after stage 1 of the install, I got a black screen (I still need to file a bug on this). Not sure if it is still required in Maverick, but I also disabled all the <features/> except <pae/> (ie, ACPI and APIC). I installed iTunes, plugged the device in and… nothing. qemu-kvm doesn’t provide a USB 2.0 controller. D’oh! Now, I could conceivably do PCI passthrough for the USB 2.0 controller on the host, but that required fiddling with the pci-stub kernel module so that the controller isn’t being used by the ehci_hcd module on the host. I’d like to play with this more, but haven’t found good documentation on it yet.
• At this point, I still didn’t have Windows handy for what I needed. The non-OSE version of VirtualBox does provide a USB 2.0 controller if you also install the ‘Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack’. So I removed the kvm* modules from the kernel, installed VirtualBox, installed the extension pack, installed Windows XP in the guest, installed iTunes, plugged the device in, then told VirtualBox to make it available in the guest. This got me to some sort of wizard to install the firmware and register the device. Using iTunes on Windows XP in VirtualBox worked flawlessly.
• From within iTunes, I did a music sync, ejected the device, then shutdown the VM
• In Ubuntu, I added Paul McEnery’s ppa and performed an upgrade. This got me several library updates, but not everything (Update: it now has everything but a new libgpod). In particular, you need (all from the ppa, except where noted):
  • usbmuxd 1.0.6
  • libimobiledevice 1.0.4 (and be sure to install libimobiledevice-utils)
  • ipheth (for tethering, not strictly needed)
  • A version of libgpod4 that supports DbVersion 5 (in progress)

Now when I plug in the device in Ubuntu, I am prompted to open F-Spot and Rhythmbox, the device is mounted via gvfs, the device shows up on the Desktop and I can read anything on the device via nautilus. ideviceinfo works, and presumably so do all the other idevice* utilities. I can also copy files to the device via nautilus (though make sure you see the ‘Sync in Progress’ message on the iPod before unplugging!), but they don’t show up in the iPod (because I am not using libgpod). Now I just need a new libgpod to support the DbVersion 5 database so it all works in Rhythmbox (I can use iTunes in the VirtualBox VM using a shared folder to add files to the device in the meantime). Beyond getting Rhythmbox to work, I’d like to be able to copy videos and pictures to the device. Maybe once libgpod is in order I can use gtkpod (there are also notes in the aforementioned wiki documentation). If I can get this to all work right, I may document all this step by step.

While the person I am setting this up for is generally happy (and patiently waiting for proper Rhythmbox support), this whole experience has felt more like Apple took a bite out of me. Here’s to hoping this blog helps others with new iPods!

Update (2010/12/28): While I haven’t personally tried any of the following, I’m told:

• You can activate Apple devices from within a Linux system using iDeviceActivate (referenced in the How to Compile iDeviceRestore on Ubuntu HOWTO).
• You can create the initial hash with http://ihash.marcansoft.com/ (from the makers of usbmuxd).
• The libgpod development for DbVersion 5 support seems to be happening in libgpod’s Gitorious.

Update (2010/12/29): As pointed out by Christophe in the comments, ideviceactivate does not work with the iTouch 4G, but patches have been submitted. Also, he says you only need to do the ihash/sync music step with DbVersion 4 devices, so other devices will be ok.