Missing Library in Rhythmbox on F17

One of the bugs I encountered after upgrading to Fedora 17 is that my music library seemed to have disappeared in Rhythmbox. In fact, everything appeared in the Missing Files view. The strange thing is that all the files appeared exactly where Rhythmbox thought they should be. In fact, I could even play them directly from the Missing Files view!

Anyway, after a bit of searching, it appears that I am facing this bug. You see, on my laptop, I encrypt the /home directory. Back on Fedora 16, some bug in gvfs or udisks caused things to think that that directory was a removable partition. Something must have been fixed in Fedora 17, because it's no longer marked that way (which is good, because I hate seeing it in nautilus.) However, Rhythmbox still thinks those files should be on a separate partition and doesn't correctly update itself (the aforementioned bug.)

This is pretty easy to fix, in fact. Since /home is now no longer considered a separate mount point, you just have to remove those indicators from Rhythmbox's config. The config is an XML file that's been pretty-printed, so the offending information appears on a single line (per file entry). That makes fixing it a pretty simple thing to do:

$ cd ~/.local/share/rhythmbox/
$ cp rhythmdb.xml rhythmdb.xml.backup
$ sed -i -e '/<mountpoint>/d' rhythmdb.xml


The Fedora 17 "Beefy Miracle"

So, a new release of Fedora was set free just last week, the so-called "Beefy Miracle". A couple of releases ago (or maybe last release), I had really thought I might wait at least a month after a release before upgrading. Turns out my curiosity got the best of me, and I went and upgraded my laptop mere days after the release.

Fortunately, this release has been much much better right out of the box. Having read the release notes, I was prepared for the one major bug, namely the kernel being stuck at an old version. Since it only affected shutdown, it wasn't too big of a deal, just a bit annoying.

Being the third release with GNOME3, it's finally starting to come together into something usable. With the right extensions, it's even better than GNOME2 was. I can't say exactly what changes went into it, but I guess it's a lot of behind the scenes changes to make things Just Work.

There are still a few things that need work, of course. I'm still not a big fan of the grey theme, but at least the window decoration is not huge anymore. It's still a bit difficult to find good complete themes that fix that. Also, it seems they introduced a fade-out of windows that are not in focus. It's sort of like it's disabled-in-appearance-only. I'm sort of yes-no on this change. It seems like it might be good for accessibility, but they don't seem to have tried this out all too much. Sometimes widgets remain faded even when the rest of the application is back in focus. Using some dialogs (like gedit's Replace) do weird things.

One last point is going to be about the wallpaper. I know I don't see it all too much, but the last three releases have had wallpapers that are really quite nice. Fedora's wallpapers have been traditionally mostly blue, the trademark colour. This one deviates a bit with a splash of pink, but I think it's just enough to make it pop. And who could not like fireworks?

Jumping on the Fedora 14 bandwagon

Fedora 14 was released just a day or two ago. I figured I'd be cutting-edge and upgrade on pretty much the day it came out.

Of course, as with last time, I'm trying to do so on a laptop with little room for any more stuff. This time though, I tried as hard as I could to get preupgrade to work instead of doing a new install.

The Free Space Issue

/boot

The first issue was, of course, the lack of free space, on both /boot and /. /boot is slightly less than 100MB, meaning I wouldn't be able to fit the install image on it. Fortunately this time, it correctly realized I didn't have room, and set up anaconda to download it after booting.

/var/cache/yum

The RPMs get downloaded to /var/cache/yum before being installed, but I didn't really have enough room for that. What I did have was a 4GB USB key. So I edited /etc/fstab to overlay the key there. I was a bit concerned that anaconda wouldn't mount it when I rebooted, but it did so correctly, so I didn't run into any trouble there.

Now, preupgrade seems to download a separate directory for each repository, then link the files into one large thing. Since the USB key was FAT32, it couldn't really do that linking. Since preupgrade is written in Python, it was easy for me to hack it to copy instead of linking. That was a bit slower and naturally took up double the space, but there was enough on the key for it.

/usr

But then there's the more pressing issue of the space on /usr.

Clearly I'm really pushing it on the free space front here. With the help of Baobab, it was pretty clear where I could find the most "dead" space. These locale files are huge, and a pretty small portion is actually in my usual locale, en_CA. So of course, that was the first thing I axed.

Everything seemed peachy, but it turns out that preupgrade sucks at estimating required disk space. I then removed OpenOffice and kdelibs, and anaconda stopped complaining. However, it still ran out of space, though I managed to clean up some stuff and restart it.

The Download Issue

Preupgrade indicates that it will resume from where it was if you cancel it. While this may be true of the RPMs, it's not of the metadata, kernel image, or installer image. I don't really know why it needs to update the metadata all the time, and I don't really know what it is. But I do know it shouldn't have to download the image every time. The problem is that it wants to check the file size, and then downloads the whole file just to do so. This is to ensure that you have enough space to hold the images on /boot. So if there's enough room, it downloads the file again to place it on the drive!

Since I didn't have enough room on /boot, it had to download the installer image whenever I started upgrade process. Obviously, I didn't want to do that too many times after it failed the first time. So I downloaded the image and placed it on the USB key. Then it was a simple matter of modifying the kickstart file in /boot just before rebooting so that it would specify the right location. The anaconda installer took care of the rest.

The Cleanup

After rebooting, yum thought I had several fc13 packages still installed. I tried rebuilding the database but that really didn't help. I tried writing a small script to pick out the extras and remove them, but then I realized that package-cleanup would do that and do it better. A run with --cleandupes took care of the problem. It also took care of some phantom missing storage space.

Why bother?

So I'm sure you're all wondering why I even bothered, instead of just installing fresh. I'll admit that would have been easier, as with last time. Mostly, I just wanted to see if I could. But arguably more importantly, I was able to see whether I could do the upgrade via a USB key.

In other words, one could get preupgrade to download everything to the USB key and install from there. It wouldn't be too difficult, probably just requiring an option to tell anaconda where the package cache was. It may already exist in the kickstart configuration, but I haven't looked into it. You can already place the installer image on a USB key, as I did above.

Got a Bus Pirate!

Update: The ModemManager rule should really only apply to the USB device, not the TTY. I updated the rule below to be a bit more specific.

Around the end of July, I ordered a Bus Pirate from Seeed Studio. It took about a week to ship, which is pretty quick, I think. I picked it up from the post office on Tuesday.

After opening the package and plugging it in, I was greeted with nice blinky lights, and my computer didn't start smoking, so that was a good sign. However, I seemed to run into a bug, where every few lines or so, the display would be screwed up. I started with minicom, until I found out that I could talk to the bus using screen instead. But the output was still not working. Even if I did something simple like press Enter repeatedly, after 5 or so lines, the prompt would show incorrectly. That was kind of disappointing, since Seeed had put a sticker on it for passing QC, so I thought I had a bad cable or set up something wrong.

But today, I finally hit a stroke of luck. One time, upon connecting, I noticed something writing to the port. The text started with AT... and I of course recognized those as modem commands. I remembered a bug report or blog post about ModemManager messing with (things that look like) serial ports.

ModemManager works alongside NetworkManager to provide support for connecting through GSM and CDMA modems. The problem is that a lot of modems suck and ModemManager needs to probe serial ports to see if it's really a modem. Since I'm on a desktop and I probably wouldn't ever use a modem, I could just remove it, but then what would I do on the laptop?

So I set out to find how to blacklist a serial port from ModemManager. I couldn't just get ModemManager to ignore all Bus Pirates, because it identifies itself as a generic FTDI USB-to-serial converter. While I can let Dangerous Prototypes go for that since it's a little hacking board, I'm sure there are actual companies using an FTDI chip who don't identify their modem devices uniquely. Thus, blacklisting the entire FTDI series was out.

All was not lost however, as ModemManager's blacklist basically works through udev. Because USB device numbers are not static, I was already using a udev rule to give me a static device name for the Bus Pirate (from the Arch Linux wiki). After a quick look at the ModemManager blacklist rules, I found that it uses ENV{ID_MM_DEVICE_IGNORE}="1" to signal to ModemManager to ignore a device.

So, to sum it all up, you need to create a udev rule in /etc/udev/rules.d/98-bus-pirate.rules:

SUBSYSTEM=="tty", ATTRS{idVendor}=="0403",\
    ATTRS{idProduct}=="6001", ATTRS{serial}=="...",\
    GROUP="users", MODE="0660", SYMLINK+="buspirate"

ACTION=="add|change", SUBSYSTEM=="usb",\
    ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="0403",\
    ATTRS{idProduct}=="6001", ATTRS{serial}=="...",\
    ENV{ID_MM_DEVICE_IGNORE}="1"

Check the Arch wiki for how to get the proper serial number. The first rule sets up a /dev/buspirate device node link so you have a memorable name to use (instead of /dev/ttyUSB#). It also sets permissions so that it's accessible to the users group. Make sure to add your user to that group and you won't need to be root to use the Bus Pirate. The second rule sets a variable on the USB device so that ModemManager doesn't attempt to probe it.

To connect, just use screen. I set up an alias to run screen /dev/buspirate 115200 8N1 and it works great. Now I just need to get something to hack on with it.

Switching from Gentoo to Fedora

Round about last week, I switched my desktop from Gentoo to Fedora. It took a few days to get everything the way I wanted. But it only took that long because I had lots of data to backup and (selectively) restore, and I only worked in the evenings. Fortunately, a "re-install" is way less painful than Windows, since all you really need to keep is your home directory. I just don't know how Windows users live through it, especially without all their programs in a convenient package manager. I can happily say I have never re-formatted Windows for any reason (and that's not because I don't used it).

I guess there are a few reasons for the switch. I'd say it basically boils down to the following three:

1. Gentoo wasn't updating fast enough for things I wanted. I can't say whether this is a general trend, but at least for packages I wanted, it was slow, and I could see that Fedora was getting those updates.
2. My computer is getting older and packages are getting bigger. It's not been too bad (except for god-awful C++ programs that use Boost), but I don't want to work my system so hard as it gets older.
3. And the most important: I got lazier. Not lazy enough to install Ubuntu though!

I guess I'll see how well Fedora fairs. I've just got an audio bug or two to fix (already reported) and it should be good. Eventually, I should go see if my cx18 works, too.

Upgrading to Fedora 13

Well, this didn't go as smoothly as the live USB versions, mostly due to preupgrade.

I don't have a "standard" install, you might say. This is on a laptop that's not mine and Fedora is not the primary OS. So it's very space-constrained. I don't even have the recommended 200MB (now the recommendation is a whopping 500MB) space for /boot. Personally, I think that's a ridiculous amount of room to hold a kernel and initrd (even if they are generic), but that's the default install size.

So the problem is that preupgrade downloads everything to /boot, but I don't have enough room for it. The other problem is that preupgrade didn't bother to tell me either. There are some bug reports for it already, but I'm pretty sure it worked for F11->F12, so it's an annoying regression.

Fortunately, that preupgrade page has instructions for setting up the correct grub command line. I had enough room for the kernel and initrd, but not the install image. I'm not sure if it's because the downloader failed to notice it ran out of space, but the installer wouldn't auto-download the install image. I had to set a URL source, find a mirror, and point it to <mirror>/releases/13/Fedora/i386/os/.

So I've got the installer image set up and everything's good to go, right? Well, no. In the end, my space-constrained root didn't have enough room to upgrade. Oh well, might as well see how nice a full install looks. I still remember what I had installed well enough to tweak it after.

How did that go? It was pretty nice and simple, actually. The only problem I ran into was that it didn't figure out my boot loader configuration. I don't know why that is, but replacing it seemed to have worked. The only hiccup was I didn't remember which partition should have been Windows. It would have been nice if it had shown partition labels.

Anyway, after dropping stuff I didn't need (who needs all graphics drivers on a laptop?), and installing all the stuff I could remember (like devel packages), I ended up with 1.2G free. I even have OpenOffice installed this time. I don't know how I managed to go from ~100M to 1.2G, but that's definitely a great improvement.

Upgrading Linux images...

So there's been quite a few new releases of distributions recently. Last month, there's been Ubuntu 10.04 and Linux Mint 9. This month it's Fedora 13.

Anyway, I just updated my live boot drive with all of those and everything works pretty well.

Booting multiple Fedora images from USB

Well, I've been trying to boot Fedora from ISO, but haven't exactly been successful. On the bug, you can see a possible fix, but it's a bit complicated and requires messing with the image from an existing install.

Instead, I figured out a way to do it that uses the extracted files. It's pretty simple. First, mount the ISO and extract three files:

1. the LiveOS image: squashfs.img and possibly osmin.img,
2. the compressed kernel: vmlinuz0, and;
3. the initial ramdisk initrd0.img

In case you were wondering, the copies in the EFI and isolinux directories are the same. You can then place all these files in a subdirectory on the drive. Then you add an entry to GRUB pointing to the files. In this case, I placed the files in /boot/F12-$arch and the drive's label is LiveLinux.

menuentry "Fedora 12 32bit" {
    linux /boot/F12-i686/vmlinuz0 root=live:LABEL=LiveLinux \
        live_dir=boot/F12-i686 rootfstype=auto ro liveimg quiet rhgb
    initrd /boot/F12-i686/initrd0.img
}

Trying to boot Fedora from ISO

Last time I managed to boot Ubuntu and Linux Mint from ISO pretty easily. Unfortunately, Fedora does not work so simply. There is no iso-scan parameter that can be used for it.

Fedora uses Dracut to create its initramfs infrastructure. From that page, I found the rdshell parameter that could be added to the kernel command line to get into a shell before switching to the real root and init. Once there, I was able to mount the USB drive, then the ISO, and finally the actual live image. So I knew that it should be possible to boot from ISO, but the initramfs doesn't have any parameter to look for it. And useful for later, I found the the version of dracut was 002-13.4.git8f397a9b.fc12.

I went looking at the Dracut pages a little more, and it turns out that support for booting off an ISO was added, and is in dracut-004. Which is great, except of course, it means it's not on the existing image. The commit was easy to find, and seemed self-contained, so I figured I might be able to patch the existing initramfs myself. It would probably be better to just build it from scratch, but I don't think I have everything installed for it and it didn't look like too big of a change to do manually. Modifying the initramfs was simple:

mount -o loop Fedora-12-i686-Live.iso /mnt/cdrom
zcat /mnt/cdrom/isolinux/initrd0.img | cpio -iV
# Modify files here
find . | cpio -oVH newc | gzip > /media/LiveLinux/boot/F12-i686-initrd.img

Now for that modify step, you need to look at the install script of the 90dmsquash-live module to see where dracut would be installing the modified files. Anyway, that seemed to work up until getting past dracut. I saw plymouth start up, but then the kernel complained about the invalid root. Obviously, the ISO was not actually mounted in the end. I even booted to the dracut shell and found no trace of the USB drive. I'm waiting to see if the comments on bug 557426 will help explain something.

SELinux Re-labelling

So yesterday, I downgraded from ext4 to ext3. I think I forgot to set SELinux to permissive, so a couple things ended up wrong. I re-labelled everything. This is what it looked like:

Now, there are a few things wrong there. First, the text really doesn't doesn't fit in. I think this is because KMS sets up the blue background, but isn't quite connected to the virtual console nicely. There's always work going on there, but it's not quite right yet. It's always getting better though.

The second thing, you might not notice at first. But if you look between the first paragraph and the lines of stars, then you'll notice some random characters. Those were typed by me. You see, nothing happens between printing the first two lines and the first asterisk. I think I read somewhere that each one corresponds to a few thousand files, but it took so long, I almost thought something had frozen. Fortunately, I let it continue and it finished correctly. But I really think there should be a better indication of activity in between asterisks.

Downgrading ext4 to ext3

So, a little while ago, I replaced Gentoo with Fedora 11 on my laptop. The laptop is a little older and prone to overheating when compiling, so I didn't see the value in using Gentoo there. And I never quite figured out the cross-compile for my desktop since it was AMD64->x86.

I ran into one very annoying problem, and that was with ext4. Every so often, any disk I/O would just stop. I could no longer load programs, and if anything attempted to write a file, it would freeze (technically, it would enter the "uninterruptible" state and get stuck). I found a few kernel OOPS in the log, and reported those with the Fedora tool for that, but it didn't seem like it happened for many people.

In any case, the real problem is that I had to force a reboot, and ext4 would always lose something. Sometimes it was some just-created data that could be replaced, and other times, the whole file system seemed messed up. Random libraries would crash or fail to load because they were corrupted. It was quite annoying.

So, I've been looking for a way to 'downgrade' to ext3. I've used ext3 for a long time on my desktop, and also on the laptop with Gentoo, and never ran into anything quite so bad. The only problem was no-one seemed to have a guide to go to ext 3. The best I could find is to create a new partition and copy files there. Unfortunately, I didn't have enough free space to get that done. (Re-partitioning is not that hard since Fedora uses LVM.)

After much searching, I have managed to find a solution. Fsarchiver provides an easy way to back up a complete file system. Once you have that backup, you can recreate it using a different file system format without losing any of the metadata (assuming the new one supports it all). Also, because it uses compression, I was able to fit what I needed onto a USB stick, and restore from there.

I used the Fedora 11 LiveCD (on the same USB stick I used for the backup) to do all of this. Fortunately, fsarchiver is provided in a Fedora package that can be installed from the default repositories. I'm doing this from memory, but it should go something like this (I'll only show the fsarchiver bits):

yum install fsarchiver
fsarchiver savefs /media/live/rootfs.fsa /dev/mapper/lvm_name
<re-format partition>
fsarchiver restfs /media/live/rootfs.fsa id=0,dest=/dev/mapper/lvm_name,mkfs=ext3