Scientific Computation with Linux on the PlayStation 3

[Hikaru79]

As a few of you know, I'm involved in some Bioinformatics research for the Applied Math department at Waterloo; basically just helping to write code to parallelize some RNA-folding code (not written by us) and spread it among large grids, like Sharcnet.

Furthermore, as recent stats have shown, it turns out that the PlayStation 3 is actually very effective at this sort of thing relative to its cost. So we've convinced the Math Endowment Fund to buy us a few PS3's, and if we get good results with them, they'll buy a lot more. Of course, in order to run this code (a mixture of C and Java) on them, we need to install an Operating System on them that can run this. Yellow Dog Linux to the rescue!

I took one of the PlayStatons home with me over the weekend and spent a few hours installing YDL on it; it was supremely easy. In fact, the PS3's own OS facilitated most of the process. It's such a refreshing change to see console manufacturers actually go out of their way to HELP people doing this sort of stuff, rather than doing their best to hinder it.

Anyway, in a short time, it was up and running (SSH even came prebundled). I've attached some screenshots below of SSH sessions from my Macbook Pro.

A picture's worth a thousand words, so ON TO THE PICTURES =)

http://i141.photobucket.com/albums/r59/hikaru79/Picture1.png
This is just my first ever SSH entry into the PS3. /proc/cpuinfo output is included just to make sure I'm not lying

Here's a more interesting one: http://i141.photobucket.com/albums/r59/hikaru79/Picture2.png
The top terminal is the code for a really quick benchmark I wrote in C (an intentionally inefficient recursive fibonacci).

The bottom-left terminal is an SSH session showing the new PS3 workhorse running that benchmark.

The bottom-right is my local computer (the Macbook Pro) running the exact same benchmark.

As you can see (if the picture isn't too small), my 2.16 Ghz Macbook Pro is running the benchmark almost six times faster than the Playstation 3. This goes to show that in order to make these things worthwhile, we have to take advantage of the SPE's. To this end, we've been recommended to use RapidMind, which is Waterloo's C++ platform for "stream programming". I have no idea how to do this (yet), but I'll keep showing screenshots and benchmarks as they show improvement =)

I'll also put up some photographs of the Playstation running on an actual TV with Linux as proof. (It's running Gnome at the moment, but it will just be in terminal mode most of the time). I'll do that in the morning; I dunno where the digital camera is right now and I don't want to wake up the whole house looking for it.

[klopyrev]

Wow... That looks really interesting!!! I want to do stuff like that when I go to university! Are you a student at Waterloo or are you just researching there? If you are a student, what program are you in?

KL

[Clayton]

He's a student at UWaterloo and is double majoring in CS and Combinatorics (I believe?)

Anyways, very nice Hikaru79! What exactly made it so easy to install Linux on the PS3 anyways? Either way, I'm intrigued

[Hikaru79]

Oh my, cs.ca sure is slow ever since the digg

Clayton @ Mon Apr 23, 2007 9:24 am wrote:

What exactly made it so easy to install Linux on the PS3 anyways? Either way, I'm intrigued

I already talked to you about this online, but I'll re-iterate for the benefit of everyone else.

Essentially, Sony has gone out of their way to make it easy to install other platforms on the Playstation 3; their GameOS has a built in partitioner (which will automatically split the drive to make room for something else). A bootloader is available from their website (that'll allow you to boot from USB to launch the installer), and so on. 'twas very enjoyable.

On to more stuff; as I said before, the PS3 is terribly slow with a vanilla Linux install because most of its usefulness comes from its SPE processors which the kernel does not automatically tap. The solution is RapidMind, a C++ platform that abstracts away the hardware and spreads the work among the Cell processor cores.

Yesterday, I got that platform up and running. Here's a screenshot: http://i141.photobucket.com/albums/r59/hikaru79/Picture3.png
On the left is the modified version of a simple C++ program that operates on all the terms of some 10,000-long arrays. As you can see on the right, the unmodified version runs in 18.52 seconds; the stream'ed version shown runs in under half a second.

I'll keep you posted as I learn more about RapidMind; the eventual goal is to port 'zepto' (an RNA-folding simulation program) to it, and transfer the current grid that is running on Sharcnet, to a grid based on PS3's.

[rdrake]

Sounds to me like you're doing something similar to what the Fold@Home project is doing. Ars had a nice article on the subject about a month back.

[Skynet]

How many PS3s, if I may ask? Doesn't Sharcnet have 4000-odd processors, with 1500 or so in the Phys-Eng building?

[Hikaru79]

rdrake @ Tue Apr 24, 2007 4:21 pm wrote:

Sounds to me like you're doing something similar to what the Fold@Home project is doing. Ars had a nice article on the subject about a month back.

Yes, in fact we're doing something very similar to Folding@Home. The main differences are that we're folding RNA, not proteins, and that (for the moment at least) it's not being opened up to the general public; it's only being run on Sharcnet clusters (hosted by Universities) running GridBase software. The project simply isn't large enough to justify getting something as big as Folding@Home started.

Skynet wrote:

How many PS3s, if I may ask? Doesn't Sharcnet have 4000-odd processors, with 1500 or so in the Phys-Eng building?

I'm not exactly sure how large Waterloo's entire Sharcnet node is; at the moment, this research only has access to the 'Vidal' node, about 80 CPU's. It seems that the PS3's success in other environments has been encouraging for the department here, because good results with the PS3's we have now will presumably lead to more of them being purchased and used in much the same way Sharcnet/Vidal is being used now.

NOTE: 1,000 posts now Yay!

[Andy]

If you're using the PS3s for parallel programming, I suggest you read a bit into GPGPU, apparently one of the profs at waterloo is an world authority on it.

[Hikaru79]

Andy @ Tue Apr 24, 2007 7:06 pm wrote:

If you're using the PS3s for parallel programming, I suggest you read a bit into GPGPU, apparently one of the profs at waterloo is an world authority on it.

Yeah; you're probably referring to Michael McCool (that really is his name!) We're already in contact with him; in fact, he's the one who recommended RapidMind to us. (In fact, he's one of the founders of RapidMind's parent company). He's the closest to a 'world authority' I think we have.

[Martin]

Hikaru, you might want to talk to the CM club as well (they're down that hall past MathSoc). They have two PS3s, and as far as I know haven't gotten around to doing much with them yet, as far as I know, and I'm sure they'd be up for some cool suggestions.

[Hikaru79]

Martin @ Wed Apr 25, 2007 4:43 pm wrote:

Hikaru, you might want to talk to the CM club as well (they're down that hall past MathSoc). They have two PS3s, and as far as I know haven't gotten around to doing much with them yet, as far as I know, and I'm sure they'd be up for some cool suggestions.

I'm a step ahead The test machine used in the screenshots was one of the machines Aly of the CM club gave us.

[Martin]

Oh, score.

You've inspired me, I'm going to install Linux on my housemate's PS3 this evening (if he lets me... otherwise, tomorrow morning )

[Clayton]

Out of curiosity, what size are the hard drives in the PS3's that you guys got?

Martin, ninja'ing your own housemate's electronics? For shame...

[rdrake]

Clayton @ Wed Apr 25, 2007 4:34 pm wrote:

Out of curiosity, what size are the hard drives in the PS3's that you guys got?

Any newer ones they'll buy will probably be 60 GB. Sony decided it would be wise to only keep the higher model.

Oh, and you guys made me decide to finally put Linux on my Xbox. It's currently got lighttpd, MySQL, PHP, and a few other things running.



(Crappy photo I know.)

[Hikaru79]

Wow It's a freaking revolution!

Clayton wrote:

Out of curiosity, what size are the hard drives in the PS3's that you guys got?

The test machine I'm using right now only has a 20GB hard drive. This is enough, though; all the result files are stored remotely to an SQL database anyway, not kept in physical result files on disk.