----------------------------------- Hikaru79 Mon Apr 23, 2007 1:22 am Scientific Computation with Linux on the PlayStation 3 ----------------------------------- As a few of you know, I'm involved in some Bioinformatics research for the Applied Math department at Waterloo; basically just helping to write code to parallelize some RNA-folding code (not written by us) and spread it among large grids, like Sharcnet. Furthermore, as recent stats have shown, it turns out that the PlayStation 3 is actually very effective at this sort of thing relative to its cost. So we've convinced the Math Endowment Fund to buy us a few PS3's, and if we get good results with them, they'll buy a lot more. Of course, in order to run this code (a mixture of C and Java) on them, we need to install an Operating System on them that can run this. Yellow Dog Linux to the rescue! I took one of the PlayStatons home with me over the weekend and spent a few hours installing YDL on it; it was supremely easy. In fact, the PS3's own OS facilitated most of the process. It's such a refreshing change to see console manufacturers actually go out of their way to HELP people doing this sort of stuff, rather than doing their best to hinder it. Anyway, in a short time, it was up and running (SSH even came prebundled). I've attached some screenshots below of SSH sessions from my Macbook Pro. A picture's worth a thousand words, so ON TO THE PICTURES =) http://i141.photobucket.com/albums/r59/hikaru79/Picture1.png This is just my first ever SSH entry into the PS3. /proc/cpuinfo output is included just to make sure I'm not lying :) Here's a more interesting one: http://i141.photobucket.com/albums/r59/hikaru79/Picture2.png The top terminal is the code for a really quick benchmark I wrote in C (an intentionally inefficient recursive fibonacci). The bottom-left terminal is an SSH session showing the new PS3 workhorse running that benchmark. The bottom-right is my local computer (the Macbook Pro) running the exact same benchmark. As you can see (if the picture isn't too small), my 2.16 Ghz Macbook Pro is running the benchmark almost six times faster than the Playstation 3. This goes to show that in order to make these things worthwhile, we have to take advantage of the SPE's. To this end, we've been recommended to use RapidMind, which is Waterloo's C++ platform for "stream programming". I have no idea how to do this (yet), but I'll keep showing screenshots and benchmarks as they show improvement =) I'll also put up some photographs of the Playstation running on an actual TV with Linux as proof. (It's running Gnome at the moment, but it will just be in terminal mode most of the time). I'll do that in the morning; I dunno where the digital camera is right now and I don't want to wake up the whole house looking for it. ----------------------------------- klopyrev Mon Apr 23, 2007 2:46 am Re: Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Wow... That looks really interesting!!! I want to do stuff like that when I go to university! Are you a student at Waterloo or are you just researching there? If you are a student, what program are you in? KL ----------------------------------- Clayton Mon Apr 23, 2007 8:24 am RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- He's a student at UWaterloo and is double majoring in CS and Combinatorics (I believe?) Anyways, very nice Hikaru79! What exactly made it so easy to install Linux on the PS3 anyways? Either way, I'm intrigued :) ----------------------------------- Hikaru79 Tue Apr 24, 2007 1:20 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Oh my, cs.ca sure is slow ever since the digg :( What exactly made it so easy to install Linux on the PS3 anyways? Either way, I'm intrigued :) I already talked to you about this online, but I'll re-iterate for the benefit of everyone else. Essentially, Sony has gone out of their way to make it easy to install other platforms on the Playstation 3; their GameOS has a built in partitioner (which will automatically split the drive to make room for something else). A bootloader is available from their website (that'll allow you to boot from USB to launch the installer), and so on. 'twas very enjoyable. On to more stuff; as I said before, the PS3 is terribly slow with a vanilla Linux install because most of its usefulness comes from its SPE processors which the kernel does not automatically tap. The solution is [url=http://www.rapidmind.net/]RapidMind, a C++ platform that abstracts away the hardware and spreads the work among the Cell processor cores. Yesterday, I got that platform up and running. Here's a screenshot: http://i141.photobucket.com/albums/r59/hikaru79/Picture3.png On the left is the modified version of a simple C++ program that operates on all the terms of some 10,000-long arrays. As you can see on the right, the unmodified version runs in 18.52 seconds; the stream'ed version shown runs in under half a second. I'll keep you posted as I learn more about RapidMind; the eventual goal is to port 'zepto' (an RNA-folding simulation program) to it, and transfer the current grid that is running on Sharcnet, to a grid based on PS3's. ----------------------------------- rdrake Tue Apr 24, 2007 3:21 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Sounds to me like you're doing something similar to what the Fold@Home project is doing. [url=http://arstechnica.com/news.ars/post/20070326-why-the-playstation-3-owns-the-pc-in-fh.html]Ars had a nice article on the subject about a month back. ----------------------------------- Skynet Tue Apr 24, 2007 3:23 pm Re: Scientific Computation with Linux on the PlayStation 3 ----------------------------------- How many PS3s, if I may ask? Doesn't Sharcnet have 4000-odd processors, with 1500 or so in the Phys-Eng building? ----------------------------------- Hikaru79 Tue Apr 24, 2007 4:33 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Sounds to me like you're doing something similar to what the Fold@Home project is doing. Yes, in fact we're doing something very similar to Folding@Home. The main differences are that we're folding RNA, not proteins, and that (for the moment at least) it's not being opened up to the general public; it's only being run on Sharcnet clusters (hosted by Universities) running GridBase software. The project simply isn't large enough to justify getting something as big as Folding@Home started. How many PS3s, if I may ask? Doesn't Sharcnet have 4000-odd processors, with 1500 or so in the Phys-Eng building? I'm not exactly sure how large Waterloo's entire Sharcnet node is; at the moment, this research only has access to the 'Vidal' node, about 80 CPU's. It seems that the PS3's success in other environments has been encouraging for the department here, because good results with the PS3's we have now will presumably lead to more of them being purchased and used in much the same way Sharcnet/Vidal is being used now. NOTE: 1,000 posts now :D Yay! ----------------------------------- Andy Tue Apr 24, 2007 6:06 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- If you're using the PS3s for parallel programming, I suggest you read a bit into GPGPU, apparently one of the profs at waterloo is an world authority on it. ----------------------------------- Hikaru79 Tue Apr 24, 2007 6:37 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- If you're using the PS3s for parallel programming, I suggest you read a bit into GPGPU, apparently one of the profs at waterloo is an world authority on it. Yeah; you're probably referring to [url=http://www.cgl.uwaterloo.ca/~mmccool/]Michael McCool (that really is his name!) We're already in contact with him; in fact, he's the one who recommended RapidMind to us. (In fact, he's one of the founders of RapidMind's parent company). He's the closest to a 'world authority' I think we have. ----------------------------------- Martin Wed Apr 25, 2007 3:43 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Hikaru, you might want to talk to the CM club as well (they're down that hall past MathSoc). They have two PS3s, and as far as I know haven't gotten around to doing much with them yet, as far as I know, and I'm sure they'd be up for some cool suggestions. ----------------------------------- Hikaru79 Wed Apr 25, 2007 3:54 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Hikaru, you might want to talk to the CM club as well (they're down that hall past MathSoc). They have two PS3s, and as far as I know haven't gotten around to doing much with them yet, as far as I know, and I'm sure they'd be up for some cool suggestions. I'm a step ahead :) The test machine used in the screenshots was one of the machines Aly of the CM club gave us. ----------------------------------- Martin Wed Apr 25, 2007 3:59 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Oh, score. You've inspired me, I'm going to install Linux on my housemate's PS3 this evening (if he lets me... otherwise, tomorrow morning ;) ) ----------------------------------- Clayton Wed Apr 25, 2007 4:34 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Out of curiosity, what size are the hard drives in the PS3's that you guys got? Martin, ninja'ing your own housemate's electronics? For shame... :P ----------------------------------- rdrake Wed Apr 25, 2007 7:24 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Out of curiosity, what size are the hard drives in the PS3's that you guys got?Any newer ones they'll buy will probably be 60 GB. Sony decided it would be wise to only keep the higher model. Oh, and you guys made me decide to finally put Linux on my Xbox. It's currently got lighttpd, MySQL, PHP, and a few other things running. http://rdrake.org/compsci.ca/DSCI0083.JPG (Crappy photo I know.) ----------------------------------- Hikaru79 Wed Apr 25, 2007 8:18 pm Re: Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Wow :D It's a freaking revolution! Out of curiosity, what size are the hard drives in the PS3's that you guys got? The test machine I'm using right now only has a 20GB hard drive. This is enough, though; all the result files are stored remotely to an SQL database anyway, not kept in physical result files on disk. ----------------------------------- Clayton Wed Apr 25, 2007 8:30 pm Re: Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Wow :D It's a freaking revolution! Heh, well, if I ever get a PS3 (which I may if I can go halfway over the summer with my parents), I'm definitely installing YDL. How has your PS3 been handling the test runs so far (assuming you've started)? ----------------------------------- Martin Thu Apr 26, 2007 8:30 am RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- My roommate's is 60GB. So, I installed Ubuntu. It went great, but there are no wireless drivers. And I wiped out everyone's save games. Oops. ----------------------------------- Clayton Thu Apr 26, 2007 9:35 am RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Well done Martin. A PS3 has an ethernet port, no? ----------------------------------- rdrake Thu Apr 26, 2007 10:42 am Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- A PS3 has an ethernet port, no?Yes. Oh, and [url=http://darkmatterirc.no-ip.org/sysinfo/]here is a link to my Xbox which is currently running Lighty. It might be up, might not be. Kinda loads slowly since it's emerging xfce currently :?. ----------------------------------- Martin Thu Apr 26, 2007 2:54 pm RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Yeah, but the router is upstairs and I'd need about 50 ft of cat 5 to reach it. ----------------------------------- Hikaru79 Thu Apr 26, 2007 5:26 pm Re: RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Yeah, but the router is upstairs and I'd need about 50 ft of cat 5 to reach it. I recommend YDL over Ubuntu for the PS3; they have deals with Sony to give them access to docs that the Ubuntu team doesn't have as far as I know. Moreover, RapidMind is only supported on FC6+ and YDL5+ (If you're planning on using RM, which I recommend) ----------------------------------- Martin Fri Apr 27, 2007 9:10 am RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- Okay, sounds good. How much does a RapidMind license cost, and how do I get one? ----------------------------------- g_man1988 Wed Jan 09, 2008 1:41 am RE:Scientific Computation with Linux on the PlayStation 3 ----------------------------------- O_O Wow, it's been a long time since I've posted on compsci.ca. A RapidMind license is free for development, so you can download it any time via developer.rapidmind.net. *thnx to Hikaru79 for helping out during my first co-op term about the PS3 stuffs ^^. Currently, the CM club owns 2 PS3s. Both have RapidMind installed. I also own a PS3 (20GB upgraded to 120GB) with RapidMind. Anyone interested is highly encouraged to help out. This project is really interesting. Also I think Sharcnet is getting some Nvidia video cards when they come out with double float optimization (which the current generation lacks.) ----------------------------------- sirkit Fri Feb 22, 2008 10:01 pm Re: Scientific Computation with Linux on the PlayStation 3 ----------------------------------- As a few of you know, I'm involved in some Bioinformatics research for the Applied Math department at Waterloo; basically just helping to write code to parallelize some RNA-folding code (not written by us) and spread it among large grids, like Sharcnet. Furthermore, as recent stats have shown, it turns out that the PlayStation 3 is actually very effective at this sort of thing relative to its cost. So we've convinced the Math Endowment Fund to buy us a few PS3's, and if we get good results with them, they'll buy a lot more. Of course, in order to run this code (a mixture of C and Java) on them, we need to install an Operating System on them that can run this. Yellow Dog Linux to the rescue! I took one of the PlayStatons home with me over the weekend and spent a few hours installing YDL on it; it was supremely easy. In fact, the PS3's own OS facilitated most of the process. It's such a refreshing change to see console manufacturers actually go out of their way to HELP people doing this sort of stuff, rather than doing their best to hinder it. Anyway, in a short time, it was up and running (SSH even came prebundled). I've attached some screenshots below of SSH sessions from my Macbook Pro. A picture's worth a thousand words, so ON TO THE PICTURES =) http://i141.photobucket.com/albums/r59/hikaru79/Picture1.png This is just my first ever SSH entry into the PS3. /proc/cpuinfo output is included just to make sure I'm not lying :) Here's a more interesting one: http://i141.photobucket.com/albums/r59/hikaru79/Picture2.png The top terminal is the code for a really quick benchmark I wrote in C (an intentionally inefficient recursive fibonacci). The bottom-left terminal is an SSH session showing the new PS3 workhorse running that benchmark. The bottom-right is my local computer (the Macbook Pro) running the exact same benchmark. As you can see (if the picture isn't too small), my 2.16 Ghz Macbook Pro is running the benchmark almost six times faster than the Playstation 3. This goes to show that in order to make these things worthwhile, we have to take advantage of the SPE's. To this end, we've been recommended to use RapidMind, which is Waterloo's C++ platform for "stream programming". I have no idea how to do this (yet), but I'll keep showing screenshots and benchmarks as they show improvement =) I'll also put up some photographs of the Playstation running on an actual TV with Linux as proof. (It's running Gnome at the moment, but it will just be in terminal mode most of the time). I'll do that in the morning; I dunno where the digital camera is right now and I don't want to wake up the whole house looking for it. Sorry to go a little bit off topic here but... WOW, this type of project sounds really fascinating to me. It's the type of thing I would immensely enjoy doing in my spare time, whether it's related to school or not. In fact, it's what I thought I would be doing as part of my studies, but unfortunately, I'm not. Let me explain... I'm a first year Computer Engineering student at UofT, and frankly my experience has been far from what I was hoping for. I'm taking math, physics, chemistry and programming courses for the most part and I don't find any of them interesting... There are no connections to research or interesting upper year stuff and until NOW I've been wondering why I chose computer engineering in the first place... Thanks to your post about your project, I remember now. I was always fascinated by hardware, chip design etc. but you would never know from the curriculum of courses I'm taking right now. My question is, does it get more interesting in later years? Are projects like yours common? I'm especially interested in hearing from people who have done Comp/Elec Eng or Comp Sci at UofT. Again sorry for sort of hijacking this thread but I felt compelled to say this!