clusterKnoppix, A Clustering Success

It has been a while since I have posted, because we have been working on our final project, making a cluster out of few computers. As I said before, we were having difficulties getting OSCAR set up. At one point it seemed to have imaged the nodes correctly, however upon testing the cluster using OSCAR’s feature to do so, we found we had several errors. We were unable to fix them, and so we decided to move on for now and try different tools. From here we tried using Rocks Cluster Distribution. You can either download the DVD with the Operating System and all the packages, CDs, or pick and choose what software you wanted and burn them to individual CDs. We originally went with the DVD choice, however, during installation, it asked the source from which it would get the packages, and it wouldn’t recognize our DVD as a choice. We tried CDs, but it would keep restarting in the middle without doing anything. Both Rocks and OSCAR had extensive documentation which I appreciated, however, since we could not get them working, we looked for another solution. During my research, clusterKnoppix always showed up, however, since it was more of a LiveCD based solution, I kept it on hold. Although from what I read, it should be easy to get one going by using LiveCDs. So we burned two and popped them into two computers, and followed these instructions. However, we would keep getting errors setting it up. I went back and did some more research, and found these instructions, which were to just boot from the CDs and they would see each other as long as they were assigned ip addresses. I immediately tried again, and they worked without any problems. I tried out the simple load balancing script that was in the first set of instructions, and it all seemed to work. The next step was to make it a permanent solution by installing it to the hard drive. We followed these instructions (the 7th post, a long set of instructions). Instead of using QTparted to partition our hard drive, we used GPart from the Ubuntu LiveCd because QTparted would not see our hard drive. We did not implement a separate partition for the cd image and we did not copy the clusterKnoppix CD contents to the hard drive the way that was suggested. Once the hard drive install was complete, we followed the instructions up to where it said “3. new: create directory for export: OK” which we selected, and we left the CD in the drive, from which it copied files. However, it created a folder called “diskless” in the root folder. After we found our cluster didn’t work, we deleted the folder and made a new “diskless” in the same location, but we copied the files from the CD to this location. In addition, for the “Client Hardware” section of the how-to, we just left the defaults, and eventually at the end everything works. We skipped some instructions, and went to where it said “Your terminal will complain about /mfs not being set-up correctly when…” and followed these steps. In addition, we also edited a script from these instructions (4th post) which we chose to implement before completing the cluster set-up. Eventually, we restarted terminalopenmosixserver (you may have to select stop, and then start) and did a openmosix restart. We booted up the nodes and watched from openmosixview as they began to appear. Our cluster was a success! During the terminalopenmosixserver and openmosix restarts, and also when booting up the nodes, there may have been small errors or warnings, but we let them be unless our cluster did not work. clusterKnoppix is a great and easy way to cluster; you can have a temporary cluster just to see it work by using the LiveCDs, or have a more permanent hard drive install.

OSCAR…not working quite yet

Yesterday, I used the start_over script in OSCAR to…start over. I did the log out and back in it said, and I proceeded to reinstall and reconfigure OSCAR. This time, though, I opted to use a mirror for the rpms it needed, and I then proceeded to build the client image. After a very long time, an error was reported, it told me to check a log file which is really a script. Meanwhile, the client computers don’t even boot past the bios with the hard drives in. Today, we will have to somehow format the hard drives and start over with OSCAR.

Getting OSCAR ready

Our next task is to set up a small cluster, a “supercomputer”. Right now, there are two software based solutions we are going to look at, both involving Linux. The first one which is supposed to be easier to set up, and very well documented, is OSCAR. I found the instructions to setting it up here. We decided on using Fedora 5 as our OS base, since it was fully supported, so we downloaded it and installed it on the Dell server. Once this was complete, I began setting up OSCAR. Once a lot of the tedious set up was over, I attempted to run the set up script like it said in the documentation. However, the name in the documentation was wrong as I soon discovered, I did find it, though. A few more errors popped up, however, I just looked at the log and made the necessary changes. Once this was complete, the gui based setup came up, and I proceeded to install the server package, and next was the image for the client nodes. Setting it up though, required some more legwork. I had originally set it up so the necessary rpms for the client nodes would be downloaded from a mirror. I thought this would be too slow, so I got the rpms off the cds for Fedora. However, I was getting errors that dependencies, other packages that packages need, were missing. Therefore, we tried a combination of both cds and mirror, and after a long wait, it said completed successfully. Next was the deployment of the images. After setting it so it would get the image off the server, I turned on three clients. The first one got errors, which I believe occurred because the first one has to be the server. The other two seemed to have installed. However, those two no longer can boot from the hard disk, in fact, they don’t boot past the HP screen unless the hard drive is removed. Brian removed the CMOS battery, and I put them back and tried the computers again this morning, however, to no avail. I was waiting to post some good news in our supercomputing ventures, but here is where we’re at.