Test Software

[DemonWasp]

At work, I've been assigned a project - I'm to supply (whether by writing it myself or by some free, open-source solution) a software solution for our testing environment. The environment is fairly complex: our product is deployed to several platforms on several different physical machines.

Ideally, I want to be able to run tests and builds through a website, with test results stored for later checking and analysis. It should be possible to run tests on all machines (so that you can easily run regression testing on your recent change, for example). I'd like for there to be features that allow builds to come from anywhere, or allow the software to build the project from a given source tree, potentially even one on another system (so that you can make a change on one system, then quickly check that it compiles and works on other systems).

I've looked at CruiseControl, but it looks like it's a single-machine solution. There are hacks to get it to report results across platforms, but they're not very convenient. There's also no capacity for on-demand build/tests or similar.


The alternative is that I continue my effort to develop a system that meets my requirements. Development has been going well, but if I can deliver something based on an existing FOSS project in 1 month rather than my own effort in 3 months, that's probably better.


Does anyone have experience with a test suite that comes even remotely close to fulfilling my needs?

[Tony]

Take a look at how major FOSS cross-platform projects do it.

https://developer.mozilla.org/en/Tinderbox

Quote:

Mozilla runs build computers ("Tinderboxen") that continually build the latest source code; the tinderbox webtool shows the status of these builds.