Data intensive simulation on EGEE with SimExplorer

Distribution of a data intensive simulation on EGEE (biomed) with the development version of SimExplorer (Workflow version).

data size : 33 MB of data / job

total : 50 GB of results

Note : SimExplorer, provides transparent data compression during file transfers. It hasn't exchanged 50 GB of raw data with the grid.

A simulation of in self propelled has been encapsulated into SimExplorer (workflow version) from a version running on a SGE (Sun Grid Engine) cluster. The integration effort has long last than an hour.

1562 jobs have been executed the following design :

design.addExperimentalFactor(new ExperimentalFactor<BigDecimal>(eta,new RangeBigDecimal("0.0","0.3","0.01")));
design.addExperimentalFactor(new ExperimentalFactor<BigDecimal>(angle,new RangeBigDecimal("0.0","5.0","0.1")));

The launching command is described has follow:

"./follow_me_2d_gaussianNnr 1045674222 1000 4 128 ${eta} ${angle} 50 0.01 1000000 fm_${eta}_${angle}.dat"

Each job runs in 45 minutes. The execution of the 1562 experiments would have require 48,8 day on a single computer (48,8 days / CPU). It has been executed in 9,8 hours on the biomed grid. The crunching factor is 118,7. The bottleneck was the slow Internet connection the experiment has been executed from (1 MB/s upstream speed transfer).