Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The IceProd framework: Distributed data processing for the IceCube neutrino observatory
Show others and affiliations
Responsible organisation
2015 (English)In: Journal of Parallel and Distributed Computing, ISSN 0743-7315, E-ISSN 1096-0848, Vol. 75Article in journal (Refereed) Published
Abstract [en]

IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. (C) 2014 Elsevier Inc. All rights reserved.

Place, publisher, year, edition, pages
2015. Vol. 75
Keyword [en]
Data management Grid computing Monitoring Distributed computing Computer Science
Research subject
SWEDARP, IceCube
Identifiers
URN: urn:nbn:se:polar:diva-1964DOI: 10.1016/j.jpdc.2014.08.001OAI: oai:DiVA.org:polar-1964DiVA: diva2:820703
Note

ISI Document Delivery No.: AW9BK Times Cited: 0 Cited Reference Count: 18 Aartsen, M. G. Abbasi, R. Ackermann, M. Adams, J. Aguilar, J. A. Ahlers, M. Altmann, D. Arguelles, C. Auffenberg, J. Bai, X. Baker, M. Barwick, S. W. Baum, V. Bay, R. Beatty, J. J. Tjus, J. Becker Becker, K. -H. BenZvi, S. Berghaus, P. Berley, D. Bernardini, E. Bernhard, A. Besson, D. Z. Binder, G. Bindig, D. Bissok, M. Blaufuss, E. Blumenthal, J. Boersma, D. J. Bohm, C. Bose, D. Boeser, S. Botner, O. Brayeur, L. Bretz, H. -P. Brown, A. M. Bruijn, R. Casey, J. Casier, M. Chirkin, D. Christov, A. Christy, B. Clark, K. Classen, L. Clevermann, F. Coenders, S. Cohen, S. Cowen, D. F. Silva, A. H. Cruz Danninger, M. Daughhetee, J. Davis, J. C. Day, M. De Clercq, C. De Ridder, S. Desiati, P. de Vries, K. D. de With, M. DeYoung, T. Diaz-Velez, J. C. Dunkman, M. Eagan, R. Eberhardt, B. Eichmann, B. Eisch, J. Euler, S. Evenson, P. A. Fadiran, O. Fazely, A. R. Fedynitch, A. Feintzeig, J. Feusels, T. Filimonov, K. Finley, C. Fischer-Wasels, T. Flis, S. Franckowiak, A. Frantzen, K. Fuchs, T. Gaisser, T. K. Gallagher, J. Gerhardt, L. Gladstone, L. Glusenkamp, T. Goldschmidt, A. Golup, G. Gonzalez, J. G. Goodman, J. A. Gora, D. Grandmont, D. T. Grant, D. Gretskov, P. Groh, J. C. Gross, A. Ha, C. Ismail, A. Haj Hallen, P. Hallgren, A. Halzen, F. Hanson, K. Hebecker, D. Heereman, D. Heinen, D. Helbing, K. Hellauer, R. Hickford, S. Hill, G. C. Hoffman, K. D. Hoffmann, R. Homeier, A. Hoshina, K. Huang, F. Huelsnitz, W. Hulth, P. O. Hultqvist, K. Hussain, S. Ishihara, A. Jacobi, E. Jacobsen, J. Jagielski, K. Japaridze, G. S. Jero, K. Jlelati, O. Kaminsky, B. Kappes, A. Karg, T. Karle, A. Kauer, M. Kelley, J. L. Kiryluk, J. Klaes, J. Klein, S. R. Koehne, J. -H. Kohnen, G. Kolanoski, H. Koepke, L. Kopper, C. Kopper, S. Koskinen, D. J. Kowalski, M. Krasberg, M. Kriesten, A. Krings, K. Kroll, G. Kunnen, J. Kurahashi, N. Kuwabara, T. Labare, M. Landsman, H. Larson, M. J. Lesiak-Bzdak, M. Leuermann, M. Leute, J. Luenemann, J. Macias, O. Madsen, J. Maggi, G. Maruyama, R. Mase, K. Matis, H. S. McNally, F. Meagher, K. Merck, M. Merino, G. Meures, T. Miarecki, S. Middell, E. Milke, N. Miller, J. Mohrmann, L. Montaruli, T. Morse, R. Nahnhauer, R. Naumann, U. Niederhausen, H. Nowicki, S. C. Nygren, D. R. Obertacke, A. Odrowski, S. Olivas, A. Omairat, A. O'Murchadha, A. Paul, L. Pepper, J. A. de los Heros, C. Perez Pfendner, C. Pieloth, D. Pinat, E. Posselt, J. Price, P. B. Przybylski, G. T. Quinnan, M. Raedel, L. Rae, I. Rameez, M. Rawlins, K. Redl, P. Reimann, R. Resconi, E. Rhode, W. Ribordy, M. Richman, M. Riedel, B. Rodrigues, J. P. Rott, C. Ruhe, T. Ruzybayev, B. Ryckbosch, D. Saba, S. M. Sander, H. -G. Santander, M. Sarkar, S. Schatto, K. Scheriau, F. Schmidt, T. Schmitz, M. Schoenen, S. Schoeneberg, S. Schoenwald, A. Schukraft, A. Schulte, L. Schultz, D. Schulz, O. Secke, D. Sestayo, Y. Seunarine, S. Shanidze, R. Sheremata, C. Smith, M. W. E. Soldin, D. Spiczak, G. M. Spiering, C. Stamatikos, M. Stanev, T. Stanisha, N. A. Stasik, A. Stezelberger, T. Stokstad, R. G. Stoessl, A. Strahler, E. A. Stroem, R. Strotjohann, N. L. Sullivan, G. W. Taavola, H. Taboada, I. Tamburro, A. Tepe, A. Ter-Antonyan, S. Tesic, G. Tilav, S. Toale, P. A. Tobin, M. N. Toscano, S. Tselengidou, M. Unger, E. Usner, M. Vallecorsa, S. van Eijndhoven, N. van Overloop, A. van Santen, J. Vehring, M. Voge, M. Vraeghe, M. Walck, C. Waldenmaier, T. Wallraff, M. Weaver, Ch. Wellons, M. Wendt, C. Westerhoff, S. Whitehorn, N. Wiebe, K. Wiebusch, C. H. Williams, D. R. Wissing, H. Wolf, M. Wood, T. R. Woschnagg, K. Xu, D. L. Xu, X. W. Yanez, J. P. Yodh, G. Yoshida, S. Zarzhitsky, P. Ziemann, J. Zierke, S. Zoll, M. Auffenberg, Jan/D-3954-2014 Auffenberg, Jan/0000-0002-1185-9094 U.S. National Science Foundation-Office of Polar Programs; U.S. National Science Foundation-Physics Division; University of Wisconsin Alumni Research Foundation; Grid Laboratory Of Wisconsin (GLOW) grid infrastructure at the University of Wisconsin Madison; Open Science Grid (OSG) grid infrastructure; U.S. Department of Energy, and National Energy Research Scientific Computing Center; Louisiana Optical Network Initiative (LONI) grid computing resources; Natural Sciences and Engineering Research Council of Canada; WestGrid and Compute/Calcul Canada; Swedish Research Council; Swedish Polar Research Secretariat; Swedish National Infrastructure for Computing (SNIC); Knut and Alice Wallenberg Foundation, Sweden; German Ministry for Education and Research (BMBF); Deutsche Forschungsgemeinschaft (DFG); Helmholtz Alliance for Astroparticle Physics (HAP); Research Department of Plasmas with Complex Interactions (Bochum), Germany; Fund for Scientific Research (FNRS-FWO); FWO Odysseus programme; Flanders Institute to encourage scientific and technological research in industry (IWT); Belgian Federal Science Policy Office (Belspo); University of Oxford, United Kingdom; Marsden Fund, New Zealand; Australian Research Council; Japan Society for Promotion of Science (JSPS); Swiss National Science Foundation (SNSF), Switzerland; National Research Foundation of Korea (NRF); Danish National Research Foundation, Denmark (DNRF) We acknowledge the support from the following agencies: U.S. National Science Foundation-Office of Polar Programs, U.S. National Science Foundation-Physics Division, University of Wisconsin Alumni Research Foundation, the Grid Laboratory Of Wisconsin (GLOW) grid infrastructure at the University of Wisconsin Madison, the Open Science Grid (OSG) grid infrastructure; U.S. Department of Energy, and National Energy Research Scientific Computing Center, the Louisiana Optical Network Initiative (LONI) grid computing resources; Natural Sciences and Engineering Research Council of Canada, WestGrid and Compute/Calcul Canada; Swedish Research Council, Swedish Polar Research Secretariat, Swedish National Infrastructure for Computing (SNIC), and Knut and Alice Wallenberg Foundation, Sweden; German Ministry for Education and Research (BMBF), Deutsche Forschungsgemeinschaft (DFG), Helmholtz Alliance for Astroparticle Physics (HAP), Research Department of Plasmas with Complex Interactions (Bochum), Germany; Fund for Scientific Research (FNRS-FWO), FWO Odysseus programme, Flanders Institute to encourage scientific and technological research in industry (IWT), Belgian Federal Science Policy Office (Belspo); University of Oxford, United Kingdom; Marsden Fund, New Zealand; Australian Research Council; Japan Society for Promotion of Science (JSPS); the Swiss National Science Foundation (SNSF), Switzerland; National Research Foundation of Korea (NRF); Danish National Research Foundation, Denmark (DNRF). 0 Academic press inc elsevier science San diego Computer Science, Theory & Methods

Available from: 2015-06-12 Created: 2015-05-13 Last updated: 2015-06-12

Open Access in DiVA

No full text

Other links

Publisher's full texthttp://<Go to ISI>://WOS:000346552500017
In the same journal
Journal of Parallel and Distributed Computing

Search outside of DiVA

GoogleGoogle Scholar

Altmetric score

Total: 275 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf