Previous | 
Home
Section: New Software and Platforms
Other Software
Omnisc'IO
Participants : Matthieu Dorier, Shadi Ibrahim, Gabriel Antoniu.
- Contact:
- Presentation:
-
Omnisc'IO is a middleware integrated in the POSIX and MPI-I/O stacks to observe, model and predict the I/O behavior of any HPC application transparently. It is based on formal grammars, implementing a modified version of the Sequitur algorithm. Omnisc'IO has been used on Grid'5000 with the CM1 atmospheric simulation, the LAMMPS molecular dynamics simulation, the GTC fusion simulation and the Nek5000 CFD simulation. Omnisc'IO was subject to a publication at SC14.
- Users:
-
Omnisc'IO is currently used only within the KerData team and at Argonne National Lab.
- URL:
- License:
- Status:
-
Currently unavailable for distribution (subject to major changes). Version 1.0 (released in November 2015) registered with APP: IDDN.FR.001.540003.000.S.P.2015.000.10000.
JetStream
Participants : Radu Tudoran, Alexandru Costan, Gabriel Antoniu.
- Contact:
- Presentation:
-
JetStream is a middleware solution for batch-based, high-performance streaming across cloud data centers. JetStream implements a set of context-aware strategies for optimizing batch-based streaming, being able to self-adapt to changing conditions. Additionally, the system provides multi-route streaming across cloud data centers for aggregating bandwidth by leveraging the network parallelism. It enables easy deployment across .Net frameworks and seamless binding with event processing engines such as StreamInsight.
- Users:
-
JetStream is currently used at Microsoft Research ATLE Munich for the management of the Azure cloud infrastructure.
- License:
- Status:
OverFlow
Participants : Radu Tudoran, Alexandru Costan, Gabriel Antoniu.
- Contact:
- Presentation:
-
OverFlow is a uniform data management system for scientific workflows running across geographically distributed sites, aiming to reap economic benefits from this geo-diversity. The software is environment-aware, as it monitors and models the global cloud infrastructure, offering high and predictable data handling performance for transfer cost and time, within and across sites. OverFlow proposes a set of pluggable services, grouped in a data-scientist cloud kit. They provide the applications with the possibility to monitor the underlying infrastructure, to exploit smart data compression, deduplication and geo-replication, to evaluate data management costs, to set a tradeoff between money and time, and optimize the transfer strategy accordingly. In 2015, OverFlow was extended with support for efficient metadata operations: the newly implemented strategies leverage workflow semantics in a 2-level metadata partitioning hierarchy that combines distribution and replication.
- Users:
-
Currently, OverFlow is used for data transfers by the Microsoft Research ATLE Munich team as well as for synthetic benchmarks at the Politehnica University of Bucharest.
- License:
- Status:
iHadoop
Participants : Tien Dat Phan, Shadi Ibrahim.
- Contact:
- Presentation:
-
iHadoop is a Hadoop simulator developed in Java on top of SimGrid to simulate the behavior of Hadoop and therefore accurately predict the performance of Hadoop in normal scenarios and under failures. In 2015, iHadoop was extended to simulate the execution and predict the performance of multiple Map-Reduce applications, sharing the same Hadoop cluster. Two schedulers (Fifo, Fair) are now available in iHadoop.
- Users:
-
iHadoop is an internal software prototype, which was initially developed to validate our idea for exploring the behavior of Hadoop under failures. iHadoop has preliminarily evaluated within our group and it has shown very high accuracy when predicating the execution time of a Map-Reduce application. iHadoop was discussed with the SimGrid community during the SimGrid user days in Lyon (June 2015). We intend to add iHadoop to the contributions site of the SimGrid project and make it available to the SimGrid community.
- License:
- Status:
-
Available on Inria's forge. Registration of the latest version with APP is in progress.