Section: New Software and Platforms

NewMadeleine

NewMadeleine: An Optimizing Communication Library for High-Performance Networks

Keywords: High-performance calculation - MPI communication

Functional Description: NewMadeleine is the fourth incarnation of the Madeleine communication library. The new architecture aims at enabling the use of a much wider range of communication flow optimization techniques. Its design is entirely modular: drivers and optimization strategies are dynamically loadable software components, allowing experimentations with multiple approaches or on multiple issues with regard to processing communication flows.

The optimizing scheduler SchedOpt targets applications with irregular, multi-flow communication schemes such as found in the increasingly common application conglomerates made of multiple programming environments and coupled pieces of code, for instance. SchedOpt itself is easily extensible through the concepts of optimization strategies (what to optimize for, what the optimization goal is) expressed in terms of tactics (how to optimize to reach the optimization goal). Tactics themselves are made of basic communication flows operations such as packet merging or reordering.

The communication library is fully multi-threaded through its close integration with PIOMan. It manages concurrent communication operations from multiple libraries and from multiple threads. Its MPI implementation MadMPI fully supports the MPI_THREAD_MULTIPLE multi-threading level.

• Participants: Alexandre Denis, Clément Foyer, Nathalie Furmento, Raymond Namyst, Adrien Guilbaud, Florian Reynier and Philippe Swartvagher

• Contact: Alexandre Denis

• Publications: NewMadeleine: a Fast Communication Scheduling Engine for High Performance Networks - Ordonnancement et qualité de service pour réseaux rapides - Improving Reactivity and Communication Overlap in MPI using a Generic I/O Manager - PIOMan : un gestionnaire d'entrées-sorties générique - A multithreaded communication engine for multicore architectures - A multicore-enabled multirail communication engine - About the interactions between communication and thread scheduling in clusters of multicore machines - Scalability of the NewMadeleine Communication Library for Large Numbers of MPI Point-to-Point Requests - An analysis of the impact of multi-threading on communication performance - A scalable and generic task scheduling system for communication libraries - A Generic and High Performance Approach for Fault Tolerance in Communication Library - A High-Performance Superpipeline Protocol for InfiniBand - A sampling-based approach for communication libraries auto-tuning - High performance checksum computation for fault-tolerant MPI over InfiniBand - pioman: a Generic Framework for Asynchronous Progression and Multithreaded Communications - pioman: a pthread-based Multithreaded Communication Engine - Updating MadMPI to MPI-3: Remote Memory Access - Portage de StarPU sur la bibliothèque de communication NewMadeleine

• URL: http://pm2.gforge.inria.fr/newmadeleine/