High Performance Computing

ZDV participates in the development of intelligent adaptive storage systems

The ZDV is part of the European joint project ADMIRE, which has set itself the goal of developing a new adaptive storage system for high-performance computers. In cooperation with fourteen institutions from six countries, ADMIRE aims to significantly increase the running time of applications in the areas of weather forecasting, molecular dynamics, turbulence simulation, cartography, brain research and software cataloging.

The research team at the ZDV "Efficient Computing and Storage" focuses on the so-called ad-hoc file systems, which are controlled by the overall ADMIRE system for applications if required. This should significantly increase the overall I / O performance for applications and relieve the central parallel file systems (such as Luster). The GekkoFS file system, a ZDV in-house development, is used as the basis. This is adapted to the requirements of modern applications in high-performance computers so that the file system provides the highest possible I / O performance and can react dynamically to decisions made by the overall ADMIRE system.

                                                     The ADMIRE project at a glance © ADMIRE

Additional information: https://www.uni-mainz.de/presse/aktuell/13932_ENG_HTML.php


More news from the Data Center → may be found here.

SSH Gate

Some servers are only accessible inside the university network. JGU provides an SSH jumphost server you can use to connect to these servers.
For authentication, only → public key authentication is allowed.

First you need to create an SSH key. If you already own an SSH key, you can go to the next step.
A guide on how to create an SSH key can be found → here.

You need to link your SSH key to your user account.
To do so, visit https://account.uni-mainz.de/my-account/add-ssh-key
On this site you find an input field named 'SSH-Key hinzufügen'.

Paste your public key into this input field. The comment of this key must contain SSHGATE. You can edit your key inside the field after you pasted it. An SSH key comment is always at the end of the key. If you want to connect to multiple servers using this authentication you need to separate them using ,.
Example: ... SSHGATE,HPCGATE,HPCLOGIN

💡 The server name declares the servers the key is deployed to.
If the same key should be used on multiple servers all servers must be added to the comment. If multiple keys should be used each key comment must contain the corresponding server name.

When you have finished, click on SSH-Key Speichern.

To connect to a server using SSHGATE as a jumphost use the following command.
ssh -J username@sshgate.zdv.uni-mainz.de loginname@Targetserver
The -J option tells ssh to use the first server as a jumphost.

You can add the SSHGATE server to your ssh config file. This way the connection can be called with a shortcut.
To add a shortcut you need to edit the file ~/.ssh/config.
You need to add the following lines.
After the file is edited you can call the shortcut with ssh ShortcutName.

Posted on

ZDV’s in-house development maintains top position at the "10 Node Challenge" of the IO500

The GekkoFS file system, a cooperation between the ZDV’s "Efficient Computing and Storage" team and the Barcelona Supercomputing Center (BSC), has taken a top ten position in the "10-Node Challenge" of the IO500 world ranking.

Mainz, 13th August 2020

The new IO500 rankings in the field of HPC storage systems were published at this year's ISC High Performance in Frankfurt, an international, scientific conference and exhibition on supercomputing and high-performance computing. ZDV’s "Efficient Computing and Storage" team and the "Storage Systems for Extreme Computing" team from the Barcelona Supercomputing Center (BSC) ranked No. 4 in the "10-Node Challenge" with the GekkoFS burst buffer file system.

GekkoFS is an ephemeral file system capable of aggregating the local I/O capacity and performance of an HPC cluster’s compute node to create an ephemeral high-performance storage space that can be accessed by an application in a distributed manner. This storage space allows HPC applications and simulations to run in isolation from each other regarding I/O, which reduces interference and improves performance.

Marc-André Vef, who has worked extensively on GekkoFS for several years, is delighted and is already looking forward to the future: "We are not resting on our current status and want to continue to develop the file system with our partners in Barcelona. We are already planning for the next two years."

GekkoFS was funded by the priority program SPPEXA of the German Research Foundation (DFG) within the ADA-FS project and the European NEXTGenIO project.

IO500 - the benchmark of the storage community

The goal of IO500 is to compare various storage systems with each other. In the past years, IO500 has become the de facto benchmarking standard for HPC storage to ensure forward progress of the storage community towards common goals. The IO500 list is composed of different HPC and storage systems, ranking each submission based on its achieved metadata and data throughput. The "10 Node Challenge" plays a unique role and focuses on the file system’s efficiency, limiting the parallelism of the storage nodes to only ten nodes.

The score of 152 places GekkoFS in 6th place on the IO500's "10-Node Challenge List" and 26th place on the complete IO500 list, published annually at the ISC High Performance in Frankfurt and the SC Conference in the USA.


More news from the Data Center → may be found here.

Analog Computing – Past, Present, Future?

Lecture by Prof. Bernd Ulmann on Thursday. This lecture had been planned for 19.03.2020, 7pm. Location: Meeting Room (Konferenzraum) 3 of the Data Center. The lecture will be delayed. We shall post information about the new date here as soon as possible.

As classic stored-program digital computers are reaching physical and practical limits with respect to clock frequency, integration density, and suffer from problems like Amdahl's law, unconventional approaches to high-speed and/or high-energy-efficient computing can offer a path to more computing power at lower power consumption.One of these approaches are analog computers which solve problems by means of analog electronic models of the underlying mathematical equations. Largely forgotten since the late 1970s, analog computers are about to return in the form of fully reconfigurable integrated circuits. Coupling these with traditional digital computers yields so-called hybrid computers which combine the best of two worlds, the programmability and vast program libraries available for digital computers as well as the high performance and low power consumption of analog computers.

This talk briefly covers the history of analog computing, gives examples of current applications and future developments.


More news from the Data Center → may be found here.

GekkoFS ranked #4 in the IO500’s “10-Node Challenge” at SC19

The GekkoFS file system jointly developed by the ZDV and the Barcelona Supercomputing Center (BSC) has been ranked No. 4 in the IO500’s “10-Node Challenge”. GekkoFS is a file system capable of aggregating the local I/O capacity and performance of each compute node in a HPC cluster to produce a high-performance storage space that can be accessed in a distributed manner. This storage space allows HPC applications and simulations to run in isolation from each other with regards to I/O, which reduces interferences and improves performance.

The development has been funded in the context of the SPPEXA programme and the Europeans Horizon 2020’s NEXTGenIO project. The IO500’s “10-Node Challenge” list is a global ranking that uses multiple concurrent processes running in 10 compute nodes to benchmark the I/O performance of a HPC storage system in terms or bandwidth and throughput. GekkoFS’ score of 125 ranks it 4th in IO500’s 10-Node Challenge List and 9th in IO500’s full list .

The IO500 benchmark was run on the 34 compute nodes of the NEXTGenIO prototype cluster. Each of the prototype’s 34 nodes is equipped with two 2nd generation Intel® Xeon® Scalable processors and 3 TBytes of Intel® Optane™ DC persistent memory, thus providing approximately 102 TBytes of persistent I/O capacity to HPC applications.


More news from the Data Center → may be found here.