Software Engineer: Scientific Data Compression and Reduction | French national synchrotron facility

Nov 09 2021

SOLEIL is the French national synchrotron facility, located on the Saclay Plateau near Paris. It is a multi-disciplinary instrument and a research laboratory, whose mission is to conduct research programs using synchrotron radiation, to develop cutting-edge instrumentation on the beamlines, and to make those available to the scientific community. SOLEIL, a unique tool for both academic research and industrial applications across a wide range of disciplines including physics, biology, chemistry etc., is used by over 5 000 researchers coming from France and abroad. The facility is a “public” company employing about 500 people, founded by the CNRS and the CEA, and partner of the Paris-Saclay University.

This position is open in the frame of the WP7 “data compression and reduction” activity <https://www.leaps-innov.eu/wp-7> of the European project LEAPS-INNOV, which was launched in April 2021 for a duration of 4 years. As the data produced by experiments using synchrotron radiation is increasing exponentially, LEAPS-INNOV WP7 aims to help users to analyze them faster and reduce their storage cost. The envisaged solutions will implement compression mechanisms for experiments performed with techniques where all the collected data carry useful information, and sorting mechanisms for those where many (or most) of the collected data are not scientifically useful.

This position is assigned to the SOLEIL Information Systems Strategy within the General Management and will have the technical support of the head of the Data Reduction and Analysis group of the Experiments Division.

I.Mission

You will be responsible for collecting the needs of SOLEIL’s beamlines according to their experimental techniques, evaluating the performance of existing compression methods, defining, and implementing a prototype method for on-the-fly compression/decompression, participating in the evaluation and development of new compression/reduction algorithms studied with the partners of the European LEAPS-INNOV project.

You will work at the SOLEIL site, in close collaboration with software and database engineers from SOLEIL and the other facilities involved in LEAPS-INNOV. You will interact with the beamlines’ scientists.

II.Responsibilities and tasks

As part of the European project LEAPS-INNOV, you will:

identify representative datasets of the experimental techniques practiced on the beamlines and participate in the selection of reference datasets for the evaluation of compression/reduction methods and algorithms.

participate in the definition of evaluation criteria, which will be guided by experimental requirements (compression ratio, compression ratio, decompression ratio, hardware, and platform constraints) and downstream analysis requirements (information loss, usability, portability, deployment, and integration into common analysis packages).

establish a list of compression algorithms that can be applied to experimental data; you will evaluate their effectiveness on the identified datasets.

evaluate and participate in the development of new algorithms based on modern techniques consisting of four conceptual layers: pre-processing (denoising, transforming, sorting, splitting), compression codec (lossy or lossless), storage format, decompression codec.

propose methods to ensure high processing speed, with the simplest possible use.

evaluate the constraints related to development, installation, use, maintenance, and long-term support.

collaborate with the project partners and contribute to project progress meetings or task-specific meetings, which will be held by video conference or face-to-face at one of the involved facilities.

report regularly internally and externally on the progress of the activities, and may be required to present your work at conferences or other external meetings.

This job is for an engineer, specialized in software development, massive data processing, scientific computing or equivalent.

The mission also requires the following qualities:

Listening to users and knowing how to identify their needs.

This offer corresponds to a temporary contract position of 18 months. It is to be filled immediately. The workplace is Saint-Aubin (Essonne).