Preprints
https://doi.org/10.5194/egusphere-2022-1094
https://doi.org/10.5194/egusphere-2022-1094
 
20 Dec 2022
20 Dec 2022
Status: this preprint is open for discussion.

A Comparison of Lossless Compression Algorithms for Altimeter Data

Mathieu Thevenin1, Stephane Pigoury2, Olivier Thomine3, and Flavien Gouillon4 Mathieu Thevenin et al.
  • 1CEA SPEC, Bat 772 F91191 Gif-sur-Yvette - France
  • 2Subnet SAS, 21 av. de la belle image, 94440 Marolles-en-Brie - France
  • 3Insa Rouen Normandie, 685 Av. de l’Université, 76800 Saint-Étienne-du-Rouvray - France
  • 4CNES, Centre Spatial de Toulouse, 18 avenue Edouard Belin, 31401 Toulouse, France

Abstract. Satellite data transmission is usually limited between hundreds of kilobits-per-second (kb/s) and several megabits-per-second (Mb/s) while the space-to-ground data volume is becoming larger as the resolution of the instruments increases while the bandwidth remains limited, typically. The Surface Water and Ocean Topography (SWOT) altimetry mission is a partnership between the National Aeronautics and Space Administration (NASA) and the Centre National des Études Spatiales (CNES) which uses the innovative KaRin instrument, a Ka band (35.75 GHz) synthetic aperture radar combined with an interforemeter. Its launch is expected for 2022 for oceanographic and hydrological levels measurement and it will generate 7 TeraBytes-per-day, for a lifetime total of 20 PetaBytes. That is why data compression needs to be implemented at both ends of satellite communications. This study compares the compression results obtained with 672 algorithms, mostly based on the Huff- man coding approach which constitute the state-of-the-art for scientific data manipulation, including Computational Fluid Dynamics (CFD). We also have incorporated data preprocessing such as shuffle and bitshuffle, and a novel algorithm named SL6.

Mathieu Thevenin et al.

Status: open (until 16 Feb 2023)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CEC1: 'Comment on egusphere-2022-1094', Juan Antonio Añel, 13 Jan 2023 reply
    • AC1: 'Reply on CEC1', Mathieu Thevenin, 26 Jan 2023 reply

Mathieu Thevenin et al.

Mathieu Thevenin et al.

Viewed

Total article views: 174 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
114 52 8 174 3 2
  • HTML: 114
  • PDF: 52
  • XML: 8
  • Total: 174
  • BibTeX: 3
  • EndNote: 2
Views and downloads (calculated since 20 Dec 2022)
Cumulative views and downloads (calculated since 20 Dec 2022)

Viewed (geographical distribution)

Total article views: 172 (including HTML, PDF, and XML) Thereof 172 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 03 Feb 2023
Download
Short summary
As an extension of the work presented in "Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files" (Delaunay) https://gmd.copernicus.org/articles/12/4099/2019/, this paper present a detailed bench of lossless, mostly LZ-based, compression algorithms that could be used for space-to-earth communication or data storage. The work is conducted on the SWOT altimetry data.