Preprints
https://doi.org/10.5194/egusphere-2022-1094
https://doi.org/10.5194/egusphere-2022-1094
20 Dec 2022
 | 20 Dec 2022

A Comparison of Lossless Compression Algorithms for Altimeter Data

Mathieu Thevenin, Stephane Pigoury, Olivier Thomine, and Flavien Gouillon

Abstract. Satellite data transmission is usually limited between hundreds of kilobits-per-second (kb/s) and several megabits-per-second (Mb/s) while the space-to-ground data volume is becoming larger as the resolution of the instruments increases while the bandwidth remains limited, typically. The Surface Water and Ocean Topography (SWOT) altimetry mission is a partnership between the National Aeronautics and Space Administration (NASA) and the Centre National des Études Spatiales (CNES) which uses the innovative KaRin instrument, a Ka band (35.75 GHz) synthetic aperture radar combined with an interforemeter. Its launch is expected for 2022 for oceanographic and hydrological levels measurement and it will generate 7 TeraBytes-per-day, for a lifetime total of 20 PetaBytes. That is why data compression needs to be implemented at both ends of satellite communications. This study compares the compression results obtained with 672 algorithms, mostly based on the Huff- man coding approach which constitute the state-of-the-art for scientific data manipulation, including Computational Fluid Dynamics (CFD). We also have incorporated data preprocessing such as shuffle and bitshuffle, and a novel algorithm named SL6.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Mathieu Thevenin, Stephane Pigoury, Olivier Thomine, and Flavien Gouillon

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CEC1: 'Comment on egusphere-2022-1094', Juan Antonio Añel, 13 Jan 2023
    • AC1: 'Reply on CEC1', Mathieu Thevenin, 26 Jan 2023
  • RC1: 'Comment on egusphere-2022-1094', Anonymous Referee #1, 10 Mar 2023
    • CC1: 'Reply on RC1', Stephane Pigoury, 02 Apr 2023
  • RC2: 'Comment on egusphere-2022-1094', Anonymous Referee #2, 15 Mar 2023
    • CC2: 'Reply on RC2', Stephane Pigoury, 02 Apr 2023
    • AC3: 'Reply on RC2', Mathieu Thevenin, 11 Apr 2023
  • RC3: 'Comment on egusphere-2022-1094', H. Xu, 05 Apr 2023
    • AC2: 'Reply on RC3', Mathieu Thevenin, 11 Apr 2023

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CEC1: 'Comment on egusphere-2022-1094', Juan Antonio Añel, 13 Jan 2023
    • AC1: 'Reply on CEC1', Mathieu Thevenin, 26 Jan 2023
  • RC1: 'Comment on egusphere-2022-1094', Anonymous Referee #1, 10 Mar 2023
    • CC1: 'Reply on RC1', Stephane Pigoury, 02 Apr 2023
  • RC2: 'Comment on egusphere-2022-1094', Anonymous Referee #2, 15 Mar 2023
    • CC2: 'Reply on RC2', Stephane Pigoury, 02 Apr 2023
    • AC3: 'Reply on RC2', Mathieu Thevenin, 11 Apr 2023
  • RC3: 'Comment on egusphere-2022-1094', H. Xu, 05 Apr 2023
    • AC2: 'Reply on RC3', Mathieu Thevenin, 11 Apr 2023
Mathieu Thevenin, Stephane Pigoury, Olivier Thomine, and Flavien Gouillon
Mathieu Thevenin, Stephane Pigoury, Olivier Thomine, and Flavien Gouillon

Viewed

Total article views: 885 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
434 416 35 885 17 13
  • HTML: 434
  • PDF: 416
  • XML: 35
  • Total: 885
  • BibTeX: 17
  • EndNote: 13
Views and downloads (calculated since 20 Dec 2022)
Cumulative views and downloads (calculated since 20 Dec 2022)

Viewed (geographical distribution)

Total article views: 874 (including HTML, PDF, and XML) Thereof 874 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 21 May 2024
Download
Short summary
As an extension of the work presented in "Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files" (Delaunay) https://gmd.copernicus.org/articles/12/4099/2019/, this paper present a detailed bench of lossless, mostly LZ-based, compression algorithms that could be used for space-to-earth communication or data storage. The work is conducted on the SWOT altimetry data.