Scalable Earth Observation Data Cubes for Advanced Analytics of Dynamic Earth Surface Processes: An Open-Source Package for Customized Processing of Sentinel-2 Data on HPCs and Beyond

Arisoy, Baturalp; Betz, Florian; Stauch, Georg; Klein, Doris; Dech, Stefan; Ullmann, Tobias

doi:10.5194/egusphere-2026-619

Preprints

https://doi.org/10.5194/egusphere-2026-619

Preprints

10 Feb 2026

| 10 Feb 2026

Scalable Earth Observation Data Cubes for Advanced Analytics of Dynamic Earth Surface Processes: An Open-Source Package for Customized Processing of Sentinel-2 Data on HPCs and Beyond

Baturalp Arisoy, Florian Betz, Georg Stauch, Doris Klein, Stefan Dech, and Tobias Ullmann

Abstract. Earth Observation archives now encompass petabytes of multispectral imagery, yet transforming these heterogeneous collections into analysis-ready data (ARD) cubes remains a critical bottleneck. We present an open-source Python package that unifies cloud masking, co-registration, and super-resolution into a seamless Xarray-based workflow, tailored specifically to close practical gaps in ARD cube generation. Leveraging scalable high-performance computing (HPC) infrastructure, our framework delivers rapid, reproducible cube construction and incremental updates, enabling users to build or extend large time-series data cubes without reprocessing historical scenes. Besides HPCs, our package is also suitable for local processing of Sentinel-2 data. Our approach integrates (1) s2cloudless, a probabilistic cloud-masking algorithm offering user-defined thresholds to overcome the rigid limitations of the Sentinel-2 Scene Classification Layer (SCL) and STAC metadata; (2) AROSICS, a sliding-window co-registration routine that ensures sub-pixel alignment over complex, dynamic landscapes to produce smoother temporal metrics and more consistent change detection; and (3) SEN2SR, a deep-learning super-resolution model that refines all bands to 2.5 m, revealing fine geomorphic and ecological features previously obscured at native resolutions. Together, these components address three recurring ARD cube gaps in existing Xarray-based toolkits: adaptive cloud filtering, robust time-series alignment, and integrated spatial enhancement within a single, reproducible pipeline. To maximize accessibility and reuse, the package is accompanied by well documented, interactive Python notebooks that guide users through configuration, and end-to-end cube generation. Validated on the German Aerospace Center’s terrabyte HPDA clusters, the pipeline runs equally well on local workstation and can be accessed at https://github.com/BaturalpArisoy/stac2cube.

Received: 02 Feb 2026 – Discussion started: 10 Feb 2026

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Download & links

Baturalp Arisoy, Florian Betz, Georg Stauch, Doris Klein, Stefan Dech, and Tobias Ullmann

Status: closed

RC1:
'Comment on egusphere-2026-619', Anonymous Referee #1, 31 Mar 2026
This paper presents an open-source Python package that implements a series of Sentinel-2 processing steps, covering cloud detection, co-registration, and super-resolution. End-to-end pipelines of this kind are useful contributions to the broader scientific community and help democratize access to advanced methodologies.
Unfortunately, I can currently not recommend this for publication due to several major concerns throughout this work:
Major Concerns:
The package is presented as a scalable solution suitable for HPCs. However, the study is limited to a small region that fits comfortably within the memory of a conventional workstation (~15 GB), and no benchmarks are provided showing how the framework actually scales on HPC infrastructure. For example, how does processing time change with area size, number of scenes, or number of cores? Does the processing break at some point? How is load distributed across nodes in an HPC environment? Without this, the HPC contribution cannot be properly evaluated. The authors should either provide a systematic performance analysis or scale back the HPC claims to match what is actually demonstrated.

The paper motivates the work partly on the premise that traditional data cube pipelines fail in dynamic environments, but does not provide an in-depth comparison showing how they fail or how this package specifically addresses those failures. It also does not mention or compare against other relevant Sentinel-2 preprocessing efforts such as xcube, cubo, sentle, or sen2r. The technical comparison to FORCE is limited. A discussion of how this package relates to and differs from these existing tools is necessary to properly contextualize the contribution.

The individual methods implemented are not novel — each component (s2cloudless, AROSICS, SEN2SR) is an existing tool — and the primary contribution lies in their integration into a unified pipeline. This is a legitimate and valued form of contribution, but the paper should frame itself accordingly rather than implying broader methodological novelty. Additionally, the title proposes a "customizable" framework; however, the customization offered operates within each component's parameters rather than enabling modular exchange of components. The authors should clarify and reframe the contribution to accurately reflect what is genuinely offered.

Minor:
Table 1: The methodology used to obtain the "processing time" results needs to be more clearly described. If this measurement was taken within a short timeframe and from a single location, it is of limited value. Response times from STAC servers — even within a single provider such as Planetary Computer — vary with time and request location. Server load also fluctuates, affecting measured processing time. This measurement therefore needs to be either far more comprehensive or removed. A meaningful performance assessment of response time would require globally and temporally distributed queries, and even then the results would need careful interpretation, as server-side throttling mechanisms may temporarily affect repeated requests from the same IP. Furthermore, the term "processing time" is potentially misleading here, as it only encompasses the STAC query and not the actual data access, which is arguably the more operationally relevant metric.

Table 1: It is unclear how the license column is relevant to the comparison being made, and what exactly it applies to. Please clarify or remove this column.

Line 111: STAC APIs do not store data. The servers providing a STAC API process spatio-temporal queries and return metadata items containing pointers to where the actual data — in this case Sentinel-2 tiles — are stored. The storage location can be entirely decoupled from where the STAC server is running. Please revise accordingly.

Line 116: Please review the definition of an "API." The term appears to be used incorrectly in several places throughout the manuscript.

Section 2.2: It is unclear why the detailed climate specifics of the Naryn River catchment are relevant to the HPC processing of Sentinel-2 data. If these details are not directly referenced in the methods or results, consider condensing this section.

Line 169–170: UTM coordinates are expressed in metres (easting/northing), not latitude and longitude. Please correct this.

Line 178: The description of the export step lacks sufficient methodological detail. What does "exported" mean in this context — is this a direct download followed by naive resampling? If so, the reported five minutes and 16 GB figure primarily reflects network speed rather than processing performance. If these figures are part of an evaluation, a methodology section should precede them.

Line 182: The choice of NetCDF and GeoTIFF as output formats should be briefly justified. While these remain widely used in operational workflows, newer formats such as Zarr V3 are increasingly adopted as a standard for spatio-temporal data. A sentence acknowledging this and explaining the format choice would strengthen the paper.

Figure 3 caption: Please expand the caption to walk the reader through the figure. The current description is too brief to be informative without reading the surrounding text.

Figure 3: It would be helpful to clarify whether intermediate data cubes are duplicated and deleted during the processing workflow, and if so, how this is managed. What are the implications for storage consumption, and does this approach scale to larger datasets?

Section 3: Please specify the data type and compression method used for the output cube.

Section 3.2.2: If the co-registration approach relies on a moving window strategy, please clarify how alignment is ensured across scenes spanning multiple years. As currently described, scenes within a given window will be aligned to the master, but it is not clear whether a consistent alignment is maintained across longer time spans. A creeping spatial shift over time seems possible and should be discussed.

Line 281: Please clarify how Dask is used in practice. For the 15 GB example provided, Dask is not strictly necessary, as a dataset of this size could fit into memory multiple times on a standard workstation. A discussion of when and how Dask provides meaningful benefit would be useful.

Figure 4 / Section 4.1: Without ground truth data, the statistical basis for the cloud masking comparison is unclear. Please either provide ground truth validation or more explicitly acknowledge and discuss this limitation.

Figure 6: Without reading the surrounding text, it is not clear what effect the reader is expected to observe, or why a high or low difference value is considered good or bad. Please revise the caption to make the figure interpretable on its own.

Figure 7: The font size in the plots is too small to be legible. Please increase it.
Citation: https://doi.org/10.5194/egusphere-2026-619-RC1
- AC1: 'Reply on RC1', Baturalp Arisoy, 24 Jul 2026
  
  We would like to express our gratitude for helpful comments and suggestions that helped us improving our paper. After carefully reviewing your concerns and comments, we have already made some changes by addressing the comments. Please find all the replies we have made in the attached file.
  
  Citation: https://doi.org/10.5194/egusphere-2026-619-AC1
RC2:
'Comment on egusphere-2026-619', Gilberto Câmara, 25 May 2026

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2026/egusphere-2026-619/egusphere-2026-619-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2026-619-RC2
- AC2: 'Reply on RC2', Baturalp Arisoy, 24 Jul 2026
  
  We would like to express our gratitude for helpful comments and suggestions that helped us improving our paper. After carefully reviewing your concerns and comments, we have already made some changes by addressing the comments. Please find all the replies we have made in the attached file.
  
  Citation: https://doi.org/10.5194/egusphere-2026-619-AC2

Status: closed

RC1:
'Comment on egusphere-2026-619', Anonymous Referee #1, 31 Mar 2026
This paper presents an open-source Python package that implements a series of Sentinel-2 processing steps, covering cloud detection, co-registration, and super-resolution. End-to-end pipelines of this kind are useful contributions to the broader scientific community and help democratize access to advanced methodologies.
Unfortunately, I can currently not recommend this for publication due to several major concerns throughout this work:
Major Concerns:
The package is presented as a scalable solution suitable for HPCs. However, the study is limited to a small region that fits comfortably within the memory of a conventional workstation (~15 GB), and no benchmarks are provided showing how the framework actually scales on HPC infrastructure. For example, how does processing time change with area size, number of scenes, or number of cores? Does the processing break at some point? How is load distributed across nodes in an HPC environment? Without this, the HPC contribution cannot be properly evaluated. The authors should either provide a systematic performance analysis or scale back the HPC claims to match what is actually demonstrated.

The paper motivates the work partly on the premise that traditional data cube pipelines fail in dynamic environments, but does not provide an in-depth comparison showing how they fail or how this package specifically addresses those failures. It also does not mention or compare against other relevant Sentinel-2 preprocessing efforts such as xcube, cubo, sentle, or sen2r. The technical comparison to FORCE is limited. A discussion of how this package relates to and differs from these existing tools is necessary to properly contextualize the contribution.

The individual methods implemented are not novel — each component (s2cloudless, AROSICS, SEN2SR) is an existing tool — and the primary contribution lies in their integration into a unified pipeline. This is a legitimate and valued form of contribution, but the paper should frame itself accordingly rather than implying broader methodological novelty. Additionally, the title proposes a "customizable" framework; however, the customization offered operates within each component's parameters rather than enabling modular exchange of components. The authors should clarify and reframe the contribution to accurately reflect what is genuinely offered.

Minor:
Table 1: The methodology used to obtain the "processing time" results needs to be more clearly described. If this measurement was taken within a short timeframe and from a single location, it is of limited value. Response times from STAC servers — even within a single provider such as Planetary Computer — vary with time and request location. Server load also fluctuates, affecting measured processing time. This measurement therefore needs to be either far more comprehensive or removed. A meaningful performance assessment of response time would require globally and temporally distributed queries, and even then the results would need careful interpretation, as server-side throttling mechanisms may temporarily affect repeated requests from the same IP. Furthermore, the term "processing time" is potentially misleading here, as it only encompasses the STAC query and not the actual data access, which is arguably the more operationally relevant metric.

Table 1: It is unclear how the license column is relevant to the comparison being made, and what exactly it applies to. Please clarify or remove this column.

Line 111: STAC APIs do not store data. The servers providing a STAC API process spatio-temporal queries and return metadata items containing pointers to where the actual data — in this case Sentinel-2 tiles — are stored. The storage location can be entirely decoupled from where the STAC server is running. Please revise accordingly.

Line 116: Please review the definition of an "API." The term appears to be used incorrectly in several places throughout the manuscript.

Section 2.2: It is unclear why the detailed climate specifics of the Naryn River catchment are relevant to the HPC processing of Sentinel-2 data. If these details are not directly referenced in the methods or results, consider condensing this section.

Line 169–170: UTM coordinates are expressed in metres (easting/northing), not latitude and longitude. Please correct this.

Line 178: The description of the export step lacks sufficient methodological detail. What does "exported" mean in this context — is this a direct download followed by naive resampling? If so, the reported five minutes and 16 GB figure primarily reflects network speed rather than processing performance. If these figures are part of an evaluation, a methodology section should precede them.

Line 182: The choice of NetCDF and GeoTIFF as output formats should be briefly justified. While these remain widely used in operational workflows, newer formats such as Zarr V3 are increasingly adopted as a standard for spatio-temporal data. A sentence acknowledging this and explaining the format choice would strengthen the paper.

Figure 3 caption: Please expand the caption to walk the reader through the figure. The current description is too brief to be informative without reading the surrounding text.

Figure 3: It would be helpful to clarify whether intermediate data cubes are duplicated and deleted during the processing workflow, and if so, how this is managed. What are the implications for storage consumption, and does this approach scale to larger datasets?

Section 3: Please specify the data type and compression method used for the output cube.

Section 3.2.2: If the co-registration approach relies on a moving window strategy, please clarify how alignment is ensured across scenes spanning multiple years. As currently described, scenes within a given window will be aligned to the master, but it is not clear whether a consistent alignment is maintained across longer time spans. A creeping spatial shift over time seems possible and should be discussed.

Line 281: Please clarify how Dask is used in practice. For the 15 GB example provided, Dask is not strictly necessary, as a dataset of this size could fit into memory multiple times on a standard workstation. A discussion of when and how Dask provides meaningful benefit would be useful.

Figure 4 / Section 4.1: Without ground truth data, the statistical basis for the cloud masking comparison is unclear. Please either provide ground truth validation or more explicitly acknowledge and discuss this limitation.

Figure 6: Without reading the surrounding text, it is not clear what effect the reader is expected to observe, or why a high or low difference value is considered good or bad. Please revise the caption to make the figure interpretable on its own.

Figure 7: The font size in the plots is too small to be legible. Please increase it.
Citation: https://doi.org/10.5194/egusphere-2026-619-RC1
- AC1: 'Reply on RC1', Baturalp Arisoy, 24 Jul 2026
  
  We would like to express our gratitude for helpful comments and suggestions that helped us improving our paper. After carefully reviewing your concerns and comments, we have already made some changes by addressing the comments. Please find all the replies we have made in the attached file.
  
  Citation: https://doi.org/10.5194/egusphere-2026-619-AC1
RC2:
'Comment on egusphere-2026-619', Gilberto Câmara, 25 May 2026

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2026/egusphere-2026-619/egusphere-2026-619-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2026-619-RC2
- AC2: 'Reply on RC2', Baturalp Arisoy, 24 Jul 2026
  
  We would like to express our gratitude for helpful comments and suggestions that helped us improving our paper. After carefully reviewing your concerns and comments, we have already made some changes by addressing the comments. Please find all the replies we have made in the attached file.
  
  Citation: https://doi.org/10.5194/egusphere-2026-619-AC2

Baturalp Arisoy, Florian Betz, Georg Stauch, Doris Klein, Stefan Dech, and Tobias Ullmann

Viewed

Total article views: 2,242 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
1,332	803	107	2,242	235	284

HTML: 1,332
PDF: 803
XML: 107
Total: 2,242
BibTeX: 235
EndNote: 284

Views and downloads (calculated since 10 Feb 2026)

Month	HTML	PDF	XML	Total
Feb 2026	620	214	45	879
Mar 2026	303	276	16	595
Apr 2026	96	96	9	201
May 2026	67	71	3	141
Jun 2026	18	6	2	26
Jul 2026	50	25	8	83
Aug 2026	30	22	2	54
Sep 2026	69	42	13	124
Oct 2026	79	51	9	139

Cumulative views and downloads (calculated since 10 Feb 2026)

Month	HTML	PDF	XML	Total
Feb 2026	620	214	45	879
Mar 2026	303	276	16	595
Apr 2026	96	96	9	201
May 2026	67	71	3	141
Jun 2026	18	6	2	26
Jul 2026	50	25	8	83
Aug 2026	30	22	2	54
Sep 2026	69	42	13	124
Oct 2026	79	51	9	139

Viewed (geographical distribution)

Total article views: 2,227 (including HTML, PDF, and XML) Thereof 2,227 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 01 Nov 2026

Short summary

Earth Observation archives now encompass multispectral imagery, yet producing analysis-ready Sentinel-2 time series remains a critical bottleneck. We present an open-source Python package that builds data cubes from Spatio-Temporal Asset Catalog catalogues and integrates cloud masking, co-registration, and super-resolution in one workflow. Validated on high-performance computing, it enables rapid, reproducible cube construction and updates, improving temporal consistency in a braided river.


Total:	0
HTML:	0
PDF:	0
XML:	0