New insights on the suitability of NetCDF/HDF5 as storage format for climate cloud repositories

Cimadevilla, Ezequiel; Hassell, David; Lawrence, Bryan N.

doi:10.5194/egusphere-2026-3249

Preprints

https://doi.org/10.5194/egusphere-2026-3249

Preprints

16 Jun 2026

| 16 Jun 2026

New insights on the suitability of NetCDF/HDF5 as storage format for climate cloud repositories

Ezequiel Cimadevilla, David Hassell, and Bryan N. Lawrence

Abstract. Climate data analysis increasingly relies on cloud infrastructures to offer new and efficient methods of accessing the necessary climate data. Cloud repositories allow access to such data in a remote data access basis, which allows users to retrieve and manipulate their data without requiring file downloads, reducing storage costs resulting in more efficient systems. Together, climate cloud repositories and remote data access are evolving fast, due to the necessity of collaboration that brings together diverse communities to address challenges in climate science. In recent years, a prevailing discourse has emerged suggesting that traditional climate data storage formats are inherently unsuitable for remote data access. In this work, we present new insights that challenge this discourse and demonstrate that established storage formats such as NetCDF/HDF5 can continue to operate efficiently in cloud environments when accessed remotely. These findings contrast with the widespread perception that such formats are inherently unsuitable for cloud based workflows. In the context of the onset of CMIP7, these insights have the potential to substantially enhance climate data access and analysis for the broader research community without incurring major maintenance burdens.

Received: 04 Jun 2026 – Discussion started: 16 Jun 2026

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Download & links

Ezequiel Cimadevilla, David Hassell, and Bryan N. Lawrence

Status: final response (author comments only)

RC1:
'Comment on egusphere-2026-3249', Amirhossein Nikfal, 16 Jul 2026

The code repository contains the precomputed CSV files used to construct the B-tree-offset distributions in Figures 2 and 3, but I could not identify the code or commands used to generate these CSV files from the original and repacked NetCDF files. Please provide the complete extraction script and document the meaning of the level, entry, and offset columns, including whether offset represents a B-tree node address or an individual entry address. The required Pyfive version or commit should also be specified.
Fig-1: Why are the B-tree nodes scattered throughout the linear address space of the NetCDF/HDF5 file? Could the authors explain why their locations cannot be organized sequentially or made more predictable during file creation?
Table 1: Please provide direct download links or persistent ESGF catalogue links for the three NetCDF files used in Section 3.

Citation: https://doi.org/10.5194/egusphere-2026-3249-RC1
- AC1:
  'Reply on RC1', Ezequiel Cimadevilla, 17 Jul 2026
  
  We have made available the script to generate the CSV files in the repository (tree.py, https://github.com/zequihg50/new-insights-netcdf/). Release v1.1.2 of Pyfive is required to test parallel access to data (https://github.com/NCAS-CMS/pyfive/releases/tag/v1.1.2). Per the HDF5 specification [1], the fields are defined as follows:
  level - The node level indicates the level at which this node appears in the tree (leaf nodes are at level zero). Not only does the level indicate whether child pointers point to sub-trees or to data, but it can also be used to help file consistency checking utilities reconstruct damaged trees.
  
  entry - Each entry (key and address of a child) among the entries used in the B-tree.
  
  offset - Byte offset of each entry in the B-tree.
  Fig-1: The purpose of B-trees is to grow over time as chunks are written. Because clients typically write one chunk at a time, space for future chunks is reserved in the index after the first chunks have been written. During repacking we increase the metadata block size and HDF5 attempts to place B-trees at the beginning of the file. We will clarify this in the reviewed version. For more information, please see https://github.com/Unidata/netcdf-c/issues/3386.
  Table 1: We provide links to both the original and repacked files. Please note that repacking is not a deterministic operation, as its output may vary depending on the version and behavior of h5repack.
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_Amon_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_185001-234912.nc
  
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_Amon_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_185001-234912.nc_cmip7repack4mb
  
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_day_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_18500101-23491231.nc
  
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_day_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_18500101-23491231.nc_cmip7repack4mb
  
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_3hr_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_187001010300-197001010000.nc
  
  - https://thredds.climate.ifca.es/thredds/fileServer/tests/uas_3hr_IPSL-CM6A-LR_piControl_r1i1p1f1_gr_187001010300-197001010000.nc_cmip7repack4mb
  [1] - https://support.hdfgroup.org/documentation/hdf5/latest/_f_m_t4.html#subsubsec_fmt4_infra_btrees_v1
  
  Citation: https://doi.org/10.5194/egusphere-2026-3249-AC1
  - RC2: 'Reply on AC1', Amirhossein Nikfal, 06 Aug 2026
    
    Code tree.py:
    The fs dictionary hardcodes a B-tree root offset and key size for each variable per file (e.g. "uas": [28671, 8+8*(3+1)]), but the code used to derive these values isn't included. Could you share the script/method used to locate them? This would let reviewers reproduce fs independently rather than trusting the hardcoded numbers.
    
    Citation: https://doi.org/10.5194/egusphere-2026-3249-RC2
    
    AC2: 'Reply on RC2', Ezequiel Cimadevilla, 06 Aug 2026
    
    We obtained the B-tree root node offsets by inspecting and debugging pyfive, while opening each netCDF file. Specifically, we identified the offset used to initialize chunk B-trees, which is available in the pyfive source code here [1]. Note that pyfive does not provide a public API for accessing this information, nor are there any plans to add one in the short term, as this is considered a private implementation detail of the library. The tree.py script included in the GitHub repository verifies that the TREE signature is present at the specified location, in accordance with the HDF5 specification.
    [1] - https://github.com/NCAS-CMS/pyfive/blob/667c4681685ac32d5528c93a4058c733914dadb4/pyfive/btree.py#L133
    
    Citation: https://doi.org/10.5194/egusphere-2026-3249-AC2
    
    RC4: 'Reply on AC2', Amirhossein Nikfal, 18 Aug 2026
    
    1. Distinguish the effects of rechunking and B-tree/metadata reorganization
    The repacking procedure substantially changes both the chunking and the HDF5 chunk-index structure. For the examined uas dataset, the original file contains roughly one chunk per timestep (~182,000 chunks), whereas repacking to ~4 MB chunks reduces this to only a few thousand chunks. Correspondingly, inspection with tree.py shows that the B-tree changes from three levels (2–1–0) to only two levels (1–0).
    Thus, part of the improvement comes from having far fewer chunks and consequently a smaller and shallower B-tree, while another part comes from placing the remaining B-tree metadata much more contiguously near the beginning of the file. I suggest distinguishing these two effects more clearly rather than attributing the improvement mainly to B-tree reordering/locality.
    2. Clarify B-tree entries versus B-tree nodes
    The output of tree.py appears to report byte offsets for individual entries within B-tree nodes, whereas Figures 2 and 3 refer to “Byte offsets of B-tree nodes.” Multiple entries can belong to the same node.
    I therefore suggest clarifying this terminology. In particular, the number of plotted entries should not be interpreted directly as the number of B-tree nodes or individual remote requests. Nevertheless, the results clearly demonstrate that the repacked file has a much smaller and more spatially concentrated chunk-index structure.
    3. Clarify the role of Pyfive versus repacking itself
    The manuscript demonstrates strong remote-access performance using Pyfive on the repacked files. However, the optimized file layout and the specialized access method are two separate aspects. Repacking improves the underlying HDF5 structure, while Pyfive provides asynchronous/concurrent access designed to exploit that structure efficiently.
    I suggest clarifying which performance improvements can be obtained simply by using the repacked file with conventional NetCDF/HDF5 tools, and which depend on Pyfive or an equivalent backend. This is particularly relevant for users working through higher-level libraries such as xarray.
    4. Clarify the relationship between Kerchunk and the optimized native HDF5 index
    Kerchunk and the proposed repacking approach address remote access differently. Kerchunk provides an external reference describing the locations of chunks, whereas the repacking approach improves the organization of the native HDF5 chunk index itself.
    Since Kerchunk is an important comparison in the manuscript, I suggest explaining this distinction more explicitly. This would make clear that the reorganized B-tree is not analogous to a Kerchunk auxiliary file, but remains part of the internal HDF5 structure.
    5. Qualify the interpretation of the performance comparison
    The performance section shows that appropriately repacked NetCDF/HDF5 files, when accessed using Pyfive, can achieve very competitive remote-reading performance relative to Kerchunk and Zarr. However, this result reflects the combination of chunk configuration, B-tree/metadata organization, and client access strategy.
    I therefore suggest avoiding conclusions that imply that file format alone determines the observed performance. A more precise conclusion would be that properly chunked and organized NetCDF/HDF5, together with an appropriate remote-access strategy, can achieve performance comparable to alternative cloud-oriented approaches under the tested conditions.
    
    Citation: https://doi.org/10.5194/egusphere-2026-3249-RC4
    
    AC3: 'Reply on RC4', Ezequiel Cimadevilla, 07 Sep 2026
    
    1. Distinguish the effects of rechunking and B-tree/metadata reorganization
    We agree with the reviewer that the effects of B-tree reorganization and chunk size should be more clearly distinguished, as they contribute to performance improvements in different ways. The location and organization of the B-tree metadata primarily affect the latency of retrieving the chunk index, which must be accessed before the corresponding data can be read. In contrast, larger chunks improve overall data-transfer throughput by reducing the number of remote requests required to retrieve chunks. We will clarify this distinction in the revised manuscript.
    2. Clarify B-tree entries versus B-tree nodes
    We acknowledge that there is an inconsistency in the terminology used. The stacked histograms report the byte offsets of individual B-tree entries, while multiple entries can belong to the same B-tree node. We will correct this terminology throughout the revised manuscript to make this distinction clear. We thank the reviewer for pointing this out.
    3. Clarify the role of Pyfive versus repacking itself
    We have attempted to distinguish in the manuscript between the performance benefits derived from the file layout and those provided by library-level optimizations. However, we agree that this distinction could be more clear, particularly in light of the points raised in Comment 1. Conventional netCDF/HDF5 tools can potentially benefit from the improved file layout, but their ability to exploit the layout efficiently depends on the access mechanisms and software stack used. In particular, achieving effective concurrent access through combinations such as h5py, h5netcdf, and fsspec may require careful configuration, and implementation limitations or unintended interactions between these components may prevent the expected parallelism from being achieved in practice. We will add a discussion of this aspect to the revised manuscript.
    4. Clarify the relationship between Kerchunk and the optimized native HDF5 index
    We agree with the reviewer that the distinction between Kerchunk and the proposed repacking approach should be made more explicit. Kerchunk can be viewed as an external representation of the chunk index, derived from the native HDF5 chunk index, and is particularly useful when modifying or repacking the original file is not possible or desirable. In contrast, our approach reorganizes the native HDF5 chunk index itself as part of the repacking process. The resulting B-tree therefore remains an integral part of the HDF5 file structure and does not require an additional external reference file. We will make this distinction more explicit in the revised manuscript.
    5. Qualify the interpretation of the performance comparison
    We will make this point more explicit in the revised manuscript and qualify our conclusions accordingly. In particular, we will clarify that our results demonstrate that properly chunked and organized NetCDF/HDF5 files, when combined with an appropriate remote-access strategy, can achieve performance comparable to alternative cloud-oriented approaches under the conditions tested, rather than implying that the file format alone determines performance.
    
    Citation: https://doi.org/10.5194/egusphere-2026-3249-AC3
RC3:
'Comment on egusphere-2026-3249', Anonymous Referee #2, 11 Aug 2026

Review of “New insights on the suitability of NetCDF/HDF5 as storage format for climate cloud repositories” by Ezequiel Cimadevilla , David Hassell, and Bryan N. Lawrence
Summary

In their study the authors investigate the suitability of HDF5-based netCDF-files to efficiently operate in cloud-based environments. They analysed the data structures of selected CMIP netCDF-files and identified metadata scattered throughout the file. They propose that reorganizing the metadata and storing it all together at the beginning of linear storage of the netCDF file together with a client that can exploit these file features can significantly reduce the time reading variables from netCDF-files via WAN networks.

General comments

The general idea that restructuring netCDF files and using advanced clients would make them also effectively useable via cloud services is very appealing. The authors introduced a concept and explained it very well, but used only 3 example files and a 2D variable for demonstration. As this method has the potential to use one file format, namely netCDF, efficiently on local systems, HPC-platforms, as well as cloud environments and thus helps to reduce disk space consumption by reducing metadata and data duplication (same data in various file formats), as well reducing inode consumption (by avoiding extra metadata files for each file), it deserves to be put on a broader data basis. I highly recommend increasing the set of test files and variables. This would also help to derive robust statistics and quantify the potential time reductions.
Specific Comments

Introduction: Using kerchunk and zarr extensively on large data archives significantly increases the number of files and hence, used inodes. This can pose challenges with respect to inode availability or inode/capacity trade-offs in HPC environments. Is this also an issue in cloud environments? As the suggested approach of repacking netCDF files would not consume additional inodes, which I consider a strong advantage, I'd suggest to also address inodes.
l72: Please name here explicitly cloud-native storage formats you had in mind when writing.
Table1: What about rounding up to 26 MB? Or at least using a decimal point instead of a comma.
l84/85: How did you extract the B-tree fragments from the netCDF file?
Figure 2: The tick labels and axis labels are too small. Please increase to make them readable at 100 % zoom level. I cannot see the colours for lat and lon in the figures.
Comparing the metadata size of Table 1 with the kerchunk file size in table 2, can one generally say that kerchunk increases the metadata size by a factor of approximately 2 plus generating these files the inode usage is doubled?
l133: Please explain what uas is (u-component of the near-surface wind). Why did you select a 2D variable for demonstration and not a 3D variable, e.g. ua, which is the 3D wind component? Usually the 3D variables account for the largest fraction of atmospheric netCDF files.
Table3: What is meant by “Relation of the repacked NetCDF files…”? I rather understand this table as same as Table 1, but for the repacked files. A relation would be, e.g. in the repacked file the number of chunks is only ~2 % and metadata size is only 3.6, 0.8, 0.8 % of the original netCDF files. Please clarify.
l125: Does cmip7repack work only for CMIP data sets, or can it be applied to all kinds of netCDF files? Are there other tools available that can do repacking? Would it be possible to specify another variable than time to rechunk the data? E.g. thinking about reanalysis data, where each file contains one time step, but many height levels.
l138-141: Could one say that a reduction of the kerchunk file size of more than 98 % is possible? A larger data basis would be helpful to prove this advantage.
Figure 3: Same as Figure 2, the labels are too small to read. Also, please add a sentence to the figure caption that the x-axis is only a fraction of 2 to highlight the changes. Moreover, lat, lon, time are hardly visible. I don't know if they are present in the middle and right panel. In the left panel maybe a logarithmic y-scale could make them better visible.
l153-155: Please provide examples for clients that are capable of this.
Figures 4 and 5: From how many samples are the values derived? Are they averages? Why are there some tests with relatively large error bars compared to the others? Please provide an explanation. Please consider rearranging the order, to have the results sorted by tool and file format, e.g. kerchunk first, followed by pyfive, and zarr, to facilitate the comparison. Could the error bars also depend on network speed and cloud setup? Would other cloud setups connections show less variability?
l167-170: Is it just impractical to open the original files or is it even impossible?
Figure 5: I think the clear advantage shown here is that repacking speeds up the reading process significantly. Since the scale is log-scale it would be good to provide some numbers in the text how much faster the variables from the repacked files can be read by pyfive and kerchunk.
Figure 6: Isn't Figure 6 a subset from Figure 5? Here the values for “Amon”' are hardly visible. Maybe a table would be the better choice.
l190: Please consider providing an example list of clients that can and cannot fetch chunks concurrently.
l194-196 & l220-221 & l256-257: The same conclusion is repeated here. Rather have the conclusion just in the conclusions section.
Typos

General: Please consider writing “netCDF”' instead of “NetCDF” in running text. This is what the developers do: https://www.unidata.ucar.edu/software/netcdf. Capital “N” in netCDF is only used at the beginning of a sentence or in headlines.
l41: “a NetCDF files” -> “a netCDF file”
Figures 5 & 6 caption: minimun -> minimum
l163 & 252: Please consider replacing LLM-hyphens.

Citation: https://doi.org/10.5194/egusphere-2026-3249-RC3
- AC4: 'Reply on RC3', Ezequiel Cimadevilla, 07 Sep 2026
  
  We thank the reviewer for the careful reading of our manuscript and for the positive and constructive summary of our work.
  General comments
  In the study, we effectively use only three example files to demonstrate our approach. It should be noted, however, that these files were selected to represent different temporal resolutions. Given the chunking scheme commonly used for CMIP data, where each chunk corresponds to one time step and contains all spatial locations, the temporal resolution represents the main factor affecting the size and distribution of the B-trees.
  Regarding B-tree behaviour, 2D, 3D, and nD variables, as well as different climate variables such as temperature and precipitation, are expected to behave in the same way. The B-trees are affected primarily by the number of chunks that they need to index, rather than by the dimensionality or physical meaning of the variables. Consequently, the number of chunks, and therefore the B-tree structure, varies in our test cases mainly as a function of the temporal resolution, which we have already considered.
  For this reason, we believe that adding further variables or netCDF files with the same chunking characteristics would largely replicate the results already presented, without providing substantially different insights. Nevertheless, we appreciate the reviewer’s suggestion and have clarified this point in the manuscript to better explain the rationale behind the selection of the test files and variables.
  Specific Comments
  Introduction: We agree that we have not paid sufficient attention to the issue of inode consumption associated with some data formats, particularly Zarr, in the manuscript. We will address this aspect in the revised version. High inode consumption is a well-known concern when using Zarr in HPC environments, particularly when data is stored as a large number of individual objects or files. This is one of the motivations behind sharded Zarr storage, which reduces the number of individual files. While we do not have empirical evidence to quantify the extent to which this issue affects cloud object storage, where the concept of inodes does not generally apply in the same way as in traditional file systems, the number of objects can nevertheless have implications for storage management, metadata operations, and access performance. We appreciate the reviewer’s suggestion and will discuss this aspect in the revised manuscript, including the advantages of HDF5 in avoiding the creation of additional files and, consequently, additional inode consumption in file-system-based environments.
  l72: Please name here explicitly cloud-native storage formats you had in mind when writing.
  We were referring specifically to Zarr and to other approaches that provide cloud-optimized access to netCDF data, such as Kerchunk. We will make this explicit in the revised manuscript.
  Table1: What about rounding up to 26 MB? Or at least using a decimal point instead of a comma.
  We will change it to 26 MB.
  l84/85: How did you extract the B-tree fragments from the netCDF file?
  We added the script “tree.py” to the GitHub repository, which extracts the locations of B-tree nodes and entries given the location of the root B-tree node. We obtained B-tree root node locations by debugging Pyfive while opening the selected files. The code of interest is located in https://github.com/NCAS-CMS/pyfive/blob/667c4681685ac32d5528c93a4058c733914dadb4/pyfive/btree.py#L133.
  Figure 2: The tick labels and axis labels are too small. Please increase to make them readable at 100 % zoom level. I cannot see the colours for lat and lon in the figures.
  We will change this in the reviewed version. Note that for lat and lon there is only one B-tree entry, which makes this effectively not visible in the plot. We will clarify this in textual format.
  Comparing the metadata size of Table 1 with the kerchunk file size in table 2, can one generally say that kerchunk increases the metadata size by a factor of approximately 2 plus generating these files the inode usage is doubled?
  The size of a Kerchunk file is primarily determined by the number of chunks that need to be indexed. Therefore, there is no generally applicable factor by which a Kerchunk version 1 file increases the size, as its size depends on the structure and chunking of the underlying dataset. We will clarify this in the manuscript.
  l133: Please explain what uas is (u-component of the near-surface wind). Why did you select a 2D variable for demonstration and not a 3D variable, e.g. ua, which is the 3D wind component? Usually the 3D variables account for the largest fraction of atmospheric netCDF files.
  We will clarify the meaning of “uas” (u-component of the near-surface wind) in the revised manuscript. We selected a 2D variable for the demonstration primarily for simplicity and clarity. As discussed above, we expect 2D, 3D, and nD variables, as well as different climate variables such as temperature and precipitation, to exhibit the same B-tree behaviour under a given chunking scheme. The B-tree structure is determined primarily by the number of chunks that need to be indexed, rather than by the dimensionality or physical meaning of the variable. We have also accounted for differences in file size by considering files with different temporal resolutions.
  Table3: What is meant by “Relation of the repacked NetCDF files…”? I rather understand this table as same as Table 1, but for the repacked files. A relation would be, e.g. in the repacked file the number of chunks is only ~2 % and metadata size is only 3.6, 0.8, 0.8 % of the original netCDF files. Please clarify.
  We will rewrite this as “Table of the repacked netCDF files...”.
  l125: Does cmip7repack work only for CMIP data sets, or can it be applied to all kinds of netCDF files? Are there other tools available that can do repacking? Would it be possible to specify another variable than time to rechunk the data? E.g. thinking about reanalysis data, where each file contains one time step, but many height levels.
  The current implementation of cmip7repack uses CMIP metadata conventions to identify the appropriate chunking scheme for the different variables in a netCDF file. Therefore, as currently implemented, cmip7repack is intended for CMIP datasets. However, the underlying approach is not specific to CMIP data and the tool could be readily adapted to other datasets and chunking requirements, since it consists only on a wrapper around h5repack.
  l138-141: Could one say that a reduction of the kerchunk file size of more than 98 % is possible? A larger data basis would be helpful to prove this advantage.
  As explained above, the size of a Kerchunk file is primarily determined by the number of chunks that need to be indexed. Its size depends mostly on the chunking of the underlying dataset.
  Figure 3: Same as Figure 2, the labels are too small to read. Also, please add a sentence to the figure caption that the x-axis is only a fraction of 2 to highlight the changes. Moreover, lat, lon, time are hardly visible. I don't know if they are present in the middle and right panel. In the left panel maybe a logarithmic y-scale could make them better visible.
  We will make the labels more visible in the manuscript. For coordinate variables, we acknowledge that they are not visible, due to being only one chunk for these variables. We will clarify this within the text of the figure.
  l153-155: Please provide examples for clients that are capable of this.
  To the best of our knowledge, Pyfive is the only client satisfying this property, although different configurations of other clients may achieve the same. We will clarify this in the manuscript.
  Figures 4 and 5: From how many samples are the values derived? Are they averages? Why are there some tests with relatively large error bars compared to the others? Please provide an explanation. Please consider rearranging the order, to have the results sorted by tool and file format, e.g. kerchunk first, followed by pyfive, and zarr, to facilitate the comparison. Could the error bars also depend on network speed and cloud setup? Would other cloud setups connections show less variability?
  We obtained a handful of measurements for each configuration, and all individual results are provided in the GitHub repository. The values shown in Figures 4 and 5 represent the mean for each configuration, while the error bars indicate the minimum and maximum values observed across the samples. We agree that the current ordering of the columns makes the comparison more difficult. We will reorder the results by tool and file format to facilitate comparison, with Kerchunk followed by Pyfive and Zarr.
  The relatively large error bars are primarily observed for the dataset-opening times. We consider this variability to be a natural consequence of accessing resources over a WAN, where network conditions and other factors outside the control of the tested approaches can introduce significant variability. We therefore do not interpret this variability as being indicative of the performance of any particular approach. We agree that different cloud and network configurations could exhibit different levels of variability, and we will clarify this limitation in the revised manuscript.
  l167-170: Is it just impractical to open the original files or is it even impossible?
  The original files can eventually be opened, but doing so requires thousands of HTTP requests and can take up to an hour in some cases. This is therefore not technically impossible, but clearly impractical for a simple dataset-opening operation that should take only a few seconds even in a WAN environment.
  Figure 5: I think the clear advantage shown here is that repacking speeds up the reading process significantly. Since the scale is log-scale it would be good to provide some numbers in the text how much faster the variables from the repacked files can be read by pyfive and kerchunk. Isn't Figure 6 a subset from Figure 5? Here the values for “Amon”' are hardly visible. Maybe a table would be the better choice.
  We used a logarithmic scale to illustrate the large differences between the approaches, including currently considered cloud-optimized solutions such as Kerchunk. In particular, our results show that repacking the HDF5 files can provide substantial performance improvements, and that applying Kerchunk to the repacked files can be both faster and more efficient than applying it to the original files.
  To make these differences more accessible, we provide Figure 6 without a logarithmic scale, allowing the actual performance of the configurations that we consider “production-ready” to be compared directly. This also avoids the visual compression of the differences between these configurations caused by including non-optimal solutions, such as Kerchunk applied to the original, non-repacked files.
  l190: Please consider providing an example list of clients that can and cannot fetch chunks concurrently.
  We will add some examples, such as netCDF4-python and h5py.
  l194-196 & l220-221 & l256-257: The same conclusion is repeated here. Rather have the conclusion just in the conclusions section.
  We will rewrite this in the reviewed version.
  
  Citation: https://doi.org/10.5194/egusphere-2026-3249-AC4

Ezequiel Cimadevilla, David Hassell, and Bryan N. Lawrence

Viewed

Total article views: 158 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
99	49	10	158	6	3

HTML: 99
PDF: 49
XML: 10
Total: 158
BibTeX: 6
EndNote: 3

Views and downloads (calculated since 16 Jun 2026)

Month	HTML	PDF	XML	Total
Jul 2026	55	30	3	88
Aug 2026	29	14	6	49
Sep 2026	15	5	1	21

Cumulative views and downloads (calculated since 16 Jun 2026)

Month	HTML	PDF	XML	Total
Jul 2026	55	30	3	88
Aug 2026	29	14	6	49
Sep 2026	15	5	1	21

Viewed (geographical distribution)

Total article views: 125 (including HTML, PDF, and XML) Thereof 125 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 07 Sep 2026

Short summary

Climate science is increasingly adopting cloud-based infrastructures for data access and analysis. Contrary to the widespread belief that traditional formats such as NetCDF/HDF5 are unsuitable for remote cloud access, this study demonstrates that they can perform efficiently in cloud environments. These findings challenge current assumptions and provide practical insights that could support CMIP7 data workflows while reducing the need for major changes to existing climate data infrastructures.


Total:	0
HTML:	0
PDF:	0
XML:	0