the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 23 Sep 2022
A turbulence data reduction scheme for autonomous and expendable profiling floats
Abstract. Autonomous and expendable profiling float arrays such as deployed in the Argo Program require the transmission of reliable data from remote sites. However, existing satellite data transfer rates preclude complete transmission of rapidly sampled turbulence measurements. It is therefore necessary to reduce turbulence data onboard. Here we propose a scheme for onboard data reduction and test it with existing turbulence data obtained with a newly developed version of a SOLO-II profiling float. The scheme invokes simple power law fits to (i) shear probe voltage spectra and (ii) fast thermistor voltage spectra that yield a fit value plus a quality control metric. At roughly 1 m vertical interval resolution, this scheme reduces the necessary data transfer volume 240-fold to approximately 3 kB for every 100 m of a profile (when profiling at 0.2 m s-1). Turbulent kinetic energy dissipation rate ε and thermal variance dissipation rate χ are recovered in post-processing. As a test, we apply our scheme to a dataset comprising 650 profiles and compare its output to that from our standard turbulence processing algorithm. For ε, values from the two approaches agree within a factor of two 87 % of the time; for χ, 78 %. These levels of agreement are greater than or comparable to that between the ε and χ values derived from two shear probes and two fast thermistors, respectively, on the same profiler.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2989 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2989 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-944', Anonymous Referee #1, 17 Nov 2022
I think this is an interesting paper describing a good method, but I’m not entirely convinced that there is a great reason for doing the fits on a constant frequency band, besides the convenience. The on-board processing is already complicated enough - ultimately I don’t understand why the on-board processing shouldn’t be more complete (doing the fits on fixed spatial scales on the wavenumber scaled spectra?) or much simpler (by sending back a more representative voltage spectra and doing the fit on shore)?
Getting a good estimates when the vertical velocity is not the nominal 0.2 m/s (e.g, near the top of the profiles, as the float comes to the surface) seems to be a very important aspect for the specific instrument discussed.
As far as I understand, the results are only obtained by fitting frequencies between 1 and 5 Hz. That corresponds to only 10 points in the spectrum… Separating this results on doing the fit on 5 points. Going from the full-spectrum (100 Hz for 5 sec. = 500 points) to 10 points is ultimately the core of the data reduction scheme. The paper claims that one can estimate accurate rates of dissipation from this narrow frequency range (without fancy despiking or using acceleration data). For these 10 points (for each channel), the fitting method returns 2 fitted values (factor of 5). This additional factor of 5 certainly a nice reduction. I also wonder at what precision that data is returned. Choosing a different number representation, or compression could also help here.
That being said, the paper is generally clear and the method is well documented. Particularly if the profiling (or horizontal velocity) varies a bit more widely, I would hesitate to really champion that method (because of the relatively narrow and fixed frequency band were the fits are done), but I can see how it might be useful and accurate for the application in question.
A few more comments:
Line 23-24: It would be useful to state the size of a typical dataset from an Argo float (one profile every 10 days). Something like “In contrast, a typical Argo profile (once every 10 days) contains XXX kB of data).
Line 54: Why is there a 3-axis accelerometer, compass, and pitot tube? What is done with that data? In addition to the data compression, if might be worth it to discuss power consumption…
Line 148: Could horizontal velocities impact the estimate of W? In particular, wave motion will have some horizontal component that is not present in pressure, but does advect turbulence past the sensor, no? In other words, are there situations where the flow past the sensor is not strictly vertical?
L170: “two-stage approach”. This phrasing, and the following sentence, made me expect that a description of the second stage would immediately follow. As it is now, I’m not sure I can readily identify the second stage (not mentioned until line 188).
When initial fit on the voltage is done on a frequency range past the inertial subrange, it seems that the least-square fit would be really dominated by the lower frequency elements of the band, since the spectrum rolls of so rapidly. The fit then doesn’t really help with any noise (for example, in Fig 3b). That is presumable captured in the score,
All the fits in Fig 4 have about the same value of epsilon. It might be interesting to have a column in Fig 4 for much smaller values (10^{-10}), and larger (10^{-6}, say), to see how the the fit is affected by what frequency/wavenumber range it is done over…
Raw data are typically not going to be recovered… What is reason for sampling so fast, if only data up to 5 Hz are used? Naively, perhaps, an analog filter could be used and microstructure signal could be sampled slower, no? Would that save power?
Citation: https://doi.org/10.5194/egusphere-2022-944-RC1 - AC1: 'Reply on RC1', Kenneth Hughes, 26 Jan 2023
-
CC1: 'Comment on egusphere-2022-944', Cynthia Bluteau, 05 Dec 2022
-
AC2: 'Reply on CC1', Kenneth Hughes, 26 Jan 2023
Per our response to RC1, our reponses to all reviews are in a single PDF. See that response for a link to the PDF (or follow https://doi.org/10.5194/egusphere-2022-944-AC1)
Citation: https://doi.org/10.5194/egusphere-2022-944-AC2
-
AC2: 'Reply on CC1', Kenneth Hughes, 26 Jan 2023
-
RC2: 'Comment on egusphere-2022-944', Toshiyuki Hibiya, 21 Dec 2022
Review of “A turbulence data reduction scheme for autonomous and expendable profiling floats” by H. G. Hughes, J. N. Moum, and D. L. Rudnick
Existing satellite data transfer rates prevent complete transmission of rapidly sampled turbulence measurements using autonomous and expendable profiling float arrays such as deployed in the Argo Program. It is therefore necessary to reduce turbulence data onboard. This study proposes a new scheme which invokes simple power law fits to shear probe voltage spectra and fast thermistor voltage spectra that yield a fit value as well as a quality control metric. At roughly 1 m vertical interval resolution, this scheme reduces the dataset size by a factor of ~240, namely, only 3 kB for each 100 m of a profile. The scheme is applied to a dataset comprising 650 profiles and compare its output to that from the standard turbulence processing algorithm. For ε (χ), values from the two approaches agree within a factor of two 87% (78%) of the time, which are greater than or comparable to that between the ε and χ values derived from two shear probes and two fast thermistors, respectively, on the same profiler.
Considering that continuous turbulence observations using autonomous and expendable profiling floats such as Deep Argo floats will be the norm in the near future, the development of the data reduction scheme as described in this paper is indispensable and deserves publication. Nevertheless, I am still not convinced about some aspects of the data reduction scheme described in the paper, so that I would be happy to receive some answers before publication.
Major Comment
- The data reduction scheme proposed in this paper presupposes the existence of a spectral slope with k1/3 dependence in the inertial subrange. However, Figure 4 shows that the shape of the measured shear spectrum significantly deviates from that of the Nasmyth spectrum, and does not appear to have the k1/3 slope presupposed in the inertial subrange. I am afraid that, in this case, the proposed formulation to obtain ε using the correction factor FNa defined by (17) and (20) might break down.
- Also, in this case, does the “fit score” defined in this study have any meaning? In other words, even if a good fit score is obtained by matching the ε calculated for 1-3 Hz with that for 3-5 Hz, this cannot necessarily be an indicator of a good match between the measured spectrum and the Nasmyth spectrum, and it may cause errors in the estimation of ε using (17) and (20), right?
- In section 4.1, the method for obtaining εint is not presented, and the discussion in section 4.1 proceeds without clarifying the definition of εint. Wouldn't it be easier to understand the overall flow of the discussion if the definition of εint written in section 4.2 were given first, followed by the discussion in section 4.1?
- Please add to the end of section 7 the reason why the agreement between the obtained χ values and those obtained from the standard scheme becomes worse than in the case of ε, even though the method for obtaining χ from the reduced scheme is basically the same as in the case of ε.
Minor comments
- Line 198: six times smaller than → eight times smaller than
- Although Hs(k) is defined in (3), it appears somewhat suddenly in (19) in section 4.1 without any connection to the preceding discussion, which seems a bit awkward.
- Just below color tones in Figures 7 and 10: proportion (%) → proportion (× 100 %)
Citation: https://doi.org/10.5194/egusphere-2022-944-RC2 -
AC3: 'Reply on RC2', Kenneth Hughes, 26 Jan 2023
Per our response to RC1, our reponses to all reviews are in a single PDF. See that response for a link to the PDF (or follow https://doi.org/10.5194/egusphere-2022-944-AC1)
Citation: https://doi.org/10.5194/egusphere-2022-944-AC3
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-944', Anonymous Referee #1, 17 Nov 2022
I think this is an interesting paper describing a good method, but I’m not entirely convinced that there is a great reason for doing the fits on a constant frequency band, besides the convenience. The on-board processing is already complicated enough - ultimately I don’t understand why the on-board processing shouldn’t be more complete (doing the fits on fixed spatial scales on the wavenumber scaled spectra?) or much simpler (by sending back a more representative voltage spectra and doing the fit on shore)?
Getting a good estimates when the vertical velocity is not the nominal 0.2 m/s (e.g, near the top of the profiles, as the float comes to the surface) seems to be a very important aspect for the specific instrument discussed.
As far as I understand, the results are only obtained by fitting frequencies between 1 and 5 Hz. That corresponds to only 10 points in the spectrum… Separating this results on doing the fit on 5 points. Going from the full-spectrum (100 Hz for 5 sec. = 500 points) to 10 points is ultimately the core of the data reduction scheme. The paper claims that one can estimate accurate rates of dissipation from this narrow frequency range (without fancy despiking or using acceleration data). For these 10 points (for each channel), the fitting method returns 2 fitted values (factor of 5). This additional factor of 5 certainly a nice reduction. I also wonder at what precision that data is returned. Choosing a different number representation, or compression could also help here.
That being said, the paper is generally clear and the method is well documented. Particularly if the profiling (or horizontal velocity) varies a bit more widely, I would hesitate to really champion that method (because of the relatively narrow and fixed frequency band were the fits are done), but I can see how it might be useful and accurate for the application in question.
A few more comments:
Line 23-24: It would be useful to state the size of a typical dataset from an Argo float (one profile every 10 days). Something like “In contrast, a typical Argo profile (once every 10 days) contains XXX kB of data).
Line 54: Why is there a 3-axis accelerometer, compass, and pitot tube? What is done with that data? In addition to the data compression, if might be worth it to discuss power consumption…
Line 148: Could horizontal velocities impact the estimate of W? In particular, wave motion will have some horizontal component that is not present in pressure, but does advect turbulence past the sensor, no? In other words, are there situations where the flow past the sensor is not strictly vertical?
L170: “two-stage approach”. This phrasing, and the following sentence, made me expect that a description of the second stage would immediately follow. As it is now, I’m not sure I can readily identify the second stage (not mentioned until line 188).
When initial fit on the voltage is done on a frequency range past the inertial subrange, it seems that the least-square fit would be really dominated by the lower frequency elements of the band, since the spectrum rolls of so rapidly. The fit then doesn’t really help with any noise (for example, in Fig 3b). That is presumable captured in the score,
All the fits in Fig 4 have about the same value of epsilon. It might be interesting to have a column in Fig 4 for much smaller values (10^{-10}), and larger (10^{-6}, say), to see how the the fit is affected by what frequency/wavenumber range it is done over…
Raw data are typically not going to be recovered… What is reason for sampling so fast, if only data up to 5 Hz are used? Naively, perhaps, an analog filter could be used and microstructure signal could be sampled slower, no? Would that save power?
Citation: https://doi.org/10.5194/egusphere-2022-944-RC1 - AC1: 'Reply on RC1', Kenneth Hughes, 26 Jan 2023
-
CC1: 'Comment on egusphere-2022-944', Cynthia Bluteau, 05 Dec 2022
-
AC2: 'Reply on CC1', Kenneth Hughes, 26 Jan 2023
Per our response to RC1, our reponses to all reviews are in a single PDF. See that response for a link to the PDF (or follow https://doi.org/10.5194/egusphere-2022-944-AC1)
Citation: https://doi.org/10.5194/egusphere-2022-944-AC2
-
AC2: 'Reply on CC1', Kenneth Hughes, 26 Jan 2023
-
RC2: 'Comment on egusphere-2022-944', Toshiyuki Hibiya, 21 Dec 2022
Review of “A turbulence data reduction scheme for autonomous and expendable profiling floats” by H. G. Hughes, J. N. Moum, and D. L. Rudnick
Existing satellite data transfer rates prevent complete transmission of rapidly sampled turbulence measurements using autonomous and expendable profiling float arrays such as deployed in the Argo Program. It is therefore necessary to reduce turbulence data onboard. This study proposes a new scheme which invokes simple power law fits to shear probe voltage spectra and fast thermistor voltage spectra that yield a fit value as well as a quality control metric. At roughly 1 m vertical interval resolution, this scheme reduces the dataset size by a factor of ~240, namely, only 3 kB for each 100 m of a profile. The scheme is applied to a dataset comprising 650 profiles and compare its output to that from the standard turbulence processing algorithm. For ε (χ), values from the two approaches agree within a factor of two 87% (78%) of the time, which are greater than or comparable to that between the ε and χ values derived from two shear probes and two fast thermistors, respectively, on the same profiler.
Considering that continuous turbulence observations using autonomous and expendable profiling floats such as Deep Argo floats will be the norm in the near future, the development of the data reduction scheme as described in this paper is indispensable and deserves publication. Nevertheless, I am still not convinced about some aspects of the data reduction scheme described in the paper, so that I would be happy to receive some answers before publication.
Major Comment
- The data reduction scheme proposed in this paper presupposes the existence of a spectral slope with k1/3 dependence in the inertial subrange. However, Figure 4 shows that the shape of the measured shear spectrum significantly deviates from that of the Nasmyth spectrum, and does not appear to have the k1/3 slope presupposed in the inertial subrange. I am afraid that, in this case, the proposed formulation to obtain ε using the correction factor FNa defined by (17) and (20) might break down.
- Also, in this case, does the “fit score” defined in this study have any meaning? In other words, even if a good fit score is obtained by matching the ε calculated for 1-3 Hz with that for 3-5 Hz, this cannot necessarily be an indicator of a good match between the measured spectrum and the Nasmyth spectrum, and it may cause errors in the estimation of ε using (17) and (20), right?
- In section 4.1, the method for obtaining εint is not presented, and the discussion in section 4.1 proceeds without clarifying the definition of εint. Wouldn't it be easier to understand the overall flow of the discussion if the definition of εint written in section 4.2 were given first, followed by the discussion in section 4.1?
- Please add to the end of section 7 the reason why the agreement between the obtained χ values and those obtained from the standard scheme becomes worse than in the case of ε, even though the method for obtaining χ from the reduced scheme is basically the same as in the case of ε.
Minor comments
- Line 198: six times smaller than → eight times smaller than
- Although Hs(k) is defined in (3), it appears somewhat suddenly in (19) in section 4.1 without any connection to the preceding discussion, which seems a bit awkward.
- Just below color tones in Figures 7 and 10: proportion (%) → proportion (× 100 %)
Citation: https://doi.org/10.5194/egusphere-2022-944-RC2 -
AC3: 'Reply on RC2', Kenneth Hughes, 26 Jan 2023
Per our response to RC1, our reponses to all reviews are in a single PDF. See that response for a link to the PDF (or follow https://doi.org/10.5194/egusphere-2022-944-AC1)
Citation: https://doi.org/10.5194/egusphere-2022-944-AC3
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 356 | 103 | 21 | 480 | 4 | 3 |
- HTML: 356
- PDF: 103
- XML: 21
- Total: 480
- BibTeX: 4
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Cited
1 citations as recorded by crossref.
Kenneth G. Hughes
James N. Moum
Daniel L. Rudnick
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2989 KB) - Metadata XML