Positive Matrix Factorization of Large Aerosol Mass Spectrometry Datasets Using Error-Weighted Randomized Hierarchical Alternating Least Squares

Sapper, Benjamin; Henze, Daven; Canagaratna, Manjula; Stark, Harald

doi:https://doi.org/10.5194/egusphere-2022-1221

Preprints

https://doi.org/10.5194/egusphere-2022-1221

Preprints

20 Dec 2022

| 20 Dec 2022

Positive Matrix Factorization of Large Aerosol Mass Spectrometry Datasets Using Error-Weighted Randomized Hierarchical Alternating Least Squares

Benjamin Sapper, Daven Henze, Manjula Canagaratna, and Harald Stark

Abstract. Weighted positive matrix factorization (PMF) has been used by scientists to find small sets of underlying factors in environmental data. However, as the size of the data has grown, increasing computational costs have made it impractical to use traditional methods for this factorization. In this paper, we present a new weighting method to dramatically decrease computational costs for these traditional algorithms. We then apply this weighting method with the Randomized Hierarchical Alternating Least Squares (RHALS) algorithm to a large environmental dataset, where we show that interpretable factors can be reproduced using these methods. We show this algorithm results in a computational speedup of 38, 67, and 634 compared to the Multiplicative Update (MU), deterministic Hierarchical Alternating Least Squares (HALS), and non-negative Alternating Least Squares (ALS) algorithms, respectively. We also investigate rotational ambiguity in the solution, and present a simple “pulling” method to rotate a set of factors. This method is shown to find alternative solutions, and in some cases, lower the weighted residual error of the algorithm.

Received: 05 Nov 2022 – Discussion started: 20 Dec 2022

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 3513 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (3513 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

19 May 2025

Positive matrix factorization of large real-time atmospheric mass spectrometry datasets using error-weighted randomized hierarchical alternating least squares

Benjamin C. Sapper, Sean Youn, Daven K. Henze, Manjula Canagaratna, Harald Stark, and Jose L. Jimenez

Geosci. Model Dev., 18, 2891–2919, https://doi.org/10.5194/gmd-18-2891-2025,https://doi.org/10.5194/gmd-18-2891-2025, 2025

Short summary

Benjamin Sapper, Daven Henze, Manjula Canagaratna, and Harald Stark

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2022-1221', Anonymous Referee #1, 11 Apr 2023
Could the randomized strategy be explained as a stochastic minimization approach for imposing rank constraints on the NMF solution? Specifically, can it be demonstrated that the NMF solution combined with random projection minimizes a certain cost function? This information would help in evaluating the convergence properties of the proposed method.
Regarding the potential drawback of imposing rank constraints on the NMF outputs, is it feasible to relax these constraints, perhaps by employing the nuclear norm?
Lastly, the manuscript does not do a good job of citing the related state-of-the-art methods. For example, there has been a tremendous amount of work done in relation to randomized weighted NMF. Please see the following:
Yahaya, F., Puigt, M., Delmaire, G., & Roussel, G. (2021, June). Random Projection Streams for (Weighted) Nonnegative Matrix Factorization. In IEEE ICASSP 2021.

Yahaya, F. (2021, November). Compressive informed (semi-) non-negative matrix factorization methods for incomplete and large-scale data: with application to mobile crowd-sensing data. Université du Littoral Côte d'Opale.

Yahaya, F., Puigt, M., Delmaire, G., & Roussel, G. (2020). Gaussian Compression Stream: Principle and Preliminary Results. arXiv preprint arXiv:2011.05390.
Citation: https://doi.org/10.5194/egusphere-2022-1221-RC1
- AC1: 'Reply on RC1', Benjamin Sapper, 14 Jun 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2022-1221-AC1
RC2:
'Comment on egusphere-2022-1221', Anonymous Referee #2, 20 Apr 2023

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2022-1221-RC2
- AC2: 'Reply on RC2', Benjamin Sapper, 14 Jun 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2022-1221-AC2

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2022-1221', Anonymous Referee #1, 11 Apr 2023
Could the randomized strategy be explained as a stochastic minimization approach for imposing rank constraints on the NMF solution? Specifically, can it be demonstrated that the NMF solution combined with random projection minimizes a certain cost function? This information would help in evaluating the convergence properties of the proposed method.
Regarding the potential drawback of imposing rank constraints on the NMF outputs, is it feasible to relax these constraints, perhaps by employing the nuclear norm?
Lastly, the manuscript does not do a good job of citing the related state-of-the-art methods. For example, there has been a tremendous amount of work done in relation to randomized weighted NMF. Please see the following:
Yahaya, F., Puigt, M., Delmaire, G., & Roussel, G. (2021, June). Random Projection Streams for (Weighted) Nonnegative Matrix Factorization. In IEEE ICASSP 2021.

Yahaya, F. (2021, November). Compressive informed (semi-) non-negative matrix factorization methods for incomplete and large-scale data: with application to mobile crowd-sensing data. Université du Littoral Côte d'Opale.

Yahaya, F., Puigt, M., Delmaire, G., & Roussel, G. (2020). Gaussian Compression Stream: Principle and Preliminary Results. arXiv preprint arXiv:2011.05390.
Citation: https://doi.org/10.5194/egusphere-2022-1221-RC1
- AC1: 'Reply on RC1', Benjamin Sapper, 14 Jun 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2022-1221-AC1
RC2:
'Comment on egusphere-2022-1221', Anonymous Referee #2, 20 Apr 2023

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2022-1221-RC2
- AC2: 'Reply on RC2', Benjamin Sapper, 14 Jun 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-1221/egusphere-2022-1221-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2022-1221-AC2

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Benjamin Sapper on behalf of the Authors (15 Jun 2023) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (20 Jun 2023) by Klaus Klingmüller

RR by Anonymous Referee #1 (07 Jul 2023)

RR by Anonymous Referee #2 (31 Jul 2023)

Suggestions for revision or reasons for rejection

The paper has been improved. However, in my opinion, there are still several outstanding issues. For example, not all errors that I characterized as "sloppy" (hence embarrassing) have been fixed and other similar ones remain.

In particular, in the Appendix there was a statement that H is a kxn matrix that is assumed to be full column rank (even though k was assumed to be smaller than n). In their response, the authors say that this (erroneous) statement was corrected with "H is a kxn matrix that is assumed to be full row rank". However, the revised document (line 666) still has the wrong statement. Similarly, following my suggestion, the authors now use Hadamard notation for elementwise operations. However, there seem to be cases where this was not applied, e.g. lines 158 and 161 (there might be others). There might be other "sloppy type" errors that the authors should find and but as I said, I do not consider the reviewer's task to be that of a chaperone. The authors should carefully revise their paper proactively and not only addressing the reviewers' comments.

The authors claim in the abstract that their algorithm results in computational speedups of 38, 67 and 634 compared to other algorithms. The statement is not justified by the methodology adopted in this paper. In particular, there is no detailed performance evaluation, neither a careful optimization of the methods. A modest statement, e.g. "numerical experiments with the proposed method indicate that the method is faster than competing algorithms" would be more appropriate.

This paper essentially uses a modification of the NMF to target a specific dataset (analyzed extensively by Massoli et al. https://pubs.acs.org/doi/epdf/10.1021/acsearthspacechem.8b00028) for which k=6 factors have been found to be appropriate. The paper makes no effort to consider different combinations of (m,n,k). It seems to me that this makes the paper very application and data specific and possibly not as interesting. In any case, the authors should declare this early in the paper so that the readers are not misled regarding the generality of the method.

The authors insist on using the term PMF instead of NMF (because of the use of the term by Paatero et al. However, the matrices used in the paper are not positive, they are nonnegative. Calling a matrix with some zero elements positive is mathematically incorrect, with all due respect to the pioneering work of Paatero et al. Indeed, there are results that hold for positive matrices but do not hold for nonnegative ones unless extra conditions are imposed (e.g. the Perron theorem, see https://en.wikipedia.org/wiki/Perron%E2%80%93Frobenius_theorem).

The statement in the authors' response "We have addressed the Computational Cost in 2 in response to Main Criticism 1. Both proposed postprocessing steps are O(mnk), and we found experimentally that using MATLAB’s pinv, a Krylov subspace method (Feng et al.,2018), generated slightly faster results." is wrong, at least regarding the MATLAB pinv function. It is likely that the authors were misled by the discussion in the reference by Feng et al. which argues (rightly) for using the lansvd of PROPACK instead of the Mathworks svds for performing large scale truncated svd. On the other hand, the native MATLAB pinv does not use svds but the native MATLAB svd function which is certainly not a Krylov method. Incidentally, this is another example where the choice of citation matters. This affects lines 305-311 of the text and possibly others.

Other issues - some notational (the list is not exhaustive):
- Throughout the paper, the authors use the symbol k sometimes as index and other times as the rank of the sought factorization which is confusing. Also, in some summation formulas (e.g. 2), the authors use summation (e.g. over k) without specifying the range while in others (e.g. 7) they do.
- If Q_j is defined as stated (minimizing) then the min operator on the left-hand side of relation (11) is redundant.
- Using both Q_j^i and Q_j^P (defined differently) is confusing.
- The last comma between W(:,j) and H(i,:) should be eliminated.
- The sizes of the matrices Sigma_i, Sigma_p and Sigma should be explicitly specified.
- The use of italics for Tr and arg is not consistent with the use of romans for min and max. In mathematical typesetting it is better to keep these non-italicized/sans serif.
- Notation becomes quite confusing in (27) and beyond.
- On p11, last line, it should be "Thus, the algorithm cannot handle ..." to make it clear that you are referring to the algorithm anot not to the uncertainty matrix.
- On p12, "denotes the pseudoinverses" should be "denote the pseudoinverses". It is stated that "Mathematically, these methods ..." - state which are these methods.
- On p12, line 312 and beyond, the initialization of H is not clear (also better say "initialize" rather than "initiate"). In particular, formula (37) shows H_0 on the left side and H on the right. How is H chosen on the right side?
- On p12, line 319 and possibly elsewhere, the font for H and W is inconsistent (non-bold) with the (bold) font for the same variables elsewhere in the text.
- When defining norms it is better to use some space, a variable symbol or a dot instead of using the two double vertical bars next to each other.
- In line 377, it is stated that MU is initialized with random numbers. I assume this means that the W and H factors in MU are initialized with nonnegative random values. If this is so, it needs to be said. The distribution also needs to be mentioned (e.g. are they uniformly distributed pseudorandom numbers in [0,1]?) The authors still make the error and use the term "non-negative SVD" in line 379, in contrast with the term used for NNDSVD in line 378. See also the comment labeled [page13] in my original review. It is also hard not to notice that the authors write "nonnegative" in lines 377-8 and "non-negative" in line 379.
- On p. 8, the W(:,j), H(j,:) are introduced as the j-th column and row of W and H, while in line 394, the notation becomes W_j and H_j. Incidentally, if you follow the parenthesized notation, then element at row-i, column-j should be denoted by W(i,j) and not W_{ij}.

Concerning the bibliography: Some of the references are not suitable. For example: Why refer to Tan et al. 2018 and Takacs and Tikk for HALS? Have these papers introduced HALS? To my understanding, they are only using some version of HALS for some application. Why not simply refer to the primary sources for HALS or even better references that provide a solid foundational discussion, e.g. the monograph by Gillis or the book by Cichocki, Zdunek et al. I would also assume that there must be some specific journal bibliography style.

Several references are incomplete or shown in an inconsistent manner. For example: The Ph.D. thesis by Yahaya does not show the institution. Some wordings are capitalized for no good reason (the institution for Ho's thesis). Using url citations (e.g. Burred J.) that have not undergone proper peer review or at least are posted on a reputable archive (e.g. ArXiv) is not good practice. For this particular one, either the Gillis monograph or the aforementioned monograph by Cichocki, Zdunek et al. noted earlier would be sufficient and more appropriate.

Hide

ED: Reconsider after major revisions (26 Aug 2023) by Klaus Klingmüller

AR by Sean Youn on behalf of the Authors (01 Aug 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (14 Aug 2024) by Klaus Klingmüller

RR by Anonymous Referee #3 (08 Feb 2025)

ED: Publish subject to minor revisions (review by editor) (11 Feb 2025) by Klaus Klingmüller

AR by Sean Youn on behalf of the Authors (21 Feb 2025) Author's response Author's tracked changes Manuscript

ED: Publish as is (03 Mar 2025) by Klaus Klingmüller

AR by Sean Youn on behalf of the Authors (05 Mar 2025)

Journal article(s) based on this preprint

19 May 2025

Positive matrix factorization of large real-time atmospheric mass spectrometry datasets using error-weighted randomized hierarchical alternating least squares

Benjamin C. Sapper, Sean Youn, Daven K. Henze, Manjula Canagaratna, Harald Stark, and Jose L. Jimenez

Geosci. Model Dev., 18, 2891–2919, https://doi.org/10.5194/gmd-18-2891-2025,https://doi.org/10.5194/gmd-18-2891-2025, 2025

Short summary

Benjamin Sapper, Daven Henze, Manjula Canagaratna, and Harald Stark

Viewed

Total article views: 1,005 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
662	300	43	1,005	40	45

HTML: 662
PDF: 300
XML: 43
Total: 1,005
BibTeX: 40
EndNote: 45

Views and downloads (calculated since 20 Dec 2022)

Month	HTML	PDF	XML	Total
Dec 2022	77	27	8	112
Jan 2023	32	17	0	49
Feb 2023	45	24	0	69
Mar 2023	45	18	0	63
Apr 2023	50	18	5	73
May 2023	27	19	0	46
Jun 2023	38	13	6	57
Jul 2023	22	8	1	31
Aug 2023	42	19	0	61
Sep 2023	22	18	0	40
Oct 2023	18	11	0	29
Nov 2023	6	6	0	12
Dec 2023	16	14	2	32
Jan 2024	11	5	3	19
Feb 2024	7	6	0	13
Mar 2024	14	7	2	23
Apr 2024	14	4	4	22
May 2024	14	12	2	28
Jun 2024	37	9	5	51
Jul 2024	9	7	3	19
Aug 2024	23	2	1	26
Sep 2024	14	1	0	15
Oct 2024	6	2	0	8
Nov 2024	22	1	23
Dec 2024	10	6	0	16
Jan 2025	6	5	0	11
Feb 2025	14	5	0	19
Mar 2025	12	2	0	14
Apr 2025	6	9	0	15
May 2025	3	6	0	9

Cumulative views and downloads (calculated since 20 Dec 2022)

Month	HTML	PDF	XML	Total
Dec 2022	77	27	8	112
Jan 2023	32	17	0	49
Feb 2023	45	24	0	69
Mar 2023	45	18	0	63
Apr 2023	50	18	5	73
May 2023	27	19	0	46
Jun 2023	38	13	6	57
Jul 2023	22	8	1	31
Aug 2023	42	19	0	61
Sep 2023	22	18	0	40
Oct 2023	18	11	0	29
Nov 2023	6	6	0	12
Dec 2023	16	14	2	32
Jan 2024	11	5	3	19
Feb 2024	7	6	0	13
Mar 2024	14	7	2	23
Apr 2024	14	4	4	22
May 2024	14	12	2	28
Jun 2024	37	9	5	51
Jul 2024	9	7	3	19
Aug 2024	23	2	1	26
Sep 2024	14	1	0	15
Oct 2024	6	2	0	8
Nov 2024	22	1	23
Dec 2024	10	6	0	16
Jan 2025	6	5	0	11
Feb 2025	14	5	0	19
Mar 2025	12	2	0	14
Apr 2025	6	9	0	15
May 2025	3	6	0	9

Viewed (geographical distribution)

Total article views: 978 (including HTML, PDF, and XML) Thereof 978 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 19 May 2025

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (3513 KB)
Metadata XML

Short summary

Positive Matrix Factorization (PMF) has been used by atmospheric scientists to extract underlying factors present in large datasets. This paper presents a new technique for weighted PMF that drastically reduces the computational costs of previously developed algorithms. We use this technique to deliver interpretative factors and solution diagnostics from an atmospheric chemistry dataset.


Total:	0
HTML:	0
PDF:	0
XML:	0