While deep learning (DL) models are effective in rainfall-runoff modelling, their dependence on data and lack of physical mechanisms can limit their use in hydrology. As there is yet no consensus on the consideration of the fundamental water balance for DL models, this paper presents an in-depth investigation of the effects of water balance constraint on the long-short term memory (LSTM) network. Specifically, based on the Catchment Attributes and Meteorology for Large-sample Studies (CAMELS) dataset, the LSTM and its architecturally mass-conserving variant (MC-LSTM) are trained basin-wise to provide rainfall-runoff prediction and then the robustness of the LSTM and MC-LSTM against data sparsity, random parameters initialization and contrasting climate conditions are assessed across the contiguous United States. Through large-sample tests, the results show that the water balance constraint evidently improves the robustness of the basin-wise trained LSTM. On the one hand, as the amount of training data increases from 1 year to 15 years, the incorporation of the water balance constraint into the LSTM network decreases the sensitivity from 95.0 % to 32.7 %. On the other hand, the water balance constraint contributes to the stability of the LSTM for 450 (85 %) basins when there are 3 years’ training data. In the meantime, the water balance constraint improves the transferability of the LSTM from the driest years to the wettest years for 318 (67 %) basins. Overall, the in-depth investigations of this paper facilitate insights into the use of DL models for rainfall-runoff modelling.