Decadal variation in the behaviour of the El Niño Southern Oscillation (ENSO) affects our ability to predict ENSO and to identify its response to climate change. With the aim to better characterise and understand the causes of this decadal modulation of ENSO, we vary the terms of a recharge oscillator model (ROM). We then analyse the resulting decadal variations in Bjerknes index (a measure of growth rate, influencing amplitude) and Wyrtki index (a measure of phase speed, influencing periodicity). These indices are compared across output from ROMs of varying nonlinearity, as well as observations, and output from CMIP models. We find that a linear ROM with stochastic noise forcing explains over 75% of decadal modulation in both observations and a full CMIP model. This creates a null hypothesis against which to compare additional processes to explain ENSO decadal modulation, and highlights the impacts of high-frequency atmospheric noise forcing on ENSO behaviour. Considering our findings that empirically fitted nonlinearities do little to alter ROM representation of ENSO decadal modulation, and full CMIP models exhibit the same timescale of ENSO predictability as a ROM, we suggest that the primary driver of ENSO decadal modulation in Bjerknes index and Wyrtki index is stochastic atmospheric forcing on sub-monthly timescales.