Winter Arctic polynyas, openings in the pack ice, play a crucial role for the climate from sea ice production to cloud formation and are hotspots for the ecosystem and human activity. Their area has significantly increased since satellite records began. Yet their representation has yet to be evaluated in any generation of global climate models, most likely because their automatic multi-model retrieval is challenging. We here use a newly-developed machine-learning based method and evaluate polynya activity against the satellite-derived one over 1979–2024 in the 18 models with daily sea ice concentration available. We find that models overestimate winter Arctic polynya area but underestimate its frequency, and limit their opening to the seasonally ice-covered regions. Polynya area is increasing in most models, but the bias of these trends are inconsistent. Although the model with the highest resolution has both the highest areas and frequencies, the sea ice model component is a more robust predictor of polynya activity, with most activity in the models whose thermodynamics scheme enhances ice growth/melt and least activity in models whose dynamics scheme dampens the influence of wind on ice. Accordingly, we found more polynyas in the models with a larger seasonal cycle, in particular those with a warmer autumn that would delay ice growth. Finally, we confirm preliminary findings that polynya activity does not seem to impact the representation of the water column; if anything, we find less-dense water at the bottom of the continental shelf following larger polynya activity, i.e. supposedly with dense water formation.