The recent historic minima in Antarctic sea ice extent suggest the possibility of new thermodynamic and dynamic conditions across the Southern Ocean. In investigating these thermodynamic and dynamic sea ice behaviors, it is essential to observe sea ice drift with high quality. However, while the relatively abundant and reliable buoy data in the Arctic guarantees a robust Arctic sea ice drift observation, the Antarctic sea ice drift product merely relies on passive microwave (PMW) data due to the lack of pan-Antarctic drifting buoy data. In this study, we assess the uncertainty of the Antarctic sea ice drift product from 2015 to 2023 by using drifting buoys in the Weddell Sea and synthetic aperture radar (SAR) sea ice drift across the Southern Ocean. The comparison between PMW and buoy ice drift shows that PMW-derived sea ice drift tends to underestimate drift speed by 2–3 km d^-1, particularly under low ice concentration conditions, while drift direction agrees well with a marginal bias. Based on the accurate high-resolution sea ice drift estimation from SAR imagery (-6° of angle difference and 0.1 km d^-1 of speed difference with buoy ice drift), we assess the pan-Antarctic uncertainties of PMW sea ice drift. We found a widespread ∼11 km d^-1 underestimation of ice drift speed across the Southern Ocean, particularly in the east Weddell Sea and west Ross Sea. Ice drift direction generally shows negligible bias across the Southern Ocean, but the east Weddell Sea shows 10–20° of clockwise bias. Such a wide underestimation is attributed to the optimal interpolation that smooths ice velocity and raises uncertainties around the marginal ice zone. Based on this understanding of PMW-derived Antarctic sea ice drift estimation, it is important to improve the sea ice velocity estimation in the Southern Ocean.