Through analysis of remotely sensed sea surface temperature (SST) and sea ice concentration data, we investigate the impact of winds related to the Southern Annular Mode (SAM) on sea ice extent around Antarctica. We show that positive SAM anomalies in the austral summer are associated with anomalously cold SSTs that persist and lead to anomalous ice growth in the following autumn, while negative SAM anomalies precede warm SSTs and a reduction in sea ice extent during autumn. The largest effect occurs in April, when a unit change in the detrended summertime SAM is followed by a 1.8±0.6 ×10⁵ km² change in detrended sea ice extent. We find no evidence that sea ice extent anomalies related to the summertime SAM affect the wintertime sea ice extent maximum. Our analysis shows that the wind anomalies related to the negative SAM during the 2016/2017 austral summer contributed to the record minimum Antarctic sea ice extent observed in March 2017.