California’s beaches are constantly changing, eroding, and recovering as waves move sand onshore and offshore. While it’s well known that big winter storms, especially during strong El Niño events, can cause major beach erosion, less is understood about what drives shoreline changes from year to year. In this study, we used satellite images of the California coast and computer models of ocean waves to explore how wave energy affects shoreline position over the last 37 years. We found that nearly half of the year-to-year changes in beach width across the state are tied to wave energy coming from a specific region in the central North Pacific Ocean. Interestingly, while El Niño years clearly bring stronger waves and narrower beaches, the reverse is not true: La Niña years do not reliably lead to beach recovery. Instead, beach recovery often occurs during neutral climate years, when North Pacific storm activity is lower. These insights can help planners better anticipate when erosion or recovery is most likely and support more effective timing of coastal protection and restoration efforts.