Accurately predicting corn yield is essential for managing food supply and supporting agricultural planning. However, traditional methods often require many years of data and struggle with complex growing conditions. In this study, we developed a new deep learning model that uses satellite imagery, environmental data, and information about crop growth stages to estimate corn yields at the county level in the U.S. Corn Belt. Our model combines several advanced techniques—3D convolution, temporal modeling, and vision transformers—within a multi-stream structure that processes different types of data more effectively. It performs well even when using fewer years of data, making it useful in regions with limited records. This approach shows promise for improving climate-smart agriculture and more resilient food systems through better yield forecasting.