Accurate flood maps are vital for emergency planning. Existing NOAA Office of Water Prediction tools for checking flood map quality are highly capable but face scaling challenges, can be difficult to deploy on the cloud, and only work for one specific model. The NextGen Water Prediction program has designed a new flood map evaluation system using cloud technology. Key improvements include: the use of containers to enable parallel processing for 15x faster evaluations; modular design that allows for integration of new evaluation techniques; cloud-native architecture with AWS and HashiCorp Nomad for massive scalability; and extensibility to the evaluation of maps from multiple types of flood models. Results from using this system to evaluate flood scenarios at the continental scale will be presented. Early results show it effectively compares maps against benchmark data while analyzing how landscape features (like watershed slope and land cover type) impact model accuracy.