Solid Earth deforms under forcings, and those deformations affect various surface processes from mountain buildings to sea level changes. Measuring the strength (viscosity) of Earth is crucial to understanding those processes. Both ice sheet loading processes during ice ages and internal dynamic processes (i.e., mantle convection) drive Earth's deformation, and those corresponding observations can be used to infer Earth's viscosity. However, there is large discrepancy in viscosity inferred from those two approaches. We show it is possible to solve the problem by using a more realistic approach than previous studies in modeling the past sea level change caused by ice sheet loading processes during ice ages. We find after incorporating the lateral variations of viscosity, the viscosity inferred from mantle convection processes is consistent with the past sea level data. This means a unified viscosity model for both processes is possible. This work will have broad implications on Earth's dynamics, as inferring viscosity is fundamental to understand Earth's dynamics.