Enter authors here: Andrew Bennett1, Bart Nijssen1
1Department of Civil and Environmental Engineering, University of Washington, Seattle, WA, USA
Corresponding author: Andrew Bennett (andrbenn@uw.edu)
Key Points:
Abstract
Deep learning (DL) methods have shown great promise for accurately predicting hydrologic processes but have not yet reached the complexity of traditional process-based hydrologic models (PBHM) in terms of representing the entire hydrologic cycle. The ability of PBHMs to simulate the hydrologic cycle makes them useful for a wide range of modeling and simulation tasks, for which DL methods have not yet been adapted. We argue that we can take advantage of each of these approaches by embedding DL methods into PBHMs to represent individual processes. We demonstrate that this is viable by developing DL-based representations of turbulent heat fluxes and coupling them into the Structure for Unifying Multiple Modeling Alternatives (SUMMA), a modular PBHM modeling framework.
We developed two DL parameterizations and integrated them into SUMMA, resulting in a one-way coupled implementation (NN1W) which relies only on model inputs and a two-way coupled implementation (NN2W), which also incorporates SUMMA-derived model states. Our results demonstrate that the DL parameterizations are able to outperform calibrated standalone SUMMA benchmark simulations. Further we demonstrate that the two-way coupling can simulate the long-term latent heat flux better than the standalone benchmark and one-way coupled configuration. This shows that DL methods can benefit from PBHM information, and the synergy between these modeling approaches is superior to either approach individually.
Plain Language Summary
Machine learning (ML) and process-based methods are two approaches to hydrologic modeling. Process-based hydrologic models (PBHMs) represent the hydrologic cycle by solving equations which have been developed from physical theory or experimentation, while ML models make predictions based on patterns learned from large amounts of data. A particular sub-field of machine learning called deep learning (DL) has been shown to often outperform process-based models. However, current DL models do not represent all aspects of the hydrologic cycle (such as streamflow, evaporation, groundwater storage, and snowpack) at once, as is often done in PBHMs. As a result, DL models in hydrology are often single purpose, while PBHMs can be used for many different scientific and/or engineering purposes.
We show how individual DL models that simulate evaporation and convective heat transport at the land surface can be incorporated into a PBHM. We show that deep learning simulated evaporation and convective heat transport better than the PBHM. We also show how the incorporation of deep learning into process-based models can further improve the DL model itself. We conclude that taking advantage of both modeling perspectives is better than either on its own.
1 Introduction
The debates amongst the hydrologic modeling community about the use and utility of machine learning (ML) to simulate hydrologic processes indicate that much work remains to be done to understand the role and potential of machine learning in hydrologic modeling (Nearing et al., 2020; Shen, 2018). While it is true that deep learning (DL) models have shown great promise and superior performance in many cases it is yet unclear how to make models that are both composable (that is, easy to combine with other models) and transferable for scientific studies (that is, the same model configuration can be used to explore disparate scientific questions). In this paper we outline an approach for coupling DL models of individual processes into existing hydrologic modeling frameworks. This coupling approach allows us to represent individual physical processes within a larger model using ML methods and to introduce feedbacks between model components. The ability to couple model components will address these composability and transferability questions, as well as allow use of these types of machine-learned models in areas which do not have readily available training data.
There are several reasons for the rapid advancement of ML-based approaches in hydrology (and other fields), including a greater abundance of publicly available data, increased computational resources, and better frameworks for selecting, fitting, and applying models. Along with this increase in interest, the community has also begun to think about how to incorporate aspects of physical theory into these data driven models. This desire for physics-based machine learning is enticing for a number of reasons. As scientists we hope that the use of models which are based in, or constrained by, physical properties will allow us to learn about the underlying processes of the systems we are modeling. Not only that, we hope that such approaches will be able to efficiently extract information from a variety of datasets, from in situ observations to satellite remote sensing data, or be able to represent complex phenomena in a more efficient way.
While inclusion of empirical or statistical relationships of individual processes in hydrologic models is common, this is not yet the case for ML methods. One reason for this is that it is not clear how to combine ML models in the same way that we have been able to include processes for which we have parsimonious descriptions. Additionally, methodologies for representing physical relationships between ML-based process representations have not been developed in the hydrology community. In part, this is not surprising since machine learning is good at resolving relationships that we have not been able to decompose into easily describable parts. This “whole-system” or “black box” approach is conceptually appealing due to its simplicity, and is exemplified by rainfall-runoff modeling, which deep learning has proven to be very good at (Hu et al., 2018; Kratzert et al., 2018; Moshe et al., 2020). However, by taking a more granular approach, we will show that DL models can be successfully incorporated as process modules into existing models. Doing so allows us to see how changes in a single component affect the entire system.
In this paper, we look at turbulent heat fluxes, for which high-quality, long-term, local observations from eddy covariance towers (here, from FluxNet; Pastorello et al., 2020) are available across a range of hydroclimates. While machine learning has been used for modeling of turbulent heat fluxes and evaporation (Jung et al., 2009; Tramontana et al., 2016; Zhao et al., 2019) there have not yet been model intercomparisons with land surface models, much less integrations into land surface models. However, Best et al. (2015) showed that even simple statistical models are often able to outperform state of the art land surface models in simulation of latent and sensible heat fluxes. Best et al. (2015) postulated that the statistical models were better able to use the information in the meteorological forcing data than the physics-based approaches. This indicates there is strong motivation for incorporating data-driven techniques into complex land surface and hydrologic models. We believe that if these types of approaches are able to provide better performance than the physically motivated relationships we should work to understand how and why this performance is better and use them where appropriate and applicable.
Despite the statistical benchmarks’ superior ability for predicting turbulent heat fluxes in Best et al. (2015), land surface models remain more suitable for a wide range of applications, because they represent a wider range of hydrologic processes and may be better suited for studies of environmental change. Such studies include drought prediction (Li et al., 2012), snow melt predictions under climate change (Musselman et al., 2017), and predicting volatile organic compound emissions (Lathière et al., 2006). That is not to say that ML models cannot be used in this way or incorporated into larger frameworks. Both Kratzert et al. (2018) and Jiang et al. (2020) make qualitative comparisons of internal ML model states to snowpack, but do not later use the models for prediction of snowpack. We believe that it is likely that ML models will be used for such purposes in the near future, but the question remains open how to extract process information from statistical models.
Because the hydrology community is still learning the best ways to build and use ML models, there remains considerable room for incorporation of machine learning into more conventional process-based hydrologic models (PBHMs), which have the flexibility needed for general purpose modeling. This approach has been adopted recently by Brenowitz & Bretherton (2018) as well as Rasp et al. (2018) for parameterizing sub-gridcell scale processes, such as cloud convection, in atmospheric circulation models. Similarly, in oceanography, neural networks have been used to parameterize the turbulent vertical mixing in the ocean surface (Ramadhan et al., 2020).
In this study, we demonstrate how coupling ML models into a hydrologic model can yield better performance at estimating turbulent heat fluxes without sacrificing mass and energy balance closure or the ability to represent other processes such as runoff or snowpack. We have developed two ML models to simulate latent and sensible heat fluxes. We embed these ML models as process parameterizations inside of a PBHM. These ML-based process parameterizations replace the turbulent heat flux equations of the original PBHM. Our first model was only allowed to learn from the same meteorological data that is used to force the hydrologic model, while our second ML model is additionally trained with the inclusion of states derived from the hydrologic model. We show that both ML models are able to outperform the routines for simulating turbulent heat fluxes at subdaily timescales. We also show that the configuration which was trained using model states is better able to reproduce the long-term water balance. Our results indicate that approaches to coupling machine learning with PBHMs offer a promising avenue, which has only begun to be explored.
2 Materials and Methods
2.1 Data and study sites
We used data from 60 FluxNet sites (Pastorello et al., 2020) to run our experiments. These sites cover a large variety of vegetation and climate classifications. Our site selection process considered several criteria. We first filtered the full FluxNet dataset to make sure we only included sites which had energy balance corrected measurements of both sensible and latent heat fluxes, which will be discussed later. We then made sure that these sites had the necessary variables to force our models, which include precipitation, air temperature, incoming shortwave radiation, incoming longwave radiation, specific humidity, air pressure, and wind speed. We then removed sites which had either fewer than three years of contiguous data or more than 20% missing observations during the longest continuous period with observations. For the remaining sites, we used gap-filled data provided as part of the FluxNet dataset. Gap-filling was based on ERA-Interim (ERAI) (Dee et al., 2011) and includes downscaling and postprocessing explicitly for the purpose of model forcing. Time steps flagged as gap-filled were excluded from our performance analysis to ensure that we did not simply measure the ability of our simulations to model ERAI data. However, the gap-filled data is included when analyzing the water balance.
We also limited our analysis to sites which had an observed ET/P ratio of less than 1.1, calculated using the mean FluxNet-reported values of ET and P over the simulation period. This was done to accommodate our model structure, which enforces mass and energy balances on a point (or lumped) scale. Larger observed ET/P ratios likely occur at sites which have strong spatial gradients and flow convergence, so that moisture available for ET is not just the result of local precipitation. Our filtering process resulted in 60 sites with 508 site-years of data. A breakdown of the site names, data periods, locations and site characteristics are given in Table 1. Figure 1 shows the locations and vegetation classes for these same sites.
Table 1. A listing of the sites, locations, IGBP vegetation types, and dates of simulation. Locations are given as (Latitude (°N), Longitude (°E)). Vegetation types are given by their IGBP codes. MF is mixed forest, ENF is evergreen needleleaf forest, CRL is croplands, GRL is grasslands, SVN is savannas, OSL is open shrublands, WLD is permanent wetlands, DBF is deciduous broadleaf forest, and WS is woody savannas. Site names are taken from FluxNet, and consist of a two-letter country code followed by a three-letter site code.