Enter authors here: Andrew Bennett1, Bart
Nijssen1
1Department of Civil and Environmental Engineering,
University of Washington, Seattle, WA, USA
Corresponding author: Andrew Bennett
(andrbenn@uw.edu)
Key Points:
- Deep learned process parameterizations of turbulent heat fluxes
outperform physically-based parameterizations.
- Deep learned process parameterizations can be dynamically coupled into
process-based hydrologic models.
- Incorporation of process-based model derived states into deep learning
introduces feedbacks that improve long-term simulations.
Abstract
Deep learning (DL) methods have shown great promise for accurately
predicting hydrologic processes but have not yet reached the complexity
of traditional process-based hydrologic models (PBHM) in terms of
representing the entire hydrologic cycle. The ability of PBHMs to
simulate the hydrologic cycle makes them useful for a wide range of
modeling and simulation tasks, for which DL methods have not yet been
adapted. We argue that we can take advantage of each of these approaches
by embedding DL methods into PBHMs to represent individual processes. We
demonstrate that this is viable by developing DL-based representations
of turbulent heat fluxes and coupling them into the Structure for
Unifying Multiple Modeling Alternatives (SUMMA), a modular PBHM modeling
framework.
We developed two DL parameterizations and integrated them into SUMMA,
resulting in a one-way coupled implementation (NN1W) which relies only
on model inputs and a two-way coupled implementation (NN2W), which also
incorporates SUMMA-derived model states. Our results demonstrate that
the DL parameterizations are able to outperform calibrated standalone
SUMMA benchmark simulations. Further we demonstrate that the two-way
coupling can simulate the long-term latent heat flux better than the
standalone benchmark and one-way coupled configuration. This shows that
DL methods can benefit from PBHM information, and the synergy between
these modeling approaches is superior to either approach individually.
Plain Language Summary
Machine learning (ML) and process-based methods are two approaches to
hydrologic modeling. Process-based hydrologic models (PBHMs) represent
the hydrologic cycle by solving equations which have been developed from
physical theory or experimentation, while ML models make predictions
based on patterns learned from large amounts of data. A particular
sub-field of machine learning called deep learning (DL) has been shown
to often outperform process-based models. However, current DL models do
not represent all aspects of the hydrologic cycle (such as streamflow,
evaporation, groundwater storage, and snowpack) at once, as is often
done in PBHMs. As a result, DL models in hydrology are often single
purpose, while PBHMs can be used for many different scientific and/or
engineering purposes.
We show how individual DL models that simulate evaporation and
convective heat transport at the land surface can be incorporated into a
PBHM. We show that deep learning simulated evaporation and convective
heat transport better than the PBHM. We also show how the incorporation
of deep learning into process-based models can further improve the DL
model itself. We conclude that taking advantage of both modeling
perspectives is better than either on its own.
1 Introduction
The debates amongst the hydrologic modeling community about the use and
utility of machine learning (ML) to simulate hydrologic processes
indicate that much work remains to be done to understand the role and
potential of machine learning in hydrologic modeling (Nearing et al.,
2020; Shen, 2018). While it is true that deep learning (DL) models have
shown great promise and superior performance in many cases it is yet
unclear how to make models that are both composable (that is, easy to
combine with other models) and transferable for scientific studies (that
is, the same model configuration can be used to explore disparate
scientific questions). In this paper we outline an approach for coupling
DL models of individual processes into existing hydrologic modeling
frameworks. This coupling approach allows us to represent individual
physical processes within a larger model using ML methods and to
introduce feedbacks between model components. The ability to couple
model components will address these composability and transferability
questions, as well as allow use of these types of machine-learned models
in areas which do not have readily available training data.
There are several reasons for the rapid advancement of ML-based
approaches in hydrology (and other fields), including a greater
abundance of publicly available data, increased computational resources,
and better frameworks for selecting, fitting, and applying models. Along
with this increase in interest, the community has also begun to think
about how to incorporate aspects of physical theory into these data
driven models. This desire for physics-based machine learning is
enticing for a number of reasons. As scientists we hope that the use of
models which are based in, or constrained by, physical properties will
allow us to learn about the underlying processes of the systems we are
modeling. Not only that, we hope that such approaches will be able to
efficiently extract information from a variety of datasets, from in situ
observations to satellite remote sensing data, or be able to represent
complex phenomena in a more efficient way.
While inclusion of empirical or statistical relationships of individual
processes in hydrologic models is common, this is not yet the case for
ML methods. One reason for this is that it is not clear how to combine
ML models in the same way that we have been able to include processes
for which we have parsimonious descriptions. Additionally, methodologies
for representing physical relationships between ML-based process
representations have not been developed in the hydrology community. In
part, this is not surprising since machine learning is good at resolving
relationships that we have not been able to decompose into easily
describable parts. This “whole-system” or “black box” approach is
conceptually appealing due to its simplicity, and is exemplified by
rainfall-runoff modeling, which deep learning has proven to be very good
at (Hu et al., 2018; Kratzert et al., 2018; Moshe et al., 2020).
However, by taking a more granular approach, we will show that DL models
can be successfully incorporated as process modules into existing
models. Doing so allows us to see how changes in a single component
affect the entire system.
In this paper, we look at turbulent heat fluxes, for which high-quality,
long-term, local observations from eddy covariance towers (here, from
FluxNet; Pastorello et al., 2020) are available across a range of
hydroclimates. While machine learning has been used for modeling of
turbulent heat fluxes and evaporation (Jung et al., 2009; Tramontana et
al., 2016; Zhao et al., 2019) there have not yet been model
intercomparisons with land surface models, much less integrations into
land surface models. However, Best et al. (2015) showed that even simple
statistical models are often able to outperform state of the art land
surface models in simulation of latent and sensible heat fluxes. Best et
al. (2015) postulated that the statistical models were better able to
use the information in the meteorological forcing data than the
physics-based approaches. This indicates there is strong motivation for
incorporating data-driven techniques into complex land surface and
hydrologic models. We believe that if these types of approaches are able
to provide better performance than the physically motivated
relationships we should work to understand how and why this performance
is better and use them where appropriate and applicable.
Despite the statistical benchmarks’ superior ability for predicting
turbulent heat fluxes in Best et al. (2015), land surface models remain
more suitable for a wide range of applications, because they represent a
wider range of hydrologic processes and may be better suited for studies
of environmental change. Such studies include drought prediction (Li et
al., 2012), snow melt predictions under climate change (Musselman et
al., 2017), and predicting volatile organic compound emissions (Lathière
et al., 2006). That is not to say that ML models cannot be used in this
way or incorporated into larger frameworks. Both Kratzert et al. (2018)
and Jiang et al. (2020) make qualitative comparisons of internal ML
model states to snowpack, but do not later use the models for prediction
of snowpack. We believe that it is likely that ML models will be used
for such purposes in the near future, but the question remains open how
to extract process information from statistical models.
Because the hydrology community is still learning the best ways to build
and use ML models, there remains considerable room for incorporation of
machine learning into more conventional process-based hydrologic models
(PBHMs), which have the flexibility needed for general purpose modeling.
This approach has been adopted recently by Brenowitz & Bretherton
(2018) as well as Rasp et al. (2018) for parameterizing sub-gridcell
scale processes, such as cloud convection, in atmospheric circulation
models. Similarly, in oceanography, neural networks have been used to
parameterize the turbulent vertical mixing in the ocean surface
(Ramadhan et al., 2020).
In this study, we demonstrate how coupling ML models into a hydrologic
model can yield better performance at estimating turbulent heat fluxes
without sacrificing mass and energy balance closure or the ability to
represent other processes such as runoff or snowpack. We have developed
two ML models to simulate latent and sensible heat fluxes. We embed
these ML models as process parameterizations inside of a PBHM. These
ML-based process parameterizations replace the turbulent heat flux
equations of the original PBHM. Our first model was only allowed to
learn from the same meteorological data that is used to force the
hydrologic model, while our second ML model is additionally trained with
the inclusion of states derived from the hydrologic model. We show that
both ML models are able to outperform the routines for simulating
turbulent heat fluxes at subdaily timescales. We also show that the
configuration which was trained using model states is better able to
reproduce the long-term water balance. Our results indicate that
approaches to coupling machine learning with PBHMs offer a promising
avenue, which has only begun to be explored.
2 Materials and Methods
2.1 Data and study sites
We used data from 60 FluxNet sites (Pastorello et al., 2020) to run our
experiments. These sites cover a large variety of vegetation and climate
classifications. Our site selection process considered several criteria.
We first filtered the full FluxNet dataset to make sure we only included
sites which had energy balance corrected measurements of both sensible
and latent heat fluxes, which will be discussed later. We then made sure
that these sites had the necessary variables to force our models, which
include precipitation, air temperature, incoming shortwave radiation,
incoming longwave radiation, specific humidity, air pressure, and wind
speed. We then removed sites which had either fewer than three years of
contiguous data or more than 20% missing observations during the
longest continuous period with observations. For the remaining sites, we
used gap-filled data provided as part of the FluxNet dataset.
Gap-filling was based on ERA-Interim (ERAI) (Dee et al., 2011) and
includes downscaling and postprocessing explicitly for the purpose of
model forcing. Time steps flagged as gap-filled were excluded from our
performance analysis to ensure that we did not simply measure the
ability of our simulations to model ERAI data. However, the gap-filled
data is included when analyzing the water balance.
We also limited our analysis to sites which had an observed ET/P ratio
of less than 1.1, calculated using the mean FluxNet-reported values of
ET and P over the simulation period. This was done to accommodate our
model structure, which enforces mass and energy balances on a point (or
lumped) scale. Larger observed ET/P ratios likely occur at sites which
have strong spatial gradients and flow convergence, so that moisture
available for ET is not just the result of local precipitation. Our
filtering process resulted in 60 sites with 508 site-years of data. A
breakdown of the site names, data periods, locations and site
characteristics are given in Table 1. Figure 1 shows the locations and
vegetation classes for these same sites.
Table 1. A listing of the sites, locations, IGBP vegetation
types, and dates of simulation. Locations are given as (Latitude (°N),
Longitude (°E)). Vegetation types are given by their IGBP codes. MF is
mixed forest, ENF is evergreen needleleaf forest, CRL is croplands, GRL
is grasslands, SVN is savannas, OSL is open shrublands, WLD is permanent
wetlands, DBF is deciduous broadleaf forest, and WS is woody savannas.
Site names are taken from FluxNet, and consist of a two-letter country
code followed by a three-letter site code.