Advances in automated image analysis using open-source computer vision tools, such as PlantCV, have greatly increased the throughput of aboveground phenotyping in a variety of crop species. However, PlantCV was largely optimized to analyze images collected under controlled laboratory conditions, and has seldom been used to analyze images collected under field conditions. Further, there are no known applications of PlantCV for analyzing images collected belowground, such as those obtained from minirhizotron imaging devices. In this study, we demonstrated applications of PlantCV for extracting plant trait information from aboveground and belowground images collected in two perennial crop mapping populations. The first population was composed of nearly 1,200 individuals of a potential perennial oilseed crop (Silphium Integrifolium x Perfoliatum ), and the second population was composed of nearly 1,700 individuals of a perennial cover crop (Trifolium ambiguum , Kura Clover). We designed and used a field-based imaging cart to collect overhead and profile images of individuals from both populations in August and October, which improved the efficiency of field-based image capture. Around the time of aboveground image collection, belowground images of root networks were collected using minirhizotron imaging devices. We then assessed the application of PlantCV for measuring aboveground traits (crop canopy area, height, leaf color and growth rates) and belowground traits (root length and growth rates), and we explored future directions of PlantCV for field-based image analysis of aboveground and belowground crop tissues.

Phenotype characterization is an exciting new field that links agriculture and data science. Thanks to advances in remote sensing and artificial intelligence, we can now accurately quantify field-scale phenotypic information and integrate this big data into predictive and prescriptive management tools.Our past work has shown that phenotypic information measured from UAS is more precise and reliable than manual field measurements. In addition, we have shown that the vast size of this data can outweigh the complexity of the problem, to the point where even a simpler algorithm can outperform a sophisticated algorithm when big enough training data are available. This is because the big data captures even very infrequent aspects of the problem of interest, which can be modeled with simple logics.The availability of UAS data makes it possible to develop digital twin models to forecast future plant growth and develop in-season management plans. A digital twin model is a virtual representation of real-world entities and processes. These models use early growth patterns of a crop as input to artificial intelligence algorithms so that the algorithms can predict crop performance during the next 10, 20, or 30 days ahead of the last data point collected by the UAS. Crop growth features forecasting results can be useful for the management of irrigation, growth regulators, maturity of the crops, and to obtain early-season yield estimation.

Image segmentation is commonly used to estimate the location and shape of plants and their external structures. Segmentation masks are then used to localize landmarks of interest and compute other geometric features that correspond to the plant’s phenotype. Despite its prevalence, segmentation-based approaches are laborious (requiring extensive annotation to train), and error-prone (derived geometric features are sensitive to instance mask integrity). Here we present a segmentation-free approach which leverages deep learning-based landmark detection and grouping, also known as pose estimation. We use a tool originally developed for animal motion capture called SLEAP (Social LEAP Estimates Animal Poses) to automate the detection of distinct morphological landmarks on plant roots. Using high-throughput phenotyping method Root Architecture 3-D Imaging Cylinder (RADICYL) across multiple species, we show that our approach can reliably and efficiently recover root system topology at greater accuracy, faster speed, and with fewer annotated samples than segmentation-based approaches. In order to make use of this landmark-based representation for root phenotyping, we developed a Python library (sleap-roots ) for trait extraction directly comparable to existing segmentation-based analysis software. We show that landmark-derived root traits are highly accurate and can be used for common downstream tasks including genotype classification and unsupervised trait mapping. Altogether, this work establishes the validity and advantages of pose estimation-based plant phenotyping. To facilitate adoption of this easy-to-use tool and to encourage further development, we make sleap-roots ,  all training data, models, and trait extraction code available at: https://github.com/talmolab/sleap-roots.

Nitrogen fertilizers are one of the top expenses for corn farmers in North America, and the highest cost of any input over a growing season. The success of developing better strategies for nitrogen-use efficiency, such as improved varieties and biologics, depends on an efficient way of measuring in-planta nitrogen content. We are testing a reflectance-based UAV hyperspectral approach and a transmittance-based hyperspectral handheld called LeafSpec to estimate corn nitrogen from hyperspectral images. The UAV approach is high throughput but with relatively low spatial resolution and has high susceptibility to environmental factors. The LeafSpec is low throughput but with high spatial resolution and is not significantly affected by environmental factors. Also, the LeafSpec can account for the nitrogen inside the corn leaves by using transmittance, whereas the UAV can only see surface effects from reflectance. We are testing well-established machine learning and state-of-the-art deep learning models for both approaches. This presentation will share our learnings from testing UAV and LeafSpec as two hyperspectral imaging approaches to estimate corn plant nitrogen from hyperspectral images regarding potential use-cases for each approach, considering precision, ease of use, throughput, and potential for further development.

Root-root interactions significantly impact the formation of architectural root phenotypes, yet are poorly understood. Phenotype formation is impacted by sensing of soil resources and exudates of neighboring plants (Nord et al., 2011; Wang et al., 2021), which motivates the need to accurately quantify this phenomenon into its underlying causes. Currently, we are developing a complete experimental system for root-root interactions. A mesh frame has been designed to support the growth of two mature plant root systems. The frame is inserted into a large mesocosm, filled with a sand/soil mixture, and two plants are grown. To harvest, the mesocosm is disassembled and the sand/soil is gently washed away. Root systems are left suspended in the mesh and using a Canon EOS Rebel T5, ~500 total photos are taken at 10 different angles ranging from below to above the roots, 360° around the frame. DIRT/3D is used to construct 3D models and extract data from individual root systems. We are in the process of improving our data extraction methods to include spatial traits relative to the two root systems. To do so, we dye the root systems right before harvesting. The difference in coloration allows the use of a deep learning model based on U-net architecture to perform image segmentation and separate roots in the 3D models. Our next step is to run a larger experiment with 10 mesh frames. This will provide statistical significance for trait identification and adaptation of DIRT/3D for root-root interaction data extraction and analysis.

The flowering date of sunflowers is a crucial trait that significantly influences crop management practices and product placement. This trait can be measured by counting the number of days from planting until 50% of plants in a given research plot have reached flowering at R5 developmental growth stage. Traditional ground methods for data collection are labor-intensive and subjective, requiring field scientists to manually estimate and record plots with 50% flowering every 1-2 days. This approach not only consumes considerable time but also potentially overlooks valuable information related to flowering rates and duration. To address these challenges, we leveraged UAVs (Unmanned Aerial Vehicles), which allow for surveying a field in a short span of time, to model flower counts over time and predict the date when the plot reached 50% flowering. The method developed employs a deep learning model trained to detect yellow sunflower heads from UAV imagery and modeling these counts over time using a logistic function to estimate the 50% flowering date. With this method, flowering date was precisely estimated with high correlation relative to the ground measurements (r = >0.92) across experiments and locations. An increase in heritability for the remote sensing trait was also observed relative to the ground trait, and more importantly we were able to gain additional insights into flowering rates and duration. This innovative approach offers a promising avenue for enhancing the efficiency and accuracy of sunflower phenotyping.

Corn (Zea Mays ) is one of the most valuable row crops grown in the United States. Thus, securing and increasing the yield of corn is necessary for the progression of society. However, blindly applying excessive nitrogen fertilizer to reach higher yield has resulted in disastrous consequences. Therefore, it is crucial to precisely monitor the nitrogen content in the corn plant and provide suggestions on nitrogen application is crucial to increase the sustainability of agriculture.Currently, spectroscopy imaging has been proved effective in measuring nitrogen content in corn plants. However, the trade-off between throughput and data quality is rarely addressed and it has become an obstacle to scale up the application of spectroscopy imaging. Hence, in this study, we have developed a new robotic system to operate a proximal imaging device to capture a spectroscopy image of a single corn leaf. The robot has 3-DoF, including one rotational and two translational actuators. One 3-D camera works as the perception system to detect and localize the targeted corn leaf. The manipulator is a uniquely designed spectroscopy imaging device that captures spectroscopy images with low-noise and high-resolution. Once deployed, this robot could capture high quality images for a single leaf within 30 seconds and free human from laborious field work.

Fusarium head blight (FHB) is an economically important disease in wheat which can cause yield losses >50%. Breeding for host resistance is the most effective control method, however time, labor, and human subjectivity limit phenotyping efforts. A novel, high-throughput phenotyping rover was used to collect in-field RGB images of inoculated wheat spikes at multiple time points in 2021 and 2022. A deep neural network pipeline was developed to classify wheat spikes, segment healthy and diseased tissue, and quantify FHB severity as the region of intersection between spike and disease masks. To validate the pipeline, model inferences on a plot and spike scale were compared to five raters who performed disease scoring in the field and on images. The precision and throughput of the phenotyping rover and FHB quantification pipeline exceeded conventional rating methods. The plot aggregate disease scores based on pipeline outputs correlated strongly with plot-level disease scores by raters in the field and imagery. When comparing disease annotations on spike images, pipeline to human disease correlations were equivalent to correlations between raters, however location tended to influence disease assessment. The pipeline has strong generalizability and performed well on images taken across environments, with different camera orientations, and throughout disease progression. These results demonstrate a breakthrough in FHB phenotyping and facilitate precise and efficient disease quantification on spikes and plot aggregates across time and imaging conditions that are unachievable using conventional methods.

Wheat (Triticum aestivum L.) is one of the key staple crops worldwide. Even though future demand for wheat is estimated to increase by 6% by 2050, wheat production might drop by 30% due to climate change. The purpose of this research is to identify canopy architecture (light capture) and anatomical (stomatal) traits that significantly increase radiation use efficiency (RUE; dry weight biomass produced per unit radiation intercepted) and improve yield under high temperature and drought stress conditions due to the growing concern over food security. This research was conducted with five contrasting wheat genotypes in a new field-based high tunnel system with 4 treatments and three replications (control, heat stress, drought stress and heat x drought stress with 3 replications = 12 tunnels) in a randomized completed block design with stress applied at the heading stage. Canopy architecture was graded according to the visual scoring scale concerning UPOV and RUE was calculated in three growth stages. New techniques of high throughput imaging of stomatal number and size using a handheld digital microscope will also be presented.

In precision agriculture and plant biology, monitoring nutrient stress in crops is of paramount importance for ensuring optimal yield and resource utilization. However, nutrient stress phenotypes can be nuanced, subtle, and display in a variety of ways. We propose using Neural Radiance Fields (NeRFs) for the organized reconstruction of plant structures to observe the changes of plant structure and color under nutrient stress.Neural Radiance Fields, a cutting-edge technique in computer vision, leverage neural networks to model complex high-frequency geometry directly from 2D images, offering high-fidelity reconstructions. This methodology holds immense potential for plant imaging, as it allows for the creation of detailed and organized 3D models that can capture subtle alterations in plant morphology associated with nutrient stress responses.The proposed methodology involves the acquisition of high-resolution images of plants under different nutrient conditions. These images are inputted to the NeRFStudio Nerfacto implementation, a NeRF model that is a aggregation of many different existing models. A 3D reconstruction of the scene is outputted from the model and can be further reduced to a point cloud containing point locations, colors, and normals. Phenotypic traits are then calculated from the point clouds. The reconstructed plant models enable the quantitative analysis of morphological changes associated with nutrient stress. This includes alterations in leaf size, branching patterns, and overall plant geometry. The utilization of NeRFs allows for non-destructive monitoring, offering a significant advantage over traditional methods that may be labor-intensive or invasive.This research not only contributes to the field of precision agriculture but also presents a powerful tool for plant biologists to deepen their understanding of how nutrient stress impacts plant architecture. The insights gained from this approach have the potential to inform precision nutrient management strategies, leading to more sustainable and efficient agricultural practices.

Walnuts are the second most produced and consumed tree nut, with over 2.6 million metric tons produced in the 2022-23 harvest cycle alone. The United States is the second largest producer, accounting for 25% of the total global supply. Nonetheless, producers face an ever-growing demand in a more uncertain climate landscape, which requires effective and efficient walnut selection and breeding of new cultivars with increased kernel content and easy-to-open shells. Past and current efforts select for these traits using hand-held calipers and eye-based evaluations. Yet there is plenty of morphology that meets the eye but goes unmeasured, such as the volume of inner air or the convexity of the kernel. Here, we study the shape of walnut fruits based on X-ray CT (Computed Tomography) 3D reconstructions. We compute 49 different morphological phenotypes for 1264 individuals comprising 149 accessions. These phenotypes are complemented by traits of breeding interest such as ease of kernel removal and kernel weight. Through allometric relationships —relative growth of one tissue to another—, we identify possible biophysical constraints at play during development. We explore multiple correlations between all morphological and commercial traits, and identify which morphological traits can explain the most variability of commercial traits. We show that using only volume and thickness-based traits, especially inner air content, we can successfully encode several of the commercial traits.

PlantCV is an open-source open-development image analysis software package for plant phenotyping written in Python that has been actively developed since 2014. A new version of PlantCV was recently released. Major goals of the version 4 release were to 1) simplify the process of developing workflows by reducing the amount of coding needed; 2) broadening the set of supported data types; and 3) introducing interactive annotation tools that can be used directly in PlantCV workflow notebooks. Here we highlight the use of point annotations that can be used to quickly collect sets of points for parameterization of functions such as regions of interest or the identification of landmark points. Another application of point annotations this for image annotation, which is a major bottleneck in plant phenomics. For example, we have used point annotations to analyze microscopy images aimed at measurement of quinoa salt bladders, the number and size of stomata, and scoring of pollen germination. These tasks have traditionally been low throughput and have required manual scoring, but our point annotation tools can be used along with traditional segmentation methods to semi-automatically detect and annotate images. The PlantCV point annotation tools also allow users to correct semi-automated detection results before classification (e.g., germinated vs non-germinated pollen) and extraction of size & color traits per object. Once images are annotated, results can be analyzed directly or potentially can be used as labeled data in supervised learning methods.

Elizabeth Buescher1, Cody Hoerning1, Kassi Kosnicki1, and Marcus Jones1

As the world’s population grows and the demand for food rises, more attention has been paid to increase crop yields and enhance global food security. Modern remote sensing technologies enable us to capture spectral features (such as NDVI) of crop canopy, which are widely used to assess crop growth, health, and stress conditions. We noticed in the literature that crop NDVI shows short-term variation within a day. Therefore, in this study, we leveraged the Spidercam Field Phenotyping Facility at University of Nebraska-Lincoln to measure and quantify the diurnal variation of canopy NDVI for corn and soybean crops. The experiments and data collection were conducted in 2022 and 2023, with canopy reflectance measured by a spectrometer (400-1000 nm) at multiple days covering different growth stages. In each day, measurements were taken at multiple time points within a time window ±3 hours centered around solar noon. Our analysis showed a clear concave-shaped, diurnal trend in NDVI for both crops, with the lowest NDVI at solar noon. More analyses will be performed to quantify this diurnal pattern and dissect the sources of variation due to solar angle and change in canopy morphology. This research will further improve the accuracy and relevance of NDVI in plant phenotyping and many other scientific disciplines and applications.

Global warming poses a substantial threat to food security, necessitating the development of crops resilient to the adverse effects of stress including, drought, heat, and their combined impact. To understand maize responses to individual and combined stress conditions of drought and heat at early vegetative stages, we did a comprehensive evaluation of the phenotypic responses of 47 diverse maize inbred lines. The plants were stressed for 13 days beginning at 7 days after planting, with heat stress conditions of 38/28°C day/night cycles, and the drought condition was achieved by reducing the water pot volume from 88 to 40% over the course of the experiment. The Bellwether Phenotyping Facility, a high-throughput phenotyping platform, was used to capture daily RGB images of the plants throughout the experiment. We extracted morphological and color-related traits from the images, and particular interest was placed on traits that exhibited high broad-sense heritability throughout the experiment. This approach allowed for the collection of a robust dataset of phenotypic responses. Our results revealed distinct responses among the maize genotypes under different stress conditions, with the combined drought and heat treatment leading to the most severe impairments in plant height and leaf area. To gain further insights, we applied a time series analysis to the extracted traits and created groups with dynamically-similar response patterns. This research is foundational for understanding maize stress responses to heat and drought conditions and will inform future work towards identifying important genes and molecular mechanisms, particularly in the context of combined stressors.

Malting is the process of controlled germination of cereal grain that is steeped, germinated, and dry kilned to develop flavor compounds, fermentable sugars, the hydrolytic enzymes necessary for the brewing process. To meet the strict standards of the brewing industry, variation in the germination of barley grains need to be minimal but also maintain rapid germination and optimum levels of \(\alpha\) and \(\beta\) amylase. Preharvest sprouting poses a signinicant threat to barley cultivars prior to harvest, resulting in premature endosperm modification, reduced enzyme content, and poor malthouse pertormance. Pregerminated grain is ditticult to detect as there are often no signs of damage, and current methods such as the Hagberg falling/Stirring number, or pearling tests, each of which ultimately leads to the destruction of the seed. Here we used time-series hyperspectral imaging of barley seeds undergoing active and or pregerminated seed and used a deep neural network to predict stirring number and alpha amylase values, which are proxy measures for sprout damage. Prediction models were made for seven genotypes tested individually and with all genotypes combined. Stirring number assays ranged from 0 to 190 and our prediction models had mean average errors of 10.5 to 23.9 depending on the model. To increase the resolution and accuracy of the predictions we transitioned from using bulk conventional molecular assays to single seed assays and prediction models. These results demonstrate that hyperspectral imaging and machine learning models can be used to predict germinated grain in bulk and single seed assays.

Stomatal conductance (gs ) is a critical plant biophysical variable that reflects plant regulation of CO2 uptake and associated water loss, yet its direct measurement is often prohibitively time-consuming. Estimating the impacts of gs indirectly through leaf temperature (Tleaf ) is a common practice, but is complicated by confounding factors such as ambient conditions, measurement aggregation scale, sample size, and measurement time. Using Tleaf measurements to instead determine parameters of a model for gs that can remove these external factors can provide quasi-traits that are more reliable and heritable. Our objective was to develop an automated pipeline for gs model parameterization using thermal data, which could be applied within a 3D biophysical model to predict the impacts of trait variation on canopy-level processes related to water-use efficiency. Field experiments were conducted on common bean, cowpea, and sorghum crops, involving high-resolution thermal measurements obtained from a robotic sensing platform. Subsequently, a deep learning algorithm was trained using synthetic thermography data generated using Helios 3D model simulations encompassing canopy structure, ambient conditions, and Tleaf , enabling the prediction of long-wave radiation and incident shortwave radiation for each thermal image pixel. Following this, a leaf-surface energy budget analysis was applied to the collected field thermal data to predict gs parameters. Validation of these predictions was performed through comparisons with ground-truth leaf-level gas exchange data. This pipeline offers a promising pathway to predictive simulations of water status and transpiration-related traits, regardless of environmental variation, ultimately enhancing our understanding of plant responses to changing environmental conditions.

Cold tolerance is of paramount importance for the survival of winter wheat, as failure to withstand low winter temperatures necessitates costly replanting during the spring season. Initial observations indicate that higher cold tolerance is associated with increased winter dormancy, resulting in slower spring growth and potentially lower grain yields. Understanding how quickly the plants grow during this critical period can provide insights into their overall health and yield potential. This study investigates the interaction between cold tolerance, winter dormancy, rate of spring growth, and final grain yield to develop cultivars with high cold tolerance and early spring growth. The plant material consists of 480 diverse soft white winter wheat varieties collected from Pacific Northwest (PNW) breeding programs. The population was cultivated for three years (2016, 2017, and 2018) at two PNW locations (Pullman, WA and Pendelton, OR). RGB and NIR images were captured weekly to generate growth curves and calculate NDVI values for each genotype. Additional data on tiller count, heading date, plant height, grain volume weight, and grain yield were collected. Modeling NDVI regression curves enabled us to quantitatively define plant growth attributes including spring growth rate, time of peak greenness, dry-down rate, time of total dry-down, time from heading to total dry-down. These data, correlated with final grain yield, will enable a better understanding of winter wheat growth patterns. This information is crucial for breeding regionally specialized winter wheat varieties that can maximize spring growth and yield potential while minimizing the risk of cold damage.