Articles
Combining 3D structural features and multimodal fusion data to correct illumination effect of plant multispectral point clouds
Article number
1360_1
Pages
1 – 14
Language
English
Abstract
Plant spectral response and three-dimensional (3D) topological relationships can be obtained by analyzing the plant morphological structure and physiological information contained in the 3D multispectral point cloud of plants through close-range imaging.
This information can be used to monitor plant growth dynamics, evaluate plant agronomic traits and provide reference indicators, as well as reveal the interactions among plant phenotypes, genotypes and environmental factors.
Currently there is no commercially available and mature sensor which can directly obtain multispectral point cloud data of plants by emitting different wavelengths of laser light.
In this study, we proposed a novel method for obtaining single-frame multispectral point clouds of plants based on multimodal image registration.
To address the differences in plant multispectral reflectance images caused by illumination inhomogeneity and light intensity difference reflected by plant leaves with different leaf positions, we proposed a method of plant spectral image reflectance correction combining 3D structural features and artificial neural network (ANN) models.
The results showed that compared with some classical image registration algorithms tested in this study, the proposed joint registration algorithm had a better registration performance for plant RGB images and multispectral images acquired by different sensors, with an average structural similarity (SSIM) of 0.931, which was better than the average registration performance of other classical algorithms (SSIM=0.889). The whiteboard reflectance distribution at different positions and orientations was simulated with high accuracy using ANN, where R-square (R2) and root mean squared error (RMSE) was 0.962 and 0.036, respectively.
Compared with the ground truth, the average RMSE of the spectra before and after correction at different leaf positions decreased from 0.182 to 0.040, a decrease of 78.0%; for the same leaf position, the average value of the range (Max-Min) of Euclidean distances between pairs of spectral reflectance curves measured from multiple viewing angles decreased from 0.140 to 0.055, a decrease of 60.7%. The proposed method of combining 3D structure features for correction of multispectral images in this study had good optimization performance for proximally acquired plant multispectral images.
This information can be used to monitor plant growth dynamics, evaluate plant agronomic traits and provide reference indicators, as well as reveal the interactions among plant phenotypes, genotypes and environmental factors.
Currently there is no commercially available and mature sensor which can directly obtain multispectral point cloud data of plants by emitting different wavelengths of laser light.
In this study, we proposed a novel method for obtaining single-frame multispectral point clouds of plants based on multimodal image registration.
To address the differences in plant multispectral reflectance images caused by illumination inhomogeneity and light intensity difference reflected by plant leaves with different leaf positions, we proposed a method of plant spectral image reflectance correction combining 3D structural features and artificial neural network (ANN) models.
The results showed that compared with some classical image registration algorithms tested in this study, the proposed joint registration algorithm had a better registration performance for plant RGB images and multispectral images acquired by different sensors, with an average structural similarity (SSIM) of 0.931, which was better than the average registration performance of other classical algorithms (SSIM=0.889). The whiteboard reflectance distribution at different positions and orientations was simulated with high accuracy using ANN, where R-square (R2) and root mean squared error (RMSE) was 0.962 and 0.036, respectively.
Compared with the ground truth, the average RMSE of the spectra before and after correction at different leaf positions decreased from 0.182 to 0.040, a decrease of 78.0%; for the same leaf position, the average value of the range (Max-Min) of Euclidean distances between pairs of spectral reflectance curves measured from multiple viewing angles decreased from 0.140 to 0.055, a decrease of 60.7%. The proposed method of combining 3D structure features for correction of multispectral images in this study had good optimization performance for proximally acquired plant multispectral images.
Authors
Pengyao Xie, Ruiming Du, Zhihong Ma, Haiyan Cen
Keywords
close-range imaging, multimodal image registration, plant multispectral point cloud, reflectance correction, illumination effect, artificial neural network
Groups involved
- Division Precision Horticulture and Engineering
- Division Plant-Environment Interactions in Field Systems
- Division Vine and Berry Fruits
- Division Tropical and Subtropical Fruit and Nuts
- Division Plant Genetic Resources, Breeding and Biotechnology
- Division Temperate Tree Nuts
- Division Temperate Tree Fruits
- Division Vegetables, Roots and Tubers
- Division Sustaining Horticulture in a Changing World
- Working Group Mechanization, Digitization, Sensing and Robotics
Online Articles (45)