New aerial image dataset provides farmers with actionable insights

A dataset of large-scale aerial images produced by Intelinair, a spinout from the University of Illinois at Urbana-Champaign, aims to give farmers visibility into the conditions of their fields. The dataset, called Agriculture-Vision, will enable agricultural pattern analysis of aerial images, providing farmers with actionable insights into the performance of their crops to improve decision-making and maximize yields.

Until now, there has been a dearth of high-quality agricultural image datasets, due in part to the large image size required to capture many acres of land, as well as the difficulty of recognizing patterns that do not occur consistently across large areas. Researchers from Illinois and the University of Oregon worked with Intelinair to develop new computer vision techniques that solve complex pattern recognition problems through deep learning methods.

“Next-gen farming has to be data-driven,” said CSL’s Naira Hovakimyan, the W. Grafton and Lillian B. Wilkins Professor of Mechanical Science and Engineering at Illinois and co-founder and chief scientist of Intelinair. “By automating the process of frequent high-resolution data collection and using the data in predictive modeling through deep learning algorithms, we’re advancing to the stage where conditions on any farm can be forecasted in the same way as weather forecasts, for example. It is just one click away.”

Not since the mid-20th century, when scientists learned how to increase yields by manipulating crop genomes and the wide use of pesticides was introduced, has a new technology shown so much promise. AI is already being used to automate farming processes and collect data on field conditions. However, ag-related visual pattern recognition has progressed slowly, in part because of a lack of large-scale and high-quality datasets.

Hovakimyan says agricultural pattern analysis poses a unique challenge because it requires recognition of patterns that do not occur consistently and are hard to distinguish – such as weeds or waterways – across large areas. For example, discerning the difference between a dog and a cat is not as complicated as distinguishing wheat from ryegrass – a weed whose color and shape are similar to those of wheat, and that looks largely the same from the air.

Professor Thomas Huang, the Maybelle Leland Swanlund Endowed Chair Emeritus in Electrical and Computer Engineering, and Humphrey Shi, an Illinois alum in electrical and computer engineering who is now at the University of Oregon, in close collaboration with Hovakimyan, led a team of ECE student researchers to curate the dataset and proposed new solutions in semantic segmentation, which is the process of clustering parts of an image together (pixel by pixel) to the same object class. For Agriculture-Vision, agronomists determined the classes and annotated the images.

The Agriculture-Vision dataset paper was accepted by the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), the highest ranked conference among all publication venues in computer science and engineering according to Google Scholar Metrics. The team is also organizing a first Agriculture-Vision workshop at CVPR at Seattle in June 2020. It has attracted a myriad of attention from both agriculture and computer vision communities.

current Agriculture-Vision dataset includes close to a hundred thousand images from thousands of corn and soybean fields across several states in the Midwest. It includes annotations for conditions such as nutrient deficiencies, drydown, weed clusters, and more. Eventually, the researchers plan to expand the dataset to include different modalities, such as soil, topographic maps, and thermal images. They say that images captured season after season, year after year, could enable creation of deep learning models that help farmers plan not only for the next season, but also for the long-term sustainable health of their soil.

Agriculture-Vision’s abilities complement the offerings of Intelinair, which provides crop intelligence via its AGMRI solution for growers, agronomists, ag retailers, and other players in the ag ecosystem. Corporate partners include Deere & Co., a Fortune 100 ag manufacturer who utilizes Intelinair’s products in its Operations Center product, and the Climate Corporation, which has integrated Intelinair’s products in its FieldView service.

“We are excited to lead the research front for agricultural pattern analysis by creating this dataset, but there is so much more we are exploring, by incorporating accurate labels and annotations, farm history, soil conditions, and crop dynamics and integrating these into deep learning models for next-gen farming intelligence,” Hovakimyan said. “We are just at the beginning of what we can do.”