The following is from Bob Nielsen, Purdue University agronomist.
Analysis of "Big Data" promises to help farmers improve crop yields and/or profit, but its success relies heavily on the "quality" or "integrity" of the "Big Data" itself. Without accurate spatial data and the background details for each field each year, the upside potential for "data mining" diminishes greatly. Factors that impact or define the "quality" of agricultural "Big Data" are varied and many.
Simple bits of background information linked to individual fields or "zones" within fields (aka "metadata") are important for understanding or interpreting the spatial data collected each year. These metadata include dates of field operations (e.g., tillage, planting, spraying, harvest), hybrids/varieties planted, seeding rates and depth, fertilizer rates, pest problems, pesticide records, rainfall records, soil moisture records, temperature records, use of winter cover crops and soil map information.
If these metadata are incomplete or non-existent, subsequent "data mining" analyses of your "Big Data" becomes more challenging and may lead to erroneous decisions. This holds true even if you are analyzing your own spatial data using "off the shelf" commercial mapping/GIS software.
Take steps now to improve the way you track and document all of this background information. It would take several articles to discuss all the factors that influence the "quality" of spatially collected data (e.g., yield data, logged application data, crop sensor data).
What I want to emphasize here is simply the importance of yield monitor calibration. The value of accurate yield monitor data was never too important "back in the day" because most of us rarely used our yield data for anything other than printing color maps. With the advent of "Big Data" and "data mining" opportunities, the importance of accurate yield estimates is becoming clearer.
Understand this one simple fact about yield monitors: They do not measure yield.
Rather, yield monitor systems estimate grain yield by translating electrical signals generated by mass flow impact sensors or optical flow sensors located in the combine's clean grain elevator. These electrical signals are generated when grain flow physically hits an impact sensor or passes by an optical sensor and are then received, along with their respective geo-positions, by the yield monitor display.
The firmware/software of the display translates the electrical signals, using a mathematical algorithm, into estimates of wet (i.e., as harvested) grain flow (pounds per second or two of travel).
In conjunction with grain moisture estimates provided by the combine's grain moisture sensor, estimates of distance traveled based on changes in DGPS position or wheel sensors and the logged width of the combine header/platform, the yield monitor calculates and logs dry grain yield per acre for every second or two of travel in the field as individual geo-positioned data points based on the DGPS position.
Re-read the previous paragraph and ponder the multiple opportunities for errors to creep into the process of estimating dry grain yield with yield monitors. Add in external influences on yield monitor accuracy due to varietal grain characteristics, field topography, harvest speed, and there is a lot of room for error in this important spatial data set.
Now, consider whether the process of faithfully calibrating every aspect of the yield monitoring system every year prior to grain harvest, and possibly multiple times during the harvest season, might pay dividends down the road when analysis of "Big Data" becomes commonplace.
So ... Dig out that user's manual for your yield monitor system now.
Before the start of harvest, devote some quality time to reading the sections on yield monitor calibration. Create your own checklist of calibration steps and follow them faithfully every time you calibrate the monitor.
Do not forget the little things like vibration settings, header height stops, offset of the DGPS antenna, etc. Recognize that accurate weighing devices (grain carts, weigh wagons, farm scales) and moisture meters are crucial for the calibration of the monitor's wet weight estimates and adjusting the combine's moisture sensor readings.
And remember the old adage about computers ... Garbage in, garbage out.