Detection of Marked Lettuce and Tomato by an Intelligent Cultivator

Figure 1. Plant labels in tray of tomato seedlings prior to transplanting. The labels and tomato plants were transplanted together in the field.

Weeds are difficult to control in lettuce and tomato due to labor shortages, increasing costs of hand weeding and limited herbicide options. Lettuce is very sensitive to weed competition, plus there is no tolerance for contamination of bagged lettuce salad mixes with weeds; therefore, weeds must be controlled if lettuce is to be harvested.

Consequently, mechanical weed control is an important part of an integrated weed management program in conventional and organic vegetable crops. Traditional inter-row cultivation, however, only removes weeds between crop rows and leaves the weeds within the crop row. The removal of in-row weeds requires hand weeding, a time-consuming and expensive process.


Vegetable Weed Control Costs

Weed control costs for conventional head lettuce production in California are estimated at $216 to $319 per acre, while weed control costs in organic leaf lettuce are $489 per acre, on average, at current labor rates. In conventional processing tomatoes, weed control costs are about $225 per acre or 12% of production costs. Additionally, hand weeding costs have increased due to labor shortages, changes in California overtime regulations and increasing minimum wages as well as decreased labor immigration from Mexico. The result is greater vulnerability of growers to crop losses due to weeds.

Automation of weed removal may be a method to contain or reduce weed control costs in vegetable crops. Intelligent intra-row cultivators (IC) provide an alternate weed management option to standard inter-row cultivation. Previous results have shown that IC can reduce the need for hand weeding compared to standard cultivators and may reduce weed control costs.

The Robovator® cultivator evaluated by Lati et al. (2016) relied on pattern recognition of the rows and crop plants within the rows based on the expected crop spacing within the rows. When these spatial cues are unavailable, as can occur in an organic field with a high weed density, this approach cannot differentiate between crops and weeds, and thus it relies on a size difference between crops and weeds, as well as a low to moderate weed population to function accurately.

Intelligent Cultivation. Intelligent intra-row cultivation requires three technologies; a machine-vision system that detects crop plants and weeds, image classification and decision algorithm that differentiates between crop plants and weeds, and an automated weed removal mechanism that controls the weed while protecting the crop. Precision guidance systems, decision algorithms, and precision in-row weed control devices are commercially available or are at an advanced level of development. Accurate crop detection and differentiation from weeds, at normal cultivation speeds, would allow for greatly improved intra-row cultivators.

Weed/Crop Differentiation. The main challenge for intelligent intra-row cultivation is to differentiate between crops and weeds using digital imagery and processing at field operation speeds of at least 1 mph in high weed density fields with travel speeds above 2 mph required for economic acceptability for low to moderate weed loads.

A new method of crop and weed differentiation called “crop signaling” is presented in the research “Crop Signaling for Automated Weed/Crop Differentiation and Mechanized Weed Control in Vegetable Crops” by Raja et al. 2019 out of UC Davis. It is based on the idea that the identity of the crop is known with certainty when it is planted, whether transplanted or seeded. Thus, if the crop has a marker or signal that an IC can reliably detect, then the IC would recognize the signal and protect the crop. Plants without the signal, i.e., weeds, would not be protected and would be removed by the IC. The objective of this work was to test a crop signaling system for crop detection accuracy and weed control efficacy by an IC in lettuce and tomato.

Marking System Descriptions. Two methods of plant signaling were tested, physical plant markers and topical markers. Biodegradable straws coated with a fluorescent marker were used as the plant markers in this study (Figure 1). The straws were then placed next to tomato seedlings in the planting trays and then transplanted together (Figure 2).

Figure 2. Holland transplanter with butterfly transfer fingers used for transplanting plant labels and tomatoes together.

The topical marker used on plant foliage was green or orange fluorescent water-based paint (Figure 3a,b). A paint sprayer was used to apply the topical marker to lettuce foliage and tomato seedlings prior to planting, while they were in trays. Another method was to spray the marker onto tomato stems as they were transplanted (Figure 4).

Figure 3. (left) Topical marker on lettuce plants, (right) Spray application of topical marker on crop plants.
Figure 4. Topical marker sprayed on tomato transplants by applicator mounted on the transplanter during the process of transplanting.

Intelligent Cultivator. The IC used in this research was developed at the University of California, Davis. It uses a machine vision system specifically designed to detect the physical labels and topical markers on the crop (Figures 5&6). Weed control was done by mechanical knives, which the IC opens (Figure 6b) to avoid the marked crop plants and closes (Figure 6a) to uproot weeds in the intra-row space.

Figure 5. Image of a tomato plant with a green label taken (a) under normal light plus UV light, and (b) under UV light only. Note the reflections of the green label in the six mirrors, and the actual label in the center of the image.
Figure 6. The actuator device used in this project: (a) Weed knives closed – uprooting weeds in crop row, (b) Weed knives open avoiding tomato plant.


Field Trials

Eight field trials in tomato at Davis, Calif., and six in lettuce at Salinas, Calif., were conducted during 2016-2018.

Tomato. Field trials in processing tomatoes were located on a silt loam soil on the UC Davis vegetable field crops research station near Davis. The tomatoes were seeded in trays and kept in a greenhouse for 45 to 60 days until they were about 10 inches tall. Tomatoes were transplanted into 60-inch beds at 15-inch spacing in a single center row. Two tomato trials were carried to yield.  Plant labels were added to seedling trays prior to transplanting (Figure 1) or the topical marker was applied to trays of tomato seedlings as described above (Figure 4). Tomato transplants were marked with paint 4 inches above the soil line. About three weeks after planting, all plots were cultivated with a standard mechanical cultivator which only removed weeds outside the plant line. The standard cultivator left a 7-inch non-cultivated band centered on the crop row.

Weed densities by species were measured before and after cultivation in four 7-inch-wide (centered on crop row) by 20-foot-long sample areas randomly placed along the length of the plots. The time required by a laborer to hand weed the 20-foot areas was recorded. Two tomato trials were maintained until harvest so that marketable yield data could be collected.

Lettuce. Field trials using Romaine lettuce were conducted in a sandy loam soil at the USDA research station in Salinas, Calif. Four weeks after seeding, the whole experiment was cultivated with a standard mechanical cultivator. The standard cultivator left a 6-inch non-cultivated band centered on the crop row (Figure 7). The IC operated within .75 inches of the lettuce plants on all sides. Pre-cultivation weed counts were measured the day before cultivation and post-cultivation weed counts were taken the day after cultivation. Weed densities were measured in a 6-inch band centered on the crop row in each of two 20 -foot-long samples in the field. Weeds that were uprooted were considered dead. After cultivation, hand weeding was performed and timed as described for the tomato trials. The time spent by a laborer to hand weed with a hoe was recorded.

Figure 7. The plant layout used in the lettuce plantings: (a) Single crop row of lettuce on 1 m beds. The control rows are with no crop signal visible, (b) physical labels in lettuce row two weeks after transplanting.

The 2017 lettuce trials were maintained until commercial maturity and number of marketable heads and weight of marketable heads were recorded. The 2018 trial was conducted at a commercial lettuce field near Salinas, Calif.

Statistical Analysis. RStudio Version 1.1.383 was used for statistical analysis. Differences between pre- and post-cultivation weed counts determined weed removal effectiveness. The most efficacious treatments removed the greatest proportion of weeds.

The difference in weed densities between pre and post cultivation were analyzed using analysis of co-variance, to measure the effect of cultivator type on weed density. Analysis of variance (ANOVA) was performed on the hand-weeding time data to measure the effect of the cultivators.  Weights were determined for both lettuce and tomato yields, and in lettuce, the number of heads was also determined.

Weed Control. The IC was more effective than the standard cultivator at removing weeds from the inter-row space. The data were pooled separately for tomato and lettuce. In tomato seed lines, 1 weed per square foot remained after IC while 10.5 weeds per square foot remained after standard cultivation. This is a 90% reduction in the number of weeds remaining after cultivation (P<0.05).  In the lettuce trials, 1.7 weeds per square foot remained in the seed line after intelligent cultivation while 5 weeds per square foot remained after standard cultivation, which is a 66% reduction in weeds remaining after cultivation (see Table 1).

Table 1: Effect of cultivator type on in-row weed densities after cultivation, time to hand weed and marketable yield in tomatoes and lettuce.

Handweeding in the tomato trials required 7.8 hours/A following the IC while the standard cultivator required 14.9 hours/A which is a 48% reduction (P<0.05). Hand weeding of lettuce required 16 hours/A following cultivation the IC while 29 hours/A was required for the standard cultivator, a 45% reduction in time (P<0.05).

The time-spent hand weeding after IC cultivation was a notably smaller percentage reduction than it was for weed densities, i.e. 48% vs. 90% in tomato. This is because the IC consistently removes the readily accessible weeds that are more than an inch from the crop; while the remaining weeds after IC cultivation are typically close to the crop plants and take more time for the field crew to remove than weeds further from the crop plant. The IC did not remove all the weeds it passed over due to some algorithmic uncertainty in the precise location of the crop’s main root and a risk-averse control strategy. Thus, weed control in close proximity to crop plants may still require some hand weeding. However, significant reductions in manual labor were achieved while maintaining effective weed control.

Crop Yields. There was no difference between the cultivators in their effect on tomato fruit yield in 2017 (P>0.05) (Table 1).  The 2018 tomato yields had marketable fruit yields in the IC and standard cultivator treatments of 44,045 and 50,217 lbs./A, respectively (P>0.05). Similarly, there were no differences between the cultivators in their effect on lettuce yields (P>0.05) (Table 1). Yield data were analyzed both as the number of marketable lettuce heads per acre and fresh weights.  

Weed/Crop Differentiation.  One of the biggest challenges for automated intra-row cultivation is to enable a computer and vision system to differentiate between crops and weeds at normal field travel speeds. The commercially available IC ‘Robovator®’ uses pattern recognition to recognize the crop row and can perform intra-row weeding at speeds of 1 mph (Lati et al. 2016). However, this requires a distinct crop pattern best found such as in a transplanted field where the crop is much larger than the weeds and the crop stand is consistent. Further, when high weed densities obscure the 2-dimensional crop row pattern, the intra-row weeding program does not work.

Two types of crop signals were tested, physical plant labels and topical markers. The methods have very low false positive error rates and the classification accuracy achieved for both techniques approaches 100%. The crop signaling technique appears to be effective in creating a reliable method for automatic detection of crop plants in vegetable fields with high weed densities. Crop signaling technology could facilitate development of automated weed control robots that are as accurate in crop/weed differentiation as human workers are.

A recommendation for future work is to develop a commercially viable marking method that is machine readable, yet does not contaminate harvested produce or the field soil and subsequent rotational crops. For transplanted stem crops like tomato, a biodegradable machine-readable tag attached to each stem as the transplanter sets the plants should be explored for commercial potential. Lettuce will probably require a machine-readable label attached to the first true leaves or a machine-readable label on the fiber-coated plant plug as it is set in the soil as is done with the Plant Tape® ( system of vegetable transplanting.

Regardless of the technology used for crop weed differentiation, development of intelligent weed removal technology has improved weed control programs for horticultural crops that continue to rely on a limited number of herbicides and hand weeding. However, there is much more to do to improve vegetable weed control.

Acknowledgments. Thanks to the USDA Institute of Food and Agriculture Specialty Crop Research Initiative (USDA-NIFA-SCRI-004530) the California Tomato Research Institute and the California Leafy Greens Research Program for financial support.



Lati, R.N., M.C. Siemens, J.S. Rachuy, and S.A. Fennimore. 2016. Intra-row Weed Removal in Broccoli and Transplanted Lettuce with an Intelligent Cultivator. Weed Technology 30:655-663

Raja R, Slaughter DC, Fennimore SA, Nguyen TT, Vuong V, Sinha N, Tourte L, Smith RF, Siemens MC (2019) Crop signaling: a novel crop recognition technique for robotic weed control.  Biosystems Engineering 187:278-291.

Steven A. Fennimore
UCCE Extension Specialist at UC Davis
David C. Slaughter
Biological and Agricultural Engineering Dept. at UC Davis