Season 1, Episode 4: What to Expect from Plant Nutrient Sensors (in English)

(00:00:00) Patty Skinkis

This is the High-Res Vineyard Nutrition Podcast, devoted to helping the grape and wine industry understand more about how to monitor and manage vineyard health through grapevine nutrition research. I am your host, Dr. Patty Skinkis, Professor and Viticulture Extension Specialist at Oregon State University.

(00:00:23) Patty Skinkis

Managing vineyards requires a keen eye towards plant development and how vines are responding to their environment. While many viticulturists enjoy walking a vineyard and checking on its status, it’s difficult to keep up with plant growth, development and issues as they arise, particularly when there are many acres involved. This is where precision farming comes in. Technologies have been developed to help researchers and farmers scan fields for a better, more complete picture of what’s going on and where within the crop land those problems exist. However, most of the applications of precision farming have been adopted by agronomic crops on a large scale and not widely used for specialty crops like grapes. Furthermore, the perennial nature of vineyards makes it difficult to pinpoint causes of plant health issues due to the compounding effect of seasons. To learn more about how precision agriculture tools and sensors are developed, we have Dr. Jan Van Art with us today to talk about his research on developing tools for precision vineyard nutrition. Dr. Van Art is professor at the Rochester Institute of Technology’s Chester F. Carlson Center for Imaging Science and is part of the Digital Imaging and Remote Sensing Group. Prior to joining RIT, he was a research group leader at the Council for Scientific and Industrial Research in South Africa. He received a Master’s and PhD degrees in forestry from Virginia Tech, right here in the U.S., and his Bachelor’s of Science degree in Forestry from the University of Stellenbosch. Much of Dr. Von Art’s research focuses on remote or proximal sensing of plant canopies and forests, production agriculture and ecosystems, but specifically, he works on sensing vegetation for chemistry composition. This is precisely why he was chosen to be one of our team members in the engineering team of the High-Res Vineyard Nutrition Project.

(00:02:12) Jan Van Aardt

Thanks for having me, Patty.

(00:002:14) Patty Skinkis

Welcome. Today I hope we can cover a number of questions related to just the sensors and the tools that are being used in vineyards, particularly knowing that there’s a lot of apprehension with midsize and smaller scale producers that always look at it as a technology that may be out of reach. Perhaps we can shed some light on what those tools are and how they might be more approachable as a technology for growers. I’m going to start out with the main question I have: What are the most important sensors for plant health monitoring used by growers today? This could be on any crop, agricultural or forestry. What are those sensors?

(00:03:02) Jan Van Aardt

That’s a great question, Patty. It’s been really fun working with Markus and the team on this project. I think one of the key aspects of our engineering group within the larger project team is to actually identify the ideal sensors, especially for vineyard nutrient monitoring. The current sensors, there are many I would say operational sensors, but one that really comes to mind is John Deere’s “Green Seeker;” that’s a sensor mounted in an approximal sensing set up right by the tractor where it’s moving forward and actually scanning across the crop and then relaying whatever sensing information back to a nutrient spreader in the back, etc. I think it’s really important to realize that that’s a very simplistic sensor, actually. It senses blue, green, red and infrared, so only four bands. That’s where we’re really diving into this project is to identify more ideal bands and different spectral ranges or color ranges, if you will, that can be applied either from a remote platform or an in situ or proximal platform, like a tractor. I think that’s part of this whole question, what does the ideal sensor look like not only from a results perspective in other words getting accurate and precise in other words, repetitive measures of vineyard nutrition. But ones that are truly operational, as you said, ones that a farmer or a service provider can actually operationally implement at a cost-effective basis, and that’s a key aspect of our project.

(00:04:28) Patty Skinkis

What kind of sensor do you use specifically for your research and are these extensible to operational use?

(00:04:37) Jan Van Aardt

The quick answer is we use research grade instrumentation; and no, they’re not really extensible to operational use, the reasons being we overdo this. We sample too many color wavelengths, too many spectral bands. Our sensor itself costs around $60,000 to $80,000. The whole drone platform touches around $150,000. I grew up in a farm and I know my dad certainly would not have invested in a $150,000 drone to fly across these crops. So, our goal really is to find that Goldilocks solution to sort of overdo this. We’re collecting too much data to identify exactly which data, which wavelength we need for a specific application. We go to farmer events, to outreach events and we fly this research-grade drone and it’s very impressive taking off. It sounds like a hive of bees when it takes off, but as soon as it starts flying I could see farmers lose interest because it flies so slow that they could immediately sense that it will not be able to cover their farm in a reasonable amount of time. But then we fly the dumbed-down version of this drone and it flies fast, it can cover say 100 acres in half an hour and suddenly that sparks their interest. You have to realize that we’re talking about an order of magnitude cost difference here. You’re talking about a sub $10,000 drone on the one end, the flying fast drone, and then the $150,000 drone on the other end. So that’s very key to our approach here, to identify those Goldilocks solutions that are not only accurate and precise but then can be cost effectively applied and in all sorts of operational environments.

(00:06:23) Patty Skinkis

So basically, throw everything at the research question and see what rises to the top in the kitchen sink.

(00:06:30) Jan Van Aardt

Exactly. It’s not only that, it’s also drones have become the buzzword, right? Everybody thinks it’s this cool, really useful technology, but it’s not necessarily that the solution will be drone-based. We’re very much open to solutions that could be, as you said, proximal sensing that could actually be an extension of this sort of John Deere approach, the “Green Seeker,” but maybe adding more specific wavelengths for specific nutrients. We’re trying to find the ideal solution, not the one that we’re trying to dictate from a research perspective.

(00:07:03) Patty Skinkis

So, this project, of course, is looking at vineyard nutrition. There’s a number of different macro and micronutrients that we are interested in managing and it depends on where you are and what type of grape you’re growing, as to what nutrient would be of concern. Do you think it’s really possible to remotely sense all of these mineral nutrients?

(00:07:25) Jan Van Aardt

We obviously focus on these primary nutrients. N, P, K, those are our key focus areas, but we’re also very much acknowledging the micronutrients and their importance in a lot of crop growing. The quick answer to that is no, I don’t think it’s possible to sense all these nutrients, and that’s the honest answer. We have to realize that anything we sense from the sky or even in close proximity to a plant, we’re assuming that it’s detectable through sensing but it might not be. I’ll give you an example: if we detect through the “Green Seeker” app on John Deere’s tractor, if we detect a plant that’s growing sub-optimally, we assume during the nitrogen or the fertilization application phase that more nitrogen will solve the problem, but that sub-optimal growth could be due to shallow soil, could be due to other nutrient stress. It could be due to disease. So, there’s this assumption that we attach to a lot of the sensing. To be specific, some of these nutrients are very specific, what we call absorption features, and you can think of them as a color signature. Some of these absorption features are so fine, so detailed that we just cannot resolve them. I sort of equate this to those connected dots things that we did in coloring books when we were kids. These days the kids don’t play with that anymore, right? But if I give you an elephant and you have 100 dots to connect the elephant with versus 1,000 dots, those 1,000 dots are going to be really detailed, very finely resolved elephant versus the 100 dot is going to be sort of a lumpy, unresolved elephant. Much of these nutrient seeking is relying on the idea that we can detect these finely defined nutrient absorption features and some nutrient absorption features are just so detailed that we cannot resolve them. So that’s part of what we’re looking at is which we can and cannot resolve, and we’re very honest about the answers. We develop the models and we say, “This model is only accurate to this level.” Then, I think it’s up to a grower to decide, is that accuracy and precision level acceptable? Or would they rather apply traditional leaf sampling methods to detect boron, for instance?

(00:09:41) Patty Skinkis

Great. I think you hit the nail on the head with the expectation of what technology can do. It’s not going to necessarily give us that same detail that a tissue sample that’s being collected in field, ground up and analyzed with analytical procedures at the tissue lab. That’s going to have to be something that’s kind of taking place of that. But, in place of that is more efficiency and getting more collection of data.

(00:10:11) Jan Van Aardt

Exactly. We actually have another project with Cornell where we’re looking at vineyard irrigation and some researchers at Cornell have developed this tensiometer that you implant in the cambium and you can measure the stem water potential of the vines. Now, speaking to that researcher, I ask him, “Why would we fly drones if you can implant the sensor all across the vineyard?” His quick answer was, “Well, we’re only sub-sampling, right? It gets expensive, so you can only plant this in, say for arguments sake, 20 vines per acre.” They really want that wall-to-wall irrigation map. So that’s where the drone comes in. Maybe part of the solution would be exactly like you said, this sort of institute, lab-based sampling that we use to calibrate some of the drone sensing with. I think this is going to be in some nutrient instances is going to be a coupled solution, almost.

(00:11:02) Patty Skinkis

Absolutely, using different integrated nutrient management. We always talk about IPM but now we could coin the integrated nutrient management, INM. What do you think is the best avenue, and we’ve touched on this a little bit, but what are the best avenues to consider for vineyard nutrient monitoring and why?

(00:11:25) Jan Van Aardt

I think, long-term, we’re going to be using what’s called “multi-spectral” systems, and those could be drone-based and they could even one day be satellite-based. “Multi-spectral” essentially means a limited set of wavelengths or limited set of color bands. Our research instrumentation or “hyper-spectral” sensors have hundreds of color bands, from the visible through the near-infrared to what’s called the shortwave infrared. Now again, we collect these hundreds of bands so we can identify, let’s say, five bands for a specific nutrient. I sort of have this pipe dream that I’m not alone in this, I think, or unique in that, but if we can identify wavelengths per nutrient, just as an example, we can then put pressure on the sensing industry, the manufacturing industry to produce these kinds of sensors for specific nutrient scenarios. For example, if a grower or a service provider could say, “Listen, I’m going to fly for nitrogen and phosphorus and potassium today,” that would imply that I’m going to take my drone with five wavelengths and first fly for nitrogen, develop a nitrogen map, then sort of replace the filters in front of the lenses with the different set, five different filters that are specific to potassium, and then fly the vineyard again and then get a very specific potassium solution. I’m sort of thinking that we’re going to have to be a little bit adaptive to the nutrient-sensing solution. We cannot just use a “one shoe fits all” type approach where you just fly one drone with one set of wavelengths and hope for the best solution. It’s possible, but the sacrifice there will be eventually that you will lose precision, accuracy and some of the nutrients. If you optimize for Nitrogen, it’s sort of inevitable that your Potassium and estimates might be sub-par. So, that’s one thing to look at is that we will have to get a more defined set of wavelengths that are operationally feasible.

Then, secondly, we’re very much open as a research team, especially the engineering group within this larger project, to say “What about proximal sensing?” Is it sensible to develop handheld tools that you could sub-sample in the vineyard and get an approximate nutrient level or status for your vineyard, even location specific, and then apply nutrients accordingly? Could be much cheaper, but at the cost of sacrificing that wall-to-wall type map that you would get from a drone.

(00:14:00) Patty Skinkis

I think that leads to my next question with sampling. Right now, growers do very minimal sampling with either soil or a paired tissue sampling, and obviously it’s not wall-to-wall – they try to represent their vineyard. But, with some of these tools, how should we do sampling? Should it be a few locations within a vineyard or complete, wall-to-wall? What would be the benefit and the costs to doing this?

(00:14:30) Jan Van Aardt

You know, it’s a very good question. It’s a difficult one to answer but, sort of taking away from a little bit of the forestry background, I always tell students that if you need to estimate volume in, say, a pine forest stand, and it’s from the same clonal material, the soils are extremely homogeneous, it has the same microclimate across the stand. Guess what? If you know how at what density you plant the trees you need to sample one tree. That’s it. So, it all depends on the variability in your stand. It all depends on the variability in your vineyard, and if there’s one thing we’ve learned quickly: farmers know their farm. They know their growing environment. I could tell a farmer, “Oh, we’ve sensed a spot in your vineyard that there’s sub-optimal growth in the farms.” “Oh yes, I know that’s actually a shallow soil or a water-depressed area, whatever,” and they know about these things. So, I think we’re adding cherry on the cake-type information just for a farmer to really dial in that specific nutrient application, saving a lot of money that way, as well, and optimizing yield and quality in grapes. So, I think ultimately, we would have to rely on a lot of the in-situ knowledge of the farmer, related to a specific vineyard and say, I know this is a variable vineyard. I’ve seen variable production rates across the vineyard itself. I’ve seen variable nitrogen soil sample results. I would suggest that you would say up the number of samples when you fly this vineyard, in situ samples that you use to calibrate the drone model with. The easy answer to this is that it all will depend on the variability within that vineyard.

I personally think we will probably never get away completely from field samples. I hope that we can get away from it to the extent that it becomes way less of a cost burden on the grower and that we only use it sort of routinely or every other year, maybe even just to double-check the model that we’re replying that’s drone-based. But just trying to verify the results to see if there’s not something that has changed in the meantime.

(00:16:36) Patty Skinkis

Sure, and maybe even only sampling trouble areas and where we know the background where everything looks normal. It’s just kind of sampling those abnormalities. So, what would be required of a farmer or a service provider to map vineyard level nutrients, if they wanted to do it? Is there something they could do and get off the shelf today? Or is it still down the road a bit?

(00:17:03) Jan Van Aardt

There are some off-the-shelf products. There’s a company called MicaSense. They developed, for instance, a camera that has five, what we call, silicon cameras on their platform. Now, silicon is exactly the same sensing tool that you have in your cell phone camera. It is sensitive from say, blue-visible all through near-infrared [light spectrum], so all you need to do is if you have five silicon cameras like your cell phone camera you just apply specific filters to each of those cameras and you can sense a very specific wavelength. That’s what MicaSense has done. They’ve developed a camera with five, or a platform, rather, with five cameras, each with a different filter and those filters span blue, green, red, near-infrared which is very important for vegetation nutrient sensing, and then something called the “red edge.” So, the “red edge” is right between the red wavelengths and the near-infrared wavelengths; also, a very important region for vegetation nutrient sensing. So that camera can actually give you very good estimates of especially nitrogen. If you want to go into specifics of some of the other macro and micronutrients, that becomes a little trickier. You might have to sacrifice some, as I said, model quality. So, accuracy and precision quality as if you just apply those off-the-shelf wavelengths. And that’s where I’m hoping that as an industry, and even a larger agricultural industry, not only vineyards, we can put pressure on a company like MicaSense, in a good sense and capitalistic way and say, “Listen, there’s a market for this.” There’s a market for potassium sensing, but for us to do that as a grower or as a service provider we need you to adjust the cameras that you sell us. You can sell us a nitrogen camera or a potassium camera, or better yet, something that’s a single camera with different lenses that we can just swap out if we want to fly for specific nutrients. So, what I would has to happen first: we have to be able to make the use case to the camera industry, so to speak, the developmental industry so that they can actually see that use and I would say market in agricultural and especially the vineyard industry. Now, for a specific farmer, I think it depends on the interest of the farmer in drone technology. These solutions, in my opinion, will be sub $10,000 solutions and then plus the farmer’s time to go and fly. Many of these drones now are veering towards doing what we call “real-time calibration.” For lack of going into too much detail, but at the moment, if you’re a farmer or a grower or as a service provider looking at acquiring data, one of the key things is that you have to calibrate your data and what that means is it becomes robust against illumination changes. If I fly today and it’s cloudy outside my results should be similar to when I fly tomorrow and it’s sunny outside and what we do there is we convert what’s called reflectance. Reflectance should be if the vine hasn’t changed from day-to-day in illumination conditions. It should look the same, irrespective of those illumination conditions. Now, the good news is a lot of the work we do in imaging science and at other places are looking at building in into the sensor itself into the platform itself real-time calibration, so the farmer doesn’t have to worry about that anymore, or the service provider, whoever flies the drone. So that actually relaxes a huge burden of the person who does the data collection. Depending on that, you could have a farmer fly, upload the data to the cloud for instance, and then apply the nitrogen model and download the map for a specific vineyard and I would say possibly in as little as an hour, depending on bandwidth, etc. Or if you’re more interested in sort of a corporate type buy-in where a service provider provides this kind of service to an entire region, that could also be a viable business model for something like this.

(00:21:09) Patty Skinkis

When using some of these sensors, is there an ideal time that they should be used, from a plant growth stage? Right now, when we do nutrient testing we always do bloom or veraison, and that’s kind of where we encourage growers to stick because that’s where our recommendations come in--our guidelines. With the sensing, obviously it would be ground truth with some of those traditional time points if we’re trying to map it to the traditional methods and our content of our tissues. But do you see that perhaps this would be a tool that could be used at any time, or is there a critical window of time that’s being focused on in the research?

(00:21:52) Jan Van Aardt

The critical time windows, in terms of these kinds of applications, to be honest, are more related to sort of daily patterns. It would be really important if you can, if you’re able to fly these kinds of sensors in, let’s say, not dusk environments or dawn environments but rather sort of let’s say midday in a perfect scenario when we fly our sensors for research purposes, for this and other projects, actually, we fly what we call plus minus two hours of solar noon. So, whenever solar noon is, one, that’s the ideal illumination scenario because then you have almost direct sunlight. Not a lot of shadows cast in the vineyard or the forest, so there’s that little restriction. We are looking at making these algorithms robust against that, because we know a farmer cannot be bound by too much by “When can I fly my drone?” because they’ve got a lot going on in their growing environment and their farms, so they might have an open hour at 9 am and then say, “I’m going to fly.” I realize this might not be the ideal situation, but hopefully this project will yield results that would say, listen, yes, you can fly at 9 am and you will still be able to get viable results that you can sort of hang your hat on and use for whatever fertilization regime you want to apply.

In terms of the seasonal timing, I think, to be honest, from an imaging perspective, we have to rely a lot on the viticulturists. We have to rely a lot on the folks who know the plant physiology better than we do, and if bloom and veraison are the best times to sense, then that’s when we sense. But, to answer you straight up, we can fly anytime you want. If we determine that we need to fly earlier in the season, that’s open. It’s all going to be I guess weather dependent and unless it’s raining cats and dogs, we can fly, you know?

(00:23:43) Patty Skinkis

Well, that’s great. So, you’ve talked a bit about the technologies, but I want to take this opportunity to just have you tell us a little bit about your specific research in the High-Res Vineyard Nutrition Project. What are the short-term and long-term goals of your lab’s work?

(00:24:01) Jan Van Aardt

We are very much a group that’s focused on imaging science. Although my background is forestry, I’ve worked in remote sensing as they call it, both drone aircraft and satellite-based remote sensing for more than probably two decades by now. I’m getting old, I guess. The specific goals have become much less academic-based and much more operationally oriented. We learned through another project that was NSF-funded. It was actually a disaster response project, believe it or not, and one of the key fundamentals of that project was to, say, a person responding to a disaster, what do they want? How quickly do they want it? As opposed to us in academic scenarios telling them, “This is what we think the ideal solution should look like.” We really put our whole approach on its head and say, “No, you as an operational implementer of disaster products, you tell us what should it look like, when you need it by and what is your tolerance for error?” We learned, for instance, in the disaster response community a good product today is better than a perfect product a week from now, and that sort of makes sense, right? So, what is that tolerance from a grower perspective? What is that tolerance for precision, for accuracy and precision? Are they willing to have a 10% error in our nutrient estimation because that still gives them a good spatial distribution understanding of, for instance, nitrogen needs? Or does it have to be within 2%? So, a lot of what we do now is actually defining how we can squeeze that very expensive sensor with a lot of wavelengths down to something that’s truly operational, truly cost effective. And that when you now fly it at a field day for an ag extension, folks say, “Now, that I can buy into,” as opposed to saying, as you said earlier in the podcast, “It’s still out of reach, it’s still unattainable technology.” So that’s a lot of what we focus on, not only in our group here in Imaging Science at RIT, but also as a project team group with folks at UC Davis, folks at Washington State University and Virginia Tech, and especially Cornell. We work very closely with Terry Bates and Justine Vanden Heuvel, who are the true experts in vineyard growth and physiology. We work very closely with them to better understand what the needs of the growers are out there. I have a post-doc on the project there, Dr. Rob Chancia. He’s working especially with Terry Bates at Cornell to say, if we develop these nutrient maps, can Terry then put them in the hands of a grower, in say, a web-based application? And I was surprised, to be honest, at the coarseness of that web-based application. Yet, that’s exactly what the grower needs, and a grower doesn’t necessarily need a nutrient estimate for every single vine. If you get sort of a spatial distribution of the nitrogen needs, etc., then that’s actually extremely useful already. You know, it’s this huge step forward from where we are now. So that’s what we’re focusing on is moving away from an academic approach where we just publish another paper that sits and gathers dust on the shelf to something that can be really useful and operationally implementable, so to speak.

(00:27:31) Patty Skinkis

Well, that’s fantastic. I think that’s exactly what needs to happen because as you know that some of this sensing has been around for quite some time and it’s still not in the hands of a lot of producers. So, it’s great to see this work being done. So, in addition I wanted to ask you – this is a little bit of a deviation from the main questions--but what is your favorite host plant to work with? You’ve worked with different crops and you’ve worked with forests so, what is your absolute favorite plant?

(00:28:02) Jan Van Aardt

Oh, that’s a good question. It’s a trick question because I like wine, so the easy answer is actually vineyards. I actually like the vineyard environment, and I’m not saying this because this is a vineyard-specific project. I like it because it’s a capital-intensive environment. It’s a capital-intensive growing industry, and that means that growers, as far as I can see, are actually amenable to the idea of high-end technology, so it makes for a potential segue for these research drones into something that could potentially be used operation. We work across snap bean also with Cornell. Dr. Sarah Pethybridge and pathology for disease detection. We work with corn quite a bit in defining yield stability zones with Dr. Corine Kettering, also Cornell, so I really enjoy the precision ag avenues that are opening up. But, I found that there’s still a long way to go for the large-area crops like corn to move to a drone-based solution. Most of those, I personally think, will eventually sort of segue into satellite-based solutions because they’re amenable to coarser resolutions, bigger pixels, more infrequent revisit times. But my understanding and now I sort of engaged with the industry today is that in the vineyard growing environment, there’s the need to be rapidly responsive. I mean, take irrigation as an example. So, there’s the need for rapid response solutions. You cannot wait for the satellite to pass over in ten days’ time, you need to know today what kind of drip irrigation rescheduling you need to apply. So, I think that’s part of what I really enjoy. It’s extremely challenging, but I enjoy the fact that the needs are there, and the potential for adoption is there because the market is more amenable or accepting of this idea of high-end type technologies, if that makes any sense. I’m not saying that just because it’s this project, I really do enjoy that and red wine.

(00:30:11) Patty Skinkis

I totally understand. I came from the dairy industry and then when going to grad school I almost was considering vegetables, tree fruits or grapes and I decided grapes sounded more challenging and thus, more interesting. So, I can totally relate to that. Thank you so much for sharing your project and the information with us today.

If you’re interested in learning more about Jan’s work and the High-Res Vineyard Nutrition Project, see our website at https://highresvineyardnutrition.com/.