Introduction
Producers – whether calf raisers, dairy farmers or cattle producers – are usually cautious about new technology. Before investing in a new technology (for the sake of this discussion, we’ll discuss a new feed product), most producers like to try that technology on the farm to see if they can “see” a positive response of some kind on their own operation. These on-farm trials are usually the last part of the sales process and a positive on-farm trial will usually mean that a producer will begin using the product.
On farm trial success
In my experience, I’ve seen hundreds of on-farm trials conducted by calf raisers, dairy farmers, and other animal producers. Unfortunately, many on-farm trials fail to show the expected results. I don’t have any statistics regarding success rates for on-farm trials so my estimate of less than 50% success rate is only my guess.
Why do so many on-farm trials fail? Well, there are several possibilities to consider. Here are some of them:
- The product doesn’t work. This is the first conclusion for many producers who try a product on their operations that doesn’t show a positive response. They are unable to see a response on their farm and assume that the product won’t work on ANY farm. Undoubtedly, there are products that are poorly conceived, poorly formulated or with terrible quality controls so that the product won’t work. My own pet peeve is companies that take a good, functional product and incorporate it into a product at such a low rate that it has no chance of working. However, there are many good products that don’t show a response on specific farms under specific conditions. I’ve done several studies wherein I’ve fed calves a milk replacer containing antibiotics and saw no significant growth or health benefit. Now, antibiotics in milk replacers are a well accepted technology and have shown in many studies to improve efficiency and growth of young milk-fed calves. So, if they didn’t work in my studies, does that mean that they never work? Not at all. There are other possible reasons, outlined below.
- The product you received was defective. Companies doing on-farm trials should be aware that results of their studies – even on-farm studies – will become public knowledge and available to many producers. So, they should be careful to ensure the product you’re testing will work. However, I’ve seen several occasions that the product tested was defective in some way. One common example is when producers have tested probiotic products. Since these products contain live microorganisms, it’s important to handle the product to protect organism viability. In a recent test with which I am familiar, a producer tested a yeast product in his pelleted calf starter. This yeast was not heat stable and none of the yeast cells were alive after pelleting.
It’s a good idea to document the active component of the product you’re testing. The company you’re working with should be able to provide you with a Certificate of Analysis – a document that shows the important components of the lot of product being tested. If you’re going to test a product, a C of A is an essential part of the trial and you should request one prior to starting the project. - You didn’t apply the product properly on the farm. The way products are handled, stored, and fed on the farm has a huge effect on the success of a trial. Some issues are fairly simple – e.g., adding probiotic bacteria to waste milk (which contained significant amounts of antibiotics) will usually mean that the probiotic bacteria will not be effective. Others may be more complex – storage conditions and or mixing problems with low inclusion ingredients, for example.
Consistency of administration is essential. During the planning process, it’s important to ensure that ingredients are properly sourced, that they arrive on the farm on time (I’ve seen expensive trials fail because the test farm ran out of test product), that the product is mixed properly and delivered so that all test animals have access to the product. Consistency every day during the trial is essential. - You couldn’t see the response. This is the most common problem with on-farm trials. Your inability to see a response to a test product may be that there is too much variability in your test measurement (e.g., change in body weight, number of veterinary treatments per calf) for you to see a difference between groups. That doesn’t mean that the product doesn’t work – it simply means that you couldn’t measure the effect because there is a lot of variability on your farm.
In previous Calf Notes (#127 and 128) we discussed the issue of variability – sometimes referred to as “noise” in a trial. Consider how you are going to measure the success or failure of a trial. What parameter are you going to measure? Can you measure it accurately? Can you control the other factors that will affect it? Will factors out of your control (e.g., weather, changes in feeds, delivery schedules) affect the parameter? Do you have good records? If you answer “no” to any of these questions, you should reconsider whether you can conduct an on-farm trial.
Some items to consider
- Consider the need for a test. Many salespeople offer producers the opportunity to do an on-farm trial, assuming that the chance to “see and feel” the product on the farm will ease the purchasing decision. It’s simply as part of the overall sales process. Many know that even if the trial doesn’t show definitive results, it’s easier for a producer to commit to a product they’ve tried already. Something like the test drive of your new car…
If the technology that you’re considering is well established scientifically (e.g., pasteurizing waste milk), maybe it’s not really necessary to do a trial of your own. Remember, you will make a significant investment in time and resources to conduct the trial. - Plan carefully. If you want to simply try the product on all of your animals, this is great. It doesn’t, however, constitute an on-farm trial. A properly conducted trial will require you to split your animals into two or more groups (or individuals) – at least one of which will be fed the test product and another group that will be fed without the test product. Making sure that all of the proper statistical rules are followed (e.g., randomization) and you have enough “replicates” is critical to making sure you will be able to identify a difference at the end of the trial if one exists.
Proper planning of how you’ll get, store, mix, transport and feed the test product will be important. Also, how you’ll measure the results on the farm. How long will you need to apply the test product? To how many animals? All of these details should be worked out in a written plan before the on-farm trial gets started. - Watch the “noise” on the farm. Understanding the concept of variation – “noise” – is essential to making an on-farm trial work properly. Here’s a great example. In a previous situation, I worked with a dairy farmer who had tried a feed additive on his operation. The product literature indicated that he should be able to get a 3 lb increase in milk production per cow per day by feeding this feed additive. So, he decided to do an on-farm trial by blending the product to his ration for all cows. He didn’t have the time or facilities that would allow him to split the herd into two groups and feed only one the product. So, he fed all of his cows and tested total her milk production. After a month on the trial, he couldn’t tell a big difference in the bulk tank shipments (his criterion for success). So, he considered the product to be a big failure. Unfortunately, he didn’t take into consideration that his overall herd days in milk was getting longer because fewer cows were calving. He also switched silos during the trial and there was a run of bad weather – all of which affected the bulk tank. Does that mean that the product didn’t work? Nobody is sure. There was just too much noise during the trial.
Beware those savvy folks who ask you to do on-farm trials at a time when things are going to go well – in spring, for example, when the weather gets good, when calf mortality declines, growth rates go up, etc. I’ve seen more than one example where a trial was scheduled and organized at a particular time of the year to take advantage of positive changes taking place on the farm – and giving credit of those changes to the test product.
A written plan for conducting the experiment can help you identify the sources of noise and take steps (to the best of your ability) to control them. I believe taking the time to write out a plan for the on-farm trial is the best bet to ensuring a successful trial.
Summary
Getting a handle on new technologies that come to the market requires some understanding how products work on the farm. However, on-farm trials can be much more complex than initially meets the eye. If you’re interested in a trial, you should consider the amount of effort involved, plan carefully and have a great relationship with the company you’re working with. Best of luck!