I never really understood what certification means until I heard it described by a psychometrician. I have been a Certified Crop Advisor (CCA) for nearly two decades and felt I knew what it means to be a certified professional. I recently attended the North American Certified Crop Advisors On-line Board Meeting. I listened to a presentation by Scott Thayn, Ph.D., CMS, a psychometrician, or a statistician specializing in distinguishing the differences between individuals. Thayn is the president of Certification Management Services, the third-party agency hired by the Agronomy Society to help develop and manage the testing required for certification. The presentation addressed a proposal to give rankings on how well a test taker did on the CCA exam. Board members were hearing from potential members who were unable to pass the exams and wanted more feedback to help them study for their next attempt. The proposal, and the way the statistician took it apart, were a revelation, and it got me thinking that the mechanics behind certification are not well understood.
Certification Exam Intricacies
The Home page for Certifications under the website Agronomy.org states, “Certification is the standard by which professionals are judged. The purpose of a certification program is to protect the public and the profession. It is a voluntary enhancement to a person’s career credentials. Being certified adds credibility and shows that you are serious about what you do.”
A prospective candidate digging deeper would find they need to meet certain criteria to be considered a CCA: academic, experience and examination. Simply speaking, certification indicates one has demonstrated the knowledge and experience to perform at a higher level than their peers.
Hearing the proposal to give test takers feedback on their performance was familiar to me as a board member who participates on the Exam Committee for the Western Region. I have heard from many colleagues who did not pass one or both certification exams and are frustrated by the lack of a score or indication where they underperformed. I struggled to explain to my friends why the exams were pass/fail and why they just had to keep trying. I believe the frustration lies in the expectations of an academic testing experience clashing with the reality of certification exams.
Data indicates most people who take the certification exams are recent college graduates. Having a college degree in agriculture is a requirement for becoming a certified crop advisor. College graduates have spent most of their lives with graded exams. Academic testing presents a broad range of questions to both examine a student’s proficiency and encourage them to improve. A student who gets a low grade on a test will hopefully review the questions marked incorrect and study the subject to raise their grade on the final exam. This familiar approach to testing is contrary to certification exams.
The distribution of difficulty of certification exam questions is quite narrow compared to an academic exam (See Figure 1). The certification exam begins by defining competency areas, the major subjects that define the everyday work of the crop advisor. Performance objectives rest under the competency areas. Each performance objective spawns several possible exam questions. Each exam question must be tied to a performance objective to accurately test one’s comprehensive knowledge of agronomy.
Where an academic exam contains a large variation in question difficulty, certification exam questions ask, “What is the minimum knowledge a professional must have to be proficient in this area.” This is determined by groups of volunteer CCAs, with guidance by the Agronomy Society’s excellent statistician Dawn Gibas, Ph.D., who reviews the performance of each exam question. Questions that nearly everyone gets right are eliminated as well as those that almost no one answers correctly. A complete exam review process takes place every four to five years.
An illustration of the difference between academic and certification exams can be given with a sports analogy. An academic exam is comparable to a high school physical education track and field program, where everyone is expected and encouraged to participate. A certification exam, on the other hand, is like the selection process for the Olympic high jumping team. The high school physical education program sets the bar low and gradually raises it to help students practice their technique and jump higher. But when the world competition is jumping over seven feet, the US team would set the bar at a level near that to select the most competitive team. During the selection process, if the bar is set too high, then they don’t have a team, but set the bar too low and the team has a poor chance of winning. When the psychometrician used this example, the proposal to classify the specific abilities of test takers was withdrawn.
The complexity of 21st-century agriculture practiced in the Western Region of the US supports the need for the most qualified field people providing the best recommendations for our growers so we can continue to deliver the highest-quality, safest agricultural products in the world.
