We’re quickly approaching the final stages of our first five new certifications, and I would like to share a few details from last week’s Section Leadership Update Meeting. We covered a lot of exciting ground and invited psychometrician Christine Depascale from our partner Professional Testing Inc. in Orlando, FL to give the group an update on the process from her perspective.
You’re probably thinking …psycho what? I had the same reaction. A psychometrician is an expert in developing exams to measure knowledge, skills and attributes. Christine’s company has been providing assessment, evaluation and certification services for 50 years to businesses, governments, and associations like ours, and I’m confident they are helping us create the highest quality exams possible.
In the meeting, Christine broke down the complicated certification creation process into four basic steps:
- Exam Analysis and Design – This is when exam developers conduct the Job Task Analysis, interviewing our IMSA members to break down each job into tasks, and then the specific knowledge, skills and tools needed to complete them. Once this process is complete the developers create an exam blueprint, which is used to communicate required test content with exam and curriculum developers, as well as IMSA members.
- Question Writing – IMSA subject matter experts (SMEs) work with psychometricians using the exam blueprint to create questions for the exam that are relevant, current, accurate and free of bias.
- Exam Form Development – Taking the finished questions, developers then create two versions of the test. Each test form will have 100 questions.
- Implementation and Evaluation – This is when we Beta test each exam, review and revise questions as necessary, and develop a standard score for passing the test.
While this is an oversimplified view of the process, I hope it gives you some insight into the amount of work that goes into the new certifications.
Right now, we’re in the middle of step two, and we’ll be finalizing the exam questions through the holidays. I encourage you to check out this video on the question-writing sessions we recently held in Orlando. I think it captures the enthusiasm our volunteers have for making IMSA a better organization.
It’s taken several months to get to this point, however, the next steps are going to move quickly. Once we have the final questions, we’ll load the tests into our IMSA online learning management system for Beta testing by the entire IMSA membership. People won’t be able to access scores, but the idea is to make sure that the questions hold up to scrutiny in the real world. If someone passes our certification exam, we need to be confident that it’s reasonable and legally defensible to say that this person has the right skills to do the job. So, once we get past the holidays, please keep an eye out for an email asking you to take the Beta tests because we’ll need as much input as possible to ensure each exam’s integrity.
While all this is going on for the first five exams, we’ve just begun our Job Task Analysis meetings for the next five certifications. They are:
- Traffic Signal Tech Level II Construction
- Traffic Signal Inspector
- Signs and Pavement Markings Technician Level II
- Transportation Center System Specialist Level I
- Traffic Signal Senior Field Technician Level III
Volunteers flew to Orlando from all around the country and Canada to begin laying out the foundations for these next certifications, and I can’t thank them enough. But we need MORE of you to get involved. One of the most difficult challenges we’ve faced so far is finding volunteers to help, and frankly, it has slowed down the entire process. l know people lead busy lives with jobs and families, but if you can help, please drop me a quick note. We’ll try to make it as easy as possible for your schedule, and some tasks can even be done remotely. One of the perks of participating is that you’ll receive Continuing Education Credits for your certifications, and sometimes, there’s even a gift card or two involved.
During our section leader meeting, we also addressed teaching materials for the exams. People are used to having manuals, and to make these certifications legally defensible and eligible for accreditation, teachers can’t help develop the new tests. While we don’t have the resources to deliver manuals at the same time we’re developing the tests, the Board and I are looking at a number of different options.
First and foremost, the Detail Design Documents (DDDs) have been approved by our subject matter experts and will be finalized by the IMSA Educational Advisory Council very soon. As you may remember, the DDDs spell out what specific topics students need to know for the exams, and how much time to reasonably spend on each topic of study. It’s all tied to the exam blueprint and will give our section moderators the appropriate substance they need to develop their own courses and materials. Sections are also free to work together to develop materials so that they’re not duplicating efforts.
Finally, I’ll leave you with a few thoughts from our IMSA Board President Kevin Musick. He wanted everyone to know that there were 14 new openings on IMSA committees, and I encourage you to reach out to Laura Jeffcoat Sosa at ljsosa@imsasafety.org if you’re interested in serving. These roles are extremely important to the function of our organization and to be honest, they support the lifesaving roles our members play in communities across cities, states and countries. As to the collective volunteer and section leadership work of our IMSA members, Kevin thanked everyone who took the time to move the organization forward, and he closed his segment of the meeting by saying, “I believe from this point on, it will be the best organization that it has been in the last 126 years.” I agree.
Toby