Current requal testing for operators, RP, NL and Lic.

[ddm502001]

Curiosity brings me to start this discussion;

When did the need arise to go to 'Higher Order' questions with deviously designed wording to intentionally trip up the test taker instead of attempting to discern the true absorption and comprehension of the materials presented during requal training?  More and more the questions seem to be more aligned to how devious the tester can be or how intense the test can be made rather than test the subject and determine the capability of the student/operator.  Is this a developing trend or has my plant training just lagged behind the rest?

Why does the testing have to go this route, we lose enough candidates for employees to the POSS and TECH tests, now they seem to be screening out those that do not test or read tests well.  The industry is already hurting for employees, why make it even harder?

[Fermi2]

About 4 years ago, and to say higher order is designed to trip one up is BS. If you know the material you'll pass the exam.

Mike

[ddm502001]

Mike,

I wasn't stating that we did not know the material, it is how the test answers are 'reworded' or altered in the use of english alternative wordings to distract one.  If the need is to be distractive, why bother with the training?  Hi stress, hi anxiety, some are giving in to the pressure and leaving for other than nuke digs due to this, and we can ill afford the personnel losses today.  Quite a few of my plant's retirees cite this very situation as a determiner for their filing their papers early, a knowledge loss that cannot be replaced as they brought this unit online initially.

Dave

[kp88]

Generally, Training plays the "most accurate response" card when they know less than their students.

[Fermi2]

Generally the most accurate response is the correct answer, at least in my experience.

[JohnK87]

When did the need arise to go to 'Higher Order' questions with deviously designed wording to intentionally trip up the test taker instead of attempting to discern the true absorption and comprehension of the materials presented during requal training?  More and more the questions seem to be more aligned to how devious the tester can be or how intense the test can be made rather than test the subject and determine the capability of the student/operator.  Is this a developing trend or has my plant training just lagged behind the rest?

The NRC changed its requirements in the late 90's to drive this.  Having been the author of numerous license exams, the ideal question from the NRC's perspective would have ~80-90% of qualified people select the correct answer, with the wrong answers being evenly split.  The distractors need to be "plausible" which means there has to be a reason why it could be picked, i.e. common misconception, confusion of equipment, etc.  It takes a lot of effort (sometimes mistaken as deviousness or trickiness) to come up with 3 plausible distractors for every question.

The key to prevent 'trick questions' is rigorous validation by qualified individuals.  I have had many a time where a question has gone through three or more iterations before the operators would agree it was satisfactory AND a high enough percentage got it right.  The result is a test where someone can walk out thinking they bombed it yet score a 94 (which has happened, I just had to ignore the comments he made as he was walking past me post-exam).

The rules also changed so you limited the number of "log cognitive level" questions to no more than 50%.  These are simple recall, like setpoints or rules.  The rest have to be "high cognitive level" where you have to remember multiple things, predict or troubleshoot, etc.  These take more time to take, but with proper validation the results aren't much different than simple memory questions.

In the old days, you could memorize the answers or immediately throw out one or two distractors.  That doens't happen anymore.

[ddm502001]

Thanks JohnK, I couldn't get this level of info from my training department, or rather they dodged answering me to it.

Too bad they had to have it bumped another notch, the number of functional operators just went down at the least opportune point in the US nuclear industry history.  With an additional 12-25 units, maybe more, the demand for the quality individual just shot up and the availability of plausible candidates just went down, at least my pay will eventually show it!

We have lost a few good men at my plant in the last few years, this was a BIG contributor.  I too will retire early having to deal with the 'higher level' to maintain quals.