4/6/2014 19:43

The Role and Limits of Objectivity

When I was a teacher, I always hated the process of grading students. The traditional methods all seemed too much like judging people. I have never flet that I was called to be a judge of anybody's personal worth. The solution I found as a teacher was to make the grade as truly objective as it could be. The course objectives were detailed, extensive, and clear: "Write a COBOL program using a simple PERFORM statement." "Name the four DIVISIONs of a COBOL program." "Write a program which reads a specific input file and prints specific data in a specific order." (It helped that I was teaching programming, but the concept is more broadly applicable.) The objectives in that course were, in fact, objective.

Now consider another set of criteria which seem to be objective. These criteria are for evaluating history projects entered into a contest.

One of the criteria is that the project is "historically accurate". That sounds objective, doesn't it? Either the project is or it is not accurate. But what do we accept as "accurate" and how do we apply that to a complete project? Any interesting historical subject is hotly debated. What was President Jackson's culpability in the Trail of Tears? Was the Civil War a fight over slavery or about states' rights? Or something else? Perhaps a better criterion would be that every assertion of fact is supported by reliable sources, although that raises the question of how to define a source as being reliable. Supposing that we reach a workable agreement on these issues, it remains unclear what proportion of the statements must be adjudged to be accurate in order to rate the project as "superior" or how many might be doubtful before being rated "needs improvement".

Another criterion is that the project "clearly relates to the theme" of the overall contest. That sounds objective as well, since the theme was given to all the participants. Any yet, to whom is this clarity visible? A person deeply familiar with the subject matter might easily discern a connection which a person off the street would not see. The audience is an essential part of the objective. The audience, however, is not defined. Perhaps the target audience is the contest judge, but the contestant does not know who that judge will be or what level of expertise (if any) the judge may have.

The last criterion I'll discuss is that the project "uses available primary sources". There is some ambiguity in whether that means the project uses all available sources or any of the primary sources, but fundamentally this criterion should be objective. You need only compare a list of primary sources available to the sources actually used in the project -- unless you do not have a list of available sources. In that case, any judgement is wholly and totally subjective: "I would guess that there might not be many primary sources", or "I would think that there must be more primary sources available for this topic."

Because of such considerations, I myself do not much like this particular contest. It is not as objective as it seems to be, and (much worse) it is not nearly as objective as I would prefer.

Yet it seems that some people appear truly to enjoy participating.

How could this be? Are they deceiving themselves and each other? Perhaps some people are deceived into participating, but the ones enjoying their participation probably are not deceived.

In the actual reality game different people choose different styles of play. My strategy moves me toward the objective and certain, but other people enjoy bingo. This history contest falls somewhere between; success in the contest depends both on the contestants' efforts and on the luck of drawing the right judges. There is strategy involved in winning, but there is also an element of chance.

I can legitimately complain only if such a contest is presented as being more objective than it truly is. Provided that there is no deceit, there is nothing illegitimate in this style of play.

On the other hand, I do not have to play this way.


Links