Are You Puzzled by the Judges? - inconsistencies in amateur photography contest judging - Column

Judging is on my mind today. Why? Have you entered a slide circuit lately and compared the scores that were awarded by the various panels of judges? I have and was amazed at the inconsistency and diversity of the scores.

Why is it that a slide may be awarded a medal or an HM by one panel of judges only to be rejected by another? Could it be that the judges are making subjective choices rather than exercising unbiased objectivity? Should not a slide that obeys the rules of composition, is in sharp focus, has good color, no distractions and contains an interesting subject be deserving of an acceptance or perhaps an HM or a medal? Most would agree that it should, yet the results suggest otherwise. Perhaps the variance is due to subject matter which may be of interest to one judge but not to another. Moreover, the variance becomes even greater if judges from other regions of the world are included. To illustrate this point, send four of your prize slides to a slide circuit. Better yet, enter circuits in Europe, Asia, India and USA with the same four slides and try to predict the outcome. Don't be surprised if the outcome is unpredictable.

My friend, Wade Clutton, FPSA, offers the following erudite observations on judging:

"1. The perception of beauty, quality and impact is an individual thing. There will be tremendous differences in how a group of people perceives these basic elements. Our perceptions are the cumulative result of major influences such as education, personality, training, camera club experiences, judging experiences, religion, advertising, personal preferences (some like red, some like blue, etc.) and no doubt a whole host of other elements.

"The perceptions will evolve over a period of time: Remember when we thought an all-white car was a piece of nondescript junk and today they are fashionable? Remember when guys wore pink shirts and charcoal pants? Not many would be caught in them today! When one looks at the Permanent Slide Collection you sure can tell how tastes have evolved.

"If we all saw and reacted to images in the same way, then all our pictures would look very much alike. For my money, the differences are to be celebrated!

"2. As far as I am concerned, a salon judging panel is there for a single purpose: To pick a variety of slides which, hopefully, captures a taste of the submitted work. Regardless of how good the Maroon Bell pictures are, there comes a time when enough is enough, and good images might well go down the drain. After all, how many pictures of the same subject from virtually the same shooting spot can you have in a show without putting the audience to sleep?

"3. Since retiring I have made a major effort to take more kinds of pictures. The results have been enjoyable and profitable, but the big shocker is that I do a better job of judging because I am more sensitive to more subjects than I used to be. A lot of judges don't have a large variety of subjects and techniques in their arsenal.

"4. A lot of judges are afraid to stray from what they think is OK or what they think their friends think is OK. A lot of judges, especially the inexperienced, often lack confidence and conviction. And some are trying to prove the opposite.

"5. My scoring system is based on a two-phase scheme: First, the judges are acting as individuals reviewing the slides and making one of three decisions: (1) I want the slide to be in the show; (2) I don't really care if the slide is in the show or not, (3) I don't want the slide to be in the show. Some judges are afraid to say they really would like to see a slide in the show. Contrary to the thoughts of some, empirical evidence suggests that judges do not die, do not go to jail or suffer mental hardships from pushing the `yes' button!

"In the second phase, the judges are acting as a unit reviewing the top slides to determine which will be awarded certain high honors and medals. If the judging panel has done a good job on the first phase, there will be a lot of slides that all three judges agreed to have in the show. These slides, by definition, are honor slides. It is from this universe that the judges should pick the top of the top.

"When selecting the award slides, the judges must be conscious about selecting variety. Not only does the exhibition audience want variety, but if portraits are the only type of image awarded medals this year, next year's exhibition will be flooded with portraits. An entrant never knows the chemistry of a judging panel. Some panels are well balanced, some are in continual conflict. An entrant never knows what has come before his work; if his image is preceded by a bunch of so-so images, his image might really look super to the judges; if his image is preceded by a bunch of super images, his image might look lousy to the judges; if his image is one of bright colors, it might appeal to the judges, or they might well be tired of bright colors.

"Some panels are instructed to be very, very selective in the number of slides they accept; some panels are instructed to be more generous so as to have a larger show. The entrant never really knows these equations but they do impact the outcome.

"The larger the scoring system, the greater the ability of one judge to unduly influence the scoring results. The ultimate in neutralization is the `in or out' system. Two or three `in' votes and it is in. No judge can force the slide into or force the slide out of the show. This generally works to provide a more balanced selection. After all, a judge is always making the `yes, I want it, or no, I don't want it' decision.

"Scores mean nothing. I remember one exhibition in which the minimum score was 5 and the maximum was 15 with a five-member panel. The accepted score cutoff was 7. That meant that roughly 75% of the work received either a 5 or a 6 and 25% of the work received a 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15. Photographic images will always tend to fall into something approximating a normal bell-shaped curve. Under this system the 10s and 11s should have been the high numbers with fewer 12s, fewer 13s, fewer 14s and fewer 15s, with a cutoff point of probably 12 or 13. When the judge's scores are so distorted, you know, by definition, their cumulative performance was unsatisfactory.

"The entry can be influenced by topics for medals and the judging can be influenced by the topics for medals. The exhibitor really never knows what the image universe is.

"Some exhibitors believe that where a slide is in the competition is a great factor in the outcome. Some say, don't be in the first 25 or the last 100; some say put your best work in round 1, some say put your best work in round 4, etc.

"I concluded several years ago that one should not show Californians Mexican images. One should show them `green' images; Mexican images went fine in Ohio! Don't show an Indian judge `Jenne Farm' as New England just doesn't click with him, but the Red Rock Country of western movies works really well. Americans like simpler pictures; Europeans like busier pictures.

"To presuppose there is a universality of results would be to presuppose that the community of man is uniform and we know that is untrue."

It appears to me that some judges award scores based on their own preferences, perhaps honed by the opinions of other photographers in their own region. On some occasions, objectivity seems to have vanished. Not long ago I heard a judge express disdain for still life images "because they were too easy." On another occasion a judge commented that he favored still life portraits. In my opinion, a judge who makes choices based on personal preferences is not fulfilling the rules of judging.

Suppose you were the judge and another one of those often-seen images of the Maroon Bells or the Jenne Farm appeared on the screen. Should it be marked down simply because you have seen the same image too frequently in the recent past and the image no longer holds the interest for you that it once did?

Certainly we all have our preference of subject matter. Some may favor a landscape or wildlife shot or portrait of a child while others may prefer a good photojournalism action subject. However, regardless of our personal likes or dislikes, it is incumbent upon all of us who accept the responsibility of judging to exercise complete impartiality and objectivity when assigning a score to each entry. This is a lesson many judges have yet to learn.

Thank you for letting me express my views and those of Wade Clutton, FPSA. Wade has certainly helped to put the judging phenomena in a clearer light!

COPYRIGHT 1999 Photographic Society of America, Inc.
COPYRIGHT 2004 Gale Group