How should IAP Contests be Judged? Please Read Then Vote

[mbroberg]

Please read this post before voting

IAP contest winners are currently chosen by one of several methods. The chair-person of each contest is the person who decides the method in which contest winners are determined. There are currently three ways IAP contest winners are selected.

One way is by a popular vote (poll). It is fairly straight forward. The judging criteria are posted, the membership at large votes and the winner(s) are selected based upon number of votes received. Whether the winner is selected in one round of voting or several rounds of voting is immaterial. The important thing is that the members determine the winner. Proponents of this method say this is a fair and transparent way to select the winners of contests. Opponents say the entries are not judged consistently by the judging criteria and the winner is ultimately selected by "Wow Factor"

Another way winners of IAP contests have been selected is by a panel of judges. The judges are often anonymous but it is not really necessary that they be anonymous. Proponents of this method say that this is fairer because all of the entries are judged on the same set of criteria. The "Wow Factor doesn't influence the judges like it would the membership at large unless, of course, "Wow Factor was one of the criteria that the judges are supposed to look for. Opponents of this method want more transparency in the selection process. They want to have a voice in selecting a winner. They want to be involved. They believe that this community is made up of pen makers of all experience levels and they are very capable of selecting a winner based upon defined judging criteria.

Lastly, there is a hybrid of Judges and open polls. The judges select a number of 'Finalists" from all the entries and the membership determines the winner(s) by open voting. The pros and cons of both the other methods come into play.

It has been argued that the Photo Contest MUST be judged by a panel of judges who know what to look for when determining whether or not the photo is composed properly. There is a fear that if the Photo Contest were to be judged by open voting then it would be a "pretty picture of a pen contest", not a true photo contest where all the technical elements that make up a professional quality photo are taken into consideration. Therefore, a couple of the responses are offered twice; once to include the photo contest and once to exclude the photo contest. Please read the responses carefully so you will vote the way you intend to.

This poll will remain open indefinitely. Future IAP activities will be influenced by the results.

 

[Joe S.]

But your polling the community at large, why not have a panel of anonymous judges decide this? :biggrin:





(To be clear, that was a joke.)

 

[mbroberg]

But your polling the community at large, why not have a panel of anonymous judges decide this? :biggrin:





(To be clear, that was a joke.)

I got it. Chuckle!

 

[Joe S.]

I guess I like the hybrid idea, judges decide the finalists, people decide the winner. Or possibly the other way around, people vote on all the pens, judges pick from the top 5 based on quality of craftsmanship.




(To be clear, that was not a joke :biggrin

 

[SerenityWoodWorks]

If its a contest and you are truely looking for the best pens that fit the 3 Fs then judges are the way to go, they will look past the pretty blank and check for proper fit (undercut/overcut). In the case of advanced pens, by having open polling you are asking for people that in many cases might have never ventured beyond a slim line to judge and that is not fair to the people entering the contest as it just turned into a pretty blank contest. Its the same in casting, I felt I did not have the knowledge to know what goes into the casting or blank making and it turned into a beauty contest rather than a skill contest. So it reallygoes back to..if you are looking for the Most Beautiful Pen the its open..if your looking for the Skill level it took to create that pen the it should be the same judges looking at each pen and they should be people that have the knowledge it takes to make the pen so they know what to look for.

 

[mredburn]

I voted, finalists selected by judge and winners by members. I have had times where the membership favors a pen that doesnt meet the criteria of the contest but is very nice over a pen that does. Neither are bad pens. Yes, color preferences, time of day, kit preferences can all sway the final choice of a judge but that is true of all the members choosing from the entries as well.

 

[SerenityWoodWorks]

And I vote that Mike is not allowed to enter any contest I want to enter.

 

[mredburn]

To quote Ms. Piggy......Mwah?

 

[mrcook4570]

The results of any subjective contest will always be open to some degree of scrutiny. Perhaps one of the biggest factors in determining a winner, regardless of the voting method, has absolutely nothing to do with pen making. It is the photograph. A spectacular pen in a mediocre photograph will most likely not make the cut. However, a mediocre pen can look like a spectacular pen in a good photograph. Major flaws can be hidden quite easily in a photograph. As much as a pen contest should be about the pen, in reality, it is also very much a photography contest.

Unfortunately, I can think of only one alternative which would be to collect the pens and physically examine them. This, of course, is not practical and creates more logistical problems than solutions.

Don't get me wrong. I love the contests and look forward to them every year. Over the years, I have entered some and sat on judging panels. I almost always vote when given the opportunity.

With all of that said, just 'build it and they will come'. Set the rules and people will happily participate because of their desire to compete. Opening the rules up for debate, in my experience, only leads to dissension.

 

[lyonsacc]

Winners except for photo contest winners determined by members after the filed os narrowed by judges.

To this I might add one or two variations:
The Russ Fairfield challenge could be done by judges - it is a more technical contest since it follows Russ' styles.

Freestyle should be 100% member vote

 

[Sylvanite]

There are legitimate arguments to be made for the fairness of each method. There is no stand-out method to pick the "best" when the choice is inherently subjective.

I think the Bash is more fun, however, when the members at large get to participate in the selection. It's a livelier party when the whole crowd gets to play.

Just my $0.02,
Eric

 

[mbroberg]

So it reallygoes back to..if you are looking for the Most Beautiful Pen the its open..if your looking for the Skill level it took to create that pen the it should be the same judges looking at each pen and they should be people that have the knowledge it takes to make the pen so they know what to look for.

This statement got me to thinking..................................

What if each contest were judged on 2 levels? Member sends photo in then sends the actual pen to the contest chairperson. Contest chairperson posts photos as he receives them or all at the same time (however that question is decided). Once the entry deadline has arrived and the contest chairperson has all the pens, the pens start making their rounds to 3 or 4 judges.

The members vote, during the Bash, for "The Member's Choice" in each contest, 1st, 2nd and 3rd place.

The judges, independant of the members votes, make their selections in every contest for 1st, 2nd and 3rd place.

Six prizes are awarded for each contest, 1st, 2nd, 3rd in both judging categories. It is possible that one pen could place in the top three in both categories.

Logistics would dictate that the judges winners would not be announced until after the Bash is over. Each entry would have to be shipped from judge to judge for independant scoring.

What this procedure would do is allow the members their vote during the Bash based on the photograph while at the same time allowing the entries be judged for technical excellance by qualified individuals who actually habdle the pen.

The same principle could be applied to the Photo Contests. The only difference would be nothing would have to actually be shipped between judges.

View this as an "Other" resonse to the poll above. If it receives enough verbal approval or a good number of "Likes" in this thread you may see another poll down the road with this option up against the winner of this poll.

 

[mredburn]

One of my winning entries was from the Ukraine I believe. Now under your proposed new double judging he has to ship it to my Judges. After they are all collected who pays to ship them on to the next judge? Who pays to ship them Home when its all done? Who is responsible should they get lost in shipping. I can see this taking 6 months to complete and needing an entry fee to pay the cost.. Way to long and complicated. People will have long forgotten the contest by the time the judges get done.

 

[Ed McDonnell]

This was responding to Mike B, not Mike R (who was faster to type than me).....

Mike - It seems to me that the Bash contests would take a couple of months if pen mailing gets involved. It would also eliminate entries from those who have a buyer waiting for the pen they want to enter. It might also make it harder to find contest moderators and judges given the huge increase in time and expense it would entail.

Maybe a solution would be as easy as adding a "Viewer's Choice" award to the prize list. Top three would be chosen by judges (with or without final member input). Viewer's Choice would be chosen by the members from the total group of submissions.

Rather than a poll for Viewer's Choice, I would open a thread where people have to post their choice and some minimum number of words of why they made their choice. It would be a little more work than a poll to tally the votes, but it would certainly up the banter.

Ed

 

[Haynie]

Stan summed up my thoughts a lot better than I could.

 

[mredburn]

Ed the "Peoples Choice" awards was going through my mind when you posted. However I believe I have seen on one of the talent shows where the dancing dog won over the talented singer because the dog was cuter.

 

[mbroberg]

I should have hit the Quote button on Mike R's earlier post. This is in response to that.


We are exploring ideas right now. I don't have the answers to those questions. At this point those questions are not important. The question to be answered is whether ot not this method of doing things would be a fair way to address all the issues surrounding contests being judged by open vote vs. judges.

Your Ukrane example is valid, but I would say that it would be the exception, not the rule. Details of any of the options of how contests are judges can be addressed later. What do you think about the idea? We need to figure out WHAT we (as members) want to do before we (as contest administrators) worry about HOW to do it. It may turn out that it can't be done. Don't jump to that conclusion right out of the gate because maybe, if we give it some thought, it can be done. What I want to know is IF it should be done.

 

[gallianp]

Why hide the results of this poll? It would be interesting to see results as votes come in.

 

[Ed McDonnell]

Ed the "Peoples Choice" awards was going through my mind when you posted. However I believe I have seen on one of the talent shows where the dancing dog out won over the talented singer because the dog was cuter.

It could also end up being a popularity contest where a beloved member whose style is easily recognized gets all the votes. But, if the "Viewer's Choice' is just one award in addition to the top 3 placers and the prize awarded is commensurate with a 4th place finish, then why the viewers chose won't really matter. To me, anyway.

Ed

 

[mbroberg]

This has some possibilities

This was responding to Mike B, not Mike R (who was faster to type than me).....

Mike - It seems to me that the Bash contests would take a couple of months if pen mailing gets involved. It would also eliminate entries from those who have a buyer waiting for the pen they want to enter. It might also make it harder to find contest moderators and judges given the huge increase in time and expense it would entail.

Maybe a solution would be as easy as adding a "Viewer's Choice" award to the prize list. Top three would be chosen by judges (with or without final member input). Viewer's Choice would be chosen by the members from the total group of submissions.

Rather than a poll for Viewer's Choice, I would open a thread where people have to post their choice and some minimum number of words of why they made their choice. It would be a little more work than a poll to tally the votes, but it would certainly up the banter.

Ed

 

[mbroberg]

Why hide the results of this poll? It would be interesting to see results as votes come in.

Some people think that a blind poll is more objective than an open poll. Others will argue that it doesn't make any difference. I happen to believe that the blind poll is more objective, and it's my poll.:biggrin::biggrin:

I do plan on revealing the results of all the polls once the votes drop down to nothing and I close them.

 

[Displaced Canadian]

I think members voting is more fun. If there is too many to reasonably put in a poll have a panel narrow it down. However there are other contests that are more technical in nature. The photo contest is a great example. I look at a picture I look at the item photographed. If it looks good I say good picture. My FIL is a semi professional photographer and he looks at things that I don't realize are taking away from the photo. The Russ Fairfield contest is the same but to a lesser extent. With whatever method used there is always the human element. Put the Mona Lisa beside an impressionist sculpture and both items will get votes.

 

[mbroberg]

If there is too many to reasonably put in a poll have a panel narrow it down.

For everyone's information, Jeff just recently (right before the Bash began) modified the poll module of the forum software to allow up to 30 choices. That should be enough to handle all the entries of any contest.

 

[Brooks803]

One of my winning entries was from the Ukraine I believe. Now under your proposed new double judging he has to ship it to my Judges. After they are all collected who pays to ship them on to the next judge? Who pays to ship them Home when its all done? Who is responsible should they get lost in shipping. I can see this taking 6 months to complete and needing an entry fee to pay the cost.. Way to long and complicated. People will have long forgotten the contest by the time the judges get done.

Mike R, the cost to ship Yaroslaw his prize was $24 via small flat rate box. I think anywhere outside North America will be at or around that price. With Customs and potential duties it could be more. Now I'm not bringing that up to gripe about it. I don't mind the cost at all. I was happy to ship them out anywhere and was aware that it could happen. But it does show that if mailing out entries does happen it really could become costly for someone.

I don't like the idea of an entry fee. When I first read some of the options my first thought went to the last Best of IAP contest and got that sour taste in my mouth again.

Now for my opinion on how judging. I have mixed emotions. From my experience hosting the casting contest the past 2 years I think a mix of both methods would be best. 1st year of the contest I hand picked judges (who I KNEW could judge casting not just a turned pen) and we selected the winners without public input. This year I went ahead and did everything via public vote.

I think having each contest chair hand pick 3-4 judges that they KNOW are capable of juding pick vote on who goes into the finals. Then let the public decide from there. Atleast even if it became a pretty photo contest they'd still be voting on the best of that category.

 

[David M]

I would pick , letting a pannel of judges narrow the field then the top few be voted on ( but I do have some problems with that ) or members voting -

 

[mbroberg]

I would pick , letting a pannel of judges narrow the field then the top few be voted on ( but I do have some problems with that ) or members voting -

Care to elaborate? You may have concerns that others have not thought of.

 

[SerenityWoodWorks]

Forget shipping the pen,( it would be nice but costly) what I think needs to be is 2 sets of photos...one for (viewers choice) the other for (Judges) The pics that go to the judges, lets say 5, one must show the top final joint, one must show bottom final joint, one must center joints, one must show capped view, one must show uncapped view. ect. ( examples of the exact shots can be placed online so all will photograph the same shot) then the judges will have the features they are looking for and not a doctored view. Also I propose an entry fee..what contest in the world doesnt have some sort of entry fee. The fee would be paid to the IAP for use they deem fit. We all spend money like water on blanks and material, why would anyone balk at a $10 entry fee?

 

[walshjp17]

Another thought -- already expressed in another thread -- is to have members vote for 1st, 2nd and 3rd place with points assigned for each position (e.g., 5 for 1st, 3 for 2nd, 1 for 3rd). Top point getter would win first place prize and/or bragging rights if prizes not included.

A hybrid approach would be for judges to select to 5 or 10 (depending upon number of entrants) and then members vote for 1, 2 & 3. Photo contests excepted as there are -- as noted -- technical matters that need to be considered.

My $.02.

 

[mbroberg]

Forget shipping the pen,( it would be nice but costly) what I think needs to be is 2 sets of photos...one for (viewers choice) the other for (Judges) The pics that go to the judges, lets say 5, one must show the top final joint, one must show bottom final joint, one must center joints, one must show capped view, one must show uncapped view. ect. ( examples of the exact shots can be placed online so all will photograph the same shot) then the judges will have the features they are looking for and not a doctored view. Also I propose an entry fee..what contest in the world doesnt have some sort of entry fee. The fee would be paid to the IAP for use they deem fit. We all spend money like water on blanks and material, why would anyone balk at a $10 entry fee?

This is what I love about these types of discussions. Solutions to problems to emerge from little bits and pieces of opinions building on eachother. This is a strong idea. It won't be too long before someone comes along and makes it even stronger.

 

[lyonsacc]

Forget shipping the pen,( it would be nice but costly) what I think needs to be is 2 sets of photos...one for (viewers choice) the other for (Judges) The pics that go to the judges, lets say 5, one must show the top final joint, one must show bottom final joint, one must center joints, one must show capped view, one must show uncapped view. ect. ( examples of the exact shots can be placed online so all will photograph the same shot) then the judges will have the features they are looking for and not a doctored view. Also I propose an entry fee..what contest in the world doesnt have some sort of entry fee. The fee would be paid to the IAP for use they deem fit. We all spend money like water on blanks and material, why would anyone balk at a $10 entry fee?

Really like the idea of multiple shots for the judges.

I think an entry fee, at least for most of the competitions, would decrease participation.

 

[SerenityWoodWorks]

Really like the idea of multiple shots for the judges.

I think an entry fee, at least for most of the competitions, would decrease participation.[/quote]

I dont think anyone would balk if they knew that lets say..10% goes to buying the judge of that contest a "thank you" and the other 80% goes to expanding the prizes...so we now have 1st through 5th or even 10th..and 10% donation to keep this site going. but say there are 50 entries and the fee is $5

 

[hunter-27]

If there is too many to reasonably put in a poll have a panel narrow it down.

For everyone's information, Jeff just recently (right before the Bash began) modified the poll module of the forum software to allow up to 30 choices. That should be enough to handle all the entries of any contest.

Disagree, one of my contests had nearly double that amount.

 

[mbroberg]

If there is too many to reasonably put in a poll have a panel narrow it down.

For everyone's information, Jeff just recently (right before the Bash began) modified the poll module of the forum software to allow up to 30 choices. That should be enough to handle all the entries of any contest.

Disagree, one of my contests had nearly double that amount.

Except Landon's :tongue:

 

[powertool-junkie]

For me 1/2 the fun was being able to vote on the contest polls. If it is done only by judges that takes away some of the fun for me at least. I think most of us enjoy our craft enough that we can professionally decide a winner ourselves based on fit & finish over just a pretty picture. We're all big kids right? An entry fee would be fine by me if it would benefit the IAP in any way. Lastly I do appreciate all of the hard work the bash planners do & I'm fine with whatever they decide. thanks

 

[jttheclockman]

To me this Bash Party is for the membership to enjoy. I voted to let all contests be voted on by the members. This has been pointed out before that any 2 sets or more of judges will not always pick the same pens for final vote so it comes down to tastes. So let it come down to a more wider group of tastes. When it comes to great looking blanks or pen, the pen was done professionally and will not have flaws or at least noticable ones. So as they say the cream always rises to the top. Basically all contests are popularity contests.

I say have rules in place that discourage any signs of giving away who did the pen by using familar surrounds or props. Have the pen photograghed on a plain white piece of cloth or paper or other neutral color such as grey for better color effects. No props allowed. Now there will always be some pens that no matter what you do we all are going to know who did them because they dabble in areas not all of us are privy to. Such as Brad.

If it is a technical contest then have a set of judges to pick a certain group of finals and let the crowd decide the winners. Judging is so subjective and this way you take the judges out of play. Unless you have the pen in your hand you really can not see fit or finish with photos unless the obvious such as OOR is present or gaps in between segments or kit parts.

Prizes are just a small part of what these contests are about. I think the bigger picture is the acceptance by your peers that your work is top shelf and by placing lets you know this. Just my thoughts.

 

[hunter-27]

To me this Bash Party is for the membership to enjoy. I voted to let all contests be voted on by the members. This has been pointed out before that any 2 sets or more of judges will not always pick the same pens for final vote so it comes down to tastes. So let it come down to a more wider group of tastes. When it comes to great looking blanks or pen, the pen was done professionally and will not have flaws or at least noticable ones. So as they say the cream always rises to the top. Basically all contests are popularity contests.

I say have rules in place that discourage any signs of giving away who did the pen by using familar surrounds or props. Have the pen photograghed on a plain white piece of cloth or paper or other neutral color such as grey for better color effects. No props allowed. Now there will always be some pens that no matter what you do we all are going to know who did them because they dabble in areas not all of us are privy to. Such as Brad.

If it is a technical contest then have a set of judges to pick a certain group of finals and let the crowd decide the winners. Judging is so subjective and this way you take the judges out of play. Unless you have the pen in your hand you really can not see fit or finish with photos unless the obvious such as OOR is present or gaps in between segments or kit parts.

Prizes are just a small part of what these contests are about. I think the bigger picture is the acceptance by your peers that your work is top shelf and by placing lets you know this. Just my thoughts.

I believe you almost described my contest rules in that post. :biggrin::wink:

 

[NewLondon88]

I dont think anyone would balk if they knew that lets say..10% goes to buying the judge of that contest

Oh. Well, when you put it that way ... :tongue:

 

[Jjartwood]

3 step process.
1. Blind, fit and finish, pens will be placed in a box with a hole large enough to fit a hand in to allow the judge to feel the fit and finish, score 1 to 25 points.
2. writing and balance,same as above except pen is placed in box open and judge writes something with the pen (still blind) score 1 to 25.
3. visual score 1 to 50
From that point the judges can discuss the pros and cons of each entry.
Mark

 

[David M]

this is going to be out there but , if the contest had 20 pens . in round one to narrow it down say to the top 5 , only the ones that entered the contest could vote on the finalist pens . all enterants of that contest has to vote but they cant vote on there own pen . the final 5 would be open voting .

 

[John Den]

Originally Posted by SerenityWoodWorks

......The pics that go to the judges, lets say 5, one must show the top final joint, one must show bottom final joint, one must center joints, one must show capped view, one must show uncapped view. ect. ( examples of the exact shots can be placed online so all will photograph the same shot) then the judges will have the features they are looking for and not a doctored view.

I found the "3 pictures only" very restricting and this is what bothered me most.
5 would be nice and 6 better to show fits and details etc.

Regards
John