Sunday, January 10, 2016

Tournament Point System: Mistaking Motion for Progress?


The Nobel Prize for Peace awaits the inventor of a handicap system that ensures equity and identifies and punishes scoundrels.  The search for such a handicap system has been as productive as that for the Loch Ness Monster.  The futility does not stem from a lack of trying.   The USGA introduced the Reduction in Player Index for Exceptional Tournament Performance (Sec. 10-3) to penalize players who have handicaps higher than their potential would suggest.   Penalties under this Section are easily avoided by the skilled sandbagger, however, so it has little or no effect on tournament equity.1] 

The most common complaint faced by Tournament Committees is about the “same old winners.”   Certain players always seem to finish high in the tournament standings which draws the wrath of other players who feel the playing field is not even.  The USGA Handicap System provides one solution for the besieged Committee.  The Committee can reduce a player’s handicap if it believes the player is deliberately manipulating his scores (Sec. 8-4c).  Such reductions are rarely imposed because the Committee typically does not have the ability to collect and analyze large data sets for evidence of manipulation.  It may also want to avoid the personal confrontation of labeling a member as a “cheater.”  

To avoid the appearance of being feckless, some Committees have adopted a tournament point system (TPS) patterned after the point system developed by Dean Knuth, former Director of Handicapping at the USGA.  Under such a system, a player accumulates points based on where he finishes in the tournament.  After certain levels of points are accumulated, his handicap is reduced.   Initially Knuth’s system was applied to large stroke play events—i.e., more than 50 players.  If a player consistently finishes high in such events, the statistical evidence strongly suggests the player’s handicap is out of line and a reduction is warranted.
 
Clubs, however, do not usually hold large stroke play events.  The theory behind the TPS breaks down when a club only has a few tournaments a year, the flight size within the tournament is small, and the tournaments are sporadically entered by the membership.  In such cases, many problems can occur: 1) the TPS may not punish the guilty, 2) the TPS may punish the innocent, 3) the TPS can drive down tournament participation, and 4) the TPS can adversely affect the culture of the club. 

To examine the likelihood and severity of each of these problems, a case study of a club that recently adopted a TPS has been undertaken. The parameters of the adopted TPS fit those where the TPS is not likely to be effective.  Only four tournaments a year are part of the TPS.  The size of each flight ranges from 5 to 8 competitors.  And on average, a member only enters two tournaments a year.  The points awarded vary by flight size as shown in Table 1. The reduction in handicap for points accumulated over two years is shown in Table 2.   Now let’s examine each potential problem:


Table 1
Points Awarded by Finish

Finish
Flight Size up to 8
Flight Size Greater than 8
1st
2
3
2nd
1
2
3rd
0
1

Table 2
Reduction in Handicap for Accumulated Point Totals

Accumulated Points
Handicap Reduction
7
2
9
3
11
4
13
5

Not Punishing the Guilty – In theory, the more tournaments a player enters, the more points he should be allowed before a reduction in handicap is imposed.  When the number of tournaments considered is only eight, however, both the Knuth Point System and the TPS studied here do not consider how many tournaments are entered.  This anomaly in the TPS allows the unethical player to manage his tournament schedule to greatly decrease his chances of a handicap reduction.  If he only enters two tournaments a year, his chance of receiving a handicap reduction is quite small.  And since most tournament players do not enter all of the tournaments, the TPS effectiveness will be greatly limited (i.e., even if a player has a suspect handicap, he will not play in a sufficient number of tournaments to be caught by the TPS).

Moreover, the TPS does not stipulate the handicap from which any strokes must be taken.[2]  If a 15-handicap receives a two-stroke reduction, he only has to increase his handicap to 17 to avoid any punishment.  To counteract such manipulation, the club has to keep a record of the player’s handicap at each tournament, make a decision on whether the player’s current handicap has been manipulated, and if so, confront the player.   The TPS was supposed to relieve the Committee of such duties.  It does not.

Punishing the Innocent– Punishing the innocent has been a problem for modified TPSs.  One club in Oregon, for example, implemented a TPS where most members had a handicap reduction by the end of the season.  Knuth was so irate he wrote the Oregon Golf Association:

“I am quite happy when clubs use my Knuth Tournament Point System.  However, I am not happy when the system gets bastardized to the point that it doesn't work.  Would you please contact this club and tell them to use the system properly.”[3]

The club under study has followed the Knuth Point System, but it still has a problem.  When the flights are small (e.g., 6 or less), the TPS has a good chance of returning a false positive (i.e., identifying a player as a sandbagger when his good fortune is due to chance).   A typical player if he enters all 8 events has about a 13 percent chance of receiving a handicap reduction if the flight size is 6 (see Appendix).  Perhaps that is a reasonable price to pay if the TPS also captured the flagrant sandbagger.  As noted above, however, that may not be the case.  The TPS may impose costs without any accompanying benefit.

When the flight size is even smaller (e.g., 4), approximately 40 percent of the competitors would receive a reduction in handicap.  In this case, the TPS goes from a method of detecting suspect handicaps to one of ensuring “everyone wins a trophy.”   

 Tournament Participation – The TPS could suppress tournament participation.  The unethical player is unlikely to play in all eight events.  His absence should work to the benefit of ethical players.  But this assumes all events are of approximately equal value (e.g., a weekly sweeps).  That is not the case here.  While three of the tournaments require significant entry fees and award substantial prizes,[4] the fourth (Net Club Championship) does neither.  In the Net Club Championship the entry fee is $40 and the payouts are $100 for first and $75 for second.  .  Yet the winners of the Net Club Championship flights are assigned the same TPS points as the flight winners in the Member/Guest with a $1095 entry fee per team.  The prudent player who wants to minimize his chances for a reduction while maximizing his economic well-being will skip the Net Club Championship.  Players on the cusp may also skip tournaments to avoid a reduction in handicap. 

The TPS could increase participation if ethical players enter more tournaments believing they now have a fair chance.  While a stronger case can be made for declining participation, the actual participation is an empirical question.  This question should be answered at the end of year two of the TPS.

Club Culture –Adopting a TPS is an admission that “cockroaches live among us.”  This is never a good thing for a club.  Knuth developed his system for large statewide events (e.g., Pacific Northwest Golf Championship).  In such events, the players typically do not know one another or whether the Committee has reduced someone’s handicap.  This is not the case when the TPS is employed at the club level.  Point totals can be readily viewed by the membership.   Will club members hang a scarlet “S” for sandbagger on a player whose only sins were being lucky and playing in all of the tournaments?  Will the penalized player give up tournaments to avoid the grief?   Will a penalized player be forsaken for partner events?  Will the TPS end the carping over suspect handicaps that pervade every tournament?  (Answer: No.)  The effect of the TPS on the culture of the club is unknown, but it is something the Committee should examine closely in the early years of implementation.

Measuring the effectiveness of the TPS is difficult.  If no player gets a reduction, can it be said the TPS has eliminated sandbagging?  Alternatively, can it be argued the TPS was a waste of time?  Let’s look at the club under study to see how many players could get a reduction in handicap.

In implementing the TPS, the Club used tournament results from 2015.[5]  If the point totals are duplicated in 2016, there would be 6 players with 8 points, and 1 player with 10 points.  Four of the players are unlikely to repeat their 2015 performance due to regression toward the mean—i.e., a good year is more likely to be followed by a bad year than another good year.  The others can avoid the TPS reduction in handicap penalty by skipping some tournaments when they are on the cusp of a penalty.  If some players do receive a reduction, the Committee will have to determine if these players have been on the Committee’s radar or are innocents snared by their heavy tournament participation.  The tournament season will end in April 2016.  Then it can be determined if the adoption of the TPS was just another case of mistaking motion for progress. 

Update: June 3, 2016
1. One player received 8 points and 4 players received 7 points. At least 3 of the players participated in all of the tournaments.  The TPS penalty may be for participation as much as for performance.
2. Only one penalty was applied during the season.  That player was assigned a higher Index after the reduction that his Index in the previous tournament (i.e., his current Index had increased so the reduction was nullified).
3. Players were not informed of their point status unless they went to the pro shop and asked.  Most players were not aware to the TPS.  Participation was down in the tournaments, but it is not clear the TPS was a cause.  The Tournament Committee never explained why the TPS was kept such a secret.
4. The pro shop was charged with keeping track of the points.  The pro shop, however, made several mistakes.  It would be best to assign this task to a member.  Assistant pros are not all that interested in performing this task.  If the point totals were posted in the locker room, peer review would have caught the mistakes.
5. The Tournament Committee never evaluated the efficacy of the TPS at the end of the season.  Since the Committee never established any criteria for success, an analytical evaluation was not in the cards.  The Committee just assumed the TPS was working and was looking forward to next year.  And so it goes.

Update: March 6, 2017
1. A penalized player won his flight by 11 strokes even with a two stroke reduction in handicap.
2. Some players played have played to a  higher reduced handicap than the handicap with which they won previous events.
3. The tournament results would have been the same in the absence of the TPS.  I believe every player with a reduced handicap except one (i.e., see #1 above) skipped the handicap flights.

Is the TPS worth the effort?  That is for higher powers to decide.


Appendix
The Probability of Tournament Outcomes

In this analysis it is assumed a flight contains 6 competitors.  The player competes in 8 events.  He can finish first and gain 2 points with a probability of 1/6.  He can finish second and gain 1 point also with a probability of 1/6.  He can finish third or worse and gain no points with a probability of 2/3.  The various outcomes are shown in Table A-1.  In the third row, for example, the player earns 14 points by finishing first 6 times and second 2 times.  There are 28 ways this performance pattern can be achieved.  The probability of achieving 14 points in this exact manner would 28*(1/6)5*(1/6)3.  The player can also achieve 14 points with seven first place finishes and one third place or worse finish.  The probability of gaining 14 points is the sum the probabilities of the two ways to score 14 points.  The probabilities of gaining various point levels are shown in Table A-2.  The probability a player gains 7 or more points and receives a reduction is approximately 13 percent.  The probabilities of gaining 7 or more points for various flight sizes are presented in Table A-3.

Table A-1
Tournament Outcomes


Pts.

Ways
Tournament Number

1
2
3
4
5
6
7
8

16
1
2
2
2
2
2
2
2
2
15
8
2
2
2
2
2
2
2
1
14
28
2
2
2
2
2
2
1
1
13
56
2
2
2
2
2
1
1
1
12
70
2
2
2
2
1
1
1
1
11
56
2
2
2
1
1
1
1
1
10
28
2
2
1
1
1
1
1
1
9
8
2
1
1
1
1
1
1
1
8
1
1
1
1
1
1
1
1
1
12
28
2
2
2
2
2
2
0
0
10
56
2
2
2
2
2
0
0
0
8
70
2
2
2
2
0
0
0
0
6
56
2
2
2
0
0
0
0
0
4
28
2
2
0
0
0
0
0
0
2
8
2
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
13
56
2
2
2
2
2
2
1
0
12
168
2
2
2
2
2
1
1
0
11
280
2
2
2
2
1
1
1
0
10
280
2
2
2
1
1
1
1
0
9
168
2
2
1
1
1
1
1
0
8
56
2
1
1
1
1
1
1
0
7
8
1
1
1
1
1
1
1
0
11
168
2
2
2
2
2
1
0
0
10
420
2
2
2
2
1
1
0
0
9
560
2
2
2
1
1
1
0
0
8
420
2
2
1
1
1
1
0
0
7
168
2
1
1
1
1
1
0
0
6
28
1
1
1
1
1
1
0
0
9
280
2
2
2
2
1
0
0
0
8
560
2
2
2
1
1
0
0
0
7
560
2
2
1
1
1
0
0
0
6
280
2
1
1
1
1
0
0
0
5
56
1
1
1
1
1
0
0
0
7
280
2
2
2
1
0
0
0
0
6
420
2
2
1
1
0
0
0
0
5
280
2
1
1
1
0
0
0
0
4
70
1
1
1
1
0
0
0
0
5
168
2
2
1
0
0
0
0
0
4
168
2
1
1
0
0
0
0
0
3
56
1
1
1
0
0
0
0
0
3
56
2
1
0
0
0
0
0
0
2
28
1
1
0
0
0
0
0
0
1
8
1
0
0
0
0
0
0
0



Table A-2
Probability of Each Point Total

Points
Probability
Points
Probability
16
.00000
7
.06563
15
.00000
6
.10909
14
.00004
5
.14723
13
.00017
4
.18137
12
.00071
3
.17070
11
.00230
2
.14632
10
.00682
1
.07804
9
.01641
0
.03901
8
.03614




Table A-3
Probability of Gaining 7 or More Points by Flight Size

Flight Size
Probability of 7 or More Points
5
22.2
6
12.8
7
9.4
8
5.0




[1] See “Why Sec. 10-3 is Ineffective,”  www.ongolfhandicaps, April 28, 2014.
[2] The USGA’s Reduction of Handicap Index for Exceptional Tournament Scores penalizes a player if his handicap goes up after making two qualifying T-scores. The TPS does not. 
[3] E-mail from Dean Knuth to Kelly Neely, Director of Handicapping, Oregon Golf Association, June 27, 2005.
[4] Significant is defined here as the size of an average Time Warner cable bill. 
[5] Using tournament results from a period when the TPS was not operational biases the results.  Players are likely to behave differently when they know their performance is being studied, a phenomenon termed the Hawthorne Effect.  It does provide a baseline for tournament participation, however, from which to make judgments about the TPS’s effect.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)