The Unmaking of the USGA's Appendix E

The only thing the USGA hates more than making a mistake is admitting to one.  Examples of this behavior are numerous.  The most famous (or infamous) was the USGA’s defense of the Masters for not disqualifying Tiger Woods in 2013. The USGA argued this was an exceptional case and would determine whether any “adjustment to the Rules and/or Decisions is appropriate.”[1]  The USGA’s curious logic, however, was never memorialized in the Decisions.  Its omission was due to the only exceptional circumstance being the Masters Tournament Committee was headed by a former president of the USGA.[2]  Since this is a circumstance unlikely to be repeated, the USGA apparently thought it best to paper over the error and move on.

The USGA’s proclivity for dissembling even reaches down to those responsible for the handicap system.  This post examines the truthfulness of the USGA in defending Appendix E, Exceptional Tournament Score Probability Table, of the USGA Handicap System.  This example is not chosen because of its importance to the handicap system.  It really has none.  Its selection is based on two other reasons.  First, if the USGA is less than straightforward on minor issues like Appendix E, it does not portend well for its actions on major issues.  Second, Appendix E should be a place for the USGA to demonstrate its competence in probability theory which is the backbone of the handicap system.  If the USGA errs here, similar missteps can be inferred in the more important elements of the handicap system (e.g., validity of the Slope System, allocation of handicap strokes, handicap allowances).

For many years the probability table in the Appendix E looked like that shown in Table 1 below.  The text explained “The values in the table are the odds (sic) of shooting a net differential equal to or better than the number in the left column.”[3]  The USGA wrote “odds” when it meant “probability.”  When this error was pointed out, the USGA made the correction without admitting its previous error. 

Table 1

Exceptional Tournament Score Probability Table 2006-2007 (Abbreviated)

Net Differential	Handicap Index Ranges
	0-5	6-12	13-21	22-30	30 or Greater
0	5	5	5	5	5
-1	10	10	10	8	7
-2	23	22	21	13	10

While the probabilities (i.e., the inverse of the table values) presented in the Table 1 could be questioned, at least they made sense.  This changed with the 2012-2015 edition of the USGA Handicap System where the probability table took on a new form as shown in Table 2.

Table 2

Exceptional Tournament Score Probability Table 2012-2015 (Abbreviated)

Net Differential	Handicap Index Ranges
	5.9 or Less	6.0-12.9	13.0-21.9	22.0-30.9	31.0 or Greater
0 to -.9	5	5	5	5	5
-1.0 to -1.9	10	10	10	8	7
-2.0 to -2.9	23	22	21	13	10

The text in Appendix E again explained “The values in the table represent the probability of shooting a Net Differential equal to or better than the number in the left column.”[4]  The problem is there is no longer a “number” in the left column but a “range” of numbers.  A simple examination of the normal curve would show the probability of shooting a net differential of 0 or better differs greatly from the probability of shooting a net differential of -.9 or better.   This discrepancy in the Appendix was pointed out to the USGA along with many typographical errors in the table.[5]    Scott Hovde, Assistant Director of Handicap and Course Rating, did not accept the criticism and defended the change as an improvement:

When reviewing the entire Handicap manual for the 2012 update, we looked at the table in Appendix E to determine if it needed some tweaking or updating.  The initial thought was to remove the table from the book altogether and only supply an online version once the table was fully reviewed.  In consulting with a few of our HRT members (who were involved with the original table), it was decided to keep Appendix E in the book, but expand the rows to ranges, as the formula (and example) listed underneath the table to determine the net differential is taken to the tenth place.  The values in each cell represented an average of the probability of net differentials within that range, and not just a single net differential.  In reality each 0.1 difference in net differential will have a separate probability, but that table would be far too large to be of use.  

In the interest of ease of use by our target audience, the ranges were there to keep them from having to interpolate and just plug in the net differential, and probability was listed in each cell as the divisor, instead of probability itself (so 1200 instead of .0008333…)…The typos in the manual were unfortunately made at the printer when they formatted the table for final printing, the table we submitted and that was reviewed by all involved had the correct values. [6] 

Hovde does admit to the typos, but argues the mistake was not the USGA’s, but the printer’s.  For this to be true, it must be believed the USGA did not request, receive, and copy edit a proof copy before publication.  In other words, Hovde’s defense implies the USGA was guilty of gross negligence.    

The most charitable interpretation of Hovde’s meandering syntax is that cell values represent the probability of a Net Differential equal to or better than some average value within the range.  In the case of a range from 0 to -.9, for example, the cell value represents the probability of a Net Differential equal to or better than -.45 (i.e., an average value).   The user will still have to interpolate if he wants a more accurate estimate of the probability.  So listing ranges is not a benefit to the user as Hovde claims.

Hovde argues members of the Handicap Research Team (HRT) agreed with this new formulation.  The probabilities, however, did not change with the new edition.  If the cell values are really “average” probabilities, all previous editions have been incorrect.  For example, the USGA has previously maintained a player should play to his Index 20 percent of the time.  The 2012-2015 Edition now implies a player has a much better chance to play to his Index than .20.  It is difficult to believe the HRT would be party to such a ruse. 

Hovde also maintains two or more of the originators of the table agreed with using ranges.  The only surviving members of the HRT who did research on outlier identification are Dean Knuth and Frank Engel.  Knuth has implied he had nothing to do with the change.[7]  Perhaps Engel supported the change, but that would still be far from the unanimous approval Hovde alleges.  It is more likely Hovde tried to use the imprimatur of the HRT to add credence to his argument.

The final piece of evidence showing Hovde’s defense was disingenuous came with the publication of the USGA Handicap System 2016-2017.  The table in Appendix E now looks like that shown in Table 3 below.[8]  Table 3 shows ranges have disappeared and the left column has reverted to integers.  If there were major benefits to be gained by using ranges, why have they disappeared? 

Table 3

Score Frequency and Probability Table 2016-2017 (Abbreviated)

Net Diff.	Handicap Index Ranges
	<  0.0	0.0- 4.9	5.0-9.9	10.0-14.9	15-19.9	20.0-24.9	25.0-29.9	>29.9
0	4.8	4.3	4.3	4.7	5.2	6.2	7.1	6.8
-1	8.8	7.6	7.2	7.6	8.3	9.7	11	9.8
-2	19	15	13	13	14	16	17	15

In early 2012, the USGA must have realized Appendix E was in error.  To make a correction, however, would mean running a revised Appendix E by the Handicap Procedure Committee and admitting to a mistake.  The better option, and the one chosen, would be to bury the change in the 2016-2017 Edition without explanation and hope no one would notice.  And so it goes…

---

[1] “USGA, The R&A Issue Statement Addressing Tiger Woods Ruling at the 2013 Masters Tournament,” USGA, Far Hills, N.J., May 1, 2013. 

[2] For a full discussion of this ruling see “Bureaucracy, Augusta National, and Tiger Woods,” www.ongolfhandicps.com, April 8, 2013.

[3] The USGA Handicap System, 2006-2007, USGA, Par Hills, NJ, p. 126.

[4] The USGA Handicap System, 2012-2015, USGA, Far Hills, NJ, p. 117.  There is some evidence the column headings were once Handicap Ranges and were switched to Index Ranges to be consistent with Slope System nomenclature. See “The Reliability and Accuracy of USGA Handicap Research,” www.ongolfhandicaps.com, March 6, 2012.  The USGA has not made any study supporting Appendix E available to the public, so its validity remains in question.

[5] Dougharty, Laurence, e-mail to Scott Hovde of the SSGA, February 1, 2012.

[6] Hovde, Scott, e-mail to author, February 9, 2012.

[7] Knuth, Dean, e-mail to author, February 13, 2012.

[8] The USGA will not release the study behind the revised Appendix.  Even without seeing the study, however, some problems with its results are evident-- see “The USGA’s Appendix E: Problematic at Best,” www.ongolfhandicaps.com, forthcoming.