Tuesday, March 24, 2015

The Selling of the USGA’s Slope Handicap System: The Big Con?

Introduction and Summary - The USGA has sold the Slope Handicap System as the solution to the portability problem.  By measuring the “slope effect” a player’s handicap will be adjusted to reflect the difficulty of any course and equity will be achieved.  The USGA, however, has never proven the existence of the slope effect nor demonstrated it could measure it with any accuracy.   If the Slope Handicap System was a consumer product, the Federal Trade Commission would step in and demand proof of the USGA’s claim before the Slope System could be marketed to consumers.  Sadly, the USGA is an unregulated monopoly and can do pretty much what it wants—no questions asked.  

This post asks those questions and presents a methodology for examining the existence, size, and efficacy of the Slope system.  A small and unrepresentative sample of scoring records is used to address three questions:  1) Is there a measurable slope effect, 2) Does the slope effect increase with handicap?  3) Does the slope effect have a significant impact on a player’s handicap?  

This limited study found it is likely there is a slope effect, but it only explains a small percentage of the variance in a player’s scoring differentials.  The slope effect does increase with a player’s Index as Slope Theory would predict.   A player’s index, however, does not explain much of the variance in the marginal effect of the Slope Rating on a player’s scoring differentials (i.e., the Slope Rating is not a perfect indicator of the difficulty of a course for player’s of the same Index).  Attributes of an individual’s game (e.g., tendency to slice, distance of the tee) probably explain more of this variance than the player’s Index.
 
There are two conditions that can negate the effect of the Slope System.  First, if a player plays courses with a narrow range Slope Ratings, the Slope System will not have a significant effect on the player’s handicap.  Second, if a player competes against players with about the same Index, the Slope Rating will not have much effect on the equity of competition.  The efficacy of the Slope system depends upon those conditions not being met.  How often that occurs is an empirical question beyond the scope of this study. 

Why does the Slope System which has only marginal benefits sell so well?  There are probably three reasons.  First, it has intuitive appeal to golfers who believe there are courses where they deserve more strokes.  Second, it sounds like it was developed with scientific rigor and is therefore credible.  And third, since a player’s index is typically lower than his pre-Slope Handicap, the Slope System caters to his vanity.   The estimate of a player’s handicap is so riddled with errors that tacking on the Slope System will not do any harm or make for a significant increase in equity.  An analogy for the introduction of the Slope System is “putting lipstick on a pig.”  It looks good, but the results are pretty much the same.

Is There a Slope Effect? – Before the introduction of the Slope System, a player’s handicap was just a function of his scoring differentials (Adjusted Score – Course Rating).  The USGA argued scoring differentials should be adjusted for the difficulty a bogey player encounters relative to the scratch player.  Without the adjustment for the slope effect the scoring differentials would not reflect a player’s true ability.  If there is a slope effect then, the scoring differentials of typical player (i.e., seldom if ever has a gross score below the course rating) should be positively correlated with the Slope Rating.  For example, if you are a 10.0 index player, your scoring differential should be higher on a course with a Slope Rating of 142 than on a course with a Slope Rating of 113.[1] In equation form this would be:

1)  Scoring Differential = a + b·(Slope Rating – 113) + є

Where є is the random error.

If there is a strong slope effect, the estimate of the coefficient “b” will be positive and statistically significant.  To estimate the coefficient with any validity would take a large random sample of players.  Since such a study is not practical for researchers outside of the USGA, a much smaller study was undertaken to 1) demonstrate the methodology, and 2) get some hint about the existence of the slope effect.  The small study consisted of the scoring records of handicap administrators and golf journalists, .  Each player in the sample was required to have played courses with a range of Slope Ratings. The sample is detailed in Table 1.

Table 1
The Slope Effect As Measured by Scoring Records (1/15/2015)

Position
Estimate of “b”
Significant?
R2
Director of Handicapping, Oregon GA
-.162
No
.06
Director of Handicapping, Washington State GA
-.238
No
.04
Director of Handicapping, NCGA
.054
No
.02
Director of Handicapping, SCGA
.160
No
.04
Exec. Director, SCGA
.040
No
.00
Executive Director, SCGA Ret.
.180
No
.05
Executive Director, USGA
.150
Yes
.17
Executive Director, USGA Ret.
.020
No
.00
Member, USGA Handicap Research Team
.120
No
.05
Editor, Golf Digest
.050
No
.03
Columnist, Wall street Journal
.310
No
.11
Member, Golf Digest Course Rating Comm.
-.046
No
.01
Director of Handicap, USGA Ret.
.140
No
.02
Asst. Director of Handicapping, USGA
.410
Yes
.33

Equation 1 was estimated using the scoring records of each player.  The estimates of “b” for each player are shown in Table 1.  For 11 of the 14 players sampled, the estimate of “b” was positive and therefore consistent with Slope Theory.  The estimate of “b”, however, is only statistically significant at the 5 percent level of confidence for two players. 

Another test of the power of the slope effect is the coefficient of determination (R2). This statistic measures the percentage of the variation in scoring differentials that is explained by the  Slope Rating.  The average value of R2 for the sample is .07.  That is, only 7 percent of the variation in scoring differentials is explained by the Slope Rating.  In summary, there likely is a slope effect, but it is small in comparison to the random error in determining a player’s scoring differential.

There are two possible explanations for the low values of R2.  First, there is not much variation in the Slope Ratings of the courses played by each player.  Players do not play a course with a Slope Rating of 70 one day, and 150 the next day.  More likely, a player will choose a set of tees within a fairly narrow range of Slope Ratings. The slope effect will be small in such cases.  Second, errors in the Slope Ratings can lead to small R2.   Regression analysis assumes the independent variable (i.e., the Slope Rating) is measured without error.   The Slope Ratings, however, are prone to error.[2]  If the Slope Ratings are listed as 125 and 130, but the true Slope Ratings are 127 and 127, regression analysis would assume there was no slope effect if the scoring differentials at the two courses were the same. 


Does the slope effect increase with handicap? - If you accept the accuracy of Slope Ratings, then the slope effect should increase with a player’s Index.  For example, the USGA assumes a large increase in the Slope Rating will have no effect on the scoring differentials of a scratch player, but should increase the scoring differentials of a high-handicap player.  Keeping with the USGA’s assumption that a player’s average score is 113 percent of the average of ten best out of twenty scores, a player’s scoring differential is expected to be:[3]

2)            Expected Scoring Differential = Index· (Slope Rating/113) ·1.13 = Index·Slope Rating·.01

The marginal increase in the scoring differential for a one point increase in the Slope Rating is then:

3)            Marginal Increase = Index·.01

Now the estimates of “b” shown in Table 1 are estimates of the marginal increase in a player’s scoring differential for a one point increase in the Slope Rating.  In equation form:

4)            “b” = C + D·Index

To estimate the coefficients C and D, the values of “b” shown in Table 1 were regressed against the player’s Index. The estimated regression equation was:

5)            “b” = -.01 + .008·Index

Equation 5) is an empirical test of the validity of the assumption behind the Slope System as represented in equation 3).  In Slope Theory, the constant C should be zero.  The estimate of C is .01, but is not statistically different from zero.  The theoretical value of D should be .01 as shown in eq. 3.  The estimated value of D is .008.  The estimated value of D is close to the theoretical value, and is statistically significant (i.e., the hypothesis that the true value is .01 cannot be rejected with a 5 percent level of confidence.).

The Figure below shows the plot of the regression line and the values of “b”.  As the figure indicates, a player’s Index does not explain much in the variance of “b”—i.e., the values of “b” are not clustered close to the regression line.  The value of R2 for this equation is only .04.  This indicates there may be a large random error associated with each player’s “b.”   The Slope Rating is for some average player, but the true Slope Rating for an individual player may vary by his particular playing characteristics.

With such a small and unscientific sample as used here, no final conclusions can be drawn.  It appears the slope effect is an increasing function of a player’s handicap, but does not explain a major part of the variance in the marginal effect of the Slope Rating.

Does the Slope Effect have a significant impact on a player’s handicap?   Table 2 (column 3) below shows the player’s handicaps computed without the Slope Rating of a course (i.e., his or her pre-Slope handicap).  Column 4 shows the range of Slope Ratings where a player’s Slope adjusted handicap would be the same as his pre-Slope handicap.  Column 5 presents the percentage of the 20 courses in a player’s scoring file where his Slope Adjusted handicap and his Old Handicap were the same.   The percentage will vary depending upon a player’s index and the variance in the Slope Ratings of the courses played.  The higher a player’s index, the smaller the range of Slope Ratings that will make the two handicaps equal and therefore the greater the chance that a Slope Rating will fall outside of this range.  If a player has a small range in the Slope Ratings he encounters, the two handicaps are more likely to be equal.  In no case, however, did a player’s pre-Slope Handicap differ from his Slope Adjusted Handicap by more than one stroke.

Therefore, the efficacy of the Slope System depends upon the Slope Rating being accurate and a player having a large range in the Slope Ratings of the courses he plays.   It is an empirical question as to whether a player’s scoring record has a wide range in Slope Ratings.   If a large percentage of players have a wide range, the Slope System may be correcting an inequity.  If the percentage is small, a lot of effort t is being made for little or no increase in equity.
   
Table 2
Handicaps and the Slope System


(1)
Position

(2)
Index
(3)
Pre-Slope Handicap

(4)
Slope Range

(5)
Percent Same
Director of Handicap, Oregon GA
6.1
7
121-138
75
Director of Handicap, Washington GA
7.3
8
117-131
100
Director of Handicap, NCGA
4.5
5
113-138
80
Director of Handicap, SCGA
9.6
11
124-135
85
Exec. Director, SCGA
14.9
17
126-132
45
Executive Director, SCGA Ret.
18.2
20
122-127
55
Executive Director, USGA
3.0
4
132-155
65
Executive Director, USGA Ret.
11.6
13
122-131
55
Member, USGA Hand.  Research Team
11.3
12
116-124
75
Editor, Golf Digest
7.6
9
127-140
45
Columnist, Wall street Journal
4.9
6
127-149
90
Member, Golf Digest Course Rating
4.8
6
130-153
45
Director of Handicap, USGA Ret.
9.5
11
125-136
95
Asst. Director of Handicapping, USGA
11.0
12
119-128
55

Why does the Slope System Sell so Well? -  The Slope System was adopted in the United States with no serious opposition.  It has now spread to many more countries.  There are a few notable holdouts, but they may go the way of the Betamax as the drive for a universal handicap system gains momentum.  There are probably three reasons for its acceptance.  First, it has intuitive appeal to golfers who believe there are courses where they deserve more strokes.  Whether the Slope Rating is an accurate measure of how many strokes they should receive is not questioned.  If they get more stokes, this group is satisfied.

Second, it sounds like the Slope System was developed with scientific rigor and is therefore credible.   The ubiquitous “113,” gives the illusion of precision, though in reality it was just a number picked out of the air.    And third, since a player’s index is typically lower than his Old Handicap, the Slope System caters to his vanity.  For example, a player is no longer an 11 handicap, but a 9 index.  The 9 index sounds like a better player than an 11 handicap to the uninitiated though they are in fact equivalent.

The estimate of a player’s handicap is riddled with errors.  The USGA Handicap System contains many self-admitted biases.  There is sampling error in using only twenty scores.  The handicap estimate is also a lagging indicator of a player’s current ability.  There are numerous rounding errors made in the calculation of a player’s index and his subsequent course handicap.[4]  There are errors in the estimates of the Course and Slope Ratings that in turn lead to errors in the estimate of a player’s handicap.  Then there the errors caused by the dubious character of some players (sandbaggers).  So tacking on the Slope System will not do much harm or bring a significant increase in equity.  Incorporating the Slope System into the existing Handicap system is like “putting lipstick on a pig.”   It looks good, but the results are pretty much the same.





[1] If you are a player with a plus index, your differential should be negatively related to the Slope Rating.  This is an anomaly of the Slope System explained in Dougharty, Laurence, “Why the Slope Handicap System Doesn't Work for Plus Indices, www.ongolfhandicaps.com, August 26, 2012.
[2] The accuracy of Slope Ratings is examined in Dougharty, Laurence, “How Accurate is the Slope System,” www.ongolfhandicaps.com, October 8, 2012.
[3] Stroud, R.C., and L.J. Riccio, “Mathematical underpinnings of the slope handicap,” Science and Golf, The Proceedings of the World Scientific Congress of Golf, E.FN. Spon, London, 1990, pp. 135-140.
[4] If Player A (Index=8.9) played Player B (Index=5.1) at a benign course with a Slope Rating of 121, Player A would get 5 strokes. If the course had a Slope Rating of 146, Player A would only get 4 strokes.  Rounding produces this result which is contrary to the philosophy of the Slope System that supposedly aids the higher-handicap player on courses with a higher Slope Rating.

5 comments:

  1. Just discovered your blog… a wealth of information here!

    I’ve been developing a golf application that uses differentials from the GHIN system to forecast probabilities for how players will score in a tournament. However, when comparing these forecasts to the actual results for the tournament, I’ve discovered that there can be significant errors in the course rating. So I’ve developed a method that determines the true course rating. For example, here are the results for a tournament in Scottsdale, AZ:

    Official rating/slope: 68.4/126
    True rating/slope: 70.6/139

    That’s not an insignificant difference! Higher index golfers would receive as many as four additional handicap strokes than with the official course rating. (And in this particular tournament, a different player would have won low net.)

    I’ve forwarded my findings on to the USGA, but I’m not getting a lot of traction yet. Do you know anyone there that is receptive to new ideas? I’m thinking that a method for updating course ratings statistically, instead of manually like they do now, would be of value.

    ReplyDelete
  2. The USGA is a very rigid bureaucracy and by its nature is not receptive to new ideas. There are two reasons for this. First, the USGA takes great pride in its Handicap System. Any idea that suggests the Handicap System is not accurate will not be looked upon kindly. Second, the Handicap section is not staffed by personnel that will understand criticism based on statistics. The head of the Handicap Section has a degree in marketing. His second in command has a degree physiology and started his career as a physical trainer. There is the USGA Handicap Team, but its members have not published a peer reviewed paper in over thirty years.
    There are two posts that may be relevant to your research: 1) "The Randomness of Course Ratings" (11/16/14) and 2) "Empirically Verifying Course and Slope Ratings" (4/8/2013). Additionally Peter Preston who is associated with Golf Australia has been trying to develop a course rating system based on the scores of visitors (i.e., non members). No finished product yet. Golf Australia does have a Daily Scratch Rating (the Course Rating is changed based on scores that day) you may find interesting.
    I don't fully understand your results. If I travel to a course for a tournament, my handicap is not a function of the Course Rating. The four stroke difference in handicap comes only from the error in the Slope Rating. You believe there is a very large error in the Slope Rating. You might want to compare your test course with other courses that have a Slope Rating in the 139 range. Are the yardages and obstacle values similar?
    If all scores on which a player's Index is based were made at the tournament course (i.e., this is a member event), the Slope Rating does not matter. Using the true Course Rating would reduce every player's handicap by approximately two strokes.
    Anyway, I have empathy for the player who lost low net since that sounds like me. Keep up the good work.

    ReplyDelete
  3. Would appreciate an 'n' size for each golfer's regression. Also for model 1), why (113-rating) instead of rating/113? I don't really see any reason to use level differences.

    (very old post but I can't help but to comment, I'm reading through all your posts and learning a lot)

    In any case firm agree on the course rating system being bad and untested, but I'm optimistic that a more carefully estimated rating model would lead to a benefit for golfers, and believe in the general idea of slope.

    ReplyDelete
  4. The value of "n" is 20, the number of scores in a player's file. Slope Rating/133 would aways be positive. Slope Theory would predict that the scoring differential would go negative for Slope Ratings less than 113. For example, assume you have the ability of a 10.0 Index player. If you play a course with a lope rating of 80, it is likely your adjusted score minus the Course Rating would be negative. Model 1 takes that into account by making the independent variable become negative in certain instances.

    ReplyDelete