Empirically Verifying Course and Slope Ratings

Most golfers have played a course where they thought the Course Rating and/or the Slope Rating were too low.  These ratings errors would explain why their scoring differential was so much higher than their index.  The golf association responsible for the ratings, however, would argue the ratings are accurate and the player’s poor performance is more properly explained by the random nature of scoring.[1] 

Course and Slope Ratings are determined through the use of estimating models.  Estimating models in both the hard sciences and the social sciences are validated by how well they predict.  There are models, for example, that attempt to predict the winning team in games of the National Football League.   If the predictions of a model are better than chance over a long period of time, the model is said to be empirically validated.  Interestingly, the United state Golf Association (USGA) has never published any evidence validating it Ratings models.

There are probably two reasons for the USGA’s lack of diligence.  First, a research finding that the models are inaccurate would tarnish the image of omnipotence  the USGA likes to project. Secondly, there are severe theoretical and logistical problems in designing an experiment to verify the ratings. These problems have been discussed in a previous post – How Accurate is the Slope System? [2]  In theory, a large group of scratch players could play a course many times and it would be possible to solve for the Course Rating that would make these players scratch at this course.  It is not known with any certainty, however, that these players are actually scratch or came about that handicap due to an error in the Course Rating at their home courses.  The logistical problem is getting a reference group (i.e., the scratch players) at least 400 (a minimum of 20 players having 20 rounds each) starting times at the test course. Because of these problems, golf associations cannot be easily challenged on the accuracy of their Ratings. 

There is one special case, however, that eliminates the logistical problem and allows for the examination of Rating errors.  That case is where a golf club has two or more courses.  By examining the performance of the same golfers on different courses, it should be possible to estimate the size of any errors in the Ratings.    Slope Theory assumes the average differential ((Average Score – Course Rating) x 113/Slope Rating) at each course should be the same for a player.  To test the hypothesis that a player’s differentials are equal, data were collected on 34 four players who alternate playing two courses under competitive conditions.  The courses are named Course1 and Course2 in this analysis.  A player’s Average Course1 Differential is plotted against his Average Course2 Differential in Figure 1 below.

  Figure 1 – Average Course1 Differential versus Average Course2 Differential

If Slope Theory is correct the regression line should have a zero intercept and a slope of 1.0.  In other words the best estimate of the Average Differential on the Course 1 is the Average Differential on the Course 2. The regression line fitted to the date shown in Figure 1 was estimated to be:

                1)            Avg. Course1 Differential = 2.1 + .85 · Avg. Course2 Differential

The regression does not have a zero intercept.  The constant “2.1” in the regression equation, indicates a possible error in the Course Ratings.[3]

Rewriting the equation 1,

                2)            Avg. Course1 Differential - .85 · Avg. Course2 Differential = 2.1

To eliminate the constant, 2.1, the average differential on Course1 must be decreased or the average differential on Course2 increased or some combination of the two.  A reasonable assumption is that the error in the fixed component should be split equally between the two courses.  Therefore, the Average Course1 Differential is reduced by 1.0 (i.e. Course1 is more difficult than it is rated), and the Average Course2 Differential is increased by 1.2 (i.e., Course2 is easier than it is rated).  The plot of these adjusted differentials for each player is shown in Figure 2 below.

Figure 2 – Average Course1 Differential Reduced by 1.0 versus Average Course2 Differential Increased by 1.2

The new regression equation is:

                3)            Average Course1 Differential = 0.03 + .85 Average Course2 Differential

The fixed component of the relationship is now close to zero.  The ratio of the two differentials, however, should be 1.0 and not .85.  To achieve that end, the Slope Rating of Course2 would have to increase or the Slope Rating on Course1 decrease -- or some combination of the two.  There are a large number of new Slope Ratings that would equalize the ratio of adjusted differentials.[4]  Again, it is assumed the errors in Slope Ratings are approximately the same size though in different directions.   The postulated Slope Ratings under this assumption are 115 for Course1 and 132 for Course2.  The plot of the differentials adjusted for the new Slope Ratings is shown in Figure 3.  The new regression line is:

                4)         Average Adjusted Course1 Differential = 0.2 +.98 Average Adjusted Course2    Differential

The new regression line eliminates the fixed component, makes the ratio of the two differentials approximately equal to 1.0, and therefore satisfies the requirements of Slope Theory.

Figure 3 – Plot of Differentials Adjusted for New Slope Ratings

Given these adjustments, what would be the estimates of the new Course Ratings?  To reduce the Average Differential on Course1 by 1.0, the Course Rating must be increased by 1.0 x 124/113 or 1.1 strokes.  The new Course Rating on Course1 would be 71.1 (from 70.0).  To increase the Average Differential on Course2 by 1.2, the Course Rating would have to be decreased by 1.2 x 122/113 or 1.3 strokes.  The new Course Rating on Course1 would be 68.3 (from 69.6).[5]  A summary of the estimated and existing Ratings are presented in the table below.

Table

Estimated and Existing Ratings

Course	Course Ratings		Slope Ratings
	Estimated	Existing	Estimated	Existing
Course1	71.1	70.0	115	124
Course2	68.3	69.6	132	122

The estimated ratings in Table 1 are purely speculative.  There are a large number of Ratings that would equalize player differentials between the two courses.  It does appear, however, that the current Ratings are not correct.  There are three possible explanations for the apparent errors:

1. The Ratings are in Error – As discussed in How Accurate is the Slope System?, the Course Rating models of the USGA are seriously flawed.  The models are deemed so imperfect the results can be changed by the Course Rating Review Committee at each golf association. [6]

2. Sampling Error – The small size of the sample (34 players) makes any conclusion tentative at best.

3. Slope Theory Is Not Valid - Under Slope Theory, a players’ average score at a course is a linear function of his index. All players have their index increased (or decreased) by the same percentage (Slope Rating/113) to find their handicap. But what if a player’s average score is a curvilinear function of his index? In that case, even if the Course Rating, Bogey Rating, and Slope Rating were estimated with precision, the average differentials at the two courses would not necessarily be equal.

There is anecdotal evidence that reason 1 is the most plausible explanation.  Individual and team (2 best balls of 4) net scores are generally lower on Course2 than on Course1.  The current Ratings would not predict such a disparity in net scores.   Reason 2 can only be eliminated by expanding the sample size.  It would be helpful if a golf association with access to the General Handicap Information Network (GHIN) would undertake a similar study.  Since this is unlikely, the most reasonable course of action is to repeat the experiment each year and examine whether the results are consistent across time.    

               

---

[1] The USGA maintains the standard error in the Course Ratings is .285 strokes, and therefore an error of more than .6 strokes is extremely unlikely.  (See Knuth, D, “A two parameter golf course rating system,” Science and Golf, edited by A.J. Cochran and M.R. Farally, E & FN Spon, London, 1990, p. 145.)

[2] See Ongolfhandicaps.blogspot.com, October 8, 2012.
[3] The constant 2.1 is only statistically significant at the 90 percent level of confidence. The variable coefficient (.85) is statistically significant at the 95 percent level of confidence.

[4] The current Slope Ratings are 122 for Course2 and 124 for Course1.  The ratio of the adjusted Course1 Differential to the Adjusted Course2 Differential is .85.  To make the ratio of the adjusted differentials equal to 1.0, the ratio of the new Course2 Slope Rating to the new Course1 Slope Rating must be approximately 1.16.

[5] The errors seem large and the USGA would argue they are not within the realm of reason.  What is often forgotten is that a small error in the Course Rating can lead to what looks like a substantial error in the Slope Rating.  Let’s look at the Course2.  The Slope Rating is given by the formula:

                Slope Rating = 5.381 (Bogey Rating – Course Rating)

If the Course Rating is overestimated by 1.3 strokes, the Slope Rating will be underestimated by 7 rating points.  That would put the Slope Rating at 129 which is not too far from the 132 used in this analysis.

[6] USGA Handicap System, 2012-2015, Unites States Golf Association, Far Hills, NJ, p. 98.