Thursday, September 20, 2012

Is the Slope System Worth the Effort?


 
(Note: This is the first in a series of four posts examining possible deficiencies in the Slope Handicap System and detailing efforts by the USGA to suppress criticism of the System. This post presents an empirical study on the efficacy of the System.  The next post, The USGA - The Dishonorable Company of Golfers, documents how the USGA went to great lengths to bury the study.  The third post, The Francis Sheid Affair, presents an example of the poor quality of handicap research performed by the USGA.  Lastly, How Accurate is the Slope System, uses USGA data to show the Slope System may be doing more harm than good to the equity of competition.)  
 
Introduction - The Slope System for golf handicapping has been introduced by the United States Golf Association (USGA) as a more equitable method of assessing the relative abilities of competing golfers.  This introduction, however, has not been without cost.  There has been increased data handling costs for associations maintaining handicaps, to say nothing of the time and effort spent by individual golfers and tournament committees converting indexes to handicaps.

            Are these costs commensurate with the benefits from the Slope System?  Unfortunately,. The USGA has not released any research validating the Slope System.  Instead they have produced normative articles explaining the reasoning behind the Slope System.  In essence, the golfers of the United States are asked to accept the Slope System as an of religious faith.  It is not without irony that the leading proponent of the Slope System within the USGA, and its Director of Handicapping, is known as the “Pope of the Slope.”[1]

            It is difficult to imagine why the USGA has stifled scientific inquiry on the Slope System.  Would the USGA, for example, recommend a type of grass and not make the field test public?  Of course not.  Yet the USGA has forced the Slope System on the country without providing any compelling evidence of its efficacy.

            The purpose here is to start the debate on the merits of the Slope System.  This research empirically tests the size and direction of the impact of the Slope System on the equity of competition.

            To set the state for later analysis, a brief description of the assumptions underlying the Slope System is presented.  Then two tenets of the Slope System are tested.  First, the Slope System assumes the handicap of a given player would be an increasing function of the slope rating.  Data from a club in Southern California are used to see if this “slope effect” (i.e., higher handicaps increasing proportionately more that lower handicaps on difficult courses) can be measured.

            Second, the Slope System presumes that players from “high slope” courses should have an advantage over players from “low slope” courses before the advent of the.  Inter-club matches from Southern California are examined to see if indeed players from the more difficult courses had the advantage slope theory would predict.

            Lastly, conclusions are drawn from the research to address the title of this article – Is the Slope System Worth the Effort?

The Slope System - Before the Slope System was introduced, handicaps were based on the difference between the player’s score and the course rating.[2]  The course rating was determined by the difficulty of the course for a scratch golfer.  Difficulty was estimated by a distance formula and then modified by obstacle factors such as topography, width of fairways, out-of-bounds, water hazards, trees, bunkers, and size of greens.

            This handicapping system assumed a player’s “expected” score would exceed the course rating by the same number of strokes regardless of the difficulty of the course for the non-scratch player.  If a player scored 10 strokes over the course rating at the local municipal golf course, he would be expected to exceed the course rating by 10 strokes from the back tees at Pine Valley.

            Proponents of the Slope System argued, however, the course rating may not be a good measure of difficulty of a course for a bogey golfer (i.e., a player with a 18 handicap).  Some hazards can be disastrous for the bogey golfer, but be of little consequence to the scratch player.  A water hazard in front of the tee, for example requiring an 180 yard carry may not be noticed by the scratch player, but may cause the body golfer to use up his supply of water balls.  Therefore, it was argued, another parameter should be used in handicapping to indicate the difficulty for the bogey golfer.  That parameter is the “slope” rating of the course.

            The slope system is designed to give the bogey golfer more strokes the higher the slope rating.[3]  The Slope System assumes players coming from easier courses need more strokes than their handicaps would provide when they travel to a more difficult course (difficulty will be assumed to be synonymous with high slope ratings for this discussion).  Proponents of the Slope System give the following example of two courses: Perfect Valley – a medium difficulty course with a moderate rating of 70.0, and 2)Panther Mountain – a difficult test of golf with a high course rating of 72.0.[4]

            Figure 2-1 is a hypothetical plot of scores against handicap for players from Perfect Valley and Panther Mountain playing at Panther Mountain.  The graph shows players from Perfect Valley are at a disadvantage to players with the same handicap from Panther Mountain.  This disparity increases as the handicaps of players increase.
 
Fig. 2-1 - Average Score Versus Handicap for Play at Panther Mountain

 

            Under the Slope System, every course receives a slope rating indicating the difficulty of the course.  For example, Panther Mountain might be rated at a 139 slope while Perfect Valley is rated at 113.  The old handicap differential  is converted to an index differential by multiplying the ratio of the standard course slope rating of 113 and the slope rating of the course having been played.  In equation form:

            Index Differential = (Adjusted Score - Course Rating) x (113/Slope Rating)

            A player’s index is computed by taking the average of the l0 lowest differentials out of the last 20 rounds, multiplying by .96, and then truncating the result after the first decimal.  A player’s handicap is found by multiplying the player’s index by the ratio of the slope rating of the course to be played and the standard slope rating of 113 – the resulting product is rounded to the nearest integer to find the handicap.

            Under the Slope System, players from Panther Mountain would tend to get fewer strokes when playing Perfect Valley that they do when they play their home course.  While the Slope System solves the problem depicted in Figure 2-1, three nagging questions remain:

1.    Is there a slope effect?  Figure 2-1 showed that Perfect Valley members could not play to their handicap at the tougher Panther Mountain.  Is that indeed the case?  No empirical evidence validating the slope effect has ever been published by the USGA.  If the slope effect does not exist, then the Slope System will lead to less equitable competition.

2.    Is the assumption of linearity in the Slope System reasonable?  The slope rating is based on the difficulty of a course for the bogey golfer.  The Slope System assumes the ratio of difficulty between course is linear over all levels of ability.  For examples, assume that a player from Perfect Valley had an index of 3.0 for a handicap of 3 when playing Perfect Valley.  At Panther Mountain under the Slope System the player’s handicap would be 4.  Realistically, the playing hazards for a 3 handicap should by the same as for a scratch player.  To give this player an extra stroke at Panther Mountain because of difficulties encountered by the bogey golfer is an assumption embedded in the Slope system.

If the slope rating adjustment is justified for high handicappers but not for low handicappers, then the Slope System would lead to less equitable competition for the latter group of players.

3.    Can the Slope Rating be estimated accurately?  Even if the slope phenomenon exists, if it cannot be estimated with sufficient accuracy, the Slope System could lead to less equitable competition.  For example, assume two courses actually had the same slope rating.  If one course was rated higher than the other, then that club would be at a disadvantage when playing members for the course with the lower rating. 

            These questions are difficult to answer, but the empirical studies can shed some light on these issues.  

 

A One-Course Test of the Slope System - To test the slope hypothesis represented in Figure 2-1, it would be necessary for members of one course (e.g. perfect Valley) play a large number of rounds at another course (e.g. Panther Mountain).  Then statistical methods could be used to test whether scores of the visiting players followed the slope line shown in the figure.  Such a test has two problems.,  First, few clubs would want to give-up so many starting times for visitors from another course.  Second, the scores by the visiting players are likely to be higher because they lack local knowledge.  If there is an upward bias in scores due to a local knowledge factor, it would be difficult to isolate the “slope effect.”

            One way overcome these two problems is to have players from the same club play two different sets of tees at their home course.  In essence, one set of tees becomes Panther Mountain and the other set Perfect Valley.  By using the same course for the test, moreover, the bias of “local knowledge” is eliminated. 

            To test the slope hypothesis data from a private club in Southern California was collected.  Of approximately 500 members, 59 members had a sufficient number of rounds (20) within one calendar year from both sets of tees to compute a handicap.  Handicap were computed for each player from the scores from the blue tees (slope 119, 6520 yards) and from the white tees (slope=111, 6059 yards).  Handicaps were computed to the nearest tenth rather than the nearest integer in order to preserve any minor variation that may be due to the slope phenomenon.

            A player’s handicap from the white tees was plotted against the difference between the player’s blue tee and white tee handicaps as shown in Figure 2-2.  Under slope theory, these points should follow a line with a zero intercept (i.e. the scratch golfer is not affected by the slope rating) and with a slope equal to the ratio of the two slopes minus one (i.e. ,119/111 -1) or .072.  That is the handicap from the blue tees should exceed that from the white tees by approximately 7 percent.


Figure 2-2 - Handicap Difference Versus White Tee Handicap

 

In equation form:

 

            DIFF = a + b WTH                                                                                                     

 

Where,

                                 DIFF= Difference between a player's blue and white tee                     handicap

                                 WTH= Player's Handicap from the White Tees

 

The coefficient "b" represents the slope effect.  .

            The coefficients of this model were estimated using standard linear regression techniques.  The estimated equation was:

 

            DIFF= .89 - .024 WTH                                                                                               

 

            The estimated model shows a slope of -2.4 percent rather than the 7 percent slope theory would predict.  The hypothesis that the true slope is 7 percent can be rejected at the 1 percent level of significance.  An alternative hypothesis is there is no slope phenomenon at work so that the coefficient "b" should be zero.  This hypothesis cannot be rejected at the 20 percent level of significance.  In other words, it is likely that there is no slope effect present.

            The introduction of the Slope System has not led to more equitable competition at this club.  White tee players are given additional strokes when playing the blue tees.  Similarly, strokes are taken from blue tee players when they play the white tees.  Neither adjustment can be justified by the data.  The Slope System, in this one case, has made competition less equitable.

            Clearly the results form one club out of thousands in not definitive.  It is hoped, however that the results will encourage the USGA and other golf associations to conduct similar studies.  The methodology used above would be helpful in determined if the slope effect existed and the accuracy of the relative slope ratings among different sets of tees at a course.

 

An Inter-Club Test of the Slope System - A purported major benefit of the Slope System is its elimination of the handicap advantage that players from difficult courses enjoyed over players from easy courses.  To test for the existence of this advantage, data from inter-club team matches sponsored by the Southern California Golf Association (SCGA) were used.  If the slope theory is valid, the courses with higher slope ratings should have had an advantage before the introduction of the Slope System.  The introduction of the Slope System should have erased any advantage belonging to the higher slope courses and led to more equitable competition.

            Data from three years of team matches were analyzed to test the influence of the Slope System on team match results.  In the years 1988 and 1989 the SCGA was not on the Slope System.  Therefore, an advantage for teams from higher slope courses would be expected in those years.  With the introduction of the Slope System in 1990, the advantage would be expected to disappear.

            Before examining data from the team matches, an explanation of how they are scored is in order.  A total of 96 points is available in two matches played on a home-and-home basis (48 points at each club).  The impact of the Slope System was estimated using a model where the number of points won over 48 (i.e., the number of points won over those expected by chance) is considered to be a linear function of the difference in course Slope Ratings.

            Slope theory would predict that courses with higher slopes would do better in the years before the Slope System was introduced.  The size of the impact would be measured by the coefficient of the variable measuring the difference in slope.  In equation form the model was:

 

            (Score1 - 48) = a + b(Slope1 -Slope2)                                                                      

 

Where,

                        Score1 = Points won by Course 1 Against Course 2

                        Slope1 = Slope of Course 1

                        Slope2 = Slope of Course 2

 

            The SCGA conducts competition for both Thursday and Saturday teams.  The data from three years for both teams were fitted to the model using standard linear regression techniques.

            In 1988 (a pre-Slope System year) the coefficient measuring the impact of the slope rating (i.e., "b") was not statistically significant.  That is, teams from courses with higher slopes did not outperform teams from courses with lower slopes.

            In 1989, the slope coefficient was significant for both Thursday and Saturday Team results.  The impact was small, however.  The model predicts a ten-point differential in slope (e.g., 128 versus 118) would lead to only a total differential of three points over the 96 point match.

            In 1990, the first year of the Slope System, the slope effect would be expected to disappear.  That is, the Slope System should have equalized competition.  The slope coefficient was not significant (10 percent level of significance) indicating that the slope was not a factor in explaining match results.

            Any conclusion concerning the efficacy of the Slope System depends in part on which two years are being compared.  Between 1988 and 1990, the introduction of the Slope System had no measurable impact on the results of team play.  Between 1989 and 1990, however, the Slope System appears to have made a minor contribution to more equitable competition.  The inconsistency and the small size of the impact, however, raises doubts about the net benefits of the Slope System.

            Another test of the impact of the Slope System is to examine whether “higher slope” teams were more successful in team play.  Table 2-1 presents the winning percentage (i.e., where the team score 49 or more points) of higher slope competitors.

Table 2-1

Winning Percentage of Higher Slope Teams

 
 
Winning Percentage
 
Year
 
Team
Slope Difference
>0
Slope Difference
>9
1988
Saturday
.469
.607
1988
Thursday
.500
.516
1989
Saturday
.533
.632
1989
Thursday
.504
.628
1990
Saturday
.541
.520
1990
Thursday
.508
.607

 

            The winning percentage is computed for two cases: 1) the slope difference is greater than zero, and 2) the slope difference is greater than nine  In the case where the slope difference is greater than zero, teams with a higher slope do no not have a significant edge in any of the years studied.

            When the set of matches is reduced to those where the difference in slope is greater than nine, the teams from the higher slope courses do better than chance would predict.[5]  But, most important, it appears they do better over all years.  That is, any slope effect was not corrected by the Slope System.

            In summary, the Slope System has not led to any large or consistent improvement in the equity of team play competition.

 

Conclusions - No definite conclusion on the Slope System can be drawn from the two tests reported here.  The research did show, however, that little benefit has been brought to the equity of competition in the two instances.

            In the “one-course test” of the Slope System problems in measuring the slope were revealed.  Since the Slope System relies in large part on human judgment, the possibility of error seems quite large.  Can the formulae used to calculate the slope really distinguish between courses?  Can ratings committees consistently and reliably distinguish between a 130 and 125 slope rating? 

The results of this research suggest that such distinctions cannot be made with precision.  In the one course test no actual difference in slope was found, the slope ratings differed by eight points.  In the test using inter-club matches, any effect of the Slope System was marginal at best.

There may be something else at work other than the slope effect.  In the inter-club matches, teams with high slopes (i.e., greater nine point differential over their competitor) continued to do well even after the introduction of the Slope System.  One possible explanation is that higher slope course may also have a “local knowledge effect” (LKE).  LKE means that players score gradually does down with the number of times a course is played until a steady state is reached.  High LKE are associated with courses where there is a large penalty for choosing the wrong club, where there is strategic value in being on certain parts of the fairways and greens, and where breaks in the green are not easily read. Figure 2-3 show the pattern of scores for courses with high and low LKE.  If the slope rating is based on the steady state score then the LKE may overwhelm any contribution made by the Slope System.

Figure 2-3 - Score Versus the Number of Rounds Played
 

            The failure of the Slope System may also be due to the wide variety of “bogey golfers.”  Some bogey golfers are slicer while others are hookers.  Some hit the ball short and straight while others hit long and wild.  A course should have a slope rating for each category of bogey golfer, but instead a slope rating for a mythical average bogey golfer.  This implies the slope rating will be too high for some and too low for others.

             For those for who the slope rating is too low, the Slope System will underestimate their ability.  An example will make this clear.  Assume a golfer who slices the ball on a course where all the trouble is on the right.  The slope rating for the average golfer at this course would be 120, but for the slicer the slope rating should be 130.  His index differential is found by adjusting is handicap differential by the ratio of 113/120.  His index will then be higher than if the “true” slope ratio of 130 were used.  His ability on away courses is underestimated (i.e., his handicap is overestimated)f by the Slope System.  The effectiveness of the Slope System depends in part upon a uniformity among bogey golfers.  The USGA has never presented any evidence that this assumption is met.

            The Slope System brought more complexity to the handicapping system, but not necessarily more accuracy in evaluation the ability of competing golfers.  The evidence presented here cannot say conclusively that the Slope system was not worth the effort, but it certainly raises doubt.  It also raises the question  as to the size of the contribution further refinements in the handicapping system can make.  The inherent measurement errors in course ratings and the wide distribution in golfing characteristics among players of the same scoring ability may make refinements such as the Slope System only second order corrections at best.



[1] Golf Digest, June 1991, p. 146.  This referred to Dean Knuth who has since been replaced as the Director of Handicapping).
[2] Handicaps were then determined by first taking the lowest 10 differentials (adjusted score minus the course rating) out of the last 20 scores.  The total of the 10 differentials was divided by 10 to find the average differential.  The handicap was computed by multiplying the average differential by .96 and rounding to the nearest integer.
[3] This is true for stoke play, but not necessarily true for match play.  Under the Slope System a low handicapper can give a higher handicapped fewer strokes on a more difficult course.  For example a 10 index would give a 15 index 6 strokes on a course rating of 117.  On a course with a slope rating of 131 (i.e., a more difficult course he would only have to give 5 strokes.
[4] Stroud, R.C., and L.J. Riccio, “Mathematical Underpinnings of the Slope Handicap System,” in Science and Golf: The Proceeding so the first World Scientific Congress, Rutledge, Chapman, and Hall, London, 1990.
[5] Pooling data over all years and teams, the teams with a greater that nine point slope differential won 58.7 of the time (128 out of 218 matches).  The null hypothesis that the probability of winning is less than or equal to .5 can be rejected at the 2.5 percent level of confidence.

No comments:

Post a Comment