To begin the examination, an alternative theory to explain the difference between success rates of various shot values (par, birdie, and bogey) is posited. The alternative theory assumes the more information a golfer has about the speed and curvature of the green, the better his chances of making a putt. Golf professionals have more information when attempting par putts than birdie putts and that explains the difference in success rates. For the purposes of this paper, this theory is termed Rational Information Theory (RIT).
Both LAT and RIT predict a player will be more proficient on his par putts than his birdie putts. They differ, however, on their success rates of their bogey putts relative to par putts. LAT predicts a player will be better on his par putts than his bogey putts. RIT predicts the player will have a higher success rate on his bogey putts than his par putts. The data in Pope confirms players are better on their bogey putts than on their par putts. This directly contradicts LAT. Typically, when a theory fails to predict an experimental outcome, the theory is either revised or at least a corollary added to explain the unexpected outcome. With a certain amount of academic hubris, however, Pope ignores the theory’s failing and concentrates only on the difference between par and birdie putts as proof of the paper’s thesis.
The various analyses conducted in Pope to affirm LAT are examined. All of the models used in Pope do not include a variable measuring the information gained by a player from shots around the green. It is possible the loss aversion effect measured in Pope could in fact be the contribution of this omitted variable.
Pope also claims risk aversion is shown by the player hitting birdie putts more softly than par and bogey putts. This, it is implied, explains why players have a better success rate at par putts than birdie putts. The data shows, however, players hit bogey putts more softly than par putts (and birdie putts) and come up short of the hole more often on bogey putts than par putts. If timidity (i.e., loss aversion) hurts a player on birdie putts, it should also harm his success rate on bogey putts. Just the reverse is true, however, as players are better at bogey putts than par putts. The findings on bogey putts refute LAT, but are simply dismissed in Pope as a statistical anomaly.
The American Economic Review supposedly requires articles to describe how final data sets were developed. Pope, however, fails to adequately define the control variables making replication of the final data set impossible. The problems with several of the control variables are presented in the appendix. The appendix also contains a listing of errors found in Pope. Many of these errors are minor and do not affect the central thesis of the paper. The number of errors, however, gives an indication of the care with which Pope was written and reviewed.
Perhaps the lesson here is data from the PGA Tour should be used only for the purposes it was intended. This data is excellent at identifying good putters, long drivers, and a myriad of other golf statistics. To use the data to identify quirks in the human psyche is not likely to be productive.
Theories and Predictions – In order to assess the validity of two competing theories, we first describe each theory and the prediction it makes on the relative success rates of putts of different shot values (i.e., eagle, birdie, par).
Loss Aversion Theory (LAT) – This theory assumes a golfer will choose an optimal level of effort by setting the marginal cost of effort equal to the marginal benefit of effort when putting for birdie or eagle. However, when putting for par, bogey or double bogey, the golfer chooses a higher effort since he derives a higher marginal benefit from making those putts in the loss domain than eagle or birdie putts which are in the gain domain. There is also diminishing sensitivity in both domains—i.e., a player will try less hard on a bogey putt than a par putt. LAT would then predict:
Controlling for putt characteristics, the probability of making a birdie putt is greater than the probability of making an eagle putt. In addition, the probability of making a par putt is greater than the probability of making a bogey putt…[4]
Rational Information Theory (RIT) – This theory assumes a golfer tries equally hard on each putt. A player’s success, however, is a function of the information he has about the green. Given the nature of the game, a player will usually have more information when he attempts a putt for par than when he attempts a similarly situated putt for birdie. For example, a putt for par has usually been preceded by a previous putt or an approach shot from near the green. On most occasions, a player cannot collect the same information for a birdie putt. Therefore, RIT would predict:
A player would have greater accuracy on a par putt than a birdie putt. This prediction is consistent with LAT. RIT would predict a player is more accurate in putting for bogey than when putting for par since he typically will have more information when putting for bogey. This prediction is in direct contradiction of LAT.
Findings - The major difference between the two theories is in the relative accuracy of par and bogey putts. Under LAT, the probability of making a par putt is greater than the probability of making a bogey putt. Under RIT, the player would have a greater probability of making a bogey putt than a par putt. Interestingly, Pope would appear to validate RIT and not the premise of the paper. Table 2 of Pope shows the following estimates of the probability of making a putt compared with the probability of making a par putt.[5]
Table 2 (abridged) --The Effect of Different Shot Values on Putt Success
Putt Type
|
Probability of Making Putt Compared With Par Putt
|
Putt for Eagle
|
-0.024 (0.002)
|
Putt for Birdie
|
-0.019 (0.001)
|
Putt for Bogey
|
0.009 (0.001)
|
The table also shows the estimated probability of making an eagle putt is less than the probability of making a birdie putt, and the probability of making a birdie putt is less than the probability of making a par putt. This is what both theories would predict.
Table 2 also shows, however, the probability of making a bogey putt is greater than the probability of making a par putt. This is inconsistent with LAT, but consistent with RIT. This finding is confirmed in the ordinary least squares analysis presented in Table 3 shown in abridged form below.
Table 3 (abridged) – The Effect of Different Shot Values on Putt Success –Robustness Checks
Putt
Type
|
Model Values (OLS Estimation)
| |||||||
1
|
2
|
3
|
4
|
5
|
6
|
7
|
9
| |
Eagle
|
-.040
(.002)
|
-.039
(.002)
|
-.030
(.002)
|
-.042
(.002)
|
-.039
(.002)
|
-.036
(.002)
|
-.036
(.003)
|
-.064
(.003)
|
Birdie
|
-.036
(.001)
|
-.036
(.001)
|
-.026
(.001)
|
-.029
(.001)
|
-.028
(.001)
|
-.028
(.001)
|
-.028
(.001)
|
-.030
(.001)
|
Bogey
|
.004
(.001)
|
.005
(.001)
|
.001
(.001)
|
.003
(.001)
|
.002
(.001)
|
.002
(.001)
|
.002
(.001)
|
.006
(.001)
|
In each of the eight models presented, the probability of making a bogey putt is greater than the probability of making a par putt. The table also shows the diminishing value of information as RIT would predict. By the time a player hits his par putt, he has already gained a great deal of information from his previous shots or the previous shots of his fellow competitors. The marginal information gained when attempting a bogey putt is likely to be small. This is reflected in the very small difference in success rates between par and bogey putts (.006 in Model 8).
Pope attempts to buttress his argument with a round-by-round analysis shown in an abridged Table 5 below:
Table 5 (abridged) – The Effect of Different Shot Values on Putt Success – By Round
Value
|
4-Round Players Only
| |||
Round 1
|
Round 2
|
Round 3
|
Round 4
| |
Eagle
|
-.065
|
-.057
|
-.028
|
-.031
|
Birdie
|
-.046
|
-.035
|
-.024
|
-.021
|
Bogey
|
.003
|
.006
|
.004
|
.003
|
Pope argues:[6]
“…the finding that the round of play moderates our effect is interesting because it is consistent with our reference point story. In the first round, the reference point of par is likely to be very salient for golfers. By the fourth round, however, other reference points such as the scores of other golfers are likely to be more salient. These competing reference points are likely to diminish the influence of par on performance.”
Pope seems to admit his theory is temporal in nature. At some bewitching time, players might switch to a new reference point. Since the difference in par and birdie success is still evident in round 4, however, at least some of the players are still clinging to their “par” reference point.
A better and more coherent explanation of the declining difference in the birdie/par success rate is the declining value of additional information as the tournament proceeds. A player learns from day to day about the peculiarities of the green. By the fourth round, a player has a good idea of the speed and breaks of the green—certainly better than in the first round. Shots from around the green in Round 4 may not provide additional new information to the player as they would in Round 1. Under RIT, the difference in success rate by shot value should tend to zero as more rounds are played, and this is reflected in Table 5.
Table 5 is also inconsistent with LAT as players are better at bogey putts than par putts. Pope does not explain why this inconsistency is found in Tables 2, 3, and 5. Pope promises to revisit this curious result in the robustness section,[7] but does not do so.
In Defense of LAT - Pope contends “learning” has been accounted for. The counter claim is it has not been properly controlled for. Essentially, there is an omitted variable representing “learning” which is responsible for the findings in the paper. Pope defends LAT by arguing learning has been controlled for, and loss aversion is also shown in an analysis of how far putts of various shot values are hit. The inadequacy of each argument is described below:
1. The Prior Putts Argument – Pope assumes a player learns from prior putts when dummy variables for the number of putts already attempted on the green by the player and the player’s fellow competitors are included in the model. Pope writes:[8]
“The results of this specification suggest that learning is important. As we report in column 3 (Model 3 as shown here) in Table 3, golfers are significantly more likely to make second and third putts on the green than they are to make otherwise similar first putts on the green.[9] By including controls for prior putts on the green, the point estimates for birdie and eagle putts are reduced by 20 to 30 percent.”
Pope clearly admits learning is important. The problem is his model does not account for short approach shots around the green. If any player in the group putts from the fringe, for example, that shot is not accounted for in the model though it may yield information to the players. This is not a negligible error. If players have a 70 percent chance of being on the green in regulation, then there is a 65 percent chance at least one or more players will have missed the green. Information gathered from their (most probably) short approach shots is not accounted for in the model.
2. Eliminating the Short Approach Shot Argument - Pope does admit a player can learn from short approach shots. He attempts to correct for the deficiency in the model by:[10]
“…restricting our sample to par or birdie putt attempts with very long approach shots (more than 50 or more than 100 yards) where learning about the green is unlikely. ..Although the point estimates are smaller,…we continue to find highly significant differences between par, birdie, and eagles success when we include these controls.”
Pope does not document which controls were used in this analysis. Let’s assume the complete set of controls was included. By restricting the sample to putts preceded by long approach shots, Pope has only eliminated information a player gains from his own approach shot, and not that from his fellow competitors. Therefore, the findings are interesting, but not compelling.
3. The Matching Model Argument - Pope uses a matching model to compare birdie and par putts taken from the same spot on a particular hole.[11] Across these analyses, Pope found players made their par putts between 1.5 percent and 3.1 percent more often than they made their birdie putts. This is not unexpected and would be predicted by both LAT and RIT (i.e., a player typically will have more information when he attempts a par putt). In the matched samples, birdie putts were made approximately 85 percent of the time. In the overall sample, birdie putts are only made 28 percent of the time. The matched sample then must be made up of fairly short putts. The par putt is likely to be a second putt while the birdie putt is a first putt. The additional learning gained by the player putting for par could explain the differences found in Pope.
4. The How Hard You Hit the Putt Argument – Pope argues golfers are more risk averse when putting for birdie or eagle than when they hit par, bogey, and double bogey putts. Pope defines risk averse putts as putts that are hit short. Pope also expects birdie and eagle putts to be hit less hard than par, bogey and double bogey putts. The results of the regression analysis are shown in an abridged Table 6 below:
Table 6 – Abridged
The Effect of Different Shot Values on Risk Aversion (All Missed Putts)
Shot Value
|
Putt Length
|
Left Short
|
Eagle
|
-0.800(.032)
|
0.013(.003)
|
Birdies
|
-0.190(.080)
|
0.003(.001)
|
Bogey
|
-0.365(.190)
|
0.007(.003)
|
Double Bogey
|
-0.053(.290)
|
0.008(.004)
|
The table shows a birdie putt, on average, will travel 0.19 inches less far than a par putt all other things being equal. To give an idea of the magnitude of this measure of risk aversion, a ball will travel 5.27 inches in one revolution. The effect of loss aversion is then approximately 4/100th of one revolution of a golf ball. [12]
The table, by the same reasoning, also shows a player is more risk averse on bogey putts than on birdie putts. A bogey putt will travel 0.175 inches less far than a similarly situated birdie putt. A bogey putt, according to the table, will be short 0.4 percent more often than a birdie putt. This is contradictory to the theory of the paper, but no explanation of this curious result is given except to argue it is a statistical anomaly.
Pope does conclude:[13]
“…birdie putts are more likely to be laid up short of the hole, but they are also significantly more likely to miss the hole to the left or right.”
This would be expected under RIT. A Player has more information on the break of the putt on a par putt which tends to lower the size of the left/right error.
Appendix
Errors and Omissions
Below, issues in Pope are shown in italics followed by a correction or explanation of the problem.
p. 132 – On scorecards, golfers draw a circle for holes they shot under par and a square for holes they shot over par.
Golfers, as a rule, do not make such artistic marks on their scorecards. Television will often employ a variety of marks and colors to designate scores over or under par. And to be technically correct, a player does not keep his own scorecard.
p. 132 - After the second round golfers with a score that places them in the bottom third are eliminated.
For most tournaments, the cut is the low 70 and ties. Therefore, approximately half the field plays the final two rounds.
p. 137 -This table provides summary statistics for putts taken on the PGA Tour between 2004 and 2009.
It is not clear how Pope handled data from tournaments that differ from the norm. The International Tournament was played with a modified Stableford format that assigned large gains to birdie and eagles. Since this tournament does not fit the assumptions of the paper, it should have been deleted from the sample. In the analysis that looked at putting success over four rounds, how was data from the fifth round of the Hope Classic treated? Pope could have mentioned data were not available from the most important tournaments of the PGA season (the Masters, the U.S. Open, the British Open, and the PGA). Pope assumes putting is the same across all tournaments regardless of their importance. It is unfortunate data are not available to test this assumption.
p. 137 – In this regression, and across all of our regressions, we include a seventh-order polynomial for the distance to the hole. Goodness-of-fit tests suggest that a seventh order polynomial is necessary and sufficient to control for this important variable.
Goodness-of-fit is not an acceptable reason for adopting the functional form of a variable. There should be some theoretical reason why a seventh order polynomial should be chosen. Moreover, the particular seventh order polynomial is never delineated, nor is the estimated coefficient for this variable ever identified in any of the regressions.
p. 138 Table 2 – The coefficient for double bogey putts is consistent with diminishing sensitivity, but the coefficient for bogey putts is not.
The “coefficients” of a logit regression do not represent probabilities, but the ratio of logged odds between the dummy variable and the reference group.[14]
p. 139 – Differences in Player Ability – Some players may be good drivers (hitting long shots from the tee to the green) but bad putters, and others may be bad drivers but good putters. If this were true, player differences could account for our finding that birdie putts are less accurate than par putts. To address this question we include player fixed effects, and we report results from this regression in column 2.
What is a “player fixed effect”? In theory, Pope must determine for each of the 2.5 million putts in the sample if was hit by a good driver or a good putter. Pope gives no indication how players were put into various putting and driving classes. And to be correct, good drivers do not hit long shots to the green. Driving proficiency is rated by length, accuracy, and length and accuracy in combination. Pope probably meant players who have a relatively high percent of greens -in-regulation.
p. 140 – We restrict our sample to birdie and par putts with very long approach shots…Although the point estimates are smaller…we continue to find highly significant differences between par, birdie and eagle success…
If the sample is restricted to birdie and par putts, how can there be a finding on “eagle” success?
p. 140 - …we include separate dummy variables for the number of putts already attempted on the green by the player and his partner.
The term is not “partner” but fellow competitor. In the early rounds, a player has two fellow competitors and not one as the singular form of “partner” implies.
p. 142 –Position in the Tournament – It is possible that golfers may be more likely to attempt birdie putts when they are far behind and are exerting less effort.
This sentence makes no sense unless Pope is attempting to use the teachings of a Zen master to explain performance. There is no other reason why a player who is well behind would get more birdie chances—i.e., start hitting greens-in-regulation.
Pope does create dummy variables indicating the player’s total score if the putt is made. These variables, however, are not necessarily good indicators of a player’s position in the tournament. Being 5-under par on the first day is much different than being 5-under par on the fourth day of a tournament. Being 5-under par at the U.S. Open is far different that being 5-under at the Phoenix Open. Pope concludes the main result is not affected by these additional controls. Later (p.146), Pope argues in the fourth round competing reference points are likely to diminish the influence of par on performance. That is, players near the lead are likely to use the scores of other golfers as reference points. Pope does not explain why position in the tournament is not important determinant of birdie success on p. 142, but very important on p.146.
p. 145 Table 5 – Comparison of full sample with 4-round players only.
The putts in round 3 and round 4 of the full sample are the exact same putts in round 3 and 4 of the sample restricted to players who played 4 rounds (e.g., the number of observations is 477,732 for round 3 of both samples). Given the same sample, the estimates from the regression analysis should also be the same. The coefficients and standard errors for the double bogey dummy variable, however, differ in the round 3 regressions.
p. 148 - This figure provides a graphical illustration of where each missed putt ---taken from over 220 (sic) inches away…
Everywhere else, the distance is cited as 270 inches.
p. 153 – Panel A and B
A player in Panel A has a birdie coefficient of .032. This player is not shown in Panel B.
p. 154 - …the table (7) provides the additional earnings each player would have earned had he increased (sic) his score by one stroke per tournament relative to the rest of the golfers.
The paper should have read “decrease” his score by one stroke. This error was pointed out when the paper was in draft form.[15]
p. 154 – Table 7 – Additional earnings for Tiger Woods are reported to be $945,532.
In a draft of this paper sent to the New York Times, Pope made large errors in this Table. Pope explained to Alan Schwarz of the N. Y. Times that the estimates in Table 7 were made by a research assistant, and would be corrected in the published paper. [16] An analysis of Tiger Woods’ earnings in 2007,[17] however reveals errors still exist. Table 7 estimates Tiger Woods would have increased earning s $945,532 he improved by one stroke in each tournament. A more careful analysis presented in the table below shows, Woods would have increased earnings of $1,139,841.
Tiger Woods 2007
Tournament
|
Winnings
|
Additional Winnings if 1
Stroke Better
|
Comments
|
Buick Invitational
|
$936,000.00
|
$0
|
Won Tournament
|
WGC-Match Play
|
$130,000.00
|
$0
|
Match Play Tournament
|
Arnold Palmer Invitational
|
$51,058.33
|
$20,441.67
|
Move from a tie for 22nd to a 5-way tie for 18th. Places 18 through 22nd are paid 6.5% of the total purse of $5.5M. Tiger would have earned ($5.5 *.065)/5 =$71,500.
|
WGC-CA
|
$1,350,000.00
|
$0
|
Won Tournament
|
Masters
|
$541,333.00
|
$268,667.00
|
Move from a 3-way tie for 2nd, to second alone. Second alone would be paid $810,000. (Based on 2008 distribution results.)
|
Wachovia
|
$1,134,000.00
|
$0
|
Won Tournament
|
The Players
|
$38,700.00
|
$54,900.00
|
Move from a tie for 37th to a 10-way tie for 28th. Places 28 through 37 are paid 6.1% of the total purse of $9M. Tiger would have won ($9M*.061)/10
|
Memorial
|
$93,000.00
|
$23,000.00
|
Move from a tie for 15th to a 3-way tie for 13th. Places 13 through 15 are paid 5.8% of the total purse of $6M. Tiger would have won ($6M*.058)/3=$116,000.
|
U.S. Open
|
$611,336.00
|
$375,664.00
|
Move from a tie for 2nd to a tie for 1st. Places 1 through 2 are paid 28.2% of the total purse of $7M. Tiger would have the “expected” winnings of ($7M*.282)/2=$987,000.
|
AT&T
|
$208,500.00
|
$64,500.00
|
Move from a tie for 6th to a 4-way tied for 3rd. Places 3 through 6 are paid 18.2% of the total purse of $6M. Tiger would have won *$6M*.182)/4=$273,000
|
British Open*
|
$120,458.00
|
$113,335.00
|
Move from a tie for 12th to a 5-way tie for 8th. Places 8 through 12 are paid 13.5% of the total purse of $8.659M. Tiger would have won ($8.569*.135)/5=$233,793.
|
WGC –Bridgestone
|
$1,350,000.00
|
$0
|
Won Tournament
|
PGA
|
$1,260,000.00
|
$0
|
Won Tournament
|
Deutsche Bank
|
$522,666.67
|
$219,333.33
|
Move from a tie for 2nd to solo second. Second place would have won 10.6% of the $7M total purse or $742,000.
|
BMW
|
$1,260,000.00
|
$0
|
Won Tournament
|
Tour Championship
|
$1,260,000.00
|
$0
|
Won Tournament
|
Total
|
$10,867,052.00
|
$1,139,841.00
|
*It was assumed that the British Open purse was distributed by the same rules as the PGA Tour follows.
[1] Pope, Devin G. and Schweitzer, Maurice E., Is Tiger Woods Loss Averse? {Persistent Bias in the Face of Experience, Competition, and High Stakes, American Economic Review (February 2011), pp. 129-157.
[2] Schwarz, Alan, “Most Pro Golfers Try to Avoid the Bogey Man,” N.Y. Times, reprinted in the Bend Bulletin, Bend Oregon, June 17, 2009.
[3] Brooks, David, “We Have Much Less Control Over Ourselves Than We Thought,” New York Times, October 20, 2011.
[4]Pope, op. cit., p.135.
[5] Putts for double bogey are not considered here. RIT assumes a player gets more information the more actual strokes he takes. If a player is putting for double bogey, however, he could have incurred a penalty stroke. Therefore, a player may not have more information putting for double bogey than for par. For example, a player hits a ball out of bounds, drives again, misses the green, and then chips on. He now is putting for double bogey, but has the same information a player would have putting for par without the out-of-bounds penalty.
[6] Pope, op. cit., p. 146.
[7] Pope, op. cit., p. 138.
[8] Pope, op. cit., p. 140.
[9] Actually, Table 3 does not show a player is more likely to make second and third putts on the green than similar first putts. Pope does not report the estimated coefficients of the dummy variables representing putting order.
[10] Pope, op. cit., p. 140.
[11] Pope, op. cit., p. 142.
[12] It is not clear whether the Shotlink System has sufficient accuracy to detect such minute differences. Shotlink is operated by 250 volunteers at each tournament so “operator error” is not an insignificant problem. There are times when the laser cannot shoot a ball, and the ball is only placed in a one-yard square quadrant. Finally, if the laser operator cannot get a shot on the ball, he is asked to shoot the ground where he believes the ball was at rest.
[13] Pope, op. cit. p. 149.
[14] Pampel, F. C., Logistic Regression: A Primer, Sage University Papers Series on Quantitative Applications in the Social Sciences, 07-132, Thousand Oaks, CA, p. 27.
[15] Letter to Devin Pope from the author, June 30, 2009.
[16] Email from Alan Schwarz of the N.Y. Times to the author, July 7, 2009.
[17] In the draft paper, earnings from 2008 were used. In that year, a one-stroke improvement by Tiger Woods did not lead to a large increase in earnings (only a 4 percent increase in earnings). It is assumed Pope switched to 2007 for dramatic effect.
This comment has been removed by a blog administrator.
ReplyDelete