“What a coach does is attempt to increase the index of probability when it comes to winning a match. As a coach all you can [do] is deny fortune as much of its role as you possibly can”
Such was the philosophy Juanma Lillo expressed in a great interview by Sid Lowe in Issue 1 of The Blizzard. One of the main ways a coach or manager can do this is a team built through more costly transfers each year. Earlier this year the Transfer Price Index was used to show that a greater percentage of the average Premier League club is made up of transfers with each passing year, that contrary to Soccernomics’ assertion transfer costs do indeed correlate to success, and it demonstrated that an escalating squad and starting XI cost (in terms of transfer fees) is required to finish in the top of the Premier League Table. Without a doubt, the cost of fielding a competitive Premier League side in the long-term is escalating rapidly in terms of transfer fees and shows no sign of slowing down.
But what about the short term? How much of an effect does a team’s cost in terms of transfer fees have on their ability to earn points in a match? To go back to Juanma Lillo’s quote, how much money does a club need to spend on transfers to “increase the index of probability when it comes to winning a match”? Luckily, Graeme Riley has been able to provide just the data set to answer that question. This post uses that data to provide a mathematical model for the effect of club transfer costs on match outcome. The model is then used to examine the biggest mismatches and upsets, as well as summarize two very different experiences this past season: Liverpool under Hodgson and Dalglish.
Graeme’s data set is quite comprehensive. It has the following data for every Premier League match since the league was formed in 1992:
- Home Team
- Away Team
- Goals For Home Team
- Goals For Away Team
- Home Team Starting XI Cost (2011 CTPP in £’s)
- Away Team Starting XI Cost (2011 CTPP in £’s)
The match data was then split up into two data points each – one for each team involved in the match. The goal differential in the match was converted to a match outcome – loss, tie, or win. Each pair of teams in a match were assigned a value for venue – 1 indicated the home team, 0 indicated the away team.
It is well understood that home teams have a higher probability of tying and/or winning versus away teams. This means that any mathematical model must take into account venue to accurately reflect the effects of any other variables. This reality will be reflected in a later discussion of the model and its outcome.
There are a variety of options when it comes to evaluating the impacts of transfer costs on match outcome. In previous posts, squad and starting XI costs were converted into a multiple of the league average for that season. That made sense when seasonal data was being used, but it doesn’t make much sense when deriving a mathematical model for individual match impacts. What matters is the relationships within the individual matches. The difference in squad costs, that is (Squad Cost 1 – Squad Cost 2), could be used. Similar to previous posts, the ratio between the two squads costs could also be used. Both were tested within the statistical model, and it turns out the ratio of the two teams starting XI transfer costs is the statistically significant term of the two. This ratio will be used to evaluate impacts of starting XI cost on match outcome. The match £XI ratio will be abbreviated via the term “m£XIR” in the remainder of this post. As an example, in the October 3rd, 2010 match between Liverpool (£XI = 108.7M) and Blackpool (£XI = 4.1M) yields m£XIR’s of 26.55 for Liverpool and 0.038 for Blackpool.
In line with Pay As You Play, the data was divided into two eras. All seasons through 2002/03 were labeled as “pre-Abramovich”, while seasons 2003/04 or later were labeled as “post-Abramovich”. The two data sets were normalized to show the percentage of data in each histogram bin as the sample sizes for the two eras are nowhere near equal. Summary statistics and a histogram of m£XIR were generated for each era, a plot of which is shown below.
The historgram shows graphically what we have known intuitively – the gap between the rich and poor clubs in the Premier League has grown since Roman Abramovich bought Chelsea. This is evidenced by the higher percentages of higher m£XIR values in the post-Abramovich era – the red line is above the black line for all m£XIR values above 5.0. Harder to see is the similar increase in extremely low m£XIR values – these are the clubs facing those with the higher m£XIR values given the inverse relationship between two teams’ m£XIR values in an individual match. Because of the increase in the percentage of these two extremes, the percentage of values in the peak region around 2.0 went down in the post-Abramovich era.
Another way to quantify this is in looking at the summary statistics in each box. The standard deviation (σ) is higher in the post-Abramovich era, and the interquartile range (Q3-Q1) that represents the bounds of the middle 50% of data is also wider in the post-Abramovich era. Also notice the maximum values of the two eras – through the 2002/03 season the maximum m£XIR for a match was 33.4, while fewer than eight years later it had increased more than six-fold to 206.5!
All of these statistics have important impacts on any mathematical model generated from the data. Clearly, a shift in m£XIR values has happened since Roman Abramovich purchased Chelsea. To ensure the highest possible accuracy of the model, the focus of the model will remain on the post-Abramovich data.
The Model: Ordinary Logistic Regression of Venue and m£XIR
This blog post opened with Juanma Lillo’s quote that emphasized “increas[ing] the index of probability when it comes to winning a match”. The mathematical model used to quantify the impacts of m£XIR on match outcome does just this – it estimates the probability of match outcome. The statistical method used to create the model is called ordinal logistic regression, which sounds like a mouthful of statistical gobbledygook but is understandable if it is broken down into its basic parts.
- Ordinal (or ordered): the set of outcomes in the model has a particular order or hierarchy (low, medium, high; 1,2,3). In the case of this model it is lose (L), tie (T), win (W).
- Logistic: This simply means that a logarithmic function is involved, in this case the natural logarithm. One might hear such a model’s output described as the “log odds” – there’s that probability (or odds) term again!
- Regression: Here’s a term with which many people are familiar, although it’s often linear regression with which they’re most familiar. In this case, regression is a general term used to describe a mathematical model quantifying the relationship between input and output variables.
Combining all the above definitions, an ordinal logistic regression model of m£XIR may be described as a regression model that uses the inputs to the model to generate the (log) odds of a team falling into one of three match outcomes: lose, tie, or win. What’s very useful about such a model is that it can easily be translated to a real world metric with which everyone is familiar: match points. If probability of an event is described by P(X), the following equation can be used to translate the probability of match outcomes to expected points.
Expected Match Points = 3*P(W) + 1*P(T) + 0*P(L)
Therefore, a relationship between a club’s match venue (home or away) and m£XIR can now be directly translated to expected match points.
One final comment should be made before a dissection of the post-Abramovich data takes place. The fact that the model uses the m£XIR data as an input to the odds (or probability) model requires that the most accurate inputs be used. To illustrate this point, ordinal logistic models were created to demonstrate the effect on expected points from a match and swept from the minimum to the maximum values of m£XIR throughout the eras. Three models were used to evaluate the impact of m£XIR – pre-Abromvich, post-Abramovich, and one that used all the data from the nineteen years of the Premier League (All Years). The graph below shows the results for away matches – similar trends are observed in home match data.
Notice that the “Pre-Abramovich” and “All Years” data level off to the maximum value of 3.0 well before the Post-Abromoich data does. This is due to the fact that Pre-Abromovich data is included in both of those analyses, and it weakens the impact of the much higher m£XIR values seen in the post-Abramovich era. In reality, we know that the gap in squad transfer costs has grown in the post-Abramovich. Using data from the prior era would create a model that would over-estimate the expected match points for lower m£XIR values from what we know the reality to be over the last eight seasons. Based upon this analysis, one could argue that an individual model should be created for each post-Abramovich season as we’ve seen the number of teams spending “Chelsea money” escalate the last few seasons. For the purposes of simplicity I have stuck with a single model in this series of posts – the conclusions aren’t any different than if models had been created for each season.
With that explanation out of the way, let’s dive in to the results of the post-Abramovich model.
The two graphs below summarize the probability of match outcome based upon m£XIR – the first for home matches and the second for away matches. Note that the plots have been restricted to m£XIR values less than or equal to 10. This is done to ensure that that the plots for probabilities remain in the linear region of the model – recall from the previous graph that once m£XIR values are greater than 10 they become non-linear. Even with such a compromise, more than 90% of the m£XIR values in the post-Abramovich are covered in the graphs below.
There are several conclusions that can be derived from the above graphs:
- Venue still matters. An away team must have an m£XIR of 7 to 8 to have a probability of winning equal to a home team with a starting XI transfer cost equal to the opposition (m£XIR = 1.0). Home teams never have a higher chance of losing than winning.
- An away team must have an m£XIR greater than 5 to have a greater chance of winning a match than losing it.
- Advantages in m£XIR for home teams primarily come in probabilities of winning at the expense of losing. For away teams, it comes in probabilities of winning or tying at the expense of losing, with winning not exceeding tying until m£XIR is greater than 2.5.
What are currently two plots of three lines each can be translated into one plot of two lines using the expected points/probability relationship described earlier in the post. Such a graph is provided below.
A mathematical relationship between points and starting XI transfer cost has now been established. For every multiple of m£XIR within a match, a team’s expected points increase by 0.1. This relationship holds no matter the venue – only the total number of points expected changes by venue. It also establishes the importance of picking up points for the likes of the Big Six against clubs like Blackpool, West Bromwich Albion, and Wolverhampton (the three lowest average £XI’s in the 2010-11 season). Picking up points from them at home is expected, while winning on the road helps pad their lead versus other teams in the league that don’t spend as much on transfers. This also produces an arms race of sorts, given what seems to be relatively low returns for each multiple of £XI. A club must spend “Chelsea money” to have a large impact on their expected match points – what matters is very large gaps to the majority of the clubs in the league if the end goal is higher expected match points.
An increase of 0.1 for each multiple of m£XIR may not sound like a lot, but it does add up over a season. The table below shows the average m£XIR for each club in the Post-Abramovich era, arranged alphabetically by club.
Take Chelsea and Liverpool as examples of how m£XIR advantages add up over a season. Given Chelsea’s average m£XIR value of 9.78, they are projected to earn nearly one additional point per match than teams evenly matched in terms of £XI. Over a 38 match season, this means they earn nearly 38 additional points due to their transfer expenditure advantage. Liverpool has about half of Chelsea’s m£XIR advantage on average, resulting in 18 additional points. These numbers certainly fluctuate match-to-match with m£XIR, resulting in a deviation from what the average value would have predicted. The low resolution of a match outcome – only 0, 1, or 3 points can be earned – also contributes from deviation from the predicted value. Nonetheless, the model quantifies the average effect team payroll has in pre-determining the average match outcome. It’s quantifiable, and it is significant over the course of a season.
The model can now be used to analyze past results to identify the biggest mismatches and upsets, as well as evaluate Liverpool’s 2010/11 season under two different managers.
Biggest Mismatches and Upsets
The first comparison made via the model is the identification of the biggest financial mismatches in the post-Abramovich era. In this study, a mismatch is measured via the m£XIR – the smaller the m£XIR, the bigger the financial mismatch. Data for every match was arranged in ascending order of m£XIR. The top ten mismatches are presented in the table below. Within the table the following terms are used:
- Date: Date of match in month/day/year format
- Team1: team with lower £XI
- Team 2: team with higher £XI
- Home?: Denotes whether Team1 is home (1) or Away (0)
- £XI Difference (2011 CTPP): defined by (£XI Team 1) – (£XI Team 2)
- m£XIR: defined by (Team 1 £XI)/(Team 2 £XI)
- P(X): Probability of X, where L = Lose, T = Tie, and W = Win
- Expected points: Expected match points defined by 3*P(W)+1*P(T)+0*P(L)
The Team 2 column is populated by the expected clubs – Chelsea and Manchester United – but a few surprises can be found. The first is in the Liverpool/Watford match taking the top spot. Watford had a miniscule £518,626 worth of players in their starting XI for that match (Mahon and Stewart), which meant their talent on the pitch cost less than one half of one percent of Liverpool’s. Watford brought in two of their more costly players, Henderson (£1.05M) and Hoskins (£1.11M), right after halftime. It was all in vain though – Liverpool were already up 2-0 at the half due to goals from Bellamy and Crouch and they added a third in the 48th minute from Crouch to sail to a 3-0 victory in front of 19,746 at Vicarage Road. Watford was similarly mismatched away from home less than three weeks later when they faced Manchester United at Old Trafford on January 31st, 2007. This was the 10th largest mismatch, ending in a 4-0 loss for Watford. Over an 18-day period Watford had a -7 goal differential while facing a combined £333M in talent beyond what their small club was able to put on the pitch.
The other outlier of sorts in the mismatch category is Bolton’s 1-0 win against Newcastle on March 28, 2004. Bolton only had six players all season that had cost them any transfer money since earning Premier League promotion, and three of them started that day (Jaaskelainen, N’Gotty, and Pederson) for a £XI of £2.3M. No matter Newcastle’s advantage in costly personnel and their form in the UEFA Cup that season, they conceded the only goal of the day. This loss played a roll in Newcastle missing out on automatic Champions League group stage qualification at the end of the season.
The remainder of the spots are held by the big spenders (Chelsea, Manchester United) and all time lowest spenders (2004/04 Bolton and 2010/11 Blackpool). Bolton was hit especially hard by absences in the month of March 2004, as demonstrated by three of their five appearances coming during the time period that also happened to coincide with matches against well financed Arsenal, Newcastle, and Chelsea.
What about the biggest upsets? Venue still plays a large part in determining match odds, thus the biggest upsets all come from teams playing away from home. The data set is sorted for away teams that won, and ordered from lowest to highest m£XIR. The table below displays the top 10 upsets of the post-Abramovich era. All column definitions are the same as the previous table, except “Deviation” which represents the difference between actual points earned and the expected points.
This table is a bit more straightforward than the last one. Yet again, Bolton from 2003/04 and last season’s Blackpool team dominate the list of teams doing the upsetting. It didn’t seem to matter who they beat, as the any and every team from those seasons that was beat on their home turf by Bolton or Blackpool shows up in the Team 2 column. It’s the rarity of the away win that makes for such a mix of clubs. The difference between the two clubs’ season-long performances couldn’t be more stark though. Blackpool finished in 19th position and was relegated this season with a nearly identical home and away record (H: 5-5-9, A: 5-4-10) while Bolton finished 8th in 2003/4 on the strength of their road performances (H: 6-8-5, A: 8-3-8).
A Tale of Two Half Seasons: Liverpool, 2010/11
Beyond summarizing under and over performance in the pre-Abramovich era, the model can also be used to analyze individual club’s performance throughout an entire season. Given the two wildly different half seasons that Liverpool saw in 2010/11, it is only appropriate to see how they performed against the model. The graph below provides 4-match moving averages for Liverpool’s match points as well as expected match point differential versus the m£XIR model (defined by actual points – expected points). Using a 4-match moving average provides a picture of how the club’s performed over the previous month’s worth of matches.
Using the above graph we see numerically why Hodgson was sacked just after the new year, and how well Kenny Dalglish did after taking over the club.
A record of 1-3-4 through the first eight games had Liverpool relegation bound and averaging about a quarter of a point per match when Hodgson reached his nadir for the season via a loss at Everton on October 17, 2010. At this point in the season Liverpool were averaging about 1.66 fewer points than expected per match based upon their m£XIR. Perhaps a loss at Manchester United on September 19th could have been tolerated, given the match was at Old Trafford and the Red Devils had a £57.6M advantage in terms of £XI. However, a September 25th tie at home against Sunderland (£68.4M £XI advantage for Liverpool) and an October 3rd loss at home to Blackpool (£105M £XI advantage for Liverpool) surely were under performances versus financial expectations. By the time October 17, 2010 rolled around, Hodgson had already left more than 6 points on the table compared to the performance predicted by the model.
A run of wins against Blackburn, Bolton, and Chelsea and a tie away to Wigan from mid-October to mid-November resulted in a brief bump in match points for Hodgson, which also moved him back to the good when it came to expected match point differential. The positive news was to be short lived. Beginning with a November 13th loss to Stoke, Hodgson’s Liverpool proceeded to a 3-0-5 record over it’s next eight matches. Losses to Stoke, Spurs, Newcastle, Wolves, and Blackburn more than cancelled out the wins against West Ham, Aston Villa, and Bolton. By the time he was sacked, Hodgson was back to earning less than a point per match (averaged over the previous four matches), while his under performance versus the model was back to the nadir of two months prior. Fenway Sports Group had seen enough, and Hodgson got sacked after the Blackburn match.
Hodgson famously claimed “no one can do a better job than me at Liverpool – unless owners splash the cash“, yet Kenny Dalglish did just that with the exact same squad except for the Torres-for-Carrol-and-Suarez swap. Paul has written plenty on this topic over at The Tomkins Times, but the stats from the m£XIR model further prove the point. The first few points in the moving average in the Dalglish era are weighed down by the final matches of the Hodgson era as well as the initial loss he suffered at Blackpool to open his second stint as Liverpool manager. Given a couple of matches to get his feet wet again, Dalglish ran off a blistering four wins against Wolves, Fulham, Stoke, and Chelsea to achieve the magical 3.0 four match running average by February 6th. The Wolves and Chelsea matches were away, so unexpected wins were all the more impressive. In fact, given the away match against a more expensive Chelsea side that was beginning their resurgence only offered a 21% chance for a Liverpool victory. No matter – what Hodgson couldn’t do at home to a psychologically weaker Chelsea in November would be done by Dalglish against a stronger Chelsea side at Stamford Bridge in February. Dalglish would amass a 6-2-2 record for the next ten matches, staying to the postitive on expected point differential and earning between 1.75 and 2.5 points per match. Only a stumble in the final two matches against Spurs and Aston Villa lowered his average to a more pedestrian 1.5 points per match.
The earlier graph does not emphasize enough how well Kenny Dalglish did on a match-by-match basis given the mess he inherited from Roy Hodgson. To get a true impression of how each manager performed, their cumulative point difference to the m£XIR model is plotted below. There’s no four match average here, as the cumulative nature of point differential accounts for the previous matches’ results.
Blue line (Hodgson) begins August 15th. Red line (Dalglish) begins January 12th.
The difference couldn’t be more stark. Hodgson never went to the positive for cumulative point difference to the m£XI model during his entire tenure, not even during his impressive run in October and November. In fact, one can now see clearly why FSG sacked him when they did. Their purchase of the club went through on October 15, 2010, which was only two day’s before Hodgson’s cumulative point difference reached its nadir in the team’s eighth match under him.
We all suspect FSG put Hodgson on a short leash when they took over the club, balancing fairness to him being a product of the previous ownership group with a need to protect their investment. When the club started coming off the rails again at match 17 (a December 11th loss to Newcastle), alarm bells had to be going off. Certainly, the performance by match 20 showed a trajectory back to the manager’s low experienced at match 8 (note: if Dalglish’s first match loss to Blackpool were included in Hodgson’s tenure a new low of -7.7 would have been realized).
Not wanting to entertain a relegation battle and faced with a team not performing up to financial expectations, FSG had no other choice than to sack Hodgson. Re-setting the cumulative differential when Kenny Dalglish took over, we see his consistent over performance versus the model. He added nearly 0.6 points per match in cumulative over performance versus the m£XI model, even when accounting for the drop off in performance in the last two matches. The R² value of the regression equation speaks volumes to the club’s consistency under Dalglish – the erratic ups and downs of Hodgson’s tenure doesn’t even warrant a regression plot (for those wanting to keep score, his R² value was 0.11).
Based on the cost of his £XIs and those of the opposition, Hodgson performed five points worse than expected. By contrast, Dalglish performed six points better than expected, meaning a swing of 11 points. Extrapolated over the course of a full season, Liverpool would have been roughly 10 points worse than expected under Hodgson, and 12 points better under Dalglish: a swing of roughly 22 points over 38 games.
The data is conclusive – Hodgson did not know how to manage Liverpool from match-to-match, while Dalglish did. Such an improvement in form given nearly the same set of tools at his disposal proved to FSG that Dalglish still had the managerial touch and they rewarded him with a three year contract for his efforts.
The m£XIR model is a highly useful tool in evaluating how clubs have done versus the financial expectations their starting XI set within a match. It gives us a way to understand who the biggest underdogs are, as well as the magnitude of upsets beyond what the normal scoreline presents. Most importantly, it gives us a way to evaluate managerial effectiveness given the price of players at their disposal. Add this tool’s ability to evaluate short term performance from match-to-match to the longer term regressions based upon MSq£ and M£XI, and a complete picture of the impacts of transfers and their costs within the Premier League begins to emerge. We now understand how much clubs must spend to measurably “deny fortune as much of its role as [they] possibly can.”