One of the many purposes of the Transfer Price Index has been to make comparisons of relative performance across multiple seasons of the Premier League using a constant-pound basis for transfer fees. Such an approach is very useful in helping to settle discussions of which clubs and managers have made the best use of their money and which have not.
There have been several TPI-based models introduced for this purpose. The MSq£ and M£XI models have looked at the multiple of league average squad and starting XI costs as the inputs, and table position as an output. These are effective at looking at longer-term trends over several seasons, but cannot be used to assess performance within individual matches or seasons.
To address this latter need, the m£XIR model has been introduced over the last few months. It takes into account the ratio of the cost of each team’s starting XI in a match (hence the acronym that means “[m]atch [£XI] [R]atio”), whether or not each team in a match is home or away, and the outcome of the match for each team (loss, tie, or win). The data for each match in a season is used as input to an ordinal logistic regression model. The output of this model is the likelihood of a team losing, tieing, or winning a match based upon the venue and the team’s m£XIR. These probabilities can then easily be converted into an expected number of points from each match (probability of tie x 1.0 + probability of win x 3.0). Knowing both the expected points and actual points from a match, the residual (actual points – expected points) is calculated for an individual match. At any point in a season, the point per match (PPM) residual can be calculated by totaling the residual and dividing it by the number of matches. It’s this PPM to the m£XIR model that allows for a comparison between clubs, managers, and seasons.
In previous posts, the performance of individual managers on the PPM residual scale has been used as a comparator. In this analysis, the reverse approach is taken. The PPM residual is calculated for each club in each season. Then, on a seasonal basis the teams are ranked in descending PPM residual in a table that shows the best over performers and worst under performers. In a slightly different analysis, the clubs performances by season are arranged from highest to lowest overall differentials to provide a comparison of clubs, regardless of which season their performance occurred. The results allow us to identify the best and worst performing teams of the Premier League era based upon their relative expenditures in the transfer market.
The table below shows the finish position of each club in each Premier League season based upon m£XIR PPM residual. Clubs with higher PPM residuals finish higher in the table, and those with lower residuals finish lower in the table. Cells marked in red indicate a club that finished in the bottom three positions, and would have faced relegation from the Premier League on an m£XIR PPM residual basis. Cells marked in green indicate clubs who finished in the top position each season versus the m£XIR PPM residual. These clubs represented the best bang-for-the-pound on a points basis within an individual season. Clubs marked in yellow represent ones that finished in spots 2 through 5 in the table – good enough recently to qualify the club for participation in UEFA tournaments. A club’s average table position and average PPM residual over the 19 seasons of the Premier League are shown on the right hand side of the graph. For those interested in a closer view of the graph, the original version of it can be found here.
The m£XIR Champions by Season
One of the more natural comparisons to make is in the actual champions of each Premier League season versus those who would have finished at the top of a table based upon the m£XIR PPM residual model. The table below provides just such a comparison, highlighting any difference between the actual and m£XIR champions in yellow.
Overall, there are eight instances where a team other than the league champion has finished with the best m£XIR PPM residual (a rate of 42%). This is primarily due to the fact that model is recalculated for each season, meaning that the model is somewhat tailored to the results seen in each season rather than a longer term average from the 19 seasons of data. It’s more difficult for a team to over perform versus a model specific to a season than one that is more generalized over multiple seasons. This result is also a bit biased by Manchester United’s championships in the pre-Abramovich era, which Pay as You Play expertly explains were a rare occurrence of a wealth of trainee talent that delivered above average results for Alex Ferguson.
By the time Manchester United and the rest of the league had to start competing in the escalated transfer fee environment of the post-Abramovich era, Ferguson’s transfer costs had escalated to the point of wiping out much of his m£XIR advantage. Not surprisingly, the league champion has only been the m£XIR champion four times (50%), while a similar feat occurred in 7 of the 11 pre-Abramovich seasons (64%).
An overall accounting of each champion’s tally is now provided, with observations about key championships providing further detail.
Manchester United (10): Manchester United earns two fewer m£XIR championships than their actual haul during the Premier League era. This comes down to the fact that two of their post-Abramovich championships – 2007/08 and 2008/09 – would be ceded to their arch-rivals Arsenal and Liverpool, respectively. A swap in championships with Arsenal in the 1997/98 and 1998/99 seasons leads to no net change in their total m£XIR championships. Manchester United’s most dominating m£XIR championship came in their title-winning 1999/2000 season where they earned 0.82 more points per match than the m£XIR model predicted. This is good enough for second best all-time behind Arsenal’s undefeated season of 2003/04.
Overall, United has the second lowest average PPM of any team that has won an m£XIR championship (0.56), which has declined when comparing pre-Abramovich versus post-Abramovich eras (0.57 versus 0.54). This is symptomatic of Ferguson’s overall difference in performance in the two eras. In the pre-Abramovich era he benefited from great academy players and averaged a PPM differential of 0.52, while in the post-Abramovich era he’s had to spend a good bit more in the transfer market and has averaged a lower 0.42 PPM differential. Surprisingly enough, he’s shown the same consistency in both eras (Pre-Abramovich standard deviation = 0.12, post-Abramovich = 0.11).
Arsenal (4): Not surprisingly, all of Arsenal’s m£XIR titles come under Arsene Wenger. Perhaps a bigger surprise was just how much Arsenal was underachieving before Wenger arrived. In the first four seasons of the Premier League the Gunners finished 14th, 5th, 19th, and 10th on a m£XIR PPM residual basis. Since Wenger’s arrival during the 1996/97 season, Arsenal has only finished outside the top 3 vs. the m£XIR metric twice – 2001/02 and 2005/06.
Only one of Arsenal’s m£XIR championships came in the same year as an actual Premier League championship. It was in their Invicibles season of 2003/04 that they achieved this feat – setting the record for m£XIR overachievement at 0.96 PPM. Two of their other m£XIR championships came at the expense of Manchester United (1998/99 and 2007/08) and one at the expense of Chelsea (2009/10). Three of the four come in the post-Abramovich era, with two of them coming in the last four seasons. Perhaps this is minor consolation for Gunner supporters who haven’t seen a major trophy since 2004/05.
Where Wenger has excelled versus the m£XIR metric is in the post-Abramovich era, especially when compared to his managerial nemesis, Alex Ferguson. He’s bettered him by 0.07 PPM on average (0.49 vs. 0.42), and nearly half a finish position on average (2.5 vs. 2.9). He’s also greatly improved his PPM performance versus his record during the pre-Abramovich era by a tenth of a point (0.49 versus 0.39) or nearly a win and a draw over a 38 match season. Wenger is a bit less consistent in the Post-Abramovich era (Post-Ambramovich standard deviation = 0.26 versus pre-Abramovich of 0.15).
Liverpool (3): Perhaps less of a consolation than Wenger’s m£XIR championships during the “Trophyless Era” is Liverpool’s three championships in the Premier League Era. Sitting on 18 first division trophies since 1990, the Reds have been bedeviled by Manchester United’s haul in the Premier League era that has seen them go to 19 total championships. Nonetheless, the Reds have been the best team for the money in three of the Premier League seasons.
The first championship came in 2001/2002, when Gerard Houllier guided them to second place finish in the league (behind Arsenal) with 80 points. This was done with an average squad cost of £186.9M and an average £XI of £99.8M, 23% and 21% less than Arsenal’s Sq£ and £XI values, respectively. Houllier would not come close to such a feat the next two seasons, regressing to PPM values of 0.15 and 0.18.
Rafael Benitez would experience even worse results his first season as manager, earning a meager 0.06 PPM residual while rebuilding Liverpool. This was offset by the improbable Champions League trophy won in classic form at the end of the season. Benitez would build upon the Champions League success the following season, winning his first of two m£XIR championships in the 2005/06 season. Liverpool finished third in the actual table and only one point behind second-place Machester United, who had 1.52 times the squad cost and 1.73 times the average starting XI cost of the Reds.
Benitez would finish at the top of the m£XIR table again in 2008/09 with the highest point total (86) to not win a Premier League title. This point haul also gives them the distinction of having the highest PPM differential to the m£XIR (0.66) of any club not winning a Premier League title. Premier League winners Manchester United finished with a PPM 0.25 lower than Liverpool – the equivalent of 9.5 points less over a full season.
Chelsea (1): Chelsea’s one m£XIR title demonstrates that sometimes a team bought through expensive transfers can truly live up to financial expectations and outpace the rest of the league when their financial resources are accounted for. After spending big prior to 2003-04 and coming up second against Arsenal’s Invincibles, Chelsea spent big again prior to the 2004-05 season and went as far as sacking Claudio Ranieri when expectations weren’t met. Enter Jose Mourinho.
Chelsea steamrolled the league that year, registering 95 points (most in the history of the Premier League) on the back of a squad that cost £498.1M and a starting XI that cost an average of £248.6M. This allowed Chelsea to register a PPM differential of 0.72, good enough for 6th all time. Just how good was Chelsea? No team that season other than Chelsea and Arsenal earned more than 0.36 PPM greater than the m£XIR predicted. Arsenal’s 0.61 PPM, good enough for second that season, would have been enough to earn them the top spot in 14 of the 19 Premier League seasons played to date. Chelsea was that good in 2004/05, even when the cost to assemble their squad is considered.
Nottingham Forrest (1): While Chelsea’s 2004-2005 m£XIR title shows that sometimes a club can be as good as the money spent on it, Blackburn Rovers’ loss of their one Premier League championship in 1994/95 to Nottingham Forrest on an m£XIR basis confirms that most of the time the expectations set by transfer spending cannot be met. That season Blackburn outstripped everyone’s squad cost in the league except Liverpool (£180.4M to £183.8M), while their average £XI was tops in the league at £111.5M. It was good enough to win the Premier League, but only good enough to finish third in the m£XIR residual table. Tiny Nottingham Forrest – 12th in squad cost (£84.8M) and average starting XI (£49.3M) – finished third their first season back in the top flight after relegation at the end of the inaugural Premier League season. They’d only finish eight points off the pace, and behind two clubs (Blackburn and Manchester United) that had squads that cost more than twice as much as theirs.
Top 30 m£XIR Performances
Now that the champions for each season have been identified, what about the all time ranking of all 386 club seasons in Premier League history? Which teams would normally have won the m£XIR championship if not for another team having an even better performance?
The table below presents this data for just such an all-time ranking, with the Top 30 teams presented. The teams are ordered from overall highest-to-lowest m£XIR PPM residual regardless of season. The rank within seasons and the actual year-end table positions are also presented for reference.
The Top 30 clubs encompass the following positions within the m£XIR league for each season:
- All of the m£XIR champions (the final one being the 2002/03 Manchester United team in overall position 29)
- Eight of the 19 seasons’ runners up
- Two third place teams (both Manchester United, in 2003/04 and 1998/99)
- One fourth place team (the 1998/99 Leeds squad)
There are only two teams who had a better m£XIR PPM residual than half of the seasonal champions and yet did not finish tops within their seasons – 2004/2005 Arsenal and 2003/2004 Chelsea. Astute observers will note this is a flip-flop of the actual finish positions in two campaigns where the actual champion far outpaced the rest of the league (including the runner up). An indication of the competitiveness of the 2003/2004 campaign can be found in Manchester United’s third place finish in the m£XIR league that season – the 12th overall best finish against the metric, the highest third place seasonal finish in the overall table, and a very respectable 0.561 m£XIR residual PPM that would have earned them an m£XIR championship in half of the Premier League seasons to date.
Liverpool own the distinction of placing the highest in the table without having won an actual championship (4th in their m£XIR championship season of 2008/09). Arsenal is right behind them with the second highest finish without a championship with their 2007/08 campaign that is arguably their best performance since their last actual championship in 2003/04. In both seasons Manchester United had a squad and average starting XI valuation at least two-thirds more expensive than Liverpool and Arsenal. United’s position in the m£XIR league was 5th (58th overall) and 3rd (36th overall) those two seasons.
Ipswich’s 2000/01 squad that finished fifth in the actual table is the biggest over performer in the m£XIR PPM residual Top 30, with a runner-up finish that season in the m£XIR league that is good enough for 18th overall in the Premier League. Ipswich’s team was built on £42.2M worth of transfers with an average £XI of £27.9M that earned them 66 points, all of which placed them 17th in the league in transfer value but fifth based upon points. The champions that season – Manchester United – had to spend £233M on their squad (tops in league) and achieved an average £XI of £102.6M (fourth behind Arsenal, Newcastle, and Chelsea).
It’s also interesting to note that before their financial implosion in the early 2000′s, Leeds had back-to-back finishes in the top four of the m£XIR league. This is good enough for 25th and 30th overall in the history of the Premier League. This suggests they were getting reasonably good value for their money, but clearly their revenue couldn’t support such expenditures.
Overall, the m£XIR metric provides for a reasonably good estimate of the points a club should expect to earn in a match based upon their and the opponent’s starting XI transfer fees. Looking at a club’s residual points, or the difference between the actual points earned and the predicted points earned, can be very effective in differentiating between clubs with similar overall performances. Overall, Manchester United’s championship haul during the Premier League Era remains largely intact. The biggest overachiever in terms of m£XIR championships would be Liverpool with three such championships compared to their lack of actual championships since the start of the Premier League.
Applying a similar methodology to managers’ tenure at a club over multiple seasons would provide a metric for judging which managers got the best bang for the pound both within a club and across multiple clubs. Such an analysis will be the subject of a future post.