Photo: Barb Colligon
Earlier this week, PSP reader Paul Baker sent me an email noting Jack McInerney’s high ranking on the latest Castrol Weekly Top 20, in which the young Philadelphia Union striker came in at No. 9 (Josue Martinez came in at No. 3 behind Chris Wondolowski and Robbie Keane). Paul described how he sees little commentary on the Castrol Index, despite the fact that it claims to be be based on objective analytics. Paul wondered how the rankings might compare with the player ratings that PSP does after each game. I link to the Castrol Weekly Top 20 in our news roundups so I decided to find out.
How the Castrol Index works
As “the Official Performance Index of Major League Soccer,” the Castrol Index “uses the latest technology to objectively analyze player performance” by tracking “every move on the field and assesses whether it has a positive or negative impact on a team’s ability to score or concede a goal.” The Castrol Index FAQ page on the league website notes, “A key factor for all areas of performance in the Castrol Index is in which zone on the pitch the action takes place.” So, events such as passes, tackles, interceptions, blocked shots, and so on, are first awarded or deducted points by “Castrol’s team of performance analysts” depending on whether or not they are successful. But a successful pass in the box leading to a shot on goal is given more weight than, say, a possession pass between two defenders their own defensive third.
Leaving aside reasonable questions about the subjective interpretations an analyst might have to make in deciding, in the example of a misplaced or intercepted pass, “how much trouble the mistake is likely to land the team in,” not to mention whether or not (or how) elements of the game such as off-the-ball movement are measured, the Castrol Index is an effort to engage in a meaningful, yet objective, interpretation player performance from basic match statistics.
Castrol Index vs PSP player ratings
While informed by match statistics, the kind of player ratings that PSP has produced for three seasons are at their heart inherently subjective in that that are based on a writer’s viewpoint. They also take into account the kind of intangibles that statistics have difficulty measuring. And, because they are opinion based, they’re usually more fun to read, producing the kind of conversations websites such as PSP value so much in the comments section.
Acknowledging the fact that the Castrol Index and PSP’s player ratings are different animals, how do they compare?
Broadly speaking, I think they compare pretty well. The principal difference to me is the placement of Jack McInerney and Zac MacMath in the first and second spots on the Castrol Index. In PSP player ratings, MacMath comes in at No. 7, two spots above McInerney. But, with a few exceptions, the rankings of the PSP player ratings averages look like a typical Union Starting XI under John Hackworth. Certainly, the high rankings of the Union backline reflect the fact that the Union defense has been the most consistent part of the team (especially since Porfirio Lopez disappeared from the Starting XI). The exceptions—Raymon Gaddis, Antoine Hoppenot, and Chandler Hoffman—have each gotten starts at various points in the season. The Castrol Index rates Gabriel Gomez higher than many Union fans would probably rate him, particularly on recent form. The Index also includes Freddy Adu, Michael Lahoud, and Danny Cruz in the top 11 whereas the rankings of the average of PSP’s player ratings does not.
It is also interesting to note that the difference between Hoffman at No. 11 and Josue Martinez at No. 18 is only .53 points.
So, how to explain some of the differences?
The Castrol Index FAQ page explains that “every move on the field” is assessed based on “whether it has a positive or negative impact on a team’s ability to score or concede a goal.” No particulars are described but it is easy to assume that scoring a goal boosts a player’s numbers. With Adu and Gomez leading in goals scored on a team that has had trouble scoring goals much of the season, they presumably gain points in the Index. But both players, to what I think would be wide agreement among Union fans, have disappointed more than they have scored.
This falling short of expectations is precisely the kind of thing subjective player ratings take into account. Similarly, Gaddis and Hoppenot are likely to rank better in the averages of PSP player ratings because, as rookies, they have exceeded expectations. (And I assure you, Hoppenot’s ratings have dropped sharply as the shockwaves from his explosion on the scene as the Union’s spark plug have diminished through the recent long scoring drought.) That Williams is slightly below Gaddis is probably because his performance suffered early in the season under the tactical experiments of Nowak and also because he played much of the season with an injured foot.
Additionally, the FAQ page says players “who have not played a specified amount of minutes each month (equal to approx 60% of season game time) will have their Castrol Index score divided by that amount of minutes, therefore penalizing players who are either injured, not selected or suspended for a considerable period of time.” PSP’s player ratings make no such claim, which helps in understanding why Gaddis and Hoffman might land higher in PSP’s rankings than in the Castrol Index. In other words, we just thought they generally played better than some of their teammates when they were on the pitch.
In the end, whether one ranking is objective and the other is not does not in itself make one better than the other and both rankings provide good fodder for conversation. Looking at the Castrol Index, does Martinez “deserve” a higher ranking than Hoppenot? Looking at PSP’s rankings, does Hoppenot “deserve” a higher ranking than McInerney (however slim the difference might be)? Comparing the two rankings, do you think Adu has had a better season than, say, Gaddis? And so on.
We have our own favorite players just as, no doubt, you do.
Feel free to discuss below. I’ll do another comparison after the end of the season.
My first observation is that goals weight the Castrol index far too heavily. It’s a low-scoring game. So much happens without a goal scored. We’ve watched the Union play too many games with too few goals this season, but does that mean we can’t see bright performances in the midst of a poor result? I think not.
I agree with you, but the point of the game is to score goals, so given that the game is low scoring that further rewards a team and/or player for scoring a goal. It doesn’t mean you get a bad rating if you DON’T score, it just means your rating increases more if you DO score. I almost feel like their scale is based on a 0-0 game, with a goal scored giving bonus points and a goal conceded losing you points. That’s my take on it, at least.
I know PSP is a fan site and thats why I come here but when it comes to player rankings/ratings I really prefer an objective approach.
I find PSP’s ratings to be artificially inflated or deflated depending upon whether the team won or lost. When a player has a solid but unexceptional game, and the team wins 3-1, that player is liable to get a 6.0. Same level of performance, but the team loses 2-0, and that player will get a 4.0. That’s the main flaw I find with the subjectivity of PSP’s ratings.
Wait … wtf? MacMath has a PSP average of 7+!?!? Is that right?
Ed- Thanks again for the foray into analytics. I, too, am not a math whiz, but this stuff is enjoyable. It is interesting to see the compilations of the two different reports. I see them both showing, for example, that Zach MacMath is ranking high in the scales, despite the criticism that many of us have been leveling at him for the occasional misplays and positioning errors. It is also interesting to see where Freddy Adu winds up, below so many of his teammates and so many other players in the league. (I will need to see if I can locate a comparable rating system that addresses his game with the U-23 national team. The suggestion has been that he has difficulty here beceause of the lack of highly skilled payers around him, and his feeling that he needs to carry more himself. It would be neat to see if his rankings are then higher when his teammates are better at receiving his passes or supporting his runs.) Thanks again.
It isn’t a statistical comparison, but after this morning’s news roundup was published, SBI posted a link to a YouTube video someone had created that shows all the chances Adu had created for the Union in 2012 and suggested the same thing, that if the Union’s finishing had been better, Adu would have looked better. There are some really lovely balls (and the video is also a reminder that Pajoy missed a depressing number of sitters.) I counted 19 chances created from 12 games in the video. While better finishing would surely have helped the Union’s fortunes, if the video is comprehensive, the troubling thing is that Adu played in 11 more games. You can see the video at http://www.youtube.com/watch?v=scsDNMzJewc
Depressing is definitely the right word. Not just Pajoy. And not just sitters. DEAD sitters. Do not watch this video if you have been drinking, or if you firearms within easy reach. It’s like a lowlight reel.
I really do think if Jack was getting those minutes, a LOT more could have been goals.
And for Pajoy, he really was a Nowak guy. And from all the shady things we are hearing about Nowak, his transfer dealings and the shady relationship with Guiterriz + his columbians, I really wonder how much Nowak actually saw of Pajoy and how much he thought Pajoy could help us.
OUCH! I watched about half of it and saw a couple of things. First, those balls that he was playing into the box are very nice. Second, the old issues about lack of height from the strikers is painfully obvious- Pajoy, Jack Mac and the Farfan boys each bottom struck balls that a taller player might have been able to drive home. (The question then is whether it is a good ball played to a player that cannot reach it, or a bad ball because it was played to a man that can’t reach it.) Also, I noticed that there seems to be a real lack of contact between him and even the players that he does serve successfully. Rarely were there the acknowledging thumbs-up signals to Freddy, thanking him for the effort when things didn’t work out; and he seems to stay away and celebrate by himself when there are goals scored from his service. I will need to watch the whole thing when I can set the time aside to do the whole thing. I wonder if he has reached the same conclusion that we are examining- is he simply trying to assist those that cannot handle the assists? Finally, I absolutely agree that the number of these opportunities is not what one would expect from the “lead player”, if he is not to be treated as a DP. He is still young, and can learn even from this clip collection, as long as he doesn’t take the lesson that he is a diamond buried in a pile of coal. Thanks for the clip.