Photo: Daniel Gajdamowicz
What is the role of an MLS manager? How much of an impact does a coach really make on teams? Is it more important for them to make the proper signings, choose the right tactics, or individually manage and motivate players?
There are many strong opinions about John Hackworth, and an increasing percentage of those are calling for his removal. In my day job, I’m trained to make decisions not on my subjective feelings, but rather on the merit of the available evidence. Therefore, I set about to figure some objective way of evaluating Coach Hackworth.
Roster budget vs. results
One way to do this would be to look at a team’s salary and compare that to their league position. Theoretically, if a team underperforms what is expected based on the quality of its players, you could assume at least part of the blame is credited to the manager.
In leagues without tightly regulated financial controls, player spending is the greatest predictor of club success. In the Premier League, player wages account for around 85% of the variance in league position. Most teams achieve just about what is expected based on their wage bill. There are of course a few notable exceptions (Moyes at Everton for instance), but they are rare. In their wonderful book, Soccernomics, Simon Kuper and Stefan Szymanski showed that the majority of managers make no statistical difference to the position of their club, and those few that do (including Sir Alex) only make a very small impact.
Unfortunately, we can’t use this measuring stick for Coach Hackworth. The problem is that, in MLS, we do not see the same correlation between wages and achievement. Player wages only account for about 3% of the variance in league position. This is understandable. Utilizing all three DPs leaves less salary cap room for the remainder of your roster — about $35,000 less per player. That amount of money can make a significant difference when it comes to the quality of your non-DPs.
There are some ways of trying to control for the DP problem, but in the end, the best you can say is that the Union have been achieving at about the same level of their spending compared to other clubs during Hackworth’s tenure. This season’s improvement in player quality would be expected to produce a better position in the table, but other teams have improved as well, we are still in the typical “rebuilding year” phase of adjustment, and two-thirds of the season still remains. Furthermore, as evidenced by Moyes’ shockingly bad results at Manchester United, just because a team is overachieving beyond its wage bill, it does not necessarily mean that is due to the manger’s effect on the team.
Does he have the locker room?
How else could you evaluate a coach? Poll the players? Some would argue that a manager’s most important role is to manage the personalities in the locker room and try to get the best out of them. From what we can understand, the vast majority of players under Hackworth have had nothing but positive things to say about him. We have not seen major drama out of the locker room under his tenure. Even players that could have easily lashed out (Adu, McInerney) did not openly criticize him. That is rare, even in MLS. The impression is that Hackworth is straightforward with the players, doesn’t play mind games, and is supportive of them in the media. I can’t recall a single time in the last two and a half seasons that a player has openly criticized his tactics. I can not think of any players that achieved significantly better after moving away from Hackworth’s influence. There are however, several examples of players that have surprised us and achieved more than would have been expected of them under his coaching.
Public face
It may be that the manager’s biggest impact comes in his role as the public face of the team leadership. It is not an enviable job. When a team performs well, the players receive the majority of the credit, but when performing badly, the manager often feels the brunt of the critique. I think Hackworth has done well in this arena. He is one of the more open managers in MLS and doesn’t throw his players under the bus.
Tactics
One of the aspects of soccer that makes it such a beautiful sport is that it is so devoid of a manager’s influence. Coaches in other other major sports have a significantly bigger direct impact on the game — whether it is calling the plays in American football, signaling pitching strategy in baseball, or mapping out set plays in basketball. In soccer, a manager can set the shape and overall strategy for the game, but it’s up to the players on the pitch to adapt to the changing game and make the plays that count. Substitutions can sometimes make an impact, but often primarily because they bring fresh legs. In most situations, the actual player selection isn’t as important as which player is taken off.
You could certainly look to lineup selections and substitutions as a major area of influence from the manager, but how can you objectively evaluate it? The most difficult aspect is that we are not privy to the behind-the-scenes performance in training. I have heard many call for certain players to be inserted in the lineup so that we can see what they are capable of. I disagree. This is the job of the manager to determine during practices. Good managers should know what they have before game day and rarely need to experiment when it counts. It is frustrating as fans for us to wonder why we didn’t see Roger Torres get more minutes or why Austin Berry is not starting over Aaron Wheeler, but you have to trust that there is a reason for this of which we are unaware. If those decisions were as obvious as some of us feel, then we would all be coaching MLS teams.
A cruel game
Soccer is a finicky sport. As players, we learned at a young age that results aren’t often fair. There are so many variables and so much randomness that you can never evaluate a single piece of a team in isolation — coach included. You typically need a very long time for results to accumulate before you can begin to see patterns that are due to more than just chance. For a coach, with a recent roster overhaul, I think that time frame has to be more than a single season.
In the end, I would argue that we just don’t have enough objective data available for us to decide if John Hackworth should be fired. What we do know is that, statistically speaking, a manager typically has very little effect on a team’s win percentage and that the majority of teams end up worse off after changing managers. In the areas that managers do make an impact (player management and media representation), I would say Coach Hackworth has for the most part been very good.
Some would argue that results often improve after a manager change. This actually is true, but only for a very brief honeymoon period. Just like the stock market, a club’s results can go through periods of poor performance followed by good performance. As the bad results pile up, owners feel pressured to do something, so they make a change and results improve. However, this is expected with normal statistical variation, and the results probably would’ve improved anyway. Kuper and Szymanski show that over the course of a season, results tend to drift back to the average. When you figure in the added cost of changing managers and the inherit wasting of money on another set of new players, the owners would have been better off doing nothing and waiting for the tide to turn. However, when there is a vocal and impatient fan base beating the drum with an axe, doing nothing is very difficult.
In the words of the late great Giorgio Chinaglia, “If you’re gonna sack him, who you gonna replace him with?” Maybe if Sir Alex himself wanted to move to Philadelphia, OK, I could agree with sacking Hackworth. But short of that, I am not convinced it would make a significant impact in the end. More concerning is that firing Hackworth certainly has the potential for leaving us even worse off.
The midfield formation/rotation in the first half against DCU was just bizarre. There was no left mid and the players were all rotating in the middle.
I love a well thought-out, rational read on a small sample professional situation. Very refreshing.
Thanks Chris. I’m here preparing for the flaming arrows (especially since I wrote this before Saturday’s loss), so I appreciate this comment tremendously.
Arrows are lit! Lol, good job on the article even if I don’t agree. It’s a website for the people, by the people!
I like Coach Hackworth, like you said the game is gruel last year jack was on fire but our mid field was suspect. This year we have a refitted mid field and jack was ice cold.
–
Imagine this roster with jack on fire from the beginning and the fans along with S.O.B would be fund raising for a statue of Coach Hackworth the mighty…
–
As we all know reality sucks, we can only hope for something to change be it players, coach or a bit of luck fingers crossed rubbing a rabbits foot etc..
So your basically saying the only person in the world who could coach this team is a retired football god in the UK?! Not one coach in this hemisphere?
I’d almost agree with you if it wasn’t just so bad, but it is. This is a team regressing and the coaches bizarre moves and roster shuffling and excuses (maidana’s kids are wild, the fans boo and it affected the players even though the game was over) show me he’s out of control.
Not saying that no one else could coach, I’m just arguing that a manager’s influence may not be as important as we think. As for the roster moves, Hackworth is in a lose-lose situation. If he does nothing, people will criticize sticking with same formation that gives bad results. If he makes changes and “tries stuff” outside the box, he gets lambasted as well.
It’s a lose-lose situation when you only have 1 win.
It’s really more of a lose-draw-draw-draw-lose-draw-lose-lose-lose situation.
+1
The premise of this article is to use objective data to evaluate Hackworth’s performance, yet the conclusion regarding tactics is that fans need to trust the coaches decisions? Tactics is the area that I believe is Hackworth’s greatest shortcoming, so I’d be interested to see if there actually is a way to objectively assess a coach’s tactical performance. This article, unfortunately, doesn’t present a way to do so.
I’d also like to see some objective justification for the assertion that the impact that substitutes make on a game is “primarily because they bring fresh legs”. My subjective opinion is that this is nonsense, at least most of the time.
I agree with you and tried to find some way of doing that. If you consider possession in the attacking half to be a result of solid tactics, the Union rank third in the league currently. Unfortunately, the stats that ultimately matter are goals and wins. There are different ways to accomplish this tactically (grind out defensive results vs. possess and attack), so in the end the evaluation of tactics becomes a subjective matter.
What about some sort of ratio between shots (or shots on target even) and possesion time (or possession time in the attacking half)? You could also maybe consider the reverse shots surrendered compared to possession time by the opponent.
.
I don’t know if that would be an effect metric or not. Just tossing out an idea off the top of my head…
I believe the real measure for a manager in the mls is to get everyone playing together. I have said it before but hack has different styles of players on the pitch and its obvious when you watch. Its almost comical when maidana and nogueira drift together in order to exclude other players to increase the chance there will be a third completed pass in a row.
A managers major job IMO is to get everyone on the same page and then the players must make it happen. I agree that on game day the manager can’t do a lot besides choose subs to make a real large impact. All this being said I am not sure that the XI he chooses to start all have the same mindset on how to build, pass and score. That is IMO hacks biggest failing and that’s a big one.
Great article on an extremely difficult subject.
+1
One thing that I’m really struggling with is the fact that MLS teams, and the Union in particular are very new on a relative scale. European teams have decades and decades of culture on which to fall back, while MLS clubs simply do not.
.
Thus, while game in and game out, the importance/effectiveness can be debated, I think it would be very hard to look at Jason Kreis, Bruce Arena, Dominic Kinnear or most recently Peter Vermes and dismiss the enormous impact that they have had on their respective clubs. At every level, these coaches have helped shape and define their programs.
.
So while between April and June, or 2013 and 2014, the individual decisions they make my not be of huge importance, their work to building a program, a culture and an organization is invaluable.
.
Obviously, this is suggesting that their is a ton of pressure on MLS coaches to do so much more than coach, but that is the reality of this league, and the Union at this particularly point in the team’s history.
My concern about Coach Hackworth is that in his three main responsibilities — player acquisition, tactics, and lineup selection — he seems at odds with himself. He wants to develop young players, but he is really hesitant about putting them on the field. He wants to play a possession, tiki-taka style, but his starting XI choices seem to prefer ‘hustle’ more than touch. He wants to spread the field and play in from the wings, but his major acquisitions are all central midfielders.
.
I get the feeling Nick Sak is interfering in just enough of the decisions to throw off Hackworth’s plans, but is letting Hackworth ‘wear’ all the decisions. Regardless, we need management of this team that puts consistency into their personnel choice, tactics and roster decisions. Until then, the club will keep foundering.
I agree and disagree with you. I agree that Hackworth is at doss with himself, but I believe this to be true because he just isn’t a good manager. He is reaching too much and can’t decide on just how to make his vision work. That comes from having no tactical ability.
.
I disagree that Sack is involved in game day tactical ideas. I don’t have much to support my idea, I just don’t think there is any to support him pulling strings (regarding game day tactics) behind the scenes.
.
In fairness, I guess we could both be right. (or wrong, ha!)
Worse off than 1 win in 11? I guess MLS could take that win away.In his first full season, he “guided” the team into 14th place in the SS standings. This year, under his leadership,they sit in 18th after a third of the season. I think we define “very good”, very differently.
I took a crack last week at objectively looking at what you say is tough to do – the subs a manager uses and what impact that has.
https://drive.google.com/file/d/0B8qcroTqBcDIN29tclU0emZtSVk/edit?usp=sharing
Basically, and if others want to build off it rock on, what was the score at the time of each sub and what was +/- impact by full time. So if a team loses early (like the DC match) and the subs dont score the manager gets a 0. If they score after subbing, +1. Simple. Hack is below average by that standard in MLS.I haven’t updated it since Friday for the weekend games, but the plan is to keep tracking just how awful his management actually is. It’s right there on the scoreboard, objectively, every week.
Awesome work. I hadn’t seen that before and love trying to apply data to these questions. I think it’s potentially a good measure, although obviously a lot of variables and vulnerable to small sample size. By the current table, you would rank Caleb Porter as the worst in the league. I think last year many would have said he was the second coming of Pep Guardiola which emphasizes my point that there are too many variables to be able to objectively evaluate, IMHO.
It tracks pretty closely to the current standings, which one could interpret as exactly how much influence a manager has on the match. So that doesn’t penalize a team for not being as talented to start, but rather what is the net result after subs. But absolutely sample size is small, though not as much for the Union as they have played more matches. If anything, the numbers could shift more dramatically against as the other clubs catch up on the schedule.
I’m also not sure how you’re distancing the manager who is buying the players to fit his system/philosophy/formation with the manager who is getting results. If Hack says I need Edu, they get him, and they don’t win because he insists on pairing him with Carroll, is that on Edu or Hack? If Hack the player evaluator says we’ve improved after dealing the leading scorer and yet we score less, wouldn’t that be an effective measurement? We aren’t alleging that Hack is still hamstrung by Nowak right? It’s time to put that bogeyman back in the closet and stop scaring the kids with those campfire tales of 2012. This is his team, and on their current trajectory they would need to average,what, 1.6ppg, 1.8PPG to reach the playoffs? Tall order for a team that only gets 3.4 shots on goal.
I like your analysis, thanks for posting. I think that this certainly points to Hackworth not being very effective at impacting a game through his use of substitutions. I think you should give a 0 to the RSL game since they both let in a goal and scored a goal post-substitutions. That leaves one game (Columbus) where the Union scored post-substitution. The lack of scoring, be it from starters or substitutes, is the reason we only have 1 win.
This is a refreshing antidote to the usual “Sack Hack!” ranting that fills the comment section of this excellent blog. And I say that as somehow who is agnostic on John Hackworth. I think he seems to have an excellent eye for American talent development, and I am not at all convinced that he is a good manager.
—-
As you say, I think the influence of the manager is likely very overrated. On the one hand I think it’s hard to argue Eli’s point that certain individuals seem to consistently get the most out of their squads. On the other hand, I think it’s hard to argue your point that most managerial changes leave their teams flat or net negative. So whether or not Hackworth is a good manager, firing him would be unlikely to help us this season.
“How could it be any worse?!?!….Jehovah Jehovah Jehovah”. Any Monty Python fans?
I’m warning you! If you say “Jehovah” one more time…doink
As you point out, Hackworth’s strength (based on previous experience) is to identify and develop youth talent. I think he could play a role in the organization as a youth scout, but I think the team needs a more capable team manager. Someone who
1) can get the most out of his players, especially supporting role players (ex. Bruce Arena having success with incorporating Michael Stephens, Chris Birchall, etc.)
2) puts his players in the best position to succeed (ex. either getting the right players who can play in a 4-3-3 or switching back to a 4-4-2).
Damn it, Scott! Stop being all rational and scientitic at a time like this! More fire and brimstone, less logic and analysis!!!
.
I very much like that Hackworth publicly protects his players. It’s the same thing, really, as Andy Reid’s “I gotta do a better job,” but with more charisma. Your point about Freddy Adu, actually, is interesting. Because if anybody was going to lambast Hackworth, he was the guy. He could call up anybody in the media – local or national – and be given the platform to speak, even if he were to be quoted anonymously. And we just didn’t see it. That does indeed say something.
.
I think we’ll know within a month or so what the outcome will be. I see two “windows” where the FO could reasonably make a change. One is right after Saturday’s game. That means the team sort of pisses away the LA road trip, but at least there’s a week between games there so a new guy can get his feet on the ground. The other, of course, is the WC break. If Hackworth survives the break, he’s here for the rest of the year.
.
The question, I think, is who is deciding that the team’s offense should be, “Get to the corner, fire in a cross, run back on defense after the cross is intercepted.” If that’s Hackworth’s plan, he needs to go because he’s not playing to the strengths of his roster. If that’s the players, Hack stays and the players need to change – either by the ones on the field doing something different, or by Hackworth replacing them with players who will do something different. But that, I think, is the question that needs to be answered.
Thanks John. As to your question about the crosses, I think both the players and Hack have said that it isn’t the gameplan at all, but a result of feeling like there aren’t better options on the field at the time. That could be a result of guys up front not running off the ball well and creating enough space, or it could be a tactical fault of having the wrong type of guys in the wrong places on the field. I, personally, dunno.
Correct. I asked him about that with the last question of the press conference, and I think the reason it’s not in the Delco Times transcript is that, by that time, 2-3 reporters had dropped off the call. De George was probably among them.
Hackworth just said in his press conference that he wants the time to stop forcing crosses so much…
It seems like the Union start each game playing positive, possession-based soccer, and then at some point they kind of give up and go back to ‘cross-and-pray’. And that point where they give up their plan seems to be coming earlier and earlier every game.
I don’t see Hack as being supportive of his players in the media, at least not all the time. I seem to remember a recent piece criticizing Maidana’s bedtime and parenting skills.
.
On the flip side, I see him being overly eager to heap praise upon players who don’t really deserve it: Wheeler this season, Keon last, for a couple examples.
.
I think he’s always had the locker room because he’s a nice guy. That, in my opinion, is what made him a decent assistant coach.
He likes to heap praise on how well the players are handling being forced to play out of position
I just see a team with a lack of tactical imagination on offense.
It could be chemistry as none of the players have played together for an extended amount of time. However, if chemistry is already an issue, its generally not a good idea to start experimenting with converting a Fwd to CB.
Okugo to CB and Captain Carroll – This didnt make sense the day it happened and it doesnt make sense now. Carroll’s inclusion in the XI is redundant and uninspired. It’s forced our youngest/most promising player into a new role and created a size issue where you never want one.
I love the U. But something isn’t clicking. I swear to god if they trade Amobi for some promising striker that Hackworth drags into the 4-3-3 graveyard, all hope is lost.
don’t lose hope, cause if Coach is serious about wanting to trade for a good striker, he has to choose among amobi or a goalkeeper and that’s about it
If Amobi Okugo goes I go.
Getting away from Hack for a minute, does anyone think this team just doesn’t shoot? Why is that? That can’t all be Hack’s fault, right? Just a random thought I had.
The Union is actually third in shots, according to MLSsoccer.com. But they’re 14th in shots on goal. So they’re shooting, just not straight.
Yeah I guess that’s what I meant. Shots actually on the goal. I can’t tell you how many times I scream for them to make the keeper work… or that I would be happy to see it go just over (even though it’s technically not “on”) instead of 12 yards wide.
So I just want to see if I’m summing this up correctly, even if I am being a little generic… “A manager in football has little to no effect on the outcome of a game or season. John Hackworth is a nice, open manager, who the players like. With those things in mind, why bother trying to change anything?”. If that’s the case, then I just couldn’t disagree more.
.
I really appreciate the attempt at rational thought and diagnosis in this article, even if i disagree. And I have actually always appreciated the fact that Hackworth was pretty open and didn’t hide from problems. Some of the things were completely illogical and infuriating, but he said what he thought.
.
However, you simply can’t keep a manager around just because he’s a nice guy, that’s insane. We currently sit second to last in the league, with games in hand. We’re behind Chivas for God’s sake. Simply put, it couldn’t really get worse. I would kill for a honeymoon period right now. Sometimes those are just what is needed for a team that is talented but slogging.
.
The other thing that baffled me last year, and baffles me again now that you are defending it, is the continued use of certain players, even when faced with their futility. We are doing it again this year. As with the manager, if certain guys just simply are not producing, TRY SOMEONE ELSE. I mean we tried Wheeler didn’t we? What would happen if younger guys were played up front, we wouldn’t score in 16 hours of game time??
.
All in all a great article and caused lots of thoughts, I just don’t agree with the thinking. I would love for the team to turn it around and start winning and for Hackworth to stay. This would mean my weekends, and in turn my first year as a STH, wouldn’t be wasted. The problem is I just don’t see it happening. For that reason, give me someone new.
Yes, I think you did get my main point which is that a manager may not have as much impact as we think. However, I’m not saying we should keep him around just because he’s a nice guy. I’m saying that the evidence doesn’t support the traditional owner’s decision to fire a manager just because of poor results.
.
Your comment about trying someone else is a little confusing. You want him to try someone else, and then you acknowledged that he has (Wheeler). He has also tried Leo this year despite everyone thinking that Leo was terrible last year.
.
The cost of changing managers could be this: we bring in a new manager. Maybe we get a couple of wins, but ultimately still miss the playoffs. New manager wants new guys, so the sacrifices that were made to bring in Edu, Maidana and Nogueira and now thrown away and new, more costly sacrifices have to be made to bring in the new manager’s preferred players. Support staff are fired to have the new manager hire his/her new people. All of this not only costs money, but TIME as well. Look at the ridiculous cycle of managers going on in the Premiership. For every one Gus Poyet, there are another 10 new managers who haven’t improved a clubs’ position.
.
I’m not arguing that Hack is the best coach ever. I’m just saying that the available data, while limited, would predict that firing him could be worse than keeping him.
Here we go again: managers DO have nearly as much tactical effect as football. They don’t call out a named play into a microphone, OK. But in Football, a coach can’t force the QB to throw to the open target, or force the WR to run sharp routes. But just like in Soccer and every other sport, games and situations are studied and rehearsed and filmed and reviewed ALL WEEK,EVERY WEEK. When a guy crosses a ball in practice and he’s supposed to be playing inside to feet, he is corrected and trained to follow the script. This is how coaching is done.
–
So either Hack doesn’t do this (needs to be fired then) or this is his style of play (needs to be fired).
–
If coaches weren’t needed in a pro environment, trust me, they wouldn’t be there. At least give the man the credit that he has influence over the game!
I believe that Hack has gotten some very viable criticism from some fans. He has also been on the short end of fans who just do not like him at all. I am reminded of DC United last season, how bad was their record, and their coach was not fired. Why fire a coach who has only had two years to try to fix the mess that the previous manager left (hint, it was going to take more than two years to fix the problem)?
.
What is hurting the Union this season is that they spent money like never before (to the casual fan) by bringing in the two DP signings, and the third ‘big signing’ as well. Casual fans were all of a sudden interested in the team. Then they come out and look good in the first two games and the casual fan gets even more excited. Then the team gets a stomach virus and has diarrhea all over the pitch for nine straight games. The casual fan got tired of looking at the excrement smeared all over the field by the Union, and turned the team off. They are gone now.
.
What is left is the cacophony of angry, sometimes drunk, fans who think that ‘sacking hack’ will change the fortune of the team. I disagree with the sentiment at this time, I believe Hack should have another year to see how he can set up his team within the constraints of his salary cap space.
.
It is going to take an incredibly run of play for the team to get back the good will it lost by being so horridly awful for two calendar months, while casual fans fell off the good will wagon at a weekly pace. Unless something within the team dynamic changes, they can all (players, coaches, front office) expect the drunk angry fans to drown out any positive fans who still see that it takes more than one, or two, years to recover a roster from what that last fool did to it.
Caught between “change for its own sake” and “too predictable & same poor results for too long.” Which is worse?