First Pitch: Why Are Projection Systems Always Down on the Pirates?

Yesterday I asked Andrew McCutchen for his input on the hype that the Cubs were receiving this off-season, along with whether he agreed with the Pirates being forgotten behind the Cubs and Cardinals. His response, from the article, was interesting.

“How many times have we been on the back burner these past few years? I think almost every year, if I’m not mistaken,” McCutchen said. “No one ever expects us to do that well. We won 98 games last year. It wasn’t enough for us, but I don’t think many people saw us doing what we did.”

Interestingly enough, the PECOTA projections came out today at Baseball Prospectus, and they weren’t favorable for the Pirates. The original projections had the Pirates winning 79 games. That was updated to 83 wins by the end of the night. This new total was enough to place them one win ahead of the Cardinals, but would have them missing the playoffs by several games.

It’s pretty common for the Pirates to be underrated by projection systems. PECOTA has always been lower on them than other systems, even throughout the year last year when they were on their way to 98 wins. The projections actually predicted a .500 record before the season started. So you probably shouldn’t put too much stock in the projections as a concrete number, and instead use them as a guide. And if you’re using the projections as a guide, then you have to ask a very important question: Why do these projections always underrate the Pirates?

Looking at this year’s PECOTA breakdown, a few things stand out.

First, there are some players who look like they are just flat-out projected way too low. Take the outfield, for example. Andrew McCutchen is projected for a 4.0 WARP, even though he was 6.0 or better from 2011-14, and 4.8 in a down year in 2015. McCutchen also had a 4.3 in 2010, and a 2.6 in half a season in 2009. A 4.0 projection would be a career worst, and an extremely disappointing year.

The same goes for Starling Marte. He’s projected for a 2.0 WARP, even though he’s been in the 2.5-3.3 range in his first three seasons. Of course, the other side of this is that Jordy Mercer is projected for a 2.0, even though he had a 1.6 in 2013 and an 0.9 in 2015. He did have a 3.7 in 2014, which brings his three-year average to 2.0, but that 2014 season looks like an outlier compared to the other two years. At any rate, the Pirates seem to have more players projected too low than too high, and the guys projected lower have a bigger difference than the guys projected higher.

But the truth about this is that I’m only looking at the Pirates. It’s very possible that this same problem could be taking place all around the league, with players on each team getting projected lower than their career numbers. For example, the Cubs have similar problems. Anthony Rizzo averaged 5.4 WARP the last two years, and was projected for 3.8 this year. Kris Bryant and Jason Heyward were also projected low, with Heyward getting about half his normal value. So if the Pirates are underrated for this reason, then the Cubs and every other team could have the same argument.

The next thing I’d look at would be the pitching staff, specifically the rotation. The projected back of the rotation looks pretty horrible, with Jon Niese, Jeff Locke, and Ryan Vogelsong combining for 1.9 WARP. Looking at their previous season numbers, these aren’t totally out of the question, and aren’t low predictions. But the Pirates seem to get the best out of pitchers, and that is something which won’t be included in projections. I’m less confident that they’ll get a big breakout from someone like Ryan Vogelsong, but I could see Jon Niese returning to his best self, which alone would be worth more than that 1.9 WARP combined projection for all three guys.

Beyond the pitching magic would be the defensive shifts and framing. The Pirates focus heavily on both, and I’m not sure how much they are incorporated in the projections. The projections do highlight Fielding Runs Above Average, and the Pirates have the second best NL number here (and it’s not even close between them and the third best NL team). Francisco Cervelli also gets good marks, but it’s hard to say if that’s incorporating his framing skills. These are two areas where most projections leave the Pirates short (and these areas also help boost the pitching). I can only imagine PECOTA leaves them short as well.

One thing about any projection system is that they’re supposed to be fun, and aren’t supposed to be taken as gospel. They also don’t project the best possible outcome, although even that disclaimer leaves questions when the Pirates are projected to finish sixth in the NL and four games out of the Wild Card game. The good thing here is that the Pirates have found ways to beat their projections in large fashion in previous years. Perhaps they can do it again in 2016.

**The Pirates Prospects App is Now Available on Android. If you missed it from this weekend, our app is now available on Android. The iOS version for Apple devices will be out in the next week.

**Can Cory Luebke be the Next Pitcher to Receive the Ray Searage Magic? An interesting look at how and why Luebke ended up signing with the Pirates, and his thoughts on their success rebuilding pitchers.

**Wyatt Mathisen Getting Some Work at First Base, But Still More Valuable at Third. The work at first base is unofficial, as he’s just trying out a new position on his own. I explain why the Pirates wouldn’t make such a move, since Mathisen has a lot of value to them at third base.

**Keith Law’s Top 20 Pirates Prospects, Plus More From Baseball America. Some interesting prospect rankings here, specifically Luis Escobar, who we’ve got as a breakout candidate, but have no where close to 12th in the system.

**Baseball America Announces Draft and International Bonus Pools For Pirates. These will be important numbers to keep in mind over the summer.

  • Really good article on this very topic:

    http://www.fangraphs.com/blogs/team-win-projections-vs-actual-win-totals-2007-present/

    Long story short, the projections are only “wrong” for the Pirates if you choose to use a small enough sample.

    • Before I started this site, I worked for a company that ran computer simulations to make projections on daily games, along with season standings. My job was to analyze the projections and see what they might be missing. There were always teams that the projection system missed big on, because there was information beyond the numbers that you had to consider.

      The Pirates just seem to be one of those teams in recent years where you have to look beyond the numbers and see what the projection systems are missing. There is a sample size issue here, as the projections won’t always be wrong on the Pirates. But you need further analysis of the individual projection to see how much they make sense.

      In this case, the 2016 projections don’t make sense when you look beyond what the projection systems can see.

      • That sounds like a very logical conclusion for someone to make who has no statistical background.

        The 2016 projections *absolutely* make sense unless you believe you’ve proven that the Pittsburgh Pirates have found a sustainable way to beat sequencing. You may have done that, and if you have, go make a bunch of money working for some Major League club and not writing for this site!

  • Based on us making no major signings or trades before April 1st- My predication is 87 wins.

  • kinda hard to make that assumption for Mercer given he missed 6 weeks last year because of injury and certainly didn’t come back at full strength…. the .9 would be the outlier, and he wasn’t a full time starter in 2013 either and it was his first full year in the majors. I’m just wondering what you are making these assumptions based on

    • Well, he did suck worse *before* injury than after, so there’s that.

      • Well yeah. but the .9 would still be higher if he played another 40+ games since the stat is cumulative, unless you think those extra games wouldn’t have added at all to the .9 and i think we both know that’s really not likely, so maybe 1.5 would be his “real” score last year over a whole season, and if averaged between the 2014 year, somewhere around 2 is still pretty likely

        • That’s a common WAR misconception. WAR *is* cumulative, but it *is not* linear. Ironically, I can remember guys on this board projecting Mercer himself to be a 4ish-WAR shortstop after 2013 based on this same projection logic.

          • True, but i did word my statement carefully. It certainly is possible he would have given us 0.0 WAR during those 40 games, but it’s highly unlikely as he had found his stroke at that point and was actually hitting fairly decently, and his defense has absolutely no reason to have been anything other than above average as it never slumped. It is not like he got hurt in the middle of April/May when a monkey with a breathing reed could have outhit him. No reason to expect or project anything other than a linear projection in this case.

      • …and his sucking is what led to his .9 WAR. Unless you think he actually would have sucked worse had he been healthy, it’s a fair assumption

  • I pay about as much attention to those “Projections” as I do to Daytime soaps.

  • Framing and defense *are* rolled into Baseball Prospectus’ version of WAR (thus Cervelli’s WARP > WAR), but I’m still not sure how they work this into pitchers value. WAR is a zero sum calc; you can’t give the catcher and pitcher the same credit for a strike.

    The answer to why PECOTA has been “low” seems to be Searage, Tony Watson, and Mark Melancon:

    http://www.baseballprospectus.com/article.php?articleid=28457

    PECOTA does not project “clutch”. In two of the past three seasons where PECOTA has been “low”, the Pirates actual record has far outpaced their pythag and BaseRuns record while putting up excellent marks in one run games. The one common denominator, as far as I can figure, has been extremely clutch bullpen performances.

    PECOTA also has no way of seeing the reclamation arms as what they’ve turned out to be, not to mention huge out-of-nowhere performances by Harrison, Cervelli, and Kang as of late.

    • To which some credit must be directed to “the best management team in baseball” for putting all the “surprises” together. It will be interesting to see if this out performance continues, or “regresses to the mean”.

    • I didn’t think they put framing into a catcher’s WARP. When that first came out, they wrote about it in a way where framing was in addition to WARP. And Cervelli’s numbers don’t appear to be including the framing, unless they’re projecting him to be worse than Jordy Mercer without the framing.

      Unless they’ve changed this recently, I don’t think framing is included.

      • Surprised me as well, but I just recently read an article that explicitly said the opposite citing Cervelli’s ’15 WARP of 5.7, which is significantly higher than his FanGraphs or Bref WAR.

        BP has wonky defensive stats in general – I mean, Pedro Alvarez with +11.3 FRAA over the last two years? – but they do appear to include framing.

  • The great thing about predictions are if you are correct you look like a genius, if you are wrong all you have to say it that it is tough to predict the future.

  • They are just predictions. Frankly, I don’t think the predictions are too optimistic or too pessimistic as they relate to the Pittsburgh Pirates.

    I don’t think it’s surprising that the Pirates were projected as a .500 team last year. They had significant questions to be answered at the beginning of the year in multiple positions. They were able to win 98 games largely because they got surprisingly good years from AJ, Kang and Cervelli. They also made key additions during the year which made them much better including the addition of Ramirez and Happ.

    This year I also don’t think it is surprising that they are projected to be a .500 team. They’ve lost significantly more than they’ve added at least on paper. Again, they have major question marks at first base and in the rotation. The only players that I think are “sure things” are Cutch, Marte and Cole. Probably Watson too. But I don’t think it would be a tremendous surprise if players like Cervelli, Liriano Melanceon or Kang demonstrated a significant drop off in production this year. I don’t think it would be a surprise if the rookie call ups were disappointing or inconsistent. I don’t think it would be a surprise if Vogelsong, Locke and Neise all proved to be below average pitchers. And I don’t think it is a stretch to think none of the projects at first will turn out to be all that productive.

    Now, if all of those things go wrong for the Pirates this year, they will be significantly below .500 at the end of the year. I don’t think it is reasonable to assume everything will go wrong but I don’t think it is reasonable to assume everything will go right either, which is what happened last year. If we are going to predict, I think predicting the Pirates to be right around .500 sounds reasonable Once the games start to be played and we start to see the team gel, we can always revise our predictions later.

  • Tim: It is a business and everybody out there seems to have their own method of projecting future performances – usually most will take the immediate past into strong consideration, but I cannot figure out why the projected drop in performance for guys like ‘Cutch, Marte, Heyward and Rizzo – these are guys with 3 or more years of solid results. Sensationalism Factor? WAG’s (Wild Ass Guess)?

    I guess they have to use WARP so that it does look like something from Fangraphs, but on MLB just now, one of the guest commentators stated that their numbers from last year – giving a range of plus or minus 3 wins or losses, was correct in only 5 of the 30 teams last year. I think they had KC as a .500 team?

  • Yea, I would say Nate Silver hasn’t figured out the #SearageMagic formula yet.

    Me personally, I’m just not a fan of Vogelsong, Locke, and Niese. Along with a bunch of Indy rookies at midseason. But I’m no Leo Walter either.

  • They say 83 I say this94 or over

Menu