HoopDon wrote:HoopNerd, I’ve scrolled through your website and stats multiple times, but I couldn’t find any data on how accurate they were (in terms of correlation to current and future team success), so I ignored them. If your numbers have excellent in-season accuracy, and can predict future performance as well as ASPM/BPM or even (gasp) XRAPM/RPM, I’d use them regularly. If your numbers can do it without using prior-seasons and artificial adjustments (all rookies getting an automatic negative value), I’ll be your #1 fan.More info on how you calculate your numbers will also be appreciated, cheers.http://hoopdon.weebly.com/
Well, stay tuned, I’ll just have to see how comfortable I am about how much I’ll reveal in terms of calculations. I might just, at the very least, project the next season of every single past season (since 1980) in terms of production per minute (adjusted for pace/blah blah blah) – & since we know the actual minutes played of every guy in their following seasons, plug in the projections tied to actual minutes and see how the compiled projected team results compare to actual results. I haven’t done this because, honestly, no one has ever asked until recently about the predictive nature of the results until the last couple of days – & I don’t know how much people trust someone “testing” their own work & giving results. But obviously it’s something I should do & try my best to do a comprehensive write up to show the honesty behind the results.
Now, the problem with this is that there has to be some trust in my testing my own projections. I think if I give ALL results of every player season projection from the previous season (all the good and all the bad) in a massive spreadsheet – then it’ll be pretty obvious that I’m trying to show full disclosure and not fudging things on the fly. There will be a ton of hits from player to player, but obviously a bunch of misses too. It’ll be interesting to see how close, in general, the projections are & where the standard deviations fall.
Similarly – I plan on projecting the NBA careers EVERY SINGLE D1 player since 1997 & ranking them all in terms of their NBA career projections. I’ll limit the posted results to just guys that project, at some point in a made up NBA career, to be above replacement level. This is a huge project (getting all the DoB’s for all guys that could project above replacement level in a NBA career might be a serious pain – since a good number might not ever appear on basketball-reference) – since I’m not going to try to do what it appears everyone else does & limit my scope to the 80 or so college guys on NBA draft radars (often just for simplicity, but sometimes masking the flaws of their own metric) – but EVERY guy. How much can my methodology be trusted if I pick & choose which college players I apply it to. A good methodology should give solid predictive results on any & every player. If I limit my data set to avoid players (say Javon McCrea last year – I noticed a couple draft projection metrics ignore him, I’m guessing possibly because he measured out too high) who might break the system, then what’s the point? I want to catch that undrafted guy – the Udonis Haslem & the like.
The good thing is, when I finish the above project, I’ll be able to rank every college prospect for every past season – & see if my rankings (without ANY scout, combine, or prospect rank bias) compare to the actual drafts. Since I did already do the college ratings of the last season of every guy that played at least 1 minute the following season as an NBA rookie (in order to create the conversions) – I am pretty confident the projections could very well out perform the actual draft positions. Most “busts” (occasional caveats – Beasley for example projected great) were obvious bust material by the projections. Most sleepers (2nd round studs – Boozer, Blair, etc) were very highly projected – MUCH higher than many of the guys drafted before them (Jarnell Stokes in this last draft would be a comp to those two btw). I can’t get them all right by any means (Bledsoe) – I just think I can do better than the general draft history of the past gms. I wanted to do all this before this last draft – never got anywhere close. Just producing college ratings for every single guy that played in the NBA as a rookie was majorly time consuming (I have to compile all the college ratings at the team level for all those guys). I will have all this done well before the next NBA draft – come hell or high water.
Also, another good thing about the draft project – I could run similarity scores with all past college players since ’97 – see which players who appear most similar to guys that succeeded in the NBA. I could run similarity scores of every projected NBA season to all NBA seasons at every age group since 1980, see if certain player types are being overlooked. This might be a 2nd approach I may eventually merge with the first if it improves the predictability. Have zero idea if I’ll have this fully workable by the next NBA draft. If I were working for a team, it would be because all other projects would probably be back burnered.
So, to answer your point before I went off on a massive tangent, yes HoopDon – I should run all the past projections and show how much they correlate to actual results, & post all the results. I’ll post on this board & on Twitter when I do if you are curious about the results.
If you are curious about how to apply the predictive power of the player ratings (if you trusted them at all) – scroll back to any of my numerous March madness game write ups. I predict the final score of in each write up based on predicted player minutes. Here’s the write up I did for the NCAA championship game:
Note that my general team ratings going into that game had Kentucky winning by 1.1 points. But, when I compiled the optimized lineup player ratings, & with Kentucky without Cauley-Stein – my prediction came out to UConn by 1.3. I surmised going into that game that losing Cauley-Stein cost UK, theoretically, 2.1 points.
Hey – I forgot that I attempted to quantify how good Jack Taylor (the guys that scored over 100 points twice in a college game) really was relative to an average D1 player. I think some of my methodology & how it adjusts for team & pace (that is an EXTREME team in terms of pace) is made a little more evident there. I never saw anyone even dare to try to quantify his play. I noticed this because it was the article before the NCAA championship game. Good times: