ADS Ratings: 2021-2022

This forum should be use to discust players ratings. / Ce forum devrait être utilisé par discuté des côtes des joueurs.
Post Reply
doogiski
New in Town / Le Ptit Nouveau
Posts: 13
Joined: Thu Sep 09, 2021 12:12 pm

ADS Ratings: 2021-2022

Post by doogiski »

After being involved in the SIM world for the past 5 years, I decided to help out the first league I join and attempted to created my first SIM ratings package last year. This packages was used in VHL, PHL and PFHL with great success. In my home league - VHL - input was taken from the league to the tune of a thread with over 75 responses which numerous points of feedback were fed back into the ratings package to improve it. Now that the "beta" year is over, I've been able to tweak the package a bit further and feel comfortable releasing it to the greater STHS world for the 2022-2023 season.

How this package it built?

A total of 27 different datasets using basic, advanced and micro stats from NaturalStatTrick, NHL.com's API, Dobber, MoneyPuck, Elite Prospects, Evolving Hockey and Corey Sznajder are calculated on in Python with over 3900 lines of code. With the package model written in Python instead of macros in Excel, it allows for quick iteration to make the package more accurate without redoing hours of calculations with each change.

Using this large swath of data, I feel I've created some of the most accurate ratings for contentious categories such as SK and particularly DF.

This package includes a complete rating set for any player that played a single game in the AHL or NHL this past season.

The ratings only reflect the most recent season of play and doesn't come in a multi-year weighted flavour.

This package doesn't include a PO rating.

For this year, it also has all forwards set to "C". This will be changed for next year's package.

Why this package?

The ADS ratings package is a 100% stats based approach. Yes, that includes the SK category. I've been able to find a combination of basic, advanced and micro stats that accurately act as a proxy to calculate SK for all NHL players. This means it is 100% objective and has taken human bias out of the equation.

Additionally, since the ratings model is written in Python. It allows for superior consistency year-over-year as the same model is used yearly with minor tweaks instead of the ratings being built from the ground-up at the end of every season.

I am a proponent of transparency and open source, so included in the package is slider settings for each category as well as a breakdown of the formulas used for each category and which stats were used. Additionally, for those interested, I can link my GitHub repo which holds the Jupyter Notebook of the ratings model as well as any scripts used to pull data from NHL.com and Elite Prospects.

Can I see a sample?

Here is a link to a Google Sheet that has the current rosters (as of July 16th, 2022) of the Canucks and Avalanche based on the ratings to give you a feel of what they look like. https://docs.google.com/spreadsheets/d/ ... sp=sharing

What is the cost?

The package costs $50CAD. If interested you can email me at [email protected].

Since I'm the new kid on the block, I asked a couple of commissioners their thoughts on the ratings package after using them for the most recently finished season. Here's what they had to say:
I have ran the Virtual Hockey League since 2006, and in my 15+ years as a commissioner, I have never been more impressed with an NHL ratings package before. Adam demonstrates insane commitment and accuracy in generating authentic ratings. His first package was developed for the 2021-2022 season which we used, and they went above and beyond expectation. If you want authentic ratings that are pulled from raw NHL datasets with no subjective or personal opinion factored in, Adam's ratings are bar none the most accurate set you'll ever find in the sim hockey world. There is even a detailed glossary index demonstrating how each individual category was determined, and if you have any questions, Adam is always happy to help answer your concerns thoroughly. You won't be disappointed!
- Eric, Commissioner of the Virtual Hockey League
The PHL is a league that has been going for 15 years now and having used Adam's for the most recent season, I can say they are some of the most accurate we've used. Every rating has a clear and concise reason for being what it is...and Adam is very transparent about the formula. He is able to blend traditional stats with advanced metrics in a way that makes sense, giving each player a true representation of their real life self. If you are a real life re-rate league like we are, I would highly recommend Adam's rating package.
- Tom,
Commissioner of the Perennial Hockey League

Additional Bits
If you have any questions, comments or concerns, please let me know and I'd be more than happy to chat. Thanks!
august
The Crazy / Le Fou
Posts: 440
Joined: Mon Oct 23, 2006 7:00 am

Re: ADS Ratings: 2021-2022

Post by august »

What version of simulator are the rating for?
doogiski
New in Town / Le Ptit Nouveau
Posts: 13
Joined: Thu Sep 09, 2021 12:12 pm

Re: ADS Ratings: 2021-2022

Post by doogiski »

august wrote: Sun Jul 17, 2022 6:29 am What version of simulator are the rating for?
This rating package is for v1.5 only currently, but hoping to expand to other versions in future years.
JimToupet
STHS Dynamic Website Contributor
Posts: 363
Joined: Wed Jul 02, 2014 10:55 am

Re: ADS Ratings: 2021-2022

Post by JimToupet »

Any plan to make ratings based on 2 years ?
BeardCommish
The Accomplished One / L'Accompli
Posts: 136
Joined: Sun Jul 12, 2020 12:22 pm

Re: ADS Ratings: 2021-2022

Post by BeardCommish »

Honest review of these ratings - they aren't bad. However, I found a decent amount of misspelled names, some incorrect or out-of-date vitals, and a good amount of duplicate players (easy enough to fix, though). And while I get these are built for 1.5, I think that's such a small subset of the sim community now (from what I've seen) that I'm not sure how effective these will be for most without a general tweak across the board.

Just a personal opinion, but too many "microstats" I think cause some ratings to be a bit flatter than they should. There are stats, and then there are interpretative advanced stats that aren't still as reliable to determine a player's abilities as some may feel they do - that's just my opinion (obviously there are some advanced stats that are quite good - it's subjective, is my point).

One thing I will say, though, is that I do applaud these folks for putting an alternative out to the public. It's high time, and I hope healthy competition is a rising tide that lifts all boats. We are also nearing completion on a set of ratings that will be heading to the STHS community soon, so honestly, the more, the merrier. It's all going to end up benefiting the community in the long run, so kudos for the effort folks. :)
doogiski
New in Town / Le Ptit Nouveau
Posts: 13
Joined: Thu Sep 09, 2021 12:12 pm

Re: ADS Ratings: 2021-2022

Post by doogiski »

JimToupet wrote: Tue Jul 26, 2022 11:39 am Any plan to make ratings based on 2 years ?
Yes there are. I have created packages now for the 2019-2020, 2020-2021 and 2021-2022 seasons so if required, I can create a blended package. If you are interested please send me a message with the weights you'd prefer to have on each season.

Next season, I plan on having a 1 year and 2 year weight ratings package available from the start.
doogiski
New in Town / Le Ptit Nouveau
Posts: 13
Joined: Thu Sep 09, 2021 12:12 pm

Re: ADS Ratings: 2021-2022

Post by doogiski »

BeardCommish wrote: Tue Jul 26, 2022 1:45 pm Honest review of these ratings - they aren't bad. However, I found a decent amount of misspelled names, some incorrect or out-of-date vitals, and a good amount of duplicate players (easy enough to fix, though). And while I get these are built for 1.5, I think that's such a small subset of the sim community now (from what I've seen) that I'm not sure how effective these will be for most without a general tweak across the board.

Just a personal opinion, but too many "microstats" I think cause some ratings to be a bit flatter than they should. There are stats, and then there are interpretative advanced stats that aren't still as reliable to determine a player's abilities as some may feel they do - that's just my opinion (obviously there are some advanced stats that are quite good - it's subjective, is my point).

One thing I will say, though, is that I do applaud these folks for putting an alternative out to the public. It's high time, and I hope healthy competition is a rising tide that lifts all boats. We are also nearing completion on a set of ratings that will be heading to the STHS community soon, so honestly, the more, the merrier. It's all going to end up benefiting the community in the long run, so kudos for the effort folks. :)
Thank you very for your honest review. I really do appreciate it. It will help me in building a better package moving forward. I will counter with some of your points though.

"Decent amount of misspelled names" is an exaggeration IMO. The methodology for this package was to rate NHL players and AHL players accordingly with their cohorts. If a player was determined to be of NHL quality, their name was pulled from NHL.com. If a player was determined to be of AHL quality, their name was pulled from Elite Prospects.

A prime example of this is the goalie Jean-Francois Berube. Depending on which service you pull his data from, his name is spelled 3 different ways:

1. NHL.com = J-F Berube https://www.nhl.com/player/j-f-berube-8475234
2. Elite Prospects = Jean-Francois Berube https://www.eliteprospects.com/player/2 ... ois-berube
3. HockeyDB = J.F. Berube https://www.hockeydb.com/ihdb/stats/pdi ... pid=116737

This discrepancy is also true of many Russian names and is highlighted by Vladimir Tkachyov.

1. NHL.com = Vladimir Tkachev https://www.nhl.com/player/vladimir-tkachev-8478250
2. Elite Prospects = Vladimir Tkachyov https://www.eliteprospects.com/player/2 ... r-tkachyov
3. HockeyDB = Vladimir Tkachev https://www.hockeydb.com/ihdb/stats/pdi ... pid=163405

Depending on which service you consider the "source of truth", one or two other services have spelled his name wrong and they should notified to correct this mistake.

Additionally, I agree that microstats are contentious for some, but I would rather have those included than someone's biased eye-test. Having said that, I'm unsure of how there are too many microstats since they were used only in the CK and SK ratings.

Finally, if you are creating your own ratings package, why did you purchase this one? Was it to take my framework and formulas to build upon? If so, I'll happily take a commission cut when you release yours ;)
BeardCommish
The Accomplished One / L'Accompli
Posts: 136
Joined: Sun Jul 12, 2020 12:22 pm

Re: ADS Ratings: 2021-2022

Post by BeardCommish »

Oh believe you me, the battle between websites of name spellings is one I've waged many times before haha. It's an unending battle. :P

LOL...we've created ours from scratch and are happy to show the work once we get there. The formulas have been in process for years - we got to a point recently of whether or not to actually go through with the work or not, and once we saw another entry into the fire, we thought we'd take a flier and see if it was worth continuing on.

Ultimately, we decided to continue pursuing our own. We've purchased packs before and ultimately never used them. Not only do I not mind personally in terms of helping toss a few bucks to the folks putting in the hard work, but the vast differences I think actually help everyone involved. You'll see once we get ours out that they are quite different.

Here's a random selection of our current worksheet. It's not fancy coding or open source (my wife actually works for Red Hat, but none of her expertise sadly has rubbed off on me), but it works in a similar way. We've got over 220 columns of stats collated, though. :)

Image
mr.hockey17
New in Town / Le Ptit Nouveau
Posts: 15
Joined: Fri Jun 22, 2018 1:10 pm

Re: ADS Ratings: 2021-2022

Post by mr.hockey17 »

I haven't use these ratings yet, but doogiski helped me a lot for my contracts problem!

I'll certainly give it a try one day! I am really grateful for the help and support he gave me!

Good job man! :D
ynohtna
The Addict / Le Drogué
Posts: 770
Joined: Mon Aug 14, 2006 5:15 pm
Location: Vancouver, BC
Contact:

Re: ADS Ratings: 2021-2022

Post by ynohtna »

doogiski wrote: Sun Jul 17, 2022 9:22 pm
august wrote: Sun Jul 17, 2022 6:29 am What version of simulator are the rating for?
This rating package is for v1.5 only currently, but hoping to expand to other versions in future years.
Looking forward to someone focusing on the V2.1 sim engine :)
BFHL Admin/Commish
http://www.thebfhl.ca/bfhl
Now in Season 15, using BRHL Player v2.1 Ratings modified.
doogiski
New in Town / Le Ptit Nouveau
Posts: 13
Joined: Thu Sep 09, 2021 12:12 pm

Re: ADS Ratings: 2021-2022

Post by doogiski »

ynohtna wrote: Sun Oct 30, 2022 2:56 am
doogiski wrote: Sun Jul 17, 2022 9:22 pm
august wrote: Sun Jul 17, 2022 6:29 am What version of simulator are the rating for?
This rating package is for v1.5 only currently, but hoping to expand to other versions in future years.
Looking forward to someone focusing on the V2.1 sim engine :)
I am currently road-mapping this project for next spring and can confirm there will be a decent chunk of time put towards testing all engine versions to deliver the package with recommended playability settings.
Post Reply