Initial testing
Posted: Sat Mar 26, 2011 6:23 pm
Since there seems to be so little interest in running tests with the new engine I figured I might as well give it the old go.
In these tests the ratings from my league have been used. As far as I know these ratings aren't used anywhere else so they are exclusive to our league. The tool used is the Automatic Test command on a 30 team league where each team plays 330 games per season. The target for the tests is to get all of the different statistical outcomes as close to the numbers listed for the NHL as possible, primarily by adjusting the sliders in the Sim and then by the Mass Edition option for further tweaks that go beyond what the sliders are capable of doing with the ratings as they are in their original state.
So lets proceed.
The tests began with the following settings: Shots - 50, Goals - 75, Penalties - 55, Hits - 45, Fighting 10
That produced a season where we got the following results: (NHL targets in brackets)
Goals per game: 3.15 (5.68)
Powerplay goals per game: 0.67 (1.58)
Powerplay efficiency: 9.49% (18.95%)
Powerplays per game: 7.11 (8.32)
Shorthanded goals per game: 0.15 (0.19)
Even strength goals per game: 2.33 (3.92)
Shots per game: 58.55 (60.40)
Penalty minutes per game: 22.79 (27.52)
Hits per game: 43.30 (41.92)
Blocked shots per game: 19.39 (26.28)
Fights per game: 0.37 (0.53) * the test rosters had very few fighters on them which skews this stat
Average Save Percentage: 0.946 (0.906)
Given these initial stats it became clear that scoring was way too low, as was the efficiency on the powerplay and shotblocking. The other factors were fairly accurate for a first run.
After a few more seasons had been run with variations of the sliders we ended up on season 4 where the following settings were used: Shots - 53, Goals - 99, Penalties - 62, Hits - 46, Fighting 40
These settings produced the following:
Goals per game: 4.28 (5.68)
Powerplay goals per game: 0.84 (1.58)
Powerplay efficiency: 9.98% (18.95%)
Powerplays per game: 8.39 (8.32)
Shorthanded goals per game: 0.24 (0.19)
Even strength goals per game: 3.20 (3.92)
Shots per game: 60.46 (60.40)
Penalty minutes per game: 27.17 (27.52)
Hits per game: 41.89 (41.92)
Blocked shots per game: 20.09 (26.28)
Fights per game: 0.45 (0.53)
Average Save Percentage: 0.929 (0.906)
As you can see there are a lot of categories that are pretty much right where you want them to be. But still the scoring, powerplay and shotblocking was nowhere in place and with the scoring now up to 99 there was nothing more the sliders could do to up the scoring. So we turned to Mass Edition.
Given that the csv-files the Testing tool provides has a percentage of scoring done by defensemen I could tell that defensemen were scoring 12.14% of all goals, compared to 14.6% in the NHL. So the first thing done in Mass Edition was to increase the scoring rating for defensemen by 10%.
That was the only ratings edited in a season run with these settings: Shooting - 53, Goals - 99, Penalties - 63, Hits - 46, Fighting - 45
The results were as follows:
Goals per game: 3.00 (5.68)
Powerplay goals per game: 0.55 (1.58)
Powerplay efficiency: 6.49% (18.95%)
Powerplays per game: 8.51 (8.32)
Shorthanded goals per game: 0.17 (0.19)
Even strength goals per game: 2.28 (3.92)
Shots per game: 61.55 (60.40)
Penalty minutes per game: 27.74 (27.52)
Hits per game: 42.61 (41.92)
Blocked shots per game: 20.46 (26.28)
Fights per game: 0.47 (0.53)
Average Save Percentage: 0.951 (0.906)
Not being a seasoned veteran at this Sim I didn't see that coming. By increasing Scoring for defensemen, goals per game dropped from 4.28 to 3.00 You can imagine my surprise. To make it even more surprising, the percentage of total goals scored by defensemen actually increased from 12.14% to 13.05%, still far from the 14.6% of the NHL but still remarkable. By increasing the Scoring rating for defensemen, forwards scored a LOT less despite the fact that their ratings had not been touched. Peculiar.
So obviously there was only one thing to do, run a season where Scoring for defensemen was reduced by 10% instead of increased by 10%. Settings used in this season was identical to the one above and produced the following:
Goals per game: 6.24 (5.68)
Powerplay goals per game: 1.28 (1.58)
Powerplay efficiency: 14.68% (18.95%)
Powerplays per game: 8.74 (8.32)
Shorthanded goals per game: 0.34 (0.19)
Even strength goals per game: 4.61 (3.92)
Shots per game: 59.57 (60.40)
Penalty minutes per game: 29.15 (27.52)
Hits per game: 41.50 (41.92)
Blocked shots per game: 19.82 (26.28)
Fights per game: 0.54 (0.53)
Average Save Percentage: 0.895 (0.906)
Goals per game rocket to 6.24 per game and we see the powerplay become a lot more effective as well, things we needed to see but didn't expect to see by reducing scoring. The effect of playing around with Scoring for defensemen is outlined below:
Original rating: 4.28 goals per game given settings of Shooting - 53, Goals - 99, 12.14% of all goals are scored by defensemen.
Defensemen with 10% bonus on Scoring: 3.00 goals per game given above settings, 13.05% of all goals are scored by defensemen.
Defensemen with 10% decrease in Scoring: 6.24 goals per game given above settings, 12.06% of all goals are scored by defensemen.
Defensemen with 5% decrease in Scoring: 5.15 goals per game given above settings, 12.08% of all goals are scored by defensemen.
The next step was to see if reducing Scoring for forwards would have the same effect so in the next season Scoring was reduced by 5% for both defensemen and forwards. Settings used for the season were: Shooting - 53, Goals - 99, Penalties - 62, Hits - 46, Fighting - 42
Goals per game: 7.45 (5.68)
Powerplay goals per game: 1.58 (1.58)
Powerplay efficiency: 17.37% (18.95%)
Powerplays per game: 9.07 (8.32)
Shorthanded goals per game: 0.43 (0.19)
Even strength goals per game: 5.44 (3.92)
Shots per game: 58.89 (60.40)
Penalty minutes per game: 29.24 (27.52)
Hits per game: 41.48 (41.92)
Blocked shots per game: 19.53 (26.28)
Fights per game: 0.49 (0.53)
Average Save Percentage: 0.874 (0.906)
By now it's pretty obvious that to increase scoring in the league, the Scoring ratings should be lowered. Quite surprising for me personally, but maybe that's just me.
With this season we also suddenly find ourselves having a powerplay that is exactly as effective as it should be based on the NHL standard of 1.58 powerplay goals per game. And the rate of efficiency with the man advantage reaches an all time high for these tests with a success rate of 17.37%.
There are downsides though as the goalies, who had been superhuman until now, suddenly are mediocre when the skaters they face are reduced for Scoring.
Another major problem is the offensive efficiency of the penalty killers. In the NHL they score 0.19 shorthanded goals per game. In this latest test they score 0.43, which is 126% higher than in the NHL. I believe this might be an area where the new engine needs to be tweaked as this number is consistently too high.
The next step was then to lower the slider for Scoring from the value of 99 used up until now. Over the course of the next three seasons the following settings were used: Shooting - 53 53 53, Goals - 90 80 72, Penalties - 61 61 61, Hits - 47 47 47, Fighting - 43 43 43
During these three seasons, result shifted as follows:
Goals per game: 6.69, 6.07, 5,52 (5.68)
Powerplay goals per game: 1.45, 1.47, 1.39 (1.58)
Powerplay efficiency: 16.51%, 16.55%, 15.68% (18.95%)
Powerplays per game: 8.80, 8.87, 8.85 (8.32)
Shorthanded goals per game: 0.36, 0.33, 0.31 (0.19)
Even strength goals per game: 4.88, 4.27, 3.83 (3.92)
Shots per game: 58.85, 58.96, 58.87 (60.40)
Penalty minutes per game: 29.60, 31.07, 32.07 (27.52)
Hits per game: 42.24, 42.33, 42.33 (41.92)
Blocked shots per game: 19.61, 19.67, 19.70 (26.28)
Fights per game: 0.57, 0.66, 0.74 (0.53)
Average Save Percentage: 0.886, 0.897, 0.906 (0.906)
Starting with the positives we now have a fairly accurate amount of goals being scored and the powerplay has come to life while still keeping a good amount of shots on goal. We've also managed to turn the goaltenders into just the right amount of awesome that we were looking for.
Now if lowering the Scoring rating had a surprising effect in that it actually raised the amount of goals being scored, it's even more strange to notice another effect here. When we lower the slider for Goals and keep everything else constant, we see fighting increase from 0.49 fights per game to 0.74. Remember that this increase in fighting comes without anything being done to either the Fighting or Discipline rating or from a change to any other slider but scoring being lowered from 99 to 72. Puzzling.
With this test done I tweaked the sliders to run for a season with the following: Shooting - 54, Goals - 73, Penalties - 61, Hits - 46, Fighting - 40
Goals per game: 5.71 (5.68)
Powerplay goals per game: 1.42 (1.58)
Powerplay efficiency: 16.09% (18.95%)
Powerplays per game: 8.81 (8.32)
Shorthanded goals per game: 0.31 (0.19)
Even strength goals per game: 3.98 (3.92)
Shots per game: 59.85 (60.40)
Penalty minutes per game: 31.06 (27.52)
Hits per game: 41.90 (41.92)
Blocked shots per game: 19.85 (26.28)
Fights per game: 0.68 (0.53)
Average Save Percentage: 0.905 (0.906)
For now this is where it stands. More testing will be done tomorrow.
It would be interesting to hear from Simon and jwr on a few of the things above. Especially interested in hearing if in your tests the PK is as offensively effective as it has been during these tests and if you have ideas on how to increase shotblocking which is on my list to do for tomorrow. I'm guessing the DF rating plays a part in that but will play around with different things to get it right.
I'd also like to hear from you on the matter of scoring actually going up by lowering the SC-ratings and how lowering the slider for Goals leads to an increase in fighting. Is this something you've seen in your tests as well? Now granted I haven't run multiple seasons to test these findings yet as I want to find the proper range of settings and ratings before I do so, but the effects on scoring and fighting that these changes have had is so dramatic that it can't be down to randomness of having too small of a sample size I believe.
Anyway, hell of a long thread of messing about from a complete tosser so it's time to turn it off.
Alright?
In these tests the ratings from my league have been used. As far as I know these ratings aren't used anywhere else so they are exclusive to our league. The tool used is the Automatic Test command on a 30 team league where each team plays 330 games per season. The target for the tests is to get all of the different statistical outcomes as close to the numbers listed for the NHL as possible, primarily by adjusting the sliders in the Sim and then by the Mass Edition option for further tweaks that go beyond what the sliders are capable of doing with the ratings as they are in their original state.
So lets proceed.
The tests began with the following settings: Shots - 50, Goals - 75, Penalties - 55, Hits - 45, Fighting 10
That produced a season where we got the following results: (NHL targets in brackets)
Goals per game: 3.15 (5.68)
Powerplay goals per game: 0.67 (1.58)
Powerplay efficiency: 9.49% (18.95%)
Powerplays per game: 7.11 (8.32)
Shorthanded goals per game: 0.15 (0.19)
Even strength goals per game: 2.33 (3.92)
Shots per game: 58.55 (60.40)
Penalty minutes per game: 22.79 (27.52)
Hits per game: 43.30 (41.92)
Blocked shots per game: 19.39 (26.28)
Fights per game: 0.37 (0.53) * the test rosters had very few fighters on them which skews this stat
Average Save Percentage: 0.946 (0.906)
Given these initial stats it became clear that scoring was way too low, as was the efficiency on the powerplay and shotblocking. The other factors were fairly accurate for a first run.
After a few more seasons had been run with variations of the sliders we ended up on season 4 where the following settings were used: Shots - 53, Goals - 99, Penalties - 62, Hits - 46, Fighting 40
These settings produced the following:
Goals per game: 4.28 (5.68)
Powerplay goals per game: 0.84 (1.58)
Powerplay efficiency: 9.98% (18.95%)
Powerplays per game: 8.39 (8.32)
Shorthanded goals per game: 0.24 (0.19)
Even strength goals per game: 3.20 (3.92)
Shots per game: 60.46 (60.40)
Penalty minutes per game: 27.17 (27.52)
Hits per game: 41.89 (41.92)
Blocked shots per game: 20.09 (26.28)
Fights per game: 0.45 (0.53)
Average Save Percentage: 0.929 (0.906)
As you can see there are a lot of categories that are pretty much right where you want them to be. But still the scoring, powerplay and shotblocking was nowhere in place and with the scoring now up to 99 there was nothing more the sliders could do to up the scoring. So we turned to Mass Edition.
Given that the csv-files the Testing tool provides has a percentage of scoring done by defensemen I could tell that defensemen were scoring 12.14% of all goals, compared to 14.6% in the NHL. So the first thing done in Mass Edition was to increase the scoring rating for defensemen by 10%.
That was the only ratings edited in a season run with these settings: Shooting - 53, Goals - 99, Penalties - 63, Hits - 46, Fighting - 45
The results were as follows:
Goals per game: 3.00 (5.68)
Powerplay goals per game: 0.55 (1.58)
Powerplay efficiency: 6.49% (18.95%)
Powerplays per game: 8.51 (8.32)
Shorthanded goals per game: 0.17 (0.19)
Even strength goals per game: 2.28 (3.92)
Shots per game: 61.55 (60.40)
Penalty minutes per game: 27.74 (27.52)
Hits per game: 42.61 (41.92)
Blocked shots per game: 20.46 (26.28)
Fights per game: 0.47 (0.53)
Average Save Percentage: 0.951 (0.906)
Not being a seasoned veteran at this Sim I didn't see that coming. By increasing Scoring for defensemen, goals per game dropped from 4.28 to 3.00 You can imagine my surprise. To make it even more surprising, the percentage of total goals scored by defensemen actually increased from 12.14% to 13.05%, still far from the 14.6% of the NHL but still remarkable. By increasing the Scoring rating for defensemen, forwards scored a LOT less despite the fact that their ratings had not been touched. Peculiar.
So obviously there was only one thing to do, run a season where Scoring for defensemen was reduced by 10% instead of increased by 10%. Settings used in this season was identical to the one above and produced the following:
Goals per game: 6.24 (5.68)
Powerplay goals per game: 1.28 (1.58)
Powerplay efficiency: 14.68% (18.95%)
Powerplays per game: 8.74 (8.32)
Shorthanded goals per game: 0.34 (0.19)
Even strength goals per game: 4.61 (3.92)
Shots per game: 59.57 (60.40)
Penalty minutes per game: 29.15 (27.52)
Hits per game: 41.50 (41.92)
Blocked shots per game: 19.82 (26.28)
Fights per game: 0.54 (0.53)
Average Save Percentage: 0.895 (0.906)
Goals per game rocket to 6.24 per game and we see the powerplay become a lot more effective as well, things we needed to see but didn't expect to see by reducing scoring. The effect of playing around with Scoring for defensemen is outlined below:
Original rating: 4.28 goals per game given settings of Shooting - 53, Goals - 99, 12.14% of all goals are scored by defensemen.
Defensemen with 10% bonus on Scoring: 3.00 goals per game given above settings, 13.05% of all goals are scored by defensemen.
Defensemen with 10% decrease in Scoring: 6.24 goals per game given above settings, 12.06% of all goals are scored by defensemen.
Defensemen with 5% decrease in Scoring: 5.15 goals per game given above settings, 12.08% of all goals are scored by defensemen.
The next step was to see if reducing Scoring for forwards would have the same effect so in the next season Scoring was reduced by 5% for both defensemen and forwards. Settings used for the season were: Shooting - 53, Goals - 99, Penalties - 62, Hits - 46, Fighting - 42
Goals per game: 7.45 (5.68)
Powerplay goals per game: 1.58 (1.58)
Powerplay efficiency: 17.37% (18.95%)
Powerplays per game: 9.07 (8.32)
Shorthanded goals per game: 0.43 (0.19)
Even strength goals per game: 5.44 (3.92)
Shots per game: 58.89 (60.40)
Penalty minutes per game: 29.24 (27.52)
Hits per game: 41.48 (41.92)
Blocked shots per game: 19.53 (26.28)
Fights per game: 0.49 (0.53)
Average Save Percentage: 0.874 (0.906)
By now it's pretty obvious that to increase scoring in the league, the Scoring ratings should be lowered. Quite surprising for me personally, but maybe that's just me.
With this season we also suddenly find ourselves having a powerplay that is exactly as effective as it should be based on the NHL standard of 1.58 powerplay goals per game. And the rate of efficiency with the man advantage reaches an all time high for these tests with a success rate of 17.37%.
There are downsides though as the goalies, who had been superhuman until now, suddenly are mediocre when the skaters they face are reduced for Scoring.
Another major problem is the offensive efficiency of the penalty killers. In the NHL they score 0.19 shorthanded goals per game. In this latest test they score 0.43, which is 126% higher than in the NHL. I believe this might be an area where the new engine needs to be tweaked as this number is consistently too high.
The next step was then to lower the slider for Scoring from the value of 99 used up until now. Over the course of the next three seasons the following settings were used: Shooting - 53 53 53, Goals - 90 80 72, Penalties - 61 61 61, Hits - 47 47 47, Fighting - 43 43 43
During these three seasons, result shifted as follows:
Goals per game: 6.69, 6.07, 5,52 (5.68)
Powerplay goals per game: 1.45, 1.47, 1.39 (1.58)
Powerplay efficiency: 16.51%, 16.55%, 15.68% (18.95%)
Powerplays per game: 8.80, 8.87, 8.85 (8.32)
Shorthanded goals per game: 0.36, 0.33, 0.31 (0.19)
Even strength goals per game: 4.88, 4.27, 3.83 (3.92)
Shots per game: 58.85, 58.96, 58.87 (60.40)
Penalty minutes per game: 29.60, 31.07, 32.07 (27.52)
Hits per game: 42.24, 42.33, 42.33 (41.92)
Blocked shots per game: 19.61, 19.67, 19.70 (26.28)
Fights per game: 0.57, 0.66, 0.74 (0.53)
Average Save Percentage: 0.886, 0.897, 0.906 (0.906)
Starting with the positives we now have a fairly accurate amount of goals being scored and the powerplay has come to life while still keeping a good amount of shots on goal. We've also managed to turn the goaltenders into just the right amount of awesome that we were looking for.
Now if lowering the Scoring rating had a surprising effect in that it actually raised the amount of goals being scored, it's even more strange to notice another effect here. When we lower the slider for Goals and keep everything else constant, we see fighting increase from 0.49 fights per game to 0.74. Remember that this increase in fighting comes without anything being done to either the Fighting or Discipline rating or from a change to any other slider but scoring being lowered from 99 to 72. Puzzling.
With this test done I tweaked the sliders to run for a season with the following: Shooting - 54, Goals - 73, Penalties - 61, Hits - 46, Fighting - 40
Goals per game: 5.71 (5.68)
Powerplay goals per game: 1.42 (1.58)
Powerplay efficiency: 16.09% (18.95%)
Powerplays per game: 8.81 (8.32)
Shorthanded goals per game: 0.31 (0.19)
Even strength goals per game: 3.98 (3.92)
Shots per game: 59.85 (60.40)
Penalty minutes per game: 31.06 (27.52)
Hits per game: 41.90 (41.92)
Blocked shots per game: 19.85 (26.28)
Fights per game: 0.68 (0.53)
Average Save Percentage: 0.905 (0.906)
For now this is where it stands. More testing will be done tomorrow.
It would be interesting to hear from Simon and jwr on a few of the things above. Especially interested in hearing if in your tests the PK is as offensively effective as it has been during these tests and if you have ideas on how to increase shotblocking which is on my list to do for tomorrow. I'm guessing the DF rating plays a part in that but will play around with different things to get it right.
I'd also like to hear from you on the matter of scoring actually going up by lowering the SC-ratings and how lowering the slider for Goals leads to an increase in fighting. Is this something you've seen in your tests as well? Now granted I haven't run multiple seasons to test these findings yet as I want to find the proper range of settings and ratings before I do so, but the effects on scoring and fighting that these changes have had is so dramatic that it can't be down to randomness of having too small of a sample size I believe.
Anyway, hell of a long thread of messing about from a complete tosser so it's time to turn it off.
Alright?