Updating Game Report

[EmptyNet]

I'm looking into writing up a program that lets you watch games in real time and I was wondering if it was possible to update the output of the game page. Currently the Play By Play section shows every event down to the second (ex: 0:04 of 1st period - Gustav Nyquist is hit by Hampus Lindholm) whereas the Full Play By Play shows all the events in blocks of one minute and (so it's hard to tell at which point they occured - ex: "Time : 1. Pass to Jason Pominville. Pass to Brad Richards. Snap shot by Brad Richards. Shot Blocked by Connor Murphy.") Also the formatting starts at "Time : 1", while events are listed for "Time : 0" there's no actual "Time : 0" entry.

Basically I'm looking to scrape both Play By Plays from the game page and parse it - would be very helpful if both could be output in the same format so I can display events as they happen.

Just wondering if it would be possible to make a change like this.

Thank you!

[SimonT]

The Full Play by Play can be parse by using the dot as separator no?

[EmptyNet]

The Full Play by Play can be parse by using the dot as separator no?

That encounters problems with players who have a . in their name (ex: J.T. Brown) but the real issue is that it shows all events that happen in a given minute. So everything in "Time : 8" happens at some point during Minute 8 but it's not clear when. Basically I'd like to parse all the events on a second-by-second basis so I can reply them in real time and display them on the screen (and try to create an animated representation of the game).

This would work well from the play by play as it is printed now, it just lacks the full information from the full play by play (as the name suggests).

[SimonT]

Did you try to look at the XML files instead of the HTML? You should get better information in there.

[ynohtna]

So cool... I posted something like this a few years back about somehow getting the play by play to come out bit by bit! Nice that you can tackle this!

[EmptyNet]

Did you try to look at the XML files instead of the HTML? You should get better information in there.

I took a look at the XML file for a game but it seems to be broken down in the same way without additional timing information I'm looking for.

Here is an example of what I'm trying to do. Sample output for a game (taken from rendered page for ease of reading but data is same in XML file)

Play by Play

1st period


0:01 of 1st period - Jeff Carter wins face-off versus Mike Hoffman in neutral zone.
0:32 of 1st period - Snap shot by Mike Hoffman.
0:32 of 1st period - Stopped by Semyon Varlamov without a rebound.
0:33 of 1st period - Nicklas Backstrom wins face-off versus Brad Richards in TEAM B zone.
0:50 of 1st period - Slapshot by Brad Richards.

Full Play by Play

1st period

Normal Forward Lineup #1 - Mike Hoffman, Joe Pavelski, Lee Stempniak are on ice for TEAM A. Normal Defense Lineup #1 - Hampus Lindholm, Christopher Tanev are on ice for TEAM A. Normal Forward Lineup #1 - Jeff Carter, Gustav Nyquist, T.J. Oshie are on ice for TEAM B. Normal Defense Lineup #1 - Drew Doughty, Duncan Keith are on ice for TEAM B. Jeff Carter wins face-off versus Mike Hoffman in neutral zone. Pass to Gustav Nyquist. Gustav Nyquist moves puck in TEAM A zone. Pass to T.J. Oshie. Pass to Drew Doughty. Pass to Gustav Nyquist. Pass by Gustav Nyquist intercepted by Hampus Lindholm. Pass to Mike Hoffman. Mike Hoffman moves puck in neutral zone. Pass to Joe Pavelski. Pass to Lee Stempniak in TEAM B zone. Pass to Joe Pavelski. Pass to Mike Hoffman. Snap shot by Mike Hoffman. Stopped by Semyon Varlamov without a rebound. Normal Forward Lineup #2 - Nicklas Backstrom, Cody Eakin, Zemgus Girgensons are on ice for TEAM A. Normal Defense Lineup #2 - Connor Murphy, Jonas Brodin are on ice for TEAM A. Normal Forward Lineup #2 - Brad Richards, Jason Chimera, Jason Pominville are on ice for TEAM B. Normal Defense Lineup #2 - Ryan Suter, Marc-Edouard Vlasic are on ice for TEAM B. Nicklas Backstrom wins face-off versus Brad Richards in TEAM B zone. Pass to Zemgus Girgensons. Pass to Nicklas Backstrom. Pass by Nicklas Backstrom intercepted by Ryan Suter. Pass to Jason Pominville in neutral zone. Jason Pominville moves puck in TEAM A zone. Pass to Ryan Suter. Pass to Jason Pominville. Pass to Ryan Suter. Pass to Brad Richards. Slapshot by Brad Richards. Stopped by Devan Dubnyk with a rebound. Free Puck Retrieved by Jason Pominville for TEAM B. Pass to Ryan Suter. Pass to Jason Pominville. Pass to Marc-Edouard Vlasic.

Time : 1. Pass to Brad Richards. Pass to Ryan Suter. Pass to Jason Pominville. Connor Murphy stole the puck from Jason Pominville...

I have bolded two corresponding entries in both PbP and FPbP logs. I'll only look at the first one but the second one is just another example. I'm most interested in having my game visualizer display important events (those that make the cut to graduate from just FPbP to the PbP section). So here we see there's a faceoff that Mike Hoffman loses 1 second into the game. Then there's nothing for 31 seconds until Hoffman makes a snap shot. To see what happened in between the face off and the snap shot we look at the FPbP.

There is no "Time : 0" entry as there is for every other minute (it is easy for a human reader to determine this but not a computer parser). Looking at the section with the implied "Time : 0" entry covers all game events from 00:00 to 00:59. We see a series of passes/puck movements through the different zones until Pavelski passes to Hoffman who takes his shot. All we know is that all those passes happened between 00:01 and 00:32 when Hoffman took his shot but there's no way to know if Pavelski passed to Hoffman at 00:02 and he held onto it for 30 seconds before shooting or if the pass took place at 00:32.

It would be most helpful if the FPbP log could be output in the same format as the PbP log.

So instead of "Time : 1" and then all the events in that minute jumbled together, a modified version of the FPbP log from above could say:

0:01 of 1st period - Jeff Carter wins face-off versus Mike Hoffman in neutral zone.
0:03 of 1st period - Pass to Gustav Nyquist.
0:05 of 1st period - Gustav Nyquist moves puck in TEAM A zone.
0:08 of 1st period - Pass to T.J. Oshie.
0:09 of 1st period - Pass to Drew Doughty.

Is it possible to break the FPbP log down like that? Without all this information the game vizualizer is very jerky and not always sensible (in this example after the face-off the vizualizer would show the puck stay in the neutral zone after the face-off doing nothing until 00:32 seconds when Hoffman takes a shot (and even then, without parsing the FPbP log I don't know in which zone the shot came from to properly display it on screen as the PbP log does not mention zones). Hopefully this is a better explanation of what I am looking to do and what would help me achieve that.

Also I see it does make mention of which zone the puck is in sometimes (attacking, defending or neutral) but not always. Is it correct to assume that if not mentioned the zone has not changed? (ie: until the next time a zone is mentioned, all entries reference events in the "current" zone?)

Thank you for your help with my issue as well as all the hard work you have put into making a great piece of software!

[SimonT]

Hi.

I have a look at the XML files and I understand you. But the change are you asking me will be very long to do for me and will only benefit you.

[36Henry]

Is it correct to assume that if not mentioned the zone has not changed? (ie: until the next time a zone is mentioned, all entries reference events in the "current" zone?)

Yes. The puck stays in the zone until we're told otherwise.

Is it possible to break the FPbP log down like that?

the change are you asking me will be very long to do for me and will only benefit you.

I don't think such a change is really needed as, technically, you could probably add time stamps yourself. Let me explain.

Here's a couple of examples of "timed" plays:

Matt Cullen wins face-off versus Mikael Backlund in Calgary Flames zone.
Pass to Pierre-Marc Bouchard.
Pass to Devin Setoguchi.
Pass to Matt Cullen.
Wrist shot by Matt Cullen.
Stopped by Miikka Kiprusoff with a rebound.
Free Puck Retrieved by Pierre-Marc Bouchard for Minnesota Wild.
Wrist shot by Pierre-Marc Bouchard.
Shot Blocked by TJ Brodie.
Free Puck Retrieved by Matt Cullen for Minnesota Wild.
Snap shot by Matt Cullen.
Shot Misses the Net.
Free Puck Retrieved by Tom Gilbert for Minnesota Wild.
Pass to Devin Setoguchi.
Snap shot by Devin Setoguchi.
Shot Misses the Net.

1:19 of 1st period - Matt Cullen wins face-off versus Mikael Backlund in Calgary Flames zone.
1:26 of 1st period - Wrist shot by Matt Cullen.
1:26 of 1st period - Stopped by Miikka Kiprusoff with a rebound.
1:28 of 1st period - Wrist shot by Pierre-Marc Bouchard.
1:28 of 1st period - Shot Blocked by TJ Brodie.
1:30 of 1st period - Snap shot by Matt Cullen.
1:30 of 1st period - Shot Misses the Net.
1:34 of 1st period - Snap shot by Devin Setoguchi.
1:34 of 1st period - Shot Misses the Net.

We can learn quite a few things from just these 15 seconds of play.

1. The two shots taken at 1:28 & 1:30 were plays where a player picked up a rebound and fired a shot on net. These plays take two seconds.

2. The play at 1:34 was a play where a player picked up a rebound and passed the puck to another player who then took a shot on net. The added play of the pass took an extra two seconds.

3. The play at 1:26 took seven seconds to develop from face-off to shot on net. The play consisted of a face-off win, three passes and a shot. If we give a passing play two seconds to develop, we can then afford a face-off win one second, giving us the seven seconds needed to account for this shot from Matt Cullen to develop.

Here's how the play breaks down if we add Start and End time stamps to each play:

Start 1:18 - End 1:19 - Matt Cullen wins face-off versus Mikael Backlund in Calgary Flames zone.
Start 1:20 - End 1:21 - Pass to Pierre-Marc Bouchard.
Start 1:22 - End 1:23 - Pass to Devin Setoguchi.
Start 1:24 - End 1:25 - Pass to Matt Cullen.
Start 1:26 - End 1:26 - Wrist shot by Matt Cullen.
Start 1:26 - End 1:26 - Stopped by Miikka Kiprusoff with a rebound.
Start 1:27 - End 1:27 - Free Puck Retrieved by Pierre-Marc Bouchard for Minnesota Wild.
Start 1:28 - End 1:28 - Wrist shot by Pierre-Marc Bouchard.
Start 1:28 - End 1:28 - Shot Blocked by TJ Brodie.
Start 1:29 - End 1:29 - Free Puck Retrieved by Matt Cullen for Minnesota Wild.
Start 1:30 - End 1:30 - Snap shot by Matt Cullen.
Start 1:30 - End 1:30 - Shot Misses the Net.
Start 1:31 - End 1:31 - Free Puck Retrieved by Tom Gilbert for Minnesota Wild.
Start 1:32 - End 1:33 - Pass to Devin Setoguchi.
Start 1:34 - End 1:34 - Snap shot by Devin Setoguchi.
Start 1:34 - End 1:34 - Shot Misses the Net.

If we instead focus on the Mike Hoffman shot at 0:32 of the game you referenced and break it down we see that the play developed like this:

Jeff Carter wins face-off versus Mike Hoffman in neutral zone.
Pass to Gustav Nyquist.
Gustav Nyquist moves puck in TEAM A zone.
Pass to T.J. Oshie.
Pass to Drew Doughty.
Pass to Gustav Nyquist.
Pass by Gustav Nyquist intercepted by Hampus Lindholm.
Pass to Mike Hoffman.
Mike Hoffman moves puck in neutral zone.
Pass to Joe Pavelski.
Pass to Lee Stempniak in TEAM B zone.
Pass to Joe Pavelski.
Pass to Mike Hoffman.
Snap shot by Mike Hoffman.

0:01 of 1st period - Jeff Carter wins face-off versus Mike Hoffman in neutral zone.
0:32 of 1st period - Snap shot by Mike Hoffman.

The play consists of one face-off win, ten passes, one interception and two zone entries where players carried the puck into the zone. In total these plays took 32 seconds.

If we give each passing play two seconds and the face-off win one second as per the examples above, that gives us a total of 21 seconds accounted for, leaving 11 seconds for one interception and two zone entries. We might assume that an interception is similar to picking up a loose puck as with the rebounds above, which would leave 9 seconds for two zone entries.

Obviously, the sample size is terribly small here and I haven't looked at it in any detail what so ever so the notion of all passing plays taking the same amount of time may very well be completely false, but for arguments sake let's assume they do. The zone entries are interesting then because I would imagine that they would not be fixed in the same fashion in terms of their duration (and having 9 seconds for two zone entries in this example points us in this direction). When Nyquist for example carried the puck into the Team A zone we don't know for how long he held on to it before passing it to Oshie. Did he just cross the blue line and then make the pass, or did he carry the puck deep into the zone, execute a cycle play with Oshie in the corner before Oshie passed it back to Doughty on the point? Or perhaps more importantly; for how long did he carry the puck in the neutral zone before crossing the blue line? We just don't know.

Here's another example where a player carried the puck into the offensive zone:

Matt Stajan wins face-off versus Mikko Koivu in neutral zone.
Pass to Alex Tanguay.
Alex Tanguay moves puck in Minnesota Wild zone.
Wrist shot by Alex Tanguay.
Stopped by Niklas Backstrom with a rebound.

8:11 of 1st period - Matt Stajan wins face-off versus Mikko Koivu in neutral zone.
8:14 of 1st period - Wrist shot by Alex Tanguay.
8:14 of 1st period - Stopped by Niklas Backstrom with a rebound.

Clearly we see how Tanguay carried the puck for a much shorter time than what we can expect to have happened in your game leading up to the Hoffman shot. If we add Start and End stamps to the Tanguay shot it looks like this:

Start 8:10 - End 8:11 - Matt Stajan wins face-off versus Mikko Koivu in neutral zone.
Start 8:12 - End 8:13 - Pass to Alex Tanguay.
Start 8:13 - End 8:13 - Alex Tanguay moves puck in Minnesota Wild zone.
Start 8:14 - End 8:14 - Wrist shot by Alex Tanguay.
Start 8:14 - End 8:14 - Stopped by Niklas Backstrom with a rebound.

This play quite clearly hints at how the duration of puck transports can be flexible. But if we were to assign time stamps to these plays, it wouldn't be terribly difficult to give that type of play a flexible duration in order to fit all the fixed plays into the time frames provided by the Play by play.

An example for the Hoffman shot could look like this:

Start 0:00 - End 0:01 - Jeff Carter wins face-off versus Mike Hoffman in neutral zone.
Start 0:02 - End 0:03 - Pass to Gustav Nyquist.
Start 0:04 - End 0:08 - Gustav Nyquist moves puck in TEAM A zone.
Start 0:09 - End 0:10 - Pass to T.J. Oshie.
Start 0:11 - End 0:12 - Pass to Drew Doughty.
Start 0:13 - End 0:14 - Pass to Gustav Nyquist.
Start 0:15 - End 0:16 - Pass by Gustav Nyquist intercepted by Hampus Lindholm.
Start 0:17 - End 0:18 - Pass to Mike Hoffman.
Start 0:19 - End 0:23 - Mike Hoffman moves puck in neutral zone.
Start 0:24 - End 0:25 - Pass to Joe Pavelski.
Start 0:26 - End 0:27 - Pass to Lee Stempniak in TEAM B zone.
Start 0:28 - End 0:29 - Pass to Joe Pavelski.
Start 0:30 - End 0:31 - Pass to Mike Hoffman.
Start 0:32 - End 0:32 - Snap shot by Mike Hoffman.

Interesting project. Wish I had time to play around with it some more.

[EmptyNet]

Hi.
I have a look at the XML files and I understand you. But the change are you asking me will be very long to do for me and will only benefit you.

I was afraid that might be the case. I do understand that you do not have time to make significant changes that have not been requested by anyone else!

Instead of adding all time stamps is it possible for you to change the Full Play By Play to include an explicit "Time : 0" entry instead of just leaving it out? If that is something you can do without taking very much time it would be very much appreciated!

I don't think such a change is really needed as, technically, you could probably add time stamps yourself. Let me explain.

If we give each passing play two seconds and the face-off win one second as per the examples above, that gives us a total of 21 seconds accounted for, leaving 11 seconds for one interception and two zone entries. We might assume that an interception is similar to picking up a loose puck as with the rebounds above, which would leave 9 seconds for two zone entries.

Thank you for the detailed reply, it gave much to think about!

My main issue is that I want to do as little guessing/assumption as possible (the stricter the requirements of the data set the easier it is to write a parser rather than having to try and deal with a ton of different "edge cases").

Instead of trying to guess a reasonable time for every play, your post made me think perhaps the best approach in this situation would be to take the known gap (say it's 30 seconds between the faceoff and when Hoffman takes his shot) and then evenly distribute all the intervening events. So if there were 15 passes in between the faceoff and Hoffman's shot, we'd manually assign them in 2-second intervals.

Although this might not be the most accurate way (vs a deeper analysis like you suggested) it does have a big benefit when looking at visualizing the game which is to smooth the action out across any information gaps so there are no big awkward pauses or jumps.

[SimonT]

There is other program that parses the HTML data of STHS games. Changing it would break them.

Again, the changed are you asking me will be very long to do for me, will only benefit you and all for this for a program that I didn’t see and I don’t even know if you’ll given it or sell it to the rest of the STHS community.