I can't reproduce your problem. I suspect a cache issue.
I would probably have put a colored solid above the hyperlapse footage, set the blend mode of the solid to screen or multiply so you can easily see through the layer, animated a mask on the solid, then used that as a track matte for the train. I find it easier to manipulate the masks if I work this way. I would also not set a keyframe at every frame unless there is no other way to pull off the matte. Take this shot for example - 162965230, It is the closest thing I could quickly find that kind of looked like your shot.
It took me just under 5 minutes to animate the mask for the end of the bridge. Just a few keyframes. As you can see in the screenshot
If you are interested you can download the clip from Adobe Stock (free preview) and take a look at the project file. All that is left is to turn the solid into a track matte and use that for whatever effect you want. Here's the project file: Dropbox - Hand RotoCC (15.x).aep
(note: if your browser adds a .txt extension to the .aep file you can just delete it. Here's the final...
