I don't think this issue has anything to do with your selected video player. The error is coming from Gmail's integrated video player- it tries to play the video file within the browser.
As to why it would be failing- Apparently people have had this issue for a while. It seems like it has to do with some kind of redesign of Gmail that Google did recently- can you find any option to switch to "Classic Gmail"?
It's also possible that previously it was automatically downloading and "running" the video file, which would in that case show the video file in VLC after it downloads.