As long as its just light background noise it shouldnt be a problem, but if your competing for your voice to be louder than your background then it could be annoying for both you and the viewer. However if you had a field trip lecture where you were at a lab that had lots of noisy equipment then that sound adds to the experience vs silence of a lab and just your voice.
I was an instructor for a couple years at a job and having to talk over the noise was annoying to both me and my guests. So I changed it to showing them the environment which is loud so they could see the automation working, and then back in the conference room I was able to instruct and answer questions without having to yell to them.
There are also ways to take a recorded video and edit the audio to remove or add in audio, however its best to record it live if you can without having to do so much editing to clean it up when possible.