There are so many scenarios.
Yes, it could be possible. No, not everyone can do that without big efforts and a lot of time - but it depends, it could be as easy as reading your neighbor number on his door, or it could be as hard as trying to figure who sent a letter by just reading the letter (no addresses on the envelope).
(Spoiler said it right).