Well, for starters- Part of the problem here is that the text that TREE outputs is not unicode. It's ASCII text that uses extended code page 437.
Code pages indicate what the extended ASCII set contains (eg characters from 128 upwards). You may be familiar with the smiley faces and blocks and shaded block characters on older MS-DOS systems, in addition to the line drawing characters- that was in Code page 437 as well.
UTF-8 and Unicode in general wasn't built upon that standard extended ASCII character set. Those characters were kind of useless and so it was built on top of a standardized international code page, ANSI Latin 1, I think.
in Code page 437, ASCII code 162 was └. But, in UTF-8, code 162 is À. You might see the problem. The text you output is being interpreted as UTF-8, rather than code-page 437 ASCII. But you cannot realistically fix that.
Notepad works because it seems that it detects the input text when you paste it. It converts └ to Unicode character E2 94 94 for example. That's why it warns you about loss of information if you try to save as ANSI, since it's no longer the same input ASCII text.
Anyway, onto finding a solution.
The easiest workaround is to simply use the /A switch on TREE so that it uses standard ASCII characters that are part of UTF-8.
If you really must have the special line-drawing characters, you need to figure out how to effectively force the Code page 437 characters to instead be their unicode equivalents. Basically you need to automatically do the conversion notepad does.
One way that seems to work is simply running it under powershell. if you run the command through powershell, the output appears to be proper unicode in the file:
powershell -command "tree > treeout.txt"