find files on 1 drive that are not on another

[Geek-9pm]

This problem can be solved. Either with batch files or other tools.
What tools does the OP have and use? (No special order)
Perl
Java
Python
VbScript
C++
Excel
Visual Basic
..and others.
All or any of these could be used to control some batch files and build a database and find duplications in the database.
But what about file size? Date?
And once the duplications are found, what kind of report should be generated?

Not knowing what the OP needs to do, my typical response is use more hardware. Hardware is cheaper tan working by the hour. This  poblem would take me at lest three hours on a good day. If I pay myself $41 per hour, it comes to over $120. For that amount of money I buy a portable 1TB external drive at the local Walt-Mart.

Dear OP, do you have less that one  tetra byre of data? Are you interested in the hardware solution?

[Salmon Trout]

Gee, are we giving up on this? This seems like it ought to be simple. It's simply like the opposite of a duplicate finder application.

If it was all that simple you would have coded it yourself in the more than 1 month since you first asked. You are asking for a script to be written, and this means asking someone to do work for you for free, very often for no thanks or a lot of abuse, or several pages of "it doesn't work" type posts. Having said that, this simple script will search all folders and sub folders on a drive for files, and for each file, check if a file of that name exists on another drive, and if it does not, write the whole path to a report file.

Code: [Select]

@echo off
setlocal enabledelayedexpansion
set Drive1=D
set Drive2=F
set FileMask=*.avi
set ShowNamesOnScreen=yes
set reportfile=D:\Report.txt
if exist "%reportfile%" del "%reportfile%"
Echo Looking for files which are on drive %Drive1% but not on drive %Drive2%
for /f "delims=" %%A in ('dir /b /s /a-d /tc %Drive1%:\%FileMask%') do (
set fname=%%~nxA
for /f "delims=" %%B in ('dir /b /s "%drive2%:\!fname!" 2^>^&1 ^>nul') do (
if "%%B" == "File Not Found" (
ECHO %%~dpnxA >> "%reportfile%"
if "%ShowNamesOnScreen%"=="yes" echo %%~dpnxA
)
)
)
Echo Finished searching drive %drive1%
Pause

This script will read the report file and either copy or move (edit set filemode=) the files to a destination folder. In both scripts, you will need to do some editing.

Code: [Select]

@echo off
setlocal enabledelayedexpansion
set filemode=Copy
set DestFolder=F:\FileFolder
set reportfile=D:\Report.txt
for /f "delims=" %%A in ('type "%reportfile%"') do (
%filemode% "%%~dpnxA" "%Destfolder%"
)
Pause


[Geek-9pm]

Salmon Trout, that is great!
But is the OP going to give you anything in return for your work?

[BC_Programmer]

I probably misunderstood the question, but could xcopy be made to do this?

xcopy has the ability to not copy files, and instead list files that it would copy. it doesn't provide a way to say that it should only copy files that do not exist in the new location, but you can say the opposite, and tell it to copy only files that already exist in the destination.

Then, you can use the /EXCLUDE switch to exclude those files, and just list the files that would be copied.

Code: [Select]

xcopy /s /l /u E:\ F:\ > existingfiles.txt
xcopy /s /l E:\ F:\ /EXCLUDE:existingfiles.txt >> unique.txt

Assuming I thought it out properly, the result should be that the files listed the second time will be all the files, excluding the files that exist in both places.

Note that this will only find files on the source drive that do not exist on the target drive, and not files that exist on the target but not on the source.

You could do both and merge the results, though:

Code: [Select]

xcopy /s /l /u E:\ F:\ > existingfiles.txt
xcopy /s /l E:\ F:\ /EXCLUDE:existingfiles.txt > unique.txt

xcopy /s /l /u F:\ E:\ > existingfiles2.txt
xcopy /s /l /EXCLUDE:existingfiles2.txt > unique2.txt

copy unique.txt+unique2.txt results.txt

Don't have access to Windows at the moment so not able to properly test my assumptions.

[Geek-9pm]

That looks good, BC. There is always another way to kin a cat.
Yea, I can't try it either. I am on Ubuntu on a USB  flash.

Earlier I mentioned getting a Terra byte eternal drive. Thee is an option in
MS SyncToy to contribute from a USB to a large hard drive, Or any two drives,. You would just keep contributing from each USB drive until you had done them all.

But there is a catch. You have to move the files out of the directories first.

By default, to identical files in different directories are not the same file. I understand the OP wants to change that rule and find duplicates even in other directories.

I think that rather than copying or  moving files, one would make shortcuts to all files on all USB sticks and then just parse the shortcuts looking for duplicates and build a database(s).  Visual Basic could do it. Maybe with a SOL server. I thought he said he has 50,000 MP3 files.

Still, the choice to use batch was that of the OP.  He must enjoy the work. Or he has too much spare time on his hands.

[BC_Programmer]

I understand the OP wants to change that rule and find duplicates even in other directories.

Oh... well in that case the XCOPY method will not work.