Computer Hope
Software => Computer software => Topic started by: sagra on February 29, 2012, 11:11:40 AM
-
Can someone please help me solve this. i need to find what files are on one drive but are not on another drive i am comparing it with. thanks in advance.
-
PS - I cannot use syncing software because the drives do not have the same folders or trees.
-
Welcome!
Computer Hope is the number one location for free computer help.
The forum will help everyone with all computer questions.
You can use features built into the OS or you can use additional programs that compare two almost identical folders.
For windows users there is a program called 'Sync Toy' and can be downloaded from the Microsoft site.
http://www.microsoft.com/download/en/details.aspx?id=15155
The purpose of the program is to resolve then differences between the two folders or volumes. You have the options o looking at the proposed changes beforehand.
Is that what you want?
EDIT: I missed your second post. So you want to compare files that are not in the same directory structure? By name only? Sorry, That is too hard for me.
-
Allan recommended some software for that a while back: http://www.computerhope.com/forum/index.php/topic,128500.0.html
-
Raptor - thank you - actually that is also a sync program. It will only work if the folders are the same on both drives. It should be a simple application, right?...a program that will find files that are on one, but not on another drive?
-
If you don't mind using Linux, this might do: http://linuxcommando.blogspot.com/2008/05/compare-directories-using-diff-in-linux.html
-
Hi Raptor, thanks. I am not sure what Unix is or how to use it, but I am ready to try. Where can I get some help or tutorials on how to use it?
-
Hi Raptor, thanks. I am not sure what Unix is or how to use it, but I am ready to try. Where can I get some help or tutorials on how to use it?
Well, you can run Linux off a bootable CD without having to install it by making a Live CD: http://www.ubuntu.com/download/ubuntu/download
It won't make any changes your hard drives unless you specifically tell it to do so.
Once you're in Ubuntu, you can press ctrl + alt + d to bring up a terminal and try out the instructions on that blog.
It's a pretty complex operating system, though.
-
So you strictly want to go off of file name for the compare.
If it Exists on E: but doesn't exist on F: you want to know that it doesn't exist on F:
Should be able to do this with a batch file or vbscript. I will round up the troops.
-
sagra - let's clarify a few things. Do you want to examine every single file on one drive, and see if a file with the same name (size?) (contents?) exists on a different drive? Is the first drive a system drive? Could you perhaps say why you need to do this?
-
I will round up the troops.
Rolls eyes. Way to "help" when you don't know how to help. Can't the "troops" read the threads and decide for themselves which ones to enter? You felt entitled to PM me to rope me in to this, yet you blocked the PM I sent in reply? Please don't do this again.
-
Rolls eyes. Way to "help" when you don't know how to help. Can't the "troops" read the threads and decide for themselves which ones to enter? You felt entitled to PM me to rope me in to this, yet you blocked the PM I sent in reply? Please don't do this again.
Sorry about the blocked PM. I had set my PM settings to Administrators only because I was getting inundated with PM's from all kinds of newbies who were just sending me messages to piss me off. I kept ignoring them and making settings in my Profile to block them but they kept coming back. So I set my profile to Administrators only. The forum Admin got back to me about blocking this person for good and i just forgot to set my profile back to allow PM's.
Your post in this thread is very offensive if you don't mind me saying. I am asking politely for you to edit or remove it.
I am a Trusted Advisor on the TechGuy forums and one of my duties is to alert other experts on the forum to posts I feel they can answer. I just send them a PM and let them know they may be able to help with this topic. I don't post in the thread that I am sending so and so over to help them.
I wasn't calling you out in the thread or saying "Hey I will tell Salmon Trout he has to write this script for you."
I wasn't going to start writing the batch file until he got back to me on my question. I don't like starting a script until I know all the facts.
I was just alerting you to the thread because I know you are good with VBscript and you would possibly have a better solution with Vbscript.
-
I am a Trusted Advisor on the TechGuy forums and one of my duties is to alert other experts on the forum to posts I feel they can answer.
So you're like the bell boy!
-
Hi Squashman and Salmon Trout - thanks for your kind help. Here are more specifics. The drives would both be large external USB drives. I would at least like to search by name (it would be even more ideal to be able to search by name, size, date....but name and size would be sufficient). I have many drives with various media files (many many thousands) on them. I am trying to consolidate the files on to one 3TB drive. I simply need to be able to do a search to see what files are on Drive E for example, but that are not on Drive F. Then I want to be able to copy those files on Drive E, as a group, onto Drive F. That will be a step in the process of consolidating my 500,000+ media files. The Unix route sounds complex, but I will try anything. I just need detailed instructions.Thanks again.
-
Gee, are we giving up on this? This seems like it ought to be simple. It's simply like the opposite of a duplicate finder application.
-
This problem can be solved. Either with batch files or other tools.
What tools does the OP have and use? (No special order)
Perl
Java
Python
VbScript
C++
Excel
Visual Basic
..and others.
All or any of these could be used to control some batch files and build a database and find duplications in the database.
But what about file size? Date?
And once the duplications are found, what kind of report should be generated?
Not knowing what the OP needs to do, my typical response is use more hardware. Hardware is cheaper tan working by the hour. This poblem would take me at lest three hours on a good day. If I pay myself $41 per hour, it comes to over $120. For that amount of money I buy a portable 1TB external drive at the local Walt-Mart.
Dear OP, do you have less that one tetra byre of data? Are you interested in the hardware solution?
-
Gee, are we giving up on this? This seems like it ought to be simple. It's simply like the opposite of a duplicate finder application.
If it was all that simple you would have coded it yourself in the more than 1 month since you first asked. You are asking for a script to be written, and this means asking someone to do work for you for free, very often for no thanks or a lot of abuse, or several pages of "it doesn't work" type posts. Having said that, this simple script will search all folders and sub folders on a drive for files, and for each file, check if a file of that name exists on another drive, and if it does not, write the whole path to a report file.
@echo off
setlocal enabledelayedexpansion
set Drive1=D
set Drive2=F
set FileMask=*.avi
set ShowNamesOnScreen=yes
set reportfile=D:\Report.txt
if exist "%reportfile%" del "%reportfile%"
Echo Looking for files which are on drive %Drive1% but not on drive %Drive2%
for /f "delims=" %%A in ('dir /b /s /a-d /tc %Drive1%:\%FileMask%') do (
set fname=%%~nxA
for /f "delims=" %%B in ('dir /b /s "%drive2%:\!fname!" 2^>^&1 ^>nul') do (
if "%%B" == "File Not Found" (
ECHO %%~dpnxA >> "%reportfile%"
if "%ShowNamesOnScreen%"=="yes" echo %%~dpnxA
)
)
)
Echo Finished searching drive %drive1%
Pause
This script will read the report file and either copy or move (edit set filemode=) the files to a destination folder. In both scripts, you will need to do some editing.
@echo off
setlocal enabledelayedexpansion
set filemode=Copy
set DestFolder=F:\FileFolder
set reportfile=D:\Report.txt
for /f "delims=" %%A in ('type "%reportfile%"') do (
%filemode% "%%~dpnxA" "%Destfolder%"
)
Pause
-
Salmon Trout, that is great!
But is the OP going to give you anything in return for your work?
-
I probably misunderstood the question, but could xcopy be made to do this?
xcopy has the ability to not copy files, and instead list files that it would copy. it doesn't provide a way to say that it should only copy files that do not exist in the new location, but you can say the opposite, and tell it to copy only files that already exist in the destination.
Then, you can use the /EXCLUDE switch to exclude those files, and just list the files that would be copied.
xcopy /s /l /u E:\ F:\ > existingfiles.txt
xcopy /s /l E:\ F:\ /EXCLUDE:existingfiles.txt >> unique.txt
Assuming I thought it out properly, the result should be that the files listed the second time will be all the files, excluding the files that exist in both places.
Note that this will only find files on the source drive that do not exist on the target drive, and not files that exist on the target but not on the source.
You could do both and merge the results, though:
xcopy /s /l /u E:\ F:\ > existingfiles.txt
xcopy /s /l E:\ F:\ /EXCLUDE:existingfiles.txt > unique.txt
xcopy /s /l /u F:\ E:\ > existingfiles2.txt
xcopy /s /l /EXCLUDE:existingfiles2.txt > unique2.txt
copy unique.txt+unique2.txt results.txt
Don't have access to Windows at the moment so not able to properly test my assumptions.
-
That looks good, BC. There is always another way to kin a cat.
Yea, I can't try it either. I am on Ubuntu on a USB flash.
Earlier I mentioned getting a Terra byte eternal drive. Thee is an option in
MS SyncToy (http://en.wikipedia.org/wiki/SyncToy) to contribute from a USB to a large hard drive, Or any two drives,. You would just keep contributing from each USB drive until you had done them all.
But there is a catch. You have to move the files out of the directories first.
By default, to identical files in different directories are not the same file. I understand the OP wants to change that rule and find duplicates even in other directories.
I think that rather than copying or moving files, one would make shortcuts to all files on all USB sticks and then just parse the shortcuts looking for duplicates and build a database(s). Visual Basic could do it. Maybe with a SOL server. I thought he said he has 50,000 MP3 files.
Still, the choice to use batch was that of the OP. He must enjoy the work. Or he has too much spare time on his hands.
-
I understand the OP wants to change that rule and find duplicates even in other directories.
Oh... well in that case the XCOPY method will not work.