Why Doesn't This Batch File Work?

[gmgdr11]

Hi everyone,

I'm having problems with the final step of my batch file and I can't figure out why it doesn't work.

The final step is to remove the larger directories and to keep the smallest. The code is as follows:

Code: [Select]

@echo off
setlocal EnableDelayedExpansion

rem Get size of all folders
set smallestSize=9999999999
for /D %%a in (*) do (
   set size=0
   for %%b in (%%a\*.*) do set /A size+=%%~Zb
   if !size! lss !smallestSize! (
      set smallestSize=!size!
      set smallestName=%%a
   )
)

echo Deleted folder: "%smallestName%"
pause

rem Delete all folders, except the smallest one
for /D %%a in (*) do (
   if "%%a" neq "%smallestName%" rmdir /S /Q "%%a"
It properly identifies the smallest directory, but after the point where it displays the smallest and pauses awaiting a key press, it fails to remove the larger directories.

Any help would be greatly appreciated.

Thanks.

[Lemonilla]

You forgot to close the last for loop with a ).

rem Delete all folders, except the smallest one
for /D %%a in (*) do (
   if "%%a" neq "%smallestName%" rmdir /S /Q "%%a"
)

[gmgdr11]

^^Yes that was so obvious--thank you!

Now that this works, I have another issue that I would like to correct with this batch program.

I have the following directory structure where this batch file will be executed:

Code: [Select]

DIR1
   DIRA
   DIRB
   DIRC
DIR2
   DIRD
   DIRE
   DIRF
   DIRG
DIR3
   DIRH
   DIRI
When run, it deletes DIR1 and DIR3 and only keeps DIR2 because it is the largest directory. What I'd like the result to be is as follows:

Code: [Select]

DIR1
   DIRC
DIR2
   DIRE
DIR3
   DIRH[code]

In other words, I want to keep all folders in the root directory where the batch file located and only keep the smallest directory within each of the subdirectories.

Is this possible? If not, then what I'd have to do is copy the batch file into each of the main root folders (DIR1, DIR2, DIR3) and run it each time in each of those folders.

Any further (and final) help would be very helpful.

[Lemonilla]

Look at the for /r command, I believe this is what you are looking for.  I'm not very well versed in it's uses, so I can't write you one off the top of my head.

[gmgdr11]

Look at the for /r command, I believe this is what you are looking for.  I'm not very well versed in it's uses, so I can't write you one off the top of my head.

It's ok, I appreciate your help.

Maybe someone else knows and can help me out with this one.

[Lemonilla]

If no one else replies when I get back on tomorrow I'll start running some tests and see if I can get it for you.  Good luck though.

[foxidrive]

Batch math fails above a number around 2 GB.

An accurate description of the file size limits would help to provide something that works.



This question has been multiposted:

http://stackoverflow.com/questions/26695993/final-step-of-batch-file-doesnt-work-why
http://stackoverflow.com/questions/26689398/batch-file-commands-to-keep-smallest-directory-and-delete-all-others

[gmgdr11]

Batch math fails above a number around 2 GB.

An accurate description of the file size limits would help to provide something that works.



This question has been multiposted:

http://stackoverflow.com/questions/26695993/final-step-of-batch-file-doesnt-work-why
http://stackoverflow.com/questions/26689398/batch-file-commands-to-keep-smallest-directory-and-delete-all-others

Yes, I had posted this question elsewhere. The original problem I had has been resolved. The only thing I need to figure out is what I asked in post #3 of this thread.

The sizes of the directories are not over 2GB. Comparing the sizes and deleting the smallest directories (DIRA, DIRB, DIRC, etc.) is what I am trying to achieve. Comparing the sizes and deleting the smallest directories (DIR1, DIR2, DIR3, etc.) which are in the same folder as the batch file is NOT what I want to achieve but that is what the batch file does now in the way that it's written.

[foxidrive]

Remove the echo keyword after testing - at present it will echo the RD commands to the console.

Code: [Select]

    @echo off
     :: keeps only the smallest folder, deletes all the largest folders, in all the next level folders below this folder
    setlocal EnableDelayedExpansion
    for /d %%a in (*) do (
      (for /d %%b in ("%%a\*") do (
         for /f "tokens=3" %%c in ('dir "%%b" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
         echo !size:~-15! "%%b"
        )
       )>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do echo rd /s /q "%%~e"
    del "%temp%\checksize?.bin"
    )
    pause


[gmgdr11]

Remove the echo keyword after testing - at present it will echo the RD commands to the console.

Code: [Select]

    @echo off
     :: keeps only the smallest folder, deletes all the largest folders, in all the next level folders below this folder
    setlocal EnableDelayedExpansion
    for /d %%a in (*) do (
      (for /d %%b in ("%%a\*") do (
         for /f "tokens=3" %%c in ('dir "%%b" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
         echo !size:~-15! "%%b"
        )
       )>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do echo rd /s /q "%%~e"
    del "%temp%\checksize?.bin"
    )
    pause


Created a new batch file with your code. When executed, the result is line after line of this:

Code: [Select]

'find' is not recognized as an internal or external command,
operable program or batch file.

[foxidrive]

Your Windows seems broken for some reason.

A likely reason is when someone uses the PATH variable for some other use, or the path variable has been edited in the Windows GUI and parts removed.

[Salmon Trout]

Created a new batch file with your code. When executed, the result is line after line of this:

Code: [Select]

'find' is not recognized as an internal or external command,
operable program or batch file.

Do you have a variable called %path% in your script?

[gmgdr11]

Do you have a variable called %path% in your script?

Well no. What I did to test that and generate that error was to copy the code in the post into a blank Notepad window, save it as "test.bat" in root folder of DIR1, DIR2, DIR3 and executed.

I apologize for my very basic knowledge of batch file writing. Programming, even at the most basic level, is something I am not very familiar with.

[foxidrive]

Well no. What I did to test that and generate that error was to copy the code in the post into a blank Notepad window, save it as "test.bat" in root folder of DIR1, DIR2, DIR3 and executed.

Open a cmd window and type find /? and sort /?

See if they both give you help.

[gmgdr11]

Open a cmd window and type find /? and sort /?

See if they both give you help.

Ok! Fixed it. The issue was that I had the PATH environment variable set to C:\Cygwin64\.

It displays the correct folders to be deleted. I get the following:

Code: [Select]

rd /s /q "DIR1\DIRA"
rd /s /q "DIR1\DIRC"
rd /s /q "DIR2\DIRG"
rd /s /q "DIR2\DIRF"
rd /s /q "DIR2\DIRE"
rd /s /q "DIR3\DIRH"
How does the batch file need to be modified then to execute those commands rather than display them now that it is confirmed that it accurately finds the proper directories to delete?

Thanks.

[Lemonilla]

Remove echo from 'echo rd /s /q "%%~e"'

[gmgdr11]

Remove echo from 'echo rd /s /q "%%~e"'

Oh duh!!!

Thank you guys for all of your help. I am very appreciative!

[foxidrive]

It displays the correct folders to be deleted.

How does the batch file need to be modified then to execute those commands rather than display them now that it is confirmed that it accurately finds the proper directories to delete?

Good to hear.

I could tell you how to modify it, as it's very simple, but I already told you in the post with the code. 

EDIT: Too late, Lemonilla blabbed

(thanks).

[Lemonilla]

Good to hear.

I could tell you how to modify it, as it's very simple, but I already told you in the post with the code. 

EDIT: Too late, Lemonilla blabbed

(thanks).

It's easy to overlook those things.  I've done it quite often enough, as you well know.

[gmgdr11]

Hello once again.

I guess I have just one more question (but it's a quick one).

Will this batch file work in a temp directory with 30,000 or so directories? In other words, keeping the same directory structure in the temp directory it looks something like this:

Code: [Select]

DIR1
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
DIR2
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
...
DIR33359
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
I ask only because this is a huge project and it is very difficult and time consuming to check if in every case, for all 33,359 directories, it is correctly selecting the smallest folder to keep.

Also, is there a directory size limit that, when exceeded, causes the calculation and selection of smallest directory to fail? The largest directory (with 9 directories) is approximately 198,001,310 bytes.

[foxidrive]

What it will not do is tell you if more than one directory has exactly the same size.

Will this batch file work in a temp directory with 30,000 or so directories? In other words, keeping the same directory structure in the temp directory it looks something like this:

The flaw in your task description that you aren't aware of, is that by not giving exact details of the folder names and range of characters sets being used, and the files and file attributes, nobody can tell you with certainty if it will work in your situation.

It should work with 30,000 folders in a directory - but you are probably going to get more than one folder with the same filesize, and that could be the smallest filesize.

[gmgdr11]

What it will not do is tell you if more than one directory has exactly the same size.

The flaw in your task description that you aren't aware of, is that by not giving exact details of the folder names and range of characters sets being used, and the files and file attributes, nobody can tell you with certainty if it will work in your situation.

It should work with 30,000 folders in a directory - but you are probably going to get more than one folder with the same filesize, and that could be the smallest filesize.

Ok, let me explain the entire task in detail and see if perhaps there is a better way of doing all of this:

I have a directory with 33,359 RAW files. These range from the smallest (34 bytes) to the largest (7,508,609 bytes).

An executable program that I have has 9 possible compression options to process one of these RAW files. The options are:

Code: [Select]

c1
c2
c3
c4
c5
c6
c7
c8
c9
When any one of those options is used with a RAW file, 2 output files are created.

Code: [Select]

00001.rif
00001.hup
The command-line syntax would be:

Code: [Select]

rawcomp c1 00001.raw 00001.rif 00001.hup
In every case, the result is different--That is, for 00001.raw, c9 produces the smallest set of files (rif + hup). But for 00002.raw, c5 is the optimal option.

So what I have been doing is creating a set of directories for each RAW file and subdirectories for each rawcomp option (c1-c9):

Code: [Select]

00001
   1
   2
...
  9
00002
  1
  2
...
  9
The following code is used for this:

Code: [Select]

FOR %%b IN (temp\*.raw) DO mkdir temp\%%~nb
FOR /D %%a IN (temp\*) DO mkdir %%a\1,%%a\2,%%a\3,%%a\4,%%a\5,%%a\6,%%a\7,%%a\8,%%a\9
Then once all directories are created, the following is executed next on the batch file:

Code: [Select]

FOR %%l IN (1 2 3 4 5 6 7 8 9) DO FOR %%a IN (temp\*.raw) DO rawcomp.exe c%%l %%a temp\%%~na\%%l\%%l.hup temp\%%~na\%%l\%%l.rif
After that process is completed, then the code I was so generously provided here is executed with a separate batch file.

So there it is...This is the entire process. If needed, I can provide every individual filesize for every individual RAW file.

So is the way I am attempting to go about this the most ideal/efficient/optimal? Is Should the code provided here work perfectly in this situation?

Any and all help will be greatly appreciated as usual guys!

EDIT: One quick important note: Every option (c1-c9) are different compression levels meaning that the result is always smaller than the original RAW. So in the case of the 34 byte RAW file, all 9 files will be smaller. So when the step comes to identify the smallest directory and delete the others, it will be dealing with extremely small directories/files.

[foxidrive]

Hmmm.  It's a fairly trivial script to launch the rawcomp on each file, with all 9 compression ratios, and keep the smallest files, with that filesize range.

All 33,359 RAW files are in a single folder, right?

Where do you want the smallest set of rip-and-huf files stored?
Do you want them stored in an archive file with zero compression, just for a crc envelope?

[gmgdr11]

Hmmm.  It's a fairly trivial script to launch the rawcomp on each file, with all 9 compression ratios, and keep the smallest files, with that filesize range.

All 33,359 RAW files are in a single folder, right?

Where do you want the smallest set of rip-and-huf files stored?

Yes all RAW files are in one directory.

You mentioned the filesize range: Let me clarify that in other cases with a different collection of RAW files, the filesize range may be different. I don't know if that is important when writing the batch script or not but I thought I'd mention it just in case.

What I'd like is for the rip and huf files to be named with the compression number used in a subdirectory with the RAW file's name. Like this:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.huf
Thank you for your help.

[foxidrive]

Let me clarify that in other cases with other RAW files, the filesize range may be different.

You mentioned that the largest raw filesize is less than 8 megabytes, is that no longer the case?

What I'd like is for the rip and huf files to be named with the compression number used in a subdirectory with the RAW file's name. Like this:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.huf

Just asking here if the filenames could be something like this.  Or is a folder the best solution for your needs?

Code: [Select]

00313-2.rip
00313-2.huf
01981-8.rip
01981-8.huf
They could be stored in a ZIP file with zero compression too, just for a CRC envelope and to keep the files together.

[gmgdr11]

You mentioned that the largest raw filesize is less than 8 megabytes, is that no longer the case?

For this project, that is correct. But in future projects, with a completely different collectin of RAW files, the filesize range may be from 11 bytes to 4,991,844 bytes (for example).

Just asking here if the filenames could be something like this.  Or is a folder the best solution for your needs?

The only reason I am opposed to the idea of:

Code: [Select]

00313-2.rip
00313-2.huf
01981-8.rip
01981-8.hufis because of the future restoration batch script that would restore all original RAW files with a separate program (rawres.exe). For rawres.exe, the compression level needs to be inputted in the command-line syntax for it to properly restore the RAW from the .huf and .rip files. So I would prefer to have it as:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.hufif possible.

They could be stored in a ZIP file with zero compression too, just for a CRC envelope and to keep the files together.

The reason I do not want to use any ZIP files is because this project is for maximum compression levels--Keeping the files separate and untouched is the absolute best and only way to achieve the goal needed with this project.

Thanks.

EDIT: One more thing to note is that all of the RAW files are in a directory named "temp". The created directories that will have a .huf and .rip file in them should also be inside of "temp". The batch file is to remain outside of the "temp" folder but do all of its work with files inside of "temp".

[foxidrive]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


[foxidrive]

For this project, that is correct. But in future projects, with a completely different collectin of RAW files, the filesize range may be from 11 bytes to 4,991,844 bytes (for example).

Ok, the file size isn't an issue with the method I used.

If there are two (or more) sets of files (from one raw file) that have exactly the same size, then it will keep the files from the lowest compression ratio.

The reason I do not want to use any ZIP files is because this project is for maximum compression levels--Keeping the files separate and untouched is the absolute best and only way to achieve the goal needed with this project.

You misunderstand.  ZIP and other archivers have a zero compression level and this could have been used to store the smallest files from rawcomp in an archive file, exactly as they are, in a very quick process.

This also protects the file data with a CRC calculation - so in the future when the files are extracted, the integrity of the files can be determined by the correct CRC/errorlevel test.

[gmgdr11]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


Thank you so very much. It works perfectly well.

I so much appreciate everyone's help through this process to figuring out the best way to attack this. In retrospect, explaining the entire process from the get-go probably would've made this go much faster (my mistake!).

Anyway, thanks again!

[gmgdr11]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


Hello once again everyone.

I have one final (I promise) problem that I need help solving. I thought I could figure it out on my own, but no luck!

The batch script that you wrote for me works perfectly, but now I need to reverse the process.

The rawcomp syntax for decompression (i.e. reversing the original process) is:

Code: [Select]

rawcomp d* *.hup *.rif *.raw
What I would like is for the RAW files be created in teh base (temp\) directory--not in the subdirectories where the RIF and HUP files are. So in the following case:

Code: [Select]

temp\000A\6.hup
temp\000A\6.rif
I want the RAW file to be created like this:

Code: [Select]

temp\000A.raw
And then, after rawcomp completes with that particular set RIF/HUP files and creates the RAW file, delete that subdirectory like this:

Code: [Select]

rmdir /s /q temp\000A
So all that would remain is a file called 000A.raw in temp\.

Also, in this script, I'd like not to have to place the script and rawcomp in the temp folder for it to work. In the script that you so kindly wrote for me, the batch file and rawcomp have to be copied to temp\ and then deleted once the batch file completes.

Again, thanks everyone for your help up to this point and please take my word taht will be the last problem I need solved on this project.

[foxidrive]

In the script that you so kindly wrote for me, the batch file and rawcomp have to be copied to temp\

Not at all.  The batch file can be launched from anywhere.

Change rawcomp to "c:\bin\rawcomp.exe" or where you have it located and then it will work from anywhere.



The folowing should solve the decompression - test it on sample-folders-and-files that are in a different folder.  Change the c:\temp

BTW, is d* the correct syntax for decompression?


EDITED: to handle the dNUMBER compression value.  Be sure to change c:\temp everywhere to the folder you are using.

Code: [Select]

@echo off
pushd "c:\temp" || (echo aborting - the folder doesn't exist & pause & goto :EOF)
if exist *.huf (echo Why are there *.huf files in the main folder? & pause & goto :EOF)
for /f "delims=" %%a in ('dir *.huf /b /s /a-d ') do (
   echo Working with "%%a"
      for %%b in ("%%~dpa\.") do (
          if exist  "%%~dpa\%%~na.rif" ("c:\bin\rawcomp.exe" d%%~nb "%%a" "%%~dpa\%%~na.rif" "%%~nb.raw" & rd /s /q "%%~dpa")
      )
)
popd
pause


[gmgdr11]

Not at all.  The batch file can be launched from anywhere.

Change rawcomp to "c:\bin\rawcomp.exe" or where you have it located and then it will work from anywhere.



The folowing should solve the decompression - test it on sample-folders-and-files that are in a different folder.  Change the c:\temp

BTW, is d* the correct syntax for decompression?

Code: [Select]

@echo off
pushd "c:\temp"
if exist *.huf echo Why are there *.huf files in the main folder? & pause & goto :EOF
for /f "delims=" %%a in ('dir *.huf /b /s /a-d ') do (
   echo Working with "%%a"
   if exist  "%%~dpa\%%~na.rif"   "c:\bin\rawcomp.exe" d* "%%a" "%%~dpa\%%~na.rif" "%%~na.raw" & rd /s /q "%%~dpa"
)
popd
pause


No, by d* I meant that the * is the filename of the rif/hup file. So in the following example:

Code: [Select]

8.hup
the syntax would be

Code: [Select]

rawcomp d8 ...
I apologize for not making that part clear.

Also, thanks for the note about the original script.

[foxidrive]

See my edited post above.  Test it as shown and see how it goes.

If you use cygwin then the DLL will need to be on the path, I think.

[gmgdr11]

See my edited post above.  Test it as shown and see how it goes.

If you use cygwin then the DLL will need to be on the path, I think.

Works great!

Thank so very much for all your help.