Why Doesn't This Batch File Work?

[Lemonilla]

Remove echo from 'echo rd /s /q "%%~e"'

[gmgdr11]

Remove echo from 'echo rd /s /q "%%~e"'

Oh duh!!!

Thank you guys for all of your help. I am very appreciative!

[foxidrive]

It displays the correct folders to be deleted.

How does the batch file need to be modified then to execute those commands rather than display them now that it is confirmed that it accurately finds the proper directories to delete?

Good to hear.

I could tell you how to modify it, as it's very simple, but I already told you in the post with the code. 

EDIT: Too late, Lemonilla blabbed

(thanks).

[Lemonilla]

Good to hear.

I could tell you how to modify it, as it's very simple, but I already told you in the post with the code. 

EDIT: Too late, Lemonilla blabbed

(thanks).

It's easy to overlook those things.  I've done it quite often enough, as you well know.

[gmgdr11]

Hello once again.

I guess I have just one more question (but it's a quick one).

Will this batch file work in a temp directory with 30,000 or so directories? In other words, keeping the same directory structure in the temp directory it looks something like this:

Code: [Select]

DIR1
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
DIR2
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
...
DIR33359
   DIRA
   DIRB
   DIRC
   DIRD
   DIRE
   DIRF
   DIRG
   DIRH
   DIRI
I ask only because this is a huge project and it is very difficult and time consuming to check if in every case, for all 33,359 directories, it is correctly selecting the smallest folder to keep.

Also, is there a directory size limit that, when exceeded, causes the calculation and selection of smallest directory to fail? The largest directory (with 9 directories) is approximately 198,001,310 bytes.

[foxidrive]

What it will not do is tell you if more than one directory has exactly the same size.

Will this batch file work in a temp directory with 30,000 or so directories? In other words, keeping the same directory structure in the temp directory it looks something like this:

The flaw in your task description that you aren't aware of, is that by not giving exact details of the folder names and range of characters sets being used, and the files and file attributes, nobody can tell you with certainty if it will work in your situation.

It should work with 30,000 folders in a directory - but you are probably going to get more than one folder with the same filesize, and that could be the smallest filesize.

[gmgdr11]

What it will not do is tell you if more than one directory has exactly the same size.

The flaw in your task description that you aren't aware of, is that by not giving exact details of the folder names and range of characters sets being used, and the files and file attributes, nobody can tell you with certainty if it will work in your situation.

It should work with 30,000 folders in a directory - but you are probably going to get more than one folder with the same filesize, and that could be the smallest filesize.

Ok, let me explain the entire task in detail and see if perhaps there is a better way of doing all of this:

I have a directory with 33,359 RAW files. These range from the smallest (34 bytes) to the largest (7,508,609 bytes).

An executable program that I have has 9 possible compression options to process one of these RAW files. The options are:

Code: [Select]

c1
c2
c3
c4
c5
c6
c7
c8
c9
When any one of those options is used with a RAW file, 2 output files are created.

Code: [Select]

00001.rif
00001.hup
The command-line syntax would be:

Code: [Select]

rawcomp c1 00001.raw 00001.rif 00001.hup
In every case, the result is different--That is, for 00001.raw, c9 produces the smallest set of files (rif + hup). But for 00002.raw, c5 is the optimal option.

So what I have been doing is creating a set of directories for each RAW file and subdirectories for each rawcomp option (c1-c9):

Code: [Select]

00001
   1
   2
...
  9
00002
  1
  2
...
  9
The following code is used for this:

Code: [Select]

FOR %%b IN (temp\*.raw) DO mkdir temp\%%~nb
FOR /D %%a IN (temp\*) DO mkdir %%a\1,%%a\2,%%a\3,%%a\4,%%a\5,%%a\6,%%a\7,%%a\8,%%a\9
Then once all directories are created, the following is executed next on the batch file:

Code: [Select]

FOR %%l IN (1 2 3 4 5 6 7 8 9) DO FOR %%a IN (temp\*.raw) DO rawcomp.exe c%%l %%a temp\%%~na\%%l\%%l.hup temp\%%~na\%%l\%%l.rif
After that process is completed, then the code I was so generously provided here is executed with a separate batch file.

So there it is...This is the entire process. If needed, I can provide every individual filesize for every individual RAW file.

So is the way I am attempting to go about this the most ideal/efficient/optimal? Is Should the code provided here work perfectly in this situation?

Any and all help will be greatly appreciated as usual guys!

EDIT: One quick important note: Every option (c1-c9) are different compression levels meaning that the result is always smaller than the original RAW. So in the case of the 34 byte RAW file, all 9 files will be smaller. So when the step comes to identify the smallest directory and delete the others, it will be dealing with extremely small directories/files.

[foxidrive]

Hmmm.  It's a fairly trivial script to launch the rawcomp on each file, with all 9 compression ratios, and keep the smallest files, with that filesize range.

All 33,359 RAW files are in a single folder, right?

Where do you want the smallest set of rip-and-huf files stored?
Do you want them stored in an archive file with zero compression, just for a crc envelope?

[gmgdr11]

Hmmm.  It's a fairly trivial script to launch the rawcomp on each file, with all 9 compression ratios, and keep the smallest files, with that filesize range.

All 33,359 RAW files are in a single folder, right?

Where do you want the smallest set of rip-and-huf files stored?

Yes all RAW files are in one directory.

You mentioned the filesize range: Let me clarify that in other cases with a different collection of RAW files, the filesize range may be different. I don't know if that is important when writing the batch script or not but I thought I'd mention it just in case.

What I'd like is for the rip and huf files to be named with the compression number used in a subdirectory with the RAW file's name. Like this:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.huf
Thank you for your help.

[foxidrive]

Let me clarify that in other cases with other RAW files, the filesize range may be different.

You mentioned that the largest raw filesize is less than 8 megabytes, is that no longer the case?

What I'd like is for the rip and huf files to be named with the compression number used in a subdirectory with the RAW file's name. Like this:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.huf

Just asking here if the filenames could be something like this.  Or is a folder the best solution for your needs?

Code: [Select]

00313-2.rip
00313-2.huf
01981-8.rip
01981-8.huf
They could be stored in a ZIP file with zero compression too, just for a CRC envelope and to keep the files together.

[gmgdr11]

You mentioned that the largest raw filesize is less than 8 megabytes, is that no longer the case?

For this project, that is correct. But in future projects, with a completely different collectin of RAW files, the filesize range may be from 11 bytes to 4,991,844 bytes (for example).

Just asking here if the filenames could be something like this.  Or is a folder the best solution for your needs?

The only reason I am opposed to the idea of:

Code: [Select]

00313-2.rip
00313-2.huf
01981-8.rip
01981-8.hufis because of the future restoration batch script that would restore all original RAW files with a separate program (rawres.exe). For rawres.exe, the compression level needs to be inputted in the command-line syntax for it to properly restore the RAW from the .huf and .rip files. So I would prefer to have it as:

Code: [Select]

00313\2.rip
00313\2.huf
01981\8.rip
01981\8.hufif possible.

They could be stored in a ZIP file with zero compression too, just for a CRC envelope and to keep the files together.

The reason I do not want to use any ZIP files is because this project is for maximum compression levels--Keeping the files separate and untouched is the absolute best and only way to achieve the goal needed with this project.

Thanks.

EDIT: One more thing to note is that all of the RAW files are in a directory named "temp". The created directories that will have a .huf and .rip file in them should also be inside of "temp". The batch file is to remain outside of the "temp" folder but do all of its work with files inside of "temp".

[foxidrive]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


[foxidrive]

For this project, that is correct. But in future projects, with a completely different collectin of RAW files, the filesize range may be from 11 bytes to 4,991,844 bytes (for example).

Ok, the file size isn't an issue with the method I used.

If there are two (or more) sets of files (from one raw file) that have exactly the same size, then it will keep the files from the lowest compression ratio.

The reason I do not want to use any ZIP files is because this project is for maximum compression levels--Keeping the files separate and untouched is the absolute best and only way to achieve the goal needed with this project.

You misunderstand.  ZIP and other archivers have a zero compression level and this could have been used to store the smallest files from rawcomp in an archive file, exactly as they are, in a very quick process.

This also protects the file data with a CRC calculation - so in the future when the files are extracted, the integrity of the files can be determined by the correct CRC/errorlevel test.

[gmgdr11]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


Thank you so very much. It works perfectly well.

I so much appreciate everyone's help through this process to figuring out the best way to attack this. In retrospect, explaining the entire process from the get-go probably would've made this go much faster (my mistake!).

Anyway, thanks again!

[gmgdr11]

Put some sample files in a folder, change "c:\temp" to that folder and test this: it creates a log file in c:\test also.

Code: [Select]

    @echo off
    :: This processes all *.raw files in c:\temp
    :: For each raw file it uses rawcomp to create 9 folders, one for each compression level
    :: It keeps the smallest folder contents and moves them into a folder with the same name as the raw file
    setlocal EnableDelayedExpansion
    pushd "c:\temp"
    for %%m in (*.raw) do (
         for /L %%o in (1,1,9) do (
            md "%%~nm\%%o" 2>nul
            rawcomp c%%o "%%m" "%%~nm\%%o\%%o.rif" "%%~nm\%%o\%%o.hup"
         )   
         ( for /d %%a in ("%%~nm\*") do (
            for /f "tokens=3" %%c in ('dir "%%a" /s /-c ^|find "File(s)" ') do set "size=000000000000000%%c"
            echo !size:~-15! "%%a"
           ))>"%temp%\checksize.bin"
    sort <"%temp%\checksize.bin" >"%temp%\checksize2.bin"
    >>"file.log" (type "%temp%\checksize.bin"&set /p var=<"%temp%\checksize2.bin"&echo keeping:    !var!&echo/)
    for /f "usebackq skip=1 tokens=1,*" %%d in ( "%temp%\checksize2.bin") do rd /s /q "%%~e"
    for /d %%a in ("%%~nm\*") do echo keeping compression level %%~na of file "%%~nm.raw" & move "%%a\*" "%%~nm" >nul & rd "%%a"
    del "%temp%\checksize?.bin"
    )
    popd
    pause


Hello once again everyone.

I have one final (I promise) problem that I need help solving. I thought I could figure it out on my own, but no luck!

The batch script that you wrote for me works perfectly, but now I need to reverse the process.

The rawcomp syntax for decompression (i.e. reversing the original process) is:

Code: [Select]

rawcomp d* *.hup *.rif *.raw
What I would like is for the RAW files be created in teh base (temp\) directory--not in the subdirectories where the RIF and HUP files are. So in the following case:

Code: [Select]

temp\000A\6.hup
temp\000A\6.rif
I want the RAW file to be created like this:

Code: [Select]

temp\000A.raw
And then, after rawcomp completes with that particular set RIF/HUP files and creates the RAW file, delete that subdirectory like this:

Code: [Select]

rmdir /s /q temp\000A
So all that would remain is a file called 000A.raw in temp\.

Also, in this script, I'd like not to have to place the script and rawcomp in the temp folder for it to work. In the script that you so kindly wrote for me, the batch file and rawcomp have to be copied to temp\ and then deleted once the batch file completes.

Again, thanks everyone for your help up to this point and please take my word taht will be the last problem I need solved on this project.