going crazy!! is this possible??

[Dias de verano]

Don't worry.

[Dias de verano]

03/31/09 14:00:30  UPLOAD (C:\TEMP\P2207247  DATA TRANSFER\119073399 message.txt)(This line can be ignored)

The lines that can be ignored - do they always contain "message.txt"?

[rancrazy]

Yes UNLOAD with single lines contains message.txt always

[devcom]

man, thats big script, some programers would want to pay them. Good luck Dias 

[Dias de verano]

man, thats big script, some programers would want to pay them. Good luck Dias 

Once I got all the answers, this has taken me 25 minutes

test2.txt

Code: [Select]

03/31/09 14:00:28  UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28  DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

03/31/09 07:03:02  UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02  DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

03/31/09 15:00:28  UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28  DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip

03/31/09 20:03:02  UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 20:03:02  DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip

03/31/09 14:00:30  UPLOAD (C:\TEMP\P2207247  DATA TRANSFER\119073399 message.txt)(This line can be ignored)

Batch file

Code: [Select]

@echo off
setlocal enabledelayedexpansion
set filename=test2.txt
REM pass 1 - determine latest date
echo Examining file - determining latest date
echo.
set maxdatenum=0
set lastdate=
for /f "delims=" %%L in ( ' type "%filename%" ' ) do (
set thisline=%%L
set datestamp=!thisline:~0,8!
set dateD=!datestamp:~0,2!
set dateM=!datestamp:~3,2!
set dateY=20!datestamp:~6,2!
set datenum=!dateY!!dateM!!dateD!
if !datenum! GTR !maxdatenum! (
set maxdatenum=!datenum!
set lastdate=!datestamp!
echo found date        :   !datestamp!
)
)
echo Latest date found : %lastdate%

REM filenames cannot have / character
set filenamedate=%lastdate:/=-%
echo.
REM pass 2 extract latest date's lines
REM containing DOWNLOAD because only pairs have this line

for /f "tokens=1-26 delims= " %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "DOWNLOAD"' ) do (

set filename=%%~nxQ

Echo Found filename    : !filename!

for /f "delims=_" %%A in ("!filename!") do (

set number=%%A

Echo File number part  : !number!

Echo Creating file     : %filenamedate%-!number!.txt

          REM If prior file exists for same date and file number
          REM it will be DELETED and replaced with latest one.
if exist "%filenamedate%-!number!.txt" del "%filenamedate%-!number!.txt"

REM group each filenumber pair together, ignore lines containing message.txt

for /f "delims=" %%B in ( ' type "%filename%" ^| find "%lastdate%" ^| find "!number!" ^| find /v "message.txt"' ) do (

Echo %%B>>%filenamedate%-!number!.txt

)

echo File contents:

type "%filenamedate%-!number!.txt"

echo.
)
)



Code: [Select]

Examining file - determining latest date

found date:   03/31/09
Latest date found : 03/31/09

Found filename    : P2207247_TRF_31Mar2009.zip
File number part  : P2207247
Creating file     : 03-31-09-P2207247.txt
File contents:
03/31/09 14:00:28  UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28  DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

Found filename    : P33108205_TRF_31Mar2009.zip
File number part  : P33108205
Creating file     : 03-31-09-P33108205.txt
File contents:
03/31/09 07:03:02  UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02  DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

Found filename    : PP2107244_TRF_31Mar2009.zip
File number part  : PP2107244
Creating file     : 03-31-09-PP2107244.txt
File contents:
03/31/09 15:00:28  UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28  DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip

Found filename    : P3106230_TRF_31Mar2009.zip
File number part  : P3106230
Creating file     : 03-31-09-P3106230.txt
File contents:
03/31/09 20:03:02  UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 20:03:02  DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip

[rancrazy]

Hats off to you!!!You are a real GURU in Batch Scripting.thanks a million

[Dias de verano]

Does that mean it worked?

[rancrazy]

Yes it works perfectly Thanks a lot

[ghostdog74]

this is an alternative, if you can download gawk for windows

Code: [Select]

FNR==NR{a[++d]=$1;next}
/UPLOAD/ && $1 == a[d] {
 org=$0; gsub(/.*\\|)|\.zip/,"")
 m=split($0,f,"_"); file=f[1]; getline line
 if ( line ~ /DOWNLOAD/ && line ~ file) {   
   print org > file"-"f[3]
   print line > file"-"f[3]
   print "File contents of " file"_"f[3]
   print org
   print line
   print "----------------------------------"
 }
}

save the above as script.awk and on command line

Code: [Select]

C:\test>gawk -f testawk.bat file.txt file.txt
File contents of P2207247_31Mar2009
03/31/09 14:00:28  UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28  DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip
----------------------------------
File contents of P33108205_31Mar2009
03/31/09 07:03:02  UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02  DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip
----------------------------------
File contents of PP2107244_31Mar2009
03/31/09 15:00:28  UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28  DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip
----------------------------------
File contents of P3106230_31Mar2009
03/31/09 20:03:02  UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 20:03:02  DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
----------------------------------



[BC_Programmer]

lol

little late now.

[ghostdog74]

better late than never

[Reno]

wow, that's a one *censored* of a script. great stuff with perfect coding. 
and you must have some very liquid brain to do it in just 25minutes.
DIAS is the man.

and i notice you didn't use findstr anymore.

[Dias de verano]

wow, that's a one *censored* of a script. great stuff with perfect coding. 
and you must have some very liquid brain to do it in just 25minutes.
DIAS is the man.

and i notice you didn't use findstr anymore.

Hold your horses, the colossus has stumbled,    I modified the test txt so it had an appended duplicated upload/download pair at 22:00 and the result is a 4 line output file for that file number. That is, it does not select the latest pair. I am modifying the code as we speak.

I haven't got a liquid brain, it was only typing the code out & testing it as I built it up that took 25 minutes; the actual thinking about what it was going to do took place sort of subconsciously over about a day I suppose, as I asked questions and got answers from rancrazy.

I am going to add a little bit of code to verify that each output file has only 2 lines and if it has more, delete all but the last 2.

So, rancrazy, watch this space!

I never really got the findstr habit, if find does the job I tend to stick with that.

[Dias de verano]

test5.txt

Note: in the example supplied, some of the filenumbers don't match

e.g.

03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)

Code: [Select]

03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE1_19Mar2009.zip)
03/20/09 10:31:54 DOWNLOAD FILE1_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE1_19Mar2009.zip
03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE2_19Mar2009.zip)
03/20/09 10:31:54 DOWNLOAD FILE2_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE2_19Mar2009.zip
03/21/09 11:20:52 UPLOAD C:\TEMP\UPLOAD\FILE3_20Mar2009.zip)
03/21/09 11:21:54 DOWNLOAD FILE3_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE3_20Mar2009.zip
03/21/09 12:31:52 UPLOAD C:\TEMP\UPLOAD\FILE4_20Mar2009.zip)
03/21/09 12:31:54 DOWNLOAD FILE4_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE4_20Mar2009.zip
03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip
03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip
03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip
03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247  DATA TRANSFER\119073399 message.txt)(This line can be ignored)
03/31/09 22:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 22:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip
03/31/09 22:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 22:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
03/31/09 23:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 23:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

Batch file

Code: [Select]

@echo off
setlocal enabledelayedexpansion
set filename=test5.txt
REM pass 1 - determine latest date
echo Examining file - determining latest date
set maxdatenum=0
set lastdate=
for /f "delims=" %%L in ( ' type "%filename%" ' ) do (
set thisline=%%L
set datestamp=!thisline:~0,8!
set dateD=!datestamp:~0,2!
set dateM=!datestamp:~3,2!
set dateY=20!datestamp:~6,2!
set datenum=!dateY!!dateM!!dateD!
if !datenum! GTR !maxdatenum! (
set maxdatenum=!datenum!
set lastdate=!datestamp!
echo Found date        : !datestamp!
)
)
echo Latest date found : %lastdate%
REM filenames cannot have / character
set filenamedate=%lastdate:/=-%
REM pass 2 extract latest date's lines
REM containing DOWNLOAD because only pairs have this line
set alreadydone=
REM The filename transferred is token 6
for /f "tokens=1-26 delims= " %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "DOWNLOAD"' ) do (
set filename=%%~nxQ
REM Echo Found filename    : !filename!
REM Find number part within transferred filename
REM Delimiter is underscore character
for /f "delims=_" %%A in ("!filename!") do (
set number=%%A
echo !alreadydone! | find ",!number!,">nul || (
set alreadydone=,!number!,!alreadydone!
echo.
REM Echo File number part  : !number!
Set Outfile=%filenamedate%-!number!.txt
Echo Creating file     : !outfile!
if exist "!outfile!.tmp" del "!outfile!.tmp"
if exist "!outfile!" del "!outfile!"
REM Examine entire log file and extract lines containing current
REM transferred file number
REM group each filenumber pair together, ignore lines containing message.txt
REM Write to temp file first
for /f "delims=" %%B in ( ' type "%filename%" ^| find "%lastdate%" ^| find "!number!" ^| find /v "message.txt"' ) do (
Echo %%B>>"!outfile!.tmp"
)
REM Only take latest pair for each file number
REM Count lines in output
set numlines=0&for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do set /a numlines+=1
                REM Subtract 2, this gives the first line to take
set /a startline=!numlines!-2
REM Transfer last 2 lines of tmp file to
REM real output file
set line=1
for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do (
if !line! GTR !startline! (
echo %%C>>!outfile!
echo %%C
)
set /a line+=1
)
del "!outfile!.tmp"
)
)
)
Echo.


Output

Code: [Select]

Examining file - determining latest date
Found date        : 03/20/09
Found date        : 03/21/09
Found date        : 03/31/09
Latest date found : 03/31/09

Creating file     : 03-31-09-P2207247.txt
03/31/09 22:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 22:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

Creating file     : 03-31-09-P33108205.txt
03/31/09 23:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 23:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

Creating file     : 03-31-09-PP2107244.txt
03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip

Creating file     : 03-31-09-P3106230.txt
03/31/09 22:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 22:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip



[rancrazy]

wow!!!This was a good surprise.Thank you so much Dias.This is excellent batch script.
I have tested and it works perfectly.Thanks to every one guys.

In the example 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) there was typo actually it is
03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108205_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)

But still this code works fine.