Computer Hope

Microsoft => Microsoft DOS => Topic started by: rancrazy on March 31, 2009, 12:24:33 PM

Title: going crazy!! is this possible??
Post by: rancrazy on March 31, 2009, 12:24:33 PM: Hi
Can any one let me know ,is there any possibility that I can I extract lines with latest date and each file name into different files?
for example I want lines 03/21/09 with FILE4 to one file
and lines 03/21/09 with FILE3 to other file.
03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE1_19Mar2009.zip)
03/20/09 10:31:54 DOWNLOAD FILE1_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE1_19Mar2009.zip
03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE2_19Mar2009.zip)
03/20/09 10:31:54 DOWNLOAD FILE2_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE2_19Mar2009.zip
03/21/09 11:20:52 UPLOAD C:\TEMP\UPLOAD\FILE3_20Mar2009.zip)
03/21/09 11:21:54 DOWNLOAD FILE3_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE3_20Mar2009.zip
03/21/09 12:31:52 UPLOAD C:\TEMP\UPLOAD\FILE4_20Mar2009.zip)
03/21/09 12:31:54 DOWNLOAD FILE4_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE4_20Mar2009.zip

Its very urgent and please help me!!!!!
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on March 31, 2009, 02:28:57 PM: 1. test.txt

Code: [Select]
03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE1_19Mar2009.zip) 03/20/09 10:31:54 DOWNLOAD FILE1_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE1_19Mar2009.zip 03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE2_19Mar2009.zip) 03/20/09 10:31:54 DOWNLOAD FILE2_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE2_19Mar2009.zip 03/21/09 11:20:52 UPLOAD C:\TEMP\UPLOAD\FILE3_20Mar2009.zip) 03/21/09 11:21:54 DOWNLOAD FILE3_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE3_20Mar2009.zip 03/21/09 12:31:52 UPLOAD C:\TEMP\UPLOAD\FILE4_20Mar2009.zip) 03/21/09 12:31:54 DOWNLOAD FILE4_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE4_20Mar2009.zip
2. Batch file

Code: [Select]
@echo off setlocal enabledelayedexpansion set filename=test.txt REM pass 1 - determine latest date echo Examining file - determining latest date echo. set maxdatenum=0 set lastdate= for /f "delims=" %%L in ( ' type "%filename%" ' ) do ( set thisline=%%L set datestamp=!thisline:~0,8! set dateD=!datestamp:~0,2! set dateM=!datestamp:~3,2! set dateY=20!datestamp:~6,2! set datenum=!dateY!!dateM!!dateD! if !datenum! GTR !maxdatenum! ( set maxdatenum=!datenum! set lastdate=!datestamp! echo found date: !datestamp! ) ) echo Latest date : %lastdate% echo. REM pass 2 - extract lines from latest date containg FILE3_ echo Lines starting with date %lastdate% containing FILE3_ : echo. for /f "delims=" %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "FILE3_" ' ) do echo %%L echo. REM pass 3 - extract lines from latest date containg FILE4_ echo Lines starting with date %lastdate% containing FILE4_ : echo. for /f "delims=" %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "FILE4_" ' ) do echo %%L echo.
Code: [Select]
Examining file - determining latest date found date: 03/20/09 found date: 03/21/09 Latest date : 03/21/09 Lines starting with date 03/21/09 containing FILE3_ : 03/21/09 11:20:52 UPLOAD C:\TEMP\UPLOAD\FILE3_20Mar2009.zip) 03/21/09 11:21:54 DOWNLOAD FILE3_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE3_20Mar2009.zip Lines starting with date 03/21/09 containing FILE4_ : 03/21/09 12:31:52 UPLOAD C:\TEMP\UPLOAD\FILE4_20Mar2009.zip) 03/21/09 12:31:54 DOWNLOAD FILE4_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE4_20Mar2009.zip
Title: Re: going crazy!! is this possible??
Post by: rancrazy on March 31, 2009, 05:01:54 PM: Thanks a lot.It worked wonderfully.Is there any possibility that we can apply this to a case where
FILE3_ and FILE4_ are replaced with P2210223_ and P3104234_ etc But instead of manually finding the string for FILE3_ and FILE4_ this batch script should automate to seperate such files upload and download lines with latest date.I want seperate extractions for P2210223_ UPLOAD and DOWNLOAD lines from test.txt and same for the other P3104234_.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 12:18:15 AM: Quote from: rancrazy on March 31, 2009, 05:01:54 PM
Thanks a lot.It worked wonderfully.Is there any possibility that we can apply this to a case where
FILE3_ and FILE4_ are replaced with P2210223_ and P3104234_ etc But instead of manually finding the string for FILE3_ and FILE4_ this batch script should automate to seperate such files upload and download lines with latest date.I want seperate extractions for P2210223_ UPLOAD and DOWNLOAD lines from test.txt and same for the other P3104234_.

To clarify:

You want 4 extractions:

Latest date P2210223_ UPLOAD
Latest date P2210223_ DOWNLOAD
Latest date P3104234_ UPLOAD
Latest date P3104234_ DOWNLOAD

Is that correct?

It would help if you supply a specimen file like you did before.
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 08:09:36 AM: This is a sample file extactly looks like my data
03/24/09 13:54:23 UPLOAD (C:\TEMP\P2207246 DATA TRANSFER\119073327 message.txt))

03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\119073399 message.txt)

03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

I need UPLOAD and DOWNLOAD lines for P2207247 in seperate file and P33108245 in seperate file.My requirement is to automate this process.single UPLOAD lines can be ignored.I have seperated the data for clear image of my requirement.My Log file contains several other lines with different names starting with P31...etc I hope I did not mess up.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 09:46:26 AM: 6 questions below:

1. Two files therefore?

P2207247 upload and download P33108245 upload and download

2. Ignore single upload lines, only extract matching upload / download pairs like these below?
3. Will they always be on consecutive lines like this?
4. Do they always have identical date and time stamps?

03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

Quote from: rancrazy
My Log file contains several other lines with different names starting with P31...etc

5. I hope all lines will start with a date. Hopefully the names starting P31... etc are in the places where P22... and P33... are in the example above?

6. Such lines starting P31... etc to be ignored?
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 10:32:06 AM: First of all thank you so much for helping me.
1. Two files therefore? YES for this example but there are more line starting with P31 or P22,P15 etc having UPLOAD and DOWNLOAD lines with latest date

P2207247 upload and download
P33108245 upload and download

2. Ignore single upload lines, only extract matching upload / download pairs like these below? YES
3. Will they always be on consecutive lines like this? YES
4. Do they always have identical date and time stamps? almost same but some times one second diff may be there

03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

5. I hope all lines will start with a date. Hopefully the names starting P31... etc are in the places where P22... and P33... are in the example above? Yes all the lines start with date.names with P31, P21,P15 etc

6. Such lines starting P31... etc to be ignored? no only lines with single UPLOAD unlike pairing UPLOAD and DOWNLOAD.These single lines show that data was not DOWNLOADED.so not required.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 10:48:50 AM: Now I am more confused.

Do you just want these 2 files?

P2207247 upload and download
P33108245 upload and download

All other Pxx numbers ignored?

Only upload/download pairs?
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 10:57:41 AM: I mean to say I have several additional line like below.I need a batch script to get UPLOAD/DOWNLOAD pairs of each one into seperate file.If any single line with UPLOAD is found can be ignored.

03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip)
03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip

03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)
03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip

03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip)
03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip

03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip)
03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip

03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\119073399 message.txt)(This line can be ignored)
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 11:08:56 AM: 1. So ALL upload / download pairs?
2. whatever the Pxxxxx number they share?
3. and into separate files? One file for each pair? Like this (see below)?
4. And how shall the files be named? (By the Pxxxxx number maybe?)
5. Is UPLOAD always the first line of such a pair?
6. Is the file format always Pxxxxxxx_TRF_date.zip ?
7. Is each Pxxxxxxx number unique? (could it occur more than once?)

file1
Code: [Select]
03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip
file2
Code: [Select]
03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip
file3
Code: [Select]
03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip
file4 etc
Code: [Select]
03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 11:21:27 AM: 1. So ALL upload / download pairs? Yes
2. whatever the Pxxxxx number they share? Yes
3. and into separate files? One file for each pair? Like this (see below)? Yes one file for each pair
4. And how shall the files be named? (By the Pxxxxx number maybe?) Pxxxxx_date
5. Is UPLOAD always the first line of such a pair? Yes Upload is first line
6. Is the file format always Pxxxxxxx_TRF_date.zip ? Yes
7. Is each Pxxxxxxx number unique? (could it occur more than once?) No It can occur more than once some times
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 11:44:35 AM: Quote from: rancrazy on April 01, 2009, 11:21:27 AM
7. Is each Pxxxxxxx number unique? (could it occur more than once?) No It can occur more than once some times

More than once on the same date?
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 11:49:04 AM: Yes it can be on same date but we have to extract most current time stamp from those two.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 12:12:28 PM: Quote from: rancrazy on April 01, 2009, 11:49:04 AM
Yes it can be on same date but we have to extract most current time stamp from those two.

The task keeps growing... Have you thought about employing a programmer? This is high grade work.
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 12:19:52 PM: well this is not a growing task but my actual requirement and as I am new to batch scripting it is looking like impossible task for me. ???Even if you can help me one occurence (most current stamp not required) I will mange some how.please help me out
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 12:32:09 PM: Don't worry.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 01:33:03 PM: Quote
03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\119073399 message.txt)(This line can be ignored)

The lines that can be ignored - do they always contain "message.txt"?
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 01:37:01 PM: Yes UNLOAD with single lines contains message.txt always
Title: Re: going crazy!! is this possible??
Post by: devcom on April 01, 2009, 01:49:37 PM: man, thats big script, some programers would want to pay them. Good luck Dias ;)
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 01:58:58 PM: Quote from: devcom on April 01, 2009, 01:49:37 PM
man, thats big script, some programers would want to pay them. Good luck Dias ;)

Once I got all the answers, this has taken me 25 minutes

test2.txt

Code: [Select]
03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip 03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip 03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip 03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\119073399 message.txt)(This line can be ignored)
Batch file

Code: [Select]
@echo off setlocal enabledelayedexpansion set filename=test2.txt REM pass 1 - determine latest date echo Examining file - determining latest date echo. set maxdatenum=0 set lastdate= for /f "delims=" %%L in ( ' type "%filename%" ' ) do ( set thisline=%%L set datestamp=!thisline:~0,8! set dateD=!datestamp:~0,2! set dateM=!datestamp:~3,2! set dateY=20!datestamp:~6,2! set datenum=!dateY!!dateM!!dateD! if !datenum! GTR !maxdatenum! ( set maxdatenum=!datenum! set lastdate=!datestamp! echo found date : !datestamp! ) ) echo Latest date found : %lastdate% REM filenames cannot have / character set filenamedate=%lastdate:/=-% echo. REM pass 2 extract latest date's lines REM containing DOWNLOAD because only pairs have this line for /f "tokens=1-26 delims= " %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "DOWNLOAD"' ) do ( set filename=%%~nxQ Echo Found filename : !filename! for /f "delims=_" %%A in ("!filename!") do ( set number=%%A Echo File number part : !number! Echo Creating file : %filenamedate%-!number!.txt REM If prior file exists for same date and file number REM it will be DELETED and replaced with latest one. if exist "%filenamedate%-!number!.txt" del "%filenamedate%-!number!.txt" REM group each filenumber pair together, ignore lines containing message.txt for /f "delims=" %%B in ( ' type "%filename%" ^| find "%lastdate%" ^| find "!number!" ^| find /v "message.txt"' ) do ( Echo %%B>>%filenamedate%-!number!.txt ) echo File contents: type "%filenamedate%-!number!.txt" echo. ) )
Code: [Select]
Examining file - determining latest date found date: 03/31/09 Latest date found : 03/31/09 Found filename : P2207247_TRF_31Mar2009.zip File number part : P2207247 Creating file : 03-31-09-P2207247.txt File contents: 03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip Found filename : P33108205_TRF_31Mar2009.zip File number part : P33108205 Creating file : 03-31-09-P33108205.txt File contents: 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip Found filename : PP2107244_TRF_31Mar2009.zip File number part : PP2107244 Creating file : 03-31-09-PP2107244.txt File contents: 03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip Found filename : P3106230_TRF_31Mar2009.zip File number part : P3106230 Creating file : 03-31-09-P3106230.txt File contents: 03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 02:07:04 PM: Hats off to you!!!You are a real GURU in Batch Scripting.thanks a million :)
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 01, 2009, 02:39:53 PM: Does that mean it worked?
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 01, 2009, 03:31:10 PM: Yes it works perfectly Thanks a lot
Title: Re: going crazy!! is this possible??
Post by: ghostdog74 on April 01, 2009, 08:20:27 PM: this is an alternative, if you can download gawk for windows (http://gnuwin32.sourceforge.net/packages/gawk.htm)
Code: [Select]
FNR==NR{a[++d]=$1;next} /UPLOAD/ && $1 == a[d] { org=$0; gsub(/.*\\|)|\.zip/,"") m=split($0,f,"_"); file=f[1]; getline line if ( line ~ /DOWNLOAD/ && line ~ file) { print org > file"-"f[3] print line > file"-"f[3] print "File contents of " file"_"f[3] print org print line print "----------------------------------" } }save the above as script.awk and on command line
Code: [Select]
C:\test>gawk -f testawk.bat file.txt file.txt File contents of P2207247_31Mar2009 03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip ---------------------------------- File contents of P33108205_31Mar2009 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip ---------------------------------- File contents of PP2107244_31Mar2009 03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip ---------------------------------- File contents of P3106230_31Mar2009 03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip ----------------------------------
Title: Re: going crazy!! is this possible??
Post by: BC_Programmer on April 01, 2009, 09:10:29 PM: lol

little late now.
Title: Re: going crazy!! is this possible??
Post by: ghostdog74 on April 01, 2009, 09:20:51 PM: better late than never
Title: Re: going crazy!! is this possible??
Post by: Reno on April 01, 2009, 10:50:01 PM: wow, that's a one *censored* of a script. great stuff with perfect coding. :)
and you must have some very liquid brain to do it in just 25minutes.
DIAS is the man.

and i notice you didn't use findstr anymore.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 02, 2009, 12:43:22 AM: Quote from: Reno on April 01, 2009, 10:50:01 PM
wow, that's a one *censored* of a script. great stuff with perfect coding. :)
and you must have some very liquid brain to do it in just 25minutes.
DIAS is the man.

and i notice you didn't use findstr anymore.

Hold your horses, the colossus has stumbled, ;) I modified the test txt so it had an appended duplicated upload/download pair at 22:00 and the result is a 4 line output file for that file number. That is, it does not select the latest pair. I am modifying the code as we speak.

I haven't got a liquid brain, it was only typing the code out & testing it as I built it up that took 25 minutes; the actual thinking about what it was going to do took place sort of subconsciously over about a day I suppose, as I asked questions and got answers from rancrazy.

I am going to add a little bit of code to verify that each output file has only 2 lines and if it has more, delete all but the last 2.

So, rancrazy, watch this space!

I never really got the findstr habit, if find does the job I tend to stick with that.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 02, 2009, 01:30:26 AM: test5.txt

Note: in the example supplied, some of the filenumbers don't match

e.g.

03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)

Code: [Select]
03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE1_19Mar2009.zip) 03/20/09 10:31:54 DOWNLOAD FILE1_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE1_19Mar2009.zip 03/20/09 10:31:52 UPLOAD C:\TEMP\UPLOAD\FILE2_19Mar2009.zip) 03/20/09 10:31:54 DOWNLOAD FILE2_19Mar2009.zip to C:\TEMP\DOWNLOAD\FILE2_19Mar2009.zip 03/21/09 11:20:52 UPLOAD C:\TEMP\UPLOAD\FILE3_20Mar2009.zip) 03/21/09 11:21:54 DOWNLOAD FILE3_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE3_20Mar2009.zip 03/21/09 12:31:52 UPLOAD C:\TEMP\UPLOAD\FILE4_20Mar2009.zip) 03/21/09 12:31:54 DOWNLOAD FILE4_20Mar2009.zip to C:\TEMP\DOWNLOAD\FILE4_20Mar2009.zip 03/31/09 14:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 14:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 07:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip 03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip 03/31/09 20:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 20:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip 03/31/09 14:00:30 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\119073399 message.txt)(This line can be ignored) 03/31/09 22:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 22:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip 03/31/09 22:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 22:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip 03/31/09 23:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 23:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip
Batch file

Code: [Select]
@echo off setlocal enabledelayedexpansion set filename=test5.txt REM pass 1 - determine latest date echo Examining file - determining latest date set maxdatenum=0 set lastdate= for /f "delims=" %%L in ( ' type "%filename%" ' ) do ( set thisline=%%L set datestamp=!thisline:~0,8! set dateD=!datestamp:~0,2! set dateM=!datestamp:~3,2! set dateY=20!datestamp:~6,2! set datenum=!dateY!!dateM!!dateD! if !datenum! GTR !maxdatenum! ( set maxdatenum=!datenum! set lastdate=!datestamp! echo Found date : !datestamp! ) ) echo Latest date found : %lastdate% REM filenames cannot have / character set filenamedate=%lastdate:/=-% REM pass 2 extract latest date's lines REM containing DOWNLOAD because only pairs have this line set alreadydone= REM The filename transferred is token 6 for /f "tokens=1-26 delims= " %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "DOWNLOAD"' ) do ( set filename=%%~nxQ REM Echo Found filename : !filename! REM Find number part within transferred filename REM Delimiter is underscore character for /f "delims=_" %%A in ("!filename!") do ( set number=%%A echo !alreadydone! | find ",!number!,">nul || ( set alreadydone=,!number!,!alreadydone! echo. REM Echo File number part : !number! Set Outfile=%filenamedate%-!number!.txt Echo Creating file : !outfile! if exist "!outfile!.tmp" del "!outfile!.tmp" if exist "!outfile!" del "!outfile!" REM Examine entire log file and extract lines containing current REM transferred file number REM group each filenumber pair together, ignore lines containing message.txt REM Write to temp file first for /f "delims=" %%B in ( ' type "%filename%" ^| find "%lastdate%" ^| find "!number!" ^| find /v "message.txt"' ) do ( Echo %%B>>"!outfile!.tmp" ) REM Only take latest pair for each file number REM Count lines in output set numlines=0&for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do set /a numlines+=1 REM Subtract 2, this gives the first line to take set /a startline=!numlines!-2 REM Transfer last 2 lines of tmp file to REM real output file set line=1 for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do ( if !line! GTR !startline! ( echo %%C>>!outfile! echo %%C ) set /a line+=1 ) del "!outfile!.tmp" ) ) ) Echo.
Output

Code: [Select]
Examining file - determining latest date Found date : 03/20/09 Found date : 03/21/09 Found date : 03/31/09 Latest date found : 03/31/09 Creating file : 03-31-09-P2207247.txt 03/31/09 22:00:28 UPLOAD (C:\TEMP\P2207247 DATA TRANSFER\P2207247_TRF_31Mar2009.zip) 03/31/09 22:00:28 DOWNLOAD P2207247_TRF_31Mar2009.zip to C:\TEMP\P2207247_TRF_31Mar2009.zip Creating file : 03-31-09-P33108205.txt 03/31/09 23:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) 03/31/09 23:03:02 DOWNLOAD P33108245_TRF_31Mar2009.zip to C:\TEMP\P33108205_TRF_31Mar2009.zip Creating file : 03-31-09-PP2107244.txt 03/31/09 15:00:28 UPLOAD (C:\TEMP\P2107244 DATA TRANSFER\PP2107244_TRF_31Mar2009.zip) 03/31/09 15:00:28 DOWNLOAD P2107244_TRF_31Mar2009.zip to C:\TEMP\PP2107244_TRF_31Mar2009.zip Creating file : 03-31-09-P3106230.txt 03/31/09 22:03:02 UPLOAD (C:\TEMP\P3106230_TRF_31Mar2009\P3106230_TRF_31Mar2009.zip) 03/31/09 22:03:02 DOWNLOAD P3106230_TRF_31Mar2009.zip to C:\TEMP\P3106230_TRF_31Mar2009.zip
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 02, 2009, 07:46:58 AM: wow!!!This was a good surprise.Thank you so much Dias.This is excellent batch script.
I have tested and it works perfectly.Thanks to every one guys.

In the example 03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108245_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip) there was typo actually it is
03/31/09 07:03:02 UPLOAD (C:\TEMP\P33108205_TRF_31Mar2009\P33108205_TRF_31Mar2009.zip)

But still this code works fine.
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 12, 2009, 10:46:45 AM: Dias I need your help!!I need to understand how the "maxdatenum" works as I am getting problem when latest date is 4/1/09 instead of 3/31/09.It is still extracting only 3/31/09 even if latest date is 4/1/09 or later.
please help me understand.I have googled for this but no use.
Title: Re: going crazy!! is this possible??
Post by: Dias de verano on April 12, 2009, 11:51:26 AM: Apologies for the stupid error in my code. The script was supposed to extract the first 8 characters of each line which are MM/DD/YY e.g. 04/01/09 and re arrange them to make a number for the date in the form YYYYMMDD e.g. 20090331 or 20090401 such that a later date makes a greater number. However by a stupid error I had the month and day transposed. I would have caught it if I had tested with a month later than March. Fixed below.

Code: [Select]
@echo off setlocal enabledelayedexpansion set filename=test0401.txt REM pass 1 - determine latest date echo Examining file - determining latest date set maxdatenum=0 set lastdate= for /f "delims=" %%L in ( ' type "%filename%" ' ) do ( set thisline=%%L set datestamp=!thisline:~0,8! REM Fixed set dateM=!datestamp:~0,2! REM was previously !datestamp:~3,2! REM Fixed set dateD=!datestamp:~3,2! REM was previously !datestamp:~0,2! set dateY=20!datestamp:~6,2! REM Datenum=YYYYMMDD REM e.g. 20093103 REM e.g. 20090401 set datenum=!dateY!!dateM!!dateD! if !datenum! GTR !maxdatenum! ( set maxdatenum=!datenum! set lastdate=!datestamp! echo Found date : !datestamp! ) ) echo Latest date found : %lastdate% REM filenames cannot have / character set filenamedate=%lastdate:/=-% REM pass 2 extract latest date's lines REM containing DOWNLOAD because only pairs have this line set alreadydone= REM The filename transferred is token 6 for /f "tokens=1-26 delims= " %%L in ( ' type "%filename%" ^| find "%lastdate%" ^| find "DOWNLOAD"' ) do ( set filename=%%~nxQ REM Echo Found filename : !filename! REM Find number part within transferred filename REM Delimiter is underscore character for /f "delims=_" %%A in ("!filename!") do ( set number=%%A echo !alreadydone! | find ",!number!,">nul || ( set alreadydone=,!number!,!alreadydone! echo. REM Echo File number part : !number! Set Outfile=%filenamedate%-!number!.txt Echo Creating file : !outfile! if exist "!outfile!.tmp" del "!outfile!.tmp" if exist "!outfile!" del "!outfile!" REM Examine entire log file and extract lines containing current REM transferred file number REM group each filenumber pair together, ignore lines containing message.txt REM Write to temp file first for /f "delims=" %%B in ( ' type "%filename%" ^| find "%lastdate%" ^| find "!number!" ^| find /v "message.txt"' ) do ( Echo %%B>>"!outfile!.tmp" ) REM Only take latest pair for each file number REM Count lines in output set numlines=0&for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do set /a numlines+=1 REM Subtract 2, this gives the first line to take set /a startline=!numlines!-2 REM Transfer last 2 lines of tmp file to REM real output file set line=1 for /f "delims=" %%C in ( ' type "!outfile!.tmp"' ) do ( if !line! GTR !startline! ( echo %%C>>!outfile! echo %%C ) set /a line+=1 ) del "!outfile!.tmp" ) ) ) Echo.
Title: Re: going crazy!! is this possible??
Post by: rancrazy on April 12, 2009, 12:14:14 PM: Yes I guessed you have add the date format numbers but when I tried to correct it and refered to other topics in google I did not find any.Thank you so much for your time.It works perfect now.