Welcome guest. Before posting on our computer help forum, you must register. Click here it's easy and free.

Author Topic: Seek help on how to use this batch file...  (Read 7595 times)

0 Members and 1 Guest are viewing this topic.

Clutch Cargo

    Topic Starter


    Rookie

    • Experience: Beginner
    • OS: Windows 7
    Seek help on how to use this batch file...
    « on: April 17, 2015, 05:23:15 PM »
    Hi there,

    Yes a nube with very little experience so I am seeking help.  Here's some background info:

    I have a python script 'gdalcopyproj.py' (I did not write it), and it works great.  Basically it takes a tiff geo-referenced image file and transfers that information into another tiff image that does not have geo-referenced data.

    The script is written to apply this one file at a time "gdalcopyproj.py source_file dest_file".  One uses it inside the command prompt of a program called OSGeo4W to implement the script.  For example, it looks like so in the command window.

    “C:\OSGeo4W>python gdalcopyproj.py SOCAL_CZ62.tif SOCAL_CZ62_LM.tif”

    My goal is to process a whole folder of files at once.  I do have a batch file that has been created for another user.  Appears to be somewhat custom for his needs. Here is that batch file:

    @ECHO OFF
    SETLOCAL EnableDelayedExpansion
    @for /f "delims=" %%i in ('2^>nul dir/a-d/b ??????.tif^| findstr/i "^......\.tif$"') do @(
    for /f "delims=" %%j in ('2^>nul dir/a-d/b "%%~ni_*.tif"') do @(
    gdalcopyproj.py "%%i" "%%j"
    )
    )

    What I do not understand is how to use/setup this batch file.  It appears to be seeking a certain type of file which has six characters and converts similarly named files which have custom filenames after the “_”.  Is this correct?  Can I customized that to fit my needs?

    Second, where does one put this batch file.  I have tried in the same folder as the files. I have tried in the directory of OSGeo4W but no luck.  I have tried running the batch directly or from inside a command window.  Still no luck.  Either I get an error message or simply nothing happens.  This where I am stuck.

    Can someone explain how to use this batch file somewhat step-by-step?

    Clutch

    P.S. – not sure if you need the python script to help explain but here she is:


    try:
        from osgeo import gdal
    except ImportError:
        import gdal
    import sys
    if len(sys.argv) < 3:
        print("Usage: gdalcopyproj.py source_file dest_file")
        sys.exit(1)
    input = sys.argv[1]
    dataset = gdal.Open( input )
    if dataset is None:
        print('Unable to open', input, 'for reading')
        sys.exit(1)
    projection   = dataset.GetProjection()
    geotransform = dataset.GetGeoTransform()
    if projection is None and geotransform is None:
        print('No projection or geotransform found on file' + input)
        sys.exit(1)
    output = sys.argv[2]
    dataset2 = gdal.Open( output, gdal.GA_Update )
    if dataset2 is None:
        print('Unable to open', output, 'for writing')
        sys.exit(1)
    if geotransform is not None and geotransform != (0,1,0,0,0,1):
        dataset2.SetGeoTransform( geotransform )
    if projection is not None and projection != '':
        dataset2.SetProjection( projection )
    gcp_count = dataset.GetGCPs()
    if gcp_count != 0:
        dataset2.SetGCPs( gcp_count, dataset.GetGCPProjection() )
    dataset = None
    dataset2 = None


    P.S.S -  Here is even another batch file for the same purpose if this helps but still no luck... sorry coding is not my best suit.  Appears this one would take any file name up to the point of a "_"?  am I understanding this correct?

    @echo off
    for /f "delims=" %%a in ('dir /b *.tif ^| findstr /v /c:"_.tif"') do gdalcopyproj.py "%%~a" "%%~na_.tif"

    Lemonilla



      Apprentice

    • "Too sweet"
    • Thanked: 70
    • Computer: Specs
    • Experience: Experienced
    • OS: Windows 7
    Re: Seek help on how to use this batch file...
    « Reply #1 on: April 17, 2015, 06:22:29 PM »
    Not sure what your first batch file is doing, but try modifying your second one as such:
    Code: [Select]
    @echo off
    for /f "delims=" %%a in ('dir /a-d /b *.tif ^| find /v "_." ') do python gdalcopyproj.py "%%~a" "%%~na_.tif"

    the segment 'for /f "delims=" %%A in () do' tells the program to loop through the list in the parenthesis.  In this case that is 'dir /a-d /b *.tif ^| find /v "_."' which is a list of everything in the current directory, excluding files whose names contain the string "_." and folders.  For each of these, it will run the command 'python gdalcopyproj.py "%%~a" "%%~na_.tif"' where %%~a is replaced with each thing in the list, and %%~na is replaced with only that file's name.

    If you run into the problem where nothing shows up, try removing the "@echo off" at the beginning and post the screen.  (you can do this by right clicking and using the 'mark' function)
    Quote from: patio
    God Bless the DOS Helpers...
    Quote
    If it compiles, send the files.

    Clutch Cargo

      Topic Starter


      Rookie

      • Experience: Beginner
      • OS: Windows 7
      Re: Seek help on how to use this batch file...
      « Reply #2 on: April 17, 2015, 09:36:32 PM »
      Hey Lemonilla,

      Thank you so much for taking a stab at this.  Tried a few times without success so let me give you a play-by-play:

      Created a folder on my S: drive called Geotest.  In there I placed two files: ALBQ_T20.tif (contains geo info), and a second file ALBQ_T20_SH.tiff (which contains no geo info).  I placed your bat file there and ran it.  I got a quick "blip" of the command window but then nothing.  Same thing happens if I remove the @ECHO off text.  No difference.

      2nd test.  I moved the tiff files and the bat file into the C:\OSGeo4W directory.  Same thing happens there as well.  Are my filenames different than what you expected them to be?



      Lemonilla



        Apprentice

      • "Too sweet"
      • Thanked: 70
      • Computer: Specs
      • Experience: Experienced
      • OS: Windows 7
      Re: Seek help on how to use this batch file...
      « Reply #3 on: April 17, 2015, 10:02:58 PM »
      Adding a 'pause' command at the end of a batch file is a good way to debug it.  If you wouldn't mind, remove "@echo off" and put the pause at the end and paste the error message it gives.  This can also be accomplished by running the batch file from a cmd.exe window (command prompt window).

      When I run the program (with a slight edit) it looks like it should run.  Try running the below outside of the batch file and see if there is a syntax error in calling the python script.
      Code: [Select]
      python gdalcopyproj.py "ALBQ_T20.tif" "ALBQ_T20_.tif"


      Code: (my code) [Select]
      @echo off
      for /f "delims=" %%a in ('dir /a-d /b *.tif ^| find /v "_." ') do (
      REM by changing ".tif" to "%%~xa" we are maintaining the exact extension of the previous file.
      REM e.g. if we call it on a .tiff file instead of .tif, it will maintain the double f at the end
      REM the 'echo' is there for debugging purposes only, it shows me how the python script will be called, but bypasses the actual execution
      echo python gdalcopyproj.py "%%~a" "%%~na_%%~xa"
      )
      pause
      Code: (output) [Select]
      python gdalcopyproj.py "ALBQ_T20.tif" "ALBQ_T20_.tif"
      python gdalcopyproj.py "ALBQ_T20_SH.tiff" "ALBQ_T20_SH_.tiff"
      Press any key to continue . . .



      EDIT: You might try removing the quotes around the file names, that might be messing with your python script.  Just know then that your file names cannot have spaces or they will be interpreted as new arguments.
      Quote from: patio
      God Bless the DOS Helpers...
      Quote
      If it compiles, send the files.

      foxidrive



        Specialist
      • Thanked: 268
      • Experience: Experienced
      • OS: Windows 8
      Re: Seek help on how to use this batch file...
      « Reply #4 on: April 18, 2015, 02:15:27 AM »
      Oh look!  Somebody else with exactly the same question!  What are the odds??

      http://www.dostips.com/forum/viewtopic.php?f=3&t=6409

      Lemonilla



        Apprentice

      • "Too sweet"
      • Thanked: 70
      • Computer: Specs
      • Experience: Experienced
      • OS: Windows 7
      Re: Seek help on how to use this batch file...
      « Reply #5 on: April 18, 2015, 08:06:09 AM »
      Oh look!  Somebody else with exactly the same question!  What are the odds??

      http://www.dostips.com/forum/viewtopic.php?f=3&t=6409
      At least he appears to be sharing information between the two.  Can't fault him for covering his bases.
      Quote from: patio
      God Bless the DOS Helpers...
      Quote
      If it compiles, send the files.

      Clutch Cargo

        Topic Starter


        Rookie

        • Experience: Beginner
        • OS: Windows 7
        Re: Seek help on how to use this batch file...
        « Reply #6 on: April 20, 2015, 11:27:34 AM »
        Hey, I am back.  Got tied up with a few other pressing issues.  Yes, I did post this issue at a few different forums.  Looking for options.

        Here is what I did with your latest script.  Called it T2GT-CH.bat.  I placed this in a workfolder I call Geotest along with the python script and the files I wish to process.  They were the following files:

        ALBQ_T20.tif                (has the geo info I need)
        ALBQ_T20_BM.tif         (needs geo info)
        ALBQ_T20_LM.tif          (needs geo info)
        ALBQ_T20_WM.tif        (needs geo info)

        I tried running your bat inside the regular command window and also tried this using the OSGeo4W command window and had the same results.  Nothing was processed.  This was in the command window:

        S:\WORKFOLDER\Geotest>T2GT-CH.bat
        gdalcopyproj.py "ALBQ_T20.tif" "ALBQ_T20_.tif"
        gdalcopyproj.py "ALBQ_T20_BM.tif" "ALBQ_T20_BM_.tif"
        gdgdalcopyproj.py "ALBQ_T20_LM.tif" "ALBQ_T20_LM_.tif"
        gdalcopyproj.py "ALBQ_T20_WM.tif" "ALBQ_T20_WM_.tif"
        press any key to continue...

        An Update - the author of the original bat got back to me and I do have his bat file working.  You weren't sure what he was up to with his file?  I noticed there were some emoticons in there in this post for the "??????", was that why?  Anywho, that was so he could determine how many characters for the filename to search for.  I changed that number to match mine and the "....." as well.  I placed that bat file in the same folder as the files to process along with the python script.  Opening the OSGeo4W command window and to the path of the bat I was able to process successfully.

        But I like your idea in your bat file of changing the finished file to signify a change, the "_".  Tell me, could that be optional say I want to add a small "g" at the end rather than a "_"?    So That is why I am still trying to get you bat to work... or maybe just change his to add an optional character at the end?

        Clutch

        Lemonilla



          Apprentice

        • "Too sweet"
        • Thanked: 70
        • Computer: Specs
        • Experience: Experienced
        • OS: Windows 7
        Re: Seek help on how to use this batch file...
        « Reply #7 on: April 20, 2015, 04:36:26 PM »
        It didn't process anything because I put a print statement to make sure you got the output you wanted before you messed with your files (It's a common batch practice). If the output are the commands you want to run, remove the 'echo' in the 6th line.

        Yes, you can use g instead of _, but it won't work as well, as it will then not process anything that ends in a g.  This however could end up being a file you want processed.  Instead, I'd do a '_g' if you still want the g.  To do this, simply change all instances of _ to _g in my code. (even the one in line 6).

        I'm glad you got it working though.
        Quote from: patio
        God Bless the DOS Helpers...
        Quote
        If it compiles, send the files.

        foxidrive



          Specialist
        • Thanked: 268
        • Experience: Experienced
        • OS: Windows 8
        Re: Seek help on how to use this batch file...
        « Reply #8 on: April 21, 2015, 12:37:05 AM »
        Hey, I am back.  Got tied up with a few other pressing issues.  Yes, I did post this issue at a few different forums.  Looking for options.

        Those of us that recall how to use the T word, and why giving feedback to your question is important, simply shake their head.

        Maybe the younger people think all this is typed by AI bots and there isn't a real person answering at all.


        Clutch Cargo

          Topic Starter


          Rookie

          • Experience: Beginner
          • OS: Windows 7
          Re: Seek help on how to use this batch file...
          « Reply #9 on: April 21, 2015, 08:11:54 AM »
          HI foxidrive,

          Not sure I understand your last?  Did I do something incorrect here?  What is the "T" word?

          Clutch Cargo

            Topic Starter


            Rookie

            • Experience: Beginner
            • OS: Windows 7
            Re: Seek help on how to use this batch file...
            « Reply #10 on: April 21, 2015, 10:45:45 AM »
            Hi Lemonilla,

            An update.  I removed the "echo" as you instructed.  It would not process.  So I added "python" in front of the gdalcopyproj.py and there it appears it tried to process but got the following error message:

            <'Unable to open', 'ALBQ_T20_.tif', 'for writing'>     then it said

            'ALBQ_T20_BM_.tif' does not exist in the file system.     It repeated that message for the _LM and _SH file as well.  It sounds like the bat is trying to match up a one-to-one filename to process where the _BM, _LM, and _SH should be matching up only to the ALBQ_T20.tif which has the geo info I need transferred. 

            Hope that makes sense.

            Clutch

            Lemonilla



              Apprentice

            • "Too sweet"
            • Thanked: 70
            • Computer: Specs
            • Experience: Experienced
            • OS: Windows 7
            Re: Seek help on how to use this batch file...
            « Reply #11 on: April 21, 2015, 02:34:52 PM »
            The error you are getting is a python code error, and it is running on the _SH, _BM, and _LM files because I didn't know they were to be excluded.  Change this line:
            for /f "delims=" %%a in ('dir /a-d /b *.tif ^| find /v "_." ') do (
            to this:
            for /f "delims=" %%a in ('dir /a-d /b *.tif ^| find /v "_SH." ^| find /v "_BM." ^| find /v "_LM." ') do (
            to exclude files ending in those characters.


            To fix your python error, try changing this line:
            dataset = gdal.Open( input )
            to this:
            dataset = gdal.Open( input.replace("\"","") )
            Make sure that you don't mess up the indentation, as that is how python interprets blocks of code.

            Or what might be the problem is that it is reading the arguments wrong.  You can also try changing any instance of sys.argv
            • to sys.argv[#-1]




            Quote from: patio
            God Bless the DOS Helpers...
            Quote
            If it compiles, send the files.

            Clutch Cargo

              Topic Starter


              Rookie

              • Experience: Beginner
              • OS: Windows 7
              Re: Seek help on how to use this batch file...
              « Reply #12 on: April 22, 2015, 10:56:34 AM »
              Ok, tired the latest.  What I got was:

              <'Unable to open', 'ALBQ_T20_.tif', 'for writing'>

              Sounds like to me it is still trying to find a  file that does not exist in the list?.  Also, it sounds maybe the bat file is becoming too literal in the sense of only excluding/including certain filenames.  I fear my range of filenames that can vary from "XX_XX.tif" to "XXXXX_XXXX.tiff" for the files with geo info, to "XX_XX_XX.tiff" to "XXXXXX_XXXX_XXXX.tif" for files that need info is making this quite cumbersome.  The only constant character is that second "_" for files that would need geo info inserted.

              Since the first bat file I posted in my very first post now works, would it be easier on your part to just modify that one to allow me the option to add text of my choice (for example, like a "_" or a"g" or whatever), just before the ".tif"?   I just don't want to have you continue to have to work on this while I am sure others are waiting for your expertise.

              So for example, if I have a file with geo called CA_T20.tif and the file I want to add geo is called CA_T20_LM, I would have the option to call the processed file CA_T20_LMg.  It would not create a new file but just rename the file after being processe

              I really do appreciate you being so patient with me on this and you have put so much work already into this... that's why I'm thinking just modify the first.  What do you think?

              Lemonilla



                Apprentice

              • "Too sweet"
              • Thanked: 70
              • Computer: Specs
              • Experience: Experienced
              • OS: Windows 7
              Re: Seek help on how to use this batch file...
              « Reply #13 on: April 22, 2015, 11:54:03 AM »
              I believe the isssue is somewhere between passing info from the batch file to the python script. Would it be possable for you to post the python acrtipt (in code brackets) along with any dependancies and test files so that i can do some teating on my end.
              Quote from: patio
              God Bless the DOS Helpers...
              Quote
              If it compiles, send the files.

              Clutch Cargo

                Topic Starter


                Rookie

                • Experience: Beginner
                • OS: Windows 7
                Re: Seek help on how to use this batch file...
                « Reply #14 on: April 23, 2015, 12:11:10 PM »
                OK,.... if you really want to do this  ;D

                Sorry for my ignorance but when you said in brackets I wasn't sure if you mean literally just put the code in brackets "[]".  The python script is actually in my first post at the top.  I have also attached the file as well and just gave it a txt suffix so I could upload it if that's easier for you.

                I have created some very-very small sample files for you to test with.  Actual files we use run anywhere from 327MB to 4GB in size ( take awhile to upload!).  Here is a list.  As you can see the filenames that need geo can be quite different looking but the basic format convention is the same:

                SOCAL_DD66.tif
                SOCAL_DD66_BM3.tif
                SOCAL_DD66_LM.tif
                SOCAL_DD66_WM1.tif
                SOCAL_DD66_aSCC.tif

                1. Any file that has only one underscore "_" would be the file that contains the geo information.
                2. Any file that has a second underscore needs geo info.

                I guess if I had my "ultimate" bat file it would be able to drop a bunch of files into a folder like so:

                CA_A1.tif
                CA_A1_LM.tif
                CA_A1_BM.tif
                CA_A1_WM.tif
                CA_A1_aSSC.tif
                NV_AA3.tif
                NV_AA3_LM1.tif
                NV_AA3_BM2.tif
                NV_AA3_WM2a.tif
                NV_AA3_ATCC.tif
                SOCAL_DD66.tif
                SOCAL_DD66_BM3.tif
                SOCAL_DD66_LM.tif
                SOCAL_DD66_WM1.tif
                SOCAL_DD66_aSCC.tif

                From there I could just drop the folder onto the batch file and it would process automatically and results would be in the same directory.  The bat file would allow me the option to add a single character or even text at the end of the processed file or to just leave the filename as is.  Again, for example going from SOCAL_DD66_aSCC.tif to SOCAL_DD66_aSCCg.tif if I wish.

                The program I use OSGeo4W, is really just a collection of geographic tools for processing data.  It is a free download.  It can be a rather large download and I am really only using the Python portion of the toolset.   You can choose which tools you wish to install during the install if you choose "advanced" if (I remember correctly), rather than an "auto" install.  So that can make it a lot smaller and quicker to install should you care to give it a try. 

                The link is:  http://trac.osgeo.org/osgeo4w/ or just google OSGeo4W.   I downloaded and installed the 32bit version.

                I did try just installing Python itself from the python website but I can't get it the python script to work that way.  So I use OSGeo4W.  Just was thinking that could save time rather than installing the toolset.  Maybe not a big deal.

                Because of the severe size limitiations and types I am making several posts with attachments.  Named them all TXT so they would upload.

                [attachment deleted by admin to conserve space]

                Clutch Cargo

                  Topic Starter


                  Rookie

                  • Experience: Beginner
                  • OS: Windows 7
                  Re: Seek help on how to use this batch file...
                  « Reply #15 on: April 23, 2015, 12:11:55 PM »
                  More attachments...

                  [attachment deleted by admin to conserve space]

                  Clutch Cargo

                    Topic Starter


                    Rookie

                    • Experience: Beginner
                    • OS: Windows 7
                    Re: Seek help on how to use this batch file...
                    « Reply #16 on: April 23, 2015, 12:12:25 PM »
                    One more...

                    [attachment deleted by admin to conserve space]

                    Geek-9pm


                      Mastermind
                    • Geek After Dark
                    • Thanked: 1026
                      • Gekk9pm bnlog
                    • Certifications: List
                    • Computer: Specs
                    • Experience: Expert
                    • OS: Windows 10
                    Re: Seek help on how to use this batch file...
                    « Reply #17 on: April 23, 2015, 02:09:22 PM »
                    Clutch Cargo, these files may be much too large for anybody.
                    About attachments:
                    Quote
                    Allowed file types: doc, gif, jpg, mpg, pdf, png, txt, zip, log, bat, bmp
                    Restrictions: 4 per post, maximum total size 700KB, maximum individual size 700KB
                    Yes, you did keep them under the limit. But it is a lot of stuff to look at. I was not able to used Notepad to look at it. It has a bunch of stuff that looks funny.

                    Clutch Cargo

                      Topic Starter


                      Rookie

                      • Experience: Beginner
                      • OS: Windows 7
                      Re: Seek help on how to use this batch file...
                      « Reply #18 on: April 23, 2015, 02:53:08 PM »
                      No, none of those files are meant to be looked at under notepad.  Just remove the .txt .  So for example, SOCAL_DD66.tif.txt should be changed to SOCAL_DD66.tif and viewed through a graphics program.

                      patio

                      • Moderator


                      • Genius
                      • Maud' Dib
                      • Thanked: 1769
                        • Yes
                      • Experience: Beginner
                      • OS: Windows 7
                      Re: Seek help on how to use this batch file...
                      « Reply #19 on: April 23, 2015, 02:55:41 PM »
                      Why do it that way at all ? ?
                      " Anyone who goes to a psychiatrist should have his head examined. "

                      Geek-9pm


                        Mastermind
                      • Geek After Dark
                      • Thanked: 1026
                        • Gekk9pm bnlog
                      • Certifications: List
                      • Computer: Specs
                      • Experience: Expert
                      • OS: Windows 10
                      Re: Seek help on how to use this batch file...
                      « Reply #20 on: April 23, 2015, 03:25:07 PM »
                      Clutch Cargo,
                      ZIP is the universal format for attachments on Computer Hope.
                      Next time please use the following procedure to send attachments.
                      Using Windows  explorer, select the file you wish to attach.
                      Right-click and send to compressed (zipped) folder.
                      This will create a new ZIP folder in the current directory. It contains the file yhou selected. The ZIP folder can be used as an attachment. ZIP does not destroy any information. The original files can be fully restored. It works for any file format.



                      Clutch Cargo

                        Topic Starter


                        Rookie

                        • Experience: Beginner
                        • OS: Windows 7
                        Re: Seek help on how to use this batch file...
                        « Reply #21 on: April 23, 2015, 03:55:23 PM »
                        Funny, I have no idea why I did that instead of a zip, ha!  Thx, will do.
                        « Last Edit: April 23, 2015, 04:43:47 PM by Clutch Cargo »

                        Lemonilla



                          Apprentice

                        • "Too sweet"
                        • Thanked: 70
                        • Computer: Specs
                        • Experience: Experienced
                        • OS: Windows 7
                        Re: Seek help on how to use this batch file...
                        « Reply #22 on: April 25, 2015, 10:08:23 AM »
                        I wanted to let you know that I've been very busy and probably won't get to look at this for another week or two.
                        Quote from: patio
                        God Bless the DOS Helpers...
                        Quote
                        If it compiles, send the files.

                        Clutch Cargo

                          Topic Starter


                          Rookie

                          • Experience: Beginner
                          • OS: Windows 7
                          Re: Seek help on how to use this batch file...
                          « Reply #23 on: April 25, 2015, 10:24:04 AM »
                          Thanks for the update.  No problem at all.  Take your time.  Talk with ya in a few weeks.

                          Clutch