Subtract images in batch process(leap year and regular year)

[Geek-9pm]

Does to OP want to destroy or reduce? Subtract may imply destroy or remove.
Alternative ways to reduce something without using the word subtract.

    depreciate
    derogate
    detract
    discountenance
    disesteem
    disfavor

    disparage
    expostulate
    frown
    object
    pooh-pooh
    rip
    disapprove of
    discommend
    mudsling
    not go for
    poor mouth

    protest against
    put down
    run down
    take dim view of
    take down
    take exception to

There are more. But the above is enough. 

[_unknown_]

Does to OP want to destroy or reduce? Subtract may imply destroy or remove.
Alternative ways to reduce something without using the word subtract.There are more. But the above is enough. 

Subtract in this case will be used mathematically. Means to take, deduct, take from, take out etc.

[foxidrive]

Well nothing you said replied to the question I posed.

My question wasn't very clear I have to say.

What I wanted to know is are the files created or changed in date order?

So Tuesday 2nd Dec file is created or changed on Tuesday 2nd Dec and is then processed when the code is run, and
Wednesday 3rd Dec file is created or changed on Wednesday 3rd Dec and is then processed when the code is run etc....

[_unknown_]

My question wasn't very clear I have to say.

What I wanted to know is are the files created or changed in date order?

So Tuesday 2nd Dec file is created or changed on Tuesday 2nd Dec and is then processed when the code is run, and
Wednesday 3rd Dec file is created or changed on Wednesday 3rd Dec and is then processed when the code is run etc....

The files were downloaded from an ftp site. So everyday there will be files added. I don't know if I'm getting your point.

[foxidrive]

.

[_unknown_]

Do your files get modified so that the most recent file was the last one that was created, or modified?

Anyway, these goal isn't included anymore. The goal now are:

• Create an output directory for the output files
• Create a proc_path directory for the processed files
• Subtract image from the in_path to another image from another directory(e.g. 2 - 1 = 1, image1 - image2 = difference(some part of the image2 that weren't on the image 1 were taken/deducted or vice versa)

I hope I explained it well. Hoping for your help 

[Geek-9pm]

What sort of real world application requites subtractive if one image from another?
Do these  images represent 2 dimensional objects? Are the images abstractions? Do the represent some kind of matrix? Are these images' Fourier transformations?
http://en.wikipedia.org/wiki/Fourier_transform
If they are Fourier transformation, subtraction would make sense.

Fourier transformation is used in the process of MRI systems. An yes, they are called images. Is this what the topic is about?

[_unknown_]

What sort of real world application requites subtractive if one image from another?
Do these  images represent 2 dimensional objects? Are the images abstractions? Do the represent some kind of matrix? Are these images' Fourier transformations?
http://en.wikipedia.org/wiki/Fourier_transform
If they are Fourier transformation, subtraction would make sense.

Fourier transformation is used in the process of MRI systems. An yes, they are called images. Is this what the topic is about?

These are raster images downloaded from NASA which contains specific data sets needed in subtraction. It represents the whole image of a country.

[Geek-9pm]

These are raster images downloaded from NASA which contains specific data sets needed in subtraction. It represents the whole image of a country.

Thank you! Now it all makes sense. NASA  image subtraction is part of the 
Fourier transformation algorithms. 
https://cloud1.arc.nasa.gov/solve/payload/dc8/ftir.html

Measurement Description: The NCAR Fourier transform spectrometer will record high-resolution solar infrared absorption spectra of the atmosphere above the DC-8 aircraft. From these absorption spectra column abundances of a number of stratospheric gases important to polar ozone chemistry will be derived. The spectra also will be used to attempt to detect the infrared absorption signature of polar stratospheric clouds so that the phase and composition of those clouds can be determined. The instrument has flown successfully on the DC-8 on three polar programs since 1987. Details of the column retrieval method may be found in Mankin and Coffey, 1989; and Coffey et al., 1989. Stratospheric gases that have been retrieved during the airborne field programs are listed in Table 1.

That kind of thing can be used to make weather maps and other things that require sets of still images made over a period of time.
Now we understand what the name of the topic. He wants to order sets of images to be subtracted in an algorithm that yields  the difference.

[Squashman]

Sounds like you will need a Rocket Surgeon to do this.
You would assume that someone else has already scripted something like this. Have you asked around in your community field of work or even asked your NASA contacts.

[Geek-9pm]

The OP has to use
gdal_calculate
to subtract images. Possibly written in Python.
See here:
http://stackoverflow.com/questions/13439357/extract-point-from-raster-in-gdal

It is used in a batch script on
http:// Earthwithsun.com
It is beyond my training to explain how it works.
But maybe in a few years I will catch on.

[Geek-9pm]

For the benefit of all reading this thread.
The question about subtraction of images is also found here:
http://earthwithsun.com/questions/847009/batch-process-subtraction-of-images

I have numerous images from a directory that needs to be subtracted to another image(another directory) and I don't know how to do it automatically in batch process. Any idea?
The following are the goals for the batch file:
    Create an output directory
    Create a proc_path directory for the processed files only
    Subtract image from the directory to another image from another directort
...

And yes, they use a batch file to  order the files and make backups.
The is a library of stuff in GDAL.
GDAL is GDAL: GDAL - Geospatial Data Abstraction Library
http://www.gdal.org/

GDAL is a translator library for raster and vector geospatial data formats that is released under an X/MIT style Open Source license by the Open Source Geospatial Foundation. As a library, it presents a single raster abstract data model and vector abstract data model to the calling application for all supported formats. It also comes with a variety of useful commandline utilities for data translation and processing. The NEWS page describes the September 2014 GDAL/OGR 1.11.1 release.

Traditionally GDAL used to design the raster part of the library, and OGR the vector part for Simple Features. Starting with GDAL 2.0, both sides have been more tightly integrated. You can still refer to the documentation of GDAL 1.X if needed.

Master: http://www.gdal.org
Download: ftp at remotesensing.org, http at download.osgeo.org

This use of two or more images to create a differential images is called subtraction. One might be tempted to call it a product or the result of a multiplication. But in Fourier vocabulary is is called subtraction. I think.

Please, anybody with a degree in differential calculus step in and save the thread.
I have to go back to bed and take some pills. 

[Geek-9pm]

I have to give up on this. I do not have any experience with the GDAL library. The GDAL library is a collection of functions that can be used to process images received from satellites. The images are in a form of digital information containing vector values along with Raster information. This type of mathematics is associated with a special set of calculus stuff.
The key issue is the original poster of this thread has a program that he can use from a command line, but cannot be used in a batch file. From what little research. I've been able to do on this, this seems to be typical of some script languages. One example could be Python. When using Python. One has a command line interface in Python that accepts Python programs. However, Python does not have a DOS shell that executes DOS programs for Windows. Well, at least not to my knowledge. The singular issue would be if one was using VBA, visual basic for applications, another script language that has power similar to that of Python and other interpretive languages.
My own limited experience is to use some kind of a hack. When I need to combine a DOS batch file with some kind of interpreter that does not have a usable DOS shell a hack usually means creating files that contain the parameters needed by the interpreter and having the interpreter get its parameters from the file and then passing it on to the function.
But I am not able to help the original poster with this. I don't know which is the script language being used here. I think it is Python. Buffett is Python, it would make more sense to the in entire job inside the Python rather than shelling out to the Windows DOS command.
So I'm kind of stuck here and I like anybody to step in and try to sort this out.
Another problem here is the original poster is making a big fuss about dating system. It should not be necessary to have to resort to copulating dates of leap years and ordinary years and that kind of stuff. Most programming languages have some way to deal with that without using a lot of script commands trying to figure out which days are leap you days in which years early. Pearson that kind of stuff.
In other words, this treaty should not be a batch program. The program that deals with image transformation should be written in that language, and not have to resort to DOS to create a list of objects to be processed. I have no idea why someone would have told him to do it in batch instead of doing it in the language of the interpreter.

About GDAL
http://en.wikipedia.org/wiki/GDAL

gdal_calculate
Extract Point From Raster in GDAL
It is very annoying when people do not say what language the are using.
And even when you know which, there is the question of implementation and library.
version.  Here is part of what I said to the OP in a PM.

Presently you can use
  the gdal_calculate command directly on command line
... _right?
My research indicates that the command itself is a script and  can not be executed in a batch file. Where is you get the bath file?

In windows a limited number of external commands are recognized.  Such fhave to be in tghe directory or path and must be one of:
.COM
.EXE
.BAT
.VBS
.XML
Python is not a standard feature in Widows.
Sorry I can not solve this. My mental posers are not a sharp as the used to be.

[_unknown_]

Someone answered this one. Haven't tried this yet because I don't know in which part of this script should I add the gdal_calculate command. Any idea?

Code: [Select]

@ECHO OFF >NUL
@SETLOCAL enableextensions enabledelayedexpansion

set "in_path=E:\Proc\Mer"
set "out_path=E:\Proc\Abcde"
set "two_path=E:\Proc\Me"
set "proc_path=E:\Proc\Proc_Mer_Fi"
md %in_path%   2> NUL
md %out_path%  2> NUL
md %two_path%  2> NUL
md %proc_path% 2> NUL

pushd "%in_path%\"

set "yearDay="
set "fileName="

::Get list of all YearDays in input path
set "yearDayList=x"
for /F "tokens=* delims=" %%Q in ('dir /B ????????*.tif') do (
  set "fileName=%%Q"
  set "yearDay=!fileName:~1,7!"
  Call :ItemToList !yearDayList! !yearDay!
)
@set yearDayList=%yearDayList:x= %
@echo yearDayList=%yearDayList%

::Process all *.tif files in input path day by day
for %%p in ( %yearDayList%) Do (
  set "yearDay=%%p"
  @echo .
  set /A "julYr=!yearDay:~0,4!"
  set "julDayS=!yearDay:~4,3!"
  set "month="
  set "monthDay="
  rem avoid octal conversions
  if "!julDayS:~0,2!"=="00" (
    set /A "julDn=!julDayS:~2,1!"
  ) else (
    if "!julDayS:~0,1!"=="0" (
      set /A "julDn=!julDayS:~1,2!"
    ) else (
      set /A "julDn=!julDayS!"
    )
  )
  call :months !julYr! !julDn!
  set "mDay2=0!monthDay!"
  set "mDay2=!mDay2:~-2!"
  @echo p^:  yearDay !yearDay!   yyyy-mmm-dd !julYr!-!month!-!mDay2!
  @rem p^:  yearDay !yearDay!   yyyy-mmm-d !julYr!-!month!-!monthDay!

  rem Process all *.tif files of the same YearDay in input path
  set "fileList="
  for /F "tokens=* delims=" %%G in ('dir /B "?%%p*.tif"') do (
    set "fileName=%%G"
    set "fileList=!fileList!!fileName! "
    @echo G^: !fileName!
  )
  REM   @echo fileList=!fileList!
  REM   @echo Move processed files ^(one day^) to a different directory
  REM   for %%a in (!fileList!) do (
  REM     @echo move %%a "%proc_path%\"
  REM   )
)
popd
goto :allcommon

:allcommon
@ENDLOCAL
@goto :eof

:ItemToList
rem yearDayList yearDay
SETLOCAL enableextensions enabledelayedexpansion
  set "myYDList=%1"
  set "myYearDay=%2"
  set "myYList="
  call :myset "myYList=%%myYDList:%myYearDay%=%%"
  if "%myYList%"=="%myYDList%" set "myYDList=%myYDList%x%myYearDay%"
ENDLOCAL & set "yearDayList=%myYDList%"
exit /B

:months
rem %1=julYr
rem %2=julDn
@SETLOCAL enableextensions enabledelayedexpansion
set "mymonth=XXX"
set /a "dayom=%2"
set "allmonths=Xjanfebmaraprmayjunjulaugsepoctnovdec"
set /a "ii=1"
rem leap year test makes use of integer only arithmetic
set /A "yearModi=(%1/4)*4"
If "%1"=="%yearModi%" (
  rem leap year
  set "daycounts=32 61 92 122 153 183 214 245 275 306 336 367"
) Else (
  rem non-leap year
  set "daycounts=32 60 91 121 152 182 213 244 274 305 335 366"
)
For %%G in (%daycounts%) do (
  if %2 lss %%G (
    call :myset "mymonth=%%allmonths:~!ii!,3%%"
    rem set /a "dayom+=1"
    goto :commmonths
  )
  set /a ii=!ii!+3
  set /a "dayom=%2-%%G+1"
)
:commmonths
ENDLOCAL & (set "month=%mymonth%"
set "monthDay=%dayom%")
exit /B

:myset
rem procedure to set indirect variable replace/substring
rem i.e. dynamic %StrToFind% instead of literal StrToFind
rem common: set "varNew=%varOld:StrToFind=NewStr%"
rem call :myset "varNew=%%varOld:%varToFind%=NewStr%%"
rem call :myset "varNew=%%varOld:!varToFind!=NewStr%%"
rem applicable to %NewStr% as well
rem i.e. dynamic %CharsToSkip% instead of literal CharsToSkip
rem common: set "varNew=%varOld:~CharsToSkip,chars_to_keep%"
rem call :myset "varNew=%%varOld:~%CharsToSkip%,chars_to_keep%%"
rem call :myset "varNew=%%varOld:~!CharsToSkip!,chars_to_keep%%"
rem applicable to %chars_to_keep% as well
set %1
exit /B

[_unknown_]

Still finding ways to solve this. It's giving me a hard time.