Subtract images in batch process(leap year and regular year)

[Squashman]

Sometimes you just need to bite the bullet and hire a contractor to program what you need. We contract from a resource company for a lot of the projects we work on where I work.

[foxidrive]

Is the first post still relevant to the task, or has the task changed?
I had another look and tried to understand what you want to do.

In essence it seems that you have a day number in a year, and you want to figure out which month it corresponds to, for the year in question.
After doing that you have some other files - that have a month in the filename - and you want to run a command line using each file and the correct-month-file.

Is that the actual task, or has it morphed in the following pages?

[foxidrive]

Sometimes you just need to bite the bullet and hire a contractor to program what you need.

If the task can be explained simply then it may be doable.  I found the first post quite hard to follow.

[patio]

The tasks always change here in the DOS section...you should know that foxidrive....

[foxidrive]

The tasks always change here in the DOS section...you should know that foxidrive....

Hehe  It was rather optimistic of me, wasn't it?!

[_unknown_]

Is the first post still relevant to the task, or has the task changed?
I had another look and tried to understand what you want to do.

In essence it seems that you have a day number in a year, and you want to figure out which month it corresponds to, for the year in question.
After doing that you have some other files - that have a month in the filename - and you want to run a command line using each file and the correct-month-file.

Is that the actual task, or has it morphed in the following pages?

Thank you foxidrive.
The task was never changed. I'll explain it again with my very best.

1. The main task/goal is to subtract(as in mathematics; to take, deduct, take from, take out etc.) two images using a gdal_calculate command and perform this in batch process.

• gdal_calculate Purpose: Perform simple tiled raster calculations (AKA "map algebra")from the commandline.
• raster - A raster image, also called a bitmap, is a way to represent digital images.The image is represented in a series of bits of information that translate into pixels on the screen. These pixels form points of color that create an overall finished image. When a raster image is created, the image on the screen is converted into pixels. Each pixel is assigned a specific value that determines its color. This format uses the red, green, blue (RGB) color system. An RGB value of 0,0,0 would be black, and the values go all the way through to 256 for each color, allowing the expression of a wide range of values. In photographs with subtle shading, this can be extremely valuable.
  

2. There should be an input directory for the input files, an output directory for the output files, second directory for the files that will subtracted from the input files and the processed directory for the files that we're only processed. The input directory and the second directory has numerous files already so this should be set only while the output directory and the processed directory should be made.

3. The template of the file names from the input directory is something like this: C2011060.A1_ABC.ABCD.tif where C represents the dataname, 2011 represents the year of the file either its a leap year or regular year, 060 represents the day/julian date deepending on the year whether its a leap year or regular year and A1_ABC.ABCD.tif is the extension name.

4. The template of the file names from the second directory is something like this: sample_file_month.tif where file represents the dataname and the month which represents the month of the file.


The following are the LEAP YEARS: 2000, 2004, 2008, 2012, 2016, 2020 and so on...
The following are the day/julian dates;
January - 1-31
February - 32-60   
March - 61-91   
April - 92-121     
May - 122-152 
June - 153-182
July - 183-213   
August - 214-244   
September - 245-274
October - 275-305
November - 306-335
December - 336-366

The following are the REGULAR YEARS:  2001-2003, 2005-2007, 2009-2011, 2013-2015 and so on....
The following are the day/julian dates;
January - 1-31                         
February - 32-59
March - 60-90   
April - 91-120   
May - 121-151 
June - 152-181 
July - 182-212
August - 213-243   
September - 244-273
October - 274-304
November - 305-334
December - 335-365

For the subtraction/calculation process:

Input Directory: C2011060.A1_ABC.ABCD.tif

If the year in the file name falls on 2003 this means that these file is a regular year. If the julian date in the file name is 001, 002..031 these means that the file is a regular year for the month of january.
If the year in the file name falls on 2005 this means that these file is a regular year. If the julian date in the file name is 032, 033..59 these means that the file is a regular year for the month of february.
If the year in the file name falls on 2011 this means that these file is a regular year. If the julian date in the file name is 121, 122..151 these means that the file is a regular year for the month of may.
If the year in the file name falls on 2014 this means that these file is a regular year. If the julian date in the file name is 335, 336..365 these means that the file is a regular year for the month of december.

AND SO ON. . . . . . .

If the year in the file name falls on 2000 this means that these file is a leap year. If the julian date in the file name is 001, 002..031 these means that the file is a leap year for the month of january.
If the year in the file name falls on 2004 this means that these file is a leap year. If the julian date in the file name is 032, 033..060 these means that the file is a leap year for the month of february.
If the year in the file name falls on 2008 this means that these file is a leap year. If the julian date in the file name is 122, 123..152 these means that the file is a leap year for the month of february.
If the year in the file name falls on 2012 this means that these file is a leap year. If the julian date in the file name is 336, 337..366 these means that the file is a leap year for the month of march.

AND SO ON. . . .  . .

Second Directory: sample_file_month.tif

sample_file_jan.tif
sample_file_feb.tif
sample_file_mar.tif
sample_file_apr.tif
sample_file_may.tif
sample_file_jun.tif
sample_file_jul.tif
sample_file_aug.tif
sample_file_sep.tif
sample_file_oct.tif
sample_file_nov.tif
sample_file_dec.tif


If the day in the file name(from input directory) is from 001-031 which belongs to january(leap year and regular year) this will be subtracted(gdal_calculate command) to the sample_file_jan.tif(from second directory).
If the day in the file name(from input directory) is from 032-060 which belongs to february(leap year) this will be subtracted(gdal_calculate command) to the sample_file_feb.tif(from second directory).
If the day in the file name(from input directory) is from 061-091 which belongs to march(leap year) this will be subtracted(gdal_calculate command) to the sample_file_mar.tif(from second directory).
If the day in the file name(from input directory) is from 092-121 which belongs to apri(leap year) this will be subtracted(gdal_calculate command) to the sample_file_apr.tif(from second directory).
If the day in the file name(from input directory) is from 122-152 which belongs to may(leap year) this will be subtracted(gdal_calculate command) to the sample_file_may.tif(from second directory).
AND SO ON. . . . . .

If the day in the file name(from input directory) is from 032-059 which belongs to february(regular year) this will be subtracted(gdal_calculate command) to the sample_file_feb.tif(from second directory).
If the day in the file name(from input directory) is from 060-090 which belongs to march(regular year) this will be subtracted(gdal_calculate command) to the sample_file_mar.tif(from second directory).
If the day in the file name(from input directory) is from 091-120 which belongs to april(regular year) this will be subtracted(gdal_calculate command) to the sample_file_apr.tif(from second directory).
If the day in the file name(from input directory) is from 121-151 which belongs to may(regular year) this will be subtracted(gdal_calculate command) to the sample_file_may.tif(from second directory).

AND SO ON. . . . . .

The result/output from this calculation will be saved to the output directory and those processed files only will be saved to the processed files directory.

Sorry for the longer explanation. I hope I explained it well.

[foxidrive]

Thank you foxidrive.
The task was never changed. I'll explain it again with my very best.

I'm curious why you wrote all that and made no comment on my synopsis?

If you can follow what I wrote in a couple of sentences - was it accurate?

[Geek-9pm]

For anybody following this, here is some relevant information.
The OP (original poster)  is using data that is dated by day of year. He wants  to hare a conversion to conventional year, m month and day notation.
This table shows how the conversion is done.
http://disc.gsfc.nasa.gov/julian_calendar.shtml

From PMs exchanged, then OP indicated he may be working in Oceanography with NASA images.

[_unknown_]

I'm curious why you wrote all that and made no comment on my synopsis?

If you can follow what I wrote in a couple of sentences - was it accurate?

I can't follow. Can you explain it again if its okay? Sorry.

For anybody following this, here is some relevant information.
The OP (original poster)  is using data that is dated by day of year. He wants  to hare a conversion to conventional year, m month and day notation.
This table shows how the conversion is done.
http://disc.gsfc.nasa.gov/julian_calendar.shtml

From PMs exchanged, then OP indicated he may be working in Oceanography with NASA images.

Thank you Geek-9pm for this info.

[foxidrive]

I can't follow. Can you explain it again if its okay? Sorry.

It is kinder to describe which part was hard to follow and which parts you did follow. 

In essence it seems that you have a day number in a year, and you want to figure out which month it corresponds to, for the year in question.
After doing that you have some other files - that have a month in the filename - and you want to run a command line using each file and the correct-month-file.

What this means is that:

• You have files, and in the name they have a section which shows the day number in a year.
• You would like to determine the month that the day fell on.
• Then you have another 12 files and each one has the name of a month somewhere in the filename.
• The ultimate task is to match up each file with one of the 12 files - and run the command you showed using the correct pair of files.

Does this describe the task properly?

[_unknown_]

It is kinder to describe which part was hard to follow and which parts you did follow. 

What this means is that:

• You have files, and in the name they have a section which shows the day number in a year.
• You would like to determine the month that the day fell on.
• Then you have another 12 files and each one has the name of a month somewhere in the filename.
• The ultimate task is to match up each file with one of the 12 files - and run the command you showed using the correct pair of files.

Does this describe the task properly?

Yes exactly. But I'm a little confused with this:

• You would like to determine the month that the day fell on.

Do you mean determine the month that the files fell on whether they fell on a month from leap year or regular year?
Can you explain it a little more. Sorry again.

[foxidrive]

Do you mean determine the month that the files fell on whether they fell on a month from leap year or regular year?

Your files have the portion that shows the day number of the year
but I don't follow your need to determine a leap year:

Do you want to process files that only fall on a leap year, or perhaps process files that do not fall on a leap year?
Or do you want to process all years?

[_unknown_]

Your files have the portion that shows the day number of the year
but I don't follow your need to determine a leap year:

Do you want to process files that only fall on a leap year, or perhaps process files that do not fall on a leap year?
Or do you want to process all years?

All the files should be process depending on the year day of the file name whether it falls on leap year or regular year. If the day of the file name corresponds to the day from the month where it falls regardless whether its a leap year or regular year then process the files.  Maybe the leap year or regular year here will be discarded. Sorry I don't know how to explained it well.

[foxidrive]

All the files should be process depending on the year day of the file name whether it falls on leap year or regular year. If the day of the file name corresponds to the day from the month where it falls regardless whether its a leap year or regular year then process the files.  Maybe the leap year or regular year here will be discarded. Sorry I don't know how to explained it well.

Just to clarify - for each file you want to determine the correct month in that year, using the day of year.

A high level computer language will automatically handle leap years and non leap years - and there are batch script that can also handle this.

[Squashman]

A high level computer language will automatically handle leap years and non leap years - and there are batch script that can also handle this.

I think you are right.