xan/awips2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Issue #2147 Changes to allow option hour field in category tag and change to Retentions end time.

Browse source 
Change-Id: I7f878a0e71c0989e41aaf943b1520c93f45f1859

Former-commit-id: 2ba7117d13 [formerly a76fc08e93] [formerly 2ba7117d13 [formerly a76fc08e93] [formerly e65b8a9f6e [formerly 6a6159e07eddbac712e4696023c464d58ad250f7]]]
Former-commit-id: e65b8a9f6e
Former-commit-id: 34ea90f18a [formerly 71737f069d]
Former-commit-id: 0d86e00182
This commit is contained in:
Roger Ferrel 2013-10-01 15:59:33 -05:00
parent 541ff7c791
commit efde4f9297
4 changed files with 40 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
cave/com.raytheon.uf.viz.archive/src/com/raytheon/uf/viz/archive/ui/ArchiveRetentionDlg.java
View file

@ -37,6 +37,7 @@ import org.eclipse.swt.widgets.Spinner;
import com.raytheon.uf.common.archive.config.ArchiveConstants.Type;
import com.raytheon.uf.common.archive.config.DisplayData;
import com.raytheon.uf.common.time.util.TimeUtil;
/**
* Archive retention dialog.
@ -53,6 +54,7 @@ import com.raytheon.uf.common.archive.config.DisplayData;
* Jul 24, 2013 #2220 rferrel Add recompute size button.
* Jul 24, 2013 #2221 rferrel Changes for select configuration.
* Aug 26, 2013 #2225 rferrel Make dialog perspective independent.
* Oct 01, 2013 #2147 rferrel Change getEnd() to pick up files with future time stamps.
*
* </pre>
*
@ -305,7 +307,10 @@ public class ArchiveRetentionDlg extends AbstractArchiveDlg {
@Override
protected Calendar getEnd() {
// display all elements so no end bound
return null;
Calendar endCal = TimeUtil.newCalendar();
// Back off an hour so latter rounding doesn't cause overflow.
endCal.setTimeInMillis(Long.MAX_VALUE - TimeUtil.MILLIS_PER_HOUR);
return endCal;
}
/*

17
edexOsgi/com.raytheon.uf.common.archive/src/com/raytheon/uf/common/archive/config/CategoryDataSet.java
View file

@ -42,6 +42,7 @@ import com.raytheon.uf.common.time.util.TimeUtil;
* Date Ticket# Engineer Description
* ------------ ---------- ----------- --------------------------
* Aug 6, 2013 #2224 rferrel Initial creation
* Oct 02, 2013 #2147 rferrel Allow Date to ignore hour in time stamp.
*
* </pre>
*
@ -160,7 +161,12 @@ public class CategoryDataSet {
if (timeIndices.length > 0) {
String[] indexValues = getDateGroupIndices().split("\\s*,\\s*");
for (int index = 0; index < timeIndices.length; ++index) {
timeIndices[index] = Integer.parseInt(indexValues[index]);
if (indexValues.length > index) {
timeIndices[index] = Integer
.parseInt(indexValues[index]);
} else {
timeIndices[index] = -1;
}
}
}
}
@ -233,8 +239,15 @@ public class CategoryDataSet {
.group(timeIndices[CategoryDataSet.MONTH_INDEX])) - 1;
int day = Integer.parseInt(matcher
.group(timeIndices[CategoryDataSet.DAY_INDEX]));
int hour = Integer.parseInt(matcher
// Default to last hour of the day.
int hour = 23;
if (timeIndices[CategoryDataSet.HOUR_INDEX] >= 0) {
hour = Integer.parseInt(matcher
.group(timeIndices[CategoryDataSet.HOUR_INDEX]));
}
fileCal.set(year, month, day, hour, 0, 0);
fileTime = fileCal.getTimeInMillis();
break;

18
edexOsgi/com.raytheon.uf.edex.archive/utility/common_static/base/archiver/purger/PROCESSED_DATA.xml
View file

@ -24,7 +24,8 @@
* Date Ticket# Engineer Description
* ============ ========== =========== ==========================
* Jun 20, 2013 1966 rferrel Initial creation
* Aug 05, 2012 2224 rferrel Changes to add dataSet tags.
* Aug 05, 2013 2224 rferrel Changes to add dataSet tags.
* Oct 01, 2013 2147 rfrrel Date time stamp no longer requires an hour field.
*
* @author rferrel
* @version 1.0
@ -69,7 +70,8 @@
EpochMS - The time stamp has one group in the patterns which is the epoch time in milliseconds.
File - No group is used to get the time stamp. Instead use the files date of last modification.
<dateGroupIndicies> - Required tag when <timeType> has any value but File.
Date - A comma separated list of 4 numbers which are in order the index for year, month, day and hour.
Date - A comma separated list of 3 or 4 numbers which are in order the index for year, month, day and hour.
When only 3 numbers the hour is value is 23.
EpochSec - A number which is the index for the epoch in seconds.
EpochMS - A number which is the index for the epoch in milliseconds.
File - Not needed since no group is used to get the time stamp.
@ -143,12 +145,16 @@
<name>GFE</name>
<extRetentionHours>168</extRetentionHours>
<dataSet>
<dirPattern>(gfe)/(.*)/.*</dirPattern>
<dirPattern>(gfe)/(.*)</dirPattern>
<filePattern>.*_(\d{4})(\d{2})(\d{2})_(\d{2}).*</filePattern>
<displayLabel>{2}</displayLabel>
<dirPattern>gfe/(.*)/(.*)/(\d{4})_(\d{2})_(\d{2})_(\d{2})\d{2}</dirPattern>
<displayLabel>{1} - {2}</displayLabel>
<dateGroupIndices>3,4,5,6</dateGroupIndices>
</dataSet>
<dataSet>
<dirPattern>gfe/(.*)/(Fcst|Official)</dirPattern>
<filePattern>.*_(\d{4})(\d{2})(\d{2})_.*</filePattern>
<displayLabel>{1} - {2}</displayLabel>
<dateGroupIndices>3,4,5</dateGroupIndices>
</dataSet>
</category>
<category>
<name>Local</name>

16
edexOsgi/com.raytheon.uf.edex.archive/utility/common_static/base/archiver/purger/RAW_DATA.xml
View file

@ -24,7 +24,8 @@
* Date Ticket# Engineer Description
* ============ ========== =========== ==========================
* Jun 20, 2013 1966 rferrel Initial creation
* Aug 05, 2012 2224 rferrel Changes to add dataSet tags.
* Aug 05, 2013 2224 rferrel Changes to add dataSet tags.
* Oct 01, 2013 2147 rfrrel Date time stamp no longer requires an hour field.
*
* @author rferrel
* @version 1.0
@ -45,7 +46,7 @@
<dataSet> - Required to have a least one. Each one contains a set of tags explained below.
<selectedDisplayNames> - A directory matching <dirPattern>. These are selected directories from the Retention GUI. The purger
will used the category's <extRetentionHours> instead of the Arhivie's <minRetentionHours>.
An optional tag; may have more then one.(NOTE these are set internally when a selection configuration
An optional tag; may have more then one. (NOTE these are set internally when a selection configuration
file is loaded. They should not appear in the configuration file.)
The <dataSet> contains ? types of tags:
<dirPattern> - Required to have at least one. A regex pattern for finding directories for this category.
@ -69,7 +70,8 @@
EpochMS - The time stamp has one group in the patterns which is the epoch time in milliseconds.
File - No group is used to get the time stamp. Instead use the files date of last modification.
<dateGroupIndicies> - Required tag when <timeType> has any value but File.
Date - A comma separated list of 4 numbers which are in order the index for year, month, day and hour.
Date - A comma separated list of 3 or 4 numbers which are in order the index for year, month, day and hour.
When only 3 numbers the hour is value is 23.
EpochSec - A number which is the index for the epoch in seconds.
EpochMS - A number which is the index for the epoch in milliseconds.
File - Not needed since no group is used to get the time stamp.
@ -107,19 +109,13 @@
<name>Observation</name>
<extRetentionHours>168</extRetentionHours>
<dataSet>
<dirPattern>(airep|airmet|taf)</dirPattern>
<dirPattern>(acars|airep|airmet|taf)</dirPattern>
<dirPattern>(bufrsigwx|sfcobs)/.*</dirPattern>
<displayLabel>{1}</displayLabel>
<timeType>Date</timeType>
<dateGroupIndices>2,3,4,5</dateGroupIndices>
<filePattern>[^/]*-(\d{4})-(\d{2})-(\d{2})-(\d{2})\..*</filePattern>
</dataSet>
<dataSet>
<dirPattern>(acars)/(.*)</dirPattern>
<displayLabel>{1}</displayLabel>
<filePattern>[^/]*-(\d{4})-(\d{2})-(\d{2})-(\d{2})\..*</filePattern>
finish this
</dataSet>
</category>
The first <dirPattern> looks for files matching the <filePattern> in the directories acars, airep, airmet or taf.
The second <dirPattern> expects to find the files in subdirectories of bufrsigwx or sfcobs such as bufrsigwx/SWH.