-Changed some of the default values for grids so it will process MRMS better/faster (ingestGrib.sh)
-Changed the storing of CONUS MRMS to be concatenated instead of individual files to process faster (pqact.conf and pqact.mrms)
-Added the "lma" plugin to be excluded in the pluginExclude mode (which both the ingestGoesR and ingestRadar calls)
-Added the "lma" plugin to be excluded in the pluginModelSoundingExclude mode (which ingestGrids calls)
-Removed the "lma" plugin directly from the ingestRadar mode since it's referenced in the pluginExclude
-On the Default EDEX machine, the lma plugin will start up
-Added/moved from some spi files from configured or site to base
-Removed some spi files from cave/etc/basemaps to utility tree so we could update on the EDEX side
-Had to add back in the com.raytheon.uf.edex.plugin.loctables plugin so it knew what to do with some of the files in /awips2/edex/data/ndm
-Added files from the awips-ndm svn repo to be built by the awips2-edex rpm and get installed to /awips2/edex/data/ndm so that EDEX picks up the files and updates the databases and spi files.
buildEnvironment.sh - changed the release from 1 to 2
build/setup.sh - added code to call awips_spalshscreen_updater jar and update splash.bmp to current build and release
- remove copy of rpms from robin to fserv
feature.xml (edex dataplugins feature) - remove reference to sports glmdecoder which is depricated
wrapper.conf - this is the file that has been dumping heap logs to /data/fxa so I changed the path to /awips2/data/fxa so it didn't fill up the main / mount
ldmd.conf - added GLMF entry (this is for the ISatSS gridded GLM data - for local use only, but won't hurt to have in)
- commented out GLM data from Eric Bruning
pqact.conf - Commented out Eric Bruning GLM entry since it's depricated
- Added new entry for ISatSS gridded GLM data
pqact.goesr - updated to match the pqact.conf
- updated comments to relfect which products were available
parameters.xml - changed all references of 1/s to /s
- modified the build.properties files for the common.aviation and common.dataplugin.satellite jars
Changes to re-enable the proper decoding of modelsounding data
- Two changes were made wrt this:
- Rolled back changes from our big merge from MJ's computer, which included some raytheon commits in 2018 to both the ModelSoundingDataAdapter.java and SoundingStations.java classes
- Then added two small new changes to the ModelSoundingDataAdapter to support the changes to GFS Bufr files earlier this year
The previous check-in of this file, the units were updated to 1/s which was causing issues in CAVE displaying. For the native pressure heights (000, 850, 700, 500, 250 mb) the units were wrong and it wasn't displaying any data, but the interpolated heights were showing up correct. I also noticed this when running python-awips and getting the "unit" of AV as "count" instead of /s.
On systems where this is already installed, I had to manually update the parameter database table:
update parameter set unit = '/s' where abbreviation like 'AV';
-Added HRRR-Smoke domain to the gribModels_NCEP-7.xml
-Added new HRRR-Smoke products to the grib tables
4.2.0.4.table
4.2.0.20.table
-Added new entry in ldmd.conf and pqact.conf to request and store data
-New LDM script to process the HRRR Smoke products (changeHrrrSmokeProcessId.sh)
-Added new menu entry in the models menu for the new products
-Added new styleRules for the new products
-Added new bundle files for the new products
-Added new colormaps for the new products
-Updated product name for Radar Accumulation Quality Index
-Updated pqact entries to concatenate the OCONUS MRMS products, but not CONUS
-Updated checkFile.pl script to check for files that haven't been touched in 2 minutes instead of 5
-Updated cronjob that runs the checkFile.pl to run every 2 minutes instead of 5
Originally I had copied in everything from the vlab_20.2.3 branch, but was getting "Generated Warning is null" which I wasn't sure why. I then copied back in the original templates that we were using on edex1, then copied in the following impact based templates:
-impactSevereThunderstormWarning*
-impactSevereWeatherStatement.*
-impactSignificantWeatherAdvisory.*
-impactTornadoWarning.*
And updated the config.xml file to use these impact based templates.
NOTE: I did not include the non-convective flash flood warning because that doesn't work since we don't have the dam information.
rpms/awips2.upc/Installer.ldm/patch/etc/pqact.conf
rpms/awips2.upc/Installer.ldm/patch/etc/pqact.grids
-In LDM store GFS1.0 Global data from NOAAPort only - not CONDUIT
edexOsgi/com.raytheon.uf.edex.plugin.grid/utility/common_static/base/bundles/grid/Precip-GFS.xml
-Removed colortable reference here because it's specified in the styleRules
edexOsgi/com.raytheon.edex.plugin.grib/utility/common_static/base/grib/postProcessModels/postProcessedModels.xml
edexOsgi/com.raytheon.edex.plugin.grib/utility/common_static/base/grib/postProcessModels/precipitationAccumulation.xml
-Removed typo of a "mapScale" tag
-Updated some comments
build/repos - had to add awips2-cimss to the list of repos to look at
build/rsync.cave - had to add the awips2-cimss directories to sync into the docker (common, edex, features, viz)
build/rsync.dirs - had to add the awips2-cimss directories to sync into the docker (common, edex, features, viz)
cave/build/features.txt - had to add the viz feature to this file
cave/build/p2-build.xml - had to add the viz feature to this file
cave/com.raytheon.viz.feature.awips.developer/feature.xml - had to add the viz feature to this file
cave/com.raytheon.viz.product.awips/awips.product - had to add the viz feature to this file
edexOsgi/build.edex/build.xml - had to add the edex feature to this file
NOTE: I tried adding it to the edexOsgi/com.raytheon.edex.feature.uframe/feature.xml but that ended up not working, so I removed it
edexOsgi/build.edex/edex/common.properties - had to create the awips2-cimss repo, add it to the optional repos, and then add the previx edu.wisc
gridImageryStyleRules
modified the lightning probability value (removed the *100)
added new styleRule for AzShear products
updated VIL Denstiy entry
added Seamless HSR Height
Update FLASH FFG to label as "", the unit was set to % previously which was causing wrong scaling
Corrected CONUSMergedRHV parameter
Corrected colortable for CONUSMergedZDR
updated styleRules for PrecipRate
Added new mrmsAzShear colormap
Added new VIL Density colormap
Added new Seamless HSR Height colormap
Updated mrmsSPRDP colormap to black out 0
mrmsProducts.xml
Added VIL Density and VIL 120 min and 1440 min swaths to menu
Remove 5km Composite Reflectivity and RALA from menu
Removed Cube
Removed Precip-->QPE Gauge Only, QPE Radar w/ Gauge Bias Correction, QPE Mountain Mapper
Added Gauge Influence Index Pass 1 and 2
Added QPE Radar QPI
Added Seamless HSR Height
FLASH - remove QPF Models from menu
Added in quite a few new reflectivity products
Reorganized menus
Updated/Reoganized AK, CA, HI, and GU product menus
Caribbean - FALSH QPE models are on 500m domain instead of 1km
Had to add new domains for Hawaii and Guam
MRMS-500m-GUAM2.xml 10042 : 17.995 | 500.005004
MRMS-500m-HAWAII2.xml 10022 : 25.9975| 196.0025
MRMS-500m-HAWAII3.xml 10023 : 25.995|196.005004
Original location of menus, bundles, scales, volumebrowser awips2/cave/com.raytheon.viz.volumebrowser/localization
New location edexOsgi/com.raytheon.uf.edex.plugin.grid/utility/common_static/base
There were quite a few npp plugins, but I moved them to their respective edex plugin. Specifically:
+ awips2/cave/com.raytheon.uf.viz.npp/localization --> edexOsgi/com.raytheon.uf.edex.plugin.npp/utility/common_static/base
+ awips2/cave/com.raytheon.uf.viz.npp.nucaps/localization --> edexOsgi/com.raytheon.uf.edex.plugin.npp.nucaps/utility/common_static/base
+ awips2/cave/com.raytheon.uf.viz.npp.viirs/localization --> edexOsgi/com.raytheon.uf.edex.plugin.npp.viirs/utility/common_static/base
Localization information has been moved from /awips2/cave/etc to /awips2/edex/data/utility/common_static/base
-Update ldmd.conf to grab VIIRS data:
REQUEST NOTHER "^TI[A-Q]... KNES"
-Update pqact.conf VIIRS entry TI[A-Q]
-Updated viirsImagery.xml to include SNPP and NOAA-20 (JPSS) for Alaska, CONUS, Pacific, Puerto Rico domains
-Added a "region" tag in the bundle files
-Added submenu in main satellite menu for S-NPP and NOAA-20
-Removed submenus for products that only had one menu entry
-Removed extra Alaska imagery channels that we aren't getting data for
While the composite imagery (mcidas) data being sent via UNIWISC includes the new GOES 16 and 17 imagery, the areaNames.xml and mcidas plugin is not set up to "read" and store the different sectors. Because all of the GOES 16 and 17 single channel data is available for each sector already it did not seem necessary at this time to also have the mcidas imagery. For simplicity the mcidas composite imagery for the new GOES was removed. The Global, Arctic and Antarctic imagery remain available here. I did have to comment out some of the data in the areaNames.xml so the sector (AREA####) could be stored correctly for global.
The legacy GOES West (GOES-15) submenu was removed as we are no longer receiving the data.
The legacy sounder imagery submenu was removed as we are no longer receiving the legacy data. GOES 16 derived products can be found in the GOES menus.
The legacy satellite winds got consolodated to only the ASACT winds as we are no longer receiving legacy satellite wind data. GOES 16/17 satellite derived winds can be found in the GOES menus.
Moved config references from viz plugin and common dataplugin (stored in /awips2/cave/etc) to edex plugins (stored in /awips2/edex/data/utility), which included moving some files from the awips2 repo to awips2-goesr. Having the configs on the EDEX side will allow for minor config updates without major rpm releases.
I also cleaned up the NWS version of the GOES Menus. Separated out each sector based on what products were available so the menus shouldn't have "extra" menu items that don't exist. I've also reduced the number of submenus so it's easier to load a product.
Summary of changes:
com.raytheon.viz.satellite
-Removed all references of satellite base/goesr/glm/himiwari bundle files (now in awips2-goesr/edexOsgi/com.raytheon.uf.edex.plugin.goesr)
-Removed extra copy of goesr menus (localization/menus/satellite/goesr/goesr)
-Removed main goesr/glm/himawari menus (now in awips2-goesr/edexOsgi/com.raytheon.uf.edex.plugin.goesr)
-Removed all base satellite menus (now in awips2/edexOsgi/com.raytheon.edex.plugin.satellite)
com.raytheon.uf.common.dataplugin.satellite
-Removed all satellite (base) colormaps references (now in awips2/edexOsgi/com.raytheon.edex.plugin.satellite)
-Removed all base satellite derived parameters references (now in awips2/edexOsgi/com.raytheon.edex.plugin.satellite)
-Removed all base satellite menu references (now in awips2/edexOsgi/com.raytheon.edex.plugin.satellite)
-Removed satelliteImageryStyleRules.xml (now in awips2/edexOsgi/com.raytheon.edex.plugin.satellite)
com.raytheon.edex.plugin.satellite
+Added base satellite menus from awips2/cave/com.raytheon.viz.satellite here
+Added base satellite menus from com.raytheon.uf.common.dataplugin.satellite here
+Added base satellite colormaps from com.raytheon.uf.common.dataplugin.satellite here
+Added base satellite derived parameters from com.raytheon.uf.common.dataplugin.satellite here
+Added satelliteImageryStyleRules.xml from com.raytheon.uf.common.dataplugin.satellite here
Modified: edexOsgi/com.raytheon.uf.edex.dataplugins.feature/feature.xml
included the following pirep edex/common features to feature.xml:
awips2-ncep/common/gov.noaa.nws.ncep.common.dataplugin.pirep
awips2-ncep/edex/gov.noaa.nws.ncep.edex.plugin.pirep
/awips2/edex/data/utility/common_static/base/grid/master_grib2_lookup.txt
Added the following lines at the end of the file:
// GFSFV3 transition from buckets
TP_T170L42A-NCEP TP_GFS
CP_T170L42A-NCEP CP_GFS
/awips2/edex/data/utility/common_static/base/grib/postProcessModels/postProcessedModels.xml
Commented out:
<!--Tiff updated the model name because of changes in the GFS-Fv3 #20200625-->
<!-- modelName>GFS20|AK-GFS22|GFS20-.*</modelName -->
Added:
<modelName>GFS229|GFS20|AK-GFS22|GFS20-.*</modelName>
/awips2/edex/data/utility/common_static/base/grib/postProcessModels/precipitationAccumulation.xml
Commented out:
<!-- Tiff added the new GFS-FV3 updates for precip accum # 20200625 -->
<!--
<accumulation>
<model>GFS20|AK-GFS22|GFS20-.*</model>
<create forecastPeriodHours="3" accumulationParam="TP3hr" minuendParam="TP6hr" subtrahendParam="TP3hr" />
<create forecastPeriodHours="3" accumulationParam="CP3hr" minuendParam="CP6hr" subtrahendParam="CP3hr" />
</accumulation>
-->
Added
<accumulation>
<model>GFS229|GFS20|AK-GFS22|GFS20-.*</model>
<create forecastPeriodHours="3" accumulationParam="TP3hr" minuendParam="TP-GFS" subtrahendParam="TP-GFS" />
<create forecastPeriodHours="6" accumulationParam="TP6hr" minuendParam="TP-GFS" subtrahendParam="TP-GFS" />
<create forecastPeriodHours="3" accumulationParam="CP3hr" minuendParam="CP-GFS" subtrahendParam="CP-GFS" />
<create forecastPeriodHours="6" accumulationParam="CP6hr" minuendParam="CP-GFS" subtrahendParam="CP-GFS" />
</accumulation>
common_static/base/menus/grid/index.xml
Just changed the menu for GFS20 and GFS to now point to allFamilies-GFS.xml
common_static/base/menus/grid/allFamilies-GFS.xml
This is a new file, referencing a new bundle VerticalVelocity-GFS.xml and Precip-GFS.xml
common_static/base/bundles/grid/VerticalVelocity-GFS.xml
This is a new file, referencing GVV instead of PVV
common_static/base/bundles/grid/Precip-GFS.xml
This is a new file, referencing TP-GFS instead of TP/TPrun and removed the colortable
/awips2/edex/data/utility/common_static/base/styleRules/gridImageryStyleRules.xml
Added TP-GFS and CP-GFS to the existing TP/CP styleRule
<styleRule>
<paramLevelMatches>
<parameter>tpHPCndfd</parameter>
<parameter>TP</parameter>
<parameter>TP6hr</parameter>
<parameter>TP120hr</parameter>
<parameter>TP168hr</parameter>
<parameter>CP</parameter>
<parameter>St-Pr</parameter>
<parameter>TP1hr</parameter>
<parameter>TP3hr</parameter>
<parameter>TP6hr</parameter>
<parameter>SPAcc</parameter>
<parameter>TP-GFS</parameter>
<parameter>CP-GFS</parameter>
</paramLevelMatches>
<imageStyle>
<!-- filterLow="true" -->
<displayUnits>in</displayUnits>
<range scale="LOG">
<minValue>0.005</minValue>
<maxValue>12</maxValue>
</range>
<defaultColormap>Radar/UPC One Hour Precip</defaultColormap>
<colorbarLabeling>
<values>0.01 0.03 0.1 0.3 1 3 10</values>
</colorbarLabeling>
</imageStyle>
</styleRule>
/awips2/edex/data/utility/common_static/base/styleRules/d2dContourStyleRules.xml
<styleRule>
<paramLevelMatches>
<parameter>St-Pr</parameter>
<parameter>CP</parameter>
<parameter>TP</parameter>
<parameter>TP1hr</parameter>
<parameter>TP3hr</parameter>
<parameter>TP6hr</parameter>
<parameter>TP-GFS</parameter>
<parameter>CP-GFS</parameter>
</paramLevelMatches>
<contourStyle>
<displayUnits>in</displayUnits>
<displayFlags>NoPlane</displayFlags>
<contourLabeling labelSpacing="4" minMaxLabelFormat="#0.00"
maxLabel=".">
<values>0.01 0.1 0.25 0.5 0.75 1 1.5 2 3 4 6 8 10</values>
</contourLabeling>
</contourStyle>
</styleRule>
README.md
Michael modified, and removed:
git clone https://github.com/Unidata/awips2-cimss.git
build/repos
Michael removed these repos that are needed to build the rpms:
awips2-cimss
awips2-collaboration
awips2-hazards
awips2-nasa
awips2-nws
awips2-ohd
awips2-swpc
Tiff added back in awips2-nws because it was still needed:
awips2-nws
build/rsync.cave
Michael removed the syncing of awips2-cimss, awips2-hazards, awips2-nasa, awips2-ohd, awips2-swpc directories
build/rsync.dirs
Michael removed the syncing of awips2-hazards, awips2-nasa, awips2-ohd, awips2-swpc, awips2-nws directories
Tiff re-added back in the awips2-nws directories?
cave/build/features.txt
Michael removed edu.wisc.ssec.cimss.viz.convectprob.feature
cave/build/p2-build.xml
Michael removed the cimss convectprob feature
<antcall target="p2.build.repo">
<param name="feature" value="edu.wisc.ssec.cimss.viz.convectprob.feature" />
</antcall>
cave/com.raytheon.viz.feature.awips.developer/feature.xml
Michael removed the cimss convectprob feature
cave/com.raytheon.viz.product.awips/awips.product
Michael removed the cimss convectprob feature
docs/dev/awips-development-environment.md
Michael removed the cimss repo
edexOsgi/build.edex/edex/common.properties
Michael updated the repos needed to build within eclipse
edexOsgi/com.raytheon.uf.edex.dataplugins.feature/feature.xml
Michael removed awips2-cimss features
