xan/awips2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #2 from srcarter3/new-grib-doc-changes

Browse source 
New grib doc changes
This commit is contained in:
srcarter3 2022-04-08 10:08:19 -06:00 • committed by GitHub
commit 623cd32d76
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 423 additions and 276 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

279
docs/edex/new-grid-grib1-old.md Normal file
View file

@ -0,0 +1,279 @@
# Ingest a New Grid Using .grib Files
Unrecognized grids can be decoded by EDEX simply by dropping `*.grib` or `*.grib2` files into `/awips2/data_store/ingest/`
To add support for a new grid, two edits must be made:
* **Geospatial projection** must be defined in a [***grid navigation file***](#create-grid-projection-file)
* **Grid name**, **center**, **subcenter**, and **process ID** must be defined in a [***model definition file***](#create-model-definition)
If the parameters in the grib file haven't been previously specified, another change *may* be needed as well:
* **Center**, **subcenter**, **discipline**, **category**, and possibly **parameter ID** information may need to be defined in a [***table***](#adding-a-table)
---
## Ingest an Unsupported Grid
### Download Test Data
Download an example grib1 file and rename to a `*.grib` extension, then copy to the manual ingest point `/awips2/data_store/ingest/`
wget https://downloads.unidata.ucar.edu/awips2/current/files/14102318_nmm_d01.GrbF00600 -O wrf.grib
cp wrf.grib /awips2/data_store/ingest/
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
### Check Grib Logs
Confirm that the grib file decodes in the grib log file:
edex log grib
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1200 (sec) Latency: 21.8080 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1180 (sec) Latency: 21.8140 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.4230 (sec) Latency: 21.8360 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.2240 (sec) Latency: 21.9140 (sec)
...
### Check HDF5 Data
Check that the hdf5 data directory exists for our unnamed grid
ls -latr /awips2/edex/data/hdf5/grid/GribModel:7:0:89
Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (7, 0, 89, respectively).
---
## Determine Grid Projection
When the grid was ingested a record was added to the `grid_coverage` table with its navigation information:
psql metadata
metadata=# select nx,ny,dx,dy,majoraxis,minoraxis,la1,lo1,lov,latin1,latin2 from gridcoverage where id=(select distinct(location_id) from grid_info where datasetid='GribModel:7:0:89');
nx | ny | dx | dy | majoraxis | minoraxis | la1 | lo1 | lov | latin1 | latin2
-----+-----+------------------+------------------+-----------+-----------+------------------+-------------------+-------------------+------------------+------------------
201 | 155 | 4.29699993133545 | 4.29699993133545 | 6378160 | 6356775 | 42.2830009460449 | -72.3610000610352 | -67.0770034790039 | 45.3680000305176 | 45.3680000305176
(1 row)
Compare with the projection info returned by wgrib on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above):
<!--
Using html for this code block so that certain sections within the code can be emphasized (bolded)
-->
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 <b>nx 201 ny 155</b> GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid)
<b>Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000</b>
<b>Latin1 45.368000 Latin2 45.368000</b> LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) <b>Dx 4.297000 Dy 4.297000</b> scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
Notice that our grib file has a **Lambert Conformal** projection. We will need these values for the next step.
!!! note "**There is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage area."
---
## Create Grid Projection File
### Projection Types
Grid projection files are stored in `/awips2/edex/data/utility/common_static/base/grib/grids/` and there are four grid coverage types available:
1. **lambertConformalGridCoverage** (example: `RUCIcing.xml`)
<lambertConformalGridCoverage>
<name>305</name>
<description>Regional - CONUS (Lambert Conformal)</description>
<la1>16.322</la1>
<lo1>-125.955</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>151</nx>
<ny>113</ny>
<dx>40.63525</dx>
<dy>40.63525</dy>
<spacingUnit>km</spacingUnit>
<minorAxis>6356775.0</minorAxis>
<majorAxis>6378160.0</majorAxis>
<lov>-95.0</lov>
<latin1>25.0</latin1>
<latin2>25.0</latin2>
</lambertConformalGridCoverage>
2. **polarStereoGridCoverage** (example `seaice_south1_grid.xml`)
<polarStereoGridCoverage>
<name>405</name>
<description>Sea Ice south 690X710 13km grid</description>
<la1>-36.866</la1>
<lo1>139.806</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>690</nx>
<ny>710</ny>
<dx>12.7</dx>
<dy>12.7</dy>
<spacingUnit>km</spacingUnit>
<minorAxis>6371229.0</minorAxis>
<majorAxis>6371229.0</majorAxis>
<lov>100.0</lov>
</polarStereoGridCoverage>
3. **latLonGridCoverage** (example `UkmetHR-SHemisphere.xml`)
<latLonGridCoverage>
<name>864162002</name>
<description>UKMet HiRes combined - Southern Hemisphere
Longitude range 71.25E - 70.416E </description>
<la1>-89.721</la1>
<lo1>71.25</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>864</nx>
<ny>162</ny>
<dx>0.833</dx>
<dy>0.556</dy>
<spacingUnit>degree</spacingUnit>
<la2>-0.278</la2>
<lo2>70.416</lo2>
</latLonGridCoverage>
4. **mercatorGridCoverage** (example `gridNBM_PR.xml`)
<mercatorGridCoverage>
<name>NBM_PR</name>
<description> National Blend Grid over Puerto Rico - (1.25 km)</description>
<la1>16.9775</la1>
<lo1>-68.0278</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>339</nx>
<ny>225</ny>
<dx>1.25</dx>
<dy>1.25</dy>
<la2>19.3750032477232</la2>
<lo2>-63.984399999999994</lo2>
<latin>20</latin>
<spacingUnit>km</spacingUnit>
<minorAxis>6371200</minorAxis>
<majorAxis>6371200</majorAxis>
</mercatorGridCoverage>
### Creating a New Projection File
Copy an existing xml file with the same grid projection type (in this case **lambertConformalGridCoverage**) to a new file `wrf.xml`:
cd /awips2/edex/data/utility/common_static/base/grib/grids/
cp RUCIcing.xml wrf.xml
And edit the new `wrf.xml` to define the projection values using the [output from wgrib or the database](#determine-grid-projection) (example provided):
vi wrf.xml
<lambertConformalGridCoverage>
<name>201155</name>
<description>Regional - CONUS (Lambert Conformal)</description>
<la1>42.2830009460449</la1>
<lo1>-72.3610000610352</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>201</nx>
<ny>155</ny>
<dx>4.29699993133545</dx>
<dy>4.29699993133545</dy>
<spacingUnit>km</spacingUnit>
<minorAxis>6356775.0</minorAxis>
<majorAxis>6378160.0</majorAxis>
<lov>-67.0770034790039</lov>
<latin1>45.3680000305176</latin1>
<latin2>45.3680000305176</latin2>
</lambertConformalGridCoverage>
!!! note "Notice the `<name>201155</name>` tag was created by using the number of grid points (201 and 155). This name can be anything as long as it is unique and will be used to match against in the model definition."
---
## Create Model Definition
Model definition XML files are found in **/awips2/edex/data/utility/common_static/base/grib/models/**.
Since our grib file has a center ID of 7 (NCEP) we will edit the **gribModels_NCEP-7.xml** file.
cd /awips2/edex/data/utility/common_static/base/grib/models/
vi gribModels_NCEP-7.xml
In `<gribModelSet>` add an entry:
<model>
<name>WRF</name>
<center>7</center>
<subcenter>0</subcenter>
<grid>201155</grid>
<process>
<id>89</id>
</process>
</model>
Save the file and restart EDEX for the changes to take effect:
sudo service edex_camel restart ingestGrib
Now copy the `wrf.grib` file *again* to **/awips2/data_store/ingest/**. If everything is correct we will not see any persistence errors since the grid is now named **WRF** and not **GribModel:7:0:89**.
cp wrf.grib /awips2/data_store/ingest/
edex log grib
After you have confirmed that the grid was ingested with the given name, you can [edit the D2D product menus to display the new grid](../cave/d2d-edit-menus.md).
---
## Adding a Table
If you ingest a piece of data and the parameter appears as unknown in the metadata database, ensure that the correct parameter tables are in place for the center/subcenter.
The tables are located in **/awips2/edex/data/utility/common_static/base/grib/tables/**. They are then broken into subdirectories using the following structure: **/[Center]/[Subcenter]/4.2.[Discipine].[Category].table**.
The center and subcenter have been identified previously [here](#determine-grid-projection), as 7 and 0, respectively. So, the corresponding directory is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/
To find the **discipline** of a grib product, you need the **process** and **table** values from the grib file. These are output with the `wgrib -V` command:
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 <b>process 89 Table 2</b> scan: WE:SN winds(grid)
Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000
Latin1 45.368000 Latin2 45.368000< LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
For our example, the process is **89** and table is **2**. Next, take a look in:
/awips2/edex/data/utility/common_static/base/grid/grib1ParameterConvTable.xml
And find the entry that has grib1 data with TableVersion 2 and Value 89:
<grib1Parameter>
<center>7</center>
<grib1TableVersion>2</grib1TableVersion>
<grib1Value>89</grib1Value>
<grib2discipline>0</grib2discipline>
<grib2category>3</grib2category>
<grib2Value>10</grib2Value>
</grib1Parameter>
Here, we can see the discipline and category values (referred to as x above) are 0 and 3, respectively.
So, the table needed for our example file is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/4.2.0.3.table

280
docs/edex/new-grid.md
View file

@ -2,6 +2,8 @@
Unrecognized grids can be decoded by EDEX simply by dropping `*.grib` or `*.grib2` files into `/awips2/data_store/ingest/`
!!! note "This page explains how to ingest `.grib2` products. To view information about `.grib` products, [please see this page](/awips2/edex/new-grid-grib1-old)."
To add support for a new grid, two edits must be made:
* **Geospatial projection** must be defined in a [***grid navigation file***](#create-grid-projection-file)
@ -15,105 +17,52 @@ If the parameters in the grib file haven't been previously specified, another ch
## Ingest an Unsupported Grid
### Grib Products
### Download Test Data
1. Download an example grib1 file and rename to a `*.grib` extension, then copy to the manual ingest point `/awips2/data_store/ingest/`
Download an example grib2 file (make sure the extension is `.grib2` or EDEX may not recognize it), and then copy to the manual ingest point `/awips2/data_store/ingest/`
wget https://downloads.unidata.ucar.edu/awips2/current/files/14102318_nmm_d01.GrbF00600 -O wrf.grib
wget https://downloads.unidata.ucar.edu/awips2/current/files/CPTI_00.50_20180502-000144.grib2 -O cpti.grib2
cp cpti.grib2 /awips2/data_store/ingest/
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
### Check Grib Logs
Confirm that the grib file decodes in the grib log file:
cp wrf.grib /awips2/data_store/ingest/
edex log grib
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1200 (sec) Latency: 21.8080 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1180 (sec) Latency: 21.8140 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.4230 (sec) Latency: 21.8360 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.2240 (sec) Latency: 21.9140 (sec)
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
2. Confirm that the grib file decodes in the grib log file:
...
edex log grib
!!! warning "This step may fail if the parameter is not yet defined. The error will look like:"
<pre>
INFO 2020-07-20 20:34:17,710 2565 [GribPersist-1] GridDao: EDEX - Discarding record due to missing or unknown parameter mapping: /grid/2018-05-02_00:01:44.0_(0)/GribModel:161:0:97/null/null/403/Missing/FH/500.0/-999999.0
INFO 2020-07-20 20:34:17,710 2566 [GribPersist-1] Ingest: EDEX: Ingest - grib2:: /awips2/data_store/ingest/CPTI_00.50_20180502-000144.grib2 processed in: 2.3550 (sec)
INFO 2020-07-20 20:34:17,827 2567 [Ingest.GribDecode-6] grib: EDEX - No parameter information for center[161], subcenter[0], tableName[4.2.209.3], parameter value[61]
</pre>
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1200 (sec) Latency: 21.8080 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1180 (sec) Latency: 21.8140 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.4230 (sec) Latency: 21.8360 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.2240 (sec) Latency: 21.9140 (sec)
...
In order to successfully ingest the example file, you must [**define the appropriate table**](#adding-a-table).
3. Check that the hdf5 data directory exists for our unnamed grid
### Check HDF5 Data
ls -latr /awips2/edex/data/hdf5/grid/GribModel:7:0:89
Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (7, 0, 89, respectively).
### Grib2 Products
1. Download an example grib2 file and rename to a `*.grib2` extension, then copy to the manual ingest point `/awips2/data_store/ingest/`
wget https://downloads.unidata.ucar.edu/awips2/current/files/CPTI_00.50_20180502-000144.grib2 -O cpti.grib2
cp cpti.grib2 /awips2/data_store/ingest/
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
2. Confirm that the grib file decodes in the grib log file:
edex log grib
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1200 (sec) Latency: 21.8080 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1180 (sec) Latency: 21.8140 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.4230 (sec) Latency: 21.8360 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.2240 (sec) Latency: 21.9140 (sec)
...
**Note:** This step will likely fail, because the parameter is not yet defined. The error will look like:
INFO 2020-07-20 20:34:17,710 2565 [GribPersist-1] GridDao: EDEX - Discarding record due to missing or unknown parameter mapping: /grid/2018-05-02_00:01:44.0_(0)/GribModel:161:0:97/null/null/403/Missing/FH/500.0/-999999.0
INFO 2020-07-20 20:34:17,710 2566 [GribPersist-1] Ingest: EDEX: Ingest - grib2:: /awips2/data_store/ingest/CPTI_00.50_20180502-000144.grib2 processed in: 2.3550 (sec)
INFO 2020-07-20 20:34:17,827 2567 [Ingest.GribDecode-6] grib: EDEX - No parameter information for center[161], subcenter[0], tableName[4.2.209.3], parameter value[61]
In order to successfully ingest the example file, [**define the appropriate table**](#grib2-products_4).
3. Check that the hdf5 data directory exists for our unnamed grid
Check that the hdf5 data directory exists for our unnamed grid
ls -latr /awips2/edex/data/hdf5/grid/GribModel:161:0:97
Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (161, 0, 97, respectively).
Though the grib file has been decoded, it has been given a generic name with its **center, subcenter, and process IDs** (161, 0, 97, respectively).
---
## Determine Grid Projection
### Grib Products
When the grid was ingested a record was added to the `grid_coverage` table with its navigation information:
psql metadata
metadata=# select nx,ny,dx,dy,majoraxis,minoraxis,la1,lo1,lov,latin1,latin2 from gridcoverage where id=(select distinct(location_id) from grid_info where datasetid='GribModel:7:0:89');
nx | ny | dx | dy | majoraxis | minoraxis | la1 | lo1 | lov | latin1 | latin2
-----+-----+------------------+------------------+-----------+-----------+------------------+-------------------+-------------------+------------------+------------------
201 | 155 | 4.29699993133545 | 4.29699993133545 | 6378160 | 6356775 | 42.2830009460449 | -72.3610000610352 | -67.0770034790039 | 45.3680000305176 | 45.3680000305176
(1 row)
Compare with the projection info returned by wgrib on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above):
<!--
Using html for this code block so that certain sections within the code can be emphasized (bolded)
-->
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 <b>nx 201 ny 155</b> GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid)
<b>Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000</b>
<b>Latin1 45.368000 Latin2 45.368000</b> LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) <b>Dx 4.297000 Dy 4.297000</b> scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
Notice that our grib file has a **Lambert Conformal** projection. We will need these values for the next step. Note that **there is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage area.
### Grib2 Products
When the grid was ingested a record was added to the `grid_coverage` table with its navigation information:
When a grid is ingested a record is added to the `grid_coverage` table with its navigation information:
psql metadata
@ -130,12 +79,12 @@ When the grid was ingested a record was added to the `grid_coverage` table with
<pre>
wgrib2 -grid -nxny cpti.grib2
1:0:grid_template=0:winds(N/S):
<b>lat-lon grid:(600 x 640)</b> units 1e-06 input WE:NS output WE:SN res 48
<b>lat-lon grid:(600 x 640)</b> units 1e-06 <b>input WE:NS</b> output WE:SN res 48
lat <b>40.799999</b> to 37.599999 by <b>0.005000</b>
lon <b>260.999999</b> to 263.999999 by <b>0.005000</b> #points=384000:(600 x 640)
...
</pre>
Notice that our grib2 file has a **Lat/lon Grid** projection. Where:
Notice that our grib2 file has a **Lat/lon Grid** projection, that starts in the **UpperLeft** corner (as defined by input West to East, North to South). Where:
* **nx** is **600**
* **ny** is **640**
@ -144,12 +93,16 @@ Notice that our grib2 file has a **Lat/lon Grid** projection. Where:
* **la1** is **40.799999**
* **lo1** is **261**
We will need these values for the next step. Note that **there is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage (la1 and lo1) area.
We will need these values for the next step.
!!! note "**There is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage (la1 and lo1) area."
---
## Create Grid Projection File
### Projection Types
Grid projection files are stored in `/awips2/edex/data/utility/common_static/base/grib/grids/` and there are four grid coverage types available:
1. **lambertConformalGridCoverage** (example: `RUCIcing.xml`)
@ -228,45 +181,14 @@ Grid projection files are stored in `/awips2/edex/data/utility/common_static/bas
<majorAxis>6371200</majorAxis>
</mercatorGridCoverage>
### Grib Products
Copy an existing xml file with the same grid projection type (in this case **lambertConformalGridCoverage**) to a new file `wrf.xml`:
cd /awips2/edex/data/utility/common_static/base/grib/grids/
cp RUCIcing.xml wrf.xml
And edit the new `wrf.xml` to define the projection values using the [output from wgrib or the database](#determine-grid-projection) (example provided):
vi wrf.xml
<lambertConformalGridCoverage>
<name>201155</name>
<description>Regional - CONUS (Lambert Conformal)</description>
<la1>42.2830009460449</la1>
<lo1>-72.3610000610352</lo1>
<firstGridPointCorner>LowerLeft</firstGridPointCorner>
<nx>201</nx>
<ny>155</ny>
<dx>4.29699993133545</dx>
<dy>4.29699993133545</dy>
<spacingUnit>km</spacingUnit>
<minorAxis>6356775.0</minorAxis>
<majorAxis>6378160.0</majorAxis>
<lov>-67.0770034790039</lov>
<latin1>45.3680000305176</latin1>
<latin2>45.3680000305176</latin2>
</lambertConformalGridCoverage>
> **Note**: Notice the `<name>201155</name>` tag was defined from the number of grid points (201 and 155). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. WRF).
### Grib2 Products
### Creating a New Projection File
Copy an existing xml file with the same grid projection type (in this case **latLonGridCoverage**) to a new file `cpti.xml`:
cd /awips2/edex/data/utility/common_static/base/grib/grids/
cp MRMS-1km.xml cpti.xml
And edit the new `cpti.xml` to define the projection values using the [output from wgrib2 or the database](#grib2-products_1) (example provided):
And edit the new `cpti.xml` to define the projection values using the [output from wgrib2 or the database](#determine-grid-projection) (example provided):
vi cpti.xml
@ -283,7 +205,7 @@ And edit the new `cpti.xml` to define the projection values using the [output fr
<spacingUnit>degree</spacingUnit>
</latLonGridCoverage>
> **Note**: Notice the `<name>384000</name>` tag was defined from the number of grid points (600 and 640). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. CPTI).
!!! note "Notice the `<name>384000</name>` tag was created by using the number of grid points (600 and 640). This name can be anything as long as it is unique and will be used to match against in the model definition."
---
@ -291,40 +213,6 @@ And edit the new `cpti.xml` to define the projection values using the [output fr
Model definition XML files are found in **/awips2/edex/data/utility/common_static/base/grib/models/**.
### Grid Prodcuts
Since our grib file has a center ID of 7 (NCEP) we will edit the **gribModels_NCEP-7.xml** file.
cd /awips2/edex/data/utility/common_static/base/grib/models/
vi gribModels_NCEP-7.xml
In `<gribModelSet>` add an entry:
<model>
<name>WRF</name>
<center>7</center>
<subcenter>0</subcenter>
<grid>201155</grid>
<process>
<id>89</id>
</process>
</model>
Save the file and restart EDEX for the changes to take effect:
sudo service edex_camel restart ingestGrib
Now copy the `wrf.grib` file *again* to **/awips2/data_store/ingest/**. If everything is correct we will not see any persistence errors since the grid is now named **WRF** and not **GribModel:7:0:89**.
cp wrf.grib /awips2/data_store/ingest/
edex log grib
After you have confirmed that the grid was ingested with the given name, you can [edit the D2D product menus to display the new grid](../cave/d2d-edit-menus.md).
### Grib2 Products
Since our grib2 file has a center of 161 (NOAA) we will edit the **gribModels_NOAA-161.xml** file.
cd /awips2/edex/data/utility/common_static/base/grib/models/
@ -357,52 +245,12 @@ If you ingest a piece of data and the parameter appears as unknown in the metada
The tables are located in **/awips2/edex/data/utility/common_static/base/grib/tables/**. They are then broken into subdirectories using the following structure: **/[Center]/[Subcenter]/4.2.[Discipine].[Category].table**.
### Grib products
The center and subcenter have been identified previously [here](#grib-products) and [here](#grib-products_1), as 7 and 0, respectively. So, the corresponding directory is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/
To find the **discipline** of a grib product, you need the **process** and **table** values from the grib file. These are output with the `wgrib -V` command:
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 <b>process 89 Table 2</b> scan: WE:SN winds(grid)
Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000
Latin1 45.368000 Latin2 45.368000< LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
For our example, the process is **89** and table is **2**. Next, take a look in:
/awips2/edex/data/utility/common_static/base/grid/grib1ParameterConvTable.xml
And find the entry that has grib1 data with TableVersion 2 and Value 89:
<grib1Parameter>
<center>7</center>
<grib1TableVersion>2</grib1TableVersion>
<grib1Value>89</grib1Value>
<grib2discipline>0</grib2discipline>
<grib2category>3</grib2category>
<grib2Value>10</grib2Value>
</grib1Parameter>
Here, we can see the discipline and category values (referred to as x above) are 0 and 3, respectively.
So, the table needed for our example file is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/4.2.0.3.table
### Grib2 Products
If you are using a grib2 file, then you can use either the log output or the `-center`, `-subcenter`, and `-full_name` options on `wgrib2` to get the center, subcenter, discipline, category, and parameter information:
The table would be found in the directory structure using this file's center and subcenter. The center can be found by either:
The table would be found in the directory structure using this file's center and subcenter.
### Finding Center
The center can be found by either:
* Running the following command:
@ -415,13 +263,15 @@ The table would be found in the directory structure using this file's center and
**OR:**
* Running the following command:
<pre>
<pre>
wgrib2 -varX cpti.grib2
1:0:var209_255_1_<b>161</b>_3_61
...
</pre>
1:0:var209_255_1_<b>161</b>_3_61
...</pre>
Where the 4th argument after "var" is the center id, in this case **161**.
### Finding Subcenter
To get the subcenter, simply run:
<pre>
wgrib2 -subcenter cpti.grib2
@ -430,10 +280,12 @@ wgrib2 -subcenter cpti.grib2
</pre>
The subcenter of this file is **0**.
So based on the center and subcenter, the corresponding directory is:
Based on the center and subcenter, the corresponding directory is:
/awips2/edex/data/utility/common_static/base/grib/tables/161/0/
### Finding Discipline and Category
To find the exact table, we need the discipline and category:
<pre>
@ -445,8 +297,10 @@ wgrib2 -full_name cpti.grib2
In this case the **discipline is 209** and **category is 3**, so the corresponding table is:
4.2.209.3.table
So, the full path to the corresponding table would be:
### Corresponding Table
The full path to the corresponding table would be:
/awips2/edex/data/utility/common_static/base/grib/tables/161/0/4.2.209.3.table
@ -464,10 +318,24 @@ You will have to restart ingestGrib for the changes to take place:
sudo service edex_camel restart ingestGrib
Now you can try [re-ingesting the grib2 file](#grib2-products).
Now you can try [re-ingesting the grib2 file](#download-test-data).
---
## Creating Menu Items
After you have confirmed that the grid was ingested with the given name, you can [edit the D2D product menus to display the new grid](../cave/d2d-edit-menus.md).
---
## Using wgrib2
Mentioned in this page are a few command parameters for `wgrib2` such as `-center`, `-subcenter`, and `-full_name`.
A complete [list of all available parameters can be found here](https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/long_cmd_list.html).
---
## Troubleshooting Grib Ingest
* Make sure the latitude and longitude entries in your coverage specification file match those of your ingested raw grib file. There is a tolerance of +/- 0.1 degree to keep in mind when defining your coverage area.

4
environment.yml
View file

@ -6,6 +6,6 @@ dependencies:
- python=3.8
- pip
- pip:
- mkdocs==0.17.5
- mkdocs-unidata==1.5.7
- mkdocs
- mkdocs-unidata
- fontawesome_markdown

136
mkdocs.yml
View file

@ -1,5 +1,5 @@
site_name: Unidata AWIPS User Manual
site_url: http://unidata.github.io/awips2
site_url: http://unidata.github.io/awips2/
site_description: Documentation for all things AWIPS.
site_author: support-awips@unidata.ucar.edu
theme:
@ -28,73 +28,73 @@ extra:
- type: facebook
link: https://facebook.com/Unidata
pages:
- Home: index.md
- Install CAVE: install/install-cave.md
- Install EDEX: install/install-edex.md
- Common Problems: install/common-problems.md
- CAVE User Manual:
- D2D Perspective: cave/d2d-perspective.md
- Maps, Views, Projections: cave/maps-views-projections.md
- Displays and Procedures: cave/bundles-and-procedures.md
- Localization Perspective: cave/localization-perspective.md
# - NCP Perspective: cave/ncp-perspective.md
- NSHARP: cave/nsharp.md
- WarnGen Walkthrough: cave/warngen.md
- GOES 16/17: cave/goes-16-17-satellite.md
- GIS and Shapefiles: cave/d2d-gis-shapefiles.md
- Volume Browser: cave/d2d-gridded-models.md
- Display Tools: cave/d2d-tools.md
- Radar Tools: cave/d2d-radar-tools.md
- Editing Menus: cave/d2d-edit-menus.md
- Change Localization: cave/cave-localization.md
- Import/Export: cave/import-export.md
- EDEX User Manual:
- EDEX Basic Commands: install/start-edex.md
- EDEX Settings: edex/settings.md
- Distributed EDEX: edex/distributed-computing.md
- Docker EDEX: edex/edex-ingest-docker-container.md
- LDM Feeds: edex/ldm.md
- Data Distribution Files: edex/data-distribution-files.md
- Ingest a New Grid: edex/new-grid.md
- Purging and Retention: edex/data-purge.md
- Monitor Users: edex/edex-users.md
- Data Plugins: edex/data-plugins.md
- Archive Case Studies: edex/case-studies.md
- Python-AWIPS: python/overview.md
- Educational Resources: appendix/educational-resources.md
#- Hazard Services:
# - CAVE Display: cave/hazard-services-display.md
# - Hazard Creation: cave/hazard-services-create.md
# - Hazard Alerts: cave/hazard-services-alert.md
# - Hazard Settings: cave/hazard-services-settings.md
# - Examples: cave/hazard-services-example.md
# - Supported Data Types:
# - Gridded Model Display: cave/d2d-grids.md
# - Surface Obs: cave/d2d-pointdata-surface-obs.md
# - Satellite Imagery: cave/d2d-satellite.md
# - NEXRAD Radar: cave/d2d-radar.md
# - Upper Air: cave/d2d-uair.md
# - NCEP/Hydro: cave/d2d-hydro.md
# - Map Overlays: cave/d2d-map-resources.md
- Development: dev/awips-development-environment.md
# - Overview: python/python-awips-data-access.md
# - AWIPS Grids and Cartopy: python/awips-grids-and-cartopy.md
# - Satellite Imagery: python/satellite-imagery.md
# - NEXRAD Level 3 Radar: python/nexrad-level-3-radar.md
# - Upper air BUFR Soundings: python/upper-air-bufr-soundings.md
# - Model Sounding Data: python/model-sounding-data.md
# - Map Resources and Topography: python/map-resources-and-topography.md
# - Surface Obs Plot with MetPy: python/surface-obs-plot-metpy.md
- Appendix:
- AWIPS Grid Parameters: appendix/appendix-grid-parameters.md
- Maps Database: python/maps-database.md
- Acronyms and Abbreviations: appendix/appendix-acronyms.md
# - COTS and FOSS: appendix/appendix-cots.md
- WSR-88D Product Table: appendix/appendix-wsr88d.md
- Keyboard Shortcuts: cave/cave-keyboard-shortcuts.md
- "Raytheon: CAVE User's Manual": raytheon/cave_d2d.md
- "Raytheon: AWIPS System Manager's Manual": raytheon/smm.md
nav:
- Home: index.md
- Install CAVE: install/install-cave.md
- Install EDEX: install/install-edex.md
- Common Problems: install/common-problems.md
- CAVE User Manual:
- D2D Perspective: cave/d2d-perspective.md
- Maps, Views, Projections: cave/maps-views-projections.md
- Displays and Procedures: cave/bundles-and-procedures.md
- Localization Perspective: cave/localization-perspective.md
# - NCP Perspective: cave/ncp-perspective.md
- NSHARP: cave/nsharp.md
- WarnGen Walkthrough: cave/warngen.md
- GOES 16/17: cave/goes-16-17-satellite.md
- GIS and Shapefiles: cave/d2d-gis-shapefiles.md
- Volume Browser: cave/d2d-gridded-models.md
- Display Tools: cave/d2d-tools.md
- Radar Tools: cave/d2d-radar-tools.md
- Editing Menus: cave/d2d-edit-menus.md
- Change Localization: cave/cave-localization.md
- Import/Export: cave/import-export.md
- EDEX User Manual:
- EDEX Basic Commands: install/start-edex.md
- EDEX Settings: edex/settings.md
- Distributed EDEX: edex/distributed-computing.md
- Docker EDEX: edex/edex-ingest-docker-container.md
- LDM Feeds: edex/ldm.md
- Data Distribution Files: edex/data-distribution-files.md
- Ingest a New Grid: edex/new-grid.md
- Purging and Retention: edex/data-purge.md
- Monitor Users: edex/edex-users.md
- Data Plugins: edex/data-plugins.md
- Archive Case Studies: edex/case-studies.md
- Python-AWIPS: python/overview.md
- Educational Resources: appendix/educational-resources.md
#- Hazard Services:
# - CAVE Display: cave/hazard-services-display.md
# - Hazard Creation: cave/hazard-services-create.md
# - Hazard Alerts: cave/hazard-services-alert.md
# - Hazard Settings: cave/hazard-services-settings.md
# - Examples: cave/hazard-services-example.md
# - Supported Data Types:
# - Gridded Model Display: cave/d2d-grids.md
# - Surface Obs: cave/d2d-pointdata-surface-obs.md
# - Satellite Imagery: cave/d2d-satellite.md
# - NEXRAD Radar: cave/d2d-radar.md
# - Upper Air: cave/d2d-uair.md
# - NCEP/Hydro: cave/d2d-hydro.md
# - Map Overlays: cave/d2d-map-resources.md
- Development: dev/awips-development-environment.md
# - Overview: python/python-awips-data-access.md
# - AWIPS Grids and Cartopy: python/awips-grids-and-cartopy.md
# - Satellite Imagery: python/satellite-imagery.md
# - NEXRAD Level 3 Radar: python/nexrad-level-3-radar.md
# - Upper air BUFR Soundings: python/upper-air-bufr-soundings.md
# - Model Sounding Data: python/model-sounding-data.md
# - Map Resources and Topography: python/map-resources-and-topography.md
# - Surface Obs Plot with MetPy: python/surface-obs-plot-metpy.md
- Appendix:
- AWIPS Grid Parameters: appendix/appendix-grid-parameters.md
- Maps Database: python/maps-database.md
- Acronyms and Abbreviations: appendix/appendix-acronyms.md
# - COTS and FOSS: appendix/appendix-cots.md
- WSR-88D Product Table: appendix/appendix-wsr88d.md
- Keyboard Shortcuts: cave/cave-keyboard-shortcuts.md
- "Raytheon: CAVE User's Manual": raytheon/cave_d2d.md
- "Raytheon: AWIPS System Manager's Manual": raytheon/smm.md
extra_css:
- css/extra.css