xan/awips2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #300 from Unidata/unidata_18.1.1

Browse source 
Merge Document Changes from undiata_18.1.1 into unidata_18.2.1
This commit is contained in:
tiffanycmeyer13 2020-11-16 19:30:37 -05:00 • committed by GitHub
commit 03cacbe0bd
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
93 changed files with 1747 additions and 859 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

54
.github/workflows/deploy_docs.yml vendored Normal file
View file

@ -0,0 +1,54 @@
name: publish mkdocs to github pages
on:
push:
branches:
- unidata_18.1.1
paths:
- 'docs/**'
- 'mkdocs.yml'
jobs:
deploy:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
- name: Setup Python and mkdocs
uses: actions/setup-python@v2
with:
python-version: '3.8'
- name: Update pip
run: |
# install pip=>20.1 to use "pip cache dir"
python3 -m pip install --upgrade pip
- name: Create mkdocs_requirements.txt
run: |
echo "mkdocs==0.17.5" >> mkdocs_requirements.txt
echo "mkdocs-unidata==1.5.7" >> mkdocs_requirements.txt
echo "fontawesome_markdown" >> mkdocs_requirements.txt
- name: Get pip cache dir
id: pip-cache
run: echo "::set-output name=dir::$(pip cache dir)"
- name: Cache dependencies
uses: actions/cache@v1
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('**/mkdocs_requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install python dependencies
run: python3 -m pip install -r ./mkdocs_requirements.txt
- run: mkdocs build
- name: Deploy to gh-pages
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./site

2
README.md
View file

@ -30,7 +30,7 @@ Through a grant provided by [Jetstream](https://jetstream-cloud.org/), Unidata i
* [The D2D Perspective](http://unidata.github.io/awips2/cave/d2d-perspective)
* [The NCP Perspective](http://unidata.github.io/awips2/cave/ncp-perspective)
* [The Localization Perspective](http://unidata.github.io/awips2/cave/localization-perspective)
* [python-awips Data Access Framework](http://python-awips.readthedocs.io/)
* [python-awips Data Access Framework](http://unidata.github.io/python-awips/)
* [awips2-users Mailing List Archives](https://www.unidata.ucar.edu/mailing_lists/archives/awips2-users/)
* [(click to subscribe)](mailto:awips2-users-join@unidata.ucar.edu)

1
cave/build/features.txt
View file

@ -43,3 +43,4 @@ com.raytheon.uf.viz.d2d.skewt.feature
gov.noaa.gsd.viz.ensemble.feature
gov.noaa.nws.mdl.viz.boundaryTool.common.feature
com.raytheon.uf.viz.satellite.goesr.feature
com.raytheon.uf.viz.gisdatastore.feature

3
cave/build/p2-build.xml
View file

@ -277,10 +277,11 @@
<antcall target="p2.build.repo">
<param name="feature" value="gov.noaa.nws.mdl.viz.boundaryTool.common.feature" />
</antcall>
<!--
<!--Tiff added -->
<antcall target="p2.build.repo">
<param name="feature" value="com.raytheon.uf.viz.gisdatastore.feature" />
</antcall>
<!--
<antcall target="p2.build.repo">
<param name="feature" value="gov.noaa.nws.obs.viz.geodata.feature" />
</antcall>

1
cave/com.raytheon.viz.product.awips/awips.product
View file

@ -118,6 +118,7 @@
<feature id="com.raytheon.uf.viz.d2d.skewt.feature"/>
<feature id="gov.noaa.nws.mdl.viz.boundaryTool.common.feature"/>
<feature id="com.raytheon.uf.viz.satellite.goesr.feature"/>
<feature id="com.raytheon.uf.viz.gisdatastore.feature"/>
</features>
<configurations>

3
docs/assets/javascripts/application-0b7df094bf.js Normal file
View file

File diff suppressed because one or more lines are too long

27
docs/cave/bundles-and-procedures.md
View file

@ -1,31 +1,32 @@
AWIPS contains two methods for saving and loading data resources: **Bundles** are a simple way to save loaded resources to access in future CAVE sessions. **Procedures** are similar to Bundles, but can be thought of a *groups of bundles* and allows the user to manage saved resources with more control.
# Bundles and Procedures
AWIPS contains two methods for saving and loading data resources: **Bundles** are a simple way to save loaded resources to access in future CAVE sessions. **Procedures** are similar to Bundles, but can be thought of as *groups of bundles* and allows the user to manage saved resources with more control.
## Bundles
### **File &gt; Open Bundle**
### **File > Load Display**
Load a previously-saved bundle from within the AWIPS system. The Open Bundle dialog allows you to select your own saved bundles as well as those saved by other users.
Each selected bundle will load its contents to new tabs which are named after the bundle file name (e.g. NAM_ThetaE)
Each selected bundle will load its contents to new tabs which are named after the bundle file name (e.g. NAM_ThetaE).
![image](../images/KTZZHoCV5S.gif)
Most saved bundles will consist of a single Map Editor (tab), but with multiple tabs saved each will open again in its own Map Editor
Most saved bundles will consist of a single Map Editor (tab), but with multiple tabs saved each will open again in its own Map Editor.
---
### **File &gt; Save Bundle**
### **File > Save Display**
Save a product display within the AWIPS system, synching the bundle between CAVE and the EDEX server.
---
### **File &gt; Delete Bundle**
### **File > Manage Bundles**
Select and remove a saved bundle under File &gt; Delete Bundle. Select the file name and click **OK** and then confirm deletion to remove the saved file permanently.
Select and remove a saved bundle under File > Manage Bundles, this will open the Delete Bundle dialog. Select the file name and click **OK** and then confirm deletion to remove the saved file permanently.
![image](../images/delete_bundle1.png)
![image](../images/fileMenuManageBundles.png)
![image](../images/delete_bundle2.png)
@ -33,7 +34,7 @@ Select and remove a saved bundle under File &gt; Delete Bundle. Select the file
### Load Bundle from Local Disk
To load a previously-saved display from a path within the file directory of the workstation, select **File &gt; Open Bundle** and then select the **File** button on the right to browse your local directories.
To load a previously-saved display from a path within the file directory of the workstation, select **File > Load Dislay** and then select the **File** button on the right to browse your local directories.
![image](../images/XB6vQf78pl.gif)
@ -41,7 +42,7 @@ To load a previously-saved display from a path within the file directory of the
### Save Bundle to Local Disk
To save a product display to a path within the file directory of the workstation, select **File &gt; Save Bundle** and then select the **File** button on the right.
To save a product display to a path within the file directory of the workstation, select **File > Save Display** and then select the **File** button on the right.
---
@ -49,7 +50,7 @@ To save a product display to a path within the file directory of the workstation
### New Procedure
* Select the menu **File &gt; Procedures &gt; New**
* Select the menu **File > Procedures > New**
* Select **Copy Into** to add all loaded resources to the Procedure Stack
* Select **Save** (or **Save As**) and then enter a name for the Procedure before clicking **OK** to save.
@ -57,8 +58,8 @@ To save a product display to a path within the file directory of the workstation
### Open Procedure
Similar to creating a new Procedure, select **File &gt; Procedures &gt; Open**, select the saved resources and click **Load** to load them to CAVE.
Similar to creating a new Procedure, select **File > Procedures > Open**, select the saved resources and click **Load** to load them to CAVE.
### Delete Procedure
From the menu **File &gt; Procedures &gt; Delete** you can delete existing Procedure files in a way similar to deleting saved Bundle files.
From the menu **File > Procedures > Delete** you can delete existing Procedure files in a way similar to deleting saved Bundle files.

90
docs/cave/cave-keyboard-shortcuts.md
View file

@ -1,4 +1,6 @@
# Keyboard Shortcuts
<!--
Action|Command
----------------|------------
Open bundle | Ctrl+O
@ -13,4 +15,90 @@ Image combine | Insert
Loop properties | Ctrl+L
Image properties| Ctrl+I
Export image | Ctrl+E
Exporrt KML | Ctrl+K
Exporrt KML | Ctrl+K
-->
## D2D Menu Shortcuts
Action|Command
------------------------------|---------------
Open a New Map | Ctrl + N
Open a Bundle | Ctrl + O
Save Bundle | Ctrl + S
Save Bundle Locally | Ctrl + Shift + S
Save KML | Ctrl + K
Exit CAVE | Alt + F4
Exit CAVE | Ctrl + Q
Clear Data | Ctrl + C
First Frame | Ctrl + &#8592;
Last Frame | Ctrl + &#8594;
Step Back | &#8592;
Step Forward | &#8592;
Increase Loop Speed | Page Up
Decrease Loop Speed | Page Down
Open Time Options | Ctrl + T
Toggle Image Combination | Insert
Open Loop Properties | Ctrl + L
Open Image Properties | Ctrl + I
## D2D All Tilts Shortcuts
>**Note**: Requires all tilts product in main display panel
Action|Command
--------------------------------|---------------
Step Back 1 Volume | &#8592;
Step Forward 1 Volume | &#8594;
Step up 1 Elevation Angle | &#8593;
Step down 1 Elevation Angle | &#8595;
Jump to First Frame | Ctrl + &#8592;
Jump to Last Frame | Ctrl + &#8594;
Jump to Highest Elevation Angle | Ctrl + &#8593;
Jump to Lowest Elevation Angle | Ctrl + &#8595;
## D2D Numeric Keypad Shortcuts
>**Note**: Num Lock must be enabled for these keystrokes to work
Action|Command
--------------------------------------------------|---------------
Increase Brightness of Image 1, Decrease Image 2 | [Numpad] +
Decrease Brightness of Image 1, Increase Image 2 | [Numpad] -
Toggle Image Producted in Main Map On/Off | [Numpad] 0
Toggle First 9 Graphic Products On/Off | [Numpad] 1-9
Toggle Next 10 Graphic Prodcuts On/Off | Shift + [Numpad] 0-9
Toggle Between Images 1/2 at Full Brightness | [Numpad] .
Toggle Legend | [Numpad] Enter
## Panel Combo Rotate (PCR) Shortcuts
>**Note**: These numbers refer to the ones at the top of the Keyboard
Action|Command
---------------------------|------
Cycle Through PCR Products | Delete
Return to 4 Panel View | End
Cycle Back Through PCR Products | Backspace
Display Corresponding Product | 1-8
## Text Editor Shortcuts
Action|Command
--------------------------|------
Extend Selection to Start of Line | Shift + Home
Extend Selection to End of Line | Shift + End
Extend Selection to Start of Document | Ctrl + Shift + Home
Extend Selection to End of Document | Ctrl + Shift + End
Extend Selection Up 1 Screen | Shift + Page Up
Extend Selection Down 1 Screen | Shift + Page Down
Extend Selection to Previous Character | Shift + &#8592;
Extend Selection by Previous Word | Ctrl + Shift + &#8592;
Extend Selection to Next Character | Shift + &#8594;
Extend Selection by Next Word | Ctrl + Shift + &#8594;
Extend Selection Up 1 Line | Shift + &#8593;
Extend Selection Down 1 Line | Shift + &#8595;
Delete Previous Word | Ctrl + Backspace
Delete Next Word | Ctrl + Delete
Close the Window | Ctrl + Shift + F4
Undo | Ctrl + Z
Copy | Ctrl + C
Paste | Ctrl + V
Cut | Ctrl + X

10
docs/cave/cave-localization.md
View file

@ -1,14 +1,16 @@
# Change Localization
## Localization Preferences
The default localization site for Unidata AWIPS is BOU (Boulder, Colorado, where UCAR is located). When you are prompted to connect to an EDEX server, you can change the WFO ID as well.
The default localization site for Unidata AWIPS is OAK (Omaha, Nebraska, where the Raytheon team is located). When you are prompted to connect to an EDEX server, you can change the WFO ID as well.
![](../images/connect.png)
![](../images/edexServerDialog.png)
Since release 16.1.4, CAVE users can switch the localization site to any valid NWS WFO from **CAVE > Preferences > Localization**, where edits canbe made to both the site ID and EDEX server name. Click **Restart** after changes are applied.
Since release 16.1.4, CAVE users can switch the localization site to any valid NWS WFO from **CAVE > Preferences > Localization**, where edits can be made to both the site ID and EDEX server name. Click **Restart** after changes are applied.
This window also has the option to **Prompt for settings on startup**, which if checked, would ask for the EDEX Server and Site location every time CAVE is started (this can be useful if you are used to switching between servers and/or sites).
![](../images/cave-localization-3.png)
Change the site (example shows TBW Tampa Bay) and click **Okay** or **Apply** and confirm the popup dialog, which informs you that you must *restart* CAVE for the changes to take effect.
Change the site (example shows TBW Tampa Bay) and click **Apply** or **OK** and confirm the popup dialog, which informs you that you must *restart* CAVE for the changes to take effect.
![](../images/cave-localization-4.png)

71
docs/cave/d2d-edit-menus.md
View file

@ -1,54 +1,49 @@
# Editing Menus
Any of the data menus can be customized in the Localization Perspective.
Any of the menus in the menubar can be customized in the **Localization Perspective**.
## Changing the models displayed with D2D bundles
---
1. In the Localization Perspective you can navigate to **D2D** > **Volume Browser** > **VbSources** and click on **volume.xml** to expand. Then right-click on **BASE** and select **Copy To...** **User** (awips).
## Modifying Menus
2. You can then right-click again on the **VbSources** folder and select **Refresh** to see your new localization file called **USER (awips)**.
Once in the **Localization Perspective**, menus can be modified by going to the **D2D** > **Menus** directory in the File Browser. Here there are submenus for different data types and menu structures. Usually the **index.xml** file found in these submenus is the *master* file which the actual menu is based off of. This file can reference other xml files and you may have to modify these *child* xml files to get the results you are looking for.
3. Open the file and add or edit the model names:
In order to modify any file, you must right click on it and
select **Copy To > USER (my-username)**. Then you may open this copy and begin to modify it. Once this process has been completed and a change has been made and saved, CAVE will need to be restarted and opened in the D2D perspective to see the change.
<menuContributionFile>
<include installTo="menu:models"
fileName="menus/volume/allFamilies.xml">
<substitute key="modelName" value="CMC" />
<substitute key="menuName" value="CMC" />
<substitute key="frameCount" value="41" />
<substitute key="TP" value="TP"/>
</include>
<include installTo="menu:models"
fileName="menus/volume/allFamilies.xml">
<substitute key="modelName" value="GFS" />
<substitute key="menuName" value="GFS" />
<substitute key="frameCount" value="41" />
<substitute key="TP" value="TP"/>
</include>
...
This example covers how to add a new menu entry to an existing menu.
Notice that you specify both the `modelName` (what it's named inside EDEX, such as NAM12), and `menuName`, if you prefer a more detailed menu entry, such as "NAM 12km". When you are finished, save the file and restart CAVE for the changes to take effect.
1. Switch to the **Localization Perspective**
2. Find the **grid** folder under **D2D** > **Menus**
3. Double-click to expand **index.xml**
4. Right-click to **BASE (common_static)** and select **Copy To...**, then select **USER** level
5. Double-click **USER** to open the editor and copy an existing include tag, and update the *modelName* (this must match an existing product found in the Product Browser) and the *menuName* (this can be anything)
<include installTo="menu:models" fileName="menus/grid/allFamilies.xml">
<substitute key="modelName" value="GEFS" />
<substitute key="menuName" value="GEFS" />
<substitute key="frameCount" value="41" />
<substitute key="TP" value="TP"/>
</include>
6. Once this is completed, save the file and restart CAVE
7. Navigate to the **Models** menu and you should see a new entry with **GEFS**
Your new localization file exists on the EDEX server (assuming username awips) as
`/awips2/edex/data/utility/cave_static/users/awips/volumebrowser/VbSources/index.html`
---
and will exist on the local CAVE machine under `~/caveData/etc/user/awips/volumebrowser/VbSources/index.html`
## Removing menus from the menubar
## Removing Menus
This example covers how to remove a menu (in this case **MRMS**) from D2D:
1. switch to the Localization Perspective
2. find the **mrms** folder under CAVE > Menus
3. double-click to expand **index.html**
4. right-click **BASE** and select **Copy To...**, then select **USER** level
5. right-click refresh the **mrms** entry
6. double click **USER** to open the editor and change
1. Switch to the **Localization Perspective**
2. Find the **mrms** folder under **D2D** > **Menus**
3. Double-click to expand **index.xml**
4. Right-click **BASE** and select **Copy To...**, then select **USER** level
5. Right-click refresh the **mrms** entry
6. Double click **USER** to open the editor and change
<menuContributionFile>
<include installTo="menu:mrms?after=MRMS_MENU_START"
fileName="menus/mrms/mrms.xml">
</include>
<include installTo="menu:mrms?after=MRMS_MENU_START" fileName="menus/mrms/mrms.xml"/>
</menuContributionFile>
to
@ -56,4 +51,4 @@ This example covers how to remove a menu (in this case **MRMS**) from D2D:
<menuContributionFile>
</menuContributionFile>
With this you can restart CAVE and will not see the MRMS menu anymore. Repeat this example for other product menus, such as **local**, **hydro**, **scan**, etc., to further customize D2D data menus for any level of localization.
With this completed, you can now restart CAVE and will not see the MRMS menu anymore. Repeat this example for other product menus, such as **radar**, **upperair**, **tools**, etc., to further customize D2D data menus for any level of localization.

62
docs/cave/d2d-gridded-models.md
View file

@ -1,15 +1,20 @@
# Volume Browser
The Volume Browser provides access to numerical models, other gridded data, and selected point data sources, such as RAOB, METAR, and Profiler. Through the Browser interface, you can choose the data source(s), field(s), plane(s), and point(s), and generate a customized list of model graphics or images for display.
The Volume Browser provides access to numerical models, other gridded data, sounding data, and selected point data sources, such as RAOB, METAR, and Profiler. Through the Browser interface, you can choose the data source(s), field(s), plane(s), and point(s), and generate a customized list of model graphics or images for display.
The Volume Browser window is divided into four areas:
* The Menu Bar along the top
* The Data Selection Menus
* The Product Selection List
* The Load Buttons (Diff and Load) to load items from the Product Selection List
![](../images/volumeBrowserLabeled.png)
1. The [**Menu Bar**](#volume-browser-menu-bar) along the top
2. The Data Selection Menus
3. The Product Selection List
4. The Load Buttons (Diff and Load) to load items from the Product Selection List
Each area is then subdivided into menu components. The menu bar along the top of the Volume Browser window has dropdown lists that contain options for controlling all the various menu choices of the Volume Browser.
---
## Volume Browser Menu Bar
The dropdown menus in the Volume Browser menu bar contain options for controlling and manipulating the Volume Browser or the products chosen through the Volume Browser
@ -25,10 +30,17 @@ The dropdown menus in the Volume Browser menu bar contain options for controllin
* Select None
* Select All
* Find (Ctrl+F)
* [**Tools**](#vb-tools)
* [**Display Types**](#display-types)
* [**Animation Types**](#animation-types)
---
## VB Tools
>### Baselines
![](../images/volumeBrowserToolsMenu.png)
### Baselines
Selecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be
constructed from within the Volume Browser. These baseline resources are **editable**.
@ -37,7 +49,7 @@ If you are zoomed in over an area when you load baselines and none appear, press
are working with a baseline, a second click with B3 will return you to the original baseline, even
if you modified another baseline.
>### Points
### Points
Points are used to generate model soundings, time-height cross-sections, time series, and variable vs.
height plots using the Volume Browser. As with the Baselines, the locations of these Points can be
@ -55,39 +67,49 @@ Points may be created, deleted, hidden, and manipulated (location, name, font, a
not limited in terms of number, location, or designation. Points may also be assigned to different
groups to facilitate their use.
### Choose By ID
Choose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and "Home" for conventional locations like METARs and RAOBs (Radiosonde Observations), but its primary use is for the WSR-88D-identified mesocyclone locations.
---
## Display Types
>### Plan View (default)
![](../images/volumeBrowserDisplayMenu.png)
The default option for the Volume Browser. From the Plan-view perspective, data are plotted onto horizontal surfaces. The Plan view Planes are discussed in the planes section that follows. The additional options menu that appears in the Volume Browser menu bar allows you to choose whether you want the Plan view data to Animate in Time or Animate in Space.
### Plan View (default)
>### Cross Section
This is the default option for the Volume Browser. From the Plan-view perspective, data are plotted onto horizontal surfaces. The additional options menu that appears in the Volume Browser menu bar allows you to choose whether you want the Plan view data to Animate in Time or Animate in Space.
Allows you to view gridded data as vertical slices along specific baselines. You need to use either the Interactive Baseline Tool or the predefined latitude/longitude baselines to specify the slice you wish to see. One of the additional options menus that appear in the Volume Browser menu bar allows you to choose whether you want the cross-section data to animate in time or space, while the other options menu allows you to adjust the vertical resolution. Descriptions of these options follows. (Note that the Fields and Planes submenu labels have changed after selecting "Cross section.")
### Cross Section
>### Time Height
Allows you to view gridded data as vertical slices along specific baselines. You need to use either the Interactive Baseline Tool or the predefined latitude/longitude baselines to specify the slice you wish to see. One of the additional options menus that appear in the Volume Browser menu bar allows you to choose whether you want the cross-section data to animate in time or space, while the other options menu allows you to adjust the vertical resolution. Descriptions of these options follows. (Note that the Fields and Planes submenu labels have changed after selecting "Cross section.")
Used in conjunction with the Interactive Points Tool to enable you to view a time height cross section of a full run of gridded model data for a specific location. Additional options menus in the Volume Browser menu bar allow you to choose the direction in which you want the data to be plotted, and to adjust the vertical resolution.
### Time Height
>### Var vs Hgt
Used in conjunction with the Interactive Points Tool to enable you to view a time height cross section of a full run of gridded model data for a specific location. Additional options menus in the Volume Browser menu bar allow you to choose the direction in which you want the data to be plotted, and to adjust the vertical resolution.
### Var vs Hgt
Enables you to view a profile of a meteorological model field as it changes through height, which is displayed in millibars. By using the Interactive Points Tool, you can select one or more locations from which to plot the data.
>### Sounding
### Sounding
Works in conjunction with the Interactive Points Tool to enable you to generate a Skew-T chart for a specific location, no additional menus appear in the Volume Browser when the Soundings setting is chosen.
>### Time Series
### Time Series
Used in conjunction with the Interactive Points Tool to enable you to plot gridded data on a time versus data value graph for a specified point.
## Animation Types
>### Time
![](../images/volumeBrowserAnimationMenu.png)
### Time
The default option for the Volume Browser. It allows you to view model data through time
>### Space
### Space
Allows you to loop through a series of predefined latitude or longitude cross-sectional slices at a fixed time.
Allows you to loop through a series of predefined latitude or longitude cross-sectional slices at a fixed time.

129
docs/cave/d2d-perspective.md
View file

@ -1,14 +1,15 @@
# D2D Perspective
D2D (Display 2-Dimensions) is the default AWIPS CAVE perspective, designed to mimmic the look and feel of the legacy AWIPS I system.
D2D (Display 2-Dimensions) is the default AWIPS CAVE perspective, designed to mimmic the look and feel of the legacy AWIPS I system.
![image](../images/iuNDS6J.png)
![image](../images/caveHeader.png)
System menus include **CAVE**, **File**, **View**, **Options**, **Tools**, and **Help** (far right)
System menus include **CAVE**, **File**, **View**, **Options**, and **Tools**.
Data menus include **[Models](d2d-grids)**, **[Surface](d2d-obs)**, **[NCEP/Hydro](d2d-hydro)**, **[Upper Air](d2d-uair)**, **[Satellite](d2d-satellite)**, **[Radar](d2d-radar)**, **[MRMS](d2d-mrms)**, and **[Maps](d2d-maps)**.
Data menus include **[Models](d2d-grids)**, **Surface**, **[NCEP/Hydro](d2d-hydro)**, **[Upper Air](d2d-uair)**, **[Satellite](d2d-satellite)**, **Local Radar Stations**, **[Radar](d2d-radar)**, **MRMS**, and **Maps**.
Frame control, map projection, image properties, and a few featured applications (**Warngen**, **Nsharp**, **Ncdata**, and **Browser**) make up the the primary D2D toolbar.
Map projection, image properties, frame control, and a few featured applications (**Warngen**, **Nsharp**, and **Browser**) make up the the primary D2D toolbar.
>**Note**: Depending on which Operating System version of CAVE there may be other application options (**PGEN**, **GEMPAK**).
---
@ -17,46 +18,21 @@ Frame control, map projection, image properties, and a few featured applications
At bottom-right of the map window the the Resource Stack, which displays all loaded resources and map overlays, and allows for interaction and customization with the resource via a **right-click menu**.
There are three available views of the Resource Stack, the default will show all Product Resources. **Right Click** the mouse on the map background to switch to a Simple View. **Right Click** again to show all Map Resources. **Right Click** again to toggle back to Product Resources.
There are three available views of the Resource Stack, the default will show all Product Resources. The other two views are the Simple view, which shows the time, and the Map Resources. To switch between views see the [Right-Click Functionality](#right-click-background-to-cycle-resource-views).
It's important to understand that Product Resources and Map Resources are handled differently given the time-based nature of Products, compared to the static nature of maps. Selecting the **Clear** button will remove all Products but not remove any Map Products.
It's important to understand that Product Resources and Map Resources are handled differently given the time-based nature of Products, compared to the static nature of maps. Selecting the **Clear** button will remove all Products but not remove any Map Products.
> Left-Click Resource Name to Hide
### Left-Click Resource Name to Hide
A left click on any resource in the stack will hide the resource and turn the label gray. Clicking the name again makes the resource visible.
![image](../images/rASkR3Rp6y.gif)
> Left-Click Background to Hide Resource Types
### Right-Click Background to Cycle Resource Views
A left click of the mouse anywhere on the map will hide the Resource Stack display names, which may help to declutter the view if a number of data types are loaded at once.
The default display in the resource stack is the Product Resources. Right Click the mouse on the map background (anywhere but on the stack itself) to switch to a Simple View, which just shows the current displayed time if product data is loaded. Right Click again to show all Map Resources. Right Click again to switch back to Product Resources.
> Hold-Right-Click Background for Display Menu
Holding down the right mouse button anywhere in the map view will open a right-click menu
![image](../images/rightclickmenu.gif)
> Show Map Legends
From the above menu select **Show Map Legends** and watch the Resource Stack show only map resources which are loaded to the view.
![image](../images/maplayers.png)
> Toggle 2 or 4-Panel Layout
Right-click hold in the view and select **Two Panel Layout** or **Four Panel Layout** to create duplicates of the current view (note that any data loaded to the *view* will be loaded to *both displays within the view*).
From this multi-pane display, hold-right-click again and you will see the **Single Panel Layout** option to switch back to a standard view (defaulting to the left of two, and top-left of four).
> Sample Loaded Resources
Most data types have a right-click menu option for reading out the pixel value, displayed as multi-line text for multiple resources.
![image](../images/ui4fNI3X0C.gif)
> Hold-Right-Click Resource Name for Menu
### Hold-Right-Click Resource Name for Menu
Drag the mouse over a loaded resource and **hold** the right mouse button until a menu appears (simply clicking the resource with the right mouse button will toggle its visibility).
@ -66,22 +42,57 @@ The hold-right-click menu allows you to control individual resource **Image Prop
---
## Display Menu
The display menu has many options which can alter the functionality in CAVE.
### Hold-Right-Click Background for Display Menu
Holding down the right mouse button anywhere in the map view will open a right-click menu
![image](../images/rightclickmenu.gif)
### Show Map Legends
From the above menu select **Show Map Legends** and watch the Resource Stack show only map resources which are loaded to the view.
![image](../images/maplayers.png)
### Toggle 2 or 4-Panel Layout
Right-click hold in the view and select **Two Panel Layout** or **Four Panel Layout** to create duplicates of the current view (note that any data loaded to the *view* will be loaded to *both displays within the view*).
From this multi-pane display, hold-right-click again and you will see the **Single Panel Layout** option to switch back to a standard view (defaulting to the left of two, and top-left of four).
### Sample Loaded Resources
Most data types have a right-click menu option for reading out the pixel value, displayed as multi-line text for multiple resources. This can be toggled on and off by selecting the **Sample** option in the Display Menu.
![image](../images/ui4fNI3X0C.gif)
---
## Product Browser
The Product Browser allows users to browse a complete data inventory in a side window, organized by data type. Selections for **GFE**, **Grids**, **Lightning**, **Map Overlays**, **Radar**, **Satellite**, **Redbook**, and **VIIRS** are available. All products loaded with the Product Browser are given default settings.
The Product Browser allows users to browse a complete data inventory in a side window, organized by data type. To open the Product Browser, either select the icon in the toolbar (![](../images/productBroswerIcon.png)), or go to the menu: **CAVE** > **Data Broswers** > **Product Broswers**.
Selections for **Grid**, **Lightning**, **Maps**, **Radar**, **Redbook**, and **Satellite** are available. All products loaded with the Product Browser are given default settings.
>**Note**: The Linux and Mac version also have a selection for **GFE** available.
![image](../images/vPeaMsn9ZT.gif)
---
## Switch Pane Layouts
<!--- ## Switch Pane Layouts
will switch between the single page (default) view and the 5-panel WFO view (centered on OAX by default).
---
--- -->
## Time Options (Ctrl + T)
## Options Menu
There are several toggle options and options dialogs that are available under the **Options** menu found at the top of the application.
### Time Options (Ctrl + T)
This check button enables/disables the ability to select the time interval between frames of real-time or model data. This feature has the added benefit of allowing you to view extended amounts of data (temporally) but stay within the limits of 64 frames. For example, METAR surface plots, which typically display every hour, can be set to display every three hours via the Select Valid Time and Time Resolution Dialog Box.
@ -104,52 +115,45 @@ box appears, providing the following options:
* **Tolerance**: The options in this column refer to how strict the time matching is. "None" means an exact match, while "Infinite" will put the closest match in each frame, regardless of how
far off it is.
---
## Data Scale (Ctrl + S)
<!--- ## Data Scale (Ctrl + S)
This check button enables/disables the ability to display data on its native scale. For example, if you enable Data Scaling and select a product from an alternate radar, the data will be displayed with that radar in the center of the screen. Other data can be overlaid on this "dynamic" scale until the Main Display Pane is cleared or a non-plan-view product is loaded.
---
--- -->
## Image Combination (Insert)
### Image Combination (Insert)
This check button enables/disables the ability to display two images at once. You can also enable/disable the ability to combine images by using the Toggle Image Combination iconified button on the Toolbar.
This check button enables/disables the ability to display two images at once.
Combined-image displays have been improved by removing the valid time for non-forecast products and removing the date string (time is kept) from the left side of the legend. In particular, this
makes All-Tilts radar legends more usable.
Combined-image displays have been improved by removing the valid time for non-forecast products and removing the date string (time is kept) from the left side of the legend. In particular, this makes All-Tilts radar legends more usable.
---
## Display Properties
### Display Properties
This menu option opens the Display Properties dialog box. All the options available in this dialog box are also available on the Toolbar.
![image](../images/x9uQNAI.png)
---
## Loop Properties (Ctrl + L)
### Loop Properties (Ctrl + L)
Loop Properties is another dialog box that can be opened from the Options menu or from the Loop Properties iconified button on the D2D Toolbar, or by using the Ctrl + L keyboard shortcut. The dialog allows you to adjust the forward and backward speeds, with 0 = off and 10 = maximum speed. You can set the duration of the first and last frame dwell times to between zero and 2.5 seconds.
You can turn looping on or off by checking the Looping check button. There is also a Looping button located on the Toolbar that enables/disables the animation in the large display pane. Finally, you can turn looping on and increase/decrease forward speed by pressing Page Up/Page Down on your keyboard, and turn looping off with the Left or Right Arrow keys. On the toolbar, you can use the button to start/stop looping.
---
## Image Properties (Ctrl + I)
### Image Properties (Ctrl + I)
The Image Properties dialog box can be opened here or by using the Image Properties iconified button on the D2D Toolbar, or using using the Ctrl + I keyboard shortcut. This dialog box provides options that allow you to change the color table; adjust the brightness, contrast, and alpha of either a single image or combined images; fade between combined images; and/or interpolate the displayed data.
The Image Properties dialog box can be opened here (in the Options menu) or by using the Image Properties iconified button on the D2D Toolbar (![](../images/imagePropsIcon.png)), or using using the Ctrl + I keyboard shortcut. This dialog box provides options that allow you to change the color table; adjust the brightness, contrast, and alpha of either a single image or combined images; fade between combined images; and/or interpolate the displayed data.
---
## Set Time
### Set Time
This option allows you to set the CAVE clock, located on the bottom of the screen, to an earlier time for reviewing archived data.
---
## Set Background Color
### Set Background Color
You can now set the background display color on your workstation. You can also set the background display color for a single pane via mouse Button 3 (B3).
@ -157,11 +161,12 @@ You can now set the background display color on your workstation. You can also s
## Switching Perspectives
**CAVE > Perspective**
Switching perspectives in CAVE can be found in the **CAVE > Perspective** menu.
D2D is one of many available CAVE perspectives. By selecting the **CAVE** > **Perspective** menu you can switch into the **GFE**, **Hydro**, **Localization**, **MPE**, or **National Centers Perspective** (which is available in the **Other...** submenu. Nobody seems to know why the NCP is not listed with the other perspectives, or how to make it appear with them).
D2D is one of many available CAVE perspectives. By selecting the **CAVE** > **Perspective** menu you can switch into the **GFE**, or **Localization** perspective.
>**Note**: The **National Centers Perspective** (which is available in the **Other...** submenu) is available on the Linux version of CAVE. And the **GFE** perspective is not available on the Windows version.
![image](../images/OU6rWMD.png)
![image](../images/perspectivesMenu.png)
---
@ -177,7 +182,7 @@ Set the Localization Site and server for the workstation; configure mouse operat
## Load Mode
Under the Display Properties option is Load Mode, which provides different ways to display data by manipulating previous model runs and inventories of data sets. The selected load mode is shown on the toolbar when the Load Mode menu is closed.
Within the Display Properties dialog is the Load Mode option, which provides different ways to display data by manipulating previous model runs and inventories of data sets. The selected load mode is shown on the toolbar when the Load Mode menu is closed, and can also be changed by using this toolbar option as well.
A description of the Load Mode options follow.

129
docs/cave/d2d-radar-tools.md
View file

@ -1,14 +1,88 @@
# Radar Tools
These programs are accessible though the **Tools** dropdown menu, and in individual site radar menus.
The radar tools are a subset of the tools available in CAVE. These programs are accessible though the **Tools** dropdown menu, and in individual site radar menus.
![image](../images/toolsMenuRadarTools.png)
---
## Estimated Actual Velocity (EAV)
A velocity (V) display from the radar shows only the radial component of the wind, so the indicated
speed depends on the direction of the wind and the azimuth (direction) from the radar. Consider, for example, a north wind. Straight north of the radar, the full speed of the wind will be seen on the V product. As one moves around to the east of the radar, the radial component gets smaller, eventually reaching zero straight east of the radar. If the wind direction is known, then the actual wind speed can be computed by dividing the observed radial speed by the cosine of the angle between the radar radial and the actual direction. The EAV tool allows you to provide that angle and use the sampling function of the display to show the actual wind speed.
A velocity (V) display from the radar shows only the radial component of the wind, so the indicated speed depends on the direction of the wind and the azimuth (direction) from the radar. Consider, for example, a north wind. Straight north of the radar, the full speed of the wind will be seen on the V product. As one moves around to the east of the radar, the radial component gets smaller, eventually reaching zero straight east of the radar. If the wind direction is known, then the actual wind speed can be computed by dividing the observed radial speed by the cosine of the angle between the radar radial and the actual direction. The EAV tool allows you to provide that angle and use the sampling function of the display to show the actual wind speed.
![image](../images/qLuJnK2aVF.gif)
## V-R Shear
---
## Radar Display Controls
<span style="float:right;">
![image](../images/radarDisplayControls.png)
</span>
The Radar Display Controls dialog box is derived from the Radar Tools submenu and provides options that control the appearance of the Storm Track Information (STI), the Hail Index (HI), the Tornado Vortex Signature (TVS), the Digital Mesocyclone Display (DMD) products, the Microburst Alert (MBA) products, the Storm Relative Motion (SRM), and the SAILS products. The Radar Display Controls dialog box options are described below.
> **Note**: Our version of CAVE may not have all the products that these options are applicable to.
The Radar Display Controls dialog box is divided into eight sections: [STI](#sti-storm-track-information), [HI](#hi-hail-index), [TVS](#tvs-tornado-vortex-signature), [DMD/MD/TVS](#dmd-md-tvs), [DMD](#dmd-digital-mesocyclone-display), [MBA](#mba-microburst-alert), [SRM](#srm-storm-relative-motion), and [SAILS](#sails-supplemental-adaptive-intra-volume-low-level-scan). Each section has the following options:
### STI (Storm Track Information)
This section has options to adjust the appearance of the STI graphic product.
* **Number of storms to show**: This slider bar lets you choose the maximum number of storms (0 to 100) you wish to display on the STI product. The default value is 20 storms.
* **Type of track to show**: This options menu allows you to choose the type of storm track that you want displayed.
### HI (Hail Index)
This portion of the Radar Display Controls dialog box contains options that alter the appearance of the HI radar graphic product. You can set the low and high algorithm thresholds of the Probability of Hail (POH) and the Probability of Severe Hail (POSH). Storms that meet the low POH threshold are indicated by small open triangles, while small solid triangles mark those that meet the high POH threshold. Similarly, large open triangles or solid triangles are plotted for the POSH low and high thresholds, respectively.
* **Low hail probability (POH)**: The storms that meet or exceed the threshold are indicated by small open triangles. The default setting is 30.
* **Low severe hail probability (POSH)**: The storms that meet or exceed the threshold are indicated by large open triangles. The default setting is 30.
* **High hail probability**: The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50.
* **High severe hail probability**: The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50.
### TVS (Tornado Vortex Signature)
There is one option in this section of the Radar Display Controls dialog box.
* **Show elevated TVS**: This toggle button lets you control the appearance of the elevated TVS radar graphic product.
### DMD, MD, TVS
There is one option in this section of the Radar Display Controls dialog box.
* **Show extrapolated features**: With this option, you can choose whether to show the time-extrapolated features using DMD, MD, or TVS.
### DMD (Digital Mesocyclone Display)
* **Minimum feature strength**: A mesocyclone clutter filter which specifies the minimum 3D strength rank use to display a mesocyclone (default is 5).
* **Show overlapping Mesos**: Toggles whether to show overlapping mesocyclones.
* **Type of track to show**: This dropdown has option available for whether to display past and/or forcast tracks.
### MBA (Microburst Alert)
* **Show Wind Shear**: This option allows you to choose whether to display wind shear associated with microburts alerts.
### SRM (Storm Relative Motion)
The first three options in the SRM section allow you to choose where you want to derive the storm motion from.
* **Storm Motion from WarnGen Track**: Selecting this option will display the storm motion from a WarnGen Track.
* **Average Storm Motion from STI**: Selecting this option will display the average storm motion from from the storm track information (STI).
* **Custom Storm Motion**: Selecting this option allows you to specify a custom storm motion with the selections below.
- **Direction**: This slider allows you to choose the direction (in degrees??) of the storm motion.
- **Speed**: This slider allows you to specify the speed (in mph??) of the storm motion.
### SAILS (Supplemental Adaptive Intra-Volume Low Level Scan)
* **Enable SAILS Frame Coordinator**:
**Enabled (default)**: keyboard shortcuts change where tilting up from 0.5 degree SAILS tilt will step to the next higher tilt (similar to GR2 Analyst) and Ctrl right arrow will step to the most recent tilt available for any elevation angle.
**Disabled**: keyboard shortcuts change where tilting up from 0.5 degree SAILS tilt will not go anywhere (old confusing behavior) and Ctrl right arrow will step to the most recent time of the current tilt.
---
## VR - Shear
This tool is used in conjunction with Doppler velocity data to calculate the velocity difference (or "shear") of the data directly under the end points. As with the Baselines, this feature comes up editable and the end points can be dragged to specific gates of velocity data. When in place, the speed difference (kts), distance between end points (nautical miles), shear (s-1), and distance from radar (Nmi) are automatically plotted next to the end points and in the upper left corner of the Main Display Pane. A positive shear value indicates cyclonic shear, while a negative value indicates anticyclonic shear. If either end point is not directly over velocity data, the phrase "no data" is reported for the shear value. This tool is also useful in determining gate-to-gate shear. Simply place the two end points directly over adjacent gates of velocity data.
@ -17,50 +91,3 @@ This tool is used in conjunction with Doppler velocity data to calculate the vel
mode. The VR - Shear overlay is loaded in different colors for each panel. There are actually
four copies of the program running, and each behaves independently. This means that you can
get accurate readings in any one of the four panels — one VR - Shear panel is editable at a time. To activate, click the center mouse button on the VR - Shear legend in the desired panel and position the query line to the echoes of interest.
## Radar Display Controls
<span style="float:right;">
![image](../images/44N8kJq.png)
</span>
The Radar Display Controls dialog box is derived from the Radar Tools submenu provides options that control the appearance of the Storm Track Information (STI), the Hail Index (HI), the Tornado Vortex Signature (TVS), and the Digital Mesocyclone Display (DMD) products, along with Storm
Relative Motion (SRM) options. The Radar Display Controls dialog box options are described below.
The Radar Display Controls dialog box is divided into six sections: STI, HI, TVS, DMD/MD/TVS, DMD, and SRM. Each section has the following options:
## STI (Storm Track Information)
This section has options to adjust the appearance of the STI graphic product.
* **Number of storms to show**: This slider bar lets you choose the maximum number of storms (0 to 100) you wish to display on the STI product. The default value is 20 storms.
* **Type of track to show**: This options menu allows you to choose the type of storm track that you want displayed.
## HI (Hail Index)
This portion of the Radar Display Controls dialog box contains options that alter the appearance of the HI radar graphic product. You can set the low and high algorithm thresholds of the Probability of Hail (POH) and the Probability of Severe Hail (POSH). Storms that meet the low POH threshold are indicated by small open triangles, while small solid triangles mark those that meet the high POH threshold. Similarly, large open triangles or solid triangles are plotted for the POSH low and high thresholds, respectively.
* **Low hail probability (POH)**: The storms that meet or exceed the threshold are indicated by small open triangles. The default setting is 30.
* **Low severe hail probability (POSH)**: The storms that meet or exceed the threshold are indicated by large open triangles. The default setting is 30.
* **High hail probability**: The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50.
* **High severe hail probability**: The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50.
## TVS (Tornado Vortex Signature)
There is one option in this section of the Radar Display Controls dialog box.
* **Show elevated TVS**: This toggle button lets you control the appearance of the elevated TVS radar graphic product.
## DMD, MD, TVS
There is one option in this section of the Radar Display Controls dialog box.
Show extrapolated Features: With this option, you can choose whether to show the time-extrapolated
features using DMD, MD, or TVS.

76
docs/cave/d2d-tools.md
View file

@ -1,9 +1,14 @@
# Display Tools
These programs are accessible though the **Tools** dropdown menu.
The display tools are a subset of the tools available in CAVE. These programs are accessible though the **Tools** dropdown menu.
![image](../images/xiXbVEr.png)
![image](../images/toolsMenuDisplayTools.png)
Many of the tools listed under the Tools menu can be placed into an editable state. Do not enable the "Hide Legends" feature if you want to place a tool in an editable state, because access to editability is by clicking the center mouse button over the Product Legend
Many of the tools listed under the Tools menu can be placed into an **editable state**. Do not enable the "Hide Legends" feature if you want to place a tool in an editable state, because access to editability is done by clicking the center mouse button, or right-clicking over the **Product Legend**.
> **Note**: To see information about some of the other options in the Tools menu, check out the [**Radar Tools**](d2d-radar-tools) page.
---
## Az/Ran Overlay
@ -11,6 +16,8 @@ This tool displays a movable azimuth/range radar map overlay. The overlay is in
![image](../images/xVAgVCCHwH.gif)
---
## Baselines
Selecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be constructed from within the Volume Browser. Baselines come up editable.
@ -19,18 +26,24 @@ Selecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-s
![image](../images/hK6RakT.png)
---
## Choose By ID
Choose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and "Home" for conventional locations like METARs and RAOBs (Radiosonde Observations), but its primary use is for the WSR-88D-identified mesocyclone locations. You can also access the Choose By ID tool from the Tools menu on the Volume Browser.
![image](../images/E6cghWV.png)
---
## Distance Bearing
Selecting this tool displays six editable lines, each of which shows the azimuth and range of the labeled end of the line relative to the unlabeled end of the line. You can make the lines editable by clicking the center mouse button over the legend at the lower right of the display. Once in edit mode, a line can be moved as a unit and/or either of its end points can be adjusted.
![image](../images/fgaCqrd.png)
---
## Distance Speed
This tool can be used to determine the speed and direction of a storm or any other meteorological feature of interest. Selecting Distance Speed displays a Centroid Marker to move to the location of the storm or feature of interest in any two or more frames of displayed imagery (e.g., a satellite or radar loop). The system then displays a storm track with the direction (degrees) and speed (knots) of movement. When you select the Distance Speed option, the Distance Speed dialog box opens.
@ -49,27 +62,30 @@ This tool can be used to determine the speed and direction of a storm or any oth
* **Speed**: A radio button that allows you to display speed with the Centroid Marker.
---
## Distance Scale
This tool can be used to determine the size of a storm or any other meteorological feature of interest.
Enabling this feature adds a scalebar to the bottom right hand of the main D2D display. This tool can be used to determine the size of a storm or any other meteorological feature of interest.
![image](../images/iYYc68NN9w.gif)
---
## Feature Following Zoom
When you zoom in over a small area to be able to view a feature in detail, animation will often cause the feature to move into and then out of the field of view. This tool allows you to follow a feature of interest even when zoomed in to a small area.
To use this feature, first, you need to identify the location and motion of the feature, using Distance Speed or the WarnGen tracker. Once satisfied that the tracking icon is following the feature of interest, load this tool, and the center of the zoom area will track with the Distance Speed icon. Toggling the overlay off will resume the standard zooming behavior, and toggling it back on will reinvoke the feature following zoom.
## Time of Arrival / Lead Time
Selecting the Time Of Arrival / Lead Time option displays a tracking line from a feature's initial starting point in a past frame to its final position in the current frame. Once the final position is set, an Arrival Point is displayed. You can drag this point anywhere along the line to get the Time Of Arrival / Lead Time and Distance. You can also change the Mode from Point to Circular Front or Polyline anywhere along the line to better represent the feature(s).
---
## Home
Selecting the Home option displays a marker, which is an "X" with the word "Home" next to it.
Clicking on the Home Location Legend with the center mouse button makes the marker editable; drag the "X" or click with the right mouse button to change its location. When the Home Marker is displayed, use the Sample feature (clock and hold while moving the pointer around the screen) to display the range in miles and azimuth (in degrees) of the pointer location relative to the Home location.
Clicking on the Home Location Legend with the center mouse button makes the marker editable; drag the "X" or click with the right mouse button to change its location. When the Home Marker is displayed, use the **Sample** feature (click and hold to access the menu to turn on sampling) to display the range in miles and azimuth (in degrees) of the pointer location relative to the Home location.
---
## Points
@ -86,24 +102,22 @@ Points may be created, deleted, hidden, and manipulated (location, name, font, a
1. **Create Point Dialog Box**: The Create Point dialog box is opened by clicking and holding the right mouse button on the map (but not on any exisiting Point) and selecting the "New Point..." option.
The Create Point dialog box opens with the Lat and Lon text boxes populated with the latitude
and longiture values at the point where you had clicked the right mouse button. The latitude and longitude values can be viewed in "Degrees : Minutes : Seconds," "Degrees : Minutes," or "Degrees Only" (N and S refer to North and South; W and E refer to West and East).
The Create Point dialog box opens with the Lat and Lon text boxes populated with the latitude and longiture values at the point where you had clicked the right mouse button. The latitude and longitude values can be viewed in "Degrees : Minutes : Seconds," "Degrees : Minutes," or "Degrees Only" (N and S refer to North and South; W and E refer to West and East).
In the Create Point dialog box, you must:
In the Create Point dialog box, you **must**:
* Enter the Point's name
And **may** do any of the following:
* Modify the latitude and longitude values
* Assign the Point's color and font use
* Assign the Point to a group
* Select whether the Point is movable or hidden
By default, individual Points do not have an assigned color. They inherit the color
of the Interactive Points layer reflected in the Interactive Points product legend. You can
change the color of the Interactive Points layer by right clicking on the Interactive Points
product legend and selecting a color from the dropdown list. The selected color then
changes all points not having an assigned color to the new color.
By default, individual Points do not have an assigned color. They inherit the color of the Interactive Points layer reflected in the Interactive Points product legend. You can change the color of the Interactive Points layer by right clicking on the Interactive Points product legend and selecting a color from the dropdown list. The selected color then changes all points not having an assigned color to the new color.
Points can be assigned to "<No Group>," which will organize them in the root location containing the group names when accessed by the Edit Points dialog box (see below).
Points can be assigned to "**<No Group\>**" which will organize them in the root location containing the group names when accessed by the Edit Points dialog box (see below).
2. **Edit Point Dialog Box**: The Edit Point dialog box is opened by clicking and holding the right mouse button on a Point on the map and selecting the "Edit Point..." option. The latitude and longitude values can be viewed in "Degrees : Minutes : Seconds," "Degrees : Minutes," or "Degrees Only" (N and S refer to North and South; W and E refer to West and East).
@ -127,6 +141,8 @@ or disassemble subgroups. The Points List dialog box also includes three columns
moved.
* **Hidden**: Checking the checkbox adjacent to the Point hides the Point on the map.
---
## Put home cursor
The Put home cursor tool provides an easy way to locate a METAR observation station, a city and
@ -141,26 +157,46 @@ Station, City/State, or Lat/Lon radio button, an Entry Box is activated next to
* **Go**: This menu button initiates the search for the desired station, city/state, or latitude/longitude. The Home marker jumps to the newly specified location.
---
## Range Rings
The Range Rings Tool displays adjustable range rings around locations of interest to your local office. When you select Range Rings from the Tools dropdown menu, the Range Rings legend appears in the Main Display Pane. The tool comes up editable, and the rangeRing dialog box opens. (Clicking B2 over the legend toggles tool editability and closes/opens the rangeRing dialog box.) Within this dialog box, you can toggle on/off any of the target locations using the square selectors. Adjust the size of the radii (in nautical miles) by typing a new value in the entry boxes associated with each location and pressing the Apply button. You can also add labels at the center of the range ring and/or at any of the radial distances using the Labels Options menu associated with each location. Using the Movable Rings, you can add a new location at a specific point by using the Interactive Points Tool, or by typing in latitude/longitude coordinates. There is no practical limit on the number of new locations you can add to the display. The list of locations is pre-set but can be customized at a field site
The Range Rings Tool displays adjustable range rings around locations of interest to your local office. When you select Range Rings from the Tools dropdown menu, the Range Rings legend appears in the Main Display Pane. The tool comes up editable, and the rangeRing dialog box opens. (Clicking the middle mouse button over the legend toggles tool editability and closes/opens the rangeRing dialog box.) Within this dialog box, you can toggle on/off any of the target locations using the square selectors. Adjust the size of the radii (in nautical miles) by typing a new value in the entry boxes associated with each location and pressing the Apply button. You can also add labels at the center of the range ring and/or at any of the radial distances using the Labels Options menu associated with each location. Using the Movable Rings, you can add a new location at a specific point by using the Interactive Points Tool, or by typing in latitude/longitude coordinates. There is no practical limit on the number of new locations you can add to the display. The list of locations is pre-set but can be customized at a field site.
![image](../images/d0HFeY8.png)
---
## Sunset/Sunrise
By typing a date, as well as the latitude and longitude of a location into the Sunrise/Sunset Tool dialog box, you can obtain the time (for any time zone) of sunrise and sunset, as well as the total length of daylight for that date. Additional features include the ability to calculate the sunrise/sunset in a different hemisphere, and the azimuthal angles, relative to true north, of the sunrise and sunset.
![image](../images/jkzIvS0.png)
---
## Text Window
Selecting this option brings up a Text Display window that behaves in the same way, except for scripts, as a window on the Text Workstation.
Selecting this option brings up a Text Display window that behaves in the same way as a window on the [Text Workstation](#text-workstation), except that the scripts menu is disabled.
---
## Time of Arrival / Lead Time
Selecting the Time Of Arrival / Lead Time option displays a tracking line from a feature's initial starting point in a past frame to its final position in the current frame. Once the final position is set, an Arrival Point is displayed. You can drag this point anywhere along the line to get the Time Of Arrival / Lead Time and Distance. You can also change the Mode from Point to Circular Front or Polyline anywhere along the line to better represent the feature(s).
---
## Units Calculator
This tool converts the units of the first column into differing units of the second column. The units are grouped into temperature, velocity, distance, time, and atmospheric pressure. First, simply type the number and select the units of the value you wish to convert in the firstcolumn entry box. Then in the second column, select the desired units to which you want the original value converted. The new value will appear in the second column entry box.
This tool converts the units of the first column into differing units of the second column. The units are grouped into temperature, speed, distance, time, and atmospheric pressure. First, simply type the number and select the units of the value you wish to convert in the firstcolumn entry box. Then in the second column, select the desired units to which you want the original value converted. The new value will appear in the second column entry box.
![image](../images/pdSH69h.png)
---
## Text Workstation
By selecting one of the "Text" buttons, a text window opens up. In National Weather Service operations, the text workstation is used to edit new warning text as well as look up past warnings, METARs, and TAFs. This functionality is **disabled** in the Unidata AWIPS version.
![](../images/textWorkstation.png)

188
docs/cave/goes-16-satellite.md
View file

@ -1,26 +1,46 @@
# GOES 16/17
The GOES-R decoder supports the ingest and display of NOAAport provisional products (currently on the NOTHER feed), as well as Level 2b netCDF files. A GOES-R Geostationary Lightning Mapper (GLM) decoder is also provided to handle sensor lightning observations.
The GOES-R decoder supports the ingest and display of NOAAport products (currently on the **NIMAGE** feed), Derived products (Level 2b netCDF files), and the Geostationary Lightning Mapper (GLM) products.
GOES-R products are accessible in the menu **Satellite** > **GOES-16 Provisional Products**
GOES-R products are accessible in the **Satellite** menu. The menu is broken into sections and has submenus for each of the separate geospatial products: CONUS, Full Disk, Mesoscale Sectors, Hawaii, Alaska, and Puerto Rico. Each submenu has products for [**individual channels**](#individual-channels) and [**RGB Composites**](#rgb-composites), as well as submenus for [**channel differences**](#channel-differences) and [**derived products**](#derived-products). [**GLM data**](#geostationary-lightning-mapper-glm) can also be found with its own submenu option a little lower down the menu and under the **Surface** menu.
![](../images/goes_16_menu.png)
> **NOTE**: The RGB products are not available on MacOS.
![](../images/satelliteMenu.png)
---
## NOAAport Provisional Products
## LDM Pattern Actions
### LDM Pattern Action
This is found in the LDM's pqact.conf file which is located in /awips2/ldm/etc. The entries for GOES data are shown below. There are three different feeds that supply the individual channel data, derived products, and GLM data: NIMAGE, HDS and DIFAX, respectively.
NOTHER ^(TI[RS]...) (KNES) (......) (...)
FILE -close -edex
/awips2/data_store/satellite/goes16/\1_\2_\3_\4_(seq).gini
#
# GOES 16/17 ABI
#
NIMAGE ^/data/ldm/pub/native/satellite/GOES/([^/]*)/Products/CloudAndMoistureImagery/([^/]*)/([^/]*)/([0-9]{8})/([^/]*).nc
FILE -close -edex
/awips2/data_store/GOES/CMI/\5.nc4
#
# GOES derived products
#
HDS ^(IXT.99) KNES (......)
FILE -close -edex
/awips2/data_store/GOES/derived/KNES_\1_\2-(seq)
#
# GLM
#
DIFAX ^/data/cspp-geo/(EAST|WEST|GRB-R)/OR_GLM-L2-([^/]*).nc
FILE -close -edex
/awips2/data_store/GOES/GLM/\1_OR_GLM-L2-\2.nc
### Puerto Rico sector (PRREGI)
---
![](../images/goes_16_pr.png)
## Individual Channels
### CONUS 1km
All geospatial sectors have individual channel products that can be viewed. Below are samples of Channel 02 (0.64&mu;m) for each of the sectors. These products come through the **NIMAGE** feed in the LDM.
### CONUS 1km
![](../images/goes_16_conus.png)
@ -34,98 +54,136 @@ Two floating mesoscale sectors (will vary from image shown)
![](../images/goes_16_meso.png)
### Puerto Rico (PRREGI)
![](../images/goes_16_pr.png)
### Alaska
![](../images/goes_17_ak.png)
### Hawaii
![](../images/goes_17_hi.png)
---
## RGB Composites
RGB Composites are also available for each sector. The three RGB products are: Icing - Ch 5, 3, 2 (1.61&mu;m, 0.87&mu;m, 0.64&mu;m), Composite - Ch 2, 5, 14 #1 (0.64&mu;m, 1.61&mu;m, 11.20&mu;m) and Composite #5 - Ch 2, 3, 2 (0.64&mu;m, 0.87&mu;m, 0.64&mu;m). These products are generated **on the fly in AWIPS** using the existing channel products from EDEX.
!!! warning "GOES RGB Imagery is NOT SUPPORTED on macOS"
OpenGL Shading Language limitations prevent multi-channel imagery from displaying correctly on Mac. Please use the Linux or Windows installs to view RGB products.
### Icing Composite
Channels 5,3,2 (1.61u, 0.87u, 0.64u)
![](../images/goes_16_comp1.png)
### Daytime Composite 1
Channels 2,5,14 (0.64u, 1.61u, 11.20u)
![](../images/goes_16_comp2.png)
### Daytime Composite 2
Channels 2,3,2 (0.64u, 0.87u, 0.64u)
### Daytime Composite 5
![](../images/goes_16_comp3.png)
---
!!! warning "GOES-16 Composite Imagery NOT SUPPORTED on macOS"
## Channel Differences
OpenGL Shading Language limitations prevent multi-channel imagery from displaying correctly on Mac:
* Icing (1.61, 0.87, 0.64)
* Daytime Composite (0.64, 1.61, 11.20)
* Daytime Composite (0.64, 0.87, 0.64)
Channel differences are the result of subtracting one channel from another to produce a new product. These products are generated **on the fly in AWIPS** using the existing channel products from EDEX.
To display multi-channel composites requires CAVE for Linux or Windows.
>**NOTE**: These may not be available for all sectors.
### HDF5 Data Store
There currently 10 channel differences that are offered in CAVE:
Decoded GOES-R satellite images are stored in `/awips2/edex/data/hdf5/satellite/` under sector subdirectories:
* Split Window (10.3 - 12.3 &mu;m)
* Split Cloud Top Phase (11.2 - 8.4 &mu;m)
* Night Fog (10.3 - 2.9 &mu;m)
* Day Fog (3.9 - 10.3 &mu;m)
* Split Fire (2.2 - 1.6 &mu;m)
* Split Ozone (9.6 - 10.3 &mu;m)
* Split Water Vapor (6.19 - 7.3 &mu;m)
* Split Snow (1.6 - 0.64 &mu;m)
* Vegetation (0.64 - 0.87 &mu;m)
* Upper Level Info (11.2 - 6.19 &mu;m)
drwxr-xr-x 18 awips fxalpha PRREGI
drwxr-xr-x 18 awips fxalpha ECONUS
drwxr-xr-x 18 awips fxalpha EFD
drwxr-xr-x 18 awips fxalpha EMESO-1
drwxr-xr-x 18 awips fxalpha EMESO-2
> **NOTE**: The rendering of these products uses the **Jep** package in Python, which has known to have issues installing on the MacOS and Windows versions.
---
## Derived Products
Derived products are also known as **Level 2+** products and are come through the **HDS** feed in the LDM. There are over 20 products currently available in AWIPS. To find out some more information on some of the products please the [**Quick Guides**](http://rammb.cira.colostate.edu/training/visit/quick_guides/) compiled by CIRA.
>**NOTE**: These may not all be available for each sector.
The current products offered in CAVE are:
* Aerosol Detection
* Aerosol Optical Depth
* Clear Sky Mask
* Cloud Optical Depth
* Cloud Particle Size
* Cloud Top Height
* Cloud Top Phase
* Cloud Top Pressure
* Cloud Top Temperature
* Derived CAPE
* Derived K-Index
* Derived Lifted Index
* Derived Showalter Index
* Derived Total Totals
* Fire Area
* Fire Power
* Fire Temperature
* Land Skin Temperature
* RR/QPE
* Sea Surface Temperature
* Snow Cover
* Total Precip Water
* Ash Cloud Height
* Ash Mass Load
---
## Level 2+ Products
## HDF5 Data Store
Level 2+ products are described as derived environmental variables which will be created and disseminated when GOES-16 is used operationally (compared to **Level 0**, described as unprocessed instrument data at full resolution, and **Level 1b** products, described as radiometric and geometric correction applied to produce parameters in physical units).
Decoded GOES-R satellite images are stored in `/awips2/edex/data/hdf5/satellite/` under sector subdirectories:
Unidata does not currently have access to these products, but EDEX can support their ingest if a feed is available.
drwxr-xr-x. 18 awips fxalpha AKREGI
drwxr-xr-x. 235 awips fxalpha ECONUS
drwxr-xr-x. 38 awips fxalpha EFD
drwxr-xr-x. 30 awips fxalpha EMESO-1
drwxr-xr-x. 30 awips fxalpha EMESO-2
drwxr-xr-x. 18 awips fxalpha HIREGI
drwxr-xr-x. 18 awips fxalpha PRREGI
drwxr-xr-x. 18 awips fxalpha WCONUS
drwxr-xr-x. 18 awips fxalpha WFD
drwxr-xr-x. 18 awips fxalpha WMESO-1
drwxr-xr-x. 18 awips fxalpha WMESO-2
[Read more about GOES-R data levels...](http://www.goes-r.gov/ground/overview.html)
### Level 2+ Decoder Regular Expression
From `/awips2/edex/data/utility/common_static/base/distribution/goesr.xml`
^OR_ABI-L2-\w{3,5}(C|F|M1|M2)-M[34]_G\d\d_s\d{14}_e\d{14}_c\d{14}.nc$
---
## Geostationary Lightning Mapper (GLM)
NASA's SPoRT MSFC Earth Science Office has contributed plugins to decode GLM level2 products, displayed as point data in CAVE.
While Unidata is not currently distributing GLM products, you can [download a sample netCDF file](https://www.unidata.ucar.edu/software/awips2/OR_GLM-L2-LCFA_G16_s20170402339144_e20170402339307_c20170402339509.nc) and copy it to `/awips2/data_store/ingest/` in order to test the decoding and display of GOES-R lightning data.
GLM data is located in the menu structure: **Satellite** > **Geostationary Lightning Data (GLM)**. Data are displayable is available for **Flash**, **Event**, and **Group** products.
### Ingest Sample Data
There is also additional GLM data available in the **Surface** > **GLM - Geostationary Lightning Mapper** submenus.
Run the single command from your EDEX server to ingest a single-time sample data set:
![Satellite GLM Menu](../images/goes_16_glmmenu.png)
wget https://www.unidata.ucar.edu/software/awips2/OR_GLM-L2-LCFA_G16_s20170402339144.nc -P /awips2/data_store/ingest/
### Display GLM Lightning Data
Load GLM data from the menu **Satellite** > **GOES-16 Provisional** > **GLM-Lightning**. Data are displayable as **Flash**, **Event**, and **Group** and 1min, 5min, 15min, and 1hr intervals.
![](../images/goes_16_glmmenu.png)
![Surface GLM Menu](../images/surfaceGLMMenu.png)
![](../images/goes_16_glm.png)
<!---
---
## Derived Motion Winds
At the bottom of the menu **Satellite** > **GOES-16 Provisional Products** are sub-menus for GOES-16 Derived Motion Wind (DMW) products. Select the sub-menu **GOES-Test** and then region (CONUS, Full Disk, Mesoscale), and then select the wind product by level:
* By Pressure
* By Mandatory Levels
* By Channel
![](../images/goesr_dmw.png)
--->

57
docs/cave/import-export.md
View file

@ -1,21 +1,18 @@
# Import/Export
## Export Images
## Export Images/GIFs
**File > Export > Image**
The D2D screen can be exported as a PNG image as well as an animated GIF using the **File > Export > Image** menu option. This captures the current state of the screen, and allows you to set animation options (frame number, dwell time, etc) for exporting GIFs. If you choose to animate, you will either need to rename the destination file to have the *.gif* extension, or CAVE will pop up a dialog when you go to save, asking you to confirm that you want to output a GIF.
which captures a screenshot of the current view
> **Note**: This functionality **does not** currently work on **Mac OS** because it implements OGL libraries which are not compatible on Mac.
![image](../images/UxmboZS.png)
![image](../images/exportImageDialog.png)
---
## Export KML
**File > Export > KML**
The "Export" submenu also includes a "KML" option, which allows users to save D2D displays or
GFE grids in the KML (Keyhole Markup Language) file format. When zipped (compressed), the KML
file format forms a KMZ file, which can be used in applications such as Google Earth.
The *Export* submenu also includes a *KML* option (**File > Export > KML**), which allows users to save D2D displays or GFE grids in the KML (Keyhole Markup Language) file format. When zipped (compressed), the KML file format forms a KMZ file, which can be used in applications such as Google Earth.
![image](../images/YRP9kOM.png)
@ -37,38 +34,28 @@ tiles when loaded in Google Earth.
## CAVE Import Formats
CAVE supported the following geo-referenced data files. CAVE can import the following through formats through the **CAVE** -> **Import** menu.
CAVE supported the following geo-referenced data files. CAVE can import the following through formats through the **File > Import** menu.
1. **GIS Data**
1. **Background...**
2. **Image...**
3. **BCD File**
4. **GeoTIFF**
5. **LPI File**
6. **SPI File**
7. **Displays**
2. **BCD File**
3. **GeoTIFF**
4. **LPI File**
5. **SPI File**
6. **Displays**
![image](../images/image_14.png)
![image](../images/importMenu.png)
---
## CAVE Export Formats
CAVE can export to the following through the **CAVE** -> **Export** menu.
CAVE can export to the following through the **File > Export** menu.
1. **KML**
1. **Image**
2. **Print Screen**
3. **KML**
4. **Editor Display...**
5. **Perspective Displays...**
2. **Editor Display**
3. **Perspective Display**
4. **Capture Current Frame**
5. **Capture All Frames**
6. **Print Screen**
![image](../images/image_15.png)
![image](../images/exportMenu.png)

25
docs/cave/localization-perspective.md
View file

@ -1,7 +1,7 @@
# Localization perspective
## Localization Levels
AWIPS uses a hierarchical system known as *Localization* to configure many aspects of EDEX and CAVE, such as available menu items, color maps, and derived parameters. This system allows a user to override existing configurations and customize CAVE. For example, a *User-level* localization file will supercede any similar file in a lower level (such as *Site*).
AWIPS uses a hierarchical system known as *Localization* to configure many aspects of EDEX and CAVE, such as available menu items, color maps, and derived parameters. This system allows a user to override existing configurations and customize CAVE. For example, a _**User**-level_ localization file will supercede any similar file in a higher level (such as **Site**).
!!! note "There are three *levels of localization*, starting with the default **BASE**"
@ -13,9 +13,9 @@ AWIPS uses a hierarchical system known as *Localization* to configure many aspec
## Localization Editor
The Localization Perspective acts as file editor for the XML, Python, and text files which customize the look and feel of CAVE.
The Localization Perspective acts as file editor for the XML, Python, and text files which customize the look and feel of CAVE. This perspective is available in the menu **CAVE > Perspective > Localization**.
Users may copy and add files to available directories at their own *User* localization version.
Users may copy and add files to available directories at their own **User** localization version.
Examples of things that can be accessed through the perspective include (this list is not all-inclusive):
@ -39,15 +39,17 @@ There may be several versions of each file including **BASE**, **CONFIGURED** (G
The **_File Editor_** view opens the selected configuration file in an appropriate editor. For example, a Python file is opened in a Python editor, and an XML file is opened in an XML editor.
--
---
## Customizing CAVE Menus
Navigate to **CAVE** > **Menus** and select a submenu (e.g. **satellite**). This directory lists all of the menu file contributions made by this data plugin. Most data menu directories will have an `index.xml` file from which you can investigate the menu structure and made needed changes.
Navigate to **D2D > Menus** and select a submenu (e.g. **satellite**). This directory lists all of the menu file contributions made by this data plugin. Most data menu directories will have an `index.xml` file from which you can investigate the menu structure and make needed changes.
Selecting a file such as **index.xml** will show a sub-menu with a default localization level (typically **BASE** or **CONFIGURED**). Double-click this tab to open in the file editor (you may need to click **Source** at the bottom of the view to see the raw XML). Right-click this tab and select **Copy To** > **User (awips)** and you will see the file localization versions update with the new copy. Select this file to edit, and override, the existing version.
Selecting a file such as `index.xml` (by double clicking, or expanding) will show a sub-menu with a default localization level (typically **BASE** or **CONFIGURED**). Double-click this file to open in the file editor (you may need to click **Source** at the bottom of the view to see the raw XML). Right-click this file and select **Copy To** > **User (_username_)** and you will see the file localization versions update with the new copy. Select this file to edit, and override, the existing version.
--
![copy to](../images/copyToMenu.png)
<!---
## Add new Predefined Area to NCP
@ -55,9 +57,9 @@ In the Localization Perspective, navigate to ‘**NCEP - Predefined Area Menus**
![image alt text](../images/image_4.png)
You will see a new ‘**USER (username)**’ entry for the file **AreaMenus.xml**.
You will see a new ‘**USER (username)**’ entry for the file **AreaMenus.xml**.
![image alt text](../images/image_5.png)
![image alt text](../images/image_5.png)
Double-click the new file and copy or create a new AreaMenuItem entry, such as a new CONUS Mercator projection called **CONUS_Mercator**:
@ -71,7 +73,7 @@ Double-click the new file and copy or create a new AreaMenuItem entry, such as a
![image alt text](../images/image_6.png)
Now you can either copy and post an existing xml area file, or simple create one in the NCP. Switch back to the NCP, and select ‘**Area - World**’ to load a known Mercator projection. Zoom in to the continental United States and then from the menu bar select ‘**File - Save Area**’ and name it **CONUS_Mercator** so it matches the name string given in the above AreaMenuItem.
Now you can either copy and paste an existing xml area file, or simple create one in the NCP. Switch back to the NCP, and select ‘**Area - World**’ to load a known Mercator projection. Zoom in to the continental United States and then from the menu bar select ‘**File - Save Area**’ and name it **CONUS_Mercator** so it matches the name string given in the above AreaMenuItem.
![image alt text](../images/image_7.png)
@ -95,3 +97,4 @@ On your workstation you can find the new USER localization files in ~/caveData
./etc/user/mjames/ncep/PredefinedAreas/CONUS_Mercator.xml
./.localization/NCEP/Predefined Areas/CONUS_Mercator.xml
-->

36
docs/cave/maps-views-projections.md
View file

@ -1,20 +1,32 @@
# Maps, Views, Projections
## Default Map Scales
The first toolbar menu item is a dropdown menu for different geographic areas and map projections. The efault view is always **CONUS**, which is a North Polar Steregraphic projection centered on the Continental United States.
The first toolbar menu item is a dropdown menu for different geographic areas and map projections. The default view is always **CONUS**, which is a North Polar Steregraphic projection centered on the Continental United States.
Default projections and areas available in the menu
* **CONUS**
* **N. Hemisphere** (North Polar Stereographic)
* **Regional** (for the selected localization site)
* **WFO** (for the selected localization site)
* **World Mercator**
* **World CED**
* **World Mollweide**
* **Geostationary (GOES East & West Full Disk)**
* **Regional** Mercator projections for **Africa**, **Australia/NZ**, **South America**, **Europe**, **Alaska**, **Hawaii**, **Puerto Rico**, **Japan**, **Pacific Ocean**, and the north and south poles.
There is also a **WFO** submenu which contains a map scale for every NWS localization site.
* **World - Mercator**
* **World - CED**
* **World - Mollweide**
* **GOES East Full Disk** (Geostationary)
* **GOES West Full Disk** (Geostationary)
* **Regional** Mercator projections for
- **Africa**
- **Alaska**
- **Antarctica**
- **Arctic**
- **Australia,New Zealand**
- **Europe**
- **Hawaii**
- **Japan**
- **Pacific Ocean**
- **Puerto Rico**
- **South America**
* **WFO** (Has a submenu which contains a map scale for every NWS localization site)
![image](../images/map_scales.png)
@ -25,9 +37,9 @@ There is also a **WFO** submenu which contains a map scale for every NWS localiz
**File > New Map**
Opens a new map editor tab with the default projection (CONUS Polar Stereographic).
Opens a new map editor tab with the default projection (CONUS Polar Stereographic).
![image](../images/67Wwz3L.png)
![image](../images/fileMenuNewMap.png)
This can also be done by **right-click** on any tab and selecting **New Editor**
@ -41,4 +53,4 @@ This can also be done by **right-click** on any tab and selecting **New Editor**
Create a new map projection.
![image](../images/VANzMW2.png)
![image](../images/createProjection.png)

93
docs/cave/nsharp.md
View file

@ -1,67 +1,108 @@
# NSHARP
NSHARP, which stands for the **N**ational Center **S**ounding and **H**odograph **A**nalysis and **R**esearch **P**rogram, is an AWIPS plugin originally based on NAWIPS NSHAREP, SPCs *BigSHARP* sounding display tool, and the Python package [SHARpy](https://github.com/aeroelastics/SHARPy).
NSHARP, which stands for the **N**ational Center **S**ounding and **H**odograph **A**nalysis and **R**esearch **P**rogram, is an AWIPS plugin originally based on NAWIPS NSHAREP, SPCs *BigSHARP* sounding display tool, and the Python package [SHARpy](https://github.com/aeroelastics/SHARPy).
NSHARP is available a number of ways in CAVE:
* From the **D2D toolbar** select the NSHARP icon
* From the **Upper Air** menu select **NSHARP Soundings**
* From the **Upper Air** menu select a station from the RAOB menus
* From the **Upper Air** menu select **NUCAPS Soundings**
* From within **Models &gt; Volume Browser**
* From the **Maps** menu select **Sounding Locs** to
* From the **D2D toolbar** select the NSHARP icon: ![](../images/nsharpIcon.png)
* From the **Upper Air** menu select **NSHARP Soundings**
* From the **Upper Air** menu select a station from the _RAOB_ menus
* From the **Upper Air** menu select **NUCAPS Soundings**
* From the **Models** or **Tools** menu select **Volume Browser**
- Make sure **Sounding** is selected from the menu at the top
![](../images/volumeBrowserSoundingMenu.png)
- Select a source that has data (signified by a green box to the right)
![](../images/volumeBrowserSources.png)
- Select **Soundings** from the Fields menu
![](../images/volumeBrowserFields.png)
- Select any point from the Planes menu and an option will load in the table
- To create a new point go to select **Tools** > **Points** and use the right-click-hold menu to create a new point anywhere on the map
- Use the **Load** button to load data and open the NSharp display
![](../images/volumeBrowserLoad.png)
---
## NSHARP Configurations
NSHARP has four configurations for use in different operational settings:
* **SPC Wide** - more insets and graphs at the expense of timeline/station inventory.
* **D2D Skewt Standard** - default for WFOs, larger SkewT with inventory, no Wind/Height, temperature advection, insets, or graphs.
* **D2D Lite** - Skew-T, table, and inventory only.
* **OPC** - Ocean Prediction Center display.
> If you would like to interactively explore the different graphical areas in&nbsp;NSHARP&nbsp;<strong>on the Web</strong>, see the&nbsp;<a href="http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html" style="font-weight: 200;">NSHARP Interactive Overview</a>
**To change the NSHARP confiuguration:**
!!! tip "To change the NSHARP confiuguration:"
* Click the **Configure** button
* Click **Display Pane Configuration** (second from bottom)
* Choose configuration, apply, save, close
* Open the **NSHARP(D2D)** controls tab by clicking on the Nsharp toolbar (![](../images/nsharpIcon.png)) icon again
* Click the **Configure** button
* Click **Display Pane Configuration** (third from the bottom)
* Use the dropdown to choose a configuration, apply, save, close
If you would like to interactively explore the different graphical areas in NSHARP **on the Web**, see the [NSHARP Interactive Overview](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html).
---
## Skew-T Display
The Skew-T display renders a vertical profile of temperature, dew point, and wind for RAOBs and model point soundings using a Skew-T Log-P diagram. Skew-T is the default upper air chart in AWIPS, and can be changed to *turbulence display* ("T") or an *icing display* ("I").
![](../images/nsharpSkewT.png)
The upper-left red box is linked to the cursor readout when over the SkewT chart, reported as temperature, dewpoint, wind direction and speed, pressure, height AGL, and RH of the trace.
The Skew-T display renders a vertical profile of temperature, dew point, and wind for RAOBs and model point soundings using a Skew-T Log-P diagram. The box in the upper-left of the main display is linked to the cursor readout when over the SkewT chart. It reports the temperature, dewpoint, wind direction and speed, pressure, height AGL, and relative humidity of the trace.
Skew-T is the default upper air chart in AWIPS, and can be changed to *turbulence display* (**T**) or an *icing display* (**I**). These options are available as buttons at the bottom of the NSHARP(D2D) controls tab (mentioned in [NSHARP Configurations](#nsharp-configurations)). Use the [AWIPS-2 NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information about the Skew-T display.
---
## Windspeed vs Height and Inferred Temperature Advection
The windspeed vs height and inferred temperature advection with height plot (referred to by label "iii" in the NSHARP schematic images) is situated next to the SkewT to show the values at the same heights. Inferred temperature advection is from the thermal wind. Use the AWIPS-2 NSHARP Interactive Overview page for more information about the Skew-T display.
![](../images/nsharpWindHeightTemp.png)
The windspeed vs height and inferred temperature advection with height plot is situated next to the SkewT to show the values at the same heights. Inferred temperature advection is from the thermal wind. Use the [AWIPS-2 NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information.
---
## Hodograph Display
This panel contains the hodograph display from the sounding data (referred to by label "iv" in the NSHARP schematic images). The rings in the hodograph represent the wind speed in 20 knot increments. The hodograph trace uses different colors to highlight wind observations in 3 km height increments. This display also contains information such as the mean wind, Bunkers Left/Right Moving storm motion, upshear and downshear Corfidi vectors, and a user-defined motion. Use the AWIPS NSHARP Interactive Overview page for more information about the hodograph display.
![](../images/nsharpHodograph.png)
This panel contains the hodograph display from the sounding data. The rings in the hodograph represent the wind speed in 20 knot increments. The hodograph trace uses different colors to highlight wind observations in 3 km height increments. This display also contains information such as the mean wind, Bunkers Left/Right Moving storm motion, upshear and downshear Corfidi vectors, and a user-defined motion. Use the [AWIPS NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information about the hodograph display.
---
## Insets
In the SPC Wide Screen Configuration there are four small insets beneath the hodograph containing storm-relative windspeed versus height, a Storm Slinky, Theta-E vs Pressure, Possible Watch Type, Thea-E vs Height, and storm-relative wind vectors (referred to by label "v" in the NSHARP schematic images). There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the four boxes. Use the AWIPS NSHARP Interactive Overview page for more information on the tables and a list/definition of the parameters available.
![](../images/nsharpInsets.png)
In the SPC Wide Screen Configuration there are four small insets beneath the hodograph containing storm-relative windspeed versus height, a Storm Slinky, Theta-E vs Pressure, Possible Watch Type, Thea-E vs Height, and storm-relative wind vectors. There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the four boxes. There are also two buttons **PvIn** and **NxIn** in the control tab that can be used to cycle through the previous and next inset.
<center>![](../images/nsharpControlsInsets.png)</center>
Use the [AWIPS NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information.
---
## Table Output Displays
The Table Output Displays (referred to by label "vi" in the NSHARP schematic images) contains five different pages of parameters ranging from parcel instability to storm relative shear to severe hazards potential. Use the AWIPS NSHARP Interactive Overview page for more information on the tables and a list/definition of the parameters available.
![](../images/nsharpTables.png)
The Table Output Displays contains five different pages of parameters ranging from parcel instability to storm relative shear to severe hazards potential. There are two buttons **PtDt** and **NxDt** in the controls tab that can be used to cycle through the previous and next tables.
<center>![](../images/nsharpControlsTables.png)</center>
Use the [AWIPS NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information on the tables and a list/definition of the parameters available.
---
## Graphs/Statistics
In the SPC Wide Screen Configuration there are two graphs boxes under the insets (referred to by label "vii" in the NSHARP schematic images), and they can display information on Enhanced Bulk Shear, Significant Tornado Parameter, Significant Hail Parameter (SHIP), Winter Weather, Fire Weather, Hail model (not implemented), and the Sounding Analog Retrieval System (SARS). There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the two boxes. Use the AWIPS NSHARP Interactive Overview page for more information on the tables and a list/definition of the parameters available.
![](../images/nsharpGraphs.png)
In the SPC Wide Screen Configuration there are two graphs boxes under the insets, and they can display information on Enhanced Bulk Shear, Significant Tornado Parameter, Significant Hail Parameter (SHIP), Winter Weather, Fire Weather, Hail model (not implemented), and the Sounding Analog Retrieval System (SARS). There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the two boxes.
<center>![](../images/nsharpControlsGraphs.png)</center>
Use the [AWIPS NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information.
---
## Sounding Inventory
This section (referred to by label "viii" in the NSHARP schematic images) controls the inventory of the soundings that have been loaded for potential display in NSHARP. The different colors of the text represent variously that a sounding/station is being displayed, available for display, or not available for display. Use the AWIPS NSHARP Interactive Overview page for more information on how to use the sounding inventory and time line.
![](../images/nsharpInventory.png)
This section controls the inventory of the soundings that have been loaded for potential display in NSHARP. The different colors of the text represent variously that a sounding/station is being displayed, available for display, or not available for display. Use the [AWIPS NSHARP Interactive Overview page](http://www.wdtd.noaa.gov/buildTraining/nsharp-interactive/interactive.html) for more information on how to use the sounding inventory and time line.

117
docs/cave/warngen.md
View file

@ -1,96 +1,116 @@
WarnGen is an AWIPS graphics application for creating and issuing warnings as is done by National Weather Service offices. In the Unidata AWIPS release it is a *non-operational* forecasting tool, meaning it allows users to experiment and simulate with the drawing and text-generation tools, but prevents you from transmitting a generated warning upstream.
# WarnGen Walkthrough
WarnGen is an AWIPS graphics application for creating and issuing warnings as is done by National Weather Service offices. In the Unidata AWIPS release it is a *non-operational* forecasting tool, meaning it allows users to experiment and simulate with the drawing and text-generation tools, but **prevents you from transmitting a generated warning upstream**.
!!! warning "In order to select a feature it must be within your *CAVE localization* coverage (load **Maps** > **County Warning Areas** to see coverages)"
Steps involved in using WarnGen in Unidata AWIPS CAVE
## Quick Steps - Using WarnGen in Unidata AWIPS CAVE
* Load NEXRAD Display from the Radar menu.
* Choose a WFO site with active severe weather (LKZ is used in the video).
* Re-localize to this site in the CAVE &gt; Preferences &gt; Localization menu.
* Load NEXRAD Display again and select a WSR-88D site to load a 2-panel display of 0.5 degree Reflectivity (N0Q) and Velocity (N0U).
* Click the WarnGen toolbar button or load from Tools &gt; WarnGen.
* Drag the storm marker to the center of a storm feature.
* Step through frames back and forth and adjust the marker to match the trajectory of the storm feature.
* From the WarnGen dialog select the type of warning to generate, the time range, the basis of the warning, and any threats to add (wind, hail, etc).
* Click "Create Text" at the bottom of the WarnGen dialog to generate a text warning product in a new window.
* Click "Reset" at the top of the WarnGen dialog to reset the storm marker at any time.
* Select "Line of Storms" to enable a two-pointed vector which is to be positioned parallel to a storm line.
1. [**Load NEXRAD Display**](#load-nexrad-level-3-display) from the Radar menu
2. Choose a CWA with active severe weather (PAH is used in the video below)
3. [**Re-localize**](#select-site-localization) to this site in the **CAVE** &gt; **Preferences** &gt; **Localization** menu
4. Exit out of CAVE and reload (you should notice the new CWA at the top of CAVE)
5. [**Load radar data**](#load-single-radar-data-from-the-local-radars) from the local radar menu **kpah** &gt; **Z + SRM8**
6. Use the "period" key in the number pad to toggle between the 0.5 Reflectivity and SRM
7. Click [**WarnGen**](#launch-warngen) toolbar button or load from **Tools** &gt; **WarnGen**
8. [**Drag the storm marker**](#generate-a-storm-motion-vector) to the center of a storm feature
9. Step through frames back and forth and adjust the marker to match the trajectory of the storm feature
10. Click **Track** in the Warngen GUI to update the polygon shape and trajectory
11. From the WarnGen dialog select the type of warning to generate, time range, basis of the warning, and any threats (wind, hail, etc)
12. Click [**Create Text**](#text-window) at the bottom of the WarnGen dialog to generate a text warning product in a new window
> Note: Since you are not "issuing" the warning, leave the top to rows blank ("TTAAii" and "CCCC") and Click "Enter" and a separate text window should open
![](../images/warngenHeaderBlock.png)
<iframe width="700" height="450"
src="https://www.youtube.com/embed/KSKmIMjkVCU?autoplay=1">
13. Click [**Reset**](#redrawing-a-polygon) at the top of the WarnGen dialog to reset the storm marker at any time
14. Select **Line of Storms** to enable a two-pointed vector which is to be positioned parallel to a storm line
15. To [**add another vertex**](#add-and-remove-vertex-points), middle button click along the polygon
## Video - Using WarnGen in AWIPS
The video below walks through creating a warning polygon and text in AWIPS. More detailed information can be found in the text below the video.
<iframe width="700" height="350"src="https://www.youtube.com/embed/hjytnJOxrOg?autoplay=0&rel=0" title="Tutorial: Using WarnGen in AWIPS" frameborder="0" allowfullscreen>
</iframe>
## Load NEXRAD level 3 display
Select the menu **Radar** > **NEXRAD Display** and note coverage areas of current severe weather. We choose a WFO ID that contains some active severe weather (KLZK Little Rock, Arkansas, in this example).
Select the menu **Radar** > **NEXRAD Display** and note coverage areas of current severe weather. We choose a CWA ID that contains some active severe weather (PAH Paducah, Kentucky, in this example).
## Select SITE Localization
Open **CAVE** > **Preferences** > **Localization**, select the WFO ID (KLZK) for the coverage area you want to use, followed by **Apply** and then **Restart**.
Open **CAVE** > **Preferences** > **Localization**, select the CWA site ID (PAH) for the coverage area you want to use, followed by **Apply** and **Okay** and restart CAVE. Once CAVE is restarted, you should notice the new CWA at the top of the CAVE window.
## Select the nearest WSR-88D location
![](../images/warngenLocalization.png)
Click on any NEXRAD Display station to load a two-pane Reflectivity/Velocity view.
## Load single radar data from the local radars
Click on the local radar **kpah** &gt; **Z + SRM8**. Use the "period" key in the number pad to toggle between the 0.5 Reflectivity and SRM.
![](../images/warngenLocalRadar.png)
## Launch WarnGen
Select **WarnGen** from the D2D Toolbar or from the **Tools** menu. When started, the storm centroid marker appears and the WarnGen GUI will pop up as a separate window.
| | |
:--------------------------:|:---------------------------------:|
|![](../images/Warngen_Basic.png) | ![](../images/warngen.png) |
Select **WarnGen** from the D2D Toolbar or from the **Tools** &gt; **WarnGen** menu. When started, the storm centroid marker appears and the WarnGen GUI will pop up as a separate window.
![](../images/warngen.png)
## Generate a Storm Motion Vector
1. Click and drag **Drag Me to Storm** to the feature you want to track (WarnGen uses a dot to track a single storm and a line to track a line of storms).
2. Step back 3 to 4 frames.
3. Drag the dot to the previous position of the feature you first marked.
4. Review the product loop and make adjustments to ensure the vector is accurate.
3. Drag the dot to the previous position of the feature you first marked to create the storm motion vector.
4. Click the **Track** button in the WarnGen GUI to update the polygon based off the storm motion.
5. Review the product loop and make adjustments to ensure the vector is accurate.
## Restart
![](../images/warngenDMTS.png)
Click the **Reset** button to clear all current polygons and vectors and reset the storm centroid marker.
## Redrawing a Polygon
Select the **Track** button in the **"Redraw Box on Screen"** section to redraw a default warning polygon oriented along the storm motion track and containing the storm centroid dot for the most recent volume scan. The initial polygon may have unhatched areas that will be removed from the warning due to crossing CWAs or not meeting area thresholds in the county for inclusion. The Warned/Hatched Area button allow you to preview the polygon shape that will be issued, so you can make further edits.
The initial polygon may have unhatched areas that will be removed from the warning due to crossing CWAs or not meeting area thresholds in the county for inclusion. The Warned/Hatched Area button allows you to preview the polygon shape that will be issued, so you can make further edits.
## Moving Vertex Points
Vertices can be moved by clicking and dragging with the mouse. The warning polygon, including stippling, will update automatically.
![](../images/warnpoly.png)
![](../images/warngenPoly.png)
When reshaping your warning polygon in this manner, the philosophy is to include all areas that are at risk of experiencing severe weather covered by that warning type. Effective polygons account for uncertainty over time and typically widen downstream. There will be a lot of training provided on Storm-Based Warning Fundamentals in RAC and in the workshop, and this lesson is more focused on the basic mechanics.
When reshaping your warning polygon in this manner, the philosophy is to include all areas that are at risk of experiencing severe weather covered by that warning type. Effective polygons account for uncertainty over time and typically widen downstream.
## Add and Remove Vertex Points
There will be some occasions where you will want to add vertices to your warning polygon. Most often, these situations will involve line warnings with bowing segments or single storm warnings where you want to account for storm motion uncertainty or multiple threat areas that may have differing storm motions.
New vertices are added to the warning polygon using a context relative menu accessed by selecting the warning polygon line segments with a **Right Mouse Button "click and hold"**.
1. Hold right-click and select **add vertex**.
2. Click the left mouse button at the location you desire for the new vertex and the point is created there.
3. Clicking the middle mouse button (or scroll wheel) along an edge of the polygon will also place a vertex at that point.
New vertices are added to the warning polygon two ways. Either by **Right Mouse Button** "click and hold" or a simple **Middle Mouse Button** click on the warning polygon line segment where you want to add the vertex.
Vertex points are removed from the warning polygon using the same context relative menu. Instead of selecting a line segment, you select the vertex you wish to remove and then **click and hold** with right mouse button. Hold right-click and select **remove vertex**.
Vertex points are removed from the warning polygon using the same context relative menu. Instead of selecting a line segment, you select the vertex you wish to remove and then right mouse button **click and hold** and select **remove vertex**.
## Restoring a Polygon
## Redrawing a Polygon
When you are customizing a polygon in WarnGen, speed is critical. Everyone, even veteran warning forecasters, will occasionally draw a warning polygon one way and then realize, prior to sending the warning, that they want to draw the warning boundaries differently. Depending on the complexity of the changes you make to a warning polygon, sometimes it’s faster to just start over from scratch. The easiest way to proceed, especially if you are still comfortable with the storm motion, is to select the “Track” button from the “Redraw Box on Screen from” section of the WarnGen GUI.
Click the **Reset** button to clear the current polygon and vector and reset the storm centroid marker. Generate a new storm motion by moving the storm markers and select the **Track** button in the WarnGen GUI to draw the new polygon.
<!--
# Restoring a Polygon
When you are customizing a polygon in WarnGen, speed is critical. Everyone, even veteran warning forecasters, will occasionally draw a warning polygon one way and then realize, prior to sending the warning, that they want to draw the warning boundaries differently. Depending on the complexity of the changes you make to a warning polygon, sometimes it’s faster to just start over from scratch. The easiest way to proceed, especially if you are still comfortable with the storm motion, is to select the “Track” the WarnGen GUI.
-->
## Text Window
Using the customized settings in the WarnGen GUI, WarnGen translates the information into a text product that is displayed in a text window on the Text Display. Initially the AWIPS Header Block window appears, and you will just click Enter to modify the text of the warning. The auto-generated text contains the storm speed and direction, the counties and cities affected by the warning/advisory, the valid times of the product, the warning/advisory body text (including any optional bullets selected in the GUI), and additional code to help our partners to efficiently process and disseminate the warning/advisory. The locked parts of the text are highlighted in blue and most of your text should not need to be edited if you configured your WarnGen window correctly.
Once you are satisfied with your polygon and have chosen your selections, click **Create Text** in the WarnGen GUI. Initially the AWIPS Header Block window appears. Leave the top two rows bank and click **Enter** for the text window to open.
![](../images/warngenHeaderBlock.png)
Using the customized settings in the WarnGen GUI, WarnGen translates the information into a text product that is displayed in a text window on the Text Display. The auto-generated text contains the storm speed and direction, the counties and cities affected by the warning/advisory, the valid times of the product, the warning/advisory body text (including any optional bullets selected in the GUI), and additional code to help our partners to efficiently process and disseminate the warning/advisory. The locked parts of the text are highlighted in blue and most of your text should not need to be edited if you configured your WarnGen window correctly.
!!! danger "The Unidata AWIPS release is *non-operational*. You will be allowed to simulate the drawing and text-generation of warnings, but are **prevented from transmitting** a generated warning upstream"
<!--
For some products like Severe Weather Statements, there are parts of text that must be edited that have wild-card characters around them that need to be removed after modifying the text inside (e.g. !** WEAKENED.MOVED OUT OF THE WARNED AREA. **!). When you attempt to send a product, WarnGen will instruct you to modify this text if you have not. Once the text looks ready for submission you click the Send button, and, on a live system, the warning will be transmitted for public broadcast after you press the Go Ahead button on the final WarnGen check. WES-2 Bridge has been implemented in a way that cannot interact with the live AWIPS processes and communications, so warnings issued on the WES-2 Bridge will not be transmitted as live products.
-->
NOTE: Edits made to product text in the editor window should be limited to items such as forecaster name/initials, call-to-action text, etc. If changes are warranted for items such as storm motion, warned counties, or Latitude/Longitude points, close the editor window and make changes using the D-2D and WarnGen graphical tools, then recreate the polygon and/or the text.
> Note: Edits made to product text in the editor window should be limited to items such as forecaster name/initials, call-to-action text, etc. If changes are warranted for items such as storm motion, warned counties, or Latitude/Longitude points, close the editor window and make changes using the D-2D and WarnGen graphical tools, then recreate the polygon and/or the text.
<!--
## "Issuing" a Warning
With a tracked storm in WarnGen:
@ -98,10 +118,9 @@ With a tracked storm in WarnGen:
1. Select **CAVE &gt; New &gt; Text Workstation**
2. Select the **Track** button to preview the polygon
3. Select the **Create Text** button.
4. Select **Enter** on the AWIPS Header Block window to enter the text editor.
4. Select **Enter** on the AWIPS Header Block window to enter the text editor.
- Blue text is locked and uneditable. You should not need to edit most text, and you need to be careful not to make the hazards inconsistent with the locked intensity summary at the bottom of the warning.
- If WarnGen has text you need to edit, there will be wildcards around the text, and WarnGen will not let you send the warning until you modify it.
5. Read the text of the warning, identify your ETN number after the “SV.W.” (e.g. 0004) at the top of the warning, and make changes to the warning text if WarnGen tells you to.
6. Click the **Send** button (will not actually send). Sending warnings on the WES-2 Bridge is safe because it is not connected to the live AWIPS communications route, and the transmission capabilities have been disabled, firewalled, and modified to prevent any risk of warnings going out. Unidata AWIPS always disables the sending of warnings.
-->

169
docs/dev/awips-development-environment.md
View file

@ -1,66 +1,127 @@
Quick instructions on how to deploy CAVE from Eclipse.
# AWIPS Development Environment (ADE)
Quick instructions on how to download the latest source code and run CAVE from Eclipse.
1. Change `/etc/yum.repos.d/awips2.repo` to
> **Note**: It is important to keep in mind these instructions are intended for a system that is specifically used for developing AWIPS. It should not be used in conjunction with installed production versions of AWIPS.
[awips2repo]
name=AWIPS II Repository
baseurl=https://www.unidata.ucar.edu/repos/yum/el6-dev/
enabled=1
protect=0
gpgcheck=0
proxy=_none_
2. `yum clean all && yum groupinstall awips2-ade`
## 1. Remove AWIPS Instances
This will install Eclipse (4.6.1), Java (1.8), Ant (1.9.6), Maven, Python 2.7 and its modules (Numpy, Matplotlib, Shapely, others).
First, make sure to remove any instances of AWIPS that are already installed, this can potentially cause problems when setting up the development environment. Below is an example that had CAVE installed.
Uninstall with yum:
3. `git clone https://github.com/Unidata/awips2.git`
The full list of repositories required:
git clone https://github.com/Unidata/awips2.git
git clone https://github.com/Unidata/awips2-core.git
git clone https://github.com/Unidata/awips2-core-foss.git
git clone https://github.com/Unidata/awips2-foss.git
git clone https://github.com/Unidata/awips2-ncep.git
git clone https://github.com/Unidata/awips2-nws.git
git clone https://github.com/Unidata/awips2-gsd.git
git clone https://github.com/Unidata/awips2-drawing.git
4. Run `/awips2/eclipse/eclipse.sh`
* Preferences > Java
Set to **/awips2/java**
* Preferences > PyDev > Python Interpreter
Set to **/awips2/python/bin/python** (should be resolved by Auto-Config)
* File > Import > General > Existing Projects Into Workspace
Import all of the git cloned project folders **EXCEPT** for the main (first) **github.com/Unidata/awips2.git** directory (which should be **~/awips2**).
You'll want to import **~/awips2** in two parts to ensure a clean and error-free Eclipse build:
1. Import **awips2/cave** > Select All Projects > Finish
2. Import **awips2/edexOsgi** > Select All Projects > Finish
yum clean all
yum groupremove awips2-cave
Check to make sure all rpms have been removed:
rpm -qa | grep awips2
Now import all other repositories fully:
Remove the awips2 directory:
rm -rf /awips2
Select **awips2-core**, **awips2-core-foss**, **awips2-foss**, **awips2-ncep**, etc. > Select All Projects > Finish
---
* Project > Clean
Run a clean build and ensure no errors are reported.
5. Run **com.raytheon.viz.product.awips/developer.product**
## 2. Set Up AWIPS Repo
Double-click the **developer.product** file to open the Product View in Eclipse. Select **Overview** > **Synchronize** and then right-click the file in the left-side package explorer:
Create a repo file named `/etc/yum.repos.d/awips2.repo`, and set the contents to the following:
<pre>
[awips2repo]
name=AWIPS II Repository
baseurl=https://www.unidata.ucar.edu/repos/yum/<b>el7-dev</b>/
enabled=1
protect=0
gpgcheck=0
proxy=_none_
</pre>
>**Note**: This file may already exist if AWIPS had been previously installed on the machine, so make sure to edit the baseurl.
---
## 3. Install the ADE
Install the AWIPS Development Environment (ADE) using yum. This will install Eclipse (4.6.1), Java (1.8), Ant (1.9.6), Python 2.7 and its modules (Numpy, Matplotlib, Shapely, Jep, and others).
yum clean all
yum groupinstall awips2-ade
---
## 4. Download the Source Code
If it's not already installed, install git:
Select **Run As** > **Eclipse Application** to launch CAVE in the development environment.
yum install git
Select **Debug** > **Eclipse Application** to launch CAVE in in debug mode.
Next clone all of the required repositories for AWIPS:
git clone https://github.com/Unidata/awips2.git
git clone https://github.com/Unidata/awips2-core.git
git clone https://github.com/Unidata/awips2-core-foss.git
git clone https://github.com/Unidata/awips2-foss.git
git clone https://github.com/Unidata/awips2-ncep.git
git clone https://github.com/Unidata/awips2-nws.git
git clone https://github.com/Unidata/awips2-gsd.git
git clone https://github.com/Unidata/awips2-drawing.git
---
## 5. Set Up Eclipse
Open eclipse by running: `/awips2/eclipse/eclipse.sh`
Verify or make the following changes to set up eclipse for AWIPS development:
* Preferences > Java
Set to **/awips2/java**
* Preferences > PyDev > Python Interpreter
Set to **/awips2/python/bin/python**
* There might be some unresolved errors. These should be made to warnings instead.
Preferences > Java > Compiler > Building > **Circular Dependencies** > Change to Warning
Preferences > Plug-in Development > API Baselines > **Missing API Baseline** > Change to Warning
* **Turn off automatic building** (you will turn this back on after importing the repos)
Project > Uncheck "Build Automatically"
* File > Import > General > Existing Projects Into Workspace
Import all of the git cloned project folders **EXCEPT** for the main (first) **github.com/Unidata/awips2.git** directory (which should be **~/awips2**).
Select **awips2-core**, **awips2-core-foss**, **awips2-foss**, **awips2-ncep**, etc. > Select All Projects > Finish
You'll want to import **~/awips2** in two parts to ensure a clean and error-free Eclipse build:
1. Import **awips2/cave** > Select All Projects > Finish
2. Import **awips2/edexOsgi** > Select All Projects > Finish
* Project > Clean
Clean the build and ensure no errors are reported.
* Turn automatic building back on
Project > Check "Build Automatically"
---
## 6. Run CAVE
Launch CAVE from eclipse using **com.raytheon.viz.product.awips/developer.product**.
Double-click the **developer.product** file to open the Product View in Eclipse. Select **Overview** > **Synchronize** and then right-click the file in the left-side package explorer:
Select **Run As** > **Eclipse Application** to launch CAVE in the development environment.
Select **Debug** > **Eclipse Application** to launch CAVE in in debug mode.
---
## Troubleshooting
* If you are getting a lot of errors, try changing your Java Compiler to 1.7, build the project, then change back to 1.8 and rebuild.

43
docs/edex/case-studies.md
View file

@ -1,40 +1,47 @@
# Case Study Server Configuration
This document covers what is necessary to install and run AWIPS EDEX as an archive and case study server (no purging of data).
This document covers what is necessary to install and run AWIPS EDEX as an archive and case study server (no purging of processed data).
---
## Quick Install
Follow the [EDEX Install Instructions](../install-edex/) including iptables config and an optional SSD mount (for large data volumes)
Follow the [EDEX Install Instructions](../install/install-edex.md) including iptables config and an optional SSD mount (for large data volumes).
groupadd fxalpha && useradd -G fxalpha awips
mkdir -p /awips2/data_store
wget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/awips2.repo
wget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/el7.repo
yum clean all
yum groupinstall awips2-server -y
---
## Disable Data Purging
The easiest way to disable data purging is to add an **&lt;exclude&gt;purge.*&lt;/exclude&gt;** entry in **/awips2/edex/conf/modes/ingest-modes.xml** so that the purge plugin is not loaded when the EDEX ingest JVM is started:
The easiest way to disable data purging is to add an `<exclude>purge.*</exclude>` entry in `/awips2/edex/conf/modes/modes.xml` so that the purge plugin is not loaded when the EDEX ingest JVM is started:
vi /awips2/edex/conf/modes/ingest-modes.xml
vi /awips2/edex/conf/modes/modes.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<edexModes>
<mode name="ingest">
<exclude>.*request.*</exclude>
<exclude>edex-security.xml</exclude>
<exclude>(taf|shef).*</exclude>
<exclude>purge.*</exclude>
<exclude>.*request.*</exclude>
<exclude>edex-security.xml</exclude>
...
<exclude>purge.*</exclude>
</mode>
...
</edexModes>
---
## Start EDEX
without the LDM
Start EDEX without running the LDM, since we do not want current data. Run the following command:
edex start base
monitor services
Double check everything is running, except the LDM:
edex
@ -47,6 +54,7 @@ monitor services
EDEXrequest :: running :: pid 6566 44303 44599
ldmadmin :: not running
---
## Ingest Case Study Data
@ -54,15 +62,17 @@ Raw data files of any type can be copied or moved into `/awips2/data_store/inges
Individual files can be ingested on the command line with the regex header/pattern supplied as the last argument:
qpidNotify.py /full/path/to/data.file <regex match>
qpidNotify.py /full/path/to/data.file [regex match]
for example
For example:
qpidNotify.py /home/awips/uniwisc_U5_132GOES-15_IMG10.7um_4km_20171024_1830.area.png uniwisc
qpidNotify.py /awips2/data_store/grid/NAM12/conduit/NAM_CONUS_12km_conduit_20171025_1200Z_F084_TMPK-7.000007.grib2 grib
qpidNotify.py /awips2/data_store/radar/FTG_N0Q_20171015_1815 Level3
---
## Viewing Archive Data in CAVE
@ -70,19 +80,18 @@ Because we are installing and configuring a standalone EDEX archive server witho
However, to display specific time-based data (in case you ingest more than one case study), there are two options:
### 1. Load Mode &gt; Inventory
### Set Load Mode to Inventory
In the top-left toolbar change **Valid time seq** to **Inventory**.
![](/images/load_mode_inventory1.png)
Now any data product selected from the menus or the Product Browser will prompt you to select the exact time.
Now any data product selected from the menus or the Product Browser should prompt you to select the exact time.
![](/images/load_mode_inventory2.png)
### 2. Set Data Display Time in CAVE
### Set Data Display Time in CAVE
At the bottom of the CAVE application, double-click the **Time:** entry to bring up a dialog window where you can set CAVE to a previous time, and choose the option of freezing CAVE at that time or allowing CAVE to "move forward in time" from that position as if it were real-time.
![](/images/cave_set_time.png)

150
docs/edex/data-distribution-files.md
View file

@ -1,56 +1,66 @@
# Data Distribution Files
## Overview
EDEX uses **distribution files** to alert the appropriate decoding plug-in that new data has been recieved. These files do so by use of XML and regular expressions. If the WMO header, or file name*, matches a regular expression listed in a distribution XML, then EDEX will put a message into the QPID queue for its corresponding decoder to recognize and process. It is worth noting that more than one distribution file can recognize a single peice of data and notify their decoders to act.
EDEX alerts the appropriate decoding plug-ins of new data by use of XML files that describe data based on regular expressions. If the WMO header, or file name, matches a regular expression listed in a distribution XML, then EDEX will put a message into the QPID queue for the decoder which indicates that a piece of data is available. If a piece of data does not match any distribution XML, EDEX will:
> \*Sometimes the distribution file will not look at the filename. If this file is coming in through the LDM using a proper FILE action, then the it is possible the distribution file will only look at the header and not the filename. If the file is ingested using the *manual* endpoint (/awips2/data_store/ingest/), then this behaviour could be different.
If a piece of data **does not** match any distribution XML, EDEX will:
* Create an entry in `/awips2/edex/logs/edex-ingest-unrecognized-files-yyyymmdd.log`
* Skip processing of the unrecognized file.
Distribution filters are stored in the **edex_static** branch of the Localization Store, and a list of available files can be found in the base-level directory.
Distribution files are stored in the **common_static** branch of the *Localization Store* (series of directories that exist in **/awips2/edex/data/utility/**), and a list of available files can be found in the base-level directory. The base directory is: `/awips2/edex/data/utility/common_static/base/distribution/`.
For each plug-in, the distribution file is named `<data-type>.xml`; for example, the distribution file for satellite data is `satellite.xml`. The distribution files follow the AWIPS base/site localization pattern:
For each plug-in, the distribution file is named `[data-type].xml`. For example, the distribution file for radar data is `radar.xml`. The distribution files follow the AWIPS base/site localization pattern:
[root@edex]# cd /awips2/edex/data/utility/edex_static/base/distribution
[root@edex]# cd /awips2/edex/data/utility/common_static/base/distribution
[root@edex distribution]# ls
acars.xml ffg.xml poessounding.xml
airep.xml goessounding.xml profiler.xml
airmet.xml grib.xml q2.xml
atcf.xml hpe.xml radar.xml
aww.xml idft.xml recco.xml
acars.xml goesr.xml poessounding.xml
airep.xml goessounding.xml profiler.xml
airmet.xml grib.xml radar.xml
atcf.xml intlsigmet.xml redbook.xml
aww.xml lsr.xml satellite.gini.xml
...
* base files are located in **/awips2/edex/data/utility/edex_static/base/distribution **
---
* site override distribution files are located in **/awips2/edex/data/utility/edex_static/site/XXX/distribution**, where **XXX** is the site identifier.
## Creating a Site Override
* *Base* files are located in **/awips2/edex/data/utility/common_static/base/distribution/**
* *Site* override distribution files are located in **/awips2/edex/data/utility/common_static/_site/XXX_/distribution/**, where **_XXX_** is the site identifier.
Note that site-level files override the base files; as a result, local modifications to distribution files must be made as follows:
1. The base distribution file must be copied from **/awips2/edex/data/utility/edex_static/base/distribution** to **/awips2/edex/data/utility/edex_static/site/XXX/distribution**
1. The base distribution file must be copied from **/awips2/edex/data/utility/common_static/base/distribution** to **/awips2/edex/data/utility/common_static/site/XXX/distribution**
2. The local modification must be made to the file in **/awips2/edex/data/utility/edex_static/site/XXX/distribution**
2. The local modification must be made to the file in **/awips2/edex/data/utility/common_static/site/XXX/distribution**
The basic structure of the distribution file is:
<requestPatterns xmlns:ns2="group">
<regex>pattern</regex>
<regex>pattern</regex>
<regex>[pattern]</regex>
<regex>[pattern]</regex>
</requestPatterns>
In each **<regex></regex> **tag, **_pattern_** is replaced with a regular expression that will match either the filename or the WMO header of the raw data. Only data that matches a pattern in the distribution file will be processed.
In each **<regex></regex> **tag, **_[pattern]_** is replaced with a regular expression that will match either the filename or the WMO header of the raw data. Only data that matches a pattern in the distribution file will be processed.
The contents of the base version of the radar distribution file:
[root@edex]# cd /awips2/edex/data/utility/edex_static/base/distribution/
[root@edex]# cd /awips2/edex/data/utility/common_static/base/distribution/
[root@edex]# tail -4 radar.xml
<requestPatterns xmlns:ns2="group">
<requestPatterns >
<regex>^SDUS[234578]. .*</regex>
<regex>^Level3.*</regex>
</requestPatterns>
Looking at the base radar.xml distribution file in the previous example, there are two regular expressions. The first regular expression matches the standard WMO ID of radar products. Via the edexBridge the LDM will place a message in the external.dropbox QPID queue, indicating a radar product has arrived. EDEX will then take the message containing the radar WMO ID and compare it against the regular expressions in radar.xml. If a match is found, EDEX places a message in the QPID queue Ingest.radar. The radar decoder will then consume the message and process the radar data accordingly.
Looking at the base radar.xml distribution file in this example, there are two regular expressions. The first regular expression matches the standard WMO ID of radar products. Using edexBridge the LDM will place a message in the **external.dropbox** QPID queue, indicating a radar product has arrived. EDEX will then take the message containing the radar WMO ID (which comes from the file header) and compare it against the regular expressions in radar.xml. If a match is found, EDEX places a message in the QPID queue Ingest.radar. The radar decoder will then consume the message and process the radar data accordingly.
---
## Adding a REGEX to the Satellite Data Distribution File
@ -58,73 +68,121 @@ As a quick example, suppose we have a local data source for satellite imagery th
To add this locally produced satellite data file to the EDEX distribution; perform the following steps.
1. Copy the base version of **satellite.gini.xml** from the base distribution directory **/awips2/edex/data/utility/edex_static/base/distribution** into the site distribution directory **/awips2/edex/data/utility/edex_static/site/XXX/distribution**
1. Copy the base version of **satellite.gini.xml** from the base distribution directory **/awips2/edex/data/utility/common_static/base/distribution** into the site distribution directory **/awips2/edex/data/utility/common_static/site/XXX/distribution**
2. Edit the site version of **satellite.xml**, adding a new `<regex></regex>` tag immediately below the existing regular expression (`<regex></regex>`) tag. The contents of the tag will be **LOCAL.sat**. The final result will be:
<requestPatterns xmlns:ns2="group">
<regex>TI[CGT]... ....</regex>
<regex>.*NEXRCOMP.*</regex>
<regex>^LOCAL.sat.*</regex>
</requestPatterns>
2. Edit the site version of **satellite.gini.xml**, adding a new `<regex> </regex>` tag immediately below the existing regular expression (`<regex> </regex>`) tag. The contents of the tag will be **^LOCAL.sat**. The final result will be:
<!---
Note: this next code block is formatted using html so that one of the lines of code can be bolded for emphasis. This is why &lt; and &gt; are used instead of < and >.
-->
<pre>
&lt;requestPatterns xmlns:ns2="group"&gt;
&lt;regex&gt;TI[CGT]... ....&lt;/regex&gt;
&lt;regex&gt;rad/NEXRCOMP&lt;/regex&gt;
&lt;regex&gt;.\*.gini.\*&lt;/regex&gt;
<b>&lt;regex&gt;^LOCAL.sat.*&lt;/regex&gt;</b>
&lt;/requestPatterns&gt;
</pre>
3. Save the file and exit the editor. EDEX will automatically pick up the new distribution pattern.
Raw files are written to **/data_store**, and a message is sent via QPID to the EDEX distribution service from the LDM. When a regular expression match is found in a data distribution file, the raw data file is placed in a queue for the matching plugin to decode and process. The distribution files are used to match file headers as well as filenames, which is how files dropped into EDEX's manual endpoint (**/awips2/data_store/ingest**) are processed.
Raw files are written to subdirectories in **/awips2/data_store/**, and a message is sent via QPID to the EDEX distribution service from the LDM. When a regular expression match is found in a data distribution file, the raw data file is placed in a queue for the matching plugin to decode and process. The distribution files are used to match file headers as well as filenames, which is how files dropped into EDEX's manual endpoint (**/awips2/data_store/ingest/**) are processed.
---
## Editing an EDEX Data Distribution File
Because these files are in the **edex_static/** directory, they have to be manually edited using a text editor. You should not edit the base files; rather, you should copy the base version to your site and then edit the site version. The regular expressions in the distribution files need to correspond with the regular expressions in the LDM **_pqact.conf_**** **file.
Because these files are in the **common/_static** directory, they have to be manually edited using a text editor. You should not edit the base files; rather, as stated above, [you should copy the base version to your site and then edit the site version](#creating-a-site-override).
If patterns exist in **_pqact.conf_** but are not in the distribution files, then raw data files will be written to **/data_store** but will not be ingested and processed by EDEX. Entries for these non-ingested files would be written to the unrecognized files log in **/awips/edex/logs**.
The regular expressions in the distribution files do not necessarily need to correspond with the regular expressions in the LDM **_pqact.conf_** file. It is important to note that:
* The regex in the **pqact.conf** file applies to the *productID* that is passed through the LDM.
and
* The regex in the **distribution** files (.xml) typically applies to the header in the file. It can also apply to the filename, if the file is coming through the manual endpoint, or if the data has no header to begin with.
If patterns exist in **_pqact.conf_** but there are no corresponding matching regex expressions in any distribution file, then raw data files will be written to **/awips2/data_store/** but will not be ingested and processed by EDEX. Entries for these non-ingested files would be written to the unrecognized files log in **/awips/edex/logs**.
---
## Examples
### Surface Obs
**/awips2/edex/data/utility/edex_static/base/distribution/obs.xml** Processes any file header that starts with **SA** or **SP**, which should match any WMO header that contains METAR data (e.g.**SAUS**, **SPUS**, **SACN**, **SAMX**).
Its distribution file is located at: **/awips2/edex/data/utility/common_static/base/distribution/obs.xml**:
<requestPatterns xmlns:ns2="group">
<regex>^S[AP].*</regex>
</requestPatterns>
It will process any file header that starts with **SA** or **SP**, which should match any WMO header that contains METAR data (e.g.**SAUS**, **SPUS**, **SACN**, **SAMX**).
---
### Text Data
**/awips2/edex/data/utility/edex_static/base/distribution/text.xml** Processes lots of WM patterns. The second pattern ^S[A-CEG-Z].* matches any header that starts with **S** except for **SD**or **SF**, so it also matches the **SA** and **SP** files that the **obs.xml** plugin matches. This means that METARs are processed by both plugins simultaneously.
Its distribution file is located at **/awips2/edex/data/utility/common_static/base/distribution/text.xml**:
<requestPatterns>
<regex>^[ACFNRUW][A-Z].*</regex>
<regex>^S[ACEG-Z].*</regex>
<regex>^[ACFNRUW][A-Z][A-Z0-9]{4} [A-Z0-9]{4}</regex>
<regex>^S[A-CEG-Z].*</regex>
<!-- Only have AFOS mapping for T[BCX] -->
<regex>^T[BCX].*</regex>
<regex>^SF[A-OQ-TVZ].*</regex>
<regex>^SF[A-OQ-TV-Z].*</regex>
<regex>^SDUS1.*</regex>
<regex>^SDUS4[1-6].*</regex>
<regex>^SDUS9[^7].*</regex>
<regex>^SFU[^S].*</regex>
<regex>^SFUS4[^1].*</regex>
<regex>^SFUS4[^1].*</regex>
<regex>^SFP[^A].*</regex>
<regex>^SFPA[^4].*</regex>
<regex>^SFPA4[^1].*</regex>
<regex>^BMBB91.*</regex>
<regex>^N.*</regex>
<regex>^F[EHIJKLMQVWX].*</regex>
<regex>^SFPA[^4].*</regex>
<regex>^SFPA4[^12].*</regex>
<regex>^BMBB91.*</regex>
<regex>^N[A-Z][A-Z0-9]{4} [A-Z0-9]{4}</regex>
<regex>^F[EHIJKLMQVWX].*</regex>
<regex>wcl_decrypted</regex>
<regex>ecmwf_mos_decrypted</regex>
</requestPatterns>
Processes lots of WM patterns. The second pattern ^S[A-CEG-Z].\* matches any header that starts with **S** except for **SD** or **SF**. This is because it matches A through C (*A-C*), E, and G through Z (*G-Z*). So it also matches the **SA** and **SP** files that the **obs.xml** plugin matches. This means that METARs are processed by both plugins simultaneously.
---
### Grib Data
**/awips2/edex/data/utility/edex_static/base/distribution/grib.xml** The grib/grid decoder distribution file matches all numerical grids distributed over the IDD NGRID feed by matching WMO header, and from CONDUIT by matching the *.grib* file extension.
Its distribution file is located at **/awips2/edex/data/utility/common_static/base/distribution/grib.xml**:
<requestPatterns>
<!-- Super Set of all possible WMO grib patterns -->
<!-- Is specifically not restricting on CCCC since HPE isn't populating it -->
<regex>^[EHLMOYZ][A-Z]{3}\d{2}</regex>
<!-- This to match Unidata CONDUIT products w/o standard headers -->
<!-- Exclude Data Delivery specific patterns -->
<regexExclude>^LZ[ABC][ABC]9[123] (KWBC|KNCF)</regexExclude>
<!-- ECMWF decrypted -->
<regex>ecmwf_decrypted</regex>
<!-- NWPS pattern -->
<regex>\p{Alpha}{3}_nwps_CG1</regex>
<regex>\p{Alpha}{3}_nwps_CG0_Trkng</regex>
<!-- grib files without WMO headers -->
<regex>.*grib.*</regex>
<regex>.*GRIB.*</regex>
<regex>.*grb.*</regex>
<regex>^US058.*</regex>
<regex>^CMC_reg.*</regex>
</requestPatterns>
The grib/grid decoder distribution file matches all numerical grids distributed over the IDD NGRID feed by matching WMO header, and from CONDUIT by matching various *.grib* file extensions. It also includes an example of a *regexExclude* message which can be used to single out matching values that aren't to be included.
---
### Addtional Information
Important notes about regular expressions:
* Any time a new entry is placed in the **pqact.conf** file on LDM, a corresponding entry needs to be added to the appropriate Data Distribution file in the data distribution directory, or the data file will be logged to **edex-ingest-unrecognized-files-YYYYMMDD.log**
* Any time a new entry is placed in the **pqact.conf** file on LDM, it is likely a corresponding entry needs to be added to the appropriate Data Distribution file in the data distribution directory, or the data file will be logged to **edex-ingest-unrecognized-files-YYYYMMDD.log**.
> The exception to this rule is if the new data coming from the LDM is a type of data that already exists and EDEX already has a distribution file with a matching regex that will recognize it.
* Any time an entry is removed from the **pqact.conf** file, the corresponding entry should be removed from the appropriate Data Distribution file in the data distribution directory.
* If you have written a new regex for a distribution file to match on a filename, and it is not matching, then the file most likely has a header. In this case EDEX will only look at the header to match the regex. You must change your regex to something that matches the header, not the filename.

279
docs/edex/data-purge.md
View file

@ -1,62 +1,96 @@
# Purging and Retention
AWIPS uses a plugin-based purge strategy for HDF5 data, allowing the user to change the purge frequency for each plugin individually, and specific products for a particular plugin.
## Purge Types
> Purge is triggered by a quartz timer event that fires at 30 minutes after each hour.
There are two main forms of data puring in AWIPS. The most often thought of is the purging for [**processed data**](#processed-data-purging). This has to do with how long data is stored for **after** it has been decoded and processed.
Purging rules are defined in XML files in the Localization Store, accessible from the CAVE localization perspective. On EDEX, most are located in `/awips2/edex/data/utility/common_static/base/purge`, and follow the **base/site** localization pattern (e.g. site purge files are in `site/XXX/purge` rather than `base/purge`, where XXX is the site identifier.
The second type of purging has to do with [**raw data**](#raw-data-purging). This has to do with how long data is stored for **before** it has been decoded.
## Time-base purge
## Processed Data Purging
If a plugin has no XML file, the default rule of 1 day (24 hours) is used, from `/awips2/edex/data/utility/common_static/base/purge/defaultPurgeRules.xml`
AWIPS uses a plugin-based purge strategy for processed **HDF5 data**. This allows the user to change the purge frequency for each plugin individually, and even set purge rules for specific products for a particular plugin. There is also a default purge rules file for those products which do not have specific rules written.
<purgeRuleSet>
<defaultRule>
<period>01-00:00:00</period>
</defaultRule>
</purgeRuleSet>
> **Note**: Purging is triggered by a quartz timer event that fires at 30 minutes after each hour.
Time-based purging uses the *reference time* of the data, and dtermination of the reference time is decoder based.
Purging rules are defined in XML files in the Localization Store. On EDEX, most are located in `/awips2/edex/data/utility/common_static/base/purge`, and follow the **base/site** localization pattern (e.g. site purge files are in `site/XXX/purge` rather than `base/purge`, where XXX is the site identifier).
Each data set can have a purge rule defined, and the xml file is named after the data set:
## 30-day NEXRAD3 Example
Modify `/awips2/edex/data/utility/common_static/base/purge/radarPurgeRules.xml` to increase the data retention period from 7 to 31 days:
<purgeRuleSet>
<defaultRule>
<period>31-00:00:00</period>
</defaultRule>
</purgeRuleSet>
**Note**: you do NOT have to restart EDEX when you change a purge rule!
## Frame-based purge
Some plugins use frame-base purging, retaining and certain number of prpduct "versions".
`/awips2/edex/data/utility/common_static/base/purge/satellitePurgeRules.xml`
ls /awips2/edex/data/utility/common_static/base/purge/
<purgeRuleSet>
<key>sectorID</key>
<key>physicalElement</key>
<defaultRule>
<versionsToKeep>196</versionsToKeep>
</defaultRule>
<rule>
<keyValue>NEXRCOMP</keyValue>
<versionsToKeep>288</versionsToKeep>
</rule>
<rule>
<keyValue>Alaska National</keyValue>
<versionsToKeep>48</versionsToKeep>
</rule>
acarsPurgeRules.xml bufruaPurgeRules.xml pirepPurgeRules.xml
acarssoundingPurgeRules.xml ccfpPurgeRules.xml poessoundingPurgeRules.xml
aggregatePurgeRules.xml convsigmetPurgeRules.xml pointsetPurgeRules.xml
airepPurgeRules.xml cwaPurgeRules.xml profilerPurgeRules.xml
...
> In the above example, notice a *default rule* (196) as well as specific sectors with their own rules.
---
## Purge Logs
### Time-based purge
Data purge events are logged to the file `edex-ingest-purge-<yyyymmdd>.log`, where `<yyyymmdd>` is the date stamp.
If a plugin has no XML file, the default rule of 1 day (24 hours) is used, from `/awips2/edex/data/utility/common_static/base/purge/defaultPurgeRules.xml`:
<!--
Use html <pre> tag to be able to bold a specific section. Make sure to replace '<' with '&lt;' and '>' with '&gt;'.
-->
<pre>
&lt;purgeRuleSet&gt;
&lt;defaultRule&gt;
<b>&lt;period&gt;01-00:00:00&lt;/period&gt;</b>
&lt;/defaultRule&gt;
&lt;/purgeRuleSet&gt;
</pre>
Time-based purging is set with the *period* tag and uses the *reference time* of the data. The reference time of the data is determined by the decoder.
---
### 30-day NEXRAD3 Example
Modify `/awips2/edex/data/utility/common_static/base/purge/radarPurgeRules.xml` to increase the data retention period from 1 to 31 days:
<pre>
&lt;purgeRuleSet&gt;
&lt;defaultRule&gt;
<b>&lt;period&gt;31-00:00:00&lt;/period&gt;</b>
&lt;/defaultRule&gt;
&lt;/purgeRuleSet&gt;
</pre>
>**Note**: you do NOT have to restart EDEX when you change a purge rule!
---
### Frame-Based Purge
Some plugins use frame-base purging, retaining and certain number of product "versions".
`/awips2/edex/data/utility/common_static/base/purge/gridPurgeRules.xml`
<pre>
<b>&lt;defaultRule&gt;
&lt;versionsToKeep&gt;2&lt;/versionsToKeep&gt;</b>
&lt;period&gt;07-00:00:00&lt;/period&gt;
&lt;/defaultRule&gt;
&lt;rule&gt;
<b>&lt;keyValue&gt;LAPS&lt;/keyValue&gt;
&lt;versionsToKeep&gt;30&lt;/versionsToKeep&gt;</b>
&lt;/rule&gt;
&lt;rule regex="true"&gt;
<b>&lt;keyValue&gt;NAM(?:12|20|40)&lt;/keyValue&gt;
&lt;versionsToKeep&gt;2&lt;/versionsToKeep&gt;
&lt;modTimeToWait&gt;00-00:15:00&lt;/modTimeToWait&gt;</b>
&lt;/rule&gt;
...
</pre>
In the above example, notice a *default rule* (2) is specified, as well as specific models with their own rules.
The tag *modTimeToWait* can be used in conjunction with *versionsToKeep* and will increase the versionsToKeep by 1 if data matching this rule has been stored within modTimeToWait.
---
### Purge Logs
Data purge events are logged to the file `edex-ingest-purge-[yyyymmdd].log`, where `[yyyymmdd]` is the date stamp.
tail -f edex-ingest-purge-20120327.log
@ -68,93 +102,84 @@ Data purge events are logged to the file `edex-ingest-purge-<yyyymmdd>.log`, whe
INFO 2012-03-27 00:31:23,155 [DefaultQuartzScheduler_Worker-3] PurgeLogger: EDEX - PURGE LOGS::Skipped processing 1 files
INFO 2012-03-27 00:31:23,155 [DefaultQuartzScheduler_Worker-3] PurgeLogger: EDEX - PURGE LOGS::---------END LOG PURGE-----------
---
## All Purge Rules
### All Purge Rules
To see all purge rule directories (base, site, configured):
find /awips2/edex/data/utility -name purge
/awips2/edex/data/utility/edex_static/base/purge
/awips2/edex/data/utility/edex_static/configured/OAX/purge
/awips2/edex/data/utility/edex_static/site/OAX/purge
/awips2/edex/data/utility/common_static/base/purge
/awips2/edex/data/utility/common_static/configured/OAX/purge
/awips2/edex/data/utility/common_static/site/AFC/purge
/awips2/edex/data/utility/common_static/site/OAX/purge
To see a list of the data plug-ins that have purge rules:
If any overrides have been made, then it's possible that *site* directories may show up as results from the find command as well.
---
## Raw Data Purging
Raw data are files that have been brought in by the LDM and recognized by an action in the **pqact.conf** file. These files are written to subdirectories of `/awips2/data_store/`. This data will wait here until it is purged, from the purging rules defined in `/awips2/edex/data/utility/common_static/base/archiver/purger/RAW_DATA.xml`.
If the purge time is too short, and the processing latencies on EDEX are too long, it is possible that EDEX will miss some of this data, and the purge times will need to be adjusted by changing the [`<defaultRetentionHours>`](#default-retention) or [`<selectedRetentionHours>`](#selected-retention) tag on the relevent data sets.
---
### Default Retention
The **defaultRetentionHours** tag is defined at the beginning of the RAW_DATA.xml file. It is the duration that will apply to any piece of data that does not fall under an explicitly defined [category](#selected-retention).
The default value for our EDEX is 1 hour:
<pre>
&lt;archive&gt;
&lt;name&gt;Raw&lt;/name&gt;
&lt;rootDir&gt;/awips2/data_store/&lt;/rootDir&gt;
<b>&lt;defaultRetentionHours&gt;1&lt;/defaultRetentionHours&gt;</b>
&lt;category&gt;
...
</pre>
find /awips2/edex/data/utility -name "*PurgeRules.xml"
/awips2/edex/data/utility/edex_static/base/purge/airmetPurgeRules.xml
/awips2/edex/data/utility/edex_static/base/purge/convsigmetPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufruaPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/obsPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/nctextPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ncscatPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/vaaPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosHPCPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrhdwPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/cwaPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/airmetPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ffmpPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/lsrPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/profilerPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/poessoundingPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/gpdPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/mcidasPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/acarssoundingPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ffgPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/sgwhvPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/intlsigmetPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/aggregatePurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/radarPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/idftPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/wcpPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/warningPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/satellitePurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/sshaPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ccfpPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/atcfPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrssmiPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/sgwhPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/geomagPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/modelsoundingPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosAVNPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ncuairPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/acarsPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/airepPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrquikscatPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/cwatPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/nonconvsigmetPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/pirepPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/statsPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmthdwPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrascatPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/gridPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosGFSPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ncpafmPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrsigwxPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/pgenPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/dmwPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosMRFPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosETAPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/sfcobsPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/modisPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/awwPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/defaultPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/convsigmetPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/redbookPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrncwfPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/tcmPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/bufrmosLAMPPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/goessoundingPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/binlightningPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/stormTrackPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ghcdPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/ntransPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/regionalsatPurgeRules.xml
/awips2/edex/data/utility/common_static/base/purge/solarimagePurgeRules.xml
/awips2/edex/data/utility/common_static/site/AFC/purge/regionalsatPurgeRules.xml
---
### Selected Retention
Data sets are broken up into *categories* in the RAW_DATA.xml file. These categories are groupings of similar data. Each category has a **selectedRetentionHours** tag which specifies how long the matching data will be kept for.
For example, there is a **Model** category which sets the purge time to 3 hours for all grib, bufrmos, and modelsounding data:
<pre>
...
<b>&lt;category&gt;
&lt;name&gt;Model&lt;/name&gt;
&lt;selectedRetentionHours&gt;3&lt;/selectedRetentionHours&gt;</b>
&lt;dataSet&gt;
<b>&lt;dirPattern&gt;(grib|grib2)/(\d{4})(\d{2})(\d{2})/(\d{2})/(.*)&lt;/dirPattern&gt;</b>
&lt;displayLabel&gt;{1} - {6}&lt;/displayLabel&gt;
&lt;dateGroupIndices&gt;2,3,4,5&lt;/dateGroupIndices&gt;
&lt;/dataSet&gt;
&lt;dataSet&gt;
<b>&lt;dirPattern&gt;(bufrmos|modelsounding)/(\d{4})(\d{2})(\d{2})/(\d{2})&lt;/dirPattern&gt;</b>
&lt;displayLabel&gt;{1}&lt;/displayLabel&gt;
&lt;dateGroupIndices&gt;2,3,4,5&lt;/dateGroupIndices&gt;
&lt;/dataSet&gt;
&lt;/category&gt;
...
</pre>
### Logging
Raw data purging can be seen in the **purge** logs as well (`/awips2/edex/logs/edex-ingest-purge-[yyyymmdd].log` where `[yyyymmdd]` is the date stamp).
[centos@tg-atm160027-edex-dev purge]$ grep -i 'archive' /awips2/edex/logs/edex-ingest-purge-20200728.log
INFO 2020-07-28 20:05:23,959 2329 [Purge-Archive] ArchivePurgeManager: EDEX - Start purge of category Raw - Observation, directory "/awips2/data_store/bufrhdw".
INFO 2020-07-28 20:05:23,960 2330 [Purge-Archive] ArchivePurgeManager: EDEX - End purge of category Raw - Observation, directory "/awips2/data_store/bufrhdw", deleted 0 files and directories.
INFO 2020-07-28 20:05:23,961 2331 [Purge-Archive] ArchivePurgeManager: EDEX - Unlocked: "/awips2/data_store/bufrhdw"
INFO 2020-07-28 20:05:23,963 2332 [Purge-Archive] ArchivePurgeManager: EDEX - Locked: "/awips2/data_store/xml"
INFO 2020-07-28 20:05:23,963 2333 [Purge-Archive] ArchivePurgeManager: EDEX - Start purge of category Raw - Products, directory "/awips2/data_store/xml".
INFO 2020-07-28 20:05:23,964 2334 [Purge-Archive] ArchivePurgeManager: EDEX - End purge of category Raw - Products, directory "/awips2/data_store/xml", deleted 5 files and directories.
INFO 2020-07-28 20:05:23,967 2335 [Purge-Archive] ArchivePurgeManager: EDEX - Unlocked: "/awips2/data_store/xml"
INFO 2020-07-28 20:05:23,967 2336 [Purge-Archive] ArchivePurger: EDEX - Raw::Archive Purged 28387 files in 23.8s.
INFO 2020-07-28 20:05:23,979 2337 [Purge-Archive] ArchivePurgeManager: EDEX - Purging directory: "/awips2/edex/data/archive".
INFO 2020-07-28 20:05:23,992 2338 [Purge-Archive] ArchivePurger: EDEX - Processed::Archive Purged 0 files in 25ms.
INFO 2020-07-28 20:05:23,992 2339 [Purge-Archive] ArchivePurger: EDEX - Archive Purge finished. Time to run: 23.9s
...

51
docs/edex/distributed-computing.md
View file

@ -1,24 +1,30 @@
# Distributed EDEX
AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. While originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS could consist of a dozen servers or more, the early Unidata releases were stripped of operations-specific configurations and plugins, and released as a standalone server. This worked, since (at the time) a single EDEX instance with an attached SSD could handle most of NOAAport. However, with GOES-R(16) coming online in 2017, and more gridded forecast models being created at finer temporal and spatial resolutions, there is now a need to distribute the data decoding across multiple machine to handle this firehose of data.
AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. While originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS could consist of a dozen servers or more, the early Unidata releases were stripped of operations-specific configurations and plugins, and released as a standalone server. This worked, since (at the time) a single EDEX instance with an attached SSD could handle most of NOAAport. However, with GOES-R(16) coming online in 2017, and more gridded forecast models being created at finer temporal and spatial resolutions, there is now a need to distribute the data decoding across multiple machines to handle this firehose of data.
---
This walkthrough will install different EDEX components on two machines in the XSEDE Jetstream Cloud, the first is used to **ingest and decode** while the second is used to **store and serve** data.
## Unidata's Current EDEX Server
Currently, with our specific EDEX server we use a Database/Request instance that also decodes and ingests a good portion of the data. It handles all data requests from CAVE users, as well as the majority of the decoding and ingesting for data feeds coming down on the LDM. The **radar** data has been specifically exluded (from the decoding and ingest) and it has its own [**Ingest/Decode Server**](#ingestdecode-server) which is explained in more detail below.
For our EDEX we have designated an instance of the ingest/decoding server to be dedicated to handling the radar data. Our *Radar-EDEX* recieves and decodes all radar down from the LDM and then stores it back on our main [**Database/Request EDEX**](#databaserequest-server) in the form of HDF5 data files and PostgreSQL metadata.
---
## Example Installation
This walkthrough will install different EDEX components on two machines in the XSEDE Jetstream Cloud, the first is used to **store and serve** while the second is used to **ingest and decode** data.
![](/images/awips2_distributed.png)
---
## Database/Request Server
### Database/Request Server
!!! note "Specs"
* IP address **10.0.0.9**
* CentOS 6.9
* m1.medium (CPU: 6, Mem: 16 GB)
* 1000GB attached storage for `/awips2/edex/data/hdf5`
For this example, this server will be referred to by the IP address **10.0.0.9**.
### 1. Install
#### 1. Install
groupadd fxalpha && useradd -G fxalpha awips
mkdir /awips2
@ -26,7 +32,7 @@ This walkthrough will install different EDEX components on two machines in the X
yum clean all
yum groupinstall awips2-database
### 2. IPtables Config
#### 2. IPtables Config
It is required that ports 5432 and 5672 be open for the specific IP addresses of outside EDEX ingest servers. It is *not recommended* that you leave port 5432 open to all connections (since the default awips database password is known, and is not meant as a security measure). Further, it *is recommended* that you change the default postgres awips user password (which then requires a reconfiguration of every remote EDEX ingest server in order to connect to this database/request server).
@ -56,7 +62,7 @@ Note the line **`-A INPUT -s 10.0.0.7 -j EDEX`** as well as the following **`-A
!!! Note "The two ports left open to all connections (9581,9582) in addition to default port 22 are for outside CAVE client connections"
### 3. Database Config
#### 3. Database Config
In the file `/awips2/database/data/pg_hba.conf` you define remote connections for all postgres tables with as `<IP address>/32`, after the block of IPv4 local connections:
@ -71,13 +77,13 @@ In the file `/awips2/database/data/pg_hba.conf` you define remote connections fo
hostssl all all ::1/128 cert clientcert=1
hostnossl all all ::1/128 md5
### 4. Start EDEX
#### 4. Start EDEX
edex start database
This will start PostgreSQL, httpd-pypies, Qpid, and the EDEX Request JVM (and will not start the LDM or the EDEX Ingest and IngestGrib JVMs)
### 5. Monitor Services
#### 5. Monitor Services
The command `edex` will show which services are running, and for a Database/Request server, will not include the LDM, EDEXingest, or EDEXgrib:
@ -94,7 +100,7 @@ The command `edex` will show which services are running, and for a Database/Requ
Since this Database/Request server is not running the main *edexIngest* JVM, we won't see anything from `edex log`, instead watch the Request Server with the command
edex log reqeust
edex log request
!!! warning "Confirm that EDEX Request connects to PostgreSQL!"
With the above `edex log request`, ensure that the log progresses **past this point**:
@ -113,21 +119,18 @@ Since this Database/Request server is not running the main *edexIngest* JVM, we
---
## Ingest/Decode Server
### Ingest/Decode Server
!!! note "Specs"
* IP address **10.0.0.7**
* CentOS 6.9
* m1.xxlarge (CPU: 44, Mem: 120 GB)
For this example, this server will be referred to by the IP address **10.0.0.7**.
### 1. Install
#### 1. Install
groupadd fxalpha && useradd -G fxalpha awips
wget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/awips2.repo
yum clean all
yum groupinstall awips2-ingest
### 2. EDEX Config
#### 2. EDEX Config
`vi /awips2/edex/bin/setup.env`
@ -147,13 +150,13 @@ Here you should redefine `DB_ADDR` and `PYPIES_SERVER` to point to the **Databas
Notice that `EDEX_SERVER` and `BROKER_ADDR` (qpid) should remain defined as the *localhost* IP address (10.0.0.7)
### 3. Start EDEX
#### 3. Start EDEX
edex start ingest
This will start Qpid and the EDEX Ingest and IngestGrib JVMs (and not start PostgreSQL, httpd-pypies, or the EDEX Request JVM)
### 4. Monitor Services
#### 4. Monitor Services
Watch the edex JVM log with the command

62
docs/edex/edex-ingest-docker-container.md
View file

@ -1,12 +1,14 @@
# Docker EDEX
Project home: [https://github.com/Unidata/edex-docker](https://github.com/Unidata/edex-docker)
---
EDEX can be run inside a docker container, which allows you to process data into an AWIPS system without requiring a full CentOS installation and configuration.
EDEX can be run inside a docker container, which allows you to process data into an AWIPS system without requiring accessing and altering the machine's native CentOS installation and configuration.
The [EDEX Docker Image](https://github.com/Unidata/edex-docker) is built on CentOS 7 and contains the latest Unidata AWIPS release.
The [EDEX Docker Image](https://github.com/Unidata/edex-docker) is built on CentOS 7 and contains the latest Unidata AWIPS release (18.1.1).
This container is an *ingest-only* install, meaning there is *no database or request server*. This example requires a Database/Request server be configured for you to access remotely. See the [Distributed EDEX](https://unidata.github.io/awips2/edex/distributed-computing/) document for more.
This container is an **ingest-only** install, meaning there is **no database or request server**. This example requires a Database/Request server be configured for you to access remotely. See the [**Distributed EDEX**](https://unidata.github.io/awips2/edex/distributed-computing/) document for more.
---
@ -19,38 +21,40 @@ Download and install Docker and Docker Compose:
* [Docker for Windows](https://docs.docker.com/docker-for-windows/install/)
* [docker-compose](https://docs.docker.com/compose/) (it should be bundled with Docker by default on Mac and Windows)
---
## Run the EDEX Ingest Container
Clone this repository
Clone the source repository:
git clone https://github.com/Unidata/edex-docker.git
cd edex-docker
Run the container with docker-compose
Run the container with docker-compose:
docker-compose up -d edex-ingest
Confirm the container is running
Confirm the container is running:
docker ps -a
Enter the container
Enter the container:
docker exec -it edex-ingest bash
Stop the container
Stop the container:
docker-compose stop
Delete the container (keep the image)
Delete the container (keep the image):
docker-compose rm -f
Run commands inside the container, such as
Run commands inside the container, such as:
docker exec edex-ingest edex
which should return something like
which should return something like:
[edex status]
qpid :: running :: pid 22474
@ -68,6 +72,8 @@ docker-compose stop
docker-compose up -d edex-ingest
```
---
## Configuration and Customization
The file `docker-compose.yml` defines files to mount to the container and which ports to open:
@ -87,32 +93,34 @@ The file `docker-compose.yml` defines files to mount to the container and which
soft: 1024
hard: 1024
---
## Mounted Files
- `etc/ldmd.conf`
### etc/ldmd.conf
Defines which data feeds to receive. By default there is only one active request line (`REQUEST IDS|DDPLUS ".*" idd.unidata.ucar.edu`) to not overwhelm small EDEX containers ingesting large volumes of radar and gridded data files. Any updates to the file `etc/ldmd.conf` will be read the next time you restart the container.
Defines which data feeds to receive. By default there is only one active request line (`REQUEST IDS|DDPLUS ".*" idd.unidata.ucar.edu`) to not overwhelm small EDEX containers ingesting large volumes of radar and gridded data files. Any updates to the file `etc/ldmd.conf` will be read the next time you restart the container.
- `etc/pqact.conf`
### etc/pqact.conf
Defines how products are processed and where they are written to on the filesystem. This is the full set of pattern actions used in Unidata AWIPS, and generally you do not need to edit this file. Instead control which data feeds are requested in `ldmd.conf` (above).
Defines how products are processed and where they are written to on the filesystem. This is the full set of pattern actions used in Unidata AWIPS, and generally you do not need to edit this file. Instead control which data feeds are requested in `ldmd.conf` (above).
- `bin/setup.env`
### bin/setup.env
Defines the remote EDEX Database/Request server:
Defines the remote EDEX Database/Request server:
### EDEX localization related variables ###
export AW_SITE_IDENTIFIER=OAX
export EXT_ADDR=js-157-198.jetstream-cloud.org
### EDEX localization related variables ###
export AW_SITE_IDENTIFIER=OAX
export EXT_ADDR=js-157-198.jetstream-cloud.org
**EXT_ADDR** must be set to an allowed EDEX Database/Request Server. In this example we are using a JetStream Cloud instance, which controls our *edex-ingest* access with IPtables, SSL certificates, and PostgreSQL **pg_hba.conf** rules (this server is used in software training workshop environments and will not allow outside connections).
> **Note**: *EXT_ADDR* must be set to an allowed EDEX Database/Request Server. In this example we are using a JetStream Cloud instance, which controls our *edex-ingest* access with IPtables, SSL certificates, and PostgreSQL pg_hba.conf rules. This server will not allow outside connections, you must change this to point to an appropriate server.
- `bin/runedex.sh`
### bin/runedex.sh
The default script run when the container is started, acts as a sort-of service manager for EDEX and the LDM (see `ENTRYPOINT ["/awips2/edex/bin/runedex.sh"]` in *Dockerfile.edex*), essentially:
The default script run when the container is started, acts as a sort-of service manager for EDEX and the LDM (see `ENTRYPOINT ["/awips2/edex/bin/runedex.sh"]` in *Dockerfile.edex*), essentially:
/awips2/qpid/bin/qpid-wrapper &
/awips2/edex/bin/start.sh -noConsole ingest &
ldmadmin mkqueue
ldmadmin start
/awips2/qpid/bin/qpid-wrapper &
/awips2/edex/bin/start.sh -noConsole ingest &
ldmadmin mkqueue
ldmadmin start

20
docs/edex/edex-users.md
View file

@ -1,5 +1,6 @@
# Monitor Users
To see a list of clients connecting to your EDEX server, use the `edex users [YYYYMMDD]` command, where `YYYYMMDD` is the optional date string.
To see a list of clients connecting to your EDEX server, use the `edex users [YYYYMMDD]` command, where `[YYYYMMDD]` is the optional date string.
edex users
@ -12,19 +13,22 @@ To see a list of clients connecting to your EDEX server, use the `edex users [YY
# Logging Daily EDEX Users
## Logging Daily EDEX Users
Create a short script to run once daily at 20 minutes after 00 UTC, appending each day's `edex users` list to a logfile `/home/awips/edex-users.log`.
To get a running log of who has accessed EDEX, you can create a short script.
The example below is a script that runs once daily at 20 minutes after 00 UTC, appending each day's `edex users` list to a logfile `/home/awips/edex-users.log`:
1. `vi ~/edexUsers.sh`
1.
vi~/edexUsers.sh
#!/bin/bash
/awips2/edex/bin/edex users >> /home/awips/edex-users.log
2. `crontab -e`
2.
crontab -e
0 20 * * * /home/awips/edexUsers.sh 1>> /dev/null 2>&1

41
docs/edex/ldm.md
View file

@ -1,42 +1,51 @@
# LDM Feeds
## Default LDM Feeds for EDEX
from `/awips2/ldm/etc/ldmd.conf`
Data feeds are defined by the **ldmd.conf** file in `/awips2/ldm/etc/ldmd.conf`. The default feeds that come "turned on" with our EDEX are the following:
REQUEST NEXRAD3 "./p(DHR|DPR|DSP|DTA|DAA|DVL|EET|HHC|N0Q|N0S|N0U|OHA|NVW|NTV|NST)." idd.unidata.ucar.edu
REQUEST FNEXRAD|IDS|DDPLUS|UNIWISC ".*" idd.unidata.ucar.edu
REQUEST NEXRAD3 "./p(DHR|DPR|DSP|DTA|DAA|HHC|N.P|N.Q|N.S|N.U|OHA)." idd.unidata.ucar.edu
REQUEST FNEXRAD|IDS|DDPLUS ".*" idd.unidata.ucar.edu
REQUEST UNIWISC|NIMAGE ".*" idd.unidata.ucar.edu # AREA/GINI
REQUEST EXP "WwWind" idd.unidata.ucar.edu # ESPL/PSD Profilers
REQUEST DIFAX "GLM" idd.unidata.ucar.edu # GOES GLM
REQUEST EXP ".*" lead.unidata.ucar.edu # GOES ABI netCDF4 (full sector)
REQUEST NGRID ".*" idd.unidata.ucar.edu
REQUEST NOTHER "^TIP... KNES.*" idd.unidata.ucar.edu
REQUEST HDS|NIMAGE ".*" idd.unidata.ucar.edu
REQUEST HDS ".*" idd.unidata.ucar.edu
REQUEST CONDUIT "nam" idd.unidata.ucar.edu # NAM12
REQUEST CONDUIT "pgrb2" idd.unidata.ucar.edu # GFS0p25
REQUEST CONDUIT "nam" idd.unidata.ucar.edu
REQUEST CONDUIT "rap" idd.unidata.ucar.edu
REQUEST CONDUIT "pgrb2" idd.unidata.ucar.edu
REQUEST CONDUIT "nwstg" idd.unidata.ucar.edu
> Remember than LDM commands such as these require **TAB SEPARATION** between items.
---
## Optional LDM Feeds
FNMOC and CMC models
Some additional feeds are included but commented out using '#'. To activate the feed, simply remove the #, save the file, and [restart the LDM](#restart-the-ldm).
### FNMOC and CMC models
REQUEST FNMOC ".*" idd.unidata.ucar.edu
REQUEST CMC ".*" idd.unidata.ucar.edu
Lightning (restricted to educational use with rebroadcasting restricted)
### Lightning (restricted to educational use with rebroadcasting restricted)
REQUEST LIGHTNING ".*" striker2.atmos.albany.edu
REQUEST LIGHTNING ".*" idd.unidata.ucar.edu
FSL/GSD Experimental HRRR (Sub-hourly)
### FSL/GSD Experimental HRRR (Sub-hourly)
REQUEST FSL2 "^GRIB2.FSL.HRRR" hrrr.unidata.ucar.edu
---
## Restart the LDM
Use the following commands to restart the LDM:
sudo service edex_ldm restart
ldmadmin restart
---
## Monitor Incoming Data Feeds
@ -51,9 +60,13 @@ To watch for a specific product and feed and time (360 sec = 6 min):
To watch the same on a remote queue:
notifyme -vl - -h idd.unidata.ucar.edu -f NEXRAD3 -p DHR -o 360
---
## LDM Logging
To open a real-time readout of LDM logging you can run use the `edex` command. To exit, press `CTRL+C`.
edex log ldm
[edex] EDEX Log Viewer

339
docs/edex/new-grid.md
View file

@ -1,21 +1,29 @@
# Ingest a New Grid
Unrecognized grids can be decoded by EDEX simply by dropping `*.grib` or `*.grib2` files into `/awips2/data_store/ingest/`
To add support for a new grid, two edits must be made:
* **Geospatial projection** must be defined in a *grid navigation file*
* **Grid name**, **center**, **subcenter**, and **process ID** must be defined in a *model definition file*.
* **Geospatial projection** must be defined in a [***grid navigation file***](#create-grid-projection-file)
* **Grid name**, **center**, **subcenter**, and **process ID** must be defined in a [***model definition file***](#create-model-definition)
If the parameters in the grib file haven't been previously specified, another change *may* be needed as well:
* **Center**, **subcenter**, **discipline**, **category**, and possibly **parameter ID** information may need to be defined in a [***table***](#adding-a-table)
---
## Ingest an Unsupported Grid
### Grib Products
1. Download an example grib1 file and rename to a `*.grib` extension, then copy to the manual ingest point `/awips2/data_store/ingest/`
wget https://www.unidata.ucar.edu/staff/mjames/14102318_nmm_d01.GrbF00600 -O wrf.grib
wget https://www.unidata.ucar.edu/software/awips2/14102318_nmm_d01.GrbF00600 -O wrf.grib
cp wrf.grib /awips2/data_store/ingest/
Remember that the data distribution file (`/awips2/edex/data/utility/edex_static/base/distribution/grib.xml`) will match filename which have the `*.grib` extension.
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
2. Confirm that the grib file decodes in the grib log file:
@ -33,9 +41,48 @@ To add support for a new grid, two edits must be made:
ls -latr /awips2/edex/data/hdf5/grid/GribModel:7:0:89
Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (7, 0, 89, respectively).
### Grib2 Products
1. Download an example grib2 file and rename to a `*.grib2` extension, then copy to the manual ingest point `/awips2/data_store/ingest/`
wget https://www.unidata.ucar.edu/software/awips2/CPTI_00.50_20180502-000144.grib2 -O cpti.grib2
cp cpti.grib2 /awips2/data_store/ingest/
Remember that the data distribution file (`/awips2/edex/data/utility/common_static/base/distribution/grib.xml`) will match filenames which have the `*.grib*` extension.
2. Confirm that the grib file decodes in the grib log file:
edex log grib
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1200 (sec) Latency: 21.8080 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1180 (sec) Latency: 21.8140 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.4230 (sec) Latency: 21.8360 (sec)
INFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.2240 (sec) Latency: 21.9140 (sec)
...
**Note:** This step will likely fail, because the parameter is not yet defined. The error will look like:
INFO 2020-07-20 20:34:17,710 2565 [GribPersist-1] GridDao: EDEX - Discarding record due to missing or unknown parameter mapping: /grid/2018-05-02_00:01:44.0_(0)/GribModel:161:0:97/null/null/403/Missing/FH/500.0/-999999.0
INFO 2020-07-20 20:34:17,710 2566 [GribPersist-1] Ingest: EDEX: Ingest - grib2:: /awips2/data_store/ingest/CPTI_00.50_20180502-000144.grib2 processed in: 2.3550 (sec)
INFO 2020-07-20 20:34:17,827 2567 [Ingest.GribDecode-6] grib: EDEX - No parameter information for center[161], subcenter[0], tableName[4.2.209.3], parameter value[61]
In order to successfully ingest the example file, [**define the appropriate table**](#grib2-products_4).
3. Check that the hdf5 data directory exists for our unnamed grid
ls -latr /awips2/edex/data/hdf5/grid/GribModel:161:0:97
Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (161, 0, 97, respectively).
---
## Determine Grid Projection
### Grib Products
When the grid was ingested a record was added to the `grid_coverage` table with its navigation information:
psql metadata
@ -46,26 +93,66 @@ When the grid was ingested a record was added to the `grid_coverage` table with
201 | 155 | 4.29699993133545 | 4.29699993133545 | 6378160 | 6356775 | 42.2830009460449 | -72.3610000610352 | -67.0770034790039 | 45.3680000305176 | 45.3680000305176
(1 row)
Compare with the projection info returned by wgrib on the original file:
Compare with the projection info returned by wgrib on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above):
<!--
Using html for this code block so that certain sections within the code can be emphasized (bolded)
-->
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 <b>nx 201 ny 155</b> GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid)
<b>Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000</b>
<b>Latin1 45.368000 Latin2 45.368000</b> LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) <b>Dx 4.297000 Dy 4.297000</b> scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
wgrib -V wrf.grib
Notice that our grib file has a **Lambert Conformal** projection. We will need these values for the next step. Note that **there is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage area.
### Grib2 Products
When the grid was ingested a record was added to the `grid_coverage` table with its navigation information:
psql metadata
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid)
Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000
Latin1 45.368000 Latin2 45.368000 LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
metadata=# select nx,ny,dx,dy,majoraxis,minoraxis,la1,lo1,lov,latin1,latin2 from gridcoverage where id=(select distinct(location_id) from grid_info where datasetid='GribModel:161:0:97');
nx | ny | dx | dy | majoraxis | minoraxis | la1 | lo1 | lov | latin1 | latin2
-----+-----+-------+-------+-----------+-----------+-----------+-----+-----+--------+--------
600 | 640 | 0.005 | 0.005 | | | 40.799999 | 261 | | |
(1 row)
Compare with the projection info returned by wgrib2 on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above):
<!--
Using html for this code block so that certain sections within the code can be emphasized (bolded)
-->
<pre>
wgrib2 -grid -nxny cpti.grib2
1:0:grid_template=0:winds(N/S):
<b>lat-lon grid:(600 x 640)</b> units 1e-06 input WE:NS output WE:SN res 48
lat <b>40.799999</b> to 37.599999 by <b>0.005000</b>
lon <b>260.999999</b> to 263.999999 by <b>0.005000</b> #points=384000:(600 x 640)
...
</pre>
Notice that our grib2 file has a **Lat/lon Grid** projection. Where:
Notice that our grib1 file is a **Lambert Conformal** projection. We will need these values for the next step. Note that **there is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage area.
* **nx** is **600**
* **ny** is **640**
* **dx** is **0.005**
* **dy** is **0.005**
* **la1** is **40.799999**
* **lo1** is **261**
We will need these values for the next step. Note that **there is a tolerance of +/- 0.1 degrees** to keep in mind when defining your coverage (la1 and lo1) area.
---
## Create Grid Projection File
Grid projection files are stored in `/awips2/edex/data/utility/edex_static/base/grib/grids/` and there are four grid coverage types available:
Grid projection files are stored in `/awips2/edex/data/utility/common_static/base/grib/grids/` and there are four grid coverage types available:
1. **lambertConformalGridCoverage** example
1. **lambertConformalGridCoverage** (example: `RUCIcing.xml`)
<lambertConformalGridCoverage>
<name>305</name>
@ -85,7 +172,7 @@ Grid projection files are stored in `/awips2/edex/data/utility/edex_static/base/
<latin2>25.0</latin2>
</lambertConformalGridCoverage>
2. **polarStereoGridCoverage** example
2. **polarStereoGridCoverage** (example `seaice_south1_grid.xml`)
<polarStereoGridCoverage>
<name>405</name>
@ -103,7 +190,7 @@ Grid projection files are stored in `/awips2/edex/data/utility/edex_static/base/
<lov>100.0</lov>
</polarStereoGridCoverage>
3. **latLonGridCoverage** example
3. **latLonGridCoverage** (example `UkmetHR-SHemisphere.xml`)
<latLonGridCoverage>
<name>864162002</name>
@ -121,7 +208,7 @@ Grid projection files are stored in `/awips2/edex/data/utility/edex_static/base/
<lo2>70.416</lo2>
</latLonGridCoverage>
4. **mercatorGridCoverage** example
4. **mercatorGridCoverage** (example `gridNBM_PR.xml`)
<mercatorGridCoverage>
<name>NBM_PR</name>
@ -140,13 +227,15 @@ Grid projection files are stored in `/awips2/edex/data/utility/edex_static/base/
<minorAxis>6371200</minorAxis>
<majorAxis>6371200</majorAxis>
</mercatorGridCoverage>
Copy an existing file file with the same grid projection type (in this case **lambertConformalGridCoverage**) to a new file `wrf.xml`
cd /awips2/edex/data/utility/edex_static/base/grib/grids/
### Grib Products
Copy an existing xml file with the same grid projection type (in this case **lambertConformalGridCoverage**) to a new file `wrf.xml`:
cd /awips2/edex/data/utility/common_static/base/grib/grids/
cp RUCIcing.xml wrf.xml
And edit the new `wrf.xml` to define the projection values (example provided):
And edit the new `wrf.xml` to define the projection values using the [output from wgrib or the database](#determine-grid-projection) (example provided):
vi wrf.xml
@ -168,17 +257,49 @@ And edit the new `wrf.xml` to define the projection values (example provided):
<latin2>45.3680000305176</latin2>
</lambertConformalGridCoverage>
> Notice `<name>201155</name>` defined from the number of grid points (201 x 155). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. WRF).
> **Note**: Notice the `<name>201155</name>` tag was defined from the number of grid points (201 and 155). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. WRF).
### Grib2 Products
Copy an existing xml file with the same grid projection type (in this case **latLonGridCoverage**) to a new file `cpti.xml`:
cd /awips2/edex/data/utility/common_static/base/grib/grids/
cp MRMS-1km.xml cpti.xml
And edit the new `cpti.xml` to define the projection values using the [output from wgrib2 or the database](#grib2-products_1) (example provided):
vi cpti.xml
<latLonGridCoverage>
<name>600640</name>
<description>Small domain for CPTI products</description>
<la1>40.799999</la1>
<lo1>261</lo1>
<firstGridPointCorner>UpperLeft</firstGridPointCorner>
<nx>600</nx>
<ny>640</ny>
<dx>0.005</dx>
<dy>0.005</dy>
<spacingUnit>degree</spacingUnit>
</latLonGridCoverage>
> **Note**: Notice the `<name>384000</name>` tag was defined from the number of grid points (600 and 640). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. CPTI).
---
## Create Model Definition
Model definition XML files are found in **/awips2/edex/data/utility/edex_static/base/grid/models/**. Since our grib1 file has a center ID of 7 (NCEP) we will edit the **gribModels_NCEP-7.xml** file.
Model definition XML files are found in **/awips2/edex/data/utility/common_static/base/grib/models/**.
cd /awips2/edex/data/utility/edex_static/base/grib/models/
### Grid Prodcuts
Since our grib file has a center ID of 7 (NCEP) we will edit the **gribModels_NCEP-7.xml** file.
cd /awips2/edex/data/utility/common_static/base/grib/models/
vi gribModels_NCEP-7.xml
in `<gribModelSet>` add an entry
In `<gribModelSet>` add an entry:
<model>
<name>WRF</name>
@ -190,22 +311,170 @@ in `<gribModelSet>` add an entry
</process>
</model>
save the file and restart EDEX for the changes to take effect.
Save the file and restart EDEX for the changes to take effect:
sudo service edex_camel restart
sudo service edex_camel restart ingestGrib
Now copy the `wrf.grib` file *again* to **/awips2/data_store/ingest/**. If everything is correct we will not see any persistence errors since the grid is now named **WRF** and not **GribModel:7:0:89**.
cp wrd.grib /awips2/data_store/ingest/
cp wrf.grib /awips2/data_store/ingest/
edex log grib
After you have confirmed that the grid was ingested with the given name, you can [edit the D2D product menus to display the new grid](../cave/d2d-edit-menus.html).
After you have confirmed that the grid was ingested with the given name, you can [edit the D2D product menus to display the new grid](../cave/d2d-edit-menus.md).
## Troubleshooting Grib Ingest
### Grib2 Products
Since our grib2 file has a center of 161 (NOAA) we will edit the **gribModels_NOAA-161.xml** file.
cd /awips2/edex/data/utility/common_static/base/grib/models/
vi gribModels_NOAA-161.xml
In `<gribModelSet>`, under the `<-- Subcenter 0 -->` comment, add an entry:
<model>
<name>CPTI</name>
<center>161</center>
<subcenter>0</subcenter>
<grid>600640</grid>
<process>
<id>97</id>
</process>
</model>
Save the model file and restart edex:
sudo service edex_camel restart ingestGrib
Now if you drop `cpti.grib2` into the manual endpoint again, it should ingest without any persistence errors.
---
## Adding a Table
If you ingest a piece of data and the parameter appears as unknown in the metadata database, ensure that the correct parameter tables are in place for the center/subcenter.
Make sure the latitude and longitude entries in your coverage specification file match those of your ingested raw grib file. There is a tolerance of +/- 0.1 degree to keep in mind when defining your coverage area.
The tables are located in **/awips2/edex/data/utility/common_static/base/grib/tables/**. They are then broken into subdirectories using the following structure: **/[Center]/[Subcenter]/4.2.[Discipine].[Category].table**.
If some of the information is unknown, using a grib utility application such as *wgrib* and *wgrib2* (not delivered) can be useful in determining the information that must be added to correctly process a new grib file.
### Grib products
The center and subcenter have been identified previously [here](#grib-products) and [here](#grib-products_1), as 7 and 0, respectively. So, the corresponding directory is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/
To find the **discipline** of a grib product, you need the **process** and **table** values from the grib file. These are output with the `wgrib -V` command:
<pre>
wgrib -V wrf.grib
rec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef
ALBDO=Albedo [%]
timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 <b>process 89 Table 2</b> scan: WE:SN winds(grid)
Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000
Latin1 45.368000 Latin2 45.368000< LatSP 0.000000 LonSP 0.000000
North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8
min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0
</pre>
For our example, the process is **89** and table is **2**. Next, take a look in:
/awips2/edex/data/utility/common_static/base/grid/grib1ParameterConvTable.xml
And find the entry that has grib1 data with TableVersion 2 and Value 89:
<grib1Parameter>
<center>7</center>
<grib1TableVersion>2</grib1TableVersion>
<grib1Value>89</grib1Value>
<grib2discipline>0</grib2discipline>
<grib2category>3</grib2category>
<grib2Value>10</grib2Value>
</grib1Parameter>
Here, we can see the discipline and category values (referred to as x above) are 0 and 3, respectively.
So, the table needed for our example file is:
/awips2/edex/data/utility/common_static/base/grib/tables/7/0/4.2.0.3.table
### Grib2 Products
If you are using a grib2 file, then you can use either the log output or the `-center`, `-subcenter`, and `-full_name` options on `wgrib2` to get the center, subcenter, discipline, category, and parameter information:
The table would be found in the directory structure using this file's center and subcenter. The center can be found by either:
* Running the following command:
wgrib2 -center cpti.grib2
1:0:center=US NOAA Office of Oceanic and Atmospheric Research
...
And then looking up the corresponding value for "US NOAA Office of Oceanic and Atmospheric Research" at [**this website**](https://www.nco.ncep.noaa.gov/pmb/docs/on388/table0.html), where it happens to be **161**.
**OR:**
* Running the following command:
<pre>
wgrib2 -varX cpti.grib2
1:0:var209_255_1_<b>161</b>_3_61
...
</pre>
Where the 4th argument after "var" is the center id, in this case **161**.
To get the subcenter, simply run:
<pre>
wgrib2 -subcenter cpti.grib2
1:0:subcenter=<b>0</b>
...
</pre>
The subcenter of this file is **0**.
So based on the center and subcenter, the corresponding directory is:
/awips2/edex/data/utility/common_static/base/grib/tables/161/0/
To find the exact table, we need the discipline and category:
<pre>
wgrib2 -full_name cpti.grib2
1:0:var<b>209</b>_<b>3</b>_<b>61</b>.500_m_above_mean_sea_level
...
</pre>
In this case the **discipline is 209** and **category is 3**, so the corresponding table is:
4.2.209.3.table
So, the full path to the corresponding table would be:
/awips2/edex/data/utility/common_static/base/grib/tables/161/0/4.2.209.3.table
The parameter ID was also listed in that output as **61**. Make sure that specific parameter information is defined in the table:
<pre>
...
56:56:Reflectivity at -20C:dBZ:ReflectivityM20C
57:57:Reflectivity At Lowest Altitude (RALA):dBZ:ReflectivityAtLowestAltitude
58:58:Merged Reflectivity At Lowest Altitude (RALA):dBZ:MergedReflectivityAtLowestAltitude
59:59:CPTI 80mph+:%:CPTI80mph
<b>61:61:CPTI 110mph+:%:CPTI110mph</b>
</pre>
You will have to restart ingestGrib for the changes to take place:
sudo service edex_camel restart ingestGrib
Now you can try [re-ingesting the grib2 file](#grib2-products).
---
## Troubleshooting Grib Ingest
* Make sure the latitude and longitude entries in your coverage specification file match those of your ingested raw grib file. There is a tolerance of +/- 0.1 degree to keep in mind when defining your coverage area.
* If some of the information is unknown, using a grib utility application such as *wgrib* and *wgrib2* can be useful in determining the information that must be added to correctly process a new grib file.
* If you are experiencing `Segmentation fault` errors when running wgrib2, it may be best to install the latest version using the following command:
yum install wgrib2
And then you may either need to change where `wgrib2` points to, or use `/bin/wgrib2` to run the recently downloaded version.

57
docs/edex/settings.md
View file

@ -1,25 +1,35 @@
# EDEX Settings
## Plugin Configuration
The directory `/awips2/edex/conf/resources` contains configuration text files for specific plugins, which allow for user-defined values which are read by AWIPS plugins on EDEX start:
The directory **/awips2/edex/conf/resources** contains configuration text files for specific plugins, which allow for user-defined values which are read by AWIPS plugins on EDEX start:
acarssounding.properties
autobldsrv.properties
com.raytheon.edex.plugin.gfe.properties
com.raytheon.edex.text.properties
com.raytheon.uf.common.registry.ebxml.properties
com.raytheon.uf.edex.archive.cron.properties
com.raytheon.uf.edex.database.properties
com.raytheon.uf.edex.registry.ebxml.properties
distribution.properties
edex-localization-http.properties
edex-ogc.properties
edex-requestsrv.properties
edex-uengine.properties
eventBus.properties
ftp.properties
goesr.properties
grib.properties
maintenance.properties
proxy.properties
purge.properties
quartz.properties
radar.properties
stats.properties
textdbsrv.properties
warning.properties
Look at *purge.properties* for example:
Look at **purge.properties** for example:
# Master switch to enable and disable purging
purge.enabled=true
@ -38,8 +48,10 @@ Look at *purge.properties* for example:
# Number of days older than the earliest known data to delete.
purge.orphan.buffer=7
...
In *grib.properties*, *goesr.properties*, and *radar.properties* you can adjust the number of decoder threads for each plugin.
In **grib.properties**, **goesr.properties**, and **radar.properties** you can adjust the number of decoder threads for each plugin.
cat radar.properties
@ -52,11 +64,11 @@ In *grib.properties*, *goesr.properties*, and *radar.properties* you can adjust
By default, EDEX starts three "modes": *ingest*, *ingestGrib*, and *request* (each as its own JVM).
The file `/awips2/edex/conf/modes/modes.xml` contains all available mode definitions, including some specific modes for Hydro Server Applications, ebXML Registries, and Data Delivery.
The file **/awips2/edex/conf/modes/modes.xml** contains all available mode definitions, including some specific modes for Hydro Server Applications, ebXML Registries, Data Delivery, and more.
EDEX services are registered through spring, and by including or excluding specific spring files (usually by datatype plugin name) we can finely customize EDEX startup.
In `/awips2/edex/conf/modes/modes.xml` there are a number of unused plugin decoders excluded because the data are not available outside of the SBN:
In **/awips2/edex/conf/modes/modes.xml** there are a number of unused plugin decoders excluded because the data are not available outside of the SBN:
...
<mode name="ingest">
@ -65,26 +77,36 @@ In `/awips2/edex/conf/modes/modes.xml` there are a number of unused plugin decod
<exclude>ebxml.*\.xml</exclude>
<exclude>grib-decode.xml</exclude>
<exclude>grid-staticdata-process.xml</exclude>
<exclude>.*(taf|nctext).*</exclude>
<exclude>.*(dpa|taf|nctext).*</exclude>
<exclude>webservices.xml</exclude>
<exclude>ebxml.*\.xml</exclude>
<exclude>.*datadelivery.*</exclude>
<exclude>.*bandwidth.*</exclude>
<exclude>.*sbn-simulator.*</exclude>
<exclude>hydrodualpol-ingest.xml</exclude>
<exclude>grid-metadata.xml</exclude>
<exclude>.*ogc.*</exclude>
<exclude>obs-ingest-metarshef.xml</exclude>
<exclude>ffmp-ingest.xml</exclude>
<exclude>scan-ingest.xml</exclude>
<exclude>cwat-ingest.xml</exclude>
<exclude>fog-ingest.xml</exclude>
<exclude>vil-ingest.xml</exclude>
<exclude>preciprate-ingest.xml</exclude>
<exclude>qpf-ingest.xml</exclude>
<exclude>fssobs-ingest.xml</exclude>
<exclude>cpgsrv-spring.xml</exclude>
</mode>
...
In this example, OGC, Data Delivery, request, ebXML, and grib plugins are excluded because they are included in their own mode/JVM.
In this example, request, ebXML, grib plugins, OGC and other plugins are excluded because they are included in their own mode/JVM.
> TAF and NCTEXT plugins are disabled here due to performance issues.
> **Note**: TAF and NCTEXT plugins are disabled here due to performance issues.
---
## JVM Memory
The directory `/awips2/edex/etc/` contains files which define the amount of memory used for each of the three EDEX JVMs (ingest, ingestGrib, request):
The directory **/awips2/edex/etc/** contains files which define the amount of memory used for each of the three EDEX JVMs (ingest, ingestGrib, request):
ls -al /awips2/edex/etc/
-rw-r--r-- 1 awips fxalpha 1287 Jul 24 18:41 centralRegistry.sh
@ -94,13 +116,14 @@ The directory `/awips2/edex/etc/` contains files which define the amount of memo
-rw-r--r-- 1 awips fxalpha 848 Jul 24 18:42 profiler.sh
-rw-r--r-- 1 awips fxalpha 1188 Jul 24 18:41 registry.sh
-rw-r--r-- 1 awips fxalpha 601 Jul 24 18:36 request.sh
-rw-r--r-- 1 awips fxalpha 1124 Jul 23 17:22 sbnSimulator.sh
Each file contains the **Xmx** definition for maximum memory:
...
export INIT_MEM=512 # in Meg
export MAX_MEM=4096 # in Meg
...
export INIT_MEM=512 # in Meg
export MAX_MEM=4096 # in Meg
After editing these files, you must restart edex (`service edex_camel restart`).
After editing these files, you must restart : `service edex_camel restart`.
---

BIN
docs/images/caveHeader.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 65 KiB

BIN
docs/images/copyToMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 125 KiB

BIN
docs/images/createProjection.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 74 KiB

BIN
docs/images/edexServerDialog.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
docs/images/errorPurgingLogs.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
docs/images/errorWorkbenchNull.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
docs/images/exportImageDialog.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
docs/images/exportMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 37 KiB

BIN
docs/images/fileMenuManageBundles.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 138 KiB

BIN
docs/images/fileMenuNewMap.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 164 KiB

BIN
docs/images/goes_16_conus.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 504 KiB

After

Width:  |  Height:  |  Size: 365 KiB

BIN
docs/images/goes_16_glmmenu.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 448 KiB

BIN
docs/images/goes_17_ak.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 434 KiB

BIN
docs/images/goes_17_hi.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 484 KiB

BIN
docs/images/imagePropsIcon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 7.3 KiB

BIN
docs/images/importMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 34 KiB

BIN
docs/images/ipv6ProblemStep2.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 15 KiB

BIN
docs/images/ipv6ProblemStep3a.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 124 KiB

BIN
docs/images/ipv6ProblemStep3b.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 72 KiB

BIN
docs/images/ipv6ProblemStep4.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 39 KiB

BIN
docs/images/ipv6ProblemStep5.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 77 KiB

BIN
docs/images/ipv6ProblemStep6.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 140 KiB

BIN
docs/images/nsharpControlsGraphs.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
docs/images/nsharpControlsInsets.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 14 KiB

BIN
docs/images/nsharpControlsTables.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
docs/images/nsharpGraphs.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/nsharpHodograph.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/nsharpIcon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 7.8 KiB

BIN
docs/images/nsharpInsets.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/nsharpInventory.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 960 KiB

BIN
docs/images/nsharpSkewT.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/nsharpTables.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/nsharpWindHeightTemp.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
docs/images/perspectivesMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 150 KiB

BIN
docs/images/productBroswerIcon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 7.7 KiB

BIN
docs/images/radarDisplayControls.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 144 KiB

BIN
docs/images/satelliteMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 174 KiB

BIN
docs/images/surfaceGLMMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 117 KiB

BIN
docs/images/textWorkstation.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 57 KiB

BIN
docs/images/toolsMenuDisplayTools.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 220 KiB

BIN
docs/images/toolsMenuRadarTools.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 229 KiB

BIN
docs/images/volumeBrowserAnimationMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
docs/images/volumeBrowserDisplayMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 37 KiB

BIN
docs/images/volumeBrowserFields.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
docs/images/volumeBrowserLabeled.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 305 KiB

BIN
docs/images/volumeBrowserLoad.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 83 KiB

BIN
docs/images/volumeBrowserSoundingMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 93 KiB

BIN
docs/images/volumeBrowserSources.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 58 KiB

BIN
docs/images/volumeBrowserToolsMenu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 30 KiB

BIN
docs/images/warngen.png Normal file → Executable file
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 118 KiB

BIN
docs/images/warngenDMTS.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 392 KiB

BIN
docs/images/warngenHeaderBlock.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 38 KiB

BIN
docs/images/warngenLocalRadar.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
docs/images/warngenLocalization.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 26 KiB

0
docs/images/warnpoly.png → docs/images/warngenPoly.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 193 KiB

After

Width:  |  Height:  |  Size: 193 KiB

38
docs/index.md
View file

@ -1,6 +1,6 @@
# Unidata AWIPS User Manual
[https://www.unidata.ucar.edu/software/awips2](https://www.unidata.ucar.edu/software/awips2)
[https://www.unidata.ucar.edu/software/awips2](https://www.unidata.ucar.edu/software/awips2)
---
@ -13,18 +13,20 @@
[postgres]: www.postgresql.org
[hdf5]: http://www.hdfgroup.org/HDF5/
[eclipse]: http://www.eclipse.org
[camel]: http://camel.apache.org/
[spring]: http://www.springsource.org/
[hibernate]: http://www.hibernate.org/
[qpid]: http://qpid.apache.org
[idd]: https://www.unidata.ucar.edu/projects/#idd
[camel]: http://camel.apache.org/
[spring]: http://www.springsource.org/
[hibernate]: http://www.hibernate.org/
[qpid]: http://qpid.apache.org
AWIPS (the Advanced Weather Interactive Processing System) is a meteorological decoding, display, and analysis package originally developed by the National Weather Service and Raytheon. UCAR's Unidata Program Center develops and supports a modified non-operational version of AWIPS for use in research and education by [UCAR member institutions](http://president.ucar.edu/governance/members/universities-representatives), which is released as open source software, free to download and use by anyone.
AWIPS takes a unified approach to data ingest, most data ingested into the system comes through the [LDM](#ldm) client pulling data feeds from the [Unidata IDD](#idd). Various raw data and product files (netCDF, grib, BUFR, ASCII text, gini, AREA) are decoded and stored as HDF5 and Postgres metadata by [EDEX](install/install-edex), which serves products and data over http.
The Advanced Weather Interactive Processing System (AWIPS) is a meteorological software package. It is used for decoding, displaying, and analyzing data, and was originally developed for the National Weather Service (NWS) by Raytheon. There is a division here at UCAR called the Unidata Program Center (UCP) which develops and supports a modified non-operational version of AWIPS for use in research and education by [UCAR member institutions](http://president.ucar.edu/governance/members/universities-representatives). This is released as open source software, free to download and use by anyone.
AWIPS takes a unified approach to data ingest, where most data ingested into the system comes through the [LDM](#ldm) client pulling data feeds from the [Unidata IDD](https://www.unidata.ucar.edu/projects/#idd). Various raw data and product files (netCDF, grib, BUFR, ASCII text, gini, AREA) are decoded and stored as HDF5 files and Postgres metadata by [EDEX](install/install-edex), which serves products and data over http.
Unidata supports two data visualization frameworks: [CAVE](install/install-cave) (an Eclipse-built Java application which runs on Linux, Mac, and Windows), and the [python-awips](http://python-awips.readthedocs.io).
> **Note**: Our version of CAVE is a **non-operational** version. It does not support some features of NWS AWIPS. Warnings and alerts cannot be issued from Unidata's CAVE. Additional functionality may not be available as well.
Unidata supports two data visualization frameworks: [CAVE](install/install-cave) (an Eclipse-built Java application which runs on Linux, Mac, and Windows), and the [python-awips](http://python-awips.readthedocs.io).
![CAVE](https://www.unidata.ucar.edu/software/awips2/images/Unidata_AWIPS2_CAVE.png)
@ -40,13 +42,13 @@ Unidata supports two data visualization frameworks: [CAVE](install/install-cave)
## License
Unidata AWIPS source code and binaries (RPMs) are considered to be in the public domain, meaning there are no restrictions on any download, modification, or distribution in any form (original or modified). Unidata AWIPS contains no proprietery content and is therefore not subject to export controls as stated in the Master Rights licensing file.
Unidata AWIPS source code and binaries (RPMs) are considered to be in the public domain, meaning there are no restrictions on any download, modification, or distribution in any form (original or modified). Unidata AWIPS contains no proprietery content and is therefore not subject to export controls as stated in the Master Rights licensing file.
---
## AWIPS Data in the Cloud
Unidata and XSEDE Jetstream have partnered to offer a EDEX data server in the cloud, open to the Unidata university community. Select the server in the Connectivity Preferences dialog, or enter **`edex-cloud.unidata.ucar.edu`** (without *http://* before, or *:9581/services* after).
Unidata and XSEDE Jetstream have partnered to offer an EDEX data server in the cloud, open to the community. Select the server in the Connectivity Preferences dialog, or enter **`edex-cloud.unidata.ucar.edu`** (without *http://* before, or *:9581/services* after).
![EDEX in the cloud](/images/boEbFSf28t.gif)
@ -54,13 +56,14 @@ Unidata and XSEDE Jetstream have partnered to offer a EDEX data server in the cl
## Distributed Computing
AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. While originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS can consist of a dozen servers or more, because the AWIPS source code was hard-coded with the NWS network configuration, the early Unidata releases were stripped of operation-specific configurations and plugins, and released specifically for standalone installation. This made sense given that a single EDEX instance with a Solid State Drive could handle most of the entire NOAAport data volume. However, with GOES-R(16) coming online, and more gridded forecast models being created at finer temporal and spatial resolutions, there was now a need to distribute EDEX data decoding in order to handle this firehose of data.
AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. Because AWIPS was originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS can consist of a dozen servers or more, Unidata modified the package to be more applicable in the University setting. Because the AWIPS source code was hard-coded with the NWS network configuration, the early Unidata releases were stripped of operation-specific configurations and plugins, and released specifically for standalone installation. This made sense given that a single EDEX instance with a Solid State Drive (SSD) could handle most of the entire NOAAport data volume. However, with GOES-R(16) now online, and more gridded forecast models being created at finer temporal and spatial resolutions, there was a need to distribute EDEX data decoding in order to handle this firehose of data.
* Read More: [Distributed EDEX](edex/distributed-computing)
---
<img style="float:right;width:450px;" src="https://www.unidata.ucar.edu/software/awips2/images/awips2_coms.png">
## Software Components
* [EDEX](#edex)
@ -111,11 +114,10 @@ edexBridge, invoked in the LDM configuration file `/awips2/ldm/etc/ldmd.conf`, i
[http://www.hdfgroup.org/HDF5/](http://www.hdfgroup.org/HDF5/)
[**Hierarchical Data Format (v.5)**][hdf5] is the primary data storage format used by AWIPS for processed grids, satellite and radar imagery and other products. Similar to netCDF, developed and supported by Unidata, HDF5 supports multiple types of data within a single file. For example, a single HDF5 file of radar data may contain multiple volume scans of base reflectivity and base velocity as well as derived products such as composite reflectivity. The file may also contain data from multiple radars. HDF5 is stored in `/awips2/edex/data/hdf5/`
[**Hierarchical Data Format (v.5)**][hdf5] is the primary data storage format used by AWIPS for processed grids, satellite and radar imagery and other products. Similar to netCDF, developed and supported by Unidata, HDF5 supports multiple types of data within a single file. For example, a single HDF5 file of radar data may contain multiple volume scans of base reflectivity and base velocity as well as derived products such as composite reflectivity. The file may also contain data from multiple radars. HDF5 data is stored on the EDEX server in `/awips2/edex/data/hdf5/`.
### PyPIES (httpd-pypies)
### PyPIES
**PyPIES**, Python Process Isolated Enhanced Storage, was created for AWIPS to isolate the management of HDF5 Processed Data Storage from the EDEX processes. PyPIES manages access, i.e., reads and writes, of data in the HDF5 files. In a sense, PyPIES provides functionality similar to a DBMS (i.e PostgreSQL for metadata); all data being written to an HDF5 file is sent to PyPIES, and requests for data stored in HDF5 are processed by PyPIES.
PyPIES is implemented in two parts: 1. The PyPIES manager is a Python application that runs as part of an Apache HTTP server, and handles requests to store and retrieve data. 2. The PyPIES logger is a Python process that coordinates logging. PyPIES is started and stopped by `edex start` and `edex stop`, and is controlled by the system script `/etc/rc.d/init.d/httpd-pypies`
**PyPIES**, Python Process Isolated Enhanced Storage, (httpd-pypies) was created for AWIPS to isolate the management of HDF5 Processed Data Storage from the EDEX processes. PyPIES manages access, i.e., reads and writes, of data in the HDF5 files. In a sense, PyPIES provides functionality similar to a DBMS (i.e PostgreSQL for metadata); all data being written to an HDF5 file is sent to PyPIES, and requests for data stored in HDF5 are processed by PyPIES.
PyPIES is implemented in two parts: 1. The PyPIES manager is a Python application that runs as part of an Apache HTTP server, and handles requests to store and retrieve data. 2. The PyPIES logger is a Python process that coordinates logging. PyPIES is started and stopped by `edex start` and `edex stop`, and is controlled by the system script `/etc/rc.d/init.d/httpd-pypies`.

40
docs/install/common-problems.md Normal file
View file

@ -0,0 +1,40 @@
# Common Problems
## Windows CAVE Start Up Error
One common error some users are seeing manifests itself just after selecting an EDEX server to connect to. The following error dialogs may show up:
![](../images/errorPurgingLogs.png)
![](../images/errorWorkbenchNull.png)
These errors are actually happening because the Windows machine is using IPv6, which is not compatible with AWIPS at this time.
**To fix the issue simply follow these steps:**
>**Note**: These screenshots may vary from your system.
**1. Close all error windows and any open windows associated with CAVE.**
**2. In the Windows 10 search field, search for "control panel".**
![](../images/ipv6ProblemStep2.png)
**3. Once in the Control Panel, look for "Network and Sharing Center".**
![](../images/ipv6ProblemStep3a.png)
![](../images/ipv6ProblemStep3b.png)
**4. Select the adapter for your current connection (should be either "Ethernet" or "Wi-Fi").**
![](../images/ipv6ProblemStep4.png)
**5. Click on "Properties".**
![](../images/ipv6ProblemStep5.png)
**6. Uncheck "Internet Protocol Version 6 (TCP/IPv6)" and select OK.**
![](../images/ipv6ProblemStep6.png)
**7. Restart CAVE.**
---

17
docs/install/install-cave.md
View file

@ -1,11 +1,13 @@
# Install CAVE
## Download and Install CAVE
> [Release 18.1.1-7, March 25, 2019](https://www.unidata.ucar.edu/blogs/news/category/AWIPS)
## Linux
## Linux
| | |
|:----------------------------------------:|:--|
| <h1><i class="fa fa-linux"></i></h1> | <h4>[awips_install.sh --cave <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/software/awips2/awips_install.sh) </h4> <p>For CentOS/Red Hat 6 and 7. Installs to /awips2/cave and writes files to ~/caveData.</p> <tt><code>chmod 755 awips_install.sh<br>sudo ./awips_install.sh --cave</code></tt><p>Run CAVE from the Linux Desktop menu Applications > Internet > AWIPS CAVE, or from the command line as simply `cave`.</p> <div class="admonition note"><p class="admonition-title">System Requirements</p><ul><li>x86_64 CentOS/RHEL 6 or 7</li><li>OpenGL 2.0 capable device</li><li>4GB RAM</li><li><a href="http://www.nvidia.com/Download/index.aspx?lang=en-us">Latest NVIDIA driver</a></li><li>approx. 2GB disk space for data caching (~/caveData)</li></ul></div><p>You can reset CAVE at any time by removing the **~/caveData** directory (on macOS **~/Library/caveData**) and reconnecting to an EDEX server. </p> |
| <h1><i class="fa fa-linux"></i></h1> | <h4>[awips_install.sh --cave <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/software/awips2/awips_install.sh) </h4> <p>For CentOS/Red Hat 6 and 7. Installs to /awips2/cave and writes files to ~/caveData.</p> <tt><code>chmod 755 awips_install.sh<br>sudo ./awips_install.sh --cave</code></tt><p>Run CAVE from the Linux Desktop menu Applications > Internet > AWIPS CAVE, or from the command line as simply `cave`.</p> <div class="admonition note"><p class="admonition-title">System Requirements</p><ul><li>x86_64 CentOS/RHEL 6 or 7</li><li>OpenGL 2.0 capable device</li><li>4GB RAM</li><li><a href="http://www.nvidia.com/Download/index.aspx?lang=en-us">Latest NVIDIA driver</a></li><li>NVIDIA Quadro graphics card is recommended for full visualization quality</li><li>approx. 2GB disk space for data caching (~/caveData)</li></ul></div><p>You can reset CAVE at any time by removing the **~/caveData** directory (on macOS **~/Library/caveData**) and reconnecting to an EDEX server. </p> |
## macOS
| | |
@ -13,10 +15,10 @@
| <h1><i class="fa fa-apple"></i></h1>|<h4> Download and install both<br>[awips-cave.dmg <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/downloads/awips2/awips-cave.dmg)<br>[awips-python.pkg <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/downloads/awips2/awips-python.pkg)</h4> <p>**Supported Graphics Devices for macOS**<li>Intel HD Graphics</li><li>Intel Iris</li><li>NVIDIA GeForce</li></p><p>**Unsupported Graphics Devices for macOS**<li>AMD Radeon R9</li><li>AMD Radeon Pro</li><li>AMD FirePro D300</li></p> <p>Writes and syncs files to ~/Library/caveData.</p> <p>**awips-python.pkg** is not a prerequisite, and CAVE will still run and display data without it, but to use any derived parameter functions such as wind barbs/arrows and grid parameters on various vertical coordinates, jep must be installed in some way (it is assumed in /Library/Python/2.7/site-packages/jep/)</p>|
## Windows
## Windows
| | |
|:----------------------------------------:|:--|
| <h1><i class="fa fa-windows"></i> </h1> | <h4> [awips-cave.msi <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/downloads/awips2/awips-cave.msi)</h4><p>Writes files to **~/caveData** (in your user home directory)</p> <p>Requires <b>Python 3</b>, <b>Numpy</b>, and <b>Jep</b> be installed</p> <p>Requires **PYTHONHOME** be defined</p> <p>In addition to the application directory, the MSI installer will attempt to copy the *[gridslice](https://github.com/mjames-upc/gridslice)* shared library to `$PYTHONHOME/Dlls`. If the `$PYTHONHOME` environmental variable is not defined, *gridslice* will not be installed. You can always rerun the installer after defining `$PYTHONHOME` and then check that the file `gridslice.pyd` is installed in `$PYTHONHOME/Dlls`.</p> <p>CAVE will still run without gridslice, but certain bundles which use derived parameters, such as [isentropic analyses](../cave/d2d-grids/#isentopic-analysis-270k-320k), will not load. </p> |
| <h1><i class="fa fa-windows"></i> </h1> |<b>Method 1: Install Directly on Windows</b> <h4> [awips-cave.msi <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/downloads/awips2/awips-cave.msi)</h4><p>Writes files to **~/caveData** (in your user home directory)</p> <p>Requires <b>Python 3</b>, <b>Numpy</b>, and <b>Jep</b> be installed</p> <p>Requires **PYTHONHOME** be defined</p> <p>In addition to the application directory, the MSI installer will attempt to copy the *[gridslice](https://github.com/mjames-upc/gridslice)* shared library to `$PYTHONHOME/Dlls`. If the `$PYTHONHOME` environmental variable is not defined, *gridslice* will not be installed. You can always rerun the installer after defining `$PYTHONHOME` and then check that the file `gridslice.pyd` is installed in `$PYTHONHOME/Dlls`.</p> <p>CAVE will still run without gridslice, but certain bundles which use derived parameters, such as [isentropic analyses](../cave/d2d-grids/#isentopic-analysis-270k-320k), will not load. </p> |
@ -26,9 +28,13 @@
<div class="admonition note">
<h4>1) Download and install <strong><a href="https://conda.io/miniconda.html">Miniconda Python 3.7 for Windows</a></strong><a class="headerlink" href="#1-download-and-install-miniconda-python-37-for-windows" title="Permanent link"></a></h4><ul><li>Allow Miniconda3 to set <strong>PATH</strong> and other environment variables.</li><li><p>Ensure that <strong>PYTHONHOME</strong> is set to the Miniconda3 location.</p><p><img alt="" src="../../images/windows_envvars.png" /></p><p>If <strong>PYTHONHOME</strong> is not set, the <strong>gridslice</strong> Python module will not be installed or available.</p></li></ul><h4>2) Install dependent Python packages</h4><ul><li><code>pip install numpy==1.15.1 jep==3.8.2</code></li></ul><h4>3) Run <a href="https://www.unidata.ucar.edu/downloads/awips2/awips-cave.msi">awips-cave.msi <i class="fa fa-download"></i></a><a class="headerlink" href="#3-run-awips-cave-1amd64msi" title="Permanent link"></a></h4>
<p><b>**NOTE:</b> It has been noted that these instructions may result in the failed installation of jep and/or other required python components. We are working on a solution to install the python dependencies properly. If this failure occurs, then numerous products will not render properly. At this time we offer a second option below for Windows users -- download a pre-packaged Linux VM which contains CAVE. We're sorry for the inconvenience and appreciate your patience.</p>
</div>
---
| | |
|:----------------------------------------:|:--|
| <h1><i class="fa fa-windows"></i> </h1> | <b>Method 2: Install Linux VM on Windows</b> <h4> [unidata_cave.zip <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/downloads/awips2/unidata_cave.zip)</h4><p>Requires **VMWare Workstation Player** to be installed (free software)</p><p>Once zipped file is downloaded, unzip the folder by right-clicking and selecting "Extract All". All files will be extracted into their own folder.</p><p>Open VMWare Player and go to Player >> File... >> Open, and then find the folder that was created from extracting the zipped file. There should be one file called "CentOS 7 - Unidata CAVE". Select that and it'll add a new option available in the right hand side VMWare Menu. Run that new option, if it asks if it's been moved or copied, select copied.</p><p>The user is named awips, the password is awips. The root password is unidataAWIPS. There should be a CAVE application icon on the desktop that will run CAVE. It should also run from the commandline with the command "cave".</p> |
## AWIPS Data in the Cloud
@ -47,4 +53,3 @@ You can reset CAVE by removing the **caveData** directory and reconnecting to an
* Linux: `/home/<user>/caveData`
* macOS: `/Users/<user>/Library/caveData`
* Windows: `C:\Users\<user>\caveData`

45
docs/install/install-edex.md
View file

@ -1,7 +1,8 @@
# Install EDEX
## Download and Install EDEX
> [Release 18.1.1-7, MArch 25, 2019](https://www.unidata.ucar.edu/blogs/news/category/AWIPS)
> [Release 18.1.1-7, March 25, 2019](https://www.unidata.ucar.edu/blogs/news/category/AWIPS)
| | |
|:----------------------------------------:|:--|
@ -28,12 +29,12 @@ All of these command should be run as **root**
### 2. Install EDEX
>
>Download and run [sudo ./awips_install.sh --edex <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/software/awips2/awips_install.sh)
>
>
> wget https://www.unidata.ucar.edu/software/awips2/awips_install.sh
> chmod 755 awips_install.sh
> sudo ./awips_install.sh --edex
> /usr/bin/edex setup
>
>
>
>
>!!! note "**awips_install.sh --edex** will perform the following steps (it's always a good idea to review downloaded shell scripts):"
@ -47,7 +48,7 @@ All of these command should be run as **root**
### 3. EDEX Setup
>
> The command `edex setup` will try to determine your fully-qualified domain name and set it in `/awips2/edex/bin/setup.env`. EDEX Server Administrators should double-check that the addresses and names defined in setup.env are resolvable from both inside and outside the server, and make appropriate edits to `/etc/hosts` if necessary.
> The command `edex setup` will try to determine your fully-qualified domain name and set it in `/awips2/edex/bin/setup.env`. EDEX Server Administrators should double-check that the addresses and names defined in setup.env are resolvable from both inside and outside the server, and make appropriate edits to `/etc/hosts` if necessary.
>
>For example, in the XSEDE Jetstream cloud, the fully-qualified domain name defined in `/awips2/edex/bin/setup.env`
>
@ -64,13 +65,13 @@ All of these command should be run as **root**
### 4. Configure iptables
>
>Configure iptables to allow TCP connections on ports 9581 and 9582 if you want to serve data to CAVE clients and the Python API.
>
>
>If you are running a Registry (Data Delivery) server, you will also want to open port **9588**.
>
>- **To open ports to all connections**
>
>
> vi /etc/sysconfig/iptables
>
>
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
@ -87,11 +88,11 @@ All of these command should be run as **root**
> COMMIT
>
>- **To open ports to specific IP addresses**
>
>
> In this example, the IP range `128.117.140.0/24` will match all 128.117.140.* addresses, while `128.117.156.0/24` will match 128.117.156.*.
>
>
> vi /etc/sysconfig/iptables
>
>
> *filter
> :INPUT DROP [0:0]
> :FORWARD DROP [0:0]
@ -111,7 +112,7 @@ All of these command should be run as **root**
> #-A EDEX -m state --state NEW -p tcp --dport 9588 -j ACCEPT # for registry/dd
> -A EDEX -j REJECT
> COMMIT
>
>
>**Restart iptables**
>
> service iptables restart
@ -120,7 +121,7 @@ All of these command should be run as **root**
>Failed to restart iptables.service: Unit iptables.service failed to load: No such file or directory.*
>
>The solution is:
>
>
> yum install iptables-services
> systemctl enable iptables
> service iptables restart
@ -137,7 +138,7 @@ All of these command should be run as **root**
> service qpidd start
> service edex_camel start
>
>The fifth service, **edex_ldm**, does **not run at boot** to prevent filling up disk space if EDEX is not running.
>The fifth service, **edex_ldm**, does **not run at boot** to prevent filling up disk space if EDEX is not running.
>
> ldmadmin start
>
@ -148,7 +149,7 @@ All of these command should be run as **root**
>To restart EDEX
>
> edex restart
---
@ -157,7 +158,7 @@ All of these command should be run as **root**
### /etc/security/limits.conf
**/etc/security/limits.conf** defines the number of user processes and files (this step is automatically performed by `./awips_install.sh --edex`). Without these definitions, Qpid is known to crash during periods of high ingest.
awips soft nproc 65536
awips soft nofile 65536
@ -184,11 +185,11 @@ All of these command should be run as **root**
### SSD Mount
Though a Solid State Drive is not required, it is *strongly encouraged* in order to handle the amount of disk IO for real-time IDD feeds.
Though a Solid State Drive is not required, it is *strongly encouraged* in order to handle the amount of disk IO for real-time IDD feeds.
The simplest configuration would be to mount an 500GB+ SSD to **/awips2** to contain both the installed software (approx. 20GB) and the real-time data (approx. 150GB per day).
The default [purge rules]() are configured such that **/awips2** does not exceed 450GB. **/awips2/data_store** is scoured every hour and should not exceed 50GB.
The default [purge rules]() are configured such that **/awips2** does not exceed 450GB. **/awips2/data_store** is scoured every hour and should not exceed 50GB.
If you want to increase EDEX data retention you should mount a large disk to **/awips2/edex/data/hdf5** since this will be where the archived processed data exists, and any case studies created.
@ -200,7 +201,7 @@ If you want to increase EDEX data retention you should mount a large disk to **/
---
### Configure LDM Feeds
### Configure LDM Feeds
EDEX installs its own version of the LDM to the directory **/awips2/ldm**. As with a the default LDM configuration, two files are used to control what IDD feeds are ingested:
@ -210,7 +211,7 @@ EDEX installs its own version of the LDM to the directory **/awips2/ldm**. As w
REQUEST FNEXRAD|IDS|DDPLUS|UNIWISC ".*" idd.unidata.ucar.edu
REQUEST NGRID ".*" idd.unidata.ucar.edu
REQUEST NOTHER "^TIP... KNES.*" idd.unidata.ucar.edu
!!! note "[read more about ldmd.conf in the LDM User Manual](https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/ldmd.conf.html)"
* **/awips2/ldm/etc/pqact.conf** - specifies the WMO headers and file pattern actions to request:
@ -230,13 +231,13 @@ EDEX installs its own version of the LDM to the directory **/awips2/ldm**. As w
### Directories to Know
* `/awips2` - Contains all of the installed AWIPS software.
* `/awips2` - Contains all of the installed AWIPS software.
* `/awips2/edex/logs` - EDEX logs.
* `/awips2/httpd_pypies/var/log/httpd` - httpd-pypies logs.
* `/awips2/database/data/pg_log` - PostgreSQL logs.
* `/awips2/qpid/log` - Qpid logs.
* `/awips2/edex/data/hdf5` - HDF5 data store.
* `/awips2/edex/data/utility` - Localization store and configuration files.
* `/awips2/edex/data/hdf5` - HDF5 data store.
* `/awips2/edex/data/utility` - Localization store and configuration files.
* `/awips2/ldm/etc` - Location of **ldmd.conf** and **pqact.conf**
* `/awips2/ldm/logs` - LDM logs.
* `/awips2/data_store` - Raw data store.

17
docs/install/start-edex.md
View file

@ -1,24 +1,25 @@
# EDEX Basic Commands
To start all EDEX services
To start all EDEX services:
edex start
To stop all EDEX services
To stop all EDEX services:
edex stop
These command will start and stop five EDEX service files installed into `/etc/init.d/`, four of which are run on boot:
These commands will start and stop five EDEX service files installed into **/etc/init.d/**, four of which are run on boot:
service postgres start
service httpd-pypies start
service qpidd start
service edex_camel start
The fifth, `edex_ldm`, does **not run at boot** to prevent filling up disk space if EDEX is not running.
The fifth, `edex_ldm`, does **not run at boot** to prevent filling up disk space if EDEX is not running:
service edex_ldm start
All of these services are started and stopped by a single program `edex`,
All of these services are started and stopped by the single program: `edex` as mentioned above.
---
@ -71,7 +72,7 @@ To start all EDEX services *except* the LDM:
This command configures and/or confirms that the EDEX hostname and IP address definitions exist (`edex setup` is run by `edex start`).
> If your EDEX server is running but you see the message "Connectivity Error: Unable to validate localization preferences" in CAVE, it may mean that the domain name defined in `/awips2/edex/bin/setup.env` can not be resolved from *outside* the server. Some machines have different **internally-resolved** and **externally-resolved** domain names (cloud-based especially). The name defined in `setup.env` must be **externally-resolvable**.
> **Note**: If your EDEX server is running but you see the message **"Connectivity Error: Unable to validate localization preferences"** in CAVE, it may mean that the domain name defined in **/awips2/edex/bin/setup.env** can not be resolved from *outside* the server. Some machines have different *internally-resolved* and *externally-resolved* domain names (cloud-based especially). The name defined in **setup.env** must be *externally-resolvable*.
---
@ -125,7 +126,7 @@ Shows a list of the the Qpid message queue to monitor data ingest (messages in v
### edex users
To see a list of clients connecting to your EDEX server, use the `edex users [YYYYMMDD]` command, where `YYYYMMDD` is the optional date string.
To see a list of clients connecting to your EDEX server, use the `edex users [YYYYMMDD]` command, where `[YYYYMMDD]` is the optional date string.
edex users
@ -139,4 +140,4 @@ To see a list of clients connecting to your EDEX server, use the `edex users [YY
### edex purge
To view any stuck purge jobs in PortgreSQL (a rare but serious problem if your disk fills up). The solution to this is to run `edex purge reset`.
To view any stuck purge jobs in PortgreSQL (a rare but serious problem if your disk fills up). The solution to this is to run `edex purge reset`.

5
docs/python/overview.md Normal file
View file

@ -0,0 +1,5 @@
# Python API
The **python-awips** package provides a data access framework (DAF) for requesting grid and geometry datasets from an EDEX server. The DAF can be used to interact in command line, or scripting form, with the EDEX. This is an alternative to using CAVE to interact with the data.
Thorough documentation for python-awips can be found [**here**](http://unidata.github.io/python-awips/).

2
environment.yml
View file

@ -3,7 +3,7 @@ name: awips-docs
channels:
- conda-forge
dependencies:
- python=2.7
- python=3.8
- pip
- pip:
- mkdocs==0.17.5

57
mkdocs.yml
View file

@ -1,8 +1,10 @@
site_name: Unidata AWIPS User Manual
site_url: http://unidata.github.io/awips2
site_description: Documentation for all things AWIPS.
site_author: mjames@ucar.edu
theme: unidata
site_author: support-awips@unidata.ucar.edu
theme:
name: unidata
search_index_only: true
repo_url: https://github.com/Unidata/awips2
edit_uri: edit/unidata_18.2.1/docs/
site_favicon: images/favicon.ico
@ -30,16 +32,17 @@ pages:
- Home: index.md
- Install CAVE: install/install-cave.md
- Install EDEX: install/install-edex.md
- Common Problems: install/common-problems.md
- CAVE User Manual:
- D2D Perspective: cave/d2d-perspective.md
- Maps, Views, Projections: cave/maps-views-projections.md
- Bundles and Procedures: cave/bundles-and-procedures.md
- Localization Perspective: cave/localization-perspective.md
- NCP Perspective: cave/ncp-perspective.md
# - NCP Perspective: cave/ncp-perspective.md
- NSHARP: cave/nsharp.md
- WarnGen Walkthrough: cave/warngen.md
- GOES 16/17: cave/goes-16-satellite.md
- GIS and Shapefiles: cave/d2d-gis-shapefiles.md
# - GIS and Shapefiles: cave/d2d-gis-shapefiles.md
- Volume Browser: cave/d2d-gridded-models.md
- Display Tools: cave/d2d-tools.md
- Radar Tools: cave/d2d-radar-tools.md
@ -47,7 +50,7 @@ pages:
- Change Localization: cave/cave-localization.md
- Import/Export: cave/import-export.md
- EDEX User Manual:
- EDEX Start and Stop: install/start-edex.md
- EDEX Basic Commands: install/start-edex.md
- EDEX Settings: edex/settings.md
- Distributed EDEX: edex/distributed-computing.md
- Docker EDEX: edex/edex-ingest-docker-container.md
@ -64,40 +67,42 @@ pages:
# - Hazard Alerts: cave/hazard-services-alert.md
# - Hazard Settings: cave/hazard-services-settings.md
# - Examples: cave/hazard-services-example.md
- Supported Data Types:
- Gridded Model Display: cave/d2d-grids.md
- Surface Obs: cave/d2d-pointdata-surface-obs.md
- Satellite Imagery: cave/d2d-satellite.md
- NEXRAD Radar: cave/d2d-radar.md
- Upper Air: cave/d2d-uair.md
- NCEP/Hydro: cave/d2d-hydro.md
- Map Overlays: cave/d2d-map-resources.md
- Python API:
- Overview: python/python-awips-data-access.md
- AWIPS Grids and Cartopy: python/awips-grids-and-cartopy.md
- Satellite Imagery: python/satellite-imagery.md
- NEXRAD Level 3 Radar: python/nexrad-level-3-radar.md
- Upper air BUFR Soundings: python/upper-air-bufr-soundings.md
- Model Sounding Data: python/model-sounding-data.md
- Map Resources and Topography: python/map-resources-and-topography.md
- Surface Obs Plot with MetPy: python/surface-obs-plot-metpy.md
- Development:
- AWIPS Development Environment (ADE): dev/awips-development-environment.md
# - Supported Data Types:
# - Gridded Model Display: cave/d2d-grids.md
# - Surface Obs: cave/d2d-pointdata-surface-obs.md
# - Satellite Imagery: cave/d2d-satellite.md
# - NEXRAD Radar: cave/d2d-radar.md
# - Upper Air: cave/d2d-uair.md
# - NCEP/Hydro: cave/d2d-hydro.md
# - Map Overlays: cave/d2d-map-resources.md
- Development: dev/awips-development-environment.md
- Python API: python/overview.md
# - Overview: python/python-awips-data-access.md
# - AWIPS Grids and Cartopy: python/awips-grids-and-cartopy.md
# - Satellite Imagery: python/satellite-imagery.md
# - NEXRAD Level 3 Radar: python/nexrad-level-3-radar.md
# - Upper air BUFR Soundings: python/upper-air-bufr-soundings.md
# - Model Sounding Data: python/model-sounding-data.md
# - Map Resources and Topography: python/map-resources-and-topography.md
# - Surface Obs Plot with MetPy: python/surface-obs-plot-metpy.md
- Appendix:
- AWIPS Grid Parameters: appendix/appendix-grid-parameters.md
- Maps Database: python/maps-database.md
- Acronyms and Abbreviations: appendix/appendix-acronyms.md
- COTS and FOSS: appendix/appendix-cots.md
# - COTS and FOSS: appendix/appendix-cots.md
- WSR-88D Product Table: appendix/appendix-wsr88d.md
- Keyboard Shortcuts: cave/cave-keyboard-shortcuts.md
extra_css:
- css/extra.css
extra_javascript:
- assets/javascripts/application-0b7df094bf.js
markdown_extensions:
- fontawesome_markdown:
- toc:
permalink:
- admonition:
copyright: Copyright &copy; 2017 Unidata Program Center.
copyright: Copyright &copy; 2020 Unidata Program Center.