mirror of
https://github.com/Unidata/python-awips.git
synced 2025-02-23 14:57:56 -05:00
181 lines
7.6 KiB
ReStructuredText
181 lines
7.6 KiB
ReStructuredText
===================
|
||
About Unidata AWIPS
|
||
===================
|
||
|
||
AWIPS is a weather forecasting display and analysis package
|
||
developed by the National Weather Service and Raytheon. AWIPS is a
|
||
Java application consisting of a data-rendering client (CAVE, which runs
|
||
on Red Hat/CentOS Linux and Mac OS X) and a backend data server (EDEX,
|
||
which runs only on Linux)
|
||
|
||
AWIPS takes a unified approach to data ingest, and most data types
|
||
follow a standard path through the system. At a high level, data flow
|
||
describes the path taken by a piece of data from its source to its
|
||
display by a client system. This path starts with data requested and
|
||
stored by an `LDM <#ldm>`_ client and includes the decoding of the data
|
||
and storing of decoded data in a form readable and displayable by the
|
||
end user.
|
||
|
||
The AWIPS ingest and request processes are a highly distributed
|
||
system, and the messaging broken `Qpid <#qpid>`_ is used for
|
||
inter-process communication.
|
||
|
||
.. figure:: http://www.unidata.ucar.edu/software/awips2/images/awips2_coms.png
|
||
:align: center
|
||
:alt: image
|
||
|
||
image
|
||
|
||
License
|
||
-------
|
||
|
||
The AWIPS software package released by the Unidata Program Center is considered to
|
||
be in the public domain since it is released without proprietary code. As such, export
|
||
controls do not apply. Any person is free to download, modify, distribute, or share
|
||
Unidata AWIPS in any form. Entities who modify or re-distribute Unidata AWIPS
|
||
software are encouraged to conduct their own FOSS/COTS entitlement/license review
|
||
to ensure that they remain compatible with the associated terms (see
|
||
FOSS_COTS_License.pdf at `https://github.com/Unidata/awips2 <https://github.com/Unidata/awips2>`_).
|
||
|
||
|
||
About AWIPS
|
||
-----------
|
||
|
||
The primary AWIPS application for data ingest, processing, and
|
||
storage is the Environmental Data EXchange (**EDEX**) server; the
|
||
primary AWIPS application for visualization/data manipulation is the
|
||
Common AWIPS Visualization Environment (**CAVE**) client, which is
|
||
typically installed on a workstation separate from other AWIPS
|
||
components.
|
||
|
||
In addition to programs developed specifically for AWIPS, AWIPS uses
|
||
several commercial off-the-shelf (COTS) and Free or Open Source software
|
||
(FOSS) products to assist in its operation. The following components,
|
||
working together and communicating, compose the entire AWIPS system.
|
||
|
||
EDEX
|
||
----
|
||
|
||
The main server for AWIPS. Qpid sends alerts to EDEX when data stored
|
||
by the LDM is ready for processing. These Qpid messages include file
|
||
header information which allows EDEX to determine the appropriate data
|
||
decoder to use. The default ingest server (simply named ingest) handles
|
||
all data ingest other than grib messages, which are processed by a
|
||
separate ingestGrib server. After decoding, EDEX writes metadata to the
|
||
database via Postgres and saves the processed data in HDF5 via PyPIES. A
|
||
third EDEX server, request, feeds requested data to CAVE clients. EDEX
|
||
ingest and request servers are started and stopped with the commands
|
||
``edex start`` and ``edex stop``, which runs the system script
|
||
``/etc/rc.d/init.d/edex_camel``
|
||
|
||
CAVE
|
||
----
|
||
|
||
Common AWIPS Visualization Environment. The data rendering and
|
||
visualization tool for AWIPS. CAVE contains of a number of different
|
||
data display configurations called perspectives. Perspectives used in
|
||
operational forecasting environments include **D2D** (Display
|
||
Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP**
|
||
(National Centers Perspective). CAVE is started with the command
|
||
``/awips2/cave/cave.sh`` or ``cave.sh``
|
||
|
||
.. figure:: http://www.unidata.ucar.edu/software/awips2/images/Unidata_AWIPS2_CAVE.png
|
||
:align: center
|
||
:alt: CAVE
|
||
|
||
CAVE
|
||
|
||
Alertviz
|
||
--------
|
||
|
||
**Alertviz** is a modernized version of an AWIPS I application, designed
|
||
to present various notifications, error messages, and alarms to the user
|
||
(forecaster). AlertViz can be executed either independently or from CAVE
|
||
itself. In the Unidata CAVE client, Alertviz is run within CAVE and is
|
||
not required to be run separately. The toolbar is also **hidden from
|
||
view** and is accessed by right-click on the desktop taskbar icon.
|
||
|
||
LDM
|
||
---
|
||
|
||
`http://www.unidata.ucar.edu/software/ldm/ <http://www.unidata.ucar.edu/software/ldm/>`_
|
||
|
||
The **LDM** (Local Data Manager), developed and supported by Unidata, is
|
||
a suite of client and server programs designed for data distribution,
|
||
and is the fundamental component comprising the Unidata Internet Data
|
||
Distribution (IDD) system. In AWIPS, the LDM provides data feeds for
|
||
grids, surface observations, upper-air profiles, satellite and radar
|
||
imagery and various other meteorological datasets. The LDM writes data
|
||
directly to file and alerts EDEX via Qpid when a file is available for
|
||
processing. The LDM is started and stopped with the commands
|
||
``edex start`` and ``edex stop``, which runs the commands
|
||
``service edex_ldm start`` and ``service edex_ldm stop``
|
||
|
||
edexBridge
|
||
----------
|
||
|
||
edexBridge, invoked in the LDM configuration file
|
||
``/awips2/ldm/etc/ldmd.conf``, is used by the LDM to post "data
|
||
available" messaged to Qpid, which alerts the EDEX Ingest server that a
|
||
file is ready for processing.
|
||
|
||
Qpid
|
||
----
|
||
|
||
`http://qpid.apache.org <http://qpid.apache.org>`_
|
||
|
||
**Apache Qpid**, the Queue Processor Interface Daemon, is the messaging
|
||
system used by AWIPS to facilitate communication between services.
|
||
When the LDM receives a data file to be processed, it employs
|
||
**edexBridge** to send EDEX ingest servers a message via Qpid. When EDEX
|
||
has finished decoding the file, it sends CAVE a message via Qpid that
|
||
data are available for display or further processing. Qpid is started
|
||
and stopped by ``edex start`` and ``edex stop``, and is controlled by
|
||
the system script ``/etc/rc.d/init.d/qpidd``
|
||
|
||
PostgreSQL
|
||
----------
|
||
|
||
`http://www.postgresql.org <http://www.postgresql.org>`_
|
||
|
||
**PostgreSQL**, known simply as Postgres, is a relational database
|
||
management system (DBMS) which handles the storage and retrieval of
|
||
metadata, database tables and some decoded data. The storage and reading
|
||
of EDEX metadata is handled by the Postgres DBMS. Users may query the
|
||
metadata tables by using the termainal-based front-end for Postgres
|
||
called **psql**. Postgres is started and stopped by ``edex start`` and
|
||
``edex stop``, and is controlled by the system script
|
||
``/etc/rc.d/init.d/edex_postgres``
|
||
|
||
HDF5
|
||
----
|
||
|
||
`http://www.hdfgroup.org/HDF5/ <http://www.hdfgroup.org/HDF5/>`_
|
||
|
||
**Hierarchical Data Format (v.5)** is
|
||
the primary data storage format used by AWIPS for processed grids,
|
||
satellite and radar imagery and other products. Similar to netCDF,
|
||
developed and supported by Unidata, HDF5 supports multiple types of data
|
||
within a single file. For example, a single HDF5 file of radar data may
|
||
contain multiple volume scans of base reflectivity and base velocity as
|
||
well as derived products such as composite reflectivity. The file may
|
||
also contain data from multiple radars. HDF5 is stored in
|
||
``/awips2/edex/data/hdf5/``
|
||
|
||
PyPIES (httpd-pypies)
|
||
---------------------
|
||
|
||
**PyPIES**, Python Process Isolated Enhanced Storage, was created for
|
||
AWIPS to isolate the management of HDF5 Processed Data Storage from
|
||
the EDEX processes. PyPIES manages access, i.e., reads and writes, of
|
||
data in the HDF5 files. In a sense, PyPIES provides functionality
|
||
similar to a DBMS (i.e PostgreSQL for metadata); all data being written
|
||
to an HDF5 file is sent to PyPIES, and requests for data stored in HDF5
|
||
are processed by PyPIES.
|
||
|
||
PyPIES is implemented in two parts: 1. The PyPIES manager is a Python
|
||
application that runs as part of an Apache HTTP server, and handles
|
||
requests to store and retrieve data. 2. The PyPIES logger is a Python
|
||
process that coordinates logging. PyPIES is started and stopped by
|
||
``edex start`` and ``edex stop``, and is controlled by the system script
|
||
``/etc/rc.d/init.d/https-pypies``
|