xan/python-awips
1
0
Fork 0
mirror of https://github.com/Unidata/python-awips.git synced 2025-02-23 14:57:56 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

merge docs from master to master-python2

Browse source
This commit is contained in:
mjames-upc 2017-07-07 10:46:00 -06:00
parent e8e27fe0b7
commit 4f39cdeecc
16 changed files with 211 additions and 80 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

117
docs/source/about.rst
View file

@ -1,16 +1,14 @@
=====
About AWIPS II
=====
===================
About Unidata AWIPS
===================
.. raw:: html
AWIPS II is a weather forecasting display and analysis package being
developed by the National Weather Service and Raytheon. AWIPS II is a
AWIPS is a weather forecasting display and analysis package being
developed by the National Weather Service and Raytheon. AWIPS is a
Java application consisting of a data-rendering client (CAVE, which runs
on Red Hat/CentOS Linux and Mac OS X) and a backend data server (EDEX,
which runs only on Linux)
AWIPS II takes a unified approach to data ingest, and most data types
AWIPS takes a unified approach to data ingest, and most data types
follow a standard path through the system. At a high level, data flow
describes the path taken by a piece of data from its source to its
display by a client system. This path starts with data requested and
@ -18,7 +16,7 @@ stored by an `LDM <#ldm>`_ client and includes the decoding of the data
and storing of decoded data in a form readable and displayable by the
end user.
The AWIPS II ingest and request processes are a highly distributed
The AWIPS ingest and request processes are a highly distributed
system, and the messaging broken `Qpid <#qpid>`_ is used for
inter-process communication.
@ -28,65 +26,76 @@ inter-process communication.
image
The primary AWIPS II application for data ingest, processing, and
License
-------
The AWIPS software package released by the Unidata Program Center is considered to
be in the public domain since it is released without proprietary code. As such, export
controls do not apply.  Any person is free to download, modify, distribute, or share
Unidata AWIPS in any form. Entities who modify or re-distribute Unidata AWIPS
software are encouraged to conduct their own FOSS/COTS entitlement/license review
to ensure that they remain compatible with the associated terms (see
FOSS_COTS_License.pdf at `https://github.com/Unidata/awips2 <https://github.com/Unidata/awips2>`_).
About AWIPS
-----------
The primary AWIPS application for data ingest, processing, and
storage is the Environmental Data EXchange (**EDEX**) server; the
primary AWIPS II application for visualization/data manipulation is the
primary AWIPS application for visualization/data manipulation is the
Common AWIPS Visualization Environment (**CAVE**) client, which is
typically installed on a workstation separate from other AWIPS II
typically installed on a workstation separate from other AWIPS
components.
In addition to programs developed specifically for AWIPS, AWIPS II uses
In addition to programs developed specifically for AWIPS, AWIPS uses
several commercial off-the-shelf (COTS) and Free or Open Source software
(FOSS) products to assist in its operation. The following components,
working together and communicating, compose the entire AWIPS II system.
working together and communicating, compose the entire AWIPS system.
AWIPS II Python Stack
---------------------
AWIPS Python Stack
------------------
A number of Python packages are bundled with the AWIPS II EDEX and CAVE
A number of Python packages are bundled with the AWIPS EDEX and CAVE
installations.
====================== ============== ==============================
Package Version RPM Name
====================== ============== ==============================
Python 2.7.10 awips2-python
**awips** **0.9.8** **awips2-python-awips**
basemap 1.0.7 awips2-python-basemap
cartopy 0.13.0 awips2-python-cartopy
cherrypy 3.1.2 awips2-python-cherrypy
Python 2.7.11 awips2-python
**awips** **0.9.10** **awips2-python-awips**
cartopy 0.15.1 awips2-python-cartopy
cherrypy 3.8.2 awips2-python-cherrypy
cycler 0.10.0 awips2-python-cycler
cython 0.23.4 awips2-python-cython
dateutil 2.5.0 awips2-python-dateutil
gfe 1.0.0 awips2-python-gfe
cython 0.25.2 awips2-python-cython
dateutil 2.4.2 awips2-python-dateutil
gfe 16.4.1 awips2-python-gfe
h5py 1.3.0 awips2-python-h5py
jep 3.4.1 awips2-python-jep
jep 3.5.3 awips2-python-jep
matplotlib 1.5.1 awips2-python-matplotlib
metpy 0.3.0 awips2-python-metpy
nose 0.11.1 awips2-python-nose
numpy 1.8.0 awips2-python-numpy
pil 1.1.6 awips2-python-pil
metpy 0.5.0 awips2-python-metpy
nose 1.3.7 awips2-python-nose
numpy 1.9.2 awips2-python-numpy
pint 0.7.2 awips2-python-pint
pmw 1.3.2 awips2-python-pmw
pupynere 1.0.13 awips2-python-pupynere
pycairo 1.2.2 awips2-python-pycairo
pygtk 2.8.6 awips2-python-pygtk
pyparsing 2.1.0 awips2-python-pyparsing
pupynere 1.0.15 awips2-python-pupynere
pyparsing 2.0.3 awips2-python-pyparsing
qpid 0.32 awips2-python-qpid
scientific 2.8 awips2-python-scientific
scipy 0.9.0 awips2-python-scipy
shapely 1.5.9 awips2-python-shapely
scipy 0.15.1 awips2-python-scipy
shapely 1.4.4 awips2-python-shapely
six 1.10.0 awips2-python-six
tables 2.1.2 awips2-python-tables
tpg 3.1.2 awips2-python-tpg
werkzeug 3.1.2 awips2-python-werkzeug
tables 3.3.0 awips2-python-tables
tpg 3.2.2 awips2-python-tpg
werkzeug 0.12.1 awips2-python-werkzeug
====================== ============== ==============================
EDEX
-------------------
----
The main server for AWIPS II. Qpid sends alerts to EDEX when data stored
The main server for AWIPS. Qpid sends alerts to EDEX when data stored
by the LDM is ready for processing. These Qpid messages include file
header information which allows EDEX to determine the appropriate data
decoder to use. The default ingest server (simply named ingest) handles
@ -99,10 +108,10 @@ ingest and request servers are started and stopped with the commands
``/etc/rc.d/init.d/edex_camel``
CAVE
-------------------
----
Common AWIPS Visualization Environment. The data rendering and
visualization tool for AWIPS II. CAVE contains of a number of different
visualization tool for AWIPS. CAVE contains of a number of different
data display configurations called perspectives. Perspectives used in
operational forecasting environments include **D2D** (Display
Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP**
@ -116,7 +125,7 @@ Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP**
CAVE
Alertviz
-------------------
--------
**Alertviz** is a modernized version of an AWIPS I application, designed
to present various notifications, error messages, and alarms to the user
@ -126,14 +135,14 @@ not required to be run separately. The toolbar is also **hidden from
view** and is accessed by right-click on the desktop taskbar icon.
LDM
-------------------
---
`http://www.unidata.ucar.edu/software/ldm/ <http://www.unidata.ucar.edu/software/ldm/>`_
The **LDM** (Local Data Manager), developed and supported by Unidata, is
a suite of client and server programs designed for data distribution,
and is the fundamental component comprising the Unidata Internet Data
Distribution (IDD) system. In AWIPS II, the LDM provides data feeds for
Distribution (IDD) system. In AWIPS, the LDM provides data feeds for
grids, surface observations, upper-air profiles, satellite and radar
imagery and various other meteorological datasets. The LDM writes data
directly to file and alerts EDEX via Qpid when a file is available for
@ -142,7 +151,7 @@ processing. The LDM is started and stopped with the commands
``service edex_ldm start`` and ``service edex_ldm stop``
edexBridge
-------------------
----------
edexBridge, invoked in the LDM configuration file
``/awips2/ldm/etc/ldmd.conf``, is used by the LDM to post "data
@ -150,12 +159,12 @@ available" messaged to Qpid, which alerts the EDEX Ingest server that a
file is ready for processing.
Qpid
-------------------
----
`http://qpid.apache.org <http://qpid.apache.org>`_
**Apache Qpid**, the Queue Processor Interface Daemon, is the messaging
system used by AWIPS II to facilitate communication between services.
system used by AWIPS to facilitate communication between services.
When the LDM receives a data file to be processed, it employs
**edexBridge** to send EDEX ingest servers a message via Qpid. When EDEX
has finished decoding the file, it sends CAVE a message via Qpid that
@ -164,7 +173,7 @@ and stopped by ``edex start`` and ``edex stop``, and is controlled by
the system script ``/etc/rc.d/init.d/qpidd``
PostgreSQL
-------------------
----------
`http://www.postgresql.org <http://www.postgresql.org>`_
@ -178,12 +187,12 @@ called **psql**. Postgres is started and stopped by ``edex start`` and
``/etc/rc.d/init.d/edex_postgres``
HDF5
-------------------
----
`http://www.hdfgroup.org/HDF5/ <http://www.hdfgroup.org/HDF5/>`_
**Hierarchical Data Format (v.5)** is
the primary data storage format used by AWIPS II for processed grids,
the primary data storage format used by AWIPS for processed grids,
satellite and radar imagery and other products. Similar to netCDF,
developed and supported by Unidata, HDF5 supports multiple types of data
within a single file. For example, a single HDF5 file of radar data may
@ -193,10 +202,10 @@ also contain data from multiple radars. HDF5 is stored in
``/awips2/edex/data/hdf5/``
PyPIES (httpd-pypies)
-------------------
---------------------
**PyPIES**, Python Process Isolated Enhanced Storage, was created for
AWIPS II to isolate the management of HDF5 Processed Data Storage from
AWIPS to isolate the management of HDF5 Processed Data Storage from
the EDEX processes. PyPIES manages access, i.e., reads and writes, of
data in the HDF5 files. In a sense, PyPIES provides functionality
similar to a DBMS (i.e PostgreSQL for metadata); all data being written

7
docs/source/api/DataAccessLayer.rst Normal file
View file

@ -0,0 +1,7 @@
===============
DataAccessLayer
===============
.. automodule:: awips.dataaccess.DataAccessLayer
:members:
:undoc-members:

7
docs/source/api/DateTimeConverter.rst Normal file
View file

@ -0,0 +1,7 @@
=================
DateTimeConverter
=================
.. automodule:: awips.DateTimeConverter
:members:
:undoc-members:

7
docs/source/api/IDataRequest.rst Normal file
View file

@ -0,0 +1,7 @@
===============================
IDataRequest (newDataRequest())
===============================
.. autoclass:: awips.dataaccess.IDataRequest
:members:
:special-members:

7
docs/source/api/PyData.rst Normal file
View file

@ -0,0 +1,7 @@
======================
PyData
======================
.. automodule:: awips.dataaccess.PyData
:members:
:undoc-members:

7
docs/source/api/PyGeometryData.rst Normal file
View file

@ -0,0 +1,7 @@
======================
PyGeometryData
======================
.. automodule:: awips.dataaccess.PyGeometryData
:members:
:undoc-members:

7
docs/source/api/PyGridData.rst Normal file
View file

@ -0,0 +1,7 @@
======================
PyGridData
======================
.. automodule:: awips.dataaccess.PyGridData
:members:
:undoc-members:

7
docs/source/api/RadarCommon.rst Normal file
View file

@ -0,0 +1,7 @@
======================
RadarCommon
======================
.. automodule:: awips.RadarCommon
:members:
:undoc-members:

7
docs/source/api/SoundingsSupport.rst Normal file
View file

@ -0,0 +1,7 @@
======================
SoundingsSupport
======================
.. automodule:: awips.dataaccess.SoundingsSupport
:members:
:undoc-members:

7
docs/source/api/ThriftClientRouter.rst Normal file
View file

@ -0,0 +1,7 @@
======================
ThriftClientRouter
======================
.. automodule:: awips.dataaccess.ThriftClientRouter
:members:
:undoc-members:

18
docs/source/api/index.rst Normal file
View file

@ -0,0 +1,18 @@
#################
API Documentation
#################
.. toctree::
:maxdepth: 2
DataAccessLayer
IDataRequest
PyData
PyGridData
PyGeometryData
SoundingsSupport
ThriftClientRouter
RadarCommon
DateTimeConverter
* :ref:`genindex`

16
docs/source/conf.py
View file

@ -56,16 +56,16 @@ source_suffix = '.rst'
master_doc = 'index'
# General information about the project.
project = u'python-awips'
copyright = u'2016, Unidata'
author = u'Unidata'
project = 'python-awips'
copyright = '2016, Unidata'
author = 'Unidata'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'0.9.8'
version = '0.9.10'
# The full version, including alpha/beta/rc tags.
# The language for content autogenerated by Sphinx. Refer to documentation
@ -231,8 +231,8 @@ latex_elements = {
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'python-awips.tex', u'python-awips Documentation',
u'Unidata', 'manual'),
(master_doc, 'python-awips.tex', 'python-awips Documentation',
'Unidata', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
@ -261,7 +261,7 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'python-awips', u'python-awips Documentation',
(master_doc, 'python-awips', 'python-awips Documentation',
[author], 1)
]
@ -275,7 +275,7 @@ man_pages = [
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'python-awips', u'python-awips Documentation',
(master_doc, 'python-awips', 'python-awips Documentation',
author, 'python-awips', 'One line description of project.',
'Miscellaneous'),
]

6
docs/source/dev.rst
View file

@ -1,6 +1,6 @@
Dev Guide
=========
Development Guide
=================
The Data Access Framework allows developers to retrieve different types
of data without having dependencies on those types of data. It provides
@ -266,7 +266,7 @@ contributing the necessary code for the framework to support its data.
For example, the satellite plugin provides a factory class for
interacting with the framework and registers itself as being compatible
with the Data Access Framework. This concept is similar to how EDEX in
AWIPS II expects a plugin developer to provide a decoder class and
AWIPS expects a plugin developer to provide a decoder class and
record class and register them, but then automatically manages the rest
of the ingest process including routing, storing, and alerting on new
data. This style of plugin architecture effectively enables the

4
docs/source/examples/index.rst
View file

@ -1,8 +1,8 @@
.. _examples-index:
################
######################
Data Plotting Examples
################
######################
.. toctree::
:maxdepth: 1

43
docs/source/index.rst
View file

@ -2,9 +2,45 @@
Python AWIPS Data Access Framework
==================================
The `python-awips <https://github.com/Unidata/python-awips>`_ package provides a Data Access Framework (DAF) for requesting data from a remote AWIPS II EDEX server.
`AWIPS <http://unidata.github.io/awips2>`_ is a weather display and analysis package developed by the National Weather Service for operational forecasting. UCAR's `Unidata Program Center <http://www.unidata.ucar.edu/software/awips2/>`_ supports a non-operational open-source release of the AWIPS software (`EDEX <http://unidata.github.io/awips2/#edex>`_, `CAVE <http://unidata.github.io/awips2/#cave>`_, and `python-awips <https://github.com/Unidata/python-awips>`_).
The `AWIPS II Python Stack <about.html#awips-ii-python-stack>`_ installed via RPM contains the DAF, matplotlib, numpy, scipy, basemap, pint, shapely, and other packages.
The python-awips package provides a data access framework for requesting grid and geometry datasets from an `EDEX <http://unidata.github.io/awips2>`_ server.
.. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks
Install
-------
- pip install python-awips
Requirements
~~~~~~~~~~~~
- Python 2.7+
- Shapely 1.4+
- MetPy and enum34 to run the `Jupyter Notebook`_ examples
How to Use
~~~~~~~~~~
::
from awips.dataaccess import DataAccessLayer
DataAccessLayer.changeEDEXHost("edex-cloud.unidata.ucar.edu")
request = DataAccessLayer.newDataRequest()
dataTypes = DataAccessLayer.getSupportedDatatypes()
request.setDatatype("grid")
request.addLocationNames("RAP40")
request.setParameters("T")
request.setLevels("0.0SFC")
cycles = DataAccessLayer.getAvailableTimes(request, True)
times = DataAccessLayer.getAvailableTimes(request)
response = DataAccessLayer.getGridData(request, times[-1])
for grid in response:
data = grid.getRawData()
lons, lats = grid.getLatLonCoords()
See the `API Documentation <api/DataAccessLayer.html>`_ for more information.
-------------
Documentation
@ -14,7 +50,8 @@ Documentation
:maxdepth: 2
install
api/index
examples/index
about
dev
gridparms
about

22
docs/source/install.rst
View file

@ -1,6 +1,6 @@
.. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks
Install python-awips
Installation
------------------
- pip install python-awips
@ -15,19 +15,23 @@ Requirements
Install from Github
~~~~~~~~~~~~~~~~~~~~~~~~~~~
- git clone https://github.com/Unidata/python-awips.git
- cd python-awips
- git clone https://github.com/Unidata/python-awips.git && cd python-awips
- python setup.py install
Install for AWIPS (/awips2/python)
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Install for AWIPS
~~~~~~~~~~~~~~~~~
AWIPS II >=15.1.3 (March 2016) has `python-awips` bundled with the awips2-server and awips2-cave groups, as well as a full meteorological data stack (metpy, matplotlib, numpy, etc).
On standalone AWIPS systems, the full `AWIPS Python Stack <about.html#awips-ii-python-stack>`_ is installed to ``/awips2/python`` as RPM packages.
Easy install on an AWIPS system
- wget https://bootstrap.pypa.io/ez_setup.py -O - | /awips2/python/bin/python
- /awips2/python/bin/easy_install pip
- /awips2/python/bin/pip install python-awips
* For Unidata AWIPS release **16.2.2+**:
* /awips2/python/bin/easy_install pip
* /awips2/python/bin/pip install python-awips
* For releases before and up to **16.1.5** you may need to run
* yum install awips2-python-setuptools