merge docs from master to master-python2

This commit is contained in:
mjames-upc 2017-07-07 10:46:00 -06:00
parent e8e27fe0b7
commit 4f39cdeecc
16 changed files with 211 additions and 80 deletions

View file

@ -1,16 +1,14 @@
===== ===================
About AWIPS II About Unidata AWIPS
===== ===================
.. raw:: html AWIPS is a weather forecasting display and analysis package being
developed by the National Weather Service and Raytheon. AWIPS is a
AWIPS II is a weather forecasting display and analysis package being
developed by the National Weather Service and Raytheon. AWIPS II is a
Java application consisting of a data-rendering client (CAVE, which runs Java application consisting of a data-rendering client (CAVE, which runs
on Red Hat/CentOS Linux and Mac OS X) and a backend data server (EDEX, on Red Hat/CentOS Linux and Mac OS X) and a backend data server (EDEX,
which runs only on Linux) which runs only on Linux)
AWIPS II takes a unified approach to data ingest, and most data types AWIPS takes a unified approach to data ingest, and most data types
follow a standard path through the system. At a high level, data flow follow a standard path through the system. At a high level, data flow
describes the path taken by a piece of data from its source to its describes the path taken by a piece of data from its source to its
display by a client system. This path starts with data requested and display by a client system. This path starts with data requested and
@ -18,7 +16,7 @@ stored by an `LDM <#ldm>`_ client and includes the decoding of the data
and storing of decoded data in a form readable and displayable by the and storing of decoded data in a form readable and displayable by the
end user. end user.
The AWIPS II ingest and request processes are a highly distributed The AWIPS ingest and request processes are a highly distributed
system, and the messaging broken `Qpid <#qpid>`_ is used for system, and the messaging broken `Qpid <#qpid>`_ is used for
inter-process communication. inter-process communication.
@ -28,65 +26,76 @@ inter-process communication.
image image
The primary AWIPS II application for data ingest, processing, and License
-------
The AWIPS software package released by the Unidata Program Center is considered to
be in the public domain since it is released without proprietary code. As such, export
controls do not apply.  Any person is free to download, modify, distribute, or share
Unidata AWIPS in any form. Entities who modify or re-distribute Unidata AWIPS
software are encouraged to conduct their own FOSS/COTS entitlement/license review
to ensure that they remain compatible with the associated terms (see
FOSS_COTS_License.pdf at `https://github.com/Unidata/awips2 <https://github.com/Unidata/awips2>`_).
About AWIPS
-----------
The primary AWIPS application for data ingest, processing, and
storage is the Environmental Data EXchange (**EDEX**) server; the storage is the Environmental Data EXchange (**EDEX**) server; the
primary AWIPS II application for visualization/data manipulation is the primary AWIPS application for visualization/data manipulation is the
Common AWIPS Visualization Environment (**CAVE**) client, which is Common AWIPS Visualization Environment (**CAVE**) client, which is
typically installed on a workstation separate from other AWIPS II typically installed on a workstation separate from other AWIPS
components. components.
In addition to programs developed specifically for AWIPS, AWIPS II uses In addition to programs developed specifically for AWIPS, AWIPS uses
several commercial off-the-shelf (COTS) and Free or Open Source software several commercial off-the-shelf (COTS) and Free or Open Source software
(FOSS) products to assist in its operation. The following components, (FOSS) products to assist in its operation. The following components,
working together and communicating, compose the entire AWIPS II system. working together and communicating, compose the entire AWIPS system.
AWIPS II Python Stack AWIPS Python Stack
--------------------- ------------------
A number of Python packages are bundled with the AWIPS II EDEX and CAVE A number of Python packages are bundled with the AWIPS EDEX and CAVE
installations. installations.
====================== ============== ============================== ====================== ============== ==============================
Package Version RPM Name Package Version RPM Name
====================== ============== ============================== ====================== ============== ==============================
Python 2.7.10 awips2-python Python 2.7.11 awips2-python
**awips** **0.9.8** **awips2-python-awips** **awips** **0.9.10** **awips2-python-awips**
basemap 1.0.7 awips2-python-basemap cartopy 0.15.1 awips2-python-cartopy
cartopy 0.13.0 awips2-python-cartopy cherrypy 3.8.2 awips2-python-cherrypy
cherrypy 3.1.2 awips2-python-cherrypy
cycler 0.10.0 awips2-python-cycler cycler 0.10.0 awips2-python-cycler
cython 0.23.4 awips2-python-cython cython 0.25.2 awips2-python-cython
dateutil 2.5.0 awips2-python-dateutil dateutil 2.4.2 awips2-python-dateutil
gfe 1.0.0 awips2-python-gfe gfe 16.4.1 awips2-python-gfe
h5py 1.3.0 awips2-python-h5py h5py 1.3.0 awips2-python-h5py
jep 3.4.1 awips2-python-jep jep 3.5.3 awips2-python-jep
matplotlib 1.5.1 awips2-python-matplotlib matplotlib 1.5.1 awips2-python-matplotlib
metpy 0.3.0 awips2-python-metpy metpy 0.5.0 awips2-python-metpy
nose 0.11.1 awips2-python-nose nose 1.3.7 awips2-python-nose
numpy 1.8.0 awips2-python-numpy numpy 1.9.2 awips2-python-numpy
pil 1.1.6 awips2-python-pil
pint 0.7.2 awips2-python-pint pint 0.7.2 awips2-python-pint
pmw 1.3.2 awips2-python-pmw pmw 1.3.2 awips2-python-pmw
pupynere 1.0.13 awips2-python-pupynere pupynere 1.0.15 awips2-python-pupynere
pycairo 1.2.2 awips2-python-pycairo pyparsing 2.0.3 awips2-python-pyparsing
pygtk 2.8.6 awips2-python-pygtk
pyparsing 2.1.0 awips2-python-pyparsing
qpid 0.32 awips2-python-qpid qpid 0.32 awips2-python-qpid
scientific 2.8 awips2-python-scientific scientific 2.8 awips2-python-scientific
scipy 0.9.0 awips2-python-scipy scipy 0.15.1 awips2-python-scipy
shapely 1.5.9 awips2-python-shapely shapely 1.4.4 awips2-python-shapely
six 1.10.0 awips2-python-six six 1.10.0 awips2-python-six
tables 2.1.2 awips2-python-tables tables 3.3.0 awips2-python-tables
tpg 3.1.2 awips2-python-tpg tpg 3.2.2 awips2-python-tpg
werkzeug 3.1.2 awips2-python-werkzeug werkzeug 0.12.1 awips2-python-werkzeug
====================== ============== ============================== ====================== ============== ==============================
EDEX EDEX
------------------- ----
The main server for AWIPS II. Qpid sends alerts to EDEX when data stored The main server for AWIPS. Qpid sends alerts to EDEX when data stored
by the LDM is ready for processing. These Qpid messages include file by the LDM is ready for processing. These Qpid messages include file
header information which allows EDEX to determine the appropriate data header information which allows EDEX to determine the appropriate data
decoder to use. The default ingest server (simply named ingest) handles decoder to use. The default ingest server (simply named ingest) handles
@ -99,10 +108,10 @@ ingest and request servers are started and stopped with the commands
``/etc/rc.d/init.d/edex_camel`` ``/etc/rc.d/init.d/edex_camel``
CAVE CAVE
------------------- ----
Common AWIPS Visualization Environment. The data rendering and Common AWIPS Visualization Environment. The data rendering and
visualization tool for AWIPS II. CAVE contains of a number of different visualization tool for AWIPS. CAVE contains of a number of different
data display configurations called perspectives. Perspectives used in data display configurations called perspectives. Perspectives used in
operational forecasting environments include **D2D** (Display operational forecasting environments include **D2D** (Display
Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP** Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP**
@ -116,7 +125,7 @@ Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP**
CAVE CAVE
Alertviz Alertviz
------------------- --------
**Alertviz** is a modernized version of an AWIPS I application, designed **Alertviz** is a modernized version of an AWIPS I application, designed
to present various notifications, error messages, and alarms to the user to present various notifications, error messages, and alarms to the user
@ -126,14 +135,14 @@ not required to be run separately. The toolbar is also **hidden from
view** and is accessed by right-click on the desktop taskbar icon. view** and is accessed by right-click on the desktop taskbar icon.
LDM LDM
------------------- ---
`http://www.unidata.ucar.edu/software/ldm/ <http://www.unidata.ucar.edu/software/ldm/>`_ `http://www.unidata.ucar.edu/software/ldm/ <http://www.unidata.ucar.edu/software/ldm/>`_
The **LDM** (Local Data Manager), developed and supported by Unidata, is The **LDM** (Local Data Manager), developed and supported by Unidata, is
a suite of client and server programs designed for data distribution, a suite of client and server programs designed for data distribution,
and is the fundamental component comprising the Unidata Internet Data and is the fundamental component comprising the Unidata Internet Data
Distribution (IDD) system. In AWIPS II, the LDM provides data feeds for Distribution (IDD) system. In AWIPS, the LDM provides data feeds for
grids, surface observations, upper-air profiles, satellite and radar grids, surface observations, upper-air profiles, satellite and radar
imagery and various other meteorological datasets. The LDM writes data imagery and various other meteorological datasets. The LDM writes data
directly to file and alerts EDEX via Qpid when a file is available for directly to file and alerts EDEX via Qpid when a file is available for
@ -142,7 +151,7 @@ processing. The LDM is started and stopped with the commands
``service edex_ldm start`` and ``service edex_ldm stop`` ``service edex_ldm start`` and ``service edex_ldm stop``
edexBridge edexBridge
------------------- ----------
edexBridge, invoked in the LDM configuration file edexBridge, invoked in the LDM configuration file
``/awips2/ldm/etc/ldmd.conf``, is used by the LDM to post "data ``/awips2/ldm/etc/ldmd.conf``, is used by the LDM to post "data
@ -150,12 +159,12 @@ available" messaged to Qpid, which alerts the EDEX Ingest server that a
file is ready for processing. file is ready for processing.
Qpid Qpid
------------------- ----
`http://qpid.apache.org <http://qpid.apache.org>`_ `http://qpid.apache.org <http://qpid.apache.org>`_
**Apache Qpid**, the Queue Processor Interface Daemon, is the messaging **Apache Qpid**, the Queue Processor Interface Daemon, is the messaging
system used by AWIPS II to facilitate communication between services. system used by AWIPS to facilitate communication between services.
When the LDM receives a data file to be processed, it employs When the LDM receives a data file to be processed, it employs
**edexBridge** to send EDEX ingest servers a message via Qpid. When EDEX **edexBridge** to send EDEX ingest servers a message via Qpid. When EDEX
has finished decoding the file, it sends CAVE a message via Qpid that has finished decoding the file, it sends CAVE a message via Qpid that
@ -164,7 +173,7 @@ and stopped by ``edex start`` and ``edex stop``, and is controlled by
the system script ``/etc/rc.d/init.d/qpidd`` the system script ``/etc/rc.d/init.d/qpidd``
PostgreSQL PostgreSQL
------------------- ----------
`http://www.postgresql.org <http://www.postgresql.org>`_ `http://www.postgresql.org <http://www.postgresql.org>`_
@ -178,12 +187,12 @@ called **psql**. Postgres is started and stopped by ``edex start`` and
``/etc/rc.d/init.d/edex_postgres`` ``/etc/rc.d/init.d/edex_postgres``
HDF5 HDF5
------------------- ----
`http://www.hdfgroup.org/HDF5/ <http://www.hdfgroup.org/HDF5/>`_ `http://www.hdfgroup.org/HDF5/ <http://www.hdfgroup.org/HDF5/>`_
**Hierarchical Data Format (v.5)** is **Hierarchical Data Format (v.5)** is
the primary data storage format used by AWIPS II for processed grids, the primary data storage format used by AWIPS for processed grids,
satellite and radar imagery and other products. Similar to netCDF, satellite and radar imagery and other products. Similar to netCDF,
developed and supported by Unidata, HDF5 supports multiple types of data developed and supported by Unidata, HDF5 supports multiple types of data
within a single file. For example, a single HDF5 file of radar data may within a single file. For example, a single HDF5 file of radar data may
@ -193,10 +202,10 @@ also contain data from multiple radars. HDF5 is stored in
``/awips2/edex/data/hdf5/`` ``/awips2/edex/data/hdf5/``
PyPIES (httpd-pypies) PyPIES (httpd-pypies)
------------------- ---------------------
**PyPIES**, Python Process Isolated Enhanced Storage, was created for **PyPIES**, Python Process Isolated Enhanced Storage, was created for
AWIPS II to isolate the management of HDF5 Processed Data Storage from AWIPS to isolate the management of HDF5 Processed Data Storage from
the EDEX processes. PyPIES manages access, i.e., reads and writes, of the EDEX processes. PyPIES manages access, i.e., reads and writes, of
data in the HDF5 files. In a sense, PyPIES provides functionality data in the HDF5 files. In a sense, PyPIES provides functionality
similar to a DBMS (i.e PostgreSQL for metadata); all data being written similar to a DBMS (i.e PostgreSQL for metadata); all data being written

View file

@ -0,0 +1,7 @@
===============
DataAccessLayer
===============
.. automodule:: awips.dataaccess.DataAccessLayer
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
=================
DateTimeConverter
=================
.. automodule:: awips.DateTimeConverter
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
===============================
IDataRequest (newDataRequest())
===============================
.. autoclass:: awips.dataaccess.IDataRequest
:members:
:special-members:

View file

@ -0,0 +1,7 @@
======================
PyData
======================
.. automodule:: awips.dataaccess.PyData
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
======================
PyGeometryData
======================
.. automodule:: awips.dataaccess.PyGeometryData
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
======================
PyGridData
======================
.. automodule:: awips.dataaccess.PyGridData
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
======================
RadarCommon
======================
.. automodule:: awips.RadarCommon
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
======================
SoundingsSupport
======================
.. automodule:: awips.dataaccess.SoundingsSupport
:members:
:undoc-members:

View file

@ -0,0 +1,7 @@
======================
ThriftClientRouter
======================
.. automodule:: awips.dataaccess.ThriftClientRouter
:members:
:undoc-members:

18
docs/source/api/index.rst Normal file
View file

@ -0,0 +1,18 @@
#################
API Documentation
#################
.. toctree::
:maxdepth: 2
DataAccessLayer
IDataRequest
PyData
PyGridData
PyGeometryData
SoundingsSupport
ThriftClientRouter
RadarCommon
DateTimeConverter
* :ref:`genindex`

View file

@ -56,16 +56,16 @@ source_suffix = '.rst'
master_doc = 'index' master_doc = 'index'
# General information about the project. # General information about the project.
project = u'python-awips' project = 'python-awips'
copyright = u'2016, Unidata' copyright = '2016, Unidata'
author = u'Unidata' author = 'Unidata'
# The version info for the project you're documenting, acts as replacement for # The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the # |version| and |release|, also used in various other places throughout the
# built documents. # built documents.
# #
# The short X.Y version. # The short X.Y version.
version = u'0.9.8' version = '0.9.10'
# The full version, including alpha/beta/rc tags. # The full version, including alpha/beta/rc tags.
# The language for content autogenerated by Sphinx. Refer to documentation # The language for content autogenerated by Sphinx. Refer to documentation
@ -231,8 +231,8 @@ latex_elements = {
# (source start file, target name, title, # (source start file, target name, title,
# author, documentclass [howto, manual, or own class]). # author, documentclass [howto, manual, or own class]).
latex_documents = [ latex_documents = [
(master_doc, 'python-awips.tex', u'python-awips Documentation', (master_doc, 'python-awips.tex', 'python-awips Documentation',
u'Unidata', 'manual'), 'Unidata', 'manual'),
] ]
# The name of an image file (relative to this directory) to place at the top of # The name of an image file (relative to this directory) to place at the top of
@ -261,7 +261,7 @@ latex_documents = [
# One entry per manual page. List of tuples # One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section). # (source start file, name, description, authors, manual section).
man_pages = [ man_pages = [
(master_doc, 'python-awips', u'python-awips Documentation', (master_doc, 'python-awips', 'python-awips Documentation',
[author], 1) [author], 1)
] ]
@ -275,7 +275,7 @@ man_pages = [
# (source start file, target name, title, author, # (source start file, target name, title, author,
# dir menu entry, description, category) # dir menu entry, description, category)
texinfo_documents = [ texinfo_documents = [
(master_doc, 'python-awips', u'python-awips Documentation', (master_doc, 'python-awips', 'python-awips Documentation',
author, 'python-awips', 'One line description of project.', author, 'python-awips', 'One line description of project.',
'Miscellaneous'), 'Miscellaneous'),
] ]

View file

@ -1,6 +1,6 @@
Dev Guide Development Guide
========= =================
The Data Access Framework allows developers to retrieve different types The Data Access Framework allows developers to retrieve different types
of data without having dependencies on those types of data. It provides of data without having dependencies on those types of data. It provides
@ -266,7 +266,7 @@ contributing the necessary code for the framework to support its data.
For example, the satellite plugin provides a factory class for For example, the satellite plugin provides a factory class for
interacting with the framework and registers itself as being compatible interacting with the framework and registers itself as being compatible
with the Data Access Framework. This concept is similar to how EDEX in with the Data Access Framework. This concept is similar to how EDEX in
AWIPS II expects a plugin developer to provide a decoder class and AWIPS expects a plugin developer to provide a decoder class and
record class and register them, but then automatically manages the rest record class and register them, but then automatically manages the rest
of the ingest process including routing, storing, and alerting on new of the ingest process including routing, storing, and alerting on new
data. This style of plugin architecture effectively enables the data. This style of plugin architecture effectively enables the

View file

@ -1,8 +1,8 @@
.. _examples-index: .. _examples-index:
################ ######################
Data Plotting Examples Data Plotting Examples
################ ######################
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1

View file

@ -2,9 +2,45 @@
Python AWIPS Data Access Framework Python AWIPS Data Access Framework
================================== ==================================
The `python-awips <https://github.com/Unidata/python-awips>`_ package provides a Data Access Framework (DAF) for requesting data from a remote AWIPS II EDEX server. `AWIPS <http://unidata.github.io/awips2>`_ is a weather display and analysis package developed by the National Weather Service for operational forecasting. UCAR's `Unidata Program Center <http://www.unidata.ucar.edu/software/awips2/>`_ supports a non-operational open-source release of the AWIPS software (`EDEX <http://unidata.github.io/awips2/#edex>`_, `CAVE <http://unidata.github.io/awips2/#cave>`_, and `python-awips <https://github.com/Unidata/python-awips>`_).
The `AWIPS II Python Stack <about.html#awips-ii-python-stack>`_ installed via RPM contains the DAF, matplotlib, numpy, scipy, basemap, pint, shapely, and other packages. The python-awips package provides a data access framework for requesting grid and geometry datasets from an `EDEX <http://unidata.github.io/awips2>`_ server.
.. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks
Install
-------
- pip install python-awips
Requirements
~~~~~~~~~~~~
- Python 2.7+
- Shapely 1.4+
- MetPy and enum34 to run the `Jupyter Notebook`_ examples
How to Use
~~~~~~~~~~
::
from awips.dataaccess import DataAccessLayer
DataAccessLayer.changeEDEXHost("edex-cloud.unidata.ucar.edu")
request = DataAccessLayer.newDataRequest()
dataTypes = DataAccessLayer.getSupportedDatatypes()
request.setDatatype("grid")
request.addLocationNames("RAP40")
request.setParameters("T")
request.setLevels("0.0SFC")
cycles = DataAccessLayer.getAvailableTimes(request, True)
times = DataAccessLayer.getAvailableTimes(request)
response = DataAccessLayer.getGridData(request, times[-1])
for grid in response:
data = grid.getRawData()
lons, lats = grid.getLatLonCoords()
See the `API Documentation <api/DataAccessLayer.html>`_ for more information.
------------- -------------
Documentation Documentation
@ -14,7 +50,8 @@ Documentation
:maxdepth: 2 :maxdepth: 2
install install
api/index
examples/index examples/index
about
dev dev
gridparms gridparms
about

View file

@ -1,6 +1,6 @@
.. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks .. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks
Install python-awips Installation
------------------ ------------------
- pip install python-awips - pip install python-awips
@ -15,19 +15,23 @@ Requirements
Install from Github Install from Github
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
- git clone https://github.com/Unidata/python-awips.git - git clone https://github.com/Unidata/python-awips.git && cd python-awips
- cd python-awips
- python setup.py install - python setup.py install
Install for AWIPS (/awips2/python) Install for AWIPS
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
AWIPS II >=15.1.3 (March 2016) has `python-awips` bundled with the awips2-server and awips2-cave groups, as well as a full meteorological data stack (metpy, matplotlib, numpy, etc). On standalone AWIPS systems, the full `AWIPS Python Stack <about.html#awips-ii-python-stack>`_ is installed to ``/awips2/python`` as RPM packages.
Easy install on an AWIPS system Easy install on an AWIPS system
- wget https://bootstrap.pypa.io/ez_setup.py -O - | /awips2/python/bin/python * For Unidata AWIPS release **16.2.2+**:
- /awips2/python/bin/easy_install pip
- /awips2/python/bin/pip install python-awips * /awips2/python/bin/easy_install pip
* /awips2/python/bin/pip install python-awips
* For releases before and up to **16.1.5** you may need to run
* yum install awips2-python-setuptools