diff --git a/docs/source/about.rst b/docs/source/about.rst index 20ae52b..9c5f9f3 100644 --- a/docs/source/about.rst +++ b/docs/source/about.rst @@ -1,16 +1,14 @@ -===== -About AWIPS II -===== +=================== +About Unidata AWIPS +=================== -.. raw:: html - -AWIPS II is a weather forecasting display and analysis package being -developed by the National Weather Service and Raytheon. AWIPS II is a +AWIPS is a weather forecasting display and analysis package being +developed by the National Weather Service and Raytheon. AWIPS is a Java application consisting of a data-rendering client (CAVE, which runs on Red Hat/CentOS Linux and Mac OS X) and a backend data server (EDEX, which runs only on Linux) -AWIPS II takes a unified approach to data ingest, and most data types +AWIPS takes a unified approach to data ingest, and most data types follow a standard path through the system. At a high level, data flow describes the path taken by a piece of data from its source to its display by a client system. This path starts with data requested and @@ -18,7 +16,7 @@ stored by an `LDM <#ldm>`_ client and includes the decoding of the data and storing of decoded data in a form readable and displayable by the end user. -The AWIPS II ingest and request processes are a highly distributed +The AWIPS ingest and request processes are a highly distributed system, and the messaging broken `Qpid <#qpid>`_ is used for inter-process communication. @@ -28,65 +26,76 @@ inter-process communication. image -The primary AWIPS II application for data ingest, processing, and +License +------- + +The AWIPS software package released by the Unidata Program Center is considered to +be in the public domain since it is released without proprietary code. As such, export +controls do not apply.  Any person is free to download, modify, distribute, or share +Unidata AWIPS in any form. Entities who modify or re-distribute Unidata AWIPS +software are encouraged to conduct their own FOSS/COTS entitlement/license review +to ensure that they remain compatible with the associated terms (see +FOSS_COTS_License.pdf at `https://github.com/Unidata/awips2 `_). + + +About AWIPS +----------- + +The primary AWIPS application for data ingest, processing, and storage is the Environmental Data EXchange (**EDEX**) server; the -primary AWIPS II application for visualization/data manipulation is the +primary AWIPS application for visualization/data manipulation is the Common AWIPS Visualization Environment (**CAVE**) client, which is -typically installed on a workstation separate from other AWIPS II +typically installed on a workstation separate from other AWIPS components. -In addition to programs developed specifically for AWIPS, AWIPS II uses +In addition to programs developed specifically for AWIPS, AWIPS uses several commercial off-the-shelf (COTS) and Free or Open Source software (FOSS) products to assist in its operation. The following components, -working together and communicating, compose the entire AWIPS II system. +working together and communicating, compose the entire AWIPS system. -AWIPS II Python Stack ---------------------- +AWIPS Python Stack +------------------ -A number of Python packages are bundled with the AWIPS II EDEX and CAVE +A number of Python packages are bundled with the AWIPS EDEX and CAVE installations. ====================== ============== ============================== Package Version RPM Name ====================== ============== ============================== -Python 2.7.10 awips2-python -**awips** **0.9.8** **awips2-python-awips** -basemap 1.0.7 awips2-python-basemap -cartopy 0.13.0 awips2-python-cartopy -cherrypy 3.1.2 awips2-python-cherrypy +Python 2.7.11 awips2-python +**awips** **0.9.10** **awips2-python-awips** +cartopy 0.15.1 awips2-python-cartopy +cherrypy 3.8.2 awips2-python-cherrypy cycler 0.10.0 awips2-python-cycler -cython 0.23.4 awips2-python-cython -dateutil 2.5.0 awips2-python-dateutil -gfe 1.0.0 awips2-python-gfe +cython 0.25.2 awips2-python-cython +dateutil 2.4.2 awips2-python-dateutil +gfe 16.4.1 awips2-python-gfe h5py 1.3.0 awips2-python-h5py -jep 3.4.1 awips2-python-jep +jep 3.5.3 awips2-python-jep matplotlib 1.5.1 awips2-python-matplotlib -metpy 0.3.0 awips2-python-metpy -nose 0.11.1 awips2-python-nose -numpy 1.8.0 awips2-python-numpy -pil 1.1.6 awips2-python-pil +metpy 0.5.0 awips2-python-metpy +nose 1.3.7 awips2-python-nose +numpy 1.9.2 awips2-python-numpy pint 0.7.2 awips2-python-pint pmw 1.3.2 awips2-python-pmw -pupynere 1.0.13 awips2-python-pupynere -pycairo 1.2.2 awips2-python-pycairo -pygtk 2.8.6 awips2-python-pygtk -pyparsing 2.1.0 awips2-python-pyparsing +pupynere 1.0.15 awips2-python-pupynere +pyparsing 2.0.3 awips2-python-pyparsing qpid 0.32 awips2-python-qpid scientific 2.8 awips2-python-scientific -scipy 0.9.0 awips2-python-scipy -shapely 1.5.9 awips2-python-shapely +scipy 0.15.1 awips2-python-scipy +shapely 1.4.4 awips2-python-shapely six 1.10.0 awips2-python-six -tables 2.1.2 awips2-python-tables -tpg 3.1.2 awips2-python-tpg -werkzeug 3.1.2 awips2-python-werkzeug +tables 3.3.0 awips2-python-tables +tpg 3.2.2 awips2-python-tpg +werkzeug 0.12.1 awips2-python-werkzeug ====================== ============== ============================== EDEX -------------------- +---- -The main server for AWIPS II. Qpid sends alerts to EDEX when data stored +The main server for AWIPS. Qpid sends alerts to EDEX when data stored by the LDM is ready for processing. These Qpid messages include file header information which allows EDEX to determine the appropriate data decoder to use. The default ingest server (simply named ingest) handles @@ -99,10 +108,10 @@ ingest and request servers are started and stopped with the commands ``/etc/rc.d/init.d/edex_camel`` CAVE -------------------- +---- Common AWIPS Visualization Environment. The data rendering and -visualization tool for AWIPS II. CAVE contains of a number of different +visualization tool for AWIPS. CAVE contains of a number of different data display configurations called perspectives. Perspectives used in operational forecasting environments include **D2D** (Display Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP** @@ -116,7 +125,7 @@ Two-Dimensional), **GFE** (Graphical Forecast Editor), and **NCP** CAVE Alertviz -------------------- +-------- **Alertviz** is a modernized version of an AWIPS I application, designed to present various notifications, error messages, and alarms to the user @@ -126,14 +135,14 @@ not required to be run separately. The toolbar is also **hidden from view** and is accessed by right-click on the desktop taskbar icon. LDM -------------------- +--- `http://www.unidata.ucar.edu/software/ldm/ `_ The **LDM** (Local Data Manager), developed and supported by Unidata, is a suite of client and server programs designed for data distribution, and is the fundamental component comprising the Unidata Internet Data -Distribution (IDD) system. In AWIPS II, the LDM provides data feeds for +Distribution (IDD) system. In AWIPS, the LDM provides data feeds for grids, surface observations, upper-air profiles, satellite and radar imagery and various other meteorological datasets. The LDM writes data directly to file and alerts EDEX via Qpid when a file is available for @@ -142,7 +151,7 @@ processing. The LDM is started and stopped with the commands ``service edex_ldm start`` and ``service edex_ldm stop`` edexBridge -------------------- +---------- edexBridge, invoked in the LDM configuration file ``/awips2/ldm/etc/ldmd.conf``, is used by the LDM to post "data @@ -150,12 +159,12 @@ available" messaged to Qpid, which alerts the EDEX Ingest server that a file is ready for processing. Qpid -------------------- +---- `http://qpid.apache.org `_ **Apache Qpid**, the Queue Processor Interface Daemon, is the messaging -system used by AWIPS II to facilitate communication between services. +system used by AWIPS to facilitate communication between services. When the LDM receives a data file to be processed, it employs **edexBridge** to send EDEX ingest servers a message via Qpid. When EDEX has finished decoding the file, it sends CAVE a message via Qpid that @@ -164,7 +173,7 @@ and stopped by ``edex start`` and ``edex stop``, and is controlled by the system script ``/etc/rc.d/init.d/qpidd`` PostgreSQL -------------------- +---------- `http://www.postgresql.org `_ @@ -178,12 +187,12 @@ called **psql**. Postgres is started and stopped by ``edex start`` and ``/etc/rc.d/init.d/edex_postgres`` HDF5 -------------------- +---- `http://www.hdfgroup.org/HDF5/ `_ **Hierarchical Data Format (v.5)** is -the primary data storage format used by AWIPS II for processed grids, +the primary data storage format used by AWIPS for processed grids, satellite and radar imagery and other products. Similar to netCDF, developed and supported by Unidata, HDF5 supports multiple types of data within a single file. For example, a single HDF5 file of radar data may @@ -193,10 +202,10 @@ also contain data from multiple radars. HDF5 is stored in ``/awips2/edex/data/hdf5/`` PyPIES (httpd-pypies) -------------------- +--------------------- **PyPIES**, Python Process Isolated Enhanced Storage, was created for -AWIPS II to isolate the management of HDF5 Processed Data Storage from +AWIPS to isolate the management of HDF5 Processed Data Storage from the EDEX processes. PyPIES manages access, i.e., reads and writes, of data in the HDF5 files. In a sense, PyPIES provides functionality similar to a DBMS (i.e PostgreSQL for metadata); all data being written diff --git a/docs/source/api/DataAccessLayer.rst b/docs/source/api/DataAccessLayer.rst new file mode 100644 index 0000000..e000038 --- /dev/null +++ b/docs/source/api/DataAccessLayer.rst @@ -0,0 +1,7 @@ +=============== +DataAccessLayer +=============== + +.. automodule:: awips.dataaccess.DataAccessLayer + :members: + :undoc-members: diff --git a/docs/source/api/DateTimeConverter.rst b/docs/source/api/DateTimeConverter.rst new file mode 100644 index 0000000..75949fc --- /dev/null +++ b/docs/source/api/DateTimeConverter.rst @@ -0,0 +1,7 @@ +================= +DateTimeConverter +================= + +.. automodule:: awips.DateTimeConverter + :members: + :undoc-members: diff --git a/docs/source/api/IDataRequest.rst b/docs/source/api/IDataRequest.rst new file mode 100644 index 0000000..d818819 --- /dev/null +++ b/docs/source/api/IDataRequest.rst @@ -0,0 +1,7 @@ +=============================== +IDataRequest (newDataRequest()) +=============================== + +.. autoclass:: awips.dataaccess.IDataRequest + :members: + :special-members: diff --git a/docs/source/api/PyData.rst b/docs/source/api/PyData.rst new file mode 100644 index 0000000..4097955 --- /dev/null +++ b/docs/source/api/PyData.rst @@ -0,0 +1,7 @@ +====================== +PyData +====================== + +.. automodule:: awips.dataaccess.PyData + :members: + :undoc-members: diff --git a/docs/source/api/PyGeometryData.rst b/docs/source/api/PyGeometryData.rst new file mode 100644 index 0000000..24ac092 --- /dev/null +++ b/docs/source/api/PyGeometryData.rst @@ -0,0 +1,7 @@ +====================== +PyGeometryData +====================== + +.. automodule:: awips.dataaccess.PyGeometryData + :members: + :undoc-members: diff --git a/docs/source/api/PyGridData.rst b/docs/source/api/PyGridData.rst new file mode 100644 index 0000000..860e7b9 --- /dev/null +++ b/docs/source/api/PyGridData.rst @@ -0,0 +1,7 @@ +====================== +PyGridData +====================== + +.. automodule:: awips.dataaccess.PyGridData + :members: + :undoc-members: diff --git a/docs/source/api/RadarCommon.rst b/docs/source/api/RadarCommon.rst new file mode 100644 index 0000000..4d0509e --- /dev/null +++ b/docs/source/api/RadarCommon.rst @@ -0,0 +1,7 @@ +====================== +RadarCommon +====================== + +.. automodule:: awips.RadarCommon + :members: + :undoc-members: diff --git a/docs/source/api/SoundingsSupport.rst b/docs/source/api/SoundingsSupport.rst new file mode 100644 index 0000000..527a1da --- /dev/null +++ b/docs/source/api/SoundingsSupport.rst @@ -0,0 +1,7 @@ +====================== +SoundingsSupport +====================== + +.. automodule:: awips.dataaccess.SoundingsSupport + :members: + :undoc-members: diff --git a/docs/source/api/ThriftClientRouter.rst b/docs/source/api/ThriftClientRouter.rst new file mode 100644 index 0000000..d65f137 --- /dev/null +++ b/docs/source/api/ThriftClientRouter.rst @@ -0,0 +1,7 @@ +====================== +ThriftClientRouter +====================== + +.. automodule:: awips.dataaccess.ThriftClientRouter + :members: + :undoc-members: diff --git a/docs/source/api/index.rst b/docs/source/api/index.rst new file mode 100644 index 0000000..c18d4aa --- /dev/null +++ b/docs/source/api/index.rst @@ -0,0 +1,18 @@ +################# +API Documentation +################# + +.. toctree:: + :maxdepth: 2 + + DataAccessLayer + IDataRequest + PyData + PyGridData + PyGeometryData + SoundingsSupport + ThriftClientRouter + RadarCommon + DateTimeConverter + +* :ref:`genindex` diff --git a/docs/source/conf.py b/docs/source/conf.py index 142f2e8..83b1578 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -56,16 +56,16 @@ source_suffix = '.rst' master_doc = 'index' # General information about the project. -project = u'python-awips' -copyright = u'2016, Unidata' -author = u'Unidata' +project = 'python-awips' +copyright = '2016, Unidata' +author = 'Unidata' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. -version = u'0.9.8' +version = '0.9.10' # The full version, including alpha/beta/rc tags. # The language for content autogenerated by Sphinx. Refer to documentation @@ -231,8 +231,8 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'python-awips.tex', u'python-awips Documentation', - u'Unidata', 'manual'), + (master_doc, 'python-awips.tex', 'python-awips Documentation', + 'Unidata', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of @@ -261,7 +261,7 @@ latex_documents = [ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - (master_doc, 'python-awips', u'python-awips Documentation', + (master_doc, 'python-awips', 'python-awips Documentation', [author], 1) ] @@ -275,7 +275,7 @@ man_pages = [ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - (master_doc, 'python-awips', u'python-awips Documentation', + (master_doc, 'python-awips', 'python-awips Documentation', author, 'python-awips', 'One line description of project.', 'Miscellaneous'), ] diff --git a/docs/source/dev.rst b/docs/source/dev.rst index e737c8a..e01cd5a 100644 --- a/docs/source/dev.rst +++ b/docs/source/dev.rst @@ -1,6 +1,6 @@ -Dev Guide -========= +Development Guide +================= The Data Access Framework allows developers to retrieve different types of data without having dependencies on those types of data. It provides @@ -266,7 +266,7 @@ contributing the necessary code for the framework to support its data. For example, the satellite plugin provides a factory class for interacting with the framework and registers itself as being compatible with the Data Access Framework. This concept is similar to how EDEX in -AWIPS II expects a plugin developer to provide a decoder class and +AWIPS expects a plugin developer to provide a decoder class and record class and register them, but then automatically manages the rest of the ingest process including routing, storing, and alerting on new data. This style of plugin architecture effectively enables the diff --git a/docs/source/examples/index.rst b/docs/source/examples/index.rst index 80a4f80..6c9ba8f 100644 --- a/docs/source/examples/index.rst +++ b/docs/source/examples/index.rst @@ -1,8 +1,8 @@ .. _examples-index: -################ - Data Plotting Examples -################ +###################### +Data Plotting Examples +###################### .. toctree:: :maxdepth: 1 diff --git a/docs/source/index.rst b/docs/source/index.rst index 9fb703b..a02255e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,9 +2,45 @@ Python AWIPS Data Access Framework ================================== -The `python-awips `_ package provides a Data Access Framework (DAF) for requesting data from a remote AWIPS II EDEX server. +`AWIPS `_ is a weather display and analysis package developed by the National Weather Service for operational forecasting. UCAR's `Unidata Program Center `_ supports a non-operational open-source release of the AWIPS software (`EDEX `_, `CAVE `_, and `python-awips `_). -The `AWIPS II Python Stack `_ installed via RPM contains the DAF, matplotlib, numpy, scipy, basemap, pint, shapely, and other packages. +The python-awips package provides a data access framework for requesting grid and geometry datasets from an `EDEX `_ server. + +.. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks + +Install +------- + +- pip install python-awips + +Requirements +~~~~~~~~~~~~ + +- Python 2.7+ +- Shapely 1.4+ +- MetPy and enum34 to run the `Jupyter Notebook`_ examples + +How to Use +~~~~~~~~~~ + +:: + + from awips.dataaccess import DataAccessLayer + DataAccessLayer.changeEDEXHost("edex-cloud.unidata.ucar.edu") + request = DataAccessLayer.newDataRequest() + dataTypes = DataAccessLayer.getSupportedDatatypes() + request.setDatatype("grid") + request.addLocationNames("RAP40") + request.setParameters("T") + request.setLevels("0.0SFC") + cycles = DataAccessLayer.getAvailableTimes(request, True) + times = DataAccessLayer.getAvailableTimes(request) + response = DataAccessLayer.getGridData(request, times[-1]) + for grid in response: + data = grid.getRawData() + lons, lats = grid.getLatLonCoords() + +See the `API Documentation `_ for more information. ------------- Documentation @@ -14,7 +50,8 @@ Documentation :maxdepth: 2 install + api/index examples/index - about dev gridparms + about diff --git a/docs/source/install.rst b/docs/source/install.rst index 5ea8457..3a36ebc 100644 --- a/docs/source/install.rst +++ b/docs/source/install.rst @@ -1,6 +1,6 @@ .. _Jupyter Notebook: http://nbviewer.jupyter.org/github/Unidata/python-awips/tree/master/examples/notebooks -Install python-awips +Installation ------------------ - pip install python-awips @@ -15,19 +15,23 @@ Requirements Install from Github ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- git clone https://github.com/Unidata/python-awips.git -- cd python-awips +- git clone https://github.com/Unidata/python-awips.git && cd python-awips - python setup.py install -Install for AWIPS (/awips2/python) -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Install for AWIPS +~~~~~~~~~~~~~~~~~ -AWIPS II >=15.1.3 (March 2016) has `python-awips` bundled with the awips2-server and awips2-cave groups, as well as a full meteorological data stack (metpy, matplotlib, numpy, etc). +On standalone AWIPS systems, the full `AWIPS Python Stack `_ is installed to ``/awips2/python`` as RPM packages. Easy install on an AWIPS system -- wget https://bootstrap.pypa.io/ez_setup.py -O - | /awips2/python/bin/python -- /awips2/python/bin/easy_install pip -- /awips2/python/bin/pip install python-awips +* For Unidata AWIPS release **16.2.2+**: + + * /awips2/python/bin/easy_install pip + * /awips2/python/bin/pip install python-awips + +* For releases before and up to **16.1.5** you may need to run + + * yum install awips2-python-setuptools