{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Python-AWIPS Tutorial Notebook"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"# Objectives\n",
"\n",
"* Use python-awips to connect to an edex server\n",
"* Create a plot for a regional area of the United States (Florida)\n",
"* Define and filter data request for METAR and Synoptic surface obs\n",
"* Use the maps database to request and draw state boundaries (no use of Cartopy.Feature in this example)\n",
"* Stylize and plot surface data using Metpy\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Table of Contents\n",
"\n",
"[1 Imports](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#imports)
\n",
"[2 Function: get_cloud_cover()](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#function-get-cloud-cover)
\n",
"[3 Function: make_map()](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#function-make-map)
\n",
"[4 Function: extract_plotting_data()](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#function-extract-plotting-data)
\n",
"[5 Function: plot_data()](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#function-plot-data)
\n",
"[6 Initial Setup](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#initial-setup)
\n",
" [6.1 Initial EDEX Connection](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#initial-edex-connection)
\n",
" [6.2 Maps Request and Response](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#maps-request-and-response)
\n",
" [6.3 Define Geographic Filter](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#define-geographic-filter)
\n",
" [6.4 Define Time Filter](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#define-time-filter)
\n",
" [6.5 Define Common Parameters for Data Requests](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#define-common-parameters-for-data-requests)
\n",
" [6.6 Define METAR Request](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#define-metar-request)
\n",
" [6.7 Define Synoptic Request](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#define-synoptic-request)
\n",
"[7 Get the Data!](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#get-the-data)
\n",
" [7.1 Get the EDEX Responses](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#get-the-edex-responses)
\n",
" [7.2 Extract Plotting Data](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#extract-plotting-data)
\n",
"[8 Plot the Data](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#get-the-data)
\n",
" [8.1 Draw the Region](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#draw-the-region)
\n",
" [8.2 Plot METAR Data](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#plot-metar-data)
\n",
" [8.3 Plot Synoptic Data](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#plot-synoptic-data)
\n",
" [8.4 Plot both METAR and Synoptic Data](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#plot-both-metar-and-synopitc-data)
\n",
"[9 See Also](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#see-also)
\n",
" [9.1 Related Notebooks](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#related-notebooks)
\n",
" [9.2 Additional Documentation](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html#additional-documentation)
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Imports\n",
"\n",
"The imports below are used throughout the notebook. Note the first two imports are coming directly from python-awips and allow us to connect to an EDEX server, and define a timrange used for filtering the data. The subsequent imports are for data manipulation and visualization. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from awips.dataaccess import DataAccessLayer\n",
"from dynamicserialize.dstypes.com.raytheon.uf.common.time import TimeRange\n",
"from datetime import datetime, timedelta\n",
"import numpy as np\n",
"import cartopy.crs as ccrs\n",
"from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER\n",
"from cartopy.feature import ShapelyFeature\n",
"from shapely.geometry import Polygon\n",
"import matplotlib.pyplot as plt\n",
"from metpy.units import units\n",
"from metpy.calc import wind_components\n",
"from metpy.plots import simple_layout, StationPlot, StationPlotLayout, sky_cover\n",
"import warnings"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Top](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Function: get_cloud_cover()\n",
"\n",
"Returns the cloud coverage values as integer codes (0 through 8)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def get_cloud_cover(code):\n",
" if 'OVC' in code:\n",
" return 8\n",
" elif 'BKN' in code:\n",
" return 6\n",
" elif 'SCT' in code:\n",
" return 4\n",
" elif 'FEW' in code:\n",
" return 2\n",
" else:\n",
" return 0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Top](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Function: make_map()\n",
"\n",
"In order to plot more than one image, it's easiest to define common logic in a function. Here, a new function called **make_map** is defined. This function uses the [matplotlib.pyplot package (plt)](https://matplotlib.org/3.3.3/api/_as_gen/matplotlib.pyplot.html) to create a figure and axis. The geographic extent is set and lat/lon gridlines are added for context."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"def make_map(bbox, proj=ccrs.PlateCarree()):\n",
" fig, ax = plt.subplots(figsize=(16,12),subplot_kw=dict(projection=proj))\n",
" ax.set_extent(bbox)\n",
" gl = ax.gridlines(draw_labels=True, color='#e7e7e7')\n",
" gl.top_labels = gl.right_labels = False\n",
" gl.xformatter = LONGITUDE_FORMATTER\n",
" gl.yformatter = LATITUDE_FORMATTER\n",
" return fig, ax"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Top](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Function: extract_plotting_data()\n",
"\n",
"Grab the simple variables out of the response data we have (attaching correct units), and\n",
"put them into a dictionary that we will hand the plotting function later:\n",
"\n",
"- Get wind components from speed and direction\n",
"- Convert cloud coverage values to integer codes [0 - 8]\n",
"- Assign temperature, dewpoint, and sea level pressure the the correct units\n",
"- Account for missing values (by using `nan`)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"def extract_plotting_data(arr, datatype):\n",
" \"\"\"\n",
" Extract all necessary data for plotting for either\n",
" datatype: 'obs' or 'sfcobs'\n",
" \"\"\"\n",
" \n",
" data = dict()\n",
" data['latitude'] = np.array(arr['latitude'])\n",
" data['longitude'] = np.array(arr['longitude'])\n",
" tmp = np.array(arr['temperature'], dtype=float)\n",
" dpt = np.array(arr['dewpoint'], dtype=float)\n",
" direction = np.array(arr['windDir'])\n",
"\n",
" # Suppress nan masking warnings\n",
" warnings.filterwarnings(\"ignore\",category =RuntimeWarning)\n",
" \n",
" # Account for missing values\n",
" tmp[tmp == -9999.0] = 'nan'\n",
" dpt[dpt == -9999.] = 'nan'\n",
" direction[direction == -9999.0] = 'nan'\n",
" \n",
" data['air_pressure_at_sea_level'] = np.array(arr['seaLevelPress'])* units('mbar')\n",
" u, v = wind_components(np.array(arr['windSpeed']) * units('knots'),\n",
" direction * units.degree)\n",
" \n",
" data['eastward_wind'], data['northward_wind'] = u, v\n",
" data['present_weather'] = arr['presWeather']\n",
"\n",
" \n",
" # metars uses 'stationName' for its identifier and temps are in deg C\n",
" # metars also has sky coverage\n",
" if datatype == \"obs\":\n",
" data['stid'] = np.array(arr['stationName'])\n",
" data['air_temperature'] = tmp * units.degC\n",
" data['dew_point_temperature'] = dpt * units.degC\n",
" data['cloud_coverage'] = [int(get_cloud_cover(x)) for x in arr['skyCover']]\n",
" \n",
" # synoptic obs uses 'stationId', and temps are in Kelvin\n",
" elif datatype == \"sfcobs\":\n",
" data['stid'] = np.array(arr['stationId'])\n",
" data['air_temperature'] = tmp * units.kelvin\n",
" data['dew_point_temperature'] = dpt * units.kelvin\n",
" \n",
" return data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Top](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Function: plot_data()\n",
"\n",
"This function makse use of Metpy.StationPlotLayout and Metpy.StationPlot to add all surface observation data to our plot. The logic is very similar for both METAR and Synoptic data, so a `datatype` argument is used to distinguish between which data is being drawn, and then draws the appropriate features.\n",
"\n",
"This function plots:\n",
"- Wind barbs\n",
"- Air temperature\n",
"- Dew point temperature\n",
"- Precipitation\n",
"- Cloud coverage (for METARS)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"def plot_data(data, title, axes, datatype):\n",
" custom_layout = StationPlotLayout()\n",
" custom_layout.add_barb('eastward_wind', 'northward_wind', units='knots')\n",
" custom_layout.add_value('NW', 'air_temperature', fmt='.0f', units='degF', color='darkred')\n",
" custom_layout.add_value('SW', 'dew_point_temperature', fmt='.0f', units='degF', color='darkgreen')\n",
" custom_layout.add_value('E', 'precipitation', fmt='0.1f', units='inch', color='blue')\n",
" # metars has sky coverage\n",
" if datatype == 'obs':\n",
" custom_layout.add_symbol('C', 'cloud_coverage', sky_cover)\n",
" axes.set_title(title)\n",
" stationplot = StationPlot(axes, data['longitude'], data['latitude'], clip_on=True,\n",
" transform=ccrs.PlateCarree(), fontsize=10)\n",
" custom_layout.plot(stationplot, data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Top](https://unidata.github.io/python-awips/examples/generated/Regional_Surface_Obs_Plot.html)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initial Setup\n",
"\n",
"Connect to an EDEX server and define several [new data request objects](http://unidata.github.io/python-awips/api/IDataRequest.html).\n",
"\n",
"In this example we're using multiple different datatypes from EDEX, so we'll create a request object for each of the following:\n",
"- [The states outlines (datatype **maps**)](#Define-Maps-Request)\n",
"- [The METAR data (datatype **obs**)](#Define-METAR-Request)\n",
"- [The Synoptic data (datatype **sfc**)](#Define-Synoptic-Request)\n",
"\n",
"Some of the request use filters, so we'll also create several filters than can be used for the various data requests as well."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Initial EDEX Connection\n",
"\n",
"First we establish a connection to Unidata's public EDEX server."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"# EDEX connection\n",
"edexServer = \"edex-cloud.unidata.ucar.edu\"\n",
"DataAccessLayer.changeEDEXHost(edexServer)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Maps Request and Response\n",
"The maps data request will give us data to draw our state outlines of interest (Florida and its neighboring states). We will retrieve the data response object here so we can create a geographic filter for the METAR and Synoptic data requests."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 6 MultiPolygons\n"
]
}
],
"source": [
"# Define the maps request\n",
"maps_request = DataAccessLayer.newDataRequest('maps')\n",
"# filter for multiple states\n",
"maps_request = DataAccessLayer.newDataRequest('maps')\n",
"maps_request.addIdentifier('table', 'mapdata.states')\n",
"maps_request.addIdentifier('geomField', 'the_geom')\n",
"maps_request.addIdentifier('inLocation', 'true')\n",
"maps_request.addIdentifier('locationField', 'state')\n",
"maps_request.setParameters('state','name','lat','lon')\n",
"maps_request.setLocationNames('FL','GA','MS','AL','SC','LA')\n",
"maps_response = DataAccessLayer.getGeometryData(maps_request)\n",
"print(\"Found \" + str(len(maps_response)) + \" MultiPolygons\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Define Geographic Filter\n",
"\n",
"The previous EDEX request limited the data by using a **parameter** for the maps database called **state**. We can take the results from that filter and get a geographic **envelope** based on the Florida polygon that was returned from the previous cell.\n",
"\n",
"
\n",
"