Maps.py - Map Background Configuration

January 18, 2006

Organization

Maps.py Format
Site Information
Shapefile Names
Map Definitions
Putting it all together with the maps list
Importing localMaps
AWIPS standard names
Shapefile Description
Filenames
Catalog Entry Description
Attributes

Overview

The Maps.py file defines the attributes of each map.  The server when started looks through the list of maps as defined in the Maps.py file and generates maps.  These maps may be retrieved through the server protocol.  The map data available through the protocol includes polygons, points, and arcs.  The information contains the geographical data and the set of attributes that define the shape.

The map generation software uses shapefiles as the input.  These shapefiles are filtered by domain (based on the site) and attributes.  There are different attributes available for each shapefile.  Refer to the AWIPS Map Background Database provided by the National Weather Service for details on the available shapefiles and the attributes contained within each shapefile.  This information will be needed in order to tailor a map to your site.

The Maps.py file also is used to define and automatically generate edit areas based on the shapefile polygons.  The user can select which polygons will be converted into edit areas and which edit area groups they will be assigned.

This information is provided for you to help you understand the format of Maps.py.  You should NEVER change the original Maps.py since your changes will be overwritten with the next upgrade.  See the server configuration overview for information on how to make changes that are supported to the map backgrounds.

To override changes in Maps.py, use the localMaps.py technique. To override the default map filenames, i.e., source filenames for the shapefiles, use the localMapFiles.py technique.

To update shapefiles, refer to the MapFiles documentation, specifically the section on Updating Shapefiles.


Software releases after Jan 1, 2006 separate the map shapefile from the CORE GFE.   In some applications, such as AWIPS, the map shapefile configuration becomes the responsibility of the site since the map shapefiles will no longer be shipped with the GFE software.   A separate install is available for non-AWIPS purposes and is posted with the software releases.   If the associated Map Shapefile install is not installed at your site, then sites will be responsible for:

Maps.py Format

Site Information

Maps.py inputs the siteConfig file which provides the site identifier, map directory, and the site's specific grid domains. Several variables are computed:


CWA = siteConfig.GFESUITE_SITEID
MAPDIR = siteConfig.GFESUITE_DATDIR + "/maps/"
 

Shapefile Names

The names of the shapefiles are contained in MapFiles.py and overridden in localMapFiles.py.  The section in Maps.py imports these files.

#-----------------------------------
# DO NOT CHANGE THE FOLLOWING SECTION
#------------------------------------
# Filename Translations
from MapFiles import *
try:
    from localMapFiles import *
except ImportError:
    pass
 

Map Definitions

The map definition section defines each map that is desired.  Only a small portion of the Maps.py is shown here. Numerous examples with differing capabilities are shown.

Basic Map

This definition is for a basic map, with no filtering of attributes and no automatic creation of edit areas. The Shapefile object is created in the first line. The name of the shapefile (ZoneMapName) is assigned to using the filename in the second line.  The name of the map is defined in the third line.  All polygons in the shapefile will be used to create the name of the map.  The name of the map is the name of the CWA, e.g., BOI, appended with _zones, such as BOI_zones.
 

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.name = CWA + '_zones'

Filtered Map

This definition is for a filtered map with no automatic creation of edit areas.  The filter line performs the filtering.  The basic format of this line is as follows:

     pythonMapName.filter(lambda x : x['ATTRIBUTENAME'] = AttributeValue)

where ATTRIBUTENAME is one of the attributes in the shapefile, and AttributeValue is the value that must match in the shapefile for this shape to be kept in the map. Attributes for the NWS shapefiles can be obtained from the AWIPS Map Catalog AWIPS Map Background Database.

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.filter(lambda x : x['CWA'] == CWA)
CWAzones.name = CWA + '_zones'

The "lambda" function is a shortcut method of writing a function, the code snippet below performs the identical function:

def cwaZoneFilt(x):
    return x['CWA'] == CWA

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.filter(cwaZoneFilt)
CWAzones.name = CWA + '_zones'
 

Since this is all Python code, there is another technique you can use to write a filter function which is complex.  Here is an example of writing a filter function for the cities map background to only include a specific set of cities:

mycities = ['Akron', 'Denver', 'Boulder']
map9 = ShapeFile(MAPDIR)
map9.filename('cities')
map9.filter(lambda x : x['NAME'] in mycities)
map9.name = 'MyCities'
 

An alternative method of the above city filter is shown below:

def map2Filt(x):
    zcities = ['Akron', 'Denver', 'Boulder']
    return x['NAME'] in zcities

Ycounties = ShapeFile(MAPDIR)
Ycounties.filename('cities')
Ycounties.filter(map2Filt)
Ycounties.name = 'Map2'

Complex filters can also be generated in any of the above formats.  For example, here is a filter that pulls certain counties (Summit, Sandusy, Huron, and Medina) out of a shapefile for a particular state (Ohio) and cwa (Cleveland):

def exampleFilt(x):
    myCounties = ['Summit', 'Sandusky', 'Huron', 'Medina']
    return x['COUNTYNAME'] in myCounties and x['ST'] == "OH" and x['CWA'] = 'CLE'

OHcounties = ShapeFile(MAPDIR)
OHcounties.filename(CountyMapName)
OHcounties.filter(exampleFilt)
OHcounties.name = 'ExampleOHCounties'

Here is another complex filter, for a cities map, that uses the population filter, but excludes certain cities.

excludedCities  =  ['Clinton', 'Pearl', 'Brandon', 'Ridgeland']
cities  =  ShapeFile(MAPDIR)
cities.filename('cities')
cities.filter(lambda x,y=excludedCities: float(x['POP_1990']) > 10000 and  x['NAME'] not in y)
cities.name = "JanCities"
 
Note that the POP_1990 attribute is really a string value, and that the float() function converts it to a number.  All attributes in the shapefiles are considered to be strings.  If you want to do number comparisons, then you need to promote the attribute to a number using the float() function.

Filtered Map with Automatically Generated Edit Areas

The ifpServer can automatically generate edit areas based on the information in the shapefiles. The user specifies the "editAreaName" in the Maps.py or localMaps.py files, and the ifpServer uses that information to determine the name of each edit area.   There are three forms to the editAreaName line as shown:

mapVariable.editAreaName = "string"
 Single String Format
mapVariable.editAreaName = ["string1", "string2", "string3"]
 List of Strings Format
mapVariable.editAreaName = functionName
 Function Format

Single String Format
This definition is for a filtered map with automatic generation of edit areas.  All of the generated areas will go into the Misc. edit area group. In the example below, the name of each generated edit area is obtained from the attribute 'ZONE' in the field editAreaName. The program looks up the value for the attribute ZONE in the shapefile for each polygon and creates an edit area with that name.  The example below uses the simple format for the editAreaName, in which only one attribute is specified.
 

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.filter(lambda x : x['CWA'] == CWA)
CWAzones.name = CWA + '_zones'
CWAzones.editAreaName = 'ZONE'

Examples of the names of the edit areas generated from the above snippet is shown in the following table:
 
Zone value from shapefile State (not used in this example) Name of Edit Area
034 CO ZONE034
041 CO ZONE041
041 NE ZONE041

Note that if a CWA covers more than one state, this simple form of generating edit area names will not work.  The "NE" example above illustrates that the same edit area name was generated even though the zones were from different states.  Also note that the naming of the edit area includes the name "ZONE" only because the value of the zone value begins with a number.

NOTE: Changing the default set of edit area names as supplied in the baseline may render your text formatters and VTEC inoperable.  The text formatters and VTEC assume standard UGC-named edit areas, such as COZ023 and UTC013.

List of Strings Format

The more complex form of the editAreaName involves using a Python list.  The naming of  an edit area is determined by using more than one attribute.  This example uses the more complex form for the editAreaName, in which two attributes are specified.

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.filter(lambda x : x['CWA'] == CWA)
CWAzones.name = CWA + '_zones'
CWAzones.editAreaName = ['STATE','ZONE']

Examples of the names of the edit areas generated from the above snippet is shown in the following table:
 
Zone value from shapefile State (not used in this example) Name of Edit Area
034 CO CO_034
041 CO CO_041
041 NE NE_041

The edit area name is determined from the value of the STATE attribute, followed by an underscore character, and then the value of the ZONE attribute.

Here is the software algorithm for naming of edit areas:

The ordering of the list of attributes can change the naming as shown in the following table, which assumes ZONE=034, STATE=CO, WFO=BOU, NAME=Summit County/Mosquito Range/Indian Peaks:
 
editAreaName string Name of Edit Area
'ZONE' ZONE034
['ZONE','STATE'] ZONE034_CO
['STATE','ZONE'] CO_034
['WFO','NAME'] BOU_SummitCountyMosquitoRangeIndianPeaks
['ZONE','NAME','STATE'] ZONE034_SummitCountyMosquitoRangeIndianPeaks_CO

Note that specification of an attribute that doesn't exist will simply use the name in the string.  For example, if you are creating edit areas and you want them of the form ISC_xxx, where xxx is the WFO identifier.  The editAreaName string would be the following:
ISCareas.editAreaName = ['ISC', 'WFO']

NOTE: Changing the default set of edit area names as supplied in the baseline may render your text formatters and VTEC inoperable.  The text formatters and VTEC assume standard UGC-named edit areas, such as COZ023 and UTC013.

Function Format
Occasionally there is a need to calculate a special edit area name and the single or list of strings methods are not sufficient.  The function format requires the user to name a function and then define a function.  The function has one argument, which receives a dictionary of attribute names and their values.   The return value of the function is a string which becomes the edit area name.

For example, lets say that we want edit area names based on the county FIPS code.  The desired format is stateCfips, such as COC013, for Colorado County #13.  The county shapefile has a 'STATE' attribute, such as "CO", and it has a 'FIPS' attribute, such as 08013 for Colorado County #13.   Using the list of strings format, ['STATE', 'C', 'FIPS'], you would end up with CO_C_08013, when we really want COC013.

This is how the code would be implemented:

def cwaEAN(atts):
    fips = atts['FIPS'][-3:]  #take just the last three characters from the FIPS code
    s = atts['STATE'] + "C" + fips  #assemble the name
    return s    #return the complete edit area name

CWAcounties = ShapeFile(MAPDIR)
CWAcounties.filename(CountyMapName)
CWAcounties.filter(lambda x : x['CWA'] == CWA)
CWAcounties.name = CWA + '_counties'
CWAcounties.editAreaName = cwaEAN

The "atts" dictionary looks simlilar to the following:
{'COUNTYNAME': 'Delta', 'LON': -86.909679999999994, 'TIME_ZONE': 'E', 'FE_AREA': 'sr', 'LAT': 45.82085, 'STATE': 'MI', 'FIPS': '26041', 'CWA':
'MQT', 'recnum': 9}

and the final string is:  MIC041

 

Filtered Map with Automatically Generated Edit Areas with group definition

This is identical to the Filtered Map with Automatically Generated Edit Areas case except that the user has specified a group name.  All of the generated edit areas will be placed in the edit area group called 'Zones'.

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.filter(lambda x : x['CWA'] == CWA)
CWAzones.name = CWA + '_zones'
CWAzones.editAreaName = ['STATE','ZONE']
CWAzones.groupName = 'Zones'
 

Basic Map with Automatically Generated Edit Areas

An example of a basic map with automatically generated edit areas:

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.name = CWA + '_zones'
CWAzones.editAreaName = ['STATE','ZONE']
 

Basic Map with Automatically Generated Edit Areas with group definition

An example of a basic map with automatically generated edit areas with group definition:

CWAzones = ShapeFile(MAPDIR)
CWAzones.filename(ZoneMapName)
CWAzones.name = CWA + '_zones'
CWAzones.editAreaName = 'ZONE'
CWAzones.groupName = 'Zones'
 

Expanded Maps

Maps are automatically clipped by the latitude/longitude domain that is derived from the site information in serverConfig.py.  If none of the map polygons overlap this lat/lon domain, then the polygon will not be part of the map by default.  The expandDomain attribute for any map defined in Maps.py will expand the latitude/longitude domain by that specified.  The user can control the expansion factor in each of the four compass directions.

The format of the expandDomain tuple is shown below:

mapName.expandDomain = (northExpand, eastExpand, southExpand, westExpand)

For example, to expand the map domain 2 degrees to the north, 3 degrees to the east, no expansion to the south, and 1 1/2 degrees to the west, for the Counties map, the following line would be used.

Counties.expandDomain = (2, 3, 0, 1.5)

Note that expanding the maps will result in map backgrounds that are larger.  This translates into slower load times for maps in the GFE.
 

Precision

The ifpServer will throw out vertex points of polyline and polygon maps which are closer togeather than needed for the GFE. Since opinions differ on how close points are before they are not needed, a ShapeFile can be assigned an optional attribute to define this. The attribute is called precision and is an integer or None assigned like so:

CWAzones.precision = 3

The server will then round all values in the data to this number of decimal places. Adjacent duplicate values will then be removed. This helps reduce the amount of data contained in the maps making them faster to display.

By default all polygon and polyline maps will be rounded to a precision of 2. The above example overrides this to 3. If a site wishes to do no rounding then precision can be set to None and this data reduction will be skipped entirely.

In summary, precision can be set to an integer greater than or equal to zero or None. The best way to change this value is in your localMaps.py file. The above example could be placed into localMaps.py to change the precision of the CWAzones map to 3.
 
 
 

Putting it all Together with the map list

In order for the system to recognize the list of defined maps, they must be put all together in a list.  The list MUST be called maps.  This example list, shows that there are three maps defined.  The entries within the list are the Python variable names (name to the left of the equal sign on the ShapeFile line).

maps = [ CWAcounties, Counties, CWAzones ]
 

Importing localMaps

The final section of the Maps.py file is the software that checks for a localMaps configuration and uses it if it exists.  Do not change this section of the file.

#-----------------------------------
# DO NOT CHANGE THE FOLLOWING SECTION
#------------------------------------
# import the local maps file
try:
    import localMaps
except:
    pass
 
 

AWIPS standard names

The AWIPS standard names table shows a sample filename, as loaded from the NWS map database, and the attribute fields. The user must download the files from the NWS map databases, unzip it, and either copy the resulting uncompressed files, or gzip -9 them to save space, to the appropriate directory. This list is an example, refer to the NWSH Map GIS page for complete up-to-date information. The link to the NWS Map GIS page is http://awips.nws.noaa.gov/mapdata/newcat/
Shapefile Contents Example File Name (without extensions) Attributes
Cities web_city LAND_WATER, ST_FIPS, COUNTY_FIP, PL_FIPS, NAME, LON, LAT, POP_1990, ST, WATCH_WARN, PROG_DISC
Counties c_19nv01 STATE, CWA, COUNTYNAME, FIPS, TIME_ZONE, FE_AREA, LON, LAT
County Warning Areas w_30nv01 WFO, CWA, LON, LAT
Coastal and Offshore Marine Zones mz15ja02 ID, WFO, NAME, LON, LAT, WFO_AREA
High Seas Marine Zones hz02oc01 WFO, LON, LAT, HEADING
Interstates roads LENGTH, TYPE, ADMN_CLASS, TOLL_RD, RTE_NUM1, RTE_NUM2, ROUTE
Lakes lk17de98 NAME, FEATURE, LON, LAT
NWS River Forecast Basins ba18oc01 ID, NAME, CWA, RFC, LON, LAT
Public Forecast Zones z_14ja02 STATE, ZONE, CWA, NAME, STATE_ZONE, TIME_ZONE, FE_AREA, LON, LAT
River Forecast Center Regions rf19fe99 SITE_ID, STATE, RFC_NAME, RFC_CITY, BASIN_ID
States and Territories s_24ja01 STATE, NAME, FIPS, LON, LAT
Canadian Provinces province CODE, NAME
Mexican States mx_state CODE, NAME
Fire Weather Zones fz28my02 STATE, ZONE, STATE_ZONE, NAME, TIME_ZONE, FE_AREA, LAT, LON
ISC Combined Public and Marine Areas cm15au02 WFO, LON, LAT
Fire Weather Area of Responsibility
 cf26ap04
 CWA

The Example File Name is the name of the file matching the name in the AWIPS Map Catalog AWIPS Map Background Database.  Several of these files are tagged with a date, therefore their name will change when updated.

Shapefile description

Filenames

Shapefiles provide a complete non-topological description of 2-dimensional geometric shapes, in three files:

filename.shp - shape description
filename.shx - index (shape to attributes)
filename.dbf - attributes

 The shape of each geometric (spatial) object is either a point, line, or polygon, and is described completely and in sequential order by the file *.shp. Attributes for each shape are stored sequentially in the same order by the file *.dbf, using the dBASE III+ Database File Structure. An index of shapes to attributes is maintained in the *.shx file. The Shapefile format is documented on-line at http://www.esri.com. By convention, a Shapefile is associated by one prefix filename. The contents of the *.dbf file can be changed without damaging the index relationship to *.shp as long as dBASE records are not added, deleted or re-ordered. Shapefiles can be renamed, provided that all three components are given the same filename and the extensions are not changed. We have restricted the filenames to 8 characters for compatibility with DOS systems.

 Each catalog entry provides an image of the Shapefile. When the geographic area is not obvious, the Shapefile is shown as darker points or lines on a lighter background. Space is conserved using the "zip" compression technique. Public domain software to extract *.zip files are now available for most operating systems and hardware environments from ftp://ftp.uu.net/pub/archiving/zip/. for NWS users. Make sure that the *.zip file has been downloaded using a binary transfer method. Previous archives in the *.tar.Z (UNIX) and self-extracting *.exe (DOS) formats have been deleted.

filename.zip - compressed (zip)
filename.gif - Graphics Interchange Format
filename.txt - flat text listing of filename.dbf
 

Catalog Entry Description

Naming Convention - a description of the name of the Shapefile. Most data in the catalog have a two letter prefix for the type of data and the date in ddmmyy format.
Location - directory in Catalog (URL)
Shapefile Type - type of shape {point, polyline, polygon}
Source - origin of shape and attribute information
Last Modified - last date shapes or attributes modified
File size information: - size of all relavant files in bytes and the number of records in the Dbase file (.dbf)for shapefiles or number of features in a text file.
 

Attributes

field - dBASE field name (1 to 10 characters)
type - field type {character, numeric, logical, date}
width,dec - field type specifications:
{character: width = # characters} {logical: width = 1} {date: width = 8}
{numeric: width = # digits, dec = # decimal places}
Description - brief statement of field usage.
 
 
 


 
 Back To Top
 Back To TOC