"text":"UnidataAWIPSUserManual\n\uf0c1\n\n\nhttps://www.unidata.ucar.edu/software/awips2\n \n\n\n\n\nThe Advanced Weather Interactive Processing System (AWIPS) is a meteorological software package. It is used for decoding, displaying, and analyzing data, and was originally developed for the National Weather Service (NWS) by Raytheon. There is a division here at UCAR called the Unidata Program Center (UCP) which develops and supports a modified non-operational version of AWIPS for use in research and education by \nUCAR member institutions\n. This is released as open source software, free to download and use by anyone.\n\n\nAWIPS takes a unified approach to data ingest, where most data ingested into the system comes through the \nLDM\n client pulling data feeds from the \nUnidata IDD\n. Various raw data and product files (netCDF, grib, BUFR, ASCII text, gini, AREA) are decoded and stored as HDF5 files and Postgres metadata by \nEDEX\n, which serves products and data over http.\n\n\nUnidata supports two data visualization frameworks: \nCAVE\n (an Eclipse-built Java application which runs on Linux, Mac, and Windows), and the \npython-awips\n.\n\n\n\n\nNote\n: Our version of CAVE is a \nnon-operational\n version. It does not support some features of NWS AWIPS. Warnings and alerts cannot be issued from Unidata's CAVE. Additional functionality may not be available as well.\n\n\n\n\n\n\n\n\nDownload and Install CAVE\n\uf0c1\n\n\n\n\nDownload and Install EDEX\n\uf0c1\n\n\n\n\nLicense\n\uf0c1\n\n\nUnidata AWIPS source code and binaries (RPMs) are considered to be in the public domain, meaning there are no restrictions on any download, modification, or distribution in any form (original or modified). Unidata AWIPS contains no proprietery content and is therefore not subject to export controls as stated in the Master Rights licensing file.\n\n\n\n\nAWIPS Data in the Cloud\n\uf0c1\n\n\nUnidata and XSEDE Jetstream have partnered to offer an EDEX data server in the cloud, open to the community. Select the server in the Connectivity Preferences dialog, or enter \nedex-cloud.unidata.ucar.edu\n (without \nhttp://\n before, or \n:9581/services\n after).\n\n\n\n\n\n\nDistributed Computing\n\uf0c1\n\n\nAWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. Because AWIPS was originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS can consist of a dozen servers or more, Unidata modified the package to be more applicable in the University setting. Because the AWIPS source code was hard-coded with the NWS network configuration, the early Unidata releases were stripped of operation-specific configurations and plugins, and released specifically for standalone installation. This made sense given that a single EDEX instance with a Solid State Drive (SSD) could handle most of the entire NOAAport data volume. However, with GOES-R(16) now online, and more gridded forecast models being created at finer temporal and spatial resolutions, there was a need to distribute EDEX data decoding in order to handle this firehose of data.\n\n\n\n\nRead More: \nDistributed EDEX\n\n\n\n\n\n\n\n\nSoftware Components\n\uf0c1\n\n\n\n\nEDEX\n\n\nCAVE\n\n\nLDM\n\n\nedexBridge\n\n\nQpid\n\n\nPostgreSQL\n\n\nHDF5\n\n\nPyPIES\n\n\n\n\nEDEX\n\uf0c1\n\n\nThe main server for AWIPS. Qpid sends alerts to EDEX when data stored by the LDM is ready for processing. These Qpid messages include file header information which allows EDEX to determine the appropriate data decoder to use. The default ingest server (simply named ingest) handles all data ingest other than grib messages, which are processed by a separate ingestGrib server. After decoding, EDEX writes metadata to the database via Postgres and saves the processed data in HDF5 via PyPIES. A third EDEX server, request, feeds requested data to CAVE clients. EDEX ingest and request servers are started and stopped with the commands \nedex start\n and \nedex stop\n, which runs the system scr
"text":"https://www.unidata.ucar.edu/software/awips2 The Advanced Weather Interactive Processing System (AWIPS) is a meteorological software package. It is used for decoding, displaying, and analyzing data, and was originally developed for the National Weather Service (NWS) by Raytheon. There is a division here at UCAR called the Unidata Program Center (UCP) which develops and supports a modified non-operational version of AWIPS for use in research and education by UCAR member institutions . This is released as open source software, free to download and use by anyone. AWIPS takes a unified approach to data ingest, where most data ingested into the system comes through the LDM client pulling data feeds from the Unidata IDD . Various raw data and product files (netCDF, grib, BUFR, ASCII text, gini, AREA) are decoded and stored as HDF5 files and Postgres metadata by EDEX , which serves products and data over http. Unidata supports two data visualization frameworks: CAVE (an Eclipse-built Java application which runs on Linux, Mac, and Windows), and the python-awips . Note : Our version of CAVE is a non-operational version. It does not support some features of NWS AWIPS. Warnings and alerts cannot be issued from Unidata's CAVE. Additional functionality may not be available as well.",
"text":"Unidata AWIPS source code and binaries (RPMs) are considered to be in the public domain, meaning there are no restrictions on any download, modification, or distribution in any form (original or modified). Unidata AWIPS contains no proprietery content and is therefore not subject to export controls as stated in the Master Rights licensing file.",
"text":"Unidata and XSEDE Jetstream have partnered to offer an EDEX data server in the cloud, open to the community. Select the server in the Connectivity Preferences dialog, or enter edex-cloud.unidata.ucar.edu (without http:// before, or :9581/services after).",
"text":"AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. Because AWIPS was originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS can consist of a dozen servers or more, Unidata modified the package to be more applicable in the University setting. Because the AWIPS source code was hard-coded with the NWS network configuration, the early Unidata releases were stripped of operation-specific configurations and plugins, and released specifically for standalone installation. This made sense given that a single EDEX instance with a Solid State Drive (SSD) could handle most of the entire NOAAport data volume. However, with GOES-R(16) now online, and more gridded forecast models being created at finer temporal and spatial resolutions, there was a need to distribute EDEX data decoding in order to handle this firehose of data. Read More: Distributed EDEX",
"text":"The main server for AWIPS. Qpid sends alerts to EDEX when data stored by the LDM is ready for processing. These Qpid messages include file header information which allows EDEX to determine the appropriate data decoder to use. The default ingest server (simply named ingest) handles all data ingest other than grib messages, which are processed by a separate ingestGrib server. After decoding, EDEX writes metadata to the database via Postgres and saves the processed data in HDF5 via PyPIES. A third EDEX server, request, feeds requested data to CAVE clients. EDEX ingest and request servers are started and stopped with the commands edex start and edex stop , which runs the system script /etc/rc.d/init.d/edex_camel Read More: How to Install EDEX",
"text":"Common AWIPS Visualization Environment. The data rendering and visualization tool for AWIPS. CAVE contains of a number of different data display configurations called perspectives. Perspectives used in operational forecasting environments include D2D (Display Two-Dimensional), GFE (Graphical Forecast Editor), and NCP (National Centers Perspective). CAVE is started with the command /awips2/cave/cave.sh or cave.sh Read More: How to Install CAVE",
"text":"https://www.unidata.ucar.edu/software/ldm/ The LDM (Local Data Manager), developed and supported by Unidata, is a suite of client and server programs designed for data distribution, and is the fundamental component comprising the Unidata Internet Data Distribution (IDD) system. In AWIPS, the LDM provides data feeds for grids, surface observations, upper-air profiles, satellite and radar imagery and various other meteorological datasets. The LDM writes data directly to file and alerts EDEX via Qpid when a file is available for processing. The LDM is started and stopped with the commands edex start and edex stop , which runs the commands service edex_ldm start and service edex_ldm stop",
"text":"edexBridge, invoked in the LDM configuration file /awips2/ldm/etc/ldmd.conf , is used by the LDM to post \"data available\" messaged to Qpid, which alerts the EDEX Ingest server that a file is ready for processing.",
"text":"http://qpid.apache.org Apache Qpid , the Queue Processor Interface Daemon, is the messaging system used by AWIPS to facilitate communication between services. When the LDM receives a data file to be processed, it employs edexBridge to send EDEX ingest servers a message via Qpid. When EDEX has finished decoding the file, it sends CAVE a message via Qpid that data are available for display or further processing. Qpid is started and stopped by edex start and edex stop , and is controlled by the system script /etc/rc.d/init.d/qpidd",
"text":"http://www.postgresql.org PostgreSQL , known simply as Postgres, is a relational database management system (DBMS) which handles the storage and retrieval of metadata, database tables and some decoded data. The storage and reading of EDEX metadata is handled by the Postgres DBMS. Users may query the metadata tables by using the termainal-based front-end for Postgres called psql . Postgres is started and stopped by edex start and edex stop , and is controlled by the system script /etc/rc.d/init.d/edex_postgres",
"text":"http://www.hdfgroup.org/HDF5/ Hierarchical Data Format (v.5) is the primary data storage format used by AWIPS for processed grids, satellite and radar imagery and other products. Similar to netCDF, developed and supported by Unidata, HDF5 supports multiple types of data within a single file. For example, a single HDF5 file of radar data may contain multiple volume scans of base reflectivity and base velocity as well as derived products such as composite reflectivity. The file may also contain data from multiple radars. HDF5 data is stored on the EDEX server in /awips2/edex/data/hdf5/ .",
"text":"PyPIES , Python Process Isolated Enhanced Storage, (httpd-pypies) was created for AWIPS to isolate the management of HDF5 Processed Data Storage from the EDEX processes. PyPIES manages access, i.e., reads and writes, of data in the HDF5 files. In a sense, PyPIES provides functionality similar to a DBMS (i.e PostgreSQL for metadata); all data being written to an HDF5 file is sent to PyPIES, and requests for data stored in HDF5 are processed by PyPIES. PyPIES is implemented in two parts: 1. The PyPIES manager is a Python application that runs as part of an Apache HTTP server, and handles requests to store and retrieve data. 2. The PyPIES logger is a Python process that coordinates logging. PyPIES is started and stopped by edex start and edex stop , and is controlled by the system script /etc/rc.d/init.d/httpd-pypies .",
"text":"Install CAVE\n\uf0c1\n\n\nCAVE is the \nC\nommon \nA\nWIPS \nV\nisualization \nE\nnvironment that is used for rendering and analyzing data for AWIPS. Unidata supports CAVE to work on three platforms: \nCentos (Redhat) Linux\n, \nWindows\n, and \nmacOS\n. The installer may require administrator priviledges to install and may require other system changes (environment variables, etc) as well.\n\n\n\n\nGeneral Requirements\n\uf0c1\n\n\nRegardless of what Operating System CAVE is running on, these general requirements are recommended in order for CAVE to perform optimally:\n\n\n\n\nJava 1.8\n\n\nOpenGL 2.0 Compatible Devices\n\n\nAt least 4GB RAM\n\n\nAt least 2GB Disk Space for Caching\n\n\nNVIDIA Graphics Card\n\n\nLatest NVIDIA Driver\n\n\nNote: While other graphics cards \nmay\n work, NVIDIA Quadro graphics card is recommended for full visualization capability\n\n\n\n\n\n\n\n\n\n\nLinux \n\uf0c1\n\n\nSystem Requirements\n\uf0c1\n\n\n\n\n64 bit CentOS/Red Hat 7\n\n\n\n\nDownload and Installation Instructions\n\uf0c1\n\n\n\n\nDownload the following installer: \nawips_install.sh\n \n\n\nIn a terminal, go to the download directory \n\n\nMake the installer an executable by running: \nchmod 755 awips_install.sh\n\n\nRun the installer: \nsudo ./awips_install.sh --cave\n\n\nThis will install the application in \n/awips2/cave/\n and set the local cache to \n~/caveData/\n\n\n\n\n\n\n\n\nTo run CAVE either:\n\n\n\n\nUse the terminal and type the command \ncave\n\n\nFind the application in the Linux Desktop menu: Applications > Internet > AWIPS CAVE\n\n\n\n\n\n\nWindows \n\uf0c1\n\n\nFor Windows, Unidata offers two installation options: a \nLinux Virtual Machine\n, or a \nDirect Windows Installation\n.\n\n\nCurrently, the \nvirtual machine (VM)\n is the recommended form of install for those who do not have administrative priviledges on the machine, or beginners who want a simpler installation process. \n\n\n\n\nNote: At the moment, the VM option may not render all products in CAVE (ex. RGB composites of satellite imagery)\n\n\n\n\nThe \ndirect installation method\n is recommended for those who have administrative priviledges and a little bit of experience installing more software.\n\n\nMethod 1: Linux Virtual Machine\n\uf0c1\n\n\nThis method is recommended for beginners, or those with less computer knowledge as it is a very simple installation, however at this time, some CAVE functionality may be missing (ex: rendering RGB satellite images).\n\n\nSystem Requirements\n\uf0c1\n\n\n\n\nVMWare Workstation Player\n must be installed (free software)\n\n\n\n\nDownload and Installation Instructions\n\uf0c1\n\n\n\n\nDownload the zipped file containing the virtual machine: \nunidata_cave.zip\n \n\n\nUnzip the folder by right-clicking and selecting \"Extract All\". All files will be extracted into a new folder.\n\n\nOpen VMWare Player and go to \nPlayer\n > \nFile...\n > \nOpen\n and locate the folder that was created from the downloaded zipped file. Select the file called \n\"CentOS 7 - Unidata CAVE\"\n.\n\n\nRun this new VM option. If it asks if it's been moved or copied, select \ncopied\n.\n\n\nThere will be a user in the Linux machine named \"awips\" and the password is \"awips\"\n\n\nThe root password is \"unidataAWIPS\"ifeverneeded\n\n\n\n\n\n\n\n\nOnceinsidetheVM,torunCAVEeither:\n\n\n\n\nUsethedesktopicon\n\n\nUsetheterminalandtypethecommand\ncave\n\n\nFindtheapplicationintheLinuxDesktopmenu:Applications>Internet>AWIPSCAVE\n\n\n\n\nMethod2:DirectWindowsInstall\n\uf0c1\n\n\nThismethodisrecommendedforpersonaluseandrequiresAdministrativepriviledges.ItshouldenablefullCAVEcapability,butitisabitlengthyandmighttakeabout20minutesorsotocomplete.\n\n\nSystemRequirements\n\uf0c1\n\n\n\n\n64-bitMiniconda3(4.8.2)\n\n\nPython3(comeswithMinicondainstallation)\n\n\n64-bitJavaJDK1.8(1.8_181)\n\n\n64-bitVisualC++BuildTools2015Update3(14.1)\n\n\nNumpy(1.15.1)\n\n\nJep(3.8.2)\n\n\nUserVariablePATHmusthaveminiconda3location\n\n\nUs
"text":"CAVE is the C ommon A WIPS V isualization E nvironment that is used for rendering and analyzing data for AWIPS. Unidata supports CAVE to work on three platforms: Centos (Redhat) Linux , Windows , and macOS . The installer may require administrator priviledges to install and may require other system changes (environment variables, etc) as well.",
"text":"Regardless of what Operating System CAVE is running on, these general requirements are recommended in order for CAVE to perform optimally: Java 1.8 OpenGL 2.0 Compatible Devices At least 4GB RAM At least 2GB Disk Space for Caching NVIDIA Graphics Card Latest NVIDIA Driver Note: While other graphics cards may work, NVIDIA Quadro graphics card is recommended for full visualization capability",
"text":"Download the following installer: awips_install.sh In a terminal, go to the download directory Make the installer an executable by running: chmod 755 awips_install.sh Run the installer: sudo ./awips_install.sh --cave This will install the application in /awips2/cave/ and set the local cache to ~/caveData/ To run CAVE either: Use the terminal and type the command cave Find the application in the Linux Desktop menu: Applications > Internet > AWIPS CAVE",
"text":"For Windows, Unidata offers two installation options: a Linux Virtual Machine , or a Direct Windows Installation . Currently, the virtual machine (VM) is the recommended form of install for those who do not have administrative priviledges on the machine, or beginners who want a simpler installation process. Note: At the moment, the VM option may not render all products in CAVE (ex. RGB composites of satellite imagery) The direct installation method is recommended for those who have administrative priviledges and a little bit of experience installing more software.",
"text":"This method is recommended for beginners, or those with less computer knowledge as it is a very simple installation, however at this time, some CAVE functionality may be missing (ex: rendering RGB satellite images).",
"text":"Download the zipped file containing the virtual machine: unidata_cave.zip Unzip the folder by right-clicking and selecting \"Extract All\". All files will be extracted into a new folder. Open VMWare Player and go to Player > File... > Open and locate the folder that was created from the downloaded zipped file. Select the file called \"CentOS 7 - Unidata CAVE\" . Run this new VM option. If it asks if it's been moved or copied, select copied . There will be a user in the Linux machine named \"awips\" and the password is \"awips\" The root password is \"unidataAWIPS\" if ever needed Once inside the VM, to run CAVE either: Use the desktop icon Use the terminal and type the command cave Find the application in the Linux Desktop menu: Applications > Internet > AWIPS CAVE",
"text":"This method is recommended for personal use and requires Administrative priviledges. It should enable full CAVE capability, but it is a bit lengthy and might take about 20 minutes or so to complete.",
"text":"64-bit Miniconda3 (4.8.2) Python3 (comes with Miniconda installation) 64-bit Java JDK 1.8 (1.8_181) 64-bit Visual C++ Build Tools 2015 Update 3 (14.1) Numpy (1.15.1) Jep (3.8.2) User Variable PATH must have miniconda3 location User Variables PYTHONHOME and PYTHONPATH must be defined System Variable JAVA_HOME must be defined",
"text":"Download and install 64-bit Miniconda Python 4.8.2 for Windows Allow Miniconda3 to set PATH and other environment variables Register miniconda as the default python Download and install the 64-bit Java JDK 1.8_181 (this is necessary so Jep can install properly). Select Development Tools as the installation options Make note of where it installs on your computer (the default is C:\\ProgramFiles\\Java) Set the environment variables: Access the Environment Variables window by typing \"env\" in the start bar, hitting enter, and clicking on the \"Environment Variables...\" button at the bottom of the \"System Properties\" window User Variables: PYTHONPATH and PYTHONHOME System Variable: JAVA_HOME Note: If PYTHONHOME is not set, the gridslice Python module will not be installed or available Download and install 64-bit Microsoft Visual Studio C++ Build Tools To access the page linked above you will need a Microsoft account Once at that webpage, search for \"build tools c+\" in order for the proper download to be returned Download 64-bit Visual C++ Build Tools 2015 Update 3 When running the installer, choose the Default Installation Install dependent Python packages Open a terminal by typing \"cmd\" into the start bar and hitting enter Run the following command: pip install numpy==1.15.1 jep=3.8.2 Download and install: awips-cave.msi In addition to the application directory, the MSI installer will attempt to copy the gridslice shared library to $PYTHONHOME/Dlls/ . If the $PYTHONHOME environmental variable is not defined gridslice will not be installed. You can check to see if it was installed in the Dlls directory after you have completed steps 1-3. Note: CAVE will still run without gridslice, but certain bundles which use derived parameters, such as isentropic analysis , will not load. To run CAVE, either: Type \"cave\" in the start bar and hit enter Find and run CAVE app in the file browser: C:\\Program Files\\Unidata\\AWIPS CAVE\\cave.exe",
"text":"Download and install both: awips-cave.dmg and awips-python.pkg This will install CAVE as an application and set the local cache to ~/Library/caveData Note: The awips-python.pkg is not necessarily required, and CAVE will still run without it, but any derived data such as barbs, arrows, and various grid products will not render without having jep installed (it is assumed to be in /Library/Python/2.7/site-packages/jep/) To run CAVE either: Use the System Menu Go > Applications > CAVE Type \u2318 + Spacebar and then type \"cave\", the application should appear and you can hit enter to run it",
"text":"Unidata and XSEDE Jetstream have partnered to offer a EDEX data server in the cloud, open to the Unidata university community. Select the server in the Connectivity Preferences dialog, or enter edex-cloud.unidata.ucar.edu (without http:// before, or :9581/services after).",
"text":"After connecting to an EDEX server, you will have a local directory named caveData which contains files synced from EDEX as well as a client-side cache for data and map resources. You can reset CAVE by removing the caveData directory and reconnecting to an EDEX server. Your local files have been removed, but if you are re-connecting to an EDEX server you have used before, the remote files will sync again to your local ~/caveData (bundles, colormaps, etc.). Linux: /home/<user>/caveData/ macOS: /Users/<user>/Library/caveData/ Windows: C:\\Users\\<user>\\caveData\\",
"text":"InstallEDEX\n\uf0c1\n\n\nEDEXisthe\nE\nnvironmental\nD\nata\nEx\nchangesystemthatrepresentsthebackendserverforAWIPS.EDEXisonlysupportedforLinuxsystems:CentOSandRHEL,andideally,itshouldbeonitsowndedicatedmachine.Itrequiresadministratorpriviledgestomakeroot-levelchanges.EDEXcanrunonasinglemachineorbespreadacrossmultiplemachines.Tolearnmoreaboutthatpleaselookat\nDistributedEDEX,InstallingAcrossMultipleMachines\n\n\n\n\nSystemrequirements\n\uf0c1\n\n\n\n\n64-bitCentOS/RHEL7\n\n\n16+CPUcores(eachCPUcorecanrunadecorderinparallel)\n\n\n24GBRAM\n\n\n700GB+DiskSpace\n\n\ngcc-c++package\n\n\nRun\nrpm-qa|grepgcc-c++\ntoverifyifthepackageisinstalled\n\n\nIfitisnotinstalled,run\nyuminstallgcc-c++\ntoinstallthepackage\n\n\n\n\n\n\nA\nSolidStateDrive(SSD)\nisrecommended\n\n\nASSDshouldbemountedeitherto\n/awips2\n(tocontaintheentireEDEXsystem)orto\n/awips2/edex/data/hdf5\n(tocontainthelargefilesinthedecodeddatastore).EDEXcanscaletoanysystembyadjustingtheincomingLDMdatafeedsoradjustingtheresources(CPUthreads)allocatedtoeachdatatype.\n\n\n\n\n\n\n\n\n\n\nNote:EDEXisonlysupportedfor64-bitCentOSandRHEL7OperationSystems.YoumayhaveluckwithFedoraCore12to14andScientificLinux.\n\n\n\n\n\n\nEDEXis\nnot\nsupportedinDebian,Ubuntu,SUSE,Solaris,macOS,orWindows.\n\n\n\n\n\n\nDownloadandInstallationInstructions\n\uf0c1\n\n\nAllofthesecommandshouldberunas\nroot\n\n\n1.CreateAWIPSUser\n\uf0c1\n\n\nCreateuserawipsandgroupfxalpha\n\n\ngroupaddfxalpha&&useradd-Gfxalphaawips\n\n\n\noriftheawipsaccountalreadyexists:\n\n\ngroupaddfxalpha&&usermod-Gfxalphaawips\n\n\n\n2.InstallEDEX\n\uf0c1\n\n\nDownloadtheandruntheinstaller:\nawips_install.sh\n\n\n\nwgethttps://www.unidata.ucar.edu/software/awips2/awips_install.sh\nchmod 755 awips_install.sh\nsudo ./awips_install.sh --edex\n\n\n\n\n\nawips_install.sh --edex\n will perform the following steps (it's always a good idea to review downloaded shell scripts):\n\n\n\n\nSaves the appropriate Yum repo file to \n/etc/yum.repos.d/awips2.repo\n\n\nIncreases process and file limits for the the \nawips\n account in \n/etc/security/limits.conf\n\n\nCreates \n/awips2/data_store\n if it does not exist already\n\n\nRuns \nyum groupinstall awips2-server\n\n\nAttempts to configure the EDEX hostname defined in \n/awips2/edex/bin/setup.env\n\n\nAlerts the user if the \nawips\n account does not exist (the RPMs will still install)\n\n\n\n\n\n\n3. EDEX Setup\n\uf0c1\n\n\nChange user and run edex setup:\n\n\nsudo su - awips\nsudo edex setup\n\n\n\nThe command \nedex setup\n will try to determine your fully-qualified domain name and set it in \n/awips2/edex/bin/setup.env\n. EDEX Server Administrators should double-check that the addresses and names defined in setup.env are resolvable from both inside and outside the server, and make appropriate edits to \n/etc/hosts\n if necessary.\n\n\nSetup Example\n\uf0c1\n\n\nFor example, in the XSEDE Jetstream cloud, the fully-qualified domain name defined in \n/awips2/edex/bin/setup.env\n\n\nexport EXT_ADDR=js-196-132.jetstream-cloud.org\nexport DB_ADDR=localhost\nexport DB_PORT=5432\nexport BROKER_ADDR=localhost\nexport PYPIES_SERVER=http://${EXT_ADDR}:9582\n\n\n\nThe external address needs to direct to localhost in \n/etc/hosts\n\n\n127.0.0.1 localhost localhost.localdomain js-196-132.jetstream-cloud.org\n\n\n\n4. Configure iptables\n\uf0c1\n\n\nConfigure iptables to allow TCP connections on ports 9581 and 9582 if you want to serve data to CAVE clients and the Python API.\n\n\nOpen Port 9588\n\uf0c1\n\n\nIf you are running a Registry (Data Delivery) server, you will also want to open port \n9588\n.\n\n\nTo open ports to all connections\n\uf0c1\n\n\nvi /etc/sysconfig/iptables\n\n*filter\n:INPUT ACCEPT [0:0]\n:FORWARD ACCEPT [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT\n-A INPUT -p icmp
"text":"EDEX is the E nvironmental D ata Ex change system that represents the backend server for AWIPS. EDEX is only supported for Linux systems: CentOS and RHEL, and ideally, it should be on its own dedicated machine. It requires administrator priviledges to make root-level changes. EDEX can run on a single machine or be spread across multiple machines. To learn more about that please look at Distributed EDEX, Installing Across Multiple Machines",
"text":"64-bit CentOS/RHEL 7 16+ CPU cores (each CPU core can run a decorder in parallel) 24GB RAM 700GB+ Disk Space gcc-c++ package Run rpm -qa | grep gcc-c++ to verify if the package is installed If it is not installed, run yum install gcc-c++ to install the package A Solid State Drive (SSD) is recommended A SSD should be mounted either to /awips2 (to contain the entire EDEX system) or to /awips2/edex/data/hdf5 (to contain the large files in the decoded data store). EDEX can scale to any system by adjusting the incoming LDM data feeds or adjusting the resources (CPU threads) allocated to each data type. Note: EDEX is only supported for 64-bit CentOS and RHEL 7 Operation Systems. You may have luck with Fedora Core 12 to 14 and Scientific Linux. EDEX is not supported in Debian, Ubuntu, SUSE, Solaris, macOS, or Windows.",
"text":"Create user awips and group fxalpha groupadd fxalpha && useradd -G fxalpha awips or if the awips account already exists: groupadd fxalpha && usermod -G fxalpha awips",
"text":"Download the and run the installer: awips_install.sh wget https://www.unidata.ucar.edu/software/awips2/awips_install.sh\nchmod 755 awips_install.sh\nsudo ./awips_install.sh --edex awips_install.sh --edex will perform the following steps (it's always a good idea to review downloaded shell scripts): Saves the appropriate Yum repo file to /etc/yum.repos.d/awips2.repo Increases process and file limits for the the awips account in /etc/security/limits.conf Creates /awips2/data_store if it does not exist already Runs yum groupinstall awips2-server Attempts to configure the EDEX hostname defined in /awips2/edex/bin/setup.env Alerts the user if the awips account does not exist (the RPMs will still install)",
"text":"Change user and run edex setup: sudo su - awips\nsudo edex setup The command edex setup will try to determine your fully-qualified domain name and set it in /awips2/edex/bin/setup.env . EDEX Server Administrators should double-check that the addresses and names defined in setup.env are resolvable from both inside and outside the server, and make appropriate edits to /etc/hosts if necessary.",
"text":"For example, in the XSEDE Jetstream cloud, the fully-qualified domain name defined in /awips2/edex/bin/setup.env export EXT_ADDR=js-196-132.jetstream-cloud.org\nexport DB_ADDR=localhost\nexport DB_PORT=5432\nexport BROKER_ADDR=localhost\nexport PYPIES_SERVER=http://${EXT_ADDR}:9582 The external address needs to direct to localhost in /etc/hosts 127.0.0.1 localhost localhost.localdomain js-196-132.jetstream-cloud.org",
"text":"In this example, the IP range 128.117.140.0/24 will match all 128.117.140.* addresses, while 128.117.156.0/24 will match 128.117.156.*. vi /etc/sysconfig/iptables\n\n*filter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n:EXTERNAL - [0:0]\n:EDEX - [0:0]\n-A INPUT -i lo -j ACCEPT\n-A INPUT -p icmp --icmp-type any -j ACCEPT\n-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT\n-A INPUT -s 128.117.140.0/24 -j EDEX\n-A INPUT -s 128.117.156.0/24 -j EDEX\n-A INPUT -j EXTERNAL\n-A EXTERNAL -j REJECT\n-A EDEX -m state --state NEW -p tcp --dport 22 -j ACCEPT\n-A EDEX -m state --state NEW -p tcp --dport 9581 -j ACCEPT\n-A EDEX -m state --state NEW -p tcp --dport 9582 -j ACCEPT\n#-A EDEX -m state --state NEW -p tcp --dport 9588 -j ACCEPT # for registry/dd\n-A EDEX -j REJECT\nCOMMIT",
"text":"For CentOS 7 error: Redirecting to /bin/systemctl restart iptables.service \nFailed to restart iptables.service: Unit iptables.service failed to load: No such >file or directory. The solution is: yum install iptables-services\nsystemctl enable iptables\nservice iptables restart",
"text":"edex start To manually start, stop, and restart: service edex_postgres start\nservice httpd-pypies start\nservice qpidd start\nservice edex_camel start The fifth service, edex_ldm , does not run at boot to prevent filling up disk space if EDEX is not running. ldmadmin start To start all services except the LDM (good for troubleshooting): edex start base To restart EDEX edex restart",
"text":"/etc/security/limits.conf defines the number of user processes and files (this step is automatically performed by ./awips_install.sh --edex ). Without these definitions, Qpid is known to crash during periods of high ingest. awips soft nproc 65536\nawips soft nofile 65536",
"text":"This step is no longer necessary with version LDM-6.13 or higher. The version shipped with Unidata's EDEX is higher than this cutoff. vi /etc/sysconfig/selinux\n\n# This file controls the state of SELinux on the system.\n# SELINUX= can take one of these three values:\n# enforcing - SELinux security policy is enforced.\n# permissive - SELinux prints warnings instead of enforcing.\n# disabled - No SELinux policy is loaded.\nSELINUX=disabled\n# SELINUXTYPE= can take one of these two values:\n# targeted - Targeted processes are protected,\n# mls - Multi Level Security protection.\nSELINUXTYPE=targeted Read more about selinux at redhat.com",
"text":"Though a Solid State Drive is not required, it is strongly encouraged in order to handle the amount of disk IO for real-time IDD feeds. The simplest configuration would be to mount an 500GB+ SSD to /awips2 to contain both the installed software (approx. 20GB) and the real-time data (approx. 150GB per day). The default purge rules are configured such that the processed data in /awips2 does not exceed 450GB. The raw data is located in /awips2/data_store , and is scoured every hour and should not exceed 50GB. If you want to increase EDEX data retention you should mount a large disk to /awips2/edex/data/hdf5 since this will be where the archived processed data exists, and any case studies created. Filesystem Size Used Avail Use% Mounted on\n/dev/sda1 30G 2.5G 26G 9% /\ntmpfs 28G 0 28G 0% /dev/shm\n/dev/sdc1 788G 81G 667G 11% /awips2\n/dev/sdb1 788G 41G 708G 10% /awips2/edex/data/hdf5",
"text":"EDEX installs its own version of the LDM to the directory /awips2/ldm . As with a the default LDM configuration, two files are used to control what IDD feeds are ingested:",
"text":"This file specifies an upstream LDM server to request data from, and what feeds to request: REQUEST NEXRAD3 \"./p(DHR|DPR|DSP|DTA|DAA|DVL|EET|HHC|N0Q|N0S|N0U|OHA|NVW|NTV|NST).\" idd.unidata.ucar.edu\nREQUEST FNEXRAD|IDS|DDPLUS|UNIWISC \".*\" idd.unidata.ucar.edu\nREQUEST NGRID \".*\" idd.unidata.ucar.edu\nREQUEST NOTHER \"^TIP... KNES.*\" idd.unidata.ucar.edu Read more about ldmd.conf in the LDM User Manual",
"text":"Common Problems\n\uf0c1\n\n\nWindows CAVE Start Up Error\n\uf0c1\n\n\nOne common error some users are seeing manifests itself just after selecting an EDEX server to connect to. The following error dialogs may show up:\n\n\n\n\n\n\nThese errors are actually happening because the Windows machine is using IPv6, which is not compatible with AWIPS at this time.\n\n\nTo fix the issue simply follow these steps:\n\n\n\n\nNote\n: These screenshots may vary from your system.\n\n\n\n\n1. Close all error windows and any open windows associated with CAVE.\n\n\n2. In the Windows 10 search field, search for \"control panel\".\n\n\n\n\n3. Once in the Control Panel, look for \"Network and Sharing Center\".\n\n\n\n\n\n\n4. Select the adapter for your current connection (should be either \"Ethernet\" or \"Wi-Fi\").\n\n\n\n\n5. Click on \"Properties\".\n\n\n\n\n6. Uncheck \"Internet Protocol Version 6 (TCP/IPv6)\" and select OK.\n\n\n\n\n7. Restart CAVE.",
"text":"One common error some users are seeing manifests itself just after selecting an EDEX server to connect to. The following error dialogs may show up: These errors are actually happening because the Windows machine is using IPv6, which is not compatible with AWIPS at this time. To fix the issue simply follow these steps: Note : These screenshots may vary from your system. 1. Close all error windows and any open windows associated with CAVE. 2. In the Windows 10 search field, search for \"control panel\". 3. Once in the Control Panel, look for \"Network and Sharing Center\". 4. Select the adapter for your current connection (should be either \"Ethernet\" or \"Wi-Fi\"). 5. Click on \"Properties\". 6. Uncheck \"Internet Protocol Version 6 (TCP/IPv6)\" and select OK. 7. Restart CAVE.",
"text":"D2D (Display 2-Dimensions) is the default AWIPS CAVE perspective, designed to mimmic the look and feel of the legacy AWIPS I system. System menus include CAVE , File , View , Options , and Tools . Data menus include Models , Surface , NCEP/Hydro , Upper Air , Satellite , Local Radar Stations , Radar , MRMS , and Maps . Map projection, image properties, frame control, and a few featured applications ( Warngen , Nsharp , and Browser ) make up the the primary D2D toolbar. Note : Depending on which Operating System version of CAVE there may be other application options ( PGEN , GEMPAK ).",
"text":"At bottom-right of the map window the the Resource Stack, which displays all loaded resources and map overlays, and allows for interaction and customization with the resource via a right-click menu . There are three available views of the Resource Stack, the default will show all Product Resources. The other two views are the Simple view, which shows the time, and the Map Resources. To switch between views see the Right-Click Functionality . It's important to understand that Product Resources and Map Resources are handled differently given the time-based nature of Products, compared to the static nature of maps. Selecting the Clear button will remove all Products but not remove any Map Products.",
"text":"A left click on any resource in the stack will hide the resource and turn the label gray. Clicking the name again makes the resource visible.",
"text":"The default display in the resource stack is the Product Resources. Right Click the mouse on the map background (anywhere but on the stack itself) to switch to a Simple View, which just shows the current displayed time if product data is loaded. Right Click again to show all Map Resources. Right Click again to switch back to Product Resources.",
"title":"Right-Click Background to Cycle Resource Views"
"text":"Drag the mouse over a loaded resource and hold the right mouse button until a menu appears (simply clicking the resource with the right mouse button will toggle its visibility). The hold-right-click menu allows you to control individual resource Image Properties , Change Colormaps , change resource color, width, density, and magnification, move resources up and down in the stack, as well as configure custom options with other interactive resources.",
"title":"Hold-Right-Click Resource Name for Menu"
},
{
"location":"/cave/d2d-perspective/#display-menu",
"text":"The display menu has many options which can alter the functionality in CAVE.",
"text":"Right-click hold in the view and select Two Panel Layout or Four Panel Layout to create duplicates of the current view (note that any data loaded to the view will be loaded to both displays within the view ). From this multi-pane display, hold-right-click again and you will see the Single Panel Layout option to switch back to a standard view (defaulting to the left of two, and top-left of four).",
"text":"Most data types have a right-click menu option for reading out the pixel value, displayed as multi-line text for multiple resources. This can be toggled on and off by selecting the Sample option in the Display Menu.",
"text":"The Product Browser allows users to browse a complete data inventory in a side window, organized by data type. To open the Product Browser, either select the icon in the toolbar ( ), or go to the menu: CAVE > Data Broswers > Product Broswers . Selections for Grid , Lightning , Maps , Radar , Redbook , and Satellite are available. All products loaded with the Product Browser are given default settings. Note : The Linux and Mac version also have a selection for GFE available.",
"text":"This check button enables/disables the ability to select the time interval between frames of real-time or model data. This feature has the added benefit of allowing you to view extended amounts of data (temporally) but stay within the limits of 64 frames. For example, METAR surface plots, which typically display every hour, can be set to display every three hours via the Select Valid Time and Time Resolution Dialog Box. When the Time Options check button is selected, the next product you choose to display in the Main Display Pane launches either the Select Valid Time and Time Resolution dialog box or the Select\nOffset and Tolerance dialog box. When you are loading data to an empty display and the Time Options check button is enabled, the Select Valid Time and Time Resolution dialog box opens. Valid Time: In this column of dates/times, you may choose the one that will be the first frame loaded onto the Large Display Pane. The Default option is the most recent data. Time Resolution: This column contains various time increments in which the data can be displayed. Once you make a selection, the Valid Time Column indents the exact times that will\nbe displayed. The Default resolution displays the most recent frames available. With the Time Options check button enabled for a display that already contains data, when you choose the data to be overlaid in the Main Display Pane, the Select Offset and Tolerance dialog\nbox appears, providing the following options: Offset : This column contains various time increments at intervals before, at, or after the time you selected for the first product that is displayed in the Main Display Pane. Tolerance : The options in this column refer to how strict the time matching is. \"None\" means an exact match, while \"Infinite\" will put the closest match in each frame, regardless of how\nfar off it is.",
"text":"This check button enables/disables the ability to display two images at once. Combined-image displays have been improved by removing the valid time for non-forecast products and removing the date string (time is kept) from the left side of the legend. In particular, this makes All-Tilts radar legends more usable.",
"text":"Loop Properties is another dialog box that can be opened from the Options menu or from the Loop Properties iconified button on the D2D Toolbar, or by using the Ctrl + L keyboard shortcut. The dialog allows you to adjust the forward and backward speeds, with 0 = off and 10 = maximum speed. You can set the duration of the first and last frame dwell times to between zero and 2.5 seconds. You can turn looping on or off by checking the Looping check button. There is also a Looping button located on the Toolbar that enables/disables the animation in the large display pane. Finally, you can turn looping on and increase/decrease forward speed by pressing Page Up/Page Down on your keyboard, and turn looping off with the Left or Right Arrow keys. On the toolbar, you can use the button to start/stop looping.",
"text":"The Image Properties dialog box can be opened here (in the Options menu) or by using the Image Properties iconified button on the D2D Toolbar ( ), or using using the Ctrl + I keyboard shortcut. This dialog box provides options that allow you to change the color table; adjust the brightness, contrast, and alpha of either a single image or combined images; fade between combined images; and/or interpolate the displayed data.",
"text":"You can now set the background display color on your workstation. You can also set the background display color for a single pane via mouse Button 3 (B3).",
"text":"Switching perspectives in CAVE can be found in the CAVE > Perspective menu. D2D is one of many available CAVE perspectives. By selecting the CAVE > Perspective menu you can switch into the GFE , or Localization perspective. Note : The National Centers Perspective (which is available in the Other... submenu) is available on the Linux version of CAVE. And the GFE perspective is not available on the Windows version.",
"text":"Preferences and settings for the CAVE client can be found in the CAVE > Preferences menu. Set the Localization Site and server for the workstation; configure mouse operations, change performance levels, font magnification, and text workstation hostname.",
"text":"Within the Display Properties dialog is the Load Mode option, which provides different ways to display data by manipulating previous model runs and inventories of data sets. The selected load mode is shown on the toolbar when the Load Mode menu is closed, and can also be changed by using this toolbar option as well. A description of the Load Mode options follow. Latest : Displays forecast data only from the latest model run, but also backfills at the beginning of the loop with available frames from previous runs to satisfy the requested number of\nframes. Valid time seq : Displays the most recent data and fills empty frames with previous data. For models, it provides the product from the latest possible run for every available valid time. No Backfill : Displays model data only from the most recent model run time with no backfilling to fill out a loop. Using this Load Mode prevents the mixing of old and new data. Previous run : Displays the previous model run, backfilling with frames from previous runs at the beginning of the loop to satisfy the requested number of frames. Prev valid time seq : Displays the previous model run and fills empty frames with previous model data or analyses. Prognosis loop : Shows a sequence of n-hour forecasts from successive model runs. Analysis loop : Loads a sequence of model analyses but no forecasts. dProg/dt : Selects forecasts from different model runs that all have the same valid times. This load mode is available only when there are no other products loaded in the large display\npane. Forced : Puts the latest version of a selected product in all frames without time-matching. Forecast match : Overlays a model product only when its forecast times match those of an initially loaded product. This load mode is available only when another product is already\nloaded in the large display pane. Inventory : Selecting a product when the load mode is set to Inventory brings up a Dialog Box with the available forecast and inventory times from which you can select the product you\nwant. Inventory loads into the currently displayed frame. Slot : Puts the latest version of a selected product in the currently displayed frame.",
"text":"Maps, Views, Projections\n\uf0c1\n\n\nDefault Map Scales\n\uf0c1\n\n\nThe first toolbar menu item is a dropdown menu for different geographic areas and map projections. The default view is always \nCONUS\n, which is a North Polar Steregraphic projection centered on the Continental United States.\n\n\nDefault projections and areas available in the menu\n\n\n\n\nCONUS\n\n\nN. Hemisphere\n (North Polar Stereographic)\n\n\nRegional\n (for the selected localization site)\n\n\nWFO\n (for the selected localization site)\n\n\nWorld - Mercator\n\n\nWorld - CED\n\n\nWorld - Mollweide\n\n\nGOES East Full Disk\n (Geostationary)\n\n\nGOES West Full Disk\n (Geostationary)\n\n\nRegional\n Mercator projections for\n\n\nAfrica\n\n\nAlaska\n\n\nAntarctica\n\n\nArctic\n\n\nAustralia,New Zealand\n\n\nEurope\n\n\nHawaii\n\n\nJapan\n\n\nPacific Ocean\n\n\nPuerto Rico\n\n\nSouth America\n\n\n\n\n\n\nWFO\n (Has a submenu which contains a map scale for every NWS localization site)\n\n\n\n\n\n\n\n\nNew Map Editor / View\n\uf0c1\n\n\nFile > New Map\n\n\nOpens a new map editor tab with the default projection (CONUS Polar Stereographic).\n\n\n\n\nThis can also be done by \nright-click\n on any tab and selecting \nNew Editor\n\n\n\n\n\n\nNew Projection\n\uf0c1\n\n\nFile > New Projection\n\n\nCreate a new map projection.",
"text":"The first toolbar menu item is a dropdown menu for different geographic areas and map projections. The default view is always CONUS , which is a North Polar Steregraphic projection centered on the Continental United States. Default projections and areas available in the menu CONUS N. Hemisphere (North Polar Stereographic) Regional (for the selected localization site) WFO (for the selected localization site) World - Mercator World - CED World - Mollweide GOES East Full Disk (Geostationary) GOES West Full Disk (Geostationary) Regional Mercator projections for Africa Alaska Antarctica Arctic Australia,New Zealand Europe Hawaii Japan Pacific Ocean Puerto Rico South America WFO (Has a submenu which contains a map scale for every NWS localization site)",
"text":"File > New Map Opens a new map editor tab with the default projection (CONUS Polar Stereographic). This can also be done by right-click on any tab and selecting New Editor",
"text":"Bundles and Procedures\n\uf0c1\n\n\nAWIPS contains two methods for saving and loading data resources: \nBundles\n are a simple way to save loaded resources to access in future CAVE sessions. \nProcedures\n are similar to Bundles, but can be thought of as \ngroups of bundles\n and allows the user to manage saved resources with more control.\n\n\nBundles\n\uf0c1\n\n\nFile > Load Display\n\uf0c1\n\n\nLoad a previously-saved bundle from within the AWIPS system. The Open Bundle dialog allows you to select your own saved bundles as well as those saved by other users.\n\n\nEach selected bundle will load its contents to new tabs which are named after the bundle file name (e.g. NAM_ThetaE).\n\n\n\n\nMost saved bundles will consist of a single Map Editor (tab), but with multiple tabs saved each will open again in its own Map Editor.\n\n\n\n\nFile > Save Display\n\uf0c1\n\n\nSave a product display within the AWIPS system, synching the bundle between CAVE and the EDEX server.\n\n\n\n\nFile > Manage Bundles\n\uf0c1\n\n\nSelect and remove a saved bundle under File > Manage Bundles, this will open the Delete Bundle dialog. Select the file name and click \nOK\n and then confirm deletion to remove the saved file permanently.\n\n\n\n\n\n\n\n\nLoad Bundle from Local Disk\n\uf0c1\n\n\nTo load a previously-saved display from a path within the file directory of the workstation, select \nFile > Load Dislay\n and then select the \nFile\n button on the right to browse your local directories.\n\n\n\n\n\n\nSave Bundle to Local Disk\n\uf0c1\n\n\nTo save a product display to a path within the file directory of the workstation, select \nFile > Save Display\n and then select the \nFile\n button on the right.\n\n\n\n\nProcedures\n\uf0c1\n\n\nNew Procedure\n\uf0c1\n\n\n\n\nSelect the menu \nFile > Procedures > New\n\n\nSelect \nCopy Into\n to add all loaded resources to the Procedure Stack\n\n\nSelect \nSave\n (or \nSave As\n) and then enter a name for the Procedure before clicking \nOK\n to save.\n\n\n\n\n\n\nOpen Procedure\n\uf0c1\n\n\nSimilar to creating a new Procedure, select \nFile > Procedures > Open\n, select the saved resources and click \nLoad\n to load them to CAVE.\n\n\nDelete Procedure\n\uf0c1\n\n\nFrom the menu \nFile > Procedures > Delete\n you can delete existing Procedure files in a way similar to deleting saved Bundle files.",
"text":"AWIPS contains two methods for saving and loading data resources: Bundles are a simple way to save loaded resources to access in future CAVE sessions. Procedures are similar to Bundles, but can be thought of as groups of bundles and allows the user to manage saved resources with more control.",
"text":"Load a previously-saved bundle from within the AWIPS system. The Open Bundle dialog allows you to select your own saved bundles as well as those saved by other users. Each selected bundle will load its contents to new tabs which are named after the bundle file name (e.g. NAM_ThetaE). Most saved bundles will consist of a single Map Editor (tab), but with multiple tabs saved each will open again in its own Map Editor.",
"text":"Select and remove a saved bundle under File > Manage Bundles, this will open the Delete Bundle dialog. Select the file name and click OK and then confirm deletion to remove the saved file permanently.",
"text":"To load a previously-saved display from a path within the file directory of the workstation, select File > Load Dislay and then select the File button on the right to browse your local directories.",
"text":"To save a product display to a path within the file directory of the workstation, select File > Save Display and then select the File button on the right.",
"text":"Select the menu File > Procedures > New Select Copy Into to add all loaded resources to the Procedure Stack Select Save (or Save As ) and then enter a name for the Procedure before clicking OK to save.",
"text":"Localization perspective\n\uf0c1\n\n\nLocalization Levels\n\uf0c1\n\n\nAWIPS uses a hierarchical system known as \nLocalization\n to configure many aspects of EDEX and CAVE, such as available menu items, color maps, and derived parameters. This system allows a user to override existing configurations and customize CAVE. For example, a \nUser\n-level\n localization file will supercede any similar file in a higher level (such as \nSite\n).\n\n\n\n\nThere are three \nlevels of localization\n, starting with the default \nBASE\n\n\n\n\nBASE\n - default\n\n\nSITE\n - 3-letter WFO ID (required) overrides base\n\n\nUSER\n - user-level localization overrides site and base\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nLocalization Editor\n\uf0c1\n\n\nThe Localization Perspective acts as file editor for the XML, Python, and text files which customize the look and feel of CAVE. This perspective is available in the menu \nCAVE > Perspective > Localization\n.\n\n\nUsers may copy and add files to available directories at their own \nUser\n localization version.\n\n\nExamples of things that can be accessed through the perspective include (this list is not all-inclusive):\n\n\n\n\n\n\nNCP Predefined Areas, Color Maps and Style Rules\n\n\n\n\n\n\nD2D Volume Browser Controls\n\n\n\n\n\n\nD2D Bundles - Scales (WFO, State(s), etc.)\n\n\n\n\n\n\nCAVE Map Overlays, Color Maps and Style Rules\n\n\n\n\n\n\nGFE Tools and Utilities\n\n\n\n\n\n\n\n\nThe left panel contains a directory heirarchy of CAVE files for D2D, GFE, and NCP, which can be copied and edited as \nuser\n localization files.\n\n\n\n\nThere may be several versions of each file including \nBASE\n, \nCONFIGURED\n (GFE only), \nSITE\n, and \nUSER\n. Each file version is listed separately under the actual file name.\n\n\nThe \nFile Editor\n view opens the selected configuration file in an appropriate editor. For example, a Python file is opened in a Python editor, and an XML file is opened in an XML editor.\n\n\n\n\nCustomizing CAVE Menus\n\uf0c1\n\n\nNavigate to \nD2D > Menus\n and select a submenu (e.g. \nsatellite\n). This directory lists all of the menu file contributions made by this data plugin. Most data menu directories will have an \nindex.xml\n file from which you can investigate the menu structure and make needed changes.\n\n\nSelecting a file such as \nindex.xml\n (by double clicking, or expanding) will show a sub-menu with a default localization level (typically \nBASE\n or \nCONFIGURED\n). Double-click this file to open in the file editor (you may need to click \nSource\n at the bottom of the view to see the raw XML). Right-click this file and select \nCopy To\n > \nUser (\nusername\n)\n and you will see the file localization versions update with the new copy. Select this file to edit, and override, the existing version.",
"text":"AWIPS uses a hierarchical system known as Localization to configure many aspects of EDEX and CAVE, such as available menu items, color maps, and derived parameters. This system allows a user to override existing configurations and customize CAVE. For example, a User -level localization file will supercede any similar file in a higher level (such as Site ). There are three levels of localization , starting with the default BASE BASE - default SITE - 3-letter WFO ID (required) overrides base USER - user-level localization overrides site and base",
"text":"The Localization Perspective acts as file editor for the XML, Python, and text files which customize the look and feel of CAVE. This perspective is available in the menu CAVE > Perspective > Localization . Users may copy and add files to available directories at their own User localization version. Examples of things that can be accessed through the perspective include (this list is not all-inclusive): NCP Predefined Areas, Color Maps and Style Rules D2D Volume Browser Controls D2D Bundles - Scales (WFO, State(s), etc.) CAVE Map Overlays, Color Maps and Style Rules GFE Tools and Utilities The left panel contains a directory heirarchy of CAVE files for D2D, GFE, and NCP, which can be copied and edited as user localization files. There may be several versions of each file including BASE , CONFIGURED (GFE only), SITE , and USER . Each file version is listed separately under the actual file name. The File Editor view opens the selected configuration file in an appropriate editor. For example, a Python file is opened in a Python editor, and an XML file is opened in an XML editor.",
"text":"Navigate to D2D > Menus and select a submenu (e.g. satellite ). This directory lists all of the menu file contributions made by this data plugin. Most data menu directories will have an index.xml file from which you can investigate the menu structure and make needed changes. Selecting a file such as index.xml (by double clicking, or expanding) will show a sub-menu with a default localization level (typically BASE or CONFIGURED ). Double-click this file to open in the file editor (you may need to click Source at the bottom of the view to see the raw XML). Right-click this file and select Copy To > User ( username ) and you will see the file localization versions update with the new copy. Select this file to edit, and override, the existing version.",
"text":"NSHARP, which stands for the N ational Center S ounding and H odograph A nalysis and R esearch P rogram, is an AWIPS plugin originally based on NAWIPS NSHAREP, SPCs BigSHARP sounding display tool, and the Python package SHARpy . NSHARP is available a number of ways in CAVE: From the D2D toolbar select the NSHARP icon: From the Upper Air menu select NSHARP Soundings From the Upper Air menu select a station from the RAOB menus From the Upper Air menu select NUCAPS Soundings From the Models or Tools menu select Volume Browser Make sure Sounding is selected from the menu at the top Select a source that has data (signified by a green box to the right) Select Soundings from the Fields menu Select any point from the Planes menu and an option will load in the table To create a new point go to select Tools > Points and use the right-click-hold menu to create a new point anywhere on the map Use the Load button to load data and open the NSharp display",
"text":"NSHARP has four configurations for use in different operational settings: SPC Wide - more insets and graphs at the expense of timeline/station inventory. D2D Skewt Standard - default for WFOs, larger SkewT with inventory, no Wind/Height, temperature advection, insets, or graphs. D2D Lite - Skew-T, table, and inventory only. OPC - Ocean Prediction Center display. To change the NSHARP confiuguration: Open the NSHARP(D2D) controls tab by clicking on the Nsharp toolbar ( ) icon again Click the Configure button Click Display Pane Configuration (third from the bottom) Use the dropdown to choose a configuration, apply, save, close If you would like to interactively explore the different graphical areas in NSHARP on the Web , see the NSHARP Interactive Overview .",
"text":"The Skew-T display renders a vertical profile of temperature, dew point, and wind for RAOBs and model point soundings using a Skew-T Log-P diagram. The box in the upper-left of the main display is linked to the cursor readout when over the SkewT chart. It reports the temperature, dewpoint, wind direction and speed, pressure, height AGL, and relative humidity of the trace. Skew-T is the default upper air chart in AWIPS, and can be changed to turbulence display ( T ) or an icing display ( I ). These options are available as buttons at the bottom of the NSHARP(D2D) controls tab (mentioned in NSHARP Configurations ). Use the AWIPS-2 NSHARP Interactive Overview page for more information about the Skew-T display.",
"text":"The windspeed vs height and inferred temperature advection with height plot is situated next to the SkewT to show the values at the same heights. Inferred temperature advection is from the thermal wind. Use the AWIPS-2 NSHARP Interactive Overview page for more information.",
"text":"This panel contains the hodograph display from the sounding data. The rings in the hodograph represent the wind speed in 20 knot increments. The hodograph trace uses different colors to highlight wind observations in 3 km height increments. This display also contains information such as the mean wind, Bunkers Left/Right Moving storm motion, upshear and downshear Corfidi vectors, and a user-defined motion. Use the AWIPS NSHARP Interactive Overview page for more information about the hodograph display.",
"text":"In the SPC Wide Screen Configuration there are four small insets beneath the hodograph containing storm-relative windspeed versus height, a Storm Slinky, Theta-E vs Pressure, Possible Watch Type, Thea-E vs Height, and storm-relative wind vectors. There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the four boxes. There are also two buttons PvIn and NxIn in the control tab that can be used to cycle through the previous and next inset. Use the AWIPS NSHARP Interactive Overview page for more information.",
"text":"The Table Output Displays contains five different pages of parameters ranging from parcel instability to storm relative shear to severe hazards potential. There are two buttons PtDt and NxDt in the controls tab that can be used to cycle through the previous and next tables. Use the AWIPS NSHARP Interactive Overview page for more information on the tables and a list/definition of the parameters available.",
"text":"In the SPC Wide Screen Configuration there are two graphs boxes under the insets, and they can display information on Enhanced Bulk Shear, Significant Tornado Parameter, Significant Hail Parameter (SHIP), Winter Weather, Fire Weather, Hail model (not implemented), and the Sounding Analog Retrieval System (SARS). There are buttons in the NSHARP(D2D) control button tab that toggle the six possible contents in the two boxes. Use the AWIPS NSHARP Interactive Overview page for more information.",
"text":"This section controls the inventory of the soundings that have been loaded for potential display in NSHARP. The different colors of the text represent variously that a sounding/station is being displayed, available for display, or not available for display. Use the AWIPS NSHARP Interactive Overview page for more information on how to use the sounding inventory and time line.",
"text":"WarnGen Walkthrough\n\uf0c1\n\n\nWarnGen is an AWIPS graphics application for creating and issuing warnings as is done by National Weather Service offices. In the Unidata AWIPS release it is a \nnon-operational\n forecasting tool, meaning it allows users to experiment and simulate with the drawing and text-generation tools, but \nprevents you from transmitting a generated warning upstream\n.\n\n\n\n\nIn order to select a feature it must be within your \nCAVE localization\n coverage (load \nMaps\n > \nCounty Warning Areas\n to see coverages)\n\n\n\n\nQuick Steps - Using WarnGen in Unidata AWIPS CAVE\n\uf0c1\n\n\n\n\nLoad NEXRAD Display\n from the Radar menu\n\n\nChoose a CWA with active severe weather (PAH is used in the video below)\n\n\nRe-localize\n to this site in the \nCAVE\n > \nPreferences\n > \nLocalization\n menu\n\n\nExit out of CAVE and reload (you should notice the new CWA at the top of CAVE)\n\n\nLoad radar data\n from the local radar menu \nkpah\n > \nZ + SRM8\n\n\nUse the \"period\" key in the number pad to toggle between the 0.5 Reflectivity and SRM\n\n\nClick \nWarnGen\n toolbar button or load from \nTools\n > \nWarnGen\n\n\nDrag the storm marker\n to the center of a storm feature\n\n\nStep through frames back and forth and adjust the marker to match the trajectory of the storm feature\n\n\nClick \nTrack\n in the Warngen GUI to update the polygon shape and trajectory\n\n\nFrom the WarnGen dialog select the type of warning to generate, time range, basis of the warning, and any threats (wind, hail, etc)\n\n\n\n\nClick \nCreate Text\n at the bottom of the WarnGen dialog to generate a text warning product in a new window\n\n\n\n\nNote: Since you are not \"issuing\" the warning, leave the top to rows blank (\"TTAAii\" and \"CCCC\") and Click \"Enter\" and a separate text window should open \n\n\n\n\n\n\n\n\n\n\nClick \nReset\n at the top of the WarnGen dialog to reset the storm marker at any time\n\n\n\n\nSelect \nLine of Storms\n to enable a two-pointed vector which is to be positioned parallel to a storm line\n\n\nTo \nadd another vertex\n, middle button click along the polygon\n\n\n\n\nVideo - Using WarnGen in AWIPS\n\uf0c1\n\n\nThe video below walks through creating a warning polygon and text in AWIPS. More detailed information can be found in the text below the video.\n\n\n\n\n\n\n\nLoad NEXRAD level 3 display\n\uf0c1\n\n\nSelect the menu \nRadar\n > \nNEXRAD Display\n and note coverage areas of current severe weather. We choose a CWA ID that contains some active severe weather (PAH Paducah, Kentucky, in this example).\n\n\nSelect SITE Localization\n\uf0c1\n\n\nOpen \nCAVE\n > \nPreferences\n > \nLocalization\n, select the CWA site ID (PAH) for the coverage area you want to use, followed by \nApply\n and \nOkay\n and restart CAVE. Once CAVE is restarted, you should notice the new CWA at the top of the CAVE window.\n\n\n\n\nLoad single radar data from the local radars\n\uf0c1\n\n\nClick on the local radar \nkpah\n > \nZ + SRM8\n. Use the \"period\"keyinthenumberpadtotogglebetweenthe0.5ReflectivityandSRM.\n\n\n\n\nLaunchWarnGen\n\uf0c1\n\n\nSelect\nWarnGen\nfromtheD2DToolbarorfromthe\nTools\n>\nWarnGen\nmenu.Whenstarted,thestormcentroidmarkerappearsandtheWarnGenGUIwillpopupasaseparatewindow.\n\n\n\n\nGenerateaStormMotionVector\n\uf0c1\n\n\n\n\nClickanddrag\nDragMetoStorm\ntothefeatureyouwanttotrack(WarnGenusesadottotrackasinglestormandalinetotrackalineofstorms).\n\n\nStepback3to4frames.\n\n\nDragthedottothepreviouspositionofthefeatureyoufirstmarkedtocreatethestormmotionvector.\n\n\nClickthe\nTrack\nbuttonintheWarnGenGUItoupdatethepolygonbasedoffthestormmotion.\n\n\nReviewtheproductloopandmakeadjustmentstoensurethevectorisaccurate.\n\n\n\n\n\n\nTheinitialpolygonmayhaveunhatchedareasthatwillberemovedfromthewarningduetocrossingCWAsornotmeetingareathresholdsinthecountyforinclusion.TheWarned/HatchedAreabuttonallowsyoutoprevie
"text":"WarnGen is an AWIPS graphics application for creating and issuing warnings as is done by National Weather Service offices. In the Unidata AWIPS release it is a non-operational forecasting tool, meaning it allows users to experiment and simulate with the drawing and text-generation tools, but prevents you from transmitting a generated warning upstream . In order to select a feature it must be within your CAVE localization coverage (load Maps > County Warning Areas to see coverages)",
"text":"Load NEXRAD Display from the Radar menu Choose a CWA with active severe weather (PAH is used in the video below) Re-localize to this site in the CAVE > Preferences > Localization menu Exit out of CAVE and reload (you should notice the new CWA at the top of CAVE) Load radar data from the local radar menu kpah > Z + SRM8 Use the \"period\" key in the number pad to toggle between the 0.5 Reflectivity and SRM Click WarnGen toolbar button or load from Tools > WarnGen Drag the storm marker to the center of a storm feature Step through frames back and forth and adjust the marker to match the trajectory of the storm feature Click Track in the Warngen GUI to update the polygon shape and trajectory From the WarnGen dialog select the type of warning to generate, time range, basis of the warning, and any threats (wind, hail, etc) Click Create Text at the bottom of the WarnGen dialog to generate a text warning product in a new window Note: Since you are not \"issuing\" the warning, leave the top to rows blank (\"TTAAii\" and \"CCCC\") and Click \"Enter\" and a separate text window should open Click Reset at the top of the WarnGen dialog to reset the storm marker at any time Select Line of Storms to enable a two-pointed vector which is to be positioned parallel to a storm line To add another vertex , middle button click along the polygon",
"text":"The video below walks through creating a warning polygon and text in AWIPS. More detailed information can be found in the text below the video.",
"text":"Select the menu Radar > NEXRAD Display and note coverage areas of current severe weather. We choose a CWA ID that contains some active severe weather (PAH Paducah, Kentucky, in this example).",
"text":"Open CAVE > Preferences > Localization , select the CWA site ID (PAH) for the coverage area you want to use, followed by Apply and Okay and restart CAVE. Once CAVE is restarted, you should notice the new CWA at the top of the CAVE window.",
"text":"Select WarnGen from the D2D Toolbar or from the Tools > WarnGen menu. When started, the storm centroid marker appears and the WarnGen GUI will pop up as a separate window.",
"text":"Click and drag Drag Me to Storm to the feature you want to track (WarnGen uses a dot to track a single storm and a line to track a line of storms). Step back 3 to 4 frames. Drag the dot to the previous position of the feature you first marked to create the storm motion vector. Click the Track button in the WarnGen GUI to update the polygon based off the storm motion. Review the product loop and make adjustments to ensure the vector is accurate. The initial polygon may have unhatched areas that will be removed from the warning due to crossing CWAs or not meeting area thresholds in the county for inclusion. The Warned/Hatched Area button allows you to preview the polygon shape that will be issued, so you can make further edits.",
"text":"Vertices can be moved by clicking and dragging with the mouse. The warning polygon, including stippling, will update automatically. When reshaping your warning polygon in this manner, the philosophy is to include all areas that are at risk of experiencing severe weather covered by that warning type. Effective polygons account for uncertainty over time and typically widen downstream.",
"text":"There will be some occasions where you will want to add vertices to your warning polygon. Most often, these situations will involve line warnings with bowing segments or single storm warnings where you want to account for storm motion uncertainty or multiple threat areas that may have differing storm motions. New vertices are added to the warning polygon two ways. Either by Right Mouse Button \"click and hold\" or a simple Middle Mouse Button click on the warning polygon line segment where you want to add the vertex. Vertex points are removed from the warning polygon using the same context relative menu. Instead of selecting a line segment, you select the vertex you wish to remove and then right mouse button click and hold and select remove vertex .",
"text":"Click the Reset button to clear the current polygon and vector and reset the storm centroid marker. Generate a new storm motion by moving the storm markers and select the Track button in the WarnGen GUI to draw the new polygon.",
"text":"Once you are satisfied with your polygon and have chosen your selections, click Create Text in the WarnGen GUI. Initially the AWIPS Header Block window appears. Leave the top two rows bank and click Enter for the text window to open. Using the customized settings in the WarnGen GUI, WarnGen translates the information into a text product that is displayed in a text window on the Text Display. The auto-generated text contains the storm speed and direction, the counties and cities affected by the warning/advisory, the valid times of the product, the warning/advisory body text (including any optional bullets selected in the GUI), and additional code to help our partners to efficiently process and disseminate the warning/advisory. The locked parts of the text are highlighted in blue and most of your text should not need to be edited if you configured your WarnGen window correctly. The Unidata AWIPS release is non-operational . You will be allowed to simulate the drawing and text-generation of warnings, but are prevented from transmitting a generated warning upstream Note: Edits made to product text in the editor window should be limited to items such as forecaster name/initials, call-to-action text, etc. If changes are warranted for items such as storm motion, warned counties, or Latitude/Longitude points, close the editor window and make changes using the D-2D and WarnGen graphical tools, then recreate the polygon and/or the text.",
"text":"The GOES-R decoder supports the ingest and display of NOAAport products (currently on the NIMAGE feed), Derived products (Level 2b netCDF files), and the Geostationary Lightning Mapper (GLM) products. GOES-R products are accessible in the Satellite menu. The menu is broken into sections and has submenus for each of the separate geospatial products: CONUS, Full Disk, Mesoscale Sectors, Hawaii, Alaska, and Puerto Rico. Each submenu has products for individual channels and RGB Composites , as well as submenus for channel differences and derived products . GLM data can also be found with its own submenu option a little lower down the menu and under the Surface menu. NOTE : The RGB products are not available on MacOS.",
"text":"This is found in the LDM's pqact.conf file which is located in /awips2/ldm/etc. The entries for GOES data are shown below. There are three different feeds that supply the individual channel data, derived products, and GLM data: NIMAGE, HDS and DIFAX, respectively. #\n# GOES 16/17 ABI\n#\nNIMAGE ^/data/ldm/pub/native/satellite/GOES/([^/]*)/Products/CloudAndMoistureImagery/([^/]*)/([^/]*)/([0-9]{8})/([^/]*).nc\n FILE -close -edex\n /awips2/data_store/GOES/CMI/\\5.nc4\n#\n# GOES derived products\n#\nHDS ^(IXT.99) KNES (......)\n FILE -close -edex\n /awips2/data_store/GOES/derived/KNES_\\1_\\2-(seq)\n#\n# GLM\n#\nDIFAX ^/data/cspp-geo/(EAST|WEST|GRB-R)/OR_GLM-L2-([^/]*).nc\n FILE -close -edex\n /awips2/data_store/GOES/GLM/\\1_OR_GLM-L2-\\2.nc",
"text":"All geospatial sectors have individual channel products that can be viewed. Below are samples of Channel 02 (0.64\u03bcm) for each of the sectors. These products come through the NIMAGE feed in the LDM.",
"text":"RGB Composites are also available for each sector. The three RGB products are: Icing - Ch 5, 3, 2 (1.61\u03bcm, 0.87\u03bcm, 0.64\u03bcm), Composite - Ch 2, 5, 14 #1 (0.64\u03bcm, 1.61\u03bcm, 11.20\u03bcm) and Composite #5 - Ch 2, 3, 2 (0.64\u03bcm, 0.87\u03bcm, 0.64\u03bcm). These products are generated on the fly in AWIPS using the existing channel products from EDEX. GOES RGB Imagery is NOT SUPPORTED on macOS OpenGL Shading Language limitations prevent multi-channel imagery from displaying correctly on Mac. Please use the Linux or Windows installs to view RGB products.",
"text":"Channel differences are the result of subtracting one channel from another to produce a new product. These products are generated on the fly in AWIPS using the existing channel products from EDEX. NOTE : These may not be available for all sectors. There currently 10 channel differences that are offered in CAVE: Split Window (10.3 - 12.3 \u03bcm) Split Cloud Top Phase (11.2 - 8.4 \u03bcm) Night Fog (10.3 - 2.9 \u03bcm) Day Fog (3.9 - 10.3 \u03bcm) Split Fire (2.2 - 1.6 \u03bcm) Split Ozone (9.6 - 10.3 \u03bcm) Split Water Vapor (6.19 - 7.3 \u03bcm) Split Snow (1.6 - 0.64 \u03bcm) Vegetation (0.64 - 0.87 \u03bcm) Upper Level Info (11.2 - 6.19 \u03bcm) NOTE : The rendering of these products uses the Jep package in Python, which has known to have issues installing on the MacOS and Windows versions.",
"text":"Derived products are also known as Level 2+ products and are come through the HDS feed in the LDM. There are over 20 products currently available in AWIPS. To find out some more information on some of the products please the Quick Guides compiled by CIRA. NOTE : These may not all be available for each sector. The current products offered in CAVE are: Aerosol Detection Aerosol Optical Depth Clear Sky Mask Cloud Optical Depth Cloud Particle Size Cloud Top Height Cloud Top Phase Cloud Top Pressure Cloud Top Temperature Derived CAPE Derived K-Index Derived Lifted Index Derived Showalter Index Derived Total Totals Fire Area Fire Power Fire Temperature Land Skin Temperature RR/QPE Sea Surface Temperature Snow Cover Total Precip Water Ash Cloud Height Ash Mass Load",
"text":"NASA's SPoRT MSFC Earth Science Office has contributed plugins to decode GLM level2 products, displayed as point data in CAVE. GLM data is located in the menu structure: Satellite > Geostationary Lightning Data (GLM) . Data are displayable is available for Flash , Event , and Group products. There is also additional GLM data available in the Surface > GLM - Geostationary Lightning Mapper submenus.",
"text":"Volume Browser\n\uf0c1\n\n\nThe Volume Browser provides access to numerical models, other gridded data, sounding data, and selected point data sources, such as RAOB, METAR, and Profiler. Through the Browser interface, you can choose the data source(s), field(s), plane(s), and point(s), and generate a customized list of model graphics or images for display.\n\n\nThe Volume Browser window is divided into four areas:\n\n\n\n\n\n\nThe \nMenu Bar\n along the top\n\n\nThe Data Selection Menus\n\n\nThe Product Selection List\n\n\nThe Load Buttons (Diff and Load) to load items from the Product Selection List\n\n\n\n\nEach area is then subdivided into menu components. The menu bar along the top of the Volume Browser window has dropdown lists that contain options for controlling all the various menu choices of the Volume Browser.\n\n\n\n\nVolume Browser Menu Bar\n\uf0c1\n\n\nThe dropdown menus in the Volume Browser menu bar contain options for controlling and manipulating the Volume Browser or the products chosen through the Volume Browser\n\n\n\n\nFile\n\n\nClone\n\n\nExit\n\n\n\n\n\n\nEdit\n\n\nClear All\n\n\nClear Sources\n\n\nClear Fields\n\n\nClear Panes\n\n\nSelect None\n\n\nSelect All\n\n\nFind (Ctrl+F)\n\n\n\n\n\n\nTools\n\n\nDisplay Types\n\n\nAnimation Types\n\n\n\n\n\n\nVB Tools\n\uf0c1\n\n\n\n\nBaselines\n\uf0c1\n\n\nSelecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be\nconstructed from within the Volume Browser. These baseline resources are \neditable\n.\n\n\nIf you are zoomed in over an area when you load baselines and none appear, press the middle mouse button (B3) to \"snap\" a baseline to where the mouse cursor is. The system chooses a baseline that has not been recently used. If you\nare working with a baseline, a second click with B3 will return you to the original baseline, even\nif you modified another baseline.\n\n\nPoints\n\uf0c1\n\n\nPoints are used to generate model soundings, time-height cross-sections, time series, and variable vs.\nheight plots using the Volume Browser. As with the Baselines, the locations of these Points can be\nedited in the following manner:\n\n\n\n\n\"Snapping\" an Interactive Point\n: If you are zoomed in over an area when you load Interactive\nPoints and no Points appear, click B3 to \"snap\" a Point to where the mouse cursor is positioned.\nThe system chooses a Point that has not been recently used. If you are currently working with a\nPoint, then a second B3 click will place another Point at the location of your cursor.\n\n\nDynamic Reference Map\n: When you generate a model sounding, a time-height cross-section, a\ntime series, or a variable vs. height plot, a small reference map indicating the location(s) of the\nplotted sounding(s) is provided in the upper left corner of the Main Display Pane.\n\n\n\n\nPoints may be created, deleted, hidden, and manipulated (location, name, font, and color). Points are\nnot limited in terms of number, location, or designation. Points may also be assigned to different\ngroups to facilitate their use.\n\n\nChoose By ID\n\uf0c1\n\n\nChoose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and \"Home\"forconventionallocationslikeMETARsandRAOBs(RadiosondeObservations),butitsprimaryuseisfortheWSR-88D-identifiedmesocyclonelocations.\n\n\n\n\nDisplayTypes\n\uf0c1\n\n\n\n\nPlanView(default)\n\uf0c1\n\n\nThisisthedefaultoptionfortheVolumeBrowser.FromthePlan-viewperspective,dataareplottedontohorizontalsurfaces.TheadditionaloptionsmenuthatappearsintheVolumeBrowsermenubarallowsyoutochoosewhetheryouwantthePlanviewdatatoAnimateinTimeorAnimateinSpace.\n\n\nCrossSection\n\uf0c1\n\n\nAllows
"text":"The Volume Browser provides access to numerical models, other gridded data, sounding data, and selected point data sources, such as RAOB, METAR, and Profiler. Through the Browser interface, you can choose the data source(s), field(s), plane(s), and point(s), and generate a customized list of model graphics or images for display. The Volume Browser window is divided into four areas: The Menu Bar along the top The Data Selection Menus The Product Selection List The Load Buttons (Diff and Load) to load items from the Product Selection List Each area is then subdivided into menu components. The menu bar along the top of the Volume Browser window has dropdown lists that contain options for controlling all the various menu choices of the Volume Browser.",
"text":"The dropdown menus in the Volume Browser menu bar contain options for controlling and manipulating the Volume Browser or the products chosen through the Volume Browser File Clone Exit Edit Clear All Clear Sources Clear Fields Clear Panes Select None Select All Find (Ctrl+F) Tools Display Types Animation Types",
"text":"Selecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be\nconstructed from within the Volume Browser. These baseline resources are editable . If you are zoomed in over an area when you load baselines and none appear, press the middle mouse button (B3) to \"snap\" a baseline to where the mouse cursor is. The system chooses a baseline that has not been recently used. If you\nare working with a baseline, a second click with B3 will return you to the original baseline, even\nif you modified another baseline.",
"text":"Points are used to generate model soundings, time-height cross-sections, time series, and variable vs.\nheight plots using the Volume Browser. As with the Baselines, the locations of these Points can be\nedited in the following manner: \"Snapping\" an Interactive Point : If you are zoomed in over an area when you load Interactive\nPoints and no Points appear, click B3 to \"snap\" a Point to where the mouse cursor is positioned.\nThe system chooses a Point that has not been recently used. If you are currently working with a\nPoint, then a second B3 click will place another Point at the location of your cursor. Dynamic Reference Map : When you generate a model sounding, a time-height cross-section, a\ntime series, or a variable vs. height plot, a small reference map indicating the location(s) of the\nplotted sounding(s) is provided in the upper left corner of the Main Display Pane. Points may be created, deleted, hidden, and manipulated (location, name, font, and color). Points are\nnot limited in terms of number, location, or designation. Points may also be assigned to different\ngroups to facilitate their use.",
"text":"Choose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and \"Home\" for conventional locations like METARs and RAOBs (Radiosonde Observations), but its primary use is for the WSR-88D-identified mesocyclone locations.",
"text":"This is the default option for the Volume Browser. From the Plan-view perspective, data are plotted onto horizontal surfaces. The additional options menu that appears in the Volume Browser menu bar allows you to choose whether you want the Plan view data to Animate in Time or Animate in Space.",
"text":"Allows you to view gridded data as vertical slices along specific baselines. You need to use either the Interactive Baseline Tool or the predefined latitude/longitude baselines to specify the slice you wish to see. One of the additional options menus that appear in the Volume Browser menu bar allows you to choose whether you want the cross-section data to animate in time or space, while the other options menu allows you to adjust the vertical resolution. Descriptions of these options follows. (Note that the Fields and Planes submenu labels have changed after selecting \"Cross section.\")",
"text":"Used in conjunction with the Interactive Points Tool to enable you to view a time height cross section of a full run of gridded model data for a specific location. Additional options menus in the Volume Browser menu bar allow you to choose the direction in which you want the data to be plotted, and to adjust the vertical resolution.",
"text":"Enables you to view a profile of a meteorological model field as it changes through height, which is displayed in millibars. By using the Interactive Points Tool, you can select one or more locations from which to plot the data.",
"text":"Works in conjunction with the Interactive Points Tool to enable you to generate a Skew-T chart for a specific location, no additional menus appear in the Volume Browser when the Soundings setting is chosen.",
"text":"Used in conjunction with the Interactive Points Tool to enable you to plot gridded data on a time versus data value graph for a specified point.",
"text":"Display Tools\n\uf0c1\n\n\nThe display tools are a subset of the tools available in CAVE. These programs are accessible though the \nTools\n dropdown menu.\n\n\n\n\nMany of the tools listed under the Tools menu can be placed into an \neditable state\n. Do not enable the \"Hide Legends\" feature if you want to place a tool in an editable state, because access to editability is done by clicking the center mouse button, or right-clicking over the \nProduct Legend\n.\n\n\n\n\nNote\n: To see information about some of the other options in the Tools menu, check out the \nRadar Tools\n page.\n\n\n\n\n\n\nAz/Ran Overlay\n\uf0c1\n\n\nThis tool displays a movable azimuth/range radar map overlay. The overlay is in the \"editable\" state when displayed, and can be relocated by clicking the right mouse button.\n\n\n\n\n\n\nBaselines\n\uf0c1\n\n\nSelecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be constructed from within the Volume Browser. Baselines come up editable.\n\n\n\"Snapping\" an Interactive Baseline: If you are zoomed in over an area when you load Interactive Baselines and no Baselines appear, press the right mouse button to \"snap\" a Baseline to where the mouse cursor is. The system chooses a Baseline that has not been recently used. If you are working with a Baseline, a second click with the right mouse button will return you to the original Baseline, even if you modified another Baseline in the meantime.\n\n\n\n\n\n\nChoose By ID\n\uf0c1\n\n\nChoose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and \"Home\"forconventionallocationslikeMETARsandRAOBs(RadiosondeObservations),butitsprimaryuseisfortheWSR-88D-identifiedmesocyclonelocations.YoucanalsoaccesstheChooseByIDtoolfromtheToolsmenuontheVolumeBrowser.\n\n\n\n\n\n\nDistanceBearing\n\uf0c1\n\n\nSelectingthistooldisplayssixeditablelines,eachofwhichshowstheazimuthandrangeofthelabeledendofthelinerelativetotheunlabeledendoftheline.Youcanmakethelineseditablebyclickingthecentermousebuttonoverthelegendatthelowerrightofthedisplay.Onceineditmode,alinecanbemovedasaunitand/oreitherofitsendpointscanbeadjusted.\n\n\n\n\n\n\nDistanceSpeed\n\uf0c1\n\n\nThistoolcanbeusedtodeterminethespeedanddirectionofastormoranyothermeteorologicalfeatureofinterest.SelectingDistanceSpeeddisplaysaCentroidMarkertomovetothelocationofthestormorfeatureofinterestinanytwoormoreframesofdisplayedimagery(e.g.,asatelliteorradarloop).Thesystemthendisplaysastormtrackwiththedirection(degrees)andspeed(knots)ofmovement.WhenyouselecttheDistanceSpeedoption,theDistanceSpeeddialogboxopens.\n\n\n\n\n\n\n\n\nMode\n:Youhavethefollowingselectionsfromthisoption.\n\n\n\n\n\n\nPoint\n:AradiobuttonthatallowsyoutosettheCentroidMarkerasasinglepoint.\n\n\n\n\n\n\nPolyline\n:AradiobuttonthatallowsyoutosettheCentroidMarkerasapolyline.\n\n\n\n\n\n\n\n\n\n\nLegend\n:Youhavethefollowingselectionsfromthisoption.\n\n\n\n\n\n\nTime\n:AradiobuttonthatallowsyoutodisplaytimewiththeCentroidMarker.\n\n\n\n\n\n\nSpeed\n:AradiobuttonthatallowsyoutodisplayspeedwiththeCentroidMarker.\n\n\n\n\n\n\n\n\n\n\n\n\nDistanceScale\n\uf0c1\n\n\nEnablingthisfeatureaddsascalebartothebottomrighthandofthemainD2Ddisplay.Thistoolcanbeusedtodeterminethesizeofastormoranyothermeteorologicalfeatureofinterest.\n\n\n\n\n\n\nFeatureFollowingZoom\n\uf0c1\n\n\nWhenyouzoominoverasmallareatobeabletoviewafeatureindetail,animationwilloftencausethefeaturetom
"title":"Display Tools"
},
{
"location":"/cave/d2d-tools/#display-tools",
"text":"The display tools are a subset of the tools available in CAVE. These programs are accessible though the Tools dropdown menu. Many of the tools listed under the Tools menu can be placed into an editable state . Do not enable the \"Hide Legends\" feature if you want to place a tool in an editable state, because access to editability is done by clicking the center mouse button, or right-clicking over the Product Legend . Note : To see information about some of the other options in the Tools menu, check out the Radar Tools page.",
"text":"This tool displays a movable azimuth/range radar map overlay. The overlay is in the \"editable\" state when displayed, and can be relocated by clicking the right mouse button.",
"text":"Selecting Baselines displays 10 lines, labeled A-A' to J-J', along which cross-sections can be constructed from within the Volume Browser. Baselines come up editable. \"Snapping\" an Interactive Baseline: If you are zoomed in over an area when you load Interactive Baselines and no Baselines appear, press the right mouse button to \"snap\" a Baseline to where the mouse cursor is. The system chooses a Baseline that has not been recently used. If you are working with a Baseline, a second click with the right mouse button will return you to the original Baseline, even if you modified another Baseline in the meantime.",
"text":"Choose By ID, which is a function of DMD (Digital Mesocyclone Display), is a method of selecting feature locations. The tool is used to monitor the same feature at a certain location. Without the Choose By ID tool, a monitored feature (over a period of time) could move away from its monitored location and another feature could move in its place. You can use Choose By ID to set points, baselines, and \"Home\" for conventional locations like METARs and RAOBs (Radiosonde Observations), but its primary use is for the WSR-88D-identified mesocyclone locations. You can also access the Choose By ID tool from the Tools menu on the Volume Browser.",
"text":"Selecting this tool displays six editable lines, each of which shows the azimuth and range of the labeled end of the line relative to the unlabeled end of the line. You can make the lines editable by clicking the center mouse button over the legend at the lower right of the display. Once in edit mode, a line can be moved as a unit and/or either of its end points can be adjusted.",
"text":"This tool can be used to determine the speed and direction of a storm or any other meteorological feature of interest. Selecting Distance Speed displays a Centroid Marker to move to the location of the storm or feature of interest in any two or more frames of displayed imagery (e.g., a satellite or radar loop). The system then displays a storm track with the direction (degrees) and speed (knots) of movement. When you select the Distance Speed option, the Distance Speed dialog box opens. Mode : You have the following selections from this option. Point : A radio button that allows you to set the Centroid Marker as a single point. Polyline : A radio button that allows you to set the Centroid Marker as a polyline. Legend : You have the following selections from this option. Time : A radio button that allows you to display time with the Centroid Marker. Speed : A radio button that allows you to display speed with the Centroid Marker.",
"text":"Enabling this feature adds a scalebar to the bottom right hand of the main D2D display. This tool can be used to determine the size of a storm or any other meteorological feature of interest.",
"text":"When you zoom in over a small area to be able to view a feature in detail, animation will often cause the feature to move into and then out of the field of view. This tool allows you to follow a feature of interest even when zoomed in to a small area. To use this feature, first, you need to identify the location and motion of the feature, using Distance Speed or the WarnGen tracker. Once satisfied that the tracking icon is following the feature of interest, load this tool, and the center of the zoom area will track with the Distance Speed icon. Toggling the overlay off will resume the standard zooming behavior, and toggling it back on will reinvoke the feature following zoom.",
"text":"Selecting the Home option displays a marker, which is an \"X\" with the word \"Home\" next to it.\nClicking on the Home Location Legend with the center mouse button makes the marker editable; drag the \"X\" or click with the right mouse button to change its location. When the Home Marker is displayed, use the Sample feature (click and hold to access the menu to turn on sampling) to display the range in miles and azimuth (in degrees) of the pointer location relative to the Home location.",
"text":"The Points option initially displays a circular 10-point pattern, labeled A through J on the Map display. Points are used to generate model soundings, time-height cross-sections, time series, and variable vs. height plots using the Volume Browser. As with the Baselines, the locations of these Points can be edited in the following manner: \"Snapping\" an Interactive Point : If you are zoomed in over an area when you load Interactive\nPoints and no Points appear, click the right mouse button to \"snap\" a Point to where the mouse cursor is positioned. The system chooses a Point that has not been recently used. If you are currently working with a Point, then a second right mouse button click will place another Point at the location of your cursor. Dynamic Reference Map : When you generate a model sounding, a time-height cross-section, a\ntime series, or a variable vs. height plot, a small reference map indicating the location(s) of the\nplotted sounding(s) is provided in the upper left corner of the Main Display Pane. Points may be created, deleted, hidden, and manipulated (location, name, font, and color). Points are not limited in terms of number, location, or designation. Points may also be assigned to different groups to facilitate their use. Once the Points tools have been loaded, the addition, deletion, or manipulation of Points can be accomplished in three ways: Create Point Dialog Box : The Create Point dialog box is opened by clicking and holding the right mouse button on the map (but not on any exisiting Point) and selecting the \"New Point...\" option. The Create Point dialog box opens with the Lat and Lon text boxes populated with the latitude and longiture values at the point where you had clicked the right mouse button. The latitude and longitude values can be viewed in \"Degrees : Minutes : Seconds,\" \"Degrees : Minutes,\" or \"Degrees Only\" (N and S refer to North and South; W and E refer to West and East). In the Create Point dialog box, you must : Enter the Point's name And may do any of the following: Modify the latitude and longitude values Assign the Point's color and font use Assign the Point to a group Select whether the Point is movable or hidden By default, individual Points do not have an assigned color. They inherit the color of the Interactive Points layer reflected in the Interactive Points product legend. You can change the color of the Interactive Points layer by right clicking on the Interactive Points product legend and selecting a color from the dropdown list. The selected color then changes all points not having an assigned color to the new color. Points can be assigned to \" <No Group> \" which will organize them in the root location containing the group names when accessed by the Edit Points dialog box (see below). Edit Point Dialog Box : The Edit Point dialog box is opened by clicking and holding the right mouse button on a Point on the map and selecting the \"Edit Point...\" option. The latitude and longitude values can be viewed in \"Degrees : Minutes : Seconds,\" \"Degrees : Minutes,\" or \"Degrees Only\" (N and S refer to North and South; W and E refer to West and East). Besides the option of selecting the Edit Points dialog box, you also have the option\nof selecting \"Hide Point,\" \"Delete Point,\" or \"Move Point.\" Once hidden, the Point can be\nunhidden using the Points List dialog box, where you would uncheck the checkbox under\nthe \"Hidden\" column adjacent to the Point that was hidden (see below). If \"Delete Point\" is\nselected, a pop-up opens to confirm whether you want to delete the Point. Selecting the\n\"Move Point\" option moves the Point to wherever you place the cursor on the map. Points List Dialog Box : The Points List dialog box is opened by clicking and holding the right mouse button on the Interactive Points product legend and selecting the \"Edit Points...\"option.ThePointsListdialogboxlistsalltheavailablegroupsandPoints.Groupscanbeexpandedto\nreviewthelistofPointsassignedto
"text":"The Put home cursor tool provides an easy way to locate a METAR observation station, a city and\nstate, or a latitude/longitude coordinate. For Canada and Mexico, only the METAR observation stations and latitude/longitude coordinates are accessible. When you select Put home cursor from the Tools dropdown menu, the Home marker X is displayed and the Put Home Cursor dialog box opens. You can use the Home marker, as previously described in the Home Tool, and the new Home location\n(station, city/state, or latitude/longitude) is identified in the Put Home Cursor dialog box.\nAnother way to use this tool is to type in the station, city and state, or latitude and longitude, and select Go, or hit Enter on the keypad, to move the Home marker to the specified location. The new location's nearest METAR site, city and state, and latitude and longitude appear in the Put Home Cursor dialog box. The Put Home Cursor dialog box contains the following options. Location Selection : There are three ways to find a desired location. Once you choose the\nStation, City/State, or Lat/Lon radio button, an Entry Box is activated next to the respective label within the Put Home Cursor dialog box. Enter the desired location information. Go : This menu button initiates the search for the desired station, city/state, or latitude/longitude. The Home marker jumps to the newly specified location.",
"text":"The Range Rings Tool displays adjustable range rings around locations of interest to your local office. When you select Range Rings from the Tools dropdown menu, the Range Rings legend appears in the Main Display Pane. The tool comes up editable, and the rangeRing dialog box opens. (Clicking the middle mouse button over the legend toggles tool editability and closes/opens the rangeRing dialog box.) Within this dialog box, you can toggle on/off any of the target locations using the square selectors. Adjust the size of the radii (in nautical miles) by typing a new value in the entry boxes associated with each location and pressing the Apply button. You can also add labels at the center of the range ring and/or at any of the radial distances using the Labels Options menu associated with each location. Using the Movable Rings, you can add a new location at a specific point by using the Interactive Points Tool, or by typing in latitude/longitude coordinates. There is no practical limit on the number of new locations you can add to the display. The list of locations is pre-set but can be customized at a field site.",
"text":"By typing a date, as well as the latitude and longitude of a location into the Sunrise/Sunset Tool dialog box, you can obtain the time (for any time zone) of sunrise and sunset, as well as the total length of daylight for that date. Additional features include the ability to calculate the sunrise/sunset in a different hemisphere, and the azimuthal angles, relative to true north, of the sunrise and sunset.",
"text":"Selecting this option brings up a Text Display window that behaves in the same way as a window on the Text Workstation , except that the scripts menu is disabled.",
"text":"Selecting the Time Of Arrival / Lead Time option displays a tracking line from a feature's initial starting point in a past frame to its final position in the current frame. Once the final position is set, an Arrival Point is displayed. You can drag this point anywhere along the line to get the Time Of Arrival / Lead Time and Distance. You can also change the Mode from Point to Circular Front or Polyline anywhere along the line to better represent the feature(s).",
"text":"This tool converts the units of the first column into differing units of the second column. The units are grouped into temperature, speed, distance, time, and atmospheric pressure. First, simply type the number and select the units of the value you wish to convert in the firstcolumn entry box. Then in the second column, select the desired units to which you want the original value converted. The new value will appear in the second column entry box.",
"text":"By selecting one of the \"Text\" buttons, a text window opens up. In National Weather Service operations, the text workstation is used to edit new warning text as well as look up past warnings, METARs, and TAFs. This functionality is disabled in the Unidata AWIPS version.",
"text":"The radar tools are a subset of the tools available in CAVE. These programs are accessible though the Tools dropdown menu, and in individual site radar menus.",
"text":"A velocity (V) display from the radar shows only the radial component of the wind, so the indicated speed depends on the direction of the wind and the azimuth (direction) from the radar. Consider, for example, a north wind. Straight north of the radar, the full speed of the wind will be seen on the V product. As one moves around to the east of the radar, the radial component gets smaller, eventually reaching zero straight east of the radar. If the wind direction is known, then the actual wind speed can be computed by dividing the observed radial speed by the cosine of the angle between the radar radial and the actual direction. The EAV tool allows you to provide that angle and use the sampling function of the display to show the actual wind speed.",
"text":"The Radar Display Controls dialog box is derived from the Radar Tools submenu and provides options that control the appearance of the Storm Track Information (STI), the Hail Index (HI), the Tornado Vortex Signature (TVS), the Digital Mesocyclone Display (DMD) products, the Microburst Alert (MBA) products, the Storm Relative Motion (SRM), and the SAILS products. The Radar Display Controls dialog box options are described below. Note : Our version of CAVE may not have all the products that these options are applicable to. The Radar Display Controls dialog box is divided into eight sections: STI , HI , TVS , DMD/MD/TVS , DMD , MBA , SRM , and SAILS . Each section has the following options:",
"text":"This section has options to adjust the appearance of the STI graphic product. Number of storms to show : This slider bar lets you choose the maximum number of storms (0 to 100) you wish to display on the STI product. The default value is 20 storms. Type of track to show : This options menu allows you to choose the type of storm track that you want displayed.",
"text":"This portion of the Radar Display Controls dialog box contains options that alter the appearance of the HI radar graphic product. You can set the low and high algorithm thresholds of the Probability of Hail (POH) and the Probability of Severe Hail (POSH). Storms that meet the low POH threshold are indicated by small open triangles, while small solid triangles mark those that meet the high POH threshold. Similarly, large open triangles or solid triangles are plotted for the POSH low and high thresholds, respectively. Low hail probability (POH) : The storms that meet or exceed the threshold are indicated by small open triangles. The default setting is 30. Low severe hail probability (POSH) : The storms that meet or exceed the threshold are indicated by large open triangles. The default setting is 30. High hail probability : The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50. High severe hail probability : The storms that meet or exceed the threshold are indicated by small solid triangles. The default setting is 50.",
"text":"There is one option in this section of the Radar Display Controls dialog box. Show elevated TVS : This toggle button lets you control the appearance of the elevated TVS radar graphic product.",
"text":"There is one option in this section of the Radar Display Controls dialog box. Show extrapolated features : With this option, you can choose whether to show the time-extrapolated features using DMD, MD, or TVS.",
"text":"Minimum feature strength : A mesocyclone clutter filter which specifies the minimum 3D strength rank use to display a mesocyclone (default is 5). Show overlapping Mesos : Toggles whether to show overlapping mesocyclones. Type of track to show : This dropdown has option available for whether to display past and/or forcast tracks.",
"text":"The first three options in the SRM section allow you to choose where you want to derive the storm motion from. Storm Motion from WarnGen Track : Selecting this option will display the storm motion from a WarnGen Track. Average Storm Motion from STI : Selecting this option will display the average storm motion from from the storm track information (STI). Custom Storm Motion : Selecting this option allows you to specify a custom storm motion with the selections below. Direction : This slider allows you to choose the direction (in degrees??) of the storm motion. Speed : This slider allows you to specify the speed (in mph??) of the storm motion.",
"text":"Enable SAILS Frame Coordinator : Enabled (default) : keyboard shortcuts change where tilting up from 0.5 degree SAILS tilt will step to the next higher tilt (similar to GR2 Analyst) and Ctrl right arrow will step to the most recent tilt available for any elevation angle. Disabled : keyboard shortcuts change where tilting up from 0.5 degree SAILS tilt will not go anywhere (old confusing behavior) and Ctrl right arrow will step to the most recent time of the current tilt.",
"text":"This tool is used in conjunction with Doppler velocity data to calculate the velocity difference (or \"shear\") of the data directly under the end points. As with the Baselines, this feature comes up editable and the end points can be dragged to specific gates of velocity data. When in place, the speed difference (kts), distance between end points (nautical miles), shear (s-1), and distance from radar (Nmi) are automatically plotted next to the end points and in the upper left corner of the Main Display Pane. A positive shear value indicates cyclonic shear, while a negative value indicates anticyclonic shear. If either end point is not directly over velocity data, the phrase \"no data\" is reported for the shear value. This tool is also useful in determining gate-to-gate shear. Simply place the two end points directly over adjacent gates of velocity data. \"Snapping\" VR Shear : If you are zoomed in over an area when you load VR - Shear, and the VR - Shear Baseline does not appear, click the right mouse button to \"snap\" the Baseline to where the mouse cursor is located. VR - Shear in 4 Panel : You can use the VR - Shear Tool when the large display is in 4 panel\nmode. The VR - Shear overlay is loaded in different colors for each panel. There are actually\nfour copies of the program running, and each behaves independently. This means that you can\nget accurate readings in any one of the four panels \u2014 one VR - Shear panel is editable at a time. To activate, click the center mouse button on the VR - Shear legend in the desired panel and position the query line to the echoes of interest.",
"text":"Editing Menus\n\uf0c1\n\n\nAny of the menus in the menubar can be customized in the \nLocalization Perspective\n. \n\n\n\n\nModifying Menus\n\uf0c1\n\n\nOnce in the \nLocalization Perspective\n, menus can be modified by going to the \nD2D\n > \nMenus\n directory in the File Browser. Here there are submenus for different data types and menu structures. Usually the \nindex.xml\n file found in these submenus is the \nmaster\n file which the actual menu is based off of. This file can reference other xml files and you may have to modify these \nchild\n xml files to get the results you are looking for. \n\n\nIn order to modify any file, you must right click on it and \nselect \nCopy To > USER (my-username)\n. Then you may open this copy and begin to modify it. Once this process has been completed and a change has been made and saved, CAVE will need to be restarted and opened in the D2D perspective to see the change.\n\n\nThis example covers how to add a new menu entry to an existing menu.\n\n\n\n\nSwitch to the \nLocalization Perspective\n\n\nFind the \ngrid\n folder under \nD2D\n > \nMenus\n\n\nDouble-click to expand \nindex.xml\n\n\nRight-click to \nBASE (common_static)\n and select \nCopy To...\n, then select \nUSER\n level\n\n\n\n\nDouble-click \nUSER\n to open the editor and copy an existing include tag, and update the \nmodelName\n (this must match an existing product found in the Product Browser) and the \nmenuName\n (this can be anything)\n\n\n<include installTo=\"menu:models\" fileName=\"menus/grid/allFamilies.xml\">\n <substitute key=\"modelName\" value=\"GEFS\" />\n <substitute key=\"menuName\" value=\"GEFS\" />\n <substitute key=\"frameCount\" value=\"41\" />\n <substitute key=\"TP\" value=\"TP\"/>\n</include>\n\n\n\n\n\n\n\nOnce this is completed, save the file and restart CAVE\n\n\n\n\nNavigate to the \nModels\n menu and you should see a new entry with \nGEFS\n\n\n\n\n\n\nRemoving Menus\n\uf0c1\n\n\nThis example covers how to remove a menu (in this case \nMRMS\n) from D2D:\n\n\n\n\nSwitch to the \nLocalization Perspective\n\n\nFind the \nmrms\n folder under \nD2D\n > \nMenus\n\n\nDouble-click to expand \nindex.xml\n\n\nRight-click \nBASE\n and select \nCopy To...\n, then select \nUSER\n level\n\n\nRight-click refresh the \nmrms\n entry\n\n\n\n\nDouble click \nUSER\n to open the editor and change\n\n\n<menuContributionFile>\n <include installTo=\"menu:mrms?after=MRMS_MENU_START\" fileName=\"menus/mrms/mrms.xml\"/>\n</menuContributionFile>\n\n\n\nto \n\n\n<menuContributionFile>\n</menuContributionFile>\n\n\n\n\n\n\n\nWith this completed, you can now restart CAVE and will not see the MRMS menu anymore. Repeat this example for other product menus, such as \nradar\n, \nupperair\n, \ntools\n, etc., to further customize D2D data menus for any level of localization.",
"text":"Once in the Localization Perspective , menus can be modified by going to the D2D > Menus directory in the File Browser. Here there are submenus for different data types and menu structures. Usually the index.xml file found in these submenus is the master file which the actual menu is based off of. This file can reference other xml files and you may have to modify these child xml files to get the results you are looking for. In order to modify any file, you must right click on it and \nselect Copy To > USER (my-username) . Then you may open this copy and begin to modify it. Once this process has been completed and a change has been made and saved, CAVE will need to be restarted and opened in the D2D perspective to see the change. This example covers how to add a new menu entry to an existing menu. Switch to the Localization Perspective Find the grid folder under D2D > Menus Double-click to expand index.xml Right-click to BASE (common_static) and select Copy To... , then select USER level Double-click USER to open the editor and copy an existing include tag, and update the modelName (this must match an existing product found in the Product Browser) and the menuName (this can be anything) <include installTo=\"menu:models\" fileName=\"menus/grid/allFamilies.xml\">\n <substitute key=\"modelName\" value=\"GEFS\" />\n <substitute key=\"menuName\" value=\"GEFS\" />\n <substitute key=\"frameCount\" value=\"41\" />\n <substitute key=\"TP\" value=\"TP\"/>\n</include> Once this is completed, save the file and restart CAVE Navigate to the Models menu and you should see a new entry with GEFS",
"text":"This example covers how to remove a menu (in this case MRMS ) from D2D: Switch to the Localization Perspective Find the mrms folder under D2D > Menus Double-click to expand index.xml Right-click BASE and select Copy To... , then select USER level Right-click refresh the mrms entry Double click USER to open the editor and change <menuContributionFile>\n <include installTo=\"menu:mrms?after=MRMS_MENU_START\" fileName=\"menus/mrms/mrms.xml\"/>\n</menuContributionFile> to <menuContributionFile>\n</menuContributionFile> With this completed, you can now restart CAVE and will not see the MRMS menu anymore. Repeat this example for other product menus, such as radar , upperair , tools , etc., to further customize D2D data menus for any level of localization.",
"text":"Change Localization\n\uf0c1\n\n\nLocalization Preferences\n\uf0c1\n\n\nThe default localization site for Unidata AWIPS is OAK (Omaha, Nebraska, where the Raytheon team is located). When you are prompted to connect to an EDEX server, you can change the WFO ID as well. \n\n\n\n\nSince release 16.1.4, CAVE users can switch the localization site to any valid NWS WFO from \nCAVE > Preferences > Localization\n, where edits can be made to both the site ID and EDEX server name. Click \nRestart\n after changes are applied.\n\nThis window also has the option to \nPrompt for settings on startup\n, which if checked, would ask for the EDEX Server and Site location every time CAVE is started (this can be useful if you are used to switching between servers and/or sites).\n\n\n\n\nChange the site (example shows TBW Tampa Bay) and click \nApply\n or \nOK\n and confirm the popup dialog, which informs you that you must \nrestart\n CAVE for the changes to take effect.",
"text":"The default localization site for Unidata AWIPS is OAK (Omaha, Nebraska, where the Raytheon team is located). When you are prompted to connect to an EDEX server, you can change the WFO ID as well. Since release 16.1.4, CAVE users can switch the localization site to any valid NWS WFO from CAVE > Preferences > Localization , where edits can be made to both the site ID and EDEX server name. Click Restart after changes are applied. \nThis window also has the option to Prompt for settings on startup , which if checked, would ask for the EDEX Server and Site location every time CAVE is started (this can be useful if you are used to switching between servers and/or sites). Change the site (example shows TBW Tampa Bay) and click Apply or OK and confirm the popup dialog, which informs you that you must restart CAVE for the changes to take effect.",
"text":"Import/Export\n\uf0c1\n\n\nExport Images/GIFs\n\uf0c1\n\n\nThe D2D screen can be exported as a PNG image as well as an animated GIF using the \nFile > Export > Image\n menu option. This captures the current state of the screen, and allows you to set animation options (frame number, dwell time, etc) for exporting GIFs. If you choose to animate, you will either need to rename the destination file to have the \n.gif\n extension, or CAVE will pop up a dialog when you go to save, asking you to confirm that you want to output a GIF.\n\n\n\n\nNote\n: This functionality \ndoes not\n currently work on \nMac OS\n because it implements OGL libraries which are not compatible on Mac.\n\n\n\n\n\n\n\n\nExport KML\n\uf0c1\n\n\nThe \nExport\n submenu also includes a \nKML\n option (\nFile > Export > KML\n), which allows users to save D2D displays or GFE grids in the KML (Keyhole Markup Language) file format. When zipped (compressed), the KML file format forms a KMZ file, which can be used in applications such as Google Earth.\n\n\n\n\nThe KML dialog box includes options to select frames to export. This includes exporting all frames,\nthe current/displayed frame, a range of frames, and, in GFE, the selected time range as highlighted in\nthe Grid Manager. Additional options are available for selection under the \"Other Options\" section:\n\n\n\n\n\n\nExport Hidden\n: When selected, all displayed and hidden products listed in the Product Legend section of the Main Display Pane will be exported.\n\n\n\n\n\n\nExport Maps\n: When selected, all enabled maps displayed within the Main Display Pane will be\nexported.\n\n\n\n\n\n\nShade Earth\n: When selected, a shaded background is applied to the exported product. If loaded in Google Earth, the earth will be overlaid with a black backdrop, and data will be displayed as it would in D2D with a black background.\n\n\n\n\n\n\nShow Background Tiles\n: When selected, data (such as plot data) will display on top of black\ntiles when loaded in Google Earth.\n\n\n\n\n\n\n\n\nCAVE Import Formats\n\uf0c1\n\n\nCAVE supported the following geo-referenced data files. CAVE can import the following through formats through the \nFile > Import\n menu.\n\n\n\n\nBackground...\n\n\nImage...\n\n\nBCD File\n\n\nGeoTIFF\n\n\nLPI File\n\n\nSPI File\n \n\n\nDisplays\n\n\n\n\n\n\n\n\nCAVE Export Formats\n\uf0c1\n\n\nCAVE can export to the following through the \nFile > Export\n menu.\n\n\n\n\nImage\n\n\nPrint Screen\n\n\nKML\n\n\nEditor Display...\n\n\nPerspective Displays...",
"text":"The D2D screen can be exported as a PNG image as well as an animated GIF using the File > Export > Image menu option. This captures the current state of the screen, and allows you to set animation options (frame number, dwell time, etc) for exporting GIFs. If you choose to animate, you will either need to rename the destination file to have the .gif extension, or CAVE will pop up a dialog when you go to save, asking you to confirm that you want to output a GIF. Note : This functionality does not currently work on Mac OS because it implements OGL libraries which are not compatible on Mac.",
"text":"The Export submenu also includes a KML option ( File > Export > KML ), which allows users to save D2D displays or GFE grids in the KML (Keyhole Markup Language) file format. When zipped (compressed), the KML file format forms a KMZ file, which can be used in applications such as Google Earth. The KML dialog box includes options to select frames to export. This includes exporting all frames,\nthe current/displayed frame, a range of frames, and, in GFE, the selected time range as highlighted in\nthe Grid Manager. Additional options are available for selection under the \"Other Options\" section: Export Hidden : When selected, all displayed and hidden products listed in the Product Legend section of the Main Display Pane will be exported. Export Maps : When selected, all enabled maps displayed within the Main Display Pane will be\nexported. Shade Earth : When selected, a shaded background is applied to the exported product. If loaded in Google Earth, the earth will be overlaid with a black backdrop, and data will be displayed as it would in D2D with a black background. Show Background Tiles : When selected, data (such as plot data) will display on top of black\ntiles when loaded in Google Earth.",
"text":"CAVE supported the following geo-referenced data files. CAVE can import the following through formats through the File > Import menu. Background... Image... BCD File GeoTIFF LPI File SPI File Displays",
"text":"EDEX Basic Commands\n\uf0c1\n\n\nUnidata's EDEX install also comes with a \nsimple \nedex\n program\n that can help execute basic EDEX utilities. The most basic of the commands are the following:\n\n\nTo start all EDEX services:\n\n\nedex start\n\n\n\nTo stop all EDEX services:\n\n\nedex stop\n\n\n\n\n\nService and Boot Settings\n\uf0c1\n\n\nThese commands will start and stop five EDEX service files installed into \n/etc/init.d/\n, four of which are run on boot:\n\n\nservice postgres start\nservice httpd-pypies start\nservice qpidd start\nservice edex_camel start\n\n\n\nThe fifth, \nedex_ldm\n, does \nnot run at boot\n to prevent filling up disk space if EDEX is not running:\n\n\nservice edex_ldm start\n\n\n\nAll of these services are started and stopped by the single program: \nedex\n as mentioned above.\n\n\nLDM Troubleshooting\n\uf0c1\n\n\nIf the EDEX machine is shut down abruptly, when restarted, it should start up the processes mentioned \nabove\n. If \nsudo service edex_ldm start\n does not start up LDM smoothly, please try these steps:\n\n\n\n\nAll of the following commands should be run as user \nawips\n and the \nservice\n commands may need to be run with \nsudo\n.\n\n\n\n\n\n\n\n\nRun \nsudo service edex_ldm start\n or \nldmadmin start\n and recieve this message:\n\n\nldmadmin start\n\nstart_ldm(): PID-file \"/awips2/ldm/ldmd.pid\" exists. Verify that all \nis well and then execute \"ldmadmin clean\" to remove the PID-file.\n\n\n\n\n\n\n\nRun \nldmadmin clean\n and \nsudo service edex_ldm start\n and receive this error:\n\n\nldmadmin clean\nsudo service edex_ldm start\n\nChecking the product-queue...\nThe writer-counter of the product-queue isn't zero. Either a process\nhas the product-queue open for writing or the queue might be corrupt.\nTerminate the process and recheck or use\n pqcat -l- -s -q /awips2/ldm/var/queues/ldm.pq && pqcheck -F -q\n /awips2/ldm/var/queues/ldm.pq\nto validate the queue and set the writer-counter to zero.\nLDM not started\n\n\n\n\n\n\n\nTo resolve the above, run:\n\n\npqcat -l- -s -q /awips2/ldm/var/queues/ldm.pq && pqcheck -F -q /awips2/ldm/var/queues/ldm.pq \nldmadmin delqueue\nldmadmin mkqueue\nsudo service edex_ldm start\n\n\n\n\n\n\n\n\n\nEDEX Commands\n\uf0c1\n\n\nUnidata's version of EDEX installs with a helpful \nedex\n script that can be used for basic EDEX tasks.\n\n\nedex start\n\uf0c1\n\n\nedex start\n\nStarting EDEX PostgreSQL: [ OK ]\nStarting httpd: [ OK ]\nStarting QPID [ OK ]\nStarting EDEX Camel (request): \nStarting EDEX Camel (ingest): \nStarting EDEX Camel (ingestGrib): \nStarting AWIPS LDM:The product-queue is OK.\n...\n\n\n\nedex start base\n\uf0c1\n\n\nTo start all EDEX services \nexcept\n the LDM:\n\n\nedex start base\n\n\n\n\n\nedex stop\n\uf0c1\n\n\nedex stop\n\nStopping EDEX Camel (request): \nStopping EDEX Camel (ingest): \nStopping EDEX Camel (ingestGrib): \nStopping QPID [ OK ]\nStopping httpd: [ OK ]\nStopping EDEX PostgreSQL: [ OK ]\nStopping AWIPS LDM:Stopping the LDM server...\n...\n\n\n\n\n\nedex setup\n\uf0c1\n\n\nedex setup\n\n[edex] EDEX IP and Hostname Setup\n Checking /awips2/database/data/pg_hba.conf [OK]\n Checking /awips2/edex/bin/setup.env [OK]\n\n[edit] Hostname edex.unidata.ucar.edu added to /awips2/ldm/etc/ldmd.conf\n[done]\n\n\n\nThis command configures and/or confirms that the EDEX hostname and IP address definitions exist (\nedex setup\n is run by \nedex start\n).\n\n\n\n\nNote\n: If your EDEX server is running but you see the message \n\"Connectivity Error: Unable to validate localization preferences\"\ninCAVE,itmaymeanthatthedomainnamedefinedin\n/awips2/edex/bin/setup.env\ncannotberesolvedfrom\noutside\ntheserver.Somemachineshavedifferent\ninternally-resolved\nand\nexternally-resolved\ndomainnames(cloud-basedespecially
"text":"Unidata's EDEX install also comes with a simple edex program that can help execute basic EDEX utilities. The most basic of the commands are the following: To start all EDEX services: edex start To stop all EDEX services: edex stop",
"text":"These commands will start and stop five EDEX service files installed into /etc/init.d/ , four of which are run on boot: service postgres start\nservice httpd-pypies start\nservice qpidd start\nservice edex_camel start The fifth, edex_ldm , does not run at boot to prevent filling up disk space if EDEX is not running: service edex_ldm start All of these services are started and stopped by the single program: edex as mentioned above.",
"text":"If the EDEX machine is shut down abruptly, when restarted, it should start up the processes mentioned above . If sudo service edex_ldm start does not start up LDM smoothly, please try these steps: All of the following commands should be run as user awips and the service commands may need to be run with sudo . Run sudo service edex_ldm start or ldmadmin start and recieve this message: ldmadmin start\n\nstart_ldm(): PID-file \"/awips2/ldm/ldmd.pid\" exists. Verify that all \nis well and then execute \"ldmadmin clean\" to remove the PID-file. Run ldmadmin clean and sudo service edex_ldm start and receive this error: ldmadmin clean\nsudo service edex_ldm start\n\nChecking the product-queue...\nThe writer-counter of the product-queue isn't zero. Either a process\nhas the product-queue open for writing or the queue might be corrupt.\nTerminate the process and recheck or use\n pqcat -l- -s -q /awips2/ldm/var/queues/ldm.pq && pqcheck -F -q\n /awips2/ldm/var/queues/ldm.pq\nto validate the queue and set the writer-counter to zero.\nLDM not started To resolve the above, run: pqcat -l- -s -q /awips2/ldm/var/queues/ldm.pq && pqcheck -F -q /awips2/ldm/var/queues/ldm.pq \nldmadmin delqueue\nldmadmin mkqueue\nsudo service edex_ldm start",
"title":"LDM Troubleshooting"
},
{
"location":"/install/start-edex/#edex-commands",
"text":"Unidata's version of EDEX installs with a helpful edex script that can be used for basic EDEX tasks.",
"text":"edex setup\n\n[edex] EDEX IP and Hostname Setup\n Checking /awips2/database/data/pg_hba.conf [OK]\n Checking /awips2/edex/bin/setup.env [OK]\n\n[edit] Hostname edex.unidata.ucar.edu added to /awips2/ldm/etc/ldmd.conf\n[done] This command configures and/or confirms that the EDEX hostname and IP address definitions exist ( edex setup is run by edex start ). Note : If your EDEX server is running but you see the message \"Connectivity Error: Unable to validate localization preferences\" in CAVE, it may mean that the domain name defined in /awips2/edex/bin/setup.env can not be resolved from outside the server. Some machines have different internally-resolved and externally-resolved domain names (cloud-based especially). The name defined in setup.env must be externally-resolvable .",
"text":"Shows a list of the the Qpid message queue to monitor data ingest (messages in vs messages out, i.e. decoded): [centos@js-156-89 ~]$ edex qpid\nQueues\n queue dur excl msg msgIn msgOut bytes bytesIn bytesOut cons bind\n ================================================================================================\n external.dropbox Y Y 11 1.26m 1.26m 621 79.6m 79.6m 5 1\n Ingest.Radar Y Y 4 589k 589k 184 27.1m 27.1m 5 1\n Ingest.GribDecode Y Y 0 370k 370k 0 103m 103m 11 1\n Ingest.GribSplit Y Y 2 361k 361k 201 31.9m 31.9m 5 1\n Ingest.modelsounding Y Y 0 100k 100k 0 6.54m 6.54m 1 1\n Ingest.Text Y Y 0 97.8k 97.8k 0 5.25m 5.25m 2 1\n Ingest.GOESR Y Y 0 83.4k 83.4k 0 6.92m 6.92m 2 1\n Ingest.obs Y Y 0 46.2k 46.2k 0 2.40m 2.40m 1 1\n Grid.PostProcess Y Y 0 20.2k 20.2k 0 6.68m 6.68m 1 1\n Ingest.sfcobs Y Y 0 10.5k 10.5k 0 577k 577k 1 1\n Ingest.goessounding Y Y 0 6.68k 6.68k 0 427k 427k 1 1\n Ingest.Glm Y Y 0 5.61k 5.61k 0 581k 581k 1 1\n Ingest.aww Y Y 0 3.32k 3.32k 0 182k 182k 1 1",
"text":"To see a list of clients connecting to your EDEX server, use the edex users [YYYYMMDD] command, where [YYYYMMDD] is the optional date string. edex users\n\n -- EDEX Users 20160826 --\nuser@101.253.20.225\nuser@192.168.1.67\nawips@0.0.0.0\nawips@sdsmt.edu",
"text":"To view any stuck purge jobs in PortgreSQL (a rare but serious problem if your disk fills up). The solution to this is to run edex purge reset .",
"text":"EDEX Settings\n\uf0c1\n\n\nPlugin Configuration\n\uf0c1\n\n\nThe directory \n/awips2/edex/conf/resources\n contains configuration text files for specific plugins, which allow for user-defined values which are read by AWIPS plugins on EDEX start:\n\n\nacarssounding.properties\nautobldsrv.properties\ncom.raytheon.edex.plugin.gfe.properties\ncom.raytheon.edex.text.properties\ncom.raytheon.uf.common.registry.ebxml.properties\ncom.raytheon.uf.edex.archive.cron.properties\ncom.raytheon.uf.edex.database.properties\ncom.raytheon.uf.edex.registry.ebxml.properties\ndistribution.properties\nedex-localization-http.properties\nedex-requestsrv.properties\nedex-uengine.properties\neventBus.properties\nftp.properties\ngoesr.properties\ngrib.properties\nmaintenance.properties\nproxy.properties\npurge.properties\nquartz.properties\nradar.properties\nstats.properties\ntextdbsrv.properties\nwarning.properties\n\n\n\nLook at \npurge.properties\n for example:\n\n\n# Master switch to enable and disable purging\npurge.enabled=true\n\n# Interval at which the purge job kicks off\npurge.cron=0+0/15+*+*+*+?\n\n# Interval at which the outgoing files are purged\npurge.outgoing.cron=0+30+*+*+*+?\n\n# Interval at which the logs are purged\npurge.logs.cron=0+30+0+*+*+?\n\n# Interval at which hdf5 orphans are purged\npurge.orphan.period=24h\n\n# Number of days older than the earliest known data to delete.\npurge.orphan.buffer=7\n...\n\n\n\nIn \ngrib.properties\n, \ngoesr.properties\n, and \nradar.properties\n you can adjust the number of decoder threads for each plugin.\n\n\ncat radar.properties\n\n# Number threads for radar products ingested from the SBN\nradar-decode.sbn.threads=5\n\n\n\n\n\nIngest Modes\n\uf0c1\n\n\nBy default, EDEX starts three \"modes\": \ningest\n, \ningestGrib\n, and \nrequest\n (each as its own JVM).\n\n\nThe file \n/awips2/edex/conf/modes/modes.xml\n contains all available mode definitions, including some specific modes for Hydro Server Applications, ebXML Registries, Data Delivery, and more.\n\n\nEDEX services are registered through spring, and by including or excluding specific spring files (usually by datatype plugin name) we can finely customize EDEX startup. \n\n\nIn \n/awips2/edex/conf/modes/modes.xml\n there are a number of unused plugin decoders excluded because the data are not available outside of the SBN:\n\n\n...\n<mode name=\"ingest\">\n<exclude>.*request.*</exclude>\n<exclude>edex-security.xml</exclude>\n<exclude>ebxml.*\\.xml</exclude>\n<exclude>grib-decode.xml</exclude>\n<exclude>grid-staticdata-process.xml</exclude>\n<exclude>.*(dpa|taf|nctext).*</exclude>\n<exclude>webservices.xml</exclude>\n<exclude>.*datadelivery.*</exclude>\n<exclude>.*bandwidth.*</exclude>\n<exclude>.*sbn-simulator.*</exclude>\n<exclude>hydrodualpol-ingest.xml</exclude>\n<exclude>grid-metadata.xml</exclude>\n<exclude>.*ogc.*</exclude>\n<exclude>obs-ingest-metarshef.xml</exclude>\n<exclude>ffmp-ingest.xml</exclude>\n<exclude>scan-ingest.xml</exclude>\n<exclude>cwat-ingest.xml</exclude>\n<exclude>fog-ingest.xml</exclude>\n<exclude>vil-ingest.xml</exclude>\n<exclude>preciprate-ingest.xml</exclude>\n<exclude>qpf-ingest.xml</exclude>\n<exclude>fssobs-ingest.xml</exclude>\n<exclude>cpgsrv-spring.xml</exclude>\n</mode>\n...\n\n\n\nInthisexample,request,ebXML,gribplugins,OGCandotherpluginsareexcludedbecausetheyareincludedintheirownmode/JVM.\n\n\n\n\nNote\n:TAFandNCTEXTpluginsaredisabledhereduetoperformanceissues.\n\n\n\n\n\n\nJVMMemory\n\uf0c1\n\n\nThedirectory\n/awips2/edex/etc/\ncontainsfileswhichdefinetheamountofmemoryusedforeachofthethreeEDEXJVMs(ingest,ingestGrib,request):\n\n\nls-al/awips2/edex/etc/\n-rw-r--r--1awipsfxalpha1287Jul2418:41centralRegistry.sh\n-rw-r--r--1awipsfxalpha1155Jul2418:42default.sh\n-rw-r--r--1awipsfxalpha1956Jul2418:41ingestGrib.sh\n-rw-r--
"text":"The directory /awips2/edex/conf/resources contains configuration text files for specific plugins, which allow for user-defined values which are read by AWIPS plugins on EDEX start: acarssounding.properties\nautobldsrv.properties\ncom.raytheon.edex.plugin.gfe.properties\ncom.raytheon.edex.text.properties\ncom.raytheon.uf.common.registry.ebxml.properties\ncom.raytheon.uf.edex.archive.cron.properties\ncom.raytheon.uf.edex.database.properties\ncom.raytheon.uf.edex.registry.ebxml.properties\ndistribution.properties\nedex-localization-http.properties\nedex-requestsrv.properties\nedex-uengine.properties\neventBus.properties\nftp.properties\ngoesr.properties\ngrib.properties\nmaintenance.properties\nproxy.properties\npurge.properties\nquartz.properties\nradar.properties\nstats.properties\ntextdbsrv.properties\nwarning.properties Look at purge.properties for example: # Master switch to enable and disable purging\npurge.enabled=true\n\n# Interval at which the purge job kicks off\npurge.cron=0+0/15+*+*+*+?\n\n# Interval at which the outgoing files are purged\npurge.outgoing.cron=0+30+*+*+*+?\n\n# Interval at which the logs are purged\npurge.logs.cron=0+30+0+*+*+?\n\n# Interval at which hdf5 orphans are purged\npurge.orphan.period=24h\n\n# Number of days older than the earliest known data to delete.\npurge.orphan.buffer=7\n... In grib.properties , goesr.properties , and radar.properties you can adjust the number of decoder threads for each plugin. cat radar.properties\n\n# Number threads for radar products ingested from the SBN\nradar-decode.sbn.threads=5",
"text":"By default, EDEX starts three \"modes\": ingest , ingestGrib , and request (each as its own JVM). The file /awips2/edex/conf/modes/modes.xml contains all available mode definitions, including some specific modes for Hydro Server Applications, ebXML Registries, Data Delivery, and more. EDEX services are registered through spring, and by including or excluding specific spring files (usually by datatype plugin name) we can finely customize EDEX startup. In /awips2/edex/conf/modes/modes.xml there are a number of unused plugin decoders excluded because the data are not available outside of the SBN: ...\n<mode name=\"ingest\">\n <exclude>.*request.*</exclude>\n <exclude>edex-security.xml</exclude>\n <exclude>ebxml.*\\.xml</exclude>\n <exclude>grib-decode.xml</exclude>\n <exclude>grid-staticdata-process.xml</exclude>\n <exclude>.*(dpa|taf|nctext).*</exclude>\n <exclude>webservices.xml</exclude>\n <exclude>.*datadelivery.*</exclude>\n <exclude>.*bandwidth.*</exclude>\n <exclude>.*sbn-simulator.*</exclude>\n <exclude>hydrodualpol-ingest.xml</exclude>\n <exclude>grid-metadata.xml</exclude>\n <exclude>.*ogc.*</exclude>\n <exclude>obs-ingest-metarshef.xml</exclude>\n <exclude>ffmp-ingest.xml</exclude>\n <exclude>scan-ingest.xml</exclude>\n <exclude>cwat-ingest.xml</exclude>\n <exclude>fog-ingest.xml</exclude>\n <exclude>vil-ingest.xml</exclude>\n <exclude>preciprate-ingest.xml</exclude>\n <exclude>qpf-ingest.xml</exclude>\n <exclude>fssobs-ingest.xml</exclude>\n <exclude>cpgsrv-spring.xml</exclude>\n</mode>\n... In this example, request, ebXML, grib plugins, OGC and other plugins are excluded because they are included in their own mode/JVM. Note : TAF and NCTEXT plugins are disabled here due to performance issues.",
"text":"Distributed EDEX\n\uf0c1\n\n\nAWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. While originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS could consist of a dozen servers or more, the early Unidata releases were stripped of operations-specific configurations and plugins, and released as a standalone server. This worked, since (at the time) a single EDEX instance with an attached SSD could handle most of NOAAport. However, with GOES-R(16) coming online in 2017, and more gridded forecast models being created at finer temporal and spatial resolutions, there is now a need to distribute the data decoding across multiple machines to handle this firehose of data.\n\n\n\n\nUnidata's Current EDEX Server\n\uf0c1\n\n\nCurrently, with our specific EDEX server we use a Database/Request instance that also decodes and ingests a good portion of the data. It handles all data requests from CAVE users, as well as the majority of the decoding and ingesting for data feeds coming down on the LDM. The \nradar\n data has been specifically exluded (from the decoding and ingest) and it has its own \nIngest/Decode Server\n which is explained in more detail below.\n\n\nFor our EDEX we have designated an instance of the ingest/decoding server to be dedicated to handling the radar data. Our \nRadar-EDEX\n recieves and decodes all radar down from the LDM and then stores it back on our main \nDatabase/Request EDEX\n in the form of HDF5 data files and PostgreSQL metadata.\n\n\n\n\nExample Installation\n\uf0c1\n\n\nThis walkthrough will install different EDEX components on two machines in the XSEDE Jetstream Cloud, the first is used to \nstore and serve\n while the second is used to \ningest and decode\n data.\n\n\n\n\n\n\nDatabase/Request Server\n\uf0c1\n\n\nFor this example, this server will be referred to by the IP address \n10.0.0.9\n.\n\n\n1. Install\n\uf0c1\n\n\ngroupadd fxalpha && useradd -G fxalpha awips\nmkdir /awips2\nwget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/awips2.repo\nyum clean all\nyum groupinstall awips2-database\n\n\n\n2. IPtables Config\n\uf0c1\n\n\nIt is required that ports 5432 and 5672 be open for the specific IP addresses of outside EDEX ingest servers. It is \nnot recommended\n that you leave port 5432 open to all connections (since the default awips database password is known, and is not meant as a security measure). Further, it \nis recommended\n that you change the default postgres awips user password (which then requires a reconfiguration of every remote EDEX ingest server in order to connect to this database/request server).\n\n\nvi /etc/sysconfig/iptables\n\n*filter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n:EXTERNAL - [0:0]\n:EDEX - [0:0]\n-A INPUT -i lo -j ACCEPT\n-A INPUT -p icmp --icmp-type any -j ACCEPT\n-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 9581 -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 9582 -j ACCEPT\n-A INPUT -s 10.0.0.7 -j EDEX\n-A INPUT -j EXTERNAL\n-A EXTERNAL -j REJECT\n-A EDEX -m state --state NEW -p tcp --dport 5432 -j ACCEPT\n-A EDEX -m state --state NEW -p tcp --dport 5672 -j ACCEPT\n-A EDEX -j REJECT\nCOMMIT\n\n\n\nNote the line \n-A INPUT -s 10.0.0.7 -j EDEX\n as well as the following \n-A EDEX ...\n rules for ports 5432 (PostgreSQL) and 5672 (PyPIES/HDF5). \n\n\n\n\nThe two ports left open to all connections (9581,9582) in addition to default port 22 are for outside CAVE client connections\n\n\n\n\n3. Database Config\n\uf0c1\n\n\nIn the file \n/awips2/database/data/pg_hba.conf\n you define remote connections for all postgres tables with as \n<IP address>/32\n, after the block of IPv4 local connections:\n\n\nvi /awips2/database/data/pg_hba.conf\n\n# \"local\"isforUnixdomainsocketconnectionsonly\nlocalallall
"text":"AWIPS makes use of service-oriented architecture to request, process, and serve real-time meteorological data. While originally developed for use on internal NWS forecast office networks, where operational installations of AWIPS could consist of a dozen servers or more, the early Unidata releases were stripped of operations-specific configurations and plugins, and released as a standalone server. This worked, since (at the time) a single EDEX instance with an attached SSD could handle most of NOAAport. However, with GOES-R(16) coming online in 2017, and more gridded forecast models being created at finer temporal and spatial resolutions, there is now a need to distribute the data decoding across multiple machines to handle this firehose of data.",
"text":"Currently, with our specific EDEX server we use a Database/Request instance that also decodes and ingests a good portion of the data. It handles all data requests from CAVE users, as well as the majority of the decoding and ingesting for data feeds coming down on the LDM. The radar data has been specifically exluded (from the decoding and ingest) and it has its own Ingest/Decode Server which is explained in more detail below. For our EDEX we have designated an instance of the ingest/decoding server to be dedicated to handling the radar data. Our Radar-EDEX recieves and decodes all radar down from the LDM and then stores it back on our main Database/Request EDEX in the form of HDF5 data files and PostgreSQL metadata.",
"text":"This walkthrough will install different EDEX components on two machines in the XSEDE Jetstream Cloud, the first is used to store and serve while the second is used to ingest and decode data.",
"text":"It is required that ports 5432 and 5672 be open for the specific IP addresses of outside EDEX ingest servers. It is not recommended that you leave port 5432 open to all connections (since the default awips database password is known, and is not meant as a security measure). Further, it is recommended that you change the default postgres awips user password (which then requires a reconfiguration of every remote EDEX ingest server in order to connect to this database/request server). vi /etc/sysconfig/iptables\n\n*filter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n:EXTERNAL - [0:0]\n:EDEX - [0:0]\n-A INPUT -i lo -j ACCEPT\n-A INPUT -p icmp --icmp-type any -j ACCEPT\n-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 9581 -j ACCEPT\n-A INPUT -m state --state NEW -m tcp -p tcp --dport 9582 -j ACCEPT\n-A INPUT -s 10.0.0.7 -j EDEX\n-A INPUT -j EXTERNAL\n-A EXTERNAL -j REJECT\n-A EDEX -m state --state NEW -p tcp --dport 5432 -j ACCEPT\n-A EDEX -m state --state NEW -p tcp --dport 5672 -j ACCEPT\n-A EDEX -j REJECT\nCOMMIT Note the line -A INPUT -s 10.0.0.7 -j EDEX as well as the following -A EDEX ... rules for ports 5432 (PostgreSQL) and 5672 (PyPIES/HDF5). The two ports left open to all connections (9581,9582) in addition to default port 22 are for outside CAVE client connections",
"text":"In the file /awips2/database/data/pg_hba.conf you define remote connections for all postgres tables with as <IP address>/32 , after the block of IPv4 local connections: vi /awips2/database/data/pg_hba.conf\n\n# \"local\" is for Unix domain socket connections only\nlocal all all trust\nhostssl all all 10.0.0.7/32 cert clientcert=1\nhostssl all all 162.0.0.0/8 cert clientcert=1\nhostssl all all 127.0.0.1/32 cert clientcert=1\n# IPv6 local connections:\nhostssl all all ::1/128 cert clientcert=1\nhostnossl all all ::1/128 md5",
"text":"edex start database This will start PostgreSQL, httpd-pypies, Qpid, and the EDEX Request JVM (and will not start the LDM or the EDEX Ingest and IngestGrib JVMs)",
"text":"The command edex will show which services are running, and for a Database/Request server, will not include the LDM, EDEXingest, or EDEXgrib: edex\n\n[edex status]\npostgres :: running :: pid 571\npypies :: running :: pid 639\nqpid :: running :: pid 674\nEDEXingest :: not running\nEDEXgrib :: not running\nEDEXrequest :: running :: pid 987 1029 23792 Since this Database/Request server is not running the main edexIngest JVM, we won't see anything from edex log , instead watch the Request Server with the command edex log request Confirm that EDEX Request connects to PostgreSQL! With the above edex log request , ensure that the log progresses past this point : Spring-enabled Plugins:\n-----------------------\nacars-common, acars-common-dataaccess, acarssounding-common, activetable-common,\nactivetable-request, airep-common, airep-common-dataaccess, airmet-common, \natcf-common, atcf-request, auth-request, awipstools-request, aww-common...\n\nJAXB context for PersistencePathKeySet inited in: 5ms\nINFO 20:21:09,134 5584 [EDEXMain] Reflections: Reflections took 436 ms to scan 258 urls, producing 31 keys and 3637 values\nFound 499 db classes in 720 ms If the log stops at the Found db classes... line, that means EDEX is not connecting to PostgreSQL - double-check DB_ADDR in /awips2/edex/bin/setup.env",
"text":"vi /awips2/edex/bin/setup.env Here you should redefine DB_ADDR and PYPIES_SERVER to point to the Database/Request server (10.0.0.9) export EDEX_SERVER=10.0.0.7\n\n# postgres connection\nexport DB_ADDR=10.0.0.9\nexport DB_PORT=5432\n\n# pypies hdf5 connection\nexport PYPIES_SERVER=http://10.0.0.9:9582\n\n# qpid connection\nexport BROKER_ADDR=${EDEX_SERVER} Notice that EDEX_SERVER and BROKER_ADDR (qpid) should remain defined as the localhost IP address (10.0.0.7)",
"text":"edex start ingest This will start Qpid and the EDEX Ingest and IngestGrib JVMs (and not start PostgreSQL, httpd-pypies, or the EDEX Request JVM)",
"text":"Watch the edex JVM log with the command edex log Confirm that EDEX connects to PostgreSQL! With the above edex log , ensure that the log progresses past this point : Spring-enabled Plugins:\n-----------------------\nacars-common, acars-common-dataaccess, acarssounding-common, activetable-common,\nactivetable-ingest, airep-common, airep-common-dataaccess, airmet-common, \natcf-common, atcf-ingest, aww-common...\n\nJAXB context for PersistencePathKeySet inited in: 5ms\nINFO 20:21:09,134 5584 [EDEXMain] Reflections: Reflections took 436 ms to scan 258 urls, producing 31 keys and 3637 values\nFound 499 db classes in 720 ms If the log stops at the Found db classes... line, that means EDEX is not connecting to the remote PostgreSQL instance - double-check DB_ADDR in /awips2/edex/bin/setup.env You can manually check remote PostgreSQL connectivity on any EDEX Ingest server from the command line: su - awips\npsql -U awips -h <remote IP address> -p 5432 metadata Where the default passwd is awips and is defined in files in /awips2/edex/conf/db/hibernateConfig/",
"text":"Be mindful of what IP address and hostnames are used in /awips2/edex/bin/setup.env and /awips2/database/data/pg_hba.conf , and that they are resolvable from the command line. Consult or edit /etc/hosts as needed. You can install multiple awips2-ingest servers, each decoding a different dataset or feed, all pointing to the same Database/Request server ( DB_ADDR and PYPIES_SERVER in /awips2/edex/bin/setup.env ): Every EDEX Ingest IP address must be allowed in both iptables and pg_hba.conf as shown above .",
"text":"Docker EDEX\n\uf0c1\n\n\nProject home: \nhttps://github.com/Unidata/edex-docker\n\n\n\n\nEDEX can be run inside a docker container, which allows you to process data into an AWIPS system without requiring accessing and altering the machine's native CentOS installation and configuration.\n\n\nThe \nEDEX Docker Image\n is built on CentOS 7 and contains the latest Unidata AWIPS release (18.1.1). \n\n\nThis container is an \ningest-only\n install, meaning there is \nno database or request server\n. This example requires a Database/Request server be configured for you to access remotely. See the \nDistributed EDEX\n document for more. \n\n\n\n\nDownload and Install Docker\n\uf0c1\n\n\nDownload and install Docker and Docker Compose:\n\n\n\n\nDocker for CentOS 7 Linux\n\n\nDocker for Mac\n\n\nDocker for Windows\n\n\ndocker-compose\n (it should be bundled with Docker by default on Mac and Windows)\n\n\n\n\n\n\nRun the EDEX Ingest Container\n\uf0c1\n\n\nClone the source repository:\n\n\ngit clone https://github.com/Unidata/edex-docker.git\ncd edex-docker\n\n\n\nRun the container with docker-compose:\n\n\ndocker-compose up -d edex-ingest\n\n\n\nConfirm the container is running:\n\n\ndocker ps -a\n\n\n\nEnter the container:\n\n\ndocker exec -it edex-ingest bash\n\n\n\nStop the container:\n\n\ndocker-compose stop\n\n\n\nDelete the container (keep the image):\n\n\ndocker-compose rm -f\n\n\n\nRun commands inside the container, such as:\n\n\ndocker exec edex-ingest edex\n\n\n\nwhich should return something like:\n\n\n[edex status]\n qpid :: running :: pid 22474\n EDEXingest :: running :: pid 21860 31513\n EDEXgrib :: not running\n ldmadmin :: running :: pid 22483\n\n edex (status|start|stop|setup|log|purge|qpid|users)\n\n\n\nTo update to the latest version and restart:\n\n\ndocker pull unidata/edex-ingest:latest\ndocker-compose stop\ndocker-compose up -d edex-ingest\n\n\n\n\n\nConfiguration and Customization\n\uf0c1\n\n\nThe file \ndocker-compose.yml\n defines files to mount to the container and which ports to open:\n\n\nedex-ingest:\n image: unidata/edex-ingest:latest\n container_name: edex-ingest\n volumes:\n - ./etc/ldmd.conf:/awips2/ldm/etc/ldmd.conf\n - ./etc/pqact.conf:/awips2/ldm/etc/pqact.conf\n - ./bin/setup.env:/awips2/edex/bin/setup.env\n - ./bin/runedex.sh:/awips2/edex/bin/runedex.sh\n ports:\n - \"388:388\"\n ulimits:\n nofile:\n soft: 1024\n hard: 1024\n\n\n\n\n\nMounted Files\n\uf0c1\n\n\netc/ldmd.conf\n\uf0c1\n\n\nDefines which data feeds to receive. By default there is only one active request line (\nREQUEST IDS|DDPLUS \".*\" idd.unidata.ucar.edu\n) to not overwhelm small EDEX containers ingesting large volumes of radar and gridded data files. Any updates to the file \netc/ldmd.conf\n will be read the next time you restart the container.\n\n\netc/pqact.conf\n\uf0c1\n\n\nDefines how products are processed and where they are written to on the filesystem. This is the full set of pattern actions used in Unidata AWIPS, and generally you do not need to edit this file. Instead control which data feeds are requested in \nldmd.conf\n (above).\n\n\nbin/setup.env\n\uf0c1\n\n\nDefines the remote EDEX Database/Request server:\n\n\n### EDEX localization related variables ###\nexport AW_SITE_IDENTIFIER=OAX\nexport EXT_ADDR=js-157-198.jetstream-cloud.org\n\n\n\n\n\nNote\n: \nEXT_ADDR\n must be set to an allowed EDEX Database/Request Server. In this example we are using a JetStream Cloud instance, which controls our \nedex-ingest\n access with IPtables, SSL certificates, and PostgreSQL pg_hba.conf rules. This server will not allow outside connections, you must change this to point to an appropriate server. \n\n\n\n\nbin/runedex.sh\n\uf0c1\n\n\nThe default script run when the container is started, acts as a sort-of service manager for EDEX and the LDM (see \nENTRYPOINT [\"/awips2/edex/bin/runedex.sh\"]\n in \nDockerfile.edex\n), essentially:\n\n\n/awips2/qpid/bin/qpid-wrapper &\n/awips2/edex/bin/start.sh -noConsole ingest &\nldmadmin mkqueue\nldmadmin start",
"text":"Project home: https://github.com/Unidata/edex-docker EDEX can be run inside a docker container, which allows you to process data into an AWIPS system without requiring accessing and altering the machine's native CentOS installation and configuration. The EDEX Docker Image is built on CentOS 7 and contains the latest Unidata AWIPS release (18.1.1). This container is an ingest-only install, meaning there is no database or request server . This example requires a Database/Request server be configured for you to access remotely. See the Distributed EDEX document for more.",
"text":"Download and install Docker and Docker Compose: Docker for CentOS 7 Linux Docker for Mac Docker for Windows docker-compose (it should be bundled with Docker by default on Mac and Windows)",
"text":"Clone the source repository: git clone https://github.com/Unidata/edex-docker.git\ncd edex-docker Run the container with docker-compose: docker-compose up -d edex-ingest Confirm the container is running: docker ps -a Enter the container: docker exec -it edex-ingest bash Stop the container: docker-compose stop Delete the container (keep the image): docker-compose rm -f Run commands inside the container, such as: docker exec edex-ingest edex which should return something like: [edex status]\n qpid :: running :: pid 22474\n EDEXingest :: running :: pid 21860 31513\n EDEXgrib :: not running\n ldmadmin :: running :: pid 22483\n\n edex (status|start|stop|setup|log|purge|qpid|users) To update to the latest version and restart: docker pull unidata/edex-ingest:latest\ndocker-compose stop\ndocker-compose up -d edex-ingest",
"text":"Defines which data feeds to receive. By default there is only one active request line ( REQUEST IDS|DDPLUS \".*\" idd.unidata.ucar.edu ) to not overwhelm small EDEX containers ingesting large volumes of radar and gridded data files. Any updates to the file etc/ldmd.conf will be read the next time you restart the container.",
"text":"Defines how products are processed and where they are written to on the filesystem. This is the full set of pattern actions used in Unidata AWIPS, and generally you do not need to edit this file. Instead control which data feeds are requested in ldmd.conf (above).",
"text":"Defines the remote EDEX Database/Request server: ### EDEX localization related variables ###\nexport AW_SITE_IDENTIFIER=OAX\nexport EXT_ADDR=js-157-198.jetstream-cloud.org Note : EXT_ADDR must be set to an allowed EDEX Database/Request Server. In this example we are using a JetStream Cloud instance, which controls our edex-ingest access with IPtables, SSL certificates, and PostgreSQL pg_hba.conf rules. This server will not allow outside connections, you must change this to point to an appropriate server.",
"text":"The default script run when the container is started, acts as a sort-of service manager for EDEX and the LDM (see ENTRYPOINT [\"/awips2/edex/bin/runedex.sh\"] in Dockerfile.edex ), essentially: /awips2/qpid/bin/qpid-wrapper &\n/awips2/edex/bin/start.sh -noConsole ingest &\nldmadmin mkqueue\nldmadmin start",
"text":"LDM Feeds\n\uf0c1\n\n\nDefault LDM Feeds for EDEX\n\uf0c1\n\n\nData feeds are defined by the \nldmd.conf\n file in \n/awips2/ldm/etc/ldmd.conf\n. The default feeds that come \"turned on\" with our EDEX are the following:\n\n\nREQUEST NEXRAD3 \"./p(DHR|DPR|DSP|DTA|DAA|HHC|N.P|N.Q|N.S|N.U|OHA).\" idd.unidata.ucar.edu\nREQUEST FNEXRAD|IDS|DDPLUS \".*\" idd.unidata.ucar.edu\nREQUEST UNIWISC|NIMAGE \".*\" idd.unidata.ucar.edu # AREA/GINI\nREQUEST EXP \"WwWind\" idd.unidata.ucar.edu # ESPL/PSD Profilers\nREQUEST DIFAX \"GLM\" idd.unidata.ucar.edu # GOES GLM\nREQUEST EXP \".*\" lead.unidata.ucar.edu # GOES ABI netCDF4 (full sector)\nREQUEST NGRID \".*\" idd.unidata.ucar.edu\nREQUEST HDS \".*\" idd.unidata.ucar.edu\nREQUEST CONDUIT \"nam\" idd.unidata.ucar.edu # NAM12\nREQUEST CONDUIT \"pgrb2\" idd.unidata.ucar.edu # GFS0p25\n\n\n\n\n\nOptional LDM Feeds\n\uf0c1\n\n\nSome additional feeds are included but commented out using '#'. To activate the feed, simply remove the #, save the file, and \nrestart the LDM\n.\n\n\nFNMOC and CMC models\n\uf0c1\n\n\nREQUEST FNMOC \".*\" idd.unidata.ucar.edu\nREQUEST CMC \".*\" idd.unidata.ucar.edu\n\n\n\nLightning (restricted to educational use with rebroadcasting restricted)\n\uf0c1\n\n\nREQUEST LIGHTNING \".*\" striker2.atmos.albany.edu\nREQUEST LIGHTNING \".*\" idd.unidata.ucar.edu\n\n\n\nFSL/GSD Experimental HRRR (Sub-hourly)\n\uf0c1\n\n\nREQUEST FSL2 \"^GRIB2.FSL.HRRR\" hrrr.unidata.ucar.edu\n\n\n\n\n\nRestart the LDM\n\uf0c1\n\n\nUse the following commands to restart the LDM:\n\n\nsudo service edex_ldm restart\n\nldmadmin restart\n\n\n\n\n\nMonitor Incoming Data Feeds\n\uf0c1\n\n\nTo watch incoming data in real-time:\n\n\nnotifyme -vl -\n\n\n\nTo watch for a specific product and feed and time (360 sec = 6 min):\n\n\nnotifyme -vl - -h localhost -f NEXRAD3 -p DHR -o 360\n\n\n\nTo watch the same on a remote queue:\n\n\nnotifyme -vl - -h idd.unidata.ucar.edu -f NEXRAD3 -p DHR -o 360\n\n\n\n\n\nLDM Logging\n\uf0c1\n\n\nTo open a real-time readout of LDM logging you can run use the \nedex\n command. To exit, press \nCTRL+C\n.\n\n\nedex log ldm\n\n[edex] EDEX Log Viewer\n\n :: Viewing /awips2/ldm/logs/ldmd.log. Press CTRL+C to exit\n\nAug 26 15:05:10 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_MXUPHL01-21387192.grib2\": 406227 20160826210510.477 NGRID 21387192 YZCG86 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/MXUPHL01/5000-2000 m HGHT\nAug 26 15:05:11 edextest edexBridge[5812] NOTE: Sent 2 messages (0 at the end of the queue, 2 normally).\nAug 26 15:05:11 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_CICEP-21387200.grib2\": 369464 20160826210511.484 NGRID 21387200 YMCG98 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/CICEP/0 - NONE\nAug 26 15:05:12 edextest edexBridge[5812] NOTE: Sent 9 messages (0 at the end of the queue, 9 normally).\nAug 26 15:05:12 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_LTNG-21387205.grib2\": 482800 20160826210512.254 NGRID 21387205 YZCG98 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/LTNG/0 - EATM\nAug 26 15:05:13 edextest edexBridge[5812] NOTE: Sent 1 messages (0 at the end of the queue, 1 normally).",
"text":"Some additional feeds are included but commented out using '#'. To activate the feed, simply remove the #, save the file, and restart the LDM .",
"text":"To watch incoming data in real-time: notifyme -vl - To watch for a specific product and feed and time (360 sec = 6 min): notifyme -vl - -h localhost -f NEXRAD3 -p DHR -o 360 To watch the same on a remote queue: notifyme -vl - -h idd.unidata.ucar.edu -f NEXRAD3 -p DHR -o 360",
"text":"To open a real-time readout of LDM logging you can run use the edex command. To exit, press CTRL+C . edex log ldm\n\n[edex] EDEX Log Viewer\n\n :: Viewing /awips2/ldm/logs/ldmd.log. Press CTRL+C to exit\n\nAug 26 15:05:10 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_MXUPHL01-21387192.grib2\": 406227 20160826210510.477 NGRID 21387192 YZCG86 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/MXUPHL01/5000-2000 m HGHT\nAug 26 15:05:11 edextest edexBridge[5812] NOTE: Sent 2 messages (0 at the end of the queue, 2 normally).\nAug 26 15:05:11 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_CICEP-21387200.grib2\": 369464 20160826210511.484 NGRID 21387200 YMCG98 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/CICEP/0 - NONE\nAug 26 15:05:12 edextest edexBridge[5812] NOTE: Sent 9 messages (0 at the end of the queue, 9 normally).\nAug 26 15:05:12 edextest pqact[5811] NOTE: Filed in \"/awips2/data_store/grid/HRRR/HRRR_CONUS_2p5km_201608262000_F006_LTNG-21387205.grib2\": 482800 20160826210512.254 NGRID 21387205 YZCG98 KWBY 262000 !grib2/ncep/HRRR/#255/201608262000F006/LTNG/0 - EATM\nAug 26 15:05:13 edextest edexBridge[5812] NOTE: Sent 1 messages (0 at the end of the queue, 1 normally).",
"text":"Data Distribution Files\n\uf0c1\n\n\nOverview\n\uf0c1\n\n\nEDEX uses \ndistribution files\n to alert the appropriate decoding plug-in that new data has been recieved. These files do so by use of XML and regular expressions. If the WMO header, or file name*, matches a regular expression listed in a distribution XML, then EDEX will put a message into the QPID queue for its corresponding decoder to recognize and process. It is worth noting that more than one distribution file can recognize a single peice of data and notify their decoders to act.\n\n\n\n\n*Sometimes the distribution file will not look at the filename. If this file is coming in through the LDM using a proper FILE action, then the it is possible the distribution file will only look at the header and not the filename. If the file is ingested using the \nmanual\n endpoint (/awips2/data_store/ingest/), then this behaviour could be different.\n\n\n\n\nIf a piece of data \ndoes not\n match any distribution XML, EDEX will:\n\n\n\n\n\n\nCreate an entry in \n/awips2/edex/logs/edex-ingest-unrecognized-files-yyyymmdd.log\n\n\n\n\n\n\nSkip processing of the unrecognized file.\n\n\n\n\n\n\nDistribution files are stored in the \ncommon_static\n branch of the \nLocalization Store\n (series of directories that exist in \n/awips2/edex/data/utility/\n), and a list of available files can be found in the base-level directory. The base directory is: \n/awips2/edex/data/utility/common_static/base/distribution/\n.\n\n\nFor each plug-in, the distribution file is named \n[data-type].xml\n. For example, the distribution file for radar data is \nradar.xml\n. The distribution files follow the AWIPS base/site localization pattern: \n\n\n[root@edex]# cd /awips2/edex/data/utility/common_static/base/distribution\n[root@edex distribution]# ls\nacars.xml goesr.xml poessounding.xml\nairep.xml goessounding.xml profiler.xml\nairmet.xml grib.xml radar.xml\natcf.xml intlsigmet.xml redbook.xml\naww.xml lsr.xml satellite.gini.xml\n...\n\n\n\n\n\nCreating a Site Override\n\uf0c1\n\n\n\n\n\n\nBase\n files are located in \n/awips2/edex/data/utility/common_static/base/distribution/\n\n\n\n\n\n\nSite\n override distribution files are located in \n/awips2/edex/data/utility/common_static/\nsite/XXX\n/distribution/\n, where \nXXX\n is the site identifier.\n\n\n\n\n\n\nNote that site-level files override the base files; as a result, local modifications to distribution files must be made as follows:\n\n\n\n\n\n\nThe base distribution file must be copied from \n/awips2/edex/data/utility/common_static/base/distribution\n to \n/awips2/edex/data/utility/common_static/site/XXX/distribution\n\n\n\n\n\n\nThe local modification must be made to the file in \n/awips2/edex/data/utility/common_static/site/XXX/distribution\n\n\n\n\n\n\nThe basic structure of the distribution file is:\n\n\n<requestPatterns xmlns:ns2=\"group\">\n<regex>[pattern]</regex>\n<regex>[pattern]</regex>\n</requestPatterns>\n\n\n\nIneach\n\ntag,\n[pattern]\nisreplacedwitharegularexpressionthatwillmatcheitherthefilenameortheWMOheaderoftherawdata.Onlydatathatmatchesapatterninthedistributionfilewillbeprocessed.\n\n\nThecontentsofthebaseversionoftheradardistributionfile:\n\n\n[root@edex]#cd/awips2/edex/data/utility/common_static/base/distribution/\n[root@edex]#tail-4radar.xml\n\n<requestPatterns>\n<regex>^SDUS[234578]..*</regex>\n<regex>^Level3.*</regex>\n</requestPatterns>\n\n\n\nLookingatthebaseradar.xmldistributionfileinthisexample,therearetworegularexpressions.ThefirstregularexpressionmatchesthestandardWMOIDofradarproducts.UsingedexBridgetheLDMwillplaceamessageinthe\nexternal.dropbox\nQPIDqueue,indicatingaradarproducthasarrived.EDEXwillthentakethemessagecontainingtheradarWMOID(whichcomesfromthefileheader)andcompareitagainsttheregularexpressionsinradar.xml.Ifamatchisfound,EDEXplacesa
"text":"EDEX uses distribution files to alert the appropriate decoding plug-in that new data has been recieved. These files do so by use of XML and regular expressions. If the WMO header, or file name*, matches a regular expression listed in a distribution XML, then EDEX will put a message into the QPID queue for its corresponding decoder to recognize and process. It is worth noting that more than one distribution file can recognize a single peice of data and notify their decoders to act. *Sometimes the distribution file will not look at the filename. If this file is coming in through the LDM using a proper FILE action, then the it is possible the distribution file will only look at the header and not the filename. If the file is ingested using the manual endpoint (/awips2/data_store/ingest/), then this behaviour could be different. If a piece of data does not match any distribution XML, EDEX will: Create an entry in /awips2/edex/logs/edex-ingest-unrecognized-files-yyyymmdd.log Skip processing of the unrecognized file. Distribution files are stored in the common_static branch of the Localization Store (series of directories that exist in /awips2/edex/data/utility/ ), and a list of available files can be found in the base-level directory. The base directory is: /awips2/edex/data/utility/common_static/base/distribution/ . For each plug-in, the distribution file is named [data-type].xml . For example, the distribution file for radar data is radar.xml . The distribution files follow the AWIPS base/site localization pattern: [root@edex]# cd /awips2/edex/data/utility/common_static/base/distribution\n[root@edex distribution]# ls\nacars.xml goesr.xml poessounding.xml\nairep.xml goessounding.xml profiler.xml\nairmet.xml grib.xml radar.xml\natcf.xml intlsigmet.xml redbook.xml\naww.xml lsr.xml satellite.gini.xml\n...",
"text":"Base files are located in /awips2/edex/data/utility/common_static/base/distribution/ Site override distribution files are located in /awips2/edex/data/utility/common_static/ site/XXX /distribution/ , where XXX is the site identifier. Note that site-level files override the base files; as a result, local modifications to distribution files must be made as follows: The base distribution file must be copied from /awips2/edex/data/utility/common_static/base/distribution to /awips2/edex/data/utility/common_static/site/XXX/distribution The local modification must be made to the file in /awips2/edex/data/utility/common_static/site/XXX/distribution The basic structure of the distribution file is: <requestPatterns xmlns:ns2=\"group\">\n <regex>[pattern]</regex>\n <regex>[pattern]</regex>\n</requestPatterns> In each tag, [pattern] is replaced with a regular expression that will match either the filename or the WMO header of the raw data. Only data that matches a pattern in the distribution file will be processed. The contents of the base version of the radar distribution file: [root@edex]# cd /awips2/edex/data/utility/common_static/base/distribution/\n[root@edex]# tail -4 radar.xml\n\n<requestPatterns >\n <regex>^SDUS[234578]. .*</regex>\n <regex>^Level3.*</regex>\n</requestPatterns> Looking at the base radar.xml distribution file in this example, there are two regular expressions. The first regular expression matches the standard WMO ID of radar products. Using edexBridge the LDM will place a message in the external.dropbox QPID queue, indicating a radar product has arrived. EDEX will then take the message containing the radar WMO ID (which comes from the file header) and compare it against the regular expressions in radar.xml. If a match is found, EDEX places a message in the QPID queue Ingest.radar. The radar decoder will then consume the message and process the radar data accordingly.",
"text":"As a quick example, suppose we have a local data source for satellite imagery that does not have a WMO header; also suppose that the data source writes to files whose names start with LOCAL.sat . To add this locally produced satellite data file to the EDEX distribution; perform the following steps. Copy the base version of satellite.gini.xml from the base distribution directory /awips2/edex/data/utility/common_static/base/distribution into the site distribution directory /awips2/edex/data/utility/common_static/site/XXX/distribution Edit the site version of satellite.gini.xml , adding a new <regex> </regex> tag immediately below the existing regular expression ( <regex> </regex> ) tag. The contents of the tag will be ^LOCAL.sat . The final result will be: \n <requestPatterns xmlns:ns2=\"group\">\n <regex>TI[CGT]... ....</regex>\n <regex>rad/NEXRCOMP</regex>\n <regex>.\\*.gini.\\*</regex>\n <regex>^LOCAL.sat.*</regex> \n </requestPatterns> Save the file and exit the editor. EDEX will automatically pick up the new distribution pattern. Raw files are written to subdirectories in /awips2/data_store/ , and a message is sent via QPID to the EDEX distribution service from the LDM. When a regular expression match is found in a data distribution file, the raw data file is placed in a queue for the matching plugin to decode and process. The distribution files are used to match file headers as well as filenames, which is how files dropped into EDEX's manual endpoint ( /awips2/data_store/ingest/ ) are processed.",
"text":"Because these files are in the common/_static directory, they have to be manually edited using a text editor. You should not edit the base files; rather, as stated above, you should copy the base version to your site and then edit the site version . The regular expressions in the distribution files do not necessarily need to correspond with the regular expressions in the LDM pqact.conf file. It is important to note that: The regex in the pqact.conf file applies to the productID that is passed through the LDM. and The regex in the distribution files (.xml) typically applies to the header in the file. It can also apply to the filename, if the file is coming through the manual endpoint, or if the data has no header to begin with. If patterns exist in pqact.conf but there are no corresponding matching regex expressions in any distribution file, then raw data files will be written to /awips2/data_store/ but will not be ingested and processed by EDEX. Entries for these non-ingested files would be written to the unrecognized files log in /awips/edex/logs .",
"text":"Its distribution file is located at: /awips2/edex/data/utility/common_static/base/distribution/obs.xml : <requestPatterns xmlns:ns2=\"group\">\n <regex>^S[AP].*</regex>\n</requestPatterns> It will process any file header that starts with SA or SP , which should match any WMO header that contains METAR data (e.g. SAUS , SPUS , SACN , SAMX ).",
"text":"Its distribution file is located at /awips2/edex/data/utility/common_static/base/distribution/text.xml : <requestPatterns>\n <regex>^[ACFNRUW][A-Z][A-Z0-9]{4} [A-Z0-9]{4}</regex>\n <regex>^S[A-CEG-Z].*</regex>\n <!-- Only have AFOS mapping for T[BCX] -->\n <regex>^T[BCX].*</regex>\n <regex>^SF[A-OQ-TV-Z].*</regex>\n <regex>^SDUS1.*</regex>\n <regex>^SDUS4[1-6].*</regex>\n <regex>^SDUS9[^7].*</regex>\n <regex>^SFU[^S].*</regex>\n <regex>^SFUS4[^1].*</regex>\n <regex>^SFP[^A].*</regex>\n <regex>^SFPA[^4].*</regex>\n <regex>^SFPA4[^12].*</regex>\n <regex>^BMBB91.*</regex>\n <regex>^N[A-Z][A-Z0-9]{4} [A-Z0-9]{4}</regex>\n <regex>^F[EHIJKLMQVWX].*</regex>\n <regex>wcl_decrypted</regex>\n <regex>ecmwf_mos_decrypted</regex>\n</requestPatterns> Processes lots of WM patterns. The second pattern ^S[A-CEG-Z].* matches any header that starts with S except for SD or SF . This is because it matches A through C ( A-C ), E, and G through Z ( G-Z ). So it also matches the SA and SP files that the obs.xml plugin matches. This means that METARs are processed by both plugins simultaneously.",
"text":"Its distribution file is located at /awips2/edex/data/utility/common_static/base/distribution/grib.xml : <requestPatterns>\n <!-- Super Set of all possible WMO grib patterns -->\n <!-- Is specifically not restricting on CCCC since HPE isn't populating it -->\n <regex>^[EHLMOYZ][A-Z]{3}\\d{2}</regex>\n <!-- Exclude Data Delivery specific patterns -->\n <regexExclude>^LZ[ABC][ABC]9[123] (KWBC|KNCF)</regexExclude>\n\n <!-- ECMWF decrypted -->\n <regex>ecmwf_decrypted</regex>\n\n <!-- NWPS pattern -->\n <regex>\\p{Alpha}{3}_nwps_CG1</regex>\n <regex>\\p{Alpha}{3}_nwps_CG0_Trkng</regex>\n\n <!-- grib files without WMO headers -->\n <regex>.*grib.*</regex>\n <regex>.*GRIB.*</regex>\n <regex>.*grb.*</regex>\n <regex>^US058.*</regex>\n <regex>^CMC_reg.*</regex>\n</requestPatterns> The grib/grid decoder distribution file matches all numerical grids distributed over the IDD NGRID feed by matching WMO header, and from CONDUIT by matching various .grib file extensions. It also includes an example of a regexExclude message which can be used to single out matching values that aren't to be included.",
"text":"Important notes about regular expressions: Any time a new entry is placed in the pqact.conf file on LDM, it is likely a corresponding entry needs to be added to the appropriate Data Distribution file in the data distribution directory, or the data file will be logged to edex-ingest-unrecognized-files-YYYYMMDD.log . The exception to this rule is if the new data coming from the LDM is a type of data that already exists and EDEX already has a distribution file with a matching regex that will recognize it. If you have written a new regex for a distribution file to match on a filename, and it is not matching, then the file most likely has a header. In this case EDEX will only look at the header to match the regex. You must change your regex to something that matches the header, not the filename.",
"text":"IngestaNewGrid\n\uf0c1\n\n\nUnrecognizedgridscanbedecodedbyEDEXsimplybydropping\n*.grib\nor\n*.grib2\nfilesinto\n/awips2/data_store/ingest/\n\n\nToaddsupportforanewgrid,twoeditsmustbemade:\n\n\n\n\nGeospatialprojection\nmustbedefinedina\ngridnavigationfile\n\n\nGridname\n,\ncenter\n,\nsubcenter\n,and\nprocessID\nmustbedefinedina\nmodeldefinitionfile\n\n\n\n\nIftheparametersinthegribfilehaven'tbeenpreviouslyspecified,anotherchange\nmay\nbeneededaswell:\n\n\n\n\nCenter\n,\nsubcenter\n,\ndiscipline\n,\ncategory\n,andpossibly\nparameterID\ninformationmayneedtobedefinedina\ntable\n\n\n\n\n\n\nIngestanUnsupportedGrid\n\uf0c1\n\n\nGribProducts\n\uf0c1\n\n\n\n\n\n\nDownloadanexamplegrib1fileandrenametoa\n*.grib\nextension,thencopytothemanualingestpoint\n/awips2/data_store/ingest/\n\n\n\nwgethttps://www.unidata.ucar.edu/software/awips2/14102318_nmm_d01.GrbF00600 -O wrf.grib\n\ncp wrf.grib /awips2/data_store/ingest/\n\n\n\nRemember that the data distribution file (\n/awips2/edex/data/utility/common_static/base/distribution/grib.xml\n) will match filenames which have the \n*.grib*\n extension.\n\n\n\n\n\n\nConfirm that the grib file decodes in the grib log file:\n\n\nedex log grib\n\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1200 (sec) Latency: 21.8080 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1180 (sec) Latency: 21.8140 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.4230 (sec) Latency: 21.8360 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.2240 (sec) Latency: 21.9140 (sec)\n\n...\n\n\n\n\n\n\n\nCheck that the hdf5 data directory exists for our unnamed grid\n\n\nls -latr /awips2/edex/data/hdf5/grid/GribModel:7:0:89\n\n\n\nThough the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (7, 0, 89, respectively). \n\n\n\n\n\n\nGrib2 Products\n\uf0c1\n\n\n\n\n\n\nDownload an example grib2 file and rename to a \n*.grib2\n extension, then copy to the manual ingest point \n/awips2/data_store/ingest/\n \n\n\nwget https://www.unidata.ucar.edu/software/awips2/CPTI_00.50_20180502-000144.grib2 -O cpti.grib2\n\ncp cpti.grib2 /awips2/data_store/ingest/\n\n\n\nRemember that the data distribution file (\n/awips2/edex/data/utility/common_static/base/distribution/grib.xml\n) will match filenames which have the \n*.grib*\n extension.\n\n\n\n\n\n\nConfirm that the grib file decodes in the grib log file:\n\n\nedex log grib\n\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1200 (sec) Latency: 21.8080 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1180 (sec) Latency: 21.8140 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.4230 (sec) Latency: 21.8360 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.2240 (sec) Latency: 21.9140 (sec)\n\n...\n\n\n\nNote:\n This step will likely fail, because the parameter is not yet defined. The error will look like:\n\n\nINFO 2020-07-20 20:34:17,710 2565 [GribPersist-1] GridDao: EDEX - Discarding record due to missing or unknown parameter mapping: /grid/2018-05-02_00:01:44.0_(0)/GribModel:161:0:97/null/null/403/Missing/FH/500.0/-999999.0\nINFO 2020-07-20 20:34:17,710 2566 [GribPersist-1] Ingest: EDEX: Ingest - grib2:: /awips2/data_store/ingest/CPTI_00.50_20180502-000144.grib2 processed in: 2.3550 (sec)\nINFO 2020-07-20 20:34:17,827 2567 [Ingest.GribDecode-6] grib: EDEX - No parameter information for center[161], subcenter[0], tableName[4.2.209.3], parameter value[61]\n\n\n\nIn order to successfully ingest the example file, \ndefine the appropriate table\n.\n\n\n\n\n\n\nCheck that the hdf5 data directory exists for our unnamed grid\n\n\nls -latr /awips2/edex/data/hdf5/grid/GribModel:161:0:97
"text":"Unrecognized grids can be decoded by EDEX simply by dropping *.grib or *.grib2 files into /awips2/data_store/ingest/ To add support for a new grid, two edits must be made: Geospatial projection must be defined in a grid navigation file Grid name , center , subcenter , and process ID must be defined in a model definition file If the parameters in the grib file haven't been previously specified, another change may be needed as well: Center , subcenter , discipline , category , and possibly parameter ID information may need to be defined in a table",
"text":"Download an example grib1 file and rename to a *.grib extension, then copy to the manual ingest point /awips2/data_store/ingest/ wget https://www.unidata.ucar.edu/software/awips2/14102318_nmm_d01.GrbF00600 -O wrf.grib\n\ncp wrf.grib /awips2/data_store/ingest/ Remember that the data distribution file ( /awips2/edex/data/utility/common_static/base/distribution/grib.xml ) will match filenames which have the *.grib* extension. Confirm that the grib file decodes in the grib log file: edex log grib\n\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1200 (sec) Latency: 21.8080 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.1180 (sec) Latency: 21.8140 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.4230 (sec) Latency: 21.8360 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/grib/20141026/14/wrf.grib processed in: 0.2240 (sec) Latency: 21.9140 (sec)\n\n... Check that the hdf5 data directory exists for our unnamed grid ls -latr /awips2/edex/data/hdf5/grid/GribModel:7:0:89 Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (7, 0, 89, respectively).",
"title":"Grib Products"
},
{
"location":"/edex/new-grid/#grib2-products",
"text":"Download an example grib2 file and rename to a *.grib2 extension, then copy to the manual ingest point /awips2/data_store/ingest/ wget https://www.unidata.ucar.edu/software/awips2/CPTI_00.50_20180502-000144.grib2 -O cpti.grib2\n\ncp cpti.grib2 /awips2/data_store/ingest/ Remember that the data distribution file ( /awips2/edex/data/utility/common_static/base/distribution/grib.xml ) will match filenames which have the *.grib* extension. Confirm that the grib file decodes in the grib log file: edex log grib\n\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1200 (sec) Latency: 21.8080 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.1180 (sec) Latency: 21.8140 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.4230 (sec) Latency: 21.8360 (sec)\nINFO [Ingest.GribDecode] /awips2/data_store/ingest/cpti.grib2 processed in: 0.2240 (sec) Latency: 21.9140 (sec)\n\n... Note: This step will likely fail, because the parameter is not yet defined. The error will look like: INFO 2020-07-20 20:34:17,710 2565 [GribPersist-1] GridDao: EDEX - Discarding record due to missing or unknown parameter mapping: /grid/2018-05-02_00:01:44.0_(0)/GribModel:161:0:97/null/null/403/Missing/FH/500.0/-999999.0\nINFO 2020-07-20 20:34:17,710 2566 [GribPersist-1] Ingest: EDEX: Ingest - grib2:: /awips2/data_store/ingest/CPTI_00.50_20180502-000144.grib2 processed in: 2.3550 (sec)\nINFO 2020-07-20 20:34:17,827 2567 [Ingest.GribDecode-6] grib: EDEX - No parameter information for center[161], subcenter[0], tableName[4.2.209.3], parameter value[61] In order to successfully ingest the example file, define the appropriate table . Check that the hdf5 data directory exists for our unnamed grid ls -latr /awips2/edex/data/hdf5/grid/GribModel:161:0:97 Though the grib file has been decoded, it has been given a generic name with its center, subcenter, and process IDs (161, 0, 97, respectively).",
"text":"When the grid was ingested a record was added to the grid_coverage table with its navigation information: psql metadata\n\nmetadata=# select nx,ny,dx,dy,majoraxis,minoraxis,la1,lo1,lov,latin1,latin2 from gridcoverage where id=(select distinct(location_id) from grid_info where datasetid='GribModel:7:0:89');\n nx | ny | dx | dy | majoraxis | minoraxis | la1 | lo1 | lov | latin1 | latin2 \n-----+-----+------------------+------------------+-----------+-----------+------------------+-------------------+-------------------+------------------+------------------\n 201 | 155 | 4.29699993133545 | 4.29699993133545 | 6378160 | 6356775 | 42.2830009460449 | -72.3610000610352 | -67.0770034790039 | 45.3680000305176 | 45.3680000305176\n(1 row) Compare with the projection info returned by wgrib on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above): \nwgrib -V wrf.grib \nrec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef\n ALBDO=Albedo [%]\n timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0\n center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid) \n Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000 \n Latin1 45.368000 Latin2 45.368000 LatSP 0.000000 LonSP 0.000000\n North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8\n min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0 Notice that our grib file has a Lambert Conformal projection. We will need these values for the next step. Note that there is a tolerance of +/- 0.1 degrees to keep in mind when defining your coverage area.",
"text":"When the grid was ingested a record was added to the grid_coverage table with its navigation information: psql metadata\n\nmetadata=# select nx,ny,dx,dy,majoraxis,minoraxis,la1,lo1,lov,latin1,latin2 from gridcoverage where id=(select distinct(location_id) from grid_info where datasetid='GribModel:161:0:97');\n\nnx | ny | dx | dy | majoraxis | minoraxis | la1 | lo1 | lov | latin1 | latin2 \n-----+-----+-------+-------+-----------+-----------+-----------+-----+-----+--------+--------\n600 | 640 | 0.005 | 0.005 | | | 40.799999 | 261 | | | \n(1 row) Compare with the projection info returned by wgrib2 on the original file (look at the bolded sections below and make sure they match up with the corresponding entries returned from the database above): \nwgrib2 -grid -nxny cpti.grib2\n1:0:grid_template=0:winds(N/S):\n lat-lon grid:(600 x 640) units 1e-06 input WE:NS output WE:SN res 48\n lat 40.799999 to 37.599999 by 0.005000 \n lon 260.999999 to 263.999999 by 0.005000 #points=384000:(600 x 640)\n ... Notice that our grib2 file has a Lat/lon Grid projection. Where: nx is 600 ny is 640 dx is 0.005 dy is 0.005 la1 is 40.799999 lo1 is 261 We will need these values for the next step. Note that there is a tolerance of +/- 0.1 degrees to keep in mind when defining your coverage (la1 and lo1) area.",
"text":"Copy an existing xml file with the same grid projection type (in this case lambertConformalGridCoverage ) to a new file wrf.xml : cd /awips2/edex/data/utility/common_static/base/grib/grids/\ncp RUCIcing.xml wrf.xml And edit the new wrf.xml to define the projection values using the output from wgrib or the database (example provided): vi wrf.xml\n\n<lambertConformalGridCoverage>\n <name>201155</name>\n <description>Regional - CONUS (Lambert Conformal)</description>\n <la1>42.2830009460449</la1>\n <lo1>-72.3610000610352</lo1>\n <firstGridPointCorner>LowerLeft</firstGridPointCorner>\n <nx>201</nx>\n <ny>155</ny>\n <dx>4.29699993133545</dx>\n <dy>4.29699993133545</dy>\n <spacingUnit>km</spacingUnit>\n <minorAxis>6356775.0</minorAxis>\n <majorAxis>6378160.0</majorAxis>\n <lov>-67.0770034790039</lov>\n <latin1>45.3680000305176</latin1>\n <latin2>45.3680000305176</latin2>\n</lambertConformalGridCoverage> Note : Notice the <name>201155</name> tag was defined from the number of grid points (201 and 155). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. WRF).",
"title":"Grib Products"
},
{
"location":"/edex/new-grid/#grib2-products_2",
"text":"Copy an existing xml file with the same grid projection type (in this case latLonGridCoverage ) to a new file cpti.xml : cd /awips2/edex/data/utility/common_static/base/grib/grids/\ncp MRMS-1km.xml cpti.xml And edit the new cpti.xml to define the projection values using the output from wgrib2 or the database (example provided): vi cpti.xml\n\n<latLonGridCoverage>\n <name>600640</name>\n <description>Small domain for CPTI products</description>\n <la1>40.799999</la1>\n <lo1>261</lo1>\n <firstGridPointCorner>UpperLeft</firstGridPointCorner>\n <nx>600</nx>\n <ny>640</ny>\n <dx>0.005</dx>\n <dy>0.005</dy>\n <spacingUnit>degree</spacingUnit>\n</latLonGridCoverage> Note : Notice the <name>384000</name> tag was defined from the number of grid points (600 and 640). This value will be matched against an entry in our models file (below) to set the name of the model (e.g. CPTI).",
"text":"Since our grib file has a center ID of 7 (NCEP) we will edit the gribModels_NCEP-7.xml file. cd /awips2/edex/data/utility/common_static/base/grib/models/\n\nvi gribModels_NCEP-7.xml In <gribModelSet> add an entry: <model>\n <name>WRF</name>\n <center>7</center>\n <subcenter>0</subcenter>\n <grid>201155</grid>\n <process>\n <id>89</id>\n </process>\n </model> Save the file and restart EDEX for the changes to take effect: sudo service edex_camel restart ingestGrib Now copy the wrf.grib file again to /awips2/data_store/ingest/ . If everything is correct we will not see any persistence errors since the grid is now named WRF and not GribModel:7:0:89 . cp wrf.grib /awips2/data_store/ingest/\n\nedex log grib After you have confirmed that the grid was ingested with the given name, you can edit the D2D product menus to display the new grid .",
"title":"Grid Prodcuts"
},
{
"location":"/edex/new-grid/#grib2-products_3",
"text":"Since our grib2 file has a center of 161 (NOAA) we will edit the gribModels_NOAA-161.xml file. cd /awips2/edex/data/utility/common_static/base/grib/models/\n\nvi gribModels_NOAA-161.xml In <gribModelSet> , under the <-- Subcenter 0 --> comment, add an entry: <model>\n <name>CPTI</name>\n <center>161</center>\n <subcenter>0</subcenter>\n <grid>600640</grid>\n <process>\n <id>97</id>\n </process>\n</model> Save the model file and restart edex: sudo service edex_camel restart ingestGrib Now if you drop cpti.grib2 into the manual endpoint again, it should ingest without any persistence errors.",
"title":"Grib2 Products"
},
{
"location":"/edex/new-grid/#adding-a-table",
"text":"If you ingest a piece of data and the parameter appears as unknown in the metadata database, ensure that the correct parameter tables are in place for the center/subcenter. The tables are located in /awips2/edex/data/utility/common_static/base/grib/tables/ . They are then broken into subdirectories using the following structure: /[Center]/[Subcenter]/4.2.[Discipine].[Category].table .",
"title":"Adding a Table"
},
{
"location":"/edex/new-grid/#grib-products_3",
"text":"The center and subcenter have been identified previously here and here , as 7 and 0, respectively. So, the corresponding directory is: /awips2/edex/data/utility/common_static/base/grib/tables/7/0/ To find the discipline of a grib product, you need the process and table values from the grib file. These are output with the wgrib -V command: \nwgrib -V wrf.grib \nrec 799:27785754:date 2014102318 ALBDO kpds5=84 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 6hr fcst: bitmap: 736 undef\n ALBDO=Albedo [%]\n timerange 0 P1 6 P2 0 TimeU 1 nx 201 ny 155 GDS grid 3 num_in_ave 0 missing 0\n center 7 subcenter 0 process 89 Table 2 scan: WE:SN winds(grid) \n Lambert Conf: Lat1 42.283000 Lon1 -72.361000 Lov -67.077000\n Latin1 45.368000 Latin2 45.368000 < LatSP 0.000000 LonSP 0.000000\n North Pole (201 x 155) Dx 4.297000 Dy 4.297000 scan 64 mode 8\n min/max data 5 21.9 num bits 8 BDS_Ref 50 DecScale 1 BinScale 0 For our example, the process is 89 and table is 2 . Next, take a look in: /awips2/edex/data/utility/common_static/base/grid/grib1ParameterConvTable.xml And find the entry that has grib1 data with TableVersion 2 and Value 89: <grib1Parameter>\n <center>7</center>\n <grib1TableVersion>2</grib1TableVersion>\n <grib1Value>89</grib1Value>\n <grib2discipline>0</grib2discipline>\n <grib2category>3</grib2category>\n <grib2Value>10</grib2Value>\n </grib1Parameter> Here, we can see the discipline and category values (referred to as x above) are 0 and 3, respectively. So, the table needed for our example file is: /awips2/edex/data/utility/common_static/base/grib/tables/7/0/4.2.0.3.table",
"text":"If you are using a grib2 file, then you can use either the log output or the -center , -subcenter , and -full_name options on wgrib2 to get the center, subcenter, discipline, category, and parameter information: The table would be found in the directory structure using this file's center and subcenter. The center can be found by either: Running the following command: wgrib2 -center cpti.grib2\n 1:0:center=US NOAA Office of Oceanic and Atmospheric Research\n... And then looking up the corresponding value for \"US NOAA Office of Oceanic and Atmospheric Research\" at this website , where it happens to be 161 . OR: Running the following command: \nwgrib2 -varX cpti.grib2\n 1:0:var209_255_1_ 161 _3_61\n... Where the 4th argument after \"var\" is the center id, in this case **161**. To get the subcenter, simply run: \nwgrib2 -subcenter cpti.grib2\n 1:0:subcenter= 0 \n... The subcenter of this file is 0 . So based on the center and subcenter, the corresponding directory is: /awips2/edex/data/utility/common_static/base/grib/tables/161/0/ To find the exact table, we need the discipline and category: \nwgrib2 -full_name cpti.grib2\n 1:0:var 209 _ 3 _ 61 .500_m_above_mean_sea_level\n ... In this case the discipline is 209 and category is 3 , so the corresponding table is: 4.2.209.3.table So, the full path to the corresponding table would be: /awips2/edex/data/utility/common_static/base/grib/tables/161/0/4.2.209.3.table The parameter ID was also listed in that output as 61 . Make sure that specific parameter information is defined in the table: \n...\n56:56:Reflectivity at -20C:dBZ:ReflectivityM20C\n57:57:Reflectivity At Lowest Altitude (RALA):dBZ:ReflectivityAtLowestAltitude\n58:58:Merged Reflectivity At Lowest Altitude (RALA):dBZ:MergedReflectivityAtLowestAltitude\n59:59:CPTI 80mph+:%:CPTI80mph 61:61:CPTI 110mph+:%:CPTI110mph You will have to restart ingestGrib for the changes to take place: sudo service edex_camel restart ingestGrib Now you can try re-ingesting the grib2 file .",
"text":"Make sure the latitude and longitude entries in your coverage specification file match those of your ingested raw grib file. There is a tolerance of +/- 0.1 degree to keep in mind when defining your coverage area. If some of the information is unknown, using a grib utility application such as wgrib and wgrib2 can be useful in determining the information that must be added to correctly process a new grib file. If you are experiencing Segmentation fault errors when running wgrib2, it may be best to install the latest version using the following command: yum install wgrib2 And then you may either need to change where wgrib2 points to, or use /bin/wgrib2 to run the recently downloaded version.",
"text":"Purging and Retention\n\uf0c1\n\n\nPurge Types\n\uf0c1\n\n\nThere are two main forms of data puring in AWIPS. The most often thought of is the purging for \nprocessed data\n. This has to do with how long data is stored for \nafter\n it has been decoded and processed.\n\n\nThe second type of purging has to do with \nraw data\n. This has to do with how long data is stored for \nbefore\n it has been decoded.\n\n\nProcessed Data Purging\n\uf0c1\n\n\nAWIPS uses a plugin-based purge strategy for processed \nHDF5 data\n. This allows the user to change the purge frequency for each plugin individually, and even set purge rules for specific products for a particular plugin. There is also a default purge rules file for those products which do not have specific rules written.\n\n\n\n\nNote\n: Purging is triggered by a quartz timer event that fires at 30 minutes after each hour. \n\n\n\n\nPurging rules are defined in XML files in the Localization Store. On EDEX, most are located in \n/awips2/edex/data/utility/common_static/base/purge\n, and follow the \nbase/site\n localization pattern (e.g. site purge files are in \nsite/XXX/purge\n rather than \nbase/purge\n, where XXX is the site identifier).\n\n\nEach data set can have a purge rule defined, and the xml file is named after the data set:\n\n\nls /awips2/edex/data/utility/common_static/base/purge/\n\nacarsPurgeRules.xml bufruaPurgeRules.xml pirepPurgeRules.xml\nacarssoundingPurgeRules.xml ccfpPurgeRules.xml poessoundingPurgeRules.xml\naggregatePurgeRules.xml convsigmetPurgeRules.xml pointsetPurgeRules.xml\nairepPurgeRules.xml cwaPurgeRules.xml profilerPurgeRules.xml\n...\n\n\n\n\n\nTime-based purge\n\uf0c1\n\n\nIf a plugin has no XML file, the default rule of 1 day (24 hours) is used, from \n/awips2/edex/data/utility/common_static/base/purge/defaultPurgeRules.xml\n:\n\n\n\n\n\n<purgeRuleSet>\n <defaultRule>\n \n<period>01-00:00:00</period>\n\n </defaultRule>\n</purgeRuleSet>\n\n\n\n\nTime-based purging is set with the \nperiod\n tag and uses the \nreference time\n of the data. The reference time of the data is determined by the decoder. \n\n\n\n\n30-day NEXRAD3 Example\n\uf0c1\n\n\nModify \n/awips2/edex/data/utility/common_static/base/purge/radarPurgeRules.xml\n to increase the data retention period from 1 to 31 days:\n\n\n\n<purgeRuleSet>\n <defaultRule>\n \n<period>31-00:00:00</period>\n\n </defaultRule>\n</purgeRuleSet>\n\n\n\n\n\n\nNote\n: you do NOT have to restart EDEX when you change a purge rule!\n\n\n\n\n\n\nFrame-Based Purge\n\uf0c1\n\n\nSome plugins use frame-base purging, retaining and certain number of product \"versions\". \n\n\n/awips2/edex/data/utility/common_static/base/purge/gridPurgeRules.xml\n\n\n \n\n<defaultRule>\n <versionsToKeep>2</versionsToKeep>\n\n <period>07-00:00:00</period>\n </defaultRule>\n <rule>\n \n<keyValue>LAPS</keyValue>\n <versionsToKeep>30</versionsToKeep>\n\n </rule>\n <rule regex=\"true\">\n\n<keyValue>NAM(?:12|20|40)</keyValue>\n<versionsToKeep>2</versionsToKeep>\n<modTimeToWait>00-00:15:00</modTimeToWait>\n\n</rule>\n...\n\n\n\n\nIntheaboveexample,noticea\ndefaultrule\n(2)isspecified,aswellasspecificmodelswiththeirownrules.\n\nThetag\nmodTimeToWait\ncanbeusedinconjunctionwith\nversionsToKeep\nandwillincreasetheversionsToKeepby1ifdatamatchingthisrulehasbeenstoredwithinmodTimeToWait.\n\n\n\n\nPurgeLogs\n\uf0c1\n\n\nDatapurgeeventsareloggedtothefile\nedex-ingest-purge-[yyyymmdd].log\n,where\n[yyyymmdd]\nisthedatestamp.\n\n\ntail-fedex-ingest-purge-20120327.log\n\n--------STARTLOGPURGE---------\nINFO2012-03-2700:30:00,027[DefaultQuartzScheduler_Worker-3]PurgeLogger:EDEX-PURGELOGS::SkippedfilewithinvalidfileName:afos-trigger.log\nINFO2012-03-2700:30:00,193[DefaultQuartzScheduler_Worker-3]PurgeLogger:EDEX-PURGELOGS::Removed1oldfiles\nINFO2012-03-2700:31:23,155[DefaultQuartzScheduler_Worker-3]
"text":"There are two main forms of data puring in AWIPS. The most often thought of is the purging for processed data . This has to do with how long data is stored for after it has been decoded and processed. The second type of purging has to do with raw data . This has to do with how long data is stored for before it has been decoded.",
"text":"AWIPS uses a plugin-based purge strategy for processed HDF5 data . This allows the user to change the purge frequency for each plugin individually, and even set purge rules for specific products for a particular plugin. There is also a default purge rules file for those products which do not have specific rules written. Note : Purging is triggered by a quartz timer event that fires at 30 minutes after each hour. Purging rules are defined in XML files in the Localization Store. On EDEX, most are located in /awips2/edex/data/utility/common_static/base/purge , and follow the base/site localization pattern (e.g. site purge files are in site/XXX/purge rather than base/purge , where XXX is the site identifier). Each data set can have a purge rule defined, and the xml file is named after the data set: ls /awips2/edex/data/utility/common_static/base/purge/\n\nacarsPurgeRules.xml bufruaPurgeRules.xml pirepPurgeRules.xml\nacarssoundingPurgeRules.xml ccfpPurgeRules.xml poessoundingPurgeRules.xml\naggregatePurgeRules.xml convsigmetPurgeRules.xml pointsetPurgeRules.xml\nairepPurgeRules.xml cwaPurgeRules.xml profilerPurgeRules.xml\n...",
"text":"If a plugin has no XML file, the default rule of 1 day (24 hours) is used, from /awips2/edex/data/utility/common_static/base/purge/defaultPurgeRules.xml : \n<purgeRuleSet>\n <defaultRule>\n <period>01-00:00:00</period> \n </defaultRule>\n</purgeRuleSet> Time-based purging is set with the period tag and uses the reference time of the data. The reference time of the data is determined by the decoder.",
"text":"Modify /awips2/edex/data/utility/common_static/base/purge/radarPurgeRules.xml to increase the data retention period from 1 to 31 days: \n<purgeRuleSet>\n <defaultRule>\n <period>31-00:00:00</period> \n </defaultRule>\n</purgeRuleSet> Note : you do NOT have to restart EDEX when you change a purge rule!",
"text":"Some plugins use frame-base purging, retaining and certain number of product \"versions\". /awips2/edex/data/utility/common_static/base/purge/gridPurgeRules.xml <defaultRule>\n <versionsToKeep>2</versionsToKeep> \n <period>07-00:00:00</period>\n </defaultRule>\n <rule>\n <keyValue>LAPS</keyValue>\n <versionsToKeep>30</versionsToKeep> \n </rule>\n <rule regex=\"true\">\n <keyValue>NAM(?:12|20|40)</keyValue>\n <versionsToKeep>2</versionsToKeep>\n <modTimeToWait>00-00:15:00</modTimeToWait> \n </rule>\n ... In the above example, notice a default rule (2) is specified, as well as specific models with their own rules. \nThe tag modTimeToWait can be used in conjunction with versionsToKeep and will increase the versionsToKeep by 1 if data matching this rule has been stored within modTimeToWait.",
"text":"To see all purge rule directories (base, site, configured): find /awips2/edex/data/utility -name purge\n\n/awips2/edex/data/utility/common_static/base/purge If any overrides have been made, then it's possible that site directories may show up as results from the find command as well.",
"text":"Raw data are files that have been brought in by the LDM and recognized by an action in the pqact.conf file. These files are written to subdirectories of /awips2/data_store/ . This data will wait here until it is purged, from the purging rules defined in /awips2/edex/data/utility/common_static/base/archiver/purger/RAW_DATA.xml . If the purge time is too short, and the processing latencies on EDEX are too long, it is possible that EDEX will miss some of this data, and the purge times will need to be adjusted by changing the <defaultRetentionHours> or <selectedRetentionHours> tag on the relevent data sets.",
"title":"Raw Data Purging"
},
{
"location":"/edex/data-purge/#default-retention",
"text":"The defaultRetentionHours tag is defined at the beginning of the RAW_DATA.xml file. It is the duration that will apply to any piece of data that does not fall under an explicitly defined category . The default value for our EDEX is 1 hour: \n<archive>\n <name>Raw</name>\n <rootDir>/awips2/data_store/</rootDir>\n <defaultRetentionHours>1</defaultRetentionHours> \n <category>\n ...",
"text":"Data sets are broken up into categories in the RAW_DATA.xml file. These categories are groupings of similar data. Each category has a selectedRetentionHours tag which specifies how long the matching data will be kept for. For example, there is a Model category which sets the purge time to 3 hours for all grib, bufrmos, and modelsounding data: \n... <category>\n <name>Model</name>\n <selectedRetentionHours>3</selectedRetentionHours> \n <dataSet>\n <dirPattern>(grib|grib2)/(\\d{4})(\\d{2})(\\d{2})/(\\d{2})/(.*)</dirPattern> \n <displayLabel>{1} - {6}</displayLabel>\n <dateGroupIndices>2,3,4,5</dateGroupIndices>\n </dataSet>\n <dataSet>\n <dirPattern>(bufrmos|modelsounding)/(\\d{4})(\\d{2})(\\d{2})/(\\d{2})</dirPattern> \n <displayLabel>{1}</displayLabel>\n <dateGroupIndices>2,3,4,5</dateGroupIndices>\n </dataSet>\n</category>\n...",
"text":"Monitor Users\n\uf0c1\n\n\nTo see a list of clients connecting to your EDEX server, use the \nedex users [YYYYMMDD]\n command, where \n[YYYYMMDD]\n is the optional date string.\n\n\nedex users\n\n -- EDEX Users 20160826 --\nuser@101.253.20.225\nuser@192.168.1.67\nawips@0.0.0.0\nawips@sdsmt.edu\n...\n\n\n\nLogging Daily EDEX Users\n\uf0c1\n\n\nTo get a running log of who has accessed EDEX, you can create a short script. \n\n\nThe example below is a script that runs once daily at 20 minutes after 00 UTC, appending each day's \nedex users\n list to a logfile \n/home/awips/edex-users.log\n:\n\n\n\n\n\n\nvi~/edexUsers.sh\n\n#!/bin/bash\n/awips2/edex/bin/edex users >> /home/awips/edex-users.log\n\n\n\n\n\n\n\ncrontab -e\n\n0 20 * * * /home/awips/edexUsers.sh 1>> /dev/null 2>&1",
"title":"Monitor Users"
},
{
"location":"/edex/edex-users/#monitor-users",
"text":"To see a list of clients connecting to your EDEX server, use the edex users [YYYYMMDD] command, where [YYYYMMDD] is the optional date string. edex users\n\n -- EDEX Users 20160826 --\nuser@101.253.20.225\nuser@192.168.1.67\nawips@0.0.0.0\nawips@sdsmt.edu\n...",
"text":"To get a running log of who has accessed EDEX, you can create a short script. The example below is a script that runs once daily at 20 minutes after 00 UTC, appending each day's edex users list to a logfile /home/awips/edex-users.log : vi~/edexUsers.sh\n\n#!/bin/bash\n/awips2/edex/bin/edex users >> /home/awips/edex-users.log crontab -e\n\n0 20 * * * /home/awips/edexUsers.sh 1>> /dev/null 2>&1",
"text":"NAME DESCRIPTION aqi Air Quality Index data bufrmos Model Output Statistics bufrua Upper air radiosonde data climate-hmdb Climate text products geodata NetCDF JTS Geometry records geomag SWPC Geomagnetic Forecast (RTKP) gfe Graphical Forecast Editor grids ghcd SWPC Generic High Cadence Data gpd NCEP Generic Point Data grid Binary gridded data grib1/grib2 idft Ice Drift Forecasts madis NCEP Meteorological Assimilation Data Ingest System ( MADIS ) manualIngest Manual data ingest plugin metartohmdb Adds metar records to the Verification and Climate database modelsounding Individual grid point soundings from the GFS and NAM models mping Meteorological Phenomena Identification Near the Ground ( mPING ) ncpafm Point/Area Forecast Matrices data nctext NCEP Text decoders ncuair NCEP Upper Air decoder ndm National Dataset Maintenance ingester ntrans NCCEP Ntrans Metafiles obs Surface observations from METARs pgen NCEP NAWIPS PGEN decoder redbook Redbook graphics sfcobs Surface observations other than METAR format including buoys solarimage SWPC Solar imagery ssha NCEP Sea Surface Height Anomaly BUFR data text Various Text Products vaa Volcanic ash advisories",
"text":"NAME DESCRIPTION binlightning Lightning data from the National Lightning Detection Network bufrascat Advanced Scatterometer wind data bufrhdw GOES High Density Winds bufrmthdw MTSAT (Japanese Multi-Functional Transport Satellite) High Density Winds bufrssmi Special Sensor Microwave/Imager data from DMSP (Defesne Meteorological Satellite Program) satellites crimss NPP/NPOESS CrIMSS (Cross Track Infrared and Microwave Sounding Suite) soundings dmw GOES-R Derived Motion Winds glm GOES Geostationary Lightning Mapper goesr Plugins to decode and display GOES-R products goessounding GOES Satellite Soundings lma Lightning Mapping Array mcidas NCEP decoder for McIDAS AREA files modis NASA Moderate-resolution Imaging Spectroradiometer ncscat NCEP ASCAT/Quikscat records npp National Polar-Orbiting Partnership Satellites Soundings nucaps Soundings from NOAA Unique CrIS/ATMS Processing System from NPP (National Polar-Orbiting Partnership) Satellites poessounding Polar Operational Environmental Satellite soundings radar WSR-88D and TDWR Level 3 data regionalsat Decoder implementation for netcdf3 files generated by the Alaska Region and GOES-R Proving Ground satellite-gini GINI-formatted satellite imagery (GOES, POES, VIIRS, FNEXRAD) satellite-mcidas McIDAS area files (Raytheon/D2D-developed) viirs NPP Visible Infrared Imaging Radiometer Suite data sgwh NCEP BUFR Significant Wave Height data - SGWH (Jason-1), SGWHA (Altika), SGWHC (CryoSat), SGWHE (Envisat), SGWHG (GFO), SGWH2 (Jason-2), or Jason-3 textlightning Text lightning data",
"text":"NAME DESCRIPTION atcf Automated Tropical Cyclone Forecast convectprob NOAA/CIMSS Prob Severe Model editedregions Hazard Services Edited Regions editedevents Hazard Services Edited Events cwat County Warning Area Threat produced by SCAN (System for Convection Analysis and Nowcasting). CWAT was formerly called SCAN Convective Threat Index (SCTI). ffg Flash flood guidance metadata (countybased ffg from RFCs) ffmp Flash Flood Monitoring and Prediction data (raw data inputs: radar, gridded flash flood guidance from River Forecast Centers, highresolution precipitation estimates [HPE] and nowcasts [HPN], QPF from SCAN and gage data from the IHFS [Integrated Hydrologic Forecast System] database. Radar data [with WSR-88D product mnemonics and numbers] needed for FFMP are Digital Hybrid Reflectivity [DHR, 32] and Digital Precipitation Rate [DPR, 176]. The raw GRIB files containing RFC Flash Flood Guidance are identified in the tables in Part 2 of this document as NWS_151 or FFG-XXX, where XXX is an RFC identifier such as TUA, KRF, or ALR. The WMO header for the RFC FFG begins with \u201cZEGZ98\u201d. ) fog Fog Monitor . Raw data inputs: METAR, Mesonet, maritime, buoys, MAROBs, and satellite [visible, 3.9 \u00b5m, and 10.7 \u00b5m]) freezingLevel MPE Rapid Refresh Freezing Level scheduled process (MpeRUCFreezingLevel) fssobs Observations for the Fog monitor, SNOW, and SAFESEAS (raw data inputs: METAR, Mesonet, maritime, buoys, MAROBs). lsr Local Storm Reports mpe Multi-sensor Precipitation Estimation preciprate Precipitation Rate from SCAN. Raw data input: radar data [with WSR-88D product mnemonic and number] needed for preciprate are Digital Hybrid Reflectivity [DHR, 32]. qpf Quantitative Precipitation Forecast from SCAN. (raw data inputs: radar and some RAP13 fields. Radar data [with WSR-88D product mnemonics and numbers] needed for SCAN\u2019s QPF are 0.5 degree Base Reflectivity [Z, 19], 4 km Vertically Integrated Liquid [VIL, 57], and Storm Track [STI, 58]. The RAP13 field needed is 700 mb Wind, as defined in the SCANRunSiteConfig.xml file.) satpre Satellite-estimated Pecipiration (hydroApps) scan SCAN (System for Convection Analysis and Nowcasting). (Inputs for the SCAN Table include radar, cloud-to-ground lightning from the NLDN, fields from RAP13, and CWAT. Specific radar products [with WSR-88D product mnemonics and numbers] are: 1 km Composite Reflectivity [CZ, 37]; 0.5 degree Base Reflectivity [Z, 19]; 4 km Vertically Integrated Liquid [VIL, 57]; Storm Track [STI, 58]; Mesocyclone Detections [MD, 141]; and Tornadic Vortex Signature [TVS, 61]. shef Standard Hydrometeorological Exchange Format data. warning Watches, Warnings, and Advisories wcp SPC Convective Watches svrwx SPC Local Storm Report Summaries tcg Tropical Cyclone Guidance tcm Tropical Cyclone Forecast/Advisory tcs Tropical Cyclone Forecast/Advisory stormtrack NCEP StormTrack Plug-In (Automatic Tropical Cyclone Forecast & Ensemble cyclones) vil Cell-based Vertically Integrated Liquid from SCAN (Input is radar) spc Storm Prediction Center Convective Outlook KML files",
"text":"NAME DESCRIPTION acars Aircraft Communications Addressing and Reporting System (ACARS) observations acarssounding Vertical profiles derived from ACARS data airep Automated Aircraft Reports airmet \u201cAirmen\u2019s Meteorological Information\u201d: aviation weather advisories for potentially hazardous, but non-severe weather asdi FAA Aircraft Situation Data for Industry aww Airport Weather Warning bufrncwf National Convective Weather Forecast for Aviation bufrsigwx Aviation Significant Weather ccfp Aviation Collaborative Convective Forecast Product convsigmet Aviation Significant Meteorological Information for convective weather cwa Aviation Center Weather Advisory, issued by CWSUs (Center Weather Service Units) intlsigmet International Significant Meteorological Information for Aviation nctaf NCEP TAF decoders nonconvsigmet Aviation Significant Meteorological Information for non-convective weather pirep Pilot Reports taf Terminal Aerodrome Forecasts",
"text":"Case Study Server Configuration\n\uf0c1\n\n\nThis document covers what is necessary to install and run AWIPS EDEX as an archive and case study server (no purging of processed data).\n\n\n\n\nQuick Install\n\uf0c1\n\n\nFollow the \nEDEX Install Instructions\n including iptables config and an optional SSD mount (for large data volumes).\n\n\ngroupadd fxalpha && useradd -G fxalpha awips\nmkdir -p /awips2/data_store\nwget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/el7.repo\nyum clean all\nyum groupinstall awips2-server -y\n\n\n\n\n\nDisable Data Purging\n\uf0c1\n\n\nThe easiest way to disable data purging is to add an \n<exclude>purge.*</exclude>\n entry in \n/awips2/edex/conf/modes/modes.xml\n so that the purge plugin is not loaded when the EDEX ingest JVM is started:\n\n\nvi /awips2/edex/conf/modes/modes.xml\n\n<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<edexModes>\n <mode name=\"ingest\">\n <exclude>.*request.*</exclude>\n <exclude>edex-security.xml</exclude>\n ...\n <exclude>purge.*</exclude>\n </mode>\n ...\n</edexModes>\n\n\n\n\n\nStart EDEX\n\uf0c1\n\n\nStart EDEX without running the LDM, since we do not want current data. Run the following command:\n\n\nedex start base\n\n\n\nDouble check everything is running, except the LDM:\n\n\nedex\n\n[edex status]\n postgres :: running :: pid 43644\n pypies :: running :: pid 3557\n qpid :: running :: pid 43742\n EDEXingest :: running :: pid 6564 44301 44597\n EDEXgrib :: running :: pid 6565 44302 44598\n EDEXrequest :: running :: pid 6566 44303 44599\n ldmadmin :: not running\n\n\n\n\n\nIngest Case Study Data\n\uf0c1\n\n\nRaw data files of any type can be copied or moved into \n/awips2/data_store/ingest/\n to be picked up and decoded by EDEX. Most data types are recognized by regular expression matching of the WMO Header or filename. \n\n\nIndividual files can be ingested on the command line with the regex header/pattern supplied as the last argument:\n\n\nqpidNotify.py /full/path/to/data.file [regex match]\n\n\n\nFor example:\n\n\nqpidNotify.py /home/awips/uniwisc_U5_132GOES-15_IMG10.7um_4km_20171024_1830.area.png uniwisc\n\nqpidNotify.py /awips2/data_store/grid/NAM12/conduit/NAM_CONUS_12km_conduit_20171025_1200Z_F084_TMPK-7.000007.grib2 grib\n\nqpidNotify.py /awips2/data_store/radar/FTG_N0Q_20171015_1815 Level3\n\n\n\n\n\nViewing Archive Data in CAVE\n\uf0c1\n\n\nBecause we are installing and configuring a standalone EDEX archive server without real-time LDM data ingest (and with purge disabled), any case study data that is ingested will be the \"latest available\" to CAVE, and you will see CAVE product menu time fill in with the latest of all data ingested.\n\n\nHowever, to display specific time-based data (in case you ingest more than one case study), there are two options:\n\n\nSet Load Mode to Inventory\n\uf0c1\n\n\nIn the top-left toolbar change \nValid time seq\n to \nInventory\n.\n\n\n\n\nNow any data product selected from the menus or the Product Browser should prompt you to select the exact time.\n\n\n\n\nSet Data Display Time in CAVE\n\uf0c1\n\n\nAt the bottom of the CAVE application, double-click the \nTime:\n entry to bring up a dialog window where you can set CAVE to a previous time, and choose the option of freezing CAVE at that time or allowing CAVE to \"move forward in time\" from that position as if it were real-time.",
"text":"Follow the EDEX Install Instructions including iptables config and an optional SSD mount (for large data volumes). groupadd fxalpha && useradd -G fxalpha awips\nmkdir -p /awips2/data_store\nwget -O /etc/yum.repos.d/awips2.repo https://www.unidata.ucar.edu/software/awips2/doc/el7.repo\nyum clean all\nyum groupinstall awips2-server -y",
"text":"The easiest way to disable data purging is to add an <exclude>purge.*</exclude> entry in /awips2/edex/conf/modes/modes.xml so that the purge plugin is not loaded when the EDEX ingest JVM is started: vi /awips2/edex/conf/modes/modes.xml\n\n<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<edexModes>\n <mode name=\"ingest\">\n <exclude>.*request.*</exclude>\n <exclude>edex-security.xml</exclude>\n ...\n <exclude>purge.*</exclude>\n </mode>\n ...\n</edexModes>",
"text":"Raw data files of any type can be copied or moved into /awips2/data_store/ingest/ to be picked up and decoded by EDEX. Most data types are recognized by regular expression matching of the WMO Header or filename. Individual files can be ingested on the command line with the regex header/pattern supplied as the last argument: qpidNotify.py /full/path/to/data.file [regex match] For example: qpidNotify.py /home/awips/uniwisc_U5_132GOES-15_IMG10.7um_4km_20171024_1830.area.png uniwisc\n\nqpidNotify.py /awips2/data_store/grid/NAM12/conduit/NAM_CONUS_12km_conduit_20171025_1200Z_F084_TMPK-7.000007.grib2 grib\n\nqpidNotify.py /awips2/data_store/radar/FTG_N0Q_20171015_1815 Level3",
"text":"Because we are installing and configuring a standalone EDEX archive server without real-time LDM data ingest (and with purge disabled), any case study data that is ingested will be the \"latest available\" to CAVE, and you will see CAVE product menu time fill in with the latest of all data ingested. However, to display specific time-based data (in case you ingest more than one case study), there are two options:",
"text":"In the top-left toolbar change Valid time seq to Inventory . Now any data product selected from the menus or the Product Browser should prompt you to select the exact time.",
"text":"At the bottom of the CAVE application, double-click the Time: entry to bring up a dialog window where you can set CAVE to a previous time, and choose the option of freezing CAVE at that time or allowing CAVE to \"move forward in time\" from that position as if it were real-time.",
"text":"AWIPS Development Environment (ADE)\n\uf0c1\n\n\nQuick instructions on how to download the latest source code and run CAVE from Eclipse.\n\n\n\n\nNote\n: It is important to keep in mind these instructions are intended for a system that is specifically used for developing AWIPS. It should not be used in conjunction with installed production versions of AWIPS.\n\n\n\n\n1. Remove AWIPS Instances\n\uf0c1\n\n\nFirst, make sure to remove any instances of AWIPS that are already installed, this can potentially cause problems when setting up the development environment. Below is an example that had CAVE installed.\n\n\nUninstall with yum:\n\n\nyum clean all\nyum groupremove awips2-cave\n\n\n\nCheck to make sure all rpms have been removed:\n\n\nrpm -qa | grep awips2\n\n\n\nRemove the awips2 directory:\n\n\nrm -rf /awips2\n\n\n\n\n\n2. Set Up AWIPS Repo\n\uf0c1\n\n\nCreate a repo file named \n/etc/yum.repos.d/awips2.repo\n, and set the contents to the following:\n\n\n\n[awips2repo]\nname=AWIPS II Repository\nbaseurl=https://www.unidata.ucar.edu/repos/yum/\nel7-dev\n/\nenabled=1\nprotect=0\ngpgcheck=0\nproxy=_none_\n\n\n\n\n\nNote\n: This file may already exist if AWIPS had been previously installed on the machine, so make sure to edit the baseurl.\n\n\n\n\n\n\n3. Install the ADE\n\uf0c1\n\n\nInstall the AWIPS Development Environment (ADE) using yum. This will install Eclipse (4.6.1), Java (1.8), Ant (1.9.6), Python 2.7 and its modules (Numpy, Matplotlib, Shapely, Jep, and others). \n\n\nyum clean all\nyum groupinstall awips2-ade\n\n\n\n\n\n4. Download the Source Code\n\uf0c1\n\n\nIf it's not already installed, install git:\n\n\nyum install git\n\n\n\nNext clone all of the required repositories for AWIPS:\n\n\ngit clone https://github.com/Unidata/awips2.git\ngit clone https://github.com/Unidata/awips2-core.git\ngit clone https://github.com/Unidata/awips2-core-foss.git\ngit clone https://github.com/Unidata/awips2-foss.git\ngit clone https://github.com/Unidata/awips2-ncep.git\ngit clone https://github.com/Unidata/awips2-nws.git\ngit clone https://github.com/Unidata/awips2-gsd.git\ngit clone https://github.com/Unidata/awips2-drawing.git\ngit clone https://github.com/Unidata/awips2-cimss.git\n\n\n\n\n\n5. Set Up Eclipse\n\uf0c1\n\n\nOpen eclipse by running: \n/awips2/eclipse/eclipse.sh\n\n\nVerify or make the following changes to set up eclipse for AWIPS development:\n\n\n\n\n\n\nPreferences > Java \n\n\nSet to \n/awips2/java\n\n\n\n\n\n\nPreferences > PyDev > Python Interpreter\n\n\nSet to \n/awips2/python/bin/python\n\n\n\n\n\n\nThere might be some unresolved errors. These should be made to warnings instead.\n\n\nPreferences > Java > Compiler > Building > \nCircular Dependencies\n > Change to Warning\nPreferences > Plug-in Development > API Baselines > \nMissing API Baseline\n > Change to Warning\n\n\n\n\n\n\nTurn off automatic building\n (you will turn this back on after importing the repos)\n\n\nProject > Uncheck \"Build Automatically\"\n\n\n\n\n\n\nFile > Import > General > Existing Projects Into Workspace\n\n\nImport all of the git cloned project folders \nEXCEPT\n for the main (first) \ngithub.com/Unidata/awips2.git\n directory (which should be \n~/awips2\n).\n\nSelect \nawips2-core\n, \nawips2-core-foss\n, \nawips2-foss\n, \nawips2-ncep\n, etc. > Select All Projects > Finish \n\n\nYou'll want to import \n~/awips2\n in two parts to ensure a clean and error-free Eclipse build:\n\n\n\n\nImport \nawips2/cave\n > Select All Projects > Finish\n\n\nImport \nawips2/edexOsgi\n > Select All Projects > Finish\n\n\n\n\n\n\n\n\nProject > Clean\n\n\nClean the build and ensure no errors are reported. \n\n\n\n\n\n\nTurn automatic building back on\n\n\nProject > Check \"Build Automatically\"\n\n\n\n\n\n\n\n\n6.RunCAVE\n\uf0c1\n\n\nLaunchCAVEfromeclipseusing\ncom.raytheon.viz.product.awips/developer.product\n.\n\n\nDouble-clickthe\ndeveloper.product\nfiletoopentheProductViewinEclipse.Select\nOverview\n>\nSynchronize\nandthenright-clickthefileintheleft-sidepackageexplorer:\n\n\nSelect\nRu
"text":"Quick instructions on how to download the latest source code and run CAVE from Eclipse. Note : It is important to keep in mind these instructions are intended for a system that is specifically used for developing AWIPS. It should not be used in conjunction with installed production versions of AWIPS.",
"text":"First, make sure to remove any instances of AWIPS that are already installed, this can potentially cause problems when setting up the development environment. Below is an example that had CAVE installed. Uninstall with yum: yum clean all\nyum groupremove awips2-cave Check to make sure all rpms have been removed: rpm -qa | grep awips2 Remove the awips2 directory: rm -rf /awips2",
"text":"Create a repo file named /etc/yum.repos.d/awips2.repo , and set the contents to the following: \n[awips2repo]\nname=AWIPS II Repository\nbaseurl=https://www.unidata.ucar.edu/repos/yum/ el7-dev /\nenabled=1\nprotect=0\ngpgcheck=0\nproxy=_none_ Note : This file may already exist if AWIPS had been previously installed on the machine, so make sure to edit the baseurl.",
"text":"Install the AWIPS Development Environment (ADE) using yum. This will install Eclipse (4.6.1), Java (1.8), Ant (1.9.6), Python 2.7 and its modules (Numpy, Matplotlib, Shapely, Jep, and others). yum clean all\nyum groupinstall awips2-ade",
"text":"Open eclipse by running: /awips2/eclipse/eclipse.sh Verify or make the following changes to set up eclipse for AWIPS development: Preferences > Java Set to /awips2/java Preferences > PyDev > Python Interpreter Set to /awips2/python/bin/python There might be some unresolved errors. These should be made to warnings instead. Preferences > Java > Compiler > Building > Circular Dependencies > Change to Warning\nPreferences > Plug-in Development > API Baselines > Missing API Baseline > Change to Warning Turn off automatic building (you will turn this back on after importing the repos) Project > Uncheck \"Build Automatically\" File > Import > General > Existing Projects Into Workspace Import all of the git cloned project folders EXCEPT for the main (first) github.com/Unidata/awips2.git directory (which should be ~/awips2 ). \nSelect awips2-core , awips2-core-foss , awips2-foss , awips2-ncep , etc. > Select All Projects > Finish You'll want to import ~/awips2 in two parts to ensure a clean and error-free Eclipse build: Import awips2/cave > Select All Projects > Finish Import awips2/edexOsgi > Select All Projects > Finish Project > Clean Clean the build and ensure no errors are reported. Turn automatic building back on Project > Check \"Build Automatically\"",
"text":"Launch CAVE from eclipse using com.raytheon.viz.product.awips/developer.product . Double-click the developer.product file to open the Product View in Eclipse. Select Overview > Synchronize and then right-click the file in the left-side package explorer: Select Run As > Eclipse Application to launch CAVE in the development environment. Select Debug > Eclipse Application to launch CAVE in in debug mode.",
"text":"Python API\n\uf0c1\n\n\nThe \npython-awips\n package provides a data access framework (DAF) for requesting grid and geometry datasets from an EDEX server. The DAF can be used to interact in command line, or scripting form, with the EDEX. This is an alternative to using CAVE to interact with the data.\n\n\nThorough documentation for python-awips can be found \nhere\n.",
"title":"Python API"
},
{
"location":"/python/overview/#python-api",
"text":"The python-awips package provides a data access framework (DAF) for requesting grid and geometry datasets from an EDEX server. The DAF can be used to interact in command line, or scripting form, with the EDEX. This is an alternative to using CAVE to interact with the data. Thorough documentation for python-awips can be found here .",
"text":"Column Type arpt_id character varying(4) name character varying(42) city character varying(40) state character varying(2) siteno character varying(9) site_type character varying(1) fac_use character varying(2) owner_type character varying(2) elv integer latitude character varying(16) longitude character varying(16) lon double precision lat double precision the_geom geometry(Point,4326) ok",
"text":"Column Type ihabbsrf_i double precision rr character varying(11) huc integer type character varying(1) pmile double precision pname character varying(30) owname character varying(30) pnmcd character varying(11) ownmcd character varying(11) dsrr double precision dshuc integer usdir character varying(1) lev smallint j smallint termid integer trmblv smallint k smallint the_geom geometry(MultiLineString,4326)",
"text":"Column Type artcc character varying(4) alt character varying(1) name character varying(30) type character varying(5) city character varying(40) id double precision the_geom geometry(MultiPolygon,4326)",
"text":"Column Type rfc character varying(7) cwa character varying(5) id character varying(8) name character varying(64) lon double precision lat double precision the_geom geometry(MultiPolygon,4326) the_geom_0 geometry(MultiPolygon,4326) the_geom_0_064 geometry(MultiPolygon,4326) the_geom_0_016 geometry(MultiPolygon,4326) the_geom_0_004 geometry(MultiPolygon,4326) the_geom_0_001 geometry(MultiPolygon,4326)",
"text":"Column Type f_code character varying(5) name_en character varying(25) nom_fr character varying(25) country character varying(3) cgns_fid character varying(32) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type st_fips character varying(4) sfips character varying(2) county_fip character varying(4) cfips character varying(4) pl_fips character varying(7) id character varying(20) name character varying(39) elevation character varying(60) pop_1990 numeric population character varying(30) st character varying(6) warngenlev character varying(16) warngentyp character varying(16) watch_warn character varying(3) zwatch_war double precision prog_disc integer zprog_disc double precision comboflag double precision land_water character varying(16) recnum double precision lon double precision lat double precision f3 double precision f4 character varying(254) f6 double precision state character varying(25) the_geom geometry(Point,4326)",
"text":"Column Type state character varying(2) cwa character varying(9) countyname character varying(24) fips character varying(5) time_zone character varying(2) fe_area character varying(2) lon numeric lat numeric the_geom geometry(MultiPolygon,4326)",
"text":"Column Type bullet character varying(16) name character varying(64) cwa character varying(12) rfc character varying(8) lon numeric lat numeric the_geom geometry(MultiPolygon,4326)",
"text":"Column Type cwa character varying(9) wfo character varying(3) lon numeric lat numeric region character varying(2) fullstaid character varying(4) citystate character varying(50) city character varying(50) state character varying(50) st character varying(2) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type state character varying(2) zone character varying(3) cwa character varying(3) name character varying(254) state_zone character varying(5) time_zone character varying(2) fe_area character varying(2) lon numeric lat numeric the_geom geometry(MultiPolygon,4326)",
"text":"Column Type id character varying(30) type character varying(2) use character varying(5) state character varying(2) min_alt integer latitude character varying(16) longitude character varying(16) lon double precision lat double precision the_geom geometry(Point,4326)",
"text":"Column Type awy_des character varying(2) awy_id character varying(12) awy_type character varying(1) airway character varying(16) newfield1 double precision the_geom geometry(MultiLineString,4326)",
"text":"Column Type wfo character varying(3) name character varying(250) lat numeric lon numeric id character varying(5) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type prefix character varying(2) pretype character varying(6) name character varying(30) type character varying(6) suffix character varying(2) class character varying(1) class_rte character varying(1) hwy_type character varying(1) hwy_symbol character varying(20) route character varying(25) the_geom geometry(MultiLineString,4326)",
"text":"Column Type prefix character varying(2) pretype Ushy Hwy Ave Cord Rt Loop I Sthy name character varying(30) type character varying(6) suffix character varying(2) hwy_type I U S hwy_symbol character varying(20) route character varying(25) the_geom geometry(MultiLineString,4326)",
"text":"Column Type name character varying(40) feature character varying(40) lon double precision lat double precision the_geom geometry(MultiPolygon,4326)",
"text":"Column Type awy_des character varying(2) awy_id character varying(12) awy_type character varying(1) airway character varying(16) newfield1 double precision the_geom geometry(MultiLineString,4326)",
"text":"Column Type st character varying(3) name character varying(50) prog_disc bigint warngenlev character varying(14) the_geom geometry(Point,4326)",
"text":"Column Type id character varying(6) wfo character varying(3) gl_wfo character varying(3) name character varying(254) ajoin0 character varying(6) ajoin1 character varying(6) lon numeric lat numeric the_geom geometry(MultiPolygon,4326)",
"text":"Column Type area double precision perimeter double precision st_mx_ double precision st_mx_id double precision name character varying(66) country character varying(127) continent character varying(127) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type id character varying(30) clscode character varying(11) city character varying(40) elv integer freq double precision name character varying(30) status character varying(30) type character varying(25) oprhours character varying(11) oprname character varying(50) latdms character varying(16) londms character varying(16) airway character varying(254) sym smallint lon double precision lat double precision the_geom geometry(Point,4326)",
"text":"Column Type id character varying(50) wfo character varying(10) lon numeric lat numeric location character varying(70) name character varying(90) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type site_id character varying(3) state character varying(2) rfc_name character varying(18) rfc_city character varying(25) basin_id character varying(5) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type name character varying(32) code character varying(16) yn smallint alt_desc character varying(128) artcc character varying(4) ctr_agen character varying(128) sch_agen character varying(128) state character varying(2) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type state character varying(2) name character varying(24) fips character varying(2) lon numeric lat numeric the_geom geometry(MultiPolygon,4326)",
"text":"Column Type name character varying(50) time_zone character varying(1) standard character varying(9) advanced character varying(10) unix_time character varying(19) lon double precision lat double precision the_geom geometry(MultiPolygon,4326)",
"text":"Column Type name character varying(254) st character varying(3) state character varying(20) population integer warngenlev integer cwa character varying(4) goodness double precision lat numeric lon numeric usedirs numeric(10,0) supdirs character varying(20) landwater character varying(3) recnum integer the_geom geometry(MultiPolygon,4326)",
"text":"Column Type name character varying(30) count double precision first_coun character varying(2) first_regi character varying(1) the_geom geometry(MultiPolygon,4326)",
"text":"Column Type state character varying(2) cwa character varying(9) time_zone character varying(2) fe_area character varying(2) zone character varying(3) name character varying(254) state_zone character varying(5) lon numeric lat numeric shortname character varying(32) the_geom geometry(MultiPolygon,4326)",
"text":"D2D - Display 2 Dimensions DFM - Digital Forecast Matrix DMD - Digital Mesocyclone Display DMS - Data Monitoring System DOC - Department of Commerce DPA - Digital Precipitation Array",
"text":"ECMWF - European Centre for Medium-Range Forecasts EDEX - Environmental Data EXchange EMC - Environmental Modeling Center EL - Equilibrium Level ESA - Electronic Systems Analyst ESRL - Earth System Research Laboratory",
"text":"GFE - Graphical Forecast Editor GFS - Global Forecasting Systems GHG - Graphical Hazards Generator GIS - Geographic Information Systems GMT - Greenwich Mean Time GOES - Geostationary Operational Environmental Satellite GSD - Global System Division",
"text":"ICAO - International Civil Aviation Organization IFP - Interactive Forecast Program IFPS - Interactive Forecast Preparation System IHFS - Integrated Hydrologic Forecast System IMET - Incident Meteorologist IR - infrared ISS - Incident Support Specialist IST - Interactive Skew-T",
"text":"LAC - Listening Area Code LAMP - Localized Aviation MOS Program LAN - Local Area Network LAPS - Local Analysis and Prediction System LARC - Local Automatic Remote Collector LCL - Lifting Condensation Level LDAD - Local Data Acquisition and Dissemination LFC - Level of Free Convection LSR - Local Storm Report",
"text":"NAM - North American Mesoscale model NCEP - National Centers for Environmental Prediction NCF - Network Control Facility NDFD - National Digital Forecast Database NE-PAC - Northeastern Pacific NESDIS - National Environmental Satellite, Data and Information Service NH - Northern Hemisphere nMi - nautical miles NOAA - National Oceanic and Atmospheric Administration NPN - NOAA Profiler Network NPP - Suomi National Polar-orbiting Partnership NUCAPS - NOAA Unique CrIS/ATMS Processing Systems NWP - Numerical Weather Prediction NWR - NOAA Weather Radio NWS - National Weather Service NWRWAVES - NOAA Weather Radio With All-Hazards VTEC Enhanced Software NWSRFS - National Weather Service River Forecast System NWWS - NOAA Weather Wire Service",
"text":"OCP - Ocean Prediction Center OH - Office of Hydrology OPC - Ocean Prediction Center ORPG - Open Radar Products Generator OSD - One Hour Snow Depth OSW - One Hour Snow Water OTR - One Time Request",
"text":"PID - Product Identification PIL - Product Inventory List PIREP - Pilot Weather Report POES - Polar Operational Environmental Satellite POSH - Probability of Severe Hail POH - Probability of Hail POP - Probability of Precipitation PQPF - Probabilistic QPF PRF - Pulse Repetition Frequency",
"text":"SAFESEAS - System on AWIPS for Forecasting and Evaluation of Seas and Lakes SBN - Satellite Broadcast Network SCAN - System for Convection Analysis and Nowcasting SCD - Supplementary Climatological Data SCID - Storm Cell Identification Display SCP - Satellite Cloud Product SCTI - SCAN CWA Threat Index SDC - State Distribution Circuit SNOW - System for Nowcasting Of Winter Weather SOO - Science and Operations Officer SPC - Storm Prediction Center SPE - Satellite Precipitation Estimate SREF - Short Range Ensemble Forecast SRG - Supplemental Product Generator SRM - Storm Relative Motion SSD - Storm-Total Snow Depth SSM/I - Special Sensor Microwave/Imager SSW - Storm-Total Snow Water STI - Storm Track Information Suomi NPP - Suomi National Polar-orbiting Partnership SW - Spectrum Width SWEAT Index - Severe Weather Threat Index SWP - Severe Weather Probability",
"text":"Keyboard Shortcuts\n\uf0c1\n\n\n\n\n\nD2D Menu Shortcuts\n\uf0c1\n\n\n\n\n\n\n\n\nAction\n\n\nCommand\n\n\n\n\n\n\n\n\n\n\nOpen a New Map\n\n\nCtrl + N\n\n\n\n\n\n\nOpen a Bundle\n\n\nCtrl + O\n\n\n\n\n\n\nSave Bundle\n\n\nCtrl + S\n\n\n\n\n\n\nSave Bundle Locally\n\n\nCtrl + Shift + S\n\n\n\n\n\n\nSave KML\n\n\nCtrl + K\n\n\n\n\n\n\nExit CAVE\n\n\nAlt + F4\n\n\n\n\n\n\nExit CAVE\n\n\nCtrl + Q\n\n\n\n\n\n\nClear Data\n\n\nCtrl + C\n\n\n\n\n\n\nFirst Frame\n\n\nCtrl + \u2190\n\n\n\n\n\n\nLast Frame\n\n\nCtrl + \u2192\n\n\n\n\n\n\nStep Back\n\n\n\u2190\n\n\n\n\n\n\nStep Forward\n\n\n\u2190\n\n\n\n\n\n\nIncrease Loop Speed\n\n\nPage Up\n\n\n\n\n\n\nDecrease Loop Speed\n\n\nPage Down\n\n\n\n\n\n\nOpen Time Options\n\n\nCtrl + T\n\n\n\n\n\n\nToggle Image Combination\n\n\nInsert\n\n\n\n\n\n\nOpen Loop Properties\n\n\nCtrl + L\n\n\n\n\n\n\nOpen Image Properties\n\n\nCtrl + I\n\n\n\n\n\n\n\n\nD2D All Tilts Shortcuts\n\uf0c1\n\n\n\n\nNote\n: Requires all tilts product in main display panel\n\n\n\n\n\n\n\n\n\n\nAction\n\n\nCommand\n\n\n\n\n\n\n\n\n\n\nStep Back 1 Volume\n\n\n\u2190\n\n\n\n\n\n\nStep Forward 1 Volume\n\n\n\u2192\n\n\n\n\n\n\nStep up 1 Elevation Angle\n\n\n\u2191\n\n\n\n\n\n\nStep down 1 Elevation Angle\n\n\n\u2193\n\n\n\n\n\n\nJump to First Frame\n\n\nCtrl + \u2190\n\n\n\n\n\n\nJump to Last Frame\n\n\nCtrl + \u2192\n\n\n\n\n\n\nJump to Highest Elevation Angle\n\n\nCtrl + \u2191\n\n\n\n\n\n\nJump to Lowest Elevation Angle\n\n\nCtrl + \u2193\n\n\n\n\n\n\n\n\nD2D Numeric Keypad Shortcuts\n\uf0c1\n\n\n\n\nNote\n: Num Lock must be enabled for these keystrokes to work\n\n\n\n\n\n\n\n\n\n\nAction\n\n\nCommand\n\n\n\n\n\n\n\n\n\n\nIncrease Brightness of Image 1, Decrease Image 2\n\n\n[Numpad] +\n\n\n\n\n\n\nDecrease Brightness of Image 1, Increase Image 2\n\n\n[Numpad] -\n\n\n\n\n\n\nToggle Image Producted in Main Map On/Off\n\n\n[Numpad] 0\n\n\n\n\n\n\nToggle First 9 Graphic Products On/Off\n\n\n[Numpad] 1-9\n\n\n\n\n\n\nToggle Next 10 Graphic Prodcuts On/Off\n\n\nShift + [Numpad] 0-9\n\n\n\n\n\n\nToggle Between Images 1/2 at Full Brightness\n\n\n[Numpad] .\n\n\n\n\n\n\nToggle Legend\n\n\n[Numpad] Enter\n\n\n\n\n\n\n\n\nPanel Combo Rotate (PCR) Shortcuts\n\uf0c1\n\n\n\n\nNote\n: These numbers refer to the ones at the top of the Keyboard\n\n\n\n\n\n\n\n\n\n\nAction\n\n\nCommand\n\n\n\n\n\n\n\n\n\n\nCycle Through PCR Products\n\n\nDelete\n\n\n\n\n\n\nReturn to 4 Panel View\n\n\nEnd\n\n\n\n\n\n\nCycle Back Through PCR Products\n\n\nBackspace\n\n\n\n\n\n\nDisplay Corresponding Product\n\n\n1-8\n\n\n\n\n\n\n\n\nText Editor Shortcuts\n\uf0c1\n\n\n\n\n\n\n\n\nAction\n\n\nCommand\n\n\n\n\n\n\n\n\n\n\nExtend Selection to Start of Line\n\n\nShift + Home\n\n\n\n\n\n\nExtend Selection to End of Line\n\n\nShift + End\n\n\n\n\n\n\nExtend Selection to Start of Document\n\n\nCtrl + Shift + Home\n\n\n\n\n\n\nExtend Selection to End of Document\n\n\nCtrl + Shift + End\n\n\n\n\n\n\nExtend Selection Up 1 Screen\n\n\nShift + Page Up\n\n\n\n\n\n\nExtend Selection Down 1 Screen\n\n\nShift + Page Down\n\n\n\n\n\n\nExtend Selection to Previous Character\n\n\nShift + \u2190\n\n\n\n\n\n\nExtend Selection by Previous Word\n\n\nCtrl + Shift + \u2190\n\n\n\n\n\n\nExtend Selection to Next Character\n\n\nShift + \u2192\n\n\n\n\n\n\nExtend Selection by Next Word\n\n\nCtrl + Shift + \u2192\n\n\n\n\n\n\nExtend Selection Up 1 Line\n\n\nShift + \u2191\n\n\n\n\n\n\nExtend Selection Down 1 Line\n\n\nShift + \u2193\n\n\n\n\n\n\nDelete Previous Word\n\n\nCtrl + Backspace\n\n\n\n\n\n\nDelete Next Word\n\n\nCtrl + Delete\n\n\n\n\n\n\nClose the Window\n\n\nCtrl + Shift + F4\n\n\n\n\n\n\nUndo\n\n\nCtrl + Z\n\n\n\n\n\n\nCopy\n\n\nCtrl + C\n\n\n\n\n\n\nPaste\n\n\nCtrl + V\n\n\n\n\n\n\nCut\n\n\nCtrl + X",
"text":"Action Command Open a New Map Ctrl + N Open a Bundle Ctrl + O Save Bundle Ctrl + S Save Bundle Locally Ctrl + Shift + S Save KML Ctrl + K Exit CAVE Alt + F4 Exit CAVE Ctrl + Q Clear Data Ctrl + C First Frame Ctrl + \u2190 Last Frame Ctrl + \u2192 Step Back \u2190 Step Forward \u2190 Increase Loop Speed Page Up Decrease Loop Speed Page Down Open Time Options Ctrl + T Toggle Image Combination Insert Open Loop Properties Ctrl + L Open Image Properties Ctrl + I",
"text":"Note : Requires all tilts product in main display panel Action Command Step Back 1 Volume \u2190 Step Forward 1 Volume \u2192 Step up 1 Elevation Angle \u2191 Step down 1 Elevation Angle \u2193 Jump to First Frame Ctrl + \u2190 Jump to Last Frame Ctrl + \u2192 Jump to Highest Elevation Angle Ctrl + \u2191 Jump to Lowest Elevation Angle Ctrl + \u2193",
"text":"Note : Num Lock must be enabled for these keystrokes to work Action Command Increase Brightness of Image 1, Decrease Image 2 [Numpad] + Decrease Brightness of Image 1, Increase Image 2 [Numpad] - Toggle Image Producted in Main Map On/Off [Numpad] 0 Toggle First 9 Graphic Products On/Off [Numpad] 1-9 Toggle Next 10 Graphic Prodcuts On/Off Shift + [Numpad] 0-9 Toggle Between Images 1/2 at Full Brightness [Numpad] . Toggle Legend [Numpad] Enter",
"text":"Note : These numbers refer to the ones at the top of the Keyboard Action Command Cycle Through PCR Products Delete Return to 4 Panel View End Cycle Back Through PCR Products Backspace Display Corresponding Product 1-8",
"text":"Action Command Extend Selection to Start of Line Shift + Home Extend Selection to End of Line Shift + End Extend Selection to Start of Document Ctrl + Shift + Home Extend Selection to End of Document Ctrl + Shift + End Extend Selection Up 1 Screen Shift + Page Up Extend Selection Down 1 Screen Shift + Page Down Extend Selection to Previous Character Shift + \u2190 Extend Selection by Previous Word Ctrl + Shift + \u2190 Extend Selection to Next Character Shift + \u2192 Extend Selection by Next Word Ctrl + Shift + \u2192 Extend Selection Up 1 Line Shift + \u2191 Extend Selection Down 1 Line Shift + \u2193 Delete Previous Word Ctrl + Backspace Delete Next Word Ctrl + Delete Close the Window Ctrl + Shift + F4 Undo Ctrl + Z Copy Ctrl + C Paste Ctrl + V Cut Ctrl + X",
