xan/awips2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

checkout unidata_18.1.1-osx docs

Browse source
This commit is contained in:
mjames-upc 2018-09-05 11:41:31 -06:00
parent 78505033b4
commit 11c4f85d12
5 changed files with 120 additions and 90 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

35
docs/edex/distributed-computing.md
View file

@ -61,32 +61,15 @@ Note the line **`-A INPUT -s 10.0.0.7 -j EDEX`** as well as the following **`-A
In the file `/awips2/database/data/pg_hba.conf` you define remote connections for all postgres tables with as `<IP address>/32`, after the block of IPv4 local connections:
vi /awips2/database/data/pg_hba.conf
# IPv4 local connections:
host fxatext all 127.0.0.1/32 trust
host hd_ob92oax all 127.0.0.1/32 trust
host dc_ob7oax all 127.0.0.1/32 trust
host hmdb all 127.0.0.1/32 trust
host metadata all 127.0.0.1/32 md5
host maps all 127.0.0.1/32 md5
host postgres all 127.0.0.1/32 md5
host ncep all 127.0.0.1/32 md5
host ebxml all 127.0.0.1/32 trust
host replication replication 127.0.0.1/32 md5
# Remote connections
host fxatext all 10.0.0.7/32 md5
host hd_ob92oax all 10.0.0.7/32 md5
host dc_ob7oax all 10.0.0.7/32 md5
host hmdb all 10.0.0.7/32 md5
host metadata all 10.0.0.7/32 md5
host maps all 10.0.0.7/32 md5
host postgres all 10.0.0.7/32 md5
host ncep all 10.0.0.7/32 md5
host ebxml all 10.0.0.7/32 md5
host replication replication 10.0.0.7/32 md5
# IPv6 local connections:
host all all ::1/128 md5
host replication replication ::1/128 md5
# "local" is for Unix domain socket connections only
local all all trust
hostssl all all 10.0.0.7/32 cert clientcert=1
hostssl all all 162.0.0.0/8 cert clientcert=1
hostssl all all 127.0.0.1/32 cert clientcert=1
# IPv6 local connections:
hostssl all all ::1/128 cert clientcert=1
hostnossl all all ::1/128 md5
### 4. Start EDEX

106
docs/edex/settings.md Normal file
View file

@ -0,0 +1,106 @@
## Plugin Configuration
The directory `/awips2/edex/conf/resources` contains configuration text files for specific plugins, which allow for user-defined values which are read by AWIPS plugins on EDEX start:
com.raytheon.edex.plugin.gfe.properties
com.raytheon.edex.plugin.grib.properties
com.raytheon.edex.plugin.radar.properties
com.raytheon.edex.text.properties
com.raytheon.uf.edex.archive.cron.properties
com.raytheon.uf.edex.database.properties
com.raytheon.uf.edex.registry.ebxml.properties
distribution.properties
edex-localization-http.properties
edex-ogc.properties
edex-requestsrv.properties
goesr.properties
purge.properties
warning.properties
Look at *purge.properties* for example:
# Master switch to enable and disable purging
purge.enabled=true
# Interval at which the purge job kicks off
purge.cron=0+0/15+*+*+*+?
# Interval at which the outgoing files are purged
purge.outgoing.cron=0+30+*+*+*+?
# Interval at which the logs are purged
purge.logs.cron=0+30+0+*+*+?
# Interval at which hdf5 orphans are purged
purge.orphan.period=24h
# Number of days older than the earliest known data to delete.
purge.orphan.buffer=7
In *com.raytheon.edex.plugin.grib.properties*, *com.raytheon.edex.plugin.radar.properties*, and *com.raytheon.edex.plugin.radar.properties* you can adjust the number of decoder threads for each plugin.
cat com.raytheon.edex.plugin.radar.properties
# Number threads for radar products ingested from the SBN
radar-decode.sbn.threads=5
---
## Ingest Modes
By default, EDEX starts three "modes": *ingest*, *ingestGrib*, and *request* (each as its own JVM).
The file `/awips2/edex/conf/modes/modes.xml` contains all available mode definitions, including some specific modes for Hydro Server Applications, ebXML Registries, and Data Delivery.
EDEX services are registered through spring, and by including or excluding specific spring files (usually by datatype plugin name) we can finely customize EDEX startup.
In `/awips2/edex/conf/modes/modes.xml` there are a number of unused plugin decoders excluded because the data are not available outside of the SBN:
...
<mode name="ingest">
<exclude>.*request.*</exclude>
<exclude>edex-security.xml</exclude>
<exclude>ebxml.*\.xml</exclude>
<exclude>grib-decode.xml</exclude>
<exclude>grid-staticdata-process.xml</exclude>
<exclude>.*(taf|nctext).*</exclude>
<exclude>webservices.xml</exclude>
<exclude>ebxml.*\.xml</exclude>
<exclude>.*datadelivery.*</exclude>
<exclude>.*bandwidth.*</exclude>
<exclude>.*sbn-simulator.*</exclude>
<exclude>grid-metadata.xml</exclude>
<exclude>.*ogc.*</exclude>
</mode>
...
In this example, OGC, Data Delivery, request, ebXML, and grib plugins are excluded because they are included in their own mode/JVM.
> TAF and NCTEXT plugins are disabled here due to performance issues.
---
## JVM Memory
The directory `/awips2/edex/etc/` contains files which define the amount of memory used for each of the three EDEX JVMs (ingest, ingestGrib, request):
ls -al /awips2/edex/etc/
-rw-r--r-- 1 awips fxalpha 1287 Jul 24 18:41 centralRegistry.sh
-rw-r--r-- 1 awips fxalpha 1155 Jul 24 18:42 default.sh
-rw-r--r-- 1 awips fxalpha 1956 Jul 24 18:41 ingestGrib.sh
-rw-r--r-- 1 awips fxalpha 337 Jul 24 18:36 ingest.sh
-rw-r--r-- 1 awips fxalpha 848 Jul 24 18:42 profiler.sh
-rw-r--r-- 1 awips fxalpha 1188 Jul 24 18:41 registry.sh
-rw-r--r-- 1 awips fxalpha 601 Jul 24 18:36 request.sh
-rw-r--r-- 1 awips fxalpha 1124 Jul 23 17:22 sbnSimulator.sh
Each file contains the **Xmx** definition for maximum memory:
export INIT_MEM=512 # in Meg
export MAX_MEM=4096 # in Meg
After editing these files, you must restart edex (`service edex_camel restart`).
---

6
docs/install/install-edex.md
View file

@ -5,7 +5,7 @@
| | |
|:----------------------------------------:|:--|
| <h1><i class="fa fa-linux"></i> Linux </h1> | <h4>[install.sh --edex <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/software/awips2/install.sh) </h4> <p>Installs to /awips2/ directories.</p> <tt><code>chmod 755 install.sh<br>sudo ./install.sh --edex</code></tt><p>Start and Stop:</p><p><tt>edex start<br>edex stop</tt></p><div class="admonition note"><p class="admonition-title">System Requirements</p><ul><li>x86_64 CentOS/RHEL 6 or 7</li><li>16+ CPU cores (each CPU core is one more decoder which can run in parallel)</li><li>24GB RAM</li><li>700GB+ disk space</li><li>A **Solid State Drive (SSD)** is highly recommended</li></ul></div> <p>An **SSD** should be mounted either to `/awips2` (to contain the entire EDEX system) or to `/awips2/edex/data/hdf5` (to contain the large files in the decoded data store). EDEX can scale to any system by adjusting the incoming LDM data feeds or adjusting the resources (CPU threads) allocated to each data type.</p><p>**64-bit CentOS/RHEL 6 and 7** are the only supported operating systems for EDEX. You may have luck with Fedora Core 12 to 14 and Scientific Linux.</p><p>EDEX is not supported on Debian, Ubuntu, SUSE, Solaris, OS X, or Windows.</p>|
| <h1><i class="fa fa-linux"></i> Linux </h1> | <h4>[install.sh --edex <i class="fa fa-download"></i>](https://www.unidata.ucar.edu/software/awips2/install.sh) </h4> <p>Installs to /awips2/ directories.</p> <tt><code>chmod 755 install.sh<br>sudo ./install.sh --edex</code></tt><p>Start and Stop:</p><p><tt>edex start<br>edex stop</tt></p><div class="admonition note"><p class="admonition-title">System Requirements</p><ul><li>x86_64 CentOS/RHEL 6 or 7</li><li>16+ CPU cores (each CPU core is one more decoder which can run in parallel)</li><li>24GB RAM</li><li>700GB+ disk space</li><li>A **Solid State Drive (SSD)** is highly recommended</li></ul></div> <p>An **SSD** should be mounted either to `/awips2` (to contain the entire EDEX system) or to `/awips2/edex/data/hdf5` (to contain the large files in the decoded data store). EDEX can scale to any system by adjusting the incoming LDM data feeds or adjusting the resources (CPU threads) allocated to each data type.</p><p>**64-bit CentOS/RHEL 6 and 7** are the only supported operating systems for EDEX. You may have luck with Fedora Core 12 to 14 and Scientific Linux.</p><p>EDEX is not supported on Debian, Ubuntu, SUSE, Solaris, OS X, or Windows.</p> |
> ### [Read More: Distributed EDEX, Installing Across Multiple Machines](/edex/distributed-computing/)
@ -87,6 +87,8 @@ All of these command should be run as **root**
>
>- **To open ports to specific IP addresses**
>
> In this example, the IP range `128.117.140.0/24` will match all 128.117.140.* addresses, while `128.117.156.0/24` will match 128.117.156.*.
>
> vi /etc/sysconfig/iptables
>
> *filter
@ -108,8 +110,6 @@ All of these command should be run as **root**
> #-A EDEX -m state --state NEW -p tcp --dport 9588 -j ACCEPT # for registry/dd
> -A EDEX -j REJECT
> COMMIT
>
> In this example, the IP range `128.117.140.0/24` will match all 128.117.140.* addresses, while `128.117.156.0/24` will match 128.117.156.*.
>
>**Restart iptables**
>

62
docs/install/start-edex.md
View file

@ -90,64 +90,4 @@ To see a list of clients connecting to your EDEX server, use the `edex users [YY
### edex purge
To view any stuck purge jobs in PortgreSQL (a rare but serious problem if your disk fills up). The solution to this is to run `edex purge reset`.
---
## EDEX Memory Configuration
The directory `/awips2/edex/etc/` contains files which define the amount of memory used for each of the three EDEX JVMs (ingest, ingestGrib, request):
ls -al /awips2/edex/etc/
-rw-r--r-- 1 awips fxalpha 1501 Dec 7 00:37 default.sh
-rw-r--r-- 1 awips fxalpha 1655 Dec 12 19:47 ingestGrib.sh
-rw-r--r-- 1 awips fxalpha 937 Dec 12 19:46 ingest.sh
-rw-r--r-- 1 awips fxalpha 1231 Dec 12 19:47 request.sh
Each file contains the **Xmx** definition for maximum memory:
export INIT_MEM=512 # in Meg
export MAX_MEM=4096 # in Meg
After editing these files, you must restart edex (`service edex_camel restart`).
---
## EDEX Plugin Configuration
The directory `/awips2/edex/conf/modes` contains XML files with rules defining which plugins are included or excluded with each JVM (ingest, ingestGrid, request):
ls -la /awips2/edex/conf/modes
-rw-r--r-- 1 awips fxalpha 1982 Dec 6 21:26 grid-modes.xml
-rw-r--r-- 1 awips fxalpha 928 Dec 6 21:24 ingest-modes.xml
-rw-r--r-- 1 awips fxalpha 1689 Dec 6 21:24 request-modes.xml
EDEX services are all registered through spring. By including or excluding specific spring files we can determine at startup which services the EDEX instance should start.
All mode files are merged at startup. Modes files with modes that have the same name are combined so the end result is an aggregate of patterns in all files. Include and exclude tags should have regular expressions that are compatible with Java's Pattern class. If you provide no `<include>` tag for a particular mode, the include defaults to `.*`.
An example of `/awips2/edex/conf/modes/ingest-modes.xml`, with a number of unused plugin decoders excluded because the data are not available outside of the SBN:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<edexModes>
<mode name="ingest">
<exclude>.*request.*</exclude>
<exclude>edex-security.xml</exclude>
<exclude>taf.*</exclude>
<exclude>modis.*</exclude>
<exclude>shef.*</exclude>
<exclude>idft.*</exclude>
<exclude>ffmp.*</exclude>
<exclude>stormtrack.*</exclude>
<exclude>retrieval.*</exclude>
<exclude>regionalsat.*</exclude>
<exclude>pointset-netcdf.*</exclude>
<exclude>ncscat.*</exclude>
<exclude>bufrobs.*</exclude>
<exclude>bufrmthdw.*</exclude>
<exclude>sgwh.*</exclude>
</mode>
</edexModes>
To view any stuck purge jobs in PortgreSQL (a rare but serious problem if your disk fills up). The solution to this is to run `edex purge reset`.

1
docs/mkdocs.yml Symbolic link
View file

@ -0,0 +1 @@
../mkdocs.yml