nexrad-archive/README.md

![WSR-88D: Come And Take It](doc/come-and-take-it.png)

# NEXRAD Level II Archive Tool

This tool uses
[Spatialite](https://www.gaia-gis.it/fossil/libspatialite/index)
in conjunction with the
[NCEI Storm Events Database](https://www.ncei.noaa.gov/pub/data/swdi/stormevents/csvfiles/)
to archive only NEXRAD Level II data for which storms were recorded
by the National Weather Service.  Using the start and end times and
coordinates of a given storm event, this tool is able to query and
filter objects in the 
[NEXRAD Level II Amazon bucket](https://registry.opendata.aws/noaa-nexrad/),
allowing one to only archive volume scans for which storms were noted,
from the appropriate radar sites.

## Building

Simply run `make` to produce a Docker image pre-populated with the
NEXRAD radar site database.

## Running

Invoke `sh run.sh` to run this archival tool, passing two arguments:

* The path to a storm events database CSV export file, whose name
  as obtained from the NCEI begins with `StormEvents_details_`.  Only
  these files contain the start and end coordinates of a storm which
  are used to select data from the correct radar sites.
  
* A path to a destination directory where archived data shall be
  downloaded.  Note that files downloaded under this directory will
  be organized in the `YYYY/MM/DD/SITE/` hierarchy.
  
Other optional arguments will be passed through to the
`bin/nexrad-archive` executable within the Docker image:

* `--dry-run`

  This will avoid actually performing any archiving; instead, the
  tool will indicate what actions it would take.
  
* `--quiet`

  Suppresses standard output.

* `--exclude="Event Type"`

  Exclude one or more event types from consideration for archival.
  These event types are
  [documented](https://www.ncei.noaa.gov/pub/data/swdi/stormevents/csvfiles/Storm-Data-Bulk-csv-Format.pdf)
  by the NCEI.  Cannot be specified alongside `--type`. Can be
  specified more than once.

* `--type="Event Type"`

  Explicitly specify which event types to archive.  Cannot be
  specified alongside `--exclude`.  Can be specified more than once.