Using wget with the Datamart data in GRIB format
Here is a brief set of examples and usage notes, offered here to illustrate the possibilities for selective batch downloading from the public GRIB database. Please refer to the official wget manual for detailed usage and further general examples. Examples shown here were tested under Linux and the Windows XP command line.
- Italics (emphasized text) indicate user-supplied fields.
- Some examples have been broken into multiple lines for readability. A backslash at the end of a line indicates such a break.
- Users of the Windows command shell should replace the single quotes (') in the examples with double quotes (").
Retrieving a single file in the 00Z GDPS (Global Deterministic Prediction System) model database:
wget -nd -r -l1 -A '*HGT_ISBY*aaaammjjhh_P036.grib2' \
- yyyymmddhh refers to the production run date. hh is either 00 or 12.
- The -r and -A options enable the use of wildcards. It is possible to take advantage of powerful automatic file selection capabilities by combining wildcarding with knowledge of the file naming convention. When passing an argument to the -A switch, double quotes should be used under the Windows command line, and single quotes under Unix-like systems.
- In order to perform file selection according to a given set of wildcards, wget downloads the index.html file. With some older versions of wget, the index.html file is left lying around after the download is completed. Updating to the latest version should resolve this problem. Also, see next example for a way to avoid downloading index.html altogether.
Retrieving fields from a locally generated list of file names:
wget -nd -nc -i file-list -P target-directory -B base-URL
- file-list is a plain text file containing a list of desired files
- the file names in file-list must be complete and conform to the naming convention, including a current date-stamp. This requires you to generate a new list every day, a step that can be automated. The payoff is in a savings of bandwidth and download time.
- here is an example of a correct base-URL value:
Retrieving 00Z 00-hour RDPS (Regional Deterministic Prediction System) temperature at surface and all isobaric levels:
wget -nd -r -l1 \
-R '*ISBL_00*' \
- Note the use of the -R switch to reject isobaric fields at 100 and 200 mb.
What we are doing
- Date modified: