GPS/GNSS File Server Access Examples

The NSF GAGE Facility, operated by EarthScope Consortium, has transitioned from FTP to HTTPS for access to our data archive file system. We are also now requiring user registration and log in for file server access. For more information, see our notices about the transition.

By downloading files from the NSF GAGE Facility, you are agreeing to abide by the NSF GAGE Data Policy.

To browse the HTTPS file server, simply visit GPS/GNSS file server to log in (see User Authentication below) and view our holdings.

Please send your issues, comments, and suggestions to: data-help@earthscope.org.

User Authentication

We now require all data users to register and login to access the GAGE Data File Server (this new authentication system is known as Identity Management or simply IdM). Please see the link above to the notice. To access the file server from the command line you will need to:

1. Register a user account with us, if you haven't already, to create a profile.
2. Access data:

   • For casual access

     • Use a web browser to login to your account and browse data files and download them.

       OR

     • Include an authorization header parameter with your command line data file requests that includes your access token:
       • Use a http file transfer client (Wget or cURL) with your temporary access key. From your profile API Key page, you can view and copy your short-lived access token for trial and testing purposes. You can copy and paste this token into a http request header:
         

             --header "Authorization: Bearer <PASTE YOUR ACCESS TOKEN HERE>"

   • For scripted access

     Good for scripts that run regularly and unattended to access data files.
   1. Command Line Interface (CLI)
      1. On standard servers:
         1. Download and setup our Python based Command Line Interface (CLI) on your server to simplify access token acquisition, maintenance, and usage (Requires Python 3.7+ and a bash-like shell). See the README for usage instructions
         2. Use the CLI to log in and, via command substitution, automatically insert valid (refreshed) tokens in your request's header parameter:
            --header "Authorization: Bearer $(es sso access --token)"
            
            NOTE: your shell environment may require different command substitution syntax, e.g., using backticks, e.g.,
            --header "Authorization: Bearer `es sso access --token`".
            
            
            TROUBLESHOOTING

            • If you are having trouble with command substitution, try running the es sso access --token command by itself to see if the token is being returned correctly. If it is, then the issue is likely with the command substitution syntax in your shell environment. See above note.
            • If you get an error regarding not being able to refresh your tokens, run the following EarthScope CLI commands and get new tokens:
              1. es sso logout
              2. es sso login
      2. For containers without persistent storage:
         1. You basically will want to share your access tokens across your servers. You will keep a centralized server or lambda function's CLI tokens up-to-date daily via cron and make those tokens available to your other servers/containers via persistent storage or a secrets manager.
   2. Software Development Kit (SDK)
      • For python script developers who want to authenticate their python scripts without the CLI.
      • See our SDK code repository README which includes an example python download script utilizing our SDK.
      • Note: For non-Python coders, you can look at the SDK's Python code as an example of the basic logic needed to implement similar functionality in your coding language of choice. You can also see Auth0's quickstarts.
3. See the access examples below that utilize our CLI with both cURL and Wget. See our SDK code repository README for an example python download script.

Examples

The following examples illustrate access to the NSF GAGE Facility GPS/GNSS data file server using standard file/data transfer utilities in a bash-like shell (see bash for Windows). See our SDK code repository README for an example python download script.

$ curl -L -O -f https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/p1230600.22d.Z --header "Authorization: Bearer $(es sso access --token)"

$ xargs -n 1 curl -O --header "Authorization: Bearer $(es sso access --token)" < file_urls_to_download.txt

$ curl -L "https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/?list" --header "Authorization: Bearer $(es sso access --token)"

$ curl -L -O "https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/p12{3,4,6}0600.22d.Z" --header "Authorization: Bearer $(es sso access --token)"

$ wget -N -r -np -erobots=off --reject="tmp,index.html*" https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/ --header "Authorization: Bearer $(es sso access --token)"

$ wget https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/p1230600.22d.Z --header "Authorization: Bearer $(es sso access --token)"

$ wget -i file_urls_to_download.txt --header "Authorization: Bearer $(es sso access --token)"

$ wget "https://gage-data.earthscope.org/archive/gnss/rinex/obs/2022/060/?list" --header "Authorization: Bearer $(es sso access --token)"

$ wget -N -r -l3 -np -nd -erobots=off --reject="tmp,index.html*" -A "p123*" https://gage-data.earthscope.org/archive/gnss/rinex/obs/2020/060/ --header "Authorization: Bearer $(es sso access --token)"

Last modified: 2024-11-08  17:54:40  America/Denver