For a lot more detail about NetCDf and the GEON IDV, NetCDF Data for the GEON IDV

Seismic tomography data grids (2D or 3D) used in the IDV are in a NetCDF format. NetCDF data formats are used by hundreds of scientific institutions. NetCDF files are an excellent format for 2D and 3D grids mapped on the Earth. One NetCDF file may contain several data variables, such as seismic velocities, density, and temperature.

NetCDF files must include basic metadata, including latitude, longitude, and depth for grid point locations.

NetCDF files may include other metadata, such as recognized names for units of the data values. The IDV can use approved unit names in NetCDF files to do computations with grids of differing units and differing latitude-longitude mappings; the conversions of units and grid point locations are automatically done for you. But if you will never share your data and will not use the IDV for computations you may use only the basic metadata.

Some NetCDF file formats have special "conventions" which specify certain required features to ensure data is complete for some uses. IDV NetCDF files often are in the CF convention.

Users of GMT, a geology mapping tool, may have NetCDF on their system. GMT "grd" files are NetCDF, which can be made into NetCDF files for the IDV. "grd" files are a simple early form of NetCDF files, using y and x for the latitude and longitude variables, but they do not match NetCDF format conventions they were designed before there were NetCDF conventions. It may be easy to convert between IDV NetCDF files and GMT grid files.

NetCDF is a binary format which allows you to use the same data files on different operating systems, such as Windows, Mac, and Linux. NetCDF can be written directly (as binary files) when the data is created, using code libraries for C, Fortran, C++ and Java from Unidata's NetCDF web site. Unidata maintains the NetCDF code and detailed NetCDF documentation online. For more about NetCDf and the UNAVCO IDV, NetCDF Data for the UNAVCO GEON IDV

Putting data into NetCDF

To put data in NetCDF format, use an intermediate step, NetCDF's CDL ASCII format. A CDL file is plain ascii text with the same information as a NetCDF file, in human-readable format. See Making NetCDF 3D Grids for the IDV. This section will use CDL examples.

NetCDF and CDL files have "dimensions," "variables," "attributes," and a "data:" section. "Dimensions" are name-integer pairs such as latitude=120, giving the length or size of variable arrays. "Attributes" are metadata.

"Variables" are one-dimensional arrays of data, both the model results, and also coordinate values - grid point locations (and times if needed). For the IDV, every data value in a 3D grid must have a latitude, longitude, and depth value ( and a time value for a 4D grid). A netcdf file and its CDL equivalent have a head naming the dimensions and giving metadate for all variables. Earth coordinates are some of the variables. Special names identify the Earth coordinates. The "data:" section has the 1D arrays of values for the variables.

Here is an sample CDL file of a lat-long-depth grid of Vp, Vs, and density (and no time values). It happens to be a global grid but regional grids use the same format. (Some data values in this sample have been replaced with "..." to save space on this page.)

netcdf AK135_3D_global_grid {
dimensions:
	depth =  136;
	latitude = 7 ;
	longitude = 13 ;
variables:
	float depth(depth) ;
		depth:units = "kilometer" ;
		depth:positive = "up" ;
	float latitude(latitude) ;
		latitude:long_name = "Latitude, positive north" ;
		latitude:units = "degrees_north" ;
		latitude:standard_name = "latitude" ;
	float longitude(longitude) ;
		longitude:long_name = "Longitude, positive East" ;
		longitude:units = "degrees_east" ;
		longitude:standard_name = "longitude" ;
	float Vp(depth, latitude, longitude) ;
		Vp:long_name = "AK135 P wave velocity" ;
		Vp:valid_range = 5.9f, 13.7f ;
		Vp:units = "kilometer sec^-1" ;
	float Vs(depth, latitude, longitude) ;
		Vs:long_name = "AK135 S wave velocity" ;
		Vs:valid_range = 0.f, 7.3f ;
		Vs:units = "kilometer sec^-1" ;
        float density(depth, latitude, longitude) ;
                density:long_name = "AK135 density" ;
                density:valid_range = 2.40f, 13.1f ;
                density:units = "gram centimeter^-3" ;
data:

depth = -0.000, -20.000, -20.000, -35.000, -35.000, -77.500, -120.000, -165.000, -210.000, -210.000, ...  -6320.290, -6371.000; 

latitude = -90.0, -60.0, -30.0, 0.0, 30.0, 60.0, 90.0; 

longitude = 0.0, 30.0, 60.0, 90.0, 120.0, 150.0, 180.0, 210.0, 240.0, 270.0, 300.0, 330.0, 360.0;

Vp = 5.8000, 5.8000, 5.8000, 5.8000, 5.8000, 5.8000, 5.8000, 5.8000, ... 11.2622;

Vs = 3.4600, 3.4600, 3.4600, 3.4600, 3.4600, 3.4600, 3.4600, 3.4600, ... 3.6678, 3.6678;

density = 2.4490, 2.4490, 2.4490, 2.4490, 2.4490, 2.4490, 2.4490, 2.4490, 2.4490, ... 13.0122;
}

Note:
- the first line "netcdf AK135_3D_global_grid {" has a name you make up.
- all the material after that name is inside a { } pair.
- every data line ends with ";"
- each geophysics variable shows the names and order of its Earth coordinate values, such as "(depth)."
- The "variables" "depth," "latitude," and "longitude" are 1D arrays of the Earth position coordinates.
- All position coordinate arrays are recognized in NetCDF by having the same name as their dimension size value name.
- the parameter values such as Vp have their data in a 1D array with values ordered as indicated by (depth, latitude, longitude).
- In this example the Vp array will have 137 * 7 * 13 = 12376 values.
- In this example, the values in the 1D parameter arrays have longitude changing most rapidly.
- each variable has "attributes." In this example we see attributes of units, depth:positive, long_name, standard_name, and valid_range.
- the IDV likes depth to increase upwards; generally depth=0 is at mean sea level. Depths inside the Earth are negative, increasing downwards.
- NetCDF may use recognized units; unit abbreviated names come from "UD units"; and see Supported Units.
- "long_name" is only for informational and display labeling purposes.
- "standard_name" is a NetCDF attribute to help insure that position corordinates are recognized correctly.
- "valid_range" means data values outside the range are not displayed. This can be used with a special value meaning undefined data; such values are not displayed or used in calculations such as contouring or interpolation, or differences between model grids.

CDL files have extension .cdl.

CDL documents

For the netCDF documentation about CDL, see Section 2.1.2 Network Common Data Form Language (CDL), CDL Syntax, and CDL data types (for float, int, etc.). Other parts of the complete NetCDF documentation may be useful to you. CDL files end with extension ".cdl". NetCDF files (binary) end with extension ".nc".

NetCDF: the CF Convention

There are several different "conventions" for NetCDF files. Conventions are best practices for a netCDF file to represent certain kinds of data. Conventions are created by users' working groups, with Unidata advice in some cases. The IDV works best with gridded data in the CF convention, designed for 2D, 3D and 4D grids with Earth locations.

Converting CDL files to NetCDF files: The ncgen and ncdump Utilities

The NetCDF program "ncgen" makes a NetCDF (.nc) binary file from an ASCII CDL (.cdl) file; the NetCDF program "ncdump" does the reverse.

You use ncgen to make a binary NetCDF file from your ASCII CDL file. You can get ncgen by downloading and installing NetCDF on your system from NetCDF downloads. After installation you will find ncgen and ncdump in for example a Linux directory like /home/user22/netcdf/netcdf-3.6.0/bin/ or on Windows in My Documents\netcdf\netcdf-3.6.0\bin\.

You make a NetCDF file from a CDL file with the command

ncgen -o outputNetCDFfile.nc inputCDLfile.cdl

Note that the cdl file has extension .cdl and the output NetCDF file has extension .nc. You can even use ncgen to create a C or FORTRAN code fragment to write the cdl file.

Ncgen is described in the online NetCDF ncgen documentation and in the ncgen "Man" page.

Ncdump

What if you have a NetCDF binary file you want to examine? Ncdump is a utility program to examine the contents of a NetCDF binary file by creating the CDL equivalent. The ncdump tool generates the CDL version of a netCDF file on standard output, optionally excluding some or all of the variable data in the output. (The output from ncdump is also intended to be acceptable as input to ncgen. Thus ncdump and ncgen can be used as inverses to transform data representation between binary and text representations.) To get a new CDL file from a NetCDF file, do
ncdump datafile.nc > datafile.cdl
Often ncdump is used to reveal only the header in CDL, ithe part above the "data:" section, so you can see how it works, or why it does not work. i Add "-h" after "ncdump "
ncdump -h datafile.nc > datafile.cdl
to see only the the header.

See the NetCDF header in the IDV

You can see the header of a NetCDF file in the IDV. In the Dasboard open the "Field Selector" tab. Do a right click over the data source name in the "Data Sources" column (left side). Click on "Properties." Click on "Metadata" in the new window that pops up.

Mapping X-Y-Z Grids to Latitude-Longitude-Depth grids with NetCDF

Here is an example of a CDL file header, showing how to store a rectangular x-y-depth grid in NetCDF so that it can be used by programs that require latitude-longitude-depth grids. NetCDF readers automatically convert the x-y-z locations in the file to latitude-longitude-depth for the program. You have to know exactly how your grid maps on the Earth, in this case, as transverse mercator. The official NetCDF documentation for this feature is Standard Coordinate Transforms.

netcdf tomography_m2.nc {
 dimensions:
   y = 86;   // (has coord.var)
   x = 70;   // (has coord.var)
   Depth = 101;   // (has coord.var)
 variables:
   float Vp(Depth=101, y=86, x=70);
     :units = "km/s";
     :long_name = "P wave velocity";
     :missing_value = -9999.0; // double
     :coordinates = "lat lon";
     :grid_mapping = "TM";
   int TM;
     :_CoordinateTransformType = "Projection";
     :grid_mapping_name = "transverse_mercator";
     :scale_factor_at_central_meridian = 1.0; // double
     :longitude_of_central_meridian = 138.97; // double
     :latitude_of_projection_origin = 36.05; // double
     :_CoordinateAxisTypes = "GeoX GeoY";
   float y(y=86);
     :units = "km";
     :_CoordinateAxisType = "GeoY";
     :long_name = "y coordinate of projection";
   float x(x=70);
     :_CoordinateAxisType = "GeoX";
     :units = "km";
     :long_name = "x coordinate of projection";
   float Depth(Depth=101);
     :units = "km";
     :_CoordinateAxisType = "Height";
     :_CoordinateZisPositive = "up";
   float lat(y=86, x=70);
     :units = "degrees_north";
     :long_name = "latitude coordinate";
     :standard_name = "Latitude";
     :_CoordinateAxisType = "Lat";
   float lon(y=86, x=70);
     :units = "degrees_east";
     :long_name = "longitude coordinate";
     :standard_name = "Longitude";
     :_CoordinateAxisType = "Lon";

Note that this has only one parameter, Vp; more may be used in one file. Also note the new attribute "missing_value = -9999.0." Grid values of -9999 will not be displayed or used fror calculations in the IDV.