How to read a VIIRS L2 CLDPROP netcdf file using python ?

An example of how to read a VIIRS L2 CLDPROP netcdf file using python 3:

Download a VIIRS L2 CLDPROP netcdf file

To download a VIIRS L2 CLDPROP netcdf file, a solution is to go on LAADS DAAC. Example of granule selected randomly that will be used hereafter:

CLDPROP_L2_VIIRS_SNPP.A2019001.1742.001.2019062201640.nc

Read a netCDF file with python

To read the CLDPROP_L2_VIIRS_SNPP netCDF file with python, a solution is to use the netCDF4 module.

Note: if you work with the python anaconda distribution, the netCDF4 is already installed:

$ python
Python 3.6.0 |Anaconda custom (x86_64)| (default, Dec 23 2016, 13:19:00) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import netCDF4
>>> netCDF4.__version__
'1.2.4'
>>>

Open a VIIRS L2 CLDPROP netcdf file

To open the file:

import netCDF4

f = netCDF4.Dataset('CLDPROP_L2_VIIRS_SNPP.A2019001.1742.001.2019062201640.nc')

At this point, it is possible to use print to check the file:

print(f)

or

import pprint

pprint.pprint(f)

Get a list of class attributes

To get something more readable a solution is to use the method dict:

print(f.__dict__.keys())

or

print(f.__dir__()

which returns:

['platform', 'processing_level', 'processing_version', 'cdm_data_type', 'keywords_vocabulary', 'license', 'stdname_vocabulary', 'naming_authority', 'Level1B_bias_correction_factors', 'StartTime', 'EndTime', 'Conventions', 'instrument', 'creator_name', 'creator_email', 'creator_url', 'project', 'DayNightFlag', 'SouthBoundingCoordinate', 'NorthBoundingCoordinate', 'EastBoundingCoordinate', 'WestBoundingCoordinate', 'history', 'source', 'date_created', 'product_name', 'LocalGranuleID', 'ShortName', 'product_version', 'AlgorithmType', 'identifier_product_doi', 'identifier_product_doi_authority', 'input_files', 'ancillary_files', 'l1_version', 'l1_lut_version', 'l1_lut_created', 'DataCenterId', 'creator_institution', 'publisher_name', 'publisher_url', 'publisher_email', 'publisher_institution', 'GRingPointSequenceNo', 'GRingPointLatitude', 'GRingPointLongitude', 'geospatial_lat_units', 'geospatial_lon_units', 'geospatial_lat_min', 'geospatial_lat_max', 'geospatial_lon_min', 'geospatial_lon_max', 'time_coverage_start', 'time_coverage_end', 'startDirection', 'endDirection', 'OrbitNumber', 'xmlmetadata', 'title', 'l2_luts', 'CHIMAERA_version', 'long_name', '__repr__', '__getattribute__', '__setattr__', '__delattr__', '__init__', '__getitem__', '__new__', '__enter__', '__exit__', 'filepath', '__unicode__', 'close', 'isopen', 'sync', '_redef', '_enddef', 'set_fill_on', 'set_fill_off', 'createDimension', 'renameDimension', 'createCompoundType', 'createVLType', 'createEnumType', 'createVariable', 'renameVariable', 'createGroup', 'ncattrs', 'setncattr', 'setncattr_string', 'setncatts', 'getncattr', 'delncattr', '__getattr__', 'renameAttribute', 'renameGroup', 'set_auto_maskandscale', 'set_auto_mask', 'set_auto_scale', 'get_variables_by_attributes', '_grpid', '_isopen', 'groups', 'dimensions', 'variables', 'disk_format', 'path', 'parent', 'file_format', 'data_model', 'cmptypes', 'vltypes', 'enumtypes', '__orthogonal_indexing__', 'keepweakref', '__doc__', '__hash__', '__str__', '__lt__', '__le__', '__eq__', '__ne__', '__gt__', '__ge__', '__reduce_ex__', '__reduce__', '__subclasshook__', '__init_subclass__', '__format__', '__sizeof__', '__dir__', '__class__']

It is then possible to get some useful information:

print(f.platform)
print(f.DayNightFlag)
print(f.product_version)
print(f.instrument)
print(f.license)

return for example:

Suomi-NPP
Day
1.0
VIIRS
http://science.nasa.gov/earth-science/earth-science-data/data-information-policy/

Read the CLDPROP L2 Products

The CLDPROP L2 Products are in 4 groups (see MODIS and VIIRS Cloud Properties: User Guide pages 40-43):

print(f.groups.keys())

returns

odict_keys(['scan_line_attributes', 'geolocation_data', 'geophysical_data', 'cloud_model_data'])

Select a group:

gd_group = f.groups['geophysical_data']

Print group variables:

print(gd_group.variables.keys())

returns

odict_keys(['Cloud_Top_Temperature', 'Cloud_Top_Height', 'Cloud_Top_Pressure', 'Surface_Pressure', 'Cloud_Effective_Emissivity', 'Cloud_Top_Pressure_Uncertainty', 'Cloud_Top_Temperature_Uncertainty', 'Cloud_Top_Height_Uncertainty', 'Cloud_Phase_Cloud_Top_Properties', 'Cloud_Optical_Thickness', 'Cloud_Optical_Thickness_PCL', 'Cloud_Optical_Thickness_16', 'Cloud_Optical_Thickness_16_PCL', 'Cloud_Optical_Thickness_37', 'Cloud_Optical_Thickness_37_PCL', 'Cloud_Optical_Thickness_1621', 'Cloud_Optical_Thickness_1621_PCL', 'Cloud_Effective_Radius', 'Cloud_Effective_Radius_PCL', 'Cloud_Effective_Radius_16', 'Cloud_Effective_Radius_16_PCL', 'Cloud_Effective_Radius_37', 'Cloud_Effective_Radius_37_PCL', 'Cloud_Effective_Radius_1621', 'Cloud_Effective_Radius_1621_PCL', 'Cloud_Water_Path', 'Cloud_Water_Path_PCL', 'Cloud_Water_Path_16', 'Cloud_Water_Path_16_PCL', 'Cloud_Water_Path_37', 'Cloud_Water_Path_37_PCL', 'Cloud_Water_Path_1621', 'Cloud_Water_Path_1621_PCL', 'Cloud_Effective_Radius_Uncertainty', 'Cloud_Effective_Radius_Uncertainty_16', 'Cloud_Effective_Radius_Uncertainty_37', 'Cloud_Effective_Radius_Uncertainty_1621', 'Cloud_Optical_Thickness_Uncertainty', 'Cloud_Optical_Thickness_Uncertainty_16', 'Cloud_Optical_Thickness_Uncertainty_37', 'Cloud_Optical_Thickness_Uncertainty_1621', 'Cloud_Water_Path_Uncertainty', 'Cloud_Water_Path_Uncertainty_16', 'Cloud_Water_Path_Uncertainty_37', 'Cloud_Water_Path_Uncertainty_1621', 'IRW_Low_Cloud_Temperature_From_COP', 'Cloud_Phase_Optical_Properties', 'Retrieval_Failure_Metric', 'Retrieval_Failure_Metric_16', 'Retrieval_Failure_Metric_37', 'Retrieval_Failure_Metric_1621', 'Atm_Corr_Refl', 'Cloud_Mask', 'Quality_Assurance'])

Example: read the Cloud_Top_Temperature

Select the Cloud_Top_Temperature

cloud_top_temperature = gd_group.variables['Cloud_Top_Temperature']

Print Cloud_Top_Temperature info

print(cloud_top_temperature)

int16 Cloud_Top_Temperature(number_of_lines, number_of_pixels)
    long_name: Cloud Top Temperature from NOAA CLAVR-x AWG algorithm
    _FillValue: -9999
    valid_min: 0
    valid_max: 32767
    scale_factor: 0.00793499965221
    add_offset: 100.0
    units: K
path = /geophysical_data
unlimited dimensions: 
current shape = (3248, 3200)
filling on

Get Cloud_Top_Temperature attributes

ctt_FillValue = cloud_top_temperature._FillValue
ctt_scale_factor = cloud_top_temperature.scale_factor
ctt_add_offset = cloud_top_temperature.add_offset

print(ctt_FillValue)
print(ctt_scale_factor)
print(ctt_add_offset)

return

-9999
0.00793499965221
100.0

Get the cloud_top_temperaturedata

cloud_top_temperature_data = cloud_top_temperature[:]

print(cloud_top_temperature_data)

print(np.ma.getdata(cloud_top_temperature_data))

return

[[-9999.         -9999.         -9999.         ...,   264.98451277
    264.65917778   264.82581278]
 [-9999.         -9999.         -9999.         ...,   264.98451277
    264.65917778   264.82581278]
 [-9999.         -9999.         -9999.         ...,   264.98451277
    264.65124278   264.82581278]
 ..., 
 [-9999.         -9999.           294.43923148 ...,   293.34420153
    292.36819657   293.65366651]
 [-9999.         -9999.           294.48684148 ...,   293.35213653
    292.41580657   293.64573151]
 [-9999.         -9999.           294.43129648 ...,   293.66160151
    292.36026157   293.64573151]]

Note: I am not sure if it is still necessary to read scale_factor and add_offset since the data seems to be already correctly formatted.

Plot the data using matplotlib

Check the data using maptlolib:

import matplotlib.pyplot as plt

cloud_top_temperature_data = np.fliplr(cloud_top_temperature_data)
cloud_top_temperature_data = np.flipud(cloud_top_temperature_data)

plt.imshow(cloud_top_temperature_data)
plt.show()

An example of python script

import netCDF4
import numpy as np

import pprint

#----------------------------------------------------------------------------------------#
# Open a VIIRS L2 CLDPROP netcdf file

f = netCDF4.Dataset('CLDPROP_L2_VIIRS_SNPP.A2019001.1742.001.2019062201640.nc')

#print(type(f))

#pprint.pprint(f)

#----------------------------------------------------------------------------------------#
# Get a list of class attributes

print(f.__dict__.keys())

pprint.pprint(f.__dir__())

print(f.__dir__())

print(f.platform)
print(f.DayNightFlag)
print(f.product_version)
print(f.instrument)
print(f.license)

#----------------------------------------------------------------------------------------#
# Read the CLDPROP L2 Products

print(f.groups.keys())

# select a group:

gd_group = f.groups['geophysical_data']

# print list of variables

print(gd_group.variables.keys())

#----------------------------------------------------------------------------------------#
# Example: get the Cloud_Top_Temperature

cloud_top_temperature = gd_group.variables['Cloud_Top_Temperature']

print(cloud_top_temperature)

# Get Cloud_Top_Temperature attributes

ctt_FillValue = cloud_top_temperature._FillValue
ctt_scale_factor = cloud_top_temperature.scale_factor
ctt_add_offset = cloud_top_temperature.add_offset

print(ctt_FillValue)
print(ctt_scale_factor)
print(ctt_add_offset)

# Get the data

cloud_top_temperature_data = cloud_top_temperature[:]

print(cloud_top_temperature_data)

print(np.ma.getdata(cloud_top_temperature_data))

#----------------------------------------------------------------------------------------#
# Example: plot the Cloud_Top_Temperature using matplotlib

import matplotlib.pyplot as plt

cloud_top_temperature_data = np.fliplr(cloud_top_temperature_data)
cloud_top_temperature_data = np.flipud(cloud_top_temperature_data)

plt.imshow(cloud_top_temperature_data)
plt.show()

References