terrautils

Python library for TERRA-REF specific modules and methods, e.g. those shared by multiple extractors.

Installation instructions for terrautils can be found here.

betydb.py

• get_sites() -- Gets list of stations from BETYdb, filtered by city or sitename prefix if provided.
• submit_traits() -- Submit a CSV containing traits to the BETYdb API.
• get_sitename_boundary() -- Retrieve the clip boundary dynamically from betyDB API given sitename and turns the obtained json data into a geojson polygon.

extractors.py

• build_metadata() -- Construct extractor metadata object ready for submission to a Clowder file/dataset.
• get_output_directory() -- Determine output directory path given root path and dataset name.
• get_output_filename() -- Determine output filename given input information.
• is_latest_file() -- Check whether the extractor-triggering file is the latest file in the dataset.
• load_json_file() -- Load contents of a .json file on disk into a JSON object.
• error_notification() -- Send an error message notification, e.g. to Slack.
• log_to_influxdb() -- Send extractor job detail summary to InfluxDB instance.
• trigger_file_extractions_by_dataset() -- Manually trigger an extraction on all files in a dataset.
• trigger_dataset_extractions_by_collection() -- Manually trigger an extraction on all datasets in a collection.
• _search_for_key() -- Check for presence of any key variants in metadata. Does basic capitalization check.

formats.py

• create_geotiff() -- Generate output GeoTIFF file given a numpy pixel array and GPS boundary.
• create_netcdf() -- Generate output netCDF file given an input numpy pixel array.
• create_image() -- Generate output JPG/PNG file given an input numpy pixel array.

gdal.py

• array_to_image() -- Converts a gdalnumeric array to a Python Imaging Library (PIL) Image.
• image_to_array() -- Converts a Python Imaging Library (PIL) array to a gdalnumeric image.
• world_to_pixel(geo_matrix, x, y) -- Uses a gdal geomatrix (gdal.GetGeoTransform()) to calculate the pixel location of a geospatial coordinate.
• clip_raster(rast_path, features_path, gt=None, nodata=-9999) -- Clip a raster and return the clipped result in form of numpy array.
• get_raster_extents(fname) -- Calculates the extent and the center of the given raster.

metadata.py

• clean_metadata() -- Returns a standarized metadata object.
• get_terraref_metadata() -- Combines cleaned metadata with fixed metadata.
• get_extractor_metadata() -- Returns Clowder extractor metadata.
• get_sensor_fixed_metadata() -- Returns fixed metadata from Clowder.
• calculate_scan_time() --

sensors.py

• get_sensors(station) -- Get all sensors for a given station.
• get_sensor_filename(station, sensor, date, mode="full") -- Gets the filename for the image for the given date, sensor and station from the database. If the mode is full, choose the full resolution image, otherwise the reduced resolution version.
• get_sitename(station, date, range_=None, column=None) -- Returns a full sitename for the plot (or fullfield image) corresponding to the given station, date, range and column.
• check_site(station) -- Checks for valid station given the station name, and return its path in the file system.
• check_sensor(station, sensor, date=None) -- Checks for valid sensor with optional date, and return its path in the file system.
• get_sensor_product(site, sensor) -- Returns the downloadable product for each site-sensor pair.
• get_attachment_name(site, sensor, date, product) -- Encodes site, sensor, and date to create a unique attachment name.
• plot_attachment_name(sitename, sensor, date, product) -- Encodes sitename, sensor, and date to create a unqiue attachment name.

spatial.py

• calculate_bounding_box() -- Given a set of GPS boundaries, return array of 4 vertices representing the polygon.
• calculate_centroid() -- Given a set of GPS boundaries, return lat/lon of centroid.
• calculate_gps_bounds() -- Extract bounding box geometry, depending on sensor type.
• geom_from_metadata() -- Parse location elements from metadata.
• _get_bounding_box_with_formula() -- Convert scannerbox center position & sensor field-of-view to actual bounding box.

files.py

• get_sensor_list -- Returns a list of sensors from the geostream database.
• unique_sensor_names -- Returns a unique set of sensors by removing plot id.
• get_sensor -- Returns a stream dictionary given sensor and sitename (optional).
• get_file_listing -- Return a list of clowder file records for the a sensor
• extract_file_paths -- Returns list of absolute paths given a file listing