REMOTE SENSING

Interface Control Document


Abstract

This document contains information describing the SA/Remote Sensing Interface.

26 August 2019


Overview

This document describes the interfaces between the Remote Sensing aircraft/satellites and SA (Situation Analyst). Intterra uses Amazon S3 as the data exchange mechanism for remote sensing data.

Mission Operations

Combined, SA and the Remote Sensing systems will provide near-real-time information to operations and partner agency personnel on wildfire and public safety events. SA and the Remote Sensing systems have been designed to allow a SA user to track and receive intelligence from a Remote Sensing mission from take-off to landing as well as to access data from previous missions.

Figure 1 describes the stages of an individual Remote Sensing mission as tracked within SA.


Remote Sensing Products

The Remote Sensing aircraft are equipped with an Electro-Optical/ Infrared Sensor and mission software capable of producing real-time or near-real-time airborne fire intelligence for day and night surveillance. The systems are capable of recording and mapping the location of aerially applied water or fire retardant, mapping a fire’s progression over time, and shall provide incident reports directly to the State’s information management system.

During initial operating period, the Remote Sensing aircraft products include:
  • Mission Designator File - an empty .txt formatted file that signifies that an active mission is pending or underway. File name will be used to associate data products collected for an active mission.
  • Aircraft Position – a .kml formatted file describing aircraft location and flight parameters. This file includes ‘cookie trail’ as provided by aircraft.
  • Fire Products
    • Active Fire Line – a .kml formatted RED polyline file describing an active fire front.
    • Fire Perimeter – a .kml formatted BLUE polyline file describing the perimeter of a fire.
    • Hot Spot – a .kml formatted points file containing single RED points representing hot spots.
    • Detection - a .kml formatted points file representing fire detection reports from wide area missions. These are displayed using a detection icon and include report information within the pop-up menu.
    • Fire Retardant Line - a .kml formatted PURPLE polyline file describing a fire retardant application on the ground.
  • Image Products – these images can be designated as ‘to be reprojected onto the map’ or ‘depicted as-is with no reprojection’. Image products can be provided as a geotiff or a png/aux combination.
    • Electro-optical Image – a color image from the sensor.
    • Infrared Image – a black and white image generated by the sensor’s infrared systems.
    • Heat Source Image – an image derived from sensor’s infrared system that highlights heat sources.
  • Video Products
    • Video
      • Option 1: a video product with a .kml formatted file designating video location
      • Option 2: a transport stream video with a valid video segment, a valid KLV segment, and an optional audio segment
  • Additional Products
    • Department of Public Safety – a .kml formatted file depicting a BLUE point for a public safety incident.
    • Life Safety Point, Line, Polygon – a .kml formatted file depicting YELLOW points, lines, or polygons for a life safety event.
    • All Hazard Point, Line, Polygon – a .kml formatted file depicting GREEN points, lines, or polygons for an all hazards event.


Naming Conventions

Remote Sensing products will utilize the following naming conventions.  There can be no spaces or special characters with the [] below.  Hyphens are OK.  ‘_’ are used only as delimiters between the 5 file name elements [].  
  • Remote Sensing Missions are labeled as [Remote Sensing Aircraft Number]_[date time group of mission].
    • Example - Remote Sensing01_090120142304Z.txt
  • Aircraft position files are labeled as a [Remote Sensing Mission]_AIRCRAFT.kml.
    • Example -  Remote Sensing01_091920141818Z_AIRCRAFT.kml
  • Tactical products are labeled as [Remote Sensing Mission]_[product type]_[serialized number based on date time group]_[display name].kml. Display name is optional.
    • Example - Remote Sensing0X_090120142304Z_PERIM_002446082_BigFire.kml
    • Product Type is defined as
      • Fire Products
        • Fire Perimeter = PERIM
        • Hot Spots = HOTSPOT
        • Detection = DETECTION
        • Active Fire Line = AFL
        • Retardant = RETARDANT
      • Additional Products
        • DPS = DPS
        • Life Safety = LIFESAFETY
        • All Hazard = ALLHAZARD
  • Imagery products are labeled as [Remote Sensing Mission]_[product type] _[serialized number based on time]_[display name].png and [Remote Sensing Mission]_[product type] _[serialized number based on time] _[display name].png.aux.xml. Display name is optional.
    • Examples - 
      • RemoteSensing01_091620141717Z_IRImage_190435942_burnout.png & RemoteSensing01_091620141717Z_IRImage_190435942_ burnout.png.aux.xml
      • RemoteSensing01_091620141717Z_IRImage_190435942_ burnout.png & RemoteSensing01_091620141717Z_IRImage_190435942_ burnout.tif
    • Product Type is defined as
      • Color Imagery = EOImage
      • Infrared Imagery = IRImage
      • Heat Source Imagery = HSImage
    • Video products are labeled as [Remote Sensing Mission]_[product type] _[serialized number based on time] _[displayname].mp4 and [Remote Sensing Mission]_[product type] _[serialized number based on time] _[displayname].kml. Display name is optional.
      • Video = VIDEOCLIP


Product Specifications

Mission File

The Mission file is an MS Windows .txt file with no content and a structured name using naming conventions. It is loaded by the MSO at the time a mission is planned and deleted at the conclusion of a mission.

Aircraft Position Product

The aircraft position product is a .kml formatted file. It is automatically loaded by the aircraft every 30 seconds. A sample is saved as /SampleProducts/.

Tactical Products

Remote Sensing tactical products are .kml formatted files of vector products. They are loaded when the MSO operator creates a new product based on the situation.
  • Hot Spot – Single or multiple points of active heat sources. A sample is saved as /SampleProducts/.
  • Detection – Single or multiple points of active heat sources representing the detection of a fire event during a wide area survey. A sample is saved as /SampleProducts/.
    • Note that these products should take advantage of the Optional Metadata Specification described below to include a formatted table of reported information.
  • Perimeter – Polygon of perimeter of a fire. A sample is saved as /SampleProducts/.
  • Retardant – A line of retardant. A sample is saved as /SampleProducts/.
  • Active Fire Line – A line designating and active fire front. A sample is saved as /SampleProducts/.
  • DPS – Points, lines, or polygons of public safety interest. A sample is saved as /SampleProducts/.
  • Life Safety – Points, line, or polygons of life safety interest. A samples is saved as /SampleProducts/.
  • All Hazard – Points, line, or polygons of representing ‘All Hazard’ events. A samples is saved as /SampleProducts/.
Metadata Specification (Optional): Tactical products may contain metadata per feature that can transfer into SA features. The expected convention is to use the data-field tag within a KML element to specify for SA which target field to apply a metadata value to.

Example:

<Placemark>
<description><![CDATA[<table>
<tr><td>Sensor</td><td data-field="Sensor">Burma</td></tr>
<tr><td>Name</td><td data-field="Name">Detection</td></tr>
<tr><td>DateCollected</td><td data-field="DateCollected">2016-09-21 12:00:00
]]></description>

Imagery Products

Imagery products can be provided as:
  • OCG-formatted GEOTIFF files. See OCG specification.
    • Required Projection: WGS-1984, EPSG 4326, Decimal Degrees (not UTM)
    • Required Size: <5MB (to preserve functionality in low comms environments)
    • Preferred Image Format: Interleave by Pixel, Byte, 3-band, JPEG Compression
GeoTiff files currently do not support the addition of narratives or the option to NO WRAP.

  • A pair of formatted files consisting of a .png graphic and a aux.xml file with metadata. Both files are required for SA to generate and display a product. SA processes the imagery and produces a .kmz file for display. The .kmz is saved using same naming convention and reloaded to the imagery bucket.
.png.aux.xml file – must contain the following data for processing. A sample is provided as /SampleProducts/. The aux.xml file must be structure and include the below:
    • Coordinates – these must be provided to identify location of image
<PAMDataset>
  <Metadata domain="IMAGE_STRUCTURE">
    <MDI key="INTERLEAVE">PIXEL</MDI>
  </Metadata>
  <Metadata>
  <GCPList Projection="GEOGCS[&quot;WGS 84&quot;,DATUM[&quot;WGS_1984&quot;,SPHEROID[&quot;WGS 84&quot;,6378137,298.257223563,AUTHORITY[&quot;EPSG&quot;,&quot;7030&quot;]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[&quot;EPSG&quot;,&quot;6326&quot;]],PRIMEM[&quot;Greenwich&quot;,0,AUTHORITY[&quot;EPSG&quot;,&quot;8901&quot;]],UNIT[&quot;degree&quot;,0.0174532925199433,AUTHORITY[&quot;EPSG&quot;,&quot;9108&quot;]],AUTHORITY[&quot;EPSG&quot;,&quot;4326&quot;]]">
    <GCP Id="UpperLeft" Pixel="0.5000" Line="0.5000" X="-1.079226860000E+002" Y="4.000461400000E+001" />
    <GCP Id="UpperRight" Pixel="1919.5000" Line="0.5000" X="-1.079261730000E+002" Y="4.000763300000E+001" />
    <GCP Id="LowerRight" Pixel="1919.5000" Line="1079.5000" X="-1.079226840000E+002" Y="4.001002500000E+001" />
    <GCP Id="LowerLeft" Pixel="0.5000" Line="1079.5000" X="-1.079193800000E+002" Y="4.000705500000E+001" />
  </GCPList>
</PAMDataset>

    • Annotation – optional metadata element to add annotations to images
<PAMDataset>
  <Metadata>
    <MDI key="DESCRIPTION">Enter comments &amp; notes here</MDI>
  </Metadata>
</PAMDataset>

    • No Warp Flag – optional metadata element to flag images as project to map or not. Deflault behavior is to warp all images. Setting to “1” will designate an image to no be reprojected onto the map. This is usually for quick snapshot images taken off nadir.
<PAMDataset>
  <Metadata>
    <MDI key="NO_WARP">1</MDI>
  </Metadata>
</PAMDataset>

Video Products

Video products are formatted files derived from the raw video data collected by the aircraft systems. They can be provided as:
  • An .mp4 formatted video file and a formatted .kml file containing registration information. Both files are required for SA to generate and display a product. A sample of .kml  file is provided as /SampleProducts/.
  • A .ts formatted video file
    • The .ts file format allows for embedding multiple streams of data – for proper processing, the following streams are needed
      • Video – required
      • KLV Data – required for geospatial display
      • Audio – optional
    • The following steps are performed to process .ts files
      • Stream Identification – determine the contents of the file
      • Creation of mp4 – using video and audio
      • Creation of a reformatted .ts file containing only one video stream, one audio stream, and one KLV stream – saved in /SA for use by external systems like ATAK and ESRI.
      • Extraction of KLV –
      • Creation of a JSON from KLV –
      • Creation of a VIDEOCLIP.kml – placed in /VIDEO folder to allow SA to zoom to video location.
        • If a narrative .xml file is included, the narrative information is inserted into the KML.
        • .xml file – must contain the following data for processing. A sample is provided as /SampleProducts/. The .xml file must be structure and include the below:
<PAMDataSet>
    <Metadata>
        <MDI key="DESCRIPTION">narrative text</MDI>
    </Metadata>
</PAMDataset>

        • URL of the reformatted .ts file (/SA) is inserted into the KML.
      • Creation of a DPS kml file to display the footprint of the video in SA.




SA Products

SA will transfer tactical information every 5 minutes for viewing on Aircraft systems.  These products will consist of .kmz files of the Operations Markup (tactical fire information), Special Risk (areas of defined risk), and Resource Locations (as available). They will be transferred to the S3 data exchange mechanism for direct ingest by aircraft systems. 
The SA products include:
  • Operations Incidents – a .kmz file containing active incidents and tactical mapping information.
  • Special Risk – a .kmz file containing damage assessment information as mapped in SA.
  • Resources – a .kmz file resources tracked in SA.

Naming Conventions

SA products will utilize the following naming conventions.
  • Operations Incidents.
    • Example = OperationsIncidents.kmz
  • Special Risk.
    • Example = SpecialRisk.kmz
  • Resources.
    • Example = Resources.kmz

SA Products

SA products are .kmz formatted files.  Samples are provided as /SampleProducts/.


Data Exchange Mechanism

Amazon S3

A secure Amazon S3 file sharing system will be used by Remote Sensing and SA to exchange data. The Amazon S3 system requires an authorization credential issued by SA for data exchange.
The S3 system is structured into the following file folders  - commonly called ‘buckets’:

/snccostate/

/AIRCRAFT/ – for Aircraft position files
/SA/ – for data exchange FROM SA to Remote Sensing [future]
/IMAGERY/ – for Imagery products
/MISSION/ – for Mission files
/TACTICAL/ – For Tactical products
/VIDEO/ - For Video products

Data Transfer Schedule

Intterra will review and check the status of the S3 system on the following schedule.
  • Mission file list is updated every 2 minutes.
  • Live Mission Status Information is posted/updated based on following:
    • Normal Operations – every 10 minutes
    • Once a Mission File has been posted until Mission File is deleted – every 30 seconds
  • At any time, an uploaded Imagery or Tactical product from aircraft will prompt Intterra to add data to system based on name of file.
    • Uploaded files will be matched to Incident Pull Down List based on following schedule
      • Every hour
      • Every 2 minutes after a Mission File is uploaded and date of file is <1 hour old
    • Aircraft products will be associated with Incidents from Operations Markup that are within 10KM of the product
Remote Sensing systems will upload data to S3 based on the following schedule:
  • Mission File – loaded when mission is planned, deleted when mission is completed
  • Aircraft Position – every 30 seconds during mission
  • Imagery, Video and Tactical Products – as produced
SA will send Operations Data to S3 based on the following schedule:
  • Every hour
  • Every 2 minutes after a Mission File is uploaded and date of file is <1 hour old

Data Retention

The S3 files will be managed based on content: 
  • Mission File/Aircraft Files – The MSO will manage the content of the Mission and Aircraft buckets to reflect current and active missions.
  • Tactical/Imagery Files – Operations and the SA project manager (or designated personnel) will review the contents of the tactical and imagery files weekly to ensure that expired missions are removed from the system and high interest products remain. All data removed from S3 will be archived by the SA project manager (or designated personnel).
Per Remote Sensing Operating Guide, all mission critical data generated by Remote Sensing systems will be stored on a separate storage device at the conclusion of a mission.


SA Display

The SA Aircraft Intel Tool provides status updates to users using the data stored by the Remote Sensing systems within the Amazon S3 file storage.
  • Inactive – no current missions
  • Planned – a mission file has been submitted for an upcoming mission
  • Airborne – aircraft is flying and transmitting location but not products have been created
  • Collecting – aircraft is flying, transmitting location and at least one product has been created
To preserve bandwidth for mobile devices, SA will display the ‘most recent’ image projected onto the map. All other images will be represented by a polygon showing their location on the ground. Images and products can be turned on/off or downloaded to browser using the interface.