Onboard Autonomous Health Assessment and Global Localization for the Mars CL24_5990.pdf

Onboard Autonomous Health
Assessment and Global
Localization for the Mars Helicopter

1. i-SAIRAS 2024
Onboard Autonomous Health
Assessment and Global
Localization for the Mars
Helicopter: Towards Multi-
Flight OperationsConnor Basich, Cecilia Mauceri, Gerik Kubiak,
Juan Delfa, Alberto Candela, Pedro Proenca,
Barry Ridge and Steve Chien.
Presented by Alberto Candela
For required markings, please visit https://mh.jpl.nasa.gov

2. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Background
Mars Helicopter, Ingenuity
•First powered, controlled flight on
another planet.
•Potential for aerial mobility
completely alters space of future
science missions!
•Cover multiple dimensions of
exploration.
•Access to hazardous but
scientifically interesting terrain.
•Opportunity for in-situ atmospheric
science.
2For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Figure 1: The Mars Helicopter, Ingenuity. NASA's Mars
Perseverance rover acquired this image using its Left
Mastcam-Z camera. Mastcam-Z is a pair of cameras
located high on the rover's mast. This image was acquired
onApril6,2021(Sol45). 
Image Credit: NASA/JPL-Caltech/ASU

3. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Background
Mars Helicopter, Ingenuity
•Initially intended to perform 5
flights.
•Successfully completed 72
flights!
•Longevity maintained by
extensive post-flight health
assessment and localization
manually performed by engineers
on ground after each flight.
3For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Figure 1: The Mars Helicopter, Ingenuity. NASA's Mars
Perseverance rover acquired this image using its Left
Mastcam-Z camera. Mastcam-Z is a pair of cameras
located high on the rover's mast. This image was acquired
onApril6,2021(Sol45). 
Image Credit: NASA/JPL-Caltech/ASU

4. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Ingenuity OPS Process (right):
1.Ingenuity performs flight specified by
flight sequence.
2.Telemetry, logs, and images from
flight are downlinked to Earth.
•Generally same-day for low-rate,
up to a week for all high-rate.
3.Telemetry and logs manually
reviewed to assess flight readiness
for next flight.
4.Navcam images taken during
descent are manually cross-
referenced with global reference
map to determine system pose due
to VIO drift between flights.
5.Upon success, next flight sequence
is encoded and uplinked.
4For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Enabling Multi-Flight Operations

5. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
5For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Problem: Cannot perform multiple flights
per ground cycle, severely limiting
scientific productivity on future missions!
Ingenuity OPS Process (right):
1.Ingenuity performs flight specified by
flight sequence.
2.Telemetry, logs, and images from
flight are downlinked to Earth.
•Generally same-day for low-rate,
up to a week for all high-rate.
3.Telemetry and logs manually
reviewed to assess flight readiness
for next flight.
4.Navcam images taken during
descent are manually cross-
referenced with global reference
map to determine system pose due
to VIO drift between flights.
5.Upon success, next flight sequence
is encoded and uplinked.
Enabling Multi-Flight Operations

6. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Enabling Multi-Flight Operations
Proposed OPS Process (Right):
1.Post-flight, perform global localization
and health assessment onboard the
helicopter.
7For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Solution: Automate core assessment
criteria onboard the spacecraft.

7. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Enabling Multi-Flight Operations
Proposed OPS Process (Right):
1.Post-flight, perform global localization
and health assessment onboard the
helicopter.
1.Global localization will autonomously
determine the heading and pose of the
spacecraft from navcam images during
flight.
8For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Solution: Automate core assessment
criteria onboard the spacecraft.

8. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Enabling Multi-Flight Operations
Proposed OPS Process (Right):
1.Post-flight, perform global localization
and health assessment onboard the
helicopter.
1.Global localization will autonomously
determine the heading and pose of the
spacecraft from navcam images during
flight.
2.Health assessment will be
autonomously determined via a
classification model performing
inference onboard based on flight
telemetry and logs.
9For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Solution: Automate core assessment
criteria onboard the spacecraft.

9. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Enabling Multi-Flight Operations
Proposed OPS Process (Right):
1.Post-flight, perform global localization
and health assessment onboard the
helicopter.
1.Global localization will autonomously
determine the heading and pose of the
spacecraft from navcam images during
flight.
2.Health assessment will be
autonomously determined via a
classification model performing
inference onboard based on flight
telemetry and logs.
2.If positive assessment and successful
localization, ready next flight sequence.
10For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Solution: Automate core assessment
criteria onboard the spacecraft.

10. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Motivation
Enabling Multi-Flight Operations
Proposed OPS Process (Right):
1.Post-flight, perform global localization
and health assessment onboard the
helicopter.
1.Global localization will autonomously
determine the heading and pose of the
spacecraft from navcam images during
flight.
2.Health assessment will be
autonomously determined via a
classification model performing
inference onboard based on flight
telemetry and logs.
2.If positive assessment and successful
localization, ready next flight sequence.
3.If negative assessment or unsuccessful
localization, downlink all data and
perform standard ground cycle.
11For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
Solution: Automate core assessment
criteria onboard the spacecraft.

11. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Proposed Work
•OUTLAST (Onboard Automated Health Assessment)
•System that autonomously performs health assessment onboard the
spacecraftusingpriorflight’sdata.
•Utilizes decision trees to address operational concerns:
•Data efficiency
•Interpretability
•Ability to directly encode manual decision criteria
•AGL (Automated Global Localization)
•Systemthatautonomouslylocalizesthespacecraft’sposeandheading
onboardusingpriorflight’snavigationcamera(navcam)images.
•Utilizes two-phase census matchings to more efficiently identify most
likely pose from global reference map.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 12

12. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Onboard Automated Health Assessment
(OUTLAST)
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 13
•Objective: automate health assessment process onboard spacecraft.
•By running post-flight, can perform health assessments that are based on the
collective data over the course of the entire flight.
•In contrast to traditional reactive FDIR methods.
•This will also reduce the bandwidth burden in downlinking all data after each flight.
•Limited operational uptime.
•Places additional burden on Perseverance by transmitting all data through the
rover.
•OUTLAST (Onboard Automated Health Assessment)
•System that autonomously performs health assessment onboard the spacecraft
usingpriorflight’sdata.
•Utilizes decision trees to address operational concerns:
•Data efficiency
•Interpretability
•Ability to directly encode manual decision criteria

13. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Onboard Automated Health Assessment
(OUTLAST)Proposed Approach
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 14

14. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Onboard Automated Health Assessment
(OUTLAST)Proposed Approach
Offline:
1.Configuration file is created to specify decision
tree parameters or manual rules.
2.Rules are processed using the config file.
3.Decision tree model is generated and saved.
Online:
1.Onboard telemetry and log files from previous
flight are collected as binaries and converted to
usable format (csv files).
2.Data files are then processed to handle broken
data, bin by (known) flight mode, and process
into desired decision variables.
3.Decision tree model is run on the processed
data input to produce either a GO or NOGO
decision as well as a diagnostic file for
downlink.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 15

15. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Onboard Automated Health Assessment
(OUTLAST)Testing and Results
Outlast was tested over several stages:
1.Successfully run on all 7 complete prior flight datasets
1.And correctly identified off-nominal behavior in a flight previously
marked nominal  provides valuable insight into flight results that
may not be immediately apparent.
2.OUTLAST was validated and verified on the Ingenuity FlatSat.
3.OUTLAST was successfully run onboard Ingenuity on January 5,
2024 just prior to system
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 16

16. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Automated Onboard Global Localization
•Objective: automate global localization
process onboard spacecraft.
•Upon waking, Ingenuity had no sense of
position or orientation.
•Ingenuity was localized on the ground using
navcam images (see Right) taken during
descent system.
•These images would be manually referenced
against a global reference map (the Mars
2020 Terrain Relative Navigation HiRISE
Orthorectified Image Mosaic) to localize the
spacecraft.
17For required markings, please visit https://mh.jpl.nasa.gov11/24/2024
•AGL (Automated Global Localization)
•Systemthatautonomouslylocalizesthespacecraft’sposeandheading
onboardusingpriorflight’snavigationcamera(navcam)images.
•Utilizes two-phase census matchings to more efficiently identify most likely
pose from global reference map.

17. jpl.nasa.gov
Automated Onboard Global Localization
Proposed Approach
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 18
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations

18. jpl.nasa.gov
Automated Onboard Global Localization
Process
1.Global census match compares
navcam images to whole area
map, finding map location with
highest correlation to whole
image.
2.Local census match takes output
and computes the homography
between the image and map
using brute force feature matcher.
Results:
1.Improved23/36flights’pose
estimates.
2.Improvement in mean heading by
1.5
o
3.Unable to deploy onboard as
could not reach 100% precision
solution with < 10m translation
error and <1
o
error.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 19
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations

19. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Conclusion
•Ingenuity’ssuccesshasdemonstratedthevastpotentialforspace
rotorcraft in future missions.
•Ground-in-the-loop (GitL) cycle after each flight helped maintain longevity
of system.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 20

20. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Conclusion
•Ingenuity’ssuccesshasdemonstratedthevastpotentialforspace
rotorcraft in future missions.
•Ground-in-the-loop (GitL) cycle after each flight helped maintain longevity
of system.
•Requiring GitL cycle between each flight prevents multi-flight
operations
•Limits potential science productivity of future missions like Mars Science
Helicopter.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 21

21. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Conclusion
•Ingenuity’ssuccesshasdemonstratedthevastpotentialforspace
rotorcraft in future missions.
•Ground-in-the-loop (GitL) cycle after each flight helped maintain longevity
of system.
•Requiring GitL cycle between each flight prevents multi-flight
operations
•Limits potential science productivity of future missions like Mars Science
Helicopter.
•To support multi-flight operations, we developed:
•An autonomous system for global localization (AGL) using census
matching on navcam imagery.
•An autonomous onboard health assessment system (OUTLAST) using
decision tree classification on flight data.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 22

22. jpl.nasa.gov
Onboard Autonomous Health Assessment and Global Localization for the Mars Helicopter: Towards Multi-Flight Operations
Conclusion
•Ingenuity’ssuccesshasdemonstratedthevastpotentialforspace
rotorcraft in future missions.
•Ground-in-the-loop (GitL) cycle after each flight helped maintain longevity
of system.
•Requiring GitL cycle between each flight prevents multi-flight
operations
•Limits potential science productivity of future missions like Mars Science
Helicopter.
•To support multi-flight operations, we developed:
•An autonomous system for global localization (AGL) using census
matching on navcam imagery.
•An autonomous onboard health assessment system (OUTLAST) using
decision tree classification on flight data.
•Both AGL and OUTLAST were tested in simulation on prior flight data.
•OUTLAST was run successfully onboard Ingenuity in January 2024,
demonstrating the potential for the method to be successful on future
missions.
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 23

23. jpl.nasa.gov
11/24/2024 For required markings, please visit https://mh.jpl.nasa.gov 24