## Overview

My research involves using the concept of ergodicity to understand the way in which search or exploration intention can be connected to movement control. Ergodicity can be used to compare the statistics of a search trajectory, the movement or actions of a robot or animal, to a map of expected information density. Using this principle I have developed an open-loop method of generating optimal control strategies for sensory systems subject to both hybrid and continuous dynamics.

Ergodic theory relates the time-averaged behavior of a system to the space of all possible states of the system, primarily used in the study of fluid systems and mixing. As a control objective, the basic idea is that an efficient exploration strategy—e.g. the path followed by a robotic arm—should spend more time exploring regions of space with a higher information density (darker regions), or where useful measurements are most likely to be found. The distinction between ergodic optimal control and previous work in active sensing is that in ergodic optimal control, a system should not only visit the highest information region, but distribute the amount of time spent searching proportional to a distribution (e.g. of expected information). This is in contrast to how many alternative algorithms work, which directly maximize information gain based on the current best estimate (shown graphically above), whether or not that estimate is actually correct. This distinction is illustrated graphically below.

Ergodic theory relates the time-averaged behavior of a system to the space of all possible states of the system, primarily used in the study of fluid systems and mixing. As a control objective, the basic idea is that an efficient exploration strategy—e.g. the path followed by a robotic arm—should spend more time exploring regions of space with a higher information density (darker regions), or where useful measurements are most likely to be found. The distinction between ergodic optimal control and previous work in active sensing is that in ergodic optimal control, a system should not only visit the highest information region, but distribute the amount of time spent searching proportional to a distribution (e.g. of expected information). This is in contrast to how many alternative algorithms work, which directly maximize information gain based on the current best estimate (shown graphically above), whether or not that estimate is actually correct. This distinction is illustrated graphically below.

Ergodic optimal control involves two major (separable) components:

I use a metric on ergodicity that relies on Fourier decomposition, originally proposed by George Mathew and Igor Mezic. Using methods from optimal control, it is possible to directly calculate trajectories that are ergodic with respect to a given information objective. A projection-based trajectory optimization technique makes it possible to calculate optimal trajectories for systems with arbitrary dynamics. Examples of optimally ergodic trajectories are shown below for four different distributions. For more information, please take a look at the related publications listed below.

- A metric on ergodicity, that relates a trajectory to a distribution

- A control strategy to minimize the ergodic metric

I use a metric on ergodicity that relies on Fourier decomposition, originally proposed by George Mathew and Igor Mezic. Using methods from optimal control, it is possible to directly calculate trajectories that are ergodic with respect to a given information objective. A projection-based trajectory optimization technique makes it possible to calculate optimal trajectories for systems with arbitrary dynamics. Examples of optimally ergodic trajectories are shown below for four different distributions. For more information, please take a look at the related publications listed below.

## Features

Ergodic trajectory optimization:

- is suitable for both coverage (diffuse) or “hotspot” (concentrated) sampling, without modification (see examples above)

- is suitable for systems with linear or nonlinear, kinematic or dynamic motion constraints (see example below)
- does not rely on discretization of the search space, the action space, or the belief space (discretization is an implementation decision as opposed to a part of the problem statement or solution)
- does not depend on heuristics, e.g. selecting waypoints, to distribute measurements among different regions of high expected information when planning over a long time horizon

Ergodic trajectory optimization extends to nonlinear, dynamic systems. The optimal trajectory for a two-link robotic arm, with respect to a planar distribution (grayscale), is plotted in red above. The trajectory at different iterations of the optimization are shownare plotted at different iterations of optimization.

## Variations and extensions

**Optimal contact decisions for ergodic exploration:**In this scenario, the control variable is a set of transition times between hybrid dynamic modes and/or measurement modes. An example would be tactile sensing, where measurements can only be collected during contact, but ballistic motion between regions of high information may be faster or less energetically costly than remaining in contact. Using techniques from switching-time optimization, one can determine optimal transitions between such modes, assuming order is fixed.

**Closed-loop estimation using ergodic optimal control:**Using tools from probability theory, information theory, and optimal control, ergodicity can be used as an objective function to create an adaptable algorithm for autonomous robotic exploration. See information-based search for more information.

**Optimization in higher dimensions or on Lie Goups:**Ergodic trajectory optimization can be extended to higher dimensions and Lie groups, provided the Fourier transform exists. An optimal trajectory for a 3D distribution is shown below

**Time-varying Distributions:**Ergodic trajectory optimization can be easily extend to apply to time-varying distributions as well as static distributions (unpublished--coming soon!)

## Related Publications

L.M. Miller and T.D. Murphey, “Trajectory optimization for continuous ergodic exploration on the motion group SE(2),” in IEEE Int. Conf. on Decision and Control (CDC), 2013, pp. 4517 – 4522

L.M. Miller and T.D. Murphey, “Trajectory optimization for continuous ergodic exploration,” in American Controls Conf. (ACC), 2013, pp. 4196–4201

L.M. Miller and T.D. Murphey, “Optimal contact decisions for ergodic exploration,” in IEEE Int. Conf. on Decision and Control (CDC), 2012, pp. 5091–5097

L.M. Miller and T.D. Murphey, “Trajectory optimization for continuous ergodic exploration,” in American Controls Conf. (ACC), 2013, pp. 4196–4201

L.M. Miller and T.D. Murphey, “Optimal contact decisions for ergodic exploration,” in IEEE Int. Conf. on Decision and Control (CDC), 2012, pp. 5091–5097