Transfer Learning by Discovering Latent Task Parametrizations

Introduction

In many real-world applications of learning for control, an agent is expected to repeatedly encounter tasks with dynamics that are similar, but never quite the same. For example, when learning to manipulate glasses of water, an agent might encounter glasses with different masses and different amounts of liquid. Similarly, when learning to drive a vehicle, an agent will encounter many different individual vehicles, each with (for example) brakes that behave slightly differently. Over the course of learning to fly a plane, an agent will encounter may different planes, each with slightly different handling characteristics, and carrying different loads.

In all of these scenarios, it makes little sense of the agent to start learning afresh when it encounters a new glass, a new vehicle, or a new plane. Indeed, one would hope that the more manipulation, driving, or flying tasks the agent performed, the more quickly and reliably the agent could adapt to new instances of the same type of tasks.

Tasks with these closely-related dynamics provide an interesting regime for transfer learning. Intuitively, these tasks seem to have some low-dimensional latent factors that parametrize the Dynamics in structured ways: for example, the amount of liquid in the cup can be thought of a one-dimensional latent parameter which causes smooth changes in the cup’s dynamics. Furthermore, in many interesting problems, either these latent parameters remain fixed for the duration of the task (the state of a car’s brakes will not change significantly during a single trip), or the agent will know when a change has occurred (the agent is driving a different car).
 
Authors: Finale Doshi-Velez and George Konidaris
Source and Full Article:

0 yorum: