Adapting on the Fly