Natural Evolution Strategies

Abstract:
We introduce a novel stochastic search method, Natural Evolution Strategies, which constitutes a principled alternative to conventional evolutionary methods. Like evolutionary algorithms, this method is used to optimize an unknown `fitness' function. But instead of using evolutionary concepts such as mutation and selection, our algorithm maintains, batch-by-batch, a multinormal search distribution whose characterizing weights are updated using the natural gradient in the direction of higher expected fitness. Heavily inspired by recent developments in the policy gradient framework, this method is derived from first principles. Experimental work shows the viability of this approach on a standard benchmark set of both unimodal en multimodal functions, and shows superior results on learning the weights of a difficult reinforcement-learned recurrent neural network for non-Markov double pole balancing.

back