(Smooth) Fictitious Play in Repeated Games

In this review we compare two common methods for playing repeated games: Fictitious Play (FP) and Smooth (or Stochastic) Fictitious Play (sFP). We formulate both techniques theoretically and describe how FP can be considered as a limit of sFP. We then study their convergence in two-player zero-sum games by both surveying the literature on the theoretical guarantees of convergence, and investigating the convergence via simulation. In particular we use simulations to exhibit the difference between convergence in belief (for FP and sFP) and convergence in behaviour (for sFP but not FP). We discuss how this lack of convergence of FP is due to the play of pure strategies. We also discuss convergence of these techniques in more general bimatrix games, where there exist cases where they do not converge. We consider a simple $2 \times 2$ coordination game, where although FP and sFP both converge, sFP favours the pure strategy equilibria, and FP converges to the mixed equilibrium in belief, but fails to converge to the value of the game when the mixed NE is played. We also consider the example of Shapley’s game and see that there are initial conditions where neither FP nor sFP converge in any sense. We investigate how the temperature parameter can be adjusted to prevent sFP from falling into a cycle in Shapley’s game.

You can find the paper here.