I’m having trouble locating an old post about optimization daemons that I know I read at one point. I believe it was written by Eliezer. It featured a lot of visual imagery of the daemon steering the path of the model through parameter-space by e.g. forcing it through narrow paths towards low loss surrounded by walls of high loss. Does anyone know which post I’m remembering?
This probably isn’t the thing you mean, but your description kinda sounds like tessellating hills and its predecessor demons in imperfect search.
Comment
Ah, it was John’s post I was thinking of; thanks! (Apologies to John for not remembering it was his writing, although I suppose mistaking someone’s visual imagery on a technical topic for Eliezer’s might be considered an accidental compliment :).)