While training small neural networks to perform binary addition, a surprising solution emerged that allows the network to solve the problem very effectively. This post explores the mechanism behind that solution and how it relates to analog electronics.