We benchmark many-body perturbation theory by studying neutral, as well as non-neutral, excitations of finite lattice systems. The neutral excitation spectra are obtained by time-propagating the Kadanoff–Baym equations in the Hartree–Fock and the second Born approximations. Our method is equivalent to solving the Bethe–Salpeter equation with a high-level kernel while respecting self-consistency, which guarantees the fulfillment of a frequency sum rule. As a result, we find that a time-local method, such as Hartree–Fock, can give incomplete spectra, while already the second Born approximation, which is the simplest time-non-local approximation, reproduces well most of the additional excitations, which are characterized as double excitations.