The origin of the band gap bowing in dilute nitrogen doped gallium based III-V semiconductors is largely debated. In this paper we show the dilute GaAs1−xNx and GaP1−xNx as representative examples that the nitrogen-induced states close to the conduction band minimum propagate along the zigzag chains on the {110} planes. Thereby states originating from different N atoms interact with each other resulting in broadening of the nitrogen-induced states which narrows the band gap. Our modeling based on ab initio theoretical calculations explains the experimentally observed N concentration dependent band gap narrowing both qualitatively and quantitatively.