Paying Attention to the Noise