Hello Shaurya Thapliyal,
Following might be the reason for your error:
The possible reason could be that the recognizer_instance.energy_threshold property is probably set to a value that is too high to start off with. You should decrease this threshold, or call recognizer_instance.adjust_for_ambient_noise(source, duration = 1).
If it's a continious loop of audio, the stream might never leave audio = r.listen(source) or your identation is wrong. Consider using a non-blocking object such as the select library. If you're on Windows I'd might thread the output some how.
Use this ambient noise suppression/adjustment;
Ensure pyaudio.h is installed by running the below command
sudo apt-get install portaudio19-dev python-pyaudio python3-pyaudio
Hope it helps!!