This is an extremely common anxiety symptom. Most here have experienced it too. Doctors noted this occurrence in anxiety attacks, and did some studies about it.
It turns out that when we experience this breathless state, there is absolutely no physical change to anything. we are breathing normally. We just feel like we aren't. Its not entirely clear why this happens, but that it has no physical cause is without dispute.
I know that won't make your symptom stop, but hopefully the knowledge that there is no way for it to harm you will give you some peace of mind.