It could be a little of both, but honestly it's most likely just anxiety. I know this is harder than it sounds, but try to do yoga or some other hobby that will take your mind off of the dizziness. If you can forget about it and it's no longer there, then it's most likely the anxiety. Even if it's your inner-ear don't panic because it cannot kill you. It just makes you feel a bit dizzy once in awhile, and there are exercises and nose sprays to help with that.
Hope you feel better! Dizziness, I feel, is the worse part of anxiety.