Yeah, everybody has a different tolerance for caffeine. Maybe 3 cups is OK for you, but 4 is overkill. Next time, maybe see how you feel with each cup -- if you're too anxious, or if you feel good. You might need less coffee than you realize to get that energized feeling.
As a general rule for anxiety, cutting out stimulants like caffeine can help keep your mood more stable and calm.