I've had many panic attacks while driving. Actually, that's where my first one happened and I think it left a bad taste in my mouth because driving alone is still really problematic for me. I've gone through periods where leaving the house to go to the mailbox is just out of the question, but that seems silly when you really think about it. Then again, saying I can't go to Walmart by myself seems pretty silly, too, but I still can't do it. I've had panic attacks for the past three years, but for about six months it has been an everyday issue, not just a once in a while panic attack. Not spending much time outside of my house would drive me crazy, but I totally understand where you're coming from. Doing things with friends really helps me to get out and have a social life. It is uncomfortable at times, but what's really going to happen? Nothing. I find myself rationalizing a lot, but I feel like you have to talk yourself through it sometimes instead of giving into it.
Best of luck!