Hey there. I know exactly what you're going through. I dropped out of college this past fall because it was beginning to become unbearably stressful. My panic attacks were so severe Id pull out chunks of my hair and claw at myself until the skin bruised. It was just a god awful mess, and as much as I didn't want to drop out, I just had to. And it ended up being a good decision! It gave me room to breath, and although I still attend therapy and take medication my anxiety has subsided a great deal. College isn't always the right choice for everyone. Personally, I don't think it's a bad idea to take a few years to yourself and learn to grow as pesto before you decide on what you want to do with your life. Take some time for yourself: travel, expand your horizons, seek help if you feel it would benefit you, and just have fun. Life is what happens when you're busy making other plans.
As for your boyfriend, don't take it too personally. Our anxiety issues may be hard on us, but I wouldn't doubt that it's even harder on our loved ones. I imagine it can get frustrating when a person doesn't know what to do or how to help and it just ends up making everything worse in the end. Some people are more equipped to handle certain things than others.
Hope I was able to offer some good advice. Keep your chin up, things will get better.