I agree with gcalex. I've been using Xanax for my anxiety and panic attacks for many years. I used to use it quite frequently, but now I'm down to only taking it when I really need it. I can tell you from personal experience that it can be an awful process tapering off of benzos like Xanax (alprazolam), Klonopin (clonazepam), Ativan (lorazepam), etc. Benzodiazepines are pretty habit forming, and you never want to stop taking them abruptly or taper too quickly. Some of the symptoms I experienced when tapering down my dose included headaches, nausea, dizziniess / lightheadedness, irritability, muscle tremors, and increased anxiety / panic attacks. I felt like crap! And I was just tapering from 2mg a day to 0.5mg a day! You've been taking twice that amount! You should talk to your doctor about taking a little more time to wean yourself off of them.
Also, talk to your doc about maybe taking a natural supplement to help ease the process. Xanax alters the GABA levels in our brains. Because you've been taking it for a while, your GABA levels have been heightened for an extended period of time. So, naturally, when you start tapering off the Xanax, your brain's GABA levels are getting lower and lower, and even if they're just coming back down to a normal level, if they've been higher for a sustained time period, normal levels can feel too low. This is very much like weaning yourself off of SSRI's (such as Zoloft or Celexa), but instead of the drugs altering your seretonin levels, they're altering your GABA instead. The withdrawal symptoms can be very similar though.
There are GABA supplements you can find in the vitamin / supplement section of most grocery stores, online, and at supplement specialty stores. They may ease your symptoms a little bit, and eventually you may want to switch to these completely. Also, Sam-e, B Vitamin Complexes, Valerian Root, and melatonin are some other great natural alternatives to prescription benzos. Of course, you should always talk to your doc first before you start any new supplements or before you take anything new in combination with meds you're already being prescribed.