Rest assured, if they did imaging and didn't see anything abnormal, there's nothing there to worry about. One of my close friends is an ER RN and she tells me they see imaging every day: young, old, tall short, thin, big, and of course: sick and healthy. They've learned not only through training, but by experience, how to spot anomalies in images. I told her that I worried about a brain tumor because of how my head would feel (pressure, tingling, etc) and she assured me that if it was a tumor and I was feeling strong effects from it, I wouldn't be able to hold a normal conversation with her or hold eye contact. My neurologist said something along the same lines, that major ailments have specific side effects that are obvious from anxiety.
I think dr's offices are almost required to give the "Go to ER" response. They can't diagnose you over the phone, and by small chance it is something life threatening, that's the best course for immediate care. They have to play it safe. My neurologist's office told me the same thing when I was having strong lightheadedness while the dr was on vacation.