You probably have a sinus headache. Does this happen when you change positions? Cause that'll do it. And no, by itself, its not anything to worry about.
Also check out the list of questions on this message board. You'll see a significant number very recently that also have this complaint. Theres a flu or some such going around. I have it too, its miserable.