I have IBS and although I haven't had the same symptoms, I've had others that are scary including white mucous in my stool among other things.
You had a good work up with your Dr. and you should feel satisfied with that. It's not like he just passed you off - he actually did some tests to ensure nothing was wrong. If you are still concerned, however, I would recommend finding a GI specialist and see if you can have a colonoscopy done - that will be the best way to determine if something is going on within your bowels. Just a suggestion.
As for going days without going to the bathroom, then going and having a normal movement followed by diarrhea, that is classic IBS. I get that too, a lot. I take a strong probiotic daily which helps my bowel movements a lot.
Also, anxiety plays a role in your digestive health as the majority of seratonin is in your gut. Every time I get nervous/anxiety attack, I have loose stool.