I am younger than you, and I have these thoughts a lot. That, or I worry that I will be physically unable to have kids when I'm ready.
It helps to remember that having a biological family, or really any sort of family, is not necessary for you to have a successful life. Many people are happy without children. Many people also do not need a spouse to complete them. You have to follow your heart, what you want, rather than stressing yourself about being perhaps a little different than others.