This just popped into my mind when i was reading your post, especially this bit
" I worry so much about my kids not being able to make friends and relate to other people and other kids finding them weird and them not understanding why. My cousin was like this at school and I suspect he might have had Aspergers although he was never diagnosed and he had a horrible time at school. He's fine now he's an adult. I just worry so so much about the future."
Were you yourself a child like this and now worry that it will be the same for your son as it was for you and your cousin or maybe you were at the other end, popular and sociable, and you find it hard to see it from the opposite point of view and think that your son will suffer from this even though it may not be the case.
I"'m not sure if i'm being logical but i just thought that maybe you need to look into your own childhood and find if there's something there what may be bringing on these fears. I can partly understand you as i have a 3 year old son myself. I am scared that he will turn into a bully and i am struggling because i'm not sure how i should be bringing him up and disciplining him. He's a very strong-willed, energetic and maybe a little mischievous boy but definitely not bad or naughty. But as i was bullied myself for maybe being, not the same, but similar as you described, i am scared of him becoming a bully. Also i have found so many reasons from my childhood and youth for my anxiety and HA, even though my childhood was pretty normal. I still dont know how to help myself and my HA with these realisations but maybe i've come one step further.
Your son may become this personality that you described whether he has Aspergers or not , and maybe it is your job to find how to make the best out of whatever personality he is, we all have our strenghts, and your son sounds quite talented, and you need to focus on his strengths instead of worrying over something that may not be worth worrying. Hope this makes sense, i am not very good at putting my thoughts into words.