Oh I hear you so very much on this. I'm 26, and was told around a year ago that I have very high cholesterol, which is obviously a concern. I'm greatly overweight - I was going on for 350lbs, which is clearly insanely high(BMI something like 45 ><). I started dieting, and have lost over 40lbs so far, as well as getting my cholesterol down, but I was always very scared of exercising. I've never been completely idle - in college I used to have to regularly walk up five sets of stairs three times a day, and I've always tried to walk 30-60 minutes a day. Yet I was scared - increasing it, I was convinced, would result in a heart attack.
I spoke to my doctor, and he reminded me that our hearts are very strong, and that an undamaged one can easily handle to strain of daily living, and that of people a lot more active that me - think of what, for example, rugby players put theirs through. Some of them weigh as much, if not more than me, and they're pushing themselves to the limit day after day. That made me feel a lot better, and with his assurances I've upped my activity levels a little, small steps at a time. It's hard, and I often feel it really hard - weakness, chest pain, difficulty breathing, palpitations - but I know it's just my body struggling from the lack of exercise I've done over the last few months.
From your levels, it sounds like your heart is healthy, despite your activity levels. The symptoms you describe having with exercise can certainly be your body merely struggling with increased activity, or even simply anxiety and stress at what you're doing. Also note, that your age is a big factor. Having heart problems as young as we are is a pretty rare thing.