How do I know If I've been sexually abused as a child?

When I was younger my older brother and I would have "sleepovers" where he'd let me sleep in his bed and stay up late, and I loved it because I felt like a big kid. I was so young I've forgotten most of what happened or how often it happened, but I have one image in my head of my brother and I laying in his bed and looking down each others pants. I also recall touching his penis, but that may have been just innocent childhood curiosity. I can't remember if he ever touched me, and I feel that because I was curious about 'boy parts' nothing was done against my will. However is it possible worse things happened that I've blocked out of my memory? I find myself very uneasy when it comes to people touching me or getting too close to me, and often feel violated by guys, especially older ones even if they're never physical. I have nightmares about being taken advantage of and raped, and have always felt as though I was prone to be taken advantage of or "supposed" to be taken advantage of. I've suppressed these thoughts and feelings for a few years now, because up until recently I'd forgotten about the experiences with my brother. Now, I'm really looking for some possible answers and opinions. Please help! Thank you