Welcome to the forums! It is nice to meet you.
It sounds like you have been feeling very lonely and in some way, I guess I am glad you've been able to find your way here. I think most people feel lonely at some point or another, but these forums are a place where people feel more able to openly talk about just how hard it can be.
From your post, I think I have a fair picture of how loneliness feels to you and there are certainly many parts that I also have felt or feel sporadically at times. Things like fear of rejection and pre-emptively isolating myself, or kind of feeling like (in my words) a watcher than someone participating.
I don't really know the answer to this, but I think loneliness is quite normal and it is also normal to really struggle with it. I think we then individually find what helps - some reach out to new people as you've suggested, while others try to forge stronger bonds with their existing friends. My own experience has been to go back into myself and really face these feelings of loneliness head on, not always successfully, but overall with a much better ability to be alone and content. I'll never be a hermit, but I can happily spend weekends on my own, without worrying that I don't have enough friends.
Is there anything in particular you feel you could benefit from? You mentioned striving to improve yourself rather than agonising over what you haven't got - what are your thoughts there?