Hi pinkbutterfly
You sound like the most beautiful person, so easily able to see the best in people, including your boyfriend.
It sounds like your boyfriend's facing a significant challenge in his life which involves coming to know himself better, so as to understand why he has these outbursts and how he needs to manage his frustration, levels of intolerance and so on. One thing I believe which is so important for you to consider is - he's never been led to hit you, so far. 'So far' is something you need to consider. For example, just say you stay with him and eventually have a baby with him. As a mum, I can tell you a baby can really push your buttons, especially when they're crying a lot with colic or hunger or the need for sleep etc. Your boyfriend, I imagine, hasn't faced this type of stress in his life. Imagine you discover he's not at all good with it and blames you for the baby crying. The baby keeps crying and he blames you more. You become more and more anxious as he becomes angrier. The baby continues crying, even louder now, sensing the building tension. Baby's are highly sensitive. The crying gets louder and all of a sudden your boyfriend hits you as he says 'I told you to shut it up!'. Of course, this may not happen but there are no guarantees.
What can you see happening in the future, based on your boyfriend not accepting the challenge when it comes to better understanding himself, his triggers, his feelings, his thoughts, his tolerance levels etc? I imagine he'll stay the same or maybe even get worse. There's not doubt, he needs to face this challenge constructively, perhaps on his own.
While you see the best in him, you know deep down, the best in him won't cancel out his toxic nature without him addressing this nature.
To say to him 'While I can easily see the best in you, I can't live with the worst in you' is fair, I believe. For you, if the worst in him is becoming depressing, anxiety inducing, incredibly stressful and is leading you to low self esteem, he's taking away the best in you. How much of you are you going to allow his behaviour to take?
I imagine, if you were to speak to others who've faced domestic abuse for a number of years, they'd most likely say 'My partner took the best in me and destroyed it. He/she took my confidence, my happiness, my fearlessness, my fun loving nature and more and just destroyed it all'. Pinkbutterfly, don't stay for that, don't settle for that. You are too beautiful for that :)