Hi and welcome to these forums.
Reading your post, I get the impression that anxiety is your major hurdle. Loneliness accentuates feelings of insecurity and vulnerability.This is often enough to trigger anxiety.
It seems also that your bf doesn't understand what anxiety is all about. Most people are unfamiliar with the condition. They often think it is due to some flaw of personality, do not take it seriously and end up doing/saying unhelpful things. A good place to start would be to inform him. Where relationships are concerned, honesty is the way to go. Written information is available. If you scroll down to the bottom of this page, you can order info resources (Get support). These booklets are free.
Meanwhile, keeping yourself occupied and distracted during his absences is the way to go. Nights are usually more difficult. Acquiring a pet has helped many people but it is not for everybody. Perhaps a friend could be talked into sometimes staying overnight ?
Are you on a medical plan for your anxiety ? If you are, taking your partner along to one of your appointments would be helpful. If not, talking to a GP yourself would be wise. It would also answer your own questions. There's no need to struggle alone. With the right help and support, anxiety can be managed.
I also suggest you navigate the anxiety section of the forums. You will find that many are living with the condition. Knowing how others in similar situations cope is always helpful. Many caring, supportive people interact via the forums...not a bad way to spend lonely times.