Hello Avondale, and welcome to the site.
After being in a 5 year relationship you would imagine that you knew everything about each other, but as soon as you move in together, then the pendulum changes, because now you both learn to know how you and your b/friend actually live and what each of you does that you didn't know of before.
This can increase your anxiety with the worry of whether you'll be able to handle living with him, because love and living together are two different issues here.
In other words, you can love someone when you're dating or have been going with, but as soon as you're with them 24/7 365 days a year, the situation can change immensely.
Write down what you believe makes you anxious living with him and ask him to do the same, then you can discuss whether or not it's feasible this relationship is achievable and any trouble happens then get a third person to adjudicate a meeting.
Hope to hear back from you.