Hello Ocd queen, anybody who doesn't have this illness can not believe how we think, they believe our thoughts are unreasonable and can not accept what we have to do, so how can understand why we have to check the door lock not once, but 3 or 4 times, they don't know the pressure OCD puts on us.
I also have someone close to me who also has this OCD asbestos fear, fortunately, he had the shed removed by qualified tradesmen after much worry.
If the asbestos is wet it doesn't prove to be a problem, but once it's cut or broken then it is a real concern, however, your only option is to live somewhere else and for your partner to take all of his clothes off before he sees you, place them in a plastic bag and then put them through the wash without anything else in the washing.
Ask him to read 'Asbestos OCD' and to understand that it's not only you who has this fear but many others.
Geoff.