Really hate stories like this. You had plenty of time to decide to drop out before buyers sent over signed contracts to you. In the meantime, the buyer has probably paid for a bank valuation, arranged for a loan offer, paid for a survey and engaged a solicitor. They might well have made decisions on their current living arrangements.
Everyone ie entitled to pull out but if you have numerous reasons to pull out like you claim, you should probably never have started the process and you certainly shouldn't have allowed it to get to this point. You wasted the estate agents time and you cost someone else a lot of money.