To fully retire at 50 you would, in effect, have to "pre-fund" around 35-40 years of living expenses.
So, if you currently spend, say, €40k per annum in today's money terms that means you need a pot of between €1.4m and €1.6m (depending on how conservative you want to be). Obviously you would include the capitalised value of your accrued pension entitlements as savings for this purpose.
If you expect to have a reduced income for at least part of your (semi-)retirement then deduct that amount (allowing for taxes) from your projected expenditure and run the numbers again.
We can argue around the fringes about the "best" investment strategies to maintain (and hopefully grow) the real value of your savings but ultimately you need to save (or inherit) a serious amount of money to even consider retiring at 50.
By owning your own home, you effectively pre-fund your accommodation costs - but you can't eat a house! There are a large number of other future expenses to consider (food, healthcare, transport, etc.).
I hope that doesn't sound overly pessimistic but I think it's important to be realistic. The dream of eventually (semi-)retiring to a self-build house in the north west looks entirely achievable based on what you have told us. You just need to be patient.
I personally wouldn't build a holiday let with a view to converting it into your home on (semi-)retirement. To many expenses involved in maintaining two houses, too much risk of things going awry.
Incidentally, I think you have done great so far financially. If I was in your shoes, I would apply your savings in the following order of priority:-
1. Retain around six months of your typical expenses in a savings account as an emergency buffer;
2. Get your husband to start a pension - I would be surprised if his employer doesn't offer one - and max out his contributions;
3. Pay down your mortgage ahead of schedule - and refinance to a lower rate in the meantime!
Hopefully somebody else will chime in on the wisdom of making AVCs - public sector pensions are most definitely a specialised subject!
Hope that's of some benefit and don't give up on the dream house! You'll get there...