You should get professional advice on anything non-standard like this.
The 50k you bought ex out for is irrelevant. I assume it was 50k plus taking on the mortgage? It's the market value at the time if its not an arms length transaction - you don't say if 'ex' means ex-wife or ex-girlfriend.
So it cost you 310k (135k in 2004 plus 175k in 2006), and as you lived in it for 2 years, 3/19ths of the gain is exempt from CGT. Adjust for all you cost of buying & selling.
One could argue it should be treated as 2 separate assets; it would mean that the '2nd half' of the house would not be exempt from CGT. The difference in tax is less than 1,500 so I can't see anyone getting too caught up in it.
As I said, get professional advice if you don't know.