Only you can know if you have got a gem or not. Why did you buy this particular property? Did you just jump on the buy-to-let bandwagon or did you do your research and chose the property carefully, because of it's location, yield, potential, whatever?
Generally my advice would be - if you have made a bad investment decision, cut your losses. If the decision was good but the timing was bad and you can afford to ride out the storm, keep it.
You are lucky that your mortgage is on interest only for a long time, unlike a lot of posters here, so you are not in as much trouble as if you suddenly had to come up with capital repayments as well as interest.