Hiviz
Are you really getting hassled?
If they did a VAT audit and gave you the all clear, then you should have peace of mind. They would have approved, by default, the way you invoice your Northern Irish customers.
If someone is in a refund situation, they are quite right to verify it from time to time. There have been a lot of VAT reclaim frauds recently. But if you are not doing anything wrong, there is really no hassle.
By and large, the Revenue have improved their game in recent years and they treat their compliant customers very well.
Brendan