I can't give any specific advice on whether the tax is owed or not. The only advice I can give is to beware the man in the pub talk! Don't ignore the correspondence from the Inland Revenue. Sort it out (with the help of a good tax advisor), put it behind you and enjoy your retirement. Some of the other posters have given good advice on appealing assessments etc.
A woman I know fobbed off calls from the (Irish) Revenue for months - both telephone calls and personal calls. She lived in fear of the phone ringing which was far worse than the final tax bill. As it turned out, when she eventually dealt with it, the liability wasn't as much as she thought and she was sorry she didn't sort it out sooner.