Seems like a coincidence, theres no corolation between PTSB branches and brokers, in fact if the branch rang you directly then it is likely they are competing for the business against the broker. The broker should be dealing directly with head office and not a branch.. ( unless its a very small brokers, but to my knowledge all brokers deal with head office and not branches)
not sure why its causing you concern, if you dont want to go via a broker your not obliged to.
Put it in writing to the broker that you do not want to deal with them, and go direct to either PTSB or AIB, which ever is offering you the best deal.