Section 18(2) Housing (Miscellaneous Provisions) Act 1992
(2) A person authorised by a housing authority for the purposes of this section may at all reasonable times enter and inspect a house to which regulations under this section apply.
There is no requirement, that I can see, to give notice. "at all reasonable times" suggests no notice is required.
Having said that, it is at the least bad manners to turn up unannounced. My local authority gave me 2 weeks notice, and was very accommodating when I wanted to change the proposed time.
I suggest that you write to them and ask if it is council policy to do unannounced inspections. This would make me think twice about using HAP. Which LA are you talking about if I can ask.