LOL Complainer!
Seriously though, if you fill in the same hole the dog is going to think "wow great mum's filled my hole, so I can have hours of fun digging it back up again".
Perhaps a better bet would be to plant a new shrub/tree in the hole and tell him in no uncertain words to "leave" - you have to break his association of joy with this hole. Maybe also block it off for a while. Even then chances are he'll probably just start his digging elsewhere.
I have a garden full of dog holes, and they always return to the same ones. I know if I filled them in with clay, they wouldn't last an hour never mind a day.