I can't understand how you got your keys without paying the fees and stamp duty. If you were getting a mortgage, then the solicitor should not have released the loan cheque ( in exchange for the keys) without making sure he was in funds to pay the stamp duty. And very few solicitors close sales any more without getting their fees.
If you don't have a mortgage, then lucky you!
Stamp duty is payable up to roughly 30 days after actual date of completion and thereafter there are penalties and interest payable.
While solicitor may have been remiss and may not have told you this, now that you know you need to get on to him straight away to sort out payment. At least of the stamp duty.
mf