If he is a domestic contractor to the builder it is the builders problem to sort out a replacement
if he is nominated the contractor may have a claim for delay .
Its important to terminate his appointment formally for non performance and advise you may counter claim against him for losses...