I assume that this is a standard voluntary organisation run by volunteers etc. and isnt a registered company.
who is responsible if something does show up -
You would need to be more specific about this.
If its a case that the outgoing committee made some poor financial decisions, did not have the skills to keep adequate records etc, maybe even lost some money through not following up on things like outstanding membership owed etc., then there is little you can do.
The current committee is usually responsible for getting the accounts audited. If anything irregular does turn up, they can ask the previous committee for an explanation, but there may not be one other than the reasons outlined above.
My advice would be to get the past couple of years done as best as records will allow, put in some new procedures to ensure that correct records are kept from this point forward and do things correctly from now on.
Lack of accounting skills is a common deficit on many voluntary committees. If there is nobody in your organisation who has the skills to prepare accounts, you might consider hiring an accountant, if funds allow.