Saying that if you haven't signed a contract with an EA or signed a letter then you will not be liable for fees is extremely misleading.
Most residential agencies issue a letter setting out their terms at the start of the marketing campaign. Typically it will lay out what the advertising costs, asking price and terms of their agency are. You should hold onto this letter and be sure that you are clear what the term "sole agency" refers to.
If you have agreed to a sole agency selling with your estate agent, then it will not matter if you sell the property off market to your neighbour or not, you still have to pay your estate agents fees regardless. The term sole agency refers to the fact that the agent has sole selling rights on your property. Many people are not aware of this fact and get caught out and still have to pay agency fees or a proportion of these.
You are always liable for marketing of your property when you have entered into an agreement (even verbally) with an estate agent. They have marketed the property on your behalf and incurred costs such as signboards, brochures, advertising on your behalf.
Personally speaking if you are sure that your neighbours offer still stands I would check the terms that you have agreed to - is it a sole agency property or not - then consider what the implications would be of selling to your neighbour or AN Other at the same price. You may have to pay the fees anyway so it could be worth your while bidding each party against one another to get the optimum price.