My understanding (non-lawyer, so not formal advice) is:
You have a valid and enforceable contract.
By sending the PO they were offering to buy the goods described at the price stated: absent other terms, a PO sent to a supplier is by definition a duly authorised order for purchase.
An offer remains open until it is (i) withdrawn by the offeror or (ii) accepted by the offeree,
whichever comes first.
Once the terms of an offer have been accepted in their entirety (ie, you don't say, 'yes but this also'), a valid contract is formed and both are bound: neither your customer nor you can withdraw except by mutual consent, or by a procedure agreed to be integral to the contract (eg, your standard Ts and Cs, or theirs, if either of you have made the other aware of these).
Acceptance of an offer can be oral, written (did you email back and say, Thanks, we'll start work?),
or by conduct. In your case, you went ahead and made the goods before they withdrew their offer: that is acceptance of their offer, and binds them into a contract.
NB, even if the offer had been withdrawn at their end, and you had not been adequately notified (eg, their letter was in the post), you would still be able to hold them to it. I also infer that they were to collect - so they can't argue that you're in breach by not delivering (anyway, that would not go to the essence of the contract imo, and
Sale of Goods Act is in your favour).
As has been said, you will need to look for other clauses that may give them a right of cancellation (that is cancellation of the now-existent contract, not a withdrawal of their pre-contractual offer, which has been extinguished by the contract).
But on this info, looks like you have a very strong case. You can read more
here.
PS you can probably start charging them storage at some point soon. That may encourage them to collect.