Yes. You're supposed to register in every country of the EU you export to, but I would say the vast majority of e-commerce sellers don't bother, and just let customs deal with it. However, there is a chance they won't let your orders through.
You can use the IOSS system to register in all of the EU countries at once.
I don't believe it is correct to say you're supposed to register for VAT in every country you export to.
My understanding is that you are required to register for VAT in every country you
hold stock in. After months of agonising and procrastinating about how to do business in the EU post-Brexit, I'm reasonably confident my interpretation is correct.
To the OP's issue, he/she has a number of options if they continue to sell into the EU:
a) If items are below €150 in value then IOSS registration may be suitable. You collect the VAT relative to the country you're selling to and max returns to the IOSS who then divvy it up to each member state where you sold items/collect VAT
b) If Items are above €150 then IOSS won't work and you have three options that I can think of:
1) Sell at ex-VAT price but make your customers aware they will liable for taxes/duties when the items enter their country. Easy for you but it puts customers off.
2) Send the items DDP (Delivery Duty Paid) via DPD or DHL. This is expensive for you but the customer will not be bothered by customs duties or taxes when the goods arrive so good for them.
3) Register for VAT in one EU country (Germany is usually recommended), send items from the UK to there paying VAT on them as they enter the country and then sell them as normal but ship them from your location in Germany (we use Amazon Germany to hold stock, for example). As you sell them you are collecting VAT for the various EU countries they are sold to and this is paid via an OSS return (NB OSS is different to IOSS) done quarterly I believe.
In the case in point I think you leave this one as it is. You've just earned 20% more on your item and if the customer isn't that bothered then you've been lucky on this occasion. But, as I understand it, you have no reporting or VAT registration obligations in the EU. Should you wish to have a more consistent business with the EU then you probably need to work out a better solution. HTH.