1. Welcome to UKBF! Check out our FAQ guide.

HSBS Secure Epayments failure

Discussion in 'E-Commerce Forum' started by HowardJB, Jan 8, 2008.

  1. HowardJB

    HowardJB UKBF Regular Free Member

    Posts: 15 Likes: 2
    On Saturday 05/01/08 at 9am the HSBC secure epayments system crashed and wasn't restored until the following day at 1 am. This is obviously a major incident for HSBC but an email I have received informs me that all the transactions we took on the Friday 4th have been lost and should be manually re-imput. Short of contactcting all the customers who placed orders on the Friday and asking for their card details again I am not sure how to do this. In addition our website was unable to accept orders on Saturday which is usually a busy day for us causing us to lose a days trading. Has anyone else on the forum been affected by this?


  2. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    Yep, me too. Was terrible customer service, I wasn't told about the problem and the only thing they've done is put a little note on the website after you log in saying 'sorry for our recent problems'. I also have to contact everyone manually - sounds great doesn't it, sorry we've lost all your card details, can we have them again? I have to do it though, as the orders have already been despatched. Pah.
    • Thanks Thanks x 1
  3. PayVector

    PayVector UKBF Regular Full Member

    Posts: 588 Likes: 135
    The HSBC platform is the old Clear Commerce software, a company that is no longer in existence. So your problem as a merchant of them is that when something goes wrong there is no one around with any great knowledge on how to fix it.

    Not a great place to be for HSBC. Unfortunately its you the Merchants who suffer.
  4. HowardJB

    HowardJB UKBF Regular Free Member

    Posts: 15 Likes: 2
    Hi Pelenna
    Further to my first posting I have been contacted by HSBC who have been able to re-instate all of my transactions for the missing day except a couple which had been referred to ‘Fraud Review’ but they say I will not be able to view any of them on their site. I am waiting to hear from them to see if these transactions (The biggest of the day of course) can be recovered as well.
    As is life wasn’t busy enough!
    • Thanks Thanks x 1
  5. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    Hi there - that's good news, I'm still waiting to hear back from them myself. Thanks for letting me know!
  6. Sue F

    Sue F UKBF Regular Free Member

    Posts: 405 Likes: 6
    Both myself and my father in law use HSBC CCP and my hubby (who works for his dad) noticed on Saturday that orders were being rejected.

    At 10am they told us that it would be fixed in three hours but orders were still being rejected on Sunday morning.

    I only know that Friday's transactions are missing because my father in law's business partner noticed on Monday by accident.

    Monday was spent ringing customers to ask for their card details again ... not the most fun I've ever had "Thank you for spending £1000 on furniture ... we've lost your payment". Inspires a lot of customer confidence I'm sure. I didn't manage to catch everyone until 8pm last night (Tuesday) hampered by conveniently getting the cold that's going round so my voice now sounds like an axe murderer making the explanation sound even less believable.

    I'm concerned that HowardJB's transactions have been reinstated and he can't see them on his CCP gateway as I don't fancy having to face customers to explain their payment has now been taken twice!

    Great time for it to happen ... January sale time ... it's bad enough for me but my father in law's company lost 70+ orders on Saturday and hubby had a similar number to phone on Monday to retake payments.

    The thing that really annoys me is the lack of communication. There's an email address on the gateway page but the email I sent was returned with a stock reply that didn't answer my question at all ... brill!
  7. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    Completely - I only had 3 orders disappear (for some reason most people were using Paypal that day) and it's easy to miss something like that. They didn't let anyone know - a simple email sent out to all customers explaining the situation wouldn't have been that hard would it?
  8. Dwebs-Ltd

    Dwebs-Ltd UKBF Regular Full Member

    Posts: 2,069 Likes: 263
    Well Protx had a major balls up last year, so nothings bullet proof so is it not wise to use two solutions i.e. hsbc epayments and something like protx then if one fails you can switch to the other.

    Our payment system is built in such a way that we can disable payment methods instantly and can plug any payment gateway into it :)

    I know shopping carts like cubecart, zencart offer multiple payment gateway support.

    It's worth while having multiple options if you would loose a lot of business if it wasn't working.
  9. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    Good point, but the thing is that we weren't told the system wasn't working - I do have Paypal also on my website, had I known I could have just put up a temporary notice saying 'please use Paypal' or whatever. Also, it's the payments that were made using HSBC when it was working that have disappeared - another gateway wouldn't have solved that!
  10. Dwebs-Ltd

    Dwebs-Ltd UKBF Regular Full Member

    Posts: 2,069 Likes: 263
    Indeed it wouldn't have solved the vanishing transactions but if your clients were getting bounces which i believe they were i.e. the systems stuffed you could then switch to the second gateway.

    Again nothing is bullet proof :(

    We use Verisign’s system and touch wood we haven't had a failure yet but its always possible. Currently we don't push enough credit card transactions to warrant a second gateway but if the need was there we could have one up and running very quickly :)
  11. boho

    boho UKBF Regular Free Member

    Posts: 1,411 Likes: 135
    Yep friend of mine had their busiest day of sales on Saturday and then bump go the orders as no one could put their card details through, they managed to rescue their regular customers but still lost orders from the new customers who were trying. Interesting that the issue is one of an old unsupported platform, I'll have to let them know that!
  12. tall_c

    tall_c UKBF Regular Free Member

    Posts: 1 Likes: 0
    This whole HSBC failure is a nightmare, and not at all helped by the frighteningly poor customer service from HSBC.

    I spoke to them a short while ago today, Wednesday 9th, and they were able to tell me the number of transactions that have successfully gone through on Friday, and the monetary total of these transactions. If you contact them and have your Client ID handy, they should be able to give you this info.

    One thing I was told, which I think it's best to mention, and I don't think anyone previously has mentioned in this thread:
    If you're transactions were put through as status 'pre-auth' then all these transactions are now lost. If you try and capture them now you won't be able to.

    Also, a question regarding a further issue I've just encountered: how can I refund a transaction that was successfully captured on Friday? Normally I would process it through our API which would give a full credit against the original transaction [that way we don't need the card details - just the transaction ID].
    I can't do it through the API [it just fails], and I can't do it manually through secure e-payments as you can't view any of the transactions!

    It's a nightmare! Without the original card details we can't give the man his money back. And of course [like everyone else, right?] we don't store CC details as we are PCI compliant.

    It will be very interesting to see what the result of the HSBC "Major Incident Review" are!!
  13. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    Is it just me or is it still not working one hundred percent? I've had a couple of customers say it crashes when they try to return to the website after making payment, and it just crashed on me while trying to ship orders on the virtual terminal... could be user error though I suppose, as it seems to working fine now most of the time :|
  14. HowardJB

    HowardJB UKBF Regular Free Member

    Posts: 15 Likes: 2
    I haven't been at work today but orders seem to have been coming in at a normal rate. Do you get many transactions that do not return the customer to your site, i.e. you get a payment but no confirmed order?

  15. Pelenna

    Pelenna UKBF Regular Free Member

    Posts: 152 Likes: 10
    I've only had 2 in the past 2 days that have done this, both rang me to make sure their orders had gone through and said the system crashed when they tried to return to my website - however there have been quite a few other orders that have gone through fine, so it's probably not the HSBC system. It's probably just I'm a bit over-cautious now after this whole debacle!
  16. HowardJB

    HowardJB UKBF Regular Free Member

    Posts: 15 Likes: 2
    I suppose I will have to try and sort it out when I go back in tomorrow, hopefully it will all be back to normal and all the money will be in my bank account!
  17. snorestore

    snorestore UKBF Regular Free Member

    Posts: 1 Likes: 1
    Just to let you know that I have had conversations with some very high up (so they tell me) people at HSBC over the last week regarding the repeated failures of EPayments.

    To quote one of them "Our service has been well below standard. I apologise".

    He then went on to treat me like a naughty child (he'd listened to a recording of one of my calls to the so-called help desk - in India of course - during which I lost it with them), at which point I decided enough was enough.

    So now, folks, I am trialling Protx. So far so good. It's easy to sign up - especially as all HSBC EPayments users have a merchant number which they just transfer across - and feedback from customers is excellent. It took 48 hours from web sign up to going live.

    We shall see how it goes. It will cost more than HSBC, marginally, but the freedom from hassle more than makes up for it.

    I still have World Pay as a stand-by just in case. I have no plans to retain HSBC.
    • Thanks Thanks x 1
  18. Sue F

    Sue F UKBF Regular Free Member

    Posts: 405 Likes: 6
    Thanks for posting this.

    I'm getting increasingly frustrated with the HSBC epayments system. Since the massive failure in early January our transactions have been littered with "No processor response" entries. We started off contacting the customers who had experienced problems but they had already moved on and bought else where ... and who can blame them ... I do the same thing myself.

    Please would you keep us updated on your Protx progress as I'm seriously thinking of moving on too.

    The highlight of the whole episode was a call from a market research company just over a week a go asking me to rate the HSBC card processing service. The lady asking the questions was clueless to the problems that we are having at the moment and couldn't write fast enough to keep up with my ranting ... bless her.
  19. Blagger

    Blagger UKBF Regular Free Member

    Posts: 860 Likes: 31
    So thats what all the emails from Protx were regarding HSBC settlements. Didn't know what they were on about!
  20. CLSLTD

    CLSLTD UKBF Regular Free Member

    Posts: 16 Likes: 1
    Its looking like legal action is required. I am still trying to get compensation (£3+) for the Jan incident and now theres been a second complete failure of ePayments. Went down 12.22 on Fri 28/3/08.
    We switched our WorldPay gateway on as soon as we detected the failure...
    My branch manager rang head of ePayments and had no luck with info. ePayments have decided to not repeat the black propaganda they tried to use last time - they said it was OUR websites that were the problem!!. They later told me "...you are the only customer that is complaining. You are very rude".
    My regional Merchant services manager was informed at 12.22 that a 'level 1' incident had occurred - but seemed to not actually know what this meant!
    I have been in lengthy discussion with HSBC to try and get them to understand that a massive failure of management... or fraud has been occurring. We have written scripts to constantly monitor ePayments and have kept data on the continued poor responses coming from their server(s) since the Jan incident.
    Anyone that is interested in a class action we would be happy to co-ordinate with our lawyers.
    • Thanks Thanks x 1