Querying Inventory Systems - Caching, Performance, Failover, and Scalability

[jonathan.aotearoa]

Hi,


I have a couple of issues around inventory levels that I'd really appreciate some feedback on.


About 90% of our product catalogue is held in stock. The other 10% we order from suppliers/third-party manufacturers when a customer places and order. I want to display inventory levels for all our saleable items, regardless of whether they're held in stock or not. As a result I need to query our own, pre-existing, inventory system, as well as third-party inventory systems.


One strategy I'm contemplating with regard to own inventory system is an observer pattern type approach. In this scenario our inventory system will be modified to fire “inventory level change” events over some message-oriented middleware. The store-front application will then cache these values. I'm quite keen on this approach as it minimises network traffic and scales extremely well. I've looked at the documentation for IBM's WebSphere Commerce product and they provide a similar Web service for third party inventory systems to send notifications to. Unfortunately I've been unable to find any other information on this type of approach, i.e. whether or not it's been successfully implemented and/or widely adopted. Does anyone have experience of this type of approach?


I'm also looking to make heavy use of caching with regard to third party inventory systems as I want to minimise the number of calls to remote systems, primarily for performance and scalability reasons. One of my main concerns is what kind of failover strategy to implement in the event a third party system going offline or failing to respond in a timely manner. I'm assuming the best approach is to simply display no inventory details and let customers order any quantity. If the order cannot be fulfilled due to insufficient stock levels it'll be handled further down the order processing chain when an attempt is made to allocation the stock. What's the general consensus on this?


Finally, is anyone out there using EhCache for caching inventory levels? If so what kind of cache do you find most effective; Blocking or Self Populating?


Kind regards,


Jonathan

[edmondscommerce]

You make this sound so complicated!

It's really not complicated at all...

Yes you store inventory data in your webstore (often as simple as adding another column in your products table).

Yes you make sure your inventory is kept up to date (an SQL query, web service request or whatever)

Yes you regularly grab the latest inventory info from your suppliers and update your local inventory system and then push updates to the webstore via the above method.

For this kind of thing I always prefer bulk operations over lots of small operations.

[jonathan.aotearoa]

Hi,

Thanks for your reply.

Following your approach, how do you handle failures in third party inventory systems? I've been thinking about implementing some kind of heartbeat message. If the system is deemed to have failed I simply fall back to a default behaviour, such as displaying "available" and allowing any quantity to be added to the cart. In your system the webstore will always see the local inventory system value, which may be significantly out off date. Do you have some mechanism for purging stale data?

Regarding bulk operations, do you bother with different time-to-live values for individual products or do you have a set refresh rate for all non-stocked items?

Thanks,

Jonathan

[edmondscommerce]

The way I have handled this, if a suppliers system is down then you simply continue working with the existing data - what other choice do you have?

If you wanted to get more advanced though or were exceptionally concerned about the accuracy of the data then yes falling back to a more generic message would work nicely.

Regarding the stale data, my approach is to rebuild all data. You can do that in the webstore or in your local inventory info which then pushes to the webstore. It's really up to you.

Systems that I have set up for this kind of requirement rebuild the entire system on a four hourly basis.

If its coded well the whole operation can be done in about 5 minutes or less (on lets say 100k products)

[jonathan.aotearoa]

On reflection, sticking with the most recent data in cases where a third-party system fails is probably the bests approach. One would expect most failures to be relatively short lived, in which case there's no value in disrupting the shopping experience unnecessarily.

Regardless, I think I'll implement a pluggable strategy that lets the business change this according to their needs.

Thanks again,

Jonathan

[edmondscommerce]

On reflection, sticking with the most recent data in cases where a third-party system fails is probably the bests approach. One would expect most failures to be relatively short lived, in which case there's no value in disrupting the shopping experience unnecessarily

agreed

happy to help :-)