Watching Competitors Online prices?

[Shopcetera]

Anyone already doing this that can recommend a good company?

We're thinking of giving it a go and want to know the best company to choose.

 

[mikewev]

This is totally biased but I run such a service which tracks competitor prices, provides analysis and reports and can integrate with your e-commerce system.

I'd be happy to show you a demo.

I can't post the link but you could search for Insite Track on the usual search engines.

Mike

 

[SamStones]

I had a demo from competitorpricewatch.com a few weeks ago . Wasn't any good for me, but hope it helps as the people there were really nice

 

[Shopcetera]

I had a demo from competitorpricewatch.com a few weeks ago . Wasn't any good for me, but hope it helps as the people there were really nice

Thanks Sam, can you explain why it wasnt any good for you? Are you just selling goods online like us, or maybe youre in a slightly different field.

Mike, thanks I have done some research and am aware of whom is out there, just wondered if anyone had any dealings with a company offering this service as there are quite a few and could recommend one like Sam did. Although now I've looked closer, I can see oly a couple offer what we want which is for it to be set up for us as we have 20,000 lines. Most dont offer this.

 

[movietub]

Why not automate the whole process like we did?

OutWit hub for firefox. You can design price scrape profiles for each of your competitors by setting up a profile, and set them to run early hours of every morning. It kicks out a csv file. Line those prices up against your own on an excel template, red highlights anyone cheaper, you review the price.

You can even dump the scraped data to an online db and project competitors prices live into your pages, or in theory auto update your own prices... I'd be careful with that last method though, if you did it to me I would be more than capable of feeding you dummy data

Probably best to just scrape and go through prices manually just in case they get wind and sting you with silly data. Or safer still, if they all use google base, scrape from there as the data is guaranteed to be what they display to the general public, and not just your IP.

 

[Shopcetera]

Oddly enough we have been down this route and it has its pros and cons.

Pros

Cheap to set up as long as your technically minded.
Cheap to maintain.

Cons.

Still time consuming to go through.
Not sophisticated reporting.

We are really looking for something far more sophisticated than this and most of the online services offer you reports to analyse your data in from every different angle extremely quickly, this is what interests us most.

An excel spreadsheet is ok, but its still hard going if you have more than a couple hundred products.

The one were favouring now gives you a daily breakdown by email and even provides suggested product price movements based on the market!

You cant do that from an excel spreadsheet.

You pays your money you makes your choice!

 

[patientlady]

Why not automate the whole process like we did?

OutWit hub for firefox. You can design price scrape profiles for each of your competitors by setting up a profile, and set them to run early hours of every morning. It kicks out a csv file. Line those prices up against your own on an excel template, red highlights anyone cheaper, you review the price.

You can even dump the scraped data to an online db and project competitors prices live into your pages, or in theory auto update your own prices... I'd be careful with that last method though, if you did it to me I would be more than capable of feeding you dummy data

Probably best to just scrape and go through prices manually just in case they get wind and sting you with silly data. Or safer still, if they all use google base, scrape from there as the data is guaranteed to be what they display to the general public, and not just your IP.

Hi movietub
Do I need to be technically minded to set this up? Would like to have a go at this but I am a a bit blonde! Just googgled this and there seems to be software available but I got the impression you have done this yourself. Please divulge some more please

 

[movietub]

Oddly enough we have been down this route and it has its pros and cons.

Pros

Cheap to set up as long as your technically minded.
Cheap to maintain.

Cons.

Still time consuming to go through.
Not sophisticated reporting.

We are really looking for something far more sophisticated than this and most of the online services offer you reports to analyse your data in from every different angle extremely quickly, this is what interests us most.

An excel spreadsheet is ok, but its still hard going if you have more than a couple hundred products.

The one were favouring now gives you a daily breakdown by email and even provides suggested product price movements based on the market!

You cant do that from an excel spreadsheet.

You pays your money you makes your choice!

Well providing you have a very quick system and can setup excel macros you could achieve the same. It is possible to bung all the competitor csv's into a single spreadsheet which will calculate any figure you can think of.

However, if a decent budget exists it would be better to go down the one off cost route of getting a db/dev guy in to build a mysql db for outwit post the data too directly, this can then provide a data feed for your own products, compare prices in real time, whatever you want really. This all comes down to very basic maths, none of it is complex other than designing the scrape profiles.

Possibly you used outwit hub lite last time you tried? The full version (only cheap) does a lot more. There is no scraping operation it couldn't achieve, so long as it's setup by someone that understands the construction of the pages you are scraping.

 

[Shopcetera]

Agreed, you would need a developer to do anything remotely complex as at the very least you'd neg reg exps.

It crawls text fine, but with the multitude of different layouts shop systems use the pricing grab is quite open to incorrect results.

We have competitors who try to hide their prices in images, javascript and using various different AScii characters.

For these reasons it just doesnt work very well for us, and I would imagine most others too.

Some of the price monitoring companies we've spoken to so far have already created solutions to get around this, so I guess you get what you pay for as usual!

 

[SamStones]

Hi ,

My problem with CPW was that it was too labor intensive - we'd have to add our competitors product for each new product which would be a pain, we would also have to manually review prices all the time.

My other problem was with the general concept - the race to the bottom on prices. I'm not sure this is a good thing... we used to manually price compare daily and adjust prices accordingly, we had huge turnover but little profit... "the busy fool"

 

[movietub]

Hi movietub
Do I need to be technically minded to set this up? Would like to have a go at this but I am a a bit blonde! Just googgled this and there seems to be software available but I got the impression you have done this yourself. Please divulge some more please

Outwit hub is a bolt on for firefox, so as ff opens the page, outwit hub 'reads' it as if it were a human, notes the required data, and then asks ff to generate the next page.

I have played around with a few similar programmes, outwit seemed to work best and be the easiest to setup - in truth I can't remember exactly why I drew that conclusion - but 'm more than happy with it.

Regards difficulty, it depends how familiar you are with web page construction. If very, it's quite easy. If not at all, you would effectively need to learn at least html before setting up a scrape profile (but really, anyone in ecommerce should learn at least that much anyway).

The programme itself is intuitive enough, but like all niche software, it's not as intuitive as typical mainstream product.

It's well worth the effort though. Online you compete against everyone, and visitors can swap sites within seconds. If price matters, then it's invaluable to know where you stand in the market - and there is way to much data to do it manually.

 

[movietub]

It crawls text fine, but with the multitude of different layouts shop systems use the pricing grab is quite open to incorrect results.

We have competitors who try to hide their prices in images, javascript and using various different AScii characters.

For these reasons it just doesnt work very well for us, and I would imagine most others too.

Some of the price monitoring companies we've spoken to so far have already created solutions to get around this, so I guess you get what you pay for as usual!

The layout issue is a non issue, as the data you want will virtually always be loaded into the page in the same way (per site that is, obviously you need a profile for each site). The only time this is likely to change is if they become aware of what you are doing. Using dynamic IPs is possible, but the truth is hardly anyone understands the concept of scraping, let alone detecting it happening.

Regards hiding their prices... Put simply, if you can read it on their site, there is a way of scraping it.

However the easiest option is normally to find a third party directory that they feed their prices too. For example google base, or any other shopping channel. Scrape from there.

 

[Shopcetera]

Hi ,

My problem with CPW was that it was too labor intensive - we'd have to add our competitors product for each new product which would be a pain, we would also have to manually review prices all the time.

My other problem was with the general concept - the race to the bottom on prices. I'm not sure this is a good thing... we used to manually price compare daily and adjust prices accordingly, we had huge turnover but little profit... "the busy fool"

Oh, Im guessing you were going for the self managed service they offer then? We're looking at the fully managed service as we have 1,000 products to list at least and couldn't possibly manually match them all ourselves. The quote they gave us was suprisingly small actually for the amount of work involved it seems.

We were just hoping they maybe someone on here whos already using one of the services out there that could recommend one of them thats all!

Although, Ill guess most probably dont want you to know theyre using their services of course!

I can see you have to manually review prices, but their system seems to give you a good indication of which ones need looking at, as opposed to just showing you everything. It seemed quite quick to me.

Thats got to be better than have the system update your prices for you, I wouldnt be comfortable with that!

The race to the bottom on prices I can understand, but my understanding is theres far more to it than that and surely its better to know what your competitors are doing.

 

[Shopcetera]

The layout issue is a non issue, as the data you want will virtually always be loaded into the page in the same way (per site that is, obviously you need a profile for each site). The only time this is likely to change is if they become aware of what you are doing. Using dynamic IPs is possible, but the truth is hardly anyone understands the concept of scraping, let alone detecting it happening.

Regards hiding their prices... Put simply, if you can read it on their site, there is a way of scraping it.

However the easiest option is normally to find a third party directory that they feed their prices too. For example google base, or any other shopping channel. Scrape from there.

I'd love to know how you scrape images for prices? If Google can't read images for SEO purposes (as we always are told).

 

[movietub]

My other problem was with the general concept - the race to the bottom on prices. I'm not sure this is a good thing... we used to manually price compare daily and adjust prices accordingly, we had huge turnover but little profit... "the busy fool"

Many online start-ups do indeed go down this route, many don't even realise they are losing money for a long time!

I personally grew the business very rapidly by competing on price, and making no end of day profit, in order to get a high t/o. I then used this to leverage discounts across the board and then set all price to be 'competetive' with our closet rivals, but never outright cheaper - except for products we can buy cheaper than anyone else for whatever reason.

We actually scrape price to put our own prices up! What tends to happen is the suppliers set a min online sale price, which due to various pressures we all adhere too. But then, over time some peoples price would creep up as supply price increased. We discovered that we were frequently selling for 20% below our nearest rival. The price scraping was introduced to make sure we were never more than 5% below the next price, and normally the same.

We compete now by offering prices 'as cheap' as anyone else, but generally much better service and we have more items in stock. People pay for an in stock item, as they get it sooner!

I tend to find business wisdom is rarely black and white, and often it's more effective to go down an obscure and seemingly reckless route, so long as you can make it work in the end!

 

[movietub]

I'd love to know how you scrape images for prices? If Google can't read images for SEO purposes (as we always are told).

Put simply I don't, but only because I have never needed to scrape a site that uses images for prices. Thats a very unusual setup indeed, as the price is almost always passed on somewhere.

But it's possible, although not neat, as 'robot' surfers exist that are designed to do just that. They are used to spam recaptcha protected sites/forms, by the security services, and chiefly by online poker players to read the card and auto play poker to make continuos small yield winnings.

Scary huh.

 

[Shopcetera]

Yes, youre right about putting prices up, thats exactly what we're doing!

We noticed our competitors prices were all starting to creep up from early this year and this service will help us ensure we are not too cheap, which we also noticed we were!

 

[movietub]

Yes, youre right about putting prices up, thats exactly what we're doing!

We noticed our competitors prices were all starting to creep up from early this year and this service will help us ensure we are not too cheap, which we also noticed we were!

Noticing things like that, makes you a good e-retailer. So many either don't notice, or have no system in place to do anything about it at all.

 

[mikewev]

OK, so here's a typical senario.

You don't systematically monitor compeitor prices and often find yourself outmanouvered by your competitors - last week a certain product was selling well and this week sales have dried up - you go online to check a couple of competitor prices and quickly realise why. Trying to 'manually track prices is impossible unless you only have a few products and a couple of competitors - its just too time consuming.

So you go online and find a solution -that automates the process of collecting data and provides some nice analysis for you. You now have visibility which is great but ...
The amount of data is overwhelming - you are uncompetitive on 100,s of products and prices are changing every day. The problem now is, its taking you ages to review all the price changes and then ages to get the updated prices into your ecommerce system.

If your going to automate the collection of price data you will probably also need to automate the analysis and the price change process.

Movietub knows what he's talking about - he sounds like a smart guy -unfortunately not everyone is that tech savvy.

Our price tracking service can automate as much or as little of your price management process as you are comfortable with. We do all the setup and product matching and provide a complete price tracking and automation solution.

Its got to be worth a demo ?

Mike W - Insite Track Ltd

 

[Shopcetera]

Mike,
Informative post, but we found your services way too expensive when compared to others.

I think you quoted us £495 a month for 1,000 products and £500 set up for each domain!

We were quoted half that or in some cases much less for pretty much the same functionality, more actually in other cases as some others had some really innovative features.

Shame no one else on here feels they can give a review of who theyre using, but I think we know who well be going with now.

 

[mikewev]

Shopcetera,

Your numbers are not quite correct, its £495 / month for upto 10,000 prices not 1,000.

But its good feedback- thanks for that.

Hope things go well for you, sounds like you've thoroughly researched things.

 

[Shopcetera]

Ah may be we just gave you the 1,000 products we wanted to track number then as you definetly quoted us £495 a month.

Sorry, just saw you put prices not products. Can I assume you mean product x competitor = price?

In other words, we wanted to track 1,000 products against 5 competitors = 5,000 prices?

So, that would make it £495 a month plus £2500 set up? On that basis, there are others doing the same thing a lot cheaper Im afraid.

 

[mikewev]

Really appreciate the feedback, yes 10,000 prices would be the total number of prices captured each day.

We are working at getting the setup costs down but it can be very time consuming mapping 10,000 products !

You obviously know what you are doing - so I'm sure things will go well for you on this project.

Good luck.

 

[movietub]

Ah may be we just gave you the 1,000 products we wanted to track number then as you definetly quoted us £495 a month.

Sorry, just saw you put prices not products. Can I assume you mean product x competitor = price?

In other words, we wanted to track 1,000 products against 5 competitors = 5,000 prices?

So, that would make it £495 a month plus £2500 set up? On that basis, there are others doing the same thing a lot cheaper Im afraid.

To be honest, that is probably quite realistic for a well thought out process. The more effective the process (the less manual stuff there is left to do for you each day) then my definition the more complex it is. Setting up reliable scrapes for each of your competitors is not quick, especially as they will have to let you review the data, then alter the scrape when you point out some of it isn't right...

And once you have the scrape profiles, they then have to understand your pricing and how to relate the scraped data to yours to make a useful automated report.

Put simply, I'm sure you can find cheaper (and I know nothing of this guys services btw) but value is more important than price. If they understand what your business needs to grow and design a service to suite, it gives you an advantage over all of your competitors which is, for most businesses, of unlimited value.

It depends on the scale of your current operation of course. But the prices quoted are less than say 10% of your net profit on sales, you can probably afford to try it - because it won't take long to make an excellent roi. I say this as it's usually quite realistic to get 10% extra margin simply by being price perfect 99% of the time. Not to mention you can flip tactics occasionally and undercut your competitors by a high margin for short periods. Do this for one day, on 50 products a month for against each competitor and you can eat in to their market share without them having a clue. Your prices are back to normal by the time a manual checker investigates.

 

[Shopcetera]

To be honest, that is probably quite realistic for a well thought out process. The more effective the process (the less manual stuff there is left to do for you each day) then my definition the more complex it is. Setting up reliable scrapes for each of your competitors is not quick, especially as they will have to let you review the data, then alter the scrape when you point out some of it isn't right...

And once you have the scrape profiles, they then have to understand your pricing and how to relate the scraped data to yours to make a useful automated report.

Put simply, I'm sure you can find cheaper (and I know nothing of this guys services btw) but value is more important than price. If they understand what your business needs to grow and design a service to suite, it gives you an advantage over all of your competitors which is, for most businesses, of unlimited value.

It depends on the scale of your current operation of course. But the prices quoted are less than say 10% of your net profit on sales, you can probably afford to try it - because it won't take long to make an excellent roi. I say this as it's usually quite realistic to get 10% extra margin simply by being price perfect 99% of the time. Not to mention you can flip tactics occasionally and undercut your competitors by a high margin for short periods. Do this for one day, on 50 products a month for against each competitor and you can eat in to their market share without them having a clue. Your prices are back to normal by the time a manual checker investigates.

I'm not saying there's not a lot involved, I was just saying we have found others cheaper.

I agree there is a lot involved as most companies we looked at although they seemed to have a solution were one man band developers who had no clue about retail and you still had to enter all the details yourself.

We eventually found two companies to compare who did everything for you, clearly had a proper business with a team of people who werent going to suddenly decide they had enough and pull the plug like a one man band can do, and offered a sophisticated solution. Yet, one was still more than double the price of ther other.

The pring range in this field is huge. Obviosuly the one man bands can offer their solutions cheaper, and most of them are quite a poor substitute for the larger sophisticated solutions the companies offer, but even still it ranged from £30 a month to £495 a month!!

 

[Shopcetera]

Weve signed up now, first impressions seem excellent!

 

[g4joe]

Hello Shopcetera,

Who did you eventually decide to go with, sounds like they are delivering ! Keep us all updated on your findings.

 

[Shopcetera]

They're just adding everything at the mo, but the demo was very impressive, at least as good as if not better than others.

They seem really on the ball as well, and I liked the fact there was a team of people rather than one person you spoke to, clearly a well established business rather than a lone developer I mentioned in my previous posts!

I'll let you know how I get on!

 

[ecashking]

Hi Shopcetera,

Interesting thread, who did you decide to go with then?

 

[movietub]

Glad to hear the thread ended with someone taking action for once! I get the impression most 'advice' threads on here end up discarded!!!

I hope it works out well. I'm fairly confident that most etailers could benefit from such techniques, so long as the data they then posess is put to well thought out use.

Best of luck!

 

[Shopcetera]

Hi Shopcetera,

Interesting thread, who did you decide to go with then?

Sorry yeah, Competitor Price Watch.

 

[competitor_monitor]

Sorry yeah, Competitor Price Watch.

If only I'd seen this thread sooner and I could have tried to work my marketing magic

Glad you got sorted out, just for your information we (Competitor Monitor) set everything up for you, and currently have slashed our sign up fee. We also offer our accounts at a fraction of the cost of our competitors.

Did you take a look at us when researching? Any feedback would be great.

Also, we have a full team here so no one man band.

James, Head of Sales and Marketing at Competitor Monitor

 

[Shopcetera]

Im sure I spke to someone there yes.

When you say you set everything up, do you also mean you match my products against our competitors for us as well?

So 5,000 matches for £199?

 

[competitor_monitor]

Hi there.

We have completely waived our sign up fee at the moment so there is no set up fees (just need our site updating).

The prices on our site are not for full product matching, but you are right in that it is 5000 products for £75/month, no set up fees.

Anything else you want to know feel free to ask

 

[Shopcetera]

Hi there.

We have completely waived our sign up fee at the moment so there is no set up fees (just need our site updating).

The prices on our site are not for full product matching, but you are right in that it is 5000 products for £75/month, no set up fees.

Anything else you want to know feel free to ask

Sorry, Im completely confused by your answer!

Can you let me know what we get for £75 a month then?

What does set up cover? Does it include matching all my products and all my competitors products, i.e. 25,000 matches if I had 5 competitors?

Also, how do you do your matching, is it by spider or human as I dont have SKU's.

Thanks.