Enterprise search and web scraping

[Kavunka]

Hello everybody! I would like to ask you how often you need to get data from competitors' sites (prices, news, promotions). Or, for example, you have 10-100 competitors and you want to know what they are selling and how much it costs.

 

[intheTRADE]

Never. I know the value of what we offer and charge accordingly. I am not interested in what any of my competitors are doing

 

[fisicx]

Not interested in what they do. So unlikely to need your scraping services.

 

[Kavunka]

Not interested in what they do. So unlikely to need your scraping services.

Disagree with you

Statistics of queries in Google for the month (USA only):

data mining 50000.0
scrape 50000.0
scraping 50000.0
web scraping 50000.0

 

[Ozzy]

I would like to ask you how often you need to get data from competitors' sites

Disagree with you

Of course it's your purgative to disagree with someone, but that doesn't change the fact that they choose not to want to use your services, that is also their purgative too. I, for example, agree with @fisicx and I too have no need for a service screen scraping other people's websites. The output of such an activity would be pointless to me.
I'd choose instead to do a more strategic market analysis because using more accurate data. I do not trust information on competitors websites - it is easy to fake and companies tend to massage their information on their websites. So paying someone to copy/scrape a competitors website I would see as a waste of money.

 

[NickGrogan]

Disagree with you

Statistics of queries in Google for the month (USA only):

data mining 50000.0
scrape 50000.0
scraping 50000.0
web scraping 50000.0

Posting rubbish data isn't going to convert people. Common sense says that scrape and scraping which have multiple meanings will get far more searches than the more specific web scraping and data mining.

Adding a decimal point just makes it look worse. What's 0.1 of a search.

By the way, Google agrees

https://trends.google.com/trends/explore?geo=US&q=data mining,web scraping,scraping,scrape

Scrape 76%
Scraping 59%
Data mining 24%
web scraping 8%

You might want to bear that in mind next time you're making up numbers.

The fact that 3 to 4 times as many people search for data mining as web scraping should also tell you something.

 

[fisicx]

Disagree with you

Eh? I just said didn’t need your services. How can you disagree with that.

I had a project last year where the client scraped data to populate a database. Cost him a lot of money. Within days it was out of date. The whole thing was pointless.

 

[Mark T Jones]

Disagree with you

Statistics of queries in Google for the month (USA only):

data mining 50000.0
scrape 50000.0
scraping 50000.0
web scraping 50000.0

Disturbingly round and consistent numbers there - always sets alarm bells ringing!

 

[gpietersz]

Of course it's your purgative to disagree with someone, but that doesn't change the fact that they choose not to want to use your services, that is also their purgative too.

I think you meant prerogative? A purgative is something that makes you empty your bowels.

 

[JEREMY HAWKE]

We dont have rivals or competitors

We are Jeremy Hawke Couriers

 

[antropy]

Hello everybody! I would like to ask you how often you need to get data from competitors' sites (prices, news, promotions). Or, for example, you have 10-100 competitors and you want to know what they are selling and how much it costs.

I think you learned your lesson that you don't come on to a forum and in your 2nd ever post ask a question that's clearly leading to a sales pitch

Paul.

 

[Kavunka]

Posting rubbish data isn't going to convert people.

I gave the correct data. This is the data that Google provides when you set up an advertising campaign in Google Ads. You cited trend data for search terms, not search queries. I am redirecting this phrase to you "Posting rubbish data isn't going to convert people."

 

[Kavunka]

Disturbingly round and consistent numbers there - always sets alarm bells ringing!

Google does not disclose the exact data. I just copied the data I received in Google Ads. You can check them yourself ads.google.com

 

[Kavunka]

I think you learned your lesson that you don't come on to a forum and in your 2nd ever post ask a question that's clearly leading to a sales pitch

A lot of conspiracy theories. The world is much simpler )))

 

[NickGrogan]

I gave the correct data. This is the data that Google provides when you set up an advertising campaign in Google Ads. You cited trend data for search terms, not search queries. I am redirecting this phrase to you "Posting rubbish data isn't going to convert people."

You gave the data that Google Ads gave you. It's clearly wrong, something you should have spotted.

The point of the trends data is to highlight the issue, which it does perfectly.

I checked ads.google.com as you suggested.

The results?

All the phrases get between 10k and 100k searches per month in the USA. This tells you nothing.

 

[fisicx]

Google does not disclose the exact data. I just copied the data I received in Google Ads. You can check them yourself ads.google.com

Ad data is not the same a search data.

 

[Ozzy]

I think you meant prerogative? A purgative is something that makes you empty your bowels.

Don't blame me, I think my autocorrect was trying to choose the most appropriate phase for the message. Technology is getting more and more intelligent all the time.

 

[Kavunka]

You gave the data that Google Ads gave you. It's clearly wrong, something you should have spotted.

Why are you trying to make me feel like I'm wrong? I provide only the facts that I got from Google. These facts are called "number of requests per month". You have provided trends. They do not reveal the number of requests, but can be compared to another request such as "scrape" and "create website" These two queries are equally popular. Reminds me of the US elections)))

All the phrases get between 10k and 100k searches per month in the USA. This tells you nothing.

Here Google tells me that at least I can count on 10k impressions of my ad for people who are looking "web scraping". If you download the report, then Google will provide you the figure 50k

 

[Fagin2021]

All getting a bit 'angels and pin-heads' isn't it?

What is a web-scraping service anyway? I thought web-scraping was re-publishing someone else's content. It's not immediately obvious how that would help anyone.

 

[SaveCo Online Ltd]

Just rename the thread title "How Not To Start A Thread As A New User".

Problem solved.

You're welcome.

 

[gpietersz]

What is a web-scraping service anyway? I thought web-scraping was re-publishing someone else's content. It's not immediately obvious how that would help anyone.

Not necessarily. Some scrapers are plaigarists. There are others that scrape the web to find them. I have even come across people who want to scrape content they have permission to reuse because there was no easy way to get an export of it.

Google and other search engines scrape the web to provide the data they need to search the web.

There are all sorts of reasons: reputation management, manufacturers watching their products on online retailer's sites, various kinds of business and academic research, automated auction bidding and somilar, and a lot more.

I have worked with a few of these (NOT the plaigarists).