Partners needed for ScrapePro web scraper project

[scrapepro]

My web scraping, data extraction, web automation software project named ScrapePro is in beta stage (very close to release). Free trial is available. I need partners to manage/market/fund this project. If you are interested, please contact me for more information. Thank you.

 

[Optegris]

Hardly an ethical project is it? I would imagine the purpose of the tool is to scrape content that is otherwise protected by copyright or to harvest emails...

 

[fisicx]

Eh?

You me to invest in a project that scrapes MY content so that you can use MY data without permission.

Hope it fails.

 

[scrapepro]

DEVELOPING a web scraper application IS LEGAL. Using a web scraper MAY be illegal. But it can be used for many legal actions.

 

[Optegris]

So ethics in business don't bother you then?

But it can be used for many legal actions

Such as?

 

[scrapepro]

It won't fail. It's the most feature rich web scraper platform that helps the user to scrape any web site easily. It also has a server part to convert / publish / search data.

Hope it fails.

 

[DavidGiant]

I have never wished for any person to fail at a web venture until today. Scrapers are the bane of many of my clients existence.

The fact is though its pointless, you are just wasting my clients time who has to chase down the host and file a DMCA request to get the content taken down.

I mean... of all the ventures, why choose one that actively promotes illegal activities? Then why go to a business forum and promote it amongst business owners that your application is most likely to affect?

 

[WebsiteDesigner]

Where is the trail?

 

[awebapart.com]

But it can be used for many legal actions.

Such as?

Heh heh.

Search engines, like the blessed Google scrape, so some people might say "it must be OK/legal what Google are doing since Google say they 'do no evil'".

Personally I wouldn't say that what search engines do is entirely legal. Let's face it, it is difficult (impossible) to create a search engine if the ethical side of you is worried about copyright infringement. There have been some cases, around the world, of Google's search engine being taken to court for copyright infringement.

To be fair, web scraping does have ethical uses too, like screen readers making the web more accessible, web site analysis tools (e.g. w3Validator), and for hosting companies, website monitoring uptime tools (tools which not only check websites are up, but check actual text within website pages too, to ensure that the website is not only up but is working OK too)

 

[Astaroth]

Such as?

Not to defend the OP but....

Some of the aggregators like confused.com use screenscraping on their insurer clients who don't have web services set up to return quote prices. Isn't done as much these days as most the softwarehouses now have web service components but it still goes on.

Was amuzing when discussing inclusion on an aggregators site with a former client and they said they would tell the aggregator how to bypass their HIPS such as a captcha image and the aggregator laughed and said they'd worked it out themselves years ago.

 

[Optegris]

Search engines are a moot point as it's simple enough to block them crawling content if you want. As for insurance comparison sites, there is more chance of me growing breasts than the comparison site not getting permission to crawl the content in the first place

The OPs argument is that it's not illegal to develop or sell the software but it might be to use it. Nice get out...

It's also not illegal to sell marijuana seeds as long as you don't grow them into plants. Thanks for that but what the hell else are you going to do with them???

Same goes for software developers like this, they may act all innocent and blue eyed by developing the software but they know for a fact this the tool will be used by someone to breach copyright or privacy guidelines which makes the promotion and development of such tools immoral and unethical in my view.

 

[edmondscommerce]

Yep content scraping is legal and is a major part of how the web works..

If you don't want your content to be scraped then you can do various things to prevent it or at least make it difficult, but ultimately if you post content up for public consumption then that's exactly what you are doing.

There are a great many totally ethical and valid reasons to use a scraping system.

We do a lot of scraper projects to get product content from supplier web sites where the supplier doesn't supply a proper feed.

The suppliers are totally fine with that - their sites are purely designed to supply dealers with product info anyway.

 

[awebapart.com]

The OPs argument is that it's not illegal to develop or sell the software but it might be to use it. Nice get out...

It's also not illegal to sell marijuana seeds as long as you don't grow them into plants. Thanks for that but what the hell else are you going to do with them???

Same goes for software developers like this, they may act all innocent and blue eyed by developing the software but they know for a fact this the tool will be used by someone to breach copyright or privacy guidelines which makes the promotion and development of such tools immoral and unethical in my view.

It's a difficult issue that is faced by a lot of tool or service providers, i.e. do you provide a tool or a service knowing that that tool or service could be used in both an ethical way and in ways that are not ethical.

The supply of most types of tools or services require some modicum of soul searching, as there are ethical grey areas with a lot of things...

Do you provide web hosting, knowing that some of your customers could use their websites to breach other people's copyright, or use their web hosting mailing account to spam people?

Do you run a business forum, knowing that some of your members might make copy and paste posts, or link/embed images, which infringe the copyright of others?

You could try your best to monitor the above and crack down on it after it has happened, but there is still the risk that providing the service, you are helping it to happen in the first place.

Do you provide ecommerce software, knowing that some of your customers could take it away and use it to sell dodgy goods or goods that infringe copyright? If it is software which is taken away, then you have little control over its use thereafter.

I personally have made such soul searching decisions too in the areas of hosting and providing ecommerce websites (although not takeaway ecommerce software websites yet), and as for business forums, copyright infringement was one of the (many) reasons I never ventured into that area of running my own forum.

 

[Astaroth]

As for insurance comparison sites, there is more chance of me growing breasts than the comparison site not getting permission to crawl the content in the first place

Not only limited to the insurance market but your statement isn't true.... Direct Line took massive amounts of legal action against the aggregators in the early days as they were all screenscraping them.... so DL got free customers as obviously no commissions had to be paid but it went against their brand ethos and of cause the commercial managers will get their knickers in a twist when they see conversion rates plumit/ their ratings heavily exposed to the market & customer retention nose dive.

 

[garyk]

your building something that already exists and theres loads of em? why????

 

[edmondscommerce]

It's a difficult issue that is faced by a lot of tool or service providers, i.e. do you provide a tool or a service knowing that that tool or service could be used in both an ethical way and in ways that are not ethical.

True enough I suppose.

 

[estwig]

Talking of tools, what do think is the average burglars tool of choice??

A screwdriver.

Do Stanley search their souls when making and supplying screwdrivers??

 

[Optegris]

Yep you could apply the same ethical rule to anything on the planet (including bricks for bashing people round the head with ) but my point stands. Scrapers like this only serve one purpose and that is to quickly gather content without having to resort to the owners permission.

It simply cannot be compare to business forums, hosting or ecommerce platforms as those products are not specificaally designed to carry out shady practises.

 

[Optegris]

Not only limited to the insurance market but your statement isn't true.... Direct Line took massive amounts of legal action against the aggregators in the early days as they were all screenscraping them.... so DL got free customers as obviously no commissions had to be paid but it went against their brand ethos and of cause the commercial managers will get their knickers in a twist when they see conversion rates plumit/ their ratings heavily exposed to the market & customer retention nose dive.

OK I'll buy myself a new lacy bra

 

[dagaul101]

I won't be surprised if sooner or later many sites will have an authorization simply to view their pages, having to sign agreements that you promise not to use their content for any reason anywhere else

 

[edmondscommerce]

I would be very surprised indeed as it would be SEO suicide

check out all the sites like Experts Exchange that try to filter content so you can't read it without signing up, but they still want to rank in the engines...

(scroll down btw - to the bottom)