Web developer recommendations (large data set)

[superpav]

Can anyone recommend a web developer who has expertise in dealing with large data sets (over 200,000 SKUS).

Also what ecommerce platform would people suggest? We currently use Magento but our website needs updating. We have a half built website using Woo Commerce but have found this platform is not built for large data sets because the importing SKUS is such a slow process.

I would really appreciate any guidance you may have.

 

[Ozzy]

Have a look through some of the member listings here for a web developer/designer, I could also move this thread to the I'm Looking For section if you like

 

[superpav]

Have a look through some of the member listings here for a web developer/designer, I could also move this thread to the I'm Looking For section if you like

Yes, please move it. Thanks!

 

[Silk Media]

Hi - We could assist. We would probably go for Drupal Commerce for something that size

 

[gpietersz]

I think you will get better advice here if you state your requirements. What does Magento not do that you want? What specific updates do you need? Do you really need to move away from it at all? I believe that are quite a few Magento people here who can advise you on ways you can do what you need with it - if at all possible "if it aint broke dont fix it".

200,000 SKUs is not a large data set. Is there a lot of data per SKU? Is import not a one off process?

 

[fisicx]

I worked on a woo site with 450,000 skus and importing wasn’t that slow. Maybe it’s your hosting causing the problem.

 

[gpietersz]

I worked on a woo site with 450,000 skus and importing wasn’t that slow. Maybe it’s your hosting causing the problem.

I was thinking about whether it might be some sort of customisation or extension?

 

[fisicx]

I was thinking about whether it might be some sort of customisation or extension?

There are lots of ways to import product data. If you are on cheap shared hosting and use free plugin then it's going to be slow. But decent hosting with a premium importer and properly formatted data file and there is no real delay. I seem to recall about 5 minutes - the limitation being the read/write speed to the database.

I did another project for a car loans company and needed to import a huge CSV with all the vehicle data. The plugin I wrote was so fast the server/database couldn't keep up. I had to slow it down and import in 5000 row chunks.

 

[superpav]

I think you will get better advice here if you state your requirements. What does Magento not do that you want? What specific updates do you need? Do you really need to move away from it at all? I believe that are quite a few Magento people here who can advise you on ways you can do what you need with it - if at all possible "if it aint broke dont fix it".

200,000 SKUs is not a large data set. Is there a lot of data per SKU? Is import not a one off process?

We have an old version of Magento (ver. 1.9.3.8) which I understand is no longer supported so will need updating. The website runs quite slowly which isn't good for business.

We could potentially have 500,000 SKUs. We hold back from adding as many as we would like due to issues with the site.

I wouldn't say there is a lot of data per SKU. We have have many grouped products (different sizes of the same product) which causes us issues when formatting the data. Our catalogue gets updated often so we would ideally like to add and remove products regularly.

 

[fisicx]

I wouldn't say there is a lot of data per SKU. We have have many grouped products (different sizes of the same product) which causes us issues when formatting the data.

They could easily be variants. One product with a selector. This can make your catalogue a lot smaller and much easier for customers to find the right product.

 

[superpav]

I worked on a woo site with 450,000 skus and importing wasn’t that slow. Maybe it’s your hosting causing the problem.

It could be, unfortunately I don't have much expertise on these things. We leave this in the hands of our current web developer (who are leaving soon).

It seems we are using Nimbus Hosting (Argon Package).

 

[superpav]

They could easily be variants. One product with a selector. This can make your catalogue a lot smaller and much easier for customers to find the right product.

That sounds sensible! I have spent many days organising spreadsheets into grouped products.

 

[fisicx]

That sounds sensible! I have spent many days organising spreadsheets into grouped products.

There are plugins for woo that do all this for you.

I’ve used this one a number of times:

How Do I Bulk Upload Variable Products in WooCommerce?

This Cheat Sheet explains how bulk upload variable products in WooCommerce, including how to handle the Attributes and Variations tabs.

www.wpallimport.com

 

[antropy]

Also what ecommerce platform would people suggest? We currently use Magento but our website needs updating. We have a half built website using Woo Commerce but have found this platform is not built for large data sets because the importing SKUS is such a slow process.

OpenCart is incredibly efficient at dealing with large numbers of products.

 

[antropy]

OpenCart is incredibly efficient at dealing with large numbers of products.

Seems my post has been edited as I gave an example of an OpenCart site we built with a huge number of products.

I'm not happy about my posts being edited without my knowledge and will consider leaving the forum if moderators keep adjusting my words.

Paul.

 

[gpietersz]

We have an old version of Magento (ver. 1.9.3.8) which I understand is no longer supported so will need updating. The website runs quite slowly which isn't good for business.

We could potentially have 500,000 SKUs. We hold back from adding as many as we would like due to issues with the site.

I wouldn't say there is a lot of data per SKU. We have have many grouped products (different sizes of the same product) which causes us issues when formatting the data. Our catalogue gets updated often so we would ideally like to add and remove products regularly.

It should be easier to upgrade Magento than to develop a new site.

You might want better hosting, or a a custom import script that is more efficient.

The website being slow tends to point to a problem with the hosting too. What sort of hosting do you have?