Testing hosting provider performance

[ctrlbrk]

This could either go here or in IT & Internet, feel free to move it mods.

So I have gone through an analysis of a few hosting providers and I have shortlisted a couple. Since they are both very similar in terms of offering, pricing, datacentre locations, etc. what I think it comes down to is their performance.

I have uploaded an exact copy of a test website to both providers. I was thinking of using FireFox Network tab to analyse their response times (all other things being equal, such as web page opened, time of the day, etc.) with metrics such as DOMContentLoaded, finish, etc. but I'm not sure if that's going to provide me with any insight, or whether there are smarter ways to go about it?

 

[Kerwin]

This could either go here or in IT & Internet, feel free to move it mods.

So I have gone through an analysis of a few hosting providers and I have shortlisted a couple. Since they are both very similar in terms of offering, pricing, datacentre locations, etc. what I think it comes down to is their performance.

I have uploaded an exact copy of a test website to both providers. I was thinking of using FireFox Network tab to analyse their response times (all other things being equal, such as web page opened, time of the day, etc.) with metrics such as DOMContentLoaded, finish, etc. but I'm not sure if that's going to provide me with any insight, or whether there are smarter ways to go about it?

Stress Test Website Performance

Learn what a website stress test is, how to perform website load testing on your site, and what the top 3 stress testing tools are.

www.inmotionhosting.com

 

[antropy]

I have uploaded an exact copy of a test website to both providers. I was thinking of using FireFox Network tab to analyse their response times

Good idea to upload an exact copy but best to use an online speed test so that your own computer/internet connection doesn't affect it.

The idea is a good one though.

We also offer super-fast hosting so feel free to get in touch if you want another option: [email protected]

Paul.

 

[BillyH]

I'm kinda in the same situation but I've really just hunted around for opinons. My priorities are mega fast hosting as well as decent support and UK based hosting. I found a few websites who reckon they've done the legwork in testing hosting speeds and I asked around on the likes of Reddit for people's thoughts. I've arrived at Krystal hosting looking to be my preferred hosting and intend to move my UK targeting website over there soon (using their diamond package).

Interestingly (at least I think it's interesting I have a more or less identical website hosted in the UK on standard SSD hosting with Cloudways and the other on a US host (A2hosting) using NVME SSDs and that's almost a second quicker loading according to Google Analytics. So I'd def go for Nvme - which the Krystal Diamond package is.

 

[Kerwin]

I'm kinda in the same situation but I've really just hunted around for opinons. My priorities are mega fast hosting as well as decent support and UK based hosting. I found a few websites who reckon they've done the legwork in testing hosting speeds and I asked around on the likes of Reddit for people's thoughts. I've arrived at Krystal hosting looking to be my preferred hosting and intend to move my UK targeting website over there soon (using their diamond package).

Interestingly (at least I think it's interesting I have a more or less identical website hosted in the UK on standard SSD hosting with Cloudways and the other on a US host (A2hosting) using NVME SSDs and that's almost a second quicker loading according to Google Analytics. So I'd def go for Nvme - which the Krystal Diamond package is.

Hmm, I'd be surprised if the difference in speed was only because of SATA SSDs rather than NVMe SSDs. What is the network like? Differences in latency can make a massive change in perceived speed.

 

[Nico Albrecht]

I'd def go for Nvme

NVMe, although often marketed heavily, is primarily a communication protocol that enables faster transfer speeds between storage and cpu compared to SATA3. The true value of SSDs lies not solely in the communication protocol, but in factors such as the SSD's CPU, the quality of the NAND chips, the utilization of DRAM caching, and the effectiveness of ECC and XOR operations.

It's worth noting that the majority (approximately 95%) of SSDs available on the market are native SATA SSDs. Samsung, for example, employs an NVMe to SATA bridge for all their NVMe SSDs, while Toshiba is one of the few manufacturers that actually produce native NVMe SSDs.

In terms of real-world consumer applications, a high-performance SATA SSD like the Samsung Pro often outperforms mid-range NVMe SSDs without any significant challenge. A notable example is the development by Phison of a new CPU for the OEM SSD market that is advertised as 50% faster than the previous generation. However, this improvement in CPU power can result in the SSD's performance being hampered if it is coupled with low-quality memory chips. With low-quality chips, write errors increase significantly, and the additional CPU power is consumed by error correction, nullifying the expected 50% gain or potentially yielding even worse performance than the previous generation.

It's important to recognize that not all SSDs are created equal, and in recent years, there has been a trend toward using cheaper, low-quality, high-density chips paired with additional RAM for caching purposes to mask poor overall performance.

Performance outcomes depend on the specific deployment and environment. SATA utilized a separate controller for communication management, whereas NVMe relies on the CPU for management. If the computer's CPU is already under heavy load, it may struggle to handle the communication quickly enough. Consequently, many faster SSDs in RAID setups are limited by the capabilities of the computer's CPU, with the SSDs waiting for the CPU to process their requests.

These explanations provide a simplified overview, and exceptions exist in various cases. If you want to assess the performance of the NVMe SSDs used by your hosting company, it would be advisable to inquire about the specific type and size of the SSDs so that their performance can be evaluated accordingly.

In terms of testing, installing your site locally on both a SATA SSD and an NVMe SSD would likely reveal no significant difference in performance. SATA 3, in real-world scenarios, typically achieves speeds of around 580 MB/s. Considering this, it is unlikely that your website's loading requirements would surpass even 1% of that performance.

 

[dx3webs]

There are so many variables that the difference between ssd / nvme is not going to be measurable.

Raw speed is only 1 factor. In this, website performance is not unlike computer games. If you want the fastest possible page load time then you can opt for a low-core-count high-clocked processor that will BLAST the competition by a few MICROSECONDS!! ... till your visitor count goes up and those 4 / 8 high clock cores are swamped by traffic and your site grinds to a halt. At which point a lower clocked high core processor will render those microsecond differences meaningless. The same goes for other metric.. IO / Bandwidth etc.
It's all horses for courses. Speak to your perspective hosts and ask them what they think will be the best fit you particular scenario. No point paying for fastest possible page load if it can't handle your traffic levels etc.

 

[ctrlbrk]

Thanks for the input but we're going off topic now.

The original question is:

all other things being equal, what is a sensible approach to measure the performance of two hosting providers?

By 'performance' I mean: how fast any given webpage is served to the users.

 

[dx3webs]

The only real way to do this is to upload your actual website (fully optimized ie caches / varnish etc) and then use a load testing tool to generate real world test scenarios eg add to cart (if it is ecommerce) at the max visitor numbers you are expecting eg black friday numbers.

 

[ctrlbrk]

and then use a load testing tool to generate real world test scenarios

Thanks, any particular recommendation?

 

[dx3webs]

You can use zabbix to do this. However, sign up for a free trial with https://www.loadview-testing.com/ or similar (google load testing web site for others)

 

[Kerwin]

Pick a scripting language. Get the selenium bindings for said language. Write a script to automate a set of actions on your website. Execute said script in parallel with the number of times you want to test for clients doing said action on your website.

 

[antropy]

Interestingly (at least I think it's interesting I have a more or less identical website hosted in the UK on standard SSD hosting with Cloudways and the other on a US host (A2hosting) using NVME SSDs and that's almost a second quicker loading according to Google Analytics. So I'd def go for Nvme - which the Krystal Diamond package is.

Yep, NVMEs are another level up from SSDs.

What website software are you running?

Paul.

 

[Kerwin]

I was mistaken.

 

[BillyH]

NVMe, although often marketed heavily, is primarily a communication protocol that enables faster transfer speeds between storage and cpu compared to SATA3. The true value of SSDs lies not solely in the communication protocol, but in factors such as the SSD's CPU, the quality of the NAND chips, the utilization of DRAM caching, and the effectiveness of ECC and XOR operations.

It's worth noting that the majority (approximately 95%) of SSDs available on the market are native SATA SSDs. Samsung, for example, employs an NVMe to SATA bridge for all their NVMe SSDs, while Toshiba is one of the few manufacturers that actually produce native NVMe SSDs.

In terms of real-world consumer applications, a high-performance SATA SSD like the Samsung Pro often outperforms mid-range NVMe SSDs without any significant challenge. A notable example is the development by Phison of a new CPU for the OEM SSD market that is advertised as 50% faster than the previous generation. However, this improvement in CPU power can result in the SSD's performance being hampered if it is coupled with low-quality memory chips. With low-quality chips, write errors increase significantly, and the additional CPU power is consumed by error correction, nullifying the expected 50% gain or potentially yielding even worse performance than the previous generation.

It's important to recognize that not all SSDs are created equal, and in recent years, there has been a trend toward using cheaper, low-quality, high-density chips paired with additional RAM for caching purposes to mask poor overall performance.

Performance outcomes depend on the specific deployment and environment. SATA utilized a separate controller for communication management, whereas NVMe relies on the CPU for management. If the computer's CPU is already under heavy load, it may struggle to handle the communication quickly enough. Consequently, many faster SSDs in RAID setups are limited by the capabilities of the computer's CPU, with the SSDs waiting for the CPU to process their requests.

These explanations provide a simplified overview, and exceptions exist in various cases. If you want to assess the performance of the NVMe SSDs used by your hosting company, it would be advisable to inquire about the specific type and size of the SSDs so that their performance can be evaluated accordingly.

In terms of testing, installing your site locally on both a SATA SSD and an NVMe SSD would likely reveal no significant difference in performance. SATA 3, in real-world scenarios, typically achieves speeds of around 580 MB/s. Considering this, it is unlikely that your website's loading requirements would surpass even 1% of that performance.

Wow thanks for that. My world view was that NVMe = faster website. Guess I need to change that now!

 

[BillyH]

Yep, NVMEs are another level up from SSDs.

What website software are you running?

Paul.

Wordpress.

 

[antropy]

In terms of testing, installing your site locally on both a SATA SSD and an NVMe SSD would likely reveal no significant difference in performance. SATA 3, in real-world scenarios, typically achieves speeds of around 580 MB/s. Considering this, it is unlikely that your website's loading requirements would surpass even 1% of that performance.

Depends on load. A test on localhost isn't a particularly good test. Try it with 1000s of concurrent users.

Paul.

 

[antropy]

Wordpress.

WordPress can be pretty slow without loads of caching plugins and optimisations.

You could consider a different platform - check out how ridiculously fast our website is: https://www.antropy.co.uk/

Paul.

 

[BillyH]

WordPress can be pretty slow without loads of caching plugins and optimisations.

You could consider a different platform - check out how ridiculously fast our website is

Paul.

I'll give you that... that's proper fast. Checked out a couple of your customers too and they're mega fast. Good work Here's a question for you... (hope i'm not hijacking this topic as it's still website performance related!) now that Universal Analytics is retiring, is there a (free) tool that offers the same sitewide speed test results - rather than single page speed tests? BTW sorry I had to delete your link in my reply as it said I can't post links yet!

 

[ctrlbrk]

hope i'm not hijacking this topic as it's still website performance related!

Actually, you are hijacking the topic. My original question has a different purpose.

Can you please create a thread of your own?

 

[antropy]

Thanks re. your comments on our speedy site! I'd say that level of speed is hard to achieve with WordPress - we use Concrete CMS for ours and it's literally as fast as static HTML files - we've compared them both.

We've also added various other tricks to speed it up even further.

is there a (free) tool that offers the same sitewide speed test results - rather than single page speed tests?

I'm not aware of one, but as above, it could be worth creating a topic about it!

Paul.

 

[dx3webs]

This guy runs a wp competition each year for hosts

WordPress Hosting Performance Benchmarks

The most comprehensive performance test for WordPress and WooCommerce hosting performed annually.

wphostingbenchmarks.com

oh and there is no need for wp / woo to be slow.. dx3webs.com We have woo sites with hundreds of thousands of products running just as fast.

 

[antropy]

We have woo sites with hundreds of thousands of products running just as fast.

How do you configure them for that? Because out of the box WooCommerce is sooo slow.

Paul.

 

[dx3webs]

(we really are getting off topic.. but custom varnish config is the answer and the client has a really slimmed down template)

 

[antropy]

(we really are getting off topic.. but custom varnish config is the answer and the client has a really slimmed down template)

The post is literally about comparing server performance so I'd say we're still about as on-topic as it gets?

Paul.

 

[Kerwin]

You should also ensure you have combined and minified CSS and JavaScript assets. Even better would be to store them on a CDN for delivery. You can cache a page render for, say, 30 seconds which would help reduce load at busy times as it would reduce the number of concurrent requests that need to be handled at any given time.

 

[antropy]

You should also ensure you have combined and minified CSS and JavaScript assets. Even better would be to store them on a CDN for delivery. You can cache a page render for, say, 30 seconds which would help reduce load at busy times as it would reduce the number of concurrent requests that need to be handled at any given time.

That's all front-end performance. We're concerned with back-end performance when choosing a host.

Paul.

 

[Kerwin]

That's all front-end performance. We're concerned with back-end performance when choosing a host.

Paul.

That is a fair point, although server-side caching makes a huge difference and is handled on the backend. Oh, and merging and minifying JS and CSS files and serving them from a CDN makes a massive difference as it reduces the number of network requests required along with network latency. That needs to be handled on both the backend and the frontend.