Switching to cloud storage from on-premise local server

[zigojacko]

In our agency, we're looking into serious options to replace our local server and drives set up to a purely cloud based alternative.

We're a full service digital agency with staff scattered about everywhere working with large amounts of data (we'd need 5Tb to start with and scale accordingly).

I was wondering what any other (largely digital orientated) companies on here do for storage and file access between different people in the company?

I know most people will suggest the likes of Google Drive, Dropbox for Business, Microsoft OneDrive etc but this isn't what we want (we already have a system in place for file syncing between devices) but this is about mass-storage of our data in the company with network access to various people in the company that should need to access - messing around with select-sync per device is not an option really.

 

[Ryan Vant]

Hi,
A number of options are available depending on the full scope of day to day user requirements.

We have some great solutions from pay as you use to monthly commit type models, such as Acronis that has some very beneficial security features

 

[Paul Kelly ICHYB]

I was recently involved in a very similar project for an architects office with about 15TB of storage needed. It was also necessary to move everyone from an old MS SBS, so O365 for the users and Sharepoint for the data.

I am sure Google can do similar.

You could also look at open source like NextCloud to create your own store.

 

[KM-Tiger]

You could also look at open source like NextCloud to create your own store.

Second that.

The big upside of Nextcloud is that *you* remain fully in control of your own data.

 

[Ozzy]

The big upside of Nextcloud is that *you* remain fully in control of your own data.

My thoughts on this is that it's only really important if your business case demands it. If your business case doesn't demand it then it can be beneficial not to take on the overhead of managing the service.

 

[The Byre]

(we'd need 5Tb to start with and scale accordingly).

That seems to be hardly worth the effort - it is very little data by today's standards.

 

[zigojacko]

Second that.

The big upside of Nextcloud is that *you* remain fully in control of your own data.

I did have a look at Nextcloud's website but I didn't really understand what the product was...

It seems to be a lot of stuff that I'm not really after. But I'm not sure.

 

[zigojacko]

That seems to be hardly worth the effort - it is very little data by today's standards.

How do you mean "hardly worth the effort"? We just want to store all our data in the cloud (if we can find a suitable solution) instead of on local servers and hard disks which all do periodically fail and need replacing.

 

[DavidWH]

Why does it need to move into the cloud?

We have a file server in our office, with files going back to 2014, totalling about 250gb.

Rather than move to the cloud, we have our own VPN, so can connect to the server from anywhere with an internet connection. Our servers set up with RAID to mirror the main drives, and have a hot swap drive so we can take backups off site.

 

[zigojacko]

Why does it need to move into the cloud?

We have a file server in our office, with files going back to 2014, totalling about 250gb.

Rather than move to the cloud, we have our own VPN, so can connect to the server from anywhere with an internet connection. Our servers set up with RAID to mirror the main drives, and have a hot swap drive so we can take backups off site.

Because we're tired of having hard disks fail which we need to keep replacing. And recovering data/backing up data just consumes a lot of time and money.

 

[The Byre]

Rather than move to the cloud, we have our own VPN, so can connect to the server from anywhere with an internet connection. Our servers set up with RAID to mirror the main drives, and have a hot swap drive so we can take backups off site.

That's pretty much what we do and we have to have storage in the many-TB range.

We got a 16-bay SSD RAID array from Scan and two years ago and so far, nothing has failed.

We use software that allows for peer-to-peer working, so two people see the same screen and can work on the same project on different sides of Planet Earth - if they want to!

 

[zigojacko]

That's pretty much what we do and we have to have storage in the many-TB range.

We got a 16-bay SSD RAID array from Scan and two years ago and so far, nothing has failed.

We use software that allows for peer-to-peer working, so two people see the same screen and can work on the same project on different sides of Planet Earth - if they want to!

If you don't mind, please could you share a link to exactly what you got from Scan?

We just have hard disks in an HP ProLiant server - the server itself runs fine, it's just the disks (which are generally Seagate or Western Digital mostly).

 

[DavidWH]

Because we're tired of having hard disks fail which we need to keep replacing. And recovering data/backing up data just consumes a lot of time and money.

I don't know your set up, but in all the years we've had our server running, we've had no hard drive failures, and backups are automatic, and it takes a matter of minutes to swap the backup drive.

That's pretty much what we do and we have to have storage in the many-TB range.

We got a 16-bay SSD RAID array from Scan and two years ago and so far, nothing has failed.

We use software that allows for peer-to-peer working, so two people see the same screen and can work on the same project on different sides of Planet Earth - if they want to!

As above, we've no issues. Our server is approaching 50% storage capacity, over the Christmas break I'll be upgrading the OS to the 64bit system, whilst I'm at it I'll probably change it to a VLM set up, so I can add more storage in future.

Are you running server spec drives? I think ours are WD red drives.

 

[The Byre]

If you don't mind, please could you share a link to exactly what you got from Scan?

Sadly, it was a bespoke thing and I do not know what the name of the 19" 4U rack-mountable case was. The MB was Asus, that much I do remember and it had the largest Nvidia graphics card available and two 7.1 audio drives available on the rear.

It has an internal system drive and 16 removable drive caddies on the front and two USB IOs on the front and all the other IOs at the rear. The caddy system means that we can always up-grade it without doing anything other than adding SSDs to the existing ones.

The damn thing is so big that it could only be mounted in our 19" rack vertically. The on/off button is the smallest I have ever seen and I had to call their tech-help-line to locate the silly thing. I think the clitoris on a water vole must be bigger!

It's noisy, so lives in its own machine room together with assorted other things like various PSUs.

I cannot remember the exact price as we bought it as part of a bundle of stuff to get a better price, but I do not think it cost more than £4k.

 

[KM-Tiger]

We just have hard disks in an HP ProLiant server - the server itself runs fine, it's just the disks (which are generally Seagate or Western Digital mostly)

Disks can and do fail, it's when not if. Much as I dislike the word 'Enterprise' when applied to IT software or hardware, it is noticeable that enterprise disks fail less often.

And hardware from big names is IMHO nowhere near as good as you think it ought to be. For many years now I have only bought hardware from these people. It's all RollsRoyce stuff and therefore not the cheapest. They are very approachable and will give good advice.

 

[Nico Albrecht]

It's all RollsRoyce stuff

Had a look and strongly disagree what they flog, Seagate drives and Micron SSD's that's a recipe for disaster they offer to their customers. Samsung Pro series is not available but they flog a glorified intel enterprise SSD which is fancy rebranded consumer one. All in 10% of their products are good the rest is just glorified consumer tech sold under better marketing.

If data is valuable WD Black Series or Gold ones are the way to go the rest is all crap at best. Specially the WD Gold ones which are rebranded Hitachi ones are dynamite and will last 5 years 24/7. Any Seagate drive fitted is a disaster.


As we always say on here RAID IS NOT a Backup as incremental functions are missing and a million other reasons.

Get yourself a Synology NAS or HP ProLiant 10th gen server and fit some WD Gold drives or Black if you can get them. VPN on top and some off site backup solution. Run any OS on then e.g FreeNAS, Nextcloud, Windows or any Linux, Unix flavour you want.

The definition of enterprise tech might differ for some people but I would consider enterprise tech to be available for many years of spare parts, fast turn around for warranties and plenty of redundant parts e.g power supply.

In an enterprise environment a failed drive is no biggy. Taken out new drive in from in house stock , faulty drive RMA.

Last server I brought in for a smaller company was a Proliant plus 10th gen with the option for an extra £60 to get a guaranteed 24 hour turnaround time for 3 years. Either within 24 hours they have the part shipped to you or a skilled monkey will be send out to sort it. Thats something you wont get in consumer tech.

 

[NickGrogan]

How do you mean "hardly worth the effort"? We just want to store all our data in the cloud (if we can find a suitable solution) instead of on local servers and hard disks which all do periodically fail and need replacing.

AWS?

EC2 H1 if you want cheaper, HDD based storage.

i3EN if you want stupidly fast NVMe SSD storage

or

S3 storage if you want cheaper.

 

[AW-UK]

In our agency, we're looking into serious options to replace our local server and drives set up to a purely cloud based alternative.

We're a full service digital agency with staff scattered about everywhere working with large amounts of data (we'd need 5Tb to start with and scale accordingly).

I was wondering what any other (largely digital orientated) companies on here do for storage and file access between different people in the company?

I know most people will suggest the likes of Google Drive, Dropbox for Business, Microsoft OneDrive etc but this isn't what we want (we already have a system in place for file syncing between devices) but this is about mass-storage of our data in the company with network access to various people in the company that should need to access - messing around with select-sync per device is not an option really.

Local servers come with an overhead that some do not want to take on, updating, maintaining etc, I've done it for years on and off, and it can be a ball ache, that said, cloud solutions are a lot better these days and if you want to look at replacing it with a manage datacentre I'd look at something like Azure, or A N other VPS (Virtual Private Server) / VDS (Virtual Dedicated Server) offering, which gives you essentially a PC in the "cloud", we put our own servers running VMWare into data centres, and we span across the country, and we process data for TfL there, but we are responsible for the upkeep, we have a plan in place to get the local data technicians to swap disks out in the RAID array and instructions on how to do this, we run monitoring tools on the servers that tell us as part of preventative maintenance if a disk is about to fail.

So, you have several options.
1. Keep your local server and maintain it.
2. Go for a NAS solution, which essentially just acts as a large storage drive with SAMBA share mapping, some can do other things like run media servers.
3. Use a VPS / VDS from Azure, AWS, DigitalOcean, Linode etc
4. Install your own server in a datacentre, I wouldn't recommend this as it is a costly option and still needs maintaining unless you pay the datacentre to maintain it for you and supply them with spare parts.

I would look at VPS but you need a network connection, if you need local then install a NAS or at least have a backup NAS that synchronises with the cloud.

You can get pure cloud storage solutions like DropBox, but it all depends what you need it to do.

 

[DontAsk]

If you don't own the server you don't own the data.

 

[AW-UK]

If you don't own the server you don't own the data.

Technically not true, you can't stop the owners of the server from being able to see the data, but you can encrypt it so it isn't readable, you might not be able to stop them from destroying the data, but I guess it depends on how valuable that data is to someone other than yourselves and if someone knows the industry you serve to be able to repurpose that data.

You can own the server and the building it is housed in, but you can't stop someone from breaking in and stealing the server, but you can try and prevent it, same for data, encryption is a form of preventing people from stealing, and data protection, but if someone is hell bent on getting that data and they have means they will get it.

 

[DontAsk]

Technically not true, you can't stop the owners of the server from being able to see the data, but you can encrypt it so it isn't readable, you might not be able to stop them from destroying the data, but I guess it depends on how valuable that data is to someone other than yourselves and if someone knows the industry you serve to be able to repurpose that data.

You can own the server and the building it is housed in, but you can't stop someone from breaking in and stealing the server, but you can try and prevent it, same for data, encryption is a form of preventing people from stealing, and data protection, but if someone is hell bent on getting that data and they have means they will get it.

<sigh>

 

[Hooble]

In our agency, we're looking into serious options to replace our local server and drives set up to a purely cloud based alternative.

We're a full service digital agency with staff scattered about everywhere working with large amounts of data (we'd need 5Tb to start with and scale accordingly).

I was wondering what any other (largely digital orientated) companies on here do for storage and file access between different people in the company?

I know most people will suggest the likes of Google Drive, Dropbox for Business, Microsoft OneDrive etc but this isn't what we want (we already have a system in place for file syncing between devices) but this is about mass-storage of our data in the company with network access to various people in the company that should need to access - messing around with select-sync per device is not an option really.

Hello, for around 5TB if you were going to use a service like dropbox, google drive and Microsoft One Drive, you would either need to sync it or connect and download files on demand. Depending on the size of the files you may find this delays you whilst you're waiting for them to download. I would tend to suggest a hybrid cloud in that instance where you need the data on demand. But also know that you have a backup in the Cloud. May I ask what kind of files these are? Always up for a chat if you want.

 

[forevergroup]

Azure Files sounds like the most direct fit for this. It's essentially a fully managed remote SMB share service. 5TB data is going to run you about £100/m for 'Hot' tier locally-redundant UK storage. Plus transaction-related costs.

Depending on file sizes, you may find that SharePoint Online libraries mapped to client devices using Microsoft Intune / Endpoint Manager is a good fit (and cheap as chips).

Also makes things easier from a backup standpoint (because Microsoft 365 backup vendors x a trillion).

You've mentioned that you don't want to locally-sync files, but then you're just accessing them over the cloud which opens you up to literally everything (SharePoint, Dropbox, Acronis etc).

You could be clever and spin off - for example - a major current client project - or shared resources - in to their own sharepoint library and just sync those ones offline to users while needed. Everything else can be accessed on-demand. That will give you the best of both worlds in terms of sync vs local performance.

The only credible way I can think of to get on-premises file access performance but with 'full cloud' and equal remote user access is going to be using a public/private cloud scenario where users use remote desktops in the cloud, and file storage is local to those remote desktops.

Still need backups in all scenarios, though.