Call it whatever you want: a digital gold rush, a technological arms race, a new-age industrial revolution.
In today’s world of lightspeed advancements within the technology world, the companies within that world will always be in heated competition for vital resources and the spoils that accompany them.
And as the cloud continues to pervade the landscape as “The Next Big Thing,” IT giants like Amazon, Google and Microsoft are engaged in an epic and bloody battle for the hearts and minds of companies looking for a cost-effective, reliable, durable and secure home for their precious data and workflows.
As things stand today, Amazon’s AWS platform is the overall market leader for public cloud services, however Microsoft Azure and Google Cloud Platform are both growing fast, offering more services and expanding their regions as they seek to gain – or keep – their precious traction.
As the differences between these three services continue to narrow, who companies ultimately choose for their cloud services often comes down to minute details with regards to things like compute, database, or storage. For this article, we’ll focus our attention on the latter.
Storage is, of course, a key component for IT decision makers looking to move massive amounts of data to the cloud. But even between the offerings of AWS, Azure and GCP, there are so many different types of storage options available, choosing the right one for your business can be a major challenge. Luckily for you, we’re here to help shed some light on storage.
1. Apples to Apples
The first thing to do when comparing cloud storage services is to make sure you are matching like-with-like. For example, it doesn’t make sense to compare, say, Azure Blob Storage with AWS Elastic File System, because they are two completely different services.
The correct comparison for the above would be Azure Blob Storage vs. AWS S3 and Google Cloud Storage because they all perform broadly the same function – storing unstructured object data. With this in mind, you can now look at features such as cost, availability levels, durability, third party support, etc. when comparing cloud offerings.
Note: If you’re looking for a quick comparison on just Azure and AWS, Microsoft’s site can be a useful reference. But we think GCP has done more than enough to warrant being in the discussion as well, so keep reading for your full scope of options.
When comparing cloud object storage providers on cost alone, the devil is in the details. As a ballpark figure, each of the Big Three charge roughly 2 cents per gigabyte of storage, but each offer slightly different discounts for volume. The basic storage costs will also depend on where the data sits in the cloud provider’s tier system.
What is the tier system, you ask? We got you.
AWS, Azure and GCP all offer help with organizing your unstructured data via a tier system based on how frequently you need to access the data. AWS offer three tiers: ‘Standard,’ ‘Standard-infrequent Access’ and ‘One zone-infrequent Access,’ plus they also offer their Glacier archive service.
Azure opts for a simple ‘Hot,’ ‘Cool’ and ‘Archive’ tier while Google provides two high-frequency tiers (‘Multi-regional’ and ‘Regional’) and two low-frequency tiers (‘Nearline’ and ‘Coldline’).
Basically, the higher tier your data is in, the more availability and redundancy you get and the more you have to pay per gigabyte. The highest tiers are for mission critical data which is frequently accessed, whereas the lowest and cheapest tiers are reserved for rarely accessed data. In addition to being less available (although by ‘less,’ we’re talking about 99.95% vs. 99.99% here!), this tier of data will be at greater risk of loss as it will not be duplicated across regions or availability zones.
Data can be easily transferred between tiers and the transfer can be automated using object lifecycle management protocols.
Considering the above, plus a host of other data operation and transmission costs, it’s clear why a straightforward cost comparison is not always possible. Fortunately, that’s where experienced consultants like Shamrock become invaluable in sifting through the cost-related chaos to ensure that you’re getting exactly what you need at the guaranteed best price.
Winner: GCP. We estimate that GCP wins out with customers roughly 60% of the time with regards to overall costs. In roughly 40% of scenarios, though, Azure is the clear choice, especially for enterprise customers who already have large licensing agreements in place with Microsoft.
AWS is the cheapest option exactly 0% of the time.
3. Identity Access Management
We’ll keep this section short and sweet, because there’s really no competition here. Azure’s seamless integration with Active Directory is the absolute gold standard across enterprises, and it’s not even close.
4. Availability, Security and Third-Party Support
As you might expect, all three of the big cloud providers are very closely matched when it comes to data availability, durability and security. All guarantee at least 99% availability and 99.99%-plus durability, and all stored data is encrypted using either 128-bit or 256-bit AES encryption by default. You can even choose to allow the cloud provider to manage the encryption keys, or you can enable customer-managed keys for greater flexibility with access control and auditing capability.
In terms of third-party support, AWS is the clear market leader, boasting the largest number of integrated services anywhere, which is why AWS is a favorite among IoT developers.
Availability Winner 3-way tie. Availability and durability SLAs are nearly identical across all three of the big public clouds.
Security & Third-Party Support Winner AWS. AWS has the largest partner network and the most third-party tools of anyone in the marketplace.
Azure, to their credit, is catching up thanks to their massive ecosystem of partners who are starting to adopt cloud practices at an increasing rate.
GCP’s automated encryption standards give them high marks in security, but their overall security features aren’t quite on the level of AWS or Azure…yet.
5. Cloud Block Storage Services
Cloud block storage services store unstructured or structured data in volumes which can be attached to your VMs and used just like disk drives. Block storage enables incremental data changes to be easily made and is usually accessed via a file system.
As with object storage, the three big cloud providers all offer reduced costs, fast scalability, and high levels of availability and redundancy in their block storage services: AWS Elastic Block Storage (EBS), Azure Disk Storage and Google Persistent Disk.
The redundancy of Amazon’s EBS was compromised in the infamous AWS outage of 2011, when human error led to a so-called ‘re-mirroring storm.’ Despite this, AWS remains confident enough to offer 99.99% availability and durability for EBS and, in fairness, Azure and GCP have also suffered big outages to their services (they just didn’t cause such a noticeable disruption).
Whereas redundancy in EBS is restricted to one availability zone, with users advised to create regular VSS snapshots and store them in S3, Azure Disk Storage offers several replication options, including geo-redundant storage (GRS), which spans regions. Azure also offers a full backup service which is an attractive USP. Google Persistent Disk, like EBS, provides VSS snapshots.
Winner: GCP. Their magical versioning for block storage puts them best-in-class.
6. Queue Services, File Systems, Batch Transfer Appliances and Hybrid Storage
Another common purpose for storage is to manage message queues to and from applications and microservices. Both AWS and Azure offer comparable services in AWS Simple Queue Service (SQS) and Azure Queue Storage, respectively.
AWS Elastic File System (EFS) enables users to easily create and configure file systems, adding and removing files without affecting the performance of apps. As the name suggests, the service can be scaled up or down on demand, which helps keep costs under control. EC2 instances mount EFS systems using the NFSv4 protocol and each object (file, directory, link etc.) is distributed across Availability Zones for the purposes of redundancy.
Azure’s version, called Files, uses the familiar SMB protocol and is also scalable, with the option to pay only for what you use. Azure Files also offers both locally redundant and geo-redundant storage.
Google doesn’t offer a cloud-based file system, but they have developed the open source Cloud Storage FUSE, which translates object names into a simple and scalable file system. This offers fast throughput but may not deliver the user experience of its AWS and Azure counterparts.
If you anticipate having to store and move petabyte scale levels of data, both Amazon and AWS provide data warehouse services: AWS Redshift and Azure Data Lake, respectively. While Data Lake offers enhanced scalability and cost control capability, Redshift can be hosted on a VPC for extra security.
AWS and GCP also offer physical batch file transfer services, which is ideal if you want to migrate terabytes of data without the internet transfer costs and latency issues. AWS Snowball and Google Transfer Appliance (GTA) both involve ordering an NAS appliance which is then shipped back once data has been copied over, ironically slashing the time to cloud. GTA is a relative newcomer, having only come out of Beta earlier this month, but GTA already trumps Snowball’s 80TB capacity with their 100 TB or 480 TB options.
The introduction of GTA shows just how keen Google is to get their hands on enterprise data. For serious data migration though, AWS Snowtruck will transport up to 500 PB of data from your data center, delivering its hardware via a shipping container on a literal truck. And if you were wondering about Microsoft, Azure Data Box is currently in preview and will offer a capacity of 100 TB when launched. Ho hum.
Finally, if you’re looking for the halfway house version of hybrid storage, both AWS and Azure offer services that can facilitate that. Each gives you the benefit of cost reduction by replacing on-premise storage with cloud storage, as well as an on-premise virtual machine (VM) or hardware appliance. AWS Storage Gateway is downloaded as a VM and stores your main data in Amazon S3 buckets with frequently-used files cached on locally attached storage. Azure’s StorSimple performs a similar function but uses an appliance featuring a mixture of HDD and SSD disks on a premise-mounted appliance for local storage.
Winner: AWS, hands down. Between Snowball, Snowball Edge, Snowmobile and Snowtruck, AWS blankets the competition in a proverbial whiteout.
In terms of databases, all three of the big players offer a range of SQL and NoSQL databases along with associated services. In fact, the DbaaS (database-as-a-service) model is quickly becoming accepted as a standard enterprise requirement.
Amazon’s offering is AWS RDS (Relational Database Service), while Microsoft gives us Azure SQL Database and Google provides Cloud SQL. Consultancies offer a definitive bonus when deciding between these three services, as there are some fundamental differences which may not be immediately apparent.
For storage size and use case flexibility, Google Cloud SQL comes out on top with its up to 10 TB per instance. Next is AWS RDS with 1 TB per EC2 instance and customization options. In contrast, Azure SQL Database offers just 10 GB, and the way it handles its multi-tenant architecture restricts customization.
On the other hand, Azure SQL Database is fully cloud-native (rather than just cloud-compatible), offers active geo-replication and is generally less expensive and easier to scale than AWS RDS. Azure SQL Database also offers the benefit of familiarity to businesses coming from or still working within a Microsoft environment.
Touching on some of the associated database services as they relate to AWS: There’s Aurora, a hosted database solution which auto-allocates storage in 10 GB increments and streamlines query throughput; DynamoDB, a managed NoSQL database service charged on throughput rather than storage; Neptune, a powerful graph database compatible with Property Graph; and RDF and ElastiCache, both used for extremely fast caching.
With Azure, you can get SQL Data Warehouse Service for lightning fast querying; CosmosDB and Table storage, two NoSQL database services with associated key-value stores for high volumes of semi-structured data; Redis for in-memory cache; and Server Stretch Database for cutting costs by stretching transactional data from your on-premises SQL Server to the cloud.
Lastly, Google gives you Cloud Spanner; a so-called NewSQL database for blending the benefits of relational and non-relational database storage; and Cloud Datastore, a managed NoSQL database built on Cloud Bigtable, a high-performance data storage system.
Winner: AWS takes the cake here, as they support the most types of databases, have the top tools and the best analytics. Amazon’s RDS services are the big carrots being dangled so to speak; everyone else just playing catch-up at this point.
8. Machine Learning APIs
Another short section here but we did want to point this one out as machine learning becomes a bigger and more important offering within the marketplace. Google has the easiest API’s to use, and they’re also the most effective. GCP’s Vision API, Google Neural Machine Translation (GNMT) and Natural Language are all best in the biz.
Make no mistake: Amazon does have more of these APIs available, but Google’s are simply better. Azure is a solid third place here, being least built out of the three.
Finding the Best Cloud Provider for Your Business
At the end of the day, only you can make the decision on public or private cloud is right for your business. But with so many variables going into just one cloud component such a storage, we wanted to illustrate the differences between each in hopes that it might help in your decision-making process. We’re partners with AWS, Azure and GCP and work with each equally in our day-to-day, so rest assured the conclusions we’ve come to in this article are completely objective, albeit open to interpretation.
The Cloud War is one which may never be “won,” however the true victory comes with being able to maximize your business processes via the cloud while also improving your bottom line and virtually eliminating your CapEx.
That’s where Shamrock Consulting Group comes in: we’re on the forefront of the cloud revolution and we are well-versed in helping enterprise customers save as much as 70% by migrating to the cloud. We’re a vendor-neutral consultancy with the largest reach in the industry (we partner with over 100 connectivity providers, including AWS, Azure and GCP), so whoever you decide is the winner in the battle for your big data, Shamrock is on your side.