Just when you think you have a post ready for publication, a vendor comes out and releases a new product and you’re back to square one with a re-write. This was the case at the end of last week when Amazon Web Services (AWS) released news of their new Elastic File System during the AWS San Francisco Summit. Amazon had also been hitting the consumer market with free space for Prime users (for photos only) and unlimited capacity for all users for $59.99 a year. Google has been active with the announcement of Google Cloud Storage Nearline. This post was originally intended to discuss the Amazon consumer and Google offerings but that landscape has now changed to a wider scope.
Let’s look at Google’s new offering first. Google Cloud Storage Nearline is aimed at data that doesn’t need fast access times to and from the cloud, so typically archive or backup data is the target market. Access times are quoted around 3s (yes seconds) with a price point of 1c per GB per month, which is around $10/TB/month. Based on the access time, theres a fair chance this data is being stored on disk, with the drives spun down, rather than being on more offline media such as optical or tape. The reason for this is the fact that tape or optical both have load times that can mean up to 30 seconds to reaching first byte, with the additional complication (certainly for tape) of media contention between customers requesting data on the same physical tape or BD-ROM. This service is a direct competitor to AWS’ Glacier, which offers capacity at the same price point. However Glacier has a very different access profile (3-5 hours to restore, restored data only available for 24 hours), which indicates the use of tape media and a front-end disk cache. Both systems have restrictions on early data deletion, although Google are offering a more generous one month term (Glacier is 3 months).
Amazon’s Elastic File System is file storage for EC2 instances. It is priced at $0.30/GB/month, uses NFSv4 and provides storage that can easily be shared between EC2 instances. There’s no reason to assume that this storage couldn’t also be made available externally through on-premises gateways and it wouldn’t surprise me to see AWS offer this in the future. AWS are placing no minimum fee or setup costs over and above the per GB cost, making it extremely attractive to EC2 customers, especially when the platform is described as “SSD-based”. Time will tell what this means; it’s unlikely all the data resides on SSD, but more likely that SSDs are being used as an acceleration layer – either as a tier, or cache.
As with many AWS services, the EFS feature will find use in 3rd party applications. We can already imagine uses including offsite backups, data mirrors, data repositories for EC2 instance processing and NAS archiving. The benefit here is that file-based data can be kept in file format, rather than having to be converted to object format in S3 as with many other solutions. This makes the data more easily accessible (within the cloud) than data is today. All this flexibility comes at a cost; EFS is 10x more expensive than S3 and 30x more expensive than Glacier. This will probably reduce the use cases and make the service more suitable for active than archived data.
So where does this leave Enterprise storage? At first glance EFS seems expensive compared to onsite solutions. However many comparisons will be made on pure hardware/service costs without looking at total TCO. As the service scales up and prices drop (which they surely will), then EFS will be a more compelling offering. For Google, Cloud Storage Nearline (CSN) is a direct attempt to challenge Glacier and that may represent a problem for AWS, if the Glacier architecture precludes reducing data access times to seconds rather than hours. However for end users, CSN may represent an attractive alternative to buying yet another expensive NAS box when onsite capacity is exhausted. The tricky piece here will be the on-ramp for data coming from the data centre and in tracking on and off-premises content. That is an easy one to solve. These new offerings signal even more the end of on-premises Enterprise Storage as we know it; in 5 years’ time, on-prem storage will be a shadow of its former self.