The enterprise data landscape is rapidly changing. Depending on which analysts you listen to, there will be 7 to 40 zettabytes of data by 2020 and 80–90% of this data will be unstructured. To provide some perspective, a zettabyte is equal to 1,000,000,000,000 gigabytes. Adding yet another layer of complexity is the unpredictable nature of access, which is coming directly from applications over the web by millions of users with unpredictable access patterns. Compounding this issue is the expectation of “cloud-conditioned” users to store everything they create, and access it forever, from any device.
I recently hosted a webinar to discuss how object storage is being used in today’s enterprise data landscape and where object storage fits into the broader Networked Storage Market.
How people spend time on the Internet is changing every year. FreshINFOs compiled a list of some of the fastest growing applications in the world, most of which have created proprietary infrastructure solutions out of necessity. They are experiencing “web-scale” growth and need to ensure user experience and instant access or their business will suffer and possibly disappear just as fast as it appeared. Instagram users, for example, uploaded 76% more photos per minute in 2014 than they did in 2013.
So, how do we store all of this data? For decades, organizations have used file-based storage and block storage, but these technologies are struggling to keep up with the ever-increasing demands for reliable, accessible, scalable storage. More recently object storage has been developed to handle the limitations inherent to these older technologies.
In file-based storage the file system manages directories, files, and storing on disc through inodes and blocks. Because of the use of inodes, there are finite limits to many file systems and most have problems scaling beyond the petabyte range. File-based storage is usually accessed via a standard protocol like NFS or SMB. Fixed file attributes like type, size, date created and date modified are stored in the file system. File-based storage is good at shared files and shared directories over a LAN or WAN. There are cloud-based file system services emerging like Amazon Elastic File System and OpenStack Manila, but they are still relatively expensive or still in development.
In block storage, data is organized as an array of blocks. You carve out a portion of these blocks to form a volume that can be formatted as a local block device. Access for block storage is usually through a client on the operating system over Fibre Channel, iSCSI, or ATA over Ethernet (AoE). Fixed system attributes are stored with the block. Block storage is ideal for transactional or structured information like databases or running VMs and is optimized for performance or Input Output operations per second (IOPS).
Object storage is based on key value addressing. You can store an object and get a key, just like a car with a valet: give them your car, take a ticket, and then give them the ticket to retrieve your car. In object storage, the client or access method is usually an application over HTTP, and system or custom information about the file is stored in metadata. Object storage is ideal for shared, infrequently changing files and for scalable, multi-site deployments.
The nature of object storage allows it to thrive across heterogeneous clusters of commodity hardware. Nearly all of each storage device’s capacity can be fully utilized because the system does not waste any storage on managing a hierarchical file structure. Due to the flat, global address space, object storage can be spread across many devices efficiently—providing significant advantages in simplicity and cost effectiveness over block and file-based storage for massive volumes of data.
In the rapidly evolving landscape of enterprise data storage, one thing is clear: we need to store more data, more simply, more efficiently, and for a lower overall cost. Object storage addresses these issues without expensive custom hardware which will need to be replaced every few years. As we enter an age of zettabytes of data, more and more enterprises are turning to object storage as their go-to storage solution.
Is your organization ready to store, secure, and access your data over the next decade in the new age of web-scale storage? View our recent webinar to learn more about how object storage is being used in today’s enterprise and what it can do for you. If you are interested in learning more, visit us at Caringo.com and complete a request form and follow us on Twitter (@CaringoStorage).