I love shopping for a new car. The new car smell is more addicting than just about anything else out there. But as my wife will attest, the process starts many months ahead of the actual purchase. I research extensively to make sure I am choosing the car that best fits my needs of today and the next few years. One key step in the process is developing the list of must haves (high safety ratings, space for a 6’ back seat passenger, heated seats) and want to have (navigation, heated steering wheel, good gas mileage). With my list in hand, I am able to eliminate options very quickly until I find the few that fit my needs the best.
In working with thousands of customers over the years, I have found when customers take a similar approach to storage solutions as I have with my car buying, they are guaranteed to be successful. While we as vendors are experts on our solutions, only the customer can be experts on their “must haves” and “want to have.” For those evaluating object storage, I’ve put together a checklist along with some helpful tips and reference materials.
The most important question to start with is “what is the problem you are trying to solve?” Assuming the answer is that you need storage for your data, we recommend reviewing a few relevant articles to make your decision process smoother.
- Build or Buy?
- Storage & Data Management
- Data Management Interfaces
- Service Interfaces
- Unified Namespace
- Metadata Features
How to Determine What Type of Storage to Use: NAS, SAN, Tape Archive or Object Storage
There are a lot of choices for storage today. It is likely that you have already determined that object storage is a good fit for your use case because of the many benefits found in best-of-breed object storage solutions such as Swarm:
- Built-in data protection
- S3 compatibility
- Powerful metadata and search capabilities
If you are not certain that your use case is a good fit for object storage, we recommend that you contact us to talk to a storage architect or check out some of our educational resources on the topic:
- Storage Switzerland eBook: NAS vs. Object—Which is Best for Your Data Center?
- Tech Tuesday Webinar: How Does Object Storage Fit Into Your Infrastructure?
- Tech Tuesday Webinar: What Your Storage Vendor Isn’t Telling You About S3
Evaluating Object Storage for Your Use Case
In 2006 when we launched our first product, it was a simple choice as the options were limited. You could use Caringo or you could go to EMC to purchase Centera. Both of these products evolved from the work of Caringo Co-Founder Paul Carpentier, recognized as the inventor of content addressable storage (CAS).
Through the years, more and more companies started to incorporate object-based storage into their offerings with different levels of success. In 2012, we identified 5 “must haves” for object storage. This included:
- Symmetric Architecture
- Data protection for any size file, any number of files and any capacity
- Instant access, platform neutrality, NO proprietary databases!
- Cloud storage enablement
- Entire stack provided by one company
Fast-forward to 2019, and now there are a lot of object storage vendors to choose from, and that makes the task far more challenging! You not only need to identify the “must haves,” you must determine if the product will work with your existing infrastructure, meet your immediate needs and then grow with your organization or business.
Last year, I talked with Senior Consultant John Bell about this very topic on the Tech Tuesday Webinar: Evaluating Object Storage Solutions. We discussed a number of points in depth. Based on that discussion, we have put together a checklist that can assist you in your evaluation.
Checklist for Evaluating Object-Based Storage Solutions
There are significant differences between object storage platforms. Here are some of the features that you should look at as you narrow the field of products you will take the time to evaluate.
- Does the product have automated rapid recovery?
- How much of the storage hardware is utilized for actual content versus overhead?
- How is the metadata stored?
- What are the performance characteristics of the product, and what are the demands for your use case?
- What level of availability do you need for search, sharing or streaming?
- What is the minimum and maximum capacity?
Build or Buy? Appliance vs. Software Defined
You should determine if it makes more sense for you to build your system (using software-defined object storage) or buy your system (that is, have a turn-key solution where you buy an appliance with the software). There are pros and cons to each. Here are a few things to consider when making this decision:
- Appliance Approach
- “Turnkey” solution
- May not be as flexible as necessary to meet certain requirements
- Units of purchase and associated licensing may also be inflexible
- Software Defined
- Requires more work up front (e.g., hardware sizing and purchase, integration etc.)
- Highly flexible in meeting specific requirements
- Units of purchase and licensing are also very flexible (typically “on demand”)
Storage and Data Management Features
Look at the storage and data management features and compare the products you are most interested in. Here is a list of features you will most likely want to investigate:
- Combination of UI and API
- “Single Pane of Glass” Web Management Portal
- Monitoring and Event Notification
- Automated Failover and Recovery
- Full Availability
- Capacity On Demand
- Volume Portability (This is a key feature not found in “object on file system” or similar solutions!)
Data Management Interfaces
How will you manage the data? I suggest you look for a system that offers you the following:
- “Browse and Query” (Content Portal and API)
- Flexible Protection Schemes
- Combination of Replication and Erasure Coding
- Protection Policy Range (global default to individual object)
- Usage Metering and Quota Support
- Identity Management Integration (Including support for multiple IDM stores)
- Access Control
- Management Delegation
Storage and Service Interfaces
What connections will you need between your Object Storage and your other storage devices or services? Make sure to investigate this thoroughly and outline your requirements clearly.
- Native API (RESTful API based on standard HTTP 1.1)
- S3 (Ideally, a superset of what is found in Amazon S3)
- NAS (NFS & SMB)
- Service Connectors (Public Cloud)
While this feature is often overlooked, it is quite important if you want your Object Storage to function efficiently. For a true Unified Namespace, the storage must have:
- Ability to reference the object stored by the same name…
- Independent of how it was created
- Regardless of how it’s being requested (S3, NFS/SMB, Native API etc.)
- Specifically, names that are “human readable”
- Can be done with UUIDs, but this isn’t user friendly
- Allows for alignment of naming conventions across multiple protocols
- Provides automatic synchronization of name changes
Make sure to look at how the storage system manages metadata, as it is key for keeping your data searchable and accessible.
- Should include standard metadata support for system management and basic object query
- Ideally includes comprehensive support for custom metadata
- “Unlimited” custom metadata
- Ability to list and query on custom metadata
- Collections (saved queries) are a powerful tool for dynamic data/object sets (Ability to surface Collections through multiple access protocols is highly desirable.)
- Metadata should be easily managed and protected by the storage itself!
- No separate metadata servers
- No specialized controller nodes
Support and Other Considerations
As you examine products, make sure that you will have the right level of support in place, along with the professional services and training you need.
- Commercial Support vs. “Do It Yourself”
- Portals for software access and knowledge base
- Outsourced monitoring and notification
- Professional Services
- Requirements gathering
- Training (on-site, online etc., including Certification)
Getting Started with Object Storage
If you have questions or want to discuss how to get started with object storage, contact us. My team and I are ready to help.
While object storage isn’t a panacea, it is an increasingly important storage technology that enables on-demand access for video workflows. More Details »
Organizations across all industries are struggling with enabling instant on-demand access to archived video. Workflows relied upon for years, based on tape, are becoming too difficult to manage and video assets are taking longer and … More Details »