In recent years, the industry has seen an explosion of software-defined solutions, not only for storage, but for networking and even the entire data center. Software-defined data management and storage solutions such as Caringo Swarm allow you to deploy in any hardware platform that meets certain requirements or specifications. Recently, I’ve been working on Caringo’s Hardware Compatibility Certification for Western Digital’s Ultrastar Serv60+8 Hybrid Storage line of servers. Thanks to its disk density, build quality and compute power, it is an exceptionally capable platform for managing unstructured data at massive scale.
This week, Caringo announced the collaboration with Western Digital to provide a joint solution that enables organizations to manage unstructured data at massive scale like never before—they can now store up to a petabyte (PB) of data in just 4U with 18-TB drives. There are many benefits to this solution, including reduced costs and simplified data management. The solution is particularly valuable for supporting remote access requirements for collaboration, as well as for data mining and monetization of existing assets.
Why do hardware certifications matter?
From the vendors’ perspective, hardware certification shows the industry, prospects and customers that compatibility is assured and the joint solution operates as intended in critical situations meeting specific standards. From the customers’ point of view, it signals the combined solution is reliable and that it can be trusted.
What steps are taken to ensure hardware and software compatibility?
There are 4 distinct steps we take in the Caringo Hardware Compatibility Certification process, and it starts with reviewing hardware specifications and deploying Caringo Swarm so we can run verification and resilience tests. Here is a bit more detail on each of these steps:
Step 1: Review hardware specifications
Building a great object storage solution based on Caringo Swarm starts with defining the recommended hardware specifications and provisioning the servers with those recommended components: CPU, RAM, network cards, disk drives, etc.
Step 2: Deploy Caringo Swarm
Next, we install Caringo Swarm in a lab environment with the recommended hardware configuration.
Step 3: Run verification and resilience tests
Running verification tests is the most important task in the Certification process. During this process we:
- Note all the hardware components, including firmware versions
- Review BIOS configuration and make sure it is configured properly (that is, in accordance with Caringo specifications)
- Verify that the server can boot Swarm
- Make sure that all hardware is successfully recognized
- Run it through a variety of hardware failure scenarios. For example:
- Disk removal: We remove a disk drive with the server up and running and replace it with a new one. The new drive must be recognized without having to reboot the server.
- Network failure and recovery: Having at least two ports per server connected to the network, we disable one of them in the switch while uploading a set of files, and then re-enable the port. The service must not be interrupted at any time.
- Storage node failure and recovery: We cause an immediate shutdown (non-graceful) of one of the servers in the cluster. It should not cause any downtime of the service.
- Perform resilience tests: Several runs of 24 hours upload/download/delete tests with different file sizes are conducted. We make sure no issues are found.
Step 4: Submit results to the hardware vendor
After testing the hardware, the results are shared with the hardware vendor and the certification is issued.
What happens after the hardware is certified?
Once the hardware is certified with Caringo Software, we make sure to keep up with any updates in the hardware configuration, e.g., a new component becomes available. Certification/validation processes are repeated as appropriate and the hardware certification is updated as appropriate.
When Does Dense Make Sense?
You can learn more by tuning in to the next Brews & Bytes webcast, Storing Data: When Does Dense Make Sense?
Adrian J Herrera, Caringo VP Marketing, will be joined by Eric Dey, Caringo Head of Product, and Scott Hamilton, WD Sr. Director of Product Management and Marketing. They will discuss:
- Trends in HDD density and recent advancements
- The movement to store everything online
- When it makes sense to go dense given the availability of 14 and 18 TB HDDs