Although Flash, NAS and SAN may be the current “go-to” solutions for storage in the High-Performance Computing (HPC) world, that storage paradigm has proven to be limited when it comes to dealing with the rapidly growing data sets of the 21st century. As IT Pros search for cost-effective storage that easily scales with their HPC needs, object-based storage solutions are often at the top of their list.
Tuesday, July 17, I’ll be hosting our monthly Tech Tuesday webinar and this month’s topic is focused on Object Storage for HPC. Pat Ray, Caringo Swarm Integration Engineering Lead, will be my featured guest.
As Pat and I have been developing the webinar, we’ve talked a lot about those pain points and how a “cloud approach” for HPC storage helps to alleviate those issues. This approach introduces valuable benefits to a wide range of public and private organizations including educational and research institutions, laboratories, businesses and others faced with handling relentless data growth. You will find a matrix of pain points and how object storage helps address them in the table below:
|Pain Point||How Swarm Helps||Benefits|
|File system limitations (e.g., billions of files, complex directory structures, fixed block sizes etc.)||
|Data integrity concerns (multiple backups/copies usually needed)||
||Continuous built-in data integrity checking and protection delivers multiple “nines” of data durability while optimizing data footprint|
|Inefficient parallel access and storage silos||
|Lack of multi-tenant access and/or quota support (typically, data pools into single tenant apps and databases)||
|Lack of “web-accessible” storage (percolating data to web shares causes overly complex web/app/database tier deployments)||Collapse traditional web/app/database/storage layers into a simplified and streamlined RESTful/web access method||Easily support multi-protocol access (next-gen applications and workflows)|
|Difficulty operating on data subsets locally (object metadata is trapped in a database/uncoupled from the data)||Metadata and data are combined (metadata + data is created/stored as an “object”)||Both data and metadata are managed and protected by the storage system over the object’s lifecycle|
We will discuss all of these pain points as well as considerations when selecting and deploying an object storage solution for an HPC environment Tuesday. Make sure to reserve your seat for the webcast and live Q&A, or register now and you will be notified when the broadcast recording is available on demand.
Hear about our 11 Tech Tuesday webinars and make sure to register for our twelfth, Using NFS with Object Storage on December 11. More Details »
Abstract: This whitepaper presents the results from recent benchmarking of Caringo Swarm object storage on a multi-Terabit converged Ethernet Software-Defined Storage Super Data Cluster deployed by the UK Science and Technology Facilities Council’s (STFC) Scientific … More Details »