Big Data Retention for Structured and Unstructured Data
Managing disparate storage solutions for structured data (databases, log files, text, value based data) and unstructured data (audio, documents, emails, images, video) has always been a challenge for IT Organizations. Enterprises now require both types of information to remain accessible and online for years and sometimes decades with data growth rates rapidly outstripping budgets and resources. It is clear that just archiving on tape or other forms of inaccessible storage or tying up expensive SAN and NAS devices doesn’t work anymore.
Flexible and accessible long-term storage
Caringo® Inc., a global leader in object storage, and RainStor™, a provider of Big Data management software, have partnered to deliver a solution for retaining and accessing unstructured, semi-structured, and structured data in Caringo object storage software powered by Swarm. The solution is ideal for use cases where database data must be stored with file-based data for and remain accessible for long periods of time such as in:
- Financial Services: Store complete customer account information including stock tick, trade data, transactional information, and ATM/check images together for compliance purposes or dispute resolution.
- Healthcare: Store patient medical images as well as patient documents and structured database medical records for years or decades for continued patient care.
- ISVs and Service Providers: Store server logs, customer usage logs and customer files for a variety of cloud service implementations on infrastructure that scales by the terabyte with no-downtime expansion and maintenance.
- Life Science: Store value based research information with file based documents and images in a highly scalable solution with commodity economics.
Footprint reduction with rapid search
RainStor’s Big Data Management database contains patented compression and de-duplication techniques, reducing the data footprint size by 95% or more compared to traditional approaches. Unlike other forms of binary compression, data retained in RainStor can be stored and queried on-demand at a granular level via SQL or standard Business Intelligence tools without the need for re-inflation.
Massively scalable, easy to manage storage with commodity economics
Swarm is a software-based storage solution that installs on any combination of x86 hardware with up to 98% disk utilization. There are no limits in capacity, object count or object size. With integrated automated optimization, data protection and zero-provision expansion, one system administrator can easily manage over 10 PB of storage.
Automated data protection and compliance
Integrated data protection and compliance functionality enable adherence to even the strictest corporate policies or industry regulations. Swarm utilizes configurable metadata driven retention and expiration policies per object and is WORM capable supporting immutable objects with auditable access. RainStor adds to Swarm’s capabilities with granular configurable retention and expiry policies down to the record level for structured
data, thereby allowing flexibility in data retention beyond an object level where appropriate.
RainStor + Caringo
Big Data retention for structured and unstructured data
The performance figures achieved are results of Caringo Swarm’s underlying parallel architecture. Let's describe the infrastructure, methodologies and results achieved. More Details »
While object storage isn’t a panacea, it is an increasingly important storage technology that enables on-demand access for video workflows. More Details »