The use of containers via Docker and Kubernetes has grown rapidly over the past several years. Although the roots of the technology have been around a lot longer, it got its big boost in the mid-2010s when Docker, Inc. made the building, sharing and running of images very developer friendly.
<h2>How Did Docker Make Containers Developer Friendly?</h2>
Docker did a lot to democratize the deployment of server software. Rather than having to requisition or use a shared Linux server to try an application, users could run the application on their laptop, very much the way it would be run in QA and production. Instead of having to find packages for a specific flavor and version of Linux or their desktop OS, users could simply bring up a predefined and configured container. More and more applications published images on the hub at <a href="https://hub.docker.com" target="_blank" rel="noopener noreferrer">Docker.com</a> and provided docker-compose.yml files for easy deployment of related services.
<h2>How Does Use of Containers Improve Collaboration?</h2>
Getting developers, QA, support, and customers to configure and run things roughly the same way and to recognize their shared needs and tasks is key to developing successful software and reducing the frustration of “it works for me, why doesn’t it work for you?”
<h2>What are Containers?</h2>
There’s no real magic to containers—the container images are just the tarballs of a Linux OS and the application files. Running containers are just Linux processes that have a limited view of the world.
<h2>Can Containers Solve the “Golden Image” Problem?</h2>
In some ways, containers are like lighter-weight virtual machines, but they try to solve the golden image problem (referring to server software changes or upgrades that are difficult and become unmanageable). Use of Containers solves these issues by providing a simple and standard way to build images (Dockerfile) and to share them (Docker Hub or your own registry). Running containers is more efficient as they are sharing the same Linux kernel rather than running a separate copy in each.
It could be argued that containers are a declaration of surrender to package management and dependency/DLL hell. Rather than manage dependencies across a lot of unrelated software on a general Linux server, you deploy isolated images. Rather than upgrade software on your servers you deploy new container images, preserving only data volumes and configuration files.
<h2>The IT Cattle Call: Pushing to Immutable Infrastructure</h2>
This is all part of the general IT industry push towards immutable infrastructure for reliability and more regular software rollouts. It is a continuation of the movement towards treating servers as cattle, replaceable and scaled out, not pets named and cared for individually.
Although containers can use data volumes (that is, a file system directory that persists outside the lifetime of a container), it is not ideal. Preferably, containers only persist data using services like a database or object storage. Being stateless greatly simplifies container management and deployments.
<a href="https://www.caringo.com/webinars/cloud-seeding-object-storage-containers-tech-tuesday-webinar"><img src="https://i2.wp.com/www.caringo.com/wp-content/uploads/2020/05/cloud-seeding-object-storage-containers-tech-tuesday-webinar.png?resize=640%2C360&ssl=1" alt="cloud seeding with object storage containers" width="640" height="360" class="aligncenter wp-image-20024 size-full" srcset="https://i2.wp.com/www.caringo.com/wp-content/uploads/2020/05/cloud-seeding-object-storage-containers-tech-tuesday-webinar.png?w=640&ssl=1 640w, https://i2.wp.com/www.caringo.com/wp-content/uploads/2020/05/cloud-seeding-object-storage-containers-tech-tuesday-webinar.png?resize=480%2C270&ssl=1 480w, https://i2.wp.com/www.caringo.com/wp-content/uploads/2020/05/cloud-seeding-object-storage-containers-tech-tuesday-webinar.png?resize=320%2C180&ssl=1 320w" sizes="(max-width: 640px) 100vw, 640px" data-recalc-dims="1" /></a>
<h2>Cloud Seeding with Object Storage Containers</h2>
When you integrate the use of containers with object storage, it expands on the benefits of both technologies while removing the requirement for dedicated servers. This enables potential new use cases for object storage such as:
<ul>
<li>Mobile video (e.g., sportscasting and video editing on site without a server rack)</li>
<li>Mobile army hospitals and regional health systems (such as ambulances)</li>
<li>Workflows in public clouds (such as Azure, Amazon and Wasabi)</li>
<li>Complex needs such as local storage for telemetry with async feed to the hub</li>
<li>Edge clusters for IoT</li>
</ul>
On June 23, I will join Ryan Meek, Caringo Principal Solutions Architect, on our Tech Tuesday webinar: <a href="https://www.caringo.com/webinars/cloud-seeding-object-storage-containers-tech-tuesday-webinar">Cloud Seeding with Object Storage Containers</a>. We will show you how easy it is to bring up a containerized, auto-configured <a href="https://www.caringo.com/products/swarm">Swarm Object Storage</a> environment (even on a laptop) and discuss how containers can be used for continuous integration testing and deployment, all-in-one laptop environments, demos, customer setups or reproductions and production rollouts.
We’ll also show some examples of the power and flexibility of making Caringo Swarm available in your container ecosystem.
Please <a href="https://www.caringo.com/webinars/cloud-seeding-object-storage-containers-tech-tuesday-webinar">join us for the webinar</a> and, if you have any questions or suggestions, feel free to <a href="https://www.caringo.com/company/contact">get in touch</a>.

Using containers can solve the “golden image” problem—pulling you out of DLL hell—and, using object storage to make containers stateless makes running them a breeze.

Jamshid “Jam” Afshar has worked on Caringo Swarm, Gateway and UI Engineering teams. He brings deep expertise in object storage to his work and is currently a Senior Consultant working with many of our large, high-profile customers.

Jam Afshar

Caringo uses Amazon S3 protocol for seamless policy-based tiering of data from primary storage to private Swarm cloud to public cloud.

Amazon S3

Cloud architecture are the components needed for cloud computing. Typically, front-end and back-end platforms, cloud-based delivery and a network.

Cloud Architecture

Object-based storage (e.g., Amazon S3-compliant Caringo Swarm Object Storage) is the basis for all Cloud Storage⏤private, public or hybrid).

Cloud Storage

Caringo revolutionized data storage and changed the economics of storage in 2006 by launching the first Object Storage product in the marketplace. 

Data Storage

Caringo Swarm Object Storage can be used as part of a hybrid cloud solution Amazon S3-compliant cloud services such as Microsoft Azure.

Microsoft Azure

Object storage (object-based storage) is computer data storage architecture that manages data as objects, as opposed to file storage & block storage.

Object Storage

Wasabi

containers

Docker

golden image problem

immutable infrastructure

Containers & Server Software Deployment: The Push to Immutable Infrastructure

<a href="https://www.caringo.com/webinars/elasticsearch-object-storage"><img src="https://i2.wp.com/www.caringo.com/wp-content/uploads/2017/07/elasticsearch-object-storage-webinar.jpg?resize=300%2C169&ssl=1" alt="Elasticsearch and Object Storage" class="alignright wp-image-9308 size-medium" title="Elasticsearch and Object Storage Video Webinar" width="300" height="169" srcset="https://i2.wp.com/www.caringo.com/wp-content/uploads/2017/07/elasticsearch-object-storage-webinar.jpg?resize=300%2C169&ssl=1 300w, https://i2.wp.com/www.caringo.com/wp-content/uploads/2017/07/elasticsearch-object-storage-webinar.jpg?w=445&ssl=1 445w" sizes="(max-width: 300px) 100vw, 300px" data-recalc-dims="1" /></a>As Caringo Swarm Object Storage has evolved, we have continuously added smart functionality that brings value to our customers (check out our <a href="https://www.caringo.com/resources/whitepaper-emergent-behavior" target="_blank" rel="noopener noreferrer">Smarts of the Swarm whitepaper</a>). Among the most helpful for our customers is Elasticsearch—a distributed, RESTful search and analytics engine that can be used with object storage to enhance the effectiveness of metadata searching operations. Elasticsearch provides you with the ability to see what you actually have stored in your object storage and to search for streams based on metadata.
We started using Elasticsearch in its infancy (version 0.90) because we cared about distributed computing and wanted to solve a problem with listing the streams in your Swarm cluster (Swarm had a Content Router for enumerating streams, but it was not realtime and did not have query capability). We evaluated a number of solutions, including Solr, Elasticsearch and traditional SQL databases (which lacked scale). Elasticsearch was the most promising and it passed our rigorous testing standards for speed of writes and searches. It also included an extensive API to use for diagnosing issues in an Elasticsearch cluster. Validating our choice, Elasticsearch has grown in awareness and reach with system administrators far beyond our original expectations.
In Swarm version 8, we improved Swarm feeds to allow multiple feeds. This allowed us to handle an Elasticsearch schema change without downtime—Swarm updates both the current and a new elastic database, you can switch to the new one once indexing is complete. <a href="https://www.caringo.com/blog/swarm-9-cloud-object-storage-platform/" target="_blank" rel="noopener noreferrer">Updated in Swarm version 9</a>, Elasticsearch 2 is a fully integrated analytics engine designed for horizontal scalability and open access to Swarm’s operational metrics, metering, and metadata.
Elasticsearch can also be used with 3rd-party big data tools, like Kibana and Hadoop. Swarm integrates Elasticsearch and extends the Swarm API with commands for querying Swarm objects in terms of their metadata. (Learn more by watching <a href="https://www.caringo.com/webinars/power-metadata-object-storage/" target="_blank" rel="noopener noreferrer">The Power of Metadata webinar</a>.) Through this feature, Swarm indexes object metadata in near real time and lets you perform ad hoc searches on the attributes and metadata of stored objects. Our Content Portal also provides users with an easy-to-use web UI with <a href="https://www.caringo.com/videos/caringo-swarm-searchable-metadata-collections/" target="_blank" rel="noopener noreferrer">Collections for using Elasticsearch for metadata queries</a>.
Swarm builds and maintains your search data (index) through your Search Feed, and it can regenerate the search index should it ever be lost. To minimize unavailability of listings, you can take a snapshot of your Elasticsearch index so that you can restore it for instant disaster recovery. You can even store this snapshot in a Swarm cluster using the Elasticsearch AWS Cloud plugin and Swarm Gateway S3.
Want to learn more? Join me and Caringo VP of Product Tony Barbagallo as we <a href="https://www.caringo.com/webinars/elasticsearch-object-storage/" target="_blank" rel="noopener noreferrer">take an in-depth look at best practices for using Elasticsearch</a> with Caringo Swarm Object Storage.
<a href="https://www.caringo.com/webinars/elasticsearch-object-storage/" target="_blank" rel="noopener noreferrer">Elasticsearch and Object Storage Webinar</a> 
Wed, 7/26 10am PT l 1pm ET
<a href="https://www.caringo.com/webinars/elasticsearch-object-storage/" title="Watch the Elasticsearch and Object Storage Webinar" class="btn btn-caringo caringo-lg" target="_blank" rel="noopener noreferrer">WATCH WEBINAR </a>

Learn how Elasticsearch can be used with Caringo Swarm Cloud & Object Storage Platform to enhance the effectiveness of metadata searching operations.

Elasticsearch is a distributed, multitenant-capable full-text search engine with HTTP web interface; it is fully integrated into Swarm Object Storage.

Elasticsearch

Metadata is "data about data" and can be descriptive, structural, administrative, reference or statistical metadata that is auto- or user-generated.

Metadata

Swarm Storage provides a complete S3-compliant platform for data protection, management, organization & search at massive scale for unstructured data.

The Author

Jam Afshar

Author Related Topics

Stories by Jam

Containers & Server Software Deployment: The Push to Immutable Infrastructure

Elasticsearch & Object Storage: PB-Scale Search Solved