7 best practices for operating containers

7 best practices for operating containers proposed by Google. These can serve as blueprint for ensuring resilient solutions.

Below are some of the best practices for operating containers proposed by Google. These can serve as blueprint for ensuring resilient solutions.

  1. Use native logging mechanisms or containers
    • JSON logs
    • Log aggregator sidecar pattern
  2. Ensure that containers are stateless and immutable
  3. Avoid privileged containers
  4. Make application easy to monitor
    • Metrics HTTP endpoint
    • Sidecar pattern for monitoring
  5. Expose the health of your application
    • Liveness probe
    • Readiness probe
  6. Avoid running as root
  7. Carefully choose image version

For full details of each line item check this link:

https://cloud.google.com/solutions/best-practices-for-operating-containers

Kubernetes Cheat Sheet

A list of commonly used commands when working with Kubernetes cluster.

Below is a list of common and useful commands for working with kubernetes.

Note: appending <-o wide> to some of the commands below will provide more details.

Get all nodes in a cluster

kubectl get nodes
kubectl get nodes -o wide

Get all services in a cluster

kubectl get services

Get all pods in a cluster

kubectl get pods

Create a resource (Deployment, Service)

kubectl create -f <resource.yaml>

Modify a resource

kubectl apply -f <resourcce.yaml>

Get into a pod with ‘bash’ command

kubectl exec -it <podid> -- bash

Secrets

Get secrets

kubectl get secrets

Create secrets

From string literals:

kubectl create secret generic myunsafesecret --from-literal=password=Password123

From file:

# Create files
echo -n 'admin' > ./username.txt
echo -n '1f2d1e2e67df' > ./password.txt

kubectl create secret generic db-user-pass --from-file=./username.txt --from-file=./password.txt

From yaml resource:

# Content of secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  type: Opaque
stringData:
  username: {{username}}
  password: {{password}}
#####

kubectl create -f ./secret.yaml

Ref: https://kubernetes.io/docs/concepts/configuration/secret/

Storage

Get storage classes

kubectl get sc

Get persistent volumes

kubectl get pv

Get persistent volume claims

kubectl get pvc

Deployment

Get deployments

kubectl get deploy

Get deployment details

kubectl describe deploy <deploymentname>

Get replica sets

kubectl get rs -o wide

Get deployment rollout status

kubectl rollout status deploy <deploymentname>

Get deployment rollout history

kubectl rollout history deploy <deploymentname>

Tip: Adding –record flag to kubctl apply will stick the command to the object, so that it will appear under CHANGE-CAUSE in history

Get details for a particular deployment revision

kubectl rollout history deploy <deploymentname> --revision=<revisionnumber>

Rollback a deployment

kubectl rollout undo deploy <deploymentname>

Tip: It is better to avoid this rollback mechanism, and downgrade the version in the deployment.yaml itself and reposting it with “kubectl apply”. This “imperative” way could cause inconsistent environment, where the version of the running container instance doesn’t match the yaml manifest, and any redeployment could accidentally deploy unwanted version. “Declarative” approach is preferred.

Logging

Get snapshot logs for a particular pod with only one container

kubctl logs <pod name>

Run kubectl logs -h for all options

Ref: https://kubernetes.io/docs/concepts/cluster-administration/logging/

Misc

Run a container directly

kubectl run -i --tty <name> --image=<imagename> <command>
kubectl run -i --tty loader --image=busybox /bin/sh

Note: This command will create a pod and will run the container inside it. if “–replica=<number>” is used, it will also create a replication controller in the cluster and will monitor the pod. This is useful when testing (i.e.: hitting a service and generating load, checking service/pod network, etc)

Summary and References

This post contains a list of commonly used commands when interacting with a Kubernetes cluster. Shout out to @nigelpoulton and his great ACG course: “Kubernetes Deep Dive“. Also, here is a far better Cheat Sheet from original kubernetes team.

AWS vs GCP – Cloud Services Comparison

The following table provides a side-by-side comparison of the various services available on AWS and Google Cloud.

Service Category

Service

AWS

Google Cloud

Compute

IaaS

Amazon Elastic Compute Cloud

Compute Engine

 

PaaS

AWS Elastic Beanstalk

App Engine

 

FaaS

AWS Lambda

Cloud Functions

Containers

CaaS

Amazon Elastic Kubernetes Service, Amazon Elastic Container Service

Google Kubernetes Engine

 

Containers without infrastructure

AWS Fargate

Cloud Run

 

Container registry

Amazon Elastic Container Registry

Container Registry

Networking

Virtual networks

Amazon Virtual Private Cloud

Virtual Private Cloud

 

Load balancer

Elastic Load Balancer

Cloud Load Balancing

 

Dedicated interconnect

AWS Direct Connect

Cloud Interconnect

 

Domains and DNS

Amazon Route 53

Google Domains, Cloud DNS

 

CDN

Amazon CloudFront

Cloud CDN

 

DDoS firewall

AWS Shield, AWS WAF

Google Cloud Armor

Storage

Object storage

Amazon Simple Storage Service

Cloud Storage

 

Block storage

Amazon Elastic Block Store

Persistent Disk

 

Reduced-availability storage

Amazon S3 Standard-Infrequent Access, Amazon S3 One Zone-Infrequent Access

Cloud Storage Nearline and Cloud Storage Coldline

 

Archival storage

Amazon Glacier

Cloud Storage Archive

 

File storage

Amazon Elastic File System

Filestore

 

In-memory data store

Amazon ElastiCache for Redis

Memorystore

Database

RDBMS

Amazon Relational Database Service, Amazon Aurora

Cloud SQLCloud Spanner

 

NoSQL: Key-value

Amazon DynamoDB

FirestoreCloud Bigtable

 

NoSQL: Indexed

Amazon SimpleDB

Firestore

 

In-memory data store

Amazon ElastiCache for Redis

Memorystore

Data analytics

Data warehouse

Amazon Redshift

BigQuery

 

Query service

Amazon Athena

BigQuery

 

Messaging

Amazon Simple Notification Service, Amazon Simple Queueing Service

Pub/Sub

 

Batch data processing

Amazon Elastic MapReduce, AWS Batch

DataprocDataflow

 

Stream data processing

Amazon Kinesis

Dataflow

 

Stream data ingest

Amazon Kinesis

Pub/Sub

 

Workflow orchestration

Amazon Data Pipeline, AWS Glue

Cloud Composer

Management tools

Deployment

AWS CloudFormation

Cloud Deployment Manager

 

Cost management

AWS Budgets

Cost Management

Operations

Monitoring

Amazon CloudWatch

Cloud Monitoring

 

Logging

Amazon CloudWatch Logs

Cloud Logging

 

Audit logging

AWS CloudTrails

Cloud Audit Logs

 

Debugging

AWS X-Ray

Cloud Debugger

 

Performance tracing

AWS X-Ray

Cloud Trace

Security & identity

IAM

Amazon Identity and Access Management

Cloud Identity and Access Management

 

Secret management

AWS Secrets Manager

Secret Manager

 

Encrypted keys

AWS Key Management Service

Cloud Key Management Service

 

Resource monitoring

AWS Config

Cloud Asset Inventory

 

Vulnerability scanning

Amazon Inspector

Web Security Scanner

 

Threat detection

Amazon GuardDuty

Event Threat Detection (beta)

 

Microsoft Active Directory

AWS Directory Service

Managed Service for Microsoft Active Directory

Machine learning

Speech

Amazon Transcribe

Speech-to-Text

 

Vision

Amazon Rekognition

Cloud Vision

 

Natural Language Processing

Amazon Comprehend

Cloud Natural Language API

 

Translation

Amazon Translate

Cloud Translation

 

Conversational interface

Amazon Lex

Dialogflow Enterprise Edition

 

Video intelligence

Amazon Rekognition Video

Video Intelligence API

 

Auto-generated models

Amazon SageMaker Autopilot

AutoML

 

Fully managed ML

Amazon SageMaker

AI Platform

Internet of Things

IoT services

Amazon IoT

Cloud IoT

Reference: https://cloud.google.com/docs/compare/aws

Comparison of Google Cloud Storage Options

Intro

Considering key technical differentiators for storage options is critical when deciding what services to adopt to migrate workloads to cloud providers. Here is a brief and concise comparison of some of the Google Cloud Storage options in terms of technical details and use cases.

Comparison of Storage Options

Technical Details

Cloud Datastore Cloud Bigtable Cloud Storage Cloud SQL Cloud Spanner BigQuery
Type NoSQL document NoSQL wide column Blob storage Relational SQL for OLTP Relational SQL for OLTP Relational SQL for OLAP
Transactions Yes Single-row No Yes Yes No
Complex Queries No No No Yes Yes Yes
Capacity Terabytes+ Petabytes+ Petabytes+ Terabytes Petabytes Petabytes+
Unit Size 1MB/entity ~10MB/cell
~100MB/row
5TB/object Determined by DB engine 10,240 MiB/row 10MB/row

Use Cases

Cloud Datastore Cloud Bigtable Cloud Storage Cloud SQL Cloud Spanner BigQuery
Best For Semi-structured application data, durable key-value data “Flat” data, heavy read/write, events, analytical data Structured and unstructured binary or object data Web frameworks, existing apps Large-scale database apps (> ~2tb) Interactive querying, offline analytics
Use Cases Getting started, App Engine apps AdTech, Financial and IoT data Images, large media files, backups User credentials, customer orders Whenever high I/O, global consistency is needed Data warehousing

Reference: