DevOps and Microservices using AWS Services

Introduction

Over the past few years, we have seen more and more organizations of every size trying to adopt DevOps practices. Successfully implementing DevOps practices is not easy. Depending on the company's size, it can take several months or even years. Every company has its own unique history, culture, and way of working, and thus will implement it differently. Today we will look how AWS provides various tools to help us adopting to DevOps easily.

Suppose we want to create an application with microservices architecture and deploy it to servers. We want to implement this using DevOps practices. Let's see all the services AWS provides which will help us in fulfilling these requirements.

AWS CodeStar

AWS CodeStar service enables us to develop, build, and deploy applications on AWS. We can perform all the tasks in any software development like project management, development, integrated issue tracking, deployment, all in one place. We can deploy entire continuous delivery toolchain in minutes. Another perk is that here is no additional charge for using AWS CodeStar. We only pay for AWS resources we use like EC2.

We will use this service to setup our code repository and project management dashboard.

Features:

• Project templates – Readymade templates for popular languages like Java, JavaScript, Python, Ruby, and PHP. You can deploy on EC2, Lambda, Elastic Beanstalk using these templates.
• Team access management – With support of IAM, we can manage developer identities and attach various policies to them as per requirements.
• Hosted Git repository – We don't have to manage code repository as AWS CodeStar is integrated with Github and AWS CodeCommit.
• Automated continuous delivery pipeline – Faster release cycle. Each project comes pre-configured with an automated pipeline that continuously builds, tests, and deploys your code with each commit.
• IDE integrations – AWS CodeStar provides integration with Cloud9 (online ide), Visual Studio and Eclipse. So cloud development becomes easy.
• Central project dashboard – We can easily track and manage our end-to-end development toolchain. We can monitor whole project workflow using this dashoard.

Shortcomings:

• Locked to other AWS services – Sadly AWS CodeStar is tightly integrade with other AWS services like CodeDeploy, CodeCommit etc. Also, we cannot use other hosted git repositories like gitlab.
• Limited language and framework templates – Though most of the popular language like Java and Python are supported, if you want to develop your application in other less popular languages like Golang and Elixir, support for it is not supported.

AWS EC2 System Manager

AWS EC2 System Manager is a service which enables visibility and control of whole infrastructure from single point. With Systems Manager, you can group resources, like Amazon EC2 instances, Amazon S3 buckets, by application, view operational data for monitoring and troubleshooting, and act on your groups of resources. We can run single command on all grouped instances without having to take SSH and run command on each instance one by one.

In our case, we will need this to patch our production instance time to time without need to manually handle all the instances.

Features:

• Resource Groups – Instead of performing operations on resources one by one, we can use system manager to group required resources and then execute actions like Run Command, Patch Manager, etc.
• Insights Dashboards – System Manager provides a single dashboard for operation data analysis. Otherwise we have to maintain custom dashboard for each operation and handle data ingestion.
• State Manager – With System Manager we can periodically run an operation on EC2 or on-premise instances. We can define policies through console or can directly use Ansible playbooks which can be stored on GitHub or S3 buckets.

Shortcomings:

• It's good to run simple commands on your EC2 but can become tedious when you need to do some heavy lifting work without taking SSH access.

AWS Config

AWS Config continuously monitors and records your AWS resource configurations. We can track relationships and resource dependencies among resources. We can define competency rules in Config for AWS resources to validate if they meet required criteria.

We will use this service to monitor whether our production instances have only port 443(https) and if they don't, they will be shown as non-compliant in our Config dashboard.

Features:

• Multi-account, multi-region data aggregation – AWS Config can be used to monitor different accounts across all region at a single place. This is very useful as you don't have to login to different accounts or switch regions to know the status of resource compliance rules we set.
• Configuration history – AWS Config records changes to resources and provides us the configuration history. We can know the exact state of our environment at any point in time.
• Cloud governance dashboard – AWS Config provides a visual dashboard to spot non-compliant resources and then we can take appropriate action on them. This is very useful for IT Administrators, Security Experts, and Compliance Officers.

AWS X-Ray

AWS X-Ray does what its name sounds. It is used as tracing service which help analyze and debug production applications. It is useful in our microservices architecture pattern as X-Ray provides an end-to-end view of requests as they travel through your application. So it is helpful for us as we can trace at exactly what component our request is failing.

Features:

• Easy to setup – We only need to install X-Ray agent in the server and integrate the X-Ray SDK with your application.
• AWS services and database Integrations – X-Ray can also be used to capture request metadata for requests made to MySQL or PostgreSQL. It can also be used with AWS SQS and SNS services.
• Service map – AWS X-Ray creates a map of services used by our integrated application with trace data. We get a visual view of all the connections between services in the application and aggregated data for each service like average latency and failure rates.

Shortcomings:

• Currently it supports popular but languages Java, Go, Node.js, Python, Ruby, .NET. If you are using any other language for development, then you can't use X-Ray.