ECS: Simplified Communication Between Containers with Service Discovery

Communicating locally

ECS is a managed service from AWS to orchastrate (and scale) tasks in containers.
We launch a number of Services into a Cluster.
Each of these services has a task definition, which, amongst other things, defines one or more container images to be run.
Each container instance within a given task, can communicate with the other containers in the same task locally, for example by using localhost.

This is somewhat similar to how services defined in a docker composefile can communicate with one another over a private network.

However, as all the containers in a Service, are scaled and launched together, this local communication is of limited use.

Seperate Services

In most cases, our application will want to scale independently of the resources it relies on and whilst simple to achieve with docker compose- achieving this in ECS requires a different way of thinking.
In ECS, the Cluster is behaving like a namespace - consisting of multiple, indiviudally configured services; we may have a Service for our application, another for MongoDb, another running elasticSearch and so on.
These services are all configured individually, and can Scale up/down, in and out independently.
Assuming they were in the Same VPC, a Task in service A could talk to a task in service B via the network if they knew the IP addresses of one another.

Service Discovery

In order for Tasks to find each others IP addresses, we need to rely on Service Discovery.
This allows our tasks to address each other by an internal, private domain name, rather than a temporary IP address, which could change whenever deployments or scaling events occured.

To get started with Service Discovery, we need to create a Service Discovery Namespace in route 53, and a Service Discovery Service which uses it. This simple example, will create a private domain for a given VPC,

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  NamespaceName:
    Type: String
    Description: The name that you want to assign to this namespace (the TLD part of the domain)
  VpcId:
    Type: String
    Description: The ID of the VPC
Resources:
  ServiceDiscoveryNamespace:
    Type: AWS::ServiceDiscovery::PrivateDnsNamespace
    Properties:
      Name: !Ref NamespaceName
      Vpc: !Ref VpcId
 
  ServiceDiscoveryService:
  Type: AWS::ServiceDiscovery::Service
  Properties:
    DnsConfig:
      DnsRecords: [{ Type: A, TTL: '10' }]
      NamespaceId: !Ref ServiceDiscoveryNamespace
    HealthCheckCustomConfig:
      FailureThreshold: 1

Then, in our ECS Service Defintions we can specify the following two properties:

ServiceName - this is the “domain” name of our service
ServiceRegistries - A complex type defining where the registery is. Although it functions as an array it can only have one item.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  EcsService:
      Type: AWS::ECS::Service
      Properties:
        Cluster: !Ref MyCluster
        TaskDefinition: !Ref EcsTaskDefinition
        ...
        ServiceName: mongoDb
        ServiceRegistries:
          - RegistryArn: !GetAtt ServiceDiscoveryService.Arn

Now we’ve created our internal domain zone for our VPC, and configured our ECS Service to make itself discoverable on that domain as mongoDball that remains is using it.

Using a discovered service

Is quite straightforward. A Service called mongoDb in the zone prod would be accessible within the VPC under the domain name mongoDb.prod.
In the AWS console there are a couple of places you can see your discovery enabled services.
In Route53 you will see your hosted zoneand all the records within it.
In CloudMap you will see your namespaces listed, with the services within.

Conclusion

Service Discovery is a powerful way of making our containers available to other users of ou network.
While this article only focuses on service discovery for ECS tasks, similar principles apply to making a range of services, for example EC2, discoverable.