How to send docker using terraform to aws cloudwatch

Question

I'm trying to send container logs to aws cloudwatch using terrfaorm. Here's the ECS role I'm using for IAM:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": ["ecs.amazonaws.com", "ec2.amazonaws.com"]
      },
      "Effect": "Allow"
    }
  ]
}

the ECS service role policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:Describe*",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
        "ec2:Describe*",
        "ec2:AuthorizeSecurityGroupIngress",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

the task definition for docker container contains this for cloudwatch logging

  "logConfiguration": {
    "logDriver": "awslogs",
    "options": {
        "awslogs-group": "awslog-mylogs",
        "awslogs-region": "eu-west-1",
        "awslogs-stream-prefix": "awslogs-mylogs-stream"
    }
  }

(I have the awslog-mylogs log group pre-created via AWS console).

Now, when I run an aws instance without the shown logging config for the container, everything runs(except logs being send to cloudwatch). If I add the logging config, ec2 starts but the container does not start properly. It seems that the docker container bails out. Anything that I can do to get the logs to cloudwatch?

Answer 1

Check if you set all the permissions and also include cloudwatch and ecs service role in policy. To do that jus copy paste your existing ECS service policy.

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "cloudwatch:PutMetricData",
        "ec2:DescribeTags",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:DescribeLogStreams",
        "logs:PutSubscriptionFilter",
        "logs:PutLogEvents"
      ],
      "Resource": [
        "arn:aws:logs:*:*:*"
      ]
    }
  ]
}
EOF

Comments

need to install any agent for ECS-optimised AMI ?

---

@vishwanath Challa

Yes you need to have ECS container agent installed

Here is a documentation by Amazon you can follow this to install

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-install.html

---

No @Vishwanath Challa, you only need an agent when your container instance was not launched using an Amazon ECS optimized AMI. So basically when you're instance isn't launched using the ECS optimized AMI you need to install it manually, that's when you need to use the agent. To install an AMI manually using agent, have a look at this.

---

Installing the Amazon ECS Container Agent

If your container instance was not launched using an Amazon ECS-optimized AMI, you can install the Amazon ECS container agent manually using one of the following procedures.

• For Amazon Linux 2 instances, you can install the agent using the amazon-linux-extras command. For more information, see Installing the Amazon ECS Container Agent on an Amazon Linux 2 EC2 Instance.

• For Amazon Linux AMI instances, you can install the agent using the Amazon YUM repo. For more information, see Installing the Amazon ECS Container Agent on an Amazon Linux AMI EC2 Instance.

• For non-Amazon Linux instances, you can either download the agent from one of the regional S3 buckets or from Docker Hub. If you download from one of the regional S3 buckets, you can optionally verify the validity of the container agent file using the PGP signature. For more information, see Installing the Amazon ECS Container Agent on a non-Amazon Linux EC2 Instance

Note

The Amazon ECS container agent is included in the Amazon ECS-optimized AMIs and does not require installation.

---

@Viswanath Challa 

If your container instance was not launched using an Amazon ECS-optimized AMI, you can install the Amazon ECS container agent manually using one of the following procedures.

This isn't the scenario you stated. So if your instance is launched using an Amazon ECS-optimizes AMI then you dont need an external agent 

and if it is not then according to the documentation you will need to install an external container agent.

Hope this helps!

and thank you for correction, i should have asked the pre-requisites.

---

Yess @Vishwanath Challa, this is the exact documentation even I was talking about... Hope it has helped..