How to send application logs from docker to cloudwatch

+1 vote

We are using Kubernetes to deploy our application docker images.

We would like to be able to take the application logs and push it to cloudwatch.

The application logs are generated using log4j or log4js depending on language the microservice was built.

What is the right way to do this?

Jul 20, 2018 in Docker by Hannah
• 18,570 points
5,981 views

4 answers to this question.

0 votes

Build containers with the Cloudwatch Agent. To do this you will need a Dockerfile.

Make sure your base container is either Debian or RHEL based because Amazon docs seem to only support these types of distros with the agent.

Execute this when you build the container

curl https://s3.amazonaws.com//aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O

For your IAM policy concerns, Amazons example policy is below; you will need to make sure that your containers have access.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "arn:aws:logs:*:*:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::myawsbucket/*"
            ]
        }
    ]
}

Someone on GitHub has done this already:

FROM ubuntu:latest
MAINTAINER Ryuta Otaki <otaki.ryuta@classmethod.jp>, Sergey Zhukov <sergey@jetbrains.com>
...

RUN apt-get install -q -y python python-pip wget
RUN cd / ; wget https://s3.amazonaws.com/aw
answered Jul 20, 2018 by Kalgi
• 52,360 points
0 votes

You can use fluentd for this purpose

You could use a Helm chart to install Fluentd:

$ helm install --name my-release incubator/fluentd-cloudwatch
answered Oct 25, 2018 by Vedant
0 votes

Follow these steps:

  1. Open CloudWatch Logs in the Management Console.
  2. Create a log group name docker-logs.
  3. Go to IAM and create a role for the use with EC2 named docker-logs and attach the CloudWatchLogsFullAccess policy. Note: do not use the CloudWatchLogsFullAccess policy for production workloads. Restrict access to the specific resource and actions instead.
  4. Launch an EC2 Instance based on the Amazon Linux AMI 2017.03.*, select the IAM role ‘docker-logs’, and attach a security group allowing SSH access.
  5. Log into the EC2 Instance via SSH.
  6. yum install docker to install Docker.
  7. service docker start to start Docker.
  8. Start a container with docker run --log-driver=awslogs --log-opt awslogs-group=docker-logs alpine echo 'a cloudonaut.io example'
  9. Open your CloudWatch Logs group to find your log message
answered Oct 25, 2018 by akaash
+1 vote

There are 3 main steps involved it to it.

  1. Create an IAM role/User
  2. Install CloudAgent
  3. Modify docker-compose file or docker run command

I have referred an article here with steps to send the docker logs to aws cloudwatch.

answered Mar 25, 2019 by RT

Related Questions In Docker

0 votes
1 answer

How to send docker using terraform to aws cloudwatch?

Check if you set all the permissions ...READ MORE

answered Jun 7, 2018 in Docker by DareDev
• 6,890 points
2,320 views
0 votes
1 answer

How to check Docker container application logs?

Try this: docker container logs <container-id> READ MORE

answered Oct 12, 2020 in Docker by Vishwanath
720 views
+4 votes
4 answers

How To Access a Service on Host From a Docker Container?

Adding to kalgi's answer, You can also ...READ MORE

answered Oct 16, 2018 in Docker by lina
• 8,220 points

edited Oct 16, 2018 by lina 32,899 views
0 votes
1 answer

How to run a docker command from inside the container?

You must have come across the /var/run/docker.sock file, ...READ MORE

answered Jun 28, 2018 in Docker by Sophie may
• 10,610 points
3,253 views
0 votes
2 answers
0 votes
1 answer

How to obtain the Docker container's IP address from the host?

This can be done by executing the ...READ MORE

answered Jul 17, 2018 in Docker by Sophie may
• 10,610 points
8,585 views
+2 votes
1 answer
+5 votes
7 answers

Docker swarm vs kubernetes

Swarm is easy handling while kn8 is ...READ MORE

answered Aug 27, 2018 in Docker by Mahesh Ajmeria
3,059 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP