Deploying Jenkins Docker Image using Amazon EC2 Container and Registry Services

It could be any containerized application but I picked Jenkins as it is one of the popular tools to provide continuous integration services for software development. As we all know, it is a server-based system running in a servlet container such as Apache Tomcat. If you don’t use AWS CodePipeline, Jenkins is a one of the popular choices.

Amazon EC2 Container Service, ECS in short, is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances. (More on ECS Product page). You can also read this blog post to have a better understanding on ECS.

My goal in this article is to make deploying jenkins automated and repeatable by using ECS, ECR and CodeCommit.

This is what we will do:

Create an EC2 Instance as a Development Docker Box

Instead of creating an IAM user which we can use on our own computer to access AWS APIs, we will create an IAM role for EC2 instance and use that instance as our development box. This way only secret you keep on your computer is the private ssh key. You can find more about IAM Roles for Amazon EC2 here.

Create Policy for the EC2 Role

We will create a role with enough privileges to run the API calls for the services we will use.

1- Go to IAM / Policies / Create Policy

2- Select Create Your Own Policy as seen in the picture below.

Policy Step 2

3- Copy the policy as seen below on the screenshot and paste in to the editor. Name the policy docker-dev-policy

Policy Step 3

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1452528620000",
            "Effect": "Allow",
            "Action": [
                "cloudformation:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1452528644000",
            "Effect": "Allow",
            "Action": [
                "codecommit:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1452528674000",
            "Effect": "Allow",
            "Action": [
                "ec2:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "cloudwatch:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:*",
            "Resource": "*"
        },
        {
            "Sid": "Stmt1452528689000",
            "Effect": "Allow",
            "Action": [
                "ecr:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1452528701000",
            "Effect": "Allow",
            "Action": [
                "ecs:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1452528728000",
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1452616646000",
            "Effect": "Allow",
            "Action": [
                "iam:AttachRolePolicy",
                "iam:PassRole"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

4- Click Create Policy and you will see the confirmation box. Also if you search for docker you will see the policy we just created.

Policy Step 4

Create IAM Role for EC2 Instance

5- On IAM console (where you are at the moment), click Roles and then Create New Role. Name it docker-dev-ec2 click Next Step.

Policy Step 4

6- Select Amazon EC2

Policy Step 4

7- Search for docker to find the policy we created, as we will attach it to the EC2 IAM Role. Select docker-dev-policy in the search results and click Next Step.

Policy Step 4

8- Review the role and click Create Role

Policy Step 4

9- If you search for docker you should be able to find docker-dev-ec2 role we just created.

Policy Step 4

Create EC2 Instance

We will use this EC2 instance as our development box, it will provide full access to the services as we defined earlier with the policy.

10- Go to EC2 Console and click Launch Instance and select Amazon Linux AMI. (If you choose antoher AMI, commands we will use in the following steps to install docker may differ.)

Policy Step 4

11- Select t2.large or anything you see fit. Click Next. (While choosing insance type, consider we will build docker images and run containers on this instance. In any case, you can easily change instance type later on with a reboot.)

Policy Step 4

12- Choose docker-dev-ec2 role in the IAM Role section. Rest can stay same, or you may put the instance in a specific netork if you like. Click Next.

Policy Step 4

13- Add some storage as you may end up creating and keeping many docker images. When done, click Next.

Policy Step 4

14- Tag the instance as Docker-Dev. Click Next.

Policy Step 4

PS:Docer-dev in the screenshot above should be Docker-dev

15- Create a new security group, give it a name and also add http below ssh. Click Next. (Here we allow 0.0.0.0/0 for ssh access which is not a good idea in a production environment even though we have ssh key auth enabled.)

Policy Step 4

16- On the review screen, click launch.

Policy Step 4

17- The popup will ask for the key pair. Here I choose to create a key specific to this instance. You may do the same, or use an existing one. If you create a new one, after you download it, you will be able to click Launch Instance.

Policy Step 4

After a few minutes, we will have the new Docker-dev instance.

Verify IAM Role for EC2 Instance

We will check if the role actually works on EC2 instance by logging in and running aws cli.

18- Change key file’s permissions

$ chmod 600 docker-dev.pem 

19- Login to EC2 instance. You can find your instance’s DNS name by searching for Docker in EC2 Console.

Policy Step 4

$ ssh ec2-user@ec2-52-90-41-115.compute-1.amazonaws.com -i docker-dev.pem 

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/
11 package(s) needed for security, out of 27 available
Run "sudo yum update" to apply all updates.

20- Doing the updates is a good idea before movig forward.

[ec2-user@~]$ sudo yum update -Y
...
..
Complete!

21- Let’s check if AWS CLI will be able to return instance status of our new docker instance.

For the command below to run, you need to get the Instance ID of the docker instance from EC2 Console.

[ec2-user@~]$ aws ec2 describe-instance-status --instance-ids i-66c78ee7 --region us-east-1
{
    "InstanceStatuses": [
        {
            "InstanceId": "i-66c78ee7", 
            "InstanceState": {
                "Code": 16, 
                "Name": "running"
            }, 
            "AvailabilityZone": "us-east-1c", 
            "SystemStatus": {
                "Status": "ok", 
                "Details": [
                    {
                        "Status": "passed", 
                        "Name": "reachability"
                    }
                ]
            }, 
            "InstanceStatus": {
                "Status": "ok", 
                "Details": [
                    {
                        "Status": "passed", 
                        "Name": "reachability"
                    }
                ]
            }
        }
    ]
}

This output proves that Docker-dev EC2 instance has the role assigned properly and is able to call AWS API’s we mentioned in the policy.

Back To Top

Create CodeCommit Repository

Now, as good developers | engineers, we will start keeping everything under a CodeCommit code repository. You can find more about CodeCommit on the product page.

Since we had given CodeCommit rights to the IAM role we assigned to docker instance, we will use AWS CLI for the rest of the document as much as possible.

22- Create a private code repository on CodeCommit.

[ec2-user@~]$ aws codecommit create-repository --repository-name JenkinsDemo --region us-east-1
{
    "repositoryMetadata": {
        "repositoryName": "JenkinsDemo", 
        "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/JenkinsDemo", 
        "lastModifiedDate": 1452540179.276, 
        "repositoryId": "11b9cd91-a514-4b57-82b7-851f56db560b", 
        "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/JenkinsDemo", 
        "creationDate": 1452540179.276, 
        "Arn": "arn:aws:codecommit:us-east-1:<your-aws-account-id>:JenkinsDemo", 
        "accountId": "<your-aws-account-id>"
    }
}

23- Install Git Client and Clone Repo

[ec2-user@~]$ sudo yum install git -y
...
..
Complete!

24- Setup Credential Helper

The credential helper will use the default AWS credential profile or the Amazon EC2 instance role. More info is here.

[ec2-user@~]$ mkdir source; cd source
[ec2-user@source]$ git config --global credential.helper '!aws codecommit credential-helper $@'
[ec2-user@source]$ git config --global credential.UseHttpPath true

25- Clone your repo to ~/source folder

[ec2-user@source]$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/JenkinsDemo
Cloning into 'JenkinsDemo'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.

26- Configure username and email

[ec2-user@source]$ cd JenkinsDemo
[ec2-user@JenkinsDemo]$ git config --local user.name "Oz Akan"
[ec2-user@JenkinsDemo]$ git config --local user.email oz@cloudway.io

27- Check-in for commit

[ec2-user@JenkinsDemo]$ echo "Notes" > README.md
[ec2-user@JenkinsDemo]$ git add README.md
[ec2-user@JenkinsDemo]$ git commit -m "first commit"
[ec2-user@JenkinsDemo]$ git push origin master

At this point we have a file named README.md locally and also on CodeCommit repository. If you check the repo on CodeCommit Console, you would see the file is listed like below;

Codecommit

Back To Top

Build Jenkins Docker Image

We will have to install docker on the EC2 instance first.

Install Docker

28- Install docker on Amazon Linux.

[ec2-user@JenkinsDemo]$ cd
[ec2-user@~]$ sudo yum install -y docker
...
..
Complete!

29- Start docker service.

[ec2-user@~]$ sudo service docker start
Starting cgconfig service:                                 [  OK  ]
Starting docker:        .                                  [  OK  ]
[ec2-user@~]$ sudo usermod -a -G docker ec2-user

30- Logout and login again to pick up persmissions.

[ec2-user@~]$ docker info
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: devicemapper
 Pool Name: docker-202:1-263960-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 53.74 MB
 Data Space Total: 107.4 GB
 Data Space Available: 83.02 GB
 Metadata Space Used: 606.2 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.10-17.31.amzn1.x86_64
Operating System: Amazon Linux AMI 2015.09
CPUs: 2
Total Memory: 7.8 GiB
Name: ip-172-31-52-124
ID: R4HZ:QUPA:CIRX:52I5:S5E7:XLPT:DFL6:DC6Y:VLFC:7CPD:GKFY:FBLI

This output verifies we have successfully installed docker.

Create the docker image

We will use official Jenkins docker image to build ours. Offical registry is here.

31- Create the folder structure as seen below and create Dockerfile for jenkins image.

[ec2-user@jenkins]$ pwd
/home/ec2-user/source/JenkinsDemo/docker/jenkins
[ec2-user@jenkins]$ touch Dockerfile

32- Pull jenkins docker image from docker hub.

[ec2-user@jenkins]$ docker pull jenkins
...
..
Status: Downloaded newer image for jenkins:latest
[ec2-user@jenkins]$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
jenkins             latest              fc39417bd5fb        3 days ago          708.1 MB

33- Create a named storage container to have persistent data. /var/jenkins_home is the home folder for jenkis in offical jenkins docker image.

[ec2-user@jenkins]$ docker create -v /var/jenkins_home --name jenkins-data-volume jenkins
5c64e21f549bd67df0e076e92059db68753c915ef2f31f519231e2aae01f3177

34- Create Dockerfile to include plugin requirements into our own image.

/home/ec2-user/source/JenkinsDemo/docker/jenkins/Dockerfile looks like this:

FROM jenkins
MAINTAINER Oz Akan <oz@akan.me>

COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt

35- Create plugins.txt file with the plugins you want to have installed on jenkins.

/home/ec2-user/source/JenkinsDemo/docker/jenkins/plugins.txt looks like this:

dockerhub:1.0
github-api:1.71
bitbucket:1.1.4
git:2.4.1
ssh-slaves:1.10

36- Build the image. I used cloudwayio/jenkins as identifier, you are free to use anything as long as you will be persistent for the rest of the documentation.

[ec2-user@jenkins]$ docker build -t cloudwayio/jenkins .
Sending build context to Docker daemon 3.072 kB
Step 1 : FROM jenkins
 ---> fc39417bd5fb
Step 2 : MAINTAINER Oz Akan <oz@akan.me>
 ---> Running in e9c52c9f2ed5
 ---> ad6dfd38e072
Removing intermediate container e9c52c9f2ed5
Step 3 : COPY plugins.txt /usr/share/jenkins/plugins.txt
 ---> 1c2382dd21cb
Removing intermediate container bf65d3d8a30b
Step 4 : RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
 ---> Running in d15a15f85df6
Downloading dockerhub:1.0
Downloading github-api:1.71
Downloading bitbucket:1.1.4
Downloading git:2.4.1
Downloading ssh-slaves:1.10
 ---> 7c83e18fb01a
Removing intermediate container d15a15f85df6
Successfully built 7c83e18fb01a

When you check images, you will see cloudwayio/jenkins image there.

[ec2-user@jenkins]$ docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
cloudwayio/jenkins   latest              7c83e18fb01a        11 seconds ago      711.6 MB
jenkins              latest              fc39417bd5fb        3 days ago          708.1 MB

37- At this point we have a storage container and a jenkins image. Let’s start our jenkins container. With -p 80:8080 we are binding host’s port 80 to containers port 8080.

[ec2-user@jenkins]$ docker run -d -p 80:8080 --volumes-from jenkins-data-volume \
--name ecs-jenkins cloudwayio/jenkins
5558be3aa92ffe16644f51794915b559e532ccf73b6a74455daafeb7ec2c1556

We can check the running containers

[ec2-user@jenkins]$ docker ps -a
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                             NAMES
5558be3aa92f        cloudwayio/jenkins   "/bin/tini -- /usr/lo"   38 seconds ago      Up 38 seconds       50000/tcp, 0.0.0.0:80->8080/tcp   ecs-jenkins
5c64e21f549b        jenkins              "/bin/tini -- /usr/lo"   20 minutes ago      Created                                               jenkins-data-volume

Also if we go to the DNS name of the docker EC2 instance, we will see Jenkins running on port 80

Jenkins

Checkin the code to CodeCommit Repository

38- At this point we have Dockerfile and plugins.txt files; we may want to change these in time or if nothing, we would like to keep a copy. So let’s check them in to CodeCommit repository we created earlier.

[ec2-user@JenkinsDemo]$ pwd
/home/ec2-user/source/JenkinsDemo
[ec2-user@JenkinsDemo]$ git add -A
[ec2-user@JenkinsDemo]$ git commit -m "docker configuration"
[master 396447d] docker configuration
 2 files changed, 10 insertions(+)
 create mode 100644 docker/jenkins/Dockerfile
 create mode 100644 docker/jenkins/plugins.txt
[ec2-user@JenkinsDemo]$ git push origin master
Counting objects: 6, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 575 bytes | 0 bytes/s, done.
Total 6 (delta 0), reused 0 (delta 0)
remote: 
To https://git-codecommit.us-east-1.amazonaws.com/v1/repos/JenkinsDemo
   f084118..396447d  master -> master

Out of curiosity, we can check if files are really synced by checking CodeCommit console.

Jenkins

At this point we are ready to start working on ECS.

Back To Top

Create A (Private) ECS Cotainer Registry

In AWS terms, docker registry is called repository. You can find more information about Amazon EC2 Container Registry on the product page.

39- We will create a repository to keep our jenkins docker image.

[ec2-user@JenkinsDemo]$ aws ecr create-repository \
--repository-name jenkins-demo --region us-east-1
{
    "repository": {
        "registryId": "<your-aws-account-id>", 
        "repositoryName": "jenkins-demo", 
        "repositoryArn": "arn:aws:ecr:us-east-1:<your-aws-account-id>:repository/jenkins-demo"
    }
}

PS: if you get an error, it might be because of the version of aws-cli installed on the EC2 instance. As of this writing aws-cli/1.9.17 has support for ECR. pip install --upgrade awscli would do the trick.

We are almost ready to upload our docker image.

40- We need to authenticate docker to ECR regisry with get-login

[ec2-user@JenkinsDemo]$ aws ecr get-login --region us-east-1
docker login -u AWS -p CiBwm0YaISJeRtJm5n1G6uqeekXuoXXPe5UFce9Rq8/14xLaCAEBAgB4cJtGGiEiXkbSZuZ9RurqnnpF7qF1z3uVBXHvUavP9eMAAAQxMIIELQYJKoZIhvcNAQcGoIIEHjCCBBoCAQAwggQTBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDEKL0liBiC0OfkkNDQIBEICCA+QF1SyY3tNhgE3gNDCniIMnF/EvNFoQi/SgtOsDtcf5wRK9NgOTc3GrxRLnK19vxsFa92VFbyiWylEnGDlkNJWLwJrt5KENgApZFIiejAF6jxqa0Rp8mF7DD78w4utt9ydTn2uIGqMrm9sw4Ig3NAg20c3jmSt3Bmy2cOOPQk0IPrE3lQlxwKAhZV2tBWWAhTT1cxnlOMGrLHiqFtTF8vhIWsMx9vNi9lCgeiMUvkeD89LBu8mxVCTkMm97RHT+vEgykJuls/40sjfiV/yewUvztXpyaKx8JayCgBOVR7iTHviOZW7eK85AQXDm26Isx64+ypp6zs+6Ub7xV3AXvjtSnygm/mrWq7EJ+ZkKqycw9Dp7N1733ZMf7BB530qUQiDahK5RQYbebeEHu2ZjitacpoyyyMRaYLriZXez8ZnpEUVgfcgzLr/OuLeQYE9T9EgiqNTFJ9oiVondbpxv4PKTpzOl5ZuHW53t4Nn7t+rMU2V2wvfbgaCsFYWTF3F11IfziF0SmB1TYESBrmgjeiacKC+XnzSdEIXKFI8IqqHxI0u8QLxZAuYKXdw4R+El7513RaoSAHwU2FO4uzemCkBcBEO3zNPKEsTT0Vui4iSpvSmO3UWB2nhOJaZTBnxYL26Sz4wwuqDUVrmntJ5LyTDqFswaCTBh55j0noIOItK0ha1l9h22zfAdiDS1ub2QMADiTZU7KpZQdUKop5xsZpaVZUZSu7Kf7+pCkA381McMMxzU5kJgGxbArEsUdyBwyipbz5LsDjmB88Y6AHT8NBPGjoR6DbGAuZ4lgAscxb0xGojfXN1Y2ztmByQkEj5uAyiRKAbaAhv++rKO9v6m7OKVNmEr1y0bfNrRbwp+khle8oo6sWr7Ns+gCgc1+CaFjC//Z8Ui2q+N+s82vVJshtGMhmj4VCNnhuMh0+1lAceMumW0hMwWfMnbnm8X7LU4PSSHnglBcIx8jcPUgj0cbNOvozy3TqZc8WEH7tVl6jcx15FaKNuH7aNRnwMlp6KuSuAdDRHjSSOyQazmWRBzHB4flu0zeUWAYiBfyl5x28UPkMYzX3IonH6F4R13eVP3cO3i3gteVfRNfKpi4PXVFgxx5XhJdmX7/bclXjhRok/UMLzlvDuO0pHauo+VHH0HYDCqibtxWRklRGx45+OJWw92nIp4q/QTt9p+sxS8QTpKtXpcK2K50M/77UVpj3mW7hhfvxMfS6Cqq7crce5UmQbcSDY+fvoIrn+jul9RPa8iPgZGHOII8yo6+yPtdNkUoDp7EHVA4cnmmzKa0T1nSwKQbHQvvBbRy7Rl22yUFbwi2MXIhnA= -e none https://<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com

Run run the output above.

Now, docker can push/pull images to ECR registry.

41- If you still have jenkins container running, let’s stop it now.

[ec2-user@JenkinsDemo]$ docker ps -a
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                             NAMES
5558be3aa92f        cloudwayio/jenkins   "/bin/tini -- /usr/lo"   15 hours ago        Up 15 hours         50000/tcp, 0.0.0.0:80->8080/tcp   ecs-jenkins
5c64e21f549b        jenkins              "/bin/tini -- /usr/lo"   15 hours ago        Created                                               jenkins-data-volume
[ec2-user@JenkinsDemo]$ docker stop ecs-jenkins
ecs-jenkin

Finally it’s time to tag and upload the Jenkins docker image we created. We won’t be using the data volume image on ECS.

42- Tag the image; use your account id and region information accordingly while doing so.

[ec2-user@JenkinsDemo]$ docker tag cloudwayio/jenkins:latest \
<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo:latest

Let’s check tagged image.

[ec2-user@JenkinsDemo]$ docker images
REPOSITORY                                                  TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
cloudwayio/jenkins                                          latest              7c83e18fb01a        15 hours ago        711.6 MB
<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo   latest              7c83e18fb01a        15 hours ago        711.6 MB
jenkins                                                     latest              fc39417bd5fb        3 days ago          708.1 MB

43- Upload the image to ECR Registry

[ec2-user@JenkinsDemo]$ docker push <your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo:latest

This command will take a while to upload the image depending on the image size and the bandwidth of the instance/server you are running the command on. Meanwhile it is good to remind some limits of ECR; as of January 2016, you can have 1000 repositories per account and each repository can have 500 images. (reference)

Once upload is completed, you will see latest digest.

The push refers to a repository [<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo] (len: 1)
7c83e18fb01a: Pushed 
1c2382dd21cb: Pushed 
ad6dfd38e072: Pushed 
...
...
5c63804eac90: Pushed 
523ef1d23f22: Pushed 
latest: digest: sha256:b804fb1078bbaaaadf255b29dcf625ce59fa60d915f289b75b5cbb115a1dbb45 size: 81657

44- As usual, let’s check the image

[ec2-user@JenkinsDemo]$ aws ecr list-images --repository-name jenkins-demo --region us-east-1
//output
{
    "imageIds": [
        {
            "imageTag": "latest", 
            "imageDigest": "sha256:b804fb1078bbaaaadf255b29dcf625ce59fa60d915f289b75b5cbb115a1dbb45"
        }
    ]
}

Back To Top

Create ECS (Amazon EC2 Container Service) Cluster

45- We will create a cluster named JenkinsDemoCluster in us-east-1 region.

[ec2-user@JenkinsDemo]$ aws ecs create-cluster --cluster-name JenkinsDemoCluster --region us-east-1
//output
{
    "cluster": {
        "status": "ACTIVE", 
        "clusterName": "JenkinsDemoCluster", 
        "registeredContainerInstancesCount": 0, 
        "pendingTasksCount": 0, 
        "runningTasksCount": 0, 
        "activeServicesCount": 0, 
        "clusterArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:cluster/JenkinsDemoCluster"
    }
}

We must have an ECS container instance in our cluster before we can run tasks on it. We will choose amazon-ecs-optimized AMI (ami-6ff4bd05 for us-east-1) so it will come with Amazon ECS Container Agent. If you choose a different image you will have to manually install the agent on EC2 instance.

If you want to, you can return all ECS optimized images for us-east-1 with the command below:

$ aws ec2 describe-images --filters "Name=description,Values=*ECS*"  --region us-east-1

At the moment we have no container instances running in ECS.

[ec2-user@JenkinsDemo]$  aws ecs list-container-instances --cluster JenkinsDemoCluster --region us-east-1
//outputs
{
    "containerInstanceArns": []
}

So we will create an instance using an ebs-optimized-ami. We will choose the security group we created for our Docker-dev instance. We will choose m4.large; you are free to choose any size just good to have some storage and memory. We will use docker-dev-ec2 instance profile name which is created automatically by console when we created the role with the same name.

46- Create a file named user-data.txt with the content below

#!/bin/bash
echo ECS_CLUSTER=JenkinsDemoCluster >> /etc/ecs/ecs.config

47- We need to find the original block device mapping of ami-6ff4bd05 which is amazon-ecs-optimized.

[ec2-user@JenkinsDemo]$ ec2-describe-images ami-6ff4bd05
IMAGE   ami-6ff4bd05    591542846629/amzn-ami-2015.09.c-amazon-ecs-optimized    591542846629    available       public          x86_64  machine                         ebs /dev/xvda        hvm     xen
BLOCKDEVICEMAPPING      EBS     /dev/xvda               snap-d6395b9b   30      true    gp2             Not Encrypted

48- We will use the values from the output of the command above, except we will change 30 to 80 which indicates the size of volume.

[ec2-user@JenkinsDemo]$ aws ec2 run-instances \
--image-id ami-6ff4bd05 --count 1 \
--instance-type m4.large --key-name docker-dev \
--security-groups  docker-dev-SG --region us-east-1 \
--user-data file://user-data.txt --iam-instance-profile "Name=docker-dev-ec2" \
--block-device-mappings "[{\"DeviceName\":\"/dev/xvda\",\"Ebs\":{\"VolumeSize\":80,\"DeleteOnTermination\":false}}]"
//output
{
    "OwnerId": "<your-aws-account-id>",
    "ReservationId": "r-bfa21b41",
    "Groups": [],
    ...
    ...
        "IamInstanceProfile": {
            "Id": "AIPAJC5TI5ALUAHFYSXPW",
            "Arn": "arn:aws:iam::<your-aws-account-id>:instance-profile/docker-dev-ec2"
        },
        "RootDeviceName": "/dev/xvda",
        "VirtualizationType": "hvm",
        "AmiLaunchIndex": 0
    }
}    

49- Lets check running instances again.

[ec2-user@~]$ aws ecs list-container-instances --cluster JenkinsDemoCluster --region us-east-1
//output
{
    "containerInstanceArns": [
        "arn:aws:ecs:us-east-1:<your-aws-account-id>:container-instance/de47f2db-811c-49be-b69d-eab448dfcbdc"
    ]
}

50- Above de47f2db-811c-49be-b69d-eab448dfcbdc is the instance id and we can use it to get more details about the instance as seen below:

[ec2-user@~]$ aws ecs describe-container-instances --cluster JenkinsDemoCluster \
--region us-east-1 --container-instances de47f2db-811c-49be-b69d-eab448dfcbdc
//output
{
    "failures": [], 
    "containerInstances": [
        {
            "status": "ACTIVE", 
            "registeredResources": [
                {
                    "integerValue": 2048, 
                    "longValue": 0, 
                    "type": "INTEGER", 
                    "name": "CPU", 
                    "doubleValue": 0.0
                }, 
                ...
                ...
            "versionInfo": {
                "agentVersion": "1.7.0",
                "agentHash": "191dbd5",
                "dockerVersion": "DockerVersion: 1.7.1"
            }
        }
    ]
}                    

The output is pretty long so didn’t put all above. You can find running tasks, docker version, EC2 instance id there.

On the ECS Console our instance is displayed as seen below:

Jenkins

Back To Top

Run Jenkins Container on ECS

Docker has the concept of data volume containers as we’ve created one earlier.

Data volume containers, map a volume from the host and reference to it as long as data volume container is not removed. The trick is, you can reference this data volume container from other containers using --volumes-from. In this case you can share the same volume with multiple containers; you can delete all these containers but as long as you don’t remove the data volume container, your data will persist. Since data volume container is there just to reference a volume from host, it doesn’t even have to be running.

ECS has a way of creating a hidden data volume container instance. Data volume container will have an image named amazon/ecs-emptyvolume-base. It will be automatically created as soon as you create a service with an empty volume. It won’t be running. You can login to ECS instance check it out.

Here are the docker images on ECS host:

[ec2-user@ip-172-31-30-233 ~]$ docker images
REPOSITORY                                                       TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
amazon/ecs-emptyvolume-base                                      autogenerated       22dd0110832b        17 hours ago        0 B
<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo        latest              7c83e18fb01a        39 hours ago        711.6 MB

Here are the running containers on ECS host (notice no container for the data volume running):

[ec2-user@ip-172-31-30-233 ~]$ docker ps
CONTAINER ID        IMAGE                                                              COMMAND                CREATED             STATUS              PORTS                                            NAMES
3df37ad47b94        <your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo:latest   "/bin/tini -- /usr/l   20 minutes ago      Up 20 minutes       0.0.0.0:50000->50000/tcp, 0.0.0.0:80->8080/tcp   ecs-j3-task-5-j3-container-92e3b3eff5ebedf6ba01   
b8a36e56a15d        amazon/amazon-ecs-agen

We can also see what volumes ecs-emptyvolume-base has:

[ec2-user@ip-172-31-30-233 ~]$ docker inspect -f {{.Volumes}} d9e122a1caa7
map[/ecs-empty-volume/host_blank:/var/lib/docker/volumes/c9d152c5ddcdd31f4d21212a71738293db1761debab37d5820c454936ecd26e4/_data]

It is good that ECS will create a data container instance behind the scenes but since this container is fully managed, it’s volumes will be removed when service stops. This means data will be not persistent. There might be a use case for this but in our demo for jenkins we want to have persistent data.

Another important note; data will be under the root volume of the ECS host and if we terminate ECS host, it will be gone. Not much ECS can do about it. It would be a good practice to set DeleteOnTermination to false or use another EBS volume to keep persistent data. We could also mount this EBS volume to another host or clone. We can also use S3 to backup data we have in the data volume.

Ok enough reading, let use AWS CLI to run jenkins in a container on ECS.

We already created an ECS cluster named JenkinsDemoCluster. Now, we will register a task definition to run on JenkinsDemoCluster.

51- You can either create a file named task-demo.json with the following AWS CLI command and modify the content.

[ec2-user@JenkinsDemo]$ aws ecs register-task-definition --generate-cli-skeleton > task-demo.json

Or you can just copy & paste the configuration below into task-demo.json. Remember to replace <your-aws-account-id> with your account id.

{
    "family": "jenkins-demo-task",
    "containerDefinitions": [
        {
            "name": "jenkins-demo-container",
            "image": "<your-aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/jenkins-demo:latest",
            "cpu": 512,
            "memory": 2048,
            "portMappings": [
                {   
                    "containerPort": 8080,
                    "hostPort": 80,
                    "protocol": "tcp"
                },
                {   
                    "containerPort": 50000,
                    "hostPort": 50000,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "mountPoints": [
                {   
                    "sourceVolume": "jenkins_home",
                    "containerPath": "/var/jenkins_home",
                    "readOnly": false
                }
            ],
            "privileged": false,
            "user": "root",
            "readonlyRootFilesystem": false
        }
    ],
    "volumes": [
        {   
            "name": "jenkins_home",
            "host": {
                "sourcePath": "/ecs/volumes/jenkins_home"
            }
        }
    ]
}

52- Register the task definition

$ aws ecs register-task-definition --cli-input-json \
file://$HOME/source/JenkinsDemo/task-demo.json --region us-east-1

CLI will return a JSON output with a revision number 1.

"revision": 1

Revision is important as we will use it with task family name (jenkins-demo-task:1) to define a task we will use under the service we will define.

53- We can query task definitions to see the one we just created.

[ec2-user@JenkinsDemo]$ aws ecs list-task-definitions --region us-east-1
//output
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-east-1:<your-aws-account-id>:task-definition/jenkins-demo-task:1"
    ]
}

54- Create a service definition using jenkins-demo-task

We want to create a ECS service which will allow us to run and maintain a specified number of task definitions on a cluster. Since we want one jenkins instance to be up and running all the time, ECS service will ensure that a container is up and if it crashes, it will create another instance.

You can again use the command below to create a template file named service-demo.json and modify it later on.

[ec2-user@JenkinsDemo]$ aws ecs create-service --generate-cli-skeleton > service-demo.json

Or, you can copy past the JSON content below into service-demo.json

{
    "cluster": "JenkinsDemoCluster",
    "serviceName": "jenkins-service",
    "taskDefinition": "jenkins-demo-task:1",
    "desiredCount": 1,
    "clientToken": "random_value_here",
    "deploymentConfiguration": {
        "maximumPercent": 200,
        "minimumHealthyPercent": 50
    }
}

55- Create service.

[ec2-user@JenkinsDemo]$ aws ecs create-service --cli-input-json \
file://$HOME/source/JenkinsDemo/service-demo.json --region us-east-1
//output
{
    "service": {
        "status": "ACTIVE", 
        "taskDefinition": "arn:aws:ecs:us-east-1:<your-aws-account-id>:task-definition/jenkins-demo-task:1", 
        "pendingCount": 0, 
        "loadBalancers": [], 
        "desiredCount": 1, 
        "serviceName": "jenkins-service", 
        "clusterArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:cluster/JenkinsDemoCluster", 
        "serviceArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:service/jenkins-service", 
        "deploymentConfiguration": {
            "maximumPercent": 200, 
            "minimumHealthyPercent": 50
        }, 
        "deployments": [
            {
                "status": "PRIMARY", 
                "pendingCount": 0, 
                "createdAt": 1452707724.255, 
                "desiredCount": 1, 
                "taskDefinition": "arn:aws:ecs:us-east-1:<your-aws-account-id>:task-definition/jenkins-demo-task:1", 
                "updatedAt": 1452707724.255, 
                "id": "ecs-svc/9223370584147051552", 
                "runningCount": 0
            }
        ], 
        "events": [], 
        "runningCount": 0
    }
}

If you have another task running on your cluster mapped to port 80 on the host, you may get the following error:

service jenkins-service was unable to place a task because no container
instance met all of its requirements. The closest matching container-instance
de47f2db-811c-49be-b69d-eab448dfcbdc is already using a port required by your task. 
For more information, see the Troubleshooting section.

This is expected as only one application can allocate a port on the host. In this case you will have to change the port Jenkins container will listen.

As soon as service is created with desiredCount set to 1, it will start the container that we defined in the task.

56- Check the running tasks.

[ec2-user@JenkinsDemo]$ aws ecs list-tasks --cluster JenkinsDemoCluster \
--region us-east-1
//output
{
    "taskArns": [
        "arn:aws:ecs:us-east-1:<your-aws-account-id>:task/946137b2-6899-46a0-a232-ebf89fa55474"
    ]
}

57- Just the id of the task wasn’t very helpful so let’s describe this task.

[ec2-user@JenkinsDemo]$ aws ecs describe-tasks --cluster JenkinsDemoCluster \
--task 946137b2-6899-46a0-a232-ebf89fa55474 --region us-east-1
//output
{
    "failures": [], 
    "tasks": [
        {
            "taskArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:task/946137b2-6899-46a0-a232-ebf89fa55474", 
            "overrides": {
                "containerOverrides": [
                    {
                        "name": "jenkins-demo-container"
                    }
                ]
            }, 
            "lastStatus": "RUNNING", 
            "containerInstanceArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:container-instance/de47f2db-811c-49be-b69d-eab448dfcbdc", 
            "createdAt": 1452711040.511, 
            "clusterArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:cluster/JenkinsDemoCluster", 
            "startedAt": 1452711042.28, 
            "desiredStatus": "RUNNING", 
            "taskDefinitionArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:task-definition/jenkins-demo-task:1", 
            "startedBy": "ecs-svc/9223370584144225434", 
            "containers": [
                {
                    "containerArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:container/2e4d0e8f-e96d-4abe-b1da-1fe3145cfb2c", 
                    "taskArn": "arn:aws:ecs:us-east-1:<your-aws-account-id>:task/946137b2-6899-46a0-a232-ebf89fa55474", 
                    "lastStatus": "RUNNING", 
                    "name": "jenkins-demo-container", 
                    "networkBindings": [
                        {
                            "protocol": "tcp", 
                            "bindIP": "0.0.0.0", 
                            "containerPort": 50000, 
                            "hostPort": 50000
                        }, 
                        {
                            "protocol": "tcp", 
                            "bindIP": "0.0.0.0", 
                            "containerPort": 8080, 
                            "hostPort": 80
                        }
                    ]
                }
            ]
        }
    ]
}

Looking at taskDefinitionArn you can see this task belongs to jenkins-demo-task family and is version 1.

Let’s find out if jenkins is really running.

58- Go to ECS Console and click JenkinsDemoCluster and click to the task id (pointed with the arrow).

ecs console 1

59- Click the IP:PORT (pointed by the arrow).

This IP address belongs to the ECS instance (host) and there can be only one container mapped to port 80 on the host. An ELB would overcome this limitation if we want to run multiple internet facing services on the same ECS instance.

ecs console 2

60- You should see jenkins welcome screen.

jenkins on ecs

Don’t forget to enable security under Configure Global Security settings right away!

Back To Top

Final Notes

In this article, we created an EC2 instance as a development box with an EC2 role that can use various AWS products. We installed docker on this instance and created a Jenkins Docker image. We added the plugins we wanted to Jenkins. Then we created an ECS cluster, a task and a service to ensure task is always running. Finally we were able to see Jenkins web interface.

At this point, deploying new versions of Jenkins on EC2 Container Service is pretty easy. We can create new revisions of our tasks and let ECS deploy it.

  • In production, you would like to have SSL Encryption, so maybe an nginx container infront of jenkins or probably an Elastic Load Balancer with SSL.
  • You also wouldn’t like to assign an EC2 Role to ECS instances with broad access rights, like the one we assigned to Docker-dev instance.

You might also wonder how much of the code we ran above could be automated using CloudFormation.

Let’s make minor changes to secure this environment for production and automate even further in a following post.

Finally, don’t forget to push new files and changes into CodeCommit.

[ec2-user@JenkinsDemo]$ git add -A; git commit -m "ecs demo files"; git push origin master

I wish you a very productive day.

May the force (of automation) with you…

comments powered by Disqus