How Do I Set Up a Kubernetes Cluster on AWS EKS?

Kubernetes is an open-source platform. It helps us automate deployment, scaling, and managing containerized apps. AWS EKS, or Elastic Kubernetes Service, is a managed service. It allows us to run Kubernetes on AWS easily. We don’t need to install or operate our own Kubernetes control plane or nodes.

In this article, we will show how to set up a Kubernetes cluster on AWS EKS step by step. We will cover what we need before starting, how to create a cluster using AWS CLI, how to configure kubectl for EKS access, how to deploy our first application, how to scale the EKS cluster, and how to monitor and manage it. We will also talk about security best practices and common use cases for Kubernetes on AWS EKS. This guide will help us use AWS EKS for our containerized apps.

  • How to Set Up a Kubernetes Cluster on AWS EKS Step by Step?
  • What Prerequisites Do I Need for AWS EKS Setup?
  • How Do I Create an EKS Cluster Using AWS CLI?
  • How Do I Configure kubectl for EKS Access?
  • How Do I Deploy My First Application on EKS?
  • What Are Common Use Cases for Kubernetes on AWS EKS?
  • How Do I Scale My EKS Cluster?
  • How to Monitor and Manage Your EKS Cluster?
  • What Are Security Best Practices for AWS EKS?
  • Frequently Asked Questions

If we want to learn the basics of Kubernetes, we can check our article on what is Kubernetes and how it simplifies container management. Also, if we think about using Kubernetes for our apps, we should read our insights on why you should use Kubernetes for your applications.

What Prerequisites Do We Need for AWS EKS Setup?

To set up an Amazon EKS (Elastic Kubernetes Service) cluster, we need to have these prerequisites ready:

  1. AWS Account: We have to create an AWS account if we do not have one. This is important to access the AWS Management Console and use EKS.

  2. AWS CLI: We should install the AWS Command Line Interface (CLI). This helps us manage our AWS services. We can follow the installation guide here.

    # Check installation
aws --version

  3. kubectl: We need to install kubectl. This is the tool to talk with Kubernetes clusters. For how to install it, we can look at the official documentation here.

    # Check installation
kubectl version --client

  4. AWS IAM Authenticator: We need this tool for Kubernetes to work with AWS. We can install it like this:

    # On macOS
brew install aws-iam-authenticator

# On Linux
curl -o aws-iam-authenticator https://s3.us-west-2.amazonaws.com/amazon-eks/1.18.9/2019-12-01/bin/linux/amd64/aws-iam-authenticator
chmod +x ./aws-iam-authenticator
sudo mv ./aws-iam-authenticator /usr/local/bin

  5. IAM Permissions: We must check that our AWS IAM user has permissions to create and manage EKS clusters. A policy like this is needed:

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "eks:*",
        "ec2:*",
        "iam:*",
        "cloudformation:*"
      ],
      "Resource": "*"
    }
  ]
}

  6. VPC Configuration: We need a VPC for our EKS cluster. The VPC must have the right subnets (public and private) and route tables set up correctly.

  7. Node Group: We should prepare a node group setup. This will tell us which EC2 instances will be the worker nodes in our EKS cluster.

When we make sure we have these prerequisites, we will be ready to set up a Kubernetes cluster on AWS EKS. For more information about Kubernetes, we can read this article on what are the key components of a Kubernetes cluster.

How Do We Create an EKS Cluster Using AWS CLI?

We can create an Amazon EKS cluster using the AWS CLI by following these simple steps.

  1. Install the AWS CLI: First, we need to make sure we have the AWS CLI installed. We also need to configure it with the right permissions. You can find the installation guide here.

  2. Create an IAM Role for EKS: Next, we create a role that has the permissions for EKS to manage resources.

    aws iam create-role --role-name eks-cluster-role --assume-role-policy-document file://eks-trust-policy.json

    The eks-trust-policy.json file should look like this:

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

  3. Attach Policies to the Role: Now we attach the necessary policies to the role.

    aws iam attach-role-policy --role-name eks-cluster-role --policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
aws iam attach-role-policy --role-name eks-cluster-role --policy-arn arn:aws:iam::aws:policy/AmazonEKSVPCResourceController

  4. Create a VPC: We can create a VPC with public and private subnets. Or we can use an existing VPC.

    aws cloudformation create-stack --region <region> --stack-name eks-vpc --template-url https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.0/2019-09-01/amazon-eks-vpc.yaml

  5. Create the EKS Cluster: Now we create the EKS cluster.

    aws eks create-cluster --name <cluster-name> --role-arn arn:aws:iam::<account-id>:role/eks-cluster-role --resources-vpc-config subnetIds=<subnet-id-1>,<subnet-id-2>,securityGroupIds=<security-group-id>

    We need to replace <cluster-name>, <account-id>, <subnet-id-1>, <subnet-id-2>, and <security-group-id> with our real values.

  6. Check Cluster Status: We can check the status of our cluster.

    aws eks describe-cluster --name <cluster-name> --query "cluster.status"

    We should wait until the status shows ACTIVE.

  7. Update kubeconfig: Finally, we update the kubeconfig to use the EKS cluster.

    aws eks update-kubeconfig --name <cluster-name>

This way, we can set up our EKS cluster using the AWS CLI. If we want more information on Kubernetes and its benefits, we can read about why we should use Kubernetes for our applications.

How Do I Configure kubectl for EKS Access?

To set up kubectl for your EKS cluster, we can follow these simple steps.

  1. Install AWS CLI: First, we need to have the AWS CLI installed. It should be set up with access to our AWS account.

    aws --version

    If we don’t have it, we can download it from the AWS CLI installation page.

  2. Install kubectl: Next, we need kubectl installed. We can get it from the official Kubernetes website.

    curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl

  3. Update kubeconfig: Now, we use the AWS CLI to update our kubeconfig file with the EKS cluster details. We should replace your-cluster-name and your-region with our actual cluster name and region.

    aws eks --region your-region update-kubeconfig --name your-cluster-name

  4. Verify Configuration: It’s good to check if kubectl is set up right. We want to see if it can talk to our EKS cluster.

    kubectl get svc

    This command should show us the services that run in our EKS cluster.

  5. Set Context: If we have more than one context in our kubeconfig, we need to set the one we want for our EKS cluster.

    kubectl config use-context arn:aws:eks:your-region:your-account-id:cluster/your-cluster-name

By doing these steps, we will configure kubectl to access our EKS cluster. This setup helps us manage our Kubernetes resources on AWS EKS. If we want to learn more about Kubernetes, we can check what are the key components of a Kubernetes cluster.

How Do We Deploy Our First Application on EKS?

To deploy our first application on AWS EKS, we can follow these simple steps:

  1. Create a Deployment YAML File: This file tells about our application. It includes the container image and how many copies we want. Here is an example for a basic NGINX deployment:

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-nginx
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

  2. Apply the Deployment: We use kubectl to create the deployment in our EKS cluster.

    kubectl apply -f nginx-deployment.yaml

  3. Expose the Deployment: We need to make a service to expose our application.

    apiVersion: v1
kind: Service
metadata:
  name: my-nginx-service
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-nginx

    Now we apply the service config:

    kubectl apply -f nginx-service.yaml

  4. Get the Service URL: After a little bit, we can get the external IP address to access our application.

    kubectl get services

  5. Verify Deployment: We can check the status of our pods and services.

    kubectl get pods
kubectl get svc

Now our application should be running on AWS EKS. We can access it using the LoadBalancer’s external IP. If we want to learn more about Kubernetes components, we can check this article on key components of a Kubernetes cluster.

What Are Common Use Cases for Kubernetes on AWS EKS?

We see that many people use Kubernetes on AWS EKS (Elastic Kubernetes Service) for different tasks. It is popular because it can grow easily and lets us manage our apps well. Here are some common use cases:

  1. Microservices Architecture:
    • We can deploy apps as microservices. This lets us scale and manage them separately. EKS helps us to run many microservices together. This makes development faster and operations smoother.
  2. Continuous Integration and Continuous Deployment (CI/CD):
    • EKS works well with CI/CD tools like Jenkins, GitLab CI, and AWS CodePipeline. This helps us to test and deploy our apps automatically.
  3. Multi-Cloud and Hybrid Deployments:
    • We can run Kubernetes clusters on EKS while using other cloud services or on-premises systems. This helps us to use a mix of clouds.
  4. Big Data Processing:
    • EKS can handle big data apps using tools like Apache Spark or Hadoop. It gives us good scaling and resource management.
  5. Machine Learning Workloads:
    • EKS supports machine learning tools like TensorFlow and PyTorch. This makes it easy for data scientists to run training jobs and deploy models.
  6. Serverless Applications:
    • We can use Kubernetes with AWS Lambda to create apps that run on events. This helps us build serverless applications that only run when needed.
  7. Web Applications:
    • We can deploy web apps that can scale automatically. It helps to balance the load when traffic changes.
  8. API Management:
    • We can manage APIs by putting API gateways on EKS. This lets us monitor, secure, and scale our API services well.
  9. Development and Testing Environments:
    • We can quickly set up and take down dev and testing environments using EKS. This gives developers the resources they need without extra costs.
  10. Disaster Recovery:
    • EKS can help in disaster recovery. It can back up and restore app workloads in different AWS regions.

Using Kubernetes on AWS EKS gives us tools to deploy and manage apps at a large scale. This helps us keep our services available and strong. For more details on Kubernetes and its features, we can check what Kubernetes is and how it simplifies container management.

How Do We Scale Our EKS Cluster?

Scaling an Amazon EKS (Elastic Kubernetes Service) cluster is easy. We can do this by changing the number of nodes in our node group or by using Kubernetes tools like Horizontal Pod Autoscaler. Here are the steps to help us scale our EKS cluster well.

Scaling Node Groups

To change the number of nodes in our EKS cluster, we can use the AWS Management Console or AWS CLI. Let’s see how we do it with AWS CLI:

  1. List our node groups:

    aws eks list-nodegroups --cluster-name your-cluster-name

  2. Update the desired capacity of the node group:

    aws eks update-nodegroup-config --cluster-name your-cluster-name --nodegroup-name your-node-group-name --scaling-config desiredSize=new-desired-size

Horizontal Pod Autoscaler (HPA)

We can automatically change the number of pods in our deployments based on CPU usage or other metrics by setting up a Horizontal Pod Autoscaler.

  1. Check if Metrics Server is installed: First, we need to install the Metrics Server if we have not done it yet:

    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

  2. Create the HPA resource: Here is an example of how we create an HPA for a deployment:

    kubectl autoscale deployment your-deployment-name --cpu-percent=50 --min=1 --max=10

Cluster Autoscaler

For more efficiency, we can turn on the Cluster Autoscaler. It will change the size of the EKS node group when pods cannot schedule because of not enough resources.

  1. Install Cluster Autoscaler: We should follow the official instructions to install it for our setup.

  2. Set up IAM roles: We need to make sure our node group has the right IAM roles and permissions so the Cluster Autoscaler can manage scaling.

Manual Pod Scaling

If we need to scale right away, we can manually change our deployments using:

kubectl scale deployment your-deployment-name --replicas=new-replica-count

By using these methods, we can manage the scaling of our EKS cluster to meet our application needs. For more details on the main parts of a Kubernetes cluster, we can check this article.

How to Monitor and Manage Your EKS Cluster?

We need to monitor and manage our Kubernetes cluster on AWS EKS. This is important to keep our application working well and being reliable. Here are the main tools and steps we can use for good monitoring and management:

  1. AWS CloudWatch: We can use Amazon CloudWatch to gather and watch the metrics from our EKS cluster. We can set alarms to alert us if there are any problems.

    aws cloudwatch put-metric-alarm --alarm-name "HighCPUUtilization" \
--metric-name "CPUUtilization" --namespace "AWS/EKS" \
--statistic "Average" --period 300 --threshold 80 \
--comparison-operator "GreaterThanThreshold" --evaluation-periods 1 \
--alarm-actions <SNS_TOPIC_ARN> --dimensions "ClusterName=<YOUR_EKS_CLUSTER_NAME>"

  2. kubectl top: We can use kubectl to check how much resources our nodes and pods are using.

    kubectl top nodes
kubectl top pods --all-namespaces

  3. Kubernetes Dashboard: We can deploy the Kubernetes Dashboard. It gives us a visual view of our cluster’s health and resource use.

    kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml

  4. Prometheus and Grafana: We can set up Prometheus to collect metrics and Grafana to show them. This setup gives us strong monitoring abilities.

    • We install Prometheus using Helm:
    helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/prometheus
    • We install Grafana using Helm:
    helm install grafana grafana/grafana

  5. AWS EKS Best Practices: We should follow AWS best practices for managing EKS. This includes updating our clusters regularly and managing node groups. We also need to use IAM roles for service accounts for better security.

  6. Logging: We can turn on logging for our EKS cluster with Amazon CloudWatch Logs. This helps us capture logs from the Kubernetes control plane.

    aws eks update-cluster-config --name <YOUR_EKS_CLUSTER_NAME> --logging '{"clusterLogging":[{"types":["api","audit","authenticator","scheduler","controllerManager"],"enabled":true}]}'

  7. Health Checks: We must add liveness and readiness probes in our apps. This makes sure they run smoothly and are ready to take traffic.

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: my-app
        image: my-app-image
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

With these tools and steps, we can monitor and manage our EKS cluster well. This helps keep our applications running great. For more about managing Kubernetes, we can read about the key components of a Kubernetes cluster.

What Are Security Best Practices for AWS EKS?

To make our AWS EKS (Elastic Kubernetes Service) cluster safe, we should follow these best practices:

  1. Use IAM Roles for Service Accounts (IRSA): We can assign permissions to Kubernetes service accounts. This way, we reduce the need for sensitive AWS credentials in pod specs.

    apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  namespace: default
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::account-id:role/role-name

  2. Control Access with RBAC: We should use Role-Based Access Control (RBAC) to limit access to Kubernetes resources. This is based on user roles.

    apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: example-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

  3. Network Policies: We need to set up network policies to manage how pods talk to each other. This helps us allow traffic only to the services we need.

    apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-app
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: myapp
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: frontend

  4. Use Private EKS Clusters: We should make our EKS cluster private. This limits outside access. It also keeps the control plane safe from the internet.

  5. Enable Encryption: We have to use AWS Key Management Service (KMS) to encrypt important data when it is stored and when it moves. We also need to enable encryption for etcd and S3 buckets.

  6. Regularly Update Kubernetes: We must keep our EKS cluster and nodes updated. This way, we can get the latest Kubernetes versions with security fixes and new features.

  7. Audit Logs: We should turn on Kubernetes audit logging. This helps us watch API calls and changes in our EKS environment. We can use AWS CloudTrail for checking AWS service calls.

  8. Limit Node Access: We need to use security groups to limit access to the worker nodes. This means allowing only the ports and protocols we need.

  9. Implement Pod Security Standards: We can use Pod Security Policies or OPA Gatekeeper to make sure we follow security rules on pods. For example, we should not allow privileged containers.

  10. Scan Container Images: We should check container images for issues before we deploy them. We can use tools like Amazon ECR, Aqua, or Twistlock.

For more details on Kubernetes security practices, we can check what are the key components of a Kubernetes cluster.

Frequently Asked Questions

What is AWS EKS and why should we use it for Kubernetes?

AWS Elastic Kubernetes Service (EKS) is a service that helps us set up and manage Kubernetes clusters on Amazon Web Services. With AWS EKS, we can easily scale our applications, get updates automatically, and ensure high availability. This lets us focus more on building our applications. For more details, we can check our article on Why Should I Use Kubernetes for My Applications?.

How do we determine the right size for our EKS cluster?

To find the right size for our Kubernetes cluster on AWS EKS, we need to look at what our application needs. We should think about things like CPU and memory. It is good to start with a few nodes and then add more if we need them. AWS gives us tools to track and monitor our resources. This helps us make better choices about scaling. For more information on scaling, we can read our article on What Are the Key Components of a Kubernetes Cluster?.

What are the common components of an EKS cluster?

An EKS cluster has a few important parts. These are the control plane, worker nodes, and the Amazon VPC for networking. The control plane takes care of scheduling and API requests. Worker nodes run our containerized applications. Knowing these parts is important for managing our Kubernetes cluster well. We can learn more in our article on What Are the Key Components of a Kubernetes Cluster?.

How does Kubernetes on AWS EKS differ from Docker Swarm?

Kubernetes on AWS EKS is a strong tool that gives us features like auto-scaling, self-healing, and rolling updates. These features are not in Docker Swarm. Docker Swarm is easier to set up and manage. But Kubernetes gives us more options and is better for complex applications. For a deeper look, we can read our article on How Does Kubernetes Differ from Docker Swarm?.

Do we need to install any software to use AWS EKS?

To manage our EKS cluster, we need to install the AWS Command Line Interface (CLI) and kubectl. These tools help us work with AWS services and manage our Kubernetes cluster easily. For local work, we might want to set up Minikube. We can find how to do this in our article on How Do I Install Minikube for Local Kubernetes Development?.