What happens to the flow when you run the kubectl exec or logs commands?

Introduction

Thank you for clicking through to my arcticle. I've been a DevOps engineer for 2 years in dev-team of 7 engineers.

My name is MINSEOK, LEE, but I use Unchaptered as an alias on the interenet. So, you can call me anythings "MINSEOK, LEE" or "Unchaptered" to ask something.

Topic

List up of communication flows between api-server and kubenretes cluster, references on communication between node and the control plane.

• Communication from node to control plane

• Communication from control plane to node

• Communication from control plane(api-server) to node(kubelet)

• Communication from control plane(api-server) to node, pod and service

• SSH Tunel

• Konnectivity Service

Communication from node to control plane

The kubernetes has Hub and Spoke API Pattern.
The node and pods's entire api usage is completed in api server.
The other components of control plane isn't exposed to outside.

The api server is configured to received remote connection in HTTPS Ports(443) with one or more client authentication.

Hub and Spoke Pattern
The spoke who wants to use the service must be passing through the Hub.

The node must be provisioned by root certificate of cluster to connect api-server with valid credentials.

Use kubelet for client certificate format is the good way. The auto provisioning of kubelet refer to kubelet TLS bootstrap.

When pod to be instance, pod is injected by bearer token looks like root certificates using service account.

The kubernetes service, in default namepsace, is configured :

1. HTTPS Endpoint of API Server

2. IP Address to redirect HTTPS Endpoint, through kube-proxy

As a result, the default operating mode is secured and with can run over untrusted and/or public networks.

Communication from control plane to node

In this situation, kubernetes has 2 kind of basic communication path.

1. Communication from control plane(api-server) to node(kubelet)

2. Communication from control plane(api-server) to node, pod and service

Communication from control plane(api-server) to node(kubelet)

This connection is used for :

1. Get a logs from pods.

2. Connect to running pods use kubectl, generally.

3. Support kubelet's port forwarding.

These connections terminate at the kubelet's HTTP endpoint.
By default, the api server doesn't verify the kubelet's serving certificate.
 

This process make the connections:

1. Subject to main-in-middle attacks

2. Unsafe to run over untrusted and/or public networks.

To verify this connection, use --kubelet-certificate-autority flag to provide the api server with a root certificate bundle to use to verify the kubelet's serving certificate.

If that isn't possible, use SSH Tunneling between the api server and kubelet if required to avoid connecting over an untrusted or public network. Finally, kubelet authentication and/or authroization should be enabled to secure the kubelet api.

Communication from control plane(api-server) to node, pod and service

This connection is not seucre.

• Use HTTP

• Can change HTTP and HTTPS use prefix of node, pod, and service.

• Don't support TLS/SSL Handshake.

SSH Tunel

• Deprecated

• Use Konnectivity (1.18 ~ , beta)

Konnectivity Service

As a replacement to the SSH tunnels, the Konnectivity service provide TCP level proxy for the control plane to cluster communication.

The Konnectivity service consists of two parts:

1. The Konnectivity server in the control plane network

2. The Konnectivity agents in the nodes networks.

The konnectivity agents initiate connections to the Konnectivity server and maintain the network connections. After enabling the Konnectivity service, all control plane to nodes traffic goes through these connections.

• Kubernetes Docs (Docs) | Set up Konnectivity service

Q.1. kubectl exec

The kubectl exec command is used to run commands on a specific container within a Kubernetes cluster. The process is roughly as follows

1. User request
   Specify execution container and command.

2. API server
   The kubectl client send a request to kubernetes api-server.
   This request contains client requests.

3. Authentication and authorization
   The api-server validate authentication and authorization after receiving request.

4. Pass the command
   The api-server send the command into kubelet in target container.

5. Execute the command
   The kubelet execute commnand in target container.

6. Return the result
   The response of command is sent by the kubelet to api server.
   And then returned to the clients.

Q.2. kubectl logs

The kubectl logs command is used to view the logs of a specific container.
This process is as follows:

1. User request
   Same to above.

2. API server
   Same to above.

3. Authentication and authorization
   Same to above.

4. Log request
   Send a collect logs command into kubelet

5. Collect logs
   The kubelet collect log in log files in each container.

6. Return the result
   The logs is sent by the kubelet to client.

Conclusion

Most of the connections in Kubernetes are decoupled.