본문 바로가기
Compute/kubernetis

[CKA] 05.kubelet 복구 | systemctl status kubelet | restart kubelet | enable kubelet

by 조청유곽 2025. 1. 11.
반응형

01. 서비스가 되지 않는 노드를 찾고 서비스를 복구한다. 

 

[precondition]

Node가 서비스되지 않는 상황을 만들기 위해서 node1의 kubelet을 stop한다. 

[root@k8s-node1 ~]# systemctl stop kubelet
[root@k8s-node1 ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: inactive (dead) since Fri 2024-12-27 00:18:24 KST; 6s ago
     Docs: https://kubernetes.io/docs/
  Process: 26449 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (cod>
 Main PID: 26449 (code=exited, status=0/SUCCESS)

Dec 26 22:58:56 k8s-node1 kubelet[26449]: I1226 22:58:56.454439   26449 pod_startup_latency_tracker.go:102] "Observed pod startup durati>
Dec 26 22:58:57 k8s-node1 kubelet[26449]: I1226 22:58:57.069854   26449 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CS>
Dec 26 22:58:57 k8s-node1 kubelet[26449]: I1226 22:58:57.069910   26449 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with n>
Dec 26 23:28:08 k8s-node1 kubelet[26449]: I1226 23:28:08.381251   26449 topology_manager.go:212] "Topology Admit Handler"
Dec 26 23:28:08 k8s-node1 kubelet[26449]: I1226 23:28:08.502832   26449 reconciler_common.go:258] "operationExecutor.VerifyControllerAtt>
Dec 26 23:28:18 k8s-node1 kubelet[26449]: I1226 23:28:18.006206   26449 pod_startup_latency_tracker.go:102] "Observed pod startup durati>
Dec 27 00:18:24 k8s-node1 systemd[1]: Stopping kubelet: The Kubernetes Node Agent...
Dec 27 00:18:24 k8s-node1 kubelet[26449]: I1227 00:18:24.393767   26449 dynamic_cafile_content.go:171] "Shutting down controller" name=">
Dec 27 00:18:24 k8s-node1 systemd[1]: kubelet.service: Succeeded.
Dec 27 00:18:24 k8s-node1 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
lines 1-19/19 (END)

 

 

 

**kubelet 이란? 

kubelet은 각 노드에서 실행되는 기본 "노드 에이전트"이다.

호스트 이름, 호스트 이름을 재정의하는 플래그 또는 클라우드 공급자의 특정 로직 중 하나를 사용하여 노드를 apiserver에 등록할 수 있다. 

 

https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

 

Node Components

Run on every node, maintaining running pods and providing the Kubernetes runtime environment:

kubelet : Ensures that Pods are running, including their containers.

 

 

https://kubernetes.io/docs/concepts/overview/components/

 

 

[solve]

[root@k8s-master ~]# kubectl get nodes
NAME         STATUS     ROLES           AGE    VERSION
k8s-master   Ready      control-plane   108m   v1.27.2
k8s-node1    NotReady   <none>          91m    v1.27.2
k8s-node2    Ready      <none>          91m    v1.27.2
[root@k8s-master ~]# 
[root@k8s-master ~]# ssh k8s-node1
The authenticity of host 'k8s-node1 (192.168.56.31)' can't be established.
ECDSA key fingerprint is SHA256:iWamyd4b88wDh6R5jAMl6q2v2pVbGcWlSNLSvCkyFFY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'k8s-node1,192.168.56.31' (ECDSA) to the list of known hosts.
root@k8s-node1's password: 
Last login: Thu Dec 26 22:55:19 2024 from 192.168.56.1
[root@k8s-node1 ~]# 
[root@k8s-node1 ~]# 
[root@k8s-node1 ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: inactive (dead) since Fri 2024-12-27 00:18:24 KST; 10min ago
     Docs: https://kubernetes.io/docs/
  Process: 26449 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (cod>
 Main PID: 26449 (code=exited, status=0/SUCCESS)

Dec 26 22:58:56 k8s-node1 kubelet[26449]: I1226 22:58:56.454439   26449 pod_startup_latency_tracker.go:102] "Observed pod startup durati>
Dec 26 22:58:57 k8s-node1 kubelet[26449]: I1226 22:58:57.069854   26449 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CS>
Dec 26 22:58:57 k8s-node1 kubelet[26449]: I1226 22:58:57.069910   26449 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with n>
Dec 26 23:28:08 k8s-node1 kubelet[26449]: I1226 23:28:08.381251   26449 topology_manager.go:212] "Topology Admit Handler"
Dec 26 23:28:08 k8s-node1 kubelet[26449]: I1226 23:28:08.502832   26449 reconciler_common.go:258] "operationExecutor.VerifyControllerAtt>
Dec 26 23:28:18 k8s-node1 kubelet[26449]: I1226 23:28:18.006206   26449 pod_startup_latency_tracker.go:102] "Observed pod startup durati>
Dec 27 00:18:24 k8s-node1 systemd[1]: Stopping kubelet: The Kubernetes Node Agent...
Dec 27 00:18:24 k8s-node1 kubelet[26449]: I1227 00:18:24.393767   26449 dynamic_cafile_content.go:171] "Shutting down controller" name=">
Dec 27 00:18:24 k8s-node1 systemd[1]: kubelet.service: Succeeded.
Dec 27 00:18:24 k8s-node1 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.

[root@k8s-node1 ~]# systemctl restart kubelet
[root@k8s-node1 ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Fri 2024-12-27 00:29:37 KST; 10s ago
     Docs: https://kubernetes.io/docs/
 Main PID: 34400 (kubelet)
    Tasks: 11 (limit: 18865)
   Memory: 39.8M
   CGroup: /system.slice/kubelet.service
           └─34400 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.c>

Dec 27 00:29:40 k8s-node1 kubelet[34400]: I1227 00:29:40.229278   34400 reconciler_common.go:300] "Volume detached for volume \"kube-api>
Dec 27 00:29:40 k8s-node1 kubelet[34400]: I1227 00:29:40.229374   34400 reconciler_common.go:300] "Volume detached for volume \"tmp-dir\>
Dec 27 00:29:40 k8s-node1 kubelet[34400]: I1227 00:29:40.745433   34400 scope.go:115] "RemoveContainer" containerID="2906ba6aaa53ddc2c45>
Dec 27 00:29:40 k8s-node1 kubelet[34400]: I1227 00:29:40.757310   34400 scope.go:115] "RemoveContainer" containerID="2906ba6aaa53ddc2c45>
Dec 27 00:29:40 k8s-node1 kubelet[34400]: E1227 00:29:40.758125   34400 remote_runtime.go:415] "ContainerStatus from runtime service fai>
Dec 27 00:29:40 k8s-node1 kubelet[34400]: I1227 00:29:40.758208   34400 pod_container_deletor.go:53] "DeleteContainer returned error" co>
Dec 27 00:29:41 k8s-node1 kubelet[34400]: I1227 00:29:41.678355   34400 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" po>
Dec 27 00:29:41 k8s-node1 kubelet[34400]: I1227 00:29:41.679705   34400 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" po>
Dec 27 00:29:41 k8s-node1 kubelet[34400]: I1227 00:29:41.680571   34400 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" po>
Dec 27 00:29:41 k8s-node1 kubelet[34400]: I1227 00:29:41.681843   34400 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" po>

[root@k8s-node1 ~]# systemctl enable kubelet
[root@k8s-node1 ~]# exit
logout
Connection to k8s-node1 closed.
[root@k8s-master ~]# 
[root@k8s-master ~]# 
[root@k8s-master ~]# kubectl get nodes
NAME         STATUS   ROLES           AGE    VERSION
k8s-master   Ready    control-plane   109m   v1.27.2
k8s-node1    Ready    <none>          92m    v1.27.2
k8s-node2    Ready    <none>          92m    v1.27.2
[root@k8s-master ~]# 
[root@k8s-master ~]#

 

 

반응형
저작자표시 비영리 변경금지

'Compute > kubernetis' 카테고리의 다른 글

[CKA] 07. pod 생성 | kubectl apply -f | yaml로 pod 생성 | manifest로 pod 생성  (0) 2025.01.14
[CKA] 06. kubectl create | kubectl delete | kubectl edit 명령어 사용 문제  (0) 2025.01.11
[CKA] 04. kubectl describe 명령어 사용 문제  (0) 2025.01.11
[CKA] 03. kubectl logs 명령어 사용 문제  (0) 2025.01.11
[CKA] 02. kubectl get 명령어 사용 문제 | kubectl get pods -o name | kubectl get pods -o yaml | jsonpath  (0) 2025.01.11

관련글

티스토리툴바