Topics

Problem in running Edgex services

Sezal Chug <sezal17101@...>
 

Hi,
so i was trying to run the latest version of edgex on Kubernetes. But i could run only the consul, volume and config see only . when i run the logging deployment it shows a problem.
This is the description for my pod
Name:               logging-78c5748598-q87qw
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               admin1/192.168.1.9
Start Time:         Tue, 25 Jun 2019 15:23:46 +0530
Labels:             io.kompose.service=logging
                    pod-template-hash=78c5748598
Annotations:        <none>
Status:             Running
IP:                 10.42.0.35
Controlled By:      ReplicaSet/logging-78c5748598
Containers:
  edgex-support-logging:
    Container ID:  containerd://af5f410195f01b2744193c24449e22334a140cb4da356f2efefd7c0e0f5a0a69
    Image:         nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1
    Image ID:      docker.io/edgexfoundry/docker-support-logging-go@sha256:f0b2d67554547e2dd3282f4878b57a79923cf9b5ee730e19dd5a4b42946b2ee6
    Port:          48061/TCP
    Host Port:     0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 25 Jun 2019 15:23:50 +0530
      Finished:     Tue, 25 Jun 2019 15:23:50 +0530
    Ready:          False
    Restart Count:  1
    Environment:    <none>
    Mounts:
      /consul/config from consul-config (rw)
      /consul/data from consul-data (rw)
      /data/db from db-data (rw)
      /edgex/logs from log-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hrf5p (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  db-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  db-data
    ReadOnly:   false
  log-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  log-data
    ReadOnly:   false
  consul-config:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-config
    ReadOnly:   false
  consul-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-data
    ReadOnly:   false
  default-token-hrf5p:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hrf5p
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  18s                default-scheduler  Successfully assigned default/logging-78c5748598-q87qw to admin1
  Normal   Pulled     14s (x2 over 16s)  kubelet, admin1    Container image "nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1" already present on machine
  Normal   Created    13s (x2 over 15s)  kubelet, admin1    Created container edgex-support-logging
  Normal   Started    13s (x2 over 15s)  kubelet, admin1    Started container edgex-support-logging
  Warning  BackOff    11s (x2 over 12s)  kubelet, admin1    Back-off restarting failed container

please check. why is it exiting exit code 1.
the logs are below.
ERROR: 2019/06/25 09:35:02 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:03 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:04 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:05 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:06 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:07 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:08 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:09 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:10 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:11 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:12 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:13 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:14 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:15 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:16 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:17 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:18 edgex-support-logging: Service bootstrap failed!

James.White2@...
 

Sezal,

It appears that your services are unable to get to Consul.  I am not sure what / how you have set up your pods, etc.  If you message me privately, I can put you in touch with a member of my team that has some of EdgeX running in Kubernetes.

 

 

Jim White

Director, IoT Platform Development Team & Distinguished Engineer

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 

 

 

From: EdgeX-Devel@... <EdgeX-Devel@...> On Behalf Of Sezal Chug
Sent: Tuesday, June 25, 2019 4:59 AM
To: EdgeX-Devel@...
Subject: [Edgex-devel] Problem in running Edgex services

 

[EXTERNAL EMAIL]

Hi,

so i was trying to run the latest version of edgex on Kubernetes. But i could run only the consul, volume and config see only . when i run the logging deployment it shows a problem.

This is the description for my pod

Name:               logging-78c5748598-q87qw
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               admin1/192.168.1.9
Start Time:         Tue, 25 Jun 2019 15:23:46 +0530
Labels:             io.kompose.service=logging
                    pod-template-hash=78c5748598
Annotations:        <none>
Status:             Running
IP:                 10.42.0.35
Controlled By:      ReplicaSet/logging-78c5748598
Containers:
  edgex-support-logging:
    Container ID:  containerd://af5f410195f01b2744193c24449e22334a140cb4da356f2efefd7c0e0f5a0a69
    Image:         nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1
    Image ID:      docker.io/edgexfoundry/docker-support-logging-go@sha256:f0b2d67554547e2dd3282f4878b57a79923cf9b5ee730e19dd5a4b42946b2ee6
    Port:          48061/TCP
    Host Port:     0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 25 Jun 2019 15:23:50 +0530
      Finished:     Tue, 25 Jun 2019 15:23:50 +0530
    Ready:          False
    Restart Count:  1
    Environment:    <none>
    Mounts:
      /consul/config from consul-config (rw)
      /consul/data from consul-data (rw)
      /data/db from db-data (rw)
      /edgex/logs from log-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hrf5p (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  db-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  db-data
    ReadOnly:   false
  log-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  log-data
    ReadOnly:   false
  consul-config:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-config
    ReadOnly:   false
  consul-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-data
    ReadOnly:   false
  default-token-hrf5p:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hrf5p
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  18s                default-scheduler  Successfully assigned default/logging-78c5748598-q87qw to admin1
  Normal   Pulled     14s (x2 over 16s)  kubelet, admin1    Container image "nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1" already present on machine
  Normal   Created    13s (x2 over 15s)  kubelet, admin1    Created container edgex-support-logging
  Normal   Started    13s (x2 over 15s)  kubelet, admin1    Started container edgex-support-logging
  Warning  BackOff    11s (x2 over 12s)  kubelet, admin1    Back-off restarting failed container

 

please check. why is it exiting exit code 1.

the logs are below.

ERROR: 2019/06/25 09:35:02 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:03 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:04 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:05 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:06 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:07 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:08 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:09 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:10 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:11 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:12 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:13 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:14 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:15 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:16 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:17 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:18 edgex-support-logging: Service bootstrap failed!

 

espy
 

On 6/25/19 5:58 AM, Sezal Chug wrote:

Hi,
so i was trying to run the latest version of edgex on Kubernetes. But i could run only the consul, volume and config see only . when i run the logging deployment it shows a problem.

Did you use kompose to convert the docker-compose files to k8s? I ran into exactly the same volume errors when I tried using kompose a few months back.

Regards,
/tony



This is the description for my pod
Name:               logging-78c5748598-q87qw
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               admin1/192.168.1.9
Start Time:         Tue, 25 Jun 2019 15:23:46 +0530
Labels:             io.kompose.service=logging
                    pod-template-hash=78c5748598
Annotations:        <none>
Status:             Running
IP:                 10.42.0.35
Controlled By:      ReplicaSet/logging-78c5748598
Containers:
  edgex-support-logging:
    Container ID:  containerd://af5f410195f01b2744193c24449e22334a140cb4da356f2efefd7c0e0f5a0a69
    Image:         nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1
    Image ID:      docker.io/edgexfoundry/docker-support-logging-go@sha256:f0b2d67554547e2dd3282f4878b57a79923cf9b5ee730e19dd5a4b42946b2ee6
    Port:          48061/TCP
    Host Port:     0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 25 Jun 2019 15:23:50 +0530
      Finished:     Tue, 25 Jun 2019 15:23:50 +0530
    Ready:          False
    Restart Count:  1
    Environment:    <none>
    Mounts:
      /consul/config from consul-config (rw)
      /consul/data from consul-data (rw)
      /data/db from db-data (rw)
      /edgex/logs from log-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hrf5p (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  db-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  db-data
    ReadOnly:   false
  log-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  log-data
    ReadOnly:   false
  consul-config:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-config
    ReadOnly:   false
  consul-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  consul-data
    ReadOnly:   false
  default-token-hrf5p:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hrf5p
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  18s                default-scheduler  Successfully assigned default/logging-78c5748598-q87qw to admin1
  Normal   Pulled     14s (x2 over 16s)  kubelet, admin1    Container image "nexus3.edgexfoundry.org:10002/docker-support-logging-go:0.7.1" already present on machine
  Normal   Created    13s (x2 over 15s)  kubelet, admin1    Created container edgex-support-logging
  Normal   Started    13s (x2 over 15s)  kubelet, admin1    Started container edgex-support-logging
  Warning  BackOff    11s (x2 over 12s)  kubelet, admin1    Back-off restarting failed container

please check. why is it exiting exit code 1.
the logs are below.
ERROR: 2019/06/25 09:35:02 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:03 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:04 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:05 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:06 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:07 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:08 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:09 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:10 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:11 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:12 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:13 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:14 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:15 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:16 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:17 connection to Consul could not be made: Put http://edgex-core-consul:8500/v1/agent/service/register: dial tcp: lookup edgex-core-consul on 10.43.0.10:53: no such host
ERROR: 2019/06/25 09:35:18 edgex-support-logging: Service bootstrap failed!