细看seldon core所部署出来的POD在做什麽

在本篇, 我们来看一下使用seldon完成部署之後, 在k8s上会产生哪些资源

建立在k8s上的seldon资料

• POD
  有一个POD, 在这个POD里面包含2个container.

NAME                                                  READY   STATUS    RESTARTS   AGE
pod/xgboost-default-0-classifier-9688db8bf-9x2bf      2/2     Running   0          16h

• Service
  有二个service

NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/xgboost-default              ClusterIP   10.99.29.1      <none>        8000/TCP,5001/TCP   16h
service/xgboost-default-classifier   ClusterIP   10.110.241.88   <none>        9000/TCP,9500/TCP   16h

• Deployment
  有一个demployment.

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/xgboost-default-0-classifier      1/1     1            1           16h

• Replicaset

NAME                                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/xgboost-default-0-classifier-9688db8bf      1         1         1       16h

细看POD内容

在这些资源中, 我们来看一下xgboost-default-0-classifier-9688db8bf-9x2bf这个POD为我们做些什麽.

• 查看POD内容
  使用kubectl describe指令, 我们可以知道这个POD包含一个initContainer(classifier-model-initializer)与2个container(classifier与seldon-container-engine)

kubectl describe pod xgboost-default-0-classifier-9688db8bf-9x2bf

• 查看initContainer的log
  指令下如:

kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf classifier-model-initializer

[output]
可以看出, initContainer的主要功能是读取model档. 在我们的范例中使用xbgboost server, 因此必需将model的档案命名为model.bst. 可以参考这里的log判断是否可以正确读取model档.

2021/09/07 13:35:24 NOTICE: Config file "/.rclone.conf" not found - using defaults
2021/09/07 13:35:24 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "copy" "-vv" "/mnt/pvc/" "/mnt/models"]
2021/09/07 13:35:24 DEBUG : Creating backend with remote "/mnt/pvc/"
2021/09/07 13:35:24 DEBUG : Creating backend with remote "/mnt/models"
2021/09/07 13:35:25 DEBUG : Local file system at /mnt/models: Waiting for checks to finish
2021/09/07 13:35:25 DEBUG : Local file system at /mnt/models: Waiting for transfers to finish
2021/09/07 13:35:25 DEBUG : model.bst: MD5 = 0c2a00b4f880120021c164d7c238edc0 OK
2021/09/07 13:35:25 INFO  : model.bst: Copied (new)
2021/09/07 13:35:25 DEBUG : model.bst.xgbjson: MD5 = 53962cbec628e763c7d9c33534ba860c OK
2021/09/07 13:35:25 INFO  : model.bst.xgbjson: Copied (new)
2021/09/07 13:35:25 INFO  :
Transferred:        1.944M / 1.944 MBytes, 100%, 24.475 MBytes/s, ETA 0s
Transferred:            5 / 5, 100%
Elapsed time:         0.3s

2021/09/07 13:35:25 DEBUG : 2 go routines active

• 查看 classifier container
  这里的container名称是 classifier, 是我们在yaml档所给定的名称.
  指令如下:

kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf classifier

[output]
在output中可以看出, classifier这个container的主要功能是提供endpoint, port是9000.
当你存取endpoint如果发生失败时, 可以来看这里的log内容, 查看每次打endpoint时在server端发生了什麽事.

starting microservice
2021-09-07 13:35:32,077 - seldon_core.microservice:main:206 - INFO:  Starting microservice.py:main
2021-09-07 13:35:32,077 - seldon_core.microservice:main:207 - INFO:  Seldon Core version: 1.10.0
2021-09-07 13:35:32,083 - seldon_core.microservice:main:362 - INFO:  Parse JAEGER_EXTRA_TAGS []
2021-09-07 13:35:32,084 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/containerID:41bc106f063b21c2b1d2cc9a2b65545663881f02e7651b1c986e0a56c6446031
2021-09-07 13:35:32,084 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/podIP:10.244.79.109/32
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/podIPs:10.244.79.109/32
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation kubernetes.io/config.seen:2021-09-07T21:35:18.889938774+08:00
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation kubernetes.io/config.source:api
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation prometheus.io/path:/prometheus
2021-09-07 13:35:32,086 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation prometheus.io/scrape:true
2021-09-07 13:35:32,086 - seldon_core.microservice:main:365 - INFO:  Annotations: {'cni.projectcalico.org/containerID': '41bc106f063b21c2b1d2cc9a2b65545663881f02e7651b1c986e0a56c6446031', 'cni.projectcalico.org/podIP': '10.244.79.109/32', 'cni.projectcalico.org/podIPs': '10.244.79.109/32', 'kubernetes.io/config.seen': '2021-09-07T21:35:18.889938774+08:00', 'kubernetes.io/config.source': 'api', 'prometheus.io/path': '/prometheus', 'prometheus.io/scrape': 'true'}
2021-09-07 13:35:32,086 - seldon_core.microservice:main:369 - INFO:  Importing XGBoostServer
2021-09-07 13:35:32,719 - seldon_core.microservice:main:448 - INFO:  REST gunicorn microservice running on port 9000
2021-09-07 13:35:32,723 - seldon_core.microservice:main:542 - INFO:  REST metrics microservice running on port 6000
2021-09-07 13:35:32,723 - seldon_core.microservice:main:552 - INFO:  Starting servers
2021-09-07 13:35:32,742 - seldon_core.microservice:grpc_prediction_server:505 - INFO:  GRPC Server Binding to '%s' 0.0.0.0:9500 with 1 processes
2021-09-07 13:35:32,753 - seldon_core.wrapper:_set_flask_app_configs:224 - INFO:  App Config:  <Config {'ENV': 'production', 'DEBUG': False, 'TESTING': False, 'PROPAGATE_EXCEPTIONS': None, 'PRESERVE_CONTEXT_ON_EXCEPTION': None, 'SECRET_KEY': None, 'PERMANENT_SESSION_LIFETIME': datetime.timedelta(days=31), 'USE_X_SENDFILE': False, 'SERVER_NAME': None, 'APPLICATION_ROOT': '/', 'SESSION_COOKIE_NAME': 'session', 'SESSION_COOKIE_DOMAIN': None, 'SESSION_COOKIE_PATH': None, 'SESSION_COOKIE_HTTPONLY': True, 'SESSION_COOKIE_SECURE': False, 'SESSION_COOKIE_SAMESITE': None, 'SESSION_REFRESH_EACH_REQUEST': True, 'MAX_CONTENT_LENGTH': None, 'SEND_FILE_MAX_AGE_DEFAULT': datetime.timedelta(seconds=43200), 'TRAP_BAD_REQUEST_ERRORS': None, 'TRAP_HTTP_EXCEPTIONS': False, 'EXPLAIN_TEMPLATE_LOADING': False, 'PREFERRED_URL_SCHEME': 'http', 'JSON_AS_ASCII': True, 'JSON_SORT_KEYS': True, 'JSONIFY_PRETTYPRINT_REGULAR': False, 'JSONIFY_MIMETYPE': 'application/json', 'TEMPLATES_AUTO_RELOAD': None, 'MAX_COOKIE_SIZE': 4093}>
2021-09-07 13:35:32,777 - seldon_core.wrapper:_set_flask_app_configs:224 - INFO:  App Config:  <Config {'ENV': 'production', 'DEBUG': False, 'TESTING': False, 'PROPAGATE_EXCEPTIONS': None, 'PRESERVE_CONTEXT_ON_EXCEPTION': None, 'SECRET_KEY': None, 'PERMANENT_SESSION_LIFETIME': datetime.timedelta(days=31), 'USE_X_SENDFILE': False, 'SERVER_NAME': None, 'APPLICATION_ROOT': '/', 'SESSION_COOKIE_NAME': 'session', 'SESSION_COOKIE_DOMAIN': None, 'SESSION_COOKIE_PATH': None, 'SESSION_COOKIE_HTTPONLY': True, 'SESSION_COOKIE_SECURE': False, 'SESSION_COOKIE_SAMESITE': None, 'SESSION_REFRESH_EACH_REQUEST': True, 'MAX_CONTENT_LENGTH': None, 'SEND_FILE_MAX_AGE_DEFAULT': datetime.timedelta(seconds=43200), 'TRAP_BAD_REQUEST_ERRORS': None, 'TRAP_HTTP_EXCEPTIONS': False, 'EXPLAIN_TEMPLATE_LOADING': False, 'PREFERRED_URL_SCHEME': 'http', 'JSON_AS_ASCII': True, 'JSON_SORT_KEYS': True, 'JSONIFY_PRETTYPRINT_REGULAR': False, 'JSONIFY_MIMETYPE': 'application/json', 'TEMPLATES_AUTO_RELOAD': None, 'MAX_COOKIE_SIZE': 4093}>
2021-09-07 13:35:32,791 - seldon_core.microservice:_run_grpc_server:460 - INFO:  Starting new GRPC server with 1.
[2021-09-07 13:35:32 +0000] [36] [INFO] Starting gunicorn 20.1.0
[2021-09-07 13:35:32 +0000] [36] [INFO] Listening at: http://0.0.0.0:6000 (36)
[2021-09-07 13:35:32 +0000] [36] [INFO] Using worker: sync
2021-09-07 13:35:32,926 - root:download:31 - INFO:  Copying contents of /mnt/models to local
[2021-09-07 13:35:32 +0000] [46] [INFO] Booting worker with pid: 46
[2021-09-07 13:35:33 +0000] [7] [INFO] Starting gunicorn 20.1.0
[2021-09-07 13:35:33 +0000] [7] [INFO] Listening at: http://0.0.0.0:9000 (7)
[2021-09-07 13:35:33 +0000] [7] [INFO] Using worker: sync
[2021-09-07 13:35:33 +0000] [48] [INFO] Booting worker with pid: 48
2021-09-07 13:35:33,152 - seldon_core.gunicorn_utils:load:103 - INFO:  Tracing not active
2021-09-07 13:35:33,153 - root:download:31 - INFO:  Copying contents of /mnt/models to local

• 查看 seldon-container-engine
  指令如下:

  kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf seldon-container-engine
  

  [output]
  这是seldon core所以提供的功能(SeldonRestApi). 这个container应该不用花太多时间去看它, 只是确定这个container状态是running就好.

  {"level":"info","ts":1631021729.002178,"logger":"entrypoint","msg":"Hostname unset will use localhost"}
  {"level":"info","ts":1631021729.092143,"logger":"entrypoint","msg":"Starting","worker":1}
  {"level":"info","ts":1631021729.0923526,"logger":"entrypoint","msg":"Starting","worker":2}
  {"level":"info","ts":1631021729.0924156,"logger":"entrypoint","msg":"Starting","worker":3}
  {"level":"info","ts":1631021729.09253,"logger":"entrypoint","msg":"Starting","worker":4}
  {"level":"info","ts":1631021729.0925732,"logger":"entrypoint","msg":"Starting","worker":5}
  {"level":"info","ts":1631021729.095814,"logger":"entrypoint","msg":"Running http server ","port":8000}
  {"level":"info","ts":1631021729.0958958,"logger":"entrypoint","msg":"Creating non-TLS listener","port":8000}
  {"level":"info","ts":1631021729.0968246,"logger":"entrypoint","msg":"Running grpc server ","port":5001}
  {"level":"info","ts":1631021729.0968761,"logger":"entrypoint","msg":"Creating non-TLS listener","port":5001}
  {"level":"info","ts":1631021729.096972,"logger":"entrypoint","msg":"Setting max message size ","size":2147483647}
  {"level":"info","ts":1631021729.100623,"logger":"SeldonRestApi","msg":"Listening","Address":"0.0.0.0:8000"}
  

细看Service内容

前面提到有2个service

NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/xgboost-default              ClusterIP   10.99.29.1      <none>        8000/TCP,5001/TCP   16h
service/xgboost-default-classifier   ClusterIP   10.110.241.88   <none>        9000/TCP,9500/TCP   16h

我们是透过xgboost-default-classifier这个service取得推论的结果. 我们来查一下这两个serivce是连到哪个pod

kubectl get ep

结果是

NAME                                            ENDPOINTS                                            AGE
cluster.local-nfs-subdir-external-provisioner   <none>                                               25d
kubernetes                                      172.23.180.10:6443                                   25d
xgboost-default                                 10.244.79.109:5001,10.244.79.109:8000                3d17h
xgboost-default-classifier                      10.244.79.109:9000,10.244.79.109:9500                3d17h

可以看出xgboost-default-classifier这个service的endpoint是指向xgboost-default-0-classifier-9688db8bf-9x2bf这个POD(10.244.79.109), 因此我们可以在前一篇进行推论时经由service最得推论结果

到这里我们已经完成Seldon Core安装, 将model部署在seldon之上, 并且提供endpoint可供存取.