1、ingress 部署有两种方式。本次采用DaemonSet部署。

  1. apiVersion: v1
  2. kind: Namespace
  3. metadata:
  4.   name: ingress-nginx
  5.  
  6. ---
  7.  
  8. apiVersion: extensions/v1beta1
  9. kind: Deployment
  10. metadata:
  11.   name: default-http-backend
  12.   labels:
  13.     app.kubernetes.io/name: default-http-backend
  14.     app.kubernetes.io/part-of: ingress-nginx
  15.   namespace: ingress-nginx
  16. spec:
  17.   replicas: 1
  18.   selector:
  19.     matchLabels:
  20.       app.kubernetes.io/name: default-http-backend
  21.       app.kubernetes.io/part-of: ingress-nginx
  22.   template:
  23.     metadata:
  24.       labels:
  25.         app.kubernetes.io/name: default-http-backend
  26.         app.kubernetes.io/part-of: ingress-nginx
  27.     spec:
  28.       terminationGracePeriodSeconds: 60
  29.       containers:
  30.         - name: default-http-backend
  31.           # Any image is permissible as long as:
  32.           # 1. It serves a 404 page at /
  33.           # 2. It serves 200 on a /healthz endpoint
  34.           image: k8s.gcr.io/defaultbackend-amd64:1.5
  35.           livenessProbe:
  36.             httpGet:
  37.               path: /healthz
  38.               port: 8080
  39.               scheme: HTTP
  40.             initialDelaySeconds: 30
  41.             timeoutSeconds: 5
  42.           ports:
  43.             - containerPort: 8080
  44.           resources:
  45.             limits:
  46.               cpu: 10m
  47.               memory: 20Mi
  48.             requests:
  49.               cpu: 10m
  50.               memory: 20Mi
  51.  
  52. ---
  53. apiVersion: v1
  54. kind: Service
  55. metadata:
  56.   name: default-http-backend
  57.   namespace: ingress-nginx
  58.   labels:
  59.     app.kubernetes.io/name: default-http-backend
  60.     app.kubernetes.io/part-of: ingress-nginx
  61. spec:
  62.   ports:
  63.     - port: 80
  64.       targetPort: 8080
  65.  
  66.   selector:
  67.     app.kubernetes.io/name: default-http-backend
  68.     app.kubernetes.io/part-of: ingress-nginx
  69.  
  70. ---
  71.  
  72. kind: ConfigMap
  73. apiVersion: v1
  74. metadata:
  75.   name: nginx-configuration
  76.   namespace: ingress-nginx
  77.   labels:
  78.     app.kubernetes.io/name: ingress-nginx
  79.     app.kubernetes.io/part-of: ingress-nginx
  80.  
  81. ---
  82.  
  83. kind: ConfigMap
  84. apiVersion: v1
  85. metadata:
  86.   name: tcp-services
  87.   namespace: ingress-nginx
  88.   labels:
  89.     app.kubernetes.io/name: ingress-nginx
  90.     app.kubernetes.io/part-of: ingress-nginx
  91.  
  92. ---
  93.  
  94. kind: ConfigMap
  95. apiVersion: v1
  96. metadata:
  97.   name: udp-services
  98.   namespace: ingress-nginx
  99.   labels:
  100.     app.kubernetes.io/name: ingress-nginx
  101.     app.kubernetes.io/part-of: ingress-nginx
  102.  
  103. ---
  104.  
  105. apiVersion: v1
  106. kind: ServiceAccount
  107. metadata:
  108.   name: nginx-ingress-serviceaccount
  109.   namespace: ingress-nginx
  110.   labels:
  111.     app.kubernetes.io/name: ingress-nginx
  112.     app.kubernetes.io/part-of: ingress-nginx
  113.  
  114. ---
  115. apiVersion: rbac.authorization.k8s.io/v1beta1
  116. kind: ClusterRole
  117. metadata:
  118.   name: nginx-ingress-clusterrole
  119.   labels:
  120.     app.kubernetes.io/name: ingress-nginx
  121.     app.kubernetes.io/part-of: ingress-nginx
  122. rules:
  123.   - apiGroups:
  124.       - ""
  125.     resources:
  126.       - configmaps
  127.       - endpoints
  128.       - nodes
  129.       - pods
  130.       - secrets
  131.     verbs:
  132.       - list
  133.       - watch
  134.   - apiGroups:
  135.       - ""
  136.     resources:
  137.       - nodes
  138.     verbs:
  139.       - get
  140.   - apiGroups:
  141.       - ""
  142.     resources:
  143.       - services
  144.     verbs:
  145.       - get
  146.       - list
  147.       - watch
  148.   - apiGroups:
  149.       - "extensions"
  150.     resources:
  151.       - ingresses
  152.     verbs:
  153.       - get
  154.       - list
  155.       - watch
  156.   - apiGroups:
  157.       - ""
  158.     resources:
  159.       - events
  160.     verbs:
  161.       - create
  162.       - patch
  163.   - apiGroups:
  164.       - "extensions"
  165.     resources:
  166.       - ingresses/status
  167.     verbs:
  168.       - update
  169.  
  170. ---
  171. apiVersion: rbac.authorization.k8s.io/v1beta1
  172. kind: Role
  173. metadata:
  174.   name: nginx-ingress-role
  175.   namespace: ingress-nginx
  176.   labels:
  177.     app.kubernetes.io/name: ingress-nginx
  178.     app.kubernetes.io/part-of: ingress-nginx
  179. rules:
  180.   - apiGroups:
  181.       - ""
  182.     resources:
  183.       - configmaps
  184.       - pods
  185.       - secrets
  186.       - namespaces
  187.     verbs:
  188.       - get
  189.   - apiGroups:
  190.       - ""
  191.     resources:
  192.       - configmaps
  193.     resourceNames:
  194.       # Defaults to "<election-id>-<ingress-class>"
  195.       # Here: "<ingress-controller-leader>-<nginx>"
  196.       # This has to be adapted if you change either parameter
  197.       # when launching the nginx-ingress-controller.
  198.       - "ingress-controller-leader-nginx"
  199.     verbs:
  200.       - get
  201.       - update
  202.   - apiGroups:
  203.       - ""
  204.     resources:
  205.       - configmaps
  206.     verbs:
  207.       - create
  208.   - apiGroups:
  209.       - ""
  210.     resources:
  211.       - endpoints
  212.     verbs:
  213.       - get
  214.  
  215. ---
  216. apiVersion: rbac.authorization.k8s.io/v1beta1
  217. kind: RoleBinding
  218. metadata:
  219.   name: nginx-ingress-role-nisa-binding
  220.   namespace: ingress-nginx
  221.   labels:
  222.     app.kubernetes.io/name: ingress-nginx
  223.     app.kubernetes.io/part-of: ingress-nginx
  224. roleRef:
  225.   apiGroup: rbac.authorization.k8s.io
  226.   kind: Role
  227.   name: nginx-ingress-role
  228. subjects:
  229.   - kind: ServiceAccount
  230.     name: nginx-ingress-serviceaccount
  231.     namespace: ingress-nginx
  232.  
  233. ---
  234. apiVersion: rbac.authorization.k8s.io/v1beta1
  235. kind: ClusterRoleBinding
  236. metadata:
  237.   name: nginx-ingress-clusterrole-nisa-binding
  238.   labels:
  239.     app.kubernetes.io/name: ingress-nginx
  240.     app.kubernetes.io/part-of: ingress-nginx
  241. roleRef:
  242.   apiGroup: rbac.authorization.k8s.io
  243.   kind: ClusterRole
  244.   name: nginx-ingress-clusterrole
  245. subjects:
  246.   - kind: ServiceAccount
  247.     name: nginx-ingress-serviceaccount
  248.     namespace: ingress-nginx
  249.  
  250. ---
  251. apiVersion: extensions/v1beta1
  252. #apiVersion: v1
  253. kind: DaemonSet
  254. metadata:
  255.   name: nginx-ingress-controller
  256.   namespace: ingress-nginx
  257.   labels:
  258.     app.kubernetes.io/name: ingress-nginx
  259.     app.kubernetes.io/part-of: ingress-nginx
  260. spec:
  261. #  replicas: 1  注意使用DaemonSet部署不要设置re.默认为会给标签上面会有一个副本
  262. #  selector:                    #注释掉
  263. #    matchLabels:
  264. #      app.kubernetes.io/name: ingress-nginx  #注释掉
  265. #      app.kubernetes.io/part-of: ingress-nginx  #注释掉
  266. #      sIngress: true   #主调式
  267.   template:
  268.     metadata:
  269.       labels:
  270.         app.kubernetes.io/name: ingress-nginx
  271.         app.kubernetes.io/part-of: ingress-nginx
  272.       annotations:
  273.         prometheus.io/port: "10254"
  274.         prometheus.io/scrape: "true"
  275.     spec:
  276.       serviceAccountName: nginx-ingress-serviceaccount
  277.       nodeSelector:
  278.         isIngress: "true"
  279.       hostNetwork: true                     ####启用主机网络 。会霸占主机80端口，如果主机80端口已经被使用。则会启动失败。
  280.       containers:
  281.         - name: nginx-ingress-controller
  282.           image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0
  283.           args:
  284.             - /nginx-ingress-controller
  285.             - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
  286.             - --configmap=$(POD_NAMESPACE)/nginx-configuration
  287.             - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
  288.             - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
  289.             - --publish-service=$(POD_NAMESPACE)/ingress-nginx
  290.             - --annotations-prefix=nginx.ingress.kubernetes.io
  291.           securityContext:
  292.             capabilities:
  293.               drop:
  294.                 - ALL
  295.               add:
  296.                 - NET_BIND_SERVICE
  297.             # www-data -> 33
  298.             runAsUser: 33
  299.           env:
  300.             - name: POD_NAME
  301.               valueFrom:
  302.                 fieldRef:
  303.                   fieldPath: metadata.name
  304.             - name: POD_NAMESPACE
  305.               valueFrom:
  306.                 fieldRef:
  307.                   fieldPath: metadata.namespace
  308.           ports:
  309.             - name: http
  310.               containerPort: 80
  311.             - name: https
  312.               containerPort: 443
  313.           livenessProbe:
  314.             failureThreshold: 3
  315.             httpGet:
  316.               path: /healthz
  317.               port: 10254
  318.               scheme: HTTP
  319.             initialDelaySeconds: 10
  320.             periodSeconds: 10
  321.             successThreshold: 1
  322.             timeoutSeconds: 1
  323.           readinessProbe:
  324.             failureThreshold: 3
  325.             httpGet:
  326.               path: /healthz
  327.               port: 10254
  328.               scheme: HTTP
  329.             periodSeconds: 10
  330.             successThreshold: 1
  331.             timeoutSeconds: 1
  332.  
  333. ---

2、部署结果

  1. kubectl   apply   -f  mandatory.yaml 
  2.  
  3. [root@VM_0_48_centos ingress-nginx]# kubectl  get pods  -n ingress-nginx
  4. NAME                                    READY   STATUS    RESTARTS   AGE
  5. default-http-backend-85b8b595f9-j5twg   1/1     Running   1          30h
  6. nginx-ingress-controller-mrvzh          1/1     Running   0          104m
  7. nginx-ingress-controller-pgp9t          1/1     Running   0          105m
  8. nginx-ingress-controller-vd7v6          1/1     Running   0          104m
  9. [root@VM_0_48_centos ingress-nginx]#

3、测试

  1. [root@VM_0_48_centos ingress-nginx]# cat test-deploy-demon.yaml
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5.   name: myapp
  6. spec:
  7.   selector:
  8.     app: myapp
  9.     release: canary
  10.   ports:
  11.   - name: http
  12.     port: 80
  13.     targetPort: 80
  14. ---
  15. apiVersion: apps/v1
  16. kind: Deployment
  17. metadata:
  18.   name: myapp-deploy
  19. spec:
  20.   replicas: 1
  21.   selector:
  22.     matchLabels:
  23.       app: myapp
  24.       release: canary
  25.   template:
  26.     metadata:
  27.       labels:
  28.         app: myapp
  29.         release: canary
  30.     spec:
  31.       containers:
  32.       - name: myapp
  33.         image: ikubernetes/myapp:v2
  34.         ports:
  35.         - name: httpd
  36.           containerPort: 80
  1. [root@VM_0_48_centos ingress-nginx]# cat test.yaml
  2. apiVersion: extensions/v1beta1
  3. kind: Ingress
  4. metadata:
  5.   name: ingress-myapp
  6.   namespace: default                     #######注意和service 保持一致。
  7.   annotations:
  8.     kubernetes.io/ingress.class: ""
  9. spec:
  10.   tls:  #是否启用https证书。
  11.   - hosts:
  12.     - test.xiajq.com
  13.     secretName: tomcat-ingress-secret  ##
  14.   rules:
  15.   - host: test.xiajq.com #生产中该域名应当可以被公网解析
  16.     http:
  17.       paths:
  18.       - path:
  19.         backend:
  20.           serviceName: myapp
  21.           servicePort: 80

4、如果不使用https，可以直接本地hosts解析后，访问。如果https还需创建证书

  1. openssl genrsa -out tls.key 2048
  2. openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=test.xiajq.com
  3. kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key #创建secret

5、本地hosts解析后。测试访问结果：

  1. https://test.xiajq.com/

6、ingress 解析原理。本质上还是使用了nginx作为代理服务器进行解析。只是nginx采用node方式部署。同时授权了动态获取k8s ep权限。可以动态感知后端ep变化。从而动态更新nginx代理到后端pod配置。也就nginx.conf的配置。

  1. --查看node端口
  2. [root@VM_0_2_centos ~]# netstat -ntlup|grep 80
  3. tcp        0      0 172.19.0.2:2380         0.0.0.0:*               LISTEN      1187/etcd
  4. tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      10193/nginx: master 
  5.  
  6. --进入pod
  7.  kubectl  exec -it nginx-ingress-controller-mrvzh    /bin/bash  -n ingress-nginx 
  8.  
  9. --
  10.     include /etc/nginx/mime.types;
  11.         default_type text/html;
  12.  
  13.         gzip on;
  14.         gzip_comp_level 5;
  15.         gzip_http_version 1.1;
  16.         gzip_min_length 256;
  17.         gzip_types application/atom+xml application/javascript application/x-javascript application/json application/rss+xml application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/svg+xml image/x-icon text/css text/plain text/x-component;
  18.         gzip_proxied any;
  19.         gzip_vary on;
  20.  
  21.         # Custom headers for response
  22.  
  23.         server_tokens on;
  24.  
  25.         # disable warnings
  26.         uninitialized_variable_warn off;
  27.  
  28.         # Additional available variables:
  29.         # $namespace
  30.         # $ingress_name
  31.         # $service_name
  32.         # $service_port
  33.         log_format upstreaminfo '$the_real_ip - [$the_real_ip] - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id';
  34.  
  35.         map $request_uri $loggable {
  36.  
  37.                 default 1;
  38.         }
  39.  
  40.         access_log /var/log/nginx/access.log upstreaminfo if=$loggable;
  41.  
  42.         error_log  /var/log/nginx/error.log notice;
  43.  
  44.         resolver 183.60.83.19 183.60.82.98 valid=30s ipv6=off;
  45.  
  46.         # See https://www.nginx.com/blog/websocket-nginx
  47.         map $http_upgrade $connection_upgrade {
  48.                 default          upgrade;
  49.  
  50.                 # See http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
  51.                 ''               '';
  52.  
  53.         }
  54.  
  55.         # The following is a sneaky way to do "set $the_real_ip $remote_addr"
  56.         # Needed because using set is not allowed outside server blocks.
  57.         map '' $the_real_ip {
  58.  
  59.                 default          $remote_addr;
  60.  
  61.         }
  62.  
  63.         # trust http_x_forwarded_proto headers correctly indicate ssl offloading
  64.         map $http_x_forwarded_proto $pass_access_scheme {
  65.                 default          $http_x_forwarded_proto;
  66.                 ''               $scheme;
  67.         }
  68.  
  69.         map $http_x_forwarded_port $pass_server_port {
  70.                 default           $http_x_forwarded_port;
  71.                 ''                $server_port;
  72.         }
  73.  
  74.         # Obtain best http host
  75.         map $http_host $this_host {
  76.                 default          $http_host;
  77.                 ''               $host;
  78.         }
  79.  
  80.         map $http_x_forwarded_host $best_http_host {
  81.                 default          $http_x_forwarded_host;
  82.                 ''               $this_host;
  83.         }
  84.  
  85.         # validate $pass_access_scheme and $scheme are http to force a redirect
  86.         map "$scheme:$pass_access_scheme" $redirect_to_https {
  87.                 default          0;
  88.                 "http:http"      1;
  89.                 "https:http"     1;
  90.         }
  91.  
  92.         map $pass_server_port $pass_port {
  93.                 443              443;
  94.                 default          $pass_server_port;
  95.         }
  96.  
  97.         # Reverse proxies can detect if a client provides a X-Request-ID header, and pass it on to the backend server.
  98.         # If no such header is provided, it can provide a random value.
  99.         map $http_x_request_id $req_id {
  100.                 default   $http_x_request_id;
  101.  
  102.                 ""        $request_id;
  103.  
  104.         }
  105.  
  106.         server_name_in_redirect off;
  107.         port_in_redirect        off;
  108.  
  109.         ssl_protocols TLSv1.2;
  110.  
  111.         # turn on session caching to drastically improve performance
  112.  
  113.         ssl_session_cache builtin:1000 shared:SSL:10m;
  114.         ssl_session_timeout 10m;
  115.  
  116.         # allow configuring ssl session tickets
  117.         ssl_session_tickets on;
  118.  
  119.         # slightly reduce the time-to-first-byte
  120.         ssl_buffer_size 4k;
  121.  
  122.         # allow configuring custom ssl ciphers
  123.         ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
  124.         ssl_prefer_server_ciphers on;
  125.  
  126.         ssl_ecdh_curve auto;
  127.  
  128.         proxy_ssl_session_reuse on;
  129.  
  130.         upstream upstream_balancer {
  131.                 server 0.0.0.1; # placeholder
  132.  
  133.                 balancer_by_lua_block {
  134.                         balancer.balance()
  135.                 }
  136.  
  137.                 keepalive 32;
  138.  
  139.         }
  140.  
  141.         # Global filters
  142.  
  143.         ## start server _
  144.         server {
  145.                 server_name _ ;
  146.  
  147.                 listen 80 default_server reuseport backlog=511;
  148.  
  149.                 set $proxy_upstream_name "-";
  150.  
  151.                 listen 443  default_server reuseport backlog=511 ssl http2;
  152.  
  153.                 # PEM sha: 472974aa2c90732bd8c67a1e5ef601e2f5915003
  154.                 ssl_certificate                         /etc/ingress-controller/ssl/default-fake-certificate.pem;
  155.                 ssl_certificate_key                     /etc/ingress-controller/ssl/default-fake-certificate.pem;
  156.  
  157.                 location / {
  158.  
  159.                         set $namespace      "";
  160.                         set $ingress_name   "";
  161.                         set $service_name   "";
  162.                         set $service_port   "0";
  163.                         set $location_path  "/";
  164.  
  165.                         rewrite_by_lua_block {
  166.  
  167.                                 balancer.rewrite()
  168.  
  169.                         }
  170.  
  171.                         log_by_lua_block {
  172.  
  173.                                 balancer.log()
  174.  
  175.                                 monitor.call()
  176.                         }
  177.  
  178.                         if ($scheme = https) {
  179.                                 more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains";
  180.                         }
  181.  
  182.                         access_log off;
  183.  
  184.                         port_in_redirect off;
  185.  
  186.                         set $proxy_upstream_name "upstream-default-backend";
  187.  
  188.                         client_max_body_size                    1m;
  189.  
  190.                         proxy_set_header Host                   $best_http_host;
  191.  
  192.                         # Pass the extracted client certificate to the backend
  193.  
  194.                         # Allow websocket connections
  195.                         proxy_set_header                        Upgrade           $http_upgrade;
  196.  
  197.                         proxy_set_header                        Connection        $connection_upgrade;
  198.  
  199.                         proxy_set_header X-Request-ID           $req_id;
  200.                         proxy_set_header X-Real-IP              $the_real_ip;
  201.  
  202.                         proxy_set_header X-Forwarded-For        $the_real_ip;
  203.  
  204.                         proxy_set_header X-Forwarded-Host       $best_http_host;
  205.                         proxy_set_header X-Forwarded-Port       $pass_port;
  206.                         proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
  207.  
  208.                         proxy_set_header X-Original-URI         $request_uri;
  209.  
  210.                         proxy_set_header X-Scheme               $pass_access_scheme;
  211.  
  212.                         # Pass the original X-Forwarded-For
  213.                         proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
  214.  
  215.                         # mitigate HTTPoxy Vulnerability
  216.                         # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
  217.                         proxy_set_header Proxy                  "";
  218.  
  219.                         # Custom headers to proxied server
  220.  
  221.                         proxy_connect_timeout                   5s;
  222.                         proxy_send_timeout                      60s;
  223.                         proxy_read_timeout                      60s;
  224.  
  225.                         proxy_buffering                         off;
  226.                         proxy_buffer_size                       4k;
  227.                         proxy_buffers                           4 4k;
  228.                         proxy_request_buffering                 on;
  229.  
  230.                         proxy_http_version                      1.1;
  231.  
  232.                         proxy_cookie_domain                     off;
  233.                         proxy_cookie_path                       off;
  234.  
  235.                         # In case of errors try the next upstream server before returning an error
  236.                         proxy_next_upstream                     error timeout;
  237.                         proxy_next_upstream_tries               3;
  238.  
  239.                         proxy_pass http://upstream_balancer;
  240.  
  241.                         proxy_redirect                          off;
  242.  
  243.                 }
  244.  
  245.                 # health checks in cloud providers require the use of port 80
  246.                 location /healthz {
  247.  
  248.                         access_log off;
  249.                         return 200;
  250.                 }
  251.  
  252.                 # this is required to avoid error if nginx is being monitored
  253.                 # with an external software (like sysdig)
  254.                 location /nginx_status {
  255.  
  256.                         allow 127.0.0.1;
  257.  
  258.                         deny all;
  259.  
  260.                         access_log off;
  261.                         stub_status on;
  262.                 }
  263.  
  264.         }
  265.         ## end server _
  266.  
  267.         ## start server dashboard.xiajq.com
  268.         server {
  269.                 server_name dashboard.xiajq.com ;
  270.  
  271.                 listen 80;
  272.  
  273.                 set $proxy_upstream_name "-";
  274.  
  275.                 listen 443  ssl http2;
  276.  
  277.                 # PEM sha: 472974aa2c90732bd8c67a1e5ef601e2f5915003
  278.                 ssl_certificate                         /etc/ingress-controller/ssl/default-fake-certificate.pem;
  279.                 ssl_certificate_key                     /etc/ingress-controller/ssl/default-fake-certificate.pem;
  280.  
  281.                 location / {
  282.  
  283.                         set $namespace      "ingress-nginx";
  284.                         set $ingress_name   "ingress-myapp";
  285.                         set $service_name   "kubernetes-dashboard";
  286.                         set $service_port   "443";
  287.                         set $location_path  "/";
  288.  
  289.                         rewrite_by_lua_block {
  290.  
  291.                                 balancer.rewrite()
  292.  
  293.                         }
  294.  
  295.                         log_by_lua_block {
  296.  
  297.                                 balancer.log()
  298.  
  299.                                 monitor.call()
  300.                         }
  301.  
  302.                         if ($scheme = https) {
  303.                                 more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains";
  304.                         }
  305.  
  306.                         port_in_redirect off;
  307.  
  308.                         set $proxy_upstream_name "";
  309.  
  310.                         # enforce ssl on server side
  311.                         if ($redirect_to_https) {
  312.  
  313.                                 return 308 https://$best_http_host$request_uri;
  314.  
  315.                         }
  316.  
  317.                         client_max_body_size                    1m;
  318.  
  319.                         proxy_set_header Host                   $best_http_host;
  320.  
  321.                         # Pass the extracted client certificate to the backend
  322.  
  323.                         # Allow websocket connections
  324.                         proxy_set_header                        Upgrade           $http_upgrade;
  325.  
  326.                         proxy_set_header                        Connection        $connection_upgrade;
  327.  
  328.                         proxy_set_header X-Request-ID           $req_id;
  329.                         proxy_set_header X-Real-IP              $the_real_ip;
  330.  
  331.                         proxy_set_header X-Forwarded-For        $the_real_ip;
  332.  
  333.                         proxy_set_header X-Forwarded-Host       $best_http_host;
  334.                         proxy_set_header X-Forwarded-Port       $pass_port;
  335.                         proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
  336.  
  337.                         proxy_set_header X-Original-URI         $request_uri;
  338.  
  339.                         proxy_set_header X-Scheme               $pass_access_scheme;
  340.  
  341.                         # Pass the original X-Forwarded-For
  342.                         proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
  343.  
  344.                         # mitigate HTTPoxy Vulnerability
  345.                         # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
  346.                         proxy_set_header Proxy                  "";
  347.  
  348.                         # Custom headers to proxied server
  349.  
  350.                         proxy_connect_timeout                   5s;
  351.                         proxy_send_timeout                      60s;
  352.                         proxy_read_timeout                      60s;
  353.  
  354.                         proxy_buffering                         off;
  355.                         proxy_buffer_size                       4k;
  356.                         proxy_buffers                           4 4k;
  357.                         proxy_request_buffering                 on;
  358.  
  359.                         proxy_http_version                      1.1;
  360.  
  361.                         proxy_cookie_domain                     off;
  362.                         proxy_cookie_path                       off;
  363.  
  364.                         # In case of errors try the next upstream server before returning an error
  365.                         proxy_next_upstream                     error timeout;
  366.                         proxy_next_upstream_tries               3;
  367.  
  368.                         # No endpoints available for the request
  369.                         return 503;
  370.  
  371.                 }
  372.  
  373.         }
  374.         ## end server dashboard.xiajq.com
  375.  
  376.         ## start server test.xiajq.com   ############################## 在创建一个ingress后，会自动在ng.conf中默认配置
  377.         server {
  378.                 server_name test.xiajq.com ;
  379.  
  380.                 listen 80;
  381.  
  382.                 set $proxy_upstream_name "-";
  383.  
  384.                 listen 443  ssl http2;   ##启用443端口
  385.  
  386.                 # PEM sha: 6b96e02eb5f1eaa3d45486073017e7fd8ce3a40e
  387.                 ssl_certificate                         /etc/ingress-controller/ssl/default-tomcat-ingress-secret.pem;   #证书位置。这个为我们创建的证书
  388.                 ssl_certificate_key                     /etc/ingress-controller/ssl/default-tomcat-ingress-secret.pem;
  389.  
  390.                 ssl_trusted_certificate                 /etc/ingress-controller/ssl/default-tomcat-ingress-secret-full-chain.pem;  ##
  391.                 ssl_stapling                            on;
  392.                 ssl_stapling_verify                     on;
  393.  
  394.                 location / {
  395.  
  396.                         set $namespace      "default";            ####设置变量
  397.                         set $ingress_name   "ingress-myapp";    ###
  398.                         set $service_name   "myapp";             ###服务名字
  399.                         set $service_port   "80";                 ##端口
  400.                         set $location_path  "/";                   ###路径
  401.  
  402.                         rewrite_by_lua_block {
  403.  
  404.                                 balancer.rewrite()
  405.  
  406.                         }
  407.  
  408.                         log_by_lua_block {
  409.  
  410.                                 balancer.log()
  411.  
  412.                                 monitor.call()
  413.                         }
  414.  
  415.                         if ($scheme = https) {
  416.                                 more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains";
  417.                         }
  418.  
  419.                         port_in_redirect off;
  420.  
  421.                         set $proxy_upstream_name "default-myapp-80";
  422.  
  423.                         # enforce ssl on server side
  424.                         if ($redirect_to_https) {
  425.  
  426.                                 return 308 https://$best_http_host$request_uri;
  427.  
  428.                         }
  429.  
  430.                         client_max_body_size                    1m;
  431.  
  432.                         proxy_set_header Host                   $best_http_host;
  433.  
  434.                         # Pass the extracted client certificate to the backend
  435.  
  436.                         # Allow websocket connections
  437.                         proxy_set_header                        Upgrade           $http_upgrade;
  438.  
  439.                         proxy_set_header                        Connection        $connection_upgrade;
  440.  
  441.                         proxy_set_header X-Request-ID           $req_id;
  442.                         proxy_set_header X-Real-IP              $the_real_ip;
  443.  
  444.                         proxy_set_header X-Forwarded-For        $the_real_ip;
  445.  
  446.                         proxy_set_header X-Forwarded-Host       $best_http_host;
  447.                         proxy_set_header X-Forwarded-Port       $pass_port;
  448.                         proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
  449.  
  450.                         proxy_set_header X-Original-URI         $request_uri;
  451.  
  452.                         proxy_set_header X-Scheme               $pass_access_scheme;
  453.  
  454.                         # Pass the original X-Forwarded-For
  455.                         proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
  456.  
  457.                         # mitigate HTTPoxy Vulnerability
  458.                         # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
  459.                         proxy_set_header Proxy                  "";
  460.  
  461.                         # Custom headers to proxied server
  462.  
  463.                         proxy_connect_timeout                   5s;
  464.                         proxy_send_timeout                      60s;
  465.                         proxy_read_timeout                      60s;
  466.  
  467.                         proxy_buffering                         off;
  468.                         proxy_buffer_size                       4k;
  469.                         proxy_buffers                           4 4k;
  470.                         proxy_request_buffering                 on;
  471.  
  472.                         proxy_http_version                      1.1;
  473.  
  474.                         proxy_cookie_domain                     off;
  475.                         proxy_cookie_path                       off;
  476.  
  477.                         # In case of errors try the next upstream server before returning an error
  478.                         proxy_next_upstream                     error timeout;
  479.                         proxy_next_upstream_tries               3;
  480.  
  481.                         proxy_pass http://upstream_balancer;
  482.  
  483.                         proxy_redirect                          off;
  484.  
  485.                 }
  486.  
  487.         }
  488.         ## end server test.xiajq.com
  489.  
  490.         # backend for when default-backend-service is not configured or it does not have endpoints
  491.         server {
  492.                 listen 8181 default_server reuseport backlog=511;
  493.  
  494.                 set $proxy_upstream_name "-";
  495.  
  496.                 location / {
  497.                         return 404;
  498.                 }
  499.         }
  500.  
  501.         # default server, used for NGINX healthcheck and access to nginx stats
  502.         server {
  503.                 listen 18080 default_server reuseport backlog=511;
  504.  
  505.                 set $proxy_upstream_name "-";
  506.  
  507.                 location /healthz {
  508.  
  509.                         access_log off;
  510.                         return 200;
  511.                 }
  512.  
  513.                 location /is-dynamic-lb-initialized {
  514.  
  515.                         access_log off;
  516.  
  517.                         content_by_lua_block {
  518.                                 local configuration = require("configuration")
  519.                                 local backend_data = configuration.get_backends_data()
  520.                                 if not backend_data then
  521.                                 ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
  522.                                 return
  523.                                 end
  524.  
  525.                                 ngx.say("OK")
  526.                                 ngx.exit(ngx.HTTP_OK)
  527.                         }
  528.                 }
  529.  
  530.                 location /nginx_status {
  531.                         set $proxy_upstream_name "internal";
  532.  
  533.                         access_log off;
  534.                         stub_status on;
  535.                 }
  536.  
  537.                 location /configuration {
  538.                         access_log off;
  539.  
  540.                         allow 127.0.0.1;
  541.  
  542.                         deny all;
  543.  
  544.                         # this should be equals to configuration_data dict
  545.                         client_max_body_size                    10m;
  546.                         proxy_buffering                         off;
  547.  
  548.                         content_by_lua_block {
  549.                                 configuration.call()
  550.                         }
  551.                 }
  552.  
  553.                 location / {
  554.  
  555.                         set $proxy_upstream_name "upstream-default-backend";
  556.                         proxy_set_header    Host   $best_http_host;
  557.  
  558.                         proxy_pass          http://upstream_balancer;
  559.  
  560.                 }
  561.  
  562.         }
  563. }
  564.  
  565. stream {
  566.         log_format log_stream [$time_local] $protocol $status $bytes_sent $bytes_received $session_time;
  567.  
  568.         access_log /var/log/nginx/access.log log_stream;
  569.  
  570.         error_log  /var/log/nginx/error.log;
  571.  
  572.         # TCP services
  573.  
  574.         # UDP services
  575.  
  576. }

7、ingress 使用案例，添加白名单

  1. igress控制ip访问
  2.  
  3. apiVersion: extensions/v1beta1
  4. kind: Ingress
  5. metadata:
  6.   name: whitelist
  7.   annotations:
  8.     ingress.kubernetes.io/whitelist-source-range: "1.1.1.1/24"
  9.     ingress.kubernetes.io/whitelist-source-range: 192.168**/,192.168**/32
  10. spec:
  11.   rules:
  12.   - host: whitelist.test.net
  13.   http:
  14.     paths:
  15.     - path: /
  16.     backend:
  17.       serviceName: webserver
  18.       servicePort: 80
  1. yaml下载地址: https://github.com/kubernetes/ingress-nginx/tree/nginx-0.20.0/deploy

　　

k8s-ingress部署测试以及深入理解的更多相关文章

  1. k8s Ingress 理解和部署

    目录 前言 Ingress 与 ingress-controller Ingress 部署 1.部署 ingress-controller 2.部署测试 web 服务 3.部署 Ingress 4.检 ...

  2. K8S从入门到放弃系列-(15)Kubernetes集群Ingress部署

    Ingress是kubernetes集群对外提供服务的一种方式.ingress部署相对比较简单,官方把相关资源配置文件,都已经集合到一个yml文件中(mandatory.yaml),镜像地址也修改为q ...

  3. K8S中部署apisix(非ingress)

    不使用pvc的方式在K8S中部署apisix-gateway 简介 因为公司项目准备重构,现在做技术储备,之前公司项目使用的ocelot做网关,ocelot是.net平台下的一个网关,也是很不错,但是 ...

  4. 浅谈 k8s ingress controller 选型

    大家好,先简单自我介绍下,我叫厉辉,来自腾讯云.业余时间比较喜欢开源,现在是Apache APISIX PPMC.今天我来简单给大家介绍下 K8S Ingress 控制器的选型经验,今天我讲的这些内容 ...

  5. 在k8s上部署日志系统elfk

    日志系统elfk 前言 经过上周的技术预研,在本周一通过开会研究,根据公司的现有业务流量和技术栈,决定选择的日志系统方案为:elasticsearch(es)+logstash(lo)+filebea ...

  6. 优化：在k8s上部署的gitlab

    gitlab组件图 gitlab在k8s上占用资源 # kubectl top pods -n default | grep git* gitlab-gitaly-0 9m 444Mi gitlab- ...

  7. k8s 新版本 部署 Ingress-nginx controller

    k8s 新版本 部署 Ingress-nginx controller 本篇主要记录一下 k8s 新版本 1.23.5 中如何搭建 ingress controller 以及里面的注意项 新版本和老版 ...

  8. Kubernetes Ingress 部署

    Kubernetes Ingress 部署 Pod与Ingress的关系• 通过service相关联• 通过Ingress Controller实现Pod的负载均衡- 支持TCP/UDP 4层和HTT ...

  9. Kubernetes之在k8s中部署Java应用

    部署好了k8s以后 部署参考https://www.cnblogs.com/minseo/p/12055731.html 怎么在k8s部署应用 项目迁移到k8s平台是怎样的流程 1,制作镜像 2,控制 ...

随机推荐

  1. 字节Android Native Crash治理之Memory Corruption工具原理与实践

    作者:字节跳动终端技术--庞翔宇 内容摘要 ​ MemCorruption工具是字节跳动AppHealth (Client Infrastructure - AppHealth) 团队开发的一款用于定 ...

  2. JavaScript通过父节点ID递归生成JSON树

    JavaScript通过父节点ID递归生成JSON树: · 实现思路:通过递归实现(第一次递归的时候查询出所有的父节点,然后通过当前父节点id不断地去查询所有子节点,直到递归完毕返回)   · 代码示 ...

  3. 【XR-4】文本编辑器

    直接做是困难的,不妨依照部分分来思考. - Subtask 3 首先会进入一个误区:维护修改,通过循环串的性质在 \(\tt KMP\) 自动机上优化遍历. 但可以发现这样很难处理,我们不妨 直接维护 ...

  4. MySQL 新增表分区很慢，转移大表数据

    问题: MySQL (version 5.7.26) 数据库有一批表 xxx_yyy,由于评估的数据量可能比较大,因此每张表都设置了表分区,把每个月的数据保存在单独的分区里. 那么如果每年年末,没有提 ...

  5. 7.2.*PHP编译安装时常见错误解决办法，php编译常见错误

    configure: error: Cannot find ldap.h   检查下面是不是已经安装,如果没有安装之:检查:yum list openldapyum list openldap-dev ...

  6. tomcat启用apr需要的三个组件(缺少可能会报错)

    tomcat8开始支持Apr,可以提升IO性能,但若配置了使用Apr,如下图所示,则需要安装apr和aprutil和tomcat-native 如果采用nio的方式,则配置可以改成protocol=& ...

  7. feiQ发送信息

    import java.io.IOException; import java.net.DatagramPacket; import java.net.DatagramSocket; import j ...

  8. 3.6 万颗星！开源 Web 服务器后起之秀，自带免费 HTTPS 开箱即用

    众所周知,Web 服务器是 Web 开发中不可或缺的基础服务,在开发中经常会用到.耳熟能详的开源 Web 服务器有久负盛名的 Apache.性能强劲的 Nginx.而我们今天要介绍的开源项目是采用 G ...

  9. Note - Powerful Number

    Powerful Number   对于 \(n\in\mathbb N_+\),若不存在素数 \(p\) 使得 \(p\mid n~\land~p^2\not\mid n\),则称 \(n\) 为 ...

  10. 故障分析：网络闪断引发的ServiceStack.Redis死锁问题

    背景说明 某天生产环境发生进程的活跃线程数过高的预警问题,且一天两个节点分别出现相同预警.此程序近一年没出现过此类预警,事出必有因,本文就记录下此次根因分析的过程. 监控看到的线程数变化情况: 初步的 ...