在https://qingmu.io/2019/08/06/Run-Spring-Cloud-Gateway-on-kubernetes/文章中，我们搭建好了基本的网关的架子。这篇文章中
我们继续剩下的部分，将网关和部署到k8s中，并结合Ingress完成域名访问我们的网关，通过Secret完成网关HTTPS的安全加密。
流程图
- 我们可以使用阿里云的SLB作为IP入口，可以将域名直接用*.xxx.com解析到该IP上。
- IngressController采用DS+HostNetwork方式运行，管理所有的二级域名，以及HTTPS。
- 最后层就是我们的Gateway了，负责自动发现我们的业务服务，并通过Ribbon进行LB。
- SLB访问Ingress走HostNetwork，Ingress访问Gateway走IPVS，Gateway访问业务服务走pod网络。
Docker image
这里我们直接使用maven的docker插件完成一下流程.
1
|
buildFatjar --> buildDockerImage --> pushDockerImage
|
一个完整的maven插件配置如下。这个插件在我们前面的文章中已经配置好了，关于这个插件相关的详细你可以参考我的这篇文章 https://qingmu.io/2018/08/07/How-to-run-springcloud-in-docker/
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<imageName>
freemanliu/demo-gatway:v1.0.0
</imageName>
<registryUrl></registryUrl>
<workdir>/work</workdir>
<rm>true</rm>
<env>
<TZ>Asia/Shanghai</TZ>
<JAVA_OPTS>
-XX:+UnlockExperimentalVMOptions \
-XX:+UseCGroupMemoryLimitForHeap \
-XX:MaxRAMFraction=2 \
-XX:CICompilerCount=8 \
-XX:ActiveProcessorCount=8 \
-XX:+UseG1GC \
-XX:+AggressiveOpts \
-XX:+UseFastAccessorMethods \
-XX:+UseStringDeduplication \
-XX:+UseCompressedOops \
-XX:+OptimizeStringConcat
</JAVA_OPTS>
</env>
<baseImage>freemanliu/openjre:8.212</baseImage>
<cmd>
/sbin/tini java ${JAVA_OPTS} -jar ${project.build.finalName}.jar
</cmd>
<!--是否推送image-->
<pushImage>true</pushImage>
<resources>
<resource>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
<serverId>docker-hub</serverId>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
</executions>
</plugin>
|
配置好以上插件之后我们执行以下命令.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
|
$ mvn clean package -DskipTests
[INFO]
[INFO] -----------------------< io.qingmu:demo-gateway >-----------------------
[INFO] Building demo-gateway 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ demo-gateway ---
[INFO] Deleting /Users/freeman/GithubProjects/demo-gateway/target
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ demo-gateway ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ demo-gateway ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 6 source files to /Users/freeman/GithubProjects/demo-gateway/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ demo-gateway ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/freeman/GithubProjects/demo-gateway/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ demo-gateway ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /Users/freeman/GithubProjects/demo-gateway/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ demo-gateway ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ demo-gateway ---
[INFO] Building jar: /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.7.RELEASE:repackage (repackage) @ demo-gateway ---
[INFO] Replacing main artifact with repackaged archive
[INFO]
[INFO] --- docker-maven-plugin:1.2.1:build (default) @ demo-gateway ---
[INFO] Using authentication suppliers: [ConfigFileRegistryAuthSupplier, FixedRegistryAuthSupplier]
[INFO] Copying /Users/freeman/GithubProjects/demo-gateway/target/demo-gateway-0.0.1-SNAPSHOT.jar -> /Users/freeman/GithubProjects/demo-gateway/target/docker/demo-gateway-0.0.1-SNAPSHOT.jar
[INFO] Building image freemanliu/demo-gateway:v1.0.0
Step 1/6 : FROM freemanliu/openjre:8.212
---> 8a33da3f701e
Step 2/6 : ENV JAVA_OPTS -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=2 -XX:CICompilerCount=8 -XX:ActiveProcessorCount=8 -XX:+UseG1GC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseStringDeduplication -XX:+UseCompressedOops -XX:+OptimizeStringConcat
---> Using cache
---> 0f65d8120eb4
Step 3/6 : ENV TZ Asia/Shanghai
---> Using cache
---> 698b12f89dd2
Step 4/6 : WORKDIR /work
---> Using cache
---> 45e2bbd5eb93
Step 5/6 : ADD demo-gateway-0.0.1-SNAPSHOT.jar .
---> 0a76e61162bb
Step 6/6 : CMD /sbin/tini java ${JAVA_OPTS} -jar demo-gateway-0.0.1-SNAPSHOT.jar
---> Running in 034aee2b1d56
Removing intermediate container 034aee2b1d56
---> 19591ef9767b
ProgressMessage{id=null, status=null, stream=null, error=null, progress=null, progressDetail=null}
Successfully built 19591ef9767b
Successfully tagged freemanliu/demo-gateway:v1.0.0
[INFO] Built freemanliu/demo-gateway:v1.0.0
[INFO] Pushing freemanliu/demo-gateway:v1.0.0
The push refers to repository [docker.io/freemanliu/demo-gateway]
854550ddc678: Pushed
607d6bc26ce8: Pushed
e02c5cc56276: Pushed
f1b5933fe4b5: Layer already exists
v1.0.0: digest: sha256:bcc45e9b9edde415523c7e742df55486eb9d109c6864729fdf68eeefd13b3b9e size: 1158
null: null
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59.288 s
[INFO] Finished at: 2019-09-15T20:26:42+08:00
[INFO] ------------------------------------------------------------------------
|
查看
完整的执行完打包命令之后，我们的image就被推送到了docker的中央仓库
部署到K8S
完成了我们的前置工作，我们就可以编写我们的yaml描述文件
gateway.deployment.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
|
apiVersion: v1
kind: Service
metadata:
name: demo-gateway
namespace: default
labels:
app: demo-gateway
spec:
ports:
- port: 8084
name: tcp
selector:
app: demo-gateway
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-gateway
namespace: default
spec:
revisionHistoryLimit: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
replicas: 1
selector:
matchLabels:
app: demo-gateway
template:
metadata:
labels:
app: demo-gateway
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- app-gateway # 反亲和，尽量让gateway分散到不同的节点上
weight: 1
containers:
- name: demo-gateway
image: freemanliu/demo-gateway:v1.0.0
imagePullPolicy: Always
lifecycle:
preStop:
httpGet:
port: 8084
path: /spring/shutdown
livenessProbe:
httpGet:
path: /actuator/health
port: 8084
periodSeconds: 5
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /actuator/health
port: 8084
periodSeconds: 5
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 5
resources:
requests:
memory: 1Gi
limits:
memory: 1Gi
ports:
- containerPort: 8084
env:
- name: EUREKA_SERVER
value: "http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/,http://eureka-2.eureka:8761/eureka/"
- name: SAMPLER_PROBABILITY
value: "1.0"
- name: ZIPKIN
value: "http://10.96.0.13:9411/"
|
apply
1
|
$ kubectl apply -f demo.gateway.deployment.yaml
|
查看
1
2
|
$ kubectl get po -owide | grep demo-gateway
demo-gateway-c94bc5679-8ld2v 1/1 Running 0 3m35s 172.224.3.23 node1 <none> <none>
|
通过podip访问
1
2
3
4
5
|
$ curl -i 172.224.3.23:8084/demo2-service/world
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
Content-Length: 5
Date: Tue, 17 Sep 2019 06:53:46 GMT
|
查看svc
1
2
|
$ kubectl get svc | grep demo-gateway
demo-gateway ClusterIP 10.111.213.174 <none> 8084/TCP 5m10s
|
通过svcip访问
1
2
3
4
5
|
$ curl -i 10.111.213.174:8084/demo2-service/world
HTTP/1.1 200 OK
Content-Type: text/plain;charset=UTF-8
Content-Length: 5
Date: Tue, 17 Sep 2019 06:55:25 GMT
|
到此我们就将完整的demo部署到了kubernetes中。接下来我们处理ingress，以支持通过域名和https访问我们的服务。
Ingress和Https
这里我们使用域名gateway.qingmu.io访问我们的服务。首先需要申请一个HTTPS证书，自己测试使用可以在阿里云的证书服务中申请一个为期一年的，生产使用建议购买一个通配证书。
上传证书信息到K8S集群
在kubernetes中专门抽象了一个资源类型叫Secret,专门用户存储我们的上传的信息，存在这里的信息，将会被我们的ingress-controller使用，并自动配置https。
我们申请完的证书，选择下载nginx，下载之后的zip包中解压开会有两个文件2232293__gateway.qingmu.io.key,2232293__gateway.qingmu.io.pem。
1
2
|
gateway.qingmu.io_nginx freeman$ ls
2822276_gateway.qingmu.io.key 2822276_gateway.qingmu.io.pem
|
创建Secret
这里-n代表存储在哪个namespace中，这取决于你的ingress存在哪个namespace中，这里我们都是在default中。
1
2
3
4
|
$ kubectl -n default create secret tls qingmu-secret \
--key 2822276_gateway.qingmu.io.key \
--cert 2822276_gateway.qingmu.io.pem
secret/qingmu-secret created
|
查看Secret
1
2
3
|
$ kubectl get secret -ndefault qingmu-secret
NAME TYPE DATA AGE
qingmu-secret kubernetes.io/tls 2 84s
|
Ingress
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: gateway-ingress
namespace: default
annotations:
# 是否强制重定向到https
ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- gateway.qingmu.io
# 这里需要和我们上面的一致
secretName: qingmu-secret
rules:
- host: gateway.qingmu.io
http:
paths:
- backend:
serviceName: demo-gateway
servicePort: 8084
|
部署Ingress
1
2
|
$ kubectl apply -f demo.gateway.ingress.yaml
ingress.networking.k8s.io/gateway-ingress created
|
查看
1
2
3
|
$ kubectl get ingress gateway-ingress
NAME HOSTS ADDRESS PORTS AGE
gateway-ingress gateway.qingmu.io 80, 443 53s
|
通过域名访问
HTTP访问
访问http端口的时候，服务器会返回重定向308（永久重定向）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
$ curl -i gateway.qingmu.io/demo2-service/world
HTTP/1.1 308 Permanent Redirect
Server: openresty/1.15.8.1
Date: Tue, 17 Sep 2019 07:41:08 GMT
Content-Type: text/html
Content-Length: 177
Connection: keep-alive
Location: https://gateway.qingmu.io/demo2-service/world
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>
|
Https访问
1
2
3
4
5
6
7
|
$ curl -i https://gateway.qingmu.io/demo2-service/world
HTTP/2 200
server: openresty/1.15.8.1
date: Tue, 17 Sep 2019 07:40:06 GMT
content-type: text/plain;charset=UTF-8
content-length: 5
strict-transport-security: max-age=15724800; includeSubDomains
|
到此我们就完整的spring cloud gateway部署到了kubernetes中。
Github