Closed
Description
Issue description
Apisix cannot work when etcd auth is enabled, but apisix-dashboard does.
Did I miss any key configuration items?
STEP 1: create the etcd user and role
# etcdctl user get apisix
User: apisix
Roles: apisix
# etcdctl role get apisix
Role apisix
KV Read:
[/apisix, /apisiy) (prefix /apisix)
KV Write:
[/apisix, /apisiy) (prefix /apisix)
STEP 2: enable etcd auth
#etcdctl auth enable
Authentication Enabled
STEP 3: create apisix ssl pems
#cat apisix-csr.json
{
"CN": "apisix",
"hosts": [
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "DUMMY",
"L": "DUMMY",
"O": "etcd",
"OU": "DUMMY"
}
]
}
STEP 4: make the etcd tls config
# cat conf/config.yaml
nginx_config:
http:
lua_shared_dicts:
s_cache: 8m
enable_cpu_affinity: false
apisix:
node_listen: 80
ssl:
listen_port: 443
ssl_trusted_certificate: /etc/pki/tls/certs/ca-bundle.crt
key_encrypt_salt: -
allow_admin: [ 127.0.0.0/24, 0.0.0.0/0 ]
enable_ipv6: false
admin_key:
- name: "admin"
key: -
role: admin
etcd:
host: [ https://prod-k8s0.x.com:2379,https://prod-k8s1.x.com:2379,https://prod-k8s2.x.com:2379 ]
prefix: /apisix-7ac5eb62
timeout: 10
health_check_timeout: 60
ssl_trusted_certificate: /tmp/ca.pem
tls:
verify: true
cert: /tmp/apisix.pem
key: /tmp/apisix-key.pem
STEP 5 boot apisix
# apisix start
/usr/bin/apisix: line 32: which: command not found
lua /usr/local/apisix/apisix/cli/apisix.lua start
lua_shared_dicts is deprecated, use custom_lua_shared_dict instead
nginx: [error] [lua] init.lua:86: http_init(): failed to load the configuration: failed to read etcd dir
# etcdctl get --prefix /api
STEP 6 try use dashboard with same etcd TLS config create a new route
# cat
......
etcd:
endpoints:
- https://prod-k8s0.x.com:2379
- https://prod-k8s1.x.com:2379
- https://prod-k8s2.x.com:2379
mtls:
key_file: " /tmp/apisix-key.pem"
cert_file: "/tmp/apisix.pem"
ca_file: "/tmp/ca.pem"
prefix: /apisix-7ac5eb62
......
# etcdctl get --prefix /api
/apisix-7ac5eb62/routes/383017729642202494
{"id":"383017729642202494","create_time":1637825957,"update_time":1637825957,"uri":"/*","name":"1","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"test.x.om","upstream":{"nodes":{"127.0.0.1:9000":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"labels":{"API_VERSION":"1"},"status":1}
STEP 7 disable etcd auth and reboot apisix
# etcdctl get --prefix /api
/apisix-7ac5eb62/consumers/
init_dir
/apisix-7ac5eb62/global_rules/
init_dir
/apisix-7ac5eb62/plugin_configs/
init_dir
/apisix-7ac5eb62/plugin_metadata/
init_dir
/apisix-7ac5eb62/plugins/
init_dir
/apisix-7ac5eb62/proto/
init_dir
/apisix-7ac5eb62/routes/
init_dir
/apisix-7ac5eb62/services/
init_dir
/apisix-7ac5eb62/ssl/
init_dir
/apisix-7ac5eb62/stream_routes/
init_dir
/apisix-7ac5eb62/upstreams/
init_dir
/apisix-7ac5eb62/routes/383017729642202494
{"id":"383017729642202494","create_time":1637825957,"update_time":1637825957,"uri":"/*","name":"1","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"test.x.om","upstream":{"nodes":{"127.0.0.1:9000":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"labels":{"API_VERSION":"1"},"status":1}
Environment
- apisix version : 2.10.1
- OS : docker image: centos:7
- OpenResty / Nginx version : nginx version: openresty/1.19.9.1
- etcd version: etcd Version: 3.4.3
Metadata
Metadata
Assignees
Labels
No labels
Activity
tokers commentedon Nov 25, 2021
It seems that you don't configure the username and password fields in APISIX configuration, so it won't carry the credential to ETCD.
MizuhaHimuraki commentedon Nov 25, 2021
Thanks for your reply. I think username and password are not necessary. If an etcd server is launched with the option --client-cert-auth=true, the field of Common Name (CN) in the client’s TLS cert will be used as an etcd user.
I did not set the username and password on dashboard config, but dashboard works well.
And my ssl pem follows this rule.
refer to https://etcd.io/docs/v3.4/op-guide/authentication/#using-tls-common-name
I force enable the debug mode of init_etcd, and got the following errmsg:
The error was ignored since the keyword "error" was missing in the response.
details see apisix and etcd
tokers commentedon Nov 26, 2021
Just in the docs you share:
APISIX uses the gRPC gateway to communicate with ETCD.
MizuhaHimuraki commentedon Nov 29, 2021
OK, this is a limitation of the etcd itself. I will add username and password fields in the config file to workaround.
I think this error message should be recognized during init_etcd. I spent a lot of time analyzing the failure, because the script did not report any errors.
tokers commentedon Nov 30, 2021
PR's welcome! We can enhance from two aspects.