Etcd Confd实现配置文件变化自动更新prometheus服务
一、部署步骤
1、confd部署
# wget https://github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-amd64# mv confd-0.16.0-linux-amd64 /usr/local/confd# chmod x /usr/local/confd# cd /usr/local/confd# ./confd --help2、confd配置
confd通过读取后端存储的配置信息来动态更新对应的配置文件，对应的后端存储可以是etcd，redis等，其中etcd的v3版本对应的存储后端为etcdv3
（1）创建confdir
confdir底下包含两个目录:
conf.d:confd的配置文件，主要包含配置的生成逻辑，例如模板源，后端存储对应的keys，命令执行等。
templates:配置模板Template，即基于不同组件的配置，修改为符合 Golang text templates的模板文件。
# mkdir -p /etc/confd/{conf.d,templates}（2）Template Resources
模板源配置文件是TOML格式的文件，主要包含配置的生成逻辑，例如模板源，后端存储对应的keys，命令执行等。默认目录在/etc/confd/conf.d。
参数说明：
必要参数
dest (string) - The target file.
keys (array of strings) - An array of keys.
src (string) - The relative path of a configuration template.
可选参数
gid (int) - The gid that should own the file. Defaults to the effective gid.
mode (string) - The permission mode of the file.
uid (int) - The uid that should own the file. Defaults to the effective uid.
reload_cmd (string) - The command to reload config.
check_cmd (string) - The command to check config. Use {{.src}} to reference the rendered source template.
prefix (string) - The string to prefix to keys.
例如：/etc/confd/conf.d/prometheus.conf.toml
# vim /etc/confd/conf.d/prometheus.conf.toml[template]prefix = "/prometheus"src = "prometheus.yml.tmpl"dest = "/opt/prometheus/prometheus.yml"mode = "0755"keys = ["/job",]reload_cmd = "curl -XPOST 'http://10.1.43.1:9090/-/reload'"（3）Template
Template定义了单一应用配置的模板，默认存储在/etc/confd/templates目录下，模板文件符合Go的text/template格式。
模板文件常用函数有base，get，gets，lsdir，json等。具体可参考https://github.com/kelseyhightower/confd/blob/master/docs/templates.md。
例如：/etc/confd/templates/prometheus.yml.tmpl
# vim /etc/confd/templates/prometheus.yml.tmpl# my global configglobal: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s). external_labels: monitor: 'monitor1'# Alertmanager configurationalerting: alertmanagers: - static_configs: - targets: - 10.1.43.1:9093 - 10.1.43.2:9093# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.rule_files: - "rules/*.yml" - "rules/HLY/*.yml" # - "first_rules.yml" # - "second_rules.yml"# A scrape configuration containing exactly one endpoint to scrape:# Here it's Prometheus itself.scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['10.1.43.1:9090','10.1.43.2:9090'] {{range $job_name := gets "/job/*"}} {{$jobJson := json $job_name.Value}} - job_name: '{{$jobJson.name}}' scheme: '{{$jobJson.scheme}}' metrics_path: '{{$jobJson.metrics}}' static_configs: {{$target := printf "%s/*" $job_name.Key}}{{range $ins_name := gets $target}} {{$insJson := json $ins_name.Value}} - targets: ['{{$insJson.instance}}'] labels: name: '{{$insJson.name}}' ip: '{{$insJson.ip}}' {{end}} {{end}}3、创建后端存储的配置数据
以etcdv3存储为例，在etcd中创建以下prometheus数据。
先创建job# etcdctl --endpoints="10.1.43.1:12379" put /prometheus/job/test '{"scheme":"http","metrics":"/metrics","name":"test"}'OK再创建instance# etcdctl --endpoints="10.1.43.1:12379" put /prometheus/job/test/test1 '{"name":"test1","instance":"1.1.1.1:9093","ip":"1.1.1.1"}'OK4、启动confd服务
# cd /usr/local/confd# ./confd -watch -backend etcdv3 -node http://10.1.43.1:12379 -node http://10.1.43.2:12379 -node http://10.1.43.3:12379 #后端etcd集群启动5、验证
查看生成的配置文件，已自动同步并重新加载prometheus服务
- job_name: 'test' scheme: 'http' metrics_path: '/metrics' static_configs: - targets: ['1.1.1.1:9093'] labels: name: 'test1' ip: '1.1.1.1'查看prometheus页面
已创建刚才put到etcd的监控目标！
注意：启动confd的时候只指定一个etcd node节点，就算etcd挂了，confd服务起来，配置文件也不会重置