Skip to content

Commit 593b009

Browse files
MattiasAngMattiasAng
authoredSep 1, 2022
Support for scraping multiple mysqld hosts (#651)
Add multi-target exporter scrape support. Breaking change: `DATA_SOURCE_NAME` is now removed. Local configuration is now supplied by `MYSQLD_EXPORTER_PASSWORD` and command line arguments. Signed-off-by: Mattias Ängehov <mattias.angehov@castoredc.com>
·
v0.17.2v0.15.0-rc.0
1 parent c7ab579 commit 593b009

13 files changed

+640
-234
lines changed
 

‎Makefile

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ STATICCHECK_IGNORE =
2222

2323
DOCKER_IMAGE_NAME ?= mysqld-exporter
2424

25-
test-docker:
26-
@echo ">> testing docker image"
25+
.PHONY: test-docker-single-exporter
26+
test-docker-single-exporter:
27+
@echo ">> testing docker image for single exporter"
2728
./test_image.sh "$(DOCKER_IMAGE_NAME):$(DOCKER_IMAGE_TAG)" 9104
2829

2930
.PHONY: test-docker

‎README.md

Lines changed: 51 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,50 @@ NOTE: It is recommended to set a max connection limit for the user to avoid over
3030

3131
### Running
3232

33-
Running using an environment variable:
34-
35-
export DATA_SOURCE_NAME='user:password@(hostname:3306)/'
36-
./mysqld_exporter <flags>
33+
##### Single exporter mode
3734

3835
Running using ~/.my.cnf:
3936

4037
./mysqld_exporter <flags>
4138

39+
##### Multi-target support
40+
41+
This exporter supports the multi-target pattern. This allows running a single instance of this exporter for multiple MySQL targets.
42+
43+
To use the multi-target functionality, send an http request to the endpoint /probe?target=foo:5432 where target is set to the DSN of the MySQL instance to scrape metrics from.
44+
45+
To avoid putting sensitive information like username and password in the URL, you can have multiple configurations in `config.my-cnf` file and match it by adding `&auth_module=<section>` to the request.
46+
47+
Sample config file for multiple configurations
48+
49+
[client]
50+
user = foo
51+
password = foo123
52+
[client.servers]
53+
user = bar
54+
password = bar123
55+
56+
On the prometheus side you can set a scrape config as follows
57+
58+
- job_name: mysql # To get metrics about the mysql exporter’s targets
59+
params:
60+
# Not required. Will match value to child in config file. Default value is `client`.
61+
auth_module: client.servers
62+
static_configs:
63+
- targets:
64+
# All mysql hostnames to monitor.
65+
- server1:3306
66+
- server2:3306
67+
relabel_configs:
68+
- source_labels: [__address__]
69+
target_label: __param_target
70+
- source_labels: [__param_target]
71+
target_label: instance
72+
- target_label: __address__
73+
# The mysqld_exporter host:port
74+
replacement: localhost:9104
75+
76+
##### Flag format
4277
Example format for flags for version > 0.10.0:
4378

4479
--collect.auto_increment.columns
@@ -102,6 +137,8 @@ collect.heartbeat.utc | 5.1 | U
102137
### General Flags
103138
Name | Description
104139
-------------------------------------------|--------------------------------------------------------------------------------------------------
140+
mysqld.address | Hostname and port used for connecting to MySQL server, format: `host:port`. (default: `locahost:3306`)
141+
mysqld.username | Username to be used for connecting to MySQL Server
105142
config.my-cnf | Path to .my.cnf file to read MySQL credentials from. (default: `~/.my.cnf`)
106143
log.level | Logging verbosity (default: info)
107144
exporter.lock_wait_timeout | Set a lock_wait_timeout (in seconds) on the connection to avoid long metadata locking. (default: 2)
@@ -112,6 +149,15 @@ web.listen-address | Address to listen on for web interf
112149
web.telemetry-path | Path under which to expose metrics.
113150
version | Print the version information.
114151

152+
### Environment Variables
153+
Name | Description
154+
-------------------------------------------|--------------------------------------------------------------------------------------------------
155+
MYSQLD_EXPORTER_PASSWORD | Password to be used for connecting to MySQL Server
156+
157+
### Configuration precedence
158+
159+
If you have configured cli with both `mysqld` flags and a valid configuration file, the options in the configuration file will override the flags for `client` section.
160+
115161
## TLS and basic authentication
116162

117163
The MySQLd Exporter supports TLS and basic authentication.
@@ -120,12 +166,6 @@ To use TLS and/or basic authentication, you need to pass a configuration file
120166
using the `--web.config.file` parameter. The format of the file is described
121167
[in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md).
122168

123-
### Setting the MySQL server's data source name
124-
125-
The MySQL server's [data source name](http://en.wikipedia.org/wiki/Data_source_name)
126-
must be set via the `DATA_SOURCE_NAME` environment variable.
127-
The format of this variable is described at https://github.com/go-sql-driver/mysql#dsn-data-source-name.
128-
129169
## Customizing Configuration for a SSL Connection
130170

131171
If The MySQL server supports SSL, you may need to specify a CA truststore to verify the server's chain-of-trust. You may also need to specify a SSL keypair for the client side of the SSL connection. To configure the mysqld exporter to use a custom CA certificate, add the following to the mysql cnf file:
@@ -141,8 +181,6 @@ ssl-key=/path/to/ssl/client/key
141181
ssl-cert=/path/to/ssl/client/cert
142182
```
143183

144-
Customizing the SSL configuration is only supported in the mysql cnf file and is not supported if you set the mysql server's data source name in the environment variable DATA_SOURCE_NAME.
145-
146184

147185
## Using Docker
148186

@@ -157,9 +195,8 @@ docker pull prom/mysqld-exporter
157195
docker run -d \
158196
-p 9104:9104 \
159197
--network my-mysql-network \
160-
-e DATA_SOURCE_NAME="user:password@(hostname:3306)/" \
161198
prom/mysqld-exporter
162-
```
199+
--config.my-cnf=<path_to_cnf>
163200

164201
## heartbeat
165202

‎config/config.go

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
// Copyright 2022 The Prometheus Authors
2+
// Licensed under the Apache License, Version 2.0 (the "License");
3+
// you may not use this file except in compliance with the License.
4+
// You may obtain a copy of the License at
5+
//
6+
// http://www.apache.org/licenses/LICENSE-2.0
7+
//
8+
// Unless required by applicable law or agreed to in writing, software
9+
// distributed under the License is distributed on an "AS IS" BASIS,
10+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
// See the License for the specific language governing permissions and
12+
// limitations under the License.
13+
14+
package config
15+
16+
import (
17+
"crypto/tls"
18+
"crypto/x509"
19+
"fmt"
20+
"net"
21+
"os"
22+
"sync"
23+
24+
"github.com/go-kit/log"
25+
"github.com/go-kit/log/level"
26+
27+
"github.com/go-sql-driver/mysql"
28+
"github.com/prometheus/client_golang/prometheus"
29+
30+
"gopkg.in/ini.v1"
31+
)
32+
33+
var (
34+
configReloadSuccess = prometheus.NewGauge(prometheus.GaugeOpts{
35+
Namespace: "mysqld_exporter",
36+
Name: "config_last_reload_successful",
37+
Help: "Mysqld exporter config loaded successfully.",
38+
})
39+
40+
configReloadSeconds = prometheus.NewGauge(prometheus.GaugeOpts{
41+
Namespace: "mysqld_exporter",
42+
Name: "config_last_reload_success_timestamp_seconds",
43+
Help: "Timestamp of the last successful configuration reload.",
44+
})
45+
46+
cfg *ini.File
47+
48+
opts = ini.LoadOptions{
49+
// Do not error on nonexistent file to allow empty string as filename input
50+
Loose: true,
51+
// MySQL ini file can have boolean keys.
52+
AllowBooleanKeys: true,
53+
}
54+
55+
err error
56+
)
57+
58+
type Config struct {
59+
Sections map[string]MySqlConfig
60+
}
61+
62+
type MySqlConfig struct {
63+
User string `ini:"user"`
64+
Password string `ini:"password"`
65+
Host string `ini:"host"`
66+
Port int `ini:"port"`
67+
Socket string `ini:"socket"`
68+
SslCa string `ini:"ssl-ca"`
69+
SslCert string `ini:"ssl-cert"`
70+
SslKey string `ini:"ssl-key"`
71+
TlsInsecureSkipVerify bool `ini:"ssl-skip-verfication"`
72+
}
73+
74+
type MySqlConfigHandler struct {
75+
sync.RWMutex
76+
TlsInsecureSkipVerify bool
77+
Config *Config
78+
}
79+
80+
func (ch *MySqlConfigHandler) GetConfig() *Config {
81+
ch.RLock()
82+
defer ch.RUnlock()
83+
return ch.Config
84+
}
85+
86+
func (ch *MySqlConfigHandler) ReloadConfig(filename string, mysqldAddress string, mysqldUser string, tlsInsecureSkipVerify bool, logger log.Logger) error {
87+
var host, port string
88+
defer func() {
89+
if err != nil {
90+
configReloadSuccess.Set(0)
91+
} else {
92+
configReloadSuccess.Set(1)
93+
configReloadSeconds.SetToCurrentTime()
94+
}
95+
}()
96+
97+
if cfg, err = ini.LoadSources(
98+
opts,
99+
[]byte("[client]\npassword = ${MYSQLD_EXPORTER_PASSWORD}\n"),
100+
filename,
101+
); err != nil {
102+
return fmt.Errorf("failed to load %s: %w", filename, err)
103+
}
104+
105+
if host, port, err = net.SplitHostPort(mysqldAddress); err != nil {
106+
return fmt.Errorf("failed to parse address: %w", err)
107+
}
108+
109+
if clientSection := cfg.Section("client"); clientSection != nil {
110+
if cfgHost := clientSection.Key("host"); cfgHost.String() == "" {
111+
cfgHost.SetValue(host)
112+
}
113+
if cfgPort := clientSection.Key("port"); cfgPort.String() == "" {
114+
cfgPort.SetValue(port)
115+
}
116+
if cfgUser := clientSection.Key("user"); cfgUser.String() == "" {
117+
cfgUser.SetValue(mysqldUser)
118+
}
119+
}
120+
121+
cfg.ValueMapper = os.ExpandEnv
122+
config := &Config{}
123+
m := make(map[string]MySqlConfig)
124+
for _, sec := range cfg.Sections() {
125+
sectionName := sec.Name()
126+
127+
if sectionName == "DEFAULT" {
128+
continue
129+
}
130+
131+
mysqlcfg := &MySqlConfig{
132+
TlsInsecureSkipVerify: tlsInsecureSkipVerify,
133+
}
134+
if err != nil {
135+
level.Error(logger).Log("msg", "failed to load config", "section", sectionName, "err", err)
136+
continue
137+
}
138+
139+
err = sec.StrictMapTo(mysqlcfg)
140+
if err != nil {
141+
level.Error(logger).Log("msg", "failed to parse config", "section", sectionName, "err", err)
142+
continue
143+
}
144+
if err := mysqlcfg.validateConfig(); err != nil {
145+
level.Error(logger).Log("msg", "failed to validate config", "section", sectionName, "err", err)
146+
continue
147+
}
148+
149+
m[sectionName] = *mysqlcfg
150+
}
151+
config.Sections = m
152+
if len(config.Sections) == 0 {
153+
return fmt.Errorf("no configuration found")
154+
}
155+
ch.Lock()
156+
ch.Config = config
157+
ch.Unlock()
158+
return nil
159+
}
160+
161+
func (m MySqlConfig) validateConfig() error {
162+
if m.User == "" {
163+
return fmt.Errorf("no user specified in section or parent")
164+
}
165+
if m.Password == "" {
166+
return fmt.Errorf("no password specified in section or parent")
167+
}
168+
169+
return nil
170+
}
171+
172+
func (m MySqlConfig) FormDSN(target string) (string, error) {
173+
var dsn, host, port string
174+
175+
user := m.User
176+
password := m.Password
177+
if target == "" {
178+
host := m.Host
179+
port := m.Port
180+
socket := m.Socket
181+
if socket != "" {
182+
dsn = fmt.Sprintf("%s:%s@unix(%s)/", user, password, socket)
183+
} else {
184+
dsn = fmt.Sprintf("%s:%s@tcp(%s:%d)/", user, password, host, port)
185+
}
186+
} else {
187+
if host, port, err = net.SplitHostPort(target); err != nil {
188+
return dsn, fmt.Errorf("failed to parse target: %s", err)
189+
}
190+
dsn = fmt.Sprintf("%s:%s@tcp(%s:%s)/", user, password, host, port)
191+
}
192+
193+
if m.SslCa != "" {
194+
if err := m.CustomizeTLS(); err != nil {
195+
err = fmt.Errorf("failed to register a custom TLS configuration for mysql dsn: %w", err)
196+
return dsn, err
197+
}
198+
dsn = fmt.Sprintf("%s?tls=custom", dsn)
199+
}
200+
201+
return dsn, nil
202+
}
203+
204+
func (m MySqlConfig) CustomizeTLS() error {
205+
var tlsCfg tls.Config
206+
caBundle := x509.NewCertPool()
207+
pemCA, err := os.ReadFile(m.SslCa)
208+
if err != nil {
209+
return err
210+
}
211+
if ok := caBundle.AppendCertsFromPEM(pemCA); ok {
212+
tlsCfg.RootCAs = caBundle
213+
} else {
214+
return fmt.Errorf("failed parse pem-encoded CA certificates from %s", m.SslCa)
215+
}
216+
if m.SslCert != "" && m.SslKey != "" {
217+
certPairs := make([]tls.Certificate, 0, 1)
218+
keypair, err := tls.LoadX509KeyPair(m.SslCert, m.SslKey)
219+
if err != nil {
220+
return fmt.Errorf("failed to parse pem-encoded SSL cert %s or SSL key %s: %w",
221+
m.SslCert, m.SslKey, err)
222+
}
223+
certPairs = append(certPairs, keypair)
224+
tlsCfg.Certificates = certPairs
225+
}
226+
tlsCfg.InsecureSkipVerify = m.TlsInsecureSkipVerify
227+
mysql.RegisterTLSConfig("custom", &tlsCfg)
228+
return nil
229+
}

‎config/config_test.go

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
// Copyright 2022 The Prometheus Authors
2+
// Licensed under the Apache License, Version 2.0 (the "License");
3+
// you may not use this file except in compliance with the License.
4+
// You may obtain a copy of the License at
5+
//
6+
// http://www.apache.org/licenses/LICENSE-2.0
7+
//
8+
// Unless required by applicable law or agreed to in writing, software
9+
// distributed under the License is distributed on an "AS IS" BASIS,
10+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
// See the License for the specific language governing permissions and
12+
// limitations under the License.
13+
14+
package config
15+
16+
import (
17+
"fmt"
18+
"os"
19+
"testing"
20+
21+
"github.com/go-kit/log"
22+
23+
"github.com/smartystreets/goconvey/convey"
24+
)
25+
26+
func TestValidateConfig(t *testing.T) {
27+
convey.Convey("Working config validation", t, func() {
28+
c := MySqlConfigHandler{
29+
Config: &Config{},
30+
}
31+
if err := c.ReloadConfig("testdata/client.cnf", "localhost:3306", "", true, log.NewNopLogger()); err != nil {
32+
t.Error(err)
33+
}
34+
35+
convey.Convey("Valid configuration", func() {
36+
cfg := c.GetConfig()
37+
convey.So(cfg.Sections, convey.ShouldContainKey, "client")
38+
convey.So(cfg.Sections, convey.ShouldContainKey, "client.server1")
39+
40+
section, ok := cfg.Sections["client"]
41+
convey.So(ok, convey.ShouldBeTrue)
42+
convey.So(section.User, convey.ShouldEqual, "root")
43+
convey.So(section.Password, convey.ShouldEqual, "abc")
44+
45+
childSection, ok := cfg.Sections["client.server1"]
46+
convey.So(ok, convey.ShouldBeTrue)
47+
convey.So(childSection.User, convey.ShouldEqual, "test")
48+
convey.So(childSection.Password, convey.ShouldEqual, "foo")
49+
50+
})
51+
52+
convey.Convey("False on non-existent section", func() {
53+
cfg := c.GetConfig()
54+
_, ok := cfg.Sections["fakeclient"]
55+
convey.So(ok, convey.ShouldBeFalse)
56+
})
57+
})
58+
59+
convey.Convey("Inherit from parent section", t, func() {
60+
c := MySqlConfigHandler{
61+
Config: &Config{},
62+
}
63+
if err := c.ReloadConfig("testdata/child_client.cnf", "localhost:3306", "", true, log.NewNopLogger()); err != nil {
64+
t.Error(err)
65+
}
66+
cfg := c.GetConfig()
67+
section, _ := cfg.Sections["client.server1"]
68+
convey.So(section.Password, convey.ShouldEqual, "abc")
69+
})
70+
71+
convey.Convey("Environment variable / CLI flags", t, func() {
72+
c := MySqlConfigHandler{
73+
Config: &Config{},
74+
}
75+
os.Setenv("MYSQLD_EXPORTER_PASSWORD", "supersecretpassword")
76+
if err := c.ReloadConfig("", "testhost:5000", "testuser", true, log.NewNopLogger()); err != nil {
77+
t.Error(err)
78+
}
79+
80+
cfg := c.GetConfig()
81+
section := cfg.Sections["client"]
82+
convey.So(section.Host, convey.ShouldEqual, "testhost")
83+
convey.So(section.Port, convey.ShouldEqual, 5000)
84+
convey.So(section.User, convey.ShouldEqual, "testuser")
85+
convey.So(section.Password, convey.ShouldEqual, "supersecretpassword")
86+
})
87+
88+
convey.Convey("Environment variable / CLI flags error without port", t, func() {
89+
c := MySqlConfigHandler{
90+
Config: &Config{},
91+
}
92+
os.Setenv("MYSQLD_EXPORTER_PASSWORD", "supersecretpassword")
93+
err := c.ReloadConfig("", "testhost", "testuser", true, log.NewNopLogger())
94+
convey.So(
95+
err,
96+
convey.ShouldBeError,
97+
)
98+
})
99+
100+
convey.Convey("Config file precedence over environment variables", t, func() {
101+
c := MySqlConfigHandler{
102+
Config: &Config{},
103+
}
104+
os.Setenv("MYSQLD_EXPORTER_PASSWORD", "supersecretpassword")
105+
if err := c.ReloadConfig("testdata/client.cnf", "localhost:3306", "fakeuser", true, log.NewNopLogger()); err != nil {
106+
t.Error(err)
107+
}
108+
109+
cfg := c.GetConfig()
110+
section := cfg.Sections["client"]
111+
convey.So(section.User, convey.ShouldEqual, "root")
112+
convey.So(section.Password, convey.ShouldEqual, "abc")
113+
})
114+
115+
convey.Convey("Client without user", t, func() {
116+
c := MySqlConfigHandler{
117+
Config: &Config{},
118+
}
119+
os.Clearenv()
120+
err := c.ReloadConfig("testdata/missing_user.cnf", "localhost:3306", "", true, log.NewNopLogger())
121+
convey.So(
122+
err,
123+
convey.ShouldResemble,
124+
fmt.Errorf("no configuration found"),
125+
)
126+
})
127+
128+
convey.Convey("Client without password", t, func() {
129+
c := MySqlConfigHandler{
130+
Config: &Config{},
131+
}
132+
os.Clearenv()
133+
err := c.ReloadConfig("testdata/missing_password.cnf", "localhost:3306", "", true, log.NewNopLogger())
134+
convey.So(
135+
err,
136+
convey.ShouldResemble,
137+
fmt.Errorf("no configuration found"),
138+
)
139+
})
140+
}
141+
142+
func TestFormDSN(t *testing.T) {
143+
var (
144+
c = MySqlConfigHandler{
145+
Config: &Config{},
146+
}
147+
err error
148+
dsn string
149+
)
150+
151+
convey.Convey("Host exporter dsn", t, func() {
152+
if err := c.ReloadConfig("testdata/client.cnf", "localhost:3306", "", true, log.NewNopLogger()); err != nil {
153+
t.Error(err)
154+
}
155+
convey.Convey("Default Client", func() {
156+
cfg := c.GetConfig()
157+
section, _ := cfg.Sections["client"]
158+
if dsn, err = section.FormDSN(""); err != nil {
159+
t.Error(err)
160+
}
161+
convey.So(dsn, convey.ShouldEqual, "root:abc@tcp(server2:3306)/")
162+
})
163+
convey.Convey("Target specific with explicit port", func() {
164+
cfg := c.GetConfig()
165+
section, _ := cfg.Sections["client.server1"]
166+
if dsn, err = section.FormDSN("server1:5000"); err != nil {
167+
t.Error(err)
168+
}
169+
convey.So(dsn, convey.ShouldEqual, "test:foo@tcp(server1:5000)/")
170+
})
171+
})
172+
}

‎config/testdata/child_client.cnf

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
[client]
2+
user = root
3+
password = abc
4+
[client.server1]
5+
user = root

‎config/testdata/client.cnf

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
[client]
2+
user = root
3+
password = abc
4+
host = server2
5+
[client.server1]
6+
user = test
7+
password = foo

‎config/testdata/missing_password.cnf

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[client]
2+
user = abc

‎config/testdata/missing_user.cnf

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[client]
2+
password = abc

‎mysqld_exporter.go

Lines changed: 46 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -15,19 +15,14 @@ package main
1515

1616
import (
1717
"context"
18-
"crypto/tls"
19-
"crypto/x509"
20-
"fmt"
2118
"net/http"
2219
"os"
2320
"path"
2421
"strconv"
25-
"strings"
2622
"time"
2723

2824
"github.com/go-kit/log"
2925
"github.com/go-kit/log/level"
30-
"github.com/go-sql-driver/mysql"
3126
"github.com/prometheus/client_golang/prometheus"
3227
"github.com/prometheus/client_golang/prometheus/promhttp"
3328
"github.com/prometheus/common/promlog"
@@ -36,9 +31,9 @@ import (
3631
"github.com/prometheus/exporter-toolkit/web"
3732
webflag "github.com/prometheus/exporter-toolkit/web/kingpinflag"
3833
"gopkg.in/alecthomas/kingpin.v2"
39-
"gopkg.in/ini.v1"
4034

4135
"github.com/prometheus/mysqld_exporter/collector"
36+
"github.com/prometheus/mysqld_exporter/config"
4237
)
4338

4439
var (
@@ -59,11 +54,21 @@ var (
5954
"config.my-cnf",
6055
"Path to .my.cnf file to read MySQL credentials from.",
6156
).Default(path.Join(os.Getenv("HOME"), ".my.cnf")).String()
57+
mysqldAddress = kingpin.Flag(
58+
"mysqld.address",
59+
"Address to use for connecting to MySQL",
60+
).Default("localhost:3306").String()
61+
mysqldUser = kingpin.Flag(
62+
"mysqld.username",
63+
"Hostname to use for connecting to MySQL",
64+
).String()
6265
tlsInsecureSkipVerify = kingpin.Flag(
6366
"tls.insecure-skip-verify",
6467
"Ignore certificate and server verification when using a tls connection.",
6568
).Bool()
66-
dsn string
69+
c = config.MySqlConfigHandler{
70+
Config: &config.Config{},
71+
}
6772
)
6873

6974
// scrapers lists all possible collection methods and if they should be enabled by default.
@@ -104,76 +109,23 @@ var scrapers = map[collector.Scraper]bool{
104109
collector.ScrapeReplicaHost{}: false,
105110
}
106111

107-
func parseMycnf(config interface{}) (string, error) {
108-
var dsn string
109-
opts := ini.LoadOptions{
110-
// MySQL ini file can have boolean keys.
111-
AllowBooleanKeys: true,
112-
}
113-
cfg, err := ini.LoadSources(opts, config)
114-
if err != nil {
115-
return dsn, fmt.Errorf("failed reading ini file: %s", err)
116-
}
117-
user := cfg.Section("client").Key("user").String()
118-
password := cfg.Section("client").Key("password").String()
119-
if user == "" {
120-
return dsn, fmt.Errorf("no user specified under [client] in %s", config)
121-
}
122-
host := cfg.Section("client").Key("host").MustString("localhost")
123-
port := cfg.Section("client").Key("port").MustUint(3306)
124-
socket := cfg.Section("client").Key("socket").String()
125-
sslCA := cfg.Section("client").Key("ssl-ca").String()
126-
sslCert := cfg.Section("client").Key("ssl-cert").String()
127-
sslKey := cfg.Section("client").Key("ssl-key").String()
128-
passwordPart := ""
129-
if password != "" {
130-
passwordPart = ":" + password
131-
} else {
132-
if sslKey == "" {
133-
return dsn, fmt.Errorf("password or ssl-key should be specified under [client] in %s", config)
134-
}
135-
}
136-
if socket != "" {
137-
dsn = fmt.Sprintf("%s%s@unix(%s)/", user, passwordPart, socket)
138-
} else {
139-
dsn = fmt.Sprintf("%s%s@tcp(%s:%d)/", user, passwordPart, host, port)
140-
}
141-
if sslCA != "" {
142-
if tlsErr := customizeTLS(sslCA, sslCert, sslKey); tlsErr != nil {
143-
tlsErr = fmt.Errorf("failed to register a custom TLS configuration for mysql dsn: %s", tlsErr)
144-
return dsn, tlsErr
145-
}
146-
dsn = fmt.Sprintf("%s?tls=custom", dsn)
147-
}
112+
func filterScrapers(scrapers []collector.Scraper, collectParams []string) []collector.Scraper {
113+
filteredScrapers := scrapers
148114

149-
return dsn, nil
150-
}
115+
// Check if we have some "collect[]" query parameters.
116+
if len(collectParams) > 0 {
117+
filters := make(map[string]bool)
118+
for _, param := range collectParams {
119+
filters[param] = true
120+
}
151121

152-
func customizeTLS(sslCA string, sslCert string, sslKey string) error {
153-
var tlsCfg tls.Config
154-
caBundle := x509.NewCertPool()
155-
pemCA, err := os.ReadFile(sslCA)
156-
if err != nil {
157-
return err
158-
}
159-
if ok := caBundle.AppendCertsFromPEM(pemCA); ok {
160-
tlsCfg.RootCAs = caBundle
161-
} else {
162-
return fmt.Errorf("failed parse pem-encoded CA certificates from %s", sslCA)
163-
}
164-
if sslCert != "" && sslKey != "" {
165-
certPairs := make([]tls.Certificate, 0, 1)
166-
keypair, err := tls.LoadX509KeyPair(sslCert, sslKey)
167-
if err != nil {
168-
return fmt.Errorf("failed to parse pem-encoded SSL cert %s or SSL key %s: %s",
169-
sslCert, sslKey, err)
122+
for _, scraper := range scrapers {
123+
if filters[scraper.Name()] {
124+
filteredScrapers = append(filteredScrapers, scraper)
125+
}
170126
}
171-
certPairs = append(certPairs, keypair)
172-
tlsCfg.Certificates = certPairs
173127
}
174-
tlsCfg.InsecureSkipVerify = *tlsInsecureSkipVerify
175-
mysql.RegisterTLSConfig("custom", &tlsCfg)
176-
return nil
128+
return filteredScrapers
177129
}
178130

179131
func init() {
@@ -182,8 +134,20 @@ func init() {
182134

183135
func newHandler(metrics collector.Metrics, scrapers []collector.Scraper, logger log.Logger) http.HandlerFunc {
184136
return func(w http.ResponseWriter, r *http.Request) {
185-
filteredScrapers := scrapers
186-
params := r.URL.Query()["collect[]"]
137+
var dsn string
138+
var err error
139+
140+
cfg := c.GetConfig()
141+
cfgsection, ok := cfg.Sections["client"]
142+
if !ok {
143+
level.Error(logger).Log("msg", "Failed to parse section [client] from config file", "err", err)
144+
}
145+
if dsn, err = cfgsection.FormDSN(""); err != nil {
146+
level.Error(logger).Log("msg", "Failed to form dsn from section [client]", "err", err)
147+
}
148+
149+
collect := r.URL.Query()["collect[]"]
150+
187151
// Use request context for cancellation when connection gets closed.
188152
ctx := r.Context()
189153
// If a timeout is configured via the Prometheus header, add it to the context.
@@ -207,24 +171,11 @@ func newHandler(metrics collector.Metrics, scrapers []collector.Scraper, logger
207171
r = r.WithContext(ctx)
208172
}
209173
}
210-
level.Debug(logger).Log("msg", "collect[] params", "params", strings.Join(params, ","))
211-
212-
// Check if we have some "collect[]" query parameters.
213-
if len(params) > 0 {
214-
filters := make(map[string]bool)
215-
for _, param := range params {
216-
filters[param] = true
217-
}
218174

219-
filteredScrapers = nil
220-
for _, scraper := range scrapers {
221-
if filters[scraper.Name()] {
222-
filteredScrapers = append(filteredScrapers, scraper)
223-
}
224-
}
225-
}
175+
filteredScrapers := filterScrapers(scrapers, collect)
226176

227177
registry := prometheus.NewRegistry()
178+
228179
registry.MustRegister(collector.New(ctx, dsn, metrics, filteredScrapers, logger))
229180

230181
gatherers := prometheus.Gatherers{
@@ -276,13 +227,10 @@ func main() {
276227
level.Info(logger).Log("msg", "Starting mysqld_exporter", "version", version.Info())
277228
level.Info(logger).Log("msg", "Build context", version.BuildContext())
278229

279-
dsn = os.Getenv("DATA_SOURCE_NAME")
280-
if len(dsn) == 0 {
281-
var err error
282-
if dsn, err = parseMycnf(*configMycnf); err != nil {
283-
level.Info(logger).Log("msg", "Error parsing my.cnf", "file", *configMycnf, "err", err)
284-
os.Exit(1)
285-
}
230+
var err error
231+
if err = c.ReloadConfig(*configMycnf, *mysqldAddress, *mysqldUser, *tlsInsecureSkipVerify, logger); err != nil {
232+
level.Info(logger).Log("msg", "Error parsing host config", "file", *configMycnf, "err", err)
233+
os.Exit(1)
286234
}
287235

288236
// Register only scrapers enabled by flag.
@@ -298,6 +246,7 @@ func main() {
298246
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
299247
w.Write(landingPage)
300248
})
249+
http.HandleFunc("/probe", handleProbe(collector.NewMetrics(), enabledScrapers, logger))
301250

302251
level.Info(logger).Log("msg", "Listening on address", "address", *listenAddress)
303252
srv := &http.Server{Addr: *listenAddress}

‎mysqld_exporter_test.go

Lines changed: 36 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -28,124 +28,8 @@ import (
2828
"syscall"
2929
"testing"
3030
"time"
31-
32-
"github.com/smartystreets/goconvey/convey"
3331
)
3432

35-
func TestParseMycnf(t *testing.T) {
36-
const (
37-
tcpConfig = `
38-
[client]
39-
user = root
40-
password = abc123
41-
`
42-
tcpConfig2 = `
43-
[client]
44-
user = root
45-
password = abc123
46-
port = 3308
47-
`
48-
clientAuthConfig = `
49-
[client]
50-
user = root
51-
port = 3308
52-
ssl-ca = ca.crt
53-
ssl-cert = tls.crt
54-
ssl-key = tls.key
55-
`
56-
socketConfig = `
57-
[client]
58-
user = user
59-
password = pass
60-
socket = /var/lib/mysql/mysql.sock
61-
`
62-
socketConfig2 = `
63-
[client]
64-
user = dude
65-
password = nopassword
66-
# host and port will not be used because of socket presence
67-
host = 1.2.3.4
68-
port = 3307
69-
socket = /var/lib/mysql/mysql.sock
70-
`
71-
remoteConfig = `
72-
[client]
73-
user = dude
74-
password = nopassword
75-
host = 1.2.3.4
76-
port = 3307
77-
`
78-
ignoreBooleanKeys = `
79-
[client]
80-
user = root
81-
password = abc123
82-
83-
[mysql]
84-
skip-auto-rehash
85-
`
86-
badConfig = `
87-
[client]
88-
user = root
89-
`
90-
badConfig2 = `
91-
[client]
92-
password = abc123
93-
socket = /var/lib/mysql/mysql.sock
94-
`
95-
badConfig3 = `
96-
[hello]
97-
world = ismine
98-
`
99-
badConfig4 = `[hello`
100-
)
101-
convey.Convey("Various .my.cnf configurations", t, func() {
102-
convey.Convey("Local tcp connection", func() {
103-
dsn, _ := parseMycnf([]byte(tcpConfig))
104-
convey.So(dsn, convey.ShouldEqual, "root:abc123@tcp(localhost:3306)/")
105-
})
106-
convey.Convey("Local tcp connection on non-default port", func() {
107-
dsn, _ := parseMycnf([]byte(tcpConfig2))
108-
convey.So(dsn, convey.ShouldEqual, "root:abc123@tcp(localhost:3308)/")
109-
})
110-
convey.Convey("Authentication with client certificate and no password", func() {
111-
dsn, _ := parseMycnf([]byte(clientAuthConfig))
112-
convey.So(dsn, convey.ShouldEqual, "root@tcp(localhost:3308)/")
113-
})
114-
convey.Convey("Socket connection", func() {
115-
dsn, _ := parseMycnf([]byte(socketConfig))
116-
convey.So(dsn, convey.ShouldEqual, "user:pass@unix(/var/lib/mysql/mysql.sock)/")
117-
})
118-
convey.Convey("Socket connection ignoring defined host", func() {
119-
dsn, _ := parseMycnf([]byte(socketConfig2))
120-
convey.So(dsn, convey.ShouldEqual, "dude:nopassword@unix(/var/lib/mysql/mysql.sock)/")
121-
})
122-
convey.Convey("Remote connection", func() {
123-
dsn, _ := parseMycnf([]byte(remoteConfig))
124-
convey.So(dsn, convey.ShouldEqual, "dude:nopassword@tcp(1.2.3.4:3307)/")
125-
})
126-
convey.Convey("Ignore boolean keys", func() {
127-
dsn, _ := parseMycnf([]byte(ignoreBooleanKeys))
128-
convey.So(dsn, convey.ShouldEqual, "root:abc123@tcp(localhost:3306)/")
129-
})
130-
convey.Convey("Missed user", func() {
131-
_, err := parseMycnf([]byte(badConfig))
132-
convey.So(err, convey.ShouldBeError, fmt.Errorf("password or ssl-key should be specified under [client] in %s", badConfig))
133-
})
134-
convey.Convey("Missed password", func() {
135-
_, err := parseMycnf([]byte(badConfig2))
136-
convey.So(err, convey.ShouldBeError, fmt.Errorf("no user specified under [client] in %s", badConfig2))
137-
})
138-
convey.Convey("No [client] section", func() {
139-
_, err := parseMycnf([]byte(badConfig3))
140-
convey.So(err, convey.ShouldBeError, fmt.Errorf("no user specified under [client] in %s", badConfig3))
141-
})
142-
convey.Convey("Invalid config", func() {
143-
_, err := parseMycnf([]byte(badConfig4))
144-
convey.So(err, convey.ShouldBeError, fmt.Errorf("failed reading ini file: unclosed section: %s", badConfig4))
145-
})
146-
})
147-
}
148-
14933
// bin stores information about path of executable and attached port
15034
type bin struct {
15135
path string
@@ -195,7 +79,8 @@ func TestBin(t *testing.T) {
19579
}
19680

19781
tests := []func(*testing.T, bin){
198-
testLandingPage,
82+
testLanding,
83+
testProbe,
19984
}
20085

20186
portStart := 56000
@@ -216,7 +101,7 @@ func TestBin(t *testing.T) {
216101
})
217102
}
218103

219-
func testLandingPage(t *testing.T, data bin) {
104+
func testLanding(t *testing.T, data bin) {
220105
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
221106
defer cancel()
222107

@@ -225,8 +110,8 @@ func testLandingPage(t *testing.T, data bin) {
225110
ctx,
226111
data.path,
227112
"--web.listen-address", fmt.Sprintf(":%d", data.port),
113+
"--config.my-cnf=test_exporter.cnf",
228114
)
229-
cmd.Env = append(os.Environ(), "DATA_SOURCE_NAME=127.0.0.1:3306")
230115
if err := cmd.Start(); err != nil {
231116
t.Fatal(err)
232117
}
@@ -254,6 +139,38 @@ func testLandingPage(t *testing.T, data bin) {
254139
}
255140
}
256141

142+
func testProbe(t *testing.T, data bin) {
143+
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
144+
defer cancel()
145+
146+
// Run exporter.
147+
cmd := exec.CommandContext(
148+
ctx,
149+
data.path,
150+
"--web.listen-address", fmt.Sprintf(":%d", data.port),
151+
"--config.my-cnf=test_exporter.cnf",
152+
)
153+
if err := cmd.Start(); err != nil {
154+
t.Fatal(err)
155+
}
156+
defer cmd.Wait()
157+
defer cmd.Process.Kill()
158+
159+
// Get the main page.
160+
urlToGet := fmt.Sprintf("http://127.0.0.1:%d/probe", data.port)
161+
body, err := waitForBody(urlToGet)
162+
if err != nil {
163+
t.Fatal(err)
164+
}
165+
got := strings.TrimSpace(string(body))
166+
167+
expected := `target is required`
168+
169+
if got != expected {
170+
t.Fatalf("got '%s' but expected '%s'", got, expected)
171+
}
172+
}
173+
257174
// waitForBody is a helper function which makes http calls until http server is up
258175
// and then returns body of the successful call.
259176
func waitForBody(urlToGet string) (body []byte, err error) {

‎probe.go

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
// Copyright 2022 The Prometheus Authors
2+
// Licensed under the Apache License, Version 2.0 (the "License");
3+
// you may not use this file except in compliance with the License.
4+
// You may obtain a copy of the License at
5+
//
6+
// http://www.apache.org/licenses/LICENSE-2.0
7+
//
8+
// Unless required by applicable law or agreed to in writing, software
9+
// distributed under the License is distributed on an "AS IS" BASIS,
10+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
// See the License for the specific language governing permissions and
12+
// limitations under the License.
13+
14+
package main
15+
16+
import (
17+
"fmt"
18+
"net/http"
19+
20+
"github.com/go-kit/log"
21+
"github.com/go-kit/log/level"
22+
"github.com/prometheus/client_golang/prometheus"
23+
"github.com/prometheus/client_golang/prometheus/promhttp"
24+
"github.com/prometheus/mysqld_exporter/collector"
25+
)
26+
27+
func handleProbe(metrics collector.Metrics, scrapers []collector.Scraper, logger log.Logger) http.HandlerFunc {
28+
return func(w http.ResponseWriter, r *http.Request) {
29+
var dsn, authModule string
30+
var err error
31+
32+
ctx := r.Context()
33+
params := r.URL.Query()
34+
target := params.Get("target")
35+
if target == "" {
36+
http.Error(w, "target is required", http.StatusBadRequest)
37+
return
38+
}
39+
collectParams := r.URL.Query()["collect[]"]
40+
41+
if authModule = params.Get("auth_module"); authModule == "" {
42+
authModule = "client"
43+
}
44+
45+
cfg := c.GetConfig()
46+
cfgsection, ok := cfg.Sections[authModule]
47+
if !ok {
48+
level.Error(logger).Log("msg", fmt.Sprintf("Failed to parse section [%s] from config file", authModule), "err", err)
49+
http.Error(w, fmt.Sprintf("Error parsing config section [%s]", authModule), http.StatusBadRequest)
50+
}
51+
if dsn, err = cfgsection.FormDSN(target); err != nil {
52+
level.Error(logger).Log("msg", fmt.Sprintf("Failed to form dsn from section [%s]", authModule), "err", err)
53+
http.Error(w, fmt.Sprintf("Error forming dsn from config section [%s]", authModule), http.StatusBadRequest)
54+
}
55+
56+
probeSuccessGauge := prometheus.NewGauge(prometheus.GaugeOpts{
57+
Name: "probe_success",
58+
Help: "Displays whether or not the probe was a success",
59+
})
60+
61+
filteredScrapers := filterScrapers(scrapers, collectParams)
62+
63+
registry := prometheus.NewRegistry()
64+
registry.MustRegister(probeSuccessGauge)
65+
registry.MustRegister(collector.New(ctx, dsn, metrics, filteredScrapers, logger))
66+
67+
if err != nil {
68+
probeSuccessGauge.Set(1)
69+
http.Error(w, err.Error(), http.StatusInternalServerError)
70+
return
71+
}
72+
73+
h := promhttp.HandlerFor(registry, promhttp.HandlerOpts{})
74+
h.ServeHTTP(w, r)
75+
}
76+
}

‎test_exporter.cnf

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[client]
2+
host=localhost
3+
port=3306
4+
socket=/var/run/mysqld/mysqld.sock
5+
user=foo
6+
password=bar
7+
[client.server1]
8+
user = bar
9+
password = bar123

‎test_image.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,12 @@ wait_start() {
1515
sleep 1
1616
fi
1717
done
18-
18+
1919
exit 1
2020
}
2121

2222
docker_start() {
23-
container_id=$(docker run -d --network mysql-test -e DATA_SOURCE_NAME="root:secret@(mysql-test:3306)/" -p "${port}":"${port}" "${docker_image}")
23+
container_id=$(docker run -d --network mysql-test -p "${port}":"${port}" "${docker_image}" --config.my-cnf=test_exporter.cnf)
2424
}
2525

2626
docker_cleanup() {

0 commit comments

Comments
 (0)
Please sign in to comment.