Winse Blog

走走停停都是风景, 熙熙攘攘都向最好, 忙忙碌碌都为明朝, 何畏之.

Puppetdb安装配置

安装 PuppetDB 后，还得修改 PuppetServer 的配置。由于测试环境机器硬件一般般，把 PuppetDB 安装在 cu3。

  • cu2: master server, ca server, postgresql
  • cu3: puppetdb, agent
1
2
3
4
5
6
7

[root@cu3 puppet]# puppetdb -v
puppetdb version: 4.0.0

[root@cu2 ~]# puppetserver -v
puppetserver version: 2.3.1
[root@cu2 ~]# puppet -V
4.4.1

原来老的版本有资源(清单)导出的功能，到了Puppet4后被PuppetDB取代了。见官网文档: Inventory Service

同时老版本用ruby写的 puppet-dashboard 也没有必要安装了，前后端分离大势所趋：后端提供接口，前端用ajax来展现。

安装PuppetDB

https://docs.puppetlabs.com/puppetdb/latest/install_from_packages.html

由于天朝特殊环境，本地repo的创建参考第一篇文章: puppet4.4.1入门安装

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

[root@cu3 ~]# yum install puppetdb
Loaded plugins: fastestmirror
Setting up Install Process
Loading mirror speeds from cached hostfile
 * epel: ftp.cuhk.edu.hk
Resolving Dependencies
--> Running transaction check
---> Package puppetdb.noarch 0:4.0.0-1.el6 will be installed
--> Processing Dependency: java-1.8.0-openjdk-headless for package: puppetdb-4.0.0-1.el6.noarch
--> Running transaction check
---> Package java-1.8.0-openjdk-headless.x86_64 1:1.8.0.77-0.b03.el6_7 will be installed
--> Processing Dependency: tzdata-java >= 2014f-1 for package: 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el6_7.x86_64
--> Processing Dependency: jpackage-utils for package: 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el6_7.x86_64
--> Running transaction check
---> Package jpackage-utils.noarch 0:1.7.5-3.14.el6 will be installed
---> Package tzdata-java.noarch 0:2016c-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================================================================================================================================================
 Package                                                Arch                              Version                                            Repository                               Size
===========================================================================================================================================================================================
Installing:
 puppetdb                                               noarch                            4.0.0-1.el6                                        puppet-local                             21 M
Installing for dependencies:
 java-1.8.0-openjdk-headless                            x86_64                            1:1.8.0.77-0.b03.el6_7                             updates                                  32 M
 jpackage-utils                                         noarch                            1.7.5-3.14.el6                                     base                                     60 k
 tzdata-java                                            noarch                            2016c-1.el6                                        updates                                 179 k

Transaction Summary
===========================================================================================================================================================================================
Install       4 Package(s)

Total size: 53 M
Total download size: 53 M
Installed size: 126 M
Is this ok [y/N]: y
Downloading Packages:
(1/3): java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el6_7.x86_64.rpm                                                                                                  |  32 MB     00:00     
(2/3): puppetdb-4.0.0-1.el6.noarch.rpm                                                                                                                              |  21 MB     00:00     
(3/3): tzdata-java-2016c-1.el6.noarch.rpm                                                                                                                           | 179 kB     00:00     
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                       32 MB/s |  53 MB     00:01     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : tzdata-java-2016c-1.el6.noarch                                                                                                                                          1/4 
  Installing : jpackage-utils-1.7.5-3.14.el6.noarch                                                                                                                                    2/4 
  Installing : 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el6_7.x86_64                                                                                                               3/4 
  Installing : puppetdb-4.0.0-1.el6.noarch                                                                                                                                             4/4 
Config archive not found. Not proceeding with migration
PEM files in /etc/puppetlabs/puppetdb/ssl are missing, we will move them into place for you
Warning: Unable to find all puppet certificates to copy

  This tool requires the following certificates to exist:

  * /etc/puppetlabs/puppet/ssl/certs/ca.pem
  * /etc/puppetlabs/puppet/ssl/private_keys/cu3.esw.cn.pem
  * /etc/puppetlabs/puppet/ssl/certs/cu3.esw.cn.pem

  These files may be missing due to the fact that your host's Puppet
  certificates may not have been signed yet, probably due to the
  lack of a complete Puppet agent run. Try running puppet first, for
  example:

      puppet agent --test

  Afterwards re-run this tool then restart PuppetDB to complete the SSL
  setup:

      puppetdb ssl-setup -f
  Verifying  : jpackage-utils-1.7.5-3.14.el6.noarch                                                                                                                                    1/4 
  Verifying  : tzdata-java-2016c-1.el6.noarch                                                                                                                                          2/4 
  Verifying  : puppetdb-4.0.0-1.el6.noarch                                                                                                                                             3/4 
  Verifying  : 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el6_7.x86_64                                                                                                               4/4 

Installed:
  puppetdb.noarch 0:4.0.0-1.el6                                                                                                                                                            

Dependency Installed:
  java-1.8.0-openjdk-headless.x86_64 1:1.8.0.77-0.b03.el6_7                   jpackage-utils.noarch 0:1.7.5-3.14.el6                   tzdata-java.noarch 0:2016c-1.el6                  

Complete!

PuppetDB 需要与 puppetserver 通信，需要签名证书。如果安装之前本机 Puppet-agent 证书已签名，安装会自动把证书拷贝到 puppetdb/ssl 目录下。我们这里先签名agent再配置 puppetdb-ssl 。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

[root@cu3 ~]# puppet agent --server cu2.esw.cn --test
Info: Creating a new SSL key for cu3.esw.cn
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for cu3.esw.cn
Info: Certificate Request fingerprint (SHA256): 16:CB:A3:6D:21:69:78:D0:0D:37:1F:A7:C1:86:2E:55:7F:B1:60:77:05:EC:F5:37:81:12:28:73:61:1A:4F:20
Info: Caching certificate for ca
Exiting; no certificate found and waitforcert is disabled

# 服务端签名: puppet cert sign cu3.esw.cn

[root@cu3 ~]# puppet agent --server cu2.esw.cn --test
Info: Caching certificate for cu3.esw.cn
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for cu3.esw.cn
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for cu3.esw.cn
Info: Applying configuration version '1461159906'
Info: Creating state file /opt/puppetlabs/puppet/cache/state/state.yaml
Notice: Applied catalog in 0.02 seconds
[root@cu3 ~]# puppetdb ssl-setup -f
PEM files in /etc/puppetlabs/puppetdb/ssl are missing, we will move them into place for you
Copying files: /etc/puppetlabs/puppet/ssl/certs/ca.pem, /etc/puppetlabs/puppet/ssl/private_keys/cu3.esw.cn.pem and /etc/puppetlabs/puppet/ssl/certs/cu3.esw.cn.pem to /etc/puppetlabs/puppetdb/ssl
Backing up /etc/puppetlabs/puppetdb/conf.d/jetty.ini to /etc/puppetlabs/puppetdb/conf.d/jetty.ini.bak.1461159930 before making changes
Updated default settings from package installation for ssl-host in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.
Updated default settings from package installation for ssl-port in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.
Updated default settings from package installation for ssl-key in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.
Updated default settings from package installation for ssl-cert in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.
Updated default settings from package installation for ssl-ca-cert in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.
[root@cu3 ~]#

安装Postgres

配置好 ssl 后，下一步就是连接数据库。puppet4.4 默认配置里面只有 postgres 数据库。直接用 yum 安装，这里简单列出配置过程。

https://docs.puppetlabs.com/puppetdb/latest/configure.html#using-postgresql

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

[root@cu2 ~]# yum localinstall http://yum.postgresql.org/9.4/redhat/rhel-6-x86_64/pgdg-centos94-9.4-1.noarch.rpm
[root@cu2 ~]# yum install postgresql94-server
[root@cu2 ~]# yum install postgresql94-contrib

[root@cu2 ~]# service postgresql-9.4 initdb
Initializing database:                                     [  OK  ]
[root@cu2 ~]# service postgresql-9.4 status
postgresql-9.4 is stopped
[root@cu2 ~]# service postgresql-9.4 start
Starting postgresql-9.4 service:                           [  OK  ]


# 先查看 PGDATA 的目录！！
[root@cu2 data]# grep "PGDATA=" /etc/init.d/postgresql-9.4 
PGDATA=/usr/local/pgsql/data
OLDPGDATA=` sed -n 's/^PGDATA=//p' /etc/init.d/postgresql-$PGPREVMAJORVERSION`
NEWPGDATA=` sed -n 's/^PGDATA=//p' /etc/init.d/postgresql-$PGMAJORVERSION`


# 切换到 postgres 用户，先验证环境变量 PGDATA 是否正确！！否则自己修改 .bash_profile 文件！！
[root@cu2 puppet]# su - postgres
-bash-4.1$ echo $PGDATA
/usr/local/pgsql/data

# 创建用户
-bash-4.1$ createuser -DRSP puppetdb
Enter password for new role: 
Enter it again: 
-bash-4.1$ 
-bash-4.1$ createdb -E utf8 -O puppetdb puppetdb

-bash-4.1$ psql puppetdb -c 'create extension pg_trgm'
CREATE EXTENSION

# 配置连接选项（相当于mysql的privilege）
-bash-4.1$ vi $PGDATA/pg_hba.conf 
host    all             all              0.0.0.0/0               md5

# 重启
[root@cu2 puppet]# service postgresql-9.4 restart
Stopping postgresql-9.4 service:                           [  OK  ]
Starting postgresql-9.4 service:                           [  OK  ]

# 测试 
[root@cu2 puppet]# psql -h localhost puppetdb puppetdb
psql (9.4.5)
Type "help" for help.

puppetdb=> 
puppetdb=> \q

查看 postgres 的端口:

1
2
3
4
5
6
7
8
9
10
11

[root@cu2 puppet]# netstat -anp | grep post
tcp        0      0 0.0.0.0:5432                0.0.0.0:*                   LISTEN      8126/postmaster     
tcp        0      0 :::5432                     :::*                        LISTEN      8126/postmaster     
udp        0      0 ::1:39400                   ::1:39400                   ESTABLISHED 8126/postmaster     
unix  2      [ ACC ]     STREAM     LISTENING     954965338 8126/postmaster     /tmp/.s.PGSQL.5432

# 有客户端连上来后：
[root@cu2 ~]# netstat -anp | grep post
tcp        0      0 0.0.0.0:5432                0.0.0.0:*                   LISTEN      8126/postmaster     
tcp        0      0 192.168.0.214:5432          192.168.0.148:60626         ESTABLISHED 20589/postgres 
...

启动PuppetDB

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

[root@cu3 ~]# vi /etc/puppetlabs/puppetdb/conf.d/database.ini 
[database]
classname = org.postgresql.Driver
subprotocol = postgresql

# The database address, i.e. //HOST:PORT/DATABASE_NAME
subname = //cu2:5432/puppetdb

# Connect as a specific user
username = puppetdb

# Use a specific password
password = puppetdb

# How often (in minutes) to compact the database
# gc-interval = 60
# 通过api/name=num-active-nodes查询不到了，但是pgsql数据库中还没有删除。也可以通过 puppet node deactivate 手动执行
# node-ttl = 30d
# 默认没有设置，disabled。格式与node-ttl一样
# node-purge-ttl = 
# report-ttl = 14d

# Number of seconds before any SQL query is considered 'slow'; offending
# queries will not be interrupted, but will be logged at the WARN log level.
log-slow-statements = 10


# 注意修改，不然web-ui就只能localhost访问了！！
[root@cu3 ~]# vi /etc/puppetlabs/puppetdb/conf.d/jetty.ini
...
host = 0.0.0.0

# JVM 参数修改
[root@cu3 ~]# less /etc/sysconfig/puppetdb 
JAVA_BIN="/usr/local/jdk1.7.0_17/bin/java"
JAVA_ARGS="-XX:MaxPermSize=128m -Xmx2g"

[root@cu3 ~]# service puppetdb start
Starting puppetdb:                                         [  OK  ]
[root@cu3 ~]# 
[root@cu3 ~]# service puppetdb status
puppetdb (pid  8452) is running...

# 8081 为 puppetserver 写数据的https接口。8080 为http web-ui端口
[root@cu3 ~]# netstat -anp | grep 8081
tcp        0      0 :::8081                     :::*                        LISTEN      8794/java

查看 8080 端口通过网页查看集群的状态，现在还什么数据都获取不到，需要配置服务端把数据发送给puppetdb。

服务端配置

https://docs.puppet.com/puppetdb/latest/connect_puppet_master.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

# 安装Plug-in
# 服务端还要安装 puppetdb-termini ，不然会报错。
[root@cu2 puppet]# yum install puppetdb-termini
Loaded plugins: fastestmirror, priorities
Setting up Install Process
Loading mirror speeds from cached hostfile
 * epel: mirrors.opencas.cn
Resolving Dependencies
--> Running transaction check
---> Package puppetdb-termini.noarch 0:3.2.4-1.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================
 Package                                      Arch                               Version                                   Repository                                Size
==========================================================================================================================================================================
Installing:
 puppetdb-termini                             noarch                             3.2.4-1.el6                               puppet-local                              25 k

Transaction Summary
==========================================================================================================================================================================
Install       1 Package(s)

Total download size: 25 k
Installed size: 69 k
Is this ok [y/N]: y
Downloading Packages:
puppetdb-termini-3.2.4-1.el6.noarch.rpm                                                                                                            |  25 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : puppetdb-termini-3.2.4-1.el6.noarch                                                                                                                    1/1 
  Verifying  : puppetdb-termini-3.2.4-1.el6.noarch                                                                                                                    1/1 

Installed:
  puppetdb-termini.noarch 0:3.2.4-1.el6                                                                                                                                   

Complete!


# 注意这里URL的域名，要与CA中的名称对应！！ 设置成 cu3 是不正确的！！
# /etc/puppetlabs/puppet
[root@cu2 puppet]# vi puppetdb.conf 
[main]
server_urls = https://cu3.esw.cn:8081

[root@cu2 puppet]# vi puppet.conf 
# This file can be used to override the default puppet settings.
# See the following links for more details on what settings are available:
# - https://docs.puppetlabs.com/puppet/latest/reference/config_important_settings.html
# - https://docs.puppetlabs.com/puppet/latest/reference/config_about_settings.html
# - https://docs.puppetlabs.com/puppet/latest/reference/config_file_main.html
# - https://docs.puppetlabs.com/puppet/latest/reference/configuration.html
[master]
vardir = /opt/puppetlabs/server/data/puppetserver
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
codedir = /etc/puppetlabs/code

#autosign = true

storeconfigs = true
storeconfigs_backend = puppetdb
reports = store,puppetdb

[root@cu2 puppet]# puppet master --configprint route_file
/etc/puppetlabs/puppet/routes.yaml

[root@hadoop-master2 puppet]# vi routes.yaml 
---
master:
  facts:
    terminus: puppetdb
    cache: yaml

[root@cu2 puppet]# service puppetserver restart
Stopping puppetserver:                                     [  OK  ]
Starting puppetserver:                                     [  OK  ]

[root@cu2 puppet]# puppet agent --server cu2.esw.cn --test 
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for cu2.esw.cn
Info: Applying configuration version '1461162748'
Notice: Applied catalog in 0.01 seconds

如果 puppet-agent 服务没有启动，分别在各台机器上面执行 –test 连一下 PuppetServer，就可以在8080 puppetdb页面看到主机的数量了。

–END

Comments