Winse Blog

走走停停都是风景, 熙熙攘攘都向最好, 忙忙碌碌都为明朝, 何畏之.

使用Puppet安装配置Ganglia

前面写过完全纯手工和用yum安装依赖来安装ganglia的文章，最近生产安装了puppet，既然已经手上已有牛刀，杀鸡就不用再取菜刀了。今天记录下前几天使用puppet安装ganglia的经历。

前提（自己操作过熟悉怎么用）

  • 配置过私有仓库 (createrepo)
  • 安装好puppet
  • 编译过自己的rpm (rpmbuild)

编译gmetad，gmond，gweb

点击链接下载SPEC：

然后编译打包：

先手动编译安装 ganglia ，把依赖的问题处理好。编译安装没问题，然后再使用 rpmbuild 编译生成 rpm 包！！

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# 1> 建立目录结构
mkdir ganglia-build
cd ganglia-build
mkdir BUILD RPMS SOURCES SPECS SRPMS

# 2> 修改配置
# ganglia-web-3.7.1.tar.gz的makefile、conf_default.php.in修改下，根据等下要配置gmetad的参数进行修改

less ganglia-web-3.7.1/Makefile 
  # Location where gweb should be installed to (excluding conf, dwoo dirs).
  GDESTDIR = /var/www/html/ganglia

  # Location where default apache configuration should be installed to.
  GCONFDIR = /usr/local/ganglia/etc/

  # Gweb statedir (where conf dir and Dwoo templates dir are stored)
  GWEB_STATEDIR = /var/www/html/ganglia

  # Gmetad rootdir (parent location of rrd folder)
  GMETAD_ROOTDIR = /data/ganglia

  APACHE_USER = apache

# 连外网太慢，下载放到本地
less ganglia-web-3.7.1/conf_default.php.in 
  #$conf['cubism_js_path'] = "js/cubism.v1.min.js";
  $conf['jquery_js_path'] = "js/jquery.min.js";
  $conf['jquerymobile_js_path'] = "js/jquery.mobile.min.js";
  $conf['jqueryui_js_path'] = "js/jquery-ui.min.js";
  $conf['rickshaw_js_path'] = "js/rickshaw.min.js";
  $conf['cubism_js_path'] = "js/cubism.v1.min.js";
  $conf['d3_js_path'] = "js/d3.min.js";
  $conf['protovis_js_path'] = "js/protovis.min.js";

# 3> 源文件
# 把文件放到SOURCES目录下，
ls SOURCES/
  ganglia-3.7.2.tar.gz  ganglia-web-3.7.1.tar.gz

# 4> 编译生成RPM
rpmbuild -v -ba SPECS/gmetad.spec 
rpmbuild -v -ba SPECS/gmond.spec 
rpmbuild -v -ba SPECS/gweb.spec 

# 5> 查看内容
rpm -qpl RPMS/x86_64/ganglia-3.7.2-1.el6.x86_64.rpm

本地仓库

这里假设已经把系统光盘做成了本地仓库。

先安装httpd、php、createrepo，然后按照下面的步骤创建本地仓库：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# 系统带的可以从光盘拷贝，直接映射到httpd的目录下即可
[hadoop@hadoop-master1 rhel6.3]$ ls 
Packages  repodata
[hadoop@hadoop-master1 html]$ pwd
/var/www/html
[hadoop@hadoop-master1 html]$ ll
lrwxrwxrwx.  1 root root   20 2月  15 2014 rhel6.3 -> /opt/rhel6.3

[hadoop@hadoop-master1 ~]$ sudo mkdir -p /opt/dta/repo
[hadoop@hadoop-master1 ~]$ cd /opt/dta/repo
[hadoop@hadoop-master1 repo]$ ls *.rpm
gmetad-3.7.2-1.el6.x86_64.rpm  gmond-3.7.2-1.el6.x86_64.rpm  gweb-3.7.1-1.el6.x86_64.rpm  libconfuse-2.7-4.el6.x86_64.rpm

[hadoop@hadoop-master1 repo]$ sudo createrepo .
3/3 - libconfuse-2.7-4.el6.x86_64.rpm                                           
Saving Primary metadata
Saving file lists metadata
Saving other metadata

# 映射到httpd目录下
[hadoop@hadoop-master1 yum.repos.d]$ cd /var/www/html/
[hadoop@hadoop-master1 html]$ sudo ln -s /opt/dta/repo dta

# 加入本地仓库源
[hadoop@hadoop-master1 yum.repos.d]$ sudo cp puppet.repo dta.repo
[hadoop@hadoop-master1 yum.repos.d]$ sudo vi dta.repo 
[dta]
name=DTA Local
baseurl=http://hadoop-master1:801/dta
enabled=1
gpgcheck=0

注意： 在安装的时刻找不到gmond，可以先清理yum的缓冲： yum clean all

puppet模块

添加了三个模块，用于主机添加repo配置和sudo配置，以及安装配置gmond。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

[root@hadoop-master1 modules]# tree $PWD
/etc/puppetlabs/code/environments/production/modules
├── dtarepo
│   ├── manifests
│   │   └── init.pp
│   └── templates
│       └── dta.repo
├── gmond
│   ├── manifests
│   │   └── init.pp
│   └── templates
│       └── gmond.conf
└── sudo
    ├── manifests
    │   └── init.pp
    └── templates
        └── sudo.erb

都比较简单，通过init.pp来进行配置，然后加载模板，写入到同步主机本地文件中。

  • dtarepo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

./dtarepo/manifests/init.pp
class dtarepo {

file{'/etc/yum.repos.d/dta.repo':
  ensure => file,
  content => template('dtarepo/dta.repo'),
}

}

./dtarepo/templates/dta.repo
[dta]
name=DTA Local
baseurl=http://hadoop-master1:801/dta
enabled=1
gpgcheck=0
  • sudo：用于sudo切root，方便调试
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

./sudo/manifests/init.pp
class sudo {

if ( $::hostname =~ /(^cu-omc)/ ) {
  $user = 'omc'
} elsif ( $::hostname =~ /(^cu-uc)/ ) {
  $user = 'uc'
} elsif ( $::hostname =~ /(^cu-ud)/ ) {
  $user = 'ud'
} elsif ( $::hostname =~ /(^cu-db)/ ) {
  $user = 'mysql'
} else {
  $user = 'hadoop'
}


file { "/etc/sudoers.d/10_$user":
  ensure => file,
  mode => '0440', 
  content => template('sudo/sudo.erb'),
}


}

./sudo/templates/sudo.erb
<%= scope.lookupvar('sudo::user') %> ALL=(ALL) NOPASSWD: ALL
  • gmond

在默认的gmond.conf基础上修改一下两个配置: globals.deaf, cluster.name

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

./gmond/manifests/init.pp
class gmond {

$deaf = $::hostname ? {
  'hadoop-master1' => 'no',
  'cu-omc1' => 'no',
  default => 'yes',
}

if ( $::hostname =~ /(^cu-)/ ) {
  $cluster_name = 'CU'
} else {
  $cluster_name = 'HADOOP'
}

package { 'gmond':
  ensure => present,
  before => File['/usr/local/ganglia/etc/gmond.conf'],
}

file { '/usr/local/ganglia/etc/gmond.conf':
  ensure => file,
  content => template('gmond/gmond.conf'),
  notify => Service['gmond'],
}

service { 'gmond':
  ensure    => running,
  enable    => true,
}

}

./gmond/templates/gmond.conf
/* This configuration is as close to 2.5.x default behavior as possible
   The values closely match ./gmond/metric.h definitions in 2.5.x */
globals {
...
  mute = no
  deaf = <%= scope.lookupvar('gmond::deaf') %>
  allow_extra_data = yes
...
cluster {
  name = "<%= scope.lookupvar('gmond::cluster_name') %>"

参考下逻辑即可（也可以通过hiera配置）。

最后在 site.pp 引用加载编写的Module：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

[root@hadoop-master1 modules]# cd ../manifests/
[root@hadoop-master1 manifests]# cat site.pp 
file{'/etc/puppetlabs/mcollective/facts.yaml':
  owner    => root,
  group    => root,
  mode     => '400',
  loglevel => debug, # reduce noise in Puppet reports
  content  => inline_template("<%= scope.to_hash.reject { |k,v| k.to_s =~ /(uptime_seconds|timestamp|free)/ }.to_yaml %>"), # exclude rapidly changing facts
}


include dtarepo
include gmond

# include sudo

一键安装

安装gmetad：

首先在主机上安装gmetad，由于只需要在一台机器安装，配置没有整成模板，这里直接手动弄。

1
2
3
4
5
6
7
8
9
10

[root@hadoop-master1 dtarepo]# mco rpc package install package=gmetad -I cu-omc1 （或者直接yum install -y gmetad）

# 注意：主机多网卡时可能需要添加route
[root@cu-omc1 ~]# route add -host 239.2.11.71 dev bond0

[root@cu-omc1 ~]# /etc/ganglia/gmetad.conf 注意!! 这里的rrd_rootdir配置与上面gweb/makefile是对应的！！
data_source "HADOOP" hadoop-master1
data_source "CU" cu-omc1
gridname "CQCU"
rrd_rootdir "/data/ganglia/rrds"

注意： data_source 需要配合 gmond/manifests/init.pp 中的 deaf 属性值。

php的时区调整：vi /etc/php.ini date.timezone = “Asia/Shanghai”

安装gmond：

在cu-omc2上安装gmond（正则表达式，想怎么匹配就怎么写）：

1

[root@hadoop-master1 production]# mco shell -I /^cu-omc2/ run -- "/opt/puppetlabs/bin/puppet agent -t"

puppet同步好后，就安装好puppet，以及启动gmond服务。

同时看看web是否已经有图像。不要看一分钟负载，搞一个明显一点的，如磁盘容量内存容量可以明确判断数据有没有采集到的。 没有话可以试着重启gmond:

1

[root@hadoop-master1 production]# mco shell -I /cu-/ run -- service gmond restart

不要一次重启太多机器，时间比较长的话可以结合screen命令使用：

1
2
3
4
5
6

[root@hadoop-master1 ~]# screen
[root@hadoop-master1 ~]# mco shell -I hadoop-slaver2 -I hadoop-slaver3 -I hadoop-slaver4  -I hadoop-slaver5 -I hadoop-slaver6 -I hadoop-slaver7 -I hadoop-slaver8 -I hadoop-slaver9  run -- service gmond restart ; 
[root@hadoop-master1 ~]# for ((i=1;i<17;i++)) ; do mco shell -I /hadoop-slaver${i}.$/ run -- service gmond restart ; sleep 60 ; done  

[root@hadoop-master1 ~]# screen -ls 
[root@hadoop-master1 ~]# screen -r 22929

–END

Comments