Manage BDP by ambari

It’s boring and complicated to manage bigdata platforms, there are so many softwares need to be installed and coordinated to make them work well together, so I tried ambari to manage them.

1.run centos7 container

docker run -dit --name centos7 --privileged --publish 8080:8080 centos:7 /usr/sbin/init

2.operate container

2.1.enter container

docker exec -it centos7 bash

2.2.update yum and install tools

yum update -y && yum install -y wget

2.3.download the ambari repository

wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.0.0/ambari.repo -O /etc/yum.repos.d/ambari.repo

2.4.install the ambari

yum install -y ambari-server
yum install -y ambari-agent

2.5.install mysql as metastore, ,create mysql repo under /etc/yum.repos.d

cat << 'EOF' >/etc/yum.repos.d/mysql.5.7.repo
[mysql57-community]
name=MySQL 5.7 Community Server
baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/7/$basearch/
enabled=1
gpgcheck=0
EOF

2.6.install mysql server

yum install -y mysql-community-server

2.7.start mysql

systemctl start mysqld

2.8.create mysql user && init database

mysql_password=ambari
mysql_default_password=`grep 'temporary password' /var/log/mysqld.log | awk -F ': ' '{print $2}'`
mysql -u root -p${mysql_default_password} -e "set global validate_password_policy=0; set global validate_password_length=4;" --connect-expired-password
mysqladmin -u root -p${mysql_default_password} password ${mysql_password}
mysql -u root -p${mysql_password} -e "create database ambari default charset 'utf8'; flush privileges;"
mysql -u root -p${mysql_password} -e "grant all privileges on ambari.* to ambari@'' identified by 'ambari'; flush privileges;"
mysql -u root -p${mysql_password} -e "use ambari; source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;"

2.9.download mysql driver

driver_path=/usr/share/java
mkdir ${driver_path}
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.45/mysql-connector-java-5.1.45.jar -O ${driver_path}/mysql-connector.jar

2.10.setup ambari server, pay attention to database configuration, need select mysql manually

ambari-server setup

2.11.modify ambari database configuration

echo "server.jdbc.driver.path=${driver_path}/mysql-connector.jar" >> /etc/ambari-server/conf/ambari.properties

2.12.start ambari

ambari-server start
ambari-agent start
ambari-server setup --jdbc-db=mysql --jdbc-driver=${driver_path}/mysql-connector.jar

3.login, default accuont: admin/admin
http://localhost:8080

P.S.

The above steps are configured on single server,  if you wanna build cluster with several servers, you also need configure ssh key(please google for specific steps, it’s simple) and start ambari-agent on slave servers.

Below are screenshots of a mini cluster which was built by 4 servers: