Tagged: HA Toggle Comment Threads | Keyboard Shortcuts
-
Wang
-
Wang
Slurm
Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Slurm requires no kernel modifications for its operation and is relatively self-contained. As a cluster workload manager, Slurm has three key functions. First, it allocates exclusive and/or non-exclusive access to resources (compute nodes) to users for some duration of time so they can perform work. Second, it provides a framework for starting, executing, and monitoring work (normally a parallel job) on the set of allocated nodes. Finally, it arbitrates contention for resources by managing a queue of pending work.
https://slurm.schedmd.com/ -
Wang
-
Wang
-
Wang
Install Mesos/Marathon
I applied GCE recently, so I installed Mesos/Marathon for test.
Compute Engine: n1-standard-1 (1 vCPU, 3.75 GB, Intel Ivy Bridge, asia-east1-a region)
OS: CentOS 7
10.140.0.1 master 10.140.0.2 slave1 10.140.0.3 slave2 10.140.0.4 slave3
Prepare
1.install git
sudo yum install -y tar wget git
2.install and import apache maven repository
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo sudo yum install -y epel-release sudo bash -c 'cat > /etc/yum.repos.d/wandisco-svn.repo <<EOF [WANdiscoSVN] name=WANdisco SVN Repo 1.9 enabled=1 baseurl=http://opensource.wandisco.com/centos/7/svn-1.9/RPMS/$basearch/ gpgcheck=1 gpgkey=http://opensource.wandisco.com/RPM-GPG-KEY-WANdisco EOF'
3.install tools
sudo yum update systemd sudo yum groupinstall -y "Development Tools" sudo yum install -y apache-maven python-devel python-six python-virtualenv java-1.8.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel apr-util-devel
Installation
1.append hosts
cat << EOF >>/etc/hosts 10.140.0.1 master 10.140.0.2 slave1 10.140.0.3 slave2 10.140.0.4 slave3 EOF
2.zookeeper
2.1.install zookeeper on slave1/slave2/slave3
2.2.modify conf/zoo.cfg on slave1/slave2/slave3
cat << EOF > conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=./data clientPort=2181 maxClientCnxns=0 autopurge.snapRetainCount=3 autopurge.purgeInterval=0 leaderServes=yes skipAcl=no server.1=slave1:2888:3888 server.2=slave2:2889:3889 server.3=slave3:2890:3890 EOF
2.3.create data folder, and write serverid to myid on slave1/slave2/slave3, id is equals server’s sequence
mkdir data && echo ${id} > data/myid2.4.start zookeeper on slave1/slave2/slave3, check zk’s status
bin/zkServer.sh start bin/zkServer.sh status
3.mesos
3.1.install and import mesos repository on each server
rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-mesosphere
3.2.install mesos on each server
yum install mesos -y
3.3.modify mesos-master’s zk address on master/slave1
echo "zk://slave1:2181,slave2:2181,slave3:2181/mesos" >/etc/mesos/zk
3.4.modify quorum of mesos-master on master/slave1
echo 2 > /etc/mesos-master/quorum
3.5. start master and enable auto start on master/slave1
systemctl enable mesos-master.service systemctl start mesos-slave.service
3.6.start slave and enable auto start on slave1/slave2/slave3
systemctl enable mesos-slave.service systemctl start mesos-slave.service
4.marathon
4.1.install marathon on master
yum install marathon -y
4.2.config master/zk address on master
cat << EOF >>/etc/default/marathon MARATHON_MASTER="zk://slave1:2181,slave2:2181,slave3:2181/mesos" MARATHON_ZK="zk://slave1:2181,slave2:2181,slave3:2181/marathon" EOF
4.3.start marathon and enable auto start on master
systemctl enable marathon.service systemctl start marathon.service
Test
mesos: http://master:5050
marathon: http://master:8080w
Wang's Tech Blog 
Reply