Tagged: HA Toggle Comment Threads | Keyboard Shortcuts

  • Wang 19:59 on 2021-06-09 Permalink | Reply
    Tags: , , HA,   

    Slurm 

    Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Slurm requires no kernel modifications for its operation and is relatively self-contained. As a cluster workload manager, Slurm has three key functions. First, it allocates exclusive and/or non-exclusive access to resources (compute nodes) to users for some duration of time so they can perform work. Second, it provides a framework for starting, executing, and monitoring work (normally a parallel job) on the set of allocated nodes. Finally, it arbitrates contention for resources by managing a queue of pending work.

    https://slurm.schedmd.com/

     
  • Wang 22:16 on 2020-12-08 Permalink | Reply
    Tags: , HA, , ,   

    Circuit Breaker Pattern

     
  • Wang 22:41 on 2020-05-18 Permalink | Reply
    Tags: , HA, , ,   

    A Design Analysis of Cloud-based Microservices Architecture at Netflix

     
  • Wang 22:18 on 2018-01-26 Permalink | Reply
    Tags: , , , HA, marathon, Mesos, , Zookeeper   

    Install Mesos/Marathon 

    I applied GCE recently, so I installed Mesos/Marathon for test.

    Compute Engine: n1-standard-1 (1 vCPU, 3.75 GB, Intel Ivy Bridge, asia-east1-a region)

    OS: CentOS 7

    10.140.0.1 master
    10.140.0.2 slave1
    10.140.0.3 slave2
    10.140.0.4 slave3
    

    Prepare

    1.install git

    sudo yum install -y tar wget git
    

    2.install and import apache maven repository

    sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
    sudo yum install -y epel-release
    sudo bash -c 'cat > /etc/yum.repos.d/wandisco-svn.repo <<EOF
    [WANdiscoSVN]
    name=WANdisco SVN Repo 1.9
    enabled=1
    baseurl=http://opensource.wandisco.com/centos/7/svn-1.9/RPMS/$basearch/
    gpgcheck=1
    gpgkey=http://opensource.wandisco.com/RPM-GPG-KEY-WANdisco
    EOF'
    

    3.install tools

    sudo yum update systemd
    sudo yum groupinstall -y "Development Tools"
    sudo yum install -y apache-maven python-devel python-six python-virtualenv java-1.8.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel apr-util-devel
    

    Installation

    1.append hosts

    cat << EOF >>/etc/hosts
    10.140.0.1 master
    10.140.0.2 slave1
    10.140.0.3 slave2
    10.140.0.4 slave3
    EOF
    

    2.zookeeper

    2.1.install zookeeper on slave1/slave2/slave3

    2.2.modify conf/zoo.cfg on slave1/slave2/slave3

    cat << EOF > conf/zoo.cfg
    tickTime=2000
    initLimit=10
    syncLimit=5
    dataDir=./data
    clientPort=2181
    maxClientCnxns=0
    autopurge.snapRetainCount=3
    autopurge.purgeInterval=0
    leaderServes=yes
    skipAcl=no
    server.1=slave1:2888:3888
    server.2=slave2:2889:3889
    server.3=slave3:2890:3890
    EOF
    

    2.3.create data folder, and write serverid to myid on slave1/slave2/slave3, id is equals server’s sequence

    mkdir data && echo ${id} > data/myid
    

    2.4.start zookeeper on slave1/slave2/slave3, check zk’s status

    bin/zkServer.sh start
    bin/zkServer.sh status
    

    3.mesos

    3.1.install and import mesos repository on each server

    rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
    rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-mesosphere
    

    3.2.install mesos on each server

    yum install mesos -y
    

    3.3.modify mesos-master’s zk address on master/slave1

    echo "zk://slave1:2181,slave2:2181,slave3:2181/mesos" >/etc/mesos/zk
    

    3.4.modify quorum of mesos-master on master/slave1

    echo 2 > /etc/mesos-master/quorum
    

    3.5. start master and enable auto start on master/slave1

    systemctl enable mesos-master.service
    systemctl start mesos-slave.service
    

    3.6.start slave and enable auto start on slave1/slave2/slave3

    systemctl enable mesos-slave.service
    systemctl start mesos-slave.service
    

    4.marathon

    4.1.install marathon on master

    yum install marathon -y
    

    4.2.config master/zk address on master

    cat << EOF >>/etc/default/marathon
    MARATHON_MASTER="zk://slave1:2181,slave2:2181,slave3:2181/mesos"
    MARATHON_ZK="zk://slave1:2181,slave2:2181,slave3:2181/marathon"
    EOF
    

    4.3.start marathon and enable auto start on master

    systemctl enable marathon.service
    systemctl start marathon.service
    

    Test

    mesos: http://master:5050

    marathon: http://master:8080w

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: