CDH 6.3.1을 CentOS 7.7에 설치해보자

0. 네트워크 및 커널 매개변수 설정(클러스터 전체)

A. OS 버전

CentOS 7.7.1908 (3.10.0-1062.4.1.el7.x86_64)

B. 디스크 및 파티션

OS(SSD) : RAID 미러링

    /boot       500MB

    /           64GB

    /usr        128GB

    /var        64GB

    swap        설치할 메모리에 맞춰서

DATA(SAS or S-ATA) : No RAID

    /data01     48TB

C.설치 패키지

인프라 서버 : 자바 플랫폼, 가상화 하이퍼바이저, 호환성 라이브러리, 개발용 툴, 시스템 관리 도구

D. 네트워크, 호스트

vi /etc/sysconfig/network-scripts/ifcfg-eth0

IPADDR=10.211.55.101

NETMASK=255.255.255.0

GATEWAY=10.211.55.1

DNS1=1.1.1.1

DNS2=8.8.8.8

DNS3=168.126.63.1

DNS4=61.100.183.125

vi /etc/hosts

# CDH 6.3 Cluster

10.211.55.101    ivs1 nn1 MANAGER

10.211.55.102    ivs2 nn2

10.211.55.103    ivs3 dn1

vi /etc/sysconfig/network

HOSTNAME=ivs[1-3]

E. 방화벽 및 SELINUX

systemctl disable firewalld

systemctl stop firewalld

vi /etc/sysconfig/selinex

SELINUX=disabled

F. 커널 매개변수

vi /etc/sysctl.conf

vm.swappiness=10

vi /etc/rc.local

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo never > /sys/kernel/mm/transparent_hugepage/enabled

systemctl start rc-local

systemctl enable rc-local

sync; reboot

# 다음은 참고용

hostnamectl set-hostname ivs[1-3]

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo never > /sys/kernel/mm/transparent_hugepage/enabled

sysctl vm.swappiness=10

sysctl -p

저장소를 로컬로 구축한 후 관련 모듈를 사전에 설치해서 다운로드 및 설치 시간을 절약한다.

하지만 매니저 서버와 데이터베이스, 에이전트 등은 로컬 저장소를 이용해 설치하지 않는다. 

0. 웹 서버 준비(MANAGER)

yum -y install httpd

systemctl start httpd

1. 설치미디어 준비(MANAGER)

scp cloudera-repos-cdh631_20191025.tgz ivs1:/var/www/html/

cd /var/www/html

tar zxvf cloudera-repos-cdh631_20191025.tgz

cp /var/www/html/cloudera-repos/cloudera-repos.repo /etc/yum.repos.d/

yum clean all

yum update

2. cloudera-repos.repo 배포(모든 호스트)

scp /etc/yum.repos.d/cloudera-repos.repo ivs2:/etc/yum.repos.d/

scp /etc/yum.repos.d/cloudera-repos.repo ivs3:/etc/yum.repos.d/

ssh ivs2 "yum clean all"

ssh ivs3 "yum clean all"

2-1. 재설치의 경우 이전 설치된 모듈을 먼저 삭제

rm -rf /etc/cloudera* /opt/cloudera /usr/lib/oozie /usr/lib/hadoop /usr/lib/hive* /usr/lib/hue /usr/lib/parquet

rm -rf hadoop hbase* hive* zookeeper kafka hue impala

rm -rf /var/lib/cloudera-* /var/lib/hadoop* /var/lib/hbase /var/lib/flume-ng /var/lib/solr

rm -f /etc/yum.repos.d/cloudera-cdh.repo /etc/yum.repos.d/cloudera-manager.repo*

yum -y remove cloudera-manager-daemons cloudera-manager-agent

yum -y remove avro-doc avro-libs avro-tools bigtop-jsvc bigtop-utils flume-ng hadoop hadoop-hdfs  hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hadoop-kms hadoop-libhdfs hadoop-mapreduce hadoop-yarn hbase hbase-solr hive hive-hbase hive-hcatalog hive-jdbc hive-webhcat hue impala impala-shell kafka kite kudu oozie oozie-client parquet parquet-format pig search sentry sentry-hdfs-plugin solr solr-crunch solr-doc solr-mapreduce spark-core spark-python sqoop zookeeper

3. 사전 설치(모든 호스트)

yum -y install ntp

systemctl enable ntpd

systemctl start ntpd

rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

yum clean all

yum -y install libtidy

yum -y install oracle-j2sdk1.8.x86_64

yum -y install cloudera-manager-daemons 

yum -y install avro-doc avro-libs avro-tools bigtop-jsvc bigtop-utils flume-ng hadoop hadoop-hdfs  hadoop-hdfs-fuse hadoop-hdfs-nfs3 hadoop-httpfs hadoop-kms hadoop-libhdfs hadoop-mapreduce hadoop-yarn hbase hbase-solr hive hive-hbase hive-hcatalog hive-jdbc hive-webhcat hue impala impala-shell kafka kite kudu oozie oozie-client parquet parquet-format pig search sentry sentry-hdfs-plugin solr solr-crunch solr-doc solr-mapreduce spark-core spark-python sqoop zookeeper

4. Cloudera Manager 설치(MANAGER)

/var/www/html/cloudera-repos/cloudera-manager-installer-6.3.1.bin

5. Cluster 설치 & Service Role(PC)

http://manager:7180/

HDFS, HBase, Zookeeper 만 설치

6. HBase & htrace-core-3.1.0(MANAGER에서 모든 호스트로)

wget https://repo1.maven.org/maven2/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar

scp htrace-core-3.1.0-incubating.jar nn1:/usr/lib/hadoop/lib/

scp htrace-core-3.1.0-incubating.jar nn2:/usr/lib/hadoop/lib/

scp htrace-core-3.1.0-incubating.jar dn1:/usr/lib/hadoop/lib/

scp htrace-core-3.1.0-incubating.jar nn1:/usr/lib/hadoop-hdfs/lib/

scp htrace-core-3.1.0-incubating.jar nn2:/usr/lib/hadoop-hdfs/lib/

scp htrace-core-3.1.0-incubating.jar dn1:/usr/lib/hadoop-hdfs/lib/

scp htrace-core-3.1.0-incubating.jar nn1:/usr/lib/hbase/lib/client-facing-thirdparty/

scp htrace-core-3.1.0-incubating.jar nn2:/usr/lib/hbase/lib/client-facing-thirdparty/

scp htrace-core-3.1.0-incubating.jar dn1:/usr/lib/hbase/lib/client-facing-thirdparty/

7. Phoenix 설치(MANAGER에서 모든 호스트로)

wget http://apache.tt.co.kr/phoenix/apache-phoenix-5.0.0-HBase-2.0/bin/apache-phoenix-5.0.0-HBase-2.0-bin.tar.gz

cd /usr/local/src

tar zxvf apache-phoenix-5.0.0-HBase-2.0-bin.tar.gz

mv apache-phoenix-5.0.0-HBase-2.0-bin ../phoenix-5.0.0

chown -R root.root /usr/local/phoenix-5.0.0

ln -s /usr/local/phoenix-5.0.0 /usr/local/phoenix

scp -rp /usr/local/phoenix-5.0.0 nn2:/usr/local/

scp -rp /usr/local/phoenix-5.0.0 dn1:/usr/local/

ssh nn2 "ln -s /usr/local/phoenix-5.0.0 /usr/local/phoenix"

ssh dn1 "ln -s /usr/local/phoenix-5.0.0 /usr/local/phoenix"

scp /usr/local/phoenix/phoenix-5.0.0-HBase-2.0-server.jar nn1:/usr/lib/hbase/lib/

scp /usr/local/phoenix/phoenix-5.0.0-HBase-2.0-server.jar nn2:/usr/lib/hbase/lib/

scp /usr/local/phoenix/phoenix-5.0.0-HBase-2.0-server.jar dn1:/usr/lib/hbase/lib/

8. Environment Variables(.bash_profile:MANAGER에서 모든 호스트로)

export PHOENIX_HOME=/usr/local/phoenix

export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera

export PATH=$JAVA_HOME/bin:$PHOENIX_HOME/bin:$HOME/bin:$PATH

9. 설치 확인(클러스터 내 아무데서나 가능)

hbase shell

list

status 'simple'

sqlline.py ivs1

!table

CREATE TABLE test (

    today  VARCHAR NOT NULL,

    cnt    INTEGER,

    CONSTRAINT PK PRIMARY KEY(today)

) SALT_BUCKETS=3;