系统配置
在 /etc/security/limits.conf
这个文件的末尾加入一下内容:
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
在/etc/security/limits.d/90-nproc.conf
这个文件的末尾加入一下内容:
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
重启服务器之后生效,用 ulimit -n 或者 ulimit -a 查看设置结果
修改 /etc/selinux/config
然后重启服务器
SELINUX=disabled
检查CPU是否支持SSE4.2指令集,因为向量化执行需要用到这项特性:
grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
安装依赖
yum install -y libtool
yum install -y *unixODBC*
安装clickhouse单机
yum安装:
yum install yum-utils -y
rpm --import https://repo.clickhouse.tech/CLICKHOUSE-KEY.GPG
yum-config-manager --add-repo https://repo.clickhouse.tech/rpm/clickhouse.repo
yum install clickhouse-server clickhouse-client -y
安装完成后修改配置文件/etc/clickhouse-server/config.xml
<!--为避免与hdfs端口冲突 。需要修改tcp端口,如修改为9977-->
<tcp_port>9977</tcp_port>
<!--删除监听端口的行注释,允许指定主机访问-->
<listen_host>0.0.0.0</listen_host>
配置用户/etc/clickhouse-server/users.xml
<users>
<bigdata>
<password>BigData_2021</password>
<networks incl="networks" replace="replace">
<ip>::/0</ip>
</networks>
<profile>default</profile>
<quota>default</quota>
</bigdata>
</users>
启动服务端
systemctl start clickhouse-server
安装clickhouse集群
注:如安装clickhouse集群,需先安装并启动zookeeper
安装过程: 省略
如三台服务器的zookeeper均正常启动,则可继续搭建clickhouse集群
启动cluster还需要添加/etc/metrika.xml
文件
<yandex>
<clickhouse_remote_servers>
<hn_test>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>IPTV-test-hadoop1</host>
<port>9977</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>IPTV-test-hadoop2</host>
<port>9977</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>IPTV-test-hadoop3</host>
<port>9977</port>
</replica>
</shard>
</hn_test>
</clickhouse_remote_servers>
<!--zookeeper相关配置-->
<zookeeper-servers>
<node index="1">
<host>IPTV-test-hadoop1</host>
<port>2181</port>
</node>
<node index="2">
<host>IPTV-test-hadoop2</host>
<port>2181</port>
</node>
<node index="3">
<host>IPTV-test-hadoop3</host>
<port>2181</port>
</node>
</zookeeper-servers>
<!--每台节点不同配置,注意修改-->
<macros>
<layer>hn_test</layer>
<shard>hn_test_01</shard>
<replica>IPTV-test-hadoop1</replica>
</macros>
<networks>
<ip>::/0</ip>
</networks>
<clickhouse_compression>
<case>
<min_part_size>10000000000</min_part_size>
<min_part_size_ratio>0.01</min_part_size_ratio>
<method>lz4</method>
</case>
</clickhouse_compression>
</yandex>
同步到以下其他两台:
/etc/clickhouse-server/config.xml
/etc/clickhouse-server/users.xml
/etc/metrika.xml
添加完metrik.xml 重启clickhouse-server,登录客户端,通过system.cluster表可以查看集群信息
客户端登录命令
TZ=Asia/Shanghai clickhouse-client --port=9977 --user=bigdata --password=BigData_2021
ClickHouse总结
配置文件路径:
/etc/clickhouse-server/config.xml
/etc/clickhouse-server/users.xml
/etc/metrika.xml
日志文件路径:
/var/log/clickhouse-server/
表信息路径、元数据存储目录:
/var/lib/clickhouse/metadata/
表数据路径:
/var/lib/clickhouse/data/