-- 一、修改clickhouse 查询节点同时为查询节点，数据节点
先备份
IP:10.224.11.35-36

1. 以下<ck_cluster>标签内新增<shard>，其中新增的host为新增的两台数据节点IP地址
文件：/data/tsg/olap/clickhouse/clickhouse-server/metrika.xml

<ck_cluster>


<shard>
  <weight>1</weight>
  <internal_replication>false</internal_replication>
  <replica>
    <host>10.224.11.35</host>
    <port>9001</port>
    <user>default</user>
    <password>galaxy2019</password>
  </replica>
</shard>
<shard>
  <weight>1</weight>
  <internal_replication>false</internal_replication>
  <replica>
    <host>10.224.11.36</host>
    <port>9001</port>
    <user>default</user>
    <password>galaxy2019</password>
  </replica>
</shard>

</ck_cluster>

2. 修改后依次重启clickhouse 35 36节点
service clickhouse-server restart

3. 35节点执行创建local表
文件：/data/tsg/olap/clickhouse/init/create_ck_table.sql

clickhouse-client -h 127.0.0.1 --port 9001 -m -u default --password galaxy2019  --multiquery < create_ck_table.sql

4. 登录35 36 clickhouse查看是否增加了local表
--35 36 节点登录客户端
clickhouse-client -h 127.0.0.1 --port 9001 -m -n -u default --password galaxy2019

-- 查看是否有local表
use tsg_galaxy_v3;
show tables;

--查看是否增加了新增的35 36地址
select * from `system`.clusters where cluster = 'ck_cluster';



--- 二、更新NC session record flink任务配置，减少部分字段写入clickhouse

1. Flink 前两台服务器，修改文件文件
IP:10.224.11.24-25 (文件两个节点都修改，启停任务只在一个节点进行)

/data/tsg/olap/flink/topology/groot-stream/config/session_record_processed_kafka_to_clickhouse

2.先备份，再更新，可以直接下载附件中已经改好的文件替换 session_record_processed_kafka_to_clickhouse

# 新增的内容
postprocessing_pipelines:
  post_etl_processor: # [object] Processing Pipeline
    type: com.geedgenetworks.core.processor.projection.ProjectionProcessorImpl
    remove_fields: [device_tag,app_transition,tcp_s2c_rtx_pkts,tcp_c2s_rtx_bytes,tcp_c2s_o3_pkts,tcp_s2c_o3_pkts,tcp_c2s_rtx_pkts,tcp_s2c_ip_fragments,tcp_c2s_ip_fragments,tcp_c2s_lost_bytes,tcp_s2c_lost_bytes,tcp_s2c_rtx_bytes,flags_identify_info,ssl_san,app_debug_info]

# 修改的内容
sinks:
  clickhouse_sink:
    type: clickhouse
    properties:
      host: 列表新增了扩容的35 36 节点

略....

# 修改的内容

  topology:
  - name: kafka_source
    downstream: [post_etl_processor]
  - name: post_etl_processor
    downstream: [clickhouse_sink]
  - name: clickhouse_sink


3. 重启Flink session record 任务

目录/data/tsg/olap/flink/topology/groot-stream/bin
sh stop.sh ../config/session_record_kafka_to_clickhouse
sh start.sh ../config/session_record_kafka_to_clickhouse

4. 验证任务启动，为RUNNING状态
yarn application --list|grep session_record_kafka_to_clickhouse

5. 验证去掉部分字段生效，返回结果应该为空或者为默认值0
--登录35客户端
clickhouse-client -h 127.0.0.1 --port 9001 -m -n -u default --password galaxy2019

select device_tag,app_transition,tcp_s2c_rtx_pkts,tcp_c2s_rtx_bytes,tcp_c2s_o3_pkts,tcp_s2c_o3_pkts,tcp_c2s_rtx_pkts,tcp_s2c_ip_fragments,tcp_c2s_ip_fragments,tcp_c2s_lost_bytes,tcp_s2c_lost_bytes,tcp_s2c_rtx_bytes,flags_identify_info,ssl_san,app_debug_info from tsg_galaxy_v3.session_record where recv_time >= now()- 60
and recv_time < now() limit 5;