优化Kafka分区数 1:分别登录Bole-IGW、Microwave-IGW的kafka界面。 2:逐步点击到Topics页面,再点击SESSION-RECORD。 3:进入SESSION-RECORD页面后点击Add Partitions。 4:将Partitions修改为30,再点击蓝色的Add Partitions按钮完成修改。 修改完成后,再次进入SESSION-RECORD页面,可以看到Partition Information下面有30个Partition -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 优化IGW中心会话日志汇聚任务并行度,期间静默OLAP DataCenter Flink Topology Down告警 登录:Bole-IGW、Microwave-IGW 1:cd /data/tsg/olap/galaxy/workspace/topology/flink_topology/completion/config 2:优化配置 vim ETL-SESSION-RECORD-COMPLETED将 source.parallelism修改为30 sink.parallelism修改为60 3:停止任务 3.1 获取任务id: docker exec jobmanager flink list | grep "ETL-SESSION-RECORD-COMPLETED" | awk '{print $4}' 3.2 根据获取的id关闭任务: docker exec jobmanager flink cancel 任务id 3.3 确认任务已关闭,执行后无返回结果即可。 docker exec jobmanager flink list | grep "ETL-SESSION-RECORD-COMPLETED" 4:启动任务 cd /data/tsg/olap/galaxy/workspace/topology/flink_topology/completion cat config/ETL-SESSION-RECORD-COMPLETED > service_flow_config.properties jar -uvf log-completion-schema-220318-Nacos.jar service_flow_config.properties docker cp log-completion-schema-220318-Nacos.jar "jobmanager":/home/log-completion-schema-220318-Nacos.jar docker exec "jobmanager" flink run -d -c com.zdjizhi.topology.LogFlowWriteTopology -p 1 /home/log-completion-schema-220318-Nacos.jar ETL-SESSION-RECORD-COMPLETED 5:校验 5.1 确认任务已启动 docker exec jobmanager flink list | grep "ETL-SESSION-RECORD-COMPLETED" 任务状态为RUNNING 5.2 登录TSG界面,可查询到该局点会话日志。 6:恢复OLAP DataCenter Flink Topology Down告警。