flink-1.9.1单机版环境的搭建Flink基础环境
本次安装Flink版本为flink-1.9.1，可以点击此链接直接下载
前提条件
1.安装JDK
[root@CentOSA ~]# rpm -ivh jdk-8u191-linux-x64.rpm
warning: jdk-8u191-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: >>>>>>>>### [100%]
1:jdk1.8 >>>>>>>>### [100%]
Unpacking
[root@CentOSA ~]# java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
[root@CentOSA ~]# vi ~/.bashrc
[root@CentOSA ~]# source .bashrc
[root@CentOSA ~]# jps
1449 Jps
2.安装HDFS
[root@CentOSA ~]# tar -zxf hadoop-2.9. -C /usr/
[root@CentOSA ~]# vi ~/.bashrc
HADOOP_HOME=/usr/hadoop-2.9.2
JAVA_HOME=/usr/java/latest
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
CLASSPATH=.
export JAVA_HOME
export PATH
export CLASSPATH
export HADOOP_HOME
[root@CentOSA ~]# source .bashrc
[root@CentOSA ~]# vi /usr/hadoop-2.9.2/etc/l
<!--nn访问⼊⼝-->
<property>
<name>fs.defaultFS</name>
<value>hdfs://CentOSA:9000</value>
</property>
<!--hdfs⼯作基础⽬录-->linux安装jdk rpm安装
<property>
<name&p.dir</name>
<value>/usr/hadoop-2.9.2/hadoop-${user.name}</value>
</property>
[root@CentOSA ~]# vi /usr/hadoop-2.9.2/etc/hadoop/slaves
CentOSA
[root@CentOSA ~]# vi /usr/hadoop-2.9.2/etc/l
<!--block副本因⼦-->
<property>
<name&plication</name>
<value>1</value>
</property>
<!--配置Sencondary namenode所在物理主机-->
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>CentOSA:50090</value>
</property>
<!--设置datanode最⼤⽂件操作数-->
<property>
<name>dfs.ievers</name>
<value>4096</value>
</property>
<!--设置datanode并⾏处理能⼒-->
<property>
<name>dfs.unt</name>
<value>6</value>
</property>
[root@CentOSA ~]# hdfs namenode -format # 格式化
[root@CentOSA ~]# start-dfs.sh
3.Flink安装
上传并解压flink
[root@CentOSA ~]# tar -zxf flink-1.9.1-bin-scala_ -C /usr/配置flink-conf.yaml
[root@CentOSA ~]# vi /usr/flink-1.9.1/conf/flink-conf.yaml jobmanager.rpc.address: CentOSA
taskmanager.numberOfTaskSlots:4
parallelism.default:3
配置slaves
[root@CentOSA ~]# vi /usr/flink-1.9.1/conf/slaves
CentOSA
启动Flink
[root@CentOSA flink-1.9.1]# ./bin/start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host CentOSA. Starting taskexecutor daemon on host CentOSA.
[root@CentOSA flink-1.9.1]# jps
1713 DataNode
3237 TaskManagerRunner
1894 SecondaryNameNode
1622 NameNode
2794 StandaloneSessionClusterEntrypoint
3277 Jps
环境测试案例
1.引⼊依赖
<properties>
<flink.version>1.9.1</flink.version>
<scala.version>2.11</scala.version>
</properties>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_${scala.version}</artifactId>
<version>${flink.version}</version>
</dependency>
2.Client程序
//1.创建流处理的环境
val fsEnv = ExecutionEnvironment
//2.读取socket中的数据
val lines:DataStream[String]=fsEnv.socketTextStream("CentOSA",5555)
lines.flatMap(_.split("\\s+"))
.map((_,1))
.keyBy(t=>t._1)
.sum(1)
.print()
//3.执⾏流计算任务
将程序打包
[root@CentOSA ~]# cd /usr/flink-1.9.1/
[root@CentOSA flink-1.9.1]# ./bin/flink run --class Test1 --detached --parallelism 3 /root/original-FlinkStream-1.0-SNAPSHOT.jar Starting execution of program
Job has been submitted with JobID 241250bf67445ed08d1b914e7159ced4
[root@CentOSA flink-1.9.1]# ./bin/flink list -m CentOSA:8081
Waiting
------------------ Running/Restarting Jobs -------------------
28.01.2020 03:15:31 : 241250bf67445ed08d1b914e7159ced4 : test1 (RUNNING)
--------------------------------------------------------------
No scheduled jobs.
取消任务
[root@centos flink-1.9.1]# ./bin/flink cancel -m CentOSA:8081 241250bf67445ed08d1b914e7159ced4
Cancelling job 241250bf67445ed08d1b914e7159ced4.
Cancelled job 241250bf67445ed08d1b914e7159ced4.
版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。
Oracle静默安装
« 上一篇
发表评论