实验相关配置

### 相关配置

> Written by 黄俊仁，2024/04/13

本文档中后续提及的所有示例程序均基于以下Maven项目配置。该配置文件中包括了对Apache Flink的依赖，确保示例程序能够在Flink环境中顺利编译和执行。请确保项目`pom.xml`文件包含以下核心元素：

- `Apache Flink`依赖：包括用于Java API、流处理、表API以及SQL执行的Flink组件。

- `Maven Shade Plugin`：用于创建包含所有必需依赖的"uber-jar"，以简化部署和执行过程。

确保开发环境满足以下条件：

- Java 8
- Maven 3.6（或以上版本）
- Apach Flink 1.11.1

```xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

<groupId>org.nbubigdata</groupId>
    <artifactId>flink-demo</artifactId>
    <version>1.0-SNAPSHOT</version>

<dependencies>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-java</artifactId>
            <version>${flink.version}</version>
        </dependency>

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-api-java-bridge_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-clients_2.11</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner-blink_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

<dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>
    </dependencies>

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.4</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <createDependencyReducedPom>false</createDependencyReducedPom>
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                            <transformers>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>指定自己的程序入口</mainClass>
                                </transformer>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.conf</resource>
                                </transformer>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>
```

下面对部署和执行过程进行简要介绍。项目结构整体如下图所示：

![image-20240409192402463](https://hexo-img.obs.cn-east-3.myhuaweicloud.com/img/image-20240409192402463.png)

在`pom.xml`中对jar包的程序入口配置进行修改：

```xml
<transformers>
    <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
        <mainClass>指定的程序入口</mainClass>
    </transformer>
</transformers>
```

然后在项目根目录运行maven命令：

```shell
nbu@ecs:~/bigdata/flink/Flink$ mvn clean package
```

运行结束后自动生成target目录，得到打好的jar包，使用java命令运行jar包：

```shell
nbu@ecs:~/bigdata/flink/Flink/target$ java -jar flink-demo-1.0-SNAPSHOT.jar
```

大数据学习

导航

最近发表

友情链接