Flink Java Dependency Management
This document provides the complete Maven dependency configuration for Flink projects on Amazon Managed Service for Apache Flink. For new applications, default to Flink 2.2. For existing applications, use the user's current Flink version.
For Kinesis-specific API usage and code examples, see kinesis-connector-guide.md.
Version-Specific Dependency Mapping
Not all connector versions are published for every Flink version. Use the correct combination from this table:
| Dependency | Flink 1.20 | Flink 2.2 | Notes |
|---|---|---|---|
flink.version |
1.20.3 |
2.2.0 |
Core Flink artifacts use this directly |
target.java.version |
11 |
17 |
Flink 2.x requires Java 17 minimum |
flink-connector-kafka |
3.4.0-1.20 |
4.0.1-2.0 |
Connector uses separate versioning scheme |
flink-connector-aws-kinesis-streams |
5.1.0-1.20 |
6.0.0-2.0 |
Connector uses separate versioning scheme |
flink-statebackend-rocksdb |
1.20.3 (uses flink.version) |
2.2.0 (uses flink.version) |
Available for both, can also use HashMap state backend via support case in MSF for apps that have small states that can fit in memory and benefit from lower latency state access |
aws-kinesisanalytics-runtime |
1.2.0 |
1.2.0 |
Same version for both |
aws-msk-iam-auth |
2.3.5 |
2.3.5 |
Version-agnostic |
maven-compiler-plugin |
3.8.1 |
3.11.0 |
Newer version recommended for Java 17 |
Flink 2.2 additional notes:
flink-statebackend-forstis available as an alternative toflink-statebackend-rocksdbfor disaggregated state management- Logging uses
log4j-slf4j-impl(same as 1.20; older Flink versions usedslf4j-log4j12) - See
flink-2x-migration.mdfor API changes that affect application code (e.g.,open(OpenContext),Durationinstead ofTime, removedSourceFunction/SinkFunction)
Example pom.xml for a Flink project
The example below defaults to Flink 2.2 properties (recommended for new applications). To use Flink 1.20 for existing applications, replace the properties block with the Flink 1.20 values from the table above.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<!-- REPLACE: Use your application's package name -->
<groupId>com.example</groupId>
<!-- REPLACE: Use your application's artifact name -->
<artifactId>my-flink-app</artifactId>
<!-- Increment version as new changes are released -->
<version>1.0</version>
<packaging>jar</packaging>
<!-- ============================================================
Flink 2.2 properties (default for new applications)
For Flink 1.20, replace with:
target.java.version = 11
flink.version = 1.20.3
kafka.version = 3.4.0-1.20
kinesis-streams.version = 5.1.0-1.20
maven.compiler.plugin.version = 3.8.1
============================================================ -->
<properties>
<!-- Build configs -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<buildDirectory>${project.basedir}/target</buildDirectory>
<jar.finalName>${project.name}-${project.version}</jar.finalName>
<target.java.version>17</target.java.version>
<maven.compiler.source>${target.java.version}</maven.compiler.source>
<maven.compiler.target>${target.java.version}</maven.compiler.target>
<!-- Dependency versions -->
<flink.version>2.2.0</flink.version>
<kda.runtime.version>1.2.0</kda.runtime.version>
<log4j.version>2.23.1</log4j.version>
<!-- Not all kafka / flink version combinations are published on Maven -->
<kafka.version>4.0.1-2.0</kafka.version>
<msk-iam-auth.version>2.3.5</msk-iam-auth.version>
<!-- Not all Kinesis / flink version combinations are published on Maven -->
<kinesis-streams.version>6.0.0-2.0</kinesis-streams.version>
<maven.compiler.plugin.version>3.11.0</maven.compiler.plugin.version>
</properties>
<!-- Java SDK for AWS required in general -->
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.12.677</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- Apache Flink dependencies required for most projects -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime-web</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-base</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- Flink Table Runtime — required when application code references
internal Table Runtime types (InternalTypeInfo, RowDataSerializer,
internal RowData converters). Commonly needed by Iceberg sinks
that emit/consume RowData on a side output. Provided by MSF at
runtime.
Note: bridge != runtime. flink-table-api-java-bridge gives you
the public Table API/SQL surface (DDL, TableEnvironment,
DataStream<->Table conversion). flink-table-runtime gives you
the internal data-structure types referenced by sinks and
custom serializers. -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-runtime</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- Managed Service for Apache Flink Runtime required for all projects -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-kinesisanalytics-runtime</artifactId>
<version>${kda.runtime.version}</version>
<scope>provided</scope>
</dependency>
<!-- Kafka Connector required for Kafka source/sink projects -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka</artifactId>
<version>${kafka.version}</version>
</dependency>
<!-- MSK IAM Auth required for Kafka projects using MSK IAM -->
<dependency>
<groupId>software.amazon.msk</groupId>
<artifactId>aws-msk-iam-auth</artifactId>
<version>${msk-iam-auth.version}</version>
</dependency>
<!-- Kinesis connector for Kinesis Streams source/sink projects -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-aws-kinesis-streams</artifactId>
<version>${kinesis-streams.version}</version>
</dependency>
<!-- RocksDB State Backend required for all projects -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-statebackend-rocksdb</artifactId>
<version>${flink.version}</version>
</dependency>
<!-- JSON Processing required for JSON projects -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.15.2</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-json</artifactId>
<version>${flink.version}</version>
</dependency>
<!-- Logging - generally required -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
<!-- Profile for local testing - includes provided dependencies -->
<profiles>
<profile>
<id>local</id>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime-web</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-base</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
<!-- Managed Service for Apache Flink Runtime -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-kinesisanalytics-runtime</artifactId>
<version>${kda.runtime.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
</profile>
</profiles>
<!-- Most of the below is boilerplate for managing shaded JARs -->
<build>
<directory>${buildDirectory}</directory>
<finalName>${jar.finalName}</finalName>
<plugins>
<!-- Java Compiler -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${target.java.version}</source>
<target>${target.java.version}</target>
</configuration>
</plugin>
<!-- Maven Shade Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.apache.flink:force-shading</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<!-- REPLACE: Use your application's fully qualified main class -->
<mainClass>com.example.MyFlinkJob</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>