The traffic control strategy in Amazon Kinesis Client Library for Java
The traffic control strategy in Amazon Kinesis Client Library for Java
Amazon Kinesis is a completely custodian streaming data processing service that can process large -scale and real -time data streams.Amazon Kinesis Client Library for Java is a complicated logic for developers to manage the interaction with Amazon Kinesis to build a streaming data application.The library provides some traffic control strategies to ensure that the data transmission between the application and Amazon Kinesis can be smooth.
Flow control is an important part of ensuring that the system can efficiently process data flow.Amazon Kinesis Client Library for Java provides the following traffic control strategies:
1. Record Rate Limiting: This strategy is used to limit the rate of reading records from Amazon Kinesis.Developers can use the "setlimiters ()" method to set a rate limiter, which limits the number of records that consumers can read within a certain time period (such as per second).The following is a sample code segment to demonstrate how to set the flow control strategy:
import software.amazon.kinesis.coordinator.Scheduler;
import software.amazon.kinesis.retrieval.fanout.FanOutConfig;
public class RecordRateLimitingExample {
private static final int MAX_RECORDS_PER_SECOND = 100;
public static void main(String[] args) {
Scheduler scheduler = createScheduler();
scheduler.setLimiters(FanOutConfig.builder()
.rateLimiterByRecordsPerSecond(MAX_RECORDS_PER_SECOND)
.build());
}
// Example method of creating schedules (Scheduler)
private static Scheduler createScheduler() {
// Create and configure the scheduling program here
// ...
}
}
2. Consumer Rate Limiting: This strategy is used to limit the rate of consumer consumption data from Amazon Kinesis.Developers can use the "SetOffSetslimiters ()" method to set consumer restrictions for specific consumers.The limiter will limit the number of consumers consumed in a certain time period (eg, per second).The following is a sample code segment to demonstrate how to set the flow control strategy:
import software.amazon.kinesis.coordinator.Scheduler;
import software.amazon.kinesis.retrieval.fanout.FanOutConfig;
public class ConsumerRateLimitingExample {
private static final int MAX_RECORDS_PER_SECOND = 100;
private static final String CONSUMER_NAME = "consumer1";
public static void main(String[] args) {
Scheduler scheduler = createScheduler();
scheduler.fanOutConsumerRegistration().getRegistration(CONSUMER_NAME)
.ifPresent(consumer -> consumer.setLimiters(FanOutConfig.builder()
.rateLimiterByRecordsPerSecond(MAX_RECORDS_PER_SECOND)
.build()));
}
// Example method of creating schedules (Scheduler)
private static Scheduler createScheduler() {
// Create and configure the scheduling program here
// ...
}
}
3. Shard Rate Limiting: This strategy is used to limit the rate of consumer consumption data from specific shard consumption.Developers can use the "SetRatelimiter ()" method to set the rate limiter for specific shards.The limiter will limit the number of records read consumers from the film.The following is a sample code segment to demonstrate how to set the flow control strategy:
import software.amazon.kinesis.coordinator.Scheduler;
import software.amazon.kinesis.retrieval.fanout.FanOutConfig;
public class ShardRateLimitingExample {
private static final int MAX_RECORDS_PER_SECOND = 100;
private static final String SHARD_ID = "shard-123";
public static void main(String[] args) {
Scheduler scheduler = createScheduler();
scheduler.fanOutConsumerRegistration().getRegistration(CONSUMER_NAME)
.ifPresent(consumer -> consumer.getRateLimiter().setRate(MAX_RECORDS_PER_SECOND));
}
// Example method of creating schedules (Scheduler)
private static Scheduler createScheduler() {
// Create and configure the scheduling program here
// ...
}
}
These flow control strategies can be customized and configured according to the needs of the application to achieve the best data processing performance and resource utilization.Developers can choose suitable traffic control strategies according to their specific scenes and requirements, and use the API in the Java code example for configuration and application.
