How to use the Taskun SchedUler framework to implement the method and technique of distributed task scheduling
How to use the Taskun SchedUler framework to implement the method and technique of distributed task scheduling
The distributed task scheduling is the process of automatically arranging and performing tasks in a distributed system according to specific rules and needs.Taskun Scheduler is a Java -based open source framework that is specially designed for distributed task scheduling.This article will introduce how to use the Taskun Scheduler framework to achieve distributed task scheduling and provide some related skills and example code.
1. Add taskun scheduler dependencies
First, add Taskun Scheduler to your Java project.You can add the following code to the project's Build.gradle or POM.XML file to introduce Taskun Scheduler:
Gradle:
groovy
dependencies {
Implementation 'ORG.embulk.embulk: Embulk-Core: x.x.x' // Taskun Scheduler's core dependencies
Implementation 'ORG.embulk.embulk: Embulk-Parser-CSV: x.x.x' // You need to select the appropriate parser dependencies according to the task needs
// Other dependencies ...
}
Maven:
<dependencies>
<dependency>
<groupId>org.embulk.embulk</groupId>
<artifactId>embulk-core</artifactId>
<version>x.x.x</version>
</dependency>
<dependency>
<groupId>org.embulk.embulk</groupId>
<artifactId>embulk-parser-csv</artifactId>
<version>x.x.x</version>
</dependency>
<!-Other dependencies ...->
</dependencies>
2. Writing the task scheduler configuration file
Taskun Scheduler uses the configuration file of YAML format to define the behavior of the task scheduling.You can create an arbitrary yaml file, such as `taskun.yaml`, and write in accordance with the following example configuration file:
yaml
tasks:
- task_type: your-task-type
cron: "0 * * * *"
time_zone: "Asia/Shanghai"
param1: value1
param2: value2
task_types_config:
your-task-type:
your_task_type_config_key1: your_task_type_config_value1
your_task_type_config_key2: your_task_type_config_value2
In the above example:
-`tasks` partially defines the task list.Each task object contains at least the `task_type` and` cron` fields, which represent the task type and task trigger time expression.
-`time_zone` field specifies the time zone where the task is triggered. The example is set to" ASIA/SHANGHAI ".
-`Param1` and` Param2` are the customized parameters of the task, which are different depending on the type of task.You can freely define these parameters according to the actual situation.
-`task_types_config` part is used to define the configuration information of specific task types.Taking `YOUR-TASK-TYPE` as an example, two custom configuration items` your_task_config_key1 and `your_task_config_key2`.
3. Write the type of custom task
In Taskun Scheduler, you can customize the task type according to actual needs.In order to achieve custom task types, you need to inherit the `ORG.embulk.embulk.spi.executorplugin` interface and implement the method.The following is a custom task type of a example:
import org.embulk.embulk.spi.Executor;
import org.embulk.embulk.spi.ExecutorPlugin;
import org.embulk.embulk.spi.PageReader;
import org.embulk.embulk.spi.PageBuilder;
public class YourTaskTypePlugin implements ExecutorPlugin {
@Override
public void transaction(ConfigSource config, ExecutorPlugin.Control control) {
// Realize the preparations before the task execution
}
@Override
public Executor open(ConfigSource config, ExecutorPlugin.Control control) {
Return new youtasktypeexecutor (config); // Return to the task actuator
}
public static class YourTaskTypeExecutor implements Executor {
private final String yourTaskTypeConfigValue1;
public YourTaskTypeExecutor(ConfigSource config) {
this.yourTaskTypeConfigValue1 = config.get(String.class, "your_task_type_config_key1");
// Get the value of other configuration items as needed
}
@Override
public void execute() {
// Implement the specific logic of the task
// You can read the task data through PageReader
// You can build task output data through PageBuilder
}
@Override
public void cleanup() {
// Clean up after implementing task execution
}
}
}
In the above examples, we realized a custom task type called `yourtasktypeplugin`. The type of task is based on the` YOUR_TASK_TYPE_CONFIG_KEY1` and `youR_TASK_CONFIG_KEY2` in the configuration file to get the configuration information EssenceDuring the task execution, you can read data through PageReader and build output data through PageBuilder.
4. Write a starter
Finally, you need to write a promoter class to load the task scheduler and configuration file, and start a distributed task scheduling.Here are the starter class of a sample:
import org.embulk.embulk.EmbulkEmbed;
import org.embulk.embulk.spi.GuessPlugin;
import org.embulk.embulk.spi.ParserPlugin;
public class TaskSchedulerStarter {
public static void main(String[] args) {
try (final EmbulkEmbed embedded = new EmbulkEmbed.Builder()
.registerPlugin(GuessPlugin.class)
.registerPlugin(ParserPlugin.class)
.registerplugin (yourtasktypeplugin.class) // Register a custom task type
.build()) {
embedded.run("taskun", "your-config-file.yaml"); // 运行Taskun Scheduler
} catch (Throwable throwable) {
throwable.printStackTrace();
}
}
}
In the above example, we started the Taskun Scheduler through the `Embulkembed`, and contained the previously customized task type` yourtasktypeplugin` when registering a plug -in.
At this point, we have completed the method and technique of using Taskun Scheduler framework to implement distributed task scheduling.You can customize your task type and configuration items according to the actual project needs, and effectively distribute task scheduling through Taskun Scheduler.
I hope this article can help you understand and apply the Taskun Scheduler framework!
