Methods annotated with @Scheduled stops working in Open Source Spring

Products

• VMware Spring Runtime

Issue/Introduction

Symptoms:
From the sample code above every time the method is run. It logs the current date and time. In a hypothetical failure scenario the logs would show that it suddenly stops logging even though it's scheduled to run.

Environment

Cause

Spring provides a convenient way of implement scheduler with the use of ThreadPoolTaskScheduler. 

The following passage is from spring docs:

 

As a convenience, Spring also provides a ThreadPoolTaskScheduler, which internally delegates to a ScheduledExecutorService to provide common bean-style configuration along the lines of ThreadPoolTaskExecutor. These variants work perfectly fine for locally embedded thread pool setups in lenient application server environments, as well — in particular on Tomcat and Jetty.

Sometimes tasks takes longer than usual to execute and each task that will run uses up a thread from the pool. If it will take some time for a task to complete then it will take some time for that thread to be returned to pool. This delay can cause other task to wait for a thread to be available from the pool because there are no more free threads in the pool.

It is unlikely that the example above would cause thread exhaustion as it is just printing the current date and time, however, most actual scheduled tasks are performing more complicated actions.

Resolution

Here are some items to check what is causing this issue:

1. Turn on debug logs on see org.springframework.scheduling.* to get more information
2. Put more logging on code inside your annotated task. Check if there is code there that is taking longer than expected i.e query to database, writing to file, etc. If there is check if you can fine tune your code.
3. Capture several thread dumps from your application. Space them out one each couple of seconds. Examine the thread dumps to see how many threads from the pool are being used at one particular time. If the threads from the pool are consistently being used, then it's time to increase the number of threads in the pool.
4. Tune your ThreadPoolTaskScheduler pool size. The default value of ThreadPoolTaskScheduler pool size is 1.You might need to increase Pool Size to meet the needs of your scheduled tasks. Here is a sample how to specify pool size.

@Bean
public ThreadPoolTaskScheduler taskScheduler {
     ThreadPoolTaskScheduler taskScheduler = new       ThreadPoolTaskScheduler();
     taskScheduler.setPoolSize(25);
     return taskScheduler;
}

ThreadPoolTaskScheduler pool size can also be modified at runtime for example through JMX.