How to run a Linux script every few seconds under cron
From Computer Tyme Support Wiki
(→Even more precise with microsecond resolution) |
(→Even more precise with microsecond resolution) |
||
Line 330: | Line 330: | ||
fi | fi | ||
- | # Loops per minute and the next sleep | + | # Loops per minute and the next sleep till interval are passed on the command line with each loop |
loops=$1 | loops=$1 | ||
Line 336: | Line 336: | ||
# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute | # Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute | ||
- | |||
usleep $(( $next_interval - 10#$(date +%S%N) / 1000 )) | usleep $(( $next_interval - 10#$(date +%S%N) / 1000 )) |
Revision as of 23:57, 20 January 2014
Contents |
Overview
Did you ever want to run a program every few seconds under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that. You can get 1,2,3,4,5,6,10,12,15,20,30 second resolution. Just run it once a minute under crond and it just works.
Features:
- Run once per minute under cron
- Launches multiple programs in parallel
- Multiple time periods supported simultaneously
- Use of timeout kills programs that don't terminate within delay time
- It just works
- Simple, elegant, free
I'm now posting two versions of the program. The first version is simpler and is good for running as many as 30 times a minute. There are 2 issues however. crond as it turns out isn't real precise and it starts programs 1 to 1.5 seconds after the minute. Also - since the program call itself over and over during the minute there is some drift. There's enough drift so you can't really use it to run something every second. But every 2 seconds or greater is fine. Also - if you don't need to make sure the programs are killed before the next interval you can get rid of the timeout code.
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.
Simple Example
#! /bin/sh # Run all programs in a directory in parallel # Usage: run-parallel directory delay # Copyright 2013 by Marc Perkel # docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron" # Free to use with attribution if [ $# -eq 0 ] then echo echo "run-parallel by Marc Perkel" echo echo "This program is used to run all programs in a directory in parallel" echo "or to rerun them every X seconds for one minute." echo "Think of this program as cron with seconds resolution." echo echo "Usage: run-parallel [directory] [delay]" echo echo "Examples:" echo " run-parallel /etc/cron.20sec 20" echo " run-parallel 20" echo " # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute." echo echo "If delay parameter is missing it runs everything once and exits." echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed." echo echo 'If "cronsec" is passed then it runs all of these delays 2 3 4 5 6 10 12 15 20 30' echo "resulting in 30 20 15 12 10 6 5 4 3 2 executions per minute." echo exit fi # If "cronsec" is passed as a parameter then run all the delays in parallel if [ $1 = cronsec ] then for interval in 2 3 4 5 6 10 12 15 20 30 do $0 $interval & done exit fi # Set the directory to first parameter and delay to second parameter dir=$1 delay=$2 # If only parameter is 2,3,4,5,6,10,12,15,20,30 then automatically calculate # the standard directory name /etc/cron.[delay]sec if [[ "$1" =~ ^(2|3|4|5|6|10|12|15|20|30)$ ]] then dir="/etc/cron.$1sec" delay=$1 fi # Exit if directory doesn't exist or has no files if [ ! "$(ls -A $dir/ 2> /dev/null)" ] then exit fi # Sleep if $counter is set if [ ! -z $counter ] then sleep $delay fi # Run all the programs in the directory in parallel # Use of timeout ensures that the processes are killed if they run too long for program in $dir/* ; do if [ -x $program ] then if [ 0$delay -gt 0 ] then timeout $delay $program &> /dev/null & else $program &> /dev/null & fi fi done # If delay not set then we're done if [ -z $delay ] then exit fi # Add delay to counter counter=$(( $counter + $delay )) # If minute is not up - call self recursively if [ $counter -lt 60 ] then . $0 $dir $delay & fi # Otherwise we're done
You can then create the directories needed to put your programs in that you want to run. You don't have to create the directories that you aren't going to use.
mkdir /etc/cron.2sec mkdir /etc/cron.3sec mkdir /etc/cron.4sec mkdir /etc/cron.5sec mkdir /etc/cron.6sec mkdir /etc/cron.10sec mkdir /etc/cron.12sec mkdir /etc/cron.15sec mkdir /etc/cron.20sec mkdir /etc/cron.30sec
To run every minute you can edit your /etc/crontab file and add:
* * * * * root /usr/local/sbin/run-parallel cronsec
Keep in mind that the programs that you are trying need to be written to complete in the allotted time or they will be killed before completing. Also keep in mind that on a heavily loaded system that on smaller intervals there could be some time drift over the 1 minute period with unknown results. This is especially true of the 1 second interval.
Have a comment? You can email me at support@junkemailfilter.com
More Precise Implementation
Maybe I'm too picky but I want it to start right on the minute and have the intervals not be subject to drift/loading errors. So here is a slightly more complex version. The only drawback to this version is that it will take at least a full minute to start and maybe up to 2 minutes. That's because when it is launched it waits out the first minute to start processes right at the top of the minute instead of being late.
The usleep command is required for microsecond resolution sleep times.
This implementation starts on time and has 1 second resolution. Accuracy on my servers seem to be about 15 ms or better.
#! /bin/sh # Run all programs in a directory in parallel # Usage: run-parallel directory delay # Copyright 2013 by Marc Perkel # docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron" # Free to use with attribution if [ $# -eq 0 ] then echo echo "run-parallel by Marc Perkel" echo echo "This program is used to run all programs in a directory in parallel" echo "or to rerun them every X seconds for one minute." echo "Think of this program as cron with seconds resolution." echo echo "Usage: run-parallel [directory] [delay]" echo echo "Examples:" echo " run-parallel /etc/cron.20sec 20" echo " run-parallel 20" echo " # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute." echo echo "If delay parameter is missing it runs everything once and exits." echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed." echo echo 'If "cronsec" is passed then it runs all of these delays 1 2 3 4 5 6 10 12 15 20 30' echo "resulting in 60 30 20 15 12 10 6 5 4 3 2 executions per minute." echo exit fi # Runs like the sleep command but compensates for drift to start on time function syncsleep () { usleep $(( $1000000 - (10#$(date +%N) / 1000) )) } # If "cronsec" is passed as a parameter then run all the delays in parallel if [ $1 = cronsec ] then # Wait till start of next minute syncsleep $(( 60 - 10#$(date +%S) )) for interval in 1 2 3 4 5 6 10 12 15 20 30 do $0 $interval & done exit fi # Set the directory to first parameter and delay to second parameter dir=$1 delay=$2 # If only parameter is 1,2,3,4,5,6,10,12,15,20,30 then automatically calculate # the standard directory name /etc/cron.[delay]sec if [[ "$1" =~ ^(1|2|3|4|5|6|10|12|15|20|30)$ ]] then dir="/etc/cron.$1sec" delay=$1 fi # Exit if directory doesn't exist or has no files if [ ! "$(ls -A $dir/ 2> /dev/null)" ] then exit fi # Sleep if $counter is set if [ ! -z $counter ] then syncsleep $delay fi # Run all the programs in the directory in parallel # Use of timeout ensures that the processes are killed if they run too long for program in $dir/* ; do if [ -x $program ] then if [ 0$delay -gt 0 ] then timeout $delay $program &> /dev/null & else $program &> /dev/null & fi fi done # If delay not set then we're done if [ -z $delay ] then exit fi # Add delay to counter counter=$(( $counter + $delay )) # If minute is not up - call self recursively if [ $counter -lt 60 ] then . $0 $dir $delay & fi # Otherwise we're done
Even more precise with microsecond resolution
Here I changed the name of the program because the way it works is different. here the existence of the directory creates the schedule. This version supports microsecond resolution assuming your computer can handle it. It is the most accurate on start time and actually us a more simple implementation. Also the number of executions per minute does not have to to evenly divisible by 60. If you want to run something 17 times a minute - no problem.
Even though this has microsecond resolution probably 2400 per minute is as fast as you can practically go.
Keep in mind that this program takes up to 2 minutes to start.
#! /bin/sh # Microsecond Cron # Usage: cron-ms start # Copyright 2014 by Marc Perkel # docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron" # Free to use with attribution basedir=/etc/cron-ms if [ $# -eq 0 ] then echo echo "cron-ms by Marc Perkel" echo echo "This program is used to run all programs in a directory in parallel every X times per minute." echo "Think of this program as cron with microseconds resolution." echo echo "Usage: cron-ms start" echo echo "The scheduling is done by creating directories with the number of" echo "executions per minute as part of the directory name." echo echo "Examples:" echo " /etc/cron-ms/7 # Executes everything in that directory 7 times a minute" echo " /etc/cron-ms/30 # Executes everything in that directory 30 times a minute" echo " /etc/cron-ms/600 # Executes everything in that directory 10 times a second" echo " /etc/cron-ms/2400 # Executes everything in that directory 40 times a second" echo exit fi # If "start" is passed as a parameter then run all the delays in parallel # The number of the directory is the executions per minute # Since cron isn't accurate we need to start at top of next minute if [ $1 = start ] then for dir in $basedir/* ; do $0 ${dir##*/} 60000000 & done exit fi # Loops per minute and the next sleep till interval are passed on the command line with each loop loops=$1 next_interval=$2 # Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute usleep $(( $next_interval - 10#$(date +%S%N) / 1000 )) # Run all the programs in the directory in parallel for program in $basedir/$loops/* ; do if [ -x $program ] then $program &> /dev/null & fi done # Calculate next_interval next_interval=$(($next_interval % 60000000 + (60000000 / $loops) )) # If minute is not up - call self recursively if [ $next_interval -lt $(( 60000000 / $loops * $loops)) ] then . $0 $loops $next_interval & fi # Otherwise we're done
Launch from cron every minute. Edit your /etc/crontab file and add:
* * * * * root /usr/local/sbin/cron-ms start
Kind of amazing that you can do all this in less than 50 lines of code.