How to run a Linux script every few seconds under cron
From Computer Tyme Support Wiki
(→Even more precise with millisecond resolution) |
(→Even more precise with millisecond resolution) |
||
| Line 330: | Line 330: | ||
if [ $1 = start ] | if [ $1 = start ] | ||
then | then | ||
| - | for dir in | + | for dir in $basedir/* ; do |
loops=${dir:((${#basedir}+1)):5} | loops=${dir:((${#basedir}+1)):5} | ||
$0 $loops 60000 & | $0 $loops 60000 & | ||
Revision as of 16:52, 19 January 2014
Contents |
Overview
Did you ever want to run a program every few seconds under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that. You can get 1,2,3,4,5,6,10,12,15,20,30 second resolution. Just run it once a minute under crond and it just works.
Features:
- Run once per minute under cron
- Launches multiple programs in parallel
- Multiple time periods supported simultaneously
- Use of timeout kills programs that don't terminate within delay time
- It just works
- Simple, elegant, free
I'm now posting two versions of the program. The first version is simpler and is good for running as many as 30 times a minute. There are 2 issues however. crond as it turns out isn't real precise and it starts programs 1 to 1.5 seconds after the minute. Also - since the program call itself over and over during the minute there is some drift. There's enough drift so you can't really use it to run something every second. But every 2 seconds or greater is fine. Also - if you don't need to make sure the programs are killed before the next interval you can get rid of the timeout code.
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.
Simple Example
#! /bin/sh
# Run all programs in a directory in parallel
# Usage: run-parallel directory delay
# Copyright 2013 by Marc Perkel
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution
if [ $# -eq 0 ]
then
echo
echo "run-parallel by Marc Perkel"
echo
echo "This program is used to run all programs in a directory in parallel"
echo "or to rerun them every X seconds for one minute."
echo "Think of this program as cron with seconds resolution."
echo
echo "Usage: run-parallel [directory] [delay]"
echo
echo "Examples:"
echo " run-parallel /etc/cron.20sec 20"
echo " run-parallel 20"
echo " # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute."
echo
echo "If delay parameter is missing it runs everything once and exits."
echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed."
echo
echo 'If "cronsec" is passed then it runs all of these delays 2 3 4 5 6 10 12 15 20 30'
echo "resulting in 30 20 15 12 10 6 5 4 3 2 executions per minute."
echo
exit
fi
# If "cronsec" is passed as a parameter then run all the delays in parallel
if [ $1 = cronsec ]
then
for interval in 2 3 4 5 6 10 12 15 20 30
do
$0 $interval &
done
exit
fi
# Set the directory to first parameter and delay to second parameter
dir=$1
delay=$2
# If only parameter is 2,3,4,5,6,10,12,15,20,30 then automatically calculate
# the standard directory name /etc/cron.[delay]sec
if [[ "$1" =~ ^(2|3|4|5|6|10|12|15|20|30)$ ]]
then
dir="/etc/cron.$1sec"
delay=$1
fi
# Exit if directory doesn't exist or has no files
if [ ! "$(ls -A $dir/ 2> /dev/null)" ]
then
exit
fi
# Sleep if $counter is set
if [ ! -z $counter ]
then
sleep $delay
fi
# Run all the programs in the directory in parallel
# Use of timeout ensures that the processes are killed if they run too long
for program in $dir/* ; do
if [ -x $program ]
then
if [ 0$delay -gt 0 ]
then
timeout $delay $program &> /dev/null &
else
$program &> /dev/null &
fi
fi
done
# If delay not set then we're done
if [ -z $delay ]
then
exit
fi
# Add delay to counter
counter=$(( $counter + $delay ))
# If minute is not up - call self recursively
if [ $counter -lt 60 ]
then
. $0 $dir $delay &
fi
# Otherwise we're done
You can then create the directories needed to put your programs in that you want to run. You don't have to create the directories that you aren't going to use.
mkdir /etc/cron.2sec mkdir /etc/cron.3sec mkdir /etc/cron.4sec mkdir /etc/cron.5sec mkdir /etc/cron.6sec mkdir /etc/cron.10sec mkdir /etc/cron.12sec mkdir /etc/cron.15sec mkdir /etc/cron.20sec mkdir /etc/cron.30sec
To run every minute you can edit your /etc/crontab file and add:
* * * * * root /usr/local/sbin/run-parallel cronsec
Keep in mind that the programs that you are trying need to be written to complete in the allotted time or they will be killed before completing. Also keep in mind that on a heavily loaded system that on smaller intervals there could be some time drift over the 1 minute period with unknown results. This is especially true of the 1 second interval.
Have a comment? You can email me at support@junkemailfilter.com
More Precise Implementation
Maybe I'm too picky but I want it to start right on the minute and have the intervals not be subject to drift/loading errors. So here is a slightly more complex version. The only drawback to this version is that it will take at least a full minute to start and maybe up to 2 minutes. That's because when it is launched it waits out the first minute to start processes right at the top of the minute instead of being late.
The usleep command is required for microsecond resolution sleep times.
This implementation starts on time and has 1 second resolution. Accuracy on my servers seem to be about 15 ms or better.
#! /bin/sh
# Run all programs in a directory in parallel
# Usage: run-parallel directory delay
# Copyright 2013 by Marc Perkel
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution
if [ $# -eq 0 ]
then
echo
echo "run-parallel by Marc Perkel"
echo
echo "This program is used to run all programs in a directory in parallel"
echo "or to rerun them every X seconds for one minute."
echo "Think of this program as cron with seconds resolution."
echo
echo "Usage: run-parallel [directory] [delay]"
echo
echo "Examples:"
echo " run-parallel /etc/cron.20sec 20"
echo " run-parallel 20"
echo " # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute."
echo
echo "If delay parameter is missing it runs everything once and exits."
echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed."
echo
echo 'If "cronsec" is passed then it runs all of these delays 1 2 3 4 5 6 10 12 15 20 30'
echo "resulting in 60 30 20 15 12 10 6 5 4 3 2 executions per minute."
echo
exit
fi
# Runs like the sleep command but compensates for drift to start on time
function syncsleep ()
{
usleep $(( $1000000 - (10#$(date +%N) / 1000) ))
}
# If "cronsec" is passed as a parameter then run all the delays in parallel
if [ $1 = cronsec ]
then
# Wait till start of next minute
syncsleep $(( 60 - 10#$(date +%S) ))
for interval in 1 2 3 4 5 6 10 12 15 20 30
do
$0 $interval &
done
exit
fi
# Set the directory to first parameter and delay to second parameter
dir=$1
delay=$2
# If only parameter is 1,2,3,4,5,6,10,12,15,20,30 then automatically calculate
# the standard directory name /etc/cron.[delay]sec
if [[ "$1" =~ ^(1|2|3|4|5|6|10|12|15|20|30)$ ]]
then
dir="/etc/cron.$1sec"
delay=$1
fi
# Exit if directory doesn't exist or has no files
if [ ! "$(ls -A $dir/ 2> /dev/null)" ]
then
exit
fi
# Sleep if $counter is set
if [ ! -z $counter ]
then
syncsleep $delay
fi
# Run all the programs in the directory in parallel
# Use of timeout ensures that the processes are killed if they run too long
for program in $dir/* ; do
if [ -x $program ]
then
if [ 0$delay -gt 0 ]
then
timeout $delay $program &> /dev/null &
else
$program &> /dev/null &
fi
fi
done
# If delay not set then we're done
if [ -z $delay ]
then
exit
fi
# Add delay to counter
counter=$(( $counter + $delay ))
# If minute is not up - call self recursively
if [ $counter -lt 60 ]
then
. $0 $dir $delay &
fi
# Otherwise we're done
Even more precise with millisecond resolution
Here I changed the name of the program because the way it works is different. here the existence of the directory creates the schedule. This version supports millisecond resolution assuming your computer can handle it. It is the most accurate on start time and actually us a more simple implementation. (Yea Simple!) Also the number of executions per minute does not have to to evenly divisible by 60 or 60000. If you want to run something 17 times a minute no problem.
#! /bin/sh
# Millisecond Cron
# Usage: cron-ms start
# Copyright 2014 by Marc Perkel
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution
basedir=/etc/cron-ms
if [ $# -eq 0 ]
then
echo
echo "cron-ms by Marc Perkel"
echo
echo "This program is used to run all programs in a directory in parallel every X times per minute."
echo "Think of this program as cron with milliseconds resolution."
echo
echo "Usage: cron-ms start"
echo
echo "In theory this program can run a program up to 60000 times a second."
echo "However 2000 for a very fast program is a more realistic limit."
echo
echo "The scheduling is done by creating directories with the number of"
echo "executions per minute as part of the directory name."
echo
echo "Examples:"
echo " /etc/cron-ms/7 # Executes everything in that directory 7 times a minute"
echo " /etc/cron-ms/30 # Executes everything in that directory 30 times a minute"
echo " /etc/cron-ms/600 # Executes everything in that directory 10 times a second"
echo
exit
fi
# Sleeps until a specific pary of a minute with millisecond resolution. 60000 is full minute
function sleeptill ()
{
now=`date +%S%N`
usleep $(( $1000 - 10#${now:0:8} ))
}
# If "start" is passed as a parameter then run all the delays in parallel
# The number of the directory is the executions per minute
# Since cron isn't accurate we need to start at top of next minute
if [ $1 = start ]
then
for dir in $basedir/* ; do
loops=${dir:((${#basedir}+1)):5}
$0 $loops 60000 &
done
exit
fi
# Loops per minute and the next sleep til interval are passed on the command line with each loop
loops=$1
next_interval=$2
# Sleep till next_interval
sleeptill $next_interval
# Run all the programs in the directory in parallel
for program in $basedir/$loops/* ; do
if [ -x $program ]
then
$program &> /dev/null &
fi
done
# Calculate next_interval
next_interval=$(($next_interval % 60000 + (60000 / $loops) ))
# If minute is not up - call self recursively
if [ $next_interval -lt $(( 60000 / $loops * $loops)) ]
then
. $0 $loops $next_interval &
fi
# Otherwise we're done
