How to run a Linux script every few seconds under cron

From Computer Tyme Support Wiki

(Difference between revisions)
Jump to: navigation, search
(Companion Program with 1 minute resolution)
(Overview)
 
(46 intermediate revisions not shown)
Line 1: Line 1:
= Overview =
= Overview =
-
Did you ever want to run a program every few seconds, or even fractions of a second, under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that.  
+
Did you ever want to run a program every few seconds, or even fractions of a second, under a linux, unix, bsd or osx cron script? Here's an elegant bash script that does just that.  
Features:
Features:
-
* Run once per minute under cron
+
* It can run stand alone or as a service - does not need cron
* Launches multiple programs in parallel
* Launches multiple programs in parallel
-
* Multiple time periods supported simultaneously just by the directory name.
+
* Multiple time periods supported simultaneously just by the directory name
 +
* 2 companion programs to support odd time periods like 37 seconds or 37 minutes
 +
* All standard bash stuff that should run on any *nix systems
 +
* Monit script included to keep it alive
 +
* Simple, elegant, documented, complete, free
* It just works
* It just works
-
* Simple, elegant, free
 
-
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.
+
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me a Amazon Gift Certificate to marc@perkel.com.
== Precise Cron with microsecond resolution ==
== Precise Cron with microsecond resolution ==
Line 16: Line 19:
This version can be installed in the /etc/init.d directory and run as a service.
This version can be installed in the /etc/init.d directory and run as a service.
 +
 +
chkconfig cron-ms on
 +
service cron-ms start
 +
 +
You don't need to run it as a service. You can just copy it into /usr/bin and put it in a startup script somewhere.
 +
 +
You'll also need usleep. If you can't find usleep in your Linux distribution install busybox and run this:
 +
 +
ln /bin/busybox /bin/usleep
 +
 +
Here's the program. It's just a simple bash script.
<pre>
<pre>
Line 26: Line 40:
# Microsecond Cron
# Microsecond Cron
# Usage: cron-ms start
# Usage: cron-ms start
-
# Copyright 2014 by Marc Perkel
+
# Copyright 2014 by Marc Perkel - marc@perkel.com
-
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
+
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron
# Free to use with attribution
# Free to use with attribution
-
# The scheduling is done by creating directories with the number of"
+
# The scheduling is done by creating directories with the number of
-
# executions per minute as part of the directory name."
+
# executions per minute as part of the directory name.
# Examples:
# Examples:
Line 39: Line 53:
#  /etc/cron-ms/2400  # Executes everything in that directory 40 times a second
#  /etc/cron-ms/2400  # Executes everything in that directory 40 times a second
-
basedir=/etc/cron-ms
+
if [ $# -eq 0 ]
 +
then
 +
  echo "Usage $0 start|stop"
 +
  exit
 +
fi
case "$1" in
case "$1" in
Line 46: Line 64:
   $0 stop
   $0 stop
   mkdir -p /var/run/cron-ms
   mkdir -p /var/run/cron-ms
-
   for dir in $basedir/* ; do
+
 
 +
  # Here I'm adding some extra directories to the $PATH.
 +
 
 +
  PATH=/usr/local/sbin:/usr/local/bin:$PATH
 +
  export PATH
 +
 
 +
   for dir in /etc/cron-ms/* ; do
       $0 ${dir##*/} &
       $0 ${dir##*/} &
   done
   done
 +
   exit
   exit
   ;;
   ;;
Line 59: Line 84:
esac
esac
-
# Loops per minute is passed on the command line
+
# Loops per minute is passed on the command line as $1
-
loops=$1
+
microseconds=$((60000000/$1))
-
interval=$((60000000/$loops))
+
-
 
+
-
# Just a heartbeat signal that can be used with monit to verify it's alive
+
-
 
+
-
touch /var/run/cron-ms
+
# After a restart the PIDs will be different allowing old processes to terminate
# After a restart the PIDs will be different allowing old processes to terminate
 +
# Touching /var/run/cron-ms is a heartbeat signal that can be used with monit to verify it's alive
-
touch /var/run/cron-ms/$$
+
touch /var/run/cron-ms /var/run/cron-ms/$$
# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute
# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute
-
usleep $(( $interval - 10#$(date +%S%N) / 1000 % $interval ))
+
usleep $(( $microseconds - $(date +%s%N) / 1000 % $microseconds ))
# Deleting the PID files exit the program
# Deleting the PID files exit the program
Line 85: Line 106:
# Run all the programs in the directory in parallel
# Run all the programs in the directory in parallel
-
for program in $basedir/$loops/* ; do
+
for program in /etc/cron-ms/$1/* ; do
-
   if [ -x $program ]
+
   $program &> /dev/null &
-
  then
+
-
      $program &> /dev/null &
+
-
  fi
+
done
done
-
exec $0 $loops
+
exec $0 $1
-
 
+
</pre>
</pre>
Kind of amazing that you can do all this in less than 50 lines of code.
Kind of amazing that you can do all this in less than 50 lines of code.
-
=== Monit Script to make sure it keeps running ===
+
You can add or remove programs from the /etc/cron-ms directories while cron-ms is running. However if create or remove directories you will need to restart the program.
 +
 
 +
=== Monit Script to make sure cron-ms keeps running ===
 +
Cron-ms touches the /var/run/cron-ms directory every cycle so if the date is older and one minute then cron-ms has stopped running and monit can restart it.
<pre>
<pre>
-
check directory cron-ms path /var/run/cron-ms
+
# /var/run/cron-ms is touched every cycle as heartbeat signal
-
if timestamp > 1 minute then restart
+
 
 +
check directory cron-ms-run path /var/run/cron-ms
 +
if does not exist then exec "/etc/init.d/cron-ms restart"
 +
if timestamp > 1 minute then exec "/etc/init.d/cron-ms restart"
 +
 
 +
# if a directory is created or deleted in /etc/cron-ms then restart
-
start program = "/etc/init.d/cron-ms start"
+
check directory cron-ms-etc path /etc/cron-ms
 +
if timestamp < 1 minute then exec "/etc/init.d/cron-ms restart"
</pre>
</pre>
-
== Companion Program with cron-min 1 minute resolution ==
+
== Companion Program cron-min with 1 minute resolution ==
This is a companion program to run a program every X minutes. you can of course do this in crontab but this makes it easier under some circumstances. This program allows you to:
This is a companion program to run a program every X minutes. you can of course do this in crontab but this makes it easier under some circumstances. This program allows you to:
Line 114: Line 140:
* The number of minutes does not need to divide equally into 60.
* The number of minutes does not need to divide equally into 60.
* The number of minutes isn't limited to 60. You can run a program every 61 minutes or every 923 minutes if you want.
* The number of minutes isn't limited to 60. You can run a program every 61 minutes or every 923 minutes if you want.
-
 
+
* The number of minutes doesn't have to divide evenly into hours. You can run something every 17 minutes if you want.
-
Both of these programs can be started at the same time from a single crontab entry. (see below) One is for minutes and up. The other is for seconds and down. Between these two you have a very powerful time based execution extension to compliment what cron does.
+
  #! /bin/sh
  #! /bin/sh
Line 121: Line 146:
  # Minute Cron
  # Minute Cron
  # Usage: cron-min start
  # Usage: cron-min start
-
  # Copyright 2014 by Marc Perkel
+
  # Copyright 2014 by Marc Perkel - marc@perkel.com
  # docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
  # docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
  # Free to use with attribution
  # Free to use with attribution
   
   
-
  # Run this script under Cron once a minute
+
  # Run this script under Cron or cron-ms once a minute.
 +
# This program is used to run all programs in a directory in parallel every X minutes.
   
   
-
  basedir=/etc/cron-min
+
  # The scheduling is done by creating directories with the number of minutes as part of the
 +
# directory name. The minutes do not have to evenly divide into 60 or be less than 60.
   
   
-
  if [ $# -gt 0 ]
+
  # Examples:
-
then
+
# /etc/cron-min/1      # Executes everything in that directory every 1  minute
-
    echo
+
# /etc/cron-min/5      # Executes everything in that directory every 5  minutes
-
    echo "cron-min by Marc Perkel"
+
# /etc/cron-min/13    # Executes everything in that directory every 13 minutes
-
    echo
+
# /etc/cron-min/75    # Executes everything in that directory every 75 minutes
-
    echo "This program is used to run all programs in a directory in parallel every X minutes."
+
-
    echo
+
  basedir=/etc/cron-min
-
    echo "Usage: cron-min"
+
-
    echo
+
-
    echo "The scheduling is done by creating directories with the number of minutes as part of the"
+
-
    echo "directory name. The minutes do not have to evenly divide into 60 or be less than 60."
+
-
    echo
+
-
    echo "Examples:"
+
-
    echo " /etc/cron-min/1      # Executes everything in that directory every 1  minute"
+
-
    echo " /etc/cron-min/5      # Executes everything in that directory every 5  minutes"
+
-
    echo " /etc/cron-min/13    # Executes everything in that directory every 13 minutes"
+
-
    echo " /etc/cron-min/75    # Executes everything in that directory every 75 minutes"
+
-
    echo
+
-
    exit
+
-
  fi
+
   
   
  for dir in $basedir/* ; do
  for dir in $basedir/* ; do
Line 155: Line 169:
     then
     then
       for program in $basedir/$minutes/* ; do
       for program in $basedir/$minutes/* ; do
-
           if [ -x $program ]
+
           $program &> /dev/null &
-
          then
+
-
            $program &> /dev/null &
+
-
          fi
+
       done
       done
     fi
     fi
  done
  done
-
=== Launching cron-min with Cron or cron-ms ===
+
== Companion Program cron-sec with 1 second resolution ==
 +
 
 +
The same thing can be done with 1 second resolution in case you want to run something every 37 seconds or every 90 seconds. You would have to run this script once a second from the above cron-ms program. The only new feature this script adds is the ability to use intervals greater than 60 seconds.
 +
 
 +
#! /bin/sh
 +
 +
# Run this script under cron-ms once a second.
 +
# This program is used to run all programs in a directory in parallel every X seconds.
 +
 +
# Examples:
 +
#  /etc/cron-sec/37    # Executes everything in that directory every 37 seconds
 +
#  /etc/cron-sec/75    # Executes everything in that directory every 75 seconds
 +
#  /etc/cron-sec/987    # Executes everything in that directory every 987 seconds
 +
 +
basedir=/etc/cron-sec
 +
 +
for dir in $basedir/* ; do
 +
    seconds=${dir##*/}
 +
    if [ $(( $(date +%s) % $seconds )) -eq 0 ]
 +
    then
 +
      for program in $basedir/$seconds/* ; do
 +
          $program &> /dev/null &
 +
      done
 +
    fi
 +
done
 +
 
 +
== Companion Program run-parallel to execute all programs in a different directory ==
 +
 
 +
If you have another directory where you have programs that you want to run you can use this script to run all of them in parallel.
 +
 
 +
#! /bin/sh
 +
 +
# Runs all programs in a directory in parallel
 +
# Usage /usr/local/sbin/run-parallel <dir>
 +
 +
for program in $1/* ; do
 +
    $program &> /dev/null &
 +
done
 +
 
 +
== Launching cron-min or cron-sec with Cron or cron-ms ==
To launch both of the above programs from cron every minute edit your /etc/crontab file and add:
To launch both of the above programs from cron every minute edit your /etc/crontab file and add:
-
  * * * * * root /usr/local/sbin/cron-min ; /usr/local/sbin/cron-ms start
+
  * * * * * root /usr/local/sbin/cron-min
-
or you can put this script in the /etc/cron-ms/1 directory and run it under cron-ms.
+
Or you can put this script in the /etc/cron-ms/1 directory and run it under cron-ms. Of you want to run cron-sec you would put it in the /etc/cron-ms/60 directory.

Latest revision as of 22:20, 16 November 2017

Contents

Overview

Did you ever want to run a program every few seconds, or even fractions of a second, under a linux, unix, bsd or osx cron script? Here's an elegant bash script that does just that.

Features:

  • It can run stand alone or as a service - does not need cron
  • Launches multiple programs in parallel
  • Multiple time periods supported simultaneously just by the directory name
  • 2 companion programs to support odd time periods like 37 seconds or 37 minutes
  • All standard bash stuff that should run on any *nix systems
  • Monit script included to keep it alive
  • Simple, elegant, documented, complete, free
  • It just works

Even though this software is free if you find it really useful and you want to reward/encourage me you can email me a Amazon Gift Certificate to marc@perkel.com.

Precise Cron with microsecond resolution

The existence of the directory of the number of loops per minute creates the schedule. This version supports microsecond resolution assuming your computer can handle it. The number of executions per minute does not have to to evenly divisible by 60 nor is it limited to 60 I have tested it to 6000 and it works.

This version can be installed in the /etc/init.d directory and run as a service.

chkconfig cron-ms on
service cron-ms start

You don't need to run it as a service. You can just copy it into /usr/bin and put it in a startup script somewhere.

You'll also need usleep. If you can't find usleep in your Linux distribution install busybox and run this:

ln /bin/busybox /bin/usleep

Here's the program. It's just a simple bash script.

#! /bin/sh

# chkconfig: 2345 91 61
# description: This program is used to run all programs in a directory in parallel every X times per minute. \
#              Think of this program as cron with microseconds resolution.

# Microsecond Cron
# Usage: cron-ms start
# Copyright 2014 by Marc Perkel - marc@perkel.com
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron
# Free to use with attribution

# The scheduling is done by creating directories with the number of
# executions per minute as part of the directory name.

# Examples:
#   /etc/cron-ms/7      # Executes everything in that directory  7 times a minute
#   /etc/cron-ms/30     # Executes everything in that directory 30 times a minute
#   /etc/cron-ms/600    # Executes everything in that directory 10 times a second
#   /etc/cron-ms/2400   # Executes everything in that directory 40 times a second

if [ $# -eq 0 ]
then
   echo "Usage $0 start|stop"
   exit
fi

case "$1" in

   start|restart|reload)
   $0 stop
   mkdir -p /var/run/cron-ms

   # Here I'm adding some extra directories to the $PATH.

   PATH=/usr/local/sbin:/usr/local/bin:$PATH
   export PATH

   for dir in /etc/cron-ms/* ; do
      $0 ${dir##*/} &
   done

   exit
   ;;

   stop)
   rm -Rf /var/run/cron-ms
   exit
   ;;

esac

# Loops per minute is passed on the command line as $1

microseconds=$((60000000/$1))

# After a restart the PIDs will be different allowing old processes to terminate
# Touching /var/run/cron-ms is a heartbeat signal that can be used with monit to verify it's alive

touch /var/run/cron-ms /var/run/cron-ms/$$

# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute

usleep $(( $microseconds - $(date +%s%N) / 1000 % $microseconds ))

# Deleting the PID files exit the program

if [ ! -f /var/run/cron-ms/$$ ]
then
   exit
fi

# Run all the programs in the directory in parallel

for program in /etc/cron-ms/$1/* ; do
   $program &> /dev/null &
done

exec $0 $1

Kind of amazing that you can do all this in less than 50 lines of code.

You can add or remove programs from the /etc/cron-ms directories while cron-ms is running. However if create or remove directories you will need to restart the program.

Monit Script to make sure cron-ms keeps running

Cron-ms touches the /var/run/cron-ms directory every cycle so if the date is older and one minute then cron-ms has stopped running and monit can restart it.

# /var/run/cron-ms is touched every cycle as heartbeat signal

check directory cron-ms-run path /var/run/cron-ms
if does not exist then exec "/etc/init.d/cron-ms restart"
if timestamp > 1 minute then exec "/etc/init.d/cron-ms restart"

# if a directory is created or deleted in /etc/cron-ms then restart

check directory cron-ms-etc path /etc/cron-ms
if timestamp < 1 minute then exec "/etc/init.d/cron-ms restart"

Companion Program cron-min with 1 minute resolution

This is a companion program to run a program every X minutes. you can of course do this in crontab but this makes it easier under some circumstances. This program allows you to:

  • Set the timing in number of minutes just be the directory name and put as many programs in that directory as you want to start.
  • The number of minutes does not need to divide equally into 60.
  • The number of minutes isn't limited to 60. You can run a program every 61 minutes or every 923 minutes if you want.
  • The number of minutes doesn't have to divide evenly into hours. You can run something every 17 minutes if you want.
#! /bin/sh

# Minute Cron
# Usage: cron-min start
# Copyright 2014 by Marc Perkel - marc@perkel.com
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution

# Run this script under Cron or cron-ms once a minute.
# This program is used to run all programs in a directory in parallel every X minutes.

# The scheduling is done by creating directories with the number of minutes as part of the
# directory name. The minutes do not have to evenly divide into 60 or be less than 60.

# Examples:
#  /etc/cron-min/1      # Executes everything in that directory every 1  minute
#  /etc/cron-min/5      # Executes everything in that directory every 5  minutes
#  /etc/cron-min/13     # Executes everything in that directory every 13 minutes
#  /etc/cron-min/75     # Executes everything in that directory every 75 minutes

basedir=/etc/cron-min

for dir in $basedir/* ; do
   minutes=${dir##*/}
   if [ $(( ($(date +%s) / 60) % $minutes )) -eq 0 ]
   then
      for program in $basedir/$minutes/* ; do
         $program &> /dev/null &
      done
   fi
done

Companion Program cron-sec with 1 second resolution

The same thing can be done with 1 second resolution in case you want to run something every 37 seconds or every 90 seconds. You would have to run this script once a second from the above cron-ms program. The only new feature this script adds is the ability to use intervals greater than 60 seconds.

#! /bin/sh

# Run this script under cron-ms once a second.
# This program is used to run all programs in a directory in parallel every X seconds.

# Examples:
#  /etc/cron-sec/37     # Executes everything in that directory every 37 seconds
#  /etc/cron-sec/75     # Executes everything in that directory every 75 seconds
#  /etc/cron-sec/987    # Executes everything in that directory every 987 seconds

basedir=/etc/cron-sec

for dir in $basedir/* ; do
   seconds=${dir##*/}
   if [ $(( $(date +%s) % $seconds )) -eq 0 ]
   then
      for program in $basedir/$seconds/* ; do
         $program &> /dev/null &
      done
   fi
done

Companion Program run-parallel to execute all programs in a different directory

If you have another directory where you have programs that you want to run you can use this script to run all of them in parallel.

#! /bin/sh

# Runs all programs in a directory in parallel
# Usage /usr/local/sbin/run-parallel <dir>

for program in $1/* ; do
   $program &> /dev/null &
done

Launching cron-min or cron-sec with Cron or cron-ms

To launch both of the above programs from cron every minute edit your /etc/crontab file and add:

* * * * * root /usr/local/sbin/cron-min

Or you can put this script in the /etc/cron-ms/1 directory and run it under cron-ms. Of you want to run cron-sec you would put it in the /etc/cron-ms/60 directory.

Personal tools