How to Troubleshoot Crontab Issues

The other day, I updated the operating system on the server on which my crontab was installed. I thought that this would be a harmless upgrade to the system — nothing too crazy. However, I was wrong, the crontab that was working perfectly well on the old operating system now seemingly ceased to work. Nothing had changed in my setup of the scripts or the crontab, so why was it not working?

I set about troubleshooting the issue with some guidance from this article:

Disclaimer: After all my troubleshooting, I ultimately did not resolve the core of my issue and instead decided to give up on crontab and use a different solution. So, if you’re looking for a solution to my crontab problems at the end of this article, you will not find it.

  1. Verify my crontab settings: First thing I did was open up my crontab to make sure that everything was configured properly (and as I’d left it). I did this using the command…
    env EDITOR=nano crontab -e

    And sure enough the setup of my crontab was as I had left it:

    0 * * * * cd ~/Desktop/how\ to\ troubleshoot\ crontab/ && python2.7 hello_world.py

    While I was here, I thought to update the “0” in my job to “*” so that the job will run regularly every minute. This will make it easier to troubleshoot what was going on. I then closed and saved the crontab.

  2. Make sure the command can run successfully from Terminal: My second thought, after seeing that everything was all right with the file, was attempting to run the full command from my crontab file in Terminal and seeing if it worked:



    It did!
  3. Attempt to install crontab with admin permissions: Once I verified that the command could run outside of crontab without an issue, I started considering that maybe I didn’t have proper permissions to install the crontab on the server. So, I re-opened the crontab file with sudo privileges:sudo env EDITOR=nano crontab -e

    This is what I saw:

    What happened to my crontab command? Well, just to be sure, I went ahead and entered the same command into this crontab file that I had opened with the sudo permissions and checked if that produced the result I wanted.  It did not.

  4. Make sure the application has proper permissions: When I closed my last crontab edit (with sudo permissions), this little dialog box showed up:

    So, I speculated that perhaps iTerm, the Terminal application I was using to edit and install the crontab did not have proper permissions to install the crontab file on my server. I opened up my Mac’s “System Preferences”, accessible from the Apple icon in the upper left corner of the screen:

    Navigated to “Security & Privacy” tab:

    Scrolled to the “Full Disk Access” portion and unlocked the screen:

    Then added the iTerm application to the list of applications with “Full Disk Access” using the “+” button under the list of applications to the right:

    With iTerm now having “Full Disk Access”, I re-attempted to save the crontab (using sudo permissions), but had no success.

  5. Brute-force a crontab into its expected placement: So, I turned to the internet and found the article listed at the top that seemed pretty helpful. It stated, “There is a system-wide /etc/crontab file…”

    I neglected to read the rest of the sentence and instead went searching for the existence of the system-wide crontab file in the specified /etc/ directory.

    This is a directory that is otherwise hidden, so I used the “Spotlight” search feature on Mac (accessed via Cmd + Space) to search for the /etc/ directory and found it:

    And sure enough, once I opened that directory, I noticed that it had no file called “crontab” in it.

    In the absence of a “crontab” file, I decided I’d add one myself in a rather unorthodox manner. I opened a new file in Sublime Text, wrote my crontab command into the file, saved it to my “Desktop”, and then dragged it from that directory into the “etc” directory. What could go wrong?

    And yet, despite all this, nothing seemed to change! My crontab job was still failing to run.

     

  6. Check crontab run logs: So, I scanned the web article for some other ideas on what may be going on and spotted a recommendation to run the command…
    ps -ef | grep cron | grep -v grep

    …to check if my crontab has run. And sure enough it had! Multiple times even!

    To better understand this output, I looked up how to get the column headers for the grep command and found this helpful writeup. I went ahead and amended my command to the following:

    ps -ef | head -1 && ps -ef | grep cron | grep -v grep

    Now that it seemed that something was running, I did some accounting: Up to this point, it’s possible that I may have set up three different crontab files. One was set up via crontab using my own user permissions, another was set up via crontab using sudo permissions, and the last one was brute-forced into the “/etc/” directory using a text file.

    Of those three, it seemed that one or two were running. I suspected that the command labeled “(cron)” was a result of my sudo permissions manipulation and that the command labeled “/usr/sbin/cron” was coming from the one setup with my standard user permissions. As to why two instances existed for both, I was not sure. So, in an effort to verify this, I deleted my brute-forced crontab file from the “/etc/” directory and waited a minute to see if the logs looked any different.

    The logs did not look any different. Thus, my next step was to delete the crontab I added using sudo permissions. To do this, I opened the crontab file again using sudo permissions, deleted its contents, and saved. Again I waited a minute and re-ran the logs to see if anything had changed:

    This time around, I noted that duplicate commands have been removed. This let me know a few things: a) the crontab I had originally configured using my standard user permissions was running and b) it was generating two commands. I suspected two commands were showing because I was concatenating two commands in my crontab: one command to change directories and another to run a Python script.
     

  7. Split the command and leverage logs: I was more befuddled than ever at this point. It seems that all my poking around had been for nothing. My crontab was installed and running successfully. So maybe it was something in my crontab that wasn’t working well.I decided to employ logs in order to troubleshoot further. I broke up my command into different components and recorded output messages into separate logs in order to see where an error may have occurred:
    * * * * * echo 'success' >> /tmp/log1.log
    * * * * * cd ~/Desktop/how\ to\ troubleshoot\ crontab/ && echo 'success' >> /tmp/log2.log
    * * * * * cd ~/Desktop/how\ to\ troubleshoot\ crontab/ && python2.7 hello_world.py >> /tmp/log3.log
    * * * * * cd ~/Desktop/how\ to\ troubleshoot\ crontab/ && python2.7 hello_world.py && echo 'success' >> /tmp/log4.log

    First thing I noted when running the new crontab is that each command line in the file generated two commands in the crontab run logs:

    So my previous assumption that the number of commands in the log corresponded to the number of commands in a line was not correct.

    Second, I noted that while I expected 4 logs to generate, only 3 did:

    And, unfortunately, they were all empty. This was not what I expected at all since when I ran the third command line from my crontab file in Terminal…

    cd ~/Desktop/how\ to\ troubleshoot\ crontab/ && python2.7 hello_world.py >> /tmp/log3.log

    …it generated a log file with a value in it:


    I was starting to suspect that crontab was having trouble running my Python script.

  8. Try running an executable file: Better informed about what was happening within my crontab, but somewhat exasperated by my situation, I thought to convert my Python script file to an executable file. I thought that an executable file may be easier for crontab to process since it would remove the dependency on the “python2.7” command. Unfortunately, even this failed!
  9. Use a different solution: So, feeling completely out of options, I decided to give up on crontab and try something different altogether: Automator.

I apologize for the disappointing conclusion. Nonetheless, I think the troubleshooting process for my issue is helpful in understanding how crontab works and how to check that crontab is running. The main takeaways:

  • When troubleshooting with crontab, update the command to run every minute to provide a quicker feedback loop.
  • Verify the format of your crontab.
  • Attempt the commands from the crontab file in Terminal to make sure they work on their own.
  • Leverage crontab logs to check that crontab is installed and running correctly.
  • Add output logs to your commands in the crontab file to better understand where failure is occurring.

How to Leverage Automator to Work the Same Way as Crontab

After struggling for a long time with trying to make my crontab work, I eventually gave up and sought out a different method to setup an automated way for my script to run. I came across multiple articles online mentioning Automator, a MacOS application that allows you to automate certain functions on your computer. Many of these guides mentioned creating an Automator job and then scheduling it to recur via iCal.

Here is how I went about doing that:

  1. I opened Automator via Spotlight:

    For our purposes, since we want this to run on its own, I created a new Automator Application (File → New or Cmd+N and then select Application):

    Automator Applications are self-running whereas Automator Workflows require user inputs to kick off.

    Now let’s go over this screen a bit, because it was very confusing to me when I first opened this application:

    Within the application, you have three main panels. The far left panel with the expandable/collapsable directories is a way to navigate through the different actions available for automation. When you first open a new file in Automator, you are automatically placed at the top of this directory (Library) and all actions are displayed.

    The second panel, the one in the middle, is where the individual actions for automation are accessible. You can scroll up and down to see some of the actions at your disposal: you can launch an application, copy and paste files from one folder to another, download URLs, or do many other things using the actions in this column.

    The third panel on the far right of the screen is the workflow panel. This allows you to build your workflow using the actions in the middle panel. This is where Automator becomes really powerful since it allows you to string together simple actions to form complex workflows: you can download URLs, then copy the files downloaded from one directory to another, and then launch a script to run a process on those newly transferred files. In order to build these workflows, you drag and drop actions from the middle panel into the far right panel.

    At the top right corner, you have Record, Step, Stop, and Run buttons that will assist in checking your workflow once you have something down.

  2. For my Automator Application, I wanted to run a Shell script. So, I filtered the left panel directory to Utilities, then searched for the “Run Shell Script” command in the middle panel, and dragged it into the right panel to create my workflow:

  3. My next step was to customize my step. So, in the step panel, I entered my Terminal code for running my Shell script:
    /Users/mnachum/Desktop/./hello_world.sh >> /tmp/log.log

  4. I then ran my Application to verify that it worked:

    Checking the logs within Automator, everything seemed to have worked as expected. I also checked the outputted logs from my command (/tmp/log.log) and verified that those looked correct, as well.

    I saved my Application:

  5. I now had my Automator Application running my desired task, but I needed to schedule it to run regularly without input from me. Many forums online mentioned being able to create an appointment in iCal and then connecting that appointment to the running of the workflow, but this seemed like outdated advice to me. When searching for this functionality in the current iCal interface, I could not find a way to make appointments or events that linked to applications or scripts.

    However, I did note in Automator the ability to create an event in iCal that would launch an application. To get to this option, I opened a new Automator Workflow…

    …filtered to Calendar events in the directory panel, and dragged and dropped the “New Calendar Event” option into my Workflow.  Under the “Alarm” dropdown, there’s an option to “Open file”:

    I configured my New Calendar Event to open my previously configured Automator Application:

    Once the “Open file” option is selected under the “Alarm” field, an application needs to be selected. I selected my previously saved Automator Application.

  6. With my scheduling Workflow completed, I ran the Workflow:

    …and verified that the Event was added to my iCal, as expected:

    Whether or not you save this workflow is up to you. My desired outcome for this Automator Workflow was completed but I still decided to save my Workflow in case I needed to troubleshoot.

  7. From my iCal, I can now configure the Event to recur:

And that’s it!  That’s how I was able to stop using crontab to run my scripts and instead started using a combination of Automator Applications/Workflows and iCal Events. Note that from my experience, the Automator and iCal applications do not need to be open in order for the automated job to complete.

When the Automator Application does run, you will be able to see that it is running via the appearance of a sun-like icon in the toolbar:

Create a free website or blog at WordPress.com.