Automating Daily Dev Tasks with Python Scripts and Cron Jobs

In the fast-paced world of software development, repetitive tasks can sap both time and energy. Whether it’s cleaning up log files, generating reports, or syncing data between services, automating these chores frees you to focus on more creative work. In this article, we’ll explore how to harness the power of Python scripts and cron jobs on Unix-like systems to schedule and automate your daily development tasks.

---

Why Automation Matters

Manual repetition is not only tedious but also error-prone. Even well-documented step-by-step guides can be skipped or misremembered. Automating routine tasks offers three big benefits:

1. Consistency
   Scripts run the same way every time, eliminating human mistakes.
2. Efficiency
   Tasks execute in the background, often faster than when done by hand.
3. Scalability
   As your project grows, you can adapt scripts or add new ones without overhauling existing workflows.

---

Writing Python Automation Scripts

Python’s rich standard library and third‑party modules make it ideal for writing quick automation scripts.

1. File and Log Management

Use Python’s os and shutil modules to move, compress, or delete old logs:

import os, shutil
from datetime import datetime, timedelta

log_dir = “/var/log/myapp”
archive_dir = “/var/log/myapp/archive”
cutoff = datetime.now() – timedelta(days=7)

for filename in os.listdir(log_dir):
filepath = os.path.join(log_dir, filename)
if os.path.isfile(filepath):
modified = datetime.fromtimestamp(os.path.getmtime(filepath))
if modified < cutoff:
shutil.move(filepath, archive_dir)

This script moves logs older than seven days into an archive folder.

2. Database Backups

Leverage subprocess calls to dump databases:

import subprocess
from datetime import datetime

backup_file = f”/backups/db_{datetime.now():%Y%m%d}.sql”
cmd = f”pg_dump mydatabase > {backup_file}”

subprocess.run(cmd, shell=True, check=True)

Adjust for MySQL or other systems by changing the dump command accordingly.

3. API Data Fetch

Automate data retrieval from REST APIs using requests:

import requests
import json

response = requests.get(“https://api.example.com/data”)
data = response.json()

with open(“/data/latest.json”, “w”) as f:
json.dump(data, f, indent=2)

Combine with pandas for processing or plotting as needed.

---

Scheduling with Cron Jobs

On Linux and macOS, cron provides a simple way to run scripts at set intervals.

1. Editing the Crontab

Open your crontab file:

crontab -e

Each line follows the format:

• * * * * /path/to/command
  │ │ │ │ │
  │ │ │ │ └─ Day of week (0–7, both 0 and 7 = Sunday)
  │ │ │ └── Month (1–12)
  │ │ └─── Day of month (1–31)
  │ └──── Hour (0–23)
  └───── Minute (0–59)

2. Scheduling Examples

• Daily at midnight:

0 0 * * * /usr/bin/python3 /home/user/log_cleanup.py

Every hour:

0 * * * * /usr/bin/python3 /home/user/db_backup.py

Every weekday at 9 AM:

0 9 * * 1-5 /usr/bin/python3 /home/user/fetch_api_data.py

3. Logging Output

Redirect output and errors to a log file for troubleshooting:

0 0 * * * /usr/bin/python3 /home/user/log_cleanup.py >> /home/user/cron.log 2>&1

This appends both standard output and errors to cron.log.

---

Best Practices

1. Use Virtual Environments
   Ensure each script runs in the correct Python environment by activating a virtualenv or using full paths to the Python interpreter.
2. Absolute Paths
   Always reference full paths in both scripts and crontab entries to avoid “file not found” errors.
3. Permissions
   Place scripts in directories with appropriate permissions and make them executable (chmod +x script.py).
4. Notifications
   Consider sending email or Slack notifications on failure. You can integrate SMTP libraries in Python or call mail from cron:

*/30 * * * * /path/to/script.sh || echo “Task failed” | mail -s “Cron Error” you@example.com

Conclusion

By combining Python scripts with cron jobs, you can automate a wide array of development tasks—from log rotation and database backups to API data fetching. Automating these processes saves time, reduces errors, and keeps your workflows both reliable and scalable. Start small by automating one daily chore today, and you’ll soon wonder how you ever managed without it.