Podcast-Downloader/README.md

# Podcast-Downloader


A **bash script** that automatically downloads videos from YouTube playlists using `yt-dlp`, maintains a local SQLite database of downloaded content, and prevents duplicate downloads. Supports multiple playlists with individual destination folders and **cron job integration** for automated scheduling.

---

## Features

| Feature | Description |
|---------|-------------|
| **Multi-Playlist Support** | Configure and download from multiple YouTube playlists simultaneously |
| **Duplicate Prevention** | SQLite database tracks all downloaded videos to prevent re-downloading |
| **Individual Destinations** | Assign a different destination folder for each playlist |
| **Cron-Compatible** | Run automatically on a schedule with proper logging |
| **Download Sessions** | Detailed statistics on each download run (downloaded, skipped, errors) |
| **Database Statistics** | View total videos downloaded, storage used, and per-playlist breakdowns |
| **Configurable Format** | Specify video format, download delays, and audio-only mode via config |
| **Automatic Cleanup** | Removes temporary download folders after successful transfers |
| **Colored Output** | Terminal-friendly logging (disabled in cron mode) |

---

## Prerequisites

Before using this script, ensure you have the following tools installed:

### Required

- **bash** (version 4.0 or higher)
- **yt-dlp** – YouTube video downloader ([install](https://github.com/yt-dlp/yt-dlp#installation))
- **sqlite3** – Database management
- **jq** – JSON parser for config file handling

### Installation

**Ubuntu/Debian:**
```bash
sudo apt-get update
sudo apt-get install yt-dlp sqlite3 jq
```

**macOS:**
```bash
brew install yt-dlp sqlite3 jq
```

**Fedora/RHEL:**
```bash
sudo dnf install yt-dlp sqlite3 jq
```

---

## Installation

### 1. Clone or Download the Script

```bash
mkdir -p ~/.youtube_downloader
cd ~/.youtube_downloader
# Download or clone the script here
chmod +x youtube_downloader.sh
```

### 2. Create Configuration File

Create the config file at `~/.config/youtube_downloader/config.json`:

```bash
mkdir -p ~/.config/youtube_downloader
```

Then create `~/.config/youtube_downloader/config.json` with the following structure:

```json
{
  "general": {
    "db_dir": "$HOME/.youtube_downloader",
    "temp_download_dir": "/tmp/youtube_downloads",
    "video_format": "best[ext=mp4]",
    "output_template": "%(title)s.%(ext)s",
    "audio_only": false,
    "download_delay": 2,
    "debug": false
  },
  "playlists": [
    {
      "name": "My Music Playlist",
      "url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_1",
      "destination": "/home/user/Music/YouTubeDownloads",
      "enabled": true
    },
    {
      "name": "Tutorial Playlist",
      "url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_2",
      "destination": "/home/user/Videos/Tutorials",
      "enabled": true
    },
    {
      "name": "Disabled Playlist",
      "url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_3",
      "destination": "/home/user/Videos/Archive",
      "enabled": false
    }
  ]
}
```

### 3. Find Your Playlist IDs

To get a playlist ID:
1. Go to a YouTube playlist
2. Look at the URL: `https://www.youtube.com/playlist?list=PLAYLIST_ID_HERE`
3. Copy the part after `list=`

---

## Configuration Guide

### General Settings

| Setting | Description | Example |
|---------|-------------|---------|
| `db_dir` | Directory for database and logs | `$HOME/.youtube_downloader` |
| `temp_download_dir` | Temporary folder for downloads (cleaned up after) | `/tmp/youtube_downloads` |
| `video_format` | yt-dlp format specification | `best[ext=mp4]` or `worst` |
| `output_template` | Filename pattern (yt-dlp variables) | `%(title)s.%(ext)s` |
| `audio_only` | Download audio only (ignores video_format) | `true` or `false` |
| `download_delay` | Seconds between downloads (avoids rate-limiting) | `2` |
| `debug` | Enable debug logging | `true` or `false` |

### Video Format Examples

```json
"best[ext=mp4]"           // Best quality MP4
"worst"                   // Lowest quality (smallest file)
"bestvideo+bestaudio"     // Best video + best audio (merged)
"bestvideo[height<=720]"  // Max 720p
```

### Playlist Settings

| Setting | Description | Example |
|---------|-------------|---------|
| `name` | Display name for the playlist | `"My Music Playlist"` |
| `url` | Full YouTube playlist URL | `https://www.youtube.com/playlist?list=...` |
| `destination` | Where videos are saved | `/home/user/Music/YouTubeDownloads` |
| `enabled` | Enable/disable this playlist | `true` or `false` |

---

## Usage

### Interactive Usage

```bash
# Download all enabled playlists
./youtube_downloader.sh run

# Download a specific playlist
./youtube_downloader.sh run "My Music Playlist"

# View statistics
./youtube_downloader.sh stats

# List last 20 downloaded videos
./youtube_downloader.sh list 20

# Show all configured playlists
./youtube_downloader.sh playlists

# View help
./youtube_downloader.sh help

# Reset database (WARNING: deletes all download history)
./youtube_downloader.sh reset
```

### Cron Job Usage

Run downloads automatically on a schedule. Use `--cron` flag to disable colors and log only to file.

#### Example Cron Jobs

**Download all playlists daily at 2 AM:**
```bash
0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run >> /home/user/.youtube_downloader/cron.log 2>&1
```

**Download specific playlist every 6 hours:**
```bash
0 */6 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run "My Music Playlist"
```

**Download all playlists every Sunday at midnight:**
```bash
0 0 * * 0 /home/user/.youtube_downloader/youtube_downloader.sh --cron run
```

#### Setting Up Cron

1. **Open cron editor:**
   ```bash
   crontab -e
   ```

2. **Add a cron job** (example: daily at 2 AM):
   ```bash
   0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run
   ```

3. **Save and exit** (in nano: `Ctrl+X`, then `Y`, then `Enter`)

4. **Verify cron job:**
   ```bash
   crontab -l
   ```

#### Important Notes for Cron

- **Use absolute paths** – cron doesn't have the same environment as your shell
- **Specify full path** to the script: `/home/user/.youtube_downloader/youtube_downloader.sh`
- **Colors are disabled** automatically in cron mode (detected via `--cron` flag)
- **All output logged** to `$HOME/.youtube_downloader/downloader.log`
- **Ensure permissions** – make the script executable: `chmod +x youtube_downloader.sh`

---

## Database

The script maintains a **SQLite database** at `~/.youtube_downloader/downloads.db` with the following tables:

### `downloaded_videos`

Tracks all downloaded videos to prevent duplicates.

| Column | Type | Purpose |
|--------|------|---------|
| `id` | INTEGER | Unique record ID |
| `video_id` | TEXT | YouTube video ID (unique) |
| `title` | TEXT | Video title |
| `url` | TEXT | Full YouTube URL |
| `playlist_name` | TEXT | Which playlist it came from |
| `download_date` | TIMESTAMP | When it was downloaded |
| `file_path` | TEXT | Full path to saved file |
| `file_size` | INTEGER | File size in bytes |
| `status` | TEXT | Status (always `'completed'` for valid downloads) |

### `download_sessions`

Logs each download run with statistics.

| Column | Type | Purpose |
|--------|------|---------|
| `id` | INTEGER | Session ID |
| `playlist_name` | TEXT | Which playlist was downloaded |
| `playlist_url` | TEXT | Playlist URL |
| `session_start` | TIMESTAMP | When the session started |
| `session_end` | TIMESTAMP | When the session finished |
| `videos_downloaded` | INTEGER | Count of newly downloaded videos |
| `videos_skipped` | INTEGER | Count of already-downloaded videos |
| `errors` | INTEGER | Count of download failures |

---

## Logs

All activity is logged to **`~/.youtube_downloader/downloader.log`**

### Log Levels

- **[INFO]** – General information (downloads, moves, session start/end)
- **[ERROR]** – Critical issues (failed downloads, missing directories)
- **[WARNING]** – Non-critical issues (missing destinations, disabled playlists)
- **[DEBUG]** – Detailed diagnostic info (only if `debug: true` in config)

### Viewing Logs

```bash
# View entire log
cat ~/.youtube_downloader/downloader.log

# View last 50 lines
tail -50 ~/.youtube_downloader/downloader.log

# Follow logs in real-time (while script is running)
tail -f ~/.youtube_downloader/downloader.log

# View errors only
grep "ERROR" ~/.youtube_downloader/downloader.log
```

---

## Common Tasks

### Add a New Playlist

1. Get the playlist URL from YouTube
2. Edit `~/.config/youtube_downloader/config.json`
3. Add a new object to the `playlists` array:

```json
{
  "name": "New Playlist Name",
  "url": "https://www.youtube.com/playlist?list=PLAYLIST_ID",
  "destination": "/path/to/save/videos",
  "enabled": true
}
```

4. Save and run: `./youtube_downloader.sh run "New Playlist Name"`

### Disable a Playlist Temporarily

Set `"enabled": false` for that playlist in `config.json`. The playlist will be skipped during automatic runs but can still be downloaded manually by name.

### Change Download Location

Edit the `destination` field for a playlist in `config.json`. New videos will go to the new location; old videos remain where they were.

### Download Audio Only

In `config.json`, set:
```json
"audio_only": true,
"video_format": "best"
```

The script will extract audio only and save as MP3/M4A.

### Reduce File Sizes

Change `video_format` to download lower quality:
```json
"video_format": "worst[ext=mp4]"
```

Or cap at 720p:
```json
"video_format": "bestvideo[height<=720]+bestaudio"
```

### Clear Download History

**WARNING: This is irreversible!**

```bash
./youtube_downloader.sh reset
```

Then confirm with `yes`.

---

## Troubleshooting

### Issue: "Config file not found"

**Solution:** Ensure `~/.config/youtube_downloader/config.json` exists and is readable:

```bash
ls -la ~/.config/youtube_downloader/config.json
```

Create it if missing (see [Configuration Guide](#configuration-guide)).

---

### Issue: "jq is not installed"

**Solution:** Install jq:

```bash
# Ubuntu/Debian
sudo apt-get install jq

# macOS
brew install jq

# Fedora
sudo dnf install jq
```

---

### Issue: "yt-dlp is not installed"

**Solution:** Install yt-dlp:

```bash
# Ubuntu/Debian
sudo apt-get install yt-dlp

# macOS
brew install yt-dlp

# Or via pip (any OS)
pip install yt-dlp
```

---

### Issue: "Failed to fetch playlist"

**Possible causes:**
- **Invalid playlist URL** – Double-check the URL format
- **Private/restricted playlist** – YouTube playlists must be public
- **Network issue** – Check internet connection
- **Rate-limited** – Try again later

**Check logs:**
```bash
tail -20 ~/.youtube_downloader/downloader.log
```

---

### Issue: Cron job not running

**Debug steps:**

1. **Check cron is enabled:**
   ```bash
   sudo systemctl status cron
   ```

2. **Verify cron job exists:**
   ```bash
   crontab -l
   ```

3. **Check system mail for errors:**
   ```bash
   mail
   ```

4. **Test the script manually:**
   ```bash
   /home/user/.youtube_downloader/youtube_downloader.sh --cron run
   ```

5. **Add logging to cron job:**
   ```bash
   0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run >> /home/user/.youtube_downloader/cron.log 2>&1
   ```

6. **Check permissions:**
   ```bash
   chmod +x /home/user/.youtube_downloader/youtube_downloader.sh
   ```

---

### Issue: "Permission denied" when running script

**Solution:** Make the script executable:

```bash
chmod +x ~/.youtube_downloader/youtube_downloader.sh
```

---

### Issue: Videos not moving to destination folder

**Possible causes:**
- **Destination folder doesn't exist** – Script will create it automatically
- **Insufficient disk space** – Check available space
- **File permissions** – Ensure write access to destination folder

**Check permissions:**
```bash
ls -la /path/to/destination/
```

---

## Performance Tips

1. **Increase download delay** if hitting YouTube rate limits:
   ```json
   "download_delay": 5
   ```

2. **Lower video quality** to save bandwidth:
   ```json
   "video_format": "worst[ext=mp4]"
   ```

3. **Use audio-only mode** if you only need audio:
   ```json
   "audio_only": true
   ```

4. **Schedule cron jobs during off-peak hours** to avoid network congestion

5. **Monitor database size** – it grows with each download:
   ```bash
   du -sh ~/.youtube_downloader/downloads.db
   ```

---

## File Structure

```
~/Podcast-Pownloader/
├── podcast-downloader.sh    # Main script
├── downloads.db             # SQLite database
├── downloader.log           # Activity log
├── cron.log                 # Cron job output (optional)
└── podcasts.json            # Configuration file
```

---

## License

This project is provided as-is for personal use. Please respect YouTube's Terms of Service and copyright laws when downloading content.

---

## Contributing

Found a bug or have a feature request? Feel free to submit issues or improvements!

---

## Disclaimer

**This tool is for personal use only.** Users are responsible for:
- Complying with YouTube's Terms of Service
- Respecting copyright and intellectual property rights
- Not using the tool to circumvent any protections or restrictions
- Ensuring all downloads comply with local laws

The authors assume no liability for misuse of this tool.