Added V0.1
This commit is contained in:
@@ -1,3 +1,539 @@
|
||||
# Podcast-Downloader
|
||||
|
||||
A script to check for podcasts, download them and move them to a specified folder
|
||||
|
||||
A **bash script** that automatically downloads videos from YouTube playlists using `yt-dlp`, maintains a local SQLite database of downloaded content, and prevents duplicate downloads. Supports multiple playlists with individual destination folders and **cron job integration** for automated scheduling.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| **Multi-Playlist Support** | Configure and download from multiple YouTube playlists simultaneously |
|
||||
| **Duplicate Prevention** | SQLite database tracks all downloaded videos to prevent re-downloading |
|
||||
| **Individual Destinations** | Assign a different destination folder for each playlist |
|
||||
| **Cron-Compatible** | Run automatically on a schedule with proper logging |
|
||||
| **Download Sessions** | Detailed statistics on each download run (downloaded, skipped, errors) |
|
||||
| **Database Statistics** | View total videos downloaded, storage used, and per-playlist breakdowns |
|
||||
| **Configurable Format** | Specify video format, download delays, and audio-only mode via config |
|
||||
| **Automatic Cleanup** | Removes temporary download folders after successful transfers |
|
||||
| **Colored Output** | Terminal-friendly logging (disabled in cron mode) |
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before using this script, ensure you have the following tools installed:
|
||||
|
||||
### Required
|
||||
|
||||
- **bash** (version 4.0 or higher)
|
||||
- **yt-dlp** – YouTube video downloader ([install](https://github.com/yt-dlp/yt-dlp#installation))
|
||||
- **sqlite3** – Database management
|
||||
- **jq** – JSON parser for config file handling
|
||||
|
||||
### Installation
|
||||
|
||||
**Ubuntu/Debian:**
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install yt-dlp sqlite3 jq
|
||||
```
|
||||
|
||||
**macOS:**
|
||||
```bash
|
||||
brew install yt-dlp sqlite3 jq
|
||||
```
|
||||
|
||||
**Fedora/RHEL:**
|
||||
```bash
|
||||
sudo dnf install yt-dlp sqlite3 jq
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### 1. Clone or Download the Script
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.youtube_downloader
|
||||
cd ~/.youtube_downloader
|
||||
# Download or clone the script here
|
||||
chmod +x youtube_downloader.sh
|
||||
```
|
||||
|
||||
### 2. Create Configuration File
|
||||
|
||||
Create the config file at `~/.config/youtube_downloader/config.json`:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.config/youtube_downloader
|
||||
```
|
||||
|
||||
Then create `~/.config/youtube_downloader/config.json` with the following structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"general": {
|
||||
"db_dir": "$HOME/.youtube_downloader",
|
||||
"temp_download_dir": "/tmp/youtube_downloads",
|
||||
"video_format": "best[ext=mp4]",
|
||||
"output_template": "%(title)s.%(ext)s",
|
||||
"audio_only": false,
|
||||
"download_delay": 2,
|
||||
"debug": false
|
||||
},
|
||||
"playlists": [
|
||||
{
|
||||
"name": "My Music Playlist",
|
||||
"url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_1",
|
||||
"destination": "/home/user/Music/YouTubeDownloads",
|
||||
"enabled": true
|
||||
},
|
||||
{
|
||||
"name": "Tutorial Playlist",
|
||||
"url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_2",
|
||||
"destination": "/home/user/Videos/Tutorials",
|
||||
"enabled": true
|
||||
},
|
||||
{
|
||||
"name": "Disabled Playlist",
|
||||
"url": "https://www.youtube.com/playlist?list=PLAYLIST_ID_3",
|
||||
"destination": "/home/user/Videos/Archive",
|
||||
"enabled": false
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Find Your Playlist IDs
|
||||
|
||||
To get a playlist ID:
|
||||
1. Go to a YouTube playlist
|
||||
2. Look at the URL: `https://www.youtube.com/playlist?list=PLAYLIST_ID_HERE`
|
||||
3. Copy the part after `list=`
|
||||
|
||||
---
|
||||
|
||||
## Configuration Guide
|
||||
|
||||
### General Settings
|
||||
|
||||
| Setting | Description | Example |
|
||||
|---------|-------------|---------|
|
||||
| `db_dir` | Directory for database and logs | `$HOME/.youtube_downloader` |
|
||||
| `temp_download_dir` | Temporary folder for downloads (cleaned up after) | `/tmp/youtube_downloads` |
|
||||
| `video_format` | yt-dlp format specification | `best[ext=mp4]` or `worst` |
|
||||
| `output_template` | Filename pattern (yt-dlp variables) | `%(title)s.%(ext)s` |
|
||||
| `audio_only` | Download audio only (ignores video_format) | `true` or `false` |
|
||||
| `download_delay` | Seconds between downloads (avoids rate-limiting) | `2` |
|
||||
| `debug` | Enable debug logging | `true` or `false` |
|
||||
|
||||
### Video Format Examples
|
||||
|
||||
```json
|
||||
"best[ext=mp4]" // Best quality MP4
|
||||
"worst" // Lowest quality (smallest file)
|
||||
"bestvideo+bestaudio" // Best video + best audio (merged)
|
||||
"bestvideo[height<=720]" // Max 720p
|
||||
```
|
||||
|
||||
### Playlist Settings
|
||||
|
||||
| Setting | Description | Example |
|
||||
|---------|-------------|---------|
|
||||
| `name` | Display name for the playlist | `"My Music Playlist"` |
|
||||
| `url` | Full YouTube playlist URL | `https://www.youtube.com/playlist?list=...` |
|
||||
| `destination` | Where videos are saved | `/home/user/Music/YouTubeDownloads` |
|
||||
| `enabled` | Enable/disable this playlist | `true` or `false` |
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Interactive Usage
|
||||
|
||||
```bash
|
||||
# Download all enabled playlists
|
||||
./youtube_downloader.sh run
|
||||
|
||||
# Download a specific playlist
|
||||
./youtube_downloader.sh run "My Music Playlist"
|
||||
|
||||
# View statistics
|
||||
./youtube_downloader.sh stats
|
||||
|
||||
# List last 20 downloaded videos
|
||||
./youtube_downloader.sh list 20
|
||||
|
||||
# Show all configured playlists
|
||||
./youtube_downloader.sh playlists
|
||||
|
||||
# View help
|
||||
./youtube_downloader.sh help
|
||||
|
||||
# Reset database (WARNING: deletes all download history)
|
||||
./youtube_downloader.sh reset
|
||||
```
|
||||
|
||||
### Cron Job Usage
|
||||
|
||||
Run downloads automatically on a schedule. Use `--cron` flag to disable colors and log only to file.
|
||||
|
||||
#### Example Cron Jobs
|
||||
|
||||
**Download all playlists daily at 2 AM:**
|
||||
```bash
|
||||
0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run >> /home/user/.youtube_downloader/cron.log 2>&1
|
||||
```
|
||||
|
||||
**Download specific playlist every 6 hours:**
|
||||
```bash
|
||||
0 */6 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run "My Music Playlist"
|
||||
```
|
||||
|
||||
**Download all playlists every Sunday at midnight:**
|
||||
```bash
|
||||
0 0 * * 0 /home/user/.youtube_downloader/youtube_downloader.sh --cron run
|
||||
```
|
||||
|
||||
#### Setting Up Cron
|
||||
|
||||
1. **Open cron editor:**
|
||||
```bash
|
||||
crontab -e
|
||||
```
|
||||
|
||||
2. **Add a cron job** (example: daily at 2 AM):
|
||||
```bash
|
||||
0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run
|
||||
```
|
||||
|
||||
3. **Save and exit** (in nano: `Ctrl+X`, then `Y`, then `Enter`)
|
||||
|
||||
4. **Verify cron job:**
|
||||
```bash
|
||||
crontab -l
|
||||
```
|
||||
|
||||
#### Important Notes for Cron
|
||||
|
||||
- **Use absolute paths** – cron doesn't have the same environment as your shell
|
||||
- **Specify full path** to the script: `/home/user/.youtube_downloader/youtube_downloader.sh`
|
||||
- **Colors are disabled** automatically in cron mode (detected via `--cron` flag)
|
||||
- **All output logged** to `$HOME/.youtube_downloader/downloader.log`
|
||||
- **Ensure permissions** – make the script executable: `chmod +x youtube_downloader.sh`
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
The script maintains a **SQLite database** at `~/.youtube_downloader/downloads.db` with the following tables:
|
||||
|
||||
### `downloaded_videos`
|
||||
|
||||
Tracks all downloaded videos to prevent duplicates.
|
||||
|
||||
| Column | Type | Purpose |
|
||||
|--------|------|---------|
|
||||
| `id` | INTEGER | Unique record ID |
|
||||
| `video_id` | TEXT | YouTube video ID (unique) |
|
||||
| `title` | TEXT | Video title |
|
||||
| `url` | TEXT | Full YouTube URL |
|
||||
| `playlist_name` | TEXT | Which playlist it came from |
|
||||
| `download_date` | TIMESTAMP | When it was downloaded |
|
||||
| `file_path` | TEXT | Full path to saved file |
|
||||
| `file_size` | INTEGER | File size in bytes |
|
||||
| `status` | TEXT | Status (always `'completed'` for valid downloads) |
|
||||
|
||||
### `download_sessions`
|
||||
|
||||
Logs each download run with statistics.
|
||||
|
||||
| Column | Type | Purpose |
|
||||
|--------|------|---------|
|
||||
| `id` | INTEGER | Session ID |
|
||||
| `playlist_name` | TEXT | Which playlist was downloaded |
|
||||
| `playlist_url` | TEXT | Playlist URL |
|
||||
| `session_start` | TIMESTAMP | When the session started |
|
||||
| `session_end` | TIMESTAMP | When the session finished |
|
||||
| `videos_downloaded` | INTEGER | Count of newly downloaded videos |
|
||||
| `videos_skipped` | INTEGER | Count of already-downloaded videos |
|
||||
| `errors` | INTEGER | Count of download failures |
|
||||
|
||||
---
|
||||
|
||||
## Logs
|
||||
|
||||
All activity is logged to **`~/.youtube_downloader/downloader.log`**
|
||||
|
||||
### Log Levels
|
||||
|
||||
- **[INFO]** – General information (downloads, moves, session start/end)
|
||||
- **[ERROR]** – Critical issues (failed downloads, missing directories)
|
||||
- **[WARNING]** – Non-critical issues (missing destinations, disabled playlists)
|
||||
- **[DEBUG]** – Detailed diagnostic info (only if `debug: true` in config)
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
```bash
|
||||
# View entire log
|
||||
cat ~/.youtube_downloader/downloader.log
|
||||
|
||||
# View last 50 lines
|
||||
tail -50 ~/.youtube_downloader/downloader.log
|
||||
|
||||
# Follow logs in real-time (while script is running)
|
||||
tail -f ~/.youtube_downloader/downloader.log
|
||||
|
||||
# View errors only
|
||||
grep "ERROR" ~/.youtube_downloader/downloader.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Add a New Playlist
|
||||
|
||||
1. Get the playlist URL from YouTube
|
||||
2. Edit `~/.config/youtube_downloader/config.json`
|
||||
3. Add a new object to the `playlists` array:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "New Playlist Name",
|
||||
"url": "https://www.youtube.com/playlist?list=PLAYLIST_ID",
|
||||
"destination": "/path/to/save/videos",
|
||||
"enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
4. Save and run: `./youtube_downloader.sh run "New Playlist Name"`
|
||||
|
||||
### Disable a Playlist Temporarily
|
||||
|
||||
Set `"enabled": false` for that playlist in `config.json`. The playlist will be skipped during automatic runs but can still be downloaded manually by name.
|
||||
|
||||
### Change Download Location
|
||||
|
||||
Edit the `destination` field for a playlist in `config.json`. New videos will go to the new location; old videos remain where they were.
|
||||
|
||||
### Download Audio Only
|
||||
|
||||
In `config.json`, set:
|
||||
```json
|
||||
"audio_only": true,
|
||||
"video_format": "best"
|
||||
```
|
||||
|
||||
The script will extract audio only and save as MP3/M4A.
|
||||
|
||||
### Reduce File Sizes
|
||||
|
||||
Change `video_format` to download lower quality:
|
||||
```json
|
||||
"video_format": "worst[ext=mp4]"
|
||||
```
|
||||
|
||||
Or cap at 720p:
|
||||
```json
|
||||
"video_format": "bestvideo[height<=720]+bestaudio"
|
||||
```
|
||||
|
||||
### Clear Download History
|
||||
|
||||
**WARNING: This is irreversible!**
|
||||
|
||||
```bash
|
||||
./youtube_downloader.sh reset
|
||||
```
|
||||
|
||||
Then confirm with `yes`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: "Config file not found"
|
||||
|
||||
**Solution:** Ensure `~/.config/youtube_downloader/config.json` exists and is readable:
|
||||
|
||||
```bash
|
||||
ls -la ~/.config/youtube_downloader/config.json
|
||||
```
|
||||
|
||||
Create it if missing (see [Configuration Guide](#configuration-guide)).
|
||||
|
||||
---
|
||||
|
||||
### Issue: "jq is not installed"
|
||||
|
||||
**Solution:** Install jq:
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
sudo apt-get install jq
|
||||
|
||||
# macOS
|
||||
brew install jq
|
||||
|
||||
# Fedora
|
||||
sudo dnf install jq
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue: "yt-dlp is not installed"
|
||||
|
||||
**Solution:** Install yt-dlp:
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
sudo apt-get install yt-dlp
|
||||
|
||||
# macOS
|
||||
brew install yt-dlp
|
||||
|
||||
# Or via pip (any OS)
|
||||
pip install yt-dlp
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue: "Failed to fetch playlist"
|
||||
|
||||
**Possible causes:**
|
||||
- **Invalid playlist URL** – Double-check the URL format
|
||||
- **Private/restricted playlist** – YouTube playlists must be public
|
||||
- **Network issue** – Check internet connection
|
||||
- **Rate-limited** – Try again later
|
||||
|
||||
**Check logs:**
|
||||
```bash
|
||||
tail -20 ~/.youtube_downloader/downloader.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue: Cron job not running
|
||||
|
||||
**Debug steps:**
|
||||
|
||||
1. **Check cron is enabled:**
|
||||
```bash
|
||||
sudo systemctl status cron
|
||||
```
|
||||
|
||||
2. **Verify cron job exists:**
|
||||
```bash
|
||||
crontab -l
|
||||
```
|
||||
|
||||
3. **Check system mail for errors:**
|
||||
```bash
|
||||
mail
|
||||
```
|
||||
|
||||
4. **Test the script manually:**
|
||||
```bash
|
||||
/home/user/.youtube_downloader/youtube_downloader.sh --cron run
|
||||
```
|
||||
|
||||
5. **Add logging to cron job:**
|
||||
```bash
|
||||
0 2 * * * /home/user/.youtube_downloader/youtube_downloader.sh --cron run >> /home/user/.youtube_downloader/cron.log 2>&1
|
||||
```
|
||||
|
||||
6. **Check permissions:**
|
||||
```bash
|
||||
chmod +x /home/user/.youtube_downloader/youtube_downloader.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue: "Permission denied" when running script
|
||||
|
||||
**Solution:** Make the script executable:
|
||||
|
||||
```bash
|
||||
chmod +x ~/.youtube_downloader/youtube_downloader.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue: Videos not moving to destination folder
|
||||
|
||||
**Possible causes:**
|
||||
- **Destination folder doesn't exist** – Script will create it automatically
|
||||
- **Insufficient disk space** – Check available space
|
||||
- **File permissions** – Ensure write access to destination folder
|
||||
|
||||
**Check permissions:**
|
||||
```bash
|
||||
ls -la /path/to/destination/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Increase download delay** if hitting YouTube rate limits:
|
||||
```json
|
||||
"download_delay": 5
|
||||
```
|
||||
|
||||
2. **Lower video quality** to save bandwidth:
|
||||
```json
|
||||
"video_format": "worst[ext=mp4]"
|
||||
```
|
||||
|
||||
3. **Use audio-only mode** if you only need audio:
|
||||
```json
|
||||
"audio_only": true
|
||||
```
|
||||
|
||||
4. **Schedule cron jobs during off-peak hours** to avoid network congestion
|
||||
|
||||
5. **Monitor database size** – it grows with each download:
|
||||
```bash
|
||||
du -sh ~/.youtube_downloader/downloads.db
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
~/Podcast-Pownloader/
|
||||
├── podcast-downloader.sh # Main script
|
||||
├── downloads.db # SQLite database
|
||||
├── downloader.log # Activity log
|
||||
├── cron.log # Cron job output (optional)
|
||||
└── podcasts.json # Configuration file
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This project is provided as-is for personal use. Please respect YouTube's Terms of Service and copyright laws when downloading content.
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Found a bug or have a feature request? Feel free to submit issues or improvements!
|
||||
|
||||
---
|
||||
|
||||
## Disclaimer
|
||||
|
||||
**This tool is for personal use only.** Users are responsible for:
|
||||
- Complying with YouTube's Terms of Service
|
||||
- Respecting copyright and intellectual property rights
|
||||
- Not using the tool to circumvent any protections or restrictions
|
||||
- Ensuring all downloads comply with local laws
|
||||
|
||||
The authors assume no liability for misuse of this tool.
|
||||
|
||||
Reference in New Issue
Block a user