Podcast Downloader

A robust Bash script for downloading YouTube playlists with automatic video and audio extraction, deduplication tracking, and comprehensive logging.

Disclaimer

This tool is provided for educational and personal use only. Users are responsible for ensuring their use complies with all applicable laws and regulations, including but not limited to copyright, intellectual property, and terms of service agreements. Downloading content from YouTube or other platforms may violate their terms of service or local laws depending on your jurisdiction and the nature of the content. Before using this script, verify that you have the legal right to download and store the content you intend to access. The authors and contributors of this script assume no liability for any misuse or legal consequences that may arise from its use. Always respect content creators' rights and obtain appropriate permissions when necessary.

---

Overview

This script downloads YouTube playlists while maintaining a persistent database of downloaded content to prevent duplicates. It separates video and audio streams, processes them independently, and stores them in organized directories with full logging and error handling.

Features

• Playlist Processing: Downloads all videos from a YouTube playlist with progress tracking
• Dual Stream Extraction: Separates video and audio into distinct files for flexibility
• Deduplication: Maintains a database of downloaded videos to skip previously processed content
• Robust Error Handling: Comprehensive validation, retry logic, and error recovery
• Rate Limiting: Configurable bandwidth throttling to respect server resources
• Lock Mechanism: File-based locking prevents concurrent download conflicts
• Log Rotation: Automatic log file rotation when size exceeds threshold
• Time Estimation: Calculates and displays estimated completion time before processing
• Progress Tracking: Real-time feedback with countdown between downloads

Requirements

• Bash 4.0 or higher
• yt-dlp (YouTube downloader)
• Standard Unix utilities: grep, awk, find, mv, mkdir, stat, date
• Sufficient disk space for temporary and final storage

Installation

1. Clone or download this script to your system
2. Create the configuration file at /opt/podcast-downloader/PODCAST_NAME/podcast_settings.conf (if you change the config file location, remember to update the "CONFIG_FILE=" in the podcast_downloader.sh script)
3. Update the configuration with your specific paths and preferences (see Configuration section)
4. Make the script executable:

chmod +x podcast_downloader.sh

5. Ensure yt-dlp is installed

Configuration

Create a configuration file at the path specified in CONFIG_FILE. Required variables:

Variable	Description	Example
DOWNLOAD_DIR	Temporary directory for intermediate files	/tmp/
FINAL_DIR	Directory where processed videos and audio are saved	/location/to/save/directory
LOG_DIR	Directory for storing log files	/opt/podcast-downloader/PODCAST_NAME/
LOG_FILE	Path to the main activity log	/opt/podcast-downloader/PODCAST_NAME/.youtube_downloader.log
DOWNLOADS_LOG	Path to the downloads database	/opt/podcast-downloader/PODCAST_NAME/downloaded_episodes.log
DOWNLOADS_LOCK	Path to lock directory (must not exist)	/opt/podcast-downloader/PODCAST_NAME/.podcast_downloads.lock
MAX_LOG_SIZE	Maximum log file size in bytes before rotation	10485760 (10 MB)
QUALITY	yt-dlp format specification	137+140
VIDEO_FORMAT	Output video container format	mp4
AUDIO_FORMAT	Audio format (not currently used in output)	m4a
SLEEP_INTERVAL	Seconds to wait between downloads	120
RATE_LIMIT	Bandwidth limit for downloads	3M
CONNECTION_TIMEOUT	Socket timeout in seconds	30
MAX_RETRIES	Number of retries for failed downloads	3
PLAYLIST_URL	Default YouTube playlist URL	https://www.youtube.com/playlist?list=YOUR_LIST_ID

Usage

Basic Usage

Run with a playlist URL as an argument:

./podcast_downloader.sh "https://www.youtube.com/playlist?list=YOUR_LIST_ID"

Using Configuration Default

If PLAYLIST_URL is set in the configuration file, run without arguments:

./podcast_downloader.sh

Output Structure

Directory Layout

FINAL_DIR/
  video_title.mp4
  video_title.mp3
  another_video.mp4
  another_video.mp3

LOG_DIR/
  .youtube_downloader.log
  .youtube_downloader.log.1234567890
  downloaded_episodes.log

Downloads Database Format

The downloaded_episodes.log file uses pipe-delimited format:

VIDEO_ID|TITLE|FILENAME|DOWNLOADED_DATE
dQw4w9WgXcQ|Example Video|Example Video.mp4|2024-01-15 14:30:45

Log Format

[2024-01-15 14:30:45] Starting download for playlist: https://www.youtube.com/playlist?list=...
[2024-01-15 14:30:47] Found 150 videos to process
[2024-01-15 14:31:02] [1/150] Downloading: 'Example Video'
[2024-01-15 14:35:10] [1/150] Successfully processed 'Example Video'

Functions

Core Download Functions

• download_playlist(): Main function that processes all videos in a playlist
• process_downloaded_video(): Moves downloaded files to final directory and updates database
• is_downloaded(): Checks if a video ID exists in the downloads database
• record_download(): Adds a new entry to the downloads database

Utility Functions

• acquire_lock() / release_lock(): Manages concurrent access protection
• validate_config(): Verifies all required configuration variables are set
• sanitize_filename(): Removes invalid characters from filenames
• log_message(): Writes timestamped messages to log file with automatic rotation
• rotate_log(): Rotates log file when size limit is exceeded
• estimate_download_time(): Calculates approximate total processing time
• format_time(): Converts seconds to human-readable duration format
• wait_for_next(): Displays countdown between downloads
• show_log(): Displays the last 20 downloads from the database

Error Handling

The script implements multiple error handling strategies:

• Configuration validation at startup ensures all required variables are present
• Lock-based synchronization prevents concurrent execution issues
• Retry logic with configurable limits handles transient network failures
• File existence checks before processing prevent data loss
• Filename deduplication adds video ID suffix if collision is detected
• Graceful cleanup of temporary files via trap handlers

Exit Codes

• 0: Successful completion
• 1: Configuration error, missing files, or download failure

Logging

All operations are logged with timestamps. The script provides:

• Download start and completion messages
• Per-video progress indicators with current count and total count
• Detailed error messages with context
• Skipped video notifications
• Database write confirmations
• Total execution time

Access logs at the path specified by LOG_FILE. Historical logs are retained with timestamp suffixes when rotation occurs.

Notes

• The script creates separate video (mp4) and audio (mp3) files for each download
• Filenames are automatically sanitized to remove invalid filesystem characters
• Duplicate file names are handled by appending the video ID to the base name
• Lock mechanism uses directory creation atomicity for reliability
• The script respects rate limiting to avoid overwhelming download sources
• Clean shutdown is performed even if interrupted, removing temporary files

Troubleshooting

Configuration file not found: Verify the path in CONFIG_FILE variable matches your actual configuration file location.

Failed to fetch playlist: Check your internet connection and ensure the playlist URL is correct and publicly accessible.

Lock timeout: If the script repeatedly fails to acquire lock, check for stale lock directories at the DOWNLOADS_LOCK path and remove manually if necessary.

Insufficient disk space: Ensure DOWNLOAD_DIR and FINAL_DIR have adequate free space. Temporary files are cleaned up automatically after processing.

Invalid video ID format: This warning indicates yt-dlp returned unexpected data. Verify the playlist is valid and contains standard YouTube videos.