Updated for freetube/piped formats, subscriptions and chunked playlist
+943 −166 · 4 files changed
-
README.md+315 −37 -
requirements.txt+1 −0 -
src/yt_playlist_export/__main__.py+9 −2 -
src/yt_playlist_export/yt_playlist_export.py+618 −127
@@ -1,74 +1,352 @@ # yt-playlist-export -yt-playlist-export is a tool that allows to export YouTube playlists, export YouTube "Watch later" list and export YouTube "Liked videos". It supports exporting to JSON and CSV formats. +Utilities to **export YouTube data using `yt-dlp`** into formats you actually use: -## Installation +* **FreeTube** playlists (append to `playlists.db` or standalone JSON) +* **Piped** import (JSON and CSV, optional chunk split) +* **URL / ID lists** +* **NewPipe** subscriptions JSON (from your `/feed/channels`) -```bash {"id":"01J4HFTRJ1CK76GD127E1R65GX"} -python3 -m pip install yt-playlist-export +> Works with private playlists/channels via `--cookies` or `--browser-cookies` (programmatic `cookiesfrombrowser`). + +--- + +## Features + +### Exports + +* **FreeTube (append to DB)** + Append one or more playlists (or an ID list) into FreeTube’s line-delimited `playlists.db`. + +* **FreeTube (standalone JSON)** + Same structure as above, but written to a file for manual import/inspection. + +* **Piped (JSON)** + `{"format":"Piped","version":1,"playlists":[{ name, type, visibility, videos: [urls] }]}` + + * **Split mode:** chunk large playlists into many files with `--split N`. + +* **Piped (CSV)** + Minimal CSV: `videoId,addedAt` for quick bulk adds. + +* **NewPipe subscriptions** + Export your **subscriptions** as NewPipe JSON (`newpipe-subscriptions-YYYY-MM-DD.json`) using **`/feed/channels`**. + + * No HTML scraping, no OPML endpoint required. + * Requires valid login cookies. + +* **URL / ID lists** + Flat text files (`urls.txt`, `ids.txt`) for pipelines or custom tooling. + +### Inputs + +* **Playlist URL(s)** (public/private if cookies provided) +* **ID file** (`--ids-file`) with one YouTube video ID per line (tolerates lines like `youtube <id>`) + +### Auth / Cookies + +* **`--cookies`**: Netscape cookie file path +* **`--browser-cookies`**: `cookiesfrombrowser` like `"brave:Default"`, correctly passed to `yt-dlp` as a tuple +* **`--skip-authcheck`**: skip yt-dlp auth check for `youtube:tab` extractors when needed + +### Ergonomics + +* **`--pretty`** JSON +* **`--sleep`** between requests to avoid rate-limits +* **`--verbose`** to see `yt-dlp` chatter +* **Atomic-ish writes** for crash-safe `--out` paths (when applicable in some flows) +* Clean, deterministic file naming and chunking + +--- + +## Install + +```bash +python -m pip install yt-dlp colorama +# or uv/pipx/poetry as you prefer ``` -## Usage +Place the script anywhere on your PATH (e.g. `yt-playlist-export.py`) and make it executable. -```bash {"id":"01J4HFTRJ1CK76GD127F41SF28"} -yt-playlist-export -h -usage: yt-playlist-export [-h] [--browser BROWSER] [--cookies COOKIES] [-f {csv,json}] [-o OUTPUT] [-v] playlist [playlist ...] +--- -yt-playlist-export exports YouTube playlists to JSON or CSV +## Quick Start -positional arguments: - playlist playlist URL +### 1) Export **NewPipe subscriptions** (uses `/feed/channels`) -optional arguments: - -h, --help show this help message and exit - --browser BROWSER browser where you logged in to YouTube - --cookies COOKIES absolute path to a Netscape formatted cookie file - -f {csv,json}, --format {csv,json} - output's format - -o OUTPUT, --output OUTPUT - filename to store output, by default it's printed to stdout - -v, --verbose verbose +```bash +python yt-playlist-export.py \ + --browser-cookies "brave:Default" \ + -e newpipe-subs \ + -o newpipe-subscriptions-2025-11-10.json \ + --pretty ``` -### Use case: Exporting public playlist +* Requires you’re logged into YouTube in the specified browser/profile. +* You can also use `--cookies /path/to/cookies.txt` instead. + +### 2) Export a **YouTube playlist → FreeTube DB** ```bash -yt-playlist-export -f csv -o test.csv https://www.youtube.com/playlist?list=PLO4kDC0EWkeCnf8PBRk7XDPhrzmAxHLah +python yt-playlist-export.py \ + --browser-cookies "brave:Default" \ + -e freetube-db \ + "https://www.youtube.com/playlist?list=PLxxxxxxxxxxxxxxxx" ``` -### Use case: Exporting using cookies from default browser (firefox) +This appends to your OS-specific FreeTube `playlists.db` (auto-located). +Override with `--path /path/to/FreeTube/playlists.db` or `--path /dir/` to create if missing. + +### 3) Export a **YouTube playlist → FreeTube JSON** ```bash -yt-playlist-export -f csv -o test.csv https://www.youtube.com/playlist?list=PLO4kDC0EWkeCnf8PBRk7XDPhrzmAxHLah +python yt-playlist-export.py \ + --cookies cookies.txt \ + -e freetube-json \ + -o my_playlist.freetube.json \ + "https://www.youtube.com/playlist?list=PLxxxxxxxxxxxxxxxx" ``` -### Use case: Exporting using Chrome browser +### 4) Export a **YouTube playlist → Piped JSON** (single file) -At the moment of writing this supported browsers are: firefox, safari, brave, chrome, chromium, edge, opera, vivaldi, whale. +```bash +python yt-playlist-export.py \ + --browser-cookies "brave:Default" \ + -e piped-json \ + -o playlist-piped.json \ + "https://www.youtube.com/playlist?list=PLxxxxxxxxxxxxxxxx" \ + --pretty +``` -For the `--browser` agrument one can also pass `BROWSER[+KEYRING][:PROFILE][::CONTAINER]` as it's described in [yt-dlp](https://github.com/yt-dlp/yt-dlp) documentation. -Check out `--cookies-from-browser` option in yt-dlp documentation to understand how it works. +### 5) Export a **YouTube playlist → Piped JSON (split into chunks of 500)** ```bash -yt-playlist-export --browser chrome -f csv -o test.csv https://www.youtube.com/playlist?list=PLO4kDC0EWkeCnf8PBRk7XDPhrzmAxHLah +python yt-playlist-export.py \ + --browser-cookies "brave:Default" \ + -e piped-json \ + --split 500 \ + -o my_piped.json \ + --split-dir ./chunks \ + "https://www.youtube.com/playlist?list=PLxxxxxxxxxxxxxxxx" ``` -### Use case: Exporting using cookie file +Outputs: `./chunks/my_piped_<sanitized-name>_001.json`, `..._002.json`, etc. -The cookie file must be in a Netscape format. You can follow [this guide](https://github.com/yt-dlp/yt-dlp/issues/10927) and use [this extension](https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc) for Chrome. +### 6) Export **IDs file → FreeTube / Piped / lists** + +Given `ids.txt`: -```bash -yt-playlist-export --cookies /home/daydiff/cookie.txt -f csv -o test.csv https://www.youtube.com/playlist?list=PLO4kDC0EWkeCnf8PBRk7XDPhrzmAxHLah ``` +dQw4w9WgXcQ +youtube 1DxWY0nLEF0 +``` + +* **FreeTube JSON**: + ```bash + python yt-playlist-export.py -f ids.txt -e freetube-json --name "Imported IDs" -o imported.freetube.json --pretty + ``` -## Show case +* **Piped CSV**: -```bash {"id":"01J4HFTRJ1CK76GD127FGZF1W0"} -yt-playlist-export https://www.youtube.com/playlist?list=PLO4kDC0EWkeCnf8PBRk7XDPhrzmAxHLah + ```bash + python yt-playlist-export.py -f ids.txt -e piped-csv -o imported.csv + ``` + +* **URLs**: + + ```bash + python yt-playlist-export.py -f ids.txt -e urls -o urls.txt + ``` + +--- + +## CLI + +```text +usage: yt-playlist-export.py [-h] [-f IDS_FILE] [--name NAME] [--description DESCRIPTION] + [-c BROWSER_COOKIES] [--cookies COOKIES] [--skip-authcheck] [--sleep SLEEP] + [-e {freetube-db,freetube-json,piped-json,piped-csv,urls,ids,newpipe-subs}] + [-o OUTPUT] [--path PATH] [--split N] [--split-dir SPLIT_DIR] [--pretty] + [--newpipe-version NEWPIPE_VERSION] [--newpipe-version-int NEWPIPE_VERSION_INT] + [-q] [--verbose] + [playlist_url ...] ``` - +**Inputs** + +* `playlist_url ...` One or more YouTube playlist URLs +* `-f, --ids-file FILE` Text file with video IDs (one per line; also accepts lines like `youtube <id>`) +* `--name NAME` Override/assign playlist name (single URL or IDs mode) +* `--description TEXT` Optional description + +**Auth / yt-dlp** + +* `-c, --browser-cookies "brave:Default"` Use logged-in browser cookies (programmatic tuple) +* `--cookies PATH` Netscape cookie file +* `--skip-authcheck` `youtubetab:skip=authcheck` for certain private listings +* `--sleep SECS` Sleep between requests + +**Exports** + +* `-e, --export` `freetube-db | freetube-json | piped-json | piped-csv | urls | ids | newpipe-subs` +* `-o, --output` Output file (not used for `freetube-db`) +* `--path PATH` Override FreeTube `playlists.db` (file OR directory) + +**Piped JSON** + +* `--split N` Chunk size (e.g. 500) +* `--split-dir DIR` Where to put the chunks (default `chunks/<basename>`) +* `--pretty` Pretty JSON + +**NewPipe** + +* `--newpipe-version` String, default `0.26.0` +* `--newpipe-version-int` Int, default `1200` + +**Misc** + +* `-q, --quiet` Reduce logging +* `--verbose` More `yt-dlp` logging + +--- + +## Output Formats + +### FreeTube playlist JSON (single object, used for DB and file export) + +```json +{ + "playlistName": "My Playlist", + "protected": false, + "description": "", + "videos": [ + { + "videoId": "1DxWY0nLEF0", + "title": "10 Most Effective Pushup Variations You Really Need", + "author": "CHRIS HERIA", + "authorId": "@CHRISHERIA", + "lengthSeconds": 724, + "published": 0, + "timeAdded": 1731048855213, + "playlistItemId": "uuid-…", + "type": "video" + } + ], + "_id": "ft-playlist--uuid…", + "createdAt": 1731048855213, + "lastUpdatedAt": 1731048855213 +} +``` + +### Piped JSON + +```json +{ + "format": "Piped", + "version": 1, + "playlists": [ + { + "name": "My Playlist", + "type": "playlist", + "visibility": "private", + "videos": [ + "https://youtube.com/watch?v=1DxWY0nLEF0", + "https://youtube.com/watch?v=WzFMnRUzYog" + ] + } + ] +} +``` + +### NewPipe subscriptions + +```json +{ + "app_version": "0.26.0", + "app_version_int": 1200, + "subscriptions": [ + { + "service_id": 0, + "url": "https://www.youtube.com/channel/UC-0igYFlnYv1XCF7wk30BTg", + "name": "Ace" + } + ] +} +``` + +--- + +## Notes & Gotchas + +* **Cookies:** + + * For `--browser-cookies`, use strings like `"brave:Default"`, `"firefox:default-release"`, `"chromium:Default"` + * Internally, the script converts to the **tuple** format `yt-dlp` expects. + * If you see “failed to load cookies”, confirm the profile name and that you’re logged in. + +* **Private / unlisted:** + Use cookies. Consider `--skip-authcheck` if yt-dlp complains for certain tabs. + +* **Rate limits:** + Use `--sleep 0.5` (or similar) for large exports. + +* **FreeTube DB path:** + Auto-detected per OS. Override with `--path /path/to/playlists.db` or a directory to create one. + +* **yt-dlp version:** + Keep it current. Old versions can fail on `/feed/channels` or playlist tabs. + +--- + +## Examples + +* Multiple playlists → FreeTube JSON directory: + + ```bash + python yt-playlist-export.py \ + --browser-cookies "firefox:default-release" \ + -e freetube-json \ + -o ./ft_json \ + --pretty \ + "https://www.youtube.com/playlist?list=PL1..." \ + "https://www.youtube.com/playlist?list=PL2..." + ``` + +* One playlist → Piped JSON (split, pretty): + + ```bash + python yt-playlist-export.py \ + --cookies cookies.txt \ + -e piped-json \ + -o piped_export.json \ + --split 500 \ + --pretty \ + "https://www.youtube.com/playlist?list=PL..." + ``` + +* IDs file → URL list: + + ```bash + python yt-playlist-export.py -f ids.txt -e urls -o urls.txt + ``` + +--- + +## License + +MIT (or your preferred license — add a `LICENSE` file). + +--- + +## Changelog (high-level) + +* Add NewPipe subscriptions export via `/feed/channels` (no OPML scraping) +* Robust `cookiesfrombrowser` parsing → correct programmatic tuple +* Piped JSON split + dir controls +* FreeTube DB append + standalone JSON +* IDs-file workflow; placeholder entries on partial failures +* Pretty JSON, sleep, verbose, quiet modes ## Dependencies @@ -1 +1,2 @@ yt_dlp +colorama @@ -1,3 +1,10 @@ +import sys +from . import main + if __name__ == "__main__": - from yt_playlist_export import main - main() + try: + main() + except KeyboardInterrupt: + print("\nAborted by user.\n") + sys.exit(130) + @@ -1,139 +1,630 @@ -#!/bin/env python3 +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +ytft-export.py + +Export YouTube playlists/IDs → FreeTube DB/JSON, Piped JSON/CSV, URL/ID lists +Export YouTube subscriptions → NewPipe subscriptions JSON (via /feed/channels) + +Requires: + - yt-dlp + - colorama (optional; for colored console output) +""" + +from __future__ import annotations import argparse import csv +import json +import os import re import sys -import json +from dataclasses import dataclass, asdict +from time import time as _time +from typing import Any, Dict, Iterable, List, Optional, Tuple +from uuid import uuid4 + +# ---- optional colors ---- +try: + from colorama import init as color_init, Fore + color_init() + HAVE_COLOR = True +except Exception: + HAVE_COLOR = False + import yt_dlp from yt_dlp.cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS -parser = argparse.ArgumentParser(description='yt-playlist-export exports YouTube playlists to JSON or CSV') -parser.add_argument('--browser', type=str, required=False, default='firefox', help='browser where you logged in to YouTube') -parser.add_argument('--cookies', type=str, required=False, help='absolute path to a Netscape formatted cookie file') -parser.add_argument('-f', '--format', type=str, choices=['csv', 'json'], required=False, default='csv', help='output\'s format') -parser.add_argument('-o', '--output', type=str, required=False, help='filename to store output, by default it\'s printed to stdout') -parser.add_argument('-v', '--verbose', action='store_true', help='verbose') -parser.add_argument('playlist', nargs='+', help='playlist URL') -args = parser.parse_args() - - -# print(args) -# exit(0) - -playlist = args.playlist[0] -format = args.format -output = args.output -verbose = args.verbose - -id = 'id' -url = 'url' -title = 'title' -duration = 'duration' -channel_id = 'channel id' -channel_name = 'channel name' -channel_url = 'channel url' -duration = 'duration (seconds)' -view_count = 'view count' -thumbnail_xs = 'thumbnail xs' -thumbnail_s = 'thumbnail s' -thumbnail_m = 'thumbnail m' -thumbnail_l = 'thumbnail l' - -header = [ - id, - url, - title, - duration, - channel_id, - channel_name, - channel_url, - view_count, - thumbnail_xs, - thumbnail_s, - thumbnail_m, - thumbnail_l, -] - -# Converts ytl JSON structure to yt-playlist-export, in particular removing all not populated field -def convert_to_csv(info: any) -> any: - for entry in info['entries']: - yield { - id: entry['id'], - url: entry['url'], - title: entry['title'], - duration: entry['duration'], - channel_id: entry['channel_id'], - channel_name: entry['channel'], - channel_url: entry['channel_url'], - view_count: entry['view_count'], - thumbnail_xs: entry['thumbnails'][0]['url'] if len(entry['thumbnails']) > 0 else "", - thumbnail_s: entry['thumbnails'][1]['url'] if len(entry['thumbnails']) > 1 else "", - thumbnail_m: entry['thumbnails'][2]['url'] if len(entry['thumbnails']) > 2 else "", - thumbnail_l: entry['thumbnails'][3]['url'] if len(entry['thumbnails']) > 3 else "", + +# ========================= +# Console utils +# ========================= + +def cprint(head: str, rest: str = "", color: str = "RESET", quiet: bool = False) -> None: + if quiet: + return + if not HAVE_COLOR: + print(f"{head}{rest}") + return + cmap = {"RED": Fore.RED, "GREEN": Fore.GREEN, "YELLOW": Fore.YELLOW, "CYAN": Fore.CYAN, "RESET": Fore.RESET} + sys.stdout.write(cmap.get(color, Fore.RESET) + head + Fore.RESET) + if rest: + print(rest) + + +# ========================= +# Text helpers +# ========================= + +ZERO_WIDTH = re.compile(r"[\u200B-\u200F\uFEFF]") + +def clean_text(s: Any) -> str: + if s is None: + return "" + s = str(s).strip().replace("\u00A0", " ").replace("\u2024", ".") + return ZERO_WIDTH.sub("", s) + +def to_int_str(val: Any) -> str: + if val is None: + return "" + return re.sub(r"[^\d]", "", str(val)) + +def sanitize_name(name: str) -> str: + return re.sub(r"[^A-Za-z0-9._-]+", "_", name).strip("_") + +def now_ms() -> int: + return int(round(_time() * 1000)) + + +# ========================= +# cookiesfrombrowser parsing (shared) +# ========================= + +_CFB_RE = re.compile(r'''(?x) + (?P<name>[^+:]+) + (?:\s*\+\s*(?P<keyring>[^:]+))? + (?:\s*:\s*(?!:)(?P<profile>.+?))? + (?:\s*::\s*(?P<container>.+))? +''') + +def parse_cookiesfrombrowser_spec(cfb: str) -> tuple[str, Optional[str], Optional[str], Optional[str]]: + """ + Convert "brave:Default" (or similar) to a tuple (name, profile, keyring, container) + suitable for yt_dlp programmatic options. + """ + m = _CFB_RE.fullmatch(cfb.strip()) + if not m: + raise ValueError(f"Invalid cookiesfrombrowser spec: {cfb}") + name, keyring, profile, container = m.group("name", "keyring", "profile", "container") + name = name.lower() + if name not in SUPPORTED_BROWSERS: + raise ValueError(f'Unsupported browser "{name}". Supported: {", ".join(sorted(SUPPORTED_BROWSERS))}') + if keyring: + keyring = keyring.upper() + if keyring not in SUPPORTED_KEYRINGS: + raise ValueError(f'Unsupported keyring "{keyring}". Supported: {", ".join(sorted(SUPPORTED_KEYRINGS))}') + return (name, profile, keyring, container) + + +# ========================= +# Data models (FreeTube) +# ========================= + +@dataclass +class FreeTubeVideo: + videoId: str + title: str = "N/A" + author: str = "N/A" + authorId: str = "N/A" + lengthSeconds: int | str = 0 + published: int | str = 0 + timeAdded: int = 0 + playlistItemId: str = "" + type: str = "video" + + @staticmethod + def from_ytdlp_entry(e: Dict[str, Any]) -> "FreeTubeVideo": + return FreeTubeVideo( + videoId=clean_text(e.get("id") or "N/A"), + title=clean_text(e.get("title") or "N/A"), + author=clean_text(e.get("channel") or e.get("uploader") or "N/A"), + authorId=clean_text(e.get("channel_id") or e.get("uploader_id") or "N/A"), + lengthSeconds=int(to_int_str(e.get("duration") or 0) or 0), + published=int(to_int_str(e.get("timestamp") or 0) or 0), + timeAdded=now_ms(), + playlistItemId=str(uuid4()), + ) + + @staticmethod + def placeholder(video_id: str) -> "FreeTubeVideo": + return FreeTubeVideo(videoId=video_id, timeAdded=now_ms(), playlistItemId=str(uuid4())) + + +@dataclass +class FreeTubePlaylist: + playlistName: str + protected: bool = False + description: str = "" + videos: List[FreeTubeVideo] = None + _id: str = "" + createdAt: int = 0 + lastUpdatedAt: int = 0 + + @staticmethod + def new(name: str, description: str = "") -> "FreeTubePlaylist": + ts = now_ms() + return FreeTubePlaylist( + playlistName=name or "Imported Playlist", + protected=False, + description=description or "", + videos=[], + _id="ft-playlist--" + str(uuid4()), + createdAt=ts, + lastUpdatedAt=ts, + ) + + def add_video(self, v: FreeTubeVideo) -> None: + self.videos.append(v) + self.lastUpdatedAt = now_ms() + + def to_json_obj(self) -> Dict[str, Any]: + d = asdict(self) + d["videos"] = [asdict(v) for v in self.videos] + return d + + +# ========================= +# yt-dlp client +# ========================= + +class YTDLPClient: + def __init__( + self, + cookies_file: Optional[str], + browser_cookies: Optional[str], + skip_authcheck: bool, + sleep_requests: Optional[float], + quiet: bool, + ) -> None: + self.quiet = quiet + self.opts: Dict[str, Any] = { + "ignoreerrors": "only_download", + "extract_flat": True, + "quiet": True, } + if cookies_file: + self.opts["cookies"] = cookies_file + else: + # Use tuple spec programmatically + self.opts["cookiesfrombrowser"] = parse_cookiesfrombrowser_spec(browser_cookies or "firefox") + if skip_authcheck: + self.opts["extractor_args"] = {"youtubetab": {"skip": ["authcheck"]}} + if sleep_requests is not None: + self.opts["sleep_interval_requests"] = float(sleep_requests) -def print_csv(info: any): - global output - if output is None: - csv_file = sys.stdout - else: - csv_file = open(output, mode='w', newline='', encoding='utf-8') - writer = csv.DictWriter(csv_file, fieldnames=header) - writer.writeheader() - for row in convert_to_csv(info): - writer.writerow(row) - -def print_json(info: any): - global output - if output is None: - print(json.dumps(info)) + def extract_playlist(self, url: str) -> Dict[str, Any]: + with yt_dlp.YoutubeDL(self.opts) as ydl: + info = ydl.extract_info(url, download=False) + if not info: + raise RuntimeError(f"yt-dlp returned no data for {url}") + return ydl.sanitize_info(info) + + def extract_video_min(self, video_id: str) -> Dict[str, Any]: + url = f"https://www.youtube.com/watch?v={video_id}" + try: + with yt_dlp.YoutubeDL(self.opts) as ydl: + info = ydl.extract_info(url, download=False) + if not info: + return {"id": video_id} + return ydl.sanitize_info(info) + except Exception: + return {"id": video_id} + + +# ========================= +# Inputs +# ========================= + +YT_ID_RE = re.compile(r"([A-Za-z0-9_-]{10}[AEIMQUYcgkosw048])") + +def read_ids_file(path: str) -> List[str]: + ids: List[str] = [] + with open(path, "r", encoding="utf-8") as f: + for raw in f: + line = raw.strip() + if not line or line.startswith("#"): + continue + m = YT_ID_RE.search(line) + if m: + ids.append(m.group(1)) + return ids + + +# ========================= +# Exporters +# ========================= + +def freetube_db_path(user_path: Optional[str]) -> str: + database_name = "playlists.db" + if user_path: + if os.path.isfile(user_path) or user_path.lower().endswith((".db", ".json")): + return user_path + return os.path.join(user_path, database_name) + home = os.path.expanduser("~") + if sys.platform.startswith("win"): + base = os.path.join(os.getenv("APPDATA") or "", "FreeTube") + elif sys.platform.startswith("linux"): + flatpak = os.path.join(home, ".var/app/io.freetubeapp.FreeTube/config/FreeTube") + base = flatpak if os.path.exists(flatpak) else os.path.join(home, ".config/FreeTube") + elif sys.platform == "darwin": + base = os.path.join(home, "Library/Application Support/FreeTube") else: - with open(output, 'w') as f: - f.write(json.dumps(info)) - -def parse_browser_arg(cookiesfrombrowser): - # taken from yt-dlp: https://github.com/yt-dlp/yt-dlp/blob/6ca23ffaa4663cb552f937f0b1e9769b66db11bd/yt_dlp/__init__.py#L382-L402 - if cookiesfrombrowser: - container = None - mobj = re.fullmatch(r'''(?x) - (?P<name>[^+:]+) - (?:\s*\+\s*(?P<keyring>[^:]+))? - (?:\s*:\s*(?!:)(?P<profile>.+?))? - (?:\s*::\s*(?P<container>.+))? - ''', cookiesfrombrowser) - if mobj is None: - raise ValueError(f'invalid cookies from browser arguments: {cookiesfrombrowser}') - browser_name, keyring, profile, container = mobj.group('name', 'keyring', 'profile', 'container') - browser_name = browser_name.lower() - if browser_name not in SUPPORTED_BROWSERS: - raise ValueError(f'unsupported browser specified for cookies: "{browser_name}". ' - f'Supported browsers are: {", ".join(sorted(SUPPORTED_BROWSERS))}') - if keyring is not None: - keyring = keyring.upper() - if keyring not in SUPPORTED_KEYRINGS: - raise ValueError(f'unsupported keyring specified for cookies: "{keyring}". ' - f'Supported keyrings are: {", ".join(sorted(SUPPORTED_KEYRINGS))}') - return (browser_name, profile, keyring, container) - -# See help(yt_dlp.YoutubeDL) for a list of available options and public functions -ydl_opts = { - 'ignoreerrors': 'only_download', - 'extract_flat': True, - 'quiet': not verbose -} - -if args.cookies: - ydl_opts['cookies'] = args.cookies -else: - ydl_opts['cookiesfrombrowser'] = parse_browser_arg(args.browser) - -def main(): - with yt_dlp.YoutubeDL(ydl_opts) as ydl: - info = ydl.extract_info(playlist, download=False) - # ydl.sanitize_info makes the info json-serializable - info_sanitized = ydl.sanitize_info(info) - if format == 'json': - print_json(info_sanitized) + base = os.path.join(home, ".config/FreeTube") + return os.path.join(base, database_name) + +def ensure_file_exists(path: str) -> None: + parent = os.path.dirname(path) or "." + if not os.path.isdir(parent): + raise FileNotFoundError(f"Parent directory does not exist: {parent}") + if not os.path.exists(path): + open(path, "a", encoding="utf-8").close() + +def export_freetube_db(pl: FreeTubePlaylist, user_path: Optional[str], quiet: bool) -> str: + db_path = freetube_db_path(user_path) + ensure_file_exists(db_path) + with open(db_path, "a", encoding="utf-8") as f: + json.dump(pl.to_json_obj(), f, separators=(",", ":")) + f.write("\n") + f.flush() + os.fsync(f.fileno()) + cprint("✓ ", f"Playlist '{pl.playlistName}' appended to {db_path}", "GREEN", quiet) + return db_path + +def export_freetube_json(pl: FreeTubePlaylist, out_path: str, pretty: bool, quiet: bool) -> str: + path = out_path or (sanitize_name(pl.playlistName) + ".freetube.json") + with open(path, "w", encoding="utf-8") as f: + if pretty: + json.dump(pl.to_json_obj(), f, ensure_ascii=False, indent=2) else: - print_csv(info_sanitized) + json.dump(pl.to_json_obj(), f, ensure_ascii=False, separators=(",", ":")) + cprint("✓ ", f"Wrote FreeTube JSON: {path}", "GREEN", quiet) + return path + +def export_piped_json(playlists: List[Tuple[str, List[str]]], out_path: Optional[str], + pretty: bool, quiet: bool) -> str: + payload = { + "format": "Piped", + "version": 1, + "playlists": [ + {"name": name, "type": "playlist", "visibility": "private", + "videos": [f"https://youtube.com/watch?v={v}" for v in vids]} + for (name, vids) in playlists + ], + } + path = out_path or "piped.json" + with open(path, "w", encoding="utf-8") as f: + if pretty: + json.dump(payload, f, ensure_ascii=False, indent=2) + else: + json.dump(payload, f, ensure_ascii=False, separators=(",", ":")) + cprint("✓ ", f"Wrote Piped JSON: {path}", "GREEN", quiet) + return path + +def export_piped_json_split(playlists: List[Tuple[str, List[str]]], base_out: str, + split: int, split_dir: Optional[str], pretty: bool, quiet: bool) -> str: + base = os.path.splitext(os.path.basename(base_out or "piped.json"))[0] + outdir = split_dir or os.path.join("chunks", base) + os.makedirs(outdir, exist_ok=True) + total = 0 + for name, vids in playlists: + for i in range(0, len(vids), split): + chunk = vids[i:i+split] + payload = {"format": "Piped", "version": 1, + "playlists": [{"name": name, "type": "playlist", "visibility": "private", + "videos": [f"https://youtube.com/watch?v={v}" for v in chunk]}]} + out_name = f"{base}_{sanitize_name(name)}_{(i//split)+1:03}.json" + path = os.path.join(outdir, out_name) + with open(path, "w", encoding="utf-8") as f: + if pretty: + json.dump(payload, f, ensure_ascii=False, indent=2) + else: + json.dump(payload, f, ensure_ascii=False, separators=(",", ":")) + total += 1 + if not quiet: + print(f" → {path} ({len(chunk)} videos)") + cprint("✓ ", f"Split Piped JSON finished: {total} files in {outdir}", "GREEN", quiet) + return outdir + +def export_piped_csv(video_ids: Iterable[str], out_path: Optional[str], quiet: bool) -> str: + path = out_path or "piped.csv" + with open(path, "w", encoding="utf-8", newline="") as f: + w = csv.writer(f, delimiter=",", quotechar='"', lineterminator="\r\n") + w.writerow(["videoId", "addedAt"]) + for vid in video_ids: + w.writerow([vid, ""]) + cprint("✓ ", f"Wrote Piped CSV: {path}", "GREEN", quiet) + return path + +def export_urls(video_ids: Iterable[str], out_path: Optional[str], quiet: bool) -> str: + path = out_path or "urls.txt" + with open(path, "w", encoding="utf-8") as f: + for vid in video_ids: + f.write(f"https://www.youtube.com/watch?v={vid}\n") + cprint("✓ ", f"Wrote URL list: {path}", "GREEN", quiet) + return path + +def export_ids(video_ids: Iterable[str], out_path: Optional[str], quiet: bool) -> str: + path = out_path or "ids.txt" + with open(path, "w", encoding="utf-8") as f: + for vid in video_ids: + f.write(f"{vid}\n") + cprint("✓ ", f"Wrote ID list: {path}", "GREEN", quiet) + return path + + +# ========================= +# NewPipe subscriptions (yt-dlp /feed/channels) +# ========================= + +def fetch_subscriptions_via_ytdlp( + browser_cookies: str | None, + cookies_file: str | None, + verbose: bool = False, +) -> list[tuple[str, str]]: + """ + Use yt-dlp to extract the /feed/channels subscription list for a logged-in account. + Returns a list of (channel_url, name). + """ + url = "https://www.youtube.com/feed/channels" + opts: Dict[str, Any] = { + "extract_flat": True, + "skip_download": True, + "quiet": not verbose, + "ignoreerrors": "only_download", + "lazy_playlist": True, + } + if cookies_file: + opts["cookies"] = cookies_file + elif browser_cookies: + # IMPORTANT: pass a tuple, not a raw string + opts["cookiesfrombrowser"] = parse_cookiesfrombrowser_spec(browser_cookies) + + results: list[tuple[str, str]] = [] + with yt_dlp.YoutubeDL(opts) as ydl: + info = ydl.extract_info(url, download=False) + if not info or "entries" not in info: + raise RuntimeError("Could not extract subscriptions list (login cookies may be invalid).") + for e in info.get("entries", []): + if not e: + continue + ch_id = e.get("channel_id") or e.get("id") + name = e.get("title") or e.get("channel") or "Unknown" + if not ch_id: + continue + ch_url = f"https://www.youtube.com/channel/{ch_id}" + results.append((ch_url, name)) + return results + +def export_newpipe_subscriptions( + subs: list[tuple[str, str]], + out_path: str, + app_ver: str = "0.26.0", + app_ver_int: int = 1200, + pretty: bool = False, + quiet: bool = False, +): + payload = { + "app_version": app_ver, + "app_version_int": app_ver_int, + "subscriptions": [{"service_id": 0, "url": url, "name": name} for (url, name) in subs], + } + with open(out_path, "w", encoding="utf-8") as f: + if pretty: + json.dump(payload, f, ensure_ascii=False, indent=2) + else: + json.dump(payload, f, ensure_ascii=False, separators=(",", ":")) + cprint("✓ ", f"Wrote NewPipe subscriptions: {out_path} ({len(subs)} channels)", "GREEN", quiet) + + +# ========================= +# Builders +# ========================= + +def build_freetube_from_entries(info: Dict[str, Any], name: Optional[str], description: str) -> FreeTubePlaylist: + entries = info.get("entries") or [] + pl_name = name or clean_text(info.get("title") or "") or "Imported Playlist" + ft = FreeTubePlaylist.new(pl_name, description) + for e in entries: + vid = clean_text(e.get("id") or "") + if not vid: + continue + ft.add_video(FreeTubeVideo.from_ytdlp_entry(e)) + return ft + +def build_freetube_from_ids(ids: List[str], ytdlp: YTDLPClient, + name: str, description: str, quiet: bool) -> FreeTubePlaylist: + ft = FreeTubePlaylist.new(name or f"Imported IDs ({len(ids)} videos)", description) + for vid in ids: + meta = ytdlp.extract_video_min(vid) + if "id" in meta and meta.get("title") is not None: + ft.add_video(FreeTubeVideo.from_ytdlp_entry(meta)) + else: + ft.add_video(FreeTubeVideo.placeholder(vid)) + if not quiet: + print(f"{vid} | {ft.videos[-1].title} | {ft.videos[-1].author}") + return ft + + +# ========================= +# CLI +# ========================= + +def build_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser(description="Export YouTube playlists/IDs → FreeTube, Piped, NewPipe") + # Inputs + p.add_argument("playlist_url", nargs="*", help="YouTube playlist URL(s)") + p.add_argument("-f", "--ids-file", help="File with YouTube IDs (one per line)") + p.add_argument("--name", help="Playlist name (single URL or IDs mode)") + p.add_argument("--description", default="", help="Playlist description (IDs mode or override)") + + # yt-dlp auth/behavior + p.add_argument("-c", "--browser-cookies", help='cookiesfrombrowser, e.g. "brave:Default"') + p.add_argument("--cookies", help="Path to a Netscape cookie file") + p.add_argument("--skip-authcheck", action="store_true", help="yt-dlp: youtubetab:skip=authcheck") + p.add_argument("--sleep", type=float, help="Sleep between requests (seconds)") + + # Output type + p.add_argument("-e", "--export", + choices=["freetube-db", "freetube-json", "piped-json", "piped-csv", "urls", "ids", "newpipe-subs"], + default="freetube-db", help="Export format") + p.add_argument("-o", "--output", help="Output path (file). Not used for freetube-db.") + p.add_argument("--path", help="Override FreeTube playlists.db path (file or dir)") + + # Piped JSON controls + p.add_argument("--split", type=int, metavar="N", help="Split Piped JSON into chunks of N videos") + p.add_argument("--split-dir", help="Output directory for split JSON files (default chunks/<basename>)") + p.add_argument("--pretty", action="store_true", help="Pretty-print JSON outputs") + + # NewPipe subscriptions + p.add_argument("--newpipe-version", default="0.19.8", help='NewPipe app_version') + p.add_argument("--newpipe-version-int", type=int, default=953, help="NewPipe app_version_int") + + # Verbosity + p.add_argument("-q", "--quiet", action="store_true", help="Reduce log output") + p.add_argument("--verbose", action="store_true", help="Increase yt-dlp verbosity for debugging") + return p + + +# ========================= +# Main +# ========================= + +def main() -> None: + args = build_parser().parse_args() + + # NewPipe subscriptions export + if args.export == "newpipe-subs": + cprint("⋯ ", "Fetching subscriptions via yt-dlp /feed/channels…", "CYAN", args.quiet) + try: + subs = fetch_subscriptions_via_ytdlp( + browser_cookies=args.browser_cookies, + cookies_file=args.cookies, + verbose=args.verbose, + ) + except Exception as e: + cprint("ERR: ", f"Failed to fetch subscriptions: {e}", "RED") + sys.exit(1) + + if not subs: + cprint("WARN: ", "No subscriptions found — ensure cookies are logged in.", "YELLOW", args.quiet) + sys.exit(2) + + export_newpipe_subscriptions( + subs=subs, + out_path=args.output or "newpipe-subscriptions.json", + app_ver=args.newpipe_version, + app_ver_int=args.newpipe_version_int, + pretty=args.pretty, + quiet=args.quiet, + ) + return + + # Playlist / IDs exporters + have_urls = len(args.playlist_url) > 0 + have_ids = bool(args.ids_file) + # Show help if no input arguments provided + if not have_urls and not have_ids and args.export != "newpipe-subs": + build_parser().print_help() + sys.exit(0) + + if not have_urls and not have_ids: + cprint("ERR: ", "Provide playlist URL(s) or --ids-file, or use -e newpipe-subs", "RED") + sys.exit(2) + + ytdlp_client = YTDLPClient( + cookies_file=args.cookies, + browser_cookies=args.browser_cookies, + skip_authcheck=args.skip_authcheck, + sleep_requests=args.sleep, + quiet=args.quiet, + ) + + freetube_playlists: List[FreeTubePlaylist] = [] + piped_sets: List[Tuple[str, List[str]]] = [] + + if have_ids: + ids = read_ids_file(args.ids_file) + if not ids: + cprint("ERR: ", "No valid YouTube IDs found.", "RED") + sys.exit(1) + default_name = args.name or os.path.splitext(os.path.basename(args.ids_file))[0] or f"Imported IDs ({len(ids)})" + ft = build_freetube_from_ids(ids, ytdlp_client, default_name, args.description, args.quiet) + freetube_playlists.append(ft) + piped_sets.append((ft.playlistName, [v.videoId for v in ft.videos])) + + for url in args.playlist_url: + info = ytdlp_client.extract_playlist(url) + name = args.name if len(args.playlist_url) == 1 and args.name else None + ft = build_freetube_from_entries(info, name, args.description) + freetube_playlists.append(ft) + piped_sets.append((ft.playlistName, [v.videoId for v in ft.videos])) + + if args.export == "freetube-db": + for pl in freetube_playlists: + export_freetube_db(pl, args.path, args.quiet) + return + + if args.export == "freetube-json": + if len(freetube_playlists) == 1: + export_freetube_json(freetube_playlists[0], args.output or "", args.pretty, args.quiet) + else: + outdir = args.output or "freetube_json_playlists" + os.makedirs(outdir, exist_ok=True) + for pl in freetube_playlists: + path = os.path.join(outdir, sanitize_name(pl.playlistName) + ".json") + export_freetube_json(pl, path, args.pretty, args.quiet) + return + + if args.export == "piped-json": + if args.split: + export_piped_json_split(piped_sets, args.output or "piped.json", + args.split, args.split_dir, args.pretty, args.quiet) + else: + export_piped_json(piped_sets, args.output or "piped.json", args.pretty, args.quiet) + return + + if args.export == "piped-csv": + vids: List[str] = [] + for _, v in piped_sets: + vids.extend(v) + export_piped_csv(vids, args.output or "piped.csv", args.quiet) + return + + if args.export == "urls": + vids: List[str] = [] + for _, v in piped_sets: + vids.extend(v) + export_urls(vids, args.output or "urls.txt", args.quiet) + return + + if args.export == "ids": + vids: List[str] = [] + for _, v in piped_sets: + vids.extend(v) + export_ids(vids, args.output or "ids.txt", args.quiet) + return + + cprint("ERR: ", f"Unknown export type: {args.export}", "RED") + sys.exit(2) + +if __name__ == "__main__": + try: + main() + except BrokenPipeError: + try: sys.stdout.close() + except Exception: pass + try: sys.stderr.close() + except Exception: pass + sys.exit(0) +