Compare commits

..

40 commits

Author SHA1 Message Date
f134abe8f6
Merge remote-tracking branch 'search/url-search' into custom 2024-08-18 02:36:51 -07:00
7209bae58c
Merge remote-tracking branch 'oauth/master' into custom 2024-08-18 02:36:43 -07:00
9da7e7b227
Merge remote-tracking branch 'chapters/add-chapters' into custom 2024-08-18 02:36:05 -07:00
Samantaz Fox
85deea5aca
Search: Change smart search inhibitor to a backslash 2024-08-17 19:22:40 +02:00
Samantaz Fox
78c5ba93c7
Misc: Clean some code in UrlSanitizer 2024-08-17 19:22:40 +02:00
Samantaz Fox
31a80420ec
Search: Add URL search inhibition logic 2024-08-17 19:22:40 +02:00
Samantaz Fox
4c0b5c314d
Search: Add support for youtu.be and youtube.com URLs 2024-08-17 19:22:40 +02:00
Samantaz Fox
eb0f651812
Add a youtube URL sanitizer 2024-08-17 19:22:40 +02:00
syeopite
36ed5d3418
Lint 2024-04-28 21:07:03 -07:00
syeopite
f48aa0a2c2
Don't use generic click handler for chapter widget 2024-04-28 20:15:38 -07:00
syeopite
93a6464bbe
Code quality fixes 2024-04-28 20:15:38 -07:00
syeopite
503ace90f5
Refactor: Add object to represent chapters
Prior to this commit we used an Array of Chapter structs to represent
a video's chapters. However, as we often needed to apply operations on
the entire sequence of chapters, multiple isolated functions had to be
created and in turn clogged up the code.

By grouping everything together under a chapters struct that stores a
sequence of chapters, these functions can be grouped together, and can
be simplifed due to instance variables containing the data that they need.

Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2024-04-28 20:15:38 -07:00
syeopite
2744ea2244
Use Time::Span for timestamps in chapter struct 2024-04-28 20:15:38 -07:00
syeopite
b0e0e19017
Use WebVTT.build for chapters vtt file 2024-04-28 20:15:38 -07:00
syeopite
310825997f
Use proxy url for chapter thumbnails in JSON API 2024-04-28 20:15:38 -07:00
syeopite
2dc17d7409
Add data for chapters to JSON endpoint for videos 2024-04-28 20:15:37 -07:00
syeopite
6bddcea178
Add separate method for constructing chapters json 2024-04-28 20:15:37 -07:00
syeopite
3860c69a52
Move parsed chapters info to "extra video infos" 2024-04-28 20:15:36 -07:00
syeopite
9535009864
Use short-hand block notation for parsing chapters 2024-04-28 20:14:40 -07:00
syeopite
6f62de36d7
Properly camelcase auto gen chapters attribute 2024-04-28 20:14:40 -07:00
syeopite
6f295bb33b
Escape localization for desc chapters widget 2024-04-28 20:14:40 -07:00
syeopite
e53a483bcb
Change order of chapters button within the player 2024-04-28 20:14:40 -07:00
syeopite
5732a2a394
Add field for whether or not chapter is auto gen 2024-04-28 20:14:39 -07:00
syeopite
65d9914329
Remove extraneous space between desc and date 2024-04-28 20:14:39 -07:00
syeopite
9f43a74871
Localize chapters label 2024-04-28 20:14:38 -07:00
syeopite
a569b8f3d9
Add message for when chapters are auto generated 2024-04-28 20:14:09 -07:00
syeopite
00d16dff1f
Add data-jump-time attribute to chapters component
Allows automatically jumping to specified time instead of reloading the
page.
2024-04-28 20:14:09 -07:00
syeopite
08d82cc749
Fix missing timestamp on first chapter 2024-04-28 20:14:09 -07:00
syeopite
3fba6f5728
Fix description chapter component design 2024-04-28 20:14:09 -07:00
syeopite
2cd3ded93b
Remove whitespace in chapters component 2024-04-28 20:14:09 -07:00
syeopite
ddd931573a
Fetch chapter thumbnails for selector in desc 2024-04-28 20:14:09 -07:00
syeopite
48ba6373df
Remove initial whitespace from video description 2024-04-28 20:14:09 -07:00
syeopite
0afd95fb78
Add initial html for chapters selector in desc 2024-04-28 20:14:09 -07:00
syeopite
d9aeb2c360
Add chapters track to player.ecr 2024-04-28 20:14:09 -07:00
syeopite
39b0229835
Add method to convert chapters to vtt 2024-04-28 20:14:09 -07:00
syeopite
98c6cee383
Add chapters data to API 2024-04-28 20:14:08 -07:00
syeopite
371dbd73fe
Add logic to parse video chapters 2024-04-28 20:10:25 -07:00
soraefir
8953c105be
Implemented requested changes and added 'auth_enforce_source' option 2023-07-13 12:48:18 +02:00
soraefir
4d14789e7b
Added login flow functions and fix cookies 2023-07-12 01:09:29 +02:00
soraefir
bee301a6f4
Implemented Oauth 2023-07-12 00:38:03 +02:00
25 changed files with 707 additions and 70 deletions

View file

@ -160,7 +160,9 @@ body a.pure-button {
button.pure-button-primary,
body a.pure-button-primary,
.channel-owner:hover,
.channel-owner:focus {
.channel-owner:focus,
.chapter:hover,
.chapter:focus {
background-color: #a0a0a0;
color: rgba(35, 35, 35, 1);
}
@ -797,5 +799,26 @@ h1, h2, h3, h4, h5, p,
}
#download_widget {
width: 100%;
width: 100%;
}
.description-chapters-section {
white-space: normal;
}
.description-chapters-content-container {
display: flex;
flex-direction: row;
gap: 5px;
overflow: scroll;
overflow-y: hidden;
}
.chapter {
padding: 3px;
}
.chapter .thumbnail {
width: 200px;
}

View file

@ -114,6 +114,10 @@ ul.vjs-menu-content::-webkit-scrollbar {
order: 5;
}
.vjs-chapters-button {
order: 5;
}
.vjs-share-control {
order: 6;
}

View file

@ -17,6 +17,7 @@ var options = {
'remainingTimeDisplay',
'Spacer',
'captionsButton',
'ChaptersButton',
'audioTrackButton',
'qualitySelector',
'playbackRateMenuButton',

View file

@ -191,3 +191,9 @@ addEventListener('load', function (e) {
comments.innerHTML = '';
}
});
const chapter_widget_buttons = document.getElementsByClassName("chapter-widget-buttons")
Array.from(chapter_widget_buttons).forEach(e => e.addEventListener("click", function (event) {
event.preventDefault();
player.currentTime(e.getAttribute('data-jump-time'));
}))

View file

@ -323,6 +323,40 @@ https_only: false
##
#enable_user_notifications: true
##
## List of Enabled Authentication Backend
## If not provided falls back to default
##
## Supported Values:
## - invidious
## - oauth
## - ldap (Not implemented !)
## - saml (Not implemented !)
##
## Default: ["invidious","oauth"]
##
# auth_type: ["oauth"]
##
## OAuth Configuration
##
## Notes:
## - Supports multiple OAuth backends
## - Requires external_port and domain to be configured
##
## Default: []
##
# oauth:
# example:
# host: oauth.example.net
# field : email
# auth_uri: /oauth/authorize/
# token_uri: /oauth/token/
# info_uri: https://api.example.net/oauth/userinfo/
# client_id: CLIENT_ID
# client_secret: CLIENT_SECRET
# -----------------------------
# Background jobs
# -----------------------------

View file

@ -496,5 +496,7 @@
"toggle_theme": "Toggle Theme",
"carousel_slide": "Slide {{current}} of {{total}}",
"carousel_skip": "Skip the Carousel",
"carousel_go_to": "Go to slide `x`"
"carousel_go_to": "Go to slide `x`",
"video_chapters_label": "Chapters",
"video_chapters_auto_generated_label": "These chapters are auto-generated"
}

View file

@ -8,6 +8,18 @@ struct DBConfig
property dbname : String
end
struct OAuthConfig
include YAML::Serializable
property host : String
property field : String = "email"
property auth_uri : String
property token_uri : String
property info_uri : String
property client_id : String
property client_secret : String
end
struct ConfigPreferences
include YAML::Serializable
@ -137,6 +149,10 @@ class Config
# poToken for passing bot attestation
property po_token : String? = nil
property auth_type : Array(String) = ["invidious", "oauth"]
property auth_enforce_source : Bool = true
property oauth = {} of String => OAuthConfig
# Saved cookies in "name1=value1; name2=value2..." format
@[YAML::Field(converter: Preferences::StringToCookies)]
property cookies : HTTP::Cookies = HTTP::Cookies.new
@ -159,6 +175,14 @@ class Config
end
end
def auth_oauth_enabled?
return (@auth_type.find(&.== "oauth") && @oauth.size > 0)
end
def auth_internal_enabled?
return (@auth_type.find(&.== "invidious"))
end
def self.load
# Load config from file or YAML string env var
env_config_file = "INVIDIOUS_CONFIG_FILE"

View file

@ -0,0 +1,53 @@
require "oauth2"
module Invidious::OAuthHelper
extend self
def get_provider(key)
if provider = CONFIG.oauth[key]?
provider
else
raise Exception.new("Invalid OAuth Endpoint: " + key)
end
end
def make_client(key)
if HOST_URL == ""
raise Exception.new("Missing domain and port configuration")
end
provider = get_provider(key)
redirect_uri = "#{HOST_URL}/login/oauth/#{key}"
OAuth2::Client.new(
provider.host,
provider.client_id,
provider.client_secret,
authorize_uri: provider.auth_uri,
token_uri: provider.token_uri,
redirect_uri: redirect_uri
)
end
def get_uri_host_pair(host, url)
if (url.starts_with?(/https*\:\/\//))
uri = URI.parse url
[uri.host || host, uri.path || "/"]
else
[host, url]
end
end
def get_info(key, token)
provider = self.get_provider(key)
uri_host_pair = self.get_uri_host_pair(provider.host, provider.info_uri)
client = HTTP::Client.new(uri_host_pair[0], tls: true)
token.authenticate(client)
response = client.get uri_host_pair[1]
client.close
response.body
end
def info_field(key, token)
info = JSON.parse(self.get_info(key, token))
info[self.get_provider(key).field].as_s?
end
end

View file

@ -212,6 +212,14 @@ module Invidious::JSONify::APIv1
end
end
if !video.chapters.nil?
json.field "chapters" do
json.object do
video.chapters.to_json(json)
end
end
end
if !video.music.empty?
json.field "musicTracks" do
json.array do

View file

@ -415,4 +415,41 @@ module Invidious::Routes::API::V1::Videos
end
end
end
def self.chapters(env)
id = env.params.url["id"]
region = env.params.query["region"]? || env.params.body["region"]?
if id.nil? || id.size != 11 || !id.matches?(/^[\w-]+$/)
return error_json(400, "Invalid video ID")
end
format = env.params.query["format"]?
begin
video = get_video(id, region: region)
rescue ex : NotFoundException
haltf env, 404
rescue ex
haltf env, 500
end
begin
chapters = video.chapters
rescue ex
haltf env, 500
end
if chapters.nil?
return error_json(404, "No chapters are defined in video \"#{id}\"")
end
if format == "json"
env.response.content_type = "application/json"
return chapters.to_json
else
env.response.content_type = "text/vtt; charset=UTF-8"
return chapters.to_vtt
end
end
end

View file

@ -160,6 +160,12 @@ module Invidious::Routes::Images
id = env.params.url["id"]
name = env.params.url["name"]
# Some thumbnails such as the ones for chapters requires some additional queries.
query_params = HTTP::Params.new
{"sqp", "rs"}.each do |name|
query_params[name] = env.params.query[name] if env.params.query[name]?
end
headers = HTTP::Headers.new
if name == "maxres.jpg"
@ -173,7 +179,7 @@ module Invidious::Routes::Images
end
end
url = "/vi/#{id}/#{name}"
url = "/vi/#{id}/#{name}?#{query_params}"
REQUEST_HEADERS_WHITELIST.each do |header|
if env.request.headers[header]?

View file

@ -3,11 +3,10 @@
module Invidious::Routes::Login
def self.login_page(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
user = env.get? "user"
referer = get_referer(env, "/feed/subscriptions")
return env.redirect referer if user
if !CONFIG.login_enabled
@ -19,7 +18,13 @@ module Invidious::Routes::Login
captcha = nil
account_type = env.params.query["type"]?
account_type ||= "invidious"
account_type ||= ""
if CONFIG.auth_type.size == 0
return error_template(401, "No authentication backend enabled.")
elsif CONFIG.auth_type.find(&.== account_type).nil? && CONFIG.auth_type.size == 1
account_type = CONFIG.auth_type[0]
end
captcha_type = env.params.query["captcha"]?
captcha_type ||= "image"
@ -27,9 +32,38 @@ module Invidious::Routes::Login
templated "user/login"
end
def self.login_oauth(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
authorization_code = env.params.query["code"]?
provider_k = env.params.url["provider"]
if authorization_code.nil?
return error_template(403, "Missing Authorization Code")
end
begin
token = OAuthHelper.make_client(provider_k).get_access_token_using_authorization_code(authorization_code)
if email = OAuthHelper.info_field(provider_k, token)
if user = Invidious::Database::Users.select(email: email)
if CONFIG.auth_enforce_source && user.password != ("oauth:" + provider_k)
return error_template(401, "Wrong provider")
else
user_flow_existing(env, email)
end
else
user_flow_new(env, email, nil, "oauth:" + provider_k)
end
end
rescue ex
return error_template(500, "Internal Error")
end
env.redirect referer
end
def self.login(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
if !CONFIG.login_enabled
@ -41,9 +75,22 @@ module Invidious::Routes::Login
password = env.params.body["password"]?
account_type = env.params.query["type"]?
account_type ||= "invidious"
account_type ||= ""
if CONFIG.auth_type.size == 0
return error_template(401, "No authentication backend enabled.")
elsif CONFIG.auth_type.find(&.== account_type).nil? && CONFIG.auth_type.size == 1
account_type = CONFIG.auth_type[0]
end
case account_type
when "oauth"
provider_k = env.params.body["provider"]
env.redirect OAuthHelper.make_client(provider_k).get_authorize_uri("openid email profile")
when "saml"
return error_template(501, "Not implemented")
when "ldap"
return error_template(501, "Not implemented")
when "invidious"
if email.nil? || email.empty?
return error_template(401, "User ID is a required field")
@ -53,24 +100,14 @@ module Invidious::Routes::Login
return error_template(401, "Password is a required field")
end
user = Invidious::Database::Users.select(email: email)
if user
if Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
Invidious::Database::SessionIDs.insert(sid, email)
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if user = Invidious::Database::Users.select(email: email)
if user.password.not_nil!.starts_with? "oauth"
return error_template(401, "Wrong provider")
elsif Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
user_flow_existing(env, email)
else
return error_template(401, "Wrong username or password")
end
# Since this user has already registered, we don't want to overwrite their preferences
if env.request.cookies["PREFS"]?
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
else
if !CONFIG.registration_enabled
return error_template(400, "Registration has been disabled by administrator.")
@ -147,32 +184,7 @@ module Invidious::Routes::Login
end
end
end
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
user, sid = create_user(sid, email, password)
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
user.preferences.locale = language.header
end
end
Invidious::Database::Users.insert(user)
Invidious::Database::SessionIDs.insert(sid, email)
view_name = "subscriptions_#{sha256(user.email)}"
PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS #{MATERIALIZED_VIEW_SQL.call(user.email)}")
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if env.request.cookies["PREFS"]?
user.preferences = env.get("preferences").as(Preferences)
Invidious::Database::Users.update_preferences(user)
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
user_flow_new(env, email, password, "internal")
end
env.redirect referer
@ -211,4 +223,49 @@ module Invidious::Routes::Login
env.redirect referer
end
def self.user_flow_existing(env, email)
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
Invidious::Database::SessionIDs.insert(sid, email)
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
# Since this user has already registered, we don't want to overwrite their preferences
if env.request.cookies["PREFS"]?
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
end
def self.user_flow_new(env, email, password, provider)
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
if provider == "internal"
user, sid = create_internal_user(sid, email, password)
else
user, sid = create_user(sid, email, provider)
end
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
user.preferences.locale = language.header
end
end
Invidious::Database::Users.insert(user)
Invidious::Database::SessionIDs.insert(sid, email)
view_name = "subscriptions_#{sha256(user.email)}"
PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS #{MATERIALIZED_VIEW_SQL.call(user.email)}")
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if env.request.cookies["PREFS"]?
user.preferences = env.get("preferences").as(Preferences)
Invidious::Database::Users.update_preferences(user)
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
end
end

View file

@ -51,6 +51,12 @@ module Invidious::Routes::Search
else
user = env.get? "user"
# An URL was copy/pasted in the search box.
# Redirect the user to the appropriate page.
if query.is_url?
return env.redirect UrlSanitizer.process(query.text).to_s
end
begin
items = query.process
rescue ex : ChannelSearchException

View file

@ -55,6 +55,7 @@ module Invidious::Routing
def register_user_routes
# User login/out
get "/login", Routes::Login, :login_page
get "/login/oauth/:provider", Routes::Login, :login_oauth
post "/login", Routes::Login, :login
post "/signout", Routes::Login, :signout
@ -236,6 +237,7 @@ module Invidious::Routing
get "/api/v1/annotations/:id", {{namespace}}::Videos, :annotations
get "/api/v1/comments/:id", {{namespace}}::Videos, :comments
get "/api/v1/clips/:id", {{namespace}}::Videos, :clips
get "/api/v1/chapters/:id", {{namespace}}::Videos, :chapters
# Feeds
get "/api/v1/trending", {{namespace}}::Feeds, :trending

View file

@ -20,6 +20,9 @@ module Invidious::Search
property region : String?
property channel : String = ""
# Flag that indicates if the smart search features have been disabled.
@inhibit_ssf : Bool = false
# Return true if @raw_query is either `nil` or empty
private def empty_raw_query?
return @raw_query.empty?
@ -48,10 +51,18 @@ module Invidious::Search
)
# Get the raw search query string (common to all search types). In
# Regular search mode, also look for the `search_query` URL parameter
if @type.regular?
@raw_query = params["q"]? || params["search_query"]? || ""
else
@raw_query = params["q"]? || ""
_raw_query = params["q"]?
_raw_query ||= params["search_query"]? if @type.regular?
_raw_query ||= ""
# Remove surrounding whitespaces. Mostly useful for copy/pasted URLs.
@raw_query = _raw_query.strip
# Check for smart features (ex: URL search) inhibitor (backslash).
# If inhibitor is present, remove it.
if @raw_query.starts_with?('\\')
@inhibit_ssf = true
@raw_query = @raw_query[1..]
end
# Get the page number (also common to all search types)
@ -85,7 +96,7 @@ module Invidious::Search
@filters = Filters.from_iv_params(params)
@channel = params["channel"]? || ""
if @filters.default? && @raw_query.includes?(':')
if @filters.default? && @raw_query.index(/\w:\w/)
# Parse legacy filters from query
@filters, @channel, @query, subs = Filters.from_legacy_filters(@raw_query)
else
@ -136,5 +147,22 @@ module Invidious::Search
return params
end
# Checks if the query is a standalone URL
def is_url? : Bool
# If the smart features have been inhibited, don't go further.
return false if @inhibit_ssf
# Only supported in regular search mode
return false if !@type.regular?
# If filters are present, that's a regular search
return false if !@filters.default?
# Simple heuristics: domain name
return @raw_query.starts_with?(
/(https?:\/\/)?(www\.)?(m\.)?youtu(\.be|be\.com)\//
)
end
end
end

View file

@ -14,6 +14,7 @@ struct Invidious::User
return HTTP::Cookie.new(
name: "SID",
domain: domain,
path: "/",
value: sid,
expires: Time.utc + 2.years,
secure: SECURE,
@ -28,6 +29,7 @@ struct Invidious::User
return HTTP::Cookie.new(
name: "PREFS",
domain: domain,
path: "/",
value: URI.encode_www_form(preferences.to_json),
expires: Time.utc + 2.years,
secure: SECURE,

View file

@ -4,7 +4,6 @@ require "crypto/bcrypt/password"
MATERIALIZED_VIEW_SQL = ->(email : String) { "SELECT cv.* FROM channel_videos cv WHERE EXISTS (SELECT subscriptions FROM users u WHERE cv.ucid = ANY (u.subscriptions) AND u.email = E'#{email.gsub({'\'' => "\\'", '\\' => "\\\\"})}') ORDER BY published DESC" }
def create_user(sid, email, password)
password = Crypto::Bcrypt::Password.create(password, cost: 10)
token = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
user = Invidious::User.new({
@ -13,7 +12,7 @@ def create_user(sid, email, password)
subscriptions: [] of String,
email: email,
preferences: Preferences.new(CONFIG.default_user_preferences.to_tuple),
password: password.to_s,
password: password,
token: token,
watched: [] of String,
feed_needs_update: true,
@ -22,6 +21,11 @@ def create_user(sid, email, password)
return user, sid
end
def create_internal_user(sid, email, password)
password = Crypto::Bcrypt::Password.create(password.not_nil!, cost: 10)
create_user(sid, email, password.to_s)
end
def get_subscription_feed(user, max_results = 40, page = 1)
limit = max_results.clamp(0, MAX_ITEMS_PER_PAGE)
offset = (page - 1) * limit

View file

@ -15,7 +15,7 @@ struct Video
# NOTE: don't forget to bump this number if any change is made to
# the `params` structure in videos/parser.cr!!!
#
SCHEMA_VERSION = 2
SCHEMA_VERSION = 3
property id : String
@ -26,6 +26,9 @@ struct Video
@[DB::Field(ignore: true)]
@captions = [] of Invidious::Videos::Captions::Metadata
@[DB::Field(ignore: true)]
@chapters : Invidious::Videos::Chapters? = nil
@[DB::Field(ignore: true)]
property adaptive_fmts : Array(Hash(String, JSON::Any))?
@ -254,6 +257,24 @@ struct Video
return @captions
end
def chapters
# As the chapters key is always present in @info we need to check that it is
# actually populated
if @chapters.nil?
chapters = @info["chapters"].as_a
return nil if chapters.empty?
@chapters = Invidious::Videos::Chapters.from_raw_chapters(
chapters,
self.length_seconds,
# Should never be nil but just in case
is_auto_generated: @info["autoGeneratedChapters"].as_bool? || false
)
end
return @chapters
end
def hls_manifest_url : String?
info.dig?("streamingData", "hlsManifestUrl").try &.as_s
end

View file

@ -0,0 +1,108 @@
module Invidious::Videos
# A `Chapters` struct represents an sequence of chapters for a given video
struct Chapters
record Chapter, start_ms : Time::Span, end_ms : Time::Span, title : String, thumbnails : Array(Hash(String, Int32 | String))
property? auto_generated : Bool
def initialize(@chapters : Array(Chapter), @auto_generated : Bool)
end
# Constructs a chapters object from InnerTube's JSON object for chapters
#
# Requires the length of the video the chapters are associated to in order to construct correct ending time
def Chapters.from_raw_chapters(raw_chapters : Array(JSON::Any), video_length : Int32, is_auto_generated : Bool = false)
video_length_milliseconds = video_length.seconds.total_milliseconds
parsed_chapters = [] of Chapter
raw_chapters.each_with_index do |chapter, index|
chapter = chapter["chapterRenderer"]
title = chapter["title"]["simpleText"].as_s
raw_thumbnails = chapter["thumbnail"]["thumbnails"].as_a
thumbnails = raw_thumbnails.map do |thumbnail|
{
"url" => thumbnail["url"].as_s,
"width" => thumbnail["width"].as_i,
"height" => thumbnail["height"].as_i,
}
end
start_ms = chapter["timeRangeStartMillis"].as_i
# To get the ending range we have to peek at the next chapter.
# If we're the last chapter then we need to calculate the end time through the video length.
if next_chapter = raw_chapters[index + 1]?
end_ms = next_chapter["chapterRenderer"]["timeRangeStartMillis"].as_i
else
end_ms = video_length_milliseconds.to_i
end
parsed_chapters << Chapter.new(
start_ms: start_ms.milliseconds,
end_ms: end_ms.milliseconds,
title: title,
thumbnails: thumbnails,
)
end
return Chapters.new(parsed_chapters, is_auto_generated)
end
# Calls the given block for each chapter and passes it as a parameter
def each(&)
@chapters.each { |c| yield c }
end
# Converts the sequence of chapters to a WebVTT representation
def to_vtt
vtt = WebVTT.build do |build|
self.each do |chapter|
build.cue(chapter.start_ms, chapter.end_ms, chapter.title)
end
end
end
# Dumps a JSON representation of the sequence of chapters to the given JSON::Builder
def to_json(json : JSON::Builder)
json.field "autoGenerated", @auto_generated.to_s
json.field "chapters" do
json.array do
@chapters.each do |chapter|
json.object do
json.field "title", chapter.title
json.field "startMs", chapter.start_ms.total_milliseconds
json.field "endMs", chapter.end_ms.total_milliseconds
json.field "thumbnails" do
json.array do
chapter.thumbnails.each do |thumbnail|
json.object do
json.field "url", URI.parse(thumbnail["url"].as(String)).request_target
json.field "width", thumbnail["width"]
json.field "height", thumbnail["height"]
end
end
end
end
end
end
end
end
end
# Create a JSON representation of the sequence of chapters
def to_json
JSON.build do |json|
json.object do
json.field "chapters" do
json.object do
to_json(json)
end
end
end
end
end
end
end

View file

@ -248,11 +248,12 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
end
end
player_overlays = player_response.dig?("playerOverlays", "playerOverlayRenderer")
# If nothing was found previously, fall back to end screen renderer
if related.empty?
# Container for "endScreenVideoRenderer" items
player_overlays = player_response.dig?(
"playerOverlays", "playerOverlayRenderer",
end_screen_watch_next_array = player_overlays.try &.dig?(
"endScreen", "watchNextEndScreenRenderer", "results"
)
@ -394,6 +395,32 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
.try &.as_s.split(" ", 2)[0]
end
# Chapters
chapters_array = [] of JSON::Any
chapters_auto_generated = nil
# Yes,`decoratedPlayerBarRenderer` is repeated twice.
if player_bar = player_overlays.try &.dig?("decoratedPlayerBarRenderer", "decoratedPlayerBarRenderer", "playerBar")
if markers = player_bar.dig?("multiMarkersPlayerBarRenderer", "markersMap")
potential_chapters_array = markers.as_a.find(&.["key"]?.try &.== "DESCRIPTION_CHAPTERS")
# Chapters that are manually created should have a higher precedence than automatically generated chapters
if !potential_chapters_array
potential_chapters_array = markers.as_a.find(&.["key"]?.try &.== "AUTO_CHAPTERS")
end
if potential_chapters_array
if potential_chapters_array["key"] == "AUTO_CHAPTERS"
chapters_auto_generated = true
else
chapters_auto_generated = false
end
chapters_array = potential_chapters_array["value"]["chapters"].as_a
end
end
end
# Return data
if live_now
@ -414,13 +441,15 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
"lengthSeconds" => JSON::Any.new(length_txt || 0_i64),
"published" => JSON::Any.new(published.to_rfc3339),
# Extra video infos
"allowedRegions" => JSON::Any.new(allowed_regions.map { |v| JSON::Any.new(v) }),
"allowRatings" => JSON::Any.new(allow_ratings || false),
"isFamilyFriendly" => JSON::Any.new(family_friendly || false),
"isListed" => JSON::Any.new(is_listed || false),
"isUpcoming" => JSON::Any.new(is_upcoming || false),
"keywords" => JSON::Any.new(keywords.map { |v| JSON::Any.new(v) }),
"isPostLiveDvr" => JSON::Any.new(post_live_dvr),
"allowedRegions" => JSON::Any.new(allowed_regions.map { |v| JSON::Any.new(v) }),
"allowRatings" => JSON::Any.new(allow_ratings || false),
"isFamilyFriendly" => JSON::Any.new(family_friendly || false),
"isListed" => JSON::Any.new(is_listed || false),
"isUpcoming" => JSON::Any.new(is_upcoming || false),
"keywords" => JSON::Any.new(keywords.map { |v| JSON::Any.new(v) }),
"isPostLiveDvr" => JSON::Any.new(post_live_dvr),
"autoGeneratedChapters" => JSON::Any.new(chapters_auto_generated),
"chapters" => JSON::Any.new(chapters_array),
# Related videos
"relatedVideos" => JSON::Any.new(related),
# Description

View file

@ -0,0 +1,34 @@
<% if chapters = video.chapters %>
<div class="description-chapters-section">
<hr class="description-content-separator"/>
<h4><%=HTML.escape(translate(locale, "video_chapters_label"))%></h4>
<% if chapters.auto_generated? %>
<h5><%=HTML.escape(translate(locale, "video_chapters_auto_generated_label"))%> </h5>
<% end %>
<div class="description-chapters-content-container">
<% chapters.each do | chapter | %>
<%- start_in_seconds = chapter.start_ms.total_seconds.to_i %>
<a href="/watch?v=<%-= video.id %>&t=<%=start_in_seconds %>" data-jump-time="<%=start_in_seconds%>" class="chapter-widget-buttons">
<div class="chapter">
<div class="thumbnail">
<%- if !env.get("preferences").as(Preferences).thin_mode -%>
<img loading="lazy" class="thumbnail" src="<%-=URI.parse(chapter.thumbnails[-1]["url"].to_s).request_target %>" alt="<%=chapter.title%>"/>
<%- else -%>
<div class="thumbnail-placeholder"></div>
<%- end -%>
</div>
<%- if start_in_seconds > 0 -%>
<p><%-= recode_length_seconds(start_in_seconds) -%></p>
<%- else -%>
<p>0:00</p>
<%- end -%>
<p><%-=chapter.title-%></p>
</div>
</a>
<% end %>
</div>
<hr class="description-content-separator"/>
</div>
<% end %>

View file

@ -63,6 +63,10 @@
<% captions.each do |caption| %>
<track kind="captions" src="/api/v1/captions/<%= video.id %>?label=<%= caption.name %>" label="<%= caption.name %>">
<% end %>
<% if !video.chapters.nil? %>
<track kind="chapters" src="/api/v1/chapters/<%= video.id %>">
<% end %>
<% end %>
</video>

View file

@ -7,7 +7,18 @@
<div class="pure-u-1 pure-u-lg-3-5">
<div class="h-box">
<% case account_type when %>
<% else # "invidious" %>
<% when "oauth" %>
<form class="pure-form pure-form-stacked" action="/login?referer=<%= URI.encode_www_form(referer) %>&type=oauth" method="post">
<fieldset>
<select name="provider" id="provider">
<% CONFIG.oauth.each_key do |key| %>
<option value="<%= key %>"><%= key %></option>
<% end %>
</select>
<button type="submit" class="pure-button pure-button-primary"><%= translate(locale, "Sign In via OAuth") %></button>
</fieldset>
</form>
<% when "invidious" %>
<form class="pure-form pure-form-stacked" action="/login?referer=<%= URI.encode_www_form(referer) %>&type=invidious" method="post">
<fieldset>
<% if email %>
@ -70,6 +81,14 @@
<% end %>
</fieldset>
</form>
<% else %>
<% if CONFIG.auth_internal_enabled? %>
<a class="pure-button pure-button-secondary" href="/login?referer=<%= URI.encode_www_form(referer) %>&type=invidious">Internal</a>
<% end %>
<% if CONFIG.auth_oauth_enabled? %>
<a class="pure-button pure-button-secondary" href="/login?referer=<%= URI.encode_www_form(referer) %>&type=oauth">OAuth</a>
<% end %>
<label></label>
<% end %>
</div>
</div>

View file

@ -254,10 +254,14 @@ we're going to need to do it here in order to allow for translations.
<div id="description-box"> <!-- Description -->
<% if video.description.size < 200 || params.extend_desc %>
<div id="descriptionWrapper"><%= video.description_html %></div>
<div id="descriptionWrapper"><%-= video.description_html %>
<%= rendered "components/description_chapters_widget" %>
</div>
<% else %>
<input id="descexpansionbutton" type="checkbox"/>
<div id="descriptionWrapper"><%= video.description_html %></div>
<div id="descriptionWrapper"><%-= video.description_html %>
<%= rendered "components/description_chapters_widget" %>
</div>
<label for="descexpansionbutton">
<a></a>
</label>

View file

@ -0,0 +1,121 @@
require "uri"
module UrlSanitizer
extend self
ALLOWED_QUERY_PARAMS = {
channel: ["u", "user", "lb"],
playlist: ["list"],
search: ["q", "search_query", "sp"],
watch: [
"v", # Video ID
"list", "index", # Playlist-related
"playlist", # Unnamed playlist (id,id,id,...) (embed-only?)
"t", "time_continue", "start", "end", # Timestamp
"lc", # Highlighted comment (watch page only)
],
}
# Returns whether the given string is an ASCII word. This is the same as
# running the following regex in US-ASCII locale: /^[\w-]+$/
private def ascii_word?(str : String) : Bool
return false if str.bytesize != str.size
str.each_byte do |byte|
next if 'a'.ord <= byte <= 'z'.ord
next if 'A'.ord <= byte <= 'Z'.ord
next if '0'.ord <= byte <= '9'.ord
next if byte == '-'.ord || byte == '_'.ord
return false
end
return true
end
# Return which kind of parameters are allowed based on the
# first path component (breadcrumb 0).
private def determine_allowed(path_root : String)
case path_root
when "watch", "w", "v", "embed", "e", "shorts", "clip"
return :watch
when .starts_with?("@"), "c", "channel", "user", "profile", "attribution_link"
return :channel
when "playlist", "mix"
return :playlist
when "results", "search"
return :search
else # hashtag, post, trending, brand URLs, etc..
return nil
end
end
# Create a new URI::Param containing only the allowed parameters
private def copy_params(unsafe_params : URI::Params, allowed_type) : URI::Params
new_params = URI::Params.new
ALLOWED_QUERY_PARAMS[allowed_type].each do |name|
if unsafe_params[name]?
# Only copy the last parameter, in case there is more than one
new_params[name] = unsafe_params.fetch_all(name)[-1]
end
end
return new_params
end
# Transform any user-supplied youtube URL into something we can trust
# and use across the code.
def process(str : String) : URI
# Because URI follows RFC3986 specifications, URL without a scheme
# will be parsed as a relative path. So we have to add a scheme ourselves.
str = "https://#{str}" if !str.starts_with?(/https?:\/\//)
unsafe_uri = URI.parse(str)
unsafe_host = unsafe_uri.host
unsafe_path = unsafe_uri.path
new_uri = URI.new(path: "/")
# Redirect to homepage for bogus URLs
return new_uri if (unsafe_host.nil? || unsafe_path.nil?)
breadcrumbs = unsafe_path
.split('/', remove_empty: true)
.compact_map do |bc|
# Exclude attempts at path trasversal
next if bc == "." || bc == ".."
# Non-alnum characters are unlikely in a genuine URL
next if !ascii_word?(bc)
bc
end
# If nothing remains, it's either a legit URL to the homepage
# (who does that!?) or because we filtered some junk earlier.
return new_uri if breadcrumbs.empty?
# Replace the original query parameters with the sanitized ones
case unsafe_host
when .ends_with?("youtube.com")
# Use our sanitized path (not forgetting the leading '/')
new_uri.path = "/#{breadcrumbs.join('/')}"
# Then determine which params are allowed, and copy them over
if allowed = determine_allowed(breadcrumbs[0])
new_uri.query_params = copy_params(unsafe_uri.query_params, allowed)
end
when "youtu.be"
# Always redirect to the watch page
new_uri.path = "/watch"
new_params = copy_params(unsafe_uri.query_params, :watch)
new_params["id"] = breadcrumbs[0]
new_uri.query_params = new_params
end
return new_uri
end
end