Compare commits

...

73 commits

Author SHA1 Message Date
soraefir
bd028430aa
Implemented requested changes and added 'auth_enforce_source' option 2024-08-27 13:07:48 -07:00
soraefir
6afab5a386
Added login flow functions and fix cookies 2024-08-27 13:07:48 -07:00
soraefir
f9ec5ba246
Implemented Oauth 2024-08-27 13:07:48 -07:00
syeopite
c885147306
Lint 2024-08-27 13:07:47 -07:00
syeopite
266d4cc998
Don't use generic click handler for chapter widget 2024-08-27 13:07:47 -07:00
syeopite
dc7811f9bc
Code quality fixes 2024-08-27 13:07:47 -07:00
syeopite
f258a9c7c2
Refactor: Add object to represent chapters
Prior to this commit we used an Array of Chapter structs to represent
a video's chapters. However, as we often needed to apply operations on
the entire sequence of chapters, multiple isolated functions had to be
created and in turn clogged up the code.

By grouping everything together under a chapters struct that stores a
sequence of chapters, these functions can be grouped together, and can
be simplifed due to instance variables containing the data that they need.

Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2024-08-27 13:07:46 -07:00
syeopite
c4424fc07f
Use Time::Span for timestamps in chapter struct 2024-08-27 13:07:46 -07:00
syeopite
0bc058400d
Use WebVTT.build for chapters vtt file 2024-08-27 13:07:46 -07:00
syeopite
e638469352
Use proxy url for chapter thumbnails in JSON API 2024-08-27 13:07:45 -07:00
syeopite
d0de17be50
Add data for chapters to JSON endpoint for videos 2024-08-27 13:07:45 -07:00
syeopite
43c0b4e85b
Add separate method for constructing chapters json 2024-08-27 13:07:45 -07:00
syeopite
27b44a2890
Move parsed chapters info to "extra video infos" 2024-08-27 13:07:44 -07:00
syeopite
02282eb188
Use short-hand block notation for parsing chapters 2024-08-27 13:07:44 -07:00
syeopite
4a8d77c157
Properly camelcase auto gen chapters attribute 2024-08-27 13:07:44 -07:00
syeopite
09b5972924
Escape localization for desc chapters widget 2024-08-27 13:07:43 -07:00
syeopite
cebfdf8243
Change order of chapters button within the player 2024-08-27 13:07:43 -07:00
syeopite
a2f6735734
Add field for whether or not chapter is auto gen 2024-08-27 13:07:43 -07:00
syeopite
fdb122bb67
Remove extraneous space between desc and date 2024-08-27 13:07:42 -07:00
syeopite
fbb4c9193e
Localize chapters label 2024-08-27 13:07:42 -07:00
syeopite
6ad2661e0c
Add message for when chapters are auto generated 2024-08-27 13:07:42 -07:00
syeopite
85407cdd7d
Add data-jump-time attribute to chapters component
Allows automatically jumping to specified time instead of reloading the
page.
2024-08-27 13:07:41 -07:00
syeopite
bcc9fdaaea
Fix missing timestamp on first chapter 2024-08-27 13:07:41 -07:00
syeopite
da861b539d
Fix description chapter component design 2024-08-27 13:07:41 -07:00
syeopite
f74f8f778f
Remove whitespace in chapters component 2024-08-27 13:07:40 -07:00
syeopite
2aff77f361
Fetch chapter thumbnails for selector in desc 2024-08-27 13:07:40 -07:00
syeopite
5bebacb263
Remove initial whitespace from video description 2024-08-27 13:07:40 -07:00
syeopite
72383f8b73
Add initial html for chapters selector in desc 2024-08-27 13:07:39 -07:00
syeopite
3f5d8776d5
Add chapters track to player.ecr 2024-08-27 13:07:39 -07:00
syeopite
eb1c121f77
Add method to convert chapters to vtt 2024-08-27 13:07:39 -07:00
syeopite
f114f9c3a0
Add chapters data to API 2024-08-27 13:07:39 -07:00
syeopite
0bd89cead0
Add logic to parse video chapters 2024-08-27 13:07:33 -07:00
Samantaz Fox
4782a67038
Release v2.20240825.2 2024-08-26 22:52:50 +02:00
Samantaz Fox
5baaedfa39
CI: Fix docker container tags (#4883)
Closes issue 4880
2024-08-26 22:48:14 +02:00
Samantaz Fox
4f066e880c
CI: Fix docker container tags 2024-08-26 21:55:43 +02:00
Samantaz Fox
3e17d04875
Release v2.20240825.1 2024-08-25 22:30:46 +02:00
syeopite
cec905e95e
Allow manual trigger of release-container build (#4877) 2024-08-25 19:55:52 +00:00
Samantaz Fox
80958aa0d8
Release v2.20240825 2024-08-25 21:25:48 +02:00
Samantaz Fox
c5fdd9ea65
HTML: Sort playlists alphabetically in watch page drop down (#4853)
Closes issue 4708
2024-08-24 20:50:46 +02:00
Samantaz Fox
2876ee0f9f
HTML: Fix XSS vulnerability in description/comments (#4852)
Before this PR, the comment/description content was not HTML escaped when 'parse_description()'
was called with a JSON object lacking the "commandRuns" entry.

Closes issue 4727
2024-08-24 20:50:05 +02:00
Samantaz Fox
0699e5fc27
YtAPI: Bump client versions (#4849)
This might help reducing the amount of playback errors.

No related issue
2024-08-24 20:47:01 +02:00
Samantaz Fox
15669acccf
SigHelper: Fix inverted time comparison in 'check_update' (#4845)
Closes issue 4840
2024-08-24 20:44:52 +02:00
Samantaz Fox
cd2daf4adb
Storyboards: Various fixes and code cleaning (#4153)
Closes issue 3441
2024-08-24 20:43:05 +02:00
syeopite
ccecc6d318
Fix lint errors introduced in #4146 and #4295 (#4876)
* Ameba: Fix Naming/VariableNames

Introduced in #4295

* Ameba: Fix Naming/PredicateName

Introduced in #4146
2024-08-24 18:11:11 +00:00
Samantaz Fox
3c6a662aaf
Search: Add support for Youtube URLs (#4146)
Closes issue 3300
2024-08-24 19:44:59 +02:00
Samantaz Fox
9e55799269
Channel: Render age restricted channels (#4295)
This PR:
 * gets thumbnail and channel name from the initial request
 * gets videos, shorts and streams via autogenerated channel playlists

Test Url: /channel/UCbfnHqxXs_K3kvaH-WlNlig

Closes issue 3513
2024-08-24 19:43:59 +02:00
Samantaz Fox
da70c9b7b0
Ameba: Miscellaneous fixes (#4807)
End of a series of PRs meant to improve code quality.

Related to issue 2231
2024-08-24 19:42:10 +02:00
Samantaz Fox
828da3c6ce
API: Proxy formatStreams URLs too (#4859)
The /api/v1/videos endpoint does not proxy the formatStreams URLs when
'local=true' is passed, whereas the adaptiveFormats URLs are correctly proxied.

The Web UI does proxy when clicking "Download" with 'fmt=18' for example, so
this is probably an oversight. This PR aims to fix that.

No related issue
2024-08-24 19:39:36 +02:00
Samantaz Fox
febf18cbf7
UI: Add search button to search bar (#4706)
Closes issue 529
2024-08-24 19:38:48 +02:00
Samantaz Fox
21ab5dc668
Storyboard: Revert cue timing "fix" 2024-08-22 00:29:15 +02:00
Samantaz Fox
b200ebfb6b
CSS: Remove extra space in default.css 2024-08-21 20:23:45 +00:00
syeopite
ecbea0b67b
Ameba: Fix Lint/ShadowingOuterLocalVar 2024-08-21 02:43:26 -07:00
syeopite
d1cd790388
Ameba: Fix Lint/RedundantStringCoercion 2024-08-21 02:43:26 -07:00
syeopite
f66068976e
Ameba: Fix Naming/PredicateName 2024-08-21 02:43:08 -07:00
syeopite
22b35c453e
Ameba: Fix Style/WhileTrue 2024-08-21 02:43:08 -07:00
Colin Leroy-Mira
c606465708 Proxify formatStreams URLs too 2024-08-19 09:37:24 +02:00
Samantaz Fox
764965c441
Storyboards: Fix lint error 2024-08-17 12:20:53 +02:00
Samantaz Fox
b795bdf2a4
HTML: Sort playlists alphabetically in watch page drop down 2024-08-16 12:10:22 +02:00
Samantaz Fox
5b05f3bd14
Storyboards: Workarounds for videojs-vtt-thumbnails
The workarounds are as follow:
  * Unescape HTML entities
  * Always use 0:00:00.000 for cue start/end
2024-08-16 11:36:01 +02:00
Samantaz Fox
a335bc0814
Storyboards: Fix some small logic mistakes 2024-08-16 10:05:49 +02:00
Samantaz Fox
7b50388eaf
Storyboards: Fix broken first storyboard 2024-08-16 10:05:48 +02:00
Samantaz Fox
da3d58f03c
Storyboards: Cleanup and document code 2024-08-16 10:05:47 +02:00
Samantaz Fox
8327862697
Storyboards: Use replace the NamedTuple by a struct 2024-08-16 10:04:40 +02:00
Samantaz Fox
6878822c4d
Storyboards: Move parser to its own file 2024-08-16 10:02:52 +02:00
Samantaz Fox
0b28054f8a
videos: Fix XSS vulnerability in description/comments
Patch provided by e-mail, thanks to an anonymous user whose cats are named
Yoshi and Yasuo.

Comment is mine
2024-08-15 18:26:17 +02:00
Samantaz Fox
cc33d3f074
YtAPI: Also update User-Agent string 2024-08-15 18:14:29 +02:00
Samantaz Fox
acbb625866
YtAPI: Update clients to latest version 2024-08-15 12:57:36 +02:00
Samantaz Fox
466bfbb306
SigHelper: Fix inverted time comparison in 'check_update' 2024-08-14 21:43:37 +02:00
ChunkyProgrammer
e31053e812 Use dig to get properties
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2024-08-13 15:09:16 -04:00
ChunkyProgrammer
96ade642fa Channel: Render age restricted channels 2024-08-13 15:09:16 -04:00
thansk
1ce2d10c50
fix: use ion icon for search icon 2024-05-20 14:17:30 +00:00
thansk
5abafb8296
fix: use a search icon instead of text 2024-05-20 11:49:56 +00:00
thansk
9cd2e93a2e
feat: allow submitting search with mouse 2024-05-19 11:46:55 +00:00
41 changed files with 1214 additions and 317 deletions

View file

@ -1,6 +1,7 @@
name: Build and release container
on:
workflow_dispatch:
push:
tags:
- "v*"
@ -46,9 +47,11 @@ jobs:
uses: docker/metadata-action@v5
with:
images: quay.io/invidious/invidious
flavor: |
latest=false
tags: |
type=semver,pattern={{version}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=raw,value=latest
labels: |
quay.expires-after=12w
@ -70,10 +73,11 @@ jobs:
with:
images: quay.io/invidious/invidious
flavor: |
latest=false
suffix=-arm64
tags: |
type=semver,pattern={{version}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=raw,value=latest
labels: |
quay.expires-after=12w

View file

@ -1,6 +1,189 @@
# CHANGELOG
## 2024-04-26
## v2.20240825.2 (2024-08-26)
This releases fixes the container tags pushed on quay.io.
Previously, the ARM64 build was released under the `latest` tag, instead of `latest-arm64`.
### Full list of pull requests merged since the last release (newest first)
CI: Fix docker container tags ([#4883], by @SamantazFox)
[#4877]: https://github.com/iv-org/invidious/pull/4877
## v2.20240825.1 (2024-08-25)
Add patch component to be [semver] compliant and make github actions happy.
[semver]: https://semver.org/
### Full list of pull requests merged since the last release (newest first)
Allow manual trigger of release-container build ([#4877], thanks @syeopite)
[#4877]: https://github.com/iv-org/invidious/pull/4877
## v2.20240825.0 (2024-08-25)
### New features & important changes
#### For users
* The search bar now has a button that you can click!
* Youtube URLs can be pasted directly in the search bar. Prepend search query with a
backslash (`\`) to disable that feature (useful if you need to search for a video whose
title contains some youtube URL).
* On the channel page the "streams" tab can be sorted by either: "newest", "oldest" or "popular"
* Lots of translations have been updated (thanks to our contributors on Weblate!)
* Videos embedded in local HTML files (e.g: a webpage saved from a blog) can now be played
#### For instance owners
* Invidious now has the ability to provide a `po_token` and `visitordata` to Youtube in order to
circumvent current Youtube restrictions.
* Invidious can use an (optional) external signature server like [inv_sig_helper]. Please note that
some videos can't be played without that signature server.
* The Helm charts were moved to a separate repo: https://github.com/iv-org/invidious-helm-chart
* We have changed how containers are released: the `latest` tag now tracks tagged releases, whereas
the `master` tag tracks the most recent commits of the `master` branch ("nightly" builds).
[inv_sig_helper]: https://github.com/iv-org/inv_sig_helper
#### For developpers
* The versions of Crystal that we test in CI/CD are now: `1.9.2`, `1.10.1`, `1.11.2`, `1.12.1`.
Please note that due to a bug in the `libxml` bindings (See [#4256]), versions prior to `1.10.0`
are not recommended to use.
* Thanks to @syeopite, the code is now [ameba] compliant.
* Ameba is part of our CI/CD pipeline, and its rules will be enforced in future PRs.
* The transcript code has been rewritten to permit transcripts as a feature rather than being
only a workaround for captions. Trancripts feature is coming soon!
* Various fixes regarding the logic interacting with Youtube
* The `sort_by` parameter can be used on the `/api/v1/channels/{id}/streams` endpoint. Accepted
values are: "newest", "oldest" and "popular"
[ameba]: https://github.com/crystal-ameba/ameba
[#4256]: https://github.com/iv-org/invidious/issues/4256
### Bugs fixed
#### User-side
* Channels: fixed broken "subscribers" and "views" counters
* Watch page: playback position is reset at the end of a video, so that the next time this video
is watched, it will start from the beginning rather than 15 seconds before the end
* Watch page: the items in the "add to playlist" drop down are now sorted alphabetically
* Videos: the "genre" URL is now always pointing to a valid webpage
* Playlists: Fixed `Could not parse N episodes` error on podcast playlists
* All external links should now have the [`rel`] attibute set to `noreferrer noopener` for
increased privacy.
* Preferences: Fixed the admin-only "modified source code" input being ignored
* Watch/channel pages: use the full image URL in `og:image` and `twitter:image` meta tags
[`rel`]: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel
#### API
* fixed the `local` parameter not applying to `formatStreams` on `/api/v1/videos/{id}`
* fixed an `Index out of bounds` error hapenning when a playlist had no videos
* fixed duplicated query parameters in proxied video URLs
* Return actual video height/width/fps rather than hard coded values
* Fixed the `/api/v1/popular` endpoint not returning a proper error code/message when the
popular page/endpoint are disabled.
### Full list of pull requests merged since the last release (newest first)
* HTML: Sort playlists alphabetically in watch page drop down ([#4853], by @SamantazFox)
* Videos: Fix XSS vulnerability in description/comments ([#4852], thanks _anonymous_)
* YtAPI: Bump client versions ([#4849], by @SamantazFox)
* SigHelper: Fix inverted time comparison in 'check_update' ([#4845], by @SamantazFox)
* Storyboards: Various fixes and code cleaning ([#4153], by SamantazFox)
* Fix lint errors introduced in #4146 and #4295 ([#4876], thanks @syeopite)
* Search: Add support for Youtube URLs ([#4146], by @SamantazFox)
* Channel: Render age restricted channels ([#4295], thanks @ChunkyProgrammer)
* Ameba: Miscellaneous fixes ([#4807], thanks @syeopite)
* API: Proxy formatStreams URLs too ([#4859], thanks @colinleroy)
* UI: Add search button to search bar ([#4706], thanks @thansk)
* Add ability to set po_token and visitordata ID ([#4789], thanks @unixfox)
* Add support for an external signature server ([#4772], by @SamantazFox)
* Ameba: Fix Naming/VariableNames ([#4790], thanks @syeopite)
* Translations update from Hosted Weblate ([#4659])
* Ameba: Fix Lint/UselessAssign ([#4795], thanks @syeopite)
* HTML: Add rel="noreferrer noopener" to external links ([#4667], thanks @ulmemxpoc)
* Remove unused methods in Invidious::LogHandler ([#4812], thanks @syeopite)
* Ameba: Fix Lint/NotNilAfterNoBang ([#4796], thanks @syeopite)
* Ameba: Fix unused argument Lint warnings ([#4805], thanks @syeopite)
* Ameba: i18next.cr fixes ([#4806], thanks @syeopite)
* Ameba: Disable rules ([#4792], thanks @syeopite)
* Channel: parse subscriber count and channel banner ([#4785], thanks @ChunkyProgrammer)
* Player: Fix playback position of already watched videos ([#4731], thanks @Fijxu)
* Videos: Fix genre url being unusable ([#4717], thanks @meatball133)
* API: Fix out of bound error on empty playlists ([#4696], thanks @Fijxu)
* Handle playlists cataloged as Podcast ([#4695], thanks @Fijxu)
* API: Fix duplicated query parameters in proxied video URLs ([#4587], thanks @absidue)
* API: Return actual stream height, width and fps ([#4586], thanks @absidue)
* Preferences: Fix handling of modified source code URL ([#4437], thanks @nooptek)
* API: Fix URL for vtt subtitles ([#4221], thanks @karelrooted)
* Channels: Add sort options to streams ([#4224], thanks @src-tinkerer)
* API: Fix error code for disabled popular endpoint ([#4296], thanks @iBicha)
* Allow embedding videos in local HTML files ([#4450], thanks @tomasz1986)
* CI: Bump Crystal version matrix ([#4654], by @SamantazFox)
* YtAPI: Remove API keys like official clients ([#4655], by @SamantazFox)
* HTML: Use full URL in the og:image property ([#4675], thanks @Fijxu)
* Rewrite transcript logic to be more generic ([#4747], thanks @syeopite)
* CI: Run Ameba ([#4753], thanks @syeopite)
* CI: Add release based containers ([#4763], thanks @syeopite)
* move helm chart to a dedicated github repository ([#4711], thanks @unixfox)
[#4146]: https://github.com/iv-org/invidious/pull/4146
[#4153]: https://github.com/iv-org/invidious/pull/4153
[#4221]: https://github.com/iv-org/invidious/pull/4221
[#4224]: https://github.com/iv-org/invidious/pull/4224
[#4295]: https://github.com/iv-org/invidious/pull/4295
[#4296]: https://github.com/iv-org/invidious/pull/4296
[#4437]: https://github.com/iv-org/invidious/pull/4437
[#4450]: https://github.com/iv-org/invidious/pull/4450
[#4586]: https://github.com/iv-org/invidious/pull/4586
[#4587]: https://github.com/iv-org/invidious/pull/4587
[#4654]: https://github.com/iv-org/invidious/pull/4654
[#4655]: https://github.com/iv-org/invidious/pull/4655
[#4659]: https://github.com/iv-org/invidious/pull/4659
[#4667]: https://github.com/iv-org/invidious/pull/4667
[#4675]: https://github.com/iv-org/invidious/pull/4675
[#4695]: https://github.com/iv-org/invidious/pull/4695
[#4696]: https://github.com/iv-org/invidious/pull/4696
[#4706]: https://github.com/iv-org/invidious/pull/4706
[#4711]: https://github.com/iv-org/invidious/pull/4711
[#4717]: https://github.com/iv-org/invidious/pull/4717
[#4731]: https://github.com/iv-org/invidious/pull/4731
[#4747]: https://github.com/iv-org/invidious/pull/4747
[#4753]: https://github.com/iv-org/invidious/pull/4753
[#4763]: https://github.com/iv-org/invidious/pull/4763
[#4772]: https://github.com/iv-org/invidious/pull/4772
[#4785]: https://github.com/iv-org/invidious/pull/4785
[#4789]: https://github.com/iv-org/invidious/pull/4789
[#4790]: https://github.com/iv-org/invidious/pull/4790
[#4792]: https://github.com/iv-org/invidious/pull/4792
[#4795]: https://github.com/iv-org/invidious/pull/4795
[#4796]: https://github.com/iv-org/invidious/pull/4796
[#4805]: https://github.com/iv-org/invidious/pull/4805
[#4806]: https://github.com/iv-org/invidious/pull/4806
[#4807]: https://github.com/iv-org/invidious/pull/4807
[#4812]: https://github.com/iv-org/invidious/pull/4812
[#4845]: https://github.com/iv-org/invidious/pull/4845
[#4849]: https://github.com/iv-org/invidious/pull/4849
[#4852]: https://github.com/iv-org/invidious/pull/4852
[#4853]: https://github.com/iv-org/invidious/pull/4853
[#4859]: https://github.com/iv-org/invidious/pull/4859
[#4876]: https://github.com/iv-org/invidious/pull/4876
## v2.20240427 (2024-04-27)
Major bug fixes:
* Videos: Use android test suite client (#4650, thanks @SamantazFox)

View file

@ -160,7 +160,9 @@ body a.pure-button {
button.pure-button-primary,
body a.pure-button-primary,
.channel-owner:hover,
.channel-owner:focus {
.channel-owner:focus,
.chapter:hover,
.chapter:focus {
background-color: #a0a0a0;
color: rgba(35, 35, 35, 1);
}
@ -278,7 +280,14 @@ div.thumbnail > .bottom-right-overlay {
display: inline;
}
.searchbar .pure-form fieldset { padding: 0; }
.searchbar .pure-form {
display: flex;
}
.searchbar .pure-form fieldset {
padding: 0;
flex: 1;
}
.searchbar input[type="search"] {
width: 100%;
@ -310,6 +319,16 @@ input[type="search"]::-webkit-search-cancel-button {
background-size: 14px;
}
.searchbar #searchbutton {
border: none;
background: none;
margin-top: 0;
}
.searchbar #searchbutton:hover {
color: rgb(0, 182, 240);
}
.user-field {
display: flex;
flex-direction: row;
@ -797,5 +816,26 @@ h1, h2, h3, h4, h5, p,
}
#download_widget {
width: 100%;
width: 100%;
}
.description-chapters-section {
white-space: normal;
}
.description-chapters-content-container {
display: flex;
flex-direction: row;
gap: 5px;
overflow: scroll;
overflow-y: hidden;
}
.chapter {
padding: 3px;
}
.chapter .thumbnail {
width: 200px;
}

View file

@ -114,6 +114,10 @@ ul.vjs-menu-content::-webkit-scrollbar {
order: 5;
}
.vjs-chapters-button {
order: 5;
}
.vjs-share-control {
order: 6;
}

View file

@ -17,6 +17,7 @@ var options = {
'remainingTimeDisplay',
'Spacer',
'captionsButton',
'ChaptersButton',
'audioTrackButton',
'qualitySelector',
'playbackRateMenuButton',

View file

@ -191,3 +191,9 @@ addEventListener('load', function (e) {
comments.innerHTML = '';
}
});
const chapter_widget_buttons = document.getElementsByClassName("chapter-widget-buttons")
Array.from(chapter_widget_buttons).forEach(e => e.addEventListener("click", function (event) {
event.preventDefault();
player.currentTime(e.getAttribute('data-jump-time'));
}))

View file

@ -323,6 +323,40 @@ https_only: false
##
#enable_user_notifications: true
##
## List of Enabled Authentication Backend
## If not provided falls back to default
##
## Supported Values:
## - invidious
## - oauth
## - ldap (Not implemented !)
## - saml (Not implemented !)
##
## Default: ["invidious","oauth"]
##
# auth_type: ["oauth"]
##
## OAuth Configuration
##
## Notes:
## - Supports multiple OAuth backends
## - Requires external_port and domain to be configured
##
## Default: []
##
# oauth:
# example:
# host: oauth.example.net
# field : email
# auth_uri: /oauth/authorize/
# token_uri: /oauth/token/
# info_uri: https://api.example.net/oauth/userinfo/
# client_id: CLIENT_ID
# client_secret: CLIENT_SECRET
# -----------------------------
# Background jobs
# -----------------------------

View file

@ -496,5 +496,7 @@
"toggle_theme": "Toggle Theme",
"carousel_slide": "Slide {{current}} of {{total}}",
"carousel_skip": "Skip the Carousel",
"carousel_go_to": "Go to slide `x`"
"carousel_go_to": "Go to slide `x`",
"video_chapters_label": "Chapters",
"video_chapters_auto_generated_label": "These chapters are auto-generated"
}

View file

@ -15,7 +15,8 @@ record AboutChannel,
allowed_regions : Array(String),
tabs : Array(String),
tags : Array(String),
verified : Bool
verified : Bool,
is_age_gated : Bool
def get_about_info(ucid, locale) : AboutChannel
begin
@ -45,46 +46,102 @@ def get_about_info(ucid, locale) : AboutChannel
end
tags = [] of String
tab_names = [] of String
total_views = 0_i64
joined = Time.unix(0)
if auto_generated
author = initdata["header"]["interactiveTabbedHeaderRenderer"]["title"]["simpleText"].as_s
author_url = initdata["microformat"]["microformatDataRenderer"]["urlCanonical"].as_s
author_thumbnail = initdata["header"]["interactiveTabbedHeaderRenderer"]["boxArt"]["thumbnails"][0]["url"].as_s
# Raises a KeyError on failure.
banners = initdata["header"]["interactiveTabbedHeaderRenderer"]?.try &.["banner"]?.try &.["thumbnails"]?
banner = banners.try &.[-1]?.try &.["url"].as_s?
description_base_node = initdata["header"]["interactiveTabbedHeaderRenderer"]["description"]
# some channels have the description in a simpleText
# ex: https://www.youtube.com/channel/UCQvWX73GQygcwXOTSf_VDVg/
description_node = description_base_node.dig?("simpleText") || description_base_node
tags = initdata.dig?("header", "interactiveTabbedHeaderRenderer", "badges")
.try &.as_a.map(&.["metadataBadgeRenderer"]["label"].as_s) || [] of String
if age_gate_renderer = initdata.dig?("contents", "twoColumnBrowseResultsRenderer", "tabs", 0, "tabRenderer", "content", "sectionListRenderer", "contents", 0, "channelAgeGateRenderer")
description_node = nil
author = age_gate_renderer["channelTitle"].as_s
ucid = initdata.dig("responseContext", "serviceTrackingParams", 0, "params", 0, "value").as_s
author_url = "https://www.youtube.com/channel/#{ucid}"
author_thumbnail = age_gate_renderer.dig("avatar", "thumbnails", 0, "url").as_s
banner = nil
is_family_friendly = false
is_age_gated = true
tab_names = ["videos", "shorts", "streams"]
auto_generated = false
else
author = initdata["metadata"]["channelMetadataRenderer"]["title"].as_s
author_url = initdata["metadata"]["channelMetadataRenderer"]["channelUrl"].as_s
author_thumbnail = initdata["metadata"]["channelMetadataRenderer"]["avatar"]["thumbnails"][0]["url"].as_s
author_verified = has_verified_badge?(initdata.dig?("header", "c4TabbedHeaderRenderer", "badges"))
if auto_generated
author = initdata["header"]["interactiveTabbedHeaderRenderer"]["title"]["simpleText"].as_s
author_url = initdata["microformat"]["microformatDataRenderer"]["urlCanonical"].as_s
author_thumbnail = initdata["header"]["interactiveTabbedHeaderRenderer"]["boxArt"]["thumbnails"][0]["url"].as_s
ucid = initdata["metadata"]["channelMetadataRenderer"]["externalId"].as_s
# Raises a KeyError on failure.
banners = initdata["header"]["interactiveTabbedHeaderRenderer"]?.try &.["banner"]?.try &.["thumbnails"]?
banner = banners.try &.[-1]?.try &.["url"].as_s?
# Raises a KeyError on failure.
banners = initdata["header"]["c4TabbedHeaderRenderer"]?.try &.["banner"]?.try &.["thumbnails"]?
banners ||= initdata.dig?("header", "pageHeaderRenderer", "content", "pageHeaderViewModel", "banner", "imageBannerViewModel", "image", "sources")
banner = banners.try &.[-1]?.try &.["url"].as_s?
description_base_node = initdata["header"]["interactiveTabbedHeaderRenderer"]["description"]
# some channels have the description in a simpleText
# ex: https://www.youtube.com/channel/UCQvWX73GQygcwXOTSf_VDVg/
description_node = description_base_node.dig?("simpleText") || description_base_node
# if banner.includes? "channels/c4/default_banner"
# banner = nil
# end
tags = initdata.dig?("header", "interactiveTabbedHeaderRenderer", "badges")
.try &.as_a.map(&.["metadataBadgeRenderer"]["label"].as_s) || [] of String
else
author = initdata["metadata"]["channelMetadataRenderer"]["title"].as_s
author_url = initdata["metadata"]["channelMetadataRenderer"]["channelUrl"].as_s
author_thumbnail = initdata["metadata"]["channelMetadataRenderer"]["avatar"]["thumbnails"][0]["url"].as_s
author_verified = has_verified_badge?(initdata.dig?("header", "c4TabbedHeaderRenderer", "badges"))
description_node = initdata["metadata"]["channelMetadataRenderer"]?.try &.["description"]?
tags = initdata.dig?("microformat", "microformatDataRenderer", "tags").try &.as_a.map(&.as_s) || [] of String
ucid = initdata["metadata"]["channelMetadataRenderer"]["externalId"].as_s
# Raises a KeyError on failure.
banners = initdata["header"]["c4TabbedHeaderRenderer"]?.try &.["banner"]?.try &.["thumbnails"]?
banners ||= initdata.dig?("header", "pageHeaderRenderer", "content", "pageHeaderViewModel", "banner", "imageBannerViewModel", "image", "sources")
banner = banners.try &.[-1]?.try &.["url"].as_s?
# if banner.includes? "channels/c4/default_banner"
# banner = nil
# end
description_node = initdata["metadata"]["channelMetadataRenderer"]?.try &.["description"]?
tags = initdata.dig?("microformat", "microformatDataRenderer", "tags").try &.as_a.map(&.as_s) || [] of String
end
is_family_friendly = initdata["microformat"]["microformatDataRenderer"]["familySafe"].as_bool
if tabs_json = initdata["contents"]["twoColumnBrowseResultsRenderer"]["tabs"]?
# Get the name of the tabs available on this channel
tab_names = tabs_json.as_a.compact_map do |entry|
name = entry.dig?("tabRenderer", "title").try &.as_s.downcase
# This is a small fix to not add extra code on the HTML side
# I.e, the URL for the "live" tab is .../streams, so use "streams"
# everywhere for the sake of simplicity
(name == "live") ? "streams" : name
end
# Get the currently active tab ("About")
about_tab = extract_selected_tab(tabs_json)
# Try to find the about metadata section
channel_about_meta = about_tab.dig?(
"content",
"sectionListRenderer", "contents", 0,
"itemSectionRenderer", "contents", 0,
"channelAboutFullMetadataRenderer"
)
if !channel_about_meta.nil?
total_views = channel_about_meta.dig?("viewCountText", "simpleText").try &.as_s.gsub(/\D/, "").to_i64? || 0_i64
# The joined text is split to several sub strings. The reduce joins those strings before parsing the date.
joined = extract_text(channel_about_meta["joinedDateText"]?)
.try { |text| Time.parse(text, "Joined %b %-d, %Y", Time::Location.local) } || Time.unix(0)
# Normal Auto-generated channels
# https://support.google.com/youtube/answer/2579942
# For auto-generated channels, channel_about_meta only has
# ["description"]["simpleText"] and ["primaryLinks"][0]["title"]["simpleText"]
auto_generated = (
(channel_about_meta["primaryLinks"]?.try &.size) == 1 && \
extract_text(channel_about_meta.dig?("primaryLinks", 0, "title")) == "Auto-generated by YouTube" ||
channel_about_meta.dig?("links", 0, "channelExternalLinkViewModel", "title", "content").try &.as_s == "Auto-generated by YouTube"
)
end
end
end
is_family_friendly = initdata["microformat"]["microformatDataRenderer"]["familySafe"].as_bool
allowed_regions = initdata
.dig?("microformat", "microformatDataRenderer", "availableCountries")
.try &.as_a.map(&.as_s) || [] of String
@ -102,52 +159,6 @@ def get_about_info(ucid, locale) : AboutChannel
end
end
total_views = 0_i64
joined = Time.unix(0)
tab_names = [] of String
if tabs_json = initdata["contents"]["twoColumnBrowseResultsRenderer"]["tabs"]?
# Get the name of the tabs available on this channel
tab_names = tabs_json.as_a.compact_map do |entry|
name = entry.dig?("tabRenderer", "title").try &.as_s.downcase
# This is a small fix to not add extra code on the HTML side
# I.e, the URL for the "live" tab is .../streams, so use "streams"
# everywhere for the sake of simplicity
(name == "live") ? "streams" : name
end
# Get the currently active tab ("About")
about_tab = extract_selected_tab(tabs_json)
# Try to find the about metadata section
channel_about_meta = about_tab.dig?(
"content",
"sectionListRenderer", "contents", 0,
"itemSectionRenderer", "contents", 0,
"channelAboutFullMetadataRenderer"
)
if !channel_about_meta.nil?
total_views = channel_about_meta.dig?("viewCountText", "simpleText").try &.as_s.gsub(/\D/, "").to_i64? || 0_i64
# The joined text is split to several sub strings. The reduce joins those strings before parsing the date.
joined = extract_text(channel_about_meta["joinedDateText"]?)
.try { |text| Time.parse(text, "Joined %b %-d, %Y", Time::Location.local) } || Time.unix(0)
# Normal Auto-generated channels
# https://support.google.com/youtube/answer/2579942
# For auto-generated channels, channel_about_meta only has
# ["description"]["simpleText"] and ["primaryLinks"][0]["title"]["simpleText"]
auto_generated = (
(channel_about_meta["primaryLinks"]?.try &.size) == 1 && \
extract_text(channel_about_meta.dig?("primaryLinks", 0, "title")) == "Auto-generated by YouTube" ||
channel_about_meta.dig?("links", 0, "channelExternalLinkViewModel", "title", "content").try &.as_s == "Auto-generated by YouTube"
)
end
end
sub_count = 0
if (metadata_rows = initdata.dig?("header", "pageHeaderRenderer", "content", "pageHeaderViewModel", "metadata", "contentMetadataViewModel", "metadataRows").try &.as_a)
@ -177,6 +188,7 @@ def get_about_info(ucid, locale) : AboutChannel
tabs: tab_names,
tags: tags,
verified: author_verified || false,
is_age_gated: is_age_gated || false,
)
end

View file

@ -8,6 +8,18 @@ struct DBConfig
property dbname : String
end
struct OAuthConfig
include YAML::Serializable
property host : String
property field : String = "email"
property auth_uri : String
property token_uri : String
property info_uri : String
property client_id : String
property client_secret : String
end
struct ConfigPreferences
include YAML::Serializable
@ -137,6 +149,10 @@ class Config
# poToken for passing bot attestation
property po_token : String? = nil
property auth_type : Array(String) = ["invidious", "oauth"]
property auth_enforce_source : Bool = true
property oauth = {} of String => OAuthConfig
# Saved cookies in "name1=value1; name2=value2..." format
@[YAML::Field(converter: Preferences::StringToCookies)]
property cookies : HTTP::Cookies = HTTP::Cookies.new
@ -159,6 +175,14 @@ class Config
end
end
def auth_oauth_enabled?
return (@auth_type.find(&.== "oauth") && @oauth.size > 0)
end
def auth_internal_enabled?
return (@auth_type.find(&.== "invidious"))
end
def self.load
# Load config from file or YAML string env var
env_config_file = "INVIDIOUS_CONFIG_FILE"

View file

@ -140,6 +140,7 @@ module Invidious::Database::Playlists
request = <<-SQL
SELECT id,title FROM playlists
WHERE author = $1 AND id LIKE 'IV%'
ORDER BY title
SQL
PG_DB.query_all(request, email, as: {String, String})

View file

@ -0,0 +1,53 @@
require "oauth2"
module Invidious::OAuthHelper
extend self
def get_provider(key)
if provider = CONFIG.oauth[key]?
provider
else
raise Exception.new("Invalid OAuth Endpoint: " + key)
end
end
def make_client(key)
if HOST_URL == ""
raise Exception.new("Missing domain and port configuration")
end
provider = get_provider(key)
redirect_uri = "#{HOST_URL}/login/oauth/#{key}"
OAuth2::Client.new(
provider.host,
provider.client_id,
provider.client_secret,
authorize_uri: provider.auth_uri,
token_uri: provider.token_uri,
redirect_uri: redirect_uri
)
end
def get_uri_host_pair(host, url)
if (url.starts_with?(/https*\:\/\//))
uri = URI.parse url
[uri.host || host, uri.path || "/"]
else
[host, url]
end
end
def get_info(key, token)
provider = self.get_provider(key)
uri_host_pair = self.get_uri_host_pair(provider.host, provider.info_uri)
client = HTTP::Client.new(uri_host_pair[0], tls: true)
token.authenticate(client)
response = client.get uri_host_pair[1]
client.close
response.body
end
def info_field(key, token)
info = JSON.parse(self.get_info(key, token))
info[self.get_provider(key).field].as_s?
end
end

View file

@ -90,7 +90,7 @@ struct SearchVideo
json.field "lengthSeconds", self.length_seconds
json.field "liveNow", self.live_now
json.field "premium", self.premium
json.field "isUpcoming", self.is_upcoming
json.field "isUpcoming", self.upcoming?
if self.premiere_timestamp
json.field "premiereTimestamp", self.premiere_timestamp.try &.to_unix
@ -109,7 +109,7 @@ struct SearchVideo
to_json(nil, json)
end
def is_upcoming
def upcoming?
premiere_timestamp ? true : false
end
end

View file

@ -10,10 +10,8 @@ class Invidious::DecryptFunction
end
def check_update
now = Time.utc
# If we have updated in the last 5 minutes, do nothing
return if (now - @last_update) > 5.minutes
return if (Time.utc - @last_update) < 5.minutes
# Get the amount of time elapsed since when the player was updated, in the
# event where multiple invidious processes are run in parallel.

View file

@ -63,7 +63,7 @@ module Invidious::JSONify::APIv1
json.field "isListed", video.is_listed
json.field "liveNow", video.live_now
json.field "isPostLiveDvr", video.post_live_dvr
json.field "isUpcoming", video.is_upcoming
json.field "isUpcoming", video.upcoming?
if video.premiere_timestamp
json.field "premiereTimestamp", video.premiere_timestamp.try &.to_unix
@ -109,7 +109,7 @@ module Invidious::JSONify::APIv1
# On livestreams, it's not present, so always fall back to the
# current unix timestamp (up to mS precision) for compatibility.
last_modified = fmt["lastModified"]?
last_modified ||= "#{Time.utc.to_unix_ms.to_s}000"
last_modified ||= "#{Time.utc.to_unix_ms}000"
json.field "lmt", last_modified
json.field "projectionType", fmt["projectionType"]
@ -162,7 +162,13 @@ module Invidious::JSONify::APIv1
json.array do
video.fmt_stream.each do |fmt|
json.object do
json.field "url", fmt["url"]
if proxy
json.field "url", Invidious::HttpServer::Utils.proxy_video_url(
fmt["url"].to_s, absolute: true
)
else
json.field "url", fmt["url"]
end
json.field "itag", fmt["itag"].as_i.to_s
json.field "type", fmt["mimeType"]
json.field "quality", fmt["quality"]
@ -212,6 +218,14 @@ module Invidious::JSONify::APIv1
end
end
if !video.chapters.nil?
json.field "chapters" do
json.object do
video.chapters.to_json(json)
end
end
end
if !video.music.empty?
json.field "musicTracks" do
json.array do
@ -271,17 +285,17 @@ module Invidious::JSONify::APIv1
def storyboards(json, id, storyboards)
json.array do
storyboards.each do |storyboard|
storyboards.each do |sb|
json.object do
json.field "url", "/api/v1/storyboards/#{id}?width=#{storyboard[:width]}&height=#{storyboard[:height]}"
json.field "templateUrl", storyboard[:url]
json.field "width", storyboard[:width]
json.field "height", storyboard[:height]
json.field "count", storyboard[:count]
json.field "interval", storyboard[:interval]
json.field "storyboardWidth", storyboard[:storyboard_width]
json.field "storyboardHeight", storyboard[:storyboard_height]
json.field "storyboardCount", storyboard[:storyboard_count]
json.field "url", "/api/v1/storyboards/#{id}?width=#{sb.width}&height=#{sb.height}"
json.field "templateUrl", sb.url.to_s
json.field "width", sb.width
json.field "height", sb.height
json.field "count", sb.count
json.field "interval", sb.interval
json.field "storyboardWidth", sb.columns
json.field "storyboardHeight", sb.rows
json.field "storyboardCount", sb.images_count
end
end
end

View file

@ -46,8 +46,14 @@ struct PlaylistVideo
XML.build { |xml| to_xml(xml) }
end
def to_json(locale : String?, json : JSON::Builder)
to_json(json)
end
def to_json(json : JSON::Builder, index : Int32? = nil)
json.object do
json.field "type", "video"
json.field "title", self.title
json.field "videoId", self.id
@ -67,6 +73,7 @@ struct PlaylistVideo
end
json.field "lengthSeconds", self.length_seconds
json.field "liveNow", self.live_now
end
end

View file

@ -27,10 +27,21 @@ module Invidious::Routes::API::V1::Channels
# Retrieve "sort by" setting from URL parameters
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
begin
videos, _ = Channel::Tabs.get_videos(channel, sort_by: sort_by)
rescue ex
return error_json(500, ex)
if channel.is_age_gated
begin
playlist = get_playlist(channel.ucid.sub("UC", "UULF"))
videos = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
videos = [] of PlaylistVideo
end
next_continuation = nil
else
begin
videos, _ = Channel::Tabs.get_videos(channel, sort_by: sort_by)
rescue ex
return error_json(500, ex)
end
end
JSON.build do |json|
@ -84,6 +95,7 @@ module Invidious::Routes::API::V1::Channels
json.field "joined", channel.joined.to_unix
json.field "autoGenerated", channel.auto_generated
json.field "ageGated", channel.is_age_gated
json.field "isFamilyFriendly", channel.is_family_friendly
json.field "description", html_to_content(channel.description_html)
json.field "descriptionHtml", channel.description_html
@ -142,12 +154,23 @@ module Invidious::Routes::API::V1::Channels
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
continuation = env.params.query["continuation"]?
begin
videos, next_continuation = Channel::Tabs.get_60_videos(
channel, continuation: continuation, sort_by: sort_by
)
rescue ex
return error_json(500, ex)
if channel.is_age_gated
begin
playlist = get_playlist(channel.ucid.sub("UC", "UULF"))
videos = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
videos = [] of PlaylistVideo
end
next_continuation = nil
else
begin
videos, next_continuation = Channel::Tabs.get_60_videos(
channel, continuation: continuation, sort_by: sort_by
)
rescue ex
return error_json(500, ex)
end
end
return JSON.build do |json|
@ -176,12 +199,23 @@ module Invidious::Routes::API::V1::Channels
# Retrieve continuation from URL parameters
continuation = env.params.query["continuation"]?
begin
videos, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
)
rescue ex
return error_json(500, ex)
if channel.is_age_gated
begin
playlist = get_playlist(channel.ucid.sub("UC", "UUSH"))
videos = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
videos = [] of PlaylistVideo
end
next_continuation = nil
else
begin
videos, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
)
rescue ex
return error_json(500, ex)
end
end
return JSON.build do |json|
@ -211,12 +245,23 @@ module Invidious::Routes::API::V1::Channels
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
continuation = env.params.query["continuation"]?
begin
videos, next_continuation = Channel::Tabs.get_60_livestreams(
channel, continuation: continuation, sort_by: sort_by
)
rescue ex
return error_json(500, ex)
if channel.is_age_gated
begin
playlist = get_playlist(channel.ucid.sub("UC", "UULV"))
videos = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
videos = [] of PlaylistVideo
end
next_continuation = nil
else
begin
videos, next_continuation = Channel::Tabs.get_60_livestreams(
channel, continuation: continuation, sort_by: sort_by
)
rescue ex
return error_json(500, ex)
end
end
return JSON.build do |json|

View file

@ -1,3 +1,5 @@
require "html"
module Invidious::Routes::API::V1::Videos
def self.videos(env)
locale = env.get("preferences").as(Preferences).locale
@ -116,7 +118,7 @@ module Invidious::Routes::API::V1::Videos
else
caption_xml = XML.parse(caption_xml)
webvtt = WebVTT.build(settings_field) do |webvtt|
webvtt = WebVTT.build(settings_field) do |builder|
caption_nodes = caption_xml.xpath_nodes("//transcript/text")
caption_nodes.each_with_index do |node, i|
start_time = node["start"].to_f.seconds
@ -136,7 +138,7 @@ module Invidious::Routes::API::V1::Videos
text = "<v #{md["name"]}>#{md["text"]}</v>"
end
webvtt.cue(start_time, end_time, text)
builder.cue(start_time, end_time, text)
end
end
end
@ -187,15 +189,14 @@ module Invidious::Routes::API::V1::Videos
haltf env, 500
end
storyboards = video.storyboards
width = env.params.query["width"]?
height = env.params.query["height"]?
width = env.params.query["width"]?.try &.to_i
height = env.params.query["height"]?.try &.to_i
if !width && !height
response = JSON.build do |json|
json.object do
json.field "storyboards" do
Invidious::JSONify::APIv1.storyboards(json, id, storyboards)
Invidious::JSONify::APIv1.storyboards(json, id, video.storyboards)
end
end
end
@ -205,35 +206,48 @@ module Invidious::Routes::API::V1::Videos
env.response.content_type = "text/vtt"
storyboard = storyboards.select { |sb| width == "#{sb[:width]}" || height == "#{sb[:height]}" }
# Select a storyboard matching the user's provided width/height
storyboard = video.storyboards.select { |x| x.width == width || x.height == height }
haltf env, 404 if storyboard.empty?
if storyboard.empty?
haltf env, 404
else
storyboard = storyboard[0]
end
# Alias variable, to make the code below esaier to read
sb = storyboard[0]
WebVTT.build do |vtt|
start_time = 0.milliseconds
end_time = storyboard[:interval].milliseconds
# Some base URL segments that we'll use to craft the final URLs
work_url = sb.proxied_url.dup
template_path = sb.proxied_url.path
storyboard[:storyboard_count].times do |i|
url = storyboard[:url]
authority = /(i\d?).ytimg.com/.match!(url)[1]?
url = url.gsub("$M", i).gsub(%r(https://i\d?.ytimg.com/sb/), "")
url = "#{HOST_URL}/sb/#{authority}/#{url}"
# Initialize cue timing variables
# NOTE: videojs-vtt-thumbnails gets lost when the cue times don't overlap
# (i.e: if cue[n] end time is 1:06:25.000, cue[n+1] start time should be 1:06:25.000)
time_delta = sb.interval.milliseconds
start_time = 0.milliseconds
end_time = time_delta
storyboard[:storyboard_height].times do |j|
storyboard[:storyboard_width].times do |k|
current_cue_url = "#{url}#xywh=#{storyboard[:width] * k},#{storyboard[:height] * j},#{storyboard[:width] - 2},#{storyboard[:height]}"
vtt.cue(start_time, end_time, current_cue_url)
# Build a VTT file for VideoJS-vtt plugin
vtt_file = WebVTT.build do |vtt|
sb.images_count.times do |i|
# Replace the variable component part of the path
work_url.path = template_path.sub("$M", i)
start_time += storyboard[:interval].milliseconds
end_time += storyboard[:interval].milliseconds
sb.rows.times do |j|
sb.columns.times do |k|
# The URL fragment represents the offset of the thumbnail inside the storyboard image
work_url.fragment = "xywh=#{sb.width * k},#{sb.height * j},#{sb.width - 2},#{sb.height}"
vtt.cue(start_time, end_time, work_url.to_s)
start_time += time_delta
end_time += time_delta
end
end
end
end
# videojs-vtt-thumbnails is not compliant to the VTT specification, it
# doesn't unescape the HTML entities, so we have to do it here:
# TODO: remove this when we migrate to VideoJS 8
return HTML.unescape(vtt_file)
end
def self.annotations(env)
@ -415,4 +429,41 @@ module Invidious::Routes::API::V1::Videos
end
end
end
def self.chapters(env)
id = env.params.url["id"]
region = env.params.query["region"]? || env.params.body["region"]?
if id.nil? || id.size != 11 || !id.matches?(/^[\w-]+$/)
return error_json(400, "Invalid video ID")
end
format = env.params.query["format"]?
begin
video = get_video(id, region: region)
rescue ex : NotFoundException
haltf env, 404
rescue ex
haltf env, 500
end
begin
chapters = video.chapters
rescue ex
haltf env, 500
end
if chapters.nil?
return error_json(404, "No chapters are defined in video \"#{id}\"")
end
if format == "json"
env.response.content_type = "application/json"
return chapters.to_json
else
env.response.content_type = "text/vtt; charset=UTF-8"
return chapters.to_vtt
end
end
end

View file

@ -36,12 +36,24 @@ module Invidious::Routes::Channels
items = items.select(SearchPlaylist)
items.each(&.author = "")
else
sort_options = {"newest", "oldest", "popular"}
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_videos(
channel, continuation: continuation, sort_by: (sort_by || "newest")
)
if channel.is_age_gated
sort_by = ""
sort_options = [] of String
begin
playlist = get_playlist(channel.ucid.sub("UC", "UULF"))
items = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
items = [] of PlaylistVideo
end
next_continuation = nil
else
sort_options = {"newest", "oldest", "popular"}
items, next_continuation = Channel::Tabs.get_videos(
channel, continuation: continuation, sort_by: (sort_by || "newest")
)
end
end
selected_tab = Frontend::ChannelPage::TabsAvailable::Videos
@ -58,14 +70,27 @@ module Invidious::Routes::Channels
return env.redirect "/channel/#{channel.ucid}"
end
# TODO: support sort option for shorts
sort_by = ""
sort_options = [] of String
if channel.is_age_gated
sort_by = ""
sort_options = [] of String
begin
playlist = get_playlist(channel.ucid.sub("UC", "UUSH"))
items = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
items = [] of PlaylistVideo
end
next_continuation = nil
else
# TODO: support sort option for shorts
sort_by = ""
sort_options = [] of String
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
)
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
)
end
selected_tab = Frontend::ChannelPage::TabsAvailable::Shorts
templated "channel"
@ -81,13 +106,26 @@ module Invidious::Routes::Channels
return env.redirect "/channel/#{channel.ucid}"
end
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
sort_options = {"newest", "oldest", "popular"}
if channel.is_age_gated
sort_by = ""
sort_options = [] of String
begin
playlist = get_playlist(channel.ucid.sub("UC", "UULV"))
items = get_playlist_videos(playlist, offset: 0)
rescue ex : InfoException
# playlist doesnt exist.
items = [] of PlaylistVideo
end
next_continuation = nil
else
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
sort_options = {"newest", "oldest", "popular"}
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_60_livestreams(
channel, continuation: continuation, sort_by: sort_by
)
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_60_livestreams(
channel, continuation: continuation, sort_by: sort_by
)
end
selected_tab = Frontend::ChannelPage::TabsAvailable::Streams
templated "channel"

View file

@ -160,6 +160,12 @@ module Invidious::Routes::Images
id = env.params.url["id"]
name = env.params.url["name"]
# Some thumbnails such as the ones for chapters requires some additional queries.
query_params = HTTP::Params.new
{"sqp", "rs"}.each do |name|
query_params[name] = env.params.query[name] if env.params.query[name]?
end
headers = HTTP::Headers.new
if name == "maxres.jpg"
@ -173,7 +179,7 @@ module Invidious::Routes::Images
end
end
url = "/vi/#{id}/#{name}"
url = "/vi/#{id}/#{name}?#{query_params}"
REQUEST_HEADERS_WHITELIST.each do |header|
if env.request.headers[header]?

View file

@ -3,11 +3,10 @@
module Invidious::Routes::Login
def self.login_page(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
user = env.get? "user"
referer = get_referer(env, "/feed/subscriptions")
return env.redirect referer if user
if !CONFIG.login_enabled
@ -19,7 +18,13 @@ module Invidious::Routes::Login
captcha = nil
account_type = env.params.query["type"]?
account_type ||= "invidious"
account_type ||= ""
if CONFIG.auth_type.size == 0
return error_template(401, "No authentication backend enabled.")
elsif CONFIG.auth_type.find(&.== account_type).nil? && CONFIG.auth_type.size == 1
account_type = CONFIG.auth_type[0]
end
captcha_type = env.params.query["captcha"]?
captcha_type ||= "image"
@ -27,9 +32,38 @@ module Invidious::Routes::Login
templated "user/login"
end
def self.login_oauth(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
authorization_code = env.params.query["code"]?
provider_k = env.params.url["provider"]
if authorization_code.nil?
return error_template(403, "Missing Authorization Code")
end
begin
token = OAuthHelper.make_client(provider_k).get_access_token_using_authorization_code(authorization_code)
if email = OAuthHelper.info_field(provider_k, token)
if user = Invidious::Database::Users.select(email: email)
if CONFIG.auth_enforce_source && user.password != ("oauth:" + provider_k)
return error_template(401, "Wrong provider")
else
user_flow_existing(env, email)
end
else
user_flow_new(env, email, nil, "oauth:" + provider_k)
end
end
rescue ex
return error_template(500, "Internal Error")
end
env.redirect referer
end
def self.login(env)
locale = env.get("preferences").as(Preferences).locale
referer = get_referer(env, "/feed/subscriptions")
if !CONFIG.login_enabled
@ -41,9 +75,22 @@ module Invidious::Routes::Login
password = env.params.body["password"]?
account_type = env.params.query["type"]?
account_type ||= "invidious"
account_type ||= ""
if CONFIG.auth_type.size == 0
return error_template(401, "No authentication backend enabled.")
elsif CONFIG.auth_type.find(&.== account_type).nil? && CONFIG.auth_type.size == 1
account_type = CONFIG.auth_type[0]
end
case account_type
when "oauth"
provider_k = env.params.body["provider"]
env.redirect OAuthHelper.make_client(provider_k).get_authorize_uri("openid email profile")
when "saml"
return error_template(501, "Not implemented")
when "ldap"
return error_template(501, "Not implemented")
when "invidious"
if email.nil? || email.empty?
return error_template(401, "User ID is a required field")
@ -53,24 +100,14 @@ module Invidious::Routes::Login
return error_template(401, "Password is a required field")
end
user = Invidious::Database::Users.select(email: email)
if user
if Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
Invidious::Database::SessionIDs.insert(sid, email)
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if user = Invidious::Database::Users.select(email: email)
if user.password.not_nil!.starts_with? "oauth"
return error_template(401, "Wrong provider")
elsif Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
user_flow_existing(env, email)
else
return error_template(401, "Wrong username or password")
end
# Since this user has already registered, we don't want to overwrite their preferences
if env.request.cookies["PREFS"]?
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
else
if !CONFIG.registration_enabled
return error_template(400, "Registration has been disabled by administrator.")
@ -147,32 +184,7 @@ module Invidious::Routes::Login
end
end
end
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
user, sid = create_user(sid, email, password)
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
user.preferences.locale = language.header
end
end
Invidious::Database::Users.insert(user)
Invidious::Database::SessionIDs.insert(sid, email)
view_name = "subscriptions_#{sha256(user.email)}"
PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS #{MATERIALIZED_VIEW_SQL.call(user.email)}")
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if env.request.cookies["PREFS"]?
user.preferences = env.get("preferences").as(Preferences)
Invidious::Database::Users.update_preferences(user)
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
user_flow_new(env, email, password, "internal")
end
env.redirect referer
@ -211,4 +223,49 @@ module Invidious::Routes::Login
env.redirect referer
end
def self.user_flow_existing(env, email)
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
Invidious::Database::SessionIDs.insert(sid, email)
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
# Since this user has already registered, we don't want to overwrite their preferences
if env.request.cookies["PREFS"]?
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
end
def self.user_flow_new(env, email, password, provider)
sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
if provider == "internal"
user, sid = create_internal_user(sid, email, password)
else
user, sid = create_user(sid, email, provider)
end
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
user.preferences.locale = language.header
end
end
Invidious::Database::Users.insert(user)
Invidious::Database::SessionIDs.insert(sid, email)
view_name = "subscriptions_#{sha256(user.email)}"
PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS #{MATERIALIZED_VIEW_SQL.call(user.email)}")
env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
if env.request.cookies["PREFS"]?
user.preferences = env.get("preferences").as(Preferences)
Invidious::Database::Users.update_preferences(user)
cookie = env.request.cookies["PREFS"]
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
end
end
end

View file

@ -53,7 +53,7 @@ module Invidious::Routes::Search
# An URL was copy/pasted in the search box.
# Redirect the user to the appropriate page.
if query.is_url?
if query.url?
return env.redirect UrlSanitizer.process(query.text).to_s
end

View file

@ -131,7 +131,7 @@ module Invidious::Routes::VideoPlayback
end
# TODO: Record bytes written so we can restart after a chunk fails
while true
loop do
if !range_end && content_length
range_end = content_length
end

View file

@ -55,6 +55,7 @@ module Invidious::Routing
def register_user_routes
# User login/out
get "/login", Routes::Login, :login_page
get "/login/oauth/:provider", Routes::Login, :login_oauth
post "/login", Routes::Login, :login
post "/signout", Routes::Login, :signout
@ -236,6 +237,7 @@ module Invidious::Routing
get "/api/v1/annotations/:id", {{namespace}}::Videos, :annotations
get "/api/v1/comments/:id", {{namespace}}::Videos, :comments
get "/api/v1/clips/:id", {{namespace}}::Videos, :clips
get "/api/v1/chapters/:id", {{namespace}}::Videos, :chapters
# Feeds
get "/api/v1/trending", {{namespace}}::Feeds, :trending

View file

@ -149,7 +149,7 @@ module Invidious::Search
end
# Checks if the query is a standalone URL
def is_url? : Bool
def url? : Bool
# If the smart features have been inhibited, don't go further.
return false if @inhibit_ssf

View file

@ -14,6 +14,7 @@ struct Invidious::User
return HTTP::Cookie.new(
name: "SID",
domain: domain,
path: "/",
value: sid,
expires: Time.utc + 2.years,
secure: SECURE,
@ -28,6 +29,7 @@ struct Invidious::User
return HTTP::Cookie.new(
name: "PREFS",
domain: domain,
path: "/",
value: URI.encode_www_form(preferences.to_json),
expires: Time.utc + 2.years,
secure: SECURE,

View file

@ -115,7 +115,7 @@ struct Invidious::User
playlists.each do |item|
title = item["title"]?.try &.as_s?.try &.delete("<>")
description = item["description"]?.try &.as_s?.try &.delete("\r")
privacy = item["privacy"]?.try &.as_s?.try { |privacy| PlaylistPrivacy.parse? privacy }
privacy = item["privacy"]?.try &.as_s?.try { |raw_pl_privacy_state| PlaylistPrivacy.parse? raw_pl_privacy_state }
next if !title
next if !description
@ -161,7 +161,7 @@ struct Invidious::User
# Youtube
# -------------------
private def is_opml?(mimetype : String, extension : String)
private def opml?(mimetype : String, extension : String)
opml_mimetypes = [
"application/xml",
"text/xml",
@ -179,7 +179,7 @@ struct Invidious::User
def from_youtube(user : User, body : String, filename : String, type : String) : Bool
extension = filename.split(".").last
if is_opml?(type, extension)
if opml?(type, extension)
subscriptions = XML.parse(body)
user.subscriptions += subscriptions.xpath_nodes(%q(//outline[@type="rss"])).map do |channel|
channel["xmlUrl"].match!(/UC[a-zA-Z0-9_-]{22}/)[0]

View file

@ -4,7 +4,6 @@ require "crypto/bcrypt/password"
MATERIALIZED_VIEW_SQL = ->(email : String) { "SELECT cv.* FROM channel_videos cv WHERE EXISTS (SELECT subscriptions FROM users u WHERE cv.ucid = ANY (u.subscriptions) AND u.email = E'#{email.gsub({'\'' => "\\'", '\\' => "\\\\"})}') ORDER BY published DESC" }
def create_user(sid, email, password)
password = Crypto::Bcrypt::Password.create(password, cost: 10)
token = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
user = Invidious::User.new({
@ -13,7 +12,7 @@ def create_user(sid, email, password)
subscriptions: [] of String,
email: email,
preferences: Preferences.new(CONFIG.default_user_preferences.to_tuple),
password: password.to_s,
password: password,
token: token,
watched: [] of String,
feed_needs_update: true,
@ -22,6 +21,11 @@ def create_user(sid, email, password)
return user, sid
end
def create_internal_user(sid, email, password)
password = Crypto::Bcrypt::Password.create(password.not_nil!, cost: 10)
create_user(sid, email, password.to_s)
end
def get_subscription_feed(user, max_results = 40, page = 1)
limit = max_results.clamp(0, MAX_ITEMS_PER_PAGE)
offset = (page - 1) * limit

View file

@ -15,7 +15,7 @@ struct Video
# NOTE: don't forget to bump this number if any change is made to
# the `params` structure in videos/parser.cr!!!
#
SCHEMA_VERSION = 2
SCHEMA_VERSION = 3
property id : String
@ -26,6 +26,9 @@ struct Video
@[DB::Field(ignore: true)]
@captions = [] of Invidious::Videos::Captions::Metadata
@[DB::Field(ignore: true)]
@chapters : Invidious::Videos::Chapters? = nil
@[DB::Field(ignore: true)]
property adaptive_fmts : Array(Hash(String, JSON::Any))?
@ -177,65 +180,8 @@ struct Video
# Misc. methods
def storyboards
storyboards = info.dig?("storyboards", "playerStoryboardSpecRenderer", "spec")
.try &.as_s.split("|")
if !storyboards
if storyboard = info.dig?("storyboards", "playerLiveStoryboardSpecRenderer", "spec").try &.as_s
return [{
url: storyboard.split("#")[0],
width: 106,
height: 60,
count: -1,
interval: 5000,
storyboard_width: 3,
storyboard_height: 3,
storyboard_count: -1,
}]
end
end
items = [] of NamedTuple(
url: String,
width: Int32,
height: Int32,
count: Int32,
interval: Int32,
storyboard_width: Int32,
storyboard_height: Int32,
storyboard_count: Int32)
return items if !storyboards
url = URI.parse(storyboards.shift)
params = HTTP::Params.parse(url.query || "")
storyboards.each_with_index do |sb, i|
width, height, count, storyboard_width, storyboard_height, interval, _, sigh = sb.split("#")
params["sigh"] = sigh
url.query = params.to_s
width = width.to_i
height = height.to_i
count = count.to_i
interval = interval.to_i
storyboard_width = storyboard_width.to_i
storyboard_height = storyboard_height.to_i
storyboard_count = (count / (storyboard_width * storyboard_height)).ceil.to_i
items << {
url: url.to_s.sub("$L", i).sub("$N", "M$M"),
width: width,
height: height,
count: count,
interval: interval,
storyboard_width: storyboard_width,
storyboard_height: storyboard_height,
storyboard_count: storyboard_count,
}
end
items
container = info.dig?("storyboards") || JSON::Any.new("{}")
return IV::Videos::Storyboard.from_yt_json(container, self.length_seconds)
end
def paid
@ -254,6 +200,24 @@ struct Video
return @captions
end
def chapters
# As the chapters key is always present in @info we need to check that it is
# actually populated
if @chapters.nil?
chapters = @info["chapters"].as_a
return nil if chapters.empty?
@chapters = Invidious::Videos::Chapters.from_raw_chapters(
chapters,
self.length_seconds,
# Should never be nil but just in case
is_auto_generated: @info["autoGeneratedChapters"].as_bool? || false
)
end
return @chapters
end
def hls_manifest_url : String?
info.dig?("streamingData", "hlsManifestUrl").try &.as_s
end
@ -280,7 +244,7 @@ struct Video
info["genreUcid"].try &.as_s? ? "/channel/#{info["genreUcid"]}" : nil
end
def is_vr : Bool?
def vr? : Bool?
return {"EQUIRECTANGULAR", "MESH"}.includes? self.projection_type
end
@ -361,6 +325,21 @@ struct Video
{% if flag?(:debug_macros) %} {{debug}} {% end %}
end
# Macro to generate ? and = accessor methods for attributes in `info`
private macro predicate_bool(method_name, name)
# Return {{name.stringify}} from `info`
def {{method_name.id.underscore}}? : Bool
return info[{{name.stringify}}]?.try &.as_bool || false
end
# Update {{name.stringify}} into `info`
def {{method_name.id.underscore}}=(value : Bool)
info[{{name.stringify}}] = JSON::Any.new(value)
end
{% if flag?(:debug_macros) %} {{debug}} {% end %}
end
# Method definitions, using the macros above
getset_string author
@ -382,11 +361,12 @@ struct Video
getset_i64 likes
getset_i64 views
# TODO: Make predicate_bool the default as to adhere to Crystal conventions
getset_bool allowRatings
getset_bool authorVerified
getset_bool isFamilyFriendly
getset_bool isListed
getset_bool isUpcoming
predicate_bool upcoming, isUpcoming
end
def get_video(id, refresh = true, region = nil, force_refresh = false)

View file

@ -0,0 +1,108 @@
module Invidious::Videos
# A `Chapters` struct represents an sequence of chapters for a given video
struct Chapters
record Chapter, start_ms : Time::Span, end_ms : Time::Span, title : String, thumbnails : Array(Hash(String, Int32 | String))
property? auto_generated : Bool
def initialize(@chapters : Array(Chapter), @auto_generated : Bool)
end
# Constructs a chapters object from InnerTube's JSON object for chapters
#
# Requires the length of the video the chapters are associated to in order to construct correct ending time
def Chapters.from_raw_chapters(raw_chapters : Array(JSON::Any), video_length : Int32, is_auto_generated : Bool = false)
video_length_milliseconds = video_length.seconds.total_milliseconds
parsed_chapters = [] of Chapter
raw_chapters.each_with_index do |chapter, index|
chapter = chapter["chapterRenderer"]
title = chapter["title"]["simpleText"].as_s
raw_thumbnails = chapter["thumbnail"]["thumbnails"].as_a
thumbnails = raw_thumbnails.map do |thumbnail|
{
"url" => thumbnail["url"].as_s,
"width" => thumbnail["width"].as_i,
"height" => thumbnail["height"].as_i,
}
end
start_ms = chapter["timeRangeStartMillis"].as_i
# To get the ending range we have to peek at the next chapter.
# If we're the last chapter then we need to calculate the end time through the video length.
if next_chapter = raw_chapters[index + 1]?
end_ms = next_chapter["chapterRenderer"]["timeRangeStartMillis"].as_i
else
end_ms = video_length_milliseconds.to_i
end
parsed_chapters << Chapter.new(
start_ms: start_ms.milliseconds,
end_ms: end_ms.milliseconds,
title: title,
thumbnails: thumbnails,
)
end
return Chapters.new(parsed_chapters, is_auto_generated)
end
# Calls the given block for each chapter and passes it as a parameter
def each(&)
@chapters.each { |c| yield c }
end
# Converts the sequence of chapters to a WebVTT representation
def to_vtt
vtt = WebVTT.build do |build|
self.each do |chapter|
build.cue(chapter.start_ms, chapter.end_ms, chapter.title)
end
end
end
# Dumps a JSON representation of the sequence of chapters to the given JSON::Builder
def to_json(json : JSON::Builder)
json.field "autoGenerated", @auto_generated.to_s
json.field "chapters" do
json.array do
@chapters.each do |chapter|
json.object do
json.field "title", chapter.title
json.field "startMs", chapter.start_ms.total_milliseconds
json.field "endMs", chapter.end_ms.total_milliseconds
json.field "thumbnails" do
json.array do
chapter.thumbnails.each do |thumbnail|
json.object do
json.field "url", URI.parse(thumbnail["url"].as(String)).request_target
json.field "width", thumbnail["width"]
json.field "height", thumbnail["height"]
end
end
end
end
end
end
end
end
end
# Create a JSON representation of the sequence of chapters
def to_json
JSON.build do |json|
json.object do
json.field "chapters" do
json.object do
to_json(json)
end
end
end
end
end
end
end

View file

@ -36,7 +36,13 @@ def parse_description(desc, video_id : String) : String?
return "" if content.empty?
commands = desc["commandRuns"]?.try &.as_a
return content if commands.nil?
if commands.nil?
# Slightly faster than HTML.escape, as we're only doing one pass on
# the string instead of five for the standard library
return String.build do |str|
copy_string(str, content.each_codepoint, content.size)
end
end
# Not everything is stored in UTF-8 on youtube's side. The SMP codepoints
# (0x10000 and above) are encoded as UTF-16 surrogate pairs, which are

View file

@ -248,11 +248,12 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
end
end
player_overlays = player_response.dig?("playerOverlays", "playerOverlayRenderer")
# If nothing was found previously, fall back to end screen renderer
if related.empty?
# Container for "endScreenVideoRenderer" items
player_overlays = player_response.dig?(
"playerOverlays", "playerOverlayRenderer",
end_screen_watch_next_array = player_overlays.try &.dig?(
"endScreen", "watchNextEndScreenRenderer", "results"
)
@ -394,6 +395,32 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
.try &.as_s.split(" ", 2)[0]
end
# Chapters
chapters_array = [] of JSON::Any
chapters_auto_generated = nil
# Yes,`decoratedPlayerBarRenderer` is repeated twice.
if player_bar = player_overlays.try &.dig?("decoratedPlayerBarRenderer", "decoratedPlayerBarRenderer", "playerBar")
if markers = player_bar.dig?("multiMarkersPlayerBarRenderer", "markersMap")
potential_chapters_array = markers.as_a.find(&.["key"]?.try &.== "DESCRIPTION_CHAPTERS")
# Chapters that are manually created should have a higher precedence than automatically generated chapters
if !potential_chapters_array
potential_chapters_array = markers.as_a.find(&.["key"]?.try &.== "AUTO_CHAPTERS")
end
if potential_chapters_array
if potential_chapters_array["key"] == "AUTO_CHAPTERS"
chapters_auto_generated = true
else
chapters_auto_generated = false
end
chapters_array = potential_chapters_array["value"]["chapters"].as_a
end
end
end
# Return data
if live_now
@ -414,13 +441,15 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
"lengthSeconds" => JSON::Any.new(length_txt || 0_i64),
"published" => JSON::Any.new(published.to_rfc3339),
# Extra video infos
"allowedRegions" => JSON::Any.new(allowed_regions.map { |v| JSON::Any.new(v) }),
"allowRatings" => JSON::Any.new(allow_ratings || false),
"isFamilyFriendly" => JSON::Any.new(family_friendly || false),
"isListed" => JSON::Any.new(is_listed || false),
"isUpcoming" => JSON::Any.new(is_upcoming || false),
"keywords" => JSON::Any.new(keywords.map { |v| JSON::Any.new(v) }),
"isPostLiveDvr" => JSON::Any.new(post_live_dvr),
"allowedRegions" => JSON::Any.new(allowed_regions.map { |v| JSON::Any.new(v) }),
"allowRatings" => JSON::Any.new(allow_ratings || false),
"isFamilyFriendly" => JSON::Any.new(family_friendly || false),
"isListed" => JSON::Any.new(is_listed || false),
"isUpcoming" => JSON::Any.new(is_upcoming || false),
"keywords" => JSON::Any.new(keywords.map { |v| JSON::Any.new(v) }),
"isPostLiveDvr" => JSON::Any.new(post_live_dvr),
"autoGeneratedChapters" => JSON::Any.new(chapters_auto_generated),
"chapters" => JSON::Any.new(chapters_array),
# Related videos
"relatedVideos" => JSON::Any.new(related),
# Description

View file

@ -0,0 +1,122 @@
require "uri"
require "http/params"
module Invidious::Videos
struct Storyboard
# Template URL
getter url : URI
getter proxied_url : URI
# Thumbnail parameters
getter width : Int32
getter height : Int32
getter count : Int32
getter interval : Int32
# Image (storyboard) parameters
getter rows : Int32
getter columns : Int32
getter images_count : Int32
def initialize(
*, @url, @width, @height, @count, @interval,
@rows, @columns, @images_count
)
authority = /(i\d?).ytimg.com/.match!(@url.host.not_nil!)[1]?
@proxied_url = URI.parse(HOST_URL)
@proxied_url.path = "/sb/#{authority}/#{@url.path.lchop("/sb/")}"
@proxied_url.query = @url.query
end
# Parse the JSON structure from Youtube
def self.from_yt_json(container : JSON::Any, length_seconds : Int32) : Array(Storyboard)
# Livestream storyboards are a bit different
# TODO: document exactly how
if storyboard = container.dig?("playerLiveStoryboardSpecRenderer", "spec").try &.as_s
return [Storyboard.new(
url: URI.parse(storyboard.split("#")[0]),
width: 106,
height: 60,
count: -1,
interval: 5000,
rows: 3,
columns: 3,
images_count: -1
)]
end
# Split the storyboard string into chunks
#
# General format (whitespaces added for legibility):
# https://i.ytimg.com/sb/<video_id>/storyboard3_L$L/$N.jpg?sqp=<sig0>
# | 48 # 27 # 100 # 10 # 10 # 0 # default # rs$<sig1>
# | 80 # 45 # 95 # 10 # 10 # 10000 # M$M # rs$<sig2>
# | 160 # 90 # 95 # 5 # 5 # 10000 # M$M # rs$<sig3>
#
storyboards = container.dig?("playerStoryboardSpecRenderer", "spec")
.try &.as_s.split("|")
return [] of Storyboard if !storyboards
# The base URL is the first chunk
base_url = URI.parse(storyboards.shift)
return storyboards.map_with_index do |sb, i|
# Separate the different storyboard parameters:
# width/height: respective dimensions, in pixels, of a single thumbnail
# count: how many thumbnails are displayed across the full video
# columns/rows: maximum amount of thumbnails that can be stuffed in a
# single image, horizontally and vertically.
# interval: interval between two thumbnails, in milliseconds
# name: storyboard filename. Usually "M$M" or "default"
# sigh: URL cryptographic signature
width, height, count, columns, rows, interval, name, sigh = sb.split("#")
width = width.to_i
height = height.to_i
count = count.to_i
interval = interval.to_i
columns = columns.to_i
rows = rows.to_i
# Copy base URL object, so that we can modify it
url = base_url.dup
# Add the signature to the URL
params = url.query_params
params["sigh"] = sigh
url.query_params = params
# Replace the template parts with what we have
url.path = url.path.sub("$L", i).sub("$N", name)
# This value represents the maximum amount of thumbnails that can fit
# in a single image. The last image (or the only one for short videos)
# will contain less thumbnails than that.
thumbnails_per_image = columns * rows
# This value represents the total amount of storyboards required to
# hold all of the thumbnails. It can't be less than 1.
images_count = (count / thumbnails_per_image).ceil.to_i
# Compute the interval when needed (in general, that's only required
# for the first "default" storyboard).
if interval == 0
interval = ((length_seconds / count) * 1_000).to_i
end
Storyboard.new(
url: url,
width: width,
height: height,
count: count,
interval: interval,
rows: rows,
columns: columns,
images_count: images_count,
)
end
end
end
end

View file

@ -110,13 +110,13 @@ module Invidious::Videos
"Language" => @language_code,
}
vtt = WebVTT.build(settings_field) do |vtt|
vtt = WebVTT.build(settings_field) do |builder|
@lines.each do |line|
# Section headers are excluded from the VTT conversion as to
# match the regular captions returned from YouTube as much as possible
next if line.is_a? HeadingLine
vtt.cue(line.start_ms, line.end_ms, line.line)
builder.cue(line.start_ms, line.end_ms, line.line)
end
end

View file

@ -0,0 +1,34 @@
<% if chapters = video.chapters %>
<div class="description-chapters-section">
<hr class="description-content-separator"/>
<h4><%=HTML.escape(translate(locale, "video_chapters_label"))%></h4>
<% if chapters.auto_generated? %>
<h5><%=HTML.escape(translate(locale, "video_chapters_auto_generated_label"))%> </h5>
<% end %>
<div class="description-chapters-content-container">
<% chapters.each do | chapter | %>
<%- start_in_seconds = chapter.start_ms.total_seconds.to_i %>
<a href="/watch?v=<%-= video.id %>&t=<%=start_in_seconds %>" data-jump-time="<%=start_in_seconds%>" class="chapter-widget-buttons">
<div class="chapter">
<div class="thumbnail">
<%- if !env.get("preferences").as(Preferences).thin_mode -%>
<img loading="lazy" class="thumbnail" src="<%-=URI.parse(chapter.thumbnails[-1]["url"].to_s).request_target %>" alt="<%=chapter.title%>"/>
<%- else -%>
<div class="thumbnail-placeholder"></div>
<%- end -%>
</div>
<%- if start_in_seconds > 0 -%>
<p><%-= recode_length_seconds(start_in_seconds) -%></p>
<%- else -%>
<p>0:00</p>
<%- end -%>
<p><%-=chapter.title-%></p>
</div>
</a>
<% end %>
</div>
<hr class="description-content-separator"/>
</div>
<% end %>

View file

@ -63,6 +63,10 @@
<% captions.each do |caption| %>
<track kind="captions" src="/api/v1/captions/<%= video.id %>?label=<%= caption.name %>" label="<%= caption.name %>">
<% end %>
<% if !video.chapters.nil? %>
<track kind="chapters" src="/api/v1/chapters/<%= video.id %>">
<% end %>
<% end %>
</video>

View file

@ -6,4 +6,7 @@
title="<%= translate(locale, "search") %>"
value="<%= env.get?("search").try {|x| HTML.escape(x.as(String)) } %>">
</fieldset>
<button type="submit" id="searchbutton" aria-label="<%= translate(locale, "search") %>">
<i class="icon ion-ios-search"></i>
</button>
</form>

View file

@ -7,7 +7,18 @@
<div class="pure-u-1 pure-u-lg-3-5">
<div class="h-box">
<% case account_type when %>
<% else # "invidious" %>
<% when "oauth" %>
<form class="pure-form pure-form-stacked" action="/login?referer=<%= URI.encode_www_form(referer) %>&type=oauth" method="post">
<fieldset>
<select name="provider" id="provider">
<% CONFIG.oauth.each_key do |key| %>
<option value="<%= key %>"><%= key %></option>
<% end %>
</select>
<button type="submit" class="pure-button pure-button-primary"><%= translate(locale, "Sign In via OAuth") %></button>
</fieldset>
</form>
<% when "invidious" %>
<form class="pure-form pure-form-stacked" action="/login?referer=<%= URI.encode_www_form(referer) %>&type=invidious" method="post">
<fieldset>
<% if email %>
@ -70,6 +81,14 @@
<% end %>
</fieldset>
</form>
<% else %>
<% if CONFIG.auth_internal_enabled? %>
<a class="pure-button pure-button-secondary" href="/login?referer=<%= URI.encode_www_form(referer) %>&type=invidious">Internal</a>
<% end %>
<% if CONFIG.auth_oauth_enabled? %>
<a class="pure-button pure-button-secondary" href="/login?referer=<%= URI.encode_www_form(referer) %>&type=oauth">OAuth</a>
<% end %>
<label></label>
<% end %>
</div>
</div>

View file

@ -62,7 +62,7 @@ we're going to need to do it here in order to allow for translations.
"params" => params,
"preferences" => preferences,
"premiere_timestamp" => video.premiere_timestamp.try &.to_unix,
"vr" => video.is_vr,
"vr" => video.vr?,
"projection_type" => video.projection_type,
"local_disabled" => CONFIG.disabled?("local"),
"support_reddit" => true
@ -254,10 +254,14 @@ we're going to need to do it here in order to allow for translations.
<div id="description-box"> <!-- Description -->
<% if video.description.size < 200 || params.extend_desc %>
<div id="descriptionWrapper"><%= video.description_html %></div>
<div id="descriptionWrapper"><%-= video.description_html %>
<%= rendered "components/description_chapters_widget" %>
</div>
<% else %>
<input id="descexpansionbutton" type="checkbox"/>
<div id="descriptionWrapper"><%= video.description_html %></div>
<div id="descriptionWrapper"><%-= video.description_html %>
<%= rendered "components/description_chapters_widget" %>
</div>
<label for="descexpansionbutton">
<a></a>
</label>

View file

@ -1,6 +1,6 @@
def add_yt_headers(request)
request.headers.delete("User-Agent") if request.headers["User-Agent"] == "Crystal"
request.headers["User-Agent"] ||= "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
request.headers["User-Agent"] ||= "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
request.headers["Accept-Charset"] ||= "ISO-8859-1,utf-8;q=0.7,*;q=0.7"
request.headers["Accept"] ||= "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"

View file

@ -6,10 +6,10 @@ module YoutubeAPI
extend self
# For Android versions, see https://en.wikipedia.org/wiki/Android_version_history
private ANDROID_APP_VERSION = "19.14.42"
private ANDROID_USER_AGENT = "com.google.android.youtube/19.14.42 (Linux; U; Android 12; US) gzip"
private ANDROID_SDK_VERSION = 31_i64
private ANDROID_APP_VERSION = "19.32.34"
private ANDROID_VERSION = "12"
private ANDROID_USER_AGENT = "com.google.android.youtube/#{ANDROID_APP_VERSION} (Linux; U; Android #{ANDROID_VERSION}; US) gzip"
private ANDROID_SDK_VERSION = 31_i64
private ANDROID_TS_APP_VERSION = "1.9"
private ANDROID_TS_USER_AGENT = "com.google.android.youtube/1.9 (Linux; U; Android 12; US) gzip"
@ -17,9 +17,9 @@ module YoutubeAPI
# For Apple device names, see https://gist.github.com/adamawolf/3048717
# For iOS versions, see https://en.wikipedia.org/wiki/IOS_version_history#Releases,
# then go to the dedicated article of the major version you want.
private IOS_APP_VERSION = "19.16.3"
private IOS_USER_AGENT = "com.google.ios.youtube/19.16.3 (iPhone14,5; U; CPU iOS 17_4 like Mac OS X;)"
private IOS_VERSION = "17.4.0.21E219" # Major.Minor.Patch.Build
private IOS_APP_VERSION = "19.32.8"
private IOS_USER_AGENT = "com.google.ios.youtube/#{IOS_APP_VERSION} (iPhone14,5; U; CPU iOS 17_6 like Mac OS X;)"
private IOS_VERSION = "17.6.1.21G93" # Major.Minor.Patch.Build
private WINDOWS_VERSION = "10.0"
@ -48,7 +48,7 @@ module YoutubeAPI
ClientType::Web => {
name: "WEB",
name_proto: "1",
version: "2.20240304.00.00",
version: "2.20240814.00.00",
screen: "WATCH_FULL_SCREEN",
os_name: "Windows",
os_version: WINDOWS_VERSION,
@ -57,7 +57,7 @@ module YoutubeAPI
ClientType::WebEmbeddedPlayer => {
name: "WEB_EMBEDDED_PLAYER",
name_proto: "56",
version: "1.20240303.00.00",
version: "1.20240812.01.00",
screen: "EMBED",
os_name: "Windows",
os_version: WINDOWS_VERSION,
@ -66,7 +66,7 @@ module YoutubeAPI
ClientType::WebMobile => {
name: "MWEB",
name_proto: "2",
version: "2.20240304.08.00",
version: "2.20240813.02.00",
os_name: "Android",
os_version: ANDROID_VERSION,
platform: "MOBILE",
@ -74,7 +74,7 @@ module YoutubeAPI
ClientType::WebScreenEmbed => {
name: "WEB",
name_proto: "1",
version: "2.20240304.00.00",
version: "2.20240814.00.00",
screen: "EMBED",
os_name: "Windows",
os_version: WINDOWS_VERSION,
@ -147,8 +147,8 @@ module YoutubeAPI
ClientType::IOSMusic => {
name: "IOS_MUSIC",
name_proto: "26",
version: "6.42",
user_agent: "com.google.ios.youtubemusic/6.42 (iPhone14,5; U; CPU iOS 17_4 like Mac OS X;)",
version: "7.14",
user_agent: "com.google.ios.youtubemusic/7.14 (iPhone14,5; U; CPU iOS 17_6 like Mac OS X;)",
device_make: "Apple",
device_model: "iPhone14,5",
os_name: "iPhone",
@ -161,7 +161,7 @@ module YoutubeAPI
ClientType::TvHtml5 => {
name: "TVHTML5",
name_proto: "7",
version: "7.20240304.10.00",
version: "7.20240813.07.00",
},
ClientType::TvHtml5ScreenEmbed => {
name: "TVHTML5_SIMPLY_EMBEDDED_PLAYER",