Refactor the way custom lists are fetched

Allow public custom lists to be fetched without an authenticated user and add the ability to fetch a list from a specific user by authenticating as another user.
pull/20/head
Mitchell Klijs 7 years ago
parent 1c5e68e438
commit 8e641d6690

@ -8,7 +8,7 @@ Trakt lists currently supported:
- popular
- trending
Furthermore, watchlist and other types of list from multiple users are supported.
Furthermore, watchlists and custom list from multiple users are supported.
# Requirements
@ -30,10 +30,7 @@ Install Traktarr to be run with `traktarr` command.
7. `traktarr` - run once to generate a default a config.json file.
8. `nano config.json` - edit preferences.
## 2. Authenticate Trakt (optional)
If you want to acces private lists (watchlists or other user lists),
you'll need to authenticate Traktarr to access your personal lists.
## 2. Create app authentication
1. Create an Trakt application by going [here](https://trakt.tv/oauth/applications/new)
2. Enter a name for your application; for example `Traktarr`
@ -50,6 +47,10 @@ Client ID in the api_key and the Client Secret in the api_secret. Like this:
}
```
## 3. Authenticate users (optional)
If you want to be able to access private lists, you have to authentcate that user.
Repeat the following steps for every user you want to authenticate:
1. Run `traktarr trakt_authenticate` (from the installation path)
2. Go to https://trakt.tv/activate.
@ -61,7 +62,7 @@ Repeat the following steps for every user you want to authenticate:
You've now authenticated the user.
You can repeat this process for as many users as you want.
## 3. Setup Schedule
## 4. Setup Schedule
To have Traktarr get Movies and Shows for you automatically, on set interval.
@ -74,6 +75,8 @@ To have Traktarr get Movies and Shows for you automatically, on set interval.
# Configuration
Here is some default configuration you can use.
```json
{
"core": {
@ -85,25 +88,13 @@ To have Traktarr get Movies and Shows for you automatically, on set interval.
"boxoffice": 10,
"interval": 24,
"popular": 3,
"trending": 2,
"watchlist": {
"username": 10
},
"my-custom-list" {
"username": 10
}
"trending": 2
},
"shows": {
"anticipated": 10,
"interval": 48,
"popular": 1,
"trending": 2,
"watchlist": {
"username": 10
},
"my-custom-list" {
"username": 10
}
"trending": 2
}
},
"filters": {
@ -201,6 +192,22 @@ To have Traktarr get Movies and Shows for you automatically, on set interval.
```
## Watchlist
Traktarr can fetch the watchlist for as many users as you like.
You'll have to authenticate every user from whome you want to fetch the watchlist,
by following the steps described [here]((#authenticate-users-optional)).
When all users are authenticated you can fetch their watchlist either
with the automatic task or with the manual commands (see examples below).
## Other custom user lists
Traktarr can also fetch any number of other custom lists.
If the custom list is private, you'll have to authenticate a user that is allowed to
access that list by following the steps described [here]((#authenticate-users-optional)).
## Core
```json
@ -226,43 +233,124 @@ Movies can be run on a separate schedule from Shows.
"interval": 24,
"popular": 3,
"trending": 2,
"watchlist": {
"user1": 10
"user2": 10
},
"my-custom-list": {
"user1": 10
},
"another-custom-list": {
"user2": 10
}
"watchlist": {},
"lists": {}
},
"shows": {
"anticipated": 10,
"interval": 48,
"popular": 1,
"trending": 2,
"watchlist": {},
"lists": {}
}
},
```
`interval` - specify how often (in hours) to run Traktarr task.
`anticipated`, `popular`, `trending`, `boxoffice` (movies only) - specify how many items from each Trakt list to find.
`watchlist` - specify which watchlists to fetch (see explanation below)
`lists` - specify which custom lists to fetch (see explanation below)
### Watchlist
The watchlist task can be scheduled with a differtent item limit for every user.
For every user you've to add: `"username": limit` to the watchlist key. For example:
```json
"automatic": {
"movies": {
"watchlist": {
"user1": 10
"user2": 10
"user1": 10,
"user2": 5
}
},
"shows": {
"watchlist": {
"user1": 2,
"user3": 1
}
}
},
"my-custom-list": {
"user1": 10
```
Of course you can combine this with running the other list types as well.
### Custom lists
You can also schedule any number of public or private custom lists.
For both public and private lsits you'll need the url to that list.
You can copy this url from the address bar in you browser when viewing
the list on Trakt.
Public lists can be added by specifying the url and the item limit like this:
```json
"automatic": {
"movies": {
"lists": {
"https://trakt.tv/users/rkerwin/lists/top-100-movies": 10
}
},
"another-custom-list": {
"user2": 10
"shows": {
"lists": {
"https://trakt.tv/users/claireaa/lists/top-100-tv-shows-of-all-time-ign": 10
}
}
},
```
`interval` - specify how often (in hours) to run Traktarr task.
Private lists can be added in two ways:
`anticipated`, `popular`, `trending`, `boxoffice` (movies only) - specify how many items from each Trakt list to find.
1. If there is only one authenticated user to Traktarr, you can add
the private list just like any other public list:
`watchlist` - add every user you want to fetch items from and specify how many items to fetch
```json
"automatic": {
"movies": {
"lists": {
"https://trakt.tv/users/user/lists/my-private-movies-list": 10
}
},
"shows": {
"lists": {
"https://trakt.tv/users/user/lists/my-private-shows-list": 10
}
}
},
```
2. If there are multiple authenticated users to Traktarr, you'll need
to specify with which user Traktarr should authenticate when fetching
the list. The user should have acces to the list (either own the list,
or friends with the owner of the list and the list is specified to be
shared with friends)
_Note that the specified user has to be authenticated_
```json
"automatic": {
"movies": {
"lists": {
"https://trakt.tv/users/user/lists/my-private-movies-list": {
"authenticate_as": "user2",
"limit": 10
}
}
},
"shows": {
"lists": {
"https://trakt.tv/users/user/lists/my-private-shows-list": {
"authenticate_as": "user2",
"limit": 10
}
}
}
},
```
You can add every (private) list you want by adding the list key.
## Filters
@ -501,7 +589,8 @@ Finally, we will edit the Traktarr config and assign the `AMZN` tag to certain n
```json
"trakt": {
"api_key": ""
"api_key": "",
"api_scret": ""
}
```
@ -509,6 +598,9 @@ Finally, we will edit the Traktarr config and assign the `AMZN` tag to certain n
`api_secret` - Fill in your Trakt Secret key (_Client Scret_)
_Note that when users authenticate to Traktarr, their token information
will be added to this._
# Usage
@ -552,7 +644,7 @@ Usage: traktarr movies [OPTIONS]
Options:
-t, --list-type TEXT Trakt list to process. For example, anticipated,
trending, popular, boxoffice, watchlist or any
other user list [required]
URL to a list [required]
-l, --add-limit INTEGER Limit number of movies added to Radarr.
[default: 0]
-d, --add-delay FLOAT Seconds between each add request to Radarr.
@ -561,9 +653,9 @@ Options:
-f, --folder TEXT Add movies with this root folder to Radarr.
--no-search Disable search when adding movies to Radarr.
--notifications Send notifications.
--user TEXT Specify which user to use for the personal Trakt
lists. Default: first user in the config
--help Show this message and exit.
--authencate-user TEXT Specify which user to authenticate with to
retrieve Trakt lists. Default: first user in the
config
```
@ -576,7 +668,7 @@ Usage: traktarr shows [OPTIONS]
Options:
-t, --list-type TEXT Trakt list to process. For example, anticipated,
trending, popular, watchlist or any other user
trending, popular, watchlist or any URL to a
list [required]
-l, --add-limit INTEGER Limit number of shows added to Sonarr.
[default: 0]
@ -586,19 +678,39 @@ Options:
-f, --folder TEXT Add shows with this root folder to Sonarr.
--no-search Disable search when adding shows to Sonarr.
--notifications Send notifications.
--user TEXT Specify which user to use for the personal Trakt
lists. Default: first user in the config
--authencate-user TEXT Specify which user to authenticate with to
retrieve Trakt lists. Default: first user in the
config
--help Show this message and exit.
```
## Examples
- Fetch boxoffice movies labeld with the comedy genere, limit to 10 items and send notifications
```
traktarr movies -t boxoffice -g comedy -l 10 --notifications
```
- Fetch popular shows, limit to 2 items and don't start the search in Sonarr
```
traktarr shows -t popular -l 2 --no-search
```
- Fetch all shows from the watchlist from user1
```
traktarr shows -t watchlist --authenticate-user user1
```
- Fetch all movies from the public https://trakt.tv/users/rkerwin/lists/top-100-movies list
```
traktarr shows -t https://trakt.tv/users/rkerwin/lists/top-100-movies
```
- Fetch all movies from the private https://trakt.tv/users/user1/lists/private-movies-list list
```
traktarr shows -t https://trakt.tv/users/user1/lists/private-movies-list --authenticate-user=user1
```

@ -161,9 +161,7 @@ class Trakt:
def oauth_headers(self, user):
headers = self.headers
if user is None or user not in self.cfg['trakt'].keys():
log.debug('No authenticated user corresponds to "%s", so the first user in the config to authenticated.', user)
if user is None:
users = self.cfg['trakt']
if 'api_key' in users.keys():
@ -172,8 +170,21 @@ class Trakt:
if 'api_secret' in users.keys():
users.pop('api_secret')
if len(users) > 0:
user = list(users.keys())[0]
log.debug('No user provided, so default to the first user in the config (%s)', user)
elif user not in self.cfg['trakt'].keys():
log.error('The user %s you specified to use for authentication is not authenticated yet. Authenticate the user first, before you use it to retrieve lists.', user)
exit()
# If there is no default user, try without authentication
if user is None:
log.info('Using no authentication')
return headers, user
token_information = self.cfg['trakt'][user]
# Check if the acces_token for the user is expired
expires_at = token_information['created_at'] + token_information['expires_in']
@ -188,7 +199,7 @@ class Trakt:
headers['Authorization'] = 'Bearer ' + token_information['access_token']
return headers
return headers, user
############################################################
# Shows
@ -255,7 +266,7 @@ class Trakt:
return None
@backoff.on_predicate(backoff.expo, lambda x: x is None, max_tries=4, on_backoff=backoff_handler)
def get_watchlist_shows(self, user=None, limit=1000, languages=None):
def get_watchlist_shows(self, authenticate_user=None, limit=1000, languages=None):
try:
processed_shows = []
@ -269,10 +280,12 @@ class Trakt:
# make request
while True:
headers, authenticate_user = self.oauth_headers(authenticate_user)
req = requests.get('https://api.trakt.tv/sync/watchlist/shows', params=payload,
headers=self.oauth_headers(user),
headers=headers,
timeout=30)
log.debug("Request User: %s", user)
log.debug("Request User: %s", authenticate_user)
log.debug("Request URL: %s", req.url)
log.debug("Request Payload: %s", payload)
log.debug("Response Code: %d", req.status_code)
@ -304,12 +317,12 @@ class Trakt:
exit()
else:
log.error("Failed to retrieve shows on watchlist from %s, request response: %d", user,
log.error("Failed to retrieve shows on watchlist from %s, request response: %d", authenticate_user,
req.status_code)
break
if len(processed_shows):
log.debug("Found %d shows on watchlist from %s", len(processed_shows), user)
log.debug("Found %d shows on watchlist from %s", len(processed_shows), authenticate_user)
return processed_shows
return None
@ -318,7 +331,7 @@ class Trakt:
return None
@backoff.on_predicate(backoff.expo, lambda x: x is None, max_tries=4, on_backoff=backoff_handler)
def get_user_list_shows(self, list_id, user=None, limit=1000, languages=None):
def get_user_list_shows(self, list_url, authenticate_user=None, limit=1000, languages=None):
try:
processed_shows = []
@ -330,13 +343,26 @@ class Trakt:
if languages:
payload['languages'] = ','.join(languages)
try:
import re
list_user = re.search('\/users\/([^/]*)', list_url).group(1)
list_key = re.search('\/lists\/([^/]*)', list_url).group(1)
except:
log.error('The URL "%s" is not in the correct format', list_url)
exit()
log.debug('Fetching %s from %s', list_key, list_user)
# make request
while True:
req = requests.get('https://api.trakt.tv/users/' + user + '/lists/' + list_id + '/items/shows',
headers, authenticate_user = self.oauth_headers(authenticate_user)
req = requests.get('https://api.trakt.tv/users/' + list_user + '/lists/' + list_key + '/items/shows',
params=payload,
headers=self.oauth_headers(user),
headers=headers,
timeout=30)
log.debug("Request User: %s", user)
log.debug("Request User: %s", authenticate_user)
log.debug("Request URL: %s", req.url)
log.debug("Request Payload: %s", payload)
log.debug("Response Code: %d", req.status_code)
@ -748,7 +774,7 @@ class Trakt:
return None
@backoff.on_predicate(backoff.expo, lambda x: x is None, max_tries=4, on_backoff=backoff_handler)
def get_watchlist_movies(self, user=None, limit=1000, languages=None):
def get_watchlist_movies(self, authenticate_user=None, limit=1000, languages=None):
try:
processed_movies = []
@ -762,10 +788,12 @@ class Trakt:
# make request
while True:
headers, authenticate_user = self.oauth_headers(authenticate_user)
req = requests.get('https://api.trakt.tv/sync/watchlist/movies', params=payload,
headers=self.oauth_headers(user),
headers=headers,
timeout=30)
log.debug("Request User: %s", user)
log.debug("Request User: %s", authenticate_user)
log.debug("Request URL: %s", req.url)
log.debug("Request Payload: %s", payload)
log.debug("Response Code: %d", req.status_code)
@ -797,12 +825,12 @@ class Trakt:
exit()
else:
log.error("Failed to retrieve movies on watchlist from %s, request response: %d", user,
log.error("Failed to retrieve movies on watchlist from %s, request response: %d", authenticate_user,
req.status_code)
break
if len(processed_movies):
log.debug("Found %d movies on watchlist from %s", len(processed_movies), user)
log.debug("Found %d movies on watchlist from %s", len(processed_movies), authenticate_user)
return processed_movies
return None
@ -811,7 +839,7 @@ class Trakt:
return None
@backoff.on_predicate(backoff.expo, lambda x: x is None, max_tries=4, on_backoff=backoff_handler)
def get_user_list_movies(self, list_id, user=None, limit=1000, languages=None):
def get_user_list_movies(self, list_url, authenticate_user=None, limit=1000, languages=None):
try:
processed_movies = []
@ -823,13 +851,24 @@ class Trakt:
if languages:
payload['languages'] = ','.join(languages)
try:
import re
list_user = re.search('\/users\/([^/]*)', list_url).group(1)
list_key = re.search('\/lists\/([^/]*)', list_url).group(1)
except:
log.error('The URL "%s" is not in the correct format', list_url)
log.debug('Fetching %s from %s', list_key, list_user)
# make request
while True:
req = requests.get('https://api.trakt.tv/users/' + user + '/lists/' + list_id + '/items/movies',
headers, authenticate_user = self.oauth_headers(authenticate_user)
req = requests.get('https://api.trakt.tv/users/' + list_user + '/lists/' + list_key + '/items/movies',
params=payload,
headers=self.oauth_headers(user),
headers=headers,
timeout=30)
log.debug("Request User: %s", user)
log.debug("Request User: %s", authenticate_user)
log.debug("Request URL: %s", req.url)
log.debug("Request Payload: %s", payload)
log.debug("Response Code: %d", req.status_code)
@ -861,12 +900,12 @@ class Trakt:
exit()
else:
log.error("Failed to retrieve movies on watchlist from %s, request response: %d", user,
log.error("Failed to retrieve movies on watchlist from %s, request response: %d", authenticate_user,
req.status_code)
break
if len(processed_movies):
log.debug("Found %d movies on watchlist from %s", len(processed_movies), user)
log.debug("Found %d movies on watchlist from %s", len(processed_movies), authenticate_user)
return processed_movies
return None

@ -75,7 +75,7 @@ def trakt_authentication():
@app.command(help='Add new shows to Sonarr.')
@click.option('--list-type', '-t',
help='Trakt list to process. For example, anticipated, trending, popular, watchlist or any other user list',
help='Trakt list to process. For example, anticipated, trending, popular, watchlist or any URL to a list',
required=True)
@click.option('--add-limit', '-l', default=0, help='Limit number of shows added to Sonarr.', show_default=True)
@click.option('--add-delay', '-d', default=2.5, help='Seconds between each add request to Sonarr.', show_default=True)
@ -83,10 +83,10 @@ def trakt_authentication():
@click.option('--folder', '-f', default=None, help='Add shows with this root folder to Sonarr.')
@click.option('--no-search', is_flag=True, help='Disable search when adding shows to Sonarr.')
@click.option('--notifications', is_flag=True, help='Send notifications.')
@click.option('--user',
help='Specify which user to use for the personal Trakt lists. Default: first user in the config')
@click.option('--authenticate-user',
help='Specify which user to authenticate with to retrieve Trakt lists. Default: first user in the config')
def shows(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_search=False, notifications=False,
user=None):
authenticate_user=None):
from media.sonarr import Sonarr
from media.trakt import Trakt
from misc import helpers
@ -165,9 +165,15 @@ def shows(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_sea
elif list_type.lower() == 'popular':
trakt_series_list = trakt.get_popular_shows()
elif list_type.lower() == 'watchlist':
trakt_series_list = trakt.get_watchlist_shows(user)
trakt_series_list = trakt.get_watchlist_shows(authenticate_user)
elif list_type.lower() == 'lists':
trakt_series_list = trakt.get_user_list_shows(list_type, authenticate_user)
else:
trakt_series_list = trakt.get_user_list_shows(list_type, user)
log.error("Aborting due to unknown Trakt list type")
if notifications:
callback_notify({'event': 'abort', 'type': 'shows', 'list_type': list_type,
'reason': 'Failure to determine Trakt list type'})
return None
if not trakt_series_list:
log.error("Aborting due to failure to retrieve Trakt %s shows list", list_type)
@ -252,7 +258,7 @@ def shows(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_sea
@app.command(help='Add new movies to Radarr.')
@click.option('--list-type', '-t',
help='Trakt list to process. For example, anticipated, trending, popular, boxoffice, watchlist or any other user list',
help='Trakt list to process. For example, anticipated, trending, popular, boxoffice, watchlist or any URL to a list',
required=True)
@click.option('--add-limit', '-l', default=0, help='Limit number of movies added to Radarr.', show_default=True)
@click.option('--add-delay', '-d', default=2.5, help='Seconds between each add request to Radarr.', show_default=True)
@ -260,10 +266,10 @@ def shows(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_sea
@click.option('--folder', '-f', default=None, help='Add movies with this root folder to Radarr.')
@click.option('--no-search', is_flag=True, help='Disable search when adding movies to Radarr.')
@click.option('--notifications', is_flag=True, help='Send notifications.')
@click.option('--user',
help='Specify which user to use for the personal Trakt lists. Default: first user in the config')
@click.option('--authenticate-user',
help='Specify which user to authenticate with to retrieve Trakt lists. Default: first user in the config')
def movies(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_search=False, notifications=False,
user=None):
authenticate_user=None):
from media.radarr import Radarr
from media.trakt import Trakt
from misc import helpers
@ -334,9 +340,15 @@ def movies(list_type, add_limit=0, add_delay=2.5, genre=None, folder=None, no_se
elif list_type.lower() == 'boxoffice':
trakt_movies_list = trakt.get_boxoffice_movies()
elif list_type.lower() == 'watchlist':
trakt_movies_list = trakt.get_watchlist_shows(user)
trakt_movies_list = trakt.get_watchlist_movies(authenticate_user)
elif list_type.lower() == 'lists':
trakt_movies_list = trakt.get_user_list_movies(list_type, authenticate_user)
else:
trakt_movies_list = trakt.get_user_list_shows(list_type, user)
log.error("Aborting due to unknown Trakt list type")
if notifications:
callback_notify({'event': 'abort', 'type': 'movies', 'list_type': list_type,
'reason': 'Failure to determine Trakt list type'})
return None
if not trakt_movies_list:
log.error("Aborting due to failure to retrieve Trakt %s movies list", list_type)
@ -441,34 +453,49 @@ def automatic_shows(add_delay=2.5, no_search=False, notifications=False):
log.info("Started")
for list_type, value in cfg.automatic.shows.items():
added_shows = None
if list_type.lower() == 'interval':
continue
if list_type.lower() not in Trakt.non_user_lists:
type_amount = value
if list_type.lower() in Trakt.non_user_lists:
limit = value
if type_amount <= 0:
if limit <= 0:
log.info("Skipped Trakt's %s shows list", list_type)
continue
else:
log.info("Adding %d shows from Trakt's %s list", type_amount, list_type)
log.info("Adding %d shows from Trakt's %s list", limit, list_type)
# run shows
added_shows = shows.callback(list_type=list_type, add_limit=type_amount,
added_shows = shows.callback(list_type=list_type, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications)
else:
for user, type_amount in value:
if type_amount <= 0:
log.info("Skipped Trakt's %s for &s", list_type, user)
elif list_type.lower() == 'watchlist':
for authenticate_user, limit in value.items():
if limit <= 0:
log.info("Skipped Trakt's %s for %s", list_type, authenticate_user)
continue
else:
log.info("Adding %d shows from the %s from &s", type_amount, list_type, user)
log.info("Adding %d shows from the %s from %s", limit, list_type, authenticate_user)
# run shows
added_shows = shows.callback(list_type=list_type, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications, authenticate_user=authenticate_user)
elif list_type.lower() == 'lists':
for list, v in value.items():
if isinstance(v, dict):
authenticate_user = v['authenticate_user']
limit = v['limit']
else:
authenticate_user = None
limit = v
# run shows
added_shows = shows.callback(list_type=list_type, add_limit=type_amount,
added_shows = shows.callback(list_type=list_type, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications, user=user)
notifications=notifications, authenticate_user=authenticate_user)
if added_shows is None:
log.error("Failed adding shows from Trakt's %s list", list_type)
@ -497,34 +524,49 @@ def automatic_movies(add_delay=2.5, no_search=False, notifications=False):
log.info("Started")
for list_type, value in cfg.automatic.movies.items():
added_movies = None
if list_type.lower() == 'interval':
continue
if list_type.lower() not in Trakt.non_user_lists:
type_amount = value
if list_type.lower() in Trakt.non_user_lists:
limit = value
if type_amount <= 0:
if limit <= 0:
log.info("Skipped Trakt's %s movies list", list_type)
continue
else:
log.info("Adding %d movies from Trakt's %s list", type_amount, list_type)
log.info("Adding %d movies from Trakt's %s list", limit, list_type)
# run movies
added_movies = movies.callback(list_type=list_type, add_limit=type_amount,
added_movies = movies.callback(list_type=list_type, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications)
else:
for user, type_amount in value:
if type_amount <= 0:
log.info("Skipped Trakt's %s for &s", list_type, user)
elif list_type.lower() == 'watchlist':
for authenticate_user, limit in value.items():
if limit <= 0:
log.info("Skipped Trakt's %s for %s", list_type, authenticate_user)
continue
else:
log.info("Adding %d movies from the %s from &s", type_amount, list_type, user)
log.info("Adding %d movies from the %s from %s", limit, list_type, authenticate_user)
# run movies
added_movies = movies.callback(list_type=list_type, add_limit=type_amount,
added_movies = movies.callback(list_type=list_type, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications, authenticate_user=authenticate_user)
elif list_type.lower() == 'lists':
for list, v in value.items():
if isinstance(v, dict):
authenticate_user = v['authenticate_user']
limit = v['limit']
else:
authenticate_user = None
limit = v
# run shows
added_movies = movies.callback(list_type=list, add_limit=limit,
add_delay=add_delay, no_search=no_search,
notifications=notifications, user=user)
notifications=notifications, authenticate_user=authenticate_user)
if added_movies is None:
log.error("Failed adding movies from Trakt's %s list", list_type)

Loading…
Cancel
Save