mirror of https://github.com/x89/Shreddit
Scott Hand
8 years ago
committed by
GitHub
commit
01e96903d0
@ -1,112 +1,107 @@
|
||||
Shreddit
|
||||
========
|
||||
|
||||
Description
|
||||
-----------
|
||||
Shreddit is a Python command line program which will take a user's post history
|
||||
on the website [Reddit](http://reddit.com), and will systematically go through
|
||||
the user's history deleting one post/submission at a time until only those
|
||||
whitelisted remain.
|
||||
|
||||
**Note:** When it became known that post edits were *not* saved but post
|
||||
deletions *were* saved, code was added to edit your post prior to deletion. In
|
||||
fact you can actually turn off deletion all together and just have lorem ipsum
|
||||
(or a message about Shreddit) but this will increase how long it takes the
|
||||
script to run as it will be going over all of your messages every run!
|
||||
|
||||
It allows you to maintain your normal reddit account while having your history
|
||||
# Shreddit
|
||||
|
||||
Shreddit is a Python command line program which will take a user's post history on the website
|
||||
[Reddit](http://reddit.com), and will systematically go through the user's history deleting one post/submission at a
|
||||
time until only those whitelisted remain. It allows you to maintain your normal reddit account while having your history
|
||||
scrubbed after a certain amount of time.
|
||||
|
||||
User Login deprecation
|
||||
----------------------
|
||||
Reddit intends to disable username-password based authentication to access its
|
||||
APIs in the near future. You can specify your username and password in the
|
||||
`shreddit.yml` or the `praw.ini` to make it work **FOR NOW**. But consider
|
||||
looking at the [OAuth2 instructions](#oauth2-instructions) if you intend to use
|
||||
this program in the future.
|
||||
|
||||
Installation ([Click here for Windows instructions](#for-windows-users))
|
||||
------------------------------------------------------------------------
|
||||
1. Clone the repository
|
||||
2. Run `python setup.py install`. Usually this is run in the context of a
|
||||
virtualenv or with administrative permissions for system-wide installation.
|
||||
virtual environment)
|
||||
3. Copy `shreddit.yml.example` to `shreddit.yml` and edit it to your liking.
|
||||
- Make sure you specify your credentials in the file.
|
||||
- See the [OAuth2 instructions](#oauth2-instructions) if you don't want to
|
||||
use username-password based authentication.
|
||||
|
||||
- It's useful to have it run as an event, you can set this up as you like but I
|
||||
suggest `cron` via `crontab -e` and adding a line such as
|
||||
`@hourly cd $HOME/Shreddit && source bin/activate && shreddit` See below for
|
||||
more.
|
||||
- Adding your password to the praw.ini and adding the additional output line
|
||||
can provide extra debugging help.
|
||||
|
||||
Cron examples
|
||||
-------------
|
||||
|
||||
- Run `crontab -e` to edit your cron file. If you have access to something like
|
||||
vixie-cron then each user can have their own personal cron job!
|
||||
When it became known that post edits were *not* saved but post deletions *were* saved, code was added to edit your post
|
||||
prior to deletion. In fact you can actually turn off deletion all together and just have lorem ipsum (or a message
|
||||
about Shreddit) but this will increase how long it takes the script to run as it will be going over all of your messages
|
||||
every run.
|
||||
|
||||
## User Login deprecation
|
||||
|
||||
Reddit intends to disable username-password based authentication to access its APIs in the near future. You can specify
|
||||
your username and password in the `shreddit.yml` or the `praw.ini` to make it work **FOR NOW**. But consider looking at
|
||||
the [OAuth2 instructions](#oauth2-instructions) if you intend to use this program in the future.
|
||||
|
||||
## Pip Installation
|
||||
|
||||
`pip install -U shreddit` will install the package and its dependencies, and it will add a `shreddit` command line
|
||||
utility to your PATH. This is typically either run in a virtualenv or using administrative privileges for global
|
||||
installation.
|
||||
|
||||
## Manual Installation
|
||||
|
||||
1. Clone the `shreddit` repository to a directory.
|
||||
2. From the directory, run `pip install -r requirements.txt`
|
||||
3. Run `python setup.py install` to install the package and the `shreddit` command line utility. This is typically
|
||||
either run in a virtualenv or using administrative privileges for global installation.
|
||||
|
||||
## Usage
|
||||
|
||||
After installing the `shreddit` command line utility, the first step is setting up the tool's configuration file. Simply
|
||||
typing `shreddit` will print a message with an example config. Copy the message from `---` onwards and save it as
|
||||
`shreddit.yml`. Now, the tool may be used by simply typing `shreddit` from this directory. Alternatively, if you named
|
||||
the configuration file something different such as `config.yml`, you may use it with `shreddit -c config.yml`.
|
||||
|
||||
### Automating
|
||||
|
||||
The easiest way to automate this tool after the first run is by using the cron utility. Run `crontab -e` to edit your
|
||||
user's crontab settings.
|
||||
|
||||
**Examples:**
|
||||
|
||||
- Run every hour on the hour
|
||||
`0 * * * * shreddit -c <full path to shreddit.yml>`
|
||||
`0 * * * * shreddit -c <full path to shreddit.yml>`
|
||||
|
||||
- Run at 3am every morning
|
||||
`0 3 * * * shreddit -c <full path to shreddit.yml>`
|
||||
`0 3 * * * shreddit -c <full path to shreddit.yml>`
|
||||
|
||||
- Run once a month on the 1st of the month
|
||||
`0 0 1 * * shreddit -c <full path to shreddit.yml>`
|
||||
`0 0 1 * * shreddit -c <full path to shreddit.yml>`
|
||||
|
||||
If virtualenv was used, be sure to add `source /full/path/to/venv/bin/activate &&` before the command. For example:
|
||||
|
||||
If virtualenv was used, be sure to add
|
||||
`source /full/path/to/venv/bin/activate &&`
|
||||
before the command. For example:
|
||||
`0 * * * * source /full/path/to/venv/bin/activate && shreddit -c <full path to shreddit.yml>`
|
||||
|
||||
`0 * * * * source /full/path/to/venv/bin/activate &&
|
||||
shreddit -c <full path to shreddit.yml>`
|
||||
### Command Line Options
|
||||
|
||||
For Windows users
|
||||
-----------------
|
||||
```
|
||||
$ shreddit --help
|
||||
usage: shreddit [-h] [-c CONFIG] [-p PRAW] [-t]
|
||||
|
||||
Command-line frontend to the shreddit library.
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-c CONFIG, --config CONFIG
|
||||
Config file to use instead of the default shreddit.yml
|
||||
-p PRAW, --praw PRAW PRAW config (if not ./praw.ini)
|
||||
-t, --test-oauth Perform OAuth test and exit
|
||||
```
|
||||
|
||||
## For Windows users
|
||||
|
||||
1. Make sure you have Python installed.
|
||||
[Click here for the Python download page](https://www.python.org/downloads/).
|
||||
- **Note:** Install either `python 2.x` or `python 3.x`, not both.
|
||||
2. Clone the repository (or download and extract the zip file)
|
||||
3. Open command prompt to the folder with the zip file (Shreddit-master.zip),
|
||||
and type `pip install -U Shreddit-master.zip`
|
||||
4. Open `shreddit.yml.example` in the zip edit it to your liking, and rename the
|
||||
file to `shreddit.yml`.
|
||||
- Make sure you specify credentials in the file.
|
||||
- See the [OAuth2 instructions](#oauth2-instructions) if you don't want to
|
||||
use username-password based authentication.
|
||||
5. Type `shreddit` in the open command prompt window to run the program.
|
||||
|
||||
OAuth2 Instructions
|
||||
-------------------
|
||||
- **Note:** Install either `python 2.x` or `python 3.x`, not both.
|
||||
2. Follow the [pip installation](#pip-installation) instructions.
|
||||
3. Open a new command prompt and verify that the `shreddit` command works before moving on to the [usage](#usage)
|
||||
section.
|
||||
|
||||
## OAuth2 Instructions
|
||||
|
||||
1. Visit: https://www.reddit.com/prefs/apps
|
||||
2. Click on 'Create app'.
|
||||
- Fill in the name and select the 'script' option
|
||||
- Under "redirect uri" put http://127.0.0.1:65010
|
||||
3. Copy from or rename `praw.ini.example` to `praw.ini` and open it. Enter the
|
||||
values from the Reddit page.
|
||||
- oauth\_client\_id = { The ID displayed next to the icon thingy (under
|
||||
- Fill in the name and select the 'script' option
|
||||
- Under "redirect uri" put http://127.0.0.1:65010
|
||||
3. Copy from or rename `praw.ini.example` to `praw.ini` and open it. Enter the values from the Reddit page.
|
||||
- oauth\_client\_id = { The ID displayed next to the icon thingy (under
|
||||
"personal use script") }
|
||||
- oauth\_client\_secret = { The secret }
|
||||
- oauth\_redirect\_uri = http://127.0.0.1:65010
|
||||
- Save the file.
|
||||
- oauth\_client\_secret = { The secret }
|
||||
- oauth\_redirect\_uri = http://127.0.0.1:65010
|
||||
- Save the file.
|
||||
4. Run `python get_secret.py` in the command prompt.
|
||||
5. Your browser will open to a page on Reddit listing requested permissions.
|
||||
6. Click 'Allow'.
|
||||
|
||||
|
||||
Caveats
|
||||
-------
|
||||
## Caveats
|
||||
|
||||
- Certain limitations in the Reddit API and the PRAW library make it difficult
|
||||
to delete more than 1,000 comments. While deleting >1000 comments is planned,
|
||||
it is necessary right now to rerun the program until they are all deleted.
|
||||
- Certain limitations in the Reddit API and the PRAW library make it difficult to delete more than 1,000 comments.
|
||||
While deleting >1000 comments is planned, it is necessary right now to rerun the program until they are all deleted.
|
||||
|
||||
- We are relying on Reddit admin words that they do not store edits, deleted
|
||||
posts are still stored in the database they are merely inaccessible to the
|
||||
public.
|
||||
- We are relying on Reddit admin words that they do not store edits, deleted posts are still stored in the database
|
||||
they are merely inaccessible to the public.
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
arrow
|
||||
decorator
|
||||
arrow==0.9.0
|
||||
decorator==4.0.10
|
||||
praw==3.6.0
|
||||
PyYAML
|
||||
requests
|
||||
six
|
||||
PyYAML==3.12
|
||||
requests==2.12.1
|
||||
six==1.10.0
|
||||
backports-abc==0.4
|
||||
tornado==4.3
|
||||
update-checker==0.11
|
||||
update-checker==0.15
|
||||
wheel==0.24.0
|
||||
|
||||
@ -0,0 +1,83 @@
|
||||
---
|
||||
# Login details for Reddit. Fill out if you don't wish
|
||||
# to be prompted for a login every time you run Shreddit.
|
||||
### NOTE: This may be deprecated as you can specify in praw.ini instead
|
||||
username:
|
||||
password:
|
||||
|
||||
# How many hours of comments you want to keep
|
||||
# 24 hours in a day,
|
||||
# 168 hours in a week,
|
||||
# 672 hours in two fortnights,
|
||||
# 720 hours in a month (30 days),
|
||||
# 8766 hours in a year (365.25 days)
|
||||
hours: 24
|
||||
|
||||
# Max score, higher than this and it'll be kept.
|
||||
max_score: 100
|
||||
|
||||
# Options: new, top, controversial, more?
|
||||
sort: new
|
||||
|
||||
# Enables print statements to notify you of what's going on
|
||||
verbose: True
|
||||
|
||||
# Removes your vote before deleting the item
|
||||
clear_vote: False
|
||||
|
||||
# Options: comments, submitted, overview
|
||||
# See: https://github.com/mellort/reddit_api/blob/master/reddit/objects.py#L359
|
||||
# Overview: both submissions and comments. Comments / Submitted are as expected
|
||||
item: overview
|
||||
|
||||
# Anything in this list won't be deleted, coma delimited
|
||||
# spaces should work as .strip() is called after splitting
|
||||
# on comma.
|
||||
whitelist: [AskScience, TheCulture, redditdev, programming, charity, netsec]
|
||||
|
||||
# If you want any specific posts to be whitelisted stick 'em in here
|
||||
whitelist_ids: []
|
||||
|
||||
# If you set this then no editing or deleting will be done
|
||||
# but the output from the program will be shown as an example
|
||||
trial_run: False
|
||||
|
||||
# Don't delete but *do* edit, could prove... interesting to see a comment
|
||||
# with 5000 upvotes and it's just a lorem ipsum!
|
||||
edit_only: False
|
||||
|
||||
# Ignore distinguished comments.
|
||||
whitelist_distinguished: True
|
||||
|
||||
# Ignore gilded (gold) comments
|
||||
whitelist_gilded: True
|
||||
|
||||
# Delete everything older that this date, **This ignores whitelists**.
|
||||
# Can be used as a second deletion, as in "delete items older than 24 hours
|
||||
# except on whitelisted subreddits but after 3 months delete everything.
|
||||
nuke_hours: 720
|
||||
|
||||
# Save a copy to disk of comments and posts before deleting them.
|
||||
keep_a_copy: False
|
||||
save_directory: /tmp
|
||||
|
||||
# Replacement text format
|
||||
# Defines what you want to edit deleted content with pre-deletion (to ensure
|
||||
# it's not saved in their database).
|
||||
# Default: Random string. But this can be detected as spam in some cases.
|
||||
# options: [random, dot, "user entered string"]
|
||||
replacement_format: random
|
||||
|
||||
# Debug level, how much output you want
|
||||
# See: https://docs.python.org/3/library/logging.html#logging-levels
|
||||
debug: DEBUG
|
||||
|
||||
# Define your own wordlist to use as substitution text when
|
||||
# replacement_format == random
|
||||
wordlist: []
|
||||
|
||||
# Batch cooldown
|
||||
# This controls how long (in seconds) to wait between each set of 1000 deletions.
|
||||
batch_cooldown: 10
|
||||
|
||||
# vim: syntax=yaml ts=2
|
||||
Loading…
Reference in new issue