changedetection.io/changedetectionio/tests/test_backend.py

#!/usr/bin/python3

import time
from flask import url_for
from urllib.request import urlopen
from . util import set_original_response, set_modified_response, live_server_setup

sleep_time_for_fetch_thread = 3

# Basic test to check inscriptus is not adding return line chars, basically works etc
def test_inscriptus():
    from inscriptis import get_text
    html_content="<html><body>test!<br/>ok man</body></html>"
    stripped_text_from_html = get_text(html_content)
    assert stripped_text_from_html == 'test!\nok man'


def test_check_basic_change_detection_functionality(client, live_server):
    set_original_response()
    live_server_setup(live_server)

    # Add our URL to the import page
    res = client.post(
        url_for("import_page"),
        data={"urls": url_for('test_endpoint', _external=True)},
        follow_redirects=True
    )

    assert b"1 Imported" in res.data

    time.sleep(sleep_time_for_fetch_thread)

    # Do this a few times.. ensures we dont accidently set the status
    for n in range(3):
        client.get(url_for("api_watch_checknow"), follow_redirects=True)

        # Give the thread time to pick it up
        time.sleep(sleep_time_for_fetch_thread)

        # It should report nothing found (no new 'unviewed' class)
        res = client.get(url_for("index"))
        assert b'unviewed' not in res.data
        assert b'test-endpoint' in res.data

        # Default no password set, this stuff should be always available.

        assert b"SETTINGS" in res.data
        assert b"BACKUP" in res.data
        assert b"IMPORT" in res.data

    #####################

    # Make a change
    set_modified_response()

    res = urlopen(url_for('test_endpoint', _external=True))
    assert b'which has this one new line' in res.read()

    # Force recheck
    res = client.get(url_for("api_watch_checknow"), follow_redirects=True)
    assert b'1 watches are queued for rechecking.' in res.data

    time.sleep(sleep_time_for_fetch_thread)

    # Now something should be ready, indicated by having a 'unviewed' class
    res = client.get(url_for("index"))
    assert b'unviewed' in res.data

    # #75, and it should be in the RSS feed
    res = client.get(url_for("rss"))
    expected_url = url_for('test_endpoint', _external=True)
    assert b'<rss' in res.data

    # re #16 should have the diff in here too
    assert b'(into   ) which has this one new line' in res.data
    assert b'CDATA' in res.data
    
    assert expected_url.encode('utf-8') in res.data

    # Following the 'diff' link, it should no longer display as 'unviewed' even after we recheck it a few times
    res = client.get(url_for("diff_history_page", uuid="first"))
    assert b'Compare newest' in res.data

    time.sleep(2)

    # Do this a few times.. ensures we dont accidently set the status
    for n in range(2):
        client.get(url_for("api_watch_checknow"), follow_redirects=True)

        # Give the thread time to pick it up
        time.sleep(sleep_time_for_fetch_thread)

        # It should report nothing found (no new 'unviewed' class)
        res = client.get(url_for("index"))
        assert b'unviewed' not in res.data
        assert b'head title' not in res.data # Should not be present because this is off by default
        assert b'test-endpoint' in res.data

    set_original_response()

    # Enable auto pickup of <title> in settings
    res = client.post(
        url_for("settings_page"),
        data={"extract_title_as_title": "1", "minutes_between_check": 180, 'fetch_backend': "html_requests"},
        follow_redirects=True
    )

    client.get(url_for("api_watch_checknow"), follow_redirects=True)
    time.sleep(sleep_time_for_fetch_thread)

    res = client.get(url_for("index"))
    assert b'unviewed' in res.data
    # It should have picked up the <title>
    assert b'head title' in res.data

    #
    # Cleanup everything
    res = client.get(url_for("api_delete", uuid="all"), follow_redirects=True)
    assert b'Deleted' in res.data
hack of pytest implementation - doesnt work yet 4 years ago			`#!/usr/bin/python3`

			`import time`
Finalse pytest methods 4 years ago			`from flask import url_for`
			`from urllib.request import urlopen`
Apprise notifications (#43) * issue #4 Adding settings screen for apprise URLS * Adding test notification mechanism * Move Worker module to own class file * Adding basic notification URL runner * Tests for notifications * Tweak readme with notification info * Move notification test to main test_backend.py * Fix spacing * Adding notifications screenshot * Cleanup more files from test * Offer send notification test on individual edits and main/default * Process global notifications * All branches test * Wrap worker notification process in try/catch, use global if nothing set * Fix syntax * Handle exception, increase wait time for liveserver to come up * Fixing test setup * remove debug * Split tests into their own totally isolated setups, if you know a better way to make live_server() work, MR :) * Tidying up lint/imports 4 years ago			`from . util import set_original_response, set_modified_response, live_server_setup`
Finalse pytest methods 4 years ago
Activate workflow on all branches 4 years ago			`sleep_time_for_fetch_thread = 3`

Better text/plain detection and refactor tests (#443) 3 years ago			`# Basic test to check inscriptus is not adding return line chars, basically works etc`
			`def test_inscriptus():`
			`from inscriptis import get_text`
			`html_content="<html><body>test!<br/>ok man</body></html>"`
			`stripped_text_from_html = get_text(html_content)`
			`assert stripped_text_from_html == 'test!\nok man'`

Activate workflow on all branches 4 years ago
Minor cleanup 4 years ago			`def test_check_basic_change_detection_functionality(client, live_server):`
Finalse pytest methods 4 years ago			`set_original_response()`
Apprise notifications (#43) * issue #4 Adding settings screen for apprise URLS * Adding test notification mechanism * Move Worker module to own class file * Adding basic notification URL runner * Tests for notifications * Tweak readme with notification info * Move notification test to main test_backend.py * Fix spacing * Adding notifications screenshot * Cleanup more files from test * Offer send notification test on individual edits and main/default * Process global notifications * All branches test * Wrap worker notification process in try/catch, use global if nothing set * Fix syntax * Handle exception, increase wait time for liveserver to come up * Fixing test setup * remove debug * Split tests into their own totally isolated setups, if you know a better way to make live_server() work, MR :) * Tidying up lint/imports 4 years ago			`live_server_setup(live_server)`
Finalse pytest methods 4 years ago
			`# Add our URL to the import page`
			`res = client.post(`
			`url_for("import_page"),`
			`data={"urls": url_for('test_endpoint', _external=True)},`
			`follow_redirects=True`
			`)`
Security update - Use CSRF token protection for forms, make "remove password" use HTTP Post (#484) 3 years ago
Finalse pytest methods 4 years ago			`assert b"1 Imported" in res.data`

Add test delay for github action test 4 years ago			`time.sleep(sleep_time_for_fetch_thread)`

Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`# Do this a few times.. ensures we dont accidently set the status`
			`for n in range(3):`
			`client.get(url_for("api_watch_checknow"), follow_redirects=True)`
Finalse pytest methods 4 years ago
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`# Give the thread time to pick it up`
			`time.sleep(sleep_time_for_fetch_thread)`
Finalse pytest methods 4 years ago
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`# It should report nothing found (no new 'unviewed' class)`
			`res = client.get(url_for("index"))`
			`assert b'unviewed' not in res.data`
			`assert b'test-endpoint' in res.data`
Finalse pytest methods 4 years ago
Re #44 - Broke the menu by accident, adding tests and fixing. 4 years ago			`# Default no password set, this stuff should be always available.`

			`assert b"SETTINGS" in res.data`
			`assert b"BACKUP" in res.data`
			`assert b"IMPORT" in res.data`

Fixing a few more easy lint wins 4 years ago			`#####################`
hack of pytest implementation - doesnt work yet 4 years ago
Finalse pytest methods 4 years ago			`# Make a change`
			`set_modified_response()`
hack of pytest implementation - doesnt work yet 4 years ago
Finalse pytest methods 4 years ago			`res = urlopen(url_for('test_endpoint', _external=True))`
			`assert b'which has this one new line' in res.read()`
hack of pytest implementation - doesnt work yet 4 years ago
Finalse pytest methods 4 years ago			`# Force recheck`
			`res = client.get(url_for("api_watch_checknow"), follow_redirects=True)`
Language fix "watches are rechecking." it actually puts them into an internal queue "watches are QUEUED for rechecking" 3 years ago			`assert b'1 watches are queued for rechecking.' in res.data`
hack of pytest implementation - doesnt work yet 4 years ago
Finalse pytest methods 4 years ago			`time.sleep(sleep_time_for_fetch_thread)`
hack of pytest implementation - doesnt work yet 4 years ago
Finalse pytest methods 4 years ago			`# Now something should be ready, indicated by having a 'unviewed' class`
			`res = client.get(url_for("index"))`
			`assert b'unviewed' in res.data`
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago
Re #75 - Adding test to confirm watched URL appears in RSS feed 4 years ago			`# #75, and it should be in the RSS feed`
RSS allow access via token (#310) Allow access via a token * New RSS URL * Redirect the old RSS feed URL * fix tests 3 years ago			`res = client.get(url_for("rss"))`
Re #75 - Adding test to confirm watched URL appears in RSS feed 4 years ago			`expected_url = url_for('test_endpoint', _external=True)`
			`assert b'<rss' in res.data`
RSS feed CDATA should contain difference output 3 years ago
			`# re #16 should have the diff in here too`
			`assert b'(into ) which has this one new line' in res.data`
			`assert b'CDATA' in res.data`

Re #75 - Adding test to confirm watched URL appears in RSS feed 4 years ago			`assert expected_url.encode('utf-8') in res.data`

Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`# Following the 'diff' link, it should no longer display as 'unviewed' even after we recheck it a few times`
Activate workflow on all branches 4 years ago			`res = client.get(url_for("diff_history_page", uuid="first"))`
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`assert b'Compare newest' in res.data`

Add wait for threads in test 4 years ago			`time.sleep(2)`

Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`# Do this a few times.. ensures we dont accidently set the status`
Activate workflow on all branches 4 years ago			`for n in range(2):`
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`client.get(url_for("api_watch_checknow"), follow_redirects=True)`

			`# Give the thread time to pick it up`
			`time.sleep(sleep_time_for_fetch_thread)`

			`# It should report nothing found (no new 'unviewed' class)`
			`res = client.get(url_for("index"))`
			`assert b'unviewed' not in res.data`
Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago			`assert b'head title' not in res.data # Should not be present because this is off by default`
Extending tests to cover resetting the diff/unviewed status correctly 4 years ago			`assert b'test-endpoint' in res.data`
Add wait for threads in test 4 years ago
			`set_original_response()`

Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago			`# Enable auto pickup of <title> in settings`
			`res = client.post(`
			`url_for("settings_page"),`
Chrome/Webdriver support for Javascript websites (#114) JS Support via fetching the page over WebDriver/Selenium network Refactor forms (Split into logical tabs) 3 years ago			`data={"extract_title_as_title": "1", "minutes_between_check": 180, 'fetch_backend': "html_requests"},`
Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago			`follow_redirects=True`
			`)`

Add wait for threads in test 4 years ago			`client.get(url_for("api_watch_checknow"), follow_redirects=True)`
			`time.sleep(sleep_time_for_fetch_thread)`
Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago
Add wait for threads in test 4 years ago			`res = client.get(url_for("index"))`
Activate workflow on all branches 4 years ago			`assert b'unviewed' in res.data`
Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago			`# It should have picked up the <title>`
			`assert b'head title' in res.data`
Activate workflow on all branches 4 years ago
Auto extract html title as title (#102) * Auto extract <title> as watch title, Minor refactor for html tooling 4 years ago			`#`
Move logic 4 years ago			`# Cleanup everything`
Clean up after test case 4 years ago			`res = client.get(url_for("api_delete", uuid="all"), follow_redirects=True)`
			`assert b'Deleted' in res.data`
Password protection / login support (#34) Issue #24 Password login hashlib.pbkdf2_hmac implementation 4 years ago