Commit Graph

583 Commits (5104030aef8d459c745a0a37d4da2ccade43c90b)

Author SHA1 Message Date
dgtlmoon 4841c79b4c Adding extra check when updating DB on ReplyWithContentButNoText
3 years ago
dgtlmoon 2ba00d2e1d
Notifications log - log what was sent after applying all cleanups
3 years ago
dgtlmoon 19c96f4bdd Re #555 - tgram:// notifications - strip added HTML tag which is not supported by Telegram
3 years ago
dgtlmoon 82b900fbf4 Give more helpful error message when a page doesnt load
3 years ago
dgtlmoon 358a365303 Tweaks to playwright fetch code - better timeout handling
3 years ago
dgtlmoon a07ca4b136
Re #580 - New functionality - Random "jitter" delay to requests (#681)
3 years ago
dgtlmoon ba8cf2c8cf 0.39.15
3 years ago
dgtlmoon 3106b6688e Watch overview list - adding spinner to make it easier to see whats currently being 'Checked'
3 years ago
dgtlmoon 2c83845dac
Preview section - add helpful check
3 years ago
dgtlmoon 111266d6fa Send test notification - improved handling of errors
3 years ago
dgtlmoon ead610151f
Notification log - also log normal requests and make the log easier to find
3 years ago
dgtlmoon 327cc4af34 Use correct RSS CDATA handling (#662)
3 years ago
dgtlmoon 6008ff516e
Improve logging (#671)
3 years ago
dgtlmoon cdcf4b353f
New [scrub] button when editing a watch - scrub single watch history (#672)
3 years ago
dgtlmoon 1ab70f8e86 Diff + Preview - Hide date selector widget when viewing screenshots as its not yet possible to compare screenshots (but will be soon!)
3 years ago
dgtlmoon 8227c012a7 Diff + Preview - Fixing screenshot behaviour after preference change
3 years ago
dgtlmoon c113d5fb24 Screenshot handling on the diff/preview section refactor (#630)
3 years ago
dgtlmoon 8c8d4066d7 Shared watches - include "Extract text" filter
3 years ago
dgtlmoon 277dc9e1c1
Improve error message when filter not found in page result (#666)
3 years ago
dgtlmoon fc0fd1ce9d "Extract text" filter - improve placeholder example
3 years ago
dgtlmoon bd6127728a Visual selector - 'clear selection' button should clear the filter also
3 years ago
dgtlmoon 4101ae00c6
New feature - "Extract text" filter ability (#624)
3 years ago
dgtlmoon 62f14df3cb
Fixing RSS feed HTML content formatting (#662)
3 years ago
dgtlmoon 7929aeddfc
'Mark all viewed' button was missing in this version, added test also. (#652)
3 years ago
dgtlmoon 8294519f43 Content fetcher - Handle when a page doesnt load properly
3 years ago
dgtlmoon 8ba8a220b6 Playwright - Correctly close browser context/sessions on exceptions
3 years ago
dgtlmoon aa3c8a9370
Move history data to a textfile, improves memory handling (#638)
3 years ago
dgtlmoon 329c7620fb Remove UK Covid news
3 years ago
Amos (LFlare) Ng 1f974bfbb0
Visual Selector fix: Firefox compatibility - Visual Selector (#646)
3 years ago
Tim Loderhose 437c8525af
Remove group tag arbitrary length limit (#645)
3 years ago
dgtlmoon a2a1d5ae90 Distill.io import bug fix when no tags assigned to a watch (#557)
3 years ago
dgtlmoon 2566de2aae Ignore whitespace on by default
3 years ago
dgtlmoon dfec8dbb39 Visual Selector - clear events when changing tabs
3 years ago
dgtlmoon 5cefb16e52 Minor code cleanup
3 years ago
dgtlmoon 341ae24b73 Re #616 - content trigger - adding extra test (#620)
3 years ago
dgtlmoon 9d742446ab Playwright - ByPass CSP for more reliable JS scraping, disable accept downloads
3 years ago
dgtlmoon e3e022b0f4 VisualSelector - Better handling of filter targets that are no longer available in the HTML
3 years ago
dgtlmoon 7983675325 Visual Selector - be more resilient when sites interfere with the xPath scraping
3 years ago
dgtlmoon eef56e52c6 Adding new Visual Selector for choosing the area of the webpage to monitor - playwright/browserless only (#566)
3 years ago
dgtlmoon 8e3195f394 0.39.14
3 years ago
dgtlmoon e17c2121f7 Fix encoding errors with XPath filters from UTF-8 responses (#619)
3 years ago
dgtlmoon 07e279b38d
API Interface (#617)
3 years ago
dgtlmoon 2c834cfe37 Add note that changedetection is not performed on the screenshot just yet (WIP https://github.com/dgtlmoon/changedetection.io/pull/419 )
3 years ago
dgtlmoon dbb5c666f0 Fixing edit template HTML
3 years ago
dgtlmoon 70b3493866
Proxy settings on watch should have a "[ ] default" option (#610)
3 years ago
dgtlmoon 3b11c474d1
Input field tidyup (#611)
3 years ago
dgtlmoon 890e1e6dcd Update wiki link for 'More info' about sharing a watch and its configuration
3 years ago
dgtlmoon 6734fb91a2
Option to control if pages with no renderable content are a change (example: JS webapps that dont render any text sometimes) (#608)
3 years ago
dgtlmoon 16809b48f8 Playwright - raise EmptyReply on empty reply, no need to process further
3 years ago
dgtlmoon 67c833d2bc
Re #214 - configurable wait extra seconds for webdriver requests before extracting text (#606)
3 years ago
weeix 31fea55ee4
Fix PLAYWRIGHT_DRIVER_URL default value (cf. #587) (#599)
3 years ago
dgtlmoon 034507f14f
Fixing Pip install problem - Update MANIFEST to include model/ subdir, improving imports (#593)
3 years ago
dgtlmoon 0e385b1c22 0.39.13
3 years ago
dgtlmoon f28c260576
Distill.io JSON export file importer (#592)
3 years ago
dgtlmoon 18f0b63b7d
Ability to specify a list of proxies to choose from, always using the first one by default, See wiki (#591)
3 years ago
dgtlmoon 9807cf0cda Playwright - code fix
3 years ago
dgtlmoon d4b5237103 Playwright fetcher - more reliable by just waiting arbitrary seconds after the last network IO
3 years ago
dgtlmoon dc6f76ba64
Make proxy configuration more consistent - see https://github.com/dgtlmoon/changedetection.io/wiki/Proxy-configuration (#585)
3 years ago
dgtlmoon 1f2f93184e Playwright fetcher - use the correct default User-Agent
3 years ago
dgtlmoon 0f08c8dda3
Toggle visibility of extra requests options/settings when not in use (#584)
3 years ago
dgtlmoon 68db20168e
Add new fetch method: Playwright Chromium (Selenium/WebDriver alternative) (#489)
3 years ago
dgtlmoon 1d4474f5a3
Simplify scrub operation (simply cleans all) (#575)
3 years ago
dgtlmoon 613308881c
Bugfix - dont update record when deleted during check
3 years ago
dgtlmoon c0d0424e7e
Data storage bug fix #569
3 years ago
dgtlmoon 06517bfd22
Ability to 'Share' a watch by a generated link, this will include all filters and triggers - see Wiki (#563)
3 years ago
dgtlmoon ffc4215411 Unify MINIMUM_SECONDS_RECHECK_TIME env var variable to 60 seconds
3 years ago
dgtlmoon 9e708810d1
Seconds/minutes/hours/days between checks form field upgrade from 'minutes' only (#512)
3 years ago
dgtlmoon 1e8aa6158b Form styling improvements
3 years ago
dgtlmoon 015353eccc Form field handling improvements - fixing field list handler for empty lines
3 years ago
dgtlmoon 501183e66b Fix "Add email" button in main global notification settings
3 years ago
dgtlmoon def74f27e6 Test notification button fixed in main settings (#556)
3 years ago
dgtlmoon 37775a46c6 tgram:// be sure total notification size is always under their 4096 size limit
3 years ago
dgtlmoon e4eaa0c817
Shows which items are already in the queue, disables adding to the queue if already in the recheck queue (#552)
3 years ago
dgtlmoon 9e71f2aa35 Discord:// notification size limit - also includes the notification title
3 years ago
dgtlmoon 2a9fb12451
Import speed improvements, and adding an import URL batch size of 5,000 to stop accidental CPU overload (#549)
3 years ago
dgtlmoon 6c3c5dc28a Ability to set the default fetch mode via the DEFAULT_FETCH_BACKEND variable
3 years ago
dgtlmoon 8f062bfec9
Refactor form handling (#548)
3 years ago
dgtlmoon 380c512cc2
Adding support for change detection of HTML source-code via "source:https://website.com" prefix (#540)
3 years ago
dgtlmoon d7ed7c44ed Re-label the quick-add widget placeholder 'tag' to 'watch group'
3 years ago
dgtlmoon 34a87c0f41
HTTP Fetcher code improvements
3 years ago
dgtlmoon 4074fe53f1 Adding RSS metadata auto-discovery
3 years ago
Tristan Hill 44d599d0d1
Upgrade WTforms form handler to v3 (#523)
3 years ago
dgtlmoon 615fe9290a 0.39.12
3 years ago
dgtlmoon 2cc6955bc3
Miscellaneous settings form visual improvements (#535)
3 years ago
dgtlmoon 9809af142d
Option to render links as [Some Text ](/link), adds the ability to change-detect on hyperlink changes
3 years ago
dgtlmoon 1890881977 Specify our Discord avatar_url as default avatar_url
3 years ago
dgtlmoon 9fc2fe85d5 Minor git updates
3 years ago
dgtlmoon bb3c546838 Fix screenshot tab name
3 years ago
dgtlmoon 165f794595
Discord:// notifications should be cut to 2000 chars or Discord will not process them. (#531 + #323)
3 years ago
dgtlmoon a440eece9e Make long reports in the notification error log easier to read
3 years ago
dgtlmoon 34c83f0e7c
[Add email] button in notification settings with a prefix set from NOTIFICATION_MAIL_BUTTON_PREFIX env variable when defined. (#528)
3 years ago
dgtlmoon 63e91a3d66 Skip processing a watch into the RSS feed if there's not enough data to examine (fixes Internal Server Error when accessing the RSS feed) (#521)
3 years ago
dgtlmoon 3034d047c2
Introduce an AJAX button for sending test notifications instead of the checkbox (#519)
3 years ago
dgtlmoon 2620818ba7 Make text tab always available at default
3 years ago
dgtlmoon 9fe4f95990
When fetching a snapshot via Chrome, make the most recent screenshot available on the Diff and Preview pages (#516)
3 years ago
dgtlmoon ffd2a89d60
Remove 'unviewed' status in watch table when Diff link clicked (#514)
3 years ago
dgtlmoon 8f40f19328
RSS feed CDATA should contain difference output
3 years ago
dgtlmoon 082634f851
Fix - {diff} and {diff_full} notifications tokens were not always including the full output
3 years ago
dgtlmoon c79d6824e3
Minor UI cleanups (mobile tabs, font sizing) (#503)
3 years ago
zznidar 946377d2be Fix typo in Filters & Triggers settings. (#495)
3 years ago
zznidar 5db9a30ad4 Add autofocus attribute to password login field (#496)
3 years ago
dgtlmoon 1d060225e1 0.39.11
3 years ago
dgtlmoon 7e0f0d0fd8
Microsoft Windows installation fixes (#492)
3 years ago
dgtlmoon 942c3f021f
Allow changedetector to ignore status codes as a per-site setting (#479) (#485)
3 years ago
dgtlmoon 5483f5d694
Security update - Use CSRF token protection for forms, make "remove password" use HTTP Post (#484)
3 years ago
dgtlmoon f2fa638480
Security update - Protect against file:/// type access by webdriver/chrome. (#483)
3 years ago
dgtlmoon 3e8a15456a
Detect byte-encoding when the server mishandles the content-type header reply (#472)
3 years ago
dgtlmoon 2a03f3f57e Improving form/edit example markup
3 years ago
dgtlmoon ffad5cca97
JSON diff/preview should use utf-8 encoding where possible (#465)
3 years ago
Tim Loderhose 60a9a786e0
Fix typo in settings form
3 years ago
dgtlmoon c25294ca57 0.39.10
3 years ago
Tim Loderhose d4359c2e67
Add filter to remove elements by CSS rule from HTML before change detection is run (#445)
3 years ago
dgtlmoon 44fc804991
Minor updates to filters form text
3 years ago
dgtlmoon 7ce9e4dfc2
Testing - Refactor HTTP Request Type test (#453)
3 years ago
dgtlmoon 3cc6586695 Make table header font size the same as content
3 years ago
dgtlmoon 09204cb43f Adjust background colours
3 years ago
dgtlmoon a709122874
Handle the case where the visitor is already logged-in and tries to login again (#447)
3 years ago
dgtlmoon efbeaf9535 Make the Request Override settings easier to understand
3 years ago
dgtlmoon 1a19fba07d Minor tweak to notification token table
3 years ago
dgtlmoon eb9020c175 Style tweak to watch form
3 years ago
dgtlmoon 13bb44e4f8 Login form style fixes
3 years ago
dgtlmoon 69aec23d1d Style fix for background image relative to X-Forwarded-Prefix when running via reverse proxy subdirectory
3 years ago
dgtlmoon f85ccffe0a Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon 0005131472 Re-arranging primary links so the important ones are easier to find on mobile
3 years ago
dgtlmoon 3be1f4ea44
Set authentication cookie path relative to X-Forwarded-Prefix when running via reverse proxy subdirectory (#446)
3 years ago
dgtlmoon 96664ffb10
Better text/plain detection and refactor tests (#443)
3 years ago
dgtlmoon 615fa2c5b2
Tweak support tabs and text (#440)
3 years ago
dgtlmoon fd45fcce2f
Include link to changedetection.io hosted option (#439)
3 years ago
dgtlmoon 75ca7ec504
Improved CPU usage around the loop responsible for what sites needs to be checked
3 years ago
dgtlmoon 883aa968fd 0.39.9
3 years ago
dgtlmoon 3240ed2339
Minor reliability upgrade for large datasets - retry deepcopy (#436)
3 years ago
dgtlmoon a89ffffc76
"Recheck" button should work when entry is in paused state
3 years ago
dgtlmoon fda93c3798 Better file exception handling on saving index JSON
3 years ago
dgtlmoon a51c555964 Fix small issue in highlight trigger/ignore preview page with setting the background colours, add test
3 years ago
dgtlmoon b401998030 Ensure string matching on the ignore filter is always case-INsensitive
3 years ago
dgtlmoon 014fda9058
Ability to visualise trigger and filter rules against the current snapshot on the preview page
3 years ago
Michael 85715120e2
XPath RegularExpression support
3 years ago
dgtlmoon a0e4f9b88a better checking of JSON type
3 years ago
dgtlmoon 04bef6091e
Make system level errors from the HTTP fetchers easier to find (#421)
3 years ago
dependabot[bot] 536948c8c6
Bump node-sass from 6.0.1 to 7.0.0 in /changedetectionio/static/styles (#415)
3 years ago
dgtlmoon d4f4ab306a
Dont allow redirect on login, it's safer and more reliable this way (#414)
3 years ago
dgtlmoon 8d2e240a2a When using Env. FETCH_WORKERS or WEBDRIVER_DELAY_BEFORE_CONTENT_READY , it should be type int
3 years ago
dgtlmoon d7ed479ca2 0.39.8
3 years ago
dgtlmoon f25cdf0a67
Number of fetching workers can be overriden by Env "FETCH_WORKERS" (#413)
3 years ago
dgtlmoon 5214a7e0f3 Adding Env var "WEBDRIVER_DELAY_BEFORE_CONTENT_READY" to wait n seconds before extracting the text from the browser
3 years ago
dgtlmoon eb3dca3805 Language fix "watches are rechecking." it actually puts them into an internal queue "watches are QUEUED for rechecking"
3 years ago
dgtlmoon a580c238b6 Use flask url_for() for webdriver chrome icon instead of relative path
3 years ago
dgtlmoon 22ef9afb93
Refactor tests for notification error log handler (#404)
3 years ago
dgtlmoon abaec224f6
Notification error log handler (#403)
3 years ago
dgtlmoon 14db60e518 Add notification note - tgram:// bots cant send messages to other bots, so you should specify chat ID of non-bot user.
3 years ago
dgtlmoon 8e54a17e14 /preview format doesnt need <pre> - fixing too many returnlines in content on diff/preview page
3 years ago
dgtlmoon 73101a47e7
Ability to use a generated salted password in deployments as env var SALTED_PASS (#397)
3 years ago
dgtlmoon 03f776ca45
#323 Adding note about discord:// 2000 char limit (#392)
3 years ago
dgtlmoon 39b7be9e7a
plaintext mime type fix - Don't attempt to extract HTML content from plaintext, this will remove lines and break changedetection (#391)
3 years ago
dgtlmoon 6611823962 Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon c1c453e4fe .add_watch() can accept empty tag
3 years ago
Tim Loderhose 4887180671
Add option for tags on import (#377)
3 years ago
Travis Howse abe88192f4
Fix bug where diff and diff_full were switched in notification templates. (#380)
3 years ago
dgtlmoon bbc2875ef3 0.39.7
3 years ago
dgtlmoon b7ca10ebac
Scrub watch snapshot fixes
3 years ago
dgtlmoon a896493797
Simple HTTP auth (#372)
3 years ago
dgtlmoon e5fe095f16 Adding note about JS pages
3 years ago
dgtlmoon 271181968f
Notification settings defaults and validation (#361)
3 years ago
dgtlmoon 8206383ee5 Filters settings helper text tidy-up
3 years ago
dgtlmoon ecfc02ba23 Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon 3331ccd061 Add test for low-level network error text handling
3 years ago
Unpublished bd8f389a65
Add API endpoint for current snapshot (#359)
3 years ago
dgtlmoon bc74227635 Clarify notice/messages around changing ignore text
3 years ago
dgtlmoon 07c60a6acc Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon 7916faf58b 0.39.6
3 years ago
dgtlmoon febb2bbf0d
Heroku tweaks (backup download) (#356)
3 years ago
dgtlmoon 59d31bf76f
XPath support (#355)
3 years ago
dgtlmoon f87f7077a6
Better handling of EmptyReply exception, always bump 'last_checked' in the case of an error (#354)
3 years ago
dgtlmoon b33105d576 Re #348 - Add test for backup, use proper datastore path
3 years ago
Unpublished 023951a10e
Be sure that documents returned with a application/json header are not parsed with inscriptis (#337)
3 years ago
dgtlmoon fbd9ecab62
Re #340 - snapshot should not be modified by ignore text (#344)
3 years ago
dgtlmoon b5c1fce136
Re #133 Option for ignoring whitespacing (#345)
3 years ago
dgtlmoon 489671dcca
Re #342 notification encoding (#343)
3 years ago
dgtlmoon 0439acacbe
Adding global ignore text (#339)
3 years ago
dgtlmoon 735fc2ac8e Adding new proxyType to selenium mappings
3 years ago
Calvin Bui feb8e6c76c
Add socksVersion mapping (#331)
3 years ago
dgtlmoon a37a5038d8 Fix broken RSS link fields
3 years ago
dgtlmoon f1933b786c RSS Link links you back to the difference UI/JS page, RSS Description is the page you're watching, and RSS Title is the page you're watching
3 years ago
dgtlmoon d6a6ef2c1d Unify Filters and Triggers tabs into a single tab
3 years ago
dgtlmoon cf9554b169 Move 'request type' field to the new 'Requests' tab
3 years ago
dgtlmoon d602cf4646 Aligning call signatures #325
3 years ago
Simon Caron dfcae4ee64
Extend Request Parameters to add Body & Method (#325)
3 years ago
dgtlmoon 8ec17a4c83
Re #267 - Pass settings for the proxy setup for webdriver (#326)
3 years ago
dgtlmoon 43c9cb8b0c 0.39.5
3 years ago
dgtlmoon ce383bcd04 W3C HTML validation issue around RSS icon
3 years ago
dgtlmoon 0b0cdb101b Closes #323 adds link to wiki
3 years ago
dgtlmoon 20647ea319 improve theming docs
3 years ago
dgtlmoon 9815fc2526
RSS allow access via token (#310)
3 years ago
dgtlmoon 35d0c74454
Re #308 - Adding test and including settings in clone operation (#309)
3 years ago
Glassed Silver dd450b81ad
fixing too small font in diff UI (#260)
3 years ago
dgtlmoon 5a10acfd09
Send diff in notifications (#296)
3 years ago
dgtlmoon a7c09c8990 Fix scrub form theme
3 years ago
dgtlmoon 9235eae608 Scrub dates: Fix date regex limit handler parsing
3 years ago
dgtlmoon 5bbd82be79 Wait 60 seconds or until stop_thread is set
3 years ago
dgtlmoon 7f8c0fb2fa
Check that a notification URL is set when sending the test notification (#300)
3 years ago
Tristan Hill 489eedf34e
Flask 2 (#299)
3 years ago
dgtlmoon 3956b3fd68
Re #269 - Show current/correct BASE_URL information (#271)
3 years ago
dgtlmoon 61c1d213d0 Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon e07f573f64 Re #269 - Fix env var comment name
3 years ago
ghjklw ecba130fdb
Enable Markdown and HTML notifications. (#288)
3 years ago
dgtlmoon ff6dc842c0 0.39.4 release
3 years ago
dgtlmoon 4659993ecf
Re #286 - Solving lost data/corrupted data - Tweak timing and try to write to a temp file first (#292)
3 years ago
jeremysherriff 0a29b3a582
Fix element paths when using reverse proxy subfolder (#272)
3 years ago
dgtlmoon c55bf418c5 0.39.3 release
3 years ago
dgtlmoon 4bbb7d99b6 Re #264 - fixing clone watch operation
3 years ago
dgtlmoon a8e92e2226
Re #265 - extended jsonpath support (#266)
3 years ago
dgtlmoon c17327633f Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon 56d1dde7c3 Re #265 - wasnt catching the jsonpath exception due to invalid jsonpath expressions properly
3 years ago
dgtlmoon 6e4ddacaf8
Re #257 - Handle bool val of json path better (#263)
3 years ago
dgtlmoon 5d5ceb2f52 Form helper - explain where the webdriver setting comes from
3 years ago
dgtlmoon 2dc43bdfd3 version 0.39.2
3 years ago
dgtlmoon 95e39aa727
Configurable BASE_URL (#228)
3 years ago
dgtlmoon cd7df86f54
Re #242 - app was treating notification field defaults as the field value (#244)
3 years ago
dgtlmoon ca0ab50c5e
Re #239 - Individual GUID for watch+changeevent (#241)
3 years ago
dgtlmoon fe59ac4986
Re #232 - Use a copy of the datastore incase it changes while we iterate through it (#234)
3 years ago
dgtlmoon 25476bfcb2
Setting for Extract <title> as title option on individual watches (#229)
3 years ago
dgtlmoon 7618081e83 v0.39.1
3 years ago
dgtlmoon b903280cd0
Re #185 - [feature] Custom notifications templates per watch (#226)
3 years ago
dependabot[bot] dfd34d2a5b Bump tar from 6.1.6 to 6.1.9 in /changedetectionio/static/styles (#209)
3 years ago
dgtlmoon 98f6f0c80d Re #225 - Notifications refactor token replacement fix possible missing value for watch_title
3 years ago
Matthias Langhard 05f7e123ed
Adds 'Create Copy' feature to clone a watch (#184)
3 years ago
Minty b9e0ad052f
New notification tokens - watch_uuid, watch_title, watch_tag, (#201)
3 years ago
dgtlmoon f8937e437a Merge branch 'master' of github.com:dgtlmoon/changedetection.io
3 years ago
dgtlmoon fbe9270528
Re #203 - validate tokens (#204)
3 years ago
dgtlmoon 58c3bc371d No point hiding the notifications customisation area because it's now in its own tab
3 years ago
dgtlmoon 5fb9bbdfa3 Test - prove that notifications are not being sent when content does not change
3 years ago
dgtlmoon 4ed5d4c2e7 WebDriver fetcher - settings - when an alternative one is configured, show it in the label
3 years ago
dgtlmoon a11b6daa2e
Installation via pip (#186)
3 years ago