Commit Graph

468 Commits (aaad71fc19b09d756f496dab069cd1143e8e2468)

Author SHA1 Message Date
dgtlmoon 77fdf59ae3 Improve Proxy minimum time debug output
2 years ago
dgtlmoon 0e194aa4b4 Default proxy settings fixes
2 years ago
dgtlmoon 2ba55bb477
Use proxies.json instead of proxies.txt - see wiki Proxies section (#945)
2 years ago
dgtlmoon 22638399c1 0.39.19.1
2 years ago
dgtlmoon e3381776f2 Notification - code tidyup
2 years ago
dgtlmoon 26e2f21a80 Watch list & notification - Adding extra list batch operations for Mute, Unmute, Reset-to-default
2 years ago
dgtlmoon b6009ae9ff Notification - Reset defaults button should be on edit page only
2 years ago
dgtlmoon b046d6ef32 Notification watch settings - add button to make watch use defaults (empties the settings)
2 years ago
dgtlmoon e154a3cb7a Notification system update - set watch to use defaults if it is the same as the default
2 years ago
dgtlmoon 434c5813b9 0.39.19
2 years ago
dgtlmoon bd0fbaaf27
Use play and pause separate icons (#919)
2 years ago
dgtlmoon 0c111bd9ae
Further notification settings refinement (#910)
2 years ago
dgtlmoon ed9ac0b7fb
Reliability improvement - Check watch UUID exists when reporting missing path (#915)
2 years ago
dgtlmoon fefc39427b
Test improvement - Visual selector data loads as JSON (#895)
2 years ago
dgtlmoon 2c6faa7c4e
Cleaner separation of watch/global notification settings (#894)
2 years ago
dgtlmoon 6168cd2899
Code maintenance - Removing old function (#875)
2 years ago
dgtlmoon f3c7c969d8 Show screenshot age in [preview]
2 years ago
dgtlmoon 67d2441334 0.39.18
2 years ago
dgtlmoon 3c30bc02d5
More data saving pre-checks (#863)
2 years ago
dgtlmoon b1e32275dc Checkbox operations - reorder buttons for safety
2 years ago
dgtlmoon e2a6865932
UI feature - Basic checkbox/group operations (#861)
2 years ago
dgtlmoon f04adb7202
Bug fix - automatically queued watch checks weren't always being processed sequentially
2 years ago
dgtlmoon 1193a7f22c Playwright - Support proxy auth mechanisms (#859)
2 years ago
dgtlmoon 5494e61a05 Skip processing when watch was deleted
2 years ago
dgtlmoon e461c0b819
Playwright fetcher didn't report low level HTTP errors correctly (like Connection Refused) (#852)
2 years ago
dgtlmoon d67c654f37
Be sure visual-selector data is set when xPath/CSS filter is not yet found (#851)
2 years ago
dgtlmoon 06ab34b6af Visual selector data not being saved by refactor
2 years ago
dgtlmoon ba8676c4ba
'Save chrome screenshot' checkbox never used, removing, we always save the screenshot. (#844)
2 years ago
dgtlmoon 4899c1a4f9
Crash fix: Data store sub-directories werent always being created when needed (#842)
2 years ago
dgtlmoon 9bff1582f7
Make the table header easier to understand when sorting (#840)
2 years ago
dgtlmoon 269e3bb7c5
Column sorting (#838)
2 years ago
dgtlmoon 1f250aa868 Revert "don't process paused entries after queue", so we can still manually recheck a paused watch
2 years ago
dgtlmoon 1c08d9f150
Remove 'last-changed' from url-watches.json and always calculate from history index (#835)
2 years ago
dgtlmoon 9942107016
Massive improvements to error handling - show separate output for non HTTP 200 status replies
2 years ago
dgtlmoon 1eb5726cbf Execute JS should happen after waiting seconds
2 years ago
dgtlmoon f82d3b648a
Crash protection - handle the case where watch was deleted while being checked (#833)
2 years ago
dgtlmoon 034b1330d4
Don't process a watch if it was paused after being queued (#825)
2 years ago
dgtlmoon 4026575b0b 0.39.17.2
2 years ago
dgtlmoon 8c466b4826 Test fix - Remove debug from test
2 years ago
dgtlmoon 6f072b42e8
Security update - Password could be unset from settings form unexpectedly (#808)
2 years ago
dgtlmoon e318253f31 Disable SIGCHLD Handler for now - keeping SIGTERM for DB writes
2 years ago
dgtlmoon f0f2fe94ce
Handle SIGTERM for cleaner shutdowns (#737)
2 years ago
dgtlmoon 26f5c56ba4
Remove [save & preview] button, the preview is not updated live so it can lead to confusion (#801)
2 years ago
dgtlmoon a1c3107cd6
Feature - priority queue - edited and added watches should get checked before automatically queued watches (#799)
2 years ago
dgtlmoon 8fef3ff4ab
[preview current] cleanup code and add test
2 years ago
dgtlmoon baa25c9f9e
Feature - mute notifications (#791)
2 years ago
dgtlmoon 488699b7d4 Test improvement - remove unnecessary step
2 years ago
dgtlmoon cf3a1ee3e3 0.39.17.1
2 years ago
dgtlmoon daae43e9f9
Bug fix: Filter failure detection notification was interfering with change-detection results, added test case (#786)
2 years ago
dgtlmoon 3c9d2ded38 0.39.17
2 years ago
dgtlmoon 9f4364a130 Add https://discord.com/api notification hook to the automatic truncation due to Discords 2000 char limit
2 years ago
dgtlmoon 5bd9eaf99d
UI Feature - Add watch in "paused" state, saving then unpauses (#779)
2 years ago
dgtlmoon b1c51c0a65
Enhancement - support xPath text() function filter, for example "//title/text()" in RSS feeds (#778)
2 years ago
dgtlmoon 232bd92389
Bug fix - Filter "Only trigger when new lines appear" should check all history, not only the first item (#777)
2 years ago
dgtlmoon e6173357a9 Visual Selector direct element finder fix
2 years ago
dgtlmoon 1f27865fdf Filter failure notification send default enable now controlled by setting Env var
2 years ago
dgtlmoon faa42d75e0 Refactor of extract text filter - Regex, support Regex (groups) and all python regex flags via /something/aiLmsux (#773)
2 years ago
dgtlmoon 291700554e
Bug fix for alerting when xPath based filters are no longer present (#772)
2 years ago
dgtlmoon a82fad7059 Send notification when CSS/xPath filter is missing after more than 6 (configurable) attempts (#771)
2 years ago
dgtlmoon c2fe5ae0d1 mailto plaintext handling fix for 'plaintext' apprise integration
2 years ago
dgtlmoon 5beefdb7cc
Minor code cleanups
2 years ago
dgtlmoon 872bbba71c
Notifications - email - Correctly send plaintext notification email with plaintext header (#767)
2 years ago
Jonathon Sisson d578de1a35
Form text tweak - Regex clarification (#766)
2 years ago
dgtlmoon dd0eeca056 Handle simple obfuscations - HomeDepot.com style price obfuscation (#764)
2 years ago
Brandon Wees ace44d0e00
Notifications fix - Discord - added discord webhook base url to truncation rules (#753)
2 years ago
dgtlmoon 52d3d375ba removing package-lock.json - not required to be in git
2 years ago
dgtlmoon 08117089e6 Share-icon cleanups
2 years ago
dgtlmoon 2ba3a6d53f Test improvement: Extract text should return all matches
2 years ago
dgtlmoon 2f636553a9 Bug fix: RSS Feed should also announce utf-8 charset
2 years ago
Freddie Leeman 0bde48b282
Regex extract filter: Return all regex results instead of first match (#730)
2 years ago
dgtlmoon fae1164c0b
Ability to specify JS before running change-detection (#744)
2 years ago
dgtlmoon 169c293143 Playwright - log console errors to output
2 years ago
dgtlmoon 46cb5cff66 UI Improvement - Clarifying "Visual Filter" tool as "Visual Selector Filter"
2 years ago
Simo Elalj 05584ea886 Use environment variables to override new watch settings defaults (user-agent, timeout, workers) (#742)
2 years ago
dgtlmoon 15569f9592 0.39.16
2 years ago
dgtlmoon 5f9e475fe0
Fix notification apprise application name to changedetection.io #731
2 years ago
dgtlmoon 2b054ced8c
[new filter] Filter option - Trigger only when NEW content (lines) are detected ( compared to earlier text snapshots ) (#685)
2 years ago
dgtlmoon 6553980cd5
Playwright - Use HTTP Request Headers override (Cookie, etc)
2 years ago
jtagcat 7c12c47204
lang: prefer 'clear (snap) history' to 'scrub' (#721)
2 years ago
dgtlmoon dbd9b470d7
Minor diff page improvements - list should be sorted 'newest first' and no need to include the current version to compare against (#716)
2 years ago
dgtlmoon 83555a9991
bug fix: `last_changed` was being set on the first fetch, should only be set on the change after the first fetch #705
2 years ago
dgtlmoon 7da32f9ac3 New filter - Block change-detection if text matches - for example, block change-detection while the text "out of stock" is on the page, know when the text is no longer on the page (#698)
2 years ago
dgtlmoon 76996b9eb8 Some changes werent getting triggered because the previous checksum only recorded when an event occured
2 years ago
dgtlmoon fba2b1a39d Notifications regression bug in 0.39.15 - only sent the first notification URL
2 years ago
dgtlmoon 4a91505af5 Playwright screenshots - no need for high-res "bug workaround" screenshot, use lower quality/faster configurable image quality env var
2 years ago
dgtlmoon 4841c79b4c Adding extra check when updating DB on ReplyWithContentButNoText
2 years ago
dgtlmoon 2ba00d2e1d
Notifications log - log what was sent after applying all cleanups
2 years ago
dgtlmoon 19c96f4bdd Re #555 - tgram:// notifications - strip added HTML tag which is not supported by Telegram
2 years ago
dgtlmoon 82b900fbf4 Give more helpful error message when a page doesnt load
2 years ago
dgtlmoon 358a365303 Tweaks to playwright fetch code - better timeout handling
2 years ago
dgtlmoon a07ca4b136
Re #580 - New functionality - Random "jitter" delay to requests (#681)
2 years ago
dgtlmoon ba8cf2c8cf 0.39.15
2 years ago
dgtlmoon 3106b6688e Watch overview list - adding spinner to make it easier to see whats currently being 'Checked'
2 years ago
dgtlmoon 2c83845dac
Preview section - add helpful check
2 years ago
dgtlmoon 111266d6fa Send test notification - improved handling of errors
2 years ago
dgtlmoon ead610151f
Notification log - also log normal requests and make the log easier to find
2 years ago
dgtlmoon 327cc4af34 Use correct RSS CDATA handling (#662)
2 years ago
dgtlmoon 6008ff516e
Improve logging (#671)
2 years ago
dgtlmoon cdcf4b353f
New [scrub] button when editing a watch - scrub single watch history (#672)
2 years ago
dgtlmoon 1ab70f8e86 Diff + Preview - Hide date selector widget when viewing screenshots as its not yet possible to compare screenshots (but will be soon!)
2 years ago
dgtlmoon 8227c012a7 Diff + Preview - Fixing screenshot behaviour after preference change
2 years ago
dgtlmoon c113d5fb24 Screenshot handling on the diff/preview section refactor (#630)
2 years ago
dgtlmoon 8c8d4066d7 Shared watches - include "Extract text" filter
2 years ago
dgtlmoon 277dc9e1c1
Improve error message when filter not found in page result (#666)
2 years ago
dgtlmoon fc0fd1ce9d "Extract text" filter - improve placeholder example
2 years ago
dgtlmoon bd6127728a Visual selector - 'clear selection' button should clear the filter also
2 years ago
dgtlmoon 4101ae00c6
New feature - "Extract text" filter ability (#624)
2 years ago
dgtlmoon 62f14df3cb
Fixing RSS feed HTML content formatting (#662)
2 years ago
dgtlmoon 7929aeddfc
'Mark all viewed' button was missing in this version, added test also. (#652)
2 years ago
dgtlmoon 8294519f43 Content fetcher - Handle when a page doesnt load properly
2 years ago
dgtlmoon 8ba8a220b6 Playwright - Correctly close browser context/sessions on exceptions
2 years ago
dgtlmoon aa3c8a9370
Move history data to a textfile, improves memory handling (#638)
2 years ago
dgtlmoon 329c7620fb Remove UK Covid news
2 years ago
Amos (LFlare) Ng 1f974bfbb0
Visual Selector fix: Firefox compatibility - Visual Selector (#646)
2 years ago
Tim Loderhose 437c8525af
Remove group tag arbitrary length limit (#645)
2 years ago
dgtlmoon a2a1d5ae90 Distill.io import bug fix when no tags assigned to a watch (#557)
2 years ago
dgtlmoon 2566de2aae Ignore whitespace on by default
3 years ago
dgtlmoon dfec8dbb39 Visual Selector - clear events when changing tabs
3 years ago
dgtlmoon 5cefb16e52 Minor code cleanup
3 years ago
dgtlmoon 341ae24b73 Re #616 - content trigger - adding extra test (#620)
3 years ago
dgtlmoon 9d742446ab Playwright - ByPass CSP for more reliable JS scraping, disable accept downloads
3 years ago
dgtlmoon e3e022b0f4 VisualSelector - Better handling of filter targets that are no longer available in the HTML
3 years ago
dgtlmoon 7983675325 Visual Selector - be more resilient when sites interfere with the xPath scraping
3 years ago
dgtlmoon eef56e52c6 Adding new Visual Selector for choosing the area of the webpage to monitor - playwright/browserless only (#566)
3 years ago
dgtlmoon 8e3195f394 0.39.14
3 years ago
dgtlmoon e17c2121f7 Fix encoding errors with XPath filters from UTF-8 responses (#619)
3 years ago
dgtlmoon 07e279b38d
API Interface (#617)
3 years ago
dgtlmoon 2c834cfe37 Add note that changedetection is not performed on the screenshot just yet (WIP https://github.com/dgtlmoon/changedetection.io/pull/419 )
3 years ago
dgtlmoon dbb5c666f0 Fixing edit template HTML
3 years ago
dgtlmoon 70b3493866
Proxy settings on watch should have a "[ ] default" option (#610)
3 years ago
dgtlmoon 3b11c474d1
Input field tidyup (#611)
3 years ago
dgtlmoon 890e1e6dcd Update wiki link for 'More info' about sharing a watch and its configuration
3 years ago
dgtlmoon 6734fb91a2
Option to control if pages with no renderable content are a change (example: JS webapps that dont render any text sometimes) (#608)
3 years ago
dgtlmoon 16809b48f8 Playwright - raise EmptyReply on empty reply, no need to process further
3 years ago
dgtlmoon 67c833d2bc
Re #214 - configurable wait extra seconds for webdriver requests before extracting text (#606)
3 years ago
weeix 31fea55ee4
Fix PLAYWRIGHT_DRIVER_URL default value (cf. #587) (#599)
3 years ago
dgtlmoon 034507f14f
Fixing Pip install problem - Update MANIFEST to include model/ subdir, improving imports (#593)
3 years ago
dgtlmoon 0e385b1c22 0.39.13
3 years ago
dgtlmoon f28c260576
Distill.io JSON export file importer (#592)
3 years ago
dgtlmoon 18f0b63b7d
Ability to specify a list of proxies to choose from, always using the first one by default, See wiki (#591)
3 years ago
dgtlmoon 9807cf0cda Playwright - code fix
3 years ago
dgtlmoon d4b5237103 Playwright fetcher - more reliable by just waiting arbitrary seconds after the last network IO
3 years ago
dgtlmoon dc6f76ba64
Make proxy configuration more consistent - see https://github.com/dgtlmoon/changedetection.io/wiki/Proxy-configuration (#585)
3 years ago
dgtlmoon 1f2f93184e Playwright fetcher - use the correct default User-Agent
3 years ago
dgtlmoon 0f08c8dda3
Toggle visibility of extra requests options/settings when not in use (#584)
3 years ago
dgtlmoon 68db20168e
Add new fetch method: Playwright Chromium (Selenium/WebDriver alternative) (#489)
3 years ago
dgtlmoon 1d4474f5a3
Simplify scrub operation (simply cleans all) (#575)
3 years ago
dgtlmoon 613308881c
Bugfix - dont update record when deleted during check
3 years ago
dgtlmoon c0d0424e7e
Data storage bug fix #569
3 years ago
dgtlmoon 06517bfd22
Ability to 'Share' a watch by a generated link, this will include all filters and triggers - see Wiki (#563)
3 years ago
dgtlmoon ffc4215411 Unify MINIMUM_SECONDS_RECHECK_TIME env var variable to 60 seconds
3 years ago
dgtlmoon 9e708810d1
Seconds/minutes/hours/days between checks form field upgrade from 'minutes' only (#512)
3 years ago
dgtlmoon 1e8aa6158b Form styling improvements
3 years ago
dgtlmoon 015353eccc Form field handling improvements - fixing field list handler for empty lines
3 years ago
dgtlmoon 501183e66b Fix "Add email" button in main global notification settings
3 years ago
dgtlmoon def74f27e6 Test notification button fixed in main settings (#556)
3 years ago
dgtlmoon 37775a46c6 tgram:// be sure total notification size is always under their 4096 size limit
3 years ago
dgtlmoon e4eaa0c817
Shows which items are already in the queue, disables adding to the queue if already in the recheck queue (#552)
3 years ago
dgtlmoon 9e71f2aa35 Discord:// notification size limit - also includes the notification title
3 years ago
dgtlmoon 2a9fb12451
Import speed improvements, and adding an import URL batch size of 5,000 to stop accidental CPU overload (#549)
3 years ago
dgtlmoon 6c3c5dc28a Ability to set the default fetch mode via the DEFAULT_FETCH_BACKEND variable
3 years ago
dgtlmoon 8f062bfec9
Refactor form handling (#548)
3 years ago
dgtlmoon 380c512cc2
Adding support for change detection of HTML source-code via "source:https://website.com" prefix (#540)
3 years ago
dgtlmoon d7ed7c44ed Re-label the quick-add widget placeholder 'tag' to 'watch group'
3 years ago
dgtlmoon 34a87c0f41
HTTP Fetcher code improvements
3 years ago
dgtlmoon 4074fe53f1 Adding RSS metadata auto-discovery
3 years ago
Tristan Hill 44d599d0d1
Upgrade WTforms form handler to v3 (#523)
3 years ago
dgtlmoon 615fe9290a 0.39.12
3 years ago
dgtlmoon 2cc6955bc3
Miscellaneous settings form visual improvements (#535)
3 years ago
dgtlmoon 9809af142d
Option to render links as [Some Text ](/link), adds the ability to change-detect on hyperlink changes
3 years ago
dgtlmoon 1890881977 Specify our Discord avatar_url as default avatar_url
3 years ago
dgtlmoon 9fc2fe85d5 Minor git updates
3 years ago
dgtlmoon bb3c546838 Fix screenshot tab name
3 years ago
dgtlmoon 165f794595
Discord:// notifications should be cut to 2000 chars or Discord will not process them. (#531 + #323)
3 years ago
dgtlmoon a440eece9e Make long reports in the notification error log easier to read
3 years ago
dgtlmoon 34c83f0e7c
[Add email] button in notification settings with a prefix set from NOTIFICATION_MAIL_BUTTON_PREFIX env variable when defined. (#528)
3 years ago
dgtlmoon 63e91a3d66 Skip processing a watch into the RSS feed if there's not enough data to examine (fixes Internal Server Error when accessing the RSS feed) (#521)
3 years ago
dgtlmoon 3034d047c2
Introduce an AJAX button for sending test notifications instead of the checkbox (#519)
3 years ago
dgtlmoon 2620818ba7 Make text tab always available at default
3 years ago
dgtlmoon 9fe4f95990
When fetching a snapshot via Chrome, make the most recent screenshot available on the Diff and Preview pages (#516)
3 years ago
dgtlmoon ffd2a89d60
Remove 'unviewed' status in watch table when Diff link clicked (#514)
3 years ago
dgtlmoon 8f40f19328
RSS feed CDATA should contain difference output
3 years ago
dgtlmoon 082634f851
Fix - {diff} and {diff_full} notifications tokens were not always including the full output
3 years ago
dgtlmoon c79d6824e3
Minor UI cleanups (mobile tabs, font sizing) (#503)
3 years ago
zznidar 946377d2be Fix typo in Filters & Triggers settings. (#495)
3 years ago
zznidar 5db9a30ad4 Add autofocus attribute to password login field (#496)
3 years ago
dgtlmoon 1d060225e1 0.39.11
3 years ago
dgtlmoon 7e0f0d0fd8
Microsoft Windows installation fixes (#492)
3 years ago
dgtlmoon 942c3f021f
Allow changedetector to ignore status codes as a per-site setting (#479) (#485)
3 years ago
dgtlmoon 5483f5d694
Security update - Use CSRF token protection for forms, make "remove password" use HTTP Post (#484)
3 years ago
dgtlmoon f2fa638480
Security update - Protect against file:/// type access by webdriver/chrome. (#483)
3 years ago
dgtlmoon 3e8a15456a
Detect byte-encoding when the server mishandles the content-type header reply (#472)
3 years ago
dgtlmoon 2a03f3f57e Improving form/edit example markup
3 years ago
dgtlmoon ffad5cca97
JSON diff/preview should use utf-8 encoding where possible (#465)
3 years ago
Tim Loderhose 60a9a786e0
Fix typo in settings form
3 years ago
dgtlmoon c25294ca57 0.39.10
3 years ago
Tim Loderhose d4359c2e67
Add filter to remove elements by CSS rule from HTML before change detection is run (#445)
3 years ago
dgtlmoon 44fc804991
Minor updates to filters form text
3 years ago
dgtlmoon 7ce9e4dfc2
Testing - Refactor HTTP Request Type test (#453)
3 years ago
dgtlmoon 3cc6586695 Make table header font size the same as content
3 years ago