Commit Graph

57 Commits (2ba061c7673aedcf93d215354fcc772973197962)

Author SHA1 Message Date
dgtlmoon 51cb83a20a Refactor preview code to its own module
3 months ago
dgtlmoon 2842ffb205
Restock - Use the scraped 'Not in stock' product status over the metadata version (many website lie in the metadata) (#2684)
3 months ago
dgtlmoon f255165571 Code - Small improvements in logging
3 months ago
dgtlmoon 00458b95c4
UI - Improvements to live preview of Filters text
3 months ago
dgtlmoon 5b34aece96
UI - Live preview - misc improvements (Adding test, fixes to filters) (#2663)
3 months ago
dgtlmoon 1b625dc18a UI - "Filters & Triggers" - Live preview of text filters (Preview the output of the filters section in realtime) (#2612)
3 months ago
dgtlmoon 367afc81e9
Reversing subprocess execution - saved a little memory but used a LOT more CPU (#2659)
3 months ago
dgtlmoon e173954cdd
Restock monitor - Only try to process restock information (like scraping for "out of stock" keywords) if the page was actually rendered correctly. (#2645)
3 months ago
dgtlmoon e830fb2320
Text filters - Adding filters "Trim whitespace" and "Remove duplicate lines"
3 months ago
Felipe Tuffani 8ecfd17973
Restock/Price detection - Fix duplicated prices with different data type on single page product #2636 (#2638)
3 months ago
dgtlmoon 19f3851c9d
Memory management improvements - LXML and other libraries can leak allocation, wrap in a sub-process (#2626)
4 months ago
dgtlmoon 55fe2abf42
Restock/Price detection - Better catching of errors when parsing metadata documents for restock/price check (#2602)
4 months ago
dgtlmoon 932cf15e1e
Price and restock scraping - small price fix scraper (#2575)
4 months ago
dgtlmoon 8a35d62e02
Handle zero-byte/empty content responses with "`[ ] Empty pages are a change`" option, the same as when the HTML doesnt render any useful text (#2530)
5 months ago
dgtlmoon 44e7e142f8 Restock/Price detection - Improving text information snapshot value
6 months ago
dgtlmoon fe704e05a3 Restock - Tweaking storage of "original price"
6 months ago
dgtlmoon 321426dea2
Ability to use restock and price amounts in notifications as tokens (for example {{restock.price}} ) (#2503)
6 months ago
dgtlmoon bde27c8a8f
Restock & Price detection - Ability to set up a tag/group that applies to all watches with price + restock limits
6 months ago
dgtlmoon 6f2186b442
UI - Restock/price following text cleanups
6 months ago
dgtlmoon cf0ff26275
UI - Extract <title> as title should work on all processors (#2490)
6 months ago
dgtlmoon cffb6d748c
Restock & Price monitor - Huge refactor, set upper and lower price alert limits, set % change, follow the prices and restock amounts directly in the watch-overview list
6 months ago
dgtlmoon f07ff9b55e
UI - Visual Selector should still update when elements were not found (#2476)
6 months ago
dgtlmoon 343e359b39
Now saving last two HTML snapshots for future reference, refactor, dont write screenshots and xpath to disk when no change detected (saves disk IO) (#2431)
6 months ago
Hritik Vijay ffd160ce0e
Filters - Implement jqraw: filter (use this to output nicer JSON format when selecting/filtering by JSON) (#2430)
6 months ago
dgtlmoon c6ee6687b5
Fetching/Requests - Fixing user agent header overrides per-watch of global settings (#2409)
7 months ago
dgtlmoon f49eb4567f
Ability to set default User-Agent for either fetching types directly in the UI (#2375)
7 months ago
dgtlmoon 9449c59fbb
Code - Getting ready for newer python versions - packing our own strtobool (#2291)
9 months ago
Constantin Hong 52c895b2e8
text_json_diff/fix: Keep an order of filter and remove duplicated filters. 2 (#2178)
10 months ago
dgtlmoon 904ef84f82
Build fix - Pinning package versions and Custom browser endpoints should not have a proxy set (#2204)
10 months ago
dgtlmoon ccb42bcb12 Fetching pages - Custom browser endpoints should not have default proxy info added
11 months ago
dgtlmoon 823a0c99f4
Code - Split content fetcher code up (playwright, puppeteer and requests), fix puppeteer direct chrome support (#2169)
11 months ago
dgtlmoon 616c0b3f65
New text filter - Sort text alphabetically filter (#2153)
11 months ago
dgtlmoon c90b27823a
Filtering - include_filters in group and watch settings should not duplicate (#2151 #1845)
11 months ago
dgtlmoon fe3cf5ffd2
Logging - Adding extra debug logging to change detection (#2136)
11 months ago
dgtlmoon 399cdf0fbf
Logging loguru output tweaks (#2112)
12 months ago
Constantin Hong 4be0fafa93
Support Loguru as a logger (#2036)
12 months ago
dgtlmoon 1749c07750
Restock detection - Check all elements for text to get stock status from, only consider elements inside the viewport, only consider elements more than 100px from the top (avoid menu) , trim any text returned (#2040)
12 months ago
dgtlmoon 273bd45ad7
Fetching - Custom browser on experimental/puppeteer fetcher - Don't switch to custom puppeteer mode if external browser URL is active (#2068)
1 year ago
dgtlmoon 599291645d
PDF Fetcher for change detection - Always use plain requests for PDF because otherwise we cant access the embed PDF in the browser (#2020)
1 year ago
dgtlmoon 5e2049c538 Fix build issue
1 year ago
Constantin Hong 26931e0167
feature: Support XPath2.0 to 3.1 (#1774)
1 year ago
dgtlmoon 5229094e44
New functionanlity - Selectable browser / ability to add extra browser connections (good for using "scraping browsers"/ etc) (#1943)
1 year ago
dgtlmoon c8dcc072c8
Code refactor for fetchers (#1941)
1 year ago
dgtlmoon 7220afab0a
RSS fetch - RSS field <title> was not rendering as text correctly, added workaround #1879
1 year ago
dgtlmoon 1c0fe4c23e
PDF Fetching - Handle when the PDF is given as inline content without a proper mime header (#1875)
1 year ago
dgtlmoon f707c914b6
RSS Fetching - Handle CDATA (commented out text) in RSS correctly, generally handle RSS better (#1866)
1 year ago
dgtlmoon ceac8c21e4
LD JSON Price followers - Handle incorrectly created LD-JSON price structures better (#1837)
1 year ago
dgtlmoon 2b948c15c1
Backend - Regular expression / string filtering refactor for Python 3.11 and deprecation warnings since Python 3.6 (#1786)
1 year ago
dgtlmoon e30b17b8bc
UI + Fetching - Be more helpful when a filter contains no text, suggest ways to deal with images in filters (#1819)
1 year ago
dgtlmoon 1987e109e8
New feature - Helper button to trigger a scan/access test of all proxies for a particular watch (#1685)
1 year ago