Commit Graph

  • 72c4d357c9 Merge branch 'master' into restock-visualselector-refactor dgtlmoon 2024-02-20 22:12:32 +0100
  • 904ef84f82
    Build fix - Pinning package versions and Custom browser endpoints should not have a proxy set (#2204) dgtlmoon 2024-02-20 22:11:17 +0100
  • 1f4bc31d84 dnspython pinning #2204 fix-prefer-fetch dgtlmoon 2024-02-20 20:30:17 +0100
  • b850bad038 also locking urllib dgtlmoon 2024-02-20 20:25:26 +0100
  • 381ccc4033 lock version dgtlmoon 2024-02-20 20:24:18 +0100
  • de63b86ba2 lock version dgtlmoon 2024-02-20 20:18:03 +0100
  • 238b909c0b Custom browser endpoints should not have a proxy set dgtlmoon 2024-02-20 20:06:06 +0100
  • d2569ba715
    Update stock-not-in-stock.js dgtlmoon 2024-02-20 20:00:31 +0100
  • b25928d9c4 Using 2.0.0rc4 pyppeteerstealth for puppeteer fetches dgtlmoon 2024-02-20 19:52:30 +0100
  • 20a8b86db3 requirements/chore: keep under dnspython2.6 #2202 Constantin Hong 2024-02-21 00:36:30 +0900
  • 258f227d05 requirements/chore: Unpin dnspython #2029 Constantin Hong 2024-02-16 00:56:05 +0900
  • eccf60b729 Merge branch 'master' into gevent Constantin Hong 2024-02-16 00:55:18 +0900
  • 290fd844a4 __init__/fix: The app will shutdown if wsgi fail. Constantin Hong 2024-02-16 00:04:12 +0900
  • b08165beac workflows/test-only.yml/test: Add wsgi fail case: Address already in use Constantin Hong 2024-02-16 00:00:38 +0900
  • 8aa675cbf0 Maintain the pool of workers in its own thread for more reliability and flexibility #2191 adjustable-workers dgtlmoon 2024-02-14 20:25:36 +0100
  • 491715b9b9 more tweaks dgtlmoon 2024-02-14 17:27:12 +0100
  • 71a46130b4 add handy comment dgtlmoon 2024-02-14 10:31:57 +0100
  • b045a72460 More JS tweaks dgtlmoon 2024-02-14 10:27:15 +0100
  • 4012eafeda processors/text_json_diff/fix: Keep an order of filter and remove duplicated filters. Constantin Hong 2024-02-14 17:11:47 +0900
  • 04485ae493 tests/test_group/test: Add test for tag filter and watch filter altogether. Constantin Hong 2024-02-14 17:00:36 +0900
  • 62b55df7e5 Refactor to make chrome happy dgtlmoon 2024-02-14 01:23:21 +0100
  • 01bb85b451 update_worker/feat: Detailed loguru log. #2184 Constantin Hong 2024-02-14 00:11:06 +0900
  • ccb42bcb12 Fetching pages - Custom browser endpoints should not have default proxy info added dgtlmoon 2024-02-12 19:05:10 +0100
  • bd5bbd8c5c Add time limit to puppeteer fetch #2182 puppeteer-catch-timeout dgtlmoon 2024-02-12 14:18:15 +0100
  • 4163030805 Puppeteer - fixing wait times dgtlmoon 2024-02-12 13:02:24 +0100
  • 140d375ad0 Puppeteer - more improvements to proxy and authentication dgtlmoon 2024-02-12 12:54:11 +0100
  • d8ad19c650 Puppeteer - setting cache enabled should be awaited misc-fixes dgtlmoon 2024-02-12 12:41:41 +0100
  • 1a608d0ae6
    Puppeteer - client fixes for proxy and caching (#2181) dgtlmoon 2024-02-12 12:40:31 +0100
  • e6ed91cfe3
    dependabot - Bump the all group with 1 update (artifact store) (#2180) dependabot[bot] 2024-02-12 12:40:03 +0100
  • 60baf3a1d2 Misc puppeteer client fixes #2181 dgtlmoon 2024-02-12 09:56:17 +0100
  • 67438677cd
    Bump the all group with 1 update #2180 dependabot[bot] 2024-02-12 01:11:52 +0000
  • 008272cd77
    Puppeteer fetch - fixing exception names dgtlmoon 2024-02-11 11:18:36 +0100
  • 823a0c99f4
    Code - Split content fetcher code up (playwright, puppeteer and requests), fix puppeteer direct chrome support (#2169) dgtlmoon 2024-02-11 00:09:12 +0100
  • 00717788ab Remove accidental workaround #2169 dgtlmoon 2024-02-10 23:47:52 +0100
  • 1e3f1cfe3d Add some nicer exception handling dgtlmoon 2024-02-10 23:19:08 +0100
  • 1a87a288de Add comment dgtlmoon 2024-02-10 21:18:11 +0100
  • b099224d46 Merge branch 'master' into split-content-fetchers-add-puppeteer-correctly dgtlmoon 2024-02-10 20:57:37 +0100
  • 1f57d9d0b6
    Alpine linux build - adding JPEG development headers to fix build errors dgtlmoon 2024-02-10 20:57:04 +0100
  • a635a15a0a downgrade to python3.10 needed dgtlmoon 2024-02-10 20:56:11 +0100
  • cbfd42b674 test these files #2176 alpina-libjpeg dgtlmoon 2024-02-10 19:47:26 +0100
  • 377860e88b Fixes build error "The headers or library files could not be found for jpeg, a required dependency when compiling Pillow from source." dgtlmoon 2024-02-10 19:42:40 +0100
  • a36f0d8809 use newer version dgtlmoon 2024-02-10 19:31:52 +0100
  • cb06cef463 Revert "html_tools/feat: Add parameter fragment explicitly into elementpath.select" #2175 Constantin Hong 2024-02-11 02:28:17 +0900
  • 1747225c67 html_tools/feat: Add parameter fragment explicitly into elementpath.select Constantin Hong 2024-02-11 01:41:35 +0900
  • e5f87dff50 requirements/chore: Skip 4.2 version and make sure 4.1.5 as minimum Constantin Hong 2024-02-11 01:29:15 +0900
  • cfed57feb7 fix test dgtlmoon 2024-02-10 15:15:55 +0100
  • 979b5c34f4 use smarter tests dgtlmoon 2024-02-10 15:04:55 +0100
  • 667b992b02 attempt to store container logs as artifacts dgtlmoon 2024-02-10 14:41:07 +0100
  • b38dca3e7f Improve error texts for humans dgtlmoon 2024-02-10 13:57:19 +0100
  • 09a4d4b610 fixes for proxy connections dgtlmoon 2024-02-10 13:56:55 +0100
  • 0ba104433c requirements/fix: Skip elementpath 4.2 Constantin Hong 2024-02-10 20:47:29 +0900
  • 5556af8f0c text_json_diff/fix: Keep order of filter and remove duplicated filters. #2171 Constantin Hong 2024-02-10 01:53:21 +0900
  • 0988ba3987 test_group/test: Add test for merge_filters Constantin Hong 2024-02-10 02:21:45 +0900
  • 16dbd69f57 text_json_diff/refactor: Add merger_filters function Constantin Hong 2024-02-10 01:49:11 +0900
  • ce1c9c9dc9 Revert "processors/text_json_diff/fix: Keep order of filter and remove duplicated filters." Constantin Hong 2024-02-10 01:45:10 +0900
  • c2c873cfa4 Revert "text_json_diff/refactor: Add merge_filters function" Constantin Hong 2024-02-10 01:44:47 +0900
  • 17d4e3dac2 custom proxy with auth could use playwright dgtlmoon 2024-02-09 17:41:16 +0100
  • 60568daa61 proxy test improvements dgtlmoon 2024-02-09 17:36:39 +0100
  • 020ed31c10 text_json_diff/refactor: Add merge_filters function Constantin Hong 2024-02-10 01:35:08 +0900
  • 8c4874b9cd Tweaks for proxy tests dgtlmoon 2024-02-09 17:18:40 +0100
  • 359ea6889c Also kill proxy containers dgtlmoon 2024-02-09 16:56:11 +0100
  • cf73a1f35a custom url fixes dgtlmoon 2024-02-09 16:48:15 +0100
  • 0315e0b356 adjust asyncio loop dgtlmoon 2024-02-09 14:29:07 +0100
  • ab781e523d Adding playwright fallback for browsersteps always dgtlmoon 2024-02-09 13:37:59 +0100
  • 1301e21b8c unify os get env dgtlmoon 2024-02-09 10:59:32 +0100
  • b2e8319287 try our dev/test version dgtlmoon 2024-02-09 10:59:17 +0100
  • 156e05043b bump back to modern versions dgtlmoon 2024-02-09 09:23:28 +0100
  • f752f8e25d processors/text_json_diff/fix: Keep order of filter and remove duplicated filters. Constantin Hong 2024-02-09 16:12:48 +0900
  • 1f991b8b87
    Merge 65bc76f11b into 3287283065 #2111 dgtlmoon 2024-02-08 18:52:10 +0000
  • e8fe365751 lock these two dgtlmoon 2024-02-08 19:31:50 +0100
  • 7a0e00b7c8 try custom pacakge dgtlmoon 2024-02-08 19:21:47 +0100
  • 34290fb065 lock to 3.11 dgtlmoon 2024-02-08 18:25:41 +0100
  • a6a368e455 tweaks to imports dgtlmoon 2024-02-08 18:17:39 +0100
  • 8742777a08 attempt pip upgrade dgtlmoon 2024-02-08 18:12:14 +0100
  • 1d9b8b4fa0 asyncio is standard lib dgtlmoon 2024-02-08 18:07:20 +0100
  • b6a2fca92d try locked v dgtlmoon 2024-02-08 18:03:39 +0100
  • 2a7a1180e1 rename tests dgtlmoon 2024-02-08 17:59:44 +0100
  • d693c23445 Adding requirements dgtlmoon 2024-02-08 17:59:33 +0100
  • 47a6017f53 Rename tests dgtlmoon 2024-02-08 17:44:28 +0100
  • 6a8bbc5527 improve tests dgtlmoon 2024-02-08 17:40:40 +0100
  • ea4653e91e Fix default dgtlmoon 2024-02-08 17:36:57 +0100
  • 5874ae2498 fix event loop dgtlmoon 2024-02-08 17:35:22 +0100
  • b981b87091 WIP dgtlmoon 2024-02-08 17:18:23 +0100
  • 9672b46caf rename module dgtlmoon 2024-02-08 16:10:04 +0100
  • 59eff6dc5d tidy up imports dgtlmoon 2024-02-08 16:09:21 +0100
  • b0a8d113df comment out unused slab dgtlmoon 2024-02-08 15:48:48 +0100
  • 23f1e9e6cc fix paths dgtlmoon 2024-02-08 15:47:37 +0100
  • e63b5e6f38 Split content_fetchers to their own clean modules, much easier to understand dgtlmoon 2024-02-08 15:45:45 +0100
  • 3287283065
    Plawright content fetcher - Fixes for status codes and screenshot info (#2168) dgtlmoon 2024-02-08 15:15:04 +0100
  • 62bbf78d08 Fixes for status code and screenshot info #2168 more-status-code-fix-screenshots dgtlmoon 2024-02-08 13:51:38 +0100
  • c5a4e0aaa3
    Fetching - Prefer to use SockPuppetBrowser (#2163) dgtlmoon 2024-02-07 20:58:21 +0100
  • 5119efe4fb 0.45.14 0.45.14 dgtlmoon 2024-02-07 12:43:23 +0100
  • cddc95193d Add more debug params to browser fetch #2163 prefer-sockpuppet dgtlmoon 2024-02-07 12:42:34 +0100
  • f45a1b485a Tidy up queries dgtlmoon 2024-02-07 12:13:58 +0100
  • 1fe9501b4f fix test dgtlmoon 2024-02-07 12:13:46 +0100
  • 0b7dacce3a small tidyup dgtlmoon 2024-02-07 12:06:19 +0100
  • 44bf2c9a47 remove unused vars dgtlmoon 2024-02-07 10:56:03 +0100
  • 78a2dceb81
    Bug fix - fix missing default var (#2162/ #2118/ #2122 ) dgtlmoon 2024-02-06 17:25:44 +0100
  • 536791b0d5 remove experimental disk cache test dgtlmoon 2024-02-06 16:48:55 +0100
  • 04b7d98e6c Removing experimental puppeteer fetching browser dgtlmoon 2024-02-06 16:35:48 +0100