Skip to main content
Version: Next

Changelog

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.6.2 (2023-11-26)​

Note: Version bump only for package @crawlee/basic

3.6.1 (2023-11-15)​

Note: Version bump only for package @crawlee/basic

3.6.0 (2023-11-15)

Features​

3.5.8 (2023-10-17)​

Note: Version bump only for package @crawlee/basic

3.5.7 (2023-10-05)​

Bug Fixes​

  • add warning when we detect use of RL and RQ, but RQ is not provided explicitly (#2115) (6fb1c55), closes #1773
  • ensure the status message cannot stuck the crawler (#2114) (9034f08)
  • RQ request count is consistent after migration (#2116) (9ab8c18), closes #1855 #1855

3.5.6 (2023-10-04)​

Note: Version bump only for package @crawlee/basic

3.5.5 (2023-10-02)​

Bug Fixes​

Features​

3.5.4 (2023-09-11)​

Features​

  • remove side effect from the deprecated error context augmentation (#2069) (f9fb5c4)

3.5.3 (2023-08-31)​

Bug Fixes​

  • browser-pool: improve error handling when browser is not found (#2050) (282527f), closes #1459
  • clean up inProgress cache when delaying requests via sameDomainDelaySecs (#2045) (f63ccc0)
  • pin all internal dependencies (#2041) (d6f2b17), closes #2040
  • respect current config when creating implicit RequestQueue instance (845141d), closes #2043

Features​

  • core: add default dataset helpers to BasicCrawler (#2057) (e2a7544)

3.5.2 (2023-08-21)​

Note: Version bump only for package @crawlee/basic

3.5.1 (2023-08-16)​

Features​

  • exceeding maxSessionRotations calls failedRequestHandler (#2029) (b1cb108), closes #2028

3.5.0 (2023-07-31)

Features​

3.4.2 (2023-07-19)​

Bug Fixes​

  • basic-crawler: limit internalTimeoutMillis in addition to requestHandlerTimeoutMillis (#1981) (8122622), closes #1766

Features​

  • core: add RequestQueue.addRequestsBatched() that is non-blocking (#1996) (c85485d), closes #1995
  • retryOnBlocked detects blocked webpage (#1956) (766fa9b)

3.4.1 (2023-07-13)​

Note: Version bump only for package @crawlee/basic

3.4.0 (2023-06-12)

Note: Version bump only for package @crawlee/basic

3.3.3 (2023-05-31)​

Bug Fixes​

  • set status message every 5 seconds and log it via debug level (#1918) (32aede6)

Features​

  • core: add Request.maxRetries to allow overriding the maxRequestRetries (#1925) (c5592db)

3.3.2 (2023-05-11)​

Bug Fixes​

  • respect config object when creating SessionPool (#1881) (db069df)

Features​

  • allow running single crawler instance multiple times (#1844) (9e6eb1e), closes #765
  • router: allow inline router definition (#1877) (2d241c9)

3.3.1 (2023-04-11)​

Bug Fixes​

  • start status message logger after the crawl actually starts (5d1df7a)
  • status message - total requests (#1842) (710f734)

3.3.0 (2023-03-09)

Features​

  • add basic support for setStatusMessage (#1790) (c318980)
  • move the status message implementation to Crawlee, noop in storage (#1808) (99c3fdc)

3.2.2 (2023-02-08)​

Note: Version bump only for package @crawlee/basic

3.2.1 (2023-02-07)​

Note: Version bump only for package @crawlee/basic

3.2.0 (2023-02-07)

Bug Fixes​

  • declare missing dependency on tslib (27e96c8), closes #1747

3.1.4 (2022-12-14)​

Bug Fixes​

3.1.3 (2022-12-07)​

Bug Fixes​

Features​

  • always show error origin if inside the userland (#1677) (bbe9045)

3.1.2 (2022-11-15)​

Note: Version bump only for package @crawlee/basic

3.1.1 (2022-11-07)​

Note: Version bump only for package @crawlee/basic

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/basic

3.0.4 (2022-08-22)​

Note: Version bump only for package @crawlee/basic