Crawling placeholder even after long delays

Help! Browsertrix Crawler
1

I am not able to crawl happy cow even after giving long post load delays. When I see the screencast, I see the page rendered correctly. However in the replayweb.page, I see placeholders. Would be great to get some suggestions to make this work.

Config

seeds:
  - url: https://www.happycow.net/north_america/usa/new_york/new_york_city/?filters=vegan-vegetarian
    depth: 0
  - url: https://www.happycow.net/north_america/usa/new_york/new_york_city/?filters=vegan-vegetarian&page=2
    depth: 0
  - url: https://www.happycow.net/north_america/usa/new_york/new_york_city/?filters=vegan-vegetarian&page=3
    depth: 0
  - url: https://www.happycow.net/north_america/usa/new_york/new_york_city/?filters=vegan-vegetarian&page=4
    depth: 0
blockAds: true
collection: happycow-nyc
combineWARC: true
saveState: always
postLoadDelay: 24
profile: /crawls/profiles/happycow0.tar.gz
saveProfile: true
redisDedupeUrl: redis://10.89.1.2:6379/12
screencastPort: 9054
[user@vm-dev ~/Download

Command

podman run -it --rm -v <path>/browsertrix:/crawls:z webrecorder/browsertrix-crawler crawl --redisDedupeUrl redis://10.x.x.x:6379/12 --config /crawls/happycow-nyc-0.yaml

Output

hcnyc-Screenshot_2026-09-18_00-55-04
hcnyc-Screenshot_2026-09-18_00-55-041600×850 197 KB

2

Thanks for reporting, this is a replay issue. We should have a fix for it via this issue:

You don’t need to add a delay at all during crawl time, as its just not replaying correctly in this case, and should be fixed when we do a next release of ReplayWeb.page

1 Like
3

Thanks for fixing this.