50,000 max pages

[pchan3]

It seems that Browsertrix can crawl a maximum of 50,000 pages. Is there a workaround for this?"

[tessa-webrecorder]

Hi Peter,

In our hosted Browsertrix service, the maximum number of pages per crawl is one of the distinguishing features between subscription tiers, so it’s possible this is just a limit of your current subscription. If that doesn’t seem right, feel free to send us an email at support@webrecorder.net.

In a self-hosted instance, this limit is set by the max_pages_per_crawl value in the Helm chart. Smaller quotas can also be set per-organization in the admin-only org list by clicking on “Edit Quotas” in the dropdown menu to the right of the organization’s name.

[pchan3]

I changed the limit to 200,000, and I can see the update on the Limits page (see attached image). However, the system still stops caching more pages when it reaches 50,000. When I check Review Settings, the limit is still set to 50,000 instead of 200,000.