Embedded Videos - youtube asks: sign in to confirm you're not a bot - browserprofile doesn't help

Help! Browsertrix
1

Hi,
Me and my collegues from landesbibliothek Berlin are relatively new to Webarchiving and Browsertrix. We have this issue, maybe someone can help:

With more and more pages we have the problem that the embedded youtube videos are not archived. Instead in the replay we see that youtube is asking for registration. We tried creating browserprofiles with added registration information for youtube. de and youtube. com but the issue stays the same.

It feels like we did not have that issue when we started using browsertrix last year. Maybe youtube got smarter in recognizing the crawler?

A page that we had that issue with is for eaxmple

peaceof.land

etc.

Are there other work arounds?

Thanks for your help

Leonie

1 Like
2

Hi all,

does anybody have any idea for a solution for the issue yet?

The problem of getting the videos crawled is quite challenging.

Anybody out there with a lot of expertise with browsertrixing websites with a lot of video content??

Best,
Michael

3

Hey!

The only workaround we know of at this time is making sure you are signed into YouTube with a browser profile prior to crawling… But if you’ve tried that I have no additional advice at this time. We can confirm that YouTube has changed some bot-detection code to make this harder in recent months, sorry it’s affecting your crawls :frowning:

4

Just an aside for anyone having this issue: I did have success archiving a Youtube video when running the Browsertrix stack locally. So that’s a potential option.

I’m not sure if the bot-detection is based on the IP address for the Browsertrix service or something else though.