Instagram advice with autopilot?

Hello!

After hearing of a few others’ experiences capturing Instagram discussed on the last Community Call, I attempted the rather ambitious task of capturing the Frick Collection’s Instagram account, dating all the way to 2013 (!) using the ArchiveWeb.page Desktop App and Autopilot.

It worked rather well for the most part! However, a little over halfway through the task (around 3 hours total) as each post was successively displayed on the screen, posts began to show up as gray boxes, as seen here:

I left the autopilot task to finish, but in the replay view they are appearing thusly as well.

Wondering if anyone has had any similar experiences or advice. I imagine the best method of ensuring capture may be to do it manually (as a few people mentioned on the call) but it’s of course a rather large account at about 2400 posts. Curious to hear about others’ experience in this situation, or similar.

Thanks for reading.

Jack Patterson
Web Archives Technician
NYARC

1 Like

Hi Jack,

The issue you’re running into is likely rate limiting from Instagram, but I’m not 100% sure if that’s what happened. Of course, it’s tricky to debug as its something that happens hours in.

One way to figure out if that’s what happened is to see if there were a bunch of 429 error code results that were recorded in the WARC. If you have a WACZ or WARC to share, we could take a look and see if that is what was happening.

I know that @pchan3 has also been archiving many Instagram accounts, but not sure if he ran across this issue.

1 Like