I’m going through the process of archiving an old web property of ours. It contains some links that do javascript-based pop-up windows: javascript:showBio('Author Name')
I’ve been following the tutorial, and I’ve written a basic behavior that will gather those links, loop through and click on them. However, this seemed like perhaps a bit of a headache with having to close the window as part of the behavior. I’m not sure how to go about doing that.
As an alternate strategy, I figured i could just rewrite the href to the regular destination URL, since I know what it’ll be based on what’s passed to showBio(). If I do it that way, do i need to visit each of those links (and come back) as part of the behavior, is it the href rewriting enough?
Unfortunately behaviors run after link extraction, so you wouldn’t be able to just rewrite the href alone.
There’s an issue in the crawler GitHub repo in which we discussed adding a command line/YAML option for the crawler to specify selectors for extracting links. If this were implemented, it would make your task easier, as no behavior would be needed at all. Not sure if you’d have the development bandwidth to submit a PR for it: Specifying selectors for extracting links. · Issue #217 · webrecorder/browsertrix-crawler · GitHub. If not I could see if it could fit it into one of our next sprints.
Otherwise, likely your best bet at the moment is the first approach of using a behavior to gather and click on the links. Do the pop-up windows prevent your behavior from working as expected, or are you more concerned about the memory implications of keeping windows open? @ilya likely has a better idea on what is possible with the current utility methods
This was a local project, so to save time I just cut out the javascript in favour of straight-up links. I still hope to learn more about behaviours though.
With respect to opening windows, I could not get the behaviour to work beyond the first open window. My assumption is that you need to close the window and refocus back on the main browser window. I could be wrong about that, so any info you or Ilya can provide would be great.