Hello all,
I created a Warc-Wacz collection on a Twitter account using ArchiveWeb.page. How can I analyze text or topic from this collection?
Thanks
Hi @aykutkaya thanks for this question! I think the answer might depend a bit on what type of analysis you want to do.
One thing that’s nice about the WACZ file that is created with ArchiveWeb.page is that it includes a pages.jsonl file that lists the pages in the archive and the text that is on the page. It is used when you search the collection.
If you unzip your WACZ file (yes it’s just a ZIP file), you should find the pages directory. It may be that the pages.jsonl file will be suitable for the type of analysis you are doing? I’m happy to continue the conversation here if you have questions.
Hello @edsu
Thank you for your answer. I want to analyze the most used words in the pages I archive. I searched many places but couldn’t find the right method. I opened the WACZ file with Replayweb.page, but I could not access a file with the pages.jsonl extension. Is it possible that I archived the files without the extension you mentioned?
Can you remind me what tool you used to create the WACZ file?
I used Archiveweb.page
Ah ok. To open or “unpack” the WACZ file you need to unzip it. The easiest way to do this is probably to change the file name to have a .zip extension and then double click on it to launch your operating systems ZIP utility. Then if you look in the extracted files you should find a pages.jsonl?
I did as you said and reached the pages.jsonl file, but there is not enough data for analysis.
@aykutkaya interesting – are you able to share the WACZ file with me, either here or at ehs@pobox.com?