It’s hard to say exactly, it seems perhaps one of the cdxj may be corrupt…
Can you reindex the WARCs using the wb-manager reindex <collection> command, or directly using the new cdxj-indexer with cdxj-indexer -s </path/to/warcs> (the -s flag is to generate a sorted output). Hopefully that will fix the issue…
Thanks for your reply @ilya. I was using cdxj-indexer and re-indexed again with cdxj-indexer, however Pywb fails when I create more than 1.000 cdxj files. Anyway I decided to run my own ReplayWeb.page.