Unicode Replacement Character � Appearing in Crawls

abbiec.arch · August 6, 2026, 1:33am

Hello! I just completed my first crawl, and it seems characters such as quotation marks, apostrophes, and umlaut characters are being represented with �. My browser is able to read these characters fine, but they can’t seem to be rendered correctly in the crawl. Are there any settings I can use, or any tips on how I may remedy this issue? Or does anyone have any insight into what’s going on here?

I am using this Browsertrix version: v1.24.2-643de02

I also tried changing the language under Browser Settings to see if that would work, but to no avail. If anyone has any tips for a beginner, please let me know!

Edit: Checking the Crawl QA Overview, it seems as if the � don’t appear in the screenshot or the text. Perhaps this may be a topic for the Replay Webpage category, please let me know if I should relocate my post.

ilya · August 6, 2026, 2:31am

Hi, yes, this is likely an encoding issue in replay with ReplayWeb.page - can you share the example, or the a URL where this is happening?

abbiec.arch · August 6, 2026, 2:16pm

Hi Ilya!

Here is the page where I took the example from: Drinking Verse, Mettlach 1745 | Mettlach Stein Catalog | The Beer Stein Library

ilya · August 7, 2026, 5:38am

Thanks for sharing! This is actually a tricky issue, but we should have a solution, though it’ll likely involve re-running the crawl in this case.
You can track the issue here:

github.com/webrecorder/replayweb.page

[Replay Bug]: Incorrect charset due to inheritance of charset from parent frame.

opened 05:28AM - 07 Aug 26 UTC

ikreymer

bug replay bug

### ReplayWeb.page Version v2.4.x ### What did you expect to happen? What happ…ened instead? If a page specifies just `Content-Type: text/html`, the browser would normally detect the charset. Unfortunately, when replayed in an iframe, the browser will use charset of the parent frame (probably UTF-8) and *not* run the detection, thus breaking replay. This is due to Step 6 from [this section of the spec](https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding) ``` If the HTML parser for which this algorithm is being run is associated with a Document d whose container document is non-null: Let parentDocument be d's container document. If parentDocument's origin is same origin with d's origin and parentDocument's character encoding is not UTF-16BE/LE, then return parentDocument's character encoding, with the confidence tentative. ``` Not setting the parent frame's charset causes other problems and is often not feasible (when using embedded replay) Somehow inserting a Using UTF-16 frame *does* work in Chrome and FF, but not Safari, but is a very very ugly hack.. The best solution is to detect and save the charset when the page is rendered in the browser at capture time, eg. via `document.characterSet` and apply that value during replay. ### Step-by-step reproduction instructions _No response_ ### Additional details _No response_

abbiec.arch · August 7, 2026, 3:28pm

Thanks Ilya! I’ll keep an eye on it, and hope it can be resolved soon Additionally, I read ikreymar’s current solution in the bug report:

The best solution is to detect and save the charset when the page is rendered in the browser at capture time, eg. via document.characterSet and apply that value during replay.

That isn’t something I can do, correct?