Home / Guides / Why you see ’ and é in text, and how to fix it

Why you see ’ and é in text, and how to fix it

In short

Sequences such as ’ and é happen when text saved as UTF-8 is read as an older encoding like Windows-1252. The cure is to read the file with the right encoding, not to edit the garbled text. If a character has already been replaced by a question mark, the original is gone.

You open a file or a web page and an apostrophe has turned into three odd symbols, an accented letter into two, and every space into a capital A with a hat. The words are all still there, but they are wearing the wrong clothes.

What you are seeing

This kind of damage is called mojibake, from the Japanese for character change. It happens when text is stored in one encoding and read in another. Today most text is stored as UTF-8, which uses one byte for plain English letters and two to four bytes for most other characters. If a program decides those bytes are Windows-1252 instead, each byte is shown as its own character.

That is why the damage follows a pattern. A curly apostrophe (U+2019) becomes ’. A left curly quote becomes “ and an em dash becomes —. An é becomes é. A non-breaking space becomes a capital A with a circumflex (Â) followed by a space. And if you see  at the very start of a file, that is the byte order mark, which is invisible when read correctly.

Where it comes from

  • A CSV file saved as UTF-8 and opened in Excel by double-clicking, which on many systems assumes an older encoding.
  • A web page with no character set declared, so the browser guesses.
  • Emails or exports where the encoding in the header does not match the content.
  • Text saved as ANSI by an old program and read as UTF-8, or the reverse.
  • Text copied through several systems, each one converting it once more.

How to fix it at the source

  • On a web page, add a meta tag declaring the character set as utf-8 near the top of the head, and serve the page with the same setting.
  • In Excel, don't double-click a CSV. Use Data, then From Text/CSV, and set the file origin to 65001: Unicode (UTF-8) so it reads the bytes correctly.
  • When saving a CSV that others will open in Excel, choose CSV UTF-8 (Comma delimited) from the Save As list.
  • In Visual Studio Code, use the encoding shown in the status bar and choose Reopen with Encoding to try a different one.
  • In Notepad or TextEdit, reopen the file and choose the encoding explicitly, then save it as UTF-8.

Repairing text that is already garbled

If the garbled text was produced by exactly one wrong decoding, it can often be reversed by encoding it back with the wrong encoding and decoding it with the right one. In Python, for example, encoding the garbled string as cp1252 and then decoding the result as utf-8 turns ’ back into an apostrophe. Libraries such as ftfy automate this and cope with cases where the simple recipe fails, such as characters whose bytes have no Windows-1252 equivalent.

There is an important limit. If the text has gone through a step that replaced characters with a question mark or the black diamond replacement character (U+FFFD), the original information is gone, and no tool can bring it back. In that case you need the original file or source.

What Paste & Wash does and does not do

Paste & Wash removes hidden characters, odd spaces, repeated line breaks, citation markers and similar clutter. It does not repair an encoding mix-up, because that is a problem with how a file was read rather than with the characters pasted. If your text is garbled, fix the encoding first.

Once the text reads correctly, a cleaning pass is still useful. The non-breaking spaces that appear as  in a garbled copy are ordinary non-breaking spaces in a correct one, and the cleaner will replace them with normal spaces if you want that.

A short troubleshooting order

  • Check the character set declaration or the file's saved encoding.
  • Reopen or import the file with UTF-8 selected.
  • If the damage is already in the text and uses patterns like ’, try a re-decoding tool or script.
  • If you see question marks or replacement characters, go back to the original source.
  • When the text looks right, run it through a cleaner for spacing and invisible characters.

Quick reference: what the garbled version means

  • ’ is a curly apostrophe (’).
  • “ is an opening curly double quote (“). The closing one is less predictable: one of its bytes has no character in Windows-1252, so it often shows as †followed by an invisible or odd symbol.
  • – is an en dash (–), and — is an em dash (—).
  • … is an ellipsis (…).
  • é is é, è is è, ü is ü, ñ is ñ, ä is ä, ö is ö and ß is ß.
  • £ is a pound sign (£), and € is a euro sign (€).
  • Â on its own, usually before a space, is a non-breaking space that has been read in the wrong encoding.
  •  at the start of a file is a byte order mark.

Try the cleaner to see what is hiding in your own text.

Related guides

Browse all 20 guides