"find" piped to a file mangles Unicode characters

azdruid · 02-11-2009, 10:35 PM

Hi folks, I'm having some trouble with "find" in a bash script. The command I am trying to run is

Code:

find /Users/andrew/Music/ -iname "*.mp3" -or -iname "*.flac"

. Some of the filenames contain Unicode characters. These characters print out perfectly fine in the Terminal, like so:

Code:

/Users/andrew/Music//Brazilian Girls/Brazilian Girls/04 - Sirènes de la Fête.flac
/Users/andrew/Music//Brazilian Girls/Brazilian Girls/05 - Corner Store.flac

HOWEVER, when I pipe this output to a file (like so)

Code:

find /Users/andrew/Music/ -iname "*.mp3" -or -iname "*.flac" > mkplaylist

, the characters are mangled. When I open the file in TextEdit (I am on a Mac), I get this:

Code:

/Users/andrew/Music//Brazilian Girls/Brazilian Girls/04 - SireÃÄnes de la FeÃÇte.flac

How can I make bash print these characters properly? Thanks very much in advance!

David the H. · 02-12-2009, 06:55 AM

I assume your language environment is set to UTF-8? But what encoding does "file filename" give you? It might be saving the file in, say iso-8859, or something instead. Then, are you sure the encoding display of TextEdit is set properly? It may not be automatically detecting the encoding of the file.

You might try piping the find output, or the file, through iconv to ensure that the data is in the encoding that you want.

I myself try to avoid filenames with non-ascii or restricted characters like spaces in them. It saves a lot of hassle in situations like this.