xxd usage change magic byte to JPEG

Question

I create a simple test file like this:

$ cat > test
blah

Now I run vi, and then :%!xxd to edit first bytes with FFD8 FF

00000000: ffd8 ffe0 0a                             blah.

and the I run :%!xxd -r.

file gives me NOT jpeg:

$ file test
test: Unicode text, UTF-8 text

And if I manage to get hexdump:

$ xxd test
00000000: c3bf c398 c3bf c3a0 0a                   .........

What am I doing wrong with xxd? Thank you