If you interpret the contents of a "plain text file" through anything more complex than a Unicode text encoding, then you now have "a special format". (And if you don't, you still at least need some encoding, even if it's an implicit ASCII encoding. "There ain't no such thing as plain text", as they say.)

There are also plenty of "special formats" out there that are really just zipping up a directory tree of something simpler.