nicruediger,
I haven't used this myself, but if the Help file is to be believed (which can be a stretch), setting ENCODE to true is to convert non-standard characters to HTML entities.
Given the number of entities and the mixed support most browsers have for named entities (as opposed to numeric ones), it's not surprising that HTMLIB01 might be a bit, um, "underpowered" in its support for character encoding.
If it's not working, the only thing I can suggest would be to manually parse the resulting HTML file using a TextStream variable and then manually encode entities as appropriate.
In my experience, numeric entities are supported in more browsers than the named entities.
Hope this helps...
-- Lance