Fix utf-8 encoding problem

[shtse8]

Due to the urgent need in my project, hope it can help. #18

[ventrec]

I've ran into the same issue myself now, and this fix would be highly apprectiated.

@wasinger Is it possible to get this merged?

[shtse8]

I finally gave up this project and wrote my own html dom parse. But I am new on starting up a open source project.

[kukungkung]

I think it can help you.

curl_setopt($ch, CURLOPT_ENCODING, 'UTF-8');

[shtse8]

@kukungkung setting curlopt is just try to request with encoding utf-8, you have to decode the utf8 yourself. and the response may not follow your encoding. Also, mostly site are sending utf8 to you. here, the main problem is, htmlpagedom parse cannot support utf8, but not the curl.

[glensc]

the problem is not that utf8 is not parsed, just that result is encoded with html entities.

i solved this in my code:

$html = html_entity_decode((string)$crawler, ENT_NOQUOTES, 'UTF-8');

[glensc]

the PRs seems broken because created from @shtse8 master branch, thus changes from #19 and #20 mixed in both pull requests. and perhaps even changes not related to neither of the PRs.