Bug#931827: lighttpd: server returnd 400, if %C0 is included in the URL

[Helmut Grohne <helmut.grohne@intenta.de>]

Hi,

On Thu, Jul 11, 2019 at 02:38:19AM +0200, OHNO Tetsuji wrote:
> lighttpd server is returnd ”400 Bad Request", if %C0 (or any other
> char.) is included in the URL.
> 
> for example,
> http://localhost/index.lighttpd.html : return OK (display index page)
> http://localhost/index.lighttpd.html?%C0 : 400 Bad Request
> http://localhost/index.lighttpd.html?%C1 : 400 Bad Request
> http://localhost/index.lighttpd.html?%C2 : OK
> 
> I can't understand this behavior.

Thank you for the detailed report. I don't fully understand this either
and am thus Ccing Glenn Strauss (upstream).

> -- Configuration Files:

Thank you for including the configuration.

> server.http-parseopts = (
>   "header-strict"           => "enable",# default
>   "host-strict"             => "enable",# default
>   "host-normalize"          => "enable",# default
>   "url-normalize-unreserved"=> "enable",# recommended highly
>   "url-normalize-required"  => "enable",# recommended
>   "url-ctrls-reject"        => "enable",# recommended
>   "url-path-2f-decode"      => "enable",# recommended highly (unless breaks app)
>  #"url-path-2f-reject"      => "enable",
>   "url-path-dotseg-remove"  => "enable",# recommended highly (unless breaks app)
>  #"url-path-dotseg-reject"  => "enable",
>  #"url-query-20-plus"       => "enable",# consistency in query string
> )

You are using the new parsing defaults that Glenn implemented for
buster. I suspect that by changing one of these to disable, you can
restore the previous behaviour. (<- workaround)

I guess that the behaviour is connected to buffer_is_valid_UTF8 in some
way. If you pass the decoded buffer to buffer_is_valid_UTF8 you get 0
(invalid) for "\xc0" and for "\xc1", but not for "\xc2". However, those
cases where buffer_is_valid_UTF8 is involved would typically result in a
502 code rather than 400, so maybe this is wrong.

Glenn, can you comment on this?

Helmut