test.c:2210:9: warning: Value stored to 'char_len' is never read
char_len = 2;
^ ~
test.c:2921:3: warning: Value stored to 'pass' is never read
pass &= (parsed == 0);
^ ~~~~~~~~~~~~~
Reviewed-By: Fedor Indutny <fedor@indutny.com>
RFC-7230 Sec 3.2.4 expressly forbids line-folding in header
field-names.
This change no longer allows obsolete line-folding between the
header field-name and the colon. If HTTP_PARSER_STRICT is unset,
the parser still allows space characters.
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Always discard leading whitespace in a header value, even if it
is folded.
Pay attention to values of interesting headers (Connection,
Content-Length, etc.) even when they come on a continuation line.
Add a test case to check that requests and responses using only LF
to separate lines are handled correctly.
The support for folding of multi-line header values does not conform
to the specs. Given a request containing
Multi-Line-Header: foo<CRLF>
bar<CRLF>
http-parser will eliminate the whitespace breaking the header value to
yield a header value of "foobar". This is confirmed by the
LINE_FOLDING_IN_HEADER case in tests.c.
But from rfc2616, section 2.2:
A CRLF is allowed in the definition of TEXT only as part of a header
field continuation. It is expected that the folding LWS will be
replaced with a single SP before interpretation of the TEXT value.
And from draft-ietf-httpbis-p1-messaging-25, section 3.2.4:
A server that receives an obs-fold in a request message that is not
within a message/http container MUST either reject the message by
sending a 400 (Bad Request), preferably with a representation
explaining that obsolete line folding is unacceptable, or replace
each received obs-fold with one or more SP octets prior to
interpreting the field value or forwarding the message downstream.
So in the example above, the header value should be interpreted as
"foo bar", possibly with multiple spaces. The current http-parser
behaviour of eliminating the LWS altogether clearly deviates from the
specs.
For http-parser itself to confirm exactly would involve significant
changes in order to synthesize replacement SP octets. Such changes
are unlikely to be worth it to support what is an obscure and
deprecated feature. But http-parser should at least preserve some
separating whitespace when folding multi-line header values, so that
applications using http-parser can conform to the specs.
This commit is a minimal change to preserve whitespace when folding
lines. It eliminates the CRLF, but retains any trailing and leading
whitespace in the header value.
Remove the HTTP_PARSER_DEBUG macro for two reasons:
* It changes the size of struct http_parser, resulting in spurious memory
corruption bugs if part of your application is built with HTTP_PARSER_DEBUG=1
and other parts with HTTP_PARSER_DEBUG=0.
* It's a debugging tool for maintainers. It should never have been exposed in
the API in the first place.
the v6 parsing works by adding extra states for working with the
[] notation for v6 addresses. hosts and ports cannot be 0-length
because we url parsing from ending when we expect those fields to
begin.
http_parser_parse_url gets a free check for the correctness of
CONNECT urls (they can only be host:port).
this addresses the following issues:
i was bored and had my head in this space.
Before this change it would include the last slash in the separator between the
schema and host as part of the host. we cant use the trick used for skipping the
separator before ports, query strings, and fragments because if it was a CONNECT
style url string (host:port) it would skip the first character of the hostname.
Work around this by introducing a few more states to represent these separators
in a url differently to what theyre separating. this in turn lets us simplify
the url parsing so can simply skip what it considers delimiters rather than
having to special case certain types of url parts and skip their prefixes.
Add tests for the http_parser_parse_url().
This compares the http_parser_url struct that http_parser_parse_url()
produces against one that we expect from the test. If they differ
then http_parser_parse_url() misbehaved.
Summary:
- Add http_parser_pause() API. A callback may invoke this at any time.
This will cause http_parser_parse() to return indicating that it
parsed less than the number of requested bytes and set an error to
HBE_PAUSED. A paused parser with fail with HBE_PAUSED until it is
un-paused with http_parser_pause().
- Stop using 'state', 'header_state', 'index', and 'nread' shadow
variables and then updating their http_parser fields when we're done.
Instead, update the live values as we go. This will make it possible
to return from anywhere in the parser (say, due to EPAUSED) and have
valid/expected state.
- Update state before making callbacks so that if the want to pause,
we'll know the correct state already.
- Make sure that every callback has a state that uniquely identifies the
next step so that we can resume in the right place if we were suppoed
to be paused.
- Clean and re-factor up CALLBACK() macros.
- Use CALLBACK() macros for (almost) all callbacks; on_headers_complete
is still a special case. This includes on_body which we used to invoke
manually with a long run of bytes. We now use a 'body' mark and hit
its callback just like every other data callback.
- Clean up (most) gotos and replace with real states.
- Add some unit tests.
Fixes#70
- Break EOF handling out of http_should_keep_alive() into
http_message_needs_eof(), which we now use when determining what to do
with a message of unknown length. This prevents us from falling into
the s_body_identity_eof state in the cases where we actually *do* know
the length of the message (e.g. because the response status was 204).
- Add an http_parser_parse_url() method to parse a URL into its
constituent components. This uses the same underlying parser
as http_parser_parse() and doesn't do any data copies.
- Re-add the URL components in various test.c structures; validate
them when parsing.
- Treat ' ' specially, as apparently IIS6.0 can send this in headers.
Allow this character through if we're not in strict mode.
- Move some test code around so that test indices don't break when
HTTP_PARSER_STRICT changes.
Fixes#13.
With HTTP/1.1, if neither Content-Length nor Transfer-Encoding is present,
section 4.4 of RFC 2616 suggests http-parser needs to read a response body
until the connection is closed (except the response must not include a body)
See also joyent/node#2457.
Fixes#72
Jon Kolb
Alexis La Goutte
Jeff Pinner
David Wragg
Alexis Campailla
Fedor Indutny
Patrik Stutz
Ben Noordhuis
Chris Dickinson
Tóth Tamás
Corey Richardson
BogDan Vatra
Bertrand Paquet
Martell Malone
Nathan Rajlich
David Gwynne
Ivo Raisr
Peter Griess
koichik
Felix Geisendörfer