Parser

An incremental HTTP/1.1 response parser. Feed it bytes as they arrive, in pieces of any size; it never copies the body and never allocates. The client uses it internally, and it is exported for programs with their own transport.

static char line[1024];
lwhttp_parser_t parser;

lwhttp_parser_init(&parser, line, sizeof(line), &callbacks, &state);
lwhttp_parser_set_request(&parser, "GET");
while (more_data) {
  size_t used = lwhttp_parser_execute(&parser, data, len);
  if (lwhttp_parser_error(&parser)) break;
  data += used;
  len -= used;
}

Types

struct lwhttp_parser

Incremental HTTP/1.x response parser.

The parser does not allocate. It copies one line at a time (status line, header, trailer, chunk size) into a caller-supplied buffer and streams body bytes straight from the input. Initialise it with lwhttp_parser_init(). Fields documented as internal must not be modified; the others may be read at any time, typically from callbacks.

const lwhttp_parser_callbacks_t *callbacks

Callback table given to lwhttp_parser_init(); may be NULL.

void *user

Opaque pointer for the application, given to lwhttp_parser_init().

char *line

Line buffer given to lwhttp_parser_init().

size_t line_size

Size of lwhttp_parser.line in bytes.

size_t line_len

Internal: bytes currently held in the line buffer.

uint32_t content_length

Value of the last Content-Length header parsed, or 0. Only defines the body length when LWHTTP_MSG_CONTENT_LENGTH is set (Transfer-Encoding takes precedence).

uint32_t remaining

Internal: body bytes left in the current length-framed body or chunk.

uint16_t status

Status code of the current message, or 0 before its status line has been parsed.

uint16_t flags

LWHTTP_MSG_* flags of the current message, valid from on_headers_complete onward.

uint16_t internal

Internal: header bookkeeping bits.

uint8_t state

Internal: parser state.

uint8_t request

Internal: request method mode set by lwhttp_parser_set_request().

uint8_t version_major

Major HTTP version from the status line (always 1 for an accepted message).

uint8_t version_minor

Minor HTTP version from the status line (0 for HTTP/1.0, 1 for HTTP/1.1).

uint8_t error

Internal: lwhttp_error_t of the failure, read it with lwhttp_parser_error().

type lwhttp_parser_t

Response parser state. See lwhttp_parser.

struct lwhttp_parser_callbacks_t

Callbacks invoked by lwhttp_parser_execute() and lwhttp_parser_finish().

Every member may be NULL. Except where noted, a callback returns 0 to continue; any other value stops parsing and puts the parser in the error state with LWHTTP_ERR_ABORTED. Callbacks run for every message, including interim (1xx) responses. All pointers passed to callbacks point into the parser’s line buffer or the caller’s input and are valid only for the duration of the call.

int (*on_status_line)(lwhttp_parser_t *parser, uint16_t status, const char *reason)

Called once the status line has been parsed.

Param parser:

The parser.

Param status:

Three-digit status code (100 to 999).

Param reason:

NUL-terminated reason phrase, possibly empty.

Return:

0 to continue, nonzero to abort.

int (*on_header)(lwhttp_parser_t *parser, const char *name, size_t name_len, const char *value, size_t value_len)

Called for each header field, in order. Obsolete folded continuation lines are joined to the field with a space.

Param parser:

The parser.

Param name:

Field name, NUL-terminated.

Param name_len:

Length of name.

Param value:

Field value with surrounding whitespace removed, NUL-terminated.

Param value_len:

Length of value.

Return:

0 to continue, nonzero to abort.

int (*on_headers_complete)(lwhttp_parser_t *parser)

Called after the empty line that ends the header section.

lwhttp_parser.flags already holds the final framing, keep-alive and upgrade flags.

Param parser:

The parser.

Return:

0 to continue, LWHTTP_PARSER_SKIP_BODY to treat the message as bodiless, any other value to abort.

int (*on_body)(lwhttp_parser_t *parser, const uint8_t *data, size_t len)

Called with body data as it arrives, possibly many times per message. Chunk framing is already removed.

Param parser:

The parser.

Param data:

Body bytes; valid only during the call.

Param len:

Number of bytes in data, never 0.

Return:

0 to continue, nonzero to abort.

int (*on_trailer)(lwhttp_parser_t *parser, const char *name, size_t name_len, const char *value, size_t value_len)

Called for each trailer field after the last chunk. Same arguments as lwhttp_parser_callbacks_t.on_header; trailer fields are not interpreted by the parser.

Return:

0 to continue, nonzero to abort.

int (*on_message_complete)(lwhttp_parser_t *parser)

Called when a message is complete. lwhttp_parser_is_complete() already returns true.

Param parser:

The parser.

Return:

0 to continue, nonzero to abort (the parser then enters the error state).

Message flags

LWHTTP_MSG_CHUNKED

Message flag: the body uses chunked transfer coding (chunked is the final Transfer-Encoding).

The parser removes the chunk framing; lwhttp_parser_callbacks_t.on_body receives only chunk data. Chunk extensions are validated and ignored.

LWHTTP_MSG_CONTENT_LENGTH

Message flag: the body is framed by Content-Length (stored in lwhttp_parser.content_length, possibly 0).

LWHTTP_MSG_KEEP_ALIVE

Message flag: the connection may stay open for another request after this message.

HTTP/1.1 is persistent unless Connection: close is present; HTTP/1.0 only with Connection: keep-alive and no close. Never set for upgrades, close-delimited bodies, or a Transfer-Encoding combined with Content-Length or sent by an HTTP/1.0 server.

LWHTTP_MSG_UPGRADE

Message flag: the connection switches protocol after this response head.

Set for 101 Switching Protocols and for any 2xx answer to a CONNECT request. The message has no body, and every byte after the head belongs to the new protocol: lwhttp_parser_execute() consumes nothing further.

LWHTTP_MSG_CLOSE_DELIMITED

Message flag: the body runs until the connection closes.

Used when there is neither Content-Length nor Transfer-Encoding, or when the final transfer coding is not chunked. Call lwhttp_parser_finish() at end of stream to complete the message.

LWHTTP_MSG_TRANSFER_CODED

Message flag: Transfer-Encoding lists a coding other than chunked (for example gzip).

The parser does not decode it, so the body bytes are still encoded. Combined with LWHTTP_MSG_CHUNKED or LWHTTP_MSG_CLOSE_DELIMITED. The client rejects such responses with LWHTTP_ERR_UNSUPPORTED.

LWHTTP_MSG_NO_BODY

Message flag: the message has no body.

Set for 1xx, 204 and 304 responses, responses to HEAD, upgrades, and when lwhttp_parser_callbacks_t.on_headers_complete returned LWHTTP_PARSER_SKIP_BODY. After the header section exactly one of LWHTTP_MSG_NO_BODY, LWHTTP_MSG_CONTENT_LENGTH, LWHTTP_MSG_CHUNKED and LWHTTP_MSG_CLOSE_DELIMITED is set.

LWHTTP_PARSER_SKIP_BODY

Return value for lwhttp_parser_callbacks_t.on_headers_complete that tells the parser the message has no body.

The framing flags are replaced by LWHTTP_MSG_NO_BODY and the message completes immediately after the header section. Use it when the parser was not told the request method (for example a response to HEAD, see lwhttp_parser_set_request()).

Functions

void lwhttp_parser_init(lwhttp_parser_t *parser, char *line, size_t line_size, const lwhttp_parser_callbacks_t *callbacks, void *user)

Initialise a parser for a new stream of responses.

Clears the request mode (see lwhttp_parser_set_request()) and resets all message state.

Parameters:

• parser – Parser to initialise.

• line – Buffer for one protocol line at a time; must stay valid while the parser is used. Callback strings point into it.

• line_size – Size of line. Each status, header, trailer or chunk-size line (including its CR, and with folded continuation lines joined) must be shorter than this, or parsing fails with LWHTTP_ERR_HEADERS_TOO_LARGE (LWHTTP_ERR_TOO_LARGE for chunk-size lines).

• callbacks – Callback table, or NULL; must stay valid while the parser is used.

• user – Stored in lwhttp_parser.user for the callbacks.

void lwhttp_parser_reset(lwhttp_parser_t *parser)

Prepare the parser for a new message, clearing status, flags, version and any error.

Keeps the callbacks, user pointer, line buffer and request mode. Not needed between consecutive messages on one stream: lwhttp_parser_execute() resets automatically after a complete message.

Parameters:

• parser – The parser.

void lwhttp_parser_set_request(lwhttp_parser_t *parser, const char *method)

Tell the parser which request method the responses answer, which affects body framing.

"HEAD" makes every response bodiless; "CONNECT" makes a 2xx response an upgrade (LWHTTP_MSG_UPGRADE). Any other method, or NULL, selects normal framing. The comparison is case-sensitive. The mode survives lwhttp_parser_reset(); call this before the response arrives.

Parameters:

• parser – The parser.

• method – Request method, or NULL.

size_t lwhttp_parser_execute(lwhttp_parser_t *parser, const uint8_t *data, size_t len)

Feed response bytes to the parser, invoking callbacks as elements are recognised.

Input can be split anywhere. Parsing stops after each complete message, so with pipelined or interim (1xx) responses the caller must call again with the remaining bytes; that call starts the next message. Empty lines before a status line are skipped. After an upgrade message the parser consumes nothing more, and after an error every call returns 0.

Parameters:

• parser – The parser.

• data – Input bytes.

• len – Number of input bytes.

Returns:

Bytes consumed. Less than len when a message completed or an error occurred; check lwhttp_parser_is_complete() and lwhttp_parser_error().

lwhttp_error_t lwhttp_parser_finish(lwhttp_parser_t *parser)

Tell the parser the stream has ended.

Completes a close-delimited body (calling on_message_complete). A message that is otherwise unfinished puts the parser in the error state.

Parameters:

• parser – The parser.

Returns:

LWHTTP_OK if no message was in progress or the message is now complete, LWHTTP_ERR_CLOSED for a truncated message, LWHTTP_ERR_ABORTED if on_message_complete aborted, or the earlier error if the parser had already failed.

bool lwhttp_parser_is_complete(const lwhttp_parser_t *parser)

Check whether the last lwhttp_parser_execute() or lwhttp_parser_finish() call ended a message. Stays true until the next message starts.

Parameters:

• parser – The parser.

Returns:

true if a message is complete.

bool lwhttp_parser_keep_alive(const lwhttp_parser_t *parser)

Check LWHTTP_MSG_KEEP_ALIVE for the current message. Meaningful once the header section has been parsed.

Parameters:

• parser – The parser.

Returns:

true if the connection may be reused after this message.

lwhttp_error_t lwhttp_parser_error(const lwhttp_parser_t *parser)

Get the error that stopped the parser.

Parameters:

• parser – The parser.

Returns:

LWHTTP_OK if the parser has not failed, otherwise LWHTTP_ERR_PROTOCOL, LWHTTP_ERR_VERSION, LWHTTP_ERR_TOO_LARGE, LWHTTP_ERR_HEADERS_TOO_LARGE, LWHTTP_ERR_ABORTED or LWHTTP_ERR_CLOSED.