LIBWEB_HTTP(7)

NAME
libweb_http - libweb HTTP connection handling and utilities

SYNOPSIS

#include <libweb/http.h>

DESCRIPTION
As one of its key features, libweb provides a HTTP/1.1-compatible server implementation that can be embedded into applications as a library. While not a complete HTTP/1.1 server implementation, the following features are supported:

•   GET.

•   HEAD.

•   PUT.

•   POST. multipart/form-data-encoded data is also supported.

•   Cookies. An optional payload size limit can be defined for PUT and POST requests (see section HTTP server configuration).

  Utility functions
The functions listed below are meant for library users:

•   http_response_add_header(3).

•   http_cookie_create(3).

•   http_encode_url(3).

•   http_decode_url(3).

  HTTP connection-related functions
The functions listed below are meant for internal use by libweb:

•   http_alloc(3).

•   http_free(3).

•   http_update(3). However, this component alone does not provide a working web server. For example, a list of endpoints is required to define its behaviour, and struct http objects must be stored somewhere as long as the connections are active. libweb_handler(7) is the component meant to provide the missing pieces that conform a working web server.

  HTTP server configuration
A HTTP server is contained into a struct http_ctx, and can be allocated by calling http_alloc(3). This function requires a valid pointer to a struct http_cfg object. This flexible configuration allows the library to run on top of any reliable transport layer, including TCP. struct http_cfg is defined as:

    struct http_cfg
    {
        int (*read)(void *buf, size_t n, void *user);
        int (*write)(const void *buf, size_t n, void *user);
        int (*payload)(const struct http_payload *p, struct http_response *r, void *user);
        int (*length)(unsigned long long len, const struct http_cookie *c, struct http_response *r, void *user);
        const char *tmpdir;
        void *user;
        size_t max_headers;
        struct http_cfg_post
        {
            size_t max_pairs, max_files;
        } post;
    };

All of the function pointers listed above define user as a parameter, an opaque pointer to user-defined data previously defined by member user (see definition below). Unless noted otherwise, all pointers must be valid. read is a function pointer to a read(2)-like function that must read up to n bytes from the client into a buffer pointed to by buf. The function pointed to by read returns the number of bytes that could be read from the client, which could be from zero up to n. On error, a negative integer is returned. write is a function pointer to a write(2)-like function that must write up to n bytes to the client from a buffer pointed to by buf. It returns the number of bytes that could be written to the client, which could be from zero to n. On error, a negative integer is returned. payload is a function pointer called by libweb when a new HTTP request has been received. p is a read-only pointer to a struct http_payload object, which describes the HTTP request (see section HTTP payload). r is a pointer to a struct http_response object that must be initialized by the function pointed to by payload, which includes the HTTP response parameters to be returned to the client. The function pointed to by read returns the number of bytes that could be read from the client, which could be from zero to n. This function returns zero on success. On error, a negative integer is returned. length is a function pointer called by libweb when an incoming HTTP request from a client requires to store one or more files on the server. In the case of a POST request, len defines the length of the multipart/form-data (see section Content-Length design limitations for multipart/form-data). In the case of a PUT request, len defines the length of the file body. c is a read-only pointer to a struct http_cookie object, containing at most one (see section Limitations on the number of HTTP cookies) HTTP cookie. If no cookies are defined, its members shall contain null pointers. r is a pointer to a struct http_response object that must be initialized by the function pointed to by payload only when the function returns a positive integer. This function returns zero on success, a negative integer in case of a fatal error or a positive integer in case of a non-fatal error caused by a malformed request, or to indicate a lack of support for this feature. When a positive integer is returned, the connection against the client shall be closed. tmpdir is a null-terminated string defining the path to a directory where files uploaded by clients shall be stored temporarily. tmpdir can be a null pointer if this feature is not supported by the application. user is an opaque pointer to a user-defined object that shall be passed to other function pointers defined by struct http_cfg. user can be a null pointer. max_headers refers to the maximum number of header fields that shall be passed to the struct http_payload object passed to the function pointed to by payload. Any extra headers sent by the client outside this maximum value shall be silently ignored by libweb. post contains configuration parameters specific to POST requests: max_pairs refers to the maximum number of key/value pairs that shall be accepted by libweb on POST multipart/form-data-encoded requests. If the maximum number of pairs is exceeded by the request, libweb shall terminate the connection. max_files refers to the maximum number of files that shall be accepted by libweb on POST multipart/form-data-encoded requests. If the maximum number of files is exceeded by the request, libweb shall terminate the connection.

  HTTP payload
When a client submits a request to the server, libweb prepares a high-level data structure, called struct http_payload, and passes it to the function pointer defined by struct http_cfg member payload. struct http_payload is defined as:

    struct http_payload
    {
        enum http_op op;
        const char *resource;
        struct http_cookie cookie;
        union
        {
            struct http_post post;
            struct http_put put;
        } u;
        const struct http_arg *args;
        size_t n_args, n_headers;
        const struct http_header *headers;
        bool expect_continue;
    };

op describes the HTTP/1.1 operation. See the definition for enum http_op for an exhaustive list of supported operations. resource describes which resource is being requested by the client. For example: /index.html. cookie contains at most one HTTP cookie, defined as a key-value pair. Its members shall be null pointers if no cookie is present. Also, see section Limitations on the number of HTTP cookies. u defines a tagged union with operation-specific data. For example, post refers to data sent by a client on a POST request (see section HTTP POST payload), whereas put refers to data sent by a client on a PUT request (see section HTTP PUT payload). args defines a list of key-value pairs containing URL parameters. Its length is defined by n_args. headers defines a list of key-value pairs containing header fields. Its length is defined by n_headers. expect_continue shall be set to true if an Expect: 100-continue HTTP header is received, false otherwise (see section Expect: 100-continue handling). This field is only relevant for PUT or POST requests.

  HTTP POST payload
As opposed to payload-less HTTP/1.1 operations, such as GET, POST operations might or might not include payload data. Moreover, such payload can be encoded in two different ways, which libweb handles differently:

•   application/x-www-form-urlencoded: suggested for smaller payloads. libweb shall store the payload in memory, limiting its maximum size to 7999 octets.

•   multipart/form-data: suggested for larger and/or binary payloads. libweb shall store each non-file name-value pair in memory, limiting the value length to 8000 octets. On the other hand, libweb shall store each file into the temporary directory defined by struct http_cfg member tmpdir. This information is contained into a struct http_post object, defined as:

    struct http_post
    {
        const char *data;
        size_t nfiles, npairs;
        const struct http_post_pair
        {
            const char *name, *value;
        } *pairs;
        const struct http_post_file
        {
            const char *name, *tmpname, *filename;
        } *files;
    };

When application/x-www-form-urlencoded-data is included, data shall contain a null-terminated string with the user payload. Such data can be decoded with the functions provided by libweb_form(7). Otherwise, data shall be a null pointer. In the case of multipart/form-data, files shall contain a list of files that were uploaded by the client, each one stored by the server to a temporary file, defined by tmpname. The final name for the uploaded file is defined by filename. The key name used for each requested file is defined by name. The length of this list is defined by nfiles. If no files are defined, files shall be a null pointer. In the case of multipart/form-data, pairs shall contain a list of name-value pairs that were uploaded by the client, defined by name and value, respectively. The length of this list is defined by npairs. If no name-value pairs are defined, pairs shall be a null pointer.

  HTTP PUT payload
For PUT requests, and as opposed to POST requests, libweb shall always store the request body into a temporary file, defined by struct http_put member tmpname. This information is contained into a struct http_put object, defined as:

    struct http_put
    {
        const char *tmpname;
    };

  Expect: 100-continue handling
As opposed to other HTTP headers, the Expect: 100-continue HTTP header requires the server to respond before the request body is sent by the client. Typically, this is meant to allow the server to check the request resource and headers beforehand, without waiting to receive the request body, which can be long. This is usual for resource uploads, as typically done with PUT requests. Therefore, when this header is received, libweb shall trigger the user-defined callback as defined by struct http_cfg member payload at least once. Then, users are expected to check this flag and act accordingly. The example below shows a user-defined callback that replies with HTTP_STATUS_CONTINUE when struct http_payload member expect_continue is set:

    static int on_payload(const struct http_payload *const p,
        struct http_response *const r, void *const user)
    {
        if (p->expect_continue)
        {
            *r = (const struct http_response)
            {
                .status = HTTP_STATUS_CONTINUE
            };
            return 0;
        }
        /* Handle request body. */
    }

Then, once the request body is received, libweb shall trigger the callback again with struct http_payload member expect_continue assigned to false.

  HTTP responses
Some function pointers used by libweb require to initialize a struct http_response object that defines the response that must be sent to the client. This structure is defined as:

    struct http_response
    {
        enum http_status status;
        struct http_header
        {
            char *header, *value;
        } *headers;
        union
        {
            const void *ro;
            void *rw;
        } buf;
        FILE *f;
        unsigned long long n;
        size_t n_headers;
        void (*free)(void *);
        void *args;
        int (*chunk)(void *buf, size_t n, bool *done, void *user, void *args);
        union http_step
        {
            int (*length)(unsigned long long len, const struct http_cookie *c, struct http_response *r, void *user, void *args);
            int (*payload)(const struct http_payload *p, struct http_response *r, void *user, void *args);
        } step;
    };

status is the response code to be returned to the client. A list of possible values is defined by enum http_status. headers is a pointer to an array of struct http_header whose length is defined by n_headers, containing the HTTP headers to be included into the response. Note that headers is not meant to be modified directly by library users. Instead, the http_response_add_header(3) utility function shall update the struct http_response object accordingly. buf is a union containing two possible values, with minor semantic differences: ro is a read-only opaque pointer to a buffer in memory, whose length is defined by n (see definition below). libweb shall select ro as the output payload if f, free and chunk are null pointers, and n is non-zero. rw is an opaque pointer to a buffer in memory, whose length is defined by n (see definition below). libweb shall select rw as the output payload if chunk and f are null pointers, free is a valid pointer to a function that frees the memory used by rw, and n is non-zero. f is a FILE pointer opened for reading that defines the payload to be sent to the client, whose length is defined by n. libweb shall select f as the output payload if chunk, ro, rw and free are null pointers, and n is non-zero. n is the length of the output payload, which can be either a buffer in memory (see definitions for ro and rw) or a file (see definition for f). If n equals zero, no payload shall be sent. n_headers defines the number of HTTP headers contained in the response. This field is not meant to be manipulated directly. Instead, the http_response_add_header(3) utility function shall update the struct http_response object accordingly. free is a pointer to a function that frees the memory used by rw or args only if rw or args are valid pointers. Otherwise, free must be a null pointer. args is an opaque pointer to user-defined data, as consumed by chunk and step. chunk allows library users to generate the response body dynamically, therefore relying on the Transfer-Encoding: chunked HTTP header. When used, chunk must point to a function that reads up to n bytes into a buffer provided by libweb, namely buf. The function must return how many bytes were written into buf, and must be between zero and n. In case of error, a negative integer must be returned. When no more chunks are to be sent, users must assign the flag pointed to by done to true. If not null, free shall be called when either all chunks have been transferred or whenever an error occurs, with args as its argument. step allows implementations to deal with responses asynchronously i.e., without blocking other clients. For example, this can be useful to generate a response in a non-blocking manner. In other words, when a response is not immediately available when a payload is received, step must be assigned to a function that can generate it later. libweb shall then call this function immediately later, without blocking other clients. If not null, free shall be called whenever an error occurs, with args as its argument. Do not confuse step with chunk, where chunk is meant for dynamically generated message bodies only, where the response HTTP status and headers have already been defined. Assigning step.length or step.payload to a null pointer falls back to the default behaviour i.e., a response is returned immediately. Note that a non-null step.length or step.payload shall always take priority, thus ignoring any other information inside the struct http_response instance. step can be configured under the following situations:

•   Inside the length callback defined by the struct http_cfg instance. In this case, step.length must be assigned to a function to be later called by libweb.

•   Inside the payload callback defined by the struct http_cfg instance. In this case, step.payload must be assigned to a function to be later called by libweb. Functions must assign all unused members in a struct http_response instance to 0 or NULL. Otherwise, the behaviour is undefined. For example, the following assignment sets three members, namely status, buf.ro and n, while leaving all unused members to 0 or NULL:

    static const char body[] = "Hi there!\n";
    *r = (const struct http_response)
    {
        .status = HTTP_STATUS_OK,
        .buf.ro = body,
        .n = sizeof body - 1
    };

  Transport Layer Security (TLS)
By design, libweb does not implement TLS (Transport Layer Security). It is assumed this should be provided by a reverse proxy instead, a kind of project that is usually maintained by a larger community than libweb and audited for security vulnerabilities.

NOTES

  Comparing against other HTTP server implementations
While it is well understood that other solutions provide fully-fledged server implementations as standalone executables, libweb strives to be as small and easy to use as possible, intentionally limiting its scope while covering a good range of use cases.

  Content-Length design limitations for multipart/form-data
HTTP/1.1 defines the Content-Length for a multipart/form-data POST request as the sum of:

•   The length of all files.

•   The length of all boundaries.

•   The length of all headers included on each part.

•   All separator tokens, such as LFCR or --. This means it is not possible for libweb to determine the number of files or their lengths in a HTTP request unless the whole request is read, which not might be possible for large requests. Therefore, the Content-Length is the only rough estimation libweb can rely on, and therefore is the value passed to the length function pointer in struct http_cfg.

FUTURE DIRECTIONS

  Limitations on the number of HTTP cookies
So far, libweb shall only append at most one HTTP cookie to a struct http_payload object. This is due to arbitrary design limitations on the library. Future versions of this library shall replace the struct http_cookie object inside struct http_payload with a pointer to an array of struct http_cookie, plus a size_t object containing the number of HTTP cookies in the request.

SEE ALSO
handler_alloc(3), http_alloc(3), http_free(3), http_update(3), http_response_add_header(3), http_cookie_create(3), http_encode_url(3), http_decode_url(3), libweb_form(7).

COPYRIGHT
Copyright (C) 2023-2025 libweb contributors

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.