Even if libweb already parses some common headers, such as
Content-Length, some users might find it interesting to inspect which
headers were received from a request.
Since HTTP/1.1 does not define a limit on the number of maximum headers
a client can send, for security reasons a maximum value must be provided
by the user. Any extra headers shall be then discarded by libweb.
An example application showing this new feature is also provided.
- http_memmem must not check strlen(a) > n because, in case of a partial
boundary, it would wrongfully return NULL.
- If one or more characters from a partial boundary are found at the end
of a buffer, but the next buffer does not start with the rest of the
boundary, the accumulated boundary must be reset, and then look for a
new boundary.
- Writing to m->boundary[len] did not make any sense, as len is not
meant to change between calls to read_mf_boundary_byte.
- For the same reason, memset(3)ing "len + 1" did not make any sense.
- When a partial boundary is found, http_memmem must still return st.
- Calling reset_boundary with prev == 0 did not make sense, since that
case typically means a partial boundary was found on a previous
iteration, so m->blen must not be reset.
This macro would return a positive integer on failure. However,
functions called by http_update should only return a positive integer
for user input-related errors, not fatal errors such as those related to
failed memory allocations.
It was found out there was another project of the same name around
(https://git.sr.ht/~strahinja/slweb/), also related to website
generation.
In order to avoid confusion, a new name has been chosen for this
project. Surprisingly, libweb was not in use by any distributions
(according to https://repology.org and AUR index), and it should
reflect well the intention behind this project i.e., being a library
to build web-related stuff.
application/x-www-form-urlencoded-data is (or should be) always text, so
it is preferrable to define struct http_post member "data" as a null-
terminated string.
For applications already making this assumption, this change should now
remove the need for string duplication.
Now, slweb accepts requests such as:
--boundary
Content-Disposition: form-data; name="field1"
value1
--boundary
Content-Disposition: form-data; name="field2"
value2
--boundary
Content-Disposition: form-data; name="field3"; filename="example.txt"
The following breaking changes have been introduced:
Member "dir" from struct http_post was a leftover from the days where
slcl and slweb were one project. It did not make sense for slweb, since
it should not decide which Content-Disposition names are allowed. In
other words, "dir" was only relevant in the scope of slcl.
Member "n" from struct http_post used to have two meanings:
- The length of a URL-encoded request.
- The number of files on a multipart/form-data request.
Since "npairs" had to be introduced to struct http_post, it did not make
sense to keep this dual meaning any more. Therefore, "n" has been
restricted to the former, whereas a new member, called "nfiles", has
been introduced for the latter.
According to C99 7.19.1p3:
BUFSIZ is a macro that expands to an integer constant expression that is
the size of the buffer used by the setbuf function.
In other words, this means BUFSIZ is the most optimal length for a
buffer that reads a file into memory in chunks using fread(3).
Note: the number of bytes sent to the client might be less than BUFSIZ,
so this would act as a bottleneck, no matter how large the buffer passed
to fread(3) is.
Since slweb is meant as a library, it is advisable to keep public header
files under their own directory in order to avoid name clashing i.e.,
#include "something.h"
Now becomes:
#include "slweb/something.h"
- '.' or '..' must not be used for filenames.
- Filenames must not contain forward slashes ('/').
- Filenames must not contain asterisks ('*') to avoid confusion with
wildcard expressions.
Under some circumstances, clients could cause SIGPIPE to slcl. Since
this signal was not handled by server.c (i.e., via sigaction(3)), slcl
would crash without any error messages printed to stderr.
In such situation, SIGPIPE should not be usually considered a fatal
error, so it is preferrable to close the connection and keep working.
Given the following contrived example request:
/example%FB%DC&arg%DE1=examplevalue%AA
slcl must decode each token separately, so that percent-encoded
characters '&', '=' or '?' do not get accidently intepreted.
Otherwise, write_body_mem and write_body_mem would silently fail,
causing undefined behaviour.
Notes:
The return value for write_ctx_free is currently assigned to that of
fclose(3), which can be either 0 on success or EOF on failure.
However, it makes sense for write_body_mem and write_body_mem to simply
check against non-zero.
Also, it would not be sensible to return EOF to caller functions, which
expect either 0 (success), -1 (fatal error) or 1 (input error).
Considering http.h defined HTTP/1.1-only responses such as "303 See
Other", as well as incoming HTTP/1.1-only features (e.g.: byte serving),
it did not make much sense to keep a somewhat broken compatibility
against HTTP/1.0.
Unfortunately, this breaks support with some existing clients such
as lynx(1), even if HTTP/1.0 was already deprecated many years ago.
However, even lynx(1) can be configured to support HTTP/1.1.
Now, http_payload includes a list of human-readable parameters that can
be read (but not modified) by users. Given the following example link:
/test?key1=value1&key2=value2
This will generate two parameters, with the following values:
{
.args =
{
[0] = {.key = "key1", .value = "value1"},
[1] = {.key = "key2", .value = "value2"}
},
.n_args = 2
}
As expected, if any URL parameters are given, struct http_payload member
"resource" is accordingly trimmed so as not to include any parameters.
Therefore, considering the example above:
{.args = {...}, .resource = "/test"}
Limitations:
- Since the definition of struct http_arg is both shared by http.h
(as a read-only pointer within struct http_payload) and http.c
(as a read/write pointer within struct ctx), its members (namely key
and value) must remain as read/write pointers, even if they must not
be modified by users of http.h.
So far, slcl would just close the connection with a client when the
Content-Length of an incoming request exceeded the user quota, without
any meaningful information given back to the user.
Now, slcl responds with a HTML file with meaningful information about
the error.
Limitations:
- While this commits has been successfully tested on ungoogled-chromium,
LibreWolf (and I assume Firefox and any other derivates too) does not
seem to receive the response from the server.
- However, this issue only occurred during local testing, but not
on remote instances.
Until now, f->tmpname was removed by move_file when the move
operation succeeded. However, since a HTTP operation can fail before
move_file is called, the temporary file must also be removed.
This feature allows admins to set a specific quota for each user, in
MiB. This feature is particularly useful for shared instances, where
unlimited user storage might be unfeasible or even dangerous for the
server.
Also, a nice HTML5 <progress> element has been added to the site that
shows how much of the quota has been consumed.
If no quota is set, slcl falls back to the default behaviour i.e.,
assume unlimited storage.
Limitations:
- While HTTP does specify a Content-Length, which determines the length
of the whole request, it does not specify how many files are involved
or their individual sizes.
- Because of this, if multiple files are uploaded simultaneously, the
whole request would be dropped if user quota is exceeded, even if not
all files exceeded it.
- Also, Content-Length adds the length of some HTTP boilerplate
(e.g.: boundaries), but slcl must rely on this before accepting the
whole request. In other words, this means some requests might be
rejected by slcl because of the extra bytes caused by such boilerplate.
- When the quota is exceeded, slcl must close the connection so that
the rest of the transfer is cancelled. Unfortunately, this means no
HTML can be sent back to the customer to inform about the situation.
Cookies without "Expires" are considered non-persistent and thus can be
removed by the web browser. Instead, slcl now sets persistent cookies
that last for 1 year.
According to C99 §7.20.3.4:
If memory for the new object cannot be allocated, the old object is not
deallocated and its value is unchanged.
Therefore, a temporary pointer must be used to ensure the original
object can still be deallocated should realloc(3) return a null pointer.
