micro HTTP server in C

The new version is out ! HTTaP_src.20200416.tgz solves many little bugs, handles MIME type better (default is no "Content-Type:" unless there is an explicit file or link), it obsoletes the version from yesterday.

I even wrote documentation in the example index.html page :-)

Two major things are still missing though :

1. The POST method for the loopback
2. The HTTaP framework

These might be developed in the next few days...

The confinement is a good period to refresh this software-only project and here is the new version !

HTTaP_src.20200415.tgz

3 years after the last version, I have cleaned up the TCP/IP code. IPv6 handling is removed because never used in practice, I added more checks and a few portability helpers that would help with MacOS and Win32. I also simplified several cases where strlen() can be replaced by sizeof() because the strings are constants.

I have a few more days to work on this and hopefully there will be more significant enhancements !

BTW, does this new version work on your computer(s) ?

Many things have happened since the last major update !

For example I have worked for a startup where I have enhanced my skills with sockets and other related fields, that will greatly benefit this server. A lot of rewrite should happen !

The updates will happen here and also on github where @glaudistong  opened a repo. He recently joined to help, so the server can be used in other situations :-)

Jeff Harris made a version of the vintage DOOM game that is controlled by a HTTP link with JSON !

http://1amstudios.com/2017/08/01/restful-doom/

The underlying TCP/IP stack is handled by the SDL framework.

• C source code
• Real-time processing, no lag allowed
• simple set of commands

Now, what makes my source code interesting ? Look at the video : each command is written in text. There is a sort of GUI but it's not tailored to the application. OTOH, using a file server allows the developer to also embed a GUI in HTML/JS that makes the interface smoother and more powerful. Instead of writing each command, write JS to handle dynamic HTML elements that react to events in real-time on your browser !

The file server is now mostly working and serves files. It's a matter of fine tuning and we can load web applications. User HTTaP resources however are a totally different matter and I know it's a terrible thing to modify the server's source code for each application. It's practical in the early stages but it's confusing and even sometimes scaring for newcomers.

The idea is to design a kind of API for adding services without touching the server's core. It must be "object-style" and totally C-compatible too.

My idea is to design a sort of tree, using a hierarchy of linked lists. Each node is easy to describe in plain C with links to the next node and the eventual sub-lists. It can be made static for compile-time generation and/or generated on the fly by the wrapper...

It's April 1st and it's not a joke !

I have a seemingly working version of the server, with the crude MIME support outlined in MIME type handling

As expected, a critical prerequisite of the code was to get the timeout and the connection persistence working well. Then a thorough parsing of request, and countless checks of boundaries and return values...

You will find these lines in the beginning of HTTaP.h :

#ifndef  ACCESS_CONTROL_ALLOW_ORIGIN
  #ifndef HTTAP_SERVE_FILES
    #define ACCESS_CONTROL_ALLOW_ORIGIN "Access-Control-Allow-Origin: *\x0d\x0a"
  #else
    #define ACCESS_CONTROL_ALLOW_ORIGIN
  #endif
#endif

This code covers two usual situations :

• #define HTTAP_SERVE_FILE :
  The server includes the static file server, the browser can fetch files and send HTTaP commands to the same IP address and port, everything is great. No need to declare CORS.
• #undef HTTAP_SERVE_FILE ;
  The server doesn't implement the file server : the browser gets the HTML pages from a different IP/port pair and the "Same Origins Policy" enforced by web clients will block HTTaP traffic. You must enable CORS so the HTML/JS files that are served by a different server (maybe Apache, on a different port and IP address) can use this server.

There are other corner cases and you are free to combine the parameters, but you must examine the risks and benefits of each configuration. If you want to override the default behaviour, just define ACCESS_CONTROL_ALLOW_ORIGIN yourself on the compiler's command line for example.

I have finally implemented the connection's persistence along with the timeout mechanism !

I'm also including the basic features of the HTTaP protocol, which I test using telnet:

The first request is the typical "GET /?" command, which returns the JSON-formatted list of features and informations. The format of this list will evolve quickly.

Any invalid or malformed request closes the connection, which happened when I hit "Enter" (resulting in nothing written on my side).

I still have to implement a lot of things !

I have chosen to avoid using a run-time configuration file. Most parameters are defined in a .h file and some can be overridden through the environment variables. Here is the list of the parameters:

• $HTTAP_TCPPORT : sets the listening port of the server. Similar to Apache's Listen in ports.conf.
• $HTTAP_KEEPALIVE : number of seconds for the HTTP timeout (bound between 3 and 200, default is 15). This is equivalent to KeepAliveTimeout in Apache's configuration file.

If you enable the "file server" (by compiling with the -DHTTAP_SERVE_FILES option), more parameters appear:

• $HTTAP_STATICPATH : when set, sets the working directory where files are stored. If running as root, chroot() to it. Similar to Apache's DocumentRoot
• $HTTAP_ROOTPAGE : specifies the file that is handed when the resource path "/" is requested.
• $HTTAP_USER : if running as root, drop admin rights and change to this user (hopefully the system is configured to give this user minimal rights) Similar to APACHE_RUN_USER
• $HTTAP_GROUP : if $HTTAP_USER is set, also change the current group ID (similar to APACHE_RUN_GROUP)

Environment variables are both simpler than a configuration file and does not interfere with command line parameters, which are often critical to the main/host application. Furthermore, it's a bit more script-friendly and allows automation. This is excellent when you run several programs simultaneously, just increase some variables in a loop and you're done.

You could configure the server (before running it) with the following script:

# change the default TCP port and keepalive values
export HTTAP_TCPPORT=61234
export HTTAP_KEEPALIVE=123
# use the current directory to serve the files
unset HTTAP_STATICPATH
# use the default root page
unset HTTAP_ROOTPAGE
# create an empty root page
touch index.html
# don't forget to run useradd and groupadd
export HTTAP_USER=httap_sandbox
export HTTAP_GROUP=httap_sandbox
# start the server's program
./HTTaP

This will (one day) serve an empty page for the root in the current directory.

The call graph is :

MyApplication.c       // contains your main() and infinite loop
   |__ HTTaP.h        // user configurable values
         |__ HTTaP.c  // the server's boring posixy code
               |___ your code to process dynamic addresses 

It also contains compile-time flags that enable certain features:

• HTTAP_VERBOSE : printf() some useful information (good for debugging)
• SHOW_CLIENT : add more verbosity
• ENABLE_LOOPBACK : defined if a Loopback server is implemented.
• HTTAP_SERVE_FILES : this is the most important switch. This enables the static files server.
  

Inside your code, you can switch from polled mode to blocking mode (and vice versa) with the following functions:

void HTTaP_polled()
void HTTaP_blocking()

The server starts in polled mode, which requires repetitive calls to the HTTaP_server() function. If your program has finished working and waits for more commands, you can save CPU cycles (and battery ?) by switching to the blocking mode. You can revert to polled when a valid command is received.

The server is designed to run almost like a simplified file server in a POSIX machine. This implies a significant attack surface and a lot of potential for abuse.

(D)DOS attacks are pretty easy to create : the server is single-threaded and uses persistent connections but no authentication. This is not a security hazard, at least in a lab environment where there should be only one user.

There are many kinds of known, unknown and potential risks that require careful coding and safe development practices. The attack surface is reduced by limiting the system to its core functionality and keeping things as simple as possible (KISS).

But what about the unknown bugs ?

One solution is to use sandboxing techniques and implement inherent UNIX protection mechanisms. This way, if the system ever goes bad, the potential damage is contained to a portion of the system. Two methods protect this server:

• If the server is started with root privileges, and if the necessary informations are provided, the current user and group ID are changed to a non-privileged user, with limited rights.
• If the server is started with root privileges, the program is chroot()ed to the current working directory (or another one, if provided) so it can't access the rest of the system.

These are "last lines of defence" methods, which are complemented by many routine checks :

• The server (if correctly configured) will not serve files or directories that it doesn't own, which prevent accidental exfiltration of data.
• URIs must be filtered, rejecting any path containing two consecutive dots.
• No open directories. This reduce coding efforts and bugs, as well as removes a potential exfiltration method.

Of course, a user can always configure the server badly...

Many security considerations are covered in « The Tangled Web - AGuide to Securing Modern Web Applications » by Zalewski, Michał