Announcing: http-reverse-proxy, unix-process-conduit, and more
October 4, 2012
I'm happy to announce the release of three new packages: http-reverse-proxy, unix-process-conduit, and network-conduit-tls. Additionally, there is a new version of network-conduit which provides an improved interface.
Believe it or not, there's a reason that I'm announcing these four different packages at the same time: put together, they provide a lot of the functionality necessary to create a virtual hosting system working over both HTTP and HTTPS. This work directly benefits the keter deployment system. I'm still testing these changes to keter, so the new version 0.3 has not yet been released.
These new packages should be considered experimental. They are the basis of some real life code at FP Complete, but it's early in the testing cycle, so I cannot give any strong assurances that it will work perfectly. Nonetheless, these packages provide some interesting features that could be useful to others.
network-conduit and network-conduit-tls
network-conduit has been around for a while now. It provides a simple
conduit-based interface for creating TCP servers and clients. This 0.6 release
enhances the interface a bit:
- You can register an action to be called after binding the listening socket. This is useful for apps which bind to a restricted port and then setuid to a non-privileged user.
SockAddris provided to both server and client applications.
- By using settings types, it should be easy to make additions to the API going forward without breaking backwards compatibility.
The new addition here is
network-conduit-tls, which allows you to create TCP
servers over an SSL connection. What's very nice is that it uses the same
interface for applications as the regular version, so you can write
applications that will run over either secure or insecure. This comes into play
with the next library.
network-conduit-tls doesn't provide a client interface.
There are some complications involved there, such as providing a certificate
approval mechanism. These aren't difficult problems necessarily
http-conduit already does this, for example) but I decided not to include
the functionality right now. If people are interested in adding it, please
This library provides two different ways of reverse proxying. The more traditional approach leverages both WAI and http-conduit: it provides a WAI application and makes requests to some backend host. In this scenario, the request and response are fully parsed by the reverse proxy, which provides opportunities for making modifications, but can also introduce a performance overhead. Additionally, this approach can get in the way of tunneling non-HTTP data, such as websockets.
The second approach is more light-weight. Only the initial HTTP headers are parsed, in order to determine how to route your request. Then, all further communications are simply passed to and from the backend server without any further parsing. This means that, after the initial HTTP headers, the content could be completely invalid HTTP... which in the case of websockets is actually an advantage.
Both approaches allow you to have a very dynamic routing table: you must provide an action which will be run on each connection to determine where to route to. You could even go so far as to modify the routing table based on incoming HTTP requests if you wanted to.
One final feature is the ability to convert a WAI application to a raw
network-conduit-style application. This is done by using some low-level
functions provided by Warp to parse and render the requests and responses. This
makes it possible to provide a single server which can either locally serve
responses, or pass on to a different server, depending on the request headers.
The main motivation behind the creation of this library was actually not for
conduit interface. It was mainly to be able to have better control of
termination of processes. To quote
This function should not be used under normal circumstances - no guarantees are given regarding how cleanly the process is terminated.
For my use cases, I needed guarantees that a process was really going to die.
We actually ran into this problem historically with
yesod devel, and used
some complicated techniques with flag files to work around it. If we can come
up with a Windows counterpart to
unix-process-conduit, we'll be able to avoid
But once I was writing a library for processes, I decided to go ahead and model
the input and output streams via conduits. The
forkExecuteFile function takes
Maybe (Source IO ByteString) parameter for stdin. If it's
the subprocess keeps the same standard input as the parent. Otherwise, stdin is
taken from the given
Source. If all you want to do is provide no input to the
child, you can use
Just $ return ().
Likewise, for stdout and stderr the parameter type is
Maybe (Sink ByteString
IO ()). To simply ignore all output, you can use
As I mentioned, all of the new libraries are still to be considered experimental. If you have ideas for API improvements or find any bugs, please let know.