I wanted to post details while it's still fresh, so if you skip my technical posts, I am hereby warning you that this is a technical post (hopefully you have time to sometimes watch the videos I post though, I try to make them at least interesting).
Today's topic is Websockets. This is a technology that operates like UN*X sockets, but between a web browser running Javascript and some sort of server, and possibly tunneling through the same port as the HTTP (or HTTPS) protocol (depending on how it is set up). The fact it can use port 80 (HTTP) or 443 (HTTPS) is important because if it used a different port, people trying to connect with it (especially from work) are often blocked from other ports. So, using the same ports as web services means they have access to Websockets facilities on the server. I started trying to figure out how to do this, oh, back in February. There is a paucity, and possibly dearth, of documentation on how to get it running, so I am going to document what I did here. In particular, I needed to set up encrypted Websockets through the Apache web server I'm running for a bunch of virtual domains (no half measures here... and because sensitive information might be going through the link, using SSL encryption was critical for me). I can't believe how many sites I had to visit and how much documentation I had to read, and how much of it was nearly useless for my needs before I found an extremely elegant solution to doing it. The hint about how to do this without six weeks of coding was here:
Websocketd behind Apache (2.4.x). It was not a slam dunk, but it set me on the path to getting this working.
So, step one was figuring out what
websocketd was, and how it worked. It was also the key to making the job as painless as possible. While I may need to do something different should I ever require more performance (would that I should be so lucky to need that), this will allow me to develop all the web user interface code I need for the product I'm working on and a bunch of other ideas I've had. Because of the "socket" interface paradigm, it means that the client running in the web browser and the code running on my server are one seamless application, with the web browser acting as the user interface and the server having access to databases and other "backend" stuff. Where
websocketd is just a beautiful piece of software, is that it handles all the network stuff, and the server code just uses
stdin and
stdout to talk to the client running in the web browser (again, written in Javascript)... all of the nasty network stuff is handled by
websocketd (and Apache in my case, but
websocketd can also serve up content as a standalone web server if desired). Every time a secure Websockets request is made, a new instance of the server program is invoked and, as stated above, does all its communications over Websockets through its UN*X standard I/O. It is so elegant, and allows the functionality of the server code to be tested (using
stdin and
stdout like a proper UN*X program) without needing any network or weird jigs running in Javascript. It also means I can just write code and not have to worry about encryption, connections, HTTPS, or anything else. It will "just work".
Of course, nothing ever "just works". Ever. There are binaries available for
websocketd for a number of operating systems, but because of the potential sensitivity of the data I will be handling (people's schedules and other personal information), I don't want to use any product that I can't inspect the source code for.
websocketd is open source, and it is written in
Go (the language). By having the source code, I can read the source code to make sure there are no loggers or packet mirrors or anything in it (theoretically, it's possible the GO compiler may insert back doors, but that's being extra paranoid). To build the code, I downloaded
websocketd from GitHub,
here (I just downloaded a .zip file, but you could use a
git client just as easily). It downloaded "websocketd-master.zip", which unzips into the "./websocket-master" subdirectory. Change directory into the subdirectory and all that is apparently needed is to run "make". This downloads the Go language compiler and anything else needed and builds
websocketd. Of course, this didn't work. The fix, however, is trivial (after an hour and a half of searching for solutions... my bug report is
here). It was barfing saying "imports context: unrecognized import path 'context'". After some DuckDuckGoing (I use
DuckDuckGo instead of Google because they don't make money by tracking your searches and selling the information, thus I was not Googling), I found a reference saying that the "context" module was not introduced until version 1.7 of Go, and typing "./go --version" (in the "websocketd-master" subdirectory, where Go was successfully installed for the build) said it was version 1.4. So, the fix was to simply change the "GO_VER=1.4" line in the
Makefile to a version 1.7 or greater. I tried "GO_VER=1.7" and that worked, but the latest Go is version 1.11, so I tried "GO_VER=1.11" and that worked too. To jump ahead a little, the
websocketd compiled with Go version 1.11 worked like a charm. The binary needed is "./websocketd-master/bin/websocketd". There is a
10 Second Tutorial on how to create a client/server system that uses Websockets (just scroll down). I chose the C code, because that will be what I run for my backend (this example is right off the
websocketd web site, I called the program "wstest"):
#include <stdio.h>
#include <unistd.h>
int main() {
int i;
// Disable output buffering.
setbuf(stdout, NULL);
for (i = 1; i <= 10; i++) {
printf("%d\n", i);
usleep(500000);
}
return 0;
}Writing a server is as simple as that. To write a client, just a few lines of Javascript and HTML were needed. Here's my full file (the
websocketd "10 Second Tutorial" only includes the Javascript), which I called "wstest.html":
<!DOCTYPE html>
<html>
<head>
<title>WS test</title>
<meta charset="UTF-8">
</head>
<body>
<script>
var ws = new WebSocket('ws://<my server's local hostname on my LAN>:8080/');
ws.onmessage = function(event) {
console.log('Count is: ' + event.data);
};
</script>
</body>
</html>And there's the necessary code for the client (okay, it's not good code, but it was good enough for a "hello world" sanity test). Sweet! Since I was running the browser on a computer in my living room, and not on my server, I needed to use the server's local hostname on my LAN. If I had been running the web browser on my server, I could have used "ws://localhost:8080/".
To test the server, I just needed to run the
websocketd program thusly (port 8080 is a standard test port when testing web apps, most any port number could be used):
./websocketd --port=8080 ./wstest
Done. I could load the "wstest.html" locally using Firefox, and I saw the glorious count up: 1 2 3 4 5 6 7 8 9 10. Truly amazing! To be explicit, the count was output to the "Web Developer->Web Console" window, that you need to open to see the count (that's just the way the simple Javascript code was written, there are other more clever ways, but this is cheap and dirty). Now, this was an easy test, because it was my web browser running on a computer on my LAN, talking to the
websocketsd program running on my server on my LAN, but it did prove that
websocketd did work as advertised (it created a socket connection between my browser and the server code I wrote). The real test was to put it on the other side of my Apache web server and do the same thing. This took the rest of the day... there is no clear documentation on how to get it all to work together, especially using secure Websockets in a virtual host SSL environment. Part of the problem was that, because of my web server configuration, I had to skip straight to getting everything running (encrypted Websockets on a virtual domain) and couldn't do any of the intermediate steps that would have been logical progressions. Luckily, I didn't have to resort to munching on my Apache configuration to do those intermediate steps, but it did take a lot of trial and error. Skipping to the solution, I had to add the following lines to my Apache virtual server configuration file (inside the definition for the domain I wanted to use, and inside the HTTPS port 443 configuration in particular):
<VirtualHost *:443>
<virtual hosting stuff... see previous post on subject>
SSLEngine on
SSLProxyEngine on
# Edit 2022-10-02: with Apache 2.4.43, the following flag had to be added
SSLProxyCheckPeerName off
ProxyRequests Off
ProxyPass "/wss/wss-test/" "wss://localhost:8080/"
ProxyPassReverse "/wss/wss-test/" "wss://localhost:8080/"
<more virtual hosting stuff>
</VirtualHost>
After changing the configuration, Apache does need to be restarted, fyi. Now because I was doing secure Websockets, I needed to change the client code to use the "wss" protocol (which goes to HTTPS port 443, rather than HTTP port 80, and thus will connect to my virtual server on port 443 as configured above). I called this version "wsstest.html":
<!DOCTYPE html>
<html>
<head>
<title>WSS test</title>
<meta charset="UTF-8">
</head>
<body>
<script>
var wss = new WebSocket('wss://www.<my domain name>.com/wss/wss-test/');
wss.onmessage = function(event) {
console.log('Count is: ' + event.data);
};
</script>
</body>
</html>The "/wss/wss-test" directory does
not need to exist on the server, it's just a key to the proxy tunnel for Websockets in Apache to know it has to forward the communications to a particular port ("localhost:8080" in this case, since it's all running on one server) to handle the secure Websockets request. Again, because I was using secure Websockets for the test, I needed to change the
websocketd invocation to let it know it needed to do secure Websockets as well. So, instead of the simple invocation above, the proper invocation for SSL-enabled Websockets is:
./websocketd --port=8080 --ssl --sslcert=<my certificate file>.crt --sslkey=<my key file>.key ./wstest
Firstly, notice that I didn't need to change the "wstest" program at all, it works exactly the same way, and all the complexity is hidden from it. So elegant! The second thing to notice is the SSL directives on the command line. The certificate file path is exactly the same as is specified in the virtual host on port 443 configuration, specifically in the "SSLCertificateFile" directive. Similarly, the key file is exactly the same as given to the "SSLCertificateKeyFile" directive in the virtual host configuration. Again, this took hours of trial and error, but I loaded the "wsstest.html" file, and 1 2 3 4 5 6 7 8 9 10. Beers all around!
One thing to mention, that cost me a lot of time, is that because I have a virtual host setup on my server through Apache, I had to use the external domain name for the virtual host whose configuration I changed to support Websockets. I had not thought it through and had been using the local host name on my LAN and, of course, couldn't get it working. The hint was that the access request was being logged in my generic access log file, and not the access log file for the virtual domain I was trying to test on. Once I realized that, I guessed what the problem was and changed the client code to the above.
One last note, to run Websockets through the Apache server, you need to have both the
mod_proxy and
mod_proxy_wstunnel loaded in Apache. For my configuration (I run Slackware), I needed the following two lines in my global configuration file ("httpd.conf" in my case):
LoadModule proxy_module lib64/httpd/modules/mod_proxy.so
LoadModule proxy_wstunnel_module lib64/httpd/modules/mod_proxy_wstunnel.so
Getting Websockets running was one of those things that took me a lot longer that it should have, and was way harder than needed, because there was no documentation that I could find that covered all of my particular requirements (which don't seem that "out there" to me, so I can only assume many others struggle with this exact scenario, thus this post): a need for Secure Websockets running through an Apache web server running a virtual hosts configuration. I do like that this solution is pretty close to being "on the bare metal" compared to many of the middleware approaches I've read about, which is where I personally like to operate if I can.
And now, the video... this one is their tamest one, and it's a pretty entertaining "dance" music video (almost kid friendly even, you'll have to be the judge). Funny as heck, imo :). As a warning, if you go and look at their other videos, you may not want to do it at work (some are
very NSFW!). From Russia with wtf.
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
