Proxying WebSockets with Nginx

WebSockets are an exciting new technology designed to make it easier to create real time applications by providing a full-duplex communication channel between the browser and the server. In layman’s terms that means that information can be sent and received by the application at the same time. One of the easiest ways to write apps using WebSockets is to make use of the excellent Socket.io library, which is used in conjunction with Node. The usual strategy when building a Node app is to put Nginx in front of Node as a reverse proxy that serves any static content. This was a problem if you wanted to use WebSockets though, as Nginx didn’t know how to proxy those requests. Until now…

Last week, with the 1.3.13 release, Nginx has gained the ability to proxy these WebSocket requests (thanks to Apcera and CloudBees for sponsoring this work). It’s quite easy to do, and I’ll show you how. I’m assuming you’re running a reasonably new release of Ubuntu Linux. The first thing you’ll want to do is install the needed software. You can do this easily by using my PPA repositories for Node and Nginx respectively which have ready made packages.

We’ll also need an example app that uses Socket.io. Michael Mukhin has an easy, ready to go chat app tutorial that we’ll be making use of. Be sure to get the latest code from his github repository. You shouldn’t need to go through the installation steps for Node since my packages will take care of that for you. I’ll be assuming you have his chatroom app up and running correctly at:

http://localhost:8080

Now, if you wanted to serve this app in production, you obviously would want it to listen on port 80, and this is where Nginx comes in. We’ll make a basic configuration file called “chat” located at:

/etc/nginx/sites-enabled/chat

It should contain the following:

server {
    listen 80;
    root /home/chl/chat;
    index index.html index.htm;
 
    server_name _;
 
    location / {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
    }
}

You’ll want to change the root parameter to wherever you installed the chatroom app. Also, you should edit the index.html file in the chatroom app so that the Socket.io connection is made on port 80, so the 4th line should look like this:

<script src="/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js"></script>
<script>
        var socket = io.connect('http://localhost');
        // on connection to server, ask for the user's name with an anonymous callback

Once this is done, you should be able to start Nginx, and simply visit:

http://localhost

and things should work correctly. The important part of the Nginx config for all of this is the lines:

proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

The first line tells Nginx to use HTTP/1.1 when communicating to the Node backend, which is required for WebSockets. The next two tell Nginx to respond to the Upgrade request which is initiated over HTTP by the browser when it wants to use a WebSocket.

In production, you’d likely want to add additional location stanzas to Nginx to tell it where to serve static assets from, set expires headers, and so on. You’d also likely want to manage the Node process(es) with an init script or supervisor, so that the app would start automatically when the server booted up. But, in an nutshell, this is pretty much it for using Nginx with your WebSocket enabled application! Questions and comments are always welcome of course.

43 thoughts on “Proxying WebSockets with Nginx

    1. chris lea Post author

      I’m sorry but I’m not sure I understand your question. You can use proxy_pass without the Upgrade header, but that is designed specifically to proxy HTTP. It is not a generic TCP proxy, so it won’t work properly with WebSockets. If you want it to proxy WebSockets correctly you must use HTTP/1.1, set the Upgrade header, and set the Connection header. Is that what you’re asking?

      Reply
  1. Audun

    Hi.. Thanks for the great post! I have a question for you.. When using upstreams in nginx and socket.io (multiple processes) it seems like it connects to all the online nodejs processes.
    So i have an expressjs site running on port 3001->3005 and nginx available on :80. Firebug goes nuts with all the heartbeats from all the four servers. Do you know how I can instruct the client or nginx to keep the socket.io connection to just one of these nodejs processes?

    Thanks

    Reply
    1. chris lea Post author

      It’s possible your issue has something to do with the way you’re using Nginx as a load balancer. Specifically, you should probably be using the ip_hash directive in your upstream declaration. Check out the upstream docs for more information.

      Reply
      1. Audun

        Hi and sorry for the late reply.. The ip_hash seems to work as you described in my small scale test.. Hopefully it will in the full scale to hehe.. Thanks Chris!

        Reply
  2. Steve Edson (@SteveEdson)

    Hi Chris,

    This may be an issue with socket.io but I am having some trouble. I have setup the latest node + nginx packages linked above. I have managed to get the proxy working to some extent, my socket.io client JS is being served through the proxy successfully. Also, I get the message `Connection Accepted` in my terminal.

    However, my client doesn’t think that its connected. Everything works fine when switching back to my 8080 port.

    Any idea why this could be? Is it to do with my config? Or could it be something else, like AWS security groups?

    Reply
    1. chris lea Post author

      Hi Steve -

      Hrm, I’m afraid that’s not enough to really go on for me to troubleshoot. I’d recommend using tcpdump to sniff the traffic and make sure it’s sending what you expect in terms of the upgrade request and response.

      Reply
  3. Jason Valdron

    This feature would be really useful in a project I’m working on.

    I’m trying to setup WebSockets behind Nginx on my EC2 server. I’m running on an Elastic Beanstalk app right now. I’ve updated my AMI image to use Nginx 1.3.13, which should support WebSockets just fine. I’m using Socket.IO with Node.JS.

    However, when trying to connect to my WebSocket, in Chrome, I get: WebSocket connection to ‘wss://api-hostname/socket.io/1/websocket/GEe1c0_kwVWcZHHhq0P3′ failed: Unexpected response code: 502.

    In my config file, I have:

    upstream nodejs {
    server 127.0.0.1:8081;
    keepalive 256;
    }

    server {
    listen 8080;

    location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://nodejs;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection “upgrade”;
    }

    }

    Would you have any idea what’s causing this?

    Reply
    1. Alexander Kuritsyn

      I had a similar issue. After some debugging I found this in my logs:

      [error] 2708#0: *66 upstream prematurely closed connection while reading response header from upstream, client: 46.37.133.246, server: api.seinnova.eu, request: “GET /socket.io/1/websocket/DcxDHRoz1GkUiZpf3pHy HTTP/1.1″, upstream: “http://127.0.0.1:3000/socket.io/1/websocket/DcxDHRoz1GkUiZpf3pHy”, host: “api.mysite.com”

      It appears to be a problem with the buffer size used for reading from upstream

      I solved this problem by adding following directives to my nginx vhost config:

      In the server section I’ve added following:

      large_client_header_buffers 8 32k;

      In the location section I’ve added this:

      proxy_buffers 8 32k;
      proxy_buffer_size 64k;

      Of course, numbers can be different in your case.

      HTH

      Reply
    1. chris lea Post author

      Sure, but 1) compiling in an external module in isn’t something that most people do, and 2) not being able to listen on port 80 screws up a lot of out-of-the-box functionality provided by many WebSocket libraries. Kind of a deal breaker.

      Reply
  4. Robert

    Chris, how would I do this if I wanted to deploy a socket.io app with nginx under mydomain.com/foo/ (location /foo/) ? I guess I have to configure the socket.io resource option but I failed to get it to work correctly. My web page loads but the socket.io connection fails. Any advice?

    Reply
  5. aarellanor

    Hey Chris, good post! I’m facing the very same issue as the one Jason Valdron posted about a month ago.

    I’m running nginx 1.3.15 using your PPA, the very same configuration you showed, and a node.js+socket.io+express application. But I kept getting the error “failed: Unexpected response code: 502.”.

    After a few seconds it works, but I assume socket.io is ussing xhr-polling at that point.

    Any ideas?

    Reply
    1. Jason Valdron

      I’m running under EC2 behind a load balancer. The load balancer was setup to forward HTTP requests from 80 to 8080, I had to change it to forward TCP requests from 80 to 8080.

      Reply
      1. aarellanor

        I’m also running under a EC2 instance, but afaik I’m not behind a load balancer. I’m though listening on port 80 for several servers within nginx, proxying them to different ports. I don’t know if this could cause a problem with proxying websockets.

        Reply
        1. chris lea Post author

          My understanding is that this shouldn’t cause a problem. As long as you’re using a new enough version of Nginx, you should be able to proxy things however you want as far as I know. As long as you’re following the above instructions it should work.

          Reply
  6. Uday Phalgun

    Hi chris, really helpful post. I was wondering if the nginx configuration work for an application on tomcat?We have an app running on port 8080 and one of our customers wouldnt want to open their firewall, would be great If you can advise on it.

    Reply
    1. chris lea Post author

      I don’t have much experience with Tomcat but I certainly can’t see why that would be a problem. You’d just want to make sure that port was specified in the proxy_pass directive as shown in the example above. Specifically, something like:

      location / {
          proxy_pass http://localhost:8080;
          ...
      }
      

      I’d be curious if it didn’t seem to be working, but I’d need hand holding to set up a test case and try and figure out what’s going on (I’m really not a Java guy, and know little about operating Tomcat).

      Reply
  7. jaisis

    Can nginx be used to proxy a websocket server running on windows.

    I have tried to set up IIS 8 in Winserver 2012, for reverse proxying a websocket server app on windows. But cannot get it to work, and there’s little information on how to do it, or if it even works.

    I want to try nginx on Windows. Seems like it can only be run on the console, as opposed to a service. Can it be setup for proxying webseockets?

    If I install nginx on linux, and follow details in your article, can it be used to proxy to a different webserver. In your examples, it seemed like all configs had localhost, with redirection to a different port. Opening between servers within the local intranet, is not an issue for my deployment.

    Thanks.

    Reply
    1. chris lea Post author

      I have essentially zero experience with running IIS at this point, so I don’t know what would or would not work there with respect to Nginx. However, you should absolutely be able to set up a Linux server running Nginx and have it proxy to your IIS server. There’s nothing that says you can only proxy to localhost, any IP address or FQDN will work correctly. So if your IIS server was, for example, at IP address 10.7.15.5, then something like

      1
      2
      3
      4
      5
      6
      7
      
          location / {
              proxy_pass http://10.7.15.5:80;
              proxy_http_version 1.1;
              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection "upgrade";
              proxy_set_header Host $host;
          }

      should work just fine.

      Reply
      1. jaisis

        Chris, thank you for replying. I am setting up a linux server with nginx, and will configure for windows websocket server.

        Reply
        1. jaisis

          I have set up nginx and the websocket server. In my client, when I connect to ws://nginxservername, I get an error Unexpected response code: 200.

          There is a record of this connection in nginx access.log, but I don’t see any connections to my websocket server.
          There is nothing in the nginx error.log

          Is there any other way to debug this.
          Thanks for any insight.

          Reply
    1. chris lea Post author

      I’ve actually never tried this so I don’t know. I’d ask on the Nginx mailing list if you don’t have an easy way of testing it yourself.

      Reply
    1. chris lea Post author

      I would guess so. As I understand it, the WebSockets protocol doesn’t rely on any sort of HTTP keepalive functionality, so I don’t think they should interfere with each other.

      Reply
  8. code rider

    Hi Chris, Thanks for the tutorial. I have just one question.
    Does it handle data framing and unmasking so at the server we could receive data in text format.
    Thanks

    Reply
    1. chris lea Post author

      My understanding is that is is just a transparent proxy. Meaning that it doesn’t do anything involving framing or masking, it just sends the data directly through to the upstream back end service.

      Reply
  9. Bhushan

    Hi, I need a suggestion from you,
    In our institution, we have a server already configured and running with apache. (I do not have access to modify the apache config). We have a webpage(http://www.facweb.iitkgp.ernet.in/~rbahadur/prince/home.html?home) in this server which is up and running. Now my interest is to take request from user through this apache server and redirect the request (run a bio related application in other server on intranet) to my personal workstation in which I have a django site deployed with nginx. In my workstation I will do the processing with my algorithms and return the results to clients through the apache server. With my workstation i can not directly speak to outside world, so what are the best practices to deal with this issue (using ajax or ??). Please suggest me which direction should i go. I can configure my nginx only.

    Reply
  10. alle

    Hi there,

    i have this configuration:

    Nodejs listen to port 8080

    Nginx configuration:

    location / {
    root html;
    index index.html index.htm;
    }

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    location /socket.io/ {

    proxy_pass http://127.0.0.1:8080;
    proxy_http_version 1.1;

    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection “upgrade”;
    proxy_read_timeout 86400;

    access_log off;
    error_log /opt/nginx/logs/websockets.error.log;
    }

    the application on node is:

    var io = require(‘socket.io’).listen(8080);

    io.sockets.on(‘connection’, function (socket) {
    socket.on(‘set nickname’, function (name) {
    socket.set(‘nickname’, name, function () {
    socket.emit(‘ready’);
    });
    });

    socket.on(‘message’, function (data) {
    console.log(data);
    });

    var iCounter = 0;

    setInterval(function() {
    socket.emit(‘message’, { counter: iCounter++ } );
    console.log(‘Counter: ‘ + iCounter++);
    }, 1000);

    socket.on(‘msg’, function () {
    socket.get(‘nickname’, function (err, name) {
    console.log(‘Chat message by ‘, name);
    });
    });
    });

    so when a client connect to the nodejs print each second the counter.
    The problem is this:

    - If i connect from the client to the address http://IP_ADDRESS:8080, the client application work well
    - if i connect from the client to the address http://IP_ADDRESS:80/socket.io/, the client application don’t print the counter. it seems that not receive the message from the server

    i seen a lot of configurations around the internet, but i never seen a configuration where the nginx work as normal web server and the directory /socket.io/ serve the nodejs application.
    Someone can tell me if socket.io work with nginx with a configuration not based on the root ?
    My configuration would be:

    Nginx — Socket.io –> Nodejs
    \– Apache

    many thanks to all !

    alle

    Reply
      1. alle

        I’m using the following versions:

        - Nodejs 0.10.21
        - Nginx 1.3.11
        - Socket.io 0.9.13

        the problems is a bit strange. For example with firefox it seems that not establish the connection, with chrome if you inspect with the Developer toolbar and you go to network, you see the connection and in the “Frames” section i can see the replies from the server.
        I’ve implemented the message:

        var socket = io.connect(‘ws:///socket.io/’);
        socket.on(‘message’, function (data) {
        console.log(data);
        });
        socket.on(‘message’, function (data) {
        console.log(data);
        });

        but in the console i don’t see anything.
        Otherwise if you connect with:

        var socket = io.connect(‘ws://:8080′);

        i see in the console the response from the server (when i connect, the server start a counter and every 1 second, as i posted in my last post, send to client the count).

        thanks

        alle

        Reply
  11. Pingback: Using Proxy Protocol With Nginx | chris lea

I'd really love to hear from you...