WebSockets are an exciting new technology designed to make it easier to create real time applications by providing a full-duplex communication channel between the browser and the server. In layman’s terms that means that information can be sent and received by the application at the same time. One of the easiest ways to write apps using WebSockets is to make use of the excellent Socket.io library, which is used in conjunction with Node. The usual strategy when building a Node app is to put Nginx in front of Node as a reverse proxy that serves any static content. This was a problem if you wanted to use WebSockets though, as Nginx didn’t know how to proxy those requests. Until now…
Last week, with the 1.3.13 release, Nginx has gained the ability to proxy these WebSocket requests (thanks to Apcera and CloudBees for sponsoring this work). It’s quite easy to do, and I’ll show you how. I’m assuming you’re running a reasonably new release of Ubuntu Linux. The first thing you’ll want to do is install the needed software. You can do this easily by using my PPA repositories for Node and Nginx respectively which have ready made packages.
We’ll also need an example app that uses Socket.io. Michael Mukhin has an easy, ready to go chat app tutorial that we’ll be making use of. Be sure to get the latest code from his github repository. You shouldn’t need to go through the installation steps for Node since my packages will take care of that for you. I’ll be assuming you have his chatroom app up and running correctly at:
http://localhost:8080
Now, if you wanted to serve this app in production, you obviously would want it to listen on port 80, and this is where Nginx comes in. We’ll make a basic configuration file called “chat” located at:
/etc/nginx/sites-enabled/chat
It should contain the following:
server {
listen 80;
root /home/chl/chat;
index index.html index.htm;
server_name _;
location / {
proxy_pass http://localhost:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
} |
You’ll want to change the root parameter to wherever you installed the chatroom app. Also, you should edit the index.html file in the chatroom app so that the Socket.io connection is made on port 80, so the 4th line should look like this:
<script src="/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js"></script>
<script>
var socket = io.connect('http://localhost');
// on connection to server, ask for the user's name with an anonymous callback
Once this is done, you should be able to start Nginx, and simply visit:
http://localhost
and things should work correctly. The important part of the Nginx config for all of this is the lines:
proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade";
The first line tells Nginx to use HTTP/1.1 when communicating to the Node backend, which is required for WebSockets. The next two tell Nginx to respond to the Upgrade request which is initiated over HTTP by the browser when it wants to use a WebSocket.
In production, you’d likely want to add additional location stanzas to Nginx to tell it where to serve static assets from, set expires headers, and so on. You’d also likely want to manage the Node process(es) with an init script or supervisor, so that the app would start automatically when the server booted up. But, in an nutshell, this is pretty much it for using Nginx with your WebSocket enabled application! Questions and comments are always welcome of course.
55 Comments
Thank you so much! I have waited for this for a long time.
Congrats on being the first documentation on how to use this week-old feature! Thanks so much!
is it proxyfies conditionally only on upgrade?, can it be done so?
I’m sorry but I’m not sure I understand your question. You can use proxy_pass without the Upgrade header, but that is designed specifically to proxy HTTP. It is not a generic TCP proxy, so it won’t work properly with WebSockets. If you want it to proxy WebSockets correctly you must use HTTP/1.1, set the Upgrade header, and set the Connection header. Is that what you’re asking?
Hi.. Thanks for the great post! I have a question for you.. When using upstreams in nginx and socket.io (multiple processes) it seems like it connects to all the online nodejs processes.
So i have an expressjs site running on port 3001->3005 and nginx available on :80. Firebug goes nuts with all the heartbeats from all the four servers. Do you know how I can instruct the client or nginx to keep the socket.io connection to just one of these nodejs processes?
Thanks
It’s possible your issue has something to do with the way you’re using Nginx as a load balancer. Specifically, you should probably be using the
ip_hashdirective in yourupstreamdeclaration. Check out the upstream docs for more information.Hi and sorry for the late reply.. The ip_hash seems to work as you described in my small scale test.. Hopefully it will in the full scale to hehe.. Thanks Chris!
socket.io use the redisstore. so you can share your session with multiple process
Hi Chris,
This may be an issue with socket.io but I am having some trouble. I have setup the latest node + nginx packages linked above. I have managed to get the proxy working to some extent, my socket.io client JS is being served through the proxy successfully. Also, I get the message `Connection Accepted` in my terminal.
However, my client doesn’t think that its connected. Everything works fine when switching back to my 8080 port.
Any idea why this could be? Is it to do with my config? Or could it be something else, like AWS security groups?
Hi Steve –
Hrm, I’m afraid that’s not enough to really go on for me to troubleshoot. I’d recommend using tcpdump to sniff the traffic and make sure it’s sending what you expect in terms of the upgrade request and response.
Oh nevermind, I used the `ppa:nginx/development`, it installed version 1.3.11. Fixed this and its working great. Thanks for the article.
Ah, yes, that would do it. I believe the WebSocket proxy functionality was actually introduced in version 1.3.13, which is in my PPA.
Yeah, I knew I had to use your PPA, but the `deb` command to add it didn’t work. When I googled it, I came across the nginx PPA and got a bit mixed up :). Thanks for the swift reply anyway though
This feature would be really useful in a project I’m working on.
I’m trying to setup WebSockets behind Nginx on my EC2 server. I’m running on an Elastic Beanstalk app right now. I’ve updated my AMI image to use Nginx 1.3.13, which should support WebSockets just fine. I’m using Socket.IO with Node.JS.
However, when trying to connect to my WebSocket, in Chrome, I get: WebSocket connection to ‘wss://api-hostname/socket.io/1/websocket/GEe1c0_kwVWcZHHhq0P3’ failed: Unexpected response code: 502.
In my config file, I have:
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
server {
listen 8080;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://nodejs;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
}
}
Would you have any idea what’s causing this?
I had a similar issue. After some debugging I found this in my logs:
[error] 2708#0: *66 upstream prematurely closed connection while reading response header from upstream, client: 46.37.133.246, server: api.seinnova.eu, request: “GET /socket.io/1/websocket/DcxDHRoz1GkUiZpf3pHy HTTP/1.1”, upstream: “http://127.0.0.1:3000/socket.io/1/websocket/DcxDHRoz1GkUiZpf3pHy”, host: “api.mysite.com”
It appears to be a problem with the buffer size used for reading from upstream
I solved this problem by adding following directives to my nginx vhost config:
In the server section I’ve added following:
large_client_header_buffers 8 32k;
In the location section I’ve added this:
proxy_buffers 8 32k;
proxy_buffer_size 64k;
Of course, numbers can be different in your case.
HTH
I’ve created a gist with sample config https://gist.github.com/aku/5466527
You could already do it by patching nginx 2 years ago with this module https://github.com/yaoweibin/nginx_tcp_proxy_module .. There’s a good explanation here: http://www.letseehere.com/reverse-proxy-web-sockets
The only problem was you couldn’t listen on the same port as your http server.
Sure, but 1) compiling in an external module in isn’t something that most people do, and 2) not being able to listen on port 80 screws up a lot of out-of-the-box functionality provided by many WebSocket libraries. Kind of a deal breaker.
Chris, how would I do this if I wanted to deploy a socket.io app with nginx under mydomain.com/foo/ (location /foo/) ? I guess I have to configure the socket.io resource option but I failed to get it to work correctly. My web page loads but the socket.io connection fails. Any advice?
Hey Chris, good post! I’m facing the very same issue as the one Jason Valdron posted about a month ago.
I’m running nginx 1.3.15 using your PPA, the very same configuration you showed, and a node.js+socket.io+express application. But I kept getting the error “failed: Unexpected response code: 502.”.
After a few seconds it works, but I assume socket.io is ussing xhr-polling at that point.
Any ideas?
I’m running under EC2 behind a load balancer. The load balancer was setup to forward HTTP requests from 80 to 8080, I had to change it to forward TCP requests from 80 to 8080.
I’m also running under a EC2 instance, but afaik I’m not behind a load balancer. I’m though listening on port 80 for several servers within nginx, proxying them to different ports. I don’t know if this could cause a problem with proxying websockets.
My understanding is that this shouldn’t cause a problem. As long as you’re using a new enough version of Nginx, you should be able to proxy things however you want as far as I know. As long as you’re following the above instructions it should work.
Yes, ELB load balancers are known to not work with WebSockets when doing HTTP. You’ll need to always have them just do straight TCP load balancing and let Nginx handle the WebSockets part.
Had the same issue. I had to get Lea’s nginx-devel ppa to make it work. Running 1.4 and it is working on an EC2 instance.
Hi chris, really helpful post. I was wondering if the nginx configuration work for an application on tomcat?We have an app running on port 8080 and one of our customers wouldnt want to open their firewall, would be great If you can advise on it.
I don’t have much experience with Tomcat but I certainly can’t see why that would be a problem. You’d just want to make sure that port was specified in the
proxy_passdirective as shown in the example above. Specifically, something like:location / { proxy_pass http://localhost:8080; ... }I’d be curious if it didn’t seem to be working, but I’d need hand holding to set up a test case and try and figure out what’s going on (I’m really not a Java guy, and know little about operating Tomcat).
Can nginx be used to proxy a websocket server running on windows.
I have tried to set up IIS 8 in Winserver 2012, for reverse proxying a websocket server app on windows. But cannot get it to work, and there’s little information on how to do it, or if it even works.
I want to try nginx on Windows. Seems like it can only be run on the console, as opposed to a service. Can it be setup for proxying webseockets?
If I install nginx on linux, and follow details in your article, can it be used to proxy to a different webserver. In your examples, it seemed like all configs had localhost, with redirection to a different port. Opening between servers within the local intranet, is not an issue for my deployment.
Thanks.
I have essentially zero experience with running IIS at this point, so I don’t know what would or would not work there with respect to Nginx. However, you should absolutely be able to set up a Linux server running Nginx and have it proxy to your IIS server. There’s nothing that says you can only proxy to
localhost, any IP address or FQDN will work correctly. So if your IIS server was, for example, at IP address10.7.15.5, then something likeshould work just fine.
Chris, thank you for replying. I am setting up a linux server with nginx, and will configure for windows websocket server.
I have set up nginx and the websocket server. In my client, when I connect to ws://nginxservername, I get an error Unexpected response code: 200.
There is a record of this connection in nginx access.log, but I don’t see any connections to my websocket server.
There is nothing in the nginx error.log
Is there any other way to debug this.
Thanks for any insight.
That’s not really enough info to be able to give much of a diagnosis. I’d recommend using
straceon thenginxprocess family to see what’s going on under the hood.Does the nginx proxy handle binary data over websockets?
I’ve actually never tried this so I don’t know. I’d ask on the Nginx mailing list if you don’t have an easy way of testing it yourself.
Can the websocket proxying be used together with keep-alive? I am a bit confused about the Connection header. Here it says the Connection header should be cleared:
http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
But for websockets it needs to be set to upgrade?
I want to run both normal http and websocket over the same port.
I would guess so. As I understand it, the WebSockets protocol doesn’t rely on any sort of HTTP keepalive functionality, so I don’t think they should interfere with each other.
Hi Chris, Thanks for the tutorial. I have just one question.
Does it handle data framing and unmasking so at the server we could receive data in text format.
Thanks
My understanding is that is is just a transparent proxy. Meaning that it doesn’t do anything involving framing or masking, it just sends the data directly through to the upstream back end service.
Hi, I need a suggestion from you,
In our institution, we have a server already configured and running with apache. (I do not have access to modify the apache config). We have a webpage(http://www.facweb.iitkgp.ernet.in/~rbahadur/prince/home.html?home) in this server which is up and running. Now my interest is to take request from user through this apache server and redirect the request (run a bio related application in other server on intranet) to my personal workstation in which I have a django site deployed with nginx. In my workstation I will do the processing with my algorithms and return the results to clients through the apache server. With my workstation i can not directly speak to outside world, so what are the best practices to deal with this issue (using ajax or ??). Please suggest me which direction should i go. I can configure my nginx only.
Hi there,
i have this configuration:
Nodejs listen to port 8080
Nginx configuration:
location / {
root html;
index index.html index.htm;
}
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
location /socket.io/ {
proxy_pass http://127.0.0.1:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_read_timeout 86400;
access_log off;
error_log /opt/nginx/logs/websockets.error.log;
}
the application on node is:
var io = require(‘socket.io’).listen(8080);
io.sockets.on(‘connection’, function (socket) {
socket.on(‘set nickname’, function (name) {
socket.set(‘nickname’, name, function () {
socket.emit(‘ready’);
});
});
socket.on(‘message’, function (data) {
console.log(data);
});
var iCounter = 0;
setInterval(function() {
socket.emit(‘message’, { counter: iCounter++ } );
console.log(‘Counter: ‘ + iCounter++);
}, 1000);
socket.on(‘msg’, function () {
socket.get(‘nickname’, function (err, name) {
console.log(‘Chat message by ‘, name);
});
});
});
so when a client connect to the nodejs print each second the counter.
The problem is this:
– If i connect from the client to the address http://IP_ADDRESS:8080, the client application work well
– if i connect from the client to the address http://IP_ADDRESS:80/socket.io/, the client application don’t print the counter. it seems that not receive the message from the server
i seen a lot of configurations around the internet, but i never seen a configuration where the nginx work as normal web server and the directory /socket.io/ serve the nodejs application.
Someone can tell me if socket.io work with nginx with a configuration not based on the root ?
My configuration would be:
Nginx — Socket.io –> Nodejs
— Apache
many thanks to all !
alle
At first blush that looks correct to me. What version of Nginx are you using?
I’m using the following versions:
– Nodejs 0.10.21
– Nginx 1.3.11
– Socket.io 0.9.13
the problems is a bit strange. For example with firefox it seems that not establish the connection, with chrome if you inspect with the Developer toolbar and you go to network, you see the connection and in the “Frames” section i can see the replies from the server.
I’ve implemented the message:
var socket = io.connect(‘ws:///socket.io/’);
socket.on(‘message’, function (data) {
console.log(data);
});
socket.on(‘message’, function (data) {
console.log(data);
});
but in the console i don’t see anything.
Otherwise if you connect with:
var socket = io.connect(‘ws://:8080’);
i see in the console the response from the server (when i connect, the server start a counter and every 1 second, as i posted in my last post, send to client the count).
thanks
alle
Pingback: Using Proxy Protocol With Nginx | chris lea
thanks, this helped a lot 🙂
Thanks for the tutorial!
These 3 lines:
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
just solve the problem I had before:
socket.io WebSocket connection to … failed: Error during WebSocket handshake: Unexpected response code: 400
I’m running some benchmarks for the following stack.
nginx/1.6.0
node v0.10.29
socket.io 1.0.6
nginx.conf is:
server {
listen [::]:80;
listen 80;
server_name servername;
large_client_header_buffers 8 32k;
location / {
proxy_pass http://myserver;
proxy_http_version 1.1;
proxy_buffers 8 32k;
proxy_buffer_size 64k;
proxy_connect_timeout 43200000;
proxy_read_timeout 43200000;
proxy_send_timeout 43200000;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Request-Start $msec;
}
I’ve changed the ulimit to 65535 on both client(bench mark tests) and server ubuntu 14.04 machines.
While using bench marking library https://www.npmjs.org/package/websocket-bench built for socket.io 1.0.6.
I’m running 100 concurrent connections with various total connection values. I’ve noticed that after about 2000-2500 connections my nginx logs are returning 502/503 errors and all subsequent client request fail.
In my nginx error logs I see the following as well as the 502/503 responses.
*666966 http run request: “/socket.io/?EIO=3&transport=polling&t=1404697867869-18643&b64=1&sid=efCH0LHJhl2JcwvAAAi6”
2014/07/06 21:51:08 [debug] 32333#0: *666966 http upstream check client, write event:1, “/socket.io/”
2014/07/06 21:51:08 [debug] 32333#0: *666966 http upstream recv(): -1 (11: Resource temporarily unavailable)
2014/07/06 21:51:08 [debug] 32330#0: *666907 http lingering close handler
2014/07/06 21:51:08 [debug] 32330#0: *666907 recv: fd:1009 0 of 4096
Is there anything wrong with my nginx.conf that is not allowing me to handle more connections? 2000 connections is very underwhelming. I was hoping to supporting 100,000. Is that realistic?
I’m running a digtalocean droplet.
2GB Ram 40GB SSD Disk New York 2 Docker 1.0 on Ubuntu 14.04 2 CORES.
I’ve followed the post and it works great for HTTP. But I can’t get it to work for HTTPS. Is there anything different that I need to add?
Hey! did you ever have problem with nginx after php update? i got “502 Bad Gateway” after php update and i dont have idea how to fix this
@Asysteo you need to give us more information.and we will try to help
Thank you so much It helps me to solve my websocket issue in rails application
Pingback: Nginx下配置webssh | 独木の白帆
Pingback: 6.24 WebSockets – 全栈Python
Thanks, it worked
https://stackoverflow.com/questions/51037402/connect-socket-io-android-client-through-a-reverse-proxy-server-to-socket-server
Not working for me.
Pingback: Websocket support in golang from dev till deployment – GoCoderSuresh