Aside from an inability to run websockets on Django, I've been running "real time" websites for quite some time. AJAX calls are dirt simple to handle with your typical Django setup.
Scaling and blocking are handled pretty easily by running Django on FCGI using Flup and a Nginx frontend. No blocking problems since they're running in processes and threads, redis for caching and pub/sub, and a database for the backend. Works a charm.
Now then, this isn't a high volume site, getting only in the medium hundreds of requests per minute, but it's been working without problems on a small AWS instance. DB backups take more CPU than Django ever has.
Websockets, on the other hand, took me over to Go. Certainly not giving up Django for the rest of the site, however, until it really can't handle the load anymore.
Wouldn't a websocket middleware solve the issue? Client starts a socket and passes the id through HTTP to the Django app. When something happens in Django, the event is piped through the previously created socket and the problem is neatly solved. Could even have some sophisticated publish/subscribe mechanics in here.
There is a bit of middleware out there which enables websockets, but:
1) Doesn't currently work with Django 1.6+
2) Websockets and WSGI don't mix well. It's possible to capture the socket and use it further up the stack, but it requires some really nasty hacks.
3) Requires you to use a custom version of runserver, which allows the raw socket to be passed up into the handler code.
Not worth it to me. I'm sure the code could be made to work again, but then you loose a lot of the benefits of running it behind something like fcgi (since you need access to the socket for the persistent two way communication).
I wasn't thinking about Django middleware but a more generic message routing engine running as a separate process that speaks websockets on one end and has a bi-directional REST interface on the other exchanging messages with the Django backend.
edit: karneges points out Hookbox and Pushpin can fill this role.
Interesting idea, but you still end up with something performing polling against the backend, if you want the ability to send messages to the client without receiving a request first (the real strength of websockets).
Well, you always need the client to reach out first, since it is not otherwise addressable. ;) But this doesn't mean it has to poll. With a separate gateway that supports Websockets or Server Sent Events, it should be enough for the client to make an initial request to bootstrap the connection, and then the server can send as many messages as it wants downstream.
> it should be enough for the client to make an initial request to bootstrap the connection, and then the server can send as many messages as it wants downstream.
Yes, that's the point of websockets; but what was discussed here is a program acting as a bridge between django and websockets using http to speak to django. http does not support such asynchronous communication. You get one response for one request. How can a backend send a new response if there is no open request from the intermediary?
Sure, you can hack http by refusing to close the stream of a response and sending data intermittently (well, if you're using a django->apache/nginx protocol that supports streaming responses), but then you're no longer speaking http; you're speaking your own protocol over http.
Sure, you can have your intermediary poll django, which reduces some of the overhead since you're bypassing the external network stack, but you're still relying on only sending messages every $poll_interval.
Sure, you can create some secondary process through manage.py that runs and communicates with the intermediary directly, but then you're no longer speaking http to Django.
If you want async communication between your client and your server using websockets, you can't rely on speaking http to anybody; it just isn't compatible with truly asynchronous websocket communication.
The HTTP site of the middleware server should be able to make and receive HTTP requests. If something causes an event in Django, it can make a request to the HTTP server side of the middleware and send a message to all listening websocket clients.
Aaah, I think my confusion was from your misuse of the term middleware. When used in the context of Django, middleware is a layer in a stack of WSGI calls, not a standalone daemon which accepts and receives http posts and websockets.
I could see such a standalone daemon working, but it would seem like more straightforward to just write a daemon which handles websockets and your application logic on its own.
Django is missing websockets, and little else. All that ode about not repeating code at the client and the server isn't that relevant because the view (client) operates on a completely different environment from the model (server), and does a completely different kind of data manipulation. Very little code repeats, and the little that does is trivial.
Ok, a better way to represent the client code is always welcome, but Django has already a lot of power and flexibility here, and it couples well with Javascript capabilities.
Scaling and blocking are handled pretty easily by running Django on FCGI using Flup and a Nginx frontend. No blocking problems since they're running in processes and threads, redis for caching and pub/sub, and a database for the backend. Works a charm.
Now then, this isn't a high volume site, getting only in the medium hundreds of requests per minute, but it's been working without problems on a small AWS instance. DB backups take more CPU than Django ever has.
Websockets, on the other hand, took me over to Go. Certainly not giving up Django for the rest of the site, however, until it really can't handle the load anymore.