Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Netsil's stream-processor is programmed using Datalog, which is a subset of Prolog.

Our architecture/use-case: At Netsil, stateful packet processing pipelines are written in declarative rules and materialized tables are backed by SQL compatible embedded in-mem DB. Tuples are executed in parallel and parallelism is controlled by specifying context constraints in rules (e.g. packets within same TCP flow should be processed in order). Further, Datalog workflows are distributable by providing "location specifier" in rules -- i.e. Tuples and attributes serialize to protocol buffers and can be sent/received over ZMQ. Also, the materialized tables in Datalog can be made to sync up with Zookeeper, allowing distributed stream processors to do service discovery and so on. It's a pretty sophisticated runtime/compiler, written primarily in C/C++ for optimal performance. The underlying runtime uses a combination of Intel TBB and Boost ASIO.

We are in general big fans of declarative approaches as they have saved us a lot of time, allowing our small team to leapfrog the competition. You can learn more about our architecture here: https://netsil.com/blog/listen-to-your-apis-see-your-apps/

Disclaimer: I am co-founder of Netsil (www.netsil.com).



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: