Everything Will Flow
- Zach Tellman, Factual
- queues: didn’t deal with directly in clojure until core.async
- queues are everywhere: even software threads have queues for their execution, and correspond to hardware threads that have their own buffers (queues)
- queueing theory: a lot of math, ignore most
- performance modeling and design of computer systems: queueing theory in action
- closed systems: when produce something, must wait for consumer to deal with it before we can produce something else
- ex: repl, web browser
- open systems: requests come in without regard for how fast the consumer is using them
- adding consumers makes the open systems we build more robust
- but: because we’re often adding producers and consumers, our systems may respond well for a good while, but then suddenly fall over (can keep up better for longer, but when gets unstable, does so rapidly)
- lesson: unbounded queues are fundamentally broken
- three responses to too much incoming data:
- drop: valid if new data overrides old data, or if don’t care
- reject: often the only choice for an application
- pause (backpressure): often the only choice for a closed system, or sub-system (can’t be sure that dropping or rejecting would be the right choice for the system as a whole)
- this is why core.async has the puts buffer in front of their normal channel buffer
- in fact, queues don’t need buffer, so much as they need the puts and takes buffers; which is the default channel you get from core.async
Clojure At Scale
- Anthony Moocar, Walmart Labs
- redis and cassandra plus clojure
- 20 services, 70 lein projects, 50K lines of code
- prefer component over global state
