A Decade of Clojure at Studio71

What is Clojure and why did it fit for Studio71?

Clojure is a programming language (a dialect of Lisp) that excels at concurrency and data processing. Clojure runs on top of Java so it’s runs in all of the places Java runs and can use all of the Java libraries already out there (hello, Google and AWS libraries!), and that along with Clojure’s features (immutability, lazy sequences, easy concurrency), and the concise nature of the Lisp syntax made it a nice fit for Studio71’s API and data collection processes.

Studio71 is a digital media company that wears many hats and our technology needs to make sure all those hats fit! It has to connect to a huge amount of platforms and partners, parse “big data”, augment that data, and securely make it available to our team and partners. The Clojure API powers all of the Studio71 Arrow platform, AI applications such as Context, various partner applications, while processing a massive amount of information from around the internet every day. Clojure’s feature set is well aligned with those tasks.

Frankly, it’s also a fun language to work with. I wrote the original version (there’s some of my code still banging around in there today) and while I was confident it was the right call technically, I also wanted to work with something fun that had a great open source community around it.

What were the successes?

Well, for starters, it’s been 10 years and it’s still running stronger than ever!

The initial overarching technical challenge at Studio71 (back then it was called The Collective and Collective Digital Studio) was that it had big needs, big competitors, and we were starting with zero infrastructure. Building from a strong, performant, flexible API layer wasn’t the fastest strategy, and I had to work to sell it internally, but it’s one that has led to our success time and time again over the last decade. Clojure was a big part of that, as it helped a very small team (we’ve never had more than two full time backend developers) compete with teams 2 - 10x larger in terms of development speed, data processing, operational costs, and stability.

We’ve never been on the bleeding edge of Clojure, always waiting to see if a new library or feature was proven stable before we tried it, but technology such as Leiningen, Ring, and core.async have been huge in helping us process a number of requests and gigabytes of social data so large that I can’t even venture a guess.

What were the challenges?

The biggest challenge was always the most obvious one: finding developers. To be clear, this is a challenge for every technical endeavor, but this isn’t a cop out answer. If you think it’s hard to find talent in Golang, Javascript, or PHP, ask around to see if any code schools are offering classes in Lisp these days. Hell, even Computer Science bachelor degrees usually only briefly cover Lisp, unless you’re lucky enough that a professor goes out of their way to put a full class on the schedule! I’ve had to be very creative over the last 10 years to find talented developers who knew Clojure, or at least were up and capable of the challenge to get up to speed quickly.

Through a mix of hard work, and if I’m being honest, some amazing luck, I couldn’t be happier about the roster of Clojure developers that have come through the doors at Studio71 over the last decade. Erik Jacobsen, Matt Fellows, Ben Slawski, Daniel Overstreet, and Jeremiah Krayzel are all fantastic developers that joined our scrappy team to use (or learn on the fly) their Clojure skills for the Studio71 mission, and it is/was a pleasure to see the platform grow and mature to the point it is today, powering systems like Arrow and Context.

Other, less critical issues with Clojure are that because of the JVM it’s pretty slow to start up, so it’s not a good fit for various tooling requirements. We tried, but eventually moved to Golang for any backend tasks that aren’t the API itself. Also, Clojure is great, but sometimes it likes to remind you that it’s Java under the hood, and those are challenging days.

Would you still pick Clojure today?

The short answer is yes, but the decision would be harder than it was in 2013. Golang and Rust are viable options for a strong API platform that weren’t mature enough to be considered ten years ago. In particular, I’ve really learned to appreciate Golang over the years, and I do recommend that as my first choice for backend developer to friends and clients. It would be a tough decision, but for the Studio71 use case which is a lot of churning through massive data files, you still can’t beat Clojure’s performance, brevity, and core library.

Clojure is not only still going strong at Studio71 after a decade, the platform is better than it’s ever been and shows no signs of slowing down!