The Deflect service is built around defense-in-depth principles to keep your website online, no matter the traffic coming in. Our network edges are located with multiple providers in data centers around the globe. Every edge on the Deflect network caches static webpage resources and can reply very quickly to a multitude of simultaneous requests. As traffic arrives at the edge, two separate modules are always on the lookout for malicious bots and attacks. One of these is Baskerville – powered by machine lead anomaly predictions. We have a dedicated page explaining how that works. The other is Banjax – a curated list of regex patterns with associated rate limits. This allows us, for example, to instantly block IPs sending requests with user agents from a list of vulnerability scanners. Or we can block IPs that request an expensive /search/ endpoint too often, or send an unreasonable amount of POST or GET requests to the network. It’s simple but very efficient.
Banjax was originally coded in C++ and created as an Apache Traffic Server (ATS) plugin. These initial choices have made it difficult for third parties (who were not running ATS) to adopt. In refactoring Banjax we decided to use Go – a more modern language that still provided all the necessary functionality and made it easier to maintain the library in the long term. So now, we are please to present Banjax-Go built to for the 2020s and working happily in concert with Baskerville and Deflect caching or as a standalone module in your nginx setup.
So the list of decisions Banjax can make are: Allow, Block, or Challenge. The decision lists are populated from the config file (useful for allowlisting or blocklisting known good or bad IPs), from the results of the regex rate limit rules (so breaking a rule can result in a Block or a Challenge, or even an Allow), and from messages received on a Kafka topic (this is how Baskerville talks to banjax-next).
In addition to blocking requests (at the HTTP level) or blocking IPs (at the iptables/ netfilter level), Banjax also supports sending a “challenge” HTML page which contains either a basic password challenge (useful as an extra line of defense in front of admin sections) or a proof-of-work challenge (useful for blocking bots that cannot execute JavaScript, while allowing web browsers through).
An intitial concern with moving away from C++ was performance – during an attack, Banjax often has to processes thousands of requests per second, on every edge. We ran a set of synthetic tests to see how Banjax-Go performed. We used a series of worst-case scenarios, coming from our past experiences on Deflect. Our goal was to process 1,000 unique IPs per second, on an average virtual machine (a Digital Ocean droplet).
We first tested iptables directly to see how quickly it can process direct requests – deleting 2000 rules – without any other system interfering. We ended up with the following results:
Next, we tested how quickly Banjax-Go is able to process different types of common requests (again, under worst-case scenario conditions):
- Every request generates a challenge: 800 req/sec
- Every request passes through to the origin without any caching: 1200 req/sec
- Every request passes through and is served a cached version of a web page: 2800 req/sec
At the same time we decided to evolve our caching mechanism from using Apache Traffic Server to Nginx. These and many other modules will make up our release of Deflect-Core – a project deliverable that we hope to present by the end of spring. For now our efforts concentrate on the mitigation toolkit banjax-next.