1. Home
  2. >
  3. Press Release
Categories
DDoS Deflect Deflect Labs Press Release

Deflecting attacks against Israeli and Palestinian websites

DoS/DDoS attack report against Deflect protected websites between Oct 7 to Oct 22, 2023

INTRODUCTION

Violence that engulfed Israel and Gaza in recent weeks has permeated the digital commons as well. From horrifying footage of murder on our computer screens to hateful discourse throughout social media platforms. The Deflect infrastructure has for many years been a secure home for Israeli and Palestinian human rights groups, media and civic institutions. Deflect staff continue to apply our project’s principles and terms of service to ensure that the network is not used as a platform for promoting violence or hate. We also seek our clients’ explicit permission before publicizing their association with Deflect and reporting on attacks that aims to silence them.

Since Oct 7, 2023, Deflect recorded six significant DoS/DDoS attacks against Israeli human rights organizations (btselem.org) that culminated with 54 million attack events hitting our edge servers. We also recorded 11 significant DoS/DDoS attacks against the Palestinian news website (palestinechronicle.com), with a total of 7 million malicious hits in various attack formation.

COVERAGE

  1. This report covers only L7 HTTP/HTTPS logs. There may be more attack traffic below L7, but is not covered in this report. Therefore we don’t provide traffic size information (such as 1GB of traffic per second).
  2. Attack with a higher “Ban rate” might underestimate the original scale of the attack. As after the Deflect ban, attacking IP will be banned on the firewall level and preventing any further request from that IP to hit our server.
  3. Sites with different tech parameters may result in different logging behavior. Site with JS challenger constantly enabled, challenging every request but do not firewall ban IP that failed too many challenges, may result in more attack traffic logged.

METHODOLOGY

To identify attacks from normal traffic, we employ the following methodology:

  1. Identify if a spike of total traffic / ban log existed over a 24 hour window.
  2. Narrow down to that time range for anomaly, which often includes:
    1. Excessive request hitting certain URL (such as root /)
    2. Excessive request with identical User-Agent from different IPs
    3. Evenly distributed User-Agent / HTTP Method that is too perfect to be true
    4. Excessive unique query string (such as ?v={rand}) to avoid cache
  3. Confirm if top traffic IPs triggered any of our rate limiting rules.
  4. Cross check with Baskerville system, a machine learning  system that detects anomalous traffic.

ATTACK: BTSELEM.ORG

Parameters: JS Challenger: On / Fail challenger or hitting rate limit result: No ban

#DateStart (+0)Duration (s)HTTP ReqRPSUnique IPUnique bansBan rate
B110/9/202320:37:5199752,497,38052,64424516567.35%
B210/13/202315:37:262665291,19210911100.00%
B310/16/202315:02:08123146,0661,1861,8331,41677.25%
B410/16/202322:32:554831,068,4362,2113,6112,40366.55%
B510/18/20230:03:12141165,1711,1683,1332,75587.93%
B610/20/202313:24:30181133,9307392,6062,28187.53%

Chart A: Deflect / Banjax ban log visualization of attack #B1

Attack #B1 stands as the most potent attack documented in this report. It achieved an average Request Per Second (RPS) of 52,644. The top 6 originating IPs dispatched an average of 3 million requests within a 10-minute duration. The assailants deployed a “Randomized Nocache Flood” strategy, using varying query strings to bypass caching. Notably, the same query string was observed being used by different IPs from various global locations.

Attack #B2 originated from a single IP: 46.210.30.130. However, an apparent misconfiguration in the attacker’s tool resulted in all their requests being rejected by our server. 

Attack #B3 featured user-agent strings with minor variations in their version numbers, keeping a consistent foundational structure. Still, these weren’t entirely unique; the same user-agent string was detected being used by 37 different IPs.

Attack #B4 adopted a strategy akin to Attack #B3, but showcased a broader spectrum of user-agents and specifically targeted the /hebrew endpoint, as opposed to the website’s root directory (/).

Chart B: Baskerville Reaction to Attack #B4

Attack #B5 mirrored the tactics seen in Attack #B3 but employed a different set of user-agents.

Attack #B6 shared three identical User-agent string among the 2606 IPs.

ATTACK: PALESTINECHRONICLE.COM

Parameters: Js Challenger: Off / Hitting rate limit result: Firewall ban

#DateStart (+0)Duration (s)HTTP ReqRPSUnique IPUnique bansBan rate
P110/8/20238:26:5351588,0141711,87991748.80%
P210/8/202314:42:2692586,9919411100.00%
P310/9/202310:16:30299364,2411,2181,6321,44588.54%
P410/9/202322:34:091541,198,7527,76411100.00%
P510/10/202313:11:02739230,6433122,0021,72185.96%
P610/10/202317:06:396682,869,1764,29470853275.14%
P710/12/202320:27:52272711,5112,6131,50686757.57%
P810/12/202320:57:58248738,3802,9771,14293882.14%
P910/13/20230:32:16181458,3542,53382874690.10%
P1010/13/20239:25:37177291,2911,64875971093.54%
P1110/21/202316:31:55117269,0272,3052,2281,34760.46%

Chart C: Deflect / Banjax ban log visualization of attack #P6

Attack #P2 and #P4 was perpetrated by a single IP. Both targeted the HTTP port 80 and did not adhere to the 301 redirection to HTTPS. Excessive 301 requests were only subject to bans after October 14.

Attack #P6 was primarily executed by a single IP, which likewise did not adhere to the 301 redirects issued by Deflect.

Attacks #P7, #P8, #P9, and #P10 exhibited similarities in their approach; all employed a uniformly distributed user-agent string, implying that identical user-agent strings were observed across various IPs.

ATTACK CORRELATION

We observed significant overlaps in attack IPs across various DDoS attacks on palestinechronicle.com and btselem.org websites, suggesting coordinated attempts by the perpetrators. Here are the findings:

  1. Attack #P9 and #P10 shared approximately 50 common attack IPs.
  2. Attack #P7 and #P8 had about 30 identical attack IPs.
  3. Notably, attack #P7, #P8, #P9, and #P10 seems to originate from the same attacking source, evidenced by a strong overlap of source IPs.
  4. Attack #P3 and #P6 had six IPs in common. While attack #P1 and #P5 also shared six identical IPs. The recurrence of shared IPs in separate attacks suggests a possible, albeit weak, connection of a common attack source or affiliated entities.
  5. Attack #B4, #B5 and #B6 had 32 shared attack IPs, hinting that they might be from the same attacking source.
  6. There were also IPs that attacked both sites:
    1. IPs 186.121.235.66, 187.141.184.235, 201.91.82.155, and 36.91.45.11 targeted both #B3 and #P6.
    2. IPs 186.121.235.66, 187.141.184.235, 201.91.82.155, 36.91.45.11, 123.126.158.50, 223.112.53.2, 5.95.66.74, 79.107.146.14, and 190.90.8.74 attacked both #B3 and #P3.
  7. Of the 13 IPs that targeted attack #B1, three also attacked atack #P6 and six targeted #P3.

TOP ATTACKING IPs

This is a list of IP with excessive request logged on Deflect, associated with individual indecent (See # for matching attack ID).

#IPASRequests Count
B1198.50.121.146iWeb Technologies Inc.3,936,297
B1202.134.19.50CMC Telecom Infrastructure Company3,077,579
B1209.126.124.140HEG US Inc.2,908,415
P6104.199.133.2Google LLC2,802,394
B1185.191.236.162Rack Sphere Hosting S.A.2,751,354
B1200.30.138.54MILLICOM CABLE EL SALVADOR S.A. DE C.V.2,502,015
B1103.74.121.88The Corporation for Financing & Promoting Technology2,480,702
P491.227.40.198Data Invest sp. z o.o. S.K.A1,198,752
B1113.125.82.11Cloud Computing Corporation848,330
B137.211.21.205Ooredoo Q.S.C.831,118
B1173.212.197.82Contabo GmbH662,370
B1212.92.204.54A1 Hrvatska d.o.o.589,828
B1193.41.88.58Kyiv National Taras Shevchenko University542,676
B1109.70.189.70JSC Elektrosvyaz497,125
B1186.121.235.66AXS Bolivia S. A.417,661
B193.180.220.67Intertelecom Ltd417,072
B1177.126.129.43Net Aki Internet Ltda399,074
B246.210.30.130Cellcom Fixed Line Communication L.P.291,192
P2223.233.84.97Bharti Airtel Ltd., Telemedia Services86,991
P723.247.35.2Global Frag Networks28,408
P9209.17.114.78Network Solutions, LLC25,476
P10209.17.114.78Network Solutions, LLC12,392

CONCLUSION

From October 7th to 22nd, 2023, both Israeli and Palestinian websites were subjected to coordinated and severe cyber-attacks, intended to overwhelm and take down these websites. These kinds of attacks, known as Distributed Denial of Service (DDoS) attacks, function like a traffic jam clogging up a highway, preventing regular users from accessing the website.

  1. Scale of Attacks: The Israeli human rights website faced attacks resulting in 54 million web requests, while the Palestinian news website experienced 7 million web requests. Think of these as millions of unwanted phone calls jamming up a hotline.
  2. Tactics and Techniques: The attackers adapted and used varied methods to bypass Deflect defences. Some tried to vary the attack requests in minute ways to fool manual rule-sets. Others used a more straightforward approach of sending a massive number of requests rapidly. In some instances, attackers tried to disguise their harmful requests by making them look like regular user visits.
  3. Shared Attack Patterns: We noticed that many of the attacks on both websites seemed to come from the same sources or groups. This is like recognizing the same group of troublemakers causing disruptions in multiple places. Specifically, the methods and even some of the internet addresses (IPs) used in the attacks were common across the two websites.
  4. Efficiency of Defenses: Our protective measures, think of them as security guards or filters, worked well in most cases. They were able to identify and block these harmful requests, preventing significant disruptions. However, attackers are persistent, and they keep trying various methods to bypass our defenses.

Over the recent period, our protective system, Deflect, has stood as a robust guardian for websites under its watch. Using sophisticated techniques, which include the power of machine learning, it has adeptly differentiated between regular and malicious traffic. This not only ensured that these cyber attackers were effectively thwarted but also maintained the uninterrupted service of the websites in question. It’s a testament to Deflect’s capability to handle intricate and aggressive cyber-attacks, safeguarding the essence and uninterrupted function of online platforms, and thereby supporting the freedom of expression online.

  1. Home
  2. >
  3. Press Release
Categories
Blog Deflect Press Release

Introducing Deflect-next

After several years of laborious effort, we are proud to announce the public release of the new Deflect software ecosystem. In our Github organization https://github.com/deflect-ca you will find all official Deflect related open source repositories that allow you to stand up an entire website protection infrastructure or its individual components. Deflect offers a high-performance content caching and delivery network, cyber attack mitigation tools powered by Banjax and the Baskerville machine learning clearinghouse, user dashboards, APIs and much else. You are reading this post on a rebuilt and refactored Deflect infrastructure and we’re very proud of that!

Below is the story of the how the new Deflect came to be and rationale for making various software choices.

Deflect-next

Created in 2011 Deflect was a relatively simple solution to the many-against-one problem of distributed denial of service (DDoS) attacks targeting civil society’s web servers. By running Deflect’s caching and mitigation software on constantly rotating edge servers strategically located in some of the world’s biggest data centers, we offered a many-against-many scenario, utilizing the widespread nature of the Internet in a similar manner to those organizing attacks against our clients – leveling the playing field and bringing a little more ‘equalitie’ for our clients’ rights to freedom of expression and association with our audience. Deflect edges memorized previous requests for website data (by virtue of reverse-proxy caching techniques) and removed the load from our clients’ web servers. To block the onslaught of bot-driven attacks, we developed rule based attack mitigation software – Banjax – identifying algorithmic behavior that we considered malicious and banning the IPs that exhibited them. 

As we scaled our infrastructure and hundreds of independent media, human rights groups and other non-profits joined the service, millions of daily requests were received on the Deflect network from around the world. This growth was matched by a higher frequency and sophistication of attacks. The Deflect infrastructure was continuously and laboriously patched, improved and upgraded multiple times. As often the case, the software code underpinning the service became cumbersome and replete with complicated configurations and fixes. Moreover, we moved further and further away from our secondary project ambition – to see other technology groups run their own instances of Deflect using our code base. The software stack required a lot of manual configuration and ‘insider knoweledge’. Beginning in 2019 we began to architect and develop a new version of Deflect, using an entirely new method of provisioning, managing and configuring network components, maintaining our agility and adding reproduce-ability as a primary design philosophy.

From ATS to Nginx

The primary tool for serving Deflect clients’ websites is the caching software installed on every network edge. It does all the heavy lifting in our design – fielding requests from the Internet on behalf of our clients’ websites in a reverse proxy fashion. Initially we had opted for the Apache Traffic Server (ATS) built by Yahoo and released open source in early 2000s. The choice was made primarily for its performance levels under stress tests. The software itself was not yet widespread and a little more difficult to set-up and maintain, with configuration for single actions often spread across multiple files. It required every Deflect network operator to dig deep into documentation and source code to figure out what will happen with every change.

Another caching and proxy solution – Nginx – was a more attractive choice for our new network design, with expressive configuration formats and a much larger technical userbase.

From Ansible and Bash to Python and Docker

Deflect clients customize their caching and attack mitigation settings via the Deflect Dashboard, which sends snapshots of these settings to the network controller. Previously, the configuration engine was a mix of Ansible and Bash, and it had overgrown in logic and depth more than what was destined to be managed in these languages. We have now rebuilt the configuration module in Python, supporting better scaling in the future and API communications.

We rebuilt the orchestration module of Deflect – for installing packages, starting and stopping processes, and sending new configuration to network edges, using Docker.

The benefits of containerization are well-known, but in short the advantages to us were: decoupling applications from the host OS (upgrading between Debian versions has historically been a pain point), making our various development and staging environments more reproducible, and letting us create and destroy multiple copies of a container on the same server easily. Once our applications were containerized, we needed a way to start and stop these containers, deciding to write it imperatively in our go-to general-purpose language, Python. There’s a library, docker-py, which connects to the Docker daemon on remote hosts over SSH and provides an API for building images, creating volumes, starting/stopping containers, and everything else we needed. The result is not as simple as Docker Compose or Swarm, but not as complicated as Kubernetes, and is written in a language everyone on our dev team already knows.

From Banjax to Banjax-go

Our primary method of mitigation – Banjax – was previously an ATS plugin, and as such was tightly coupled to the internal details of ATS’s request processing state machine and event loop. Written as C++ code it was tightly coupled with ATS internals and often limited in functionality by the caching server itself. Answering a simple question like “does a request count against a rate limit even if the result has previously been cached, or only if the request goes through to the origin?” required a close reading of Banjax and ATS source code. To port our attack mitigation logic to another server like Nginx would require a similarly detailed understanding of its internals. In addition, we wanted to share Banjax tooling with others – but could not decouple it from ATS – no one from our partners and peers were running this caching software.

We explored an alternate architecture: where attack mitigation logic lives in a separate process (developed in any language, and decoupled from the internals of any specific server) and talks to the server over some inter-process communication channel. We explored using an Nginx plugin which was specialized for this purpose (and developed a proof-of-concept ATS plugin in Lua which did the same thing) but found that a combination between the very widely used `proxy_pass` directive and `X-Accel-Redirect` header was more flexible (authentication responses can redirect the client to arbitrary locations) and probably more portable across servers. As for the choice of language for the authentication service, Python and the Flask framework would have been nice because we use it elsewhere in our stack, but some benchmarks showed Go and Gin being a lot faster (we were aiming for a worst-case overhead of about 1ms on top of requests with a 50th percentile response time of 50ms, and Go/Gin achieves this).

The interface between Nginx and Banjax-go is a good demonstration of Nginx’s expressive configuration file. This first code block says to match on every incoming request (`/`) and proxy the request to `http://banjax-go/auth-request`.

location / {
    proxy_pass http://banjax-go/auth_request
}

Banjax-go then checks the client IP and site name against its lists of IPs to block or challenge. It responds to Nginx with a header that looks like `X-Accel-Redirect: @access_granted` or `X-Accel-Redirect: @access_denied`. These are names of other location blocks in the Nginx configuration, and Nginx performs an internal redirect to one of them.

location @access_granted {
    proxy_pass https://origin-server
}
location @access_denied {
    return 403 "access denied";
}

This is already a lot easier to understand than a plugin which hooks into the internals of Nginx’s or ATS’s request processing logic (reading and writing a configuration file is easier than reading and writing code). Furthermore it composes nicely with the other concepts that can be expressed with Nginx’s scoped configuration: you can control the logging, caching, error-handling and more in each location block and its clear whether it applies to the request to banjax-go, the request to the origin server, or the static 403 access denied message.

Here’s a diagram that shows the above proxy_pass + X-Accel-Redirect flow (follow the red numbers 1, 2, and 3) along with the other interfaces Banjax-go has: the log tailing and the Kafka connection. The log tailing enforces the same regex + rate limit rules that the ATS plugin did, but asynchronously (outside of the client’s request and response) rather than synchronously. The Kafka channel is for receiving decisions from Baskerville (“challenge this IP”) and for reporting
 whether a challenged IP then passed or failed the challenge.

Baskerville client

The machine lead anomaly prediction clearinghouse – Baskerville – is an innovative infrastructure that has been working in production on Deflect for over a year. It is a complicated set-up reliant on edge servers reporting logs to the clearinghouse, where the pre-processing for feature extraction (looking for anomalous behavior in web logs) creates vectors which are then run through the learning model. An anomaly prediction is generated and communicated back to the network edge. The clearinghouse runs on a Kubernetes cluster and requires a large amount of resources for processing.

Recently, we have split the software base into two components – the clearinghouse and the client software (operating on any Linux+nginx web server). The idea was to allow third-party clients, not using Deflect, to benefit from the clearginhouse’s predictions and the Banjax mitigation tool. In this new model, the Baskerville client is installed independently of Deflect and performs:

  • Processes nginx web server logs and calculates statistical features.
  • Sends features to a clearing house instance of Baskerville.
  • Receives predictions from a clearing house for every IP.
  • Issues challenge commands for every malicious IP in a separate Kafka topic.
  • Monitors attacks in Grafana dashboards.
Anyone can benefit from Baskerville’s anomaly predictions and Banjax’s mitigation tools

Deflect-next open source components

  • Deflect – all necessary components to set up your network controller and edge servers – essentially acting as a reverse proxy to you or your clients’ origin web servers.
  • Deflect-API – an interface to Deflect components
  • Edgemanage – a tool for managing the HTTP availability of a cluster of web servers via DNS. If a machine is found to be under-performing, it is replaced by a new host to ensure maximum network availability.
  • Banjax – basic rate-limiting on incoming requests according to a configurable set of regex patterns.
  • Baskerville – an analytics engine that leverages machine learning to distinguish between normal and abnormal web traffic behavior. Used in concert with Banjax for challenging and banning IPs that breach an operator defined threshold.
  • Baskerville client – edge software for pre-processing behaivoural features from web logs and communicating with the Baskerville clearinghouse for anomaly predictions.
  • Baskerville dashboard – A dashboard for users running the Baskerville Client software offering setup, labeling behavior and communicating feedback to the clearinghouse

Happy coding everyone!

  1. Home
  2. >
  3. Press Release
Categories
Blog Deflect Press Release Uncategorized

Deflect partners with technology and media groups

June 01, 2021 – Deflect partners with technology and media groups

Since 2010, Deflect has specialized in protecting online platforms from cyber attacks. Today, our mission and time-tested tooling reaches further and wider than ever before! We are honoured to announce strategic partnerships with well-known Internet Service Providers and digital media entrepreneurs in the Americas and Europe. Our combined service offering includes all manner of web hosting and online collaboration platforms, technical consultancy and web security services. With over a hundred years of collective technology expertise and a dozen common languages between us, this is a partnership that will serve a global clientele and meet the challenges of shrinking online spaces for expression and self-determination.

Our mission is strengthened through this mutually beneficial partnership. We stand together, stronger and ever more resilient, to protect our clients’ platforms with ethical technology solutions, multilingual human resources and a common belief in principles before profits.

Dmitri Vitaliev, Founder deflect.ca

Find out more about our partners’ individual services and mission from the list below. Check out Deflect’s partnership opportunities and write to us!

@colnodo

Colnodo is a non for profit organization working since 1994 providing Internet infrastructure services to activists and civil society organizations.  Colnodo’s main objective is the access, use and appropriation of information and communication technologies (ICT) for social development, human development and the improvement of people’s living conditions through the strengthening of capacities and competencies, education for work, information and knowledge exchange, increased citizen participation, sustainable development and innovation.

@greenhost

Greenhost (Netherlands) is an established infrastructure provider focusing on digital human rights and sustainability. By providing (infrastructure) services to a wide range of organisations supporting human rights, free press and/or censorship circumvention while preserving privacy guarantees. Greenhost makes sure to keep the internet an open and innovative space.

@greennetisp

GreenNet (UK) have been networking people and activist groups for peace, the environment, equality and human rights since 1986 – providing internet services, web design and hosting. Our hardware and software choices are based on expert technical judgment, our ecological sustainability and ethical business values.

@cloud68hq

(Tirana, Tallinn, Worldwide) Cloud68.co provides reliable open source digital infrastructure to for-purpose small & medium teams, organizations and individuals with responsive and friendly support. As a team of long time contributors to digital privacy and open knowledge projects we are committed to help you migrate from big tech as easy as possible.

@sembramedia

SembraMedia is a nonprofit dedicated to empowering diverse voices in Spanish media to publish news and information with independence, journalistic integrity, and a positive impact on the communities they serve. They conduct research, provide training, consulting, and financial support to help media leaders develop more sustainable business models in Latin America, Spain, and the U.S. Hispanic market.

At MainMicro, our goal is to ensure customer satisfaction by providing ongoing support and cost effective solutions for our partners. We take great pride in having a customer retention rate that is among the highest in the industry. For us, when you become a customer you also become a friend, and we become the one-stop shop for all of your IT related needs.

At Black Crow Labs we construct your brand’s ecosystem and tell your story.  By engaging with prospective customers on targeted platforms we integrate your brand into their lives and conversations.

  1. Home
  2. >
  3. Press Release
Categories
Advocacy DDoS Press Release

Deflect website security services for free in response to COVID-19

In response and solidarity with numerous efforts that have sprung up to help with communications, coordination and outreach during the COVID-19 epidemic, eQualitie is offering Deflect website security and content delivery services for free until the end of 2020 to organizations and individuals working to help others during this difficult time. This includes:

  • Availability: as demand for your content grows, our worldwide infrastructure will ensure that your website remains accessible and fast
  • Security: protecting your website from malicious bots and hackers
  • Hosting: for existing or new WordPress sites
  • Aanalytics: view real-time statistics in the Deflect dashboard

Deflect is always offered free of charge to not-for-profit entities that meet our eligibility requirements. This offer extends our free services to any business or individual that is responding to societal needs during the pandemic, including media organizations, government, online retail and hospitality services, etc. We will review all applications to make sure they align with Deflect’s Terms of Use.

It takes 15 minutes to set up and we’ll have you protected on the same day. Our support team can help you in English, French, Chinese, Spanish and Russian. If you have any questions please contact us.