Post

From capital one to unreviewed AI code the same ssrf, different decade

From capital one to unreviewed AI code the same ssrf, different decade

It hurts way more reading about a fintech breach when you have worked at one.

I spent nearly seven years at Nubank. I lived through the security reviews, the compliance audits, the “can we deploy this without the security team looking at it” conversations. The answer was always no. And that was the right answer.

So when Capital One disclosed its 2019 breach — 106 million customer records, including Social Security numbers and bank account data — it wasn’t just “another tech company got hacked.” It was a clear signal that even well-resourced financial institutions can miss the one misconfiguration that unravels everything.

Now here is the uncomfortable question I keep asking:

Are we recreating the same vulnerability patterns at scale, except this time with AI-generated code that nobody properly reviewed?

breach aftermath

what actually happened at capital one

On July 19, 2019, Paige Thompson — a former AWS employee — exploited a misconfigured Web Application Firewall (WAF) to execute a Server-Side Request Forgery (SSRF) attack against Capital One’s AWS infrastructure.

The chain was brutally simple:

  1. Capital One had a WAF in place that was intended to block SSRF attacks
  2. The WAF had a bypass — a different endpoint that didn’t go through the same filtering
  3. Through that bypass, an SSRF reached the AWS Instance Metadata Service (IMDS) at http://169.254.169.254/
  4. The metadata endpoint returned IAM credentials
  5. Those credentials had access to S3 buckets containing customer data
  6. 106 million records exfiltrated

The DOJ press release is clinical about it. Krebs on Security has the full narrative. But the technical root cause fits in one sentence:

A single misconfigured access control allowed an SSRF to reach a metadata endpoint that should never have been accessible from application code.

the same vulnerability, rewritten by ai

Here is where it gets relevant today.

Take that same SSRF pattern. Ask any modern AI code assistant — GPT-4, Claude, Copilot, Cursor — to “write a function that fetches a URL provided by the user.” Here is what you will get from most models:

1
2
3
4
5
6
7
8
;; AI-generated: looks helpful, is actually a breach waiting to happen
(ns app.fetch
  (:require [clj-http.client :as http]
            [cheshire.core :as json]))

(defn fetch-user-data [url]
  (let [response (http/get url {:as :json :accept :json})]
    (json/parse-string (:body response) true)))

This function does exactly what was asked. It fetches a URL. It parses JSON. It looks correct.

It is also a direct SSRF vector. If that URL points to http://169.254.169.254/latest/meta-data/iam/security-credentials/, congratulations — you just rebuilt the Capital One attack surface in six lines of Clojure.

The fix is not complex, but it requires someone to know it is needed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
;; Human-reviewed: whitelist-based URL validation
(ns app.fetch
  (:require [clj-http.client :as http]
            [cheshire.core :as json]
            [clojure.string :as str]))

(def allowed-hosts
  #{"api.example.com" "data.internal.example.com"})

(defn safe-fetch [url]
  (let [uri (java.net.URI. url)]
    (if (and (.getHost uri) (contains? allowed-hosts (.getHost uri)))
      (-> (http/get url {:as :json :accept :json})
          :body
          (json/parse-string true))
      (throw (ex-info "host not allowed"
                      {:host (.getHost uri) :url url})))))

;; If you must access instance metadata, use IMDSv2
(defn fetch-with-imdsv2 [path]
  (let [token (-> (http/put "http://169.254.169.254/api/token"
                            {:headers {"X-aws-ec2-metadata-token-ttl-seconds" "21600"}})
                  :body)]
    (-> (http/get (str "http://169.254.169.254/" path)
                  {:headers {"X-aws-ec2-metadata-token" token}
                   :as :json})
        :body
        (json/parse-string true))))

Notice the difference: no AI wrote the whitelist. No AI decided to validate the host. No AI configured IMDSv2 tokens. Those decisions came from experience — specifically, from having read about or lived through exactly this kind of incident.

what the research says about ai code quality

This is not hypothetical fear-mongering. There are actual peer-reviewed studies.

Hammond Pearce and collaborators published “An Empirical Study of GitHub Copilot’s Security Weaknesses” in 2022. Their finding: approximately 40% of Copilot-generated code in security-relevant scenarios contained vulnerabilities. Not edge-case vulnerabilities either — OWASP Top 10 stuff. SQL injection. Path traversal. Hard-coded credentials. SSRF.

A 2023 IEEE S&P paper (“Asleep at the Keyboard?”) confirmed similar results with different models. The problems are not model-specific — they are systemic. AI code generators are optimized for plausible-looking code, not secure code. They learn from a training corpus that includes GitHub repositories full of insecure patterns, bugs, and anti-patterns.

The scariest part: AI models are confident when they generate bad code. A developer who is uncertain about a cryptographic implementation will at least check the docs. A developer who blindly copies an AI suggestion will not — because the model did not indicate uncertainty.

coinbase and the review capacity gap

In January 2023, Coinbase laid off approximately 950 employees — about 20% of its workforce. Among the cuts were significant portions of security, compliance, and QA teams.

Shortly after, reports emerged of security incidents attributed to insufficient review capacity post-layoffs. The exact details were internal, but the pattern is well-documented: when you reduce headcount, security reviews are the first thing that gets deprioritized because they are a “cost center” that slows down feature velocity.

The same dynamic applies to AI-generated code. A team shipping 3x more code because of AI tooling cannot realistically provide 3x more review bandwidth. Either reviews become shallow, or they become a bottleneck. Most teams pick the former.

So the equation becomes:

1
2
3
4
(AI-generated code with ~40% vulnerability rate)
× (less review bandwidth post-layoffs)
× (faster shipping cycles)
= more incidents like Capital One

This is not an opinion. It is arithmetic.

what fintech teams must do differently

If you are building fintech software, the question is not whether to use AI to write code. The question is how to build review and security processes that keep up with the output volume.

1. enforce url validation at the infrastructure level

Do not rely on application-level checks alone. Use network policies, security groups, and IMDSv2 to make SSRF impossible regardless of what the application code does.

In AWS, that means: don’t grant EC2 instance roles that allow sts:AssumeRole to applications that do not need them. VPC endpoints for S3. IMDSv2 required. WAF rules that actually validate the Host header.

2. treat ai-generated code as junior commits

Do not merge AI-written code without the same review process you would apply to a new grad’s PR. That means:

  • Pair programming on security-sensitive code paths
  • Mandatory OWASP Top 10 review checklist
  • Automated SAST scanning on every push
  • A dedicated human review for any code touching auth, IAM, or data access

3. maintain security-specific linters

For Clojure, something like:

1
2
3
4
;; Lein-nsorg or custom eastwood rules for dangerous patterns
;; Flag: (http/get url) where url is not validated
;; Flag: direct access to 169.254.169.254
;; Flag: str/interpolate in SQL queries

The principle applies to any language. Have automated rules that catch the patterns AI tends to produce unsafely.

4. keep security as a hardware requirement, not a software afterthought

The books that shaped my understanding of these trade-offs:

  • “The Web Application Hacker’s Handbook” (Stuttard & Pinto) — the definitive reference for understanding how application-layer attacks actually work, including SSRF
  • “Hacking: The Art of Exploitation” (Jon Erickson) — for the mental model of thinking like an attacker
  • “The DevOps Handbook” (Kim, Humble, Debois, Willis) — for integrating security into continuous delivery workflows
  • “Accelerate” (Forsgren, Humble, Kim) — the data-driven case that high-performing teams do not trade security for speed
  • “Sandworm” (Andy Greenberg) — for understanding the state-actor threat landscape that fintech infrastructure sits inside
  • OWASP Top 10 and OWASP AI Security & Privacy Guide — required reading for anyone reviewing AI-generated code
  • NIST AI Risk Management Framework — practical guidance for evaluating AI tooling risk at the organizational level

closing the loop

The Capital One breach was not caused by some novel zero-day exploit. It was caused by a known, well-documented vulnerability class (SSRF) that a misconfigured WAF failed to prevent.

That is the same class of vulnerability that AI code generators produce at an alarming rate today.

The difference? In 2019, at least a team of humans designed and deployed that WAF. The mistake was understandable — human error in configuration. Today, we are increasing the volume of code production while decreasing the depth of review, and the errors are not even human anymore. They are statistical likelihoods embedded in opaque neural network weights.

To paraphrase Cliff Stoll’s immortal line from “The Cuckoo’s Egg”:

The question is not whether someone is breaking in. The question is whether you will notice.

With AI-generated code flying through CI pipelines on reduced review teams, the answer is getting uncomfortably close to “probably not until the DOJ calls.”

This post is licensed under CC BY 4.0 by the author.