On this page
01 Why fingerprint at all
A user-agent string is a sentence the client writes about itself. A fingerprint is what the client does without thinking: the order it lists ciphers, the window it opens, the pause between clicks. Declared crawlers are handled by identity checks, covered in verifying AI crawlers by DNS and address range. Fingerprinting is for everything that does not declare itself: scrapers, credential stuffers, inventory bots and headless browsers that claim to be a desktop browser. The work sits in the monitoring hub because it starts as measurement, not blocking.
02 TLS ClientHello fingerprints
The ClientHello is sent in clear, even under TLS 1.3. It lists the TLS version, cipher suites, extensions, supported groups and signature algorithms, each in an order fixed by the client library. Hash those fields and you get a compact label for the library. The JA3 method hashes them in the order sent; the later JA4 approach sorts extensions and adds readable prefixes, because modern browsers now randomise extension order and would otherwise produce a new hash per connection. GREASE values (RFC 8701) must be dropped before hashing for the same reason.
The value is in mismatch. A request whose user-agent claims a current desktop browser but whose ClientHello matches a scripting language HTTP library is almost certainly automated. The limit is that anyone can load a browser TLS profile into a client library, and every browser version shifts the fingerprint slightly, so keep the reference table current.
03 HTTP/2 SETTINGS and header order
An HTTP/2 client opens with a SETTINGS frame: initial window size, header table size, maximum concurrent streams and a few others. Browsers use distinct values and send pseudo-headers (:method, :authority, :scheme, :path) in a fixed order that differs between engines. Regular headers follow a stable order too. A client library that sends a browser user-agent with its own default SETTINGS and alphabetical headers gives itself away. Reading any of this requires terminating TLS, so the collection point is the proxy or a Layer 7 balancer, a point made in what a Layer 7 device can read.
04 TCP/IP stack signals
The SYN is a passive operating system fingerprint. Stacks pick an initial TTL (commonly 64 or 128, sometimes 255), a starting window, an MSS and an option order. Observed TTL counts down by one per hop, so round up to the nearest common starting value. A browser user-agent claiming a desktop OS over a SYN that looks like a Linux server stack is a mismatch worth a point. The signal is weak alone: load balancers, NAT devices and proxies rewrite or originate the SYN you see.
src 198.51.100.23 ttl 52 -> initial 64, 12 hops win 64240 mss 1460 wscale 7 options: mss,sackOK,ts,nop,wscale claimed UA: desktop browser on a 128-TTL OS verdict: stack mismatch, +2
Collect SYNs from the balancer or from a mirrored copy of the uplink; sampling is fine because the stack does not change between connections from one client, and sampled sFlow headers include the TCP options of any SYN they happen to catch.
05 Timing, sessions and headless browsers
People are irregular. They load a page, its assets arrive in a burst within a second, then nothing happens for tens of seconds while they read. Scripts are regular or impossibly fast: evenly spaced requests, no asset loads, deep URLs in sequence, no cookies carried between requests. Plot inter-request intervals per client and the difference is usually visible without any statistics.
Headless browsers close most of these gaps. They run a real engine, so TLS, HTTP/2 and asset loading all look correct. What remains is behaviour: navigation that never scrolls, sessions that start at a deep URL with no referrer, identical timings across hundreds of addresses, and automation flags left set in the engine. Expect to catch the careless ones and to raise the cost for the rest.
06 False positives
- Corporate NAT and carrier-grade NAT. Thousands of people behind one address look like one very fast client. Never score on per-address rate alone.
- Forward proxies and security gateways. They terminate and re-originate TLS, so the fingerprint is the proxy, not the browser behind it.
- Accessibility tools. Screen readers and assistive browsers can skip assets, move through pages in a linear order and have unusual timing. Blocking them is a failure, not a success.
- Privacy browsers and extensions. Blocked scripts and trimmed headers produce odd but human sessions.
- Monitoring and uptime checks. Your own probes are bots. Allowlist them by address before scoring.
07 Combining signals into a score
Give each signal a weight, add them per client key and act on thresholds. A mismatch between claimed browser and TLS fingerprint might score 3; a SYN mismatch 2; no asset loads over ten pages 2; regular sub-second intervals 2. Illustrative bands: 0 to 3 pass, 4 to 6 challenge or shape, 7 and above block. Tune the weights against a labelled sample of your own traffic, review what lands in the middle band by hand, and feed the result to the balancer as a key for per-client rate limits rather than as an outright block.
Fingerprints are identifiers. Hash them, keep them for days rather than months, do not join them to personal accounts unless there is a stated reason, and document the processing in your privacy notice where the law requires it.
Agents that drive a real browser defeat most of these signals; the behaviour of AI agent sessions explains what still works.
08 Questions
What is a JA3 or JA4 fingerprint?
A hash or structured label built from fields in the TLS ClientHello: version, cipher suites, extensions, groups and signature algorithms. It identifies the TLS library a client uses. JA4-style labels sort extensions so browsers that randomise order still produce a stable value.
Can TLS fingerprinting work without decrypting traffic?
Yes. The ClientHello is sent before encryption begins, even in TLS 1.3, so a passive tap or the balancer can read it. HTTP/2 SETTINGS and header order, by contrast, require terminating TLS.
Why does corporate NAT cause false positives?
Many users share one public address, so per-address request rates look like a single aggressive client. Combine rate with fingerprint and session signals, and key limits on more than the address.
Can headless browsers be detected reliably?
Not reliably. They run a real browser engine, so protocol fingerprints match. Behaviour, automation flags and consistency across many addresses catch careless ones; well-built automation raises the cost of detection rather than evading it entirely.
Should a high bot score always mean a block?
No. Use bands: pass, shape or challenge, block. Shaping keeps false positives usable while limiting damage, and the middle band is where you learn whether the weights are right.