Tailscale Exit Node Benchmark

A benchmarking tool for comparing Tailscale exit node performance, specifically optimized for AI service proxying (Claude, ChatGPT, Gemini).

Why This Tool?

When using Tailscale exit nodes to access AI services from regions with restricted access, raw speed tests don't tell the whole story. AI chat interfaces are highly sensitive to:

• TTFB (Time to First Byte) — determines how fast the first token appears
• Jitter — affects streaming smoothness; spikes cause stuttering
• Connection stability — packet loss causes retries and delays

This tool measures what actually matters for AI usage, not just marketing bandwidth numbers.

Features

• Test 1 to N exit nodes sequentially
• Direct baseline comparison — tests without exit node for reference
• Measure ping, TTFB, download speed, jitter, and packet loss
• Test against real AI API endpoints (Anthropic, OpenAI, Google Gemini)
• Test against general websites (US and HK sites, with/without anycast)
• Auto-skip unavailable nodes — detects if node doesn't offer exit capability
• JSON config file for easy sharing and customization
• Generates JSON results + Markdown comparison report
• Quick mode for fast comparisons (~2-3 min)
• Full mode with jitter analysis (~5-10 min per node)

Quick Start

# 1. Clone the repo
git clone https://github.com/wiiiimm/tailscale-exitnode-benchmark.git
cd tailscale-exitnode-benchmark

# 2. Install dependencies
./setup.sh

# 3. Edit config with your exit nodes
nano config.json

# 4. Run benchmark
./benchmark.sh

Installation

Prerequisites

• Linux (Ubuntu, Debian, Fedora, Arch, Alpine)
• Tailscale installed and authenticated
• sudo access (for switching exit nodes)

Setup

# Install all dependencies (tailscale, curl, jq, bc, ping)
./setup.sh

# Or install manually on Ubuntu/Debian:
sudo apt install curl jq bc iputils-ping
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

Configuration

Edit config.json to define your exit nodes and test parameters:

{
  "exit_nodes": [
    {
      "name": "US-West",
      "ip": "100.64.114.22",
      "dns": "us-west-node.tailnet.ts.net",
      "provider": "Provider A"
    },
    {
      "name": "US-East",
      "ip": "100.124.197.57",
      "dns": "us-east-node.tailnet.ts.net",
      "provider": "Provider B"
    }
  ],
  "ai_endpoints": [
    "https://api.anthropic.com",
    "https://api.openai.com",
    "https://generativelanguage.googleapis.com",
    "https://claude.ai",
    "https://chat.openai.com"
  ],
  "general_endpoints": [
    "https://www.google.com",
    "https://www.cloudflare.com",
    "https://github.com",
    "https://vercel.com",
    "https://news.ycombinator.com",
    "https://www.irs.gov",
    "https://www.gov.hk",
    "https://www.hku.hk"
  ],
  "test_params": {
    "ping_count": 20,
    "download_runs": 3,
    "download_size_mb": 10,
    "jitter_duration_seconds": 60
  }
}

Configuration Fields

Field	Description
exit_nodes[].name	Friendly name for the node (used in reports)
exit_nodes[].ip	Tailscale IP address (100.x.x.x)
exit_nodes[].dns	Tailscale MagicDNS name
exit_nodes[].provider	Hosting provider name (optional)
ai_endpoints	AI service URLs to test TTFB against
general_endpoints	General websites for comparison
test_params.ping_count	Number of pings per test
test_params.download_runs	Number of download speed tests
test_params.download_size_mb	Download test file size
test_params.jitter_duration_seconds	Duration of jitter test

Usage

Quick Test (~2-3 minutes)

Fast comparison with minimal tests:

./quick-test.sh

Full Benchmark (~5-10 minutes)

Comprehensive benchmark with jitter analysis:

./benchmark.sh

Command Line Options

./benchmark.sh [OPTIONS]

Options:
  -h, --help          Show help
  -c, --config FILE   Use custom config file
  -q, --quick         Quick mode (fewer pings, skip jitter)
  -n, --no-jitter     Skip jitter test
  --skip-direct       Skip direct connection baseline test
  --node NAME         Test only a specific node

Examples

# Full benchmark (includes direct baseline + all exit nodes)
./benchmark.sh

# Quick benchmark
./benchmark.sh -q

# Test only one node (still includes direct baseline)
./benchmark.sh --node "US-West"

# Skip direct baseline test
./benchmark.sh --skip-direct

# Use a different config file
./benchmark.sh -c ~/my-nodes.json

# Quick test, specific node, no jitter
./benchmark.sh -q -n --node "US-East"

Output

Results are saved to results/<timestamp>/:

results/20241218_153045/
├── Direct.json           # Baseline (no exit node)
├── US-West.json          # Raw data for US-West node
├── US-East.json          # Raw data for US-East node
└── report.md             # Markdown comparison table

Sample Report Output

## Summary

| Metric | *Direct* | US-West | US-East | Winner |
|--------|----------|---------|---------|--------|
| Ping (avg) | *27.0ms* | 156.2ms | 189.4ms | **US-West** |
| Packet Loss | *0%* | 0% | 0% | **—** |
| Claude API TTFB | *0.421s* | 0.342s | 0.418s | **US-West** |
| OpenAI API TTFB | *N/A* | 0.298s | 0.356s | **US-West** |
| Gemini API TTFB | *0.297s* | 0.312s | 0.298s | **US-East** |
| Avg Website TTFB | *0.589s* | 0.423s | 0.512s | **US-West** |
| Download Speed | *40.3Mbps* | 245.6Mbps | 312.4Mbps | **US-East** |
| Avg Jitter | *14.2ms* | 2.4ms | 8.7ms | **US-West** |

> *Direct* column shown for reference only, not included in winner calculation.

Sample JSON Output

{
  "meta": {
    "node_name": "US-West",
    "node_ip": "100.64.114.22",
    "public_ip": "203.0.113.45",
    "location": "Los Angeles, California, US",
    "timestamp": "2024-12-18T15:30:45Z"
  },
  "ping_external": {
    "min": 145.2,
    "avg": 156.2,
    "max": 178.9,
    "mdev": 8.4,
    "loss": 0
  },
  "ai_endpoints": {
    "api_anthropic_com": {
      "dns": 0.012,
      "connect": 0.156,
      "tls": 0.298,
      "ttfb": 0.342,
      "total": 0.345,
      "http_code": 200
    }
  },
  "download_mbps": {
    "runs": [241.2, 248.9, 246.7],
    "average": 245.6
  },
  "jitter": {
    "samples": 60,
    "avg_jitter_ms": 2.4,
    "spikes_over_50ms": 0
  }
}

Metrics Explained

Metric	What It Measures	Why It Matters
Ping (avg)	Round-trip latency to 8.8.8.8	Base network latency
Ping (mdev)	Latency standard deviation	Connection consistency
Packet Loss	% of dropped packets	Reliability
TTFB	Time to first byte from AI APIs	How fast first token appears
Avg Website TTFB	Average TTFB across general websites	Overall web performance
Download Speed	Throughput (Mbps)	Bulk data transfer
Jitter	Latency variance over time	Streaming smoothness
Spikes	Latency jumps >50ms	Causes visible stuttering

Note: TTFB includes DNS lookup, TCP connection, TLS handshake, and server processing time. The detailed breakdown (dns, connect, tls) is stored in JSON files for debugging but not shown in summaries since TTFB already encompasses them.

Interpreting Results

For AI Chat Usage

Prioritize (in order):

1. Lower TTFB — Most important for "chat feels fast"
2. Lower jitter — Smooth token streaming
3. Fewer spikes — No random freezes
4. Lower ping — General responsiveness

De-prioritize:

• Raw download speed (AI responses are small)
• Speedtest.net results (marketing numbers)

Decision Guide

If you see...	It means...
Lower TTFB + Higher jitter	Fast start, choppy streaming
Higher TTFB + Lower jitter	Slow start, smooth streaming
High spike count	Random freezes during chat
High packet loss	Retries, delays, timeouts

Rule of Thumb

Stable > Fast. A node with 200ms consistent latency beats one with 150ms average but frequent 500ms spikes.

Troubleshooting

"Tailscale is not running"

sudo systemctl start tailscaled
sudo tailscale up

"Permission denied" when setting exit node

The script requires sudo to switch exit nodes. It will prompt for your password automatically when needed. If you're running in an environment without interactive sudo:

sudo -v  # Cache credentials first
./benchmark.sh

Exit node not switching

Verify the node allows exit traffic:

tailscale status
# Check that the node shows "offers exit node"

"Node does not offer exit node capability"

The target machine needs to advertise itself as an exit node. On that machine, run:

sudo tailscale up --advertise-exit-node

Then approve the exit node in the Tailscale admin console if required.

All TTFB values are 0

The endpoint may be blocking or timing out. Check:

curl -v https://api.anthropic.com

Results vary wildly between runs

Network conditions fluctuate. Run multiple benchmarks at different times:

./benchmark.sh  # Morning
./benchmark.sh  # Evening
./benchmark.sh  # Peak hours

Finding Your Exit Node IPs

# List all Tailscale nodes
tailscale status

# Output:
# 100.64.114.22   my-node-1    linux   -
# 100.124.197.57  my-node-2    darwin  exit node

Use the IP addresses from this output in your config.json.

License

MIT