Rate Limits

ComplianceLayer enforces rate limits to ensure fair usage and protect API stability. Limits are based on your subscription plan and apply across multiple time windows.

Rate Limits vs Scan Quotas

ComplianceLayer has two separate usage controls:

Monthly Scan Quota: The number of security scans you can perform per month (e.g., 10 for Free plan). This is your primary usage limit.
Rate Limits: The maximum API requests per time window. These prevent abuse and excessive polling, but are set high enough to not interfere with normal usage.

In practice: Your monthly scan quota will be the limiting factor for most users. Rate limits are only hit when making unusually high numbers of API calls (e.g., polling results very frequently or running automated scripts).

Rate Limit Tiers

Plan	Per Minute	Per Hour	Per Day	Concurrent
Free	120	1,200	5,000	5
Starter	180	3,000	15,000	10
Professional	300	10,000	50,000	20
Enterprise	600	30,000	200,000	50
Custom	1,000+	100,000+	1,000,000+	100+

Rate Limit Headers

Every API response includes headers showing your current rate limit status:

HTTP/1.1 200 OK
X-RateLimit-Limit: 180
X-RateLimit-Remaining: 142
X-RateLimit-Reset: 1710000000
X-RateLimit-Plan: starter

# Limit: Maximum requests allowed in current window
# Remaining: Requests left before hitting limit
# Reset: Unix timestamp when limit resets
# Plan: Your current subscription tier

Handling Rate Limits

When you exceed your rate limit, you'll receive a 429 Too Many Requests response:

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 180
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1710000060
Retry-After: 45

{
  "error": "rate_limit_exceeded",
  "message": "Rate limit exceeded. Retry in 45 seconds.",
  "status": 429
}

Implementing Exponential Backoff

When you hit a rate limit, wait for the time specified in Retry-After before retrying:

async function fetchWithRetry(url, options, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch(url, options);

    if (response.ok) {
      return await response.json();
    }

    if (response.status === 429) {
      const retryAfter = parseInt(response.headers.get('Retry-After') || '60');
      console.log(`Rate limited. Waiting ${retryAfter}s...`);
      await new Promise(r => setTimeout(r, retryAfter * 1000));
      continue;
    }

    throw new Error(`Request failed: ${response.status}`);
  }

  throw new Error('Max retries exceeded');
}

// Usage
const result = await fetchWithRetry(
  'https://api.compliancelayer.net/v1/scan',
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({ domain: 'example.com' })
  }
);

Monitoring Your Usage

Track your rate limit usage in real-time by checking the response headers:

async function makeRequest(url, options) {
  const response = await fetch(url, options);

  // Extract rate limit info
  const limit = parseInt(response.headers.get('X-RateLimit-Limit'));
  const remaining = parseInt(response.headers.get('X-RateLimit-Remaining'));
  const reset = parseInt(response.headers.get('X-RateLimit-Reset'));

  // Calculate usage percentage
  const used = limit - remaining;
  const usagePercent = (used / limit * 100).toFixed(1);

  console.log(`Rate limit: ${used}/${limit} (${usagePercent}% used)`);
  console.log(`Remaining: ${remaining}`);
  console.log(`Resets at: ${new Date(reset * 1000).toISOString()}`);

  // Warn if approaching limit
  if (remaining < limit * 0.1) {
    console.warn('⚠️  Approaching rate limit! Consider slowing down requests.');
  }

  return await response.json();
}

Best Practices

1. Spread Out Requests

Don't burst all your requests at once. Spread them evenly across the time window:

class RateLimiter {
  constructor(requestsPerMinute) {
    this.requestsPerMinute = requestsPerMinute;
    this.queue = [];
    this.processing = false;
  }

  async request(fn) {
    return new Promise((resolve, reject) => {
      this.queue.push({ fn, resolve, reject });
      this.processQueue();
    });
  }

  async processQueue() {
    if (this.processing || this.queue.length === 0) return;

    this.processing = true;
    const delay = 60000 / this.requestsPerMinute; // ms between requests

    while (this.queue.length > 0) {
      const { fn, resolve, reject } = this.queue.shift();

      try {
        const result = await fn();
        resolve(result);
      } catch (error) {
        reject(error);
      }

      // Wait before next request
      if (this.queue.length > 0) {
        await new Promise(r => setTimeout(r, delay));
      }
    }

    this.processing = false;
  }
}

// Usage
const limiter = new RateLimiter(180); // 180 req/min for Starter plan

for (const domain of domains) {
  await limiter.request(() =>
    fetch('https://api.compliancelayer.net/v1/scan', {
      method: 'POST',
      headers: { 'Authorization': `Bearer ${API_KEY}` },
      body: JSON.stringify({ domain })
    })
  );
}

2. Cache Results

Cache scan results to avoid unnecessary API calls. A good strategy:

Cache results for 1-24 hours depending on domain criticality
Use ETags or Last-Modified headers to validate cache
Invalidate cache on-demand when needed

3. Use Webhooks Instead of Polling

Don't poll for scan results. Use webhooks to receive notifications when scans complete. This dramatically reduces API calls.

4. Batch Operations

When possible, batch operations together instead of making individual requests for each item.

5. Monitor and Alert

Set up monitoring to alert you when:

You're consistently hitting rate limits
Usage approaches your plan's quota
Unusual spikes in API calls occur

Upgrading Your Plan

If you consistently hit rate limits, consider upgrading your plan:

Free → Starter: 1.5x rate limits + 250 scans/month
Starter → Professional: 1.7x rate limits + 1,000 scans/month
Professional → Enterprise: 2x rate limits + 5,000 scans/month
Enterprise → Custom: Custom limits and pricing

Visit your billing settings to upgrade.

Enterprise Plans

Need higher limits? Enterprise plans offer:

Custom rate limits tailored to your needs
Dedicated IP addresses
SLA guarantees
Priority support

Contact [email protected] to discuss enterprise pricing.

API Documentation