in-solana by Supanode

Solana RPC Failover and High Availability

TL;DR

  • - Never depend on a single RPC endpoint — always have at least one fallback
  • - Route reads to the fastest responding provider, writes to the most reliable
  • - Health-check endpoints continuously and remove degraded providers from rotation

A single RPC provider going down should never take your application offline. Building failover and load balancing into your RPC layer is essential for production Solana applications.

Failover architecture

Your Application
       │
  ┌────▼────┐
  │ RPC Proxy│ ← health checks, routing, retries
  └────┬────┘
       │
  ┌────┼─────────┬─────────┐
  ▼    ▼         ▼         ▼
Provider A   Provider B   Provider C
(primary)    (secondary)  (tertiary)

Implementation

Basic failover client

class FailoverRPC {
  private providers: RpcProvider[];
  private current = 0;

  constructor(urls: string[]) {
    this.providers = urls.map((url) => ({
      url,
      healthy: true,
      latencyMs: 0,
      errorCount: 0,
    }));
  }

  async call(method: string, params: any[]) {
    const healthy = this.providers.filter((p) => p.healthy);
    if (healthy.length === 0) {
      // All providers down — try all of them
      this.providers.forEach((p) => (p.healthy = true));
    }

    for (const provider of this.getOrderedProviders()) {
      try {
        const start = performance.now();
        const result = await this.sendRequest(provider.url, method, params);
        provider.latencyMs = performance.now() - start;
        provider.errorCount = 0;
        return result;
      } catch (err: any) {
        provider.errorCount++;
        if (provider.errorCount >= 3) {
          provider.healthy = false;
          this.scheduleHealthCheck(provider);
        }
        continue; // Try next provider
      }
    }
    throw new Error("All RPC providers failed");
  }

  private getOrderedProviders(): RpcProvider[] {
    return this.providers
      .filter((p) => p.healthy)
      .sort((a, b) => a.latencyMs - b.latencyMs);
  }

  private scheduleHealthCheck(provider: RpcProvider) {
    setTimeout(async () => {
      try {
        await this.sendRequest(provider.url, "getHealth", []);
        provider.healthy = true;
        provider.errorCount = 0;
      } catch {
        this.scheduleHealthCheck(provider); // Retry later
      }
    }, 30000);
  }
}

Read/write splitting

Route read and write operations to different providers:

class SplitRPC {
  private readProviders: string[];
  private writeProviders: string[];

  async read(method: string, params: any[]) {
    // Reads go to fastest provider (latency-sorted)
    return this.failoverCall(this.readProviders, method, params);
  }

  async write(method: string, params: any[]) {
    // Writes go to most reliable provider (staked connections)
    return this.failoverCall(this.writeProviders, method, params);
  }
}

Write operations (sendTransaction) benefit from staked connections and low-latency paths to the leader. Read operations benefit from fast response times regardless of stake.

Health checking

Active health checks

Periodically probe each provider:

async checkHealth(provider: RpcProvider) {
  try {
    const start = performance.now();
    const health = await fetch(`${provider.url}`, {
      method: "POST",
      body: JSON.stringify({
        jsonrpc: "2.0",
        id: 1,
        method: "getHealth",
      }),
    });
    provider.latencyMs = performance.now() - start;
    provider.healthy = health.ok;
  } catch {
    provider.healthy = false;
  }
}

Run health checks every 10–30 seconds. More frequent checks detect failures faster but add load.

Passive health monitoring

Track error rates from actual requests:

  • 5xx errors: server issues, mark unhealthy after 3 consecutive.
  • 429 errors: rate limited, back off but keep in rotation.
  • Timeout: increase latency estimate, mark unhealthy after 3 consecutive.
  • 200 with error result: RPC-level error, usually not a provider issue.

Load balancing strategies

StrategyBest forDrawback
Round-robinEven distributionIgnores provider health
Weighted round-robinPreferred provider + fallbackManual weight tuning
Least connectionsHigh-throughput appsComplexity
Latency-basedLow-latency requirementsRequires continuous monitoring
Active-passiveSimplicityWastes backup capacity

For most Solana applications, latency-based routing with failover provides the best balance of performance and reliability.

Multi-provider considerations

Consistency

Different providers may be at slightly different slots. For operations that depend on recent state:

// Ensure consistency for sendTransaction
const { blockhash } = await readRPC.getLatestBlockhash();
// Use the same blockhash for transaction building
// Send through writeRPC (which may be a different provider)
await writeRPC.sendTransaction(tx);

This works because blockhashes are valid for ~60–90 seconds, spanning any reasonable provider slot difference.

WebSocket failover

WebSocket connections need separate failover logic:

  1. Maintain connections to 2+ providers.
  2. Keep one active, others on standby.
  3. On disconnect from active, promote a standby.
  4. Resubscribe all subscriptions on the new active connection.
  5. Poll RPC to fill any gap during switchover.

FAQ

How many RPC providers should I use? At least two for production. Three provides comfortable redundancy. More than four adds complexity without proportional benefit, unless you have specific geographic requirements.

Should I use the same provider in different regions? Yes. Many providers offer regional endpoints. Using the same provider’s US and EU endpoints gives you geographic redundancy within a single account.

What is a reasonable health check interval? 10–30 seconds for active checks. Passive monitoring (tracking errors from real requests) provides faster detection of degradation.

Does failover add latency? The first request to a failed provider adds one timeout duration of latency before failover. With health checks, degraded providers are removed from rotation before they affect requests. The failover logic itself adds <1ms.

Updated: