To resolve this CloudFront conflict, you need to disassociate your custom domain from your CloudFront distribution.

Since AWS only allows one CloudFront distribution per hostname, and FusionAuth Cloud is implementing CloudFront on their infrastructure, you have two options:

1. Remove the custom domain from your CloudFront distribution — This allows FusionAuth to add the domain to their CloudFront distribution
2. Set up your CloudFront as a proxy — Configure your distribution to proxy requests to FusionAuth's infrastructure

The recommended approach is to remove your custom domain from your CloudFront distribution entirely, as FusionAuth will handle CloudFront configuration as part of their cloud service.

How FusionAuth Custom Domains Work

When you configure a custom domain with FusionAuth Cloud, you create a CNAME record pointing from your custom domain to your deployment's durable URL (something like <random-string>.durable.fusionauth.dev). FusionAuth manages the CloudFront distribution and SSL certificates on their infrastructure, so you don't need to maintain your own CloudFront distribution for the custom domain.

The proper DNS setup should be:

• Custom domain → CNAME → FusionAuth durable URL

This allows FusionAuth to provision and manage the entire CloudFront infrastructure, including SSL certificate validation and renewal, as part of their cloud service.

Migration Timeline

This change is important but not urgent. A timeline of 1-2 weeks for investigation and implementation is reasonable. FusionAuth support can assist you through the migration process if you have any questions during implementation.

---

Related Documentation

• FusionAuth Cloud Custom Domains - Official guide for setting up custom domains on FusionAuth Cloud
• Adding a Custom Domain - Step-by-step walkthrough of the custom domain configuration process
• How to set up a Custom Domain for FusionAuth using AWS CloudFront - Guide for self-hosted FusionAuth installations (note: this is for self-hosted, not FusionAuth Cloud)

Note: If you're on a High Availability plan and want to use the Unlimited Custom Domains feature, or if you need assistance with the migration, open a support ticket.

We are using FusionAuth Cloud with a custom domain, and we've set up our own CloudFront distribution for that domain. However, we received notification that FusionAuth is implementing CloudFront as part of their cloud infrastructure.

The issue is that AWS only allows one CloudFront distribution per hostname. Our custom domain is registered in our CloudFront distribution, but the DNS record points to the FusionAuth instance.

What changes do we need to make to resolve this conflict and allow FusionAuth to enable CloudFront on their side for our instance?

The SocketTimeoutException: Read timed out error indicates that FusionAuth successfully initiates a connection to the external identity provider's discovery endpoint, but the provider doesn't respond within the configured timeout period. This is an intermittent connectivity issue between FusionAuth and the external provider.

Investigation Steps

1. Check FusionAuth Event Logs: Navigate to System → Event Log to find specific instances of the timeout errors with timestamps. The Event Log contains messages from asynchronous code execution, including connection errors to external services.
2. Verify the external endpoint: Test the discovery endpoint manually (e.g., via curl) to confirm it's responding correctly
3. Look for patterns: Note the times when errors occur to identify if there's a pattern
4. Enable debug logging: Turn on debugging in FusionAuth to get more detailed information about the OIDC connection attempts. This is a recommended first step when troubleshooting any OIDC connection issues.

Root Cause

Based on investigation, when the external provider's endpoint:

• DNS resolves correctly
• TLS handshake completes successfully
• Returns HTTP 200 with valid JSON during manual testing
• But still fails intermittently from FusionAuth

This indicates the external provider may be rate-limiting, blocking, or experiencing intermittent service issues that affect automated requests from FusionAuth.

Workaround

Instead of using the OpenID Connect Discovery URL, manually configure the endpoints in your FusionAuth identity provider settings. This bypasses the discovery mechanism and eliminates the timeout errors during the configuration resolution phase.

To configure manual endpoints:

1. Go to your OIDC Identity Provider configuration (Settings → Identity Providers)
2. Toggle Discover endpoints to Off (disabled)
3. Explicitly set the three required endpoints:
   • Authorization endpoint: https://accounts.example.com/tenant/oauth/authorize
   • Token endpoint: https://accounts.example.com/tenant/oauth/token
   • Userinfo endpoint: https://accounts.example.com/tenant/oauth/userinfo

This manual configuration approach is commonly used with providers like GitHub and Discord that don't implement standard discovery endpoints, and can also be used to work around discovery endpoint reliability issues.

Note: If your external provider uses RS256 to sign tokens (rather than HS256), be aware that FusionAuth currently doesn't allow manual configuration of the JWKS URL when discovery is disabled. This may cause id_token signature verification issues. If you encounter this, you may need to continue using discovery or contact FusionAuth support for alternatives.

Next Steps

Contact your external identity provider to:

• Report the intermittent timeout issues
• Share the timeout error logs and timestamps
• Ask if they're experiencing service issues or if FusionAuth's IP range needs to be whitelisted
• Inquire about any rate limiting policies that might affect discovery endpoint calls

Related Documentation

• Add an OpenID Connect Identity Provider - Complete guide to configuring OIDC IdPs
• OIDC Troubleshooting - First steps for troubleshooting OIDC connections
• OpenID Connect API - API reference for managing OIDC identity providers
• Event Log API - How to access and query event logs programmatically

We are using OpenID Connect to integrate with an external identity provider for SSO. Since a specific date, we've seen an escalating number of timeout errors when FusionAuth attempts to discover the OpenID Connect configuration.

The configuration and FusionAuth version have not changed. Here is an example error from the Event Log:

Unable to resolve OpenID Connect configuration using issuer [https://accounts.example.com/tenant] for [tenant/provider/ProviderName].
Request to the [https://accounts.example.com/tenant/.well-known/openid-configuration] endpoint failed.
Status code [-1]

Exception encountered.

java.net.SocketTimeoutException : Message: Read timed out

The errors appear intermittently and occur at various times. When testing manually, the endpoint sometimes responds successfully and quickly (within milliseconds), but the timeouts persist in production.

Is this a known issue with external OpenID Connect identity providers? Could there be network-level issues between FusionAuth and the external provider causing intermittent connectivity problems?

Rollback Process for FusionAuth Cloud

For FusionAuth Cloud deployments, the rollback process must be handled by FusionAuth support—it is not something you can perform yourself.

Plan Eligibility and Backup Retention

• Business and High Availability (HA) Deployments: Upgrade backups are retained and available for 30 days
• Basic Deployments: Backups are not included, so rollbacks cannot be performed

How to Request a Rollback

If you identify an issue after performing the upgrade:

1. Open a support ticket with FusionAuth
2. The support team will perform the complete database rollback for you

Expected Downtime

The rollback operation typically takes 20 minutes to 2 hours, depending on:

• Database size
• How quickly the operations team can complete the rollback

Note that this involves a complete database rollback, so plan for 1-2 hours of downtime.

Support During Upgrades

For production upgrades, FusionAuth can provide:

• An engineer on standby during your scheduled upgrade window
• An on-call support number for emergency assistance

Pre-Upgrade Best Practices

Before upgrading to 1.69.1:

1. Review Release Notes for breaking changes, database migrations, and API modifications
2. Choose an Upgrade Strategy:
   • Incremental (version-by-version) for lower risk
   • Direct upgrade for speed (but requires thorough testing)
3. Test in Staging to verify:
   • Authentication flows, webhooks, and APIs function correctly
   • Templates render properly
   • Database migrations complete successfully
4. Coordinate with Support to schedule the upgrade and have assistance available if needed

We are planning to upgrade our FusionAuth Cloud instance from version 1.49.0 to 1.69.1 and want to understand the rollback process in case we encounter issues during the upgrade.

Specifically:

• How long are rollback backups retained after an upgrade?
• Can we perform a rollback ourselves on FusionAuth Cloud, or does it need to be done by FusionAuth support?
• What is the typical downtime for a rollback operation?
• Are there any plan-specific limitations for rollback availability?

Currently, FusionAuth does not support recovery code regeneration within the hosted account pages, and there are no plans on the public roadmap for this feature.

Regarding your specific questions:

1. Hosted account page support: There is no current support for recovery code regeneration as a themeable template in the hosted account pages. Recovery codes are only generated and displayed when a user first enables an MFA method. As of version 1.68.0, recovery codes are hashed at rest using salted-pbkdf2-hmac-sha256 and cannot be retrieved after initial generation—they can only be regenerated through the API. This would need to be submitted as a feature request.

2. JWT authentication for the generate endpoint: The generate recovery codes endpoint (POST /api/user/two-factor/recovery-code/{userId}) currently only supports API key authentication. There is no JWT authentication variant available, and nothing in the current documentation or public roadmap indicates one is planned.

Recommended approach: Submit feature requests for both capabilities through the FusionAuth GitHub Issues repository. These features could be valuable for self-service MFA management scenarios. Note that feature requests, if accepted, do not have guaranteed timelines for implementation.

Current workarounds: You would need to implement this functionality in your own application with server-side code that uses an API key to call the generate recovery codes endpoint, though this moves the functionality outside the hosted pages where other MFA management occurs. When new codes are generated, all existing recovery codes are invalidated and replaced with a new set of 10 codes.

Related Documentation

• Multi-Factor Authentication (MFA) - Recovery Codes - Overview of how recovery codes work in FusionAuth
• Generate Recovery Codes API - API endpoint documentation for programmatic recovery code generation
• Self-Service Account Management - Documentation on the hosted account pages and available features
• Customizing Self-Service Account Management - Guide for customizing hosted account page templates
• API Authentication - Documentation on API key and JWT authentication methods
• FusionAuth 1.68 Release Notes - Hashed Recovery Codes - Information about recovery code hashing security enhancement

We want to allow users to self-service the regeneration of their MFA recovery codes. The API endpoint for generating recovery codes (POST /api/user/two-factor/recovery-code/{userId}) exists, but there appears to be no way to invoke it from the hosted theme pages.

Specific issues:

• The hosted account pages have no route for recovery code regeneration
• There is no corresponding template in the theme set to customize
• The generate endpoint requires API key authentication only (no JWT option), so it cannot be called directly from the browser

Placing this functionality on our own website doesn't seem ideal since all other MFA management lives on the FusionAuth hosted pages. We considered using a custom block on the hosted two-factor page, but this requires workarounds that could introduce security concerns.

Are there any current or planned solutions for:

1. Recovery code regeneration within hosted account pages (ideally as a themeable template alongside existing accountTwoFactor templates)?
2. A JWT-authenticated variant of the generate recovery codes endpoint to simplify browser-based implementation?

This issue is not actually related to FusionAuth itself, but rather to GCP Cloud NAT port allocation limits.

By default, Cloud NAT only allocates 64 TCP ports per VM for outbound connections. When your application makes many concurrent connections to FusionAuth (or any external service), you can quickly exhaust these ports, leading to connection timeouts.

Solution

Enable dynamic port allocation and increase the port range with this gcloud command:

gcloud compute routers nats update cloud-nat \
  --router=<ROUTER> \
  --region=<REGION> \
  --project=<PROJECT> \
  --enable-dynamic-port-allocation \
  --min-ports-per-vm=1024 \
  --max-ports-per-vm=32768

Replace <ROUTER>, <REGION>, and <PROJECT> with your actual GCP resource names.

This increases the available ports from 64 to between 1024-32768 per VM, which should resolve the connection timeout issues.

Additional Considerations

While this is primarily a GCP infrastructure issue, if you continue to experience connection problems after adjusting your NAT configuration, consider reviewing:

• Network latency between FusionAuth and your database - High latency or unstable network connectivity can cause database connection pool exhaustion, which may manifest as API timeouts
• Connection pooling settings - Ensure your application is properly reusing HTTP connections to FusionAuth rather than creating new connections for each request
• Load patterns - Monitor whether timeouts occur during specific load patterns that might indicate resource constraints

Related Documentation

• FusionAuth Networking Configuration - Configure how FusionAuth determines client IP addresses and network settings
• Troubleshooting Connection Issues - General guidance on troubleshooting API calls and connectivity
• Deploying FusionAuth on Google Kubernetes Engine - Best practices for running FusionAuth on GCP
• Google Cloud Platform with FusionAuth - Overview of deploying FusionAuth in GCP environments

External Resources

• GCP Cloud NAT port allocation documentation
• Consider monitoring your NAT port usage to right-size these settings for your workload

When running FusionAuth in containers on Google Cloud Platform with Cloud NAT configured for static IP assignment, we're experiencing intermittent connection timeouts and failures when making outbound connections from our application to FusionAuth.

The issue appears to be related to network connectivity, but FusionAuth itself seems to be running correctly. The timeouts occur sporadically under load.

Our setup:

• FusionAuth running in containers on GCP
• Cloud NAT configured to assign static IP addresses to containers
• Intermittent connection failures to FusionAuth APIs

Has anyone encountered similar networking issues when using FusionAuth with GCP Cloud NAT?