Skip to content

OpenEverest: Troubleshooting and diagnostics

This page is your go-to resource for tackling common issues and finding solutions.

For limitations, see the known limitations section.

General troubleshooting guidelines

Before troubleshooting, it’s important to understand how OpenEverest works at a high level. Understanding how it works can help narrow down potential areas to investigate. You can refer to the Kubernetes concepts guide for background. This context will help you interpret Everest’s behavior when debugging.

  • Refer to the kubectl quick reference for commonly used commands to inspect and troubleshoot Kubernetes resources when working with OpenEverest.

  • Use logs and events for debugging.

Key logs and commands

You can review different logs for additional information depending on the specific issue.

  1. Everest Core Components

    Logs Command
    OpenEverest controller kubectl logs -f deploy/everest-controller -n everest-system
    OpenEverest server kubectl logs -f deploy/everest-server -n everest-system
  2. Providers (database operators)

    Providers are installed as Helm releases in the everest-system namespace, and each provider bundles a database operator. List the installed providers and their operator deployments, then follow the operator logs:

    helm list -n everest-system
    kubectl get deploy -n everest-system
    kubectl logs -f deploy/<database-operator-deployment> -n everest-system
    
  3. Monitoring

    Component Command
    VictoriaMetrics Operator kubectl logs -f deploy/vm-operator -n everest-monitoring
    VM Agent kubectl logs -f deploy/vmagent-everest-monitoring -n everest-monitoring
  4. Database and Proxy Pods

    kubectl logs -f <pod-name of database or proxy> -c <database-container>
    
  5. Kubernetes Events

    Important

    Events (Events are stored only for 60 mins; if there are any events older than 60 minutes, they will be lost).

    kubectl get events --sort-by=".lastTimestamp"
    

Troubleshooting key areas

Installation issues

For troubleshooting OpenEverest installation issues with the Helm chart, the following steps may be helpful:

  1. Permissions and privileges

    Installing OpenEverest and its providers may require appropriate privileges.

    • OpenEverest installation may require cluster-admin privileges to create:
      • CustomResourceDefinitions (CRDs)
      • Cluster-scoped resources such as Provider objects

    If you encounter failures during installation, ensure your user account has the appropriate permissions.

    Note

    Application teams frequently encounter problems when installing operators due to insufficient privileges.

    Run the following command to check if the required privileges are granted:

    kubectl auth can-i <verb> <resource> --namespace <namespace>
    

    To verify if you can create CRDs, run the following command:

    kubectl auth can-i create crd
    

    Ensure that your cluster administrator grants the required privileges before you retry the installation.

  2. Helm Chart validation

    • Verify the installation status of the Helm chart. A properly functioning chart should be in a Deployed status.

      To verify the values used during the chart installation, run the following command:

      helm list -n everest-system
      NAME            NAMESPACE         REVISION    UPDATED                                 STATUS      CHART                    APP VERSION
      everest-core    everest-system    1           2026-09-01 16:24:56.577713 +0000 UTC    deployed    openeverest-2.0.0-dev.3   2.0.0-dev.3
      
    • The OpenEverest installation has many components, so it will fail if any subcomponent installations fail. Check the relevant namespace where components are installed, along with the logs and events.

  3. Resource availability

    If the cluster has no available resources to run pods, the Helm installation will wait for the specified --timeout (5 minutes by default) before failing.

    In such cases, check if any pod is stuck in the Pending state due to insufficient resources:

    kubectl get pods -A --field-selector=status.phase=Pending
    

    Note

    This behavior is not confined to jobs alone. If the cluster lacks sufficient resources, any component may remain in a Pending state.

  4. Local kind setup initialization timing

    When using local Kubernetes environments such as kind, Helm installation may initially time out while cluster components are still stabilizing. This behavior may be more noticeable in local environments such as WSL2.

    Example:

    Error: INSTALLATION FAILED: failed post-install:
    timed out waiting for the condition
    

    During startup, everest-server may temporarily restart if Kubernetes API connectivity is not yet available. This does not always indicate an installation failure.

    Before retrying the installation, verify whether OpenEverest components are still progressing:

    kubectl get pods -n everest-system
    

    Review logs if required:

    kubectl logs -f deploy/everest-server -n everest-system
    

    Note

    Temporary pod restarts during local kind setup may occur while Kubernetes components initialize. Allow the cluster to stabilize before reinstalling OpenEverest.

For detailed information on the installation process, see Installation overview

UI, API and authorization issues

To troubleshoot issues with the OpenEverest UI, API, or authorization, check the everest-server deployment.

  1. Check everest-server Pod health

    If the OpenEverest API is not working, check the status of the everest-server pod, specifically its Status and Restarts.

    kubectl get po -l app.kubernetes.io/name=everest-server -n everest-system
    
    NAME                            READY STATUS  RESTARTS AGE
    everest-server-78699679d4-kgqk5 1/1   Running  0       4d23h
    
  2. Check everest-server logs

    kubectl logs -f deploy/everest-server -n everest-system
    
  3. RBAC validation

    To resolve authorization and access issues, check the OpenEverest server logs. If Role-Based Access Control (RBAC) is enabled, review the RBAC policy stored in the everest-rbac ConfigMap:

    kubectl get configmap everest-rbac -n everest-system -o yaml
    
  4. Local access via Port Forwarding

    If you experience any access issues or lag in the OpenEverest UI or API, try port-forwarding to the service and check the latency compared to accessing it via a LoadBalancer or NodePort.

    kubectl port-forward svc/everest 8080:8080
    
    Once you have set up the port-forward, access the webpage using http://localhost:8080.

Database operation issues

Here are the common issues related to the database operations:

  1. Check the everest-controller logs

    Check the everest-controller logs if the Instance object has not been created or if there are any issues.

    kubectl logs -f deploy/everest-controller -n everest-system
    
  2. Check the Instance object

    Verify the status and events of the Instance object. If it is not healthy, further investigation is required. Describing the object provides details about the database configuration.

    kubectl get instance <instance-name> -n everest-system
    kubectl describe instance <instance-name> -n everest-system
    kubectl get instance <instance-name> -n everest-system -o yaml
    
  3. Check the provider’s underlying database object

    From the Instance, the provider creates the database operator’s own custom resource (for example, a PerconaServerMongoDB). Inspect it and the provider’s operator logs:

    kubectl get <db-cr> <name> -n everest-system
    kubectl describe <db-cr> <name> -n everest-system
    kubectl logs -f deploy/<database-operator-deployment> -n everest-system
    
  4. Check the database pod logs

    kubectl logs -f <database-pod-name> -c <database-container-name>