FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

[KVM] StringIndexOutOfBoundsException in IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath makes the first Stop VM fail when a disk path contains "-iscsi-" (e.g. a SharedMountPoint pool mount directory) · Issue #14664 · apache/cloudstack · GitHub

Repository navigation

[KVM] StringIndexOutOfBoundsException in IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath makes the first Stop VM fail when a disk path contains "-iscsi-" (e.g. a SharedMountPoint pool mount directory) #14664

Open

Description

problem

Stopping a VM that has a volume from a SharedMountPoint (iSCSI-backed) pool attached fails the first time with:

java.lang.StringIndexOutOfBoundsException: Range [21, -1) out of bounds for length 58
  at com.cloud.hypervisor.kvm.storage.IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath(IscsiAdmStorageAdaptor.java:362)
  at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.disconnectPhysicalDiskByPath(KVMStoragePoolManager.java:213)
  at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3971)
  at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:98)

The exception is deterministic (same message every time, reproduced 4 times). The second Stop attempt succeeds. If the volume is detached first, the Stop succeeds on the first attempt. The index pair (start 21, end -1) looks like an indexOf(...) returning -1 and being used as the end of a substring, which would mean the disk path format does not match what the parser expects. We have not confirmed this.

versions

  • Apache CloudStack 4.22.1.1
  • Hypervisor: KVM
  • Primary storage: pool of type SharedMountPoint, backed by iSCSI LUNs

The steps to reproduce the bug

  1. Create a VM with a data volume on a SharedMountPoint (iSCSI-backed) pool and attach it.
  2. Stop the VM. The stop fails with the exception above.
  3. Stop the VM again. It stops normally.
  4. Start the VM, detach the volume, stop the VM. It stops on the first attempt.

What to do about it?

Validate the path in disconnectPhysicalDiskByPath before calling substring, and return false (or log and skip) for paths that are not iSCSI device paths instead of throwing. It would also help to understand why the second Stop works.

Activity

  1. famerico commented on Nov 10, 2026

    Author

    We found the root cause by reading the code on tag 4.22.1.1, and it is narrower than the title of this issue suggests.

    KVMStoragePoolManager.disconnectPhysicalDiskByPath(path) passes the local path of every disk of the VM to every registered adaptor. IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath only skips a path if it does not contain -iscsi-:

    String search1 = "/dev/disk/by-path/ip-";
    ...
    if (!localPath.contains(search3)) {   // search3 = "-iscsi-"
        return false;
    }
    int index = localPath.indexOf(search2);   // search2 = ":"
    String host = localPath.substring(search1.length(), index);

    It then assumes a path shaped like /dev/disk/by-path/ip-<host>:<port>-iscsi-<iqn>-lun-<n>.

    In our setup the disk of the SharedMountPoint pool is /mnt/<pool dir>/<uuid>, and the pool directory name contains -iscsi-. The path is 58 characters long, passes the contains("-iscsi-") check and has no :. indexOf(":") returns -1 and search1.length() is 21, which gives exactly the observed Range [21, -1) out of bounds for length 58.

    So the trigger is a mount path containing -iscsi-, not the storage being iSCSI-backed. A SharedMountPoint pool whose directory name does not contain that string should not hit this. We have not run that variant yet.

    Suggested fix: guard on the expected prefix (localPath.startsWith(search1)) and check that the indexOf results are not -1 before calling substring, returning false otherwise.

    Why the second Stop works is still not verified. Our hypothesis is that the first attempt already stops the domain and fails only in cleanupDisk, and the retry finds the VM stopped and does not run the cleanup again.

  2. changed the title [-][KVM] StringIndexOutOfBoundsException in IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath makes the first Stop VM fail when a SharedMountPoint/iSCSI volume is attached[/-] [+][KVM] StringIndexOutOfBoundsException in IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath makes the first Stop VM fail when a disk path contains "-iscsi-" (e.g. a SharedMountPoint pool mount directory)[/+] on Nov 10, 2026
  3. famerico commented on Nov 10, 2026

    Author

    Additional confirmation of the root cause from a controlled test on our lab (CloudStack 4.22.1.1, KVM):

    • Same filesystem (same multipath device, same files and UUIDs) bind-mounted at /mnt/block-primary (no -iscsi- in the name) and registered as a new SharedMountPoint primary storage.
    • A volume was created in that pool and attached to a test VM.
    • Stop Instance succeeded on the first attempt, with no exception (Running -> Stopping -> Stopped, no retry).
    • With the original mount directory /mnt/fa-iscsi-primary, the first Stop failed every time with Range [21, -1) out of bounds for length 58.

    This matches the code analysis above: the only variable that changed is the mount directory name, so the trigger is the -iscsi- substring in the disk path, not the pool type.

    Not tested: VM migration and volume detach, which also call cleanupDisk (LibvirtMigrateCommandWrapper, KVMStorageProcessor). Our lab has no spare hosts, so they may be affected in the same way.

  4. added this to the 4.22.3 milestone on Nov 11, 2026
  5. github-actions commented on Nov 11, 2026

    Warning

    threat detection engine error
    The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

    What happened

    The threat detection engine failed to produce results.

    Review the workflow run logs for details.

    🎯 Triage report

    On KVM, the first Stop of a VM with a disk on a SharedMountPoint pool fails with a StringIndexOutOfBoundsException in IscsiAdmStorageAdaptor.disconnectPhysicalDiskByPath. The method checks only that the path contains -iscsi-, then assumes the /dev/disk/by-path/ip-...:port-iscsi-... shape. A mount directory containing -iscsi- passes that check, and indexOf(":") returns -1. The reporter's lab confirmed that the same storage mounted at a directory without -iscsi- stops on the first attempt.

    📊 Assessment

    Dimension Value Reasoning
    Type type:bug Unguarded substring on an unexpected path shape
    Component component:kvm Thrown in the KVM agent's iSCSI storage adaptor
    Severity Severity:Minor Stop fails once, and the retry succeeds
    Labels type:bug, component:kvm, Severity:Minor
    Coding agent Suitable The guard is specified: check the prefix and indexOf results, return false otherwise

    🔗 Similar issues

    💡 Notes and suggestions
    • Reproduction: mount an iSCSI-backed SharedMountPoint under a directory that contains -iscsi-, attach a data volume, then stop the VM.
    • The reporter's hypothesis for why the second stop works (the domain is already stopped) is not verified.
    • Migration and volume detach also call cleanupDisk; check them with the same guard.

    Generated by Daily Issue Triage · copilot · haiku55 · ~6 AIC · ⊞ 3.8K · ◷

    Add this agentic workflow to your repo

    To install this agentic workflow, run

    gh aw add githubnext/agentics/workflows/daily-issue-triage.md@139fa09ef17cde4135e696ff8e7af09bc5ddfa6e
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions


      Back | FazBrowse Home | New Git URL