This guide is for business module developers. It focuses on:

1. What Loader solves.
2. When to use it.
3. What each core class is responsible for.
4. How to integrate it in business modules.
5. How to diagnose common failures.

fe-extension-loader is the reusable plugin runtime foundation for Doris FE.

It unifies repeated loading logic across modules, including:

1. Scanning pluginRoots.
2. Resolving jars under plugin directories.
3. Building child-first classloaders.
4. Discovering typed factories via ServiceLoader.
5. Aggregating load successes and failures.

Use Loader if your module needs:

1. External plugin loading from directories.
2. Shared loading framework across multiple business modules.
3. Standardized failure semantics (scan, resolve, discover, etc.).
4. Less duplicated runtime loading code.

Loader may not be a direct fit if:

1. Plugin source is not directory-based and you already have a custom loading pipeline.
2. You only need SPI contracts and no runtime loading.

In those cases, use fe-extension-spi only.

This is the runtime entry class and unified facade.

Primary methods:

1. loadAll(...)
2. get(pluginName)
3. list()

Business modules should depend on this class directly.

The manager performs:

1. Scan plugin subdirectories under each root.
2. Collect jars from pluginDir/*.jar and pluginDir/lib/*.jar.
3. Create classloader.
4. Discover and validate factory (exactly one per directory).
5. Handle duplicate names and record failures.
6. Return LoadReport.

Low-level utility class. It does not scan directories. It only handles:

1. Classloader creation.
2. Typed factory discovery.

Use it when you already own classloader lifecycle externally.

Classloading behavior:

1. Child-first by default.
2. Parent-first for allowlisted package prefixes.

Purpose:

1. Reduce dependency conflict with FE process classpath.
2. Keep SPI interface class source consistent and avoid type-isolation ClassCastException.

Used to configure parent-first prefixes:

1. Mandatory prefixes (always included by default).
2. Business prefixes (append per module).

Common business examples:

1. org.apache.doris.authentication.
2. org.apache.doris.authorization.

Represents one successfully loaded plugin, including:

1. pluginName
2. pluginDir
3. resolvedJars
4. classLoader
5. factory
6. loadedAt

Business modules typically consume pluginName + factory for registration.

Represents one failed plugin directory load, including:

1. pluginDir
2. stage
3. message
4. cause

Failure stages:

1. scan
2. resolve
3. createClassLoader
4. discover
5. instantiate
6. conflict

Represents the full result of one loadAll call, including:

1. Success list: successes
2. Failure list: failures
3. Statistics: rootsScanned, dirsScanned

DirectoryPluginRuntimeManager stores loaded handles in an internal concurrent map.

No separate PluginRuntimeRegistry abstraction is exposed in current implementation.

Your business factory interface should extend PluginFactory.

DirectoryPluginRuntimeManager<MyPluginFactory> runtime =
        new DirectoryPluginRuntimeManager<>();

ClassLoadingPolicy policy = new ClassLoadingPolicy(
        Collections.singletonList("org.apache.doris.mybiz."));

LoadReport<MyPluginFactory> report = runtime.loadAll(
        pluginRoots,
        Thread.currentThread().getContextClassLoader(),
        MyPluginFactory.class,
        policy);

for (LoadFailure failure : report.getFailures()) {
    LOG.warn("plugin load failure: dir={}, stage={}, message={}",
            failure.getPluginDir(), failure.getStage(), failure.getMessage(), failure.getCause());
}

for (PluginHandle<MyPluginFactory> handle : report.getSuccesses()) {
    factoryMap.putIfAbsent(handle.getPluginName(), handle.getFactory());
}

Recommended layout:

<pluginRoot>/
  <pluginA>/
    pluginA.jar
    lib/
      dep1.jar
      dep2.jar
  <pluginB>/
    pluginB.jar

Rules:

1. Only direct subdirectories under pluginRoot are scanned.
2. Each plugin directory must contain at least one jar.
3. Each plugin directory must discover exactly one factory.

Current default strategy:

1. Keep the first successfully loaded plugin.
2. Record later duplicates as conflict.
3. Continue loading other directories.

Business recommendations:

1. Treat conflict as warning/alert.
2. Avoid duplicate plugin names in production plugin roots.

Loader returns LoadReport and does not force exception.

Business modules choose policy by semantics:

1. Tolerant mode: log failures and continue startup.
2. Strict mode: fail-fast when no successful plugin exists.

LoadReport should be startup decision input, not just logs.

Recommended goals:

1. All successful plugins are registered.
2. Every failure is traceable by stage and directory.
3. Startup behavior is deterministic (strict or tolerant).
4. Conflict/failure paths do not leak resources.

Recommended processing order:

1. Log summary metrics: rootsScanned, dirsScanned, successes.size, failures.size.
2. Log each failure with pluginDir + stage + message + cause.
3. Register successful plugins to business map (pluginName -> factory).
4. Apply startup decision logic (strict/tolerant).

Suggested stage severity grouping:

1. Environment/config issues: scan, resolve.
2. SPI/implementation issues: discover, instantiate.
3. Runtime construction issues: createClassLoader.
4. Naming conflict: conflict (usually warning, not immediate stop).

Recommended decision rules:

1. dirsScanned == 0: often means empty roots or no external setup.
2. dirsScanned > 0 && successes.isEmpty() in strict mode: fail-fast.
3. dirsScanned > 0 && successes.isEmpty() in tolerant mode: warn and continue only if business allows no external plugin.
4. If requiredPluginNames exists, enforce presence even when partial loads succeeded.

public static <F extends PluginFactory> void processLoadReport(
        LoadReport<F> report,
        Map<String, F> factoryMap,
        boolean strictMode,
        Set<String> requiredPluginNames) {
    Objects.requireNonNull(report, "report");
    Objects.requireNonNull(factoryMap, "factoryMap");
    Objects.requireNonNull(requiredPluginNames, "requiredPluginNames");

    // Step 1: summary metrics
    LOG.info("plugin load summary: rootsScanned={}, dirsScanned={}, successCount={}, failureCount={}",
            report.getRootsScanned(),
            report.getDirsScanned(),
            report.getSuccesses().size(),
            report.getFailures().size());

    // Step 2: failure details
    LoadFailure firstNonConflictFailure = null;
    for (LoadFailure failure : report.getFailures()) {
        LOG.warn("plugin load failure: dir={}, stage={}, message={}",
                failure.getPluginDir(), failure.getStage(), failure.getMessage(), failure.getCause());
        if (!LoadFailure.STAGE_CONFLICT.equals(failure.getStage()) && firstNonConflictFailure == null) {
            firstNonConflictFailure = failure;
        }
    }

    // Step 3: register successful plugins
    int registered = 0;
    for (PluginHandle<F> handle : report.getSuccesses()) {
        F existing = factoryMap.putIfAbsent(handle.getPluginName(), handle.getFactory());
        if (existing != null) {
            // If business map already contains the name, close discarded external classloader.
            closeClassLoaderQuietly(handle.getClassLoader());
            LOG.warn("skip duplicated plugin name in business map: {}", handle.getPluginName());
            continue;
        }
        registered++;
    }

    // Step 4: startup decision (strict/tolerant)
    if (strictMode && report.getDirsScanned() > 0 && registered == 0 && firstNonConflictFailure != null) {
        throw new IllegalStateException(
                "No plugin loaded in strict mode: stage=" + firstNonConflictFailure.getStage()
                        + ", dir=" + firstNonConflictFailure.getPluginDir()
                        + ", message=" + firstNonConflictFailure.getMessage(),
                firstNonConflictFailure.getCause());
    }

    // Step 5: required plugin checks
    for (String required : requiredPluginNames) {
        if (!factoryMap.containsKey(required)) {
            throw new IllegalStateException("Required plugin is missing: " + required);
        }
    }
}

The closeClassLoaderQuietly implementation pattern can be referenced from:

../fe-authentication/fe-authentication-handler/src/main/java/org/apache/doris/authentication/handler/AuthenticationPluginManager.java

Current authentication module handling is:

1. Iterate report.getFailures() and log warnings.
2. Iterate report.getSuccesses() and register factories (close duplicated external classloader if needed).
3. Throw AuthenticationException when directories were scanned but no external plugin was loaded.

Reference implementation:

../fe-authentication/fe-authentication-handler/src/main/java/org/apache/doris/authentication/handler/AuthenticationPluginManager.java

Supported:

1. loadAll
2. get
3. list

Not supported:

1. reload
2. unload

Do not depend on runtime hot-reload semantics in V1.

Check:

1. Plugin jar includes META-INF/services/<factoryType>.
2. Service file class name is correct.
3. Provider class is included in final jar.

Check:

1. Multiple jars may declare the same factory type.
2. Parent classpath may pollute service resources.

Note:

DirectoryPluginRuntimeManager includes parent service-resource filtering and prefers plugin-directory-local discovery.

Check:

1. Business layer may keep stale handle references.
2. Conflict/failure paths may skip classloader close.
3. Plugin instance close() may not release resources.

Authentication integration sample:

../fe-authentication/fe-authentication-handler/src/main/java/org/apache/doris/authentication/handler/AuthenticationPluginManager.java

Key integration points:

1. Use DirectoryPluginRuntimeManager<AuthenticationPluginFactory>.
2. Append authentication parent-first prefix.
3. Register successful factories into authentication factory map.
4. Close classloader for discarded conflicting handles.

1. Chinese developer guide: README_CN.md
2. Unified runtime design (CN): ../fe-authentication/EXTENSION_LOADER_UNIFIED_DESIGN_CN.md
3. SPI contracts: ../fe-extension-spi/README.md