Description

This PR adds grad_ellm, a target-dependent, token-level attribution method for decoder-only Llama and Mistral models.

The method combines internal Q/K/V projections with gradients of the selected next-token objective and sums attribution scores over the last n_layers layers. The paper-reproduction configuration uses attributed_fn="logit" and normalize=True.

The integration includes:

• Native method registration, allowing users to select grad_ellm through inseq.load_model.
• Dedicated token-level output classes that preserve both token axes and support Inseq visualization, aggregation, and serialization.
• Internal helpers for exact token-ID replay and indexed CUDA device handling.
• Tests covering the reference equations, MHA/GQA, batching and padding, attribution objectives, hook cleanup, method switching, and output processing.
• A method card documenting the algorithm, configuration, and supported scope.

This PR also updates the Value Zeroing hidden-state hook to handle Tensor, tuple, and list block outputs while preserving the full batch, device, and dtype. This addresses compatibility with Transformer layers that return a Tensor directly.

The initial scope is standard floating-point Llama and Mistral models on a single CPU or CUDA device. Quantization, model sharding/offload, training mode, and other model families are outside this implementation’s supported scope.

Related Issue

N/A.

Type of Change

• 🚀 New feature
• 🔧 Bug fix
• 📚 Documentation update

Validation

• Complete make test & fast-test validation.
• The build docs hook passed in that run.
• make lint currently fails at the Safety check, which reports 113 findings across 20 packages.

Comparison of the provided dependency files against upstream Inseq v0.7.1 (dcf6cd4bd4a5c9134af6fad38e930f3671c713b3) shows that 112 findings concern 19 exact package versions already present in the upstream lockfile.

The remaining finding concerns wheel 0.45.1, which is absent from both lockfiles but is bundled with their shared setuptools 80.9.0 version. Its installed origin still needs confirmation.

A clean upstream environment scan has not yet been reproduced. These dependency findings remain unresolved, and make lint is not claimed to pass.

Checklist

• Read the [Code of Conduct](https://github.com/inseq-team/inseq/blob/main/CODE_OF_CONDUCT.md).
• Read the [contribution guide](https://github.com/inseq-team/inseq/blob/main/CONTRIBUTING.md).
• Successfully run the style checks with make fix-style.
• Add tests for the new methods and classes, and successfully run the complete make test suite.
• Use Google-style docstrings for the new methods and classes.