FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Introduce index parameter in place of only_last_token by JRosenkranz · Pull Request #445 · foundation-model-stack/foundation-model-stack · GitHub

Repository navigation

Introduce index parameter in place of only_last_token - #445

Open
JRosenkranz wants to merge 4 commits into
mainfrom
only_last_token_rework
Open

JRosenkranz wants to merge 4 commits into
mainfrom
only_last_token_rework

Conversation

Copy link
Copy Markdown
Collaborator

this PR deprecates the only_last_token parameter in favor of an index which can be much more generic. The index can be one of an int, a tensor, or None where an int will gather the same index (token) for each sequence in the batch, a tensor will gather potentially different index's (token) for each sequence in the batch, and None will return the whole sequence. This was primarily added in the case a user uses right padding, where the last token exist somewhere in the middle of the sequence.

Signed-off-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
Signed-off-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
Signed-off-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
JRosenkranz requested a review from ani300 July 9, 2025 19:49
JRosenkranz self-assigned this Jul 9, 2025

This branch has not been deployed

No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL