Scope DeepCompile compiler state to graph and engine lifecycles - #8159
Conversation
Sorry, something went wrong.
Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f8f69327a
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Sorry, something went wrong.
Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com>
|
The changes look good to me. One question about the assumptions of use. This PR seems relevant to use cases where multiple engines are created within a single process. Do we assume that the forward of two engines do not overlap? If they do, the patched torch.autograd.Function will be active when another model has completed compilation, probably leading to recompilation issues similar to what #7980 had seen. I have not seen any use case running multiple engines in an overlapping manner, though. So the question is just out of curiosity. |
Sorry, something went wrong.
|
Hi @eternalNight, This PR only makes captured backward inputs and cleanup owner-specific, but the torch.autograd.Function patch is still process-global. This can cause the same kind of repeated recompilation that #7980 fixed. I haven’t seen a real workload relying on that pattern either, but it is a real limitation. To address the issue, I think we could limit the behavior to the owning engine/frame. For example, only resetting compiled_bw while that engine's frame_key is still pending. Since we are not aware of a workload that interleaves multiple engines in this way, I would prefer to avoid adding it in this PR and revisit it if such a use case arises. |
Sorry, something went wrong.
DeepCompile uses several process-global PyTorch compiler mechanisms while building graph-specific state. Scheduled recompilation also replaces previously compiled graphs.
These lifetimes did not have explicit ownership boundaries.
This PR gives those existing mechanisms explicit graph and engine lifecycle ownership.
Fixes: