| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
WalkthroughAdds generated opcode enums (RealOpcode, PseudoOpcode) and an Opcode wrapper to compiler/core, a Python generator script to emit the generated file, stdlib and VM updates to use the new opcode types, marks the generated file in .gitattributes, adds dependencies, and updates a spelling dictionary token. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Caller
participant Std as stdlib::opcode::stack_effect
participant Core as compiler_core::opcode::Opcode
participant Real as compiler_core::opcodes::RealOpcode
participant Pseudo as compiler_core::opcodes::PseudoOpcode
Caller->>Std: stack_effect(raw_id, oparg)
Std->>Core: Opcode::try_from(raw_id)
alt Pseudo (u16)
Core->>Pseudo: PseudoOpcode::try_from(raw)
Pseudo-->>Std: PseudoOpcode
Std->>Std: apply pseudo-specific stack logic
Std-->>Caller: stack delta
else Real (u8)
Core->>Real: RealOpcode::try_from(raw_u8)
Real-->>Std: RealOpcode
Std->>Std: use RealOpcode::num_popped/num_pushed/deopt
Std-->>Caller: stack delta
else Invalid
Std-->>Caller: invalid / error
end
sequenceDiagram
autonumber
actor Dev
participant Script as scripts/gen_opcodes.py
participant Output as compiler/core/src/opcodes.rs
participant Repo as Git
Dev->>Script: run generator (analyze CPython)
Script->>Output: emit enums, predicates, TryFrom impls, helpers
Script->>Repo: write file & run cargo fmt
Repo-->>Output: file added (marked in .gitattributes)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
📜 Recent review details Configuration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro 📥 CommitsReviewing files that changed from the base of the PR and between 3daec5a and 9436354. 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. ❤️ ShareComment @coderabbitai help to get the list of available commands and usage tips. |
Sorry, something went wrong.
|
I have updated the main PR comment |
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)compiler/core/src/opcode.rs (3)📜 Review details7-8: Consider adding accessor and additional trait implementations.
The OpcodeId newtype lacks a public accessor for the inner u16 value. Additionally, consider implementing Hash, Display, and potentially FromStr for better ergonomics.
Apply this diff to add an accessor:
#[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct OpcodeId(u16); impl OpcodeId { + /// Returns the raw opcode ID value. + #[must_use] + pub const fn get(self) -> u16 { + self.0 + } +For Hash, update the derives:
-#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[derive(Clone, Copy, Debug, Eq, PartialEq, Hash)] pub struct OpcodeId(u16);
235-243: Enhance safety documentation.
The safety documentation for new_unchecked should reference the is_valid method to clarify what constitutes a valid opcode ID.
Apply this diff:
/// Creates a new opcode ID without checking the value is a valid opcode ID. /// /// # Safety /// - /// The value should be a valid opcode ID. + /// The caller must ensure that `id` satisfies `Self::is_valid(id)`. #[must_use] pub const unsafe fn new_unchecked(id: u16) -> Self {
545-560: Consider a more descriptive error type for TryFrom.
The TryFrom implementations use () as the error type, providing no information about why a conversion failed. While this is acceptable, a custom error type could improve debuggability.
Consider defining an error type:
/// Error type for invalid opcode ID conversions. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct InvalidOpcodeId; impl std::fmt::Display for InvalidOpcodeId { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { write!(f, "invalid opcode ID") } } impl std::error::Error for InvalidOpcodeId {}Then update the macro:
macro_rules! opcode_id_try_from_impl { ($t:ty) => { impl TryFrom<$t> for OpcodeId { - type Error = (); + type Error = InvalidOpcodeId; fn try_from(value: $t) -> Result<Self, Self::Error> { - let id = value.try_into().map_err(|_| ())?; + let id = value.try_into().map_err(|_| InvalidOpcodeId)?; if Self::is_valid(id) { Ok(Self(id)) } else { - Err(()) + Err(InvalidOpcodeId) } } }
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📥 CommitsReviewing files that changed from the base of the PR and between 3a6fda4 and db2775a.
⛔ Files ignored due to path filters (1)📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Format Rust code with the default rustfmt style (run cargo fmt)
Run clippy and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use RustPython macros (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust
Files:
compiler/core/src/opcode.rs (9)compiler/core/src/opcode.rs (1)stdlib/src/opcode.rs (8)
- is_pseudo (253-269)
- is_valid (247-249)
- has_arg (273-426)
- has_const (430-435)
- has_name (439-465)
- has_jump (469-487)
- has_free (491-500)
- has_local (504-519)
- has_exc (523-542)
- is_valid (70-72)
- has_arg (75-77)
- has_const (80-82)
- has_name (85-87)
- has_jump (90-92)
- has_free (95-97)
- has_local (100-102)
- has_exc (105-107)
stdlib/src/opcode.rs (8)stdlib/src/opcode.rs (3)
- is_valid (70-72)
- has_arg (75-77)
- has_const (80-82)
- has_name (85-87)
- has_jump (90-92)
- has_free (95-97)
- has_local (100-102)
- has_exc (105-107)
scripts/gen_opcodes.py (1)🪛 Flake8 (7.3.0) scripts/gen_opcodes.pycompiler/core/src/bytecode.rs (1)
- Instruction (81-119)
compiler/core/src/opcode.rs (8)
- try_from (814-820)
- is_valid (247-249)
- has_arg (273-426)
- has_const (430-435)
- has_name (439-465)
- has_jump (469-487)
- has_free (491-500)
- has_local (504-519)
- has_exc (523-542)
[error] 256-256: indentation contains mixed spaces and tabs
(E101)
[error] 257-257: indentation contains mixed spaces and tabs
(E101)
[error] 258-258: indentation contains mixed spaces and tabs
(E101)
[error] 259-259: indentation contains mixed spaces and tabs
(E101)
[error] 260-260: indentation contains mixed spaces and tabs
(E101)
[error] 261-261: indentation contains mixed spaces and tabs
(E101)
[error] 262-262: indentation contains mixed spaces and tabs
(E101)
[error] 263-263: indentation contains mixed spaces and tabs
(E101)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)scripts/gen_opcodes.py (2)compiler/core/src/opcode.rs (1)75-78: ...
75-78: ...
11-233: Refine opcode verification scope
Adaptive and instrumented opcodes (e.g. CALL_ALLOC_AND_ENTER_INIT, INSTRUMENTED_*, etc.) are RustPython-specific and must not be removed; only verify that the standard CPython 3.13.7 opcodes (those without ADAPTIVE_ or INSTRUMENTED_ prefixes) match their CPython values.Likely an incorrect or invalid review comment.
Sorry, something went wrong.
| /// Whether opcode ID have 'HAS_PURE_FLAG' set. | ||
| #[must_use] | ||
| pub const fn has_exc(&self) -> bool { |
There was a problem hiding this comment.
⚠️ Potential issue | 🟡 Minor
Fix misleading documentation comment.
The documentation comment says "Whether opcode ID have 'HAS_PURE_FLAG' set" but the method is named has_exc. This appears to be a copy-paste error.
Apply this diff to fix the comment:
- /// Whether opcode ID have 'HAS_PURE_FLAG' set.
+ /// Whether opcode ID have 'HAS_EXC_FLAG' set.
#[must_use]
pub const fn has_exc(&self) -> bool {‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Whether opcode ID have 'HAS_PURE_FLAG' set. | |
| #[must_use] | |
| pub const fn has_exc(&self) -> bool { | |
| /// Whether opcode ID have 'HAS_EXC_FLAG' set. | |
| #[must_use] | |
| pub const fn has_exc(&self) -> bool { |
In compiler/core/src/opcode.rs around lines 521 to 523, the doc comment above
pub const fn has_exc(&self) -> bool is incorrect and looks copy-pasted from
another method; update the comment to correctly describe this method (e.g.,
"Whether opcode ID has 'HAS_EXC_FLAG' set.") and fix grammar ("has" instead of
"have") so the comment matches the method name and purpose.
Sorry, something went wrong.
There was a problem hiding this comment.
this is intended
Sorry, something went wrong.
There was a problem hiding this comment.
🏁 Script executed:
#!/bin/bash # Check the generation script to see how has_exc method is generated cat scripts/gen_opcodes.py | grep -A 10 -B 5 "has_exc\|HAS_PURE"
Length of output: 1411
🏁 Script executed:
#!/bin/bash # Search for any references to HAS_PURE_FLAG or has_exc in the codebase rg -n "HAS_PURE|has_exc" --type=py --type=rust -C 3
Length of output: 5081
@ShaharNaveh, understood! I can see from the generation script that this is intentional:
def generate_has_exc(self) -> str:
return self._generate_has_attr("exc", flag_override="HAS_PURE_FLAG")The method name has_exc() reflects the semantic meaning (opcodes related to exception handling), while the doc comment accurately documents which CPython flag is actually being checked (HAS_PURE_FLAG). This design choice maintains alignment with CPython's internal flag naming while providing a more descriptive API method name.
Thank you for the clarification!
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)stdlib/src/opcode.rs (1)📜 Review details26-26: Track the TODO for auto-generated Instruction enum.
This TODO references future work to leverage the auto-generated Instruction enum. Consider tracking this as a follow-up task.
Do you want me to open a new issue to track this task?
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📥 CommitsReviewing files that changed from the base of the PR and between db2775a and 7fa003b.
📒 Files selected for processing (1)📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Format Rust code with the default rustfmt style (run cargo fmt)
Run clippy and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use RustPython macros (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust
Files:
scripts/gen_opcodes.py (1)⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)compiler/core/src/bytecode.rs (1)
- Instruction (81-119)
compiler/core/src/opcode.rs (8)
- try_from (814-820)
- is_valid (247-249)
- has_arg (273-426)
- has_const (430-435)
- has_name (439-465)
- has_jump (469-487)
- has_free (491-500)
- has_local (504-519)
- has_exc (523-542)
stdlib/src/opcode.rs (3)57-59: LGTM! TypeError preservation implemented correctly.
The error handling now correctly preserves TypeError when the opcode argument is not an integer, while only converting Instruction::try_from failures to ValueError. This directly addresses the feedback from the previous review.
14-14: LGTM! Correct type for the flag.
Changing ENABLE_SPECIALIZATION from i8 to u8 is appropriate for a boolean-like flag constant.
71-101: LGTM! Consistent pattern across all predicates.
The has_* functions follow a consistent pattern using OpcodeId::try_from(opcode).is_ok_and(|oid| !oid.is_pseudo() && oid.has_*()), which correctly filters out pseudo-opcodes and delegates to the generated predicates.
Sorry, something went wrong.
| #[pyfunction] | ||
| fn is_valid(opcode: i32) -> bool { | ||
| Opcode::is_valid(opcode) | ||
| OpcodeId::try_from(opcode).is_ok() |
There was a problem hiding this comment.
⚠️ Potential issue | 🟠 Major
Consider filtering pseudo-opcodes in is_valid for consistency.
All the has_* predicates (lines 71-101) explicitly filter out pseudo-opcodes with !oid.is_pseudo(), but is_valid does not. Pseudo-opcodes are typically not valid for runtime bytecode execution. For consistency with the other predicates and to match typical CPython semantics, is_valid should likely also exclude pseudo-opcodes.
Apply this diff to filter pseudo-opcodes:
fn is_valid(opcode: i32) -> bool {
- OpcodeId::try_from(opcode).is_ok()
+ OpcodeId::try_from(opcode).is_ok_and(|oid| !oid.is_pseudo())
}‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| OpcodeId::try_from(opcode).is_ok() | |
| fn is_valid(opcode: i32) -> bool { | |
| OpcodeId::try_from(opcode).is_ok_and(|oid| !oid.is_pseudo()) | |
| } |
In stdlib/src/opcode.rs around line 66, update the is_valid implementation to exclude pseudo-opcodes like the other has_* predicates: after converting opcode to an OpcodeId via try_from, return true only if the conversion succeeds and the resulting OpcodeId is not a pseudo-opcode (i.e., check !oid.is_pseudo()); ensure you handle the Err case from try_from by returning false.
Sorry, something went wrong.
There was a problem hiding this comment.
@coolreader18 What will be the best way to solve it?
@ShaharNaveh Do we need to have different type for Instruction and RealOpCode? Otherwise, just in case, can it be defined like:
struct RealOpCode(bytecode::Instruction);
Sorry, something went wrong.
| impl TryFrom<i8> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: i8) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<i16> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: i16) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<i32> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: i32) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<i64> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: i64) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<i128> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: i128) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<isize> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: isize) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<u8> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(id: u8) -> Result<Self, Self::Error> { | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<u16> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: u16) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<u32> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: u32) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<u64> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: u64) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<u128> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: u128) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<usize> for RealOpcode { | ||
| type Error = (); | ||
|
|
||
| fn try_from(raw: usize) -> Result<Self, Self::Error> { | ||
| let id = raw.try_into().map_err(|_| ())?; | ||
| if Self::is_valid(id) { | ||
| // SAFETY: We just validated that we have a valid opcode id. | ||
| Ok(unsafe { Self::new_unchecked(id) }) | ||
| } else { | ||
| Err(()) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
is this applicable?
https://docs.rs/num_enum/latest/num_enum/derive.TryFromPrimitive.html
Also for the psuedo opcode
Sorry, something went wrong.
There was a problem hiding this comment.
Yeah, this is great!
will modify the code
Sorry, something went wrong.
Yes, because opcode an instruction is comprised out of an opcode and a optional oparg. imo opcode should be it's own enum. An instruction should contain a RealOpcode + Option<T>. where T can be:
For Instruction we have a Instruction::new constructor that looks like: impl<T: OpargType> Instruction<T> {
pub fn new(opcode: RealOpcode, raw_oparg: u32) -> Result<Self, MarshalError> {
if !opcode.has_arg() {
return Ok(Self {opcode, oparg: None });
}
let oparg = match opcode {
RealOpcode::Resume => ResumeOpcode::try_from(raw_oparg)?,
// More typed oparg checks for specific opcodes
_ => Oparg(raw_oparg)
};
Ok(Self { opcode, oparg: Some(oparg) })
}
} |
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)stdlib/src/opcode.rs (1)📜 Review detailsscripts/gen_opcodes.py (1)131-133: Filter pseudo-opcodes in is_valid to match CPython semantics.
Past review comments correctly noted that is_valid should reject pseudo-opcodes, as they are not valid for runtime bytecode execution and CPython's opcode.is_valid excludes them. Currently, Opcode::try_from(opcode).is_ok() returns true for pseudo-opcodes.
Apply this diff to filter pseudo-opcodes:
#[pyfunction] fn is_valid(opcode: i32) -> bool { - Opcode::try_from(opcode).is_ok() + matches!(Opcode::try_from(opcode), Ok(Opcode::Real(_))) }53-78: Guard against empty input in group_ranges.
The function unconditionally accesses nums[0] on line 73, raising IndexError if the input iterable is empty. While CPython's opcode metadata likely never produces empty ranges in practice, defensive coding would prevent potential future issues.
Apply this diff to add an early return:
def group_ranges(it: "Iterable[int]") -> "Iterator[range]": """ Group consecutive numbers into ranges. Parameters ---------- it : Iterable[int] Numbers to group into ranges. Notes ----- Numbers in `it` must be sorted in ascending order. Examples -------- >>> nums = [0, 1, 2, 3, 17, 18, 42, 50, 51] >>> list(group_ranges(nums)) [range(0, 4), range(17, 19), range(42, 43), range(50, 52)] """ nums = list(it) + if not nums: + return start = prev = nums[0]
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📥 CommitsReviewing files that changed from the base of the PR and between 1ae0c7e and aca4508.
⛔ Files ignored due to path filters (1)📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Format Rust code with the default rustfmt style (run cargo fmt)
Run clippy and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use RustPython macros (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust
Files:
compiler/core/src/bytecode.rs (2)scripts/gen_opcodes.py (1)compiler/core/src/opcode.rs (2)
- stack_effect (1391-1516)
- try_from (814-820)
compiler/core/src/opcodes.rs (14)
- try_from (13-20)
- try_from (26-29)
- has_arg (303-445)
- has_arg (1026-1038)
- has_const (449-454)
- has_const (1042-1044)
- has_name (527-549)
- has_name (1079-1087)
- has_jump (490-506)
- has_jump (1067-1069)
- has_free (477-486)
- has_free (1061-1063)
- has_local (510-523)
- has_local (1073-1075)
- has_exc (458-473)
- has_exc (1048-1057)
compiler/core/src/opcodes.rs (1)compiler/core/src/opcodes.rs (1)
- deopt (228-299)
stdlib/src/opcode.rs (8)⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- has_arg (136-138)
- has_const (141-143)
- has_exc (166-168)
- has_free (156-158)
- has_jump (151-153)
- has_local (161-163)
- has_name (146-148)
- is_valid (131-133)
compiler/core/Cargo.toml (1)stdlib/src/opcode.rs (3)20-21: LGTM! Dependencies align with the new opcode system.
The addition of num_enum and num-traits appropriately supports the generated opcode infrastructure. num_enum provides the TryFromPrimitive derive macro used in opcodes.rs, and num-traits supplies the PrimInt trait bound for the error constructor.
scripts/gen_opcodes.py (1)7-11: LGTM! Updated imports and type change.
The imports now correctly reference the new Opcode, PseudoOpcode, and RealOpcode types from the vm::opcode module. The change of ENABLE_SPECIALIZATION from i8 to u8 aligns with CPython's unsigned value and removes unnecessary signedness.
23-117: Stack effect implementation aligns well with CPython.
The refactored stack_effect correctly handles RealOpcode and PseudoOpcode separately, mirrors CPython's special-case treatment of ExitInitCheck, rejects specialized opcodes via deopt(), and properly computes stack effects for pseudo-opcodes with jump-dependent logic.
119-168: LGTM! Macro-based real opcode checks.
The real_opcode_check! macro correctly filters out pseudo-opcodes by attempting conversion to RealOpcode via u8::try_from and delegates to the appropriate has_* predicate. This ensures all has_arg, has_const, has_name, has_jump, has_free, has_local, and has_exc functions only operate on real opcodes, consistent with CPython semantics.
compiler/core/src/opcodes.rs (1)112-343: Well-structured generator design.
The abstraction via InstructionsMeta, RealInstructions, and PseudoInstructions cleanly separates concerns and generates comprehensive Rust opcode scaffolding including predicates, stack effects, deoptimization mappings, and TryFrom conversions. The CPython variant-to-Rust identifier substitution and automatic cargo fmt invocation are nice touches for maintainability.
compiler/core/src/opcode.rs (2)1-1109: Generated opcode definitions are comprehensive and correct.
The auto-generated RealOpcode (repr(u8), 223 variants) and PseudoOpcode (repr(u16), 12 variants) enums provide exhaustive CPython 3.13.7 opcode coverage. The implementation includes:
- deopt() mappings for specialized opcodes
- Predicate methods (has_arg, has_const, has_exc, has_free, has_jump, has_local, has_name)
- Stack effect calculations (num_popped, num_pushed) with oparg-dependent logic
- is_valid() range checks
- TryFromPrimitive conversions with custom MarshalError constructor
The generated code follows Rust best practices (const methods where possible, exhaustive matching, proper repr annotations).
1-8: LGTM! Clean opcode wrapper design.
The Opcode enum unifies RealOpcode and PseudoOpcode under a single type, enabling unified handling while preserving type safety. The public re-export of RealOpcode and PseudoOpcode maintains flexibility for code that needs direct access to the underlying types.
32-48: LGTM! Macro reduces TryFrom boilerplate.
The impl_try_from! macro generates consistent TryFrom implementations for signed and unsigned integer types by first converting to u16 and delegating to the existing TryFrom<u16> implementation. This reduces code duplication and ensures uniform error handling.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)scripts/gen_opcodes.py (1)🧹 Nitpick comments (4)49-50: Avoid assigning lambdas to methods (E731); use named defs.
Define helpers and assign them to StackOffset for clarity and lint compliance. (Similar to prior feedback.)
-StackOffset.pop = lambda self, item: self.popped.append(_var_size(item)) -StackOffset.push = lambda self, item: self.pushed.append(_var_size(item)) +def _stackoffset_pop(self, item): + self.popped.append(_var_size(item)) + +def _stackoffset_push(self, item): + self.pushed.append(_var_size(item)) + +StackOffset.pop = _stackoffset_pop +StackOffset.push = _stackoffset_push
scripts/gen_opcodes.py (3)📜 Review detailscompiler/core/src/opcodes.rs (1)14-21: Make CPython path configurable and fail with a clear message.
Hardcoding CPYTHON_PATH risks opaque failures. Allow an env override and validate the path.
-CPYTHON_PATH = ( - pathlib.Path(__file__).parents[2] / "cpython" -) +CPYTHON_PATH = pathlib.Path( + sys.environ.get("RUSTPYTHON_CPYTHON_PATH", "") +) or (pathlib.Path(__file__).parents[2] / "cpython") + +if not CPYTHON_PATH.exists(): + sys.exit(f"cpython checkout not found at: {CPYTHON_PATH}. " + "Set RUSTPYTHON_CPYTHON_PATH to override.")
253-256: Escape and order tokens in the regex replacement.
Unescaped alternation can mis-match; ordering prevents short-name shadowing.
- replacements = {name: enum_variant_name(name) for name in analysis.opmap} - inner_pattern = "|".join(replacements) + replacements = {name: enum_variant_name(name) for name in analysis.opmap} + # Longest-first to avoid partial matches (e.g., LOAD vs LOAD_CONST) + names = sorted(replacements.keys(), key=len, reverse=True) + inner_pattern = "|".join(re.escape(n) for n in names) pattern = re.compile(rf"\b({inner_pattern})\b") out = pattern.sub(lambda m: replacements[m.group(0)], out)
258-259: Surface formatting failures and keep output consistent.
Use check=True to fail fast; if cargo is missing, at least warn.
- print("Running `cargo fmt`") - subprocess.run(["cargo", "fmt"], cwd=ROOT) + print("Running `cargo fmt`") + try: + subprocess.run(["cargo", "fmt"], cwd=ROOT, check=True) + except Exception as e: + print(f"Warning: `cargo fmt` failed: {e}", file=sys.stderr)227-299: Optional: simplify deopt for readability.
Current Some(match ... _ => return None) works; consider a direct match to avoid early return inside the expression.
- pub const fn deopt(&self) -> Option<Self> { - Some(match *self { + pub const fn deopt(&self) -> Option<Self> { + match *self { /* arms unchanged */ - _ => return None, - }) + _ => return None, + } }
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📥 CommitsReviewing files that changed from the base of the PR and between aca4508 and 3daec5a.
📒 Files selected for processing (2)📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Format Rust code with the default rustfmt style (run cargo fmt)
Run clippy and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
stdlib/src/opcode.rs (7)scripts/gen_opcodes.py (2)
- has_arg (136-138)
- has_const (141-143)
- has_exc (166-168)
- has_free (156-158)
- has_jump (151-153)
- has_local (161-163)
- has_name (146-148)
compiler/core/src/marshal.rs (1)🪛 Flake8 (7.3.0) scripts/gen_opcodes.pycompiler/core/src/opcodes.rs (1)
- instructions (188-195)
- deopt (228-299)
[error] 12-12: 'collections.abc.Iterable' imported but unused
(F401)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)compiler/core/src/opcodes.rs (2)9-12: Good use of num_enum for fallible conversions.
Derive + error_type hook is clean and const-friendly. LGTM.
988-1005: Approve PseudoOpcode derivation; num_enum dependency confirmed
repr(u16) + TryFromPrimitive matches design and num_enum is declared in Cargo.toml.
Sorry, something went wrong.
| import typing | ||
|
|
||
| if typing.TYPE_CHECKING: | ||
| from collections.abc import Iterable, Iterator |
There was a problem hiding this comment.
⚠️ Potential issue | 🟡 Minor
Fix unused import (flake8 F401).
Iterable isn’t used. Keep only Iterator.
-if typing.TYPE_CHECKING:
- from collections.abc import Iterable, Iterator
+if typing.TYPE_CHECKING:
+ from collections.abc import Iterator‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| from collections.abc import Iterable, Iterator | |
| if typing.TYPE_CHECKING: | |
| from collections.abc import Iterator |
[error] 12-12: 'collections.abc.Iterable' imported but unused
(F401)
🤖 Prompt for AI AgentsIn scripts/gen_opcodes.py around line 12, the import statement brings in both Iterable and Iterator but Iterable is unused; remove Iterable from the import and keep only Iterator so the line imports Iterator from collections.abc to satisfy flake8 F401.
Sorry, something went wrong.
| const fn new_invalid_bytecode<T: num_traits::int::PrimInt>(_: T) -> MarshalError {{ | ||
| MarshalError::InvalidBytecode | ||
| }} |
There was a problem hiding this comment.
⚠️ Potential issue | 🔴 Critical
🧩 Analysis chainConfirm num_traits is declared for generated code.
new_invalid_bytecode uses num_traits::int::PrimInt; ensure compiler/core/Cargo.toml includes num_traits.
You can verify with:
🏁 Script executed:
#!/bin/bash
rg -nP '^\s*num_traits\s*=' -g 'compiler/**/Cargo.toml'Length of output: 0
🏁 Script executed:
#!/bin/bash
# List all Cargo.toml files under compiler/
find compiler -type f -name Cargo.toml
# Search for any num_traits references in those toml files
rg -n 'num_traits' -g 'compiler/**/*.toml'Length of output: 127
Add num_traits dependency to compiler/core/Cargo.toml: Generated code invokes num_traits::int::PrimInt but no num_traits entry exists in any compiler/*.toml—add it under [dependencies].
🤖 Prompt for AI AgentsIn scripts/gen_opcodes.py around lines 248 to 250, the generated Rust uses num_traits::int::PrimInt but compiler/core/Cargo.toml has no num-traits dependency; add num-traits to the [dependencies] section (e.g. num-traits = "0.2") in compiler/core/Cargo.toml, run cargo update/build to ensure the crate resolves, and regenerate/verify the generated code compiles.
Sorry, something went wrong.
|
Somehow, our Instruction is a u8 backed type because Arg is a zero-sized type. This is a trick to type the argument but not actually owning a value. |
Sorry, something went wrong.
So, if I understand it correctly we must keep Instruction as an enum and make the variants to be equal to their opcode. So there's no point in having an Opcode enum, correct? |
Sorry, something went wrong.
|
I am currently not perfectly understanding how they are different in Python. |
Sorry, something went wrong.
Some opcodes that we have does not exist in Cpython & Some were changed. I thought it would be better if we would use CPython Lib/dis.py and test it with Lib/test_dis.py, as CPython tests are more in-depth than what we currently have. imo we can mimic CPython behavior of opcodes tiers and adaptive bytecode execution, it will probably improve our performance.
Completely agree. Using Rust safety features is better than forcing rust to behave like C. For now, I'll convert this PR to draft and try to see if I'm able to auto-generate the Instruction enum in a proper way |
Sorry, something went wrong.
|
I agree it will be great if dis.py perfectly work.
I agree here too. But it may not be always possible especially when CPython is updated. Because CPython doesn't guarantee bytecode compatibility, every python version can have different bytecode. We can patch bytecode to be aligned to CPython. But if we have incompatibility, |
Sorry, something went wrong.
Sounds good. I'll invest more time to see how we can auto generate the Instruction enum. and if it'll be too hard I'll try to make dis.py to work in another way 🥲 |
Sorry, something went wrong.
|
Now Python has exception table for code object, which is related to the newer except instructions. |
Sorry, something went wrong.
Are we aiming now for Python 3.14? what are the steps to achieve it? Updating the Instructions can't be only thing needed, right? |
Sorry, something went wrong.
|
If you are interested in 3.14 support, #5484 is the guideline |
Sorry, something went wrong.
|
Oh, to prevent miscommunication, the exception table is not related to 3.14. It is probably a 3.11 feature which we didn't implement yet. |
Sorry, something went wrong.
No worries. I think that this PR will stay open for a while as the changes required to use the updated instructions is very large. meanwhile I'll open small PRs to get us there slowly but surely |
Sorry, something went wrong.
|
@ShaharNaveh is this replaced by your recent refactorings? |
Sorry, something went wrong.
yup! |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
This PR's original goal was to auto generate an enum that will replace:
RustPython/compiler/core/src/bytecode.rs
Lines 539 to 795 in 3a6fda4
RustPython/compiler/core/src/bytecode.rs
Lines 1335 to 1697 in 3a6fda4
But turned out to be more work than originally anticipated.
From what I see ATM our largest gaps are that we have opcodes that doesn't exist in Cpython (3.13.7) such as
RustPython/compiler/core/src/bytecode.rs
Lines 620 to 625 in 3a6fda4
So, I have converted this PR to address #6156 (review)
cc: @youknowone @coolreader18
Followup for #6156 (comment)
Summary by CodeRabbit
New Features
Bug Fixes
Chores