FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

add arrayiter.__reduce__ by maong0927 · Pull Request #3868 · RustPython/RustPython · GitHub

  1. RustPython
  2. RustPython

Repository navigation

add arrayiter.__reduce__ #3868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 8 commits into from
Jul 18, 2022
Conversation 21 Commits 8 Checks 0 Files changed
Merged

add arrayiter.__reduce__ #3868

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0f24b18
Add arrayiter __reduce__
maong0927 Jul 11, 2022
ab284c8
update test_iterator_pickle
maong0927 Jul 11, 2022
821f3e7
fix position value to struct value
maong0927 Jul 12, 2022
1a526af
Fix: Modify iter next to be implemented only internal
Yaminyam Jul 16, 2022
50aebef
Update stdlib/src/array.rs
Yaminyam Jul 16, 2022
2417d3e
Feat: impl `PyArrayIter.setstate`
Yaminyam Jul 16, 2022
cdd937c
Refactor: convert indent from tab to space
Yaminyam Jul 16, 2022
2bf61aa
Refactor: delete unnecessary clones
Yaminyam Jul 16, 2022
File filter

Filter by extension

Filter by extension .py  (1) .rs  (1) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
2 changes: 0 additions & 2 deletions Lib/test/test_array.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,6 @@ def test_pickle_for_empty_array(self):
self.assertEqual(a.x, b.x)
self.assertEqual(type(a), type(b))

# TODO: RUSTPYTHON
@unittest.expectedFailure
def test_iterator_pickle(self):
orig = array.array(self.typecode, self.example)
data = list(orig)
Expand Down
53 changes: 32 additions & 21 deletions stdlib/src/array.rs
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ mod array {
common::{
atomic::{self, AtomicUsize},
lock::{
PyMappedRwLockReadGuard, PyMappedRwLockWriteGuard, PyRwLock, PyRwLockReadGuard,
PyRwLockWriteGuard,
PyMappedRwLockReadGuard, PyMappedRwLockWriteGuard, PyMutex, PyRwLock,
PyRwLockReadGuard, PyRwLockWriteGuard,
},
str::wchar_t,
},
vm::{
builtins::{
PyByteArray, PyBytes, PyBytesRef, PyDictRef, PyFloat, PyInt, PyIntRef, PyList,
PyListRef, PyStr, PyStrRef, PyTupleRef, PyTypeRef,
PositionIterInternal, PyByteArray, PyBytes, PyBytesRef, PyDictRef, PyFloat, PyInt,
PyIntRef, PyList, PyListRef, PyStr, PyStrRef, PyTupleRef, PyTypeRef,
},
class_or_notimplemented,
convert::{ToPyObject, ToPyResult, TryFromObject},
Expand Down Expand Up @@ -1258,9 +1258,8 @@ mod array {

impl Iterable for PyArray {
fn iter(zelf: PyRef<Self>, vm: &VirtualMachine) -> PyResult {
Ok(PyArrayIterator {
position: AtomicUsize::new(0),
array: zelf,
Ok(PyArrayIter {
internal: PyMutex::new(PositionIterInternal::new(zelf, 0)),
}
.into_pyobject(vm))
}
Expand All @@ -1278,24 +1277,36 @@ mod array {
#[pyattr]
#[pyclass(name = "arrayiterator")]
#[derive(Debug, PyPayload)]
pub struct PyArrayIterator {
position: AtomicUsize,
array: PyArrayRef,
pub struct PyArrayIter {
internal: PyMutex<PositionIterInternal<PyArrayRef>>,

youknowone Jul 12, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

do you really need both position and internal? Though I didn't look in details, those fields look like duplication.

Yaminyam Jul 12, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Is it because internal contains position because it is PositionIterInternal as its name suggests?

youknowone Jul 13, 2022 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Check the PositionIterInternal type. it contains position field. By this change, the code have duplicated position fields but most of code uses position and new code uses internal. I don't think it can correctly work

Snowapril Jul 15, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

With comparison with other types' iterator, we really don't need position and array field because PyMutex<PositionIterInternal<PyArrayRef>> will do their roles. Because of __next__ and __setstate__, we need to modify position and array fields. It leads to the need for PyMutex.

Snowapril Jul 15, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

If we replace position and array fields with PyMutex<PositionIterInternal<PyArrayRef>>, we can implement other methods as usual we did (__next__ and __reduce__)

youknowone Jul 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Does arrayiterator require closed field? I thought it only need position.
closed is useful when it include another iterator in it.
}

#[pyimpl(with(IterNext))]
impl PyArrayIterator {}
#[pyimpl(with(IterNext), flags(HAS_DICT))]
impl PyArrayIter {
#[pymethod(magic)]
fn setstate(&self, state: PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
self.internal
.lock()
.set_state(state, |obj, pos| pos.min(obj.len()), vm)
}

impl IterNextIterable for PyArrayIterator {}
impl IterNext for PyArrayIterator {
#[pymethod(magic)]
fn reduce(&self, vm: &VirtualMachine) -> PyTupleRef {
self.internal
.lock()
.builtins_iter_reduce(|x| x.clone().into(), vm)
}
}

impl IterNextIterable for PyArrayIter {}
impl IterNext for PyArrayIter {
fn next(zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<PyIterReturn> {
let pos = zelf.position.fetch_add(1, atomic::Ordering::SeqCst);
let r = if let Some(item) = zelf.array.read().get(pos, vm) {
PyIterReturn::Return(item?)
} else {
PyIterReturn::StopIteration(None)
};
Ok(r)
zelf.internal.lock().next(|array, pos| {
Ok(match array.read().get(pos, vm) {
Some(item) => PyIterReturn::Return(item?),
None => PyIterReturn::StopIteration(None),
})
})
}
}

Expand Down
Back | FazBrowse Home | New Git URL