FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-45040: Simplify `sqlite3` transaction control functions by erlend-aasland · Pull Request #28019 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

bpo-45040: Simplify sqlite3 transaction control functions #28019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pablogsal merged 1 commit into from
Sep 19, 2021
+28 −63
Conversation 7 Commits 1 Checks 0 Files changed 2
Merged

bpo-45040: Simplify sqlite3 transaction control functions #28019

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension .c  (2) All 1 file type selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
68 changes: 21 additions & 47 deletions Modules/_sqlite/connection.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -396,43 +396,29 @@ static PyObject *
pysqlite_connection_commit_impl(pysqlite_Connection *self)
/*[clinic end generated code: output=3da45579e89407f2 input=39c12c04dda276a8]*/
{
int rc;
sqlite3_stmt* statement;

if (!pysqlite_check_thread(self) || !pysqlite_check_connection(self)) {
return NULL;
}

if (!sqlite3_get_autocommit(self->db)) {
int rc;

Py_BEGIN_ALLOW_THREADS
sqlite3_stmt *statement;
rc = sqlite3_prepare_v2(self->db, "COMMIT", 7, &statement, NULL);
Py_END_ALLOW_THREADS
if (rc != SQLITE_OK) {
_pysqlite_seterror(self->state, self->db);
goto error;
if (rc == SQLITE_OK) {
(void)sqlite3_step(statement);
rc = sqlite3_finalize(statement);
}

rc = pysqlite_step(statement);
if (rc != SQLITE_DONE) {
_pysqlite_seterror(self->state, self->db);
}

Py_BEGIN_ALLOW_THREADS
rc = sqlite3_finalize(statement);

pablogsal Aug 30, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Seems that sqlite3_finalize was protected with the GIL previously and the check for PyErr_Occurred makes me thing that some callback or similar could end calling Python code. Could you confirm if this is indeed safe?

erlend-aasland Aug 30, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hm, sqlite3_finalize was called outside of the GIL previously, and it still is. Previously, we would release/acquire GIL before/after every SQLite API call; now we release, do all SQLite API calls, then acquire.

Previously

  1. allow threads (unlock GIL), prepare statement, disallow threads (lock GIL)
  2. raise exception and bail on error
  3. unlock GIL, execute SQL statement (sqlite3_step), lock GIL
  4. raise exception and "fall through" on error
  5. unlock GIL, finalise statement (release resources, etc.), lock GIL
  6. raise exception, unless sqlite3_step already did, on error => this is the extra PyErr_Occurred

Since sqlite3_finalize will pass on the error set by sqlite3_step, there is no need to check for errors after stepping. We only need to do it after finalising the statement.

Now

  1. unlock GIL, create, execute, and finalise SQL statement, lock GIL
  2. raise exception and bail on error

pablogsal Aug 30, 2021 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can sqlite3_finalize raise Python errors on its own (via cleanups, callbacks or otherwise)?

erlend-aasland Aug 30, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Not the way we currently use the SQLite API. If we'd added SQLite profile support, sqlite3_finalize (and other API's) would invoke our _trace_callback() C callback with argument type set to SQLITE_TRACE_PROFILE. We could add an assert(type != SQLITE_TRACE_PROFILE) and a comment about this to _trace_callback().
Py_END_ALLOW_THREADS
if (rc != SQLITE_OK && !PyErr_Occurred()) {
_pysqlite_seterror(self->state, self->db);
}

if (rc != SQLITE_OK) {
(void)_pysqlite_seterror(self->state, self->db);
return NULL;
}
}

error:
if (PyErr_Occurred()) {
return NULL;
} else {
Py_RETURN_NONE;
}
Py_RETURN_NONE;
}

/*[clinic input]
Expand All @@ -445,44 +431,32 @@ static PyObject *
pysqlite_connection_rollback_impl(pysqlite_Connection *self)
/*[clinic end generated code: output=b66fa0d43e7ef305 input=12d4e8d068942830]*/
{
int rc;
sqlite3_stmt* statement;

if (!pysqlite_check_thread(self) || !pysqlite_check_connection(self)) {
return NULL;
}

if (!sqlite3_get_autocommit(self->db)) {
pysqlite_do_all_statements(self);

int rc;

Py_BEGIN_ALLOW_THREADS
sqlite3_stmt *statement;
rc = sqlite3_prepare_v2(self->db, "ROLLBACK", 9, &statement, NULL);
Py_END_ALLOW_THREADS
if (rc != SQLITE_OK) {
_pysqlite_seterror(self->state, self->db);
goto error;
}

rc = pysqlite_step(statement);
if (rc != SQLITE_DONE) {
_pysqlite_seterror(self->state, self->db);
if (rc == SQLITE_OK) {
(void)sqlite3_step(statement);
rc = sqlite3_finalize(statement);
}

Py_BEGIN_ALLOW_THREADS
rc = sqlite3_finalize(statement);
Py_END_ALLOW_THREADS
if (rc != SQLITE_OK && !PyErr_Occurred()) {
_pysqlite_seterror(self->state, self->db);

if (rc != SQLITE_OK) {
(void)_pysqlite_seterror(self->state, self->db);
return NULL;
}

}

error:
if (PyErr_Occurred()) {
return NULL;
} else {
Py_RETURN_NONE;
}
Py_RETURN_NONE;
}

static int
Expand Down
23 changes: 7 additions & 16 deletions Modules/_sqlite/cursor.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -431,31 +431,22 @@ static int
begin_transaction(pysqlite_Connection *self)
{
int rc;
sqlite3_stmt *statement;

Py_BEGIN_ALLOW_THREADS
sqlite3_stmt *statement;
rc = sqlite3_prepare_v2(self->db, self->begin_statement, -1, &statement,
NULL);
Py_END_ALLOW_THREADS

if (rc != SQLITE_OK) {
_pysqlite_seterror(self->state, self->db);
goto error;
if (rc == SQLITE_OK) {
(void)sqlite3_step(statement);
rc = sqlite3_finalize(statement);
}

Py_BEGIN_ALLOW_THREADS
sqlite3_step(statement);
rc = sqlite3_finalize(statement);
Py_END_ALLOW_THREADS

if (rc != SQLITE_OK && !PyErr_Occurred()) {
_pysqlite_seterror(self->state, self->db);
}

error:
if (PyErr_Occurred()) {
if (rc != SQLITE_OK) {
(void)_pysqlite_seterror(self->state, self->db);
return -1;
}

return 0;
}

Expand Down
Back | FazBrowse Home | New Git URL