12.6. `sqlite3` SQLite DB-API 2.0

SQLite C SQL SQLite SQLite PostgreSQL Oracle

sqlite3 Gerhard Hring PEP 249 DB-API 2.0 SQL

Connection example.db :

import sqlite3
conn = sqlite3.connect('example.db')

:memory: RAM

Connection Cursor execute() SQL :

c = conn.cursor()

# Create table
c.execute('''CREATE TABLE stocks
             (date text, trans text, symbol text, qty real, price real)''')

# Insert a row of data
c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")

# Save (commit) the changes
conn.commit()

# We can also close the connection if we are done with it.
# Just be sure any changes have been committed or they will be lost.
conn.close()

:

import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()

SQL Python Python SQL (https://xkcd.com/327/ )

DB-API ? execute() 2( %s :1 ) :

# Never do this -- insecure!
symbol = 'RHAT'
c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)

# Do this instead
t = ('RHAT',)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(c.fetchone())

# Larger example that inserts many records at a time
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
             ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
             ('2006-04-06', 'SELL', 'IBM', 500, 53.00),
            ]
c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)

SELECT 3 (iterator) fetchone() fetchall() 3

:

>>> for row in c.execute('SELECT * FROM stocks ORDER BY price'):
        print(row)

('2006-01-05', 'BUY', 'RHAT', 100, 35.14)
('2006-03-28', 'BUY', 'IBM', 1000, 45.0)
('2006-04-06', 'SELL', 'IBM', 500, 53.0)
('2006-04-05', 'BUY', 'MSFT', 1000, 72.0)

https://github.com/ghaering/pysqlite: pysqlite sqlite3 pysqlite
https://www.sqlite.org: SQLite SQL
http://www.w3schools.com/sql/: SQL
PEP 249 - Database API Specification 2.0: Marc-Andre Lemburg PEP

12.6.1.

sqlite3.version: SQLite

sqlite3.version_info: SQLite

sqlite3.sqlite_version: SQLite

sqlite3.sqlite_version_info: SQLite

sqlite3.PARSE_DECLTYPES

connect() detect_types

sqlite3 "integer primary key" "integer" "number(10)" "number"

sqlite3.PARSE_COLNAMES

connect() detect_types

SQLite [mytype] 'mytype' 'mytype' Cursor.description 'as "x [datetime]"' SQL "x"

sqlite3.connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri])

database SQLite ":memory:" RAM

SQLite timeout 5.0 (5)

isolation_level Connection isolation_level

SQLite TEXTINTEGERREALBLOB NULL detect_types register_converter()

detect_types 0 () PARSE_DECLTYPES PARSE_COLNAMES

By default, check_same_thread is True and only the creating thread may use the connection. If set False, the returned connection may be shared across multiple threads. When using multiple threads with the same connection writing operations should be serialized by the user to avoid data corruption.

sqlite3 connect Connection Connection factory connect()

SQLite Python

sqlite3 SQL cached_statements SQL 100

uri database URI :

db = sqlite3.connect('file:path/to/database?mode=ro', uri=True)

More information about this feature, including a list of recognized options, can be found in the SQLite URI documentation.

3.4 : uri

sqlite3.register_converter(typename, callable): Registers a callable to convert a bytestring from the database into a custom Python type. The callable will be invoked for all database values that are of the type typename. Confer the parameter detect_types of the connect() function for how the type detection works. Note that the case of typename and the name of the type in your query must match!

sqlite3.register_adapter(type, callable): Python type SQLite (callable) callable Python int, float, str bytes

sqlite3.complete_statement(sql)

sql SQL True SQL

SQLite :

# A minimal SQLite shell for experiments

import sqlite3

con = sqlite3.connect(":memory:")
con.isolation_level = None
cur = con.cursor()

buffer = ""

print("Enter your SQL commands to execute in sqlite3.")
print("Enter a blank line to exit.")

while True:
    line = input()
    if line == "":
        break
    buffer += line
    if sqlite3.complete_statement(buffer):
        try:
            buffer = buffer.strip()
            cur.execute(buffer)

            if buffer.lstrip().upper().startswith("SELECT"):
                print(cur.fetchall())
        except sqlite3.Error as e:
            print("An error occurred:", e.args[0])
        buffer = ""

con.close()

sqlite3.enable_callback_tracebacks(flag): flag True sys.stderr False

12.6.2. Connection

class sqlite3.Connection

SQLite :

isolation_level: None "DEFERRED", "IMMEDIATE", "EXLUSIVE"

in_transaction: () True False

3.2 .

cursor(factory=Cursor): cursor factory 1 Cursor

commit(): commit()

rollback(): commit()

close(): commit() commit()

execute(sql[, parameters]): This is a nonstandard shortcut that creates a cursor object by calling the cursor() method, calls the cursors execute() method with the parameters given, and returns the cursor.

executemany(sql[, parameters]): This is a nonstandard shortcut that creates a cursor object by calling the cursor() method, calls the cursors executemany() method with the parameters given, and returns the cursor.

executescript(sql_script): This is a nonstandard shortcut that creates a cursor object by calling the cursor() method, calls the cursors executescript() method with the given sql_script, and returns the cursor.

create_function(name, num_params, func)

Creates a user-defined function that you can later use from within SQL statements under the function name name. num_params is the number of parameters the function accepts (if num_params is -1, the function may take any number of arguments), and func is a Python callable that is called as the SQL function.

The function can return any of the types supported by SQLite: bytes, str, int, float and None.

:

import sqlite3
import hashlib

def md5sum(t):
    return hashlib.md5(t).hexdigest()

con = sqlite3.connect(":memory:")
con.create_function("md5", 1, md5sum)
cur = con.cursor()
cur.execute("select md5(?)", (b"foo",))
print(cur.fetchone()[0])

create_aggregate(name, num_params, aggregate_class)

The aggregate class must implement a step method, which accepts the number of parameters num_params (if num_params is -1, the function may take any number of arguments), and a finalize method which will return the final result of the aggregate.

The finalize method can return any of the types supported by SQLite: bytes, str, int, float and None.

:

import sqlite3

class MySum:
    def __init__(self):
        self.count = 0

    def step(self, value):
        self.count += value

    def finalize(self):
        return self.count

con = sqlite3.connect(":memory:")
con.create_aggregate("mysum", 1, MySum)
cur = con.cursor()
cur.execute("create table test(i)")
cur.execute("insert into test(i) values (1)")
cur.execute("insert into test(i) values (2)")
cur.execute("select mysum(i) from test")
print(cur.fetchone()[0])

create_collation(name, callable)

name callable -1 0 1 (SQL ORDER BY) SQL

Python UTF-8

:

import sqlite3

def collate_reverse(string1, string2):
    if string1 == string2:
        return 0
    elif string1 < string2:
        return 1
    else:
        return -1

con = sqlite3.connect(":memory:")
con.create_collation("reverse", collate_reverse)

cur = con.cursor()
cur.execute("create table test(x)")
cur.executemany("insert into test(x) values (?)", [("a",), ("b",)])
cur.execute("select x from test order by x collate reverse")
for row in cur:
    print(row)
con.close()

callable None create_collation :

con.create_collation("reverse", None)

interrupt()

set_authorizer(authorizer_callback)

SQLITE_OK SQL SQLITE_DENY NULL SQLITE_IGNORE sqlite3

None ("main", "temp", etc.) SQL None

SQLite sqlite3

set_progress_handler(handler, n)

SQLite n GUI SQLite

progress handler handler None

Returning a non-zero value from the handler function will terminate the currently executing query and cause it to raise an OperationalError exception.

set_trace_callback(trace_callback)

SQL SQLite trace_callback

SQL () Cursor.execute() SQL Python

trace_callback None

3.3 .

enable_load_extension(enabled)

SQLite SQLite SQLite 1 SQLite

SQLite [1]

3.2 .

import sqlite3

con = sqlite3.connect(":memory:")

# enable extension loading
con.enable_load_extension(True)

# Load the fulltext search extension
con.execute("select load_extension('./fts3.so')")

# alternatively you can load the extension using an API call:
# con.load_extension("./fts3.so")

# disable extension loading again
con.enable_load_extension(False)

# example from SQLite wiki
con.execute("create virtual table recipe using fts3(name, ingredients)")
con.executescript("""
    insert into recipe (name, ingredients) values ('broccoli stew', 'broccoli peppers cheese tomatoes');
    insert into recipe (name, ingredients) values ('pumpkin stew', 'pumpkin onions garlic celery');
    insert into recipe (name, ingredients) values ('broccoli pie', 'broccoli cheese onions flour');
    insert into recipe (name, ingredients) values ('pumpkin pie', 'pumpkin sugar flour butter');
    """)
for row in con.execute("select rowid, name, ingredients from recipe where name match 'pie'"):
    print(row)

load_extension(path)

SQLite enable_load_extension()

SQLite [1]

3.2 .

row_factory

:

import sqlite3

def dict_factory(cursor, row):
    d = {}
    for idx, col in enumerate(cursor.description):
        d[col[0]] = row[idx]
    return d

con = sqlite3.connect(":memory:")
con.row_factory = dict_factory
cur = con.cursor()
cur.execute("select 1 as a")
print(cur.fetchone()["a"])

row_factory sqlite3.Row Row db

text_factory

TEXT str sqlite3 TEXT Unicode bytes

:

import sqlite3

con = sqlite3.connect(":memory:")
cur = con.cursor()

AUSTRIA = "\xd6sterreich"

# by default, rows are returned as Unicode
cur.execute("select ?", (AUSTRIA,))
row = cur.fetchone()
assert row[0] == AUSTRIA

# but we can make sqlite3 always return bytestrings ...
con.text_factory = bytes
cur.execute("select ?", (AUSTRIA,))
row = cur.fetchone()
assert type(row[0]) is bytes
# the bytestrings will be encoded in UTF-8, unless you stored garbage in the
# database ...
assert row[0] == AUSTRIA.encode("utf-8")

# we can also implement a custom text_factory ...
# here we implement one that appends "foo" to all strings
con.text_factory = lambda x: x.decode("utf-8") + "foo"
cur.execute("select ?", ("bar",))
row = cur.fetchone()
assert row[0] == "barfoo"

total_changes

iterdump()

SQL test sqlite3 .dump

:

# Convert file existing_db.db to SQL dump file dump.sql
import sqlite3

con = sqlite3.connect('existing_db.db')
with open('dump.sql', 'w') as f:
    for line in con.iterdump():
        f.write('%s\n' % line)

12.6.3.

class sqlite3.Cursor

Cursor

execute(sql[, parameters])

SQL SQL ( SQL (placeholder) ) sqlite3 2(qmark )(named )

:

import sqlite3

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("create table people (name_last, age)")

who = "Yeltsin"
age = 72

# This is the qmark style:
cur.execute("insert into people values (?, ?)", (who, age))

# And this is the named style:
cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age})

print(cur.fetchone())

execute() will only execute a single SQL statement. If you try to execute more than one statement with it, it will raise a Warning. Use executescript() if you want to execute multiple SQL statements with one call.

executemany(sql, seq_of_parameters)

Executes an SQL command against all parameter sequences or mappings found in the sequence seq_of_parameters. The sqlite3 module also allows using an iterator yielding parameters instead of a sequence.

import sqlite3

class IterChars:
    def __init__(self):
        self.count = ord('a')

    def __iter__(self):
        return self

    def __next__(self):
        if self.count > ord('z'):
            raise StopIteration
        self.count += 1
        return (chr(self.count - 1),) # this is a 1-tuple

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("create table characters(c)")

theIter = IterChars()
cur.executemany("insert into characters(c) values (?)", theIter)

cur.execute("select c from characters")
print(cur.fetchall())

(generator) :

import sqlite3
import string

def char_generator():
    for c in string.ascii_lowercase:
        yield (c,)

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("create table characters(c)")

cur.executemany("insert into characters(c) values (?)", char_generator())

cur.execute("select c from characters")
print(cur.fetchall())

executescript(sql_script)

SQL COMMIT SQL

sql_script can be an instance of str.

:

import sqlite3

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.executescript("""
    create table person(
        firstname,
        lastname,
        age
    );

    create table book(
        title,
        author,
        published
    );

    insert into book(title, author, published)
    values (
        'Dirk Gently''s Holistic Detective Agency',
        'Douglas Adams',
        1987
    );
    """)

fetchone(): row 1 None

fetchmany(size=cursor.arraysize)

row

row size cursor arraysize size row fetch row row

size arraysize size fetchmany()

fetchall(): () row cursor arraysize row

close()

Close the cursor now (rather than whenever __del__ is called).

The cursor will be unusable from this point forward; a ProgrammingError exception will be raised if any operation is attempted with the cursor.

rowcount

sqlite3 Cursor /

executemany() rowcount

Python DB API rowcount executeXX() rowcount -1 SELECT

SQLite 3.6.5 DELETE FROM table rowcount 0

lastrowid

This read-only attribute provides the rowid of the last modified row. It is only set if you issued an INSERT or a REPLACE statement using the execute() method. For operations other than INSERT or REPLACE or when executemany() is called, lastrowid is set to None.

If the INSERT or REPLACE statement failed to insert the previous successful rowid is returned.

3.6 : Added support for the REPLACE statement.

arraysize: Read/write attribute that controls the number of rows returned by fetchmany(). The default value is 1 which means a single row would be fetched per call.

description

Python DB API 76 None

SELECT row 1

connection

Cursor SQLite Connection con.cursor() Cursor con connection :

>>> con = sqlite3.connect(":memory:")
>>> cur = con.cursor()
>>> cur.connection == con
True

12.6.4. Row

class sqlite3.Row

Row Connection row_factory row

, , repr(), , len()

2 Row

keys(): Cursor.description

3.5 :

Row:

conn = sqlite3.connect(":memory:")
c = conn.cursor()
c.execute('''create table stocks
(date text, trans text, symbol text,
 qty real, price real)''')
c.execute("""insert into stocks
          values ('2006-01-05','BUY','RHAT',100,35.14)""")
conn.commit()
c.close()

Row :

>>> conn.row_factory = sqlite3.Row
>>> c = conn.cursor()
>>> c.execute('select * from stocks')
<sqlite3.Cursor object at 0x7f4e7dd8fa80>
>>> r = c.fetchone()
>>> type(r)
<class 'sqlite3.Row'>
>>> tuple(r)
('2006-01-05', 'BUY', 'RHAT', 100.0, 35.14)
>>> len(r)
5
>>> r[2]
'RHAT'
>>> r.keys()
['date', 'trans', 'symbol', 'qty', 'price']
>>> r['qty']
100.0
>>> for member in r:
...     print(member)
...
2006-01-05
BUY
RHAT
100.0
35.14

12.6.5.

exception sqlite3.Warning: A subclass of Exception.

exception sqlite3.Error: Exception

exception sqlite3.DatabaseError: Exception raised for errors that are related to the database.

exception sqlite3.IntegrityError: Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. It is a subclass of DatabaseError.

exception sqlite3.ProgrammingError: Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. It is a subclass of DatabaseError.

12.6.6. SQLite Python

12.6.6.1.

SQLite : NULL, INTEGER, REAL, TEXT, BLOB

Python SQLite :

Python	SQLite
`None`	`NULL`
`int`	`INTEGER`
`float`	`REAL`
`str`	`TEXT`
`bytes`	`BLOB`

SQLite Python :

SQLite	Python
`NULL`	`None`
`INTEGER`	`int`
`REAL`	`float`
`TEXT`	`text_factory` `str`
`BLOB`	`bytes`

sqlite3 (adaptation) Python SQLite (converter) sqlite3 SQLite Python

12.6.6.2. Python SQLite

SQLite Python SQLite sqlite3 NoneType, int, float, str, bytes

sqlite3 Python

12.6.6.2.1.

:

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

SQLite __conform__(self, protocol) protocol PrepareProtocol

import sqlite3

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __conform__(self, protocol):
        if protocol is sqlite3.PrepareProtocol:
            return "%f;%f" % (self.x, self.y)

con = sqlite3.connect(":memory:")
cur = con.cursor()

p = Point(4.0, -3.2)
cur.execute("select ?", (p,))
print(cur.fetchone()[0])

12.6.6.2.2.

register_adapter()

import sqlite3

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

def adapt_point(point):
    return "%f;%f" % (point.x, point.y)

sqlite3.register_adapter(Point, adapt_point)

con = sqlite3.connect(":memory:")
cur = con.cursor()

p = Point(4.0, -3.2)
cur.execute("select ?", (p,))
print(cur.fetchone()[0])

sqlite3 Python datetime.date datetime.datetime datetime.datetime ISO Unix

import sqlite3
import datetime
import time

def adapt_datetime(ts):
    return time.mktime(ts.timetuple())

sqlite3.register_adapter(datetime.datetime, adapt_datetime)

con = sqlite3.connect(":memory:")
cur = con.cursor()

now = datetime.datetime.now()
cur.execute("select ?", (now,))
print(cur.fetchone()[0])

12.6.6.3. SQLite Python

Python SQLite Python SQLite Python (roundtrip)

(converter)

Point x, y SQLite

Point

SQLite bytes

def convert_point(s):
    x, y = map(float, s.split(b";"))
    return Point(x, y)

sqlite3 :

PARSE_DECLTYPES PARSE_COLNAMES

import sqlite3

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __repr__(self):
        return "(%f;%f)" % (self.x, self.y)

def adapt_point(point):
    return ("%f;%f" % (point.x, point.y)).encode('ascii')

def convert_point(s):
    x, y = list(map(float, s.split(b";")))
    return Point(x, y)

# Register the adapter
sqlite3.register_adapter(Point, adapt_point)

# Register the converter
sqlite3.register_converter("point", convert_point)

p = Point(4.0, -3.2)

#########################
# 1) Using declared types
con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES)
cur = con.cursor()
cur.execute("create table test(p point)")

cur.execute("insert into test(p) values (?)", (p,))
cur.execute("select p from test")
print("with declared types:", cur.fetchone()[0])
cur.close()
con.close()

#######################
# 1) Using column names
con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_COLNAMES)
cur = con.cursor()
cur.execute("create table test(p)")

cur.execute("insert into test(p) values (?)", (p,))
cur.execute('select p as "p [point]" from test')
print("with column names:", cur.fetchone()[0])
cur.close()
con.close()

12.6.6.4.

datetime date datetime ISO / ISO SQLite

datetime.date "date" datetime.datetime "timestamp"

Python / SQLite date/time

import sqlite3
import datetime

con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
cur = con.cursor()
cur.execute("create table test(d date, ts timestamp)")

today = datetime.date.today()
now = datetime.datetime.now()

cur.execute("insert into test(d, ts) values (?, ?)", (today, now))
cur.execute("select d, ts from test")
row = cur.fetchone()
print(today, "=>", row[0], type(row[0]))
print(now, "=>", row[1], type(row[1]))

cur.execute('select current_date as "d [date]", current_timestamp as "ts [timestamp]"')
row = cur.fetchone()
print("current_date", row[0], type(row[0]))
print("current_timestamp", row[1], type(row[1]))

SQLite 6

12.6.7.

By default, the sqlite3 module opens transactions implicitly before a Data Modification Language (DML) statement (i.e. INSERT/UPDATE/DELETE/REPLACE).

sqlite3 BEGIN () connect() isolation_level isolation_level

isolation_level None

BEGIN SQLite "DEFERRED", "IMMEDIATE" "EXCLUSIVE"

The current transaction state is exposed through the Connection.in_transaction attribute of the connection object.

3.6 : sqlite3 used to implicitly commit an open transaction before DDL statements. This is no longer the case.

12.6.8. `sqlite3`

12.6.8.1.

Connection execute(), executemany(), executescript() () Cursor Cursor SELECT Connection

import sqlite3

persons = [
    ("Hugo", "Boss"),
    ("Calvin", "Klein")
    ]

con = sqlite3.connect(":memory:")

# Create the table
con.execute("create table person(firstname, lastname)")

# Fill the table
con.executemany("insert into person(firstname, lastname) values (?, ?)", persons)

# Print the table contents
for row in con.execute("select firstname, lastname from person"):
    print(row)

print("I just deleted", con.execute("delete from person").rowcount, "rows")

12.6.8.2.

sqlite3 sqlite3.Row

():

import sqlite3

con = sqlite3.connect(":memory:")
con.row_factory = sqlite3.Row

cur = con.cursor()
cur.execute("select 'John' as name, 42 as age")
for row in cur:
    assert row[0] == row["name"]
    assert row["name"] == row["nAmE"]
    assert row[1] == row["age"]
    assert row[1] == row["AgE"]

12.6.8.3.

Connection :

import sqlite3

con = sqlite3.connect(":memory:")
con.execute("create table person (id integer primary key, firstname varchar unique)")

# Successful, con.commit() is called automatically afterwards
with con:
    con.execute("insert into person(firstname) values (?)", ("Joe",))

# con.rollback() is called after the with block finishes with an exception, the
# exception is still raised and must be caught
try:
    with con:
        con.execute("insert into person(firstname) values (?)", ("Joe",))
except sqlite3.IntegrityError:
    print("couldn't add Joe twice")

12.6.9.

12.6.9.1.