| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Query Django model trees using adjacency lists and recursive common table expressions. Supports PostgreSQL, sqlite3 (3.8.3 or higher) and MariaDB (10.2.2 or higher) and MySQL (8.0 or higher, if running without ONLY_FULL_GROUP_BY).
Supports Django 3.2 or better, Python 3.8 or better. See the GitHub actions build for more details.
Here's a blog post offering some additional insight (hopefully) into the reasons for django-tree-queries' existence.
The following two examples both extend the TreeNode which offers a few agreeable utilities and a model validation method that prevents loops in the tree structure. The common table expression could be hardened against such loops but this would involve a performance hit which we don't want -- this is a documented limitation (non-goal) of the library after all.
from tree_queries.models import TreeNode
class Node(TreeNode):
name = models.CharField(max_length=100)Nodes with the same parent may be ordered among themselves. The default is to order siblings by their primary key but that's not always very useful.
Manual position management:
from tree_queries.models import TreeNode
class Node(TreeNode):
name = models.CharField(max_length=100)
position = models.PositiveIntegerField(default=0)
class Meta:
ordering = ["position"]Automatic position management:
For automatic position management, use OrderableTreeNode which automatically assigns sequential position values to new nodes:
from tree_queries.models import OrderableTreeNode
class Category(OrderableTreeNode):
name = models.CharField(max_length=100)
# position field and ordering are inherited from OrderableTreeNodeWhen creating new nodes without an explicit position, OrderableTreeNode automatically assigns a position value 10 units higher than the maximum position among siblings. The increment of 10 (rather than 1) makes it explicit that the position values themselves have no inherent meaning - they are purely for relative ordering, not a sibling counter or index.
If you need to customize the Meta class (e.g., to add verbose names or additional ordering fields), inherit from OrderableTreeNode.Meta:
from tree_queries.models import OrderableTreeNode
class Category(OrderableTreeNode):
name = models.CharField(max_length=100)
class Meta(OrderableTreeNode.Meta):
verbose_name = "category"
verbose_name_plural = "categories"
# ordering = ["position"] is inherited from OrderableTreeNode.Meta# Create nodes - positions are assigned automatically
root = Category.objects.create(name="Root") # position=10
child1 = Category.objects.create(name="Child 1", parent=root) # position=10
child2 = Category.objects.create(name="Child 2", parent=root) # position=20
child3 = Category.objects.create(name="Child 3", parent=root) # position=30
# Manual reordering is still possible
child3.position = 15 # Move between child1 and child2
child3.save()This approach is identical to the pattern used in feincms3's AbstractPage.
from tree_queries.models import TreeNode
from tree_queries.query import TreeQuerySet
class NodeQuerySet(TreeQuerySet):
def active(self):
return self.filter(is_active=True)
class Node(TreeNode):
is_active = models.BooleanField(default=True)
objects = NodeQuerySet.as_manager()All examples assume the Node class from above.
# Basic usage, disregards the tree structure completely.
nodes = Node.objects.all()
# Fetch nodes in depth-first search order. All nodes will have the
# tree_path, tree_ordering and tree_depth attributes.
nodes = Node.objects.with_tree_fields()
# Fetch any node.
node = Node.objects.order_by("?").first()
# Fetch direct children and include tree fields. (The parent ForeignKey
# specifies related_name="children")
children = node.children.with_tree_fields()
# Fetch all ancestors starting from the root.
ancestors = node.ancestors()
# Fetch all ancestors including self, starting from the root.
ancestors_including_self = node.ancestors(include_self=True)
# Fetch all ancestors starting with the node itself.
ancestry = node.ancestors(include_self=True).reverse()
# Fetch all descendants in depth-first search order, including self.
descendants = node.descendants(include_self=True)
# Temporarily override the ordering by siblings.
nodes = Node.objects.order_siblings_by("id")
# Revert to a queryset without tree fields (improves performance).
nodes = Node.objects.with_tree_fields().without_tree_fields()When using with_tree_fields(), each node gets three additional attributes:
The key difference between tree_path and tree_ordering:
# Example tree structure:
# Root (pk=1, order=0)
# ├── Child A (pk=2, order=10)
# │ └── Grandchild (pk=4, order=5)
# └── Child B (pk=3, order=20)
# For the Grandchild node:
grandchild = Node.objects.with_tree_fields().get(pk=4)
# tree_path shows the route through primary keys: Root -> Child A -> Grandchild
assert grandchild.tree_path == [1, 2, 4] # [root.pk, child_a.pk, grandchild.pk]
# tree_ordering shows ordering values at each level: Root's order, Child A's order, Grandchild's order
assert grandchild.tree_ordering == [0, 10, 5] # [root.order, child_a.order, grandchild.order]Important note: When not using an explicit ordering (like a position field), siblings are ordered by their primary key by default. This means tree_path and tree_ordering will contain the same values. While this may be fine for your use case consider adding an explicit ordering field:
class Node(TreeNode):
id = models.UUIDField(primary_key=True, default=uuid.uuid4)
name = models.CharField(max_length=100)
position = models.PositiveIntegerField(default=0)
class Meta:
ordering = ["position"]Tree fields (tree_depth, tree_path, tree_ordering) are only available on objects returned by queries that use with_tree_fields() or a manager configured with with_tree_fields=True. They are NOT available after Model.objects.create(), instance.save(), or instance.refresh_from_db().
Why? Tree fields are calculated by the recursive CTE at query time and are not stored in the database. They only exist as annotations on the queryset results.
# Tree fields are NOT available after creation
node = Node.objects.create(name="New Node", parent=root)
# node.tree_depth # AttributeError: 'Node' object has no attribute 'tree_depth'
# refresh_from_db() only updates database fields, not tree fields
node.refresh_from_db()
# node.tree_depth # Still AttributeError
# To get tree fields, re-query the object with with_tree_fields()
node = Node.objects.with_tree_fields().get(pk=node.pk)
print(node.tree_depth) # Now it works! e.g., 1
# Or use a manager with tree fields enabled by default
class Node(TreeNode):
name = models.CharField(max_length=100)
objects = TreeQuerySet.as_manager(with_tree_fields=True)
# Now tree fields are available on all queries automatically
node = Node.objects.get(pk=some_pk)
print(node.tree_depth) # Works!
# But still not after create/save/refresh_from_db
new_node = Node.objects.create(name="Another")
# new_node.tree_depth # Still AttributeError - need to re-queryCommon pattern when creating nodes:
# Create a new node
new_node = Node.objects.create(name="New Child", parent=parent_node)
# Re-query to get tree fields if you need them
new_node = Node.objects.with_tree_fields().get(pk=new_node.pk)
# Now you can access tree fields
print(f"Depth: {new_node.tree_depth}")
print(f"Path: {new_node.tree_path}")IMPORTANT: For large tables, always use tree_filter() or tree_exclude() to limit which nodes are processed by the recursive CTE. Without these filters, the database evaluates the entire table, which can be extremely slow.
# Get a specific tree from a forest by filtering on root category
product_tree = Node.objects.with_tree_fields().tree_filter(category="products")
# Get organizational chart for a specific department
engineering_tree = Node.objects.with_tree_fields().tree_filter(department="engineering")
# Exclude entire trees/sections you don't need
content_trees = Node.objects.with_tree_fields().tree_exclude(category="archived")
# Chain multiple tree filters for more specific trees
recent_products = (Node.objects.with_tree_fields()
.tree_filter(category="products")
.tree_filter(created_date__gte=datetime.date.today()))
# Get descendants within a filtered tree subset
product_descendants = (Node.objects.with_tree_fields()
.tree_filter(category="products")
.descendants(some_product_node))
# Filter by site/tenant in multi-tenant applications
site_content = Node.objects.with_tree_fields().tree_filter(site_id=request.site.id)Performance note: tree_filter() and tree_exclude() filter the base table before the recursive CTE processes relationships, dramatically improving performance for large datasets compared to using regular filter() after with_tree_fields(). Best used for selecting complete trees or tree sections rather than scattered nodes.
The tree queryset doesn't support all types of queries Django supports.
Updating all descendants directly isn't supported because the recursive CTE isn't added to the UPDATE query correctly. Use a subquery workaround instead:
# Doesn't work:
node.descendants().update(is_active=False)
# Use this workaround instead:
Node.objects.filter(pk__in=node.descendants()).update(is_active=False)Using select_related() works when querying from the tree model to fetch related objects. However, querying from a related model and trying to get tree fields on the tree model via select_related() is not supported.
# This works - tree model is the base, select_related fetches the category
nodes = Node.objects.with_tree_fields().select_related("category")
for node in nodes:
print(node.tree_depth, node.category.name)
# This doesn't work - ReferenceModel is the base, tree fields won't be present
# on the related Node objects
references = ReferenceModel.objects.select_related("tree_field")
for ref in references:
# ref.tree_field.tree_depth # AttributeError - tree fields not availableWorkaround: Query from the tree model and use prefetch_related() or Prefetch() to fetch the related objects:
from django.db.models import Prefetch
# Approach 1: Query from tree model, prefetch references
nodes = Node.objects.with_tree_fields().prefetch_related("referencemodel_set")
for node in nodes:
print(node.tree_depth)
for ref in node.referencemodel_set.all():
print(f" Reference: {ref.id}")
# Approach 2: Query references, then fetch tree nodes separately
references = ReferenceModel.objects.all()
tree_node_ids = [ref.tree_field_id for ref in references]
nodes_by_id = {
node.pk: node
for node in Node.objects.with_tree_fields().filter(pk__in=tree_node_ids)
}
for ref in references:
node = nodes_by_id[ref.tree_field_id]
print(f"Reference {ref.id}: tree depth = {node.tree_depth}")Set operations like union(), intersection(), and difference() are not currently supported with tree querysets due to how Django passes the elide_empty parameter. Attempting to use these operations will result in errors.
# This doesn't work:
qs1 = Node.objects.with_tree_fields().filter(name__startswith="A")
qs2 = Node.objects.with_tree_fields().filter(name__startswith="B")
combined = qs1.union(qs2) # Will raise an errorWorkaround: Use regular Django querysets without tree fields, then add tree fields after the set operation:
# Perform the set operation without tree fields
qs1 = Node.objects.filter(name__startswith="A")
qs2 = Node.objects.filter(name__startswith="B")
combined_ids = qs1.union(qs2).values_list("pk", flat=True)
# Then fetch with tree fields
result = Node.objects.with_tree_fields().filter(pk__in=combined_ids)For more details and discussion about adding support for these operations, see GitHub issue #55.
To limit the depth of the tree returned by a query, use .extra() with a WHERE clause on the tree_depth field:
# Get only nodes up to depth 2 (root is depth 0)
nodes = Node.objects.with_tree_fields().extra(
where=["__tree.tree_depth <= %s"],
params=[2],
)
# Get nodes within a depth range
nodes = Node.objects.with_tree_fields().extra(
where=["__tree.tree_depth BETWEEN %s AND %s"],
params=[1, 3], # Only depths 1, 2, and 3
)To enforce a maximum number of children per node, implement validation in your model's clean() method:
from django.core.exceptions import ValidationError
class Node(TreeNode):
def clean(self):
super().clean()
max_children = 10
# Check if this would exceed the limit
if self.parent_id:
sibling_count = (
Node.objects
.filter(parent_id=self.parent_id)
.exclude(pk=self.pk)
.count()
)
if sibling_count >= max_children:
raise ValidationError(
f"Parent already has {sibling_count} children "
f"(maximum {max_children})"
)If you need to validate or check depth without tree fields:
# Check depth by counting ancestors
depth = node.ancestors().count()
# Or check depth manually by traversing parent chain
def get_depth(node):
depth = 0
current = node
while current.parent_id:
depth += 1
current = current.parent
return depthThe recursive CTE used by django-tree-queries builds the tree top-down from root nodes to descendants. This makes certain operations, like aggregating values bottom-up (from descendants to ancestors) or using tree fields in complex subqueries, difficult or impossible.
# This doesn't work - can't use tree fields in subqueries with OuterRef
Branch.objects.annotate(
descendant_sum=Subquery(
Branch.objects.descendants(of=OuterRef('pk')).aggregate(Sum('amount'))
)
)Workaround for small trees: Load the tree into memory and compute in Python:
# For small trees, load root nodes and calculate in Python
roots = Branch.objects.filter(parent=None)
for root in roots:
descendants = root.descendants(include_self=True)
total = sum(node.amount for node in descendants)
# Use the total...For complex query requirements: If you need extensive CTE customization, subqueries with tree fields, or bottom-up aggregations, consider using django-cte which provides more control over recursive CTE construction, or evaluate whether a different tree storage approach (like django-treebeard with nested sets) better fits your needs.
For discussion of this limitation, see GitHub issue #75.
Nobody wants breadth-first search but if you still want it you can achieve it as follows:
nodes = Node.objects.with_tree_fields().extra(
order_by=["__tree.tree_depth", "__tree.tree_ordering"]
)If you only want nodes from the top two levels:
nodes = Node.objects.with_tree_fields().extra(
where=["__tree.tree_depth <= %s"],
params=[1],
)By default, .values() only returns model fields, not tree fields. If you need tree fields in a .values() call, you can access them using RawSQL:
from django.db.models.expressions import RawSQL
# Include tree fields in values() output
data = Node.objects.with_tree_fields().values(
"name",
tree_depth=RawSQL("tree_depth", ()),
tree_path=RawSQL("tree_path", ()),
)
# Returns: [{'name': 'root', 'tree_depth': 0, 'tree_path': [1]}, ...]Important caveats:
If you need tree field values for application logic, prefer accessing them as attributes on model instances rather than through .values():
# Preferred approach
nodes = Node.objects.with_tree_fields()
for node in nodes:
depth = node.tree_depth
path = node.tree_path # Consistent across all databases
# Only use RawSQL with values() when you need dictionary output
data = Node.objects.with_tree_fields().values(
"name",
tree_depth=RawSQL("tree_depth", ()),
)Use tree_fields() to aggregate values from ancestor nodes into arrays. This is useful for collecting paths, permissions, categories, or any field that should be inherited down the tree hierarchy.
# Aggregate names from all ancestors into an array
nodes = Node.objects.with_tree_fields().tree_fields(
tree_names="name",
)
# Each node now has a tree_names attribute: ['root', 'parent', 'current']
# Aggregate multiple fields
nodes = Node.objects.with_tree_fields().tree_fields(
tree_names="name",
tree_categories="category",
tree_permissions="permission_level",
)
# Build a full path string from ancestor names
nodes = Node.objects.with_tree_fields().tree_fields(tree_names="name")
for node in nodes:
full_path = " > ".join(node.tree_names) # "Root > Section > Subsection"
# Combine with tree filtering for better performance
active_nodes = (Node.objects.with_tree_fields()
.tree_filter(is_active=True)
.tree_fields(tree_names="name"))The aggregated fields contain values from all ancestors (root to current node) in hierarchical order, including the current node itself.
django-tree-queries ships a model field and some form fields which augment the default foreign key field and the choice fields with a version where the tree structure is visualized using dashes etc. Those fields are tree_queries.fields.TreeNodeForeignKey, tree_queries.forms.TreeNodeChoiceField, tree_queries.forms.TreeNodeMultipleChoiceField.
django-tree-queries includes template tags to help render tree structures in Django templates. These template tags are designed to work efficiently with tree querysets and respect queryset boundaries.
Add tree_queries to your INSTALLED_APPS setting:
INSTALLED_APPS = [
# ... other apps
'tree_queries',
]Then load the template tags in your template:
{% load tree_queries %}The tree_info filter provides detailed information about each node's position in the tree structure. It's useful when you need fine control over the tree rendering.
{% load tree_queries %}
<ul>
{% for node, structure in nodes|tree_info %}
{% if structure.new_level %}<ul><li>{% else %}</li><li>{% endif %}
{{ node.name }}
{% for level in structure.closed_levels %}</li></ul>{% endfor %}
{% endfor %}
</ul>The filter returns tuples of (node, structure_info) where structure_info contains:
Example showing ancestor information:
{% for node, structure in nodes|tree_info %}
{{ node.name }}
{% if structure.ancestors %}
(Path: {% for ancestor in structure.ancestors %}{{ ancestor }}{% if not forloop.last %} > {% endif %}{% endfor %})
{% endif %}
{% endfor %}The recursetree tag provides recursive rendering similar to django-mptt's recursetree tag, but optimized for django-tree-queries. It only considers nodes within the provided queryset and doesn't make additional database queries.
Basic usage:
{% load tree_queries %}
<ul>
{% recursetree nodes %}
<li>
{{ node.name }}
{% if children %}
<ul>{{ children }}</ul>
{% endif %}
</li>
{% endrecursetree %}
</ul>The recursetree tag provides these context variables within the template:
Using is_leaf for conditional rendering:
{% recursetree nodes %}
<div class="{% if is_leaf %}leaf-node{% else %}branch-node{% endif %}">
<span class="node-name">{{ node.name }}</span>
{% if children %}
<div class="children">{{ children }}</div>
{% elif is_leaf %}
<span class="leaf-indicator">🍃</span>
{% endif %}
</div>
{% endrecursetree %}Advanced example with depth information:
{% recursetree nodes %}
<div class="node depth-{{ node.tree_depth }}"
data-id="{{ node.pk }}"
data-has-children="{{ children|yesno:'true,false' }}">
<h{{ node.tree_depth|add:1 }}>{{ node.name }}</h{{ node.tree_depth|add:1 }}>
{% if children %}
<div class="node-children">{{ children }}</div>
{% endif %}
</div>
{% endrecursetree %}Both template tags respect queryset boundaries and work efficiently with filtered or limited querysets:
# Only nodes up to depth 2
limited_nodes = Node.objects.with_tree_fields().extra(
where=["__tree.tree_depth <= %s"], params=[2]
)
# Only specific branches
branch_nodes = Node.objects.descendants(some_node, include_self=True)When using these limited querysets:
Example with depth-limited queryset:
<!-- Template -->
{% recursetree limited_nodes %}
<li>
{{ node.name }}
{% if is_leaf %}
<small>(leaf in limited view)</small>
{% endif %}
{{ children }}
</li>
{% endrecursetree %}This is particularly useful for creating expandable tree interfaces or rendering only portions of large trees for performance.
django-tree-queries includes a TreeAdmin class for Django's admin interface that provides an intuitive tree management experience with drag-and-drop style node moving capabilities.
To use the admin functionality, install with the admin extra:
pip install django-tree-queries[admin]With automatic position management:
For the best admin experience with proper ordering, use OrderableTreeNode:
from django.contrib import admin
from tree_queries.admin import TreeAdmin
from tree_queries.models import OrderableTreeNode
class Category(OrderableTreeNode):
name = models.CharField(max_length=100)
# position field and ordering are inherited from OrderableTreeNode
@admin.register(Category)
class CategoryAdmin(TreeAdmin):
list_display = [*TreeAdmin.list_display, "name"]
position_field = "position" # Enables sibling ordering controlsWith manual position management:
If you prefer to manage positions yourself:
from django.contrib import admin
from django.db.models import Max
from tree_queries.admin import TreeAdmin
from tree_queries.models import TreeNode
class Category(TreeNode):
name = models.CharField(max_length=100)
position = models.PositiveIntegerField(default=0)
class Meta:
ordering = ["position"]
def save(self, *args, **kwargs):
# Custom position logic here
if not self.position:
self.position = (
10
+ (
self.__class__._default_manager.filter(parent_id=self.parent_id)
.order_by()
.aggregate(p=Max("position"))["p"]
or 0
)
)
super().save(*args, **kwargs)
save.alters_data = True
@admin.register(Category)
class CategoryAdmin(TreeAdmin):
list_display = [*TreeAdmin.list_display, "name"]
position_field = "position"The TreeAdmin provides:
Configuration:
Required list_display columns:
These are included by default in TreeAdmin.list_display.
When migrating from django-mptt to django-tree-queries, you'll need to populate the position field (or whatever field you use for sibling ordering) based on the existing MPTT lft values. Here's an example migration:
def fill_position(apps, schema_editor):
ModelWithMPTT = apps.get_model("your_app", "ModelWithMPTT")
db_alias = schema_editor.connection.alias
position_map = ModelWithMPTT.objects.using(db_alias).annotate(
lft_rank=Window(
expression=RowNumber(),
partition_by=[F("parent_id")],
order_by=["lft"],
),
).in_bulk()
# Update batches of 2000 objects.
batch_size = 2000
qs = ModelWithMPTT.objects.all()
batches = (qs[i : i + batch_size] for i in range(0, qs.count(), batch_size))
for batch in batches:
for obj in batch:
obj.position = position_map[obj.pk].lft_rank
ModelWithMPTT.objects.bulk_update(batch, ["position"])
class Migration(migrations.Migration):
dependencies = [...]
operations = [
migrations.RunPython(
code=fill_position,
reverse_code=migrations.RunPython.noop,
)
]This migration uses Django's Window function with RowNumber() to assign position values based on the original MPTT lft ordering, ensuring that siblings maintain their relative order after the migration.
Note that the position field is used purely for ordering siblings and is not an index. By default, django-tree-queries' admin interface starts with a position value of 10 and increments by 10 (10, 20, 30, etc.) to make it explicit that the position values themselves have no inherent meaning - they are purely for relative ordering, not a sibling counter or index.
Replacing add_related_count():
django-mptt's add_related_count() method for cumulative related object counts is not directly supported in django-tree-queries. The implementation would be complex and database-specific. For community-contributed solutions and discussion of alternative approaches, see mptt-related issues on GitHub.
| Back | FazBrowse Home | New Git URL |