fix: don't sleep for a half second every time we update meilisearch [FC-0117] by bradenmacdonald · Pull Request #38576 · openedx/openedx-platform · GitHub

Description

Various actions in the platform like changing a course's schedule or importing a course from modulestore into a library seem much slower than they should be at the moment, due to the fact that devstack runs all tasks synchronously (not in separate celery workers) and the tasks that update Meilisearch specifically are slow.

When investigating various slowness, I discovered that every update to Meilisearch was taking ~0.5s almost exactly, and then I found the culprit:

openedx-platform/openedx/core/djangoapps/content/search/api.py

Lines 156 to 159 in 6efb92c

    
           client = _get_meilisearch_client() 
        
           current_status = client.get_task(info.task_uid) 
        
           while current_status.status in ("enqueued", "processing"): 
        
               time.sleep(0.5)

It turns out that the Meilisearch task is never immediately complete, so this always results in a wait of 500ms every time we call Meilisearch 🤦🏻‍♂️. But, Meilisearch updates are actually pretty fast and usually done in 10-20ms, so we simply need to reduce the sleep delay and we see a huge boost in performance.

Note: it would obviously be better to use asyncio.sleep() here, but we aren't set up for that in openedx-platform.

Supporting information

Found in the context of openedx/openedx-core#584

Testing instructions

The easiest way to test this is:

Go to any course's Schedule & Details page. Change the start date and press save. Time how long it takes.

Repeat with and without the fix.

Why this test: updating a course's start date results in 19 XBLOCK_UPDATED events for the meta blocks in each course: type@course+block@course, type@about+block@subtitle, type@about+block@duration, type@about+block@description, type@about+block@short_description,type@about+block@overview, type@about+block@entrance_exam_enabled, type@about+block@entrance_exam_id, type@about+block@entrance_exam_minimum_score_pct, type@about+block@about_sidebar_html. For some reason, all of these but the course block actually have two XBLOCK_UPDATED events for each block, hence the total of 19. Note: some of these shouldn't even be XBlocks, and I'm not sure why we're getting two XBLOCK_UPDATED events for each one, but I want to keep this PR focused on the main source of delay.

Deadline

ASAP, need to backport this to Verawood.

Other information

Private ref BB-10823

Thanks for the pull request, @bradenmacdonald!

This repository is currently maintained by @openedx/wg-maintenance-openedx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

Thanks for the pull request, @bradenmacdonald!

This repository is currently maintained by @openedx/wg-maintenance-openedx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

Good catch @bradenmacdonald!

LGTM 👍

I tested this using the instructions from the PR
I read through the code
I checked for accessibility issues
Includes documentation

Definitely needs a backport, though I imagine that can wait until you're done with your perf fixes on master. Thanks @bradenmacdonald!

* fix: ValidationError when trying to add components to a collection (#38579) * fix: don't sleep for a half second every time we update meilisearch (#38576) * fix: various issues with modulestore migrator and content library events (#38508) * fix: update modulestore migrator to not publish in draft context * fix: use correct date/user info for modulestore migrator * fix: allow our event handling tasks to call subtasks * fix: better handling of deletion/un-deletion using openedx-core fixes * fix: exception raised when mass-deleting all items in a content library (ContentLibraryBlockNotFound was being raised and not caught, propagating up to celery task error) * fix: don't update the 'collections' field of deleted entities when modifying a collection * fix: keep collection "# of entities" count updated when entities deleted * refactor: use LIBRARY_COLLECTION_UPDATED event to update collections on entity (un)delete * fix: unable to delete "ghost" component in library if it exists but has no draft * fix: error when deleting lib component used in a container (#38607) * fix: bump openedx-core to v1.0.2 --------- Co-authored-by: Claude <noreply@anthropic.com>

fix: don't sleep for a half second every time we call meilisearch

846d4d5

bradenmacdonald requested review from ormsbee and rpenido May 6, 2026 22:45

bradenmacdonald added the FC Relates to an Axim Funded Contribution project label May 6, 2026

openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels May 6, 2026

openedx-webhooks added this to Contributions May 6, 2026

github-project-automation Bot moved this to Needs Triage in Contributions May 6, 2026

bradenmacdonald mentioned this pull request May 6, 2026

Modulestore migrator performance openedx/openedx-core#584

Closed

rpenido approved these changes May 6, 2026

View reviewed changes

Comment thread openedx/core/djangoapps/content/search/api.py

ormsbee approved these changes May 6, 2026

View reviewed changes

fix: with multiple meilisearch tasks, only wait for the final one.

a1ae25a

mphilbrick211 moved this from Needs Triage to In Eng Review in Contributions May 7, 2026

bradenmacdonald merged commit aee47ed into openedx:master May 7, 2026
41 of 42 checks passed

bradenmacdonald deleted the braden/fix-meilisearch-waits branch May 7, 2026 02:32

github-project-automation Bot moved this from In Eng Review to Done in Contributions May 7, 2026

bradenmacdonald mentioned this pull request Jun 3, 2026

Backport library fixes to Verawood #38695

Merged

bradenmacdonald mentioned this pull request Aug 17, 2026

[Ulmo] Library v2 commit endpoint timeout with large Meilisearch indexes #38993

Open

fix: don't sleep for a half second every time we update meilisearch [FC-0117] by bradenmacdonald · Pull Request #38576 · openedx/openedx-platform · GitHub

fix: don't sleep for a half second every time we update meilisearch [FC-0117] - #38576

fix: don't sleep for a half second every time we update meilisearch [FC-0117]#38576
bradenmacdonald merged 2 commits into
openedx:masterfrom
open-craft:braden/fix-meilisearch-waits

bradenmacdonald commented May 6, 2026 •
edited
Loading

Uh oh!

Uh oh!

openedx-webhooks commented May 6, 2026

Uh oh!

openedx-webhooks commented May 6, 2026

Uh oh!

rpenido left a comment

Uh oh!

Uh oh!

ormsbee commented May 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	client = _get_meilisearch_client()
	current_status = client.get_task(info.task_uid)
	while current_status.status in ("enqueued", "processing"):
	time.sleep(0.5)

fix: don't sleep for a half second every time we update meilisearch [FC-0117] by bradenmacdonald · Pull Request #38576 · openedx/openedx-platform · GitHub

fix: don't sleep for a half second every time we update meilisearch [FC-0117] - #38576

fix: don't sleep for a half second every time we update meilisearch [FC-0117]#38576bradenmacdonald merged 2 commits intoopenedx:masteropenedx/openedx-platform:masterfrom open-craft:braden/fix-meilisearch-waitsopen-craft/openedx-platform:braden/fix-meilisearch-waitsCopy head branch name to clipboard

Conversation

bradenmacdonald commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Supporting information

Testing instructions

Deadline

Other information

Uh oh!

openedx-webhooks commented May 6, 2026

Uh oh!

openedx-webhooks commented May 6, 2026

Uh oh!

rpenido left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ormsbee commented May 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fix: don't sleep for a half second every time we update meilisearch [FC-0117]#38576
bradenmacdonald merged 2 commits into
openedx:masterfrom
open-craft:braden/fix-meilisearch-waits

bradenmacdonald commented May 6, 2026 •
edited
Loading