Enabling SQLite FTS5 module by default. by serpulga · Pull Request #199 · sql-js/sql.js · GitHub

Compiled using EMSCRIPTEN 1.37.9.

Tests:

$ ./test/run.sh 
Testing ./test/test_blob.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_database.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_errors.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_extension_functions.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_functions.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_issue128.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_issue55.js...	Passed.
Testing ./test/test_issue73.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_issue76.js...	Passed.
Testing ./test/test_node_file.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_statement.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_transactions.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!
Passed.
Testing ./test/test_worker.js...	writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!

assert.js:90
  throw new assert.AssertionError({
  ^
AssertionError: Reading BLOB
    at Worker.worker.onmessage (/Users/serpulga/Documents/src/sql.js/test/test_worker.js:23:14)
    at ChildProcess.<anonymous> (/Users/serpulga/Documents/src/sql.js/node_modules/workerjs/index.js:25:38)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at handleMessage (internal/child_process.js:695:10)
    at Pipe.channel.onread (internal/child_process.js:440:11)
Fail!\e[0m
Warning\e[0m : 12 tests passed out of 13

Same error I get on master [6dc4ac5].

Compiled using EMSCRIPTEN 1.37.9.

+1 for this; if the FTS5 engine is on, that'd allow search indexes created by other languages to be used in JS in the browser.

sql.js is already quite a huge dependency to have. I don't think adding other modules by default is a good idea.

But compiling different versions for different needs is a good idea.

I believe I am able to have FTS5 running. Here are the procedures:

modify Makefile by adding "-DSQLITE_ENABLE_FTS5" in the CFLAGS line. So, the complete CFLAGS line will become CFLAGS=-O2 -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_DISABLE_LFS -DLONGDOUBLE_TYPE=double -DSQLITE_THREADSAFE=0 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS5
make
A few .js files will be created at the js directory. I tested worker.sql.js only so far
In order to test it, I cloned http://kripken.github.io/sql.js/GUI/ to my server and then delete the line <script src="codemirror/mode/sql/sql.js"></script>
Modify gui.js to point to my own worker.sql.js at this line <a href="https://github.com/kripken/sql.js"> to var worker = new Worker("worker.sql.js");
run the gui page
I typed the commands from http://www.sqlitetutorial.net/sqlite-full-text-search/ into the textarea box. All FTS5 commands work perfectly.

In the next few weeks, I will try to test the performance of FTS5 by adding thousands of documents.

@3CE8D2BAC65BDD6AA9 Thank you for posting the steps. I was also able to run FTS5 but this pull request was to enable it by default inside sql.js; but since I agree with @lovasoa (we shouldn't add modules that most people won't use) I didn't even try to fix the failing unit tests.

Why change the SQLITE_THREADSAFE default setting? Was this intended?

I noticed the same thing but no clue why there is such a different.

I am open to the idea of making such compile time extensions into different versions of sql.js (such as memory-growth, debug, etc).

Would that be a good solution here? Can we expect future extensions to pass the standard test suite? Would a standard procedure for adding such extensions help in the future?

Would be nice to have an extra flavor with FTS5 enabled.

I also wonder if that would be a good idea to include it by default, but I definitely would love to have a ready-made version with FTS5 enabled.

Enabling SQLite FTS5 module by default.

7e7ce61

Compiled using EMSCRIPTEN 1.37.9.

brody2consult reviewed Mar 9, 2018

View reviewed changes

kimown mentioned this pull request Apr 16, 2018

Uncaught Error: no such function: json_extract #237

Closed

pamelafox mentioned this pull request Sep 19, 2018

Make more strings translatable Khan/live-editor#685

Merged

lovasoa force-pushed the master branch from b3211a6 to e902ec2 Compare February 29, 2020 11:04

lovasoa force-pushed the master branch from 351ffdd to 6e7ec33 Compare October 11, 2020 20:00

arcanis mentioned this pull request Sep 18, 2024

Enables FTS5 by default #594

Open

Enabling SQLite FTS5 module by default. by serpulga · Pull Request #199 · sql-js/sql.js · GitHub

Uh oh!

Enabling SQLite FTS5 module by default. - #199

Enabling SQLite FTS5 module by default.#199
serpulga wants to merge 1 commit into
sql-js:masterfrom
serpulga:enable-fts5

serpulga commented May 15, 2017

Uh oh!

petermolnar commented Jan 15, 2018

Uh oh!

lovasoa commented Jan 15, 2018

Uh oh!

3CE8D2BAC65BDD6AA9 commented Mar 9, 2018 •
edited
Loading

Uh oh!

Uh oh!

serpulga commented Mar 9, 2018

Uh oh!

brody2consult Mar 9, 2018

Uh oh!

3CE8D2BAC65BDD6AA9 Mar 10, 2018

Uh oh!

dinedal commented Aug 2, 2018

Uh oh!

jrocha commented Oct 4, 2023

Uh oh!

jerefrer commented Nov 16, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

               EMCC=$(EMSCRIPTEN)/emcc
-              CFLAGS=-DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_DISABLE_LFS -DLONGDOUBLE_TYPE=double -DSQLITE_INT64_TYPE="long long int" -DSQLITE_THREADSAFE=0 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS
+              CFLAGS=-DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_DISABLE_LFS -DLONGDOUBLE_TYPE=double -DSQLITE_INT64_TYPE="long long int" -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS5

Enabling SQLite FTS5 module by default. by serpulga · Pull Request #199 · sql-js/sql.js · GitHub

Uh oh!

Enabling SQLite FTS5 module by default. - #199

Enabling SQLite FTS5 module by default.#199serpulga wants to merge 1 commit intosql-js:mastersql-js/sql.js:masterfrom serpulga:enable-fts5serpulga/sql.js:enable-fts5Copy head branch name to clipboard

Conversation

serpulga commented May 15, 2017

Uh oh!

petermolnar commented Jan 15, 2018

Uh oh!

lovasoa commented Jan 15, 2018

Uh oh!

3CE8D2BAC65BDD6AA9 commented Mar 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serpulga commented Mar 9, 2018

Uh oh!

brody2consult Mar 9, 2018

Choose a reason for hiding this comment

Uh oh!

3CE8D2BAC65BDD6AA9 Mar 10, 2018

Choose a reason for hiding this comment

Uh oh!

dinedal commented Aug 2, 2018

Uh oh!

jrocha commented Oct 4, 2023

Uh oh!

jerefrer commented Nov 16, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Enabling SQLite FTS5 module by default.#199
serpulga wants to merge 1 commit into
sql-js:masterfrom
serpulga:enable-fts5

3CE8D2BAC65BDD6AA9 commented Mar 9, 2018 •
edited
Loading