Use Qscintilla instead of implementing our own code editor by MKleusberg · Pull Request #260 · sqlitebrowser/sqlitebrowser · GitHub

This adds the QScintilla library to our project. I've changed the SqlTextEdit class to use the QScintilla editor instead of depending on a plain Qt widget and trying to improve it for code editing. This should build fine on Qt4 and Qt5 using either cmake or qmake - so everybody should have the chance to test this (and give feedback) :)

Most of the time the new code behaves just like the old one but a few things changed. Right now these features are missing:

No separate syntax highlighting colours for functions and tables
No auto completion for table fields

These features are new:

Better auto completion for keywords, functions and tables (icons, help texts, ...)
Shift+Tab works as expected
Brace matching
Auto indentation

The main thing however is that this saves us from adding more and more code for all sorts of things (search & replace, error highlighting, ...) which has already been written by other people.

As mentioned in #258 I don't see any license issues and especially none which we wouldn't have had anyway because of the plotting library...

Squash them into one commit? 😄

Yep, but because I'm still working on it I feel like it's better to wait because this way I can always go back to e.g. yesterday's version if something breaks. As soon as I'm happy with the changes I can out them into one commit 😉

Ahh yeah, makes sense. Didn't think of that. 😀

"... because of the plotting library."

Uh oh... ?

@justinclift
See https://github.com/sqlitebrowser/sqlitebrowser/blob/master/libs/qcustomplot-source/GPL.txt or http://www.qcustomplot.com/ for example - they are using the GPL as well. But as I said: it's an external library which we can always link dynamically if it should be required. Then it's like we're not touching it at all; if that was a problem the Apple Store would probably be empty 😉

k, no worries then. 😄

Just did a bit of additional code cleanup, rebased the entire branch to the current master and fixed syntax highlighting in the Edit Table dialog -> Progress :)

Is it easy to strip the unused qscintilla sources? I guess we could spare all the python stuff. I see you already removed the qt3 support.

Yes, I've removed Qt3 code, documentation, examples, and the Qt Creator plugin. Now I've also removed the Python stuff (which wasn't used anywhere), the API definitions for Python code (not used anywhere), and all the lexers except for the SQL one. Removing the lexers required a bit more work, making it necessary to edit both build files as well as the libs/qscintilla/src/Catalogue.cpp file. But man, this saves us a lot of code in the repository! Thanks for drawing my attention to this :)

After removing the QScintilla files we don't need I merged all the commit so far into one to make the diff of this entire pull request much smaller.

Since then I've done a bit of restructuring in the code to make it easier to understand and compile faster. Then I've added the own syntax highlighting styles for SQLite function names and table names from the database and finally I've restored the auto completion feature for column names. It's so much simpler with QScintilla than it used to be with our original code 😄

So I think everything in this branch is working more or less as before now. So I think it's ready for testing now! If there are no remaining major bugs (like no building on some platforms, etc.) I would suggest merging this in a couple of days to avoid a lot of rabasing and have it tested by more people than when doing separate builds.

Compile fails on OSX:

clang++ -headerpad_max_install_names -stdlib=libc++ -mmacosx-version-min=10.7 -arch x86_64 -o sqlitebrowser.app/Contents/MacOS/sqlitebrowser release/main.o release/sqlitedb.o release/MainWindow.o release/CreateIndexDialog.o release/EditTableDialog.o release/PreferencesDialog.o release/AboutDialog.o release/EditDialog.o release/ExportCsvDialog.o release/ImportCsvDialog.o release/sqltextedit.o release/sqlitetypes.o release/csvparser.o release/ExtendedTableWidget.o release/Sqlite3Lexer.o release/Sqlite3Parser.o release/sqlitetablemodel.o release/FilterTableHeader.o release/SqlExecutionArea.o release/VacuumDialog.o release/DbStructureModel.o release/Application.o release/CipherDialog.o release/ExportSqlDialog.o release/SqlUiLexer.o release/moc_sqlitedb.o release/moc_MainWindow.o release/moc_CreateIndexDialog.o release/moc_AboutDialog.o release/moc_EditTableDialog.o release/moc_PreferencesDialog.o release/moc_EditDialog.o release/moc_ExportCsvDialog.o release/moc_ImportCsvDialog.o release/moc_sqltextedit.o release/moc_ExtendedTableWidget.o release/moc_sqlitetablemodel.o release/moc_FilterTableHeader.o release/moc_SqlExecutionArea.o release/moc_VacuumDialog.o release/moc_DbStructureModel.o release/moc_Application.o release/moc_CipherDialog.o release/moc_ExportSqlDialog.o release/moc_SqlUiLexer.o release/qrc_icons.o release/qrc_flags.o   -F/usr/local/Cellar/qt/4.8.6/lib -L/usr/local/Cellar/qt/4.8.6/lib -lsqlite3 -ldl -L/usr/local/lib -L/usr/local/opt/sqlite/lib -framework Carbon -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qhexedit -L/Users/jc/git_repos/sqlitebrowser/src/../libs/antlr-2.7.7 -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qcustomplot-source -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qscintilla/Qt4Qt5 -lantlr -lqhexedit -lqcustomplot -lqscintilla2 -framework QtGui -L/usr/local/Cellar/qt/4.8.6/lib -F/usr/local/Cellar/qt/4.8.6/lib -framework QtCore -framework QtNetwork 
/usr/local/Cellar/qt/4.8.6/bin/moc -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -D__APPLE__ -D__GNUC__ Qsci/qscilexer.h -o release/moc_qscilexer.cpp
/usr/local/Cellar/qt/4.8.6/bin/moc -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -D__APPLE__ -D__GNUC__ Qsci/qscilexersql.h -o release/moc_qscilexersql.cpp
ld: library not found for -lqscintilla2
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [sqlitebrowser.app/Contents/MacOS/sqlitebrowser] Error 1
make[1]: *** [release] Error 2
make: *** [sub-src-make_default] Error 2
make: *** Waiting for unfinished jobs....

Weird, the snippet you pasted really looks odd to me because in the first line it looks like it's trying to link it all together to an application but in the two following lines it looks like it's still compiling the QScintilla library. The error message which must be thrown by the first line says that the library can't be found (yet). Not sure why it doesn't wait for it to be built... Can you maybe try again and post the entire log?

Will do. I was using -j9, which is probably responsible for the weird log. Gimme a few minutes though, as I'm doing something else atm. 😉

Just did a git pull from this branch and it compiled without error this time. *whew* 😉

Trying this new QtScintilla thing out, it seems pretty nice. The tab and shift-tab thing are working, and better code completion already:

Note - I'm going to adjust the expected date for the next release to be one week from now instead of this weekend.

I definitely don't have time to do stuff for a release this weekend. Need to get an upgrade done for Gluster + Jenkins, now that Google is shutting off their OpenID 2.0 support. 😦

Cool! Glad to hear it's finally working :)
As far as I see this branch has all the features of master in it now, plus adding a few new ones like these:

Try typing 'substr(' (or any other function name) and it should show you a short explanation of the parameters and the function's purpose
Code folding to allow folding of multi-line statements
Errors in the Execute SQL tabs are highlighted (sometimes because SQLite won't recognise completely invalid statements)

Regex?

Do changes in the font selector in the Preferences dialog correctly update stuff in the new widget?

I tested quickly and it didn't seem like it. 😉

works pretty good, I just have 2 small issues.

Could we bring the auto completion popup only after 3? entered characters?
It seems the sqlite datatypes are missing in the autocompletion list, maybe were already missing before.

The changes in the preferences dialog only affected the editor after restarting the application... This is fixed now, so you get an immediate effect! Thanks for reminding me of that :) The small arrow indeed indicates a drop down menu but you have to hold your mouse button down for some time to have it opened. A single click performs the default operation (Save As when not saved yet, and Save when already saved).

As @rp- suggested I've also changed the auto completion to start only after three characters and include some SQLite datatypes.

Does regex still work? (as per #215)

I'm trying to think of how to test it myself, but my brain is not clicking for it (focused on other stuff atm 😉).

what does REGEXP have todo with this?

Gah yeah... you're right. It's been a long day. 😉

Yes, this is a frontend-only change (or at least it should be), things like the regular expressions are not affected. So this means we should test the following:

Does it build well? (an entire library has been added...)
Typing, scrolling, Tab, etc. (i.e. basic text editor features)
Syntax highlighting
Auto completion (keywords, functions, tables, columns)
Other code editing features (brace matching, help texts, code folding...) [these are mostly new, so there's no comparison to the master branch possible here]
Accessing the contents correctly (the execute current line button and the execute selected SQL only feature!!)
Settings (colours, fonts, ...)
All this should work for all SQL editor widgets in the application. Besides the Execute SQL tabs, these are the SQL log views and the SQL preview in the Create/Edit Table dialog.

This is a first rough implementation of QScintilla support as SQL text editor. It should work mostly and build fine with qmake as well as cmake. The new code supports all the features of the old one plus adding a few subtle improvements. The main point of this, however, is reducing the code we have to maintain and making it easier to add new features to the editor.

See issue #261.

If a syntax error is reported by SQLite during execution in the Execute SQL tab of the main window try to locate the problematic SQL statement and highlight it using a red squiggling line.

Only start auto completion after a three letter threshold. Only take words for auto completion from the API word list, not from the current document.

Just reorganised the pull request: The actual transition to the new widget is done in one commit now (the first one) while the other commits add new features or make improvements, so they can be viewed separately. I've also updated the QScintilla source code to the newly released version 2.9.

As I haven't heard of any objections I'll merge this now and see what happens :)

Use Qscintilla instead of implementing our own code editor

Really nice enhancement @MKleusberg!

Excellent. Lets get some people to test this. 😄

Hmmmm, just noticed an interesting behaviour with this. When compiling this on OSX single threaded (eg "make") it's compiling fine. But when compiling multi-threaded (eg "make -j9") it's sometimes barfing:

[sqlitebrowser (master)]$ qmake
[sqlitebrowser (master)]$ make -j9
[...]
clang++ -c -pipe -stdlib=libc++ -mmacosx-version-min=10.7 -O2 -arch x86_64 -fPIC -w -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -o release/moc_SciClasses.o release/moc_SciClasses.cpp
clang++ -c -pipe -stdlib=libc++ -mmacosx-version-min=10.7 -O2 -arch x86_64 -fPIC -w -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -o release/moc_ScintillaQt.o release/moc_ScintillaQt.cpp
clang++ -headerpad_max_install_names -stdlib=libc++ -mmacosx-version-min=10.7 -arch x86_64 -o sqlitebrowser.app/Contents/MacOS/sqlitebrowser release/main.o release/sqlitedb.o release/MainWindow.o release/CreateIndexDialog.o release/EditTableDialog.o release/PreferencesDialog.o release/AboutDialog.o release/EditDialog.o release/ExportCsvDialog.o release/ImportCsvDialog.o release/sqltextedit.o release/sqlitetypes.o release/csvparser.o release/ExtendedTableWidget.o release/Sqlite3Lexer.o release/Sqlite3Parser.o release/sqlitetablemodel.o release/FilterTableHeader.o release/SqlExecutionArea.o release/VacuumDialog.o release/DbStructureModel.o release/Application.o release/CipherDialog.o release/ExportSqlDialog.o release/SqlUiLexer.o release/moc_sqlitedb.o release/moc_MainWindow.o release/moc_CreateIndexDialog.o release/moc_AboutDialog.o release/moc_EditTableDialog.o release/moc_PreferencesDialog.o release/moc_EditDialog.o release/moc_ExportCsvDialog.o release/moc_ImportCsvDialog.o release/moc_sqltextedit.o release/moc_ExtendedTableWidget.o release/moc_sqlitetablemodel.o release/moc_FilterTableHeader.o release/moc_SqlExecutionArea.o release/moc_VacuumDialog.o release/moc_DbStructureModel.o release/moc_Application.o release/moc_CipherDialog.o release/moc_ExportSqlDialog.o release/moc_SqlUiLexer.o release/qrc_icons.o release/qrc_flags.o   -F/usr/local/Cellar/qt/4.8.6/lib -L/usr/local/Cellar/qt/4.8.6/lib -lsqlite3 -ldl -L/usr/local/lib -L/usr/local/opt/sqlite/lib -framework Carbon -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qhexedit -L/Users/jc/git_repos/sqlitebrowser/src/../libs/antlr-2.7.7 -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qcustomplot-source -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qscintilla/Qt4Qt5 -lantlr -lqhexedit -lqcustomplot -lqscintilla2 -framework QtGui -L/usr/local/Cellar/qt/4.8.6/lib -F/usr/local/Cellar/qt/4.8.6/lib -framework QtCore -framework QtNetwork 
ld: library not found for -lqscintilla2
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [sqlitebrowser.app/Contents/MacOS/sqlitebrowser] Error 1
make[1]: *** [release] Error 2
make: *** [sub-src-make_default] Error 2
make: *** Waiting for unfinished jobs....
rm -f libqscintilla2.a
ar cq libqscintilla2.a release/qsciscintilla.o release/qsciscintillabase.o release/qsciabstractapis.o release/qsciapis.o release/qscicommand.o release/qscicommandset.o release/qscidocument.o release/qscilexer.o release/qscilexersql.o release/qscimacro.o release/qsciprinter.o release/qscistyle.o release/qscistyledtext.o release/MacPasteboardMime.o release/InputMethod.o release/SciClasses.o release/ListBoxQt.o release/PlatQt.o release/ScintillaQt.o release/LexSQL.o release/Accessor.o release/CharacterCategory.o release/CharacterSet.o release/LexerBase.o release/LexerModule.o release/LexerNoExceptions.o release/LexerSimple.o release/PropSetSimple.o release/StyleContext.o release/WordList.o release/AutoComplete.o release/CallTip.o release/CaseConvert.o release/CaseFolder.o release/Catalogue.o release/CellBuffer.o release/CharClassify.o release/ContractionState.o release/Decoration.o release/Document.o release/Editor.o release/EditModel.o release/EditView.o release/ExternalLexer.o release/Indicator.o release/KeyMap.o release/LineMarker.o release/MarginView.o release/PerLine.o release/PositionCache.o release/RESearch.o release/RunStyles.o release/ScintillaBase.o release/Selection.o release/Style.o release/UniConversion.o release/ViewStyle.o release/XPM.o release/moc_qsciscintilla.o release/moc_qsciscintillabase.o release/moc_qsciabstractapis.o release/moc_qsciapis.o release/moc_qscilexer.o release/moc_qscilexersql.o release/moc_qscimacro.o release/moc_SciClasses.o release/moc_ScintillaQt.o
ranlib -s libqscintilla2.a
[sqlitebrowser (master)]$

Any ideas what could cause that?

On my system -j9 seems to reliably trigger the failure. -j5 and -j7 don't. Weird.

MKleusberg force-pushed the qscintilla branch from 79dbd7a to b6d89af Compare April 15, 2015 17:42

MKleusberg added the enhancement Feature requests. label Apr 15, 2015

MKleusberg self-assigned this Apr 15, 2015

MKleusberg force-pushed the qscintilla branch from 488829e to fb4ee9a Compare April 16, 2015 09:45

justinclift added this to the 3.6 milestone Apr 17, 2015

MKleusberg added 7 commits April 23, 2015 19:51

EditTableDialog: Fix 'fooling' of list selection by dragging

24ce374

See issue #261.

qscintilla: Enable code folding

6fed5ec

Mark syntax errors in Execute SQL tab

649b179

If a syntax error is reported by SQLite during execution in the Execute SQL tab of the main window try to locate the problematic SQL statement and highlight it using a red squiggling line.

qscintilla: Finetune auto completion

9484633

Only start auto completion after a three letter threshold. Only take words for auto completion from the API word list, not from the current document.

qscintilla: Reload settings properly after they have changed

cf412a2

qscintilla: Add some missing keywords to auto completion word list

e714eee

MKleusberg force-pushed the qscintilla branch from 8709ebe to e714eee Compare April 23, 2015 17:53

MKleusberg added a commit that referenced this pull request Apr 23, 2015

Merge pull request #260 from sqlitebrowser/qscintilla

506804a

Use Qscintilla instead of implementing our own code editor

MKleusberg merged commit 506804a into master Apr 23, 2015

justinclift mentioned this pull request Apr 24, 2015

Suggestion: 'Tab' indents multiline selection in SQL editor #255

Closed

MKleusberg deleted the qscintilla branch April 27, 2015 09:49

Use Qscintilla instead of implementing our own code editor by MKleusberg · Pull Request #260 · sqlitebrowser/sqlitebrowser · GitHub

Uh oh!

Uh oh!

Use Qscintilla instead of implementing our own code editor - #260

Use Qscintilla instead of implementing our own code editor#260MKleusberg merged 7 commits intomastersqlitebrowser/sqlitebrowser:masterfrom qscintillasqlitebrowser/sqlitebrowser:qscintillaCopy head branch name to clipboard

Conversation

MKleusberg commented Apr 15, 2015

Uh oh!

MKleusberg commented Apr 15, 2015

Uh oh!

justinclift commented Apr 15, 2015

Uh oh!

MKleusberg commented Apr 15, 2015

Uh oh!

justinclift commented Apr 15, 2015

Uh oh!

justinclift commented Apr 15, 2015

Uh oh!

MKleusberg commented Apr 15, 2015

Uh oh!

justinclift commented Apr 15, 2015

Uh oh!

MKleusberg commented Apr 15, 2015

Uh oh!

rp- commented Apr 15, 2015

Uh oh!

MKleusberg commented Apr 16, 2015

Uh oh!

MKleusberg commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

MKleusberg commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

MKleusberg commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

justinclift commented Apr 17, 2015

Uh oh!

rp- commented Apr 19, 2015

Uh oh!

MKleusberg commented Apr 19, 2015

Uh oh!

justinclift commented Apr 19, 2015

Uh oh!

rp- commented Apr 19, 2015

Uh oh!

justinclift commented Apr 19, 2015

Uh oh!

MKleusberg commented Apr 20, 2015

Uh oh!

MKleusberg commented Apr 23, 2015

Uh oh!

ghost commented Apr 23, 2015

Uh oh!

justinclift commented Apr 24, 2015

Uh oh!

justinclift commented Apr 24, 2015

Uh oh!

justinclift commented Apr 24, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use Qscintilla instead of implementing our own code editor#260
MKleusberg merged 7 commits into
masterfrom
qscintilla