On some pages like Qt's Gerrit, Indiegogo or Telegram Web, this caused a crash
with QtWebEngine and Qt 5.10.1 in
QtWebEngineCore::WebContentsAdapter::webContents().
I'm not sure what causes the crash exactly, but I'm guessing it's some kind of
race condition between loading the URL initially and deserializing the history,
which both ends up loading the URL.
Since restoring the history means we end up on the given URL anyways, let's just
not open the URL beforehand, which seems to fix this.
Fixes#3619.
(cherry picked from commit d44ff5ba01)
Always interpret the first word in the command string as the command to
offer completions for, even if that word looks like a flag.
Fixes#3460, where the command string `:-w open` would attempt to offer
completions for `open` but crash because the parsing was thrown off.
By moving the flag-stripping logic to _after_ we determine the command,
`:-w open` interprets `:-w` as the command. Since that is not a valid
command, we won't offer any completions.
Added spaces to comments, use python naming conventions, remove
_load_progress_fake as unnecessary, rename _on_load_*_fake to
_on_load_*_workaround (for less confusion), and use qtutils rather than
QT_VERSION_STR
This uses the much more reliable `loadProgress(100)` in place of
`loadFinished(true)` for WebEngine, with `loadProgressFake` and
`loadFinishedFake` used instead of the 'official' variants.
This broke in #3382 since re.fullmatch does a different thing for trailing
newlines:
>>> line
'===========\n'
>>> re.match(r'^=+$', line)
<_sre.SRE_Match object; span=(0, 11), match='==========='>
>>> re.fullmatch(r'=+', line)
>>>
This now strips the line by default, and adds newlines if needed.
Update the description to mention the number of columns and change the
default to ["white", "white", "white"] to make it more obvious that
multiple colors can be specified. This also satisfies the config test
that expects the default value for ListOrValue types to be a list.
One other test had to be tweaked to use a config option that is still
just a QtColor rather than a ListOrValue.
While it is possible to provide just two colors, it is "undefined
behavior". It will use the first color as the third color, but that is
an artifact of the implementation and therefore not documented (though
also not an error, as it is harmless).
colors.completion.fg is now a list instead of a QColor. As this test
specifically wanted to test a QColor, I just changed it to a different
config option.
When QtWebEngine shuts down, it calls pending callbacks, which means we access
an invalid 'settings' object when that happens. The stack would look something
like this:
0 QtWebEngineCore::WebEngineSettings::setAttribute(QtWebEngineCore::WebEngineSettings::Attribute, bool)
12 QtWebEngineCore::CallbackDirectory::invokeEmptyInternal<QVariant const&>(QtWebEnginePrivate::QWebEngineCallbackPrivateBase<QVariant const&>*)
14 QtWebEngineCore::CallbackDirectory::~CallbackDirectory()
19 QWebEnginePage::~QWebEnginePage()
If we instead get the settings from the view freshly, we get a RuntimeError from
PyQt telling us that it's dead. Not sure why it doesn't know about settings
being dead...
With that, we'd get a RuntimeError, which we can simply ignore as it doesn't
matter anyways if the tab is gone.
Fixes#3399
This means something like this:
:hint ;; later 20 follow-hint a ;; later 20 tab-close
Won't crash anymore, as the webelem.Error will be shown in the statusbar.
This gives us slightly different values it seems, but I think they are more
correct (and don't need the "+ 5" above).
Taking the width directly (not the sizeHint width) returned values like 100 or
so when the scrollbar wasn't shown yet, so that seems wrong.
The situation where there's no scroll bar at the point the column widths are
calculated is a very rare one. What happens more often is that the scroll bar
disappears due to filtering, in which case we didn't recalculate the column
widths anyways.
Furthermore, we can even go wrong with this calculation, when we calculate the
column widths while the entire completion is invisible - then the scroll bar
won't be visible either, and we won't subtract the space needed for it.
Let's not try to optimize for this uncommon case, and just always subtract the
scrollbar, even if it's not there initially.
Fixes#3359Closes#3389
Now colors.completion.fg may be set to a list to specify a different
color for each completion column. For example:
:set colors.completion.fg [black,blue,white] will use black text for the
first column, blue for the second, and white for the third.
Setting to a single value still works and behaves as before. The default
is unchanged from 'white'.
Resolves#1794.
command-accept --rapid will run the command without clearing the prompt,
allowing "rapid fire" commands. For example, one could open completion
for `open -t` and open several tabs in a row.
The default binding is ctrl+enter.
Resolves#588.
I mistakenly checked the length of wheres instead of words. This fixes
that check, renames 'wheres' to 'where_clause' to be clear
that it is a string and not an array, and adds a test.
Previously this simply crashed if there was ever malformed utf-8 in the
stderr or stdout streams, perhaps as a result of an incorrectly spawned
command. See e.g. #3222
There are various small changes here:
- If the process is already finished, we don't try to terminate it.
- On Windows, we use QProcess::kill instead of QProcess::terminate, as terminate
will only work with processes which have a GUI loop.
- We assert that quitting the suprocess actually worked.
Fixes#3384
re.match features an implicit left anchor, which can be surprising.
re.fullmatch features implicit anchors on both sides, but is aptly named
and unsurprising.
re.search has no such implicit anchors, which ought to be the default
even if a single anchor is needed.
Perviously, 'foo bar' would match 'foo/bar' but not 'bar/foo'. Now it
will match both, using a query with a WHERE clause like:
WHERE ((url || title) like '%foo%' AND (url || title) like '%bar%')
This does not seem to change the performance benchmark. However, it does
create a new query for every character added rather than re-running the
same query with different parameters. We could re-use queries if we
maintained a list like self._queries=[1_arg_query, 2_arg_query, ...].
However, it isn't clear that such a complexity would be necessary.
Resolves#1651.
This change makes it so that stderr and stdout is unconditionally read
from for a completed process, and sent to qute://spawn-output. This
allows the user to see the results of the previous process, even if they
had forgotten to use --output.
This fixes the following problems found in a review:
1. Manual modification of the asciidoc has been undone.
2. --output-to-tab has been renamed to the less verbose --output.
3. spawn_output has been changed to spawn-output in the url.
4. Erroneous newline in imports has been removed.
5. output in guiprocess.py has been marked private.
6. If there is no output for either stderr or stdout, say so.
7. Missing space in a text line was added.
8. Redundant initialising of an empty string removed.
This test is supposed to ensure that user scripts don't run on iframes
when the @noframes directive is set in the greasemonkey metadata. It is
failing sometimes on travis but passing on local test runs. Personally I
haven't actually ran the whole test suite through, just the javascript
tests. It maybe be some stale state that only shows up when you run the
whole suite. It may be some timing issue that only shows up on travis
because ???. Hopefully this stops the red x from showing up on the PR.
For some reason, if we don't wait for about:blank to be fully loaded with
Qt 5.10, we get the next LoadStatus.finished notification with about:blank as
URL.
This is most likely caused by the changes in
https://codereview.qt-project.org/#/c/202924/
See #3003
This eliminates all separate pytest envs in favor of conditionals in
[testenv]. This requires renaming some environments to make the lack of
certain functionality explicit:
- instead of omitting pyqt{version}, use pyqtlink to use host PyQt
tox.ini: eliminate -nocov
It is possible to set the `PYTEST_ADDOPTS` environment variable to
enable coverage checks, rather than a new command.
This implements the orphaned editor fix for WebKit. Webkit wasn't
crashing before, but this causes webkit to show the same warning
webengine does if the editor is orhpaned (rather than silently
continuing). This allows the same BDD test to pass for both webkit and
webengine.
Instead of rewiring signals on tab.shutting_down, have the webelem check
if its parent tab is deleted, and throw a specific exception.
This is only necessary in WebEngine, Webkit does not crash when the
editor is orphaned.
I tried to write a test for is_deleted, but could not get it to pass:
```
def test_is_deleted(qtbot, view, config_stub, tab_registry, mode_manager):
tab_w = Tab(win_id=0, mode_manager=mode_manager)
qtbot.add_widget(tab_w)
tab_w._set_widget(view)
assert not tab_w.is_deleted()
sip.delete(view)
#assert tab_w.is_deleted()
```
The qtbot post-test cleanup would error due to the deleted view.
This reverts commit e72e8b8556.
Now that the SQL category works in isolation, it is possible to hide
quickmarks/bookmarks when those categories are empty.
Fixes#960
While QSortFilterProxyModel emits layoutChanged when changing the
pattern, QSqlQueryModel emits modelReset. As only layoutChanged was
connected, a HistoryCategory would only work in a model that also had at
least one ListCategory.
The simplest solution is to have the parent model emit the signal
directly. This also emits a single signal on a pattern change rather
that one for each child model.
Resolves#3016.
If we don't do this, earlier tests can affect later ones when e.g. using "...
should not be logged", as we don't really wait until a test has been fully
finished.
The signal we were using to inject greasemonkey scripts registered to
run at document-start (javaScriptWindowObjectCleared) was unreliable to
non-existant. The initialLayoutCompleted signal is a bit of an odd duck
too I suppose. Anyway, we don't anticipate any scripts would break from
being injected when the page is finished loaded that wouldn't already
have been flaky due to the complexities of the modern web. If there is
an issue hopefully someone raises an issue and we can look into it.
Test document-end and noframes. Because coverage.py told me to.
Hopefully this doesn't slow the test run down too much, particularly the
"should not be logged" bit.
I'm just reusing and existing test html page that used an iframe because
I'm lazy.
We weren't actually picking up the @noframes greasemonkey directive
because of this. I haven't tested this very extensively but it seems to
work for making the property value optional.
Use the `QWebPage.frameCreated` signal to get notifications of subframes
and connect the javascript injection triggering signals on those frames
too.
I had to add a `url = url() or requestedUrl()` bit in there because the
inject_userjs method was getting called to early or something when
frame.url() wasn't set or was set to the previous page so we were
passing the wrong url to greasemonkey.scripts_for().
I ran into a bizarre (I maybe it is completely obvious and I just don't
see it) issue where the signals attached to the main frame that were
connected to a partial function with the main frame as an argument were
not getting emitted, or at least those partial functions were not being
called. I worked around it by using None to mean defaulting to the main
frame in a couple of places.
Just runs a greasemonkey script on a test page and uses console.log to
ensure it is running.
Tests @include, and basic happy path greasemonkey.py operation (loading
and parsing script, scrip_for on webkit), only testing document-start
injecting point but that is the troublsome one at this point.
Tested on py35 debian unstable (oldwebkit and qtwebengine5.9) debian
stable qtwebengine5.7.
Note the extra :reload call for qt5.7 because document-start scripts
don't seem to run on the first page load with the current insertion
point. I need to look into this more to look at ways of fixing this.
Thanks to @sandrosc. A few breaking changes fixed (default method to
GM_xhr not working, GM_listvalues not cleaning up output, GM_setvalue
param checking logic wrong) and a few hygenic changes made.
Add qute version to GM_info object in GM wrapper.
Support using the greasemonkey @namespace metadata for its intended
purpose of avoiding name collisions.
Get a nice utf8 encoded string from a QUrl more better.
QTWebEngine 5.8 added support for parsing greasemonkey metadata blocks
and scripts added to the QWebEngineScriptCollection of a page or its
profile and then deciding what urls to run those scripts on and at what
point in the load process to run them. For earlier versions we must do
that work ourselves. But with the additional handicap of the less rich
qtwebengine api.
We have acceptNavigationRequest, loadStarted, loadProgress,
loadFinished, urlChanged to choose from regarding points at which to
register scripts for the current page.
Adding scripts on acceptNavigation loadStarted and loadFinished causes
scripts to run too early or too late (eg on the pages being navigated
from/to) and not run on the desired page at the time they are inserted.
We could maybe do some more sophisticated stuff with loadProgress but it
didn't have any better behaviour in the brief testing I gave it.
Registering scripts on the urlChanged event seems to work fine. Even if
it seems like there could be problems with the signal firing too often,
due to not necessarily being tied to the page load progress, that
doesn't seem to have an effect in practice. The event is fired when, for
example, the url fragment changes and even if we add a new script to the
collection (or remove an existing one) it doesn't have an effect on what
is running on the page.
I suspect all of those timing issues is due to the signals being
forwarded fairly directly from the underlying chomium/blink code but the
webengine script stuff only being pushed back to the implementation on
certain events.
Anyway, using urlChanged seems to work fine due to some quirk(s) of the
implementation. That might change with later development but this
codepath is only ever going to be used for version 5.7.
There are other potential optimizations like not removing and then
re-adding scripts for the current page. But they probably wouldn't do
anything anyway, or at least anything that you would expect.
This change requires urls specified in @include, @exclude and @matches
directives in metadata blocks to be in the same form that
QUrl.toEncoded() returns. That is a punycoded domain and percent encoded
path and query. This seems to be what Tampermonkey on chrome expects to.
Also changes the scripts_for() function to take a QUrl arg so the caller
doesn't need to worry about encodings.
This regex was broken since the original PR and subsequent code seemed to be
working around it. Before re.split was returning [everything up to
/UserScript, everything else], now it returns [before UserScript, metadata,
after /UserScript], which is good.
Also I added the check for the UserScript line starting at column 0 as per
spec.
These argument type restrictions are mentioned on the greasespot pages for
these value storage functions. We could call JSON.dumps() instead but better
to push that onto the caller so we don't have to try handle deserialization.
Also removes the check for localstorage because everyone has supported that
for years.
WebEngine only. Previously we were just removing every script from the
main world. But some other scripts might got here in the future so new
we are overriding the name field to add a GM- prefix so hopefully we
only remove greasemonkey scripts before adding new ones.
For qtwebengine 5.8+ only. This is because as of 5.8 some greasemonkey
script support is in upstream. That is, qtwebenginescript(collection)
parses the greasemonkey metadata block and uses @include/match/exclude
to decide what sites to inject a script onto and @run-at to decide when
to inject it, which saves us the trouble. Notes on doing this in <5.8
are below.
Scripts are currently injected into the main "world", that is the same
world as the javascript from the page. This is good because it means
userscripts can modify more stuff on the page but it would be nice if we
could have more isolation without sacrificing functionality. I'm still
looking into why my more feature-full scripts are not having any effect
on the page while running in a separate world.
Userscripts are added to both the default and private profile because I
that if people have scripts installed they want them to run in private mode
too.
We are grabbing the scripts from the greasemonkey module, as opposed to
reading them directly from disk, because the module adds some GM_* functions
that the scripts may expect, and because that is used for webkit anyway.
I have code to support qtwebengine <5.8 but didn't because I am not
happy with the timing of some of the signals that we are provided
regarding page load state, and the actual load state. While the
difference between document-end and document-idle isn't so bad,
injecting document-start scripts when a urlChanged event is emitted
results in the script being injected into the environment for the page
being navigated away from. Anyway, if anyone wants this for earlier
webengines I can oblige them.
Supposed to be after all the assets have finished loading and in page js
has run. Not that we can garuntee that last bit. If a script misbehaves
because a precondition isn't yet met I suggest adding a defer method to
the script that adds a timer until the precondition is met.
Also changed the map/filter calls to use list comprehensions to keep
pylint happy. Even if it does look uglier.
WebKit backend only for now. Loads all .js files from a directory,
specified in the greasemonkey-directory key in the storage section,
defaulting to data/greasemonkey, and wraps them in a minimal environment
providing some GM_* functions. Makes those scripts available via the
"greasemonkey" registered object in objreg and injects scripts at appropriate
times in a page load base on @run-at directives.
I can't reproduce this, but someone on KDE reported always getting a crash (as
msg.splitlines()[0] gives an IndexError) when trying to select a file with
Qt 5.9.3.
If an editor is open on a form in a tab and that tab is closed, rewire
the callback to print a warning. Previously, the callback would access a
deleted C++ object and cause a crash.
Resolves#2758.
Updated requirements and adjusted the configuration in `.flake8`; other
files have been modified where the lack of per-file auto-ignore caused
problems, where putty's `# flake8: disable=` syntax could be replaced
with a simpler `noqa`, or where pylint directives already suppressed the
same error.
Instead of checking, if *any* qutebrowser process is running (which may or may not have an IPC socket where we expect it), simply launch a new instance *if socat fails*.
Which it does, if:
* the socket file doesn't exist (qutebrowser simply not running), or
* the socket isn't connectable (qutebrowser crashed, left orphaned socket)
Also put new instances into background, so the script behaves a bit more consistently. (Else it *sometimes* blocks and *sometimes doesn't*, when run.)
Sometimes when qutebrowser crashes, it leaves the IPC socket file behind. In those cases this script still tried to use it, failed, and didn't open your URL at all.
These links mostly occur within qutebrowser's documentation. But the
are also written to the auto generated config file. Clicking them in
any application that consults the desktop database (or uses xdg-open)
will thus open them in qutebrowser correctly.
Show an error message if the user edits the command such that it is
missing a start character (:, /, or ?). Previously, this would cause the
browser to crash.
Resolves#3326.
While 64-bit values are allowed with QtWebKit/QNetworkDiskCache, QtWebEngine
only allows 32-bit values here. With the updated sip's strict overflow checking,
that means we get an exception when setting a too big value.
:edit-command opens the current command line in an editor, and updates
the command line if the editor exits successfully. If --run is passed,
the command is executed when the editor exits sucessfully.
Resolves#2453.
If there are no quickmarks/bookmarks, hide the entire category in url
completion. Note that this only hides the category if
quickmarks/bookmarks is empty to begin with. An empty category is still
shown if the completion pattern filters out all items in that category.
See #960.
This adds Chrome/Chromium support to the importer (which ought to be the
last of these). Bookmarks are read from JSON, while keywords/search
engines (the same thing here) are read from the Web Data sqlite3
database, and converted from OpenSearch format.
importer: add tests for opensearch
This changes the undo stack from a list of UndoEntry objects to a list
of lists of UndoEntry objects, so groups of tabs can be added. Only
:tab-only does that, but it's exposed by TabbedBrowser.close_tab as a
keyword argument.
With subprocess.run, we wait until the subprocess has completed, which means the
parent process will hang on :restart.
Since we *don't* want to wait for the subprocess here, using subprocess.Popen
seems the right thing to do.
This was introduced in bb54a954fe / #3203
Probably doesn't affect #3210 and #3220.
By default, ctrl+c will yank the selection from the completion menu onto
the clipboard, and ctrl+shift+c will yank it onto the primary selection.
Unfortunately ctrl+y was already taken by rl-yank (which,
counter-intuitively to vim users, will paste the last deleted text).
Custom location installed castnow can't be killed with the command
`pkill -f /usr/bin/castnow`. Now recover the path to the binary in the path for
calling and killing the program.
When doing :unbind with a default keybinding the first time, it gets inserted
into bindings.commands with None as value.
When then doing :unbind a second time, instead of just leaving that None value
as-is, we removed it again (because it got treated as a custom binding).
Fixes#3162
This reverts commit acfb3aa26f.
The related code doesn't really belong in webenginesettings.py after all, and
for some reason I don't understand right now this breaks tests in
javascript.feature because window._qutebrowser is undefined when running them.
* Keep descriptions concise.
* Prefer starting descriptions with a noun.
* Don't explain that placeholders are placeholders/get replaced.
* Introduce placeholder list by the following line:
'The following placeholders are defined:'
* List placeholders in a markdown-style list:
'* `{<placeholder>}`: <description>.'
* for selections, use appropriate determiner/pro-adverb/pronoun:
* 'When to [...].'
* 'How to [...].'
* 'Which [...] to [...].'
* 'What [...] to [...].'
* available placeholders are defined for tabs.title.format
* tabs.title.format_pinned refers to that definition
* window.title.format repeats the identical(!) definition
* this replaces the repetion by another reference
We really just need to check that the row exists here, the date doesn't
matter. Checking the date here is actually flaky with regards to time.
When running locally at 11:50 EST, it failed with:
```
assert self._model.data(self._model.index(row, col)) == item
AssertionError: assert '1969-12-31' == '1970-01-01'
- 1969-12-31
+ 1970-01-01
```
It was wrong to assume that an atime of 0 would always format to
1970-01-01.
I previously removed the sorting logic from SortFilter thinking it was
unnecessary if we construct the model with a sorted list. However, this
only worked when no pattern was set, and the items are misordered as
soon as a pattern is input.
This patch reintroduces alpha-sorting, which can be disabled by passing
sort=False to the ListCategory constructor. The session completion test
had to be tweaked as it simulated the incorrect assumption that the
session list is not alpha-ordered; sessions come out of the
session-manager pre-sorted so we may as well use alpha-sorting in the
session completion model.
Resolves#3156.
This adds supports for the places.sqlite format as used by Firefox,
Seamonkey, Pale Moon, and presumably others. Search engine support is
limited to keyword-style '%s' functionality.
vulture whitelist for row_factory
This now works correctly in XML documents. The stylesheet is applied at
document creation to reduce flickering, and is updated if the
user_stylesheets setting is changed after page load.
The test needed to be fixed because of how the completer behaviour
changed.
Before:
completer always scheduled a completion update on selection changed,
but the updates themselves were ignored if not needed.
Now:
completer only schedules completion updates when actually needed, but
never ignores a completion update.
So, before it was correct to check whether `set_model` was called, now
we must check if the completion was actually scheduled. This can be
done by checking the parameters with which `_change_completed_part`
is called, since a completion is scheduled only when `immediate=True`
The command :edit-url --private (or :edit-url -p) will spawn a new
private window with the url input from the editor.
I had to add 'Given I have a fresh instance' to the feature file to
ensure tests were not interfering.
Resolves#3185.
When we did "Given I have a fresh instance", we immediately did run the commands
without waiting for the initial about:blank load again.
With Qt 5.10, this causes issues as the loadFinished signal is emitted with
about:blank then, and not the real URL.
See #3003
Instead of setting `_ignore_change` to `True` before calling
`_change_completed_part` we just stop `_cmd` from emitting
`update_completion`.
This has the nice side-effect that when writing a complete command
`_ignore_change` was set to `True` regardless, and thus hitting space
would not update the completion view.
Now, hitting space will (as always) schedule a completion update that
now will not be incorrectly ignored
The radius for the keyhint dialog box should be configurable vi via
c.keyhint.radius. The default was set to 6px, which is the previous
hardcoded value.
* Line breaks reinserted
* None in place of ''
* Check for browser before selecting default input format (to fix
KeyError)
* Remove redundant -S option and clarify help to make it slightly more
obvious what output formats make sense
* Added long-form arguments and slightly more sensible names (not really a
fix, but I personally like having them)
This restructures things to better support future implementations of
other input formats. The default formats are specified in a global dict
of browsers, which prevents duplicating the list of choices for browser
in bother get_args() and main(), and a new option enables overriding of
the default.
This allows importer.py to process Netscape HTML exports from Firefox
(and other Mozilla browsers) with three distinct types:
* bookmarks (sans shortcuturl attribute)
* keywords (bookmarks with a shortcuturl attribute)
* searches (keywords with a URL containing a %s substitution)
The first two can be combined at will in either quickmark or bookmark
output formats, the only difference being that keywords will be used in
place of titles when exporting to quickmark format. Searches are
exported to qutebrowser.conf format, or the new config.py format.
Dictionaries are used in the import function for readability's sake, but
the command line arguments follow the same general formula of true-false
flags used to select input bookmark types and the output format.
Added placeholders are:
* `{file}` has the same function as `{}`
* `{line}` is the 1-based line number
* `{column}` is the 1-based column number
* `{line0}` like `{line}`, but 0-based
* `{column0}` like `{column}` but 0-based
With the previous commit, we also checked that PyQt was >= 5.7.1, but we want to
support PyQt 5.7.0. Instead, we now check the individual components by hand.
Also, the previous check accidentally allowed PyQt >= 5.2.0 instead of 5.7.0.
With an older PyQt built against a newer Qt, we still don't have its features
available.
This also drops support for exact=True with compiled=True as the semantics for
that are unclear, and it's not used.
This also renames 'strict' to 'compiled' to be more descriptive.
It also fixes a crash when starting qutebrowser with an older compiled Qt
version which was introduced recently (calling setSpellCheckEnabled).
QtWebEngine emits scrollPositionChanged a lot during smooth scrolling, and
there's no reason we need to update percentages when they didn't *actually*
change.
This reduces the updates with a single spacebar press from 6-7 to 2-3 on my
machine, which might not be enough though.
See #2233
The keyhintwidget was not showing up when a keychain was prefixed with a
count. For example, 'g' would show a keyhint but '5g' would not. Now
keyhints are shown even when a count is given.
Resolves#3045.
Turns out --force is just in the way for most people, and at least for default
bindings it's easy to reset them.
Also, it makes :config-source fail when config.py contains keybindings.
Closes#3049
Ideally, we'd update all existing tables to add the new constraints, but sqlite
doesn't offer an easy way to do so: https://www.sqlite.org/lang_altertable.html
Since that migration really isn't worth the effort, we only set the constraint
for new tables...
Before, we allowed :set to take multiple values, which often lead to confusing
error messages when a user forgot to quote the value.
Now, we instead have a dedicated :config-cycle command for that.
See #1840, #2794
This renames SqlException to SqlError (to be more consistent with how Python
names exceptions), and adds an utility function which logs a few more useful
details about errors.
See #3004
Previously, indicator_padding was not taken into account, causing
problems when using a indicator_padding too small
Also removed icon padding to width calculation (seemed to be overestimating)
We removed various caches in b5eac744b5 but the
completion delegate stylesheet gets rendered a lot, causing things to slow down.
The rendering takes around 1ms, but it gets done ~10k times with a simple
profiling run, so that adds up quickly.
We don't use a functools.lru_cache here as the stylesheet template never
changes.
Thanks a lot to gilbertw1 for tracking this down!
See #2812 - there's probably more possible, but this should fix the performance
regression some people saw with the new config.
Previously, a successful import of the text history into sqlite would
move 'history' to 'history.bak'. If history.bak already existed, this
would overwrite it on unix and fail on windows.
With this patch, the most recently imported history is appended to
history.bak to avoid data loss.
Resolves#3005.
A few other options I considered:
1. os.replace:
- fast, simple, no error on Windows
- potential data loss
2. numbered backups (.bak.1, .bak.2, ...):
- fast, no data loss, but more complex
3. append each line to the backup as it is read:
- more efficient than current patch (no need to read history twice)
- potentially duplicate data if backup fails
This is done so config.py can import other python files in the config
directory. For example, config.py can 'import theme' which would load
a theme.py.
The previous path is restored at the end of this function, to avoid
tainting qutebrowser's path
Windows: The instructions are outdated and not really relevant anymore with the
standalone packages;
pip: Let's recommend tox/virtualenv by just linking to the install docs.
Closes#2998
There are just way too many gotchas related to valid modes, aliases, and
circular dependencies when validating aliases/bindings in the config.
Let's just remove this and let invalid commands fail late, when they're actually
used.
I considered introducing another list of deleted options (or a "deleted: True"
in configdata.yml), similar to what we had with the old config.
However, let's take the easier route and just delete everything we don't know
from configdata.yml. If someone edits it by hand, it's their fault :P
See #2772, #2847
The parsing bind() did manually is now available through CommandParser.
Resolves#2952.
This also adds a unit test for the case when there is no current
binding, as I broke that while working on this and there was no test to
catch it :)
Importing qtutils for version_check needs pkg_resources, so we need to check
that's available earlier.
Also, import pkg_resources also shows warnings on older setuptools versions
because of invalid escapes, so we need to import it with warnings suppressed.
Includes a test for persistence of intermediate mutations in a
configuration file (i.e. more than one update) and a switch of the
_mutable attribute in configurations to a dictionary of (old, new)
values rather than (name, old, new). get_obj() now checks for an
existing mutable value and returns a reference to that value, only
making an initial copy; this preserves changes between update_mutables()
We use fake_runtime_dir which simply patches XDG_RUNTIME_DIR for this test.
Since we patch QApplication.applicationName() during the tests, but standarddir
doesn't use that anymore, we get a different name.
Versions before v0.9.0 (which didn't even support hinting with QtWebEngine!)
used to write QtWebEngine data to:
~/.local/share/qutebrowser/qutebrowser/QtWebEngine/Default
~/.cache/qutebrowser/qutebrowser/QtWebEngine/Default
In v0.9.0 this was changed to:
~/.local/share/qutebrowser/webengine
~/.cache/qutebrowser/webengine
Now we don't try to migrate data from the old location anymore.
We were only rendering .html files before, so the old _guess_autoescape function
had the effect of always autoescaping .render() (from a file) but never
autoescaping .from_string(). However, most places using .from_string() actually
render (Qt-)HTML via jinja, so they should escape stuff!
Now, we always autoescape, except when the caller uses the
jinja.environment.no_autoescape() context manager, which places rendering
stylesheets now do.
This impacted:
- Confirm quit texts (no HTML here)
- config.py loading errors
(where this was found because of an error containing - a <keybinding>)
- Certificate error prompts
(should be fine from what I can tell, as the only user-controllable output is
the hostname, which cannot contain HTML)
This also changes qute://help to show the documentation generation error if a
help page wasn't found. This way, people who pull from git but not re-generate
the documentation hopefully get the idea.
Those now look at the history again.
Looking at the behavior in different applications:
- vim: History
- spacemacs: Completion if open, else history
- luakit: Completion if open, else history
- dwb: Always completion (has no history?)
- vimb: Nothing if completion open, else history
- vimperator: Always history
So this is consistent with at least some of them - the much more important
factor is that <Tab> is probably intuitively easy to discover if up/down doesn't
do what's expected, but <ctrl-p>/<ctrl-n> are not.
"Downloading with SSL errors" from downloads.feature still fails, but like this,
at least all other tests pass without the need to modify LD_LIBRARY_PATH.
We accidentally did show the command as a list in to_str(). However, after
correcting that to use shlex.escape, we got ugly qutebrowser command lines
when tabbing to the default value, because of how shlex handles double-escaping:
>>> print(shlex.quote("gvim -f '{}'"))
'gvim -f '"'"'{}'"'"''
While in this case, outputting "gvim -f '{}'" would be much more appropriate, it
doesn't look like we can teach shlex.quote to do that.
Instead, we now only accept a list as input for ShellCommand, at the price that
the user needs to do
:set editor.command '["gvim", "-f", "{}"]'
instead of
:set editor.command 'gvim -f {}'
Fixes#2962.
While the test worked again with eb4691adfc, it
broke again immediately because of 40ee89bddc.
With that fix in, the lower-case monospace in the set value was immediately
replaced by the full list of fonts again. With an upper-case Monospace, this
won't happen.
Fixes#2825, for real this time.
Should default to -1, not 1000 as the new history completion is better
able to handle large numbers of entries. I believe this was acidentally
reset to 1000 while fixing a merge conflict.
Also re-run src2asciidoc.
In python3.4, there is a circular dependency between the config module
and configmodel.bind. This is resolved by dependency injection. The
config/keyconfig instances are embedded in a struct passed to every
completion function, so the functions no longer depend on the modules.
This will also enable completion functions to access other previously
inaccessible info, such as the window id.
See #2814.
When upgrading from an old table that used different url formatting, two
entries might map to the same key, so we'll need to replace the previous
entry to avoid a primary key conflict.
Incrementing _USER_VERSION in the source will cause the
HistoryCompletion table to regenerate when users update.
This is currently necessary to support some recent formatting fixes, but
could be incremented again in the future for other changes.
When in miscmodels, the config module was unable to find the function.
It appears to be some sort of circular import issue:
```
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/app.py", line 44, in <module>
from qutebrowser.completion.models import miscmodels
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/miscmodels.py", line 24, in <module>
from qutebrowser.completion.models import completionmodel, listcategory, util
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/util.py", line 24, in <module>
from qutebrowser.config import config
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 223, in <module>
class ConfigCommands:
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 314, in ConfigCommands
@cmdutils.argument('command', completion=miscmodels.bind)
AttributeError: module 'qutebrowser.completion.models.miscmodels' has no attribute 'bind'
```
As configmodel imports util (and thereby config as well) it is unclear
to me why moving bind() to configmodel actually fixes this, but it does.
If the HistoryCompletion table is removed, regenerate it from the
History table. This allows users to manually edit History, then remove
HistoryCompletion to prompt regeneration.
See #2903.
Encode urls that are inserted into the history, but do not encode urls
for completion (other than removing passwords).
Also ensure that urls read from the history text file are formatted
consistenly with those added while browsing.
Fixes#2903.
Get qutebrowser to the point where it can at least start
- Declare _messages earlier in MessageView.__init__ so it is set before
the config trigger tries to access it.
- Remove unused configmodel completion functions
- Move bind completion to configmodel to avoid a circular import with
the config module
- Fix some config accesses (forgot to use .val)
- Fix old Completion.CompletionKind references
Instead, expect the data to be given in the desired order. Completion
functions should sort their data _if_ they want it sorted in the
completion. This has a few implications:
- {book,quick}marks appear in the same order they do in the text file.
This means users can rearrange their mark files for custom sorting.
Fixes#2354
- Sessions are sorted as they appear in the session manager
- Tabs are sorted numerically, not alphabetically (Fixes#2883)
Note that prefix-based filter sorting is still performed, so items
starting with the filter pattern come first.
The added/removed signals for the urlmark managers are no longer used as
the completion models are generated on-the-fly. The changed signal is
still needed so the save-manager knows when to trigger a write to disk.
Also removes session_manager.update_completion, which is no longer
needed for the same reason as above.
keyconf.changed cannot be removed, as it is still wired up to
basekeyparser.
Resolves#2874.
Fixes#2868, where pressing <shift-tab> then <ctrl-d> in history
completion (with > 256 items) would cause later items to disappear (and
cause a crash if you try to delete again).
Cause:
Scrolling to the bottom would fetch an additional 256 items (in addition
to the 256 that are fetched at first). Deleting causes the query to
re-run, but it only fetches the initial 256 items, so the current index
is now invalid.
Fix:
After deleting from the history category, call fetchMore until it has
enough rows populated that the current index is valid.
When tabbing to the last index of history completion, call expandAll
which will call fetchMore to retrieve more query results, if available.
Calling fetchMore directly will not update the view, and for some
reason self.expand(idx.parent()) and
self.expand(self.model().index(idx.row(), 0)) did not work, so I'm using
expandAll.
Fixes#2841.
When a key is bound to a command line that includes one or more
arguments to a command, bind completion should show the whole command
for the "Current" category, and use only the command name to look up the
description.
Fixes#2859, where a crash was caused by looking up the description by
the full command text rather than just the name.
Sometimes, we get another error with "Renderer process was killed" and the data:
URL for the error page. This is probably because the renderer process wasn't
restarted yet. This hopefully helps.
Override removeRows instead of removeRow.
> removeRow is not virtual in C++, so if this gets called by Qt
> internally for some reason, it wouldn't use the overloaded version -
> so I think it'd be better to implement removeRows and then use
> removeRow without overloading that
- The-Compiler
It doesn't make sense to have an active selection while you are
filtering by entering text. You should be in one of two states:
1. Tabbing through completions (valid selection)
2. Entering a filter pattern (invalid selection)
Fixes#2843, where a crash would occur after the following:
1. tab to an item other than the first
2. <backspace>
3. re-type last character
4. <ctrl-d>
This would try to delete an out of range index.
Even though no item was deleted, it was manipulating the completion
model because beginRemoveRows was called before the exception was
raised.
This fixes that problem by moving the removal logic (and delete_func
check) into the parent model, so it can check whether deletion is
possible before calling beginRemoveRows.
Fixes#2839.
We get no last_atime limit at all otherwise:
qutebrowser.misc.sql.SqlException: Failed to prepare query "SELECT url, title,
strftime('%Y-%m-%d', last_atime, 'unixepoch', 'localtime') FROM
CompletionHistory WHERE (url LIKE :pat escape '\' or title LIKE :pat escape '\')
AND last_atime >= ORDER BY last_atime DESC": "near "ORDER": syntax error Unable
to execute statement"
We need to tell sqlite to convert the timestamps to localtime during
formatting, otherwise it formats them as though you are in UTC.
Also fix up a few uses of mktime.
For some reason, calling search.clear() while no search is displayed causes the
backends to un-focus inputs, and with QtWebKit, even hinting can't focus them
again after that.
For performance, re-introduce web-history-max-items.
As the history query has now become a very specific multi-part query and
history completion was the only consumer of SqlCategory, SqlCategory is
now replaced by a HistoryCategory class.
The javascript history page was requesting the new start_time in ms, but
the python code was expecting seconds. This is fixed by removing all the
millisecond translations in the python code and only translating to
milliseconds in the javascript code that formats dates.
- Fix flake8
- history.clear should also clear completion table
- call _resize_columns in set_model, not set_pattern
- add more unit-testing for the history completion table
- Ignore invalid variable name in flake8 (pylint already checks this and
we don't want to have to double-ignore)
- Fix and test completion bug with `:set asdf `
- Remove unused import
- Use `assert not func.called` instead of `func.assert_not_called` for
backwards compatibility
- Fix comment and empty line check in _parse_entry
- connect layoutAboutToBeChanged signal
- assert sort_order is None if sort_by is None
- modify sql init failure message to ask about Qt sqlite support.
Check for interval being positive instead of checking for it to be
non-zero. So if somehow some unexpected thing happend and made
message-timeout negative, the bug doesn't cascade.
- use mocker.Mock instead of mock.Mock to avoid an extra import
- attach model to validator sooner so it can validate changes in the
model during the test
- remove outdated comment
- fix sql init error message
- clean up history text import code
- fix test_history file path in coverage check
- use real web history, not stub, for completion model tests
- use qtmodeltester in sql/list_category tests
- test url encoding in history tests
- fix test_clear by using a callable mock
- remove test_debug_dump_history_oserror as the check is now the same as
for the file not existing
- rename nonempty to data in test_completionmodel
- add more delete_cur_item tests
- test empty option/value completion
If creating the sql database fails, show an error dialog assuming sqlite
is not installed.
This removes the isDriverAvailable check as it was true even with sqlite
uninstalled.
sql.version now inits itself if sql is not already initialized and
prints 'UNAVAILABLE (<error message>)' if init fails. This is to avoid
cascading errors, where one error would create a crash dialog, which
calls sql.version, which would create another error.
This is also needed to make the docs environment work on Travis - as otherwise,
doc generation wasn't deterministic because of changing dict key order.
In Dict.to_str() and List.to_str() we use json.dump to get a value. However,
JSON includes surrogate escapes in the dumped values, which breaks round trips.
>>> yaml.load(json.dumps({'\U00010000': True}))
{'\ud800\udc00': True}
>>> yaml.load(json.dumps({'\U00010000': True}, ensure_ascii=False))
yaml.reader.ReaderError: unacceptable character #x10000: special characters are not allowed
See:
https://stackoverflow.com/a/38552626/2085149https://news.ycombinator.com/item?id=12798032
This required some changes on how URLs are handled during those tests. Before,
we simply could return a path and (since we had a patched QNAM), nobody
complained.
Now this actually needs to be a valid URL, so we use
https://www.example.com/path everywhere instead.
Now that the StyleSheetObserver is a child of the object it observes, it should
get cleaned up properly when the object is deleted.
This means this is hopefully not needed anymore, even on Qt 5.2.
This gets called a lot, and caused some :bind calls to take ~3s.
Stats after starting with a bit of :bind:
CacheInfo(hits=25917, misses=139, maxsize=256, currsize=139)
Now the "object" kind of value (like in YAML) is stored internally, and that's
the canonical value. The methods changed their meaning slightly, see the
docstring in configtypes.py for details.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
related, it could be webengine.
Show a graphical error box with install instructions if PyQt.QtSql is
not found, rather than failing with CLI errors. Also show an error box
if the sqlite driver is not available.
This ensures we actually know when an AttributeError happens.
It also changes most external code to use the correct environment, rather than
simply creating a jinja2.Template, which wouldn't use the more tightened
environment.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
With the new completion API, we no longer need a custom filterAcceptsRow
function. This was necessary to handle the tree structure of the model,
but now we use a separate QSortFilterProxyModel for each category, so
the data it filters is flat. We can simplify the code by using the
builtin setFilterRegExp.
This changes the behavior a little, as now all list categories filter on
all columns. This should be beneficial if anything. For example, help
topics are now filtered on description in addition to name.
This also seems to slightly speed up filtering, according to the url
model benchmark.
Before:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.2806 1.3817 1.3195 0.0390 1.3068 0.0487 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
After:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.1183 1.1508 1.1281 0.0132 1.1241 0.0142 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
Taking the completion widget as an argument was overly complex.
The process now looks like:
1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.
This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.
The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.
The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
For QSqlQueryModel, the argument should always be an invalid index:
http://doc.qt.io/qt-5/qsqlquerymodel.html#canFetchMore
For a QStandardItemModel, it doesn't matter. Either way, passing the
top-level parent index was wrong.
_insert_query gets called with a query and dict of values such as:
{'val': 1, 'lucky': False, 'name': 'one'}
Via bindValues(), we only assign a placeholder in the query string to a value,
so we get a query with bindings like:
INSERT INTO Foo values(:lucky,:val,:name)
{':name': 'one', ':val': 1, ':lucky': False}
So what we're executing is something like:
INSERT INTO Foo values(false,1,"one")
However, if the column order in the database doesn't happen to be the order
we're passing the values in, we get the wrong values in the wrong columns.
Instead, we now do:
INSERT INTO Foo (lucky, val, name) values(false,1,"one")
Which inserts the values in the order we intended.
With Python 3.6, this just happened to work before because we always passed the
keyword arguments in the table column order, and in 3.6 dicts
(and thus **kwargs) happen to be ordered:
https://mail.python.org/pipermail/python-dev/2016-September/146327.html
On Travis CI we are sometimes seeing:
```
CompletionView.selection_changed[str].emit():
argument 1 has unexpected type 'int'
```
Cast the data to a string before emitting it just to be safe.
This allows replace to be a named parameter and allows consolidating
some duplicate code between various insert methods.
This also fixes some tests that broke because batch insert was broken.
No longer needed with sql backend. Query results build their own
namedtuple from the returned columns, and inserting new entries is just
done with named parameters.
Fix the issue where pressing `o<esc>o` would show a url completion
dialog where attempting to <Tab> select items would do nothing but show
a Qt warning.
The fix is to ensure we set _last_completion_func to None whenever we
clear completion (there was a case I missed).
It also ensures we always delete the old model and adds a safety to
prevent deleting an in-use model is set_model is called with the current
model.
This was changed during code review but was causing Qt errors while
TAB-completing in the selection view:
08:42:34 WARNING qt Unknown module:none:0 Can't select indexes from different model or with different parents
before
------
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.072 user 0.063334 sys 0.010000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SCAN TABLE History
0|0|0|USE TEMP B-TREE FOR ORDER BY
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory
0|0|0|USE TEMP B-TREE FOR ORDER BY
after
-----
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.000 user 0.000000 sys 0.000000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SEARCH TABLE History USING INDEX AtimeIndex (atime>? AND atime<?)
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory USING INDEX CompletionAtimeIndex
When loading heise.de, for some crazy reason QtWebKit calls historyContains
about 16'000 times.
With this cache (which we simply clear when *any* page has been loaded, as then
the links which have been visited can change), that's down to 250 or so...
- Show an error message when import fails, not a generic crash dialog
- Raise CommandError when debug-dump-history fails
- Check that the path exists for debug-dump-history
This is called often, hopefully a prepared query will speed it up.
This also modifies Query.run to return self for easier chaining, so you
can use `query.run.value()` instead of `query.run` ; query.value()`.
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.
It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.
Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
The history completion query is extended to pick only the most recent item for
a given url.
The tests in test_models now check for ordering of elements.
The old implementation was looping through the whole history list, which for
SQL was selecting every row in the database. The history benchmark was taking
~2s. If this is rewritten as a specialized SQL query, the benchmark takes
~10ms, an order of magnitude faster than the original non-SQL implementation.
Vulture exposed the following dead code:
- AppendLineParse was only used for reading the history text file, which is now
a sql database (and the import code for the old text file is simpler and does
not need a complex line parser)
- async_read_done is no longer used as importing the history text file is
synchronous (and should only happen once)
- config._init_key_config is unused as it was moved to keyconf.init
Calling sql.init() in version.version() would replace the existing sql
connection and cause a crash when accessed by opening qute://version.
Now version relies on sql already being initted, and app.py inits sql early if
the --version arg is given.
Two history end2end tests are failing because sqlite is not flushing to disk in
time to be read by the test process. My understanding is that sqlite should
take an exclusive lock while writing, so it is difficult to understand why this
is happening. This can be fixed by adding a delay, but that seems flaky.
I'm fixing it by checking qute://history instead of reading the database file.
See:
https://github.com/qutebrowser/qutebrowser/pull/2295#issuecomment-292786138
and the following discussion.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
Instead of skipping bad history lines during the import to sql, fail hard. We
don't want to delete the user's old history file if we couldn't parse all of
the lines.
Now that sql is only used for history (not quickmarks/bookmarks) a number of
functions are no longer needed. In addition, primary key support was removed as
we actually need to support multiple entries for the same url with different
access times. The completion model will have to handle this by selecting
something like (url, title, max(atime)).
This also fixes up a number of tests that were broken with the last few
sql-related commits.
If qutebrowser detects a history text file when it starts
(~/.local/share/qutebrowser/history by default on Linux), it will import this
file into the new sqlite database, then delete it.
The read is done as a coroutine as it can take some time.
Instead of reading sqlite history from a file and storing it in an in-memory
database, just directly use an on-disk database. This resolves#755, where
history entries don't pop in to the completion menu immediately as they are
still being read asynchronously for a few seconds after the browser starts.
Deleting a history entry should do nothing, but we want a test to ensure this
and get 100% branch coverage for urlmodel.
This also un-skips the bookmark/quickmark tests.
Instead of add_list and add_sqltable, the completion model now supports
add_category, and callees either pass in a SqlCategory or ListCategory. This
makes unit testing much easier.
This also folds CompletionFilterModel into the ListCategory class.
The RFC on moving from plaintext to SQL storage (#2340) showed that many would
be upset if bookmarks and quickmarks were no longer stored in plaintext.
This commit uses list-based completion for quickmarks and bookmarks. Now the
history storage can be moved from plaintext to an on-disk SQL database while
leaving bookmarks and quickmarks as-is.
Now all completion models are of a single type called CompletionModel.
This model combines one or more categories. A category can either be a
ListCategory or a SqlCategory.
This simplifies the API, and will allow the use of models that combine simple
list-based and sql sources. This is important for two reasons:
- Adding searchengines to url completion
- Using an on-disk sqlite database for history, while keeping bookmarks and
quickmars as text files.
This was a performance optimization that shouldn't be needed with the new SQL
history backend. This also removes support for the LIMIT feature from SqlTable
as it only existed to support web-history-max-items.
Respond to the low-hanging code review fruit:
- Clean up some comments
- Remove an acidentally added duplicate init_autosave
- Combine two test_history tests
- Move test_init cleanup into a fixture to ensure it gets called.
- Name the _ argument of bind(_) to _key
- Ensure index is valid for first_item/last_item
- Move SqlException to top of module
- Rename test_index to test_getitem
- Return QItemFlags.None instead of None
- Fix copyright dates (its 2017 now!)
- Use * to force some args to be keyword-only
- Make some returns explicit
- Add sql to LOGGER_NAMES
- Add a comment to explain the sql escape statement
newest_slice is no longer needed after the completion refactor. Now that
history is based on the SQL backend, LIMIT is used instead.
StatusBar._option is not used, though I'm not sure why vulture only caught it
now.
The new function-based completion API introduced a circular import:
config -> keyconf -> miscmodels -> config.
config only depended on keyconf so it could initialize it as part of
config.init. This can be resolved by moving this to keyconf.init and
initializing keyconf as part of app.init.
test_history.test_init also leaked state by leaving the instantiated history as
the parent of the QApp, which was causing test_debug to fail because it was
trying to dump the history object left from test_history.
test_selectors and test_get_all_objects were running fine on my machine, but
for some reason is failing with "Driver not loaded" on Travis. Let's try
initializing SQL and see what happens.
Change the logging to report the completion function name and have the end2end
tests check for this.
Remove the tests for realtime completion, as it was decided this is not an
important feature and the code is much simpler without it.
This just forwards canFetchMore and fetchMore to the underlying tables.
It seems to be returning True and fetching in some cases (with a large
history), so I guess it is useful?
Instead of returning a regular tuple and trying to remember which index maps to
which field, return named tuples that allow accessing the fields by name.
Allow categories to specify a WHERE clause that applies in addition to the
pattern filter. This allows the url completion model to filter out redirect
entries.
This also fixed the usage of ESCAPE so it applies to all the LIKE statements.
A SQL completion category can now provide a customized column expression for
the select statement. This enables the url model to format timestamps, as well
as rearrange the name and url in the quickmark section.
This allows setting the query as a QSqlQuery instead of a string, which allows:
- Escaping quotes
- Using LIMIT (needed for history-max-items)
- Using ORDER BY (needed for sorting history)
- SELECTing columns (needed for quickmark completion)
- Creating a custom select (needed for history timestamp formatting)
From @TheCompiler:
To expand on this: I think it's fine to use KeyError on a lower level, i.e.
with the SqlTable object with a dict-like interface. However, on this higher
level, I think it makes sense to re-raise them as more specific exceptions.
- Adjust _check_completions to work for CompletionModel and SqlCompletionModel
- Move sql initialization into a reusable fixture
- Remove the bookmark/quickmark/history stubs, as they're now handled by sql
- Disable quickmark/bookmark model tests until their completion is ported to
sql.
- Disable urlmodel tests for features that have to be implemented in SQL:
- LIMIT (for history-max-items)
- Configurable column order (for quickmarks)
- Configurable formatting (for timestamp-format
SQL is included in the Archlinux pyqt5 package, but not in Debian.
We need this so the debian-based CI builds will pass with the new
sql-based completion implementation.
For URL completion, time-based sorting is handled by the SQL model.
All the other models use simple alphabetical sorting. This allowed cleaning up
some logic in the sortfilter, removing DUMB_SORT, and removing the
completion.Role.sort.
This also removes the userdata completion field as it was only used in url
completion and is no longer necessary with the SQL model.
Store quickmarks and bookmarks in an in-memory sql database instead of a
python dict. Long-term storage is not affected, bookmarks and
quickmarks are still persisted in a text file.
The added and deleted signals were removed, as once sql completion
models are used the models will no longer need to update themselves.
This will set the stage for SQL-based history completion.
See #1765.
The browser-wide in-memory web history is now stored in an in-memory sql
database instead of a python dict. Long-term storage is not affected, it
is still persisted in a text file of the same format.
This will set the stage for SQL-based history completion.
See #1765.
When qutebrowser starts, it creates an in-memory sqlite database. One
can instantiate a SqlTable to create a new table in the database. The
object provides an interface to query and modify the table.
This intended to serve as the base class for the quickmark, bookmark,
and history manager objects in objreg. Instead of reading their data
into an in-memory dict, they will read into an in-memory sql table.
Eventually the completion models for history, bookmarks, and quickmarks
can be replaced with SqlQuery models for faster creation and filtering.
See #1765.
The new completion API no longer needs either of these. Instead of
referencing an enum member, cmdutils.argument.completion now points to
a function that returnsthe desired completion model.
This vastly simplifies the addition of new completion types. Previously
it was necessary to define the new model as well as editing usertypes
and completion.models.instances. Now it is only necessary to define a
single function under completion.models.
This is the next step of Completion Model/View Revamping (#74).
There was a circular import from
config -> keyconf -> miscmodels -> config.
This is resolved by scoping config's keyconf import to the one function
that uses it.
First step of Completion Model/View revamping (#74). Rewrite the
completion models as functions that each return an instance of a
CompletionModel class.
Caching is removed from all models except the UrlModel. Models other
than the UrlModel can be generated very quickly so caching just adds
needless complexity and can lead to incorrect results if one forgets to
wire up a signal.
2017-06-19 07:42:12 -04:00
466 changed files with 30419 additions and 23525 deletions
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at mail@qutebrowser.org. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
* `storage -> offline-storage-database` (merged into `storage -> local-storage`)
Fixed
~~~~~
@@ -485,7 +820,7 @@ Fixed
- Fix crash when pressing enter without a command
- Adjust error message to point out QtWebEngine is unsupported with the OS
X .app currently.
- Hide Harfbuzz warning with the OS X .app
- Hide Harfbuzz warning with the macOS .app
v0.8.0
------
@@ -783,7 +1118,7 @@ Added
- New `:fake-key` command to send a fake keypress to a website or to
qutebrowser.
- New `--mhtml` argument for `:download` to download a page including all
ressources as MHTML file.
resources as MHTML file.
- New option `tabs -> title-alignment` to change the alignment of tab titles.
Changed
@@ -848,7 +1183,7 @@ Fixed
- Fixed scrolling to the very left/right with `:scroll-perc`.
- Using an external editor should now work correctly with some funny chars
(U+2028/U+2029/BOM).
- Movements in caret mode now should work correctly on OS X and Windows.
- Movements in caret mode now should work correctly on macOS and Windows.
- Fixed upgrade from earlier config versions.
- Fixed crash when killing a running userscript.
- Fixed characters being passed through when shifted with
@@ -923,7 +1258,7 @@ Changed
- The completion widget doesn't show a border anymore.
- The tabbar doesn't display ugly arrows anymore if there isn't enough space
for all tabs.
- Some insignificant Qt warnings which were printed on OS X are now hidden.
- Some insignificant Qt warnings which were printed on macOS are now hidden.
- Better support for Qt 5.5 and Python 3.5.
Fixed
@@ -983,7 +1318,7 @@ Added
- New argument `--no-err-windows` to suppress all error windows.
- New arguments `--top-navigate` and `--bottom-navigate` (`-t`/`-b`) for `:scroll-page` to specify a navigation action (e.g. automatically go to the next page when arriving at the bottom).
- New flag `-d`/`--detach` for `:spawn` to detach the spawned process so it's not closed when qutebrowser is.
- New flag `-v`/`--verbose` for `:spawn` to print informations when the process started/exited successfully.
- New flag `-v`/`--verbose` for `:spawn` to print information when the process started/exited successfully.
- Many new color settings (foreground setting for every background setting).
- New setting `ui -> modal-js-dialog` to use the standard modal dialogs for javascript questions instead of using the statusbar.
- New setting `colors -> webpage.bg` to set the background color to use for websites which don't set one.
@@ -1034,7 +1369,7 @@ Fixed
- Fixed AssertionError when closing many windows quickly.
- Various fixes for deprecated key bindings and auto-migrations.
- Workaround for qutebrowser not starting when there are NUL-bytes in the history (because of a currently unknown bug).
- Fixed handling of keybindings containing Ctrl/Meta on OS X.
- Fixed handling of keybindings containing Ctrl/Meta on macOS.
- Fixed crash when downloading a URL without filename (e.g. magnet links) via "Save as...".
- Fixed exception when starting qutebrowser with `:set` as argument.
- Fixed horrible completion performance when the `shrink` option was set.
@@ -1132,7 +1467,7 @@ Changed
- Add a `:search` command in addition to `/foo` so it's more visible and can be used from scripts.
- Various improvements to documentation, logging, and the crash reporter.
- Expand `~` to the users home directory with `:run-userscript`.
- Improve the userscript runner on Linux/OS X by using `QSocketNotifier`.
- Improve the userscript runner on Linux/macOS by using `QSocketNotifier`.
- Add luakit-like `gt`/`gT` keybindings to cycle through tabs.
- Show default value for config values in the completion.
- Clone tab icon, tab text and zoom level when cloning tabs.
@@ -1152,7 +1487,7 @@ Changed
* `init_venv.py` and `run_checks.py` have been replaced by http://tox.readthedocs.org/[tox]. Install tox and run `tox -e mkvenv` instead.
* The tests now use http://pytest.org/[pytest]
* Many new tests added
* Mac Mini buildbot to run the tests on OS X.
* Mac Mini buildbot to run the tests on macOS.
* Coverage recording via http://nedbatchelder.com/code/coverage/[coverage.py].
* New `--pdb-postmortem argument` to drop into the pdb debugger on exceptions.
* Use https://github.com/ionelmc/python-hunter[hunter] for line tracing instead of a selfmade solution.
@@ -1288,7 +1623,7 @@ Fixed
* Fix rare exception when a key is pressed shortly after opening a window
* Fix exception with certain invalid URLs like `http:foo:0`
* Work around Qt bug which renders checkboxes on OS X unusable
* Work around Qt bug which renders checkboxes on macOS unusable
* Fix exception when a local files can't be read in `:adblock-update`
* Hide 2 more Qt warnings.
* Add `!important` to hint CSS so websites don't override the hint look
@@ -1324,7 +1659,7 @@ Changes
* Set zoom to default instead of 100% with `:zoom`/`=`.
* Adjust page zoom if default zoom changed.
* Force tabs to be focused on `:undo`.
* Replace manual installation instructions on OS X with homebrew/macports.
* Replace manual installation instructions on macOS with homebrew/macports.
* Allow min-/maximizing of print preview on Windows.
http://wiki.qt.io/QtWebEngine[QtWebEngine] by default (and supports
http://wiki.qt.io/QtWebKit[QtWebKit] optionally). QtWebEngine is based on
Google's https://www.chromium.org/Home[Chromium]. With an up-to-date Qt, it has
much more man-power behind it than WebKitGTK+ has, and thus supports more modern
web features - it's also arguably more secure.
What's wrong with https://www.mozilla.org/en-US/firefox/new/[Firefox] and link:http://5digits.org/pentadactyl/[Pentadactyl]/link:http://www.vimperator.org/vimperator[Vimperator]?::
Firefox likes to break compatibility with addons on each upgrade, gets
@@ -125,9 +129,9 @@ When using quickmark, you can give them all names, like
without having to remember the exact website title or address.
How do I use spell checking?::
Qutebrowser's support for spell checking is somewhat limited at the moment
(see https://github.com/qutebrowser/qutebrowser/issues/700[#700]), but it
can be done.
Configuring spell checking in qutebrowser depends on the backend in use
(see https://github.com/qutebrowser/qutebrowser/issues/700[#700] for
a more detailed discussion).
+
For QtWebKit:
@@ -145,15 +149,14 @@ For QtWebKit:
+
For QtWebEngine:
. Not yet supported unfortunately :-( +
Adding it shouldn't be too hard though, since QtWebEngine 5.8 added an API for
If you are on a Debian or Ubuntu based system, you might experience some crashes
visiting these sites. This is caused by various bugs in Qt which have been
fixed in Qt 5.4. However Debian and Ubuntu are slow to adopt or upgrade
some packages. On Debian Jessie, it's recommended to use the experimental
repos as described in https://github.com/qutebrowser/qutebrowser/blob/master/INSTALL.asciidoc#on-debian--ubuntu[INSTALL].
When using QtWebEngine, qutebrowser reports "Render Process Crashed" and the console prints a traceback on Gentoo Linux or another Source-Based Distro::
As stated in https://gcc.gnu.org/gcc-6/changes.html[GCC's Website] GCC 6 has introduced some optimizations that could break non-conforming codebases, like QtWebEngine. +
As a workaround, you can disable the nullpointer check optimization by adding the -fno-delete-null-pointer-checks flag while compiling. +
On gentoo, you just need to add it into your make.conf, like this: +
* There's also a https://www.shortcutfoo.com/app/dojos/qutebrowser[free training
course] on shortcutfoo for the keybindings - note that you need to be in
insert mode (i) for it to work.
* Run `:adblock-update` to download adblock lists and activate adblocking.
* If you just cloned the repository, you'll need to run
`scripts/asciidoc2html.py` to generate the documentation.
* Go to the link:qute://settings[settings page] to set up qutebrowser the way you want it. (Currently not available with the QtWebEngine backend and on the OS X build - use the `:set` command instead)
* Go to the link:qute://settings[settings page] to set up qutebrowser the way you want it.
* Subscribe to
https://lists.schokokeks.org/mailman/listinfo.cgi/qutebrowser[the mailinglist] or
^Geoclue error: Process org\.freedesktop\.Geoclue\.Master exited with status 127
^QDBusConnection: name 'org.freedesktop.Geoclue.Master' had owner '' but we thought it was ':1.1'
^QObject::connect: Cannot connect \(null\)::stateChanged\(QNetworkSession::State\) to QNetworkReplyHttpImpl::_q_networkSessionStateChanged\(QNetworkSession::State\)
^QXcbClipboard: Cannot transfer data, no data available
^load glyph failed
^Error when parsing the netrc file
^Image of format '' blocked because it is not considered safe. If you are sure it is safe to do so, you can white-list the format by setting the environment variable QTWEBKIT_IMAGEFORMAT_WHITELIST=
^QPainter::end: Painter ended with \d+ saved states
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
