Quick Search, Advanced Search: "OR" and (explicit) "AND" not working

[kinook]

It's working as expected in my test.

Please send the info from https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

So this is the last residual from my (allegedly DLL) disaster which I discovered on Christmas, even after a re-install which I had assumed to be "complete" finally; I'll further check the registry then.

[Spliff]

I can't resolve the problems I have with UR Search.

I de-installed UR by the UR de-install run, then rebooted and deleted any residuals I could find (Wise Care 365 and Voidtools Everything), then did a fresh UR install, left it at its factory settings, then tried my UR Search again, with the same (faulty / "no-") results as before. (I then only re-installed my UR settings: registry key, and the 4 UR settings files.)

I then installed UR on a second PC, with UR's factory settings (and left it at that), and I encounter strictly the same problems there; on that PC, AHK has never even been installed, let alone run, and it was just shortly connected to the internet 2 or 3 times, just in order to update Windows 10 from the MS site (and it was never connected to my "net" (LAN), not even to my printer, all software installs, except for W10 which came with the PC, by USB stick), and no problem whatsoever (except now for the UR Search), whilst my main PC had had some c: drive problems indeed (as described by me here, but no other problem with dozens of programs though, except for UR).

Thus it's obvious that my UR Search problems are not caused by problems on either of my PCs but by some setting or settings interaction; Tools - Options - Search is:
General: Automatically no (default is yes, same result), Select first no, Focus yes, Highlight no
Quick Search (QS): Incremental 0ms, Max 0
Then: Enable no, Perform no, Match no.
(Compact-and-Repair settings: Compact yes, Repair yes, Enable yes.)

Your help page "Matches Wildcard" (to be accessed by searching "matches wildcard" in the help's search field) states,
"The Matches Wildcard [Does Not Match Wildcard] comparison operator locates Info Items or Info Items with the specified Attribute that match the specified wildcard expression." - from my understanding as a non-native speaker, this means that ANY "wildcard operator" (i.e. plural) (i.e. ? or * or the regex [some] or the regex [^some], ditto for [some-some]) will trigger, when entered in QS, the respective "matches wildcard" AS (AdvancedSearch), whilst in all cases of the absence of these ?,*,[,] (or their "escaping" by [] or a second [/]), respectively) = in all "default cases", the QS will trigger the default "contains keywords" AS; a quick toggle between "Quick" and "Advanced" would confirm this switching between the two "Comparison" alternatives, i.e. would show that the "Comparison" column value / setting / choice in the AS "list" (always just one row here, for those QSs, "translated to AS") switched accordingly; your
"A Quick Search uses an (Item) Contains Keywords <keywords entered> type of criteria, unless one of the wildcard operators, or the " (double quote) symbol is included in the criteria, in which case a Matches Wildcard criteria will be used instead.", would confirm that I have correctly understood that somewhat not-too-evident "The Matches Wildcard [Does Not Match Wildcard] comparison operator" part: Any ? or * within the QS is understood as a wildcard operator, except when it's escaped by [] around it, and similar for [] when not escaped by doubling those.

Thus, the UR code analyzes the QS string, in order to decide which AS to apply, so far so good, and I verified by toggling from QS to AS; also, any space is "translated" as (implicit) AND, and any OR (surrounded by spaces) as (explicit) "OR".

Unfortunately, this "translation" from entered string to search command does not work correctly on both of my systems, but equally faulty:

1) (As already said above,) OR is not translated, but invariably considered as being literal (being written or or OR), so some OR other will neither find some nor other items, but only items comprising some AND or AND other; ditto (also said above) for explicit AND: some AND other will NOT find items with some AND other, but only items with some AND other AND and.

2) * is correctly translated, i.e. when the search string contains a *, then the AS switches from default "contains keywords" to "matches wildcard"; this is not also the case with everything else: AS is not switched accordingly, but remains at "contains keywords", with, as value, the literal string, e.g.:

A) a?b > contains keywords: a?1
B)(when I then, in AS, manually switch to "matches wildcard", and click on start, the AS does NOT find any ab1 or ac1 item, which are clearly there, but "0 matching items")

A) a[a-z]1 > contains keywords: a[a-z]1 > and no find, obviously
B) manual switch to "matches wildcard" > no find either, e.g. for ab1 (which exists)

etc. etc.

Thus, we have TWO / three problems here, at obviously TWO different positions within UR's code:

A) QS search string is not correctly translated to AS search (switch to "matches wildcard" is not made when necessary, i.e. when ? or [] BUT is made when * though; obviously, a one OR two would not need that switch to begin with):
* only is processed correctly

B2) Even the correct AS search then (triggered manually, with "Comparison" as "match wildcard") will not find any result since here again, ? and [] are interpreted literally, not as wildcard/regex codes
here again, * only is processed correctly

B1) Similar for correct AS search:
contains keywords: a1 OR a2 > the OR is processed literally, thus no search result


THUS, MORE CONCISE:

Two problems:

AND and OR systematically processed literally (in QS and in AS, both in lowercase and uppercase), so no search result

? and [] systematically processed literally (in QS and in AS), thus NO switch to "match wildcard" when necessary, and no search result even with "match wildcard" (just * processed correctly)

That, on old PC and new PC, with old UR and UR from March, 2024.

Obviously, within UR's code, there are glitches / interactions (persistent over the UR versions) which on two of my systems (1 perhaps to be considered "problematic", the other one certainly not) cause the above-described problems, even whilst obviously, on your system(s), they do not.



I now have de-installed (with the UR de-install program) UR from PC 2, incl. the user settings (so reverted to trial). I re-installed (newest) UR (default settings) on PC 2, and I created a NEW trial .urd (not using the UR default (i.e. "sample") .urd I had used to do my previous Search tries - never used UR otherwise on PC 2 anyway).

I created two notes, One and Two.

Screenshot 1 shows "One OR Two" not working in QS (screenshot shows the automatic translation to AS where NO switch to "matches wildcard" would have been necessary, but where none of the two to-be-expected search results shows up; ditto for the search "one OR two"; ditto (i.e. NO result) for the search "eins OR zwei", considering that in the "text" of the two items, I had put "eins" and "zwei", respectively (always without the double-quotes), but a screenshot showing those texts AND the (empty) search result at the same time does not seem to be possible.

Screenshot 2 shows "O?e" not working in QS (screenshot shows the automatic translation to AS where the necessary switch to "matches wildcard" has NOT been made, and the lack of result, whilst the "One" item should have been listed).

Since my UR installation of PC 2 is "factory", without any setting, or any possibility of components to be damaged, it seems there is a glitch for OR, (explicit) AND, ? and [] in UR, showing on SOME system, not on others (and unfortunately on both of mine), whilst implicit AND and * are correctly processed on any system.

(Needless to say that the "?" placeholder would be needed for "tags" in the form of e.g. xab where x is the tag code, a the value (here) for a first tag category and b the value (here) for a second tag category (and so on): searching for xa* works fine for "a" then "any", but ? would be needed for "any" then "b".)

(I also tried 2 items, with xxxa1 and xxxb1, respectively; I then searched for xxx?1 and for xxx[?]1 (considering that "literal vs. placeholder" for "?" might have been mixed up); for both, I got the same 5 (faulty) "xxx" results (since they just contain xxx), but none of the 2 needed ones, and, needless to say, the necessary switch from "contains keywords" to "matches wildcard" was not done.)

[Spliff]

To clarify:

NO "font" problem intervening here, I use originally installed Arial, in which my "?" and [] are the original ASCII / ANSI characters 63, 91, 93 (i.e. no fiddling with character substitution so that UR could not recognize my "special" ?[] anymore - just adding this info for "completeness").


NO "stop word" problem intervening here; I use "real" words, not just "a" and "b" and "one" and "two" and the like, but words that are correctly found when I just search for them, and which are also correctly found when I search for them with implicit (!) "AND", so

a > found

a b > (implicit "AND") is found when I put a and b into the same item

a AND b > (explicit "AND") is NOT found since "AND" treated literally (and of course, IS "found" when (!) ball three, a, b, and AND, are present in an item)

a OR b > as the previous "a AND b": treated literally, so "found" (only) IF item contains all three: a, or, and b

a?b > the "?" is treated literally, thus is "found" if item contains string "a?b"

just similar, not identical, for any [...]: some item with [[x]] >
[[x]] > "finds" any items with single "x" (AS translation: "contains keywords") but not the item with string "[[x]]"
> seems to indicate that string-analysis code is not perfectly discrete but ambiguous in some cases


4 work situations (all with fts "on"), faulty behavior identical (!) in all four (whilst amelioration had been expected from a) to any of the others, naturally):

a) PC 1, with internet, UR heavily used (many UR DBs), AHK running, lots of "services" running; explicit-AND, and OR, problems since many months, over several UR versions, but OR worked as expected, many months ago; "?" and "[]" problems just discovered recently i.e. never used, or tried to use them, before; W10 Pro english, latest version

b) as a), "clean" UR-re-install (version of March 8, 2024), but possibility of third-party "services" or whatever interfering

c) PC 2, no internet (except for rare W10 updates), perfectly "clean", just some programs installed, AHK never even installed (let alone used), almost never used (i.e. my "reserve PC"), (months-old) UR version installed but never used, never copied (or created) a UR db (onto) here; UR try just with 2, 3 newly created items within the default "sample" db; W10 Pro english, some version about 6 months old

d) as c), "clean" UR-re-install (version of March 8, 2024), UR try with newly created db with just 2, 3 items created (over the default items which are already present at start)


Identical problems:

QS (QuickSearch) > AS (Advanced Search):

QS: a OR b > no result
again QS: a OR b > AS > contains keywords a OR b (=correct) > no result > OR obviously treated literally

QS: a?b > no result (except for items containing literal string "a?b"
again QS: a?b > AS > contains keywords a?b (=necessary switch to "matches wildcard" NOT done)
> switch to "matches wildcard" done manually > no result either > even "matches wildcard" treats "?" (and []) literally, NOT as placeholders / wildcard chars


Hence two code problems in all 4 situations:

QS: occurrence of "?" (or [ and ]) in QS string does NOT switch the internal (= AS) processing "contains keywords" to "matches wildcard"

AS: any OR, AND, ?, [...] within AS "matches wildcard" are treated literally though (i.e. not only when "escaped" but all the time) (and additional problems with "[" escaping, not clearly identified by me)


In the other hand:

"OR", done by AS in 2 or more rows, does work as expected, i.e. AS
contains keyword a
OR contains keyword b
OR contains keyword c
etc
works correctly, whilst
contains keyword a OR b > does NOT work correctly (OR being treated literally)

"*" is correctly treated correctly, i.e. as wildcard, BOTH times:
* in QS string > switch, in AS, is made to "matches wildcard", and then,
in AS, the "some*" string is treated correctly, i.e. the * as wildcard > correct results


P.S.:

Obviously, given my previous problems with UR on PC 1 (situations a and b), I had expected that UR Search on PC 2 (situations c and d) would work correctly, and so I was ready to switch to PC 2 (=my "reserve" PC, as said) for that reason; I then was very surprised to encounter the same problems there (situation c), and then again, even after a new UR install (situation d); I now must expect that even on a brand-new PC (PC 3), which I would have to buy, the same problem either presents itself on-the-spot, or then will occur after some time of use; I can NOT re-install Windows on PC 2 since that would break my license for another, special, rarely-used but expensive, program, which is "bound" to the current "system" on that PC (I even lost 2 licenses on my PC 1, by a Windows re-install on there, some years ago...).

Would UR be another Delphi program? (= more or less the "standard" framework at the time of its creation) - With some Delphi DLLs or similar being damaged, under certain (but obviously not all) conditions, by Windows 10 updates? (and then not systematically updated, too, by UR re-installs?) ;-(

[Spliff]

EDIT:

1)

I got the idea to run UR from a USB stick, to PC 1, finally (it didn't work but after I had closed UR on PC 1 and ran it again from the stick then):

It opened the UR DBs I had last opened on PC 1; I QS-searched
a OR b
> it (almost*) correctly displayed the items which contained either a or b
*="almost" since it only failed to also list, as search results, two NEW items, one with a, the other one with b, created by me immediately before the search, i.e. editing the DB (on PC 1) "from" the stick, so this "almost" obviously is a "fts index not updated" problem: It seems that when running UR from a stick, the DBs should also be on the stick, not on the PC.

Thus, I again de-installed UR from PC 2 (on which I don't need it), again incl. "Delete customization data? YES" (= just as I had done before situation d) described above), then checked PC 2's registry for the UR key, in order to verify if before above-described situation d), the UR key had been deleted or not, so that the new UR install on PC 2 had re-created the registry key, or had just updated it: The key is completely gone, so also situation d) had been after a complete, "fresh" new UR re-install (and without copying the registry key or parts of it, or any of the 4 UR ini files from your list, or any parts of it).

Thus, situation d), AFTER a complete UR re-install, without (!) any settings beyond the default ones, AND showing the same problems as in situations a, b and c, seems to prove that NONE of my UR settings (i.e. be it in the reg key, or in any one of the 4 ini files), seems to cause the problem, and on PC 2, NO other program had been running in situation d).

Thus, I really don't know where to search for possible causes of this awful problem - and running UR from stick is very slow (but could be sped up somewhat by buying a better stick).

2)

Off-topic here: MyInfo / MyInfoApp (according to the "manual" available on their site) seems to do something what you do, filtering in the tree pane ("Data Explorer" in UR), but whilst you only allow for that filtering by flags, MI/MIA only allows it for text-within-title, so this text-in-title (i.e. not also within the content or in other elements) might also be a user's "tag"; thus it's obvious that their tree filtering is - I hate to say this! - FAR superior to UR's indeed, since IF the user knows about that limitation, they can put "textual core info" (tags) within the titles, then filter the tree by those:

- UR: 20 flags, MI: hundreds of different tags / sub-tags in case

- UR: 20 flags combinable by OR only (by clicking together them by AND in "Tools-Flags" > only ONE flag possible per item), MI: those hundreds of tags also combinable by AND (several (!) tags / title-sub-strings possible per item)

Thus, MI doesn't have "tree filtering by general search" either (and neither has RightNote, as far as I can see from their help file: trial reverts to free-version, and then you can't trial the "pro" version anymore, even many years later...), BUT by "search in titles" at least, which multiplies, as explained, the filter possibilities in comparison to UR at least.

(Also, refer to MI's "manual" (number 5.1 in there) in order to compare how they accept users' "search" input, and which is obviously the only really good way to do it... as for the search results' presentation, it's then not so brilliant indeed, and RightNote's are simply awful in comparison...)

It would be really a good thing to think about amending filtering, be it in tree or in search: My original assumption that it was more or less "standard", was very mistaken, BUT it IS standard in editors, grep tools: The original text's order is NOT scrambled by them, by/in filtering, and the additional milliseconds due to to the additional processing would be happily borne by the users: always as an option, the "regular" = faster representation being "tree-destructing", so it would be the user's decision to wait a little second on top, for MUCH better search results - also bearing in mind that the necessary RE-sort, BACK to tree-order, and AFTER the collection of the search results, could certainly not take THAT long, considering normal searches would bring just some, or some dozens of results, rarely hundreds, almost never more... and then, it would always be possible to include some code lines for, "if result_count > 500 then message 'Results not tree-sorted since too many' instead"... ;-)

[kinook]

UR is developed with Visual C++, and I don't see how a Windows update could affect searching within UR. The SQLite library is statically linked into UltraRecall.exe.

I'm still unable to reproduce the problem with AND and OR not working. Can you post the details from

https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

My question re framework was because 25 years ago, .Net hadn't been there yet; I hadn't thought of MS tools before .Net; thanks for clarifying!

Sorry, my bad: The "stick version" (= situation e) = a fith one) is as faulty as the (newly and "from scratch") installed version, "new" items created on purpose weren't "found" since they didn't contain "or", as string, so "a or or", being processed in all 5 situations a)...e) as "a AND b AND or", wasn't found, obviously: after hours of trying, unsuccessfully, I then unfortunately tend to make even very stupid mistakes; a good night's sleep afterwards then clears the mind.

Btw, I tried with "dieses" and "jenes" (= "this" and "that", but certain to not introduce a "stop word" problem), and then, neither of the items just containing "dieses" OR "jenes" was listed as a "find", BUT when I then selected either of those two items manually, the "dieses" or "jenes" (in the text = content) was highlighted (which is only expected for search results, not also for the first item the user navigates to, after a search), whatever it was: if I selected the "dieses" item after the search (title = "v1"), "dieses" was highlighted, and if I selected the "jenes" item after the search (title = "v2"), "jenes" was highlighted, and vice versa: highlighted "search find" (but not also appearing within the search results) for the very first item selected after the search, whatever it was.

For a "dies?s" search, or for a "jen?s" search, I get some search results indeed, but with nothing being highlighted, since the whole content of any of those "hits" doesn't contain any string which could be translated to "dies[a-z]s" or "jen[a-z]s", ditto for searches for "dies[a-z]s" > "contains keywords" > LONG "hit" lists which don't make any sense, without any hit, neither placeholder- nor literally-wise, and then, changed to "matches wildcard" manually, no hits anymore - all this with fresh installs, not only after my "adjusting" the UR settings to my wishes.

The next step would obviously be to buy a third W10/11 PC.

(As said, OR works fine by AS in 2 independent rows, and I would have needed the ?-placeholder for "tag-pivoting": sometagA[a-z] being possible by searching for sometagA, but for searching / filtering by sometag[a-z]A I would have needed a "working" ?-placeholder, and, obviously, such "compound" tags would be (=for other UR users; would have been, for me) extremely useful for FILTERING-by-searching.

I then thought I could replace them by TWO tags instead, then searching = filtering for them by implicit AND:
sometag* someothertagA [1]
or
someothertagA sometag* [2]
= instead of
uniquetag?A
that is, and the * assuring the switch to "matches wildcard" will be made (it is made indeed).

Since 1 didn't work, I tried 2 (i.e. the * at the end), and that didn't work either, i.e. correct translation to AS, but no result, and:

When, in AS, I then click on the "Quick" button, I see that the QS search string has been enclosed in double quotes, so it reads now
"someothertagA sometag*"
(for both versions, 1 and 2, i.e. the * being "within" the string, or at the end of the string, does NOT have any implication).

Thus, it's obvious that the code which translates the search string, makes false assumptions, caused by the * here, or then caused by the translation to "matches wildcard", itself (correctly) caused by the *). (When I search for "sometagA someothertagB", it's found, and back to QS, it's without the "", but of course, we're in, and stay in, "contains keywords" then.)

Thus, even * does not work correctly BUT when it's only one single search string, with the * at its end:
sometagA* [that works indeed]

When the two tags are in the same line of the content, i.e. not in different lines, then
sometagA someothertag*
is successful ("matches wildcard" is done, and item is found), but back to QS, it now reads
"sometagA someothertag*" [= phrase search]

whilst (logically then)
someothertag* sometagA
doesn't find the item (which contains sometagA and someothertagB)
since, whilst AS shows, matches wildcard someothertag* sometagA
back-to-QS button brings
"someothertag* sometagA"
adding (unwanted) phrase search to placeholder/ wildcard use again, but possibly treating the * literally here because it's not at the very end of the (as said, unwanted) phrase search;

on the other hand, when I put
someothertag* sometagA
into the item content, into the same line = as "phrase", and expecting that "since" UR treats the * literally, that "phrase" should be found then, AS
matches wildcard someothertag* sometagA
does NOT find anything, albeit, back to QS, the search now reads,
"someothertag* sometagA"
where clicking on "Start" brings "0 matching items", and back to AS, it now reads
matches wildcard "someothertag* sometagA"
and back to QS, it now reads
""someothertag* sometagA""
and with this back-and-forth, I can multiply the "" at will, whilst it has never been intended as a phrase item to begin with!


I hope you can replicate (parts of?) my "a * brings unwanted double quotes (but doesn't treat the * literally anyway, and treats it as placeholder just at the very end of the forced phrase)" problem (which seems to indicate the search string might be processed faultily, internally)?

(Obviously, I closely verified my tag strings, and my search strings, in order to not make any mistake here.)

[kinook]

Again, please provide the info from

https://www.kinook.com/Forum/showthread.php?t=3038