sbp-jm/sbp-flarum-search
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 2 Packages Wiki Activity

sbp-hardfork #2

Open
sbp-jm wants to merge 0 commits from sbp-hardfork into main
pull from: sbp-hardfork
merge into: sbp-jm:main
Conversation 5 Commits - Files changed -
sbp-jm commented 2026-04-28 06:53:56 +00:00
Owner
Copy link

nach main!

nach main!
Without minimum_should_match=1 on the inner post bool query, Elasticsearch
defaults MSM to 0 whenever a filter clause is present. This caused has_child
to score every non-hidden post on every search request, saturating CPU on
shared ES nodes with large corpora (3.7M+ posts).

- Add BoolQuery subclass with create() override and minimumShouldMatch()
  (spatie/elasticsearch-query-builder 1.x uses `new self()` in create(),
  so subclassing requires overriding it)
- Set minimumShouldMatch(1) on the inner post query after adding the
  is_hidden filter, so only genuinely matching posts are scored
- Remove the operator('or') clause from buildShouldClauses() — with
  Turkish min_ngram=2, 'or' generates 2-gram tokens that match nearly
  every post, causing near-total index scans
- Add ES client timeouts (connect: 2s, query: 10s) to prevent Apache
  mod_php worker saturation when ES is slow or unreachable
- Default sort changed from relevance to updated_at desc. This lets
  has_child use score_mode:none, which skips child scoring entirely
  and lets ES short-circuit early on large corpora.
- When a field sort is requested, score_mode is also none for the same
  reason. score_mode:sum is only used when sorting by relevance.
- track_total_hits:false avoids a full-index count on every query,
  allowing ES to stop once it has collected enough results.
- Strip all gambit operators (tag:foo, author:bar, is:unread, etc.)
  from the ES query string, not just is:private. Leaving them in caused
  operator:and to require the gambit tokens to appear literally in post
  content, producing zero results when gambits were combined with text.
- Search with no explicit sort now defaults to latest (updated_at desc)
  on both backend and frontend, matching the forum UX expectation.
- "All discussions" link from the search box routes to /?q=...&sort=latest
  so the sort dropdown shows "Latest" immediately after submission.
- extendDiscussionState injects sort=-lastPostedAt into API params when
  no sort is present, keeping direct-URL navigation (?q=anan) consistent
  with the backend default.
- Backend guard changed from phpSortField===null to empty($sorts), which
  is the correct sentinel since $sorts is the authoritative ES sort list.
When params.q is set, Flarum core suppresses TerminalPost and only shows an
excerpt from mostRelevantPost. This leaves the info section empty when the post
is null (e.g. hidden/deleted first post in production) or when results are
ordered by a field sort where an excerpt is meaningless.

Extend DiscussionListItem.infoItems to:
- Replace excerpt with TerminalPost when a field sort (latest, oldest, top) is
  active — relevance excerpts are not meaningful when sorted by date/count.
- Fall back to TerminalPost when no excerpt was added (mostRelevantPost null).
Replace the asymmetric ngram (index) / language (search) setup on content
with a symmetric language analyzer at both index and search time, eliminating
zero-hit cases caused by stemmed tokens falling below min_gram (e.g. Turkish).

Add a title field to discussion documents with a title.autocomplete edge-ngram
subfield (min=2, max=15) for prefix instant search, and a flarum_title_search
analyzer (lowercase only) as its search-time counterpart.

Remove flarum_analyzer_partial / partial_search_filter / search_analyzer from
the content mapping — no longer needed with symmetric analysis.

Bump INDEX_COMPAT_VERSION to v3. Also fix MatchQuery::toArray() to serialize
the analyzer property which was stored but never emitted.
When filter[autocomplete]=1 is present, bypass the full has_child query
and run a single MatchQuery on title.autocomplete (edge-ngram prefix match).
Results are sorted by updated_at desc to stay consistent with full search.
Add a stem-exclusion textarea to the admin settings page. Words entered here
(one per line) are passed as stem_exclusion to the language analyzer, preventing
them from being stemmed. Requires a full re-index to take effect.

The two language-dependent fields (stem exclusion, min search length) are
disabled when CJK is selected, since neither applies to bigram analysis.

Settings are reordered: search-scope switches now appear above the language
block so the language-dependent fields sit directly below the language selector.

Replace the lowercase ES identifier labels in the language dropdown with proper
display names (e.g. "Brazilian Portuguese", "CJK (Chinese, Japanese, Korean)",
"Sorani (Kurdish)"). Drift detection now tracks stem exclusion instead of the
removed min_gram index setting.
When searching inside comments is ON, the autocomplete dropdown now returns
nothing from ES (only usernames show). This avoids showing title-only matches
that are inconsistent with the full comment-body search results.

Also passes filter[autocomplete]=1 to the API so the server uses the fast
title.autocomplete path instead of the full has_child query.
sbp-jm requested review from jm 2026-04-28 06:54:49 +00:00
sbp-jm commented 2026-04-28 06:55:05 +00:00
Author
Owner
Copy link

ja machen!

ja machen!
jm approved these changes 2026-04-28 06:58:15 +00:00
Dismissed
jm left a comment
Copy link

los!

los!
jm commented 2026-04-28 06:58:26 +00:00
Owner
Copy link

jetzt los

jetzt los
jm requested review from jm 2026-04-28 06:58:26 +00:00
jm approved these changes 2026-04-28 07:03:12 +00:00
jm left a comment
Copy link

alle angeschaut

alle angeschaut
jm commented 2026-04-28 07:03:26 +00:00
Owner
Copy link

und jetzt?

und jetzt?
This pull request has changes conflicting with the target branch.
  • js/dist/admin.js
  • js/dist/admin.js.map
  • js/dist/forum.js.map
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin sbp-hardfork:sbp-hardfork
git switch sbp-hardfork

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff sbp-hardfork
git switch sbp-hardfork
git rebase main
git switch main
git merge --ff-only sbp-hardfork
git switch sbp-hardfork
git rebase main
git switch main
git merge --no-ff sbp-hardfork
git switch main
git merge --squash sbp-hardfork
git switch main
git merge --ff-only sbp-hardfork
git switch main
git merge sbp-hardfork
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
jm
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sbp-jm/sbp-flarum-search!2
No description provided.