Refactors the skip detection logic to precompute and reuse line splits and non-empty line lists, reducing repeated operations and improving efficiency and readability, while also fixing edge cases for empty messages and ensuring more robust pattern detection.
Improves clarity in example 3 by specifying the origin city in the CREATE
operation for more complete context, and updates example 6 to better reflect
the distinction between technical requests and personal statements, ensuring
the documentation accurately guides memory handling logic.
refactor(memory_system): rewrite category descriptions for clarity and conciseness
Category descriptions in NON_PERSONAL_CATEGORY_DESCRIPTIONS and
PERSONAL_CATEGORY_DESCRIPTIONS are rewritten to be more concise,
generalized, and easier to parse, reducing verbosity and removing
example-heavy phrasing. This improves maintainability, readability,
and consistency, making the intent of each category clearer for
future development and review.
docs: expand and clarify examples of personal information categories
Additional examples are added to better illustrate the types of
personal information covered, improving clarity for users and
developers about what constitutes sensitive data in various contexts.
Emojis are added to log messages for better visual distinction and quick scanning in logs, and several docstrings are introduced or improved to clarify method purposes, enhancing code readability and maintainability. Unused parameters are removed for cleaner function signatures.
Status level constants are consolidated into a single STATUS_LEVEL dict for easier access and reduced redundancy, log messages are clarified and made more concise for better diagnostics, and threshold multipliers are adjusted to fine-tune memory retrieval and reranking behavior. These changes improve code maintainability, readability, and operational transparency.
Refactors deduplication to batch embedding generation for efficiency, consolidates user message extraction into a helper for reuse, replaces statistics.median with numpy for consistency, simplifies memory operation execution, and removes redundant logging and unused imports to improve maintainability and performance.
Updates content extraction logic for robustness, ensures all hash operations
handle non-string input safely, filters out empty memory content, and bumps
required Open WebUI version for compatibility. These changes address edge
cases in content processing, prevent potential errors, and ensure only valid
memories are processed and embedded.
Redundant checks and exception handling are removed to streamline code paths, making embedding normalization, user memory retrieval, and LLM response decoding more concise and maintainable while preserving error propagation at higher levels.
Skip detection and embedding-related methods are now asynchronous, allowing
non-blocking execution and better concurrency; embedding function wrappers
and initialization routines are updated to support async/await, and
shared skip detector caching is adapted accordingly. These changes are
necessary to ensure compatibility with async embedding functions, prevent
blocking the event loop, and improve scalability and responsiveness in
high-concurrency environments.
Adds an "Intermediate" status level between "Basic" and "Detailed" to provide
summaries and key details, updates all relevant mappings and usages to support
this new level, and adjusts default verbosity for status messages to improve
clarity and reduce noise for users who do not require full diagnostics.
Eliminate unused or redundant valve options such as max_message_chars,
relaxed_semantic_threshold_multiplier, and enable_llm_reranking to
streamline configuration, clarify that llm_reranking_trigger_multiplier
disables reranking when set to 0.0, and update documentation and code to
reflect these changes for improved maintainability and user clarity.
Allowing skip_category_margin to be set in Filter enables dynamic
adjustment of skip detection sensitivity, improving flexibility for
different deployment scenarios.
Eliminates the "Intermediate" status level for message verbosity, consolidating to "Basic" and "Detailed" for clarity and simplicity; updates all related mappings, enums, and status emission calls to reflect this streamlined approach, reducing confusion and improving maintainability.
Category descriptions are made more comprehensive and precise to improve classification accuracy, the skip margin is adjusted for better thresholding, and skip state caching is now performed before early returns to ensure consistent cache updates regardless of skip conditions. The user message extraction logic is also improved for robustness.
Introduce status_emit_level to control status message verbosity, allowing
users to select Basic, Intermediate, or Detailed output; refactor all
status emission calls to respect this setting, improving clarity and
customizability of system feedback for different user needs.
The SKIP_CATEGORY_MARGIN constant is raised from 0.10 to 0.15 to make skip
category classification more selective, reducing false positives by requiring
a higher margin above personal similarity.
Removes redundant try/except blocks and input validation in several
methods to simplify logic and improve readability. Moves error handling
to higher levels where appropriate. Adds a skip state cache to track
when memory operations should be skipped, improving efficiency by
avoiding repeated skip checks. Cleans up batch operation execution and
cache clearing to include the new skip state. These changes reduce
unnecessary code complexity and improve maintainability, while also
optimizing memory operation flow and cache management.
refactor(skip-detection): condense and generalize personal category descriptions for maintainability and easier updates
fix(constants): lower SKIP_CATEGORY_MARGIN from 0.20 to 0.10 to reduce false positives in skip classification
refactor(skip-detection): clarify logging for skip/allow decisions and threshold calculation for better traceability
refactor: use consistent double-quote style for operator and code-ending character lists
fix(filter): ensure embeddings are normalized before use in skip detector to improve classification accuracy
Non-personal category descriptions are now more comprehensive and organized by topic, improving the classifier's ability to detect impersonal content. Personal category descriptions are condensed into broader, clearer categories, making them easier to maintain and update. The SKIP_CATEGORY_MARGIN is reduced to make the skip detector less aggressive, addressing previous false positives. Logging is improved for easier debugging and understanding of skip decisions. Embedding normalization is enforced to ensure consistent similarity calculations.
Expand and clarify the "Filter for Intent" rule to ensure only direct,
personally significant facts are stored, explicitly excluding messages
where the user's primary intent is instructional, technical, or
analytical. Update processing and decision frameworks to reinforce
selectivity based on user intent. Revise and annotate examples to
demonstrate correct application of the new rules, making it clear that
requests for advice, recommendations, or technical tasks are ignored.
These changes improve the precision and reliability of memory
consolidation, reducing the risk of storing irrelevant or transient
information.
- Unify skip detection to a binary classifier (personal vs non-personal)
for improved maintainability and clarity. Remove multiple technical/
instruction/translation/etc. categories and consolidate into
NON_PERSONAL and PERSONAL.
- Adjust skip detection margin for more precise classification.
- Add semantic deduplication for memory operations using embedding
similarity, preventing duplicate memory creation and updates.
- Normalize and validate embedding dimensions for robustness.
- Add per-user async locks to prevent race conditions during memory
consolidation.
- Refactor requirements.txt to remove version pinning for easier
dependency management.
- Improve logging and error handling for embedding and deduplication
operations.
These changes improve the reliability and accuracy of memory
classification and deduplication, reduce false positives in skip
detection, and prevent duplicate or conflicting memory operations in
concurrent environments. Dependency management is simplified for
compatibility.
- Reorder and group imports for clarity and PEP8 compliance.
- Standardize string quoting and whitespace for consistency.
- Refactor long function signatures and dictionary constructions for better readability.
- Use double quotes for all string literals and dictionary keys.
- Improve formatting of multiline statements and function calls.
- Add or adjust line breaks to keep lines within recommended length.
- Reformat class and method docstrings for clarity.
- Use consistent indentation and spacing throughout the file.
These changes improve code readability, maintainability, and consistency, making it easier for future contributors to understand and modify the codebase. No functional logic is changed.
Introduce a dev-check.sh script to automate code formatting and import
sorting using Black and isort. Add a pyproject.toml file to configure
Black and isort settings for consistent code style. Update
requirements.txt to include Black and isort as development dependencies
and remove version pinning for easier dependency management.
These changes streamline the development workflow, enforce code style
consistency, and make it easier for contributors to run formatting and
import checks locally.
