mtayfur
3f9b4c6d48
♻️ (memory_system): refactor skip detection and add semantic deduplication
...
- Unify skip detection to a binary classifier (personal vs non-personal)
for improved maintainability and clarity. Remove multiple technical/
instruction/translation/etc. categories and consolidate into
NON_PERSONAL and PERSONAL.
- Adjust skip detection margin for more precise classification.
- Add semantic deduplication for memory operations using embedding
similarity, preventing duplicate memory creation and updates.
- Normalize and validate embedding dimensions for robustness.
- Add per-user async locks to prevent race conditions during memory
consolidation.
- Refactor requirements.txt to remove version pinning for easier
dependency management.
- Improve logging and error handling for embedding and deduplication
operations.
These changes improve the reliability and accuracy of memory
classification and deduplication, reduce false positives in skip
detection, and prevent duplicate or conflicting memory operations in
concurrent environments. Dependency management is simplified for
compatibility.
2025-10-27 00:27:33 +03:00
mtayfur
bb1bd01222
♻️ (memory_system.py): reformat code for consistency, readability, and maintainability
...
- Reorder and group imports for clarity and PEP8 compliance.
- Standardize string quoting and whitespace for consistency.
- Refactor long function signatures and dictionary constructions for better readability.
- Use double quotes for all string literals and dictionary keys.
- Improve formatting of multiline statements and function calls.
- Add or adjust line breaks to keep lines within recommended length.
- Reformat class and method docstrings for clarity.
- Use consistent indentation and spacing throughout the file.
These changes improve code readability, maintainability, and consistency, making it easier for future contributors to understand and modify the codebase. No functional logic is changed.
2025-10-27 00:20:35 +03:00
mtayfur
189c6d4226
🔧 (dev-check.sh, pyproject.toml, requirements.txt): add development tooling and configuration
...
Introduce a dev-check.sh script to automate code formatting and import
sorting using Black and isort. Add a pyproject.toml file to configure
Black and isort settings for consistent code style. Update
requirements.txt to include Black and isort as development dependencies
and remove version pinning for easier dependency management.
These changes streamline the development workflow, enforce code style
consistency, and make it easier for contributors to run formatting and
import checks locally.
2025-10-27 00:20:05 +03:00
mtayfur
c0bfb3927b
Refactor memory creation guidelines for improved clarity and conciseness in contextual completeness section.
2025-10-19 05:13:45 +03:00
mtayfur
d05ed8a16e
Update semantic retrieval thresholds in Constants class for improved accuracy
2025-10-19 04:57:36 +03:00
mtayfur
7e2209633d
Refactor logger initialization in memory_system.py to use module name for better context in log messages.
2025-10-18 19:25:22 +03:00
mtayfur
505c443050
Update README.md to enhance clarity on privacy and cost considerations; restructure sections for better readability and add relevant details.
2025-10-15 14:33:33 +03:00
mtayfur
0726293446
Update README.md for improved clarity and accuracy; revise privacy notice, cache descriptions, and model support details.
2025-10-15 14:13:55 +03:00
mtayfur
e3709fe677
Refactor cache management in Filter class; reduce maximum cache entries and concurrent user caches for improved performance and clarity. Update cache management methods for consistency and better logging.
2025-10-15 14:05:01 +03:00
mtayfur
2deba4fb2c
Refactor Filter class to use async for pipeline context setup; implement locking mechanism for shared skip detector cache to enhance concurrency safety.
2025-10-12 23:24:58 +03:00
mtayfur
849dd71a01
Refactor memory selection logic in LLMRerankingService for improved clarity; streamline response handling by directly using response.ids.
2025-10-12 23:03:36 +03:00
mtayfur
158f0d1983
Refactor memory operations in Filter class for improved readability and consistency; utilize statistics.median for score calculation and streamline operation details formatting.
2025-10-12 22:54:18 +03:00
mtayfur
2db2d3f2c8
Refactor SkipDetector to streamline skip detection logic and improve clarity; update method signature for better integration with memory system.
2025-10-12 21:44:51 +03:00
mtayfur
1390505665
Refactor SkipDetector to use a callable embedding function instead of SentenceTransformer; update requirements to remove unnecessary dependencies.
2025-10-10 15:11:41 +03:00
mtayfur
5c0ca1f4ab
Update model loading to use 'auto' device for SentenceTransformer, enhancing compatibility across environments.
2025-10-09 21:50:17 +03:00
mtayfur
86de7bad03
Add README.md to document the Memory System for Open WebUI, detailing core features, model support, configuration options, performance optimizations, and memory quality management.
2025-10-09 14:19:57 +03:00
mtayfur
6820ef9dcd
Refactor SkipDetector category descriptions for enhanced clarity and specificity, improving technical accuracy and readability across various categories.
2025-10-09 13:43:44 +03:00
mtayfur
e89a9b29b9
Refactor cache configuration constants for improved clarity and consistency; adjust cache entry limits and hash prefix length
2025-10-09 13:10:58 +03:00
mtayfur
74f7ededcd
Refactor memory retrieval logging to include retrieval method, enhancing clarity in LLM and semantic decision processes
2025-10-09 11:41:20 +03:00
mtayfur
bb992906ad
Refactor LLM analysis logging and status emissions for improved clarity and accuracy in memory retrieval processes
2025-10-09 11:33:50 +03:00
mtayfur
765408cea4
Refactor SkipDetector category descriptions for improved clarity and specificity, enhancing semantic classification and detection accuracy across technical, instruction, math, translation, grammar, and conversational categories.
2025-10-06 17:52:05 +03:00
mtayfur
50c38f39d1
Refactor SkipDetector category descriptions for enhanced clarity and specificity, updating technical and personal context entries to improve semantic classification and detection accuracy.
2025-10-06 14:34:48 +03:00
mtayfur
34d11a3be6
Implement code changes to enhance functionality and improve performance
2025-10-06 12:18:45 +03:00
mtayfur
7c495adca5
Refactor SkipDetector category descriptions for improved clarity and specificity, consolidating meta instruction and factual query categories into a single instruction category, enhancing semantic classification and detection accuracy.
2025-10-06 03:03:19 +03:00
mtayfur
9a5c6b8f58
Refactor SkipDetector category descriptions for enhanced specificity and clarity, improving semantic classification and detection accuracy.
2025-10-06 02:24:59 +03:00
mtayfur
5c91a53ded
Enhance SkipDetector category descriptions by adding new entries for debugging, CI/CD, translation, and personal context, improving detection capabilities and coverage.
2025-10-06 01:10:49 +03:00
mtayfur
05e75d1991
Refactor SkipDetector category descriptions for improved clarity and specificity, updating meta conversation to meta instruction and enhancing technical detection logic
2025-10-05 23:43:44 +03:00
mtayfur
826cd6abf7
Refactor SkipDetector category descriptions for clarity and conciseness, enhancing semantic classification accuracy
2025-10-05 21:21:46 +03:00
mtayfur
c98809f807
Refactor SkipDetector to remove creative fiction and roleplay categories, streamlining skip detection logic
2025-10-05 21:08:15 +03:00
mtayfur
2c9e7141b3
Refactor SkipDetector category descriptions for clarity and conciseness
2025-10-05 20:50:17 +03:00
mtayfur
a13302e0b5
Refactor SkipDetector to enhance skip detection patterns and add new categories for pure math, translation, grammar, fiction, and roleplay
2025-10-05 16:32:59 +03:00
mtayfur
f00e25bf44
Refactor Filter class to improve error handling for empty text inputs and optimize embedding cache logic
2025-10-05 10:58:38 +03:00
mtayfur
c3d6a36f2a
Update duration threshold for memory retrieval logging to improve accuracy
2025-10-05 00:14:04 +03:00
mtayfur
03a5388c5e
Enhance SkipDetector with fast-path detection for technical messages and add confident margin for early skips
2025-10-04 22:55:28 +03:00
mtayfur
70daca777f
Refactor SkipDetector logging to simplify message output for skipped messages
2025-10-03 23:21:20 +03:00
mtayfur
d9409c2aee
Add logging and status emission for cases with no relevant memories found
2025-10-03 23:00:42 +03:00
mtayfur
75f48a4112
Enhance SkipDetector with additional skip categories and descriptions for improved message handling
2025-10-03 22:42:35 +03:00
mtayfur
e546a1508d
Refactor Filter class configuration fields for clarity and redundancy removal
2025-10-03 21:30:22 +03:00
mtayfur
1980461f22
Initial commit
2025-10-03 21:26:41 +03:00
