Refactor SkipDetector category descriptions for enhanced clarity and specificity, improving technical accuracy and readability across various categories.

memory_system.py

@@ -343,39 +343,39 @@ class SkipDetector:
     """Semantic-based content classifier using zero-shot classification with category descriptions."""
 
     TECHNICAL_CATEGORY_DESCRIPTIONS = [
-        "programming language syntax, implementation, algorithm logic, function, method, class, object-oriented, variable scope, data type, control flow, import, module, package, library, framework, recursion, iteration, memoization, paradigm",
-        "software design patterns, creational: singleton, factory, builder, prototype; structural: adapter, decorator, facade, proxy; behavioral: observer, strategy, command, mediator, chain of responsibility; interface, polymorphism, composition",
-        "error handling, exception, traceback, stack trace, TypeError, NullPointerException, IndexError, segmentation fault, core dump, stack overflow, runtime vs compile-time error, assertion failed, syntax error, undefined reference, memory leak",
+        "programming language syntax, data types like string or integer, algorithm logic, function, method, programming class, object-oriented paradigm, variable scope, control flow, import, module, package, library, framework, recursion, iteration",
+        "software design patterns, creational: singleton, factory, builder; structural: adapter, decorator, facade, proxy; behavioral: observer, strategy, command, mediator, chain of responsibility; abstract interface, polymorphism, composition",
+        "error handling, exception, stack trace, TypeError, NullPointerException, IndexError, segmentation fault, core dump, stack overflow, runtime vs compile-time error, assertion failed, syntax error, null pointer dereference, memory leak, bug",
         "HTTP status codes: 404 Not Found, 500 Internal Server Error, 403 Forbidden, 401 Unauthorized, 200 OK, 201 Created. API response, 502 Bad Gateway, 503 Service Unavailable, 400 Bad Request, 429 Too Many Requests, timeout, CORS",
-        "terminal command line shell prompt, bash, zsh, powershell, cmd. Filesystem navigation: cd, ls, pwd. File management: mkdir, rm, cp, mv, chmod, chown. Text processing: grep, sed, awk, cat, echo. User permissions: sudo, root",
+        "terminal command line shell prompt, bash, zsh, powershell, cmd. Filesystem navigation: cd, ls, pwd. File management: mkdir, rm, cp, mv, chmod, chown. Text processing with grep, sed, awk, cat. User permissions: sudo, root access",
         "developer CLI tools, package manager, install, update. Network requests with curl, wget. Secure shell access with SSH. Version control with git: clone, commit, push, pull. Containerization with docker: run, build, compose; npm, pip",
-        "data interchange formats, serialization, deserialization, parsing. JSON object, array, key-value pair. XML tags, attributes. YAML indentation, TOML, CSV, .ini properties. Config files, env variables, dictionary, map, protocol buffers",
-        "WebSocket real-time bidirectional communication, server-client connection on a port, binary message protocol, handshake, upgrading from HTTP, socket programming, TCP, UDP, listening, binding, accepting, streaming, pub-sub, broadcast channel",
-        "API design, endpoint, REST, GraphQL, SOAP, RPC. HTTP methods: GET, POST, PUT, DELETE, PATCH. Request-response cycle, payload, authentication, authorization token, bearer, JWT, OAuth, API key, query parameters, path variables, body",
-        "file system paths, directory structure, config files, log files, binary executables, absolute vs relative paths, operating system, mount point, home, /tmp, /var, /usr, shared library, symbolic link, inode, file permissions, owner, group",
+        "data interchange formats, serialization, deserialization, parsing. JSON object, array, key-value pair. XML tags, attributes. YAML indentation, TOML, CSV, .ini properties. Config file, env variables, dictionary, map, protocol buffers",
+        "WebSocket real-time bidirectional communication, server-client connection on a port, binary message protocol, handshake, HTTP upgrade, socket programming, TCP, UDP, listening, binding, accepting, streaming, pub-sub, broadcast channel",
+        "API design, endpoint, REST, GraphQL, SOAP, RPC. HTTP methods: GET, POST, PUT, DELETE, PATCH. Request-response cycle, payload, authentication token, bearer, JWT, OAuth, API key, query parameters, path variables, request body",
+        "file system path, directory structure, config log bin, absolute vs relative path, operating system, filesystem, mount point, home, /tmp, /var, shared library, symbolic link, inode, file permissions, owner, group, read write execute",
         "algorithm analysis, O(log n) time complexity, space complexity, data structures, hash table, array, linked list, queue, stack, heap, priority queue, graph, adjacency matrix, depth-first search (DFS), breadth-first search (BFS)",
-        "sorting algorithms performance and implementation, including merge sort, quicksort, insertion sort, selection sort, bubble sort, heap sort. Understanding stable vs unstable sorts, in-place operations, and comparison-based sorting concepts",
-        "markdown syntax for text formatting, horizontal rule, separator using dashes or asterisks, headings, fenced code blocks with triple backticks, inline code, emphasis with bold and italic, strikethrough, blockquote, nested lists, task list",
-        "code formatting and style, indentation with whitespace, tabs vs spaces, nested function bodies, class methods, structured code, syntax highlighting for languages like Python, JavaScript, Java, C++, Go, Rust, TypeScript, Prettier, ESLint",
-        "container orchestration, cluster management, service scaling, replication, load balancing, namespace, pod, deployment, infrastructure, Kubernetes (K8s), Docker Swarm, Mesos, container runtime (CRI-O, containerd), image registry, Dockerfile",
-        "database query, SQL statement, table, column, row, index, primary key, foreign key, relationship, join, filter, select, insert, update, delete, relational vs NoSQL, MongoDB, PostgreSQL, MySQL, Redis, schema, normalization, transaction",
-        "application logging, log output, stack trace levels like INFO, WARN, ERROR, DEBUG, FATAL. Log message components: timestamp, level, module, line number. Diagnostic telemetry, monitoring, and observability for system health and debugging",
-        "regular expression (regex) patterns, matching groups, capturing, backslash escapes, metacharacters, wildcards, quantifiers, character classes, lookaheads, lookbehinds, alternation, anchors, word boundaries, multiline flag, global search",
+        "sorting algorithms performance and implementation, including merge sort, quicksort, insertion sort, selection sort. Understanding stable vs unstable sorts, in-place operations, comparison-based sorting, and computational complexity",
+        "markdown syntax for text formatting, horizontal rule, separator using dashes, headings, fenced code block with triple backticks, inline code, emphasis with bold and italic, strikethrough, blockquote, nested list, task list, markdown table",
+        "code formatting and style, indentation with whitespace, tabs vs spaces, nested function body, class method, structured code, syntax highlighting for languages like Python, JavaScript, Java, C++, Go, Rust, TypeScript, Prettier, ESLint",
+        "container orchestration, cluster management, service scaling, replication, load balancing, namespace, pod, deployment, infrastructure, Kubernetes (K8s), Docker Swarm, container runtime (CRI-O, containerd), image registry, Dockerfile",
+        "querying a database, SQL statement, database table, column, row, index, primary key, foreign key relationship, join, filter, select, insert, update, delete, relational vs NoSQL, MongoDB, PostgreSQL, MySQL, Redis, schema, transaction",
+        "application logging, log output, stack trace levels like INFO, WARN, ERROR, DEBUG, FATAL. Log message components: timestamp, module, line number. Diagnostic telemetry, monitoring, and observability for system health and debugging",
+        "regex pattern, regular expression matching, groups, capturing, backslash escapes, metacharacters, wildcards, quantifiers, character classes, lookaheads, lookbehinds, alternation, anchors, word boundary, multiline flag, global search",
         "software testing, unit test, assertion, mock, stub, fixture, test suite, test case, verification, automated QA, validation framework, JUnit, pytest, Jest. Integration, end-to-end (E2E), functional, regression, acceptance testing",
-        "cloud computing platforms, infrastructure as a service (IaaS), platform as a service (PaaS), AWS, Azure, GCP, compute instance, region, availability zone, scaling, elasticity, distributed system, virtual machine, serverless, Lambda",
+        "cloud computing platforms, infrastructure as a service (IaaS), PaaS, AWS, Azure, GCP, compute instance, region, availability zone, elasticity, distributed system, virtual machine, container, serverless, Lambda, edge computing, CDN",
     ]
 
     INSTRUCTION_CATEGORY_DESCRIPTIONS = [
-        "format the output as a structured data type. Return the answer as JSON with specific keys and values, or as YAML. Organize the information into a CSV file or a database-style table with specific columns and rows. Present a list of objects.",
-        "style the presentation of the text. Use markdown formatting like bullet points, a numbered list, or a task list. Organize the content into a grid or tabular layout with proper alignment. Create a hierarchical structure with nested elements.",
-        "adjust the length of the response. Make the answer shorter, more concise, brief, or condensed. Summarize the key points. Trim down the text to reduce the overall word count or meet a specific character limit. Be less verbose and more direct.",
-        "change the depth of the explanation. Make the response more detailed, comprehensive, and elaborate. Expand on the previous points and go into more depth. Provide a thorough, in-depth analysis. Explain the topic with more complexity and nuance.",
-        "rewrite the previous response. Rephrase, paraphrase, or reformulate the answer using different wording. Restate the information in another way to offer an alternative perspective. Express the same meaning but with a new structure and vocabulary.",
-        "alter the tone of the response. Change the writing style to be more formal, academic, or professional. Alternatively, make it more casual, friendly, and conversational. Adapt the register and voice to suit a specific audience or context.",
-        "explain the concept in simpler terms. Break down the topic step-by-step for a beginner. Clarify a confusing point. Explain it like I'm five years old (ELI5). Use an analogy or provide a concrete example to help me understand the idea.",
-        "continue the response. Keep going with the explanation or list. Provide more information and finish your thought. Complete the rest of the content or story. Proceed with the next steps. Do not stop until you have reached the conclusion.",
-        "act as a specific persona or role. Respond as if you were a pirate, a scientist, a travel guide, or a Shakespearean actor. Adopt the character's voice, style, and knowledge base in your answer. Maintain the persona throughout the response.",
-        "compare and contrast two or more topics. Explain the similarities and differences between A and B. Provide a detailed analysis of what they have in common and how they diverge. Create a table to highlight the key distinctions and resemblances.",
+        "format the output as structured data. Return the answer as JSON with specific keys and values, or as YAML. Organize information into a CSV file or a database-style table with columns and rows. Present as a list of objects or an array.",
+        "style the text presentation. Use markdown formatting like bullet points, a numbered list, or a task list. Organize content into a grid or tabular layout with proper alignment. Create a hierarchical structure with nested elements for clarity.",
+        "adjust the response length. Make the answer shorter, more concise, brief, or condensed. Summarize the key points. Trim down the text to reduce the overall word count or meet a specific character limit. Be less verbose and more direct.",
+        "change the explanation depth. Make the response more detailed, comprehensive, and elaborate. Expand on previous points and go into more depth. Provide a thorough, in-depth analysis. Explain the topic with more complexity and nuance.",
+        "rewrite the previous response. Rephrase, paraphrase, or reformulate the answer using different wording. Restate the information in another way to offer an alternative perspective. Express the same meaning but with a new structure or vocabulary.",
+        "alter the response tone. Change the writing style to be more formal, academic, or professional. Alternatively, make it more casual, friendly, and conversational. Adapt the register and voice to suit a specific audience or context level.",
+        "explain the concept in simpler terms. Break down the topic step-by-step for a beginner. Clarify a confusing point. Explain it like I'm five years old (ELI5). Use an analogy or a concrete example to help me understand the idea clearly.",
+        "continue the generated response. Keep going with the explanation or list. Provide more information and finish your thought. Complete the rest of the content or story. Proceed with the next steps. Do not stop until you have concluded.",
+        "act as a specific persona or role. Respond as if you were a pirate, a scientist, or a travel guide. Adopt the character's voice, style, and knowledge base in your answer. Maintain the persona throughout the entire response.",
+        "compare and contrast two or more topics. Explain the similarities and differences between A and B. Provide a detailed analysis of what they have in common and how they diverge. Create a table to highlight the key distinctions.",
     ]
 
     PURE_MATH_CALCULATION_CATEGORY_DESCRIPTIONS = [
@@ -406,10 +406,10 @@ class SkipDetector:
 
     GRAMMAR_PROOFREADING_CATEGORY_DESCRIPTIONS = [
         "proofread the following text for errors. Here is my draft, please check it for typos and mistakes: 'Teh quick brown fox jumpped'. Review, revise, and correct any misspellings or grammatical issues you find in the provided passage.",
-        "correct the grammar in this specific sentence: 'She don't like it' or 'Their going too the store'. Fix issues with subject-verb agreement, incorrect verb tense, pronoun reference errors, or misplaced modifiers in the provided text.",
-        "check the spelling and punctuation in this passage. Please review the following text and fix any errors: 'its a beautiful day, isnt it'. Correct missing commas, periods, apostrophes, quotation marks, colons, or capitalization.",
+        "correct the grammar in this sentence: 'She don't like it'. Resolve grammatical issues like subject-verb agreement, incorrect verb tense, pronoun reference errors, or misplaced modifiers in the provided text. Address faulty sentence structure.",
+        "check the spelling and punctuation in this passage. Please review the following text and correct any textual errors: 'its a beautiful day, isnt it'. Amend mistakes with commas, periods, apostrophes, quotation marks, colons, or capitalization.",
         "review this sentence and tell me if it is grammatically correct. Is the sentence 'There going to they're house' proper? Validate the grammar, check word usage (like their/there/they're), and verify that the sentence is well-formed.",
-        "proofread my email before I send it. Here is the draft. Please check for clarity, flow, coherence, and readability. Improve my writing, make it better, and polish the text to ensure it sounds professional and is free of errors.",
+        "proofread my email before I send it. Here is the draft. Please check for clarity, flow, coherence, and readability. Improve my writing, make it better, and polish the text to ensure it sounds professional and is free of textual errors.",
         "fix the punctuation in this run-on sentence or comma splice. Correct sentence fragments and ensure proper use of capitalization. Address errors with apostrophes, quotation marks, periods, semicolons, dashes, and other punctuation marks.",
         "suggest a better word choice or alternative phrasing. Can you help me improve my vocabulary and diction in this sentence? Replace words with more precise or impactful synonyms. Refine the expression for better clarity, tone, or style.",
         "rewrite this sentence from passive voice to active voice. Help me make my writing more direct and concise by eliminating passive constructions. Restructure the sentence to be more engaging and clear. Identify and fix faulty parallelism.",
@@ -427,7 +427,7 @@ class SkipDetector:
         "sharing my personal background, like my hometown, childhood upbringing, or education. My cultural heritage, ethnicity, or where I grew up. Information about the university I graduated from or formative life experiences that define my identity.",
         "asking for personal advice about a specific life situation, relationship, family decision, or career choice. Seeking guidance on a personal challenge, problem, or dilemma I'm facing. Needing help or counsel on a difficult issue or conflict.",
         "requesting personalized recommendations based on my stated context, preferences, or needs. For example, suggesting a movie based on genres I like, or a restaurant that fits my dietary restrictions, budget, and location requirements.",
-        "talking about my personal learning journey or educational pursuits. A course I'm taking, a certification I'm working on, or a degree program I'm enrolled in. My efforts in personal development, skill acquisition, or knowledge building.",
+        "talking about my personal learning journey or educational pursuits. A course or class I'm taking, a certification I'm working on, or a degree program. My efforts in personal development, skill acquisition, or knowledge building.",
         "discussing my child, spouse, or other family member's interests or needs. Helping my son with a school project, finding a hobby for my daughter, or supporting my partner's career goals. Questions related to supporting my loved ones.",
         "describing my personal challenges with a work task, learning a new skill, or a technology problem. Feeling confused, stressed, or overwhelmed. Dealing with imposter syndrome, self-doubt, or needing assistance with a difficult project.",
         "planning a personal event like a party, celebration, or family gathering. Organizing my daughter's birthday, my son's graduation, or a wedding anniversary. Discussing arrangements for a special occasion or festive milestone commemoration.",
@@ -442,6 +442,7 @@ class SkipDetector:
         "sharing my personal values and what I care about deeply. I believe strongly in environmental sustainability, social justice, or equality. Causes I support, my principles, ethics, morals, and convictions that shape my worldview and priorities.",
         "discussing a personal achievement or milestone. I got promoted, received an award, won a competition, or completed a marathon. A significant accomplishment I am proud of, a goal I reached, or a success that marked a personal triumph.",
         "referencing my social preferences. I am an introvert, an extrovert, or an ambivert. I prefer small groups over large crowds. My personality trait regarding socializing, my interaction style, and where I get my energy in social settings.",
+        "discussing everyday problems or logistics. Dealing with a car repair, a household issue like a broken appliance, losing my keys, managing appointments, or troubleshooting a personal device. Life's daily challenges and practical solutions.",
     ]
 
     class SkipReason(Enum):