- Renamed fields in LLMResponseUsage struct to better reflect their purpose, changing InputTokens and OutputTokens to PromptTokens and CompletionTokens.
- Modified GetUsage method in ChatMessage to accept an optional contents parameter, improving flexibility in usage tracking.
- Ensured consistent field naming and improved clarity in token usage aggregation logic.
- Added BSON tags to fields in LLMResponseUsage struct to ensure proper serialization with MongoDB.
- Updated the Usage field in ChatMessage model to exclude it from JSON serialization while retaining BSON compatibility.
- Introduced GetUsage method in ChatMessage to aggregate token usage from contents, enhancing usage tracking capabilities.
- Added LLMResponseUsage struct to encapsulate token usage details for language model responses.
- Updated ChatMessageContent and ChatMessage models to include a Usage field of type *entity.LLMResponseUsage, enhancing message handling capabilities.
- Improved structure and clarity of models by ensuring consistent field definitions and descriptions.
- Removed type definitions for ChatMessageContentType and ChatMessageContentActionStatus, replacing them with string fields for flexibility.
- Added a new 'Key' field to ChatMessageContent for enhanced content identification.
- Updated the 'Content' field description in ChatMessage to specify its context for user/system messages.
- Ensured consistency in field naming and descriptions across models for better maintainability.
- Added ChatMessageContent model to encapsulate message content types and action statuses, improving the structure of chat messages.
- Updated ChatMessage model to include a slice of ChatMessageContent, allowing for multiple content items per message.
- Implemented GetContent method to handle content retrieval and formatting based on content type, enhancing message processing capabilities.
- Ensured alignment with existing coding standards and added necessary comments for clarity and maintainability.
- Introduced ChatConversation and ChatMessage structs to represent conversation and message data in the application.
- Included relevant fields with JSON and BSON annotations for proper serialization and database integration.
- Enhanced model definitions with descriptions for better understanding of their purpose and usage.
- Ensured alignment with existing coding standards and practices for maintainability.
- Added descriptive annotations to various parameters across controllers to improve API documentation clarity.
- Standardized field definitions in models by including descriptions for better understanding of their purpose.
- Updated validation patterns for ID fields to ensure consistency and improve data integrity.
- Enhanced overall code readability and maintainability by aligning naming conventions and adding necessary comments.
- Updated Go version in go.work and backend/go.mod to 1.23.7
- Updated various dependencies in go.sum and backend/go.sum
- Refactored models to remove generic type parameters from BaseModel
- Introduced new utility functions for consistent API responses
- Removed unused utility files from controllers
Updated LLMProvider struct to include a new 'DeploymentName' field:
- Added 'deployment_name' tag for JSON and BSON serialization
- Modified IsUnset() method to check for deployment name in Azure OpenAI configuration
- Ensures more comprehensive validation for Azure OpenAI provider setup
Introduced a new method to check if an LLM provider is considered unset:
- Added Unset field to LLMProvider struct
- Implemented IsUnset() method with logic to validate provider configuration
- Handles special cases for different provider types (azure-openai, openai-compatible)
- Checks for empty API key, missing models, and provider-specific configuration requirements
Simplified LLMProvider model:
- Removed description, config schema, default config, and supported features
- Added new field for API version
- Cleaned up and focused on core provider attributes
Restructured LLMProvider and removed LLMModel:
- Added collection tag to LLMProvider
- Renamed and reorganized provider fields
- Removed separate LLMModel struct
- Simplified model and provider attributes
Introduced two new data models for managing Language Learning Models (LLMs):
- LLMProvider: Represents LLM providers like OpenAI, Anthropic
- LLMModel: Represents specific models within a provider
Models include key attributes such as:
- Naming and display information
- Enabled/priority status
- Supported features
- Token pricing
- Configuration schemas
- Updated the Spider model to introduce a new SpiderTemplateParams struct for improved template handling.
- Refactored string utility functions in utils/string.go to include a new replaceChars function, streamlining character replacement across multiple functions.
- Enhanced ToSnakeCase and ToKebabCase functions to utilize the new replaceChars function for better maintainability and readability.
- Added splitStringWithQuotes function to facilitate string manipulation with quotes, improving overall utility in string processing.
- Replaced direct calls to getBaseFileFsSvc with a new method fs.GetBaseFileFsSvc in base_file.go for improved clarity and maintainability.
- Introduced SpiderTemplateService interface and implemented registry service for managing spider templates, enhancing template handling in the spider controller.
- Added template-related fields to the Spider model to support template functionality.
- Created utility functions for string case conversions in utils/string.go to facilitate consistent formatting across the codebase.
- Updated environment configuration to retrieve the Python path dynamically, improving flexibility in the task runner's setup.
- Eliminated the default_version field from the DependencyConfig struct in dependency_config.go to streamline the configuration model.
- This change simplifies the dependency management process and aligns with recent updates in the application structure.
- Deleted the db module, consolidating database-related functionality into the core/mongo package for better organization and maintainability.
- Updated all import paths across the codebase to replace references to the removed db module with core/mongo.
- Cleaned up unused code and dependencies, enhancing overall project clarity and reducing complexity.
- This refactor improves the structure of the codebase by centralizing database operations and simplifying module management.
- Replaced all instances of apex/log with a structured logger interface in various services, including Api, Server, Config, and others, to enhance logging consistency and context.
- Updated logging calls to utilize the new logger methods, improving error tracking and service monitoring.
- Added logger initialization in services and controllers to ensure proper logging setup.
- Improved error handling and logging messages for better clarity during service operations.
- Removed unused apex/log imports and cleaned up related code for better maintainability.
- Added DependencyActionSync and DependencyActionSetup constants to improve dependency management.
- Refactored GrpcClient to utilize a logger interface for consistent logging across connection states and errors.
- Updated Start, Stop, and connection methods to replace direct log calls with logger methods, enhancing log context and readability.
- Simplified test cases by removing error checks on gRPC client start, ensuring cleaner test setup.
- Added TotalDependencies and SearchReady fields to the DependencyConfig struct for improved dependency management.
- Retained the Setup field for backward compatibility while ensuring proper JSON and BSON handling.
- Added utility function to normalize BSON ObjectId in query parameters for gRPC methods.
- Updated GetOne, GetMany, DeleteOne, DeleteMany, UpdateOne, UpdateMany, ReplaceOne, and UpsertOne methods to utilize the new normalization function.
- Introduced new DependencyConfigSetup model instance in base model definitions for improved structure.
- Replaced direct gRPC client instantiation with a call to GetGrpcClient().Start() for improved consistency and maintainability across test cases.
- Cleaned up unnecessary imports in model_service.go to streamline the codebase.
- Enhanced test setup by ensuring the gRPC client is properly initialized before executing service methods.
- Introduced a new POST endpoint '/:id/run' in the router to trigger the execution of a schedule.
- Implemented the PostScheduleRun function to handle the execution logic, including validation of input parameters and scheduling tasks.
- Refactored the PostSpiderRun function to streamline the scheduling process by directly calling the admin service.
- Updated the FsFileInfo struct to clarify the children field description.
- Modified the Task model to make NodeIds field optional in JSON serialization.
- Added WaitForReady method to GrpcClient for blocking until the client is ready.
- Updated WorkerService to utilize WaitForReady for ensuring gRPC client readiness before starting.
- Refactored ModelService to consistently use GetGrpcClient for context management.
- Changed logging level for received metrics in MetricServiceServer from Info to Debug.
- Modified error handling in HandleError to conditionally print errors based on the environment.
- Cleaned up unused GrpcClient references in various services, improving code clarity.
