- Updated the description for the GetSpiderResults route to clarify the type of data returned.
- Added a new route for GetTaskResults to retrieve scraped or crawled results data for tasks.
- Refactored the GetSpiderResults and GetTaskResults functions to improve query handling and error management.
- Ensured consistent usage of MongoDB query methods across the controllers for better maintainability.
- Changed the base image from golang:1.22.9 to golang:1.23 to leverage the latest features and improvements.
- Ensured alignment with existing coding standards and maintained consistency in the Dockerfile structure.
- Removed redundant comment for GetListParams struct to improve clarity.
- Updated Response, ListResponse, and VoidResponse types to use omitempty for the Error field, streamlining JSON output.
- Introduced ToJSON method for Response and ListResponse types to facilitate easy serialization of response data.
- Ensured alignment with existing coding standards and improved code maintainability through consistent struct definitions.
- Removed type definitions for ChatMessageContentType and ChatMessageContentActionStatus, replacing them with string fields for flexibility.
- Added a new 'Key' field to ChatMessageContent for enhanced content identification.
- Updated the 'Content' field description in ChatMessage to specify its context for user/system messages.
- Ensured consistency in field naming and descriptions across models for better maintainability.
- Added ChatMessageContent model to encapsulate message content types and action statuses, improving the structure of chat messages.
- Updated ChatMessage model to include a slice of ChatMessageContent, allowing for multiple content items per message.
- Implemented GetContent method to handle content retrieval and formatting based on content type, enhancing message processing capabilities.
- Ensured alignment with existing coding standards and added necessary comments for clarity and maintainability.
- Changed the summary conversion method in getIDForAction from PascalCase to SnakeCase for consistency with naming conventions.
- Introduced BaseResponse interface and implemented GetData and GetDataString methods for Response, ListResponse, and VoidResponse types, enhancing response handling.
- Simplified the Contains function to specifically handle string slices, improving clarity and maintainability of utility functions.
- Ensured alignment with existing coding standards and added necessary comments for better understanding.
- Replaced the StatsService interface with direct MongoDB queries in the stats controller, enhancing performance and reducing complexity.
- Introduced structured response types for GetStatsOverview and GetStatsTasks, improving clarity and maintainability of the API responses.
- Removed outdated service implementation files, streamlining the codebase and eliminating redundancy.
- Enhanced error handling for MongoDB operations to ensure robustness in data retrieval processes.
- Added a ping check after establishing a MongoDB client connection to ensure the connection is valid.
- Implemented error logging for ping failures to improve debugging and provide clearer feedback on connection issues.
- Ensured alignment with existing error handling practices in the codebase for consistency and maintainability.
- Replaced JSON unmarshalling with string parsing to handle sorting parameters more flexibly.
- Improved handling of sorting direction by allowing both ascending and descending specifications through prefixes.
- Enhanced code readability and maintainability by simplifying the sorting logic and removing unnecessary error handling.
- Replaced "conditions" parameter with "filter" in various controller methods to unify the filtering approach.
- Updated related functions to utilize the new filter parameter, enhancing consistency in query handling.
- Improved overall code readability and maintainability by aligning naming conventions and refactoring filter-related logic.
- Enhanced error handling in BaseController to return a more descriptive "item not found" message when no documents are found in MongoDB.
- Standardized naming conventions for configuration constants by updating DefaultMcpServerBaseUrl to DefaultMCPServerBaseUrl and introducing DefaultMCPClientBaseUrl for clarity and consistency.
- Ensured alignment with existing coding standards and improved code readability.
- Updated the error handling logic in the tonic error hook to unwrap nested errors, ensuring that the most relevant error message is returned.
- Enhanced the clarity of the error response by directly using the unwrapped error message, improving debugging and user feedback.
- Updated DESCENDING constant from "dsc" to "desc" for clarity and consistency.
- Changed JSON annotation for Direction field in Sort struct from "d" to "direction" to improve readability and maintainability.
- Ensured alignment with existing coding standards and practices across the codebase.
- Introduced ChatConversation and ChatMessage structs to represent conversation and message data in the application.
- Included relevant fields with JSON and BSON annotations for proper serialization and database integration.
- Enhanced model definitions with descriptions for better understanding of their purpose and usage.
- Ensured alignment with existing coding standards and practices for maintainability.
- Added descriptive annotations to various parameters across controllers to improve API documentation clarity.
- Standardized field definitions in models by including descriptions for better understanding of their purpose.
- Updated validation patterns for ID fields to ensure consistency and improve data integrity.
- Enhanced overall code readability and maintainability by aligning naming conventions and adding necessary comments.
- Updated GetOpenAPI function to improve clarity by storing the Fizz instance in a variable.
- Added security middleware to FizzWrapper for API token authentication, enhancing security measures.
- Configured sorting of tags in the OpenAPI generator to maintain consistency in API documentation.
- Improved comments for better understanding of the changes made in the OpenAPI wrapper.
- Changed DefaultMcpServerBaseUrl from "http://localhost:9000/sse" to "http://localhost:9000" for improved clarity and consistency in configuration.
- Ensured alignment with other base URL definitions in the config.go file.
- Added github.com/getkin/kin-openapi v0.130.0 to go.mod and updated go.sum for improved OpenAPI integration.
- Introduced DefaultOpenAPIUrl constant in config.go for easier configuration of OpenAPI URL.
- Implemented GetOpenAPIUrl function to retrieve the OpenAPI URL from configuration, enhancing flexibility in API documentation handling.
- Introduced a consistent approach for sorting in multiple controller methods by utilizing GetSortOptionFromString for improved clarity and maintainability.
- Updated the GetListParams structure to ensure sorting is handled uniformly across different entities.
- Enhanced error handling for sorting parameters to provide clearer feedback on invalid formats.
- Improved comments for better understanding of sorting logic and its integration within the controller methods.
- Introduced go.opentelemetry.io/auto/sdk v1.1.0 to go.mod and updated go.sum accordingly.
- Enhanced observability capabilities by including the OpenTelemetry SDK for automatic instrumentation.
- Ensured consistency in dependency management across the project.
- Added nil return for empty condition strings in GetFilterQueryFromConditionString and GetFilterFromConditionString functions to enhance error handling.
- Improved clarity and maintainability by ensuring consistent behavior when handling empty input conditions.
- Updated comments for better understanding of the changes made.
- Updated router groups to utilize the new crawlab-team/fizz package for improved routing capabilities.
- Refactored controller methods to accept Fizz router groups, enhancing consistency and maintainability.
- Simplified route registration by incorporating OpenAPI metadata directly into group definitions.
- Improved error handling and response generation in sync controller methods for better clarity.
- Enhanced overall code structure by standardizing route definitions and improving parameter handling.
- Updated Go version to 1.23.7 in backend/go.mod.
- Upgraded various dependencies to their latest versions for improved stability and performance.
- Adjusted indirect dependencies for better compatibility.
- Enhanced overall clarity by standardizing import paths across the project.
- Updated various dependencies in go.mod and go.sum for improved stability and performance.
- Replaced references to the old fizz package with the new crawlab-team/fizz package in multiple files.
- Adjusted indirect dependencies to their latest versions for better compatibility.
- Enhanced overall code clarity by standardizing import paths across the project.
- Updated multiple controller methods to return VoidResponse instead of generic Response[any].
- Consolidated error handling to utilize GetErrorVoidResponse for consistent error responses.
- Enhanced parameter handling in export and file management functions for improved clarity and maintainability.
- Refactored health check and login/logout methods to align with new response structure.
- Improved overall consistency in response formatting across various endpoints.
- Renamed filter operation constants for clarity and consistency.
- Introduced short forms for certain filter operations.
- Updated test cases to reflect changes in filter operation handling.
- Enhanced type handling in filter functions to support int64 values for conditions.
- Improved query generation logic to accommodate new constant names and types.
- Updated GetHealthFn to return an error for better error handling and clarity.
- Introduced a new test file for schedule management, covering various endpoints including creation, retrieval, updating, and deletion of schedules.
- Added tests for task management, including task creation, retrieval, updating, and cancellation.
- Implemented utility tests for filtering and response generation to ensure consistent API behavior.
- Improved logging in the task scheduler service for better traceability.
- Refactored export and filter controller methods to utilize structured parameter types for improved clarity and maintainability.
- Consolidated error handling and response generation, ensuring consistent API responses across methods.
- Updated function signatures to eliminate unnecessary context parameters, enhancing type safety and readability.
- Improved handling of query parameters and conditions, streamlining the logic for better performance and maintainability.
- Refactored multiple controller methods to accept structured parameters for improved clarity and maintainability.
- Consolidated error handling and response generation across various endpoints.
- Updated function signatures to eliminate unnecessary context parameters and enhance type safety.
- Improved consistency in response formatting and error handling across controllers.
- Enhanced file handling methods to support multipart file uploads and directory operations more effectively.
- Updated Go version in go.work and backend/go.mod to 1.23.7
- Updated various dependencies in go.sum and backend/go.sum
- Refactored models to remove generic type parameters from BaseModel
- Introduced new utility functions for consistent API responses
- Removed unused utility files from controllers
- Introduced a new OpenAPI wrapper using Fizz for improved API documentation
- Refactored base controller to support more flexible route handling
- Added dynamic route registration with OpenAPI metadata
- Implemented generic response types for consistent API responses
- Updated router initialization to support OpenAPI documentation endpoint
- Improved route and resource naming utilities
- Migrated existing controllers to use new routing and response mechanisms
Added a new configuration utility function GetMcpServerBaseUrl() to:
- Define a default MCP server base URL
- Allow custom configuration via Viper
- Provide a consistent way to retrieve the MCP server base URL
Updated LLMProvider struct to include a new 'DeploymentName' field:
- Added 'deployment_name' tag for JSON and BSON serialization
- Modified IsUnset() method to check for deployment name in Azure OpenAI configuration
- Ensures more comprehensive validation for Azure OpenAI provider setup
Introduced a new method to check if an LLM provider is considered unset:
- Added Unset field to LLMProvider struct
- Implemented IsUnset() method with logic to validate provider configuration
- Handles special cases for different provider types (azure-openai, openai-compatible)
- Checks for empty API key, missing models, and provider-specific configuration requirements
Simplified LLMProvider model:
- Removed description, config schema, default config, and supported features
- Added new field for API version
- Cleaned up and focused on core provider attributes
Restructured LLMProvider and removed LLMModel:
- Added collection tag to LLMProvider
- Renamed and reorganized provider fields
- Removed separate LLMModel struct
- Simplified model and provider attributes
Introduced two new data models for managing Language Learning Models (LLMs):
- LLMProvider: Represents LLM providers like OpenAI, Anthropic
- LLMModel: Represents specific models within a provider
Models include key attributes such as:
- Naming and display information
- Enabled/priority status
- Supported features
- Token pricing
- Configuration schemas
Implemented a new GetTokenList handler in the token controller to:
- Fetch tokens created by the current user
- Support pagination and filtering
- Return tokens with total count
- Handle cases with no documents gracefully
Adds an early return after handling bad request errors in the GetExportDownload function to avoid unnecessary error processing and improve error handling clarity.
- Deleted CHANGELOG-zh.md and CHANGELOG.md
- Added version-specific changelog files in the changelog/ directory for v0.2, v0.3, v0.4, v0.5, and v0.6
- Included both Chinese and English versions for some changelog files
- Organized changelogs by version with detailed feature and bug fix descriptions
Reverts the previous change that set DefaultInstallEnvs to "node,browser", returning it to an empty default value to maintain configuration flexibility.
- Introduced a new function GetInstallEnvs in config.go to retrieve installation environment variables
- Added DefaultInstallEnvs constant with an empty default value
- Implemented fallback mechanism to split DefaultInstallEnvs if no configuration is found
- Enhances configuration flexibility for installation environments
