Commit Graph

353 Commits

Author SHA1 Message Date
Marvin Zhang
14a94ff798 refactor: enhance error logging in writeLogLines to respect circuit breaker state 2025-09-12 14:34:27 +08:00
Marvin Zhang
c0e230e5d8 refactor: rename PING code to HEARTBEAT in node service and update related proto files 2025-09-12 14:17:49 +08:00
Marvin Zhang
d39c265483 feat: add PING message handling for connection health checks
- Implemented PING message handling in TaskServiceServer to acknowledge health check pings.
- Updated isConnectionHealthy method in Runner to use a non-blocking approach for health checks, preventing interference with log streams.
- Introduced lastConnCheck timestamp to optimize health check frequency based on recent activity.
- Added PING code to TaskServiceConnectCode enum in proto definition and generated files.
- Updated gRPC client and server interfaces to support new PING functionality.
2025-09-12 13:58:16 +08:00
Marvin Zhang
333dfd44c0 refactor: implement circuit breaker for log connections to prevent flooding during failures 2025-09-12 13:55:44 +08:00
Marvin Zhang
3edd2a1210 refactor: optimize connection health checks to reduce log stream interference; adjust health check intervals and implement non-blocking pings 2025-08-16 17:42:07 +08:00
Marvin Zhang
65aeb3ed8c feat: add PING mechanism for connection health checks; update proto and generated files
- Introduced PING code in TaskServiceConnectCode enum for health checks.
- Updated Runner to use proper PING messages instead of fake log messages for connection health checks.
- Modified TaskServiceServer to handle PING requests and acknowledge them.
- Adjusted generated gRPC files to reflect changes in proto definitions and ensure compatibility.
2025-08-16 17:19:21 +08:00
Marvin Zhang
45913ad7e4 refactor: implement health service for master and worker nodes; add health check script and integrate health checks into service lifecycle 2025-08-08 00:05:00 +08:00
Marvin Zhang
78f9e0ca8d refactor: update task worker pool to support dynamic max workers and improve queue management; enhance configuration defaults for node runners and task queue size 2025-08-07 18:16:23 +08:00
Marvin Zhang
6340a9b880 refactor: Move context initialization for graceful shutdown to appropriate locations 2025-08-07 17:27:11 +08:00
Marvin Zhang
6912b92501 refactor: enhance context handling across task runner and service components; ensure proper cancellation chains and prevent goroutine leaks 2025-08-07 15:40:48 +08:00
Marvin Zhang
e1251d808b refactor: update method receivers to value type for cleanup and connection methods; enhance context usage for task client operations 2025-08-07 11:53:42 +08:00
Marvin Zhang
d042bc8cd7 refactor: improve connection readiness check and enhance goroutine management in gRPC client; ensure proper context handling in stream listeners 2025-08-07 11:12:46 +08:00
Marvin Zhang
060396af3d refactor: enhance data structure annotations and improve layout responsiveness in Home component 2025-08-07 09:58:11 +08:00
Marvin Zhang
44dd68918f refactor: improve goroutine management and context handling in task and stream operations; ensure graceful shutdown and prevent leaks 2025-08-07 00:16:46 +08:00
Marvin Zhang
784ffc8b52 feat: implement task management service operations, stream manager, and worker pool
- Added service_operations.go for task management including run, cancel, and execution logic.
- Introduced stream_manager.go to handle task streams and manage cancellation signals.
- Created worker_pool.go to manage a bounded pool of workers for executing tasks concurrently.
- Implemented graceful shutdown and cleanup mechanisms for task runners and streams.
- Enhanced error handling and logging throughout the task management process.
2025-08-06 18:29:08 +08:00
Marvin Zhang
3678d14082 feat: implement bounded goroutine pools for task execution and notification handling; enhance task scheduler with graceful shutdown and cleanup routines; update metric component for new time range options 2025-08-06 17:57:37 +08:00
Marvin Zhang
9745129e33 feat: configure test database as master node for testing 2025-07-23 15:28:51 +08:00
Marvin Zhang
cf5ec81250 fix: add nil checks for logger in config initialization and logging methods 2025-07-23 15:07:39 +08:00
Marvin Zhang
a2d13fae36 feat: temporarily disable batch file saving route and implement alternative handler in spider controller 2025-07-23 14:55:04 +08:00
Marvin Zhang
b4288b08a5 fix: Update FsFileInfo to use pointer slices for children and remove redundant getter methods 2025-07-23 09:42:07 +08:00
Marvin Zhang
3c3ff09723 feat: enhance gRPC client with health check functionality and improved connection handling 2025-07-09 14:38:53 +08:00
Marvin Zhang
20ba390cf6 refactor: improve mongo client connection error logging format and remove redundant gRPC server start in MasterService 2025-07-09 14:06:10 +08:00
Marvin Zhang
46c0cd6298 refactor: update gRPC client access patterns to use safe getter methods for improved error handling 2025-07-08 18:08:46 +08:00
Marvin Zhang
8bd3ef0b72 feat: add goroutine count metric to the MetricService and update related files 2025-07-08 14:08:31 +08:00
Marvin Zhang
00daa0ed96 fix: enhance gRPC client reconnection logic and add goroutine monitoring for potential leaks 2025-07-08 13:39:39 +08:00
Marvin Zhang
f8e9c45a85 fix: enhance gRPC client connection management with circuit breaker and keep-alive settings 2025-07-08 13:34:43 +08:00
Marvin Zhang
92046a8c2e fix: improve task cancellation and connection health check logic with timeout handling 2025-06-27 14:02:24 +08:00
Marvin Zhang
9f251f3ebe fix: enhance task cancellation logic with graceful termination and stuck task cleanup 2025-06-27 13:50:21 +08:00
Marvin Zhang
89514b0154 feat: implement zombie process prevention and cleanup mechanisms in task runner 2025-06-23 13:54:43 +08:00
Marvin Zhang
1008886715 fix: enhance task service resilience with connection health monitoring and periodic cleanup 2025-06-23 11:57:05 +08:00
Marvin Zhang
5837472de5 fix: update API client constructor and improve error handling for missing API token and URL
chore: update package version to 0.1.1-dev.6 and adjust dependencies in pnpm-lock.yaml
refactor: change import paths to relative in tools.ts for consistency
2025-06-20 15:55:50 +08:00
Marvin Zhang
13038d4a1a fix: unable to sync files from master in worker nodes 2025-06-20 14:42:52 +08:00
Marvin Zhang
09cfe37272 fix: unable to sync files to worker nodes when running tasks 2025-06-18 21:50:53 +08:00
Marvin Zhang
22497d22f7 fix: unable to filter tasks issue 2025-06-17 09:28:25 +08:00
Marvin Zhang
24452bff23 refactor: extract token retrieval logic into GetAPITokenFromContext utility function 2025-06-16 21:49:27 +08:00
Marvin Zhang
6d65cc13a2 feat: add aggregation methods to ModelService for enhanced data retrieval 2025-06-15 18:03:35 +08:00
Marvin Zhang
ab5e65f2fb chore: update dependencies to latest versions across modules 2025-06-15 18:03:13 +08:00
Marvin Zhang
335050e72e refactor: improve schedule service initialization, validation, and error handling 2025-06-13 21:53:08 +08:00
Marvin Zhang
0bd2b62792 feat: add DTOs for models and update project spiders type to array 2025-06-12 19:58:21 +08:00
Marvin Zhang
79b7e074e1 refactor: update filter parameter in API requests and improve component structure 2025-06-12 00:17:40 +08:00
Marvin Zhang
622dce51c3 fix: goroutine cleanup and error handling during shutdown 2025-06-11 22:30:37 +08:00
Marvin Zhang
de69c2b01b refactor: replace activeNodes with allList in dependency dialogs and related components 2025-06-11 13:34:08 +08:00
Marvin Zhang
1f3a9586b3 feat: enhance data model associations and remove unused components 2025-06-09 23:00:47 +08:00
Marvin Zhang
975c601f38 refactor: simplify list retrieval methods and enhance error handling 2025-06-09 10:21:00 +08:00
Marvin Zhang
8a00af115c feat: refactor to remove unused 'getAllList' methods and related state properties 2025-06-09 10:20:39 +08:00
Marvin Zhang
8d32d54fe8 feat: add development and index generation run configurations; update SVG files and improve icon styles 2025-06-06 14:59:59 +08:00
Marvin Zhang
062077a9bd feat: restructure user parameters for PostUser and simplify email validation; refactor UserAvatar styles and improve user name handling 2025-05-29 22:16:27 +08:00
Marvin Zhang
512f246094 feat: implement AutoProbe components and enhance LLM provider functionality
- Introduced AutoProbeForm and AutoProbeTaskStatus components for managing AutoProbe configurations and task statuses.
- Updated LLMProvider model to include a default model field for improved provider management.
- Enhanced AutoProbeList and AutoProbeDetail components for better user interaction and data display.
- Refactored related Vue components and services to streamline AutoProbe functionality and improve state management.
- Added internationalization support for new AutoProbe features and updated existing translations.
2025-05-12 17:37:30 +08:00
Marvin Zhang
9e25ea9b51 refactor: standardize LLM provider key naming and enhance related components
- Renamed 'Key' fields to 'Type' in various models for consistency across the codebase.
- Updated route naming in router.go for clarity in the context of settings.
- Introduced a new LlmProviderForm component for managing LLM provider configurations.
- Refactored AssistantConsole and related components to utilize providerId instead of provider for improved state management.
- Enhanced UI elements and translations to reflect the new naming conventions and improve user experience.
2025-05-09 15:35:57 +08:00
Marvin Zhang
73b3c1bb70 refactor: add IgnoreFileRegexPattern constant for directory scanning
- Introduced a new constant `IgnoreFileRegexPattern` to specify patterns for files to ignore during directory scanning.
- Adjusted import order in `file.go` for improved organization and clarity.
2025-05-04 21:44:27 +08:00