Marvin Zhang
7cbe17723d
Merge pull request #1606 from crawlab-team/develop
...
feat: update test workflows to skip API tests and document controller…
2025-10-09 12:43:38 +08:00
Marvin Zhang
883c954b4e
feat: update Dockerfile to include 'go mod tidy' before installation
2025-10-09 12:39:12 +08:00
Marvin Zhang
44fd0809e6
feat: disable backend unit tests and document reasons for integration test requirements
2025-10-09 12:35:09 +08:00
Marvin Zhang
5bff8823a8
feat: update test workflows to skip API tests and document controller test status
2025-10-09 11:34:26 +08:00
Marvin Zhang
5a0a640900
Merge pull request #1605 from crawlab-team/develop
...
Develop
2025-10-09 11:17:07 +08:00
Marvin Zhang
2a211923da
Update core/task/handler/runner_sync.go
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-10-09 11:14:32 +08:00
Marvin Zhang
587d9d0960
Merge branch 'test' into develop
2025-10-09 11:13:51 +08:00
Marvin Zhang
4c508557e7
feat: parameterize ports in Docker Compose for better configurability
2025-09-29 14:31:33 +08:00
Marvin Zhang
29ef8d67da
feat: implement synchronization and error handling improvements in task reconciliation and file synchronization
2025-09-28 17:42:23 +08:00
Marvin Zhang
e80256aa61
feat: add support for Chinese locale in Docker setup
2025-09-17 16:00:27 +08:00
Marvin Zhang
b6e14a13fe
refactor: remove obsolete task reconciliation service tests
2025-09-17 11:05:27 +08:00
Marvin Zhang
afa5fab4c1
feat: enhance task reconciliation with worker-side status caching and synchronization
2025-09-17 11:03:35 +08:00
Marvin Zhang
8c2c23d9b6
feat: Update gRPC service definitions and implement CheckProcess method
...
- Downgraded protoc-gen-go-grpc and protoc versions for compatibility.
- Added CheckProcess method to TaskService with corresponding request and response types.
- Updated Subscribe and Connect methods to use new generic client stream types.
- Refactored server and client implementations for Subscribe and Connect methods.
- Ensured backward compatibility by maintaining existing method signatures where applicable.
- Added necessary handler for CheckProcess in the service descriptor.
2025-09-17 10:37:03 +08:00
Marvin Zhang
c6834e9964
feat: enhance task reconciliation logic with improved status handling and error messaging
2025-09-17 10:18:13 +08:00
Marvin Zhang
8ebdd98f99
feat: enhance ORM functionality with toggle support and UI updates
2025-09-16 16:09:33 +08:00
Marvin Zhang
bfe40e7c67
fix: comment out AI Assistant toggle button in Header.vue
2025-09-16 15:45:56 +08:00
Marvin Zhang
39f83d71b1
fix: update NotificationRequestDTO to include BSON field names for setting and channel
2025-09-16 15:43:43 +08:00
Marvin Zhang
196273c423
feat: implement ORM support with toggle functionality and UI updates
2025-09-16 15:18:35 +08:00
Marvin Zhang
875ca290b5
fix: update UseORM description to specify supported databases (MySQL, PostgreSQL, SQL Server)
2025-09-16 14:04:01 +08:00
Marvin Zhang
293e630f6f
feat: add UseORM field to Database struct for ORM support
2025-09-16 13:30:50 +08:00
Marvin Zhang
8450b074c0
fix: comment out unused 'models' menu item in SystemDetail.vue
2025-09-16 09:33:15 +08:00
Marvin Zhang
56277e47be
fix: remove unused build scripts to streamline the build process
2025-09-16 09:32:23 +08:00
Marvin Zhang
9ade564d0a
Remove unused TypeScript declaration files for task, token, and user components in the Crawlab UI, streamlining the codebase and improving maintainability.
2025-09-16 09:31:34 +08:00
Marvin Zhang
7d1a61581e
feat: add support for multi-architecture Docker builds with configurable input
2025-09-14 16:39:29 +08:00
Marvin Zhang
72177b2728
fix: update node disconnected status styling and behavior
2025-09-14 15:20:01 +08:00
Marvin Zhang
437c30b699
fix: ensure worker services depend on healthy master service
2025-09-14 15:02:06 +08:00
Marvin Zhang
829fcac3ff
feat: add multi-platform support for Docker builds
2025-09-14 14:49:53 +08:00
Marvin Zhang
7c33fec784
refactor: remove unused fields from WorkerService struct
2025-09-12 18:17:36 +08:00
Marvin Zhang
e221e3c640
feat: enhance gRPC client handling with improved reconnection logic and monitoring
2025-09-12 18:16:52 +08:00
Marvin Zhang
07bb7f8ba9
fix: enhance node metrics handling by checking state before accessing metrics map
2025-09-12 16:34:40 +08:00
Marvin Zhang
316878e129
test: add comprehensive tests for task reconciliation service handling offline nodes
2025-09-12 16:10:00 +08:00
Marvin Zhang
60be5072e5
feat: add node disconnection handling and update task statuses accordingly
2025-09-12 15:40:29 +08:00
Marvin Zhang
05a71da26e
refactor: comment out AI chat features in NormalLayout.vue for future implementation
2025-09-12 15:01:29 +08:00
Marvin Zhang
b5f10cb6a8
feat: add TypeScript interfaces for Vuex store modules
...
- Introduced new interfaces for various store modules including environment, file, git, layout, node, notification alerts, channels, requests, settings, plugins, projects, roles, schedules, spiders, systems, tags, tasks, tokens, and users.
- Each module includes state, getters, mutations, and actions definitions to enhance type safety and maintainability.
- Added utility interfaces for file handling and view-specific types for database, git, login, node, project, result, schedule, spider, task, and user.
- Improved overall structure and organization of TypeScript typings for better developer experience.
2025-09-12 14:57:42 +08:00
Marvin Zhang
14a94ff798
refactor: enhance error logging in writeLogLines to respect circuit breaker state
2025-09-12 14:34:27 +08:00
Marvin Zhang
c0e230e5d8
refactor: rename PING code to HEARTBEAT in node service and update related proto files
2025-09-12 14:17:49 +08:00
Marvin Zhang
d39c265483
feat: add PING message handling for connection health checks
...
- Implemented PING message handling in TaskServiceServer to acknowledge health check pings.
- Updated isConnectionHealthy method in Runner to use a non-blocking approach for health checks, preventing interference with log streams.
- Introduced lastConnCheck timestamp to optimize health check frequency based on recent activity.
- Added PING code to TaskServiceConnectCode enum in proto definition and generated files.
- Updated gRPC client and server interfaces to support new PING functionality.
2025-09-12 13:58:16 +08:00
Marvin Zhang
333dfd44c0
refactor: implement circuit breaker for log connections to prevent flooding during failures
2025-09-12 13:55:44 +08:00
Marvin Zhang
3edd2a1210
refactor: optimize connection health checks to reduce log stream interference; adjust health check intervals and implement non-blocking pings
2025-08-16 17:42:07 +08:00
Marvin Zhang
65aeb3ed8c
feat: add PING mechanism for connection health checks; update proto and generated files
...
- Introduced PING code in TaskServiceConnectCode enum for health checks.
- Updated Runner to use proper PING messages instead of fake log messages for connection health checks.
- Modified TaskServiceServer to handle PING requests and acknowledge them.
- Adjusted generated gRPC files to reflect changes in proto definitions and ensure compatibility.
2025-08-16 17:19:21 +08:00
Marvin Zhang
babecc46c0
refactor: update DependencySetupDialog to use language key for tag label; modify useDependencyList to dispatch node retrieval on mount
2025-08-08 11:12:05 +08:00
Marvin Zhang
45913ad7e4
refactor: implement health service for master and worker nodes; add health check script and integrate health checks into service lifecycle
2025-08-08 00:05:00 +08:00
Marvin Zhang
78f9e0ca8d
refactor: update task worker pool to support dynamic max workers and improve queue management; enhance configuration defaults for node runners and task queue size
2025-08-07 18:16:23 +08:00
Marvin Zhang
6340a9b880
refactor: Move context initialization for graceful shutdown to appropriate locations
2025-08-07 17:27:11 +08:00
Marvin Zhang
6912b92501
refactor: enhance context handling across task runner and service components; ensure proper cancellation chains and prevent goroutine leaks
2025-08-07 15:40:48 +08:00
Marvin Zhang
e1251d808b
refactor: update method receivers to value type for cleanup and connection methods; enhance context usage for task client operations
2025-08-07 11:53:42 +08:00
Marvin Zhang
f5f83a010f
refactor: update metric list structure for improved layout and responsiveness; enhance styling with grid display
2025-08-07 11:12:50 +08:00
Marvin Zhang
d042bc8cd7
refactor: improve connection readiness check and enhance goroutine management in gRPC client; ensure proper context handling in stream listeners
2025-08-07 11:12:46 +08:00
Marvin Zhang
060396af3d
refactor: enhance data structure annotations and improve layout responsiveness in Home component
2025-08-07 09:58:11 +08:00
Marvin Zhang
44dd68918f
refactor: improve goroutine management and context handling in task and stream operations; ensure graceful shutdown and prevent leaks
2025-08-07 00:16:46 +08:00
