git-sync/crawlab

mirror of https://github.com/crawlab-team/crawlab.git synced 2026-01-22 17:31:03 +01:00

Files

History

Marvin Zhang 18c5eb3956 fix: replace string slicing with filepath.Dir() in gRPC file sync

- Fix directory path calculation bug in downloadFileGRPC()
- Bug caused nested directory creation to fail (e.g., crawlab_project/spiders/)
- String slicing incorrectly truncated paths mid-character
- Now uses filepath.Dir() for correct parent directory extraction
- Fixes 'no such file or directory' errors during worker file sync
- Resolves spider task failures on worker nodes after gRPC migration

Validated by: REL-004, REL-005 test cases

2025-10-30 15:22:53 +08:00

..

.github/workflows

feat: disable backend unit tests and document reasons for integration test requirements

2025-10-09 12:35:09 +08:00

feat: enhance API routing and OpenAPI documentation support

2025-03-11 23:45:06 +08:00

feat: add logo printing functionality and update logger usage

2024-12-25 13:08:56 +08:00

fix: add nil checks for logger in config initialization and logging methods

2025-07-23 15:07:39 +08:00

feat: add node disconnection handling and update task statuses accordingly

2025-09-12 15:40:29 +08:00

fix(core): default to _id descending sort when sort is nil or parsing fails

2025-10-28 11:55:00 +08:00

refactor: renamed files and services

2024-11-05 11:15:27 +08:00

feat: add dependency installer service and enhance task runner with dependency management

2025-01-01 20:51:55 +08:00

fix: Update FsFileInfo to use pointer slices for children and remove redundant getter methods

2025-07-23 09:42:07 +08:00

refactor: code cleanup

2024-11-01 15:19:48 +08:00

refactor: standardize timestamp fields and improve code clarity

2025-04-21 18:13:22 +08:00

fix: Update FsFileInfo to use pointer slices for children and remove redundant getter methods

2025-07-23 09:42:07 +08:00

fix(grpc/server): add keepalive enforcement and params to match client

2025-10-23 10:58:22 +08:00

fix: unable to sync files to worker nodes when running tasks

2025-06-18 21:50:53 +08:00

refactor: extract token retrieval logic into GetAPITokenFromContext utility function

2025-06-16 21:49:27 +08:00

fix(core): default to _id descending sort when sort is nil or parsing fails

2025-10-28 11:55:00 +08:00

refactor: improve mongo client connection error logging format and remove redundant gRPC server start in MasterService

2025-07-09 14:06:10 +08:00

fix(node/service): implement worker client wrapper using local TaskServiceServer

2025-10-27 16:26:32 +08:00

fix: update NotificationRequestDTO to include BSON field names for setting and channel

2025-09-16 15:43:43 +08:00

refactor: enhance OpenAPI integration and security handling

2025-03-18 18:01:22 +08:00

feat: implement bounded goroutine pools for task execution and notification handling; enhance task scheduler with graceful shutdown and cleanup routines; update metric component for new time range options

2025-08-06 17:57:37 +08:00

refactor: enhance parameter handling and improve code clarity

2025-04-17 18:03:15 +08:00

refactor: update setting routes and enhance dependency management

2025-01-01 22:37:44 +08:00

fix: replace string slicing with filepath.Dir() in gRPC file sync

2025-10-30 15:22:53 +08:00

refactor: replace apex/log with structured logger across multiple services

2024-12-24 19:11:19 +08:00

feat(sync): add gRPC file synchronization service and integrate end-to-end

2025-10-20 12:48:53 +08:00

.editorconfig

feat: added modules

2024-06-14 15:42:50 +08:00

.gitignore

feat: added modules

2024-06-14 15:42:50 +08:00

BACKEND_TESTS_DISABLED.md

feat: disable backend unit tests and document reasons for integration test requirements

2025-10-09 12:35:09 +08:00

docker-compose.yml

ci: updated unit test

2024-10-20 18:55:02 +08:00

go.mod

fix: unable to filter tasks issue

2025-06-17 09:28:25 +08:00

go.sum

fix: unable to filter tasks issue

2025-06-17 09:28:25 +08:00

LICENSE

feat: added modules

2024-06-14 15:42:50 +08:00

main.go

refactor: code cleanup

2024-10-29 12:59:45 +08:00

README.md

feat: added modules

2024-06-14 15:42:50 +08:00

README.md

crawlab-core

Backend core modules for Crawlab