143 Commits

Author	SHA1	Message	Date
陈景阳	d977252eba	fix	2019-09-27 10:14:48 +08:00
陈景阳	412e0fd983	fix bug	2019-09-26 21:13:25 +08:00
陈景阳	f14085666b	完成爬虫列表	2019-09-26 20:54:50 +08:00
陈景阳	41e9edf940	完成爬虫列表	2019-09-26 20:53:05 +08:00
陈景阳	5416b7ac60	fix 上传的问题	2019-09-26 19:44:12 +08:00
陈景阳	79ee09aacb	完成爬虫获取	2019-09-26 19:12:02 +08:00
陈景阳	d27f3c04d6	fix 可能直接退出的问题	2019-09-26 16:49:01 +08:00
陈景阳	6c37731cca	优化爬虫获取逻辑	2019-09-26 16:43:32 +08:00
陈景阳	6df6155daf	优化爬虫获取逻辑	2019-09-26 16:26:32 +08:00
陈景阳	475bf4c69e	优化爬虫获取逻辑	2019-09-26 11:38:13 +08:00
陈景阳	9237e62a48	爬虫逻辑修改为从GridFS获取	2019-09-26 11:28:20 +08:00
陈景阳	c37c0b963b	fix 无法终止任务的问题	2019-09-26 10:29:58 +08:00
陈景阳	cde06a0000	fix 无法终止任务的问题	2019-09-26 10:23:08 +08:00
陈景阳	52b7577e0e	添加日志打印	2019-09-26 09:37:10 +08:00
陈景阳	48dcf5a952	添加日志打印	2019-09-25 16:19:58 +08:00
陈景阳	613b5eb139	fix 无法获取task的错误	2019-09-25 15:11:27 +08:00
陈景阳	fe221ffd6f	fix	2019-09-25 14:38:46 +08:00
陈景阳	7e17adf6b0	fix 空指针引用	2019-09-10 14:47:23 +08:00
陈景阳	f20039fefd	fix 消息无法订阅问题 ... fix 可能出现重复爬虫的问题	2019-09-10 14:26:50 +08:00
陈景阳	27f36cf91f	删除日志	2019-09-10 09:51:27 +08:00
陈景阳	9514a8a6af	重构msg的包	2019-09-10 09:32:48 +08:00
陈景阳	e5d33b8982	fix 删除spider没有删除task	2019-09-05 20:07:08 +08:00
Marvin Zhang	3eee129a7c	added param to task	2019-09-04 20:13:12 +08:00
hantmac	fb58cc1a28	bug fix:fix data race caused by LockList	2019-09-04 13:51:02 +08:00
Marvin Zhang	e93a471e89	Merge pull request #208 from yaziming/v0.4.0-imporve-error-response ... feat(backend): add some feat for error message system	2019-09-03 15:25:04 +08:00
yaziming	8ee33d3762	feat(backend): add some feat for error message system ... errors support custom http code and support return invalid validated error	2019-09-03 15:23:08 +08:00
yaziming	888a3cab79	refactor(backend): Use more efficient bytes to string methods and remove unnecessary type conversions ... detail: 1. add utils.BytesToString function instead of string() convert bytes to string. 2. use bytes.NewReader instead of strings.NewReader(string(sb)). 3. use w.Body.Bytes() instead of []byte(w.Body.String()).	2019-09-03 15:17:32 +08:00
hantmac	2ffd1774bf	bug fix:fix go test ./.. bug	2019-09-03 12:47:18 +08:00
陈景阳	4255b2f6b6	还原代码	2019-09-03 09:06:04 +08:00
陈景阳	9e9b117669	Merge branch 'develop' of https://github.com/crawlab-team/crawlab into develop	2019-09-03 08:57:21 +08:00
yaziming	ec328ae02b	Remove redundant fmt.Println	2019-09-02 22:57:29 +08:00
陈景阳	6675d075f2	fix 删除spider没有删除task的问题	2019-09-02 20:49:00 +08:00
yaziming	2b579cc642	break loop when subscribe count is 0	2019-09-02 18:32:47 +08:00
陈景阳	41c3b98333	还原代码	2019-09-02 18:14:34 +08:00
陈景阳	45aaaa2c46	fix 无法正常删除有问题的爬虫	2019-09-02 18:04:47 +08:00
陈景阳	69fc10fcc3	fix 如果从dir读取爬虫为空，则移除所有的爬虫	2019-09-02 17:37:48 +08:00
陈景阳	64a8fab961	fix 打印日志只有2KB	2019-09-02 16:45:56 +08:00
陈景阳	1e37d714b0	添加排序	2019-09-02 10:40:16 +08:00
yaziming	443d697c6f	backend: ... 1. Mongo dial add 5 seconds connection timeout. 2. Redis uses connection pool mode. 3. Redis pool new connection have 10 seconds write timeout and read timeout and connection timeout.	2019-09-01 17:18:08 +08:00
Marvin Zhang	d97134b288	Merge pull request #195 from hantmac/master ... bug fix:fix read log file oom	2019-09-01 11:09:02 +08:00
yaziming	7574ceb0cf	Backend: ... upgrade gin framework depend validator version : v8 -> v9	2019-09-01 10:13:53 +08:00
Jeremy	0d6bd8fb0a	Merge branch 'develop' into master	2019-09-01 00:13:19 +08:00
hantmac	d5dfe648af	add unit test for GetLocalLog	2019-09-01 00:07:18 +08:00
hantmac	7c0c490f1c	bug fix:fix read log file oom	2019-09-01 00:06:26 +08:00
yaziming	9f43e08ff9	Backend: ... improve - AuthMiddleware 注入当前用户的信息 - 增加Context服务支持快捷获取当前登录者信息 - 重构Login/GetMe接口逻辑避免重复的数据库查询 - 规范化error信息声明(向下兼容,旧代码可逐渐迁移规范化) - 修正部分不符合规范的代码	2019-08-31 21:26:56 +08:00
陈景阳	e027aeb71f	添加日志打印	2019-08-31 17:56:42 +08:00
陈景阳	1a383dac17	fix 爬虫目录无法打开的问题	2019-08-31 17:04:49 +08:00
陈景阳	18d66d8ad3	Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0	2019-08-31 16:54:26 +08:00
陈景阳	7a901ff257	添加备注 ... 添加上次运行状态	2019-08-31 16:09:41 +08:00
陈景阳	c0201f566a	优化代码	2019-08-31 15:59:43 +08:00