git-sync/crawlab
1
0
Fork 0
mirror of https://github.com/crawlab-team/crawlab.git synced 2026-01-23 17:31:11 +01:00
Code Issues Packages Projects Releases Wiki Activity
770 Commits 6 Branches 35 Tags
39d8a9ffd53833ee2df00d6ebdaab690bda45153
Commit Graph

49 Commits

Author SHA1 Message Date
Marvin Zhang
3eee129a7c added param to task 2019-09-04 20:13:12 +08:00
hantmac
fb58cc1a28 bug fix:fix data race caused by LockList 2019-09-04 13:51:02 +08:00
Marvin Zhang
e93a471e89 Merge pull request #208 from yaziming/v0.4.0-imporve-error-response 
feat(backend): add some feat for error message system
 2019-09-03 15:25:04 +08:00
yaziming
8ee33d3762 feat(backend): add some feat for error message system 
errors support custom http code and support return invalid validated
error
 2019-09-03 15:23:08 +08:00
yaziming
888a3cab79 refactor(backend): Use more efficient bytes to string methods and remove unnecessary type conversions 
detail:
    1. add utils.BytesToString function instead of string() convert bytes to string.
    2. use bytes.NewReader instead of strings.NewReader(string(sb)).
    3. use w.Body.Bytes() instead of []byte(w.Body.String()).
 2019-09-03 15:17:32 +08:00
陈景阳
4255b2f6b6 还原代码 2019-09-03 09:06:04 +08:00
陈景阳
9e9b117669 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into develop 2019-09-03 08:57:21 +08:00
陈景阳
41c3b98333 还原代码 2019-09-02 18:14:34 +08:00
陈景阳
45aaaa2c46 fix 无法正常删除有问题的爬虫 2019-09-02 18:04:47 +08:00
陈景阳
69fc10fcc3 fix 如果从dir读取爬虫为空，则移除所有的爬虫 2019-09-02 17:37:48 +08:00
陈景阳
64a8fab961 fix 打印日志只有2KB 2019-09-02 16:45:56 +08:00
yaziming
443d697c6f backend: 
1. Mongo dial add 5 seconds connection timeout.
 2. Redis uses connection pool mode.
 3. Redis pool new connection have 10 seconds write timeout and read timeout and connection timeout.
 2019-09-01 17:18:08 +08:00
Jeremy
0d6bd8fb0a Merge branch 'develop' into master 2019-09-01 00:13:19 +08:00
hantmac
d5dfe648af add unit test for GetLocalLog 2019-09-01 00:07:18 +08:00
hantmac
7c0c490f1c bug fix:fix read log file oom 2019-09-01 00:06:26 +08:00
yaziming
9f43e08ff9 Backend: 
improve
     - AuthMiddleware 注入当前用户的信息
     - 增加Context服务支持快捷获取当前登录者信息
     - 重构Login/GetMe接口逻辑避免重复的数据库查询
     - 规范化error信息声明(向下兼容,旧代码可逐渐迁移规范化)
     - 修正部分不符合规范的代码
 2019-08-31 21:26:56 +08:00
陈景阳
e027aeb71f 添加日志打印 2019-08-31 17:56:42 +08:00
陈景阳
1a383dac17 fix 爬虫目录无法打开的问题 2019-08-31 17:04:49 +08:00
陈景阳
c0201f566a 优化代码 2019-08-31 15:59:43 +08:00
陈景阳
1877677844 fix 无法获取日志的问题 2019-08-31 15:03:57 +08:00
陈景阳
ecaf31f0f0 优化代码 2019-08-31 13:57:14 +08:00
陈景阳
494601ab62 fix 节点注册异常情况 2019-08-31 13:49:34 +08:00
陈景阳
e43332b34f FIX #178 
FIX 当日志文件小于2048的时候无法正常拉取日志的问题
 2019-08-31 12:42:53 +08:00
陈景阳
5af4311b39 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-31 12:09:06 +08:00
陈景阳
c634e2b5cf fix 前端控制台报错的问题 \ 
fix 无法打印中文的问题
 2019-08-31 12:04:12 +08:00
Marvin Zhang
1ba497b8f6 Merge pull request #189 from wo10378931/v0.4.0 
两个fix
 2019-08-31 11:44:45 +08:00
hantmac
444784d5b6 Add unit test for log.go 2019-08-31 11:30:41 +08:00
hantmac
a9fc1951c8 Add delete log files periodically 2019-08-31 11:29:24 +08:00
hantmac
c481c66484 bug fix: hot fix out of memory problem caused by log read 2019-08-30 18:01:22 +08:00
hantmac
0512757166 bug fix: hot fix out of memory problem caused by log read 2019-08-30 17:58:58 +08:00
陈景阳
f098546a65 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-28 16:00:27 +08:00
陈景阳
1736d00161 添加打印日志 2019-08-28 16:00:09 +08:00
陈景阳
0186d08112 删除爬虫顺带删除文件 2019-08-27 09:39:16 +08:00
hantmac
f7c0e8310a bug fix:fix permission bug when create spiders dir caused by umask 2019-08-26 17:44:01 +08:00
陈景阳
0ad2a06d36 添加注释 2019-08-24 15:11:52 +08:00
Marvin Zhang
559d317ec5 Merge pull request #154 from wo10378931/master 
update 节点注册使用独立的key字段，而不是使用mac字段
 2019-08-21 15:09:29 +08:00
陈景阳
d913d7c40f fix TagsView.vue 导致前端报错的问题 
update 节点注册使用独立的key字段，而不是使用mac字段
 2019-08-21 14:37:55 +08:00
hantmac
21a68693c3 Code optimization:change the crawler zip file to slice read 2019-08-20 17:09:43 +08:00
hantmac
005abc68c8 Code optimization:change the crawler zip file to slice read 2019-08-20 16:59:53 +08:00
陈景阳
f80473a485 添加使用mac地址和ip地址注册的配置项 2019-08-11 20:01:25 +08:00
Marvin Zhang
b6d3d139a3 fixed upload bug 2019-07-29 20:31:44 +08:00
Marvin Zhang
8c7b597a3b updated Dockerfile 2019-07-28 15:27:44 +08:00
Marvin Zhang
a16f296167 added jwt 2019-07-27 00:00:59 +08:00
Marvin Zhang
be9339ff3f added user login/signup 2019-07-26 23:02:30 +08:00
Marvin Zhang
253c643e8e added spider stats 2019-07-26 17:07:24 +08:00
Marvin Zhang
fe017b4769 fixed bugs 2019-07-23 12:44:29 +08:00
Marvin Zhang
072e60e17e fetch sysinfo from remote 2019-07-22 22:25:29 +08:00
Marvin Zhang
8df06ba278 added sysinfo 2019-07-22 13:33:09 +08:00
Marvin Zhang
53cebde962 added golang 2019-07-22 12:51:52 +08:00