git-sync/crawlab
1
0
Fork 0
mirror of https://github.com/crawlab-team/crawlab.git synced 2026-01-23 17:31:11 +01:00
Code Issues Packages Projects Releases Wiki Activity
743 Commits 6 Branches 35 Tags
ec328ae02ba137ae9a6ea73ed159849e9e5cde4e
Commit Graph

107 Commits

Author SHA1 Message Date
yaziming
ec328ae02b Remove redundant fmt.Println 2019-09-02 22:57:29 +08:00
yaziming
2b579cc642 break loop when subscribe count is 0 2019-09-02 18:32:47 +08:00
yaziming
443d697c6f backend: 
1. Mongo dial add 5 seconds connection timeout.
 2. Redis uses connection pool mode.
 3. Redis pool new connection have 10 seconds write timeout and read timeout and connection timeout.
 2019-09-01 17:18:08 +08:00
Marvin Zhang
d97134b288 Merge pull request #195 from hantmac/master 
bug fix:fix read log file oom
 2019-09-01 11:09:02 +08:00
yaziming
7574ceb0cf Backend: 
upgrade gin framework depend validator version : v8 -> v9
 2019-09-01 10:13:53 +08:00
Jeremy
0d6bd8fb0a Merge branch 'develop' into master 2019-09-01 00:13:19 +08:00
hantmac
d5dfe648af add unit test for GetLocalLog 2019-09-01 00:07:18 +08:00
hantmac
7c0c490f1c bug fix:fix read log file oom 2019-09-01 00:06:26 +08:00
yaziming
9f43e08ff9 Backend: 
improve
     - AuthMiddleware 注入当前用户的信息
     - 增加Context服务支持快捷获取当前登录者信息
     - 重构Login/GetMe接口逻辑避免重复的数据库查询
     - 规范化error信息声明(向下兼容,旧代码可逐渐迁移规范化)
     - 修正部分不符合规范的代码
 2019-08-31 21:26:56 +08:00
陈景阳
e027aeb71f 添加日志打印 2019-08-31 17:56:42 +08:00
陈景阳
1a383dac17 fix 爬虫目录无法打开的问题 2019-08-31 17:04:49 +08:00
陈景阳
18d66d8ad3 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-31 16:54:26 +08:00
陈景阳
7a901ff257 添加备注 
添加上次运行状态
 2019-08-31 16:09:41 +08:00
陈景阳
c0201f566a 优化代码 2019-08-31 15:59:43 +08:00
Marvin Zhang
5afc0ca490 Merge pull request #192 from yaziming/forked/develop 
Some Change.
 2019-08-31 15:15:35 +08:00
陈景阳
1877677844 fix 无法获取日志的问题 2019-08-31 15:03:57 +08:00
yaziming
d8acab7753 Backend: 
1.[improve]
       使用gin提供的RouteGroup功能简化中间件逻辑
    2.[break change]
       移除Authorization Middleware中对登录注册下载特殊处理逻辑
    3.[unsafe problem]
       下载任务csv增加登录验证
Frontend:
    1. 更改csv下载方式
    2. task list页面table section 移除固定width设置,采用自适应,防止大屏空白断裂问题
 2019-08-31 15:00:38 +08:00
陈景阳
ecaf31f0f0 优化代码 2019-08-31 13:57:14 +08:00
陈景阳
494601ab62 fix 节点注册异常情况 2019-08-31 13:49:34 +08:00
陈景阳
e43332b34f FIX #178 
FIX 当日志文件小于2048的时候无法正常拉取日志的问题
 2019-08-31 12:42:53 +08:00
Marvin Zhang
88d8810b3a Merge pull request #191 from wo10378931/v0.4.0 
三个FIX
 2019-08-31 12:15:34 +08:00
陈景阳
5af4311b39 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-31 12:09:06 +08:00
陈景阳
c634e2b5cf fix 前端控制台报错的问题 \ 
fix 无法打印中文的问题
 2019-08-31 12:04:12 +08:00
Marvin Zhang
1ba497b8f6 Merge pull request #189 from wo10378931/v0.4.0 
两个fix
 2019-08-31 11:44:45 +08:00
Marvin Zhang
dcad7bf2ca Merge pull request #190 from hantmac/master 
Add delete log files periodically
 2019-08-31 11:44:17 +08:00
hantmac
444784d5b6 Add unit test for log.go 2019-08-31 11:30:41 +08:00
hantmac
80b92e1084 Init delete log files periodically 2019-08-31 11:30:01 +08:00
hantmac
a9fc1951c8 Add delete log files periodically 2019-08-31 11:29:24 +08:00
hantmac
95faaa0038 add option for config.yml to config log delete periodically 2019-08-31 11:28:16 +08:00
Marvin Zhang
8633e851a5 Merge pull request #187 from hantmac/master 
bug fix: hot fix out of memory problem caused by log read
 2019-08-30 18:05:18 +08:00
hantmac
c481c66484 bug fix: hot fix out of memory problem caused by log read 2019-08-30 18:01:22 +08:00
hantmac
0512757166 bug fix: hot fix out of memory problem caused by log read 2019-08-30 17:58:58 +08:00
陈景阳
f098546a65 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-28 16:00:27 +08:00
陈景阳
1736d00161 添加打印日志 2019-08-28 16:00:09 +08:00
Marvin Zhang
03e99d6872 Merge pull request #179 from hantmac/master 
bug fix: fix test failed for config.go
 2019-08-27 14:24:23 +08:00
hantmac
db81982725 bug fix: fix test failed for config.go 2019-08-27 14:21:39 +08:00
陈景阳
0186d08112 删除爬虫顺带删除文件 2019-08-27 09:39:16 +08:00
Marvin Zhang
2bcd147a60 Merge pull request #176 from yaziming/forked/develop 
JSON -> AbortWithStatusJSON
 2019-08-27 08:45:49 +08:00
yaziming
b11f31ec18 JSON -> AbortWithStatusJSON 2019-08-26 23:13:26 +08:00
hantmac
9530a3d9b0 add unit test for routes/spider.go 2019-08-26 21:10:55 +08:00
hantmac
f7c0e8310a bug fix:fix permission bug when create spiders dir caused by umask 2019-08-26 17:44:01 +08:00
hantmac
9855d9ef6a hot fix:fix redis lose connection problem 2019-08-26 10:38:40 +08:00
陈景阳
efedf587a4 Merge branch 'master' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-24 17:40:03 +08:00
Marvin Zhang
8a82699f06 Merge pull request #167 from hantmac/develop 
remove redundant code
 2019-08-24 16:23:28 +08:00
hantmac
366836d0aa remove redundant code 2019-08-24 15:50:53 +08:00
陈景阳
527f33abb4 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into v0.4.0 2019-08-24 15:25:05 +08:00
Marvin Zhang
2981fec602 Merge pull request #166 from hantmac/develop 
bug fix: fix redis connection dead cycle
 2019-08-24 15:21:59 +08:00
hantmac
f789352ad3 bug fix:fix redis connection dead cycle 2019-08-24 15:16:27 +08:00
陈景阳
0ad2a06d36 添加注释 2019-08-24 15:11:52 +08:00
Marvin Zhang
12016de0f5 Merge pull request #164 from hantmac/develop 
Add unit test for task.go
 2019-08-24 13:41:04 +08:00