50 Commits

Author	SHA1	Message	Date
陈景阳	bff13e96dc	Merge remote-tracking branch 'upstream/develop' into upstream-develop	2019-12-07 10:58:24 +08:00
marvzhang	f6ac1e00b3	Merge remote-tracking branch 'origin/develop' into develop	2019-12-05 11:52:08 +08:00
marvzhang	e8edf9dac4	添加运行类别	2019-12-05 11:51:54 +08:00
陈景阳	b445457243	Merge remote-tracking branch 'upstream/develop' into upstream-develop	2019-12-05 07:18:17 +08:00
陈景阳	761e5257da	Merge remote-tracking branch 'upstream/develop' into develop	2019-12-05 07:15:07 +08:00
marvzhang	7c1e52da88	Merge remote-tracking branch 'origin/develop' into develop	2019-12-02 13:32:44 +08:00
hantmac	f58c42c6be	fix	2019-12-02 11:14:15 +08:00
hantmac	0eba6754ae	fix panic because of interface{} is nil	2019-12-02 10:11:57 +08:00
hantmac	ff6ab9a281	bug fix:修复手动或者自动删除日志后的报错	2019-11-29 13:52:03 +08:00
marvzhang	87546f0c88	准备可配置爬虫自定义设置变量	2019-11-29 13:42:50 +08:00
陈景阳	f8f4d561e9	Merge remote-tracking branch 'upstream/develop' into upstream-develop ... # Conflicts: # backend/services/spider.go	2019-11-26 15:30:26 +08:00
暗音	4f9d8635e5	Merge pull request #13 from crawlab-team/develop ... Develop	2019-11-26 15:29:18 +08:00
陈景阳	fa4d8ddc50	fix 执行爬虫错误的问题	2019-11-26 15:26:01 +08:00
marvzhang	a87950e369	加入环境变量	2019-11-25 22:07:20 +08:00
marvzhang	1c32c92c54	refactor code	2019-11-25 16:45:55 +08:00
陈景阳	cb0018ef87	关闭日志打印	2019-11-04 19:58:26 +08:00
暗音	bcf3978600	Merge branch 'develop' into develop	2019-10-28 11:06:13 +08:00
陈景阳	4ab4892471	fix 环境变量的问题	2019-10-16 15:39:25 +08:00
陈景阳	3fc97e78d8	fix 时区的问题	2019-10-16 11:26:15 +08:00
陈景阳	467796b55a	fix 一些东西	2019-10-16 10:11:00 +08:00
陈景阳	f92d2d4266	优化代码	2019-10-16 09:45:50 +08:00
陈景阳	d4c152f93a	代码优化	2019-10-15 17:23:03 +08:00
陈景阳	b56399451e	fix 定时任务管理	2019-10-11 23:22:25 +08:00
yaziming	ee808e0e60	refactor(all): refactor code ... remove redundant code and some code refactor	2019-10-11 16:01:57 +08:00
陈景阳	46d89c8cce	添加日志打印	2019-10-10 19:56:04 +08:00
陈景阳	f5af83dcce	修改版本号	2019-10-08 20:26:08 +08:00
陈景阳	6b90d86ea2	fix 状态的问题	2019-10-08 20:20:48 +08:00
陈景阳	a081b02c13	fix 状态的问题	2019-10-08 20:16:49 +08:00
陈景阳	4c8b38f40b	fix 状态错误的问题	2019-10-08 19:41:29 +08:00
陈景阳	41556cab74	fix 删除爬虫的问题	2019-09-30 12:09:37 +08:00
陈景阳	5f158ddb44	完成爬虫获取	2019-09-26 19:12:02 +08:00
陈景阳	169a7d0df9	fix 可能直接退出的问题	2019-09-26 16:49:01 +08:00
陈景阳	05c28230b7	爬虫逻辑修改为从GridFS获取	2019-09-26 11:28:20 +08:00
陈景阳	96e926b486	fix 无法终止任务的问题	2019-09-26 10:29:58 +08:00
陈景阳	e235076a11	fix 无法终止任务的问题	2019-09-26 10:23:08 +08:00
陈景阳	7218a4f4de	添加日志打印	2019-09-26 09:37:10 +08:00
陈景阳	fda7e56e1a	添加日志打印	2019-09-25 16:19:58 +08:00
陈景阳	82999225de	fix 无法获取task的错误	2019-09-25 15:11:27 +08:00
陈景阳	951f6a9f07	fix	2019-09-25 14:38:46 +08:00
陈景阳	cdeef4b4a5	重构msg的包	2019-09-10 09:32:48 +08:00
Marvin Zhang	4893439ffb	added param to task	2019-09-04 20:13:12 +08:00
hantmac	2fb1754e56	bug fix:fix data race caused by LockList	2019-09-04 13:51:02 +08:00
yaziming	e10d8fd996	refactor(backend): Use more efficient bytes to string methods and remove unnecessary type conversions ... detail: 1. add utils.BytesToString function instead of string() convert bytes to string. 2. use bytes.NewReader instead of strings.NewReader(string(sb)). 3. use w.Body.Bytes() instead of []byte(w.Body.String()).	2019-09-03 15:17:32 +08:00
yaziming	a9346a0934	backend: ... 1. Mongo dial add 5 seconds connection timeout. 2. Redis uses connection pool mode. 3. Redis pool new connection have 10 seconds write timeout and read timeout and connection timeout.	2019-09-01 17:18:08 +08:00
陈景阳	e423ff564c	fix 爬虫目录无法打开的问题	2019-08-31 17:04:49 +08:00
陈景阳	c7e137a6aa	删除爬虫顺带删除文件	2019-08-27 09:39:16 +08:00
Marvin Zhang	83babba43f	fixed upload bug	2019-07-29 20:31:44 +08:00
Marvin Zhang	f3e78423f8	updated Dockerfile	2019-07-28 15:27:44 +08:00
Marvin Zhang	946bb7ceba	added spider stats	2019-07-26 17:07:24 +08:00
Marvin Zhang	56c99b314f	added golang	2019-07-22 12:51:52 +08:00