237 Commits

Author	SHA1	Message	Date
hantmac	5bb8be1ddd	fix	2019-12-04 16:48:36 +08:00
hantmac	88a22097da	修改isDeletePeriodically参数默认为N	2019-12-04 16:46:16 +08:00
marvzhang	97ca3b74b8	修复爬虫被覆盖的问题	2019-12-04 14:54:49 +08:00
marvzhang	be4a5f6667	优先调整xpath顺序	2019-12-04 13:57:27 +08:00
marvzhang	82c6b80063	更新Spiderfile模版	2019-12-03 14:25:54 +08:00
marvzhang	589190d674	加入Spiderfile模版	2019-12-03 13:37:41 +08:00
marvzhang	9c5c0bd270	更新可配置爬虫	2019-12-03 12:14:38 +08:00
marvzhang	af680d576a	更新可配置爬虫	2019-12-02 22:35:45 +08:00
marvzhang	fd27949a40	加入依赖模块	2019-12-02 17:46:57 +08:00
marvzhang	c237c0cf96	Merge remote-tracking branch 'origin/develop' into develop	2019-12-02 13:32:44 +08:00
hantmac	67ae278a66	fix	2019-12-02 11:14:15 +08:00
hantmac	8f06646d99	fix panic because of interface{} is nil	2019-12-02 10:11:57 +08:00
hantmac	5e846209d6	Merge remote-tracking branch 'crawlab-team/develop' into develop	2019-12-01 18:36:22 +08:00
Jeremy	578dcac22b	Merge pull request #330 from appleboy/patch-3 ... chore: fix fmt warning.	2019-12-01 18:29:42 +08:00
Bo-Yi Wu	1283e0483f	chore: fix fmt warning.	2019-12-01 18:16:39 +08:00
marvzhang	08c7aec49c	加入可配置爬虫阶段设置	2019-11-30 10:58:54 +08:00
hantmac	61daf732bd	bug fix:修复手动或者自动删除日志后的报错	2019-11-29 13:52:03 +08:00
marvzhang	a1bfe00eee	准备可配置爬虫自定义设置变量	2019-11-29 13:42:50 +08:00
marvzhang	e14c7fa80e	修复删除可配置爬虫问题	2019-11-28 22:20:20 +08:00
marvzhang	ba4f5fd20c	优化可配置爬虫前端界面	2019-11-28 21:36:17 +08:00
marvzhang	bc81462a80	加入修改可配置POST接口	2019-11-28 16:08:13 +08:00
hantmac	d947852cde	修改为递归修改文件夹权限	2019-11-28 10:26:14 +08:00
hantmac	7624ec4af1	bug fix	2019-11-27 23:05:35 +08:00
marvzhang	f06c8eb785	准备前端可配置爬虫数据	2019-11-27 12:42:08 +08:00
marvzhang	2630802e6b	Merge remote-tracking branch 'origin/develop' into develop	2019-11-27 12:41:51 +08:00
marvzhang	ded5703b13	准备前端可配置爬虫数据	2019-11-27 12:32:25 +08:00
marvzhang	2e569c44c4	加入前端可配置编辑器	2019-11-27 12:18:51 +08:00
marvzhang	88118cf894	准备前端可配置爬虫数据	2019-11-27 10:40:37 +08:00
Marvin Zhang	b6fba383d7	Merge pull request #321 from zhangweiii/master ... fix Cannot read property length of null when first init	2019-11-27 10:02:21 +08:00
zhangweiii	8c72395ab7	fix Cannot read property length of null when first init	2019-11-27 09:45:16 +08:00
marvzhang	870bb167a7	准备可配置爬虫后端数据	2019-11-26 13:38:54 +08:00
marvzhang	bc5203267a	加入xpath	2019-11-26 13:02:49 +08:00
marvzhang	8046d94cc3	加入Spiderfile限制条件	2019-11-26 12:44:24 +08:00
marvzhang	e2e61c621e	加入环境变量	2019-11-25 22:07:20 +08:00
marvzhang	825962f8ba	加入储存逻辑	2019-11-25 17:52:19 +08:00
marvzhang	2e5468e4c1	refactor code	2019-11-25 16:45:55 +08:00
marvzhang	178e46dd93	更新可配置爬虫：添加上传GridFS	2019-11-25 13:36:14 +08:00
marvzhang	b8592b89ce	更新可配置爬虫，修复一些问题	2019-11-24 19:45:21 +08:00
marvzhang	6a07afa279	更新可配置爬虫，修复一些问题	2019-11-24 18:51:32 +08:00
marvzhang	38d103da39	加入可配置爬虫	2019-11-24 17:57:12 +08:00
marvzhang	c03c296436	Merge remote-tracking branch 'origin/develop' into develop	2019-11-24 12:25:46 +08:00
marvzhang	442396a5ff	fixed https://github.com/crawlab-team/crawlab/issues/315	2019-11-24 12:20:44 +08:00
Marvin Zhang	966d3eaaf2	Merge pull request #313 from hantmac/develop ... bug fix	2019-11-24 11:34:50 +08:00
marvzhang	b51fa81b79	code cleanup	2019-11-23 18:16:24 +08:00
marvzhang	b5c3ca9c32	准备可配置爬虫	2019-11-23 18:13:47 +08:00
marvzhang	9311a37a55	Merge remote-tracking branch 'origin/develop' into develop	2019-11-23 16:48:19 +08:00
marvzhang	2b26f7fa7e	added vendor files	2019-11-23 16:47:05 +08:00
韩山杰	6741873f20	modity redis DialWriteTimeout to 15s	2019-11-22 22:29:02 +08:00
韩山杰	6c7d884e66	bug fix	2019-11-22 15:32:23 +08:00
Jeremy	7e406261c2	bug fix register.go ... fix data race because of register solve issue https://github.com/crawlab-team/crawlab/issues/308	2019-11-19 17:03:46 +08:00