377 Commits

Author	SHA1	Message	Date
marvzhang	e4c012cf2f	加入版本检测	2020-02-19 13:16:46 +08:00
marvzhang	51a01e1f32	fixed ssh-keygen issue	2020-02-18 15:20:57 +08:00
marvzhang	06f1d6a8ac	fixed unable to update is_scrapy issue	2020-02-18 14:57:57 +08:00
marvzhang	5c65ed4da1	更改ssh逻辑	2020-02-18 13:42:31 +08:00
marvzhang	0cb2e8134c	加入Git同步	2020-02-18 12:15:40 +08:00
marvzhang	f83a041cd1	加入git同步	2020-02-17 18:12:59 +08:00
marvzhang	bb972e98ef	加入添加scrapy爬虫	2020-02-17 14:06:16 +08:00
marvzhang	25b4c7d272	完成scrapy设置	2020-02-17 11:52:26 +08:00
marvzhang	dc38642dca	加入settings.py同步	2020-02-17 11:15:16 +08:00
marvzhang	cdaf3363c8	添加Scrapy设置后端	2020-02-17 10:31:59 +08:00
marvzhang	69e4c23115	更新scrapy参数	2020-02-16 10:38:41 +08:00
marvzhang	01fefbee88	加入scrapy日志选择	2020-02-15 21:25:21 +08:00
marvzhang	acdf4d5951	加入scrapy爬虫支持	2020-02-15 18:28:37 +08:00
marvzhang	c75bd604fa	updated version	2020-02-13 13:45:12 +08:00
marvzhang	1c7c20c11c	优化爬虫执行逻辑	2020-02-11 15:37:06 +08:00
marvzhang	cf7f30d7ce	updated spiders	2020-02-11 10:06:03 +08:00
marvzhang	7b34e47efe	updated install-nodejs.sh	2020-02-11 09:36:34 +08:00
marvzhang	c3b1c8040a	updated install-nodejs.sh	2020-02-10 19:28:47 +08:00
marvzhang	8df923fd3c	updated install-nodejs.sh	2020-02-10 17:45:47 +08:00
marvzhang	3e2bdb59f4	updated install-nodejs.sh	2020-02-10 17:41:14 +08:00
marvzhang	47a494a012	updated etl	2020-02-10 17:11:10 +08:00
marvzhang	c50a9bb63e	updated install-nodejs.sh	2020-02-10 17:04:53 +08:00
marvzhang	70e911ade6	updated install-nodejs.sh	2020-02-10 14:26:14 +08:00
marvzhang	6b32709a39	兼容Node.js爬虫	2020-02-10 13:23:34 +08:00
marvzhang	143550f3c6	added jd_mask_spider	2020-02-10 11:15:37 +08:00
marvzhang	9c2bdb077d	added node.js dependencies	2020-02-10 10:45:29 +08:00
marvzhang	12b03cba8a	fixed https://github.com/crawlab-team/crawlab/issues/524	2020-02-09 09:02:13 +08:00
marvzhang	4a62542eb1	fixed sinastock_spider issue	2020-02-08 09:51:11 +08:00
marvzhang	de41f41475	版本更新	2020-02-03 16:43:29 +08:00
marvzhang	a448c0f06b	fixed unable to sync spiders to nodes error	2020-02-03 16:08:43 +08:00
marvzhang	de2d230e1a	changed dir	2020-02-03 11:58:05 +08:00
marvzhang	819e5f56c5	fixed https://github.com/crawlab-team/crawlab/issues/202	2020-02-03 11:08:01 +08:00
marvzhang	be3235cefa	added demo for general spiders	2020-02-03 10:30:04 +08:00
marvzhang	cb6a8f79d8	added demo spiders	2020-02-03 09:21:41 +08:00
marvzhang	5740774ccc	添加demo爬虫	2020-02-02 22:56:11 +08:00
marvzhang	18c8dd6253	优化项目管理	2020-02-02 14:04:38 +08:00
marvzhang	024f9ac3b2	finish up project	2020-02-02 13:29:56 +08:00
marvzhang	9d5d0253a8	加入项目管理	2020-02-02 12:29:28 +08:00
marvzhang	633740f1c2	fixed https://github.com/crawlab-team/crawlab/issues/485	2020-02-01 19:18:30 +08:00
Marvin Zhang	1075506f66	Merge pull request #495 from wo10378931/develop ... Develop	2020-02-01 13:52:01 +08:00
陈景阳	ec5afd7985	还原代码	2020-02-01 13:08:15 +08:00
陈景阳	100e7631fb	fix 启动报错的问题	2020-02-01 12:53:44 +08:00
marvzhang	1385288b73	修复可配置爬虫无法解析 "//" 打头的URL	2020-01-30 17:36:38 +08:00
marvzhang	106646e9f5	fixed https://github.com/crawlab-team/crawlab/issues/465	2020-01-29 09:24:43 +08:00
陈景阳	fcfaf8ba77	Merge remote-tracking branch 'upstream/develop' into upstream-develop ... # Conflicts: # backend/main.go # frontend/.env.development	2020-01-28 15:51:09 +08:00
陈景阳	af190f36cc	fix 回退配置文件信息	2020-01-28 15:46:15 +08:00
陈景阳	81edf9ab72	fix 无法及时同步爬虫的问题	2020-01-28 15:43:57 +08:00
陈景阳	092c1a2617	feature 新增全局环境变量	2020-01-28 15:12:12 +08:00
陈景阳	9a748fc5cd	解决冲突	2020-01-23 11:09:35 +08:00
marvzhang	e3f4bcd0ab	updated stats	2020-01-22 11:07:38 +08:00