388 Commits

Author	SHA1	Message	Date
marvzhang	7315deefee	加入复制爬虫	2020-02-24 09:12:03 +08:00
marvzhang	6000667b7d	updated dockerpush.yml	2020-02-23 09:19:18 +08:00
marvzhang	dd3264768a	加入批量运行爬虫	2020-02-22 19:12:24 +08:00
marvzhang	2e6b6e4e70	加入批量删除爬虫任务	2020-02-22 17:35:51 +08:00
marvzhang	eb167ead1e	暂时停用自动检测Scrapy项目功能	2020-02-22 12:48:15 +08:00
marvzhang	ba720b356c	加入批量删除爬虫	2020-02-22 12:44:58 +08:00
marvzhang	cddc09405b	加入scrapy spider跳转	2020-02-21 22:28:41 +08:00
marvzhang	21323f0d28	加入scrapy pipelines支持	2020-02-21 17:12:57 +08:00
marvzhang	38af9e9173	加入scrapy items.py支持	2020-02-21 15:07:54 +08:00
marvzhang	1ddac86fe1	加入添加scrapy项目	2020-02-20 17:33:45 +08:00
marvzhang	17bdbaf8dd	优化爬虫列表页面	2020-02-20 11:52:28 +08:00
marvzhang	ec8bf33535	加入版本检测	2020-02-19 13:16:46 +08:00
marvzhang	16263a9423	fixed ssh-keygen issue	2020-02-18 15:20:57 +08:00
marvzhang	357a2d26a8	fixed unable to update is_scrapy issue	2020-02-18 14:57:57 +08:00
marvzhang	fb9b6e1619	更改ssh逻辑	2020-02-18 13:42:31 +08:00
marvzhang	2ce289c8f3	加入Git同步	2020-02-18 12:15:40 +08:00
marvzhang	3bb1fe2a74	加入git同步	2020-02-17 18:12:59 +08:00
marvzhang	534a29ebc3	加入添加scrapy爬虫	2020-02-17 14:06:16 +08:00
marvzhang	9abf577e74	完成scrapy设置	2020-02-17 11:52:26 +08:00
marvzhang	dec566d1e5	加入settings.py同步	2020-02-17 11:15:16 +08:00
marvzhang	a9277f3412	添加Scrapy设置后端	2020-02-17 10:31:59 +08:00
marvzhang	fc4d92c4d6	更新scrapy参数	2020-02-16 10:38:41 +08:00
marvzhang	3b8702ed58	加入scrapy日志选择	2020-02-15 21:25:21 +08:00
marvzhang	d46f43b05b	加入scrapy爬虫支持	2020-02-15 18:28:37 +08:00
marvzhang	88d8f1f907	updated version	2020-02-13 13:45:12 +08:00
marvzhang	fd62adc8ba	优化爬虫执行逻辑	2020-02-11 15:37:06 +08:00
marvzhang	a0c47e54e8	updated spiders	2020-02-11 10:06:03 +08:00
marvzhang	a62fec9c01	updated install-nodejs.sh	2020-02-11 09:36:34 +08:00
marvzhang	7e85f561bf	updated install-nodejs.sh	2020-02-10 19:28:47 +08:00
marvzhang	e1e363eb40	updated install-nodejs.sh	2020-02-10 17:45:47 +08:00
marvzhang	3363d08797	updated install-nodejs.sh	2020-02-10 17:41:14 +08:00
marvzhang	6f5e99c1ae	updated etl	2020-02-10 17:11:10 +08:00
marvzhang	6002c94dce	updated install-nodejs.sh	2020-02-10 17:04:53 +08:00
marvzhang	2403c69329	updated install-nodejs.sh	2020-02-10 14:26:14 +08:00
marvzhang	8d1d7c9b1d	兼容Node.js爬虫	2020-02-10 13:23:34 +08:00
marvzhang	ef991e1056	added jd_mask_spider	2020-02-10 11:15:37 +08:00
marvzhang	5590b9d97c	added node.js dependencies	2020-02-10 10:45:29 +08:00
marvzhang	8aed6c1de7	fixed https://github.com/crawlab-team/crawlab/issues/524	2020-02-09 09:02:13 +08:00
marvzhang	d4cdfda5ff	fixed sinastock_spider issue	2020-02-08 09:51:11 +08:00
marvzhang	4daeab594f	版本更新	2020-02-03 16:43:29 +08:00
marvzhang	23f0b0c9e2	fixed unable to sync spiders to nodes error	2020-02-03 16:08:43 +08:00
marvzhang	2c410feed3	changed dir	2020-02-03 11:58:05 +08:00
marvzhang	d6d164a438	fixed https://github.com/crawlab-team/crawlab/issues/202	2020-02-03 11:08:01 +08:00
marvzhang	0d7b97ead5	added demo for general spiders	2020-02-03 10:30:04 +08:00
marvzhang	6b70868326	added demo spiders	2020-02-03 09:21:41 +08:00
marvzhang	98e5ead285	添加demo爬虫	2020-02-02 22:56:11 +08:00
marvzhang	6931211cbd	优化项目管理	2020-02-02 14:04:38 +08:00
marvzhang	0b22e55e93	finish up project	2020-02-02 13:29:56 +08:00
marvzhang	075a899c2f	加入项目管理	2020-02-02 12:29:28 +08:00
marvzhang	85a4df9bfb	fixed https://github.com/crawlab-team/crawlab/issues/485	2020-02-01 19:18:30 +08:00