32 Commits

Author	SHA1	Message	Date
Marvin Zhang	823c7f6b53	added generate_urls function	2019-05-28 13:27:25 +08:00
Marvin Zhang	9ba5623d62	updated chinaz_spider	2019-05-22 23:10:33 +08:00
Marvin Zhang	8e96ad8cb6	updated sites	2019-05-22 22:33:28 +08:00
Marvin Zhang	9b23b1a4c6	updated sites_inspector	2019-05-22 20:08:38 +08:00
Marvin Zhang	d545dc2697	cleanup requirements.txt	2019-05-22 13:16:45 +08:00
Marvin Zhang	1f015730c1	refactor code	2019-05-21 12:28:33 +08:00
Marvin Zhang	73e3bc2dec	added jd spider	2019-05-10 21:25:17 +08:00
Marvin Zhang	1cd2fd47f3	link site to spider	2019-05-10 13:26:58 +08:00
Marvin Zhang	69e2e2f3c5	added chinaz spider	2019-05-10 12:39:10 +08:00
Marvin Zhang	e3d4796d8a	added chinaz spider	2019-05-09 20:52:59 +08:00
Marvin Zhang	b169f09e12	added lianjia example	2019-04-16 20:19:22 +08:00
Marvin Zhang	6712b3a5f1	added environment variables for spider page	2019-04-10 22:16:25 +08:00
Marvin Zhang	7ce74e06d7	updated README.md ... code cleanup	2019-04-03 19:58:21 +08:00
cxa	054f63d93f	修改文件的配置地址为127.0.0.1系列	2019-04-03 17:08:26 +08:00
cxa	df43a7fc96	fix bug	2019-04-03 16:32:35 +08:00
Marvin Zhang	640c6d2137	added article_spider.js	2019-03-20 20:58:27 +08:00
Marvin Zhang	9aeff8181c	added spiders	2019-03-15 20:22:09 +08:00
Marvin Zhang	53a80046d1	changed the way how task is saved to database	2019-03-13 21:33:47 +08:00
Marvin Zhang	ff631e1095	updated README	2019-03-13 19:27:14 +08:00
Marvin Zhang	5b190e5b2b	modified task status update logic	2019-03-12 13:10:24 +08:00
Marvin Zhang	0dc5c64bc1	added spider for segmentfault	2019-03-11 13:15:00 +08:00
Marvin Zhang	647fac1efe	added cron tasks for spiders	2019-03-09 14:05:14 +08:00
Marvin Zhang	af21ab4841	added Results	2019-03-06 13:14:26 +08:00
Marvin Zhang	68e0b98d8a	updated README.md	2019-03-06 10:46:41 +08:00
Marvin Zhang	a473aef621	restructured code	2019-03-06 10:11:07 +08:00
Marvin Zhang	318e5876a6	fixed sorting issue	2019-02-25 19:32:43 +08:00
Marvin Zhang	278d6e72f5	added TaskOverview	2019-02-23 18:39:22 +08:00
Marvin Zhang	913c0644c9	updated node/spider overviews	2019-02-21 14:08:48 +08:00
Marvin Zhang	5eb69a4b0d	overview pages updated	2019-02-20 21:24:28 +08:00
Marvin Zhang	a4a0fdd67b	added tasks	2019-02-18 21:16:40 +08:00
Marvin Zhang	714ccd18aa	update SpiderDetail	2019-02-17 11:14:22 +08:00
Yeqing Zhang	bbeb5ca5b5	added deploys	2019-02-13 20:53:15 +08:00