48 Commits

Author	SHA1	Message	Date
Marvin Zhang	91b9a8e78d	Merge remote-tracking branch 'origin/develop' into develop	2019-06-03 12:56:40 +08:00
Marvin Zhang	a27cef6868	burying points for analytics	2019-05-30 21:14:30 +08:00
Marvin Zhang	571acd9728	fixed issue https://github.com/tikazyq/crawlab/issues/45	2019-05-30 12:21:01 +08:00
Marvin Zhang	308173072f	updated ConfigList	2019-05-30 11:47:17 +08:00
Marvin Zhang	19679327d2	fixed bugs	2019-05-29 19:35:04 +08:00
Marvin Zhang	7492c34c4a	added extracting list fields functionality	2019-05-29 17:51:44 +08:00
Marvin Zhang	144b3600e3	Merge remote-tracking branch 'origin/develop' into develop	2019-05-29 14:45:30 +08:00
Marvin Zhang	4b08751868	updated env files	2019-05-29 14:44:33 +08:00
Marvin Zhang	f3da64d8b7	code cleanup	2019-05-29 13:10:31 +08:00
Marvin Zhang	d03197042d	allow generate URLs by pattern	2019-05-28 14:59:21 +08:00
Marvin Zhang	113be056ae	updated configurable spider	2019-05-26 19:17:02 +08:00
Marvin Zhang	e30c1c122a	updated configurable spider	2019-05-26 19:16:46 +08:00
Marvin Zhang	5daebcb39b	updated configurable spider	2019-05-26 17:10:04 +08:00
Marvin Zhang	15f9b7ef05	added configurable spider: add/edit fields, preview results	2019-05-25 20:18:27 +08:00
Marvin Zhang	04789febc8	prepared for configurable spiders	2019-05-24 17:28:25 +08:00
Marvin Zhang	433b945eeb	link site to spider	2019-05-10 13:27:13 +08:00
Marvin Zhang	0ad0303285	added site list	2019-05-09 22:14:37 +08:00
Marvin Zhang	c96dd102d9	updated analytics for spider	2019-04-28 21:25:32 +08:00
Marvin Zhang	21f7864c68	updated analytics for spider	2019-04-28 21:16:31 +08:00
Marvin Zhang	9bd819c384	updated analytics for spider	2019-04-28 19:52:10 +08:00
Marvin Zhang	60d658de5f	updated analytics for spider	2019-04-28 19:41:44 +08:00
Marvin Zhang	efc32886ae	updated analytics for spider	2019-04-28 19:32:18 +08:00
Marvin Zhang	70d392ef62	updated analytics for spider	2019-04-28 19:24:03 +08:00
Marvin Zhang	a41415a041	added analytics for spider	2019-04-28 13:17:40 +08:00
Marvin Zhang	ae29708d99	request task list every 5 seconds	2019-04-23 20:36:31 +08:00
Marvin Zhang	4dff16e9ec	added filter for TaskList	2019-04-22 21:26:58 +08:00
Marvin Zhang	daeaa05d85	added filter for TaskList	2019-04-17 21:50:30 +08:00
Marvin Zhang	63ff2d1894	enforce deploy before running a spider	2019-04-15 19:50:25 +08:00
Marvin Zhang	6712b3a5f1	added environment variables for spider page	2019-04-10 22:16:25 +08:00
Marvin Zhang	7ce74e06d7	updated README.md ... code cleanup	2019-04-03 19:58:21 +08:00
cxa	df43a7fc96	fix bug	2019-04-03 16:32:35 +08:00
Marvin Zhang	063703319c	updated Multi Language for a few pages	2019-03-27 12:35:22 +08:00
Marvin Zhang	1efc5199f9	updated Multi Language for a few pages	2019-03-26 19:33:03 +08:00
Marvin Zhang	d6cd8ff8f7	added Multi Language support	2019-03-23 23:48:08 +08:00
Marvin Zhang	c6174df927	removed a task	2019-03-16 21:30:27 +08:00
Marvin Zhang	e526b4215b	changed the way how task is saved to database	2019-03-16 21:29:30 +08:00
Marvin Zhang	53a80046d1	changed the way how task is saved to database	2019-03-13 21:33:47 +08:00
Marvin Zhang	1ec3074012	added pagination for tasks results	2019-03-12 21:20:22 +08:00
Marvin Zhang	489768d097	added pagination for tasks	2019-03-12 21:00:35 +08:00
Marvin Zhang	ce5e5d8f1d	fixed inconsistency of timestamp issue	2019-03-12 20:28:50 +08:00
Marvin Zhang	497576c913	fixed deploy issue	2019-03-12 13:01:18 +08:00
Marvin Zhang	3aba022922	updated error message	2019-03-09 16:27:23 +08:00
Marvin Zhang	0df8d84105	added stop task	2019-03-09 16:18:39 +08:00
Marvin Zhang	647fac1efe	added cron tasks for spiders	2019-03-09 14:05:14 +08:00
Marvin Zhang	a71f9488ec	support multi-node log view	2019-03-06 15:43:14 +08:00
Marvin Zhang	af21ab4841	added Results	2019-03-06 13:14:26 +08:00
Marvin Zhang	6b9c0581c2	updated methodologies of deployment and running spider	2019-03-05 13:22:23 +08:00
Marvin Zhang	8361c04de9	added setup.py	2019-03-03 10:48:04 +08:00