Please use this identifier to cite or link to this item: http://hdl.handle.net/10125/41876

Batch to Real-Time: Incremental Data Collection & Analytics Platform

File SizeFormat 
paper0727.pdf2.17 MBAdobe PDFView/Open

Item Summary

Title: Batch to Real-Time: Incremental Data Collection & Analytics Platform
Authors: Aydin, Ahmet
Anderson, Ken
Keywords: big data
data analytics
incremental
real-time
Issue Date: 04 Jan 2017
Abstract: Real-time data collection and analytics is a desirable but challenging feature to provide in data-intensive software systems. To provide highly concurrent and efficient real-time analytics on streaming data at interactive speeds requires a well-designed software architecture that makes use of a carefully selected set of software frameworks. In this paper, we report on the design and implementation of the Incremental Data Collection & Analytics Platform (IDCAP). The IDCAP provides incremental data collection and indexing in real-time of social media data; support for real-time analytics at interactive speeds; highly concurrent batch data processing supported by a novel data model; and a front-end web client that allows an analyst to manage IDCAP resources, to monitor incoming data in real-time, and to provide an interface that allows incremental queries to be performed on top of large Twitter datasets.
Pages/Duration: 10 pages
URI/DOI: http://hdl.handle.net/10125/41876
ISBN: 978-0-9981331-0-2
DOI: 10.24251/HICSS.2017.712
Rights: Attribution-NonCommercial-NoDerivatives 4.0 International
Appears in Collections:Big Data Engineering Minitrack



Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.