A Marketplace for Realtime Human-Sensing Data

View project onGitHub


This page introduces MARSA (A Marketplace for Realtime Human-Sensing Data), a dynamic marketplace platform in the cloud where different stakeholders can sell and buy dynamic near-realtime data. we present techniques for selecting data types and managing data contracts based on different pricing models, quality of data and incentive models. We design our marketplace by leveraging different data transferring platforms to enable an open and scalable communication mechanism between sellers (data service providers) and buyers (data service consumers) through a set of communication APIs

Traffic Scenario
Trafic scenario

The use of different real-time data sources in the context of a smart city is a typical example. Traffic congestion is a challenging problem that many big cities around the world are trying to solve. In order to have a good solution for this problem, the current state of traffic at any location in a city and at any time needs to be closely monitored. Traffic information can be retrieved from many different sources, such as traffic and surveillance cameras, GPS signals from GPS enabled devices on cars, buses and mobile phones, and on road pressure sensors. The data sets collected from these sources originally exist in its own format, and are constantly forwarded to corresponding processing centers. They need to be converted to usable traffic information and integrated together to form a unified source of traffic data. In turn, the traffic data set retrieved from the above process is provided to its intended consumers such as government agencies, police departments, and the public in real-time.

MARSA architecture

Realtime human-sensing data marketplace

In our general design of the marketplace for real-time IoT data, four stakeholders, which are data broker, payment controller, quality controller and manager, are mapped to the correspondent services of the marketplace, i.e., Service Discovery, Billing, Quality Analysis and Management which also manages the needed models for data contract such as pricing model, quality of data model, device description model, etc. We design Data Processor stakeholder as a set of services developed by marketplace owner or third parties. To enable the technology infrastructure for transferring data, a set of Real-Time Data Bus which plays the role of transporters is proposed as the services outside the marketplace. This allows us to profit the current IoT platforms (e.g., Xively) or Data Distribution Service middlewares as the data buses. Moreover, this also increases the opportunity of providers and/or consumers to choose the familiar/trusted protocol for their exchange. However, this design raises a list of APIs which allows the data buses to interact with marketplace to update the data status to marketplace.



In our prototype, we implemented fully two basic services for Data Discovery and Cost Model Management. To prove the realtime capability of our proposition, we also implemented, as a part of Data Quality Analysis service, an latency analytics that allows us to measure the near realtime capacity of databuses. We have implemented our marketplace APIs, which are used to enforce the cost model and support the latency analytic, using Web services to interface to different platforms for transferring data, including Mosquitto and Xively IoT platform as the realtime databuses. We used these brokers because they implemented a lightweight broker-based publish/subscribe messaging protocol, i.e., MQTT5.

Code compilation and installation

  1. Maven 2 or 3
  2. JDK 1.7+
  3. MySQL 5.0+
  4. Tomcat 7

The maven project of the source code is fetched from github repository and built by jdk v1.7. After building the project, following artifacts are created

  1. marketplace.war and pricing.war are deployed in Tomcat container.
  2. marketplace.sql is imported into a local MySQL server.
  3. agence.jar is run on distributed VMs to simulate the seller and buyer.
After the deployment, the web interface of the marketplace can be accessed via URL: http://localhost/marketplace/default. The "localhost" is changed to the right IP address if needed.


Previous work


For any query or issue, please contact