Demo WITHOUT Kinect and Recommendations!
GAIN: web service for user tracking and preference learning – a SMART TV use case
Jaroslav Kuchar and Tomas Kliegr
- Screencast Screencast
User Tracking
"Smart TV" is demoed with Youtube Player. There are two sources of interaction: remote control and user behaviour feedback. Every user interaction is immediately sent by the player via the REST API component to GAIN server along with the time in the video and video identification.
- Player Demo
Example:
Consider example televiewer watching news. During a spot covering a football match, the user clicks on a bookmark button and behaviour tracking records “Looking at screen”.
Ontology-based Content Description
The video is fragmented into smaller time-intervals: views. This segmentation is performed from start and stop time of subtitles. The text of the video, as extracted from subtitles, is processed with entity recognition tool entityclassifier.eu. The tool also assigns each object with several type attributes corresponding to one or more DBpedia Ontology classes.
Example:
Subtitle fragment: "Ronaldinho's presence in Brasil's squad is significant"
Entity dbpedia:Ronaldinho is recognized in the subtitle, the type assigned is dbpedia-owl:SoccerPlayer
Aggregation
Multidimensional semantic (taxonomical) description of tracked objects are processed along with user feedback to the lower dimensional output representation, with a tabular form suitable for analysis with mainstream data mining algorithms.
Content Description: Each view contains a varying number of objects. First, the domain ontology is used to expand the type attributes to a class membership vector.
Example: The “dbpedia-owl:SoccerPlayer” type attribute value is expanded to subvector containing non-zero values also for its superclasses: dbpedia-owl:Athlete and dbpedia-owl:Person.
User feedback: None to multiple user actions are recorded during a view. These are aggregated into a single interest value, which ranges from -1 (maximum disinteret) to 1 (max- imum interest).
Custom set of rules:
- if user clicks on bookmark, the interest is increased by 0.5
- if user looks at the screen, the interest is increased by 0.3
Example: Our user’s bookmark and look at screen actions resulted in a total 0.8 interest for the view.
Example of table:
| userId | sessionId | viewId | dbpedia-owl:Person | dbpedia-owl:Athlete | dbpedia:Ronaldinho | dbpedia:Brazil | interest |
|---|---|---|---|---|---|---|---|
| user1 | 12345678 | view1 | 1 | 1 | 1 | 1 | 0.8 |
| user1 | 12345678 | view2 | 0 | 1 | 0 | 1 | 0.5 |
| user1 | 12345678 | view3 | 0 | 1 | 0 | 1 | 0.3 |
Learning preference rules with EasyMiner
User preferences are discovered with EasyMiner, a web service and web application for association rule discovery based on LISp-Miner.
Example: Mining interactions of our user yields rules such as:
- dbpedia-owl:Athlete∧dbpedia:Brazil → interest = ⟨0.3; 0.5⟩
dbpedia:Ronaldinho → interest = (0.5; 1⟩.
Content recommendation with Business Rules
Our tool allows to export selected rules to a Business Rules Engine. Input is a description of a candidate content item with vector of concepts. The output is the predicted interest.
Current object viewed by the user:
| userId | sessionId | viewId | dbpedia-owl:Person | dbpedia-owl:Athlete | dbpedia:Ronaldinho | dbpedia:Brazil |
|---|---|---|---|---|---|---|
| user1 | 12345678 | view1 | 1 | 1 | 1 | 1 |
Candidates for recommendation:
| viewId | dbpedia-owl:Person | dbpedia-owl:Athlete | dbpedia:Ronaldinho | dbpedia:Brazil |
|---|---|---|---|---|
| view1 | 1 | 1 | 1 | 1 |
| view2 | 0 | 1 | 0 | 1 |
| view3 | 0 | 1 | 0 | 1 |
Acknowledgments
1) LinkedTV | Television Linked to the Web
2) Faculty of Information Technology| Czech Technical University in Prague
