Check out the long version of our demo video! Learn more about all the different modules of the ViSTA-TV platform:
ViSTA-TV and RapidMiner made it to the CORDIS front page!
ViSTA-TV was evaluated as a success according to our project officer and our reviewers. Is this the end? No! We will continue working on TV-statistics, recommendations, and turn ViSTA-TV into a business. Stay tuned.
Discover ViSTA-TV’s essentials in this two-minute video.
The final ViSTA-TV coding camp is about to end soon. We prepared the final showcase and the demos for the review meeting.
The recommendations team continued producing new exciting research results.
Now that the project is ending soon, we also took some time to think about the last two years…
… and we came up with a lot of cool ideas for research and devel
Today the final ViSTA-TV coding camp starts at Dortmund. We will finalize the platform and perform final evaluations. Stay tuned for some exciting demonstrators which will show the full power of the ViSTA-TV platform.
The ViSTA-TV overview talk presented by Abraham Bernstein at the European Data Forum (EDF) in Athens on March 19, was just published by Videolectures.net.
If you want to see it just click on the image below.
Vista TV – Online TV Statistics and Dynamic Joint Recommendations
Last week ViSTA-TV visited the European Data Forum (EDF) in Athens. Apart from many personal and informal interactions with other forum participants the project was presented in two different contexts.
On March 19, Abraham Bernstein gave a overview talk about ViSTA-TV. The talk was filmed by Videolectures.net but has not been posted yet. We will be posting a link to the video as soon as it will be available.
In addition, Abraham Bernstein was interviewed by the Greek Public Television channel. The interview, which was part of a report that also contains with the two main organisers and with Marta Nagy-Rothengass, the Head of Unit Data Value Chain within the EC’s DG CONNECT (which finances ViSTA-TV) , provides a succinct overview of ViSTA-TV and sheds light on some possible uses of the VISTA-TV results. The interview is available both as part of the report, on YouTube and embedded below.
Following the success of the ViSTA-TV demonstrations at IBC 2013 and RecSys 2013 the project has repeated these demonstrations to BBC staff and guests at three BBC sites in London and at MediaCityUK in Salford. This has enabled us to show our live viewing data analysis system and audience-facing recommender applications to a wide variety of staff.
The most recent event (shown above) was an exhibition at an internal conference on technical innovation in Television, held at Broadcasting House, where there were some unusual but highly effective security staff.
The ViSTA-TV coding camp ended today. Its results will enable both IPTV providers to evaluate the recommendation engines developed in the project. We solved many open issues regarding data quality and worked on stabilizing the feature extraction. Further progress was made in making the data warehouse internally available via SPARQL and automatig the data generation processes.
The ViSTA-TV development team looks forward to finalize the integration of the recommender engines in February. Stay tuned!
Today, the ViSTA-TV team assembles at Zurich to plan the coming months. From Wednesday on, the development team will finalize the recommendations engine and prepare the user evaluations, which are scheduled for spring 2014.
TV 2.0 Summit in Zurich
On November 28 ViSTA-TV was presented at the TV2.0 Summit in Zurich, Switzerland. It was an opportunity to connect with a number of people from the local TV industry.
The echo of the attendees was extremely positive and we we got excellent feedback.
ViSTA-TV will be connect with the EU FP7 project LDBC at the Third TUC Meeting, November 19 2013. This event is co-located with GraphConnect at London. Prof Abraham Bernstein (UZH) will talk about “Streams and Advanced Processing” whereas Thomas Scharrenbach (UZH) will present ViSTA-TV’s findings w.r.t. PCKS, a systematic approach for benchmarking stream processing systems.
We look forward to exchange ideas and lessons learned with the LDBC project.
ViSTA-TV will be present at the EBU DevCon2013. Thomas Scharrenbach from UZH will give a talk about the technical components of the ViSTA-TV engine.
“[The] EBU Developer Conference provide an opportunity to learn about best practices in software engineering and get hands-on with current development and deployment tools such as Puppet. Participants will pick up new ideas to run software-driven broadcast infrastructure and improve their engineering workflows.”
ViSTA-TV looks forward to meeting lots of interesting people at Geneva!
The ViSTA-TV Coding Camp at the BBC Media House, London, UK has been successfully accomplished. We finalized much of the work needed for continously giving recommendations. In particular for the BBC‘s Apps which they are going to evaluate in early 2014. Continuous recommendations for Zattoo are also on track and the necessary interfaces are under construction. Further achievements were new versions of the data warehouse ontologies (to be published along with the LOD dataset), continuous EPG data enrichment, realiability checks and — last but not least — the basic integration of the Streams framework (and hence the ViSTA-TV engine) into RapidMiner.
The ViSTA-TV team thanks our hosts Libby and Chris from BBC R&D. We look froward to another coding camp in early 2014 for preparing the M22 deliverables.
The ViSTA-TV project team will assemble at London to prepare the upcoming milestone. Besides the decisions what work will be done for the remainder of the project, the ViSATA-TV developers will have another Coding Camp at the BBC’s research facilities. If you want to meet members of the ViSTA-TV team, please write to info@vista-tv.eu.
Earlier this week, ViSTA-TV presented a demonstration paper at RecSys 2013, the world-leading conference on Recommender Systems.
The paper, titled “Design and Evaluation of a Client-side Recommender System”, concerns the BBC’s client-side recommender engine and two applications that use it. The recommender engine is a Javascript implementation of a classical Weighted Item-based k-NN algorithm and uses a downloaded collaborative filtering model. This model is created using audience viewing data obtained from the system that provides live data streams for the ViSTA-TV stream processing engine. Currently the model is trained periodically using the MyMediaLite Recommender System library but will be based on the recommendations work in ViSTA-TV when this is completed. The engine and training tools have been released as an open source project.
The development of the first application, the Sibyl Recommender, started before ViSTA-TV. Sibyl is a TV and radio programme recommender system designed for tablets and personal computers which uses a novel drag-and-drop system to elicit user preferences.
Users are presented with a scrollable list of programme recommendations which they can filter by genre and media type. A user is able to express preferences by dragging individual programmes into “like” and “dislike” boxes positioned below. These preferences are immediately used by the client-side recommender to re-rank the programmes and refresh the recommendation list. You can try Sibyl Recommender online and give us feedback via the associated questionnaire: http://sibyl.prototyping.bbc.co.uk/
The second application, called “Infinite Trailers” was developed within the ViSTA-TV project and provides a continuous sequence of programme clips.
The user can either skip each clip or add the programme shown to their playlist for later consumption. These two user actions are interpreted as negative and positive user feedback respectively. The recommender uses this user feedback to update the ranking of the clips and select the next clip to be presented. Unfortunately, this application cannot be made available online due to rights issues.
The demonstration attracted interest primarily from conference delegates involved in the television and media industries. However, some universities were also interested in using the engine with their own training algorithms or for developing alternative user interfaces for recommender systems.
One keen subject of discussion was the ability to browse the raw data and correlation values behind the nearest neighbor model. This provides editorial staff with an explanation for the recommendations and gives them confidence that the system is working correctly. The structure of the nearest neighbor model also means that it can easily be modified to reflect editorial concerns where these arise.
For more information about any of the work described here please contact Chris Newell and Libby Miller.
Poster from the ViSTA-TV RecSys 2013 demonstration
ViSTA-TV is processing a vast amount of different data formats at various time scales: EPG data that comes mostly in a rich relational data format as well as annotations represented as triples, simple key value pairs that need to be processed under real time constraints, aggregated output data which needs to be queried flexibly for presentation purposes in dashboards and applications, and ontologically represented knowledge.
All these data collections in their respective processing contexts pose very different requirements on the technologies used for storage and retrieval. These are the choices we made for the ViSTA-TV platform:
Whereas each of the above choices has a good motivation for its individual use case, it also means that clients to these need to speak in different languages, often introducing heavyweight dependencies. In order to abstract from the actual implementation details of the data warehouse, we therefore decided to add an abstraction layer which serves all of the requests through a simple REST interface to which all clients can connect.
The ViSTA-TV Data Warehouse implemented by a RapidAnalytics server talking to various databases including MySQL, Cassandra, and a Mondrian OLAP Server.
This interface is implemented by a RapidAnalytics server which under the hood manages connections to relational database systems, Cassandra, and the OLAP server. To that end, we are using its built-in mechanism to create user-defined Web services using the point-and-click interface of RapidMiner which in turn comes with various connectors to all of the above.
In addition to being an interface for the processing engine and apps, RapidAnalytics is also used for scheduled execution of data integration workflows, for example for the integration with EPG data sources, data enrichment, feature extraction and model generation.
IBC is over and it was a great success for ViSTA-TV. Also at the last day we had many interesting conversations with people from industry and research. We established many contacts and we look forward to future collaborations in the context of both ViSTA-TV as an EU FP7 project but even more as a company.
Thanks go to the whole ViSTA-TV team who made the exhibition a success. First of all to all people who showed potential collaborators, customers and investors about the power of ViSTA-TV. Special thanks go to the core development team who made the demos a bright and shining success.
If you want to still meet with members of the core ViSTA-TV development team at Amsterdam, please contact us at info@vista-tv.eu. We will be in Amsterdam until Sep 19.
The new week started with lots of interesting contacts. Interestingly, today many start-up and quite a few research institutions came to visit our stand and learn how ViSTA-TV fits their needs.
Today and tomorrow, we have a representative from Rapid-I at the stand. If you are interested in the Data Mining aspect of ViSTA-TV, please visit us at stand 8G39.
Despite it was Sunday, the ViSTA-TV stand was still well-visited. People from various industries showed great interest in the both in the research project as well as in the commercialization. The ViSTA-TV stand team changed and the fresh people look forward to show potential collaborators and customers the power of ViSTA-TV.
The flow of visitors to the ViSTA-TV stand continues also today with visitors from SONY, PICO Digital, TIE Kinetix, Rightster, iexpertus, FMX, etc
People were interested in the Zattoo online recommendations using the same viewing data to personalize viewing preferences.
As well as in the online analytics we demonstrated at the booth:
And some analytics from BBC:
The IBC2013 exhibition officially started today and ViSTA-TV booth (Future Zone, 8.G39) attracted quite a lot of interest. The curious visitors came by to learn about the tools we have developed.
We had visits from companies like ESPN, Ericsson and Cisco who are looking to solve similar problems and hence interested in our solutions and possible collaborations. ViSTA-TV was of particular interest to broadcasters, cable TV providers, program metadata providers and researchers.
We thank everyone who stopped by for your valuable feedback and informative discussions. We look forward to hearing from you in the near future. See you tomorrow!
Today we set up our stand here at IBC. Our presence here is themed under the motto:
The following teaser video summarizes our presence:
Our stand is in the making and we are attaching some of the set up photos for your enjoyment. So if you are in Amsterdam, come and visit us at IBC, hall 8, stand G.39. Look for the Future Zone (otherwise you might get lost).
We will post more information during the next few days, including a peek at our promotional material, in case you cannot make it to Amsterdam.
View of our stand as you come into the Future Zone at IBC
Our stand during setup
Providing personalized recommendations of programs to the users is an important goal of the ViSTA-TV project. Since ViSTA-TV deals with content provided over the internet it has access to valuable user viewing behavior. This provides a world of information from which we can learn the user viewing preferences and subsequently recommend the most relevant shows for that user. This ensures the user a much richer experience of television watching as he does not have to squander time in going through the myriads of shows before finding the few he is interested in. The goal here is to provide a smart TV guide so to speak.
The domain however presents a few interesting challenges. Unlike providing recommendations for more perpetual items like movies, books or clothes the ephemeral nature of the TV programs makes it more demanding to use as an item by itself. Most programs like series, news, talk shows have a short lifetime and there are new shows coming on constantly. Another challenge is that subscribed users are continuously changing and increasing in numbers. So there is a significant cold start problem.
University of Zürich (UZH) and Technische Universität Dortmund (TUDo) have been tackling these very issues and learning recommender models under these constraints to provide informed recommendations. Both UZH and TUDo are building clustering based recommender models. UZH employs agglomerative clustering of users and programs based on the user program ratings. TUDo uses Tag Clustering to cluster users and programs based on the tags associated with the users and programs. The tags are the user and program features.
Both these approaches cluster the features of the users and programs and not the individual users and programs thereby circumventing the ephemeral nature of the specific users and programs themselves and dealing more with their inherent properties which are representative of these entities. From these clusters the individual users and programs can be identified and associated with each other to provide the recommendation.
ViSTA-TV successfully organized a workshop with UZH and TUDo as the main participants from 19 Aug 2013 to 23 Aug 2013 at UZH campus in Zürich with the aim of consolidating and collaborating the efforts of UZH and TUDo in designing and developing the recommender models and recommendation engine.
We will post the results of this cooperation in a future blog entry.
A new version of Cassandra-Esper was released to Maven Central. Version 0.3 now supports insert, update and delete operations as well as proper initialization. Cassandra-Esper provides an implementation of Esper virtual data windows as a distributed key-value store for background information.
Add Cassandra-Esper to your project as a Maven dependency:
<dependency> <groupId>ch.uzh.ifi.ddis.ifp</groupId> <artifactId>cassandra-esper</artifactId> <version>0.3.0</version> </dependency>
Cassandra-Esper is licensed under the GNU General Public License, version 2.
We are pleased to announce that two papers related to ViSTA-TV research were accepted for publication at the 9th International Workshop on Scalable Semantic Web Knowledge Base Systems (SSWS2013). This workshop will be co-located with the 12th International Semantic Web Conference (ISWC 2013) at Sydney.
The first paper is about “Scalable Linked Data Stream Processing via Network-Aware Workload Scheduling”. It presents the results of our experiments for minimizing network traffic among stream processing operators on different machines.
The second paper deals with “Eviction Strategies for Semantic Flow Processing” and investigates how to prune partial results in an intelligent way in case the system can no longer handle the amount of incoming data, e.g., in the case of bursts.
Both papers will be published sometime in September via CEUR-WS.
As the Swiss newspaper “Die Nordwestschweiz” reports, MediaPulse will start publishing the TV statistics again starting from August 2, 2013. Yet, this statistics will currently not include people watching TV over the internet.
The DDIS group at UZH released the first version of the cassandra-esper library at Maven central. The cassandra-esper library integrates Apache Cassandra via virtual data windows into the Esper complex event processing engine.
The ViSTA-TV engine can now directly query the data warehouse’s key-value store from its complex event processing part. Queries can be directly expressed as part of statements in the Esper Event Processing Language (EPL).
The DDIS group at UZH released the testng4esper library at Maven central. The testng4esper library facilitates writing TestNG tests for programs using the Esper complex event processing engine.
ViSTA-TV will presents its intermediate results at the International Broadcasting Convention (IBC) – the world’s premier event for professionals in the broadcasting industry. This is a great occasion for ViSTA to showcase its demos to the over 50’000 attendees from over 160 countries.
The project will be presenting under the following heading:
ViSTA-TV: Real-Time TV Audience Behaviour Analytics
The TV industry depends on exact information about viewership behavior. Current methods for collecting this information used by national TV rating agencies rely on specialized set-top boxes. This use of specialized hardware practically limits the number of households sampled and provides a single point of failure (see the technical and legal problems in Switzerland in January). Furthermore, the data is often only available with significant delay.In the context of ViSTA-TV (Video Stream Analytics for the TV Industry), an EC-funded project, we combined linked open data (LOD) with behavioral information from IPTV. This allows ViSTA-TV to provide highly accurate, real-time and fault-tolerant market research information based on a much larger number of viewers.At IBC, ViSTA-TV will demonstrate prototype applications enabled by real-time information obtained from live streams of IPTV audience data:
The ViSTA-TV project successfully passed the review meeting at Luxembourg on June 18. ViSTA-TV is on track. The project wishes to express their gratitude to the reviewers and the project officer for their very valuable comments. The ViSTA-TV team is eager to continue making the project a success and presenting their achievements at the IBC Future Zone.
The DDIS group at UZH released the streams-esper library on Maven central. The streams-esper library allows to use the Esper complex event processing engine with the Streams platform for stream processing. It enabled the ViSTA-TV engine to query and match complex events on streams of IPTV features. The ViSTA-TV project is hence in the position to compute real-time vieweship statistics and content-based recommendations.
ViSTA-TV was invited for the Future Zone of the International Broadcasting Convention (IBC) 2013. More information to follow soon.
The European Commission released a feature story on December 18, 2012 about “TV and the internet: a marriage made in entertainment heaven“. The article discusses a set of innovations possible through the cross-fertilization of the TV and Internet technology. The article mentions a number of relevant research projects
About ViSTA-TV it mentions:
The recently launched Vista-TV (5) project is developing a system to extract, mine and analyse anonymised viewing data from connected TV users. The end result, the project team hopes, will be the creation of an entirely new SME-driven market in TV viewing-behaviour information.
We at ViSTA-TV are proud to be mentioned in such a release and are looking forward to publish first results about the project here mid 2013.
The ViSTA-TV project accomplished its first major milestone. In a collaborative effort, a stream-processing framework for the ViSTA-TV platform was established. Based on Storm, the partners created sub-topologies which can be glued together visually using RapidMiner. Further achievements include measures for dealing properly with privacy and data protection as well as creating data sets for testing the system.
Check out the public deliverable reports and learn about the project’s progress in more detail:
The kick-off ViSTA-TV Press Release (in German) has been sent out.
Live video content is increasingly consumed over IP networks in addition to traditional broadcasting. The move to IP provides a huge opportunity to discover what people are watching in much greater breadth and depth than currently possible through interviews or set-top box based data gathering by rating organizations, because it allows direct analysis of consumer behavior via the logs they produce. The ViSTA-TV project proposes to gather consumers’ anonymized viewing behavior and the actual video streams from broadcasters/IPTV-transmitters, to combine them with enhanced electronic program guide information as the input for a holistic live-stream data mining analysis.
ViSTA-TV will employ the gathered information via a stream-analytics process to generate a high-quality linked open dataset (LOD) describing live TV programming. Combining the LOD with the behavioral information gathered, ViSTA-TV will be in the position to provide highly accurate market research information about viewing behavior that can be used for a variety of analyses of high interest to all participants in the TV-industry. ViSTA-TV will employ the information gathered to build a recommendation service that exploits both usage information and personalized feature extraction in conjunction with existing metadata to provide real-time viewing recommendations.
These results will be made possible by scientific progress in data-stream mining consisting of advances in data mining for tagging, recommendations, and behavioral analyses and temporal/probabilistic RDF-triple stream processing.
ViSTA-TV is a European Union-funded research project, beginning on 1 June 2012, and lasting for two years. The partners are:
