[kdd] CFP: REDD 2014 - ACM RecSys Workshop on Recommender Systems Evaluation: Dimensions and Design

Tue May 13 08:29:46 PDT 2014

------------------ 1st Call for Papers - REDD 2014 ------------------
                 International ACM RecSys Workshop on
  Recommender Systems Evaluation: Dimensions and Design - REDD 2014
          Foster City, Silicon Valley, CA, USA, October 2014
                    http://ir.ii.uam.es/redd2014
---------------------------------------------------------------------

* Submission deadline: 21 July 2014 *

Scope
-----

Evaluation is a cardinal issue in recommender systems; as in almost any
other technical discipline, it highlights to a large extent the problems
that need to be solved by the field and, hence, leads the way for
algorithmic research and development in the community. Yet, in the field of
recommender systems, there still exists considerable disparity in
evaluation methods, metrics, and experimental designs as well as a
significant mismatch between evaluation methods in the lab and what
constitutes an effective recommendation for real users and businesses. This
workshop aims at providing an informal forum to tackle such issues and move
towards better understood and commonly agreed evaluation methodologies,
allowing one to leverage the efforts and the workforce of the academic
community on meaningful and relevant directions to real-world developments.

REDD 2014 places a specific focus, on the one hand, on the identification
and measurement of different recommendation quality dimensions that go
beyond the monolithic concept of simply matching user preferences. Novelty
and diversity, for instance, have been recognized as key components of the
utility of recommendations for users in real-world scenarios, with a direct
positive effect on business performance. Considering the business
perspective, performance metrics related to sales, revenues, and user
engagement along the recommendation funnel should also be used.
Additionally, from an engineering point of view, aspects such as
efficiency, scalability, robustness, and user interface design are
typically major concerns; often prioritized over the effectiveness of the
internal algorithms at the core of the system. On the other hand, once a
relevant target quality has been defined, a clear evaluation protocol
should be specified in detail and agreed upon, allowing for the comparison,
replicability and reproducibility of the results and experiments by
different authors and enabling incremental contributions.

REDD 2014 aims at gathering researchers and practitioners interested in
better understanding the unmet needs of the field in terms of evaluation
methodologies and experimental practices. The main goal of this workshop is
to provide an informal setting for discussing and exchanging ideas,
experiences, and viewpoints. REDD seeks to identify and better understand
the current gaps in recommender system evaluation methodologies, help lay
directions for progress in addressing them, and foster the consolidation
and convergence of experimental methods and practices.

Topics
------

We invite the submission of papers reporting original research, studies,
advances, or experiences that focus on recommender system utility
evaluation. The topics that the workshop seeks to address include--though
need not be limited to--the following:

* Recommendation quality dimensions
  - Effective accuracy, ranking quality
  - Novelty, diversity, unexpectedness, serendipity
  - Utility, gain, cost, risk, benefit
  - Robustness, confidence, coverage, ease of use, persuasiveness, etc.
* Matching metrics to tasks, needs, and goals
  - User satisfaction, user perception, human factors
  - Business-oriented evaluation
  - Multiple objective optimization, user engagement
  - Quality of service, quality of experience
* Evaluation methodology and experimental design
  - Definition and evaluation of new metrics, studies of existing ones
  - Adaptation of methodologies from related fields: IR, Machine Learning,
HCI, etc.
  - Evaluation theory
* Practical aspects of evaluation
  - Offline and online experimental approaches
  - Simulation-based evaluation
  - Datasets and benchmarks
  - Validation of metrics
  - Efficiency and scalability
  - Open evaluation platforms and infrastructures

Submission
----------

Two submission types are accepted: technical papers up to 6 pages long, and
position papers up to 3 pages. Each paper will be evaluated by at least two
reviewers from the Programme Committee. The papers will be evaluated for
their originality, contribution significance, soundness, clarity, and
overall quality. Within a required quality standard, position papers will
be evaluated based on the presentation of new perspectives and insights,
and their potential for provoking thoughts and stimulating discussion.

All submissions shall adhere to the standard ACM SIG proceedings format:
http://www.acm.org/sigs/publications/proceedings-templates. The accepted
papers will be published in the CEUR Proceedings series.

Submissions shall be sent as a pdf file through the online submission
system now open at: http://www.easychair.org/conferences/?conf=redd2014.

Important dates
---------------

Paper submission deadline: 21 July 2014
Author notification:       21 August 2014
Camera ready version due:  5 September 2014
REDD 2014 workshop:        October 2014

Programme Committee
-------------------

Marcel Blattner, Univ. of Applied Sciences, Switzerland
Iván Cantador, Universidad Autónoma de Madrid, Spain
Charles Clarke, University of Waterloo, Canada
Arjen de Vries, CWI, The Netherlands
Juan Manuel Fernández, Universidad de Granada, Spain
Zeno Gantner, Nokia, Germany
Ido Guy, IBM Haifa Research Lab, Israel
Juan Huete, Universidad de Granada, Spain
Kris Jack, Mendeley, UK
Dietmar Jannach, University of Dortmund, Germany
Jaap Kamps, University of Amsterdam, Netherlands
Alexandros Karatzoglou, Telefonica Research, Spain
Bart Knijnenburg, University of California, Irvine, USA
Till Plumbaum, TU Berlin, Germany
Filip Radlinski, Microsoft, Canada
Alan Said, TU Delft, Netherlands
Yue Shi, TU Delft, The Netherlands
Fabrizio Silvestri, Yahoo!, Spain
David Vallet, Google Inc., Australia
Jun Wang, University College London, UK
Xiang-Jun Wang, Netflix
Xiaoxue Zhao, University College London, UK

Organizers
----------

Panagiotis Adamopoulos, New York University, USA
Alejandro Bellogín, Universidad Autónoma de Madrid, Spain
Pablo Castells, Universidad Autónoma de Madrid, Spain
Paolo Cremonesi, Politecnico di Milano, Italy
Harald Steck, Netflix, USA

Contact email: redd2014 at easychair.org

More info at: http://ir.ii.uam.es/redd2014
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/kdd/attachments/20140513/a9bb9f93/attachment-0001.html>