Taobao Serendipity Dataset

I. Serendipity Data Collection

We conducted a user survey on a popular mobile e-commerce platform in China (Mobile Taobao) starting from Dec. 21, 2017. The users were able to access the survey's link after they logged in the system. If a user volunteered to take part in, s/he first received a recommended product (that was generated by one of our tested algorithms, and was from one of various product domains such as "clothes," "toys," "home appliances," "foods," etc.) including its name, image, short description, and price. The user then completed a questionnaire that assessed her/his immediate feedback on this recommendation. S/he was also asked to fill two psychological quizzes, one about her/his curiosity via ten-item Curiosity and Exploration Inventory-II (CEI-II) [1] and another about her/his Big-Five personality traits via Ten-Item Personality Inventory (TIPI) [2]. As the incentive, all of the participants were placed in a lottery draw with customized presents as awards given to the winners.

Till March 17, 2018, we received 13,741 users' responses. We carefully checked all of the responses in order to filter out invalid answers. For example, if a user did not answer all of the questions, or gave the same rating to two opposite questions, her/his response was deleted. In addition, we only kept the user's first response if s/he took the experiment more than once. We also tried to ensure that the recommendation was not clicked by the user in the past.

As a result, 11,446 users remained, among whom we further removed some outlier cases (17 cases in which the user clicked either less than 5 or more than 15,000 items before taking the survey, 12 cases where the recommendation's top-level category appeared less than 10 times within all users' profiles, and 34 cases where the recommendation's top-level category is "Uncategorized"). Finally, we have 11,383 users' records (7,769 females).

This dataset contains 11,383 users' feedback as collected through the user survey. Specifically, nine questions were included in our survey to assess a user’s perceptions of the recommendation (each responded on a 5-point Likert scale from "strongly disagree" to "strongly agree"), as in Table 1.

This dataset (or part of it) has been used for our analyses in [3] and [4].

II. Usage License

Neither Taobao, nor Hong Kong Baptist University, nor any of the researchers involved can guarantee the correctness of the data, its suitability for any particular purpose, or the validity of results based on the use of the dataset. The dataset can be used for any research purposes under the following conditions:

• The user may not state or imply any endorsement from Taobao, Hong Kong Baptist University, or any of the researchers involved.
• The user must acknowledge the use of the dataset in her/his publications resulting from the use of the dataset (see below for citation information).
• The user may not redistribute the data, as long as it is distributed under the same license conditions.
• The user may not use this information for any commercial or revenue-bearing purposes without first obtaining permission from the researchers involved.

In no event shall Taobao, Hong Kong Baptist University, and their affiliates or employees be liable to you for any damages arising out of the use or inability to use the data (including but not limited to loss of data or data being rendered inaccurately).

If you have any further questions or comments, please email nxwang@comp.hkbu.edu.hk and lichen@comp.hkbu.edu.hk.

III. Citation

To acknowledge the use of the dataset in your publications, please cite one or both of the following papers:

(Wang et al. 2020) Ningxia Wang, Li Chen, and Yonghua Yang. 2020. The Impacts of Item Features and User Characteristics on Users' Perceived Serendipity of Recommendations. In Proceedings of the 28th Conference on User Modeling Adaptation and Personalization (UMAP '20), 9 pages.

(Chen et al. 2019) Li Chen, Yonghua Yang, Ningxia Wang, Keping Yang, and Quan Yuan. 2019. How Serendipity Improves User Satisfaction with Recommendations? A Large-Scale User Evaluation. In The World Wide Web Conference (WWW '19), New York, NY, USA, 240-250. https://doi.org/10.1145/3308558.3313469.

IV. Content and Use of Survey Data File (umap2020_TSD_Wang.csv)

1. URL

https://github.com/greenblue96/Taobao-Serendipity-Dataset

2. Formatting and Encoding

These files were encoded as UTF-8.

3. User Ids, Item Ids, and Category Ids

Those ids have been anonymized.

4. Feedback Ratings

Users gave feedback to the received recommendation, by rating the nine statements (see Table 1) on a 5-point Likert scale from "strongly disagree" to "strongly agree".

5. Survey Data File Structure (umap2020_TSD_Wang.csv)

The file contains the following 34 fields:

Basic information

1. UserId – user id;
2. ItemId – the recommended item’s id;
3. CategoryId - the id of the item's category (e.g., passion fruit, toaster, dining table, etc.);
4. Clicked - whether the user clicked the recommendation (2 values: 0 for "not clicked" / 1 for "clicked");
5. Purchased - whether the user purchased the recommended item or added it to cart within the same day after taking part in the survey (2 values: 0 for "not purchased" / 1 for "purchased");

The following fields (i.e., 6-24) use a 5-point Likert scale, where 1 – "strongly disagree", 2 – "disagree", 3 – "neither agree nor disagree", 4 – "agree", 5 – "strongly agree".

Assessment questions of user perceptions of the recommendation (see Table 1)

6. Relevance
7. Novelty
8. Pur_diversity
9. Rec_diversity
10. Unexpectedness
11. Serendipity
12. Timeliness
13. User satisfaction
14. Purchase intention

Ten-item Curiosity and Exploration Inventory-II (CEI-II) [1]

15. C1 - "I actively seek as much information as I can in new situations."
16. C2 - "I am the type of person who really enjoys the uncertainty of everyday life."
17. C3 - "I am at my best when doing something that is complex or challenging."
18. C4 - "Everywhere I go, I am out looking for new things or experiences."
19. C5 - "I view challenging situations as an opportunity to grow and learn."
20. C6 - "I like to do things that are a little frightening."
21. C7 - "I am always looking for experiences that challenge how I think about myself and the world."
22. C8 - "I prefer jobs that are excitingly unpredictable."
23. C9 - "I frequently seek out opportunities to challenge myself and grow as a person."
24. C10 - "I am the kind of person who embraces unfamiliar people, events, and places."

The following fields (i.e., 25-34) use a 7-point Likert scale, where 1 – "strongly disagree", 2 – " moderately disagree", 3 - "disagree a little", 4 – "neither agree nor disagree", 5 - "agree a little", 6 – "moderately agree", 7 – "strongly agree".

Ten-Item Personality Inventory (TIPI) [2]

("I see myself as")

25. P1 - "Extraverted, enthusiastic."
26. P2 - "Critical, quarrelsome."
27. P3 - "Dependable, self-disciplined."
28. P4 - "Anxious, easily upset."
29. P5 - "Open to new experiences, complex."
30. P6 - "Reserved, quiet."
31. P7 - "Sympathetic, warm."
32. P8 - "Disorganized, careless."
33. P9 - "Calm, emotionally stable."
34. P10 - "Conventional, uncreative."

V. Ethical Considerations

1. How were Participants informed about the Data Collection?

   Before the survey, each participant was presented with a consent form that described the purpose of this experiment and data collection.

2. Was there an Ethical Review?

   The user survey was approved by Alibaba's search & recommendation platform.

3. If it relates to people, were they told what the dataset would be used for, and did they consent?

   Participants had to confirm a consent form that explains what the data is being collected for and how it is going to be used.

VI. References

[1] Todd B. Kashdan, Matthew W. Gallagher, Paul J. Silvia, Beate P. Winterstein, William E. Breen, Daniel Terhar, and Michael F. Steger. 2009. The Curiosity and Exploration Inventory-II: Development, Factor Structure, and Psychometrics. Journal of Research in Personality. 43, 6 (December 2009), 987–998. https://doi.org/10.1016/j.jrp.2009.04.011.

[2] Samuel D. Gosling, Peter J. Rentfrow, and William B Swann. 2003. A Very Brief Measure of the Big-Five Personality Domains. Journal of Research in Personality. 37, 6 (December 2003), 504–528. DOI:https://doi.org/10.1016/S0092-6566(03)00046-1.

[3] Ningxia Wang, Li Chen, and Yonghua Yang. 2020. The Impacts of Item Features and User Characteristics on Users' Perceived Serendipity of Recommendations. In Proceedings of the 28th Conference on User Modeling Adaptation and Personalization (UMAP '20), 9 pages.

[4] Li Chen, Yonghua Yang, Ningxia Wang, Keping Yang, and Quan Yuan. 2019. How Serendipity Improves User Satisfaction with Recommendations? A Large-Scale User Evaluation. In The World Wide Web Conference (WWW '19), New York, NY, USA, 240-250. https://doi.org/10.1145/3308558.3313469.