- STIVA only published a short summary of the results and a brief description of the methodology of the study, which makes it not possible to verify how the research has exactly been conducted. It also makes it impossible to replicate this research, a generally recognized principle of scientific research. EUCAM contacted STIVA for access to the original report, but received a negative response.
- The short summary given by STIVA does not indicate how is dealt with the so-called ‘inter-rater reliability’ in the analysis of a total of 5815 statements (2620 beer brands, 695 wine brands, and 2500 distilled liquor brands). Social media marketing of nine alcohol brands on four social media channels (Facebook, Twitter, Instagram and Pinterest) have been analysed at 3 different moments. Per moment, profiles of 100 ‘commenters’ on the social media advertisement were manually viewed. However, it is not clear how many researchers have been involved in this study and in what way it has been attempted to limit the differences in these measurements, which are likely to be performed by a number of different researchers.
- The STIVA study raises questions about how the age of the social media users is determined. The summary mentions that in case of doubt about the age, it is tried to establish the age reaching the profile through another channel. However, how is this done exactly, and through which other channel? Also, in what cases was there doubt about the age, based on what? And in how many cases has this occurred, and in how many cases was it not possible to establish the user’s age through another channel?
- The study does not present a complete and realistic image of the exposure to alcohol marketing through social media, because only people who responded to the advertisement by means of a ‘comment’, ‘like’ or ‘share’ have been included in the study. However, the researchers did not look at the ‘followers’ of the alcohol brand on the social media channels. The reason for this according to the researchers, is that in this way “it can be assumed that they have actually seen the advertisement message”. There is some truth in that, but young people can frequently be exposed to alcohol marketing through social media by simply following these channels, without having actual interaction.
- The study was conducted at three different moments, between April and September 2015. These moments are close to each other, whereas it is now presented as if the amended Advertising code for Alcoholic drinks had a positive effect over the past five years. The study also does not describe the times at which the social media advertisements were analyzed. For example, was this during the day, when most young people are at school?
- Earlier research has shown that many young people on social media lie about their age to gain access to these channels.- For that reason, an age on social media, says not much about the actual age of an user. This finding undermines the whole foundation of this study Moreover, many users on social media have a private profile, and therefore the personal information is not visible. The report states that it was not in all cases possible to analyze 100 commenters per measuring moment; sometimes because there were no more (or no) responses available and sometimes because relatively many users protect their profile. In those cases, the research period has been extended or a more qualitative analysis has been used. However, when was this the case, what was the consequential attrition and what did this more qualitative analysis look like?
European Centre for Monitoring Alcohol Marketing (EUCAM) Supporting organisations: Dutch Institute for Alcohol Policy STAP, Alcohol & Society Denmark, AV.OG.TIL Norway, IOGT-NTO Sweden and Eurocare Italia
Contact: D. Lenssen MSc LL.M, Researcher, email@example.com Postbus 9769, 3506 GT Utrecht
 http://stiva.nl/nieuwsberichten/zelfregulering-alcoholmarketing-social-media-al-5-jaar-succesvol/  De Bruijn, A., Engels, R., Anderson, P., Bujalski, M., Gosselt, J., Schreckenberg, D., Wohtge, J. & De Leeuw, R. (2016). Exposure to online alcohol marketing and adolescent’s drinking: a cross-sectional study in four European countries. Alcohol and Alcoholism, 1-7, http://dx.doi.org/10.1093/alcalc/agw020  De Bruijn, A., Tanghe, J., De Leeuw, R., Engels, R., Anderson, P., Beccaria, F., Bujalski, M., Celata, C., Gosselt, J., Schreckenberg, D., Słodownik, L., Wothge J. & Van Dalen, W. (2016). European longitudinal study on the relationship between adolescents’ alcohol marketing exposure and alcohol use, Addiction, 10.1111/add.13455  Winpenny, E.M., Marteau, T.M. & Nolte, E. (2014). Exposure of Children and Adolescents to Alcohol Marketing on Social Media Websites. Alcohol and Alcoholism, 49(2), 154–159, doi: 10.1093/alcalc/agt174  O’Neill, B., Grehan, S. & Ólafsson K. (2011). Risks and safety for children on the internet: the Ireland report. LSE, London: EU Kids Online.  Landon, J., Graff, H. & Westerman, L. (2015). Alcohol marketing and young people – a literature review and mapping exercise. UK Health Forum for Public Health England.