Web Scrapping as a Data Collection Technique in Communication Research

The subject of this study, which was prepared within the scope of the “Empowering Teacher Competence in Detecting and Preventing Child Abuse” Project, with the short name ProChild, funded within the scope of the Erasmus+ Program School Education Strategic Partnerships action carried out by the Turkish National Agency, is to put forward a model for the use of the Web Scraping technique in the Communication research that will reveal Turkey’s news agenda by making use of data science and thus contribute to the sustainability of the project. Considering the data to be obtained from the data collection tool as well as the contribution of the technique itself to the sustainability potential of the project, this study also has the potential to contribute to the development of communication research in Turkey. The increasing digitalization of communication has created a globally dense supply of data, significantly diversifying and intensifying the data sources that can be used in communication research. However, it can be stated that new research approaches based on the techniques of using these data sources to realize research objectives have not been sufficiently developed in communication research in Turkey. This study is motivated by this very deficiency and aims to concretize a web scraping model for communication research, which is carried out to follow the news agenda of Turkey in certain historical periods within the scope of meta-journalism research. In this context, the model, which enables the analysis of 16 internet news sites within the framework of “news content qualities” and “metadata” through a script written in Python language, is structured and presented with its stages.