Data Analysis as Part of the Co-Creative Process

Authors: Tomas Kjellqvist, Lina Rådmark, David Borgström, Reidun Norvoll & Julie Ridley

Handbook of Youth Social Citizen Science (Borgström, D., Canto-Farachala, P., Hagen, A. L., Norvoll, R., Rådmark, L. & Lorenzen, S.B. (Eds.). (2024). Handbook of Youth Citizen Social Science. Working with Young People and the Local Community for Social Change. Zenodo. https://doi.org/10.5281/zenodo.10566411)

Analysis Starts at the Beginning of the Project

In a way, making sense of the data in a co-creative youth citizen social science project starts at the very beginning of the project. When preparing the formulation of the research question, before the actual research has even started, it is important to involve a core group of young co-researchers, and to allow them to share inputs and voice concerns about how the research questions are framed and formulated. Open, clear and dialogic communication is vital in this initial phase. As a researcher, it is crucial to be reflexive and open to adapting to the ideas and suggestions of the young citizen scientists. However, it is also important to establish clear roles and responsibilities.

Ultimately, the researcher bears the responsibility for ensuring the scientific quality of the research questions, data collection, analysis, and conclusions. The research questions, once formulated, guides discussions and decisions around both data collection and analysis. Our recommendation from the YouCount project is to be pragmatic when selecting methods for data collection, bearing in mind the complexity and workload in the forthcoming analysis. Participatory data analysis takes time, and remember, these considerations must take into account the points of view of the young citizen scientists. In some studies, depending on the methodological design, doing a pilot study can be beneficial as it makes it possible to check the relevance, reliability and validity of the design, e.g. test different ways of collecting data. If a pilot study is done, it is of course critical to then take the young coresearchers’ feedback into account when designing the final study. This was a key starting point when developing both the YouCount app, as well as other methods for data collection in the project (Ridley et al., 2021). By ensuring that the study design, including methods for data collection and analysis, is feasible, you increase both the relevance and quality of the data, the possibility of systematically analysing that same data, and the likelihood that the young citizen social scientists find their participation meaningful the remaining data into a manageable set. In youth citizen social science, the process of cleaning the data for analysis is often led by academic researchers, but whenever possible, should also involve contributions from the young citizen scientists. Since the data collected is so close to their everyday lives, the young people are in a unique position to understand and interpret what is, and is not, relevant to the analysis. In the YouCount project, one such cleaning process involved sorting all the various quantitative data provided in the YouCount app and delete observations that did not come from the young citizen scientists, adjust misplaced or misspelt observations, and to validate each observation as correct and relevant. The cleaning process took a long time, but resulted in reliable, rich, and valuable data (Ridley et al., 2003).

A similar process for qualitative data involves doing an initial sorting before subjecting the data to more systematic analysis. In the YouCount project a large portion of the qualitative data was photos from workshops and meetings depicting posters, post-it notes, drawings and other forms of “unstructured” documentation. An important first step in analysing this data was to sort it into broad thematic categories, often together with the young co-researchers. Only then, a more systematic round of participatory data analysis was possible ( Pataki et al., 2023).

Qualitative Participatory Data Analysis

After these initial preparations, the next step is to co-analyse the data together with the young citizen scientists. An example of such participatory analysis is the coding of qualitative data, such as questionnaires or interview transcripts. Coding can either be conducted as an exercise where the coders – academic researchers and young citizen scientists – start by reading, discussing and categorising the statements without any predetermined themes. Comparing the different outcomes in a discussion would eventually lead to a common ground on which codes to use or to a problematisation that would require further studies through comparison. It should be noted that a free-text response may contain several statements that could be assigned different codes. The final sorting of the dataset into groups that have received similar codes would, in some cases, require a weighting of the importance the codes have for the research question.

Deductive coding is a different approach where the researchers, possibly in discussion with the core group of young citizen scientists, have decided on a set of codes to use. The tasks in the coding exercise would then be to sort the free-text statements into groups according to the predefined codes. In YouCount, the researchers, with the assistance of some young citizen scientists, utilised qualitative data coding (e.g., minutes, transcripts, and case summary reports) in the cross-case data analysis of experiences, social inclusion, and the evaluation of participation outcomes and impacts. For instance, in the evaluation study (Saumer et al., 2023), numerous interviews were initially categorised into broader predefined themes reflecting common outcomes in citizen science (such as “scientific knowledge”), before further division into sub-category topics (such as “citizen education”). The coding process was facilitated using qualitative data analysis programs, such as Nvivo, Atlas.ti, and others.

Pseudo-Quantitative Data

When it comes to analysis of pseudo-quantitative data in citizen social science more generally, a well-executed coding exercise provides opportunities to quantify free-text responses to make it possible to integrate them into the quantitative analysis. Additionally, qualitative data can be utilised in quantitative analysis through methods such as tallying response frequencies to specific questions or counting keywords within responses.

Quantitative Participatory Data Analysis

The YouCount project employed a mixed-methods approach to gain deeper insights into young people’s experiences with social inclusion and the outcomes of their participation as young citizen scientists. This encompassed the collection of quantitative data through the YouCount app and questionnaire data in the process evaluation study Quantitative data analysis often necessitates the use of computerised statistical tools, ranging from Excel to professional packages like SPSS, SAS, etc. There are also open-source opportunities available, such

as the programming language R, although it may entail a learning curve. For semi-skilled R-programmers, there are possibilities to develop user interfaces that enable participatory quantitative data analysis. However, this would require preparations that typically exceed the time constraints of most research projects. Ordinarily, quantitative analysis would be conducted by the researchers using statistical tools, and followed by a visualisation and presentation for discussion with the young citizen scientists. During these discussions, the analysis progresses from the technical stage to one of interpretation and explanation. Simpler statistical associations, like examining how a set of qualitative responses correlates with factors such as age, gender, residential area, or other distinct characteristics, could be conducted through participatory exercises. In these exercises, the young co-researchers can engage in sorting the material. This sorting process can be facilitated using digital tools or preferably through hands-on exercises using printed materials. For instance, if the material is in print, the group could divide tasks to identify specific associations, such as sorting a particular cluster of free-text responses into piles based on gender, age groups, and so forth. The responsibility for reporting the findings from the participatory data analysis lies with the professional researchers. It is crucial that all participatory processes are thoroughly documented for future reference. This documentation should encompass a detailed account of the process, including a discussion on challenges, problematic aspects and the items that sparked the most debate.

Why Participatory Data analysis

The process of conducting participatory analysis is undeniably more complex. It encompasses not only a research endeavor but also functions as an educational and dissemination process, aligning well with the objectives of citizen social science. Adequate budget allocation during project preparations is essential to accommodate the additional time required for co-creation. However, the benefits are substantial: a meticulously executed participatory process is likely to yield more reliable and valid data, resulting in findings that are more pertinent. Moreover, the transition from research to impact may be considerably expedited.