Data Collection Methods
Table of Contents
Role of the Boys & Girls Club of Hartford
Our approach
To construct a comprehensive and nuanced narrative on cultural responsiveness in the Boys and Girls Club of Hartford, we conducted a literature review, formulated and disseminated an online survey for adult BGCH stakeholders and assembled a digital culturally responsive activity library.
Our process
To build our literature review, we read, evaluated, and synthesized dozens of academic articles relating to cultural responsiveness in after-school programming. We began by inputting critical information about each article (methodology, key findings, etc.) into a spreadsheet. Afterwards, we identified emergent themes in the literature and took note of when articles diverged and converged in findings, methodology, or argument. Finally, we wrote the literature review, which included the following sections: abstract, introduction, after-school activities, after-school activities and the achievement gap, theoretical framework, cultural responsiveness, tenets of culturally responsive after school activities, discussion, areas for future research and conclusion. Our review of the literature concluded that caring relationships, a community of belonging, student voice, critical consciousness and activism and parent involvement are all critical features of robust culturally responsive afterschool programming.
We constructed our activity library using activities found by asking relevant organizations for culturally responsive teaching resources as well as by researching relevant teaching and social justice organizations online. When we identified an activity that may be relevant to the BGCH (specifically activities relating to our core research areas of Indigenous People’s Day, Women’s History Month, Pride, Hispanic Heritage Month and Black History Month) we added it to an online document of possible activities. Once the activity collection process was over, we organized the activities by subject and included information on the recommended age group, objectives and directions of the activity, as well as a picture to accompany the description.
Arguably the centerpiece of our research is the BGCH parent survey to evaluate the effectiveness of the BGCH culturally responsive programming. The idea for the survey emerged through conversations with our community partners. We then began to draft questions for the survey, as well as a consent form. The survey is completely anonymous and included a section of fixed-answer demographic questions, a section of 5-point scale cultural activity satisfaction questions on the cultural area chosen by the participant (Black History Month, Indigenous Peoples’ Day, etc.) as well as a brief section with short-answer questions in which respondents could add suggestions and expand on prior answers. At the end of the survey, respondents were invited to complete a brief Google Form with their personal information to be entered into a drawing of one of eight $50 Amazon Gift Cards. The survey was sent to our community partners at the BGCH for approval. After being submitted to and accepted by the IRB, we uploaded our survey to the online survey platform Qualtrics. Finally, the survey was distributed by email from the BGCH to a comprehensive list of BGCH partners, parents, volunteers, employees and other adult affiliates.
View our survey questions here.
Data analysis methods
Step 1: Exporting the data
To begin the data analysis process, we exported the survey responses from Qualtrics onto Excel. In doing so, we chose to represent responses numerically, to allow us to better chart our data. Many of our survey questions asked respondents to answer on a 5-point scale (e.g. strongly agree to strongly disagree). These responses were then collapsed into their numerical value rather than their text response (i.e. 3 for unsure).
Step 2: Cleaning the data
The next stage in the data analysis process was cleaning the data. We removed responses that didn’t answer the questions or answered only one question as well as responses that engaged in the survey for a very small amount of time (e.g. 30 seconds). We also flipped the 5-point scale responses, so that 1 would represent the least satisfaction and 5 would represent the most, while maintaining the content of their responses.
Step 3: Visualizing the data
Lastly, we visualized the data by creating bar charts that reflects the survey responses regarding programming satisfaction, activity enjoyment, cultural content accuracy, and interdisciplinary engagement for each of the programming areas responded to. We chose this data visualization because it provides a synthesis of parent satisfaction across each of the programming areas.
Why these methods?
To gather information on community perspectives of the BGCH’s afterschool programming, the method we chose was an online survey.
–Compared to other research tools, online surveys are relatively easy to disseminate to large numbers of people, all at once. Since including a diverse range of voices was extremely important to the information we are attempting to gather, we sought out a research tool that is conducive to high response rates: the survey.
–Surveys typically require less of a time commitment than interviews and require less planning and preparation for the respondent. This is especially important when collaborating with economically marginalized communities, where respondents may not have much time.
For these reasons, we chose a digital survey to gather information on parent perspectives of BGCH cultural programming.
Role of the Boys & Girls Club of Hartford
The Hartford Boys and Girls Club played a significant role in communicating the needs, boundaries, and expectations that Hartford residents have in terms of the standards they set for furthering diversity, equity, and inclusion in after-school activities and events.
-They provide an advanced level of expertise on the social situation of the club that our team as researchers can not replicate.
-We received ideas as to where we should begin our research proposal.
-They disseminated the survey to BGCH parents.
The Hartford Boys and Girls Club has had a substantial impact on how we defined and maintained our project purpose, selected our research methods, formulated our literature review, made decisions on the type of sources that should be included in our investigation and compared with our observations, and on how we can ensure benefits for the researched population as a result of our research findings.
Challenges and limitations
1. Small survey sample size
Due to the small sample size of our survey (n=14) our survey results cannot be claimed to be representative of all of the diverse perspectives of BGCH parents, employees, and volunteers. Such few responses make it difficult to draw definitive conclusions regarding community perceptions of BGCH cultural programming.
We hypothesize that the small sample size may be a consequence of the time constraints and limited capacities of our target population: community members affiliated with the BGCH. Residents may be working multiple jobs, have caregiving obligations, or are overburdened by the stressors associated with racism, poverty, and injustice. Because of this, encouraging participation in an optional survey may be particularly difficult.
Additionally, our survey size may be small due to the fact that the survey was digital, rather than written. For those living in entrenched poverty and those who are older, using a digital survey tool may have been a barrier to survey participation. In the future, researchers may consider providing a paper-and-pencil survey option to participants.
2. Limited time frame
Because the Liberal Arts Action Lab operates as a semester-long program, our research project was limited in time. A longer project window may have allowed for a more robust survey dissemination process and a more comprehensive culturally responsive activities library.
3. Narrow geographical focus
Due to our own limited capacity as students, we were only able to disseminate surveys to Boys and Girls Club of Hartford branches in Hartford. For this reason (and our small sample size), our findings cannot be applied to other Boys and Girls Club locations which may have a different cultural activity curriculum and participant demographic makeup.