Thinkery Connect Research Project
-
Conduct research on caregiver and child interaction and scientific thinking and learning using eye-tracking technologies
-
Evaluate visitor experience
-
Prototype exhibit design features and modifications
-
Collaborate with community partners throughout the city and the state to support early childhood education and development
Organization
Overview
Thinkery Connect is a long-term collaborative project between Thinkery (a children's museum in Austin, Texas) and the University of Texas at Austin, aimed at understanding and supporting scientific thinking in young children through informal learning experiences using eye-tracking technologies. The project centers around how parents and children explain, explore, and engage with science concepts, and how parental scaffolding, exhibit modification, and signage influence these processes.
We analyzed extensive behavioral observation and survey data from parent-child dyads to uncover how families interact with exhibits, how parents guide children's learning, and what factors influence children's engagement and inquiry. These insights inform evidence-based design recommendations aimed at enriching the museum experience for both children and caregivers.
Team
-
1 Principal Investigator,
-
1 Research Technician,
-
1 Graduate Student,
-
7 Research Assistants
My Role
Project Lead - Research Technician
Duration
Long Term Research
Tools
-
Eye-tracking Glasses,
-
Adobe Primere,
-
Datavyu,
-
R Studio
Skills
-
Quantitative and Qualitative Research
-
Project Management
-
Datavyu Software Coding
-
Data Analysis
-
R programming
Responsibilities
As a project lead, my responsibilities including oversee project planning, timeline and planning. I manage mixed-methods research initiatives including data collection, data management, data cleaning, data coding, reliability testing, and data analysis. I develop standardized research workflows, create behavioral coding rubrics, and automate data pipelines using Ruby and shell scripting. I supervise and delegate tasks to research assistants and lead cross-functional collaboration with internal team, collaborators and external stakeholders. Additionally, I am also responsible in preparing grant application and research publication writing.
Process
1. Recruitment & Data Collection
Determining research goal and study design
The research project aims to scaffold the natural behaviors that support scientific thinking and STEM learning in children through museum exhibit design and signage development. The study has two between- subjects condition: baseline condition without signage and a signage condition with signage featuring goal-related information and process prompts.
In collaboration with Process Curiosity, the research team prototyped a suite of signage tools for the Build Landscape exhibit at the Thinkery museum, which supports hands-on exploration of pulleys and simple machines. The signage included:
-
A legend of parts - illustrated images of each part labeled with their names and short descriptions of their affordances designed to encourage dialogue and experimentation of materials
-
Design challenges - illustrated pictures of simple machines with visual prompts and goal statements to spark explanation and collaborative exploration
-
part labels for storage - illustrated images of each part labeled with their names and short descriptions of their affordances placed on storage containers in the exhibit designed to promote dialogue, sorting, and experimentation of materials
-
Connect cards designed to guide caregivers in prompting scientific thinking during play
Recruiting participants, data collection and research protocol
The data collection stage is done before I joined the team. The research team recruited parent-child dyads at the Thinkery Museum using a quota sampling method. Caregiver-child dyads were randomly assigned to one of two conditions. Upon receiving consent, participants were instructed to play naturally in the exhibit while equipped with a wearable eye tracker and microphone. Video cameras were positioned around the perimeter of the exhibit to capture the interaction. This setup allowed us to collect rich multimodal data to analyze how children explore, explain, learn science concepts and how parents and children collaborate and engage with exhibit during hands-on museum experiences. Although I was not involved in this initial stage, I familiarized myself with the research goals, procedures, and dataset structure in order to align my work with the team’s objectives and contribute effectively to later phases.
2. Data Management & Integration
Generating and syncing eye-tracking and raw video files
To standardize the workflow and support the research assistants in executing each steps, I created detailed step-by-step tutorial with visuals to guide of processing of raw data. This stage began with offloading raw data from the Pupil Labs eye-tracking glasses software and four exhibit-facing cameras to a shared Box folder for further data management. Using the Pupil Player application, we then added eye-tracking visualization (eye overlay, crosshair and scene view) to the raw eye-tracking data.
To create a unified video for each participants, all the exhibit camera videos and eye-tracking videos from the software were cleaned and synced together using Adobe Premiere. I also implemented a custom script to batch combine and export the final synced video for each participant to streamline the process and reduce manual workload.
Organizing and Managing Survey Data and Project Workflow
Alongside video processing, I managed and cleaned the survey data. I examined, cleaned and consolidated multiple datasets and merged them into a single master survey file. To facilitate efficient collaboration, I also developed tracking sheets and checklists to monitor progress and guide research assistants through each tasks.
3. Coding Rubric Development & Iteration
Developing and refining coding rubric
While research assistants worked on video syncing process, I focused on developing a comprehensive behavioral coding rubric to capture the nuances of parent-child interaction and scientific thinking behaviors. I started by watching a diverse sample of video data to identify behaviors of interest - such as exploration, explanation and parent scaffolding. I also brainstormed coding categories within the research team.
From the observation, I designed a structured coding rubric by:
-
Planning coding passes and structuring the dataset to capture both parent and child behaviors
-
Creating a Datavyu coding spreadsheet template and drafting a coding manual with precise code definitions
-
Testing the coding rubric on sample videos across different signage conditions to ensure coverage and clarity
We began with pilot coding on a subset of videos to validate the coding rubric. As expected, initial criteria sometimes failed or new behaviors emerged, leading to multiple rounds of refinement.
4. Behavioral Coding & Reliability Testing
Behavioral Coding
After finalizing the coding rubric, I led the behavioral coding phase, coordinating research assistants to systematically code the data in Datavyu. To ensure a smooth and consistent process, I prepared comprehensive coding instructions and supporting materials, including:
An AI-assited transcription guide using Whisper AI to speed up transcription while maintaining accuracy through manual verification
A finalized coding rubric and FAQ document to clarify definitions and resolve frequently asked questions by coders
A coding tracker for assigning videos and monitoring coding progress
Coding process involved two phases:
-
Primary coding: The assigned coders completed full coding pass for each video and documented their progress in a coding tracker, making files as ready for reliability testing.
-
Reliability coding: Approximately 25% of all videos were then reliability coded by a second coder.
Reliability Testing
To monitor coding consistency, we will be implement an Inter-rater reliability (IRR) process. Using a customized reliability script, developed based on our data structure, we compared primary and reliability coders' code to identify disagreements in coded behaviors or timing. After discussion of each disagreement, coding errors will be corrected, ensuring final analyses were based on accurate data.
5. Data Analysis & Reporting
Preliminary Data Analysis
With primary coding completed for a subset of videos, I conducted preliminary data analysis to examine how exhibit signage influences engagement, parent-child collaboration and science learning behaviors. I exported the data from Datavyu spreadsheet and loaded them into R for further data processing. The data was merged, cleaned and analyzed. Findings were also visualized using graphs.
The preliminary findings were also submitted and accepted to a IEEE ICDL 2025 conference.
Future Data Analysis
Once the full dataset is coded and inter-rater reliability is confirmed, we will conduct a comprehensive analysis to further investigate the research question.
6. Future Work
Next step
Given the richness of the dataset, we have planned multiple research questions that will be addressed in future analyses. Each research question will leverage a different subset of the coding rubric, allowing us to explore additional dimensions of parent-child interaction and science learning.
After completing analyses for the current research focus, we will proceed with coding another layer of behaviors using another subset of coding rubric tailored for these questions.
Reflections and Takeaway
Leading this project allowed me to developed strong management and leadership skills while navigating the challenges of a large, evolving research team. I have learned some lessons such as:
-
Team coordination: I managed task assignments, provided training materials and maintained consistent progress.
-
Process automation: I automated data processing workflows using Ruby and shell scripts, reducing data processing time by 50%.
-
Collaboration: I managed collaboration with internal technical teams, research labs, and external partners.
-
Funding impact: I was also involved in preparing a $2M grant application for an AI-based educational technology project.