Prompting Analysis: Comparing AI and Human Thematic Coding in Transana

Introduction

When you work with transcripts, interviews, or debates, the amount of text can feel overwhelming. Thematic analysis is the tool that helps researchers make sense of it all.

• It finds patterns in what people are saying so we can see the big picture.
• It helps us move from raw words to clear insights about ideas, values, and concerns.
• It makes research transparent, because themes can be linked back to the exact moment in the video or transcript.
• It turns conversations into knowledge for researchers that can inform theory, practice, or decision-making.

By combining AI and Transana:

• AI speeds things up by suggesting draft themes.
• Transana provides structure — time codes, collections, transcripts — that makes those themes useful and verifiable.
• The researcher stays in charge of interpretation, ensuring the analysis is rigorous and meaningful.

In short, thematic analysis is how we move from detailed transcripts to insights that matter. And with AI + Transana, that process becomes faster, clearer, and more powerful.

While themes are important, they are not the only way to analyze qualitative data as we all know. Depending on the research question, a study might focus on:

• Turns of talk in conversation analysis (who speaks, when, and how).
• Interactional moves in classroom discussions or debates.
• Didactic phases in teaching and learning studies.
• Discursive strategies in political speech or media discourse.

Each unit of analysis offers a different lens for understanding the data.

For this blog, we are focusing on thematic analysis to show how AI and Transana can work together. The same workflow (AI draft → researcher validation → time-coded transcript) can extend to these other approaches as well.

Comparing Prompts for Theme Creation

To test the strength of the ChatGPT integration and if the result will stay the same when we manage to change the prompt for a same analysis, we applied two different prompts to the same presidential debate transcript. The first prompt (Prompt 1) is a minimalist prompt, focusing only on timestamps and theme titles.

Prompt 1 – Simplified Theme Extraction

Prompt 2 is more related to the content of the transcript and structrured in a way that will allow the search to be more specific, so we have added the definition of a theme. It frames the AI as a researcher of presidential debates, instructs it to look for “big knowledge ideas”, to pay attention to interactional dynamics (candidates + audience), and to use verbal transitions to decide boundaries. The output is shaped to be conceptually richer and methodologically closer to what a researcher would do.

Prompt 2 – Presidential Debate Researcher Framing

Running both prompts on the same transcript shows whether AI can produce consistent thematic structures across different levels of instruction.

• If both yield similar themes, that demonstrates robustness — the AI is reliably capturing the main structure of the data.
• If Prompt 2 provides more nuance, that shows how prompt design can deepen analysis, while still aligning with the simpler result.
• Together, the comparison validates that AI outputs are not random: they can be both stable (across prompts) and adaptable (to researcher framing).

Interpretation of results

This following map (Figure 1) compares thematic coding from Prompt 1 (pink), Prompt 2 (blue), and the human analysis (turquoise). Each block represents a clip extracted from the transcript of the prompt result and the primary transcript made by human analysis. They are all coded with a thematic label. This visualization shows how AI-generated themes align with human-coded themes when applied to the same episode, highlighting both convergence and variation across analyses.

What this visualization makes clear is that theme segmentation can vary in depth without altering the underlying structure of the data.

This means that while AI (and human) analysis may vary in how finely themes are divided, the structural borders of the discussion remain stable. In other words, the boundaries of thematic units are consistent, even when the themes themselves are grouped or split differently.

The implication for researchers is important: AI can provide both a high-level overview (broad themes) and a granular map (detailed themes) of the same transcript. Both perspectives are valid, and the choice depends on the researcher’s analytic goals. In short, this shows that AI can adapt to different levels of thematic detail without losing coherence with human analysis.

Comparing labels produced from Prompt 1, Prompt 2, and a human analysis result

Please note that the AI prompts characterize themes of the question and candidate responses, while the human analysis attempts only to characterize the question asked.

Table 1. Comparison of Thematic Results Across Prompts and Human Analysis.
Column 1 shows the results of Prompt 1, Column 2 shows the results of Prompt 2, and Column 3 shows the human analysis. Each row corresponds to a common theme identified across all three approaches.

1.  Overall Similarity

• Both prompts produce chronological thematic structures that cover the same major phases of the debate (Table 1):
  • Financial crisis and bailout
  • Fiscal responsibility and entitlements
  • Energy and climate
  • Health care
  • Foreign policy (Iraq, Pakistan, Afghanistan, Russia, Iran/Israel)
  • Closing reflections
• This consistency shows the AI is robust across prompt styles: it identifies the same main themes even when instructions differ.

2.  Differences

• Prompt 1 (Simple):
  • Gives longer, descriptive labels with multiple sub-topics in a single theme (e.g., “Climate change and energy policy: nuclear, alternatives, and drilling”).
  • Reads more like a summary of topics, sometimes blending several ideas into one. Example: “National security doctrine: Iraq, use of force, and humanitarian intervention”.
• Prompt 2 (Detailed):
  • Produces shorter, sharper theme titles (e.g., “Health Care: Commodity or Right”).
  • More analytically framed — closer to how a researcher might label codes or themes.
  • Breaks complex topics into smaller, sequential chunks (e.g., Pakistan and Afghanistan are separated into two themes, whereas Prompt 1 merges them).
• Both prompts reproduce the core debate structure recognized by human analysis. Prompt 1 shows that even minimal instructions produce reliable outlines. Prompt 2 demonstrates that careful prompt design present results closer in style and precision to human coding,

Ethical Use of Data and AI

All data used in this work are publicly available and were handled in accordance with ethical research principles. When working with any kind of research data—especially video, audio, or transcripts—it is essential to ensure that your practices align with your Institutional Review Board (IRB) or equivalent ethical oversight policies.

Researchers are strongly encouraged to review their institution’s guidelines on AI-assisted analysis and confirm that all data processing complies with approved protocols, data-sharing agreements, and participant consent terms. Responsible use of AI within Transana means maintaining respect for privacy, confidentiality, and intellectual integrity at every stage of analysis.

Author Profile

Zeynab Badreddine, Ph.D. is the Founder & CEO of Advanced Video-Based Research LLC, a consultancy dedicated to qualitative video data analysis. She is an official Transana trainer, and a specialist in helping researchers use video, transcripts, and digital tools to produce rigorous and innovative analysis. She works closely with the Transana developer to expand the tool’s global reach.

With a background in science education, management, and computer science, Zeynab bridges research and practice by combining deep academic expertise with hands-on experience in technology-enhanced analysis. Her work focuses on making complex video-based research methods accessible, transparent, and impactful for scholars and practitioners around the world.

Connect with Zeynab on Linkedin  or explore her work at Advanced Video Based Research