Reading several payment narratives can reveal recurring questions. The risk appears when a researcher turns their own shorthand into a judgment. A code such as ‘access difficulty reported’ describes a theme. A code such as ‘provider wrongdoing confirmed’ claims an investigation the researcher may not have performed. Build the coding vocabulary before drawing conclusions.
Define the unit of analysis
Decide whether you are coding a whole record, a passage or a distinct reported event. One record can mention several topics, so a theme count may exceed the number of records. That is not necessarily an error, but it must be explained. Otherwise readers may mistake theme totals for a count of unique people.
Use original fictional practice narratives while designing the method. For example: ‘The writer reports difficulty understanding which party should answer a payment question.’ A descriptive code might be ‘responsibility unclear.’ It should not name a cause, assign fault or claim that the provider’s instructions were legally inadequate.
Write inclusion and exclusion rules
Each code needs a short definition, one positive example and one near-miss. For ‘timing question,’ include a narrative asking when an event occurred or will occur. Exclude a passage that merely contains a date without raising a timing issue. The near-miss is important because it prevents the code from expanding to fit every record.
Add an uncertainty option. If the narrative does not support a clear classification, mark it for review instead of forcing a choice. A high agreement rate achieved by removing difficult records can misrepresent how well the codebook works. Preserve the number and nature of unresolved cases in the method note.
Compare interpretations before aggregating
Have two readers independently code a small permitted set, then discuss disagreements at the definition level. The goal is not to pressure one reader into agreement. Ask which phrase triggered each decision and whether the code definition needs revision. Recode earlier examples after a substantive change so the dataset uses one consistent rule.
No agreement statistic can prove that the underlying narratives are complete or true. It can only describe a part of the coding process under the chosen rules. Keep that limitation separate from the value of a clear method: readers can still understand how the thematic summary was produced.
Protect the boundary of the publication
For Fintwist research, do not gather private account files through an editorial contact address to make the coding exercise feel more authentic. Public-source research and individual dispute handling are different tasks. Respect source permissions, minimize quoted material and avoid publishing identifying details even when a narrative seems vivid.
A useful conclusion names the selected material and the themes reported within it. It does not claim to decide individual cases, estimate prevalence among all cardholders or prove causation. The research becomes stronger when the codebook makes those limits visible rather than allowing a dramatic category name to smuggle them away.
Continue the investigation
Sources and scope
- External official source: CFPB: how complaint data is shared — Public fields and separation from identifying information and supporting documents. Checked 2026-10-04.
- External official source: CFPB Open Tech: complaint field reference — Meaning of date, product, issue, company and response fields. Not a merits decision. Checked 2026-10-04.