Coding With the End in Mind: Quantifying Data Done Right

Quantifying qualitative data can be tempting. When we code or tag our data in Qualitative Data Analysis the computer keeps counting. With little effort we can produce overwhelming tables and great visualisations. However, we need to think of quantification before we start coding and make proper decisions about our analytical units in order not to spoil good qualitative research with inappropriate quantifications. A good strategy here is to make the coded segment reflect text immanent structures or to get advice on our analysis plan before we invest many days into coding.

Qualitative and quantitative representations of data

Sometimes it seems a deep trench is separating qualitative and quantitative methods of analysis. Yet not only in a Mixed Methods approach the boundaries become blurry. In fact, (as Kuckartz 2014 points out) also qualitative research is full of quasi-quantifications. When we say that something appears “often”, “from time to time”, “rarely” or “almost never” in a text, we could in principle assign absolute or relative values to that.

Still, qualitative research is more than just quantitative research with ambiguous counting. Especially when evaluating the importance or centrality of an argument qualitative judgments are needed. Of course we can use weights or scales to measure the importance or level of agreement in a statement, yet still this will be an approximation that cannot be translated into a quantitative value easily.

On the contrary, qualification of quantitative data plays an important role in research, although it is less often discussed. Let’s have a look at the following table. It is a quantitative representation of our data:

If we now continue to qualify one of the cases (H17) here, it could look like the following:

You see here, that the description is full of quasi quantifications. Yet still it puts the values into perspectives and already gives an idea of what might be important and how the values can be interpreted in their context.

What are we counting? The importance of analytical units

In day to day research, quantitative perspectives are commonly considered in qualitative research. In Qualitative Data Analysis the software counts all segments while we code our data. It can be tempting to produce tables, charts, diagrams and other quantified results because they are only a single click away. However, we must consider if this is indeed appropriate, and at best we should do so not after we have coded our material but right at the beginning of our research. Tables and diagrams look great in presentations, sure, yet it is very easy to create invalid results and to spoil good qualitative research with inappropriate quantifications.

When we start coding or tagging segments of text, we often start with rather intuitive units that make sense to us even when separated from their context. We look for statements that are good, clear and relevant. However, we often do not care so much whether we are coding a word, a sentence or even an entire paragraph here. Yet if we want to quantify our results later, this is crucially important. If we do not define clear analytical units we will not be able to tell in the end what our numbers mean, because they do not resemble a consistent amount of text.

Furthermore, we have to ask ourselves if we are indeed looking at all the data. If, for example, we just transcribed 20 minutes of a 1,5 hour interview and compare this with a segment from another interview we will hardly be able to say anything about numbers. When we pre selected data from a larger file and are just comparing sub segments, statements about relative or even absolute frequency become almost meaningless. This can be fine if we are just interested in qualitative statements about a specific topic but as soon as we want to quantify and compare numbers we are running into problems here.

Quantifying data done right

It’s highly important to think about what we intend to do with our data before we start coding seriously. If we do not intend to do any quantifications we have a much easier time with analytical units. We can just code the statements that make sense to us. We have to worry less if we look only at a part of the whole or if what we describe is representative for a larger group. Yet when we decide to do this, we have to stick to it.

Instead, if we want to quantify results we have to think clearly about what we code. We have to define our codes more rigorously. We must decide, how we want to analyse the numbers and visualize the data later and what valid statements we can make. When someone asks us: “What does this number mean?” we must be fully prepared to explain in detail what it reflects. I found it quite useful here to take the pre-existing structure of data into account and reflect that in my (quantitative) coding. Are we coding texts of law? Then we can count paragraphs. Do we have ten open ended questions? Then we can count answers. Do we have Twitter, Youtube or Facebook comments? Let’s assign one code to each (entire) comment!

An example of analytical units following the internal structure of documents. In these texts of law our quantifications reflect the number of paragraphs that were dealing with a topic.

Conclusion

How we code makes a serious difference for our analysis. Therefore, we are usually well advised (in accordance with Mayring 2015) to make an analysis plan even before we start coding. Bellow you find an example for a simplified analysis plan that I am using for one of my research projects:

Creating such a plan before the coding begins can be challenging, of course, especially when we are just learning a new tool or even method. Yet if we do not do this, we risk to produce invalid research or might need to start all over again.

Therefore, I think it is crucial, when learning a new method and tool to have someone with more experience take a quick look over the research and analysis plan, discuss the codes, the analytical units and the intended analysis before(!) we start coding. Because when we have done our coding, it might be already too late to change much and we can just hope to do it better next time.

You can read more about analytical units here: Sampling Unit, Coding Unit, Context Unit? Analytical Units in QDA

And more about how to define codes for quantification here: Scope & Quantity: Defining Codes for Proper Quantification


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.