Scope & Quantity: Defining Codes for Proper Quantification

If we want to quantify the results of our coding in QDA, we have to think about  two aspects first: scope and quantity. The scope forces us to consider what length our text segments should have (words, sentence, paragraph), whereas the quantity indicates what we are allowed to do with these segments later in analysis (numerical, boolean, non-numerical analysis). To quantify codes properly, this must be taken into consideration long before we complete our coding.

Quantification is a tempting way of analysing data in Qualitative Data Analysis. If we code text segments anyway, why would we not count and compare them? In fact, this is a way of analysis that can easily lead to invalid or inappropriate results. Here, we need the help of analytical units (as we have discussed here). They help us, to work in a consistent way that allows us to always keep in mind what our numbers actually mean, when we analyse them later.

My problem with analytical units as presented by Mayring (2015) and Kuckartz (2014) is, that they create the impression that the Coding Unit (the size of the segments we code) is fixed for the entire analysis and set for all codes equally. Instead, I argue, we have to adjust our analytical units to fit each of our codes. Let’s look at three examples first, to understand the problem in more detail.

Example of three different sizes of a coding unit. Do we code just one word, a group of words, a sentence or a paragraph?
Scope: How much do we code?

Assume, we have created a code “witches’ meeting”. This one we want to use to code all the paragraphs that describe in a full narrative the meeting of the witches. It’s purpose is to collect the different narratives, so that we can read them side by side under a comparative perspective. Here, the best coding unit would be the paragraph because we want to qualitatively compare narratives. Fine!

Next, we create a code “appearance of the devil” (as in the image above). With this code we want to track what the devil looks like when doing specific actions (flying, having sex, doing magic). Therefore, we are coding entire sentences, so that we not just capture his appearance but also the context in which it is situated. Therefore, our coding unit will be the sentence. Great!

Lastly, we create a code with the name of a person “Barbara Stadtschreiberin”. With this code we want to track where this person is mentioned and how often she is mentioned in each document. We use the code to capture her name or the words that are used to refer to her (e.g. “the wife of the scribe”). Here, we use the codes only to track where this person is mentioned and we are not really interested in the actual content of the text segments. Therefore our coding unit will just be limited to a few words. Excellent!

Now how should we set a general Coding Unit that servers the purpose of all three? This is hardly possible; and I don’t think any researcher would seriously insist that we do that. However, methodological literature often gives exactly that impression.

Therefore, my personal take on this problem is to define for each code it’s own Coding Unit. We will call this Coding Unit its “scope” here for clarity. In one case the scope of a code might just be a few words, in others we might need the context of a sentence and in a third case we even code the entire paragraph. Our system will be flexible, yet still we will try our best to be consistent within the usage of each individual code.

Quantity: What are we allowed to do with it?

How we set the scope, strongly depends on what we intend to do with our code later. How do we want to analyse it? Will we read the content of the coded segments under a comparative perspective? Will we differentiate it into more granular subcodes? Or will we quantify it right away? For this reason we will need to define what we intend to do with a code. This should be done at the moment when we start using it as a serious and elaborated category (read more on this here).

Therefore, we have to decide if quantifying that code would be an appropriate action, based on the way we defined and applied it. Does it make sense to say: This code appeared in this document seven times? Or is such a statement per definition invalid? In response to these questions we can differentiate between three different types of “quantities”:

Numerical, we can count this code and compare its absolute or relative number across different documents. (For example: Person 1 is mentioned 5 times in this document.)

Boolean, we are only allowed to differentiate whether a code exists in each document or not (0 or 1). (For example: In this document “living in the streets” is reported.)

Non-numerical, any statement referring to the quantity or existence of that code in a particular document does not make much sense. (For example: A marker for “In this document I found a relevant statement”.)

Conclusion

As we can see from this, whether we intend to quantify a code or not, heavily influences how strict we have to be in its application. When we want to quantify a code, we must be very strict with its scope (Coding Unit). We want a result where we can say: “In this document there are 5 sentences, that deal with the topic of this code.” If our scope varies within a code (once a sentence, then a paragraph, then a few words) such statements become invalid. If our interest is purely qualitative and we do not intend to quantify at all, it does not make such a big difference when our scope remains more flexible.

According to this system, a well defined code would look something like this:

A flying witch well defined. Within QDA-software memos or comments are often used for definitions.

As we can see, the code above does not only carry a definition, an example and coding rules (Mayring 2015, p. 44-49), but also the scope (word, sentence, paragraph etc.) and quantity (numerical, boolean, not-numerical) as we have illustrated them.

In conclusion, the Coding Unit does not have to be static or universal. Instead, it makes sense to think of the Coding Unit as something that is set for each individual code. To make our quantification more reliable we set a “scope” for each code and we consider its “quantity” (numerical, boolean, non-numerical). In my opinion these two considerations are often critical in telling good from bad and valid from invalid results.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.