Teaching qualiatative data analysis with QDAcity
How-To on using QDAcity for teaching qualitative analysis methods.
Overview
What you get
QDAcity makes it easier to teach qualitative data analysis (QDA) by offering teachers a dedicated course system which is easy to set up and share with your students. These exercises can be based on your own research projects or teaching examples you can set up. To enable you to teach QDA methods to an even larger audience we offer exercises which can automatically evaluate student's work against your own analysis using inter coder agreement. This enables a new level of scalability of practical QDA exercises.
What you need
Exercises in QDAcity are based on projects you have created, were invited to, or you have access to through a group. Therefore you need to have registered an account, have created a project, and added data to that project. If you want to create an exercise with an automated evaluation you also need to have coded the data in your project before creating an exercise based on it.
Learning objectives
QDAcity offers two types of exercises, with different learning objectives and ways of evaluating the student's work.
Codebook Utilization
This exercise type is designed to emulate the practice of inter coder agreement where a second coder re-codes parts of the data to evaluate whether the codes are all well-defined in the codebook, and the coded segments well reasoned and not due to some idiosynchracy of the original coder. Students receive an exercise project which contains all of the data of the project on which the exercise is based, but without the codings applied. They will all have the same documents to code and the same code system with the same codebook entries which define the constructs to be coded. This type of exercise allows for an automated evaluation of the results and hence allows the course to scale to a much larger audience.
Codebook Creation
This exercise type is designed to teach students to identify the relevant constructs found in the data and manifest them in their own code system and write codebook entries for them. Because each participant will (with a very high likelihood) end up with a unique solution to the analysis, this exercise type does not offer the ability to automatically evaluate the results. Instead, the teacher will be required to review the analysis artifacts of each student individually to grade and give feedback.
Courses
QDAcity offers a dedicated course management with courses and their instantiation within a given term (semester, trimester, etc). Courses can be created straight from the personal dashboard after you logged in. Click the button on the right and you will be asked to fill out the form shown in the following screenshot.
Students will be able to find your course by searching the course name in the course browser or the general search bar in the top of the app. These elements are highlighted in the following screenshot.
The course administration is available through the course configuration dashboard. As the creator of the course, you will see a cog icon (highlighted in the following screenshot) on the list element which you can click to navigate to the course dashboard. Here you can do the course administration.
On the course dashboard you can create or delete new instances (terms) of the course, and invite other users of your teaching team to be course administrators. In order to create exercises for any of the course instances you need to, again, click the green cog icon to configure the term (hightlighted in the following screenshot).
Exercises
New exercises for a given course instance can be created on the term dashboard. Just click the corresponding button highlighted in the following screenshot. As mentioned in the 'what you need'-section this button is inactive unless you have already created a project on which the exercise can be based.
In the form that appears, you can configure the type of exercise you want to run within your course. As explained above there currently exist two general types of exercises called "codebook creation" and "codebook utilization". Besides the type of exericse you need to select the project on which the exercise should be based as well as the revision of the project. You can also specify a deadline until the exercise should be completed. See the following screenshot.
Once you created an exercise you can click on the list item to get to the exercise dashboard (see screenshot below). Here you see all of the students who have started working on the exercise. By clicking on the green tag icon next to a student's name, you can see their project to give them feedback. If the exercise supports an automated form of evaluation you also see the an area for "exercise project reports".
Evaluation
For exercises supporting automated evaluation you are able to create a report based on inter coder agreement. In the screenshot below you can see that you are able to configure the agreement metric to only cover parts of the code system or a subset of the documents in the project. Or you can leave the default of evaluating all codes and all documents. You can also specify the type of agreement metric which you can chose between F-Measure, Krippendorff's Alpha and Fleiss' Kappa. For unit of analysis you have the choice between sentence and paragraph level. Selecting paragraph as evaluation unit leads to a more lenient evaluation. It is the default we would suggest, but different contexts may reqquire more fine granular distinctions in the coding.
We use cookies for a number of purposes, including analytics and performance, functionality and advertising. Learn more about QDAcity use of cookies.
Analytics:Performance:Functional: