By Shalin Hai-Jew, Kansas State University 

The prior questions are typical ones that may be asked in a marketing and product development context.  Interestingly, some version of those questions may also be asked in an educational context.  For example:  

(The above example is not to suggest that teaching learners is the same as providing a commercial service to customers per se because this line of argument may be offensive to some.  However, in an educational context, it is helpful to consider learner needs and learner expectations.)  

The first set of bulleted questions are generic ones, and the second set is comprised of hypothetical ones that could be asked.  They are also questions that could be posed through conjoint analyses.  “Conjoint” is an adjective that suggests “combining all things involved,” and a “conjoint analysis” then combines all things involved from which people may select what they may prefer.  

In the online Qualtrics Research Suite, a new feature involves a built-in “conjoint analysis” in a basic educational site license (and access to a full suite of conjoint analysis tools at additional cost).  This analytical approach originated in business (particularly applied to marketing and product development, and others), but has moved into the social sciences (political science, dietetics, and others).  Qualtrics, Inc. has provided a page on “Conjoint Analysis Theory” that backs up their Choice Simulator.  

This short article introduces the basic step-wise “self-explicated conjoint analysis.”

“Conjoint analyses” essentially present various (available or potential) attributes and levels of particular products or services to better understand what consumer preferences are, particularly in defined environments (based on specific services or specific products).  A conjoint analysis may probe preferences throughout the related “space” by engaging a number of products and services…or it may hone in on particular products or services of specific interest (such as those that have received complaints). The idea is to better understand customer needs, so their interests may be addressed to attract a larger customer base.  

A first step is to conceptualize the problem or challenge.  Then, the particular “space” is defined as a combination of “parent”-products and / or services based on their “child”-attributes or levels.  

For example, in the context of an educational conference, the top-level services may include the following:  locale (and presentation spaces), keynote speakers, scheduled presentations, food, and sponsors.  

Under locale and presentation spaces, “locale” may include “name recognition of the destination, closeness to entertainment (dining, shopping), and parking,” and “presentation spaces” may include “technologies, wireless fidelity (“wifi”), hygiene, acoustics, and lighting.”  Then, under “keynote speakers,” there may be other attributes:  name recognition, expertise, charisma, professional insights, and so on.  Under “scheduled presentations,” there may be features like “pacing, diversity of topics, catchy titles, coherent descriptions, applicability to work,” and others.  Under “food,” there may be levels of preferences:  minimal, plentiful, and maximal…low-calorie or high-calorie.  Or there may be attributes like the following:  Italian-themed, Chinese-themed, Indian-themed, or others.  For “sponsors,” the attributes may be the following:  name recognition, level of sponsorship, display table attractiveness, geographic representation, topic representation, and others.  In other words, there are the coarse-grained services or products and then the more granular attributes and levels.  

Once these have been built and vetted, the next step is to build the conjoint analysis in Qualtrics.  Once a new “project” or “survey” has been started, go to the Tools button and select “Conjoint Analyses…” in the dropdown menu.  

 

Figure 1:  “Conjoint Analyses…” in Dropdown Menu 

Once that is selected, the Conjoint Analyses window will open and show all existing created conjoint analyses.  If none have been created yet or a new one is desired, click the “Add Another Conjoint” link.  The “Add Conjoint Analysis” wizard will open to the first window.  This first window provides an explanation of the Conjoint Analysis in Qualtrics.  

Figure 2:  Built-in Context-Sensitive Explanation of a Conjoint Analysis in “Add Conjoint Analysis”   

Click “Next.”  In the next screen, type the title for the Conjoint Analysis.  Below will be an auto-filled data export tag (which is used to label the conjoint-analysis related data).  Note that conjoint analyses may be stand-alone…or they may be interspersed in a full survey.   Click “Next.”  

 

Figure 3:  Naming the Conjoint Analysis   

The four-part section to define the Conjoint Analysis follows.  In the next slide are the four tabs:  Features and Attributes, Question Text – 1, Question Text – 2, and Conjoint Options.  To add a Feature, type in the text field and click “Add.”  

 

Figure 4:  Adding Features and Related Attributes / Levels 

To add the “child” sub-attributes under each “parent” feature, highlight the target feature in the left, click into the space to the right, and add the respective attributes and levels.  To differentiate one attribute / level from another, use the “Enter” or return key.  

In the Question Text – 1 and Question Text – 2, there is already default text in the spaces, and these may be changed.  (It helps to do a walk-through to see how the questions align with how you set up the other parts of your question later on.)  

Finally, in the “Conjoint Options” tab, you have a choice of selecting one of two conjoint display options:  (1) Feature Importance Constant Sum and (2) Desirability Upgrade Constant Sum.  The first explores the most desired and least desired features in a space. The second enables respondents to indicate their level of desirability with added-on features to a basic service.  

Click “Save.”  The Conjoint Analysis now exists as a research sequence that may be added to any survey (in the same way that uploaded images, question blocks, saved surveys, and such may be added to any other project).  

Now, it is important to place the Conjoint Analysis in an actual survey sequence.  

Placing the conjoint analysis is done in the “Survey Flow” area.  

 

Figure 5:  Adding a Conjoint Analysis to a Survey Flow 

The Conjoint Block is now part of the survey.  Its appearance is somewhat different from the other blocks in the particular survey.  This may be seen in Figure #, which also has the data export tag highlighted.  

 

Figure 6:  Conjoint Services and Attributes Example in Instructional Design  

In a survey with a backwards arrow in Qualtrics, it is possible to navigate backwards and forwards throughout the survey. However, if there is a conjoint analysis sequence within that survey, it will not be possible to back up to prior to the conjoint analysis because this analysis is a stepwise process.  It depends on prior responses to reconfigure the next steps of the analysis, so respondents will get different setups of the question depending on their earlier preferences.  

The survey should be published in the “Distributions” area, and that link should be used for actual testing (not the “Preview Survey” option).  

Technological functionalities:  

The survey and its conjoint analysis(ses) should be tested across a range of web browsers for functionality.  

This should also be tested for accessibility.  

The survey should be tested for clarity (with respondents).  

Data analytics:  

Finally, it is helpful to see how the conjoint analysis functions with data. Unfortunately, the “Generate Test Responses” does not work with conjoint analyses.  

Note that the conjoint analysis data is not shown in the Reports section.  It has to be exported from the survey platform as a data file (.csv, .tsv, .xml, or .spss).  The results are reported as percentages of respondents who selected a certain option within a particular context or constraints.  

By common practice, conjoint analyses are usually practical and real-world, with the assumption of rational acting and decision-making by consumers in a micro-economics context.  This approach assumes that the respondents understand the questions and are correctly indicating their actual preferences…and further that they will act on their preferences in a commercial situation.   The interchange between the provider of the service / product and the consumer is transactional and practical. This means that the options offered are not “blue sky” but are actionable (and likely used for decision-making).  

This short article presented a walkthrough of the self-explicated conjoint analysis model, but there are others that may be deployed with the add-on of a conjoint analysis suite from Qualtrics (which may include choice-based / discrete choice, adaptive choice-based, menu-based, and MaxDiff).  (Or, those with more sophisticated statistical analyses capabilities may be able to harness the rich question types in the Qualtrics Research Suite and collect their own data and compute the fractional factorial analyses themselves using IBM’s SPSS or another tool.)  

Those interested in doing a walk-through, “A Sample Conjoint Analysis Setup on Qualtrics” may be accessed here.  

Those who use Qualtrics come from various professional contexts—the private sector, the public sector, and elsewhere.  The conjoint analysis hails from the commercial sector, but it has a variety of applications in education and research.  

Shalin Hai-Jew works as an instructional designer at Kansas State University.  Her email is shalin@k-state.edu.  

She is editing a text titled "Qualitative and Quantitative Methods in Online Survey Design and Data Analytics." Chapter proposals are being elicited through the end of August 2018.