Book review: MERLO for deep learning of meaning-equivalence concepts
By Shalin Hai-Jew, Kansas State University
Pedagogy for Conceptual Thinking and Meaning Equivalence: Emerging Research and Opportunities
Masha Etkind and Uri Shafrir
IGI Global
2020 154 pp.
Is learning about a field or a discipline mostly or partially about understanding meanings (of terms, of diagrams, of data, and other representations)? Is it important to think analogically, to understand which objects and phenomena are analogues? Is it vital to understand schemas and ways that various elements of a field are structured and interrelated? Is it crucial to understand nuanced differences and similarities among various aspects of a discipline? Is it important to be able to reason through a design and integrate signals and practices and influences from history in an articulate way?
Masha Etkind and Uri Shafrir might argue that all the priors are important for learners who aspire to be expert practitioners in Pedagogy for Conceptual Thinking and Meaning Equivalence: Emerging Research and Opportunities (2020). In this work, they and colleagues introduce a method for mapping curricula based on concepts and actual meaning equivalence (even in a context with distractors and complexity). They share various multiple-choice questions built off of their MERLO (meaning equivalence reusable learning objects) model that read like old-school IQ tests back in the day on one hand and rigorous and systematized computational thinking on another. The co-editors make a coherent case that such capabilities of discernment enable more coherent (and maybe conservative as in informed by the past) innovations.
Achieving Higher Order of Conceptual Thinking
Masha Etkind’s “High Order of Conceptual Thinking: Find the Equivalence of Meaning” (Ch. 1) focuses on so-called “multi-semiotic problems” or those represented by multiple signs and symbols given the multidimensionality of the world. Being able to engage such issues may enhance the “cognitive control of inter-hemispheric attentional processing in the lateral brain, and increase higher-order thinking” and improve design ideas with an enriched creative process (Etkind, 2020, p. 1). As such, this work introduces “a novel pedagogy for conceptual thinking and peer cooperation with meaning equivalence reusable learning objects (MERLO) that enhances higher order thinking” (p. 1). The ability to work ideas through various levels of abstraction enhances mental agility and flexibility. The ability to wield such skills benefits group work. MERLO has its roots in from-life teaching and learning examples in architecture, “the emergence of concept science, development of digital information, research in neuroscience and brain imaging” (p. 1).
MERLO is not only a pedagogical approach and a model, but it is expressed in a multi-dimensional database. Here, concepts are expressed as “multi-semiotic representations in multiple sign systems, including: exemplary target statements of particular conceptual situations” (Etkind, 2020, p. 2). The author explains:
Each node of MERLO database is an item family that includes five statements: one Target Statement (TS) that describes a conceptual situation and encodes different features of an important concept; and 4 other statements that are sorted by two sorting criteria: shared equivalence-of-meaning with TS (and) shared surface similarity with TS (Etkind, 2020, p. 2).
The items in a database, while likely painstaking to create across multimodal representations, can be accessed and used for various types of teaching and learning. These can serve as a store of knowledge into practical perpetuity. Various statements may be thematically sorted; item families may be identified around “meaning equivalence” (core importance) and distractors may be identified based on “surface characteristics”. This chapter contains a four-quadrant table on two dimensions: surface similarity (SS) and meaning equivalence (ME). Based on the focal target statement:
Q1 contains statements similar in appearance to TS (target statement) and shares equivalence-of-meaning with it.Q2 contains statements that are not similar in appearance to TS, but do share equivalence-of-meaning with it.Q3 contains statements similar in appearance to TS, but that do not sure equivalence-of-meaning with it.Q4 contains statements that, although thematically relevant to TS, are not similar in appearance to TS and do not share equivalence-of-meaning with it (Etkind, 2020, pp. 3-4).
An adapted and annotated derived table is shown in Figure 1, with a main focus on preserving the original meaning but with highlighted quadrants of relevance for understanding true meanings.
Figure 1: Template for constructing an item-family in MERLO (Etkind, 2020, p. 3, redrawn with some emphases)
There are social aspects to shared meaning making. Etkind writes: “…if a statement contains text in natural language, then by ‘surface similarity’ we mean same/similar words appearing in the same/similar order as in the TS (target sentence); and by ‘meaning equivalence’ we mean that in a community that shares a sublanguage…with a controlled vocabulary, a majority would likely agree that the meaning of the statement being sorted is equivalent to the meaning of TS” (Etkind, 2020, p. 3). The focal Target Statement can be in various semiotic or symbolic systems (text, image, map, others). Academia, for many years, has focused on textual sign systems (based on language, based on math, based on physics), but of late, there has been more focus on diagrams and image sign systems. Clearly, mulling over surface similarity vs. meaning equivalence helps clarify what a concept is, what it isn’t, and more of the underlying nuances. The mapping of a discipline is possible, such as using MERLO to map functions in math [f(x) = ].
The MERLO approach may be applied to “the full content of a course” or content area in a discipline. In a learner assessment approach, “MERLO pedagogy guides sequential teaching/learning episodes in a course by focusing learners’ attention on meaning. The format of a MERLO assessment item allows the instructor to assess deep comprehension of conceptual content by eliciting response that signal learners’ ability to recognize, and to produce, multiple representations, in multiple sign-systems—namely, multi-semiotic—that share equivalence-of-meaning” (Etkind, 2020, p. 4).
A MERLO assessment has five unmarked unlabeled statements: an “unmarked TS (target statement); plus four additional (unmarked) statements from quadrants Q2; Q3; and Q4” (p. 4) and having individuals identify those elements (two or more) with equivalence of meaning. The question does not include Q1 contents (with yes on “surface similarity” and yes on “meaning equivalence”) because “it gives away the shared meaning due to the valence-match between surface similarity and meaning equivalence—a strong indicator of shared meaning between a Q1 and TS” (Target Statement) (Etkind, 2020, p. 5). The task is to not confuse surface similarity with actual meaning (as people do in navigating a world full of distractors, to find the real vs. the fake). One example shows representations in a second-year history of architecture course including five different sign-systems: urban plan, photo, orthogonal drawing, language, and 3D sketch (Etkind, 2020, p. 5). The testing of comprehension is not only from recognition assessed in the MERLO prompt…but also production (less scaffolded) such as writing an answer.
Masha Etkind (2020) shares some powerful insights about how this might apply in an architectural context:
To recognize guiding principles of good design one must understand architectural precedent and should be able to decompose the main design idea of the precedent into numerous sets of meaningful concepts. How does one recognize the conceptual meaning of form—the elements of totality of formal expression, the distinction of the composition of parts from a whole, of the material construct, of the tangible function and intangible narrative, of the path through time and space? Pedagogy based on recognition of Meaningful Equivalence is a helpful tool that may lead the way. Pedagogy for conceptual thinking focuses learners’ attention on meaning and enhances their understanding of the difference between formal representations that ‘look alike’ but do not share the equivalence-of-meaning and those examples that share same essential characteristics without apparent similarity. To recognize meaning in architecture one needs to search for evocative concepts that constitute comprehensive architectural statement. While the final outcome of architecture is the one that constitutes physical reality of built form, a material construct that integrates ideas of the time, and represents level of available technology—the interpretation of form that conveys meaningful relations associated with place and time is an intellectual task of decoding the abstract ideas and clarifying intangible connections between the concepts. (p. 6)
In another sense, this type of learning is about bringing the mental modeling of non-experts in closer alignment with the conceptual modeling and higher precision of experts.
An example of a MERLO quiz, from the architectural angle, becomes overly complex quickly given the multi-dimensionality of the concepts and the necessity of domain knowledge to understand the example given (Etkind, 2020, p. 10).
In other applications, MERLO objects may be used to map core and peripheral concepts of a discipline and their interrelationships with each other. In a concept map, the closer a concept is to another indicates similarity of meaning. In this informational hierarchy, the “partial meaningful roles” of ideas help define the field, piece by piece (Etkind, 2020, p. 12). Etkind (2020) writes: “Renaissance is one striking example in history when ideas and concepts from different areas of knowledge and art overlapped and impacted on each other” (p. 12).
From this work, it seems that creating MERLOs require in-depth immersion in a field and a subject for a long time and training in logical reasoning and the ability to engage complexity. (I wonder how many can reason their way through complex MERLOs in their own fields.)
This work acknowledges the challenges for learners: “It will require learners to exercise a significantly higher level of conceptual comprehension to recognize meaning equivalence shared not only across different forms of representation, but across different areas and forms of knowledge” (Etkind, 2020, p. 13). The discernment has to extend far, so learners do not get fooled by the lack of surface similarity between two expressions that are actually similar in meaning.
MERLO helps differentiate “good conceptual thinkers” from “poor” ones, with a scoring metric based on z scores (how far from the mean the learner lands and in what direction) (Etkind, 2020, p. 15). Importantly, it is possible to train conceptual thinking and “attentional processing of multi-semiotic MERLO items,” based on research with students in higher education (Etkind, 2020, p. 17).
Alternate conceptual thinkers. Some who engage with MERLO objects also may form their own “alternative descriptions of the equivalence-of-meaning” including “conceptual aspects that were correct, but not within the expected and obvious context of these assessments items” (Etkind, Kenett, & Shafrir, 2010, as cited in Etkind, 2020, p. 18). The research has also surfaced an Intriguing finding that the “alternative conceptual thinkers scored significantly higher than even good conceptual thinkers did on MERLO production scores, as well as on the essay question” (Etkind, 2020, p. 18).
MERLO in the Early Digital Era with Various Enablements and Constraints
Uri Shafrir’s “Meaning Equivalence Reusable Learning Objects (MERLO) Access to Knowledge in Early Digital Era and Development of Pedagogy for Conceptual Thinking” (Ch. 2) provides a sense of empirical grounding for MERLO. This method was applied to three contexts in the early 2000s and seen to offer proof-of-concept. One application was to a risk management course in business management (in Canada), another in secondary school courses in math, physics and chemistry (in a high school in Russia), and an implementation of MERLO pedagogy and databases for learners in grades 9 – 12 in math courses (in a learning center in Canada).
The MERLO method emerges from practices in the informational sciences and is part of “concept science,” described as “a novel generic methodology for parsing and analyzing concepts, applicable to the various knowledge domains and professions; with tools designed for recognizing, representing, organizing, exploring, communicating, and manipulating knowledge encoded in controlled vocabularies of sublanguages” (Cabre, 1998; Kittredge, 1983, as cited in Shafrir, 2020, p. 23). Concept science “documents the evolution of content and structure of concepts, and categorization, knowledge representation and use” (Shafrir, 2020, p. 23). MERLO is design to teach for deep comprehension, which preempts the need for teaching to any particular test, since the knowledge should carry to various contexts. The author writes: “At the core of MERLO is a battery of reusable on-line databases that encode the conceptual content of the instructional material. Each database is for a specific discipline and a specific content, and is based on a detailed concept mapping of the specific content area…” (Shafrir, 2020, p. 26).
This chapter describes the respective setup, data, and research findings from the three different contexts; the work affirms the usefulness of the MERLO approach, with clear observed improvement of learning performance / test outcomes. Each of the unique research projects has its own nuances. There are insights on how MERLO can be integrated into various learning and training contexts and then assessed for learning efficacy. Some evocative quotes from learners have been included for color. Some screenshots provide insight into how some MERLO self-tests can work. So MERLO is portrayed as a model, a (multi)method, an expertise, and a technology here, with adaptability in implementation.
Interactive Concept Discovery
Being able to explore how concepts are interrelated within a particular context can enhance the acquisition of relevant meanings. Masha Etkind and Uri Shafrir’s “Enhancing Conceptual Thinking with Interactive Concept Discovery” (INCOD)” (Ch. 3) describes the application of “concept parsing algorithms” (CPAs) to enhance this exploration of concepts, with computational text analysis (including “concordance, collocation, co-occurrence, word frequency” and other approaches in a course knowledge repository (p. 54). The coauthors explain the importance of vocabularies in disciplines:
Code words in scholarly discourse are lexical labels of concepts in a controlled vocabulary that encode conceptual content within the body of knowledge in a discipline, a profession, a domain. A lexical label acts as proper name of a regularity, an organizing principle behind a collection of facts in context. Lexical label is often one or more common words (mostly nouns and noun phrases) used to label a recognized pattern in human experience and to communicate a well-defined meaning. (Etkind & Shafrir, 2020, p. 55)
The words are not defined directly in dictionaries. In these contexts, these lexical labels “do not encode the literal meanings associated with their constituent words in the common use of language” and “cannot be replaced by synonyms. Each label functions as a proper name of the signified concept” (Etkind & Shafrir, 2020, p. 55). The coauthors share some data visualizations of text analyses based on various seeding terms to highlight some exploration capabilities of the technologies.
Secondary School Teachers and the Design of MERLO Items
Ornella Robutti, Paola Carante, Theodosia Prodromou, and Ron S. Kenett’s “Teachers Involved in Designing MERLO Items” (Ch. 4) focuses on two cases (in Italy and Australia) of mathematics teachers designing MERLO items as part of their professional development program. This work explores how MERLO can be used in “different cultural and institutional ecosystems” (p. 61).
The research team applied the Meta-Didactical Transposition framework (Arzarello et al., 2014, as cited in Robutti, Carante, Prodromou, & Kenett, 2020, p. 62) to understand teacher practices in this work. This framework involves the study of “task, technique, and justifying discourses” to understand teacher praxeology (p. 63). Teacher practices evolve differently based on their respective “history and experience” and “institutional environment” (p. 64). Within the structure of MERLO, with the setup of statements (per Figure 1), the respective teachers approach the work in different ways. The team studied the “MERLO items designed by the teachers, video recordings of the teacher face-to-face meetings, texts by the platform (uploaded files and interventions in the forum), namely the content related to the productions of items and the explanations and justifications of the steps they followed to design them” (p. 67).
One innovation was the elicitation of an open-ended question to understand learners’ reasons for making a particular choice, which seems to enable deeper learning. There were in-depth analyses of how learners have to reason in different ways to solve different MERLO items. Besides the different strategies and tactics, this chapter affirms the capability of teachers to create and deploy MERLO items collaboratively and individually. They conclude: “These observations suggest that MERLO items can be used in different countries by adapting them to institutional contexts, while retaining their substantial structure” (Robutti, Carante, Prodromou, & Kenett, 2020, p. 83).
Evidence-Based Informed Consent with MERLO in a Clinical Setting
Myrtha Elvia Reyna Vargas, Wendy Lou, and Ron S. Kenett, in “Production of Evidence-Based Informed Consent (EBIC) with Meaning Equivalence Reusable Learning Objects (MERLO): An Application on the Clinical Setting” (Ch. 5), evaluate whether MERLO can be used to help people understand informed consent in their medical care. A common challenge in the healthcare context involves the overestimation of understanding by physicians:
This study measures level of understanding of informed consent for elective cesarean surgery using an evidence-based informed consent (EBIC) model based on six MERLO assessments. MERLO recognition and production scores and follow-up interviews of 50 patients and their partners were recorded. Statistical comparison of scores within couples was performed by weighted kappa agreement, t-tests, and Ward’s hierarchical clustering. Recognition score means were high for patients and partners with low standard deviation (SD), while production scores means were lower with higher SD. Clustering analysis showed that only 70% (35/50) of couples were assigned to the same cluster and t-test yields significant difference of scores within couple. Kappa yields moderate agreement levels on all items except for items D and C, which are lower. Follow-up interviews show that participants consider MERLO assessments to be helpful in improving comprehension. (Vargas, Lou, & Kenett, 2020, p. 86)
The research context here refers to elective caesarean section with its related knowledge domain. The potential patient and the couple need to know what they’re getting into, potential risks of any course of action, and other implications. Such potentially life-changing decisions should not be taken in an ill-informed or pro forma or non-thinking way. The focus is on six concepts related to c-section informed consent: “indications for caesarian delivery…preparation for the procedure…the surgery procedure…healing process…potential complications…potential complications in future pregnancies” (Vargas, Lou, & Kenett, 2020, p. 95). This research team explains:
MERLO assessments are a novel methodology for evaluating the essence of multi-dimensional, complex and conceptual situations. They are also known to enhance deep comprehension of the material reviewed. These assessments are designed to test not only the correct interpretation of the information or data provided during a teaching-learning process, but also the conceptual understanding of such information (Vargas, Lou, & Kenett, 2020, p. 91).
This work follows learning into non-classroom contexts. In this case, the study involved 100 participants (50 patients and 50 spouses / partners) and used questionnaires and interview forms, in a hospital, in Canada. The researchers applied various statistical techniques and machine learning to their collected data. The patient group had higher “mean and standard deviation on aggregated recognition scores” than for their partners (p. 103). Some of the six points are retained as potentially more salient: indications for caesarian delivery and the healing process. The lowest scores were on potential complications of future pregnancies, preparation for the procedure, and potential complications (of the procedure) (p. 104). This substantive research involves additional insights.
Surfacing Learner Misconceptions with Errorful Concept Maps
Figure 2. Seeable Complexity
A basic concept map (or “cmap”) illuminates conceptual relationships in a particular discipline or domain. Dealing with a complex concept map may be difficult for learners, given the many moving parts and complex definitions. To help make concept maps more accessible in a more easy-to-handle way, some researchers have created a knowledge assessment platform based on a propositional concept map with errors (like distractors). The learner assignment is to identify the errors (without visual cues) or complete missing information. Paulo Rogério Miranda Correia, Joana Aguiar, and Brian Moon’s “Using Concept Maps with Errors to Identify Misconceptions: The Role of Instructional Design to Create Large-Scale On-Line Solutions” (Ch. 6) describes their work in Sero, which was built to help learners move beyond their “safe semantic territory”! This work describes some of the work to “create Cmap-based tasks for large-scale purposes using technology” (p. 119). The research was based out of an undergraduate Natural Science Course at the University of São Paulo (Brazil) with the main give-class sequence studied as follows: evolutionism, molecular biology, ethical issues and scientific research, ethical issues and society, and assessment (with the “cmap with errors”) (p. 120). This work describes some of the relational statements and shows some physical (flowchart) maps of the focal knowledge; they provide examples of “easy” mistakes that learners make in their reasoning. This work also included the clustering of students based on similar performances, with low average performance (“scientific: 28%; non-scientific: 62%”) (p. 128). The researchers suggest creative approaches to diversifying cmap with errors, through varying task formats, error formats, “epistemological inputs,” concept selection, and timely feedback to enhance learning (p. 131).
Conclusion
Ideas take time to percolate and infuse through to practice. Masha Etkind and Uri Shafrir’s Pedagogy for Conceptual Thinking and Meaning Equivalence: Emerging Research and Opportunities gives the sense of an approach—MERLO—as practiced by a small group of those read into this method. The focus on deeper meanings (as represented in different sign systems and semiotics) for true learning is powerful.
This reads also almost like a database capability worked backwards to more practical applications. To me, it is not clear how user friendly (or how human friendly) this computational approach is to learners or to teachers. How built-out are the various MERLO systems for various learning disciplines? (I assume that there is plenty of gaps.) How are contested “truths” handled, and is this model and process only applicable for some baseline understandings? Also, it is unclear where teacher individuality and creativity come in if the authority is a computerized system of curated knowledge. How well can teachers and learners shake loose from informational formalisms and innovate wholly?
About the Author
Shalin Hai-Jew works as an instructional designer / researcher at Kansas State University. Her email is shalin@ksu.edu.
Thanks to IGI Global for a watermarked review copy of the text.
| Previous page on path | Cover, page 17 of 21 | Next page on path |
Discussion of "Book review: MERLO for deep learning of meaning-equivalence concepts"
Add your voice to this discussion.
Checking your signed in status ...