A concept map is a diagram that depicts relationships between concepts. Concept maps are considered a valuable tool for learning from texts enhancing text comprehension and summarization. During a concept mapping assisted by texts, students may be asked to construct the concept map to reflect their understanding and measurement of the text comprehension from the given resources. The problem is that the grounding reading that supports the concept construct is not explicit but in the students’ heads. To make explicit the link between teachers-provided reading material and students’ concept map we propose an annotation-based concept map construction. Our premise is that rephrasing the concept map activity as an annotation activity can help this practice. Annotations provide a link between the reading material and the developed concept map so that teachers can gain a deeper insight into students’ misconceptions. We present Concept&Go, an innovative web browser extension that together with Hypothes.is and CmapTools makes possible the application of a Web annotation-based concept map construction.
What is the problem for which a DSR project aims to deliver a possible solution? Problems ought to be defined by means of problem articulations and characterized by positioning the problem in a problem space. Previous research has identified the context, described by the domain, the stakeholder, time and place, and goodness criteria.
During a concept mapping assisted by texts, students may be asked to construct the concept map to reflect their understanding and measurement of the text comprehension from the given resources. The problem is that the grounding reading that supports the concept construct is not explicit but in the students’ heads
What are the essential activities planned (or conducted) to make the intended contribution? When the intended contribution is design entities, the process includes building and evaluating activities. These activities specifically include anchoring the design through the execution of tasks like literature reviews and meta-analyses. It is recommended that the construction and assessment activities be planned and documented together in order to facilitate concurrent design and evaluation. These actions for theorizing on the design may also be included in the process described. Activities for theorizing can draw from a variety of research methodologies and strategies of inquiry, including qualitative and quantitative empirical research, whereas activities for processing the design can draw from DSR process models like the Peffers et al.
Design Science Research is the scientific study and creation of artefacts as they are developed and used by people with the goal of solving practical problems of general interest.
What is the solution to the problem that a DSR project is investigating? The solution description identifies the representation of the solution as a construct, a model, a method, an instantiation, or a design theory and expresses concisely the fundamental mechanisms of the solution and how the solution is situated in the solution space.
We developed Concept&Go, an innovative web browser extension that together with Hypothes.is and CmapTools makes possible the application of a Web annotation-based concept map construction
What previous knowledge will be used in the DSR design? Design knowledge can be either descriptive, explicatory, prophetic (omega knowledge) or prescriptive (delta knowledge). Three different types of input — kernel theories, design theories, and design entities — can be differentiated for high-level communication about DSR projects.
The practice of annotation has been shown to be an efficient method to facilitate the understanding and interpretation of the content that is being read (Kawase et al., 2009).
What are the core concepts that were utilized in the DSR project's research? The concepts used to describe the process and input and output knowledge, as well as the words used to characterize the study, such as the problem and solution space that the DSR project focuses on, must be described precisely. To ensure a rigorous execution of the evaluation activities, it is especially crucial to have a precise definition of the key concepts.
What kind of knowledge does the DSR project produce? Naturally, DSR projects generate design knowledge, which is referred to as omega knowledge. However, unlike the solution description, the design knowledge produced by the project puts the problem and solution spaces in relation to one another. If a DSR project's goal is to create design entities rather than design theory, the descriptions of such entities do not represent design knowledge because only the outcomes of the design entity's evaluation in context do. When the project is explained, these results are subsequently documented as output knowledge.
Our premise is that rephrasing the concept map activity as an annotation activity can help students' concept mapping
1. Positioning: What problem is addressed through which solution to what confidence?
Problem: State and characterize the problem.
Problem statement: How can concept mapping be supported for helping teachers to find the causes of the students’ misconceptions?
Context description: Teachers can provide several sources of knowledge related to the subject matter that can assist learners to achieve their learning aims. One of the ways to assist a student's concept mapping process is providing texts associated with the topic or focus question. The concept mapping task carried out by the students is followed by the evaluation and validation by the teacher. The latter allows teachers to note missing or irrelevant concepts, and trivial or incorrect linking phrases. It is not odd for teachers to wonder what is behind these misunderstandings. Yet, using the concept map as the unique resource to perform the validation allows the detection of students' misunderstandings, but not to trace their origin. Here, the problem is that the grounding reading that supports the concept construct is not explicit but in the students’ heads.
Goodness description: Usefulness of design principles that helps teachers find the causes of the students’ misconceptions supporting a concept map activity that can help teachers be trace for evaluation
Solution: Outline and characterize the solution.
Solution essence: we proposed a concept mapping activity based on an annotation practice.Our premise is that rephrasing the concept map activity as an annotation activity can help. Annotations provide a link between the reading material and the developed concept map so that teachers can have a deeper insight into students’ misconceptions.
Representation description: we developed a web annotation tool named Concept&Go, which is a web extension that supports a concept mapping activity based on Web annotations.
Process description: Design of the steps to conduct the concept mapping activity using web annotations (Annotation-based concept map construction) and instantiation of an artifact that allows performing each step (Concept&Go).
Evaluation: Describe evaluation activities and results.
We have not been able to carry out an evaluation of the project because we have not had the opportunity to evaluate it with real professionals in a real environment. We are looking for researchers who can help us evaluate the tool
2. Grounding: What knowledge is informing the design?
Search Process: Which search strategy was applied?
Search Results:
3. Aligning: How do the design activities contribute to creating the DK?
Design process documentation: What activities were conducted in which sequence?
Exploration and identification of the problem (when assessing Concept maps, finding the causes of the misconception is difficult). Identification of requirements (traceability), design (development of the solution space adjusting the practice of annotation to concept mapping and developing an IT artifact). Evaluation is needed
Design process rationale: Why were the activities conducted in this sequence?
First, as we did not have the necessary knowledge about the context (concept mapping), we needed to read and learn how it is performed in the practice. We perceived that web annotation could support the concept mapping activity when it is based on reading materials. We believe that concept maps can be enriched with linked resources that can complement them and allow teachers to have more information to assess and find students’ causes of misconceptions.
4. Advancing: How does the DK chunk provided advance?
Compared to existing DK. The annotation practice has been applied in different settings to solve problems of traceability. Different annotation practices can be found in Social Sciences and Humanities, Journalism investigation, or Medical and Biological Sciences.
We depart from web annotation tools that were used to support different practices.
Highlight&Go [1] is a web extension that supports the web annotation practice to support researchers doing Systematic Literature Reviews.
Mark&Go [2] is a web extension that supports the web annotation practice to support teachers to assess assignments based on rubrics.
Review&Go [3] is a web extension that supports the web annotation practice to support reviewers reviewing papers.
We conducted the projectability of the use of web annotations to adjust it to our own domain (i.e. concept mapping assignments) and stakeholders (teachers and students). In fact, the Concept&Go was developed by extending and reusing the source code of Review&Go.
Anticipated advancements of current DK.
Anticipated projectability: can this solution be linked to a different problem space?
As shown above, the annotation practice can be applied in multiple settings. Reusing the IT artifact, as we did previously with another one, can help to accelerate the research of the application of web annotation in other settings and contexts that can be supported with it. For example, if the users of the tool were teachers instead of students, teachers could use Concept&Go to prepare concept maps that group the learning materials by web annotations within concepts and relationships. In this case, the concept map can serve as a resources hub, instead of an element to be assessed.
Anticipated fitness: what changes can be made in the solution for this problem?
Concept&Go is a first software prototype that implements the solution. It would be interesting to assess the solution with experts in the area of concept maps. We have more knowledge about the technical part of the solution than about the context of the concept mapping tasks. Therefore, more experience with concept maps could help us to provide a better solution to support traceability, or improve the application of web annotations
Anticipated confidence: what evaluations can be conducted for current DK?
It is the most important advancement to consider. This project has not yet an evaluation performed in a real-setting. Therefore, we are looking forward to finding collaborators that can help us to evaluate it in a real setting.
5. Resourcing: which are the available resources?
About Problem: see attached pptx file
About Solution: Artifact in Chrome Store; Source code; User manual
About evaluation: ---
Support&Collaboration:
- xabier.garmendiad@ehu.eus
- haritz.medina@ehu.eus
REFERENCES
[1] Díaz, O., Medina, H., & Anfurrutia, F. I. (2019). Coding-Data Portability in Systematic Literature Reviews: a W3C's Open Annotation Approach. In Proceedings of the Evaluation and Assessment on Software Engineering (pp. 178-187).
[2] https://chrome.google.com/webstore/detail/markgo/kjedcndgienemldgjpjjnhjdhfoaocfa?hl=es
[3] Díaz, O., Contell, J. P., & Medina, H. (2019, June). Performant Peer Review for Design Science Manuscripts: A Pilot Study on Dedicated Highlighters. In International Conference on Design Science Research in Information Systems and Technology (pp. 61-75). Springer, Cham.
[4] Cañas, A. J., Coffey, J. W., Carnot, M. J., Feltovich, P., Hoffman, R. R., Feltovich, J., & Novak, J. D. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. Report to the Chief of Naval Education and Training, 1-108.
[5] https://www.cmapconference.org/
PROBLEM STATEMENT
When assessing Concept maps, finding the causes of the misconception is difficult.
- "In analyzing the factors that influence the effect of teaching, the teacher can determine a student’s knowledge structure and highlight misconceptions by inspecting the concept maps and logs." [3]
- Difficulties for detecting students' misconceptions [4,5]
- Not being able to adjust learning materials [4,6,7]
- Multi-perspective [1,3,8]
- No evidence of what is in the student's mind at creating the map [3]
- Students have difficulty in the concept mapping task [5,9]
[2] McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 36(4), 475-492.
[4] Cañas, A. J., Coffey, J. W., Carnot, M. J., Feltovich, P., Hoffman, R. R., Feltovich, J., & Novak, J. D. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. Report to the Chief of Naval Education and Training, 1-108.
[5] Atapattu, T., Falkner, K., & Falkner, N. (2017). A comprehensive text analysis of lecture slides to generate concept maps. Computers & Education, 115, 96-113.
[6] Cheng, S. Y., Lin, C. S., Chen, H. H., & Heh, J. S. (2005). Learning and diagnosis of individual and class conceptual perspectives: an intelligent systems approach using clustering techniques. Computers & Education, 44(3), 257-283.
[7] Brinda, T., Napierala, S., Tobinski, D., & Diethelm, I. (2019). Student strategies for categorizing IT-related terms. Education and Information Technologies, 24(3), 2095-2125.
[8] Novak, J. D., & Cañas, A. J. (2008). The theory underlying concept maps and how to construct and use them.
[9] Atapattu, T., Falkner, K., & Falkner, N. (2014, April). An evaluation methodology for concept maps mined from lecture notes: An educational perspective. In International Conference on Computer Supported Education (pp. 68-83). Springer, Cham.
PROBLEM STATEMENT
When assessing Concept maps, finding the causes of the misconception is difficult.
- "In analyzing the factors that influence the effect of teaching, the teacher can determine a student’s knowledge structure and highlight misconceptions by inspecting the concept maps and logs." [3]
- Difficulties for detecting students' misconceptions [4,5]
- Not being able to adjust learning materials [4,6,7]
- Multi-perspective [1,3,8]
- No evidence of what is in the student's mind at creating the map [3]
- Students have difficulty in the concept mapping task [5,9]
[2] McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 36(4), 475-492.
[4] Cañas, A. J., Coffey, J. W., Carnot, M. J., Feltovich, P., Hoffman, R. R., Feltovich, J., & Novak, J. D. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. Report to the Chief of Naval Education and Training, 1-108.
[5] Atapattu, T., Falkner, K., & Falkner, N. (2017). A comprehensive text analysis of lecture slides to generate concept maps. Computers & Education, 115, 96-113.
[6] Cheng, S. Y., Lin, C. S., Chen, H. H., & Heh, J. S. (2005). Learning and diagnosis of individual and class conceptual perspectives: an intelligent systems approach using clustering techniques. Computers & Education, 44(3), 257-283.
[7] Brinda, T., Napierala, S., Tobinski, D., & Diethelm, I. (2019). Student strategies for categorizing IT-related terms. Education and Information Technologies, 24(3), 2095-2125.
[8] Novak, J. D., & Cañas, A. J. (2008). The theory underlying concept maps and how to construct and use them.
[9] Atapattu, T., Falkner, K., & Falkner, N. (2014, April). An evaluation methodology for concept maps mined from lecture notes: An educational perspective. In International Conference on Computer Supported Education (pp. 68-83). Springer, Cham.
REQUIREMENTS
Annotations are commonly made highlighting text fragments during reading activities. The practice of annotation has been shown to be an efficient method to facilitate the understanding and interpretation of the content that is being read [1]. As its name explains, Web annotations are online annotations made externally over a web resource, e.g. a web page or PDF document. This particular type of annotation is displayed over a layer on top of the Web. The annotation layer is usually provided by a Web annotation client, which allows end-users to add, modify or remove information from a Web resource without modifying the resource itself.
The annotation process does not only consist of highlighting text, but also includes activities such as adding comments and classifying the content for giving meaning to annotations. The fact of being able to write down ideas and be able to give meaning to them, made us think it could be interesting to apply them in a concept mapping activity. Rephrasing the concept mapping as an annotation activity leads to a concept map construction based on the concepts and relationships that are annotated during a reading activity (see Fig.1).
Figure 1. Overview of the idea put forward concept mapping based on annotations.
The Standard Concept Map Construction Method explained by Cañas et al. [2], the concept map construction involves: 1. definition of focus question 2. identification of concepts 3. order concepts from most general to the most specific 4. add links & linking phrases 5. add cross-links 6. and map review. Next, we propose a similar method for annotation-based concept mapping, which follow some of these steps.
Definition of focus question. The reading activity allows students to discover new concepts and ideas from an unknown topic. This step involves defining which will be the topic of the concept map and collecting documents associated with it. It is important to properly select which documents will be used for the annotation activity because they are going to be the only knowledge source of students.
Concept handling. Annotations can serve for capturing text fragments that give rise to the conception of a new concept. When a new concept appears, the concept mapper should have the opportunity to capture it by highlighting and assigning a name to it. As an example, in Fig. 1 the concepts “photosynthesis” and “plants” are captured while reading in Wikipedia.
Relationships handling. In addition to finding new concepts, reading also allows finding how these are related to each other. As well as concepts, relationships (links & cross-links) can also be captured by annotations, but it involves one addition. In this case, apart from giving the name of the linking word, it is also necessary to include in the annotation the ‘from’ and ‘to’ concepts that link it. As an example, in Fig. 1 the linking word “used by” is captured while reading in Wikipedia for “photosynthesis” and “plants”.
The Map is reviewed. During the annotation activity, annotations are made to capture concepts and relationships among them. However, annotations themselves do not form a concept map. Therefore, a further step is required. Gathered concepts and relationships must be moved to a concept map, and students can refine and manipulate the knowledge that has been acquired from lecture notes. The result of this step is the concept map and the set of annotations that teachers use to validate it.
Benefits are twofold. First, annotations provide physical support for the reading material managed by students. Second, annotations provide the link between the reading material and the concept map, so that teachers can have a deeper insight about student misconceptions that do not stop at the concept map but the ground reading that roots the misconception. On the other hand, performing that annotation-based activity by hand and without support adds too much effort to the activity. Therefore, we have developed a web annotation client that supports the annotation-based concept mapping process.
REFERENCES
[1]
[2] Cañas, A. J., Coffey, J. W., Carnot, M. J., Feltovich, P., Hoffman, R. R., Feltovich, J., & Novak, J. D. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. Report to the Chief of Naval Education and Training, 1-108
CONCEPT&GO
Concept&Go is a web browser extension that extends and tailors a web browser interface to add functionality over reading resources with the aim of supporting the students' concept map construction process explained above. It also allows generating automatically the concept map with the annotations that sustain the concepts and relationships to facilitate the validation performed by teachers. When Concept&Go is activated over a reading material opened in the web browser, it shows a sidebar on the left part of the document (see Fig. 2). This sidebar contains the collected concepts and the required tools for developing the annotation-based concept map (e.g. buttons to annotate concepts and relationships) (see Fig.2).
Figure 2. A PDF augmented by activating Concept&
Concept&Go does not support the whole process by itself. It needs to make use of two external applications to perform the whole annotation-based concept mapping process. On the one hand, Hypothes.is is used as persistent storage to save and retrieve created annotations when necessary. It is a widely-used annotation server that requires to have a user account to use it. On the other hand, Concept&Go also uses CmapCloud (the cloud service of CmapTools)to depict the developed concept map with its annotations. Next, we briefly explain how to perform the steps of concept mapping in Concept&Go.
Focus question definition. When the extension is activated for the first time over a reading material, it asks to introduce the topic of the new concept map. Once the topic is defined, the topic concept is established as the root node of the concept map and it is displayed as a button in the sidebar. In addition, Concept&Go provides the opportunity to change the focus question anytime and to set any of the already annotated concepts as the root node of the concept map.
Concept handling. The next step is to start identifying concepts. When the student identifies a new possible concept during the reading process, he/she has to select the text fragment on the document and click the ‘new concept’ button. Once the concept is created, an annotation is created over the selected text and a new button is added within the sidebar in order to annotate more evidence for that concept.
Relationship handling. The next step is to create relationships between concepts. Relationships are made up of two concepts that are linked by a linking word. As well as with concepts, users can find ‘concept-link-concept’ triples propositions while reading. These relationships can be found written explicitly, implicitly, or collected on a table. Therefore, the user has to be able to capture the relationship by annotating the linking word in the same way as with concepts. To complete a relationship, it is necessary to provide the "from" and "to" concepts together with the linking word.
Map is reviewed. After capturing map elements that are relevant for the student, the student has to export the annotated content as a concept map to review the result. Concept&Go provides the option to export a concept map with the annotations to a CmapCloud account on the top part of the sidebar. The user has to provide his/her CmapCloud user credentials to associate Concept&Go with the CmapCloud account. Once the association is done, students are able to export the concept map, which is exported in a folder that is automatically generated in the CmapCloud home folder with the name of the concept map and its version. The folder contains the file that represents the concept map and the web annotation resources (URLs) that are referenced within the concept map (see 1 in Fig 3). The resources attached to the concepts and relationships of the concept map, allow to navigate to the annotation that created when the map element was captured (see 2 in Fig 3). Concepts maps can be shared with the teacher, and this brings the opportunity to evaluate the task with the concept map and the annotations to trace the origin of the elements.
Figure 3. (1) Resources exported to CmapCloud and the visualization of the concept map in the editor. (2) The URL resource attached to the concept ‘Relationship’ allows the teacher to navigate to the point the annotation was created
Concept&Go can be installed from the Chrome Web Store[1] and it is public-available under the MIT license on GitHub[2]. Moreover, a user guideline with a video-tutorial is provided, which explains more deeply the steps for using the annotation-client[3]
TODO
Until now it has been evaluated with colleagues, but we could not perform a consistent evaluation. We are trying to finde practitioners (i.e, teachers that use concept maps) that could collaborate with us to evaluate the idea and the instantiation of the solution (i.e, Concept&Go).
Achieve or allow for [Aim A] for [User U] in [Context C], employ [Mechanisms M1, M2, ... Mn] involving [Enactors E1, E2, ... En] because of [Rationale R].
Users: those whose aims are to be achieved.
Enactors: perform actions as part of the mechanisms that are used to accomplish the aim.
Context: Boundary conditions, implementation setting, a further user characteristic
Mechanisms: the means to achieve the ends envisioned by the design principles
Rationale: a justification for believing that the mechanisms will lead to achieving the aim.