DSR Buddy

Michi Gau

Alex Maedche

Jan vom Brocke

Description

In this project, we aim to create a new and lightweight user interface for the MyDesignProcess platform in order to create new research activity descriptions. The interface is based on a conversational agent and supports different user stories. The agent should act as a research "buddy" and will support novice researchers and experienced researchers in conducting their design science research process documentation.

Design Canvas

Problem Problem What is the problem for which a DSR project aims to deliver a possible solution? Problems ought to be defined by means of problem articulations and characterized by positioning the problem in a problem space. Previous research has identified the context, described by the domain, the stakeholder, time and place, and goodness criteria. Hello :) Creating documentation needs a lot of effort and is often time-consuming (Mohammad, 2005). Lack of integration of knowledge management into individuals daily work activities (Akhavan, 2015)	Research Process Research Process What are the essential activities planned (or conducted) to make the intended contribution? When the intended contribution is design entities, the process includes building and evaluating activities. These activities specifically include anchoring the design through the execution of tasks like literature reviews and meta-analyses. It is recommended that the construction and assessment activities be planned and documented together in order to facilitate concurrent design and evaluation. A high-level list of research activities used to scope the DSR project in the process dimension can be enhanced by logs of the research process that are kept using DSR tools. These actions for theorizing on the design may also be included in the process described. Activities for theorizing can draw from a variety of research methodologies and strategies of inquiry, including qualitative and quantitative empirical research, whereas activities for processing the design can draw from DSR process models like the Peffers et al model. Design Science Research Develop a prototype and study the impact of the documentation behavior of design science researchers.	Solution Solution What is the solution to the problem that a DSR project is investigating? The solution description identifies the representation of the solution as a construct, a model, a method, an instantiation, or a design theory and expresses concisely the fundamental mechanisms of the solution and how the solution is situated in the solution space. Designing and implementing a contextual AI assistant with design features to support novice researchers in their DSR activity documentation process.
Input Knowledge Input Knowledge What previous knowledge will be used in the DSR design? Design knowledge can be either descriptive, explicatory, prophetic (omega knowledge) or prescriptive (delta knowledge). Three different types of input — kernel theories, design theories, and design entities — can be differentiated for high-level communication about DSR projects. Human AI Interaction (Human-Computer Interaction) Knowledge Management Smart Personal Assistant Design Science Research	Concepts Concepts What are the core concepts that were utilized in the DSR project's research? The concepts used to describe the process and input and output knowledge, as well as the words used to characterize the study, such as the problem and solution space that the DSR project focuses on, must be described precisely. To ensure a rigorous execution of the evaluation activities, it is especially crucial to have a precise definition of the key concepts. Knowledge Reuse (Markus, 2001) Knowledge Management Prozess	Output Knowledge Output Knowledge What kind of knowledge does the DSR project produce? Naturally, DSR projects generate design knowledge, which is referred to as omega knowledge. However, unlike the solution description, the design knowledge produced by the project puts the problem and solution spaces in relation to one another. If a DSR project's goal is to create design entities rather than design theory, the descriptions of such entities do not represent design knowledge because only the outcomes of the design entity's evaluation in context do. When the project is explained, these results are subsequently documented as output knowledge. Design features for integrated research activity documentation tool support. DSR Buddy implementation: A Conversational Agent to support research activity documentation.

More information about the DSR grid

Research Pitch

Internal presentation and discussion of the first draft of the research idea in our research group.

We pitched the idea of using conversational agents (CA) to support design science researchers in their research activity documentation tasks to increase productivity and decrease the workload. Based on the feedback we reshaped the idea and designed an initial research design. The reshaped research design is described in the following activity descriptions.

Kickoff
Pitch Presentation

Problem

We aim to support design science researchers in structuring research activities and transforming them into research activity documentation.

Additionally, we support captureing different research activities, for example, requirements, design decisions, research activity ... that contribute to the design knowledge of a DSR project.

Furthermore, we aim to facilitate the documentation process to reduce the efforts of researchers in their documentation tasks.

We first conduct a literature review to identify related work and the research gap. Based on the results we derive a research question. We then defined an initial set of design requirements.

Relevant Literatur

We collected prior research and relevant literature on how to document knowledge-intense research process activities in DSR and other research fields.

The core references are listed below.

Documentation in the field of healthcare:

https://www.researchgate.net/publication/202342455_Management_of_Knowledge-Intensive_Healthcare_Processes_on_the_Example_of_General_Medical_Documentation

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5297955/

https://www.semanticscholar.org/paper/Quality-of-nursing-documentation%3A-Paper%E2%80%90based-Akhu-Zaheya-Al-Maaitah/45b82afd0501065013a95a09a7437efc0e8b5d99

Half a day is used for documentation in health care: https://www.ama-assn.org/press-center/press-releases/type-click-tasks-drain-half-primary-care-workday and http://telecareaware.com/now-ehr-data-entry-50-of-primary-care-doctors-workday-ama-university-of-wi-report/

Knowledge Management Perspektives

Retrieving knowledge from human minds is an expensive process that is time-consuming and requires skilled personnel (Mohammad, A. H., & Al Saiyd, N. A. M. (2012). Guidelines for tacit knowledge acquisition. Journal of
Theoretical and Applied Information Technology, 38(1), 110-118. Retrieved on October 10th, 2013,
from http://www.jatit.org/volumes/Vol38No1/15Vol38No1.pdf)

Lack of integration of KM into individuals' daily work activities, creation of repositories without addressing the
need to manage content... (Akhavan, P., Jafari, M., & Fathian, M. (2005). Exploring failure factors of implementing knowledge management systems in organizations. Journal of Knowledge Management Practice, 6. Available at
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2188273)

https://www.scielo.br/scielo.php?pid=S0104-530X2017000200248&script=sci_arttext&tlng=en

Mind to paper: https://www.researchgate.net/publication/236040925_Mind-to-paper_is_an_effective_method_for_scientific_writing

Conversational Agents in Education

https://www.sciencedirect.com/science/article/pii/S0360131521000257

Source: Winkler, R., Söllner, M., & Leimeister, J. M. (2021). Enhancing problem-solving skills with smart personal assistant technology. Computers & Education, 165, 104148. https://doi.org/10.1016/j.compedu.2021.104148

Literature Review
Problem Definition

Research Question

How to design a conversational agent to support design science researchers in documenting their DSR projects?

Design Requirements

In the following, we describe an initial set of design requirements (DR) derived from the literature to support research activity documentation in DSR.

DR1: The DSR Buddy should support structured research activity documentation.

Source: vom Brocke, J., Fettke, P., Gau, M., Houy, C., Maedche, A., Morana, S., & Seidel, S. (2017). Tool-Support for Design Science Research: Design Principles and Instantiation (SSRN Scholarly Paper ID 2972803). Social Science Research Network; https://ssrn.com/abstract=2972803. https://doi.org/10.2139/ssrn.2972803

DR2: The DSR Buddy should support DSR with different levels of skills and experience to document their research activities.

Source: Herwix, A., & Rosenkranz, C. (2019). A Multi-Perspective Framework for the Investigation of Tool Support for Design Science Research. In Proceedings of the 27th European Conference on Information Systems (ECIS). https://aisel.aisnet.org/ecis2019_rp/164

DR3: The DSR Buddy should be integrated into individuals’ daily work activities.

DR4: The DSR Buddy should support the learning process of DSR novices.

Source: Winkler, R., Söllner, M., & Leimeister, J. M. (2021). Enhancing problem-solving skills with smart personal assistant technology. Computers & Education, 165, 104148. https://doi.org/10.1016/j.compedu.2021.104148

DR5: The DSR Buddy should be highly engaging so that researchers document their design decision and knowledge.

Lukyanenko, R., & Parsons, J. (2020). Design Theory Indeterminacy: What is it, how can it be reduced, and why did the polar bear drown? Journal of the Association for Information Systems, 1–59. https://doi.org/10.17705/1jais.00639

Design Requirements

Development

In the following, we describe the development and the implementation activities of the DSR buddy.

Design Features

DF1: DSR process documentation guidance (DR2, DR4):

The CA provides knowledge on well-established DSR process and activity descriptions including the original source, examples, and further readings on different DSR approaches used in the field of IS in order to provide guidance on how to execute and document DSR activities. Moreover, the CA provides process descriptions on DSR support processes, such as descriptions of the literature review process proposed by Webster and Watson, focus group research according to Trembly et al., and a description of the case study research process proposed by Benbasat et al.

DF2: Easy capturing of DSR activities (DR1, DR3):

The CA enables the creation of new research activities and aggregates them to selected DSR projects. In contrast to the MDP web form, where a user must navigate to a project and the new activity form first, the CA asks researchers to provide the necessary information to create a new activity description in a DSR project. Activities might, for example, take the form of a meeting, a new idea supporting a research project, or a planned evaluation. Activity descriptions can be provided either by text or by speech input. In the case of speech input, the system transcribes the speech to text and stores the activity description in the form of text. The automatic transcription feature supports only the English language so far. Descriptions created by the CA can be edited and refined using MDP at a later stage.

DF3: Feedback on DSR process descriptions (DR4, DR5):

The CA can provide feedback to researchers based on existing activity descriptions in a DSR project. The system analyzes all activities in a project and based on predefined rules, reports if there are any issues or improvements available for a DSR project. The current implementation contains two rules. The first rule seeks for specific activity types, for example, a research question or constructs, and reports if they are missing in the selected project. The second rule analyses the activity descriptions based on the provided text. Suggested improvements may include recommendations to provide more detail in empty (or very short) activity descriptions, or when essential activities are missing—for example, when no research question is found in a project. The CA reports on the found issues and asks the researcher to fix them if necessary.

Design Features

System Architecture

The DSR buddy is integrated into the MyDesignProcess.com (MDP) platform but is running on its own server. This separation is mainly due to the performance requirements of the conversational agent. Almost every change in the CA business logic results in a recreation of the CA model. The underlying TensorFlow framework requires a bunch of resources to retrain the model that the webserver of MDP can not fulfill.

The proposed architectures allow the use of existing MDP features like login, activity management, templates, and many more.

Design
System Architecture

Conversational Agent Implementation

The CA is implemented on top of the chatbot framework RASA.

Currently, the CA implementation contains 3 stories, 17 rules, 16 intends, 8 custom actions, and 3 forms.

The server is set up to listen to a socket channel. The action server interacts with a REST API with the MDP system.

Integration of the DSR buddy into MDP:

Screenshots of the single design feature implementations:

The DSR Buddy features are available for all registeredd users on myDesignProcess.com.

Implementation
Screenshots

Feedback Meeting

We presented the proposed design features in a research pitch presentation and the latest implementation of the DSR buddy to 11 researchers. The group contained two senior researchers and nine Ph.D. students applying DSR in their research projects.

Feedback of the researchers:

Optimize voice interface, e.g.:

User: „create new feedback activity for test project“ or „create a new activity for asdf project“

Instead of clicking through an assistant… Use intend extraction features.

Ask the DSR buddy for e.g., reference models:

How to conduct a literature review?

How to conduct a DSR project?

One researcher mentioned that he particularly likes the hint feature of the DSR buddy.

Meeting

Demonstration

To demonstrate the applicability of the DSR Buddy, we documented our DSR project retrospectively using the prototype.

The documentation of the DSR project with the title “DSR Buddy” describes the design and implementation of the conversational agent to support design science researchers in planning, structuring, and documenting their DSR projects.

Also, the project documentation provides a project overview based on the Design Grid proposed by vom Brocke and Maedche and includes a set of activity descriptions to illustrate the design process.

The initial activity descriptions were created using the DSR Buddy using the activity capturing features. All the activities were created using a smartphone and by providing a short activity description via voice input. We further edited, refined, and structured the activities using MDP at a later stage.

A demonstration video can be found here: https://youtu.be/IMFYE7xFkM4

Prototype
Demonstration

Documentation

Besides the documentation on myDesingProcess.com we also submitted and presented a manuscript describing the DSR project to the WI2022 conference. The accepted paper can be found in the AIS eLibrary: https://aisel.aisnet.org/wi2022/prototype_track/prototype_track/35/

documentaton
WI2022

Result

The resulting artifact is embedded in MyDesingProcess.com. You can log in or sign up to try and use the DSR buddy.

You can use the DSR buddy to create new activities or get inspired by some hints or help on established DSR methods.

An illustrative video of the DSR buddy action can be found here: https://youtu.be/IMFYE7xFkM4

Result
Artifact

DSR Buddy

Michi Gau

Alex Maedche

Jan vom Brocke

Description

Design Canvas

Problem

Problem

Research Process

Research Process

Solution

Solution

Input Knowledge

Input Knowledge

Concepts

Concepts

Output Knowledge

Output Knowledge