Hi Dr. Kowler! Thank you for your question.

Several theoretical principles and findings within cognitive psychology will guide the design of these games.
One significant concept I am exploring is the idea of ‘mirror neurons’. Since the primary goal in at-home stroke rehabilitation is to regain motor ability, I’d like to take a look at how observation and imitation play a role in facilitating this process. I’ll be designing a game scenario in which an avatar acts as a virtual “trainer” by performing an action and having the player synchronize with that action through imitation, similar to how sword-play, martial-arts training, and dance training occur. The user will be interacting in physical motions by using a “smart object” – “smart” in that it records input motions and provides physical feedback (imagine the object being a baton, and the game scenario being a choreographed sword fight. When the swords collide in the game world, a vibro-tactile signal is output from the object to give the user a physical sense of collision – the cue to stop movement.) The game scenarios will be guided and “adaptive” – the pace of the sword fight, for example, will change in real time to match the user’s current range of motion. The goal here is to determine if triggering the mirror neuron mechanism has an impact on motor control re-acquisition and, more importantly, motivation (via Reinforcement Theory).

I’m looking at ways to design games so that feedback on improvement is immediate and clear. If a user completes a motion at a required threshold (provided by a therapist), immediate feedback within the game (including audio, visual and haptic cues) and immediate effect on game outcomes (such as progression to the next level of difficulty) help reinforce the association between the cause and outcome, in a process consistent with Cognitive Evaluation Theory. I’m also looking very closely at the idea of “flow” and “flow theory”, an aspect of cognitive psychology that has been heavily utilized and implemented within serious game design and research. This theory is perhaps the greatest argument for real-time adaptation within game design. My task is to ramp the difficulty in the game not just between game sessions, but also directly within a game as the user is playing. To keep the player engaged (and on the optimal track to motor control recovery), I will design my games to react directly to the player’s motions. A failed motion task results in subsequently easier tasks to prevent the player from becoming frustrated, and successful motion results in an immediate increase in difficulty to maintain challenge.

In further work, I’d like to look at social gameplay using in-game cooperation tasks, and how other psychological aspects such as affect play their role in supporting motivation within game design.

Thank you for your question, Dr. Irizarry.

Previous work has established basic guidelines for assessing motivation and affect in serious games ¹. One of the guidelines mentioned, and perhaps the most important point I am considering, is that any sensing mechanism used to detect motivation or affect needs to be unobtrusive, so that it has no influence on the gameplay experience. This limitation inevitably rules out some of the sensing methods which are potentially the most reliable (such as bulky wearables that monitor heart rate or skin response or certain emotion recognition mechanisms with heavy constraints on body positioning). Instead, I want to incorporate the sensing mechanisms into the gameplay itself, as much as possible. Some potentially useful tools for this are the smart objects I am designing for user interaction. I am exploring the use of surface sensors for skin and heart response as well as pressure detection and movement quality detection on these controllers. These sensors will provide data that I can use to assess metrics such as arousal and confidence as measures of affect and motivation, respectively. Other researchers have used similar mechanisms; for example, there was a study on emotion detection through the gamepad ².

I will also explore using cameras for facial recognition and emotion detection to measure affect. Our research lab, the Center for Cognitive Ubiquitous Computing here at Arizona State University, is employing a camera detection strategy for facial expression recognition which is being used for social interaction ³. I will explore whether or not this is an effective fit for detecting affect in rehabilitation gaming. For long-term measures of motivation, I will use external evaluation strategies, including recording total play time per week for each test subject and looking for consistencies over a period of several weeks. Of course, I will also gather feedback from the players using weekly interviews as well as questionnaires. I will develop models for motivation and affect based on short term metrics (arousal, confidence, sense of self-efficacy being primary factors) and long-term metrics (hours of gameplay per week and flow rate measure over an extended period) and compare the data obtained from the smart objects, capture mechanisms, in-game logs, and regularly scheduled interviews and questionnaires with these models to evaluate whether or not my system effectively achieves motivation and affect for players.

Reference:

¹ E. Hudlicka, “Affective game engines: motivation and requirements,” in Proceedings of the 4th International Conference on Foundations of Digital Games, New York, NY, USA, 2009, pp. 299–306.

² J. Sykes and S. Brown, “Affective gaming: measuring emotion through the gamepad,” in CHI ’03 Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA, 2003, pp. 732–733.

³ S.Krishna, T.McDaniel, S. Panchanathan, “Embodied Social Interaction Assistant,” Tempe, AZ, 2010.

Hi Dr. Gauvain, and thank you for your question.

This is a critical point I am taking into consideration in my research. Prior research on stroke rehabilitation games derived a set of design criteria which must be met to maximize playability for the target population ¹. Research projects such as the ElderGames project ² have focused specifically on these criteria, since they interrelate directly with motivation. One design requirement to consider is low cognitive load. I am being careful to design my games so that at any given time, the information present on the game screen is not so complex in nature that it overwhelms the player. I plan to determine this appropriate level of cognitive load by designing multiple rapid prototypes of my game interfaces (with varying complexity) and performing user testing early on with the designs to pinpoint the simplest design which users still find engaging. Another criterion is the appropriateness of the games’ genre. Interviews and studies from prior work indicate a bias toward puzzle and arcade games with very simple objectives and explicit feedback on performance. Based on this information, I’m targeting these genres for the design of my games, although ultimately the final choice of genre will be the result of heavy feedback gathering. In the sword training example I mentioned earlier, the scenario will be designed as an arcade style game with a single objective at any given time: to match the opponent’s movement. The focus of the interface will be on that opponent, and the pacing of the exercise will be slow enough that the movements can be clearly followed. Even this design choice, however, is subject to the feedback of test participants.

I attempt to address other criteria mentioned in these studies, including sensitivity to slower response and decreased acuity and motivational feedback, using adaptive game design with immediate feedback. This choice in my design was specifically based on these criteria for designing games for older adults, and addresses the frustration and sense of hopelessness you describe. The big benefit that clinical therapy currently has over at-home therapy is that with a therapist present and guiding movement, the individual knows when he or she is completing exercises correctly, since the movements are directly guided and feedback on positive performance is provided immediately. This serves to alleviate some of the frustrations in the clinical environment, and since the therapist is not present for at-home exercises, I’m designing games to simulate this feedback. This way, at any point during an individual’s progression through a stroke rehabilitation program, and independently of that individual’s current level of motion control and recovery at any given moment, the game task should always remain challenging but feasible (this is the person-centered aspect of my research). Game elements can be controlled automatically and in real-time to provide this environment, depending on the game’s genre. For a game requiring wrist rotation as motion input, for example, the degree of wrist rotation can vary from motion to motion depending on the user’s level of success with the last motion input, as can the time frame required to input the motion. These elements can also be modified when the game detects frustration (via the detection mechanisms discussed in my previous response). Finally, if one motion is an improvement over the previous, the game will provide feedback immediately upon completion of the motion before returning the focus to the main game task. I will evaluate whether these design strategies make the games enjoyable and effective independent of age and background of the player, and whether they successfully assist in replacing the individual’s frustration with the motivation to complete rehabilitation therapy.

Reference:
¹ E. Flores, G. Tobon, E. Cavallaro, F. I. Cavallaro, J. C. Perry, and T. Keller, “Improving patient motivation in game development for motor deficit rehabilitation,” in Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, New York, NY, USA, 2008, pp. 381–384.
² Fabregat, M., 2006. “Game typologies to be developed and their relationship with the parameters to measure. Development of High Therapeutic Value IST-based Games for Monitoring and Improving the Quality of the Life of Elderly People,” Information Society Technologies project CN 034552, ElderGames Project, final report version.

Hi Dr. Waring. Thank you for your question.

I’m glad you mentioned this point, because I do need to clarify the project’s focus a bit. As an initial area of application, we are currently restricting the scope to upper limb stroke rehabilitation. We’re currently looking at motions such as finger flexion, wrist flexion and rotation, elbow movement, reaching and grasping, and each game targets one of these motions specifically so that assessment of progress is more accurate. Even among these motions there is high variability of function and control among individuals, and adaptive gameplay accounts for this. By targeting each game to a specific motion exercise and adapting to user status, I can avoid the issue of attempting to address multiple stroke-related impairments within a single game framework while allowing the therapist to assign games specific to the necessary movement exercise for each individual.
As a result of restricting the scope this way, getting access to a large user base for testing is a challenge. We do have access, through partnerships with local medical and physical therapy centers, to a population of individuals undergoing stroke therapy which is large enough to generate reasonably reliable results for some of the user studies I’m planning to complete (particularly those relating to functionality and improvement of motor control), and for other studies (usability and, most importantly, motivation) I am working with my research group and with our IGERT to establish connections with rehabilitation centers which can provide the necessary population for meaningful evaluation data.

Thanks Ramin!

Hi Dr. Gneezy, and thank you for your question.

While this research is certainly not the first attempt at applying serious games toward stroke rehabilitation, the unique aspect of this approach is in the application of person-centered design and adaptation. This approach is reflected in my work as an attempt to influence long-term motivation. “Long-term” motivation has seen little attention in prior work, as the focus of most evaluations is on measurements of immediate interest in users, and since rehabilitation therapy is a time-intensive process that requires commitment over long periods, this is an important factor with which I am attempting to link serious games. The project, from a broader perspective, is a step towards linking gaming frameworks with the cognitive processes that determine goal-commitment, confidence, dedication to physical activity, skill re-acquisition and reinforcement. Furthermore, the relation between triggering of the mirror neuron mechanism and adaptation in serious game design has been largely unexplored, and post-stroke motor rehabilitation provides a perfect area of application since the motion exercises are focused enough that these linkages can be assessed. To provide a first step in establishing this linkage between design choice and consequences for cognitive processes in long-term activity, I would claim, is the incremental contribution of this work.

Thank you as always, Jay! None of the efforts in our APAcT IGERT program would be possible without your incredible dedication and guidance.

Thanks, Debra!

This is an excellent point, and one I am trying to address with the adaptive gaming design approach. The idea is that, regardless of an individual’s level of prior skill and background in games, the gameplay experience should remain challenging but not overwhelming. I’m attempting to achieve this by having each game modify itself in real-time to the user’s level of skill and range of motion. My hypothesis is that this will eliminate individual gaming background as a potential factor in motivation. Of course, a user’s general interest in games is also an important factor and I’m hoping to get a measure of this using questionnaires prior to user studies. If this has a heavy influence on outcome, I’ll look at further simplifications I can make to game objectives and also examine the effect of changing a game’s genre on the population. I’m very interested in exploring the impact of this person-centered design style on both motivation and rate of recovery with users of varying gaming interest.