“Designers and developers of instruction targeted at deeper learning and development of transferable 21^st century competencies should begin with clearly delineated learning goals and a model of how learning is expected to develop, along with assessments to measure student progress toward and attainment of the goals. Such instruction can and should begin with the earliest grades and be sustained throughout students’ K-12 careers.” (Pellegrino and Hilton, p. 9)

 

ABSTRACT

The US education system, by several metrics, is failing. For example, US education is not even ranked in the top 20 in the world. College and university leaders state the students they get from high schools are not ready for college. Employers state the graduates they get from college and universities are not ready for a work environment. Research shows that most academic environments remain siloed by subject and schools rarely integrate them, which inhibits the critical thinking and multi-disciplinary approach many employers want and that is critical for success in a complex environment.

Several frameworks and approaches try to solve these problems and issues. These include the P21 framework and the High Impact Educational Programs (HIEP). The elements of the frameworks have high promise, based on research and work with employers, but they have not made serious penetration. Serious games prove a mechanism to expand the frameworks and achieve their potential.

This paper builds the design for a series of keystone programs at different grade levels that use serious games to challenge students and integrate concepts into progressively complex problems. The paper employs a literature review of education, serious games, and gamification to assess the status and best practices. It then develops a keystone framework based on P21 and HIEP and a central scenario for the series of keystones, starting with the third grade. The core scenario will be the colonization of a distant planet. The scenario will get progressively more complex at each level and provide a vehicle for the creative, critical thinking across disciplines complex environments require. The paper ends with a technical design that supports the scenario, provides an engaging user interface, and combines fun with a challenging environment that requires critical thinking and an integrated approach to problem solving. It will also help develop critical mathematics, reading, and writing skills that are a core weakness in US education as shown by world education ranking.

 

Education Situation

Education in the United States is in trouble. The US may still have the number one economy in the world, but at least two global ranking systems for grades K-12 rank the US below 25th in the world. Colleges and universities report that many, perhaps most, incoming students are barely qualified and need remedial courses. Likewise, many employers report that new employees rarely have the basic skills required for entry level work.

When we look at primary and secondary education, the situation is perhaps even gloomier. A January 2017 segment on a recent Hechinger Report found that high schools do not prepare many students for even the basic college courses. Many other studies and anecdotes confirm the problem. For example, an April 2016 US News and World Report shows that high schools prepare only 37% of seniors for college. This report is similar to articles in The Wall Street Journal and Fortune Magazine and other sources. Unfortunately, this story is not new. Reporting on the problem goes back many years. In 2013, the United States placed 36th in the Organizations for Economic Cooperation and Development (OECD) rankings of mathematics, reading, and science and 25th in 2015. See the OECD 2015 PISA Report. Curiously, reporting on these issues seems to be lower in the last few years.

Educators recognize the problem but seem to focus on closing achievement gaps between races and sexes (AACTE, 2010). While there are clearly gaps, they seem to grow wider, rather smaller and the US as a whole is falling farther in global rankings and competitiveness. Current methods do not work.

There are many references to capstone programs, but they are largely at the undergraduate level and mainly in the student’s senior year. If we want to do things differently to better prepare our students, we need to bring these concepts to K-12, potentially as early as the third grade. Therefore, this paper uses the term “keystone” rather than “capstone” to distinguish the initiative from existing college-level programs. Keystone events need to be tailored to the grade level learning objectives, student capabilities, and the community needs. The Keystone Initiative seeks to address these issues and provide a K-12 initiative to bridge gaps.

Figure 1. Conceptual Inputs

Conceptual Inputs

The review of the education literature, especially revolving around the learning needs for the 21^st century, drives the conceptual inputs shown in Figure 1. The literature has four common themes: critical thinking, creativity, communication and collaboration (4C). Much of it also discusses STEM (Science, Technology, Engineering, and Mathematics) as critical subjects for success in the 21st century job market. Unfortunately, many employers say workers do not receive this education and are often ill-prepared for the modern work environment. Figure 1. shows how the conceptual inputs fit together. Much of the literature echoes these thoughts, to include Pellegrino & Hilton (2013) and the Assessment and Teaching of 21^st Century Skills, sponsored by CISCO, Microsoft, INTEL, and the University of Melbourne (Care et al., 2017, 2018; Vista and Care, 2020; Griffen and Care, 2015)

STEM

STEM are vital subjects for a complex, technology focused environment. A successful country needs an education system that can train scientists and engineers and provide some fundamental understanding of the STEM concepts in the rest of the population. This fundamental understanding is critical to ensure citizens understand the issues when they go to the ballot box. We need to teach STEM how to assess the STEM programs to ensure they add value to society. To do that, we need to understand success factors and develop measures and metrics. We also need teachers and employers to work together to ensure students learn the key skills required for success in today’s economy. Games can use these inputs and assess the type of STEM programs that provide the most value to society.

Figure 2. STEM and Society

P21

The P21 initiative is one way to help strengthen students’ and teachers’ ability to understand new concepts and to integrate them into education.

Figure 3. P2 Integration

The framework document (P21, 2019) states, “The P21 Framework for 21st Century Learning was developed with input from educators, education experts, and business leaders to define and illustrate the skills, knowledge, expertise, and support systems that students need to succeed in work, life, and citizenship.” Figure 3 shows how these core functions fit together to stimulate learning.

Multi-disciplinary

Contemporary problems rarely exist within a single disciple. Complexity spans disciplines and students must be able to integrate across disciplines. Unfortunately, education is still very stovepiped and as Pellegrino and Hilton (2012, p.22) note, there is little research on how to lower transfer competencies between disciplines. As Anft (2017) notes: “We’re in the silo-busting business across academe right now,” says Donnie Horner, provost at Jacksonville University, in Florida, which has formed a handful of hybrid graduate programs in the past five years. “The world is becoming more interdisciplinary and is forcing us to change to reflect it. The marketplace is telling us that students need to be well-versed in their discipline, but also technologically competent.”

Figure 4. C4I

Employers now require more workers who can bridge the gaps between professions — engineers who understand business, journalists with computer-programming skills, health-care providers who can crunch data. An Industrial Era approach with siloed subjects still dominates much of education in the US. Few schools adapt to the post-Information Age world that requires an interdisciplinary approach to complex issues. The Keystone Initiative adds an “I”—integration—to the 4C to keep the focus clearly in mind: C4I, as shown in Figure 4.

High Impact Educational Practices

The Association of American Colleges & Universities (AACU) provided an approach to high impact educational practices (HIEP) in 2008 (Kuh, 2008). While there do not appear to be many references after that, they add capability to P21 that add capability to the Keystone Initiative. These include learning communities, global perspectives, service learning and capstone courses and projects. These are critical to developing the skills required for a complex world in the 21st century AACU states, “On many campuses, assessment of student involvement in active learning practices such as these has made it possible to assess the practices’ contribution to students’ cumulative learning. However, on almost all campuses, utilization of active learning practices is unsystematic, to the detriment of student learning.”

Keystone Programs

The keystone programs provide the vehicle to help students learn C4I and integrate them across various disciplines. Gamification and serious games provide vehicles to develop C4I and to stimulate learning in fun and challenging environments. These games need to need to be tailored to the student’s grade level learning requirements and integrate these requirements together across the disciplines appropriate to the grade using both analog and digital methods, as discussed below.

Culture Shift

These inputs and changes will require a culture shift. Changing educational paradigms and breaking down silos requires the same type of cultural shift businesses and other organizations went through to adapt to 21^st century conditions. But businesses had a compelling reason to shift the culture: survival. They also had key measures and metrics to show why they needed to do it and to measure progress. While education has many measures and metrics, politics and other dynamics often obscure them, and the educational bureaucracy seems reticent to make meaningful reforms.

The Keystone Framework

Figure 5 shows the high-level Keystone architecture. The architecture uses framework conceptual inputs discussed above and research into serious games and gamification, especially with their fit with education.

Figure 5. Keystone High Level Architecture

Learning Objectives

The serious games for education literature are clear on two things. The game must start with the educational learning objectives. However, this rarely happens, as shown in several sources, such as Farrell and Moffat (2016). The normal processes for curriculum and lesson development and planning apply to serious games in the Keystone Initiative.

The measurement/evaluation component should specify the specific metrics and data used for the evaluation. Game developers need to tie this closely together with the data analytics for effective measurement and evaluation. These objectives should drive the components of the scenarios to ensure an effective challenge and problem engages the learning objective to allow for assessment.

Figure 6. Learning Objectives

Given the capabilities of serious games, the Learning Enablement System (LES), and ePortfolios, Keystone places a special emphasis on learning objectives to enable cognitive tutoring when applicable, assess student performance, and update the ePortfolio.

Learning Interfaces

Keystone uses the P21 framework, HIEP, and educators’ experience with employers. The interfaces are not standalone events. Rather, they are the skills and group interactions players employ to accomplish challenges and solve problems. Their use in game play will vary with the scenario, grade level, and complexity. In the end, they are true skills the game should teach. The scenarios provide the means to employ them through challenges and problems. Game designers need to tie learning objectives and learning interfaces together.

Figure 7. Learning Interfaces

Keystone has three types of game interaction:

• Communications. Employers noted in several studies that communication is a key skill requirement. They highlighted written and oral communication skills and collaborative problem-solving. The goal of these interactions is to help students work well together and for students to articulate and share their ideas in various communications media.
• Technical. These include STEM, critical thinking, and problem-solving. As with communication skills, they are byproducts of the gameplay and are used to accomplish challenges and overcome obstacles. For example, players may use STEM skills to assess a problem and then engage in critical thinking to explore it and assess options. The goal of these interactions is to help students become successful in their work.
• Civic. These skills help the students to place problems in a civic context and to understand how governments set policy, balance and apply resources to achieve civic aims. These interactions help students become better citizens.

Game designers can update and change these interfaces to meet specific learning objectives and organizational requirements.

Teacher and Student Mentors

The teacher is a central figure. But to be effective, the teacher needs training in the serious games approach and how to integrate the analog and digital aspects of the game into a seamless experience. The teacher needs to understand when to intervene and when to let the students run with their activities. (Baalsrud Hauge et al., 2013)

Student mentors should be two grades above the target students. The mentors should help the younger students and provide insights based on their experience in the game. The mentors are also learning. The act of mentoring often provides key insights into problems and issues as they answer questions and make suggestions. The game learning objectives should include learning objectives for the students and the teacher should facilitate their learning.

In higher grades, the mentors should take on most of the teaching and coaching and the teacher should spend more time working with the mentors and observing the game to ensure the learners meet the learning objectives and all aspects of the game interfaces are effectively engaged in the analog portion of the game.

Game Play

Game play includes both analog and digital elements. The design includes this hybrid aspect for several reasons:

• Real life exists in a hybrid mode. We do some things online and some things in person. Even online, most of it involves some communication. One of the manifest purposes of education is to prepare students for life. Therefore, the keystone events should replicate the conditions in life where possible.
• Many employers note employees need to communicate more effectively and work collaboratively, especially in problem-solving. That is why one of the three interfaces is communication. A hybrid game facilitates learning these skills. While one can talk to an avatar and gain a great deal from the interaction if the AI and NLP are good, it does little to facilitate the give and take of collaborative problem-solving, effective writing, and presentation.
• Research shows that not all students do well in the digital game environment. A hybrid environment mitigates the potential problems and even alienation of a pure digital game.
• The military, which spends tens of millions of dollars, perhaps hundreds of millions of dollars on serious games, uses a hybrid game. Commanders and staffs do their planning and decision-making in their command posts. They issue orders just as they would in a live event. These orders are then input into the digital aspect of the game where the games execute them and generate information and feedback to the players. It is a very effective approach and one that designers can readily adapt to educational serious games.

Communities of Practice/Interest

Communities of Practice/Interest (COPI) facilitate the communications learning interface. They help students to develop and share ideas and information and can be the forum for problem-solving. Most game engines do not include them, so they need to be a third-party inclusion into the game. For example, the Army includes controlled social media interactions in some of their games, replicating Twitter, Facebook, and other venues. They can also span the range from simple chat and file sharing to an integrated system that includes synchronous communications, online group document editing, and chat and file sharing. Moodle could provide an easy and configurable approach to the COPI.

Scenario Files

Scenario-based learning provides a flexible and effective way to guide learning and development during a game. At their heart, scenarios are one or more events with associated actions that are tied to learning objectives. Figure 8 shows the core components of scenario-based learning.

• Select Learning Objectives and Interfaces. Ideally, designers develop and maintain learning objectives separately so that educators and game designers can use them in multiple scenarios. The interface between learning objectives and the scenario should specify the learning interfaces the scenario will use.
• Develop Events & Actions. Marsh (2010, p.219) provides key concepts into how objectives should drive a story and its events. The actions and events provide the means to engage the learning objectives and interfaces and to evaluate them. The execution creates data that then flows into data analytics.
• Build a narrative. The narrative is a storyline with characters, challenges, and problems. Players engage the characters and negotiate the challenges and problems that are based on the events and actions. The narrative needs to be informative and engaging to encourage learning. Designers should tailor it to the cognitive capabilities and capacity of the learning audience. Scenarios with low cognitively developed learners should be straightforward, with little or no exogenous cognitive load and lower lever cognitive learning objectives. As the learning audience’s cognitive development increases, the scenarios may have larger exogenous cognitive loads to help make the challenges and problems more difficult. Learning objectives may also increase up the cognitive chain.

Figure 8. Scenario-based Learning

• Develop Branches. Branches allow the learner to move to different learning objectives within a game. Data analytics should establish trigger points for a branch activation. In a digital component, the workflow could tell the game engine to start the branch and the associated information and actions. In an analog component, the workflow could alert the teacher and/or mentors to start the branch and provide players with new information required to them to explore the branch.
• Execute & Conduct Data Analytics. Serrano-Laguna et al. (2016, p5) provide a structure for event and action data to flow into data analytics. Designers must instrument the events and actions to capture the data needed for assessment. The assessments help to determine scenario branches to engage and evaluate performance and progress. If the game moves into AI/ML capabilities, the data analytics will become more important, and the analytical sophistication will need to grow.

Games and Gamification

Gamification is most often defined as “using game elements in non-game contexts” (Hallifax et al., 2019, p.1). Kiryakova et al. (2014) add Kapp’s concept of game-based mechanics, aesthetics and game thinking. Most cite the use of badges, progress indicators, and other aspects to generate motivation. But there is some crossover, such as Kapp’s game-based mechanics, into the actual games themselves. Several sources cite potential negative effects on motivation from gamification, so we need to be careful. The Keystone Initiative, therefore, separates the learning and motivational aspects of gamification from the use of serious games. If the gamification concepts work to increase motivation in a setting, designers should include them, with the ability to turn them on and off or change depending on learner reactions and requirements.

Game Rules

Game rules need to span the analog and the digital components of the game. They need to specify the player’s abilities to interact with the game through the learning interfaces. As a minimum, they specify what a player can and cannot do and how they can do it. If the game includes non-player characters, the rules also need to specify how the players can interact with them. Game engines have rules embedded within them to specify how the game handles inputs and actions. The game designer needs to understand these rules and whether users can change them through parameters.

Admin Files

These files provide game documentation to include comments on parameter setting and game configuration to build scenarios, game components and architecture, modifications and updates. They should also provide notes on game play and suggestions for teachers, training materials, teacher instructions and suggestions, lessons learned from prior games, and interfaces to enter new lessons learned and suggestions for improvement.

Game Engine

The literature provides a robust assessment of various game engines and how to incorporate and configure them. The keys are to ensure the game engine supports the learning environment and the game engine works without external support during the game and teachers can easily resolve issues during play. The game engine must also support the learning environment. Some of the literature expresses concerns about the ability of off-the-shelf games to support cognitive learning objectives.

Data Analytics

Data analytics are critical to assessing student performance and progress and game execution. If the game includes Artificial Intelligence (AI) components, the data analytics helps to keep the AI rules current and provides data for the AI to execution. If the game includes intelligent tutors, data analytics provide information for the intelligent tutor to alter the complexity of the game and provide suggestions to the players. Teachers and mentors should monitor real-time analytics to help them determine when to intervene and advise.

Learning Enablement System and ePortfolio

A Learning Enablement System (LES) is a Leaning Management System (LMS) enhanced to focus on the 21st century learning requirements. It includes the core LMS, plus AI/Machine Learning to provide intelligent tutors and to assess student performance to manage game complexity. It can also host and feed COPI and ePortfolio. ePortfolios follow a student throughout their academic careers. While in a school, the school may link it to the school’s LMS to update it. When the student graduates or otherwise leaves the school, the ePortfolio follows the student. The ePorfolio may be as simple as a collection of files hosted on a site such as Google Documents. Or it can be a dedicated service, such as https://portfolium.com/. The Keystone Initiative seeks to extend the ePortfolio concept to include courses, grades, games and simulations engaged with outcomes. In this light, the ePortfolio takes on some characteristics of an LMS, but is portable and follows the student.

Infrastructure

The infrastructure to house the game needs to be portable and affordable. Potentially, a game engine sits on a school server along with either a dedicated file server/SharePoint system or access to Google documents. The server could also sit on a district file server to serve all the schools in the district. The more decentralized the server, the more capability and bandwidth it will require. Classroom laptops access the game server to design and configure games and scenarios and to play the game. The specific infrastructure architecture depends on the game engine, resources, and learning objectives in the school/district/state.

An example to test the framework

An effective educational serious game revolves around an educationally valid and engaging scenario. Learning objectives geared to the grade level need to drive the scenario and test the concepts and skills appropriate for the grade. Events and actions provide the vehicle to exercise and test the objectives in both analog and digital environments.

The keystone process starts with a master scenario. This scenario provides the core backstories and setting for a range of scenarios that can grow in scope and complexity to meet the learning requirements of different grades. Using a common master scenario allows schools to lower costs and provides a way to show students the layers of complexity within a problem set. Each grade level has one or more root scenarios. The root scenarios are subsets of the master scenario and manage the events and branches of the scenario execution. They provide more detailed backstories and characters and roles for scenario chain execution. At lower levels, this is a simple chain. As complexity grows, there are more branches, and a scenario may have several branches, depending on event execution or challenge status. At the simplest level, there is one event and learning objective per scenario and one branch forward to meet the overall learning goal.

Table 1. 3rd Grade Scenario Example
3rd Grade Scenarios
Scenario	3rd Grade Planet Selection Criteria	3rd Grade Planet Selection	3rd Grade Colony Site Selection	3rd Grade Colony team selection
Learning Objectives	Describe the conditions for a habitable planet during a mission planning meeting.	Select a planet to colonize, given information on the solar system..	Select a suitable site for the colony given the information on the map.	Develop a suitable colony team that balances the skills need for a sustainable colony.
Events/ Actions	First meeting with the Mission Planning Chief (MPC). Team will use the standard Third Grade PowerPoint template for the briefing. It will contain the site selection criteria. MPC discusses the criteria with the team and either approves the selection or guides them to suitable criteria through discovery learning. Mission Planning Chief.	Second meeting with the MPC. Team selects best planet for colonization (problem solving). Team uses standard Third Grade PowerPoint template to brief the MPC. MPC discusses the selection with the team and either approves the selection or guides them to a suitable site through discovery learning.	Third meeting with the MPC Team uses problem-solving skills to reject planets that do not pass the selection criteria and to select the best planet from those that meet it. Team will use the standard Third Grade PowerPoint template to brief the MPC. It will contain the site selected and the reasons for it. MPC discusses the recommendations with the team and either approves the selection or guides them to a suitable site through discovery learning.	Fourth meeting with the MPC Team has approved site. Team uses STEM skills to conduct a more detailed survey of the site to determine the skills need on the selected site. Team reviews the colonists against the site requirements and selects 100 primary colonists. The team briefs the MPC using the standard Third Grade PowerPoint template. MPC discusses the recommendations with the team and either approves the selection or guides them to a suitable site through discovery learning.
Success Criteria	Team captures at least 90% of the conditions for a habitable planet	Team selects the right planet.	Site is secure and has access to resources and arable land.	Colony team provides for government, farming, mining, engineering, research, and security.
Analog Mode	Low level Communities of Practice for internal team discussions. Briefings to the Mission Planning Chief.
Digital Mode	Simple game to execute challenges to find information and reporting/briefing templates.
Branch	Planet Selection	Site Selection	Colony Team Selection

The 3^rd grade and the 12^th grade students use the same master scenario, but their learning objectives and the events are different. The 12 grade students have more complex learning objectives and the events and challenges are far more interactive and require higher cognitive skills.

Table 2. Comparison of 3rd and 12th Grade
Item	3rd Grade	12th Grade
Learning Objectives	Based on 3rd grade knowledge and skills. One event tied to each objective. Most objectives are on the lower end of Bloom’s Taxonomy.	Based on 12th grade knowledge and skills. Multiple events tied to each objective. Most objectives are in the middle range of Bloom’s Taxonomy.
Scenario Complexity	Simple	Moderate
Events per scenario	1	1-3
Complexity of events	Simple. Gather information, use fill in the blank templates.	Conduct experiments and research to gather data and turn it into information. Templates are free form.
Complexity of challenges and problems	Simple quests to find information and data.	Complex challenges that require game engine interface
Scenario Branching	1 branch per scenario.	1-3 branches per scenario.

Teacher Planning and Execution

Teacher planning time is precious. It has always been scarce, but I suspect in this day of ever-increasing mandates, it is even more so. Therefore, if schools want to use keystone events, they need to be simple and “off the shelf” as possible. These events, however, also need some flexibility that allows the game designer to tailor them to the learning objectives. They need to go to one spot, make a few selections, and have the configurations and resources they need to execute the event. Therefore, Greenman House developed a prototype planning tool based on the Keystone Framework. Figure 9 is a screenshot of the Keystone planning and configuration module. It allows teachers and game designers to manage all aspects of the keystone event and configure it for execution. The goal is to allow teachers to focus on teaching rather than game management.

Figure 9. Keystone Planning

The module allows teachers and game designers to develop reusable learning objectives, events, characters, and other aspects of the game to simplify development and administration.

Conclusions and recommendations

The fusion of serious games, 21 century learning frameworks and an integrated planning and execution approach provides educators with a power vehicle to meet 21^st century learning objectives. There is a volume of literature on 21^st century skills and the C4 concepts and even more on serious games and serious games in education, but almost nothing that connects them together, along with the multi-disciplinary approach. The Keystone Initiative provides this linkage point and puts together an integrated framework to use serious games, education concepts, and the 21st century skills employers require.

Pellegrino and Hilton (2012) note current research on 21^st century skills is limited and primarily correlational. As schools implement the Keystone Initiative, researchers can do two things. First, they can perform longitudinal assessments and follow students through K12 to college and work to determine the effects and value of 21^st century skills. Second, they can use the games to see their effect on learning by using treatment groups and control groups. Finally, researchers can study the effects of gamification on motivation across various schools and students to determine how and when to use gamification.

References

(AACTE), A. A. of C. for T. E. (2010). 21st Century Knowledge and. In Education (Issue September). https://doi.org/10.1787/9789264193864-en

Anft, M. (2017). Breaking professional schools out of their silos. Chronical of Higher Education, 63(35), A8–A12.

Baalsrud Hauge, J. M., Stanescu, I. A., Arnab, S., Moreno Ger, P., Lim, T., Serrano-Laguna, A., … Degano, C. (2015). Learning Analytics Architecture to Scaffold Learning Experience through Technology-based Methods. International Journal of Serious Games, 2(1). https://doi.org/10.17083/ijsg.v2i1.38

Care, E., Kim, H., Vista, A., & Anderson, K. (2018). Education system alignment for 21st century skills: Focus on assessment. Center for Universal Education at the Brookings Institution., January, 1–40. https://www.brookings.edu/wp-content/uploads/2018/11/Education-system-alignment-for-21st-century-skills-012819.pdf

Care, E., & Wednesday, A. V. (2017). Education assessment in the 21st century: New skillsets for a new millennium. 21–23. https://www.brookings.edu/blog/education-plus-development/2017/03/01/education-assessment-in-the-21st-century-new-skillsets-for-a-new-millennium/

Care, E., Kim, H., & Tuesday, A. V. (2017). How do we teach 21st century skills in classrooms? 2017–2019. https://www.brookings.edu/blog/education-plus-development/2017/10/17/how-do-we-teach-21st-century-skills-in-classrooms/

Care, E., & Lou, R. (2016). Assessment of transversal competencies: Policy and practice in the Asia-Pacific region. https://unesdoc.unesco.org/ark:/48223/pf0000246590

Farrell, D., & Moffat, D. C. (2014). Adapting cognitive walkthrough to support game based learning design. International Journal of Game-Based Learning, 4(3), 23–34. https://doi.org/10.4018/ijgbl.2014070103

Griffin, P., & Care, E. (Eds.). (2015). Assessment and Teaching of 21st Century Skills. Springer Dordrecht. https://doi.org/10.1007/978-94-017-9395-7

Hallifax, S., Serna, A., Marty, J. C., & Lavoué, É. (2019). Adaptive Gamification in Education: A Literature Review of Current Trends and Developments. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11722 LNCS, 294–307. https://doi.org/10.1007/978-3-030-29736-7_22

Kiryakova, G., Angelova, N., & Yordanova, L. (2014). Gamification in education related papers. Actas de La 9^a Conferencia Internacional de Educación y Ciencia de Los Balcanes., 1–6. https://d1wqtxts1xzle7.cloudfront.net/53993983/293-Kiryakova-with-cover-page-v2.pdf?Expires=1625349919&Signature=aIGwmdXFaZERw4pCT~rerIAdM0c5-oZPQHldlrEGrKIp1j~mguk71m~bU0ji1kfLTk~39xzrHP9AfdLYOY-3Cyx8P0xClWyuxkM74T8DoWXxcYeDDg0P1~A6YeFNYghcTqMrB9DtD0KoX9

Kuh, G. D. (2008). High-Impact Educational Practices. In High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter.

Marsh, T. (2010). Activity-based scenario design, development and assessment in serious games. In R. Van Eck (Ed.), Gaming and Cognition: Theories and Practice from the Learning Sciences (pp. 213–226). https://doi.org/https://doi.org/10.4018/978-1-61520-717-6.CH010

P21. (2019). Framework for 21st Century Learning.

Pellegrino, J. W., & Hilton, M. L. (2013). Education for life and work: Developing transferable knowledge and skills in the 21st century. In Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. https://doi.org/10.17226/13398

Serrano-Laguna, Á., Martínez-Ortiz, I., Haag, J., Regan, D., Johnson, A., & Fernández-Manjón, B. (2016). Applying standards to systematize learning analytics in serious games. Computer Standards and Interfaces, 50, 116–123. https://doi.org/10.1016/j.csi.2016.09.014

Vista, A., & Monday, E. C. (2017). Education assessment in the 21st century: Pathways to the future. https://www.brookings.edu/blog/education-plus-development/2017/03/13/education-assessment-in-the-21st-century-pathways-to-the-future/