This study is a basic research on the quality improvement and efficacy evaluation of STEAM (Science, Technology, Engineering, Arts, Mathematic), and aims to develop evaluation models based on evaluation criteria of study outcome and analytic hierarchy ...
This study is a basic research on the quality improvement and efficacy evaluation of STEAM (Science, Technology, Engineering, Arts, Mathematic), and aims to develop evaluation models based on evaluation criteria of study outcome and analytic hierarchy process (AHP) as well as evaluation tools for key competencies of study outcome. As research tools, literature review, content analysis, survey, analytic hierarchy process (AHP), and expert council were used and the results were as follows.
First, the key competencies of evaluation criteria on the STEAM study outcome were classified as 5 large areas and 26 middle areas after preliminary study and expert council, and an operational definition was devised.
Second, a validation survey was conducted 3 times with experts to verify validity of the devised key competencies. As a result, 3 large areas, 9 middle areas, and 18 evaluation items were developed based on the analysis and opinions. Validity of the operational definition was also verified using the same method.
Third, according to the rank of key competencies, capabilities of convergence accomplishment was the first (0.394), followed by capabilities of convergence cognitive (0.368) and capabilities of convergence attitude (0.238). Specifically, capabilities of convergence accomplishment comprised problem solving (0.200), cooperation (0.076), convergence tool utilization (0.076), and communication (0.059). Capabilities of convergence cognitive comprised creative thinking (0.183), understanding convergence knowledge (0.110), and critical thinking (0.075). Capabilities of convergence attitude comprised responsibility (0.153) and consideration (0.085).
Fourth, a study outcome evaluation model in STEAM was developed based on the key competencies and their analytic hierarchy process (AHP).
Fifth, a key competencies BARS-based evaluation tool was validated and developed after 2 survey results with experts and expert council. BOS method (example) and self-diagnosis evaluation tool (example) were developed after correction through reviewing prior examples in the literature and expert council.
Thus, this study developed criteria and tools to assess STEAM study outcome and provided study objective in STEAM and experience-based basic material to accomplish it. In addition, teachers' freedom in evaluation was secured since the ground of competency assessment was prepared, which would help them to observe and evaluate specific competences.
The results from this study can be utilized in connection with project-based convergence education in elementary, middle, and specialized high schools and other creative experience-based various convergence education programs. Furthermore, they can be utilized in education institutions for the gifted children and creative engineering education program in college (Capstone Design).