Students' Generated Questions Quality by Developing STEM-based E-Module in Science Learning

Article history: Received: 17 September 2020 Received in revised form: 07 December 2020 Accepted: 29 December 2020 Available online: 31 December 2020


Introduction
Science and technology were overgrowing. That is becoming a basic need for humans to gain daily information. Furthermore, society for now really engaged with the gadget and demand higher digital technology. That causes the flow of information very easy to be received to be called a revolution industry 4.0 era (Saykili, 2019;Kalolo, 2019;Hashim, 2018). The demand for education in this era is high, especially for the student's prospective teacher. Not only emphasize cognitive but also skills. One of the essential skills to develop is higher-order thinking skills, and questioning skills are included. a Asking a question is one of the essential human habits. The ability to ask questions is a thought process related to critical thinking, creative thinking, and problem-solving (Chin & Osborne, 2008). Question skills are essential because closely related to the curiosity that identifies a phenomenon or a question. In this digital era, the flow of information is easy to get, and one of the effects is hoax statements spread uncontrollably. So that, needed good question skills with critical thinking to verify the news or statements. Those can help construct knowledge and improve the motivation and interest in the topic or information (Chin & Kayalvizhi, 2005;Chin & Brown, 2000).
Especially science emphasizes curiosity, so questioning skills is an important component because it is included in the scientific method and scientific attitude. For instance, construct a problem, construct research questions, and ask a question about the found phenomenon. So, asking questions happens because there is a gap between students' knowledge and the students' new information (Cautinho & Almeida, 2014). Questioning skills are also closely related to students' cognitive. There is proof that lack of prior knowledge can affect their learning, lack of focus, and understanding, so it also affects questioning skills, but they do not know what they should ask (McQueen et al., 2014). Prior knowledge affects the question's skills while questioning skills is a strategy to improve students higher-order thinking skills because when students ask a question, they must be giving reasonable question which has an explanation to construct their higher-order thinking skills (Song, 2016;Bates et al., 2014;Papinczak et al., 2012).
In fact, according to preliminary observation to the several student's pre-service teachers in science learning, their willingness to ask a question is still low. A few students' pre-service teachers ask a question to the lecturer, and those question qualities are only technical questions, not reasoning. Lower-level questions include closed questions that consist of information and some technical questions (Bates et al., 2014). It can be said that students' reasoning still low (Herawati & Ardianto, 2017), and student question quality also low (Nurramadhani, 2019). Reasoning skills and question skills can lead the students to reflect on what they thought. This is usually happening in higher education students (Bottomley & Denny, 2011). Reasoning, critical thinking, and question skills also have relation and able to develop by reading practice (Collins et al., 2018;Gilson et al., 2014;Chin, 2001). However, reading habit that has been done by other researcher is only about reading passage with a particular topic that printed in paper (Collins et al., 2018;Gilson et al., 2014).
While they needed to adjust in the digital era, today's students are millennials or the Z generation who curious, practical, and related to digital. The current educational challenge is how to provide learning media and interesting learning materials (Açışlı et al., 2011;Warburton, 2003), but not to forget about hands-on activity that must be implemented to the students learning as one of the essences of science learning. Thus, it constructs E-Module (electronic module) to replace the module that printed before, becomes more interesting and easier to access everywhere and anytime suitable with the millennial and Z generations habits.
They will be interested in science learning, and then their higher-order thinking skills will develop critical thinking and question skills. The module consists of standard learning competencies and has a function to deepen these competencies (Daryanto, 2013). The module also exists in electronic form that includes multimedia interaction with technology and ICT to make navigation, showing a picture, audio, video, and animation easier and also there are vary quizzes with an interesting interface (Sugihartini & Jayanta, 2017;Nugraha et al., 2013;Voithofer, 2005).
There are so many approaches, methods, models, and learning material in education that can make the students interested in learning science. One of them is STEM (Science, Technology, Engineering, and Mathematics) which are all educators implement it in their classroom all around the world today. STEM itself is active learning that students centered integrated from one subject to the other. STEM also the innovative learning that leads to inquiry, project-based learning, and problem-solving (Breiner et al., 2012). STEM education is to motivate students to practice using integrated skills to solve problems and become meaningful learning in science (León et al., 2015;Moore et al., 2014;Wai et al., 2010), and also motivate students as innovator, logical thinker, problem solver, technologically literate and using technology in their life and relationship with their own culture (Stohlmann et al., 2012;Morrison, 2006). Learning with STEM approaches promotes hands-on activity that very important for students. Hands-on activities with contextual learning positively affect students' attitudes toward learning and content knowledge or cognition (Kaldi et al., 2011).
The tremendous results show that students who studied and took STEM learning in the class showed positive attitude and achievement, success in courses, and school (Domínguez & Jaime, 2010).
STEM in the form of approach, method, and model in learning science has already produced a positive action and development of students' skills and potential. On the other hand, there are still a few researchers to make STEM in teaching-learning material based on E-Module. Several former researchers have already developed about E-Module but in different aspects, such as 1) Problem Based Learning (PBL); 2) Learning Cycle 7E; 3) Laboratory guidelines on Biology (Serevina et al., 2018;Istuningsih et al., 2018;Refirman et al., 2017). But other researchers also haven't developed questioning skills optimally. This research provides the learning STEM-based E-module in science learning integrated with reading material and experimental activity, which promotes students asking questions so that students questioning skills are well developed.

Method
The method design that was used in this research to develop a STEM-based electronic module (E-module) was ADDIE (Analyze, Design, Develop, Implementation, and Evaluation) (Molenda, 2015). This research design could be adapted to develop an electronic module as teaching material that will be a learning product. This design has already done in five steps, for instance: a. Analyze In this first step, curricula and learning material were analyzed for each indicator in every aim in subtopics. Identify student's need for environmental pollution material for laboratory activity and students' question skills in science learning.

b. Design
The second steps were about designing all the components that include STEM-based E-Modul. For instance, making learning goals about environmental pollutions, identifying the learning material, and construct it with adjusting STEM inside the learning activity. Also, construct the activity inside the STEM-based E-Module, such as pojok ekspolrasi (the activity that asks students to do the exploration with a worksheet as guidance). pojok wawasan (the activity that asks students to answer all the multiple choices questions related to material that is given in every chapter), and pojok inquiry (the activity that asked students to make their own questions based on reading material provided in every chapter). All that components which was included inside in STEM-based E-Module were placed in the blueprint as guidance for developing the digital form. Besides preparing the learning material inside STEM-based E-Module, in this step also preparing the component about the validation sheet for expert judgment, as like content, language and legibility, graphic and presentation, and also STEM (Science Technology Engineering Mathematics). The other research instrument that was constructed, questionnaire for students.

c. Develop
The third step was the digitalization of STEM-based E-Module according to its blueprint that has been made in the previous step. The digitalization also including made link for students to put their answers and responses in all activities. STEM-based E-module could be read in .pdf form or with application in android or iOS system on your handphones such as playbook, moon+ reader, and lithium. Those were making it students easier to access E-Module and could read anytime and anywhere. After digitalization's process has been done, the expert gives their judgment of STEM-based E-module with content, language and legibility, graphic and presentation, and also STEM (Science Technology Engineering Mathematics) component. Then, test the E-Module to non-subject students after validation has been done. Revisions are made based on suggestions from experts and test results. The expert judgment validation was analyzed by the category based on (Supranto, 2000), that the class length is obtained 0.75. The category was explained below in Table 1. These fourth steps were all about implementing STEM-based E-module in learning science, which was done by the subject 26 students who take an environmental course in teacher and education faculty Universitas Pakuan, Bogor. They were divided into seven groups, and each group consists of 3-4 students. This implementation was done in three meetings with different topics that include environmental pollutions by using a qualitative descriptive method. Those meetings were soil pollution as activity 1, water pollution as activity 2, and air pollution as activity 3. The meetings conducted by virtual were observed, and students' question quality data were taken using video recording (free conference call application), chat transcriptions of discussion (WhatsApp application), and the results of students' responses from the pojok inquiry. The responses of students about learning activity using STEM-based E-module were recorded by questionnaire. The data is then analyzed by question quality category based on (Cautinho & Almeida, 2014). The category could be described as follow Table 2 below. Question that need explanation which help students get their concept, fact, phenomena, task, and procedure. Why…?
There is no direct and simple answer. How…?
Open Question (High Level)

Relationship
The purposes of the question are to comprehend the causes and consequences.
Which consequences...? Question has relationship with two or more concepts.
Which differences…? What happen…? Evaluation Students try to find the guide to make a decision or express their personal opinion.
Which is the best…? This question shows students' point of view, their choices, and judgement for a problem.
What is opinion…? What do we think about…? Finding and Solution The question purposes are to comprehend a complex problem.
The answer is related with identification or solution from a problem. if…then….

e. Evaluation
The last step is to evaluate the implementation of STEM based E-Module in science learning. Analyzed the strength and weakness of implementation and analyzed the effectiveness of STEM-based E-Module implementation towards questions quality of students' prospective teachers. The next steps were the final revisions of E-Modul.

Development STEM based E-Module
Based on the development design method used to construct STEM-based E-Module, ADDIE adapted from (Molenda 2015), the first steps are needed analysis about students question skills during a teaching-learning activity, especially in learning science. Based on the preliminary study about students' questioning skills in science learning, it shows that students' questioning skills are still in low-level quality (closed question) by using lecturing and laboratory activity (Nurramadhani, 2019) and reading assignment (Nurramadhani & Permana, 2020). So, pertaining to these results, in order to generated questions quality of students developed toward higher quality, there should be another teaching material that helps to learn activity which is integrated between reading, laboratory activity, science, technology, engineering, and mathematics, which those can make students higher-order thinking skills, include the critical thinking and questioning skills are improved.
Thus, based on the preliminary study results, there are notions of making teaching material that integrates and develops students' questioning skills. That notion is developing STEMbased E-Module. Before developing E-Module, it is needed to be designed first. It was suitable from the ADDIE method, the next step after analyzing designs. In this step, designing STEM-based E-Module are divided into several procedures, such as make a chapter based on the topic that will be given to the subject become three-chapter excluded the introduction.
Then, every chapter must include several activities that lead students can create critical thinking and curiosity. After that, they can construct their great questions. chapter, the layout, pojok eksplorasi (the activity that asks students to do the exploration and experiment with the worksheet as guidance), pojok wawasan (the activity that asks students to answer all the multiple choices questions related to material that is given in every chapter), and pojok inquiry (the activity that asks students to make their questions based on reading material that provided in every chapter). Besides blueprint, it also needed to design all research instruments and validation sheets for expert judgment.
The development process of digitalization of electronic module has the purpose of making students' motivation in learning and making a more comfortable to learn. His statement in line with Daghan (2017) that the students explained the benefits of technology or digitalization as facilitating communication, making their lives more comfortable, acquiring the information more manageable, and providing the sharing of information not only among them but also from others across the discipline. This electronic module can be read in .pdf form or Android or iOS system on handphones such as playbook, moon+ reader, and lithium. Students today are including in millennial or Z generation who attached with gadget and technology is around them. It was also suitable for this era, revolution industry 4.0, which needs technology integrating with science, mathematics, and engineering to solve the problem. Before this STEM-based E-Module is implemented to the subject, it is through validating expert judgment to make sure of its feasibility. The validation from STEM-based E-Modul is based on content, language and legibility, graphic and presentation, and STEM (Science Technology Engineering Mathematics) components. The results of STEM-based E-Module expert judgment validation are briefly explained in Table 3 below.  Table 3 provides that information the STEM-based E-Module is feasible to implement in science learning because all the component that has been scored by expert judgment reaches the very valid category with revisions in little aspects such as typing error or some layout for picture and illustration that has already revised. The cover and content view of STEM-based E-Module are shown in Figure 1 below.

STEM-based E-Module is an integrated teaching material that is constructed in electronic
form. All the topics and content are containing the integration of science, technology, engineering, and mathematics. STEM-based E-Module also provides the way to get used to asking a question for students by placing all the sub-topics with interrogative sentences. Not only the content and interrogative sentences but also the activity that is included in E-Module.
There are three acts in every chapter or topics, for instance, pojok eksplorasi (the activity that asks students to do the exploration with a worksheet as guidance), pojok wawasan (the activity that asks students to answer all the multiple choices questions related to material that is given in every chapter), and pojok inquiry (the activity that asks students to make their questions based on reading material that provided in every chapter) that has a link which students can fill their task, response, and answer to google form directly. There is an example of three activities in STEM-based E-Module, as shown in Figure 2. The next stages are the implementation of the STEM-based E-Module to the students. In this implementation, the learning activity is conducted in online learning because of pandemic COVID-19. So, before this implementation has begun, all the experimental apparatus and substances needed in this activity are sent to students' homes. A week before conducting the first meeting, students describe how to use the apparatus and substances and explain the application used in the learning. This learning is conducted in three meetings in three weeks.
Activity 1 about soil pollution, activity two about water pollution, and activity three about air pollution Every meeting used WhatsApp group chat and free conference call (FCC) application for learning. All the data from 7 groups of students are taken by making a transcript of students questioning in discussion sessions and presenting their experimental results. Even though the learning activity is conducted by online learning, students never lose their motivation to conduct their activity at home, so that when the meeting has begun, the discussion session was full of questions. The right procedure and good quality of experimental work by hands-on activity can promote the engagement, curiosity, and interest of students and develop a range of their skills especially higher-order thinking skills, science knowledge, and conceptual understanding (Kizilaslan et al., 2020). Those activities are divided and well distributed to each person, whether they are not conducting it together unless they share the concept as a group. After conducting the presentation on each topic, they fill the worksheet in pojok eksplorasi activity and done their reading test on pojok inquiry activity to put their questions based on what they read from the passage. From this activity, the data of student's questions quality are gained.
The last steps are evaluation from the implementation and input from the students based on the questionnaire. For example, they like STEM-based E-Module and experiment activity, but they do not like reading tests on it. They also ask to put more illustrations so that they are interested in reading. Other than to evaluate from this implementation are the experiment activity conducted from home is less effective rater than direct meeting at the classroom.
Direct meetings include the guided practice, self-reflection, sharing, and learning community directly with the emotions and full-body movement that affect the students' learning (Yeh, 2009).

Students' Generated Question Quality
The implementation of STEM-based E-Module to the students, which is divided into seven groups. Each group has done every single exploration activity for each topic. There are three topics and activity that has been done in three meetings based on pojok eksplorasi as guidance for students. All the student's questioning skills data are taken from two activities, and there are exploration activities from pojok eksplorasi activity and students' responses in pojok inquiry activity. The results of students generated questions quality are shown briefly in figure   3 below.  Pertaining from both data, Figure 3 and Figure 4 generally said that the generated questions quality of students improved after implementations of STEM-based E-Module in science learning. From both of the data above, the question quality category is analyzed between activity 1, which explains soil pollution, activity 2, which explains water pollution, and activity 3, which explains air pollution. The general result from both data shows that students' generated questions quality rose gradually from activity 1 until activity 3 during science learning by using STEM-based E-Module. It is indicated by the category results of percentages number such as information and understanding as closed questions decrease, while relationship, evaluation, and finding & solution as open questions increase. The factor that influences those results is the form of the module as the teaching material itself. E-Module is the electronic module that makes students get their motivation to learn because students in this era are a Z generation attached by their gadget and technology. Teenagers who include in millennial and Z generation are called a digital native's group, who have different expectations and behaviors toward using digital media; we can say that they are born with technology. Implementation of electronic books (e-books) such as E-module for learning and activity is needed, especially in a science subject, that younger students prefer reading ebooks for improving some skills that are important for them in everyday life (Wood et al., 2010;Woody et al., 2010;Liu & Huang, 2008). Besides that, the electronic form also can make learning easier both for students and teachers. They can open the module anywhere and anytime on their hand. Teachers can make a score for students easier than usual because all the data has already been put in digital. Students like to learn using STEM-based E-Modul from the responses of students because of the picture and easiness to learn.
Students' generated questions quality is developed generally, but it has a different trend between pojok inquiry and pojok eksplorasi activity results, specifically from activity 1 until activity 3. According to pojok inquiry activity results, activity 1 as soil pollution topics gets the greatest number of students asking questions' category is information (35%) and understanding ( Because pojok inquiry activity is all about reading the narration, students respond by their questioning skills. According to Teixeira-Dias et al. (2005), students will ask questions if they are engaged and interested in the topics they learned. If they are not interested, so they lack to ask questions. If they are lazy to read, so they are lack literacy, then leads to a lack of curiosity, after that, the quality of questions would not be improved. It means that the student's literacy in this research is still low, whereas literacy is essential for students.
Students' higher literacy skills, the more quality of students' higher-order thinking skills to decision-making, critical thinking, and inquiry are question skills (Mardliyah, 2019;Yuriza et al., 2018). Reading skill, reading interest could be affected by cognitive activity (Uusen & Müürsepp, 2012;Kintsch & Rawson, 2005). Most of the students include university students, have insufficient reading skills, they tend to skim during the reading (Huang et al., 2013;Uusen & Müürsepp, 2012).
Different from results on pojok inquiry activity, the results of pojok eksplorasi activity has improved gradually on students' questions quality category. Closed question quality are decreasing by the time activity begun from activity 1 (understanding -54%), activity 2 (information -15%), and activity 3 (information -8%), while open question quality increasing from activity 1 (finding & solution -0%), activity 2 (finding & solution -50%), and activity 3 (finding & solution -50%). It because students are motivated with hands on activity which promotes their curiosity. Implementation of hands-on experimental activity can make students improve their higher-order thinking skills, such as critical thinking skills and inquiry skills like questioning skills. STEM based E-module contains experimental activity in STEM-based will show their interest and improve their skills, for instance inquiry or questioning skills when they do something new, and they like it, such as experimental activity based on STEM learning and electronic module (Kennedy & Odell, 2014;Wai et al., 2010). In the laboratory activity integrated with science, students are using the scientific inquiry method and improve inquiry skills. Inquiry refers to scientists study the research, propose ideas, explain and justify the fact from investigation (Kaberman & Dori, 2009). It also supported by the integrated STEM material that combine all the four-science subject with mathematics, technology, and engineering together in combines that students must be created, thus it promotes student's problem-solving skills, critical thinking skills, and others higher; thus thinking skills. Students can also actively discuss in discussion forum about what higher-order in an experimental activity that is integrating with STEM. All of the theme of experimental activity is related and integrated in student's daily life. They also modified their experimental design according to their condition. The example of students' experimental design based on STEM, which students have done briefly, is shown in Figure 5 below. Figure 5. The students' experimental design from pojok eksplorasi Even though the students' question quality in the first activity remains in closed question quality, but in the next activity, the quality of their questions is improving. Because in the first time of exploration activity, students still adapted with the activity and learning methods, so the questions that they are asking us all about technical questions included in the information category as closed question quality. The novel finding is found during the research; that is, the students will pose the quality of their open questions if the lecturer also poses a good quality level of questions. The teacher questions quality might relate to students' questions quality when the teacher poses the questions about procedural and fact-based, which is included in closed questions quality, so the students do (Almeida, 2010). The lecturer poses the students' questions to encourage students to ask a question because sometimes students are ashamed to pose questions in front of the lecturer directly. In line with Becker's (2000) research, many students have stopped asking questions, and they do not express a desire to discover and debate because they are not confident. So, if this STEM-based E-Module would be implemented in the school, it must be having ascertained that teacher has already had a good question ability. The example of students' questions during pojok inquiry activity and pojok eksplorasi activity by using STEM-based E-Module in Table 4. Table 4. Example of students' questions quality from pojok inquiry activity and pojok eksplorasi activity by using STEM based e-module

Pojok Inquiry Activity Pojok Eksplorasi Activity
Closed Question (Low Level)

Information
What is shifting field activity? Is food coloring effect the plant stem become colorful? Understanding How is the oil washing tools works when it washing to maintain contaminate soil?
Why sprout changing they color become purple?
Open Question (High Level)

Relationship
What will happen if pollutant in the water and do not overcome properly in a long time?
What will happen if soil poured by vinegar, will the fungi life there?

Evaluation
Technology are invented and created with two blades, easiness human work, but have an affect to environment. Is there any possibility to crated the technology which can give an advantage for both human and environment and what kind of technology?
Container 3 there is no air circulation and with cricket inside, while container 2 there is air circulation and with cricket inside. Why cricket in container 2 died sooner than the other cricket? What is happen with the design?
Finding & Solution If the society and government has already give others counseling about land and its advantage, why there are many people still make soil damage?
I think that if I change cricket with butterfly, is it still the same for the endurance of butterfly with cricket? they are insect.

Conclusion
According to the results, it can be concluded that generally, students' generated questions quality is slightly developed, which means they have already attained the beginning of open question category as a higher level of questions. However, the close question category has the greatest number of questions which is asked by students. The student's questioning skills gradually improve in the open question's category, such as relationship, evaluation, and finding & solution. Thus, STEM-based E-Module can make students' questions quality develop, although still in a small amount. The questioning skills' quality of students will be better if the implementation of those activities already applied in this research becomes a habit. The questioning skills' quality will also improve dramatically if the lecturer also helps or scaffold the best technique of questioning to the students, which means that the teacher's lecturer should have the good ability to pose the questions.