Formative assessments are of paramo

Formative assessments are of paramount importance to gauge learning in many chemistry classrooms. Formative assessment is defined more by what one does with the results versus a specific form of assessment.1 In particular, formative assessments are primarily used to provide feedback to both teachers and learners.2 Summative assessments are primarily aimed toward evaluation of students’ learning, making them an assessment of learning as opposed to for learning.1 Evaluating the alignment of assessment items with teacher goals provides useful measurements for teachers to determine students’ understanding of concepts,2−5 particularly chemical phenomena. Without this alignment, inferences that teachers make from formative assessment data cannot inform their teaching in a valid manner.6,7 As an example, if a teacher determines that students do not have accurate mental particulate models of gas behavior (aligned with the kinetic molecular theory) by evaluating correctness of a quantitative Charles’ Law exercise, this teacher would be basing his/her conclusions on evidence (student responses to the exercise) that is not aligned with the learning goal. Furthermore, misaligned assessments lead to false positives (e.g., “students understand stoichiometry” when they do not) and false negatives (e.g., “students don’t understand relationships among gas law variables” when they do) that have a detrimental effect on teaching and learning chemistry.
The need for conceptual understanding of chemistry has been recognized by the Next Generation Science Standards, NGSS,8 which stresses the importance that students “...develop their understanding of the four core ideas in the physical sciences. These ideas include the most fundamental concepts from chemistry and physics....”8 Without a complete understanding of material, superficial understandings are sometimes evaluated using formative assessments and demonstrate that students have a level of understanding much higher than they actually do.9 In alignment with the NGSS, conceptualunderstanding is different than problem solving, which may be as simple as replacing a variable with a number.
Over the past years, instructional sensitivity has been highlighted as an integral criterion in determining the extent of alignment of assessments with their intended content goals.10 Instructional sensitivity examines the degree to which student performance and quality of instruction align.11 When teachers use items that are sensitive to their instruction, they can draw more valid conclusions about their instruction based on student performance data from the items.10 To date, no one method for evaluating instructional sensitivity has emerged as a best practice, but several authors provide broad guidelines for appropriate methods.12−14 Work on instructional sensitivity highlights the importance of evaluating the extent to which assessments and assessment items are able to inform instruction.
The alignment of assessment items and assessment purpose is one consideration in an existing assessment process known as data-driven inquiry (DDI).6,15−18 DDI is a process defined by four steps (italicized): Defining a goal for assessments, collecting evidence from students, analyzing data to make conclusions about teaching and learning, and finally taking pedagogical actions that address or support the conclusions made. The ways in which these steps are carried out are described in detail in a literature review.19 Additionally, Harshman and Yezierski argue the DDI literature does not consider disciplinary content. Coffey et al.8 make a similar claim in previous formative assessment research, including the aspect of aligning learning objectives with assessments has “focus[ed] attention to strategies and techniques...generally presumed traditional notions of disciplinary content as a body of information...with an emphasis on terminology....”19 Without full consideration to the rich disciplinary content of chemistry, it is not always possible to align assessments with the learning objectives.
Of notable exception to this lack of disciplinary substance, a study by Tomanek, Talanquer, and Novodvorsky researched the factors involved when secondary science teachers chose formative assessment tasks.20 In this study, in-service and preservice teachers were given three probes, each of which asked them to choose from a variety of assessment tasks and why they chose as they did. Interestingly, they found that both types of teachers used the factor “provides evidence of understanding/misunderstanding (makes knowledge/understanding/thinking visible)” infrequently in the selection of a task in all three probes. Similarly, the factor that specifically focused on alignment between the task, curriculum, teaching, and learning objectives was only observed in one of the three probes, indicating that alignment of these features was not readily observed. While the study by Tomanek et al.20 provides evidence that secondary science teachers do not always consider alignment using prompts that were largely biologyfocused, the degree to which chemistry teachers focus on alignment is still unaddressed.
We propose to focus on teachers as the unit of analysis (as opposed to students) as we investigate the effect of alignment of assessment items on teachers’ analyses of assessment data. In both cases of assessment types (formative and summative), teachers collect data that can be used to make data-driven conclusions and decisions. As such, our study is of relevance to both forms of assessment, although we place a heavy emphasis on the formative type, as it is more important to analyze data directly from classrooms and not restrict the tools to one type of assessment. The research question that guided this study was: To what level are the assessments of high school chemistry teachers instructionally sensitive (aligned with their learning goals)? We believe that this assessment process and considerations discussed will be of high interest and value to chemistry educators who wish to draw valid conclusions about student learning and instruction based on everyday assessment results.

Hasil (Bahasa Indonesia) 1:

Penilaian formatif adalah sangat penting untuk mengukur belajar di banyak kelas kimia. Formatif penilaian lebih didefinisikan oleh apa yang satu tidak dengan hasil versus bentuk tertentu assessment.1 khususnya, penilaian formatif terutama digunakan untuk memberikan umpan balik kepada kedua-dua guru dan learners.2 penilaian yang sumatif terutama ditujukan terhadap evaluasi pembelajaran siswa, membuat mereka penilaian belajar sebagai lawan untuk untuk mengevaluasi kesesuaian penilaian item dengan tujuan guru learning.1 menyediakan pengukuran yang berguna bagi para guru untuk menentukan siswa memahami konsep-konsep , Fenomena kimia terutama 2−5. Tanpa kesejajaran ini, kesimpulan yang membuat guru dari formatif penilaian data tidak dapat menginformasikan mereka mengajar di manner.6,7 berlaku sebagai contoh, jika seorang guru menentukan bahwa siswa tidak memiliki akurat partikulat model mental perilaku gas (selaras dengan teori kinetik molekul) dengan mengevaluasi kebenaran hukum Charles kuantitatif latihan, guru akan mendasarkan / kesimpulan pada bukti (mahasiswa tanggapan latihan) yang tidak selaras dengan tujuan belajar. Selain itu, penilaian sejajar mengarah ke positif palsu (misalnya, "siswa memahami stoikiometri" ketika mereka tidak) dan palsu negatif (misalnya, "siswa tidak mengerti hubungan antara variabel hukum gas" ketika mereka melakukan) yang memiliki efek yang merugikan pada mengajar dan belajar kimia.The need for conceptual understanding of chemistry has been recognized by the Next Generation Science Standards, NGSS,8 which stresses the importance that students “...develop their understanding of the four core ideas in the physical sciences. These ideas include the most fundamental concepts from chemistry and physics....”8 Without a complete understanding of material, superficial understandings are sometimes evaluated using formative assessments and demonstrate that students have a level of understanding much higher than they actually do.9 In alignment with the NGSS, conceptualunderstanding is different than problem solving, which may be as simple as replacing a variable with a number.Over the past years, instructional sensitivity has been highlighted as an integral criterion in determining the extent of alignment of assessments with their intended content goals.10 Instructional sensitivity examines the degree to which student performance and quality of instruction align.11 When teachers use items that are sensitive to their instruction, they can draw more valid conclusions about their instruction based on student performance data from the items.10 To date, no one method for evaluating instructional sensitivity has emerged as a best practice, but several authors provide broad guidelines for appropriate methods.12−14 Work on instructional sensitivity highlights the importance of evaluating the extent to which assessments and assessment items are able to inform instruction.The alignment of assessment items and assessment purpose is one consideration in an existing assessment process known as data-driven inquiry (DDI).6,15−18 DDI is a process defined by four steps (italicized): Defining a goal for assessments, collecting evidence from students, analyzing data to make conclusions about teaching and learning, and finally taking pedagogical actions that address or support the conclusions made. The ways in which these steps are carried out are described in detail in a literature review.19 Additionally, Harshman and Yezierski argue the DDI literature does not consider disciplinary content. Coffey et al.8 make a similar claim in previous formative assessment research, including the aspect of aligning learning objectives with assessments has “focus[ed] attention to strategies and techniques...generally presumed traditional notions of disciplinary content as a body of information...with an emphasis on terminology....”19 Without full consideration to the rich disciplinary content of chemistry, it is not always possible to align assessments with the learning objectives.Of notable exception to this lack of disciplinary substance, a study by Tomanek, Talanquer, and Novodvorsky researched the factors involved when secondary science teachers chose formative assessment tasks.20 In this study, in-service and preservice teachers were given three probes, each of which asked them to choose from a variety of assessment tasks and why they chose as they did. Interestingly, they found that both types of teachers used the factor “provides evidence of understanding/misunderstanding (makes knowledge/understanding/thinking visible)” infrequently in the selection of a task in all three probes. Similarly, the factor that specifically focused on alignment between the task, curriculum, teaching, and learning objectives was only observed in one of the three probes, indicating that alignment of these features was not readily observed. While the study by Tomanek et al.20 provides evidence that secondary science teachers do not always consider alignment using prompts that were largely biologyfocused, the degree to which chemistry teachers focus on alignment is still unaddressed.We propose to focus on teachers as the unit of analysis (as opposed to students) as we investigate the effect of alignment of assessment items on teachers’ analyses of assessment data. In both cases of assessment types (formative and summative), teachers collect data that can be used to make data-driven conclusions and decisions. As such, our study is of relevance to both forms of assessment, although we place a heavy emphasis on the formative type, as it is more important to analyze data directly from classrooms and not restrict the tools to one type of assessment. The research question that guided this study was: To what level are the assessments of high school chemistry teachers instructionally sensitive (aligned with their learning goals)? We believe that this assessment process and considerations discussed will be of high interest and value to chemistry educators who wish to draw valid conclusions about student learning and instruction based on everyday assessment results.

Hasil (Bahasa Indonesia) 2:

Penilaian formatif adalah sangat penting untuk mengukur belajar di banyak kelas kimia. Penilaian formatif didefinisikan lainnya apa yang dilakukan seseorang dengan hasil versus bentuk spesifik dari assessment.1 Secara khusus, penilaian formatif terutama digunakan untuk memberikan umpan balik bagi guru dan penilaian sumatif learners.2 terutama ditujukan terhadap evaluasi belajar siswa, membuat mereka penilaian pembelajaran sebagai lawan untuk learning.1 Mengevaluasi alignment item penilaian dengan tujuan guru memberikan pengukuran yang berguna bagi guru untuk menentukan pemahaman siswa tentang konsep-konsep, 2-5 fenomena khususnya kimia. Tanpa keselarasan ini, kesimpulan bahwa guru membuat dari data penilaian formatif tidak dapat menginformasikan mengajar mereka dalam manner.6,7 valid Sebagai contoh, jika seorang guru menentukan bahwa siswa tidak memiliki akurat model partikulat mental perilaku gas (selaras dengan kinetik molekul Teori) dengan mengevaluasi kebenaran dari Charles kuantitatif 'latihan Hukum, guru ini akan mendasarkan / nya kesimpulan nya pada bukti (tanggapan siswa untuk latihan) yang tidak selaras dengan tujuan pembelajaran. Selanjutnya, penilaian sejajar menyebabkan positif palsu (misalnya "siswa memahami stoikiometri" ketika mereka tidak) dan negatif palsu (misalnya, "siswa tidak memahami hubungan antara variabel hukum gas" ketika mereka melakukan) yang memiliki efek yang merugikan pada ajaran dan belajar kimia.
Kebutuhan pemahaman konseptual kimia telah diakui oleh Next Generation Ilmu Standar, NGSS, 8 yang menekankan pentingnya siswa "... mengembangkan pemahaman mereka tentang empat gagasan inti dalam ilmu fisika. Ide-ide ini meliputi konsep-konsep yang paling mendasar dari kimia dan fisika .... "8 Tanpa pemahaman lengkap tentang materi, pemahaman yang dangkal kadang-kadang dievaluasi menggunakan penilaian formatif dan menunjukkan bahwa siswa memiliki tingkat pemahaman yang lebih tinggi daripada mereka benar-benar do.9 Dalam keselarasan dengan NGSS, conceptualunderstanding berbeda dari pemecahan masalah, yang mungkin sederhana seperti mengganti variabel dengan nomor.
Selama beberapa tahun terakhir, sensitivitas instruksional telah disorot sebagai kriteria integral dalam menentukan sejauh mana keselarasan penilaian dengan dimaksudkan mereka konten goals.10 Instruksional sensitivitas meneliti sejauh mana kinerja siswa dan kualitas pengajaran align.11 Ketika guru menggunakan item yang sensitif terhadap instruksi mereka, mereka dapat menarik kesimpulan yang lebih valid tentang instruksi mereka berdasarkan data kinerja siswa dari items.10 yang Sampai saat ini, tidak ada satu metode untuk mengevaluasi sensitivitas instruksional telah muncul sebagai praktek terbaik, tetapi beberapa penulis memberikan panduan luas bagi methods.12-14 sesuai Bekerja pada sensitivitas instruksional menyoroti pentingnya mengevaluasi sejauh mana penilaian dan item penilaian yang mampu . menginformasikan instruksi
Penyelarasan item penilaian dan tujuan penilaian adalah salah satu pertimbangan dalam proses penilaian yang ada dikenal sebagai penyelidikan data-driven (DDI) .6,15-18 DDI adalah proses didefinisikan oleh empat langkah (dicetak miring): Mendefinisikan tujuan untuk penilaian, mengumpulkan bukti dari mahasiswa, menganalisis data untuk membuat kesimpulan tentang pengajaran dan pembelajaran, dan akhirnya mengambil tindakan pedagogis yang membahas atau mendukung kesimpulan dibuat. Cara-cara di mana langkah-langkah yang dilakukan dijelaskan secara rinci dalam review.19 literatur Selain itu, Harshman dan Yezierski berpendapat literatur DDI tidak mempertimbangkan konten disiplin. Coffey et al.8 membuat klaim serupa dalam penelitian penilaian formatif sebelumnya, termasuk aspek menyelaraskan tujuan pembelajaran dengan penilaian telah "fokus [ed] memperhatikan strategi dan teknik ... umumnya dianggap pemikiran tradisional tentang konten disiplin sebagai badan informasi ... dengan penekanan pada terminologi .... "19 Tanpa pertimbangan penuh dengan konten yang kaya disiplin kimia, itu tidak selalu mungkin untuk menyelaraskan penilaian dengan tujuan pembelajaran.
Dari pengecualian untuk kurangnya substansi disiplin, studi oleh Tomanek, Talanquer, dan Novodvorsky meneliti faktor-faktor yang terlibat ketika guru sains sekunder memilih penilaian formatif tasks.20 Dalam studi ini, in-service dan guru preservice diberi tiga probe, yang masing-masing meminta mereka untuk memilih dari berbagai tugas penilaian dan mengapa mereka memilih seperti yang mereka lakukan. Menariknya, mereka menemukan bahwa kedua jenis guru menggunakan faktor "memberikan bukti pemahaman / kesalahpahaman (membuat pengetahuan / pemahaman / berpikir terlihat)" jarang dalam pemilihan tugas di semua tiga probe. Demikian pula, faktor yang secara khusus berfokus pada keselarasan antara tugas, kurikulum, pengajaran, dan tujuan belajar hanya diamati pada salah satu dari tiga probe, yang menunjukkan bahwa keselarasan fitur ini tidak mudah diamati. Sementara studi oleh Tomanek et al.20 memberikan bukti bahwa guru ilmu sekunder tidak selalu mempertimbangkan keselarasan menggunakan petunjuk yang sebagian besar biologyfocused, sejauh mana guru kimia fokus pada keselarasan masih belum terselesaikan.
Kami mengusulkan untuk fokus pada guru sebagai unit analisis (sebagai lawan siswa) seperti yang kita menyelidiki pengaruh penyelarasan item penilaian terhadap analisis guru data penilaian. Dalam kedua kasus jenis penilaian (formatif dan sumatif), guru mengumpulkan data yang dapat digunakan untuk membuat kesimpulan dan keputusan berbasis data. Dengan demikian, penelitian kami adalah relevansi untuk kedua bentuk penilaian, meskipun kami menempatkan penekanan berat pada jenis formatif, karena lebih penting untuk menganalisis data secara langsung dari ruang kelas dan tidak membatasi alat untuk satu jenis penilaian. Pertanyaan penelitian yang dipandu penelitian ini adalah: Untuk tingkat apa adalah penilaian sekolah tinggi guru kimia instructionally sensitif (selaras dengan tujuan belajar mereka)? Kami percaya bahwa ini proses penilaian dan pertimbangan yang dibahas akan menarik tinggi dan nilai untuk pendidik kimia yang ingin menarik kesimpulan yang valid tentang belajar siswa dan pengajaran berdasarkan hasil penilaian sehari-hari.