class: left, bottom, title-slide .title[ # Module 4: Needs Assessment ] .subtitle[ ## EME5601: Introduction to Instructional Systems ] .author[ ### Dr. Bret Staudt Willet ] .date[ ### October 4, 2023 ] --- class: inverse, center, middle #
**View the slides:** [bretsw.com/eme5601-fs23-module4](https://bretsw.com/eme5601-fs23-module4) --- class: inverse, center, middle #
<br><br> Module 4 <br> Needs Assessment --- class: inverse, center, middle #
<br><br> Why Needs Assessment? --- #
Why Needs Assessment? ###
**A Climber's Survival Tale** <br> <iframe src="https://www.npr.org/player/embed/101420504/101420480" width="100%" height="240px" frameborder="0" title="NPR embedded audio player"></iframe> Source: [**NPR: Day to Day, March 4, 2009**](https://www.npr.org/2009/03/04/101420504/redux-a-climbers-survival-tale) --- class: inverse, center, middle #
<br><br> Conducting a Needs Assessment <br> (Ch. 2) --- class: inverse, center, middle #
<br><br> Gap = <br><br> [What is] - [What should be] --- #
Needs Assessment Purpose **Clear from the start:** What is the purpose of the needs assessment? <img src="img/4-assessing-needs.png" width="100%" style="display: block; margin: auto;" /> --- #
Needs Assessment Purpose Needs assessment is the **first step** in an iterative design cycle: -- <img src="img/Rothwell-fig2-1.png" width="480px" style="display: block; margin: auto;" /> <div class="caption"> Figure 2.1 from Rothwell et al. (2016, p. 20) <br><br> </div> --- #
Instructional Myths <img src="img/Rothwell-table2-1.png" width="560px" style="display: block; margin: auto;" /> <div class="caption"> Table 2.1 from Rothwell et al. (2016, p. 23) <br><br> </div> -- <hr> ###
Discuss in groups... --- #
Vocabulary of Needs -- - **Need:** performance gap -- - **Needs Assessment:** strategic level, tactical level (day-to-day operations), operational level -- - **Needs Analysis:** underlying causes -- - **Training Requirements Analysis:** If you know that training is the solution, why do a needs assessment? Which training is necessary? (p. 26) -- - **Needs Assessment Planning:** comprehensive or situation-specific -- - **Alignment:** between training assets and business goals -- - **Position Road Map:** audiences you serve, tasks they perform, outcomes of their jobs, desired behaviors, job constraints -- - **Needs Assessment Plan:** (like a research plan) objectives, target audience, sampling, data collection, instruments and protocols, data analysis, how decisions will be made based on the data --- #
Needs Assessment Steps <img src="img/4-assessing-needs.png" width="480px" style="display: block; margin: auto;" /> -- 1. Establishing objectives -- 2. Identifying the target audience: Whose instructional needs are to be addressed? -- 3. Specifying instruments and protocols -- 4. Assessing the feasibility of the needs assessment plan --- #
Ethics in Needs Assessment <img src="img/4-assessing-needs.png" width="480px" style="display: block; margin: auto;" /> ### "Can the needs assessment withstand charges it was not cost-effective, timely, or rigorous?" (p. 38) <br><br> -- <hr> ###
Discuss in groups... --- #
Needs Across Cultures <img src="img/subway.jpg" width="100%" style="display: block; margin: auto;" /> ### Are people on the train rude? --- class: inverse, center, middle #
<br><br> Collecting and Analyzing Data for Instructional Design Projects <br> (Ch. 3) --- #
Methods for Needs <img src="img/workshop.jpg" width="400px" style="display: block; margin: auto;" /> -- - **Data:** relevancy, reliability, validity (p. 42) -- - **Open process:** iterative, evolving based on feedback and revision -- - **Paradigm:** -- - Quantitative vs. qualitative approaches -- - if vs. how and why -- - worldviews and assumptions --- #
Quant vs. Qual <img src="img/Rothwell-table3-1.png" width="600px" style="display: block; margin: auto;" /> <div class="caption"> Table 3.1 from Rothwell et al. (2016, p. 44) <br><br> </div> --- #
Sampling <img src="img/spotlight.jpg" width="520px" style="display: block; margin: auto;" /> -- - Convenience or Judgmental sampling -- - Simple random sampling -- - Stratified sampling -- - Systematic sampling --- #
Data Collection <img src="img/sandbox.jpg" width="520px" style="display: block; margin: auto;" /> -- - Interviews -- - Direct observations -- - Indirect observations -- - Questionnaires --- #
Data Collection -- - Key Informant Groups or Focus groups -- - Delphi procedure -- - Critical incident method -- - Root Cause Analysis (beware bias: recency, availability, anchoring, confirmation) -- - Competency assessment -- - Assessment center -- - Exit interviews -- - Performance records -- - Work samples -- - Process mapping --- #
Data Collection - Unobtrusive measures (because many collection methods are reactive: inherently biased by their altering the situation being studied) -- - Data Analytics -- - Data mining --- #
Data Collection <img src="img/Rothwell-table3-3.png" width="360px" style="display: block; margin: auto;" /> <div class="caption"> Table 3.3 from Rothwell et al. (2016, p. 57) <br><br> </div> -- <hr> ###
Discuss in groups... --- #
Data Analysis -- - Historical -- - Descriptive (mean, mode, median) -- - Fancy math (sophisticated statistical techniques) -- - ANOVA -- - chi-square -- - t-test --- #
Key Questions on Data <img src="img/4-assessing-needs.png" width="480px" style="display: block; margin: auto;" /> -- 1. What are needs this project is attempting to satisfy? -- 2. What data or information will I need to clarify these needs? -- 3. How will I need to analyze this data to arrive at a reliable and valid conclusion? (p. 59) --- class: inverse, center, middle #
<br><br> Identifying the Target Population and Environmental Characteristics <br> (Ch. 4) --- #
Learner Assessment -- ### Who is the intended learner? -- ### What learner characteristics should be assessed? -- - **Situation**-related characteristics: -- - **Performance Analysis:** What are the possible relationships between the performance problem and the learner? -- - **Decision**-related characteristics --- #
Learner Assessment - **Learner**-related characteristics: -- - Prerequisite knowledge, skills, and attitudes -- - Physical traits -- - Demographic characteristics -- - Learner sensitivity -- - Physiological characteristics -- - Aptitudes (future capabilities to perform in certain ways) -- - Experience -- - ~~Learning styles~~ --- #
Learner Assessment - **Learner**-related characteristics: - Geographical location -- - Job category -- - Interests -- - Personal benefit to learning -- - Culture -- - Life cycle stage -- - Career stage and/or career prospects --- #
Learner Assessment - **Environmental** characteristics: -- - Physical -- - Social -- - Political -- - Cultural --- #
Learner Assessment **Learner Profiles:** normative, descriptive, historical -- <img src="img/Rothwell-exhibit4-1.png" width="360px" style="display: block; margin: auto;" /> <div class="caption"> Exhibit 4.1 from Rothwell et al. (2016, p. 72) <br><br> </div> --- #
Learner Assessment ###
Discuss in groups... <img src="img/Rothwell-exhibit4-1.png" width="360px" style="display: block; margin: auto;" /> <div class="caption"> Exhibit 4.1 from Rothwell et al. (2016, p. 72) <br><br> </div> --- #
Cultural Considerations <img src="img/Rothwell-table4-2.png" width="480px" style="display: block; margin: auto;" /> <div class="caption"> Table 4.2 from Rothwell et al. (2016, p. 77) <br><br> </div> -- ### Key question: Is the learner assessment free of bias and stereotyping? --- #
Setting Analysis -- - physical system -- - environment (developmental, delivery, application) -- - context -- - situation -- - **Whole Task Instructional Design** -- - **Universal Instructional Design:** inclusive rather than exclusive perspective -- - class climate -- - interaction -- - physical environments and products -- - delivery methods -- - assessment and feedback -- - accommodation --- #
Setting Analysis ### UID
Culturally Sustaining Instructional Design <img src="img/CSID-model.png" width="480px" style="display: block; margin: auto;" /> --- #
Developmental Assessment <img src="img/Rothwell-table4-3.png" width="420px" style="display: block; margin: auto;" /> <div class="caption"> Table 4.3 from Rothwell et al. (2016, p. 80) <br><br> </div> --- #
Setting Analysis -- ### Key questions: -- - Based on what is known about the organization, how many of these characteristics relate to the present assignment? -- - How are the characteristics relevant? What is known about how each characteristic effects on-the-job performance? -- - How should information about these characteristics subsequently be used in the instructional design process to improve the chances that learners will apply on the job what they learned during instruction? -- - How should this information influence subsequent steps in the instructional design model? (p. 87) --- class: inverse, center, middle #
<br><br> Using Analytical Techniques to Determine Instructional Content <br> (Ch. 5) --- #
Instructional Content -- - **Synthesize** (bring together) pieces of the puzzle in an efficient manner -- - **Ask**: Do the instructional products appear to meet the organization's needs? -- - **Sequence** the instruction: What should be addressed first? -- - Chronological -- - Topical -- - Whole-to-Part -- - Part-to-Whole -- - Know-to-Unknown -- - Unknown-to-Known -- - Step-by-Step -- - Part-to-Part-to-Part -- - General-to-Specific --- #
Instructional Content <img src="img/threads.png" width="720px" style="display: block; margin: auto;" /> -- - Analyze content through thematic analysis: -- - "goodness of fit" to meet performance standards --- class: inverse, center, middle #
<br><br> Expanding the <br> ISD Analysis Toolbox --- #
ISD Analysis Toolbox <img src="img/toolbench.jpg" width="450px" style="display: block; margin: auto;" /> -- - Systems Analysis - Needs Assessment / Needs Analysis - Training Requirements Analysis - Root Cause Analysis - Competency Assessment - Performance Analysis - Learner Assessment - Setting Analysis - Developmental Setting Assessment --- class: inverse, center, middle #
<br><br> Gap = <br><br> [What is] - [What should be] --- class: inverse, center, middle #
<br><br> Looking ahead --- #
Semester Schedule <img src="img/across-time.jpg" width="480px" style="display: block; margin: auto;" /> - **Module 1:** Introduction to Instructional Systems Design - **Module 2:** Systems Analysis - **Module 3:** ISD & HPT - **Module 4: Needs Assessment** - **Module 5:** Work Analysis - **Module 6:** Design & Development - **Module 7:** Evaluation --- #
Module Structure <img src="img/workshop.jpg" width="540px" style="display: block; margin: auto;" /> -
Optional Office Hours: **Mondays 3:30-4:00pm EST** -
Read -
Meet: **Wednesdays 3:05-5:40pm EST** -
Create --- #
Major Assignments <img src="img/build.jpg" width="320px" style="display: block; margin: auto;" /> - **Assignments** (70%) - Module 2 Assignment: Systems Analysis paper (150 points) - Module 3 Assignment: Annotated Bibliography 1 (50 points) - **Module 4 Assignment: Needs Assessment paper (150 points)** - Module 5 Assignment: Annotated Bibliography 2 (50 points) - Module 6 Assignment: Work Analysis paper (150 points) - Module 7 Assignment: ISD Process Model paper (150 points) --- class: inverse, center, middle #
<br><br> Questions <hr> **What questions can I answer for you now?** **How can I support you this week?** <hr>
[bret.staudtwillet@fsu.edu](mailto:bret.staudtwillet@fsu.edu) |
[bretsw.com](https://bretsw.com) |
[GitHub](https://github.com/bretsw/)