class: left, bottom, title-slide .title[ # Module 3: Learning Analytics ] .subtitle[ ## EME6356: Learning & Web Analytics ] .author[ ### Dr. Bret Staudt Willet ] .date[ ### June 5, 2023 ] --- class: inverse, center, middle #
**View the slides:** [bretsw.com/eme6356-su23-module3](https://bretsw.com/eme6356-su23-module3) --- class: inverse, center, middle #
<br><br> Looking Back at Module 2 --- #
Data Doubles <img src="img/mirror.jpg" width="640px" style="display: block; margin: auto;" /> -- - How well can data capture our multifaceted selves? -- - Do data shape our identities? --- #
Double-Edged Sword <img src="img/sword.jpg" width="600px" style="display: block; margin: auto;" /> -- - How can data **help**?
alerts and notifications -- - How can data **hurt**?
obsessive checking/tracking --- #
What's the Point? <img src="img/crosswalk.jpg" width="600px" style="display: block; margin: auto;" /> -- - What’s the purpose of knowing how many emails one sends? -- - Do parents ever show data patterns to their kids? -- - Would this even help them self-regulate? --- #
Cautions <img src="img/sold.jpg" width="640px" style="display: block; margin: auto;" /> -- - Who **owns** the data? The interpretation? -- - **Bias** exists in human interpretation and is baked into data analytics --- class: inverse, center, middle #
<br><br> Module 2 <br> Final Thoughts? --- class: inverse, center, middle #
<br><br> Module 3: <br> Learning Analytics --- class: inverse, center, middle #
<br><br> **Defining Analytics** --- class: inverse, center, middle #
<br><br> **Defining Analytics** **Measure
Collect
Analyze
Report** --- class: inverse, center, middle #
<br><br> **Learning Analytics** **(Performance at School)** --- #
Performance at School <img src="img/classroom.jpg" width="600px" style="display: block; margin: auto;" /> Measure
Collect
Analyze
Report -- ###
**What might we measure?** --- #
Performance at School <img src="img/classroom.jpg" width="600px" style="display: block; margin: auto;" /> - How did student learning changed when doing school from home, if at all? -- - With whom do students interact in class discussions? -- - From whom do teachers seek professional advice? --- #
Performance at School <img src="img/3-ye-2022.png" width="100%" style="display: block; margin: auto;" /> <div class="caption"> <p>TechTrends article: "The History and Development of Learning Analytics in Learning, Design, & Technology Field" (<a href="https://doi.org/10.1007/s11528-022-00720-1" target="_blank">Ye, 2022</a>)</p> </div> --- class: inverse, center, middle #
<br><br> **Learning Analytics** <br><br> Example 1 --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** -- ```r edgelist1 <- read_csv("data/meteakca_edgelist.csv", show_col_types = FALSE) %>% mutate(group = week, week = substr(week, 1, 1) ) %>% group_by(from, to) %>% mutate(weight = n()) glimpse(edgelist1) ``` ``` ## Rows: 317 ## Columns: 5 ## Groups: from, to [182] ## $ week <chr> "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1"… ## $ from <chr> "AL", "VW", "RN", "CH", "KW", "KJ", "Kwi", "AM", "KW", "KW", "Z… ## $ to <chr> "RN", "RN", "KW", "RN", "CH", "CH", "CH", "KW", "AM", "AL", "AL… ## $ group <chr> "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1"… ## $ weight <int> 1, 1, 1, 1, 2, 3, 1, 2, 2, 2, 1, 2, 1, 1, 1, 3, 1, 2, 1, 1, 1, … ``` --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** ```r head(edgelist1, 15) ``` ``` ## # A tibble: 15 × 5 ## # Groups: from, to [15] ## week from to group weight ## <chr> <chr> <chr> <chr> <int> ## 1 1 AL RN 1 1 ## 2 1 VW RN 1 1 ## 3 1 RN KW 1 1 ## 4 1 CH RN 1 1 ## 5 1 KW CH 1 2 ## 6 1 KJ CH 1 3 ## 7 1 Kwi CH 1 1 ## 8 1 AM KW 1 2 ## 9 1 KW AM 1 2 ## 10 1 KW AL 1 2 ## 11 1 ZT AL 1 1 ## 12 1 LW AL 1 2 ## 13 1 RS AL 1 1 ## 14 1 SP AL 1 1 ## 15 1 JK AL 1 1 ``` -- ###
**What might we measure?** --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** ```r sociogram1 <- tidygraph::as_tbl_graph(edgelist1) %>% mutate(popularity = centrality_degree(mode = 'in')) %>% ggraph(layout = 'kk') + geom_edge_arc(alpha = .2, width = .5, strength = .5, color = 'steelblue' ) + geom_node_point(alpha = .4, aes(size = popularity)) + scale_size(range = c(1,10)) + theme_wsj() + scale_colour_wsj("colors6") + theme(axis.line=element_blank(), axis.text.x=element_blank(), axis.text.y=element_blank(), axis.ticks.x =element_blank(), axis.ticks.y =element_blank(), axis.title.x=element_blank(), axis.title.y=element_blank(), panel.background=element_blank(), panel.border=element_blank(), panel.grid.major=element_blank(), panel.grid.minor=element_blank()) ``` --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** <img src="img/3-sociogram1.png" width="100%" style="display: block; margin: auto;" /> -- ###
**Next step:** Look at the network by group or by week --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** There are quite a few descriptive measures of networks: -- - **Order:** number of nodes/vertices (students, in this case) -- - **Size:** number of edges/connections (responses, in this case) -- - **Reciprocity:** mutuality -- - **Transitivity:** clustering -- - **Diameter:** similar to degrees of separation -- - **Density:** out of all possible connections, percentage that have been made -- - **Node degree:** number of connections -- - **Sentiment score:** how positive or negative in aggregate -- - Character count, Word count, Length of threads --- #
Learning Analytics Example 1 **Online Class Discussion - Social Network Analysis** Analyze the network: -- ```r graph1 <- tidygraph::as_tbl_graph(edgelist1) ``` -- - **Order:** 23 -- - **Size:** 317 -- - **Reciprocity:** 0.5874587 -- - **Transitivity:** 0.6530769 -- - **Diameter:** 4 -- - **Density:** 0.6264822 -- - **Node degree:** 26 --- class: inverse, center, middle #
<br><br> **Learning Analytics** <br><br> Example 2 --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** <img src="img/3-article-cover.png" width="600px" style="display: block; margin: auto;" /> **Article:** [A social network perspective on peer supported learning in MOOCs for educators ](http://www.irrodl.org/index.php/irrodl/article/view/1852) (Kellogg, Booth, & Oliver, 2014) --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** <img src="img/3-data-cover.png" width="600px" style="display: block; margin: auto;" /> **Data source:** [Massively Open Online Course for Educators (MOOC-Ed) network dataset](https://dataverse.harvard.edu/dataset.xhtml;jsessionid=9ad052693563b29056a88d490182?persistentId=doi%3A10.7910%2FDVN%2FZZH3UB&version=&q=&fileTypeGroupFacet=&fileAccess=&fileSortField=name&fileSortOrder=desc)] --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** <img src="img/3-example2-rq.png" width="100%" style="display: block; margin: auto;" /> --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** ```r edgelist2 <- read_csv("data/DLT1 Edgelist.csv", show_col_types = FALSE) %>% group_by(Sender, Receiver) %>% mutate(Weight = n()) %>% ungroup() %>% relocate(Sender, Receiver, Weight) glimpse(edgelist2) ``` ``` ## Rows: 2,529 ## Columns: 11 ## $ Sender <dbl> 360, 356, 356, 344, 392, 219, 318, 4, 355, 355… ## $ Receiver <dbl> 444, 444, 444, 444, 444, 444, 444, 444, 356, 4… ## $ Weight <int> 1, 2, 2, 1, 1, 3, 2, 4, 1, 2, 4, 1, 1, 2, 1, 1… ## $ Timestamp <chr> "4/4/13 16:32", "4/4/13 18:45", "4/4/13 18:47"… ## $ `Discussion Title` <chr> "Most important change for your school or dist… ## $ `Discussion Category` <chr> "Group N", "Group D-L", "Group D-L", "Group O-… ## $ `Parent Category` <chr> "Units 1-3 Discussion Groups", "Units 1-3 Disc… ## $ `Category Text` <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA… ## $ `Discussion Identifier` <chr> "Most important change for your school or dist… ## $ `Comment ID` <dbl> 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15… ## $ `Discussion ID` <chr> "2", "1", "3", "4", "5", "6", "6", "2", "3", "… ``` --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** ```r head(edgelist2, 15) ``` ``` ## # A tibble: 15 × 11 ## Sender Receiver Weight Timestamp `Discussion Title` `Discussion Category` ## <dbl> <dbl> <int> <chr> <chr> <chr> ## 1 360 444 1 4/4/13 16:32 Most important cha… Group N ## 2 356 444 2 4/4/13 18:45 Most important cha… Group D-L ## 3 356 444 2 4/4/13 18:47 DLT Resources—Comm… Group D-L ## 4 344 444 1 4/4/13 18:55 Most important cha… Group O-T ## 5 392 444 1 4/4/13 19:13 Most important cha… Group U-Z ## 6 219 444 3 4/4/13 19:16 Most important cha… Group M ## 7 318 444 2 4/4/13 19:26 Most important cha… Group M ## 8 4 444 4 4/4/13 19:44 Most important cha… Group N ## 9 355 356 1 4/4/13 20:12 DLT Resources—Comm… Group D-L ## 10 355 444 2 4/4/13 20:13 Most important cha… Group D-L ## 11 4 444 4 4/4/13 20:14 DLT Resources—Comm… Group N ## 12 310 444 1 4/4/13 20:34 Submitting Questio… Unit 1 Expert Panel ## 13 248 444 1 4/4/13 20:42 Most important cha… Group D-L ## 14 150 444 2 4/4/13 21:42 Most important cha… Group O-T ## 15 19 310 1 4/4/13 23:13 How important is t… Group A-C ## # ℹ 5 more variables: `Parent Category` <chr>, `Category Text` <chr>, ## # `Discussion Identifier` <chr>, `Comment ID` <dbl>, `Discussion ID` <chr> ``` --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** Analyze the network: -- ```r graph2 <- tidygraph::as_tbl_graph(edgelist2) ``` -- - **Order:** 442 -- - **Size:** 2529 -- - **Reciprocity:** 0.1997544 -- - **Transitivity:** 0.0888077 -- - **Diameter:** 8 -- - **Density:** 0.0129744 -- - **Node degree:** 4 --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** ```r sociogram2 <- graph2 %>% mutate(popularity = centrality_degree(mode = 'in')) %>% ggraph(layout = 'kk') + geom_edge_arc(alpha = .2, width = .5, strength = .5, color = 'steelblue' ) + geom_node_point(alpha = .4, aes(size = popularity)) + scale_size(range = c(1,10)) + theme_wsj() + scale_colour_wsj("colors6") + theme(axis.line=element_blank(), axis.text.x=element_blank(), axis.text.y=element_blank(), axis.ticks.x =element_blank(), axis.ticks.y =element_blank(), axis.title.x=element_blank(), axis.title.y=element_blank(), panel.background=element_blank(), panel.border=element_blank(), panel.grid.major=element_blank(), panel.grid.minor=element_blank()) ``` --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** <img src="img/3-sociogram2.png" width="100%" style="display: block; margin: auto;" /> --- #
Learning Analytics Example 2 **Massive Online Open Course (MOOC) Discussion - Social Network Analysis** ```r dlt1_no <- read_csv("data/DLT1 Nodes.csv", show_col_types = FALSE) glimpse(dlt1_no) ``` ``` ## Rows: 445 ## Columns: 13 ## $ UID <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,… ## $ Facilitator <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0… ## $ role1 <chr> "libmedia", "classteaching", "districtadmin", "classteachi… ## $ experience <dbl> 1, 1, 2, 2, 3, 1, 2, 1, 1, 2, 3, 3, 2, 1, 3, 1, 1, 1, 3, 1… ## $ experience2 <chr> "6 to 10", "6 to 10", "11 to 20", "11 to 20", "20+", "4 to… ## $ grades <chr> "secondary", "secondary", "generalist", "middle", "general… ## $ location <chr> "VA", "FL", "PA", "NC", "AL", "AL", "SD", "BE", "NC", "NC"… ## $ region <chr> "South", "South", "Northeast", "South", "South", "South", … ## $ country <chr> "US", "US", "US", "US", "US", "US", "US", "BE", "US", "US"… ## $ group <chr> "UZ", "DL", "OT", "N", "AC", "AC", "OT", "AC", "N", "N", "… ## $ gender <chr> "female", "female", "female", "female", "female", "female"… ## $ expert <chr> "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"… ## $ connect <chr> "1", "0", "1", "0", "0", "1", "0", "0", "0", "0", "1", "0"… ``` -- ###
**Next step:** Infer network structure from participant and network attributes -- (try using **[brms](https://paul-buerkner.github.io/brms/)**: An R package for Bayesian multilevel models using Stan) --- class: inverse, center, middle #
<br><br> **Learning Analytics** <br><br> Comparing Examples 1 & 2 --- #
Comparing Examples 1 & 2 <img src="img/3-sociograms1-2.png" width="540px" style="display: block; margin: auto;" /> --- #
Comparing Examples 1 & 2 <table class="table table-striped table-bordered" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;"> Variable </th> <th style="text-align:center;"> Example 1 (Classroom) </th> <th style="text-align:center;"> Example 2 (MOOC) </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;"> Order </td> <td style="text-align:center;"> 23.0000000 </td> <td style="text-align:center;"> 442.0000000 </td> </tr> <tr> <td style="text-align:center;"> Size </td> <td style="text-align:center;"> 317.0000000 </td> <td style="text-align:center;"> 2529.0000000 </td> </tr> <tr> <td style="text-align:center;"> Reciprocity </td> <td style="text-align:center;"> 0.5874587 </td> <td style="text-align:center;"> 0.1997544 </td> </tr> <tr> <td style="text-align:center;"> Transitivity </td> <td style="text-align:center;"> 0.6530769 </td> <td style="text-align:center;"> 0.0888077 </td> </tr> <tr> <td style="text-align:center;"> Diameter </td> <td style="text-align:center;"> 4.0000000 </td> <td style="text-align:center;"> 8.0000000 </td> </tr> <tr> <td style="text-align:center;"> Density </td> <td style="text-align:center;"> 0.6264822 </td> <td style="text-align:center;"> 0.0129744 </td> </tr> <tr> <td style="text-align:center;"> Median Degree </td> <td style="text-align:center;"> 26.0000000 </td> <td style="text-align:center;"> 4.0000000 </td> </tr> </tbody> </table> --- class: inverse, center, middle #
<br><br> Looking ahead --- #
Semester schedule <img src="img/across-time.jpg" width="720px" style="display: block; margin: auto;" /> - **Module 1:** Introduction to Analytics - **Module 2*:** Performance Analytics - **Module 3: Learning Analytics** - **Module 4:** Web Analytics - **Module 5:** Data Visualization - **Module 6:** Ethics in Learning Analytics - **Module 7:** Future of Analytics - **Module 8:** Case Discussions --- #
Module structure <img src="img/workshop.jpg" width="480px" style="display: block; margin: auto;" /> -
Watch -
Explore -
Read -
Discuss -
Create --- #
Upcoming Assignments <img src="img/build.jpg" width="600px" style="display: block; margin: auto;" /> -- ### Discussion (50 points) - Initial post on Flip due by second Friday (but post as early as possible) - At least 3 response due on Flip by end of Module 3 (second Sunday night) --- #
Upcoming Assignments <img src="img/build.jpg" width="360px" style="display: block; margin: auto;" /> ### Analytics Problem Plan (100 points) - **due end of Module 3** - "Write a brief report (approximately 750-1000 words) identifying and describing a real-world problem that might be addressed via analytics." - "The emphasis of this assignment is on the conceptualization of the problem and the clear identification of a possible data source." - "Then briefly describe how analytics will be used to solve the problem" - "You will not need to collect actual data related to this problem, just plan for how it would be done." --- class: inverse, center, middle #
<br><br> Questions <hr> **What questions can I answer for you now?** **How can I support you this week?** <hr>
[bret.staudtwillet@fsu.edu](mailto:bret.staudtwillet@fsu.edu) |
[bretsw.com](https://bretsw.com) |
[GitHub](https://github.com/bretsw/) --- class: inverse, center, middle # Learn to Code <img src="img/dsieur.jpg" width="320px" style="display: block; margin: auto;" /> **https://datascienceineducation.com/** --- class: inverse, center, middle #
<br><br> Play in the <br> [Analytics Sandbox](https://bretsw.com/sandbox)
[GitHub repository for code and data](https://github.com/bretsw/sandbox)
[Datasets for practice](https://bretsw.com/post/datasets/)