The Network Profile Summary: exploring network science through the lens of student motivation

The Network Profile Summary: exploring network science through the lens of student motivation Abstract Productive learning environments strike a balance between student motivation and the necessary learning outcomes associated with a particular course. This article explores one way to achieve such a balance and discusses some of the subtle benefits that spring from that balance. We designed a network science course that achieves course objectives and at the same time allows students to develop, test, and monitor a network of their own choosing. Rather than use canned networks, we introduce a set of assignments called the Network Profile Summary in which students continually apply newly learned course concepts to their selected networks and are therefore able to realize the utility and place of these concepts. This course design fosters student motivation and encourages independent learning. Under this pedagogical paradigm, students began to see coursework, concepts and feedback as productive and globally meaningful rather than corrective and locally meaningful. 1. Introduction In The Rhetorical Stance, Wayne Booth argues that the best writing, including the best student writing, is the result of a balance between three aspects: speaker, audience and subject. To Booth, an education in rhetoric moves far beyond the formulaic teaching of writing. In fact, he recalls that the most important lessons he learned as a writer were not a part of a formal, writing class, but rather were a collection of other experiences quite unconnected with a specific writing course [1]. These lessons were important, Booth continues, precisely because they were connected to moments ‘when I had something to say [1].’ It is this important part of education that we, as teachers, often overlook in our course design, our assignments, and our responses to student work. Booth’s argument is larger than a case for rhetorical education; it is a case for student agency, involvement, interest and motivation that transcends all subjects. It is perhaps the last of Booths three aspects, subject of good writing that becomes the most important in the following article. Students have something to say, in other words they have a stake in their own work, when the subject of that work means something to them. More importantly, as Booth shows, having a stake in their own work provides students experiences that last, that are meaningful, and that shape students future work and experience. Professors know (and research shows) that interested and motivated students are the best students. Not only do they make class interesting with their innovative insights but also they often try the hardest, are the highest achievers and earn the best grades [2]. This interest extends beyond the day-to-day classroom interactions to include the subject of the work itself. William Butler Yeats is attributed with the thought that education is a fire to be lit rather than a vessel to be filled. The more we reach our students and allow them to connect with ideas and subjects that are of interest to them, the greater chance we have of igniting and stoking their academic fire, facilitating the conflagration of life-time learning in academia and beyond. The goal of providing each individual student with the opportunity to engage with topics that mean something to them, while still exposing all students to required material, resulted in the introduction of Network Profile Summaries (NPS) to the Complex Networks course offered by the Applied Mathematics Department of the Naval Postgraduate School. These summaries, the result of a multidisciplinary approach to education, enable students to pick any network that piques their interest to serve as a network science playground throughout the duration of the course. From infrastructure to terrorism to Star Wars®, students developed a portfolio of network science concepts in the context of a topic that interests them. In the context of Booth’s formula for the best writing, the NPS gives students the opportunity to discover something they want to say. The professor provides this opportunity in two ways, both of which increase the student ‘buy-in’ that is increasingly important in generating motivation in today’s classrooms. First, the professor cedes some control of the course to students by allowing them to select the object of their network profile. Second, the relevance of course content become obvious to the student as it is applied in as many different contexts as there are students in the class. It is quite the opposite of canned textbook problems. This work describes the goals of the NPS, explains how they enrich the classroom environment and provides some indication of their efficacy in fostering student motivation and causing learning to occur in the complex networks classroom. 2. Defining and achieving educational goals 2.1 An overview of the course and its goals The Complex Networks course at the Naval Postgraduate School is a terminal course in a sequence that prepares graduate students to receive a certificate in network science, see [3]. As such, there is flexibility in the selection of the content, allowing the course to respond to, and focus on, the emerging advancements of this actively developing field. The course does not require any previous coursework and draws students pursuing their graduate degrees in a wide variety of fields including Operations Research, Mathematics, Computer Science and Electrical Engineering. These students often come into the course from a mixed set of backgrounds, and many have been out of academia for several years due to their military careers. Therefore, each student’s skill set varies in terms of their abilities and familiarity with proofs, modelling, writing and programming in R or Python. Additionally, each student has opted to enroll in this elective class with his or her own individual learning goals; each has professional and personal interests driving an academic interest in network science. Though the Complex Networks course itself has no prerequisites, those taking the course as part of the Network Science Academic Certificate will have previously taken a course in discrete mathematics and a course in either graph theory, network traffic analysis, network flows, or game theory. Conceptually, the course exposes students to a variety of real and synthetic network models as well as network metrics, including both the standard, or foundational, metrics and new metrics based on the recent literature. Students also learn the relevance of these metrics in telling the narratives that describe a particular network. In addition to addressing the course material, this class is specifically designed to develop and hone other transferable skills, such as: Connecting mathematical results to real world meaning; Comparing and contrasting results between different scenarios; Taking responsibility for learning; Developing curiosity and formulating good, investigable questions; Presenting complicated ideas through clear oral communication; Presenting complicated data sets through clear visualizations; Defending their own work; Creating a convincing, coherent and cohesive narrative based on data and analyses. The way this course approaches these goals, and another way to understand Booth’s ‘stake’ in critical learning, is through the MUSIC model of student motivation [4], which stresses: eMpowering students to learn; helping students identify the Usefulness of the material to their personal goals; instilling a belief in students that they can Succeed in mastering the material; promoting student Interest in the material; and creating a Caring learning environment. 2.2 The Network Profile Summary In order to achieve the course goals, the class is designed around a series of individual assignments which replace traditional homework. These assignments are called the NPS. For the NPS, each student develops a network of his or her choosing, analyzes it weekly as new metrics are introduced in the course, presents the weekly findings in the context of the chosen network as a single PowerPoint slide, and compiles all of these weekly reports into an end-of-course presentation that summarizes and highlights the key findings to tell the narrative of the network. To launch the course, during the first week, students are exposed to several real networks as well as synthetic network models. In addition to seeing how raw data is used to create those networks, these early examples motivate the study of the course material by demonstrating a wide range of applications that can be analyzed with the tools that are presented throughout the course. In the second week, each student leverages his or her new appreciation for network science to choose a situation, find corresponding data and use the data to create a network that is then analyzed throughout the course. Having students choose their own networks motivates students by first, providing them a stake in their work, and second by directly addressing several elements of the MUSIC model. Specifically, the freedom to choose is empowering, and students can choose something that is of personal or professional interest. Additionally, this approach demonstrates the usefulness of the course material in two ways. First, each student will see how network science is used to help them answer questions on his or her own network. Second, students also get to see how network science is used to analyze the diverse set of applications selected by their classmates. The success and caring aspects of the MUSIC motivation are also addressed as students select their networks. Students work with the instructor to get individualized feedback on the feasibility of their choices and finalize their selections, so that they are set up for success. Though the student is empowered to explore a network of his or her own interest, the professor is involved in the process of selection so the chosen network is appropriate in size and/or structure. For example, a student may be interested in the internet; however, running analyses on such a large network several times per week is computationally beyond the scope of this course. In this case, the professor may steer the student toward some subset of the internet, such as a block of IP addresses. As an additional support mechanism, the professor maintains a library of networks available for students to sample, should they be unable to choose a network appropriate for exploration throughout the course. After the networks are chosen and built, each week, as students learn about new metrics and analysis techniques in lecture, they apply their learning to their own networks. This approach allows students to immediately practice their new skills and ask the instructor about issues they may face in transitioning from theory to practice. Since students are analyzing their own networks, they are naturally interested in the results and recognize the usefulness of what they have learned. Furthermore, this approach promotes curiosity, and students, driven by a burning question that naturally arises from their chosen topic, occasionally go above and beyond, learning additional techniques to satisfy their curiosity. Each week, the students look at the results obtained by analyzing their NPS network and meaningfully interpret their mathematical answers in the context of their chosen applications. They then select the most important takeaways from their analysis and present these to the class as a single-slide PowerPoint presentation. This process elevates the level of learning in three ways. First, after simply running the analyses, each student must decide what each part of the analysis means in the context of their applications. Next, the student must synthesize the results into a cohesive narrative about the network. Lastly, he or she must determine how to choose the appropriate metrics and/or visualizations that will best convey the narrative to the class. They have to think through both data visualization and slide organization skills, since they often have a significant story to share and very limited space for that story. By orally presenting their slides to the class, students not only practice their public speaking skills but they also have the opportunity to practice the proper implementation of new terminology, increasing fluency in the language of network science. In alignment with teaching recommendations outlined in the GAIMME report, a national report on improving mathematics education through the use of modelling [5], the NPS experience is genuine. Specifically, unlike a typical course in which the instructor already knows the answers to the exercises, the student-based selection of models means that the whole class, including the instructor, is learning from each student presentation, further empowering each student by making their efforts integral to the learning of the whole community. Students are empowered to become the class’s leading expert on their own networks. Each week, during the NPS presentations, the class is exposed to a diverse set of networks, giving the students an opportunity to identify similarities and differences between those networks and resulting metrics; this promotes a broad understanding of the material despite the fact that each student only performed a single set of calculations. The presentations often lead to student-led discussions, as the students feel a bit more comfortable posing questions to other learners than to an instructor. Occasionally, students will challenge their peers’ findings or interpretations, resulting in rich conversations filled with genuine discoveries. The interactive nature of the course helps promote an atmosphere in which the students dare to ask interesting questions and be creative in their thinking, absorbing the feedback from the instructor and their peers. At the end of the course, the students take their set of single-slide NPS presentations and organize them into a final cumulative presentation. Students are able to leverage the feedback they received from the instructor and their peers throughout the semester to make improvements on their slides, ultimately presenting a cohesive narrative describing their network analysis. 3. Assessment Although increasing student interest and motivation are key objectives of the NPS, the pedagogical model accomplishes much more. By centring classroom work on the interests of individual students, a shift occurs, not only in participation and quality of work but also in the assessment paradigm. Coursework still earns a grade, but both students and professors reconceptualize feedback. Rather than justifying the grade, feedback becomes constructive. When students have a legitimate stake in their work, they cannot help but see comments as collaborative rather than corrective. The conversations about the NPS assignments do not revolve around right or wrong. Instead, peers and faculty interact with the ideas and concepts that are discussed during the presentations and provide useful and meaningful feedback. It is in this interaction, in this collaborative environment, that true learning occurs. The inevitable shift in the feedback and assessment paradigm, then, contributes to the overall quality of the educational experience. This paradigm shift also contributes to the development of the whole person. We use assessments and the associated feedback as means to mentor future members of the discipline, to help them mature both technically and academically. In contrast, the feedback given in a more traditional assessment paradigm is seen as corrective; this corrective environment hampers the mentoring relationship. A canned problem set or scenario fosters an environment of corrective assessment, as it always has an approved solution, and students cannot help but attempt to measure their solution against this perceived ideal. When, however, they are working on something original that interests them, they are denied both the comfort of an approved solution as well as the pressure of measuring up to that ideal. Instead of perceiving comments as a corrective comparison between the ideal and their work, they begin to recognize feedback as opportunities to improve their own work. They have something to say, and the feedback they receive is a means of saying it better. In practice, the course approach to assessment is holistic and provides a formative assessment of individual assignments as well as a summative assessment at course completion. This assessment practice is adapted from those used by the United States Military Academys Department of Mathematical Sciences and the Department of English and Philosophy. This section includes discussion of the theory behind this holistic model of grading, followed by an example rubric used during the assessment of presentations given during the course’s Centrality Week, the week during which various centrality measures are taught and then applied to the NPS networks. The grade for the course is split between the NPS ($$40\%$$ of the final grade), exploratory research as a group paper to be submitted to a conference ($$40\%$$ of the final grade), and in class participation ($$20\%$$ of the final grade). The NPS provides a relatively simple homework alternative to assess student work: evaluate the presentations weekly, provide feedback and assign grades. For this evaluation, we look at both content and delivery of message using the criteria for strong slides and presentations in Table 1. Table 1 Assessment rubric for slides and presentations Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Table 1 Assessment rubric for slides and presentations Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit 3.1 Guiding principles A student that presents work worthy of an A demonstrates domination over the material presented. This student demonstrates a sophisticated understanding of the material and adheres to appropriate mathematical and grammatical conventions. The student shows superior control of insights and assertions derived from the analysis of one or more appropriately applied mathematical techniques. Presentations that earn an A make sound associations among concepts, weaving those logical connections into intelligent and sometimes highly insightful synthesis and argumentation when appropriate, all while managing the space limitations of a single presentation slide. Evidence is well chosen, deftly integrated and properly cited. In terms of organization, ideas are carefully ordered and structured with transitions that move the analysis in evident directions. The author makes skilful use of data visualization to enhance reader understanding of analysis and conclusions. Few submissions are perfect, so minor errors in grammatical correctness may occur. Students that earn the mark of B demonstrate good understanding of the material. These students comprehend and address the material while adhering to appropriate mathematical and grammatical conventions. The presentation skilfully controls insights and assertions derived from the analysis of one or more appropriately applied mathematical techniques. Presentations earning a B make sound associations among concepts, in the service of exposing results. Treatment of individual concepts and ideas is clear and thorough. Evidence is well chosen, properly cited and usually effectively integrated. The presentation may attempt to synthesize; however, clear and coherent synthesis is hampered by control of subordinate concepts. In terms of organization, the slide structure may feel formulaic, instead of being optimized to clearly convey the specific narrative. The author makes good use of data visualization approaches to support reader understanding of analysis and conclusions. There are some grammatical errors in correctness that will not impede comprehension. We assess work that is passing and demonstrating short range proficiency at the grade of C. Such work addresses the material and demonstrates awareness of the concepts but may not clearly convey their knowledge to the audience. Tone can be somewhat misguided, though not usually inappropriate. The presentation controls at least one concept and/or assertion and presents a generally clear and accurate discussion thereof. Presentations earning a C seldom reach beyond the level of the primary mathematical concept to make associations or connections with related concepts, nor do they move toward synthesis. Insights and results may be simplistic or demonstrate limited flaws in logic. Evidence is generally well chosen but might not be clearly explained or effectively integrated. The organization of the slide may be ineffective. Figures may be present but are not leveraged in a manner that effectively supports reader understanding of analysis and conclusions. Some grammatical errors noted in the realm of correctness could cause minor or temporary difficulty with comprehension. Students may earn a D with a marginal performance that is passing. This work demonstrates a slight grasp of one idea or concept, but limited ability to control or explain that concept effectively. These presentations might adequately communicate a part of the idea in a clear thought or two, but they fail to produce coherent and logically connected presentation. Choice of evidence is usually confusing or flawed. Tone can be inappropriate at times. Several spoken sentences may be poorly or incorrectly structured, wordy and/or vague. Confusion regarding the audience and context could be apparent. A borderline presentation passes when it contains a clearly articulated instance of comprehension, as long as correctness errors are not rampant. Failing marks may be assigned at two levels. A 50% F, or high F, is a presentation that contains some mathematical content, but definitely fails to demonstrate understanding of the requirements. A 25% F, or low F shows some attempt at fulfilling the requirements, but provides little relevant content that indicates a definite failure to demonstrate understanding of the requirement. Students that fail to submit the assignment are awarded a 0. Five percent of the overall grade is available for award upon demonstration of elements seen in higher marks. For example, if a student’s presentation mostly aligns with the grade of C, but demonstrates some elements worthy of a B, the student is awarded a C+. A student that presents A-level work and includes work or insights that are beyond the expectations of the course, that student may earn an A+. 3.2 Sample rubric Each week in the course covers a related set of topics. During centrality week, the instructor introduces the broad topic of network centrality and presents the following centrality measures: degree, closeness, eigenvector, Katz, PageRank and betweenness. In addition to the in-class instruction, we assign the students a set of training exercises so that they can practice computing centralities on small networks. Leveraging their instruction in centralities, the students then perform centrality analyses on their selected networks, synthesize the results into a coherent narrative about their network in the context of their application and prepare a single slide for the NPS. Table 1 is a rubric for the centrality NPS, allowing the instructor to check off the topics that are addressed in the presentation as well as quickly assess the overall quality of the work. There is some space in the table for the instructor to give more general comments about the selection of the metrics, suggestions to refine the narrative itself and techniques to enhance the presentation of the narrative. 4. Examples of NPS slides In this section, we present examples of slides from the NPS of different students that were part of the $$2016$$ cohort. 4.1 Using existing networks for NPS Figure 1 shows two slides from the NPS of Ben McCaleb. The slide on the left introduces his NPS network he will analyze throughout the quarter. The slide on the right presents his work on the degree distribution in which he tries to fit a curve through the degree distribution; in this case his network appears to be a very modular network with a modularity of $$0.932$$. Fig. 1. View largeDownload slide The West Coast Power Grid introductory slide and degree distribution slide (by Ben McCaleb). Fig. 1. View largeDownload slide The West Coast Power Grid introductory slide and degree distribution slide (by Ben McCaleb). Figure 2 shows two slides from the NPS of Richard Allain. The left slide presents an overview of the Disease and Disorders network (top right) along with then two subnetworks: human disease network (nodes are diseases and edges are the genes connecting them), and disease gene network (nodes are the genes and the edges are the deceases they depicted in pairs). Through the presentation and feedback process, the student learned that this would be better presented using hypergraphs. Later in the course, he then presented the slide on the right illustrating the degree distribution and community information for his network. Fig. 2. View largeDownload slide Introductory slide to disease and disorders, and degree distribution slide (by Richard Allain). Fig. 2. View largeDownload slide Introductory slide to disease and disorders, and degree distribution slide (by Richard Allain). Figure 3 includes the introductory slide (left) and the centrality slide (right) for the European Union Airlines network [6], a multilayered network that Brian Crawford used for his NPS. Fig. 3. View largeDownload slide Introductory slide and the centrality slide for European Union Airlines network (by Brian Crawford). Fig. 3. View largeDownload slide Introductory slide and the centrality slide for European Union Airlines network (by Brian Crawford). 4.2 Creating new networks for NPS Several students used available data to create their own networks: capturing new data as political events happened, using their military background and knowledge about military distribution equipment, watching movies to create an interaction between characters, screening their hard drives and emails to create social networks, and so on. The fact that even a handful of students took this approach supports the authors’ belief that students would like to construct and pursue an analysis of information about which they are passionate, if given the chance. Moreover, students were encouraged to bring to class any data that they might consider analyzing for their thesis, and several students have done so [7–9]. Tom Knuth used www.netlytic.org [10] to capture Twitter data that was relevant to the Trump presidential campaign to create the ‘Trump Talk’ network shown in Fig. 4. His overview slide (left) presents the motivation, and the slide on the right presents some of the network’s metrics. Fig. 4. View largeDownload slide Introductory slide and network statistics of the ‘Trump Talk’ network (captured by Tom Knuth using www.netlytic.org). Fig. 4. View largeDownload slide Introductory slide and network statistics of the ‘Trump Talk’ network (captured by Tom Knuth using www.netlytic.org). Zack Luckens created the Star Wars network shown in Fig. 5. He captured the Star Wars social network by re-watching the episodes and adding edges between two characters if they talked to each other, fought together/against each other, were relatives, or worked together. At that time the website http://moviegalaxies.com/ [11] did not have the network for Star Wars, but now it contains a different version of this network. Fig. 5. View largeDownload slide Introductory slide and centralities in the Star Wars IV, V and WI episodes (captured by Zac Lukens by watching character interactions in Star Wars). Fig. 5. View largeDownload slide Introductory slide and centralities in the Star Wars IV, V and WI episodes (captured by Zac Lukens by watching character interactions in Star Wars). In Fig. 6, Ryan Miller introduces the Noordin Top network, data collected by NPS students under supervision of Everton and Roberts [12]. As this network had several attributes associated with its nodes, he created layers for the networks turning it into a multilayered network. This multilayered information is now part of his thesis [8] and served as a basis for a couple of publications co-authored by this student [13, 14]. Fig. 6. View largeDownload slide Introductory slide of the Noordin Top Network, and the layer visualization (by Ryan Miller). Fig. 6. View largeDownload slide Introductory slide of the Noordin Top Network, and the layer visualization (by Ryan Miller). Greg Allen created his hard drive social network, shown in Fig. 7, using the forensic bulk extractor [15], a digital fingerprinting tool. He used this analysis as a seed for the analysis in his thesis [7], which he analyzed several people’s hard drives by categorizing the components of the networks according to the usefulness in identifying the owner of the hard drive; this work also required the use of machine learning featuring metrics he learned in this course. Fig. 7. View largeDownload slide Greg Allen created his hard drive social network using the forensic bulk extractor [15], a digital fingerprinting tool. Fig. 7. View largeDownload slide Greg Allen created his hard drive social network using the forensic bulk extractor [15], a digital fingerprinting tool. Miguel Miranda Lopez created a network of user accounts screening his email from September through December $$2016$$. The data was uploaded and turned into a network using www.netlytic.org [10]. The network is shown in Fig. 8. This also inspired him to further this analysis in his thesis on ‘Classifying Cyber Targets in Email Networks from Digital Storage Media’. Fig. 8. View largeDownload slide Miguel Miranda Lopez created use accounts network using his email data and www.netlytic.org.. Fig. 8. View largeDownload slide Miguel Miranda Lopez created use accounts network using his email data and www.netlytic.org.. The aforementioned networks are just some examples of the types of work submitted by students, and the diversity of this limited set illustrates the broad applicability of network science for the students in the class. Additionally, given that some students used their NPS as a seed for their thesis supports the case that the NPS increases student motivation, as these students are going above and beyond as they follow the questions triggered by their work in the course. 4.3 Possible concerns and encouragement While the student-centred, motivation-based design of the course may be attractive, it is only natural that potential instructors have concerns about the implementation. We have anticipated several of these concerns and briefly addressed them. What if my students cannot find a suitable network for their NPS? Many of the students in this course arrived with ideas and were eager to choose their own network, but it was helpful to have several living networks available to offer as suggestions for students who were not prepared to select their own network. Since these networks are still adapting, the student’s work is still new and can provide valuable insights, unlike a static artificial network generated for the purposes of completing an assignment. Several sample networks, as well as a syllabus for the course, are available at http://faculty.nps.edu/rgera/MA4404/NetworkProfileSummaryResources.html What if there is resistance to learning another software tool? This can be an issue in many mathematics and science courses, including network science courses. However, learning the software is a clearly stated goal and is considered part of the course content, so there is a pre-existing expectation to learn the software. Student buy-in can be more easily achieved by clearly communicating this objective and demonstrating the value of the software, especially as it relates to helping them answer their own questions about the network that they have selected for the analysis. If every student is turning in a project on a completely different topic, will the grading be really difficult and/or time-consuming? While each submission is unique, it is the responsibility of the student to make a strong case both to the instructor and to their classmates. The assessment process is partially outsourced to the class, as they raise questions and concerns about the work of their peers. There is no obligation on the part of the instructor to regenerate the work and confirm the answer. The grading burden is further lightened through the development and implementation of a simple, generic rubric which can be shared with the students before they submit their work to clearly convey the instructor’s expectations. Additionally, the students receive actionable formative feedback throughout the semester, thereby improving the quality of their final submission and its alignment with the rubric. What if I try this approach, but my students are not receptive? Change can be difficult. Unlike a typical course, students must shoulder a larger burden for their learning, and they cannot simply compare and check their homework answers with one another. As with any new pedagogical approach, students are more likely to adapt positively when the instructor sets the tone early in the course, clearly communicates the expectations and demonstrates that he or she cares about the student success. 5. Conclusion Wayne Booth tells a story of a bright, young graduate student whose essays were particularly awful. His writing was awful, Booth suggests, because in response to canned, formulaic assignments, he found nothing worth saying. When, on the other hand, he made an impassioned argument criticizing Booths own interpretation of a text, Booth says he wrote a four page polemic, unpretentious, stimulating, organized and convincing [1]. It was only when this student had something to say and someone to say it to that his writing began to develop. The core belief that inspired the NPS is an extension of the discovery Booth made all those years ago in a graduate English seminar. That is, students analyze, evaluate and create (and do these well) if they are allowed to choose and develop networks that interest them personally rather than respond to a choice that has already been made for them. The methodology introduced here replaces the standard homework assignments with an exploratory and creative medium for understanding network science concepts. Each week, the students are presented with new concepts. Then they explore these concepts on their individual networks rather than solving problems similar to the ones worked in the book or classroom. Students further synthesize the results obtained into convincing, coherent and cohesive stories as they create their weekly power point slide. This process culminates with the summary slide that creates a profile for the studied network. By allowing the students to practice on their personal network, we empowered them to personalize their project and take responsibility for their own learning. Since there is no pre-determined answer for these networks, students rely on critical reasoning and connections to the real world to present and defend their observations to the class. The classroom discussions during students’ presentations create a genuine graduate-level learning environment. The collective feedback naturally invites students to compare and contrast the weekly learned network science concepts on different networks. Seeing several different results, students bring new questions that the facilitator might not have considered. These discussions often plant seeds for new research creation, which is the reason several students were inspired by NPS for their theses and several of these networks were used for theses [7–9] and class group publications [13, 14, 16–19]. Finally, it is obvious that students are more enthusiastic about the class. One student remarked on the student opinion form that ‘I enjoyed exploring the course concepts by implementing them on my network profile. Great way to learn!’ While another mentioned that ‘This was an engaging and interactive course...improved my understanding of the material’. Moreover, the professor has a blast teaching and facilitating, as the networks presented and the discussions provide variety in teaching. Acknowledgements The authors would like to thank the DoD for partially sponsoring this project. The authors are grateful to the referees for their valuable input on this article. References 1. Booth, W. C. ( 1963 ) The rhetorical stance. Coll. Compos. Commun ., 14 , 139 – 145 . Google Scholar CrossRef Search ADS 2. Michaels, J. W. & Miethe, T. D. ( 1989 ) Academic effort and college grades. Soc. Forces , 68 , 309 . Google Scholar CrossRef Search ADS 3. Gera, R. ( 2017 ) Leading edge learning of network science. (in press) . 4. Jones, B. ( 2009 ) Motivating students to engage in learning: the MUSIC model of academic motivation. Int. J. Teach. Learn. High. Educ. , 21 , 272 – 285 . 5. Bliss, K., Fowler, K., Galluzzo, B., Garfunkel, S., Giordano, F., Godbold, L., Gould, H., Levy, R., Libertini, J., Long, M., Malkevitch, J., Montgomery, M., Pollak, H., Teague, D., van der Kooij, H. & Zbiek, R. ( 2016 ) Guidelines for Assessment and Instruction in Mathematical Modeling Education (GAIMME Report) . http://www.siam.org/reports/gaimme.php Cited 05 June 2017 . 6. De Domenico, M., Porter, M. A. & Arenas, A. ( 2014 ) Muxviz: a tool for multilayer analysis and visualization of networks. J. Complex Netw. , 3 . 7. Allen, G. ( 2016 ) Constructing and classifying email networks from raw forensic images. Master Thesis , Naval Postgraduate School, Monterey, CA . 8. Miller, R. ( 2016 ) Purpose-driven communities in multiplex networks: thresholding user-engaged layer aggregation. Master’s Thesis , Naval Postgraduate School, Monterey, CA . 9. Warnke, S. ( 2016 ) Partial information community detection in a multilayered network. Master’s Thesis , Naval Postgraduate School, Monterey, CA . 10. Gruzd, A. ( 2016 ) Netlytic: Software for Automated Text and Social Network Analysis . http://netlytic.org. [Software Tool] . 11. Kaminski, J., Schober, M., Albaladejo, R., Zastupailo, O. & Hidalgo, C. ( 2012 ) Moviegalaxies-Social Networks in Movies . Consulté sur http://moviegalaxies.com. 12. Roberts, N. & Everton., S. F. ( 2011 ) Terrorist Data: Noordin Top Terrorist Network . https://sites.google.com/site/sfeverton18/research/appendix-1. 13. Crawford, B., Gera, R., Miller, R. & Shrestha, B. ( 2016 ) Community evolution in multiplex layer aggregation. Advances in Social Networks Analysis and Mining (ASONAM), IEEE/ACM International Conference . pp. 1229 – 1237 . http://ieeexplore.ieee.org/abstract/document/7752395/. 14. Gera, R., Miller, R., MirandaLopez, M., Saxena, A. & Warnke, S. ( 2017 ) Three is the answer: combining relationships to analyze multilayered terrorist networks. Advances in Social Networks Analysis and Mining (ASONAM), IEEE/ACM , (in press) . 15. Garfinkel, S. L. ( 2013 ) Digital media triage with bulk data analysis and bulk_extractor. Comput. Secur. 32 , 56 – 72 . Google Scholar CrossRef Search ADS 16. Allain, R., Gera, R., Hall, R. & Raffetto, M. ( 2016 ) Modeling network community evolution in YouTube comment posting. International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation (BRIMS) . http://sbp-brims.org/2016/proceedings/LB_115.pdf. 17. Berest, M., Gera, R., Lukens, Z., Martinez, N., & McCaleb, B. ( 2016 ) Predicting network evolution through temporal Twitter snapshots for paris attacks of 2015. International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation (BRIMS) . http://sbp-brims.org/2016/proceedings/LB_111.pdf. 18. Chen, S., Debnath, J, Gera, R., Greunke, B., Sharpe, N. & Warnke, S. ( 2017 ) Discovering community structure using network sampling. The 32nd ISCA International Conference on Computers and Their Applications (CATA) . 19. Crawford, B., Gera, R., House, J., Knuth, T. & Miller, R. ( 2016 ) Graph structure similarity using spectral theory. Springer International Publishing AG 2017 ( Cherifi H. et al. eds), International Workshop on Complex Networks and their Applications, Studies in Computational Intelligence , vol. 693 , Cham : Springer , pp. 209 – 221 . Google Scholar CrossRef Search ADS Published by Oxford University Press 2017. This work is written by US Government employees and is in the public domain in the US. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Complex Networks Oxford University Press

The Network Profile Summary: exploring network science through the lens of student motivation

Loading next page...
 
/lp/ou_press/the-network-profile-summary-exploring-network-science-through-the-lens-yzMSzrcxli
Publisher
Oxford University Press
Copyright
Published by Oxford University Press 2017. This work is written by US Government employees and is in the public domain in the US.
ISSN
2051-1310
eISSN
2051-1329
D.O.I.
10.1093/comnet/cnx040
Publisher site
See Article on Publisher Site

Abstract

Abstract Productive learning environments strike a balance between student motivation and the necessary learning outcomes associated with a particular course. This article explores one way to achieve such a balance and discusses some of the subtle benefits that spring from that balance. We designed a network science course that achieves course objectives and at the same time allows students to develop, test, and monitor a network of their own choosing. Rather than use canned networks, we introduce a set of assignments called the Network Profile Summary in which students continually apply newly learned course concepts to their selected networks and are therefore able to realize the utility and place of these concepts. This course design fosters student motivation and encourages independent learning. Under this pedagogical paradigm, students began to see coursework, concepts and feedback as productive and globally meaningful rather than corrective and locally meaningful. 1. Introduction In The Rhetorical Stance, Wayne Booth argues that the best writing, including the best student writing, is the result of a balance between three aspects: speaker, audience and subject. To Booth, an education in rhetoric moves far beyond the formulaic teaching of writing. In fact, he recalls that the most important lessons he learned as a writer were not a part of a formal, writing class, but rather were a collection of other experiences quite unconnected with a specific writing course [1]. These lessons were important, Booth continues, precisely because they were connected to moments ‘when I had something to say [1].’ It is this important part of education that we, as teachers, often overlook in our course design, our assignments, and our responses to student work. Booth’s argument is larger than a case for rhetorical education; it is a case for student agency, involvement, interest and motivation that transcends all subjects. It is perhaps the last of Booths three aspects, subject of good writing that becomes the most important in the following article. Students have something to say, in other words they have a stake in their own work, when the subject of that work means something to them. More importantly, as Booth shows, having a stake in their own work provides students experiences that last, that are meaningful, and that shape students future work and experience. Professors know (and research shows) that interested and motivated students are the best students. Not only do they make class interesting with their innovative insights but also they often try the hardest, are the highest achievers and earn the best grades [2]. This interest extends beyond the day-to-day classroom interactions to include the subject of the work itself. William Butler Yeats is attributed with the thought that education is a fire to be lit rather than a vessel to be filled. The more we reach our students and allow them to connect with ideas and subjects that are of interest to them, the greater chance we have of igniting and stoking their academic fire, facilitating the conflagration of life-time learning in academia and beyond. The goal of providing each individual student with the opportunity to engage with topics that mean something to them, while still exposing all students to required material, resulted in the introduction of Network Profile Summaries (NPS) to the Complex Networks course offered by the Applied Mathematics Department of the Naval Postgraduate School. These summaries, the result of a multidisciplinary approach to education, enable students to pick any network that piques their interest to serve as a network science playground throughout the duration of the course. From infrastructure to terrorism to Star Wars®, students developed a portfolio of network science concepts in the context of a topic that interests them. In the context of Booth’s formula for the best writing, the NPS gives students the opportunity to discover something they want to say. The professor provides this opportunity in two ways, both of which increase the student ‘buy-in’ that is increasingly important in generating motivation in today’s classrooms. First, the professor cedes some control of the course to students by allowing them to select the object of their network profile. Second, the relevance of course content become obvious to the student as it is applied in as many different contexts as there are students in the class. It is quite the opposite of canned textbook problems. This work describes the goals of the NPS, explains how they enrich the classroom environment and provides some indication of their efficacy in fostering student motivation and causing learning to occur in the complex networks classroom. 2. Defining and achieving educational goals 2.1 An overview of the course and its goals The Complex Networks course at the Naval Postgraduate School is a terminal course in a sequence that prepares graduate students to receive a certificate in network science, see [3]. As such, there is flexibility in the selection of the content, allowing the course to respond to, and focus on, the emerging advancements of this actively developing field. The course does not require any previous coursework and draws students pursuing their graduate degrees in a wide variety of fields including Operations Research, Mathematics, Computer Science and Electrical Engineering. These students often come into the course from a mixed set of backgrounds, and many have been out of academia for several years due to their military careers. Therefore, each student’s skill set varies in terms of their abilities and familiarity with proofs, modelling, writing and programming in R or Python. Additionally, each student has opted to enroll in this elective class with his or her own individual learning goals; each has professional and personal interests driving an academic interest in network science. Though the Complex Networks course itself has no prerequisites, those taking the course as part of the Network Science Academic Certificate will have previously taken a course in discrete mathematics and a course in either graph theory, network traffic analysis, network flows, or game theory. Conceptually, the course exposes students to a variety of real and synthetic network models as well as network metrics, including both the standard, or foundational, metrics and new metrics based on the recent literature. Students also learn the relevance of these metrics in telling the narratives that describe a particular network. In addition to addressing the course material, this class is specifically designed to develop and hone other transferable skills, such as: Connecting mathematical results to real world meaning; Comparing and contrasting results between different scenarios; Taking responsibility for learning; Developing curiosity and formulating good, investigable questions; Presenting complicated ideas through clear oral communication; Presenting complicated data sets through clear visualizations; Defending their own work; Creating a convincing, coherent and cohesive narrative based on data and analyses. The way this course approaches these goals, and another way to understand Booth’s ‘stake’ in critical learning, is through the MUSIC model of student motivation [4], which stresses: eMpowering students to learn; helping students identify the Usefulness of the material to their personal goals; instilling a belief in students that they can Succeed in mastering the material; promoting student Interest in the material; and creating a Caring learning environment. 2.2 The Network Profile Summary In order to achieve the course goals, the class is designed around a series of individual assignments which replace traditional homework. These assignments are called the NPS. For the NPS, each student develops a network of his or her choosing, analyzes it weekly as new metrics are introduced in the course, presents the weekly findings in the context of the chosen network as a single PowerPoint slide, and compiles all of these weekly reports into an end-of-course presentation that summarizes and highlights the key findings to tell the narrative of the network. To launch the course, during the first week, students are exposed to several real networks as well as synthetic network models. In addition to seeing how raw data is used to create those networks, these early examples motivate the study of the course material by demonstrating a wide range of applications that can be analyzed with the tools that are presented throughout the course. In the second week, each student leverages his or her new appreciation for network science to choose a situation, find corresponding data and use the data to create a network that is then analyzed throughout the course. Having students choose their own networks motivates students by first, providing them a stake in their work, and second by directly addressing several elements of the MUSIC model. Specifically, the freedom to choose is empowering, and students can choose something that is of personal or professional interest. Additionally, this approach demonstrates the usefulness of the course material in two ways. First, each student will see how network science is used to help them answer questions on his or her own network. Second, students also get to see how network science is used to analyze the diverse set of applications selected by their classmates. The success and caring aspects of the MUSIC motivation are also addressed as students select their networks. Students work with the instructor to get individualized feedback on the feasibility of their choices and finalize their selections, so that they are set up for success. Though the student is empowered to explore a network of his or her own interest, the professor is involved in the process of selection so the chosen network is appropriate in size and/or structure. For example, a student may be interested in the internet; however, running analyses on such a large network several times per week is computationally beyond the scope of this course. In this case, the professor may steer the student toward some subset of the internet, such as a block of IP addresses. As an additional support mechanism, the professor maintains a library of networks available for students to sample, should they be unable to choose a network appropriate for exploration throughout the course. After the networks are chosen and built, each week, as students learn about new metrics and analysis techniques in lecture, they apply their learning to their own networks. This approach allows students to immediately practice their new skills and ask the instructor about issues they may face in transitioning from theory to practice. Since students are analyzing their own networks, they are naturally interested in the results and recognize the usefulness of what they have learned. Furthermore, this approach promotes curiosity, and students, driven by a burning question that naturally arises from their chosen topic, occasionally go above and beyond, learning additional techniques to satisfy their curiosity. Each week, the students look at the results obtained by analyzing their NPS network and meaningfully interpret their mathematical answers in the context of their chosen applications. They then select the most important takeaways from their analysis and present these to the class as a single-slide PowerPoint presentation. This process elevates the level of learning in three ways. First, after simply running the analyses, each student must decide what each part of the analysis means in the context of their applications. Next, the student must synthesize the results into a cohesive narrative about the network. Lastly, he or she must determine how to choose the appropriate metrics and/or visualizations that will best convey the narrative to the class. They have to think through both data visualization and slide organization skills, since they often have a significant story to share and very limited space for that story. By orally presenting their slides to the class, students not only practice their public speaking skills but they also have the opportunity to practice the proper implementation of new terminology, increasing fluency in the language of network science. In alignment with teaching recommendations outlined in the GAIMME report, a national report on improving mathematics education through the use of modelling [5], the NPS experience is genuine. Specifically, unlike a typical course in which the instructor already knows the answers to the exercises, the student-based selection of models means that the whole class, including the instructor, is learning from each student presentation, further empowering each student by making their efforts integral to the learning of the whole community. Students are empowered to become the class’s leading expert on their own networks. Each week, during the NPS presentations, the class is exposed to a diverse set of networks, giving the students an opportunity to identify similarities and differences between those networks and resulting metrics; this promotes a broad understanding of the material despite the fact that each student only performed a single set of calculations. The presentations often lead to student-led discussions, as the students feel a bit more comfortable posing questions to other learners than to an instructor. Occasionally, students will challenge their peers’ findings or interpretations, resulting in rich conversations filled with genuine discoveries. The interactive nature of the course helps promote an atmosphere in which the students dare to ask interesting questions and be creative in their thinking, absorbing the feedback from the instructor and their peers. At the end of the course, the students take their set of single-slide NPS presentations and organize them into a final cumulative presentation. Students are able to leverage the feedback they received from the instructor and their peers throughout the semester to make improvements on their slides, ultimately presenting a cohesive narrative describing their network analysis. 3. Assessment Although increasing student interest and motivation are key objectives of the NPS, the pedagogical model accomplishes much more. By centring classroom work on the interests of individual students, a shift occurs, not only in participation and quality of work but also in the assessment paradigm. Coursework still earns a grade, but both students and professors reconceptualize feedback. Rather than justifying the grade, feedback becomes constructive. When students have a legitimate stake in their work, they cannot help but see comments as collaborative rather than corrective. The conversations about the NPS assignments do not revolve around right or wrong. Instead, peers and faculty interact with the ideas and concepts that are discussed during the presentations and provide useful and meaningful feedback. It is in this interaction, in this collaborative environment, that true learning occurs. The inevitable shift in the feedback and assessment paradigm, then, contributes to the overall quality of the educational experience. This paradigm shift also contributes to the development of the whole person. We use assessments and the associated feedback as means to mentor future members of the discipline, to help them mature both technically and academically. In contrast, the feedback given in a more traditional assessment paradigm is seen as corrective; this corrective environment hampers the mentoring relationship. A canned problem set or scenario fosters an environment of corrective assessment, as it always has an approved solution, and students cannot help but attempt to measure their solution against this perceived ideal. When, however, they are working on something original that interests them, they are denied both the comfort of an approved solution as well as the pressure of measuring up to that ideal. Instead of perceiving comments as a corrective comparison between the ideal and their work, they begin to recognize feedback as opportunities to improve their own work. They have something to say, and the feedback they receive is a means of saying it better. In practice, the course approach to assessment is holistic and provides a formative assessment of individual assignments as well as a summative assessment at course completion. This assessment practice is adapted from those used by the United States Military Academys Department of Mathematical Sciences and the Department of English and Philosophy. This section includes discussion of the theory behind this holistic model of grading, followed by an example rubric used during the assessment of presentations given during the course’s Centrality Week, the week during which various centrality measures are taught and then applied to the NPS networks. The grade for the course is split between the NPS ($$40\%$$ of the final grade), exploratory research as a group paper to be submitted to a conference ($$40\%$$ of the final grade), and in class participation ($$20\%$$ of the final grade). The NPS provides a relatively simple homework alternative to assess student work: evaluate the presentations weekly, provide feedback and assign grades. For this evaluation, we look at both content and delivery of message using the criteria for strong slides and presentations in Table 1. Table 1 Assessment rubric for slides and presentations Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Table 1 Assessment rubric for slides and presentations Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit Criteria Task Detailed step (0–10 points) Content $$\square$$ Define centrality (general) Compare and contrast: $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Katz centrality $$\square$$ PageRank centrality $$\square$$ Betweenness centrality Correct analysis synthesizing learned concepts A (9–10 points) Relevant & clearly explained findings; Insightful contextualization of findings; and sophisticated synthesis & interpretation of metrics. B (8 points) Sound work, short of sophisticated synthesis C (7 points) Short range understanding, does not connect topics D (6 points) Marginally passing performance F (0–5 points) Little to no work of merit Presentation Depict (when applicable): $$\square$$ Degree centrality $$\square$$ Closeness centrality $$\square$$ Eigenvector centrality $$\square$$ Betweenness centrality General Comments: Clarity and style of graphics (Could they be presented in a significant way?) A (9–10 points) Slide deck Clear & succinct slide; correct spelling and mathematical notation; figures and tables are labelled with descriptive captions presentation is consistent in tense and active voice; references are complete and stylistically correct Conveying the information Clear & insightful verbal explanation; correct use of terminology while explaining clear and commanding voice B (8 points) Well done; lacks insight and sophistication of A work C (7 points) Slide portrays information, articulation is unclear D (6 points) Slides and presentation are marginally passing F (0-5 points) Little to no work of merit 3.1 Guiding principles A student that presents work worthy of an A demonstrates domination over the material presented. This student demonstrates a sophisticated understanding of the material and adheres to appropriate mathematical and grammatical conventions. The student shows superior control of insights and assertions derived from the analysis of one or more appropriately applied mathematical techniques. Presentations that earn an A make sound associations among concepts, weaving those logical connections into intelligent and sometimes highly insightful synthesis and argumentation when appropriate, all while managing the space limitations of a single presentation slide. Evidence is well chosen, deftly integrated and properly cited. In terms of organization, ideas are carefully ordered and structured with transitions that move the analysis in evident directions. The author makes skilful use of data visualization to enhance reader understanding of analysis and conclusions. Few submissions are perfect, so minor errors in grammatical correctness may occur. Students that earn the mark of B demonstrate good understanding of the material. These students comprehend and address the material while adhering to appropriate mathematical and grammatical conventions. The presentation skilfully controls insights and assertions derived from the analysis of one or more appropriately applied mathematical techniques. Presentations earning a B make sound associations among concepts, in the service of exposing results. Treatment of individual concepts and ideas is clear and thorough. Evidence is well chosen, properly cited and usually effectively integrated. The presentation may attempt to synthesize; however, clear and coherent synthesis is hampered by control of subordinate concepts. In terms of organization, the slide structure may feel formulaic, instead of being optimized to clearly convey the specific narrative. The author makes good use of data visualization approaches to support reader understanding of analysis and conclusions. There are some grammatical errors in correctness that will not impede comprehension. We assess work that is passing and demonstrating short range proficiency at the grade of C. Such work addresses the material and demonstrates awareness of the concepts but may not clearly convey their knowledge to the audience. Tone can be somewhat misguided, though not usually inappropriate. The presentation controls at least one concept and/or assertion and presents a generally clear and accurate discussion thereof. Presentations earning a C seldom reach beyond the level of the primary mathematical concept to make associations or connections with related concepts, nor do they move toward synthesis. Insights and results may be simplistic or demonstrate limited flaws in logic. Evidence is generally well chosen but might not be clearly explained or effectively integrated. The organization of the slide may be ineffective. Figures may be present but are not leveraged in a manner that effectively supports reader understanding of analysis and conclusions. Some grammatical errors noted in the realm of correctness could cause minor or temporary difficulty with comprehension. Students may earn a D with a marginal performance that is passing. This work demonstrates a slight grasp of one idea or concept, but limited ability to control or explain that concept effectively. These presentations might adequately communicate a part of the idea in a clear thought or two, but they fail to produce coherent and logically connected presentation. Choice of evidence is usually confusing or flawed. Tone can be inappropriate at times. Several spoken sentences may be poorly or incorrectly structured, wordy and/or vague. Confusion regarding the audience and context could be apparent. A borderline presentation passes when it contains a clearly articulated instance of comprehension, as long as correctness errors are not rampant. Failing marks may be assigned at two levels. A 50% F, or high F, is a presentation that contains some mathematical content, but definitely fails to demonstrate understanding of the requirements. A 25% F, or low F shows some attempt at fulfilling the requirements, but provides little relevant content that indicates a definite failure to demonstrate understanding of the requirement. Students that fail to submit the assignment are awarded a 0. Five percent of the overall grade is available for award upon demonstration of elements seen in higher marks. For example, if a student’s presentation mostly aligns with the grade of C, but demonstrates some elements worthy of a B, the student is awarded a C+. A student that presents A-level work and includes work or insights that are beyond the expectations of the course, that student may earn an A+. 3.2 Sample rubric Each week in the course covers a related set of topics. During centrality week, the instructor introduces the broad topic of network centrality and presents the following centrality measures: degree, closeness, eigenvector, Katz, PageRank and betweenness. In addition to the in-class instruction, we assign the students a set of training exercises so that they can practice computing centralities on small networks. Leveraging their instruction in centralities, the students then perform centrality analyses on their selected networks, synthesize the results into a coherent narrative about their network in the context of their application and prepare a single slide for the NPS. Table 1 is a rubric for the centrality NPS, allowing the instructor to check off the topics that are addressed in the presentation as well as quickly assess the overall quality of the work. There is some space in the table for the instructor to give more general comments about the selection of the metrics, suggestions to refine the narrative itself and techniques to enhance the presentation of the narrative. 4. Examples of NPS slides In this section, we present examples of slides from the NPS of different students that were part of the $$2016$$ cohort. 4.1 Using existing networks for NPS Figure 1 shows two slides from the NPS of Ben McCaleb. The slide on the left introduces his NPS network he will analyze throughout the quarter. The slide on the right presents his work on the degree distribution in which he tries to fit a curve through the degree distribution; in this case his network appears to be a very modular network with a modularity of $$0.932$$. Fig. 1. View largeDownload slide The West Coast Power Grid introductory slide and degree distribution slide (by Ben McCaleb). Fig. 1. View largeDownload slide The West Coast Power Grid introductory slide and degree distribution slide (by Ben McCaleb). Figure 2 shows two slides from the NPS of Richard Allain. The left slide presents an overview of the Disease and Disorders network (top right) along with then two subnetworks: human disease network (nodes are diseases and edges are the genes connecting them), and disease gene network (nodes are the genes and the edges are the deceases they depicted in pairs). Through the presentation and feedback process, the student learned that this would be better presented using hypergraphs. Later in the course, he then presented the slide on the right illustrating the degree distribution and community information for his network. Fig. 2. View largeDownload slide Introductory slide to disease and disorders, and degree distribution slide (by Richard Allain). Fig. 2. View largeDownload slide Introductory slide to disease and disorders, and degree distribution slide (by Richard Allain). Figure 3 includes the introductory slide (left) and the centrality slide (right) for the European Union Airlines network [6], a multilayered network that Brian Crawford used for his NPS. Fig. 3. View largeDownload slide Introductory slide and the centrality slide for European Union Airlines network (by Brian Crawford). Fig. 3. View largeDownload slide Introductory slide and the centrality slide for European Union Airlines network (by Brian Crawford). 4.2 Creating new networks for NPS Several students used available data to create their own networks: capturing new data as political events happened, using their military background and knowledge about military distribution equipment, watching movies to create an interaction between characters, screening their hard drives and emails to create social networks, and so on. The fact that even a handful of students took this approach supports the authors’ belief that students would like to construct and pursue an analysis of information about which they are passionate, if given the chance. Moreover, students were encouraged to bring to class any data that they might consider analyzing for their thesis, and several students have done so [7–9]. Tom Knuth used www.netlytic.org [10] to capture Twitter data that was relevant to the Trump presidential campaign to create the ‘Trump Talk’ network shown in Fig. 4. His overview slide (left) presents the motivation, and the slide on the right presents some of the network’s metrics. Fig. 4. View largeDownload slide Introductory slide and network statistics of the ‘Trump Talk’ network (captured by Tom Knuth using www.netlytic.org). Fig. 4. View largeDownload slide Introductory slide and network statistics of the ‘Trump Talk’ network (captured by Tom Knuth using www.netlytic.org). Zack Luckens created the Star Wars network shown in Fig. 5. He captured the Star Wars social network by re-watching the episodes and adding edges between two characters if they talked to each other, fought together/against each other, were relatives, or worked together. At that time the website http://moviegalaxies.com/ [11] did not have the network for Star Wars, but now it contains a different version of this network. Fig. 5. View largeDownload slide Introductory slide and centralities in the Star Wars IV, V and WI episodes (captured by Zac Lukens by watching character interactions in Star Wars). Fig. 5. View largeDownload slide Introductory slide and centralities in the Star Wars IV, V and WI episodes (captured by Zac Lukens by watching character interactions in Star Wars). In Fig. 6, Ryan Miller introduces the Noordin Top network, data collected by NPS students under supervision of Everton and Roberts [12]. As this network had several attributes associated with its nodes, he created layers for the networks turning it into a multilayered network. This multilayered information is now part of his thesis [8] and served as a basis for a couple of publications co-authored by this student [13, 14]. Fig. 6. View largeDownload slide Introductory slide of the Noordin Top Network, and the layer visualization (by Ryan Miller). Fig. 6. View largeDownload slide Introductory slide of the Noordin Top Network, and the layer visualization (by Ryan Miller). Greg Allen created his hard drive social network, shown in Fig. 7, using the forensic bulk extractor [15], a digital fingerprinting tool. He used this analysis as a seed for the analysis in his thesis [7], which he analyzed several people’s hard drives by categorizing the components of the networks according to the usefulness in identifying the owner of the hard drive; this work also required the use of machine learning featuring metrics he learned in this course. Fig. 7. View largeDownload slide Greg Allen created his hard drive social network using the forensic bulk extractor [15], a digital fingerprinting tool. Fig. 7. View largeDownload slide Greg Allen created his hard drive social network using the forensic bulk extractor [15], a digital fingerprinting tool. Miguel Miranda Lopez created a network of user accounts screening his email from September through December $$2016$$. The data was uploaded and turned into a network using www.netlytic.org [10]. The network is shown in Fig. 8. This also inspired him to further this analysis in his thesis on ‘Classifying Cyber Targets in Email Networks from Digital Storage Media’. Fig. 8. View largeDownload slide Miguel Miranda Lopez created use accounts network using his email data and www.netlytic.org.. Fig. 8. View largeDownload slide Miguel Miranda Lopez created use accounts network using his email data and www.netlytic.org.. The aforementioned networks are just some examples of the types of work submitted by students, and the diversity of this limited set illustrates the broad applicability of network science for the students in the class. Additionally, given that some students used their NPS as a seed for their thesis supports the case that the NPS increases student motivation, as these students are going above and beyond as they follow the questions triggered by their work in the course. 4.3 Possible concerns and encouragement While the student-centred, motivation-based design of the course may be attractive, it is only natural that potential instructors have concerns about the implementation. We have anticipated several of these concerns and briefly addressed them. What if my students cannot find a suitable network for their NPS? Many of the students in this course arrived with ideas and were eager to choose their own network, but it was helpful to have several living networks available to offer as suggestions for students who were not prepared to select their own network. Since these networks are still adapting, the student’s work is still new and can provide valuable insights, unlike a static artificial network generated for the purposes of completing an assignment. Several sample networks, as well as a syllabus for the course, are available at http://faculty.nps.edu/rgera/MA4404/NetworkProfileSummaryResources.html What if there is resistance to learning another software tool? This can be an issue in many mathematics and science courses, including network science courses. However, learning the software is a clearly stated goal and is considered part of the course content, so there is a pre-existing expectation to learn the software. Student buy-in can be more easily achieved by clearly communicating this objective and demonstrating the value of the software, especially as it relates to helping them answer their own questions about the network that they have selected for the analysis. If every student is turning in a project on a completely different topic, will the grading be really difficult and/or time-consuming? While each submission is unique, it is the responsibility of the student to make a strong case both to the instructor and to their classmates. The assessment process is partially outsourced to the class, as they raise questions and concerns about the work of their peers. There is no obligation on the part of the instructor to regenerate the work and confirm the answer. The grading burden is further lightened through the development and implementation of a simple, generic rubric which can be shared with the students before they submit their work to clearly convey the instructor’s expectations. Additionally, the students receive actionable formative feedback throughout the semester, thereby improving the quality of their final submission and its alignment with the rubric. What if I try this approach, but my students are not receptive? Change can be difficult. Unlike a typical course, students must shoulder a larger burden for their learning, and they cannot simply compare and check their homework answers with one another. As with any new pedagogical approach, students are more likely to adapt positively when the instructor sets the tone early in the course, clearly communicates the expectations and demonstrates that he or she cares about the student success. 5. Conclusion Wayne Booth tells a story of a bright, young graduate student whose essays were particularly awful. His writing was awful, Booth suggests, because in response to canned, formulaic assignments, he found nothing worth saying. When, on the other hand, he made an impassioned argument criticizing Booths own interpretation of a text, Booth says he wrote a four page polemic, unpretentious, stimulating, organized and convincing [1]. It was only when this student had something to say and someone to say it to that his writing began to develop. The core belief that inspired the NPS is an extension of the discovery Booth made all those years ago in a graduate English seminar. That is, students analyze, evaluate and create (and do these well) if they are allowed to choose and develop networks that interest them personally rather than respond to a choice that has already been made for them. The methodology introduced here replaces the standard homework assignments with an exploratory and creative medium for understanding network science concepts. Each week, the students are presented with new concepts. Then they explore these concepts on their individual networks rather than solving problems similar to the ones worked in the book or classroom. Students further synthesize the results obtained into convincing, coherent and cohesive stories as they create their weekly power point slide. This process culminates with the summary slide that creates a profile for the studied network. By allowing the students to practice on their personal network, we empowered them to personalize their project and take responsibility for their own learning. Since there is no pre-determined answer for these networks, students rely on critical reasoning and connections to the real world to present and defend their observations to the class. The classroom discussions during students’ presentations create a genuine graduate-level learning environment. The collective feedback naturally invites students to compare and contrast the weekly learned network science concepts on different networks. Seeing several different results, students bring new questions that the facilitator might not have considered. These discussions often plant seeds for new research creation, which is the reason several students were inspired by NPS for their theses and several of these networks were used for theses [7–9] and class group publications [13, 14, 16–19]. Finally, it is obvious that students are more enthusiastic about the class. One student remarked on the student opinion form that ‘I enjoyed exploring the course concepts by implementing them on my network profile. Great way to learn!’ While another mentioned that ‘This was an engaging and interactive course...improved my understanding of the material’. Moreover, the professor has a blast teaching and facilitating, as the networks presented and the discussions provide variety in teaching. Acknowledgements The authors would like to thank the DoD for partially sponsoring this project. The authors are grateful to the referees for their valuable input on this article. References 1. Booth, W. C. ( 1963 ) The rhetorical stance. Coll. Compos. Commun ., 14 , 139 – 145 . Google Scholar CrossRef Search ADS 2. Michaels, J. W. & Miethe, T. D. ( 1989 ) Academic effort and college grades. Soc. Forces , 68 , 309 . Google Scholar CrossRef Search ADS 3. Gera, R. ( 2017 ) Leading edge learning of network science. (in press) . 4. Jones, B. ( 2009 ) Motivating students to engage in learning: the MUSIC model of academic motivation. Int. J. Teach. Learn. High. Educ. , 21 , 272 – 285 . 5. Bliss, K., Fowler, K., Galluzzo, B., Garfunkel, S., Giordano, F., Godbold, L., Gould, H., Levy, R., Libertini, J., Long, M., Malkevitch, J., Montgomery, M., Pollak, H., Teague, D., van der Kooij, H. & Zbiek, R. ( 2016 ) Guidelines for Assessment and Instruction in Mathematical Modeling Education (GAIMME Report) . http://www.siam.org/reports/gaimme.php Cited 05 June 2017 . 6. De Domenico, M., Porter, M. A. & Arenas, A. ( 2014 ) Muxviz: a tool for multilayer analysis and visualization of networks. J. Complex Netw. , 3 . 7. Allen, G. ( 2016 ) Constructing and classifying email networks from raw forensic images. Master Thesis , Naval Postgraduate School, Monterey, CA . 8. Miller, R. ( 2016 ) Purpose-driven communities in multiplex networks: thresholding user-engaged layer aggregation. Master’s Thesis , Naval Postgraduate School, Monterey, CA . 9. Warnke, S. ( 2016 ) Partial information community detection in a multilayered network. Master’s Thesis , Naval Postgraduate School, Monterey, CA . 10. Gruzd, A. ( 2016 ) Netlytic: Software for Automated Text and Social Network Analysis . http://netlytic.org. [Software Tool] . 11. Kaminski, J., Schober, M., Albaladejo, R., Zastupailo, O. & Hidalgo, C. ( 2012 ) Moviegalaxies-Social Networks in Movies . Consulté sur http://moviegalaxies.com. 12. Roberts, N. & Everton., S. F. ( 2011 ) Terrorist Data: Noordin Top Terrorist Network . https://sites.google.com/site/sfeverton18/research/appendix-1. 13. Crawford, B., Gera, R., Miller, R. & Shrestha, B. ( 2016 ) Community evolution in multiplex layer aggregation. Advances in Social Networks Analysis and Mining (ASONAM), IEEE/ACM International Conference . pp. 1229 – 1237 . http://ieeexplore.ieee.org/abstract/document/7752395/. 14. Gera, R., Miller, R., MirandaLopez, M., Saxena, A. & Warnke, S. ( 2017 ) Three is the answer: combining relationships to analyze multilayered terrorist networks. Advances in Social Networks Analysis and Mining (ASONAM), IEEE/ACM , (in press) . 15. Garfinkel, S. L. ( 2013 ) Digital media triage with bulk data analysis and bulk_extractor. Comput. Secur. 32 , 56 – 72 . Google Scholar CrossRef Search ADS 16. Allain, R., Gera, R., Hall, R. & Raffetto, M. ( 2016 ) Modeling network community evolution in YouTube comment posting. International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation (BRIMS) . http://sbp-brims.org/2016/proceedings/LB_115.pdf. 17. Berest, M., Gera, R., Lukens, Z., Martinez, N., & McCaleb, B. ( 2016 ) Predicting network evolution through temporal Twitter snapshots for paris attacks of 2015. International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation (BRIMS) . http://sbp-brims.org/2016/proceedings/LB_111.pdf. 18. Chen, S., Debnath, J, Gera, R., Greunke, B., Sharpe, N. & Warnke, S. ( 2017 ) Discovering community structure using network sampling. The 32nd ISCA International Conference on Computers and Their Applications (CATA) . 19. Crawford, B., Gera, R., House, J., Knuth, T. & Miller, R. ( 2016 ) Graph structure similarity using spectral theory. Springer International Publishing AG 2017 ( Cherifi H. et al. eds), International Workshop on Complex Networks and their Applications, Studies in Computational Intelligence , vol. 693 , Cham : Springer , pp. 209 – 221 . Google Scholar CrossRef Search ADS Published by Oxford University Press 2017. This work is written by US Government employees and is in the public domain in the US.

Journal

Journal of Complex NetworksOxford University Press

Published: Aug 29, 2017

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off