Qualtrics recently added a series of features that you will see listed as you log into Qualtrics, along with links to instructions and tutorials. To review a few of those features, let’s start with the Mobile Compatibility Checker.  This checker allows you to optimize your survey to be viewed and taken on mobile devices.  Mobile Compatibility Checker is a feature that you turn on; then as you scroll through your survey Qualtrics, it will present the mobile checker icon in the left column of a survey question if tips are detected that will help your survey display better on a mobile device.  To turn this feature on:

1. Log into Qualtrics and go to Edit Survey
2. Open the Advanced Options drop-down menu located in the upper right of the screen.
3. Select “Toggle Mobile Compatibility Check”.
4. Scroll through your survey and look for the mobile checker icon displayed in the left margin of your questions.
5. When you see the orange icon with a small mobile screen inside of it; click the icon to view suggestions.

If you use questions with heat maps, Qualtrics has added a new option, the Multi-Click Heat Map, to let you determine how many clicks to allow on the map.  To explain further, heat map questions let you load an image as part of your question.  Survey participants then click on the image to indicate their answer to your question.  You could use this for a variety of purposes; identifying points on the human body, picking the most useful areas on a web page for usability studies, locating areas on a geographical map or perhaps identifying creative components in a painting. To use the new feature for the Multi-Click Heat Map question format, follow these steps:

1. Log into Qualtrics and go to Edit Survey.
2. Use the Create a New Item link to add a question.
3. Select Insert a new Heat Map item.
4. Select and upload a graphic to use for your question.
5. In the right tool bar, select the number of clicks you would like to allow on that image by increasing the number of clicks in that option.

If you have ever wanted to add the comparison of goals to your actual data results, Qualtrics has now added Goal Lines for Graphs. This feature lets you edit the survey results graphs to show your goal values in the Y-Axis against the actual survey results values. For example, you might want to show how many students accessed certain campus services in the past year against how many they indicated accessing this year in your current survey.  For research purposes, you could show the expected use of milk in a calcium study or the expected use of safety procedures against what is actually reported.  These graphs may be easily downloaded into reports for MS Word, Excel or PowerPoint.  To include Goal Lines for Graphs:

1. Log into Qualtrics and along the top bar; go to the tab called Reporting Beta.
2. Create a new report or open an existing report.
3. Add a bar, line or control graph to the question you want to graph.
4. Click Graph Options in the top menu, then click on Goal Line.
5. Set a value for the goal line based on your Y-Axis values.

To read about other new Qualtrics features such as adding multiple data sources to a word cloud, scheduling future downloads of data or grouping page items; click here. 

We are at the end of the spring semester, which means Summer courses will begin soon and planning for the Fall semester is already underway. Now is a great time to consider how to use student activity reports in addressing participation issues or finding out what content/tools in your course(s) are getting attention and which ones might not receiving as much.

As promised in the previous article, this entry will cover the features of other reports you can run in Learn. If you have not yet read the previous article, please click this link to review it (that way, what you see below makes sense): http://blogs.itap.purdue.edu/learning/2013/03/12/evaluating-your-students-blackboard-learn-and-its-underused-unique-feature-part-1/ .

Last time, we discussed the aspects of the All User Activity inside Content Areas report, and how it displays a summary of all user activity inside Content Areas for the course.  The reports that will be covered in this entry are: Course Activity Overview, Overall Summary of User Activity, and Student Overview for Single Course. The information displayed comes from a large enrollment course, and all student names/usernames are blacked out for confidentiality reasons.

First up is the Course Activity Overview report, which displays the totals and averages of student’s time (in hours) spent in the course on Blackboard Learn.

The first part of the report shows the overview of total hours of student activity for each day of the week in the course. In addition, the amount of students in the course are displayed the top, including the date range of the report’s data. Below we can see the total time in hours in the course and the average time spent per user.

The second part of the report, which can extended to several pages after due to the amount of students, shows the total amount of hours each student spent (the blue bars) and avg amount of hours the class of students spent in the course (orange line)- given the date range of report. We can see that many students either spent many more or less hours in the course than the class average.

Note: Again, this data comes from an actual course, and the names of students included have been blacked out for confidentiality purposes.

Now, let’s take a look at the Overall Summary of User Activity report. This report is similar to the All User Activity inside Content Areas report in the last entry, but the biggest difference is that this one keeps track of course tool/mashup usage.

The first part of the report shows the total amount of hits each tool had during the date range set for the report. The report shows the tools/mashups that currently exist in Learn.

The second part of the report shows the user activity totals per user per tool/mashup.

The third part of the report shows the overall user activity per user for each month/day. A graph is shown for each day of the date range on the x-axis and the total hours of activity of all users on the y-axis.

The fourth part of the report shows an overview of user activity on which hours of the day users accessed the course the most. The table on the left lists the hours of the day and the total hits of for that hour, and finally the comparible percentage. The total hits for each hour of the day is display in the graph on the right.

The fifth, and last part of the report is an overview of user activity on which days of the week users accessed the course the most. The table on the left shows the

Now, let’s take a look at the last report type that will be covered, which is the Student Overview for Single Course report. This report is similar to the Course Activity Overview report, but focuses on one student’s level of activity and the total hours/number of times acessed for each item in the course.

The first part is similar to the first page of the Course Activity Overview report, and focuses on total hours the student had on the days of the week for a date range.

The second part of the report covers the hours the student has spent looking at items in the course, including the number of times they were accessed and the initial access date/time of the item by the studen

As you can see there are several more report types to choose from and they can be useful for seeing trends in overall student activity or pinpointing students who have certain activity levels.Again, I hope this article has been useful to you and inspires you to use these reports in your course(s). I will have one more follow-up blog post on this topic that will cover the remaining reports that Blackboard provides. If you have any questions, please contact ITaP’s Consulting & Training group at tlt-consulting@purdue.edu .

In an age of “big data” – Excel spreadsheets with a million lines of code, SPSS and SAS files computing data far beyond the reaches of the human mind, and super-computers reaching distantly into our cosmological past and future(s) – is there such thing as a moral compass? Do our technological capabilities progress at a rate far beyond the steady reflection necessary for ethical inquiry?

In a series of three previous blog postings, I explored the implications of analytics viewed with a distinctly utilitarian lens. Themes included how institutions should use data to help as many students as possible, how student success can be reframed in terms of reinforcement of certain positive behaviors, and how the student-institution relationship is renegotiated with the use of big data.

Now that I bring this series to something of a tentative close, I do so with the suggestion that this conversation is only beginning. There is nothing new about so-called big data. The Romans had big data over 2,000 years ago – called a census. What makes this a new time in history is our computing power to understand the data we possess. The science of learning analytics makes sense of massive data sets which have the ability to influence students and their behaviors. I suggest that we seriously consider the need to recast our desire to correlate previously unforeseen variables within the prism of ethical discourse. The ability to predict certain outcomes with very powerful regression models means that data holders have monumental responsibility to act ethically toward students and institutions.

I propose that what we learn from utilitarianism can offer us some key questions to consider when thinking about the intersection of learning analytics and ethics:

• Do unique identifiers within the data set compromise student or institutional trust if they are utilized?

• What steps am I ethically compelled to take to protect unique identifiers?

• What safeguards can be taken to effectively de-identify data points which may be later reconstructed into unique identifiers?

• Is it possible to know what effect my analysis of a data set will have on students? If not, what protections may be put into place to minimize or negate negative outcomes?

Purdue will upgrade to Blackboard Learn Service Pack 11 during the 2013 Summer Session. With this upgrade to Service Pack 11 will come several notable changes that faculty will see as they use Learn. While the look and feel of Blackboard will not change dramatically, several new features will be introduced to enhance the use of Learn.

New Visual Text Box Editor
Blackboard has revamped the Visual Text Box Editor to add some new features and to allow for easier formatting of information.

There are several new options within the editor:
Preview:  The Preview button is not a new feature, but it is relocated to the upper right hand corner of the menu.  Clicking the Preview button will allow you to see what you have entered in the text editor in a pop-up window. This tool can help you determine if any additional formatting work is needed.
Full Screen: The Full Screen button , also located in the upper right and corner of the text editor, will open your text editor to the whole screen, removing the restrictions on the size of the text editor in Blackboard. To return to the normal view, click the Full Screen button again.

Record from Webcam:  Blackboard has introduced it’s Video Everywhere service. which will allow for the integration of user-created videos (via an attached webcam) within the text box. When clicked, this button  will allow instructors to record a video with their webcam and save it to YouTube, where the video will be saved as a non-searchable video and will then be entered similar as a YouTube mashup video in the text box.
Emoticons: The Emoticon button will allow instructors and students to include small images that will indicate the “mood” the student or instructor was in when writing the text of the message – whether happy, sad, or even sarcastic! This can help instructor and students read postings (like discussions) with the appropriate context.

Paste from Word: There is no longer a separate “Paste from Word” mashup within the text editor. Instead. Blackboard has improved the ability for content from Microsoft Word to be directly pasted from Word into its text editor, by removing unnecessary formatting information and converting it from the XML language used by Office to HTML.

Item Analysis
Blackboard has added item analysis for assessment questions. Instructors can use the item analysis tool to help determine if there were questions within the exam that may have been problematic for students.

The Item Analysis tool provide exam statistics, including questions it finds within its analysis that are “easy”, “medium”, or “hard”, and provide statistics on individual questions to back up its analysis. Discrimination on questions is available after several students complete the assessment. The analysis can be run at any time the assessment is available.

Discussion Forums – Force Students to Create Initial Thread
For courses that use discussion forums, a new setting on discussions will prohibit students from viewing discussion forums without contributing a post! Instructors can now require students to add a new thread to a discussion forum before the student can read any of the posts within the forum. This can help ensure students’ posts are unique and are not influenced by other posts within the forum.

Calendar
Blackboard has greatly improved its Calendar feature. Now, the Blackboard Calendar will show assignments and events in all classes. All instructors need to do is ensure they include a Due Date on assignments and assessments to have those items appear on the calendar. Instructors can also add other class items to course calendars, and students can add items such as study group meetings and organization meetings to their portion of the calendar.

By default, all events from all courses will display. Previously, the calendar was restricted to individual courses within Learn.

This is the first post going over some of the new features in Learn. More information about these exciting improvements to Learn will be posted in the next several weeks!

For more information about the new features in Blackboard Learn, or to schedule a consultation, please contact us at tlt-consulting@lists.purdue.edu.

Brett Creech
Educational Technologist

WAVE is a free tool created by WebAIM to help individuals evaluate, and thus improve, the accessibility of web pages. It is an automated tool, so by itself it cannot tell you if all aspects of your web content are accessible. (Knowledgeable humans are still necessary!) However, WAVE is a powerful tool that makes it easy to check certain features for accessibility.

Using WAVE

To use WAVE, enter a web page address on the WAVE home page.

To have an example, I will evaluate www.whitehouse.gov. After I enter the URL, WAVE displays the original page with additional icons and a sidebar summary.

The sidebar shows at a glance what errors, alerts and other features it has detected. You may view details, documentation or a heading outline by selecting one of the other icons at the left of the sidebar.

Toolbar Available

A WAVE Firefox toolbar is also available. If you are going to be checking multiple web pages, a toolbar is highly recommended. Unlike the online version, the toolbar plugin will allow you to check private, intranet, password-protected, dynamically-generated, or scripted content. The toolbar has not yet been updated to WAVE 5, but it is in process.

Additional Information

Help is available from the makers of WAVE at http://wave.webaim.org/help. In addition, WebAIM’s Jared Smith presented a webinar on WAVE 5 in October 2012 when it was in beta.

Writer: Dean Brusnighan, ITaP Assistive Technology Specialist

As discussed in my previous blog posting, utilitarianism is a useful ethical model to examine academic analytics. Utilitarianism is the theory that ethical decisions ought to be made in terms of what benefits the most number of people. This is especially useful for academics because decisions are often couched between the needs of individual students and the entirety of the student body. Utilitarianism is useful to help bring together the power of analytics and decisions that affect student success as a group.

The renegotiation of the student-institution relationship means the data students provide, whether it is in the form of demography, course preferences, final grades, et al., comes under a different umbrella. What does it mean for an institution to promise a student that his/her information will be safeguarded when computing predictive analytics? FERPA and other protective regulations aside, a school has a responsibility to provide the most good (in this case, scientific data) to students to help improve learning outcomes, retention, and a host of other concerns. While this relationship may not seem “new,” I beg to differ because it is generated and generative in terms of super-computing, data-mining, and predictive power. What I mean by “generative” here is that big data provides correlations that were, before now, not possible or simply lost in the aggregate. How do schools reassure students that such correlations really are for “the good” and will not be used inappropriately? I suspect that wide-sweeping institutional policy changes and modifications are on the horizon.

These twenty-first century concerns of privacy versus useful information force ethical questions of paternalism. This means that institutions should not decide what is best for students in a vacuum. Instead, universities ought to seek out student, faculty, and other administrative viewpoints before deciding on a predictive analytics program that affects large groups of students. Reconsidering the data between the aggregate and individual student is important to contextualize what effect might occur in the group. Balancing big data’s promise of moving aggregated data into individually-useful points requires engaging with an institution’s desire to do the most good for the most students.

Next blog posting: final thoughts on utilitarianism and analytics in terms of the future of big data.

I attended the EDUCAUSE Midwest Regional Conference in Chicago in March of this year. My primary reason for attending was to deliver a presentation with Pat Reid on Gradient, a peer review tool developed by the Informatics team here at Purdue. I also went to learn more about the wonderful projects that our colleagues throughout the Midwest are integrating onto their campus.

EDUCAUSE hosted a bevy of events designed to help new attendees adjust to the conference, as well as presentations aimed at enhancing the professional development of all attendees. For example, the workshop on Successful Mentoring for Career Development, provided valuable information on how to instigate and structure mentoring opportunities within one’s organization, as well as how to foster relationships with individuals based in the field of IT.

David Ward, one of the keynote speakers, discussed the evolution of strategic developments in higher education that he has been witness to (and helped instigate) during his career. He explained the necessity of Universities to make tough decisions that may be unpopular to tradition, but imperative for survival. In his subsequent talk, he addressed how he believed course transformation models, such as the flipped classroom, would further impact higher education.

Being a gamer, the presentation by Western Michigan University (WMU) on Broncoland caught my attention.

This highly interactive game is designed as a recruitment tool to help potential students learn more about what the first year experience looks like on WMU’s campus. Students can tour the campus and interact with pedagogical agents who help simulate scenarios the average freshman would experience during their first year on campus. The 3-D environment was highly engaging.

The presentation on Social Learning Networks from Davenport University showcased a tool that is used as a professional development tool for faculty. This workshop walked us through the evolution of this home-grown tool and explained how it helps support different types of faculty on campus. In a related vein, the University of Wisconsin, Madison, described how they selected an emerging tool to use in a private social network to be used in their active learning based nursing program

There were several sections on mobile tools. I was able to attend a session on TourGuide, a GPS based web app that allows participants can construct their own tours directly from a mobile device or desktop computer. This tool can be used by anyone who has an account, and anyone can request an account from the TourGuide Website. Innovation through Configuration: Maximizing Learning Experiences and Student Engagement with Technology showcased Adrenna, a Moodle based learning management system. The presenter explained how their company was able to customize this cloud-based LMS to the specific needs of University of Southern Florida.

There are several more presentations that I haven’t discussed in this space. If I shared everything I learned, I would have enough information for a magazine or short book. I encourage you to go to the conference website where many of the presenters have shared their presentations. If you are at an institution that is a EDUCAUSE partner, you can use Federated Login https://www.educause.edu/user/login?destination=node/276605 to login with your institution’s credentials.

Instructors sometimes ask,  ”How can I create an assignment in Blackboard Learn that isn’t graded?”

It is possible to create a check mark in your Grade Center indicating a task is completed without assigning a grade value.  The trick is setting that part up after you have created the assignment.  To have an entry that looks like the image below in your Grade Center, you need to set up the assignment as normal, which requires you to enter some value in the Points Possible field.

After you have the assignment created, go to that column in your Grade Center.  Select Edit Column Information from the column header drop down menu.  As you scroll through your options, under Primary Display, you will notice an option for Complete/Incomplete.  You may select that option now and change your Points Possible to zero or another desired value.

Remember to click the Submit button to save your changes. You will now be able to click in the grade cell and add a check mark for students who have completed the assigned task.

As discussed in my previous blog posting, utilitarianism is a useful ethical model to examine academic analytics. Utilitarianism is the theory that ethical decisions ought to be made in terms of what benefits the most number of people. This is especially useful for academics because decisions are often couched between the needs of individual students and the entirety of the student body. Utilitarianism is useful to help bring together the power of analytics and decisions that affect student success as a group.

Academic analytics today employ statistical methods to harness predictive capability in terms of student success (in various measures), what a student ought to take as a curriculum sequence, and how students may improve course completion through behavior modification. The question of utility here, then, is a question of what it means to predict. In big-data prediction, it is easy to slip into thinking that the prediction is the inevitable outcome because it comes with sophisticated longitudinal and deep analysis; it is important to remember that prediction is not fact: prediction is the use of factors to establish what could be fact.

In applying a utilitarian framework to academic analytics, universities have the ability to shape student behaviors by mixing educational trial and positive reinforcement. This means universities can use predictive modeling to ascertain how educational challenges can help reinforce behaviors like resilience, perseverance, and patience; coupled with positive reinforcement, universities may harness skills that will not only benefit the student, but also the graduate who will face challenges in the “real world.” Such modeling also reinforces the fact that learning – real learning – is arduous. Learning can be made more effective, however, by universities who use predictive analytics to take stock of educational theory, motivation, and pedagogical modification to benefit their students.

The next blog posting in this series will address how the use of predictive factors to look at what could be true for a group of students means that the role of responsibility between the institution and the student must be renegotiated.

Who viewed this item? What did students look at before test day? Are any of my students utilizing the resources I’ve posted? If you are the type of instructor who is asking these questions – it could be that you teach a large lecture course, maybe an online course with students all over the nation and world, or you simply want to know if the hours you spent uploading lecture notes to your course are actually being consumed regularly. You’re in luck, as a cure to this issue has been found- really it has. Blackboard Learn has filled the void of LMS’s being able to accurately and in-detail track the activity of students in a course…something Blackboard Vista and older systems like it were very limited in their ability to compute or were completely unable to perform.

So before we dive into the technical how-to’s, it’s important to examine the benefits of activity tracking and the amount of effort you will have to expend to maximize its usefulness. This way you can determine if this is a feature you want to employ now or in the future.

?Question: Should I tell my students I’m tracking their activity or should I keep it a secret?

#Thoughts: Simply put, it depends. You could tell your students at the start of the semester and immediately see increased activity in your couse. However, you might run into the problem where students only click on items to register their activity, while not actually engaging with the content. Alternatively, you could keep the tracking a secret and be able to view the level of dedication each one of your students possesses. However, you might run into the issue that your students aren’t motivated to view your course, which wastes the time you spent uploading materials. If you do pull a student aside for not being involved, they might be upset that you didn’t notify them that the tracking was potentially negatively impacting their standing in a course.

!Solution: It’s better to let your students know you are using the activity tracking. Even if some of your students abuse the feature simply to avoid participation point deduction, you can be rest assured that many students will understand that you are serious about them utilizing the content regularly and will honestly get more involved with reading the things you post. Ultimately, you know your time is not being wasted using Learn, and students will interact more with the content (and hopefully get more out of the course too)- sounds like a win-win right? Lastly, it would a good idea to address students on what you plan on using your Learn course for and what you expect from your students when they access it.

?Question: How accurate are the reports and to what degree should I use the reports to judge online course participation?

#Thoughts: While Evaluation Reports may not be as robust or detailed as let’s say Google Analytics, it is indeed accurate. Blackboard explains that a hit is tracked every time a request is sent to Learn. Now to the second aspect of this question is, what emphasis should you place on the statistics generated by the reports to actual student performance/grades? well, it’s entirely up to you. For a class that meets entirely in-person, the reports may be most useful for viewing what students find valuable in the way of content and possibly seeing why exceptional and poorly-performing students differ in how much they access your course and the materials in it. Now, for a class that meets mainly or exclusively online, judging attendance might be a bit harder. Depending on how your class meets, whether through Adobe Connect, Google Hangout, or a similar tool, it may be hard to provide an adequate environment for student engagement. A lot of factors such as time constraint, lack of video/audio equipment from students, etc. can minimize the chances to conduct class in the same way that is done in-person. For Purdue distance courses, the online Learn course space can contain the class’ entire framework: handouts, exams, readings, the list goes on and on.  Thus, to ensure students are engaging with all of these elements, it may be best to give the course reports feature a closer look.

!Solution: As stated in the thoughts, the helpfulness of the course reports depends on what you expect out of your students, which in turn allows you to judge how their activity in your course affects their grade. It is recommended to monitor the course reports regularly for student activity, which could explain their dedication to accessing course materials. As mentioned in the thoughts area, participation grades for online courses should rely more on the course reports’ findings, while the regular courses should use it more as a guideline for students’ attentiveness and dedication to using a Learn course.

?Question: Am I breaking any policy, Purdue and/or otherwise, by using this feature and what should I avoid when running reports?

#Thoughts: Running reports out of Blackboard Learn is enabled for any instructor/assistant role on any of the Purdue campuses, thus you can use them at your discretion. Since reports display students’ first name, last name, and username, along with their activity inside your course- it’s best to keep this information confidential. You can run a report for any duration in the course and for one or more students.

!Solution: To avoid violating FERPA, http://www.purdue.edu/registrar/FERPA/FERPA_.html , keep your course reports saved onto a password protected computer, device, and/or drive to avoid any loss of confidentiality. Also, printed versions should be kept in a protected environment as well. If you plan to share your course reports with students to let them know of their activity levels in your class, only select their name when producing a report. Also, if you want to share the report findings with your whole class, keep the statistics very general and do not point out any student or students when explaining activity levels.

Your first Course Report- All User Activity inside Content Areas

Now we are ready to discuss the actual creation of report, which takes just a few steps in Learn to produce. For this article, we will stick with the first report type and cover several other types in future posts in the blog. To get started, login into Learn and go to a course where you have Instructor access.

Locate and click on the Evaluation link in the Course Management navigation bar, then when the menu expands click on the Course Reports link.

The Course Reports page will appear. Purdue’s version of Blackboard Learn offers 8 reports and we’ll stick with the top most, “All User Activity inside Content Areas”. To get started running the report, hover over the name of the report and an option button will appear.

Click on the option button and a menu with a single link will appear. Click the Run link.

The Run Reports page will appear. Review the report information to confirm this is correct report to run. Interestingly, the Elapsed Time of Last Run data gives you information on how long this type of report took to create last time it was ran. This amount of time will vary depending on the start/end date, the amount of students in the course and the amount of activity they have produced.

Continuing on to section 2, Report Specifications, we get into the meat of what the report’s contents will display. First, you must select the type of format you want the report to be produced as. The major difference between the formats is that Excel does not display charts of user activity, but HTML, PDF, Word will. Sadly, the Excel format is choppy and hard to work with when trying to sort out activity data- thus it is NOT recommended and it is something Blackboard should look to improve.

Next, you must select both a start date and end date for the report. You can  select the first day of class as a possible start date and then extend the range as far as the current day. There is a handy button on the right of the field for picking dates as well.

Lastly, you’ll want to make sure you have users included in the report. If you click on no users, then the report will display data for everyone in the list. If you only need to see data on one individual or handful of people, you can left click and hold down the CTRL button on your PC or Mac to select users.

Once you’re set with the options, go ahead and click Submit. Depending on several factors, you may see the load screen shown below for up to a minute or two in extreme circumstances, though reports are generally created in 15-30 seconds.

What does a typically “All User Activity inside Content Areas” report look like with a lot student data?

Extensive, certainly. Classes that have over 300-400 students may have a report that extends beyond 70 pages of student activity statistics. Below are smaller images of each view you would see in this report and links to the full size images as well. The different views are shown below:

A pie chart of overall views for each content area page a course may have:

Note: Both pages and folders within pages will show up in the bar chart

http://web.ics.purdue.edu/~callaha0/piechart.jpg

A breakdown of each student’s activity on each content area page and possible folders:

http://web.ics.purdue.edu/~callaha0/userviewsperpage.jpg

A bar chart of each day’s total student activity in the range of dates:

http://web.ics.purdue.edu/~callaha0/barchartofdates.jpg

A breakdown of each student’s activity per month and day for all content area pages:

http://web.ics.purdue.edu/~callaha0/dayspermonthuseractivity.jpg

I hope this was useful and will encourage you to start exploring what course reports can do for your course(s). Please look out for a follow-up blog post here in the next 2-2-3 weeks that covers more questions and report types!