Episode 2.
Leaders in all types of organizations,
education included, are concerned with performance.
They want to measure the progress that they're making towards the strategic goals they've defined for the organization.
They want to measure the degree to which individuals and groups are contributing to the logistic effectiveness and inefficiency.
They want to know if the resources they dedicate to systems and initiatives are worth the cost.
The nature of the data leaders use varies and business data are usually monetary,
but increasingly other measures are being used as well. In education, the data are almost always test scores.
The appropriateness of those tests as a measure for what really matters in education is actively debated,
but we're not going to debate that here.
The examples I've mentioned so far are of course quantitative. They deal with numbers, but many researchers also value qualitative data.
What people say when they're describing their experiences is valuable information for leaders to know as well.
While the IT folks in schools may have their opinions,
it's really not up to them to decide what data to collect, how to analyze it, or how to report it.
They do help manage databases and generate reports and perhaps give advice on how to effectively present data,
but educators decide the data to use and its interpretation.
So IT leaders in schools do have a responsibility to support their leaders' use of data,
but they also have a responsibility to use data to make their own plans and decisions,
especially with regards to system design.
We want to document and understand the degree to which our systems are helping people to do their jobs.
Fortunately for IT leaders in schools, there's a long tradition of using data to support those decisions,
and there's a rich research tradition that has elucidated models to help us understand what really matters when we're designing IT for users.
Of course, the default option for data collection in schools and most other organizations
is to ask folks to fill out a survey. The quality of that data is always suspect.
We ask questions that we think matter, but they might not.
We're careful to avoid bias in the questions that we ask, but we can never catch it all.
We usually ask everyone to answer the survey, but response rates are poor and those who do respond typically represent a special case.
Either they really like what we're doing and want to tell us about it,
or they really dislike what we're doing and they want to tell us that as well.
By adopting and perhaps adapting the instruments used by technology researchers,
we as IT leaders do enjoy several benefits.
First, the instruments have been tested for validity and reliability.
They really do measure what they claim to measure, and higher scores on these instruments really do correlate with better systems.
Second, the instruments have a theoretical basis.
If scores are low on these measures, then we can usually gain insight into what we can do to improve the performance of our systems.
Because we can recognize and understand the theory behind the measures,
we're more likely to actually improve performance rather than simply improve scores on the test.
And that is not something that we can do if we're using a survey that we've made ourselves.
In my experience, the technology acceptance model is a theoretical model that translates into easy data collection
and that can identify aspects of our IT systems that must be improved.
The technology acceptance model was first described in 1989,
and it has been refined and updated several times in the intervening years,
and it has been adapted for various populations and purposes.
I typically use the unified theory of acceptance and use of technology,
which was described in 2003 when I want to gather data about the IT that I manage.
According to the UTAUT, people tend to use technology when four things are true.
First, if they perceive it to be easy to use.
Second, if they perceive it to be effective.
Third, if others whom they respect use it or similar systems.
And fourth, if it's well supported.
Now, of course, I'm summarizing what's really an extensive body of research.
And my language is a little different from that which is used to define it,
but the ideas are the same.
If you're interested in the original research, head on over to my website at hackscience.education.
Follow the link to my blog in search for UTAUT.
Several posts have links to the papers in which this work was originally elucidated.
One common element of all variations on the technology acceptance model
is a focus on users' perceptions.
If you as an IT leader or technician believe that the systems are easy to use,
but your users do not, then it is not easy to use.
This can be hard for many IT folks to hear, but that perception is reality.
Now, technology acceptance is also based on the assumption
that these factors will positively affect people's intention to use technology,
which then affects their actual use of technology.
In many school settings, the use of IT systems is not optional.
The teachers use a grade book and the student information system that they've been directed to use,
and they don't have an option to not use it.
Interestingly, technology acceptance can influence design decisions for those compulsory systems as well.
If we customize a grade book, for example, so that it aligns with the factors associated with technology acceptance,
then teachers are likely to dig a little deeper into software
so that it more efficiently and effectively captures and communicates student progress.
In a later episode, I'll dig a little deeper into the unified theory of acceptance and use of technology
and perceived effort expectancy, performance expectancy, social influences, and facilitating conditions.
Those are the terms that were used when the theory was originally defined.
I'll do that in a later episode, however.
In this episode, I hope you understand that IT leaders in schools, as well as in other organizations,
should design systems to align with ease of use, effectiveness, social influences, and facilitating conditions.
You can save the time of writing surveys by using instruments that have already been developed to measure technology acceptance.
I haven't mentioned this yet, but you can also adopt researchers' approaches to sampling
by choosing a small random sample and get data that's more representative of the whole population
than just asking everyone to respond and having those people who are most interested responding.
You can also integrate technology acceptance into your conversations with users and your interpretations of their complaints.
Ask what changes could be made to make the system easier.
Listen to the changes that they say they need in order to do their job.
Listen to what they say about others who do the same work that they do and the systems that they have.
One last thing to remember about technology acceptance for this episode is that communication has important influences on perceptions.
Of course, ineffective communication is a problem for all departments and in all organizations,
but school IT leaders who take the steps to tell others what they've been doing
and also who listen to what users have to say are much more likely to have systems that are used.
And isn't that why we do the work that we do?