By Daniel Maxwell Despite the fact that educational technologies have been commonplace in schools for over 20 years, there remains rigorous debate regarding how effectively these technologies support learning, with the international media giving regular attention to reports which indicate technology has a negative impact on education. The difficulty of evaluating the impact of innovations in complex educational environments in which multiple factors simultaneously influence learning outcomes has long been acknowledged by educators and researchers.

Assessing the impact of technological innovations in the learning environment comes with additional challenges because these technologies are integrated and applied alongside other learning strategies, as Bodilly and Mitchell explained, the “complex environments in which technology projects are embedded make inference of causal relations between project activities and outcomes tenuous": Much of the early research on educational technologies was conducted to measure their impact on student attainment and many of these studies recorded only negligible improvements.

Sipe’s study from 1997 and Weaver’s study from 2000 are typical of the research from this period.

Weaver’s report concluded that computer use makes very little difference to student achievement, while Sipe’s research concluded that, "when compared with typical effect of innovation on educational achievement, computer innovations are not that different from the average innovation."  These early reports focused on measuring the extent to which technology influenced student attainment in traditional assessment, often neglecting a wide variety of additional learning outcomes such as learner motivation, digital literacy, student interaction and the potential of these resources to develop higher order thinking skills.

Assessing the value of technology by simply measuring student attainment in traditional assessments focuses on too narrow a set of objectives, and this approach has been criticised by numerous academics, including Joy and Garcia, who, in their report ‘Measuring Learning Effectiveness: A New Look at No-Significant-Difference Findings Ernest’, argue that that this approach is flawed and that the findings from these reports are often inaccurate. Over the past decade researchers have begun to focus on a wider of variety of outcomes from educational technologies by looking at the impact on specific areas of learning such as mathematics and literacy, as well as the impact of technology on student motivation, remedial support, differentiated learning, and the development of higher order thinking skills.

However, the conclusions from these reports have also been mixed, with many reports indicating the application of technology in the classroom continues to result in only negligible gains. In an attempt to summarise the existing literature and draw broader conclusions on the impact of technology in the classroom, numerous researchers have initiated meta-analysis studies.

The meta-analysis studies conducted by Soe in 2000 and Bayraktar in 2002 concluded that computer-assisted instruction (CAI) had a small but positive effect on students’ reading achievement and was effective in science education – indicating there were areas where the adoption of technology was more effective than others.

These conclusions have been supported by more recent research such as Higgins’ study from 2012, which concluded that over the last forty years digital technologies have had a positive impact on learning, and Cheung’s study from 2013, which indicate that computers in the classroom produce small positive improvements in reading, and modest improvements in mathematics. Higgins’ 2012 report also identified a number of circumstances in which the use of technology had been found to be particularly effective -

1. Collaborative use of technology rather than individual use;
2. Remedial use of technology with students who had, lower levels of attainment;
3. As a supplement to normal teaching rather than a replacement;
4. By subject, attainment tended to be greater in mathematics and science.

The results from these reports are useful for teachers hoping to utilise digital technologies more effectively, but there is still progress to be made. A greater willingness to recognize and embrace the potential of educational technologies is needed to ensure that learners fully benefit from the opportunities digital technologies present.

Currently, much of this potential remains untapped as schools continue to use ed-tech for simply reinforcing curricula content by having students complete ‘drill and skill’ activities.

If schools continue to employ these tools to simply improve basic skills through automated practice, the impact of technology will remain modest. In order to assess the ability of technology to support the development of 21^st Century skills, it is essential that researchers broaden their perspectives on educational technology and develop new models for assessing these resources.

This can be done by redefining the purpose of educational technologies and developing of a set of learning outcomes which encompasses a more diverse range of 21^st Century skills.

By embracing more ambitious learning outcomes for technology supported learning, greater rewards are guaranteed to follow.