ACPHIS Medal 2003 Winner - Dr Anne Rouse
In 2003, Dr Anne Rouse was awarded the inaugural ACPHIS Medal after completing her PhD thesis titled "Information Technology Outsourcing Revisited: Success Factors and Risks" at the University of Melbourne.
About the award winning thesis
Dr Rouse's research established that IT outsourcing involves substantially more risk than had previously been acknowledged, and only rarely results in substantial cost savings. She found that many purchasers failed to obtain expected benefits not because of poor management, but rather because of inherent risks and a failure to recognize trade-offs. The influence of several recommended practices were explored, revealing that while some widely recommended practices (such as “selective” outsourcing) did not have the expected effects, benchmarking, and service level management practices did improve outcomes.
This thesis examines the likelihood of success, and the risks associated with Information Technology (IT) outsourcing arrangements. It also examines the trade offs that are associated with the strategy, and the impact of certain recommended practices. While previous IT outsourcing studies have tended to adopt a case study approach, in this thesis a multi-method strategy is used within a post positivist epistemological framework. This approach marries qualitative research (a detailed single case study, and a series of 16 focus groups) with quantitative strategies, including structural equation modelling (SEM) and confirmatory factor analysis (CFA). The quantitative studies employed a range of statistical hypothesis testing; the qualitative case study and focus groups used a hermeneutic analytical strategy, which relied on the author’s own interpretations of informant reports, and (in the case study) public and press documents. The thesis involved four research components:
Literature review - A critical review of ten years’ literature on IT outsourcing was undertaken, paying particular attention to evidence supporting the likely success of the strategy, and the impact of recommended practices on IT outsourcing success. The author concluded that the dominance of inductive case study research, and the virtual absence of substantial statistical studies limited understanding of the extent to which case study findings generalised to wider populations. This limitation casts doubt on some of the widely-accepted understandings about the rates of IT outsourcing success and risks, and the impact of certain recommended practices. The literature review also established that findings related to success and risks associated with outsourcing, and the development of recommended practices, have emerged largely from studies conducted early in the 1990s. Comparatively little work has been done since the widespread introduction of the Internet and multi-tier technical platforms. As part of this review the author re-analysed cross-case comparisons conducted by earlier researchers (Lacity and Willcocks, 1998), illustrating that alternative conclusions could be drawn from those authors’ quantitative analysis of their cases.
Statistical analysis - This second component of the study involved statistical analysis of a survey of government and non-government organisations (n = 240) taken from the largest 1000 organisations in Australia. Almost all of these outsourced some IT services. Both descriptive and analytical statistics were applied to analysis of the survey. These included structural equation modelling (SEM) and confirmatory factor analysis (CFA) – using AMOS software; multiple analysis of variance (MANOVA); and several non-parametric statistics. To avoid the ambiguity associated with many different success items, the study examined the issue of success measures in depth. This involved distilling 30+ survey items into a set of relatively robust measures, and clarifying the relationship between them. As part of this analysis the outsourcing success scale developed by Grover, Cheon and Teng (1996) was examined, but found not to be unidimensional. A more comprehensive measure was subsequently developed and subjected to CFA. The analysis resulted in a reliable instrument composed of 7 inter-related dimensions of IT outsourcing success. This instrument enabled the author to more clearly establish the rates of success in the sample, and to test, statistically, the effects of certain practices. Initial analysis indicated the major benefits of outsourcing were access to skilled staff and vendor service, but not cost savings. Only 7% reported substantial cost reductions from the strategy.
Outsourcing success and risks - The relationships between the 7 dimensions of outsourcing success were examined using SEM to better understand these findings. This analysis indicated that evaluations of IT outsourcing satisfaction and value (satisfaction/value) were directly predicted by two constructs: the services provided by the vendor (vendor service), and the strategic benefits conferred on the purchaser (strategic benefits). Cost reductions did not directly predict IT outsourcing satisfaction/value, but did influence this success measure indirectly. Hence the fact that few respondents reported substantial cost reductions for outsourcing accounts for some of the lack of satisfaction/value. Another construct, access to skilled personnel, indirectly predicted outsourcing satisfaction/value through its effect on vendor service. Two additional predictor variables were technical benefits and economies of scale. The six predictor variables accounted for 67% of the variance in the criterion variable, IT outsourcing satisfaction/value. Only access to skilled personnel and vendor service were scored positively by the majority of respondents, whereas the other five success measures (satisfaction/value, strategic benefits, cost reduction, technical benefits, and economies of scale) were scored positively by only a minority of respondents. Just over one third (36%) of respondents reported a positive score for the criterion variable, satisfaction/value. The author concluded from the quantitative analysis that a range of IT outsourcing risks are more widespread than has previously been suggested. She also concluded that the low reported levels of many aspects of outsourcing success imply that outsourcing is much riskier than the literature would suggest. The structural model suggests the major problem areas are purchasers’ achievement of cost reductions, and their capacity to convert the services offered by vendors into strategic benefits.
Effect of practices - The effects of certain “best” practices on the identified dimensions of success were also examined. The research confirmed the effectiveness of some, but not all, recommendations related to clear specification of requirements (through service level agreements and benchmarking). It established that the benefits of these practices are statistically generalisable to a much wider group of purchasers than the small number of cases studied to date. Particular attention was paid to the notion of “selective” outsourcing, which has received much attention in the literature. The research failed to substantiate the claim that selective outsourcing (of between 20% and 80% of IT budget) is more successful than “total” outsourcing (80%+). There were, however, some facets of success where selective outsourcing led to greater benefits than minimal outsourcing (of less than 20% of IT budget). Overall, the survey revealed that while the prescribed “best” practices – e.g. selective sourcing, short term contracts, benchmarking and service level agreement (SLA) practices – were widely adopted in the IT community, these were necessary, but not sufficient, for outsourcing success.
Qualitative case study - A detailed case study was carried out into the Australian Federal Government’s “Whole of Government IT Infrastructure Outsourcing Initiative”, focussing on issues of success and risks. This initiative involved a complex outsourcing arrangement that met the definition of “selective” given by Lacity and Willcocks (1998). The case study employed hermeneutic analysis of a range of text documents and field notes. The study illuminates the findings of the quantitative analyses by illustrating why expected cost savings were not achieved. It also establishes that outsourcing required considerably more managerial attention than had been expected by decision-makers. The case study illustrates how certain recommended practices were more difficult to implement in the field than the literature would suggest and how the strategy involved a range of inherent trade offs that could not easily be resolved. It also demonstrates that certain recommended practices may have unexpected negative consequences. In addition, the case study illustrates how heavy promotion of outsourcing’s benefits, and the potential of recommended practices to ensure success, may have led decision-makers to under-estimate the risks involved in the venture.
Qualitative focus groups - The fourth study involved qualitative analysis of 16 focus groups made up of vendor and purchaser informants. These focus groups shed light on the findings of both the case and quantitative studies. They also help explain why the experiences of IT outsourcing are often negative, and why certain practices did not necessarily lead to outsourcing success. Together with the case study, the focus groups highlight the importance of trade offs implicit in IT outsourcing.
Integration and conclusions - Drawing on all four research components, the thesis supports the more cautionary view of outsourcing presented by Lacity and Hirschheim at the beginning of the 1990s. The research provides evidence that many of the difficulties identified then still occur, despite the widespread adoption of strategies to limit risk (like selective outsourcing, service level specification, benchmarking and shorter contracts). The thesis also highlights new difficulties associated with the outsourcing of emerging technologies. The author concludes that IT outsourcing remains considerably riskier than has been promoted in the trade, and academic, literatures, and that while certain practices are important to the success of outsourcing arrangements, they are not sufficient to guarantee success. The author also proposes that “information impactedness” associated with post-Internet technologies and skills shortages, accounts for some of the failure of organisations to reap the benefits expected, particularly cost reductions. The research has demonstrated that that there are still inherent trade offs in IT outsourcing. The author argues that it is therefore critical that decision makers acknowledge and address these trade offs explicitly, so that expectations are more realistic. This should lead to more satisfactory and successful outcomes. The author also recommends that organisations devote more attention to risk analysis, benefit projections, and to the early evaluation of initial incremental forays into outsourcing.