Ensuring Online Usability and Value:�A comparative analysis of the design standards, criteria and governance models in Denmark, Portugal, and the UK

Utilising the recommendations of the two most quoted online us- ability experts, Jacob Nielsen and Ben Shneiderman, as an analytical framework, this paper compares the approaches of Denmark, Portu- gal, and the United Kingdom (UK) to online service design standards and usability over time. The paper finds that all three countries address the recommendations of Nielsen and Shneiderman, albeit with varying degrees of detail, support, and compliance assurance. The approach to the design standard and usability criteria have changed in the Danish case, whereas in Portugal governance and compliance approach have been altered. The Danish and UK ap- proaches to design standards and usability requirements stand out as particularly successful in their reach and ability to improve online user-friendliness of content and transactional services. These two countries stand out as global leaders in their own right. Denmark shows how to reap the benefit of IT and technology investments in public sector services delivery through a combination of strong mandates, coordination, and collaboration across all levels of gov- ernment to achieve its strategic objectives. With a similar effect, the UK has set a new global standard for the design of public sector service offers online, but the impact is limited by its focus on central government services.

1. United Nations University
Operating Unit on Policy-Driven Electronic Governance
Ensuring Online Usability and Value:
A comparative analysis of the design standards, criteria and
governance models in Denmark, Portugal, and the UK
MortenMeyerhoff Nielsen andLuís Nuno Barbosa
UNU-EGOV
ICEGOV 2024, 1-4 October 2024, Pretoria (ZA)

2. 0102030405
Why is this topic
and these research
questions?
Research
Questions
What does existing
literature say, what
is the relevant
methodology and
case selection?
Conceptual
FrameworkWhat can we learn
from the case
studies?
Findings
What can we take-
away from this
research?
Recommendations
What can we do
next for even more
insights?
Future Research

3. 01
Why is this topic
and these research
questions?
Research
Questions

4. •The introduction of public sector digital services are intended to improve the access and service, the level of quality, cost-efficiency of delivery and good governance.
•To date the envisioned outcomes have not been materialised, as services often underutilised by the intended users.
•Research points to a lack of awareness, perceived trust, perceived quality, and as a result a lack of behavioural change.
•The introduction of digital services the calls for a user-friendly online experience to facilitate behavioural change.
•Better usability improves citizen use and satisfaction with public services and thereby the return on investment.
•Importance to align with international standards (e.g., the UN Convention on the Rights of Persons with Disabilities).
•Usability addresses accessibility, inclusivity, and effectiveness of governmental digital offerings.
Relevance of Online Usability in Public Sector Services

5. RQ 1
RQ 2
RQ 3
Which, if any, assessment criteria do design standards and usability guides
apply to monitor and measure online service usability?
What, if any, are the governance and compliance approaches applied?
How have the criteria and approaches changed over time in the context of the
selected case studies?
Research Questions

6. 0102
Why is this topic
and these research
questions?
Research
Questions
What does existing
literature say, what
is the relevant
methodology and
case selection?
Conceptual
Framework

7. •Literature review: A classical review of the literature to identify finding within academic and grey literature is combined with the analysis of policy documents, actual service design and usability standards and guidelines from the public sector.
•Conceptual lens: J. Nielsen and B. Shneiderman are renowned usability experts whose principles guide effective user interface design in the 1990s and early 2000s. The patterns of user-behaviour identified, and the principles synthesised, though their respect research have proven to be universal and applicable to both online information, portals, websites, apps, and transactional services. The patters identified can be linked to non-digital user-behaviour such as reading and visual perception patterns.
•Case selection: The analysis is based on three case studies (Yin, 2003). Denmark, Portugal, and the UK, selected for their diverse approaches, varying levels of compliance and usability frameworks. While operating in similar contexts (e.g. same continent, high-income, developed welfare states) they are also have applied different approaches, introduced changes over time. Lastly the UK case was the first public sector service design standard introduced globally, Denmark have in the last 20 years consistently ranked in the global top of digital government, while Portugal have performed solidly.
•Combining theory and practice: The focus is on practical applications and real-world implications of usability guidelines in public services. Using Nielsen and Shneiderman as the conceptual lens for a metasynthesis allows for such an approach.
Conceptual Framework

8. Case Study Selection

9. 010203
Why is this topic
and these research
questions?
Research
Questions
What does existing
literature say, what
is the relevant
methodology and
case selection?
Conceptual
FrameworkWhat can we learn
from the case
studies?
Findings

10. •All three countries incorporate Nielsen and Shneiderman's principles in their usability frameworks.
•All three cases incorporate the usability standards within a larger framework that is focused on digital service
delivery (the Denmark's joint-public digital infrastructure universe, the Portuguese Mosaico, and the UK's Service
Manual).
•Denmark and the UK have more detailed approaches leading to improved online service usability.
•Portugal incorporates basic principles with a focus on checklists but is less comprehensive.
•The UK sets a global benchmark for public sector service design.
Findings – Assessment Criteria for Usability

11. •Denmark and the UK mandate compliance for national authorities, fostering intergovernmental cooperation, but
while Denmark emphasizes local government inclusion, in the UK local compliance is voluntary.
•Portugal's voluntary compliance model encourages reputational pressure but lacks the robustness of mandatory
systems.
•Strong governance structures in Denmark and the UK seem to promote proactive service delivery and user-centric
design.
•Both Denmark and the UK have a strong compliance model, although the Danish model is leaner and less
expensive, as it assesses the result and not each phase of the process.
Findings – Governance and Compliance Approaches

12. •Denmark and the United Kingdom have moved towards simplifying the guidelines and focusing on the assessments
to be performed on the usability outcomes.
•The compliance models have not changed significatively in Denmark or the UK.
•The initially relatively vague Portuguese guidelines have increased levels of information and additional checklists
and tools have been developed.
•Portugal seems to have felt the need to create a compliance model, even though it is voluntary, creating a seal of
compliance that seeks to enhance user experience, while still lacking a pre-launch screening.
Findings – Evolution of Criteria and Approaches

13. 01020304
Why is this topic
and these research
questions?
Research
Questions
What does existing
literature say, what
is the relevant
methodology and
case selection?
Conceptual
FrameworkWhat can we learn
from the case
studies?
Findings
What can we take-
away from this
research?
Recommendations

14. •Establish governance and compliance structures with clear accountability for public sector service usability.
•Foster cross-governmental collaboration to enhance compliance with usability standards, the reuse of components
and government data, on a "once-only" principle.
•Engage key public and private sector partners in the design and implementation process fosters co-ownership and
a shared understanding of usability criteria.
•Create channels for feedback from authorities, vendors, and users to address issues promptly and continuously
improve standards and tools.
•Adopting lean models like Denmark’s seems to be able to reduce costs without compromising on usability.
Policy Recommendations

15. 0102030405
Why is this topic
and these research
questions?
Research
Questions
What does existing
literature say, what
is the relevant
methodology and
case selection?
Conceptual
FrameworkWhat can we learn
from the case
studies?
Findings
What can we take-
away from this
research?
Recommendations
What can we do
next for even more
insights?
Future Research

16. •Expand the analysis to other countries comparing their standards and compliance models.
•Compare and assess the cost effectiveness of different compliance models, focusing on their usability outcomes,
from the cost-heavy ones, like the UK, the leaner ones, like Denmark, and the reputation-based voluntary ones, like
Portugal.
•Conduct in-depth interviews to key stakeholders to gain insights on the reasoning behind each change in the criteria
and compliance models.
Future Research

17. MORTEN MEYERHOFF NIELSEN
United Nations University
Operating Unit on Policy-Driven Electronic
Governance
Campus de Couros, RuaVila Flor 166, 4810-445
Guimarães, Portugal
Mobile +45 23 92 22 91
[email protected]
Twitter @mortenmeyerhoff
LinkedINmortenmeyerhoff
Webhttp://egov.unu.edu/
LUÍS NUNO BARBOSA
United Nations University
Operating Unit on Policy-Driven Electronic
Governance
Campus de Couros, RuaVila Flor 166, 4810-445
Guimarães, Portugal
Mobile +351 914 027 028
[email protected]
LinkedIn luisnunobarbosa
Webhttp://egov.unu.edu/