Multiple round tables in a brightly lit room, each with 7 design leaders in conversation.

AI is a ground-breaking technology, providing unforeseen capability and opportunity. But it is not without controversy, both ethically and technologically. AI presents design leaders with a quandary, requiring us to tread a fine line between what is acceptable and useful, and what is problematic and harmful.

I think it’s time for us to develop a confidence in this technology that will allow us to have more dominant, principled engagement with it in the future. It is in its infancy, but we have no time to wait.
Ovetta Sampson, keynote speaker – Director of User Experience Machine Learning, Google

This document is not a manifesto or an agenda. It is a series of prompts written by design leaders for design leaders, conceived to help us navigate these tricky waters. It does not attempt to provide any answers, but it does offer up critical questions to ask of yourself, your colleagues and your organisation when using AI* in your workflows and the products or services you are designing.

*For the most part we are using ‘AI’ as a shorthand for generative AI (GenAI) and the large language models (LLMs) it relies on. We believe the same critical thinking should also be applied to other forms of artificial intelligence such as machine learning (ML).

The questions

Product design

Considerations to ask your design team and product colleagues when integrating AI-driven features into products and services.

Is AI necessary to solve this problem? Consider the problem you are trying to solve or the human need you are attempting to address. Ask your colleagues whether AI is the right tool for the job, or whether there might be cheaper, better, more performant non-AI alternatives. Consider whether you are using AI to improve the product, or inserting an AI feature in search of a problem to solve. AI should be an enabler not an irritant.

How can AI elevate human ability? People use digital products and services to enhance their lives, and make things easier and quicker. Consider where introducing AI could make a positive difference in that regard.

What do we lose by including or excluding AI? Disregarding AI may be a missed opportunity for the product, but including it may be to the detriment of another feature, or the service as a whole. Think through the potential consequences either way.

Who are we leaving behind? AI can help, but it can also hurt. LLMs are trained using the morass of content on the web, which means they are trained on the biases inherent in the human condition. Consider how you will counteract and design against bias, prejudice and discrimination in AI output.

How do we manage unintended consequences of using AI? AI output can be wrong, misleading and potentially offensive. Ask yourselves what systems, processes and governance needs to be in place to mitigate and handle these situations. Should there be options to turn it off for individual customers or across your entire product?

Design practice

As a design leader you have responsibility for the success of your team. These are questions to ask when introducing AI-based tools into your team’s workflow.

What opportunities does AI bring to improve tools, practices and processes? Some AI-driven features make grand claims regarding productivity and creativity gains. Consider how appropriate these tools might be for your team, and whether they could provide useful efficiencies and enhancements in the context of your organisation.

What’s the role and responsibility of your designers with GenAI? If your team is using generative AI as part of their process, define what responsibilities you and your team need to take. Consider whether GenAI can, or should, provide the final output; whether it is solely an input into the design process, or in fact a straight replacement for the creative skills of a colleague. Think about when and where to introduce quality control of GenAI output within your processes.

How do you preserve the human touch in the process? AI can generate output quickly and tirelessly, but humans bring heart, soul, empathy, imagination and their lived experience. Think about where human craft should fit into your design processes, and whether you should deliberately add friction into the process around AI.

How do you avoid sameness and ensure innovation? GenAI essentially works as a huge statistical model. By definition it delivers the most likely response based on what it has seen before in its training data. AI can expose patterns we can’t see, but it can’t genuinely reason or actively make the cognitive leaps and connections required for true innovation. Consider how you keep originality in your team’s outputs and solutions – over reliance on AI could result in a regression to the mean.

Can AI free up your designers to solve critical business problems? Designers excel when given the latitude to solve problems. If AI’s promise of significant productivity increases holds true, consider how your team could spend a portion of their time bringing critical thinking to your organisation’s decision making.

Design leadership

You are responsible for the development of your team, and influential in its role at your organisation. These questions address the impact AI might have on those aspects.

Who are you leaving behind? GenAI could potentially ease the entry of junior designers into the profession, but at what cost? Curiosity, collaboration and critical thinking are key tools of the experienced designer – they help us simplify and humanise solutions to tricky problems. Consider how you’ll ensure newer members of your team will gain these attributes on the job.

What are the new baseline qualifications? Consider how you are shaping your team to design systems of the future. What does it mean to be AI-literate as a designer, and is that a requirement to work in the design teams of today and the near future?

How much does design’s role need to change with the advent of AI? Consider how much business value there is in the design process, and how you need to communicate it across your organisation in order to deliver that value with impact and meaning.

Who are our outcome-aligned allies outside of design? Consider how to avoid fetishising deliverables and feature-itis by focusing on the outcomes your products or services are intended to deliver. Collaborating with outcome-focussed colleagues in other disciplines such as product and customer services could be a solution.

How do you maintain humanity as a designer? Creativity and human understanding stand the possibility of being condensed into the role of a prompt engineer. Consider the role of your research colleagues, and how to keep the user in user-centred design.

Organisational impact

Your organisation’s appetite for AI may vary from a dictated mandate for use, to a cautious take-up among staff. Whether top-down or from within, the ripples will spread throughout. Use of AI anywhere within an organisation, whether in its processes or its products, presents questions that should be answered at a high level.

Are we prepared to stake our reputation on our use of AI? The current crop of LLMs does not come without their controversies and questionable ethics. Their significant energy consumption may not sit well with your organisation’s environmental commitments. Almost all models were trained on copyrighted material without permission of the intellectual property owners, including journalists, artists, authors and scientists. The models themselves contain inherent biases due to the nature of their training data – the good and the bad of the world wide web. What’s more, these biases can be tweaked in the direction of the political leanings of AI company owners, which may also run counter to your organisation’s values.

What are the short- and long-term consequences of AI use? Commercial LLM providers currently run at a loss – consider what strategies you have in place to mitigate critical reliance on a loss-making technology. Some AI companies are concerned that their own models are at risk of stagnation, having been trained on all available data, and are potentially now consuming their own output. How will you work around this?

How do we ensure AI literacy is distributed across teams and departments? If AI is becoming a fundamental aspect of your products or processes, an understanding of its capabilities and limitations will be necessary throughout your organisation. How will this be achieved and maintained?

What processes do we need to fix before applying AI? Before going all in on AI, ask yourselves what fundamental parts of your product or services need to be fixed first (and could AI help with that in the background?)

Now is the perfect time to start thinking about how AI can be democratized and decentralized. This is the moment to figure out how positive things can come out of it, while also understanding the human impact of the negative – the things that can go wrong. I think that’s the hardest thing for designers to do, because we always think about ‘here are the possibilities’, but not ‘here are the risks’.
TB Bardlavens, keynote speaker – Director of Product, Product Equity, Adobe Inc

The big picture

As with the Juvet assembly convened by Clearleft eight years earlier, the group found that some questions are too big, complex or nuanced to handle on your own. This is where you’ll need to gather your peers and consider the big picture presented by AI.

Is AI the pilot, co-pilot, or on autopilot – and when?
How can we tailor AI to local or specific needs, not just global?
How might AI help to increase equality, not create more disparity?
How might we develop AI in a community that is more diverse culturally, cognitively, professionally, and geographically?
How might we keep AI (and use AI to keep ourselves) rooted in human connection?