I originally wrote and published this post on Normative Design’s Ratsnest blog as the second part of a two-part synthesis of Design+AI, a monthly meetup hosted by Normative Design to explore how we design in a world augmented by AI. On the Sept 21, 2017 edition of the meetup, Chris Noessel, author of Designing Agentive Technology, joined us to shed some light on the area of designing agentive technology; specifically:

• What is agentive tech?
• What is narrow AI versus general AI?
• How do designers need to modify their practice to design agentive tech?
• Do agentive tech pose unique operational burdens?

Part 1 of my recap covered the questions of what agentive technology is, along with a discussion of narrow versus general AI. Part 2 ventures into the pragmatic territory of what it all means for us as designers in the field.

What changes in our practice? The good news is: not a lot.

So, from a practice perspective, what are the important differences when we design for agentive technology? What parts of our design practice — process, approach, methods, and perspective — need to be adapted? What should we be thinking about and watching for? What are the gotchas?

Chris notes that what’s missing from his book Designing Agentive Technology is explicit commentary on what changes in our practice. The good news is: not a lot changes. Research should be handled identically. We still need to go out and talk to people who will be served by agents. We still need to understand how the thing that’s in question gets done. We still need to understand the goals of the people using the agent to get things done: none of that changes.

That said, Chris sees three areas that do need to change, two of which are near term, and one that’s longer term.

1. Asking about the future

During research, ask how the future would be like for the user. This is an optional question when designing tools because we tend to face limitations in what we can do for people. However, when designing agentive tech, this is the mission. Let’s imagine we’re designing a solution for our customer Carla. We could frame up a question along the lines of “Carla, if we had infinite money to hire you an assistant who’s infinitely knowledgeable and incredibly fast, how would you hope that assistant would help you?” This uncovers what people don’t like doing — stuff that we can solve with agentive solutions. Another question we could ask might be “What would you still do?” This will uncover what people love doing, tasks or outcomes that we probably shouldn’t give to an agent (perhaps an assistant would be better, something to help people do it themselves). Or, if we do want to design an agentive solution for things that Carla enjoys doing, we need to take care in designing the agentive solution. These types of questions tend to be on the tail end of our normal research routine; with agentive technology, they’re going to be more front and centre.

2. Multiple design layers, leading to more complexity

Chris acknowledges that in his book, agentive tech comes across like a pure thing. In practice, most sophisticated technologies are in layered modes: there’s an agentive mode that sits near an automated mode that sits near an assistive mode that sits near a tool mode that gives back control to humans. For example, during his research on the robo-investor work, one of the research participants wanted to set aside 10% of his theoretical money to see if he could beat the AI because there might be something special in humans that might give us a shot. In reality, there probably aren’t pure agents or pure automatic things, or not too many of them.

In designing agentive tech, we’ll need to keep in mind four modes:

• There are times when we want agents to handle things routinely and alert us if we need to make decisions or if there are problems;
• There are times when we only want to be notified when things are fundamentally broken;
• There are times when we want to do it and want help; and
• There are times when we want to do it ourselves and don’t want help.

These four modes are likely to be present in sophisticated and certainly enterprise-level projects, and thinking through them is going to be complicated. After research, our modified design process should work through these four modes. First, design it as a manual tool — how would Carla do it herself with no help. Then, add the assistive layer: how does the AI help her do it and how does she turn it off if she wants to? Then, how might we make it agentive so that we can take the burdensome or routine parts off Carla and let her pay attention to the things that she loves. Finally, what does the autonomous mode look like? If the engineering team is confident, how might we let Carla pass it off into autonomous mode and how does she re-gain control? These are complicated layers to design for, but they’re necessary.

3. Interaction design will get more complicated when users and agents proliferate

In the longer-term future, user and agents will proliferate. And they’ll interact with one another and likely with peer groups outside their immediate domains. These interactions will make design more complicated. How will we account for the touch point experiences and the outcomes when users and agents talk to one another? For instance, Chris wonders if his health agent going to rat on his insurance agent? I expect the answer likely lies in who paid for the creation and maintenance of the agent.

Selling agentive tech

Conceptually at least, business leaders like agentive tech: there are interesting opportunities for up-sell, cross-sell, lowering the cost of human resources for routine tasks, etc. In practice, getting funding is harder. The response is typically, “Cool idea! OK, let’s get back to building this piece of junk.” How do we get past this inertia?

Find an executive champion

Chris suggests that we use the same design methodology he discussed earlier. First, design the manual solution. Get agreement that the manual solution solves the pains for the user and for the business. Once we have that agreement, we can suggest doing it one better with an agentive solution. So, for example, instead of the tool that Carla could use to locate the information she needs, with the agentive solution, the information will come to her, although she’ll have options for varying degrees of intervention. If we can get to an agreement on the benefits that the agentive mode provides, we can then talk to engineering about the costs associated with the four different modes: tool mode, assistive mode, agentive mode, autonomous mode. Expect engineers to prioritize the tool over the agent, so internal champions are going to be mission-critical.

First mover advantage

In Wired for War, P.W. Singer talks about how technology is influencing the world’s concept of war. One of the concepts is “threshold technology”. For example: once a culture adopts drones for warfare, why would it ever risk flesh and blood again? In general, once we adopt a technology, we’re loath to go back. Drones are an example of a threshold technology. This notion of threshold tech applies to agents. Once you have a Roomba, pulling out the Dyson feels like a chore. The Roomba is a threshold tech.

Likewise, agentive tech is a threshold concept. It confers a competitive advantage. The first player in a domain to introduce functional agents to their users will have their attention and loyalty, or from my perspective, at least establish the inertia that prevents customers from switching or leaving. If business leaders miss out on the first mover advantage, they’ll spend time scrambling to catch up and lose market share.

Operationalizing agentive tech

What exactly does it mean to build agentive technology? In practical terms, it doesn’t require fancy specialized coding skills. What we’re building is software to watch data streams for triggers and then to act on those triggers. Devs need to design triggers and watch data streams for a set of conditions or scenarios that designers design to be smart as a default, but design controls and flows for users to modify triggers and conditions or scenarios. There are then triggers and behaviours to enact, which is at the core of object-oriented programming.

To have humans watch for conditions and triggers, then perform the pre-defined acts is orders of magnitude greater than the server space required to house the software. Delivering this continuousness is more pragmatic and feasible using software than with manual human labour. In his book, Chris cites the Mackworth Clock Test, which taught us that even if we wanted to use humans to provide this persistent service, it isn’t something the human brain is wired for:

People are terrible at vigilance. Back in the 1940s, a researcher named Norman Mackworth made formal studies about how long people could visually monitor a system for “critical signals.” He made a “clock” with a single ticking second hand that would jump every second, but which would irregularly jump forward two seconds instead of one. He would then sit his unfortunate test subjects in front of it for two hours and ask them to press a button when the clock jumped. Then he tracked each subject’s accuracy over time. It turns out that, in general, people could go for around half an hour before their vigilance would significantly decay, and they would being missing jumps. This loss of attention has little consequence in a testing environment, but Mackworth was studying the vigilance limits of WWI radar operators, when missing a critical signal could mean life or death. ( Noessel, Designing Agentive Technology, Rosenfeld Media, 2017, p.142)

From an engineering perspective, the challenge of agentive software isn’t an issue of skills but operational resources. Agents, it’s argued, can be operational sinkholes. Crashed servers and other downtime conditions require human intervention. And if we built agents to mitigate that human intervention, that could spiral into meta problems, so the argument goes. Chris points out that these problems exist today — servers go offline, software gets deprecated — we just offload them to users. We don’t need to reach far for examples: the recent release of iOS11 created an ‘appocalypse’ of apps built with 32-bit technology. Unquestionably, agentive tech imposes an operational burden, but to a large extent, every improvement for users implies some operational burden. Ops costs need to be managed, not seen as an obstacle or reason to avoid using agentive technology to improve the human experience.

Intelligence is a multivariate space

Humming below the surface of every Design and AI meetup is the fear of monolithic AI that pits human intelligence against artificial intelligence. Chris argues this is a false dichotomy. Moravec’s Paradox showed us that “contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources.” Put differently, machines / computers are better at doing some things better than humans, while humans are better at other things. Chris notes that the mistake is in thinking about it as a problem of simple task allocation; it’s more complicated than that. Citing Howard Gardner’s (contentious) hypothesis of the theory of multiple intelligences, Chris prefers to think of intelligence as a multivariate space and proposes that if we design an AI to mimic human intelligence, we’ll have missed the point.

We tend to think of an over-arching agent such as a self-driving car rather than the component parts that deliver the self-driving outcome; it’s certainly a more saleable concept — after all, people buy benefits, not features. Put it differently, agency is multi-part: we confer agency to software over a series of tasks rather than agency over an outcome or job-to-be-done. Ultimately, how much agency is given to software should be determined by user preferences. In a future when self-driving cars dominate, it’s reasonable to assume that we will want to impose some degree of practice time to ensure our driving skills don’t atrophy into a dangerous state where we aren’t able to re-take control from the software agent. It’s also reasonable to assume that individuals will want to set their preferences for how much practice they want: the utilitarian will want the minimum recommended amount while the enthusiast will want more.

Chris expects that our agentive design practice is more likely to start at the small bits and then get grouped into larger bits. Some agents’ scope will be so small that they won’t be perceived as agents at all. Take blinkers in a car, for instance: they work with a single touch; the salesperson at the auto dealership doesn’t sell it as a feature, as an automatic, agentive blinker. And yet, behind the single human touch that engages the blinker are a series of tasks and processes that get triggered. It’s only when something gets to behaviour that feels ‘more human than I thought technology could do’ that we need to have a conversation about it. Until then, software agents are just improving technologies that will likely be expressed through APIs and networked communications, rather than a singular monolithic expression. The scope of artificial intelligence is still too ill-defined to get to an infrastructure of agents from the top down. It’s more likely to be emergent: lots of small, well-contained agents will be released; things will either break down or people will introduce coordinating agents that becomes, de facto, a weird war of escalation that will start to drive top-down organization.

Instead of the human versus artificial construct, framing our design practice space as narrow AI versus general AI offers more pragmatic, productive opportunities to explore this technological frontier responsibly. Chris’ hypothesis is that narrow AI will get safer as it gets smarter, while general AI will get more dangerous as it gets smarter — that’s the existential threat that Hawking and Musk refer to. Narrow AI offers opportunities for more people to contribute to the field of AI, and this democratizing effect is critical:

Don’t think about one agent. Think about all the agents, their capabilities, and their rules. We will be building a giant, worldwide database of behaviors, rules, and contexts by which we want to be individually treated. That stack would be impossible for any human to make sense of, but it might be the exact right thing to hand to the first artificial general intelligence.

Maybe in working on triggers and rules, we’re building the Ultimate Handbook of Helping Humans, one rule at a time. Agentive technology may be the best hope of ensuring general AI doesn’t end up being our Great Filter. Instead of [Asimov’s] four laws of robotics, we’ll have four trillion laws of humanity. (Noessel, Designing Agentive Technology, Rosenfeld Media, 2017, p.187)

A final call to action

Designing agentive solutions means shifting “our day-to-day practice from building tools for people to do work themselves to building usable, effective, and humane agents.” And to do that effectively, ethically, and purposefully,

[w]e must build a community of practice so that we can get better at this new work .To build a shared vocabulary amongst ourselves to have good and productive discussions about what’s best. To share case studies, analysis, critiques successes, and yes, failures. To get some numbers about effectiveness and return-on-investment to share with our business leads. To push these ideas forward and share new, better development libraries, documentation techniques, and conceptual models. To reify the ways we talk to others about what it is we’re building and why. (Noessel, Designing Agentive Technology, Rosenfeld Media, 2017, p.192)

I couldn’t have expressed the purpose of our design and AI meetup better than Chris. So, if you’re working on software ideas that can help people, join us.

Disclaimer: The ideas and opinions expressed in this post are my synthesis of Chris Noessel’s session at a Design and AI meetup hosted by Normative, where I work. My views are not necessarily those held by Normative nor by Chris Noessel, and any technical errors or omissions are mine alone.

References

http://rosenfeldmedia.com/books/designing-agentive-technology/
https://medium.com/@christophernoessel/ani-design-skills-f0af22360570

https://articles.uie.com/new-technologies-to-consider-for-interaction/

I originally wrote and published this post on Normative Design’s Ratsnest blog as the first part of a two-part synthesis of Design+AI, a monthly meetup hosted by Normative Design to explore how we design in a world augmented by AI. On the Sept 21, 2017 edition of the meetup, Chris Noessel, author of Designing Agentive Technology, joined us to shed some light on the area of designing agentive technology; specifically:

• What is agentive tech?
• What is narrow AI versus general AI?
• How do designers need to modify their practice to design agentive tech?
• Do agentive tech pose unique operational burdens?

Before the conversation got going, the group indulged me by playing a game of “what is and isn’t agentive tech”. Players considered a pair of options and identify which was agentive.

Unsurprisingly, only one person correctly identified the agentive tech. Hint: most of the pairs are both agentive to some degree, while some of the pairs contained only one example of agentive tech, and one contained a super artificial intelligence. So be warned…

What the heck is agentive technology?

I asked Chris to level-set on what he mean when he says “agentive technology”.

Agentive tech was an idea that arose from two forces that came together. On the one end, Chris had been challenged by one of his designers on his vision of the future from a designer’s perspective. On the other end, Chris started to see a pattern in his own work over the past two decades.

In short, it’s a new mode of interaction enabled by recent advances in narrow AI, in which the technology does something on behalf of the user, persistently and in a hyperpersonalized way. To understand more, we have to go back in time a bit.

What is the largest possible context for the world of interaction design?

Chris posits that the history of interaction design starts in WWII with human factors engineering. Highly trained and competent pilots were crashing planes. We learned through research that the machines were just too complicated: they represented a level of cognitive load that competed with other tasks and objectives and overwhelmed the pilots. The legacy of that research is human factors engineering.

If interaction design started with HFE, where’s the other end? When will our jobs be moot? Chris hypothesizes that “General” AI is the end of our jobs as interaction designers. Once we have something that can do what we do but is smarter and faster and can collaborate with the hive mind around the world, that’s pretty much the end for interaction design as a specialist job (if not well, almost all jobs). The question is: where are we now as we head into general AI? How close or far are we from general AI?

A new mode of interaction: outsourcing the work of achieving outcomes

Over the course of two decades, Chris has worked on various solutions that all involved outsourcing work to software in order to achieve outcomes. Based on his work as well as his personal experiences of different consumer devices, Chris started to see an emergent pattern. So, what do underwater science robot towers, automatic cat feeders, robo-investors, and Roombas have in common? Let’s look at each of these examples.

At Microsoft, he worked with the University of Washington to design underwater robots with sensors for seismic measurements. The robots would be pre-programmed to watch for certain things and then travel to different areas to collect different sorts of data, areas where humans couldn’t go with measuring tools in hand. These robots weren’t directly controlled by scientists, but did work for them.

When he travels for work, Chris’ used an automatic cat feeder to keep his cat from going hungry. That said, early edition cat feeders had a limitations: when they worked, they did, and when they didn’t, they didn’t, but you wouldn’t know either way, and either your cat would go hungry or you’d be worried about your cat going hungry. It should put food in the cat’s belly and assurances of the same into the user’s attention.

His work on robo-investor software taught him that despite knowing that the algorithm had data and reaction times that were better than human, people still wanted to see if they can beat the algorithm. People wanted room for play, for serendipity.

Roombas promised set-it-and-forget-it convenience for a household chore that many of us would rather not do. Who doesn’t love the feeling of coming home to clean floors without having had to do the work? The Roomba is not a fancy way for you to do vacuuming.

Granting agency to software

All these things started to feel like they were “of a piece”. There’s a pattern that felt like “I’m not doing the work. It’s doing the work.” Chris observed that in the past, he would design things to help people do work, build tools for people to do work, but this new type of thing was different: he was telling things how to do the work, and they would handle it from that point forward.

For example, even the humble automatic feeder wasn’t a tool to better feed his cat. Chris told it when to dispense food, told it how much food to dispense, and to continue doing so until further notice. Roomba isn’t a tool for us to push a vacuum cleaner around. We tell it when to clean, and it does. A robo-investor isn’t a tool for data and information; we tell it our financial goals, and from that point forward, it would do its best to achieve them. We can still look in and provide feedback, change up a few parameters, but it would continue operating on its own. We are granting these objects agency.

Disambiguating agentive technology

Chris saw an emergent pattern: the things he was designing and using were not automatic because automatic things didn’t need his attention at all. Think of a pacemaker as a good example of an automatic tech. If the human needs to get involved, automation has failed, and that’s not a design problem but an engineering problem.

The types of things we’re talking about are not helping us do things the ourselves. Smart assistants help us do things. Agentive tech, in contrast, does the things for us. For example, you can tell Google Keep to remind you to do a task when you’re at a location and/or at a certain time, but that’s all it does: it reminds you but doesn’t do it for you. It’s an agentive alert because it watches for your location in space and time, but it’s not like those other agents because you, the human, still have to act on the reminder. This notion of “help me do things” versus “do things for me” (or assistants versus agents) is a way for us to explore and understand a new class of interactions.

This class or pattern of interactions is marked by software that takes our directives, then implements on our behalf. As he cast about for a way to describe this class of technology, Chris looked for the adjectival form of “agent”, which turned out to be “agentive”. Agent-ive…saying it like that helps people realize it has to do with agents. In fact, in Japan, agentive technology is translated as “agent-based AI”.

Within the context of the general trajectory of general artificial intelligence that our industry is on, this pattern of agentive interactions is a “weak” kind of AI called narrow AI. It’s narrow because it’s smart in narrowly defined domains. It can’t generalize its knowledge to new domains.

Agentive technology is persistent, always-on, domain-specific narrow AI that acts on its user’s behalf in a hyper-personal way.

AI versus Narrow AI versus General AI

Invariably, the question of “what is AI” reared its head. From Chris’ vantage point, asking the question “what is AI?” is both interesting and not interesting. It’s not interesting because the term AI is too ambiguous to be useful or pragmatic. We’ve been talking about it since the 1950s and still can’t quite agree on what we want it to mean. For starters, the term “artificial” somehow implies fake, or at least made by humans. Maybe? As for the term “intelligence”, we don’t have a grasp on what that is, even after 100 years of studying it. And now we’ve combined these two notions into something we call “artificial intelligence”, begging the question — what does that even mean?

Having an adjective to the term “AI” perhaps gets us a bit closer to understanding what we’ve been doing, the implications and ramifications, and more important, our responsibilities. Thinking about in terms of “general AI” versus “narrow AI” begins to unpack some of the types of “artificial intelligence” work that we’ve been doing. We can posit that “General AI” refers to a human-like intelligence; specifically, an intelligence that can generalize from domain to domain. Not only can it learn, but it can learn across domains. Roomba is not that: it can’t generalize what it’s doing to other domains. It can only vacuum.

When we use the term “general AI”, we tend to mean human-like. Narrow AI is the stuff that comes before it; it’s an asymptotic approach to general AI or human-like intelligence. Narrow AI is the suite of technologies that are improving and getting human-like in their intelligence in a specific narrow domain.

As a designer, I take the maker’s approach: learn about the thing by making it, playing with it, testing its limits. To channel Richard Feynman, “What I cannot create, I do not understand.” While it’s fun in a hurts-my-brain way to participate in armchair philosophy about a monolithic AI, its breadth and non-specificity leaves me struggling to answer the “so what does it mean for me” and “now what do I do” questions. It also leaves me no closer to clarifying my own thoughts on the ethical implications of technology. Framing the mission in terms of narrow AI, however, does help me get down to where the rubber meets the road and to begin to understand what it really means to use “artificial intelligence” to solve human problems.

In Part 2, we’ll explore how our design practice needs to adapt when designing agentive technology.

References

http://rosenfeldmedia.com/books/designing-agentive-technology/
https://medium.com/@christophernoessel/ani-design-skills-f0af22360570

https://articles.uie.com/new-technologies-to-consider-for-interaction/

Disclaimer: The ideas and opinions expressed in this post are my synthesis of Chris Noessel’s session at a Design and AI meetup hosted by Normative, where I work. My views are not necessarily those held by Normative nor by Chris Noessel, and any technical errors or omissions are mine alone.