Chapter C17 · Interactive Course
AI, people and society
See the human choices behind a system's behaviour.
Core Learning Concept
Inspect the fictional public service as a system that affects people. Identify who chose its objective, which groups need testing, how a user can seek review, and what resource or permission questions remain. A helpful output is one part of the evaluation; the surrounding decisions matter too.
What You’ll Learn in This Chapter
-
People shape AI systems
Identify a human choice behind an AI feature.
People choose the problem, training data, success measure, and deployment rules of an AI system. Those choices shape whose needs it serves. A reminder optimized to reduce overdue books may behave differently from one designed to minimize unnecessary messages. Ask what success was defined to mean.
-
Performance can differ across people
Recognize that an overall score can hide uneven performance.
An overall average combines outcomes across many different cases. When a large easy group scores high, the headline average hides poor results for smaller groups. Always ask for a breakdown by condition and group size.
-
Recommendations serve an objective
Identify how a recommendation goal can shape what a person sees.
A recommendation system selects options according to an objective, such as clicks, purchases, or completed reading. That objective may only partly match your goal. Ask what behavior the system is rewarded for and whether it provides the options you need, rather than treating its ranking as neutral advice.
-
Ask for human review
Identify a constructive response to a consequential automated decision.
A consequential automated decision should be examined using the facts and process behind it. Ask what information was used, whether it can be corrected, and what review route the organization provides. Available rights and procedures depend on the setting; the lesson’s task is to identify a constructive next step.
-
Respect creators and permissions
Check rights and permitted use before distributing generated or transformed material.
Generated or transformed content can still raise permission questions. Consider the service terms, rights in material supplied as input, and the interests of recognizable people depicted. Check the intended use before distributing the result. The label “AI-generated” does not by itself settle these different questions.
-
AI uses physical resources
Recognize that AI runs on equipment with resource costs.
AI queries run on physical hardware in data centers, consuming electricity and water for cooling. Resource use depends heavily on model size, prompt length, and server efficiency. One single estimate cannot describe every query.
-
Work changes task by task
Distinguish a change to a task from a certain prediction about an entire job.
A job is a broad bundle of varied tasks and human judgment. An AI tool that assists with one specific task does not automate an entire profession. Look at which specific parts are aided versus what requires human responsibility.
-
Look beyond the output
Inspect a public-facing AI service: name a human design choice, a group to test, a review route and a resource or rights tradeoff.