“It is the idealogy of Cartesian dualism in AI: where AI is narrowly understood as disembodied intelligence, removed from any relation to the material world.” – they absorb and produce knowledge independently from their creators, infrastructures, and the world at large.
“To understand how AI is fundamentally political, we need to go beyond neural nets and statistical pattern recognition to instead ask what is being optimized, and for whom, and who gets to decide.”
“As author and engineer Ellen Ullman puts it, this belief that the mind is like a computer, and vice versa, has infected decades of thinking in the computer and cognitive sciences,’ creating a kind of original sin for the field. It is the idealogy of Cartesian dualism in artificial intelligence: where AI is narrowly understood as disembodied intelligence, removed from any relation to the material world.”
“An atlas is an unusual type of book. It is a collection of disparate parts, with maps that very in resolution from a satellite view of the planet to a zoomed-in detail of an archipelago. When you open an atlas, you may be seeking specific information about a particular place – or perhaps you are wandering, following your curiosity, and finding unexpected pathways and new perspectives.”
AI is an attempt to be the atlas – the dominant way of seeing the world. We should instead try to construct “humble geography” that acknowledges one’s specific perspectives rather than claiming objectivity or mastery.
“In this sense, artificial intelligence is a registry of power… Computational reason and embodied work are deeply interlinked: AI systems both reflect and produce social relations and understanding of the world.”
# Hans: Observer-expectancy effect
“Their intuition was right: the questioner’s posture, breathing, and facial expression would subtly change around the moment Hans reached the right answer, prompting Hans to stop there.”
“The story of Clever Hans is compelling from many angles: the relationship between desire, illusion, and action, the business of spectacles, how we anthropomorphize the nonhuman, how biases emerge, and the politics of intelligence.”
“Even a system that appears to perform spectacularly in training can make terrible predictions when presented with novel data in the world.”
“Commerce follows the flag, but the flag follows the pick.”
“Those who profit from mining do so only because the costsd must be sustained by others, those living and those not yet born. It is easy to put a price on precious metals, but what is the exact value of a wilderness, a clean stream, breathable air, the health of local communities?”
“It was the ‘move fast and break things’ of a different time”
“The mines were located far from the city they enriched, and this remoteness allowed city dwellers to remain ignorant of what was happening to the mountains, rivers, and laborers that fed their fortunes.”
Megamachine: illustration of how all systems, no matter how immense, consist of the work of many individual human actors.
Humans are increasingly treated like robots or the connective tissue between robots.
Potemkin AI systems are a form of deception perpetrated by technology vendors eager to stake a claim in the lucrative tech space. The difference between ‘perceived’ ability and ‘real’ ability of these AI system are augmented by human piecework.
There is a new pervasive belief that everything is data and is there for the taking.
Data, when aggregated, become infrastructure. “The meaning or care that might be given to the image of an individual person, or the context behind a scene, is presumed to be erased at the moment it becomes part of an aggregate mass that will drive a broader system.”
ML systems work based on inductive inference. Any person who has taken a first-year course on logic will know the false conclusions that this can draw (e.g. all swans are white). Can we create systems that operate on deductive inference, which follows logically from a premise?
“Even the largest troves of data cannot escape the fundamental slippages that occur when an infinitely complex world is simplified and slices into categories.” This is a process that requires inherently political, cultural, and social choices.
“Data systems allowed scientists during wartime to operate at a psychological distance from the people ‘who would be maimed and killed by the weapons systems that would result from the ideas they communicated’.”
# Recognition Systems
“This is the danger of affect recognition tools. As we’ve seen, they take us back to the phrenological past, where spurious claims were made, allowed to stand, and deployed to support existing systems of power.”
Andrew Ferguson: “We are moving to a state where prosecutors and police are going to say ’the algorithm told me to do it, so I did, I had no idea what I was doing’.”
A false objectivity, “it’s just math, math can’t be biased, right?”
The historian of technology Alex Campolo calls this enchanted determinism: “AI systems are seen as enchanted, beyond the known world, yet determinisitc in that they discover patterns that can be applied with predictive certainty to everyday life.”