Semantics

Last updated Jul 2, 2021 Edit Source

• Seed

What do words or sentences mean

Meanings are public property: the same meaning be grasped by more than one person and by people at different times. Heavily related to representation

• If this argument is correct, then the content of our thoughts partly depends on what is in the world we live in
• Content determines object
  • What about indexical thoughts? Thoughts that depend on context like “you”, “me”, “there”, “here”
• Concept of meaning rests on two unchallenged assumptions
  • Understanding a word (knowing its intension) was just a matter of being in a certain psychological state
  • meaning of a term determines its extension
    • extension → actual physical manifestation / what it is
• proving these are false → Twin Earth Argument
  • extension of the term is not a function of the psychological state of the speaker by itself
    • water on twin earth vs earth in 1750; both oscars think it to be the same thing

• division of linguistic labour
  • some people wear gold rings
  • some people tell the difference between gold and non-gold
  • not everyone needs to tell the difference between gold and non-gold, rely on the judgement of experts
  • formal: “every linguistic community exemplifies the sort of division of linguistic labour just described; that is, it possesses at least some terms whose associated “criteria” a re known only to a subset of the speakers who acquire the terms, and whose use by the other speakers depends upon a structured cooperation between them and the speakers in the relevant subsets”
• two theories → if “water” means $H_2O$ in $W_1$ and “water” means $XYZ$ in $W_2$
  1. world-relative but constant in meaning, ‘water’ means the same thing in both world, but water is $H_2O$ in $W_1$ and water is $XYZ$ in $W_2$
  2. water is $H_2O$ in all worlds, but ‘water’ doesn’t mean the same thing in $W_1$ and $W_2$
• Twin-earth argument implies 2nd one is true
• Rigidity → if a designator in a particular sentence refers to the same individual in every possible w in W
  • Water is rigid in (2)
• Once we have discovered the nature of water, nothing counts as a possible world in which water doesn’t have nature
  • i.e. one we have discovered that water (in the actual world) is h2o, nothing counts as a possible world in which water isn’t h2o
• Water at another time or in another place or even in another possible world has to bear the relation same to our “water” in order to be water
• Types of statements
  • epistemically necessary → rationally unrevisable
  • metaphysically necessary → true in all possible worlds

• Mental lexicon: mental dictionary of word knowledge (how it sounds, grammar, definition, etc.)
• Word: symbol that refers to something
• Symbol: stands for something without being a part of that something
• Context-bound word: things tied to particular contexts (word use is more specific than actual meaning)
• Nominals: names for things
• Natural partitions hypothesis: the physical world makes obvious the things that take nouns as labels, whereas the meanings that verbs encode have to be figured out from hearing the verb in use
• Relational relativity hypothesis: possibility that verb meanings will vary from language to language (linguistic work showing that noun meanings are more similar across languages than are verb meanings)
• Word extension: to what extent is a word valid?
  • Underextensions: using words in a more restricted fashion
  • Overextensions: using words in a more broad fashion (for related study, see Naigles & Gelman 1995 study, results showed that overextensions are mistakes, they don’t indicate incorrect understanding of the words)
  • Protowords (also known as phonetically consistent forms – PCFs)
    • Phonetically consistent: the child uses the same word every time.
  • Things that help with accurate word extension:
    • Taxonomic extension: words to things are actually taxonomies (they are of the same category)
• Word spurt: see Choi & Gopnik (1995)
• Types of language use, two ends of a continuum
  • Referential language style: more object labels
  • Expressive language style: relatively fewer object object labels and more personal/social words
• Mapping problem: how do we know what the new word refers to?
  • Fast mapping: initial hypothesis about word meaning
  • Lexical principles/lexical constraints: guides that limit possible interpretations of new words
    • Whole-object assumption: words refer to whole objects
    • Assumption of mutual exclusivity: different words refer to different kinds of things. No category overlap
  • Lexical gaps: Sometimes things are not a one-to-one match – your language may not have a lexical item for something
• Age at which children learn early words (first 50-100) can vary a lot due to
  • Environmental Factors
    • Language experience and input
    • Socioeconomic status (SES)
    • Birth order
  • Individual Factors
    • Processing speed
    • Phonological memory
    • Personality and temperament

Semantics in convolutional neural networks

Hidden units often correlate semantically-meaningful concepts.

Inceptionism: what about, instead of weights, use backpropagation to take gradient with respect to $x_i$. i.e., show me what you think a banana looks like

Style Transfer: loss function matches deep latent representation of content image $C$:

• Difference between $z_i^{(m)}$ for deepest $m$ between $x_i$ and $C$

• Intuition, deep layers $z_i^{(m)}$ capture the semantics/concepts in an image, invariant to actual style

• Adversarial Examples: imperceptible noise that changes label/prediction.

  • Potentially dangerous! We could repaint a stop sign and fool self-driving cars

• It can learn bad correlations (e.g. correlating grass with cows so when it sees a cow by a beach it has no idea what it is)

  • Related: does the prediction change real-world outcomes?
  • i.e., does the doctor actually care?
  • Does “not trying to overfit” mean we perform badly on some groups?
    • If you have 99% “Group A” in your dataset, model can do well on average by only focusing on Group A
    • Treat the other 1% as outliers
    • Doing well at test-time might mean ignoring outliers and minorities
  • See also: bias bug