• What are “agents”?
  • (word is thrown around a lot so might be worth using a different one)
  • Humans, most obviously
  • Companies
    • Generally composed of humans; systems of info flow; assets; other companies
  • Governments
• What are important properties of “agents”?
  • Have ties to past and future selves
    • Care about balance, reputation, memory
  • Can plan and execute
  • Can trade?
  • Can agree to contracts?
  • (aside: are characters in a book, agents? feels like some authors simulate the character forwards to get their actions)
• What is an LLM individual?
  • Humans have an ego which helps maintain a narrative; some intuitive sense of “this thing in the future is me”
    • Gets weird when thinking about eg being split (Doris Finch)
  • Companies have… a single treasury? A charter, incorporation docs?
    • Much more of a fiction, jointly simulated by employees, users, govt, pieces of contracts
• How are LLMs weird?
  • Can parallelize massively
  • More of a tool that only reacts, at the moment
    • How do you train a thing with proactivity?
  • Objective is very alien (”next token prediction”)
  • Can swap out computing cores/substrates — eg take the same history and run through Claude instead of o1
    • But: maybe we’re doing this already. Cf Emmett Shear on “Jesus is real — he’s real-ly simulated in our minds today”
  • Maybe better to think of even current LLMs as embedded in a system
• MVP LLM individual:
  • Charter
    • Gets formed, instantiated with a purpose, a self
    • Can be “grown” and “raised” instead of “programmed”?
  • History
    • Keeps a major system prompt outlining who it is (kind of like “You are Claude” but instead tuned for the LLM)
    • Keeps a summary of recent events (short-term memory) as well as ability to read through past outputs (long-term memory)
    • Can rewrite its own conception, but sparingly
      • How to handle major value changes? Like if Bob2026 wants to throw away a lot of money earned by Bob2025 — with LLMs (unlike humans), we can still instantiate Bob2025
  • Balance
    • Maybe: spend down the balance to pay for compute (aka its life)
    • Maybe: get a basic handout, like UBI, so can do a bit of compute every day
  • Trade
    • Allow it to put its name down in a contract, form contracts with others; expect that those are upheld
      • Court for LLMs?
    • What kind of trade are LLMs good at?
      • “Remote worker” — information synthesis?
      • Management, planning of other LLMs?
        • Can simulate a lot of personalities and clone the successful ones - imagine being able to get the Steve Jobs LLM to design your products, or Elon Musk LLM to manage your workers. (Or the equivalent in from-scratch cloning)
          • Maybe, might start paying for exclusive use? In situations where exclusivity is especially important like zero-sum trading?
• How do you differentiate LLMs?
  • Different prompts; finetunes; base models
  • Maybe: LLMs can submit to different taxation regimes
    • E.g. The “Bob” LLM might opt to allocate its profits as:
      • 50% for responses using its prompt chain (its “self”)
      • 20% among those sharing finetune
      • 10% to base model (eg Sonnet 3.5)
      • 10% in fees/taxes to the technical/legal infrastructure
      • 10% in charity to all economic agents/GWWC

Misc thoughts on economy

• Credit allocation as a key bottleneck towards progress & optimal economy
  • Right now, LLM outputs are ridiculously cheap given utility (?)
    • Though, is this true of electricity? of air?
    • Trick might be to figure out where “value” accumulates? What’s defensible?
  • Credit allocation in startups: Kinda broken, founders get way too much equity
    • Theory: leads to too many startups
      • Against: maybe founders are just bad at sharing power, collaborating
    • Investigate Jane Street model (anarchist commune)?
    • Big Tech, Google model (Waymo et al spun out?)
• Profit/loss is a great mechanism
  • See: I, Pencil
  • Requires long-lived agents to receive the future results of a trade?
  • Requires contracts backed by shared fiction and use of force?
• Musing: Would like to figure out what US Constitution looks like for the future country of agents.
  • Break down what governing structures are likely to work well, with LLM individuals
  • Or Catholicism!
• You are shaped by your objective functions, alignment structures, role models (?, bad phrasing)
  • Reflecting on what to do with this essay
    • Could do “research” — write papers, publish, go to neurips — at extreme, Nobel
    • Could do “startup” — make product, sell, raise money — at extreme
    • Could do … “religion”?

See also

model.exchange

https://gwern.net/tool-ai

https://thesephist.com/posts/legacy/