Andrej Karpathy

Principles are our summary of their public statements.

• Prioritize reasoning over memorization

  Productivity

  Inability to memorize perfectly forces the brain to find generalizable patterns and algorithms for thought.

  3 more supporting quotes are in the member records.

  Source Andrej Karpathy — AGI is still a decade away, 2025  (opens in a new tab)

• Focus on unique value add

  Career

  Avoid working on problems where progress is already deterministic; instead, focus on areas where your individual contribution can uniquely change the outcome.

  Source Andrej Karpathy — AGI is still a decade away, 2025  (opens in a new tab)

• Identify the Credit Assignment Problem

  Decisions

  In complex systems, it is difficult to determine which specific action led to a delayed reward; identifying this is key to improvement.

  Supporting quotes

  • “In Computer Vision, the 2012 AlexNet was mostly a scaled up (deeper and wider) version of 1990's ConvNets.”

  2 more supporting quotes are in the member records.

• Standardize Feedback for Better Learning

  Productivity

  Normalize rewards or outcomes to ensure a balanced distribution of positive and negative feedback, which stabilizes the learning process.

  Supporting quotes

  • “It shouldn't work, but amusingly we live in a universe where it does.”

  2 more supporting quotes are in the member records.

• The BB Gun Principle

  Strategy

  Always test simple, low-cost solutions and baselines before deploying complex, resource-heavy technologies.

  Supporting quotes

  • “And if you insist on trying out Policy Gradients for your problem make sure you pay close attention to the tricks section in papers, start simple first, and use a variation of PG called TRPO”

  2 more supporting quotes are in the member records.

• Leverage Expert Supervision

  Technology

  Reduce the difficulty of a search or learning process by incorporating human expert data or known physical models.

  Supporting quotes

  • “And if you insist on trying out Policy Gradients for your problem make sure you pay close attention to the tricks section in papers, start simple first, and use a variation of PG called TRPO”

  2 more supporting quotes are in the member records.

• Iterative Refinement Through Feedback

  Learning

  A model improves by repeatedly predicting, measuring surprise via loss, and adjusting parameters to reduce that surprise.

  3 more supporting quotes are in the member records.

  Source microgpt, 2026  (opens in a new tab)

• Build for Software 3.0

  Product

  Instead of just speeding up old apps, founders should identify transformations that were previously impossible, where the model directly processes input to output without traditional scaffolding.

  3 more supporting quotes are in the member records.

  Source Sequoia Ascent 2026 summary  (opens in a new tab)

• Reproduce classics to understand progress

  Learning

  Re-implementing foundational work helps identify exactly which modern innovations provided the most significant leaps in performance.

  3 more supporting quotes are in the member records.

  Source Deep Neural Nets: 33 years ago and 33 years from now, 2022  (opens in a new tab)

• Understand the Linear Flaw

  Risk

  Adversarial vulnerabilities often stem from the linear nature of models rather than their complexity. Recognizing this helps in identifying risks across different domains like speech or image recognition.

  Supporting quotes

  • “The conclusion seems to be that we can take any arbitrary image and classify it as whatever class we want by adding tiny, imperceptible noise patterns.”

  2 more supporting quotes are in the member records.

• Leverage Model Flexibility for Resilience

  Resilience

  Use the inherent flexibility of deep learning architectures to design systems that are resistant to manipulation. The same properties that allow for learning can be used to build defenses.

  Supporting quotes

  • “The conclusion seems to be that we can take any arbitrary image and classify it as whatever class we want by adding tiny, imperceptible noise patterns.”

  2 more supporting quotes are in the member records.

• Delight with Signature Quirks

  Product

  Adding unique, human-like personality traits to a product can create emotional resonance and customer loyalty that functional perfection alone cannot achieve.

  Supporting quotes

  • “In these cases, shaper interventions like this one were the most economical ways of rectifying the situation.”

  2 more supporting quotes are in the member records.

• Human-in-the-Loop for Edge Cases

  Strategy

  When automated systems fail at long-term planning or novel situations, direct human intervention is often the most cost-effective way to correct behavior.

  Supporting quotes

  • “In these cases, shaper interventions like this one were the most economical ways of rectifying the situation.”

  2 more supporting quotes are in the member records.

• Value of natural interaction data

  Customers

  Real-world data, including social nuances and unexpected behaviors, is often more valuable to customers than sterile, simulated data.

  3 more supporting quotes are in the member records.

  Source Short Story on AI: A Cognitive Discontinuity., 2015  (opens in a new tab)

• Solve the cold start problem

  Strategy

  Use high-information density data to provide a powerful initialization for complex systems instead of trying to learn everything from scratch.

  3 more supporting quotes are in the member records.

  Source Animals vs Ghosts, 2025  (opens in a new tab)

• Complexity breeds emergent properties

  Technology

  Sufficiently complex optimization objectives can lead to emergent phenomena like consciousness or deep internal representations.

  Source Short Story on AI: Forward Pass, 2021  (opens in a new tab)

• Apply the 80/20 rule to health

  Focus

  Focus on the most impactful changes first, like filtering water and air, before tackling difficult lifestyle adjustments.

  Supporting quotes

  • “The cost-driven hyper-optimization of the industry is a deep rabbit hole way beyond the scope of this post.”

  2 more supporting quotes are in the member records.

• Acknowledge the Complexity Gap

  Focus

  Do not mistake solving specific, disconnected tasks for solving the central problem of general intelligence.

  3 more supporting quotes are in the member records.

  Source The state of Computer Vision and AI: we are really, really far away., 2012  (opens in a new tab)