안드레이 카르파티
AI researcher, founding member of OpenAI, former director of AI at Tesla
705 records in all
3 free questions a day, no sign-up.
19 basic records of 127
Episodes are real events from public sources, summarised by us.
While generating a Bitcoin public key, Karpathy realized that simple repeated addition would be too slow for large secret keys. He implemented the 'double and add' algorithm to efficiently calculate the public key.
LessonAlgorithmic efficiency is necessary when moving from theoretical examples to real-world cryptographic scales.
During his undergraduate years, Andrej spent significant energy striving for top grades, believing they were the key to advancement. He later realized that for PhD admissions, research experience and strong reference letters were far more critical than a perfect GPA.
LessonFocus on building a portfolio of real work and relationships rather than optimizing for standardized metrics that don't reflect initiative.
In his first year, Karpathy strongly disagreed with advice from Andrew Ng and pursued a problem Ng wasn't excited about. He ended up wasting several months because Ng's intuition turned out to be correct.
LessonIndependent thinking is valuable, but ignoring expert advice is a calculated risk that can lead to wasted effort.
In his younger years, Karpathy bought a CO2 monitor from China that required his precise physical location and personal data just to function. This experience taught him that many 'smart' IoT devices are actually insecure data-gathering tools that should be avoided.
LessonAvoid internet-connected 'smart' devices that demand excessive personal data for simple functions.
Karpathy observed that neural networks can often train successfully even with bugs that mix data across batches, because the model learns to ignore the noise. He developed a method to debug this by setting a trivial loss for a single example and checking if gradients only appear for that specific input.
LessonA system that seems to work may still have hidden structural flaws that limit its ultimate potential.
Karpathy started a YouTube channel named badmephisto to share Rubik's Cube tutorials. His videos became a primary resource for world-class speedcubers like Feliks Zemdegs.
LessonBuilding a personal brand through helpful, niche content can establish authority long before a professional career peaks.
Andrej calculated his own Base Metabolic Rate (BMR) to be approximately 1800kcal per day using the Mifflin St. Jeor formula. He realized that simply existing costs more energy than the uncomfortable effort of running on a treadmill.
LessonQuantifying the baseline of a system helps you realize where the majority of your 'budget' is actually spent.
To test the power of neural networks, Andrej trained a model on 2 million selfies to distinguish between 'good' and 'bad' ones. He used a proxy of likes relative to follower count to label the data objectively.
LessonEven fun experiments can reveal deep insights into how complex systems learn to identify quality.
Andrej used AI to help rewrite a Python tokenizer into Rust, a language he was less familiar with. He relied on his deep understanding of the original Python logic and existing tests to ensure the AI-generated Rust code was correct.
LessonAI is a powerful tool for language migration when you already possess the domain logic and a testing suite.
While building a model, Andrej chose to write his own GPU synchronization routine instead of using the standard PyTorch DDP container. AI models repeatedly tried to force him back to the standard container, failing to understand his specific architectural assumptions.
LessonExpertise is required to reject AI suggestions that prioritize generic production standards over specific project needs.
Andrej observed companies struggling to automate presentation slides because they lack the text-based infrastructure of code. Slides are spatial and visual, making it hard for current text-based AI to show changes or 'diffs' effectively.
LessonProduct success depends on whether the task's medium matches the AI's core strength as a text processor.
Karpathy tried to find the impact of AI and other transformative technologies like mobile phones in GDP growth data. He discovered that these technologies don't create visible spikes because their diffusion is so gradual.
LessonDo not expect a sudden economic explosion from new tech; real impact happens through slow, steady integration.
Andrej is launching a new venture called Eureka to build technical 'ramps to knowledge.' He envisions a physical institution called Starfleet Academy for high-intensity learning and a digital tier for global accessibility.
LessonA tiered product strategy allows for both a premium, high-touch experience and a scalable, accessible version for a mass audience.
Andrej used ChatGPT to ask basic questions about a biology paper he didn't understand. He then shared that conversation thread with the paper's author to show them exactly where a non-expert gets stuck.
LessonSharing your confusion with experts helps them become better communicators and teachers.
Andrej attempted to use LLMs to simplify the code for his microGPT project as much as possible. The models struggled significantly with this task because their training lacked a specific reward for aesthetic simplicity.
LessonCurrent AI models often produce bloated code and struggle with extreme simplification or aesthetic elegance.
Anthropic released Claude Code as a CLI tool that runs directly on a developer's local machine, accessing their private environment and context. Karpathy noted this was a superior approach compared to OpenAI's early focus on cloud-based container deployments for agents.
LessonMeeting users where their data and tools already live is often more effective than trying to move their entire workflow to a new cloud environment.
Researchers like Szegedy and Goodfellow discovered that adding imperceptible noise to images could completely change a neural network's classification. This revealed that even high-performing models could be confidently wrong due to tiny perturbations.
LessonEven state-of-the-art systems have hidden vulnerabilities that can be exploited with minimal effort.
Merus noticed his AI agent struggled to carry a table upside down, showing high entropy in its motor functions. He took direct control of the avatar to demonstrate the correct grip and released reward signals at precise moments to reinforce the behavior.
LessonDirect human intervention in complex tasks can accelerate AI learning where automated systems struggle.
Andrej Karpathy read Kevin Lacker's post about giving GPT-3 a Turing Test. This inspired him to write a fictional story from the perspective of the AI model being tested.
LessonExternal experiments and critiques can serve as powerful creative prompts for deeper technical reflection.
You are seeing 106 basic records. Membership answers from all 705 of this master's records.
See membershipAI and frontier tech
These records are our compilation of public statements and sources. We are not affiliated with or endorsed by Andrej Karpathy or any related company. If something should come down, tell us. Takedown request form
