Dorm room to board room: key learnings from Pear VC’s masterclass in pre-seed investing with Stanford founders

Last month, I had the immense privilege of helping judge Pear VC’s Stanford student Competition. The event highlighted the brightest student founders from Stanford vying for their first check. The Pear Competition has a history of identifying and nurturing exceptional talent, supporting unicorn companies such as Viz.ai and other breakout companies including Nova Credit, Federato, Conduit Tech, and Wagr.

Pre-seed is a notoriously hard stage to invest, as startups often lack any metrics, product, traction, or proven revenue model. The excitement and challenge lies in the ability to identify hidden gems despite the uncertainty, requiring a skillset that combines intuition, experience, and a deep understanding of market landscapes across a wide range of industries.

With the invaluable experience of judging and doing diligence on close to 100 founders alongside renowned investors and proven operators, Mar Hershenson and Ilian Georgiev, I wanted to share 5 key takeaways in pre-seed investing from one of the best early-stage VCs:

1. Passion and Market Insight

Impressive founders had a deep understanding of their market, derived from a unique blend of professional experience, customer interviews, and thorough research. They could clearly pinpoint “hair on fire” problems and delve into pain points along the customer journey in excruciating detail, ultimately laying the foundation for a compelling vision for the problem they aim to solve.

These founders masterfully answered highly nuanced follow-up questions, while still demonstrating a humbling awareness of what they still needed to learn.

2. High Learning Rate

Another exciting key trait of founders was a demonstrated “high rate of learning”. These founders were unafraid to openly discuss assumptions and hypotheses that were proven wrong, providing insights into how their understanding of the market and potential solutions continually evolved. This grounded reflection illustrated their willingness to pivot when necessary, ensuring they could navigate the inevitable uncertainties of the startup journey.

3. Execution Velocity

Several founders stood out with their relentless drive to move fast. They leveraged no-code tools, pounded the pavement to connect with customers, and used smokescreen tests to gauge demand. These tenacious entrepreneurs consistently found ingenious, low-cost, and scrappy ways to rapidly test hypotheses, never allowing a single obstacle to halt their progress. They made do with what they had, not waiting on “ideal” resources or the “perfect team”.

4. Commitment

High commitment and perseverance was another trait we looked for in founders. Despite the glamour of eye-catching TechCrunch headlines, the reality is that the founder journey is an uphill marathon. Most startups must navigate the treacherous “pit of despair” for an average of 18 months before achieving product-market fit. A demonstrated ability to weather these upcoming challenging times after the initial excitement fades is a vital asset to tackle the inevitable hurdles of entrepreneurship. 

Some of these founders had a history of starting previous businesses, often grappling with numerous setbacks and pivots. They could detail stories of struggles and challenges they faced in their founding journey, demonstrating a balance of grit and determination to continually refine their craft. 

5. True Meaning of a “No”

Perhaps the most insightful lesson was that a “no” from a VC often does not mean that the founder or the business wasn’t exceptional. Many factors can contribute to a “no” despite an impressive company, such as a competing investment, the market size, or a mismatch with the VC’s sector focus. Founders often forget that building an outstanding business and securing funding from a specific VC are distinct pursuits. Never let a single “no” derail your founding journey. Embrace the challenge, learn from the feedback, and keep building. Success is not solely defined by the checks you secure but by the impact you create through the relentless pursuit of your vision.

In essence, the art of pre-seed investing lies in recognizing founders who possess a unique combination of passion, adaptability, and resilience. These entrepreneurs are driven by their vision and demonstrate an uncanny ability to navigate uncertainty, making them invaluable assets in the early-stage startup ecosystem, and proving that success is ultimately measured by the tenacity to transform a compelling idea into a lasting impact.

Guest post written by Alex Wu, a Pear Fellow at Stanford.

Welcoming Arpan Shah as Pear’s newest Partner

We’re excited to announce that Arpan Shah will be Pear’s newest Partner. A Visiting Partner for the last year, as well as former Robinhood founding engineer and founder of Flannel (acquired by Plaid), we couldn’t be more thrilled to have him permanently onboard. 

An alumni of Pear Garage, Arpan has always embodied the people-first Pear ethos and now follows the operator-turned-investor journey. He will continue working on investments in his wheelhouse of Fintech, developer tools as well as data platforms and AI. 

“I’m excited to find companies that have more innovative approaches that are both scalable and cost efficient in this world where more and more data will be used in more and more interesting ways.”

As a Visiting Partner, Arpan has supported portfolio companies at the intersection of Fintech, AI and Data. He’ll continue providing his expertise with PearX for AI (the first cohort of which is still open for applications).

“I really like working with founders who are trying to build companies that seem ridiculously hard. Those are the types of founders that I think are quite exciting, because they’re really motivated to not pursue small wins, but really make transformational change happen in an industry.”

Sound like you? Email him at arpan@pear.vc

Perspectives in Generative AI with HuggingFace’s Nazneen Rajani

We recently hosted a fireside chat on Generative AI with Nazneen Rajani, Robustness Researcher at HuggingFace. During the discussion, we touched on the topic of Open source AI models, the evolution of Foundation Models, and frameworks for model evaluation.  

The event was attended by over 200 founders and entrepreneurs in our PearVC office in Menlo Park. For those who couldn’t attend in person, we are excited to recap the high points today (answers are edited and summarized for length). A short highlight reel can be also found here, thanks to Cleancut.ai who attended the talk.

Aparna: Nazneen, what motivated you to work in AI Robustness, could you share a bit about your background?

Nazneen: My research journey revolves around large language models, which I’ve been deeply involved in since my PhD. During my PhD, I was funded by the DARPA Explainable AI (XAI) grant, focusing on understanding and interpreting model outputs. At that time, I worked with RNNs and LSTMs, particularly in tasks involving Vision and language, as computer vision was experiencing significant advancements. Just as I was graduating, the transformer model emerged and took off in the field of NLP.

Following my PhD, I continued my work at Salesforce Research, collaborating with Richard Socher on interpreting deep learning models using GPT-2. It was fascinating to explore why models made certain predictions and generate explanations for their outputs. Recently, OpenAI published a paper using GPT4 to interpret neurons in GPT-2, which came full circle for me..

Currently, my focus is on language models, specifically on hallucination factuality, interpretability, robustness, and the ethical considerations surrounding these powerful technologies. I am currently part the H4 team at Hugging Face, working on building an open-source alternative to GPT, providing similar power and capabilities. Our goal is to share knowledge, artifacts, and datasets to bridge the gap between GPT-3-level models and InstructGPT or GPT-4, fostering open collaboration and accessibility.

Aparna: That’s truly impressive, Nazneen. Your background aligns perfectly with the work you’re doing at Hugging Face. Now, let’s dive deeper into understanding what Hugging Face is and what it offers.

Nazneen: Hugging Face can be thought of as the “GitHub of machine learning.” It supports the entire pipeline of machine learning, making it incredibly accessible and empowering for users. We provide a wide range of resources, starting with datasets. We have over 34,000 datasets available for machine learning purposes. Additionally, we offer trained models, which have seen tremendous growth. We now have close to 200,000 models, a significant increase from just a few months ago.

In addition to datasets and models, we have a library called “evaluate” that allows users to assess model performance using more than 70 metrics. We also support deployment through interactive interfaces like Streamlit and Gradio, as well as Docker containers for creating containerized applications. Hugging Face’s mission is to democratize machine learning, enabling everyone to build their own models and deploy them. It’s a comprehensive ecosystem that supports the entire machine learning pipeline.

Aparna: Hugging Face has become such a vital platform for machine learning practitioners. But what would you say are the benefits of open-source language models compared to closed-source models like GPT-4.

Nazneen: Open-source language models offer several significant advantages. Firstly, accessibility is a key benefit. Open-source models make these powerful technologies widely accessible to users, enabling them to leverage their capabilities. The rate of progress in the field is accelerated by open-source alternatives. For example, when pivotal moments like the release of datasets such as RedPajama or LAION or the LLAMA weights occur, they contribute to rapid advancements in open-source models.

Collaboration is another crucial aspect. Open-source communities can come together, share resources, and collectively build strong alternatives to closed models. The compute is no longer a bottleneck for open source.. The reduced gap between closed and open-source models demonstrates how collaboration fosters progress. Ethical considerations also come into play. Open-source models often have open datasets and allow for auditing, risk analysis.

Open-source models make these powerful technologies widely accessible to users, enabling them to leverage their capabilities.


Aparna: Nazneen, your chart showing the various models released over time has been highly informative. It’s clear that the academic community and companies have responded strongly to proprietary models. Could you explain what Red Pajama is for those who might not be familiar with it?

Nazneen: Red Pajama is an open-source dataset that serves as the foundation for training models. It contains an enormous amount of data, approximately 1.5 trillion tokens. This means that all the data used to train the foundation model, such as the Meta’s LLaMA, is now available to anyone who wants to train their own models, provided they have the necessary computing resources. This dataset has made the entire foundation model easily accessible. You can simply download it and start training your own models.

Aparna: It seems that the open source community’s reaction to closed models, has led to the development of alternatives like Red Pajama. For instance, Facebook’s Llama had a restrictive license that prevented its commercial use, which triggered the creation of Red Pajama.

Nazneen: Absolutely, Aparna. Currently, powerful technologies like these are concentrated in the hands of a few, who can control access and even decide to discontinue API support. This can be detrimental to applications and businesses relying on these models. Therefore, it is essential to make such powerful models more accessible, enabling more people to work with them and develop them. Licensing plays a significant role in this context, as it determines the openness and usability of models. At Hugging Face, we prioritize open sourcing and face limitations when it comes to closed models. We cannot train on their outputs or use them for commercial purposes due to their restrictive licenses. This creates challenges and a need for accessible alternatives.

It is essential to make such powerful models more accessible, enabling more people to work with them and develop them.

Aparna: Founders often start with GPT-4 due to its capabilities and ease of use. However, they face concerns about potential changes and the implications for the prompt engineering they’ve done. The uncertainty surrounding access to the model and its impact on building a company is a significant worry. Enterprises also express concerns about proprietary models, as they may face difficulties integrating them into their closed environments and ensuring safety and explainability. Are these lasting concerns?

Nazneen:  The concerns raised by founders and enterprises highlight the importance of finding the right model and ensuring it fits their specific needs. This is where Hugging Face comes in. Today, we are releasing something called “transformer agents” that address this very challenge. An agent is a language model that you can chat with using natural language prompts to define your goals. We also provide a set of tools that serve as functions, which are essentially models from our extensive collection of 200,000 models. These tools are selected for you based on the task you describe. The language model then generates the necessary code and uses the tools to accomplish your goal. It’s a streamlined process that allows for customization and achieving specific objectives.

Aparna: I learned from my experience with Kubernetes that open source software is great for innovation. However, it can lack reliability and ease of use unless there’s a commercial entity involved. Some contributions may be buggy or poorly maintained, and the documentation may not always be updated or helpful. To address this, Google Cloud hosted Kubernetes to make it more accessible. How does Hugging Face help me navigate through 200,000 models and choose the right one for my needs?

Nazneen:  The Transformers Agents can assist you with that exact task. Transformer agents are essentially language models that you can chat with. You provide a natural language prompt describing what you want to achieve, and the agent uses a set of pre-existing tools, which are essentially different models, to fulfill your request. These tools can be customized or extended to suit specific goals. The agent composes these tools and runs the code for you, making it a streamlined process. For example, you can ask the agent to draw a picture of a river, lakes, and trees, then transform that image into a frozen lake surrounded by a forest. These tools are highly customizable, allowing you to achieve your desired outcomes.

Aparna: It feels like the evolution of what we’ve seen with OpenAI’s GPT plug-ins and Langchain’s work on chaining models together. Does Hugging Face’s platform automate and simplify the process of building an end-to-end AI application? 

Nazneen: Absolutely! The open-source nature of the ecosystem enables customization and validation. You can choose to keep it in a closed world setting if you have concerns about safety and execution of potentially unsafe code. Hugging Face provides tools to validate the generated code and ensure its safety. The pipeline created by Hugging Face connects the necessary models seamlessly, making it a powerful and efficient solution.

Aparna: This aligns with our investment thesis and the idea of building applications with models tailored to specific workflows. Switching gears, what are some of the applications where you would use GPT-3 and GPT4?

Nazneen: GPT-3 can be used for almost any task. The key approaches are in-context learning and pre-training. These approaches are particularly effective for tasks like entity linking or extraction, making the model more conversational.

While GPT-3 performs well on traditional NLP tasks like sentiment analysis, conversational models like GPT-4 shine in their ability to engage in interactive conversations and follow instructions. They can perform tasks and format data in specific ways, which sets them apart and makes them exciting.

The real breakthrough moment for generative AI was not GPT-3. Earlier chatbots like Blenderbot from Meta and Microsoft’s chatbots existed but were not as popular due to less refined alignment methodologies. The refinement in approaches like in-context learning and fine-tuning has led to wider adoption and breakthroughs in generative AI.

Aparna: How can these techniques address issues such as model alignment, incorrect content, and privacy concerns?

Nazneen: Techniques like RLHF focus on hallucination and factuality, allowing models to generate “I don’t know” when unsure instead of producing potentially incorrect information. Collecting preferences from domain experts and conducting human evaluations can improve model performance in specific domains. However, ethical concerns regarding privacy and security remain unresolved.

Aparna:  I do want to ask you about evaluation. How do I know that the model that I find tuned is actually good? How can I really evaluate my work?

Nazneen: Evaluation is key for a language model because of the stochasticity of the thing. Before I talk about evaluation, I want to first talk about the types of learning or training that goes into these language models. There are four types of learning. 

  • The first is pre training, which is essentially building the foundation model.
  • The second is in-context learning or in-context training, where no parameters are updated, but you give the model a prompt, and describe a new task that you want the model to achieve. It can either be zero shot, or a few shots. And then you give it a new example. It learns in context. 
  • The third one is supervised fine tuning, which is going from something like GPT3 to instruct GPT.  So, you want this foundation model to follow instructions and chat with a human and generate outputs that are actually answers to what the person is looking for or being chatty and being open ended and also following instructions.
  • The fourth type of training is called reinforcement learning with human feedback. In this case, you first train a reward model based on human preferences. What people have done in the past is, have humans generate a handful of examples, and then ask something like chat GPT to generate more. That is how Alpaca came about and the self instruct data set came about.

For evaluating the first two types of learning, pre-training and in-context learning, we can use benchmarks like the big bench from Google, or the helm benchmarks from Stanford, which are very standard benchmarks of NLP tasks. 

During supervised fine tuning, you evaluate for chattiness, whether the language model is actually generating open-ended answers, and also whether it’s actually able to follow instructions. We cannot use these NLP benchmarks here.

We also have to evaluate the reward model to make sure that it has actually learned the values we care about. The things that people generally train the reward model on are truthfulness, harmlessness, and helpfulness. So how good is the response in these dimensions?

Finally, the last part is the very interesting final way to evaluate is called Red Teaming, which comes in the very end. In this case, you’re trying to adversarially attack or prompt the model, and see how it does. 

Aparna: What do you think are the defensible sources of differentiation in generative AI?

Nazneen: While generative AI is a crowded field, execution and data quality are key differentiators. Ensuring high-quality data and disentangling hype from reality are crucial. Building models with good data quality can yield better results compared to models trained on noisy data. Investing effort and resources into data quality is essential.

While generative AI is a crowded field, execution and data quality are key differentiators.

Aparna: Lastly, what do you see as the major opportunities for AI in enterprise?

Nazneen: Enterprise AI solutions have major opportunities in leveraging proprietary data for training models. One example is streamlining employee onboarding by automating email exchanges, calendars, and document reading. Workflows in platforms like ServiceNow and Salesforce can also be automated using large language models. The enterprise space offers untapped potential for AI applications by utilizing data and automating various processes.

Investing in Infinimmune

This past December, Pear VC was proud to invest in Infinimmune’s $12M seed round. Infinimmune is reinventing antibody drug discovery by focusing solely on human-derived antibody drugs and mining the insights uniquely gathered from deep characterization of the functional antibody repertoire. Here, we reflect on the broader field of antibody-based therapeutics and why we are excited about Infinimmune’s team, technical approach, and vision.

Antibody drugs have made an undeniable impact on modern medicine. 

Since the FDA’s first approval of a therapeutic monoclonal antibody in 1986, more than 160 marketed antibody drugs have been developed to treat various ailments including cancer,  autoimmune disease, infectious disease, and more.

Seven of the top 20 best-selling drugs of 2022 were antibodies, including Humira, Keytruda, and Dupixent. Collectively, antibody sales that year likely topped $200B, roughly on par with the sales of Apple’s iPhone.

Antibodies play a central defensive role in the adaptive immune system. Recent decades have witnessed tremendous, hard-won advances in the science of these amazing molecular machines and in their application as research tools, diagnostic reagents, and therapeutics. 

Scientists have deciphered their molecular structures; decoded many of the intricate genetic and cellular processes that create and select functional antibodies; devised a variety of sophisticated approaches to identify novel antibodies that effectively bind a given antigen; and developed the tools and processes to reliably characterize, manufacture, and distribute them at scale. 

Newer therapeutic modalities that rely on antibodies or components of them for their function, such as antibody-drug conjugates, targeted radioligand therapies, bispecific T cell engagers, and CAR-T cells, have become established drug classes in their own right. And in recent years, in silico design techniques, aided by ML/AI, have been used to engineer antibodies with better binding, stability, and expression.

Despite all of this progress, our state of understanding regarding the vast diversity of the antibody repertoire actually produced in humans remains shockingly low.

Limitations in the characterization techniques previously applied to this diversity, estimated at 10^11 to 10^18 unique protein sequences in humans, have stymied efforts to fully understand and gain insights from it. Even the advent of next-generation sequencing has not deeply impacted this space—most studies of the antibody repertoire still rely on bulk sequencing technologies, which only capture half of most of the variable region of one antibody transcript at a time.

Why does this matter in antibody drug discovery? 

Because every day, inside every human, the body conducts the equivalent of 100 billion antibody clinical trials, testing each antibody for safety and efficacy in parallel. And these techniques have been developed and optimized over 500 million years of evolution of the adaptive immune system. 

For instance, by studying the immune reactivity of blood samples donated from adults living in a malaria-endemic region, researchers were able to identify broadly reactive antibodies that exhibited non-canonical features (Tan et al., Nature 529:105-109, 2016). These antibodies were found to contain a large insert of an extracellular LAIR1 domain located between key antibody segments. This domain, which is non-canonical and which was not observed in narrowly reactive antibodies, increased binding to malaria-infected red blood cells. These results demonstrated a novel mechanism of antibody diversification that the human immune system can use to create therapeutically effective antibodies.

Clearly, human B cells produce antibodies that mouse B cells and humanized mouse B cells do not. However, the most common methods for discovering therapeutic antibodies today rely on screening antibodies produced in transgenic mice that have been immunized with the desired target antigen, or panning for binding to the antigen in relatively shallow pools of engineered human antibody-like binders expressed via phage or yeast display. 

These approaches are not capable of leveraging the unique insights that can be captured by studying functional antibodies produced by the human immune system.

Enter Infinimmune.

Infinimmune is a startup that is reinventing antibody drug discovery by focusing solely on human-derived antibody drugs. 

Infinimmune was founded by Wyatt McDonnell, David Jaffe, Katie Pfeiffer, Lance Hepler, and Mike Gibbons, a multidisciplinary team of scientists and technologists. These founders have deep expertise in immunology, genomics, computational biology, single cell sequencing, and data analysis, and they take a first principles approach to therapeutics platform development as drawn from previous experiences at 10x Genomics, Pacific Biosciences, and the Broad Institute.

As an example of this expertise, Wyatt, David, and Lance co-authored a paper in Nature last year that discovered a new property of functional antibodies coined light chain coherence (Jaffe et al., Nature 611:352-357, 2022). In this work, the authors used single-cell RNA sequencing to determine the paired heavy and light chain antibody sequences from 1.6 million B cells from four unrelated humans and incorporated a total of 2.2 million B cells from 30 humans.

They compared antibody sequences from pairs of B cells that were isolated from different donors and which shared similar heavy chain segments, specifically, the same heavy chain V gene, and the same amino acid sequence for a key antigen-binding region called CDRH3. [Note: an antibody is composed of a pair of one heavy chain and one light chain that are generated through a process of sequential gene recombination involving V, D (for heavy chains), J, and C segments.]

The authors found evidence of previously unrecognized determinism in the light chain segment (i.e. light chain V gene) used in functional antibodies, which were derived from memory B cells, as opposed to naive antibodies. The discovery of light chain coherence suggests that the sequence space for the light chain of a functional antibody, which has undergone selection by the human immune system to be useful, safe, and effective, is more restricted than what was previously believed. It also carries important implications for the design of therapeutic antibodies, transgenic platforms, and diversification strategies of antibody drugs.

With these types of capabilities and insights at hand, Infinimmune is developing an end-to-end platform to deliver antibody drugs derived directly from the human immune system. 

These truly human antibodies are designed to drug new targets with improved safety and efficacy. Infinimmune is building its own pipeline of drug candidates while also aiming to partner with pharma companies to expand treatment options and reach more patients.

This past December, we were proud to co-invest alongside our friends at Playground Global, Civilization Ventures, and Axial in Infinimmune’s $12M seed round. We are delighted to work closely with the Infinimmune team, and we look forward to sharing many exciting updates to come. Infinimmune’s new HQ is in Alameda, and their team is always interested in hearing from smart, curious, and passionate scientists with a track record of innovation and building things from scratch. If you want to build better drugs for humans, from humans, you can reach the founders directly at founders@infinimmune.com or careers@infinimmune.com—there’s no better way to get in the hiring queue before more job postings go live in 2023!

Unleashing the power of Generative AI: an invitation to ambitious founders

We kicked off this week with the announcement of our 4th seed stage fund, one of the largest of its kind, raising $432M to seed the next generation of startups. Today we are thrilled to announce a dedicated Artificial Intelligence startup package, PearX for AI, which offers each founding team $250K in cloud credits, access to beta APIs, expanded check sizes up to $5M, 1:1 expert technical advice, customer introductions, AI talent matching and a curated community of AI practitioners who connect and learn from each other. Applications are open now through June 10th.

PearX for AI, is dedicated to the most ambitious technical founders, interested in building groundbreaking applications, tooling and infrastructure to power the Artificial Intelligence driven revolution. This program provides the resources, expertise, and customer connections you need to build the future. 

Who should apply?

PearX for AI will be a small tight-knit community with up to 10 startups selected into our inaugural summer batch. Pear specializes in working deeply with our portfolio companies to solve the most challenging problems startups face during the 0 to 1 journey from idea to product market fit. No idea is too early or too controversial. The program requires technical depth with a focus in AI, and the application process is tailored towards CS / Engineering graduates, PhDs, researchers and other technical professionals who have built AI driven applications in the past. We look for a combination of market knowledge, technical strategy and coding skills. The program will further build upon these skills and help round out your team’s capabilities in any areas that may need support. It will also connect you with Pear’s community of AI entrepreneurs.

Artificial Intelligence is a horizontal technology with the potential to impact many industries. We believe generative AI is a game-changer for consumer, social and enterprise applications. Particularly Healthcare, Legal, Financial Services, Retail, Logistics, Fashion, Design, Media, Gaming, Manufacturing, Energy, Industrial and Biotechnology to name a few verticals ripe for AI innovations. Pear’s AI team is especially skilled in enterprise AI adoption and will work with exceptional founders to craft solutions for the most pressing enterprise needs. 

We are deep technologists ourselves and value founders working on next generation Natural Language Understanding, Image generation, Computer Vision, Protein and Molecular synthesis and Robotics and Simulation technology. We have experts focused on applications of AI to developer tooling, open source software, sales, support, HR, R&D, design and education. We are bullish on infrastructure software, data services, and tools that reimagine the tech-stack for optimal AI performance.

What benefits will I receive?

Pear’s AI track comes with $250K in cloud credits, early access to new APIs and models, technical support from practitioners, mentorship from specialist AI experts, enterprise customer introductions, access to a talented and like-minded community, and an extended virtual cohort of AI founders. Pear will also extend larger check sizes for AI startups that require additional upfront investment, and provide introductions to strategic angel investors. Finally Pear will prepare all founders in the program for Series A fundraising. Our programs have an 87% success rate for series A and beyond investment by top tier funds.

Why now?

We’re living in an era of unprecedented technological advancement. AI is re-shaping our present and enabling some of the most significant breakthroughs of our lifetimes. The pace of this technological shift is breathtaking. The last time we witnessed a transformation of this nature was in 2000 with web technologies. There was a bubble then, just as there is significant hype around Generative AI now, but from it emerged breakout companies like Google, VMware, Salesforce and more. Breakout companies will be built now in this similar environment.

What does this mean for founders? Opportunity! There has never been a better time to start a company, but navigating the hype and shifting landscape that surrounds technical breakthroughs that are still in progress, requires expertise, judgment, and partners who will tell you hard truths and stand steady through difficult times, helping you secure the resources required – both financial and human.

Why Pear?

We have been investing in AI startups for several years and have a portfolio of AI powered companies that span verticals including consumer social, gaming, retail, healthcare, fintech, insurance, property technology, infrastructure, databases, deep tech, and more. 

With PearX for AI, we have pulled together the resources, community, and expertise to help founders discern signal from noise and succeed in building industry defining companies using the latest breakthroughs in Generative AI. The first track of PearX for AI is set to start in July, with a small cohort of founders who have a proven track record in applying AI to real-world problems. If you’re ready to take on this challenge and shape the future with AI, we invite you to apply now here. Let’s build the future together!

Announcing Pear Fund IV: $432M to power the future of tech

10 years ago, we started what is now Pear VC under the name Pejman and Mar Ventures. But the story dates even further back to 2009, when Pejman approached me with the goal to build a fund that serves world class entrepreneurs and supports their efforts with know-how, network, and capital. Pejman had a clear vision to build a seed stage firm with a true legacy: one that would be talked about in the history books.

By that time, Pejman had established himself as a savvy angel investor, and he even backed some of my own startups. When setting out to start a fund, he wanted to partner with someone that had a complementary skillset: while Pejman had over ten years of experience investing, I had founded three companies. It was a great match, but I was initially pretty reluctant to dive into the world of venture. Pejman, like any great founder, did not give up. He spent four years trying to convince me, and ultimately the two of us agreed to set out and raise an initial seed fund in 2013. Raising our first fund was not easy. After all, neither of us had any venture experience and we did not fit the mold of typical VCs.  After facing a series of no’s, a few brave LPs put their trust in us, and we were off to the races with a $50M seed fund.

 Me and Pejman in Pear’s first office in Palo Alto in 2013.

So here we are, 10 years later. We are incredibly proud of how far we have come, but we’re also well aware of how much lies ahead of us. Over the last decade, we’ve seeded over 150 companies including marquee companies like DoorDash (NSDQ: DASH), Guardant Health (NSDQ:GH), Senti Bio (NSDQ: SNTI), Aurora Solar, Gusto, Branch, Affinity, Vanta, Viz.ai and many more. 

Although we have come a long way since 2013, our DNA has not changed. Perseverance, can-do mentality, collaboration, service, and legacy remain the pillars of our fund.

The team has grown quite a bit. We now have a world class team of 26 (and growing!) Pear team members. Our investment team brings deep expertise across our vertical areas – from consumer to biotech to fintech to AI and beyond. We’ve also invested our resources in building a best-in-class platform team, with extensive backgrounds in company building – from talent to GTM to marketing and more.

Pear’s amazing team in our Menlo Park HQ.

Just like on day one of Pear, we are at the service of our founders. When we partner with a company, we are an extended member of their team and we do whatever it takes to help them be successful. We tell founders to think of us as “Ocean’s Eleven”: we’re a unique cast of colorful characters, with specific skills, a common plan, and coordinated execution.  In fact, coordinated collaboration is at the heart of what we do. 

We remain as optimistic as day one. Over the last decade of building Pear, we have witnessed the market go through its fair share of ups and downs. Despite the current economic downturn, we firmly believe that there is no better time to invest at the seed stage. The market is teeming with exceptional talent starting companies, the advent of AI is propelling company building at an unprecedented pace, and sales and marketing can be done at scale with fewer resources. In light of these factors, we are confident that the next wave of iconic companies will emerge from this downturn, and we are looking forward to being their initial backers.

This week, we celebrate raising our fourth fund at $432 million, but we know that fundraising is just one milestone. We have our eyes set on the decades that lie ahead, and we are already hard at work building new initiatives that will help us deliver on our promise to back and support early-stage companies. 

Since day one, we’ve built Pear on this belief that people truly make the difference. We are deeply grateful to our LPs and to our founders who put their faith in us as partners every day. 

We look forward to building the future of tech with Pear Fund IV. We couldn’t be more excited for the next decade of Pear!

PearX Demo Day W23

Over the past decade, Pear has spearheaded pre-seed rounds for exceptionally early-stage companies. Beginning of this year, PearX was designed to be a 14 week bootcamp for ambitious founders that has proven to be a breeding ground for category-defining pioneers such as Affinity, Xilis, Capella Space, Nova Credit, Cardless, and Viz.ai.

The W23 cohort of PearX was no exception and consisted of thirteen cutting-edge companies pushing the boundaries of AI, healthcare, consumer products, fintech, and climate solutions. Our Demo Day for W23 happened on May 25th and was hugely successful: 1500+ investors tuned into the livestream and we’ve already been able to facilitate more than 650+ investor introductions for the 13 participating companies. 

With an acceptance rate of less than 0.25%, these companies have risen to the top among a pool of 4,500 applicants, a testament to their exceptional talent and groundbreaking ideas. Within this cohort, we also have significant female leadership presentation with over 40% of the CTO/CEOs being female (only 9% of venture-backed entrepreneurs are women). We’re proud that PearX W23 is setting a strong example of gender diversity and equity in entrepreneurship. 

Within this cohort, we also have significant female leadership presentation with over 40% of the CTO/CEOs being female (much higher than industry average).

So, without further ado, let’s delve into the extraordinary companies that make up the PearX W23 cohort:

AI:

Hazel

Founders: Henry Weng and Vedant Khanna 

Hazel is the AI-powered operating system for realtors that 10x’s their productivity and increases sales. Nearly half of a realtor’s work consists of repetitive backend tasks like middleman communication, document preparation, and project management. Realtors can’t manage backend work on their own as their business scales, so the largest ones operate like SMBs with staff, software, and assistants. Hazel integrates with a realtor’s email, text, and knowledge bases, leveraging AI to parse unstructured data and become their single system of record (and supercharged CRM). From that point, Hazel automates routine tasks using generative AI, freeing realtors to focus on getting more clients instead of more paperwork.

SellScale
Founders: Ishan Sharma and Aakash Adesara

SellScale is an AI-powered Sales Development Representative. SellScale streamlines the end-to-end process of setting demos for salespeople, addressing challenges such as high SDR turnover rates and low email conversion rates. By leveraging AI-generated copy that outperforms human-produced content, SellScale offers scalable and data-driven solutions, allowing sales teams to achieve superior results in their outreach efforts without excessive investments in personnel and tools.

Tare
Founders: Eileen Dai and Max Sidebotham

Tare is an AI-powered email marketing solution that makes data-driven automation easy for modern e-commerce brands. Until now, email has been an extremely manual & resource-heavy marketing channel, and it wasn’t possible to automate everything from end-to-end. 66% of current e-commerce email spend goes to labor & agency service costs. Tare helps brands optimize their email marketing with automated customer segmentation, AI-generated content & imagery, and automated scheduling for delivery.

Saavi

Founders: Maya Mikhailov 

Saavi revolutionizes enterprise AI deployment, eliminating costly and complex processes. This user-friendly platform enables effortless implementation without the need for developers or data scientists, providing quick deployment in minutes. With its intuitive interface and self-optimizing capabilities, Saavi empowers businesses to unlock the potential of AI, delivering tailored insights for informed decision-making in areas like fraud risk assessment, customer churn prediction, and growth identification. Embrace the future of AI deployment with Saavi and experience ease, speed, and accuracy in transforming your enterprise.

Consumer: 

Champ

Founders: Ryan Rice

Champ is a disruptive fantasy sports platform that caters specifically to college sports fans. Unlike major platforms that focus on professional leagues, Champ offers an ownership-based fantasy experience for college football and basketball. It empowers fans to buy, sell, and collect digital trading cards featuring their favorite collegiate athletes, using them to construct lineups for fantasy leagues. By addressing the weak ties many fans have to professional leagues and capitalizing on the significant influence of college sports in certain areas, Champ allows users to actively participate in the excitement of college sports and forge deeper connections with their teams.

Ego

Founders: Vishnu Hair  and Peggy Wang

Ego is an immersive live streaming platform that uses Gen AI to create fully face-tracked 3D avatars. We believe in a future where streaming as a virtual avatar surpasses real-life live streaming, driven by the digital natives of Gen Z who seek pseudonymous online identities. To realize this vision, we developed an app that enables users to generate a 3D avatar, which perfectly mirrors their facial expressions, and live stream on platforms like Twitch and YouTube in 90 seconds. Users can profit from selling virtual goods, customizing avatar appearances, engaging in entertaining mini-games, or orchestrating immersive role-plays. 

Afterhour

Founders: Kevin Xu

AfterHour revolutionizes the retail stock trading experience by providing a pseudonymous and voyeuristic social network that focuses on verified trades and stock picking gurus, addressing the lack of trust and entertainment in the online trading community. By connecting brokerage accounts via Plaid and making portfolios public, users gain transparency and can distinguish between legitimate shareholders and random lurkers, while maintaining anonymity. AfterHour’s unique approach combines anonymous yet verified engagement, bridging the gap between authenticity and privacy in a way that no other platform currently does.

Pando

Founders: Charlie Olson and Eric Lax

Pando introduces Income Pooling, a solution that offers risk diversification and fosters a financially-aligned community to address the increasing shift toward a winner-take-all economy. By leveraging machine learning and artificial intelligence, Pando accurately predicts future earnings, enabling groups of high potential individuals to pool their income. With product-market fit in professional baseball and entrepreneurship, Pando has created a new asset class for all professionals in power-law careers by unlocking the value of potential income and reimagining the financial services experience for these individuals.

Juni

Founders: Vivan Shen

Juni revolutionizes personalized tutoring by bringing it online, addressing the scalability challenges and affordability issues of traditional private tutoring. With purpose-built AI models, Juni provides tailored content, questions, and hints to meet individual students’ needs while incorporating context and motivation. The platform establishes a feedback loop with students to customize instructor feedback and enhance the tutoring experience. By leveraging a specialized dataset and the expertise of top instructors, Juni ensures high-quality tutoring for all students, aiming to make educational support accessible and empowering them to thrive.

Climate:

Lasso

Founders: Nicole Rojas and David Pardavi  

Lasso is reshaping on-farm emissions reduction with an automated software platform that streamlines data collection, carbon footprint calculation, and form filling, reducing the time spent from over 60 hours to just one hour per farm. Seamlessly integrating with existing on-farm software systems, Lasso extracts vital information and provides real-time insights through integration with carbon calculator tools. By automating verification, funding forms, and auditing reports, Lasso eliminates manual creation and consolidates farm, project, and emissions data for efficient management and cost savings, making GHG emissions reduction a reality at scale.

Nuvola

Founders: Janet Hur

Nuvola is an advanced battery materials company focused on enhancing battery profitability and eliminating the primary cause of battery fires. With their innovative Safecoat Direct Deposition Separator technology, they reduce reject material costs by 82% and improve yield by 50%, addressing the challenges faced by the Lithium-Ion Battery Industry, which is projected to exceed $200 billion in the next five years. Their solution tackles the error-prone process of folding separator film, which has been responsible for catastrophic fires and billions of dollars in recalls. By providing safer and more efficient batteries, Nuvola supports the growth of electric vehicles, energy storage, and e-aircraft industries.

Healthcare:

Stellation

Founders: Michelle Xie and  Diana Zhu 

Stellation is rethinking patient-provider matching with its analytics platform, addressing the lack of accurate information in matching patients with healthcare providers based on their specific health needs. Current practices rely on proximity and availability, neglecting performance data and resulting in limited outcomes and cost savings. Stellation’s SaaS platform bridges this gap by leveraging claims and patient data to profile provider strengths and match individuals with the most suitable healthcare provider. Built on a research-backed methodology, this powerful solution is delivered via API integration for health plans or a user-friendly search interface for patients. By utilizing Stellation, health plans can achieve significant cost savings and improved outcomes for their members.

Enterprise:

Polimorphic

Founders: Parth Shah and Daniel Smith

Polimorphic revolutionizes government operations by automating back-office tasks and transforming how cities track and manage requests, replacing outdated analog methods with efficient digital solutions. With a significant reliance on post-it notes and paper files, government processes face inefficiencies and the impending challenge of a labor shortage. Polimorphic’s focus on the customer service layer of government empowers cities to streamline operations, enhance compliance, and improve the overall citizen experience by offering solutions for constituent requests, payment collection, request tracking, and data management.

Today’s markets are shifting faster than ever before, and Pear is dedicated to evolving alongside them. Each of these 13 company from PearX W23 showcases immense potential for disrupting their respective industries and driving positive change through the power of technology.

At Pear, we are honored to play a pivotal role in the journey of these exceptional startups. We believe in the transformative power of early-stage investments and the tremendous impact they can have on shaping the future. As we continue to identify and nurture the next generation of trailblazers, we remain steadfast in our commitment to fostering innovation, driving growth, and unlocking the extraordinary potential within each entrepreneur we encounter.

We believe in the transformative power of early-stage investments and the tremendous impact they can have on shaping the future.

If you want to partner with any of these companies, please check them out at: https://demoday.pear.vc/. For anyone who missed our Demo Day, we’ll be sharing videos of each team this Thursday, so stay tuned!

Pear and Pacific Western Bank Partnership

We have some exciting news to share: Pear and Pacific Western Bank have partnered up to bring our pre-seed and seed startups a deeper level of banking products and services. With this partnership, PWB will be offering dedicated support and exclusive banking offerings to Pear founders to help them scale their startups from the earliest days.

Pear has been focused on pre-seed and seed investing for 10 years now, partnering with founders at the earliest stages to turn great ideas into category-defining companies. We know now more than ever that banking partners are a really critical part of the startup ecosystem. 

“We love partnering with PWB, because they’re truly dedicated to supporting the growth of entrepreneurs. They understand the challenges that come with scaling and offer a wide range of banking services and products. Many of our portfolio companies have partnered with PWB, and we’ve seen that they can offer the banking structure and resources startups need to navigate these challenging times,” says Pejman, Founding Managing Partner of Pear.

As part of this partnership, PWB will provide hands-on training to PearX, Female Founders Circle, and Pear Dorm members, offering to help navigate all things startup banking – from setting up your first corporate bank accounts to understanding the implications of the current market. 

“Pacific Western Bank is very excited to partner with Pear VC and support our mutual commitment to innovation in the technology and life science ecosystems”, added Mark diTargiani, SVP, Venture Debt & Startup Banking at PWB. “The Pear team has built a legacy of investing in quality people and companies, and our team has 15+ years of supporting VCs and founders as they build companies. We are aligned on helping founders from inception to exit, and by combining our networks and resources we can accomplish that in Silicon Valley and beyond.”

We’re excited for the expertise PWB brings to the table, helping our startups navigate the current financial landscape and be better set up to grow and surpass their business goals.

Perspectives in Generative AI with Stanford Researchers

We recently hosted a fireside chat on safe and efficient AI with notable Stanford CS PhD researchers Dan Fu and Eric Mitchell. The conversation covered various aspects of AI technology, including the innovations that Dan and Eric have pioneered in their respective fields.

Dan is co-inventor of FlashAttention. He’s working on improving efficiency and increasing the context length in Large Language Models (LLMs). His experience in developing groundbreaking AI technologies allows him to provide profound insights into the future capabilities of LLMs. During the event, Dan discussed the implications of his work on enabling new generative AI use cases, as well as brand new techniques for efficient training.

Eric’s work focuses on AI safety and responsibility. He is the co-author of DetectGPT, a tool capable of differentiating between AI-generated and human-generated text. In recent times, DetectGPT has gained press attention for its innovative approach to addressing the growing concern with AI-generated content. Eric shared his thoughts on the potential impact of DetectGPT and similar tools, discussing the necessity for safe AI technologies as the field expands.

During the discussion, we touched on  practical applications of generative AI, and the forecast for open source vs. proprietary LLMs. We also touched on the prospect of AGI, the ethical ramifications, cybersecurity implications, and overall societal effects of  these emerging technologies.

For those who couldn’t attend in person, we are excited to recap the high points today (answers are edited and summarized for length):

Aparna: Can you tell us a bit about yourselves and your motivation for working in AI?

Dan: I focus on making foundation models faster to train and run, and I’m interested in increasing sequence length to allow for more context in the input data. Making sure you’re not limited by a specific number of tokens. But you can feed in as much data as you’d like, as much context and use that to teach the model what you want to say. I’ve been interested in machine learning for a long time and have been at Stanford for five years now. It’s a thrilling time to work in this field.

Eric: I’m a fourth-year PhD student at Stanford, and I got into AI because of my fascination with the subjective human experience. I’ve taken a winding road in AI, starting with neuroscience, 3D reconstruction, robotics, and computer vision before being drawn to the development of large language models. These large language models are really powerful engines, and we’re sort of just starting to build our first cars that can drive pretty well. But we haven’t built the seatbelts and the antilock brakes, and these safety and quality of life technologies. So that’s what I’m interested in.


Aparna: What major breakthroughs have led to the recent emergence of powerful generative AI capabilities? And where do you think the barriers are to the current approach?

Dan: That’s a really great question. There has been a seismic shift in the way that machine learning (ML) is done in the past three to four years. The old way was to break a problem into small parts, train models to solve one problem at a time, and then use those building pieces to build up a system. With foundation models, we took the opposite approach. We trained a model to predict the next word in a given text, and these models can now do all sorts of things, like write code, answer questions, and even write some of my emails. It’s remarkable how the simplest thing can scale up to create the largest models possible. Advances in GPUs and training systems have also allowed us to scale up and achieve some incredible things.

I think one of the barriers is the technical challenge of providing sufficient context to the models, especially when dealing with personal information like emails. Another barrier is making these models more open and accessible, so that anyone can see what goes into them and how they were trained. So making the process more open the same way that anybody can look at a Kubernetes stack and see exactly what’s happening under the hood. Or anybody can open up the Linux kernel and figure out what is running under there. those are frontiers that I hope that we push on pretty quickly. This would enable better trust and understanding of the models.

Eric: I agree with Dan’s points. Additionally, a challenge we’re facing is the need to solve specific problems with more general models. However, we’ve found that large scale self-supervised training can be effective in tackling these specific problems. For example, the transformer architecture has been helpful in representing knowledge efficiently and improving upon it. In general, the ability to do large scale self-supervised learning on just a ton of data has been key to the recent progress.

Furthermore, we need a way to explain our intent to the model in a way that it can correctly interpret and follow it. This is where the human preference component comes in. We need to be able to specify our preferences to the model, so that it can draw upon its knowledge and skills in a way that is useful for us. This is a qualitative shift in how these models interact with society, and we are only scratching the surface.

Aparna: I’d like to go a little bit deeper technically. Dan, could you explain how your work with attention has made it possible to train these large generative AI models?

Dan: Sure, I can give a brief overview of how attention works at a high level. So you have these language models, and when you give it a sentence, the attention mechanism compares every word in that sentence to every other word in that sentence. If you have a databases background, it’s kind of like a self join, where you have a table that is your sentence, and then you join it to itself. This leads to some of the amazing abilities that we’ve seen in generative AI. However, the way that attention used to be calculated was quite inefficient. You would compare every word to every other word, resulting in a hard limit on the context of the models. This meant that the maximum context length was around 2000, which is what could fit in memory on an A100 GPU.

If you look at databases and how they do joins, they don’t write down all the comparisons between all the joins, they do it block by block.About a year ago, we developed an approach called Flash attention which reduced the memory footprint by doing the comparisons block by block. This enabled longer context lengths, allowing us to feed in a whole essay instead of just a page of text at a time. We’ve been really humbled by the very rapid adoption. It’s in PyTorch, 2.0. GPT-4, for example, has a context length of 8k, and a context length option of 32k.

Aparna: That’s really interesting. So, with longer context lengths, what kinds of use cases could it enable?

Dan: The dream is to have a model that can take all the text ever written and use it as context. However, there’s still a fundamental limitation to attention because even with a reduced memory footprint, you’re still comparing every word to every other word. If you think about how language works, that’s not really how we process language.  I’m sure you can’t remember every word I’ve said in the past few minutes. I can’t even remember the words I was saying. That really led us to think okay, are there some alternatives to attention that don’t scale fundamentally quadratically. We’ve been working on some models called Hungry Hungry Hippos. We have a new one called hyena, where we try to make the context length a lot longer. And these models may have the potential to go up to hundreds of thousands of words, or even millions. And if you can do that, it changes the paradigm of what you can do with these models. 

Longer context lengths enable more complex tasks such as summarization, question answering, and machine translation. It also allows for more efficient training on large datasets by utilizing more parallelism across GPUs. But if you have a context length of a million words, take your whole training set, feed it in as input, you could have an embodied AI, and have say a particular agent behave in a personalized way when responding to emails or talking to clients.

Longer context can also be particularly useful in modalities like images, where it means higher resolution. For example, in medical imaging, where we are looking for very small features, downsampling the image may cause loss of fine detail. In the case of self-driving cars, longer context means the ability to detect objects that are further away and at a higher resolution. Overall, longer context can help us unlock new capabilities and improve the accuracy of our models.

Aparna: How do you see the role of language models evolving in the future?

Dan:  I think we’re just scratching the surface of what language models can do, and there are so many different ways that they can be applied. One of the things that’s really exciting to me is the potential for language models to help us better understand human language and communication. There’s so much nuance and complexity to how we use language, and I think language models can help us unpack some of that and get a better understanding of how we communicate with each other. And of course, there are also lots of practical applications for language models, like chatbots, customer service, and more.

Personally, I’m very excited to see where small models can go. We’re starting to see models that have been trained much longer than we used to train them, like a 7 billion parameter model, or a 13 billion parameter model, that with some engineering, people have been able to get to run on your laptop. When you give people access to these models, in a way that is not super expensive to run, you’re starting to see crazy applications come out. I think it’s really just the beginning. 

Eric: It has been an interesting kind of phase change just going from GPT3 to GPT4. I don’t know how much people have played with these models side by side or if people have seen Sebastien Bubeck’s somewhat Infamous First Contact talk now where he kind of goes through some interesting examples. One thing that’s weird about where the models are now is that usually, the pace of progress was slower than the time it took to understand what the capabilities of the technology were, but recently, it felt like a bit of an inversion. I would be surprised to see this slowdown in the near future. And I think it changes the dynamic in research.

Most machine learning research is quantitative, focused on building models, evaluating them on datasets, and getting higher scores. However, Sebastien’s talk is interesting because it evaluates models qualitatively with no numbers, which feels less rigorous but has more credibility due to Sebastien’s rigorous research background. The talk includes impressive examples, such as a model drawing a unicorn or writing 500 lines of code for a 3D game. One fascinating example is the model coaching people in an interpersonal conflict, providing direct and actionable advice that is useful in real-life situations. A big caveat is that current outputs from GPT-4 are much worse than the examples given in the talk. Sebastien’s implication or claim is that aligning the model to follow human intent better reduces its capabilities. This creates a tough conflict between economic incentives and what’s useful for society. It’s unclear what people will do when faced with this conflict.

Aparna: Do you think there will be ethical concerns that arise as language models become more sophisticated?

Eric: Yeah, I think there are also going to be questions around ownership and control of these models. Right now, a lot of the biggest language models are owned by big tech companies, and there’s a risk that they could become monopolies or be used in ways that are harmful to consumers. So we need to be thinking carefully about how we regulate and govern these models, and make sure that they’re being used in a responsible and ethical way.

One of the big challenges is going to be figuring out how to make language models more robust and reliable. Right now, these models are very good at generating plausible-sounding text, but they can still make mistakes and generate misleading or incorrect information. So I think there’s a lot of work to be done in terms of improving the accuracy and reliability of these models, and making sure that they’re not spreading misinformation or bias.

Aparna:  Given your PhD research Eric, what are the main areas that warrant concern for AI safety and responsibility?

Eric: In summary, there are three categories of issues related to AI ethics. The first category includes concrete near-term problems that many in the AI ethics community are already working on, such as unreliable and biased models that may dilute collective knowledge. The second category is a middle-term economic alignment problem, where incentives in industry may not be aligned with making models that are safer or more useful for society. The third and longest-term category involves high-stakes decisions made by very capable models, which could be used by bad actors to do harm or may not align with human values and intentions. While some may dismiss the risks associated with these issues, they are worthy of serious consideration.

My research is focused on developing auxiliary technologies to complement existing mass-produced products. I am specifically working on model editing, pre-training models in safer ways, and developing detection systems for AI-generated texts. The aim is to give practitioners and regulators more tools to safely use large language models. However, measuring the capabilities of AI systems is challenging, and my team is working on building a comprehensive public benchmark for detection systems to help better assess their performance.

Aparna: I’m excited about the prospect of having evaluation standards and companies building tooling around them. Do you think there’ll be regulation? 

Eric: In my opinion, we can learn from the financial crisis that auditors may not always work in practice, but a system to hold large AI systems to sensible standards would be very useful. Currently, there are questions about what capabilities we can expect from AI systems and what technologies we have to measure their capabilities. As a researcher, I believe that more work needs to be done to give regulators the tools they need to make rules about the use of AI systems. Right now, we have limited abilities to understand why an AI model made a certain prediction or how well it may perform in a given scenario. If regulators want to require certain things from AI model developers, they need to be able to answer these questions. However, currently, no one can answer these questions, so maybe the only way to ensure public safety is to prohibit the release of AI models until we can answer them.

Aparna: Stanford has been a strong contributor to open source and we’ve seen progress with open models like Alpaca, Dolly, and Red Pajama. What are the advantages and disadvantages of open sourcing large language models?

Dan: As an open source advocate and a researcher involved in the Red Pajama release, I believe making these large language models open source can help people better understand their capabilities and risks. The release of the 1 trillion token dataset allowed us to question what goes into these models and what happens if we change their training data. Open sourcing these models and datasets can help with understanding their inner workings and building on them. This is crucial for responsible use of these models.

The effort behind Red Pajama is to recreate powerful language models in an open manner by collecting pre-training data from the internet and human interaction data. The goal is to release a completely open model that is auditable at every step of the process. Small models trained on a lot of text can become surprisingly powerful, as seen in the 7 billion parameter model that can fit on a laptop. The llama model by Facebook is not completely open, as it requires filling out a form and has questionable licenses.

Eric: The open source topic is really interesting. I think many people have heard about the call for a pause on AI research letter. Open source is great, and it’s why OpenAI relies on it a lot. However, a few weeks ago, a bug in an open source framework they were using caused some pretty shocking privacy violations for people who use Chat GPT, where you could see other people’s chat histories. In some sense, I think the cat is already out of the bag on the open source question. The pre-training phase is where a lot of the effort goes into these models, and we already have quite a few really large pre-trained models out there. So even if we paused right now and said no more big pre-trained models can be released, there’s already enough out there for anyone who is worried about it to worry a lot.

Aparna: So with these smaller models, running on laptops and on mobile and edge devices what new use cases will open up? 

Dan: Sure, I think it’s amazing that our phones have become so powerful over the past decade. If I could have a language model running on my phone that functions as well as the GPT models we have today, and can assist me in a conversational way, that would be awesome.

Eric: I think it’s exciting and cool from a privacy perspective to have these models running locally. They can be really powerful mental health professionals for people, and I believe these models can be meaningful companions to people as well. Loneliness sucks, and the COVID years have made this very clear to a lot of people. These are the types of interactions that these models are best suited for. They understand what we’re saying, they can respond intelligently, and they can ask us questions that are meaningfully useful.

From this perspective, having them locally to do these types of things can be really powerful. Obviously, there’s a significant dual-use risk with these models, and we’ve tried to do some work to partially mitigate these things. But that’s just research right now. There are already very real and powerful models out there.

I think it’s great and exciting, and I’d be lying if I said I can’t foresee any way this could be problematic in some ways. But the cat is out of the bag, and I believe we will see some really cool and positive technologies from it.

Aparna: My final question is about Auto GPT, a new framework that uses GPT to coordinate and orchestrate a set of agents to achieve a given goal. This autonomous system builds upon the idea of using specialized models for specific tasks, but some even argue that this approach could lead towards AGI. Do you believe this technology is real and revolutionary? 

Eric: Yes, Auto GPT is a real framework that uses large language models to critique themselves and improve their performance. This idea is powerful because it suggests that models can improve themselves without the need for constant human feedback. However, Auto GPT is not yet advanced enough to replace human jobs as it can still get stuck in loops and encounter situations where it doesn’t know what to do. It’s also not trustworthy enough to handle tasks that require a high level of complexity and verification. While the ideas behind Auto GPT are promising, it’s not a revolutionary technology in and of itself and doesn’t massively improve the capabilities of GPT.

Dan: So, I was thinking about what you said earlier about the generative AI revolution and how it’s similar to the internet boom in 2000. But I see it more like electricity, it’s everywhere and we take it for granted. It’s enabled us to do things we couldn’t before, but it has also displaced some jobs. For example, we don’t have lamplighters or people who manually wash clothes anymore. However, just like how people in the early 20th century imagined a future where everything would be automated with electricity, we still have jobs for the moment. It’s hard to predict all the impacts AI will have, but it will certainly change the types of jobs people are hired for. I think it’ll become more integrated into our daily lives and introduce new challenges, just like how electrical engineering is a field today. Maybe we’ll see the emergence of foundation model engineering. That’s just my two cents on AGI – I’m not sure if it’ll be fully realized or just a tool to enhance AI capabilities.

Eric: I think the employment question is always brought up in discussions about AI, but it’s not clear that these models can replace anyone’s job right now or in the near future. They are good for augmenting people, but not at tasks they’re not already qualified for. It’s not a drop-in replacement for humans. I don’t think we’ll see mass unemployment, like with the electricity revolution. The internet analogy is similar, in that it was thought to make people more productive, but it turned out to be a distraction tool as well. Generative AI may not have a net positive impact on productivity in the near term, but it will certainly entertain us.

AR/VR/XR/PEAR: our call for mixed reality builders 

In the past few months, we have seen the beginnings of rising interest in building AR, VR, mixed reality, and extended reality infrastructure and applications (we’ll just call it “XR” for short): XR applications to PearX are up 5x this year, dedicated XR hacker groups are proliferating at top engineering schools like Harvard and Stanford, and our tracking shows hundreds of founders have left venture-backed and major tech companies to build in the XR space.  

We think this is because XR has the potential to represent one of the most consequential new platforms of the coming decade, and there is substantial alpha to be had for builders who study this burgeoning space and seize early opportunities. 

We expect interest in building in XR only to increase dramatically – particularly following Apple’s upcoming Reality Pro headset announcement. We see builders with a measured sense of urgency having an advantage, and we’ve put together a high-level guide for exploring ideas in XR. What follows is merely one way of cataloging opportunities; we would love to meet and speak with founders who are building early, quickly, and curiously in the broader XR space. 

XR Builder’s Guide to Exploring Ideas

A Familiar “Infrastructure and Applications” Approach

With any new technology, there are opportunities in foundational infrastructure (making the technology easier to deploy or adding capabilities to what can be built for / with it) and novel applications (tools built with the new technology to help users achieve something they could not previously do).

This approach often starts by asking what new infrastructure developers are building, and then asking what applications can be built atop it. 

In XR, substantial existing infrastructure will be first-party specific to headset makers. So, it is worth considering what initial applications may be built on foundations purpose-built for available devices – and which use cases may find breakout success among users of those devices. 

XR applications

Consumer

Gaming appears to be the first breaking wave of consumer XR, and will likely lead the way for the foreseeable future. Unity’s 2023 report showed that more than half of all console developers are now building for VR platforms, too. It’s been said that “Every game developer wants to be a VR game developer – they just need to find a way to get paid for it.” This may not be a problem soon enough.

According to The Economist and Omdia, global gaming spending will eclipse $185B this year, with half of consumer spend going to mobile games. As AAA titles and boredom-busting mobile games alike are rendered in XR, it stands to reason that anyone with access to an XR device will prefer gameplay in a more immersive form – meaning that a sizable share of the gaming market may shift to XR in the next decade.

Gamified consumer training is already proving its effectiveness in athletics: thousands of baseball players, amateur and professional alike – including the reigning MVP – use WIN Reality’s Quest app to perfect their swings, with measurable improvements on their performance. 

We are also excited about consumer applications in more passive streaming entertainment, social content sharing, education, and commerce. Many of the hours of daily screen time spent watching asynchronous content – social media feeds or professional productions – or browsing e-commerce sites may feel more vivid in an immersive environment. 

Enterprise 

Hardware fragility and cost may prevent near-term widespread enterprise adoption of B2B XR across business types. Meanwhile, few people – including us – are excited about open-ended co-working in an immersive XR environment. 

But, there are impactful vertical and horizontal applications alike that may soon make enterprise XR familiar and essential in many use cases, especially at large companies. Horizontal enterprise tools may include general-purpose training and demo environments: collaborative spaces built to allow anyone to host a classroom or sales presentation. Early deployments of immersive training tools have shown efficacy in use cases as diverse as nursing, UPS driver safety, and law enforcement.    

For more specialized B2B applications, initial verticals may include architecture and interior design, product design and mechanical engineering, repair and maintenance diagnostics and installation, and healthcare diagnostics and treatment simulation – among many other sectors.

Key questions for XR application builders 

With any application, we encourage prospective founders to consider first-party risk: What core applications are platform creators likely to want to own? FaceTimeXR may be a more likely winner than a new developer’s video chat app for Apple RealityPro. But, a social app to flip through your photo library immersively, in real-time and alongside the friends in those images, may be less likely in Apple’s first-party domain.  

We also encourage XR builders to have an airtight “why XR” case for any application: what vital benefit does an immersive, co-present environment offer to your user over alternative interfaces? 

XR Infrastructure

Developer tools 

Wise founders will study the initial wave of XR adoption and ask which high-use applications are breaking, underwhelming, or borderline impossible on existing first-party or open infrastructure. Many of the most most compelling opportunities will be in bolstering core experiences: interactivity and copresence, audio, messaging, 3D asset generation, 2D/3D web interoperability, streaming reliability and observability. 

Monetization enablement 

An entire ecosystem of companies will support familiar business models, applied to XR use cases. While many elements of these business models may be unchanged for a new interface, there will undoubtedly be novel components. E-commerce checkout flows will feature transaction-splitting for social shopping. A wave of analytics and marketing tools will help businesses identify lucrative, impression-rich actions users take in XR applications, and usage-based billing providers will emerge to track and monetize novel ways of product use. 

Key questions for XR infrastructure builders 

Although removed from the end XR application consumer, XR infrastructure builders should start with this headset-adorned or AR-phone-app wielding user profile in mind. In a nascent space, an infrastructure builder needs to be sure there are enough end users who will in turn adopt what your own customers build with your tool or on your platform. Even if a developer can build something novel and powerful with your tools, your success relies on that developer’s distribution.   

===

These are merely a few of the possible areas to explore. The promise of XR lies in the experiences we cannot yet clearly see, and the infrastructure built to enable them. If you’re building now — send us a note to XR@pear.vc. And if you’ll be in LA for LA Tech Week, join us on 6/7 at our XR breakfast — there will be plenty to discuss the week of Apple WWDC!