Topic: Product & Engineering

On March 1st, Pear’s Aparna Sinha hosted a fireside chat with Hussein Mehanna, SVP of Engineering for AI/ML at Cruise for a discussion on next generation AI/ML technologies. Hussein has a long and deep history of innovation in machine learning engineering spanning speech recognition, language models, search, ads, and ML platforms at companies including Google, Facebook and Microsoft. He is currently focused on ML-driven robotics, especially autonomous vehicles at Cruise.

This is the first in a series of AI/ML events Pear is hosting. To hear about future events, please sign up for our newsletter and keep an eye on our events page.

The exciting conversation lasted for over an hour, but below is a summary with some highlights from the talk between Aparna and Hussein:

Q: You’ve been building products at the forefront of AI throughout your career, from search, to speech to ML platforms and now robotics and autonomous vehicles. Tell us a little bit about your journey, and the evolution of your work through these products?

A: My journey began with a scholarship for neural networks research in 2003, followed by a role at Microsoft. I eventually joined Facebook and worked on Ads that pushed the limits of ML and from there moved to a more broadened role of working with ML platforms across the company. I then joined Google Cloud’s AI team to explore disruption of enterprise through ML. I learned over the years that robotics is the biggest field facing disruption with machine learning, and autonomous vehicles is the biggest application of that. So I joined Cruise both out of interest in robotics and a pure interest in cars. 

Q. Ads, in fact, also at Google was the birthplace of a lot of the advanced AI. And now AI is absolutely into everything. 

A: Absolutely. There was a system in Google Ads called Smart ass. It was actually one of the first known large scale machine learning systems. And the person who developed them, Andrew Moore, eventually became my manager at Google Cloud AI. You’d be surprised how many lessons to be learned from building machine learning for ads that you could implement in something as advanced as autonomous vehicles.

You’d be surprised how many lessons to be learned from building machine learning for ads that you could implement in something as advanced as autonomous vehicles.

Q: We are seeing the emergence of many AI-assistive products, co-pilot for x, or auto-pilot for y. But you’ve spoken about AI-native products. Are AI-assistive products and AI-native products fundamentally different?

A: Yes, they are. An AI-native product is one that cannot exist, even in MVP form, without machine learning. Examples include autonomous vehicles or speech recognition software like Alexa. On the other hand, AI-assistive products can help humans in various ways without necessarily using machine learning. In fact, Google search, people may not know that, but Google Search started with more of a data mining approach than machine learning. 

Q: What is the gap between building an AI-assistive product versus an AI-native product?

A: The gap is huge. Building an AI-native product assumes full autonomy, while building an AI-enhanced product assumes a human being will still be involved. For example, the technology used for driver-assist features (level 1-3 autonomy) versus fully autonomous driving (level 4-5 autonomy) require vastly different approaches and parameters.  Autopilot is actually classified as Driver Assist. But then once you remove the driver completely, from behind the wheel, you get into level 4, level 5, autonomy. Level 5 is maybe less dependent on a predefined map, you could just throw the robot anywhere, and they’ll figure its way.  It’s very important for founders, entrepreneurs, product managers to understand, are they building something that assists human beings, and therefore assumes a human being, or something that completely replaces them.

Q: Where do generative AI and GPT technologies fall on the spectrum?

A: Generative AI and GPT – so far – are human-assisted technologies that require a human being to function properly. Today, they are not designed to replace humans like technologies used for level 4-5 autonomy.

Q: At a high level, what are the components and characteristics of a fully autonomous system? I’ve heard you call it an AI brain.

A: So let me project the problem first, from a very high level on driving, because I suspect most of us have driven before. For a full autonomous system the first component is perception, you need to understand the environment,  and essentially describe the environment as the here and now. This is a vehicle, it’s heading this direction, with this velocity; here’s a pedestrian, he or she is x distance away from you, and they’re heading that way, and that’s their velocity. Here’s a pile of dirt. And here’s a flying plastic bag. And here’s something that we don’t know what it is, right? So perception is extremely important. Because if you don’t understand the environment around you, you don’t know how to navigate it. 

Now, what’s very, very important about perception is that you can’t build a perception system that is 100% perfect, especially a rich system that describes all sorts of things around you. And so one of the lessons we’ve learned is, you can build multiple levels of perception. You can build a level of perception that is less fine grained. A machine learning system that just understands these two categories can generalize better. And it’s very important for your perception system to have some self awareness so that it tells you the rich system is confused about this thing here. So let’s go to the less  sophisticated system and understand whether it’s something that is safe to go through or go around. Now the reason why you need the rich system is because it gives you rich information. So you can zip through the environment faster, you can finish your task faster. And if your rich system is accurate, let’s say x percent of the time with a little bit of unsureness, then it’s okay to drive a little bit slower using the less rich, less refined system. So that’s number one about perception. 

The second component of autonomous driving is prediction, which involves understanding how agents in the environment will interact with each other. For example, predicting whether a car will cut you off or slow down based on its behavior. However, predicting the future behavior of other agents is dependent on how your car will behave, leading to an interactive loop. We’ve all been in this situation, you’re trying to cross the road, there seems to be a car coming up. If you’re assertive, very likely, in crossing the road, the car will stop. Or if they’re more assertive, you’ll probably back off. At Cruise, we no longer separate the prediction system from the maneuver planning system. We have combined them  to decide jointly on what is the future behavior of other agents and our future, to solve extremely complicated interactive scenarios, including intersections with what they call a “chicken dance” where cars inch up against each other. We now call this the “behaviors” component.

The third component is motion planning and controls, where the car starts actually executing on its planned trajectory with smoothness. This component plays a huge role in delivering a comfortable ride because it can accurately calculate the optimal braking speed that reduces jerk (or discomfort). Most of our riders feel the difference immediately compared to human driving where a human driver could pump the breakers harder than necessary. Simulation is also a critical component of autonomous driving, which is often considered only as a testing tool but is, in fact, a reverse autonomous vehicle problem. Simulation involves building other agents that behave intelligently, such as human drivers, pedestrians, and cyclists. At Cruise, we have seen massive improvements in simulation since we have taken a big chunk of our AI and Autonomous Vehicle talent and put them in simulation. The technology we are working on is generalizable and broadly applicable to any robotics problem, such as drones and robots inside warehouses. 

I like to tell people that by joining Cruise, people are building their ML-driven robotics career, which can be applied to many other places. The stack of perception, prediction, maneuvering, and simulation can be scaled to other robotics problems. Robotics is pushing AI technology to its limits as it requires reasoning, self-awareness, and better generative AI technologies.

Robotics is pushing AI technology to its limits as it requires reasoning, self-awareness, and better generative AI technologies.

Q: The concepts you described here of predicting and simulating, giving your AI system a reasoning model, and self awareness, in terms of how confident it should be. These are lacking in today’s generative AI technologies. Is this a future direction that could produce better results? 

A: I do believe robotics is going to push AI technology to its limits, because it is not acceptable that you build a robot that will do the operation 99% of the time, correct, the 1% of the time can introduce massive friction.

Generative AI is very impressive, because it sort of samples a distribution of outputs, for a task that is not extremely well defined. There’s so many degrees of freedom, it’s like, give me a painting about something. And then it produces a weird looking painting, which in reality is an error. But you’re like, Wow, this is so creative. That’s why I say generative AI and particularly chatGPT do not replace human beings, they actually require a human operator to refine it.  Now it may reduce the number of human beings needed to do a task. But it’s L3 at best.

Now, in order to build an L4 and above technology, especially if it has a sort of a massive safety component. Number one, you need various components of this technology to have some self awareness of how sure they are. And us as humans, we actually operate that way with a self awareness of uncertainty. L4 technologies are not going to be able to be certain about everything. So they have to be self aware about the uncertainty of whatever situation they’re in. And then they have to develop sort of policies to handle this uncertainty versus chance it up to tell you whatever, statistically, it’s learned without self awareness of its accuracy. 

Q: What do you think about the combination of generative AI and a human operator in various fields such as education and healthcare?

A: Using generative AI alongside a human operator can result in an incredible system. However, it’s important to be mindful of the system’s limitations and determine whether you’re creating an L3 system with more degrees of freedom or an L4 system with no human oversight. In the field of education, generative AI can be a valuable tool, but it’s crucial to acknowledge that education is a highly sensitive area. On the other hand, in healthcare, as long as a physician reviews the outcomes, there are considerable degrees of freedom.

Q: I’ve heard great reviews from riders using Cruise’s service in San Francisco. What was your experience like in a driverless ride?

A: My first driverless ride was in a Chevy Bolt vehicle with a decent sensor package on top. At first, I felt a little anxious, but quickly realized that the vehicle was an extremely cautious driver that obeyed stop signs and braked very well. The vehicle optimized the braking and turning speeds, which made me feel safe and comfortable. I have seen the same reaction from family and friends who have ridden in the vehicles.

I think that the new Origin car is amazing and looks like the future. It’s a purposely built car for autonomy with no steering wheel and has two rows of seating facing each other. I believe that it’s going to be a very different experience from the current driverless rides, as it becomes real that there’s no driving and the car is really moving itself. The feedback from multiple people who have experienced it is that it’s as big as their first driverless ride. I also think that people will love the Origin car because it’s more comfortable and cautious than any vehicle with a driver, and it looks like the future. The first version of the Origin car should be deployed this year, and I hope that many people will have the opportunity to experience it and enjoy it within the next year or two.

Q: What are some open questions and unsolved problems as we move forward in building autonomous vehicles?

A: One open question is how to move towards end-to-end learning for autonomous vehicles, which would involve creating a single, large machine learning model that takes in sensor inputs and produces control signals, rather than the current system, which is heavily componentized. Another question is how to create an equivalent to the convolutional operator, a key component in computer vision, for autonomous vehicles. This is still an early stage field that requires significant investment to develop.

Q: At Facebook, you pioneered a new approach to AI platforms that then also later permeated our work at Google Cloud. And I think it was a very meaningful contribution. Can you explain why platforms are important for machine learning productivity?

A: I pioneered a new approach to AI platforms at Facebook that focused on productivity and delivering machine learning models quickly. I believe that productivity is key for successful machine learning because it allows for quick iteration and a faster feedback loop. In my opinion, platforms are the best mechanism to deliver machine learning models quickly and make machine learning a reality.  I believe what is much more powerful than building one model that is centralized, that serves everybody is to empower everybody to build the models they want, and to tweak them, and to tune them the way they like. And that’s where a machine learning platform comes in. And I do believe that is very much true in our organization. And I’ve seen that happen at Facebook, where at one point, around 2017, we had 20% of the company, either interacting or building machine learning models one way or another.

Q: In summary, are we at an inflection point in machine learning? Can autonomous systems approaches influence responsible AI more broadly?

A: I believe that we are at an inflection point where machine learning is expected to have a massive impact on multiple fields, including autonomous vehicles, robotics, and generative AI. Robotics is pioneering this concept of reasoning and understanding the environment and incorporating it, simulating it, and building your machine learning system to be accurate enough and understand the externalities. All of it is on this foundational bedrock of having great platforms which will enable quick iteration and a faster feedback loop. 

I also believe that the advanced work happening in robotics and autonomous vehicles will influence the future of AI, potentially leading to a more holistic and safe system that is oriented towards reasoning. In my opinion, one potential impact of autonomous vehicle technology on machine learning is around responsible AI. We should have one strategy for product safety, rather than separate strategies for product safety and ML safety. As an autonomous vehicle engineer, I spend more time evaluating the effectiveness of the system than building and tuning the ML model. The ability to evaluate the system effectively will become increasingly important, and I hope that there will be a generation of ML engineers that are used to doing so.

I believe that we are at an inflection point where machine learning is expected to have a massive impact on multiple fields, including autonomous vehicles, robotics, and generative AI.

We’d like to extend our sincerest thanks to Hussein Mehanna for joining us for this insightful chat. His expertise and experience in the field provided valuable insights into the current and future states of AI/ML. We look forward to hosting more conversations on AI, so please keep an eye on our events page!

I recently joined Pear as a Visiting Partner focused on early-stage investments in Machine Learning / Artificial Intelligence and Developer Tools. As I kicked off this new chapter, I sought advice on how to find the next billion dollar startup. The natural place to start: ChatGPT. 

Every time I asked ChatGPT which areas in technology are likely to cause the greatest disruption in the next 5 years?”, it gave me different lists, everything from quantum computing to 5G to Biotech and medicine. But AI/ML was always on top of its list. So to get some detail on the potential of AI/ML, I asked “which problems should AI-based startups solve to maximize growth and ROI?”

I noticed that ChatGPT picked up acronyms like ROI and expanded them correctly, and correctly interpreted queries with misspelled words (e.g., ‘distributed’ was corrected to ‘disruptive’ based on the context of the chat). That made me sit up in my chair!

ChatGPT’s responses are high level. So I tried to pin it down by asking what specific products our startup should build. It provided a specific list, but I could not get it to stack rank or put numerical values against any of these.

To go deeper, I picked the ‘Personal Assistants’ product idea, and asked 

1. “How would we monetize our personal assistant?” and got this reply:

2. And, “how would we differentiate it relative to competition?”

These were surprisingly good answers: multimodal chatbots that are personalized and integrated with enterprise systems could be quite useful. Finally I asked ChatGPT to write the business plan and it did!  I shared this business plan with Pear’s healthcare investors and it even passed a few checks. Now if only ChatGPT could generate founders and funding, we’d be all set!

But let’s fast forward to the future, which is what Pear’s portfolio companies are building. First of all they are using AI to solve real problems, such as automating an existing workflow. For example, Osmos.io uses a form of generative AI to create data transformations in a no-code data ingestion platform that’s replacing fragile, hard to maintain hand-coded ETLs. Sellscale uses generative AI to create better marketing emails. Orby.ai discovers and automates repetitive tasks in Finance, HR, customer service, sales, and marketing. And CausalLabs.io is creating the infrastructure layer for UI that is increasingly generated, optimized and personalized by AI. 

These companies are building on foundational models like GPT and expanding them to get users accurate, effective outcomes that are orders of magnitude more efficient. When a technology creates orders of magnitude better outcomes, it is a game changer.

We believe AI in 2023 is what PC’s were in 1983 or the internet was in 1995 – it will power enterprises and consumers to do things they couldn’t before and generate enormous value in the next five years. Much of it will come from startups that are in the very early stages today.

This brings me back to why I joined the fast-moving startup ecosystem at PearVC in the first place: the time is now, and the opportunity to build the future with our next billion dollar startup is here.

This is a recap of our discussion with Lindsay Kriger, Director of Transformation and Business Operations, Andrew Smith, Chief Operations and Innovation Officer, and Dr. Bobbie Kumar, a family physician, from Vituity, the largest physician-owner partnership in the U.S.

Watch the full talk at pear.vc/speakers and RSVP for the next!

Healthcare innovation is local
To get started, pick an area to focus on
Recommendations for new healthcare solutions

Healthcare innovation is local

Though tackling the American healthcare system seems extremely daunting, the most important rule as a healthcare founder is to remember to keep the patients’ and the communities’ needs centered. Like any startup founder, healthcare providers need to know their users.

“The reality of healthcare innovation is that it’s local,” said Kriger, Vituity’s Director of Transformation and Business Operations. “We need to talk to real people from diverse backgrounds, geographies, cultures, and clinical care settings and build companies from passion and personal experience. That will ultimately create the healthcare that we all want.”

Often, this means bringing care directly to the patient, especially in the COVID era, when patients may be wary of coming to hospitals.

“There are many sick people in this country that aren’t coming to the hospital or asking anyone for help, so taking care of people at home, or in their grandparents’ house, or at an SNF, or anywhere they are and reaching out to them has become incredibly important,” explains Smith.

To get started, pick an area to focus on

Our healthcare system is a $6 trillion problem. We spend close to 18% of the GDP on healthcare every year and over $10,000 per capita—twice of what any other industrialized country spends.

Naturally, numerous technical solutions and innovations have emerged in this space. Because of how vast the market is, however, it can be difficult to figure out how to start. 

This is where balance comes in. After thoroughly understanding your users, pick one area to focus on. 

“If you try to solve all of the problems of healthcare in one swoop, it is going to be extremely difficult,” Smith emphasizes.

Smith underscores the importance of iteration and building feedback cycles from diverse sets of users. 

“With whatever you’re rolling out, you need to hear from patients and providers–and not just in one location. One location is a great place to start, but one of my recommendations is having a diverse set of pilots.”

Again, because of how local healthcare is, understanding the specific needs of markets you are targeting will help you build the best solutions. 

At scale, however, Vituity firmly believes that all users and all markets should be serviced. 

“We’re going to take care of everyone equally, always. And we want to build solutions and products that work for the entire community in this country.”

Recommendations for new healthcare solutions

Dr. Bobbie Kumar, Director of Clinical Innovation at Vituity, thinks a key area for innovation in the healthcare space is physician productivity.

When Dr. Kumar was an intern, she had 45 minutes to meet each patient. As a doctor, she only gets 10-15 minutes to do the same amount of work. 

“Reducing the administrative burden that myself and the care team have to experience is going to be a very key and poignant feature in how solutions are able to penetrate the healthcare space,” she added.

Beyond that, she leaves two other recommendations for innovating solutions within the healthcare industry. 

“Disrupt the industry, not the mission,” she said. “The top two reasons for choosing medicine are help people, followed by intellectual pursuit— wanting to find something that’s challenging enough for the knowledge base and the research.”

The last recommendation? “Preserve the humanism.” Technological solutions simply cannot account for the degree of human interaction inherent in physician-patient relationships. 

“Until we really start looking at healthcare as both patient-centric and provider and care team supported, we’re just going to end up missing the target. So, my hope for the future is that we disrupt the status quo and we model these solutions that not just promote, but really emphasize the value and the unique human experience.”

Cathy Polinsky, former CTO at Stitch Fix, chatted this week with Pear associate Harris, who leads our talent initiatives. This is a recap! Watch the full talk at pear.vc/speakers.

It starts as an obsession. You play with your first coding language, and you’re hooked. You keep learning new languages and building cool things with your new skills. You’re doing this in an exciting environment with enthusiastic mentors who support you.

Cathy Polinsky, former CTO at Stitch Fix, was lucky enough to grow up this way. In the 80’s, Cathy’s mother was a teacher, and Apple personal computers had arrived at school. Cathy’s mother thought the technology was interesting and bought one for their home (before they had a VCR!).

Cathay learned her first programming language, Logo, and went on to learn basic programming. Computer science was so new then that there wasn’t formal education for it in school, but growing up in Pittsburgh meant that Cathy was able to go to a summer camp at Carnegie Mellon.

“We did algorithms in the morning and more hands-on programming in the afternoon with robotics and maze solving problems, and I just really got hooked and knew I wanted to study it after high school,” said Cathy.

Sure enough, Cathy did just that, and went on to a thriving computer engineering career. In this fireside chat with Pear, she looks back at her career and highlights the building blocks she gained from each role to help future engineers make decisions about their own careers.

“A career is made in hindsight, not while you’re on that path. While you’re on it, it might not feel straight, but you can in retrospect see this arc of where you’ve come from. I’d say I have a little bit more of a clear arc than most, but I know many other people who have had different paths that have gotten there or to some other destination that they have been really excited about.”

Laying the groundwork as a software engineer scaling early Amazon
Hands-on, non-stop building at an early stage startup
Moving into management and people skills at Yahoo and beyond
Cathy’s Career Advice
Cathy’s Advice On Hiring
Tactical Hiring Tips

Laying the groundwork as a software engineer scaling early Amazon

Like many young graduates in computer science today, Cathy’s first job out of school was as a software engineer at a fast-growing company — Amazon, in 1999, right before the dotcom crash.

“I thought I’d missed the interesting times at Amazon. I thought I’d missed the early stage startup. Little did I know, I was living the interesting times. It was a gritty time. I was there during the dotcom collapse and all my stock was underwater,” Cathy recalls.

Things were still early enough for Cathy to grow fast with the company (she had a door desk when she started!). Amazon was in the hundreds of engineers then, and everyone was brilliant, but the rapid pace meant that the environment was a bit chaotic. The code base was a mess for example, with little documentation.

“It was an amazing time where I learned how to be an engineer being in a company that was scaling so fast. I learned a lot about how to hire people earlier than any other job I would have gotten,” says Cathy.

At Amazon, Cathay had the opportunity to learn from the tech giants. She developed solid, rigorous ground for her career learning to ship things at scale and having to think through everything from interview process to database and schema design, to hardware constraints and automation and testing.

But in addition to engineering skills, Amazon helped Cathy to gain a deep appreciation for the growth mindset — a mindset that serves her even today.

“Jeff [Bezos] never wanted to be written in an article and have Amazon in the same sentence with any other early stage dotcom company. He only wanted to see himself with the giant companies that were out there. I’d say that I was much more of a voice of growth and opportunity as a leader at Stitch Fix in the sense of: ‘How can we invest more, and how can we grow more?’ and making sure that we were keeping our eyes always on the long-term.”

Hands-on, non-stop building at an early stage startup

Cathy soon got the startup itch and wanted to see what early startups were like, so she went and joined a 13-person startup in San Mateo.

At first, with only two years of work experience at a scaling company, the early startup environment felt quite different. Cathy was surprised to learn that there were no automation suites and other big tools she took for granted at Amazon.

“But, I wrote way more code there than any other place, and I got to wear more hats and I got to go to a pilot customer and be involved when we were doing discussions about name changes and logos,” says Cathy. “You got to interact with people that weren’t just tech and be much more involved in the company on a different scale .”

Cathy also experienced the pain of building great products that didn’t work out.

“The company never really got off the ground. We only had two pilot customers, couldn’t get sales. There was a recession going on. It was heartbreaking when we closed the door to feel like I put so much of my blood, sweat, and tears into building something that I was really proud of.”

It was valuable first-hand experience in understanding that a successful company was not just about building a great product, but also about building the right thing and checking in with the market for early feedback.

Moving into management and people skills at Yahoo and beyond

After the startup heartbreak, Cathy turned back to big company world, though she “wouldn’t recommend to anyone to overcompensate that strongly,” she laughs.

At Oracle, Cathy realized she missed the fast pace of earlier companies, and sought to move into management. A friend pointed her to a position opening up at Yahoo.

Cathy ended up being offered her choice between two roles there — an engineering management role and an engineering lead role. She decided to try management and never looked back, going on to lead engineering at Salesforce after Yahoo, and on to the C-suite at Stitch Fix.

One of the first lessons Cathy learned in her first management role at Yahoo was to stay out of the code. The engineering manager she inherited the role from said that she regretted being so hands-on and jumping in too quickly to fix bugs or help her team with the solution.

“I really took that to heart. If I’m jumping in to heroically save the day, I’m not actually working on the fundamental issues that are going to help the team be successful in the long run. That has really influenced how I spend my time and how I look at the role,” says Cathy.

That is, transitioning into a management role means focusing your attention more on the people issues rather than the tech issues.

“I’d say that I choke often — that I would have been better off being a psychology major than a computer science major,” Cathay laughs. “Dealing with people in some of these large organizations is much more complex, and not deterministic in any way. You never know if you’re doing it the right way or not. I think that I’ve spent more sleepless nights thinking about people issues than I have about technology issues.”

In all the companies Cathy has been in, it’s been key to treat every single tech production incident as a learning opportunity.

That’s because if you’re shipping software, it’s inevitable that you are going to break something eventually. So, the question software engineers should be thinking about isn’t “How do we avoid breaking things?” but rather, “How do you make sure you don’t have the same problem happen again, and how do you make sure that you learn from it and get better and have other people around you learn from it, so they don’t have the same mistake that you had?”

Cathay is a fan of blameless postmortems.

“We get in a room and we do a postmortem of what happened, but the only rule is you can’t point fingers at anyone. You don’t name any names. You’re not trying to get anyone in trouble. If you can really approach any problem with that open mindset and learning, then you will make sure that you uncover all the problems in the areas and you won’t have the same problems again.”

Cathy’s Career Advice

• There’s no wrong answer to big tech vs. join a startup vs. start my own company.
• Take the roles where you’re going to learn the most.
• Follow your passions.
• If you are interested in something that’s different than what you’re doing today, just tell people that. People will see you differently and think of you when opportunities arise when you tell people what you’re passionate about.
• Don’t worry about where you’re going to be 10 years from now, but have an idea of what you want to do next.
• If you don’t know what you want to do next, talk to a lot of people.

Cathy’s Advice On Hiring

Anytime you can, hire someone smarter than you.

A lot of people have a hard time doing that. They think, “Oh, they have more experience, or they’re smarter than I am, what am I needed for?” You will never ever fail if you were hiring people that are smarter than you and know more than you.

Extend outside your network and find people who are going to push you.

I have worked in a lot of companies that focus on hiring within your network and telling the great track record they have for internal referrals. As a woman engineer, I think that’s really dangerous. I think that you have only a specific network of people and if you continue to hire people in your network, you’re only going to see people who look exactly like you and you’re not going to push yourself and get diverse opinions around the table. You’d be better off really trying to extend outside your network and finding people who are going to push you and bring different experiences to the table than you have in your own. That’s something that we were really intentional about at Stitch Fix — making sure that we were reaching out into diverse networks and seeing people that were different than the people that we already had.

Follow the structure of what you set out to do — don’t rush.

One of the challenging hires was not someone who worked directly for me, but was one level down, and was a referral from someone in the organization. We did a rush job hiring and leaned on the referral, but really did not show up living the values that we had as a company. We started to see some things that didn’t show up the way that we would expect for someone of their level. When we did a debrief on what happened, we realized that we hadn’t done reference checks and we hadn’t really done a full check on: ‘Is this person a good fit to the values that we had?’ It was a pretty big miss that we hadn’t followed the structure of what we had set out to do and really caused a lot of friction across the team because of it.

It’s critical to understand what you need for the stage of your company.

In the early days, Amazon would rather hire someone that had a couple of amazing interviews and a couple of bad interviews than someone who was mediocre across the board. They wanted to see someone who really excelled in a specific area, even if it meant they had holes in other areas. I like that sense of playing to someone’s strengths, especially as a larger company, you can take more liberties in that way.

It’s harder in an early stage company… you have funding for three hires. You’re going to need to hire some real generalists who like to build things, who can also answer phones and do hiring and operations and the like. Thinking about those first hires as generalists who are interested in getting to wear a lot of different hats is important. It’s kind of fun for the people who do it, to get to build things from the early stage.

Then you have to think about how you scale it. Because at some point, those people are probably not going to be happy after you get to a team of 100 where they’re not the best at any of those things. Either they scale, and they still know everything and are this amazing person that can jump in anywhere and add value, or they get really dissatisfied that they can’t really play the same role that they had before.

As you’re scaling things out, you’re hiring more narrow generalists of, “Hey, we really need someone who understands AWS deployments, or we need someone who really understands this mobile technology stack,” or whatever it might be.

So, if you’re really thinking about building and scaling with the business, as the business scales, you have to think about what stage you’re in and know that what works before is not going to be the thing that’s going to be successful after you get to 50 or you get to 100.

Tactical Hiring Tips

• Take home interview questions and pair-programming interview questions are a good way to see what people are going to do in their day to day. You don’t work by yourself — you work in a team and so seeing how people work with a team is good.
• Have candidates interview with non-technical team members and solve a problem together. It’s important for technical people to be able to talk with nontechnical folks. At Stitch Fix, this practice has enabled a much higher EQ team.
• Have inclusive language in your job description and list fewer requirements to be more welcoming to women. Avoiding bro-like or gendered language is of course the obvious thing to do, but what might be less obvious is that long requirements can rule out certain populations of people! Men will tend to apply anyway despite not meeting all requirements, and women will tend to filter themselves out because they don’t have nearly enough of the requirements on this list.
• Sit in on every debrief. You don’t necessarily need to meet every candidate, but you should listen carefully to candidate discussions and guide your team. “There were times where someone would explain the interview and then have a strong assessment on whether we should hire the person or not. I would listen to their rationale, but not agree with the outcome based on their assessment, so we could talk about that and dig in. Sometimes there were some things that changed the way that they thought about the interview, for example, something like, “They weren’t very confident in their skills.” We would ask, “Why did you think that?” And it could just be a passive word choice that they used, and someone else points out, “They’re actually really confident over here — and is that really a job requirement that you need?”

As we move into the last days of summer, with the world still very much uncertain, we highly recommend giving Bob Baxley’s Pear talk on conditions, constraints, and convictions a watch at pear.vc/speakers.

Bob shares three Cold War stories of magnificent innovation — reminders of the amazing things humans have accomplished and can still accomplish even in trying times.

Below, find insights from the Q&A session for thinking about conditions, constraints, and convictions for our own times.

When are constraints good for creativity, and when are they bad?
What can founders do to help creative teammates be most creative?
How do you figure out which problems to solve and which to defer?
What signals can you look for to indicate that you need to engage the creative side more, versus getting it done?
How do you find inspiration for design during the initial brainstorming phases?
How can we think about current pandemic conditions of not being able to see each other?

Q: Constraints may have boosted Dr. Seuss’s creativity, but it seems that a lot of conventional wisdom also suggests that having too many constraints can be bad for creativity — how do you reconcile this?

I think the question is getting at two different kinds of constraints. One is a resourcing constraint and the other one is defining the problem space. Trying to do too much with too few people is a different challenge. That’s really a question of prioritization, so I’ll leave that aside.

As a creative professional and somebody who’s been designing software products and leading teams for three decades, the worst thing you can do is to give me a lot of space and a lot of time. Every creative I know, we’re master procrastinators. We will always put off finishing the thing. If we have a big, giant open field, we don’t quite know how to get started. I’m a big believer in a well-defined problem space.

I like making sure that you have to show progress on a steady basis. With my teams at Apple and now ThoughtSpot, the team has to show me work basically every 48 hours. We are constantly looking at work. It keeps people from getting too caught up in their own heads and wandering off in the wrong direction, and it keeps us together as a team. It reduces a lot the emotional, creative pressure on people.

Structure’s the difference between an empty field and a baseball diamond. You can’t have a game if you don’t have the diamond. You can’t make progress if you don’t have the diamond. You’ve got to have some boundaries to make progress and a ton of amazing art.

If you really go back and you look at Chopin’s Preludes are Bach’s Goldberg Variations, there’s a ton of different, great artistic achievements that were set up as a technical problem that the artist was trying to solve. They go into the work that way. Most artists don’t just show up at the canvas and see where it goes — like they’re trying to accomplish something.

Q: What can we, the founders, do to help our creative teammates be most creative and set the right constraints without setting too many?

In all designed problems, what you’re trying to do is help the person to understand the problem. Don’t get prescriptive about the solutions. Try to make sure we’re trying to accomplish the same goal, and then evaluate the design solution in the context of the goal.

A story I tell is: at a company I was working at years ago, we had a homepage. There was a link on the page, and the product manager said “Bob, you have to make that link blue.”

I said, “Why do we have to make a blue?” They said, “Just make it blue.”

We got in this debate about it being blue. I said, “No, I don’t want it. That’s going to ruin the aesthetic of the page.” And they said, “Well, I want it” and went behind my back and made it blue.

Then we had a heated discussion in the hallway. I eventually said, “What was it you were trying to accomplish by making it blue?”

Turns out, people couldn’t see it. And I said, “Oh, so it wasn’t prominent enough?” And the product manager said, “Yeah. It wasn’t prominent enough.”

I said, “Okay, great. Well, we’re professional graphic designers. We have 15 ways to make it more prominent, one of which is making it blue. What you did is you leapt to a solution, and you didn’t really present the problem to us.”

For founders and executives, I spend most of the conversation trying to get them to more specifically narrow down our understanding of the problem and to make sure that we’re on the same page with the problem. The benefit of doing that is that when you find the right solution, everybody will agree.

I often get questions about how to resolve creative conflicts between different design solutions. I always say — that means you haven’t agreed on the problem. Because if you had a shared understanding of the problem, the solution would be obvious.

Q: How do you and your team tend to figure out which problems to solve and which ones to defer or ignore?

Here’s another story, about my friend Steve.

When Steve was getting ready to work on Keynote for the first time, Steve went to Roger Rosner, who was the VP of engineering responsible for iWork. Roger asked, “How should we think about Keynote? We got to compete with PowerPoint. There’s already these other presentation packages. What should we do?”

Steve said, “Roger, there’s three things with Keynote. One is it should be really difficult to make ugly presentations. Two, you should focus on beautiful cinematic transitions. And three, you should optimize for innovation over PowerPoint compatibility.”

I use those as examples of strong tenets. Each one of those statements, for example, if we just take the last one, optimize for innovation over PowerPoint compatibility — you could imagine the thousands of hours of debate that the product team would have had if they didn’t have clarity on which side of that debate to be on.

Steve just said, straight off the bat, make it difficult to make ugly presentations. That’s a whole set of things to take out of the way. So, we get beautiful cinematic transitions, which to this day is the hallmark of Keynote and why I can’t stand using Google Slides or PowerPoint. That’s a very clear direction.

I think a lot of times when you’re trying to figure out what to do, do you really have clarity around the tenets? Because if you have that clarity, the priorities fall out much quicker.

Q: What signals do you look for in your own work that indicate you need to engage on the creative side more deeply versus buckling down and getting it done?

This is an interesting question we’ve been dealing with internally at ThoughtSpot. When does something pivot from being exploratory creative work to execution mode? It’s always a delicate balance, and it’s a little bit like being a parent. When do you give your kid that extra thing to go do? When do you let them go further, versus when do you stay closer to them?

There is a point with almost every project where the team has some sense of, “No, this is really what we want to go do.” I can’t give you a rubric for how to judge that, but you can usually feel it in the room. The debates are becoming more and more myopic and you don’t seem to be making the same level of progress. There seems to be general agreement that this is the thing we want to go do. At that point, you have to pivot into execution mode. There’s a point where you just have to sit down and finish the thing.

As a creative, I enjoy working in both those modes, because it’s emotionally draining to sit in that level of uncertainty of wondering what we’re doing. There’s a point when it’s actually really rewarding to put all that uncertainty behind you and just go and finish the dang thing.

We tend to do that with deadlines. We sign up for some sort of commitment. Maybe there’s a company event that you have to have a demo ready to show. Maybe you have to give a presentation like this. You sign up for different kinds of commitments, and then you can use that commitment as a forcing function.

Q: How do you find inspiration for design during your initial brainstorming phases?

In terms of software, I spend a lot of time just looking around in the world. I’m the annoying guy at the deli that asks to look at the point of sales system. How does the Clover UI actually work? I sit and chat with people about self-checkout at Safeway and how it sucks and why it sucks. There are software products everywhere, so there’s tons of inspiration to be found just looking around the normal world.

Q: We have a lot of new constraints these days — one of which is that we’re not with each other. What are your thoughts on our current conditions, and where do you see it going?

I’m on the side that says there’s no great reason to go back to the office. I always try to get my team to separate the idea of working from anywhere from the idea of the lockdown. Imagine your kids are still going to school, and imagine you can go work at the Starbucks, and you can go see your friends for lunch. You can work from Hawaii if you want. How does that feel?

Most people say, “Oh my God, that feels awesome” — so, it’s not the working from home thing, it’s the lockdown. The lockdown is not something within our control, as individual companies or individual citizens. So you have to parse those two.

I firmly believe that we can make this work, and I think there are some unique benefits to working from anywhere. The fact that we can do things like this company events with a few hundred people from around the globe — we wouldn’t have done that in the past. If we had, it would have been a huge deal, and it would cost us an enormous amount of money.

You’ve got to dance with the constraints, and stop thinking about what you’ve lost by not being in the office, and start thinking about what you’ve gained — because you’ve gained a lot.