How to decide
Dis-correlation between decision quality and outcome quality
- Paradox of experience
- experience is necessary to be a good decision maker
- you do something, world gives you feedback, you do it again and in the process you become better
- problem: taking individual pieces of feedback and over biasing on them.
- Reason: Our brains process outcomes sequentially. Out of 10k coin tosses, you can predict the general outcome but when it's sequential, we are taking big lessons from these singular data points.
- We use quality of the outcome to rate the quality of underlying decision.
- Hindsight bias: We are good at determining the state of knowledge when we made a decision. At that point we can determine all the possible branches of our decisions. As things play out we keep pruning all the counterfactual branches. When result comes, we think that's the only outcome that could have occurred.
- Try to recreate the actual state of knowledge in both cases
- Two cases
- what do you know before making a decision
- what do you know at the end of the result
- This will help you understand what you missed and what you should have paid more attention to.
- Matrix of decision quality and outcome quality
- Good Decision, Good Outcome
- good work
- there's a lot to learn from here. To understand why we won. What information we missed. What were the causes of this win. Did we factor that in.
- Good Decision, Bad Outcome
- bad luck
- Bad Decision, Good Outcome
- dumb luck
- Bad Decision, Bad Outcome
- justice
- When you do this over time, you see
- that you are only focused on good-good and bad-bad. You are not considering the luck part.
- you are also not considering the good-good part
- If you have bad outcome, you want to deconstruct that and you want to find a bad luck in there. It helps you get a way out of that decisions. We are more eager to go explore those bad outcomes.
- When we have a good outcome, we don't apply the same process. We don't want to look at it because if we find out that we won because of luck, then we sully our win.
- Knowledge tracker - Should get auto created as part of a good decision making process
- Do it prospectively
- what do you know
- what are the possible ways it could turn out
- what are the payoffs of each of those probabilities
- how probable do we think these are
- And when you get result
- same as prospective
- Pro-con list
- It's better than nothing
- The goal is to reduce the cognitive bias but it actually amplifies it. As soon as we start on a problem, we have made up our minds. Pros-cons list amplifies the conclusion that we have made.
- Worst case: Pros are aspirational, cons are delusional
- It tries to certify a decision
- Needs 2 more things
- How good/bad
- Decision Tree
- Probability of an outcome and then
- Good way to get around this is to have more than one person build their own pros and cons list and then compare.
- What types of decisions should this be applied to?
- Most of the decisions (low impact) we make every day, should be made faster. Much faster than we actually do. Example: what to eat, what to watch, what to wear etc
- We spend a lot of time trying to make these decisions. In US ~6-7 workweeks on average.
- These decisions don't have any long term effect. If dinner at new restaurant was bad, it might be negative for you (physically, mentally) maybe next 2 days. Happiness test.
- In general the time we take to make the decision, our accuracy increases. But with these decisions, we can take the risk because we can increase the sample size and the bad outcomes don't really affect your happiness.
- We also don't know our preferences. It's good to get feedback from the world. Get a lot of response from the world and use that feedback faster to build better models. Over time it will become better.
- Decision Hygiene
- In startups, you have very little information and your decisions are going to be very crucial. With good decision hygiene the decisions are much better.
- Your whole process is built on a foundation of your beliefs (your models of the world, facts you have, knowledge you have). Our foundations have 2 problems:
- Some things that we believe in are inaccurate. Cracks in the foundation.
- We don't know very much. Flimsy foundation.
- Solution to both problems is to explore the universe that we don't know.
- Corrective information
- New information
- Best thing you can do about your decision making
- Find out things that other people know
- And what their perspectives are on the problem that you are considering
- If you don't have a good decision hygiene, then you won't be able to make it work properly.
- We are very tribal and we like to agree with each other
- Our opinions are very infectious
- When we ask for feedback, we tell the person, not just the data but our opinions on that data as well. These opinions bias the person who you are asking feedback from.
- Possible solution: Quarantined solutions or Pre-work
- Give everyone same information
- Ask them to make decisions on that
- Give them a budget to put things in priority, to be more specific
- This gives folks a reason to have something precise for their rationale about a decision.
- This allows you see agreements and disagreements in the solutions
- You can figure out what are the component parts of this decision that actually matter. Things that you need to pay attention to and get more feedback on. You are essentially mediating judgements.
- Share this with the team before the meeting so that everyone know what are the areas of agreement and disagreements. This way everyone is aware of different opinions in the group.
- Make your meeting much more efficient (time saving, sharing etc plus) because you might not have surfaced it all there in meetings.
- In meetings people talk about the topics they agree upon because everyone wants to get credit for that decision by adding their color to it.
- Focus time on the areas that you don't agree upon. It shouldn't be to convince others of your opinions. It's to inform the group. This way you can ask those who agree with it, to add more data points and those who disagree with it to get more information and make better decision. They can explain their position and mental model. Everyone benefits. Convey vs Convince. You don't need to agree to decide, you need to inform to decide. All of you can't be on the same place about an idea.
- You want diversity of opinion and if you don't tease out the different opinions, then you are making your decision and not the team.
- Teams often feel like pros-cons list. You are trying to get more people on the decisions but in such groups the decision qualities isn't better.
- Long feedback loops
- There's no such thing as long feedback look. Even for investing in a startup which might exit in 10+ years. Your decision isn't based on the final exit. It's based on everything else related to that company.
- Their idea, ability to build, hire etc
- Funding on next round
- Intermediary stages act as the decision checkpoints. You are essentially making all the cumulative decisions at once.
- It also helps you weigh against the ones that didn't work out since they die out between that 10+ year period.
- Fox-light vs Hedgehog-light
- Fox light: Looking at the world from all sorts of different perspectives and applying lots of different mental models to get to your answer.
- Hedgehog-light: You approach the world through your one big idea/thesis. Generally, this doesn't work very well.
- How fast you should go?
- Should depend on how easy it is for you to revert back from a decision, essentially opportunity cost of not choosing another branch.
- After you choose something, a new information might tell you that this is not the correct answer
- Type1 (one way door) or Type2 (two-way door) decisions. Type2 is faster to move along. Whole point is to mitigate the downside. You can get out easily vs not. The more quit-able something is, faster you can go on that route.
- At times you don't have to make a decision and you can run things in parallel. A/B testing. Based on what works better, you can make your decisions.
- You can't quit it but you can negate it. If so, think hard about it because it's tough to get out of there. If you can negate it then you can go faster.
- Decision Stacking: For when you have to make a big bet. Find out things that you can do before, that help you gather decision, so that when you have to make the decision, your model of the world is better. Get a feel for the world. Low impact decisions to get better feel. Get a lot of directional information to de-risk your decision.
- When you are 90% sure, find out what is the information that will cause you to flip the decision. If there's such information, find out if you can get it. If you can, get it. If you can't get it, then that's the state of the world, you should make that as the decision.
Last modified 2yr ago