Bookmarks

RAG Or Fine-Tuning?

There is a lot of confusion about when to apply which method.

RAG makes sense when you have a custom knowledge base and want a standard ChatGPT-like interface on top of it. RAG has multiple components to it and can be tricky to get right. However, it's definitely easier to implement than fine-tuning.

Fine-tuning makes sense when you have several supervised examples of request responses and are looking for a particular format for your responses. That is if you want the model to adapt to a particular type of response. For example, you can fine-tune a model to be good at a specific type of SQL code generation.

Sometimes, but not often, it makes sense to do both. Using something like Abacus AI makes applying either method on open-source and closed-source LLMs super simple.

Of course, we are particularly partial to open-source, especially if it can do the job!

If only someone told me this before my 1st startup:

1. Validate idea first.
I wasted at least 5 years building stuff nobody needed.

2. Kill your EGO.
It's not about me, but the user. I must want what the user wants, not what I want.

3. Don't chaise investors, chase users, and then investors will be chasing you.

4. Never hire managers.
Only hire doers until PMF.

5. Landing page is the least important thing in a startup.
Pick an average template, edit texts and that's it.
90% of the users will end up on your site coming from a blog article, social media post, a recommendation. Which means they have the intent. No need to "convert" them again.

6. Hire only fullstack devs.
There is nothing less productive in this world than a team of developers.
One full stack dev building the whole product. That's it.

7. Chase global market from day 1.
If the product and marketing are good, it will work on the global market too, if it's bad, it won't work on the local market too. So better go global from day 1, so that if it works, the upside is 100x bigger.

8. Do SEO from day 2.
As early as you can. I ignored this for 14 years. It's my biggest regret.

9. Sell features, before building them.
Ask existing users if they want this feature. I run DMs with 10-20 users every day, where I chat about all my ideas and features I wanna add. I clearly see what resonates with me most and only go build those.

10. Hire only people you would wanna hug.
My mentor said this to me in 2015. And it was a big shift. I realized that if I don't wanna hug the person, it means I dislike them. Even if I can't say why, but that's the fact. Sooner or later, we would have a conflict and eventually break up.

11. Invest all money into your startups and friends.
Not crypt0, not stockmarket, not properties.
I did some math, if I kept investing all my money into all my friends’ startups, that would be about 70 investments.
3 of them turned into unicorns eventually. Even 1 would have made the bank. Since 2022, I have invested all my money into my products, friends, and network.

12. Post on Twitter daily.
I started posting here in March this year. It's my primary source of new connections and traffic.

13. Don't work/partner with corporates.
Corporations always seem like an amazing opportunity. They're big and rich, they promise huge stuff, millions of users, etc. But every single time none of this happens. Because you talk to a regular employees there. They waste your time, destroy focus, shift priorities, and eventually bring in no users/money.

14. Don't get ever distracted by hype, e.g. crypt0.
I lost 1.5 years of my life this way.
I met the worst people along the way. Fricks, scammers, thieves. Some of my close friends turned into thieves along the way, just because it was so common in that space. I wish this didn't happen to me.

15. Don't build consumer apps. Only b2b.
Consumer apps are so hard, like a lottery. It's just 0.00001% who make it big. The rest don't.
Even if I got many users, then there is a monetization challenge. I've spent 4 years in consumer apps and regret it.

16. Don't hold on bad project for too long, max 1 year.
Some projects just don't work. In most cases, it's either the idea that's so wrong that you can't even pivot it or it's a team that is good one by one but can't make it as a team. Don't drag this out for years.

17. Tech conferences are a waste of time.
They cost money, take energy, and time and you never really meet anyone there. Most people there are the "good" employees of corporations who were sent there as a perk for being loyal to the corporation. Very few fellow makers.

18. Scrum is a Scam.
If I had a team that had to be nagged every morning with questions as if they were children in kindergarten, then things would eventually fail.
The only good stuff I managed to do happened with people who were grownups and could manage their stuff. We would just do everything over chat as a sync on goals and plans.

19. Outsource nothing at all until PMF.
In a startup, almost everything needs to be done in a slightly different way, more creative, and more integrated into the vision. When outsourcing, the external members get no love and no case for the product. It's just yet another assignment in their boring job.

20. Bootstrap.
I spent way too much time raising money. I raised more than 10 times, preseed, seed, and series A. But each time it was a 3-9 month project, meetings every week, and lots of destruction. I could afford to bootstrap, but I still went the VC-funded way, I don't know why. To be honest, I didn't know bootstrapping was a thing I could do or anyone does.

That's it.
All my projects →