https://www.indiehackers.com/post/claude-just-slashed-the-cost-of-building-ai-applications-lQHHc1S9NLLiPLJeyUWk

Community Starting Up Tech Creators E-Commerce Services Money
Ideas Database Stories Database Products Database Subscribe to IH+
Plus
Starting Up Tech Creators  
E-Commerce Services Money
Join
 
11
Likes
1 Bookmark 1 Comments

Claude just slashed the cost of building AI applications

by Darko Gjorgjievski
August 16, 2024
ai agent person

Imagine you're creating an AI SaaS/app.

You're heavily dependent on OpenAI, Claude, or Google's API, and one
of your key competitive advantages is your advanced prompt.

However, your advanced prompt is quite lengthy and includes many
examples, which help the AI generate a useful output.

This can quickly drive up your API costs since you're constantly
sending the same long input. Providers like OpenAI charge you based
on input tokens, meaning the more words you send for the AI to
process, the more it costs.

Things can get expensive fast.

Welcome to the world of prompt caching

ClaudeAI, one of the top three AI providers in the world (alongside
Google and OpenAI), has recently introduced a feature called Prompt
Caching.

This feature essentially lets you to "reuse text" across multiple
prompts.

Imagine you have a large prompt with plenty of examples. The examples
make up 90% of the prompt's length.

With this feature, you can "cache" the examples and only send the
remaining 10% as the actual prompt.

The impact of this is dramatic. Up to 90% reduction in input API
costs.

This also means that, as a developer, you can either lower your
pricing or increase your profit margins for your SaaS/app.

What is prompt caching useful for?

According to the creators of Claude, this is useful if you provide:

  * AI assistants, where you expect multiple users to enter the same
    prompt.

  * Code generation where you need to reuse the same prompt or have
    multiple users working with the same template.

  * Code reviews: When asking AI to review long chunks of code, you
    don't have to send the same code over and over again. This can
    save both time and money

  * Processing large documents: For instance, if you feed the AI a
    novel and want to ask questions about it,.

  * Any search tool: You can input data from files and ask questions,
    etc.

  * Any prompt with plenty of examples: You no longer need to worry
    about optimizing your prompt for length. You can focus on being
    thorough and getting the best results possible.

Will OpenAI follow suit and release a similar feature soon?

Avatar for Darko Gjorgjievski Darko Gjorgjievski
11
1
Share
Say something nice to zerotousers...
Post Comment

 1.  
    1

    Great examples. Just remember Claude only has 5 min cache, which
    limits the use cases based on your traffic

    Avatar for user @RasmusHjulskov RasmusHjulskov
    *
    8 hours ago
    *
    Reply
    *
    Edit
    *
    Delete

Stay informed as an indie hacker.

Market insights that help you start and grow your business.

[                    ]
[                    ]
Subscribe
---------------------------------------------------------------------
Indie Hackers logo glyph Indie Hackers logotype

Follow @IndieHackers on X for stories and insights about founders
building profitable online businesses, and to connect with others in
the Indie Hackers community.

(c) Indie Hackers, Inc. * FAQ * Terms * Privacy * Cookie Settings/
Policy *    

Community

Groups Meetups Top Today Top This Week Top This Month Join

Products

All Products Highest Revenue Side Projects Self-Funded Add Yours

Databases

Ideas Products Stories